Finding Nano-Ötzi: Cryo-Electron Tomography Visualization Guided by Learned Segmentation

Ngan Nguyen, Ciril Bohak, Dominik Engel, Peter Mindek, Ondřej Strnad, Peter Wonka, Sai Li, Timo Ropinski, Ivan Viola

View presentation:2022-10-19T19:36:00ZGMT-0600Change your timezone on the schedule page
Exemplar figure, described by caption below
The image shows a volume containing several intact SARS-CoV-2 virions acquired using cryo-electron tomography 3D imaging. From left to right: slice of the original data; direct volume rendering of the original data; foreground-background segmentation; color-coded four-class segmented data (background, spikes, membrane, lumen).

Prerecorded Talk

The live footage of the talk, including the Q&A, can be viewed on the session page, (Volume) Rendering.

Fast forward

Volume Rendering; Computer Graphics Techniques; Machine Learning Techniques; Scalar Field Data; Life Sciences


Cryo-electron tomography (cryo-ET) is a new 3D imaging technique with unprecedented potential for resolving submicron structural details. Existing volume visualization methods, however, are not able to reveal details of interest due to low signal-to-noise ratio. In order to design more powerful transfer functions, we propose leveraging soft segmentation as an explicit component of visualization for noisy volumes. Our technical realization is based on semi-supervised learning, where we combine the advantages of two segmentation algorithms. First, the weak segmentation algorithm provides good results for propagating sparse user-provided labels to other voxels in the same volume and is used to generate dense pseudo-labels. Second, the powerful deep-learning-based segmentation algorithm learns from these pseudo-labels to generalize the segmentation to other unseen volumes, a task that the weak segmentation algorithm fails at completely. The proposed volume visualization uses deep-learning-based segmentation as a component for segmentation-aware transfer function design. Appropriate ramp parameters can be suggested automatically through frequency distribution analysis. Furthermore, our visualization uses gradient-free ambient occlusion shading to further suppress the visual presence of noise, and to give structural detail the desired prominence. The cryo-ET data studied in our technical experiments are based on the highest-quality tilted series of intact SARS-CoV-2 virions. Our technique shows the high impact in target sciences for visual data analysis of very noisy volumes that cannot be visualized with existing techniques.