Differentiable Design Galleries: A Differentiable Approach to Explore the Design Space of Transfer Functions

Bo Pan, Jiaying Lu, Haoxuan Li, Weifeng Chen, Yiyao Wang, Minfeng Zhu, Chenhao Yu, Wei Chen

Room: 103

2023-10-26T22:48:00ZGMT-0600Change your timezone on the schedule page
Exemplar figure, described by caption below
We propose Differentiable Design Galleries, a transfer function design approach based on deep learning and differentiable rendering to assist users in exploring the design space of transfer functions.
Fast forward
Full Video

Transfer function, direct volume rendering, deep learning, generative models, differentiable rendering


The transfer function is crucial for direct volume rendering (DVR) to create an informative visual representation of volumetric data. However, manually adjusting the transfer function to achieve the desired DVR result can be time-consuming and unintuitive. In this paper, we propose Differentiable Design Galleries, an image-based transfer function design approach to help users explore the design space of transfer functions by taking advantage of the recent advances in deep learning and differentiable rendering. Specifically, we leverage neural rendering to learn a latent design space, which is a continuous manifold representing various types of implicit transfer functions. We further provide a set of interactive tools to support intuitive query, navigation, and modification to obtain the target design, which is represented as a neural-rendered design exemplar. The explicit transfer function can be reconstructed from the target design with a differentiable direct volume renderer. Experimental results on real volumetric data demonstrate the effectiveness of our method.