Differentiable Direct Volume Rendering

Sebastian Weiss, Rüdiger Westermann

View presentation: 2021-10-28T15:15:00Z GMT-0600 Change your timezone on the schedule page
2021-10-28T15:15:00Z
Exemplar figure, described by caption below
We present a differentiable volume rendering solution that provides differentiability of all continuous parameters of the volume rendering process. A fully differentiable direct volume renderer is used for (left) viewpoint optimization, (middle) transfer function optimization, and (right) optimization of voxel properties using an emission-absorption model with rgb-alpha transfer functions
Fast forward

Direct link to video on YouTube: https://youtu.be/5E2I8iDKVIc

Abstract

We present a differentiable volume rendering solution that provides differentiability of all continuous parameters of the volume rendering process. This differentiable renderer is used to steer the parameters towards a setting with an optimal solution of a problem-specific objective function. We have tailored the approach to volume rendering by enforcing a constant memory footprint via analytic inversion of the blending functions. This makes it independent of the number of sampling steps through the volume and facilitates the consideration of small-scale changes. The approach forms the basis for automatic optimizations regarding external parameters of the rendering process and the volumetric density field itself. We demonstrate its use for automatic viewpoint selection using differentiable entropy as objective, and for optimizing a transfer function from rendered images of a given volume. Optimization of per-voxel densities is addressed in two different ways: First, we mimic inverse tomography and optimize a 3D density field from images using an absorption model. This simplification enables comparisons with algebraic reconstruction techniques and state-of-the-art differentiable path tracers. Second, we introduce a novel approach for tomographic reconstruction from images using an emission-absorption model with post-shading via an arbitrary transfer function.