Accelerating NeRF with the Visual Hull

Neural rendering methods for learning the appearance and geometry of 3D scenes have gained tremendous popularity since 2020. In this field, NeRF or Neural Radiance Fields is the bestknown methodology. Given a collection of multi-view images and their camera models, NeRF optimizes a neural network to learn the color and scene geometry that render the input images according to classical volumetric rendering techniques. NeRF operates in a self-supervised manner and provides a remarkable level of detail, but the time-consuming optimization process remains a major limitation. This paper reviews the Voxel-Accelerated NeRF (VaxNeRF), a simple acceleration strategy for NeRF proposed in 2021. VaxNeRF reduces the number of point queries required in training and inference time by considering only the region of space corresponding to the visual hull, i.e., the maximum volume compatible with the object silhouettes given by the multi-view collection. VaxNeRF requires only coarse foreground-background segmentation masks and minimal changes to the original NeRF code to improve speed by a factor of 2-8, without any performance degradation.

The source code and documentation for this algorithm are available from the web page of this article 1 . Usage instructions are included in the README file of the archive. The authors' original method implementation is available here 2 . This is an MLBriefs article, the source code has not been reviewed!