Pruning multi-view stereo net for efficient 3D reconstruction

Abstract How can we perform an efficient 3D reconstruction with high accuracy and completeness, in the presence of non-Lambertian surface and low textured regions? This paper aims at fast quality 3D reconstruction, best near real time. While deep learning approaches perform very well in multi-view stereo (MVS), the high complexity of models makes them inapplicable in practical applications. Few works were explored to accelerate deep learning-based 3D reconstruction approaches. In this paper, we take an unprecedented attempt to compress and accelerate these models via pruning their redundant parameters. We introduce an efficient channel pruning method for 2D convolutional neural networks (CNNs) based on a mixed back propagation process, where a soft mask is learned to prune the channels using a fast iterative shrinkage-thresholding algorithm. While in 3D CNNs, we train a large multi-scale CNNs architecture and observe that only utilizing one module enough for the 3D reconstruction, which can still maintain the performance of the full-precision model. We achieve an efficient MVS reconstruction system up to 2 times faster, in contrast to the state-of-the-arts, while maintaining comparable model accuracy and even better completeness.