Gipuma: Massively Parallel Multi-view Stereo Reconstruction

We describe a method for dense multi-view matching and 3D point cloud reconstruction, which has been developed at the Institute of Geodesy and Photogrammetry of ETH Zurich. The method generates high-quality point clouds from oriented images very efficiently, by massively parallel processing on graphics processing units (GPUs). It is available as open-source code. Technically, the method is an extension of the PatchMatch Stereo algorithm: 3D depth values and surface normal vectors are iteratively propagated across the image and refined, in order to find a maximally photo-consistent depth map and normal field for each view. Photo-consistency is computed in a slanted tangent plane, such that the reconstruction does not suffer from fronto-parallel bias. We extend PatchMatch to 3D scene space, such that photo-consistency can be aggregated over multiple views, which allows for more robust and more accurate depth estimation. Moreover, the sequential propagation is replaced by a local, diffusion-like scheme, such that the computation can be massively parallelised. All computations are local, thus computation time is linear in the image size and inversely proportional to the number of parallel threads. Moreover, memory requirements are also modest, since only four values per pixel must be stored. Experiments on benchmark datasets show that our method delivers point clouds (respectively, surfaces) with high accuracy and completeness, across a range of applications.