Dense and Occlusion-Robust Multi-View Stereo for Unstructured Videos

Jian Wei, Benjamin Resch, and Hendrik P. A. Lensch

(Department of Computer Graphics, Tübingen University, Germany)

13th Conference on Computer and Robot Vision (CRV 2016)

Victoria, British Columbia

June 1-3, 2016

We present an efficient multi-view stereo system for dense reconstruction of homogeneous areas from videos with arbitrary camera trajectories. Most techniques have difficulties in recovering textureless areas, weak robustness for occlusion, and low efficiency on the massive redundant data provided by densely sampled frames. Our key idea is to use the measurable depth at object edges to recover the enclosed geometries assuming smooth surfaces, which is appropriate for most scenes. The depth values at edges are calculated by considering individual viewing rays to allow for sharp discontinuities. Then we employ perspective diffusion to create smooth surfaces between edges. The wrong interpolants between fore- and background edges are effectively invalidated by detecting large depth standard deviation which is approximated using the edge depth from other views. For reliable invalidation, the accuracy of edge depth is improved by advanced score evaluation and two-scale image sampling. Finally, we fill up holes and correct the wrongly sunken surfaces by propagating the closest valid depth across views. The pixel-level operations throughout support high parallelism, and our high-quality depth maps allow dense scene representation by point clouds.

Paper Slides (pdf) Bibtex