Challenges and advances in interactive modeling

Many important computer graphics applications rely on 3D models of real-world scenes. The current approach for modeling such scenes is first to acquire depth and color data and then to build a model off line. The approach is slow, the acquisition devices are expensive, and the resulting models are incomplete and of uneven quality. We propose an interactive modeling strategy where an operator scans the scene with a handheld device and the model is built incrementally. The evolving model is displayed continually for immediate feedback. The advantages of the interactive modeling approach are short modeling times, good scene coverage, even sampling, and low cost. Implementing the interactive modeling pipeline poses the following challenges: devising an acquisition device that is lightweight, compact and robust, real-time depth extraction, real-time registration, and real-time incremental modeling. In this paper, we analyze these advantages and challenges in light of our experience in designing an interactive modeling system.