Purposive three-dimensional reconstruction by means of a controlled environment

Retrieving 3D data using imaging devices is a relevant task for many applications in medical imaging, surveillance, industrial quality control, and others. As soon as we gain procedural control over parameters of the imaging device, we encounter the necessity of well-defined reconstruction goals and we need methods to achieve them. Hence, we enter next-best-view planning. In this work, we present a formalization of the abstract view planning problem and deal with different planning aspects, whereat we focus on using an intensity camera without active illumination. As one aspect of view planning, employing a controlled environment also provides the planning and reconstruction methods with additional information. We incorporate the additional knowledge of camera parameters into the Kanade-Lucas-Tomasi method used for feature tracking. The resulting Guided KLT tracking method benefits from a constrained optimization space and yields improved accuracy while regarding the uncertainty of the additional input. Serving other planning tasks dealing with known objects, we propose a method for coarse registration of 3D surface triangulations. By the means of exact surface moments of surface triangulations we establish invariant surface descriptors based on moment invariants. These descriptors allow to tackle tasks of surface registration, classification, retrieval, and clustering, which are also relevant to view planning. In the main part of this work, we present a modular, online approach to view planning for 3D reconstruction. Based on the outcome of the Guided KLT tracking, we design a planning module for accuracy optimization with respect to an extended E-criterion. Further planning modules endow non-discrete surface estimation and visibility analysis. The modular nature of the proposed planning system allows to address a wide range of specific instances of view planning. The theoretical findings in this work are underlined by experiments evaluating the relevant terms.

[1]  F. R. Gantmakher.  The Theory of Matrices , 1984 .

[2]  Peter K. Allen,et al.  Constraint-Based Sensor Planning for Scene Modeling , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Joachim Denzler,et al.  Online Next-Best-View Planning for Accuracy Optimization Using an Extended E-Criterion , 2010, 2010 20th International Conference on Pattern Recognition.

[4]  Youfu Li,et al.  The research of an automatic object reconstruction method based on limit visible region of the laser-scanning vision system , 2010 .

[5]  Éric Marchand,et al.  Active Vision for Complete Scene Reconstruction and Exploration , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  F. Pukelsheim.  Optimal Design of Experiments , 1993 .

[7]  Joachim Denzler,et al.  Active Self-calibration of Multi-camera Systems , 2010, DAGM-Symposium.

[8]  Shengyong Chen,et al.  Active Sensor Planning for Multiview Vision Tasks , 2008 .

[9]  Joachim Denzler,et al.  View Planning for 3D Reconstruction Using Time-of-Flight Camera Data , 2009, DAGM-Symposium.

[10]  Richard Pito,et al.  A Solution to the Next Best View Problem for Automated Surface Acquisition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Narendra Ahuja,et al.  Gross motion planning—a survey , 1992, CSUR.

[12]  Andrew Zisserman,et al.  Self-Calibration from Image Triplets , 1996, ECCV.

[13]  Simon Baker,et al.  Lucas-Kanade 20 Years On: A Unifying Framework , 2004, International Journal of Computer Vision.

[14]  Tamal K. Dey,et al.  Tight cocone: a water-tight surface reconstructor , 2003, SM '03.

[15]  Joachim Denzler,et al.  Combined GKLT Feature Tracking and Reconstruction for Next Best View Planning , 2009, DAGM-Symposium.

[16]  Joachim Denzler,et al.  Benchmarking 3D Reconstructions from Next Best View Planning , 2007, MVA.

[17]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[18]  Marc Rioux,et al.  Description of shape information for 2-D and 3-D objects , 2000, Signal Process. Image Commun..

[19]  Sim Heng Ong,et al.  3D registration of partially overlapping surfaces using a volumetric approach , 2007, Image Vis. Comput..

[20]  Emanuele Trucco,et al.  Improving Feature Tracking with Robust Statistics , 1999, Pattern Analysis & Applications.

[21]  Joachim Denzler,et al.  Experimental Evaluation of Relative Pose Estimation Algorithms , 2008, VISAPP.

[22]  Christian Bräuer-Burchardt,et al.  3D shape measurement with phase correlation based fringe projection , 2007, SPIE Optical Metrology.

[23]  Joachim Denzler,et al.  An Information Theoretic Approach for Next Best View Planning in 3-D Reconstruction , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[24]  H. C. Longuet-Higgins,et al.  A computer algorithm for reconstructing a scene from two projections , 1981, Nature.

[25]  Kamal K. Gupta,et al.  View planning for exploration via maximal C-space entropy reduction for robot mounted range sensors , 2007, Adv. Robotics.

[26]  Hein A.M. Daanen,et al.  Automatic feature detection in 3D human body scans , 2004 .

[27]  Heinrich Niemann,et al.  Active Visual Object Reconstruction using D-, E-, and T-Optimal Next Best Views , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Joachim Denzler,et al.  Coarse registration of 3D surface triangulations based on moment invariants with applications to object alignment and identification , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[29]  R. Bajcsy.  Active perception , 1988 .

[30]  Jan Flusser,et al.  Moment Invariants in Image Analysis , 2007 .

[31]  Shmuel Peleg,et al.  Lucas-Kanade without Iterative Warping , 2006, 2006 International Conference on Image Processing.

[32]  Benjamin Loriot,et al.  Non-Model Based Method for an Automation of 3D Acquisition and Post-Processing , 2009 .

[33]  Simon Winkelbach,et al.  Fast Random Sample Matching of 3d Fragments , 2004, DAGM-Symposium.

[34]  H. Kuhn.  The Hungarian method for the assignment problem , 1955 .

[35]  Mohammed Bennamoun,et al.  A Novel Representation and Feature Matching Algorithm for Automatic Pairwise Registration of Range Images , 2005, International Journal of Computer Vision.

[36]  R. Courant,et al.  Methods of Mathematical Physics , 1962 .

[37]  Samia Boukir,et al.  Structure From Controlled Motion , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Tamal Bose,et al.  Digital Signal and Image Processing , 2003 .

[39]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Jean Ponce,et al.  Accurate Camera Calibration from Multi-View Stereo and Bundle Adjustment , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[41]  Kiriakos N. Kutulakos,et al.  A Theory of Shape by Space Carving , 2000, International Journal of Computer Vision.

[42]  Jean-Paul Chilès,et al.  Wiley Series in Probability and Statistics , 2012 .

[43]  Joachim Denzler,et al.  Extending GKLT Tracking - Feature Tracking for Controlled Environments with Integrated Uncertainty Estimation , 2009, SCIA.

[44]  Enrique Dunn,et al.  Parisian camera placement for vision metrology , 2006, Pattern Recognit. Lett..

[45]  P. J. Huber.  Robust Estimation of a Location Parameter , 1964 .

[46]  Martin Kampel,et al.  Next view planning for a combination of passive and active acquisition techniques , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[47]  Jan Flusser,et al.  On the independence of rotation moment invariants , 2000, Pattern Recognit..

[48]  Roger Mohr,et al.  Optimal camera placement for accurate reconstruction , 2002, Pattern Recognit..

[49]  Joachim Denzler,et al.  A Fast Approach for Pixelwise Labeling of Facade Images , 2010, 2010 20th International Conference on Pattern Recognition.

[50]  Claus B. Madsen,et al.  A Viewpoint Planning Strategy for Determining True Angles on Polyhedral Objects by Camera Alignment , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[51]  Robert B. Fisher,et al.  A Best Next View Selection Algorithm Incorporating a Quality Criterion , 1998, BMVC.

[52]  Kostas Daniilidis,et al.  Fully Automatic Registration of 3D Point Clouds , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[53]  William R. Scott,et al.  Model-based view planning , 2007, Machine Vision and Applications.

[54]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[55]  Christophe Dumont,et al.  A next-best-view system for autonomous 3-D object reconstruction , 2000, IEEE Trans. Syst. Man Cybern. Part A.

[56]  Joachim Denzler,et al.  KLT Tracking Using Intrinsic and Extrinsic Camera Parameters in Consideration of Uncertainty , 2008, VISAPP.

[57]  Dong Xu,et al.  3-D Surface Moment Invariants , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[58]  Ruzena Bajcsy,et al.  Occlusions as a Guide for Planning the Next View , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[59]  Ming-Kuei Hu,et al.  Visual pattern recognition by moment invariants , 1962, IRE Trans. Inf. Theory.

[60]  Roberto Cipolla,et al.  Camera Calibration from Vanishing Points in Image of Architectural Scenes , 1999, BMVC.

[61]  Yohan Fougerolle,et al.  Fully automatic 3D digitization of unknown objects , 2010, Electronic Imaging.

[62]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[63]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[64]  Peter F. Sturm,et al.  A Factorization Based Algorithm for Multi-Image Projective Structure and Motion , 1996, ECCV.