Active Visual Object Reconstruction using D-, E-, and T-Optimal Next Best Views

In visual 3-D reconstruction tasks with mobile cameras, one wishes to move the cameras so that they provide the views that lead to the best reconstruction result. When the camera motion is adapted during the reconstruction, the view of interest is the next best view for the current shape estimate. We present such a next best view planning approach for visual 3-D reconstruction. The reconstruction is based on a probabilistic state estimation with sensor actions. The next best view is determined by a metric of the state estimation's uncertainty. We compare three metrics: D-optimality, which is based on the entropy and corresponds to the (D)eterminant of the covariance matrix of a Gaussian distribution, E-optimality, and T-optimality, which are based on (E)igenvalues or on the (T)race of this matrix, respectively. We show the validity of our approach with a simulation as well as real-world experiments, and compare reconstruction accuracy and computation time for the optimality criteria.

[1]  Heinrich Niemann,et al.  Efficient Feature Tracking for Long Video Sequences , 2004, DAGM-Symposium.

[2]  J. L. Roux.  An Introduction to the Kalman Filter , 2003 .

[3]  Kiriakos N. Kutulakos,et al.  A Theory of Shape by Space Carving , 2000, International Journal of Computer Vision.

[4]  J. Denzler,et al.  An Information Theoretic Approach to Optimal Sensor Data Selection for State Estimation , 2002 .

[5]  Kiriakos N. Kutulakos,et al.  Recovering shape by purposive viewpoint adjustment , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  Joachim Denzler,et al.  Information Theoretic Sensor Data Selection for Active Object Recognition and State Estimation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[8]  Kiriakos N. Kutulakos,et al.  Global surface reconstruction by purposive control of observer motion , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Samia Boukir,et al.  Structure From Controlled Motion , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Éric Marchand,et al.  Active Vision for Complete Scene Reconstruction and Exploration , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Joachim Denzler,et al.  An Information Theoretic Approach for Next Best View Planning in 3-D Reconstruction , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[12]  Andrew J. Davison,et al.  Active search for real-time vision , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[13]  Samia Boukir,et al.  Optimal estimation of 3D structures using visual servoing , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Éric Marchand,et al.  An Autonomous Active Vision System for Complete and Accurate 3D Scene Reconstruction , 1999, International Journal of Computer Vision.

[15]  Wolfgang Niem,et al.  Camera viewpoint control for the automatic reconstruction of 3D objects , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.