Sift-based Camera Localization using Reference Objects for Application in Multi-camera Environments and Robotics

In this contribution, we present a unified approach to improve the localization and the perception of a robot in a new environment by using already installed cameras. Using our approach we are able to localize arbitrary cameras in multi-camera environments while automatically extending the camera network in an online, unattended, real-time way. This way, all cameras can be used to improve the perception of the scene, and additional cameras can be added in real-time, e.g., to remove blind spots. To this end, we use the Scale-invariant feature transform (SIFT) and at least one arbitrary known-size reference object to enable camera localization. Then we apply non-linear optimization of the relative pose estimate and we use it to iteratively calibrate the camera network as well as to localize arbitrary cameras, e.g. of mobile phones or robots, inside a multi-camera environment. We performed an evaluation on synthetic as well as real data to demonstrate the applicability of the proposed approach.

[1]  Jun Liu,et al.  Automatic Camera Calibration and Scene Reconstruction with Scale-Invariant Features , 2006, ISVC.

[2]  David Nistér,et al.  An efficient solution to the five-point relative pose problem , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[3]  Yann Frauel,et al.  Vision-Based Localization for Mobile Robots Using a Set of Known Views , 2008, ISVC.

[4]  Kenneth Levenberg.  A METHOD FOR THE SOLUTION OF CERTAIN NON – LINEAR PROBLEMS IN LEAST SQUARES , 1944 .

[5]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[6]  Rainer Stiefelhagen,et al.  Visual Focus of Attention in Dynamic Meeting Scenarios , 2008, MLMI.

[7]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[8]  D. Marquardt.  An Algorithm for Least-Squares Estimation of Nonlinear Parameters , 1963 .

[9]  Pascal Fua,et al.  On benchmarking camera calibration and multi-view stereo for high resolution imagery , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Francis K. H. Quek,et al.  Meeting room configuration and multiple camera calibration in meeting analysis , 2005, ICMI '05.

[11]  Joachim Denzler,et al.  Active Self-calibration of Multi-camera Systems , 2010, DAGM-Symposium.

[12]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[13]  Jordi Luque,et al.  Multimodal identification and localization of users in a smart environment , 2008, Journal on Multimodal User Interfaces.

[14]  David Nister,et al.  Recent developments on direct relative orientation , 2006 .

[15]  Richard Szeliski,et al.  Modeling the World from Internet Photo Collections , 2008, International Journal of Computer Vision.

[16]  Rainer Stiefelhagen,et al.  Automatic Calibration of Camera Networks based on Local Motion Features , 2008 .

[17]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Tomás Svoboda,et al.  A Convenient Multicamera Self-Calibration for Virtual Environments , 2005, Presence: Teleoperators & Virtual Environments.

[19]  Thomas F. Coleman,et al.  An Interior Trust Region Approach for Nonlinear Minimization Subject to Bounds , 1993, SIAM J. Optim..

[20]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[21]  Gernot A. Fink,et al.  Multi-modal and multi-camera attention in smart environments , 2009, ICMI-MLMI '09.

[22]  V. Rodehorst,et al.  EVALUATION OF RELATIVE POSE ESTIMATION METHODS FOR MULTI-CAMERA SETUPS , 2008 .