Indoor localization and visualization using a human-operated backpack system

Automated 3D modeling of building interiors is useful in applications such as virtual reality and entertainment. Using a human-operated backpack system equipped with 2D laser scanners and inertial measurement units (IMU), we develop scan matching based algorithms to localize the backpack in complex indoor environments such as a T-shaped corridor intersection, a staircase, and two indoor hallways from two separate floors connected by a staircase. When building 3D textured models, we find that the localization resulting from scan matching is not pixel accurate, resulting in misalignment between successive images used for texturing. To address this, we propose an image based pose estimation algorithm to refine the results from our scan matching based localization. Finally, we use the localization results within an image based renderer to enable virtual walkthroughs of indoor environments using imagery from cameras on the same backpack. Our renderer uses a three-step process to determine which image to display, and a RANSAC framework to determine homographies to mosaic neighboring images with common SIFT features. In addition, our renderer uses plane-fitted models of the 3D point cloud resulting from the laser scans to detect occlusions. We characterize the performance of our image based renderer on an unstructured set of 2709 images obtained during a five minute backpack data acquisition for a T-shaped corridor intersection.

[1]  C. N. Liu,et al.  Approximating discrete probability distributions with dependence trees , 1968, IEEE Trans. Inf. Theory.

[2]  Wolfram Burgard,et al.  Efficient estimation of accurate maximum likelihood maps in 3D , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[3]  Richard Szeliski,et al.  Image-based interactive exploration of real-world environments , 2004, IEEE Computer Graphics and Applications.

[4]  Yizhou Yu,et al.  Efficient View-Dependent Image-Based Rendering with Projective Texture-Mapping , 1998, Rendering Techniques.

[5]  Reinhard Koch,et al.  Image-Based Rendering from Uncalibrated Lightfields with Scalable Geometry , 2000, Theoretical Foundations of Computer Vision.

[6]  Paul Newman,et al.  Probabilistic Appearance Based Navigation and Loop Closing , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[7]  Wolfram Burgard,et al.  Probabilistic Robotics (Intelligent Robotics and Autonomous Agents) , 2005 .

[8]  Paul Newman,et al.  FAB-MAP: Probabilistic Localization and Mapping in the Space of Appearance , 2008, Int. J. Robotics Res..

[9]  Gérard G. Medioni,et al.  Visual loop closing using multi-resolution SIFT grids in metric-topological SLAM , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Bobby Bodenheimer,et al.  Synthesis and evaluation of linear motion transitions , 2008, TOGS.

[11]  Andrea Censi,et al.  An accurate closed-form estimate of ICP's covariance , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[12]  Michael Bosse,et al.  Unstructured lumigraph rendering , 2001, SIGGRAPH.

[13]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision (2nd ed) , 2003 .

[14]  Karl Granström,et al.  Learning to detect loop closure from range data , 2009, 2009 IEEE International Conference on Robotics and Automation.

[15]  Harry Shum,et al.  Review of image-based rendering techniques , 2000, Visual Communications and Image Processing.

[16]  A. Zakhor,et al.  Fast Surface Reconstruction and Segmentation with Ground-Based and Airborne LIDAR Range Data , 2009 .

[17]  Olivier Stasse,et al.  MonoSLAM: Real-Time Single Camera SLAM , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[19]  Steven M. Seitz,et al.  Finding paths through the world's photos , 2008, SIGGRAPH 2008.

[20]  Daniel G. Aliaga,et al.  Plenoptic stitching: a scalable method for reconstructing 3D interactive walk throughs , 2001, SIGGRAPH.

[21]  Avideh Zakhor,et al.  Indoor Localization Algorithms for a Human-Operated Backpack System , 2010 .

[22]  Evangelos E. Milios,et al.  Robot Pose Estimation in Unknown Environments by Matching 2D Range Scans , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[23]  James R. Bergen,et al.  Visual odometry , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[24]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[25]  Michael Bosse,et al.  Continuous 3D scan-matching with a spinning 2D laser , 2009, 2009 IEEE International Conference on Robotics and Automation.

[26]  Joachim Hertzberg,et al.  Globally consistent 3D mapping with scan matching , 2008, Robotics Auton. Syst..

[27]  Richard Szeliski,et al.  Modeling the World from Internet Photo Collections , 2008, International Journal of Computer Vision.

[28]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[29]  Leonard McMillan,et al.  Plenoptic Modeling: An Image-Based Rendering System , 2023 .