Tracking planes with Time of Flight cameras and J-linkage

In this paper, we propose a method for detection and tracking of multiple planes in sequences of Time of Flight (ToF) depth images. Our approach extends the recent J-linkage algorithm for estimation of multiple model instances in noisy data to tracking. Instead of randomly selecting plane hypotheses in every image, we propagate plane hypotheses through the sequence of images, resulting in a significant reduction of computational load in every frame. We also introduce a multi-pass scheme that allows detecting and tracking planes of varying spatial extent along with their boundaries. Our qualitative and quantitative evaluation shows that the proposed method can robustly detect planes and consistently track the hypotheses through sequences of ToF images.

[1]  Adrien Bartoli,et al.  A random sampling strategy for piecewise planar scene segmentation , 2007, Comput. Vis. Image Underst..

[2]  Masayuki Inaba,et al.  Plane segment finder: algorithm, implementation and applications , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[3]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[4]  Richard Szeliski,et al.  Piecewise planar stereo for image-based rendering , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[5]  Andreas Birk,et al.  Fast Registration Based on Noisy Planes With Unknown Correspondences for 3-D Mapping , 2010, IEEE Transactions on Robotics.

[6]  Andreas Birk,et al.  Fast plane detection and polygonalization in noisy 3D range images , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[7]  Konrad Schindler,et al.  Piecewise planar scene reconstruction from sparse correspondences , 2006, Image Vis. Comput..

[8]  Ruzena Bajcsy,et al.  Segmentation of range images as the search for geometric parametric models , 1995, International Journal of Computer Vision.

[9]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Dan Wang,et al.  Range image segmentation based on randomized Hough transform , 2005, Pattern Recognit. Lett..

[11]  Nassir Navab,et al.  Manifold Learning for ToF-based Human Body Tracking and Activity Recognition , 2010, BMVC.

[12]  Andrew Zisserman,et al.  A PLANE-SWEEP STRATEGY FOR THE 3D RECONSTRUCTION OF BUILDINGS FROM MULTIPLE IMAGES , 2000 .

[13]  Andrea Fusiello,et al.  Real-time Incremental J-Linkage for Robust Multiple Structures Estimation , 2010 .

[14]  Jörg Stückler,et al.  Towards Semantic Scene Analysis with Time-of-Flight Cameras , 2010, RoboCup.

[15]  Panos E. Trahanias,et al.  Iterative Computation of 3D Plane Parameters , 2000, BMVC.

[16]  Andrea Fusiello,et al.  Robust Multiple Structures Estimation with J-Linkage , 2008, ECCV.

[17]  Christian Goerick,et al.  Fast detection of arbitrary planar surfaces from unreliable 3D data , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[18]  Atsushi Imiya,et al.  Voting method for planarity and motion detection , 1999, Image Vis. Comput..

[19]  Wolfram Burgard,et al.  Learning compact 3D models of indoor and outdoor environments with a mobile robot , 2003, Robotics Auton. Syst..

[20]  Zhengyou Zhang,et al.  Iterative point matching for registration of free-form curves and surfaces , 1994, International Journal of Computer Vision.

[21]  Henrik I. Christensen,et al.  Multiple Plane Segmentation Using Optical Flow , 2002, BMVC.

[22]  David F. Fouhey,et al.  Multiple Plane Detection in Image Pairs Using J-Linkage , 2010, 2010 20th International Conference on Pattern Recognition.

[23]  Patrick Rives,et al.  Real-time Robust Detection of Planar Regions in a Pair of Images , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[24]  José Luis Lerma,et al.  Unsupervised robust planar segmentation of terrestrial laser scanner point clouds based on fuzzy clustering methods , 2008 .

[25]  Jana Kosecka,et al.  Piecewise planar city 3D modeling from street view panoramic sequences , 2009, CVPR.

[26]  Andrew W. Fitzgibbon,et al.  Markerless tracking using planar structures in the scene , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[27]  Markus Vincze,et al.  Incremental Model Selection for Detection and Tracking of Planar Surfaces , 2010, BMVC.

[28]  Jan-Michael Frahm,et al.  Piecewise planar and non-planar stereo for urban scene reconstruction , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[29]  Manolis I. A. Lourakis,et al.  Detecting Planes In An Uncalibrated Image Pair , 2002, BMVC.