Exploiting depth camera for 3D spatial relationship interpretation

Interpretation of spatial relations between objects is essential to many applications such as robotics, video surveillance, spatial reasoning, and scene understanding. Current models for spatial logic are two-dimensional. With the advance in new sensing technology, inexpensive depth sensors become widely available and 3D scene reconstruction can be applied in various application scenarios. In this paper, we propose a 3D spatial logic and algorithms for interpretation of spatial relationships among objects in 3D space. More specifically, these techniques are developed for LVDBMS (Live Video DataBase Management System), a generic platform for live video computing. We extend the original directional relationships into 3D directional relationships, and introduce a simple yet effective way to build 3D object models based on depth sensors. A highly accurate and efficient algorithm is also proposed to compute the spatial relationships between two objects by sampling the entire space from the reference object. Experimental results based on a real indoor scene and an RGB-D dataset are given to demonstrate the effectiveness of our techniques.

[1]  Andrew S. Glassner,et al.  An introduction to ray tracing , 1989 .

[2]  Dieter Fox,et al.  A large-scale hierarchical multi-view RGB-D object dataset , 2011, 2011 IEEE International Conference on Robotics and Automation.

[3]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[4]  Alexei A. Efros,et al.  From 3D scene geometry to human workspace , 2011, CVPR 2011.

[5]  Alan L. Yuille,et al.  Manhattan World: compass direction from a single image by Bayesian inference , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[6]  Anca L. Ralescu,et al.  Spatial organization in 2D segmented images: representation and recognition of primitive spatial relations , 1994, CVPR 1994.

[7]  Jiming Liu,et al.  A Method of Spatial Reasoning Based on Qualitative Trigonometry , 1998, Artif. Intell..

[8]  J. Keller,et al.  Learning spatial relationships in computer vision , 1996, Proceedings of IEEE 5th International Fuzzy Systems.

[9]  Benjamin Kuipers,et al.  Navigation and Mapping in Large Scale Space , 1988, AI Mag..

[10]  Isabelle Bloch,et al.  Fuzzy spatial relationships for image processing and interpretation: a review , 2005, Image Vis. Comput..

[11]  Donald Meagher,et al.  Geometric modeling using octree encoding , 1982, Computer Graphics and Image Processing.

[12]  Kourosh Khoshelham,et al.  Accuracy analysis of kinect depth data , 2012 .

[13]  Benjamin Rosman,et al.  Learning spatial relationships between objects , 2011, Int. J. Robotics Res..

[14]  James M. Keller,et al.  Comparison of spatial relation definitions in computer vision , 1995, Proceedings of 3rd International Symposium on Uncertainty Modeling and Analysis and Annual Conference of the North American Fuzzy Information Processing Society.

[15]  Rüdiger Dillmann,et al.  Using spatial relations of objects in real world scenes for scene structuring and scene understanding , 2011, 2011 15th International Conference on Advanced Robotics (ICAR).

[16]  Derek Hoiem,et al.  Recovering the spatial layout of cluttered rooms , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[17]  Kien A. Hua,et al.  Real-Time Query Processing on Live Videos in Networks of Distributed Cameras , 2010, Int. J. Interdiscip. Telecommun. Netw..

[18]  MAX J. EGENHOFER,et al.  Point Set Topological Relations , 1991, Int. J. Geogr. Inf. Sci..

[19]  M. Egenhofer,et al.  Point-Set Topological Spatial Relations , 2001 .

[20]  Kien A. Hua,et al.  A general framework for managing and processing live video data with privacy protection , 2011, Multimedia Systems.

[21]  Isabelle Bloch,et al.  Fuzzy Relative Position Between Objects in Image Processing: A Morphological Approach , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Thorsten Joachims,et al.  Labeling 3D scenes for Personal Assistant Robots , 2011, ArXiv.

[23]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..