Towards efficient support relation extraction from RGBD images

To extract reasonable support relations from "RGB+depth" (RGBD) images, it is very important to achieve good scene understanding. This paper proposes a novel approach to extracting accurate support relationships by analyzing the RGBD images of indoor scenes. Noting that the support relations and structure classes of indoor images are inherently related to physical stability, we construct an improved energy function that embodies this stability. We then infer the support relations and structure classes from indoor RGBD images by minimizing this energy function. Moreover, the authors succeed in improving the segmentation quality of RGBD images using the inferred results as input. Compared with previous methods, our approach produces more reasonable support relations and structure classes, where physical stability function is taken into account for resolving the optimization problem. We use the NYU-Depth2 dataset as the training data, and experimental results show that the proposed RGBD image segmentation method based on support relation abstraction produces more accurate results than segmentation methods based on ground-truth support relations.

[1]  Tsuhan Chen,et al.  3D-Based Reasoning with Blocks, Support, and Stability , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Jessica B. Hamrick,et al.  Simulation as an engine of physical scene understanding , 2013, Proceedings of the National Academy of Sciences.

[3]  David A. Forsyth,et al.  Recovering free space of indoor scenes from a single image , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Mohammed Bennamoun,et al.  A novel local surface feature for 3D object recognition under clutter and occlusion , 2015, Inf. Sci..

[5]  Hema Swetha Koppula,et al.  Learning human activities and object affordances from RGB-D videos , 2012, Int. J. Robotics Res..

[6]  Alexei A. Efros,et al.  Recovering Occlusion Boundaries from an Image , 2011, International Journal of Computer Vision.

[7]  Katsushi Ikeuchi,et al.  Beyond Point Clouds: Scene Understanding by Reasoning Geometry and Physics , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Alexei A. Efros,et al.  From 3D scene geometry to human workspace , 2011, CVPR 2011.

[9]  Jonathan T. Barron,et al.  A category-level 3-D object dataset: Putting the Kinect to work , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[10]  Alexei A. Efros,et al.  Blocks World Revisited: Image Understanding Using Qualitative Geometry and Mechanics , 2010, ECCV.

[11]  Xuelong Li,et al.  GA-SIFT: A new scale invariant feature transform for multispectral image using geometric algebra , 2014, Inf. Sci..

[12]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[13]  James M. Coughlan,et al.  Manhattan World: Orientation and Outlier Detection by Bayesian Inference , 2003, Neural Computation.

[14]  Derek Hoiem,et al.  Indoor Segmentation and Support Inference from RGBD Images , 2012, ECCV.

[15]  Yun Jiang,et al.  Hallucinated Humans as the Hidden Context for Labeling 3D Scenes , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[17]  Luc Van Gool,et al.  What makes a chair a chair? , 2011, CVPR 2011.

[18]  Ke Lu,et al.  3D model retrieval and classification by semi-supervised learning with content-based similarity , 2014, Inf. Sci..

[19]  Takeo Kanade,et al.  Geometric reasoning for single image structure recovery , 2009, CVPR.

[20]  Isabelle Bloch,et al.  A constraint propagation approach to structural model based image segmentation and recognition , 2013, Inf. Sci..

[21]  Luc Vincent,et al.  Watersheds in Digital Spaces: An Efficient Algorithm Based on Immersion Simulations , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Takeo Kanade,et al.  Estimating Spatial Layout of Rooms using Volumetric Reasoning about Objects and Surfaces , 2010, NIPS.

[23]  Yun Jiang,et al.  Learning to place new objects in a scene , 2012, Int. J. Robotics Res..

[24]  Jitendra Malik,et al.  Perceptual Organization and Recognition of Indoor Scenes from RGB-D Images , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.