Automated Place Classification Using Object Detection

Places in an environment can be described by the objects they contain. This paper discusses the completely automated integration of object detection and place classification in a single system. We first perform automated learning of object-place relations from an online annotated database. We then train object detectors on some of the most frequently occurring objects. Finally, we use detection scores as well as learned object-place relations to perform place classification of images. We also discuss areas for improvement and the application of this work to informed visual search. As a whole, the system demonstrates the automated acquisition of training data containing labeled instances (i.e. bounding boxes) and the performance of a state-of-the-art object detection technique trained on this data to perform place classification of realistic indoor scenes.

[1]  James J. Little,et al.  Curious George: An attentive semantic robot , 2008, Robotics Auton. Syst..

[2]  Benjamin Kuipers,et al.  The Spatial Semantic Hierarchy , 2000, Artif. Intell..

[3]  Antonio Torralba,et al.  LabelMe: A Database and Web-Based Tool for Image Annotation , 2008, International Journal of Computer Vision.

[4]  T. Southey,et al.  Object Discovery through Motion, Appearance and Shape , 2006 .

[5]  Roland Siegwart,et al.  Bayesian space conceptualization and place classification for semantic maps in mobile robotics , 2008, Robotics Auton. Syst..

[6]  Frank Dellaert,et al.  Semantic Modeling of Places using Objects , 2007, Robotics: Science and Systems.

[7]  James J. Little,et al.  Automated Spatial-Semantic Modeling with Applications to Place Labeling and Informed Search , 2009, 2009 Canadian Conference on Computer and Robot Vision.

[8]  Barbara Caputo,et al.  Visual Servoing to Help Camera Operators Track Better , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  Barbara Caputo,et al.  Multi-modal Semantic Place Classification , 2010, Int. J. Robotics Res..

[10]  Antonio Torralba,et al.  Recognizing indoor scenes , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[12]  Barbara Caputo,et al.  SVM-based discriminative accumulation scheme for place recognition , 2008, 2008 IEEE International Conference on Robotics and Automation.

[13]  Daniel P. Huttenlocher,et al.  Spatial priors for part-based recognition using statistical models , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[14]  Antonio Torralba,et al.  Context-based vision system for place and object recognition , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[15]  David A. McAllester,et al.  A discriminatively trained, multiscale, deformable part model , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Thomas Hofmann,et al.  Multiple instance learning with generalized support vector machines , 2002, AAAI/IAAI.

[17]  Ben J. A. Kröse,et al.  A Geometrically Constrained Image Similarity Measure for Visual Mapping, Localization and Navigation , 2007, EMCR.

[18]  Ben J. A. Kröse,et al.  BIRON, where are you? Enabling a robot to learn new places in a real home environment by integrating spoken dialog and visual localization , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[19]  P. Bartlett,et al.  Probabilities for SV Machines , 2000 .

[20]  Yali Amit,et al.  POP: Patchwork of Parts Models for Object Recognition , 2007, International Journal of Computer Vision.

[21]  Robert Marti,et al.  Which is the best way to organize/classify images by content? , 2007, Image Vis. Comput..

[22]  Keiji Nagatani,et al.  Topological simultaneous localization and mapping (SLAM): toward exact localization without explicit localization , 2001, IEEE Trans. Robotics Autom..

[23]  Pietro Perona,et al.  A Bayesian hierarchical model for learning natural scene categories , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).