A multi-modal object attention system for a mobile robot

Robot companions are intended for operation in private homes with naive users. For this purpose, they need to be endowed with natural interaction capabilities. Additionally, such robots will need to be taught unknown objects that are present in private homes. We present a multi-modal object attention system that is able to identify objects referenced by the user with gestures and verbal instructions. The proposed system can detect known and unknown objects and stores newly acquired object information in a scene model for later retrieval. This way, the growing knowledge base of the robot companion improves the interaction quality as the robot can more easily focus its attention on objects it has been taught previously.

[1]  Jannik Fritsch,et al.  Supporting advanced interaction capabilities on a mobile robot with a flexible control system , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[2]  Jannik Fritsch,et al.  Recognition of Deictic Gestures with Context , 2004, DAGM-Symposium.

[3]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[4]  Pietro Perona,et al.  A Bayesian approach to unsupervised one-shot learning of object categories , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[5]  Sebastian Lang,et al.  BIRON - The Bielefeld Robot Companion , 2004 .

[6]  F. Kaplan,et al.  The challenges of joint attention , 2006 .

[7]  Tony R. Martinez,et al.  Reduction Techniques for Instance-Based Learning Algorithms , 2000, Machine Learning.

[8]  Richard T. Vaughan,et al.  The Player/Stage Project: Tools for Multi-Robot and Distributed Sensor Systems , 2003 .

[9]  Michael Isard,et al.  A mixed-state condensation tracker with automatic model-switching , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[10]  Cory D. Kidd,et al.  HUMANOID ROBOTS AS COOPERATIVE PARTNERS FOR PEOPLE , 2004 .

[11]  Michael J. Black,et al.  A Probabilistic Framework for Matching Temporal Trajectories: CONDENSATION-Based Recognition of Gestures and Expressions , 1998, ECCV.

[12]  Frank Lömker,et al.  A Multimodal System for Object Learning , 2002, DAGM-Symposium.

[13]  Pietro Perona,et al.  A Bayesian approach to unsupervised one-shot learning of object categories , 2003, ICCV 2003.

[14]  Christian Bauckhage,et al.  An active memory as a model for information fusion , 2004 .

[15]  Jannik Fritsch,et al.  Interactive object learning for robot companions using mosaic images , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[16]  Toshi Takamori,et al.  Multi-Modal Interaction of Human and Home Robot in the Context of Room Map Generation , 2002, Auton. Robots.

[17]  Sebastian Wrede,et al.  A Flexible Infrastructure for the Development of a Robot Companion with Extensible HRI-Capabilities , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[18]  Akira Utsumi,et al.  View-based detection of 3-D interaction between hands and real objects , 2004, ICPR 2004.

[19]  Sebastian Lang,et al.  Providing the basis for human-robot-interaction: a multi-modal attention system for a mobile robot , 2003, ICMI '03.