Mirror Perspective-Taking with a Humanoid Robot

The ability to use a mirror as an instrument for spatial reasoning enables an agent to make meaningful inferences about the positions of objects in space based on the appearance of their reflections in mirrors. The model presented in this paper enables a robot to infer the perspective from which objects reflected in a mirror appear to be observed, allowing the robot to use this perspective as a virtual camera. Prior work by our group presented an architecture through which a robot learns the spatial relationship between its body and visual sense, mimicking an early form of self-knowledge in which infants learn about their bodies and senses through their interactions with each other. In this work, this self-knowledge is utilized in order to determine the mirror's perspective. Witnessing the position of its end-effector in a mirror in several distinct poses, the robot determines a perspective that is consistent with these observations. The system is evaluated by measuring how well the robot's predictions of its end-effector's position in 3D, relative to the robot's egocentric coordinate system, and in 2D, as projected onto it's cameras, match measurements of a marker tracked by its stereo vision system. Reconstructions of the 3D position end-effector, as computed from the perspective of the mirror, are found to agree with the forward kinematic model within a mean of 31:55mm. When observed directly by the robot's cameras, reconstructions agree within 5:12mm. Predictions of the 2D position of the end-effector in the visual field agree with visual measurements within a mean of 18:47 pixels, when observed in the mirror, or 5:66 pixels, when observed directly by the robot's cameras.

[1]  G. G. Gallop.  Chimpanzees: self-recognition. , 1970, Science.

[2]  J. Denavit,et al.  A kinematic notation for lower pair mechanisms based on matrices , 1955 .

[3]  Diana Reiss,et al.  Self-recognition in an Asian elephant , 2006, Proceedings of the National Academy of Sciences.

[4]  E W Menzel,et al.  Chimpanzee (Pan troglodytes) spatial problem solving with the use of mirrors and televised equivalents of mirrors. , 1985, Journal of comparative psychology.

[5]  Jun'ichi Takeno,et al.  Experiments and examination of mirror image cognition using a small robot , 2005, 2005 International Symposium on Computational Intelligence in Robotics and Automation.

[6]  Bennett I. Bertenthal,et al.  Development of Self-Recognition in the Infant. , 1978 .

[7]  Aude Billard,et al.  Online Learning of the Body Schema , 2008, Int. J. Humanoid Robotics.

[8]  Hirokazu Kato,et al.  Marker tracking and HMD calibration for a video-based augmented reality conferencing system , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[9]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[10]  D. Reiss,et al.  Mirror self-recognition in the bottlenose dolphin: A case of cognitive convergence , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[11]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Brian Scassellati,et al.  A robotic model of the Ecological Self , 2011, 2011 11th IEEE-RAS International Conference on Humanoid Robots.

[13]  Gordon G. Gallup,et al.  Which Primates Recognize Themselves in Mirrors? , 2011, PLoS biology.

[14]  S. P. Mudur,et al.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[15]  Ronald C. Arkin,et al.  Robot tool behavior: a developmental approach to autonomous tool use , 2007 .

[16]  Manuel Lopes,et al.  Body schema acquisition through active learning , 2010, 2010 IEEE International Conference on Robotics and Automation.

[17]  Judith M Burkart,et al.  A new mark test for mirror self-recognition in non-human primates , 2006, Primates.

[18]  Brian Scassellati,et al.  Motion-based robotic self-recognition , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[19]  B. Scassellati,et al.  A Bayesian Robot That Distinguishes "Self" from "Other" , 2007 .