A saliency-driven robotic head with bio-inspired saccadic behaviors for social robotics

This paper presents a robotic head for social robots to attend to scene saliency with bio-inspired saccadic behaviors. Scene saliency is determined by measuring low-level static scene information, motion, and object prior knowledge. Towards the extracted saliency spots, the designed robotic head is able to turn gazes in a saccadic manner while obeying eye–head coordination laws with the proposed control scheme. The results of the simulation study and actual applications show the effectiveness of the proposed method in discovering of scene saliency and human-like head motion. The proposed techniques could possibly be applied to social robots to improve social sense and user experience in human–robot interaction.

[1]  T Raphan,et al.  Modeling control of eye orientation in three dimensions. I. Role of muscle pulleys in determining saccadic trajectory. , 1998, Journal of neurophysiology.

[2]  G. Westheimer,et al.  Kinematics of the eye. , 1957, Journal of the Optical Society of America.

[3]  Jean-Michel Morel,et al.  ASIFT: A New Framework for Fully Affine Invariant Image Comparison , 2009, SIAM J. Imaging Sci..

[4]  Thierry Pun,et al.  Integration of bottom-up and top-down cues for visual attention using non-linear relaxation , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[5]  D. Guitton,et al.  Gaze control in humans: eye-head coordination during orienting movements to targets within and beyond the oculomotor range. , 1987, Journal of neurophysiology.

[6]  Antonio Torralba,et al.  Top-down control of visual attention in object detection , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[7]  Ales Ude,et al.  The Karlsruhe Humanoid Head , 2008, Humanoids 2008 - 8th IEEE-RAS International Conference on Humanoid Robots.

[8]  Peyman Milanfar,et al.  Nonparametric bottom-up saliency detection by self-resemblance , 2009, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[9]  Matthew H Tong,et al.  SUN: Top-down saliency using natural statistics , 2009, Visual cognition.

[10]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[11]  W P Medendorp,et al.  Donders' law in torticollis. , 1999, Journal of neurophysiology.

[12]  David W. Murray,et al.  Modeling and copying human head movements , 1999, IEEE Trans. Robotics Autom..

[13]  Laurent Itti,et al.  Models of Bottom-up Attention and Saliency , 2005 .

[14]  Stefano Soatto,et al.  Dynamic Textures , 2003, International Journal of Computer Vision.

[15]  Minoru Asada,et al.  A constructive model for the development of joint attention , 2003, Connect. Sci..

[16]  M. Corbetta,et al.  Control of goal-directed and stimulus-driven attention in the brain , 2002, Nature Reviews Neuroscience.

[17]  Nikolaos G. Tsagarakis,et al.  iCub: the design and realization of an open humanoid platform for cognitive and neuroscience research , 2007, Adv. Robotics.

[18]  Manuel Lopes,et al.  Biomimetic Eye-Neck Coordination , 2009, 2009 IEEE 8th International Conference on Development and Learning.

[19]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[20]  Laurent Itti,et al.  Realistic avatar eye and head animation using a neurobiological model of visual attention , 2004, SPIE Optics + Photonics.

[21]  Alex D. Hwang,et al.  A model of top-down attentional control during visual search in complex scenes. , 2009, Journal of vision.

[22]  Bo Gao,et al.  Pareto-optimal coordination of multiple robots with safety guarantees , 2011, Autonomous Robots.

[23]  Ray Smith.  An Overview of the Tesseract OCR Engine , 2007 .

[24]  Shuzhi Sam Ge,et al.  Attention Determination for Social Robots Using Salient Region Detection , 2010, ICSR.

[25]  Laurent Itti,et al.  An Integrated Model of Top-Down and Bottom-Up Attention for Optimizing Detection Speed , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[26]  Christoph von der Malsburg,et al.  Self Calibration of the Fixation Movement of a Stereo Camera Head , 1998, Auton. Robots.

[27]  Paolo Dario,et al.  A bio-inspired predictive sensory-motor coordination scheme for robot reaching and preshaping , 2008, Auton. Robots.

[28]  Patrick Le Callet,et al.  A coherent computational approach to model bottom-up visual attention , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Sethu Vijayakumar,et al.  Optimal variable stiffness control: formulation and application to explosive movement tasks , 2012, Auton. Robots.

[30]  T Vilis,et al.  Violations of Listing's law after large eye and head gaze shifts. , 1992, Journal of neurophysiology.

[31]  C. Breazeal.  Sociable Machines: Expressive Social Ex-change Between Humans and Robots , 2000 .

[32]  Frédo Durand,et al.  Learning to predict where humans look , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[33]  Shuzhi Sam Ge,et al.  Bottom-up saliency detection for attention determination , 2011, Machine Vision and Applications.

[34]  T Vilis,et al.  Axes of eye rotation and Listing's law during rotations of the head. , 1991, Journal of neurophysiology.

[35]  D. Tweed,et al.  Three-dimensional model of the human eye-head saccadic system. , 1997, Journal of neurophysiology.

[36]  Nuno Vasconcelos,et al.  Bottom-up saliency is a discriminant process , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[37]  Shuzhi Sam Ge,et al.  Visual Attention Prediction Using Saliency Determination of Scene Understanding for Social Robots , 2011, Int. J. Soc. Robotics.

[38]  Alice Caplier,et al.  Lip contour segmentation and tracking compliant with lip-reading application constraints , 2012, Machine Vision and Applications.

[39]  A. Opstal,et al.  Human eye-head coordination in two dimensions under different sensorimotor conditions , 1997, Experimental Brain Research.

[40]  Paolo Dario,et al.  Bioinspired velocity control of fast gaze shifts on a robotic anthropomorphic head , 2008, Auton. Robots.

[41]  J. Crawford,et al.  Neural control of three-dimensional eye and head movements , 2003, Current Opinion in Neurobiology.

[42]  Garrison W. Cottrell,et al.  Visual saliency model for robot cameras , 2008, 2008 IEEE International Conference on Robotics and Automation.