Eye Gaze Based 3D Triangulation for Robotic Bionic Eyes

Three-dimensional (3D) triangulation based on active binocular vision has increasing amounts of applications in computer vision and robotics. An active binocular vision system with non-fixed cameras needs to calibrate the stereo extrinsic parameters online to perform 3D triangulation. However, the accuracy of stereo extrinsic parameters and disparity have a significant impact on 3D triangulation precision. We propose a novel eye gaze based 3D triangulation method that does not use stereo extrinsic parameters directly in order to reduce the impact. Instead, we drive both cameras to gaze at a 3D spatial point P at the optical center through visual servoing. Subsequently, we can obtain the 3D coordinates of P through the intersection of the two optical axes of both cameras. We have performed experiments to compare with previous disparity based work, named the integrated two-pose calibration (ITPC) method, using our robotic bionic eyes. The experiments show that our method achieves comparable results with ITPC.

[1]  Mansour Alsulaiman,et al.  Stereo vision SLAM based indoor autonomous mobile robot navigation , 2014, 2014 IEEE International Conference on Robotics and Biomimetics (ROBIO 2014).

[2]  Kikuo Fujimura,et al.  The intelligent ASIMO: system overview and integration , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[3]  Jean-Yves Bouguet,et al.  Camera calibration toolbox for matlab , 2001 .

[4]  Ye Tian,et al.  Reliable eyes pose measurement for robotic bionic eyes with MEMS gyroscope and AKF filter , 2017, 2017 IEEE International Conference on Cyborg and Bionic Systems (CBS).

[5]  K. Shadan,et al.  Available online: , 2012 .

[6]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  David W. Murray,et al.  Hardware development of the Yorick series of active vision systems , 1998, Microprocess. Microsystems.

[8]  Jingjun Zhang,et al.  Passive 3D reconstruction based on binocular vision , 2019, International Conference on Graphic and Image Processing.

[9]  Tamim Asfour,et al.  ARMAR-4: A 63 DOF torque controlled humanoid robot , 2013, 2013 13th IEEE-RAS International Conference on Humanoid Robots (Humanoids).

[10]  Samia Nefti-Meziani,et al.  3D perception from binocular vision for a low cost humanoid robot NAO , 2015, Robotics Auton. Syst..

[11]  Alfredo Paolillo,et al.  Covariance Propagation for the Uncertainty Estimation in Stereo Vision , 2011, IEEE Transactions on Instrumentation and Measurement.

[12]  Xu Chen,et al.  Multi-loop stabilization control of a robotic bionic eyes , 2017, 2017 IEEE International Conference on Cyborg and Bionic Systems (CBS).

[13]  Paul G. Maropoulos,et al.  Estimation of uncertainty in three-dimensional coordinate measurement by comparison with calibrated points , 2010 .

[14]  Sebastian Thrun,et al.  Simultaneous localization and mapping with active stereo vision , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[15]  Jharna Majumdar,et al.  Efficient parallel processing for depth calculation using stereo , 1997, Robotics Auton. Syst..

[16]  Fuqiang Zhou,et al.  Precise calibration of binocular vision system used for vision measurement. , 2014, Optics express.

[17]  Éric Marchand,et al.  ViSP for visual servoing: a generic software platform with a wide class of robot control skills , 2005, IEEE Robotics & Automation Magazine.

[18]  Andrew Howard,et al.  Design and use paradigms for Gazebo, an open-source multi-robot simulator , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[19]  Jörn Malzahn,et al.  WALK‐MAN: A High‐Performance Humanoid Platform for Realistic Environments , 2017, J. Field Robotics.

[20]  I Kiss,et al.  Social humanoid robot SARA: development of the wrist mechanism , 2018 .

[21]  Ingo Lütkebohle,et al.  The bielefeld anthropomorphic robot head “Flobi” , 2010, 2010 IEEE International Conference on Robotics and Automation.

[22]  Yue Wang,et al.  A Method for Extrinsic Parameter Calibration of Rotating Binocular Stereo Vision Using a Single Feature Point , 2018, Sensors.

[23]  Seth Hutchinson,et al.  Visual Servo Control Part I: Basic Approaches , 2006 .

[24]  Giorgio Cannata,et al.  MAC-EYE: a tendon driven fully embedded robot eye , 2005, 5th IEEE-RAS International Conference on Humanoid Robots, 2005..

[25]  Takayuki Okatani,et al.  Binocular motion tracking by gaze fixation control and three-dimensional shape reconstruction , 2003, Adv. Robotics.

[26]  Fumio Miyazaki,et al.  Binocular gaze holding of a moving object with the active stereo vision system , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[27]  Edwin Olson,et al.  AprilTag: A robust and flexible visual fiducial system , 2011, 2011 IEEE International Conference on Robotics and Automation.

[28]  Yang Song,et al.  An active binocular integrated system for intelligent robot vision , 2012, 2012 IEEE International Conference on Intelligence and Security Informatics.

[29]  De Xu,et al.  A new vision measurement method based on active object gazing , 2017 .

[30]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[31]  Alexandru Tupan,et al.  Triangulation , 1997, Comput. Vis. Image Underst..

[32]  Giorgio Metta,et al.  Design of the robot-cub (iCub) head , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[33]  Kenji KANEKO,et al.  Humanoid robot HRP-3 , 2004, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[34]  Alain Berthoz,et al.  Head-eyes system and gaze analysis of the humanoid robot Romeo , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[35]  Carla Schwengber ten Caten,et al.  Measurement Uncertainty: Literature Review and Research Trends , 2012, IEEE Transactions on Instrumentation and Measurement.

[36]  Qiang Huang,et al.  An Integrated Two-Pose Calibration Method for Estimating Head-Eye Parameters of a Robotic Bionic Eye , 2020, IEEE Transactions on Instrumentation and Measurement.

[37]  Richard P. Paul,et al.  Robot manipulators : mathematics, programming, and control : the computer control of robot manipulators , 1981 .

[38]  S Sasikumar,et al.  Implementation of a Stereo vision based system for visual feedback control of Robotic Arm for space manipulations , 2018 .

[39]  Kenichi Kanatani,et al.  Triangulation from Two Views Revisited: Hartley-Sturm vs. Optimal Correction , 2008, BMVC.

[40]  M. Vanrell,et al.  Gaze control in a binocular robot systems , 1999, 1999 7th IEEE International Conference on Emerging Technologies and Factory Automation. Proceedings ETFA '99 (Cat. No.99TH8467).

[41]  Xinxing Shao,et al.  A comparative study of 3D reconstruction methods in stereo digital image correlation , 2019 .

[42]  Yasuo Kuniyoshi,et al.  Learning of oculo-motor control: a prelude to robotic imitation , 1996, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems. IROS '96.

[43]  Ales Ude,et al.  The Karlsruhe Humanoid Head , 2008, Humanoids 2008 - 8th IEEE-RAS International Conference on Humanoid Robots.

[44]  C. Liguori,et al.  Propagation of uncertainty through stereo triangulation , 2010, 2010 IEEE Instrumentation & Measurement Technology Conference Proceedings.

[45]  Christian Hoffmann,et al.  Continuous Stereo Self-Calibration by Camera Parameter Tracking , 2009, IEEE Transactions on Image Processing.

[46]  Mengxiang Li,et al.  Kinematic calibration of an active head-eye system , 1998, IEEE Trans. Robotics Autom..

[47]  Pablo Bustos,et al.  Muecas: A Multi-Sensor Robotic Head for Affective Human Robot Interaction and Imitation , 2014, Sensors.

[48]  Yoshihiko Nakamura,et al.  Dense RGB-D SLAM for Humanoid Robots in the Dynamic Humans Environment , 2018, 2018 IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids).

[49]  Yi-Ping Hung,et al.  A New Approach to Automatic Reconstruction of a 3-D World Using Active Stereo Vision , 2002, Comput. Vis. Image Underst..

[50]  Nicola J. Ferrier,et al.  Robust active stereo calibration , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[51]  Xu Chen,et al.  Design of anthropomorphic robot bionic eyes , 2017, 2017 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[52]  De Xu,et al.  A New Active Visual System for Humanoid Robots , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[53]  C.M. Brown,et al.  Cooperative gaze holding in binocular vision , 1991, IEEE Control Systems.

[54]  Yue Wang,et al.  An improved two-point calibration method for stereo vision with rotating cameras in large FOV , 2019 .