Visual-model-based, real-time 3D pose tracking for autonomous navigation: methodology and experiments

This paper presents a novel 3D-model-based computer-vision method for tracking the full six degree-of-freedom (dof) pose (position and orientation) of a rigid body, in real-time. The methodology has been targeted for autonomous navigation tasks, such as interception of or rendezvous with mobile targets. Tracking an object’s complete six-dof pose makes the proposed algorithm useful even when targets are not restricted to planar motion (e.g., flying or rough-terrain navigation). Tracking is achieved via a combination of textured model projection and optical flow. The main contribution of our work is the novel combination of optical flow with z-buffer depth information that is produced during model projection. This allows us to achieve six-dof tracking with a single camera.A localized illumination normalization filter also has been developed in order to improve robustness to shading. Real-time operation is achieved using GPU-based filters and a new data-reduction algorithm based on colour-gradient redundancy, which was developed within the framework of our project. Colour-gradient redundancy is an important property of colour images, namely, that the gradients of all colour channels are generally aligned. Exploiting this property provides a threefold increase in speed. A processing rate of approximately 80 to 100 fps has been obtained in our work when utilizing synthetic and real target-motion sequences. Sub-pixel accuracies were obtained in tests performed under different lighting conditions.

[1]  Peter I. Corke,et al.  Dynamic effects in visual closed-loop systems , 1996, IEEE Trans. Robotics Autom..

[2]  Gourab Sen Gupta,et al.  Real-time identification and predictive control of fast mobile robots using global vision sensing , 2005, IEEE Transactions on Instrumentation and Measurement.

[3]  Jens-Steffen Gutmann,et al.  Model-based object tracking using stereo vision , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[4]  Gang Hua,et al.  Switching observation models for contour tracking in clutter , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[5]  Rajeev Sharma,et al.  Appearance management and cue fusion for 3D model-based tracking , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[6]  Georg Hartmann,et al.  Single view recognition and pose estimation of 3D objects using sets of prototypical views and spatially tolerant contour representations , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[7]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[8]  Takahiro Ishikawa,et al.  The template update problem , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Brian C. Lovell,et al.  Color Optical Flow , 2003 .

[10]  Ramakant Nevatia,et al.  Automatic pose estimation of complex 3D building models , 2002, Sixth IEEE Workshop on Applications of Computer Vision, 2002. (WACV 2002). Proceedings..

[11]  Reinhard Klette,et al.  Quantitative color optical flow , 2002, Object recognition supported by user interaction for service robots.

[12]  David J. Fleet,et al.  Robust Online Appearance Models for Visual Tracking , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Stephen J. McKenna,et al.  Tracking interacting people , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[14]  Yang Fan,et al.  Target tracking by underwater robots , 2001, 2001 IEEE International Conference on Systems, Man and Cybernetics. e-Systems and e-Man for Cybernetics in Cyberspace (Cat.No.01CH37236).

[15]  Mei Han,et al.  An algorithm for multiple object trajectory tracking , 2004, CVPR 2004.

[16]  J. Serrat,et al.  Multiple vehicle 3D tracking using an unscented Kalman , 2005, Proceedings. 2005 IEEE Intelligent Transportation Systems, 2005..

[17]  Markus Vincze,et al.  Vision for Robotics: a tool for model-based object tracking , 2005, IEEE Robotics & Automation Magazine.

[18]  Jin-Long Chen,et al.  Determining Pose of 3D Objects With Curved Surfaces , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Hanqing Lu,et al.  Generalized optical flow in the scale space , 2004, Third International Conference on Image and Graphics (ICIG'04).

[20]  Thomas Sugar,et al.  Mobile robot interception using human navigational principles: Comparison of active versus passive tracking algorithms , 2006, Auton. Robots.

[21]  Tieniu Tan,et al.  Model-Based Localisation and Recognition of Road Vehicles , 1998, International Journal of Computer Vision.

[22]  Stefano Soatto,et al.  Real-time 3D motion and structure of point features: a front-end system for vision-based control and interaction , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[23]  Edward Tunstel,et al.  Planetary Rover Developments Supporting Mars Exploration, Sample Return and Future Human-Robotic Colonization , 2003, Auton. Robots.

[24]  Katsushi Ikeuchi,et al.  Illumination normalization with time-dependent intrinsic images for video surveillance , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[25]  Robin R. Murphy,et al.  Cooperative Navigation of Micro-Rovers Using Color Segmentation , 2000, Auton. Robots.

[26]  Sing Bing Kang,et al.  Parameter-Free Radial Distortion Correction with Center of Distortion Estimation , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Anil K. Jain,et al.  Interacting multiple model (IMM) Kalman filters for robust high speed human motion tracking , 2002, Object recognition supported by user interaction for service robots.

[28]  Mei Han,et al.  An algorithm for multiple object trajectory tracking , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[29]  Beno Benhabib,et al.  The robotic interception of moving objects in industrial settings: strategy development and experiment , 1998 .

[30]  Ville Kyrki,et al.  Integation Methods of Model-Free Features for 3D Tracking , 2005, SCIA.

[31]  Patrick Bouthemy,et al.  A 2D-3D model-based approach to real-time visual tracking , 2001, Image Vis. Comput..

[32]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[33]  Vincent Lepetit,et al.  Point matching as a classification problem for fast and robust object pose estimation , 2004, CVPR 2004.

[34]  Ruigang Yang,et al.  Real‐Time Consensus‐Based Scene Reconstruction Using Commodity Graphics Hardware † , 2003, Comput. Graph. Forum.

[35]  Jitendra Malik,et al.  Robust computation of optical flow in a multi-scale differential framework , 1993, 1993 (4th) International Conference on Computer Vision.

[36]  Björn Johansson,et al.  Patch-duplets for object recognition and pose estimation , 2005, The 2nd Canadian Conference on Computer and Robot Vision (CRV'05).

[37]  Volker Graefe,et al.  Dynamic monocular machine vision , 1988, Machine Vision and Applications.

[38]  Gregory D. Hager,et al.  Fast and Globally Convergent Pose Estimation from Video Images , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  Jitendra Malik,et al.  Robust computation of optical flow in a multi-scale differential framework , 2005, International Journal of Computer Vision.

[40]  Peter I. Corke,et al.  A tutorial on visual servo control , 1996, IEEE Trans. Robotics Autom..

[41]  Éric Marchand,et al.  A real-time tracker for markerless augmented reality , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[42]  Dave Shreiner,et al.  OpenGL(R) 1.4 Reference Manual (4th Edition) , 2004 .

[43]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[44]  Roberto Cipolla,et al.  Real-Time Visual Tracking of Complex Structures , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[45]  Michel Dhome,et al.  Real Time Robust Template Matching , 2002, BMVC.

[46]  A. Balasuriya,et al.  Vision based autonomous vehicles target visual tracking with multiple dynamics models , 2005, Proceedings. 2005 IEEE Networking, Sensing and Control, 2005..

[47]  Takeo Kanade,et al.  A System for Video Surveillance and Monitoring , 2000 .

[48]  Li Jinzong,et al.  Robust computation of optical flow field with large motion , 2004, Proceedings 7th International Conference on Signal Processing, 2004. Proceedings. ICSP '04. 2004..

[49]  Ernst D. Dickmanns,et al.  Visual grasping with long delay time of a free floating object in orbit , 1994, Auton. Robots.

[50]  Gregory D. Hager,et al.  Real-time tracking of image regions with changes in geometry and illumination , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[51]  Ling-Zhi Liao,et al.  3D object recognition and pose estimation using kernel PCA , 2004, Proceedings of 2004 International Conference on Machine Learning and Cybernetics (IEEE Cat. No.04EX826).

[52]  In-So Kweon,et al.  Robust model-based 3D object recognition by combining feature matching with tracking , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[53]  Vincenzo Lippiello,et al.  Robust visual tracking using a fixed multi-camera system , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[54]  Danica Kragic,et al.  Object recognition and pose estimation for robotic manipulation using color cooccurrence histograms , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).