Target Tracking of Moving and Rotating Object by High-Speed Monocular Active Vision

In recent years, the importance of measuring the three-dimensional position and orientation of manipulation targets in robot systems has been increasing. There is great demand to integrate robots with high-frame-rate vision systems to improve the working efficiency. A number of high-speed vision systems have been developed for high-frame-rate visual feedback. In this study, we focus on high-speed target tracking of both 3D position and orientation using only a monocular camera. This is a lightweight, low-cost solution for many fields, such as surgical navigation and drone avoidance. We modified the pixel-wise posterior 3D (PWP3D) framework and proposed a fast-PWP3D algorithm for high-speed target tracking and pose estimation. Unlike the original PWP3D method, our method improved the tracking speed (our current implementation runs at 400 Hz on a GPU board), and we showed that the tracking accuracy against changes in the environment (e.g., partial occlusion) was improved compared with the original PWP3D algorithm. By combining the fast-PWP3D algorithm with a visual servoing controller, we realized 500 Hz target tracking of both 3D position and orientation.

[1]  Akio Namiki,et al.  Development of an Active High-Speed 3-D Vision System † , 2019, Sensors.

[2]  Takeshi Takaki,et al.  2000 fps real-time vision system with high-frame-rate video recording , 2010, 2010 IEEE International Conference on Robotics and Automation.

[3]  Henrik I. Christensen,et al.  RGB-D object tracking: A particle filter approach on GPU , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[4]  Nassir Navab,et al.  Deep Learning of Local RGB-D Patches for 3D Object Detection and 6D Pose Estimation , 2016, ECCV.

[5]  Akio Namiki,et al.  High-speed Target Tracking of 3D Object by Monocular camera , 2020, 2020 IEEE/SICE International Symposium on System Integration (SII).

[6]  Nassir Navab,et al.  Real-Time 3D Model Tracking in Color and Depth on a Single CPU Core , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Hans-Peter Seidel,et al.  A Comparison of Shape Matching Methods for Contour Based Pose Estimation , 2006, IWCIA.

[8]  Masatoshi Ishikawa,et al.  1 ms Auto Pan-Tilt – video shooting technology for objects in motion based on Saccade Mirror with background subtraction , 2015, Adv. Robotics.

[9]  Akio Namiki,et al.  Target Tracking of Moving and Rotating Object by High-Speed Monocular Active Vision , 2020, IEEE Sensors Journal.

[10]  Olaf Kähler,et al.  3D Tracking of Multiple Objects with Identical Appearance Using RGB-D Input , 2014, 2014 2nd International Conference on 3D Vision.

[11]  Masatoshi Ishikawa,et al.  3D tracking using two high-speed vision systems , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[12]  Masatoshi Ishikawa,et al.  Ultra high-speed Robot Based on 1 kHz vision system , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[13]  Bodo Rosenhahn,et al.  Three-Dimensional Shape Knowledge for Joint Image Segmentation and Pose Tracking , 2007, International Journal of Computer Vision.

[14]  Rami R. Hagege,et al.  2D-3D Pose Estimation of Heterogeneous Objects Using a Region Based Approach , 2015, International Journal of Computer Vision.

[15]  Masatoshi Ishikawa,et al.  High-speed 3D sensing with three-view geometry using a segmented pattern , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[16]  David Joseph Tan,et al.  Multi-forest Tracker: A Chameleon in Tracking , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Peter I. Corke,et al.  A tutorial on visual servo control , 1996, IEEE Trans. Robotics Autom..

[18]  Yoshihiro Watanabe,et al.  High-speed optical 3D sensing and its applications , 2016 .

[19]  Bodo Rosenhahn,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence Combined Region-and Motion-based 3d Tracking of Rigid and Articulated Objects , 2022 .

[20]  Masatoshi Ishikawa,et al.  955-fps Real-time Shape Measurement of a Moving/Deforming Object using High-speed Vision for Numerous-point Analysis , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[21]  Ulrich Schwanecke,et al.  Real-Time Monocular Segmentation and Pose Tracking of Multiple Objects , 2016, ECCV.

[22]  Olaf Kähler,et al.  Real-Time 3D Tracking and Reconstruction on Mobile Phones , 2015, IEEE Transactions on Visualization and Computer Graphics.

[23]  Nassir Navab,et al.  A Versatile Learning-Based 3D Temporal Tracker: Scalable, Robust, Online , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[24]  Ian D. Reid,et al.  PWP3D: Real-time Segmentation and Tracking of 3D Objects , 2009, BMVC.

[25]  Bodo Rosenhahn,et al.  Region-Based Pose Tracking , 2007, IbPRIA.

[26]  Ulrich Schwanecke,et al.  Real-Time Monocular Pose Estimation of 3D Objects Using Temporally Consistent Local Color Histograms , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[27]  Koichi Hashimoto,et al.  A review on vision-based control of robot manipulators , 2003, Adv. Robotics.

[28]  Masatoshi Ishikawa,et al.  High-speed batting using a multi-jointed manipulator , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[29]  Vincent Lepetit,et al.  Monocular Model-Based 3D Tracking of Rigid Objects: A Survey , 2005, Found. Trends Comput. Graph. Vis..

[30]  Masatoshi Ishikawa,et al.  High-speed throwing motion based on kinetic chain approach , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[31]  Tokuo Tsuji,et al.  High-speed 3D image acquisition using coded structured light projection , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[32]  Daniel P. Huttenlocher,et al.  Distance Transforms of Sampled Functions , 2012, Theory Comput..