Three Dimensional Microrobot Tracking Using Learning-based System

As a promising method for robotic catheter therapeutics, a controllable wireless microrobot system with a less invasive intraoperative procedure has been presented for thrombosis surgery. The controllable wireless microrobot utilizes an electromagnetic actuator system for microrobot actuation and biplane X-ray system for microrobot imaging. We propose the new 3D tracking method of microrobot for this system. As a learning-based system, cascade classifier was adapted for real-time microrobot tracking. Combination of cascade classifier and contour-based system made accurate microrobot detection method on 2D projected X-ray images. Kalman filter interpolated lost frames, and triangulation method reconstructed 3D pose (position and orientation) from 4 endpoints of microrobot pairs. Also, the semi-automatic calibration method for bi-plane C-arm devices was proposed by using modified shortest path algorithm. To find the optimal rescaling setting, 3D tracking accuracy was investigated according to the rescaling of original intensity values. The experiment results showed a good performance with tracking errors of 2.37±9.91mm in position and 6.53±13.80° in orientation under 2200 optimal width of rescaling. If tracking evaluation is constrained under frames detected by cascade classifier, 3D tracking errors improved sincerely by 0.28±1.13mm in position and 3.48±2.89° in orientation. The optimal width of the rescaling setting needed bigger value than the mean of intensity values. The proposed tracking technique accomplished a fast frame rate of 34.72 frames/sec under OpenCL implementation of OpenCV. Learning based system is robust to the change of graphics setting since training can adapt rapidly to this change. Also, proposed framework can cooperate with the change of microrobot shape if a contour-based method is optimized. Hence, the method can be used for therapeutic millimeter- or micron-sized manipulator recognition in vascular, as well as implanted objects in the human body.

[1]  Ioannis K. Kaliakatsos,et al.  Microrobots for minimally invasive medicine. , 2010, Annual review of biomedical engineering.

[2]  Deukhee Lee,et al.  Automatic spine segmentation from CT images using Convolutional Neural Network via redundant generation of class labels , 2019, J. Comput. Des. Eng..

[3]  Alberto Corbi,et al.  Evaluation of modern camera calibration techniques for conventional diagnostic X-ray imaging settings , 2016, Radiological Physics and Technology.

[4]  Rainer Lienhart,et al.  An extended set of Haar-like features for rapid object detection , 2002, Proceedings. International Conference on Image Processing.

[5]  Seong Young Ko,et al.  Real-time microrobot posture recognition via biplane X-ray imaging system for external electromagnetic actuation , 2018, International Journal of Computer Assisted Radiology and Surgery.

[6]  Mohan Manubhai Trivedi,et al.  Active learning based robust monocular vehicle detection for on-road safety systems , 2009, 2009 IEEE Intelligent Vehicles Symposium.

[7]  Le Song,et al.  Real-time 3D Finger Pointing for an Augmented Desk , 2005, AUIC.

[8]  배장표.  Advanced segmentation algorithms using numerical model and graph theory-Application to the lung images- , 2016 .

[9]  A. K. Hoshiar,et al.  A Magnetically Controlled Soft Microrobot Steering a Guidewire in a Three-Dimensional Phantom Vascular Network , 2019, Soft robotics.

[10]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Robert E. Schapire,et al.  The Boosting Approach to Machine Learning An Overview , 2003 .

[12]  Sizhong Chen,et al.  Multi-vehicle detection with identity awareness using cascade Adaboost and Adaptive Kalman filter for driver assistant system , 2017, PloS one.

[13]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[14]  Chad J Zack,et al.  Comparative outcomes of catheter-directed thrombolysis plus anticoagulation vs anticoagulation alone to treat lower-extremity proximal deep vein thrombosis. , 2014, JAMA internal medicine.

[15]  Robert F. Riley,et al.  Update on the Management of Chronic Total Occlusions in Coronary Artery Disease , 2017, Current Atherosclerosis Reports.

[16]  Dorin Comaniciu,et al.  Spine detection in CT and MR using iterated marginal space learning , 2013, Medical Image Anal..

[17]  Greg Welch,et al.  An Introduction to Kalman Filter , 1995, SIGGRAPH 2001.

[18]  Jianliang Tang,et al.  Complete Solution Classification for the Perspective-Three-Point Problem , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  A. Baggish,et al.  Strenuous Exercise and Cardiovascular Disease Outcomes , 2017, Current Atherosclerosis Reports.

[20]  Franziska Ullrich,et al.  Mobility experiments with microrobots for minimally invasive intraocular surgery. , 2013, Investigative ophthalmology & visual science.

[21]  Anupam Agrawal,et al.  Vision based hand gesture recognition for human computer interaction: a survey , 2012, Artificial Intelligence Review.

[22]  M. Fornage,et al.  Heart Disease and Stroke Statistics—2017 Update: A Report From the American Heart Association , 2017, Circulation.