Towards fully automated robotic platform for remote auscultation

Since most developed countries are facing an increase in the number of patients per healthcare worker due to a declining birth rate and an aging population, relatively simple and safe diagnosis tasks may need to be performed using robotics and automation technologies, without specialists and hospitals. This study presents an automated robotic platform for remote auscultation, which is a highly cost-effective screening tool for detecting abnormal clinical signs. The developed robotic platform is composed of a 6-degree-of-freedom cooperative robotic arm, light detection and ranging (LiDAR) camera, and a spring-based mechanism holding an electric stethoscope. The platform enables autonomous stethoscope positioning based on external body information acquired using the LiDAR camerabased multi-way registration; the platform also ensures safe and flexible contact, maintaining the contact force within a certain range through the passive mechanism. Our preliminary results confirm that the robotic platform enables estimation of the landing positions required for cardiac examinations based on the depth and landmark information of the body surface. It also handles the stethoscope while maintaining the contact force without relying on the push-in displacement by the robotic arm.

[1]  K. Tatara,et al.  Japan. Health system review. , 2009 .

[2]  Christophe Rosenberger,et al.  A tele-operated mobile ultrasound scanner using a light-weight robot , 2005, IEEE Transactions on Information Technology in Biomedicine.

[3]  Matthew W. Gilbertson,et al.  Force and Position Control System for Freehand Ultrasound , 2015, IEEE Transactions on Robotics.

[4]  Michael J. Black,et al.  On the unification of line processes, outlier rejection, and robust statistics with applications in early vision , 1996, International Journal of Computer Vision.

[5]  Fang Zhu,et al.  Artificial intelligence-assisted auscultation in detecting congenital heart disease , 2021, European heart journal. Digital health.

[6]  M. S. Swapna,et al.  Nonlinear time series and principal component analyses: Potential diagnostic tools for COVID-19 auscultation , 2020, Chaos, Solitons & Fractals.

[7]  R. Ye,et al.  Feasibility of a 5G-Based Robot-Assisted Remote Ultrasound System for Cardiopulmonary Assessment of Patients With COVID-19 , 2020 .

[8]  Vladlen Koltun,et al.  Fast Global Registration , 2016, ECCV.

[9]  John L Semmlow,et al.  Utility of an advanced digital electronic stethoscope in the diagnosis of coronary artery disease compared with coronary computed tomographic angiography. , 2013, The American journal of cardiology.

[10]  G. Darling,et al.  Surface anatomy and surface landmarks for thoracic surgery. , 2007, Thoracic surgery clinics.

[11]  Kyrre Glette,et al.  An Ultrasound Robotic System Using the Commercial Robot UR5 , 2016, Front. Robot. AI.

[12]  A. Schweikard,et al.  Towards automated ultrasound imaging—robotic image acquisition in liver and prostate for long-term motion monitoring , 2021, Physics in medicine and biology.

[13]  Carlo Alberto Avizzano,et al.  Teleoperated multimodal robotic interface for telemedicine: A case study on remote auscultation , 2017, 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[14]  Hiroyasu Iwata,et al.  Development of ultrasonography assistance robot for prenatal care , 2020, Medical Imaging: Image-Guided Procedures.

[15]  F. Bruyère,et al.  Follow-up of adults with noncritical COVID-19 two months after symptom onset , 2020, Clinical Microbiology and Infection.

[16]  Ryosuke Tsumura,et al.  Robotic fetal ultrasonography platform with a passive scan mechanism , 2020, International Journal of Computer Assisted Radiology and Surgery.

[17]  G. Poisson,et al.  Fetal tele‐echography using a robotic arm and a satellite link , 2005, Ultrasound in obstetrics & gynecology : the official journal of the International Society of Ultrasound in Obstetrics and Gynecology.

[18]  Russell H. Taylor,et al.  Force-assisted ultrasound imaging system through dual force sensing and admittance robot control , 2017, International Journal of Computer Assisted Radiology and Surgery.

[19]  A. Capri,et al.  A mechatronic tele-operated system for echography using visual navigation assistance and a model based bilateral predictive control , 2007, 2007 IEEE International Symposium on Industrial Electronics.

[20]  Gretchen A. Stevens,et al.  NCD Countdown 2030: worldwide trends in non-communicable disease mortality and progress towards Sustainable Development Goal target 3.4 , 2018, The Lancet.

[21]  X. Bao,et al.  A Constant-Force End-Effector With Online Force Adjustment for Robotic Ultrasonography , 2021, IEEE Robotics and Automation Letters.

[22]  P. Gomes,et al.  Digital Auscultation: Challenges and Perspectives , 2016 .

[23]  Thomas Neff,et al.  Towards MRI-Based Autonomous Robotic US Acquisitions: A First Feasibility Study , 2016, IEEE Transactions on Medical Imaging.

[24]  Nassir Navab,et al.  On the reproducibility of expert-operated and robotic ultrasound acquisitions , 2017, International Journal of Computer Assisted Radiology and Surgery.

[25]  M. W. Chu,et al.  Predictors of internal mammary vessel diameter: A computed tomographic angiography-assisted anatomic analysis. , 2016, Journal of plastic, reconstructive & aesthetic surgery : JPRAS.

[26]  Hengli Liu,et al.  Towards Fully Autonomous Ultrasound Scanning Robot With Imitation Learning Based on Clinical Protocols , 2021, IEEE Robotics and Automation Letters.

[27]  J. Bruce Utility of the sternum to estimate sex and age , 2014 .

[28]  Beatrice Hoffmann,et al.  Tele-Operative Low-Cost Robotic Lung Ultrasound Scanning Platform for Triage of COVID-19 Patients , 2020, IEEE Robotics and Automation Letters.

[29]  Alexandre Krupa,et al.  Robotized Tele-Echography: An Assisting Visibility Tool to Support Expert Diagnostic , 2016, IEEE Systems Journal.

[30]  A. Reinisch,et al.  Artificial Intelligence-Assisted Auscultation of Heart Murmurs: Validation by Virtual Clinical Trial , 2018, Pediatric Cardiology.

[31]  Hiroyuki Ishii,et al.  Development of robotic system for autonomous liver screening using ultrasound scanning device , 2013, 2013 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[32]  Mamoru Mitsuishi,et al.  Construction Methodology for a Remote Ultrasound Diagnostic System , 2009, IEEE Transactions on Robotics.

[33]  M. Infanger,et al.  Configuration and Localization of the Nipple‐Areola Complex in Men , 2001, Plastic and reconstructive surgery.

[34]  Jędrzej Kociński,et al.  Practical implementation of artificial intelligence algorithms in pulmonary auscultation examination , 2019, European Journal of Pediatrics.

[35]  Kevin Cleary,et al.  Robotic Arm-Assisted Sonography: Review of Technical Developments and Potential Clinical Applications. , 2017, AJR. American journal of roentgenology.

[36]  Xuelong Li,et al.  Robotic Arm Based Automatic Ultrasound Scanning for Three-Dimensional Imaging , 2019, IEEE Transactions on Industrial Informatics.

[37]  SiegwartRoland,et al.  Comparing ICP variants on real-world data sets , 2013 .

[38]  Jakub Tomasz Kaminski,et al.  Feasibility of robot-assisted ultrasound imaging with force feedback for assessment of thyroid diseases , 2020, Medical Imaging: Image-Guided Procedures.

[39]  Thomas Neff,et al.  Automatic force-compliant robotic ultrasound screening of abdominal aortic aneurysms , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[40]  A. M. Priester,et al.  Robotic ultrasound systems in medicine , 2013, IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control.