Multi-sensor fusion for body sensor network in medical human-robot interaction scenario

Abstract With the development of sensor and communication technologies, body sensor networks(BSNs) have become an indispensable part of smart medical services by monitoring the real-time state of users. Due to introducing of smart medical robots, BSNs are not related to users, but also responsible for data acquisition and multi-sensor fusion in medical human–robot interaction scenarios. In this paper, a hybrid body sensor network architecture based on multi-sensor fusion(HBMF) is designed to support the most advanced smart medical services, which combines various sensor, communication, robot, and data processing technologies. The infrastructure and system functions are described in detail and compared with other architectures. Especially, A multi-sensor fusion method based on interpretable neural network(MFIN) for BSNs in medical human–robot interaction scenario is designed and analyzed to improve the performance of fusion decision-making. Compared with the current multi-sensor fusion methods, our design guarantees both the flexibility and reliability of the service in the medical human–robot interaction scenario.

[1]  Hassan Ghasemzadeh,et al.  Multi-sensor fusion in body sensor networks: State-of-the-art and research challenges , 2017, Inf. Fusion.

[2]  Farzaneh Abdollahi,et al.  A Decentralized Cooperative Control Scheme With Obstacle Avoidance for a Team of Mobile Robots , 2014, IEEE Transactions on Industrial Electronics.

[3]  Cristian Sminchisescu,et al.  3D Human Sensing, Action and Emotion Recognition in Robot Assisted Therapy of Children with Autism , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[4]  Alex Pentland,et al.  Multi-sensor data fusion using the influence model , 2006, International Workshop on Wearable and Implantable Body Sensor Networks (BSN'06).

[5]  Hailiang Liu,et al.  The human body characteristic parameters extraction and disease tendency prediction based on multi-sensing fusion algorithms , 2016, 2016 IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems (CYBER).

[6]  M. Shamim Hossain,et al.  Green Video Transmission in the Mobile Cloud Networks , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[7]  Trung Dung Ngo,et al.  Toward Socially Aware Robot Navigation in Dynamic and Crowded Environments: A Proactive Social Motion Model , 2017, IEEE Transactions on Automation Science and Engineering.

[8]  Abdallah Makhoul,et al.  Self-Adaptive Data Collection and Fusion for Health Monitoring Based on Body Sensor Networks , 2016, IEEE Transactions on Industrial Informatics.

[9]  Ling Guan,et al.  3D Human Action Recognition Using a Single Depth Feature and Locality-Constrained Affine Subspace Coding , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  Min Chen,et al.  Enhanced Fingerprinting and Trajectory Prediction for IoT Localization in Smart Buildings , 2016, IEEE Transactions on Automation Science and Engineering.

[11]  Giancarlo Fortino,et al.  An Edge-Based Architecture to Support Efficient Applications for Healthcare Industry 4.0 , 2019, IEEE Transactions on Industrial Informatics.

[12]  Huimin Lu,et al.  PEA: Parallel electrocardiogram-based authentication for smart healthcare systems , 2018, J. Netw. Comput. Appl..

[13]  Jianye Liu,et al.  An Improved Multi-Sensor Fusion Navigation Algorithm Based on the Factor Graph , 2017, Sensors.

[14]  Wei Wang,et al.  Multi-sensor Image Fusion Algorithm Based on Multiresolution Analysis , 2018, Int. J. Online Eng..

[15]  Ashwin P. Dani,et al.  Human Intention Inference Using Expectation-Maximization Algorithm With Online Model Learning , 2017, IEEE Transactions on Automation Science and Engineering.

[16]  Wenyao Xu,et al.  A Configurable Energy-Efficient Compressed Sensing Architecture With Its Application on Body Sensor Networks , 2016, IEEE Transactions on Industrial Informatics.

[17]  M. Shamim Hossain,et al.  Artificial-Intelligence-Based Data Analytics for Cognitive Communication in Heterogeneous Wireless Networks , 2019, IEEE Wireless Communications.

[18]  Geoffrey E. Hinton,et al.  Dynamic Routing Between Capsules , 2017, NIPS.

[19]  Huasong Min,et al.  A Human–Robot-Environment Interactive Reasoning Mechanism for Object Sorting Robot , 2018, IEEE Transactions on Cognitive and Developmental Systems.

[20]  Karl J. Friston,et al.  Action perception as hypothesis testing , 2017, Cortex.

[21]  Francisco Herrera,et al.  Big Data: Tutorial and guidelines on information and process fusion for analytics algorithms with MapReduce , 2018, Inf. Fusion.

[22]  Xi Wang,et al.  Modeling Spatial-Temporal Clues in a Hybrid Deep Learning Framework for Video Classification , 2015, ACM Multimedia.

[23]  Jie Chen,et al.  Decision-Making Algorithm for Multisensor Fusion Based on Grey Relation and DS Evidence Theory , 2016, J. Sensors.

[24]  Abdul Ghafoor,et al.  Regional Image Fusion with Genetic Algorithm Optimization , 2015, 2015 9th Asia Modelling Symposium (AMS).

[25]  Min Chen,et al.  Opportunistic Task Scheduling over Co-Located Clouds in Mobile Environment , 2018, IEEE Transactions on Services Computing.

[26]  Wei Li,et al.  A Dynamic Service Migration Mechanism in Edge Cognitive Computing , 2018, ACM Trans. Internet Techn..

[27]  Sami Haddadin,et al.  An Artificial Robot Nervous System To Teach Robots How To Feel Pain And Reflexively React To Potentially Damaging Contacts , 2017, IEEE Robotics and Automation Letters.

[28]  Min Chen,et al.  Emotion Communication System , 2017, IEEE Access.

[29]  Yong Du,et al.  Hierarchical recurrent neural network for skeleton based action recognition , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Ping Lu,et al.  Audio-visual emotion fusion (AVEF): A deep efficient weighted approach , 2019, Inf. Fusion.

[31]  Yu Yang,et al.  A System for Human Gait Analysis Based on Body Sensor Network , 2014, 2014 International Conference on Wireless Communication and Sensor Network.

[32]  Liu Yan,et al.  Multi-sensor data fusion system based on Apache Storm , 2017, 2017 3rd IEEE International Conference on Computer and Communications (ICCC).

[33]  Long Hu,et al.  AI-Skin : Skin Disease Recognition based on Self-learning and Wide Data Collection through a Closed Loop Framework , 2019, Inf. Fusion.

[34]  Victor C. M. Leung,et al.  Topology design and cross-layer optimization for wireless body sensor networks , 2017, Ad Hoc Networks.

[35]  Federico Álvarez,et al.  Adaptive Fingerprinting in Multi-Sensor Fusion for Accurate Indoor Tracking , 2017, IEEE Sensors Journal.

[36]  Ivan Tashev,et al.  Unsupervised deep representation learning to remove motion artifacts in free-mode body sensor networks , 2017, 2017 IEEE 14th International Conference on Wearable and Implantable Body Sensor Networks (BSN).

[37]  Snehal Sanjay Kale,et al.  A Secured IoT Based Webcare Healthcare Controlling System using BSN , 2018, 2018 Second International Conference on Inventive Communication and Computational Technologies (ICICCT).

[38]  Victor C. M. Leung,et al.  Cognitive Information Measurements: A New Perspective , 2019, Inf. Sci..

[39]  Giancarlo Fortino,et al.  BodyCloud: A SaaS approach for community Body Sensor Networks , 2014, Future Gener. Comput. Syst..

[40]  Andrew Zisserman,et al.  Two-Stream Convolutional Networks for Action Recognition in Videos , 2014, NIPS.

[41]  Lorenzo Cominelli,et al.  Design and Evaluation of a Unique Social Perception System for Human–Robot Interaction , 2017, IEEE Transactions on Cognitive and Developmental Systems.

[42]  Rogério de Lemos,et al.  Artificial iImmune systems for data fusion: A novel biologically inspired approach , 2008, 2008 11th International Conference on Information Fusion.

[43]  Oluwarotimi Williams Samuel,et al.  Electromyogram-based method to secure wireless body sensor networks for rehabilitation systems , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[44]  Roozbeh Jafari,et al.  Enabling Effective Programming and Flexible Management of Efficient Body Sensor Network Applications , 2013, IEEE Transactions on Human-Machine Systems.

[45]  M. Shamim Hossain,et al.  Localization Based on Social Big Data Analysis in the Vehicular Networks , 2017, IEEE Transactions on Industrial Informatics.

[46]  Derek Anderson,et al.  Multi-sensor conflict measurement and information fusion , 2016, Defense + Security.

[47]  Kay Römer,et al.  Accurate Monitoring of Circardian Rhythms Using Wearable Body Sensor Networks , 2011, IPSN 2011.

[48]  Min Chen,et al.  Wearable Affective Robot , 2018, IEEE Access.

[49]  Kuo-Hui Yeh,et al.  A Secure IoT-Based Healthcare System With Body Sensor Networks , 2016, IEEE Access.

[50]  Srinivasan Alavandar,et al.  Security Aware Trusted Cluster Based Routing Protocol for Wireless Body Sensor Networks , 2018, Wirel. Pers. Commun..

[51]  Adel Belouchrani,et al.  A new multi-sensor fusion algorithm based on the Information Filter framework , 2017, 2017 Seminar on Detection Systems Architectures and Technologies (DAT).

[52]  Yixue Hao,et al.  Label-less Learning for Emotion Cognition , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[53]  Yuan-Ting Zhang,et al.  A flexible tonoarteriography-based body sensor network for cuffless measurement of arterial blood pressure , 2015, 2015 IEEE 12th International Conference on Wearable and Implantable Body Sensor Networks (BSN).

[54]  Giovanni Muscato,et al.  Human-robot interaction through kinect and graphics tablet sensing devices , 2016, 2016 24th Mediterranean Conference on Control and Automation (MED).

[55]  Ming Yang,et al.  3D Convolutional Neural Networks for Human Action Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.