A Neurorobotics Approach to Behaviour Selection based on Human Activity Recognition

Behaviour selection has been an active research topic for robotics, in particular in the field of human-robot interaction. For a robot to interact effectively and autonomously with humans, the coupling between techniques for human activity recognition, based on sensing information, and robot behaviour selection, based on decision-making mechanisms, is of paramount importance. However, most approaches to date consist of deterministic associations between the recognised activities and the robot behaviours, neglecting the uncertainty inherent to sequential predictions in real-time applications. In this paper, we address this gap by presenting a neurorobotics approach based on computational models that resemble neurophysiological aspects of living beings. This neurorobotics approach was compared to a non-bioinspired, heuristics-based approach. To evaluate both approaches, a robot simulation is developed, in which a mobile robot has to accomplish tasks according to the activity being performed by the inhabitant of an intelligent home. The outcomes of each approach were evaluated according to the number of correct outcomes provided by the robot. Results revealed that the neurorobotics approach is advantageous, especially considering the computational models based on more complex animals.

[1]  Roseli A. Francelin Romero,et al.  Activity Recognition for Ambient Assisted Living with Videos, Inertial Units and Ambient Sensors , 2021, Sensors.

[2]  Jean Liénard,et al.  A biologically constrained model of the whole basal ganglia addressing the paradoxes of connections and selection , 2013, Journal of Computational Neuroscience.

[3]  Antonio Bicchi,et al.  Combined Sensing, Cognition, Learning, and Control for Developing Future Neuro-Robotics Systems: A Survey , 2019, IEEE Transactions on Cognitive and Developmental Systems.

[4]  Alexxai V. Kravitz,et al.  A competitive model for striatal action selection , 2019, Brain Research.

[5]  Michael A. Arbib,et al.  Neurorobotics: From Vision to Action , 2008, Springer Handbook of Robotics.

[6]  Marco Aiello,et al.  Planning meets activity recognition: Service coordination for intelligent buildings , 2017, Pervasive Mob. Comput..

[7]  Basilio Sierra,et al.  Video Activity Recognition: State-of-the-Art , 2019, Sensors.

[8]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Renan C. Moioli,et al.  Evaluation of Frequency-Dependent Effects of Deep Brain Stimulation in a Cortex-Basal Ganglia-Thalamus Network Model of Parkinson’s Disease* , 2020, 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC).

[10]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[11]  Jhielson M. Pimentel,et al.  Unveiling Parkinson’s Disease Features from a Primate Model with Deep Neural Networks , 2020, 2020 International Joint Conference on Neural Networks (IJCNN).

[12]  Daniel Roggen,et al.  Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition , 2016, Sensors.

[13]  Laura Sacerdote,et al.  Mean Instantaneous Firing Frequency Is Always Higher Than the Firing Rate , 2004, Neural Computation.

[14]  U. Raghavendra,et al.  A deep learning approach for Parkinson’s disease diagnosis from EEG signals , 2018, Neural Computing and Applications.

[15]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[16]  Balasubramanian Raman,et al.  Evaluating fusion of RGB-D and inertial sensors for multimodal human action recognition , 2020, J. Ambient Intell. Humaniz. Comput..

[17]  Tetsuji Ogawa,et al.  CHARM-Deep: Continuous Human Activity Recognition Model Based on Deep Neural Network Using IMU Sensors of Smartwatch , 2020, IEEE Sensors Journal.

[18]  A. Teixeira,et al.  Animal models of neurodegenerative diseases. , 2013, Revista brasileira de psiquiatria.

[19]  Warren M. Grill,et al.  A biophysical model of the cortex-basal ganglia-thalamus network in the 6-OHDA lesioned rat model of Parkinson’s disease , 2016, Journal of Computational Neuroscience.

[20]  Nicolas Tabareau,et al.  Where neuroscience and dynamic system theory meet autonomous robotics: A contracting basal ganglia model for action selection , 2008, Neural Networks.

[21]  Alexandra B. Nelson,et al.  Circuit Mechanisms of Parkinson’s Disease , 2019, Neuron.

[22]  Roseli A. F. Romero,et al.  Uncovering Human Multimodal Activity Recognition with a Deep Learning Approach , 2020, 2020 International Joint Conference on Neural Networks (IJCNN).

[23]  Weihua Sheng,et al.  Detection of privacy-sensitive situations for social robots in smart homes , 2016, 2016 IEEE International Conference on Automation Science and Engineering (CASE).

[24]  Rong Liu,et al.  A Decision-Making Model Based on Basal Ganglia Account of Action Prediction , 2019, 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[25]  Caitlyn Clabaugh,et al.  Escaping Oz: Autonomy in Socially Assistive Robotics , 2019, Annu. Rev. Control. Robotics Auton. Syst..

[26]  Dong Seog Han,et al.  Feature Representation and Data Augmentation for Human Activity Classification Based on Wearable IMU Sensor Data Using a Deep LSTM Neural Network , 2018, Sensors.

[27]  William W. Lytton,et al.  NetPyNE, a tool for data-driven multiscale modeling of brain circuits , 2019, eLife.

[28]  Min Tan,et al.  Real-Time Human-Robot Interaction for a Service Robot Based on 3D Human Activity Recognition and Human-Mimicking Decision Mechanism , 2018, 2018 IEEE 8th Annual International Conference on CYBER Technology in Automation, Control, and Intelligent Systems (CYBER).

[29]  Horst Bischof,et al.  A Duality Based Approach for Realtime TV-L1 Optical Flow , 2007, DAGM-Symposium.

[30]  Senem Velipasalar,et al.  Autonomous Human Activity Classification From Wearable Multi-Modal Sensors , 2019, IEEE Sensors Journal.

[31]  Andrea A. Kühn,et al.  Beta burst dynamics in Parkinson’s disease OFF and ON dopaminergic medication , 2017, Brain : a journal of neurology.

[32]  Abdelghani Chibani,et al.  Hybrid Approach for Human Activity Recognition by Ubiquitous Robots , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[33]  Juan J Mariman,et al.  Oscillations in cortico-basal ganglia circuits - Implications for Parkinson's disease and other neurologic and psychiatric conditions. , 2019, Journal of neurophysiology.

[34]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[35]  Andrew Howard,et al.  Design and use paradigms for Gazebo, an open-source multi-robot simulator , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[36]  Vicente Matellán Olivera,et al.  A context‐awareness model for activity recognition in robot‐assisted scenarios , 2020, Expert Syst. J. Knowl. Eng..

[37]  Hitoshi Kita,et al.  Cortical Stimulation Evokes Abnormal Responses in the Dopamine-Depleted Rat Basal Ganglia , 2011, The Journal of Neuroscience.

[38]  Davide Bacciu,et al.  An ambient intelligence approach for learning in smart robotic environments , 2019, Comput. Intell..

[40]  Scott W. Linderman,et al.  The Striatum Organizes 3D Behavior via Moment-to-Moment Action Selection , 2018, Cell.

[41]  A. Kuznetsov,et al.  Basal ganglia role in learning rewarded actions and executing previously learned choices: Healthy and diseased states , 2019, bioRxiv.

[42]  J. Brotchie,et al.  Animal models of α-synucleinopathy for Parkinson disease drug development , 2017, Nature Reviews Neuroscience.

[43]  Abigail Morrison,et al.  Exploring the role of striatal D1 and D2 medium spiny neurons in action selection using a virtual robotic framework , 2018, The European journal of neuroscience.

[44]  Wolfram Burgard,et al.  Improved Techniques for Grid Mapping With Rao-Blackwellized Particle Filters , 2007, IEEE Transactions on Robotics.

[45]  Sacha Jennifer van Albada,et al.  Mean-field modeling of the basal ganglia-thalamocortical system. I Firing rates in healthy and parkinsonian states. , 2009, Journal of theoretical biology.

[46]  Jong-Hwan Kim,et al.  Behavior Selection of Social Robots Using Developmental Episodic Memory-Based Mechanism of Thought , 2018, 2018 IEEE International Conference on Consumer Electronics - Asia (ICCE-Asia).

[47]  Mehrtash Tafazzoli Harandi,et al.  Going deeper into action recognition: A survey , 2016, Image Vis. Comput..

[48]  Ghassan Al-Regib,et al.  TS-LSTM and Temporal-Inception: Exploiting Spatiotemporal Dynamics for Activity Recognition , 2017, Signal Process. Image Commun..

[49]  Mark D. Humphries,et al.  A robot model of the basal ganglia: Behavior and intrinsic processing , 2006, Neural Networks.

[50]  Felipe Aparecido Garcia,et al.  Temporal Approaches for Human Activity Recognition Using Inertial Sensors , 2019, 2019 Latin American Robotics Symposium (LARS), 2019 Brazilian Symposium on Robotics (SBR) and 2019 Workshop on Robotics in Education (WRE).

[51]  Paolo Sernani,et al.  Exploring the ambient assisted living domain: a systematic review , 2017, J. Ambient Intell. Humaniz. Comput..

[52]  Jhielson M. Pimentel,et al.  A Data-Driven Biophysical Computational Model of Parkinson’s Disease Based on Marmoset Monkeys , 2021, IEEE Access.