Intrinsic Robotic Introspection: Learning Internal States From Neuron Activations

We present an introspective framework inspired by the process of how humans perform introspection. Our working assumption is that neural network activations encode information, and building internal states from these activations can improve the performance of an actor-critic model. We perform experiments where we first train a Variational Autoencoder model to reconstruct the activations of a feature extraction network and use the latent space to improve the performance of an actor-critic when deciding which low-level robotic behaviour to execute. We show that internal states reduce the number of episodes needed by about 1300 episodes while training an actor-critic, denoting faster convergence to get a high success value while completing a robotic task.

[1]  Rami Bahsoon,et al.  A Survey of Self-Awareness and Its Application in Computing Systems , 2011, 2011 Fifth IEEE Conference on Self-Adaptive and Self-Organizing Systems Workshops.

[2]  Yuval Tassa,et al.  MuJoCo: A physics engine for model-based control , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[3]  P. Rochat.  Self-Unity as Ground Zero of Learning and Development , 2019, Front. Psychol..

[4]  Frank Jäkel,et al.  Introspection in Problem Solving , 2013, J. Probl. Solving.

[5]  S. Dehaene,et al.  What is consciousness, and could machines have it? , 2017, Science.

[6]  Anant Agarwal,et al.  Self-Aware Computing , 2009 .

[7]  Stefano Fusi,et al.  Emotion, cognition, and mental state representation in amygdala and prefrontal cortex. , 2010, Annual review of neuroscience.

[8]  Wojciech Zaremba,et al.  OpenAI Gym , 2016, ArXiv.

[10]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[11]  Giovanni Pilato,et al.  Humanoid Introspection: A Practical Approach , 2013 .

[12]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[13]  Andrey Sheka,et al.  Robot Self-Awareness: Exploration of Internal States , 2012 .

[14]  Bruno Lara,et al.  Exploration Behaviors, Body Representations, and Simulation Processes for the Development of Cognition in Artificial Agents , 2016, Front. Robot. AI.

[15]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[16]  Gerardo Aragon-Camarasa,et al.  On Simple Reactive Neural Networks for Behaviour-Based Reinforcement Learning , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[17]  Pierre-Yves Oudeyer,et al.  How Many Random Seeds? Statistical Power Analysis in Deep Reinforcement Learning Experiments , 2018, ArXiv.

[18]  Axel Cleeremans,et al.  Consciousness and metarepresentation: A computational sketch , 2007, Neural Networks.

[19]  Kensuke Harada,et al.  Online robot introspection via wrench-based action grammars , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[20]  Hongmin Wu,et al.  Robot introspection with Bayesian nonparametric vector autoregressive hidden Markov models , 2017, 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids).

[21]  Betty J. Mohler,et al.  How Cognitive Models of Human Body Experience Might Push Robotics , 2019, Front. Neurorobot..