Intrinsic Robotic Introspection: Learning Internal States From Neuron Activations

We present an introspective framework inspired by the process of how humans perform introspection. Our working assumption is that neural network activations encode information, and building internal states from these activations can improve the performance of an actor-critic model. We perform experiments where we first train a Variational Autoencoder model to reconstruct the activations of a feature extraction network and use the latent space to improve the performance of an actor-critic when deciding which low-level robotic behaviour to execute. We show that internal states reduce the number of episodes needed by about 1300 episodes while training an actor-critic, denoting faster convergence to get a high success value while completing a robotic task.

[1]  Gerardo Aragon-Camarasa,et al.  On Simple Reactive Neural Networks for Behaviour-Based Reinforcement Learning , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[2]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[3]  P. Rochat.  Self-Unity as Ground Zero of Learning and Development , 2019, Front. Psychol..

[4]  Stefano Fusi,et al.  Emotion, cognition, and mental state representation in amygdala and prefrontal cortex. , 2010, Annual review of neuroscience.

[5]  S. Dehaene,et al.  What is consciousness, and could machines have it? , 2017, Science.

[6]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[7]  Betty J. Mohler,et al.  How Cognitive Models of Human Body Experience Might Push Robotics , 2019, Front. Neurorobot..

[8]  Bruno Lara,et al.  Exploration Behaviors, Body Representations, and Simulation Processes for the Development of Cognition in Artificial Agents , 2016, Front. Robot. AI.

[9]  Yuval Tassa,et al.  MuJoCo: A physics engine for model-based control , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[10]  Kensuke Harada,et al.  Online robot introspection via wrench-based action grammars , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[11]  Hongmin Wu,et al.  Robot introspection with Bayesian nonparametric vector autoregressive hidden Markov models , 2017, 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids).

[12]  Rami Bahsoon,et al.  A Survey of Self-Awareness and Its Application in Computing Systems , 2011, 2011 Fifth IEEE Conference on Self-Adaptive and Self-Organizing Systems Workshops.

[13]  Giovanni Pilato,et al.  Humanoid Introspection: A Practical Approach , 2013 .

[14]  Andrey Sheka,et al.  Robot Self-Awareness: Exploration of Internal States , 2012 .

[15]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[16]  Wojciech Zaremba,et al.  OpenAI Gym , 2016, ArXiv.

[18]  Frank Jäkel,et al.  Introspection in Problem Solving , 2013, J. Probl. Solving.