Predictive Processing in Cognitive Robotics: A Review

Predictive processing has become an influential framework in cognitive sciences. This framework turns the traditional view of perception upside down, claiming that the main flow of information processing is realized in a top-down hierarchical manner. Furthermore, it aims at unifying perception, cognition, and action as a single inferential process. However, in the related literature, the predictive processing framework and its associated schemes such as predictive coding, active inference, perceptual inference, free-energy principle, tend to be used interchangeably. In the field of cognitive robotics there is no clear-cut distinction on which schemes have been implemented and under which assumptions. In this paper, working definitions are set with the main aim of analyzing the state of the art in cognitive robotics research working under the predictive processing framework as well as some related non-robotic models. The analysis suggests that, first, both research in cognitive robotics implementations and non-robotic models needs to be extended to the study of how multiple exteroceptive modalities can be integrated into prediction error minimization schemes. Second, a relevant distinction found here is that cognitive robotics implementations tend to emphasize the learning of a generative model, while in non-robotics models it is almost absent. Third, despite the relevance for active inference, few cognitive robotics implementations examine the issues around control and whether it should result from the substitution of inverse models with proprioceptive predictions. Finally, limited attention has been placed on precision weighting and the tracking of prediction error dynamics. These mechanisms should help to explore more complex behaviors ar X iv :2 10 1. 06 61 1v 2 [ cs .R O ] 2 2 Ja n 20 21 and tasks in cognitive robotics research under the predictive processing framework.

[1]  Pierre-Yves Oudeyer,et al.  Intrinsic Motivation Systems for Autonomous Mental Development , 2007, IEEE Transactions on Evolutionary Computation.

[2]  Karl J. Friston The free-energy principle: a rough guide to the brain? , 2009, Trends in Cognitive Sciences.

[3]  D. Knill,et al.  The Bayesian brain: the role of uncertainty in neural coding and computation , 2004, Trends in Neurosciences.

[4]  Anil K. Seth,et al.  Being a Beast Machine: The Somatic Basis of Selfhood , 2018, Trends in Cognitive Sciences.

[5]  G. Pazour,et al.  Ror2 signaling regulates Golgi structure and transport through IFT20 for tumor invasiveness , 2017, Scientific Reports.

[6]  Jun Tani,et al.  Accounting for the Minimal Self and the Narrative Self: Robotics Experiments Using Predictive Coding , 2020, AAAI Spring Symposium: Towards Conscious AI Systems.

[7]  Bruno Lara,et al.  Embodied Cognitive Robotics and the learning of sensorimotor schemes , 2018, Adapt. Behav..

[8]  Jun Tani,et al.  Goal-Directed Behavior under Variational Predictive Coding: Dynamic organization of Visual Attention and Working Memory , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[9]  Dare A. Baldwin,et al.  Evidence for ‘motionese’: modifications in mothers’ infant-directed action , 2002 .

[10]  Karl J. Friston What Is Optimal about Motor Control? , 2011, Neuron.

[11]  A. Clark Embodied Prediction , 2015 .

[12]  Yiannis Demiris,et al.  Hierarchical attentive multiple models for execution and recognition of actions , 2006, Robotics Auton. Syst..

[13]  Bruno Lara,et al.  Exploration Behaviors, Body Representations, and Simulation Processes for the Development of Cognition in Artificial Agents , 2016, Front. Robot. AI.

[14]  Karl J. Friston,et al.  Degeneracy and Redundancy in Active Inference , 2020, Cerebral cortex.

[15]  A. Clark Whatever next? Predictive brains, situated agents, and the future of cognitive science. , 2013, The Behavioral and brain sciences.

[16]  Daniel Williams,et al.  Predictive Processing and the Representation Wars , 2017, Minds and Machines.

[17]  Gordon Cheng,et al.  Active inference body perception and action for humanoid robots , 2019, ArXiv.

[18]  Karl J. Friston,et al.  Planning and navigation as active inference , 2017, Biological Cybernetics.

[19]  Karl J. Friston,et al.  Is the free-energy principle neurocentric? , 2010, Nature Reviews Neuroscience.

[20]  Rajesh P. N. Rao,et al.  Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. , 1999 .

[21]  Jun Tani,et al.  Investigation of Multimodal and Agential Interactions in Human-Robot Imitation, based on frameworks of Predictive Coding and Active Inference , 2020, ArXiv.

[22]  Karl J. Friston,et al.  Attention, Uncertainty, and Free-Energy , 2010, Front. Hum. Neurosci..

[23]  Jun-Cheol Park,et al.  Learning for Goal-Directed Actions Using RNNPB: Developmental Change of “What to Imitate” , 2018, IEEE Transactions on Cognitive and Developmental Systems.

[24]  Karl J. Friston,et al.  Active inference, sensory attenuation and illusions , 2013, Cognitive Processing.

[25]  H. Aarts,et al.  Moving Forward: On the Limits of Motor-Based Forward Models , 2019, Trends in Cognitive Sciences.

[26]  Gordon Cheng,et al.  Adaptive Robot Body Learning and Estimation Through Predictive Coding , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[27]  Andy Clark,et al.  Beyond Desire? Agency, Choice, and the Predictive Mind , 2019, Australasian Journal of Philosophy.

[28]  Bruno Lara-Guzmán,et al.  A Self-Organized Internal Models Architecture for Coding Sensory–Motor Schemes , 2016, Front. Robot. AI.

[29]  David Colliaux,et al.  Intrinsic motivation and episodic memories for robot exploration of high-dimensional sensory spaces , 2020, Adapt. Behav..

[30]  Karl J. Friston,et al.  Working memory, attention, and salience in active inference , 2017, Scientific Reports.

[31]  Philipp Schwartenbeck,et al.  Computational mechanisms of curiosity and goal-directed exploration , 2019, eLife.

[32]  Karl J. Friston,et al.  Active inference and robot control: a case study , 2016, Journal of The Royal Society Interface.

[33]  Hansol Choi,et al.  Predictive Coding Strategies for Developmental Neurorobotics , 2012, Front. Psychology.

[34]  Yukie Nagai,et al.  A predictive coding model of representational drawing in human children and chimpanzees , 2019, 2019 Joint IEEE 9th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob).

[35]  Yukie Nagai,et al.  How development in the Bayesian brain facilitates learning , 2019, 2019 Joint IEEE 9th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob).

[36]  Jun Tani,et al.  Dealing With Large-Scale Spatio-Temporal Patterns in Imitative Interaction Between a Robot and a Human by Using the Predictive Coding Framework , 2020, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[37]  Karl J. Friston,et al.  Predictions not commands: active inference in the motor system , 2012, Brain Structure and Function.

[38]  Michael W. Spratling A review of predictive coding algorithms , 2017, Brain and Cognition.

[39]  Bruno Lara,et al.  Body Representations for Robot Ego-Noise Modelling and Prediction. Towards the Development of a Sense of Agency in Artificial Agents , 2016, ALIFE.

[40]  Karl J. Friston,et al.  A free energy principle for the brain , 2006, Journal of Physiology-Paris.

[41]  Marco Mirolli,et al.  Intrinsically Motivated Learning in Natural and Artificial Systems , 2013 .

[42]  M. Tsakiris,et al.  The free-energy self: A predictive coding account of self-recognition , 2014, Neuroscience & Biobehavioral Reviews.

[43]  Stefano Nolfi,et al.  Learning to perceive the world as articulated: an approach for hierarchical learning in sensory-motor systems , 1998, Neural Networks.

[44]  Christopher L. Buckley,et al.  Active Inference: Computational Models of Motor Control without Efference Copy , 2019, 2019 Conference on Cognitive Computational Neuroscience.

[45]  Jun Tani,et al.  A Novel Predictive-Coding-Inspired Variational RNN Model for Online Prediction and Recognition , 2018, Neural Computation.

[46]  Karl J. Friston,et al.  In the Body’s Eye: The computational anatomy of interoceptive inference , 2019, bioRxiv.

[47]  Mateus Joffily,et al.  Emotional Valence and the Free-Energy Principle , 2013, PLoS Comput. Biol..

[48]  Corrado Pezzato,et al.  A Novel Adaptive Controller for Robot Manipulators Based on Active Inference , 2020, IEEE Robotics and Automation Letters.

[49]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[50]  Karl J. Friston,et al.  All Thinking is ‘Wishful’ Thinking , 2020, Trends in Cognitive Sciences.

[51]  Karl J. Friston,et al.  Computational psychiatry: the brain as a phantastic organ. , 2014, The lancet. Psychiatry.

[52]  Michael I. Jordan,et al.  An internal model for sensorimotor integration. , 1995, Science.

[53]  A. Clark,et al.  Getting ahead: forward models and their place in cognitive architecture , 2014, Trends in Cognitive Sciences.

[54]  Karl J. Friston,et al.  The Computational Anatomy of Psychosis , 2013, Front. Psychiatry.

[55]  Rajesh P. N. Rao,et al.  Predictive Coding , 2019, A Blueprint for the Hard Problem of Consciousness.

[56]  A. Clark A nice surprise? Predictive processing and the active pursuit of novelty , 2018 .

[57]  Jun Tani,et al.  Goal-Directed Planning for Habituated Agents by Active Inference Using a Variational Recurrent Neural Network , 2020, Entropy.

[58]  Karl J. Friston,et al.  Prediction, perception and agency , 2012, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[59]  Karl J. Friston,et al.  Active inference and agency: optimal control without cost functions , 2012, Biological Cybernetics.

[60]  Jun Tani,et al.  How can a recurrent neurodynamic predictive coding model cope with fluctuation in temporal patterns? Robotic experiments on imitative interaction , 2017, Neural Networks.

[61]  Mitsuo Kawato,et al.  Internal models for motor control and trajectory planning , 1999, Current Opinion in Neurobiology.

[62]  Karl J. Friston,et al.  Active inference and epistemic value , 2015, Cognitive neuroscience.

[63]  Bruno Lara,et al.  Tracking Emotions: Intrinsic Motivation Grounded on Multi - Level Prediction Error Dynamics , 2020, 2020 Joint IEEE 10th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob).

[64]  Verena V. Hafner,et al.  A Deep Convolutional Neural Network Model for Sense of Agency and Object Permanence in Robots , 2018, 2018 Joint IEEE 8th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob).

[65]  Michael W. Spratling Predictive coding as a model of biased competition in visual attention , 2008, Vision Research.

[66]  Alexandre Pitti,et al.  Autonomous learning and chaining of motor primitives using the Free Energy Principle , 2020, 2020 International Joint Conference on Neural Networks (IJCNN).

[67]  Karl J. Friston,et al.  A theory of cortical responses , 2005, Philosophical Transactions of the Royal Society B: Biological Sciences.

[68]  Beren Millidge,et al.  Deep Active Inference as Variational Policy Gradients , 2019, Journal of Mathematical Psychology.

[69]  J. Hohwy The Predictive Mind , 2013 .

[70]  Pablo Lanillos,et al.  End-to-End Pixel-Based Deep Active Inference for Body Perception and Action , 2020, 2020 Joint IEEE 10th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob).

[71]  Erik Rietveld,et al.  The feeling of grip: novelty, error dynamics, and the predictive brain , 2017, Synthese.

[72]  Simon McGregor,et al.  The free energy principle for action and perception: A mathematical review , 2017, 1705.09156.

[73]  Robert P. Abelson,et al.  Velocity relation : satisfaction as a function of the first derivative of outcome over time , 1991 .

[74]  Karl J. Friston,et al.  Active Inference, homeostatic regulation and adaptive behavioural control , 2015, Progress in Neurobiology.

[75]  Karl J. Friston,et al.  An Investigation of the Free Energy Principle for Emotion Recognition , 2020, Frontiers in Computational Neuroscience.

[76]  D M Wolpert,et al.  Multiple paired forward and inverse models for motor control , 1998, Neural Networks.

[77]  Karl J. Friston,et al.  Action understanding and active inference , 2011, Biological Cybernetics.

[78]  S. V. D. Cruys,et al.  Affective Value in the Predictive Mind , 2017 .

[79]  Tetsuya Ogata,et al.  A Neurorobotics Simulation of Autistic Behavior Induced by Unusual Sensory Precision , 2018, Computational Psychiatry.

[80]  Karl J. Friston,et al.  A tale of two densities: active inference is enactive inference , 2019, Adapt. Behav..

[81]  Alexander Tschantz,et al.  Learning action-oriented models through active inference , 2020, PLoS computational biology.

[82]  Angelo Cangelosi,et al.  AFA-PredNet: The Action Modulation Within Predictive Coding , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[83]  Wolfram Schenck,et al.  Bootstrapping Cognition from Behavior - A Computerized Thought Experiment , 2008, Cogn. Sci..

[84]  Tom Schaul,et al.  Artificial curiosity for autonomous space exploration , 2011 .

[85]  R. Weale Vision. A Computational Investigation Into the Human Representation and Processing of Visual Information. David Marr , 1983 .

[86]  Karl J. Friston Functional integration and inference in the brain , 2002, Progress in Neurobiology.

[87]  Felix Blankenburg,et al.  Minimal self-models and the free energy principle , 2013, Front. Hum. Neurosci..

[88]  Karl J. Friston,et al.  Waves of prediction , 2019, PLoS biology.

[89]  Shigeki Sugano,et al.  Predictive learning with uncertainty estimation for modeling infants' cognitive development with caregivers: A neurorobotics experiment , 2015, 2015 Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob).

[90]  Gordon Cheng,et al.  Robot self/other distinction: active inference meets neural networks learning in a mirror , 2020, ECAI.

[91]  Christopher L. Buckley,et al.  An active inference implementation of phototaxis , 2017, ECAL.

[92]  Karl J. Friston,et al.  Action perception as hypothesis testing , 2017, Cortex.

[93]  Karl J. Friston The free-energy principle: a unified brain theory? , 2010, Nature Reviews Neuroscience.

[94]  Karl J. Friston,et al.  Computational mechanisms of curiosity and goal-directed exploration , 2018, bioRxiv.

[95]  James C. R. Whittington,et al.  Theories of Error Back-Propagation in the Brain , 2019, Trends in Cognitive Sciences.

[96]  Giovanni Pezzulo,et al.  Model-Based Approaches to Active Perception and Control , 2017, Entropy.

[97]  Karl J. Friston,et al.  The Depressed Brain: An Evolutionary Systems Theory , 2017, Trends in Cognitive Sciences.

[98]  Homayoun Najjaran,et al.  Deep Learning Sensor Fusion for Autonomous Vehicle Perception and Localization: A Review , 2020, Sensors.