Towards Deep Developmental Learning

Deep learning techniques are having an undeniable impact on general pattern recognition issues. In this paper, from a developmental robotics perspective, we scrutinize deep learning techniques under the light of their capability to construct a hierarchy of meaningful multimodal representations from the raw sensors of robots. These investigations reveal the differences between the methodological constraints of pattern recognition and those of developmental robotics. In particular, we outline the necessity to rely on unsupervised rather than supervised learning methods and we highlight the need for progress towards the implementation of hierarchical predictive processing capabilities. Based on these new tools, we outline the emergence of a new domain that we call deep developmental learning.

[1]  Pierre-Yves Oudeyer,et al.  Intrinsic Motivation Systems for Autonomous Mental Development , 2007, IEEE Transactions on Evolutionary Computation.

[2]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[3]  Graham W. Taylor,et al.  Generative Class-conditional Autoencoders , 2015, ICLR.

[4]  Roland Memisevic,et al.  Learning to Relate Images , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  André Stuhlsatz,et al.  Discriminative feature extraction with Deep Neural Networks , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[6]  Yoshua Bengio,et al.  Shallow vs. Deep Sum-Product Networks , 2011, NIPS.

[7]  A. Noë,et al.  A sensorimotor account of vision and visual consciousness. , 2001, The Behavioral and brain sciences.

[8]  T. Järvilehto.  The theory of the organism-environment system: III. Role of efferent influences on receptors in the formation of knowledge. , 1999 .

[9]  Yoshua Bengio,et al.  Deep Learning of Representations: Looking Forward , 2013, SLSP.

[10]  Giulio Sandini,et al.  Developmental robotics: a survey , 2003, Connect. Sci..

[11]  Emilio Bizzi,et al.  Combinations of muscle synergies in the construction of a natural motor behavior , 2003, Nature Neuroscience.

[12]  Justus H. Piater,et al.  Emergent structuring of interdependent affordance learning tasks , 2014, 4th International Conference on Development and Learning and on Epigenetic Robotics.

[13]  Yoshua Bengio,et al.  Extracting and composing robust features with denoising autoencoders , 2008, ICML '08.

[14]  Haibo He,et al.  Anticipation-Based Temporal Sequences Learning in Hierarchical Structure , 2007, IEEE Transactions on Neural Networks.

[15]  Olivier Sigaud,et al.  Path Integral Policy Improvement with Covariance Matrix Adaptation , 2012, ICML.

[16]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[17]  P. H. Miller.  Theories of developmental psychology , 1983 .

[18]  J. Hawkins,et al.  On Intelligence , 2004 .

[19]  Pascal Vincent,et al.  Higher Order Contractive Auto-Encoder , 2011, ECML/PKDD.

[20]  Jürgen Schmidhuber,et al.  Evolving deep unsupervised convolutional networks for vision-based reinforcement learning , 2014, GECCO.

[21]  P. L. Adams.  THE ORIGINS OF INTELLIGENCE IN CHILDREN , 1976 .

[22]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[23]  A. Borst.  Seeing smells: imaging olfactory learning in bees , 1999, Nature Neuroscience.

[24]  Christopher W. Geib,et al.  Structural bootstrapping at the sensorimotor level for the fast acquisition of action knowledge for cognitive robots , 2013, 2013 IEEE Third Joint International Conference on Development and Learning and Epigenetic Robotics (ICDL).

[25]  B. Schölkopf,et al.  Modeling Human Motion Using Binary Latent Variables , 2007 .

[26]  Mirko Farina.  Supersizing the Mind: Embodiment, Action and Cognitive Extension. , 2010 .

[27]  Marc'Aurelio Ranzato,et al.  A Unified Energy-Based Framework for Unsupervised Learning , 2007, AISTATS.

[28]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[29]  Honglak Lee,et al.  Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations , 2009, ICML '09.

[30]  Pascal Vincent,et al.  The Difficulty of Training Deep Architectures and the Effect of Unsupervised Pre-Training , 2009, AISTATS.

[31]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[32]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[33]  Jürgen Schmidhuber,et al.  Learning to forget: continual prediction with LSTM , 1999 .

[34]  Jürgen Schmidhuber,et al.  LSTM can Solve Hard Long Time Lag Problems , 1996, NIPS.

[35]  Jürgen Schmidhuber,et al.  Curious model-building control systems , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[36]  Martin V. Butz,et al.  Anticipatory Behavior: Exploiting Knowledge About the Future to Improve Current Behavior , 2003, ABiALS.

[37]  Geoffrey E. Hinton,et al.  Learning a better representation of speech soundwaves using restricted boltzmann machines , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[38]  Ke Chen,et al.  Exploring speaker-specific characteristics with deep learning , 2011, The 2011 International Joint Conference on Neural Networks.

[39]  Isabelle Guyon,et al.  ChaLearn gesture challenge: Design and first results , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[40]  Jason Weston,et al.  Curriculum learning , 2009, ICML '09.

[41]  Hariharan Narayanan,et al.  Sample Complexity of Testing the Manifold Hypothesis , 2010, NIPS.

[42]  Martin V. Butz,et al.  Internal Models and Anticipations in Adaptive Learning Systems , 2003, ABiALS.

[43]  John R. Searle,et al.  Minds, brains, and programs , 1980, Behavioral and Brain Sciences.

[44]  Martin A. Riedmiller,et al.  Deep auto-encoder neural networks in reinforcement learning , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[45]  Pierre-Yves Oudeyer,et al.  R-IAC: Robust Intrinsically Motivated Exploration and Active Learning , 2009, IEEE Transactions on Autonomous Mental Development.

[46]  Olivier Sigaud,et al.  Gated Autoencoders with Tied Input Weights , 2013, ICML.

[47]  Jun Tani,et al.  Learning to generate articulated behavior through the bottom-up and the top-down interaction processes , 2003, Neural Networks.

[48]  Geoffrey E. Hinton.  A Practical Guide to Training Restricted Boltzmann Machines , 2012, Neural Networks: Tricks of the Trade.

[49]  Jun Nakanishi,et al.  Dynamical Movement Primitives: Learning Attractor Models for Motor Behaviors , 2013, Neural Computation.

[50]  Andrew W. Senior,et al.  Long Short-Term Memory Based Recurrent Neural Network Architectures for Large Vocabulary Speech Recognition , 2014, ArXiv.

[51]  Karl J. Friston,et al.  Predictive coding under the free-energy principle , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[52]  Rodney A. Brooks,et al.  A Robust Layered Control Syste For A Mobile Robot , 2022 .

[53]  Jonathan D. Cohen,et al.  Rubber hands ‘feel’ touch that eyes see , 1998, Nature.

[54]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[55]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[56]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[57]  M. Khamassi,et al.  Replay of rule-learning related neural patterns in the prefrontal cortex during sleep , 2009, Nature Neuroscience.

[58]  Olivier Sigaud,et al.  Robot Skill Learning: From Reinforcement Learning to Evolution Strategies , 2013, Paladyn J. Behav. Robotics.

[59]  A. Clark.  Whatever next? Predictive brains, situated agents, and the future of cognitive science. , 2013, The Behavioral and brain sciences.

[60]  Geoffrey E. Hinton,et al.  Learning Multilevel Distributed Representations for High-Dimensional Sequences , 2007, AISTATS.

[61]  Pierre-Yves Oudeyer,et al.  Object Learning Through Active Exploration , 2014, IEEE Transactions on Autonomous Mental Development.

[62]  Geoffrey E. Hinton,et al.  Gated Softmax Classification , 2010, NIPS.

[63]  Emre Ugur,et al.  Affordance learning from range data for multi-step planning , 2009, EpiRob.

[64]  Martin V. Butz,et al.  From Actions to Goals and Vice-Versa: Theoretical Analysis and Models of the Ideomotor Principle and TOTE , 2007, SAB ABiALS.

[65]  Olivier Sigaud,et al.  Deep unsupervised network for multimodal perception, representation and classification , 2015, Robotics Auton. Syst..

[66]  Alexander Stoytchev,et al.  Some Basic Principles of Developmental Robotics , 2009, IEEE Transactions on Autonomous Mental Development.

[67]  Geoffrey E. Hinton,et al.  Learning to Represent Spatial Transformations with Factored Higher-Order Boltzmann Machines , 2010, Neural Computation.

[68]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[69]  Haitham Bou-Ammar,et al.  Factored four way conditional restricted Boltzmann machines for activity recognition , 2015, Pattern Recognit. Lett..

[70]  L. Vygotsky.  Play and Its Role in the Mental Development of the Child , 1967 .

[71]  Dirk Kraft,et al.  A Survey of the Ontogeny of Tool Use: From Sensorimotor Experience to Planning , 2013, IEEE Transactions on Autonomous Mental Development.

[72]  Droniou Alain,et al.  Gated Autoencoders with Tied Input Weights , 2013, ICML 2013.

[73]  Jürgen Schmidhuber,et al.  Bidirectional LSTM Networks for Improved Phoneme Classification and Recognition , 2005, ICANN.

[74]  Alessandro Sperduti,et al.  Neural Networks for Sequential Data: a Pre-training Approach based on Hidden Markov Models , 2015, Neurocomputing.

[75]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[76]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[77]  G. Pezzulo,et al.  Thinking as the control of imagination: a conceptual framework for goal-directed systems , 2009, Psychological research.

[78]  E. Koechlin,et al.  The Architecture of Cognitive Control in the Human Prefrontal Cortex , 2003, Science.

[79]  Biing-Hwang Juang,et al.  Recurrent deep neural networks for robust speech recognition , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[80]  DarrellTrevor,et al.  End-to-end training of deep visuomotor policies , 2016 .

[81]  David J. Schwab,et al.  An exact mapping between the Variational Renormalization Group and Deep Learning , 2014, ArXiv.

[82]  Bruno Lara,et al.  Anticipation by multi-modal association through an artificial mental imagery process , 2014, Connect. Sci..

[83]  Luca Maria Gambardella,et al.  Deep Big Multilayer Perceptrons for Digit Recognition , 2012, Neural Networks: Tricks of the Trade.

[84]  Marco Wiering,et al.  2011 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN) , 2011, IJCNN 2011.

[85]  Allen Newell,et al.  Physical Symbol Systems , 1980, Cogn. Sci..

[86]  James Martens,et al.  Deep learning via Hessian-free optimization , 2010, ICML.

[87]  Luc De Raedt,et al.  Learning relational affordance models for robots in multi-object manipulation tasks , 2012, 2012 IEEE International Conference on Robotics and Automation.

[88]  A. Berthoz,et al.  Le sens du mouvement , 1997 .

[89]  Jürgen Schmidhuber,et al.  Multi-column deep neural network for traffic sign classification , 2012, Neural Networks.

[90]  J. H. Ward.  Hierarchical Grouping to Optimize an Objective Function , 1963 .

[91]  Geoffrey E. Hinton,et al.  Schemata and Sequential Thought Processes in PDP Models , 1986 .

[92]  Active Learning and Intrinsically Motivated Exploration in Robots : Advances and Challenges , 2010 .

[93]  Derek C. Rose,et al.  Deep Machine Learning - A New Frontier in Artificial Intelligence Research [Research Frontier] , 2010, IEEE Computational Intelligence Magazine.

[94]  Terrence J Sejnowski,et al.  Foundations for a New Science of Learning , 2009, Science.

[95]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[96]  B. McNaughton,et al.  Replay of Neuronal Firing Sequences in Rat Hippocampus During Sleep Following Spatial Experience , 1996, Science.

[97]  Lawrence Cayton,et al.  Algorithms for manifold learning , 2005 .

[98]  James B. Marshall,et al.  ECA: An enactivist cognitive architecture based on sensorimotor modeling , 2013, BICA 2013.

[99]  Geoffrey E. Hinton,et al.  Deep Boltzmann Machines , 2009, AISTATS.

[100]  Marco Zorzi,et al.  Emergence of a 'visual number sense' in hierarchical generative models , 2012, Nature Neuroscience.

[101]  Nikolas Hemion,et al.  Grounding object perception in a naive agent's sensorimotor experience , 2015, 2015 Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob).

[102]  Teuvo Kohonen,et al.  Self-organized formation of topologically correct feature maps , 2004, Biological Cybernetics.

[103]  Minoru Asada,et al.  Cognitive developmental robotics as a new paradigm for the design of humanoid robots , 2001, Robotics Auton. Syst..

[104]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[105]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[106]  Christina Freytag.  Being There Putting Brain Body And World Together Again , 2016 .

[107]  Michail G. Lagoudakis,et al.  Least-Squares Policy Iteration , 2003, J. Mach. Learn. Res..

[108]  Giovanni Pezzulo,et al.  Coordinating with the Future: The Anticipatory Nature of Representation , 2008, Minds and Machines.

[109]  Pascal Vincent,et al.  Contractive Auto-Encoders: Explicit Invariance During Feature Extraction , 2011, ICML.

[110]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[111]  三嶋 博之.  The theory of affordances , 2008 .

[112]  Stevan Harnad.  The Symbol Grounding Problem , 1999, ArXiv.

[113]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[114]  Christian Wolf,et al.  Multi-scale Deep Learning for Gesture Detection and Localization , 2014, ECCV Workshops.

[115]  Honglak Lee,et al.  Deep learning for detecting robotic grasps , 2013, Int. J. Robotics Res..

[116]  Olivier Sigaud,et al.  Learning a repertoire of actions with deep neural networks , 2014, 4th International Conference on Development and Learning and on Epigenetic Robotics.

[117]  J. Piaget.  The construction of reality in the child , 1954 .

[118]  Michael Gasser,et al.  The Development of Embodied Cognition: Six Lessons from Babies , 2005, Artificial Life.

[119]  Yoshua Bengio,et al.  Gradient Flow in Recurrent Nets: the Difficulty of Learning Long-Term Dependencies , 2001 .

[120]  Roland Memisevic,et al.  Learning to encode motion using spatio-temporal synchrony , 2013, ICLR 2014.

[121]  Roland Memisevic,et al.  Modeling Deep Temporal Dependencies with Recurrent "Grammar Cells" , 2014, NIPS.

[122]  Sergey Levine,et al.  Learning Neural Network Policies with Guided Policy Search under Unknown Dynamics , 2014, NIPS.

[123]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[124]  Pierre-Yves Oudeyer,et al.  Guest Editorial Active Learning and Intrinsically Motivated Exploration in Robots: Advances and Challenges , 2010, IEEE Trans. Auton. Ment. Dev..

[125]  Geoffrey Zweig,et al.  Recent advances in deep learning for speech research at Microsoft , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[126]  James L. McClelland,et al.  Autonomous Mental Development by Robots and Animals , 2001, Science.

[127]  Jürgen Schmidhuber,et al.  Sequence Labelling in Structured Domains with Hierarchical Recurrent Neural Networks , 2007, IJCAI.

[128]  Pierre-Yves Oudeyer,et al.  Intrinsically Motivated Learning of Real-World Sensorimotor Skills with Developmental Constraints , 2013, Intrinsically Motivated Learning in Natural and Artificial Systems.

[129]  Marc'Aurelio Ranzato,et al.  Building high-level features using large scale unsupervised learning , 2011, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[130]  Alberto Testolin,et al.  Modeling language and cognition with deep unsupervised learning: a tutorial overview , 2013, Front. Psychol..

[131]  Martin A. Riedmiller,et al.  Deep learning of visual control policies , 2010, ESANN.

[132]  Jürgen Schmidhuber,et al.  A Clockwork RNN , 2014, ICML.

[133]  Sergey Levine,et al.  End-to-End Training of Deep Visuomotor Policies , 2015, J. Mach. Learn. Res..

[134]  Jun Nakanishi,et al.  Movement imitation with nonlinear dynamical systems in humanoid robots , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[135]  Rajat Raina,et al.  Efficient sparse coding algorithms , 2006, NIPS.

[136]  Alexander Stoytchev,et al.  Behavior-Grounded Representation of Tool Affordances , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[137]  Geoffrey E. Hinton,et al.  Two Distributed-State Models For Generating High-Dimensional Time Series , 2011, J. Mach. Learn. Res..

[138]  Geoffrey E. Hinton,et al.  Generating Text with Recurrent Neural Networks , 2011, ICML.

[139]  Ales Ude,et al.  Task adaptation through exploration and action sequencing , 2009, 2009 9th IEEE-RAS International Conference on Humanoid Robots.

[140]  J. Born,et al.  Sleep inspires insight , 2004, Nature.

[141]  Yoshua Bengio,et al.  What regularized auto-encoders learn from the data-generating distribution , 2012, J. Mach. Learn. Res..

[142]  Geoffrey E. Hinton.  Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[143]  Marc'Aurelio Ranzato,et al.  Learning Longer Memory in Recurrent Neural Networks , 2014, ICLR.

[144]  Pascal Vincent,et al.  The Manifold Tangent Classifier , 2011, NIPS.

[145]  Paul Smolensky,et al.  Information processing in dynamical systems: foundations of harmony theory , 1986 .

[146]  A. Clark.  Supersizing the Mind , 2008 .

[147]  Pierre-Yves Oudeyer,et al.  The interaction of maturational constraints and intrinsic motivations in active motor development , 2011, 2011 IEEE International Conference on Development and Learning (ICDL).

[148]  Naftali Tishby,et al.  Deep learning and the information bottleneck principle , 2015, 2015 IEEE Information Theory Workshop (ITW).

[149]  Kristen Grauman,et al.  Learning image representations equivariant to ego-motion , 2015, ArXiv.

[150]  J. MacDonald,et al.  Hearing Lips and Seeing Voices: Illusion and Serendipity in Auditory‐Visual Perception Research , 2008 .

[151]  Geoffrey E. Hinton,et al.  Acoustic Modeling Using Deep Belief Networks , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[152]  Angelo Cangelosi,et al.  The Mechanics of Embodiment: A Dialog on Embodiment and Computational Modeling , 2011, Front. Psychology.

[153]  Geoffrey E. Hinton,et al.  Factored conditional restricted Boltzmann Machines for modeling motion style , 2009, ICML '09.

[154]  Roland Memisevic,et al.  Gradient-based learning of higher-order image features , 2011, 2011 International Conference on Computer Vision.

[155]  Justus H. Piater,et al.  Bootstrapping paired-object affordance learning with learned single-affordance features , 2014, 4th International Conference on Development and Learning and on Epigenetic Robotics.

[156]  Geoffrey E. Hinton,et al.  The Recurrent Temporal Restricted Boltzmann Machine , 2008, NIPS.

[157]  Yann LeCun,et al.  The mnist database of handwritten digits , 2005 .

[158]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[159]  Olivier Sigaud,et al.  Many regression algorithms, one unified model: A review , 2015, Neural Networks.

[160]  Michael Beetz,et al.  ORO, a knowledge management platform for cognitive architectures in robotics , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[161]  Pascal Vincent,et al.  Generalized Denoising Auto-Encoders as Generative Models , 2013, NIPS.

[162]  Angelo Cangelosi,et al.  A Deep Learning Neural Network for Number Cognition: A bi-cultural study with the iCub , 2015, 2015 Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob).

[163]  R. Bellman.  Dynamic programming. , 1957, Science.

[164]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[165]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[166]  Jan Peters,et al.  Reinforcement Learning to Adjust Robot Movements to New Situations , 2010, IJCAI.