Active Learning and Intrinsically Motivated Exploration in Robots : Advances and Challenges

L EARNING techniques are increasingly being used in today’s complex robotic systems. Robots are expected to deal with a large variety of tasks using their high-dimensional and complex bodies, to manipulate objects and also, to interact with humans in an intuitive and friendly way. In this new setting, not all relevant information is available at design time, and robots should typically be able to learn, through self-experimentation or through human–robot interaction, how to tune their innate perceptual-motor skills or to learn, cumulatively, novel skills that were not preprogrammed initially. In a word, robots need to have the capacity to develop in an open-ended manner and in an open-ended environment, in a way that is analogous to human development which combines genetic and epigenetic factors. This challenge is at the center of the developmental robotics field [7], [35]–[37]. Among the various technical challenges that are raised by these issues, exploration is paramount. Self-experimentation and learning by interacting with the physical and social world is essential to acquire new knowledge and skills. Exploration in physical robotic agents poses challenging problems due to the high-dimensional and complex dynamics of the body–environment system (especially when other agents are part of the environment), and to the open-ended nature of the sensory-motor spaces of real environments. Typically, in those spaces, the lack of adequate constraints in exploration strategies can result in at best very slow learning, and most often in no consistent learning that can even be dangerous or destructive. This special issue1 provides a discussion forum and presents novel contributions on intentional exploration, i.e., internal mechanisms and constraints that explicitly foster organized exploration. Of central interest are mechanisms that push agents to select actions that allow them to gain maximal knowledge or maximal control/competence over their bodies and environments. To this, end different fields suggested different approaches, ranging from operational heuristics that indirectly target maximal information gain, to theoretically