Learning tactile skills through curious exploration

We present curiosity-driven, autonomous acquisition of tactile exploratory skills on a biomimetic robot finger equipped with an array of microelectromechanical touch sensors. Instead of building tailored algorithms for solving a specific tactile task, we employ a more general curiosity-driven reinforcement learning approach that autonomously learns a set of motor skills in absence of an explicit teacher signal. In this approach, the acquisition of skills is driven by the information content of the sensory input signals relative to a learner that aims at representing sensory inputs using fewer and fewer computational resources. We show that, from initially random exploration of its environment, the robotic system autonomously develops a small set of basic motor skills that lead to different kinds of tactile input. Next, the system learns how to exploit the learned motor skills to solve supervised texture classification tasks. Our approach demonstrates the feasibility of autonomous acquisition of tactile skills on physical robotic platforms through curiosity-driven reinforcement learning, overcomes typical difficulties of engineered solutions for active tactile exploration and underactuated control, and provides a basis for studying developmental learning through intrinsic motivation in robots.

[1]  Scott Kuindersma,et al.  Autonomous Skill Acquisition on a Mobile Manipulator , 2011, AAAI.

[2]  Maria Chiara Carrozza,et al.  Roughness Encoding in Human and Biomimetic Artificial Touch: Spatiotemporal Frequency Modulation and Structural Anisotropy of Fingerprints , 2011, Sensors.

[3]  Kenneth O. Johnson,et al.  Neural Coding Mechanisms in Tactile Pattern Recognition: The Relative Contributions of Slowly and Rapidly Adapting Mechanoreceptors to Perceived Roughness , 1997, The Journal of Neuroscience.

[4]  E P Gardner,et al.  Simulation of motion on the skin. I. Receptive fields and temporal frequency coding by cutaneous mechanoreceptors of OPTACON pulses delivered to the hand. , 1989, Journal of neurophysiology.

[5]  M. Hollins,et al.  Pacinian representations of fine surface texture , 2005, Perception & psychophysics.

[6]  Jürgen Schmidhuber,et al.  AutoIncSFA and vision-based developmental learning for humanoid robots , 2011, 2011 11th IEEE-RAS International Conference on Humanoid Robots.

[7]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[8]  Ove Franzén,et al.  Information Processing in the Somatosensory System , 1991 .

[9]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[10]  M. Hollins,et al.  Vibrotactile adaptation impairs discrimination of fine, but not coarse, textures. , 2001, Somatosensory & motor research.

[11]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[12]  A. Goodwin,et al.  Tactile discrimination of gratings , 2004, Experimental Brain Research.

[13]  R. Klatzky,et al.  Haptic perception: A tutorial , 2009, Attention, perception & psychophysics.

[14]  Mark T. Waters,et al.  This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits distribution,andreproductioninanymedium,providedtheoriginalauthorandsourcearecredited.Thislicensedoesnot permit commercial exploitation or the creation of derivative works without sp , 2009 .

[15]  R. Johansson,et al.  Tactile sensibility in the human hand: relative and absolute densities of four types of mechanoreceptive units in glabrous skin. , 1979, The Journal of physiology.

[16]  Kenneth O. Johnson,et al.  Neural Coding Mechanisms Underlying Perceived Roughness of Finely Textured Surfaces , 2001, The Journal of Neuroscience.

[17]  J. Craig,et al.  Texture perception through direct and indirect touch: An analysis of perceptual space for tactile textures in two modes of exploration , 2007, Somatosensory & motor research.

[18]  Jürgen Schmidhuber,et al.  Developmental robotics, optimal artificial curiosity, creativity, music, and the fine arts , 2006, Connect. Sci..

[19]  C. Connor,et al.  Tactile roughness: neural codes that account for psychophysical magnitude estimates , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[20]  Jürgen Schmidhuber,et al.  Formal Theory of Creativity, Fun, and Intrinsic Motivation (1990–2010) , 2010, IEEE Transactions on Autonomous Mental Development.

[21]  Richard S. Sutton,et al.  Reinforcement Learning , 1992, Handbook of Machine Learning.

[22]  Robert G. Radwin,et al.  A new automated tactility test instrument for evaluating hand sensory function , 1993 .

[23]  Christian Cipriani,et al.  Roughness Encoding for Discrimination of Surfaces in Artificial Active-Touch , 2011, IEEE Transactions on Robotics.

[24]  M. Hollins,et al.  The vibrations of texture , 2003, Somatosensory & motor research.

[25]  A. Wing,et al.  Active touch sensing , 2011, Philosophical Transactions of the Royal Society B: Biological Sciences.

[26]  H. Bourlard,et al.  Auto-association by multilayer perceptrons and singular value decomposition , 1988, Biological Cybernetics.

[27]  B. Buchholz,et al.  Anthropometric data for describing the kinematics of the human hand. , 1992, Ergonomics.

[28]  Michail G. Lagoudakis,et al.  Least-Squares Policy Iteration , 2003, J. Mach. Learn. Res..

[29]  J. Randall Flanagan,et al.  Coding and use of tactile signals from the fingertips in object manipulation tasks , 2009, Nature Reviews Neuroscience.

[30]  K. Johnson,et al.  Neural coding of tactile texture: comparison of spatial and temporal mechanisms for roughness perception , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[31]  Pierre-Yves Oudeyer,et al.  Intrinsic Motivation Systems for Autonomous Mental Development , 2007, IEEE Transactions on Evolutionary Computation.

[32]  Ehud Ahissar,et al.  Reinforcement active learning hierarchical loops , 2011, The 2011 International Joint Conference on Neural Networks.

[33]  Susumu Tachi,et al.  Position control of manipulator with passive joints using dynamic coupling , 1991, IEEE Trans. Robotics Autom..

[34]  Gerald E. Loeb,et al.  Bayesian Exploration for Intelligent Identification of Textures , 2012, Front. Neurorobot..

[35]  Benjamin Kuipers,et al.  Autonomous Learning of High-Level States and Actions in Continuous Environments , 2012, IEEE Transactions on Autonomous Mental Development.

[36]  M. Hollins,et al.  Evidence for the duplex theory of tactile texture perception , 2000, Perception & psychophysics.