Electronic Skins for Intelligent Soft Robots

1 Department of Mechanical and Aerospace Engineering, University of California, San Diego, USA 2 Department of Mechanical Engineering & Materials Science, Yale University, USA 3 Department of Chemical Engineering & Material Science and Engineering, Stanford University, USA 4 Department of Engineering, University of Cambridge, UK, 5 Department of Mechanical and Aerospace Engineering, Seoul National University, South Korea

[1]  Matthias Scheutz,et al.  The Tactile Ethics of Soft Robotics: Designing Wisely for Human-Robot Interaction. , 2017, Soft robotics.

[2]  R. Johansson,et al.  Tactile sensibility in the human hand: relative and absolute densities of four types of mechanoreceptive units in glabrous skin. , 1979, The Journal of physiology.

[3]  Dmitry Berenson,et al.  Improving Soft Pneumatic Actuator fingers through integration of soft sensors, position and force control, and rigid fingernails , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[4]  Daniel M. Vogt,et al.  Embedded 3D Printing of Strain Sensors within Highly Stretchable Elastomers , 2014, Advanced materials.

[5]  Patricia A. Xu,et al.  Optical lace for synthetic afferent neural networks , 2019, Science Robotics.

[6]  M. J. Wells Tactile Discrimination of Surface Curvature and Shape by the Octopus , 1964 .

[7]  Jan Peters,et al.  Grip Stabilization of Novel Objects Using Slip Prediction , 2018, IEEE Transactions on Haptics.

[8]  Pierre-Yves Oudeyer,et al.  Information-seeking, curiosity, and attention: computational and neural mechanisms , 2013, Trends in Cognitive Sciences.

[9]  Yong-Lae Park,et al.  Optically Sensorized Elastomer Air Chamber for Proprioceptive Sensing of Soft Pneumatic Actuators , 2020, IEEE Robotics and Automation Letters.

[10]  Oliver Kroemer,et al.  Soft Magnetic Skin for Continuous Deformation Sensing , 2019, Adv. Intell. Syst..

[11]  Vincent Hayward,et al.  Large-Area Soft e-Skin: The Challenges Beyond Sensor Designs , 2019, Proceedings of the IEEE.

[12]  Stefan Schaal,et al.  Understanding haptics by evolving mechatronic systems. , 2011, Progress in brain research.

[13]  Brian Litt,et al.  Flexible, Foldable, Actively Multiplexed, High-Density Electrode Array for Mapping Brain Activity in vivo , 2011, Nature Neuroscience.

[14]  Gordon Cheng,et al.  Tactile-based active object discrimination and target object search in an unknown workspace , 2018, Autonomous Robots.

[15]  Jamie Kyujin Paik,et al.  An any-resolution pressure localization scheme using a soft capacitive sensor skin , 2018, 2018 IEEE International Conference on Soft Robotics (RoboSoft).

[16]  Jamie Paik,et al.  Closed-Loop Haptic Feedback Control Using a Self-Sensing Soft Pneumatic Actuator Skin. , 2020, Soft robotics.

[17]  Fumiya Iida,et al.  Non-Destructive Robotic Assessment of Mango Ripeness via Multi-Point Soft Haptics , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[18]  J. Randall Flanagan,et al.  Coding and use of tactile signals from the fingertips in object manipulation tasks , 2009, Nature Reviews Neuroscience.

[19]  Junghan Kwon,et al.  Adaptive Calibration of Soft Sensors Using Optimal Transportation Transfer Learning for Mass Production and Long‐Term Usage , 2020, Adv. Intell. Syst..

[20]  Katsu Yamane,et al.  Design and fabrication of a soft robotic hand and arm system , 2018, 2018 IEEE International Conference on Soft Robotics (RoboSoft).

[21]  Michelle C. Yuen,et al.  Morphing Robots Using Robotic Skins That Sculpt Clay , 2019, IEEE Robotics and Automation Letters.

[22]  Massimo Totaro,et al.  Toward Perceptive Soft Robots: Progress and Challenges , 2018, Advanced science.

[23]  Heinrich M. Jaeger,et al.  JSEL: Jamming Skin Enabled Locomotion , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[24]  David Kim,et al.  FlexSense: a transparent self-sensing deformable surface , 2014, UIST.

[25]  R. Klatzky,et al.  Hand movements: A window into haptic object recognition , 1987, Cognitive Psychology.

[26]  Mari Velonaki,et al.  Robotics and Autonomous Systems , 2014 .

[27]  T. Someya,et al.  Conformable, flexible, large-area networks of pressure and thermal sensors with organic transistor active matrixes. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[28]  H. Shinoda,et al.  3D shape measuring sheet utilizing gravitational and geomagnetic fields , 2008, 2008 SICE Annual Conference.

[29]  Fulvio Mastrogiovanni,et al.  Skin spatial calibration using force/torque measurements , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[30]  Carmel Majidi,et al.  Machine Learning for Soft Robotic Sensing and Control , 2020, Adv. Intell. Syst..

[31]  Yasuo Kuniyoshi,et al.  Humanoid robot which can lift a 30kg box by whole body contact and tactile feedback , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[32]  Sergey Levine,et al.  Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations , 2017, Robotics: Science and Systems.

[33]  Zhenan Bao,et al.  A bioinspired flexible organic artificial afferent nerve , 2018, Science.

[34]  Shigeki Sugano,et al.  Whole-body covering tactile interface for human robot coordination , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[35]  Mahmood Karimi,et al.  3D printed soft actuators for a legged robot capable of navigating unstructured terrain , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[36]  Gerald E. Loeb,et al.  Bayesian Exploration for Intelligent Identification of Textures , 2012, Front. Neurorobot..

[37]  Yonggang Huang,et al.  Two-dimensional materials in functional three-dimensional architectures with applications in photodetection and imaging , 2018, Nature Communications.

[38]  D. Rus,et al.  Design, fabrication and control of soft robots , 2015, Nature.

[39]  Mark R. Cutkosky,et al.  Force and Tactile Sensors , 2008, Springer Handbook of Robotics.

[40]  Robert J. Wood,et al.  Wearable soft sensing suit for human gait measurement , 2014, Int. J. Robotics Res..

[41]  Jakub W. Pachocki,et al.  Learning dexterous in-hand manipulation , 2018, Int. J. Robotics Res..

[42]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[43]  Rebecca K. Kramer,et al.  Sensor Skins: An Overview , 2016 .

[44]  Sergey Levine,et al.  One-shot learning of manipulation skills with online dynamics adaptation and neural network priors , 2015, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[45]  Abhinav Gupta,et al.  The Curious Robot: Learning Visual Representations via Physical Interactions , 2016, ECCV.

[46]  Kevin O'Brien,et al.  Optoelectronically innervated soft prosthetic hand via stretchable optical waveguides , 2016, Science Robotics.

[47]  Oussama Khatib,et al.  A hierarchically patterned, bioinspired e-skin able to detect the direction of applied pressure for robotics , 2018, Science Robotics.

[48]  Oussama Khatib,et al.  Ocean One: A Robotic Avatar for Oceanic Discovery , 2016, IEEE Robotics & Automation Magazine.

[49]  Jan Peters,et al.  Stable reinforcement learning with autoencoders for tactile and visual data , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[50]  Wojciech Matusik,et al.  Learning the signatures of the human grasp using a scalable tactile glove , 2019, Nature.

[51]  Yong-Lae Park,et al.  Design and Fabrication of Soft Artificial Skin Using Embedded Microchannels and Liquid Conductors , 2012, IEEE Sensors Journal.

[52]  Nawid Jamali,et al.  Material classification by tactile sensing using surface textures , 2010, 2010 IEEE International Conference on Robotics and Automation.

[53]  Sami Haddadin,et al.  Physical Human-Robot Interaction , 2016, Springer Handbook of Robotics, 2nd Ed..

[54]  Kevin Blankespoor,et al.  BigDog, the Rough-Terrain Quadruped Robot , 2008 .

[55]  Jürgen Schmidhuber,et al.  Learning tactile skills through curious exploration , 2012, Front. Neurorobot..

[56]  D. Dewsbury,et al.  Octopus: Physiology and behaviour of an advanced invertebrate. , 1978 .

[57]  J. C. Stevens Aging and spatial acuity of touch. , 1992, Journal of gerontology.

[58]  Sergey Levine,et al.  Path integral guided policy search , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[59]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[60]  Jivko Sinapov,et al.  Vibrotactile Recognition and Categorization of Surfaces by a Humanoid Robot , 2011, IEEE Transactions on Robotics.

[61]  Cecilia Laschi,et al.  Control Strategies for Soft Robotic Manipulators: A Survey. , 2018, Soft robotics.

[62]  Sigurd Wagner,et al.  Electronic skin: architecture and components , 2004 .

[63]  Danica Kragic,et al.  Learning of grasp adaptation through experience and tactile sensing , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[64]  Josep Amat,et al.  Human-Robot Interaction Based on a Sensitive Bumper Skin , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[65]  Andrew G. Gillies,et al.  Nanowire active-matrix circuitry for low-voltage macroscale artificial skin. , 2010, Nature materials.

[66]  Daniela Rus,et al.  Exploration of underwater life with an acoustically controlled soft robotic fish , 2018, Science Robotics.

[67]  Mehmet Remzi Dogar,et al.  Haptic identification of objects using a modular soft robotic gripper , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[68]  R. F. Shepherd,et al.  Soft optoelectronic sensory foams with proprioception , 2018, Science Robotics.

[69]  Matteo Campanella,et al.  Detection of incipient object slippage by skin-like sensing and neural network processing , 1998, IEEE Trans. Syst. Man Cybern. Part B.

[70]  Máximo A. Roa,et al.  Planning in-hand object manipulation with multifingered hands considering task constraints , 2013, 2013 IEEE International Conference on Robotics and Automation.

[71]  Michelle C. Yuen,et al.  OmniSkins: Robotic skins that turn inanimate objects into multifunctional robots , 2018, Science Robotics.

[72]  Jonathan Rossiter,et al.  Bodily Aware Soft Robots: Integration of Proprioceptive and Exteroceptive Sensors , 2017, ICRA.

[73]  Boris Murmann,et al.  Skin electronics from scalable fabrication of an intrinsically stretchable transistor array , 2018, Nature.

[74]  Gordon Cheng,et al.  3D surface reconstruction for robotic body parts with artificial skins , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[75]  Helge J. Ritter,et al.  A Control Framework for Tactile Servoing , 2013, Robotics: Science and Systems.

[76]  M. Kaltenbrunner,et al.  An ultra-lightweight design for imperceptible plastic electronics , 2013, Nature.

[77]  Jan Peters,et al.  Learning inverse dynamics models with contacts , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[78]  Yong-Lae Park,et al.  Sensorized, Flat, Pneumatic Artificial Muscle Embedded with Biomimetic Microfluidic Sensors for Proprioceptive Feedback. , 2019, Soft robotics.

[79]  R. Johansson,et al.  Properties of cutaneous mechanoreceptors in the human hand related to touch sensation. , 1984, Human neurobiology.

[80]  Katsu Yamane,et al.  Design of a soft upper body robot for physical human-robot interaction , 2015, 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids).

[81]  Giulio Sandini,et al.  Tactile Sensing—From Humans to Humanoids , 2010, IEEE Transactions on Robotics.

[82]  I. Cohen,et al.  Stretchable surfaces with programmable 3D texture morphing for synthetic camouflaging skins , 2017, Science.

[83]  Ross A. Knepper,et al.  A Deformable Interface for Human Touch Recognition Using Stretchable Carbon Nanotube Dielectric Elastomer Sensors and Deep Neural Networks. , 2017, Soft robotics.

[84]  Nan Sun,et al.  Modular and Reconfigurable Wireless E‐Tattoos for Personalized Sensing , 2019, Advanced Materials Technologies.

[85]  Jan Peters,et al.  Learning robot in-hand manipulation with tactile features , 2015, 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids).

[86]  R. Adam Bilodeau,et al.  Monolithic fabrication of sensors and actuators in a soft robotic gripper , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[87]  Razvan Pascanu,et al.  Learning to Navigate in Complex Environments , 2016, ICLR.

[88]  Benjamin C. K. Tee,et al.  Flexible polymer transistors with high pressure sensitivity for application in electronic skin and health monitoring , 2013, Nature Communications.

[89]  Kaspar Althoefer,et al.  Tactile sensing for dexterous in-hand manipulation in robotics-A review , 2011 .

[90]  Matei Ciocarlie,et al.  Data-Driven Super-Resolution on a Tactile Dome , 2018, IEEE Robotics and Automation Letters.

[91]  Si Li,et al.  A neuro-inspired artificial peripheral nervous system for scalable electronic skins , 2019, Science Robotics.

[92]  Robert J. Wood,et al.  Soft curvature sensors for joint angle proprioception , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[93]  Nikolaus Correll,et al.  A soft pneumatic actuator that can sense grasp and touch , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[94]  Sheng Xu,et al.  Three-dimensional integrated stretchable electronics , 2018, Nature Electronics.

[95]  Daniela Rus,et al.  Automated Recycling Separation Enabled by Soft Robotic Material Classification , 2019, 2019 2nd IEEE International Conference on Soft Robotics (RoboSoft).

[96]  Gordon Cheng,et al.  3D spatial self-organization of a modular artificial skin , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[97]  Carmel Majidi,et al.  A biosensing soft robot: Autonomous parsing of chemical signals through integrated organic and inorganic interfaces , 2019, Science Robotics.

[98]  Raeed H. Chowdhury,et al.  Epidermal Electronics , 2011, Science.

[99]  Zhenan Bao,et al.  Pursuing prosthetic electronic skin. , 2016, Nature materials.

[100]  Robert J. Wood,et al.  An integrated design and fabrication strategy for entirely soft, autonomous robots , 2016, Nature.

[101]  Oliver Brock,et al.  A novel type of compliant and underactuated robotic hand for dexterous grasping , 2016, Int. J. Robotics Res..

[102]  Howie Choset,et al.  Continuum Robots for Medical Applications: A Survey , 2015, IEEE Transactions on Robotics.

[103]  Gordon Cheng,et al.  Realizing whole-body tactile interactions with a self-organizing, multi-modal artificial skin on a humanoid robot , 2015, Adv. Robotics.

[104]  Allister F. McGuire,et al.  A skin-inspired organic digital mechanoreceptor , 2015, Science.

[105]  Tomonori Yamamoto,et al.  Use of tactile feedback to control exploratory movements to characterize object compliance , 2012, Front. Neurorobot..

[106]  I. Park,et al.  Stretchable, Skin‐Mountable, and Wearable Strain Sensors and Their Potential Applications: A Review , 2016 .

[107]  Kit-Hang Lee,et al.  Real-Time Surface Shape Sensing for Soft and Flexible Structures Using Fiber Bragg Gratings , 2019, IEEE Robotics and Automation Letters.

[108]  Gordon Cheng,et al.  Whole-Body Active Compliance Control for Humanoid Robots with Robot Skin , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[109]  Cecilia Laschi,et al.  Soft robot perception using embedded soft sensors and recurrent neural networks , 2019, Science Robotics.

[110]  Gordon Cheng,et al.  New materials and advances in making electronic skin for interactive robots , 2015, Adv. Robotics.

[111]  Filip Ilievski,et al.  Multigait soft robot , 2011, Proceedings of the National Academy of Sciences.

[112]  Henrik I. Christensen,et al.  Custom soft robotic gripper sensor skins for haptic object visualization , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[113]  Modris Greitans,et al.  Acceleration and Magnetic Sensor Network for Shape Sensing , 2016, IEEE Sensors Journal.

[114]  Veronica J. Santos,et al.  Biomimetic Tactile Sensor Array , 2008, Adv. Robotics.

[115]  Allison M. Okamura,et al.  A soft robot that navigates its environment through growth , 2017, Science Robotics.

[116]  Joran W. Booth,et al.  Reconfigurable soft body trajectories using unidirectionally stretchable composite laminae , 2019, Nature Communications.

[117]  Hod Lipson,et al.  Unshackling evolution: evolving soft robots with multiple materials and a powerful generative encoding , 2013, GECCO '13.

[118]  Michael T. Tolley,et al.  Design Considerations for 3D Printed, Soft, Multimaterial Resistive Sensors for Soft Robotics , 2019, Front. Robot. AI.

[119]  Yonggang Huang,et al.  Stretchable and Foldable Silicon Integrated Circuits , 2008, Science.

[120]  Gordon Cheng,et al.  A Comprehensive Realization of Robot Skin: Sensors, Sensing, Control, and Applications , 2019, Proceedings of the IEEE.

[121]  Gian Domenico Iannetti,et al.  Whole-Body Mapping of Spatial Acuity for Pain and Touch , 2014, Annals of neurology.

[122]  Jaeha Kim,et al.  Electronic skins for soft, compact, reversible assembly of wirelessly activated fully soft robots , 2018, Science Robotics.

[123]  Xiaochen Ren,et al.  A Low‐Operating‐Power and Flexible Active‐Matrix Organic‐Transistor Temperature‐Sensor Array , 2016, Advanced materials.

[124]  T. Someya,et al.  A Rubberlike Stretchable Active Matrix Using Elastic Conductors , 2008, Science.

[125]  Michael T. Tolley,et al.  Morphing Structure for Changing Hydrodynamic Characteristics of a Soft Underwater Walking Robot , 2019, IEEE Robotics and Automation Letters.

[126]  Nawid Jamali,et al.  Majority Voting: Material Classification by Tactile Sensing Using Surface Texture , 2011, IEEE Transactions on Robotics.