NeuroTac: A Neuromorphic Optical Tactile Sensor applied to Texture Recognition

Developing artificial tactile sensing capabilities that rival human touch is a long-term goal in robotics and prosthetics. Gradually more elaborate biomimetic tactile sensors are being developed and applied to grasping and manipulation tasks to help achieve this goal. Here we present the neuroTac, a novel neuromorphic optical tactile sensor. The neuroTac combines the biomimetic hardware design from the TacTip sensor which mimicks the layered papillae structure of human glabrous skin, with an event-based camera (DAVIS240, iniVation) and algorithms which transduce contact information in the form of spike trains. The performance of the sensor is evaluated on a texture classification task, with four spike coding methods being implemented and compared: Intensive, Spatial, Temporal and Spatiotemporal. We found timing-based coding methods performed with the highest accuracy over both artificial and natural textures. The spike-based output of the neuroTac could enable the development of biomimetic tactile perception algorithms in robotics as well as non-invasive and invasive haptic feedback methods in prosthetics.

[1]  M. Srinivasan,et al.  Tactile detection of slip: surface microgeometry and peripheral neural codes. , 1990, Journal of neurophysiology.

[2]  T. Delbruck,et al.  > Replace This Line with Your Paper Identification Number (double-click Here to Edit) < 1 , 2022 .

[3]  Kwabena Boahen,et al.  A silicon retina that reproduces signals in the optic nerve , 2006, Journal of neural engineering.

[4]  Mark C. W. van Rossum,et al.  A Novel Spike Distance , 2001, Neural Computation.

[5]  J. Randall Flanagan,et al.  Coding and use of tactile signals from the fingertips in object manipulation tasks , 2009, Nature Reviews Neuroscience.

[6]  Fariborz Baghaei Naeini,et al.  A Novel Dynamic-Vision-Based Approach for Tactile Sensing Applications , 2020, IEEE Transactions on Instrumentation and Measurement.

[7]  Gordon Cheng,et al.  Event-based signaling for reducing required data rates and processing power in a large-scale artificial robotic skin , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[8]  Shih-Chii Liu,et al.  Neuromorphic sensory systems , 2010, Current Opinion in Neurobiology.

[9]  Luca Citi,et al.  Restoring Natural Sensory Feedback in Real-Time Bidirectional Hand Prostheses , 2014, Science Translational Medicine.

[10]  Nathan F. Lepora,et al.  Slip Detection With a Biomimetic Tactile Sensor , 2018, IEEE Robotics and Automation Letters.

[11]  Chiara Bartolozzi,et al.  Event-driven encoding of off-the-shelf tactile sensors for compression and latency optimisation for robotic skin , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[12]  Edward H. Adelson,et al.  Sensing and Recognizing Surface Textures Using a GelSight Sensor , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  M. Hollins,et al.  Evidence for the duplex theory of tactile texture perception , 2000, Perception & psychophysics.

[14]  Hannes P. Saal,et al.  Millisecond Precision Spike Timing Shapes Tactile Perception , 2012, The Journal of Neuroscience.

[15]  Jonathan Rossiter,et al.  The TacTip Family: Soft Optical Tactile Sensors with 3D-Printed Biomimetic Morphologies , 2018, Soft robotics.

[16]  Conor J. Houghton,et al.  A New Multineuron Spike Train Metric , 2008, Neural Computation.

[17]  Alberto Mazzoni,et al.  Neuromorphic Artificial Touch for Categorization of Naturalistic Textures , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[18]  T Mano,et al.  Mechanisms of fine-surface-texture discrimination in human tactile sensation. , 1999, The Journal of the Acoustical Society of America.

[19]  Tobi Delbrück,et al.  A 128$\times$ 128 120 dB 15 $\mu$s Latency Asynchronous Temporal Contrast Vision Sensor , 2008, IEEE Journal of Solid-State Circuits.

[20]  Gert Cauwenberghs,et al.  Neuromorphic Silicon Neuron Circuits , 2011, Front. Neurosci.

[21]  Danfei Xu,et al.  Tactile identification of objects using Bayesian exploration , 2013, 2013 IEEE International Conference on Robotics and Automation.

[22]  Giulio Sandini,et al.  Tactile Sensing—From Humans to Humanoids , 2010, IEEE Transactions on Robotics.

[23]  Nawid Jamali,et al.  Majority Voting: Material Classification by Tactile Sensing Using Surface Texture , 2011, IEEE Transactions on Robotics.

[24]  Jonathan Rossiter,et al.  Development of a tactile sensor based on biologically inspired edge encoding , 2009, 2009 International Conference on Advanced Robotics.

[25]  Nitish V. Thakor,et al.  Prosthesis with neuromorphic multilayered e-dermis perceives touch and pain , 2018, Science Robotics.

[26]  Nitish V. Thakor,et al.  A kilohertz kilotaxel tactile sensor array for investigating spatiotemporal features in neuromorphic touch , 2015, 2015 IEEE Biomedical Circuits and Systems Conference (BioCAS).

[27]  Francesco Giovacchini,et al.  Artificial Roughness Encoding with a Bio-inspired MEMS- based Tactile Sensor Array , 2009, Sensors.

[28]  Tobi Delbruck,et al.  A 240 × 180 130 dB 3 µs Latency Global Shutter Spatiotemporal Vision Sensor , 2014, IEEE Journal of Solid-State Circuits.