Active Multi-Object Exploration and Recognition via Tactile Whiskers

Robotic exploration under uncertain environments is challenging when optical information is not available. In this paper, we propose an autonomous solution of exploring an unknown task space based on tactile sensing alone. We first designed a whisker sensor based on MEMS barometer devices. This sensor can acquire contact information by interacting with the environment non-intrusively. This sensor is accompanied by a planning technique to generate exploration trajectories by using mere tactile perception. This technique relies on a hybrid policy for tactile exploration, which includes a proactive informative path planner for object searching, and a reactive Hopf oscillator for contour tracing. Results indicate that the hybrid exploration policy can increase the efficiency of object discovery. Last, scene understanding was facilitated by segmenting objects and classification. A classifier was developed to recognize the object categories based on the geometric features collected by the whisker sensor. Such an approach demonstrates the whisker sensor, together with the tactile intelligence, can provide sufficiently discriminative features to distinguish objects.

[1]  Ian D. Reid,et al.  On the comparison of uncertainty criteria for active SLAM , 2012, 2012 IEEE International Conference on Robotics and Automation.

[2]  Edward H. Adelson,et al.  Sensing and Recognizing Surface Textures Using a GelSight Sensor , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Ludovic Righetti,et al.  A Dynamical Systems Approach to Learning: A Frequency-Adaptive Hopper Robot , 2005, ECAL.

[4]  Cheng Fang,et al.  Coverage path planning for harbour seabed surveys using an autonomous underwater vehicle , 2010, OCEANS'10 IEEE SYDNEY.

[5]  Hiroshi Yokoi,et al.  An active artificial whisker array for texture discrimination , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[6]  Tianmiao Wang,et al.  Parameter Synthesis of Coupled Nonlinear Oscillators for CPG-Based Robotic Locomotion , 2014, IEEE Transactions on Industrial Electronics.

[7]  Wojciech Jaskowski,et al.  Model-Based Active Exploration , 2018, ICML.

[8]  Jason P Carey,et al.  The effect of biomechanical variables on force sensitive resistor error: Implications for calibration and improved accuracy. , 2016, Journal of biomechanics.

[9]  Kuan-Ting Yu,et al.  Tactile SLAM: Real-time inference of shape and pose from planar pushing , 2020, 2021 IEEE International Conference on Robotics and Automation (ICRA).

[10]  Andrew Zisserman,et al.  A Statistical Approach to Texture Classification from Single Images , 2004, International Journal of Computer Vision.

[11]  Claudio Zito,et al.  GPAtlasRRT: A Local Tactile Exploration Planner for Recovering the Shape of Novel Objects , 2018, Int. J. Humanoid Robotics.

[12]  Wenchuan Jia,et al.  Hopf oscillator based adaptive locomotion control for a bionic quadruped robot , 2017, 2017 IEEE International Conference on Mechatronics and Automation (ICMA).

[13]  Kurt E. Weaver,et al.  Attention and Sensory Interactions within the Occipital Cortex in the Early Blind: An fMRI Study , 2007, Journal of Cognitive Neuroscience.

[14]  Rong Zheng,et al.  Informative Path Planning for Mobile Sensing with Reinforcement Learning , 2020, IEEE INFOCOM 2020 - IEEE Conference on Computer Communications.

[15]  Fumio Arisaka,et al.  The tail structure of bacteriophage T4 and its mechanism of contraction , 2005, Nature Structural &Molecular Biology.

[16]  Takayuki Okatani,et al.  3D reconstruction of reflective surface on reflection type tactile sensor using constraints of geometrical optics , 2010, Proceedings of SICE Annual Conference 2010.

[17]  Marc Toussaint,et al.  Active Multi-Contact Continuous Tactile Exploration with Gaussian Process Differential Entropy , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[18]  M. Hartmann,et al.  Ergodic Exploration Using Binary Sensing for Nonparametric Shape Estimation , 2017, IEEE Robotics and Automation Letters.

[19]  Joseph H. Solomon,et al.  Biomechanics: Robotic whiskers used to sense features , 2006, Nature.

[20]  E. Vázquez,et al.  Convergence properties of the expected improvement algorithm with fixed mean and covariance functions , 2007, 0712.3744.

[21]  S.S.M. Wang,et al.  Sensors for computer controlled mechanical assembly , 1978 .

[22]  Bohan Wu,et al.  MAT: Multi-Fingered Adaptive Tactile Grasping via Deep Reinforcement Learning , 2019, CoRL.

[23]  M. Brecht,et al.  Functional architecture of the mystacial vibrissae , 1997, Behavioural Brain Research.

[24]  Roland Siegwart,et al.  Adaptive continuous‐space informative path planning for online environmental monitoring , 2017, J. Field Robotics.

[25]  Tamim Asfour,et al.  Active Tactile Exploration Based on Cost-Aware Information Gain Maximization , 2018, Int. J. Humanoid Robotics.

[26]  Juan Pablo Wachs,et al.  Image Exploration Procedure Classification with Spike-timing Neural Network for the Blind , 2018, 2018 24th International Conference on Pattern Recognition (ICPR).

[27]  Mathew H. Evans,et al.  Tactile SLAM with a biomimetic whiskered robot , 2012, 2012 IEEE International Conference on Robotics and Automation.

[28]  Kaspar Althoefer,et al.  Novel Tactile-SIFT Descriptor for Object Shape Recognition , 2015, IEEE Sensors Journal.

[29]  William Deer,et al.  Lightweight Whiskers for Contact, Pre-Contact, and Fluid Velocity Sensing , 2019, IEEE Robotics and Automation Letters.

[30]  Giorgio Metta,et al.  A Flexible and Robust Large Scale Capacitive Tactile System for Robots , 2013, IEEE Sensors Journal.

[31]  Phillip J. McKerrow,et al.  Introduction to robotics , 1991 .

[32]  Gaurav S. Sukhatme,et al.  Informative path planning for an autonomous underwater vehicle , 2010, 2010 IEEE International Conference on Robotics and Automation.

[33]  David G. Kirkpatrick,et al.  On the shape of a set of points in the plane , 1983, IEEE Trans. Inf. Theory.

[34]  Peter Englert,et al.  Active learning with query paths for tactile object shape exploration , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[35]  Roland Siegwart,et al.  An Efficient Sampling-Based Method for Online Informative Path Planning in Unknown Environments , 2020, IEEE Robotics and Automation Letters.

[36]  Markus Vincze,et al.  3DNet: Large-scale object class recognition from CAD models , 2012, 2012 IEEE International Conference on Robotics and Automation.

[37]  Lorenzo Rosasco,et al.  Active perception: Building objects' models using tactile exploration , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).

[38]  Tony J. Prescott,et al.  Vibrissal behavior and function , 2011, Scholarpedia.

[39]  A. Ahl The role of vibrissae in behavior: A status review , 1986, Veterinary Research Communications.

[40]  Kshitij Tiwari,et al.  ViTa-SLAM: A Bio-inspired Visuo-Tactile SLAM for Navigation while Interacting with Aliased Environments , 2019, 2019 IEEE International Conference on Cyborg and Bionic Systems (CBS).

[41]  Marc Toussaint,et al.  Uncertainty aware grasping and tactile exploration , 2013, 2013 IEEE International Conference on Robotics and Automation.

[42]  Veronika Yordanova,et al.  Coverage Path Planning With Track Spacing Adaptation for Autonomous Underwater Vehicles , 2020, IEEE Robotics and Automation Letters.

[43]  Gerald E. Loeb,et al.  Bayesian Exploration for Intelligent Identification of Textures , 2012, Front. Neurorobot..

[44]  Raia Hadsell,et al.  From Pixels to Percepts: Highly Robust Edge Perception and Contour Following Using Deep Learning and an Optical Biomimetic Tactile Sensor , 2018, IEEE Robotics and Automation Letters.

[45]  Helge J. Ritter,et al.  A Probabilistic Approach to Tactile Shape Reconstruction , 2011, IEEE Transactions on Robotics.

[46]  Mitra J. Hartmann,et al.  c ○ 2001 Kluwer Academic Publishers. Manufactured in The Netherlands. Active Sensing Capabilities of the Rat Whisker System , 2022 .

[47]  Seiji Akita,et al.  Fully printed, highly sensitive multifunctional artificial electronic whisker arrays integrated with strain and temperature sensors. , 2014, ACS nano.

[48]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[49]  Tony J. Dodd,et al.  Feeling the Shape: Active Exploration Behaviors for Object Recognition With a Robotic Hand , 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[50]  Andreas Krause,et al.  Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting , 2009, IEEE Transactions on Information Theory.

[51]  Kaspar Althoefer,et al.  Implementing Tactile and Proximity Sensing for Crack Detection , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[52]  Gordon Cheng,et al.  Tactile-based active object discrimination and target object search in an unknown workspace , 2018, Autonomous Robots.

[53]  M. Ani Hsieh,et al.  A triangle histogram for object classification by tactile sensing , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[54]  B. Hassard,et al.  Theory and applications of Hopf bifurcation , 1981 .

[55]  S. LaValle Rapidly-exploring random trees : a new tool for path planning , 1998 .

[56]  Frank Kirchner,et al.  Haptic Object Recognition in Underwater and Deep-sea Environments , 2015, J. Field Robotics.

[57]  Nigel H. Lovell,et al.  A review of tactile sensing technologies with applications in biomedical engineering , 2012 .

[58]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[59]  Emilio Frazzoli,et al.  Sampling-based algorithms for optimal motion planning , 2011, Int. J. Robotics Res..

[60]  Wei Chen,et al.  Tactile Sensors for Friction Estimation and Incipient Slip Detection—Toward Dexterous Robotic Manipulation: A Review , 2018, IEEE Sensors Journal.

[61]  Shaheen Ahmad,et al.  Shape recovery from robot contour-tracking with force feedback , 1990, Proceedings., IEEE International Conference on Robotics and Automation.

[62]  S. Hirose,et al.  Titan III, A quadruped walking vehicle , 1985 .

[63]  Gordon Cheng,et al.  Tactile-based manipulation of deformable objects with dynamic center of mass , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).

[64]  Kostas Daniilidis,et al.  Active end-effector pose selection for tactile object recognition through Monte Carlo tree search , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[65]  Francesco Zanichelli,et al.  On the robustness of haptic object recognition based on polyhedral shape representations , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[66]  Salah Sukkarieh,et al.  A Gaussian process-based RRT planner for the exploration of an unknown and cluttered environment with a UAV , 2013, Adv. Robotics.

[67]  Ravinder Dahiya,et al.  Robotic Tactile Perception of Object Properties: A Review , 2017, ArXiv.

[68]  E. Adelson,et al.  Digger Finger: GelSight Tactile Sensor for Object Identification Inside Granular Media , 2021, ISER.

[69]  Allison M. Okamura,et al.  Methods to Segment Hard Inclusions in Soft Tissue During Autonomous Robotic Palpation , 2015, IEEE Transactions on Robotics.

[70]  T. Prescott,et al.  Biomimetic vibrissal sensing for robots , 2011, Philosophical Transactions of the Royal Society B: Biological Sciences.

[71]  Gregory D. Hager,et al.  Tactile-Object Recognition From Appearance Information , 2011, IEEE Transactions on Robotics.

[72]  Jan Baumbach,et al.  Comparing the performance of biomedical clustering methods , 2015, Nature Methods.

[73]  H. Ritter,et al.  Barometer-based Tactile Skin for Anthropomorphic Robot Hand , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[74]  Mark Lee,et al.  Review Article Tactile sensing for mechatronics—a state of the art survey , 1999 .