Object recognition using tactile array sensors

In this dissertation, we explore the use of tactile force sensors to understand properties of the surfaces of objects they are pressed against. We develop a model of such sensors as imaging devices, which facilitates the use of techniques from computer vision and image processing with the “tactile images” they provide. The goal is object recognition, and three approaches are presented for distinguishing amongst objects from a previouslyencountered set: The first approach is entirely geometric in nature and borrows ideas from mobile robotics. Object surfaces are modeled as occupancy grid maps, and recognition is posed as a problem of localizing the sensor within one of these maps. The second approach applies techniques from computer vision to characterize the tactile appearance of objects. Finally, the third approach combines information from the previous two to describe the spatially-varying appearance of objects’ surfaces. These methods are evaluated in experiments using a physical tactile force sensing system and in a variety of simulations based on our imaging model, and they all exhibit strong performance in their respective domains.

[1]  Ronald S. Fearing,et al.  Tactile sensing for shape interpretation , 1990 .

[2]  Antonio Criminisi,et al.  Object Class Recognition at a Glance , 2006 .

[3]  Ronald S. Fearing,et al.  Using a cylindrical tactile sensor for determining curvature , 1988, Proceedings. 1988 IEEE International Conference on Robotics and Automation.

[4]  Chin Seng Chua,et al.  Point Signatures: A New Representation for 3D Object Recognition , 1997, International Journal of Computer Vision.

[5]  Jitendra Malik,et al.  Shape matching and object recognition using shape contexts , 2010, 2010 3rd International Conference on Computer Science and Information Technology.

[6]  Frédéric Jurie,et al.  Sampling Strategies for Bag-of-Features Image Classification , 2006, ECCV.

[7]  A. Ng,et al.  Touch Based Perception for Object Manipulation , 2007 .

[8]  Cordelia Schmid,et al.  A Performance Evaluation of Local Descriptors , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Howie Choset,et al.  Principles of Robot Motion: Theory, Algorithms, and Implementation ERRATA!!!! 1 , 2007 .

[10]  A. R. Muir,et al.  The structure and function of a slowly adapting touch corpuscle in hairy skin , 1969, The Journal of physiology.

[11]  Thomas Parisini,et al.  Shape from touch by a neural net , 1992, Proceedings 1992 IEEE International Conference on Robotics and Automation.

[12]  R. S. Fearing,et al.  Tactile Sensing Mechanisms , 1990, Int. J. Robotics Res..

[13]  R. Brent Gillespie,et al.  The haptic probe: mechanized haptic exploration and automated modeling , 2003, 11th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2003. HAPTICS 2003. Proceedings..

[14]  G. Metta,et al.  Piezoelectric oxide semiconductor field effect transistor touch sensing devices , 2009 .

[15]  Vittorio Ferrari,et al.  Learning Object Categories with Generic Knowledge , 2011, VISAPP.

[16]  Wolfram Burgard,et al.  Monte Carlo Localization: Efficient Position Estimation for Mobile Robots , 1999, AAAI/IAAI.

[17]  Sergei Vassilvitskii,et al.  k-means++: the advantages of careful seeding , 2007, SODA '07.

[18]  Reid G. Simmons,et al.  Probabilistic Robot Navigation in Partially Observable Environments , 1995, IJCAI.

[19]  Dinesh K. Pai,et al.  ACME, A Telerobotic Active Measurement Facility , 1999, ISER.

[20]  John S. Bay.  Tactile shape sensing via single- and multifingered hands , 1989, Proceedings, 1989 International Conference on Robotics and Automation.

[21]  Wolfram Burgard,et al.  Probabilistic Robotics (Intelligent Robotics and Autonomous Agents) , 2005 .

[22]  Andrew E. Johnson,et al.  Spin-Images: A Representation for 3-D Surface Matching , 1997 .

[23]  Francesco Zanichelli,et al.  On the robustness of haptic object recognition based on polyhedral shape representations , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[24]  Javad Dargahi,et al.  Advances in tactile sensors design/manufacturing and its impact on robotics applications - a review , 2005, Ind. Robot.

[25]  Joost van de Weijer,et al.  Fast Anisotropic Gauss Filtering , 2002, ECCV.

[26]  Andrea Vedaldi,et al.  Vlfeat: an open and portable library of computer vision algorithms , 2010, ACM Multimedia.

[27]  Wolfram Burgard,et al.  Monte Carlo localization for mobile robots , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[28]  Wolfram Burgard,et al.  An experimental comparison of localization methods , 1998, Proceedings. 1998 IEEE/RSJ International Conference on Intelligent Robots and Systems. Innovations in Theory, Practice and Applications (Cat. No.98CH36190).

[29]  Robert W. Platt,et al.  Inferring hand-object configuration directly from tactile data , 2010 .

[30]  Viii Supervisor.  Sonar-Based Real-World Mapping and Navigation , 2001 .

[31]  Sunil R. Das,et al.  Robotic tactile recognition of pseudorandom encoded objects , 2004, IEEE Transactions on Instrumentation and Measurement.

[32]  Ruzena Bajcsy,et al.  TACTILE INFORMATION PROCESSING - THE BOTTOM UP APPROACH. , 1984 .

[33]  W. Daniel Hillis,et al.  A High-Resolution Imaging Touch Sensor , 1982 .

[34]  Hans P. Moravec.  Sensor Fusion in Certainty Grids for Mobile Robots , 1988, AI Mag..

[35]  Andrew E. Johnson,et al.  Recognizing objects by matching oriented points , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[36]  Christoph H. Lampert,et al.  Efficient Subwindow Search: A Branch and Bound Framework for Object Localization , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[38]  Ming-Kuei Hu,et al.  Visual pattern recognition by moment invariants , 1962, IRE Trans. Inf. Theory.

[39]  Cordelia Schmid,et al.  Constructing models for content-based image retrieval , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[40]  Susan J. Lederman,et al.  Haptic Classification of Facial Identity in 2D Displays: Configural versus Feature-Based Processing , 2010, IEEE Transactions on Haptics.

[41]  A. Caiti,et al.  Shape estimation with tactile sensors: a radial basis functions approach , 1992, [1992] Proceedings of the 31st IEEE Conference on Decision and Control.

[42]  J. S. Son,et al.  Tactile sensing and stiffness control with multifingered hands , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[43]  Eugenio Faldella,et al.  A neural approach to robotic haptic recognition of 3-D objects based on a Kohonen self-organizing feature map , 1997, IEEE Trans. Ind. Electron..

[44]  Joseph M. Romano,et al.  Haptography: Capturing and Recreating the Rich Feel of Real Surfaces , 2009, ISRR.

[45]  Daniel Cohen-Or,et al.  Salient geometric features for partial shape matching and similarity , 2006, TOGS.

[46]  Heinz Wörn,et al.  Haptic Object Exploration Using Attention Cubes , 2010, KI.

[47]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[48]  G. Chirikjian,et al.  Metrics on Motion and Deformation of Solid Models , 1998 .

[49]  Andrew Zisserman,et al.  Texture classification: are filter banks necessary? , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[50]  Ruzena Bajcsy,et al.  Visually-guided haptic object recognition , 1987 .

[51]  R. Klatzky,et al.  Hand movements: A window into haptic object recognition , 1987, Cognitive Psychology.

[52]  W. Eric L. Grimson,et al.  Model-based recognition and localization from tactile data , 1984, ICRA.

[53]  Gregory D. Hager,et al.  Characterization and simulation of tactile sensors , 2010, 2010 IEEE Haptics Symposium.

[54]  Gregory D. Hager,et al.  Tactile-Object Recognition From Appearance Information , 2011, IEEE Transactions on Robotics.

[55]  Heinz Wörn,et al.  Development of a flexible tactile sensor system for a humanoid robot , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[56]  K. O. Johnson,et al.  Tactile spatial resolution. III. A continuum mechanics model of skin predicting mechanoreceptor responses to bars, edges, and gratings. , 1981, Journal of neurophysiology.

[57]  R. E. Ellis,et al.  Singular-value and finite-element analysis of tactile shape recognition , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[58]  Iasonas Kokkinos,et al.  Scale-invariant heat kernel signatures for non-rigid shape recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[59]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[60]  Allison M. Okamura,et al.  Haptic exploration of unknown objects , 2000 .

[61]  Frédéric Jurie,et al.  Creating efficient codebooks for visual recognition , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[62]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[63]  Andrew Zisserman,et al.  Scene Classification Via pLSA , 2006, ECCV.

[64]  Allison M. Okamura,et al.  Haptic exploration of fine surface features , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).