What's in the container? Classifying object contents from vision and touch

Robots operating in household environments need to interact with food containers of different types. Whether a container is filled with milk, juice, yogurt or coffee may affect the way robots grasp and manipulate the container. In this paper, we concentrate on the problem of identifying what kind of content is in a container based on tactile and/or visual feedback in combination with grasping. In particular, we investigate the benefits of using unimodal (visual or tactile) or bimodal (visual-tactile) sensory data for this purpose. We direct our study toward cardboard containers with liquid or solid content or being empty. The motivation for using grasping rather than shaking is that we want to investigate the content prior to applying manipulation actions to a container. Our results show that we achieve comparable classification rates with unimodal data and that the visual and tactile data are complimentary.

[1]  Peter K. AllenDepartment,et al.  CLOSED-LOOP VISUAL GRASPING AND MANIPULATIONBillibon , 1996 .

[2]  Gordon Cheng,et al.  Making Object Learning and Recognition an Active Process , 2008, Int. J. Humanoid Robotics.

[3]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[4]  Oussama Khatib,et al.  Global Localization of Objects via Touch , 2011, IEEE Transactions on Robotics.

[5]  Dieter Fox,et al.  Manipulator and object tracking for in-hand 3D object modeling , 2011, Int. J. Robotics Res..

[6]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[7]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[8]  M. Heller.  Visual and tactual texture perception: Intersensory cooperation , 1982, Perception & psychophysics.

[9]  Wolfram Burgard,et al.  Object identification with tactile sensors using bag-of-features , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[10]  F. Massey.  The Kolmogorov-Smirnov Test for Goodness of Fit , 1951 .

[11]  Trevor Darrell,et al.  Using robotic exploratory procedures to learn the meaning of haptic adjectives , 2013, 2013 IEEE International Conference on Robotics and Automation.

[12]  Danica Kragic,et al.  Enhancing visual perception of shape through tactile glances , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[13]  Danica Kragic,et al.  Real-time tracking meets online grasp planning , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[14]  Nawid Jamali,et al.  Majority Voting: Material Classification by Tactile Sensing Using Surface Texture , 2011, IEEE Transactions on Robotics.

[15]  Javier Díaz,et al.  Real-Time Model-Based Rigid Object Pose Estimation and Tracking Combining Dense and Sparse Visual Cues , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Hideko F. Norman,et al.  The visual and haptic perception of natural object shape , 2010 .

[17]  Peter K. Allen,et al.  Integrating Vision and Touch for Object Recognition Tasks , 1988, Int. J. Robotics Res..

[18]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[19]  Jürgen Sturm,et al.  Tactile object class and internal state recognition for mobile manipulation , 2010, 2010 IEEE International Conference on Robotics and Automation.

[20]  Gregory D. Hager,et al.  Tactile-Object Recognition From Appearance Information , 2011, IEEE Transactions on Robotics.

[21]  Jonathon Shlens,et al.  A Tutorial on Principal Component Analysis , 2014, ArXiv.

[22]  Marc Toussaint,et al.  Gaussian process implicit surfaces for shape estimation and grasping , 2011, 2011 IEEE International Conference on Robotics and Automation.

[23]  John Platt,et al.  Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods , 1999 .

[24]  Danica Kragic,et al.  Data-Driven Grasp Synthesis—A Survey , 2013, IEEE Transactions on Robotics.

[25]  Helge J. Ritter,et al.  A Probabilistic Approach to Tactile Shape Reconstruction , 2011, IEEE Transactions on Robotics.