Knock-Knock: Acoustic object recognition by using stacked denoising autoencoders

This paper presents a successful application of deep learning for object recognition based on acoustic data. The shortcomings of previously employed approaches where handcrafted features describing the acoustic data are being used, include limiting the capability of the found representation to be widely applicable and facing the risk of capturing only insignificant characteristics for a task. In contrast, there is no need to define the feature representation format when using multilayer/deep learning architecture methods: features can be learned from raw sensor data without defining discriminative characteristics a-priori. In this paper, stacked denoising autoencoders are applied to train a deep learning model. Knocking each object in our test set 120 times with a marker pen to obtain the auditory data, thirty different objects were successfully classified in our experiment and each object was knocked 120 times by a marker pen to obtain the auditory data. By employing the proposed deep learning framework, a high accuracy of 91.50% was achieved. A traditional method using handcrafted features with a shallow classifier was taken as a benchmark and the attained recognition rate was only 58.22%. Interestingly, a recognition rate of 82.00% was achieved when using a shallow classifier with raw acoustic data as input. In addition, we could show that the time taken to classify one object using deep learning was far less (by a factor of more than 6) than utilizing the traditional method. It was also explored how different model parameters in our deep architecture affect the recognition performance.

[1]  Lorenzo Natale,et al.  Tapping into Touch , 2005 .

[2]  Dinesh K. Pai,et al.  Perception of Material from Contact Sounds , 2000, Presence: Teleoperators & Virtual Environments.

[3]  Jivko Sinapov,et al.  Interactive learning of the acoustic properties of household objects , 2009, 2009 IEEE International Conference on Robotics and Automation.

[4]  Edward H. Adelson,et al.  Exploring features in a Bayesian framework for material recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[5]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[6]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[8]  Sven Behnke,et al.  RGB-D object recognition and pose estimation based on pre-trained convolutional neural network features , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[9]  Arnold W. M. Smeulders,et al.  Color-based object recognition , 1997, Pattern Recognit..

[10]  L.A. Jones,et al.  Material identification using real and simulated thermal cues , 2004, The 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[11]  Bayya Yegnanarayana,et al.  Combining evidence from residual phase and MFCC features for speaker recognition , 2006, IEEE Signal Processing Letters.

[12]  Kaspar Althoefer,et al.  Surface material recognition through haptic exploration using an intelligent contact sensing finger , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[13]  Shree K. Nayar,et al.  Reflectance and texture of real-world surfaces , 1999, TOGS.

[14]  Kristin J. Dana,et al.  3D Texture Recognition Using Bidirectional Feature Histograms , 2004, International Journal of Computer Vision.

[15]  Yoshua Bengio,et al.  Practical Recommendations for Gradient-Based Training of Deep Architectures , 2012, Neural Networks: Tricks of the Trade.

[16]  Eric Krotkov,et al.  Object classification from analysis of impact acoustics , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[17]  Shree K. Nayar,et al.  Reflectance and Texture of Real-World Surfaces Authors , 1997, CVPR 1997.

[18]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[19]  Nawid Jamali,et al.  Majority Voting: Material Classification by Tactile Sensing Using Surface Texture , 2011, IEEE Transactions on Robotics.

[20]  Kaspar Althoefer,et al.  Iterative Closest Labeled Point for tactile object shape recognition , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[21]  Eric Krotkov,et al.  Robotic Perception of Material: Experiments with Shape-Invariant Acoustic Measures of Material Type , 1995, ISER.

[22]  Masahiro Ohka,et al.  Low force control scheme for object hardness distinction in robot manipulation based on tactile sensing , 2008, 2008 IEEE International Conference on Robotics and Automation.

[23]  Wolfram Burgard,et al.  Learning the elasticity parameters of deformable objects with a manipulation robot , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[24]  Afsar Saranli,et al.  Acoustic surface perception from naturally occurring step sounds of a dexterous hexapod robot , 2013 .

[25]  Shigeki Sugano,et al.  Tactile object recognition using deep learning and dropout , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.

[26]  Jitendra Malik,et al.  Representing and Recognizing the Visual Appearance of Materials using Three-dimensional Textons , 2001, International Journal of Computer Vision.

[27]  Andrew Zisserman,et al.  A Statistical Approach to Texture Classification from Single Images , 2005, International Journal of Computer Vision.

[28]  Davide Rocchesso,et al.  Size, shape, and material properties of sound models , 2003 .

[29]  Connor Schenck,et al.  Interactive object recognition using proprioceptive and auditory feedback , 2011, Int. J. Robotics Res..