Grasp stability assessment through the fusion of proprioception and tactile signals using convolutional neural networks

The growing demand in industry for robots capable of performing a variety of tasks requires an increased capability in robotic grasping. Humans are adept at interacting with novel objects, a skill attributed primarily to tactile feedback in the form of exteroception and proprioception. This paper presents a novel way to incorporate exteroception and proprioception into grasp stability assessment: by using convolutional neural networks. This method improves upon the results of a unsupervised feature learning approach that used similar tactile feedback. 1000 different grasps on 100 objects were used to train and test the network. The network achieved an overall accuracy of 88.4% while predicting the failure class with an accuracy of 92.7%.

[1]  Danica Kragic,et al.  A probabilistic framework for task-oriented grasp stability assessment , 2013, 2013 IEEE International Conference on Robotics and Automation.

[2]  Ludmila I. Kuncheva,et al.  Switching between selection and fusion in combining classifiers: an experiment , 2002, IEEE Trans. Syst. Man Cybern. Part B.

[3]  T.-J. Tarn,et al.  Integration of task scheduling, action planning, and control in robotic manufacturing systems , 2000, Proceedings of the IEEE.

[4]  Alexander C. Berg,et al.  Combining multiple sources of knowledge in deep CNNs for action recognition , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[5]  Jan Peters,et al.  Can Modular Finger Control for In-Hand Object Stabilization be accomplished by Independent Tactile Feedback Control Laws? , 2016, ArXiv.

[6]  Axaykumar Rana.  Development of highly sensitive multimodal tactile sensor , 2015 .

[7]  Leon O. Chua,et al.  Automatic detection and tracking of moving image target with CNN‐UM via target probability fusion of multiple features , 2003, Int. J. Circuit Theory Appl..

[8]  Berthold Bäuml,et al.  Robust material classification with a tactile skin using deep learning , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[9]  Danica Kragic,et al.  A strategy for grasping unknown objects based on co-planarity and colour information , 2010, Robotics Auton. Syst..

[10]  Martin V. Butz,et al.  Self-supervised regrasping using spatio-temporal tactile features and reinforcement learning , 2016, IROS 2016.

[11]  Ye Zhang,et al.  A Sensitivity Analysis of (and Practitioners’ Guide to) Convolutional Neural Networks for Sentence Classification , 2015, IJCNLP.

[12]  Peter K. Allen,et al.  Learning grasp stability , 2012, 2012 IEEE International Conference on Robotics and Automation.

[13]  Vincent Duchaine,et al.  A highly sensitive multimodal capacitive tactile sensor , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[14]  Peter K. Allen,et al.  Stable grasping under pose uncertainty using tactile feedback , 2014, Auton. Robots.

[15]  Subhransu Maji,et al.  Bilinear CNN Models for Fine-Grained Visual Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[16]  Jingjing He,et al.  Neural network fusion strategies for identifying breast masses , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[17]  Danica Kragic,et al.  Grasping known objects with humanoid robots: A box-based approach , 2009, 2009 International Conference on Advanced Robotics.

[18]  Vincent Duchaine,et al.  Grasp stability assessment through unsupervised feature learning of tactile images , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[19]  Danica Kragic,et al.  Learning the tactile signatures of prototypical object parts for robust part-based grasping of novel objects , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[20]  Jimmy A. Jørgensen,et al.  Assessing Grasp Stability Based on Learning and Haptic Data , 2011, IEEE Transactions on Robotics.

[21]  Dong Yu,et al.  Exploring convolutional neural network structures and optimization techniques for speech recognition , 2013, INTERSPEECH.

[22]  D. Fox,et al.  Unsupervised Learning of Spatio-Temporal Features for Tactile Data , 2014 .

[23]  Tony Wong,et al.  Unsupervised feature learning for classifying dynamic tactile events using sparse coding , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[24]  Vitoantonio Bevilacqua,et al.  Comparison of data-merging methods with SVM attribute selection and classification in breast cancer gene expression , 2011, BMC Bioinformatics.

[25]  Danica Kragic,et al.  Learning grasp stability based on tactile data and HMMs , 2010, 19th International Symposium in Robot and Human Interactive Communication.