Grasp Stability Prediction with Time Series Data Based on STFT and LSTM

With an increasing demand for robots, robotic grasping will has a more important role in future applications. This paper takes grasp stability prediction as the key technology for grasping and tries to solve the problem with time series data inputs including the force and pressure data. Widely applied to more fields to predict unstable grasping with time series data, algorithms can significantly promote the application of artificial intelligence in traditional industries. This research investigates models that combine short-time Fourier transform (STFT) and long short-term memory (LSTM) and then tested generalizability with dexterous hand and suction cup gripper. The experiments suggest good results for grasp stability prediction with the force data and the generalized results in the pressure data. Among the 4 models, (Data + STFT) & LSTM delivers the best performance. We plan to perform more work on grasp stability prediction, generalize the findings to different types of sensors, and apply the grasp stability prediction in more grasping use cases in real life.

[1]  Russ Tedrake,et al.  Soft-bubble: A highly compliant dense geometry tactile sensor for robot manipulation , 2019, 2019 2nd IEEE International Conference on Soft Robotics (RoboSoft).

[2]  Jan Peters,et al.  Grip Stabilization of Novel Objects Using Slip Prediction , 2018, IEEE Transactions on Haptics.

[3]  Guanglin Li,et al.  A piezoelectret-based approach for touching and slipping detection in robotic hands , 2015, 2015 IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems (CYBER).

[4]  Che-Lun Hung,et al.  Evaluation of Deep Learning Neural Networks for Surface Roughness Prediction Using Vibration Signal Analysis , 2019, Applied Sciences.

[5]  Alberto Cavallo,et al.  Slipping detection and avoidance based on Kalman filter , 2014 .

[6]  Siti Anom Ahmad,et al.  Slip detection with accelerometer and tactile sensors in a robotic hand model , 2015 .

[7]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[8]  Hussein A. Abdullah,et al.  A Slip Detection and Correction Strategy for Precision Robot Grasping , 2016, IEEE/ASME Transactions on Mechatronics.

[9]  Fernando Torres Medina,et al.  Learning Spatio Temporal Tactile Features with a ConvLSTM for the Direction Of Slip Detection , 2019, Sensors.

[10]  Yang Yang,et al.  FingerVision Tactile Sensor Design and Slip Detection Using Convolutional LSTM Network , 2018, ArXiv.

[11]  Mark R. Cutkosky,et al.  Slip classification for dynamic tactile array sensors , 2016, Int. J. Robotics Res..

[12]  Bin Fang,et al.  Multimodal grasp data set: A novel visual–tactile data set for robotic manipulation , 2019, International Journal of Advanced Robotic Systems.

[13]  Edward H. Adelson,et al.  Measurement of shear and slip with a GelSight tactile sensor , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[14]  N. Altman.  An Introduction to Kernel and Nearest-Neighbor Nonparametric Regression , 1992 .

[15]  Jonathan Becedas,et al.  Micro-Vibration-Based Slip Detection in Tactile Force Sensors , 2014, Sensors.

[16]  Tao Wang,et al.  Eagle Shoal: A new designed modular tactile sensing dexterous hand for domestic service robots , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[17]  Andrew W. Senior,et al.  Long short-term memory recurrent neural network architectures for large scale acoustic modeling , 2014, INTERSPEECH.

[18]  Wolfgang Reif,et al.  Environment-aware proximity detection with capacitive sensors for human-robot-interaction , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[19]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[20]  Jin Jiang,et al.  Time-frequency feature representation using energy concentration: An overview of recent advances , 2009, Digit. Signal Process..

[21]  J. Tukey,et al.  An algorithm for the machine calculation of complex Fourier series , 1965 .

[22]  Jan Peters,et al.  Stabilizing novel objects by learning to predict tactile slip , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[23]  Edward H. Adelson,et al.  Improved GelSight tactile sensor for measuring geometry and slip , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[24]  Andreas Christmann,et al.  Support vector machines , 2008, Data Mining and Knowledge Discovery Handbook.

[25]  Yadong Wang,et al.  Prediction for High Risk Clinical Symptoms of Epilepsy Based on Deep Learning Algorithm , 2018, IEEE Access.

[26]  Jianhua Li,et al.  Slip Detection with Combined Tactile and Visual Information , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[27]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[28]  Hua Deng,et al.  Initial Slip Detection and Its Application in Biomimetic Robotic Hands , 2016, IEEE Sensors Journal.

[29]  Pat Langley,et al.  Estimating Continuous Distributions in Bayesian Classifiers , 1995, UAI.

[30]  Karl Van Wyk,et al.  Calibration and Analysis of Tactile Sensors as Slip Detectors , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[31]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[32]  Somer M. Nacy,et al.  A Novel Fingertip Design for Slip Detection Under Dynamic Load Conditions , 2014 .