Which LSTM Type is Better for Interaction Force Estimation?

Tactile, one of the five senses classified into the main senses of human, is the first sensation developed when human beings are formed. The tactile includes various information such as pressure, temperature, and texture of objects, it also helps the person to interact with the surrounding environment. One of the tactile information, the pressure is used in various fields such as medical, beauty, mobile devices and so on. However, humans can perceive the real world with multi-modal senses such as sound, vision. In this paper, we study interaction force estimation using haptic sensor and video. Interact ion force estimation through video analysis is one of a cross-modal approach that is applicable such as a software haptic feedback method that can give haptic feedback to remote control of robot arm by predicting interaction force even in absence of haptic sensor. we compare and analyze three types of a deep neural network to predict the interaction force. In particular, the best model for the stacking structure of CNN and LSTM is selected through a detailed analysis of how the structure change of LSTM affects the video regression problem. The average error of the best suit model is MSE 0.1306, RMSE 0.2740, MAE 0.1878.

[1]  M. Fulkerson,et al.  The First Sense: A Philosophical Study of Human Touch , 2013 .

[2]  Antonio Visioli,et al.  A virtual force sensor for interaction tasks with conventional industrial robots , 2018 .

[3]  Andrew Owens,et al.  The Feeling of Success: Does Touch Sensing Help Predict Grasp Outcomes? , 2017, CoRL.

[4]  B. Lorena Villarreal,et al.  Bioinspired Smell Sensor: Nostril Model and Design , 2016, IEEE/ASME Transactions on Mechatronics.

[5]  A. Okamura Haptic feedback in robot-assisted minimally invasive surgery , 2009, Current opinion in urology.

[6]  Mark R. Cutkosky,et al.  Force and Tactile Sensors , 2008, Springer Handbook of Robotics.

[7]  Ming Yang,et al.  3D Convolutional Neural Networks for Human Action Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Seiji Chonan,et al.  Active haptic sensation for monitoring skin conditions , 2005 .

[9]  Edward H. Adelson,et al.  Connecting Look and Feel: Associating the Visual and Tactile Properties of Physical Materials , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Yann LeCun,et al.  A Closer Look at Spatiotemporal Convolutions for Action Recognition , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[11]  Chang Liu,et al.  Recent Progress in Technologies for Tactile Sensors , 2018, Sensors.

[12]  Ying Tan,et al.  Interaction Force Estimation Using Extended State Observers: An Application to Impedance-Based Assistive and Rehabilitation Robotics , 2019, IEEE Robotics and Automation Letters.

[13]  Trevor Darrell,et al.  Long-term recurrent convolutional networks for visual recognition and description , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  R. Klatzky,et al.  Hand movements: A window into haptic object recognition , 1987, Cognitive Psychology.

[15]  Tao Mei,et al.  Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[16]  King-Sun Fu,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence Publication Information , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Andrew Zisserman,et al.  Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Sven Bambach A Survey on Recent Advances of Computer Vision Algorithms for Egocentric Video , 2015, ArXiv.

[19]  Antonio Torralba,et al.  Connecting Touch and Vision via Cross-Modal Prediction , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Wonjun Hwang,et al.  Interaction Force Estimation Using Camera and Electrical Current Without Force/Torque Sensor , 2018, IEEE Sensors Journal.

[21]  A. B. Huber,et al.  How axons see their way--axonal guidance in the visual system. , 2008, Frontiers in bioscience : a journal and virtual library.

[22]  R. Klatzky,et al.  Haptic perception: A tutorial , 2009, Attention, perception & psychophysics.

[23]  Wonjun Hwang,et al.  Inferring Interaction Force from Visual Information without Using Physical Force Sensors , 2017, Sensors.