Measuring fingertip forces from camera images for random finger poses

Robust fingertip force detection from fingernail image is a critical strategy that can be applied in many areas. However, prior research fixed many variables that influence the finger color change. This paper analyzes the effect of the finger joint on the force detection in order to deal with the constrained finger position setting. A force estimator method is designed: a model to predict the fingertip force from finger joints measured from 2D cameras and 3 rectangular markers in cooperation with the fingernail images are trained. Then the error caused by the color changes of the joint bending can be avoided. This strategy is a significant step forward from a finger force estimator that requires tedious finger joint setting. The approach is evaluated experimentally. The result shows that it increases the accuracy over 10% for the force in conditions of the finger joint free movement. The estimator is used to demonstrate lifting and replacing objects with various weights.

[1]  H. Harry Asada,et al.  Measurement of finger posture and three-axis fingertip touch force using fingernail sensors , 2004, IEEE Transactions on Robotics and Automation.

[2]  Christian Osendorfer,et al.  Computing grip force and torque from finger nail images using Gaussian processes , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[3]  Alessandro Panarese,et al.  Human ability to discriminate direction of three-dimensional force stimuli applied to the finger pad. , 2011, Journal of neurophysiology.

[4]  Yu Sun,et al.  3-D force control on the human fingerpad using a magnetic levitation device for fingernail imaging calibration , 2009, World Haptics 2009 - Third Joint EuroHaptics conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.

[5]  Christian Osendorfer,et al.  Estimating finger grip force from an image of the hand using Convolutional Neural Networks and Gaussian processes , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[6]  Andriy Myronenko,et al.  Intensity-Based Image Registration by Minimizing Residual Complexity , 2010, IEEE Transactions on Medical Imaging.

[7]  Stephen A. Mascaro,et al.  Force prediction by fingernail imaging using active appearance models , 2013, 2013 World Haptics Conference (WHC).

[8]  J. Randall Flanagan,et al.  Sensorimotor control of manipulation , 2009 .

[9]  Stephen A. Mascaro,et al.  Estimation of Fingertip Force Direction With Computer Vision , 2009, IEEE Transactions on Robotics.

[10]  Zoubin Ghahramani,et al.  Sparse Gaussian Processes using Pseudo-inputs , 2005, NIPS.

[11]  Stephen A. Mascaro,et al.  3D force prediction using fingernail imaging with automated calibration , 2010, 2010 IEEE Haptics Symposium.

[12]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.