Synergy-Driven Performance Enhancement of Vision-Based 3D Hand Pose Reconstruction

In this work we propose, for the first time, to improve the performance of a Hand Pose Reconstruction (HPR) technique from RGBD camera data, which is affected by self-occlusions, leveraging upon postural synergy information, i.e., a priori information on how human most commonly use and shape their hands in everyday life tasks. More specifically, in our approach, we ignore joint angle values estimated with low confidence through a vision-based HPR technique and fuse synergistic information with such incomplete measures. Preliminary experiments are reported showing the effectiveness of the proposed integration.

[1]  Edoardo Battaglia,et al.  A Synergy-Based Optimally Designed Sensing Glove for Functional Grasp Recognition , 2016, Sensors.

[2]  Alessandro Tognetti,et al.  A Multi-Modal Sensing Glove for Human Manual-Interaction Studies , 2016 .

[3]  Bor-Shing Lin,et al.  Data glove embedded with 9-axis IMU and force sensing sensors for evaluation of hand function , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[4]  Matteo Bianchi,et al.  Synergy-based hand pose sensing: Reconstruction enhancement , 2012, Int. J. Robotics Res..

[5]  Paolo Dario,et al.  A Survey of Glove-Based Systems and Their Applications , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[6]  David Zeltzer,et al.  A survey of glove-based input , 1994, IEEE Computer Graphics and Applications.

[7]  Matteo Bianchi,et al.  On the use of postural synergies to improve human hand pose reconstruction , 2012, 2012 IEEE Haptics Symposium (HAPTICS).

[8]  Matteo Bianchi,et al.  Synergy-based optimal design of hand pose sensing , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[10]  Lale Akarun,et al.  Hand Pose Estimation and Hand Shape Classification Using Multi-layered Randomized Decision Forests , 2012, ECCV.

[11]  Matteo Bianchi,et al.  Synergy-based hand pose sensing: Optimal glove design , 2012, Int. J. Robotics Res..

[12]  Manuel G. Catalano,et al.  Adaptive synergies for the design and control of the Pisa/IIT SoftHand , 2014, Int. J. Robotics Res..

[13]  Antonis A. Argyros,et al.  Efficient model-based 3D tracking of hand articulations using Kinect , 2011, BMVC.

[14]  J. F. Soechting,et al.  Postural Hand Synergies for Tool Use , 1998, The Journal of Neuroscience.

[15]  Edoardo Battaglia,et al.  Exploiting hand kinematic synergies and wearable under-sensing for hand functional grasp recognition , 2014 .

[16]  Matteo Bianchi,et al.  Towards a synergy framework across neuroscience and robotics: Lessons learned and open questions. Reply to comments on: "Hand synergies: Integration of robotics and neuroscience for understanding the control of biological and artificial hands". , 2016, Physics of life reviews.

[17]  Jian Sun,et al.  Cascaded hand pose regression , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Matteo Bianchi,et al.  Hand synergies: Integration of robotics and neuroscience for understanding the control of biological and artificial hands. , 2016, Physics of life reviews.

[19]  E. van Lunteren,et al.  Improvement of diaphragm and limb muscle isotonic contractile performance by K+ channel blockade , 2010, Journal of NeuroEngineering and Rehabilitation.

[20]  Daniel M. Vogt,et al.  Embedded 3D Printing of Strain Sensors within Highly Stretchable Elastomers , 2014, Advanced materials.

[21]  Christian Cipriani,et al.  Principal components analysis based control of a multi-dof underactuated prosthetic hand , 2010, Journal of NeuroEngineering and Rehabilitation.