Robust hand pose estimation using visual sensor in IoT environment

In Internet of Things (IoT) environments, visual sensors with good performance have been used to create and apply various kinds of image data. Particularly, in the field of human–computer interaction, the image sensor interface using human hands is applicable to sign language recognition, games, object operation in virtual reality, and remote surgery. With the popularization of depth cameras, there has been a new interest in the research conducted in RGB images. Nevertheless, hand pose estimation is hard. Research on hand pose estimation has multiple issues, including high-dimensional degrees of freedom, shape changes, self-occlusion, and real-time condition. To address the issues, this study proposes the random forests-based method of hierarchically estimating hand pose in depth images. In this study, the hierarchical estimation method that individually handles hand palms and fingers with the use of an inverse matrix is utilized to address high-dimensional degrees of freedom, shape changes, and self-occlusion. For real-time execution, random forests using simple characteristics are applied. As shown in the experimental results of this study, the proposed hierarchical estimation method estimates the hand pose in input depth images more robustly and quickly than other existing methods.

[1]  Alexey G. Finogeev,et al.  The convergence computing model for big sensor data mining and knowledge discovery , 2017, Human-centric Computing and Information Sciences.

[2]  Andrew Gilbert,et al.  Guided optimisation through classification and regression for hand pose estimation , 2017, Comput. Vis. Image Underst..

[3]  Andrea Tagliasacchi,et al.  Robust Articulated-ICP for Real-Time Hand Tracking , 2015 .

[4]  Serafín Moral,et al.  Increasing diversity in random forest learning algorithm via imprecise probabilities , 2018, Expert Syst. Appl..

[5]  Brenda K. Wiederhold,et al.  Using Virtual Reality to Mobilize Health Care: Mobile Virtual Reality Technology for Attenuation of Anxiety and Pain , 2018, IEEE Consumer Electronics Magazine.

[6]  Antonis A. Argyros,et al.  Generative 3D Hand Tracking with Spatially Constrained Pose Sampling , 2017, BMVC.

[7]  Hazem Wannous,et al.  Skeleton-Based Dynamic Hand Gesture Recognition , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[8]  Tae-Seong Kim,et al.  Hand Gesture Recognition and Interface via a Depth Imaging Sensor for Smart Home Appliances , 2014 .

[9]  Hong Va Leong,et al.  Real-time tracking of hand gestures for interactive game design , 2009, 2009 IEEE International Symposium on Industrial Electronics.

[10]  Kathiravan Srinivasan,et al.  Robust RGB-D Hand Tracking Using Deep Learning Priors , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[11]  Antti Oulasvirta,et al.  Fast and robust hand tracking using detection-guided optimization , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Suwanna Rasmequan,et al.  Adaptive directional bounding box from RGB-D information for improving fall detection , 2017, J. Vis. Commun. Image Represent..

[13]  Jie Liu,et al.  Hand pose estimation with multi-scale network , 2017, Applied Intelligence.

[14]  João Paulo Papa,et al.  Internet of Things: A survey on machine learning-based intrusion detection approaches , 2019, Comput. Networks.

[15]  Alon Lerner,et al.  ICPIK: Inverse Kinematics based articulated-ICP , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[16]  Vincent Lepetit,et al.  Hands Deep in Deep Learning for Hand Pose Estimation , 2015, ArXiv.

[17]  Xinjun Sheng,et al.  Shared control of a robotic arm using non-invasive brain-computer interface and computer vision guidance , 2019, Robotics Auton. Syst..

[18]  Holger Regenbrecht,et al.  Towards Pervasive Augmented Reality: Context-Awareness in Augmented Reality , 2017, IEEE Transactions on Visualization and Computer Graphics.

[19]  Mircea Nicolescu,et al.  Vision-based hand pose estimation: A review , 2007, Comput. Vis. Image Underst..

[20]  Yun Tian,et al.  Calibrating effective focal length for central catadioptric cameras using one space line , 2012, Pattern Recognit. Lett..

[21]  Didier Stricker,et al.  Simultaneous Hand Pose and Skeleton Bone-Lengths Estimation from a Single Depth Image , 2017, 2017 International Conference on 3D Vision (3DV).

[22]  Li-Chen Fu,et al.  Learning a deep network with spherical part model for 3D hand pose estimation , 2017, ICRA.

[23]  Antonis A. Argyros,et al.  Hybrid One-Shot 3D Hand Pose Estimation by Exploiting Uncertainties , 2015, BMVC.

[24]  Albert Dipanda,et al.  Hand pose estimation and tracking in real and virtual interaction: A review , 2019, Image Vis. Comput..

[25]  Andrea Tagliasacchi,et al.  Low-Dimensionality Calibration through Local Anisotropic Scaling for Robust Hand Model Personalization , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[26]  Ying Zhao,et al.  An information-aware visualization for privacy-preserving accelerometer data sharing , 2018, Human-centric Computing and Information Sciences.

[27]  Quanxin Liu,et al.  Camera calibration method based on optimal polarization angle , 2019, Optics and Lasers in Engineering.

[28]  Taufik Abrão,et al.  Wavelet against random forest for anomaly mitigation in software-defined networking , 2019, Appl. Soft Comput..

[29]  Javier Ruiz Hidalgo,et al.  Correspondence matching in unorganized 3D point clouds using Convolutional Neural Networks , 2019, Image Vis. Comput..

[30]  Chunfang Liu,et al.  3D human gesture capturing and recognition by the IMMU-based data glove , 2018, Neurocomputing.

[31]  Alex H. B. Duffy,et al.  Systematic literature review of hand gestures used in human computer interaction interfaces , 2019, Int. J. Hum. Comput. Stud..

[32]  Kaizhu Huang,et al.  Banzhaf random forests: Cooperative game theory based random forests with consistency , 2018, Neural Networks.

[33]  Kechar Bouabdellah,et al.  Rotational Wireless Video Sensor Networks with Obstacle Avoidance Capability for Improving Disaster Area Coverage , 2015, J. Inf. Process. Syst..

[34]  Sergio Escalera,et al.  Occlusion Aware Hand Pose Recovery from Sequences of Depth Images , 2017, 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017).

[35]  K. V. Arya,et al.  A new heuristic for multilevel thresholding of images , 2019, Expert Syst. Appl..

[36]  Songfeng Lu,et al.  Many-objectives multilevel thresholding image segmentation using Knee Evolutionary Algorithm , 2019, Expert Syst. Appl..

[37]  Christian Wolf,et al.  Hand pose estimation through semi-supervised and weakly-supervised learning , 2015, Comput. Vis. Image Underst..

[38]  Jianrong Tan,et al.  A survey on 3D hand pose estimation: Cameras, methods, and datasets , 2019, Pattern Recognit..

[39]  Jinjun Chen,et al.  Privacy preservation in blockchain based IoT systems: Integration issues, prospects, challenges, and future research directions , 2019, Future Gener. Comput. Syst..

[40]  Yang Liu,et al.  Economical and Balanced Energy Usage in the Smart Home Infrastructure: A Tutorial and New Results , 2015, IEEE Transactions on Emerging Topics in Computing.

[41]  Saifollah Rasouli,et al.  Microlenses focal length measurement using Z-scan and parallel moiré deflectometry , 2013 .

[42]  Jessica Cantillo-Negrete,et al.  Robotic orthosis compared to virtual hand for Brain–Computer Interface feedback , 2019, Biocybernetics and Biomedical Engineering.

[43]  Andrew W. Fitzgibbon,et al.  Efficient and precise interactive hand tracking through joint, continuous optimization of pose and correspondences , 2016, ACM Trans. Graph..

[44]  Daniel Thalmann,et al.  Real-Time 3D Hand Pose Estimation with 3D Convolutional Neural Networks , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Shuang Zhang,et al.  Frequency domain point cloud registration based on the Fourier transform , 2019, J. Vis. Commun. Image Represent..

[46]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[47]  Robert Sabourin,et al.  Random forest dissimilarity based multi-view learning for Radiomics application , 2019, Pattern Recognit..

[48]  Ákos Pernek,et al.  Automatic focal length estimation as an eigenvalue problem , 2013, Pattern Recognit. Lett..

[49]  Lu Ding,et al.  Detection based visual tracking with convolutional neural network , 2019, Knowl. Based Syst..

[50]  Yong Hu,et al.  Simple very deep convolutional network for robust hand pose regression from a single depth image , 2017, Pattern Recognit. Lett..

[51]  Jeong-Joon Kim.  Spatio-temporal Sensor Data Processing Techniques , 2017, J. Inf. Process. Syst..

[52]  José A. Pino,et al.  Predicting user performance time for hand gesture interfaces , 2017 .

[53]  Sheng Xiao,et al.  3D face recognition: a survey , 2018, Human-centric Computing and Information Sciences.

[54]  Hans-Peter Seidel,et al.  Design and volume optimization of space structures , 2017, ACM Trans. Graph..

[55]  Jaspreet Singh,et al.  Optimization of sentiment analysis using machine learning classifiers , 2017, Human-centric Computing and Information Sciences.

[56]  Fernando Magno Quintão Pereira,et al.  SIoT: Securing Internet of Things through distributed systems analysis , 2017, Future Gener. Comput. Syst..

[57]  Wei T. Yue,et al.  Transformative value of the Internet of Things and pricing decisions , 2019, Electron. Commer. Res. Appl..

[58]  Andrew W. Fitzgibbon,et al.  Online generative model personalization for hand tracking , 2017, ACM Trans. Graph..

[59]  Stephen Gareth Pierce,et al.  Novel algorithms for 3D surface point cloud boundary detection and edge reconstruction , 2019, J. Comput. Des. Eng..

[60]  Andrea Tagliasacchi,et al.  Sphere-meshes for real-time hand modeling and tracking , 2016, ACM Trans. Graph..

[61]  Daniel Thalmann,et al.  3D Convolutional Neural Networks for Efficient and Robust Hand Pose Estimation from Single Depth Images , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[62]  Steve Marschner,et al.  Matching Real Fabrics with Micro-Appearance Models , 2015, ACM Trans. Graph..