Pose Recognition of 3D Human Shapes via Multi-View CNN with Ordered View Feature Fusion

Rapid pose classification and pose retrieval in 3D human datasets are important problems in shape analysis. In this paper, we extend the Multi-View Convolutional Neural Network (MVCNN) with ordered view feature fusion for orientation-aware 3D human pose classification and retrieval. Firstly, we combine each learned view feature in an orderly manner to form a compact representation for orientation-aware pose classification. Secondly, for pose retrieval, the Siamese network is adopted to learn descriptor vectors, where their L2 distances are close for pairs of shapes with the same poses and are far away for pairs of shapes with different poses. Furthermore, we also construct a larger 3D Human Pose Recognition Dataset (HPRD) consisting of 100,000 shapes for the evaluation of pose classification and retrieval. Experiments and comparisons demonstrate that our method obtains better results than previous works of pose classification and retrieval on the 3D human datasets, such as SHREC’14, FAUST, and HPRD.

[1]  A KakadiarisIoannis,et al.  3D Human pose estimation , 2016 .

[2]  Michael J. Black,et al.  MoSh: motion and shape capture from sparse markers , 2014, ACM Trans. Graph..

[3]  Cristian Sminchisescu,et al.  Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Hui Zeng,et al.  Multi-Feature Fusion Based on Multi-View Feature and 3D Shape Feature for Non-Rigid 3D Model Retrieval , 2019, IEEE Access.

[5]  Stefano Berretti,et al.  Representation, Analysis, and Recognition of 3D Humans , 2018, ACM Trans. Multim. Comput. Commun. Appl..

[6]  Michael J. Black,et al.  SMPL: A Skinned Multi-Person Linear Model , 2023 .

[7]  Hazem Wannous,et al.  3D human motion analysis framework for shape similarity and retrieval , 2014, Image Vis. Comput..

[8]  Bo Li,et al.  Shape Retrieval of Non-rigid 3D Human Models , 2014, International Journal of Computer Vision.

[9]  Lin Gao,et al.  A survey on deep geometry learning: From a representation perspective , 2020, Computational Visual Media.

[10]  C. V. Jawahar,et al.  Human pose search using deep networks , 2017, Image Vis. Comput..

[11]  Ersin Yumer,et al.  Learning Local Shape Descriptors from Part Correspondences with Multiview Convolutional Networks , 2017, ACM Trans. Graph..

[12]  Yang Liu,et al.  Adaptive O-CNN , 2018, ACM Trans. Graph..

[13]  Ralph R. Martin,et al.  Parametric modeling of 3D human body shape - A survey , 2017, Comput. Graph..

[14]  Pierre Vandergheynst,et al.  Geometric Deep Learning: Going beyond Euclidean data , 2016, IEEE Signal Process. Mag..

[15]  Suk-Hwan Lee,et al.  A 3D Shape Recognition Method Using Hybrid Deep Learning Network CNN–SVM , 2020, Electronics.

[16]  Dianhui Mao,et al.  A Novel Sketch-Based Three-Dimensional Shape Retrieval Method Using Multi-View Convolutional Neural Network , 2019, Symmetry.

[17]  Ioannis A. Kakadiaris,et al.  3D Human pose estimation: A review of the literature and analysis of covariates , 2016, Comput. Vis. Image Underst..

[18]  Andrew Zisserman,et al.  2D Articulated Human Pose Estimation and Retrieval in (Almost) Unconstrained Still Images , 2012, International Journal of Computer Vision.