K-Order Graph-oriented Transformer with GraAttention for 3D Pose and Shape Estimation

We propose a novel attention-based 2D-to-3D pose estimation network for graph-structured data, named KOG-Transformer, and a 3D pose-to-shape estimation network for hand data, named GASE-Net. Previous 3D pose estimation methods focus on various modifications of the graph convolutional kernel, such as abandoning weight sharing or increasing the receptive field. Some of these methods employ attention-based non-local modules as auxiliary modules. In order to better model the relationship between nodes in graph-structured data and fuse the information of different neighbor nodes in a differentiated way, we make targeted modifications to the attention module and propose two modules designed for graph-structured data, graph relative positional encoding multi-head self-attention (GR-MSA) and K-order graph-oriented multi-head self-attention (KOG-MSA). By stacking GR-MSA and KOG-MSA, we propose a novel network KOG-Transformer 1 for 2D-to-3D pose estimation. Further, we propose a network for shape estimation on hand data, called GraAttention Shape Estimation Network (GASE-Net), which takes a 3D pose as input and gradually models the shape of a hand from sparse to dense. We have empirically shown the superiority of KOG-Transformer through extensive experiments. The experimental results show that the KOG-Transformer significantly outperforms the previous state-of-the-art methods on the benchmark dataset Human3.6M. We evaluate the effect of GASE-Net on two hand datasets publicly available, ObMan and InterHand2.6M. The experimental results show that the GASE-Net can estimate the corresponding shapes for input poses with strong generalization ability.