mmPoint: Dense Human Point Cloud Generation from mmWave

Millimeter-wave (mmWave) radars have emerged as a promising technology for sensing humans in diverse environments, owing to their ability to easily obtain 3D information in the form of point clouds. However, mmWave point clouds are typically characterized by sparsity and irregularity, which may limit their potential for certain applications. To address this issue, we propose mmPoint, the first model capable of generating dense human point clouds from mmWave radar signals. Specifically, mmPoint takes a single radar frame of a human as input and generates a dense point cloud that accurately reflects the shape of the detected human as output. The proposed model consists of a novel Encoder-Decoder architecture that utilizes a Multi-Modal Encoder (MME) to extract features from both the radar signal and a point cloud template. A Multi-Resolution Decoder (MRD) is then utilized to gradually infer a dense point cloud in a three-step fashion, with a Lift-and-Deform Module (LDM) employed at each step to increase the number of points and deform the point cloud based on the radar feature. Experimental results demonstrate that mmPoint achieves excellent performance on dense point cloud generation from mmWave radar signals. Code and dataset are available at https://github.com/NUAAXQ/mmPoint .