Point Cloud Generation From Multiple Angles of Voxel Grids

The advancement of deep learning technology has been concentrating on deploying end-to-end solutions using high dimensional data, such as images. Recently, a number of methods have been proposed for reconstructing 3D objects using deep learning. One such technique is the method that involves recovering 3D objects as voxel grids using one or multiple images. However, there has been very little work directed towards the generation of 3D objects represented by a set of points, i.e. point cloud, from voxel grids which are ambiguous and coarse. The development of a deep learning model that generates point clouds with details from coarse voxel grids has numerous benefits to quality of life. For example, design professionals can use this model to generate detailed 3D point clouds using a sketched and coarse voxel to enable their creativities. This paper presents an algorithm to generate point clouds from voxel grids. The algorithm explicitly loads 3D objects into voxels without projection operations and associated information loss. To obtain a comprehensive understanding of the voxel grid, the grid is analyzed through various angles, as inspired by how humans observe 3D objects. The features from various angles are passed into a GRU layer to extract patterns across views, which will then be passed to a channel-wise convolutional layer and graph convolution to generated the predicted point cloud. The experimental result of the algorithm indicates that the algorithm is capable of generating high-quality point clouds by understanding the semantic features of the voxel grid.

[1]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[2]  Bo Yang,et al.  Dense 3D Object Reconstruction from a Single Depth View , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Leonidas J. Guibas,et al.  ShapeNet: An Information-Rich 3D Model Repository , 2015, ArXiv.

[5]  Leonidas J. Guibas,et al.  PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space , 2017, NIPS.

[6]  Markus H. Gross,et al.  A Network Architecture for Point Cloud Classification via Automatic Depth Images Generation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[7]  D. Hubel,et al.  Receptive fields and functional architecture of monkey striate cortex , 1968, The Journal of physiology.

[8]  Hao Su,et al.  A Point Set Generation Network for 3D Object Reconstruction from a Single Image , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Abhinav Gupta,et al.  Learning a Predictable and Generative Vector Representation for Objects , 2016, ECCV.

[10]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[11]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[12]  Chunping Hou,et al.  Open-set human activity recognition based on micro-Doppler signatures , 2019, Pattern Recognit..

[13]  Sridha Sridharan,et al.  Super-resolution for biometrics: A comprehensive survey , 2018, Pattern Recognit..

[14]  Masahiro Takatsuka,et al.  Hexpo: A vanishing-proof activation function , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[15]  Bram van Ginneken,et al.  Computer analysis of computed tomography scans of the lung: a survey , 2006, IEEE Transactions on Medical Imaging.

[16]  Shiqi Yu,et al.  GaitGANv2: Invariant gait feature extraction using generative adversarial networks , 2019, Pattern Recognit..

[17]  Junsong Yuan,et al.  Fusion of 3D-LIDAR and camera data for scene parsing , 2014, J. Vis. Commun. Image Represent..

[18]  Ulrich Neumann,et al.  Recurrent Slice Networks for 3D Segmentation of Point Clouds , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[19]  Ping Tan,et al.  DualGAN: Unsupervised Dual Learning for Image-to-Image Translation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[20]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Yue Wang,et al.  Dynamic Graph CNN for Learning on Point Clouds , 2018, ACM Trans. Graph..

[22]  Kyungeun Cho,et al.  Generative Adversarial Network-Based Method for Transforming Single RGB Image Into 3D Point Cloud , 2019, IEEE Access.

[23]  Tianqi Chen,et al.  Empirical Evaluation of Rectified Activations in Convolutional Network , 2015, ArXiv.

[24]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Dacheng Tao,et al.  Perceptual Adversarial Networks for Image-to-Image Transformation , 2017, IEEE Transactions on Image Processing.

[26]  Maggi Kelly,et al.  A New Method for Segmenting Individual Trees from the Lidar Point Cloud , 2012 .

[27]  Taku Komura,et al.  3D Incomplete Point Cloud Surfaces Reconstruction With Segmentation and Feature-Enhancement , 2019, IEEE Access.

[28]  Xiaojuan Qi,et al.  GAL: Geometric Adversarial Loss for Single-View 3D-Object Reconstruction , 2018, ECCV.

[29]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[30]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[32]  Lorenzo Torresani,et al.  Learning Spatiotemporal Features with 3D Convolutional Networks , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[33]  Oliver Grau,et al.  VConv-DAE: Deep Volumetric Shape Learning Without Object Labels , 2016, ECCV Workshops.

[34]  Ulrich Neumann,et al.  SGPN: Similarity Group Proposal Network for 3D Point Cloud Instance Segmentation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[35]  Daniel Cohen-Or,et al.  PU-Net: Point Cloud Upsampling Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[36]  Dacheng Tao,et al.  Attention-GAN for Object Transfiguration in Wild Images , 2018, ECCV.

[37]  Sebastian Scherer,et al.  VoxNet: A 3D Convolutional Neural Network for real-time object recognition , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[38]  Jianxiong Xiao,et al.  3D ShapeNets: A deep representation for volumetric shapes , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Dacheng Tao,et al.  Tag Disentangled Generative Adversarial Network for Object Image Re-rendering , 2017, IJCAI.

[40]  Jiajun Wu,et al.  Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling , 2016, NIPS.

[41]  Muhammad Moazam Fraz,et al.  DPRNet: Deep 3D Point Based Residual Network for Semantic Segmentation and Classification of 3D Point Clouds , 2019, IEEE Access.

[42]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.