Virtual Point Removal for Large-Scale 3D Point Clouds with Multiple Glass Planes

Large-scale 3D point clouds (LS3DPCs) captured by terrestrial LiDAR scanners often include virtual points which are generated by glass reflection. The virtual points may degrade the performance of various computer vision techniques when applied to LS3DPCs. In this paper, we propose a virtual point removal algorithm for LS3DPCs with multiple glass planes. We first estimate multiple glass regions by modeling the reliability with respect to each glass plane, respectively, such that the regions are assigned high reliability when they have multiple echo pulses for each emitted laser pulse. Then we detect each point whether it is a virtual point or not. For a given point, we recursively traverse all the possible trajectories of reflection, and select the optimal trajectory which provides a point with a similar geometric feature to a given point at the symmetric location. We evaluate the performance of the proposed algorithm on various LS3DPC models with diverse numbers of glass planes. Experimental results show that the proposed algorithm estimates multiple glass regions faithfully and detects the virtual points successfully. Moreover, we also show that the proposed algorithm yields a much better performance of reflection artifact removal compared with the existing method qualitatively and quantitatively.

[1]  Anat Levin,et al.  User Assisted Separation of Reflections from a Single Image Using a Sparsity Prior , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Assaf Zomet,et al.  Separating reflections from a single image using local features , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[3]  Ren Ng,et al.  Single Image Reflection Separation with Perceptual Losses , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[4]  Nico Blodow,et al.  Fast Point Feature Histograms (FPFH) for 3D registration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[5]  Michael Goesele,et al.  Image-based rendering for scenes with reflections , 2012, ACM Trans. Graph..

[6]  Jae-Young Sim,et al.  Reflection Removal Using Low-Rank Matrix Completion , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Ronen Basri,et al.  Separation of Transparent Layers using Focus , 2004, International Journal of Computer Vision.

[8]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[9]  Jian Yang,et al.  Lidar-histogram for fast road and obstacle detection , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[10]  Alston S. Householder,et al.  Unitary Triangularization of a Nonsymmetric Matrix , 1958, JACM.

[11]  Andrea Fusiello,et al.  Robust Multiple Structures Estimation with J-Linkage , 2008, ECCV.

[12]  Ayellet Tal,et al.  Saliency Detection in Large Point Sets , 2013, 2013 IEEE International Conference on Computer Vision.

[13]  Ji Wan,et al.  Multi-view 3D Object Detection Network for Autonomous Driving , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Andrew E. Johnson,et al.  Spin-Images: A Representation for 3-D Surface Matching , 1997 .

[15]  William T. Freeman,et al.  A computational approach for obstruction-free photography , 2015, ACM Trans. Graph..

[16]  Sabine Süsstrunk,et al.  Single Image Reflection Suppression , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[18]  Amit K. Agrawal,et al.  Removing photography artifacts using gradient projection and flash-exposure sampling , 2005, ACM Trans. Graph..

[19]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Nico Blodow,et al.  General 3D modelling of novel objects from a single view , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[21]  Lennart Svensson,et al.  Fast LIDAR-based road detection using fully convolutional neural networks , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[22]  Jae-Young Sim,et al.  Glass Reflection Removal Using Co-Saliency-Based Image Alignment and Low-Rank Matrix Completion in Gradient Domain , 2018, IEEE Transactions on Image Processing.

[23]  Jan Kautz,et al.  Separating Reflection and Transmission Images in the Wild , 2017, ECCV.

[24]  Yu-Wing Tai,et al.  A Physically-Based Approach to Reflection Separation: From Physical Modeling to Constrained Optimization , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Chang-Su Kim,et al.  Large-Scale 3D Point Cloud Compression Using Adaptive Radial Distance Prediction in Hybrid Coordinate Domains , 2015, IEEE Journal of Selected Topics in Signal Processing.

[26]  Jitendra Malik,et al.  Recognizing Objects in Range Data Using Regional Point Descriptors , 2004, ECCV.

[27]  Jie Yang,et al.  Seeing Deeply and Bidirectionally: A Deep Learning Approach for Single Image Reflection Removal , 2018, ECCV.

[28]  Michael S. Brown,et al.  Single Image Layer Separation Using Relative Smoothness , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Ling-Yu Duan,et al.  CRRN: Multi-scale Guided Concurrent Reflection Removal Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[30]  Jiaolong Yang,et al.  A Generic Deep Architecture for Single Image Reflection Removal and Image Smoothing (Supplementary Material) , 2017 .

[31]  Jae-Young Sim,et al.  Reflection Removal for Large-Scale 3D Point Clouds , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[32]  Xiaochun Cao,et al.  Robust Separation of Reflection from Multiple Images , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Frédo Durand,et al.  Reflection removal using ghosting cues , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Michael S. Brown,et al.  Exploiting Reflection Change for Automatic Reflection Removal , 2013, 2013 IEEE International Conference on Computer Vision.

[35]  Jae-Young Sim,et al.  Supervoxel-based saliency detection for large-scale colored 3D point clouds , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[36]  J. Besag On the Statistical Analysis of Dirty Pictures , 1986 .

[37]  Changshui Zhang,et al.  Blind Separation of Superimposed Moving Images Using Image Statistics , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Wojciech Matusik,et al.  Video Reflection Removal Through Spatio-Temporal Optimization , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).