Fully in tensor computation manner: one-shot dense 3D structured light and beyond

: Tensor computation evolves fast towards a prosperous existence in recent years, e.g. PyTorch. An immediate advantage of using tensor computation is that one does not need to implement low-level parallelism to attain efficient computation, which is of simplicity for both research and application development. The authors began with discovering that a simple manoeuvre ‘tensor shift’ could perform neighbourhood manipulation in very efficient parallel manner. Based on ‘tensor shift’, they derive the tensor version of a renowned correspondence search algorithm: semi-global matching (SGM), which they prefix the name as tensor-SGM. To evaluate their idea, they build-up a novel and practical one-shot structured light 3D acquisition system, which yields state-of-art reconstruction results using off-the-shelf hardware. This is the first fully tensorised 3D reconstruction system published to the authors’ best knowledge, and it opens new possibilities. A major one is, in the same tensorised framework, they solved the pattern interfering problem which hinders multi-structured light systems from working together. This part is marked as ‘beyond’ in this study to avoid confusing the readers the spotlight: the fully tensorised 3D structured light framework.

[1]  Pascal Monasse,et al.  Quasi-Euclidean Epipolar Rectification , 2011, Image Process. Line.

[2]  Sergio Orts,et al.  HyperDepth: Learning Depth from Structured Light without Matching , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Xuan-Li Chen,et al.  Fully in Tensor Computation Manner: A One-Shot Dense 3D Acquisition Method using Structured Light , 2019, 2019 2nd China Symposium on Cognitive Computing and Hybrid Intelligence (CCHI).

[4]  Fabrice Labeau,et al.  Discrete Time Signal Processing , 2004 .

[5]  Yasushi Yagi,et al.  Dynamic scene shape reconstruction using a single structured light pattern , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Yaser Sheikh,et al.  Texture Illumination Separation for Single-Shot Structured Light Reconstruction , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[8]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[9]  Gabriel Taubin,et al.  One-shot scanning using De Bruijn spaced grids , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[10]  Shai Avidan,et al.  Semi-Global Matching: A Principled Derivation in Terms of Message Passing , 2014, GCPR.

[11]  Marc Pollefeys,et al.  Patch Based Confidence Prediction for Dense Disparity Map , 2016, BMVC.

[12]  Johannes Stallkamp,et al.  Real-time stereo vision: Optimizing Semi-Global Matching , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[13]  H. Hirschmuller,et al.  Stereo matching in the presence of sub-pixel calibration errors , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Raúl Rojas,et al.  Weighted Semi-Global Matching and Center-Symmetric Census Transform for Robust Driver Assistance , 2013, CAIP.

[15]  Carsten Rother,et al.  PatchMatch Stereo - Stereo Matching with Slanted Support Windows , 2011, BMVC.

[16]  Jianqin Zhou,et al.  On discrete cosine transform , 2011, ArXiv.

[17]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[18]  Luc Van Gool,et al.  One-shot active 3D shape acquisition , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[19]  Ryo Furukawa,et al.  Learning-based feature extraction for active 3D scan with reducing color crosstalk of multiple pattern projections , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[20]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  André Oosterlinck,et al.  Range Image Acquisition with a Single Binary-Encoded Light Pattern , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Rainer Stiefelhagen,et al.  Kinect Unleashed: Getting Control over High Resolution Depth Maps , 2013, MVA.

[23]  Joan L. Mitchell,et al.  JPEG: Still Image Data Compression Standard , 1992 .

[24]  Yann LeCun,et al.  Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches , 2015, J. Mach. Learn. Res..

[25]  Ramin Zabih,et al.  Non-parametric Local Transforms for Computing Visual Correspondence , 1994, ECCV.

[26]  Shree K. Nayar,et al.  Micro Phase Shifting , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  K. Hosoda,et al.  Time change in strain distribution on the lateral and plantar surfaces of the foot during walking , 2016 .

[28]  Luc Van Gool,et al.  Watermarking structured light patterns for one-shot, extendable 3D scanning , 2019, Other Conferences.

[29]  Enric Meinhardt,et al.  MGM: A Significantly More Global Matching for Stereovision , 2015, BMVC.

[30]  Gabriel Taubin,et al.  Simple, Accurate, and Robust Projector-Camera Calibration , 2012, 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission.

[31]  Jean-Michel Morel,et al.  Scale Space Meshing of Raw Data Point Sets , 2011, Comput. Graph. Forum.

[32]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[33]  Ryo Furukawa,et al.  Dense 3D Reconstruction from High Frame-Rate Video Using a Static Grid Pattern , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  H. Hirschmüller.  Accurate and Efficient Stereo Processing by Semi-Global Matching and Mutual Information , 2005, CVPR.

[35]  Peter F. Sturm,et al.  Efficient Separation Between Projected Patterns for Multiple Projector 3D People Scanning , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[36]  Kim L. Boyer,et al.  Color-Encoded Structured Light for Rapid Active Ranging , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Nicolas Martin,et al.  Unstructured light scanning to overcome interreflections , 2011, 2011 International Conference on Computer Vision.

[38]  Richard I. Hartley,et al.  Theory and Practice of Projective Rectification , 1999, International Journal of Computer Vision.

[39]  Maarten Vergauwen,et al.  3D Reconstruction from Multiple Images: Part 1 - Principles , 2009, Found. Trends Comput. Graph. Vis..

[40]  Richard Szeliski,et al.  Efficient High-Resolution Stereo Matching Using Local Plane Sweeps , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.