Robust Depth Image Acquisition Using Modulated Pattern Projection and Probabilistic Graphical Models

Depth image acquisition with structured light approaches in outdoor environments is a challenging problem due to external factors, such as ambient sunlight, which commonly affect the acquisition procedure. This paper presents a novel structured light sensor designed specifically for operation in outdoor environments. The sensor exploits a modulated sequence of structured light projected onto the target scene to counteract environmental factors and estimate a spatial distortion map in a robust manner. The correspondence between the projected pattern and the estimated distortion map is then established using a probabilistic framework based on graphical models. Finally, the depth image of the target scene is reconstructed using a number of reference frames recorded during the calibration process. We evaluate the proposed sensor on experimental data in indoor and outdoor environments and present comparative experiments with other existing methods, as well as commercial sensors.

[1]  Tokuo Tsuji,et al.  High-speed 3D image acquisition using coded structured light projection , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[2]  Simon Dobrisek,et al.  Exploiting Spatio-Temporal Information for Light-Plane Labeling in Depth-Image Sensors Using Probabilistic Graphical Models , 2016, Informatica.

[3]  Richard Szeliski,et al.  High-accuracy stereo depth maps using structured light , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[4]  Benjamin Ranft.  3D perception for autonomous navigation of a low-cost MAV using minimal landmarks , 2013 .

[5]  Ales Svigelj,et al.  Modulated Acquisition of Spatial Distortion Maps , 2013, Sensors.

[6]  Henry Fuchs,et al.  3D imaging in medicine : algorithms, systems, applications , 1990 .

[7]  Ashutosh Saxena,et al.  Pose Estimation from a Single Depth Image for Arbitrary Kinematic Skeletons , 2011, ArXiv.

[8]  Masahiro Fujita,et al.  3D Perception and Environment Map Generation for Humanoid Robot Navigation , 2008, Int. J. Robotics Res..

[9]  Rok Gajsek,et al.  Gender and affect recognition based on GMM and GMM-UBM modeling with relevance MAP estimation , 2010, INTERSPEECH.

[10]  Arman Savran,et al.  Bosphorus Database for 3D Face Analysis , 2008, BIOID.

[11]  Giovanna Sansoni,et al.  State-of-The-Art and Applications of 3D Imaging Sensors in Industry, Cultural Heritage, Medicine, and Criminal Investigation , 2009, Sensors.

[12]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision (2nd ed) , 2003 .

[13]  Nelson L. Max,et al.  Structured Light-Based 3D Reconstruction System for Plants , 2015, Sensors.

[14]  Qing Zhang,et al.  A Survey on Human Motion Analysis from Depth Data , 2013, Time-of-Flight and Depth Imaging.

[15]  Tom Heskes,et al.  Fractional Belief Propagation , 2002, NIPS.

[16]  S. P. Mudur,et al.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[17]  Nikos Paragios,et al.  Handbook of Mathematical Models in Computer Vision , 2005 .

[18]  Wilhelm Burger,et al.  Digital Image Processing - An Algorithmic Introduction using Java , 2008, Texts in Computer Science.

[19]  Jean-François Giovannelli,et al.  Image reconstruction in optical interferometry , 2010, IEEE Signal Processing Magazine.

[20]  Hong Liu,et al.  3D Action Recognition Using Multi-Temporal Depth Motion Maps and Fisher Vector , 2016, IJCAI.

[21]  D B Karki,et al.  Imaging in medicine. , 2004, Kathmandu University medical journal.

[22]  Gabriel Taubin,et al.  Robust one-shot 3D scanning using loopy belief propagation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[23]  François Berry,et al.  Toward 3D Reconstruction of Outdoor Scenes Using an MMW Radar and a Monocular Vision Sensor , 2015, Sensors.

[24]  William T. Freeman,et al.  On the optimality of solutions of the max-product belief-propagation algorithm in arbitrary graphs , 2001, IEEE Trans. Inf. Theory.

[25]  Li Zhang,et al.  Rapid shape acquisition using color structured light and multi-pass dynamic programming , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[26]  Jean Ponce,et al.  Computer Vision: A Modern Approach , 2002 .

[27]  Joachim Hornegger,et al.  3-D gesture-based scene navigation in medical imaging applications using Time-of-Flight cameras , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[28]  Roland Siegwart,et al.  A state-of-the-art 3D sensor for robot navigation , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[29]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[30]  Shree K. Nayar,et al.  Shape from Focus , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Olivier D. Faugeras,et al.  Shape From Shading , 2006, Handbook of Mathematical Models in Computer Vision.

[32]  Nahum Kiryati,et al.  Toward optimal structured light patterns , 1997, Proceedings. International Conference on Recent Advances in 3-D Digital Imaging and Modeling (Cat. No.97TB100134).

[33]  Larry S. Davis,et al.  Uncalibrated stereo rectification for automatic 3D surveillance , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[34]  Joaquim Salvi,et al.  Recent progress in coded structured light as a technique to solve the correspondence problem: a survey , 1998, Pattern Recognit..

[35]  Konstantinos N. Plataniotis,et al.  Cyclic orthogonal codes in CDMA-based asynchronous Wireless Body Area Networks , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[36]  Joaquim Salvi,et al.  Pattern codification strategies in structured light systems , 2004, Pattern Recognit..

[37]  Paul J. Besl,et al.  Active, optical range imaging sensors , 1988, Machine Vision and Applications.

[38]  Vitomir Štruc,et al.  Towards Robust 3D Face Verification Using Gaussian Mixture Models , 2012 .

[39]  Rok Gajsek,et al.  Emotion recognition using linear transformations in combination with video , 2009, INTERSPEECH.

[40]  R. Lange,et al.  Solid-state time-of-flight range camera , 2001 .

[41]  François Blais.  Review of 20 years of range sensor development , 2004, J. Electronic Imaging.

[42]  Yun Yang,et al.  Action recognition from depth sequences using weighted fusion of 2D and 3D auto-correlation of gradients features , 2016, Multimedia Tools and Applications.

[43]  H. S. Wolff,et al.  iRun: Horizontal and Vertical Shape of a Region-Based Graph Compression , 2022, Sensors.

[44]  Maoying Qiao,et al.  Model-Based Human Pose Estimation with Hierarchical ICP from Single Depth Images , 2011 .

[45]  Yu Zhang,et al.  Estimating 3D Leaf and Stem Shape of Nursery Paprika Plants by a Novel Multi-Camera Photography System , 2016, Sensors.

[46]  Simon Dobrisek,et al.  Combining 3D face representations using region covariance descriptors and statistical models , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[47]  Brendan J. Frey,et al.  Factor graphs and the sum-product algorithm , 2001, IEEE Trans. Inf. Theory.

[48]  Jeffrey L. Posdamer,et al.  Surface measurement by space-encoded projected beam systems , 1982, Comput. Graph. Image Process..

[49]  Gerhard Rigoll,et al.  Surveillance and Activity Recognition with Depth Information , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[50]  Maria Luisa Merani,et al.  On the assignment of Walsh and quasi-orthogonal codes in a multicarrier DS-CDMA system with multiple classes of users , 2002, Global Telecommunications Conference, 2002. GLOBECOM '02. IEEE.