Direct Object Recognition Without Line-Of-Sight Using Optical Coherence

Visual object recognition under situations in which the direct line-of-sight is blocked, such as when it is occluded around the corner, is of practical importance in a wide range of applications. With coherent illumination, the light scattered from diffusive walls forms speckle patterns that contain information of the hidden object. It is possible to realize non-line-of-sight (NLOS) recognition with these speckle patterns. We introduce a novel approach based on speckle pattern recognition with deep neural network, which is simpler and more robust than other NLOS recognition methods. Simulations and experiments are performed to verify the feasibility and performance of this approach.

[1]  Wolfgang Heidrich,et al.  Material Classification Using Raw Time-of-Flight Measurements , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  J. Tanida,et al.  Learning-based imaging through scattering media. , 2016, Optics express.

[3]  Ran El-Yaniv,et al.  Selective Classification for Deep Neural Networks , 2017, NIPS.

[4]  Ramesh Raskar,et al.  Single view reflectance capture using multiplexed scattering and time-of-flight imaging , 2011, SA '11.

[5]  Min H. Kim,et al.  DeepToF: off-the-shelf real-time correction of multipath interference in time-of-flight imaging , 2017, ACM Trans. Graph..

[6]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Frédo Durand,et al.  Laser speckle photography for surface tampering detection , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Frédo Durand,et al.  Turning Corners into Cameras: Principles and Methods , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[9]  Wolfgang Heidrich,et al.  Diffuse Mirrors: 3D Reconstruction from Diffuse Indirect Illumination Using Inexpensive Time-of-Flight Sensors , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Omkar M. Parkhi,et al.  VGGFace2: A Dataset for Recognising Faces across Pose and Age , 2017, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[11]  Lei Tian,et al.  Deep speckle correlation: a deep learning approach toward scalable imaging through scattering media , 2018, Optica.

[12]  George Papandreou,et al.  Rethinking Atrous Convolution for Semantic Image Segmentation , 2017, ArXiv.

[13]  Ramesh Raskar,et al.  Looking Around the Corner using Ultrafast Transient Imaging , 2011, International Journal of Computer Vision.

[14]  J. Goodman.  Introduction to Fourier optics , 1969 .

[15]  Matthew O'Toole,et al.  Tracking Multiple Objects Outside the Line of Sight Using Speckle Imaging , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[16]  Gordon Wetzstein,et al.  Computational imaging with multi-camera time-of-flight systems , 2016, ACM Trans. Graph..

[17]  Ramesh Raskar,et al.  Occluded Imaging with Time-of-Flight Sensors , 2016, ACM Trans. Graph..

[18]  Ramesh Raskar,et al.  Looking around the corner using transient imaging , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[19]  J. Tanida,et al.  Speckle-learning-based object recognition through scattering media. , 2015, Optics express.

[20]  R. Raskar,et al.  All Photons Imaging Through Volumetric Scattering , 2016, Scientific Reports.

[21]  Prasanna Rangarajan,et al.  Resolving Non Line-of-Sight (NLoS) motion using Speckle , 2018 .

[22]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Shuai Li,et al.  Lensless computational imaging through deep learning , 2017, ArXiv.

[24]  Ramesh Raskar,et al.  Object classification through scattering media with deep learning on time resolved measurement. , 2017, Optics express.

[25]  Matthew O'Toole,et al.  Decomposing global light transport using time of flight imaging , 2012, CVPR.

[26]  O. Katz,et al.  Noninvasive nonlinear focusing and imaging through strongly scattering turbid layers , 2014, 1405.4826.

[27]  R R Alfano,et al.  Imaging objects hidden in highly scattering media using femtosecond second-harmonic-generation cross-correlation time gating. , 1991, Optics letters.

[28]  Ramesh Raskar,et al.  A light transport model for mitigating multipath interference in Time-of-flight sensors , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Charles Saunders,et al.  Computational periscopy with an ordinary digital camera , 2019, Nature.

[30]  Ramesh Raskar,et al.  Estimating Motion and size of moving non-line-of-sight objects in cluttered environments , 2011, CVPR 2011.

[31]  Hao Wang,et al.  Exploit imaging through opaque wall via deep learning , 2017, ArXiv.

[32]  Pingxing Chen,et al.  Imaging through scattering layers exceeding memory effect range with spatial-correlation-achieved point-spread-function. , 2018, Optics letters.

[33]  M. Fink,et al.  Non-invasive single-shot imaging through scattering layers and around corners via speckle correlations , 2014, Nature Photonics.

[34]  Yaron Silberberg,et al.  Scatterer recognition via analysis of speckle patterns , 2018 .

[35]  Lei Zhu,et al.  Tracking moving targets behind a scattering medium via speckle correlation. , 2018, Applied optics.

[36]  Gordon Wetzstein,et al.  Confocal non-line-of-sight imaging , 2018, SIGGRAPH Talks.

[37]  G. Pedrini,et al.  Looking through a diffuser and around an opaque surface: a holographic approach. , 2014, Optics express.

[38]  Frédo Durand,et al.  Capturing the human figure through a wall , 2015, ACM Trans. Graph..

[39]  R. Raskar,et al.  Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging , 2012, Nature Communications.