Non-line-of-sight tracking of people at long range

A remote-sensing system that can determine the position of hidden objects has applications in many critical real-life scenarios, such as search and rescue missions and safe autonomous driving. Previous work has shown the ability to range and image objects hidden from the direct line of sight, employing advanced optical imaging technologies aimed at small objects at short range. In this work we demonstrate a long-range tracking system based on single laser illumination and single-pixel single-photon detection. This enables us to track one or more people hidden from view at a stand-off distance of over 50 m. These results pave the way towards next generation LiDAR systems that will reconstruct not only the direct-view scene but also the main elements hidden behind walls or corners. Published by The Optical Society under the terms of the Creative Commons Attribution 4.0 License. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI. OCIS codes: (150.5670) Machine vision, Range finding; (040.1345) Optoelectronics, Avalanche photodiodes (APDs). References and links 1. A. Kirmani, T. Hutchison, J. Davis, and R. Raskar, “Looking around the corner using transient imaging,” in Proceedings of IEEE International Conference on Computer Vision (IEEE, 2009), pp. 159–166. 2. R. Pandharkar, A. Velten, A. Bardagjy, E. Lawson, M. Bawendi, and R. Raskar, “Estimating motion and size of moving non-line-of-sight objects in cluttered environments,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 265–272. 3. A. Velten, T. Willwacher, O. Gupta, A. Veeraraghavan, M. G. Bawendi, and R. Raskar, “Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging,” Nature Comms 3, 745–748 (2012). 4. O. Gupta, T. Willwacher, A. Velten, A. Veeraraghavan, and R. Raskar, “Reconstruction of hidden 3D shapes using diffuse reflections,” Opt. Express 20, 19096–19108 (2012). 5. F. Heide, M. B. Hullin, J. Gregson, and W. Heidrich, “Low-budget transient imaging using photonic mixer devices,” ACM Trans. Graph. 32, 45–50 (2013). 6. M. Buttafava, J. Zeman, A. Tosi, K. Eliceiri, and A. Velten, “Non-line-of-sight imaging using a time-gated single photon avalanche diode,” Opt. Express 23, 20997–21115 (2015). 7. M. Laurenzis, J. Klein, E. Bacher, and N. Metzger, “Multiple-return single-photon counting of light in flight and sensing of non-line-of-sight objects at shortwave infrared wavelengths,” Opt. Lett. 40, 4815–4818 (2015). 8. G. Gariepy, F. Tonolini, R. Henderson, J. Leach, and D. Faccio, “Detection and tracking of moving objects hidden from view,” Nature Photon. 10, 23–26 (2015). 9. S. Shrestha, F. Heide, W. Heidrich, and G. Wetzstein, “Computational imaging with multi-camera time-of-flight systems,” ACM Trans. Graph. 35, 1–11 (2016). 10. A. Kadambi, H. Zhao, B. Shi, and R. Raskar, “Occluded imaging with time-of-flight sensors,” ACM Trans. Graph. 35, 1–12 (2016). 11. J. Klein, C. Peters, J. Martín, and M. Laurenzis, “Tracking objects outside the line of sight using 2D intensity images,” Scientific Reports 6,32491 (2016). 12. U. Wandinger, “Introduction to Lidar,” in Lidar: Range-Resolved Optical Remote Sensing of the Atmosphere, C. Weitkamp, ed. (Springer, 2005), pp. 1–18. 13. C. Niclass, A. Rochas, P.-A. Besse, and E. Charbon, “Design and characterization of a CMOS 3-D image sensor based on single photon avalanche diodes,” IEEE J. Solid-State Circuits 40, 1847–1854 (2005). 14. F. Zappa, S. Tisa, A. Tosi, and S. Cova, “Principles and features of single-photon avalanche diode arrays,” Sens. Actuators A Phys. 140, 103–112 (2007). 15. J. Richardson, R. Walker, L. Grant, D. Stoppa, F. Borghetti, E. Charbon, M. Gersbach, and R. K. Henderson, “A 32x32 50ps resolution 10 bit time to digital converter array in 130nm CMOS for time correlated imaging,” in Vol. 25, No. 9 | 1 May 2017 | OPTICS EXPRESS 10109 #276516 https://doi.org/10.1364/OE.25.010109 Journal © 2017 Received 26 Sep 2016; revised 30 Jan 2017; accepted 30 Jan 2017; published 24 Apr 2017 Proceedings of IEEE Custom Integrated Circuits Conference (IEEE, 2009), pp. 77–80. 16. E. Charbon, M. Fishburn, R. Walker, R. K. Henderson, and C. Niclass, “SPAD-Based Sensors,” in “TOF RangeImaging Cameras,” F. Remondino and D. Stoppa, eds. (Springer, 2013), pp. 11–38. 17. J. A. Richardson, L. A. Grant, and R. K. Henderson, “Low dark count single-photon avalanche diode structure compatible with standard nanometer scale CMOS technology,” IEEE Photon. Technol. Lett. 21, 1020–1022 (2009). 18. G. Gariepy, N. Krstajić, R. Henderson, C. Li, R. R. Thomson, G. S. Buller, B. Heshmat, R. Raskar, J. Leach, and D. Faccio, “Single-photon sensitive light-in-flight imaging,” Nature Comms 6, 6021–6026 (2015). 19. W. Becker, Advanced Time-Correlated Single Photon Counting Techniques (Springer, 2005). 20. R. Cutler and L. Davis, “View-based detection and analysis of periodic motion,” in Proceedings of International Conference on Pattern Recognition (IEEE Comput. Soc, 1998), pp. 495–500. 21. R. Cucchiara, C. Grana, M. Piccardi, and A. Prati, “Detecting moving objects, ghosts, and shadows in video streams,” IEEE Trans. Pattern Anal. Mach. Intell. 25, 1337–1342 (2003). 22. T. S. Ralston, G. L. Charvat, and J. E. Peabody, “Real-time through-wall imaging using an ultrawideband multipleinput multiple-output (MIMO) phased array radar system,” in Proceedings of IEEE International Symposium on Phased Array Systems and Technology (IEEE, 2010), pp. 551–558.