Information-Driven Adaptive Structured-Light Scanners

Sensor planning and active sensing, long studied in robotics, adapt sensor parameters to maximize a utility function while constraining resource expenditures. Here we consider information gain as the utility function. While these concepts are often used to reason about 3D sensors, these are usually treated as a predefined, black-box, component. In this paper we show how the same principles can be used as part of the 3D sensor. We describe the relevant generative model for structured-light 3D scanning and show how adaptive pattern selection can maximize information gain in an open-loop-feedback manner. We then demonstrate how different choices of relevant variable sets (corresponding to the subproblems of locatization and mapping) lead to different criteria for pattern selection and can be computed in an online fashion. We show results for both subproblems with several pattern dictionary choices and demonstrate their usefulness for pose estimation and depth acquisition.

[1]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[2]  Feng Zhao,et al.  Information-driven dynamic sensor collaboration , 2002, IEEE Signal Process. Mag..

[3]  Derek Hoiem,et al.  Indoor Segmentation and Support Inference from RGBD Images , 2012, ECCV.

[4]  John W. Fisher,et al.  Information-Driven Adaptive Structured-Light Scanners , 2018, IEEE Transactions on Computational Imaging.

[5]  Sebastian Thrun,et al.  Robotic mapping: a survey , 2003 .

[6]  Sébastien Roy,et al.  A MRF formulation for coded structured light , 2005, Fifth International Conference on 3-D Digital Imaging and Modeling (3DIM'05).

[7]  Katsushi Ikeuchi,et al.  Modeling sensor detectability with the VANTAGE geometric/sensor modeler , 1989, IEEE Trans. Robotics Autom..

[8]  John W. Fisher,et al.  On the Role of Representations for Reasoning in Large-Scale Urban Scenes , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[9]  Wolfram Burgard,et al.  Improved Techniques for Grid Mapping With Rao-Blackwellized Particle Filters , 2007, IEEE Transactions on Robotics.

[10]  Hugh F. Durrant-Whyte,et al.  Simultaneous map building and localization for an autonomous mobile robot , 1991, Proceedings IROS '91:IEEE/RSJ International Workshop on Intelligent Robots and Systems '91.

[11]  G. Klein,et al.  Parallel Tracking and Mapping for Small AR Workspaces , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[12]  John W. Fisher,et al.  Approximate Dynamic Programming for Communication-Constrained Sensor Network Management , 2007, IEEE Transactions on Signal Processing.

[13]  Shree K. Nayar,et al.  Micro Phase Shifting , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  KrauseAndreas,et al.  Near-Optimal Sensor Placements in Gaussian Processes: Theory, Efficient Algorithms and Empirical Studies , 2008 .

[15]  Friedrich M. Wahl.  A Coded Light Approach for Depth Map Acquisition , 1986, DAGM-Symposium.

[16]  Mark R. Pickering,et al.  Dense depth estimation using adaptive structured light and cooperative algorithm , 2011, CVPR 2011 WORKSHOPS.

[17]  Guillermo Sapiro,et al.  Low-Cost Compressive Sensing for Color Video and Depth , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Vivek K. Goyal,et al.  Compressive depth map acquisition using a single photon-counting detector: Parametric signal processing meets sparsity , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  J. How,et al.  Information-rich Path Planning with General Constraints using Rapidly-exploring Random Trees , 2010 .

[20]  Jonathan P. How,et al.  Sensor Selection in High-Dimensional Gaussian Trees with Nuisances , 2013, NIPS.

[21]  Peter Stone,et al.  Reinforcement learning , 2019, Scholarpedia.

[22]  Pierre Graebling,et al.  Real-time structured light coding for adaptive patterns , 2013, Journal of Real-Time Image Processing.

[23]  Mac Schwager,et al.  Distributed robotic sensor networks: An information-theoretic approach , 2012, Int. J. Robotics Res..

[24]  Stefano Soatto,et al.  Information-Seeking Control Under Visibility-Based Uncertainty , 2013, Journal of Mathematical Imaging and Vision.

[25]  N KutulakosKiriakos,et al.  Homogeneous codes for energy-efficient illumination and imaging , 2015 .

[26]  Alfred M. Bruckstein,et al.  RGBD-fusion: Real-time high precision depth recovery , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Matthew O'Toole,et al.  Homogeneous codes for energy-efficient illumination and imaging , 2015, ACM Trans. Graph..

[28]  Shree K. Nayar,et al.  Programmable Imaging: Towards a Flexible Camera , 2006, International Journal of Computer Vision.

[29]  Joaquim Salvi,et al.  A state of the art in structured light patterns for surface profilometry , 2010, Pattern Recognit..

[30]  Dieter Fox,et al.  A large-scale hierarchical multi-view RGB-D object dataset , 2011, 2011 IEEE International Conference on Robotics and Automation.

[31]  DenzlerJoachim,et al.  Information Theoretic Sensor Data Selection for Active Object Recognition and State Estimation , 2002 .

[32]  Andreas Krause,et al.  Near-Optimal Sensor Placements in Gaussian Processes: Theory, Efficient Algorithms and Empirical Studies , 2008, J. Mach. Learn. Res..

[33]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[34]  Lucas Paletta,et al.  Learning temporal context in active object recognition using Bayesian analysis , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[35]  John W. Fisher,et al.  Maximum Mutual Information Principle for Dynamic Sensor Query Problems , 2003, IPSN.

[36]  F.S. Cohen,et al.  A decision theoretic approach for 3-D vision , 1988, Proceedings CVPR '88: The Computer Society Conference on Computer Vision and Pattern Recognition.

[37]  Stephen Lin,et al.  Shading-Based Shape Refinement of RGB-D Images , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[38]  Alfred O. Hero,et al.  Sensor management using an active sensing approach , 2005, Signal Process..

[39]  Ron Kimmel,et al.  Sparse Modeling of Shape from Structured Light , 2012, 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission.

[40]  Yoav Y. Schechner,et al.  The Next Best Underwater View , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Stefano Soatto,et al.  Steps Towards a Theory of Visual Information: Active Perception, Signal-to-Symbol Conversion and the Interplay Between Sensing and Control , 2011, ArXiv.

[42]  Paul R. Cohen,et al.  Toward AI research methodology: three case studies in evaluation , 1989, IEEE Trans. Syst. Man Cybern..

[43]  Frank Wolter,et al.  Exploring Artificial Intelligence in the New Millenium , 2002 .

[44]  Luc Van Gool,et al.  Real-time range acquisition by adaptive structured light , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Ronald A. Howard,et al.  Information Value Theory , 1966, IEEE Trans. Syst. Sci. Cybern..

[46]  Shree K. Nayar,et al.  Structured Light in Sunlight , 2013, 2013 IEEE International Conference on Computer Vision.

[47]  Zhiwei Xiong,et al.  Robust depth sensing with adaptive structured light illumination , 2014, J. Vis. Commun. Image Represent..

[48]  Fumie Yokota,et al.  Value of Information Analysis in Environmental Health Risk Management Decisions: Past, Present, and Future , 2004, Risk analysis : an official publication of the Society for Risk Analysis.

[49]  John W. Fisher,et al.  Performance Guarantees for Information Theoretic Active Inference , 2007, AISTATS.

[50]  John J. Leonard,et al.  Efficient scene simulation for robust monte carlo localization using an RGB-D camera , 2012, 2012 IEEE International Conference on Robotics and Automation.

[51]  Wei Hong,et al.  Model-Driven Data Acquisition in Sensor Networks , 2004, VLDB.

[52]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.