Information-Driven Adaptive Structured-Light Scanners

Sensor planning and active sensing, long studied in robotics, adapt sensor parameters to maximize a utility function while constraining resource expenditures. Here, we consider information gain as the utility function. While these concepts are often used to reason about 3D sensors, these are usually treated as a predefined black-box component. In this paper, we show how the same principles can be used as part of the 3D sensor. We describe the generative model for structured-light 3D scanning and show how adaptive pattern selection can maximize information gain in an open-loop-feedback manner. We then demonstrate how different choices of relevant variable sets (corresponding to the subproblems of localization and mapping) lead to different criteria for pattern selection and can be computed in an online fashion. We show results for both subproblems with several pattern dictionary choices and demonstrate their usefulness for pose estimation and depth acquisition.

[1]  Stefano Soatto,et al.  Information-Seeking Control Under Visibility-Based Uncertainty , 2013, Journal of Mathematical Imaging and Vision.

[2]  Ronald A. Howard,et al.  Information Value Theory , 1966, IEEE Trans. Syst. Sci. Cybern..

[3]  Hugh F. Durrant-Whyte,et al.  Simultaneous map building and localization for an autonomous mobile robot , 1991, Proceedings IROS '91:IEEE/RSJ International Workshop on Intelligent Robots and Systems '91.

[4]  F.S. Cohen,et al.  A decision theoretic approach for 3-D vision , 1988, Proceedings CVPR '88: The Computer Society Conference on Computer Vision and Pattern Recognition.

[5]  Fumie Yokota,et al.  Value of Information Analysis in Environmental Health Risk Management Decisions: Past, Present, and Future , 2004, Risk analysis : an official publication of the Society for Risk Analysis.

[6]  Stephen Lin,et al.  Shading-Based Shape Refinement of RGB-D Images , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Alfred O. Hero,et al.  Sensor management using an active sensing approach , 2005, Signal Process..

[8]  Pierre Graebling,et al.  Real-time structured light coding for adaptive patterns , 2013, Journal of Real-Time Image Processing.

[9]  Sebastian Nowozin,et al.  Bayesian Time-of-Flight for Realtime Shape, Illumination and Albedo , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  John W. Fisher,et al.  Performance Guarantees for Information Theoretic Active Inference , 2007, AISTATS.

[11]  Ron Kimmel,et al.  Sparse Modeling of Shape from Structured Light , 2012, 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission.

[12]  John J. Leonard,et al.  Efficient scene simulation for robust monte carlo localization using an RGB-D camera , 2012, 2012 IEEE International Conference on Robotics and Automation.

[13]  John W. Fisher,et al.  Information-Driven Adaptive Structured-Light Scanners , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Katsushi Ikeuchi,et al.  Modeling sensor detectability with the VANTAGE geometric/sensor modeler , 1989, IEEE Trans. Robotics Autom..

[15]  Alfred M. Bruckstein,et al.  RGBD-fusion: Real-time high precision depth recovery , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Guillermo Sapiro,et al.  Low-Cost Compressive Sensing for Color Video and Depth , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  G. Klein,et al.  Parallel Tracking and Mapping for Small AR Workspaces , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[18]  Wolfram Burgard,et al.  Improved Techniques for Grid Mapping With Rao-Blackwellized Particle Filters , 2007, IEEE Transactions on Robotics.

[19]  Shree K. Nayar,et al.  Micro Phase Shifting , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Jonathan P. How,et al.  Sensor Selection in High-Dimensional Gaussian Trees with Nuisances , 2013, NIPS.

[21]  Sébastien Roy,et al.  A MRF formulation for coded structured light , 2005, Fifth International Conference on 3-D Digital Imaging and Modeling (3DIM'05).

[22]  John W. Fisher,et al.  On the Role of Representations for Reasoning in Large-Scale Urban Scenes , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[23]  Feng Zhao,et al.  Information-driven dynamic sensor collaboration , 2002, IEEE Signal Process. Mag..

[24]  Sebastian Thrun,et al.  Robotic mapping: a survey , 2003 .

[25]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[26]  Dieter Fox,et al.  A large-scale hierarchical multi-view RGB-D object dataset , 2011, 2011 IEEE International Conference on Robotics and Automation.

[27]  Derek Hoiem,et al.  Indoor Segmentation and Support Inference from RGBD Images , 2012, ECCV.

[28]  Yoav Y. Schechner,et al.  The Next Best Underwater View , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Stefano Soatto,et al.  Steps Towards a Theory of Visual Information: Active Perception, Signal-to-Symbol Conversion and the Interplay Between Sensing and Control , 2011, ArXiv.

[30]  Paul R. Cohen,et al.  Toward AI research methodology: three case studies in evaluation , 1989, IEEE Trans. Syst. Man Cybern..

[31]  Friedrich M. Wahl.  A Coded Light Approach for Depth Map Acquisition , 1986, DAGM-Symposium.

[32]  Mark R. Pickering,et al.  Dense depth estimation using adaptive structured light and cooperative algorithm , 2011, CVPR 2011 WORKSHOPS.

[33]  Joachim Denzler,et al.  Information Theoretic Sensor Data Selection for Active Object Recognition and State Estimation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Shree K. Nayar,et al.  Structured Light in Sunlight , 2013, 2013 IEEE International Conference on Computer Vision.

[35]  Zhiwei Xiong,et al.  Robust depth sensing with adaptive structured light illumination , 2014, J. Vis. Commun. Image Represent..

[36]  Luc Van Gool,et al.  Real-time range acquisition by adaptive structured light , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Wei Hong,et al.  Model-Driven Data Acquisition in Sensor Networks , 2004, VLDB.

[38]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[39]  John W. Fisher,et al.  Approximate Dynamic Programming for Communication-Constrained Sensor Network Management , 2007, IEEE Transactions on Signal Processing.

[40]  J. How,et al.  Information-rich Path Planning with General Constraints using Rapidly-exploring Random Trees , 2010 .

[41]  Vivek K. Goyal,et al.  Compressive depth map acquisition using a single photon-counting detector: Parametric signal processing meets sparsity , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Mac Schwager,et al.  Distributed robotic sensor networks: An information-theoretic approach , 2012, Int. J. Robotics Res..

[43]  Matthew O'Toole,et al.  Homogeneous codes for energy-efficient illumination and imaging , 2015, ACM Trans. Graph..

[44]  Andreas Krause,et al.  Near-Optimal Sensor Placements in Gaussian Processes: Theory, Efficient Algorithms and Empirical Studies , 2008, J. Mach. Learn. Res..

[45]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[46]  Shree K. Nayar,et al.  Programmable Imaging: Towards a Flexible Camera , 2006, International Journal of Computer Vision.

[47]  Joaquim Salvi,et al.  A state of the art in structured light patterns for surface profilometry , 2010, Pattern Recognit..

[48]  Lucas Paletta,et al.  Learning temporal context in active object recognition using Bayesian analysis , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[49]  John W. Fisher,et al.  Maximum Mutual Information Principle for Dynamic Sensor Query Problems , 2003, IPSN.