Gradient-Based Camera Exposure Control for Outdoor Mobile Platforms

We introduce a novel method to automatically adjust camera exposure for image processing and computer vision applications on mobile robot platforms. Because most image processing algorithms rely heavily on low-level image features that are based mainly on local gradient information, we consider that gradient quantity can determine the proper exposure level, allowing a camera to capture the important image features in a manner robust to illumination conditions. We then extend this concept to a multi-camera system and present a new control algorithm to achieve both brightness consistency between adjacent cameras and a proper exposure level for each camera. We implement our prototype system with off-the-shelf machine-vision cameras and demonstrate the effectiveness of the proposed algorithms on practical applications, including pedestrian detection, visual odometry, surround-view imaging, panoramic imaging, and stereo matching.

[1]  Navid Nourani-Vatani,et al.  Automatic camera exposure control , 2007 .

[2]  D. Lowe,et al.  Fast Matching of Binary Features , 2012, 2012 Ninth Conference on Computer and Robot Vision.

[3]  Sang Uk Lee,et al.  Robust Stereo Matching Using Adaptive Normalized Cross-Correlation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Julius Ziegler,et al.  StereoScan: Dense 3d reconstruction in real-time , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[5]  In-So Kweon,et al.  Capturing Village-level Heritages with a Hand-held Camera-Laser Fusion Sensor , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[6]  In-So Kweon,et al.  Automatic Content-Aware Color and Tone Stylization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Tetsuya Kuno,et al.  A new automatic exposure system for digital still cameras , 1998 .

[8]  Davide Scaramuzza,et al.  Active exposure control for robust visual odometry in HDR environments , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[9]  In-So Kweon,et al.  Auto-adjusting camera exposure for outdoor robotics using gradient information , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[10]  Nitin Sampat,et al.  System implications of implementing auto-exposure on consumer digital cameras , 1999, Electronic Imaging.

[11]  Edmund Y. Lam,et al.  Computationally Efficient Truncated Nuclear Norm Minimization for High Dynamic Range Imaging , 2016, IEEE Transactions on Image Processing.

[12]  Jan-Michael Frahm,et al.  Joint Feature Tracking and Radiometric Calibration from Auto-Exposure Video , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[13]  Zhengyou Zhang,et al.  Flexible camera calibration by viewing a plane from unknown orientations , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[14]  Kurt Konolige,et al.  Small Vision Systems: Hardware and Implementation , 1998 .

[15]  Armando J. Pinho,et al.  Autonomous Configuration of Parameters in Robotic Digital Cameras , 2009, IbPRIA.

[16]  Marc Pollefeys,et al.  Robust Radiometric Calibration and Vignetting Correction , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Tae-Hyun Oh,et al.  Robust High Dynamic Range Imaging by Rank Minimization , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.

[19]  Gordon Wyeth,et al.  Extending the dynamic range of robotic vision , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[20]  André Kaup,et al.  High dynamic range video by spatially non-regular optical filtering , 2012, 2012 19th IEEE International Conference on Image Processing.

[21]  Peter I. Corke,et al.  High dynamic range stereo vision for outdoor mobile robotics , 2009, 2009 IEEE International Conference on Robotics and Automation.

[22]  David A. McAllester,et al.  Object Detection with Discriminatively Trained Part Based Models , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Neil Barakat,et al.  Minimal-Bracketing Sets for High-Dynamic-Range Image Capture , 2008, IEEE Transactions on Image Processing.

[24]  Eli Shechtman,et al.  Robust patch-based hdr reconstruction of dynamic scenes , 2012, ACM Trans. Graph..

[25]  Shree K. Nayar,et al.  High dynamic range imaging: spatially varying pixel exposures , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[26]  Masayuki Murakami,et al.  An exposure control system of video cameras based on fuzzy logic using color information , 1996, Proceedings of IEEE 5th International Fuzzy Systems.

[27]  Huimin Lu,et al.  Camera parameters auto-adjusting technique for robust robot vision , 2010, 2010 IEEE International Conference on Robotics and Automation.

[28]  Shree K. Nayar,et al.  Ieee Transactions on Image Processing Computational Cameras: Convergence of Optics and Processing , 2022 .

[29]  Rob Fergus,et al.  Fast Image Deconvolution using Hyper-Laplacian Priors , 2009, NIPS.

[30]  Peirong Ji.  StereoScan : Dense 3 D Reconstruction in Real-time , 2016 .

[31]  Duarte Valério,et al.  Tuning of fractional PID controllers with Ziegler-Nichols-type rules , 2006, Signal Process..

[32]  Jan-Michael Frahm,et al.  Detailed Real-Time Urban 3D Reconstruction from Video , 2007, International Journal of Computer Vision.

[33]  Gonzalo Pajares,et al.  Acquisition of Agronomic Images with Sufficient Quality by Automatic Exposure Time Control and Histogram Matching , 2013, ACIVS.

[34]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[35]  Pietro Perona,et al.  Pedestrian Detection: An Evaluation of the State of the Art , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  In-So Kweon,et al.  Partial Sum Minimization of Singular Values in Robust PCA: Algorithm and Applications , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  J. G. Ziegler,et al.  Optimum Settings for Automatic Controllers , 1942, Journal of Fluids Engineering.

[38]  Sung-Jea Ko,et al.  An advanced video camera system with robust AF, AE, and AWB control , 2001, IEEE Trans. Consumer Electron..

[39]  In-So Kweon,et al.  An Autonomous Driving System for Unknown Environments Using a Unified Map , 2015, IEEE Transactions on Intelligent Transportation Systems.

[40]  Charles Poynton,et al.  Digital Video and HDTV Algorithms and Interfaces , 2012 .

[41]  Vincent Lepetit,et al.  BRIEF: Binary Robust Independent Elementary Features , 2010, ECCV.

[42]  Jitendra Malik,et al.  Recovering high dynamic range radiance maps from photographs , 1997, SIGGRAPH '08.

[43]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[44]  Alonzo Kelly,et al.  Linearized Error Propagation in Odometry , 2004, Int. J. Robotics Res..

[45]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[46]  Stephen Lin,et al.  A New In-Camera Imaging Model for Color Computer Vision and Its Application , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Steffen Beich,et al.  Digital Video And Hdtv Algorithms And Interfaces , 2016 .

[48]  Pradeep Sen,et al.  A versatile HDR video production system , 2011, ACM Trans. Graph..

[49]  Richard Szeliski,et al.  Creating full view panoramic image mosaics and environment maps , 1997, SIGGRAPH.

[50]  Luc Van Gool,et al.  The 2005 PASCAL Visual Object Classes Challenge , 2005, MLCW.

[51]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[52]  Aapo Hyvärinen,et al.  Natural Image Statistics - A Probabilistic Approach to Early Computational Vision , 2009, Computational Imaging and Vision.

[53]  Antonio Torralba,et al.  HOGgles: Visualizing Object Detection Features , 2013, 2013 IEEE International Conference on Computer Vision.

[54]  Jianguo Zhang,et al.  The PASCAL Visual Object Classes Challenge , 2006 .

[55]  David A. McAllester,et al.  A discriminatively trained, multiscale, deformable part model , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[56]  Jun Hu,et al.  HDR Deghosting: How to Deal with Saturation? , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[57]  ZissermanAndrew,et al.  The Pascal Visual Object Classes Challenge , 2015 .

[58]  Wen-Chung Kao,et al.  Real-time image fusion and adaptive exposure control for smart surveillance systems , 2007 .