HDR Imaging Pipeline for Spectral Filter Array Cameras

Multispectral single shot imaging systems can benefit computer vision applications in needs of a compact and affordable imaging system. Spectral filter arrays technology meets the requirement, but can lead to artifacts due to inhomogeneous intensity levels between spectral channels due to filter manufacturing constraints, illumination and object properties. One solution to solve this problem is to use high dynamic range imaging techniques on these sensors. We define a spectral imaging pipeline that incorporates high dynamic range, demosaicing and color image visualization. Qualitative evaluation is based on real images captured with a prototype of spectral filter array sensor in the visible and near infrared.

[1]  Paul M. Simon.  Single Shot High Dynamic Range and Multispectral Imaging Based on Properties of Color Filter Arrays , 2011 .

[2]  Robert L. Stevenson,et al.  Dynamic range improvement through multiple exposures , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[3]  Young J. Yu,et al.  Vertical silicon nanowire photodetectors: Spectral sensitivity via nanowire radius , 2013, CLEO: 2013.

[4]  Gaurav Sharma,et al.  Digital color imaging , 1997, IEEE Trans. Image Process..

[5]  Dominique Ginhac,et al.  Real-time ghost free HDR video stream generation using weight adaptation based method , 2016, ICDSC.

[6]  Kurt Debattista,et al.  Advanced High Dynamic Range Imaging: Theory and Practice , 2011 .

[7]  Hans-Peter Seidel,et al.  Lightness Perception in Tone Reproduction for High Dynamic Range Images , 2005, Comput. Graph. Forum.

[8]  Dominique Ginhac,et al.  3D-IC: New Perspectives for a Digital Pixel Sensor , 2016, ICDSC.

[9]  Steve Mann,et al.  Realtime HDR (High Dynamic Range) video for eyetap wearable computers, FPGA-based seeing aids, and glasseyes (EyeTaps) , 2012, 2012 25th IEEE Canadian Conference on Electrical and Computer Engineering (CCECE).

[10]  Masatoshi Okutomi,et al.  Simultaneous capturing of RGB and additional band images using hybrid color filter array , 2014, Electronic Imaging.

[11]  Touradj Ebrahimi,et al.  Benchmarking of objective quality metrics for HDR image quality assessment , 2015, EURASIP Journal on Image and Video Processing.

[12]  Shree K. Nayar,et al.  Radiometric self calibration , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[13]  Yassine Ruichek,et al.  Energy balance in single exposure multispectral sensors , 2013, 2013 Colour and Visual Computing Symposium (CVCS).

[14]  Carolina Blanch,et al.  A tiny VIS-NIR snapshot multispectral camera , 2015, Photonics West - Optoelectronic Materials and Devices.

[15]  Takeo Kanade,et al.  Statistical calibration of CCD imaging process , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[16]  W.E. Snyder,et al.  Color image processing pipeline , 2005, IEEE Signal Processing Magazine.

[17]  Masatoshi Okutomi,et al.  A Practical One-Shot Multispectral Imaging System Using a Single Image Sensor , 2015, IEEE Transactions on Image Processing.

[18]  Yassine Ruichek,et al.  Energy balance in Spectral Filter Array camera design , 2017, Journal of the European Optical Society-Rapid Publications.

[19]  Dingrong Yi,et al.  Fabrication of multispectral imaging technology driven MEMS-based micro-arrayed multichannel optical filter mosaic , 2011, MOEMS-MEMS.

[20]  Pierre Gouton,et al.  Multispectral Filter Arrays: Recent Advances and Practical Implementation , 2014, Sensors.

[21]  Pierre Gouton,et al.  Spectral Characterization of a Prototype SFA Camera for Joint Visible and NIR Acquisition , 2016, Sensors.

[22]  Wen-Chung Kao,et al.  Design considerations of color image processing pipeline for digital cameras , 2006, IEEE Transactions on Consumer Electronics.

[23]  Guoping Qiu,et al.  Tone-mapping high dynamic range images by novel histogram adjustment , 2010, Pattern Recognit..

[24]  Erik Reinhard,et al.  Photographic tone reproduction for digital images , 2002, ACM Trans. Graph..

[25]  Dominique Ginhac,et al.  A 1.3 megapixel FPGA-based smart camera for high dynamic range real time video , 2013, 2013 Seventh International Conference on Distributed Smart Cameras (ICDSC).

[26]  Dominique Ginhac,et al.  HDR-ARtiSt: an adaptive real-time smart camera for high dynamic range imaging , 2014, Journal of Real-Time Image Processing.

[27]  ShirleyPeter,et al.  Photographic tone reproduction for digital images , 2002 .

[28]  Jason M. Eichenholz,et al.  Ultracompact fully integrated megapixel multispectral imager , 2009, OPTO.

[29]  Jitendra Malik,et al.  Recovering high dynamic range radiance maps from photographs , 1997, SIGGRAPH '08.

[30]  W. Jino Hans,et al.  FPGA implementation of bilinear interpolation algorithm for CFA demosaicing , 2013, 2013 International Conference on Communication and Signal Processing.

[31]  Jean-Baptiste Thomas,et al.  Illuminant estimation from uncalibrated multispectral images , 2015, 2015 Colour and Visual Computing Symposium (CVCS).

[32]  G. W. Larson,et al.  Rendering with radiance - the art and science of lighting visualization , 2004, Morgan Kaufmann series in computer graphics and geometric modeling.

[33]  Laurence Meylan,et al.  Digital camera workflow for high dynamic range images using a model of retinal processing , 2008, Electronic Imaging.

[34]  Nam Ik Cho,et al.  Probabilistic motion pixel detection for the reduction of ghost artifacts in high dynamic range images from multiple exposures , 2014, EURASIP J. Image Video Process..

[35]  Sabine Süsstrunk,et al.  Multiscale guided deblurring: Chromatic aberration correction in color and near-infrared imaging , 2015, 2015 23rd European Signal Processing Conference (EUSIPCO).

[36]  Miguel Granados,et al.  Automatic noise modeling for ghost-free HDR reconstruction , 2013, ACM Trans. Graph..

[37]  Sabine Süsstrunk,et al.  Demultiplexing Visible and Near-Infrared Information in Single- Sensor Multispectral Imaging , 2016 .

[38]  B. Dang,et al.  3D silicon integration , 2008, 2008 58th Electronic Components and Technology Conference.

[39]  Wesley E. Snyder,et al.  Binary Tree-based Generic Demosaicking Algorithm for Multispectral Filter Arrays , 2006, IEEE Transactions on Image Processing.

[40]  Dominique Ginhac,et al.  Hardware-based smart camera for recovering high dynamic range video from multiple exposures , 2014 .

[41]  Jie Jia,et al.  Fourier Spectral Filter Array for Optimal Multispectral Imaging , 2016, IEEE Transactions on Image Processing.