Multispectral imaging using a stereo camera: concept, design and assessment

This paper proposes a one-shot six-channel multispectral color image acquisition system using a stereo camera and a pair of optical filters. The two filters from the best pair, selected from among readily available filters such that they modify the sensitivities of the two cameras in such a way that they produce optimal estimation of spectral reflectance and/or color, are placed in front of the two lenses of the stereo camera. The two images acquired from the stereo camera are then registered for pixel-to-pixel correspondence. The spectral reflectance and/or color at each pixel on the scene are estimated from the corresponding camera outputs in the two images. Both simulations and experiments have shown that the proposed system performs well both spectrally and colorimetrically. Since it acquires the multispectral images in one shot, the proposed system can solve the limitations of slow and complex acquisition process, and costliness of the state of the art multispectral imaging systems, leading to its possible uses in widespread applications.

[1]  Mohan M. Trivedi,et al.  Multi-Primitive Hierarchical (MPH) Stereo Analysis , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Christoph U. Keller,et al.  Persistent spectral hole burning used for spectrally high-resolved imaging of the sun , 1995, Defense, Security, and Sensing.

[3]  Daniel L. Farkas,et al.  Microscopic and mesoscopic spectral bioimaging , 1996, Photonics West.

[4]  Jon Yngve Hardeberg.  Acquisition and reproduction of colour images: colorimetric and multi-spectral approaches , 1999 .

[5]  Javier Hernández-Andrés,et al.  Linear bases for representation of natural and artificial illuminants , 1997 .

[6]  Flávio P. Ferreira,et al.  Statistics of spatial cone-excitation ratios in natural scenes. , 2002, Journal of the Optical Society of America. A, Optics, image science, and vision.

[7]  Takeo Kanade,et al.  A Cooperative Algorithm for Stereo Matching and Occlusion Detection , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Mårten Gulliksson,et al.  The Use and Properties of Tikhonov Filter Matrices , 2000, SIAM J. Matrix Anal. Appl..

[9]  Peter D. Burns,et al.  Multispectral-based color reproduction research at the Munsell Color Science Laboratory , 1998, Other Conferences.

[10]  Shree K. Nayar,et al.  Multispectral Imaging Using Multiplexed Illumination , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[11]  L. Maloney.  Evaluation of linear models of surface spectral reflectance with small numbers of parameters. , 1986, Journal of the Optical Society of America. A, Optics and image science.

[12]  Philip H. Swain,et al.  Remote Sensing: The Quantitative Approach , 1981, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Toshio Uchiyama,et al.  Color image reproduction based on multispectral and multiprimary imaging: experimental evaluation , 2001, IS&T/SPIE Electronic Imaging.

[14]  Hideaki Haneishi,et al.  Six Band HDTV Camera System for Spectrum-Based Color Reproduction , 2004, Journal of Imaging Science and Technology.

[15]  R. Berns,et al.  A Comparative Analysis of Spectral Reflectance Estimated in Various Spaces Using a Trichromatic Camera System , 2000, Journal of Imaging Science and Technology.

[16]  Peter D. Burns,et al.  Analysis Multispectral Image Capture , 1996, CIC.

[17]  Masahiro Yamaguchi,et al.  Natural color reproduction in the television system for telemedicime , 1997, Medical Imaging.

[18]  Leonardo Vanneschi,et al.  Polynomial modeling and optimization for colorimetric characterization of scanners , 2008, J. Electronic Imaging.

[19]  Hideaki Haneishi,et al.  Beyond Red-Green-Blue (RGB) : Spectrum-Based Color Imaging Technology , 2008 .

[21]  Roy S. Berns,et al.  Challenges for Color Science in Multimedia Imaging , 1998 .

[22]  Motonori Doi,et al.  Spectral Estimation of Skin Color with Foundation Makeup , 2005, SCIA.

[23]  Arvid Lundervold,et al.  Multispectral analysis of the brain using magnetic resonance imaging , 1994, IEEE Trans. Medical Imaging.

[24]  Stephen Westland,et al.  Comparison of linear spectral reconstruction methods for multispectral imaging , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[25]  George E. Meyer,et al.  Optics in Agriculture , 1991 .

[26]  Norimichi Tsumura.  Appearance reproduction and multispectral imaging , 2006 .

[27]  Yi-Ping Hung,et al.  Multipass hierarchical stereo matching for generation of digital terrain models from aerial images , 1998, Machine Vision and Applications.

[28]  Francisco H. Imai,et al.  Multi-spectral Image Acquisition and Spectral Reconstruction using a Trichromatic Digital Camera System associated with absorption filters PART VIII General Discussion , 1998 .

[29]  M H Horman.  Temperature analysis from multispectral infrared data. , 1976, Applied optics.

[30]  Peter J. Miller,et al.  Multispectral imaging with a liquid crystal tunable filter , 1995, Other Conferences.

[31]  Raimondo Schettini,et al.  Selection of Training Sets for the Characterisation of Multispectral Imaging Systems , 2003, PICS.

[32]  Mark S. Peercy,et al.  Linear color representations for full speed spectral rendering , 1993, SIGGRAPH.

[33]  David R. Bull,et al.  Projective image restoration using sparsity regularization , 2013, 2013 IEEE International Conference on Image Processing.

[34]  David Connah,et al.  Spectral recovery using polynomial models , 2005, IS&T/SPIE Electronic Imaging.

[35]  Junko Kishimoto,et al.  Two-Shot type 6-band still image capturing system using Commercial Digital Camera and Custom Color Filter , 2008, CGIV/MCS.

[36]  Shoji Tominaga,et al.  Spectral imaging by a multichannel camera , 1998, Electronic Imaging.

[37]  Eli Saber,et al.  Automated extraction of fire line parameters from multispectral infrared images , 2007 .

[38]  Raimondo Schettini,et al.  Selection of filters for multispectral acquisition using the filter vectors analysis method , 2003, IS&T/SPIE Electronic Imaging.

[39]  Shoji Tominaga,et al.  MULTICHANNEL VISION SYSTEM FOR ESTIMATING SURFACE AND ILLUMINATION FUNCTIONS , 1996 .

[40]  Hans Brettel,et al.  Multispectral color image capture using a liquid crystal tunable filter , 2002 .

[41]  Brian V. Funt,et al.  Camera characterization for color research , 2002 .

[42]  W K Pratt,et al.  Spectral estimation techniques for the spectral calibration of a color image scanner. , 1976, Applied optics.

[43]  Francisco H. Imai,et al.  Comparative study of spectral reflectance estimation based on broad-band imaging systems , 2003 .

[44]  Paul J. Curran,et al.  Imaging spectrometry , 1994 .

[45]  No Value,et al.  IEEE International Conference on Image Processing , 2003 .

[46]  T. Stephenson.  Image analysis , 1992, Nature.

[47]  A. Ardeshir Goshtasby,et al.  Image registration by local approximation methods , 1988, Image Vis. Comput..

[48]  Kinjiro Amano,et al.  Recovering spectral data from natural scenes with an RGB digital camera and colored filters , 2007 .

[49]  Jon Y. Hardeberg,et al.  Filter Selection for Multispectral Color Image Acquisition , 2004, PICS.

[50]  David Connah,et al.  Multispectral Imaging: How Many Sensors Do We Need? , 2004, Color Imaging Conference.

[51]  David C Day.  Filter Selection for Spectral Estimation Using a Trichrmatic Camera , 2003 .

[52]  黃灝雄.  Acquisition of Multispectral Images Using Digital Cameras , 2004 .

[53]  Franck Marzani,et al.  Neural networks in two cascade algorithms for spectral reflectance reconstruction , 2005, IEEE International Conference on Image Processing 2005.

[54]  Roy S. Berns,et al.  Spectral Estimation Using Trichromatic Digital Cameras , 1999 .

[55]  Donald W. Hillger,et al.  Improved detection of airborne volcanic ash using multispectral infrared satellite data , 2003 .

[56]  Jon Y. Hardeberg,et al.  Spectrophotometric Image Analysis of Fine Art Paintings , 1996, CIC.