A single sensor based multispectral imaging camera using a narrow spectral band color mosaic integrated on the monochrome CMOS image sensor

A multispectral image camera captures image data within specific wavelength ranges in narrow wavelength bands across the electromagnetic spectrum. Images from a multispectral camera can extract a additional information that the human eye or a normal camera fails to capture and thus may have important applications in precision agriculture, forestry, medicine, and object identification. Conventional multispectral cameras are made up of multiple image sensors each fitted with a narrow passband wavelength filter and optics, which makes them heavy, bulky, power hungry, and very expensive. The multiple optics also create an image co-registration problem. Here, we demonstrate a single sensor based three band multispectral camera using a narrow spectral band red–green–blue color mosaic in a Bayer pattern integrated on a monochrome CMOS sensor. The narrow band color mosaic is made of a hybrid combination of plasmonic color filters and a heterostructured dielectric multilayer. The demonstrated camera technology has reduced cost, weight, size, and power by almost n times (where n is the number of bands) compared to a conventional multispectral camera.

[1]  Harry A Atwater,et al.  Color imaging via nearest neighbor hole coupling in plasmonic color filters integrated onto a complementary metal-oxide semiconductor image sensor. , 2013, ACS nano.

[2]  Yasuyuki Ichihashi,et al.  High-performance parallel computing for next-generation holographic imaging , 2018, Nature Electronics.

[3]  S. Maier Plasmonics: Fundamentals and Applications , 2007 .

[4]  Jeong-Bong Lee,et al.  Axially-Anisotropic Hierarchical Grating 2D Guided-Mode Resonance Strain-Sensor , 2019, Sensors.

[5]  Billy Lam,et al.  Plasmonic metasurfaces with 42.3% transmission efficiency in the visible , 2019, Light: Science & Applications.

[6]  Da Xing,et al.  Co-impulse multispectral photoacoustic microscopy and optical coherence tomography system using a single supercontinuum laser. , 2019, Optics letters.

[7]  Agnès Bégué,et al.  Can Commercial Digital Cameras Be Used as Multispectral Sensors? A Crop Monitoring Test , 2008, Sensors.

[8]  Benjamin Gallinet,et al.  Multispectral Imaging with Tunable Plasmonic Filters , 2017 .

[9]  Mingyuan Gao,et al.  Multispectral optoacoustic imaging of dynamic redox correlation and pathophysiological progression utilizing upconversion nanoprobes , 2019, Nature Communications.

[10]  G. Si,et al.  Annular aperture array based color filter , 2011 .

[11]  Mitsumasa Nakajima,et al.  High-Sensitivity Color Imaging Using Pixel-Scale Color Splitters Based on Dielectric Metasurfaces , 2019, ACS Photonics.

[12]  Harry A. Atwater,et al.  High Spectral Resolution Plasmonic Color Filters with Subwavelength Dimensions , 2019, ACS Photonics.

[13]  H. Bethe Theory of Diffraction by Small Holes , 1944 .

[14]  Timothy D. Wilkinson,et al.  Enhanced reflection from arrays of silicon based inverted nanocones , 2011 .

[15]  Xiao-Ning Pang,et al.  A broadband achromatic metalens array for integral imaging in the visible , 2019, Light: Science & Applications.

[16]  Timothy D Wilkinson,et al.  Electro-optic characteristics of a transparent nanophotonic device based on carbon nanotubes and liquid crystals. , 2010, Applied optics.

[17]  Filbert J. Bartoli,et al.  Ultrathin Nanostructured Metals for Highly Transmissive Plasmonic Subtractive Color Filters , 2014, 2014 Conference on Lasers and Electro-Optics (CLEO) - Laser Science to Photonic Applications.

[18]  Peter Nordlander,et al.  Color‐Selective and CMOS‐Compatible Photodetection Based on Aluminum Plasmonics , 2014, Advanced materials.

[19]  Tsuyoshi Nomura,et al.  Polarization independent visible color filter comprising an aluminum film with surface-plasmon enhanced transmission through a subwavelength array of holes , 2011 .

[20]  Torbjørn Skauli An upper-bound metric for characterizing spectral and spatial coregistration errors in spectral imaging. , 2012, Optics express.

[21]  Haider Butt,et al.  Can Nanotubes Make a Lens Array? , 2012, Advanced materials.

[22]  A. Rakić,et al.  Algorithm for the determination of intrinsic optical constants of metal films: application to aluminum. , 1995, Applied optics.

[23]  Qin Chen,et al.  CMOS Photodetectors Integrated With Plasmonic Color Filters , 2012, IEEE Photonics Technology Letters.

[24]  Ki-Dong Lee,et al.  Color filter based on a subwavelength patterned metal grating. , 2007, Optics express.

[25]  Na Liu,et al.  Dynamic plasmonic colour display , 2017, Nature Communications.

[26]  Robert Magnusson,et al.  Guided-mode resonant polarization-controlled tunable color filters. , 2014, Optics express.

[27]  Robert Magnusson,et al.  Efficient band-pass color filters enabled by resonant modes and plasmons near the Rayleigh anomaly. , 2014, Optics express.

[28]  Harry A Atwater,et al.  Plasmonic color filters for CMOS image sensor applications. , 2012, Nano letters.

[29]  Beibei Zeng,et al.  Angle-insensitive plasmonic color filters with randomly distributed silver nanodisks. , 2015, Optics letters.

[30]  Kaipeng Zhang,et al.  2D non‐polarising transmission filters based on GMR for optical communications , 2018, Micro & Nano Letters.

[31]  Hyunsung Park,et al.  Multispectral imaging with vertical silicon nanowires , 2013, Scientific Reports.

[32]  Harry Atwater,et al.  Hyper-selective plasmonic color filters. , 2016, Optics express.

[33]  Michael Scalora,et al.  Transparent, metallo-dielectric, one-dimensional, photonic band-gap structures , 1998 .

[34]  Wen-Kai Kuo,et al.  Two-dimensional grating guided-mode resonance tunable filter. , 2017, Optics express.

[35]  H. Lezec,et al.  Extraordinary optical transmission through sub-wavelength hole arrays , 1998, Nature.

[36]  Anastasios Koulaouzidis,et al.  Use of Hyperspectral/Multispectral Imaging in Gastroenterology. Shedding Some–Different–Light into the Dark , 2019, Journal of clinical medicine.

[37]  Efstratios Skafidas,et al.  Plasmonic Colour Filters Based on Coaxial Holes in Aluminium , 2017, Materials.

[38]  Ampalavanapillai Nirmalathas,et al.  Plasmonic Narrow Bandpass Filters Based on Metal-Dielectric-Metal for Multispectral Imaging , 2018, 2018 Conference on Lasers and Electro-Optics Pacific Rim (CLEO-PR).

[39]  Keith A. Jenkins,et al.  Flexible CMOS integrated circuits based on carbon nanotubes with sub-10 ns stage delays , 2018 .

[40]  Hideto Miyake,et al.  Fabrication and optical characterization of a 2D metal periodic grating structure for cold filter application , 2015, SPIE Micro + Nano Materials, Devices, and Applications.

[41]  Efstratios Skafidas,et al.  Filling schemes at submicron scale: Development of submicron sized plasmonic colour filters , 2014, Scientific Reports.

[42]  Igal Brener,et al.  Active tuning of all-dielectric metasurfaces. , 2015, ACS nano.

[43]  Ali Javey,et al.  Wearable sweat sensors , 2018 .

[44]  Chenghai Yang,et al.  Registration for Optical Multimodal Remote Sensing Images Based on FAST Detection, Window Selection, and Histogram Specification , 2018, Remote. Sens..

[45]  Ampalavanapillai Nirmalathas,et al.  Transmission Enhancement in Coaxial Hole Array Based Plasmonic Color Filter for Image Sensor Applications , 2018, IEEE Photonics Journal.

[46]  Chang-Hyun Park,et al.  Highly Efficient Color Filter Incorporating a Thin Metal–Dielectric Resonant Structure , 2012 .

[47]  Beibei Zeng,et al.  Effect of relative nanohole position on colour purity of ultrathin plasmonic subtractive colour filters. , 2015, Nanotechnology.

[48]  Ki-Dong Lee,et al.  Color filter based on a subwavelength patterned metal grating , 2007 .

[49]  Yuzhang Liang,et al.  Free-standing plasmonic metal-dielectric-metal bandpass filter with high transmission efficiency , 2017, Scientific Reports.

[50]  S. Goossens,et al.  Broadband image sensor array based on graphene–CMOS integration , 2017, Nature Photonics.

[51]  T. Ebbesen,et al.  Light in tiny holes , 2007, Nature.

[52]  Chung-Kun Yen,et al.  Vein Pattern Locating Technology for Cannulation: A Review of the Low-Cost Vein Finder Prototypes Utilizing near Infrared (NIR) Light to Improve Peripheral Subcutaneous Vein Selection for Phlebotomy , 2019, Sensors.