Generalized Mosaicing: Wide Field of View Multispectral Imaging

We present an approach to significantly enhance the spectral resolution of imaging systems by generalizing image mosaicing. A filter transmitting spatially varying spectral bands is rigidly attached to a camera. As the system moves, it senses each scene point multiple times, each time in a different spectral band. This is an additional dimension of the generalized mosaic paradigm, which has demonstrated yielding high radiometric dynamic range images in a wide field of view, using a spatially varying density filter. The resulting mosaic represents the spectrum at each scene point. The image acquisition is as easy as in traditional image mosaics. We derive an efficient scene sampling rate, and use a registration method that accommodates the spatially varying properties of the filter. Using the data acquired by this method, we demonstrate scene rendering under different simulated illumination spectra. We are also able to infer information about the scene illumination. The approach was tested using a standard 8-bit black/white video camera and a fixed spatially varying spectral (interference) filter.

[1]  Ralph Bernstein,et al.  Digital Image Processing of Earth Observation Sensor Data , 1976, IBM J. Res. Dev..

[2]  E. Eliason,et al.  Global color variations on the Martian surface , 1978 .

[3]  John B. Wellman.  Multispectral Mapper: Imaging Spectroscopy As Applied To The Mapping Of Earth Resources , 1981, Photonics West - Lasers and Applications in Science and Engineering.

[4]  Edward H. Adelson,et al.  A multiresolution spline with application to image mosaics , 1983, TOGS.

[5]  J. Usón,et al.  The Central Galaxy in Abell 2029: An Old Supergiant , 1990, Science.

[6]  Aram M. Mika.  Linear-wedge spectrometer , 1990, Other Conferences.

[7]  J. Colin Mahoney,et al.  Thermal Infrared Imaging Spectrometer: an advanced optics technology instrument , 1990, Other Conferences.

[8]  John C. Curlander,et al.  An automated system for mosaicking spaceborne SAR imagery , 1990 .

[9]  K. M. Merrill,et al.  Infrared Images of M17 , 1991 .

[10]  M. Landy,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[11]  J. M. Anderson,et al.  A spatially variable light-frequency-selective component-based, airborne pushbroom imaging spectrometer for the water environment , 1993 .

[12]  Sidney F. Ray,et al.  Applied Photographic Optics: Lenses and Optical Systems for Photography, Film, Video and Electronic Imaging , 1994 .

[13]  Richard Szeliski,et al.  Image mosaicing for tele-reality applications , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[14]  E. Reynoso,et al.  VLA Observations of Neutral Hydrogen in the Direction of Puppis A , 1995 .

[15]  Andrew S. Glassner,et al.  Principles of Digital Image Synthesis , 1995 .

[16]  G. Monnet.  3D Spectroscopy with large Telescopes: Past, Present and Prospects , 1995 .

[17]  P. Anandan,et al.  Efficient representations of video sequences and their applications , 1996, Signal Process. Image Commun..

[18]  Harpreet S. Sawhney,et al.  Compact Representations of Videos Through Dominant and Multiple Motion Estimation , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Richard W. Hall,et al.  Mosaic image generation on a flattened Gaussian sphere , 1996, Proceedings Third IEEE Workshop on Applications of Computer Vision. WACV'96.

[20]  Sleve Mann,et al.  'Pencigraphy' with AGC: joint parameter estimation in both domain and range of functions in same orbit of the projective-Wyckoff group , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[21]  Shmuel Peleg,et al.  Panoramic mosaics by manifold projection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  Ravi K. Sharma,et al.  Multisensor image registration , 1997 .

[23]  Marie-Lise Duplaquet,et al.  Building large image mosaics with invisible seam lines , 1998, Defense, Security, and Sensing.

[24]  P. Anandan,et al.  Robust multi-sensor image alignment , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[25]  Ronald J. Rapp,et al.  High-resolution airborne imaging spectrometer , 1998, Optics & Photonics.

[26]  A. Vasavada,et al.  Galileo Imaging of Jupiter's Atmosphere: The Great Red Spot, Equatorial Region, and White Ovals , 1998 .

[27]  Eero P. Simoncelli,et al.  Range estimation by optical differentiation. , 1998, Journal of the Optical Society of America. A, Optics, image science, and vision.

[28]  Andrew Zisserman,et al.  Automated mosaicing with super-resolution zoom , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[29]  Harpreet S. Sawhney,et al.  VideoBrush/sup TM/: experiences with consumer video mosaicing , 1998, Proceedings Fourth IEEE Workshop on Applications of Computer Vision. WACV'98 (Cat. No.98EX201).

[30]  Robert Garfinkel,et al.  Mosaicking of Aerial Photographic Maps Via Seams Defined by Bottleneck Shortest Paths , 1998, Oper. Res..

[31]  Seth J. Teller,et al.  Acquisition of a large pose-mosaic dataset , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[32]  Luciano Alparone,et al.  Color constancy from multispectral images , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[33]  K. Ikeuchi,et al.  Illumination distribution from brightness in shadows: Adaptive estimation of illumination distribution with unknown reflectance properties in shadow regions , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[34]  Glenn Healey,et al.  Material classification for 3D objects in aerial hyperspectral images , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[35]  Theo Gevers,et al.  Color Measurement by Imaging Spectrometry , 2000, Comput. Vis. Image Underst..

[36]  Nahum Gat,et al.  Imaging spectroscopy using tunable filters: a review , 2000, SPIE Defense + Commercial Sensing.

[37]  Narendra Ahuja,et al.  High dynamic range panoramic imaging , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[38]  S. Nayar,et al.  Generalized mosaicing , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[39]  Robert B. Kerr,et al.  Filters expand capabilities of infrared imaging , 2001 .

[40]  Michael J. Black,et al.  On the unification of line processes, outlier rejection, and robust statistics with applications in early vision , 1996, International Journal of Computer Vision.

[41]  Shree K. Nayar,et al.  Generalized Mosaicing : High Dynamic Range in a Wide Field of View 247 , 2001 .

[42]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[43]  Richard Szeliski,et al.  Systems and Experiment Paper: Construction of Panoramic Image Mosaics with Global and Local Alignment , 2000, International Journal of Computer Vision.

[44]  Anil K. Jain.  Fundamentals of Digital Image Processing , 2018, Control of Color Imaging Systems.