Single-Scale Fusion: An Effective Approach to Merging Images

Due to its robustness and effectiveness, multi-scale fusion (MSF) based on the Laplacian pyramid decomposition has emerged as a popular technique that has shown utility in many applications. Guided by several intuitive measures (weight maps) the MSF process is versatile and straightforward to be implemented. However, the number of pyramid levels increases with the image size, which implies sophisticated data management and memory accesses, as well as additional computations. Here, we introduce a simplified formulation that reduces MSF to only a single level process. Starting from the MSF decomposition, we explain both mathematically and intuitively (visually) a way to simplify the classical MSF approach with minimal loss of information. The resulting single-scale fusion (SSF) solution is a close approximation of the MSF process that eliminates important redundant computations. It also provides insights regarding why MSF is so effective. While our simplified expression is derived in the context of high dynamic range imaging, we show its generality on several well-known fusion-based applications, such as image compositing, extended depth of field, medical imaging, and blending thermal (infrared) images with visible light. Besides visual validation, quantitative evaluations demonstrate that our SSF strategy is able to yield results that are highly competitive with traditional MSF approaches.

[1]  Rafal Mantiuk,et al.  Display adaptive tone mapping , 2008, SIGGRAPH 2008.

[2]  Neil A. Dodgson,et al.  Cross Dissolve Without Cross Fade: Preserving Contrast, Color and Salience in Image Compositing , 2006, Comput. Graph. Forum.

[3]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[4]  J. Rolet,et al.  Extraction of spectral information from Landsat TM data and merger with SPOT panchromatic imagery: a contribution to the study of geological structures , 1993 .

[5]  Codruta O. Ancuti,et al.  Enhancing underwater images and videos by fusion , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Yulong Shen,et al.  Registration and fusion of retinal images-an evaluation study , 2003, IEEE Transactions on Medical Imaging.

[7]  S. Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, CVPR 2009.

[8]  Zhou Wang,et al.  Reduced- and No-Reference Image Quality Assessment , 2011, IEEE Signal Processing Magazine.

[9]  Leonard McMillan,et al.  Multispectral Bilateral Video Fusion , 2007, IEEE Transactions on Image Processing.

[10]  Oliver Rockinger,et al.  Image sequence fusion using a shift-invariant wavelet transform , 1997, Proceedings of International Conference on Image Processing.

[11]  Belur V. Dasarathy,et al.  Medical Image Fusion: A survey of the state of the art , 2013, Inf. Fusion.

[12]  Stavri G. Nikolov,et al.  Image fusion: Advances in the state of the art , 2007, Inf. Fusion.

[13]  Aleksandra Pizurica,et al.  Extending the Depth of Field in Microscopy Through Curvelet-Based Frequency-Adaptive Image Fusion , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[14]  Jan Kautz,et al.  Exposure Fusion , 2009, 15th Pacific Conference on Computer Graphics and Applications (PG'07).

[15]  Frédo Durand,et al.  Fast Local Laplacian Filters , 2014, ACM Trans. Graph..

[16]  David Salesin,et al.  Interactive digital photomontage , 2004, SIGGRAPH 2004.

[17]  Pramod K. Varshney,et al.  An Image Fusion Approach Based on Markov Random Fields , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Erik Reinhard,et al.  Ieee Transactions on Visualization and Computer Graphics 1 Dynamic Range Reduction Inspired by Photoreceptor Physiology , 2022 .

[19]  Erik Reinhard,et al.  Photographic tone reproduction for digital images , 2002, ACM Trans. Graph..

[20]  Alan C. Bovik,et al.  Automatic Prediction of Perceptual Image and Video Quality , 2013, Proceedings of the IEEE.

[21]  Zhou Wang,et al.  Modern Image Quality Assessment , 2006, Modern Image Quality Assessment.

[22]  James R. Bergen,et al.  Pyramid-based texture analysis/synthesis , 1995, Proceedings., International Conference on Image Processing.

[23]  Wenzhong Shi,et al.  Remote Sensing Image Fusion Using Multiscale Mapped LS-SVM , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[24]  Jianbo Shi,et al.  Generalized Random Walks for Fusion of Multi-Exposure Images , 2011, IEEE Transactions on Image Processing.

[25]  Gauthier Lafruit,et al.  High-Level Cache Modeling for 2-D Discrete Wavelet Transform Implementations , 2003, J. VLSI Signal Process..

[26]  Zhou Wang,et al.  Objective Quality Assessment of Tone-Mapped Images , 2013, IEEE Transactions on Image Processing.

[27]  Ding Liu,et al.  Image Fusion Using Higher Order Singular Value Decomposition , 2012, IEEE Transactions on Image Processing.

[28]  Jinshan Tang,et al.  A contrast based image fusion technique in the DCT domain , 2004, Digit. Signal Process..

[29]  Karol Myszkowski,et al.  Adaptive Logarithmic Mapping For Displaying High Contrast Scenes , 2003, Comput. Graph. Forum.

[30]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[31]  Zhou Wang,et al.  Information Content Weighting for Perceptual Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[32]  Ramesh Raskar,et al.  Image fusion for context enhancement and video surrealism , 2004, NPAR '04.

[33]  Shutao Li,et al.  Image Fusion With Guided Filtering , 2013, IEEE Transactions on Image Processing.

[34]  Codruta O. Ancuti,et al.  Single Image Dehazing by Multi-Scale Fusion , 2013, IEEE Transactions on Image Processing.

[35]  Jian Sun,et al.  Guided Image Filtering , 2010, ECCV.

[36]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[37]  Christophe De Vleeschouwer,et al.  Human visual system features enabling watermarking , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[38]  Rudy Lauwereins,et al.  Modeling and exploiting spatial locality trade-offs in wavelet-based applications under varying resource requirements , 2010, TECS.

[39]  Cosmin Ancuti,et al.  Image and Video Decolorization by Fusion , 2010, ACCV.

[40]  Qiang Zhang,et al.  Multifocus image fusion using the nonsubsampled contourlet transform , 2009, Signal Process..

[41]  Vladimir S. Petrovic,et al.  Gradient-based multiresolution image fusion , 2004, IEEE Transactions on Image Processing.

[42]  Alan Conrad Bovik,et al.  Referenceless Prediction of Perceptual Fog Density and Perceptual Image Defogging , 2015, IEEE Transactions on Image Processing.

[43]  Minh N. Do,et al.  Ieee Transactions on Image Processing the Contourlet Transform: an Efficient Directional Multiresolution Image Representation , 2022 .

[44]  Cedric Nishan Canagarajah,et al.  Pixel- and region-based image fusion with complex wavelets , 2007, Inf. Fusion.

[45]  Allen M. Waxman,et al.  Fusion of multi-sensor imagery for night vision: color visualization, target learning and search , 2000, Proceedings of the Third International Conference on Information Fusion.

[46]  Robert W. Heath,et al.  Design of Linear Equalizers Optimized for the Structural Similarity Index , 2008, IEEE Transactions on Image Processing.

[47]  B. S. Manjunath,et al.  Multisensor Image Fusion Using the Wavelet Transform , 1995, CVGIP Graph. Model. Image Process..

[48]  Richard Szeliski,et al.  Digital photography with flash and no-flash image pairs , 2004, ACM Trans. Graph..

[49]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[50]  Dani Lischinski,et al.  Gradient Domain High Dynamic Range Compression , 2023 .