A comparative study of conventional and deep learning approaches for demosaicing Mastcam images

Bayer pattern is a low cost approach to generating RGB images in commercial digital cameras. In NASA's mast camera (Mastcams) onboard the Mars rover Curiosity, Bayer pattern has also been used in capturing the RGB bands. It is well known that debayering (also known as demosaicing) introduces color and zipper artifacts. Currently, NASA is using a demosaicing algorithm developed in early 2000’s. It is probably the right time to assess some state-of-the-art algorithms and recommend a more recent and powerful approach to NASA for its future missions. In this paper, we present results of a comparative study on the use of conventional and deep learning algorithms for demosaicing Mastcam images. Due to lack of ground truth, subjective evaluation has been used in our study.

[1]  Chiman Kwan,et al.  Perceptually Lossless Compression for Mastcam Images , 2018, 2018 9th IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON).

[2]  Hairong Qi,et al.  A Novel and Effective Multivariate Method for Compositional Analysis using Laser Induced Breakdown Spectroscopy , 2014 .

[3]  Chiman Kwan,et al.  Debayering RGBW color filter arrays: A pansharpening approach , 2017, 2017 IEEE 8th Annual Ubiquitous Computing, Electronics and Mobile Communication Conference (UEMCON).

[4]  Frédo Durand,et al.  Deep joint demosaicking and denoising , 2016, ACM Trans. Graph..

[5]  L. Macaire,et al.  Comparison of color demosaicing methods , 2010 .

[6]  Chiman Kwan,et al.  Mastcam image enhancement using estimated point spread functions , 2017, 2017 IEEE 8th Annual Ubiquitous Computing, Electronics and Mobile Communication Conference (UEMCON).

[7]  Chiman Kwan,et al.  Development of a pipeline for generating high resolution multispectral Mastcam images , 2019, Defense + Commercial Sensing.

[8]  Chiman Kwan,et al.  A Novel Utilization of Image Registration Techniques to Process Mastcam Images in Mars Rover With Applications to Image Fusion, Pixel Clustering, and Anomaly Detection , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[9]  Eric Dubois,et al.  Frequency-domain methods for demosaicking of Bayer-sampled color images , 2005, IEEE Signal Processing Letters.

[10]  Hairong Qi,et al.  Revisiting the preprocessing procedures for elemental concentration estimation based on chemcam libs on mars rover , 2014, 2014 6th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS).

[11]  Lei Zhang,et al.  Color demosaicking by local directional interpolation and nonlocal adaptive thresholding , 2011, J. Electronic Imaging.

[12]  Chiman Kwan,et al.  Anomaly detection in hyperspectral images through spectral unmixing and low rank decomposition , 2016, 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[13]  Chiman Kwan,et al.  Comparison of Deep Learning and Conventional Demosaicing Algorithms for Mastcam Images , 2019 .

[14]  Chiman Kwan,et al.  Enhancing Mastcam Images for Mars Rover Mission , 2017, ISNN.

[15]  B. Ayhan,et al.  On The Use of a Linear Spectral Unmixing Technique For Concentration Estimation of APXS Spectrum , 2015 .

[16]  Yücel Altunbasak,et al.  Color plane interpolation using alternating projections , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[17]  Yap-Peng Tan,et al.  Color filter array demosaicking: new method and performance measures , 2003, IEEE Trans. Image Process..

[18]  J. Bednar,et al.  Alpha-trimmed means and their relationship to median filters , 1984 .

[19]  Chiman Kwan,et al.  Demosaicing enhancement using pixel-level fusion , 2018, Signal Image Video Process..

[20]  Chiman Kwan,et al.  Enhancing Stereo Image Formation and Depth Map Estimation for Mastcam Images , 2018, 2018 9th IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON).

[21]  Betina Pavri.  The Mars Science Laboratory “Curiosity” Rover , 2011 .

[22]  Chiman Kwan,et al.  Perceptually lossless compression of Mastcam images with error recovery , 2019, Defense + Commercial Sensing.

[23]  Lei Zhang,et al.  Color demosaicking via directional linear minimum mean square-error estimation , 2005, IEEE Transactions on Image Processing.

[24]  Chiman Kwan,et al.  Pansharpening of Mastcam images , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[25]  Chiman Kwan,et al.  Further Improvement of Debayering Performance of RGBW Color Filter Arrays Using Deep Learning and Pansharpening Techniques , 2019, J. Imaging.

[26]  Ning Zhang,et al.  Primary-consistent soft-decision color demosaicking for digital cameras (patent pending) , 2004, IEEE Transactions on Image Processing.

[27]  Lei Zhang,et al.  Image demosaicing: a systematic survey , 2008, Electronic Imaging.