Random Color Filter Arrays are Better than Regular Ones

Most of digital cameras today use a color filter array (CFA) and a single sensor to acquire color information of the scene. In this article, we ask which arrangement of colors in the mosaic of the CFA provides the best encoding of the scene. As a solution of the inverse problem of demosaicing, we consider a linear minimum mean squared error model. We used redundancy given by the neighborhood on the sampled image to ensure the stability of the solution. For some CFAs, LMMSE with neighborhood provides equivalent reconstruction results and less variability among the image content compared to edge-directed demosaicing on the Bayer. LMMSE allows comparing CFAs of regular pattern with random ones. We show that mosaics with random arrangement of colors and quasi equal proportion of RGB provide best reconstruction performance. c © 2016 Society for Imaging Science and Technology. [DOI: 10.2352/J.ImagingSci.Technol.2016.60.5.050406] INTRODUCTION A color image is composed with the intensity of three different channels covering three different domains of wavelength, usually in the Red, Green and Blue part of the visible spectrum. To acquire such an image with simplicity and low cost, a single sensor is used which is covered with a color filter array (CFA) to provide several color components to the acquired image, arranged in a mosaic. Thus only a single color is sampled at each pixel and reconstruction of missing colors (called demosaicing or demosaicking) is required. The Bayer’s CFA1 is the most commonly used CFA and several methods have been proposed for improving the quality of the reconstruction. Edge-directed2 methods which interpolate along contours and avoid interpolation across them are known to be the best method for the Bayer CFA. Thesemethods are usually followed by a post-processing that improves the reconstructed image.3–10 But the computation time needed for these methods makes them generally too costly for embedded systems. Moreover in practice, the CFA Received Apr. 8, 2016; accepted for publication June 20, 2016; published online Aug. 18, 2016. Associate Editor: Yeong-Ho Ha. 1062-3701/2016/60(5)/050406/6/$25.00 image produced by a sensor is less constraining than the simulated image on the Kodak database11 (the most used one on demosaicing) which contains moiré due to higher frequency content compared to the number of pixels. This is even worse for recent cameras with small pixels size.12 Because these methods are optimized for Bayer CFA they are not very useful for general CFAs such as those with random arrangement of color. Some studies show new CFA patterns, but either they are designed empirically.13–15 Many authors have proposed optimal CFAs arrangement based on the criteria of frequency representation and selection.16–20 Indeed, the mosaic arrangement of the filters in the CFA could be interpreted as a spatial multiplexing of color components and has a simple expression in the Fourier domain.21–23 The spatial Fourier representation of the CFA allows simple linear demosaicing by selecting the part of the spectrum that corresponds to luminance and color components. Some authors assume the RGB filter’s spectral sensitivity can be modified and consider composed colors as a linear combination of RGB and propose an arrangement of these new colors that optimize the frequency representation and estimation. But there is no evidence that these new colors can be easily produced from physical composition of the RGB pigments. In addition, the simple mathematical expression of spatial multiplexing is due to periodicity or regularity in the mosaic. The locality of chrominance is lost for a random arrangement of color on the CFA. This prevents the application of frequency selection method on random CFAs. Demosaicing is an inverse problem to retrieve the missing colors from the sampled ones. This problem has no general solution. To solve it, we must consider a model of the solution family (solutions appear as a functional for which a set of parameters are optimal for the problem) and provide the best estimated solution inside this family. It is almost straightforward to consider linear solutions.24 We therefore restrict here the solutions to be linear application from Rn to Rm, n will be the dimension of the mosaiced image plus neighborhood’s space andm the dimension of the J. Imaging Sci. Technol. 050406-1 Sept.-Oct. 2016 Amba, Dias, and Alleysson: Random color filter arrays are better than regular ones