How to Make Life More Colorful: From Image Quality to Atmosphere Experience

Image quality and color appearance have been extensively studied in the past decades, which has resulted in high quality displays. Although research on image quality is still ongoing, most improvements have only marginal effects. A new trend in display technology is emerging that focuses on enhancing the overall visual experience of the user. Two features that have been proven to be effective are the introduction of stereoscopic depth and dynamic surround light. In order to further enhance the user’s experience, the atmosphere of the entire room could be adapted to the emotional content of the video. This paper gives a brief overview of research from image quality to the emotional impact of light emitting devices and identifies the research challenges for creating colorful and appealing experiences. Introduction Color is an important aspect of our everyday lives. From an evolutionary point of view, animals with color vision were better suited to gather food, to spot enemies and to pass on their genes. Nowadays, color is used by humans in many areas, like art, architecture, fashion, communication and entertainment. The reason of using color can be very divers, e.g. to draw people’s attention, to transfer information or to create an experience. Since the introduction of an electrical supply network in the late 1800s, products have been developed that emit colored light, such as TVs and lamps. Nowadays, these products are a matter of course. Most households in developed countries have more than one TV, computer, mobile phone or digital camera with a color display. In outdoor spaces, color is used frequently since the introduction of neon lights for signage and city beautification. Whereas in the past light emitting devices were mainly developed for their functional benefits, the emotional value of these devices is becoming more and more important. The image quality of displays has improved drastically over the years, from blurred black and white images to colorful high resolution images. Lighting technology has been improved as well, from inefficient incandescent light bulbs to energy saving compact fluorescent lamps and LEDs. As the functional quality of these devices is reaching the level required by the average user, the next challenge is to optimize the experience of the end-user. This paper gives a brief overview of research from image quality of displays to the emotional impact of light emitting devices. Image Quality Marketing studies consistently show that image quality is one of the most important considerations for consumers to purchase a display besides costs. In order to make high quality displays, display manufacturers need to know how the technology variables of the imaging system, such as the thickness of the color filters or the size of the pixels, affect the image quality as perceived by the end-users. However, assessing the relation between image quality and technology variables appears to be a time consuming and inefficient task, especially because the optimal value of one technology variable usually depends on the values of several other technology variables. This means that the optimization of one display system does not provide knowledge on the optimization of another display system. To overcome this problem, Engeldrum [1] has developed the Image Quality Circle that breaks the relation into three measurable steps (Figure 1). In the first step, image quality is consider to be a multidimensional concept that can be described by a weighted sum of perceptual image quality attributes, such as brightness and sharpness. These attributes can only be determined by human observers and are expressed as perceived strengths (e.g. very bright or very dim). In the second step, each perceptual attribute is related to the physical characteristics of the light emitted by the display, such as the chromaticity of the red, green and blue primaries. In the third step, the physical light output of the display is described as the combination of all technology variables. For instance, changing the thickness of a color filter will affect both the luminance and the chromaticity of the corresponding primary. Image processing algorithms can also be considered as part of the technology variables. The problem of optimizing image quality is now redefined as three questions: (1) what are the image quality attributes and their relative importance for overall image quality, (2) what is the influence of physical characteristics of the light output on the perceptual attributes, and (3) what is the relation between technology variables and the physical light output? Figure 1. The Image Quality Circle model of Engeldrum [1]. Image quality attributes Experimental studies have revealed several perceptual attributes that contribute to the overall image quality of a display system, such as brightness, contrast, color appearance, sharpness and flicker [2]. The relation between these attributes and the overall image quality is, however, far from trivial. One of the 1 2 3 4 Image Quality Circle Image Quality Technology Variables Physical Image Characteristics Image Quality Attributes 17th Color Imaging Conference Final Program and Proceedings 123 reasons is that the relation depends on several factors, such as image content, ambient illumination and personal preference. Moreover, the image quality attributes are usually measured on relative scales (e.g. brighter or sharper) and not on absolute scales that are comparable. Research is underway to express the relative importance of the attributes in terms of the just noticeable difference (JND) of each attribute. In the mean time, qualitative studies have demonstrated that color appearance is one of the most important attributes that naïve viewers use to rank the quality of different high-end TV sets shown next to each other [3]. Color appearance The color appearance of an image presented on a display depends on physical characteristics of the display, but also on characteristics of the surround illumination. In this section only the display will be considered. The range of colors that can be rendered on a display is usually represented by a 3-dimensional shape in a given color space. This so-called ‘color gamut’ is determined by the chromaticity of the display’s primaries, the intrinsic white-point and the gray scale transfer function of each primary. In order to achieve the same color rendering on different displays, video material is encoded according to a standard format (e.g. EBU Tech. 3213 or ITU Rec.709), specifying the primaries, white-point and transfer function, but also the frame rate and resolution. Only displays that comply with the standardized color gamut are able to reproduce colors accurately without additional image processing. Due to technology constraints, displays can have a significantly smaller color gamut compared to the standardized gamut, as is the case for most hand-held devices. In order to provide guidelines for display manufactures, research has determined the variations that are allowed in the chromaticity coordinates of the primaries for the image to be perceived as natural [4]. People are most tolerant for a saturation reduction of the blue primary and much less tolerant for a saturation reduction of the red and green primaries. On the other hand, people are least tolerant for a hue change of the blue primary and more tolerant for a hue change of the green primary. For the red primary, hue changes towards blue are more acceptable compared to hue changes towards green. Also, the white-point of a display does not always correspond to the standardized value of D65. Research has found that deviations of the white-point are more acceptable for variations along the black body curve compared to variations perpendicular to the black body curve [5]. Recent advances in backlight technology of LCDs have made it possible to expand the color gamut towards more saturated primaries. In addition, displays with more than three primaries in a spatial or time-sequential pattern have been proposed. The added value of these wide-gamut displays is based on two observations. First, it is known that not all natural colors can be reproduced within the standardized gamut [6]. Second, people usually prefer colors to be slightly more saturated than what is natural [7]. It has been shown that using the RGB values of an image to directly drive the wide-gamut display can lead to unacceptable colors [8]. For instance, objects at high saturation and high luminance might appear to be fluorescent. Recent studies have determined the maximum gamut size that results in the most preferred or acceptable color rendering, using a large set of complex images [9] or using images containing mainly one hue [10]. Both studies show that the preferred chroma for most images is located outside the EBU gamut, which illustrates the need for wide-gamut displays. The preferred chroma and maximally acceptable chroma were found to depend on image content and personal preference, and, to a lesser extent, on hue. Color processing Once the color gamut of a display is determined, image processing algorithms can be used to change the physical light output for a given RGB input value and, hence, to improve the color appearance. When the (output) gamut of the display is smaller than the (input) gamut of the image, a combination of clipping and scaling is usually applied. Clipping out-of-gamut colors to the borders of the output gamut has the advantage of retaining the saturation of most colors at the expense of losing color detail in areas with high saturation. Scaling of the input gamut has a limited effect on color detail but reduces the saturation of all colors. Both clipping and scaling can be applied in many different ways, e.g. one could change the lightness of the input color, the chroma, the hue or a combination of these color attributes. In the past, many different gamut compression algorithms have been propo