On the Salience of Novel Stimuli: Adaptation and Image Noise

Webster1 has proposed “that adaptation increases the salience of novel stimuli by partially discounting the ambient background.” This is an excellent, concise, description of the purpose and function of chromatic adaptation in image reproduction applications. However, Webster was not limiting this proposal to just chromatic adaptation, but rather using it as a general description for all forms of perceptual adaptation. Demonstrations of adaption to other properties of image displays such as motion, blur, and spatial frequency led the authors to ponder the question of whether observers might adapt to the noise structure in images to enhance the novel stimuli — the systematic image content. This paper describes psychophysical measurements of noise adaptation in color image perception and explores mathematical prediction of the effect. The results illustrate the hypothesized pattern-dependent adaptation and its prediction through adaptation of a 2-D contrast sensitivity function in an image-appearance-model-based difference metric. Introduction Spatial frequency adaptation has been recognized for over 30 years and used as evidence for the existence of spatial-frequencyand orientation-tuned mechanisms in the human visual system.2 Figure 1 is a typical demonstration of spatial frequency adaptation. After gazing at the bar on the left side of Fig. 1 for 15-30 s., the identical patterns on the right side appear to shift in spatial frequency in directions opposite the adapting stimuli. Webster and coworkers1,3,4 have expanded the exploration of spatial frequency adaptation to the study of adaptation to complex spatial stimuli such as image blur, face expression, and face recognition. Figure 2 recreates one of Webster’s demonstrations of blur adaptation. After gazing at the bar between the upper images for 15-30s., the bottom two images, which are physically identical will appear significantly different. The image on the left will appear more blurred after adaptation to a sharp image while the image on the right will appear sharper after adaptation to a blurry image. This effect can also be seen in the form of simultaneous contrast whereby an image will appear sharper if surrounded by blurry images. Webster’s observations led the authors to hypothesize that the human visual system might be capable of adapting to noise content in images effectively enhancing the perception of image content while minimizing the perception of artifacts introduced by imaging systems. Quantitative knowledge of such adaptation effects is critical for the development of accurate image quality metrics. Figure 1. Demonstration of spatial frequency adaptation. Figure 2. Demonstration of adaptation to image blur. A visual demonstration of noise adaptation in images is easily created as illustrated in Fig. 3. Adaptation to the images at the top will result in the lower-left image appearing noisier than the lowerright image despite being physically identical. Figure 3. Demonstration of adaptation to image noise. Webster and Mollon5 measured contrast adaptation in natural images illustrating that the visual system does adapt to the range of color and lightness information in a scene. This adaptation could be considered similar to an automatic gamut mapping in the visual system. While these results suggest the possibility of adapting to the noise contrast in an image, they did not explicitly explore noise adaptation. Field and Brady6 describe an approach to perception based on the content of natural scenes that is easily extensible to the concept of adaptation to the noise in an image. Other researchers have explored related forms of adaptation, but not specifically image noise. Clifford and Weston7 studied adaptation to Glass patterns, essentially noise with some correlated structure. Anderson and Wilson8 described complex spatial frequency adaptation to identity elements in faces. Artal et al.9 have shown that neural mechanisms, presumably long-term adaptation, are capable of compensating for optical aberrations in observers’ eyes. Finally, Durgin et al.10,11 have shown adaptation to natural and artificial texture. This, and related, work comes closest to measuring noise adaptation however texture adaptation is an examination of noise adaptation in the absence of other content. The current work aims to examine the perception of the remaining image content after noise adaptation. Experimental The experiment began with the hypothesis that adaptation to spatially-structured noise would decrease the sensitivity (raise the threshold) of observers to similar noise within an image. Furthermore, it was hypothesized that adapting noise of one structure (e.g. vertically oriented) would have little, or no, effect on the sensitivity to noise of a completely different structure (e.g. horizontally oriented). A simple psychophysical experiment was designed and implemented to test these hypotheses. Observers were presented with images intermittently placed on an adapting background. Three types of adapting backgrounds were used (see Fig. 4), 2D random, horizontal, and vertical white noise with uniform luminance distribution. Additionally, a uniform gray adapting background was used. Each adapting background was used with contrast levels of 9.4, 18.9, 28.1, and 37.5 percent (Fig. 4). The adapting backgrounds filled the experimental display, a carefully-characterized 23”Apple Cinema HD Display viewed at 1 meter. The display (1920x1200 pixels) subtended 28x17 degrees of visual field with an addressability of 68 pixels/degree. The maximum display luminance was 320 cd/m2 with a white point approximating CIE Illuminant D65. The adapting backgrounds were achromatic. Figure 4. Adapting backgrounds ranging from uniform (left) to 37.5% contrast (right) for random, horizontal, and vertical white noise. Visual sensitivity to each of the three types (random, horizontal, vertical) of noise was measured using the method of adjustment. These measurements were completed using 5 different images (Fig. 5) upon which the noise was added. These images include 4 pictorial scenes and a uniform gray (equal to the adapting background mean luminance, approximately middle gray, and 128 digital counts on a Macintosh display). The images were each 512x512 pixels, or 7.5x7.5 degrees of viewing angle. Figure 5. Five images used for measurement of sensitivity to added noise (random, horizontal, and vertical). The test images were presented together with an original image having no added noise. The images were presented for 1 s. followed by 4 s. in which only the adapting background was present. This cycle repeated while the observers adjusted the noise contrast of the right image until the noise was just identifiable. Specifically observers were asked to adjust the noise contrast until they could just discriminate which of the three types of noise was being added to the image. These contrast discrimination thresholds (called visible contrast in the plotted results) were obtained for each combination of image content, background noise type, background noise contrast, and image noise type. There was a total of 195 threshold settings for a full experimental session. Observers could complete a session in about 2 hours. Once observers set the image noise level to the criterion contrast, they pressed a button and a new trial began. Trials were completely randomized in all experimental variables. Figure 6 shows an example stimulus configuration with vertical noise in the adapting background and horizontal noise (clearly above the threshold setting) in the test image. Two observers, MF and GJ, performed the experiment five times each to collect precise data on two observers and assess intraobserver variability. An additional 10 observers completed the experiment once to verify the effect and estimate inter-observer variability. All observers had normal, or corrected-to-normal, visual acuity and normal color vision. Data for two observers was discarded since the available range of noise was not sufficient for them in multiple trials. Thus, the reported inter-observer data are for a total of 10 observers. Figure 6. Example stimulus with the reference image on the left, test image with horizontal noise on the right, and adapting background with vertical noise. Results Figure 7 shows the visibility of random noise (observers MF and GJ) as a function of adapting background contrast averaged over all images for each adapting condition. Example 95% error bars are presented on one curve, the magnitude of which would be similar for the other data sets. While the error bars appear large relative to the adaptation effect, most of the variability is due to image dependent changes in the threshold. Only about 1/3 of the error is associated with random noise (see Fig. 10). The adaptation effect is statistically significant for each viewing situation. The results show that, for both observers, random noise in the adapting field elevates the threshold for random noise in the image and the effect increases with adapting contrast. Horizontal and vertical adapting noise also elevate the thresholds, but to a lesser extent as would be expected since those adapting stimuli only depress one dimension of the 2D contrast sensitivity function. Observer GJ generally shows higher thresholds (possibly a criterion effect in the method of adjustment) and larger adaptation effects. 0.04 0.05 0.06 0.07 0.08 0.09 0.00 0.10 0.20 0.30 0.40 Adapting Contrast V is ib le C o n tr a st MF Random Adapt MF Horizontal Adapt MF Vertical Adapt GJ Random Adapt GJ Horizontal Adapt GJ Vertical Adapt Figure 7. Random noise visibility for all adapting conditions. Figure 8 shows similar results for the visibility of horizontal and vertical image noise. The results are consistent with the thresholds for vertical noise elevated when adapting to vertical noise and vice versa. There is no effect of horizontal noise adaptation on the visibility of vertical