An information theory of visual communication

The fundamental problem of visual communication is that of producing the best possible picture at the lowest data rate. We address this problem by extending information theory to the assessment of the visual communication channel as a whole, from image gathering to display. The extension unites two disciplines, the electro- optical design of image gathering and display devices and the digital processing for image coding and restoration. The mathematical development leads to several intuitively attractive figures of merit for assessing the visual communication channel as a function of the critical limiting factors that constrain its performance. Multiresolution decomposition is included in the mathematical development to optimally combine the economical encoding of the transmitted signal with image gathering and restoration. Quantitative and qualitative assessments demonstrate that a visual communication channel ordinarily can be expected to produce the best possible picture at the lowest data rate only if the image-gathering device produces the maximum-realizable information rate and the image-restoration algorithm properly accounts for the critical limiting factors that constrain the visual communication. These assessments encompass (a) the electro-optical design of the image-gathering device in terms of the trade-off between blurring and aliasing in the presence of photodetector and quantization noises, (b) the compression of data transmission by redundancy reduction, (c) the robustness of the image restoration to uncertainties in the statistical properties of the captured radiance field, and (d) the enhancement of particular features or, more generally, of the visual quality of the observed image. The ‘best visual quality’ in this context normally implies a compromise among maximum-realizable fidelity, sharpness, and clarity which depends on the characteristics of the scene and the purpose of the visual communication (e.g. diagnosis versus entertainment).