Natural Images: Coding Efficiency

A wide variety of studies over the past 20 years have demonstrated that our sensory systems are remarkably efficient at coding the sensory environment. Much of this work has focused on the visual system and has demonstrated that many properties of the early visual system are extremely well matched to the statistical structure of the visual world. However, there remain many questions regarding how far this approach can be taken in understanding the full visual system, especially higher levels of visual processing. Basic theories of efficiency (e.g., decorrelation, sparseness, independence) are likely to be insufficient in accounting for the more complex nonlinear representations found in higher levels. In this article, we take a closer look at how efficiency might be defined. In particular, we consider three forms of efficiency: representational efficiency, metabolic efficiency, and learning efficiency. Although the majority of studies have focused on representational efficiency and metabolic efficiency, we argue that a complete account of visual processing must consider all three forms.