Efficient Coding of Natural Images Outline: Abstract Introduction Efficient for What Task? Defining Efficiency A. Representational Efficiency Correlation and Decorrelation Optimal Information Transfer beyond Correlations: Sparseness and Independence Optimality with Nonlinear Systems B. Metabolic Eff

A wide variety of studies over the last twenty years have demonstrated that our sensory systems are remarkably efficient at coding the sensory environment. Much of this work has focused on the visual system and it has demonstrated that many properties of the early visual system are extremely well matched to the statistical structure of the visual world. However, there remain many questions regarding how far this approach can be taken in understanding the full visual system, especially higher levels of visual processing. Basic theories of efficiency (e.g., decorrelation, sparseness, independence, etc.) are likely to be insufficient to account for the more complex nonlinear representations found in higher levels. In this chapter, we take a closer look at how efficiency might be defined. In particular, we consider three forms of efficiency: representational efficiency, metabolic efficiency, and learning efficiency. Although the majority of studies have focused on representational and metabolic efficiency, we argue that a complete account of visual processing must consider all three forms of efficiency.