The local structure of image intensity discontinuities

We present two solutions to a critical problem in computer vision: that of describing an image as the sum of an underlying piecewise-smooth image and spatially varying white noise. Both solutions employ the idea that simplicity and stability of description lead to important inferences about the processes that give rise to an image. The first solution uses a set of local operators, each hypothesizing a simplest description of the underlying image within its spatial domain by estimation-theoretical means. Hypotheses from overlapping operators are then combined to produce the simplest overall description. The second, and more powerful, solution uses an objective function based on the information-theoretic complexity of describing an image in a given language. This objective function is highly non-convex, so that standard optimization techniques cannot be used. Instead, a local, parallel, and iterative optimization algorithm is devised for finding near-optimal solutions. This algorithm is adaptive, requiring no change in parameters across a wide range of images, and has the added advantage of finding the most stable features of the simplest description first. Even though the language contains no models of three-dimensional shape, lighting, or texture, the simplest and most stable descriptions in this language are intuitively satisfying.