An analysis of the Scale Saliency algorithm

In this paper, we present an analysis of the theoretical underpinnings of the Scale Saliency algorithm recently introduced in (Kadir and Brady, 2001). Scale Saliency considers image regions salient if they are simultaneously unpredictable in some feature-space and over scale. The algorithm possesses a number of attractive properties: invariance to planar rotation, scaling, intensity shifts and translation; robustness to noise, changes in viewpoint, and intensity scalings. Moreover, the approach offers a more general model of feature saliency compared with conventional ones, such as those based on kernel convolution, for example wavelet analysis. Typically, such techniques define saliency and scale with respect to a particular set of basis morphologies. The aim of this paper is to make explicit the nature of this generality. Specifically, we aim to answer the following questions: What exactly constitutes a ‘salient feature’? How does this differ from other feature selection methods? The main result of our analysis is that Scale Saliency defines saliency and scale independently of any particular feature morphology. Instead, a more general notion is used, namely spatial unpredictability. This is determined within the geometric constraint determined by the sampling window and its parameterisation. In (Kadir and Brady, 2001) this window was a circle parameterised by a single parameter controlling the radius. Under such a scheme, features are considered salient if their feature-space properties vary rapidly with an incremental change in radius. In other words, the feature favours isotropically unpredictable features. We also present a number of variations of the original algorithm: a modification for colour images (or more generally any vector-valued image) and a generalisation of the isotropic scale constraint to the anisotropic case.