Spry features

The fundamental contribution of this thesis is Spry — a new framework for spry (agile, nimble) object/scene recognition. Spry presents a new affine invariant recognition paradigm that typically produces results at least as good as, if not better than, current state of the art algorithms while being significantly faster. To achieve this, Spry develops new technologies that stand on their own as contributions to the fields of Image Processing and Computer Vision. First, we present a novel, fully automatic, method to create box filters that achieve an excellent approximation of any arbitrary 2-D filter. We approximate the original filter by a weighted sum of individual box filters and formulate the filter approximation as an optimization problem. We present. two algorithms that can determine the optimal location of the box filters: Exhaustive Search and Directed Search. We show that both algorithms find good approximations to general fillers. Second, we develop methods that are invariant to the general affine movement by decomposing it into four distortions with geometric meaning: rotation in the object plane; rotation in the image plane; isotropic scaling; and tilt (anisotropic scaling). We develop an affine-space function that achieves invariance to all these distortions and we show that it can be computed by convolution of a filter bank with the input image. Third, we introduce iterative feature detection and description. In contrast with all relevant state of the art methods that detect features sequentially in a single pass, our method is iterative: detecting, describing, and matching features in batches. Fourth, we show that it is possible to detect and describe features iteratively without fully filtering the input image. We develop a novel greedy algorithm for iterative feature detection and description that works by randomly deciding a starting location on the scale space and then exploring the neighborhood of that location until a suitable feature point is found. We introduce a new descriptor, a modified version of the SIFT descriptor, that uses the observation information in order to be fully affine invariant. Fifth, we show that traditional methods of feature matching are not appropriate for our iterative framework and instead introduce a new feature matching algorithm based on the use of an Iterative k-Dimensional Tree. We show that this new data structure is ideal for applications in which the number of features increases at runtime and demonstrate, both theoretically and experimentally, that its performance is superior to traditional methods for matching features in growing databases. Sixth, we present three alternative approaches to the decision task. In the first approach, Nomography Estimation and Verification, we find groups of features that can be related by homographies and only report matches that agree with one of the homographies in the image. In the second approach, LASIC, we formulate the decision problem as an hypothesis test and derive the uniformly most. powerful (UMP) test that is affine invariant. We formulate the matching problem as a quadratic maximization in the space of permutation matrices and present an efficient algorithm to solve this optimization problem. In the third approach, Shapes as Empirical Distributions, we interpret the shape of an object as a probability distribution governing the location of the features of the object and interpret an image of an object as a random drawing from the shape distribution. We use Maximum Likelihood and formulate the decision problem associated with shape classification as a hypothesis test for which we characterize the performance. Finally, we demonstrate how all the above contributions conic together to create Spry. Multiple experimental results corroborate the superiority of Spry versus all current competitive state of the art algorithms in the field.