Statistical image sequence segmentation using multidimensional attributes

A novel interactive approach is presented for finding multiple objects in an image sequence. The segmentation technique requires only a small number of training samples from a user in order to model the multidimensional probability distribution function (PDF) of the image characteristics at each region. The user-supplied training samples are tracked over time, and the PDF of each region can be updated at every frame. All the unlabeled data points are classified to the most likely region, based on the distance between the PDF of each region, and the multidimensional feature value. A certainty of that classification is computed and stored at each sample for additional post-processing and analysis of the segmentation. The usefulness of such a system has immediate applications in the fields of image processing and computer vision. Image attributes such as color, texture, motion, and position are used to characterize the different regions, and constitute the basis of our multidimensional feature vector at every sample location in the image sequence. A detailed explanation of a PDF estimation technique known as expectation maximization (EM) explains the basis of how the PDF of each feature is calculated. Improvements to the basic EM algorithm including deterministic annealing are also tested. Varying the feature calculation methods, feature combinations, PDF estimation techniques, and tracking of user-supplied training for several sequences establish the basis of our experiments and results. The experiments presented in this thesis show how the segmentation result is usually robust with respect to the method of calculating the features, and the result is typically better when using all features instead of any one feature exclusively. (Copies available exclusively from MIT Libraries, Rm. 14-0551, Cambridge, MA 02139-4307. Ph. 617-253-5668; Fax 617-253-1690.)