Scale-Driven Image Decomposition With Applications to Recognition, Registration, and Segmentation

In this thesis, we propose to solve several computer vision problems using a novel fundamental idea, the scale difference between different patterns. In order to achieve our goal, we utilize the recently proposed total variation regularized L1 functional, which has an unique geometric feature of decomposing an additive image according to scales of the patterns within the image. We analyze and study the geometric properties of the TV-L1 model. We discuss different properties and provide intuitively proofs. We also discuss the properties when this model is applied to an image containing irregular shaped patterns, which were rarely discussed in literature. We then modify the TV-L1 model and develop novel algorithms to solve problems in different application areas. Other than proposing the direct use of the TV-L 1 model for uneven background correction, we develop several novel algorithms based on this scale-driven image decomposition model. Our extensions and modifications are threefold: recognition, registration, and segmentation. In recognition, instead of decomposing an additive signal, we propose to factorize an image under multiplicative illumination fields based on the TV- L1 model. The effectiveness of this factorization is validated by a significant improvement of face recognition under varying illumination. In registration, we propose a non-rigid registration framework using a novel scale hierarchy established by the TV-L 1 model. We obtain robust and accurate registration on both 2D satellite images and 3D brain MR images with this framework. At last, a probabilistic method and a multi-resolution method are used to improve the limitations of the TV-L1 model for image segmentation. The proposed segmentation method is able to extract brain regions from head images. It can also be used to extract large scale patterns in general images. Experiment results validate the effectiveness of our work in different application areas. We believe our works have significant contributions and have brought new possibilities to computer vision, public security, surveillance, medical image analysis, and other related fields.

[1]  Josef Kittler,et al.  A comparison of photometric normalisation algorithms for face verification , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[2]  Michael Elad,et al.  A Variational Framework for Retinex , 2002, IS&T/SPIE Electronic Imaging.

[3]  Stephen M Smith,et al.  Fast robust automated brain extraction , 2002, Human brain mapping.

[4]  Alex Pentland,et al.  Bayesian face recognition , 2000, Pattern Recognit..

[5]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[6]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Brendan J. Frey,et al.  Epitomic analysis of appearance and shape , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[8]  Lei Zhang,et al.  Face recognition under variable lighting using harmonic image exemplars , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[9]  Ping-Sing Tsai,et al.  Shape from Shading: A Survey , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  J. Mazziotta,et al.  MRI‐PET Registration with Automated Algorithm , 1993, Journal of computer assisted tomography.

[11]  Thomas S. Huang,et al.  Boundary correction for total variation regularized L^1 function with applications to image decomposition and segmentation , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[12]  Stefano Alliney,et al.  Digital filters as absolute norm regularizers , 1992, IEEE Trans. Signal Process..

[13]  Jan Flusser,et al.  Image registration methods: a survey , 2003, Image Vis. Comput..

[14]  Y. J. Tejwani,et al.  Robot vision , 1989, IEEE International Symposium on Circuits and Systems,.

[15]  Thomas S. Huang,et al.  Scale-Driven Iterative Optimization for Brain Extraction and Registration , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[16]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression (PIE) database , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[17]  T. Chan,et al.  Edge-preserving and scale-dependent properties of total variation regularization , 2003 .

[18]  Anders M. Dale,et al.  Cortical Surface-Based Analysis I. Segmentation and Surface Reconstruction , 1999, NeuroImage.

[19]  K. Hohn,et al.  Determining Lightness from an Image , 2004 .

[20]  T. Speed,et al.  Statistical issues in cDNA microarray data analysis. , 2003, Methods in molecular biology.

[21]  Haitao Wang,et al.  Face recognition under varying lighting conditions using self quotient image , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[22]  Alain Viari,et al.  Imagene: an integrated computer environment for sequence annotation and analysis , 1999, Bioinform..

[23]  Alan Siegel.  An Isoperimetric Theorem in Plane Geometry , 2003, Discret. Comput. Geom..

[24]  Stephen M. Smith,et al.  A global optimisation method for robust affine registration of brain images , 2001, Medical Image Anal..

[25]  Ralph Gross,et al.  An Image Preprocessing Algorithm for Illumination Invariant Face Recognition , 2003, AVBPA.

[26]  Alexei A. Efros,et al.  Fast bilateral filtering for the display of high-dynamic-range images , 2002 .

[27]  P. Hanrahan,et al.  On the relationship between radiance and irradiance: determining the illumination from images of a convex Lambertian object. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[28]  Wotao Yin,et al.  Second-order Cone Programming Methods for Total Variation-Based Image Restoration , 2005, SIAM J. Sci. Comput..

[29]  M. Stella Atkins,et al.  Fully automatic segmentation of the brain in MRI , 1998, IEEE Transactions on Medical Imaging.

[30]  Haitao Wang,et al.  Generalized quotient image , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[31]  Jean-François Aujol,et al.  Color image decomposition and restoration , 2006, J. Vis. Commun. Image Represent..

[32]  Donald Goldfarb,et al.  Second-order cone programming , 2003, Math. Program..

[33]  Wen Gao,et al.  Illumination normalization for robust face recognition against varying lighting conditions , 2003, 2003 IEEE International SOI Conference. Proceedings (Cat. No.03CH37443).

[34]  Antonin Chambolle,et al.  Nonlinear wavelet image processing: variational problems, compression, and noise removal through wavelet shrinkage , 1998, IEEE Trans. Image Process..

[35]  Rama Chellappa,et al.  Robust image based face recognition , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[36]  Pat Hanrahan,et al.  A signal-processing framework for inverse rendering , 2001, SIGGRAPH.

[37]  Max A. Viergever,et al.  Mutual-information-based registration of medical images: a survey , 2003, IEEE Transactions on Medical Imaging.

[38]  Nanning Zheng,et al.  Stereo Matching Using Belief Propagation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  Tony F. Chan,et al.  Total Variation Denoising and Enhancement of Color Images Based on the CB and HSV Color Models , 2001, J. Vis. Commun. Image Represent..

[40]  A. Evans,et al.  MRI simulation-based evaluation of image-processing and classification methods , 1999, IEEE Transactions on Medical Imaging.

[41]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[42]  Dorin Comaniciu,et al.  Illumination normalization for face recognition and uneven background correction using total variation based image models , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[43]  Michael Brady,et al.  Improved Optimization for the Robust and Accurate Linear Registration and Motion Correction of Brain Images , 2002, NeuroImage.

[44]  Wotao Yin,et al.  Background correction for cDNA microarray images using the TV+L1 model , 2005, Bioinform..

[45]  Guido Gerig,et al.  Elastic model-based segmentation of 3-D neuroradiological data sets , 1999, IEEE Transactions on Medical Imaging.

[46]  Pierre Soille,et al.  Morphological Image Analysis: Principles and Applications , 2003 .

[47]  Thomas S. Huang,et al.  A New Coarse-to-Fine Framework for 3D Brain MR Image Registration , 2005, CVBIA.

[48]  Zia-ur Rahman,et al.  A multiscale retinex for bridging the gap between color images and the human observation of scenes , 1997, IEEE Trans. Image Process..

[49]  Tony F. Chan,et al.  Structure-Texture Image Decomposition—Modeling, Algorithms, and Parameter Selection , 2006, International Journal of Computer Vision.

[50]  Terence P. Speed,et al.  Comparison of Methods for Image Analysis on cDNA Microarray Data , 2002 .

[51]  Ajay N. Jain,et al.  Fully automatic quantification of microarray image data. , 2002, Genome research.

[52]  William T. Freeman,et al.  On the optimality of solutions of the max-product belief-propagation algorithm in arbitrary graphs , 2001, IEEE Trans. Inf. Theory.

[53]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[54]  Pradeep K. Khosla,et al.  "Corefaces" - robust shift invariant PCA based correlation filter for illumination tolerant face recognition , 2004, CVPR 2004.

[55]  Richard M. Leahy,et al.  Surface-based labeling of cortical anatomy using a deformable atlas , 1997, IEEE Transactions on Medical Imaging.

[56]  Shimon Ullman,et al.  Face Recognition: The Problem of Compensating for Changes in Illumination Direction , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[57]  Vladimir Brajovic,et al.  Brightness perception, dynamic range and noise: a unifying model for adaptive image sensors , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[58]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[59]  Guodong Guo,et al.  Face recognition by support vector machines , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[60]  B. V. K. Vijaya Kumar,et al.  "Corefaces" - robust shift invariant PCA based correlation filter for illumination tolerant face recognition , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[61]  Antonin Chambolle,et al.  Dual Norms and Image Decomposition Models , 2005, International Journal of Computer Vision.

[62]  Mila Nikolova,et al.  Minimizers of Cost-Functions Involving Nonsmooth Data-Fidelity Terms. Application to the Processing of Outliers , 2002, SIAM J. Numer. Anal..

[63]  G. Hagemann,et al.  Fast, accurate, and reproducible automatic segmentation of the brain in T1‐weighted volume MRI data , 1999, Magnetic resonance in medicine.

[64]  Aly A. Farag,et al.  A shape-based segmentation approach: an improved technique using level sets , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[65]  A I Saeed,et al.  TM4: a free, open-source system for microarray data management and analysis. , 2003, BioTechniques.

[66]  Vladimir Brajovic,et al.  Model for reflectance perception in vision , 2003, SPIE Microtechnologies.

[67]  Jr. Thomas G. Stockham,et al.  Image processing in the context of a visual model , 1972 .

[68]  Wotao Yin,et al.  Image Cartoon-Texture Decomposition and Feature Selection Using the Total Variation Regularized L1 Functional , 2005, VLSM.

[69]  Andrew Zisserman,et al.  OBJ CUT , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[70]  David J. Kriegman,et al.  What is the set of images of an object under all possible lighting conditions? , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[71]  E. Land,et al.  Lightness and retinex theory. , 1971, Journal of the Optical Society of America.

[72]  Takeo Kanade,et al.  Shape from interreflections , 2004, International Journal of Computer Vision.

[73]  Nicholas Ayache,et al.  The Correlation Ratio as a New Similarity Measure for Multimodal Image Registration , 1998, MICCAI.

[74]  Tony F. Chan,et al.  Aspects of Total Variation Regularized L[sup 1] Function Approximation , 2005, SIAM J. Appl. Math..

[75]  Penio S. Penev,et al.  Local feature analysis: A general statistical theory for object representation , 1996 .

[76]  Timothy F. Cootes,et al.  Automatic Interpretation and Coding of Face Images Using Flexible Models , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[77]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[78]  Dorin Comaniciu,et al.  Total variation models for variable lighting face recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[79]  Simon R. Arridge,et al.  A survey of hierarchical non-linear medical image registration , 1999, Pattern Recognit..

[80]  Lisa M. Brown,et al.  A survey of image registration techniques , 1992, CSUR.

[81]  Michael Elad,et al.  Reduced complexity Retinex algorithm via the variational approach , 2003, J. Vis. Commun. Image Represent..

[82]  Greg Turk,et al.  LCIS: a boundary hierarchy for detail-preserving contrast reduction , 1999, SIGGRAPH.

[83]  David J. Kriegman,et al.  Nine points of light: acquiring subspaces for face recognition under variable lighting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[84]  S. Osher,et al.  Slope and $G$-set characterization of set-valued functions and applications to non-differentiable optimization problems , 2005 .

[85]  Norbert Krüger,et al.  Face recognition by elastic bunch graph matching , 1997, Proceedings of International Conference on Image Processing.

[86]  W. A. Hanson,et al.  Interactive 3D segmentation of MRI and CT volumes using morphological operations. , 1992, Journal of computer assisted tomography.

[87]  Ronen Basri,et al.  Lambertian reflectance and linear subspaces , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[88]  Amnon Shashua,et al.  The Quotient Image: Class-Based Re-Rendering and Recognition with Varying Illuminations , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[89]  Daniel P. Huttenlocher,et al.  Efficient Belief Propagation for Early Vision , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[90]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .