An Information Theory framework for two-stage binary image operator design

The design of translation invariant and locally defined binary image operators over large windows is made difficult by decreased statistical precision and increased training time. We present a complete framework for the application of stacked design, a recently proposed technique to create two-stage operators that circumvents that difficulty. We propose a novel algorithm, based on Information Theory, to find groups of pixels that should be used together to predict the output value. We employ this algorithm to automate the process of creating a set of first-level operators that are later combined in a global operator. We also propose a principled way to guide this combination, by using feature selection and model comparison. Experimental results show that the proposed framework leads to better results than single stage design.

[1]  P. Grünwald The Minimum Description Length Principle (Adaptive Computation and Machine Learning) , 2007 .

[2]  David H. Wolpert,et al.  Stacked generalization , 1992, Neural Networks.

[3]  Edward R. Dougherty,et al.  Iterative design of morphological binary image operators , 2000 .

[4]  Edward R. Dougherty,et al.  Optimal mean-square N-observation digital morphological filters : I. Optimal binary filters , 1992, CVGIP Image Underst..

[5]  Ivan Bratko,et al.  Analyzing Attribute Dependencies , 2003, PKDD.

[6]  Nina Sumiko Tomita Hirata Binary Image Operator Design Based on Stacked Generalization , 2005, XVIII Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI'05).

[7]  Nina Sumiko Tomita Hirata,et al.  Multilevel Training of Binary Morphological Operators , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  F. Fleuret Fast Binary Feature Selection with Conditional Mutual Information , 2004, J. Mach. Learn. Res..

[9]  Edward R. Dougherty,et al.  Optimal iterative increasing binary morphological filters , 1996 .

[10]  Roberto Hirata,et al.  Exploring window selection strategies for two-level binary image operator design , 2007, ISMM.

[11]  W. J. McGill Multivariate information transmission , 1954, Trans. IRE Prof. Group Inf. Theory.

[12]  David Correa Martins,et al.  W-operator window design by minimization of mean conditional entropy , 2006, Pattern Analysis and Applications.

[13]  Junior Barrera,et al.  A Maximum-Likelihood Approach for Multiresolution W-Operator Design , 2005, XVIII Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI'05).

[14]  Kari Torkkola,et al.  Feature Extraction by Non-Parametric Mutual Information Maximization , 2003, J. Mach. Learn. Res..

[15]  Edward R. Dougherty,et al.  Automatic programming of binary morphological machines by design of statistically optimal operators in the context of computational learning theory , 1997, J. Electronic Imaging.

[16]  G. Banon,et al.  Minimal representations for translation-invariant set mappings by mathematical morphology , 1991 .

[17]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[18]  R. Hirata,et al.  Two-stage Binary Image Operator Design: an Approach Based on Interaction Information , 2007 .

[19]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[20]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Jaakko Astola,et al.  Secondarily constrained Boolean filters , 1998, Signal Processing.

[22]  Edward R. Dougherty,et al.  Multiresolution Analysis for Optimal Binary Filters , 2001, Journal of Mathematical Imaging and Vision.

[23]  Jaakko Astola,et al.  Two-stage binary filters , 1999, J. Electronic Imaging.