Knowledge Reuse Mechanisms for Categorizing Related Image Sets

This chapter introduces the concept of classifier knowledge reuse as a means of exploiting domain knowledge taken from old, previously created, relevant classifiers to assist in a new classification task. Knowledge reuse helps in constructing better generalizing classifiers given few training examples and for evaluating images for search in an image database. In particular, we discuss a knowledge reuse framework in which a supra-classifier improves the performance of the target classifier using information from existing support classifiers. Soft computing methods can be used for all three types of classifiers involved. We explore supra-classifier design issues and introduce several types of supra-classifiers, comparing their relative strengths and weaknesses. Empirical examples on real world image data sets are used to demonstrate the effectiveness of the supra-classifier framework for classification and retrieval/search in image databases.

[1]  Galina L. Rogova,et al.  Combining the results of several neural network classifiers , 1994, Neural Networks.

[2]  Jonathan Baxter,et al.  Learning internal representations , 1995, COLT '95.

[3]  Sherif Hashem,et al.  Optimal Linear Combinations of Neural Networks , 1997, Neural Networks.

[4]  Kagan Tumer,et al.  Structural adaptation and generalization in supervised feed-forward networks , 1994 .

[5]  Joydeep Ghosh,et al.  Knowledge reuse in multiple classifier systems , 1997, Pattern Recognit. Lett..

[6]  David H. Wolpert,et al.  Stacked generalization , 1992, Neural Networks.

[7]  Colin Aitken.  Kernel methods for the estimation of discrete distributions , 1983 .

[8]  Vasant Honavar,et al.  Books-Received - Artificial Intelligence and Neural Networks - Steps Toward Principled Integration , 1994 .

[9]  Jerome H. Friedman,et al.  An Overview of Predictive Learning and Function Approximation , 1994 .

[10]  Joydeep Ghosh,et al.  Three techniques for extracting rules from feedforward networks , 1996 .

[11]  Michael I. Jordan,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1994, Neural Computation.

[12]  M. Hofri.  Analysis of Algorithms: Computational Methods & Mathematical Tools , 1995 .

[13]  David Mackay,et al.  Probable networks and plausible predictions - a review of practical Bayesian methods for supervised neural networks , 1995 .

[14]  David L. Waltz,et al.  Toward memory-based reasoning , 1986, CACM.

[15]  T. Ho.  A theory of multiple classifier systems and its application to visual word recognition , 1992 .

[16]  Lorien Y. Pratt,et al.  Experiments on the transfer of knowledge between neural networks , 1994, COLT 1994.

[17]  Joydeep Ghosh,et al.  On the design of supra-classifiers for knowledge reuse , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[18]  Rich Caruana,et al.  Learning Many Related Tasks at the Same Time with Backpropagation , 1994, NIPS.

[19]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[20]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[21]  Ronald Christensen,et al.  Log-Linear Models and Logistic Regression , 1997 .

[22]  Li-Min Fu,et al.  Knowledge-based connectionism for revising domain theories , 1993, IEEE Trans. Syst. Man Cybern..

[23]  Amanda J. C. Sharkey,et al.  On Combining Artificial Neural Nets , 1996, Connect. Sci..

[24]  Jude W. Shavlik,et al.  Knowledge-Based Artificial Neural Networks , 1994, Artif. Intell..

[25]  Joydeep Ghosh,et al.  A Supra-Classifier Architecture for Scalable Knowledge Reuse , 1998, ICML.

[26]  Philippe Flajolet,et al.  Analysis of algorithms , 2000, Random Struct. Algorithms.

[27]  Joydeep Ghosh,et al.  Use of localized gating in mixture of experts networks , 1998, Defense, Security, and Sensing.

[28]  Sebastian Thrun,et al.  Discovering Structure in Multiple Learning Tasks: The TC Algorithm , 1996, ICML.

[29]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[30]  Sebastian Thrun,et al.  Is Learning The n-th Thing Any Easier Than Learning The First? , 1995, NIPS.

[31]  Raymond J. Mooney,et al.  Combining Connectionist and Symbolic Learning to Refine Certainty Factor Rule Bases , 1993 .