Learning Cartesian Granule Feature Models

In the previous chapter, it was shown how Cartesian granule feature models exploit a divide-and-conquer strategy to representation, capturing knowledge in terms of a network of low-order semantically related features. Both classification and prediction problems can be modelled quite naturally in terms of these models. This chapter describes a constructive induction algorithm, G_DACG (Genetic Discovery of Additive Cartesian Granule feature models), which facilitates the learning of such models from example data [Shanahan 1998; Shanahan, Baldwin and Martin 1999]. This involves two main steps: language identification (identification of the low-order semantically related features in terms of Cartesian granule features); and parameter identification of class fuzzy sets and rules. The C_DACG algorithm achieves this by embracing the synergistic spirit of soft computing, using genetic programming to discover the language (structure) of the model fuzzy sets and evidential rules for knowledge representation, while relying on the well-developed probability theory for learning the parameters of the model.

[1]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory, Third Edition , 1989, Springer Series in Information Sciences.

[2]  J. Bezdek.  A Physical Interpretation of Fuzzy ISODATA , 1993 .

[3]  James F. Baldwin,et al.  System Identification of Fuzzy Cartesian Granules Feature Models Using Genetic Programming , 1997, Fuzzy Logic in Artificial Intelligence.

[4]  M. J. D. Powell,et al.  An efficient method for finding the minimum of a function of several variables without calculating derivatives , 1964, Comput. J..

[5]  Thomas G. Dietterich,et al.  Learning with Many Irrelevant Features , 1991, AAAI.

[6]  A. G. Ivakhnenko,et al.  Polynomial Theory of Complex Systems , 1971, IEEE Trans. Syst. Man Cybern..

[7]  Kevin Martin Bossley,et al.  Neurofuzzy modelling approaches in system identification , 1997 .

[8]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[9]  John R. Koza,et al.  Genetic Programming II , 1992 .

[10]  Trevor P Martin,et al.  Modelling with words using Cartesian granule features , 1997, Proceedings of 6th International Fuzzy Systems Conference.

[11]  Gilbert Syswerda,et al.  Uniform Crossover in Genetic Algorithms , 1989, ICGA.

[12]  Josef Kittler,et al.  Pattern recognition : a statistical approach , 1982 .

[13]  Kalyanmoy Deb,et al.  A Comparative Analysis of Selection Schemes Used in Genetic Algorithms , 1990, FOGA.

[14]  Marvin Minsky,et al.  Perceptrons: An Introduction to Computational Geometry , 1969 .

[15]  I. Jolliffe.  Principal Component Analysis , 2002 .

[16]  Trevor P Martin,et al.  Constructive induction of fuzzy Cartesian granule feature models using genetic programming with applications , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[17]  Larry A. Rendell,et al.  A Practical Approach to Feature Selection , 1992, ML.

[18]  Igor Kononenko,et al.  Attribute selection for modelling , 1997, Future Gener. Comput. Syst..

[19]  James F. Baldwin,et al.  Evidential support logic, FRIL and case based reasoning , 1993, Int. J. Intell. Syst..

[20]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[21]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[22]  Bernadette Bouchon-Meunier,et al.  Learning from Imperfect Data , 1997 .

[23]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[24]  George J. Klir,et al.  Fuzzy sets and fuzzy logic - theory and applications , 1995 .

[25]  Martin Fodslette Møller,et al.  A scaled conjugate gradient algorithm for fast supervised learning , 1993, Neural Networks.

[26]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[27]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[28]  Thomas G. Dietterich,et al.  A Comparative Review of Selected Methods for Learning from Examples , 1983 .

[29]  Michel Grabisch,et al.  Classification by fuzzy integral: performance and tests , 1994, CVPR 1994.

[30]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[31]  James F. Baldwin,et al.  Combining evidences for evidential reasoning , 1991, Int. J. Intell. Syst..

[32]  Walter Alden Tackett,et al.  Mining the Genetic Program , 1995, IEEE Expert.

[33]  T. Kavli.  ASMO—Dan algorithm for adaptive spline modelling of observation data , 1993 .

[34]  Ron Kohavi,et al.  Wrappers for feature selection , 1997 .

[35]  G. P. Fletcher,et al.  Using neural networks as a tool for constructing rule based systems , 1995, Knowl. Based Syst..

[36]  Pat Langley,et al.  Selection of Relevant Features and Examples in Machine Learning , 1997, Artif. Intell..