Prestructuring neural networks via extended dependency analysis with application to pattern classification

We consider the problem of matching domain-specific statistical structure to neural-network (NN) architecture. In past work we have considered this problem in the function approximation context; here we consider the pattern classification context. General Systems Methodology tools for finding problem-domain structure suffer exponential scaling of computation with respect to the number of variables considered. Therefore we introduce the use of Extended Dependency Analysis (EDA), which scales only polynomially in the number of variables, for the desired analysis. Based on EDA, we demonstrate a number of NN pre-structuring techniques applicable for building neural classifiers. An example is provided in which EDA results in significant dimension reduction of the input space, as well as capability for direct design of an NN classifier.

[1]  Bush Jones,et al.  Architecture of systems problem solving , 1986, Journal of the American Society for Information Science.

[2]  R. Hecht-Nielsen.  Counterpropagation networks. , 1987, Applied optics.

[3]  R. Palmer,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[4]  W. Ashby,et al.  Constraint analysis of many-dimensional relations. , 1965, Progress in biocybernetics.

[5]  George G. Lendaris,et al.  Improving ANN generalization using a priori knowledge to pre-structure ANNs , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[6]  Russell Reed,et al.  Pruning algorithms-a survey , 1993, IEEE Trans. Neural Networks.

[7]  G. Lendaris,et al.  On Prestructuring ANNs Using A Priori Knowledge , 1994 .

[8]  John N. Warfield,et al.  SOCIETAL SYSTEMS Planning, Policy and Complexity , 1978 .

[9]  George G. Lendaris,et al.  Diffraction-pattern sampling for automatic pattern recognition , 1970 .

[10]  Ashby Wr,et al.  Constraint analysis of many-dimensional relations. , 1965 .

[11]  Roger C. Conant,et al.  EXTENDED DEPENDENCY ANALYSIS OF LARGE SYSTEMS , 1988 .

[12]  George G. Lendaris,et al.  Use of a structured problem domain to explore development of modularized neural networks , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[13]  G. Lendaris,et al.  Using A Priori Knowledge to Prestructure ANNs , 1994 .

[14]  Tsu-Chang Lee,et al.  Structure level adaptation for artificial neural networks , 1991 .

[15]  G. Klir.  IDENTIFICATION OF GENERATIVE STRUCTURES IN EMPIRICAL DATA , 1976 .

[16]  Klaus Krippendorff,et al.  On the Identification of Structures in Multivariate Data by the Spectral Analysis of Relations , 1979 .

[17]  Dušan Petrovački,et al.  Evolutional development of a multilevel neural network , 1993, Neural Networks.

[18]  George G. Lendaris,et al.  Structural Modeling a Tutorial Guide , 1980, IEEE Transactions on Systems, Man, and Cybernetics.

[19]  Klaus Krippendorff,et al.  Information Theory: Structural Models for Qualitative Data. , 1988 .

[20]  M. Zwick.  CONTROL UNIQUENESS IN RECONSTRUCTABILITY ANALYSIS , 1996 .

[21]  George G. Lendaris,et al.  On Matching ANN Structure to Problem Domain Structure , 1993 .

[22]  George G. Lendaris,et al.  Structure and constraint in discrete adaptive networks , 1965 .

[23]  Robert Hecht-Nielsen,et al.  Applications of counterpropagation networks , 1988, Neural Networks.

[24]  George G. Lendaris,et al.  Prestructuring ANNs via a priori knowledge (Abstract only) , 1995, SPIE Defense + Commercial Sensing.

[25]  Martin Zwick.  Complexity Reduction in State-based Modeling , 1998 .

[26]  Shu Hui,et al.  Information-Theoretic Mask Analysis of Rainfall Time Series Data , 1995 .