Theory and scope of exact representation extraction from feed-forward networks

An algorithm to extract representations from feed-forward threshold networks is outlined. The representation is based on polytopic decision regions in the input space - and is exact not an approximation. Using this exact representation we explore scope questions, such as when and where do networks form artifacts, or what can we tell about network generalization from its representation. The exact nature of the algorithm also lends itself to theoretical questions about representation extraction in general, such as what is the relationship between factors such as input dimensionality, number of hidden units, number of hidden layers, and how the network output is interpreted to the potential complexity of the network's function.

[1]  Guido Bologna,et al.  Symbolic Rule Extraction from the DIMLP Neural Network , 1998, Hybrid Neural Systems.

[2]  Sebastian Thrun,et al.  Extracting Provably Correct Rules from Artificial Neural Networks , 1993 .

[3]  Alexander Linden.  Iterative inversion of neural networks and its applications , 1997 .

[4]  Frédéric Maire,et al.  Rule-extraction by backpropagation of polyhedra , 1999, Neural Networks.

[5]  A. Lo,et al.  THE ECONOMETRICS OF FINANCIAL MARKETS , 1996, Macroeconomic Dynamics.

[6]  Alexander Schrijver,et al.  Theory of linear and integer programming , 1986, Wiley-Interscience series in discrete mathematics and optimization.

[7]  Dennis Sanger,et al.  Contribution analysis: a technique for assigning responsibilities to hidden units in connectionist networks , 1991 .

[8]  L. Khachiyan.  Complexity of Polytope Volume Computation , 1993 .

[9]  Terrence J. Sejnowski,et al.  Analysis of hidden units in a layered network trained to classify sonar targets , 1988, Neural Networks.

[10]  Thomas R. Shultz,et al.  Analysis of Unstandardized Contributions in Cross Connected Networks , 1994, NIPS.

[11]  Joachim Diederich,et al.  Survey and critique of techniques for extracting rules from trained artificial neural networks , 1995, Knowl. Based Syst..

[12]  Stefan Wermter,et al.  A Novel Modular Neural Architecture for Rule-Based and Similarity-Based Reasoning , 1998, Hybrid Neural Systems.

[13]  Herbert Edelsbrunner,et al.  Algorithms in Combinatorial Geometry , 1987, EATCS Monographs in Theoretical Computer Science.

[14]  Rudy Setiono,et al.  Extracting Rules from Neural Networks by Pruning and Hidden-Unit Splitting , 1997, Neural Computation.

[15]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .