Structural identifiability of generalized constraint neural network models for nonlinear regression

Identifiability becomes an essential requirement for learning machines when the models contain physically interpretable parameters. This paper presents two approaches to examining structural identifiability of the generalized constraint neural network (GCNN) models by viewing the model from two different perspectives. First, by taking the model as a static deterministic function, a functional framework is established, which can recognize deficient model and at the same time reparameterize it through a pairwise-mode symbolic examination. Second, by viewing the model as the mean function of an isotropic Gaussian conditional distribution, the algebraic approaches [E.A. Catchpole, B.J.T. Morgan, Detecting parameter redundancy, Biometrika 84 (1) (1997) 187-196] are extended to deal with multivariate nonlinear regression models through symbolically checking linear dependence of the derivative functional vectors. Examples are presented in which the proposed approaches are applied to GCNN nonlinear regression models that contain coupling physically interpretable parameters.

[1]  Byron J. T. Morgan,et al.  Estimation in parameter-redundant models , 1998 .

[2]  David G. Stork,et al.  Pattern Classification , 1973 .

[3]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[4]  Arye Nehorai,et al.  On identifiability and information-regularity in parametrized normal distributions , 1997 .

[5]  Carlo Novara,et al.  Parametric and nonparametric curve fitting , 2006, Autom..

[6]  C. H. Edwards,et al.  Differential Equations and Boundary Value Problems: Computing and Modeling , 1985 .

[7]  Paul M. J. Van den Hof,et al.  Test for local structural identifiability of high-order non-linearly parametrized state space models , 1996, Autom..

[8]  Bernard Hanzon,et al.  Identifiability of homogeneous systems using the state isomorphism approach , 2005, Autom..

[9]  Ghislaine Joly-Blanchard,et al.  System Identifiability (Symbolic Computation) and Parameter Estimation (Numerical Computation) , 2003, Numerical Algorithms.

[10]  Eric Walter,et al.  Identification of Parametric Models: from Experimental Data , 1997 .

[11]  Nicolas Le Roux,et al.  Convex Neural Networks , 2005, NIPS.

[12]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[13]  Eric Walter,et al.  On the identifiability and distinguishability of nonlinear parametric models , 1996 .

[14]  Adrian Corduneanu,et al.  Stable Mixing of Complete and Incomplete Information , 2014 .

[15]  Vasileios Basios,et al.  A method for approximating one-dimensional functions , 1997 .

[16]  Keith R. Godfrey,et al.  Structural identifiability of non-linear systems using linear/non-linear splitting , 2003 .

[17]  Edison Tse,et al.  On the identifiability of parameters , 1971, CDC 1971.

[18]  G. Casella,et al.  Statistical Inference , 2003, Encyclopedia of Social Network Analysis and Mining.

[19]  Stefan Wermter,et al.  A Novel Modular Neural Architecture for Rule-Based and Similarity-Based Reasoning , 1998, Hybrid Neural Systems.

[20]  Eric Walter,et al.  Identifiability and distinguishability concepts in electrochemistry , 1996, Autom..

[21]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[22]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[23]  Stefan Wermter,et al.  Hybrid neural systems: from simple coupling to fully integrated neural networks , 1999 .

[24]  Byron J. T. Morgan,et al.  Detecting parameter redundancy , 1997 .

[25]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[26]  Tom Wansbeek,et al.  Identification in parametric models , 2001 .

[27]  C. Henry Edwards,et al.  Differential Equations Computing and Modeling , 2007 .

[28]  Byron J. T. Morgan,et al.  Solving problems in parameter redundancy using computer algebra , 2002 .

[29]  Yong Wang,et al.  A generalized-constraint neural network model: Associating partially known relationships for nonlinear regressions , 2009, Inf. Sci..