Experimental evaluation of performance improvements in abductive network classifiers with problem decomposition

Problem decomposition and divide-and-conquer strategies have been proposed to improve the performance and realization of neural network solutions for complex problems. This paper reports on an experimental evaluation of performance gains brought about by problem decomposition for abductive network classifiers that classify four noisy waveform patterns having two waveform types (sine/cosine) and two different frequencies. Two-stage problem decomposition improves overall classification accuracy from 87.2% to 99%. Problem decomposition classifiers were found to be much more tolerant to model simplification and reduction in the training set size compared to monolithic solutions. This allows trading-off some of the large gain in classification performance for some other advantages that may be quite desirable in some applications, such as simpler models that execute faster and are easier to implement, smaller training sets, and shorter training times. A problem decomposition classifier is more accurate than a monolithic classifier in spite of the former being five times simpler, executing over two times faster, requiring one-fifth of the training data, and synthesized in one-eleventh of the training time. Performance is comparable with a neural network solution using the same decomposition method and significantly superior to an abductive network committee approach.

[1]  R. E. Abdel-Aal,et al.  Automatic fitting of Gaussian peaks using abductive machine learning , 1998 .

[2]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[3]  M. Basu,et al.  Gating improves neural network performance , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[4]  A.C.P.L.F. de Carvalho,et al.  Recognition of vehicles silhouette using combination of classifiers , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[5]  H. El-Bakry.  Face detection using neural networks and image decomposition , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[6]  Andrzej Bielecki,et al.  A hybrid neural system for phonematic transformation , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[7]  Chee Peng Lim,et al.  Predicting drug dissolution profiles with an ensemble of boosted neural networks: a time series approach , 2003, IEEE Trans. Neural Networks.

[8]  A.J.R. Reis,et al.  NeuroDem-a neural network based short term demand forecaster , 2001, 2001 IEEE Porto Power Tech Proceedings (Cat. No.01EX502).

[9]  David A. Landgrebe,et al.  A decision tree classifier design for high-dimensional data with limited training samples , 1996, IGARSS '96. 1996 International Geoscience and Remote Sensing Symposium.

[10]  Pavel Pudil,et al.  Introduction to Statistical Pattern Recognition , 2006 .

[11]  C. A. Kamm,et al.  Improving a phoneme classification neural network through problem decomposition , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[12]  Zheru Chi,et al.  Neural networks with problem decomposition for finding real roots of polynomials , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[13]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[14]  Manfred M. Fischer,et al.  Neural network ensembles and their application to traffic flow prediction in telecommunications networks , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[15]  Kishan G. Mehrotra,et al.  Efficient classification for multiclass problems using modular neural networks , 1995, IEEE Trans. Neural Networks.

[16]  D. Edelman,et al.  Using neural network prediction to arbitrage the Australian All-Ordinaries Index , 1999, 1999 Third International Conference on Knowledge-Based Intelligent Information Engineering Systems. Proceedings (Cat. No.99TH8410).

[17]  P. Liang.  Problem decomposition and subgoaling in artificial neural networks , 1990, 1990 IEEE International Conference on Systems, Man, and Cybernetics Conference Proceedings.

[18]  Y. S. Tarng,et al.  An abductive network for predicting tool life in drilling , 1995, Proceedings IEEE Conference on Industrial Automation and Control Emerging Technology Applications.

[19]  Prampero,et al.  Classifier combination for vehicle silhouettes recognition , 1999 .

[20]  Keith C. Drake,et al.  Abductive networks , 1990, Defense, Security, and Sensing.

[21]  Steven Guan,et al.  Parallel growing and training of neural networks using output parallelism , 2002, IEEE Trans. Neural Networks.

[22]  A. A. Ghorbani,et al.  Stacked generalization in neural networks: generalization on statistically neutral problems , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[23]  Han Wen,et al.  Auditory brainstem response classification using modular neural networks , 1991, Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society Volume 13: 1991.

[24]  Yakov A. Pachepsky,et al.  Predicting crop yields under climate change conditions from monthly GCM weather projection , 2000, Environ. Model. Softw..

[25]  D. Jimenez,et al.  Dynamically weighted ensemble neural networks for classification , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[26]  Robert E. Jenkins,et al.  A simplified neural network solution through problem decomposition: the case of the truck backer-upper , 1993, IEEE Trans. Neural Networks.

[27]  Vladimir Radevski,et al.  Reliability control in committee classifier environment , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[28]  R. E. Abdel-Aal,et al.  Using abductive machine learning for online vibration monitoring of turbo molecular pumps , 1999 .

[29]  W. J. DeCoursey,et al.  Introduction: Probability and Statistics , 2003 .

[30]  Hong-Tzer Yang,et al.  Abductive network model-based diagnosis system for power transformer incipient fault detection , 2002 .

[31]  Stanley J. Farlow,et al.  Self-Organizing Methods in Modeling: Gmdh Type Algorithms , 1984 .

[32]  A. A. Ilumoka,et al.  Efficient and accurate crosstalk prediction via neural net-based topological decomposition of 3-D interconnect , 2001 .