论文信息 - A faster model selection criterion for OP-ELM and OP-KNN: Hannan-Quinn criterion

A faster model selection criterion for OP-ELM and OP-KNN: Hannan-Quinn criterion

The Optimally Pruned Extreme Learning Machine (OP- ELM) and Optimally Pruned K-Nearest Neighbors (OP-KNN) algorithms use the a similar methodology based on random initialization (OP-ELM) or KNN initialization (OP-KNN) of a Feedforward Neural Network fol- lowed by ranking of the neurons; ranking is used to determine the best combination to retain. This is achieved by Leave-One-Out (LOO) cross- validation. In this article is proposed to use the Hannan-Quinn (HQ) Criterion as a model selection criterion, instead of LOO. It proved to be efficient and as good as the LOO one for both OP-ELM and OP-KNN, while decreasing computations by factors of four to five for OP-ELM and up to 24 for OP-KNN.

Amaury Lendasse | Yoan Miché | Y. Miché | A. Lendasse

[1] B. G. Quinn,et al. The determination of the order of an autoregression , 1979 .

[2] Chee Kheong Siew,et al. Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[3] Kurt Hornik,et al. Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[4] R. Bhansali,et al. Some properties of the order of an autoregressive model selected by a generalization of Akaike∘s EPF criterion , 1977 .

[5] H. Akaike. A new look at the statistical model identification , 1974 .

[6] Johan A. K. Suykens,et al. Least Squares Support Vector Machines , 2002 .

[7] Timo Similä,et al. Multiresponse Sparse Regression with Application to Multidimensional Scaling , 2005, ICANN.

[8] Frank-Michael Schleif,et al. Advances in computational intelligence and learning , 2010 .

[9] G. Schwarz. Estimating the Dimension of a Model , 1978 .

[10] Amaury Lendasse,et al. A Methodology for Building Regression Models using Extreme Learning Machine: OP-ELM , 2008, ESANN.

[11] Alberto Guillén,et al. Optimal Pruned K-Nearest Neighbors: OP-KNN Application to Financial Modeling , 2008, 2008 Eighth International Conference on Hybrid Intelligent Systems.

[12] R. H. Myers. Classical and modern regression with applications , 1986 .