Online GRNN-Based Ensembles for Regression on Evolving Data Streams

In this paper, a novel procedure for regression analysis in the case of non-stationary data streams is presented. Despite numerous applications, the regression task is rarely considered in a scientific literature, e.g. compared to classification task. The proposed method applies an ensemble technique to deal with data streams (especially with concept drift). As weak learners, a nonparametric estimator of regression is used. Every single weak model (weak learner) is able to track a specific type of the non-stationarity. The experimental section demonstrates that the proposed algorithm allows for tracking different types nonstationarities and increases accuracy with respect to a single estimator.

[1]  J. Fox,et al.  Applied Regression Analysis and Generalized Linear Models , 2008 .

[2]  Piotr Duda,et al.  Heuristic Regression Function Estimation Methods for Data Streams with Concept Drift , 2017, ICAISC.

[3]  Saso Dzeroski,et al.  Learning model trees from evolving data streams , 2010, Data Mining and Knowledge Discovery.

[4]  João Gama,et al.  Ensemble learning for data stream analysis: A survey , 2017, Inf. Fusion.

[5]  Bogdan Gabrys,et al.  Local learning‐based adaptive soft sensor for catalyst activation prediction , 2011 .

[6]  Saso Dzeroski,et al.  Online tree-based ensembles and option trees for regression on evolving data streams , 2015, Neurocomputing.

[7]  Erol Egrioglu,et al.  An ARMA Type Pi-Sigma Artificial Neural Network for Nonlinear Time Series Forecasting , 2018, J. Artif. Intell. Soft Comput. Res..

[8]  Piotr Duda,et al.  Convergent Time-Varying Regression Models for Data Streams: Tracking Concept Drift by the Recursive Parzen-Based Generalized Regression Neural Networks , 2017, Int. J. Neural Syst..

[9]  Claudia Eckert,et al.  Lazy Gaussian Process Committee for Real-Time Online Regression , 2013, AAAI.

[10]  Donald F. Specht,et al.  A general regression neural network , 1991, IEEE Trans. Neural Networks.

[11]  S. Chatterjee,et al.  Regression Analysis by Example , 1979 .

[12]  Marcus A. Maloof,et al.  Using additive expert ensembles to cope with concept drift , 2005, ICML.

[13]  Piotr Duda,et al.  How to adjust an ensemble size in stream data mining? , 2017, Inf. Sci..

[14]  V. Susheela Devi,et al.  Parallel MCNN (pMCNN) with Application to Prototype Selection on Large and Streaming Data , 2017, J. Artif. Intell. Soft Comput. Res..

[15]  W. Greblicki,et al.  Nonparametric system identification , 2008 .

[16]  Stuart J. Russell,et al.  Online bagging and boosting , 2005, 2005 IEEE International Conference on Systems, Man and Cybernetics.

[17]  João Gama,et al.  Adaptive Model Rules From High-Speed Data Streams , 2014, BigMine.

[18]  Bernard Zenko,et al.  Speeding-Up Hoeffding-Based Regression Trees With Options , 2011, ICML.