On sequences of different adaptive mechanisms in non-stationary regression problems

Existing adaptive predictive methods often use multiple adaptive mechanisms as part of their coping strategy in non-stationary environments. These mechanisms are usually deployed in a prescribed order which does not change. In this work we investigate and provide a comparative analysis of the effects of using a flexible order of adaptive mechanisms' deployment resulting in varying adaptation sequences. As a vehicle for this comparison, we use an adaptive ensemble method for regression in batch learning mode which employs several adaptive mechanisms to react to the changes in data. Using real world data from the process industry we demonstrate that such flexible deployment of available adaptive methods embedded in a cross-validatory framework can benefit the predictive accuracy over time.

[1]  Bogdan Gabrys,et al.  Architecture for development of adaptive on-line prediction models , 2009, Memetic Comput..

[2]  B. Kowalski,et al.  Partial least-squares regression: a tutorial , 1986 .

[3]  Philip S. Yu,et al.  Mining concept-drifting data streams using ensemble classifiers , 2003, KDD '03.

[4]  Hiromasa Kaneko,et al.  Adaptive soft sensor based on online support vector regression and Bayesian ensemble learning for various states in chemical plants , 2014 .

[5]  Bruce Mizrach.  Forecast Comparison in L2 , 1996 .

[6]  J. M. Bates,et al.  The Combination of Forecasts , 1969 .

[7]  Lei Wu,et al.  Adaptive soft sensor modeling framework based on just-in-time learning and kernel partial least squares regression for nonlinear multiphase batch processes , 2014, Comput. Chem. Eng..

[8]  Rui Araújo,et al.  Online Mixture of Univariate Linear Regression Models for Adaptive Soft Sensors , 2014, IEEE Transactions on Industrial Informatics.

[9]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[10]  Michael J. Procopio,et al.  Learning terrain segmentation with classifier ensembles for autonomous robot navigation in unstructured environments , 2009 .

[11]  Ralf Klinkenberg,et al.  Boosting classifiers for drifting concepts , 2007, Intell. Data Anal..

[12]  Bogdan Gabrys,et al.  Soft Sensor Based on Adaptive Local Learning , 2008, ICONIP.

[13]  Bogdan Gabrys,et al.  Local learning‐based adaptive soft sensor for catalyst activation prediction , 2011 .

[14]  Robi Polikar,et al.  Incremental Learning of Concept Drift in Nonstationary Environments , 2011, IEEE Transactions on Neural Networks.

[15]  Hiromasa Kaneko,et al.  Selective Use of Adaptive Soft Sensors Based on Process State , 2014 .

[16]  Rui Araújo,et al.  A dynamic and on-line ensemble regression for changing environments , 2015, Expert Syst. Appl..

[17]  Steven D. Brown,et al.  A localized adaptive soft sensor for dynamic system modeling , 2014 .

[18]  PolikarRobi,et al.  Incremental Learning of Concept Drift in Nonstationary Environments , 2011 .

[19]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[20]  S. Qin.  Recursive PLS algorithms for adaptive data modeling , 1998 .

[21]  Bogdan Gabrys,et al.  Adaptive on-line prediction soft sensing without historical data , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[22]  Petr Kadlec,et al.  On robust and adaptive soft sensors. , 2009 .

[23]  Rui Araújo,et al.  An on-line weighted ensemble of regressor models to handle concept drifts , 2015, Eng. Appl. Artif. Intell..

[24]  Gregory Z. Grudic,et al.  Learning terrain segmentation with classifier ensembles for autonomous robot navigation in unstructured environments , 2009, J. Field Robotics.

[25]  Jesús S. Aguilar-Ruiz,et al.  Knowledge discovery from data streams: Guest Editorial , 2007 .

[26]  Ping Wang,et al.  Local Partial Least Squares Based Online Soft Sensing Method for Multi-output Processes with Adaptive Process States Division , 2014 .

[27]  Dražen Slišković,et al.  Adaptive soft sensor for online prediction and process monitoring based on a mixture of Gaussian process models , 2013, Comput. Chem. Eng..