Detecting and Reacting to Changes in Sensing Units: The Active Classifier Case

The ability to detect concept drift, i.e., a structural change in the acquired datastream, and react accordingly is a major achievement for intelligent sensing units. This ability allows the unit, for actively tuning the application, to maintain high performance, changing online the operational strategy, detecting and isolating possible occurring faults to name a few tasks. In the paper, we consider a just-in-time strategy for adaptation; the sensing unit reacts exactly when needed, i.e., when concept drift is detected. Change detection tests (CDTs), designed to inspect structural changes in industrial and environmental data, are coupled here with adaptive k-nearest neighbor and support vector machine classifiers, and suitably retrained when the change is detected. Computational complexity and memory requirements of the CDT and the classifier, due to precious limited resources in embedded sensing, are taken into account in the application design. We show that a hierarchical CDT coupled with an adaptive resource-aware classifier is a suitable tool for processing and classifying sequential streams of data.

[1]  F. Wilcoxon Individual Comparisons by Ranking Methods , 1945 .

[2]  G. Gates,et al.  The reduced nearest neighbor rule (Corresp.) , 1972, IEEE Trans. Inf. Theory.

[3]  Irving Wingate Burr Statistical quality control methods , 1976 .

[4]  R. Herrnstein,et al.  Natural concepts in pigeons. , 1976, Journal of experimental psychology. Animal behavior processes.

[5]  Ramanathan Gnanadesikan,et al.  Methods for statistical data analysis of multivariate observations , 1977, A Wiley publication in applied statistics.

[6]  R. Forthofer,et al.  Rank Correlation Methods , 1981 .

[7]  J. Flavell Piaget's Legacy , 1996 .

[8]  G. Gates The Reduced Nearest Neighbor Rule , 1998 .

[9]  Vladimir Katkovnik,et al.  A new method for varying adaptive bandwidth selection , 1999, IEEE Trans. Signal Process..

[10]  B. Manly,et al.  A cumulative sum type of method for environmental monitoring , 2000 .

[11]  Samy Bengio,et al.  SVMTorch: Support Vector Machines for Large-Scale Regression Problems , 2001, J. Mach. Learn. Res..

[12]  Maria Gabriella Xibilia,et al.  An innovative intelligent system for sensor validation in tokamak machines , 2002, IEEE Trans. Control. Syst. Technol..

[13]  Dustin Boswell,et al.  Introduction to Support Vector Machines , 2002 .

[14]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[15]  A. Goldenshluger On Spatial Adaptive Estimation of Nonparametric Regression , 2004 .

[16]  Jason Weston,et al.  Fast Kernel Classifiers with Online and Active Learning , 2005, J. Mach. Learn. Res..

[17]  Eleftheria Athanasopoulou,et al.  Probabilistic approaches to fault detection in networked discrete event systems , 2005, IEEE Transactions on Neural Networks.

[18]  Pavel Pudil,et al.  Introduction to Statistical Pattern Recognition , 2006 .

[19]  Giuseppe Acciani,et al.  Application of neural networks in optical inspection and classification of solder joints in surface mount technology , 2006, IEEE Transactions on Industrial Informatics.

[20]  Nicholas R. Jennings,et al.  A utility-based sensing and communication model for a glacial sensor network , 2006, AAMAS '06.

[21]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[22]  Benjamin Schrauwen,et al.  An overview of reservoir computing: theory, applications and implementations , 2007, ESANN.

[23]  Cesare Alippi,et al.  Just-in-Time Adaptive Classifiers—Part II: Designing the Classifier , 2008, IEEE Transactions on Neural Networks.

[24]  Wang Wei,et al.  Gird-pattern method for model selection of support vector machines , 2008 .

[25]  Nathan Srebro,et al.  SVM optimization: inverse dependence on training set size , 2008, ICML '08.

[26]  Cesare Alippi,et al.  Just-in-Time Adaptive Classifiers—Part I: Detecting Nonstationary Changes , 2008, IEEE Transactions on Neural Networks.

[27]  Mario Di Francesco,et al.  Energy conservation in wireless sensor networks: A survey , 2009, Ad Hoc Networks.

[28]  Lior Rokach,et al.  Ensemble-based classifiers , 2010, Artificial Intelligence Review.

[29]  Giuseppe Anastasi,et al.  Energy management in wireless sensor networks with energy-hungry sensors , 2009, IEEE Instrumentation & Measurement Magazine.

[30]  Cesare Alippi,et al.  Adaptive Classifiers with ICI-Based Adaptive Knowledge Base Management , 2010, ICANN.

[31]  Dimitris K. Tasoulis,et al.  Nonparametric Monitoring of Data Streams for Changes in Location and Scale , 2011, Technometrics.

[32]  Cesare Alippi,et al.  A just-in-time adaptive classification system based on the intersection of confidence intervals rule , 2011, Neural Networks.

[33]  Cesare Alippi,et al.  An effective just-in-time adaptive classifier for gradual concept drifts , 2011, The 2011 International Joint Conference on Neural Networks.

[34]  Robi Polikar,et al.  Incremental Learning of Concept Drift in Nonstationary Environments , 2011, IEEE Transactions on Neural Networks.

[35]  Dongbin Zhao,et al.  SVM-Based Just-in-Time Adaptive Classifiers , 2012, ICONIP.

[36]  Michael B. Miller Linear Regression Analysis , 2013 .

[37]  Cesare Alippi,et al.  Just-In-Time Classifiers for Recurrent Concepts , 2013, IEEE Transactions on Neural Networks and Learning Systems.