Using Control Charts for Detecting Concept Change in Streaming Data

We address adaptive online classification in the presence of concept change. An overview of the machine learning approaches reveals a deficit of methods for explicit detection of change when the only information is the classification error of the streaming data. We look to borrow answers from the longstanding research in monitoring process quality by using control charts. Four methods for change detection are detailed and compared in the paper: two from the machine learning literature and two control charts (Shewhart and Sequential Probability Ratio Test (SPRT)). Control charts would only signal a change. To examine empirically their effect on the classification accuracy, the Shewhart and SPRT methods were equipped with a window resizing heuristic. We chose to grow the window until a change has been detected, and shrink it to a batch size upon detection. Experiments with 28 real data sets were carried out, where change was simulated by swapping class labels. Paired t-test on the classification error and paired Wilcoxon signed rank test picked SPRT as the best change detection method.