Online Sequential Extreme Learning Machine With Dynamic Forgetting Factor

Online sequential extreme learning machine (OS-ELM) and its variants provide a promising way to solve data stream problems, but most of them do not take the timeliness of the problems into account, which may degrade the performance of the model. The main reason is that these algorithms are unable to adapt to the latest data accordingly when the distribution of the data stream changes. To mitigate this limitation, the forgetting factor is introduced into the relevant models, which is used to balance the relative importance of past data and new data when necessary. However, there is no efficient way to set the forgetting factor properly so far. In this paper, we have developed a novel updating strategy for setting the forgetting factor and proposed a dynamic forgetting factor based OS-ELM algorithm (DOS-ELM). In the sequential learning phase of DOS-ELM, the forgetting factor can be adjusted dynamically according to the change degree of the model accuracy in each learning epoch. This updating process does not require setting any parameters artificially and thus greatly improves the flexibility of the model. The experimental results on ten classification problems, five regression problems, one time-series problem show that DOS-ELM can deal well with both stationary and non-stationary data stream problems. In addition, we have extended DOS-ELM to an online deep model named ML-DOS-ELM, which can handle more complex tasks such as the face recognition problem and the handwritten digit recognition problem. Our experimental evaluations show that both DOS-ELM and ML-DOS-ELM can achieve higher prediction accuracy compared to the other similar algorithms.