A^2DTEL: Attention-Aware based Deep Tree Ensemble Learning

In recent years, random forest, as a classic ensemble learning method, has been widely used in many real scenarios benefited from its advantages such as superior performance and easy implementation. Random forest has high generalization ability and can achieve quantitative backtracking of the importance of decision feature parameters during the prediction processing, but it is fail to effectively learn the representation from the input like some deep learning based methods can do, which limits the prediction accuracy unavoidably. To address this issue, this paper proposes an attention-aware deep tree ensemble learning method. Specifically, it introduces the idea of deep learning into the random forest to design a multilayer stacking prediction model. In this way, the prediction result of the upper layer can be weighted to derive the attention parameter of the input sample at the current layer so that the layered importance sampling can be achieved. Compared with the traditional random forest model, the proposed method can learn representation from the input in a more effectively way. In experimental results, the proposed method can outperform other baselines in two real classification tasks significantly.