Regression rules extraction from artificial neural network based on least squares

As an intelligent approach, the artificial neural network (ANN) has been shown to exhibit superior predictive power compared to traditional approaches in many studies. However, its further development is restricted by its characteristic of “black box” because it can't explain how learning from input data was done and how performance can be consistently ensured. In order to open the “black box”, a novel approach of regression rules extraction from ANN based on Least Squares is proposed in this paper, it can uncover underlying dependencies between the input data and the output data of ANN. Its main concepts are that the nonlinear activation function of each hidden unit is approximated locally by a three-piece linear function based on Least Squares and then the regression rules are generated by decision tree approach. The piecewise linear regression rules can not only ensure the accuracy but also enhance the explanation. Experiments on two public datasets show that the proposed approach generates rules are more accurate than the existing approaches based on decision trees or linear regression.