Medical diagnosis with C4.5 rule preceded by artificial neural network ensemble

Comprehensibility is very important when machine learning techniques are used in computer-aided medical diagnosis. Since an artificial neural network ensemble is composed of multiple artificial neural networks, its comprehensibility is worse than that of a single artificial neural network. In this paper, C4.5 Rule-PANE, which combines an artificial neural network ensemble with rule induction by regarding the former as a preprocess of the latter, is proposed. At first, an artificial neural network ensemble is trained. Then, a new training data set is generated by feeding the feature vectors of original training instances to the trained ensemble and replacing the expected class labels of original training instances with the class labels output from the ensemble. Additional training data may also be appended by randomly generating feature vectors and combining them with their corresponding class labels output from the ensemble. Finally, a specific rule induction approach, i.e., C4.5 Rule, is used to learn rules from the new training data set. Case studies on diabetes, hepatitis , and breast cancer show that C4.5 Rule-PANE could generate rules with strong generalization ability, which benefits from an artificial neural network ensemble, and strong comprehensibility, which benefits from rule induction.

[1]  Joachim Diederich,et al.  Survey and critique of techniques for extracting rules from trained artificial neural networks , 1995, Knowl. Based Syst..

[2]  David W. Opitz,et al.  Actively Searching for an E(cid:11)ective Neural-Network Ensemble , 1996 .

[3]  Alberto Maria Segre,et al.  Programs for Machine Learning , 1994 .

[4]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[5]  Yoichi Hayashi,et al.  Fuzzy and Crisp Logical Rule Extraction Methods in Application to Medical Data , 2000 .

[6]  R. Setiono.  Extracting Rules from Pruned Neural Networks for Breast Cancer Diagnosis , 1996 .

[7]  Ronald L. Rivest,et al.  Inferring Decision Trees Using the Minimum Description Length Principle , 1989, Inf. Comput..

[8]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[9]  R. Schapire.  The Strength of Weak Learnability , 1990, Machine Learning.

[10]  Jianchang Mao,et al.  A case study on bagging, boosting and basic ensembles of neural networks for OCR , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[11]  Pádraig Cunningham,et al.  Stability problems with artificial neural networks and the ensemble solution , 2000, Artif. Intell. Medicine.

[12]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[13]  P. H. Sönksen,et al.  Data mining for indicators of early mortality in a database of clinical records , 2001, Artif. Intell. Medicine.

[14]  Christopher J. Merz,et al.  UCI Repository of Machine Learning Databases , 1996 .

[15]  Joachim Diederich,et al.  The truth will come to light: directions and challenges in extracting the knowledge embedded within trained artificial neural networks , 1998, IEEE Trans. Neural Networks.

[16]  Sven Loncaric,et al.  Rule-Based Labeling of CT Head Image , 1997, AIME.

[17]  Nathan Intrator,et al.  Classification of seismic signals by integrating ensembles of neural networks , 1998, IEEE Trans. Signal Process..

[18]  Kevin J. Cherkauer.  Human Expert-level Performance on a Scientiic Image Analysis Task by a System Using Combined Artiicial Neural Networks , 1996 .

[19]  Lars Kai Hansen,et al.  Ensemble methods for handwritten digit recognition , 1992, Neural Networks for Signal Processing II Proceedings of the 1992 IEEE Workshop.

[20]  Noel E. Sharkey,et al.  Adapting an Ensemble Approach for the Diagnosis of Breast Cancer , 1998 .

[21]  Tsuhan Chen,et al.  Pose invariant face recognition , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[22]  Igor Kononenko,et al.  Machine learning for medical diagnosis: history, state of the art and perspective , 2001, Artif. Intell. Medicine.

[23]  Amanda J. C. Sharkey,et al.  Combining Artificial Neural Nets: Ensemble and Modular Multi-Net Systems , 1999 .

[24]  Harris Drucker,et al.  Improving Performance in Neural Networks Using a Boosting Algorithm , 1992, NIPS.

[25]  Wei Tang,et al.  Ensembling neural networks: Many could be better than all , 2002, Artif. Intell..

[26]  D. E. Rumelhart,et al.  Learning internal representations by back-propagating errors , 1986 .

[27]  Yu-Bin Yang,et al.  Lung cancer cell identification based on artificial neural network ensembles , 2002, Artif. Intell. Medicine.

[28]  Katsumi Yoshida,et al.  A comparison between two neural network rule extraction techniques for the diagnosis of hepatobiliary disorders , 2000, Artif. Intell. Medicine.

[29]  Rudy Setiono,et al.  Generating concise and accurate classification rules for breast cancer diagnosis , 2000, Artif. Intell. Medicine.

[30]  Harry Wechsler,et al.  Face recognition using hybrid classifier systems , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[31]  Paulo J. G. Lisboa,et al.  Artificial Neural Networks in Biomedicine , 2000, Perspectives in Neural Computing.