Decision tree ensemble hardware accelerators for embedded applications

This paper presents four different architectures for the hardware acceleration of axis-parallel, oblique and non-linear decision tree ensemble classifier systems. Hardware architectures for the implementation of a number of ensemble combination rules are also presented. The proposed architectures are optimized for size, making them particularly interesting for embedded applications where the size of the system is critical constraint. Proposed architectures are suitable for the implementation using FPGA and ASIC technology. Experiment results obtained using 29 datasets from the standard UCI Machine Learning Repository database suggest that the FPGA implementations offer significant improvement in the classification time in comparison with the pure software implementations.

[1]  Simon Haykin,et al.  Neural Networks and Learning Machines , 2010 .

[2]  Maya Gokhale,et al.  Accelerating a Random Forest Classifier: Multi-Core, GP-GPU, or FPGA? , 2012, 2012 IEEE 20th International Symposium on Field-Programmable Custom Computing Machines.

[3]  Lior Rokach,et al.  Top-down induction of decision trees classifiers - a survey , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[4]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[5]  David A. Fenstermacher,et al.  Introduction to bioinformatics , 2005, J. Assoc. Inf. Sci. Technol..

[6]  Cha Zhang,et al.  Ensemble Machine Learning , 2012 .

[7]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[8]  Lior Rokach,et al.  Data Mining with Decision Trees - Theory and Applications , 2007, Series in Machine Perception and Artificial Intelligence.

[9]  Xindong Wu,et al.  The Top Ten Algorithms in Data Mining , 2009 .

[10]  Amine Bermak,et al.  A compact 3D VLSI classifier using bagging threshold network ensembles , 2003, IEEE Trans. Neural Networks.

[11]  Peter A. Flach,et al.  Machine Learning - The Art and Science of Algorithms that Make Sense of Data , 2012 .

[12]  Simon J. D. Prince,et al.  Computer Vision: Models, Learning, and Inference , 2012 .

[13]  Davide Anguita,et al.  A FPGA Core Generator for Embedded Classification Systems , 2011, J. Circuits Syst. Comput..

[14]  Simon J. D. Prince,et al.  Computer Vision: Index , 2012 .

[15]  Ian H. Witten,et al.  Data Mining: Practical Machine Learning Tools and Techniques, 3/E , 2014 .

[16]  Shigeo Abe.  Support Vector Machines for Pattern Classification , 2010, Advances in Pattern Recognition.

[17]  Cha Zhang,et al.  Ensemble Machine Learning: Methods and Applications , 2012 .

[18]  Olfa Nasraoui,et al.  Web data mining: exploring hyperlinks, contents, and usage data , 2008, SKDD.

[19]  อนิรุธ สืบสิงห์,et al.  Data Mining Practical Machine Learning Tools and Techniques , 2014 .

[20]  Shawki Areibi,et al.  A scalable pipelined architecture for real-time computation of MLP-BP neural networks , 2012, Microprocess. Microsystems.

[21]  Ching Y. Suen,et al.  The behavior-knowledge space method for combination of multiple classifiers , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[23]  J.R. Struharik,et al.  Implementing decision trees in hardware , 2011, 2011 IEEE 9th International Symposium on Intelligent Systems and Informatics.

[24]  Marios S. Pattichis,et al.  Pipelined Decision Tree Classification Accelerator Implementation in FPGA (DT-CAIF) , 2015, IEEE Transactions on Computers.

[25]  John Langford,et al.  Scaling up machine learning: parallel and distributed approaches , 2011, KDD '11 Tutorials.

[26]  Christos-Savvas Bouganis,et al.  Novel Cascade FPGA Accelerator for Support Vector Machines Classification , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[27]  Bing Liu,et al.  Web Data Mining: Exploring Hyperlinks, Contents, and Usage Data , 2006, Data-Centric Systems and Applications.

[28]  Bing Liu,et al.  Web Data Mining: Exploring Hyperlinks, Contents, and Usage Data , 2006, Data-Centric Systems and Applications.

[29]  Ran Ginosar,et al.  Scalable network-on-chip architecture for configurable neural networks , 2011, Microprocess. Microsystems.

[30]  Andreas Holzinger,et al.  Data Mining with Decision Trees: Theory and Applications , 2015, Online Inf. Rev..

[31]  Berkin Özisikyilmaz,et al.  Accelerating data mining workloads: current approaches and future challenges in system architecture design , 2011, WIREs Data Mining Knowl. Discov..