HC-CART: A parallel system implementation of data mining classification and regression tree (CART) algorithm on a multi-FPGA system

Data mining is a new field of computer science with a wide range of applications. Its goal is to extract knowledge from massive datasets in a human-understandable structure, for example, the decision trees. In this article we present an innovative, high-performance, system-level architecture for the Classification And Regression Tree (CART) algorithm, one of the most important and widely used algorithms in the data mining area. Our proposed architecture exploits parallelism at the decision variable level, and was fully implemented and evaluated on a modern high-performance reconfigurable platform, the Convey HC-1 server, that features four FPGAs and a multicore processor. Our FPGA-based implementation was integrated with the widely used “rpart” software library of the R project in order to provide the first fully functional reconfigurable system that can handle real-world large databases. The proposed system, named HC-CART system, achieves a performance speedup of up to two orders of magnitude compared to well-known single-threaded data mining software platforms, such as WEKA and the R platform. It also outperforms similar hardware systems which implement parts of the complete application by an order of magnitude. Finally, we show that the HC-CART system offers higher performance speedup than some other proposed parallel software implementations of decision tree construction algorithms.

[1]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[2]  Richard A. Olshen,et al.  CART: Classification and Regression Trees , 1984 .

[3]  Vipin Kumar,et al.  ScalParC: a new scalable and efficient parallel classification algorithm for mining large datasets , 1998, Proceedings of the First Merged International Parallel Processing Symposium and Symposium on Parallel and Distributed Processing.

[4]  Yan Luo,et al.  Acceleration of decision tree searching for IP traffic classification , 2008, ANCS '08.

[5]  Tsutomu Maruyama,et al.  An FPGA implementation of real-time K-means clustering for color images , 2007, Journal of Real-Time Image Processing.

[6]  Viktor K. Prasanna,et al.  Scalable regression tree learning on Hadoop using OpenPlanet , 2012, MapReduce '12.

[7]  Rakesh Agrawal,et al.  SPRINT: A Scalable Parallel Classifier for Data Mining , 1996, VLDB.

[8]  Xindong Wu,et al.  10 Challenging Problems in Data Mining Research , 2006, Int. J. Inf. Technol. Decis. Mak..

[9]  Alok Choudhary,et al.  Interactive presentation: An FPGA implementation of decision tree classification , 2007 .

[10]  Bingsheng He,et al.  Frequent itemset mining on graphics processors , 2009, DaMoN '09.

[11]  Jorma Rissanen,et al.  SLIQ: A Fast Scalable Classifier for Data Mining , 1996, EDBT.

[12]  Christos-Savvas Bouganis,et al.  Performance comparison of GPU and FPGA architectures for the SVM training problem , 2009, 2009 International Conference on Field-Programmable Technology.

[13]  Christos-Savvas Bouganis,et al.  A scalable FPGA architecture for non-linear SVM training , 2008, 2008 International Conference on Field-Programmable Technology.

[14]  D. Steinberg.  CART: Classification and Regression Trees , 2009 .

[15]  Gagan Agrawal,et al.  A translation system for enabling data mining applications on GPUs , 2009, ICS.

[16]  João Gama,et al.  Parallel Implementation of Decision Tree Learning Algorithms , 2001, EPIA.

[17]  Yael Ben-Haim,et al.  A Streaming Parallel Decision Tree Algorithm , 2010, J. Mach. Learn. Res..

[18]  Shuliang Wang,et al.  Data Mining and Knowledge Discovery , 2005, Mathematical Principles of the Internet.

[19]  Qing He,et al.  The High-Activity Parallel Implementation of Data Preprocessing Based on MapReduce , 2010, RSKT.

[20]  Viktor K. Prasanna,et al.  Efficient hardware data mining with the Apriori algorithm on FPGAs , 2005, 13th Annual IEEE Symposium on Field-Programmable Custom Computing Machines (FCCM'05).

[21]  Viktor K. Prasanna,et al.  An Architecture for Efficient Hardware Data Mining using Reconfigurable Computing Systems , 2006, 2006 14th Annual IEEE Symposium on Field-Programmable Custom Computing Machines.

[22]  James Theiler,et al.  Algorithmic transformations in the implementation of K- means clustering on reconfigurable hardware , 2001, FPGA '01.

[23]  Yu Wang,et al.  FPMR: MapReduce framework on FPGA , 2010, FPGA '10.

[24]  Ingo Mierswa,et al.  YALE: rapid prototyping for complex data mining tasks , 2006, KDD '06.

[25]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[26]  Miriam Leeser,et al.  K-means Clustering for Multispectral Images Using Floating-Point Divide , 2007 .

[27]  Alok N. Choudhary,et al.  An FPGA Implementation of Decision Tree Classification , 2007, 2007 Design, Automation & Test in Europe Conference & Exhibition.

[28]  JOHANNES GEHRKE,et al.  RainForest—A Framework for Fast Decision Tree Construction of Large Datasets , 1998, Data Mining and Knowledge Discovery.

[29]  T. Therneau,et al.  An Introduction to Recursive Partitioning Using the RPART Routines , 2015 .

[30]  Joseph Zambreno,et al.  Mining Association Rules with systolic trees , 2008, 2008 International Conference on Field Programmable Logic and Applications.

[31]  Roberto J. Bayardo,et al.  PLANET: Massively Parallel Learning of Tree Ensembles with MapReduce , 2009, Proc. VLDB Endow..

[32]  Vipin Kumar,et al.  Parallel Formulations of Decision-Tree Classification Algorithms , 2004, Data Mining and Knowledge Discovery.

[33]  Miriam Leeser,et al.  K-means Clustering for Multispectral Images Using Floating-Point Divide , 2007, 15th Annual IEEE Symposium on Field-Programmable Custom Computing Machines (FCCM 2007).

[34]  Matthew N. Anyanwu,et al.  Comparative Analysis of Serial Decision Tree Classification Algorithms , 2009 .

[35]  Ioannis Papaefstathiou,et al.  Novel and Highly Efficient Reconfigurable Implementation of Data Mining Classification Tree , 2011, 2011 21st International Conference on Field Programmable Logic and Applications.

[36]  John Langford,et al.  Scaling up machine learning: parallel and distributed approaches , 2011, KDD '11 Tutorials.

[37]  Joseph Zambreno,et al.  A Reconfigurable Platform for Frequent Pattern Mining , 2008, 2008 International Conference on Reconfigurable Computing and FPGAs.

[38]  Stephen Tyree,et al.  Parallel boosted regression trees for web search ranking , 2011, WWW.

[39]  Philip S. Yu,et al.  Top 10 algorithms in data mining , 2007, Knowledge and Information Systems.