Novel Cascade FPGA Accelerator for Support Vector Machines Classification

Support vector machines (SVMs) are a powerful machine learning tool, providing state-of-the-art accuracy to many classification problems. However, SVM classification is a computationally complex task, suffering from linear dependencies on the number of the support vectors and the problem's dimensionality. This paper presents a fully scalable field programmable gate array (FPGA) architecture for the acceleration of SVM classification, which exploits the device heterogeneity and the dynamic range diversities among the dataset attributes. An adaptive and fully-customized processing unit is proposed, which utilizes the available heterogeneous resources of a modern FPGA device in efficient way with respect to the problem's characteristics. The implementation results demonstrate the efficiency of the heterogeneous architecture, presenting a speed-up factor of 2-3 orders of magnitude, compared to the CPU implementation. The proposed architecture outperforms other proposed FPGA and graphic processor unit approaches by more than seven times. Furthermore, based on the special properties of the heterogeneous architecture, this paper introduces the first FPGA-oriented cascade SVM classifier scheme, which exploits the FPGA reconfigurability and intensifies the custom-arithmetic properties of the heterogeneous architecture. The results show that the proposed cascade scheme is able to increase the heterogeneous classifier throughput even further, without introducing any penalty on the resource utilization.

[1]  Markos Papadonikolakis,et al.  A novel FPGA-based SVM classifier , 2010, 2010 International Conference on Field-Programmable Technology.

[2]  I. Kukenys,et al.  Classifier cascades for support vector machines , 2008, 2008 23rd International Conference Image and Vision Computing New Zealand.

[3]  Thorsten Joachims,et al.  Training linear SVMs in linear time , 2006, KDD '06.

[4]  William M. Pottenger,et al.  Hardware-based support vector machine classification in logarithmic number systems , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[5]  Kurt Keutzer,et al.  Fast support vector machine training and classification on graphics processors , 2008, ICML '08.

[6]  Christos-Savvas Bouganis,et al.  Performance comparison of GPU and FPGA architectures for the SVM training problem , 2009, 2009 International Conference on Field-Programmable Technology.

[7]  Mong-Kai Ku,et al.  Support vector machine FPGA implementation for video shot boundary detection application , 2009, 2009 IEEE International SOC Conference (SOCC).

[8]  Christopher J. C. Burges,et al.  Simplified Support Vector Decision Rules , 1996, ICML.

[9]  S. Sathiya Keerthi,et al.  A fast iterative nearest point algorithm for support vector machine classifier design , 2000, IEEE Trans. Neural Networks Learn. Syst..

[10]  E. Gilbert.  An Iterative Procedure for Computing the Minimum of a Quadratic Form on a Convex Set , 1966 .

[11]  Chih-Jen Lin,et al.  A Practical Guide to Support Vector Classication , 2008 .

[12]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[13]  Bernhard Schölkopf,et al.  Improving the Accuracy and Speed of Support Vector Machines , 1996, NIPS.

[14]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[15]  Martin Langhammer.  Floating point datapath synthesis for FPGAs , 2008, 2008 International Conference on Field Programmable Logic and Applications.

[16]  Christos-Savvas Bouganis,et al.  A scalable FPGA architecture for non-linear SVM training , 2008, 2008 International Conference on Field-Programmable Technology.

[17]  B. Schölkopf,et al.  Efficient face detection by a cascaded support–vector machine expansion , 2004, Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[18]  Narayanan Vijaykrishnan,et al.  A Hardware Efficient Support Vector Machine Architecture for FPGA , 2008, 2008 16th International Symposium on Field-Programmable Custom Computing Machines.

[19]  Shawn Martin.  Training support vector machines using Gilbert's algorithm , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[20]  Marta Ruiz-Llata,et al.  FPGA Implementation of Support Vector Machines for 3D Object Identification , 2009, ICANN.

[21]  Shigeru Akamatsu,et al.  Invariant face detection with support vector machines , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[22]  Davide Anguita,et al.  A digital architecture for support vector machines: theory, algorithm, and FPGA implementation , 2003, IEEE Trans. Neural Networks.

[23]  Thorsten Joachims,et al.  Transductive Inference for Text Classification using Support Vector Machines , 1999, ICML.

[24]  Umaa Rebbapragada,et al.  Onboard detection of natural sulfur on a glacier via a SVM and Hyperion data , 2009, 2009 IEEE Aerospace conference.

[25]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[27]  Austin Carpenter,et al.  CUSVM: A CUDA IMPLEMENTATION OF SUPPORT VECTOR CLASSIFICATION AND REGRESSION , 2009 .

[28]  Srihari Cadambi,et al.  A Massively Parallel FPGA-Based Coprocessor for Support Vector Machines , 2009, 2009 17th IEEE Symposium on Field Programmable Custom Computing Machines.

[29]  Hyeran Byun,et al.  Applications of Support Vector Machines for Pattern Recognition: A Survey , 2002, SVM.

[30]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[31]  J. Mercer.  Functions of Positive and Negative Type, and their Connection with the Theory of Integral Equations , 1909 .