A Learning Machine with a Bit-Based Hypothesis Space

We propose in this paper a bit-based classifier, picked from an hypothesis space described accordingly to sparsity and locality princi- ples: the complexity of the corresponding space of functions is controlled through the number of bits needed to represent it, so that it will include the classifiers that will be most likely chosen by the learning procedure. Through an introductory example, we show how the number of bits, the sparsity of the representation and the local definition approach affect the complexity of the space of functions, where the final classifier is selected from.

[1]  Davide Anguita,et al.  Nature Inspiration for Support Vector Machines , 2006, KES.

[2]  Hartmut Neven,et al.  Training a Large Scale Classifier with the Quantum Adiabatic Algorithm , 2009, ArXiv.

[3]  Davide Anguita,et al.  Selecting the hypothesis space for improving the generalization ability of Support Vector Machines , 2011, The 2011 International Joint Conference on Neural Networks.

[4]  Davide Anguita,et al.  Maximal Discrepancy for Support Vector Machines , 2011, ESANN.

[5]  Narayanan Vijaykrishnan,et al.  A Hardware Efficient Support Vector Machine Architecture for FPGA , 2008, 2008 16th International Symposium on Field-Programmable Custom Computing Machines.

[6]  V. Koltchinskii.  Local Rademacher complexities and oracle inequalities in risk minimization , 2006, 0708.0083.

[7]  Peter L. Bartlett,et al.  Model Selection and Error Estimation , 2000, Machine Learning.

[8]  Peter L. Bartlett,et al.  Rademacher and Gaussian Complexities: Risk Bounds and Structural Results , 2003, J. Mach. Learn. Res..

[9]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[10]  Carlo Vercellis,et al.  Multicategory classification via discrete support vector machines , 2009, Comput. Manag. Sci..

[11]  Manfred Mücke,et al.  Effects of Reduced Precision on Floating-Point SVM Classification Accuracy , 2011, International Conference on Conceptual Structures.

[12]  Sang-Woong Lee,et al.  Real-Time Implementation of Face Recognition Algorithms on DSP Chip , 2003, AVBPA.

[13]  E. L. Lawler,et al.  Branch-and-Bound Methods: A Survey , 1966, Oper. Res..

[14]  Michael G. Epitropakis,et al.  Hardware-friendly Higher-Order Neural Network Training using Distributed Evolutionary Algorithms , 2010, Appl. Soft Comput..

[15]  Gert Cauwenberghs,et al.  Kerneltron: support vector "machine" in silicon , 2003, IEEE Trans. Neural Networks.

[16]  Josef Kittler,et al.  Audio- and Video-Based Biometric Person Authentication, 5th International Conference, AVBPA 2005, Hilton Rye Town, NY, USA, July 20-22, 2005, Proceedings , 2005, AVBPA.

[17]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[18]  Davide Anguita,et al.  A support vector machine with integer parameters , 2008, Neurocomputing.

[19]  Suresh B. Srinivasamurthy.  Methods of Solving Ill-Posed Problems , 2012, 1205.5323.