Support vector machines and the multiple hypothesis test problem

Two enhancements are proposed to the application and theory of support vector machines. The first is a method of multicategory classification based on the binary classification version of the support vector machine (SVM). The method, which is called the M-ary SVM, represents each category in binary format, and to each bit of that representation is assigned a conventional SVM. This approach requires only [log/sub 2/(K)] SVMs, where K is the number of classes. We give an example of classification on an octaphase-shift-keying (8-PSK) pattern space to illustrate the main concepts. The second enhancement is that of adding equality constraints to the conventional binary classification SVM. This allows pinning the classification boundary to points that are known a priori to lie on the boundary. Applications of this method often arise in problems having some type of symmetry, We present one such example where the M-ary SVM is used to classify symbols of a CDMA two-user, multiuser detection pattern space.

[1]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[2]  O. Mangasarian.  Linear and Nonlinear Separation of Patterns by Linear Programming , 1965 .

[3]  FRED W. SMITH,et al.  Pattern Classifier Design by Linear Programming , 1968, IEEE Transactions on Computers.

[4]  Olvi L. Mangasarian,et al.  Multisurface method of pattern separation , 1968, IEEE Trans. Inf. Theory.

[5]  D. Luenberger.  Optimization by Vector Space Methods , 1968 .

[6]  V. E. Vickers,et al.  A Technique for Generating Specialized Gray Codes , 1980, IEEE Transactions on Computers.

[7]  J. Ludman,et al.  Gray Code Generation for MPSK Signals , 1981, IEEE Trans. Commun..

[8]  John P. Robinson,et al.  Counting sequences , 1981, IEEE Transactions on Computers.

[9]  J. E. Ludman,et al.  A technique for generating Gray codes , 1981 .

[10]  Henry Stark,et al.  Probability, Random Processes, and Estimation Theory for Engineers , 1995 .

[11]  G. Lieberman,et al.  Introduction to Mathematical Programming , 1990 .

[12]  O. Mangasarian,et al.  Robust linear programming discrimination of two linearly inseparable sets , 1992 .

[13]  Kristin P. Bennett,et al.  Serial and Parallel Multicategory Discrimination , 1994, SIAM J. Optim..

[14]  O. Mangasarian,et al.  Multicategory discrimination via linear programming , 1994 .

[15]  László Györfi,et al.  A Probabilistic Theory of Pattern Recognition , 1996, Stochastic Modelling and Applied Probability.

[16]  Carla D. Savage,et al.  Balanced Gray Codes , 1996, Electron. J. Comb..

[17]  Carla Savage,et al.  A Survey of Combinatorial Gray Codes , 1997, SIAM Rev..

[18]  Paul S. Bradley,et al.  Feature Selection via Concave Minimization and Support Vector Machines , 1998, ICML.

[19]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[20]  Jason Weston,et al.  Multi-Class Support Vector Machines , 1998 .

[21]  Kristin P. Bennett,et al.  Multicategory Classification by Support Vector Machines , 1999, Comput. Optim. Appl..

[22]  Kristin P. Bennett,et al.  Combining support vector and mathematical programming methods for classification , 1999 .

[23]  Ulrich H.-G. Kreßel,et al.  Pairwise classification and support vector machines , 1999 .

[24]  Linda Kaufman,et al.  Solving the quadratic programming problem arising in support vector classification , 1999 .

[25]  Michael C. Ferris,et al.  Interfaces to PATH 3.0: Design, Implementation and Usage , 1999, Comput. Optim. Appl..

[26]  James A. Bucklew,et al.  Support vector machine techniques for nonlinear equalization , 2000, IEEE Trans. Signal Process..

[27]  David R. Musicant,et al.  Data Discrimination via Nonlinear Generalized Support Vector Machines , 2001 .