Global Optimization Methods for Extended Fisher Discriminant Analysis

The Fisher discriminant analysis (FDA) is a common technique for binary classication. A parametrized extension, which we call the extended FDA, has been introduced from the viewpoint of robust optimization. In this work, werst give a new probabilistic inter- pretation of the extended FDA. We then de- velop algorithms for solving an optimization problem that arises from the extended FDA: computing the distance between a point and the surface of an ellipsoid. We solve this problem via the KKT points, which we show are obtained by solving a generalized eigen- value problem. We speed up the algorithm by taking advantage of the matrix structure and proving that a globally optimal solution is a KKT point with the smallest Lagrange multiplier, which can be computed efficiently as the leftmost eigenvalue. Numerical exper- iments illustrate the efficiency and effective- ness of the extended FDA model combined with our algorithm.