Trace Ratio vs. Ratio Trace for Dimensionality Reduction

A large family of algorithms for dimensionality reduction end with solving a Trace Ratio problem in the form of arg max_W Tr(W^T S_PW)/Tr(WT S_IW)¹, which is generally transformed into the corresponding Ratio Trace form arg max_W Tr[ (W^TS_IW)^-1 (WTS_PW) ] for obtaining a closed-form but inexact solution. In this work, an efficient iterative procedure is presented to directly solve the Trace Ratio problem. In each step, a Trace Difference problem arg max_W Tr [W^T (S_P - lambdaS_I) W] is solved with lambda being the trace ratio value computed from the previous step. Convergence of the projection matrix W, as well as the global optimum of the trace ratio value lambda, are proven based on point-to-set map theories. In addition, this procedure is further extended for solving trace ratio problems with more general constraint W^TCW=I and providing exact solutions for kernel-based subspace learning problems. Extensive experiments on faces and UCI data demonstrate the high convergence speed of the proposed solution, as well as its superiority in classification capability over corresponding solutions to the ratio trace problem.