Stuctured Predictions Cascades

Structured prediction tasks pose a fundamental trade off between the need for model complexity to increase predictive power and the limited computational resources for inference in the exponentially-sized output spaces such models require. We formulate and develop structured prediction cascades: a sequence of increasingly complex models that progressively filter the space of possible outputs. We represent an exponentially large set of filtered outputs using max marginals and propose a novel convex loss function that balances filtering error with filtering efficiency. We provide generalization bounds for these loss functions and evaluate our approach on handwriting recognition and part-of-speech tagging. We find that the learned cascades are capable of reducing the complexity of inference by up to five orders of magnitude, enabling the use of models which incorporate higher order features and yield higher accuracy. Disciplines Computer Sciences Comments Structured Prediction Cascades, D. Weiss and B. Taskar. International Conference on Artificial Intelligence and Statistics (AISTATS), Sardinia, Italy, May 2010. Copyright is held by the authors. This conference paper is available at ScholarlyCommons: http://repository.upenn.edu/cis_papers/546 Structured Prediction Cascades David Weiss and Ben Taskar University of Pennsylvania