Efficient Maximum Entropy Training for Statistical Object Recognition

In statistical pattern recognition, we use probabilistic models within the task of assigning observations to one of a set of predefined classes, like e.g. images of handwritten digits to one of the classes ‘0’ to ‘9’. The principle of maximum entropy is a powerful framework that can be used to estimate class posterior probabilities for pattern recognition tasks. It is a conceptually simple and easily extensible model that allows to estimate a large number of free parameters reliably. We show how to apply this framework to object recognition and compare the results to other state-of-the-art approaches in experiments with the well known US Postal Service handwritten digits recognition task. We also introduce a simple but effective heuristic method for speeding up the algorithms used to determine the model parameters.