Reliable recognition of handwritten digits using a cascade ensemble classifier system and hybrid features

Aiming at a high recognition rate and a low error rate at the same time, a cascade ensemble classifier system is proposed for the recognition of handwritten digits. The tradeoff among the error, rejection and recognition rates of the recognition system is analyzed theoretically. Three solutions are proposed: (i) extracting more discriminative features to attain a high recognition rate, (ii) using ensemble classifiers to suppress the error rate, and (iii) employing a novel cascade system to enhance the recognition rate and to reduce the rejection rate. Based on these strategies, seven sets of discriminative hybrid features and three sets of randomly selected features are extracted and used in the different layers of the cascade recognition system. Novel gating networks are used to congregate the confidence values of three parallel Artificial Neural Networks (ANNs) classifiers. The weights of the gating networks are trained by the Genetic Algorithms (GAs) to achieve the overall optimal performance. Experiments are conducted on the MNIST handwritten numeral database with encouraging results: a high reliability of 99.96% with a minimal rejection, or 99.59% correct recognition rate without rejection in the last cascade layer. In the verification model, a novel multi-modal nonparametric analysis for optimal feature dimensionality reduction is proposed. The computational complexity of our proposed algorithm is much lower than that of other similar approaches found in the literature. Experiments demonstrate that our proposed method can achieve a high feature compression performance without sacrificing its discriminant ability. The results of dimensionality reduction make the ANNs converge more easily. For the verification of confusing handwritten numeral pairs, our proposed algorithm is used to congregate features, and it outperforms the PCA and compares favorably with other nonparametric discriminant analysis methods.