Cluster-Based Prototype Learning System for Multiple Applications with Flexible HW/SW Codesign

This paper proposes a novel hybrid hardware-software (HW/SW) system for K-means-based prototype learning and Nearest-Neighbor (1-NN) classification. We implement a prototype learning system instead of simplifying complex learning algorithms (e.g. neural and fuzzy networks, or SVMs) because this facilitates the adaptability to hardware capabilities and constraints. The K-means algorithm, which is implemented by HW/SW co-design, is effective in improving classification performance and reducing storage requirements. Particularly, the hardware realization is applied to obtain orders of magnitude higher speed for nearest-distance searching, which is the most burdensome performance barrier both in K-means learning and 1-NN classification. We benchmark our multi-purpose learning system against the application of handwritten digit recognition and face recognition to demonstrate its excellent performance, namely high flexibility, fast training, short recognition time and good recognition rate.

[1]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[2]  Harris Drucker,et al.  Comparison of learning algorithms for handwritten digit recognition , 1995 .

[3]  Masaki Nakagawa,et al.  Evaluation of prototype learning algorithms for nearest-neighbor classifier in application to handwritten character recognition , 2001, Pattern Recognit..

[4]  Robert M. Gray,et al.  An Algorithm for Vector Quantizer Design , 1980, IEEE Trans. Commun..

[5]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[6]  Meng Joo Er,et al.  High-speed face recognition based on discrete cosine transform and RBF neural networks , 2005, IEEE Transactions on Neural Networks.

[7]  Gerhard Rigoll,et al.  High quality face recognition in JPEG compressed images , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[8]  Cheng-Lin Liu,et al.  Handwritten digit recognition: benchmarking of state-of-the-art techniques , 2003, Pattern Recognit..

[9]  Dominique Lavenier,et al.  Experience with a Hybrid Processor: K-Means Clustering , 2004, The Journal of Supercomputing.

[10]  Maya Gokhale,et al.  Applying reconfigurable hardware to the analysis of multispectral and hyperspectral imagery , 2002, SPIE Optics + Photonics.

[12]  Uday B. Desai,et al.  Face recognition using a DCT-HMM approach , 1998, Proceedings Fourth IEEE Workshop on Applications of Computer Vision. WACV'98 (Cat. No.98EX201).

[13]  Hans Jurgen Mattausch,et al.  Mixed Digital–Analog Associative Memory Enabling Fully-Parallel Nearest Euclidean Distance Search , 2007 .

[14]  Alistair G. Rust,et al.  Image redundancy reduction for neural network classification using discrete cosine transforms , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.