An analog on-line-learning K-means processor employing fully parallel self-converging circuitry

A hardware-efficient on-line-learnable processor was developed for the K-means clustering of highly dimensional vectors. Based on our proposed sample updating strategy, an incremental number of sample vectors can be clustered by a constant set of VLSI circuits. In order to speed up the learning process, we developed an analog fully parallel self-converging circuitry to implement the K-means algorithm. Upon receiving a sample vector on-line, the K-means learning autonomously proceeds and converges within a single system clock cycle (0.1 μs at 10 MHz). Furthermore, the chip-area and inner connection explosion problem was solved by using the proposed architecture. A proof-of-concept processor was designed and verified by the HSPICE and Nanosim simulations. The images from an actual database were used as learning samples in the form of 64 dimensional feature vectors. From the simulation results, all the samples were clustered into correct categories with a randomly ill initialization. In addition, the number of sample vectors can be freely increased.

[1]  Shao-Yi Chien,et al.  Flexible Hardware Architecture of Hierarchical K-Means Clustering for Large Cluster Number , 2011, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[2]  Andreas Kolb,et al.  GPU-Based Multilevel Clustering , 2011, IEEE Transactions on Visualization and Computer Graphics.

[3]  Hiroshi Murase,et al.  Power-Efficient Hardware Architecture of K-Means Clustering With Bayesian-Information-Criterion Processor for Multimedia Processing Applications , 2011, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[4]  Tadashi Shibata,et al.  An On-Chip-Trainable Gaussian-Kernel Analog , 2010 .

[5]  Yitao Ma,et al.  A Binary-Tree Hierarchical Multiple-Chip Architecture for Real-Time Large-Scale Learning Processor Systems , 2010 .

[6]  Tsutomu Maruyama Real-time K-Means Clustering for Color Images on Reconfigurable Hardware , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[7]  Tadashi Shibata,et al.  An On-Chip-Trainable Gaussian-Kernel Analog Support Vector Machine , 2010, IEEE Transactions on Circuits and Systems I: Regular Papers.

[8]  J. MacQueen Some methods for classification and analysis of multivariate observations , 1967 .

[9]  Shiri Gordon,et al.  Applying the information bottleneck principle to unsupervised clustering of discrete and continuous image representations , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[10]  Shao-Yi Chien,et al.  Architectural analyses of K-Means silicon intellectual property for image segmentation , 2008, 2008 IEEE International Symposium on Circuits and Systems.

[11]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[12]  Bradley A. Minch,et al.  MOS Translinear Principle for All Inversion Levels , 2008, IEEE Transactions on Circuits and Systems II: Express Briefs.

[13]  Shao-Yi Chien,et al.  Bandwidth Adaptive Hardware Architecture of K-Means Clustering for Video Analysis , 2010, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[14]  Siddheswar Ray,et al.  Determination of Number of Clusters in K-Means Clustering and Application in Colour Image Segmentation , 2000 .

[15]  Huseyin Seker,et al.  FPGA implementation of K-means algorithm for bioinformatics application: An accelerated approach to clustering Microarray data , 2011, 2011 NASA/ESA Conference on Adaptive Hardware and Systems (AHS).

[16]  Jean-Jacques Quisquater,et al.  Integer Factorization Based on Elliptic Curve Method: Towards Better Exploitation of Reconfigurable Hardware , 2007 .

[17]  Orly Yadid-Pecht,et al.  Hardware-driven adaptive k-means clustering for real-time video imaging , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[18]  Miriam Leeser,et al.  K-means Clustering for Multispectral Images Using Floating-Point Divide , 2007 .

[19]  Tadashi Shibata,et al.  An image representation algorithm compatible with neural-associative-processor-based hardware recognition systems , 2003, IEEE Trans. Neural Networks.

[20]  Chang-Dong Wang,et al.  A Conscience On-line Learning Approach for Kernel-Based Clustering , 2010, 2010 IEEE International Conference on Data Mining.

[21]  Tadashi Shibata,et al.  Fully Parallel Self-Learning Analog Support Vector Machine Employing Compact Gaussian Generation Circuits , 2012 .

[22]  Willy Sansen,et al.  A simple characterization method for MOS transistor matching in deep submicron technologies , 2001, ICMTS 2001. Proceedings of the 2001 International Conference on Microelectronic Test Structures (Cat. No.01CH37153).