A compact 3D VLSI classifier using bagging threshold network ensembles

A bagging ensemble consists of a set of classifiers trained independently and combined by a majority vote. Such a combination improves generalization performance but can require large amounts of memory and computation, a serious drawback for addressing portable real-time pattern recognition applications. We report here a compact three-dimensional (3D) multiprecision very large-scale integration (VLSI) implementation of a bagging ensemble. In our circuit, individual classifiers are decision trees implemented as threshold networks - one layer of threshold logic units (TLUs) followed by combinatorial logic functions. The hardware was fabricated using 0.7-/spl mu/m CMOS technology and packaged using MCM-V micro-packaging technology. The 3D chip implements up to 192 TLUs operating at a speed of up to 48 GCPPS and implemented in a volume of (/spl omega/ /spl times/ L /spl times/ h) = (2 /spl times/ 2 /spl times/ 0.7) cm/sup 3/. The 3D circuit features a high level of programmability and flexibility offering the possibility to make an efficient use of the hardware resources in order to reduce the power consumption. Successful operation of the 3D chip for various precisions and ensemble sizes is demonstrated through an electronic nose application.

[1]  David W. Opitz,et al.  An Empirical Evaluation of Bagging and Boosting , 1997, AAAI/IAAI.

[2]  Simon Kasif,et al.  A System for Induction of Oblique Decision Trees , 1994, J. Artif. Intell. Res..

[3]  Michael Biehl,et al.  Construction algorithm for the parity-machine , 1993 .

[4]  Marc Duranton,et al.  Image processing by neural networks , 1996, IEEE Micro.

[5]  Klaus Schumacher,et al.  VLSI technologies for artificial neural networks , 1989, IEEE Micro.

[6]  D. Martinez,et al.  The Offset Algorithm: Building and Learning Method for Multilayer Neural Networks , 1992 .

[7]  Marc Duranton,et al.  Lneuro 1.0: a piece of hardware LEGO for building neural network systems , 1992, IEEE Trans. Neural Networks.

[8]  J. Beichter,et al.  Architecture of a general-purpose neural signal processor , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[9]  N. L. Seed,et al.  Electronic systems in dense three-dimensional packages , 1995 .

[10]  Yannis Tsividis,et al.  A reconfigurable VLSI neural network , 1992 .

[11]  Bernabé Linares-Barranco,et al.  A real-time clustering microchip neural engine , 1996, IEEE Trans. Very Large Scale Integr. Syst..

[12]  Nirmal K. Bose,et al.  Neural network design using Voronoi diagrams , 1993, IEEE Trans. Neural Networks.

[13]  Paolo Ienne,et al.  GENES IV: A bit-serial processing element for a built-model neural-network accelerator , 1993, Proceedings of International Conference on Application Specific Array Processors (ASAP '93).

[14]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[15]  Raúl Rojas,et al.  Hardware for Neural Networks , 1996 .

[16]  Gérard Dreyfus,et al.  Handwritten digit recognition by neural networks with single-layer training , 1992, IEEE Trans. Neural Networks.

[17]  Ishwar K. Sethi.  Neural implementation of tree classifiers , 1995, IEEE Trans. Syst. Man Cybern..

[18]  E. van Keulen,et al.  Neural network hardware performance criteria , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[19]  E. Fiesler,et al.  Hardware-friendly learning algorithms for neural networks: an overview , 1996, Proceedings of Fifth International Conference on Microelectronics for Neural Networks.

[20]  Thomas G. Dietterich.  Machine-Learning Research Four Current Directions , 1997 .

[21]  Said F. Al-Sarawi,et al.  A Review of 3-D Packaging Technology , 1998 .

[22]  Rainer Stotzka,et al.  Neural chip SAND/1 for real time pattern recognition , 1998 .

[23]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[24]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[25]  Ishwar K. Sethi,et al.  Entropy nets: from decision trees to neural networks , 1990, Proc. IEEE.

[26]  Donald Goldfarb,et al.  A numerically stable dual method for solving strictly convex quadratic programs , 1983, Math. Program..

[27]  Amine Bermak,et al.  High-density 16/8/4-bit configurable multiplier , 1997 .

[28]  Jan N. H. Heemskerk.  Overview of neural hardware , 1995 .

[29]  D. Opitz,et al.  Popular Ensemble Methods: An Empirical Study , 1999, J. Artif. Intell. Res..