An FPGA distributed implementation model for embedded SOM with on-line learning

Neuroengineering has contributed to the increasing capability of embedded hardware to efficiently execute neural computation models. These hardware structures, suitable for implementing only the recall phase or both phases — learning and recall — of artificial neural networks (ANNs) are generally identified as neuromorphic systems. Some features of the FPGA technology resemble characteristics usually associated to ANNs, such as the parallel processing and the configurability of digital circuits. Embedded neural systems elaborated to execute Self-Organizing Maps (SOMs) can take advantage of the circuit parallelism to speed-up training process and also of the design flexibility by the hardware reconfiguration enabled by FPGA chips. Hardware-based SOMs capable of executing the training process on chip are relevant because they can be employed in a wide variety of stand-alone applications. loT and mobile robotics are examples of some challenging research fields that use unsupervised learning to extract information from the surrounding environment and require the portability of embedded systems. This work presents a distributed architecture to implement SOM learning and recall algorithms into FPGAs. The distributed model does not employ any type of central control unit hence allowing a simple hardware redesign process during the configuration of the neural system. The set of experiments used for verifying the proposed architecture is specified in a data stream format. The aim of choosing streaming condition is to simulate applications in which the FPGA-based SOM must process a continuous flow of information autonomously. All the steps of the SOM learning algorithm to process an input sample are performed in a single clock cycle and implementation results show a maximum clock speed of 2.38 MHz. Synthesis reports on FPGA implementation of the distributed architecture for the experiments carried out herein also indicate that the factor of increase in consumption of chip resources is lower than the increment in the total number of SOM connections.

[1]  Teuvo Kohonen,et al.  Essentials of the self-organizing map , 2013, Neural Networks.

[2]  Bipin Rajendran,et al.  Mimicking the worm — An adaptive spiking neural circuit for contour tracking inspired by C. Elegans thermotaxis , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[3]  Leandro dos Santos Coelho,et al.  Hardware opposition-based PSO applied to mobile robot controllers , 2014, Eng. Appl. Artif. Intell..

[4]  Leibo Liu,et al.  Neural approximating architecture targeting multiple application domains , 2015, 2015 IEEE International Symposium on Circuits and Systems (ISCAS).

[5]  David Harle,et al.  Data Communications and Networks: An Engineering Approach , 2000 .

[6]  Emilio Del-Moral-Hernandez,et al.  Architecture Analysis of an FPGA-Based Hopfield Neural Network , 2014, Adv. Artif. Neural Syst..

[7]  Karl Goser,et al.  Hardware Design for Self-Organizing Feature Maps with Binary Input Vectors , 1993, IWANN.

[8]  Gert Cauwenberghs,et al.  Neuromorphic Silicon Neuron Circuits , 2011, Front. Neurosci.

[9]  Andrew Hunter,et al.  A binary Self-Organizing Map and its FPGA implementation , 2009, 2009 International Joint Conference on Neural Networks.

[10]  Jorge Peña,et al.  Digital Hardware Architectures of Kohonen's Self Organizing Feature Maps with Exponential Neighboring Function , 2006, 2006 IEEE International Conference on Reconfigurable Computing and FPGA's (ReConFig 2006).

[11]  Jose Hugo Barron-Zambrano,et al.  FPGA implementation of a configurable neuromorphic CPG-based locomotion controller , 2013, Neural Networks.

[12]  Rafael Gadea Gironés,et al.  A hardware design of a massive-parallel, modular NN-based vector quantizer for real-time video coding , 2008, Microprocess. Microsystems.

[13]  Hiroomi Hikawa,et al.  Color-space image compression with hardware Self-organizing map , 2013, 2013 International Symposium on Intelligent Signal Processing and Communication Systems.

[14]  Yiping Dong,et al.  A hybrid architecture for efficient FPGA-based implementation of multilayer neural network , 2010, 2010 IEEE Asia Pacific Conference on Circuits and Systems.

[15]  Simon Haykin,et al.  Neural Networks and Learning Machines , 2010 .

[16]  Athanasios Anastasiou,et al.  A New Method for Profile Generation in an Internet of Things Environment: An Application in Ambient-Assisted Living , 2015, IEEE Internet of Things Journal.

[17]  Pravin Dakhole,et al.  Parameterless self organizing map for improving OFDM system performance , 2011, 2011 6th International Conference on Industrial and Information Systems.

[18]  Akira Hirose,et al.  Predictive self-organizing map for vector quantization of migratory signals and its application to mobile communications , 2003, IEEE Trans. Neural Networks.

[19]  Bertrand Granado,et al.  Toward a Sparse Self-Organizing Map for Neuromorphic Architectures , 2015, ACM J. Emerg. Technol. Comput. Syst..

[20]  W. Kurdthongmee,et al.  A novel hardware-oriented Kohonen SOM image compression algorithm and its FPGA implementation , 2008, J. Syst. Archit..

[21]  Teuvo Kohonen,et al.  Self-organized formation of topologically correct feature maps , 2004, Biological Cybernetics.

[22]  Soo-Chang Pei,et al.  Color image compression and limited display using self-organization Kohonen map , 1998, IEEE Trans. Circuits Syst. Video Technol..

[23]  Henry Chen,et al.  A 75µW, 16-channel neural spike-sorting processor with unsupervised clustering , 2011, 2011 Symposium on VLSI Circuits - Digest of Technical Papers.

[24]  Volnei A. Pedroni.  Circuit Design and Simulation with VHDL , 2010 .

[25]  Erzsébet Merényi,et al.  A reconfigurable neuroprocessor for self-organizing feature maps , 2013, Neurocomputing.

[26]  Tsung-Han Tsai,et al.  Design for an intelligent surveillance system based on system-on-a-programmable-chip platform , 2015, 2015 IEEE International Symposium on Circuits and Systems (ISCAS).

[27]  S. T. Brassai.  FPGA based hardware implementation of a self-organizing map , 2014, IEEE 18th International Conference on Intelligent Engineering Systems INES 2014.

[28]  Amro Najjar,et al.  Constructivist ambient intelligent agent for smart environments , 2013, 2013 IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOM Workshops).

[29]  Jihan Zhu,et al.  FPGA Implementations of Neural Networks - A Survey of a Decade of Progress , 2003, FPL.

[30]  Haidi Ibrahim,et al.  Recent survey on crowd density estimation and counting for visual surveillance , 2015, Eng. Appl. Artif. Intell..

[31]  Hiroomi Hikawa,et al.  Novel FPGA Implementation of Hand Sign Recognition System With SOM–Hebb Classifier , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[32]  Moncef Gabbouj,et al.  On the design of Hopfield Neural Networks: Synthesis of hopfield type associative memories , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[33]  David Dye.  Partial Reconfiguration of Xilinx FPGAs Using ISE Design Suite , 2010 .

[34]  Peter J. Ashenden,et al.  The Designer's Guide to VHDL , 1995 .

[35]  Hiroomi Hikawa,et al.  Supervised learning of DPLL based winner-take-all neural network , 2014, 2014 IEEE International Conference on Evolvable Systems.

[36]  Charu C. Aggarwal,et al.  On the Surprising Behavior of Distance Metrics in High Dimensional Spaces , 2001, ICDT.

[37]  Pong P. Chu.  FPGA Prototyping by Verilog Examples: Xilinx Spartan-3 Version , 2008 .

[38]  Yutaka Maeda,et al.  Improved Learning Performance of Hardware Self-Organizing Map Using a Novel Neighborhood Function , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[39]  Steve B. Furber,et al.  Neural Systems Engineering , 2008, Computational Intelligence: A Compendium.

[40]  Jr. L.R. Litwin.  Blind channel equalization , 1999 .

[41]  W. Kurdthongmee,et al.  Utilization of a rational-based representation to improve the image quality of a hardware-based K-SOM quantizer , 2011, Journal of Real-Time Image Processing.