NEROvideo: a general-purpose CNN-UM video processing system

Emulations of cellular nonlinear networks (CNN) on digital reconfigurable hardware have proved to be adequate for highly-efficient computation of massive data, exceeding the accuracy and flexibility of full-custom designs. Based on a recently-proposed architecture for the emulation of a large-scale CNN universal machine, a new real-time video processing system has been developed. Due to its free programmability and massively-parallel architecture the system is very suitable for high-speed computation of complex algorithms that follow the idea of spatio-temporal computing. Implemented on a state-of-the-art Xilinx Zynq system-on-chip, the proposed setup is capable of processing a $$640\times 480$$640×480p video stream with up to 1,700 fps, depending on the respective algorithm.

[1]  Piotr Dudek,et al.  A general-purpose processor-per-pixel analog SIMD vision chip , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[2]  José Manuel Ferrández,et al.  Implementation of a CNN-based retinomorphic model on a high performance reconfigurable computer , 2011, Neurocomputing.

[3]  P. Strohm,et al.  On the potential of current CNN cameras for industrial surface inspection , 2012, 2012 13th International Workshop on Cellular Nanoscale Networks and their Applications.

[4]  Ronald Tetzlaff,et al.  A novel spatter detection algorithm based on typical cellular neural network operations for laser beam welding processes , 2011 .

[5]  Leon O. Chua,et al.  Autonomous cellular neural networks: a unified paradigm for pattern formation and active wave propagation , 1995 .

[6]  Leon O. Chua,et al.  Analogic CNN algorithms for some image compression and restoration tasks , 1995 .

[7]  Péter Szolgay,et al.  Implementation of embedded emulated-digital CNN-UM global analogic programming unit on FPGA and its application , 2008, Int. J. Circuit Theory Appl..

[8]  Edmund Koch,et al.  Intraoperative imaging of cortical cerebral perfusion by time-resolved thermography and multivariate data analysis. , 2011, Journal of biomedical optics.

[9]  David Kirk,et al.  NVIDIA cuda software and gpu parallel computing architecture , 2007, ISMM '07.

[10]  R. Tetzlaff,et al.  CESAR: Emulating Cellular Networks on FPGA , 2012, 2012 13th International Workshop on Cellular Nanoscale Networks and their Applications.

[11]  Péter Szolgay,et al.  Configurable multilayer CNN-UM emulator on FPGA , 2003 .

[12]  Piotr Dudek,et al.  A general-purpose vision processor with 160×80 pixel-parallel SIMD processor array , 2013, Proceedings of the IEEE 2013 Custom Integrated Circuits Conference.

[13]  Tamás Roska,et al.  The CNN universal machine: an analogic array computer , 1993 .

[14]  V. Tavsanoglu,et al.  Demonstration of the Second Generation Real-Time Cellular Neural Network Processor: RTCNNP-v2 , 2012, 2012 13th International Workshop on Cellular Nanoscale Networks and their Applications.

[15]  R.P. Kleihorst,et al.  Xetal-II: A 107 GOPS, 600 mW Massively Parallel Processor for Video Scene Analysis , 2008, IEEE Journal of Solid-State Circuits.

[16]  Liang-Gee Chen,et al.  iVisual: An Intelligent Visual Sensor SoC With 2790 fps CMOS Image Sensor and 205 GOPS/W Vision Processor , 2009, IEEE J. Solid State Circuits.

[17]  V.M. Brea,et al.  SIMD array on FPGA for B/W image processing , 2008, 2008 11th International Workshop on Cellular Neural Networks and Their Applications.

[18]  Ángel Rodríguez-Vázquez,et al.  ACE4k: An analog I/O 64×64 visual microprocessor chip with 7-bit analog accuracy: Research Articles , 2002 .

[19]  Ronald Tetzlaff,et al.  NERO mastering 300k CNN cells , 2013, 2013 European Conference on Circuit Theory and Design (ECCTD).

[20]  Ángel Rodríguez-Vázquez,et al.  The Eye-RIS CMOS Vision System , 2008 .

[21]  Gonzalo R. Arce,et al.  Nonlinear Signal Processing - A Statistical Approach , 2004 .

[22]  Ronald Tetzlaff,et al.  A new cellular nonlinear network emulation on FPGA for EEG signal processing in epilepsy , 2011, Microtechnologies.

[23]  Hussein Baher,et al.  Analog & digital signal processing , 1990 .

[24]  H. Peter Hofstee,et al.  Introduction to the Cell multiprocessor , 2005, IBM J. Res. Dev..

[25]  Ángel Rodríguez-Vázquez,et al.  ACE4k: An analog I/O 64×64 visual microprocessor chip with 7-bit analog accuracy , 2002, Int. J. Circuit Theory Appl..

[26]  Ákos Zarándy,et al.  Focal-Plane Sensor-Processor Chips , 2014 .

[27]  Piotr Dudek.  An asynchronous cellular logic network for trigger-wave image processing on fine-grain massively parallel arrays , 2006, IEEE Transactions on Circuits and Systems II: Express Briefs.

[28]  Chih-Chi Cheng,et al.  iVisual: An Intelligent Visual Sensor SoC With 2790 fps CMOS Image Sensor and 205 GOPS/W Vision Processor , 2009, IEEE Journal of Solid-State Circuits.

[29]  V. Tavsanoglu,et al.  A new approach to emulate CNN on FPGAs for real time video processing , 2008, 2008 11th International Workshop on Cellular Neural Networks and Their Applications.

[30]  Ákos Zarándy,et al.  Configurable 3D-integrated focal-plane cellular sensor–processor array architecture , 2008 .

[31]  Michiel Steyaert,et al.  Analog Circuit Design , 2005, Springer US.

[32]  Leon O. Chua,et al.  The CNN is universal as the Turing machine , 1993 .

[33]  Ángel Rodríguez-Vázquez,et al.  ACE16k: the third generation of mixed-signal SIMD-CNN ACE chips toward VSoCs , 2004, IEEE Transactions on Circuits and Systems I: Regular Papers.

[34]  Ákos Zarándy,et al.  2D operators on topographic and non‐topographic architectures—implementation, efficiency analysis, and architecture selection methodology , 2011, Int. J. Circuit Theory Appl..

[35]  Lin-Bao Yang,et al.  Cellular neural networks: theory , 1988 .