Emulating Mammalian Vision on Reconfigurable Hardware

A significant challenge in creating machines with artificial vision is designing systems which can process visual information as efficiently as the brain. To address this challenge, we identify key algorithms which model the process of attention and recognition in the visual cortex of mammals. This paper presents Cover - an FPGA framework for generating systems which can potentially emulate the visual cortex. We have designed accelerators for models of attention and recognition in the cortex and integrated them to realize an end-to-end attention-recognition system. Evaluation of our system on a Dinigroup multi-FPGA platform shows high performance and accuracy for attention and recognition systems and speedups over existing CPU, GPU and FPGA implementations. Results show that our end-to-end system which emulates the cortex can achieve near real-time speeds for high resolution images. This system can be applied to many artificial vision applications such as augmented virtual reality and autonomous vehicle navigation.

[1]  Narayanan Vijaykrishnan,et al.  An algorithm-architecture co-design framework for gridding reconstruction using FPGAs , 2011, 2011 48th ACM/EDAC/IEEE Design Automation Conference (DAC).

[2]  David G. Lowe,et al.  Multiclass Object Recognition with Sparse, Localized Features , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[3]  Narayanan Vijaykrishnan,et al.  An FPGA Implementation of Information Theoretic Visual-Saliency System and Its Optimization , 2011, 2011 IEEE 19th Annual International Symposium on Field-Programmable Custom Computing Machines.

[4]  Berin Martini,et al.  Hardware accelerated convolutional neural networks for synthetic vision systems , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[5]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[6]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[7]  Janarbek Matai,et al.  Design and Implementation of an FPGA-Based Real-Time Face Recognition System , 2011, 2011 IEEE 19th Annual International Symposium on Field-Programmable Custom Computing Machines.

[8]  Laurent Itti,et al.  Neuromorphic algorithms for computer vision and attention , 2001, SPIE Optics + Photonics.

[9]  Tomaso Poggio,et al.  CNS: a GPU-based framework for simulating cortically-organized networks , 2010 .

[10]  Narayanan Vijaykrishnan,et al.  A reconfigurable accelerator for neuromorphic object recognition , 2012, 17th Asia and South Pacific Design Automation Conference.

[11]  Narayanan Vijaykrishnan,et al.  A hardware architecture for accelerating neuromorphic vision algorithms , 2011, 2011 IEEE Workshop on Signal Processing Systems (SiPS).

[12]  Thomas Serre,et al.  Robust Object Recognition with Cortex-Like Mechanisms , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Christof Koch,et al.  Modeling attention to salient proto-objects , 2006, Neural Networks.

[14]  PoggioTomaso,et al.  Robust Object Recognition with Cortex-Like Mechanisms , 2007 .

[15]  Narayanan Vijaykrishnan,et al.  SHARC: A streaming model for FPGA accelerators and its application to Saliency , 2011, 2011 Design, Automation & Test in Europe.

[16]  Laurent Itti,et al.  Applying computational tools to predict gaze direction in interactive visual environments , 2008, TAP.

[17]  Tingting Xu,et al.  A high-speed multi-GPU implementation of bottom-up attention using CUDA , 2009, 2009 IEEE International Conference on Robotics and Automation.

[18]  Wayne Luk,et al.  Towards an embedded biologically-inspired machine vision processor , 2010, 2010 International Conference on Field-Programmable Technology.