Real-time image segmentation using a spiking neuromorphic processor

Segmentation of specific objects in an image is a key task in computer vision, for which various algorithms have been proposed. However, most of these algorithms are software-oriented and have high computational complexity that makes them difficult to implement in hardware for real-time applications. The semi-supervised graph-based random walker (RW) algorithm, which seeks the solution for a large sparse matrix, shows optimal image segmentation performance in software. In this paper, we configured our existing spiking neuromorphic processor to implement an adaptation of the RW algorithm that performs segmentation using an iterative method while preserving accuracy. Our hardware platform can be adapted to achieve an extremely fast segmentation speed of up to 1000 images per second by computing all neurons in parallel. We believe our approach will facilitate the implementation of graph-based computer vision algorithms on neuromorphic hardware for low power, real-time applications. Our approach will be immensely valuable in applications that require high-performance computing to run in real time, such as biomedical image segmentation for image-guided surgery.

[1]  John G. Harris,et al.  An analog network for continuous-time segmentation , 1993, International Journal of Computer Vision.

[2]  Ralph Etienne-Cummings,et al.  Low-power, low-mismatch, highly-dense array of VLSI Mihalas-Niebur neurons , 2017, 2017 IEEE International Symposium on Circuits and Systems (ISCAS).

[3]  Tsutomu Maruyama,et al.  Real-time image segmentation based on a parallel and pipelined watershed algorithm , 2007, Journal of Real-Time Image Processing.

[4]  Ettore Napoli,et al.  FPGA-based architecture for real time segmentation and denoising of HD video , 2013, Journal of Real-Time Image Processing.

[5]  William A. Barrett,et al.  Interactive Segmentation with Intelligent Scissors , 1998, Graph. Model. Image Process..

[6]  Christof Koch,et al.  Analog VLSI motion discontinuity detectors for image segmentation , 1996, 1996 IEEE International Symposium on Circuits and Systems. Circuits and Systems Connecting the World. ISCAS 96.

[7]  K. Schwan,et al.  FPGA based real-time image segmentation for medical systems and data processing , 2005, 14th IEEE-NPSS Real Time Conference, 2005..

[8]  Jayaram K. Udupa,et al.  An ultra-fast user-steered image segmentation paradigm: live wire on the fly , 2000, IEEE Transactions on Medical Imaging.

[9]  Supun Samarasekera,et al.  User-steered image boundary segmentation , 1996, Medical Imaging.

[10]  Gert Cauwenberghs,et al.  Neuromorphic Silicon Neuron Circuits , 2011, Front. Neurosci.

[11]  Ron Kikinis,et al.  Real-Time Image Segmentation for Image-Guided Surgery , 1998, Proceedings of the IEEE/ACM SC98 Conference.

[12]  Yousef Saad,et al.  Parallel Iterative Methods for Sparse Linear Systems , 2001 .

[13]  André van Schaik,et al.  FPGA implementation of the CAR Model of the cochlea , 2014, 2014 IEEE International Symposium on Circuits and Systems (ISCAS).

[14]  André van Schaik,et al.  A Trainable Neuromorphic Integrated Circuit that Exploits Device Mismatch , 2015, ArXiv.

[15]  Wiro J. Niessen.  Model-Based Image Segmentation for Image-Guided Interventions , 2008 .

[16]  Peter G. Doyle,et al.  Random Walks and Electric Networks: REFERENCES , 1987 .

[17]  Eric A. Vittoz,et al.  Analog VLSI signal processing: Why, where, and how? , 1994, J. VLSI Signal Process..

[18]  P. Sivakumar,et al.  A REVIEW ON IMAGE SEGMENTATION TECHNIQUES , 2016 .

[19]  Shihab A. Shamma,et al.  Sound stream segregation: a neuromorphic approach to solve the “cocktail party problem” in real-time , 2015, Front. Neurosci..

[20]  Leo Grady,et al.  Random Walks for Image Segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.