A Spiking Neural Network Architecture for Object Tracking

Spiking neural network (SNN) has the advantages of high computational efficiency, low energy consumption, low memory resource consumption, and easy hardware implementation. But its training algorithm is immature and inefficiency which limits the applications of SNN. In this paper, we propose a SNN architecture named SiamSNN for object tracking to avoid the training problems. Specifically, we propose a more comprehensive parameter conversion scheme with the processes of standardization, retraining, parameter transfer, and weight normalization, in order to convert a trained CNN to a similar SNN. Then we propose an encoder named Attention with Average Rate Over Time (AAR) in order to encoding images to spiking sequences. By using IF model, the accuracy decreases by only 0.007 on MNIST compared to the original method. Our approach applies SNN to object tracking and achieves certain effects, which is a reference for SNN applications in other computer vision areas in the future.

[1]  Deepak Khosla,et al.  Spiking Deep Convolutional Neural Networks for Energy-Efficient Object Recognition , 2014, International Journal of Computer Vision.

[2]  Wei Wu,et al.  Distractor-aware Siamese Networks for Visual Object Tracking , 2018, ECCV.

[3]  Matthew Cook,et al.  Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[4]  Luca Bertinetto,et al.  Fully-Convolutional Siamese Networks for Object Tracking , 2016, ECCV Workshops.

[5]  Kaushik Roy,et al.  Going Deeper in Spiking Neural Networks: VGG and Residual Architectures , 2018, Front. Neurosci..

[6]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[7]  Clément Farabet,et al.  Towards real-time image understanding with convolutional networks , 2013 .

[8]  Shih-Chii Liu,et al.  Conversion of Continuous-Valued Deep Networks to Efficient Event-Driven Networks for Image Classification , 2017, Front. Neurosci..

[9]  Michael Felsberg,et al.  Convolutional Features for Correlation Filter Based Visual Tracking , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[10]  Huchuan Lu,et al.  Visual tracking via shallow and deep collaborative model , 2016, Neurocomputing.

[11]  Timothée Masquelier,et al.  Deep Learning in Spiking Neural Networks , 2018, Neural Networks.

[12]  Shih-Chii Liu,et al.  Learning to be efficient: algorithms for training low-latency, low-compute deep spiking neural networks , 2016, SAC.

[13]  Shuang Wang,et al.  Weighted multifeature hyperspectral image classification via kernel joint sparse representation , 2016, Neurocomputing.

[14]  Gang Pan,et al.  CSNN: An Augmented Spiking based Framework with Perceptron-Inception , 2018, IJCAI.

[15]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Xiangyang Xue,et al.  Arbitrary-Oriented Scene Text Detection via Rotation Proposals , 2017, IEEE Transactions on Multimedia.

[17]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[18]  Wolfgang Maass,et al.  Lower Bounds for the Computational Power of Networks of Spiking Neurons , 1996, Neural Computation.

[19]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[20]  Wei Wu,et al.  High Performance Visual Tracking with Siamese Region Proposal Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.