Accurate and Efficient LIF-Nets for 3D Detection and Recognition

3D object detection and recognition are crucial tasks for many spatiotemporal processing applications, such as computer-aided diagnosis and autonomous driving. Although prevalent 3D Convolution Nets (ConvNets) have continued to improve the accuracy and sensitivity, excessive computing resources are required. In this paper, we propose Leaky Integrate and Fire Networks (LIF-Nets) for 3D detection and recognition tasks. LIF-Nets have rich inter-frame sensing capability brought from membrane potentials, and low power event-driven mechanism, which make them excel in 3D processing and save computational cost at the same time. We also develop ResLIF Blocks to solve the degradation problem of deep LIF-Nets, and employ U-LIF structure to improve the feature representation capability. As a result, we carry out experiments on the LUng Nodule Analysis 2016 (LUNA16) public dataset for chest CT automated analysis and conclude that the LIF-Nets achieve 94.6% detection sensitivity at 8 False Positives per scan and 94.14% classification accuracy while the LIF-detection net reduces 65.45% multiplication operations, 65.12% addition operations, and 65.32% network parameters. The results show that LIF-Nets have extraordinary time-efficient and energy-saving performance while achieving comparable accuracy.

[1]  Nikola Kasabov,et al.  Integrating Space, Time, and Orientation in Spiking Neural Networks: A Case Study on Multimodal Brain Data Modeling , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[2]  Ulas Bagci,et al.  Risk Stratification of Lung Nodules Using 3D CNN-Based Multi-task Learning , 2017, IPMI.

[3]  Wei Shen,et al.  Multi-crop Convolutional Neural Networks for lung nodule malignancy suspiciousness classification , 2017, Pattern Recognit..

[4]  Xiaohui Xie,et al.  DeepLung: Deep 3D Dual Path Nets for Automated Pulmonary Nodule Detection and Classification , 2018, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[5]  Andrew S. Cassidy,et al.  Conversion of artificial recurrent neural networks to spiking neural networks for low-power neuromorphic hardware , 2016, 2016 IEEE International Conference on Rebooting Computing (ICRC).

[6]  Ayman El-Baz,et al.  Computer-Aided Diagnosis Systems for Lung Cancer: Challenges and Methodologies , 2013, Int. J. Biomed. Imaging.

[7]  S. Thorpe,et al.  STDP-based spiking deep convolutional neural networks for object recognition , 2018 .

[8]  Cuicui Zhang,et al.  Computer-Aided Diagnosis (CAD) of Pulmonary Nodule of Thoracic CT Image Using Transfer Learning , 2019, Journal of Digital Imaging.

[9]  Haizhou Li,et al.  Deep Spiking Neural Network with Spike Count based Learning Rule , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[10]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[11]  Jin Hu,et al.  Spatio-temporal EEG Data Classification in the NeuCube 3D SNN Environment: Methodology and Examples , 2013, ICONIP.

[12]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  U. Rajendra Acharya,et al.  Deep convolutional neural network for the automated detection and diagnosis of seizure using EEG signals , 2017, Comput. Biol. Medicine.

[14]  Ming Yang,et al.  3D Convolutional Neural Networks for Human Action Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Raunak Dey,et al.  Diagnostic classification of lung nodules using 3D neural networks , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[16]  Hao Chen,et al.  Automated Pulmonary Nodule Detection via 3D ConvNets with Online Sample Filtering and Hybrid-Loss Residual Learning , 2017, MICCAI.

[17]  Zhe Li,et al.  Evaluate the Malignancy of Pulmonary Nodules Using the 3-D Deep Leaky Noisy-OR Network , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[18]  Xingwei Liu,et al.  An End-to-End Framework for Integrated Pulmonary Nodule Detection and False Positive Reduction , 2019, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[19]  Timothée Masquelier,et al.  Deep Learning in Spiking Neural Networks , 2018, Neural Networks.

[20]  João Manuel R. S. Tavares,et al.  Automatic 3D pulmonary nodule detection in CT images: A survey , 2016, Comput. Methods Programs Biomed..

[21]  Alberto Traverso,et al.  Clinical validation of the M5L lung Computer-Assisted Detection system , 2016 .

[22]  Maryam Gholami Doborjeh,et al.  Mapping, Learning, Visualization, Classification, and Understanding of fMRI Data in the NeuCube Evolving Spatiotemporal Data Machine of Spiking Neural Networks , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[23]  Tobi Delbrück,et al.  Training Deep Spiking Neural Networks Using Backpropagation , 2016, Front. Neurosci..

[24]  Nikola K. Kasabov,et al.  A Retinotopic Spiking Neural Network System for Accurate Recognition of Moving Objects Using NeuCube and Dynamic Vision Sensors , 2018, Front. Comput. Neurosci..

[25]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[26]  Tingting Zhao,et al.  A hybrid CNN feature model for pulmonary nodule malignancy risk differentiation. , 2017, Journal of X-ray science and technology.

[27]  Niranjan Khandelwal,et al.  A Combination of Shape and Texture Features for Classification of Pulmonary Nodules in Lung CT Images , 2016, Journal of Digital Imaging.

[28]  Yangfan Peng,et al.  Excitatory Microcircuits within Superficial Layers of the Medial Entorhinal Cortex. , 2017, Cell reports.

[29]  Lei Deng,et al.  Spatio-Temporal Backpropagation for Training High-Performance Spiking Neural Networks , 2017, Front. Neurosci..

[30]  Hesham Mostafa,et al.  Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-based optimization to spiking neural networks , 2019, IEEE Signal Processing Magazine.

[31]  Sungroh Yoon,et al.  Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detection , 2020, AAAI.

[32]  Michael Pfeiffer,et al.  Deep Learning With Spiking Neurons: Opportunities and Challenges , 2018, Front. Neurosci..

[33]  Lei Deng,et al.  Direct Training for Spiking Neural Networks: Faster, Larger, Better , 2018, AAAI.

[34]  Hao Tang,et al.  Automated pulmonary nodule detection using 3D deep convolutional neural networks , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[35]  Jie Yang,et al.  Simultaneous Accurate Detection of Pulmonary Nodules and False Positive Reduction Using 3D CNNs , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[36]  Wenrui Zhang,et al.  Hybrid Macro/Micro Level Backpropagation for Training Deep Spiking Neural Networks , 2018, NeurIPS.

[37]  L. F Abbott,et al.  Lapicque’s introduction of the integrate-and-fire model neuron (1907) , 1999, Brain Research Bulletin.

[38]  Xin Geng,et al.  Classification of Lung Nodule Malignancy Risk on Computed Tomography Images Using Convolutional Neural Network: A Comparison Between 2D and 3D Strategies , 2016, ACCV Workshops.

[39]  Hao Chen,et al.  Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: The LUNA16 challenge , 2016, Medical Image Anal..

[40]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Maryam Gholami Doborjeh,et al.  Attentional Bias Pattern Recognition in Spiking Neural Networks from Spatio-Temporal EEG Data , 2018, Cognitive Computation.

[42]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[43]  Maryam Gholami Doborjeh,et al.  Evolving, dynamic clustering of spatio/spectro-temporal data in 3D spiking neural network models and a case study on EEG data , 2018, Evol. Syst..

[44]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[45]  Aoxue Li,et al.  Accurate Pulmonary Nodule Detection in Computed Tomography Images Using Deep Convolutional Neural Networks , 2017, MICCAI.

[46]  Bernabé Linares-Barranco,et al.  Feedforward Categorization on AER Motion Events Using Cortex-Like Features in a Spiking Neural Network , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[47]  Wei Shen,et al.  Multi-scale Convolutional Neural Networks for Lung Nodule Classification , 2015, IPMI.

[48]  Wulfram Gerstner,et al.  Neuronal Dynamics: From Single Neurons To Networks And Models Of Cognition , 2014 .