Unsupervised Clustering of Time Series Signals Using Neuromorphic Energy-Efficient Temporal Neural Networks

Unsupervised time series clustering is a challenging problem with diverse industrial applications such as anomaly detection, bio-wearables, etc. These applications typically involve small, low-power devices on the edge that collect and process real-time sensory signals. State-of-the-art time-series clustering methods perform some form of loss minimization that is extremely computationally intensive from the perspective of edge devices. In this work, we propose a neuromorphic approach to unsupervised time series clustering based on Temporal Neural Networks that is capable of ultra low-power, continuous online learning. We demonstrate its clustering performance on a subset of UCR Time Series Archive datasets. Our results show that the proposed approach either outperforms or performs similarly to most of the existing algorithms while being far more amenable for efficient hardware implementation. Our hardware assessment analysis shows that in 7 nm CMOS the proposed architecture, on average, consumes only about 0.005 mm2 die area and 22 μW power and can process each signal with about 5 ns latency.

[1]  Qianli Ma,et al.  Learning Representations for Time Series Clustering , 2019, NeurIPS.

[2]  Lars Schmidt-Thieme,et al.  Learning time-series shapelets , 2014, KDD.

[3]  Malu Zhang,et al.  Neural Population Coding for Effective Temporal Classification , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[4]  S. Thorpe,et al.  STDP-based spiking deep convolutional neural networks for object recognition , 2018 .

[5]  Timothée Masquelier,et al.  Deep Learning in Spiking Neural Networks , 2018, Neural Networks.

[6]  W. B. Johnson,et al.  Extensions of Lipschitz mappings into Hilbert space , 1984 .

[7]  Matthew Cook,et al.  Unsupervised learning of digit recognition using spike-timing-dependent plasticity , 2015, Front. Comput. Neurosci..

[8]  James E. Smith,et al.  Direct CMOS Implementation of Neuromorphic Temporal Neural Networks for Sensory Processing , 2020, ArXiv.

[9]  Sander M. Bohte,et al.  Error-backpropagation in temporally encoded networks of spiking neurons , 2000, Neurocomputing.

[10]  Abbas Nowzari-Dalini,et al.  Bio-inspired digit recognition using reward-modulated spike-timing-dependent plasticity in deep convolutional networks , 2019, Pattern Recognit..

[11]  James Smith,et al.  Space-Time Algebra: A Model for Neocortical Computation , 2018, 2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA).

[12]  Dimitris Achlioptas,et al.  Database-friendly random projections: Johnson-Lindenstrauss with binary coins , 2003, J. Comput. Syst. Sci..

[13]  Chengqi Zhang,et al.  Salient Subsequence Learning for Time Series Clustering , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  James E. Smith,et al.  Space-Time Computing with Temporal Neural Networks , 2017, Synthesis Lectures on Computer Architecture.

[15]  Anthony S. Maida,et al.  Bio-Inspired Spiking Convolutional Neural Network using Layer-wise Sparse Coding and STDP Learning , 2016, ArXiv.