ResOT: Resource-Efficient Oblique Trees for Neural Signal Classification

Classifiers that can be implemented on chip with minimal computational and memory resources are essential for edge computing in emerging applications such as medical and IoT devices. This paper introduces a machine learning model based on oblique decision trees to enable resource-efficient classification on a neural implant. By integrating model compression with probabilistic routing and implementing cost-aware learning, our proposed model could significantly reduce the memory and hardware cost compared to state-of-the-art models, while maintaining the classification accuracy. We trained the resource-efficient oblique tree with power-efficient regularization (ResOT-PE) on three neural classification tasks to evaluate the performance, memory, and hardware requirements. On seizure detection task, we were able to reduce the model size by 3.4× and the feature extraction cost by 14.6× compared to the ensemble of boosted trees, using the intracranial EEG from 10 epilepsy patients. In a second experiment, we tested the ResOT-PE model on tremor detection for Parkinson's disease, using the local field potentials from 12 patients implanted with a deep-brain stimulation (DBS) device. We achieved a comparable classification performance as the state-of-the-art boosted tree ensemble, while reducing the model size and feature extraction cost by 10.6× and 6.8×, respectively. We also tested on a 6-class finger movement detection task using ECoG recordings from 9 subjects, reducing the model size by 17.6× and feature computation cost by 5.1×. The proposed model can enable a low-power and memory-efficient implementation of classifiers for real-time neurological disease detection and motor decoding.

[1]  Mahsa Shoaran,et al.  Hardware Complexity Analysis of Deep Neural Networks and Decision Tree Ensembles for Real-time Neural Data Classification , 2019, 2019 9th International IEEE/EMBS Conference on Neural Engineering (NER).

[2]  Keshab K. Parhi,et al.  Seizure detection using regression tree based feature selection and polynomial SVM classification , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[3]  Marian Verhelst,et al.  A 90 nm CMOS, $6\ {\upmu {\text{W}}}$ Power-Proportional Acoustic Sensing Frontend for Voice Activity Detection , 2016, IEEE Journal of Solid-State Circuits.

[4]  Ullrich Köthe,et al.  On Oblique Random Forests , 2011, ECML/PKDD.

[5]  Margo I. Seltzer,et al.  Optimal Sparse Decision Trees , 2019, NeurIPS.

[6]  Miguel Á. Carreira-Perpiñán,et al.  Alternating optimization of decision trees, with application to learning sparse oblique trees , 2018, NeurIPS.

[7]  Mahsa Shoaran,et al.  Cost-Efficient Classification for Neurological Disease Detection , 2019, 2019 IEEE Biomedical Circuits and Systems Conference (BioCAS).

[8]  Chunhua Deng,et al.  PermDNN: Efficient Compressed DNN Architecture with Permuted Diagonal Matrices , 2018, 2018 51st Annual IEEE/ACM International Symposium on Microarchitecture (MICRO).

[9]  Mahsa Shoaran,et al.  Improved detection of Parkinsonian resting tremor with feature engineering and Kalman filtering , 2019, Clinical Neurophysiology.

[10]  Ryan P. Adams,et al.  SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained Microcontrollers , 2019, NeurIPS.

[11]  Mahsa Shoaran,et al.  Enhanced Classification of Individual Finger Movements with ECoG , 2019, 2019 53rd Asilomar Conference on Signals, Systems, and Computers.

[12]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[13]  Incremental-Precision Based Feature Computation and Multi-Level Classification for Low-Energy Internet-of-Things , 2018, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[14]  B. Hjorth EEG analysis based on time domain properties. , 1970, Electroencephalography and clinical neurophysiology.

[15]  Ronald L. Rivest,et al.  Constructing Optimal Binary Decision Trees is NP-Complete , 1976, Inf. Process. Lett..

[16]  Prateek Jain,et al.  FastGRNN: A Fast, Accurate, Stable and Tiny Kilobyte Sized Gated Recurrent Neural Network , 2018, NeurIPS.

[17]  Tie-Yan Liu,et al.  LightGBM: A Highly Efficient Gradient Boosting Decision Tree , 2017, NIPS.

[18]  Naveen Verma,et al.  A Micro-Power EEG Acquisition SoC With Integrated Feature Extraction Processor for a Chronic Seizure Detection System , 2010, IEEE Journal of Solid-State Circuits.

[19]  Boris Murmann,et al.  Memory-Optimal Direct Convolutions for Maximizing Classification Accuracy in Embedded Applications , 2019, ICML.

[20]  Fei Zhang,et al.  An implantable neuroprocessor for multichannel compressive neural recording and on-the-fly spike sorting with wireless telemetry , 2010, 2010 Biomedical Circuits and Systems Conference (BioCAS).

[21]  Banshidhar Majhi,et al.  Brain MR image classification using two-dimensional discrete wavelet transform and AdaBoost with random forests , 2016, Neurocomputing.

[22]  Song Han,et al.  Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding , 2015, ICLR.

[23]  Naveen Verma,et al.  A Low-Power Processor With Configurable Embedded Machine-Learning Accelerators for High-Order and Adaptive Analysis of Medical-Sensor Signals , 2013, IEEE Journal of Solid-State Circuits.

[24]  Fred A. Hamprecht,et al.  End-to-end Learning of Deterministic Decision Trees , 2017, GCPR.

[25]  Yann LeCun,et al.  Optimal Brain Damage , 1989, NIPS.

[26]  Chen Zhang,et al.  A 16-Channel Patient-Specific Seizure Onset and Termination Detection SoC With Impedance-Adaptive Transcranial Electrical Stimulator , 2015, IEEE Journal of Solid-State Circuits.

[27]  Saurabh Goyal,et al.  Resource-efficient Machine Learning in 2 KB RAM for the Internet of Things , 2017, ICML.

[28]  Tim Oates,et al.  A Flexible Multichannel EEG Feature Extractor and Classifier for Seizure Detection , 2015, IEEE Transactions on Circuits and Systems II: Express Briefs.

[29]  Marian Verhelst,et al.  A 90 nm CMOS, 6µW Power-Proportional Acoustic Sensing Frontend for Voice Activity Detection , 2016, IEEE J. Solid State Circuits.

[30]  Mahsa Shoaran,et al.  Energy-Efficient Classification for Resource-Constrained Biomedical Applications , 2018, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[31]  Keshab K. Parhi,et al.  Low-Complexity Seizure Prediction From iEEG/sEEG Using Spectral Power and Ratios of Spectral Power , 2016, IEEE Transactions on Biomedical Circuits and Systems.

[32]  Shoeb Shaikh,et al.  Towards Intelligent Intra-cortical BMI (i2BMI): Low-power Neuromorphic Decoders that outperform Kalman Filters , 2019, bioRxiv.

[33]  Fred A. Hamprecht,et al.  Cost efficient gradient boosting , 2017, NIPS.

[34]  Lin Yao,et al.  Resting Tremor Detection in Parkinson's Disease with Machine Learning and Kalman Filtering , 2018, 2018 IEEE Biomedical Circuits and Systems Conference (BioCAS).

[35]  Brian Litt,et al.  Epilepsyecosystem.org: crowd-sourcing reproducible seizure prediction with long-term human intracranial EEG , 2018, Brain : a journal of neurology.

[36]  Shoeb Shaikh,et al.  Towards Intelligent Intracortical BMI (i$^2$BMI): Low-Power Neuromorphic Decoders That Outperform Kalman Filters , 2019, IEEE Transactions on Biomedical Circuits and Systems.

[37]  Nick F. Ramsey,et al.  Human Motor Cortical Activity Is Selectively Phase-Entrained on Underlying Rhythms , 2012, PLoS Comput. Biol..

[38]  Sachin S. Talathi,et al.  Fixed Point Quantization of Deep Convolutional Networks , 2015, ICML.

[39]  Mahsa Shoaran,et al.  Hardware-Efficient Seizure Detection , 2019, 2019 53rd Asilomar Conference on Signals, Systems, and Computers.

[40]  David J. Fleet,et al.  Efficient Non-greedy Optimization of Decision Trees , 2015, NIPS.

[41]  Mahsa Shoaran,et al.  Migraine classification using somatosensory evoked potentials , 2019, Cephalalgia : an international journal of headache.

[42]  Keshab K. Parhi,et al.  Seizure detection using wavelet decomposition of the prediction error signal from a single channel of intra-cranial EEG , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[43]  Kilian Q. Weinberger,et al.  The Greedy Miser: Learning under Test-time Budgets , 2012, ICML.

[44]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[45]  Anantha Chandrakasan,et al.  An 8-Channel Scalable EEG Acquisition SoC With Patient-Specific Seizure Classification and Recording Processor , 2013, IEEE Journal of Solid-State Circuits.

[46]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[47]  Antonio Criminisi,et al.  Adaptive Neural Trees , 2018, ICML.

[48]  Brian Litt,et al.  Collaborating and sharing data in epilepsy research. , 2015, Journal of clinical neurophysiology : official publication of the American Electroencephalographic Society.

[49]  Arindam Basu,et al.  A 128-Channel Extreme Learning Machine-Based Neural Decoder for Brain Machine Interfaces , 2015, IEEE Transactions on Biomedical Circuits and Systems.