Identifying Opioid Use Disorder from Longitudinal Healthcare Data using a Multi-stream Transformer

Opioid Use Disorder (OUD) is a public health crisis costing the US billions of dollars annually in healthcare, lost workplace productivity, and crime. Analyzing longitudinal healthcare data is critical in addressing many real-world problems in healthcare. Leveraging the real-world longitudinal healthcare data, we propose a novel multi-stream transformer model called MUPOD for OUD identification. MUPOD is designed to simultaneously analyze multiple types of healthcare data streams, such as medications and diagnoses, by attending to segments within and across these data streams. Our model tested on the data from 392,492 patients with long-term back pain problems showed significantly better performance than the traditional models and recently developed deep learning models.

[1]  Zhenghua Chen,et al.  Two-Stream Convolution Augmented Transformer for Human Activity Recognition , 2021, AAAI.

[2]  C. Leighton,et al.  Predicting opioid use disorder and associated risk factors in a Medicaid managed care population. , 2021, American Journal of Managed Care.

[3]  G. Koren,et al.  Development of a machine learning algorithm for early detection of opioid use disorder , 2020, Pharmacology research & perspectives.

[4]  Oscar Koller,et al.  Multi-channel Transformers for Multi-articulatory Sign Language Translation , 2020, ECCV Workshops.

[5]  Kyu J. Han,et al.  State-of-the-Art Speech Recognition Using Multi-Stream Self-Attention with Dilated 1D Convolutions , 2019, 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU).

[6]  Yuval Pinter,et al.  Attention is not not Explanation , 2019, EMNLP.

[7]  Cho-Jui Hsieh,et al.  VisualBERT: A Simple and Performant Baseline for Vision and Language , 2019, ArXiv.

[8]  Kazem Rahimi,et al.  BEHRT: Transformer for Electronic Health Records , 2019, Scientific Reports.

[9]  Michael W. Dusenberry,et al.  Learning the Graphical Structure of Electronic Health Records with Graph Convolutional Transformer , 2019, AAAI.

[10]  Jimeng Sun,et al.  Pre-training of Graph Augmented Transformers for Medication Recommendation , 2019, IJCAI.

[11]  Noah A. Smith,et al.  Is Attention Interpretable? , 2019, ACL.

[12]  Darren W. Henderson,et al.  Predicting substance use disorder using long-term attention deficit hyperactivity disorder medication records in Truven , 2019, Health Informatics J..

[13]  Ying Wang,et al.  Inpatient2Vec: Medical Representation Learning for Inpatients , 2019, 2019 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[14]  Byron C. Wallace,et al.  Attention is not Explanation , 2019, NAACL.

[15]  Fei Wang,et al.  Deep learning for healthcare: review, opportunities and challenges , 2018, Briefings Bioinform..

[16]  Jindrich Libovický,et al.  Input Combination Strategies for Multi-Source Transformer Decoder , 2018, WMT.

[17]  M. Brandeau,et al.  Modeling Health Benefits and Harms of Public Policy Responses to the US Opioid Epidemic , 2018, American journal of public health.

[18]  Andreas Spanias,et al.  Attend and Diagnose: Clinical Time Series Analysis using Attention Models , 2017, AAAI.

[19]  L. Gostin,et al.  Reframing the Opioid Epidemic as a National Emergency. , 2017, JAMA.

[20]  Fenglong Ma,et al.  Dipole: Diagnosis Prediction in Healthcare via Attention-based Bidirectional Recurrent Neural Networks , 2017, KDD.

[21]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[22]  He Zhu,et al.  Treatment utilization among persons with opioid use disorder in the United States. , 2016, Drug and alcohol dependence.

[23]  Jimeng Sun,et al.  RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism , 2016, NIPS.

[24]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[25]  Svetha Venkatesh,et al.  DeepCare: A Deep Dynamic Memory Model for Predictive Medicine , 2016, PAKDD.

[26]  Walter F. Stewart,et al.  Doctor AI: Predicting Clinical Events via Recurrent Neural Networks , 2015, MLHC.

[27]  Christopher D. Manning,et al.  Effective Approaches to Attention-based Neural Machine Translation , 2015, EMNLP.

[28]  Wojciech Zaremba,et al.  Recurrent Neural Network Regularization , 2014, ArXiv.

[29]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[30]  Joshua M Sharfstein,et al.  Confronting the stigma of opioid use disorder--and its treatment. , 2014, JAMA.

[31]  D. Wijeysundera,et al.  Rates and risk factors for prolonged opioid use after major surgery: population based cohort study , 2014, BMJ : British Medical Journal.

[32]  Alex Graves.  Generating Sequences With Recurrent Neural Networks , 2013, ArXiv.

[33]  T. Mark,et al.  Psychiatric and medical comorbidities, associated pain, and health care utilization of patients prescribed buprenorphine. , 2013, Journal of substance abuse treatment.

[34]  P. Barnett.  Comparison of costs and utilization among buprenorphine and methadone patients. , 2009, Addiction.

[35]  L. Webster,et al.  Predicting aberrant behaviors in opioid-treated patients: preliminary validation of the Opioid Risk Tool. , 2005, Pain medicine.

[36]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[37]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[38]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[39]  J. Rice.  Mathematical Statistics and Data Analysis , 1988 .

[40]  Ronghang Hu,et al.  Transformer is All You Need: Multimodal Multitask Learning with a Unified Transformer , 2021, ArXiv.

[41]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.