Handling time-varying confounding in state transition models for dynamic optimization of adaptive interdisciplinary pain management

ABSTRACT Interdisciplinary pain management combines multiple disciplines of professionals to understand the biological and psychosocial factors causing a patient's pain and to determine the best treatments among many to administer. To improve current and future pain outcomes, the developed adaptive interdisciplinary pain management framework employs approximate dynamic programming with state transition and outcome models estimated from actual patient data. The sequential treatment structure and observational nature of the data lead to a form of endogeneity, which results in biased model parameter estimates when developing state transition and outcome models. This research develops a process based on the inverse probability of treatment weighted method to address the endogeneity in estimating state transition and outcome models. This article discusses a general method developed for independent treatments. The proposed approach can potentially be employed for adaptive treatment in other sequential health care applications based on observational data. Our approach is demonstrated using data from the Eugene McDermott Center for Pain Management at the University of Texas Southwestern Medical Center at Dallas.

[1]  Yuan Bo Peng,et al.  The biopsychosocial approach to chronic pain: scientific advances and future directions. , 2007, Psychological bulletin.

[2]  Ignasi Serra,et al.  Time-dependent confounding in the study of the effects of regular physical activity in chronic obstructive pulmonary disease: an application of the marginal structural model. , 2008, Annals of epidemiology.

[3]  S A Murphy,et al.  Screening Experiments for Developing Dynamic Treatment Regimes , 2009, Journal of the American Statistical Association.

[4]  Ralph B D'Agostino,et al.  Estimating treatment effects using observational data. , 2007, JAMA.

[5]  George L. Nemhauser,et al.  A Stochastic Model of Airline Operations , 2002, Transp. Sci..

[6]  Robert J. Gatchel,et al.  Clinical Essentials Of Pain Management , 2004 .

[7]  Heiko Reichel,et al.  Intensive interdisciplinary outpatient pain management program for chronic back pain: a pilot study , 2012, Journal of pain research.

[8]  J. A. Calvin.  Regression Models for Categorical and Limited Dependent Variables , 1998 .

[9]  James M. Robins,et al.  Causal Inference from Complex Longitudinal Data , 1997 .

[10]  S. Murphy,et al.  Optimal dynamic treatment regimes , 2003 .

[11]  Bruce D. Naliboff,et al.  Effectiveness of an interdisciplinary pain management program for the treatment of chronic pelvic pain , 1990, Pain.

[12]  J. Robins,et al.  Marginal Structural Models to Estimate the Joint Causal Effect of Nonrandomized Treatments , 2001 .

[13]  Harold I Feldman,et al.  Model Selection, Confounder Control, and Marginal Structural Models , 2004 .

[14]  D. Nampiaparampil,et al.  Chronic Pain Management: Guidelines for Multidisciplinary Program Development , 2008 .

[15]  J. Robins.  A new approach to causal inference in mortality studies with a sustained exposure period—application to control of the healthy worker survivor effect , 1986 .

[16]  Patrick Richard,et al.  The economic costs of pain in the United States. , 2012, The journal of pain : official journal of the American Pain Society.

[17]  T. Lancaster,et al.  Instrumental variables and inverse probability weighting for causal inference from longitudinal observational studies , 2004, Statistical methods in medical research.

[18]  Victoria C. P. Chen,et al.  An adaptive pain management framework , 2010 .

[19]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[20]  Vincent Mor,et al.  Principles for modeling propensity scores in medical research: a systematic literature review , 2004, Pharmacoepidemiology and drug safety.

[21]  Stephen R Cole,et al.  Constructing inverse probability weights for marginal structural models. , 2008, American journal of epidemiology.

[22]  Jay M. Rosenberger,et al.  Two-stage stochastic programming for interdisciplinary pain management , 2019, IISE Transactions on Healthcare Systems Engineering.

[23]  R. Gatchel,et al.  A decision-making framework for adaptive pain management , 2013, Health Care Management Science.

[24]  Victoria C. P. Chen,et al.  Mining and modeling for a metropolitan Atlanta ozone pollution decision-making framework , 2007 .

[25]  Seoung Bum Kim,et al.  A data-integrated simulation model to evaluate nurse–patient assignments , 2009, Health care management science.

[26]  R J Little,et al.  Causal effects in clinical and epidemiological studies via potential outcomes: concepts and analytical approaches. , 2000, Annual review of public health.

[27]  P.J. Werbos,et al.  Using ADP to Understand and Replicate Brain Intelligence: the Next Level Design , 2007, 2007 IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning.

[28]  R. Gatchel,et al.  Evidence-based scientific data documenting the treatment and cost-effectiveness of comprehensive pain programs for chronic nonmalignant pain. , 2006, The journal of pain : official journal of the American Pain Society.

[29]  Michael T. Manry,et al.  Second Order Training of a Smoothed Piecewise Linear Network , 2017, Neural Processing Letters.

[30]  Andrew J. Schaefer,et al.  Modeling Medical Treatment Using Markov Decision Processes , 2005 .

[31]  Joelle Pineau,et al.  Adaptive Treatment of Epilepsy via Batch-mode Reinforcement Learning , 2008, AAAI.

[32]  Christopher Eccleston,et al.  Interdisciplinary management of adolescent chronic pain: developing the role of physiotherapy , 2004 .

[33]  Erkan ARI,et al.  PARALLEL LINES ASSUMPTION IN ORDINAL LOGISTIC REGRESSION AND ANALYSIS APPROACHES , 2014 .

[34]  Jay M. Rosenberger,et al.  Bounds for Optimal Control of a Regional Plug-in Electric Vehicle Charging Station System , 2018, IEEE Transactions on Industry Applications.

[35]  S. Murphy,et al.  An experimental design for the development of adaptive treatment strategies , 2005, Statistics in medicine.

[36]  Magnus Olason.  Outcome of an interdisciplinary pain management program in a rehabilitation clinic. , 2004, Work.

[37]  Tyler J. VanderWeele,et al.  Marginal Structural Models for the Estimation of Direct and Indirect Effects , 2009, Epidemiology.

[38]  Gazi Md Daud Iqbal,et al.  Multi-objective two-stage stochastic programming for adaptive interdisciplinary pain management with piecewise linear network transition models , 2017, IISE Transactions on Healthcare Systems Engineering.

[39]  Benjamin Pfaff,et al.  Pain Management An Interdisciplinary Approach , 2016 .

[40]  J. Robins,et al.  Adjusting for differential rates of prophylaxis therapy for PCP in high- versus low-dose AZT treatment arms in an AIDS randomized trial , 1994 .

[41]  M. Kramer,et al.  Estimating Response-Maximized Decision Rules With Applications to Breastfeeding , 2009 .

[42]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[43]  Heather Robbins,et al.  A Prospective One-Year Outcome Study of Interdisciplinary Chronic Pain Management: Compromising Its Efficacy by Managed Care Policies , 2003, Anesthesia and analgesia.

[44]  James M. Robins,et al.  Association, Causation, And Marginal Structural Models , 1999, Synthese.

[45]  M. Davidian,et al.  Marginal structural models for analyzing causal effects of time-dependent treatments: an application in perinatal epidemiology. , 2004, American journal of epidemiology.

[46]  J. Robins,et al.  Marginal Structural Models and Causal Inference in Epidemiology , 2000, Epidemiology.

[47]  Ching-Feng Lin.  Adaptive Pain Management Decision Support System , 2010 .

[48]  S. Murphy,et al.  The multiphase optimization strategy (MOST) and the sequential multiple assignment randomized trial (SMART): new methods for more potent eHealth interventions. , 2007, American journal of preventive medicine.

[49]  Joelle Pineau,et al.  Constructing evidence-based treatment strategies using methods from computer science. , 2007, Drug and alcohol dependence.

[50]  Alan Agresti,et al.  Categorical Data Analysis , 2003 .

[51]  M. Hernán,et al.  Controlling for Time-dependent Confounding using Marginal Structural Models , 2004 .

[52]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .

[53]  Joelle Pineau,et al.  Informing sequential clinical decision-making through reinforcement learning: an empirical study , 2010, Machine Learning.

[54]  Philip W. Lavori,et al.  A design for testing clinical strategies: biased adaptive within‐subject randomization , 2000 .

[55]  Susan A Murphy,et al.  Customizing treatment to the patient: adaptive treatment strategies. , 2007, Drug and alcohol dependence.

[56]  H. Flor,et al.  Efficacy of multidisciplinary pain treatment centers: a meta-analytic review , 1992, Pain.

[57]  Jong Min Lee,et al.  Approximate Dynamic Programming Strategies and Their Applicability for Process Control: A Review and Future Directions , 2004 .

[58]  Warren B. Powell,et al.  Handbook of Learning and Approximate Dynamic Programming , 2006, IEEE Transactions on Automatic Control.

[59]  P. Diggle,et al.  Latent Variable Modeling and Applications to Causality , 2017 .

[60]  Sean D Sullivan,et al.  Methods to assess intended effects of drug treatment in observational studies are reviewed. , 2004, Journal of clinical epidemiology.

[61]  Jay M. Rosenberger,et al.  Outcome And State Transition Modeling For Adaptive Interdisciplinary Pain Management , 2013 .