REFUEL: Exploring Sparse Features in Deep Reinforcement Learning for Fast Disease Diagnosis

This paper proposes REFUEL, a reinforcement learning method with two techniques: {\em reward shaping} and {\em feature rebuilding}, to improve the performance of online symptom checking for disease diagnosis. Reward shaping can guide the search of policy towards better directions. Feature rebuilding can guide the agent to learn correlations between features. Together, they can find symptom queries that can yield positive responses from a patient with high probability. Experimental results justify that the two techniques in REFUEL allows the symptom checker to identify the disease more rapidly and accurately.

[1]  Jing Peng,et al.  Function Optimization using Connectionist Reinforcement Learning Algorithms , 1991 .

[2]  Preben Alstrøm,et al.  Learning to Drive a Bicycle Using Reinforcement Learning and Shaping , 1998, ICML.

[3]  Andrew Y. Ng,et al.  Policy Invariance Under Reward Transformations: Theory and Application to Reward Shaping , 1999, ICML.

[4]  Ronald J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[5]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[6]  Daniel Kudenko,et al.  Theoretical and Empirical Analysis of Reward Shaping in Reinforcement Learning , 2009, 2009 International Conference on Machine Learning and Applications.

[7]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[8]  Matt J. Kusner,et al.  Cost-Sensitive Tree of Classifiers , 2012, ICML.

[9]  Yixin Chen,et al.  Feature-Cost Sensitive Learning with Submodular Trees of Classifiers , 2014, AAAI.

[10]  Venkatesh Saligrama,et al.  Feature-Budgeted Random Forest , 2015, ICML.

[11]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[12]  C. Gidengil,et al.  Evaluation of symptom checkers for self diagnosis and triage: audit study , 2015, BMJ : British Medical Journal.

[13]  David Silver,et al.  Deep Reinforcement Learning with Double Q-Learning , 2015, AAAI.

[14]  Kai-Fu Tang,et al.  Inquire and Diagnose : Neural Symptom Checking Ensemble using Deep Reinforcement Learning , 2016 .

[15]  Venkatesh Saligrama,et al.  Pruning Random Forests for Prediction on a Budget , 2016, NIPS.

[16]  Venkatesh Saligrama,et al.  Adaptive Classification for Prediction Under a Budget , 2017, NIPS.

[17]  Edward Y. Chang,et al.  Context-Aware Symptom Checking for Disease Diagnosis Using Hierarchical Reinforcement Learning , 2018, AAAI.

[18]  Tomás Pevný,et al.  Classification with Costly Features using Deep Reinforcement Learning , 2019, AAAI.