Using Paraphrasing and Memory-Augmented Models to Combat Data Sparsity in Question Interpretation with a Virtual Patient Dialogue System

When interpreting questions in a virtual patient dialogue system one must inevitably tackle the challenge of a long tail of relatively infrequently asked questions. To make progress on this challenge, we investigate the use of paraphrasing for data augmentation and neural memory-based classification, finding that the two methods work best in combination. In particular, we find that the neural memory-based approach not only outperforms a straight CNN classifier on low frequency questions, but also takes better advantage of the augmented data created by paraphrasing, together yielding a nearly 10% absolute improvement in accuracy on the least frequently asked questions.

[1]  Mirella Lapata,et al.  Paraphrasing Revisited with Neural Machine Translation , 2017, EACL.

[2]  David Suendermann-Oeft,et al.  Crowdsourcing Multimodal Dialog Interactions: Lessons Learned from the HALEF Case , 2017, AAAI Workshops.

[3]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[4]  Oren Etzioni,et al.  Paraphrase-Driven Learning for Open Question Answering , 2013, ACL.

[5]  Michael White,et al.  Generating Disambiguating Paraphrases for Structurally Ambiguous Sentences , 2016, LAW@ACL.

[6]  Quoc V. Le,et al.  Distributed Representations of Sentences and Documents , 2014, ICML.

[7]  Mary T. Johnson,et al.  Development of Virtual Patient Simulations for Medical Education , 2009 .

[8]  Chris Callison-Burch,et al.  PPDB: The Paraphrase Database , 2013, NAACL.

[9]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[10]  Evan Jaffe,et al.  Combining CNNs and Pattern Matching for Question Interpretation in a Virtual Patient Dialogue System , 2017, BEA@EMNLP.

[11]  Matthew D. Zeiler.  ADADELTA: An Adaptive Learning Rate Method , 2012, ArXiv.

[12]  Chris Callison-Burch,et al.  Clustering Paraphrases by Word Sense , 2016, NAACL.

[13]  Rico Sennrich,et al.  Edinburgh Neural Machine Translation Systems for WMT 16 , 2016, WMT.

[14]  Q. Mcnemar.  Note on the sampling error of the difference between correlated proportions or percentages , 1947, Psychometrika.

[15]  Douglas Danforth,et al.  Developing a Conversational Virtual Standardized Patient to Enable Students to Practice History-Taking Skills , 2017, Simulation in healthcare : journal of the Society for Simulation in Healthcare.

[16]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[17]  Anton Leuski,et al.  An Evaluation of Alternative Strategies for Implementing Dialogue Policies Using Statistical Classification and Rules , 2011, IJCNLP 2011.

[18]  David Suendermann-Oeft,et al.  From rule-based to statistical grammars: Continuous improvement of large-scale spoken dialog systems , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[19]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[20]  Oriol Vinyals,et al.  Matching Networks for One Shot Learning , 2016, NIPS.

[21]  Brent Rossen,et al.  A crowdsourcing method to develop virtual human conversational agents , 2012, Int. J. Hum. Comput. Stud..

[22]  Aurko Roy,et al.  Learning to Remember Rare Events , 2017, ICLR.

[23]  Nitin Madnani,et al.  Effective Feature Integration for Automated Short Answer Scoring , 2015, NAACL.

[24]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[25]  Eduard Hovy,et al.  Statistical QA - Classifier vs. Re-ranker: What’s the difference? , 2003, ACL 2003.

[26]  Evan Jaffe,et al.  Interpreting Questions with a Log-Linear Ranking Model in a Virtual Patient Dialogue System , 2015, BEA@NAACL-HLT.

[27]  Roberto Navigli,et al.  The English lexical substitution task , 2009, Lang. Resour. Evaluation.