Inferring Cognitive Models from Data using Approximate Bayesian Computation

An important problem for HCI researchers is to estimate the parameter values of a cognitive model from behavioral data. This is a difficult problem, because of the substantial complexity and variety in human behavioral strategies. We report an investigation into a new approach using approximate Bayesian computation (ABC) to condition model parameters to data and prior knowledge. As the case study we examine menu interaction, where we have click time data only to infer a cognitive model that implements a search behaviour with parameters such as fixation duration and recall probability. Our results demonstrate that ABC (i) improves estimates of model parameter values, (ii) enables meaningful comparisons between model variants, and (iii) supports fitting models to individual users. ABC provides ample opportunities for theoretical HCI research by allowing principled inference of model parameter values and their uncertainty.

[1]  John R. Anderson Is human cognition adaptive? , 1991, Behavioral and Brain Sciences.

[2]  Antti Oulasvirta,et al.  Model of visual search and selection time in linear menus , 2014, CHI.

[3]  J. Beck,et al.  Model Selection using Response Measurements: Bayesian Probabilistic Approach , 2004 .

[4]  Anind K. Dey,et al.  Maximum Entropy Inverse Reinforcement Learning , 2008, AAAI.

[5]  K. Rayner Eye movements in reading and information processing: 20 years of research. , 1998, Psychological bulletin.

[6]  John M. Flach,et al.  Control Theory for Humans: Quantitative Approaches To Modeling Performance , 2002 .

[7]  Erik Nilsen,et al.  Exploring the divide between two unified theories of cognition: modeling visual attention in menu selection , 1999, CHI EA '99.

[8]  Antti Oulasvirta,et al.  Informing the Design of Novel Input Methods with Muscle Coactivation Clustering , 2015, ACM Trans. Comput. Hum. Interact..

[9]  Eyal Amir,et al.  Bayesian Inverse Reinforcement Learning , 2007, IJCAI.

[10]  Andrew Howes,et al.  The adaptation of visual search to utility, ecology and design , 2015, Int. J. Hum. Comput. Stud..

[11]  ปิยดา สมบัติวัฒนา Behavioral Game Theory: Experiments in Strategic Interaction , 2013 .

[12]  Andrew Y. Ng,et al.  Pharmacokinetics of a novel formulation of ivermectin after administration to goats , 2000, ICML.

[13]  Michael U. Gutmann,et al.  Bayesian Optimization for Likelihood-Free Inference of Simulator-Based Statistical Models , 2015, J. Mach. Learn. Res..

[14]  D. Kieras,et al.  Modern computational perspectives on executive mental processes and cognitive control: Where to from here? , 2000 .

[15]  Peter Pirolli,et al.  Information Foraging , 2009, Encyclopedia of Database Systems.

[16]  Leif Azzopardi,et al.  Modelling interaction with economic models of search , 2014, SIGIR.

[17]  Antti Oulasvirta,et al.  The Emergence of Interactive Behavior: A Model of Rational Menu Search , 2015, CHI.

[18]  Shumin Zhai,et al.  Hard lessons: effort-inducing interfaces benefit spatial learning , 2007, CHI.

[19]  ELFI ELFI: Engine for Likelihood-Free Inference , 2018 .

[20]  David E. Kieras,et al.  An Overview of the EPIC Architecture for Cognition and Performance With Application to Human-Computer Interaction , 1997, Hum. Comput. Interact..

[21]  N. Chater,et al.  Ten years of the rational analysis of cognition , 1999, Trends in Cognitive Sciences.

[22]  D. Ballard,et al.  Modeling Task Control of Eye Movements , 2014, Current Biology.

[23]  Anthony J. Hornof,et al.  A Computational Model of “Active Vision” for Visual Search in Human–Computer Interaction , 2011, Hum. Comput. Interact..

[24]  O. François,et al.  Approximate Bayesian Computation (ABC) in practice. , 2010, Trends in ecology & evolution.

[25]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[26]  Richard L. Lewis,et al.  of the Annual Meeting of the Cognitive Science Society Title Bounded Optimal State Estimation and Control in Visual Search : Explaining Distractor Ratio Effects Permalink , 2013 .

[27]  Jeremy L. Wyatt,et al.  Models of gaze control for manipulation tasks , 2013, ACM Trans. Appl. Percept..

[28]  Scott P. Robertson,et al.  Proceedings of the SIGCHI Conference on Human Factors in Computing Systems , 1991 .

[29]  Wai-Tat Fu,et al.  SNIF-ACT: A Cognitive Model of User Navigation on the World Wide Web , 2007, Hum. Comput. Interact..

[30]  Anthony J. Hornof,et al.  Towards accurate and practical predictive models of active-vision-based visual search , 2014, CHI.

[31]  Andrew Howes,et al.  Adaptive Interaction: A Utility Maximization Approach to Understanding Human Interaction with Technology , 2013, Adaptive Interaction: A Utility Maximization Approach to Understanding Human Interaction with Technology.

[32]  Nick Chater,et al.  A rational analysis of the selection task as optimal data selection. , 1994 .

[33]  Peter Pirolli,et al.  Rational Analyses of Information Foraging on the Web , 2005, Cogn. Sci..

[34]  Satinder Singh,et al.  Computational Rationality: Linking Mechanism and Behavior Through Bounded Utility Maximization , 2014, Top. Cogn. Sci..

[35]  Samuel J. Gershman,et al.  Computational rationality: A converging paradigm for intelligence in brains, minds, and machines , 2015, Science.

[36]  Donald A. Norman,et al.  Simulating a Skilled Typist: A Study of Skilled Cognitive-Motor Performance , 1982, Cogn. Sci..

[37]  Allen Newell,et al.  The psychology of human-computer interaction , 1983 .

[38]  Craig S. Miller,et al.  Modeling Information Navigation: Implications for Information Architecture , 2004, Hum. Comput. Interact..

[39]  R. Baloh,et al.  Quantitative measurement of saccade amplitude, duration, and velocity , 1975, Neurology.

[40]  Michael D. Byrne,et al.  ACT-R/PM and menu selection: applying a cognitive architecture to HCI , 2001, Int. J. Hum. Comput. Stud..

[41]  Duncan P. Brumby,et al.  How does knowing what you are looking for change visual search behavior? , 2014, CHI.

[42]  Anthony J. Hornof,et al.  Cognitive Strategies for the Visual Search of Hierarchical Computer Displays , 2004, Hum. Comput. Interact..

[43]  Richard L. Lewis,et al.  Rational adaptation under task and processing constraints: implications for testing theories of cognition and action. , 2009, Psychological review.

[44]  John R. Anderson,et al.  ACT-R: A Theory of Higher Level Cognition and Its Relation to Visual Attention , 1997, Hum. Comput. Interact..

[45]  Jay I. Myung,et al.  Model Comparison in Psychology , 2016 .

[46]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[47]  Jukka Corander,et al.  Approximate Bayesian Computation , 2013, PLoS Comput. Biol..