Identifying category representations for complex stimuli using discrete Markov chain Monte Carlo with people

With the explosion of “big data,” digital repositories of texts and images are growing rapidly. These datasets present new opportunities for psychological research, but they require new methodologies before researchers can use these datasets to yield insights into human cognition. We present a new method that allows psychological researchers to take advantage of text and image databases: a procedure for measuring human categorical representations over large datasets of items, such as arbitrary words or pictures. We call this method discrete Markov chain Monte Carlo with people (d-MCMCP). We illustrate our method by evaluating the following categories over datasets: emotions as represented by facial images, moral concepts as represented by relevant words, and seasons as represented by images drawn from large online databases. Three experiments demonstrate that d-MCMCP is powerful and flexible enough to work with complex, naturalistic stimuli drawn from large online databases.

[1]  Frank Kleemann,et al.  Un(der)paid innovators: the commercial utilization of consumer work through crowdsourcing , 2008 .

[2]  Lewis D. Griffin,et al.  Optimality of the basic colour categories for classification , 2006, Journal of The Royal Society Interface.

[3]  B. Edwards,et al.  Loop Gain Predicts the Response to Upper Airway Surgery in Patients With Obstructive Sleep Apnea , 2017, Sleep.

[4]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[5]  Samuel Greengard,et al.  Following the crowd , 2011, Commun. ACM.

[6]  Tony Jebara,et al.  B-Matching for Spectral Clustering , 2006, ECML.

[7]  Daniel J. McDuff A Human-Markov Chain Monte Carlo Method For Investigating Facial Expression Categorization , 2010 .

[8]  D. Wolpert,et al.  Cognitive Tomography Reveals Complex, Task-Independent Mental Representations , 2013, Current Biology.

[9]  Thomas L. Griffiths,et al.  Testing the Efficiency of Markov Chain Monte Carlo With People Using Facial Affect Categories , 2012, Cogn. Sci..

[10]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[11]  R. Nosofsky Attention and learning processes in the identification and categorization of integral stimuli. , 1987, Journal of experimental psychology. Learning, memory, and cognition.

[12]  Aniket Kittur,et al.  Crowdsourcing user studies with Mechanical Turk , 2008, CHI.

[13]  R. Nosofsky Attention, similarity, and the identification-categorization relationship. , 1986, Journal of experimental psychology. General.

[14]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[15]  Matthew Lease,et al.  Crowdsourcing for search evaluation , 2011, SIGF.

[16]  M. Lengyel,et al.  Mind Reading by Machine Learning: A Doubly Bayesian Method for Inferring Mental Representations , 2010 .

[17]  A. Barker Monte Carlo calculations of the radial distribution functions for a proton-electron plasma , 1965 .

[18]  Peter Green,et al.  Markov chain Monte Carlo in Practice , 1996 .

[19]  J. W. Hopkins,et al.  Incomplete Block Rank Analysis: Some Taste Test Results , 1954 .

[20]  D. Luce,et al.  Detection and Recognition " ' , 2006 .

[21]  Jay I. Myung,et al.  Optimal experimental design for model discrimination. , 2009, Psychological review.

[22]  J. Santamaria,et al.  Actigraphy: a useful tool to monitor sleep-related hypermotor seizures. , 2017, Sleep medicine.

[23]  W. T. Maddox,et al.  Relations between prototype, exemplar, and decision bound models of categorization , 1993 .

[24]  Brian A. Nosek,et al.  Liberals and conservatives rely on different sets of moral foundations. , 2009, Journal of personality and social psychology.

[25]  R. Shepard,et al.  Toward a universal law of generalization for psychological science. , 1987, Science.

[26]  Adam N. Sanborn,et al.  What Sways People’s Judgment of Sleep Quality? A Quantitative Choice-Making Study With Good and Poor Sleepers , 2017, Sleep.

[27]  F. Gregory Ashby,et al.  Multidimensional Models of Perception and Cognition , 2014 .

[28]  Frank R. Clarke,et al.  Constant‐Ratio Rule for Confusion Matrices in Speech Communication , 1957 .

[29]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[30]  James L. McClelland,et al.  The TRACE model of speech perception , 1986, Cognitive Psychology.

[31]  Adam N. Sanborn,et al.  Uncovering mental representations with Markov chain Monte Carlo , 2010, Cognitive Psychology.

[32]  Kenneth Steiglitz,et al.  Combinatorial Optimization: Algorithms and Complexity , 1981 .

[33]  J. Edmonds Paths, Trees, and Flowers , 1965, Canadian Journal of Mathematics.

[34]  R. A. Bradley Incomplete Block Rank Analysis: On the Appropriateness of the Model for a Method of Paired Comparisons , 1954 .

[35]  F. Ashby,et al.  Categorization as probability density estimation , 1995 .