A Test of the Cognitive Assumptions of Magnitude Estimation: Commutativity does not Hold for Acceptability Judgments

The introduction of the psychophysical technique of magnitude estimation to the study of acceptability judgments (Bard et al. 1996) has led to a surge of interest in formal acceptability-judgment experiments over the past fifteen years. One of the primary reasons for its popularity is that it was developed as a tool to measure actual units of perception, offering the possibility of data that is inherently more informative than previous scaling tasks. However, there are several untested cognitive assumptions that must hold in order for ME to be the perceptual measurement test that it is purported to be. Building on the recent formalization of these assumptions in the psychophysics literature (Narens 1996, Luce 2002), this article presents two experiments designed to test whether these assumptions hold for acceptability-judgment experiments. The results suggest that the cognitive assumptions of magnitude estimation do not hold for participants in acceptability-judgment experiments, eliminating any reason to believe that ME could deliver inherently more meaningful data than other acceptability-judgment tasks.

[1]  R Duncan Luce.  A psychophysical theory of intensity proportions, joint presentations, and matches. , 2002, Psychological review.

[2]  Wayne Cowart,et al.  Experimental Syntax: Applying Objective Methods to Sentence Judgments , 1997 .

[3]  G. Fanselow,et al.  On the Informativity of Different Measures of Linguistic Acceptability , 2011 .

[4]  Frank Keller,et al.  Gradience in Grammar: Experimental and Computational Aspects of Degrees of Grammaticality , 2001 .

[5]  James Myers,et al.  The design and analysis of small-scale syntactic judgment experiments , 2009 .

[6]  Karin Zimmer,et al.  Examining the validity of numerical ratios in loudness fractionation , 2005, Perception & psychophysics.

[7]  Jon Sprouse.  A validation of Amazon Mechanical Turk for the collection of acceptability judgments in linguistic theory , 2010, Behavior research methods.

[8]  L. Richardson.  Imagery, Conation, and Cerebral Conductance , 1929 .

[9]  Louis Narens,et al.  A theory of ratio magnitude estimation , 1996 .

[10]  Jon Sprouse,et al.  A program for experimental syntax: Finding the relationship between acceptability and grammatical knowlege , 2007 .

[11]  M. Bader,et al.  Toward a model of grammaticality judgments1 , 2009, Journal of Linguistics.

[12]  D. Kirsh,et al.  Proceedings of the 25th annual conference of the Cognitive Science Society , 2003 .

[13]  Frank Keller,et al.  Linear Optimality Theory as a Model of Gradience in Grammar , 2006 .

[14]  A. Sorace,et al.  MAGNITUDE ESTIMATION OF LINGUISTIC ACCEPTABILITY , 1996 .

[15]  Carson T. Schütze.  The empirical base of linguistics: Grammaticality judgments and linguistic methodology , 1998 .

[16]  S. S. Stevens.  The direct estimation of sensory magnitudes-loudness. , 1956, The American journal of psychology.

[17]  Sam Featherston,et al.  Universals and grammaticality: wh-constraints in German and English , 2005 .

[18]  Frank Keller,et al.  A Psychophysical Law for Linguistic Judgments , 2003 .

[19]  Ellen F. Lau,et al.  Agreement Attraction in Comprehension: Representations and Processes. , 2009 .

[20]  S. S. Stevens.  On the psychophysical law. , 1957, Psychological review.

[21]  Sam Featherston,et al.  Magnitude estimation and what it can do for your syntax: some wh-constraints in German , 2005 .

[22]  W. Ellermeier,et al.  Empirical evaluation of axioms fundamental to Stevens’s ratio-scaling approach: I. Loudness production , 2000, Perception & psychophysics.

[23]  Carson T. Schütze.  The empirical base of linguistics , 2016 .

[24]  Eugene S. Edgington,et al.  Randomization Tests , 2011, International Encyclopedia of Statistical Science.

[25]  Jon Sprouse,et al.  A test of the relation between working-memory capacity and syntactic island effects , 2012 .

[26]  E. Gibson,et al.  Memory Limitations and Structural Forgetting: The Perception of Complex Ungrammatical Sentences as Grammatical , 1999 .

[27]  M. Just,et al.  Individual differences in syntactic processing: The role of working memory , 1991 .

[28]  J. Merkel.  Die Abhangigkeit zwischen Reiz und Empfindung , 1889 .