An Extended Cultural Consensus Theory Model to Account for Cognitive Processes in Decision Making in Social Surveys

In recent decades, cultural consensus theory (CCT) models have been extensively applied across research domains to explore shared cultural knowledge and beliefs. These models are parameterized in terms of person-specific cognitive parameters such as abilities and guessing biases as well as item difficulties. Although psychometric test theory is also formalized in terms of abilities and item difficulties, a quality that clearly sets CCT models apart from other test theory models is their specification to operate on data in which the answer key is latent. In doing so, CCT models specify the answer key as parameters of the model, and also involved with this specification are procedures to verify the integrity of the answer key that is estimated. In this article, the authors develop the following methods to propagate the application of these CCT models in the field of social surveys: (1) by extending the underlying cognitive model to be able to account for uncertainty in decision making (“don’t know” responses), (2) by allowing covariate information to be entered in the analysis, and (3) by deriving statistical inference in the hierarchical Bayesian framework. The proposed model is fit to data describing knowledge on science and on aging to demonstrate the novel findings that can be achieved by the approach.

[1]  W. Batchelder,et al.  Test theory without an answer key , 1988 .

[2]  蔡宗漢,et al.  superdiag: A Comprehensive Test Suite for Markov Chain Non-Convergence , 2012 .

[3]  J. Kruschke Doing Bayesian Data Analysis: A Tutorial with R and BUGS , 2010 .

[4]  C. Bann,et al.  Public knowledge and attitudes scale construction: Development of short forms , 2004 .

[5]  S. Weller,et al.  A Cross-Cultural Approach to the Study of the Folk Illness Nervios , 2003, Culture, medicine and psychiatry.

[6]  Michael X. Delli Carpini,et al.  Measuring Political Knowledge: Putting First Things First , 1993 .

[7]  Clifford C. Clogg,et al.  Using Association Models in Sociological Research: Some Examples , 1982, American Journal of Sociology.

[8]  Neil A. Macmillan,et al.  Detection Theory: A User's Guide , 1991 .

[9]  Douglas L. Medin,et al.  Folkbiology of freshwater fish , 2006, Cognition.

[10]  John K Kruschke,et al.  Bayesian data analysis. , 2010, Wiley interdisciplinary reviews. Cognitive science.

[11]  William H. Batchelder,et al.  Cultural Consensus Theory: Aggregating Signed Graphs under a Balance Constraint , 2012, SBP.

[12]  S. Weller,et al.  Latino beliefs about diabetes. , 1999, Diabetes care.

[13]  E. Palmore The facts on aging quiz: a review of findings. , 1980, The Gerontologist.

[14]  R. Pardo,et al.  The Cognitive Dimension of Public Perceptions of Science: Methodological Issues , 2004 .

[15]  S. Weller,et al.  Empacho in four Latino groups: a study of intra- and inter-cultural variation in beliefs. , 1993, Medical anthropology.

[16]  Mark A. Carrozza,et al.  A test of Palmore's Facts on Aging Quizzes as alternate measures , 2002 .

[17]  R. Bates American Academy of Arts and Sciences , 1934, Nature.

[18]  E. Palmore Facts on aging. A short quiz. , 1977, The Gerontologist.

[19]  Amy M. Hightower,et al.  Science and Engineering Indicators , 1993 .

[20]  William H. Batchelder,et al.  Consensus analysis of three‐way social network data , 1997 .

[21]  Jeffery J. Mondak,et al.  Knowledge variables in cross-national social inquiry , 2004 .

[22]  S. Richardson Politics and gender , 2007 .

[23]  J. Durant,et al.  The public understanding of science , 1989, Nature.

[24]  Edgar Erdfelder,et al.  Extracting the truth from conflicting eyewitness reports: a formal modeling approach. , 2012, Journal of experimental psychology. Applied.

[25]  S. Weller Cultural Consensus Theory: Applications and Frequently Asked Questions , 2007 .

[26]  J. Norris,et al.  The factor structure of the Facts on Aging Quiz. , 1987, The Gerontologist.

[27]  D. Klemmack Comment: an examination of Palmore's Facts on Aging Quiz. , 1978, The Gerontologist.

[28]  Joachim Vandekerckhove,et al.  Bayesian Cultural Consensus Theory , 2014 .

[29]  Joseph Hilbe,et al.  Data Analysis Using Regression and Multilevel/Hierarchical Models , 2009 .

[30]  W. Batchelder,et al.  Cultural consensus theory for multiple consensus truths , 2012 .

[31]  Martyn Plummer,et al.  JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling , 2003 .

[32]  Patrick Sturgis,et al.  Science knowledge and attitudes across cultures: a meta-analysis , 2008 .

[33]  N. Crowell,et al.  The American Association of Retired Persons. , 1973, Journal of psychiatric nursing and mental health services.

[34]  S. Weller Consistency and Consensus among Informants: Disease Concepts in a Rural Mexican Village , 1984 .

[35]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[36]  T. Liao The nonrandom selection of don't knows in binary and ordinal responses: Corrections with the bivariate probit model with sample selection , 1995 .

[37]  Deniz Senturk-Doganaksoy,et al.  Explanatory Item Response Models: A Generalized Linear and Nonlinear Approach , 2006, Technometrics.

[38]  C. Spearman The proof and measurement of association between two things. By C. Spearman, 1904. , 1987, The American journal of psychology.

[39]  L. Feick LATENT CLASS ANALYSIS OF SURVEY QUESTIONS THAT INCLUDE DON'T KNOW RESPONSES , 1989 .

[40]  J. Beck,et al.  Palmore's Facts on Aging Quiz: a reappraisal. , 1979, The Gerontologist.

[41]  F. Al-Shamali,et al.  Author Biographies. , 2015, Journal of social work in disability & rehabilitation.

[42]  S. Weller Cross'Cultural Concepts of Illness: Variation and Validation , 1984 .

[43]  V. Tariq,et al.  Gender differences in scientific knowledge and attitudes toward science: a comparative study of four Anglo-American nations , 2000 .

[44]  Va Arlington National Science Board. , 2010 .

[45]  M. Bauer,et al.  The Culture of Science: How the Public Relates to Science Across the Globe , 2011 .

[46]  Andrew H. Sidman,et al.  Explaining the Gender Gap in Political Knowledge , 2009, Politics & Gender.

[47]  Science knowledge and biblical literalism , 2012, Public understanding of science.

[48]  Jon D. Miller The measurement of civic scientific literacy , 1998 .

[49]  W. Batchelder,et al.  Markov chain estimation for test theory without an answer key , 2003 .

[50]  Lori A. Harris, Stephanie Dollinger PARTICIPATION IN A COURSE ON AGING: KNOWLEDGE, ATTITUDES, AND ANXIETY ABOUT AGING IN ONESELF AND OTHERS , 2001 .

[51]  D. Hruschka,et al.  When There Is More than One Answer Key: Cultural Theories of Postpartum Hemorrhage in Matlab, Bangladesh , 2008 .

[52]  E. York-Crowe,et al.  The diet and health knowledge survey: development of a short interview format. , 2006, Eating behaviors.

[53]  Jon D. Miller,et al.  Public Acceptance of Evolution , 2006, Science.

[54]  B. Courtenay,et al.  The effects of a "don't know" response on Palmore's Facts on Aging quizzes. , 1985, The Gerontologist.

[55]  Roel Bosker,et al.  Multilevel analysis : an introduction to basic and advanced multilevel modeling , 1999 .

[56]  Jeffery J. Mondak Developing Valid Knowledge Scales , 2001 .

[57]  I. Stuart-Hamilton,et al.  THE EFFECT OF AGING AWARENESS TRAINING ON KNOWLEDGE OF, AND ATTITUDES TOWARDS, OLDER ADULTS , 2003 .

[58]  S. Weller,et al.  Explanatory models of diabetes in the U.S. and Mexico: the patient-provider gap and cultural competence. , 2012, Social science & medicine.

[59]  D. Medin,et al.  Folkecology and commons management in the Maya Lowlands. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[60]  Robert C. Luskin,et al.  ''Don't Know'' Means ''Don't Know'': DK Responses and the Public's Level of Political Knowledge , 2011 .

[61]  Jeffery J. Mondak,et al.  Asked and Answered: Knowledge Levels When We Will Not Take “Don't Know” for an Answer , 2001 .

[62]  Joe D. Francis,et al.  WHAT WE NOW KNOW ABOUT “I DON'T KNOWS” , 1975 .

[63]  B. Davis,et al.  Science and engineering indicators—1996 , 1997 .

[64]  William H. Batchelder,et al.  Cultural Consensus Theory: Comparing different concepts of cultural truth , 2012 .

[65]  R. Miller,et al.  A revision of Palmore's facts on aging quiz. , 1980, The Gerontologist.

[66]  W. Batchelder,et al.  Culture as Consensus: A Theory of Culture and Informant Accuracy , 1986 .

[67]  Wm. R. Wright General Intelligence, Objectively Determined and Measured. , 1905 .

[68]  M. Bauer Socio-demographic correlates of DK-responses in knowledge surveys: self-attributed ignorance of science , 1996 .

[69]  Jon D. Miller Public Understanding of, and Attitudes toward, Scientific Research: What We Know and What We Need to Know , 2004 .

[70]  Allison L. Hopkins Use of Network Centrality Measures to Explain Individual Levels of Herbal Remedy Cultural Competence among the Yucatec Maya in Tabi, Mexico , 2011, Field methods.

[71]  Robert Cameron Mitchell,et al.  The Impact of "No Opinion" Response Options on Data Quality: Non-Attitude Reduction or an Invitation to Satisfice? , 2001 .

[72]  Jeff Gill,et al.  Bayesian Methods : A Social and Behavioral Sciences Approach , 2002 .

[73]  E. Palmore Three Decades of Research on Ageism , 2005 .

[74]  Patrick Sturgis,et al.  An experiment on the measurement of political knowledge in surveys , 2008 .

[75]  P. Berger,et al.  The Social Construction of Reality , 1966 .