Selecting single items to measure doubly concrete constructs: A cautionary tale

Single-item measures have recently become more en vogue due to studies arguing in favor of their psychometric properties vis-a-vis multi-item scales. However, their effective use requires (1) expert raters to designate the focal construct as being doubly concrete and (2) researchers to find a good single item to represent the construct. This study examines whether expert raters identify the doubly concrete nature of constructs that prior research presents as exemplary in this respect. Furthermore, the study compares the efficacy of a broad range of selection mechanisms based on expert judgment and statistical criteria for identifying the best item in a scale. The results show that expert raters do not share the commonly held belief that researchers can validly measure constructs such as attitude toward the ad, or brand, with single items. Further analyses show that neither rater assessments nor statistical criteria prove valuable regarding identifying an appropriate single item from a set of candidate items.

[1]  D. G. Gardner,et al.  Single-Item Versus Multiple-Item Measurement Scales: An Empirical Comparison , 1998 .

[2]  D. Andrich.  A rating formulation for ordered response categories , 1978 .

[3]  Marko Sarstedt,et al.  Exploring the influence of customers' time horizon perspectives on the satisfaction-loyalty link☆ , ☆☆ , 2014 .

[4]  鄭宇庭.  行銷硏究 : Marketing research , 2009 .

[5]  R. Loo.  A caveat on using single‐item versus multiple‐item scales , 2002 .

[6]  Carol C. Bienstock,et al.  Question order effects in measuring service quality , 2003 .

[7]  Thomas Salzberger,et al.  Measurement in Marketing Research: An Alternative Framework , 2009 .

[8]  W. Balzer,et al.  ISSUES AND STRATEGIES FOR REDUCING THE LENGTH OF SELF‐REPORT SCALES , 2002 .

[9]  M. Mallon,et al.  Reliability and Validity of a Single-Item Measure of Job Satisfaction , 2005, American journal of health promotion : AJHP.

[10]  Robert F. DeVellis,et al.  Scale Development: Theory and Applications. , 1992 .

[11]  M. Sprangers,et al.  Is a single-item visual analogue scale as valid, reliable and responsive as multi-item scales in measuring quality of life? , 2004, Quality of Life Research.

[12]  J. Rossiter,et al.  Tailor-made single-item measures of doubly concrete constructs , 2009 .

[13]  D. Robins,et al.  The Modified Checklist for Autism in Toddlers: An Initial Study Investigating the Early Detection of Autism and Pervasive Developmental Disorders , 2001, Journal of autism and developmental disorders.

[14]  Adamantios Diamantopoulos,et al.  Measuring Abstract Constructs in Management and Organizational Research: The Case of Export Coordination , 2008 .

[15]  A. Diamantopoulos,et al.  Guidelines for choosing between multi-item and single-item scales for construct measurement: a predictive validity perspective , 2012 .

[16]  Jagdip Singh,et al.  Tackling measurement problems with Item Response Theory: Principles, characteristics, and assessment, with an illustrative example , 2004 .

[17]  D. G. Morrison,et al.  Do We Really Need Multiple-Item Measures in Service Research? , 2001 .

[18]  William R. Darden,et al.  Mutually Adaptive Effects of Interpersonal Communication , 1971 .

[19]  Gail R. Casper,et al.  Focus on Psychometrics Single-Item Indicators in Nursing Research , 1993 .

[20]  T. Keiningham,et al.  The value of different customer satisfaction and loyalty metrics in predicting customer retention, recommendation, and share‐of‐wallet , 2007 .

[21]  M. Sarstedt,et al.  A new criterion for assessing discriminant validity in variance-based structural equation modeling , 2015 .

[22]  Adamantios Diamantopoulos,et al.  The C-OAR-SE procedure for scale development in marketing: a comment , 2005 .

[23]  John R. Rossiter,et al.  Measurement for the Social Sciences: The C-OAR-SE Method and Why It Must Replace Psychometrics , 2010 .

[24]  Marko Sarstedt,et al.  Applying the future time perspective scale to advertising research , 2014 .

[25]  Richard G. Netemeyer,et al.  Scaling Procedures: Issues and Applications , 2003 .

[26]  Thomas Salzberger,et al.  Detecting gender item bias and differential manifest response behavior: A Rasch-based solution , 2014 .

[27]  L. Crocker,et al.  Introduction to Classical and Modern Test Theory , 1986 .

[28]  Monika Koller,et al.  Towards a new paradigm of measurement in marketing , 2013 .

[29]  W. Kamakura.  Measure twice and cut once: the carpenter’s rule still applies , 2014, Marketing Letters.

[30]  J. Fleiss.  Measuring nominal scale agreement among many raters. , 1971 .

[31]  Udo Konradt,et al.  Reliability, validity, and sensitivity of a single-item measure of online store usability , 2011, Int. J. Hum. Comput. Stud..

[32]  W Revelle,et al.  Hierarchical Cluster Analysis And The Internal Structure Of Tests. , 1979, Multivariate behavioral research.

[33]  T. C. Melewar,et al.  Measuring reputation in global markets—A comparison of reputation measures’ convergent and criterion validities , 2013 .

[34]  Marko Sarstedt,et al.  An assessment of the use of partial least squares structural equation modeling in marketing research , 2012 .

[35]  H. Kwon,et al.  The Feasibility of Single-Item Measures in Sport Loyalty Research , 2005 .

[36]  J. R. Larson,et al.  Research strategies and tactics in industrial and organizational psychology. , 1990 .

[37]  Robert W. Ruekert,et al.  Reliability and Validity of Alternative Measures of Channel Member Satisfaction , 1984 .

[38]  J. Rossiter,et al.  The Predictive Validity of Multiple-Item versus Single-Item Measures of the Same Constructs , 2007 .

[39]  M. Nagy.  Using a single-item approach to measure facet job satisfaction , 2002 .

[40]  Larry Percy,et al.  A better advertising planning grid. , 1991 .

[41]  John R. Rossiter,et al.  The C-OAR-SE procedure for scale development in marketing , 2002 .

[42]  Barbara G. Dodd,et al.  Computerized Adaptive Testing With Polytomous Items , 1995 .

[43]  J. P. Wanous,et al.  Overall job satisfaction: how good are single-item measures? , 1997, The Journal of applied psychology.

[44]  A. A. Mitchell,et al.  The Effect of Verbal and Visual Components of Advertisements on Brand Attitudes and Attitude Toward the Advertisement , 1986 .

[45]  Lars Bergkvist,et al.  Appropriate use of single-item measures is here to stay , 2015 .

[46]  Scott B. MacKenzie,et al.  An Empirical Examination of the Structural Antecedents of Attitude toward the Ad in an Advertising Pretesting Context , 1989 .

[47]  E. Muraki.  Information Functions of the Generalized Partial Credit Model , 1993 .

[48]  C. Fornell,et al.  Evaluating structural equation models with unobservable variables and measurement error. , 1981 .