The Visual Analogue Scale for Rating, Ranking and Paired-Comparison (VAS-RRP): A new technique for psychological measurement

Traditionally, the visual analogue scale (VAS) has been proposed to overcome the limitations of ordinal measures from Likert-type scales. However, the function of VASs to overcome the limitations of response styles to Likert-type scales has not yet been addressed. Previous research using ranking and paired comparisons to compensate for the response styles of Likert-type scales has suffered from limitations, such as that the total score of ipsative measures is a constant that cannot be analyzed by means of many common statistical techniques. In this study we propose a new scale, called the Visual Analogue Scale for Rating, Ranking, and Paired-Comparison (VAS-RRP), which can be used to collect rating, ranking, and paired-comparison data simultaneously, while avoiding the limitations of each of these data collection methods. The characteristics, use, and analytic method of VAS-RRPs, as well as how they overcome the disadvantages of Likert-type scales, ranking, and VASs, are discussed. On the basis of analyses of simulated and empirical data, this study showed that VAS-RRPs improved reliability, response style bias, and parameter recovery. Finally, we have also designed a VAS-RRP Generator for researchers’ construction and administration of their own VAS-RRPs.

[1]  Herbert W. Marsh,et al.  Latent variable models of multitrait-multimethod data. , 1995 .

[2]  Michael R. Harwell,et al.  Rescaling Ordinal Data to Interval Data in Educational Research , 2001 .

[3]  Isabella C. M. Cunningham,et al.  The Ipsative Process to Reduce Response Set Bias , 1977 .

[4]  E. Greenleaf,et al.  MEASURING EXTREME RESPONSE STYLE , 1992 .

[5]  Peter M. Bentler,et al.  Covariance structure analysis of ordinal ipsative data , 1998 .

[6]  Chien-Ho Wu.  An Empirical Study on the Transformation of Likert-scale Data to Numerical Scores , 2007 .

[7]  Y. Sung,et al.  Identifying the Career-Interest Profiles of Junior-High-School Students Through Latent Profile Analysis , 2017, The Journal of psychology.

[8]  J. Piaget,et al.  The Kolb Learning Style Inventory — Version 3 . 1 2005 Technical Specifi cations , 2005 .

[9]  Ulf-Dietrich Reips,et al.  Why Semantic Differentials in Web-Based Research Should Be Made from Visual Analogue Scales and Not from 5-Point Scales , 2012 .

[10]  Alberto Maydeu-Olivares,et al.  Item Response Modeling of Forced-Choice Questionnaires , 2011 .

[11]  Gina J. Medsker,et al.  Development of the O*NET Paper-and-Pencil Work Importance Locator , 1999 .

[12]  Klaas Sijtsma,et al.  Playing with Data—Or How to Discourage Questionable Research Practices and Stimulate Researchers to Do Things Right , 2016, Psychometrika.

[13]  Shantanu Dutta,et al.  Does a single response category in a scale completely capture a response , 1996 .

[14]  P. Costa,et al.  NEO inventories for the NEO Personality Inventory-3 (NEO-PI-3), NEO Five-Factor Inventory-3 (NEO-FFI-3), NEO Personality Inventory-Revised (NEO PI-R) : professional manual , 2010 .

[15]  Eli P. Cox,et al.  The Optimal Number of Response Alternatives for a Scale: A Review , 1980 .

[16]  K. Glenk,et al.  Validity and Reliability , 2008, Environmental Valuation with Discrete Choice Experiments.

[17]  H. Marsh,et al.  Confirmatory Factor Analyses of Multitrait-Multimethod Data: A Comparison of Alternative Models , 1991 .

[18]  Bruce Thompson,et al.  Score Reliability in Webor Internet-Based Surveys: Unnumbered Graphic Rating Scales versus Likert-Type Scales , 2001 .

[19]  Paul van Schaik,et al.  A Comparison of Multi-Item Likert and Visual Analogue Scales for the Assessment of Transactionally Defined Coping Function , 2004 .

[20]  Duane F. Alwin,et al.  The Factor Analysis of Ipsative Measures , 1980 .

[21]  A. Colman,et al.  Optimal number of response categories in rating scales: reliability, validity, discriminating power, and respondent preferences. , 2000, Acta psychologica.

[22]  G. Guyatt,et al.  A comparison of Likert and visual analogue scales for measuring change in function. , 1987, Journal of chronic diseases.

[23]  Albert Maydeu-Olivares,et al.  Fitting a Thurstonian IRT model to forced-choice data using Mplus , 2012, Behavior Research Methods.

[24]  Carl J. Chimi,et al.  The Likert Scale : A Proposal for Improvement Using Quasi-Continuous Variables , 2009 .

[25]  Paul S. Myles,et al.  The pain visual analog scale: is it linear or nonlinear? , 1999, Anesthesia and analgesia.

[26]  R. Likert.  “Technique for the Measurement of Attitudes, A” , 2022, The SAGE Encyclopedia of Research Design.

[27]  Rohana Yusoff,et al.  Generation of an Interval Metric Scale to Measure Attitude , 2014 .

[28]  E. Carmines,et al.  Analyzing models with unobserved variables: analysis of covariance structures , 1981 .

[29]  N. Reynolds,et al.  Prototype Development, Usability, and Preference of a Culturally-relevant Pictorial Aid to Facilitate Comprehension of Likert-type Levels of Agreement in Caregivers of Children Living With HIV in Ghana. , 2019, Computers, informatics, nursing : CIN.

[30]  Michael R. Mullen,et al.  Structural equation modelling: guidelines for determining model fit , 2008 .

[31]  J. Scott Granberg-Rademacker,et al.  An Algorithm for Converting Ordinal Scale Measurement Data to Interval/Ratio Scale , 2010 .

[32]  S. S. Stevens.  Mathematics, measurement, and psychophysics. , 1951 .

[33]  G. Albaum.  The Likert scale revisited: an alternate version , 1997 .

[34]  G. Menon,et al.  Describing the Development of a Health State Valuation Protocol to Obtain Community-Derived Disability Weights , 2019, Front. Public Health.

[35]  Jamal Munshi,et al.  A Method for Constructing Likert Scales , 2014 .

[36]  Birk Diedenhofen,et al.  cocron: A Web Interface and R Package for the Statistical Comparison of Cronbach’s Alpha Coefficients , 2017 .

[37]  Diana Adler,et al.  Using Multivariate Statistics , 2016 .

[38]  Duane F. Alwin,et al.  INFORMATION TRANSMISSION IN THE SURVEY INTERVIEW: NUMBER OF RESPONSE CATEGORIES AND THE RELIABILITY OF ATTITUDE MEASUREMENT , 1992 .

[39]  Ulf-Dietrich Reips,et al.  Investigating measurement equivalence of visual analogue scales and Likert-type scales in Internet-based personality questionnaires , 2017, Behavior Research Methods.

[40]  Gina J. Medsker,et al.  Development of the O*NET ™ Computerized Work Importance Profiler , 1999 .

[41]  D. Price,et al.  The validation of visual analogue scales as ratio scale measures for chronic and experimental pain , 1983, Pain.

[42]  R. Bagozzi,et al.  On the evaluation of structural equation models , 1988 .

[43]  John P. Robinson,et al.  Measures Of Personality And Social Psychological Attitudes , 1991 .

[44]  Alberto Maydeu-Olivares,et al.  How IRT can solve problems of ipsative data in forced-choice questionnaires. , 2013, Psychological methods.

[45]  William Vance Clemans,et al.  An analytical and empirical examination of some properties of ipsative measures , 1967 .

[46]  Roger Tourangeau,et al.  Evaluating the Effectiveness of Visual Analog Scales : A Web Experiment , 2006 .

[47]  J. L. Holland,et al.  Making vocational choices : a theory of vocational personalities and work environments , 1984 .

[48]  H. Adami,et al.  Self-rating of pain in nonulcer dyspepsia. A methodological study comparing a new fixed-point scale and the visual analogue scale. , 1987, Journal of clinical gastroenterology.

[49]  Y. Sung,et al.  Constructing a Situation-Based Career Interest Assessment for Junior High School Students and Examining Their Interest Structure , 2016 .

[50]  D. Hibbs.  On analyzing the effects of policy interventions : Box-Jenkins and Box-Tiao vs. structural equation models , 1977 .

[51]  Annette Scherpenzeel,et al.  The Validity and Reliability of Survey Questions , 1997 .

[52]  R. Goffin,et al.  The Effect of Target Attractiveness and Rating Method on the Accuracy of Trait Ratings , 2011 .

[53]  P. Zimbardo,et al.  A Global Look at Time , 2014 .

[54]  W. R. Dillon,et al.  Inferring Latent Brand Dependencies , 2000 .

[55]  P. Ferrando,et al.  A Kernel Density Analysis of Continuous Typical-Response Scales , 2003 .

[56]  Roger Tourangeau,et al.  Evaluating the Effectiveness of Visual Analog Scales , 2006 .

[57]  P. Thijssen,et al.  ‘Content’ versus ‘style’: Acquiescence in student evaluation of teaching? , 2012 .

[58]  S. McKelvie,et al.  Graphic rating scales — How many categories? , 1978 .

[59]  Anna Brown.  Item Response Models for Forced-Choice Questionnaires: A Common Framework , 2014, Psychometrika.

[60]  E. Krieg.  Biases Induced by Coarse Measurement Scales , 1999 .

[61]  Donna M. Randall,et al.  The social desirability response bias in ethics research , 1991 .

[62]  H. Marsh.  Confirmatory Factor Analyses of Multitrait-Multimethod Data: Many Problems and a Few Solutions , 1989 .

[63]  Ulf-Dietrich Reips,et al.  Interval-level measurement with visual analogue scales in Internet-based research: VAS Generator , 2008, Behavior research methods.

[64]  G H Guyatt,et al.  A comparison of seven-point and visual analogue scales. Data from a randomized trial. , 1990, Controlled clinical trials.

[65]  S S Stevens,et al.  On the Theory of Scales of Measurement. , 1946, Science.

[66]  James M Olson,et al.  Is It All Relative? , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.

[67]  Emin Babakus,et al.  The Sensitivity of Confirmatory Maximum Likelihood Factor Analysis to Violations of Measurement Scale and Distributional Assumptions , 1987 .

[68]  G. Alliger,et al.  A Proposed Method to Combine Ranking and Graphic Rating in Performance Appraisal: The Quantitative Ranking Scale , 1990 .

[69]  J. Colliver.  Full‐curriculum interventions and small‐scale studies of transfer: implications for psychology‐type theory , 2004, Medical education.

[70]  Y. Sung,et al.  Development and Validation of a Work Values Scale for Assessing High School Students , 2019, European Journal of Psychological Assessment.

[71]  R. Dawis,et al.  Comparability of Multiple Rank Order and Paired Comparison Methods , 1978 .

[72]  N. Lowe,et al.  A critical review of visual analogue scales in the measurement of clinical phenomena. , 1990, Research in nursing & health.

[73]  K. Bollen,et al.  Pearson's R and Coarsely Categorized Measures , 1981 .

[74]  Helen Baron,et al.  Strengths and limitations of ipsative measurement , 1996 .

[75]  Mike W.-L. Cheung,et al.  Reducing Uniform Response Bias With Ipsative Measurement in Multiple-Group Confirmatory Factor Analysis , 2002 .

[76]  J. Rounds.  Manual for the Minnesota importance questionnaire : a measure of vocational needs and values , 1981 .

[77]  K. Widaman.  Hierarchically Nested Covariance Structure Models for Multitrait-Multimethod Data , 1985 .

[78]  A. Meade.  Psychometric problems and issues involved with creating and using ipsative measures for selection , 2004 .

[79]  Gerald Albaum,et al.  The Likert Scale Revisited , 1997 .

[80]  L. E. Hicks,et al.  Some properties of ipsative, normative, and forced-choice normative measures. , 1970 .

[81]  T. Raykov.  Estimation of Composite Reliability for Congeneric Measures , 1997 .

[82]  Robert Loo,et al.  Confirmatory factor analyses of Kolb's Learning Style Inventory (LSI-1985) , 1999 .

[83]  D. Paulhus.  Control of social desirability in personality inventories: Principal-factor deletion , 1981 .

[84]  Henry E. Brady.  Factor and ideal point analysis for interpersonally incomparable data , 1989 .

[85]  S. Jamieson.  Likert scales: how to (ab)use them , 2004, Medical education.

[86]  Ulf-Dietrich Reips,et al.  Erratum to: Investigating measurement equivalence of visual analogue scales and Likert-type scales in Internet-based personality questionnaires , 2017, Behavior Research Methods.

[87]  Bruno D. Zumbo,et al.  Coefficient Alpha as an Estimate of Test Reliability Under Violation of Two Assumptions , 1993 .