Mixing Interviews and Rasch Modeling: Demonstrating a Procedure Used to Develop an Instrument That Measures Trust

Developing psychometrically sound instruments can be difficult, especially if little is known about the constructs of interest. When constructs of interest are unclear, a mixed methods approach can be useful. Qualitative inquiry can be used to explore a construct’s meaning in a way that informs item writing and allows the strengths of one analysis method to compensate for the weaknesses of the other. Mixing method applications can be complex, however, there are few examples within the literature pertaining to the mix of interviews, Rasch modeling, and classical test theory. This article demonstrates how to mix qualitative inquiry with Rasch modeling (and classical test theory) in order to develop an instrument that measures a complex construct: patient trust.

[1]  Joan Newman,et al.  Illustrating a Mixed-Method Approach for Validating Culturally Specific Constructs , 2005 .

[2]  B. Nastasi,et al.  Contributions of qualitative research to the validity of intervention research , 2005 .

[3]  Individual Differences in Rater Decision-Making Style: An Exploratory Mixed-Methods Study , 2012 .

[4]  José-Luis Padilla,et al.  Analysis of Nonequivalent Assessments Across Different Linguistic Groups Using a Mixed Methods Approach , 2014 .

[5]  David Kaplan,et al.  The Sage handbook of quantitative methodology for the social sciences , 2004 .

[6]  John H. Hitchcock,et al.  Mixed Methods in Intervention Research: Theory to Adaptation , 2007 .

[7]  J M Linacre,et al.  Investigating rating scale category utility. , 1999, Journal of outcome measurement.

[8]  Robert F. Dedrick,et al.  Development of the Trust in Physician Scale: A Measure to Assess Interpersonal Trust in Patient-Physician Relationships , 1990, Psychological reports.

[9]  A. Strauss,et al.  The discovery of grounded theory: strategies for qualitative research aldine de gruyter , 1968 .

[10]  Anthony J. Onwuegbuzie,et al.  Mixed Research as a Tool for Developing Quantitative Instruments , 2010 .

[11]  B P O'Connor,et al.  SPSS and SAS programs for determining the number of components using parallel analysis and Velicer’s MAP test , 2000, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[12]  Xitao Fan.  Item Response Theory and Classical Test Theory: An Empirical Comparison of their Item/Person Statistics , 1998 .

[13]  L. Radwin,et al.  Development and pilot testing of four desired health outcomes scales. , 2005, Oncology nursing forum.

[14]  L. Kirisci,et al.  Robustness of Item Parameter Estimation Programs to Assumptions of Unidimensionality and Normality , 2001 .

[15]  J. Linacre,et al.  Sample size and item calibration stability , 1994 .

[16]  Melvin R. Novick,et al.  Some latent train models and their use in inferring an examinee's ability , 1966 .

[17]  S H Kaplan,et al.  Patients' participation in medical care: effects on blood sugar control and quality of life in diabetes. , 1988, Journal of general internal medicine.

[18]  I. Newman.  Qualitative-Quantitative Research Methodology : Exploring the Interactive Continuum , 1999 .

[19]  Russell T Warne,et al.  An introduction to item response theory for health behavior researchers. , 2012, American journal of health behavior.

[20]  S. R. Lavinghouze.  Practical Program Evaluation: Assessing and Improving Planning, Implementation, and Effectiveness , 2005, Preventing Chronic Disease.

[21]  David A. Harrison,et al.  Robustness of Irt Parameter Estimation to Violations of The Unidimensionality Assumption , 1986 .

[22]  Yong-Won Lee.  Examining passage-related local item dependence (LID) and measurement construct using Q3 statistics in an EFL reading comprehension test , 2004 .

[23]  L. Cronbach.  Coefficient alpha and the internal structure of tests , 1951 .

[24]  D. Dillman.  Mail and telephone surveys : the total design method , 1979 .

[25]  L. Greenberg,et al.  The psychotherapeutic process: A research handbook. , 1986 .

[26]  J. Horn.  A rationale and test for the number of factors in factor analysis , 1965, Psychometrika.

[27]  J. Kramer.  Using Mixed Methods to Establish the Social Validity of a Self-Report Assessment: An Illustration Using the Child Occupational Self-Assessment (COSA) , 2011 .

[28]  Isadore Newman,et al.  Content Validity Using a Mixed Methods Approach , 2013 .

[29]  J. Frich,et al.  [Qualitative studies]. , 2007, Tidsskrift for den Norske laegeforening : tidsskrift for praktisk medicin, ny raekke.

[30]  D E Weisburd,et al.  What about the patient? , 1999, The Journal of clinical psychiatry.

[31]  John M Linacre,et al.  Optimizing rating scale category effectiveness. , 2002, Journal of applied measurement.

[32]  Duane T. Wegener,et al.  Evaluating the use of exploratory factor analysis in psychological research. , 1999 .

[33]  Samuel DiGangi,et al.  Assessing unidimensionality: A comparison of Rasch modeling, Parallel analysis, and TETRAD , 2007 .

[34]  J. Parker,et al.  Qualitative Research in Physical Activity and the Health Professions , 2009 .

[35]  W. Applegate,et al.  A randomized trial comparing aerobic exercise and resistance exercise with a health education program in older adults with knee osteoarthritis. The Fitness Arthritis and Seniors Trial (FAST). , 1997, JAMA.

[36]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[37]  J. Kulik,et al.  Preferences for health care involvement, perceived control and surgical recovery: a prospective study. , 1990, Social science & medicine.

[38]  R. Hambleton,et al.  Fundamentals of Item Response Theory , 1991 .

[39]  RON D. HAYS,et al.  Item Response Theory and Health Outcomes Measurement in the 21st Century , 2000, Medical care.

[40]  C. Brodsky.  The Discovery of Grounded Theory: Strategies for Qualitative Research , 1968 .

[41]  D. Gill,et al.  Development of a measure of rehabilitation adherence for athletic training. , 2010, Journal of sport rehabilitation.

[42]  Laura Johnson,et al.  How Many Interviews Are Enough? , 2006 .

[43]  G. Derscheid,et al.  A statistical analysis to characterize treatment adherence of the 18 most common diagnoses seen at a sports medicine clinic. , 1987, The Journal of orthopaedic and sports physical therapy.

[44]  N. Leech,et al.  Qualitative Data Analysis: A Compendium of Techniques and a Framework for Selection for School Psychology Research and Beyond. , 2008 .

[45]  M. Sheelagh T. Carpendale,et al.  Analyzing Qualitative Data , 2017, ISS.

[46]  J. Fries,et al.  The Patient-Reported Outcomes Measurement Information System (PROMIS): Progress of an NIH Roadmap Cooperative Group During its First Two Years , 2007, Medical care.

[47]  L. Luborsky,et al.  The Penn Helping Alliance Scales. , 1986 .

[48]  L. Podlog,et al.  Certified athletic trainers' perspectives on rehabilitation adherence in collegiate athletic training settings. , 2014, Journal of sport rehabilitation.

[49]  Robert T. Jiménez,et al.  Qualitative Studies in Special Education , 2005 .

[50]  E. McColl.  Cognitive Interviewing. A Tool for Improving Questionnaire Design , 2006, Quality of Life Research.

[51]  B. Chaney,et al.  Factor retention in EFA: strategies for health behavior researchers. , 2009, American journal of health behavior.

[52]  A. Onwuegbuzie,et al.  Mixed Methods Research: A Research Paradigm Whose Time Has Come , 2004 .

[53]  B. Brewer,et al.  Preliminary psychometric evaluation of a measure of adherence to clinic-based sport injury rehabilitation☆☆☆★★★☆☆☆ , 2000 .

[54]  D. Eignor.  The standards for educational and psychological testing. , 2013 .

[55]  M. Arvinen-Barrow,et al.  Psychosocial aspects of athletic injuries as perceived by athletic trainers. , 2013, Journal of athletic training.

[56]  Huey-Tsyh Chen,et al.  Practical program evaluation : assessing and improving planning, implementation, and effectiveness , 2005 .

[57]  John W. Creswell,et al.  Designing and Conducting Mixed Methods Research , 2006 .

[58]  Daniel J Buysse,et al.  The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005-2008. , 2010, Journal of clinical epidemiology.

[59]  C. Rees.  Quantitative and qualitative approaches to research , 1996 .

[60]  Wen-Chung Wang,et al.  Exploring Local Item Dependence Using a Random-Effects Facet Model , 2005 .

[61]  Jason W. Osborne,et al.  Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. , 2005 .

[62]  Carlo P. Magno.  Demonstrating the Difference between Classical Test Theory and Item Response Theory Using Derived Test Data , 2009 .

[63]  J. Millman,et al.  The specification and development of tests of achievement and ability. , 1989 .

[64]  Colin Sharp.  Qualitative Research and Evaluation Methods (3rd ed.) , 2003 .