Introduction to bifactor polytomous item response theory analysis.

A bifactor item response theory model can be used to aid in the interpretation of the dimensionality of a multifaceted questionnaire that assumes continuous latent variables underlying the propensity to respond to items. This model can be used to describe the locations of people on a general continuous latent variable as well as on continuous orthogonal specific traits that characterize responses to groups of items. The bifactor graded response (bifac-GR) model is presented in contrast to a correlated traits (or multidimensional GR model) and unidimensional GR model. Bifac-GR model specification, assumptions, estimation, and interpretation are demonstrated with a reanalysis of data (Campbell, 2008) on the Shared Activities Questionnaire. We also show the importance of marginalizing the slopes for interpretation purposes and we extend the concept to the interpretation of the information function. To go along with the illustrative example analyses, we have made available supplementary files that include command file (syntax) examples and outputs from flexMIRT, IRTPRO, R, Mplus, and STATA. Supplementary data to this article can be found online at http://dx.doi.org/10.1016/j.jsp.2016.11.001. Data needed to reproduce analyses in this article are available as supplemental materials (online only) in the Appendix of this article.

[1]  J. Lakin,et al.  Distinguishing verbal, quantitative, and figural facets of fluid intelligence in young students , 2012 .

[2]  James J. Appleton,et al.  Measuring cognitive and psychological engagement: Validation of the Student Engagement Instrument , 2006 .

[3]  Yanyan Sheng,et al.  BAYESIAN IRT MODELS INCORPORATING GENERAL AND SPECIFIC ABILITIES , 2009 .

[4]  Li Cai,et al.  Metropolis-Hastings Robbins-Monro Algorithm for Confirmatory Item Factor Analysis , 2010 .

[5]  Pui‐wa Lei,et al.  Confirmatory factor analysis of the Early Arithmetic, Reading, and Learning Indicators (EARLI). , 2014, Journal of school psychology.

[6]  K. Holzinger,et al.  The Bi-factor method , 1937 .

[7]  Gary L. Canivez,et al.  Bifactor modeling in construct validation of multifactored tests: Implications for understanding multidimensional constructs and test interpretation. , 2016 .

[8]  S. West,et al.  A Comparison of Bifactor and Second-Order Models of Quality of Life , 2006, Multivariate behavioral research.

[9]  M. Edelen,et al.  The psychometric development and initial validation of the DCI-A short form for adolescent therapeutic community treatment process. , 2014, Journal of substance abuse treatment.

[10]  van der Ark,et al.  Mokken Scale Analysis in R , 2007 .

[11]  Tony D. Thompson,et al.  Relating Unidimensional IRT Parameters to a Multidimensional Response Space: A Review of Two Alternative Projection IRT Models for Scoring Subscales , 2011 .

[12]  M. Matteucci,et al.  Multidimensional IRT Models to Analyze Learning Outcomes of Italian Students at the End of Lower Secondary School , 2015 .

[13]  Gautam Puhan,et al.  Reporting subscores for institutions. , 2009, The British journal of mathematical and statistical psychology.

[14]  Richard E. Mayer,et al.  Intelligence and education , 2004 .

[15]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[16]  Jonathan M. Campbell.  Brief Report: Reliability and Validity of the Shared Activities Questionnaire as a Measure of Middle School Students’ Attitudes toward Autism , 2008, Journal of autism and developmental disorders.

[17]  D. Thissen,et al.  Using Logistic Approximations of Marginal Trace Lines to Develop Short Assessments , 2013 .

[18]  M. Reckase.  Multidimensional Item Response Theory , 2009 .

[19]  J. Fantuzzo,et al.  Measuring preschool cognitive growth while it's still happening: the Learning Express. , 2009, Journal of school psychology.

[20]  Dai Feng,et al.  Computing and Displaying Isosurfaces in R , 2008 .

[21]  Ian J. Deary,et al.  Simple information processing and intelligence , 2000 .

[22]  R. Philip Chalmers,et al.  mirt: A Multidimensional Item Response Theory Package for the R Environment , 2012 .

[23]  Jos M. F. ten Berge,et al.  The greatest lower bound to the reliability of a test and the hypothesis of unidimensionality , 2004 .

[24]  R. Meijer,et al.  Analyzing psychopathology items: a case for nonparametric item response theory modeling. , 2004, Psychological methods.

[25]  Shelby J. Haberman,et al.  Reporting of Subscores Using Multidimensional Item Response Theory , 2010 .

[26]  Wesley A. Sims,et al.  Confirmation of models for interpretation and use of the Social and Academic Behavior Risk Screener (SABRS). , 2015, School psychology quarterly : the official journal of the Division of School Psychology, American Psychological Association.

[27]  Jonathan M. Campbell,et al.  Peers’ Knowledge About and Attitudes Towards Students with Autism Spectrum Disorders , 2014 .

[28]  J. Gustafsson,et al.  General and Specific Abilities as Predictors of School Achievement. , 1993, Multivariate behavioral research.

[29]  H. Goldstein,et al.  The Graphical Presentation of a Collection of Means , 1995 .

[30]  Robert D Gibbons,et al.  On the psychometric validity of the domains of the PDSQ: an illustration of the bi-factor item response theory model. , 2009, Journal of psychiatric research.

[31]  D. Thissen,et al.  Likelihood-Based Item-Fit Indices for Dichotomous Item Response Theory Models , 2000 .

[32]  Marley W. Watkins,et al.  Structure of the Wechsler Intelligence Scale for Children--Fourth Edition among a national sample of referred students. , 2010, Psychological assessment.

[33]  S. Reise.  The Rediscovery of Bifactor Measurement Models , 2012 .

[34]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[35]  E. Ip.  Interpretation of the Three-Parameter Testlet Response Model and Information Function , 2010 .

[36]  P. Boeck,et al.  Explanatory item response models : a generalized linear and nonlinear approach , 2004 .

[37]  Gilles E. Gignac.  Higher-order models versus direct hierarchical models: g as superordinate or breadth factor? , 2008 .

[38]  Christine E. DeMars.  A Tutorial on Interpreting Bifactor Model Scores , 2013 .

[39]  S. West,et al.  Effect of retention in elementary grades on grade 9 motivation for educational attainment. , 2015, Journal of school psychology.

[40]  Marley W. Watkins,et al.  Bifactor structure of the Wechsler Preschool and Primary Scale of Intelligence--Fourth Edition. , 2014, School psychology quarterly : the official journal of the Division of School Psychology, American Psychological Association.

[41]  Robert J. Mokken,et al.  A Theory and Procedure of Scale Analysis. , 1973 .

[42]  A. Murray,et al.  Do the Autism Spectrum Quotient (AQ) and Autism Spectrum Quotient Short Form (AQ-S) Primarily Reflect General ASD Traits or Specific ASD Traits? A Bi-Factor Analysis , 2017, Assessment.

[43]  Howard Wainer,et al.  Using a New Statistical Model for Testlets to Score TOEFL , 2000 .

[44]  Melvin R. Novick,et al.  Some latent train models and their use in inferring an examinee's ability , 1966 .

[45]  Mitchell K. Byrne,et al.  The Effects of a Multi-Component Higher-Functioning Autism Anti-Stigma Program on Adolescent Boys , 2013, Journal of autism and developmental disorders.

[46]  Nisha C. Gottfredson,et al.  An item factor analysis and item response theory-based revision of the Everyday Discrimination Scale. , 2011, Cultural diversity & ethnic minority psychology.

[47]  Anna Brown.  Item Response Theory approaches to test scoring and evaluating the score accuracy , 2018 .

[48]  David R. Anderson,et al.  Model Based Inference in the Life Sciences: A Primer on Evidence , 1998 .

[49]  Klaas Sijtsma,et al.  A Taxonomy of IRT Models for Ordering Persons and Items Using Simple Sum Scores , 2000 .

[50]  R. D. Bock,et al.  Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm , 1981 .

[51]  S. Reise,et al.  Bifactor Models and Rotations: Exploring the Extent to Which Multidimensional Data Yield Univocal Scale Scores , 2010, Journal of personality assessment.

[52]  J. Carroll.  Human Cognitive Abilities-a sur-vey of factor-analytic studies , 1993 .

[53]  Li Cai,et al.  Generalized full-information item bifactor analysis. , 2011, Psychological methods.

[54]  Klaas Sijtsma,et al.  Introduction to Nonparametric Item Response Theory , 2002 .

[55]  S. Reise,et al.  Scoring and Modeling Psychological Measures in the Presence of Multidimensionality , 2013, Journal of personality assessment.

[56]  Steven P. Reise,et al.  The role of the bifactor model in resolving dimensionality issues in health outcomes measures , 2007, Quality of Life Research.

[57]  De Ayala,et al.  The Theory and Practice of Item Response Theory , 2008 .

[58]  Kristen Bottema‐Beutel,et al.  Adolescent Judgments and Reasoning About the Failure to Include Peers with Social Disabilities , 2015, Journal of autism and developmental disorders.

[59]  Maria Orlando,et al.  Further Investigation of the Performance of S - X2: An Item Fit Index for Use With Dichotomous Item Response Theory Models , 2003 .

[60]  P. Boeck,et al.  Modeling local item dependencies in item response theory , 1998 .

[61]  K. Widaman,et al.  A National Study of Youth Attitudes toward the Inclusion of Students with Intellectual Disabilities , 2007 .

[62]  Michael D. Toland.  Practical Guide to Conducting an Item Response Theory Analysis , 2014 .

[63]  E. Turiel,et al.  To include or not to include: Evaluations and reasoning about the failure to include peers with autism spectrum disorder in elementary students , 2017, Autism : the international journal of research and practice.

[64]  Christine E. DeMars.  Application of the Bi-Factor Multidimensional Item Response Theory Model to Testlet-Based Tests. , 2006 .

[65]  Shelby J. Haberman,et al.  When Can Subscores Have Value? , 2008 .

[66]  D. Brouwer,et al.  On the factor structure of the Beck Depression Inventory-II: G is the key. , 2013, Psychological assessment.

[67]  R. P. McDonald,et al.  Test Theory: A Unified Treatment , 1999 .

[68]  Albert Maydeu-Olivares,et al.  Limited- and Full-Information Estimation and Goodness-of-Fit Testing in 2n Contingency Tables , 2005 .

[69]  Gilles E. Gignac,et al.  Bifactor Modeling and the Estimation of Model-Based Reliability in the WAIS-IV , 2013, Multivariate behavioral research.

[70]  I. Ajzen.  The theory of planned behavior , 1991 .

[71]  Yanyan Sheng,et al.  Bayesian Estimation of MIRT Models with General and Specific Latent Traits in MATLAB , 2010 .

[72]  F. Lord.  An approach to mental test theory , 1959 .

[73]  S. Dunsmuir,et al.  The Inclusion of Children with ASD: Using the Theory of Planned Behaviour as a Theoretical Framework to Explore Peer Attitudes , 2015 .

[74]  J. Immekus,et al.  Dimensionality Assessment Using the Full-Information Item Bifactor Analysis for Graded Response Data , 2008 .

[75]  Representation of Competencies in Multidimensional IRT Models with Within-Item and Between-Item Multidimensionality , 2008 .

[76]  Fang Fang Chen,et al.  Delaware School Climate Survey-Student: its factor structure, concurrent validity, and reliability. , 2011, Journal of school psychology.

[77]  Tanya N. Beran,et al.  Development of the Bullying and Health Experiences Scale , 2012, Interactive journal of medical research.

[78]  F. Worrell,et al.  Back to “the Future”: Evidence of a Bifactor Solution for Scores on the Consideration of Future Consequences Scale , 2015, Journal of personality assessment.

[79]  Margit Wiesner,et al.  Exploratory structural equation modeling, bifactor models, and standard confirmatory factor analysis models: application to the BASC-2 Behavioral and Emotional Screening System Teacher Form. , 2013, Journal of school psychology.

[80]  F. Samejima.  Estimation of latent ability using a response pattern of graded scores , 1968 .

[81]  M. Edelen,et al.  Using Hierarchical IRT Models to Create Unidimensional Measures From Multidimensional Data , 2014 .

[82]  Donald Hedeker,et al.  Full-Information Item Bifactor Analysis of Graded Response Data , 2007 .

[83]  Frank Rijmen,et al.  Formal Relations and an Empirical Comparison among the Bi‐Factor, the Testlet, and a Second‐Order Multidimensional IRT Model , 2010 .

[84]  Lynne Steinberg,et al.  New tricks for an old measure: the development of the Barratt Impulsiveness Scale-Brief (BIS-Brief). , 2013, Psychological assessment.

[85]  Knowledge of Autism and Attitudes of Children Towards Their Partially Integrated Peers with Autism Spectrum Disorders , 2014, Journal of autism and developmental disorders.

[86]  Gautam Puhan,et al.  Subscores Based on Classical Test Theory: To Report or Not to Report , 2007 .

[87]  Hally Quinn.  Bifactor Models, Explained Common Variance (ECV), and the Usefulness of Scores from Unidimensional Item Response Theory Analyses , 2014 .

[88]  Harry H. Harman,et al.  Comparison of two factorial analyses , 1938 .

[89]  Gilles E. Gignac.  The WAIS-III as a Nested Factors Model: A Useful Alternative to the More Conventional Oblique and Hi , 2006 .

[90]  Li Cai,et al.  Limited-information goodness-of-fit testing of hierarchical item factor models. , 2013, The British journal of mathematical and statistical psychology.

[91]  Lihua Yao.  Reporting Valid and Reliable Overall Scores and Domain Scores , 2010 .

[92]  Joseph Betts,et al.  Investigating Early Literacy and Numeracy: Exploring the Utility of the Bifactor Model. , 2011 .

[93]  D. Thissen,et al.  Local Dependence Indexes for Item Pairs Using Item Response Theory , 1997 .

[94]  A. Jones,et al.  Inclusive provision options for pupils on the autistic spectrum , 2010 .

[95]  Donald Hedeker,et al.  Full-information item bi-factor analysis , 1992 .

[96]  S. Embretson,et al.  Item response theory for psychologists , 2000 .

[97]  Gautam Puhan,et al.  The Utility of Augmented Subscores in a Licensure Exam: An Evaluation of Methods Using Empirical Data , 2010 .

[98]  Marley W. Watkins,et al.  Examining the validity of the Homework Performance Questionnaire: Multi-informant assessment in elementary and middle school. , 2015, School psychology quarterly : the official journal of the Division of School Psychology, American Psychological Association.