Item Response Theory -- A Statistical Framework for Educational and Psychological Measurement

Item response theory (IRT) has become one of the most popular statistical models for psychometrics, a field of study concerned with the theory and techniques of psychological measurement. The IRT models are latent factor models tailored to the analysis, interpretation, and prediction of individuals’ behaviors in answering a set of measurement items that typically involve categorical response data. Many important questions of measurement are directly or indirectly answered through the use of IRT models, including scoring individuals’ test performances, validating a test scale, linking two tests, among others. This paper provides a review of item response theory, including its statistical framework and psychometric applications. We establish connections between item response theory and related topics in statistics, including empirical Bayes, nonparametric methods, matrix completion, regularized estimation, and sequential analysis. Possible future directions of IRT are discussed from the perspective of statistical learning.

[1]  E. Muraki.  A GENERALIZED PARTIAL CREDIT MODEL: APPLICATION OF AN EM ALGORITHM , 1992 .

[2]  H. Kaiser.  The varimax criterion for analytic rotation in factor analysis , 1958 .

[3]  William Stout,et al.  A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF , 1993 .

[4]  Sonia A. Bhaskar,et al.  Probabilistic Low-Rank Matrix Completion from Quantized Measurements , 2016, J. Mach. Learn. Res..

[5]  M. Kosinski,et al.  A decade into Facebook: where is psychiatry in the digital age? , 2016, The lancet. Psychiatry.

[6]  A. Goldberger.  STRUCTURAL EQUATION METHODS IN THE SOCIAL SCIENCES , 1972 .

[7]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[8]  P. Fayers.  Item Response Theory for Psychologists , 2004, Quality of Life Research.

[9]  F. Kong,et al.  A stochastic approximation algorithm with Markov chain Monte-carlo method for incomplete data estimation problems. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[10]  David J. Bartholomew,et al.  Latent Variable Models and Factor Analysis: A Unified Approach , 2011 .

[11]  D. Thissen,et al.  Local Dependence Indexes for Item Pairs Using Item Response Theory , 1997 .

[12]  Robert Jennrich,et al.  Rotation to simple loadings using component loss functions: The orthogonal case , 2004 .

[13]  Ying Cheng.  When Cognitive Diagnosis Meets Computerized Adaptive Testing: CD-CAT , 2009 .

[14]  David J. Weiss,et al.  Improving Measurement Quality and Efficiency with Adaptive Testing , 1982 .

[15]  J. Wolfowitz,et al.  Optimum Character of the Sequential Probability Ratio Test , 1948 .

[16]  Neal M. Kingston,et al.  The Use of Learning Map Systems to Support the Formative Assessment in Mathematics , 2017 .

[17]  J. Mckillip,et al.  Fundamentals of item response theory , 1993 .

[18]  M. Petersen,et al.  Introduction to Nonparametric Item Response Theory , 2005, Quality of Life Research.

[19]  E. Ising.  Beitrag zur Theorie des Ferromagnetismus , 1925 .

[20]  P. Robinson,et al.  Identification, estimation and large-sample theory for regressions containing unobservable variables , 1974 .

[21]  Eric R. Ziegel,et al.  Generalized Linear Models , 2002, Technometrics.

[22]  Hua-Hua Chang,et al.  A Global Information Approach to Computerized Adaptive Testing , 1996 .

[23]  Stan Lipovetsky,et al.  Generalized Latent Variable Modeling: Multilevel,Longitudinal, and Structural Equation Models , 2005, Technometrics.

[24]  Steven Andrew Culpepper,et al.  Estimating the Cognitive Diagnosis $$\varvec{Q}$$Q Matrix with Expert Knowledge: Application to the Fraction-Subtraction Dataset , 2018, Psychometrika.

[25]  อนุสรณ์ เกิดศรี,et al.  Elements of Adaptive Testing , 2015 .

[26]  Eric T. Bradlow,et al.  A Bayesian random effects model for testlets , 1999 .

[27]  M. Knott,et al.  Generalized latent trait models , 2000 .

[28]  David J. Bartholomew,et al.  The Goodness of Fit of Latent Trait Models in Attitude Measurement , 1999 .

[29]  A. Zellner.  Estimation of Regression Relationships Containing Unobservable Independent Variables , 1970 .

[30]  Robert I. Jennrich,et al.  Rotation to Simple Loadings Using Component Loss Functions: The Oblique Case , 2006 .

[31]  Robert J. Mislevy,et al.  Estimation of Latent Group Effects , 1985 .

[32]  Jonathan Templin,et al.  Diagnostic Measurement: Theory, Methods, and Applications , 2010 .

[33]  Gongjun Xu,et al.  Identifiability of Diagnostic Classification Models , 2015, Psychometrika.

[34]  D. Lawley,et al.  XXIII.—On Problems connected with Item Selection and Test Construction , 1943, Proceedings of the Royal Society of Edinburgh. Section A. Mathematical and Physical Sciences.

[35]  Donald Hedeker,et al.  Full-information item bi-factor analysis , 1992 .

[36]  D. Lawley,et al.  X.—The Factorial Analysis of Multiple Item Tests , 1944, Proceedings of the Royal Society of Edinburgh. Section A. Mathematical and Physical Sciences.

[37]  Myrsini Katsikatsou,et al.  Pairwise likelihood estimation for factor analysis models with ordinal data , 2012, Comput. Stat. Data Anal..

[38]  B. Junker,et al.  Cognitive Assessment Models with Few Assumptions, and Connections with Nonparametric Item Response Theory , 2001 .

[39]  Matthias von Davier,et al.  A general diagnostic model applied to language testing data. , 2008, The British journal of mathematical and statistical psychology.

[40]  Peter M. Bentler,et al.  Structural equation models with continuous and polytomous variables , 1992 .

[41]  Hua-Hua Chang,et al.  Detecting DIF for Polytomously Scored Items: An Adaptation of the SIBTEST Procedure , 1995 .

[42]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[43]  David Kaplan,et al.  The Sage handbook of quantitative methodology for the social sciences , 2004 .

[44]  K. Tatsuoka.  RULE SPACE: AN APPROACH FOR DEALING WITH MISCONCEPTIONS BASED ON ITEM RESPONSE THEORY , 1983 .

[45]  Kai-Fu Lee.  AI Superpowers: China, Silicon Valley, and the New World Order , 2018 .

[46]  B. Muthén.  A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators , 1984 .

[47]  Malay Ghosh,et al.  Inconsistent maximum likelihood estimators for the Rasch model , 1995 .

[48]  M. Kosinski,et al.  Musical Preferences Predict Personality: Evidence From Active Listening and Facebook Likes , 2018, Psychological science.

[49]  Francis Tuerlinckx,et al.  Copula Functions for Residual Dependency , 2007 .

[50]  J. Kiefer,et al.  CONSISTENCY OF THE MAXIMUM LIKELIHOOD ESTIMATOR IN THE PRESENCE OF INFINITELY MANY INCIDENTAL PARAMETERS , 1956 .

[51]  Jay Bartroff,et al.  Sequential Experimentation in Clinical Trials: Design and Analysis , 2012 .

[52]  Chia-Yi Chiu.  Statistical Refinement of the Q-Matrix in Cognitive Diagnosis , 2013 .

[53]  Jingchen Liu,et al.  A Rate Function Approach to Computerized Adaptive Testing for Cognitive Diagnosis , 2015, Psychometrika.

[54]  Mark Von Tress,et al.  Generalized, Linear, and Mixed Models , 2003, Technometrics.

[55]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[56]  Melissa S. Yale,et al.  Differential Item Functioning , 2014 .

[57]  Melvin R. Novick,et al.  Some latent train models and their use in inferring an examinee's ability , 1966 .

[58]  Sandip Sinharay,et al.  Assessing Item Fit for Unidimensional Item Response Theory Models Using Residuals from Estimated Item Response Functions , 2013, Psychometrika.

[59]  Ratna Nandakumar,et al.  Refinements of Stout’s Procedure for Assessing Latent Trait Unidimensionality , 1993 .

[60]  Gongjun Xu,et al.  Identifiability of restricted latent class models with binary responses , 2016, 1603.04140.

[61]  Kenneth R. Koedinger,et al.  Learning Factors Analysis - A General Method for Cognitive Model Evaluation and Improvement , 2006, Intelligent Tutoring Systems.

[62]  I JordanMichael,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008 .

[63]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[64]  Robert J. Mislevy,et al.  A Consumer's Guide to LOGIST and BILOG , 1987 .

[65]  Gongjun Xu,et al.  Identifying Latent Structures in Restricted Latent Class Models , 2018, Journal of the American Statistical Association.

[66]  Duanli Yan,et al.  Computerized multistage testing : theory and applications , 2014 .

[67]  Walter T. Federer,et al.  Sequential Design of Experiments , 1967 .

[68]  Bengt Muthen,et al.  Some uses of structural equation modeling in validity studies: Extending IRT to external variables , 1986 .

[69]  Boris Polyak,et al.  Acceleration of stochastic approximation by averaging , 1992 .

[70]  Jingchen Liu,et al.  Theory of the Self-learning Q-Matrix. , 2010, Bernoulli : official journal of the Bernoulli Society for Mathematical Statistics and Probability.

[71]  Edward H. Ip,et al.  Stochastic EM: method and application , 1996 .

[72]  Peter M. Bentler,et al.  Exploratory Bi-Factor Analysis , 2011, Psychometrika.

[73]  Ratna Nandakumar,et al.  MULTISIB: A Procedure to Investigate DIF When a Test is Intentionally Two-Dimensional , 1997 .

[74]  Edward H. Ip,et al.  Locally dependent latent trait model for polytomous responses with application to inventory of hostility , 2004 .

[75]  Matthew N. O. Sadiku,et al.  General Intelligence , 2021, A Primer on Multiple Intelligences.

[76]  Thomas S. Ferguson,et al.  Sequential classification on partially ordered sets , 2003 .

[77]  Edward H. Ip,et al.  Locally dependent latent trait model and the dutch identity revisited , 2002 .

[78]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[79]  F. Samejima.  Estimation of latent ability using a response pattern of graded scores , 1968 .

[80]  J. Templin,et al.  Measurement of psychological disorders using cognitive diagnosis models. , 2006, Psychological methods.

[81]  Jean-Paul Fox,et al.  Using Item Response Theory to Measure Extreme Response Style in Marketing Research: A Global Investigation , 2008 .

[82]  Yunxiao Chen,et al.  Computation for Latent Variable Model Estimation: A Unified Stochastic Proximal Framework , 2020, Psychometrika.

[83]  Yunxiao Chen,et al.  Joint Maximum Likelihood Estimation for High-Dimensional Exploratory Item Factor Analysis , 2018, Psychometrika.

[84]  R. Nandakumar,et al.  Evaluation of the CATSIB DIF Procedure in a Pretest Setting , 2004 .

[85]  M. W. Richardson.  The relation between the difficulty and the differential validity of a test , 1936 .

[86]  Hua-Hua Chang,et al.  A Simulation Study to Compare CAT Strategies for Cognitive Diagnosis , 2003 .

[87]  Li Cai,et al.  Metropolis-Hastings Robbins-Monro Algorithm for Confirmatory Item Factor Analysis , 2010 .

[88]  H. Akaike.  A new look at the statistical model identification , 1974 .

[89]  Chandler Davis.  The rotation of eigenvectors by a perturbation , 1963 .

[90]  Zhi Wang,et al.  Latent Feature Extraction for Process Data via Multidimensional Scaling , 2019, Psychometrika.

[91]  Kunpeng Li,et al.  STATISTICAL ANALYSIS OF FACTOR MODELS OF HIGH DIMENSION , 2012, 1205.6617.

[92]  C. Spearman.  The proof and measurement of association between two things. , 2015, International journal of epidemiology.

[93]  L. Cronbach.  Coefficient alpha and the internal structure of tests , 1951 .

[94]  I. Moustaki,et al.  A Note on Likelihood Ratio Tests for Models with Latent Variables , 2020, Psychometrika.

[95]  H. Chernoff.  On the Distribution of the Likelihood Ratio , 1954 .

[96]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[97]  Li Cai,et al.  Generalized full-information item bifactor analysis. , 2011, Psychological methods.

[98]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[99]  Matthew S. Johnson,et al.  Modeling dichotomous item responses with free-knot splines , 2007, Comput. Stat. Data Anal..

[100]  Yunxiao Chen,et al.  Determining the Number of Factors in High-Dimensional Generalized Latent Factor Models , 2020, Biometrika.

[101]  K. Pearson.  On the Criterion that a Given System of Deviations from the Probable in the Case of a Correlated System of Variables is Such that it Can be Reasonably Supposed to have Arisen from Random Sampling , 1900 .

[102]  David J. Weiss,et al.  APPLICATION OF COMPUTERIZED ADAPTIVE TESTING TO EDUCATIONAL PROBLEMS , 1984 .

[103]  Edward H. Ip,et al.  On Single Versus Multiple Imputation for a Class of Stochastic Algorithms Estimating Maximum Likelihood , 2002, Comput. Stat..

[104]  D. Andrich.  Sufficiency and Conditional Estimation of Person Parameters in the Polytomous Rasch Model , 2010 .

[105]  Jingchen Liu,et al.  Subtask Analysis of Process Data Through a Predictive Model , 2020, ArXiv.

[106]  Cun-Hui Zhang,et al.  Compound decision theory and empirical bayes methods , 2003 .

[107]  P. Holland,et al.  Classical Test Theory as a first-order Item Response Theory: Application to true-score prediction from a possibly nonparallel test , 2003 .

[108]  Frederic M. Lord.  A BROAD-RANGE TAILORED TEST OF VERBAL ABILITY , 1975 .

[109]  Y. Chang,et al.  Application of Sequential Interval Estimation to Adaptive Mastery Testing , 2005 .

[110]  RON D. HAYS,et al.  Item Response Theory and Health Outcomes Measurement in the 21st Century , 2000, Medical care.

[111]  Robert J. Mokken,et al.  A Theory and Procedure of Scale Analysis. , 1973 .

[112]  J. Douglas.  Joint consistency of nonparametric item characteristic curve and ability estimation , 1997 .

[113]  Paul De Boeck,et al.  A parametric model for local dependence among test items. , 1997 .

[114]  R. D. Bock,et al.  Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm , 1981 .

[115]  M. Kosinski,et al.  Deep Neural Networks Are More Accurate Than Humans at Detecting Sexual Orientation From Facial Images , 2018, Journal of personality and social psychology.

[116]  Tue Tjur,et al.  A Connection between Rasch's Item Analysis Model and a Multiplicative Poisson Model , 1982 .

[117]  Shelby J. Haberman,et al.  Generalized Residuals for General Models for Contingency Tables With Application to Item Response Theory , 2013 .

[118]  R. Millsap,et al.  The SAGE Handbook of Quantitative Methods in Psychology , 2009 .

[119]  S. Sinharay,et al.  Nonparametric Item Response Curve Estimation With Correction for Measurement Error , 2011 .

[120]  Zhenke Wu,et al.  Partial Identifiability of Restricted Latent Class Models , 2018 .

[121]  Albert Maydeu-Olivares,et al.  Limited Information Goodness-of-fit Testing in Multidimensional Contingency Tables , 2005 .

[122]  Jingchen Liu,et al.  A reinforcement learning approach to personalized learning recommendation systems , 2018, The British journal of mathematical and statistical psychology.

[123]  Seonghoon Kim.  A Note on the Reliability Coefficients for Item Response Model-Based Ability Estimates , 2012 .

[124]  G. Masters.  A rasch model for partial credit scoring , 1982 .

[125]  Po-Hsien Huang,et al.  A penalized likelihood method for multi-group structural equation modelling. , 2018, The British journal of mathematical and statistical psychology.

[126]  J. D. L. Torre,et al.  The Generalized DINA Model Framework. , 2011 .

[127]  L. A. Goodman.  The Analysis of Systems of Qualitative Variables When Some of the Variables Are Unobservable. Part I-A Modified Latent Structure Approach , 1974, American Journal of Sociology.

[128]  Li Cai,et al.  HIGH-DIMENSIONAL EXPLORATORY ITEM FACTOR ANALYSIS BY A METROPOLIS–HASTINGS ROBBINS–MONRO ALGORITHM , 2010 .

[129]  Louis V. DiBello,et al.  A Kernel-Smoothed Version of SIBTEST With Applications to Local DIF Inference and Function Estimation , 1996 .

[130]  L. A. Goodman.  Exploratory latent structure analysis using both identifiable and unidentifiable models , 1974 .

[131]  Jingchen Liu,et al.  Recommendation System for Adaptive Learning , 2018, Applied psychological measurement.

[132]  Xiaotong Shen,et al.  Personalized Prediction and Sparsity Pursuit in Latent Factor Models , 2016 .

[133]  Jingchen Liu,et al.  Data-Driven Learning of Q-Matrix , 2012, Applied psychological measurement.

[134]  J. Horn.  A rationale and test for the number of factors in factor analysis , 1965, Psychometrika.

[135]  Elvezio Ronchetti,et al.  Estimation of generalized linear latent variable models , 2004 .

[136]  Z. Ying,et al.  Statistical Analysis of Q-Matrix Based Diagnostic Classification Models , 2015, Journal of the American Statistical Association.

[137]  Michael C. Edwards,et al.  A Markov Chain Monte Carlo Approach to Confirmatory Item Factor Analysis , 2010 .

[138]  F. Krauss.  Latent Structure Analysis , 1980 .

[139]  Edward H. Haertel.  Using restricted latent class models to map the skill structure of achievement items , 1989 .

[140]  Yunxiao Chen,et al.  Structured Latent Factor Analysis for Large-scale Data: Identifiability, Estimability, and Their Implications , 2017, Journal of the American Statistical Association.

[141]  Suzanne Winsberg,et al.  FITTING ITEM CHARACTERISTIC CURVES WITH SPLINE FUNCTIONS , 1984 .

[142]  Xiao-Li Meng,et al.  Fitting Full-Information Item Factor Models and an Empirical Investigation of Bridge Sampling , 1996 .

[143]  Dimitris Rizopoulos,et al.  Weighted pairwise likelihood estimation for a general class of random effects models. , 2014, Biostatistics.

[144]  Yunxiao Chen.  A Continuous-Time Dynamic Choice Measurement Model for Problem-Solving Process Data. , 2020, Psychometrika.

[145]  S. Fienberg.  Contingency Tables and Log-Linear Models: Basic Results and New Developments , 2000 .

[146]  Steven P Reise,et al.  Item response theory and clinical measurement. , 2009, Annual review of clinical psychology.

[147]  Chih-Hung Chang,et al.  Item response theory (IRT): Applications in quality of life measurement, analysis and interpretation , 2002 .

[148]  Bengt Muthén,et al.  A Method for Studying the Homogeneity of Test Items with Respect to Other Relevant Variables , 1985 .

[149]  Deniz Senturk-Doganaksoy,et al.  Explanatory Item Response Models: A Generalized Linear and Nonlinear Approach , 2006, Technometrics.

[150]  Noah Kaplan,et al.  Practical Issues in Implementing and Understanding Bayesian Ideal Point Estimation , 2005, Political Analysis.

[151]  Francis Tuerlinckx,et al.  Detection of Differential Item Functioning Using the Lasso Approach , 2015 .

[152]  Chia-Yi Chiu,et al.  Cluster Analysis for Cognitive Diagnosis: Theory and Applications , 2009 .

[153]  Norman Verhelst,et al.  Maximum Likelihood Estimation in Generalized Rasch Models , 1986 .

[154]  Howard Wainer,et al.  Computerized Adaptive Testing: A Primer , 2000 .

[155]  D. Cox,et al.  A note on pseudolikelihood constructed from marginal densities , 2004 .

[156]  R. Khan,et al.  Sequential Tests of Statistical Hypotheses. , 1972 .

[157]  John T. Willse,et al.  Defining a Family of Cognitive Diagnosis Models Using Log-Linear Models with Latent Variables , 2009 .

[158]  J. Ramsay,et al.  Maximum marginal likelihood estimation for semiparametric item analysis , 1991 .

[159]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[160]  M. Browne.  An Overview of Analytic Rotation in Exploratory Factor Analysis , 2001 .

[161]  Erling B. Andersen,et al.  Sufficient statistics and latent trait models , 1977 .

[162]  Z. Ying,et al.  Accurate Assessment via Process Data , 2021, Psychometrika.

[163]  Kristopher J Preacher,et al.  Item factor analysis: current approaches and future directions. , 2007, Psychological methods.

[164]  Yang Liu,et al.  Local Dependence Diagnostics in IRT Modeling of Binary Data , 2013 .

[165]  Michael C. Mozer,et al.  Integrating latent-factor and knowledge-tracing models to predict individual differences in learning , 2014, EDM.

[166]  J. Besag.  Spatial Interaction and the Statistical Analysis of Lattice Systems , 1974 .

[167]  Noel A Cressie,et al.  Characterizing the manifest probabilities of latent trait models , 1983 .

[168]  William Stout,et al.  The theoretical detect index of dimensionality and its application to approximate simple structure , 1999 .

[169]  Ewout van den Berg,et al.  1-Bit Matrix Completion , 2012, ArXiv.

[170]  Zhiliang Ying,et al.  Latent Variable Selection for Multidimensional Item Response Theory Models via $$L_{1}$$L1 Regularization , 2016 .

[171]  P M Bentler,et al.  A two-stage estimation of structural equation models with continuous and polytomous variables. , 1995, The British journal of mathematical and statistical psychology.

[172]  W. Stout,et al.  Improved Type I Error Control and Reduced Estimation Bias for DIF Detection Using SIBTEST , 1998 .

[173]  Dorothy T. Thayer,et al.  Differential Item Performance and the Mantel-Haenszel Procedure. , 1986 .

[174]  H. Chernoff.  Sequential Analysis and Optimal Design , 1987 .

[175]  W. Stout,et al.  A new procedure for detection of crossing DIF , 1996 .

[176]  H. Kaiser.  The Application of Electronic Computers to Factor Analysis , 1960 .

[177]  T. Lai.  SEQUENTIAL ANALYSIS: SOME CLASSICAL PROBLEMS AND NEW CHALLENGES , 2001 .

[178]  P. Wedin.  Perturbation bounds in connection with singular value decomposition , 1972 .

[179]  Steven Andrew Culpepper,et al.  Bayesian Estimation of the DINA Model With Gibbs Sampling , 2015 .

[180]  P L Fidler,et al.  Goodness-of-Fit Testing for Latent Class Models. , 1993, Multivariate behavioral research.

[181]  B. Efron,et al.  Stein's Estimation Rule and Its Competitors- An Empirical Bayes Approach , 1973 .

[182]  Z. Ying,et al.  Identifiability of Bifactor Models , 2020, 2012.12196.

[183]  M. M. Meyer,et al.  Loglinear models and categorical data analysis with psychometric and econometric applications , 1983 .

[184]  H. Robbins,et al.  Adaptive Design and Stochastic Approximation , 1979 .

[185]  Silvia Cagnone,et al.  A Composite Likelihood Inference in Latent Variable Models for Ordinal Longitudinal Responses , 2012, Psychometrika.

[186]  H. Robbins.  An Empirical Bayes Approach to Statistics , 1956 .

[187]  Hua-Hua Chang,et al.  From smart testing to smart learning: how testing technology can assist the new generation of education , 2016 .

[188]  C. Mitchell Dayton,et al.  The Use of Probabilistic Models in the Assessment of Mastery , 1977 .

[189]  David Watson,et al.  The Hierarchical Taxonomy of Psychopathology (HiTOP): A Dimensional Alternative to Traditional Nosologies , 2017, Journal of abnormal psychology.

[190]  Yu He,et al.  Statistical Significance of the Netflix Challenge , 2012, 1207.5649.

[191]  Peter Brusilovsky,et al.  Integrating Knowledge Tracing and Item Response Theory: A Tale of Two Frameworks , 2014, UMAP Workshops.

[192]  J. Neyman,et al.  Consistent Estimates Based on Partially Consistent Observations , 1948 .

[193]  William Stout,et al.  A nonparametric approach for assessing latent trait unidimensionality , 1987 .

[194]  H. Joe,et al.  Limited-and Full-Information Estimation and Goodness-ofFit Testing in 2 n Contingency Tables : A Unified Framework , 2005 .

[195]  Cun-Hui Zhang.  Nearly unbiased variable selection under minimax concave penalty , 2010, 1002.4734.

[196]  K. Jöreskog,et al.  Factor Analysis of Ordinal Variables: A Comparison of Three Approaches , 2001, Multivariate behavioral research.

[197]  Chia-Yi Chiu,et al.  A General Method of Empirical Q-matrix Validation , 2016, Psychometrika.

[198]  Adel Javanmard,et al.  1-bit matrix completion under exact low-rank constraint , 2015, 2015 49th Annual Conference on Information Sciences and Systems (CISS).

[199]  Zhiliang Ying,et al.  An Exploratory Analysis of the Latent Structure of Process Data via Action Sequence Autoencoder , 2019, The British journal of mathematical and statistical psychology.

[200]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[201]  Pao-Kuei Wu,et al.  MISSING RESPONSES AND IRT ABILITY ESTIMATION: OMITS, CHOICE, TIME LIMITS, AND ADAPTIVE TESTING , 1996 .

[202]  Mark Reiser,et al.  Analysis of residuals for the multionmial item response model , 1996 .

[203]  Zhehan Jiang,et al.  Gibbs Samplers for Logistic Item Response Models via the Pólya–Gamma Distribution: A Computationally Efficient Data-Augmentation Strategy , 2018, Psychometrika.

[204]  S. Mulaik.  Foundations of Factor Analysis , 1975 .

[205]  Yan Yang,et al.  Tracking Skill Acquisition With Cognitive Diagnosis Models: A Higher-Order, Hidden Markov Model With Covariates , 2018 .

[206]  W. Haenszel,et al.  Statistical aspects of the analysis of data from retrospective studies of disease. , 1959, Journal of the National Cancer Institute.

[207]  Peter M. Bentler,et al.  A three-stage estimation procedure for structural equation models with polytomous variables , 1990 .

[208]  Yang Liu,et al.  An improved stochastic EM algorithm for large-scale full-information item factor analysis. , 2018, The British journal of mathematical and statistical psychology.

[209]  Jeffrey A Douglas,et al.  Asymptotic identifiability of nonparametric item response models , 2001 .

[210]  E. B. Andersen,et al.  Asymptotic Properties of Conditional Maximum‐Likelihood Estimators , 1970 .

[211]  M. Drton.  Likelihood ratio tests and singularities , 2007, math/0703360.

[212]  Galit Shmueli,et al.  To Explain or To Predict? , 2010, 1101.0891.

[213]  Yuguo Chen,et al.  Bayesian Estimation of the DINA Q matrix , 2018, Psychometrika.

[214]  Jingchen Liu,et al.  Robust Measurement via A Fused Latent and Graphical Item Response Theory Model , 2018, Psychometrika.

[215]  William F. Strout.  A new item response theory modeling approach with applications to unidimensionality assessment and ability estimation , 1990 .

[216]  E. Walker,et al.  Diagnostic and Statistical Manual of Mental Disorders , 2013 .

[217]  F. Lord.  Applications of Item Response Theory To Practical Testing Problems , 1980 .

[218]  Frederic M. Lord.  ROBBINS‐MONRO PROCEDURES FOR TAILORED TESTING* , 1969 .

[219]  B. Muthén.  Contributions to factor analysis of dichotomous variables , 1978 .

[220]  Handbook of Diagnostic Classification Models , 2019, Methodology of Educational Measurement and Assessment.

[221]  Frederic M. Lord,et al.  An Analysis of the Verbal Scholastic Aptitude Test Using Birnbaum's Three-Parameter Logistic Model , 1968 .

[222]  R. Bennett,et al.  Advancing Human Assessment: A Synthesis Over Seven Decades , 2017 .

[223]  R. Darrell Bock,et al.  Estimating item parameters and latent ability when responses are scored in two or more nominal categories , 1972 .

[224]  Shelby J. Haberman,et al.  Maximum Likelihood Estimates in Exponential Response Models , 1977 .

[225]  Andreas Ritter,et al.  Structural Equations With Latent Variables , 2016 .

[226]  James O. Ramsay,et al.  Binomial Regression with Monotone Splines: A Psychometric Application , 1989 .

[227]  M. Reckase.  Multidimensional Item Response Theory , 2009 .

[228]  B. Efron.  Robbins, Empirical Bayes, And Microarrays , 2001 .

[229]  G. A. Ferguson,et al.  Item selection by the constant process , 1942 .

[230]  W. D. Linden,et al.  Handbook of item response theory , 2015 .

[231]  M. R. Novick.  The axioms and principal results of classical test theory , 1965 .

[232]  S. Haberman.  Product Models for Frequency Tables Involving Indirect Observation , 1977 .

[233]  Wendy M. Yen,et al.  Effects of Local Item Dependence on the Fit and Equating Performance of the Three-Parameter Logistic Model , 1984 .

[234]  D. J. Bartholomew,et al.  Factor Analysis for Categorical Data , 1980 .

[235]  S. Haberman.  IDENTIFIABILITY OF PARAMETERS IN ITEM RESPONSE MODELS WITH UNCONSTRAINED ABILITY DISTRIBUTIONS , 2005 .

[236]  R. Cattell.  The Scree Test For The Number Of Factors. , 1966, Multivariate behavioral research.

[237]  H. Robbins.  A Stochastic Approximation Method , 1951 .

[238]  Johan Braeken,et al.  A Boundary Mixture Approach to Violations of Conditional Independence , 2011 .

[239]  Daniel J Bauer,et al.  Improving the assessment of measurement invariance: Using regularization to select anchor items and identify differential item functioning. , 2020, Psychological methods.

[240]  Anja Vogler,et al.  An Introduction to Multivariate Statistical Analysis , 2004 .

[241]  Klaas Sijtsma,et al.  A tutorial on how to do a Mokken scale analysis on your test and questionnaire data. , 2017, The British journal of mathematical and statistical psychology.

[242]  Charles Lewis,et al.  A Nonparametric Approach to the Analysis of Dichotomous Item Responses , 1982 .

[243]  K. Pearson.  Mathematical contributions to the theory of evolution. VIII. On the correlation of characters not quantitatively measurable , 2022, Proceedings of the Royal Society of London.

[244]  J. Fries,et al.  The Patient-Reported Outcomes Measurement Information System (PROMIS): Progress of an NIH Roadmap Cooperative Group During its First Two Years , 2007, Medical care.

[245]  Herman Rubin,et al.  Statistical Inference in Factor Analysis , 1956 .

[246]  R. Mislevy.  Estimating latent distributions , 1984 .

[247]  B. Lindsay,et al.  Semiparametric Estimation in the Rasch Model and Related Exponential Response Models, Including a Simple Latent Class Model for Item Analysis , 1991 .

[248]  Z. Ying,et al.  Nonlinear sequential designs for logistic item response theory models with applications to computerized adaptive tests , 2009, 0906.1859.

[249]  Paul W. Holland,et al.  The Dutch Identity: A New Tool for the Study of Item Response Models. , 1990 .

[250]  R. Jennrich,et al.  Rotation for simple loadings , 1966, Psychometrika.

[251]  Wen-Xin Zhou,et al.  A max-norm constrained minimization approach to 1-bit matrix completion , 2013, J. Mach. Learn. Res..

[252]  Gerhard Tutz,et al.  A Penalty Approach to Differential Item Functioning in Rasch Models , 2015, Psychometrika.

[253]  Matthew S. Johnson,et al.  Nonparametric Estimation of Item and Respondent Locations from Unfolding-type Items , 2006, Psychometrika.

[254]  Damon Berridge,et al.  Multivariate Generalized Linear Mixed Models Using R , 2011 .

[255]  Anders Christoffersson,et al.  Factor analysis of dichotomized variables , 1975 .

[256]  Jean-Claude Falmagne,et al.  Knowledge spaces , 1998 .

[257]  Steven Andrew Culpepper,et al.  A Hidden Markov Model for Learning Trajectories in Cognitive Diagnosis With Application to Spatial Rotation Skills , 2018, Applied psychological measurement.

[258]  Z. Ying,et al.  a-Stratified Multistage Computerized Adaptive Testing , 1999 .