Detection of Differential Item Functioning on the Kirton Adaption-Innovation Inventory Using Multiple-Group Mean and Covariance Structure Analyses

This article demonstrates how Sörbom's (1974) mean and covariance structure analysis (MACS) model can be used to detect uniform and non-uniform differential item functioning (DIF) on polytomous ordered response items that are assumed to approximate a continuous scale. Using item responses from 773 civil service employees, the MACS model was applied to three cognitive style scales on the widely-used Kirton Adaption-Innovation Inventory (KAI, Kirton, 1976; 1977). As predicted, DIF across managerial and staff support occupational groups were detected on each of the scales. As expected, there was no evidence of DIF across gender groups and across two groups formed through a random split on any of the scales. Substantive implications for the use of the KAI as well as methodological implications and future research direction for detection of DIF are discussed.

[1]  Dorothy T. Thayer,et al.  Differential Item Performance and the Mantel-Haenszel Procedure. , 1986 .

[2]  Fredinand A. Gul,et al.  Adaption-innovation as a factor in australian accounting undergraduates' subject interests and career preferences☆ , 1986 .

[3]  Allan S. Cohen,et al.  Detection of Differential Item Functioning in Multiple Groups. , 1995 .

[4]  Larry J. Williams,et al.  An Alternative Approach to Method Effects by Using Latent-Variable Models: Applications in Organizational Behavior Research , 1994 .

[5]  Neal M. Kingston,et al.  The Analysis of Item-Ability Regressions: An Exploratory IRT Model Fit Tool , 1985 .

[6]  Howard T. Everson,et al.  Isolating Gender Differences in Test Anxiety: A Confirmatory Factor Analysis of the Test Anxiety Inventory , 1991 .

[7]  Wim Jansen,et al.  Multilog: Multiple, Categorical Item Analysis and Test Scoring Using Item Response Theory , 1994 .

[8]  K. J. Evans,et al.  Adaptors and Innovators: Styles of Creativity and Problem-solving , 1990 .

[9]  D. Sorbom,et al.  Structural equation models with structured means , 1981 .

[10]  F. Drasgow.  Scrutinizing psychological tests: Measurement equivalence and equivalent relations with external variables are the central issues , 1984 .

[11]  David Chan,et al.  The Conceptualization and Analysis of Change Over Time: An Integrative Approach Incorporating Longitudinal Mean and Covariance Structures Analysis (LMACS) and Multiple Indicator Latent Growth Modeling (MLGM) , 1998 .

[12]  William Stout,et al.  A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF , 1993 .

[13]  G. J. Mellenbergh.  Generalized linear item response theory. , 1994 .

[14]  R. Goldsmith.  The Factorial Composition of the Kirton Adaption-Innovation Inventory , 1985 .

[15]  K. Lanning.  Consistency, Scalability, and Personality Measurement , 1990 .

[16]  Fritz Drasgow,et al.  Study of the measurement bias of two standardized psychological tests. , 1987 .

[17]  R E Millsap,et al.  Confirmatory Measurement Model Comparisons Using Latent Means. , 1991, Multivariate behavioral research.

[18]  R. Goldsmith.  Dimensions of consumer innovativeness : an empirical study of open processing , 1983 .

[19]  B. Byrne,et al.  Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. , 1989 .

[20]  M. Kirton.  Adaptors and Innovators: A Description and Measure. , 1976 .

[21]  L. Shepard,et al.  Methods for Identifying Biased Test Items , 1994 .

[22]  Frans J. Oort,et al.  Simulation study of item bias detection with restricted factor analysis , 1998 .

[23]  M. Browne,et al.  Alternative Ways of Assessing Model Fit , 1992 .

[24]  R. P. McDonald,et al.  Goodness-of-fit indexes in confirmatory factor analysis : The effect of sample size , 1988 .

[25]  Bengt Muthén,et al.  General Longitudinal Modeling of Individual Differences in Experimental Designs: A Latent Variable Framework for Analysis and Power Estimation , 1997 .

[26]  Keith F Widaman,et al.  Confirmatory factor analysis and item response theory: two approaches for exploring measurement invariance. , 1993, Psychological bulletin.

[27]  Robert T. Golembiewski,et al.  Measuring Change and Persistence in Human Affairs: Types of Change Generated by OD Designs , 1976 .

[28]  D. Sörbom.  A GENERAL METHOD FOR STUDYING DIFFERENCES IN FACTOR MEANS AND FACTOR STRUCTURE BETWEEN GROUPS , 1974 .

[29]  Ratna Nandakumar,et al.  Simultaneous DIF Amplification and Cancellation: Shealy-Stout's Test for DIF , 1993 .

[30]  David Chan,et al.  Cognitive Misfit of Problem-Solving Style at Work: A Facet of Person-Organization Fit , 1996 .

[31]  E. Kelloway.  Using LISREL for Structural Equation Modeling: A Researcher′s Guide , 1998 .

[32]  M. Kirton,et al.  Adaptors and Innovators in Organizations , 1980 .

[33]  George Hayward,et al.  Adaptors and innovators: Data from the Kirton Adaptor‐Innovator Inventory in a local authority setting , 1983 .

[34]  Earl R. Nason,et al.  Likability and Similarity as Potential Sources of Predictor-Related Criterion Bias in Validation Research , 1996 .

[35]  P. Bentler,et al.  Comparative fit indexes in structural models. , 1990, Psychological bulletin.

[36]  J. H. Steiger.  Structural Model Evaluation and Modification: An Interval Estimation Approach. , 1990, Multivariate behavioral research.

[37]  G. P. Previde.  Italian adaptors and innovators: Is cognitive style underlying culture? , 1991 .

[38]  Christine E. DeMars,et al.  Item Response Theory , 2010, Assessing Measurement Invariance for Applied Research.

[39]  W. H. Angoff,et al.  Perspectives on differential item functioning methodology. , 1993 .

[40]  F. Lord.  Applications of Item Response Theory To Practical Testing Problems , 1980 .

[41]  R. Hambleton,et al.  Item Response Theory , 1984, The History of Educational Measurement.

[42]  M. Weber.  From Max Weber: Essays in sociology , 1946 .

[43]  G. J. Mellenbergh.  Contingency Table Models for Assessing Item Bias , 1982 .

[44]  Gina J. Medsker,et al.  A Review of Current Practices for Evaluating Causal Models in Organizational Behavior and Human Resources Management Research , 1994 .

[45]  L. J. Williams,et al.  An Alternative Approach to Method Effects by Using Latent-Variable Models: Applications in Organizational Behavior Research , 1994 .

[46]  R. Zwick,et al.  Assessment of Differential Item Functioning for Performance Tasks , 1993 .

[47]  P. Bentler,et al.  Significance Tests and Goodness of Fit in the Analysis of Covariance Structures , 1980 .

[48]  P. J. Ferrando.  Calibration of Invariant Item Parameters in a Continuous Item Response Model Using the Extended Lisrel Measurement Submodel. , 1996, Multivariate behavioral research.

[49]  J. Helms.  Why is there no study of cultural equivalence in standardized cognitive ability testing , 1992 .

[50]  F. Drasgow,et al.  A Revision of the Job Diagnostic Survey: Elimination of a Measurement Artifact , 1987 .

[51]  M. Kirton,et al.  The Adaption-Innovation Continuum, Occupational Type, and Course Selection , 1982 .

[52]  R. MacCallum,et al.  Power analysis and determination of sample size for covariance structure modeling. , 1996 .

[53]  Howard T. Everson,et al.  Methodology Review: Statistical Approaches for Assessing Measurement Bias , 1993 .

[54]  Karl G. Jöreskog,et al.  Lisrel 8: Structural Equation Modeling With the Simplis Command Language , 1993 .

[55]  P. Holland.  Adaptors and Innovators: Application of the Kirton Adaption-Innovation Inventory to Bank Employees , 1987 .