Domain-general and domain-specific developmental assessments: do they measure the same thing?

Abstract The concept of epistemological development is useful in psychological assessment only insofar as instruments can be designed to measure it consistently, reliably, and without bias. In the psychosocial domain, most traditional stage assessment systems rely on a process of matching concepts in a scoring manual generated from a limited number of construction cases, and thus suffer from bias introduced by an over-dependence on particular content. In contrast, the Hierarchical Complexity Scoring System (HCSS) employs criteria for assessing the hierarchical complexity of texts that are independent of specific conceptual content. This paper examines whether the HCSS and a conventional stage assessment system, Kohlberg’s Standard Issue Scoring System (SISS), measure the same dimension of performance. We scored 378 moral judgment interviews with both scoring systems. We then conducted a multidimensional partial credit analysis to determine the extent to which the two scoring systems assess the same dimension of performance. The disattenuated correlation between performance estimates on the SISS and HCSS is .92. Based on this and other evidence, we conclude that a single latent trait — hierarchical complexity — is the predominant dimension assessed by the two systems.

[1]  T. Bond,et al.  Piaget and measurement III : reassessing the Méthode clinique , 1995 .

[2]  R. Enright,et al.  Handbook of Moral Behavior and Development , 1991 .

[3]  M. Keller,et al.  A Critical Note on the Conception of Preconventional Morality: The Case of Stage 2 in Kohlberg's Theory , 1989 .

[4]  Theo L. Dawson,et al.  Developing conceptions of authority and contract across the lifespan: Two perspectives ☆ , 2003 .

[5]  Theo L Dawson,et al.  A Stage Is a Stage Is a Stage: A Direct Comparison of Two Scoring Systems , 2003, The Journal of genetic psychology.

[6]  K. Fischer,et al.  Dynamic development of psychological structures in action and thought. , 1998 .

[7]  R. Gelman.  Cognitive Development 1 , 2002 .

[8]  L J Walker,et al.  A longitudinal study of moral reasoning. , 1989, Child development.

[9]  Patricia M. King,et al.  The Reflective Judgment model: Ten years of research. , 1990 .

[10]  K. Fischer.  A theory of cognitive development: The control and construction of hierarchies of skills. , 1980 .

[11]  Anne Colby.  Standard issue scoring manual , 1987 .

[12]  J. Piaget,et al.  The equilibration of cognitive structures : the central problem of intellectual development , 1985 .

[13]  Barbara Henderson,et al.  Individual Variability and Consistency in Cognitive Development: New Evidence for the Existence of Central Conceptual Structures , 1993 .

[14]  M. Davison,et al.  Sequentiality and Consistency in the development of reflective judgment: A six-year longitudinal study , 1989 .

[15]  Theo L. Dawson,et al.  Developmental Trajectories in Moral Reasoning Across the Life Span , 1997 .

[16]  C. Fox,et al.  Applying the Rasch Model: Fundamental Measurement in the Human Sciences , 2001 .

[17]  K. Fischer,et al.  Patterns of data: Sequence, synchrony, and constraint in cognitive development , 1981 .

[18]  A. Demetriou,et al.  Structure and development of causal-experimental thought: from early adolescence to youth , 1993 .

[19]  Robbie Case,et al.  Intellectual development : birth to adulthood , 1985 .

[20]  W. Damon.  Measurement and Social Development , 1977 .

[21]  Raymond J. Adams,et al.  The Multidimensional Random Coefficients Multinomial Logit Model , 1997 .

[22]  Paul van Geert,et al.  The Dynamics of General Developmental Mechanisms From Piaget and Vygotsky to Dynamic Systems Models , 2000 .

[23]  U Müller,et al.  Developmental sequences in class reasoning and propositional reasoning. , 1999, Journal of experimental child psychology.

[24]  D. Kuhn.  Short-term longitudinal evidence for the sequentiality of Kohlberg's early stages of moral judgment. , 1976 .

[25]  Phillip K. Wood,et al.  Developmental Range of Reflective Judgment: The Effect of Contextual Support and Practice on Developmental Stage , 1993 .

[26]  Karen Draney,et al.  Objective measurement : theory into practice , 1992 .

[27]  Anne Colby,et al.  A Longitudinal Study of Moral Judgment. , 1983 .

[28]  Lawrence J. Walker,et al.  The Sequentiality of Kohlberg's Stages of Moral Development. , 1982 .

[29]  Cheryl Armon.  Ideals of the Good Life: A Longitudinal/Cross-Sectional Study of Evaluative Reasoning in Children and Adults , 1984 .

[30]  James R. Rest.  Longitudinal study of the Defining Issues Test of moral judgment: A strategy for analyzing developmental change. , 1975 .

[31]  Theo Linda Dawson,et al.  New tools, new insights: Kohlberg’s moral judgement stages revisited , 2002 .

[32]  L. Kohlberg.  Stage and sequence: The cognitive-developmental approach to socialization , 1969 .

[33]  Michael Lamport Commons,et al.  Hierarchical Complexity of Tasks Shows the Existence of Developmental Stages , 1998 .

[34]  Joseph Reimer,et al.  Development of social-moral reasoning among Kibbutz adolescents: A longitudinal cross-cultural study. , 1985 .

[35]  Larry Nucci,et al.  Sociomoral development and drug and alcohol abuse , 1991 .

[36]  Monique de Saint Martin.  Uma "boa" educação , 1999 .

[37]  Theo Linda Dawson,et al.  A comparison of three developmental stage scoring systems. , 2002, Journal of applied measurement.

[38]  Dorothy Z. Ullian.  The development of conceptions of masculinity and femininity. , 1976 .

[39]  Anne Colby,et al.  The Measurement Of Moral Judgment , 1987 .