Measuring Knowledge Organization as a Method for Assessing Learning during Training

This paper applies the conceptual work of K. Kraiger, J. K. Ford, and E. Salas (1993) to the evaluation of two training programs. A method known as structural assessment (SA) was described and adapted for use in the evaluation of a training program for computer programming and a PC-based simulation of a naval decision-making task. SA represents and evaluates pairwise judgments of relatedness of concepts drawn from the training content domain. In the first study, SA scores of students (determined by similarity to an expert solution) were significantly higher after training than before but did not predict performance on a take-home exam 12 weeks later. In the second study, we manipulated training content by providing half the students with the goals and objectives of the transfer task (an advance organizer) before training and providing the other half with the same information after training. As hypothesized, SA scores were higher for those receiving the organizers before training; SA scores were also more strongly related to performance on the criterion task for this group. Implications of the results for training evaluation are discussed.

[1]  J. Reitman,et al.  Knowledge organization and skill differences in computer programmers , 1981, Cognitive Psychology.

[2]  J. Mathieu,et al.  Meeting trainees' expectations: The influence of training fulfillment on the development of commitment, self-efficacy, and motivation. , 1991 .

[3]  Thomas J. Shuell,et al.  Phases of Meaningful Learning , 1990 .

[4]  Elke Sumfleth.  Knowledge of terms and problem-solving in chemistry , 1988 .

[5]  Ray Sleet,et al.  IMPROVING THE RELATIONSHIP BETWEEN ASSESSMENT RESULTS AND STUDENT UNDERSTANDING , 1990 .

[6]  Eduardo Salas,et al.  A Performance Assessment Task for Examining Tactical Decision Making Under Stress , 1992 .

[7]  Keith J. Holyoak,et al.  The cognitive basis of knowledge transfer. , 1987 .

[8]  David H. Jonassen.  Changes in Knowledge Structures from Building Semantic Net versus Production Rule Representations of Subject Content. , 1993 .

[9]  Leopold E. Klopfer,et al.  Structural representations of students' knowledge before and after science instruction , 1981 .

[10]  L. Sjodahl.  Evaluation of Training. , 1975 .

[11]  Robert D. Tennyson,et al.  An Empirically Based Instructional Design Theory for Teaching Concepts , 1986 .

[12]  E. Salas,et al.  Shared mental models in expert team decision making. , 1993 .

[13]  Mary Jo Kane,et al.  Young children's mental models determine analogical transfer across problems with a common goal structure * , 1986 .

[14]  Ayşen Melek Aytuğ Koşan,et al.  Assessment of Learning Outcomes , 2021, Assessment Tools for Mapping Learning Outcomes With Learning Objectives.

[15]  G. Alliger,et al.  KIRKPATRICK'S LEVELS OF TRAINING CRITERIA: THIRTY YEARS LATER , 1989 .

[16]  Peder J. Johnson,et al.  Assessing Structural Knowledge. , 1991 .

[17]  Francis T. Durso,et al.  Network Structures in Proximity Data , 1989 .

[18]  Nancy J. Cooke,et al.  Network and Multidimensional Representations of the Declarative Knowledge of Human-Computer Interface Design Experts , 1992, Int. J. Man Mach. Stud..

[19]  D. Herrmann,et al.  Problem perception and knowledge structure in expert and novice mathematical problem solvers. , 1982 .

[20]  T. Goldsmith,et al.  Assessing structural similarity of graphs , 1990 .

[21]  E. Salas,et al.  Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. , 1993 .

[22]  E. Salas,et al.  Toward an Integration of Training Theory and Technique , 1991 .

[23]  James E. McDonald,et al.  The Application of Psychological Scaling Techniques to Knowledge Elicitation for Knowledge-Based Systems , 1987, Int. J. Man Mach. Stud..

[24]  Susan E. Newman,et al.  Cognitive Apprenticeship: Teaching the Craft of Reading, Writing, and Mathematics. Technical Report No. 403. , 1987 .

[25]  Richard E. Mayer,et al.  Can Advance Organizers Influence Meaningful Learning? , 1979 .

[26]  E. Yalow.  On Educational psychology: A cognitive view. , 1979 .

[27]  J. Bergeron,et al.  Models of Understanding. , 1983 .

[28]  Nancy J. Cooke,et al.  Effects of Computer Programming Experience on Network Representations of Abstract Programming Concepts , 1988, Int. J. Man Mach. Stud..

[29]  R. Glaser,et al.  Expertise in a complex skill: Diagnosing x-ray pictures. , 1988 .