Evaluating Cognitive Training Outcomes: Validity and Utility of Structural Knowledge Assessment

Conventional approaches to evaluating cognitive outcomes of training typically use paper-and-pencil tests that emphasize gains or differences in declarative knowledge. Yet a key factor in differentiating expert and novice performance is the way individuals organize their knowledge. Accordingly, the acquisition of meaningful knowledge structures and methods of assessing structural knowledge are potentially important issues for designing and evaluating training programs. Two studies were conducted to examine the validity and utility of one structural assessment technique called Pathfinder (Schvaneveldt, Durso, & Dearholt, 1989). Results from academic and organizational samples indicated that Pathfinder measures of structural knowledge quality predicted individual differences in performance self-efficacy. After controlling for differences in declarative knowledge, measures of structural knowledge quality added to the prediction of performance self-efficacy in the student sample, but not in the organizational sample. The unique features and potential advantages of structural assessment for training evaluation are discussed.

[1]  Lívia Markíczy,et al.  A Method for Eliciting and Comparing Causal Maps , 1995 .

[2]  Aharon Tziner,et al.  Economic Utility of Training Programs , 1999 .

[3]  Richard J. Shavelson,et al.  Comparison of Content Structure and Cognitive Structure in High School Students' Learning of Probability. , 1975 .

[4]  C. E. Davis.  Experience and the organization of auditors’ knowledge , 1997 .

[5]  Francis T. Durso,et al.  Network Structures in Proximity Data , 1989 .

[6]  Peder J. Johnson,et al.  Assessing Structural Knowledge. , 1991 .

[7]  Francis T. Durso,et al.  Recall and Measures of Memory Organization , 1986 .

[8]  Richard K. Wagner,et al.  Tacit knowledge in everyday intelligent behavior. , 1987 .

[9]  Mary Anne Devanna,et al.  Strategic Human Resource Management , 1984 .

[10]  Michelene T. H. Chi,et al.  How knowledge is structured and used by expert and novice children , 1986 .

[11]  A. Bandura.  Social Foundations of Thought and Action , 1986 .

[12]  Alexander D. Stajkovic,et al.  Self-efficacy and work-related performance: A meta-analysis. , 1998 .

[13]  Moshe Naveh-Benjamin,et al.  Inferring students' cognitive structures and their development using the "ordered tree technique." , 1986 .

[14]  R. Glaser.  Education and Thinking: The Role of Knowledge. , 1984 .

[15]  Eduardo Salas,et al.  Measuring Knowledge Organization as a Method for Assessing Learning during Training , 1995, Hum. Factors.

[16]  E. Day,et al.  Knowledge structures and the acquisition of a complex skill. , 2001, The Journal of applied psychology.

[17]  C. Dodrill,et al.  Long-term reliability of the Wonderlic Personnel Test. , 1983 .

[18]  Peter Warr,et al.  Predicting three levels of training outcome , 1999 .

[19]  Leopold E. Klopfer,et al.  Structural representations of students' knowledge before and after science instruction , 1981 .

[20]  D. Kirkpatrick.  Techniques for evaluating training programs , 1979 .

[21]  J. Hunter.  Cognitive ability, cognitive aptitudes, job knowledge, and job performance , 1986 .

[22]  E. Salas,et al.  Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. , 1993 .

[23]  G. Alliger,et al.  KIRKPATRICK'S LEVELS OF TRAINING CRITERIA: THIRTY YEARS LATER , 1989 .

[24]  Martha Reeves,et al.  Evaluation of Training , 1993 .

[25]  Jenenne A Geske.  The structure of introductory statistics knowledge: An investigation exploring students' and experts' cognitive organization using multidimensional scaling and pathfinder analyses , 2001 .