Comparison of the reliability and validity of scores from two concept‐mapping techniques*

This paper reports the results of a study that compared two concept-mapping techniques, one high-directed, ''fill-in-the-map,'' and one low-directed, ''construct-a-map-from-scratch.'' We examined whether: (1) skeleton map scores were sensitive to the sample of nodes or linking lines to be filled in; (2) the two types of skeleton maps were equivalent; and (3) the two mapping techniques provided similar infor- mation about students' connected understanding. Results indicated that fill-in-the-map scores were not sensitive to the sample of concepts or linking lines to be filled in. Nevertheless, the fill-in-the-nodes and fill- in-the-lines techniques were not equivalent forms of fill-in-the-map. Finally, high-directed and low-directed maps led to different interpretations about students' knowledge structure. Whereas scores obtained under the high-directed technique indicated that students' performance was close to the maximum possible, the scores obtained with the low-directed technique revealed that students' knowledge was incomplete com- pared to a criterion map. We concluded that the construct-a-map technique better reflected differences among students' knowledge structure. fl 2001 John Wiley & Sons, Inc. J Res Sci Teach 38: 260 - 278, 2001 Concept maps have been used to assess students' knowledge structure, especially in science education (Novak, 1990). The justification for assessing students' knowledge structures is based on the idea that relating concepts that belong to the same domain is an important characteristic of scientific literacy (e.g., Bybee, 1996; Moore, 1995). Theory and research have shown that understanding a subject domain such as science is associated with a rich set of relations among important concepts in the domain (Novak, 1998; Novak & Gowin, 1984; Novak, Gowin, & Johansen, 1983; Novak & Ridley, 1988). We know, for example, that successful learners develop elaborate and highly integrated frameworks of related concepts (Mintzes, Wandersee, & Novak, 1997), just as experts do (Chi, Glaser, & Farr, 1988; Glaser, 1991). Research has shown that highly organized structures facilitate problem solving and other cognitive activities (e.g., generating explanations or rapidly recognizing meaningful patterns; Baxter, Elder, & Glaser,

[1]  R. Shavelson Some Aspects of the Correspondence between Content Structure and Cognitive Structure in Physics Instruction. , 1972 .

[2]  R. Shavelson Methods for examining representations of A subject-matter structure in a student's memory , 1974 .

[3]  J. Novak,et al.  The use of concept mapping and knowledge vee mapping with junior high school science students , 1983 .

[4]  John R. Surber Mapping as a Testing and Diagnostic Device , 1984 .

[5]  Joseph D. Novak,et al.  Learning How to Learn , 1984 .

[6]  M. Chi,et al.  The Nature of Expertise , 1988 .

[7]  J. Minstrell,et al.  Teaching Science for Understanding , 1989 .

[8]  Thomas H. Anderson,et al.  On using concept maps to assess the comprehension effects of reading expository text , 1989 .

[9]  John R. McClure,et al.  Effects of an Environmental Education-Related STS Approach Instruction on Cognitive Structures of Preservice Science Teachers. , 1990 .

[10]  J. Novak Concept mapping: A useful tool for science education , 1990 .

[11]  Xiao-Li Meng,et al.  Comparing correlated correlation coefficients , 1992 .

[12]  Cultural and scientific literacy. , 1995, Molecular biology of the cell.

[13]  R. Glaser,et al.  Knowledge-based cognition and performance assessment in the science classroom , 1996 .

[14]  R. Shavelson,et al.  Problems and Issues in the Use of Concept Maps in Science Assessment. , 1996 .

[15]  Candace Schau,et al.  Use of Fill-in Concept Maps To Assess Middle School Students' Connected Understanding of Science. , 1997 .

[16]  Candace Schau,et al.  Use of Map Techniques in Teaching Applied Statistics Courses , 1997 .

[17]  Joseph D. Novak,et al.  Learning creating and using knowledge: Concept maps as facilitative tools , 1998 .