Design and performance frameworks for constructing problem-solving simulations.

Rapid advancements in hardware, software, and connectivity are helping to shorten the times needed to develop computer simulations for science education. These advancements, however, have not been accompanied by corresponding theories of how best to design and use these technologies for teaching, learning, and testing. Such design frameworks ideally would be guided less by the strengths/limitations of the presentation media and more by cognitive analyses detailing the goals of the tasks, the needs and abilities of students, and the resulting decision outcomes needed by different audiences. This article describes a problem-solving environment and associated theoretical framework for investigating how students select and use strategies as they solve complex science problems. A framework is first described for designing on-line problem spaces that highlights issues of content, scale, cognitive complexity, and constraints. While this framework was originally designed for medical education, it has proven robust and has been successfully applied to learning environments from elementary school through medical school. Next, a similar framework is detailed for collecting student performance and progress data that can provide evidence of students' strategic thinking and that could potentially be used to accelerate student progress. Finally, experimental validation data are presented that link strategy selection and use with other metrics of scientific reasoning and student achievement.

[1]  Joseph Krajcik,et al.  Proportional Reasoning and Achievement in High School Chemistry , 1987 .

[2]  Michael Shayer,et al.  Accelerating the development of formal thinking in middle and high school students , 1990 .

[3]  Susan Kanowith-Klein,et al.  Sleuthing for Science , 1998 .

[4]  A. Collins,et al.  Situated Cognition and the Culture of Learning , 1989 .

[5]  Joseph D. Novak,et al.  Learning How to Learn , 1984 .

[6]  R H Stevens,et al.  Search path mapping: a versatile approach for visualizing problem-solving behavior. , 1991, Academic medicine : journal of the Association of American Medical Colleges.

[7]  John R. Staver,et al.  The effects of reasoning, use of models, sex type, and their interactions on posttest achievement in chemical bonding after constant instruction , 1985 .

[8]  A. Lawson,et al.  Development of Scientific Reasoning in College Biology: Do Two Levels of General Hypothesis-Testing Skills Exist?. , 2000 .

[9]  Robyn Fivush,et al.  Scripts and episodes: The development of event memory , 1992 .

[10]  Peter A. Frensch,et al.  The Role of Information Reduction in Skill Acquisition , 1996, Cognitive Psychology.

[11]  R. Stevens,et al.  Artificial neural network-based performance assessments , 1999 .

[12]  R. Mayer,et al.  Cognitive constraints on multimedia learning: When presenting more material results in less understanding. , 2001 .

[13]  Ronald H. Stevens,et al.  Cognitive process validation of an online problem solving assessment , 2002, Comput. Hum. Behav..

[14]  Terry Vendlinski,et al.  A Markov Model Analysis of Problem-Solving Progress. , 2002 .

[15]  S. Ambron,et al.  Interactive multimedia , 1988 .

[16]  L M Reder,et al.  Awareness and working memory in strategy adaptivity , 2001, Memory & cognition.

[17]  J. Kolodner.  Educational implications of analogy. A view from case-based reasoning. , 1997, The American psychologist.

[18]  A S Elstein,et al.  Beyond multiple‐choice questions and essays: the need for a new way to assess clinical competence , 1993, Academic medicine : journal of the Association of American Medical Colleges.

[19]  L. Reder,et al.  The Strategy-Specific Nature of Improvement: The Power Law Applies by Strategy Within Task , 1998 .

[20]  Anton E. Lawson,et al.  The development and validation of a classroom test of formal reasoning , 1978 .

[21]  Russell G. Almond,et al.  Bayes Nets in Educational Assessment: Where Do the Numbers Come from? CSE Technical Report. , 2000 .

[22]  McCoy Jm,et al.  Solving the problem of how medical students solve problems. , 1991 .

[23]  Michael Shayer,et al.  Accelerating the development of formal thinking in middle and high school students IV: Three years after a two-year intervention , 1993 .

[24]  Russell G. Almond,et al.  Bayes Nets in Educational Assessment: Where the Numbers Come From , 1999, UAI.

[25]  Ronald H. Stevens,et al.  Exploring Alternative Models of Complex Patient Management with Artificial Neural Networks , 2000, Advances in health sciences education : theory and practice.

[26]  G V Barrett,et al.  A reconsideration of testing for competence rather than for intelligence. , 1991, The American psychologist.

[27]  D. Broadbent,et al.  Interactive tasks and the implicit‐explicit distinction , 1988 .

[28]  Ron Stevens,et al.  Assessing Student Problem-Solving Skills With Complex Computer-Based Tasks , 2002 .

[29]  B. Andersson,et al.  Science teaching and the development of thinking , 1976 .

[30]  R H Stevens,et al.  Artificial neural networks as adjuncts for assessing medical students' problem solving performances on computer-based simulations. , 1993, Computers and biomedical research, an international journal.

[31]  K. VanLehn,et al.  Cognitive skill acquisition. , 1996, Annual review of psychology.

[32]  Anton E. Lawson,et al.  What are the relative effects of reasoning ability and prior knowledge on biology achievement in expository and inquiry classes , 1998 .

[33]  J Palacio-Cayetano,et al.  UCLA's outreach program of science education in the Los Angeles schools. , 1999, Academic medicine : journal of the Association of American Medical Colleges.

[34]  John R. Anderson.  Cognitive Psychology and Its Implications , 1980 .

[35]  L. Reder,et al.  Metacognition does not imply awareness: Strategy choice is governed by implicit learning and memory , 1996 .

[36]  R. Glaser,et al.  Knowing What Students Know: The Science and Design of Educational Assessment , 2001 .

[37]  Janet L. Kolodner,et al.  Case-Based Reasoning , 1988, IJCAI 1989.

[38]  R H Stevens,et al.  Solving the problem of how medical students solve problems. , 1991, M.D. computing : computers in medical practice.

[39]  P. Johnson-Laird.  Mental models , 1989 .

[40]  C. Knox-Quinn.  International Society for Technology in Education. , 1992 .

[41]  Ron Stevens,et al.  The Use of Artificial Neural Nets (ANN) to Help Evaluate Student Problem Solving Strategies , 2000 .

[42]  John R. Anderson,et al.  The generality/specificity of expertise in scientific reasoning , 1999, Cogn. Sci..

[43]  P. Chandler,et al.  Why Some Material Is Difficult to Learn , 1994 .

[44]  Robert W. Lissitz,et al.  Estimating the Impact of Instructional Practices on Student Achievement in Science , 1999 .

[45]  Russell G. Almond,et al.  A cognitive task analysis with implications for designing simulation-based performance assessment☆ , 1999 .