Model-based tools for knowledge assessment

This chapter introduces the functions of knowledge representation and presents a critical review of the model-based tools for assessment: Pathfinder, ALA-Reader, jMAP, HIMATT, AKOVIA. For each model-based tool, foundations and applications are discussed. Next, the tools are compared in order to illustrate their advantages and disadvantages, strengths, and limitations. The latter part postulates that there is no easy and no complete way to integrate any of the model-based tools. However, the strength of good research lies in a best possible integration: Multiple perspectives on the same construct are usually needed. Thus, the further development of existing tools as well as of new ones is necessary to explore human knowledge, its change, decision-making, performance, and problem-solving as our understanding of those complex human potentials evolves.

[1]  Dirk Ifenthaler,et al.  Automated Knowledge Visualization and Assessment , 2010 .

[2]  Mark Torrance,et al.  Writing as a Knowledge-Constituting Process , 1999 .

[3]  Dirk Ifenthaler,et al.  Understanding models for learning and instruction , 2008 .

[4]  Roy B. Clariana,et al.  A Comparison of Pair-Wise, List-Wise, and Clustering Approaches for Eliciting Structural Knowledge in Information Systems Courses , 2009 .

[5]  Rob Koper,et al.  La rappresentazione del Learning Design di Unità di Apprendimento , 2006 .

[6]  K. A. Ericsson,et al.  Protocol Analysis: Verbal Reports as Data , 1984 .

[7]  Dirk Ifenthaler,et al.  Scope of Graphical Indices in Educational Diagnostics , 2010 .

[8]  Dirk Ifenthaler,et al.  Computer-Based Diagnostics and Systematic Analysis of Knowledge , 2010 .

[9]  David H. Jonassen.  Externally Modeling Mental Models , 2009 .

[10]  Dirk Ifenthaler,et al.  Text-Guided Automated Self Assessment , 2009, CELDA 2011.

[11]  Arthur B. Markman,et al.  Knowledge Representation , 1998 .

[12]  R. Clariana,et al.  A Computer-Based Approach for Deriving and Measuring Individual and Team Knowledge Structure from Essay Questions , 2007 .

[13]  Dirk Ifenthaler,et al.  Assessment of Knowledge: Do Graphical Notes and Texts Represent Different Things? , 2009 .

[14]  T. Goldsmith,et al.  Assessing structural similarity of graphs , 1990 .

[15]  Susan M. Land,et al.  Theoretical Foundations of Learning Environments. , 1999 .

[16]  Colin Tattersall,et al.  New directions for lifelong learning using network technologies , 2004, Br. J. Educ. Technol..

[17]  Timothy E. Goldsmith,et al.  Specificity of Structural Assessment of Knowledge , 2010 .

[18]  Kinshuk,et al.  Learning and Instruction in the Digital Age , 2014 .

[19]  P. Johnson-Laird,et al.  Mental Models: Towards a Cognitive Science of Language, Inference, and Consciousness , 1985 .

[20]  Pablo Pirnay-Dummer,et al.  Model-Based Knowledge Mapping , 2010 .

[21]  R W Schvaneveldt,et al.  Assessing Beliefs about 'Environmental Illness/Multiple Chemical Sensitivity' , 1996, Journal of health psychology.

[22]  Dirk Ifenthaler,et al.  Practical Solutions for the Diagnosis of Progressing Mental Models , 2008 .

[23]  D. Galbraith,et al.  Knowing What to Write , 1999 .

[24]  Fred D. Davis,et al.  Improving computer skill training: behavior modeling, symbolic mental rehearsal, and the role of knowledge structures. , 2004, The Journal of applied psychology.

[25]  Nancy J. Cooke,et al.  Toward an on-line knowledge assessment methodology: Building on the relationship between knowing and doing , 1996 .

[26]  Timothy E. Goldsmith,et al.  Structural knowledge assessment: comparison of referent structures , 1994 .

[27]  D. Jonassen,et al.  Externalizing Mental Models with Mindtools , 2008 .

[28]  M. Perkins,et al.  Visions of the Future , 1996 .

[29]  Dirk Ifenthaler,et al.  Relational, structural, and semantic analysis of graphical representations and concept maps , 2010 .

[30]  Dirk Ifenthaler,et al.  Bridging the Gap between Expert-Novice Differences , 2010 .

[31]  John G. Gammack,et al.  Expert conceptual structure: the stability of pathfinder representations , 1990 .

[32]  Pablo Pirnay-Dummer,et al.  Complete Structure Comparison , 2010 .

[33]  P. Johnson-Laird.  Mental models , 1989 .

[34]  Roy B. Clariana,et al.  A Technique for Automatically Scoring Open-Ended Concept Maps , 2006 .

[35]  G. Bower,et al.  THE PSYCHOLOGY OF LEARNING AND M·OTIVATION , 2001 .

[36]  Paul A. Kirschner.  Introduction to part II of the special issue: Design, development and implementation of electronic learning environments for collaborative learning , 2004 .

[37]  V. Shute,et al.  Innovative Assessment for the 21st Century , 2010 .

[38]  Anna Strasser,et al.  A Functional View Toward Mental Representations , 2010 .

[39]  Barry Kirwan,et al.  A Guide To Task Analysis: The Task Analysis Working Group , 1992 .

[40]  Heinz Mandl,et al.  Mental Models of Complex Systems: When Veridicality Decreases Functionality , 1995 .

[41]  Dirk Ifenthaler,et al.  Text-Guided Automated Self Assessment A Graph-Based Approach to Help Learners with Ongoing Writing , 2011 .

[42]  Kinshuk,et al.  Multiple perspectives on problem solving and learning in the digital age , 2011 .

[43]  Jonathan O. McKeown,et al.  Using annotated concept map assessments as predictors of performance and understanding of complex problems for teacher technology integration , 2009 .

[44]  Francis T. Durso,et al.  Network Structures in Proximity Data , 1989 .

[45]  Nancy J. Cooke,et al.  Network and Multidimensional Representations of the Declarative Knowledge of Human-Computer Interface Design Experts , 1992, Int. J. Man Mach. Stud..

[46]  Dirk Ifenthaler,et al.  Comparison of Two Analysis Approaches for Measuring Externalized Mental Models , 2011, J. Educ. Technol. Soc..

[47]  Russell G. Almond,et al.  Enhancing the Design and Delivery of Assessment Systems: A Four-Process Architecture , 2002 .

[48]  Timothy E. Goldsmith,et al.  Locus of the Predictive Advantage in Pathfinder-Based Representations of Classroom Knowledge. , 1994 .

[49]  Emden R. Gansner,et al.  Graphviz and Dynagraph – Static and Dynamic Graph Drawing Tools , 2003 .

[50]  Jeongmin Lee,et al.  Effects of model -centered instruction and levels of learner expertise on effectiveness, efficiency, and engagement with ill-structured problem solving: An exploratory study of ethical decision making in program evaluation , 2009 .

[51]  Norbert M. Seel,et al.  Encyclopedia of the sciences of learning , 2012 .

[52]  J. Michael Spector,et al.  Adventures and Advances in Instructional Design Theory and Practice , 2009 .

[53]  R. Clariana,et al.  A computer-based approach for translating text into concept map-like representations , 2004 .

[54]  Yvonne Rogers,et al.  External cognition: how do graphical representations work? , 1996, Int. J. Hum. Comput. Stud..

[55]  Dirk Ifenthaler,et al.  Reading guided by automated graphical representations: How model-based text visualizations facilitate learning in reading comprehension tasks , 2011 .

[56]  Dirk Ifenthaler.  Model-based feedback for improving expertise and expert performance , 2009 .

[57]  Jason Huett,et al.  Learning and Instructional Technologies for the 21st Century , 2009 .

[58]  Roy B. Clariana,et al.  Deriving Individual and Group Knowledge Structure from Network Diagrams and from Essays , 2010 .

[59]  Cristina Zucchermaglio,et al.  Organizational Learning and Technological Change , 1995, NATO ASI Series.

[60]  L. Housner,et al.  Conceptual Maps and Simulated Teaching Episodes as Indicators of Competence in Teaching Elementary Mathematics , 1996 .

[61]  R. Branch,et al.  Educational Media and Technology Yearbook , 2019, Educational Media and Technology Yearbook.

[62]  Dirk Ifenthaler,et al.  Determining the effectiveness of prompts for self-regulated learning in problem-solving scenarios , 2012, J. Educ. Technol. Soc..

[63]  Dirk Ifenthaler,et al.  Designing model-based learning environments to support mental models for learning , 2012 .

[64]  Roger W. Schvaneveldt,et al.  Pathfinder associative networks: studies in knowledge organization , 1990 .

[65]  W. Kintsch.  The role of knowledge in discourse comprehension: a construction-integration model. , 1988, Psychological review.

[66]  Lev Vygotsky.  Denken und Sprechen , 1964 .

[67]  J. Michael Spector,et al.  Highly integrated model assessment technology and tools , 2010, CELDA 2008.

[68]  Valerie J. Shute,et al.  Modeling, Assessing, and Supporting Key Competencies Within Game Environments , 2010 .

[69]  Norbert Groeben,et al.  Die Heidelberger Struktur-Lege-Technik (SLT) , 1984 .

[70]  Linda Jane Smith,et al.  Graph and property set analysis: A methodology for comparing mental model representations , 2009 .

[71]  Allan Jeong.  Assessing Change in Learners’ Causal Understanding Using Sequential Analysis and Causal Maps , 2010 .

[72]  Dirk Ifenthaler,et al.  Identifying cross-domain distinguishing features of cognitive structure , 2011 .

[73]  Dirk Ifenthaler.  Learning and instruction in the digital age: Introduction , 2010 .

[74]  Dirk Ifenthaler,et al.  Intelligent Model-Based Feedback: Helping Learners to Monitor their Individual Learning Progress , 2012 .

[75]  José J. Cañas,et al.  Structural Representations in Knowledge Acquisition. , 1994 .

[76]  J. Michael Spector,et al.  Model-Based Methods for Assessment, Learning, and Instruction: Innovative Educational Technology at Florida State University , 2009 .

[77]  D. W. Nelson,et al.  A Conceptual Framework for External Representations of Knowledge in Teaching and Learning Environments , 2004 .

[78]  Norbert M. Seel,et al.  Educational diagnosis of mental models: Assessment problems and technology-based solutions. , 1999 .

[79]  Hyewon Kim,et al.  An investigation of the effects of model-centered instruction in individual and collaborative contexts: The case of acquiring instructional design expertise , 2008 .

[80]  Robert J. Mislevy,et al.  On the roles of external knowledge representations in assessment design , 2007 .

[81]  Dirk Ifenthaler,et al.  States and Processes of Learning Communities. Engaging Students in Meaningful Reflection and Learning , 2011, Social Media Tools and Platforms in Learning Environments.

[82]  Michael J. Hannafin,et al.  Emerging technologies, ISD, and learning environments: Critical perspectives , 1992 .