A multi-expert approach for developing testing and diagnostic systems based on the concept-effect model

With the popularization of computer and communication technologies, researchers have attempted to develop computer-assisted testing and diagnostic systems to help students improve their learning performance on the Internet. In developing a diagnostic system for detecting students' learning problems, it is difficult for individual teachers to address the exact relationships between the test items and the concepts. To cope with this problem, this study proposes an innovative approach to eliciting and integrating the weightings of test item-concept relationships from multiple experts. Based on the proposed approach, a testing and diagnostic system has been implemented; moreover, an experiment was conducted to evaluate the performance of our approach. By analyzing the results from four groups of students using learning suggestions provided by different models, it was found that the learning performance of the students who received learning suggestions by applying the innovative approach was significantly better than for those who received guidance based on the original model.

[1]  Gwo-Jen Hwang,et al.  Diagnosing student learning problems based on historical assessment records , 2008 .

[2]  Gwo-Jen Hwang.  On the development of a cooperative tutoring environment on computer networks , 2002, IEEE Trans. Syst. Man Cybern. Part C.

[3]  Ya-Ting Carolyn Yang,et al.  Comprehensive evaluation criteria for English learning websites using expert validity surveys , 2008, Comput. Educ..

[4]  Gwo-Jen Hwang,et al.  A Delphi-based approach to developing expert systems with the cooperation of multiple experts , 2007, Expert Systems with Applications.

[5]  Zeinab A. Karake.  Enhancing the learning process with expert systems , 1990 .

[6]  Chenn-Jung Huang,et al.  An intelligent learning diagnosis system for Web-based thematic learning platform , 2007, Comput. Educ..

[7]  C. P. Yialouris,et al.  DIARES-IPM: a diagnostic advisory rule-based expert system for integrated pest management in Solanaceous crop systems , 2003 .

[8]  Analía Amandi,et al.  Intelligent assistance for teachers in collaborative e-learning environments , 2009, Comput. Educ..

[9]  Shyi-Ming Chen,et al.  Automatically constructing concept maps based on fuzzy rules for adapting learning systems , 2008, Expert Syst. Appl..

[10]  Gwo-Jen Hwang,et al.  A novel approach for assisting teachers in analyzing student web-searching behaviors , 2008, Comput. Educ..

[11]  Annemarie Hauf,et al.  Computers in education , 1983 .

[12]  Gwo-Jen Hwang,et al.  A conceptual map model for developing intelligent tutoring systems , 2003, Comput. Educ..

[13]  Akinobu Shimizu,et al.  A multi-expert approach for robust face detection , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[14]  Gwo-Jen Hwang,et al.  A context-aware ubiquitous learning environment for conducting complex science experiments , 2009, Comput. Educ..

[15]  Yen-Ting Lin,et al.  An adaptive testing system for supporting versatile educational assessment , 2009, Comput. Educ..

[16]  Ann Nevin,et al.  Collaborative Teaching: Critique of the Scientific Evidence , 2007 .

[17]  Yu-Bin Yang,et al.  Lung cancer cell identification based on artificial neural network ensembles , 2002, Artif. Intell. Medicine.

[18]  Margita Pavlekovic,et al.  Comparison of intelligent systems in detecting a child's mathematical gift , 2009, Comput. Educ..

[19]  Jia-Sheng Heh,et al.  Learning and diagnosis of individual and class conceptual perspectives: an intelligent systems approach using clustering techniques , 2005, Comput. Educ..

[20]  Clive L. Dym,et al.  Knowledge Acquisition from Multiple Experts , 1984, AI Mag..

[21]  Gwo-Jen Hwang,et al.  An auto-scoring mechanism for evaluating problem-solving ability in a web-based learning environment , 2009, Comput. Educ..

[22]  Gwo-Jen Hwang,et al.  Development of an adaptive learning system with two sources of personalization information , 2008, Comput. Educ..

[23]  Jiung-yao Huang,et al.  Distance learning in advanced military education: Analysis of joint operations course in the Taiwan military , 2009, Comput. Educ..

[24]  Larry R. Medsker,et al.  Knowledge acquisition from multiple experts: Problems and issues , 1995 .

[25]  Shyi-Ming Chen,et al.  Learning barriers diagnosis based on fuzzy rules for adaptive learning systems , 2009, Expert Syst. Appl..

[26]  Bin Shyan Jong,et al.  Learning Log Explorer in E-Learning Diagnosis , 2007, IEEE Transactions on Education.

[27]  Gwo-Jen Hwang,et al.  A test-sheet-generating algorithm for multiple assessment requirements , 2003, IEEE Trans. Educ..

[28]  Gwo-Jen Hwang,et al.  An innovative parallel test sheet composition approach to meet multiple assessment criteria for national tests , 2008, Comput. Educ..

[29]  Rodney L. Doran,et al.  Basic measurement and evaluation of science instruction , 1980 .

[30]  Norman Edward Gronlund,et al.  Assessment of student achievement , 1997 .

[31]  Gwo-Jen Hwang,et al.  A Computerized Approach to Diagnosing Student Learning Problems in Health Education , 2006 .

[32]  Simon Hooper,et al.  Cooperative learning and computer-based instruction , 1992 .

[33]  C. Chou,et al.  Diagnosing students' alternative conceptions in science , 2002, J. Comput. Assist. Learn..

[34]  Shian-Shyong Tseng,et al.  A new approach for constructing the concept map , 2004, IEEE International Conference on Advanced Learning Technologies, 2004. Proceedings..

[35]  Gwo-Jen Hwang,et al.  Development of a Web-Based System for Diagnosing Student Learning Problems on English Tenses , 2007, Int. J. Distance Educ. Technol..

[36]  S. Manning,et al.  Identifying students' mathematical skills from a multiple-choice diagnostic test using an iterative technique to minimise false positives , 2008, Comput. Educ..

[37]  Gwo-Jen Hwang,et al.  A Computer-Assisted Approach to Diagnosing Student Learning Problems in Science courses , 2003, J. Inf. Sci. Eng..

[38]  Gwo-Jen Hwang.  Gray Forecast Approach for Developing Distance Learning and Diagnostic Systems , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[39]  Ann Nevin,et al.  Collaborative Teaching for Teacher Educators--What Does the Research Say?. , 2009 .

[40]  Gwo-Jen Hwang,et al.  A time scale-oriented approach for building medical expert systems , 2006, Expert Syst. Appl..

[41]  Jay Liebowitz.  Knowledge‐based/expert systems technology in life support systems , 1997 .

[42]  Gudela Grote,et al.  Distributed collaboration activities in a blended learning scenario and the effects on learning performance , 2007, J. Comput. Assist. Learn..

[43]  Sue Drew,et al.  Key Skills Computerised Assessments. Guiding Principles , 2002 .

[44]  Bertrand Léger,et al.  Experimenting statecharts for multiple experts knowledge elicitation in agriculture , 2007, Expert Syst. Appl..

[45]  Judy van Biljon,et al.  A model for intelligent computer-aided education systems , 1995 .

[46]  Chun-Hsiung Lee,et al.  Application of automatically constructed concept map of learning to conceptual diagnosis of e-learning , 2009, Expert Syst. Appl..

[47]  Joseph J. Pear,et al.  Performance measures in courses using computer-aided personalized system of instruction , 2008, Comput. Educ..

[48]  Bin Shyan Jong,et al.  Diagnostic and remedial learning strategy based on conceptual graphs , 2004, J. Comput. Assist. Learn..

[49]  Beth Bos,et al.  The design and application of technology-based courses in the mathematics classroom , 2009, Comput. Educ..

[50]  Slavomir Stankov,et al.  TEx-Sys model for building intelligent tutoring systems , 2008, Comput. Educ..

[51]  George Luger,et al.  Artificial Intelligence: Structures and Strategies for Complex Problem Solving (5th Edition) , 2004 .