EduMiner: Using text mining for automatic formative assessment

Formative assessment and summative assessment are two widely accepted approaches of assessment. While summative assessment is a typically formal assessment used at the end of a lesson or course, formative assessment is an ongoing process of monitoring learners' progresses of knowledge construction. Although empirical evidence has acknowledged that formal assessment is indeed superior to summative assessment, current e-learning assessment systems however seldom provide plausible solutions for conducting formative assessment. The major bottleneck of putting formative assessment into practice lies in its labor-intensive and time-consuming nature, which makes it hardly a feasible way of achievement evaluation especially when there are usually a large number of learners in e-learning environment. In this regard, this study developed EduMiner to relieve the burdens imposed on instructors and learners by capitalizing a series of text mining techniques. An empirical study was held to examine effectiveness and to explore outcomes of the features that EduMiner supported. In this study 56 participants enrolling in a ''Human Resource Management'' course were randomly divided into either experimental groups or control groups. Results of this study indicated that the algorithms introduced in this study serve as a feasible approach for conducting formative assessment in e-learning environment. In addition, learners in experimental groups were highly motivated to phrase the contents with higher-order level of cognition. Therefore a timely feedback of visualized representations is beneficial to facilitate online learners to express more in-depth ideas in discourses.

[1]  J. Piaget,et al.  The Child's Conception of the World , 1971 .

[2]  Dan Sullivan,et al.  Document Warehousing and Text Mining: Techniques for Improving Business Operations, Marketing, and Sales , 2001 .

[3]  B. Bloom,et al.  Taxonomy of Educational Objectives. Handbook I: Cognitive Domain , 1966 .

[4]  P. Black,et al.  Assessment and Classroom Learning , 1998 .

[5]  M. Scriven,et al.  Perspectives of curriculum evaluation , 1968 .

[6]  Raymond J. Mooney,et al.  Text mining with information extraction , 2004 .

[7]  Anna Sfard,et al.  Guest Editor's Introduction: Analyzing Tools: Perspectives on the Role of Designed Artifacts in Mathematics Learning , 2002 .

[8]  Laura M. Leventhal,et al.  Implementations of the CC′01 human – computer interaction guidelines using Bloom's taxonomy , 2007, Comput. Sci. Educ..

[9]  Robert J. Marzano,et al.  Designing a New Taxonomy of Educational Objectives , 2000 .

[10]  Robert J. Marzano,et al.  Designing a New Taxonomy of Educational Objectives. Experts in Assessment. , 2001 .

[11]  Jack Conklin,et al.  A New Bloom: Transforming Learning. , 2007 .

[12]  Nian-Shing Chen,et al.  Enhancing the Quality of e-Learning in Virtual Learning Communities by Finding Quality Learning Content and Trustworthy Collaborators , 2007, J. Educ. Technol. Soc..

[13]  Benjamin S. Bloom,et al.  Handbook On Formative and Summative Evaluation of Student Learning , 1971 .

[14]  Steven R. Banks Classroom Assessment: Issues and Practices , 2004 .

[15]  T. Crooks The Impact of Classroom Evaluation Practices on Students , 1988 .

[16]  J. Fleiss,et al.  The measurement of interrater agreement , 2004 .

[17]  Koby Crammer,et al.  On the Algorithmic Implementation of Multiclass Kernel-based Vector Machines , 2002, J. Mach. Learn. Res..

[18]  L. Vygotsky Mind in Society: The Development of Higher Psychological Processes: Harvard University Press , 1978 .

[19]  L. Earl,et al.  Assessment as Learning: Using Classroom Assessment to Maximize Student Learning , 2003 .

[20]  H. Garfinkel Studies in Ethnomethodology , 1968 .

[21]  Jiawei Han,et al.  Data Mining: Concepts and Techniques , 2000 .

[22]  Peter W. Foltz,et al.  An introduction to latent semantic analysis , 1998 .

[23]  M. Scriven Evaluation thesaurus, 4th ed. , 1991 .

[24]  Marti A. Hearst Untangling Text Data Mining , 1999, ACL.

[25]  Jacob Cohen A Coefficient of Agreement for Nominal Scales , 1960 .