Monitoring Online Tests through Data Visualization

We present an approach and a system to let tutors monitor several important aspects related to online tests, such as learner behavior and test quality. The approach includes the logging of important data related to learner interaction with the system during the execution of online tests and exploits data visualization to highlight information useful to let tutors review and improve the whole assessment process. We have focused on the discovery of behavioral patterns of learners and conceptual relationships among test items. Furthermore, we have led several experiments in our faculty in order to assess the whole approach. In particular, by analyzing the data visualization charts, we have detected several previously unknown test strategies used by the learners. Last, we have detected several correlations among questions, which gave us useful feedbacks on the test quality.

[1]  Daniel A. Keim,et al.  43 Visual Data-Mining Techniques* , 2004 .

[2]  Daniel A. Keim,et al.  Visual exploration of large data sets , 2001, Commun. ACM.

[3]  Clyde A. Paul,et al.  The Relationship between the Time Taken to Complete an Examination and the Test Score Received. , 1980 .

[4]  Gary K. L. Tam,et al.  A 3D Geometry Search Engine in Support of Learning , 2007, ICWL.

[5]  Sébastien George,et al.  Tracking, analyzing, and visualizing learners' activities on discussion forums , 2007 .

[6]  Lucinda McClain.  Behavior during Examinations: A Comparison of “A,” “C,” and “F” Students , 1983 .

[7]  A KeimDaniel.  Visual exploration of large data sets , 2001 .

[8]  Hiroshi Kato,et al.  Promotion of self-assessment for learners in online discussion using the visualization software , 2005, CSCL.

[9]  Ben Shneiderman,et al.  Show Me! Guidelines for producing recorded demonstrations , 2005, 2005 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC'05).

[10]  Scott Boag,et al.  XQuery 1.0 : An XML Query Language , 2007 .

[11]  Matthew O. Ward,et al.  Introduction to data visualization , 2001 .

[12]  John A. Bath,et al.  Answer-Changing Behavior on Objective Examinations , 1967 .

[13]  Daniel A. Keim,et al.  Information Visualization and Visual Data Mining , 2002, IEEE Trans. Vis. Comput. Graph..

[14]  Jon Hill,et al.  Tracking and Visualisation of Student Use of Online Learning Materials in a Large Undergraduate Course , 2007, ICWL.

[15]  Celmar Guimaraes da Silva,et al.  Learning Management Systems' database exploration by means of Information Visualization-based query tools , 2007, Seventh IEEE International Conference on Advanced Learning Technologies (ICALT 2007).

[16]  Domenico Talia,et al.  Eureka!: an interactive and visual knowledge discovery tool , 2004, J. Vis. Lang. Comput..

[17]  Christopher A. Badurek,et al.  Review of Information visualization in data mining and knowledge discovery by Usama Fayyad, Georges G. Grinstein, and Andreas Wierse. Morgan Kaufmann 2002 , 2003 .

[18]  Ronald J. Brachman,et al.  Brief Application Description; Visual Data Mining: Recognizing Telephone Calling Fraud , 2004, Data Mining and Knowledge Discovery.

[19]  Susumu Yamasaki,et al.  A Framework for Adaptive e-Learning Systems in Higher Education with Information Visualization , 2007, 2007 11th International Conference Information Visualization (IV '07).

[20]  Hong Zhao,et al.  Applying data mining to detect fraud behavior in customs declaration , 2002, Proceedings. International Conference on Machine Learning and Cybernetics.

[21]  Haralampos Karanikas,et al.  Visual Techniques for the Interpretation of Data Mining Outcomes , 2005, Panhellenic Conference on Informatics.

[22]  Chrisina Draganova.  Asynchronous JavaScript Technology and XML (AJAX) , 2007 .

[23]  F. Ferrucci,et al.  A Web based tool for assessment and self-assessment , 2004, ITRE 2004. 2nd International Conference Information Technology: Research and Education.

[24]  John R. Anderson,et al.  What can a mouse cursor tell us more?: correlation of eye/mouse movements on web browsing , 2001, CHI Extended Abstracts.

[25]  Heidrun Schumann,et al.  A Flexible Approach for Visual Data Mining , 2002, IEEE Trans. Vis. Comput. Graph..

[26]  Ioannis Kopanakis,et al.  Visual data mining modeling techniques for the visualization of mining outcomes , 2003, J. Vis. Lang. Comput..

[27]  J. Carter,et al.  Addressing student cheating: definitions and solutions , 2002, ITiCSE-WGR '02.

[28]  Ryan Shaun Joazeiro de Baker,et al.  Off-task behavior in the cognitive tutor classroom: when students "game the system" , 2004, CHI.

[29]  Tiziana Catarci,et al.  VidaMine: a visual data mining environment , 2004, J. Vis. Lang. Comput..

[30]  Alfred Inselberg.  Visualization and data mining of high-dimensional data , 2002 .

[31]  Mao Lin Huang,et al.  A new visualization approach for supporting knowledge management and collaboration in e-learning , 2004, Proceedings. Eighth International Conference on Information Visualisation, 2004. IV 2004..

[32]  Heikki Mannila,et al.  Principles of Data Mining , 2001, Undergraduate Topics in Computer Science.

[33]  Jiawei Han,et al.  Geographic Data Mining and Knowledge Discovery , 2001 .

[34]  Urska Demsar,et al.  Data mining of geospatial data: combining visual and automatic methods , 2006 .

[35]  Maria Francesca Costabile,et al.  Visualizing Association Rules in a Framework for Visual Data Mining , 2005, From Integrated Publication and Information Systems to Virtual Information and Knowledge Environments.

[36]  Georges G. Grinstein,et al.  A survey of visualizations for high-dimensional data mining , 2001 .

[37]  D. Carpenter,et al.  The current state of research on academic dishonesty among engineering students , 2001, 31st Annual Frontiers in Education Conference. Impact on Engineering and Science Education. Conference Proceedings (Cat. No.01CH37193).

[38]  Jiawei Han,et al.  Geographic data mining and knowledge discovery: An overview , 2009 .

[39]  Download Book,et al.  Information Visualization in Data Mining and Knowledge Discovery , 2001 .

[40]  P. Mahadevan,et al.  An overview , 2007, Journal of Biosciences.

[41]  Mihael Ankerst,et al.  Visual Data Mining , 2001, Encyclopedia of GIS.

[42]  Luca Chittaro,et al.  Data mining on temporal data: a visual approach and its clinical application to hemodialysis , 2003, J. Vis. Lang. Comput..

[43]  Padhraic Smyth,et al.  From Data Mining to Knowledge Discovery in Databases , 1996, AI Mag..

[44]  SheardJudy,et al.  Addressing student cheating , 2002 .

[45]  John B. Best,et al.  Item Difficulty and Answer Changing , 1979 .

[46]  Vania Dimitrova,et al.  Visualising student tracking data to support instructors in web-based distance education , 2004, WWW Alt. '04.

[47]  Ronna C. Turner,et al.  Techniques for Detection of Cheating on Standardized Tests using SAS ® , 2001 .

[48]  Norma Banas,et al.  Visualization , 1968, Machine-mediated learning.

[49]  Daniel A. Keim,et al.  Pixel based visual data mining of geo-spatial data , 2004, Comput. Graph..