An Empirical Study About Calibration of Adaptive Hints in Web-Based Adaptive Testing Environments

In this paper we present a proposal for introducing hint adaptive selection in an adaptive web-based testing environment. To this end, a discussion of some aspects concerning the adaptive selection mechanism for hints is presented, which results in the statement of two axioms that such hints must fulfil. Then, an empirical study with real students is presented, whose goal is to evaluate a tentative bank of items with their associated hints to determine the usefulness of such hints for different knowledge levels and to calibrate both test items and hints.

[1]  Melvin R. Novick,et al.  Some latent train models and their use in inferring an examinee's ability , 1966 .

[2]  Eduardo Guzmán,et al.  A Brief Introduction to the New Architecture of SIETTE , 2004, AH.

[3]  Joel A. Michael,et al.  Hinting as a Tactic in One-on-One Tutoring , 1996 .

[4]  Carlo Strapparava,et al.  Adaptive Hypermedia and Adaptive Web-Based Systems, 5th International Conference, AH 2008, Hannover, Germany, July 29 - August 1, 2008. Proceedings , 2008, AH.

[5]  L. S. Vygotksy.  Mind in society: the development of higher psychological processes , 1978 .

[6]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[7]  Ricardo Conejo,et al.  SIETTE: A Web-Based Tool for Adaptive Testing , 2004, Int. J. Artif. Intell. Educ..

[8]  Joseph E. Beck,et al.  Macroadapting Animalwatch to Gender and Cognitive Differnces with Respect to Hint Interactivity and Symbolism , 2000, Intelligent Tutoring Systems.

[9]  Wim Jansen,et al.  Multilog: Multiple, Categorical Item Analysis and Test Scoring Using Item Response Theory , 1994 .

[10]  R. Hambleton,et al.  Fundamentals of Item Response Theory , 1991 .

[11]  Cristina Conati,et al.  Procedural Help in Andes: Generating Hints Using a Bayesian Network Student Model , 1998, AAAI/IAAI.

[12]  G. Wells.  Dialogic Inquiry: Towards a Socio-cultural Practice and Theory of Education , 1999 .

[13]  Sophiana Chua Abdullah.  Student modelling by adaptive testing : a knowledge-based approach , 2003 .

[14]  Howard Wainer,et al.  Computerized Adaptive Testing: A Primer , 2000 .

[15]  Rosemary Luckin,et al.  Getting to Know Me: Helping Learners Understand Their Own Learning Needs through Metacognitive Scaffolding , 2002, Intelligent Tutoring Systems.

[16]  Vincent Aleven,et al.  Applying cognitive modeling to meta-cognitive skills , 2004 .