Internet Research Support System: An Application for Immediate Feedback in Authentic Internet Research

A prototype Internet Research Support System is presented based on earlier work on the modeling of online reading behavior and evaluating the reading process and its outcomes. Users complete a reading task by browsing the Web and writing a short essay. The online browsing data is captured in real-time, which, along with the essay, reflects the participants’ online reading process. Verbatim Quotient Detection is possible which monitors the proportion of material copied from the Internet. A Latent Semantic Analysis component to evaluate sentence-to-sentence cohesion in the final essay is also described, and is in the process of being integrated with the main system. The system evaluation capabilities provide the participants with immediate feedback that is specific to their final essay in order to help them increase their online reading comprehension. It also archives data for research and evaluation purposes. One hundred and seventeen undergraduate students and one faculty member participated in two experiments to test the system where the outcomes were marked by the system and human markers. Analysis of the online reading activity demonstrated a negative correlation between the human marker’s score and the number of verbatim words in the final essay. Furthermore, the analysis showed that when the whole Internet was available, even with limited time and a common task, there was significant variation in participants’ search strategies, search terms and research processes.

[1]  Ion Juvina,et al.  Predicting user preferences: from semantic to pragmatic metrics of Web navigation behavior , 2004 .

[2]  John E. McEneaney.  Navigational correlates of comprehension in hypertext , 2000, HYPERTEXT '00.

[3]  Daniel E. Rose,et al.  Understanding user goals in web search , 2004, WWW '04.

[4]  Nasiroh Omar,et al.  Investigating an approach for online reading assessment , 2005, Fifth IEEE International Conference on Advanced Learning Technologies (ICALT'05).

[5]  John E. McEneaney.  New Approaches to Data Collection and Analysis in Online Reading Studies. , 1999 .

[6]  Gerry Stahl,et al.  Developing Summarization Skills through the Use of LSA-Based Feedback , 2000, Interact. Learn. Environ..

[7]  Julie Coiro.  Reading comprehension on the Internet: Expanding our understanding of reading comprehension to encompass new literacies , 2003 .

[8]  Patrick F. Reidy.  An Introduction to Latent Semantic Analysis , 2009 .

[9]  Zhenyu Liu,et al.  Automatic identification of user goals in Web search , 2005, WWW '05.

[10]  Christopher Olston,et al.  ScentTrails: Integrating browsing and searching on the Web , 2003, TCHI.

[11]  Eszter Hargittai,et al.  Beyond logs and surveys: In-depth measures of people's web use skills , 2002, J. Assoc. Inf. Sci. Technol..

[12]  Allan Collins,et al.  Assessment and technology , 1993, CACM.

[13]  Colin Harrison,et al.  Postmodern Principles for Responsive Reading Assessment. , 2004 .

[14]  Athanasios Tsintsifas,et al.  A Framework for the Computer Based Assessment of Diagram Based Coursework , 2002 .

[15]  Claudia Leacock,et al.  Automated evaluation of essays and short answers , 2001 .

[16]  Mark S. Ackerman,et al.  The perfect search engine is not enough: a study of orienteering behavior in directed search , 2004, CHI.

[17]  Laurie A. Henry.  Information Search Strategies on the Internet: A Critical Component of New Literacies , 2005, Webology.

[18]  Nasiroh Omar,et al.  Evaluating Real-time Online Research Data (RORD) and Verbatim Quotient Detection (VQD): Low Inference Tools to Monitor Outcomes of Unconstrained Authentic Internet Research , 2006 .

[19]  Mika Käki,et al.  Information search and re-access strategies of experienced web users , 2005, WWW '05.

[20]  Stuart Hepplestone,et al.  Strategies for coping with unexpected uptake of CBA , 2003 .

[21]  Amy C. Kimme Hea,et al.  After hypertext: Other ideas , 2003 .

[22]  C. Schunn.  The Presence and Absence of Category Knowledge in LSA , 2020, Proceedings of the Twenty First Annual Conference of the Cognitive Science Society.

[23]  Colin Harrison,et al.  Understanding Reading Development , 2003 .

[24]  O. Mason,et al.  Automated free text marking with Paperless School , 2002 .

[25]  Surendra Prasad,et al.  Automatic Evaluation of Students’ Answers using Syntactically Enhanced LSA , 2003, HLT-NAACL 2003.

[26]  Benoît Lemaire,et al.  A System to Assess the Semantic Content of Student Essays , 2001 .

[27]  Alan E. Farstrup,et al.  High-stakes assessments in reading: A position statement of the international reading association , 1999 .

[28]  Peter Bruza,et al.  Interactive Internet search: keyword, directory and query reformulation mechanisms compared , 2000, SIGIR '00.

[29]  Colin Higgins,et al.  Formative computer based assessment in diagram based domains , 2006, ITICSE '06.