Comprehensive evaluation criteria for English learning websites using expert validity surveys

This study aimed to develop a set of evaluation criteria for English learning websites. These criteria can assist English teachers/web designers in designing effective websites for their English courses and can also guide English learners in screening for appropriate and reliable websites to use in increasing their English ability. To fulfill our objective, we employed a three-phase research procedure: (a) establishing a preliminary set of criteria from a thorough review of the literature, (b) evaluating and refining the preliminary criteria by conducting interviews with in-service teachers and learners, and (c) validating and finalizing the criteria according to expert validity surveys. The established criteria have 46 items, classified into 6 categories (the number of items within the category) - general information (12), integrated English learning (13), listening (4), speaking (6), reading (5), and writing (6). The general information evaluates the authority, accuracy, and format of the learning websites. The integrated English learning evaluates the overall information relevant to English learning materials as well as the common features of the four language skills. The criteria for listening, speaking, reading, and writing, for example, examine the suitable intonation, skills of discourse, classification of reading articles by their attributes, and the proper use of discussion boards for students when practicing their writing skills. Based on qualitative and quantitative analysis of the interviews and expert validity surveys, we confirmed the effectiveness of the developed evaluation criteria with satisfactory indexes of inter-rater reliability, content validity, and factorial validity.

[1]  Mark Warschauer,et al.  Internet for English teaching , 2000 .

[2]  Thomas C. Reeves.  A Model of the Effective Dimensions of Interactive Learning on the World Wide Web , 1997 .

[3]  D. Rubio,et al.  Objectifying Content Validity: Conducting a Content Validity Study in Social Work Research , 2003 .

[4]  Sigmar-Olaf Tergan,et al.  Checklists for the Evaluation of Educational Software: Critical Review and Prospects. , 1998 .

[5]  Gerry Abbott,et al.  The Teaching Of English As An International Language: A Practical Guide , 1981 .

[6]  Charles M. Browne,et al.  New Perspectives on CALL for Second Language Classrooms , 2013 .

[7]  J. S. Ellenberger,et al.  The American Association of Law Libraries , 1963 .

[8]  T. Ballard.  American Association of Law Libraries , 1996 .

[9]  Steve Cushion,et al.  From Access to Acceptability: Exploiting the Web to Design a New CALL Environment , 2000 .

[10]  Lawrence J. Najjar.  Principles of Educational Multimedia User Interface Design , 1998, Hum. Factors.

[11]  Angela Chambers,et al.  ICT and language learning : a European perspective , 2001 .

[12]  Alastair G. Smith,et al.  Testing the Surf: Criteria for Evaluating Internet Information Resources , 1997 .

[13]  Carol A. Chapelle,et al.  Multimedia CALL: Lessons to be Learned from Research on Instructed SLA , 1998 .

[14]  Lysbeth B. Chuck.  Emmerce in Academia: A Cinderella Fable. , 1999 .

[15]  Rebecca L. Oxford,et al.  Language learning motivation : pathways to the new century , 1996 .

[16]  Bernard Susser.  A defense of checklists for courseware evaluation , 2001, ReCALL.

[17]  Mark Peterson,et al.  Directions for Development in Hypermedia Design , 2000 .

[18]  Kevin Oliver,et al.  Evaluation criteria and indicators of quality for Internet resources , 1997 .

[19]  L. Davis,et al.  Selection and use of content experts for instrument development. , 1997, Research in nursing & health.

[20]  Min Liu,et al.  A Look at the Research on Computer-Based Technology Use in Second Language Learning , 2002 .

[21]  Laura Gordon-Murnane.  Evaluating Net Evaluators. , 1999 .

[22]  E. Horwitz.  Psychology for Language Teachers: A Social Constructivist Approach , 2002 .

[23]  Priscilla Garrido Comer,et al.  A Methodology for Software Evaluation , 1998 .

[24]  Joseph M. Furner,et al.  The Educational Software/Website Effectiveness Survey , 2004 .

[25]  Sung Heum Lee.  Criteria for Evaluating and Selecting Multimedia Software for Instruction. , 1996 .

[26]  Mark Peterson.  Creating Hypermedia Learning Environments: Guidelines for Designers. , 1998 .

[27]  Joy Egbert,et al.  CALL environments : research, practice, and critical issues , 1999 .

[28]  Colin Lonfils,et al.  How to Design User-Friendly CALL Interfaces , 2001 .

[29]  David Lasagabaster,et al.  Students' evaluation of CALL software programs , 2003 .

[30]  L. Davis.  Instrument review: Getting the most from a panel of experts , 1992 .