A comparative analysis of the consistency and difference among teacher-assessment, student self-assessment and peer-assessment in a Web-based portfolio assessment environment for high school students

This study explored the consistency and difference of teacher-, student self- and peer-assessment in the context of Web-based portfolio assessment. Participants were 72 senior high school students enrolled in a computer application course. Through the assessment system, the students performed portfolio creation, inspection, self- and peer-assessment; three teachers reviewed portfolios and evaluated learning performances. There were significant differences in the results of the three assessment methods, among which teacher-raters adopted the most rigorous scoring standards, while peer-raters tended to use the most lax standards. The results of self- and teacher-assessment were discovered to be consistent; however, consistency was not discovered between self- and peer-assessment as well as peer- and teacher-assessment. In analyzing their consistency with the end-of-course examination, teacher- and self-assessment demonstrated high consistency, whereas peer-assessment showed a low level of consistency.

[1]  P. Shefler.  The adult learner. , 1971, Washington State journal of nursing.

[2]  K. Topping,et al.  Formative Peer Assessment of Academic Writing Between Postgraduate Students , 2000 .

[3]  Sue Bloxham *,et al.  Understanding the rules of the game: marking peer assessment as a medium for developing students' conceptions of assessment , 2004 .

[4]  Nicole A. Buzzetto-More,et al.  A Comparative Analysis of Common E-Portfolio Features and Available Platforms , 2007 .

[5]  Helen C. Barrett.  Researching Electronic Portfolios and Learner Engagement: The REFLECT Initiative , 2007 .

[6]  Jyh-Chong Liang,et al.  The development of science activities via on-line peer assessment: the role of scientific epistemological views , 2009 .

[7]  Chin-Chung Tsai,et al.  On-line peer assessment and the role of the peer feedback: A study of high school computer course , 2007, Comput. Educ..

[8]  Michael F. Young,et al.  Reliability and Validity of an Ecologically-Grounded Student Teacher Electronic Portfolio Rubric , 2008 .

[9]  Eric Zhi-Feng Liu,et al.  Web based peer assessment: attitude and achievement , 2001, IEEE Trans. Educ..

[10]  P. Miller,et al.  The Effect of Scoring Criteria Specificity on Peer and Self-assessment , 2003 .

[11]  Alain Jaillet,et al.  Can Online Peer Assessment be Trusted? , 2009, J. Educ. Technol. Soc..

[12]  F. Prins,et al.  Formative peer assessment in a CSCL environment: a case study , 2005 .

[13]  A. Gélinas,et al.  Issues of reliability and validity in using portfolio assessment to measure foreign language teacher performance , 1998 .

[14]  Helen C. Barrett,et al.  Balancing the Two Faces of ePortfolios , 2010 .

[15]  Elena Barberà,et al.  Mutual feedback in e-portfolio assessment: an approach to the netfolio system , 2009, Br. J. Educ. Technol..

[16]  N. Falchikov,et al.  Student Peer Assessment in Higher Education: A Meta-Analysis Comparing Peer and Teacher Marks , 2000 .

[17]  Inci Morgil,et al.  Reliability of Portfolio : A Closer Look at Findings from Recent Publications , 2008 .

[18]  James D. Russell,et al.  Using portfolios in education technology courses , 1999 .

[19]  Chi-Cheng Chang.  Enhancing self-perceived effects using Web-based portfolio assessment , 2008, Comput. Hum. Behav..

[20]  Robert L. Johnson,et al.  Using Portfolios in Program Evaluation: An Investigation of Interrater Reliability , 2000 .

[21]  Carol Derham,et al.  Digital Professional Portfolios of Preservice Teaching: An Initial Study of Score Reliability and Validity , 2007 .

[22]  Chi-Cheng Chang,et al.  Use and performances of Web-based portfolio assessment , 2009, Br. J. Educ. Technol..

[23]  Eric Zhi-Feng Liu,et al.  Web-based peer review: the learner as both adapter and reviewer , 2001, IEEE Trans. Educ..

[24]  C. Beth Burch.  Creating a Two-Tiered Portfolio Rubric. , 1997 .

[25]  Christian D. Schunn,et al.  Validity and reliability of scaffolded peer assessment of writing from instructor and student perspectives , 2006 .

[26]  Benjamin S. Bloom,et al.  A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives , 2000 .

[27]  Yao-Ting Sung,et al.  The design and application of a web-based self- and peer-assessment system , 2005, Comput. Educ..

[28]  Aliisa Mylonas,et al.  Developing Procedures for Implementing Peer Assessment in Large Classes Using an Action Research Process , 2002 .

[29]  Chao-hsiu Chen,et al.  The implementation and evaluation of a mobile self- and peer-assessment system , 2010, Comput. Educ..

[30]  M. Jenkins,et al.  Unfulfilled Promise: formative assessment using computer-aidedassessment , 2005 .

[31]  Annemarie Hauf,et al.  Computers in education , 1983 .

[32]  Eric Zhi-Feng Liu,et al.  Web-based peer assessment: feedback for students with various thinking-styles , 2001, J. Comput. Assist. Learn..

[33]  Chin-Chung Tsai,et al.  University Students’ Perceptions of and Attitudes Toward (Online) Peer Assessment , 2006 .

[34]  Cynthia C Gadbury-Amyot,et al.  Validity and reliability of portfolio assessment of competency in a baccalaureate dental hygiene program. , 2003, Journal of dental education.

[35]  P. Sadler,et al.  The Impact of Self- and Peer-Grading on Student Learning , 2006 .

[36]  Phil Davies,et al.  Computerized Peer Assessment , 2000 .

[37]  Yao-Ting Sung,et al.  Evaluating Proposals for Experiments: An Application of Web-Based Self-Assessment and Peer-Assessment , 2003 .

[38]  Helen C. Barrett,et al.  Online personal learning environments: structuring electronic portfolios for lifelong and life‐wide learning , 2009 .

[39]  Chin-Chung Tsai,et al.  Conceptions of and approaches to learning through online peer assessment , 2010 .

[40]  Azzedine Lansari,et al.  E-portfolio Assessment System for an Outcome-Based Information Technology Curriculum , 2009, J. Inf. Technol. Educ..

[41]  Chi-Cheng Chang,et al.  Using a Web-based portfolio assessment system to elevate project-based learning performances , 2011, Interact. Learn. Environ..

[42]  Shyan-Ming Yuan,et al.  Developing science activities through a networked peer assessment system , 2002, Comput. Educ..