Developing and validating instrument of alignment understanding with school assessment practice

Article history: Received 17 August 2016 Received in revised form 20 October 2016 Accepted 30 October 2016 This study aims to develop and validate the Understanding Alignment with School Assessment Practice instrument (2KAPS). The instrument consists of 27 items. The 2KAPS questionnaire validation involves 109 teachers who taught Form 1 and Form 2 students (teachers directly involved in the School Assessment Practice implementation) in one district in Perak. The instrument was developed in several stages such as building the understanding alignment model and an assessment practice generated based on alignment models from the literature review, determining the main constructs in the assessment expectations, determining the chosen practice in line with the assessment expectations, the use of the Likert scale with three categories (Full agreement=3, Lack of agreement=2 and No Agreement=1) which indicated that there was an alignment between the assessment practice and the teachers’ assessment understanding, acquiring the content validity from experts and the analysis of items using the Rasch Measurement Model. The instrument validity and reliability had been conducted by identifying the Rasch fit statistics, item difficulty, unidimensionality, item reliability as well as 2KAPS item map. The Rasch analysis showed that the item reliability was valued at 0.92 while the Cronbach Alpha value was 0.90. All the items fit the model as their MNSQ values were between 0.7 and 1.35. The dispersion of items from 2KAPS data was 3.29 which indicated the existence of 3 to 4 item strata. No item showed a negative point measure correlation or less than 0.2 and this generally indicated that the item discrimination was very good. The data showed that the mean for a person was measured at 1.19 logits with a standard deviation of 1.12 logits while the item mean value was zero with a standard deviation of 0.52. This indicates that the position of item and person does not fully match and thus shows a medium difficulty. The overall item quality was good and all 27 items of 2KAPS were retained.

[1]  G. Boulton‐Lewis.  Teaching for quality learning at university , 2008 .

[2]  R. Stiggins.  Assessment Crisis: The Absence of Assessment for Learning , 2002 .

[3]  C. Buckendahl,et al.  Informing State Assessment from the Local Level: A District's Reflections. , 2001 .

[4]  N. Webb.  Criteria for Alignment of Expectations and Assessments in Mathematics and Science Education. Research Monograph No. 6. , 1997 .

[5]  M. R. Espejo.  Applying the Rasch Model: Fundamental Measurement in the Human Sciences , 2004 .

[6]  Jann E. Freed,et al.  Learner-Centered Assessment on College Campuses: Shifting the Focus from Teaching to Learning , 1999 .

[7]  W. Spady.  Outcome-based education : critical issues and answers , 1995 .

[8]  James H. McMillan,et al.  Formative classroom assessment : theory into practice , 2007 .

[9]  Valentina Klenowski,et al.  Assessment for learning in the accountability era: queensland, australia , 2011 .

[10]  Michele T. Ohlsen.  Classroom Assessment Practices of Secondary School Members of Nctm , 2007 .

[11]  Descriptors Educational,et al.  of Educational Measurement , 1988 .

[12]  Lindsay I. Smith,et al.  A tutorial on Principal Components Analysis , 2002 .

[13]  Norman L. Webb,et al.  Determining Alignment of Expectations and Assessments in Mathematics and Science Education. , 1997 .

[14]  J. Biggs.  Enhancing teaching through constructive alignment , 1996 .

[15]  J. Linacre,et al.  Sample size and item calibration stability , 1994 .

[16]  R. Linn.  Educational measurement, 3rd ed. , 1989 .

[17]  P. Moss.  Shifting Conceptions of Validity in Educational Measurement: Implications for Performance Assessment , 1992 .

[18]  Fred M. Newmann.  Beyond Common Sense in Educational Restructuring , 1993 .