' Tools for the trade ' : What makes GCSE marking reliable ?

The Qualifications and Curriculum Authority requires Awarding Bodies to use standardisation procedures (e.g. coordination meetings and providing exemplar scripts marked by the Principal Examiner (PE)) to make GCSE marking reliable. AQA and UCLES collaboratively researched these procedures. In the first experiment with English examiners, it was found that if they were given exemplar scripts at the centre of the marks associated with a level descriptor rather than exemplar scripts at the lowest marks for the level their marking was more severe. In the second experiment, with History examiners it was found that different styles of co-ordination meetings and whether a co-ordination meeting was held did not affect the inter-rater reliability of the marking. This paper reports on a survey of History examiners, which was undertaken after the second experiment. Questionnaires about aspects of standardisation were distributed to the examiners who took part in the experiment and 35 examiners responded. The data were quantitatively and qualitatively analysed. The survey shows that all aspects of standardisation are important, particularly the mark scheme and the co-ordination meeting. This is in contrast to the experimental results (above). Examiners value the opportunity in the co-ordination meeting to develop a 'community of assessment practice' and learn about the application of the mark scheme. The coordination meeting is useful as it gives examiners a feeling of being part of a team, boosts confidence and provides examiners with feedback. The literature review suggests that exemplar scripts and discussion between examiners is important in facilitating reliability. In the survey of History examiners reported here exemplar scripts and discussion between examiners were rated as useful as they facilitated the understanding and application of the mark scheme. The literature says that not everything can be written down and that some understanding of how to apply the mark scheme will remain tacit. But the History examiners surveyed here pointed out that the mark scheme and how it is written are important. They thought that exemplar scripts should be annotated and related to the levels in the mark scheme. The experiment in English illustrated that the way that the exemplar scripts are tied to the mark scheme affects the severity of marking. A principle of a community of practice is that practice is negotiated in a non-hierarchical manner so that there is shared understanding and ownership. Some examiners found hierarchical discussion useful, but others found non-hierarchical discussion useful.