Interrater Objectivity for Field-Based Fundamental Motor Skill Assessment

In the past 10 years, a growing body of evidence has linked fundamental motor skills (FMS) proficiency to physical activity participation (Booth et al., 1997; Booth et al., 2006; Okely, 1999; Okely, Booth, & Patterson, 2001; van Beurden et al., 2003; Wrotniak, Epstein, Dorn, Jones, & Kondilis, 2006). With FMS proficiency becoming more significant for understanding factors that may influence young people’s physical activity, instruments and testing methods used in such research must be valid and reliable. Motor skill assessments on children and adolescents tend to be either process or product oriented. Process assessments (e.g., TGMD-2; Ulrich, 2000) are concerned with how the skill is performed (Burton & Miller, 1998), whereas product assessments (e.g., Bruininks-Oseretsky Test of Motor Proficiency; Bruininks, 1978a, 1978b; or The Movement Assessment Battery for Children M-ABC; Henderson & Sugden, 1992) are based on the skill execution outcome, such as time, distance, or successful attempts (Burton & Miller, 1998). In studies of motor skill proficiency and physical activity, researchers have used both process (Booth et al., 1997; Booth et al., 2006; Okely et al., 2001; van Beurden et al., 2003) and product assessments (Fisher et al., 2005; McKenzie, Sallis, Broyles, Zive, & Nader, 2002; Wrotniak et al., 2006). While both modes are useful in judging the physical competences of children and adolescents, process assessments have the advantage of allowing accurate identification of specific skill components that may need improving (Ulrich, 2000). An important aspect in studies concerning FMS proficiency is interrater objectivity (or interrater reliability), defined as the consistency or agreement in scores obtained from two or more raters (Goodwin, 2001; Posner, Sampson, Caplan, Ward, & Cheney, 1990). In a training setting, interrater objectivity is commonly determined as the relative number of times raters agree with an “expert” rating of skill proficiency (used as a gold standard for comparisons). However, during observation in a field setting, multiple raters may assess FMS proficiency without comparison to an expert rating, so the overall reliability of the raters is key. Additionally, examining the reliability of specific FMSs may help determine which skills are difficult to assess in the field. The Move It Groove It (MIGI) study used the Get Skilled: Get Active process-oriented motor skill assessment (New South Wales Department of Education and Training, 2000) to observe FMSs in a school setting and reported an overall kappa of .61 (van Beurden et al., 2003; van Beurden, Zask, Barnett, & Dietrich, 2002). Interrater reliability was determined during field observation periods on 48 scoring sets for every rater pair (10 raters total); results for each skill were not reported. Other motor skill studies (Booth et al., 1997; Booth et al., 2006; (New South Wales Department of Education and Training, 2000 Okely, 1999; Okely et al., 2001) used Get Skilled: Get Active protocol, but none reported interrater objectivity assessments. The purpose of the current study was to determine interrater objectivity for six FMS using the processoriented motor assessment tool Get Skilled: Get Active by live observation in a school field setting with adolescents. This study was part of a larger project known as the Physical Activity and Skills Study (PASS). Interrater Objectivity for Field-Based Fundamental Motor Skill Assessment