Overcoming Research Design Issues using Rasch Measurement: The StatSmart Project

Longitudinal research in educational settings is notoriously difficult, and when the study is on a large scale, becomes even more problematic. The StatSmart project aims to measure changes in both teachers and their students over a period of three years. Using previously validated instruments for students, three overlapping test forms will be used with a rotating design so that each student will only complete each test once. Teacher change will be measured using a profiling instrument at the start, middle and end of the study. Rasch measurement provides for this type of design and should provide good information even with the complexity of students changing classes or schools. The issues associated with complex designs of this nature are discussed.

[1]  A. Schoenfeld,et al.  Improving Educational Research:Toward a More Useful, More Influential, and Better-Funded Enterprise , 2003 .

[2]  W. Fisher.  The Rasch Debate: Validity and Revolution in Educational Measurement. , 1991 .

[3]  Jane Watson,et al.  Developing Concepts of Sampling , 2000 .

[4]  'You've been NERFed!' Dumbing down the academy: National Educational Research Forum: 'a national strategy ? consultation paper': a brief and bilious response , 2001 .

[5]  Jane Watson,et al.  Student change associated with teachers' professional learning , 2007 .

[6]  Aj McIntosh,et al.  Mental Computation Competence Across Years 3 to 10 , 2002 .

[7]  Kari Augustine.  Mystery Three, Find It, and Treasure Hunt , 2006 .

[8]  Jane Watson,et al.  A longitudinal study of student understanding of chance and data , 2006 .

[9]  L. Shulman.  Assessment for Teaching: An Initiative for the Profession. , 1987 .

[10]  Jane Watson,et al.  Fairness of dice: a longitudinal study of students' beliefs and strategies for making judgements , 2003 .

[11]  Georg Rasch,et al.  Probabilistic Models for Some Intelligence and Attainment Tests , 1981, The SAGE Encyclopedia of Research Design.

[12]  J. Hiebert.  Relationships between research and the NCTM standards , 1999 .

[13]  Sue Dockett,et al.  Australian association for research in education , 1989 .

[14]  Jane Watson.  Profiling Teachers' Competence and Confidence to Teach Particular Mathematics Topics: the Case of Chance and Data , 2001 .

[15]  Jane Watson,et al.  Students' Appreciation of Expectation and Variation as a Foundation for Statistical Understanding , 2007 .

[16]  D. C. Phillips,et al.  On the Science of Education Design Studies , 2003 .

[17]  G. Masters,et al.  Rating scale analysis , 1982 .

[18]  P. Griffin,et al.  A 20-Year Study of Mathematics Achievement. , 2006 .

[19]  G. Masters.  A rasch model for partial credit scoring , 1982 .

[20]  Margaret Wu,et al.  ACER conquest: generalised item response modelling software , 1998 .

[21]  L. Shulman.  Knowledge and Teaching: Foundations of the New Reform , 1987 .

[22]  Jane Watson,et al.  Statistical Literacy: From Idiosyncratic to Critical Thinking , 2004, Curricular Development in Statistics Education IASE Roundtable Conference.

[23]  Jane Watson,et al.  Statistical Literacy at School: Growth and Goals , 2006 .

[24]  Abbie Brown,et al.  Design experiments: Theoretical and methodological challenges in creating complex interventions in c , 1992 .

[25]  Equating of Tests , 1999 .

[26]  Jane Watson,et al.  The measurement of school students' understanding of statistical variation , 2003 .

[27]  Patrick Griffin,et al.  The Literacy Profiles in Practice: Toward Authentic Assessment , 2001 .

[28]  Harold L. Schoen,et al.  Teacher Variables That Relate to Student Achievement When Using a Standards-Based Curriculum. , 2003 .

[29]  Mark Wilson.  Measurement of Developmental Levels , 1999 .

[30]  Jane Watson,et al.  Profiling Teacher Change Resulting from a Professional Learning Program in Middle School Numeracy , 2005 .

[31]  C. Fox,et al.  Applying the Rasch Model: Fundamental Measurement in the Human Sciences , 2001 .

[32]  S. Loucks-Horsley,et al.  Research on Professional Development for Teachers of Mathematics and Science: The State of the Scene , 1999 .

[33]  S. Nisbet,et al.  Surveying primary teachers about compulsory numeracy testing: Combining factor analysis with rasch analysis , 2006 .