Diagnosing skills of statistical hypothesis testing using the Rule Space Method

Abstract This study illustrated the use of the Rule Space Method to diagnose students’ proficiencies in, skills and knowledge of statistical hypothesis testing. Participants included 96 undergraduate and, graduate students, of whom 94 were classified into one or more of the knowledge states identified by, the rule space analysis. Analysis at the level of proficiency groups showed that the critical difference, between low and medium proficiency groups was the understanding of statistical concepts and, knowledge while the critical skill discriminating the medium proficiency group from the high, proficiency group was to mange complex computational procedures. In addition, attribute profiles of, two students showed how students with the same total score can possess different strengths and, weaknesses.

[1]  I. Kostin,et al.  The Subskills of Reading: Rule‐space Analysis of a Multiple‐choice Test of Second Language Reading Comprehension , 1997 .

[2]  Bronwen Cowie,et al.  The Characteristics of Formative Assessment in Science Education. , 2001 .

[3]  K. Tatsuoka,et al.  Applying an IRT-Based Cognitive Diagnostic Model to Diagnose Students' Knowledge States in Multiplication and Division with Exponents. , 1993 .

[4]  Kikumi K. Tatsuoka,et al.  Bug distribution and statistical pattern classification , 1987 .

[5]  R. Lesh,et al.  Handbook of Research Design in Mathematics and Science Education. , 2000 .

[6]  Clifford Konold,et al.  Issues in Assessing Conceptual Understanding in Probability and Statistics , 1995 .

[7]  Angela M. O'Donnell,et al.  Representativeness in Statistical Reasoning: Identifying and Assessing Misconceptions , 2001 .

[8]  K. Tatsuoka.  Toward an Integration of Item-Response Theory and Cognitive Error Diagnosis. , 1987 .

[9]  Kay Lipson.  ASSESSING UNDERSTANDING IN STATISTICS , 2007 .

[10]  Michael E. Martinez,et al.  Extending the Rule Space Methodology to a Semantically-Rich Domain: Diagnostic Assessment in Architecture , 1998 .

[11]  Matthias von Davier,et al.  HIERARCHICAL GENERAL DIAGNOSTIC MODELS , 2007 .

[12]  Menucha Birenbaum,et al.  TASK SPECIFICATION CHART: A KEY TO A BETTER UNDERSTANDING OF TEST RESULTS , 1985 .

[13]  Beth Chance,et al.  Assessment in Statistics Education: Issues and Challenges , 2000 .

[14]  Steven P. Schacht,et al.  What's Funny about Statistics? A Technique for Reducing Student Anxiety. , 1990 .

[15]  M. J. Subkoviak.  ESTIMATING RELIABILITY FROM A SINGLE ADMINISTRATION OF A CRITERION-REFERENCED TEST* , 1976 .

[16]  Brenda Sugrue,et al.  The Lack of Fidelity between Cognitively Complex Constructs and Conventional Test Development Practice. , 2005 .

[17]  Mark J. Gierl,et al.  The Attribute Hierarchy Method for Cognitive Assessment: A Variation on Tatsuoka's Rule-Space Approach , 2004 .

[18]  Ronald A. Berk,et al.  A Guide to Criterion-Referenced Test Construction , 1984 .

[19]  Curtis Tatsuoka,et al.  Data analytic methods for latent partially ordered classification models , 2002 .

[20]  Anthony E. Kelly,et al.  DIAGNOSING KNOWLEDGE STATES IN ALGEBRA USING THE RULE SPACE MODEL , 1992 .

[21]  Kikumi Tatasuoka.  Use of Generalized Person-Fit Indexes, Zetas for Statistical Pattern Classification. , 1996 .

[22]  Anthony J. Onwuegbuzie,et al.  Attitudes toward Statistics Assessments , 2000 .

[23]  F. Davis.  Criterion-Referenced Tests. , 1971 .

[24]  Matthias von Davier,et al.  A GENERAL DIAGNOSTIC MODEL APPLIED TO LANGUAGE TESTING DATA , 2005 .

[25]  Kikumi K. Tatsuoka,et al.  Boolean Algebra Applied to Determination of Universal Set of Knowledge States , 1991 .

[26]  M. Birenbaum,et al.  Diagnostic assessment in TIMSS-R: Between-countries and within-country comparisons of eighth graders' mathematics performance , 2004 .

[27]  H. G. Schmidt,et al.  Assessing Knowledge Structures in a Constructive Statistical Learning Environment , 2002 .

[28]  D. Vere-Jones,et al.  Proceedings of the Second International Conference on Teaching Statistics , 1989 .

[29]  Joan Garfield.  ASSESSING STATISTICAL REASONING , 2003, STATISTICS EDUCATION RESEARCH JOURNAL.

[30]  S. Chipman,et al.  Cognitively diagnostic assessment , 1995 .

[31]  B Garfield Joan,et al.  Beyond Testing and Grading: Using Assessment To Improve Student Learning , 1994 .

[32]  E. Maris.  Estimating multiple classification latent class models , 1999 .

[33]  Anthony J. Nitko,et al.  Solving Problems Is Not Enough: Assessing and Diagnosing the Ways in Which Students Organize Statistical Concepts. , 1990 .

[34]  James E. Corter,et al.  Patterns of Diagnosed Mathematical Content and Process Skills in TIMSS-R Across a Sample of 20 Countries , 2004 .

[35]  Kikumi K. Tatsuoka,et al.  A Probabilistic Model for Diagnosing Misconceptions By The Pattern Classification Approach , 1985 .

[36]  P. Black,et al.  Assessment and Classroom Learning , 1998 .

[37]  Subgroup Differences on the GRE Quantitative Test Based on the Underlying Cognitive Processes and Knowledge , 2000 .