Key Practices in the English Language Arts (ELA): Linking Learning Theory, Assessment, and Instruction

This paper presents a framework intended to link the following assessment development concepts into a systematic framework: evidence-centered design (ECD), scenario-based assessment (SBA), and assessment of, for, and as learning. The context within which we develop this framework is the English language arts (ELA) for K-12 students, though the framework could easily be applied to cover reading, writing, and critical thinking skills from pre-K through college. Central to the framework is the concept of a key practice, drawn from constructivist learning theory, which emphasizes the purposeful social context within which skills are recruited and organized to carry out complex literacy tasks. We argue that key practices provide a key link between existing CBAL™ ELA learning progressions (defined as part of a student model for literacy skills) and the structure of well-designed SBAs. This structure enables us to design assessments that model a key practice, supporting the systematic creation of task sequences that can be used to support both instruction and assessment.

[1]  P. Deane,et al.  Building and Sharing Knowledge Key Practice: What Do You Know, What Don't You Know, What Did You Learn? , 2015 .

[2]  Ets Rr,et al.  Preliminary Reading Literacy Assessment Framework: Foundation and Rationale for Assessment and System Design , 2013 .

[3]  D. Jonassen,et al.  Activity theory as a framework for designing constructivist learning environments , 1999 .

[4]  Robert J. Mislevy,et al.  Evidence-Centered Design of Epistemic Games: Measurement Principles for Complex Learning Environments. , 2010 .

[5]  S. Wineburg.  Historical Problem Solving: A Study of the Cognitive Processes Used in the Evaluation of Documentary and Pictorial Evidence , 1991 .

[6]  B. Rogoff.  Apprenticeship in Thinking: Cognitive Development in Social Context , 1990 .

[7]  Susan R. Goldman,et al.  Choosing and using multiple information sources: Some new findings and emergent issues , 2011 .

[8]  Jan Herrington,et al.  An instructional design framework for authentic learning environments , 2000 .

[9]  Graham S. Maxwell,et al.  Contextualising Authentic Assessment , 1999 .

[10]  S. Wineburg,et al.  Historical Thinking and Other Unnatural Acts: Charting the Future of Teaching the Past , 2002 .

[11]  R. Mislevy.  Evidence-centered design for simulation-based assessment. , 2011, Military medicine.

[12]  J. Willinsky,et al.  Digital Literacy , 2013 .

[13]  Paul Prior.  A Sociocultural Theory of Writing. , 2005 .

[14]  N. Frederiksen,et al.  PERFORMANCE TESTS OF EDUCATIONAL ACHIEVEMENT. IN E.F., LINDQUIS (ED.) , 1951 .

[15]  Marilyn Jager Adams,et al.  Beginning To Read: Thinking and Learning about Print. , 1991 .

[16]  Maria Bertling,et al.  A CBAL™ SCIENCE MODEL OF COGNITION: DEVELOPING A COMPETENCY MODEL AND LEARNING PROGRESSIONS TO SUPPORT ASSESSMENT DEVELOPMENT , 2013 .

[17]  J. Osborne,et al.  Supporting and Promoting Argumentation Discourse in Science Education , 2002 .

[18]  Susan E. Newman,et al.  Cognitive Apprenticeship: Teaching the Craft of Reading, Writing, and Mathematics. Technical Report No. 403. , 1987 .

[19]  Elizabeth Sulzby,et al.  Emergent Literacy: Writing and Reading. Writing Research: Multidisciplinary Inquiries into the Nature of Writing Series. , 1986 .

[20]  Robert J. Mislevy,et al.  A Bayesian Network Approach to Modeling Learning Progressions and Task Performance. CRESST Report 776. , 2010 .

[21]  M. A. Britt,et al.  Studying and Using Multiple Documents in History: Effects of Discipline Expertise , 1997 .

[22]  Ets Rr,et al.  CBAL: Results From Piloting Innovative K-12 Assessments , 2011 .

[23]  Randy Elliot Bennett,et al.  Cognitively Based Assessment of, for, and as Learning (CBAL): A Preliminary Theory of Action for Summative and Formative Assessment , 2010 .

[24]  Harold F. O'Neil,et al.  Policy and validity prospects for performance-based assessment. , 1993 .

[25]  Edith Aurora Graf,et al.  Using Argumentation Learning Progressions to Support Teaching and Assessments of English Language Arts , 2013 .

[26]  C. Bazerman.  Speech Acts, Genres, and Activity Systems: How Texts Organize Activity and People , 2003 .

[27]  A. Collins,et al.  Situated Cognition and the Culture of Learning , 1989 .

[28]  D. Russell.  Rethinking Genre in School and Society , 1997 .

[29]  C. Snow.  Literacy and Language: Relationships during the Preschool Years , 1983 .

[30]  P. Gilster.  Digital Literacy , 1997, Encyclopedia of Education and Information Technologies.

[31]  Jianbin Fu,et al.  STATISTICAL REPORT OF 2011 CBAL™ MULTISTATE ADMINISTRATION OF READING AND WRITING TESTS , 2012 .

[32]  Brian E Clauser,et al.  The Introduction of Computer‐based Case Simulations into the United States Medical Licensing Examination , 2002, Academic medicine : journal of the Association of American Medical Colleges.

[33]  Allan S. Cohen,et al.  Validating Measures of Performance , 2005 .

[34]  Robert J. Mislevy,et al.  Design Patterns for Learning and Assessment: Facilitating the Introduction of a Complex Simulation-Based Learning Environment into a Community of Instructors , 2010 .

[35]  Russell G. Almond,et al.  A cognitive task analysis with implications for designing simulation-based performance assessment☆ , 1999 .

[36]  B. Street.  What's 'new' in New Literacy Studies? Critical approaches to literacy in theory and practice , 2003 .

[37]  P. Broadfoot.  Evaluating authentic assessment , 1995 .

[38]  Judith Gulikers,et al.  Developer, teacher, student and employer evaluations of competence-based assessment quality , 2009 .

[39]  Janice D. Gobert,et al.  Leveraging Educational Data Mining for Real-time Performance Assessment of Scientific Inquiry Skills within Microworlds , 2012, EDM 2012.

[40]  Robert J. Mislevy,et al.  A Bayes net approach to modeling learning progressions and task performances , 2009 .

[41]  Dania Bilal.  Children's use of the Yahooligans! Web search engine: II. cognitive and physical behaviors on research tasks , 2001 .

[42]  Valerie J. Shute,et al.  Conceptual Framework for Modeling , Assessing and Supporting Competencies within Game Environments , 2010 .

[43]  Mark Guzdial,et al.  Learner-centered design: the challenge for HCI in the 21st century , 1994, INTR.

[44]  D. Yaden,et al.  Emergent Literacy: A Matter (Polyphony) of Perspectives , 2000 .

[45]  A Antal Haans,et al.  What Does it Mean to Be Average? The Miles per Gallon versus Gallons per Mile Paradox Revisited , 2008 .

[46]  Tenaha O'Reilly,et al.  COGNITIVELY BASED ASSESSMENT OF, FOR, AND AS LEARNING: A FRAMEWORK FOR ASSESSING READING COMPETENCY , 2009 .

[47]  S. Messick.  The Interplay of Evidence and Consequences in the Validation of Performance Assessments , 1994 .

[48]  Paul Deane,et al.  COGNITIVE MODELS OF WRITING: WRITING PROFICIENCY AS A COMPLEX INTEGRATED SKILL , 2008 .

[49]  Geoffrey R. Norman,et al.  Performance-Based Assessment: Lessons From the Health Professions , 1995 .

[50]  John Willinsky,et al.  The Cambridge Handbook of Literacy: Digital Literacy , 2009 .

[51]  R. Almond,et al.  A BRIEF INTRODUCTION TO EVIDENCE-CENTERED DESIGN , 2003 .

[52]  John Sabatini,et al.  FOUR YEARS OF COGNITIVELY BASED ASSESSMENT OF, FOR, AND AS LEARNING (CBAL): LEARNING ABOUT THROUGH- COURSE ASSESSMENT (TCA) , 2011 .

[53]  P. Deane.  WRITING ASSESSMENT AND COGNITION , 2011 .

[54]  Robert L. Linn,et al.  Validation of Performance-Based Assessments , 2000 .

[55]  P. David Pearson.  Handbook of reading research. , 1990 .

[56]  Ets Rr,et al.  Statistical Report of 2011 CBAL™ Multistate Administration of Reading and Writing Tests , 2012 .

[57]  Adriana Briseno,et al.  Analyzing Literacy Practice: Grounded Theory to Model. , 2011 .

[58]  H. David,et al.  Learning within the Context of Communities of Practices: A Re-Conceptualization of Tools, Rules and Roles of the Activity System , 2002 .

[59]  Robert J. Mislevy,et al.  Implications of Evidence‐Centered Design for Educational Testing , 2007 .

[60]  Paul D. Nelson,et al.  Competency assessment models. , 2007 .

[61]  Shirley Agostinho,et al.  Design Methodology for the Implementation and Evaluation of a Scenario-Based Online Learning Environment , 2005 .

[62]  Etienne Wenger,et al.  Situated Learning: Legitimate Peripheral Participation , 1991 .

[63]  Philip B. Gough,et al.  The simple view of reading , 1990 .

[64]  Rand J. Spiro,et al.  Cognitive flexibility theory : advanced knowledge acquisition in ill-structured domains , 1988 .

[65]  Randy Elliot Bennett,et al.  Preparing for the Future: What Educational Assessment Must Do. , 2014 .

[66]  Herbert J. Walberg,et al.  Knowing, Learning, and Instruction: Essays in Honor of Robert Glaser. , 1991 .

[67]  Y. Engeström,et al.  Perspectives on activity theory: Play, learning, and instruction , 1999 .

[68]  C. Geisler.  Academic Literacy and the Nature of Expertise: Reading, Writing and Knowing in Academic Philosophy , 1994 .

[69]  J. Chall.  Stages of reading development , 1983 .

[70]  Dania Bilal.  Children's use of the Yahooligans! Web search engine: I. Cognitive, physical, and affective behaviors on fact‐based search tasks , 2000 .

[71]  Susan R. Goldman,et al.  Managing, Understanding, Applying, and Creating Knowledge in the Information Age: Next-Generation Challenges and Opportunities , 2013 .

[72]  Tenaha O'Reilly,et al.  THE CBAL READING ASSESSMENT: AN APPROACH FOR BALANCING MEASUREMENT AND LEARNING GOALS , 2011 .

[73]  Catherine E. Snow,et al.  The Theoretical Basis For Relationships Between Language and Literacy in Development , 1991 .

[74]  Grant Wiggins,et al.  Assessing student performance , 1993 .

[75]  D B Swanson,et al.  Simulations in the United States Medical Licensing Examination™ (USMLE™) , 2004, Quality and Safety in Health Care.

[76]  Thomas W. Zane.  Performance Assessment Design Principles Gleaned from Constructivist Learning Theory (Part 1) , 2009 .

[77]  Jesse R. Sparks,et al.  Cognitively Based Assessment of Research and Inquiry Skills: Defining a Key Practice in the English Language Arts , 2015 .

[78]  M. Scardamalia,et al.  Knowledge Building: Theory, Pedagogy, and Technology , 2006 .

[79]  E. A. Graf,et al.  Empirical recovery of argumentation learning progressions in scenario-based assessments of English language arts , 2014 .

[80]  Thomas C. Reeves,et al.  Patterns of engagement in authentic online learning environments , 2003, ASCILITE.

[81]  Jennifer Wiley,et al.  The Cambridge Handbook of Expertise and Expert Performance: Expertise in History , 2006 .

[82]  Robert J. Mislevy,et al.  Putting ECD into Practice: The Interplay of Theory and Data in Evidence Models within a Digital Learning Environment , 2012, EDM 2012.

[83]  Robert J. Mislevy,et al.  Some implications of expertise research for educational assessment , 2010 .

[84]  C. Chinn,et al.  Epistemologically Authentic Inquiry in Schools: A Theoretical Framework for Evaluating Inquiry Tasks , 2002 .

[85]  John Sabatini,et al.  READING FOR UNDERSTANDING: HOW PERFORMANCE MODERATORS AND SCENARIOS IMPACT ASSESSMENT DESIGN , 2013 .

[86]  Torulf Palm.  Performance Assessment and Authentic Assessment: A Conceptual Analysis of the Literature , 2008 .

[87]  C. Frederiksen,et al.  Literary expertise in the description of a fictional narrative , 1991 .

[88]  Saskia Brand-Gruwel,et al.  Information problem solving by experts and novices: analysis of a complex cognitive skill , 2005, Comput. Hum. Behav..

[89]  Paul Deane,et al.  The Key Practice, Discuss and Debate Ideas: Conceptual Framework, Literature Review, and Provisional Learning Progressions for Argumentation , 2015 .

[90]  Dania Bilal,et al.  Children's use of the Yahooligans! Web search engine: II. Cognitive and physical behaviors on research tasks , 2001, J. Assoc. Inf. Sci. Technol..