Assessment in action: A study of lecturers' and students' constructions of BTEC national assessment practice in a college engineering programme area

This research examines the nature and form of Edexcel’s BTEC National assessment policy and practice, as found within a small college Engineering Programme Area. The study investigated the salient influences and considerations underpinning both the explicit and implicit lecturer assessment constructs. The backwash effects of these constructs are considered, and how these impact on lecturers’ micro-level classroom practice, and on students’ engagement with assessment. This study also considers the effect assessment has on preparing students for progression from BTEC National programmes. BTEC National qualifications of the 2000s have their origins in the 1970s Technician Education Council’s programmes, founded on the recommendations of the Haslegrave Committee’s Report (Haslegrave, 1969). Although BTEC programmes have evolved over the past four decades, the central tenets of Haslegrave, that of unitised, teacher-assessed, broken-up summative assessment, still underpin BTEC National assessment of the 2000s. Current BTEC units are criterion-referenced, and employ formative assessment as an integral aspect of the educational ethos of the qualification. The research design involved a single site case study of assessment-in-action within a small programme area offering BTEC Nationals in Electrical and Electronic Engineering and in Manufacturing Engineering. This study used an interpretative approach, based on semi-structured interviews with seven lecturers and thirteen students during academic years 2006-2008. Findings suggest BTEC assessment practice relies significantly on the integrity of the lecturers, who construct their assessment practice by accommodating and balancing various external and internal requirements and influences placed upon them. It is through the programme area community of practice that notions of standards evolve, these being significantly influenced by cultural considerations, which impact on all aspects of assessment practice. This study finds an ethical departmental ethos in which all students should pass, and an assessment regime implicitly designed to aid student retention and achievement, but from which emanates a focus on criteria compliance. This tends to produce assessment constructs encouraging instrumental learning, where students’ achievements can be based on incremental improvement of the same assessment through multiple attempts, and where the potential for developing learning is diminished as formative assessment becomes conflated with summative intent. Both the assessment regime and the type of learning implicitly encouraged, has the potential to hamper some students’ preparedness for progression from the BTEC National programmes. Based on the findings of this research, considerations and recommendations are offered, both at the macro level of BTEC policy and at the departmental programme area micro-level of classroom practice, with the intention of enhancing students preparedness for progression from the National programmes. The study concludes that, despite radical changes in technician assessment practice having occurred since instigation of the Haslegrave recommendations, concerns emanating from assessment practice of the 1950s and 60s are still present within modern-day BTEC assessment, a case of plus ca change.

[1]  J. Biggs Assessment and Classroom Learning: a role for summative assessment? , 1998 .

[2]  A. P. Craig,et al.  Time and Learning , 2000, On Learning.

[3]  R. H. Parsons A history of the Institution of Mechanical Engineers, 1847-1947 , 1947 .

[4]  Glenn M. Hymel Instructional Design Applications in the Teaching of Psychology. , 1993 .

[5]  D. Sadler Formative assessment and the design of instructional systems , 1989 .

[6]  David James,et al.  Improving Learning Cultures in Further Education , 2007 .

[7]  P. Newton Clarifying the purposes of educational assessment , 2007 .

[8]  Etienne Wenger,et al.  Communities of practice: A brief introduction , 2009 .

[9]  D. Rowntree Assessing Students: How Shall We Know Them? , 1977 .

[10]  David Hamilton,et al.  Formative and Summative Assessment : a cybernetic viewpoint , 2005 .

[11]  Steven N. Durlauf,et al.  Current perspectives and future directions , 2006 .

[12]  P. Ramsden,et al.  THE EXPERIENCE OF LEARNING THE CONTEXT OF LEARNING IN ACADEMIC DEPARTMENTS 199198 CHAPTER THIRTEEN The Context of Learning in Academic Departments , 2005 .

[13]  Thomas R. Guskey,et al.  Cooperative Mastery Learning Strategies , 1990, The Elementary School Journal.

[14]  Barbara Crossouard,et al.  A socio‐cultural theorisation of formative assessment , 2008 .

[15]  K. Ecclestone,et al.  The impact of different modes of assessment on achievement and progress in the learning and skills sector , 2005 .

[16]  D. Bligh Learning to teach in higher education , 1993 .

[17]  Arkalgud Ramaprasad,et al.  On the definition of feedback , 1983 .

[18]  J. Pryor,et al.  Investigating Formative Assessment: Teaching, Learning and Assessment in the Classroom , 1998 .

[19]  Some difficulties in informal assessment in mathematics , 2006 .

[20]  M. Scriven The methodology of evaluation , 1966 .

[21]  A. Mcwhinnie,et al.  Bode’s Rule and Folded Sequences , 1976, The Fibonacci Quarterly.

[22]  K. B. Brown The downward slope: engineering HE and vocational qualifications in terminal decline? , 2001 .

[23]  Mary James,et al.  Beyond method: assessment and learning practices and values , 2006 .

[24]  Paul Black,et al.  Testing: Friend or Foe?: Theory and Practice of Assessment and Testing , 1997 .

[25]  K. Tonso The Impact of Cultural Norms on Women , 1996 .

[26]  G. Jessup Outcomes: Nvqs And The Emerging Model Of Education And Training , 1991 .

[27]  E. Hargreaves Assessment for learning? Thinking outside the (black) box , 2005 .

[28]  M. Taras,et al.  ASSESSMENT – SUMMATIVE AND FORMATIVE – SOME THEORETICAL REFLECTIONS , 2005 .

[29]  R. Tierney Changing practices: influences on classroom assessment , 2006 .

[30]  Maddalena Taras Challenging the status quo: Summative and Formative Assessment - Perceptions and Realities , 2005 .

[31]  Gary McCulloch,et al.  Succeeding with Your Doctorate , 2005 .

[32]  Gene V. Glass,et al.  Meta-Analysis of Research on Class Size and Achievement , 1979 .

[33]  J. Biggs Enhancing teaching through constructive alignment , 1996 .

[34]  K. Allen Technical Education in England and Wales , 1965, History of Education Quarterly.

[35]  G. Gibbs,et al.  Conditions Under Which Assessment Supports Students’ Learning , 2005 .

[36]  Catherine T. Amelink,et al.  Quantitative, Qualitative, and Mixed Research Methods in Engineering Education , 2009 .

[37]  David Boud,et al.  Moving Towards Autonomy , 2012 .

[38]  Ronald L. Miller,et al.  A New Paradigm for a New Field: Communicating Representations of Engineering Education Research , 2008 .

[39]  D. Prozesky,et al.  Assessment of Learning , 2020, The SAGE Encyclopedia of Higher Education.

[40]  A. Bathmaker,et al.  Communities of practice and the construction of learners in post-compulsory education and training , 2002 .

[41]  Mantz Yorke,et al.  Summative assessment: dealing with the ‘measurement fallacy’ , 2011 .

[42]  Is the Scientific Paper a Fraud? or: Who's Got the Paper? , 1965, Canadian Medical Association journal.

[43]  P. Black,et al.  'In praise of educational research': formative assessment , 2003 .

[44]  Benjamin S. Bloom,et al.  Taxonomy of Educational Objectives: The Classification of Educational Goals. , 1957 .

[45]  Mantz Yorke Formative assessment in higher education: Moves towards theory and the enhancement of pedagogic practice , 2003 .

[46]  R. Slavin Mastery Learning Reconsidered , 1987 .

[47]  Caroline Gipps,et al.  Developments in Educational Assessment: what makes a good test? , 1994 .

[48]  C. Gipps Beyond Testing: Towards a Theory of Educational Assessment , 1994 .

[49]  J. Losak Assessment and improvement in education , 1987 .

[50]  R. Birbeck The History and Operation of the Technician Education Council Programmes in Colleges of Technology in the United Kingdom , 1980 .

[51]  M. Townend Integrating Case Studies in Engineering Mathematics: A response to SARTOR 3 , 2001 .

[52]  Kathryn Ecclestone,et al.  'I know a 2:1 when I see it': Understanding criteria for degree classifications in franchised university programmes , 2001 .

[53]  D. Boud Sustainable Assessment: Rethinking assessment for the learning society , 2000 .

[54]  David Boud,et al.  Introduction: assessment for the longer term , 2007 .

[55]  Maddalena Taras,et al.  Summative and formative assessment , 2008 .

[56]  Jennifer Platt,et al.  On Interviewing One's Peers , 1981 .

[57]  G. Boulton‐Lewis Teaching for quality learning at university , 2008 .

[58]  Uncertain science-- uncertain world , 2003 .

[59]  B. Flyvbjerg Five Misunderstandings About Case-Study Research , 2006, 1304.1186.

[60]  Harry Torrance Formative Assessment: some theoretical problems and empirical questions , 1993 .

[61]  M. Price,et al.  Assessment standards: the role of communities of practice and the scholarship of assessment , 2005 .

[62]  E. Guba,et al.  Competing paradigms in qualitative research. , 1994 .

[63]  Brian Simon Does education matter , 1985 .

[64]  Maddalena Taras,et al.  Assessment for learning: understanding theory to improve practice , 2007 .

[65]  Hugh C. Davis,et al.  Changing assessment practice in engineering: how can understanding lecturer perspectives help? , 2004 .

[66]  Rajeev Kapila,et al.  Annual Report 2007-2008 , 2008 .

[67]  M. James,et al.  Assessment and Learning: differences and relationships between formative and summative assessment , 1997 .

[68]  Ron Dearing,et al.  Review of qualifications for 16-19 year olds: Summary report , 1996 .

[69]  Learning Cultures in Further Education , 2007 .

[70]  Robin Burgess-Limerick,et al.  Implementing Criterion-referenced Assessment Within a Multi-disciplinary University Department , 2000 .

[71]  Eleanore Hargreaves,et al.  The validity of collaborative assessment for learning , 2007 .

[72]  D. Sadler,et al.  Formative Assessment: revisiting the territory , 1998 .

[73]  D. Anderson,et al.  The hidden curriculum. , 1992, AJR. American journal of roentgenology.

[74]  Robert Stake,et al.  Standards-based & responsive evaluation , 2003 .

[75]  Terry Hyland Competence, Knowledge and Education , 1993 .

[76]  E. B. Page Teacher comments and student performance: A seventy-four classroom experiment in school motivation. , 1958 .

[77]  Robert Glaser,et al.  Instructional technology and the measurement of learing outcomes: Some questions. , 1963 .

[78]  A. Smithers All our futures: Britain’s education revolution: a dispatches report on education , 1993 .

[79]  R. F. Melton Learning outcomes for higher education: Some key issues , 1996 .

[80]  P. Sharp The Beginnings of GNVQs: an analysis of key determining events and factors , 1998 .

[81]  D. Royce Sadler,et al.  Interpretations of criteria‐based assessment and grading in higher education , 2005 .

[82]  J.K. Martin,et al.  Experiences combining technology, assessment, and feedback to improve student learning in mechanical engineering thermal science courses , 2005, Proceedings Frontiers in Education 35th Annual Conference.

[83]  Mark Weiser,et al.  The spirit of the engineering quest , 1999 .

[84]  David Boud,et al.  Reframing assessment as if learning were important , 2007 .

[85]  Harry Torrance Assessment as learning? How the use of explicit learning objectives, assessment criteria and feedback in post‐secondary education and training can come to dominate learning. 1 , 2007 .

[86]  Alison Wolf,et al.  Review of Vocational Education - The Wolf Report , 2011 .

[87]  R. W. Tyler Changing concepts of educational evaluation , 1986 .

[88]  R. Fisher From Business Education Council to Edexcel Foundation 1969–96: the short but winding road from technician education to instrumentalist technicism , 2004 .

[89]  R. Lindsay,et al.  Resource changes and academic performance at an English polytechnic , 1987 .

[90]  G. Rees,et al.  Education for All: The Future of Education and Training for 14-19 Year-Olds , 2009 .