Assessing Performance in Engineering Education: Examples Across Fifty Years of Practice

This chapter outlines the challenges associated with defining assessment in the context of engineering education. A case study undertaken in the late 1950s brought into question the validity of written examinations. One response to this was broadening the concept of coursework to include projects and the application of criterion referenced measures to assessment. It is against this backdrop that discussion takes place about recent developments in outcomes (competency)-based assessment required by regulation agencies. Studies of engineers at work show the complexity of engineering and the significance of the affective domain and tacit knowledge in acquiring competence. The idea that an engineer should be able to take on a professional role immediately after leaving college without some prior guided experience of working in industry is refuted. This chapter supports the view that engineering education may better prepare students for work in the industry if it is structured on a cooperative basis. But the experience of industry must be carefully designed. The acquisition of competence occurs through a developmental process and is subject to the experience of the conditions imposed by the organization. This has implications for the way in which formative assessment is conducted.

[1]  B. Bloom.  Taxonomy of educational objectives , 1956 .

[2]  Keith J. Bowman,et al.  All, most or some: Implementation of tiered objectives for ABET assessment in an engineering program , 2009, 2009 39th IEEE Frontiers in Education Conference.

[3]  James Trevelyan,et al.  Engineering Practice in a Global Context : Understanding the Technical and the Social , 2013 .

[4]  W. Furneaux,et al.  THE PSYCHOLOGIST AND THE UNIVERSITY , 1962 .

[5]  P. J. McVey.  The Errors in Marking Examination Scripts in Electronic Engineering , 1975 .

[6]  Jörgen Sandberg.  Understanding Human Competence at Work: An Interpretative Approach , 2000 .

[7]  R. Glaser,et al.  Knowing What Students Know: The Science and Design of Educational Assessment , 2001 .

[8]  Mats Daniels,et al.  Experiences from using constructive controversy in an open ended group project , 2010, 2010 IEEE Frontiers in Education Conference (FIE).

[9]  Holly Matusovich,et al.  Constructive Academic Controversy- What is it? Why use it? How to structure it? , 2009, 2009 39th IEEE Frontiers in Education Conference.

[10]  Louis L. Bucciarelli,et al.  Designing Engineers , 1994 .

[11]  John Stephenson,et al.  Quality in Learning: A Capability Approach in Higher Education , 1992 .

[12]  S. A. Tobias,et al.  The Project Method of Teaching Creative Mechanical Engineering , 1964 .

[13]  Thomas A. Litzinger,et al.  The Effects of a First‐Year Engineering Design Course on Student Intellectual Development as Measured by the Perry Scheme , 2000 .

[14]  D. Culver,et al.  A review of Emotional Intelligence by Daniel Goleman: implications for technical education , 1998, FIE '98. 28th Annual Frontiers in Education Conference. Moving from 'Teacher-Centered' to 'Learner-Centered' Education. Conference Proceedings (Cat. No.98CH36214).

[15]  Richard G. Carter.  Engineering curriculum design , 1984 .

[16]  Charles F. Elton,et al.  Students Who Leave Engineering. , 1971 .

[17]  John Heywood,et al.  Engineering Education : Research and Development in Curriculum and Instruction , 2005 .

[18]  M. Berg.  The Framework for Qualifications of the European Higher Education Area , 2005 .

[19]  John Heywood.  The Assessment of Learning in Engineering Education: Practice and Policy , 2016 .

[20]  David Radcliffe,et al.  Engineering education: Targeted learning outcomes or accidental competencies? , 2006 .

[21]  Susan E. Kowalski,et al.  Lessons learned when gathering real-time formative assessment in the university classroom using Tablet PCs , 2009, 2009 39th IEEE Frontiers in Education Conference.

[22]  J. Trevelyan.  Reconstructing engineering from practice , 2010 .

[23]  Deesha Chadha,et al.  Teaching Transferable Skills to Undergraduate Engineering Students: Recognising the Value of Embedded and Bolt-on Approaches* , 2006 .

[24]  Shan Barkataki,et al.  Teaching teamwork in engineering and computer science , 2011, 2011 Frontiers in Education Conference (FIE).

[25]  Judith George,et al.  A Handbook of Techniques for Formative Evaluation: Mapping the Students' Learning Experience , 1999 .

[26]  R. Oxtoby,et al.  Engineers, their jobs and training needs: A study of perceptions using the semantic differential , 1973 .

[27]  Walter G. Vincenti,et al.  What Engineers Know and How They Know It: Analytical Studies from Aeronautical History. , 1992 .

[28]  Thomas J. Brumm,et al.  Assessing and Developing Program Outcomes through Workplace Competencies , 2006 .

[29]  Nguyen Thai Nghe,et al.  A comparative analysis of techniques for predicting academic performance , 2007, 2007 37th Annual Frontiers In Education Conference - Global Engineering: Knowledge Without Borders, Opportunities Without Passports.

[30]  David Pistrui,et al.  Mapping the Behaviors, Motives, and Professional Competencies of Entrepreneurially Minded Engineers in Theory and Practice: An Empirical Investigation , 2012 .

[31]  P. R. Whitfield.  Creativity in industry , 1975 .

[32]  John Heywood,et al.  Higher Technological Education and British Policy Making: A Lost Opportunity for Curriculum Change in Engineering Education , 2014 .

[33]  Richard M. Felder,et al.  ON CREATING CREATIVE ENGINEERS , 1987 .

[34]  G. S. Bosworth.  TOWARDS CREATIVE ACTIVITY IN ENGINEERING , 1963 .

[35]  Andrew N. Hrymak,et al.  Developing Problem Solving Skills: The McMaster Problem Solving Program , 1997 .

[36]  Gretchen Hein,et al.  Enhancing Student Learning Through Self- Assessment , 2008 .

[37]  Bernard Blandin.  The competence of an engineer and how it is built through an apprenticeship program: a tentative model , 2012 .

[38]  Nicholas D. Fila,et al.  Does diversity in novice teams lead to greater innovation? , 2011, 2011 Frontiers in Education Conference (FIE).

[39]  M. R. Ramirez,et al.  Creativity in the classroom: Special concerns and insights , 1993, Proceedings of IEEE Frontiers in Education Conference - FIE '93.

[40]  Robert Cloutier,et al.  Comparing Perceptions of Competency Knowledge Development in Systems Engineering Curriculum: A Case Study , 2011 .

[41]  Karl O. Jones,et al.  Relationship between examination questions and bloom's taxonomy , 2009, 2009 39th IEEE Frontiers in Education Conference.

[42]  James Trevelyan,et al.  The Making of an Expert Engineer , 2014 .

[43]  Louis B. Barnes.  Organizational systems and engineering groups: a comparative study of two technical groups in industry , 1961 .

[44]  Stephen Ekwaro-Osire,et al.  Using design notebooks to map creativity during team activities , 2009, 2009 39th IEEE Frontiers in Education Conference.

[45]  Kathrin Figl,et al.  What competences do employers, staff and students expect from a Computer Science graduate? , 2009, 2009 39th IEEE Frontiers in Education Conference.

[46]  Michael Eraut,et al.  Concepts of competence , 1998 .

[47]  Richard G. Carter,et al.  A taxonomy of objectives for professional education , 1985 .

[48]  Benjamin S. Bloom,et al.  A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives , 2000 .

[49]  Patrick Ferro.  Use of Fink's Taxonomy in Establishing Course Objectives for a Re-designed Materials Engineering Course , 2011 .

[50]  J. Heywood,et al.  Engineering science a level in the UK. a case study in the balanced assessment of student learning, educational policies and educational scholarship , 2007, 2007 37th Annual Frontiers In Education Conference - Global Engineering: Knowledge Without Borders, Opportunities Without Passports.

[51]  C McGuire.  Perspectives in assessment , 1993, Academic medicine : journal of the Association of American Medical Colleges.

[52]  Peter R. Turner,et al.  Predictive assessment of student performance for early strategic guidance , 2011, 2011 Frontiers in Education Conference (FIE).

[53]  Steven B. Robbins,et al.  The Role of Academic and Non-Academic Factors in Improving College Retention. ACT Policy Report. , 2004 .

[54]  G. Carter.  The fall and rise of university engineering education in England in the 1980s , 1992 .

[55]  Jennifer Turns,et al.  When students choose competencies: Insights from the competency-specific engineering portfolio studio , 2012, 2012 Frontiers in Education Conference Proceedings.

[56]  J. Heywood,et al.  The value added performance of electrical engineering students in a British University , 1991 .

[57]  D. Royce Sadler,et al.  Perils in the meticulous specification of goals and assessment criteria , 2007 .

[58]  John David N. Dionisio,et al.  Standards-based grading: Preliminary studies to quantify charges in student affective and cognitive behaviors , 2012 .

[59]  C.M. Wigal.  The use of peer evaluations to measure student performance and critical thinking ability , 2007, 2007 37th Annual Frontiers In Education Conference - Global Engineering: Knowledge Without Borders, Opportunities Without Passports.

[60]  Tom R. Burns,et al.  The Management of Innovation. , 1963 .

[61]  K. Figl,et al.  Researching the development of team competencies in computer science courses , 2008, 2008 38th Annual Frontiers in Education Conference.

[62]  R. Motschnig-Pitrik,et al.  Developing team competence as part of a person centered learning course on communication and soft skills in project management , 2007, 2007 37th Annual Frontiers In Education Conference - Global Engineering: Knowledge Without Borders, Opportunities Without Passports.

[63]  Sydney Absell Gregory.  Creativity and innovation in engineering , 1972 .

[64]  Margaret Dale,et al.  Assessing management skills : guide to competencies and evalutation techniques , 1992 .

[65]  John Stephenson,et al.  Capability and quality in higher education , 1998 .

[66]  L. Fink,et al.  Creating Significant Learning Experiences: An Integrated Approach to Designing College Courses , 2003 .

[67]  Mani Mina,et al.  Learning to Think Critically to Solve Engineering Problems: Revisiting John Dewey's ideas for evaluating the engineering education , 2003 .

[68]  Timothy Highley,et al.  Discrete Mathematics assessment using learning objectives based on Bloom's taxonomy , 2009, 2009 39th IEEE Frontiers in Education Conference.

[69]  M. Pavelich,et al.  Measuring the Effect of Experiential Education Using the Perry Model , 1996 .

[70]  C.F. Yokomoto,et al.  Modeling the process of writing measurable outcomes for EC 2000 , 1999, FIE'99 Frontiers in Education. 29th Annual Frontiers in Education Conference. Designing the Future of Science and Engineering Education. Conference Proceedings (IEEE Cat. No.99CH37011.

[71]  R.E. Barr,et al.  Continuous Outcomes Assessment in an Introduction to Mechanical Engineering Course , 2006, Proceedings. Frontiers in Education. 36th Annual Conference.

[72]  G. Miller.  The assessment of clinical skills/competence/performance , 1990, Academic medicine : journal of the Association of American Medical Colleges.

[73]  R. T. Rakowski.  Assessment of student performance during industrial training placements , 1990 .

[74]  José Figueiredo,et al.  Finding workable solutions: Portuguese engineering experience , 2013 .

[75]  Sue Otter.  Learning Outcomes in Higher Education. A Development Project Report. , 1992 .

[76]  Ronald L. Miller,et al.  Defining the outcomes: a framework for EC-2000 , 2000, IEEE Trans. Educ..

[77]  Edmundo Tovar,et al.  The use of competences assessment to predict the performance of first year students , 2010, 2010 IEEE Frontiers in Education Conference (FIE).

[78]  J. Froyd,et al.  Integrated Engineering Curricula , 2005 .

[79]  Stanley Hutton,et al.  German engineers : the anatomy of a profession , 1981 .