Hunters & gatherers: Strategies for curriculum mapping and data collection for assurance of learning

Assurance of learning is a predominant feature in both quality enhancement and assurance in higher education. Assurance of learning is a process that articulates explicit program outcomes and standards, and systematically gathers evidence to determine the extent to which performance matches expectations. Benefits accrue to the institution through the systematic assessment of whole of program goals. Data may be used for continuous improvement, program development, and to inform external accreditation and evaluation bodies. Recent developments, including the introduction of the Tertiary Education and Quality Standards Agency (TEQSA) will require universities to review the methods they use to assure learning outcomes. This project investigates two critical elements of assurance of learning: 1. the mapping of graduate attributes throughout a program; and 2. the collection of assurance of learning data. An audit was conducted with 25 of the 39 Business Schools in Australian universities to identify current methods of mapping graduate attributes and for collecting assurance of learning data across degree programs, as well as a review of the key challenges faced in these areas. Our findings indicate that external drivers like professional body accreditation (for example: Association to Advance Collegiate Schools of Business (AACSB)) and TEQSA are important motivators for assuring learning, and those who were undertaking AACSB accreditation had more robust assurance of learning systems in place. It was reassuring to see that the majority of institutions (96%) had adopted an embedding approach to assuring learning rather than opting for independent standardised testing. The main challenges that were evident were the development of sustainable processes that were not considered a burden to academic staff, and obtainment of academic buy in to the benefits of assuring learning per se rather than assurance of learning being seen as a tick box exercise. This cultural change is the real challenge in assurance of learning practice.

[1]  Beverley Oliver,et al.  Final report. Teaching Fellowship: Benchmarking partnerships for graduate employability , 2010 .

[2]  Paul Hager,et al.  Nature And Development of Generic Attributes , 2006 .

[3]  Bernardo A. Huberman,et al.  Predicting the Future , 2003, Inf. Syst. Frontiers.

[4]  Assurance of Learning (AoL) Methods Just Have to Be Good Enough , 2007 .

[5]  Angel R. Martinez,et al.  Natural language processing , 2010 .

[6]  J. Biggs,et al.  Teaching For Quality Learning At University , 1999 .

[7]  Mantz Yorke,et al.  Embedding employability into the curriculum , 2006 .

[8]  Beverley Oliver Assuring graduate outcomes , 2011 .

[9]  Charles D. Pringle,et al.  Assessment Practices in AACSB-Accredited Business Schools , 2007 .

[10]  Susanne Owen,et al.  Project final report : learning and teaching in the discipline of law : achieving and sustaining excellence in a changed and changing environment , 2009 .

[11]  Shirley Alexander,et al.  Dissemination, adoption and adaptation of project innovations in higher education , 2005 .

[12]  Lesley Treleaven,et al.  Integrating the Development of Graduate Attributes Through Constructive Alignment , 2008 .

[13]  Kevin Crowston,et al.  Using natural language processing technology for qualitative data analysis , 2012, International Journal of Social Research Methodology.

[14]  Elaine Welsh,et al.  Dealing with Data: Using NVivo in the Qualitative Data Analysis Process , 2002 .

[15]  S. Barrie,et al.  The national graduate attributes report: Integration and assessment of graduate attributes in curriculum (Research Report) , 2009 .

[16]  Jennifer Sumsion,et al.  Identifying generic skills through curriculum mapping: a critical evaluation , 2004 .

[17]  Klaus Krippendorff,et al.  Content Analysis: An Introduction to Its Methodology , 1980 .

[18]  Bronwyn James,et al.  A web environment linking university teaching strategies with graduate attributes , 2004, Journal of University Teaching and Learning Practice.

[19]  S. Barrie,et al.  A conceptual framework for the teaching and learning of generic graduate attributes , 2007 .

[20]  Sarah J. Stein,et al.  Beyond mapping and embedding graduate attributes: bringing together quality assurance and action learning to create a validated and living curriculum , 2004 .

[21]  Carolyn Haynes,et al.  Targeted Assessment Rubric: An Empirically Grounded Rubric for Interdisciplinary Writing , 2009 .

[22]  Marie Kavanagh,et al.  The Roles of Some Key Stakeholders in the Future of Accounting Education in Australia , 2009 .

[23]  J. Landeta Current validity of the Delphi method in social sciences , 2006 .

[24]  Kayoko Uchiyama,et al.  Curriculum Mapping in Higher Education: A Vehicle for Collaboration , 2009 .

[25]  R M Harden,et al.  An international virtual medical school (IVIMEDS): the future for medical education? , 2002, Medical teacher.

[26]  John Biggs,et al.  What the student does: teaching for enhanced learning , 1999 .

[27]  Rachael M. Field,et al.  Bachelor of Laws : learning and teaching academic standards statement : December 2010 [Learning and Teaching Academic Standards Project] , 2010 .

[28]  Rochelle E. Tractenberg,et al.  A Mastery Rubric: guiding curriculum design, admissions and development of course objectives , 2010, Assessment and evaluation in higher education.

[29]  Kristin G. Esterberg,et al.  Qualitative Methods in Social Research , 2001 .

[30]  Kathryn A. Martell,et al.  Assessing Student Learning: Are Business Schools Making the Grade? , 2007 .

[31]  Hamish Coates,et al.  Defining and monitoring academic standards in Australian higher education , 2010 .

[32]  Melissa S Medina,et al.  A curriculum review and mapping process supported by an electronic database system. , 2008, American journal of pharmaceutical education.

[33]  J. Kotter,et al.  The Heart of Change: Real-Life Stories of How People Change Their Organizations , 2002 .

[34]  S. Barrie Understanding What We Mean by the Generic Attributes of Graduates , 2006 .

[35]  Romy Lawson,et al.  Facilitating staff and student engagement with graduate attribute development, assessment and standards in business faculties , 2009 .

[36]  Celia Popovic,et al.  Teaching for quality learning at university. (2nd Edn.) , 2013 .

[37]  Hamish Coates,et al.  An international assessment of bachelor degree graduates' learning outcomes , 2012 .

[38]  Patrick McCabe,et al.  Reference Points for the Design and Delivery of Degree Programs in Business , 2009 .