eMedical Teacher

Medical education is increasingly driven by economic models looking to maximise return on investment and other metrics of accountability. Not least because being able to account for the execution of medical education is both a professional and ethical concern. We should and will be held accountable to our students, to our staff, to our professions, to our locality and to society as a whole. In this edition of the column we look at the issue of metrics for e-learning and e-teaching and the issues that surround them. Business informatics in support of quality assurance and audit are increasingly concerned with measurable indicators of success. Typically taking the form of ‘key performance indicators’ (KPIs) these metrics are intended to be practical, quantifiable and meaningful to the audience they are meant to serve and the factors they are to represent. Examples of KPIs for e-teaching might include uptake measured as the percentage of classes and tutorials conducted online, reliability measured as percentage of the average day/month/year the online learning environment is accessible to its learners, or its acceptability measured as a proportion of favourable to unfavourable reviews or evaluations it receives. KPIs and other organizational performance metrics may be aggregated to create what has been called the ‘balanced scorecard’ (see www.balancedscorecard.org), a four-domain model comprising financial factors (such as return on investment in a particular system), customer factors (such as satisfaction in that system), internal process factors (such as uptake or availability of the system) and learning and growth factors (such as skills and capabilities required or developed around the system). Broader still is the growing concept of ‘analytics’; essentially any kind of analysis including metrics, statistics or data mining. As an example Google’s free Analytics service (see http:// www.google.com/analytics/) allows users to paste tracking code into their web pages and then analyze in detail the patterns of access for those pages including graphical analysis and dashboards for the websites in question. Many learning environments and other educational tools also provide logs, tracking and other analytic tools regarding how their users work with their systems. In higher education these various approaches are being developed into an emerging practice of ‘action analytics’ (Norris et al, 2008), which integrates educational, fiscal and institutional alignment metrics in support of improved services and outcomes for all concerned. More specific to the e-teacher are the various approaches to the benchmarking of e-learning (see http://en.wikipedia.org/wiki/benchmarking_e-learning) where specific metrics are related back to common reference points for comparing different instances (rather than setting standards for one instance). Clearly then there is much that might be, can be, and is being, measured in the e-learning and e-teaching environment for all sorts of purposes above and beyond the execution of the course or program it is there to support. Indeed, tracking data and its analysis is one of the foundations of information and communication technologies.