Design for Teaching and Learning in a Networked World: 10th European Conference on Technology Enhanced Learning, EC-TEL 2015, Toledo, Spain, September 15-18, 2015, Proceedings

In a controlled experiment on the effects of frequent and local digital annotations, 137 volunteers covered an online course at 3 conditions: no/free/question-based electronic annotations. Results show no difference in performance between groups. However, analyses conducted within treatments suggest positive impacts on performance when annotation rates are taken into consideration, and coupled with other reflective enactments.

[1]  Viktoria Pammer-Schindler,et al.  In-App Reflection Guidance for Workplace Learning , 2015, EC-TEL.

[2]  Paul Rayson,et al.  From key words to key semantic domains , 2008 .

[3]  Andy Dong,et al.  REFLECTION AND ANALYSIS IN DESIGN STUDENT BLOGS , 2006 .

[4]  Wu-Yuin Hwang,et al.  A study of multimedia annotation of Web-based materials , 2005, Comput. Educ..

[5]  Michael Prilla,et al.  Supporting Collaborative Reflection at Work: A Socio-Technical Analysis , 2015, AIS Trans. Hum. Comput. Interact..

[6]  Michael Prilla,et al.  Supporting Collaborative Reflection at Work: A Comparative Case Analysis , 2014, GROUP.

[7]  K. Lonka,et al.  Review and Process Effects of Spontaneous Note-Taking on Text Comprehension. , 1999, Contemporary educational psychology.

[8]  D. Kolb.  Experiential Learning: Experience as the Source of Learning and Development , 1983 .

[9]  David W. Smith,et al.  Facilitating reflection: issues and research. -Version of paper presented to Australian Teacher Education Association. Conference ( 1994: Brisbane )- , 1994 .

[10]  Erik Duval,et al.  Visualizing Activities for Self-reflection and Awareness , 2010, ICWL.

[11]  J. Sweller,et al.  Why Minimally Guided Teaching Techniques Do Not Work: A Reply to Commentaries , 2007 .

[12]  Matthew B. Crawford,et al.  The world beyond your head : how to flourish in an age of distraction , 2016 .

[13]  Eric Baumer,et al.  Reflective Informatics: Conceptual Dimensions for Designing Technologies of Reflection , 2015, CHI.

[14]  Olusola O. Adesope,et al.  Learning With Concept and Knowledge Maps: A Meta-Analysis , 2006 .

[15]  Michael Prilla,et al.  Collaborative Reflection Support at Work: a Socio-Technical Design Task , 2014, ECIS.

[16]  Bill N. Schilit,et al.  Beyond paper: supporting active reading with free form digital ink annotations , 1998, CHI.

[17]  Tzu-Hua Wang,et al.  What strategies are effective for formative assessment in an e-learning environment? , 2007, J. Comput. Assist. Learn..

[18]  Vera D. Khovanskaya,et al.  Reviewing reflection: on the use of reflection in interactive system design , 2014, Conference on Designing Interactive Systems.

[19]  Carmen Becker,et al.  Interactive and collaborative supports for CLIL: Towards a formal model based on digital literacy , 2014 .

[20]  MarenziIvana,et al.  Multiliteracies and Active Learning in CLIL—The Development of LearnWeb2.0 , 2012 .

[21]  H Platzer,et al.  Barriers to learning from reflection: a study of the use of groupwork with post-registration nurses. , 2000, Journal of advanced nursing.

[22]  Rudolf Wille,et al.  Restructuring Lattice Theory: An Approach Based on Hierarchies of Concepts , 2009, ICFCA.

[23]  DeSeCo Oecd.  Definition and Selection of Key Competencies-Executive Summary , 2005 .

[24]  Amy Forbes.  Evidence of learning in reflective practice: a case study of computer-assisted analysis of students' reflective blogs , 2011 .

[25]  Katharine S. Willis,et al.  WiMo: location-based emotion tagging , 2009, MUM.

[26]  Thomas Daniel Ullmann,et al.  An architecture for the automated detection of textual indicators of reflection , 2011 .

[27]  E. Zhu.  Meaning Negotiation, Knowledge Construction, and Mentoring in a Distance Learning Course , 1996 .

[28]  R. Fivush,et al.  Two-year-old talk about the past , 1987 .

[29]  Gilles Louppe,et al.  A hybrid human-computer approach for large-scale image-based measurements using web services and machine learning , 2014, 2014 IEEE 11th International Symposium on Biomedical Imaging (ISBI).

[30]  Janet E. Dyment,et al.  The Quality of Reflection in Student Journals: A Review of Limiting and Enabling Factors , 2010 .

[31]  Alyssa Friend Wise,et al.  Learning analytics for online discussions: a pedagogical model for intervention with embedded and extracted analytics , 2013, LAK '13.

[32]  C. Hmelo‐Silver,et al.  Scaffolding and Achievement in Problem-Based and Inquiry Learning: A Response to Kirschner, Sweller, and Clark (2006) , 2007 .

[33]  Rosanne Birney,et al.  Reflective Writing: Quantitative Assessment and Identification of Linguistic Features , 2012 .

[34]  Richard Freeman,et al.  Creating Learning Materials for Open and Distance Learning: A Handbook for Authors and Instructional Designers , 2005 .

[35]  Michael Prilla,et al.  Analyzing Collaborative Reflection Support: A Content Analysis Approach , 2015, ECSCW.

[36]  Sue Wharton,et al.  Presenting a united front: assessed reflective writing on a group experience , 2012 .

[37]  Henk G Schmidt,et al.  Correlates of Reflective Practice in Medicine , 2005, Advances in health sciences education : theory and practice.

[38]  L. Ruth-Sahd,et al.  Reflective practice: a critical analysis of data-based studies and implications for nursing education. , 2003, The Journal of nursing education.

[39]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[40]  Simone Braun,et al.  Mood Tracking in Virtual Meetings , 2012, EC-TEL.

[41]  George Siemens,et al.  Learning Analytics and Educational Data Mining , 2016 .

[42]  Daniele Quercia.  Urban: crowdsourcing for the good of London , 2013, WWW '13 Companion.

[43]  W. Chien,et al.  Promoting self-reflection in clinical practice among Chinese nursing undergraduates in Hong Kong , 2012, Contemporary nurse.

[44]  Mark Hawkes,et al.  Examining the reflective outcomes of asynchronous computer-mediated communication on inservice teacher development , 2001 .

[45]  Anind K. Dey,et al.  Designing for collaborative reflection , 2014, PervasiveHealth.

[46]  Fridolin Wild,et al.  Reflection - quantifying a rare good , 2013, ARTEL@EC-TEL.

[47]  Michael Prilla,et al.  Reflective TEL: Augmenting Learning Tools with Reflection Support , 2015, EC-TEL.

[48]  Päivi Häkkinen,et al.  Specifying computer-supported collaboration scripts , 2007, Int. J. Comput. Support. Collab. Learn..

[49]  Andrew Dillon,et al.  Reading from paper versus screens: a critical review of the empirical literature , 1992 .

[50]  R. J. Bogumil,et al.  The reflective practitioner: How professionals think in action , 1985, Proceedings of the IEEE.

[51]  M. Daudelin.  Learning from experience through reflection , 1996 .

[52]  Fridolin Wild,et al.  Grand Challenges in Technology Enhanced Learning: Outcomes of the 3rd Alpine Rendez-Vous , 2013 .

[53]  Hendrik Drachsler,et al.  Translating Learning into Numbers: A Generic Framework for Learning Analytics , 2012, J. Educ. Technol. Soc..

[54]  Wim Westera,et al.  Reflection amplifiers in online courses: a classification framework , 2011 .

[55]  Mark Hawkes.  An analysis of critically reflective teacher dialogue in asynchronous computer-mediated communication , 2001, Proceedings IEEE International Conference on Advanced Learning Technologies.

[56]  Elizabeth A. Davis,et al.  Scaffolding students' knowledge integration: prompts for reflection in KIE , 2000 .

[57]  J. M. Simmons,et al.  Reflective Pedagogical Thinking: How Can We Promote It and Measure It? , 1990 .

[58]  Robert B. Kottkamp.  Means for Facilitating Reflection , 1990 .

[59]  Rosamund Sutherland,et al.  A Vision and Strategy for Technology Enhanced Learning: Report from the STELLAR Network of Excellence , 2012 .

[60]  Wim Westera,et al.  Infusing Reflective Practice in eLearning Courses - Can Widgets Help? , 2011, MUPPLE@EC-TEL.

[61]  Mary E. Ryan,et al.  Reflexive writers: Re-thinking writing development and assessment in schools , 2014 .

[62]  Tobias Ley,et al.  Scaling up Learning for Sustained Impact , 2013, Lecture Notes in Computer Science.

[63]  Jasmine Luk,et al.  Assessing teaching practicum reflections: Distinguishing discourse features of the “high” and “low” grade reports , 2008 .

[64]  M. Woerkom,et al.  Operationalising critically reflective work behaviour , 2008 .

[65]  P. Robert-Jan Simons,et al.  Assessing student teachers’ reflective writing through quantitative content analysis , 2014 .

[66]  Paul T. Sowden,et al.  Perceptual learning of the detection of features in X-ray images: a functional role for improvements in adults' visual sensitivity? , 2000, Journal of experimental psychology. Human perception and performance.

[67]  Robert O. Briggs,et al.  The cognitive network model of creativity: a new causal model of creativity and a new brainstorming technique , 2000, Proceedings of the 33rd Annual Hawaii International Conference on System Sciences.

[68]  Andrew Wilson.  Embracing Bayes factors for key item analysis in corpus linguistics , 2013 .

[69]  Matthew Kam,et al.  Livenotes: a system for cooperative and augmented note-taking in lectures , 2005, CHI.

[70]  E. Terhart,et al.  Constructivism and teaching: A new paradigm in general didactics? , 2003 .

[71]  Ivana Marenzi,et al.  ESP Course Design - A Multiliteracies Approach , 2013, 2013 IEEE 13th International Conference on Advanced Learning Technologies.

[72]  J. H. Eshuis.  Seks en de evolutie , 2007 .

[73]  J. Hartley,et al.  Note‐taking: A critical review , 1978 .

[74]  Panagiotis Papapetrou,et al.  Significance testing of word frequencies in corpora , 2016, Digit. Scholarsh. Humanit..

[75]  N. Hatton,et al.  Reflection in teacher education: Towards definition and implementation , 1995 .

[76]  Marcus Specht,et al.  A first approach to “Learning Dashboards” in formal learning contexts , 2011 .

[77]  Michael Prilla.  User and Group Behavior in Computer Support for Collaborative Reflection in Practice: An Explorative Data Analysis , 2014, COOP.

[78]  Christian Glahn,et al.  Contextual support of social engagement and reflection on the Web , 2009 .

[79]  Richard E. Clark,et al.  Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching , 2006 .

[80]  B. Rosenshine,et al.  Teaching Students to Generate Questions: A Review of the Intervention Studies , 1996 .

[81]  Jodi Forlizzi,et al.  Understanding my data, myself: supporting self-reflection with ubicomp technologies , 2011, UbiComp '11.

[82]  Brendan Calandra,et al.  Examining the effects of computer-based scaffolds on novice teachers’ reflective journal writing , 2010 .

[83]  Judy Kay,et al.  Open Learner Models , 2010, Advances in Intelligent Tutoring Systems.

[84]  Marco Winckler,et al.  A Formal Model-Based Approach for Designing Interruptions-Tolerant Advanced User Interfaces , 2011, Model-Driven Development of Advanced User Interfaces.

[85]  Erik Duval,et al.  Recommender Systems for Learning , 2012, Springer Briefs in Electrical and Computer Engineering.

[86]  Wanda Pratt,et al.  Understanding quantified-selfers' practices in collecting and exploring personal data , 2014, CHI.

[87]  Margot Brereton,et al.  Proceedings of the 22nd Conference of the Computer-Human Interaction Special Interest Group of Australia on Computer-Human Interaction , 2010 .

[88]  Paul A. Kirschner,et al.  Finding your way into an open online learning community , 2008 .

[89]  M. Ryan,et al.  Improving reflective writing in higher education: a social semiotic perspective , 2011 .

[90]  Ulrike Cress,et al.  Reflection as a catalyst for organisational learning , 2013 .

[91]  Susanne G. Scott,et al.  Enhancing Reflection Skills Through Learning Portfolios: An Empirical Test , 2010 .

[92]  Wu-Yuin Hwang,et al.  The Effects of Pre-Reading and Sharing Mechanisms on Learning with the Use of Annotations , 2011 .

[93]  Vijay Verma.  On pooling of data and measures , 2009 .

[94]  C. C. Crawford.  Some experimental studies of the results of college note-taking. , 1925 .

[95]  Jennifer A. Moon.  Learning Journals: A Handbook for Reflective Practice and Professional Development , 2006 .

[96]  Geraldine Fitzpatrick,et al.  Reflecting on reflection: framing a design landscape , 2010, OZCHI '10.

[97]  Eldrid Herrington,et al.  Marginalia: Readers Writing in Books , 2002 .

[98]  Cindy K. Chung,et al.  Linguistic Inquiry and Word Count (LIWC): Pronounced “Luke,” . . . and Other Useful Facts , 2012 .

[99]  L. Ring,et al.  A categorization scheme for assessing pharmacy students' levels of reflection during internships. , 2008, American journal of pharmaceutical education.

[100]  Anthony Baldry,et al.  Multimodal transcription and text analysis : a multimedia toolkit and coursebook , 2006 .

[101]  Ted Dunning,et al.  Accurate Methods for the Statistics of Surprise and Coincidence , 1993, CL.

[102]  Susan M. Land,et al.  A Qualitative Analysis of Scaffolding Use in a Resource-Based Learning Environment Involving the World Wide Web , 2000 .

[103]  Kristina Höök,et al.  Experiencing the Affective Diary , 2009, Personal and Ubiquitous Computing.

[104]  Michael Prilla,et al.  Fostering Collaborative Redesign of Work Practice: Challenges for Tools Supporting Reflection at Work , 2013, ECSCW.

[105]  Kurt Hornik,et al.  Text Mining Package , 2015 .

[106]  Rudolf Wille,et al.  Knowledge Spaces and Formal Concept Analysis , 1996 .

[107]  Gordon B. Davis,et al.  User Acceptance of Information Technology: Toward a Unified View , 2003, MIS Q..

[108]  Kristy Kay Ballard,et al.  Using Van Manen's model to assess levels of reflectivity among preservice physical education teachers , 2006 .

[109]  Carl Reidsema,et al.  Assessing reflective writing: Analysis of reflective writing in an engineering design course , 2009 .

[110]  Parvati Dev.  Tutorial: Imaging and Visualization in Medical Education , 1999, IEEE Computer Graphics and Applications.

[111]  Nian-Shing Chen,et al.  Effects of reviewing annotations and homework solutions on math learning achievement , 2011, Br. J. Educ. Technol..

[112]  Massimo Marraffa,et al.  Organizational learning II: Theory, method and practice , 1998 .

[113]  Gregory Schraw,et al.  Assessing metacognitive awareness , 1994 .

[114]  Michael Prilla,et al.  The Functions of Sharing Experiences, Observations and Insights for Reflective Learning at Work , 2012, ARTEL@EC-TEL.

[115]  Simone Kopeinik,et al.  Applying the Formal Concept Analysis to Introduce Guidance in an Inquiry-Based Learning Environment , 2015, 2015 IEEE 15th International Conference on Advanced Learning Technologies.

[116]  Mary Kalantzis,et al.  Learning by Design: creating pedagogical frameworks for knowledge building in the twenty‐first century , 2008 .

[117]  Fridolin Wild,et al.  Comparing Automatically Detected Reflective Texts with Human Judgements , 2012, ARTEL@EC-TEL.

[118]  Maya Bialik,et al.  Sentiment in New York City: A High Resolution Spatial and Temporal View , 2013, ArXiv.

[119]  D. McKenzie‐Mohr,et al.  Fostering sustainable behavior through community-based social marketing. , 2000, The American psychologist.

[120]  Christian Wagner,et al.  Stimulating ideas through creativity software , 1994 .

[121]  Maryanne Driscoll,et al.  A method for assessing reflective journal writing. , 2005, Journal of allied health.

[122]  P. Winne,et al.  Feedback and Self-Regulated Learning: A Theoretical Synthesis , 1995 .

[123]  Michael Prilla,et al.  Prompting users to facilitate support needs in collaborative reflection , 2015, ARTEL@EC-TEL.

[124]  Lynn McAlpine,et al.  Building a metacognitive model of reflection , 1999 .

[125]  Erik Duval,et al.  LARAe: Learning analytics reflection & awareness environment , 2014, ARTEL@EC-TEL.

[126]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[127]  S. Reis,et al.  Fostering and Evaluating Reflective Capacity in Medical Education: Developing the REFLECT Rubric for Assessing Reflective Writing , 2012, Academic medicine : journal of the Association of American Medical Colleges.

[128]  D. Leutner,et al.  Is it Merely a Question of “What” to Prompt or Also “When” to Prompt? , 2009 .

[129]  Chee-Kit Looi,et al.  GroupScribbles as a rapid CSCL tool: Learning experiences of pre-service teachers , 2009 .

[130]  Pamela Effrein Sandstrom,et al.  Information Foraging Theory: Adaptive Interaction with Information , 2010, J. Assoc. Inf. Sci. Technol..

[131]  Mike Scott,et al.  PC analysis of key words — And key key words , 1997 .

[132]  Rudolf Wille,et al.  Formal Concept Analysis as Mathematical Theory of Concepts and Concept Hierarchies , 2005, Formal Concept Analysis.

[133]  Marcello Gallucci,et al.  A conceptual and empirical examination of justifications for dichotomization. , 2009, Psychological methods.

[134]  Marcel Van der Klink,et al.  Modelling self-efficacy, self-regulation, self-directed learning and career processes of adult professionals and relations with learning outcomes and labour market success , 2010 .

[135]  Viktoria Pammer-Schindler,et al.  Mood in the city - data-driven reflection on mood in relation to public spaces , 2015, ARTEL@EC-TEL.

[136]  Earl L. Wiener,et al.  Human factors of flight-deck checklists: The normal checklist , 1990 .

[137]  Mica R. Endsley,et al.  Theoretical Underpinnings of Situation Awareness, A Critical Review , 2000 .

[138]  D. Boud,et al.  Promoting Reflection in Learning: a Model , 2013 .

[139]  Maryanne Driscoll,et al.  Assessing reflective writing on a pediatric clerkship by using a modified Bloom's Taxonomy. , 2007, Ambulatory pediatrics : the official journal of the Ambulatory Pediatric Association.

[140]  Rossano Schifanella,et al.  The shortest path to happiness: recommending beautiful, quiet, and happy routes in the city , 2014, HT.

[141]  Kazuhiko Yokosawa,et al.  Visual Search of Experts in Medical Image Reading: The Effect of Training, Target Prevalence, and Expert Knowledge , 2013, Front. Psychol..

[142]  Hilary Nesi,et al.  Genres across the Disciplines: Student Writing in Higher Education , 2012 .

[143]  Steve Whittaker,et al.  Echoes from the past: how technology mediated reflection improves well-being , 2013, CHI.

[144]  Gloria Mark,et al.  The cost of interrupted work: more speed and stress , 2008, CHI.

[145]  María Jesús Rodríguez-Triana,et al.  Contextual learning analytics apps to create awareness in blended inquiry learning , 2015, 2015 International Conference on Information Technology Based Higher Education and Training (ITHET).

[146]  Terri A. Scandura,et al.  Burden or blessing? Expected costs and benefits of being a mentor. , 1999 .

[147]  Itiel E. Dror,et al.  Optimising the use of note-taking as an external cognitive aid for increasing learning , 2009, Br. J. Educ. Technol..

[148]  Hilary Nesi,et al.  A Classification of Genre Families in University Student Writing. , 2013 .

[149]  Davide Taibi,et al.  Supporting formative assessment in content and language integrated learning: the MWS-Web platform , 2014 .

[150]  Michael Prilla,et al.  The Push and Pull of Reflection in Workplace Learning: Designing to Support Transitions between Individual, Collaborative and Organisational Learning , 2012, EC-TEL.

[151]  D. Boud,et al.  Reflection, turning experience into learning , 1985 .

[152]  Dan Cosley,et al.  Pensieve: supporting everyday reminiscence , 2010, CHI.

[153]  Annie Piolat,et al.  Note Taking and Learning : A Summary of Research , 2006 .

[154]  S. Høyrup.  Reflection as a core process in organisational learning , 2004 .

[155]  Wim Westera,et al.  Self-Reported Learning Effects of a Tagging Activity Carried out in a Personal Learning Environment (PLE) by Secondary-School Pupils , 2011 .

[156]  Timothy J. Lewis,et al.  Assessing Teacher Use of Opportunities to Respond and Effective Classroom Management Strategies , 2009 .

[157]  C. J. Ravesloot,et al.  Interpretation of radiological images: towards a framework of knowledge and skills , 2014, Advances in health sciences education : theory and practice.

[158]  Matthew D. Pistilli,et al.  Course signals at Purdue: using learning analytics to increase student success , 2012, LAK.

[159]  Klaus Korossy,et al.  Modeling Knowledge as Competence and Performance , 1999 .

[160]  Thomas Daniel Ullmann,et al.  Automated detection of reflection in texts : a machine learning based approach , 2015 .

[161]  Scott P. Robertson,et al.  Proceedings of the SIGCHI Conference on Human Factors in Computing Systems , 1991 .

[162]  Michael Prilla,et al.  Understanding and Supporting Reflective Learning Processes in the Workplace: The CSRL Model , 2013, EC-TEL.

[163]  Starla McCollum.  Insights Into the Process of Guidingreflection During an Early Fieldexperience of Preservice Teachers , 1997 .

[164]  David Kember,et al.  Determining the level of reflective thinking from students' written journals using a coding scheme based on the work of Mezirow , 1999 .

[165]  Stefanie N. Lindstaedt,et al.  Reflective learning at work – A position and discussion paper , 2011 .

[166]  Zhijie Xu,et al.  Online annotation - Research and practices , 2007, Comput. Educ..

[167]  Daniel McDuff,et al.  AffectAura: an intelligent system for emotional memory , 2012, CHI.

[168]  Stefaan Ternier,et al.  Personalisation of Learning in Virtual Learning Environments , 2009, EC-TEL.

[169]  Katrin Poom-Valickis,et al.  Reflecting others and own practice: an analysis of novice teachers’ reflection skills , 2013 .

[170]  Bracha Kramarski,et al.  Construction and Application of an Evaluative Tool to Assess Reflection in Teacher-Training Courses , 2002 .

[171]  Mary E. Ryan,et al.  Conceptualising and teaching discursive and performative reflection in higher education , 2012 .

[172]  Mark Hawkes.  Linguistic Discourse Variables as Indicators of Reflective Online Interaction , 2006 .

[173]  Petri Nokelainen,et al.  A shared document-based annotation tool to support learner-centred collaborative learning , 2005, Br. J. Educ. Technol..

[174]  G. Salomon,et al.  Skill may not be enough: The role of mindfulness in learning and transfer , 1987 .

[175]  Guy Claxton,et al.  Expanding the Capacity to Learn: A new end for education? , 2006 .