Learning Analytics in Massive Open Online Courses

Educational technology has obtained great importance over the last fifteen years. At present, the umbrella of educational technology incorporates multitudes of engaging online environments and fields. Learning analytics and Massive Open Online Courses (MOOCs) are two of the most relevant emerging topics in this domain. Since they are open to everyone at no cost, MOOCs excel in attracting numerous participants that can reach hundreds and hundreds of thousands. Experts from different disciplines have shown significant interest in MOOCs as the phenomenon has rapidly grown. In fact, MOOCs have been proven to scale education in disparate areas. Their benefits are crystallized in the improvement of educational outcomes, reduction of costs and accessibility expansion. Due to their unusual massiveness, the large datasets of MOOC platforms require advanced tools and methodologies for further examination. The key importance of learning analytics is reflected here. MOOCs offer diverse challenges and practices for learning analytics to tackle. In view of that, this thesis combines both fields in order to investigate further steps in the learning analytics capabilities in MOOCs. The primary research of this dissertation focuses on the integration of learning analytics in MOOCs, and thereafter looks into examining students' behavior on one side and bridging MOOC issues on the other side. The research was done on the Austrian iMooX xMOOC platform. We followed the prototyping and case studies research methodology to carry out the research questions of this dissertation. The main contributions incorporate designing a general learning analytics framework, learning analytics prototype, records of students' behavior in nearly every MOOC's variables (discussion forums, interactions in videos, self-assessment quizzes, login frequency), a cluster of student engagement...

[1]  Mahil Carr.  Prototyping and Software Development Approaches , 1998 .

[2]  Guoyin Wang,et al.  An Efficient Piecewise Hashing Method for Computer Forensics , 2008, First International Workshop on Knowledge Discovery and Data Mining (WKDD 2008).

[3]  Vivienne L. Ming,et al.  Predicting student outcomes from unstructured data , 2012, UMAP Workshops.

[4]  Ivan Titov,et al.  Modeling online reviews with multi-grain topic models , 2008, WWW.

[5]  George Siemens,et al.  Penetrating the fog: analytics in learning and education , 2014 .

[6]  Martin Ebner,et al.  “How satisfied are you with your MOOC?” - A Research Study on Interaction in Huge Online Courses , 2013 .

[7]  Lawrence A. Machi,et al.  The literature review : six steps to success , 2009 .

[8]  Rebecca Ferguson,et al.  Social learning analytics: five approaches , 2012, LAK.

[9]  Martin Ebner,et al.  Does Gamification in MOOC Discussion Forums Work? , 2017, EMOOCs.

[10]  Devayani Tirthali,et al.  MOOCs: Expectations and Reality. Full Report. , 2014 .

[11]  Tim Vogelsang,et al.  On the validity of peer grading and a cloud teaching assistant system , 2015, LAK.

[12]  Steven Lonn,et al.  Bridging the gap from knowledge to action: putting analytics in the hands of academic advisors , 2012, LAK '12.

[13]  Fiona M. Hollands and Devayani Tirthali.  MOOCs: Expectations and Reality , 2014 .

[14]  Wilbur Schramm,et al.  Notes on Case Studies of Instructional Media Projects. , 1971 .

[15]  F. Herzberg.  One More Time: How Do You Motivate Employees? , 2008 .

[16]  Bernard J. Jansen,et al.  An Analysis of MOOC Discussion Forum Interactions from the Most Active Users , 2015, SBP.

[17]  Fred D. Davis,et al.  User Acceptance of Computer Technology: A Comparison of Two Theoretical Models , 1989 .

[18]  Martin Ebner,et al.  Why Learning Analytics for Primary Education Matters ! , .

[19]  Pierangela Samarati,et al.  Protecting privacy when disclosing information: k-anonymity and its enforcement through generalization and suppression , 1998 .

[20]  Meehyun Yoon,et al.  Analyzing the log patterns of adult learners in LMS using learning analytics , 2014, LAK.

[21]  Der deutschsprachige Open Online Course OPCO12 , 2013 .

[22]  C. Osvaldo Rodriguez,et al.  MOOCs and the AI-Stanford Like Courses: Two Successful and Distinct Course Formats for Massive Open Online Courses. , 2012 .

[23]  John Fritz,et al.  Classroom walls that talk: Using online course activity data of successful students to raise self-awareness of underperforming peers , 2011, Internet High. Educ..

[24]  Deborah Richards,et al.  Learning analytics in higher education : a summary of tools and approaches , 2013 .

[25]  Jack Mostow,et al.  Some useful tactics to modify, map and mine data from intelligent tutors , 2006, Natural Language Engineering.

[26]  P. Rousseeuw,et al.  Displaying a clustering with CLUSPLOT , 1999 .

[27]  Albert Sangrà,et al.  Building an inclusive definition of e-learning: An approach to the conceptual framework , 2012 .

[28]  C. Chinn,et al.  Innovating Pedagogy 2017: Exploring new forms of teaching, learning and assessment, to guide educators and policy makers. Open University Innovation Report 6 , 2017 .

[29]  Susan Lowes,et al.  Exploring the Link between Online Behaviours and Course Performance in Asynchronous Online High School Courses , 2015, J. Learn. Anal..

[30]  Zvisinei Sandi.  DEFINITION , 1961, A Philosopher Looks at Sport.

[31]  Jim Gaston,et al.  Sherpa: increasing student success with a recommendation engine , 2012, LAK '12.

[32]  Tanya Elias,et al.  Learning Analytics: Definitions, Processes and Potential , 2011 .

[33]  Li Yuan,et al.  MOOCs and open education: Implications for higher education , 2013 .

[34]  E. Duval.  Attention please!: learning analytics for visualization and recommendation , 2011, LAK.

[35]  Shuang-Hong Yang,et al.  Dimensionality Reduction and Topic Modeling: From Latent Semantic Indexing to Latent Dirichlet Allocation and Beyond , 2012, Mining Text Data.

[36]  Christopher Cunningham,et al.  Gamification by Design - Implementing Game Mechanics in Web and Mobile Apps , 2011 .

[37]  Ryan S. Baker,et al.  Content or platform: Why do students complete MOOCs? , 2015 .

[38]  Sebastián Ventura,et al.  Educational data mining: A survey from 1995 to 2005 , 2007, Expert Syst. Appl..

[39]  Laura Calvet Liñán,et al.  Educational Data Mining and Learning Analytics: differences, similarities, and time evolution , 2015 .

[40]  Anirban Dasgupta,et al.  Superposter behavior in MOOC forums , 2014, L@S.

[41]  Paul Ohm.  Broken Promises of Privacy: Responding to the Surprising Failure of Anonymization , 2009 .

[42]  Ulrik Schroeder,et al.  What Drives a Successful MOOC? An Empirical Examination of Criteria to Assure Design Quality of MOOCs , 2014, 2014 IEEE 14th International Conference on Advanced Learning Technologies.

[43]  Hendrik Drachsler,et al.  Translating Learning into Numbers: A Generic Framework for Learning Analytics , 2012, J. Educ. Technol. Soc..

[44]  John P. Campbell,et al.  Academic Analytics: A New Tool for a New Era. , 2007 .

[45]  W. Bruce Croft,et al.  LDA-based document models for ad-hoc retrieval , 2006, SIGIR.

[46]  Abdellatif Medouri,et al.  LASyM: A Learning Analytics System for MOOCs , 2013 .

[47]  Ulrik Schroeder,et al.  Learning Analytics: Challenges and Future Research Directions , 2014 .

[48]  V. Boiteau,et al.  Title an Examination of Indices for Determining the Number of Clusters : Nbclust Package , 2012 .

[49]  Marcia J. Simmering,et al.  E-learning: emerging uses, empirical results and future directions , 2003 .

[50]  Kenneth D. Strang,et al.  Beyond engagement analytics: which online mixed-data factors predict student learning outcomes? , 2017, Education and Information Technologies.

[51]  D. Wiley,et al.  Open Educational Resources: Enabling Universal Education. , 2008 .

[52]  Steve Joordens,et al.  Assessing the effectiveness of a voluntary online discussion forum on improving students' course performance , 2011, Comput. Educ..

[53]  Jane Sinclair,et al.  Exploring the use of MOOC discussion forums , 2014 .

[54]  Barry Fishman,et al.  Planning for success: how students use a grade prediction tool to win their classes , 2015, LAK.

[55]  Antje Baer,et al.  E Learning Strategies For Delivering Knowledge In The Digital Age , 2016 .

[56]  Pedro A. Toledo,et al.  Enhancing the engagement of intelligent tutorial systems through personalization of gamification , 2016 .

[57]  Matthew Montebello,et al.  A case study inside virtual worlds: use of analytics for immersive spaces , 2013, LAK '13.

[58]  R. B. O'Toole,et al.  Pedagogical strategies and technologies for peer assessment in Massively Open Online Courses (MOOCs) , 2013 .

[59]  B. Nonnecke,et al.  WHY LURKERS LURK , 2001 .

[60]  Dowming Yeh,et al.  What drives a successful e-Learning? An empirical investigation of the critical factors influencing learner satisfaction , 2008, Comput. Educ..

[61]  George Siemens,et al.  Analytics to literacies: The development of a learning analytics framework for multiliteracies assessment , 2014 .

[62]  Erik Duval,et al.  Tracking Data in Open Learning Environments , 2015, J. Univers. Comput. Sci..

[63]  Martin Ebner,et al.  Portraying MOOCs Learners: a Clustering Experience Using Learning Analytics , 2016 .

[64]  V. Larionova,et al.  Learning analytics in massive open online courses as a tool for predicting learner performance , 2018 .

[65]  Allison Littlejohn,et al.  Learning in MOOCs: Motivations and self-regulated learning in MOOCs , 2016, Internet High. Educ..

[66]  Bin Xu,et al.  Motivation Classification and Grade Prediction for MOOCs Learners , 2016, Comput. Intell. Neurosci..

[67]  David E. Pritchard,et al.  A MOOC based on blended pedagogy , 2016, J. Comput. Assist. Learn..

[68]  Charles B. Hodges.  Designing to Motivate: Motivational Techniques to Incorporate in E-Learning Experiences , 2004 .

[69]  Malcolm E. Brown,et al.  Learning Analytics : The Coming Third Wave , 2011 .

[70]  Justin Reich,et al.  Rebooting MOOC Research , 2015, Science.

[71]  Anil K. Jain,et al.  Algorithms for Clustering Data , 1988 .

[72]  Krzysztof Janowicz,et al.  A Linked-Data-Driven Web Portal for Learning Analytics: Data Enrichment, Interactive Visualization, and Knowledge Discovery , 2014, LAK Workshops.

[73]  Mike Sharkey,et al.  Course correction: using analytics to predict course success , 2012, LAK '12.

[74]  S. Rozendaal,et al.  Campus 2.0 , 2017 .

[75]  Xin Chen,et al.  "Twitter Archeology" of learning analytics and knowledge conferences , 2015, LAK.

[76]  Carlos Delgado Kloos,et al.  Experiences of running MOOCs and SPOCs at UC3M , 2014, 2014 IEEE Global Engineering Education Conference (EDUCON).

[77]  Sebastián Ventura,et al.  Educational Data Mining: A Review of the State of the Art , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[78]  Peter Baumgartner,et al.  Open Educational Practices and Resources : OLCOS Roadmap 2012 , 2007 .

[79]  E. Deci,et al.  Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. , 2000, The American psychologist.

[80]  George Siemens,et al.  The MOOC model for digital practice , 2010 .

[81]  Luc Bouganim,et al.  Data degradation: making private data less sensitive over time , 2008, CIKM '08.

[82]  Chong Wang,et al.  Reading Tea Leaves: How Humans Interpret Topic Models , 2009, NIPS.

[83]  Ulrik Schroeder,et al.  A reference model for learning analytics , 2012 .

[84]  Mohammad Khalil,et al.  Learning Analytics in MOOCs: Can Data Improve Students Retention and Learning? , 2016 .

[85]  Rebecca Ferguson,et al.  Learning analytics: drivers, developments and challenges , 2012 .

[86]  Zdenek Zdráhal,et al.  Improving retention: predicting at-risk students by analysing clicking behaviour in a virtual learning environment , 2013, LAK '13.

[87]  Kalina Yacef,et al.  Analysis of collaborative writing processes using revision maps and probabilistic topic models , 2013, LAK '13.

[88]  George Siemens,et al.  Ethical and privacy principles for learning analytics , 2014, Br. J. Educ. Technol..

[89]  I. Ajzen,et al.  Attitude-behavior relations: A theoretical analysis and review of empirical research. , 1977 .

[90]  Jeff Grann,et al.  Competency map: visualizing student learning to promote student success , 2014, LAK.

[91]  Florence Martin,et al.  Applying Learning Analytics to Investigate Timed Release in Online Learning , 2016, Technol. Knowl. Learn..

[92]  René F. Kizilcec,et al.  Motivation as a Lens to Understand Online Learners , 2015, ACM Trans. Comput. Hum. Interact..

[93]  Martin Ebner,et al.  What Massive Open Online Course (MOOC) Stakeholders Can Learn From Learning Analytics? , 2016, ArXiv.

[94]  Niall Sclater,et al.  Code of practice for learning analytics , 2015 .

[95]  Martin Ebner,et al.  MOOCs Completion Rates and Possible Methods to Improve Retention - A Literature Review , 2014 .

[96]  Peter Baumgartner,et al.  Developing a Taxonomy for Electronic Portfolios , 2011 .

[97]  Martin Ebner,et al.  What is Learning Analytics about? A Survey of Different Methods Used in 2013-2015 , 2016, ArXiv.

[98]  Chris Piech,et al.  Deconstructing disengagement: analyzing learner subpopulations in massive open online courses , 2013, LAK '13.

[99]  Jure Leskovec,et al.  Engaging with massive online courses , 2014, WWW.

[100]  Tim Rogers,et al.  Modest analytics: using the index method to identify students at risk of failure , 2014, LAK.

[101]  Martin Ebner,et al.  Learning Analytics: From Theory to Practice - Data Support for Learning and Teaching , 2014, CAA.

[102]  D. Rh.  International symposium on pain. , 1973 .

[103]  George Siemens,et al.  Connectivism: Learning Theory or Pastime of the Self-Amused? , 2006 .

[104]  Lewis Elton,et al.  Strategies to enhance student motivation: A conceptual analysis , 1996 .

[105]  Paul Prinsloo,et al.  Big(ger) Data as Better Data in Open Distance Learning. , 2015 .

[106]  Hendrik Drachsler,et al.  Privacy and Analytics – it’s a DELICATE issue. A Checklist to establish trusted Learning Analytics , 2016 .

[107]  Martin Ebner,et al.  Can Microblogs and Weblogs change traditional scientific writing , 2008 .

[108]  Sarah Guri-Rosenblit,et al.  ‘Distance education’ and ‘e-learning’: Not the same thing , 2005 .

[109]  Shalin Hai-Jew,et al.  Iff and Other Conditionals: Expert Perceptions of the Feasibility of Massive Open Online Courses (MOOCs) – A Modified E-Delphi Study , 2014 .

[110]  Divesh Srivastava,et al.  Anonymized Data: Generation, models, usage , 2010, 2010 IEEE 26th International Conference on Data Engineering (ICDE 2010).

[111]  Nobuko Fujita,et al.  at UWindsor , 2018 .

[112]  Björn Hartmann,et al.  Should your MOOC forum use a reputation system? , 2014, CSCW.

[113]  M. Sharples.  Innovating pedagogy: exploring new forms of teaching, learning and assessment, to guide educators and policy makers [2012-] , 2012 .

[114]  Rodney Petersen.  Policy Dimensions of Analytics in Higher Education , 2012 .

[115]  J. Harackiewicz,et al.  Goal setting, achievement orientation, and intrinsic motivation: a mediational analysis. , 1994, Journal of personality and social psychology.

[116]  Michael F. Beaudoin.  Learning or lurking?: Tracking the "invisible" online student , 2002, Internet High. Educ..

[117]  J. Manyika.  Big data: The next frontier for innovation, competition, and productivity , 2011 .

[118]  Carlos Delgado Kloos,et al.  Who are the top contributors in a MOOC? Relating participants' performance and contributions , 2016, J. Comput. Assist. Learn..

[119]  P. Prinsloo,et al.  Learning Analytics , 2013 .

[120]  Justin Reich,et al.  Privacy, Anonymity, and Big Data in the Social Sciences , 2014 .

[121]  Mathieu d'Aquin,et al.  Interpreting data mining results with linked data for learning analytics: motivation, case study and directions , 2013, LAK '13.

[122]  John W. Tukey,et al.  Exploratory Data Analysis. , 1979 .

[123]  Mansureh Kebritchi,et al.  Learning Analytics Methods, Benefits, and Challenges in Higher Education: A Systematic Literature Review. , 2016 .

[124]  W. Lichtenberger,et al.  PLATO: An Automatic Teaching Device , 1961 .

[125]  Sebastián Ventura,et al.  Data mining in education , 2013, WIREs Data Mining Knowl. Discov..

[126]  George Siemens,et al.  Learning analytics: envisioning a research discipline and a domain of practice , 2012, LAK.

[127]  Mike Moore,et al.  Toward a Theory of Independent Learning and Teaching , 1973 .

[128]  Philip H. Winne,et al.  Learning from Learning Kits: gStudy Traces of Students’ Self-Regulated Engagements with Computerized Content , 2006 .

[129]  José A. Ruipérez-Valiente,et al.  Scaling to Massiveness With ANALYSE: A Learning Analytics Tool for Open edX , 2017, IEEE Transactions on Human-Machine Systems.

[130]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[131]  Michael Derntl,et al.  Learning Analytics: Trends and Issues of the Empirical Research of the Years 2011-2014 , 2015, EC-TEL.

[132]  Michalis Nik Xenos,et al.  Considering formal assessment in learning analytics within a PLE: the HOU2LEARN case , 2013, LAK '13.

[133]  Philip S. Yu,et al.  Privacy-preserving data publishing: A survey of recent developments , 2010, CSUR.

[134]  Cristóbal Romero,et al.  Clustering for improving educational process mining , 2014, LAK.

[135]  Balaji Raghunathan,et al.  The Complete Book of Data Anonymization: From Planning to Implementation , 2013 .

[136]  Stuart Palmer,et al.  Modelling Engineering Student Academic Performance Using Academic Analytics , 2013 .

[137]  Rik Van de Walle,et al.  Monitoring Learning Activities in PLE Using Semantic Modelling of Learner Behaviour , 2013, SouthCHI.

[138]  Ryan S. Baker,et al.  The State of Educational Data Mining in 2009: A Review and Future Visions. , 2009, EDM 2009.

[139]  James C. Lester,et al.  Assessing elementary students' science competency with text analytics , 2014, LAK '14.

[140]  Dragan Gasevic,et al.  Learning Analytics – A Growing Field and Community Engagement , 2015 .

[141]  Dragan Gasevic,et al.  Ethics and Privacy as Enablers of Learning Analytics , 2016 .

[142]  Paul Prinsloo,et al.  An evaluation of policy frameworks for addressing ethical considerations in learning analytics , 2013, LAK '13.

[143]  Judy Kay,et al.  The LATUX workflow: designing and deploying awareness tools in technology-enabled learning settings , 2015, LAK.

[144]  Sean P. Goggins,et al.  Learning analytics in CSCL with a focus on assessment: an exploratory study of activity theory-informed cluster analysis , 2014, LAK.

[145]  Martin Ebner,et al.  Clustering patterns of engagement in Massive Open Online Courses (MOOCs): the use of learning analytics to reveal student categories , 2017, J. Comput. High. Educ..

[146]  Doug Clow,et al.  The learning analytics cycle: closing the loop effectively , 2012, LAK.

[147]  Stephanie D. Teasley,et al.  Proceedings of the Fifth International Conference on Learning Analytics And Knowledge , 2014, LAK 2014.

[148]  Thieme Hennis.  The Future of Delft Open Courseware , 2008 .

[149]  Rebecca Ferguson,et al.  Examining engagement: analysing learner subpopulations in massive open online courses (MOOCs) , 2015, LAK.

[150]  Shane Dawson,et al.  Learning designs and learning analytics , 2011, LAK.

[151]  Martin Ebner,et al.  Support of Video-Based lectures with Interactions - Implementation of a first prototype , 2014 .

[152]  Dragan Gasevic,et al.  Importance of Theory in Learning Analytics in Formal and Workplace Settings , 2015 .

[153]  Martin Ebner,et al.  On Using Learning Analytics to Track the Activity of Interactive MOOC Videos , 2016, SE@VBL@LAK.

[154]  David E. Pritchard,et al.  Studying Learning in the Worldwide Classroom Research into edX's First MOOC. , 2013 .

[155]  Justin Reich,et al.  Computer-Assisted Reading and Discovery for Student Generated Text in Massive Open Online Courses , 2014, J. Learn. Anal..

[156]  Rob Nadolski,et al.  Learning Analytics in Serious Gaming: Uncovering the Hidden Treasury of Game Log Files , 2013, GALA.

[157]  T. Grance,et al.  SP 800-122. Guide to Protecting the Confidentiality of Personally Identifiable Information (PII) , 2010 .

[158]  Martin Ebner,et al.  MOOCs as granular systems: design patterns to foster participant activity , 2015 .

[159]  B. Zimmerman.  Attaining self-regulation: A social cognitive perspective. , 2000 .

[160]  Patrick Siehndel,et al.  Technology Enhancing Learning: Past, Present and Future , 2014, EC-TEL.

[161]  Marion Waite,et al.  Learning in a Small, Task-Oriented, Connectivist MOOC: Pedagogical Issues and Implications for Higher Education. , 2013 .

[162]  Dursun Delen,et al.  A comparative analysis of machine learning techniques for student retention management , 2010, Decis. Support Syst..

[163]  Martin Ebner,et al.  Learning Analytics: Principles and Constraints , 2015 .

[164]  Elke Lackner,et al.  DO MOOCS NEED A SPECIAL INSTRUCTIONAL DESIGN , 2014 .

[165]  Shirley Williams,et al.  MOOCs: A systematic study of the published literature 2008-2012 , 2013 .

[166]  Jürgen Helmerich,et al.  Interaktion zwischen Lehrenden und Lernenden in Medien unterstützten Veranstaltungen , 2007 .

[167]  A. Kaplan,et al.  Higher education and the digital revolution: About MOOCs, SPOCs, social media, and the Cookie Monster , 2016 .

[168]  George Siemens,et al.  Content analytics: the definition, scope, and an overview of published research , 2017 .

[169]  Yasmin,et al.  Application of the classification tree model in predicting learner dropout behaviour in open and distance learning , 2013 .

[170]  Martin Ebner,et al.  When Learning Analytics Meets MOOCs - a Review on iMooX Case Studies , 2016, I4CS.

[171]  D. Garrison,et al.  Blended learning: Uncovering its transformative potential in higher education , 2004, Internet High. Educ..

[172]  Tobias Hecking,et al.  Analysis of dynamic resource access patterns in a blended learning course , 2014, LAK.

[173]  Erik Duval,et al.  Goal-oriented visualizations of activity tracking: a case study with engineering students , 2012, LAK.

[174]  Ralf Klamma,et al.  From micro to macro: analyzing activity in the ROLE Sandbox , 2013, LAK '13.

[175]  Christoph Meinel,et al.  openHPI – Das MOOC-Angebot des Hasso-Plattner-Instituts , 2017 .

[176]  Kristy Elizabeth Boyer,et al.  Unsupervised modeling for understanding MOOC discussion forums: a learning analytics approach , 2015, LAK.

[177]  Gautam Biswas,et al.  Early Prediction of Student Dropout and Performance in MOOCs using Higher Granularity Temporal Information , 2014, J. Learn. Anal..

[178]  L. Sweeney.  Simple Demographics Often Identify People Uniquely , 2000 .

[179]  William Wright,et al.  GeoTime Information Visualization , 2004, IEEE Symposium on Information Visualization.

[180]  Albert Sangrà,et al.  MOOC Design Principles. A Pedagogical Approach from the Learner"s Perspective , 2013 .

[181]  T. Anderson,et al.  Three Generations of Distance Education Pedagogy. , 2010 .

[182]  Martin Ebner,et al.  Lurking: An Underestimated Human-Computer Phenomenon , 2005, IEEE Multim..

[183]  Klaus Tochtermann,et al.  Addressing the long tail in empirical research data management , 2012, i-KNOW '12.

[184]  George Siemens,et al.  MOOCs in the News: A European Perspective , 2015 .

[185]  Doug Clow,et al.  MOOCs and the funnel of participation , 2013, LAK '13.

[186]  Rebecca Ferguson,et al.  Visualizing social learning ties by type and topic: rationale and concept demonstrator , 2013, LAK '13.

[187]  Paul Stacey,et al.  Pedagogy of MOOCs , 2014 .

[188]  Carlos Delgado Kloos,et al.  Analysing the Impact of Built-In and External Social Tools in a MOOC on Educational Technologies , 2013, EC-TEL.

[189]  George Siemens,et al.  Learning Analytics , 2013 .

[190]  G. Williamson.  Introduction to Social Research Quantitative and Qualitative Approaches, 2nd edn , 2006 .

[191]  Christopher Brooks,et al.  Visualizing Lecture Capture Usage: A Learning Analytics Case Study , 2013 .

[192]  Yan Zhang,et al.  Longitudinal engagement, performance, and social connectivity: a MOOC case study using exponential random graph models , 2016, LAK.

[193]  D. C. Merrill,et al.  Tutoring: Guided Learning by Doing , 1995 .

[194]  Steven Lonn,et al.  Perceptions and use of an early warning system during a higher education transition program , 2014, LAK.

[195]  H. Fournier,et al.  New dimensions to self-directed learning in an open networked learning environment , 2012 .

[196]  Sheng Tang,et al.  A density-based method for adaptive LDA model selection , 2009, Neurocomputing.

[197]  Matthew D. Pistilli,et al.  Course signals at Purdue: using learning analytics to increase student success , 2012, LAK.

[198]  Jeremy Knox,et al.  From MOOCs to Learning Analytics: Scratching the surface of the 'visual' , 2014, ELERN.

[199]  Albrecht Fortenbacher,et al.  Predicting students' success based on forum activities in MOOCs , 2015, 2015 IEEE 8th International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS).

[200]  Katharina Reinecke,et al.  Demographic differences in how students navigate through MOOCs , 2014, L@S.

[201]  R. Cole.  Issues in Web-Based Pedagogy: A Critical Primer , 2000 .

[202]  Hélène Fournier,et al.  The value of learning analytics to networked learning on a personal learning environment , 2011, LAK.

[203]  M Kalantzis,et al.  The conditions of learning , 2005 .

[204]  Ulrik Schroeder,et al.  Supporting action research with learning analytics , 2013, LAK '13.

[205]  Erik Duval,et al.  Success, activity and drop-outs in MOOCs an exploratory study on the UNED COMA courses , 2014, LAK.

[206]  Mohamed Ally,et al.  FOUNDATIONS OF EDUCATIONAL THEORY FOR ONL INE LEARNING , 2004 .

[207]  Nobuko Fujita,et al.  Scholarship at UWindsor Scholarship at UWindsor Towards Visual Analytics for Teachers’ Dynamic Diagnostic Towards Visual Analytics for Teachers’ Dynamic Diagnostic Pedagogical Decision-Making Pedagogical Decision-Making , 2011 .

[208]  Martin Ebner,et al.  De-Identification in Learning Analytics , 2016, J. Learn. Anal..

[209]  Dear Mr Sotiropoulos.  ARTICLE 29 Data Protection Working Party , 2013 .

[210]  Inmaculada Arnedillo-Sánchez,et al.  Mobile Learning [Guest editor's introduction] , 2007 .

[211]  George Siemens,et al.  Current state and future trends: a citation network analysis of the learning analytics field , 2014, LAK.

[212]  Andrew D. Ho,et al.  Changing “Course” , 2014 .

[213]  John W. Rice,et al.  The Gamification of Learning and Instruction: Game-Based Methods and Strategies for Training and Education , 2012, Int. J. Gaming Comput. Mediat. Simulations.

[214]  Michael Derntl,et al.  A Dynamic Topic Model of Learning Analytics Research , 2013, LAK.

[215]  Helene Fournier,et al.  A pedagogy of abundance or a pedagogy to support human beings? Participant support on massive open online courses , 2011 .

[216]  Sara de Freitas,et al.  Exploratory Analysis in Learning Analytics , 2015, Technology, Knowledge and Learning.

[217]  Gilly Salmon,et al.  E-moderating: the key to teaching and learning online , 2003 .

[218]  Philip C. Abrami,et al.  A methodological morass? How we can improve quantitative research in distance education , 2004 .

[219]  Martin Ebner,et al.  How to foster forum discussions within MOOCs: A case study , 2016 .

[220]  Anastasios A. Economides,et al.  Learning Analytics and Educational Data Mining in Practice: A Systematic Literature Review of Empirical Evidence , 2014, J. Educ. Technol. Soc..

[221]  Ted S. Hasselbring,et al.  Interaction design for improved analytics , 2014, LAK '14.

[222]  Jennifer Rowley,et al.  Conducting a literature review , 2004 .

[223]  Martin D. Snyder.  MOOCs ( Massive Open Online Courses ) Journals and Blogs , 2016 .

[224]  George Siemens,et al.  What public media reveals about MOOCs: A systematic analysis of news reports , 2015, Br. J. Educ. Technol..

[225]  Scott Sanner,et al.  Improving LDA topic models for microblogs via tweet pooling and automatic labeling , 2013, SIGIR.

[226]  Rebecca Ferguson,et al.  Moving Through MOOCS: Pedagogy, Learning Design and Patterns of Engagement , 2015, EC-TEL.

[227]  J. Daniel,et al.  Making Sense of MOOCs : Musings in a Maze of Myth , Paradox and Possibility Author : , 2013 .

[228]  Ryan S. Baker,et al.  Educational Data Mining and Learning Analytics , 2014 .

[229]  Michael Eagle,et al.  Exploring networks of problem-solving interactions , 2015, LAK.

[230]  George Siemens,et al.  The Open Course: Through the Open Door--Open Courses as Research, Learning, and Engagement , 2010 .

[231]  Arthur C. Graesser,et al.  How do you connect?: analysis of social capital accumulation in connectivist MOOCs , 2015, LAK.

[232]  Rada Mihalcea,et al.  Topic Modeling on Historical Newspapers , 2011, LaTeCH@ACL.

[233]  Stephen Downes.  E-learning 2.0 , 2005, ELERN.

[234]  Carlos Delgado Kloos,et al.  Inferring higher level learning information from low level data for the Khan Academy platform , 2013, LAK '13.

[236]  George Siemens,et al.  Let’s not forget: Learning analytics are about learning , 2015 .

[237]  Martin Ebner,et al.  Evaluation Grid for xMOOCs , 2015, iJET.

[238]  Zachary A. Pardos,et al.  Affective states and state tests: investigating how affect throughout the school year predicts end of year learning outcomes , 2013, LAK '13.

[239]  Martin Ebner,et al.  REVENUE VS. COSTS OF MOOC PLATFORMS. DISCUSSION OF BUSINESS MODELS FOR XMOOC PROVIDERS, BASED ON EMPIRICAL FINDINGS AND EXPERIENCES DURING IMPLEMENTATION OF THE PROJECT IMOOX , 2014 .

[240]  Kimberly A. Neuendorf,et al.  The Content Analysis Guidebook , 2001 .

[241]  Martin Ebner,et al.  How to MOOC? – A pedagogical guideline for practitioners , 2014 .

[242]  Marco Kalz,et al.  The MOOC and learning analytics innovation cycle (MOLAC): a reflective summary of ongoing research and its challenges , 2016, J. Comput. Assist. Learn..

[243]  Manuel Benito,et al.  Students' personal networks in virtual and personal learning environments: a case study in higher education using learning analytics approach , 2016, Interact. Learn. Environ..

[244]  Dennis Zielke,et al.  Design and Implementation of a Learning Analytics Toolkit for Teachers , 2012, J. Educ. Technol. Soc..

[245]  Erik Duval,et al.  Addressing learner issues with StepUp!: an evaluation , 2013, LAK '13.

[246]  A. Wise,et al.  Why Theory Matters More than Ever in the Age of Big Data , 2015, J. Learn. Anal..

[247]  Ulrik Schroeder,et al.  An Evaluation of Learning Analytics in a Blended MOOC Environment , 2015 .

[248]  George Siemens,et al.  Where is research on massive open online courses headed? A data analysis of the MOOC research initiative , 2014 .

[249]  Luc Bouganim,et al.  Data confidentiality: to which extent cryptography and secured hardware can help , 2006, Ann. des Télécommunications.

[250]  Dirk Ifenthaler,et al.  Development and Validation of a Learning Analytics Framework: Two Case Studies Using Support Vector Machines , 2014, Technology, Knowledge and Learning.

[251]  George Siemens,et al.  Courses : Innovation in Education ? , 2013 .

[252]  M. Gazzaniga,et al.  Combined spatial and temporal imaging of brain activity during visual selective attention in humans , 1994, Nature.

[253]  Hiroaki Ogata,et al.  Connecting Dots for Ubiquitous Learning Analytics , 2015, ICHL.

[254]  Dragan Gasevic,et al.  Profiling MOOC Course Returners: How Does Student Behavior Change Between Two Course Enrollments? , 2016, L@S.

[255]  S. Schön,et al.  Inverse Blended Learning bei „Gratis Online Lernen“ – über den Versuch, einen Online-Kurs für viele in die Lebenswelt von EinsteigerInnen zu integrieren , 2015 .

[256]  Stephen Aguilar,et al.  GradeCraft: what can we learn from a game-inspired learning management system? , 2013, LAK '13.

[257]  Joseph Jay Williams,et al.  HarvardX and MITx: Two Years of Open Online Courses Fall 2012-Summer 2014 , 2015 .

[258]  George Siemens,et al.  Learning analytics and educational data mining: towards communication and collaboration , 2012, LAK.

[259]  Hendrik Drachsler,et al.  The pulse of learning analytics understandings and expectations from the stakeholders , 2012, LAK.

[260]  Rebecca Eynon,et al.  Structural limitations of learning in a crowd: communication vulnerability and information diffusion in MOOCs , 2014, Scientific Reports.

[261]  Vladan Devedzic,et al.  LOCO-Analyst : semantic web technologies in learning content usage analysis , 2008 .

[262]  Martin Ebner,et al.  A STEM MOOC for school children — What does learning analytics tell us? , 2015, 2015 International Conference on Interactive Collaborative Learning (ICL).

[263]  Martin Ebner,et al.  E-Learning 2.0 = e-Learning 1.0 + Web 2.0? , 2007, The Second International Conference on Availability, Reliability and Security (ARES'07).

[264]  Martin Ebner,et al.  Driving Student Motivation in MOOCs through a Conceptual Activity-Motivation Framework , 2017 .

[265]  Rebecca Ferguson,et al.  An evaluation of learning analytics to identify exploratory dialogue in online discussions , 2013, LAK '13.

[266]  Wen-Shan Chen,et al.  Evaluating the Practices in the E-Learning Platform from the Perspective of Knowledge Management , 2012, OST.

[267]  Ruth Cobos,et al.  eMadrid project: MOOCs and learning analytics , 2016, 2016 International Symposium on Computers in Education (SIIE).

[268]  Tim O'Reilly,et al.  What is Web 2.0: Design Patterns and Business Models for the Next Generation of Software , 2007 .

[269]  Girish Balakrishnan,et al.  Predicting Student Retention in Massive Open Online Courses using Hidden Markov Models , 2013 .

[270]  Martin Ebner,et al.  Impacts of Interactions in Learning-Videos:A Subjective and Objective Analysis , 2015 .

[271]  Avanilde Kemczinski,et al.  A Systematic Mapping on the Learning Analytics Field and Its Analysis in the Massive Open Online Courses Context , 2015, Int. J. Distance Educ. Technol..