Do professors matter?: using an a/b test to evaluate the impact of instructor involvement on MOOC student outcomes

This research investigates the impact professors, and other instructional staff, have on student content knowledge acquisition in a physical science MOOC offered through the University of Illinois at Urbana-Champaign. An A/B test was used to randomly assign MOOC participants in either a control group (with no instructional interaction) or an intervention group (in which the professor and teaching assistants responded to comments in the discussion and complied summary weekly feedback statements) to identify the differences in learning outcomes, participation rates, and student satisfaction. The study found that instructor intervention had no statistically significant impact on overall completion rates, overall badge acquisition rates, student participation rates, or satisfaction with the course, but did (p<0.05) lead to a higher rate of forum badge completion, an area that was targeted by the intervention.

[1]  Marilyn E. Laiken.  Understanding and Promoting Transformative Learning: A Guide for Educators of Adults , 2013 .

[2]  Dennis Beck,et al.  The other side of the LMS: Considering implementation and use in the adoption of an LMS in online and blended learning environments , 2007 .

[3]  Roy Williams,et al.  The Ideals and Reality of Participating in a MOOC , 2010, Proceedings of the International Conference on Networked Learning.

[4]  Wolfgang Nejdl,et al.  Extendible Adaptive Hypermedia Courseware: Integrating Different Courses and Web Material , 2000, AH.

[5]  Mark Bullen,et al.  Interaction and interpersonality in online discussion forums , 2005 .

[6]  A. P. Rovai.  Sense of community, perceived cognitive learning, and persistence in asynchronous learning networks , 2002, Internet High. Educ..

[7]  David W. Johnson,et al.  A Theory of CooperATion And CompeTiTion , 2014 .

[8]  R. Palloff,et al.  Collaborating Online: Learning Together in Community , 2006 .

[9]  Beverly Park Woolf,et al.  Building Intelligent Interactive Tutors: Student-centered Strategies for Revolutionizing E-learning , 2008 .

[10]  JeongMin Lee,et al.  Personality Types and Learners' Interaction in Web-Based Threaded Discussion , 2006 .

[11]  J. Ludvigsson,et al.  McKEACHIE’S TEACHING TIPS , 2003, Journal of Pediatric Gastroenterology and Nutrition.

[12]  Marilla D. Svinicki,et al.  McKeachie''s Teaching Tips , 2006 .

[13]  S. Brookfield.  Becoming a Critically Reflective Teacher. Jossey-Bass Higher and Adult Education Series. , 1995 .

[14]  J. Hardy.  What the Best College Teachers Do , 2007 .

[15]  A. Wise,et al.  The Effects of Teacher Social Presence on Student Satisfaction, Engagement, and Learning , 2004, ICLS.

[16]  Avner Caspi,et al.  A Comparison of Non-Mandatory Online Dialogic Behavior in Two Higher Education Blended Environments , 2012 .

[17]  Margaret Mazzolini,et al.  When to jump in: The role of the instructor in online discussion forums , 2007, Comput. Educ..

[18]  Starr Roxanne Hiltz,et al.  Contextual Factors That Influence ALN Effectiveness , 2004 .

[19]  M. Saadatmand,et al.  Content aggregation and knowledge sharing in a personal learning environment: Serendipitous and emergent learning in open online networks , 2012, 2012 15th International Conference on Interactive Collaborative Learning (ICL).

[20]  A. W. Bates,et al.  Effective Teaching with Technology in Higher Education: Foundations for Success , 2003 .

[21]  Starr Roxanne Hiltz,et al.  Becoming a Virtual Professor: Pedagogical Roles and Asynchronous Learning Networks , 2002, J. Manag. Inf. Syst..

[22]  Margaret Mazzolini,et al.  Sage, guide or ghost? The effect of instructor intervention on student participation in online discussion forums , 2003, Comput. Educ..

[23]  P. Cranton,et al.  Understanding and Promoting Transformative Learning: A Guide for Educators of Adults. Jossey-Bass Higher and Adult Education Series. , 1994 .

[24]  David W. Park,et al.  Interpersonal Effects in Computer-Mediated Interaction , 1994 .

[25]  N. Lewis.  A dynamic theory of personality , 1935 .

[26]  George Siemens,et al.  The MOOC model for digital practice , 2010 .

[27]  Ann L. Brown,et al.  Guided discovery in a community of learners. , 1994 .

[28]  D. Randy Garrison,et al.  Blended Learning in Higher Education: Framework, Priciples and Guidlines , 2007 .

[29]  Mia Stern,et al.  Adaptive Content in an Online Lecture System , 2000, AH.

[30]  Janet Mannheimer Zydney,et al.  Creating a community of inquiry in online environments: An exploratory study on the effect of a protocol on interactions within asynchronous discussions , 2012, Comput. Educ..

[31]  Deana L. Molinari,et al.  The Role of Social Comments in Problem-Solving Groups in an Online Class , 2004 .

[32]  D. Randy Garrison,et al.  Critical Inquiry in a Text-Based Environment: Computer Conferencing in Higher Education , 1999, Internet High. Educ..

[33]  A. P. Rovai,et al.  Blended Learning and Sense of Community: A Comparative Analysis with Traditional and Fully Online Graduate Courses , 2004 .

[34]  Raven McCrory,et al.  The Decentered Teacher and the Construction of Social Space in the Virtual Classroom , 2005, Teachers College Record: The Voice of Scholarship in Education.

[35]  Donna J. Charlevoix,et al.  Design and Implementation of Inquiry-Based, Technology-Rich Learning Activities in a Large-Enrollment Blended Learning Course , 2009 .

[36]  Rose M. Marra,et al.  A Comparative Analysis of Online Discussion Participation Protocols , 2005 .

[37]  Karen Swan,et al.  Developing a community of inquiry instrument: Testing a measure of the Community of Inquiry framework using a multi-institutional sample , 2008, Internet High. Educ..

[38]  A. Hwang,et al.  Does "teaching presence" exist in online MBA courses? , 2006, Internet High. Educ..

[39]  F. Bell.  Connectivism: Its place in theory-informed research and innovation in technology-enabled learning , 2011 .