Using Learning Analytics to Predict At-Risk Students in Online Graduate Public Affairs and Administration Education

Abstract In this global information age, schools that teach public affairs and administration must meet the needs of students. Increasingly, this means providing students information in online classrooms to help them reach their highest potential. The acts of teaching and learning online generate data, but to date, that information has remained largely untapped for assessing student performance. Using data generated by students in an online Master of Public Administration program, drawn from the Marist College Open Academic Analytics Initiative,1 we identify and analyze characteristics and behaviors that best provide early indication of a student being academically at risk, paying particular attention to the usage of online tools. We find that fairly simple learning analytics models achieve high levels of sensitivity (over 80% of at-risk students identified) with relatively low false positive rates (13.5%). Results will be used to test interventions for improving student performance in real time.

[1]  Carol Ebdon,et al.  Teaching Public Finance Administration Online: A Case Study , 1999 .

[2]  Cathy Owens Swift,et al.  Retaining the Online Learner: Profile of Students in an Online MBA Program and Implications for Teaching Them , 2004 .

[3]  D. Granger.  Engaging the Online Learner: Acitivities and Resources for Creative Instruction , 2005 .

[4]  Eitel J. M. Lauría,et al.  Early Alert of Academically At-Risk Students: An Open Source Analytics Initiative , 2014, J. Learn. Anal..

[5]  H. Salzman,et al.  Making the grade , 2008, Nature.

[6]  Dianne Rahm,et al.  Internet-Mediated Learning in Public Affairs Programs: Issues and Implications. , 1999 .

[7]  E. Rogers,et al.  Diffusion of innovations , 1964, Encyclopedia of Sport Management.

[8]  M. Mitchell Waldrop,et al.  Massive Open Online Courses, aka MOOCs, Transform Higher Education and Science , 2014 .

[9]  Michael S. Mingus.  Toward Understanding the Culture of Internet-Mediated Learning. , 1999 .

[10]  Abdullah Karaksha,et al.  Student Engagement in Pharmacology Courses Using Online Learning Tools , 2013, American Journal of Pharmaceutical Education.

[11]  Selma Vonderwell,et al.  Asynchronous Discussions and Assessment in Online Learning , 2007 .

[12]  C. C. Robinson,et al.  New Benchmarks in Higher Education: Student Engagement in Online Learning , 2008 .

[13]  George D. Kuh.  What We're Learning About Student Engagement From NSSE: Benchmarks for Effective Educational Practices , 2003 .

[14]  M. H. Ginn,et al.  Online Education in Public Affairs: Current State and Emerging Issues , 2012 .

[15]  Maryann Jacobi.  College Student Outcomes Assessment: A Talent Development Perspective. ASHE-ERIC Higher Education Report No. 7, 1987. , 1987 .

[16]  Ralph S. Brower,et al.  Distance Learning: Some Fundamental Questions for Public Affairs Education , 2000 .

[17]  Linda F. Cornelious,et al.  Preparing Instructors for Quality Online Instruction. , 2005 .

[18]  Roger Richman,et al.  The High Tech MPA: Distance Learning Technology and Graduate Public Administration Education , 1997 .

[19]  John R. Grandzol.  Teaching MBA Statistics Online: A Pedagogically Sound Process Approach , 2004 .

[20]  Laurie P. Dringus,et al.  Using data mining as a strategy for assessing asynchronous discussion forums , 2005, Comput. Educ..

[21]  Heather E. Campbell.  Cheating, Public Administration Education, and Online Courses: An Essay and Call to Arms , 2006 .

[22]  G. Stowers.  Getting Left behind? Gender Differences in Computer Conferencing , 1995 .

[23]  Osmar R. Zaïane,et al.  Web Usage Mining for a Better Web-Based Learning Environment , 2001 .

[24]  S. Gall.  Help-seeking: An understudied problem-solving skill in children☆ , 1981 .

[25]  Catherine L. Finnegan,et al.  Predicting Retention in Online General Education Courses , 2005 .

[26]  Selma Vonderwell,et al.  Promoting Formative Assessment in Online Teaching and Learning , 2013 .

[27]  Eitel J. M. Lauría,et al.  Mining academic data to improve college student retention: an open source perspective , 2012, LAK.

[28]  Thomas J. Barth,et al.  Teaching PA Online: Reflections of a Skeptic , 2004 .

[29]  I. E. Allen,et al.  Making the Grade: Online Education in the United States, 2006. , 2006 .

[30]  Pamela A. Gibson,et al.  Creating Quality Online Course Design through a Peer-Reviewed Assessment , 2012 .

[31]  Eitel J. M. Lauría,et al.  Open academic analytics initiative: initial research findings , 2013, LAK '13.

[32]  J. W. Gikandi,et al.  Online formative assessment in higher education: A review of the literature , 2011, Comput. Educ..

[33]  Eric K. Austin,et al.  Limits to Technology-Based Distance Education in MPA Curricula , 2009 .

[34]  Ellen B. Mandinach,et al.  The Development of Effective Evaluation Methods for E-Learning: A Concept Paper and Action Plan , 2005, Teachers College Record: The Voice of Scholarship in Education.

[35]  G. Stowers.  Computer Conferencing in the Public Affairs Classroom , 1999 .

[36]  Valentina Klenowski,et al.  Creating Communities of Shared Practice: The challenges of assessment use in learning and teaching , 2002 .

[37]  Andrew Feenberg.  Distance Learning: Promise or Threat? , 1999 .

[38]  I. E. Allen,et al.  Going the Distance: Online Education in the United States, 2011. , 2011 .

[39]  Rena M. Palloff,et al.  Book Review: Building online learning communities: Effective strategies for the virtual classroom , 2007 .

[40]  T. Davenport.  Competing on analytics. , 2006, Harvard business review.

[41]  Kimberly E. Arnold.  Signals: Applying Academic Analytics. , 2010 .

[42]  Alex van der Merwe.  Can Online Learning Boost Academic Performance? A Microeconomics Study , 2011 .

[43]  Dianne Rahm,et al.  Going Remote: The Use of Distance Learning, the World Wide Web, and the Internet in Graduate Programs of Public Affairs and Administration , 1997 .

[44]  John Patrick Campbell.  Utilizing student data within the course management system to determine undergraduate student academic success: An exploratory study , 2007 .

[45]  Xin Ma,et al.  Hierarchical Linear Modelling of Student and School Effects on Academic Achievement , 2000 .