Predicting course outcomes with digital textbook usage data

Abstract Digital textbook analytics are a new method of collecting student-generated data in order to build predictive models of student success. Previous research using self-report or laboratory measures of reading show that engagement with the textbook was related to student learning outcomes. We hypothesized that an engagement index based on digital textbook usage data would predict student course grades. Linear regression analyses were conducted using data from 233 students to determine whether digital textbook usage metrics predicted final course grades. A calculated linear index of textbook usage metrics was significantly predictive of final course grades and was a stronger predictor of course outcomes than previous academic achievement. However, time spent reading, one of the variables that make up the index was more strongly predictive of course outcomes. Additionally, students who were in the top 10th percentile in number of highlights had significantly higher course grades than those in the lower 90th percentile. These findings suggest that digital textbook analytics are an effective early warning system to identify students at risk of academic failure. These data can be collected unobtrusively and automatically and provide stronger prediction of outcomes than prior academic achievement (which to this point has been the single strongest predictor of student success).

[1]  Cai Grau,et al.  Evaluation of Objective Measures of Smoking Status A Prospective Clinical Study in a Group of Head and Neck Cancer Patients Treated with Radiotherapy , 2003, Acta oncologica.

[2]  M. Deberard,et al.  Predictors of Academic Achievement and Retention among College Freshmen: A Longitudinal Study. , 2004 .

[3]  John Limber,et al.  Reading Skill, Textbook Marking, and Course Performance , 2009 .

[4]  George D. Kuh,et al.  What Student Affairs Professionals Need to Know About Student Engagement , 2009 .

[5]  Shana K. Carpenter,et al.  Using Spacing to Enhance Diverse Forms of Learning: Review of Recent Research and Implications for Instruction , 2012 .

[6]  Yu-Fen Yang,et al.  Developing a reciprocal teaching/learning system for college remedial reading instruction , 2010, Comput. Educ..

[7]  P. Pintrich,et al.  Disciplinary Differences in Self-Regulated Learning in College Students , 1996, Contemporary educational psychology.

[8]  Catherine Garrison,et al.  Formative and Summative Assessments in the Classroom , 2011 .

[9]  Reynol Junco,et al.  Comparing actual and self-reported measures of Facebook use , 2013, Comput. Hum. Behav..

[10]  Zane L. Berge,et al.  Learning analytics as a tool for closing the assessment loop in higher education , 2012 .

[11]  R. Eric Landrum,et al.  Assessments of Textbook Usage and the Relationship to Student Course Performance , 2012 .

[12]  Ruomeng Zhao,et al.  Effects of online note taking formats and self-monitoring prompts on learning from online text: Usin , 2011 .

[13]  Shane Dawson,et al.  Numbers Are Not Enough. Why e-Learning Analytics Failed to Inform an Institutional Strategic Plan , 2012, J. Educ. Technol. Soc..

[14]  Carrie Spencer,et al.  Research on Learners' Preferences for Reading from a Printed Text or from a Computer Screen. , 2006 .

[15]  Hendrik Drachsler,et al.  Translating Learning into Numbers: A Generic Framework for Learning Analytics , 2012, J. Educ. Technol. Soc..

[16]  Rebecca Ferguson,et al.  Learning analytics: drivers, developments and challenges , 2012 .

[17]  Pei-Hsuan Hsieh,et al.  The Instructional Effect of Online Reading Strategies and Learning Styles on Student Academic Achievement , 2009, J. Educ. Technol. Soc..

[18]  Carlos Salas,et al.  Objective vs. Self-Reported Physical Activity and Sedentary Time: Effects of Measurement Method on Relationships with Risk Biomarkers , 2012, PloS one.

[19]  Shane Dawson,et al.  Teaching smarter: how mining ICT data can inform and improve learning and teaching practice , 2008 .

[20]  David B. Daniel,et al.  E-textbooks at what cost? Performance and use of electronic v. print texts , 2013, Comput. Educ..

[21]  A. Taylor,et al.  Students Learn Equally Well From Digital as From Paperbound Texts , 2011 .

[22]  J. Vermunt.  Relations between student learning patterns and personal and contextual factors and academic performance , 2005 .

[23]  Matthew D. Pistilli,et al.  Course signals at Purdue: using learning analytics to increase student success , 2012, LAK.

[24]  Erik Duval,et al.  Dataset-Driven Research to Support Learning and Knowledge Analytics , 2012, J. Educ. Technol. Soc..

[25]  Anthony G. Picciano.  The Evolution of Big Data and Learning Analytics in American Higher Education , 2012 .

[26]  Reynol Junco,et al.  iSpy: seeing what students really do online , 2014 .

[27]  S. C. Ehrmann.  IMPLEMENTING THE SEVEN PRINCIPLES: Technology as Lever , 2004 .

[28]  Vernon C. Smith,et al.  Predictive Modeling to Forecast Student Outcomes and Drive Effective Interventions in Online Community College Courses , 2012 .

[29]  David B. Daniel,et al.  E-books or textbooks: Students prefer textbooks , 2010, Comput. Educ..

[30]  A. Willford.  Secondary School Course Grades and Success in College. , 2009 .

[31]  Esther Usó-Juan,et al.  Reading Printed versus Online Texts. A Study of EFL Learners' Strategic Reading Behavior , 2009 .

[32]  Kate Sandberg,et al.  College Student Academic Online Reading: A Review of the Current Literature , 2011 .

[33]  Benjamin Littenberg,et al.  Relationship Between Self‐report and an Objective Measure of Television‐viewing Time in Adults , 2010, Obesity.

[34]  Amanda J. Rockinson-Szapkiw,et al.  Electronic versus traditional print textbooks: A comparison study on the influence of university students' learning , 2013, Comput. Educ..

[35]  Nathaniel J. Bray,et al.  How College Affects Students: A Third Decade of Research (review) , 2006, The Journal of General Education.

[36]  Christian Penny,et al.  A Nook or a Book? Comparing College Students' Reading Comprehension Levels, Critical Reading, and Study Skills , 2011 .

[37]  James Longhurst,et al.  World History on the World Wide Web: A Student Satisfaction Survey and a Blinding Flash of the Obvious. , 2003 .

[38]  Sally Hamouda,et al.  Exploring students learning behavior with an interactive etextbook in computer science courses , 2014, Comput. Hum. Behav..

[39]  J. Manyika.  Big data: The next frontier for innovation, competition, and productivity , 2011 .

[40]  Jenny Bossaller,et al.  Faculty Views on eTextbooks: A Narrative Study , 2014 .

[41]  Michael Weisberg,et al.  Student Attitudes and Behaviors Towards Digital Textbooks , 2011 .

[42]  Jeffrey R. Young.  The Object Formerly Known as the Textbook. , 2013 .

[43]  K. Lonka,et al.  Aspects and Prospects of Measuring Studying and Learning in Higher Education , 2004 .

[44]  A. Astin.  Student involvement: A developmental theory for higher education. , 1999 .

[45]  Lee Rainie,et al.  Tablet and E-book reader Ownership Nearly Double Over the Holiday Gift-Giving Period , 2012 .

[46]  J. Rouet,et al.  The Impact of Presentation Format, Task Assignment, and Prior Knowledge on Students' Comprehension of Multiple Online Documents , 2007 .

[47]  Saul Geiser,et al.  Validity of High-School Grades in Predicting Student Success beyond the Freshman Year: High-School Record vs. Standardized Tests as Indicators of Four-Year College Outcomes. Research & Occasional Paper Series: CSHE.6.07. , 2007 .

[48]  P. A. Cohen.  Student Ratings of Instruction and Student Achievement: A Meta-analysis of Multisection Validity Studies , 1981 .

[49]  F. Worrell,et al.  Formative and Summative Assessment in the Classroom , 2016 .

[50]  David Dowling,et al.  In search of the key factors that influence student success at university , 2005 .

[51]  J. Alderson,et al.  The effect of students' academic discipline on their performance on ESP reading tests , 1985 .

[52]  Lori S. Cook,et al.  An Exploratory Analysis of Textbook Usage and Study Habits: Misperceptions and Barriers to Success , 2010 .

[53]  A. Chickering,et al.  Seven Principles for Good Practice in Undergraduate Education , 1987, CORE.

[54]  David G. O'Brien,et al.  Reading Multimodally: What Is Afforded? , 2011, Journal of Adolescent & Adult Literacy.