Predicting Gaps in Usage in a Phone-Based Literacy Intervention System

Educational technologies may help support out-of-school learning in contexts where formal schooling fails to reach every child, but children may not persist in using such systems to learn at home. Prior research has developed methods for predicting learner dropout but primarily for adults in formal courses and Massive Open Online Courses (MOOCs), not for children’s voluntary ed tech usage. To support early literacy in rural contexts, our research group developed and deployed a phone-based literacy technology with rural families in Côte d’Ivoire in two longitudinal studies. In this paper, we investigate the feasibility of using time-series classification models trained on system log data to predict gaps in children’s voluntary usage of our system in both studies. We contribute insights around important features associated with sustained system usage, such as children’s patterns of use, performance on the platform, and involvement from other adults in their family. Finally, we contribute design implications for predicting and supporting learners’ voluntary, out-of-school usage of mobile learning applications in rural contexts.

[1]  José Manuel Benítez,et al.  On the use of cross-validation for time series predictor evaluation , 2012, Inf. Sci..

[2]  Christopher Lange,et al.  Opportunities and Lessons from Informal and Non-formal Learning: Applications to Online Environments , 2015 .

[3]  M. Ames.  Can One Laptop per Child Save the World's Poor? , 2011 .

[4]  Y. L. Jack Lam.  PREDICTING DROPOUTS OF UNIVERSITY FRESHMEN: A LOGIT REGRESSION ANALYSIS , 1984 .

[5]  Bernhard Schölkopf,et al.  The Kernel Trick for Distances , 2000, NIPS.

[6]  Jeswin Jose.  Predicting Customer Retention of an App-Based Business Using Supervised Machine Learning , 2019 .

[7]  Wonjong Rhee,et al.  Churn prediction of mobile and online casual games using play log data , 2017, PloS one.

[8]  Alladi Venkatesh,et al.  Mobile Marketing in the Retailing Environment: Current Insights and Future Research Avenues , 2010 .

[9]  Amy Ogan,et al.  "Everyone Brings Their Grain of Salt": Designing for Low-Literate Parental Engagement with a Mobile Literacy Technology in Côte d'Ivoire , 2019, CHI.

[10]  Acheson Charles Phiri,et al.  Evaluation of a pilot project on information and communication technology for rural education development: A Cofimvaba case study on the educational use of tablets , 2014 .

[11]  Anna Guitart,et al.  Churn Prediction in Mobile Social Games: Towards a Complete Assessment Using Survival Ensembles , 2016, 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA).

[12]  Arthur Tatnall.  Editorial for EAIT issue 1, 2019 , 2018, Education and Information Technologies.

[13]  Matthew Kam,et al.  Improving literacy in developing countries using speech recognition-supported games on mobile devices , 2012, CHI.

[14]  Zhang Xiong,et al.  Time Series Model for Predicting Dropout in Massive Open Online Courses , 2018, AIED.

[15]  B. Roe,et al.  Boosted decision trees as an alternative to artificial neural networks for particle identification , 2004, physics/0408124.

[16]  Gail L. Zellman,et al.  Understanding the Impact of Parent School Involvement on Children's Educational Outcomes , 1998 .

[17]  Ernest T. Pascarella,et al.  Predicting freshman persistence and voluntary dropout decisions: A replication , 1981 .

[18]  Michael A. Herbert.  Staying the Course: A Study in Online Student Satisfaction and Retention , 2006 .

[19]  Eric W. T. Ngai,et al.  Customer churn prediction using improved balanced random forests , 2009, Expert Syst. Appl..

[20]  Judith Uchidiuno,et al.  Designing Appropriate Learning Technologies for School vs Home Settings in Tanzanian Rural Villages , 2018, COMPASS.

[21]  Jihie Kim,et al.  Predicting Student Attrition in MOOCs using Sentiment Analysis and Neural Networks , 2015, AIED Workshops.

[22]  L. Petitto,et al.  Age of Bilingual Exposure Is Related to the Contribution of Phonological and Semantic Knowledge to Successful Reading Development. , 2018, Child development.

[23]  Zane L. Berge,et al.  A Model for Sustainable Student Retention: A Holistic Perspective on the Student Dropout Problem with Special Attention to e-Learning , 2004 .

[24]  Alexandra I. Cristea,et al.  Early Dropout Prediction for Programming Courses Supported by Online Judges , 2019, AIED.

[25]  Carolyn Penstein Rosé,et al.  “ Turn on , Tune in , Drop out ” : Anticipating student dropouts in Massive Open Online Courses , 2013 .

[26]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[27]  Tapan S. Parikh,et al.  Avaaj Otalo: a field study of an interactive voice forum for small farmers in rural India , 2010, CHI.

[28]  Evelyn Yarzebinski,et al.  "You give a little of yourself": family support for children's use of an IVR literacy system , 2019, COMPASS.

[29]  Jin-Hyuk Hong,et al.  Understanding and prediction of mobile application usage for smart phones , 2012, UbiComp.

[30]  Scott Lundberg,et al.  A Unified Approach to Interpreting Model Predictions , 2017, NIPS.

[31]  P. McEwan.  Improving Learning in Primary Schools of Developing Countries , 2015 .

[32]  Katharine M. Conn.  Identifying Effective Education Interventions in Sub-Saharan Africa: A Meta-Analysis of Impact Evaluations , 2017 .

[33]  Ji-Hye Park,et al.  Factors Influencing Adult Learners' Decision to Drop Out or Persist in Online Learning , 2009, J. Educ. Technol. Soc..

[34]  Vincent Tinto,et al.  Research and Practice of Student Retention: What Next? , 2006 .

[35]  Matthew Kam,et al.  Mobile Gaming with Children in Rural India: Contextual Factors in the Use of Game Design Patterns , 2007, DiGRA Conference.

[36]  Matthew Kam,et al.  Improving literacy in rural India: cellphone games in an after-school program , 2009, 2009 International Conference on Information and Communication Technologies and Development (ICTD).

[37]  Wendy C. Kemp.  Persistence of Adult Learners in Distance Education , 2002 .

[38]  Rachel Wells,et al.  Household poverty classification in data-scarce environments: a machine learning approach , 2017, 1711.06813.

[39]  Andrew J. Saltarelli,et al.  Who Takes MOOCs , 2016 .

[40]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[41]  Mamoru Ishikawa,et al.  Schooling, basic skills and economic outcomes , 2002 .

[42]  A. Asmala,et al.  THE EFFECTS OF TRAINING SET SIZE ON THE ACCURACY OF MAXIMUM LIKELIHOOD, NEURAL NETWORK AND SUPPORT VECTOR MACHINE CLASSIFICATION , 2014 .

[43]  Keith Tyler-Smith.  Early Attrition among First Time eLearners: A Review of Factors that Contribute to Drop-out, Withdrawal and Non-completion Rates of Adult Learners undertaking eLearning Programmes , 2006 .

[44]  D. Schunk,et al.  Self-Efficacy and Academic Motivation , 1991 .

[45]  Hasan Dağ,et al.  Machine learning model to predict an adult learner’s decision to continue ESOL course , 2019, Education and Information Technologies.

[46]  Paul Denny,et al.  The effect of virtual achievements on student engagement , 2013, CHI.

[47]  Alyssa R. Gonzalez-DeHass,et al.  Examining the Relationship Between Parental Involvement and Student Motivation , 2005 .

[48]  Evelyn Yarzebinski,et al.  Collective Support and Independent Learning with a Voice-Based Literacy Technology in Rural Communities , 2020, CHI.

[49]  Leonard J. Tashman,et al.  Out-of-sample tests of forecasting accuracy: an analysis and review , 2000 .

[50]  Gwo-Dong Chen,et al.  Effects of Push Notifications on Learner Engagement in a Mobile Learning App , 2016, 2016 IEEE 16th International Conference on Advanced Learning Technologies (ICALT).

[51]  B. Zimmerman,et al.  Self-Efficacy: An Essential Motive to Learn. , 2000, Contemporary educational psychology.