Equality of Learning Opportunity via Individual Fairness in Personalized Recommendations

Online education platforms play an increasingly important role in mediating the success of individuals’ careers. Therefore, while building overlying content recommendation services, it becomes essential to guarantee that learners are provided with equal recommended learning opportunities, according to the platform principles, context, and pedagogy. Though the importance of ensuring equality of learning opportunities has been well investigated in traditional institutions, how this equality can be operationalized in online learning ecosystems through recommender systems is still under-explored. In this paper, we shape a blueprint of the decisions and processes to be considered in the context of equality of recommended learning opportunities, based on principles that need to be empirically-validated (no evaluation with live learners has been performed). To this end, we first provide a formalization of educational principles that model recommendations’ learning properties, and a novel fairness metric that combines them to monitor the equality of recommended learning opportunities among learners. Then, we envision a scenario wherein an educational platform should be arranged in such a way that the generated recommendations meet each principle to a certain degree for all learners, constrained to their individual preferences. Under this view, we explore the learning opportunities provided by recommender systems in a course platform, uncovering systematic inequalities. To reduce this effect, we propose a novel post-processing approach that balances personalization and equality of recommended opportunities. Experiments show that our approach leads to higher equality, with a negligible loss in personalization. This paper provides a theoretical foundation for future studies of learners’ preferences and limits concerning the equality of recommended learning opportunities.

[1]  Huzefa Rangwala,et al.  Grade Prediction Based on Cumulative Knowledge and Co-taken Courses , 2019, EDM.

[2]  Abraham Bernstein,et al.  Updatable, Accurate, Diverse, and Scalable Recommendations for Interactive Applications , 2016, ACM Trans. Interact. Intell. Syst..

[3]  Sunil Rai,et al.  Recommender System in eLearning: A Survey , 2020 .

[4]  George Karypis,et al.  RecWalk: Nearly Uncoupled Random Walks for Top-N Recommendation , 2019, WSDM.

[5]  Irenee R. Beattie,et al.  Connecting in Class? College Class Size and Inequality in Academic Social Capital , 2016 .

[6]  Qian Zhang,et al.  Modeling and Predicting Learning Behavior in MOOCs , 2016, WSDM.

[7]  Thorsten Joachims,et al.  Policy Learning for Fairness in Ranking , 2019, NeurIPS.

[8]  Miroslav Dudík,et al.  Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need? , 2018, CHI.

[9]  Krishna P. Gummadi,et al.  iFair: Learning Individually Fair Data Representations for Algorithmic Decision Making , 2018, 2019 IEEE 35th International Conference on Data Engineering (ICDE).

[10]  Christopher G. Brinton,et al.  Course recommendation as graphical analysis , 2018, 2018 52nd Annual Conference on Information Sciences and Systems (CISS).

[11]  Peter Brusilovsky,et al.  Learning Content Recommender System for Instructors of Programming Courses , 2018, AIED.

[12]  Kalina Yacef,et al.  Does a Peer Recommender Foster Students' Engagement in MOOCs? , 2016, EDM.

[13]  Stephen Darwin,et al.  What contemporary work are student ratings actually doing in higher education , 2017 .

[14]  Luca Oneto,et al.  Fairness in Machine Learning , 2020, INNSBDDL.

[15]  Ben Green,et al.  The Myth in the Methodology: Towards a Recontextualization of Fairness in Machine Learning , 2018, ICML 2018.

[16]  Jie Tang,et al.  Understanding Dropouts in MOOCs , 2019, AAAI.

[17]  Hao Wu,et al.  Adaptive Learning Material Recommendation in Online Language Education , 2019, AIED.

[18]  Carina Girvan,et al.  What is a virtual world? Definition and classification , 2018 .

[19]  Ludovico Boratto,et al.  On the negative impact of social influence in recommender systems: A study of bribery in collaborative hybrid algorithms , 2020, Inf. Process. Manag..

[20]  Yuen Yi Lo,et al.  Content familiarity, task repetition and Chinese EFL learners’ engagement in second language use , 2017 .

[21]  Emma Brunskill,et al.  Fairer but Not Fair Enough On the Equitability of Knowledge Tracing , 2019, LAK.

[22]  Rebecca Ferguson,et al.  Ethical and privacy issues in the application of learning analytics , 2015, LAK.

[23]  Niels Pinkwart Another 25 Years of AIED? Challenges and Opportunities for Intelligent Educational Technologies of the Future , 2016, International Journal of Artificial Intelligence in Education.

[24]  L. S. Vygotskiĭ,et al.  Mind in society : the development of higher psychological processes , 1978 .

[25]  Ghassan Beydoun,et al.  Deep-Cross-Attention Recommendation Model for Knowledge Sharing Micro Learning Service , 2020, AIED.

[26]  Francisco Iniesto,et al.  ETHICS in AIED: Who Cares? An EC-TEL workshop , 2019 .

[27]  Feng Liu,et al.  Curriculum Design in Professional Education: Theory and Practice , 2018 .

[28]  Kwankamol Nongpong,et al.  Recommender Systems for university elective course recommendation , 2017, 2017 14th International Joint Conference on Computer Science and Software Engineering (JCSSE).

[29]  Ed H. Chi,et al.  Fairness in Recommendation Ranking through Pairwise Comparisons , 2019, KDD.

[30]  Harald Steck,et al.  Calibrated recommendations , 2018, RecSys.

[31]  Bert Huang,et al.  Beyond Parity: Fairness Objectives for Collaborative Filtering , 2017, NIPS.

[32]  Diego Reforgiato Recupero,et al.  COCO: Semantic-Enriched Collection of Online Courses at Scale with Experimental Use Cases , 2018, WorldCIST.

[33]  Krishna P. Gummadi,et al.  Equity of Attention: Amortizing Individual Fairness in Rankings , 2018, SIGIR.

[34]  Abeer Alsadoon,et al.  A systematic review: machine learning based recommendation systems for e-learning , 2019, Education and Information Technologies.

[35]  A. Mood,et al.  Equality of Educational Opportunity. , 1967 .

[36]  Carrie Demmans Epp,et al.  CSCLRec: Personalized Recommendation of Forum Posts to Support Socio-collaborative Learning , 2020, EDM.

[37]  Sanjay Mohapatra,et al.  Adopting MOOCs for afforable quality education , 2017, Education and Information Technologies.

[38]  Nour-Eddine El Faddouli,et al.  Predicting Learners Need for Recommendation Using Dynamic Graph-Based Knowledge Tracing , 2020, AIED.

[39]  María Fernández-Mellizo,et al.  Inequality of educational opportunities: School failure trends in Spain (1977–2012) , 2017 .

[40]  Ashok K. Goel,et al.  The Unexpected Pedagogical Benefits of Making Higher Education Accessible , 2016, L@S.

[41]  Gerasimos Spanakis,et al.  It's a Match! Reciprocal Recommender System for Graduating Students and Jobs , 2019, EDM.

[42]  Krishna P. Gummadi,et al.  Operationalizing Individual Fairness with Pairwise Fair Representations , 2019, Proc. VLDB Endow..

[43]  Jade Goldstein-Stewart,et al.  The use of MMR, diversity-based reranking for reordering documents and producing summaries , 1998, SIGIR '98.

[44]  Francisco Iniesto,et al.  Guidance on How Learning at Scale Can be Made More Accessible , 2020, L@S.

[45]  Longbing Cao,et al.  CoupledCF: Learning Explicit and Implicit User-item Couplings in Recommendation for Deep Collaborative Filtering , 2018, IJCAI.

[46]  S. Konomi,et al.  Course Recommendation for University Environment , 2020, EDM.

[47]  Gianni Fenu,et al.  The Effect of Algorithmic Bias on Recommender Systems for Massive Open Online Courses , 2019, ECIR.

[48]  Raphaël Morsomme,et al.  Content-based Course Recommender System for Liberal Arts Education , 2019, EDM.

[49]  Peter Brusilovsky,et al.  Recommending Remedial Readings Using Student's Knowledge state , 2020, EDM.

[50]  Colin Cooper,et al.  Random walks in recommender systems: exact computation and simulations , 2014, WWW.

[51]  Kirsten Meyer,et al.  Why should we demand equality of educational opportunity? , 2016 .

[52]  Pradeep K. Sinha,et al.  Personalization Approaches for Ranking: A Review and Research Experiments , 2017, Int. J. Inf. Retr. Res..

[53]  Dietmar Jannach,et al.  Multistakeholder recommendation: Survey and research directions , 2020, User Modeling and User-Adapted Interaction.

[54]  Simon Buckingham Shum,et al.  Transitioning education’s knowledge infrastructure: Shaping design or shouting from the touchline? , 2018 .

[55]  Heinz Ulrich Hoppe,et al.  One-to-One Technology-Enhanced Learning: an Opportunity for Global Research Collaboration , 2006, Res. Pract. Technol. Enhanc. Learn..

[56]  Danah Boyd,et al.  Fairness and Abstraction in Sociotechnical Systems , 2019, FAT.

[57]  Ben Williamson,et al.  Decoding ClassDojo: psycho-policy, social-emotional learning and persuasive educational technologies , 2017 .

[58]  Tat-Seng Chua,et al.  Neural Collaborative Filtering , 2017, WWW.

[59]  James Caverlee,et al.  Fairness-Aware Tensor-Based Recommendation , 2018, CIKM.

[60]  Amelia Zafra,et al.  A Hybrid Multi-Criteria approach using a Genetic Algorithm for Recommending Courses to University Students , 2018, EDM.

[61]  Jane Golley,et al.  Inequality of opportunity in China's educational outcomes , 2016, China Economic Review.

[62]  Shayan Doroudi,et al.  Towards Accurate and Fair Prediction of College Success: Evaluating Different Sources of Student Data , 2020, EDM.

[63]  Dragan Gasevic,et al.  Learning analytics in higher education --- challenges and policies: a review of eight learning analytics policies , 2017, LAK.

[64]  Florence Martin,et al.  Award-winning faculty online teaching practices: Elements of award-winning courses , 2019 .

[65]  Shinichi Nakagawa,et al.  Missing inaction: the dangers of ignoring missing data. , 2008, Trends in ecology & evolution.

[66]  Niall Sclater,et al.  Code of practice for learning analytics , 2015 .

[67]  Dragan Gasevic,et al.  Complementing educational recommender systems with open learner models , 2020, LAK.

[68]  J. Reich,et al.  Democratizing education? Examining access and usage patterns in massive open online courses , 2015, Science.

[69]  Iván Cantador,et al.  Time-aware recommender systems: a comprehensive survey and analysis of existing evaluation protocols , 2013, User Modeling and User-Adapted Interaction.

[70]  Boi Faltings,et al.  Adaptive Sequential Recommendation for Discussion Forums on MOOCs using Context Trees , 2017, EDM.

[71]  Niall Sclater,et al.  Developing a Code of Practice for Learning Analytics , 2016, J. Learn. Anal..

[72]  Jade Goldstein-Stewart,et al.  The Use of MMR, Diversity-Based Reranking for Reordering Documents and Producing Summaries , 1998, SIGIR Forum.

[73]  Francisco Fernández-Navarro,et al.  Measuring teachers and learners’ perceptions of the quality of their online learning experience , 2016 .

[74]  Sho Fujihara,et al.  The absolute and relative values of education and the inequality of educational opportunity: Trends in access to education in postwar Japan , 2016 .

[75]  Sandra Buchholz,et al.  Secondary school differentiation and inequality of educational opportunity in Germany , 2016 .

[76]  Huzefa Rangwala,et al.  Towards Fair Educational Data Mining: A Case Study on Detecting At-risk Students , 2020, EDM.

[77]  Joanna C. Dunlap,et al.  Does Class Size Matter? An Exploration into Faculty Perceptions of Teaching High-Enrollment Online Courses , 2019, American Journal of Distance Education.

[78]  Soo-yong Byun,et al.  When Different Types of Education Matter , 2017 .

[79]  Vincent Aleven,et al.  Designing for Complementarity: Teacher and Student Needs for Orchestration Support in AI-Enhanced Classrooms , 2019, AIED.

[80]  John Shawe-Taylor,et al.  Towards Automatic, Scalable Quality Assurance in Open Education , 2019 .

[81]  Hassan Khosravi,et al.  Reciprocal peer recommendation for learning purposes , 2018, LAK.

[82]  Michael Eagle,et al.  Predicting Individualized Learner Models Across Tutor Lessons , 2018, EDM.

[83]  Soohyung Joo,et al.  Selection of information sources: Accessibility of and familiarity with sources, and types of tasks , 2009, ASIST.

[84]  Thomas W. Price,et al.  One minute is enough: Early Prediction of Student Success and Event-level Difficulty during Novice Programming Tasks , 2019, EDM.

[85]  George Karypis,et al.  Scholars Walk: A Markov Chain Framework for Course Recommendation , 2019, EDM.

[86]  Jimeng Sun,et al.  Hierarchical Reinforcement Learning for Course Recommendation in MOOCs , 2019, AAAI.

[87]  Yuchun Guo,et al.  Concept-Aware Deep Knowledge Tracing and Exercise Recommendation in an Online Learning System , 2019, EDM.

[88]  Masatoshi Yoshikawa,et al.  Course Content Analysis: An Initiative Step toward Learning Object Recommendation Systems for MOOC Learners , 2016, EDM.

[89]  Zachary A. Pardos,et al.  Combating the Filter Bubble: Designing for Serendipity in a University Course Recommendation System , 2019, ArXiv.

[90]  Neil T. Heffernan,et al.  Population validity for educational data mining models: A case study in affect detection , 2014, Br. J. Educ. Technol..

[91]  John Riedl,et al.  Item-based collaborative filtering recommendation algorithms , 2001, WWW '01.

[92]  Alan W. Black,et al.  Equity Beyond Bias in Language Technologies for Education , 2019, BEA@ACL.

[93]  K. Porayska-Pomsta,et al.  Accountability in Human and Artificial Intelligence Decision-Making as the Basis for Diversity and Educational Inclusion , 2019, Artificial Intelligence and Inclusive Education.

[94]  Krishna P. Gummadi,et al.  Fighting Fire with Fire: Using Antidote Data to Improve Polarization and Fairness of Recommender Systems , 2018, WSDM.

[95]  Pasquale Lops,et al.  Content-based Recommender Systems: State of the Art and Trends , 2011, Recommender Systems Handbook.