Artificial Neural Networks for Educational Data Mining in Higher Education: A Systematic Literature Review

ABSTRACT Efforts to raise the bar of higher education so as to respond to dynamic societal/industry needs have led to a number of initiatives, including artificial neural network (ANN) based educational data mining (EDM) inclusive. With ANN-based EDM, humongous amount of student data in higher institutions could be harnessed for informed academic advisory that promotes adaptive learning for purposes of student retention, student progression, and cost saving. Mining students’ data optimally requires predictive data mining tool and machine learning technique like ANN. However, despite acknowledging the capability of ANN-based EDM for efficiently classifying students’ learning behavior and accurately predicting students’ performance, the concept has received less than commensurate attention in the literature. This seems to suggest that there are gaps and challenges confronting ANN-based EDM in higher education. In this study, we used the systematic literature review technique to gauge the pulse of researchers from the viewpoint of modeling, learning procedure, and cost function optimization using research studies. We aim to unearth the gaps and challenges with a view to offering research direction to upcoming researchers that want to make invaluable contributions to this relatively new field. We analyzed 190 studies conducted in 2010–2018. Our findings reveal that hardware challenges, training challenges, theoretical challenges, and quality concerns are the bane of ANN-based EDM in higher education and offer windows of opportunities for further research. We are optimistic that advances in research along this direction will make ANN-based EDM in higher education more visible and relevant in the quest for higher education-driven sustainable development.

[1]  Yegor Tkachenko,et al.  Autonomous CRM Control via CLV Approximation with Deep Reinforcement Learning in Discrete and Continuous Action Space , 2015, ArXiv.

[2]  Shrideep Pallickara,et al.  Predictive analytics using statistical, learning, and ensemble methods to support real-time exploration of discrete event simulations , 2016, Future Gener. Comput. Syst..

[3]  Stefano Manetti,et al.  A Multi-Valued Neuron Based Complex ELM Neural Network , 2017, Neural Processing Letters.

[4]  Andrew Y. Ng,et al.  Parsing Natural Scenes and Natural Language with Recursive Neural Networks , 2011, ICML.

[5]  K. Rajeswari,et al.  Attributes Selection for Predicting Students' Academic Performance using Education Data Mining and Artificial Neural Network , 2014 .

[6]  M. Kubát.  An Introduction to Machine Learning , 2017, Springer International Publishing.

[7]  Jason Yosinski,et al.  Deep neural networks are easily fooled: High confidence predictions for unrecognizable images , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[9]  Larry P. Heck,et al.  Learning deep structured semantic models for web search using clickthrough data , 2013, CIKM.

[10]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[11]  Razvan Pascanu,et al.  Advances in optimizing recurrent networks , 2012, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[12]  Mohammad S. Obaidat,et al.  Authorship verification using deep belief network systems , 2017, Int. J. Commun. Syst..

[13]  Andrew W. Senior,et al.  Long short-term memory recurrent neural network architectures for large scale acoustic modeling , 2014, INTERSPEECH.

[14]  Martin Höst,et al.  A systematic review of research on open source software in commercial software product development , 2011, Inf. Softw. Technol..

[15]  Anna Lea Dyckhoff,et al.  Action research and learning analytics in higher education , 2014 .

[16]  Daniel Graupe.  LAMSTAR-1 and LAMSTAR-2 Neural Networks , 2016 .

[17]  Gulshan Kumar,et al.  The Use of Artificial-Intelligence-Based Ensembles for Intrusion Detection: A Review , 2012, Appl. Comput. Intell. Soft Comput..

[18]  Daniel Graupe,et al.  Principles of Artificial Neural Networks - 3rd Edition , 2013, Advanced Series in Circuits and Systems.

[19]  Dennis Zielke,et al.  Design and Implementation of a Learning Analytics Toolkit for Teachers , 2012, J. Educ. Technol. Soc..

[20]  John F. Pane,et al.  Making Sense of Data-Driven Decision Making in Education , 2006 .

[21]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[22]  Weiqin Chen,et al.  Learning Analytics Interoperability - looking for Low-Hanging Fruits , 2014 .

[23]  Jürgen Schmidhuber,et al.  Multi-column deep neural networks for image classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Elena,et al.  USING ARTIFICIAL NEURAL NETWORKS IN E-LEARNING SYSTEMS , 2010 .

[25]  Olawande Daramola,et al.  Design of a learning analytics system for academic advising in Nigerian universities , 2017, 2017 International Conference on Computing Networking and Informatics (ICCNI).

[26]  Davide Anguita,et al.  Advances in learning analytics and educational data mining , 2015, ESANN.

[27]  Jimeng Sun,et al.  Using recurrent neural network models for early detection of heart failure onset , 2016, J. Am. Medical Informatics Assoc..

[28]  Thomas J. Shuell,et al.  What we know about how children learn. , 1984 .

[29]  Pat Langley,et al.  The changing science of machine learning , 2011, Machine Learning.

[30]  Mansureh Kebritchi,et al.  Learning Analytics Methods, Benefits, and Challenges in Higher Education: A Systematic Literature Review. , 2016 .

[31]  Stefan T. Mol,et al.  Labour Market Driven Learning Analytics , 2014 .

[32]  Miguel-Ángel Sicilia,et al.  A framework for learning analytics in moodle for assessing course outcomes , 2016, 2016 IEEE Global Engineering Education Conference (EDUCON).

[33]  Rebecca Ferguson,et al.  Learning analytics: drivers, developments and challenges , 2012 .

[34]  Geoffrey Zweig,et al.  Using Recurrent Neural Networks for Slot Filling in Spoken Language Understanding , 2015, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[35]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[36]  Katrina Sin,et al.  Application of Big Data in Education Data Mining and Learning Analytics-A Literature Review , 2015, SOCO 2015.

[37]  P. Prinsloo,et al.  Learning Analytics , 2013 .

[38]  Y. Liu,et al.  Bilinear deep learning for image classification , 2011, ACM Multimedia.

[39]  Kenan Zengin,et al.  A sample study on applying data mining research techniques in educational science: Developing a more meaning of data , 2011 .

[40]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[41]  Dimitrios Zissis,et al.  A cloud based architecture capable of perceiving and predicting multiple vessel behaviour , 2015, Appl. Soft Comput..

[42]  Weiqin Chen,et al.  Privacy-driven Design of Learning Analytics Applications - Exploring the Design Space of Solutions for Data Sharing and Interoperability , 2016, J. Learn. Anal..

[43]  Vladimir Vapnik,et al.  Chervonenkis: On the uniform convergence of relative frequencies of events to their probabilities , 1971 .

[44]  Rui Guo,et al.  Participation-based student final performance prediction model through interpretable Genetic Programming: Integrating learning analytics, educational data mining and theory , 2015, Comput. Hum. Behav..

[45]  Pejman Tahmasebi,et al.  A hybrid neural networks-fuzzy logic-genetic algorithm for grade estimation , 2012, Comput. Geosci..

[46]  Bernard Widrow,et al.  The No-Prop algorithm: A new learning algorithm for multilayer neural networks , 2013, Neural Networks.

[47]  Heiga Zen,et al.  Unidirectional long short-term memory recurrent neural network with recurrent output layer for low-latency speech synthesis , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[48]  Joshua B. Tenenbaum,et al.  Learning with Hierarchical-Deep Models , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Sepp Hochreiter,et al.  Toxicity Prediction using Deep Learning , 2015, ArXiv.

[50]  Luca Maria Gambardella,et al.  Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks , 2013, MICCAI.

[51]  Pierre Baldi,et al.  Deep autoencoder neural networks for gene ontology annotation predictions , 2014, BCB.

[52]  Sergio Gomez Colmenarejo,et al.  Hybrid computing using a neural network with dynamic external memory , 2016, Nature.

[53]  Jürgen Schmidhuber,et al.  Multi-column deep neural network for traffic sign classification , 2012, Neural Networks.

[54]  Elvira Popescu,et al.  Using Artificial Neural Networks to Identify Learning Styles , 2015, AIED.

[55]  Dong Yu,et al.  Scalable stacking and learning for building deep architectures , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[56]  D. Graupe,et al.  Automated prediction of apnea and hypopnea, using a LAMSTAR artificial neural network. , 2010, American journal of respiratory and critical care medicine.

[57]  Xiaodong He,et al.  A Multi-View Deep Learning Approach for Cross Domain User Modeling in Recommendation Systems , 2015, WWW.

[58]  Honglak Lee,et al.  An Analysis of Single-Layer Networks in Unsupervised Feature Learning , 2011, AISTATS.

[59]  Jürgen Schmidhuber,et al.  LSTM recurrent networks learn simple context-free and context-sensitive languages , 2001, IEEE Trans. Neural Networks.

[60]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[61]  Daniel Graupe.  DEEP LEARNING NEURAL NETWORKS: DESIGN AND CASE STUDIES , 2016 .

[62]  Dong Yu,et al.  Deep Convex Net: A Scalable Architecture for Speech Pattern Classification , 2011, INTERSPEECH.

[63]  Kai Petersen,et al.  Guidelines for conducting systematic mapping studies in software engineering: An update , 2015, Inf. Softw. Technol..

[64]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[65]  Tara N. Sainath,et al.  Deep convolutional neural networks for LVCSR , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[66]  Chris Edwards,et al.  Growing pains for deep learning , 2015, Commun. ACM.

[67]  Marcus Hutter,et al.  One Decade of Universal Artificial Intelligence , 2012, ArXiv.

[68]  Tara N. Sainath,et al.  Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups , 2012, IEEE Signal Processing Magazine.

[69]  Sayan Mukherjee,et al.  Learning theory: stability is sufficient for generalization and necessary and sufficient for consistency of empirical risk minimization , 2006, Adv. Comput. Math..

[70]  Renzo Sprugnoli,et al.  Data mining models for student careers , 2015, Expert Syst. Appl..

[71]  Manpreet Singh,et al.  Classification and Prediction Based Data Mining Algorithms to Predict Slow Learners in Education Sector , 2015 .

[72]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[73]  Shane Dawson,et al.  Numbers Are Not Enough. Why e-Learning Analytics Failed to Inform an Institutional Strategic Plan , 2012, J. Educ. Technol. Soc..

[74]  Dragan Gasevic,et al.  Open Learning Analytics: an integrated modularized platform , 2011 .

[75]  Geoffrey E. Hinton,et al.  A Better Way to Pretrain Deep Boltzmann Machines , 2012, NIPS.

[76]  Dong Yu,et al.  Tensor Deep Stacking Networks , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[77]  Simon Buckingham Shum,et al.  Learning dispositions and transferable competencies: pedagogy, modelling and learning analytics , 2012, International Conference on Learning Analytics and Knowledge.

[78]  Anna Siri,et al.  Predicting students’ dropout at university using Artificial Neural Networks , 2015 .

[79]  Jun Miao,et al.  Constrained Extreme Learning Machine: A novel highly discriminative random feedforward neural network , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[80]  Yoshua Bengio,et al.  Unsupervised Models of Images by Spikeand-Slab RBMs , 2011, ICML.

[81]  Yoshua Bengio,et al.  A Neural Probabilistic Language Model , 2003, J. Mach. Learn. Res..

[82]  Lukás Burget,et al.  Recurrent neural network based language model , 2010, INTERSPEECH.

[83]  Tore Hoel,et al.  Privacy in Learning Analytics – Implications for System Architecture , 2015 .

[84]  Hendrik Drachsler,et al.  Privacy and Analytics – it’s a DELICATE issue. A Checklist to establish trusted Learning Analytics , 2016 .

[85]  Jianfeng Gao,et al.  Learning Continuous Phrase Representations for Translation Modeling , 2014, ACL.

[86]  TahmasebiPejman,et al.  A hybrid neural networks-fuzzy logic-genetic algorithm for grade estimation , 2012 .

[87]  Li Deng,et al.  A deep convolutional neural network using heterogeneous pooling for trading acoustic invariance with phonetic confusion , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[88]  Dong Yu,et al.  Automatic Speech Recognition: A Deep Learning Approach , 2014 .

[89]  Luca Maria Gambardella,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence Flexible, High Performance Convolutional Neural Networks for Image Classification , 2022 .

[90]  Xiangang Li,et al.  Constructing long short-term memory based deep recurrent neural networks for large vocabulary speech recognition , 2014, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[91]  George Siemens,et al.  Ethical and privacy principles for learning analytics , 2014, Br. J. Educ. Technol..

[92]  Wahidah Husain,et al.  A Review on Predicting Student's Performance Using Data Mining Techniques , 2015 .

[93]  Maren Scheffel,et al.  Quality Indicators for Learning Analytics , 2014, J. Educ. Technol. Soc..

[94]  Nurettin Yorek,et al.  A CFBPN Artificial Neural Network Model for Educational Qualitative Data Analyses: Example of Students' Attitudes Based on Kellerts' Typologies. , 2015 .

[95]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[96]  John P. Campbell,et al.  Analytics in Higher Education: Establishing a Common Language , 2012 .

[97]  Sérgio André Ferreira,et al.  Academic Analytics: Mapping the Genome of the University , 2014, IEEE Revista Iberoamericana de Tecnologias del Aprendizaje.

[98]  Marek Hatala,et al.  A qualitative evaluation of evolution of a learning analytics tool , 2012, Comput. Educ..

[99]  Geoffrey E. Hinton,et al.  Acoustic Modeling Using Deep Belief Networks , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[100]  Geoffrey Zweig,et al.  Recent advances in deep learning for speech research at Microsoft , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[101]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[102]  Václav Snásel,et al.  Metaheuristic design of feedforward neural networks: A review of two decades of research , 2017, Eng. Appl. Artif. Intell..

[103]  Yoshua Bengio,et al.  Learning deep physiological models of affect , 2013, IEEE Computational Intelligence Magazine.

[104]  Vive Kumar,et al.  Learning Analytics Solution for Reducing Learners' Course Failure Rate , 2015, 2015 IEEE Seventh International Conference on Technology for Education (T4E).

[105]  Sandeep Rajani,et al.  ARTIFICIAL INTELLIGENCE – MAN OR MACHINE , 2010 .

[106]  Shaojie Qu,et al.  Predicting Achievement of Students in Smart Campus , 2018, IEEE Access.

[107]  Pierre-Yves Oudeyer,et al.  On the Impact of Robotics in Behavioral and Cognitive Sciences: From Insect Navigation to Human Cognitive Development , 2010, IEEE Transactions on Autonomous Mental Development.

[108]  Janet E. Hurn,et al.  Using learning analytics to predict (and improve) student success: a faculty perspective , 2013 .

[109]  Stamos T. Karamouzis,et al.  An Artificial Neural Network for Predicting Student Graduation Outcomes , 2008 .

[110]  Christopher Potts,et al.  Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank , 2013, EMNLP.

[111]  Dong Yu,et al.  Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[112]  Jun Miao,et al.  Hierarchical Extreme Learning Machine for unsupervised representation learning , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[113]  Hélène Fournier,et al.  The value of learning analytics to networked learning on a personal learning environment , 2011, LAK.

[114]  Gerald Penn,et al.  Convolutional Neural Networks for Speech Recognition , 2014, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[115]  Haiping Lu,et al.  A survey of multilinear subspace learning for tensor data , 2011, Pattern Recognit..

[116]  Dong Yu,et al.  Deep Learning: Methods and Applications , 2014, Found. Trends Signal Process..

[117]  Dumitru Erhan,et al.  Deep Neural Networks for Object Detection , 2013, NIPS.

[118]  Yoshua Bengio,et al.  Practical Recommendations for Gradient-Based Training of Deep Architectures , 2012, Neural Networks: Tricks of the Trade.

[119]  Dong Yu,et al.  Roles of Pre-Training and Fine-Tuning in Context-Dependent DBN-HMMs for Real-World Speech Recognition , 2010 .

[120]  R. Burbaite,et al.  Educational robots as collaborative learning objects for teaching Computer Science , 2013, 2013 International Conference on System Science and Engineering (ICSSE).

[121]  Izhar Wallach,et al.  AtomNet: A Deep Convolutional Neural Network for Bioactivity Prediction in Structure-based Drug Discovery , 2015, ArXiv.

[122]  Yoshua Bengio,et al.  A Spike and Slab Restricted Boltzmann Machine , 2011, AISTATS.

[123]  Yelong Shen,et al.  A Latent Semantic Model with Convolutional-Pooling Structure for Information Retrieval , 2014, CIKM.

[124]  Phil Blunsom,et al.  Recurrent Continuous Translation Models , 2013, EMNLP.

[125]  Hendrik Drachsler,et al.  Privacy and analytics: it's a DELICATE issue a checklist for trusted learning analytics , 2016, LAK.

[126]  Ameet Talwalkar,et al.  Foundations of Machine Learning , 2012, Adaptive computation and machine learning.

[127]  Robertas Damasevicius,et al.  Teaching of Computer Science Topics Using Meta-Programming-Based GLOs and LEGO Robots , 2013, Informatics Educ..

[128]  M. Weinstein.  When Numbers are not Enough , 2016 .

[129]  Geoffrey E. Hinton.  A Practical Guide to Training Restricted Boltzmann Machines , 2012, Neural Networks: Tricks of the Trade.

[130]  M. Bolic,et al.  Comparison of Feed-Forward Neural Network training algorithms for oscillometric blood pressure estimation , 2010, 4th International Workshop on Soft Computing Applications.

[131]  Lei Xie,et al.  Photo-real talking head with deep bidirectional LSTM , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[132]  Ulrik Schroeder,et al.  Learning Analytics: Challenges and Future Research Directions , 2014 .

[133]  Niall Sclater,et al.  Code of practice for learning analytics , 2015 .

[134]  A. Cooper.  Interoperability-a survey of current literature and candidate standards , 2013 .

[135]  Bo Liu,et al.  Multiple Evaluation Models for Education Based on Artificial Neural Networks , 2015 .

[136]  R. Suchithra,et al.  Survey of Learning Analytics based on Purpose and Techniques for Improving Student Performance , 2015 .

[137]  Dragan Gasevic,et al.  Using institutional data to predict student course selections in higher education , 2016, Internet High. Educ..

[138]  Elif Bahadir,et al.  Using Neural Network and Logistic Regression Analysis to Predict Prospective Mathematics Teachers' Academic Success upon Entering Graduate Education , 2016 .

[139]  Maria-Iuliana Dascalu,et al.  A Survey on Social Learning Analytics: Applications, Challenges and Importance , 2016 .

[140]  Maren Scheffel,et al.  Developing an evaluation framework of quality indicators for learning analytics , 2015, LAK.

[141]  Vivienne Sze,et al.  Efficient Processing of Deep Neural Networks: A Tutorial and Survey , 2017, Proceedings of the IEEE.

[142]  Brian Kingsbury,et al.  New types of deep neural network learning for speech recognition and related applications: an overview , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[143]  Marek Hatala,et al.  Factors influencing beliefs for adoption of a learning analytics tool: An empirical study , 2013, Comput. Educ..

[144]  Anastasios A. Economides,et al.  Learning Analytics and Educational Data Mining in Practice: A Systematic Literature Review of Empirical Evidence , 2014, J. Educ. Technol. Soc..

[145]  Jaume Bacardit,et al.  Functional Network Construction in Arabidopsis Using Rule-Based Machine Learning on Large-Scale Data Sets[C][W][OA] , 2011, Plant Cell.

[146]  Pablo Garaizar,et al.  From Analysis to Improvement: Challenges and Opportunities for Learning Analytics , 2016, IEEE Revista Iberoamericana de Tecnologias del Aprendizaje.

[147]  Bogdan Oancea,et al.  Predicting students’ results in higher education using a neural network , 2013 .

[148]  Robertas Damasevicius,et al.  Towards the development of genuine intelligent ontology-based e-Learning systems , 2010, 2010 5th IEEE International Conference Intelligent Systems.

[149]  Robertas Damasevicius,et al.  Analysis of Academic Results for Informatics Course Improvement Using Association Rule Mining , 2008, ISD.

[150]  Alison Gopnik,et al.  Making AI More Human. , 2017, Scientific American.

[151]  Robertas Damasevicius,et al.  Educational Robots for Internet-of-Things Supported Collaborative Learning , 2014, ICIST.

[152]  Kanya Tanaka,et al.  Experimental Study on Learning of Neural Network Using Particle Swarm Optimization in Predictive Fuzzy for Pneumatic Servo System , 2019, Cognitive Internet of Things.

[153]  Gökhan Tür,et al.  Use of kernel deep convex networks and end-to-end learning for spoken language understanding , 2012, 2012 IEEE Spoken Language Technology Workshop (SLT).

[154]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[155]  Stuart Palmer,et al.  Modelling Engineering Student Academic Performance Using Academic Analytics , 2013 .

[156]  Chia-Hua Ho,et al.  Recent Advances of Large-Scale Linear Classification , 2012, Proceedings of the IEEE.

[157]  Luca Maria Gambardella,et al.  Deep, Big, Simple Neural Nets for Handwritten Digit Recognition , 2010, Neural Computation.

[158]  Wu Zhang,et al.  Prediction of Confusion Attempting Algebra Homework in an Intelligent Tutoring System through Machine Learning Techniques for Educational Sustainable Development , 2018, Sustainability.

[159]  Ladislav Dušek,et al.  Curriculum Mapping with Academic Analytics in Medical and Healthcare Education , 2015, PloS one.