Comparison Between Stochastic Gradient Descent and VLE Metaheuristic for Optimizing Matrix Factorization

Matrix factorization is used by recommender systems in collaborative filtering for building prediction models based on a couple of matrices. These models are usually generated by stochastic gradient descent algorithm, which learns the model minimizing the error done. Finally, the obtained models are validated according to an error criterion by predicting test data. Since the model generation can be tackled as an optimization problem where there is a huge set of possible solutions, we propose to use metaheuristics as alternative solving methods for matrix factorization. In this work we applied a novel metaheuristic for continuous optimization, which works inspired by the vapour-liquid equilibrium. We considered a particular case were matrix factorization was applied: the prediction student performance problem. The obtained results surpassed thoroughly the accuracy provided by stochastic gradient descent.

[1]  Dervis Karaboga,et al.  Artificial bee colony algorithm , 2010, Scholarpedia.

[2]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[3]  Gerhard Friedrich,et al.  Recommender Systems - An Introduction , 2010 .

[4]  Enrique Cortés-Toro,et al.  A New Metaheuristic Inspired by the Vapour-Liquid Equilibrium for Continuous Optimization , 2018, Applied Sciences.

[5]  Lars Schmidt-Thieme,et al.  Online-updating regularized kernel matrix factorization models for large-scale recommender systems , 2008, RecSys '08.

[6]  Celso C. Ribeiro,et al.  Greedy Randomized Adaptive Search Procedures , 2003, Handbook of Metaheuristics.

[7]  Mirjana Cangalovic,et al.  General variable neighborhood search for the continuous optimization , 2006, Eur. J. Oper. Res..

[8]  Richard F. Hartl,et al.  Simulation-based optimization methods for setting production planning parameters , 2014 .

[9]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[10]  Hong Sun,et al.  Smolign: A Spatial Motifs-Based Protein Multiple Structural Alignment Method , 2012, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[11]  Léon Bottou,et al.  Large-Scale Machine Learning with Stochastic Gradient Descent , 2010, COMPSTAT.

[12]  André Rossi,et al.  Two Iterative Metaheuristic Approaches to Dynamic Memory Allocation for Embedded Systems , 2011, EvoCOP.

[13]  Michel Gendreau,et al.  Handbook of Metaheuristics , 2010 .

[14]  Robin Smith,et al.  Chemical Process: Design and Integration , 2005 .

[15]  El-Ghazali Talbi,et al.  Metaheuristics - From Design to Implementation , 2009 .

[16]  Sachin Ahuja,et al.  Machine learning and its applications: A review , 2017, 2017 International Conference on Big Data Analytics and Computational Intelligence (ICBDAC).

[17]  Hossein Nezamabadi-pour,et al.  GSA: A Gravitational Search Algorithm , 2009, Inf. Sci..

[18]  Marco Dorigo,et al.  Ant system: optimization by a colony of cooperating agents , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[19]  Jonathan M. Garibaldi,et al.  Parameter Estimation Using Metaheuristics in Systems Biology: A Comprehensive Review , 2012, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[20]  Ying Tan,et al.  FWA Application on Non-negative Matrix Factorization , 2015 .

[21]  Fred Glover,et al.  Tabu Search - Part II , 1989, INFORMS J. Comput..

[22]  Rajesh Kumar,et al.  A review on particle swarm optimization algorithms and their applications to data clustering , 2011, Artificial Intelligence Review.

[23]  Kenneth O. Stanley,et al.  Simple Evolutionary Optimization Can Rival Stochastic Gradient Descent in Neural Networks , 2016, GECCO.

[24]  Zong Woo Geem,et al.  Overview of Harmony Search algorithm and its applications in Civil Engineering , 2014, Evol. Intell..

[25]  Lars Schmidt-Thieme,et al.  Factorization Techniques for Predicting Student Performance , 2012 .

[26]  Lior Rokach,et al.  Recommender Systems Handbook , 2010 .

[27]  John H. Holland,et al.  Genetic Algorithms and Adaptation , 1984 .

[28]  Reza Boostani,et al.  Using Genetic algorithm to enhance nonnegative matrix factorization initialization , 2010, 2010 6th Iranian Conference on Machine Vision and Image Processing.

[29]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.