An Improved Hybrid Distributed Collaborative Filtering Model for Recommender Engine using Apache Spark

The present scenario there is a serious need of scalability for efficient analytics of big data. In order to achieve this, technology like MapReduce, Pig and HIVE came into action but when the question comes to scalability; Apache Spark maintains a great position far ahead. In this research paper, it has designed and developed an improved hybrid distributed collaborative model for filtering recommender engine. Execution time, scalability and robustness of the engine are the three evaluation parameters; has been considered for this present study. The present work keeps an eye on recommender system built with help of Apache Spark. Apart from this, it has been proposed and implemented the bisecting KMeans clustering algorithms. It has discussed about the comparative analysis between KMeans and Bisecting KMeans clustering algorithms on Apache Spark environment.

[1]  Jukka Riekki,et al.  Low latency analytics for streaming traffic data with Apache Spark , 2015, 2015 IEEE International Conference on Big Data (Big Data).

[2]  Laurence T. Yang,et al.  Cloud-Based Mobile Multimedia Recommendation System With User Behavior Information , 2014, IEEE Systems Journal.

[3]  Qing Yang,et al.  Fog Data: Enhancing Telehealth Big Data Through Fog Computing , 2015, ASE BD&SI.

[4]  Yang Guo,et al.  A survey of collaborative filtering based social recommender systems , 2014, Comput. Commun..

[5]  Dongliang Ding,et al.  An overview on cloud computing platform spark for Human Genome mining , 2016, 2016 IEEE International Conference on Mechatronics and Automation.

[6]  N. B. Anuar,et al.  The rise of "big data" on cloud computing: Review and open research issues , 2015, Inf. Syst..

[7]  Zhiyang Li,et al.  Scalable Collaborative Filtering Recommendation Algorithm with MapReduce , 2014, 2014 IEEE 12th International Conference on Dependable, Autonomic and Secure Computing.

[8]  Zhi-Dan Zhao,et al.  User-Based Collaborative-Filtering Recommendation Algorithms on Hadoop , 2010, 2010 Third International Conference on Knowledge Discovery and Data Mining.

[9]  Arun B. Samaddar,et al.  FogGIS: Fog Computing for geospatial big data analytics , 2016, 2016 IEEE Uttar Pradesh Section International Conference on Electrical, Computer and Electronics Engineering (UPCON).

[10]  J. Bobadilla,et al.  Recommender systems survey , 2013, Knowl. Based Syst..

[11]  Rabindra K. Barik,et al.  FAIR: A Hadoop-based Hybrid Model for Faculty Information Retrieval System , 2017, ArXiv.

[12]  Yujie Zhang,et al.  Spark: A Big Data Processing Platform Based on Memory Computing , 2015, 2015 Seventh International Symposium on Parallel Architectures, Algorithms and Programming (PAAP).

[13]  Kishor Sadafale,et al.  An online recommendation system for e-commerce based on apache mahout framework , 2013, SIGMIS-CPR '13.

[14]  Andreas Lommatzsch,et al.  Development of a News Recommender System based on Apache Flink , 2016, CLEF.

[15]  Lior Rokach,et al.  Introduction to Recommender Systems Handbook , 2011, Recommender Systems Handbook.

[16]  Bankim Patel,et al.  Big Data Analysis: Recommendation System with Hadoop Framework , 2015, 2015 IEEE International Conference on Computational Intelligence & Communication Technology.

[17]  F. Maxwell Harper,et al.  The MovieLens Datasets: History and Context , 2016, TIIS.

[18]  Zhou Zheng,et al.  The research of recommendation system based on Hadoop cloud platform , 2014, 2014 9th International Conference on Computer Science & Education.

[19]  Sasmita Panigrahi,et al.  A Hybrid Distributed Collaborative Filtering Recommender Engine Using Apache Spark , 2016, ANT/SEIT.

[20]  Andreas Lommatzsch,et al.  Real-time News Recommendations using Apache Spark , 2016, CLEF.