Breast Cancer Identification from Patients' Tweet Streaming Using Machine Learning Solution on Spark

Twitter integrates with streaming data technologies and machine learning to add new value to healthcare. /is paper presented a real-time system to predict breast cancer based on streaming patient’s health data from Twitter. /e proposed system consists of two major components: developing an offline building model and an online prediction pipeline. For the first component, we made a correlation between the features to determine the correlation between features and reduce the number of features from the Breast Cancer Wisconsin Diagnostic dataset. Two feature selection algorithms are recursive feature elimination and univariate feature selection algorithms which are applied to features after correlation to select the essential features. Four decision trees, logistic regression, support vector machine, and random forest classifier have been used on features after correlation and feature selection. Also, hyperparameter tuning and cross-validation have been applied with machine learning to optimize models and enhance accuracy. Apache Spark, Apache Kafka, and Twitter Streaming API are used to develop the second component. /e best model with the highest accuracy obtained from the first component predicts breast cancer in real time from tweets’ streaming./e results showed that the best model is the random forest classifier which achieved the best accuracy.

[1]  Abien Fred Agarap.  On breast cancer detection: an application of machine learning algorithms on the wisconsin diagnostic dataset , 2017, ICMLSC '18.

[2]  Abbas Toloie Eshlaghy,et al.  Using Three Machine Learning Techniques for Predicting Breast Cancer Recurrence , 2013 .

[3]  T. Sridevi,et al.  A Novel Feature Selection Method for Effective Breast Cancer Diagnosis and Prognosis , 2014 .

[4]  David A. Landgrebe,et al.  A survey of decision tree classifier methodology , 1991, IEEE Trans. Syst. Man Cybern..

[5]  Hastari Utama.  Sentiment Analysis in Airline Tweets Using Mutual Information for Feature Selection , 2019, 2019 4th International Conference on Information Technology, Information Systems and Electrical Engineering (ICITISEE).

[6]  Aixia Guo,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2014 .

[7]  Bo Gao,et al.  A novel intelligent classification model for breast cancer diagnosis , 2019, Inf. Process. Manag..

[8]  P. Warner.  Ordinal logistic regression , 2008, Journal of Family Planning and Reproductive Health Care.

[9]  B. McAree,et al.  Breast cancer in women under 40 years of age: a series of 57 cases from Northern Ireland. , 2010, Breast.

[10]  Lekha R. Nair,et al.  Applying spark based machine learning model on streaming big data for health status prediction , 2017, Comput. Electr. Eng..

[11]  Ole Winther,et al.  Rare disease diagnosis: A review of web search, social media and large-scale data-mining approaches , 2015, Rare diseases.

[12]  Xin Jin,et al.  Machine Learning Techniques and Chi-Square Feature Selection for Cancer Classification Using SAGE Gene Expression Profiles , 2006, BioDM.

[13]  Marcos Dias de Assunção,et al.  Apache Spark , 2019, Encyclopedia of Big Data Technologies.

[14]  Aouatif Amine,et al.  Comparative Study of Machine Learning Algorithms Using the Breast Cancer Dataset , 2019 .

[15]  Muhammet Fatih Ak,et al.  A Comparative Analysis of Breast Cancer Detection and Diagnosis Using Data Visualization and Machine Learning Applications , 2020, Healthcare.

[16]  Sang Won Yoon,et al.  Breast cancer diagnosis based on feature extraction using a hybrid of K-means and support vector machine algorithms , 2014, Expert Syst. Appl..

[17]  Damminda Alahakoon,et al.  Machine learning to support social media empowered patients in cancer care and cancer treatment decisions , 2018, PloS one.

[18]  Matthias Sax,et al.  Apache Kafka , 2019, Encyclopedia of Big Data Technologies.

[19]  Christopher M. Danforth,et al.  A Sentiment Analysis of Breast Cancer Treatment Experiences and Healthcare Perceptions Across Twitter , 2018, ArXiv.

[20]  Dayou Liu,et al.  A support vector machine classifier with rough set-based feature selection for breast cancer diagnosis , 2011, Expert Syst. Appl..

[21]  Jochen L. Leidner,et al.  Quantifying Self-Reported Adverse Drug Events on Twitter: Signal and Topic Analysis , 2016, SMSociety.

[22]  Heyam H. Al-Baity,et al.  On the Scalability of Machine-Learning Algorithms for Breast Cancer Prediction in Big Data Context , 2019, IEEE Access.

[23]  H. Iwase,et al.  [Breast cancer]. , 2006, Nihon rinsho. Japanese journal of clinical medicine.

[24]  M. D. Shehu,et al.  Using Five Machine Learning for Breast Cancer Biopsy Predictions Based on Mammographic Diagnosis , 2017 .

[25]  Dursun Delen,et al.  Predicting breast cancer survivability: a comparison of three data mining methods , 2005, Artif. Intell. Medicine.

[26]  Sonal Jain,et al.  Analysis of k-means clustering approach on the breast cancer Wisconsin dataset , 2016, International Journal of Computer Assisted Radiology and Surgery.

[27]  Laura Rodríguez Larumbe.  Special requirements for QA in mammography with respect to CR-Systems , 2013 .

[28]  Hajar Mousannif,et al.  Using Machine Learning Algorithms for Breast Cancer Risk Prediction and Diagnosis , 2016, ANT/SEIT.

[29]  Adel Aloraini.  Different Machine Learning Algorithms for Breast Cancer Diagnosis , 2012 .

[30]  Elia Gabarron,et al.  Diabetes on Twitter: A Sentiment Analysis , 2018, Journal of diabetes science and technology.

[31]  C. Dolea,et al.  World Health Organization , 1949, International Organization.

[32]  William M. Campbell,et al.  Support vector machines for speaker verification and identification , 2000, Neural Networks for Signal Processing X. Proceedings of the 2000 IEEE Signal Processing Society Workshop (Cat. No.00TH8501).

[33]  Mehmet Fatih Akay,et al.  Support vector machines combined with feature selection for breast cancer diagnosis , 2009, Expert Syst. Appl..

[34]  Keqin Li,et al.  A task-level adaptive MapReduce framework for real-time streaming data in healthcare applications , 2015, Future Gener. Comput. Syst..

[35]  Adrian E. Raftery,et al.  Bayesian model averaging: development of an improved multi-class, gene selection and classification tool for microarray data , 2005, Bioinform..

[36]  Jae Won Lee,et al.  An extensive comparison of recent classification tools applied to microarray data , 2004, Comput. Stat. Data Anal..

[37]  Abdeltawab M. Hendawi,et al.  Heart disease identification from patients' social posts, machine learning solution on Spark , 2020, Future Gener. Comput. Syst..

[38]  Bor-Wen Cheng,et al.  Diagnosing Breast Masses in Digital Mammography Using Feature Selection and Ensemble Methods , 2012, Journal of Medical Systems.