Biomarker Discovery Based on Large-Scale Feature Selection and MapReduce

Large-scale feature selection is one of the most important fields in the big data domain that can solve real data problems, such as bioinformatics, where it is necessary to process huge amount of data. The efficiency of existing feature selection algorithms significantly downgrades, if not totally inapplicable, when data size exceeds hundreds of gigabytes, because most feature selection algorithms are designed for centralized computing architecture. For that, distributed computing techniques, such as MapReduce can be applied to handle very large data. Our approach is to scale the existing method for feature selection, Kmeans clustering and Signal to Noise Ratio (SNR) combined with optimization technique as Binary Particle Swarm Optimization (BPSO). The proposed method is divided into two stages. In the first stage, we have used parallel Kmeans on MapReduce for clustering features, and then we have applied iterative MapReduce that implement parallel SNR ranking for each cluster. After, we have selected the top ranked feature from each cluster. The top scored features from each cluster are gathered and a new feature subset is generated. In the second stage, the new feature subset is used as input to the proposed BPSO based on MapReduce which provides an optimized feature subset. The proposed method is implemented in a distributed environment, and its efficiency is illustrated through analyzing practical problems such as biomarker discovery.

[1]  Shomona Gracia Jacob,et al.  Benchmarking Classification Models for Cancer Prediction from Gene Expression Data: A Novel Approach and New Findings , 2013 .

[2]  Hsiao-Hui Tsou,et al.  Putative tumor metastasis-associated genes in human gastric cancer. , 2012, International journal of oncology.

[3]  Xiaoning Peng,et al.  SSiCP: a new SVM based Recursive Feature Elimination Algorithm for Multiclass Cancer Classification , 2014, MUE 2014.

[4]  T. Ørntoft,et al.  Molecular Profiling of ADAM12 in Human Bladder Cancer , 2006, Clinical Cancer Research.

[5]  Hanlee P. Ji,et al.  Next-generation DNA sequencing , 2008, Nature Biotechnology.

[6]  Tom White,et al.  Hadoop: The Definitive Guide , 2009 .

[7]  Ming-Tsang Wu,et al.  Identification of Biomarkers for Esophageal Squamous Cell Carcinoma Using Feature Selection and Decision Tree Methods , 2013, TheScientificWorldJournal.

[8]  Nazrul M. Ahmad,et al.  Distributed visual enhancement on surveillance video with Hadoop Mapreduce and performance evaluation in pseudo distributed mode , 2014 .

[9]  Gaizhen Yang.  The Application of MapReduce in the Cloud Computing , 2011, 2011 2nd International Symposium on Intelligence Information Processing and Trusted Computing.

[10]  Hyun Cheol Chung,et al.  Novel biomarker candidates for gastric cancer. , 2008, Oncology reports.

[11]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[12]  Ahmad Zafarullah Abdul Jalil,et al.  Household risk management strategies in the semi-urban and rural area A literature review.pdf , 2014 .

[13]  John Langford,et al.  Scaling up machine learning: parallel and distributed approaches , 2011, KDD '11 Tutorials.

[14]  Jiang Yun,et al.  A New Gene Expression Profiles Classifying Approach Based on Neighborhood Rough Set and Probabilistic Neural Networks Ensemble , 2013, ICONIP.

[15]  Sean Owen,et al.  Mahout in Action , 2011 .

[16]  Ron Bekkerman,et al.  Scaling up Machine Learning , 2011 .

[17]  Jiawei Han,et al.  Selection of interdependent genes via dynamic relevance analysis for cancer diagnosis , 2013, J. Biomed. Informatics.

[18]  David C Whiteman,et al.  Reduced expression of IL‐18 is a marker of ultraviolet radiation‐induced melanomas , 2008, International journal of cancer.

[19]  Barnali Sahu,et al.  A Novel Feature Selection Algorithm using Particle Swarm Optimization for Cancer Microarray Data , 2012 .

[20]  Pedro Larrañaga,et al.  A review of feature selection techniques in bioinformatics , 2007, Bioinform..

[21]  Ahlem Kourid.  Iterative MapReduce for Feature Selection , 2014 .