Machine Learning Approach for Bottom 40 Percent Households (B40) Poverty Classification

Malaysia citizens are categorised into three different income groups which are the Top 20 Percent (T20), Middle 40 Percent (M40), and Bottom 40 Percent (B40). One of the focus areas in the Eleventh Malaysia Plan (11MP) is to elevate the B40 household group towards the middle-income society. Based on recent studies by the World Bank, Malaysia is expected to enter the high-income economy status no later than the year 2024. Thus, it is essential to clarify the B40 population through a predictive classification as a prerequisite towards developing a comprehensive action plan by the government. This paper is aimed at identifying the best machine learning models using Naive Bayes, Decision Tree and k-Nearest Neighbors algorithm for classifying the B40 population. Several data pre-processing task such as data cleaning, feature engineering, normalisation, feature selection: Correlation Attribute, Information Gain Attribute and Symmetrical Uncertainty Attribute and sampling methods using SMOTE has been conducted to the raw dataset to ensure the quality of the training data. Each classifier is then optimized using different tuning parameter with 10-Fold Cross Validation for achieving the optimal values before the performance of the three classifiers are compared to each other. For the experiments, a dataset from National Poverty Data Bank called eKasih obtained from the Society Wellbeing Department, Implementation Coordination Unit of Prime Minister's Department (ICU JPM), consisting of 99,546 households from 3 different states: Johor, Terengganu and Pahang are used to train each of the machine learning model. The experimental results using 10-Fold Cross-Validation method demonstrates that the overall performance of Decision Tree model outperformed the other models and the significance test specified the result is statistically significance.

[1]  Sri Redjeki,et al.  Naive Bayes Classifier Algorithm Approach for Mapping Poor Families Potential , 2015 .

[2]  Daniel Berend,et al.  A finite sample analysis of the Naive Bayes classifier , 2015, J. Mach. Learn. Res..

[3]  Abdul Hadi,et al.  Enhancing Malaysia Rainfall Prediction Using Classification Techniques , 2017 .

[4]  Zadanie Záverečnej Práce,et al.  POVERTY ANALYSIS USING MACHINE LEARNING METHODS , 2016 .

[5]  R. Terano,et al.  Effectiveness of microcredit program and determinants of income among small business entrepreneurs in Malaysia , 2015, Journal of Global Entrepreneurship Research.

[6]  Francisco Herrera,et al.  SMOTE for Learning from Imbalanced Data: Progress and Challenges, Marking the 15-year Anniversary , 2018, J. Artif. Intell. Res..

[7]  Hairi Zamzuri,et al.  Pedestrian detection using triple laser range finders , 2017 .

[8]  Preksha Pareek,et al.  Classifying the population as BPL or non-BPL using Multilayer Neural Network , 2012 .

[9]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[10]  Nazri Mohd Nawi,et al.  The Effect of Pre-Processing Techniques and Optimal Parameters selection on Back Propagation Neural Networks , 2017 .

[11]  Hongyuan Cao,et al.  Assessing agreement with multiple raters on correlated kappa statistics , 2016, Biometrical journal. Biometrische Zeitschrift.

[12]  Mohd Zakree Ahmad Nazri,et al.  On machine learning technique selection for classification , 2015, 2015 International Conference on Electrical Engineering and Informatics (ICEEI).

[13]  Kamal Kant Verma,et al.  Comparative Study of Various Decision Tree Classification Algorithm Using WEKA , 2015 .

[14]  Ferat Sahin,et al.  A survey on feature selection methods , 2014, Comput. Electr. Eng..

[15]  Azuraliza Abu Bakar,et al.  Comparative Analysis of Data Mining Techniques for Malaysian Rainfall Prediction , 2016 .

[16]  Peter Willett,et al.  Calculation of Substructural Analysis Weights Using a Genetic Algorithm , 2015, J. Chem. Inf. Model..

[17]  Nazlia Omar,et al.  Feature Selection for Multi-label Document Based on Wrapper Approach through Class Association Rules , 2017 .

[18]  P. Willett,et al.  Ligand-based virtual screening using a genetic algorithm with data fusion , 2018 .

[19]  Stefan Wager,et al.  Estimation and Inference of Heterogeneous Treatment Effects using Random Forests , 2015, Journal of the American Statistical Association.

[20]  Salwani Abdullah,et al.  Hybrid feature selection algorithm using symmetrical uncertainty and a harmony search algorithm , 2016, Int. J. Syst. Sci..

[21]  Martin Kappas,et al.  Comparison of Random Forest, k-Nearest Neighbor, and Support Vector Machine Classifiers for Land Cover Classification Using Sentinel-2 Imagery , 2017, Sensors.

[22]  K. K. Sahu,et al.  Normalization: A Preprocessing Stage , 2015, ArXiv.

[23]  อนิรุธ สืบสิงห์,et al.  Data Mining Practical Machine Learning Tools and Techniques , 2014 .

[24]  N. Sani The Use of Data Fusion on Multiple Substructural Analysis Based GA Runs , 2017 .

[25]  Ying LU,et al.  Decision tree methods: applications for classification and prediction , 2015, Shanghai archives of psychiatry.

[26]  Ruben Thoplan Random Forests for Poverty Classification , 2014 .

[27]  Siti Mariyam Shamsuddin,et al.  Classification with class imbalance problem: A review , 2015, SOCO 2015.

[28]  Chih-Fong Tsai,et al.  The distance function effect on k-nearest neighbor classification for medical datasets , 2016, SpringerPlus.

[29]  F. Johari,et al.  The Effectiveness of Zakat in Reducing Poverty Incident: An Analysis in Kelantan, Malaysia , 2015 .

[30]  C. Siwar,et al.  Poverty Mapping and Characterizing the Poor Using Geographical Information System: Case Study in Terengganu, Malaysia , 2014 .

[31]  Zalmiyah Zakaria,et al.  A Review on Classification of the Urban Poverty Using the Artificial Intelligence Method , 2017 .