Explainability of a Machine Learning Granting Scoring Model in Peer-to-Peer Lending

Peer-to-peer (P2P) lending demands effective and explainable credit risk models. Typical machine learning algorithms offer high prediction performance, but most of them lack explanatory power. However, this deficiency can be solved with the help of the explainability tools proposed in the last few years, such as the SHAP values. In this work, we assess the well-known logistic regression model and several machine learning algorithms for granting scoring in P2P lending. The comparison reveals that the machine learning alternative is superior in terms of not only classification performance but also explainability. More precisely, the SHAP values reveal that machine learning algorithms can reflect dispersion, nonlinearity and structural breaks in the relationships between each feature and the target variable. Our results demonstrate that is possible to have machine learning credit scoring models be both accurate and transparent. Such models provide the trust that the industry, regulators and end-users demand in P2P lending and may lead to a wider adoption of machine learning in this and other risk assessment applications where explainability is required.

[1]  Yuehui Chen,et al.  Credit Risk Assessment Based on Long Short-Term Memory Model , 2017, ICIC.

[2]  Yufei Xia,et al.  A Novel Reject Inference Model Using Outlier Detection and Gradient Boosting Technique in Peer-to-Peer Lending , 2019, IEEE Access.

[3]  Shuiqing Yang,et al.  The relationship between soft information in loan titles and online peer-to-peer lending: evidence from RenRenDai platform , 2018, Electronic Commerce Research.

[4]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[5]  Carlos Serrano-Cinca,et al.  The use of profit scoring as an alternative to credit scoring systems in peer-to-peer (P2P) lending , 2016, Decis. Support Syst..

[6]  Panos M. Pardalos,et al.  Computational approaches and data analytics in financial services: A literature review , 2019, J. Oper. Res. Soc..

[7]  Oluwasanmi Koyejo,et al.  Examples are not enough, learn to criticize! Criticism for Interpretability , 2016, NIPS.

[8]  Cynthia Rudin,et al.  All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously , 2019, J. Mach. Learn. Res..

[9]  J. Friedman Greedy function approximation: A gradient boosting machine. , 2001 .

[10]  Hussein A. Abdou,et al.  Credit Scoring, Statistical Techniques and Evaluation Criteria: A Review of the Literature , 2011, Intell. Syst. Account. Finance Manag..

[11]  Yu Jin,et al.  A Data-Driven Approach to Predict Default Risk of Loan for Online Peer-to-Peer (P2P) Lending , 2015, 2015 Fifth International Conference on Communication Systems and Network Technologies.

[12]  Riza Emekter,et al.  Evaluating credit risk and loan performance in online Peer-to-Peer (P2P) lending , 2015 .

[13]  K. Maddulety,et al.  Machine Learning in Banking Risk Management: A Literature Review , 2019, Risks.

[14]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[15]  Shanlin Yang,et al.  Heterogeneous Ensemble for Default Prediction of Peer-to-Peer Lending in China , 2018, IEEE Access.

[16]  J. Suykens,et al.  Benchmarking state-of-the-art classification algorithms for credit scoring: An update of research , 2015, Eur. J. Oper. Res..

[17]  Yufei Xia,et al.  A boosted decision tree approach using Bayesian hyper-parameter optimization for credit scoring , 2017, Expert Syst. Appl..

[18]  Avanti Shrikumar,et al.  Learning Important Features Through Propagating Activation Differences , 2017, ICML.

[19]  Jing Duan,et al.  Financial system modeling using deep neural networks (DNNs) for effective risk assessment and prediction , 2019, J. Frankl. Inst..

[20]  Woojin Chang,et al.  Application of Instance-Based Entropy Fuzzy Support Vector Machine in Peer-To-Peer Lending Investment Decision , 2019, IEEE Access.

[21]  Xu Sun,et al.  Fast Implementation of DeLong’s Algorithm for Comparing the Areas Under Correlated Receiver Operating Characteristic Curves , 2014, IEEE Signal Processing Letters.

[22]  Ha-Nam Nguyen,et al.  Improving Credit Risk Prediction in Online Peer-to-Peer (P2P) Lending Using Feature selection with Deep learning , 2019, 2019 21st International Conference on Advanced Communication Technology (ICACT).

[23]  D. Hand,et al.  Testing the difference between two Kolmogorov–Smirnov values in the context of receiver operating characteristic curves , 2011 .

[24]  Yoichi Hayashi,et al.  High Accuracy-priority Rule Extraction for Reconciling Accuracy and Interpretability in Credit Scoring , 2018, New Generation Computing.

[25]  L. Shapley A Value for n-person Games , 1988 .

[26]  Yufei Xia,et al.  Cost-sensitive boosted tree for loan evaluation in peer-to-peer lending , 2017, Electron. Commer. Res. Appl..

[27]  Paolo Giudici,et al.  Latent factor models for credit scoring in P2P systems , 2018 .

[28]  Emil Pitkin,et al.  Peeking Inside the Black Box: Visualizing Statistical Learning With Plots of Individual Conditional Expectation , 2013, 1309.6392.

[29]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[30]  Q. Mcnemar Note on the sampling error of the difference between correlated proportions or percentages , 1947, Psychometrika.

[31]  Yijing Li,et al.  Learning from class-imbalanced data: Review of methods and applications , 2017, Expert Syst. Appl..

[32]  Kaveh Bastani,et al.  Wide and Deep Learning for Peer-to-Peer Lending , 2018, Expert Syst. Appl..

[33]  Zachary Chase Lipton The mythos of model interpretability , 2016, ACM Queue.

[34]  Daniel W. Apley,et al.  Visualizing the effects of predictor variables in black box supervised learning models , 2016, Journal of the Royal Statistical Society: Series B (Statistical Methodology).

[35]  Raquel Flórez López,et al.  Enhancing accuracy and interpretability of ensemble strategies in credit risk assessment. A correlated-adjusted decision forest proposal , 2015, Expert Syst. Appl..

[36]  Zihao Wang,et al.  Credit Risk Assessment of Peer-to-Peer Lending Borrower Utilizing BP Neural Network , 2017, EIDWT.

[37]  Xiaojun Ma,et al.  Study on a prediction of P2P network loan default based on the machine learning LightGBM and XGboost algorithms according to different high dimensional data cleaning , 2018, Electron. Commer. Res. Appl..

[38]  Zhao Wang,et al.  A Novel behavioral scoring model for estimating probability of default over time in peer-to-peer lending , 2018, Electron. Commer. Res. Appl..

[39]  Tim Miller,et al.  Explanation in Artificial Intelligence: Insights from the Social Sciences , 2017, Artif. Intell..

[40]  Xin Ye,et al.  Loan evaluation in P2P lending based on Random Forest optimized by genetic algorithm with profit score , 2018, Electron. Commer. Res. Appl..

[41]  Kjersti Aas,et al.  Explaining individual predictions when features are dependent: More accurate approximations to Shapley values , 2019, Artif. Intell..

[42]  Carlos Serrano-Cinca,et al.  Determinants of Default in P2P Lending , 2015, PloS one.

[43]  Tsendsuren Munkhdalai,et al.  An Empirical Comparison of Machine-Learning Methods on Bank Client Credit Assessments , 2019, Sustainability.

[44]  Scott M. Lundberg,et al.  Consistent Individualized Feature Attribution for Tree Ensembles , 2018, ArXiv.

[45]  Jaime S. Cardoso,et al.  Machine Learning Interpretability: A Survey on Methods and Metrics , 2019, Electronics.

[46]  Cynthia Rudin,et al.  An Interpretable Model with Globally Consistent Explanations for Credit Risk , 2018, ArXiv.

[47]  Philip Wolfe,et al.  Contributions to the theory of games , 1953 .

[48]  Huimin Zhao,et al.  A prediction-driven mixture cure model and its application in credit scoring , 2019, Eur. J. Oper. Res..

[49]  Xiaoli Yang,et al.  A rejection inference technique based on contrastive pessimistic likelihood estimation for P2P lending , 2018, Electron. Commer. Res. Appl..

[50]  Ricardo A. L. Rabêlo,et al.  Machine learning and decision support system on credit scoring , 2019, Neural Computing and Applications.

[51]  Qinghua Huang,et al.  Predicting financial distress and corporate failure: A review from the state-of-the-art definitions, modeling, sampling, and featuring approaches , 2014, Knowl. Based Syst..

[52]  Jon Frost,et al.  Fintech Credit Markets Around the World: Size, Drivers and Policy Issues , 2018 .

[53]  Marian B. Gorzalczany,et al.  A multi-objective genetic optimization for fast, fuzzy rule-based credit classification with balanced accuracy and interpretability , 2016, Appl. Soft Comput..

[54]  Gabriel Erion,et al.  Explainable AI for Trees: From Local Explanations to Global Understanding , 2019, ArXiv.

[55]  Scott M. Lundberg,et al.  Explainable machine-learning predictions for the prevention of hypoxaemia during surgery , 2018, Nature Biomedical Engineering.

[56]  Wei-Yang Lin,et al.  Machine Learning in Financial Crisis Prediction: A Survey , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[57]  Ying Wah Teh,et al.  Credit Scoring Models Using Soft Computing Methods: A Survey , 2010, Int. Arab J. Inf. Technol..

[58]  Vural Aksakalli,et al.  Risk assessment in social lending via random forests , 2015, Expert Syst. Appl..

[59]  Andreas Joseph,et al.  Parametric inference with universal function approximators , 2019, SSRN Electronic Journal.

[60]  Been Kim,et al.  Towards A Rigorous Science of Interpretable Machine Learning , 2017, 1702.08608.

[61]  Jean Paul Barddal,et al.  Improving Credit Risk Prediction in Online Peer-to-Peer (P2P) Lending Using Imbalanced Learning Techniques , 2017, 2017 IEEE 29th International Conference on Tools with Artificial Intelligence (ICTAI).

[62]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[63]  Scott Lundberg,et al.  A Unified Approach to Interpreting Model Predictions , 2017, NIPS.

[64]  Wenhua Zeng,et al.  Applications of artificial intelligence technologies in credit scoring: A survey of literature , 2014, 2014 10th International Conference on Natural Computation (ICNC).

[65]  Mohsen Naderpour,et al.  Credit risk prediction in an imbalanced social lending environment , 2018, Int. J. Comput. Intell. Syst..

[66]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.