Applying Bayesian Optimization for Machine Learning Models in Predicting the Surface Roughness in Single-Point Diamond Turning Polycarbonate

This paper deals with the prediction of surface roughness in manufacturing polycarbonate (PC) by applying Bayesian optimization for machine learning models. The input variables of ultraprecision turning—namely, feed rate, depth of cut, spindle speed, and vibration of the X-, Y-, and Z-axis—are the main factors affecting surface quality. In this research, six machine learning- (ML-) based models—artificial neural network (ANN), Cat Boost Regression (CAT), Support Vector Machine (SVR), Gradient Boosting Regression (GBR), Decision Tree Regression (DTR), and Extreme Gradient Boosting Regression (XGB)—were applied to predict the surface roughness (Ra). The predictive performance of the baseline models was quantitatively assessed through error metrics: root means square error (RMSE), mean absolute error (MAE), and coefficient of determination (R2). The overall results indicate that the XGB and CAT models predict Ra with the greatest accuracy. In improving baseline models such as XGB and CAT, the Bayesian optimization (BO) is next used to determine their best hyperparameters, and the results indicate that XGB is the best model according to the evaluation metrics. Results have shown that the performance of the models has been improved significantly with BO. For example, the values of RMSE and MAE of XGB have decreased from 0.0076 to 0.0047 and from 0.0063 to 0.0027, respectively, for the training dataset. Using the testing dataset, the values of RMSE and MAE of XGB have decreased from 0.4033 to 0.2512 and from 0.2845 to 0.2225, respectively. Moreover, the vibrations of the X, Y, and Z axes and feed rate are the most significant feature in predicting the results, which is in high accordance with the literature. We find that, in a specified value domain, the vibration of the axes has a greater influence on the surface quality than does the cutting condition.

[1]  W. Zeng,et al.  Evaluation of CatBoost method for prediction of reference evapotranspiration in humid regions , 2019, Journal of Hydrology.

[2]  Yoshua Bengio,et al.  Hierarchical Probabilistic Neural Network Language Model , 2005, AISTATS.

[3]  Tien-Thinh Le.  Prediction of tensile strength of polymer carbon nanotube composites using practical machine learning method , 2020, Journal of Composite Materials.

[4]  Andrés Bustillo,et al.  Artificial intelligence for automatic prediction of required surface roughness by monitoring wear on face mill teeth , 2017, Journal of Intelligent Manufacturing.

[5]  Tien-Thinh Le.  Practical Hybrid Machine Learning Approach for Estimation of Ultimate Load of Elliptical Concrete-Filled Steel Tubular Columns under Axial Loading , 2020 .

[6]  Herbert Kimura,et al.  Stock price prediction using support vector regression on daily and up to the minute prices , 2018, The Journal of Finance and Data Science.

[7]  Uthayasanker Thayasivam,et al.  Taxi Trip Travel Time Prediction with Isolated XGBoost Regression , 2019, 2019 Moratuwa Engineering Research Conference (MERCon).

[8]  Prince Waqas Khan,et al.  Machine Learning-Based Approach to Predict Energy Consumption of Renewable and Nonrenewable Power Sources , 2020, Energies.

[9]  Lin He,et al.  Prediction of Surface Roughness of 304 Stainless Steel and Multi-Objective Optimization of Cutting Parameters Based on GA-GBRT , 2019, Applied Sciences.

[10]  Jose Barata,et al.  Multistage Quality Control Using Machine Learning in the Automotive Industry , 2019, IEEE Access.

[11]  Tadeusz Mikolajczyk,et al.  Modeling of Cutting Parameters and Tool Geometry for Multi-Criteria Optimization of Surface Roughness and Vibration via Response Surface Methodology in Turning of AISI 5140 Steel , 2020, Materials.

[12]  V. Sudarsan.  Optical Materials: Fundamentals and Applications , 2012 .

[13]  Muammer Nalbant,et al.  Comparison of regression and artificial neural network models for surface roughness prediction with the cutting parameters in CNC turning , 2007 .

[14]  W. Zong,et al.  Influencing Factors and Theoretical Models for the Surface Topography in Diamond Turning Process: A Review , 2019, Micromachines.

[15]  Yoshua Bengio,et al.  Random Search for Hyper-Parameter Optimization , 2012, J. Mach. Learn. Res..

[16]  Surjya K. Pal,et al.  Surface roughness prediction in turning using artificial neural network , 2005, Neural Computing & Applications.

[17]  H. Brouwer,et al.  Color stability of polycarbonate for optical applications , 2015 .

[18]  Tien-Thinh Le,et al.  Prediction of Ultimate Load of Rectangular CFST Columns Using Interpretable Machine Learning Method , 2020, Advances in Civil Engineering.

[19]  B. Sahoo,et al.  Application of Support Vector Regression for Modeling Low Flow Time Series , 2019, KSCE Journal of Civil Engineering.

[20]  F. Kara,et al.  Artificial Intelligence-Based Surface Roughness Estimation Modelling for Milling of AA6061 Alloy , 2021 .

[21]  Tadeusz Mikolajczyk,et al.  Parametric optimization and process capability analysis for machining of nickel-based superalloy , 2019, The International Journal of Advanced Manufacturing Technology.

[22]  Guang-Bin Huang,et al.  Extreme Learning Machine for Multilayer Perceptron , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[23]  Niklaus E. Zimmermann,et al.  Predicting tree species presence and basal area in Utah: A comparison of stochastic gradient boosting, generalized additive models, and tree-based methods , 2006 .

[24]  Mahardhika Pratama,et al.  Financial time series forecasting using twin support vector regression , 2019, PloS one.

[25]  Mahdi S Alajmi,et al.  Predicting the Tool Wear of a Drilling Process Using Novel Machine Learning XGBoost-SDA , 2020, Materials.

[26]  Yoshua Bengio,et al.  Algorithms for Hyper-Parameter Optimization , 2011, NIPS.

[27]  Tsendsuren Munkhdalai,et al.  An Empirical Comparison of Machine-Learning Methods on Bank Client Credit Assessments , 2019, Sustainability.

[28]  N. R. Sakthivel,et al.  Chatter prediction in boring process using machine learning technique , 2017, Int. J. Manuf. Res..

[29]  R. Manicka Chezian,et al.  Support Vector Regression to Forecast the Demand and Supply of Pulpwood , 2013 .

[30]  Tien-Thinh Le,et al.  Development of Deep Learning Model for the Recognition of Cracks on Concrete Surfaces , 2021, Appl. Comput. Intell. Soft Comput..

[31]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[32]  Kevin Leyton-Brown,et al.  Sequential Model-Based Optimization for General Algorithm Configuration , 2011, LION.

[33]  Anna Veronika Dorogush,et al.  CatBoost: gradient boosting with categorical features support , 2018, ArXiv.

[34]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[35]  A M Glass,et al.  Optical Materials , 1987, Science.

[36]  Lai-Wan Chan,et al.  Support Vector Machine Regression for Volatile Stock Market Prediction , 2002, IDEAL.

[37]  Mirella Lapata,et al.  Long Short-Term Memory-Networks for Machine Reading , 2016, EMNLP.

[38]  Tien-Thinh Le,et al.  Optimization design of rectangular concrete-filled steel tube short columns with Balancing Composite Motion Optimization and data-driven model , 2020 .

[39]  Tien-Thinh Le.  Probabilistic modeling of surface effects in nano-reinforced materials , 2021 .

[40]  Christophe Guyeux,et al.  Predicting Fire Brigades Operational Breakdowns: A Real Case Study , 2020 .

[41]  R. Sharma,et al.  Rice Yield Forecasting using Support Vector Machine , 2019, International Journal of Recent Technology and Engineering.

[42]  Christoph Gerhard.  Optics Manufacturing: Components and Systems , 2017 .

[43]  Adem Çiçek,et al.  Artificial neural network based modelling of performance of a beta-type Stirling engine , 2013 .

[44]  N. R. Sakthivel,et al.  Machine Learning Approach to the Prediction of Surface Roughness Using Statistical Features of Vibration Signal Acquired in Turning , 2015 .

[45]  Chris Eliasmith,et al.  Hyperopt: a Python library for model selection and hyperparameter optimization , 2015 .

[46]  P. J. García Nieto,et al.  Application of an SVM-based regression model to the air quality study at local scale in the Avilés urban area (Spain) , 2011, Math. Comput. Model..

[47]  Zijian Liu,et al.  A deep learning based multitask model for network-wide traffic speed prediction , 2020, Neurocomputing.

[48]  H. Abbasimehr,et al.  Prediction of COVID-19 confirmed cases combining deep learning methods and Bayesian optimization , 2020, Chaos, Solitons & Fractals.

[49]  Johannes Gehrke,et al.  Data Mining with Decision Trees , 2000, ICDE.

[50]  Yuzhen Zhang,et al.  An Evaluation of Eight Machine Learning Regression Algorithms for Forest Aboveground Biomass Estimation from Multiple Satellite Data Products , 2020, Remote. Sens..

[51]  J Elith,et al.  A working guide to boosted regression trees. , 2008, The Journal of animal ecology.

[52]  Ah Chung Tsoi,et al.  Discrete time recurrent neural network architectures: A unifying review , 1997, Neurocomputing.

[53]  A. T. Abbas,et al.  Prediction Model of Cutting Parameters for Turning High Strength Steel Grade-H: Comparative Study of Regression Model versus ANFIS , 2017 .

[54]  Thomas Guenther,et al.  Review on Fabrication Technologies for Optical Mold Inserts , 2019, Micromachines.

[55]  T. Chai,et al.  Root mean square error (RMSE) or mean absolute error (MAE)? – Arguments against avoiding RMSE in the literature , 2014 .

[56]  Tien-Thinh Le,et al.  Effects of variability in experimental database on machine-learning-based prediction of ultimate load of circular concrete-filled steel tubes , 2021 .

[57]  Stanislaw Legutko,et al.  Predicting the tool life in the dry machining of duplex stainless steel , 2013 .

[58]  Tien-Thinh Le,et al.  Development of user-friendly kernel-based Gaussian process regression model for prediction of load-bearing capacity of square concrete-filled steel tubular members , 2021, Materials and Structures.

[59]  Tien-Thinh Le,et al.  An empirical model for bending capacity of defected pipe combined with axial load , 2021, International Journal of Pressure Vessels and Piping.

[60]  A. Almeshal,et al.  Prediction and Optimization of Surface Roughness in a Turning Process Using the ANFIS-QPSO Method , 2020, Materials.

[61]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[62]  Nhu Khue Vuong,et al.  Predicting Surface Roughness and Flank Wear in Turning Processes , 2020, 2020 IEEE International Conference on Prognostics and Health Management (ICPHM).

[63]  Yiming Li,et al.  A Remaining Useful Life Prediction Method Considering the Dimension Optimization and the Iterative Speed , 2019, IEEE Access.

[64]  Miran Brezocnik,et al.  A comparison of machine learning methods for cutting parameters prediction in high speed turning process , 2016, Journal of Intelligent Manufacturing.

[65]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[66]  Tien-Thinh Le.  Practical machine learning-based prediction model for axial capacity of square CFST columns , 2020, Mechanics of Advanced Materials and Structures.

[67]  Tien-Thinh Le,et al.  Nanoscale Effect Investigation for Effective Bulk Modulus of Particulate Polymer Nanocomposites Using Micromechanical Framework , 2021 .

[68]  Kevin Leyton-Brown,et al.  Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms , 2012, KDD.

[69]  Tien-Thinh Le.  Probabilistic investigation of the effect of stochastic imperfect interfaces in nanocomposites , 2020 .

[70]  Stanislaw Legutko,et al.  Predicting the surface roughness in the dry machining of duplex stainless steel (DSS) , 2013 .

[71]  Alexander J. Smola,et al.  Support Vector Regression Machines , 1996, NIPS.