gradDescentR: An R package implementing gradient descent and its variants for regression tasks

Gradient Descent (GD) is one of famous machine-learning methods used for prediction on regression tasks in many fields. However, only a few software library utilizing it can be found in the literature. Therefore, this research is aimed to implement the method and the following variants: Mini-Batch Gradient Descent (MBGD), Stochastic Gradient Descent (SGD), and Stochastic Average Gradient Descent (SAG). The package is written in the programming language R, which is a software environment offering many facilities for statistics and machine-learning tools. Moreover, we provide the use of these approaches in a case study of gas industries, which is to predict values of the compressibility factor (Z-factor) of CO2. Basically, we consider the problem as a regression task including pressure and temperature as the input attributes. The first step is generating a learning model from available training data by using the methods. Then, the model is used to predict the Z-factor over new data. A performance comparison between the package and other methods from the literature is presented as well.

[1]  Farhad Gharagheizi,et al.  Intelligent model for prediction of CO2 – Reservoir oil minimum miscibility pressure , 2013 .

[2]  Ohad Shamir,et al.  Better Mini-Batch Algorithms via Accelerated Gradient Methods , 2011, NIPS.

[4]  Francisco Herrera,et al.  Implementing algorithms of rough set theory and fuzzy rough set theory in the R package "RoughSets" , 2014, Inf. Sci..

[5]  Zhouping Xin,et al.  Step-sizes for the gradient method , 2008 .

[6]  Amir H. Mohammadi,et al.  Prediction of sour gas compressibility factor using an intelligent approach , 2013 .

[7]  G. Kennedy.  Pressure-volume-temperature relations in CO 2 at elevated temperatures and pressures , 1954 .

[8]  Ross Ihaka,et al.  Gentleman R: R: A language for data analysis and graphics , 1996 .

[9]  Mark W. Schmidt,et al.  A Stochastic Gradient Method with an Exponential Convergence Rate for Finite Training Sets , 2012, NIPS.

[10]  S. Ayatollahi,et al.  Experimental Determination of Equilibrium Interfacial Tension for Nitrogen-Crude Oil during the Gas Injection Process: The Role of Temperature, Pressure, and Composition , 2014 .

[11]  Tarek Ahmed,et al.  Reservoir Engineering Handbook , 2002 .

[12]  Sanjiban Sekhar Roy,et al.  An effective hybridized classifier for breast cancer diagnosis , 2015, 2015 IEEE International Conference on Advanced Intelligent Mechatronics (AIM).

[13]  Jyh-Shing Roger Jang,et al.  ANFIS: adaptive-network-based fuzzy inference system , 1993, IEEE Trans. Syst. Man Cybern..

[14]  J. R. Hastings,et al.  A Burnett apparatus for the accurate determination of gas compressibility factors and second virial coefficients, and an evaluation of its capability based on some results for argon and carbon dioxide , 1971 .

[15]  F. D. Monte.  Calculation of thermodynamic properties of R407C and R410A by the Martin–Hou equation of state — part I: theoretical development , 2002 .

[16]  William N. Venables,et al.  An Introduction To R , 2004 .

[17]  Amir H. Mohammadi,et al.  Efficient estimation of natural gas compressibility factor using a rigorous method , 2014 .

[18]  M. Mahmoud.  Development of a New Correlation of Gas Compressibility Factor (Z-Factor) for High Pressure Gas Reservoirs , 2014 .

[19]  Augustin-Louis Cauchy,et al.  ANALYSE MATHÉMATIQUE. – Méthodc générale pour la résolution des systèmes d'équations simultanées , 2009 .

[20]  Alexander J. Smola,et al.  Parallelized Stochastic Gradient Descent , 2010, NIPS.

[21]  Tarek Ahmed,et al.  Hydrocarbon Phase Behavior , 1989 .

[22]  Joseph Adler,et al.  R in a Nutshell - a Desktop Quick Reference , 2009 .

[23]  Léon Bottou,et al.  Large-Scale Machine Learning with Stochastic Gradient Descent , 2010, COMPSTAT.

[24]  A. Bachelor.  GLOSSARY OF TERMS GLOSSARY OF TERMS , 2010 .

[25]  Francisco Herrera,et al.  Learning from data using the R package "FRBS" , 2014, 2014 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE).

[26]  P. Siarry,et al.  Gradient descent method for optimizing various fuzzy rule bases , 1993, [Proceedings 1993] Second IEEE International Conference on Fuzzy Systems.

[27]  Boudewijn P. F. Lelieveldt,et al.  Fast Automatic Step Size Estimation for Gradient Descent Optimization of Image Registration , 2016, IEEE Transactions on Medical Imaging.

[28]  I. D. Watson,et al.  The prediction of the thermodynamic properties of fluids and fluid mixtures-I The principle of corresponding states and its extensions , 1969 .

[29]  A. Bahadori,et al.  Carbon dioxide compressibility factor determination using a robust intelligent method , 2015 .

[30]  Ge Ren,et al.  Restoration of Degraded Images Using Pupil-Size Diversity Technology With Stochastic Parallel Gradient Descent Algorithm , 2016, IEEE Photonics Journal.

[31]  Lala Septem Riza,et al.  frbs: Fuzzy Rule-Based Systems for Classification and Regression in R , 2015 .