An Efficient Self-Tuning Proposal Distribution for Random Variate Generation With Complex Density Representation

Random variate generation has been widely used in various engineering applications and model frameworks. The introduction of the proposal distribution makes this kind of methods can deal with more general sampling problems with complex probability density representation. However, the design of proposal distribution is usually difficult, especially when the target distribution is so complex that it contains asymmetric, multimodal and skewed, or even unsmooth, heavy tailed and illegal density, the sampling performance and efficiency will be greatly affected. In this paper, a novel parameter optimization method with efficient self-tuning strategy is proposed in order to automatically construct the optimal proposal distribution for any complex target density, which treats the design of proposal distribution as a parameter search process and search the optimal solution in an infeasible region by evaluating the loss of solution. The significant advantage of the proposed method is that the search based on infeasible region can converge to a good solution with only a few iterations, making our method far superior to other existing methods in efficiency, which is very suitable for the complex target distribution model with time-consuming density calculation process. Experimental results show the advantages of the proposed method in terms of search strategy, optimal solution performance, search efficiency and robustness.

[1]  Mirza Ali Shaheer,et al.  A novel hierarchical hyper-parameter search algorithm based on greedy strategy for wind turbine fault diagnosis , 2022, Expert Syst. Appl..

[2]  T. Hachisuka,et al.  Regression-based Monte Carlo integration , 2022, ACM Trans. Graph..

[3]  Y. Tong,et al.  Research on Hyper-Parameter Optimization of Activity Recognition Algorithm Based on Improved Cuckoo Search , 2022, Entropy.

[4]  W. Yin,et al.  A study on depth classification of defects by machine learning based on hyper-parameter search , 2022, Measurement.

[5]  Zonghai Chen,et al.  FasterGICP: Acceptance-Rejection Sampling Based 3D Lidar Odometry , 2022, IEEE Robotics and Automation Letters.

[6]  S. Prabu,et al.  Grid Search for Predicting Coronary Heart Disease by Tuning Hyper-Parameters , 2022, Comput. Syst. Sci. Eng..

[7]  R. Spiteri,et al.  A new very simply explicitly invertible approximation for the standard normal cumulative distribution function , 2022, AIMS Mathematics.

[8]  M. Akhmet.  Unpredictability in Markov chains , 2021, Carpathian Journal of Mathematics.

[9]  Wei-guo Zhang,et al.  Optimal futures hedging strategies based on an improved kernel density estimation method , 2021, Soft Computing.

[10]  Tao Zan,et al.  Thermal error prediction of ball screws based on PSO-LSTM , 2021, The International Journal of Advanced Manufacturing Technology.

[11]  Peter Stone,et al.  Importance sampling in reinforcement learning with an estimated behavior policy , 2021, Machine Learning.

[12]  Andreas Wichert,et al.  Quantum-like Gaussian mixture model , 2021, Soft Computing.

[13]  Shalini Batra,et al.  Deep neural network hyper-parameter tuning through twofold genetic approach , 2021, Soft Computing.

[14]  Benoit Forget,et al.  A simple method for rejection sampling efficiency improvement on SIMT architectures , 2021, Stat. Comput..

[15]  Hadi Hemmati,et al.  A pragmatic approach for hyper-parameter tuning in search-based test case generation , 2021, Empirical Software Engineering.

[16]  R. Valarmathi,et al.  Heart disease prediction using hyper parameter optimization (HPO) tuning , 2021, Biomed. Signal Process. Control..

[17]  Alfrêdo Gomes Neto,et al.  Synthesis of Multiband Frequency Selective Surfaces Using Machine Learning With the Decision Tree Algorithm , 2021, IEEE Access.

[18]  Zhili Sun,et al.  Importance Sampling for Time-Variant Reliability Analysis , 2021, IEEE Access.

[19]  N. Madani,et al.  A hierarchical cosimulation algorithm integrated with an acceptance–rejection method for the geostatistical modeling of variables with inequality constraints , 2020, Stochastic Environmental Research and Risk Assessment.

[20]  Min Li,et al.  Two-Stage Stochastic Variational Inequality Arising from Stochastic Programming , 2020, Journal of Optimization Theory and Applications.

[21]  Yangyang Li,et al.  Hyper-Parameter Optimization Using MARS Surrogate for Machine-Learning Algorithms , 2020, IEEE Transactions on Emerging Topics in Computational Intelligence.

[22]  Pei Wang,et al.  Acceptance-Rejection Sampling Based Monte Carlo Ray Tracing in Anisotropic Porous Media , 2020 .

[23]  Amel Benazza-Benyahia,et al.  Majorize–Minimize Adapted Metropolis–Hastings Algorithm , 2020, IEEE Transactions on Signal Processing.

[24]  Tung-Yu Wu,et al.  Mini-batch Metropolis-Hastings MCMC with Reversible SGLD Proposal , 2019, ArXiv.

[25]  Niao He,et al.  Sample Complexity of Sample Average Approximation for Conditional Stochastic Optimization , 2019, SIAM J. Optim..

[26]  Cheolkeun Ha,et al.  High Precision Weighted Optimum K-Nearest Neighbors Algorithm for Indoor Visible Light Positioning Applications , 2020, IEEE Access.

[27]  Ebrahim Babaei,et al.  A Hybrid Optimization Technique Using Exchange Market and Genetic Algorithms , 2020, IEEE Access.

[28]  Kevin H. Knuth,et al.  Optimal data-based binning for histograms and histogram-based probability density models , 2019, Digit. Signal Process..

[29]  Joose Rajamäki,et al.  Continuous Control Monte Carlo Tree Search Informed by Multiple Experts , 2019, IEEE Transactions on Visualization and Computer Graphics.

[30]  Ling Shao,et al.  Classification complexity assessment for hyper-parameter optimization , 2019, Pattern Recognit. Lett..

[31]  L. Martino,et al.  Adaptive rejection sampling with fixed number of nodes , 2015, Commun. Stat. Simul. Comput..

[32]  Guiding Gu,et al.  An efficient exponential twisting importance sampling technique for pricing financial derivatives , 2018, Communications in Statistics - Theory and Methods.

[33]  Theodore R Rieger,et al.  Improving the generation and selection of virtual populations in quantitative systems pharmacology models. , 2018, Progress in biophysics and molecular biology.

[34]  M. Weber,et al.  An Automatic Adaptive Importance Sampling Algorithm for Molecular Dynamics in Reaction Coordinates , 2018, SIAM J. Sci. Comput..

[35]  Denis Marcotte,et al.  Gibbs sampling on large lattice with GMRF , 2018, Comput. Geosci..

[36]  Andry Rakotonirainy,et al.  Long Short Term Memory Hyperparameter Optimization for a Neural Network Based Emotion Recognition Framework , 2018, IEEE Access.

[37]  David B. Dunson,et al.  Data augmentation for models based on rejection sampling , 2014, Biometrika.

[38]  Min-Yuan Cheng,et al.  Slope Collapse Prediction Using Bayesian Framework with K-Nearest Neighbor Density Estimation: Case Study in Taiwan , 2016, J. Comput. Civ. Eng..

[39]  Amer Ibrahim Al-Omari,et al.  Estimation of entropy using random sampling , 2014, J. Comput. Appl. Math..

[40]  Sancho Salcedo-Sanz,et al.  New validation methods for improving standard and multi-parametric support vector regression training time , 2012, Expert Syst. Appl..

[41]  Yoshua Bengio,et al.  Random Search for Hyper-Parameter Optimization , 2012, J. Mach. Learn. Res..

[42]  Carsten H. Botts,et al.  A modified adaptive accept–reject algorithm for univariate densities with bounded support , 2011 .

[43]  Luc Devroye,et al.  Chapter 4 Nonuniform Random Variate Generation , 2006, Simulation.

[44]  W. Gilks,et al.  Adaptive Rejection Sampling for Gibbs Sampling , 1992 .

[45]  M. Wand,et al.  EXACT MEAN INTEGRATED SQUARED ERROR , 1992 .