A particle swarm algorithm with broad applicability in shape-constrained estimation

In nonparametric function estimation, the inclusion of shape constraints can confer several advantages, including improved estimation accuracy, reduced sensitivity to smoothing parameters, and control over the qualitative appearance of the estimate. Finding shape-restricted estimates may require solving a difficult optimization problem, however, making these advantages hard to realize. A particle swarm algorithm is proposed to overcome this barrier and expand the possibilities for shape-constrained estimation. The algorithm uses a cooperative search strategy with two swarms, one focused on global exploration and one focused on local exploitation. The new heuristic has the added advantage of being a general tool, applicable without modification to a variety of estimators, constraints, and objective functions. The algorithm is demonstrated on a number of density estimation and regression problems, and the potential for further improvement is discussed. Supplementary materials, including source code, are available online.

[1]  Andries Petrus Engelbrecht,et al.  Fundamentals of Computational Swarm Intelligence , 2005 .

[2]  Kee-Hoon Kang,et al.  Unimodal kernel density estimation by datra sharpening , 2005 .

[3]  John W. Kern,et al.  Using the bootstrap and fast Fourier transform to estimate confidence intervals of 2D kernel densities , 2003, Environmental and Ecological Statistics.

[4]  J. Simonoff.  Smoothing Methods in Statistics , 1998 .

[5]  P. Hall,et al.  NONPARAMETRIC KERNEL REGRESSION SUBJECT TO MONOTONICITY CONSTRAINTS , 2001 .

[6]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[7]  Jérôme Morio,et al.  Optimisation of interacting particle systems for rare event estimation , 2013, Comput. Stat. Data Anal..

[8]  M. C. Jones,et al.  A reliable data-based bandwidth selection method for kernel density estimation , 1991 .

[9]  Christopher F. Parmeter,et al.  Imposing Economic Constraints in Nonparametric Regression: Survey, Implementation and Extension , 2009, SSRN Electronic Journal.

[10]  J. Kennedy,et al.  Population structure and particle swarm performance , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[11]  W. John Braun,et al.  Data Sharpening for Nonparametric Inference Subject to Constraints , 2001 .

[12]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[13]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[14]  A. Walden,et al.  Wavelet Methods for Time Series Analysis , 2000 .

[15]  Jeffrey S. Racine,et al.  Nonparametric Kernel Regression with Multiple Predictors and Multiple Shape Constraints , 2012 .

[16]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.