Particle Swarm Optimization: A Powerful Family of Stochastic Optimizers. Analysis, Design and Application to Inverse Modelling

Inverse problems are ill-posed: the error function has its minimum in a flat elongated valley or surrounded by many local minima. Local optimization methods give unpredictable results if no prior information is available. Traditionally this has generated mistrust for the use of inverse methods. Stochastic approaches to inverse problems consists in shift attention to the probability of existence of certain kinds of models (called equivalent) instead of "looking for the true model". Also, inverse problems are ill-conditioned and often the observed data are noisy. Global optimization methods have become a good alternative to sample the model space efficiently. These methods are very robust since they solve the inverse problem as a sampling problem, but they are hampered by dimensionality issues and high computational costs needed to solve the forward problem (predictions). In this paper we show how our research over the last three years on particle swarm optimizers can be used to solve and evaluate inverse problems efficiently. Although PSO is a stochastic algorithm, it can be physically interpreted as a stochastic damped mass-spring system. This analogy allowed us to introduce the PSO continuous model, to deduce a whole family of PSO algorithms, and to provide some results of its convergence based on the stochastic stability of the particle trajectories. This makes PSO a particularly interesting algorithm, different from other global algorithms which are purely heuristic. We include the results of an application of our PSO algorithm to the prediction of phosphorylation sites in proteins, an important mechanism for regulation of biological function. Our PSO optimization methods have enabled us to predict phosphorylation sites with higher accuracy and with better generalization, than other reports on similar studies in literature. Our preliminary studies on 984 protein sequences show that our algorithm can predict phosphorylation sites with a training accuracy of 92.5% and a testing accuracy 91.4%, when combined with a neural network based algorithm called Extreme Learning Machine.