Adaptive splines and genetic algorithms for optimal statistical modeling

In many statistical applications, a modeling technique is needed which can capture a relationship between two variables x and y that is more complex than a simple linear relationship. In the case where little is known about the function f which relates x and y, the modeling technique should be flexible or adaptive, i.e., able to handle a wide variety of functional shapes and behaviors. Nonparametric modeling is one such technique which has been successful in characterizing features of datasets that could not be obtainable by other means. Spatially adaptive smoothing methods involving regression splines have become a popular and rapidly developing class of such modeling techniques. Most of these methods are based on nonlinear optimization and/or stepwise selection of basis functions. Although computationally fast and spatially adaptive, stepwise knot selection is necessarily suboptimal while determining the best model over the space of adaptive splines is a very poorly behaved nonlinear optimization problem (Wahba 1988). A possible alternative is to use a genetic algorithm to perform knot selection. Genetic algorithms are stochastic search methods which, under certain design conditions, have been shown theoretically to converge to the global optimum of the evaluation function of an optimization problem (Banyopadhyay, Murthy, and Pal 1999). Hence, given a variable selection criterion and a search space of possible knot locations, a genetic algorithm has the potential to find models that are more appropriate in comparison to models selected using stepwise or nonlinear optimization techniques. In this work we explore the use of genetic algorithms for adaptive spline modeling in low dimensional settings. Chapters 3 and 4 concern only linear splines while Chapter 5 involves the optimal fitting of polynomial splines. Experimental results are compared to those of Luo and Wahba (1997), Donoho and Johnstone (1995), Friedman (1991), and Stone, Kooperberg, Hansen and Troung (1997), among others. Future research is focused on comparisons with the recent Bayesian spline methods (e.g., Denison, Mallick and Smith 1998), extensions to higher dimensions, and explorations into other areas of study such as neural networks and fractals.