Generalized Density-Estimate Memory for Dynamic Problems

Optimization systems traditionally focus on static proble ms, also known as offline or a priori optimization problems. Many real-world problems may be better model e as dynamic optimization problems, also known as stochastic, in situ, or online optimization proble ms. In these types of problems, the fitness landscape of the search space changes over time. While it may be po ssible to learn policies offline for every potential scenario, doing so may not be tractable or sufficie ntly robust. Instead, recent work has focused on optimization concurrent with dynamic processes in order to t ack moving optima. One part of the field that has seen a great deal of work in the las t decade is dynamic optimization with evolutionary algorithms (EAs). The population-based sear ch used in most EAs has proven very useful for dynamic problems, since the population helps to ease optimi zation after changes in the fitness landscape. However, normal EAs encounter several difficulties with dyn amic problems; foremost, the convergence of a population on a single peak in the fitness landscape deple tes the diversity of the population, making adaptation after a change in the fitness landscape more diffic ult. One method for countering this problem has been the use of memory to retain good solutions for future se. Memory aids dynamic optimization in several ways. By mainta ining good solutions from the past, memory may speed up search for similar solutions after a dyna mic event. Memory entries also provide additional points to search from after a change, and when the population has converged, may add a great deal of diversity to the population. Memory has been used ext ensively for dynamic optimization, and while there are many variations, a standard memory model has emerg d. In this model, a small portion of the population is reserved for memory. For many problems, this t ype of memory has helped to improve the performance of dynamic optimization. However, because the memory size is finite, and usually small, it may be difficult to store enough solutions to accurately mo del the search space over time. Also, for problems where the feasible region of the search space chang es with time, memory entries may become infeasible, making the memory less useful. This proposal examines the development of more generalized models of memory for dynamic problems. A density-estimate memory for dynamic problems which build s rich models of the dynamic search space efficiently and then uses that information to improve the qua lity of solutions returned to the search process is proposed. By building models within the memory, the searc h process can continue to interact with a limited number of entries, except now these entries are mode ls incorporating many individual solutions. The models stored in memory can be continuously refined by upd ating the models as new solutions are stored to the memory. These models can be used to help keep sea rch diverse in a more informed way and help retrieve the most useful solutions from the memory. An e xtension of memory to problems where the feasible region of the search space changes over time throug the development of an indirect memory layer is also proposed, which will open up the use of memory to many d y amic problems which could not use information from the past. The indirect memory layer will us e environmental information from the problem to create an indirect representation of a solution which can be stored in memory. The stored entry can be mapped to a feasible solution at a future time. The work propo sed here will lead to a generalized memory which can be applied to many dynamic problems.