Actively probing and modeling users in interactive coevolution

A major challenge in interactive evolution is extracting user preferences with minimal probing. We introduce an interactive multi-objective coevolutionary algorithm that actively selects the most informative probes: We simultaneously coevolve a population of candidate models that explain users' selection so far, and a population of candidate probes that cause the most divergence among model predictions, thereby elucidating model uncertainties (divergence). As progress is made, we begin selecting for probes with the highest expected outcome averaged among different models, thereby exploiting model certainties (consensus). In the evolution of pen stroke drawings, we find this technique to be highly effective at extracting preference models from very limited human interaction. Using only pair-wise preference questions, strategy and preference in pen stroke drawings are extracted in fewer than ten user probes. Our results show that the optimal questions to probe the user need not include drawings similar to the target drawing. Instead, the user models converge on trends in the user responses, thereby extrapolating strong preference for target drawings which the models are never actually trained to prefer.

[1]  Clive Richards,et al.  The Blind Watchmaker , 1987, Bristol Medico-Chirurgical Journal.

[2]  John Scott Bridle,et al.  Probabilistic Interpretation of Feedforward Classification Network Outputs, with Relationships to Statistical Pattern Recognition , 1989, NATO Neurocomputing.

[3]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[4]  Charles E. Taylor,et al.  Artificial Life II , 1991 .

[5]  John R. Koza,et al.  Genetic Programming II , 1992 .

[6]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[7]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[8]  Erik Hollnagel,et al.  Human Reliability Analysis: Context and Control , 1994 .

[9]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[10]  Ian D. Bishop,et al.  Comparing regression and neural net based approaches to modelling of scenic beauty , 1996 .

[11]  Samir W. Mahfoud Niching methods for genetic algorithms , 1996 .

[12]  R. Poli Genetic programming for image analysis , 1996 .

[13]  Greg Linden,et al.  Interactive Assessment of User Preference Models: The Automated Travel Assistant , 1997 .

[14]  Riccardo Poli,et al.  Genetic Programming with User-Driven Selection : Experiments on the Evolution of Algorithms for Image Enhancement , 1997 .

[15]  Rich Caruana,et al.  Overfitting in Neural Nets: Backpropagation, Conjugate Gradient, and Early Stopping , 2000, NIPS.

[16]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[17]  Jordan B. Pollack,et al.  Pareto Optimality in Coevolutionary Learning , 2001, ECAL.

[18]  Hideyuki Takagi,et al.  Interactive evolutionary computation: fusion of the capabilities of EC optimization and human evaluation , 2001, Proc. IEEE.

[19]  Ian F. C. Smith,et al.  Special Issue: Human–computer interaction in engineering contexts , 2002, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[20]  Eric Bonabeau,et al.  Agent-based modeling: Methods and techniques for simulating human systems , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[21]  Ian C. Parmee,et al.  Improving problem definition through interactive evolutionary computation , 2002, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[22]  Hod Lipson,et al.  Co-evolutionary variance can guide physical testing in evolutionary system identification , 2005, 2005 NASA/DoD Conference on Evolvable Hardware (EH'05).

[23]  Eric Bonabeau,et al.  Interactive estimation of agent-based financial markets models: modularity and learning , 2005, GECCO '05.