Towards a Reliable Evaluation of Mixed-Initiative Systems

Mixed-Initiative approaches are being applied in different real world domains and many systems have been developed to address specific problems. Though several successful examples of such tools encourage the use of this solving paradigm, it is worth highlighting that research in mixed-initiative interaction is still at an early stage and many important issues ne ed to be addressed. In particular, while some work has been devoted to the design of working prototypes and to identify relevant features of the mixed-initiative interaction, littl e attention has been given to the problem of evaluating the approach as a whole and the diverse aspects involved. This work aims at highlighting the need for effective evaluation studies f or this class of tools and provides a methodological contribution in this direction. In particular it uses an experimental met hodology well known in psychology and human-computer interaction for the problem of understanding users’ attitude wit h respect to mixed-initiative problem solving and investigates the importance of explanation services as a means to foster users’ involvement in the problem solving.