Justified advice: a semi-naturalistic study of advisory strategies

“Wizard of Oz” techniques were used to observe the interaction between users of a statistical package and a human playing the role of an simulated intelligent advisory system. The results emphasized the complexities of the advisory process. More than half of the clients' requests sought help on planning actions toward achieving task goals. Further, protocols collected from the advisor while advice was given revealed the importance of constructing models of the user's current and past interaction with the application, and of addressing the high-level goals that underlie clients' explicit questions. The relevance of these findings to the development of intelligent advisory systems is discussed.