Enabling context-sensitive information seeking

Information seeking is an important but often difficult task, especially when it involves large and complex data sets. We hypothesize that a context-sensitive interaction paradigm would greatly assist users in their information seeking. Such a paradigm would allow users to both express their requests and receive requested information in context. Driven by this hypothesis, we have taken rigorous steps to design, develop, and evaluate a full-fledged, context-sensitive information system. We started with a Wizard-of-OZ (WOZ) study to verify the effectiveness of our envi-sioned system. We then built a fully automated system based on the findings from our WOZ study. We targeted the development and integration of two sets of technologies: context-sensitive mul-timodal input interpretation and multimedia output generation. Finally, we formally evaluated the usability of our system in real world conditions. The results show that our system greatly improves the users' ability to perform practical information-seek-ing tasks. These results not only confirm our initial hypothesis, but they also indicate the practicality of our approaches.

[1]  Sharon Oviatt,et al.  Multimodal interactive maps: designing for human performance , 1997 .

[2]  Thomas Rist,et al.  Generating coherent presentations employing textual and visual material , 1995, Artificial Intelligence Review.

[3]  Michelle X. Zhou,et al.  A probabilistic approach to reference resolution in multimodal user interfaces , 2004, IUI '04.

[4]  Shimei Pan,et al.  Two-way adaptation for robust input interpretation in practical multimodal conversation systems , 2005, IUI '05.

[5]  James F. Allen,et al.  An architecture for more realistic conversational systems , 2001, IUI '01.

[6]  Steven K. Feiner,et al.  Dynamic space management for user interfaces , 2000, UIST '00.

[7]  Ben Shneiderman,et al.  Visual information seeking: tight coupling of dynamic query filters with starfield displays , 1994, CHI '94.

[8]  Wolfgang Wahlster,et al.  Readings in Intelligent User Interfaces , 1998 .

[9]  Michael C. McCord,et al.  Slot Grammar: A System for Simpler Construction of Practical Natural Language Grammars , 1989, Natural Language and Logic.

[10]  Xuehua Shen,et al.  Context-sensitive information retrieval using implicit feedback , 2005, SIGIR '05.

[11]  Min Chen,et al.  Automated Generation of Graphic Sketches by Example , 2003, IJCAI.

[12]  Shimei Pan A multi-layer conversation management approach for information seeking applications , 2004, INTERSPEECH.

[13]  Michelle X. Zhou,et al.  A graph-matching approach to dynamic media allocation in intelligent multimedia interfaces , 2005, IUI '05.

[14]  Gary Marchionini,et al.  Information Seeking in Electronic Environments , 1995 .

[15]  Ruth C. T. Morris Toward a User-Centered Information Service , 1994, J. Am. Soc. Inf. Sci..

[16]  Susan T. Dumais,et al.  Personalizing Search via Automated Analysis of Interests and Activities , 2005, SIGIR.

[17]  Oren Etzioni,et al.  Towards a theory of natural language interfaces to databases , 2003, IUI '03.

[18]  Mark S. Ackerman,et al.  The perfect search engine is not enough: a study of orienteering behavior in directed search , 2004, CHI.

[19]  Steven K. Feiner,et al.  Automating the generation of coordinated multimedia explanations , 1991, Computer.

[20]  Keith Houck Contextual revision in information seeking conversation systems , 2004, INTERSPEECH.

[21]  Michelle X. Zhou,et al.  An optimization-based approach to dynamic visual context management , 2005, IEEE Symposium on Information Visualization, 2005. INFOVIS 2005..

[22]  Michelle X. Zhou,et al.  An optimization-based approach to dynamic data content selection in intelligent multimedia interfaces , 2004, UIST '04.

[23]  Shimei Pan,et al.  Context-based multimodal input understanding in conversational systems , 2002, Proceedings. Fourth IEEE International Conference on Multimodal Interfaces.

[24]  C. Y. Thielman,et al.  Natural Language with Integrated Deictic and Graphic Gestures , 1989, HLT.

[25]  Hiroshi Hosobe A modular geometric constraint solver for user interface applications , 2001, UIST '01.

[26]  Carlo Strapparava,et al.  Multimodal Interaction for Information Access: Exploiting Cohesion , 1997, Comput. Intell..

[27]  Marilyn A. Walker,et al.  MATCH: An Architecture for Multimodal Dialogue Systems , 2002, ACL.

[28]  Mark T. Maybury,et al.  Planning Multimedia Explanations Using Communicative Acts , 1991, AAAI Workshop on Intelligent Multimedia Interfaces.

[29]  Candace L. Sidner,et al.  Attention, Intentions, and the Structure of Discourse , 1986, CL.

[30]  Johanna D. Moore,et al.  Integrating planning and task-based design for multimedia presentation , 1997, IUI '97.

[31]  Sharon L. Oviatt,et al.  Unification-based Multimodal Integration , 1997, ACL.

[32]  Shimei Pan,et al.  SEGUE: A Hybrid Case-Based Surface Natural Language Generator , 2004, INLG.