Ontology-Based User Modeling in an Augmented Audio Reality System for Museums

Ubiquitous computing is a challenging area that allows us to further our understanding and techniques of context-aware and adaptive systems. Among the challenges is the general problem of capturing the larger context in interaction from the perspective of user modeling and human–computer interaction (HCI). The imperative to address this issue is great considering the emergence of ubiquitous and mobile computing environments. This paper provides an account of our addressing the specific problem of supporting functionality as well as the experience design issues related to museum visits through user modeling in combination with an audio augmented reality and tangible user interface system. This paper details our deployment and evaluation of ec(h)o – an augmented audio reality system for museums. We explore the possibility of supporting a context-aware adaptive system by linking environment, interaction objects and users at an abstract semantic level instead of at the content level. From the user modeling perspective ec(h)o is a knowledge-based recommender system. In this paper we present our findings from user testing and how our approach works well with an audio and tangible user interface within a ubiquitous computing system. We conclude by showing where further research is needed.

[1]  Helene Hembrooke,et al.  Activity-Centered Design: An Ecological Approach to Designing Smart Tools and Usable Systems , 2004 .

[2]  K Sloan,et al.  The hand that rocks the cradle. , 1987, Film History.

[3]  Alfred Kobsa,et al.  User Modeling for Personalized City Tours , 2002, Artificial Intelligence Review.

[4]  J. Huizinga.  Homo Ludens: A Study of the Play-Element in Culture , 1938 .

[5]  Alfred Kobsa.  User Modeling and User-Adapted Interaction , 2005, User Modeling and User-Adapted Interaction.

[6]  Hiroshi Ishii,et al.  Token+constraint systems for tangible interaction with digital information , 2005, TCHI.

[7]  John R. Josephson,et al.  What Are They? Why Do We Need Them? , 1999 .

[8]  Pelle Ehn,et al.  Work-oriented design of computer artifacts , 1989 .

[9]  John M. Carroll,et al.  Making Use: Scenario-Based Design of Human-Computer Interactions , 2000 .

[10]  Marcus Specht,et al.  A Context-Sensitive Nomadic Exhibition Guide , 2000, HUC.

[11]  Andrew P. Sage,et al.  An overview of automated reasoning , 1990, IEEE Trans. Syst. Man Cybern..

[12]  Susanne Bødker,et al.  Through the Interface: A Human Activity Approach To User Interface Design , 1990 .

[13]  Carlo Strapparava,et al.  Modelling and Adapting to Context , 2001, Personal and Ubiquitous Computing.

[14]  Nancy Proctor,et al.  The State of the Art in Museum Handhelds in 2003. , 2003 .

[15]  Elena Not,et al.  The MacroNode Approach: Mediating Between Adaptive and Dynamic Hypermedia , 2000, AH.

[16]  P. Libby.  The Scientific American , 1881, Nature.

[17]  Elizabeth D. Mynatt,et al.  Designing audio aura , 1998, CHI.

[18]  Kenneth P. Fishkin,et al.  A taxonomy for and analysis of tangible interfaces , 2004, Personal and Ubiquitous Computing.

[19]  Stephen A. Brewster,et al.  Gestural and audio metaphors as a means of control for mobile devices , 2002, CHI.

[20]  Jude W. Shavlik,et al.  Learning users' interests by unobtrusively observing their normal behavior , 2000, IUI '00.

[21]  Allison Woodruff,et al.  Sotto voce: exploring the interplay of conversation and mobile audio spaces , 2002, CHI.

[22]  Wolfgang Wahlster,et al.  A resource-adaptive mobile navigation system , 2002, IUI '02.

[23]  Genevieve Bell,et al.  Designing Technology for Domestic Spaces: A Kitchen Manifesto , 2002 .

[24]  Marek Hatala,et al.  Rules and ontologies in support of real-time ubiquitous application , 2005, J. Web Semant..

[25]  Daniel G. Bobrow,et al.  Dimensions of Interaction , 1991 .

[26]  Lucy A. Suchman,et al.  Plans and Situated Actions: The Problem of Human-Machine Communication (Learning in Doing: Social, , 1987 .

[27]  Staffan Björk,et al.  Designing Ubiquitous Computing Games – A Report from a Workshop Exploring Ubiquitous Computing Entertainment , 2002, Personal and Ubiquitous Computing.

[28]  Henry Lieberman,et al.  Letizia: An Agent That Assists Web Browsing , 1995, IJCAI.

[29]  Marcus Specht,et al.  Location Models for Augmented Environments , 2002, Personal and Ubiquitous Computing.

[30]  Zdenek Zdráhal,et al.  Story fountain: intelligent support for story research and exploration , 2004, IUI '04.

[31]  Hiroshi Ishii,et al.  Emerging frameworks for tangible user interfaces , 2000, IBM Syst. J..

[32]  Alfred Kobsa,et al.  User Models in Dialog Systems , 1989, Symbolic Computation.

[33]  G Stix,et al.  The mice that warred. , 2001, Scientific American.

[34]  Allison Woodruff,et al.  Improving electronic guidebook interfaces using a task-oriented design approach , 2000, DIS '00.

[35]  Gerhard Eckel.  Immersive Audio-Augmented Environments , 2001 .

[36]  Elisabeth André,et al.  Proceedings of the 8th international conference on Intelligent user interfaces , 2002 .

[37]  Mirjana Spasojevic,et al.  From Informing to Remembering: Ubiquitous Systems in Interactive Museums , 2002, IEEE Pervasive Comput..

[38]  Young ae Hahn,et al.  Activity Centered Design: An Ecological Approach to Designing Smart Tools and Usable Systems , 2005 .

[39]  Balakrishnan Chandrasekaran,et al.  What are ontologies, and why do we need them? , 1999, IEEE Intell. Syst..

[40]  Lucia Terrenghi,et al.  Tailored audio augmented environments for museums , 2004, IUI '04.

[41]  Young-Woo Seo,et al.  A reinforcement learning agent for personalized information filtering , 2000, IUI '00.

[42]  Lars Erik Holmquist,et al.  Token-Based Acces to Digital Information , 1999, HUC.

[43]  Marek Hatala,et al.  Interactive Audio Content: An Approach to Audio Content for a Dynamic Museum Experience through Augmented Audio Reality and Adaptive Information Retrieval , 2004 .

[44]  Stephen A. Brewster,et al.  Multimodal 'eyes-free' interaction techniques for wearable devices , 2003, CHI '03.

[45]  Marek Hatala,et al.  Ontology and rule based retrieval of sound objects in augmented audio reality system for museum visitors , 2004, SAC '04.

[46]  Flavia Sparacino,et al.  The Museum Wearable: real-time sensor-driven understanding of visitors' interests for personalized visually-augmented museum experiences , 2002 .

[47]  L. Vygotsky.  Consciousness as a Problem in the Psychology of Behavior , 1979 .

[48]  Bonnie A. Nardi,et al.  Information Ecologies: Using Technology with Heart , 1999 .

[49]  Robin D. Burke,et al.  Hybrid Recommender Systems: Survey and Experiments , 2002, User Modeling and User-Adapted Interaction.

[50]  John M. Carroll,et al.  Human-Computer Interaction in the New Millennium , 2001 .

[51]  Carole D. Hafner,et al.  The State of the Art in Ontology Design: A Survey and Comparative Review , 1997, AI Mag..

[52]  Mark Weiser,et al.  Some computer science issues in ubiquitous computing , 1993, CACM.

[53]  Peter Thomas,et al.  Chapter 3 – The Very Idea: Informing HCI Design from Conversation Analysis , 1990 .

[54]  Lucy Suchman.  Plans and situated actions: the problem of human-machine communication , 1987 .

[55]  Paul Dourish,et al.  What we talk about when we talk about context , 2004, Personal and Ubiquitous Computing.

[56]  Martin Doerr,et al.  The CIDOC Conceptual Reference Model - A New Standard for Knowledge Sharing , 2007, ER.

[57]  Mirjana Spasojevic,et al.  The Electronic Guidebook: Using Portable Devices And A Wireless Web-based Network To Extend The Museum Experience , 2002 .

[58]  David Dean,et al.  Museum Exhibition: Theory and Practice , 2002 .

[59]  Ron Wakkary,et al.  Museum as ecology: A case study analysis of an ambient intelligent museum guide , 2005 .

[60]  William C. Regli,et al.  DAMLJessKB: A Tool for Reasoning with the Semantic Web , 2003, IEEE Intell. Syst..

[61]  Daniel G. Bobrow,et al.  Dimensions of interaction: a shift of perspective in artificial intelligence , 1991 .

[62]  Alain Karsenty,et al.  Unremarkable computing , 2002, CHI.

[63]  Ken Hinckley,et al.  Passive real-world interface props for neurosurgical visualization , 1994, International Conference on Human Factors in Computing Systems.

[64]  J. Kruse.  Book review: User Models in Dialog Systems Edited by A. Kobsa and W. Wahlster (Springer-Verlag, 1989) , 1991, SGAR.

[65]  B. Nardi.  Context and consciousness: activity theory and human-computer interaction , 1995 .

[66]  Massimo Zancanaro,et al.  Navigating by knowledge , 2003, IUI '03.

[67]  G. W. Teuscher.  The hand that rocks the cradle... , 1988, ASDC journal of dentistry for children.

[68]  John Riedl,et al.  Recommender Systems: A GroupLens Perspective , 1998 .

[69]  David Hakken.  Work‐Oriented Design of Computer Artifacts , 1989 .

[70]  Gerhard Fischer,et al.  User Modeling in Human–Computer Interaction , 2001, User Modeling and User-Adapted Interaction.

[71]  Kevin Crowley,et al.  Learning As Conversational Elaboration : A Proposal to Capture , Code , and Analyze Talk in Museums , 2000 .

[72]  Mirjana Spasojevic,et al.  Evaluating the CoolTown User Experience , .

[73]  R. Polynor,et al.  The hand that rocks the cradle , 1995 .

[74]  Hiroshi Ishii,et al.  Tangible bits: towards seamless interfaces between people, bits and atoms , 1997, CHI.

[75]  Elena Not,et al.  HIPS: hyper-interaction within physical space , 1999, Proceedings IEEE International Conference on Multimedia Computing and Systems.

[76]  Beverly Serrell.  The Question of Visitor Styles , 1993 .

[77]  Joseph A. Paradiso,et al.  PingPongPlus: design of an athletic-tangible interface for computer-supported cooperative play , 1999, CHI '99.

[78]  Orit Shaer,et al.  The TAC paradigm: specifying tangible user interfaces , 2004, Personal and Ubiquitous Computing.

[79]  C. Heath,et al.  Exhibiting Interaction: Conduct and Collaboration in Museums and Galleries , 2001 .

[80]  Brygg Ullmer,et al.  Tangible interfaces for manipulating aggregates of digital information , 2002 .

[81]  Scott P. Robertson,et al.  Proceedings of the SIGCHI Conference on Human Factors in Computing Systems , 1991 .

[82]  Brendon Towle,et al.  Knowledge Based Recommender Systems Using Explicit User Models , 2000 .

[83]  Ron Wakkary,et al.  Framing complexity, design and experience: A reflective analysis , 2005, Digit. Creativity.

[84]  Robin Burke,et al.  Knowledge-based recommender systems , 2000 .

[85]  Paul Resnick,et al.  Recommender systems , 1997, CACM.

[86]  Caroline Hummels,et al.  ISH and the search for resonant tangible interaction , 2004, Personal and Ubiquitous Computing.

[87]  Paul Dourish,et al.  Where the action is , 2001 .

[88]  Hiroshi Ishii,et al.  Bricks: laying the foundations for graspable user interfaces , 1995, CHI '95.

[89]  James A. Hendler,et al.  The Semantic Web" in Scientific American , 2001 .

[90]  James P. Carse,et al.  Finite and Infinite Games , 1986 .