The Invisible Museum: A User-Centric Platform for Creating Virtual 3D Exhibitions with VR Support

With the ever-advancing availability of digitized museum artifacts, the question of how to make the vast collection of exhibits accessible and explorable beyond what museums traditionally offer via their websites and exposed databases has recently gained increased attention. This research work introduces the Invisible Museum: a user-centric platform that allows users to create interactive and immersive virtual 3D/VR exhibitions using a unified collaborative authoring environment. The platform itself was designed following a Human-Centered Design approach, with the active participation of museum curators and end-users. Content representation adheres to domain standards such as International Committee for Documentation of the International Council of Museums (CIDOC-CRM) and the Europeana Data Model and exploits state-of-the-art deep learning technologies to assist the curators by generating ontology bindings for textual data. The platform enables the formulation and semantic representation of narratives that guide storytelling experiences and bind the presented artifacts with their socio-historic context. Main contributions are pertinent to the fields of (a) user-designed dynamic virtual exhibitions, (b) personalized suggestions and exhibition tours, (c) visualization in web-based 3D/VR technologies, and (d) immersive navigation and interaction. The Invisible Museum has been evaluated using a combination of different methodologies, ensuring the delivery of a high-quality user experience, leading to valuable lessons learned, which are discussed in the article.

[1]  Vincenzo Moscato,et al.  An Emotional Recommender System for Music , 2021, IEEE Intelligent Systems.

[2]  Ayako Hashizume.  JIS Z 8530 Human-Centred Design for Interactive Systems , 2021 .

[3]  X. Zabulis,et al.  Representation and Preservation of Heritage Crafts , 2020, Sustainability.

[4]  Sardar Jaf,et al.  Deep Learning for Natural Language Parsing , 2019, IEEE Access.

[5]  Heiko Schuldt,et al.  VIRTUE: a virtual reality museum Experience , 2019, IUI Companion.

[6]  Ivan Titov,et al.  AMR Parsing as Graph Prediction with Latent Alignment , 2018, ACL.

[7]  Tara N. Sainath,et al.  State-of-the-Art Speech Recognition with Sequence-to-Sequence Models , 2017, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[8]  Diego Reforgiato Recupero,et al.  Semantic Web Machine Reading with FRED , 2017, Semantic Web.

[9]  Emmanuel Salami,et al.  An Analysis of the General Data Protection Regulation (EU) 2016/679 , 2017 .

[10]  Flora Amato,et al.  Recommendation in Social Media Networks , 2017, 2017 IEEE Third International Conference on Multimedia Big Data (BigMM).

[11]  Thomas P. Kersten,et al.  Development of a Virtual Museum Including a 4d Presentation of Building History in Virtual Reality , 2017 .

[12]  D. Tzovaras,et al.  The Scan4Reco Virtual Museum , 2017 .

[13]  Nadia Magnenat-Thalmann,et al.  Mixed Reality and Gamification for Cultural Heritage , 2017 .

[14]  Constantine Stephanidis,et al.  Digital Cultural Heritage Experience in Ambient Intelligence , 2017, Mixed Reality and Gamification for Cultural Heritage.

[15]  Chairi Kiourt,et al.  DynaMus: A fully dynamic 3D virtual museum framework , 2016 .

[16]  Constantine Stephanidis,et al.  Accessibility of Cultural Heritage Exhibits , 2016, HCI.

[17]  Jaime G. Carbonell,et al.  CMU at SemEval-2016 Task 8: Graph-based AMR Parsing with Infinite Ramp Loss , 2016, *SEMEVAL.

[18]  Constantine Stephanidis,et al.  Adaptation and Content Personalization in the Context of Multi User Museum Exhibits , 2016, AVI*CH.

[19]  Joseph Giacomin,et al.  What Is Human Centred Design? , 2014 .

[20]  Inderjeet Mani,et al.  Computational Modeling of Narrative , 2013, Computational Modeling of Narrative.

[21]  Eneko Agirre,et al.  PATHS: Personalising access to cultural heritage spaces , 2012, 2012 18th International Conference on Virtual Systems and Multimedia.

[22]  Paul Mulholland,et al.  Storyspace: a story-driven approach for creating museum narratives , 2012, HT '12.

[23]  Lora Aroyo,et al.  Understanding Objects in Online Museum Collections by Means of Narratives , 2012 .

[24]  Pierre Nugues,et al.  Using Semantic Role Labeling to Extract Events from Wikipedia , 2011, DeRiVE@ISWC.

[25]  Valentine Charles,et al.  The Europeana Data Model (EDM) , 2010 .

[26]  Massimo Bergamasco,et al.  Beyond virtual museums: experiencing immersive virtual reality in real museums , 2010 .

[27]  Eun-Kyung Kim,et al.  Intrinsic Property-based Taxonomic Relation Extraction from Category Structure , 2010 .

[28]  Christophe Kolski,et al.  State of the Art on the Cognitive Walkthrough Method, Its Variants and Evolutions , 2010, Int. J. Hum. Comput. Interact..

[29]  Klaus Pohl,et al.  Requirements Engineering - Fundamentals, Principles, and Techniques , 2010 .

[30]  Niall O'hOisin,et al.  The Medieval Dublin Project: A Case Study , 2010 .

[31]  Steffen Staab,et al.  F--a model of events based on the foundational ontology dolce+DnS ultralight , 2009, K-CAP '09.

[32]  Raphaël Troncy,et al.  LODE: Linking Open Descriptions of Events , 2009, ASWC.

[33]  Marcelo Milrad,et al.  Integrating Co-design Practices into the Development of Mobile Science Collaboratories , 2009, 2009 Ninth IEEE International Conference on Advanced Learning Technologies.

[34]  Asbjørn Følstad.  The effect of group discussions in usability inspection: a pilot study , 2008, NordiCHI.

[35]  Pieter Jan Stappers,et al.  Co-creation and the new landscapes of design , 2008 .

[36]  Anna-Lisa Osvalder,et al.  An Analytical Approach for Predicting and Identifying Use Error and Usability Problem , 2007, USAB.

[37]  Mark Billinghurst,et al.  OSGART - A Pragmatic Approach to MR , 2006 .

[38]  Charles E. Hughes,et al.  Mixed reality in education, entertainment, and training , 2005, IEEE Computer Graphics and Applications.

[39]  C. Prahalad,et al.  Co-creation experiences: The next practice in value creation , 2004 .

[40]  Paul H. Lewis,et al.  Knowledge-Based Exploration of Multimedia Museum Collections , 2004, EWIMT.

[41]  Martin Doerr,et al.  The CIDOC Conceptual Reference Module: An Ontological Approach to Semantic Interoperability of Metadata , 2003, AI Mag..

[42]  Dan I. Moldovan,et al.  Learning Semantic Constraints for the Automatic Discovery of Part-Whole Relations , 2003, NAACL.

[43]  David E. Millard,et al.  Augmented reality as an interface to adaptive hypermedia systems , 2003, New Rev. Hypermedia Multim..

[44]  Daniel Jurafsky,et al.  Automatic Labeling of Semantic Roles , 2002, CL.

[45]  Kenneth W. Boyer,et al.  Advanced use case modeling: software systems , 2002, SOEN.

[46]  Jane Hunter,et al.  The ABC Ontology and Model , 2001, J. Digit. Inf..

[47]  Werner Schweibenz,et al.  The "Virtual Museum": New Perspectives For Museums to Present Objects and Information Using the Internet as a Knowledge Base and Communication System , 1998, ISI.

[48]  M. Forte,et al.  Virtual Archaeology: Great Discoveries Brought to Life Through Virtual Reality , 1997 .

[49]  Jakob Nielsen,et al.  Heuristic Evaluation of Prototypes (individual) , 2022 .