Exploring Intuitive Lifelog Retrieval and Interaction Modes in Virtual Reality with vitrivr-VR

The multimodal nature of lifelog data collections poses unique challenges for multimedia management and retrieval systems. The Lifelog Search Challenge (LSC) offers an annual evaluation platform for such interactive retrieval systems. They compete against one another in finding items of interest within a set time frame. In this paper, we present the multimedia retrieval system vitrivr-VR, the latest addition to the vitrivr stack, which participated in the LSC in recent years. vitrivr-VR leverages the 3D space in virtual reality (VR) to offer novel retrieval and user interaction models, which we describe with a special focus on design decisions taken for the participation in the LSC.

[1]  Heiko Schuldt,et al.  On the User-Centric Comparative Remote Evaluation of Interactive Video Search Systems , 2021, IEEE MultiMedia.

[2]  Minh-Triet Tran,et al.  Introduction to the Fourth Annual Lifelog Search Challenge, LSC'21 , 2021, ICMR.

[3]  H. Schuldt,et al.  Interactive Multimodal Lifelog Retrieval with vitrivr at LSC 2021 , 2021, LSC@ICMR.

[4]  Heiko Schuldt,et al.  3D Model Retrieval Using Constructive Solid Geometry in Virtual Reality , 2020, 2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR).

[5]  Heiko Schuldt,et al.  Cottontail DB: An Open Source Database System for Multimedia Retrieval and Analysis , 2020, ACM Multimedia.

[6]  Xirong Li,et al.  A W2VV++ Case Study with Automated and Interactive Text-to-Video Retrieval , 2020, ACM Multimedia.

[7]  Heiko Schuldt,et al.  Multi-Stage Queries and Temporal Scoring in Vitrivr , 2020, 2020 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).

[8]  Cathal Gurrin,et al.  VRLE: Lifelog Interaction Prototype in Virtual Reality: Lifelog Search Challenge at ACM ICMR 2020 , 2020, LSC@ICMR.

[9]  Heiko Schuldt,et al.  Interactive Lifelog Retrieval with vitrivr , 2020, LSC@ICMR.

[10]  Minh-Triet Tran,et al.  Introduction to the Third Annual Lifelog Search Challenge (LSC'20) , 2020, ICMR.

[11]  Luca Rossetto,et al.  Interactive Video Retrieval in the Age of Deep Learning – Detailed Evaluation of VBS 2019 , 2020, IEEE Transactions on Multimedia.

[12]  Cathal Gurrin,et al.  Baseline Analysis of a Conventional and Virtual Reality Lifelog Retrieval System , 2020, MMM.

[13]  Heiko Schuldt,et al.  Combining Boolean and Multimedia Retrieval in vitrivr for Large-Scale Video Search , 2019, MMM.

[14]  Heiko Schuldt,et al.  Multimodal Multimedia Retrieval with vitrivr , 2019, ICMR.

[15]  Heiko Schuldt,et al.  Retrieval of Structured and Unstructured Data with vitrivr , 2019, LSC@ICMR.

[16]  Ralph Gasser,et al.  Interactive Search or Sequential Browsing? A Detailed Analysis of the Video Browser Showdown 2018 , 2019, ACM Trans. Multim. Comput. Commun. Appl..

[17]  Heiko Schuldt,et al.  Deep Learning-Based Concept Detection in vitrivr , 2018, MMM.

[18]  Cathal Gurrin,et al.  User Interaction for Visual Lifelog Retrieval in a Virtual Environment , 2018, MMM.

[19]  Cathal Gurrin,et al.  Virtual Reality Lifelog Explorer: Lifelog Search Challenge at ACM ICMR 2018 , 2018, LSC@ICMR.

[20]  Cathal Gurrin,et al.  Geospatial Access to Lifelogging Photos in Virtual Reality , 2018, LSC@ICMR.

[21]  Heiko Schuldt,et al.  vitrivr: A Flexible Retrieval Stack Supporting Multiple Query Modes for Searching in Multimedia Collections , 2016, ACM Multimedia.

[22]  Heiko Schuldt,et al.  Searching in Video Collections Using Sketches and Sample Images - The Cineast System , 2016, MMM.

[23]  Heiko Schuldt,et al.  Cineast: A Multi-feature Sketch-Based Video Retrieval Engine , 2014, 2014 IEEE International Symposium on Multimedia.

[24]  Alan F. Smeaton,et al.  LifeLogging: Personal Big Data , 2014, Found. Trends Inf. Retr..

[25]  Klaus Schöffmann,et al.  Video Browsing with a 3D Thumbnail Ring Arranged by Color Similarity , 2012, MMM.

[26]  Ralph Gasser,et al.  Towards Explainable Interactive Multi-modal Video Retrieval with Vitrivr , 2021, MMM.

[27]  Ralph Gasser,et al.  Competitive Interactive Video Retrieval in Virtual Reality with vitrivr-VR , 2021, MMM.

[28]  Ralph Gasser,et al.  A System for Interactive Multimedia Retrieval Evaluations , 2021, MMM.

[29]  Heiko Schuldt,et al.  Vitrivr-Explore: Guided Multimedia Collection Exploration for Ad-hoc Video Search , 2020, SISAP.