Automatic Museum Audio Guide

An automatic “museum audio guide” is presented as a new type of audio guide for museums. The device consists of a headset equipped with a camera that captures exhibit pictures and the eyes of things computer vision device (EoT). The EoT board is capable of recognizing artworks using features from accelerated segment test (FAST) keypoints and a random forest classifier, and is able to be used for an entire day without the need to recharge the batteries. In addition, an application logic has been implemented, which allows for a special highly-efficient behavior upon recognition of the painting. Two different use case scenarios have been implemented. The main testing was performed with a piloting phase in a real world museum. Results show that the system keeps its promises regarding its main benefit, which is simplicity of use and the user’s preference of the proposed system over traditional audioguides.

[1]  Alberto Del Bimbo,et al.  Deep Artwork Detection and Retrieval for Automatic Context-Aware Audio Guides , 2017, ACM Trans. Multim. Comput. Commun. Appl..

[2]  Jamal N. Al-Karaki,et al.  Wireless Multimedia Sensor Networks: Current Trends and Future Directions , 2010, Sensors.

[3]  Ian F. Akyildiz,et al.  A survey on wireless multimedia sensor networks , 2007, Comput. Networks.

[4]  Donald G. Bailey,et al.  Design for Embedded Image Processing on FPGAs , 2011 .

[5]  Sufen Fong,et al.  MeshEye: A Hybrid-Resolution Smart Camera Mote for Applications in Distributed Intelligent Surveillance , 2007, 2007 6th International Symposium on Information Processing in Sensor Networks.

[6]  Jan O. Borchers,et al.  Maps and Location: Acceptance of Modern Interaction Techniques for Audio Guides , 2016, CHI.

[7]  Tom Drummond,et al.  Faster and Better: A Machine Learning Approach to Corner Detection , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Steve Benford,et al.  Deepening Visitor Engagement with Museum Exhibits through Hand-crafted Visual Markers , 2018, Conference on Designing Interactive Systems.

[9]  José Luis Espinosa-Aranda,et al.  Smart Doll: Emotion Recognition Using Embedded Deep Learning , 2018, Symmetry.

[10]  Allen Y. Yang,et al.  CITRIC: A low-bandwidth wireless camera network platform , 2008, 2008 Second ACM/IEEE International Conference on Distributed Smart Cameras.

[11]  J. Bier.  Implementing Vision Capabilities in Embedded Sysstems , 2011 .

[12]  Joo-Hwee Lim,et al.  Scene Recognition with Camera Phones for Tourist Information Access , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[13]  Luca Greci,et al.  An Augmented Reality Guide for Religious Museum , 2016, AVR.

[14]  Stanislav Kovacic,et al.  Towards commoditized smart-camera design , 2013, J. Syst. Archit..

[15]  Richard P. Kleihorst,et al.  Camera Mote with a High-Performance Parallel Processor for Real-Time Frame-Based Video Processing , 2007, 2007 First ACM/IEEE International Conference on Distributed Smart Cameras.

[16]  Vincent Lepetit,et al.  Point matching as a classification problem for fast and robust object pose estimation , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[17]  José Luis Espinosa-Aranda,et al.  Pulga, a tiny open-source MQTT broker for flexible and secure IoT deployments , 2015, 2015 IEEE Conference on Communications and Network Security (CNS).

[18]  Leonard Wein,et al.  Visual recognition in museum guide apps: do visitors want it? , 2014, CHI.

[19]  Keiji Yanai,et al.  Efficient Mobile Implementation of A CNN-based Object Recognition System , 2016, ACM Multimedia.

[20]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[21]  Jaime Lloret,et al.  A secure NFC application for credit transfer among mobile phones , 2012, 2012 International Conference on Computer, Information and Telecommunication Systems (CITS).

[22]  Roberto Manduchi,et al.  A Power-Aware, Self-Managing Wireless Camera Network for, Wide Area Monitoring , 2006 .

[23]  Wendi B. Heinzelman,et al.  A Survey of Visual Sensor Networks , 2009, Adv. Multim..

[24]  Shaharyar Ahmed Khan Tareen,et al.  A comparative analysis of SIFT, SURF, KAZE, AKAZE, ORB, and BRISK , 2018, 2018 International Conference on Computing, Mathematics and Engineering Technologies (iCoMET).

[25]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.

[26]  Sule Serubugo,et al.  Comparison of Wearable Optical See-through and Handheld Devices as Platform for an Augmented Reality Museum Guide , 2017, VISIGRAPP.

[27]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[28]  Dmitry P. Nikolaev,et al.  2D art recognition in uncontrolled conditions using one-shot learning , 2019, International Conference on Machine Vision.

[29]  Heedong Ko,et al.  Development and Evaluation of Mobile Tour Guide Using Wearable and Hand-Held Devices , 2016, HCI.

[30]  Zhan Ma,et al.  Smart Cameras , 2020, Encyclopedia of Biometrics.

[31]  Mark Billinghurst,et al.  Exploring the use of handheld AR for outdoor navigation , 2012, Comput. Graph..

[32]  Serge J. Belongie,et al.  Recognizing locations with Google Glass: A case study , 2014, IEEE Winter Conference on Applications of Computer Vision.

[33]  Robert P. Sheridan,et al.  Random Forest: A Classification and Regression Tool for Compound Classification and QSAR Modeling , 2003, J. Chem. Inf. Comput. Sci..

[34]  Theofilos Chrysikos,et al.  CHISTA: Cultural Heritage Information Storage and reTrieval Application , 2018, EuroMed.

[35]  Gregory D. Abowd,et al.  Cyberguide: A mobile context‐aware tour guide , 1997, Wirel. Networks.

[36]  Damianos Gavalas,et al.  Electronic mobile guides: a survey , 2010, Personal and Ubiquitous Computing.

[37]  Thomas Kunz,et al.  Wireless Multimedia Sensor Networks Testbeds and State-of-the-Art Hardware: A Survey , 2011, FGIT-FGCN.

[38]  A. Dunser,et al.  CityViewAR: A mobile outdoor AR application for city visualization , 2012, 2012 IEEE International Symposium on Mixed and Augmented Reality - Arts, Media, and Humanities (ISMAR-AMH).

[39]  Peter Schelkens,et al.  The mobile museum guide: Artwork recognition with eigenpaintings and SURF , 2011, WIAMIS 2011.

[40]  Michele Magno,et al.  A low-power wireless video sensor node for distributed object detection , 2007, Journal of Real-Time Image Processing.

[41]  Bülent Tavli,et al.  A survey of visual sensor network platforms , 2012, Multimedia Tools and Applications.

[42]  Mladen Russo,et al.  Soundscape of an Archaeological Site Recreated with Audio Augmented Reality , 2018, ACM Trans. Multim. Comput. Commun. Appl..

[43]  Luc Van Gool,et al.  Interactive Museum Guide: Accurate Retrieval of Object Descriptions , 2006, Adaptive Multimedia Retrieval.

[44]  Effrosini Kokiopoulou,et al.  Mobile Museum Guide Based on Fast SIFT Recognition , 2008, Adaptive Multimedia Retrieval.

[45]  Tiberio Uricchio,et al.  Wearable systems for improving tourist experience , 2019, Multimodal Behavior Analysis in the Wild.

[46]  Charlotte Magnusson,et al.  A real-world study of an audio-tactile tourist guide , 2012, Mobile HCI.

[47]  Aubrey K. Dunne,et al.  Eyes of Things , 2017, 2017 IEEE International Conference on Cloud Engineering (IC2E).

[48]  Kuldip K. Paliwal,et al.  Fast features for face authentication under illumination direction changes , 2003, Pattern Recognit. Lett..

[49]  Ricardo Carmona-Galán,et al.  Vision-enabled WSN Nodes: State of the Art , 2012 .

[50]  Claire O'Malley,et al.  Treasure codes: augmenting learning from physical museum exhibits through treasure hunting , 2018, Personal and Ubiquitous Computing.

[51]  Shuvra S. Bhattacharyya,et al.  Embedded Computer Vision , 2008 .

[52]  Wu-chi Feng,et al.  Panoptes: A Scalable Architecture for Video Sensor Networking Applications , 2004 .

[53]  Ricardo Carmona-Galán,et al.  Low-Power Smart Imagers for Vision-Enabled Sensor Networks , 2012 .

[54]  Andrew Zisserman,et al.  Image Classification using Random Forests and Ferns , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[55]  Peter Fröhlich,et al.  KIBITZER: a wearable system for eye-gaze-based mobile urban exploration , 2010, AH.