Automatic 3D Modeling and Reconstruction of Cultural Heritage Sites from Twitter Images

This paper presents an approach for leveraging the abundance of images posted on social media like Twitter for large scale 3D reconstruction of cultural heritage landmarks. Twitter allows users to post short messages, including photos, describing a plethora of activities or events, e.g., tweets are used by travelers on vacation, capturing images from various cultural heritage assets. As such, a great number of images are available online, able to drive a successful 3D reconstruction process. However, reconstruction of any asset, based on images mined from Twitter, presents several challenges. There are three main steps that have to be considered: (i) tweets’ content identification, (ii) image retrieval and filtering, and (iii) 3D reconstruction. The proposed approach first extracts key events from unstructured tweet messages and then identifies cultural activities and landmarks. The second stage is the application of a content-based filtering method so that only a small but representative portion of cultural images are selected to support fast 3D reconstruction. The proposed methods are experimentally evaluated using real-world data and comparisons verify the effectiveness of the proposed scheme.

[1]  Hae-Chang Rim,et al.  Identifying interesting Twitter contents using topical analysis , 2014, Expert Syst. Appl..

[2]  K. Fan,et al.  Maximum Properties and Inequalities for the Eigenvalues of Completely Continuous Operators. , 1951, Proceedings of the National Academy of Sciences of the United States of America.

[3]  Charalampos Z. Patrikakis,et al.  Employing clustering algorithms to create user groups for personalized context aware services provision , 2011, SBNMA '11.

[4]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[5]  Thomas S. Huang,et al.  Motion and structure from feature correspondences: a review , 1994, Proc. IEEE.

[6]  Sylvia Kierkegaard,et al.  Twitter thou doeth? , 2010, Comput. Law Secur. Rev..

[7]  Yiannis Kompatsiaris,et al.  Sensing Trending Topics in Twitter , 2013, IEEE Transactions on Multimedia.

[8]  Pouria Sadeghi-Tehran,et al.  Look‐a‐Like: A Fast Content‐Based Image Retrieval Approach Using a Hierarchically Nested Dynamically Evolving Image Clouds and Recursive Local Data Density , 2017, Int. J. Intell. Syst..

[9]  Nikolaos Doulamis,et al.  4D Modelling in Cultural Heritage , 2017, DCH@ICE/ITMC.

[10]  Kyriacos Themistocleous,et al.  Model reconstruction for 3d vizualization of cultural heritage sites using open data from social media: The case study of Soli, Cyprus , 2017 .

[11]  Prashant Srivastava,et al.  Integration of wavelet transform, Local Binary Patterns and moments for content-based image retrieval , 2017, J. Vis. Commun. Image Represent..

[12]  Emmanouel A. Varvarigos,et al.  Event Detection in Twitter Microblogging , 2016, IEEE Transactions on Cybernetics.

[13]  Keiji Yanai,et al.  Event photo mining from Twitter using keyword bursts and image clustering , 2016, Neurocomputing.

[14]  Heng-Da Cheng,et al.  Effective image retrieval using dominant color descriptor and fuzzy support vector machine , 2009, Pattern Recognit..

[15]  Aixin Sun,et al.  Query-Guided Event Detection From News and Blog Streams , 2011, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[16]  Marinos Ioannides,et al.  In the wild image retrieval and clustering for 3D cultural heritage landmarks reconstruction , 2014, Multimedia Tools and Applications.

[17]  C. Paige.  Computational variants of the Lanczos method for the eigenproblem , 1972 .

[18]  Karin Ibenholt,et al.  Economic impacts of cultural heritage – Research and perspectives , 2009 .

[19]  Derya Birant,et al.  ST-DBSCAN: An algorithm for clustering spatial-temporal data , 2007, Data Knowl. Eng..

[20]  Stuart E. Middleton,et al.  Vision-based production of personalized video , 2009, Signal Process. Image Commun..

[21]  Georgios Miaoulis,et al.  Dance Pose Identification from Motion Capture Data: A Comparison of Classifiers , 2018 .

[22]  Nikolaos Doulamis,et al.  Four-dimensional reconstruction of cultural heritage sites based on photogrammetry and clustering , 2017, J. Electronic Imaging.

[23]  Wenjie Li,et al.  Sequential Summarization: A Full View of Twitter Trending Topics , 2014, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[24]  Abdolreza Abhari,et al.  Cluster-discovery of Twitter messages for event detection and trending , 2015, J. Comput. Sci..

[25]  Anastasios Doulamis,et al.  Semi-Supervised Image Meta-Filtering Using Relevance Feedback in Cultural Heritage Applications , 2014 .

[26]  Lei Chen,et al.  Event detection over twitter social media streams , 2013, The VLDB Journal.

[27]  Emmanouel A. Varvarigos,et al.  Resource Selection for Tasks with Time Requirements Using Spectral Clustering , 2014, IEEE Transactions on Computers.

[28]  Zhi-Hua Zhou,et al.  Distributional features for text categorization , 2006 .

[29]  Younghoon Kim,et al.  TWILITE: A recommendation system for Twitter using a probabilistic model based on latent Dirichlet allocation , 2014, Inf. Syst..

[30]  George Kalliris,et al.  Audiovisual production, restoration-archiving and content management methods to preserve local tradition and folkloric heritage , 2014 .

[31]  Dimitrios Skoutas,et al.  MI-MERCURY: A mobile agent architecture for ubiquitous retrieval and delivery of multimedia information , 2007, Multimedia Tools and Applications.