A Reference Architecture and Model for Sensor Data Warehousing

Sensor data are becoming far more available thanks to the growth in both sensor systems and Internet of Things devices. Much of the value of sensor data comes from examining trends that occur over long timescales, ranging from hours to years. However, making use of data a long time after it has been collected has significant implications for the data-handling systems used to manage it. In particular, the data must be contextualized into the environment in which it was collected to avoid misleading (and potentially dangerous) misinterpretation. We apply data warehousing techniques to develop an extensible model to capture contextual metadata alongside sensor datasets and show how this can be used to support the analysis of datasets long after collection. We present our baseline reference framework for sensor context and derive multidimensional schemata representing different modeling and analysis scenarios. Finally, we exercise the model with two case studies.

[1]  Thanos G. Stavropoulos,et al.  BOnSAI: a smart building ontology for ambient intelligence , 2012, WIMS '12.

[2]  Lei Shu,et al.  Cloud-based Data-intensive Framework towards fault diagnosis in large-scale petrochemical plants , 2016, 2016 International Wireless Communications and Mobile Computing Conference (IWCMC).

[3]  Huang Fang Managing data lakes in big data era: What's a data lake and why has it became popular in data management ecosystem , 2015, 2015 IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems (CYBER).

[4]  Mark J. Warshawsky,et al.  A Modern Approach , 2005 .

[5]  Franco Cicirelli,et al.  Metamodeling of Smart Environments: from design to implementation , 2017, Adv. Eng. Informatics.

[6]  David Garlan,et al.  Context is key , 2005, CACM.

[7]  Simon A. Dobson,et al.  Towards Data-centric Control of Sensor Networks through Bayesian Dynamic Linear Modelling , 2015, 2015 IEEE 9th International Conference on Self-Adaptive and Self-Organizing Systems.

[8]  Martin Bauer,et al.  Proceedings of the Federated Conference on Computer Science and Information Systems pp. 949–955 ISBN 978-83-60810-22-4 Service Modelling for the Internet of Things , 2022 .

[9]  Hairong Kuang,et al.  The Hadoop Distributed File System , 2010, 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST).

[10]  Michael Scriney,et al.  Generating cubes from smart city web data , 2017, ACSW.

[11]  Martin Fowler,et al.  NoSQL Distilled: A Brief Guide to the Emerging World of Polyglot Persistence , 2012 .

[12]  Yvan Bédard,et al.  SOLAP technology: Merging business intelligence with geospatial technology for interactive spatio-temporal exploration and analysis of data , 2005 .

[13]  Umar Farooq Minhas,et al.  SQL-on-Hadoop: Full Circle Back to Shared-Nothing Database Architectures , 2014, Proc. VLDB Endow..

[14]  Carlos Eduardo Cugnasca,et al.  Insights Into IoT Data and an Innovative DWT-Based Technique to Denoise Sensor Signals , 2018, IEEE Sensors Journal.

[15]  Jerry R. Hobbs,et al.  DAML-S: Semantic Markup for Web Services , 2001, SWWS.

[16]  Fang Ming,et al.  Research on multidimensional analysis method of drilling information based on Hadoop , 2017, 2017 3rd IEEE International Conference on Computer and Communications (ICCC).

[17]  Marco Chiani,et al.  A Robust Wireless Sensor Network for Landslide Risk Analysis: System Design, Deployment, and Field Testing , 2016, IEEE Sensors Journal.

[18]  Simon A. Dobson,et al.  Discovery and recognition of unknown activities , 2016, UbiComp Adjunct.

[19]  Matteo Golfarelli,et al.  Conceptual design of data warehouses from E/R schemes , 1998, Proceedings of the Thirty-First Hawaii International Conference on System Sciences.

[20]  Matteo Golfarelli,et al.  Starry Vault: Automating Multidimensional Modeling from Data Vaults , 2016, ADBIS.

[21]  Reynold Xin,et al.  Apache Spark , 2016 .

[22]  Matteo Golfarelli,et al.  Data Warehouse Design: Modern Principles and Methodologies , 2009 .