Analysis Cloud - Running Sensor Data Analysis Programs on a Cloud Computing Infrastructure

Sensors have been used for many years to gather information about their environment. The number of sensors connected to the internet is increasing, which has led to a growing demand of data transport and storage capacity. In addition, evermore emphasis is put on processing the data to detect anomalous situations and to identify trends. This paper presents a sensor data analysis platform that executes statistical analysis programs on a cloud computing infrastructure. Compared to existing batch and stream processing platforms, it adds the notion of simulated time, i.e. time that differs from the actual, current time. Moreover, it adds the ability to dynamically schedule the analysis programs based on a single timestamp, recurring schedule, or on the sensor data itself.

[1]  Scott Shenker,et al.  Spark: Cluster Computing with Working Sets , 2010, HotCloud.

[2]  Tom White,et al.  Hadoop: The Definitive Guide , 2009 .

[3]  Leonardo Neumeyer,et al.  S4: Distributed Stream Computing Platform , 2010, 2010 IEEE International Conference on Data Mining Workshops.

[4]  Bo Sheng,et al.  Data storage placement in sensor networks , 2006, MobiHoc '06.

[5]  Petr Jan Horn,et al.  Autonomic Computing: IBM's Perspective on the State of Information Technology , 2001 .

[6]  Valeria V. Krzhizhanovskaya,et al.  Machine learning methods for environmental monitoring and flood protection , 2011 .

[7]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[8]  Kun Wang,et al.  A Distributed Self-Learning Approach for Elastic Provisioning of Virtualized Cloud Resources , 2011, 2011 IEEE 19th Annual International Symposium on Modelling, Analysis, and Simulation of Computer and Telecommunication Systems.

[9]  GhemawatSanjay,et al.  The Google file system , 2003 .

[10]  Mahadev Konar,et al.  ZooKeeper: Wait-free Coordination for Internet-scale Systems , 2010, USENIX ATC.

[11]  Howard Gobioff,et al.  The Google file system , 2003, SOSP '03.

[12]  R. Fielding,et al.  Architectural Styles and the Design of Network-based Software Architectures (CHAPTER 5) , 2000 .

[13]  William J. Blackwell,et al.  A neural-network technique for the retrieval of atmospheric temperature and moisture profiles from high spectral resolution sounding data , 2005, IEEE Transactions on Geoscience and Remote Sensing.