Real-Time Analytics

Real-time analytics is a special kind of Big Data analytics in which data elements are required to be processed and analyzed as they arrive in real time. It is important in situations where real-time processing and analysis can deliver important insights and yield business value. This chapter provides an overview of current processing and analytics platforms needed to support such analysis, as well as analytics techniques that can be applied in such environments. The chapter looks beyond traditional event processing system technology to consider a broader big data context that involves “data at rest” platforms and solutions. The chapter includes a case study showing the use of EventSwarm complex event processing engine for a class of analytics problems in finance. The chapter concludes with several research challenges, such as the need for new approaches and algorithms required to support real-time data filtering, data exploration, statistical data analysis, and machine learning.

[1]  Opher Etzion,et al.  Event Processing in Action , 2010 .

[2]  Weisi Chen,et al.  Enabling user-driven rule management in event data analysis , 2016, Inf. Syst. Frontiers.

[3]  John J. Binder.  The Event Study Methodology Since 1969 , 1997 .

[4]  Andrew Berry,et al.  Real-Time Analytics for Legacy Data Streams in Health: Monitoring Health Data Quality , 2013, 2013 17th IEEE International Enterprise Distributed Object Computing Conference.

[5]  Opher Etzion,et al.  The Situation Manager Component of Amit - Active Middleware Technology , 2002, NGITS.

[6]  Fethi A. Rabhi,et al.  An Event-Based Model to Support Distributed Real-Time Analytics: Finance Case Study , 2015, 2015 IEEE 19th International Enterprise Distributed Object Computing Conference.

[7]  Sandy Ryza.  Advanced Analytics with Spark , 2015 .

[8]  Giuseppe Psaila,et al.  Composite Events in Chimera , 1996, EDBT.

[9]  Sharma Chakravarthy,et al.  Snoop: An Expressive Event Specification Language for Active Databases , 1994, Data Knowl. Eng..

[10]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[11]  Bruce Powell Douglass,et al.  Real-time UML (2nd ed.): developing efficient objects for embedded systems , 1997 .

[12]  Paolo Mancarella,et al.  Production Systems with Negation as Failure , 2002, IEEE Trans. Knowl. Data Eng..

[13]  Calton Pu,et al.  Continual Queries for Internet Scale Event-Driven Information Delivery , 1999, IEEE Trans. Knowl. Data Eng..

[14]  T. Andersen.  Some Reflections on Analysis of High-Frequency Data , 2000 .

[15]  Adrian Paschke,et al.  Rule-Based Event Processing and Reaction Rules , 2009, RuleML.

[16]  Jennifer Widom,et al.  The CQL continuous query language: semantic foundations and query execution , 2006, The VLDB Journal.

[17]  Robert Kajic.  Evaluation of the Stream Query Language CQL , 2010 .

[18]  Tim Bollerslev,et al.  Risk, Jumps, and Diversification , 2007 .

[19]  Byron Ellis.  Real-Time Analytics: Techniques to Analyze and Visualize Streaming Data , 2014 .

[20]  Linda L. Briggs.  Documentation for users , 1991 .

[21]  Jennifer Widom,et al.  Towards a streaming SQL standard , 2008, Proc. VLDB Endow..

[22]  Tom White,et al.  Hadoop: The Definitive Guide , 2009 .

[23]  C. Goodhart,et al.  High frequency data in financial markets: Issues and applications , 1997 .

[24]  David Luckham,et al.  The power of events - an introduction to complex event processing in distributed enterprise systems , 2002, RuleML.

[25]  David J. DeWitt,et al.  NiagaraCQ: a scalable continuous query system for Internet databases , 2000, SIGMOD 2000.

[26]  Jean Bacon,et al.  Generic Support for Distributed Applications , 2000, Computer.

[27]  Klaus R. Dittrich,et al.  The active database management system manifesto: a rulebase of ADBMS features , 1995, SGMD.

[28]  D. Luckham.  Event Processing for Business: Organizing the Real-Time Enterprise , 2011 .