Live Video Analytics at Scale with Approximation and Delay-Tolerance

Video cameras are pervasively deployed for security and smart city scenarios, with millions of them in large cities worldwide. Achieving the potential of these cameras requires efficiently analyzing the live videos in real-time. We describe VideoStorm, a video analytics system that processes thousands of video analytics queries on live video streams over large clusters. Given the high costs of vision processing, resource management is crucial. We consider two key characteristics of video analytics: resource-quality tradeoff with multi-dimensional configurations, and variety in quality and lag goals. VideoStorm's offline profiler generates query resource-quality profile, while its online scheduler allocates resources to queries to maximize performance on quality and lag, in contrast to the commonly used fair sharing of resources in clusters. Deployment on an Azure cluster of 101 machines shows improvement by as much as 80% in quality of real-world queries and 7× better lag, processing video from operational traffic cameras.

[1]  G. Dantzig.  Discrete-Variable Extremum Problems , 1957 .

[2]  Jeffrey M. Jaffe,et al.  Bottleneck Flow Control , 1981, IEEE Trans. Commun..

[3]  B. Ratchford.  Cost-Benefit Models for Explaining Consumer Choice and Information Seeking Behavior , 1982 .

[4]  R. C. Merton,et al.  Continuous-Time Finance , 1990 .

[5]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[6]  Helen J. Wang,et al.  Online aggregation , 1997, SIGMOD '97.

[7]  Elisa Bertino,et al.  Query Processing , 1997, Multimedia Databases in Perspective.

[8]  Frank Kelly,et al.  Rate control for communication networks: shadow prices, proportional fairness and stability , 1998, J. Oper. Res. Soc..

[9]  Steven H. Low,et al.  Optimization flow control—I: basic algorithm and convergence , 1999, TNET.

[10]  Jay H. Lee,et al.  Model predictive control: past, present and future , 1999 .

[11]  Daphne Koller,et al.  Making Rational Decisions Using Adaptive Utility Elicitation , 2000, AAAI/IAAI.

[12]  Robert Buff.  Continuous Time Finance , 2002 .

[13]  Jim Blythe,et al.  Visual exploration and incremental utility elicitation , 2002, AAAI/IAAI.

[14]  Peter Marbach,et al.  Priority service and max-min fairness , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[15]  Michael Stonebraker,et al.  Monitoring Streams - A New Class of Data Management Applications , 2002, VLDB.

[16]  Jennifer Widom,et al.  Query Processing, Resource Management, and Approximation ina Data Stream Management System , 2002 .

[17]  Michael Stonebraker,et al.  Load Shedding in a Data Stream Manager , 2003, VLDB.

[18]  Michael Stonebraker,et al.  Operator Scheduling in a Data Stream Manager , 2003, VLDB.

[19]  Ying Xing,et al.  Scalable Distributed Stream Processing , 2003, CIDR.

[20]  Rajeev Motwani,et al.  Load shedding for aggregation queries over data streams , 2004, Proceedings. 20th International Conference on Data Engineering.

[21]  David E. Irwin,et al.  Balancing risk and reward in a market-based task service , 2004, Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004..

[22]  Craig Boutilier,et al.  Regret-based Utility Elicitation in Constraint-based Decision Problems , 2005, IJCAI.

[23]  Lothar Thiele,et al.  Real-time interfaces for interface-based design of real-time systems with fixed priority scheduling , 2005, EMSOFT.

[24]  Asser N. Tantawi,et al.  Performance management for cluster-based web services , 2005, IEEE Journal on Selected Areas in Communications.

[25]  Binoy Ravindran,et al.  On recent advances in time/utility function real-time scheduling and resource management , 2005, Eighth IEEE International Symposium on Object-Oriented Real-Time Distributed Computing (ISORC'05).

[26]  Rajarshi Das,et al.  Utility-Function-Driven Resource Allocation in Autonomic Systems , 2005, Second International Conference on Autonomic Computing (ICAC'05).

[27]  Jeffrey O. Kephart,et al.  Research challenges of autonomic computing , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[28]  Song Liu,et al.  Control-Based Quality Adaptation in Data Stream Management Systems , 2005, DEXA.

[29]  John N. Tsitsiklis,et al.  Efficiency loss in a network resource allocation game: the case of elastic supply , 2005, IEEE Trans. Autom. Control..

[30]  Karsten Schwan,et al.  Distributed Stream Management using Utility-Driven Self-Adaptive Middleware , 2005, Second International Conference on Autonomic Computing (ICAC'05).

[31]  Ying Xing,et al.  The Design of the Borealis Stream Processing Engine , 2005, CIDR.

[32]  Sang Hyuk Son,et al.  Prediction-Based QoS Management for Real-Time Data Streams , 2006, 2006 27th IEEE International Real-Time Systems Symposium (RTSS'06).

[33]  Navendu Jain,et al.  Adaptive Control of Extreme-scale Stream Processing Systems , 2006, 26th IEEE International Conference on Distributed Computing Systems (ICDCS'06).

[34]  Amin Vahdat,et al.  Evaluating the impact of inaccurate information in utility-based scheduling , 2009, Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis.

[35]  Song Liu,et al.  Load shedding in stream databases: a control-based approach , 2006, VLDB.

[36]  Kang G. Shin,et al.  Adaptive control of virtualized resources in utility computing environments , 2007, EuroSys '07.

[37]  Malgorzata Steinder,et al.  Server virtualization in autonomic management of heterogeneous workloads , 2007, 2007 10th IFIP/IEEE International Symposium on Integrated Network Management.

[38]  Stanley B. Zdonik,et al.  Staying FIT: Efficient Load Shedding Techniques for Distributed Stream Processing , 2007, VLDB.

[39]  Andrzej Kochut,et al.  Dynamic Placement of Virtual Machines for Managing SLA Violations , 2007, 2007 10th IFIP/IEEE International Symposium on Integrated Network Management.

[40]  Chris Jermaine,et al.  Scalable approximate query processing with the DBO engine , 2008, TODS.

[41]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[42]  Radu Calinescu,et al.  A Model-Based Approach to the Autonomic Management of Mobile Robot Resources , 2010 .

[43]  Rajkumar Buyya,et al.  Energy Efficient Resource Management in Virtualized Cloud Data Centers , 2010, 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing.

[44]  Roy H. Campbell,et al.  ARIA: automatic resource inference and allocation for mapreduce environments , 2011, ICAC '11.

[45]  Ju Wang,et al.  Windows Azure Storage: a highly available cloud storage service with strong consistency , 2011, SOSP.

[46]  Benjamin Hindman,et al.  Dominant Resource Fairness: Fair Allocation of Multiple Resource Types , 2011, NSDI.

[47]  Randy H. Katz,et al.  Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center , 2011, NSDI.

[48]  Bernd Freisleben,et al.  Utility-based resource allocation for virtual machines in Cloud computing , 2011, 2011 IEEE Symposium on Computers and Communications (ISCC).

[49]  Rina Panigrahy,et al.  Heuristics for Vector Bin Packing , 2011 .

[50]  Srikanth Kandula,et al.  Jockey: guaranteed job latency in data parallel clusters , 2012, EuroSys '12.

[51]  Jasper Snoek,et al.  Practical Bayesian Optimization of Machine Learning Algorithms , 2012, NIPS.

[52]  Ion Stoica,et al.  BlinkDB: queries with bounded errors and bounded response times on very large data , 2012, EuroSys '13.

[53]  David E. Culler,et al.  Hierarchical scheduling for diverse datacenter workloads , 2013, SoCC.

[54]  Scott Shenker,et al.  Discretized streams: fault-tolerant streaming computation at scale , 2013, SOSP.

[55]  Michael J. Freedman,et al.  Aggregation and Degradation in JetStream: Streaming Analytics in the Wide Area , 2014, NSDI.

[56]  Srikanth Kandula,et al.  Multi-resource packing for cluster schedulers , 2014, SIGCOMM.

[57]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[58]  Carlo Curino,et al.  Reservation-based Scheduling: If You're Late Don't Blame Us! , 2014, SoCC.

[59]  Badrish Chandramouli,et al.  Trill: A High-Performance Incremental Query Processor for Diverse Analytics , 2014, Proc. VLDB Endow..

[60]  Wei Lin,et al.  Apollo: Scalable and Coordinated Scheduling for Cloud-Scale Computing , 2014, OSDI.

[61]  Adam Wierman,et al.  This Paper Is Included in the Proceedings of the 11th Usenix Symposium on Networked Systems Design and Implementation (nsdi '14). Grass: Trimming Stragglers in Approximation Analytics Grass: Trimming Stragglers in Approximation Analytics , 2022 .

[62]  Ion Stoica,et al.  The Power of Choice in Data-Aware Cluster Scheduling , 2014, OSDI.

[63]  Abhishek Verma,et al.  Large-scale cluster management at Google with Borg , 2015, EuroSys.

[64]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[65]  Craig Chambers,et al.  The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing , 2015, Proc. VLDB Endow..

[66]  Bohyung Han,et al.  Learning Multi-domain Convolutional Neural Networks for Visual Tracking , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[67]  Wei Lin,et al.  StreamScope: Continuous Reliable Distributed Processing of Big Data Streams , 2016, NSDI.

[68]  Song Han,et al.  Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding , 2015, ICLR.

[69]  Carlo Curino,et al.  Morpheus: Towards Automated SLOs for Enterprise Clusters , 2016, OSDI.

[70]  Alec Wolman,et al.  MCDNN: An Approximation-Based Execution Framework for Deep Stream Processing Under Resource Constraints , 2016, MobiSys.

[71]  Alexander Dekhtyar,et al.  Information Retrieval , 2018, Lecture Notes in Computer Science.