Hone: Mitigating Stragglers in Distributed Stream Processing With Tuple Scheduling

Low latency stream processing on large clusters consisting of hundreds to thousands of servers is an increasingly important challenge. A crucial barrier to tackling this challenge is stragglers, i.e., tasks that are significantly straggling behind others in processing the stream data. However, prior straggler mitigation solutions have significant limitations. They balance streaming workloads among tasks but may incur imbalanced backlogs when the workloads exhibit variance, causing stragglers as well. Fortunately, we observe that carefully scheduling the outgoing tuples of different tasks can yield benefits for balancing backlogs, and thus avoids stragglers. To this end, we present Hone, a tuple scheduler that aims to minimize the maximum queue backlog of all tasks over time. Hone leverages an online Largest-Backlog-First (LBF) algorithm with a provable good competitive ratio to perform efficient tuple scheduling. We have implemented Hone based on Apache Storm and evaluated it extensively via both simulations and testbed experiments. Our results show that under the same workload balancing strategy–shuffle grouping, Hone outperforms the original Storm significantly, with the end-to-end tuple processing latency reduced by 78.7 percent on average.

[1]  Randy H. Katz,et al.  Wrangler: Predictable and Faster Jobs using Fewer Resources , 2014, SoCC.

[2]  Randy H. Katz,et al.  Heterogeneity and dynamicity of clouds at scale: Google trace analysis , 2012, SoCC '12.

[3]  Sriram Rao,et al.  Dhalion: Self-Regulating Stream Processing in Heron , 2017, Proc. VLDB Endow..

[4]  Gianmarco De Francisci Morales,et al.  The power of both choices: Practical load balancing for distributed stream processing engines , 2015, 2015 IEEE 31st International Conference on Data Engineering.

[5]  Minyi Guo,et al.  Falcon: Addressing Stragglers in Heterogeneous Parameter Server Via Multiple Parallelism , 2021, IEEE Transactions on Computers.

[6]  Vasiliki Kalavri,et al.  Three steps is all you need: fast, accurate, automatic scaling decisions for distributed streaming dataflows , 2018, OSDI.

[7]  Paolo Costa,et al.  Chi: A Scalable and Programmable Control Plane for Distributed Stream Processing Systems , 2018, Proc. VLDB Endow..

[8]  Anshul Jaiswal,et al.  Realtime Data Processing at Facebook , 2016, SIGMOD Conference.

[9]  Mianxiong Dong,et al.  SEER-MCache: A Prefetchable Memory Object Caching System for IoT Real-Time Data Processing , 2018, IEEE Internet of Things Journal.

[10]  Roberto Baldoni,et al.  Adaptive online scheduling in storm , 2013, DEBS.

[11]  Gianmarco De Francisci Morales,et al.  When two choices are not enough: Balancing at scale in Distributed Stream Processing , 2015, 2016 IEEE 32nd International Conference on Data Engineering (ICDE).

[12]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[13]  Raj Jain,et al.  A Quantitative Measure Of Fairness And Discrimination For Resource Allocation In Shared Computer Systems , 1998, ArXiv.

[14]  Michael J. Freedman,et al.  Aggregation and Degradation in JetStream: Streaming Analytics in the Wide Area , 2014, NSDI.

[15]  Scott Shenker,et al.  Usenix Association 10th Usenix Symposium on Networked Systems Design and Implementation (nsdi '13) 185 Effective Straggler Mitigation: Attack of the Clones , 2022 .

[16]  Vyas Sekar,et al.  CFA: A Practical Prediction System for Video QoE Optimization , 2016, NSDI.

[17]  Jeyhun Karimov,et al.  Benchmarking Distributed Stream Data Processing Systems , 2019, 2018 IEEE 34th International Conference on Data Engineering (ICDE).

[18]  Abhishek Chandra,et al.  Multi-Query Optimization in Wide-Area Streaming Analytics , 2018, SoCC.

[19]  Albert G. Greenberg,et al.  Reining in the Outliers in Map-Reduce Clusters using Mantri , 2010, OSDI.

[20]  Mianxiong Dong,et al.  Real-Time Awareness Scheduling for Multimedia Big Data Oriented In-Memory Computing , 2018, IEEE Internet of Things Journal.

[21]  T. N. Vijaykumar,et al.  Dart: Divide and Specialize for Fast Response to Congestion in RDMA-Based Datacenter Networks , 2018, IEEE/ACM Transactions on Networking.

[22]  Magdalena Balazinska,et al.  SkewTune: mitigating skew in mapreduce applications , 2012, SIGMOD Conference.

[23]  Adam Wierman,et al.  Hopper: Decentralized Speculation-aware Cluster Scheduling at Scale , 2015, SIGCOMM.

[24]  T. V. Lakshman,et al.  Typhoon: An SDN Enhanced Real-Time Big Data Streaming Framework , 2017, CoNEXT.

[25]  Shrideep Pallickara,et al.  NEPTUNE: Real Time Stream Processing for Internet of Things and Sensing Environments , 2016, 2016 IEEE International Parallel and Distributed Processing Symposium (IPDPS).

[26]  Hai Jin,et al.  Towards Low-Latency Batched Stream Processing by Pre-Scheduling , 2019, IEEE Transactions on Parallel and Distributed Systems.

[27]  Kun-Lung Wu,et al.  SODA: An Optimizing Scheduler for Large-Scale Stream-Based Distributed Computer Systems , 2008, Middleware.

[28]  Scott Shenker,et al.  Adaptive Stream Processing using Dynamic Batch Sizing , 2014, SoCC.

[29]  Aakanksha Chowdhery,et al.  The Design and Implementation of a Wireless Video Surveillance System , 2015, MobiCom.

[30]  Kun-Lung Wu,et al.  COLA: Optimizing Stream Processing Applications via Graph Partitioning , 2009, Middleware.

[31]  Ali Ghodsi,et al.  Drizzle: Fast and Adaptable Stream Processing at Scale , 2017, SOSP.

[32]  Kaoru Ota,et al.  DSARP: Dependable Scheduling with Active Replica Placement for Workflow Applications in Cloud Computing , 2020, IEEE Transactions on Cloud Computing.

[33]  Zhuo Tang,et al.  Optimizing Speculative Execution in Spark Heterogeneous Environments , 2019 .

[34]  Craig Chambers,et al.  The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing , 2015, Proc. VLDB Endow..

[35]  Ying Xing,et al.  Providing resiliency to load variations in distributed stream processing , 2006, VLDB.

[36]  Eric P. Xing,et al.  Addressing the straggler problem for iterative convergent parallel ML , 2016, SoCC.

[37]  Ramesh K. Sitaraman,et al.  Trading Timeliness and Accuracy in Geo-Distributed Streaming Analytics , 2016, SoCC.

[38]  Aoying Zhou,et al.  Parallel Stream Processing Against Workload Skewness and Variance , 2017, HPDC.

[39]  Bruno Sericola,et al.  Online Scheduling for Shuffle Grouping in Distributed Stream Processing Systems , 2016, Middleware.

[40]  Hai Jin,et al.  TurboStream: Towards Low-Latency Data Stream Processing , 2018, 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS).

[41]  Indranil Gupta,et al.  Stateful Scalable Stream Processing at LinkedIn , 2017, Proc. VLDB Endow..

[42]  Sandeep Chinchali,et al.  NUMFabric: Fast and Flexible Bandwidth Allocation in Datacenters , 2016, SIGCOMM.

[43]  Wei Bai,et al.  Information-Agnostic Flow Scheduling for Commodity Data Centers , 2015, NSDI.

[44]  Indranil Gupta,et al.  Henge: Intent-driven Multi-Tenant Stream Processing , 2018, SoCC.

[45]  Weng Chon Ao,et al.  Resource-Constrained Replication Strategies for Hierarchical and Heterogeneous Tasks , 2020, IEEE Transactions on Parallel and Distributed Systems.

[46]  Changjun Jiang,et al.  Improving Performance of Heterogeneous MapReduce Clusters with Adaptive Task Tuning , 2017, IEEE Transactions on Parallel and Distributed Systems.