An Integrated Transport Solution to Big Data Movement in High-Performance Networks

We propose and develop an integrated transport solution to big data movement in high-performance networks in support of data-and network-intensive applications in various science domains. This solution integrates three major components, i.e. (i) transport-support workflow optimization, (ii) transport profile generation, and (iii) transport protocol design, into a unified framework.

[1]  Mark R. Meiss.  Tsunami: A High-Speed Rate-Controlled Protocol for File Transfer , 2004 .

[2]  R. H. Farrell.  Bounded Length Confidence Intervals for the Zero of a Regression Function , 1962 .

[3]  Donald F. Towsley,et al.  Modeling TCP Reno performance: a simple model and its empirical validation , 2000, TNET.

[4]  Larry Peterson,et al.  TCP Vegas: new techniques for congestion detection and avoidance , 1994, SIGCOMM 1994.

[5]  Injong Rhee,et al.  CUBIC: a new TCP-friendly high-speed TCP variant , 2008, OPSR.

[6]  Robert L. Grossman,et al.  UDT: UDP-based data transfer for high-speed wide area networks , 2007, Comput. Networks.

[7]  Raj Jain,et al.  Analysis of the Increase and Decrease Algorithms for Congestion Avoidance in Computer Networks , 1989, Comput. Networks.

[8]  Chase Qishi Wu,et al.  A Workflow-Based Network Advisor for Data Movement with End-to-End Performance Optimization , 2012, 2012 SC Companion: High Performance Computing, Networking Storage and Analysis.

[9]  Marta Beltrán,et al.  A New CPU Availability Prediction Model for Time-Shared Systems , 2008, IEEE Transactions on Computers.

[10]  Bijan Jabbari,et al.  DRAGON: a framework for service provisioning in heterogeneous grid networks , 2006, IEEE Communications Magazine.

[11]  Qishi Wu,et al.  Network-aware data movement advisor , 2011, NDM '11.

[12]  J. Spall.  Multivariate stochastic approximation using a simultaneous perturbation gradient approximation , 1992 .

[13]  Chase Qishi Wu,et al.  A Dynamic Performance-Based Flow Control Method for High-Speed Data Transfer , 2010, IEEE Transactions on Parallel and Distributed Systems.

[14]  Chase Qishi Wu,et al.  A class of reliable UDP-based transport protocols based on stochastic approximation , 2005, Proceedings IEEE 24th Annual Joint Conference of the IEEE Computer and Communications Societies..

[15]  Chase Qishi Wu,et al.  Modeling and optimizing transport-support workflows in high-performance networks , 2012, 37th Annual IEEE Conference on Local Computer Networks.

[16]  Yasumasa Fujisaki,et al.  A stopping rule for simultaneous perturbation stochastic approximation , 2013, 2013 European Control Conference (ECC).

[17]  Van Jacobson,et al.  Congestion avoidance and control , 1988, SIGCOMM '88.

[18]  James C. Spall,et al.  Introduction to Stochastic Search and Optimization. Estimation, Simulation, and Control (Spall, J.C. , 2007 .

[19]  Greg Kroah-Hartman,et al.  Linux device drivers, third edition , 2005 .

[20]  Steven H. Low,et al.  Understanding TCP Vegas: a duality model , 2002 .

[21]  R. Srikant,et al.  TCP-Illinois: A loss- and delay-based congestion control algorithm for high-speed networks , 2008, Perform. Evaluation.

[22]  Sushant Sharma,et al.  End-System Aware, Rate-Adaptive Protocol for Network Transport in LambdaGrid Environments , 2006, ACM/IEEE SC 2006 Conference (SC'06).

[23]  Jing Wu,et al.  A User-Controlled Lightpath Provisioning System for Grid Optical Networks , 2006, 2006 10th IEEE Singapore International Conference on Communication Systems.

[24]  R. Grossman,et al.  An Analysis of AIMD Algorithm with Decreasing Increases , 2022 .

[25]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[26]  Chase Qishi Wu,et al.  CHEETAH: circuit-switched high-speed end-to-end transport architecture testbed , 2005, IEEE Communications Magazine.

[27]  Sally Floyd.  Limited Slow-Start for TCP with Large Congestion Windows , 2004, RFC.

[28]  Tim Hesterberg,et al.  Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control , 2004, Technometrics.

[29]  Qian Zhang,et al.  A Compound TCP Approach for High-Speed and Long Distance Networks , 2006, Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications.

[30]  William E. Allcock,et al.  The Globus Striped GridFTP Framework and Server , 2005, ACM/IEEE SC 2005 Conference (SC'05).

[31]  Stephen W. Poole,et al.  A technique for moving large data sets over high-performance long distance networks , 2011, 2011 IEEE 27th Symposium on Mass Storage Systems and Technologies (MSST).

[32]  Robert L. Grossman,et al.  SABUL: A Transport Protocol for Grid Computing , 2003, Journal of Grid Computing.

[33]  Chase Qishi Wu,et al.  On transport methods for peak utilization of dedicated connections , 2009, 2009 Sixth International Conference on Broadband Communications, Networks, and Systems.

[34]  J. Spall.  Implementation of the simultaneous perturbation algorithm for stochastic optimization , 1998 .

[35]  Matthew Mathis,et al.  Automatic TCP buffer tuning , 1998, SIGCOMM '98.

[36]  Brian Tierney,et al.  A TCP Tuning Daemon , 2002, ACM/IEEE SC 2002 Conference (SC'02).

[37]  Mo Dong,et al.  PCC: Re-architecting Congestion Control for Consistent High Performance , 2014, NSDI.

[38]  Stephen E. Deering,et al.  Path MTU discovery , 1990, RFC.

[39]  J. Kiefer,et al.  Stochastic Estimation of the Maximum of a Regression Function , 1952 .

[40]  Robert L. Grossman,et al.  UDTv4: Improvements in Performance and Usability , 2008, GridNets.

[41]  Lixia Zhang,et al.  Stream Control Transmission Protocol , 2000, RFC.

[42]  Biswanath Mukherjee,et al.  RAPID: an end-system aware protocol for intelligent data transfer over lambda grids , 2006, Proceedings 20th IEEE International Parallel & Distributed Processing Symposium.

[43]  Aleksandar Kuzmanovic.  HSTCP-LP: A Protocol for Low-Priority Bulk Data Transfer in High-Speed High-RTT Networks , 2004 .

[44]  Mun Choon Chan,et al.  Sync-TCP: A new approach to high speed congestion control , 2009, 2009 17th IEEE International Conference on Network Protocols.

[45]  kc claffy,et al.  Bandwidth estimation: metrics, measurement techniques, and tools , 2003, IEEE Netw..

[46]  Olivier Bonaventure,et al.  MultiPath TCP: From Theory to Practice , 2011, Networking.

[47]  Injong Rhee,et al.  Binary increase congestion control (BIC) for fast long-distance networks , 2004, IEEE INFOCOM 2004.

[48]  Tom Kelly,et al.  Scalable TCP: improving performance in highspeed wide area networks , 2003, CCRV.

[49]  Jeffrey S. Chase,et al.  End system optimizations for high-speed TCP , 2001, IEEE Commun. Mag..

[50]  Min Zhu,et al.  B4: experience with a globally-deployed software defined wan , 2013, SIGCOMM.

[51]  Robert L. Grossman,et al.  Experiences in Design and Implementation of a High Performance Transport Protocol , 2004, Proceedings of the ACM/IEEE SC2004 Conference.

[52]  Chase Qishi Wu,et al.  Profiling Optimization for Big Data Transfer over Dedicated Channels , 2016, 2016 25th International Conference on Computer Communication and Networks (ICCCN).

[53]  Nick McKeown,et al.  OpenFlow: enabling innovation in campus networks , 2008, CCRV.

[54]  Chase Qishi Wu,et al.  Ultrascience net: network testbed for large-scale science applications , 2005, IEEE Communications Magazine.

[55]  Gregory F. Lawler,et al.  Random Walk: A Modern Introduction , 2010 .

[56]  Jason Lee,et al.  Intra and Interdomain Circuit Provisioning Using the OSCARS Reservation System , 2006, 2006 3rd International Conference on Broadband Communications, Networks and Systems.

[57]  Richard P. Mount.  The Office of Science Data-Management Challenge , 2005 .

[58]  Greg Kroah-Hartman,et al.  Linux Device Drivers, 3rd Edition , 2005 .

[59]  Loa Andersson,et al.  The Multiprotocol Label Switching (MPLS) Working Group decision on MPLS signaling protocols , 2003, RFC.

[60]  Robert L. Grossman,et al.  Udt: a high performance data transport protocol , 2005 .

[61]  Manish Jain,et al.  Socket Buffer Auto-Sizing for High-Performance Data Transfers , 2003, Journal of Grid Computing.

[62]  Jason Leigh,et al.  Reliable Blast UDP : predictable high performance bulk data transfer , 2002, Proceedings. IEEE International Conference on Cluster Computing.

[63]  Cheng Jin,et al.  FAST TCP: Motivation, Architecture, Algorithms, Performance , 2006, IEEE/ACM Transactions on Networking.

[64]  Sally Floyd,et al.  HighSpeed TCP for Large Congestion Windows , 2003, RFC.

[65]  Mark Handley,et al.  How Hard Can It Be? Designing and Implementing a Deployable Multipath TCP , 2012, NSDI.

[66]  Wenji Wu,et al.  The performance analysis of linux networking - Packet receiving , 2007, Comput. Commun..

[67]  Fred W. Glover,et al.  Future paths for integer programming and links to artificial intelligence , 1986, Comput. Oper. Res..

[68]  Chase Qishi Wu,et al.  Profiling transport performance for big data transfer over dedicated channels , 2015, 2015 International Conference on Computing, Networking and Communications (ICNC).

[69]  Mark Handley,et al.  Congestion control for high bandwidth-delay product networks , 2002, SIGCOMM.

[70]  Injong Rhee,et al.  Delay-based congestion avoidance for TCP , 2003, TNET.