Third-party transfers in WLCG using HTTP

Since its earliest days, the Worldwide LHC Computational Grid (WLCG) has relied on GridFTP to transfer data between sites. The announcement that Globus is dropping support of its open source Globus Toolkit (GT), which forms the basis for several FTP client and servers, has created an opportunity to reevaluate the use of FTP. HTTP-TPC, an extension to HTTP compatible with WebDAV, has arisen as a strong contender for an alternative approach. In this paper, we describe the HTTP-TPC protocol itself, along with the current status of its support in different implementations, and the interoperability testing done within the WLCG DOMA working group's TPC activity. This protocol also provides the first real use-case for token-based authorisation for this community. We will demonstrate the benefits of such authorisation by showing how it allows HTTP-TPC to support new technologies (such as OAuth, OpenID Connect, Macaroons and SciTokens) without changing the protocol. We will also discuss the next steps for HTTP-TPC and the plans to use the protocol for WLCG transfers.

[1]  Ian T. Foster,et al.  A security architecture for computational grids , 1998, CCS '98.

[2]  Fabrizio Furano,et al.  Towards an HTTP Ecosystem for HEP Data Access , 2014 .

[3]  Farid Ould-Saada,et al.  Rucio: Scientific Data Management , 2019, Computing and Software for Big Science.

[4]  Andrew McNab,et al.  The GridSite Web/Grid security system , 2005, Softw. Pract. Exp..

[5]  Arnar Birgisson,et al.  Macaroons: Cookies with Contextual Caveats for Decentralized Authorization in the Cloud , 2014, NDSS.

[6]  Anupam Ashish,et al.  dCache, towards Federated Identities & Anonymized Delegation , 2017 .

[7]  Oliver Keeble,et al.  FTS3: New Data Movement Service For WLCG , 2014 .

[8]  Paul Millar,et al.  Bootstrapping a New LHC Data Transfer Ecosystem , 2019, EPJ Web of Conferences.

[9]  Brian Bockelman,et al.  SciTokens: Capability-Based Secure Access to Remote Scientific Data , 2018, PEARC.

[10]  Dick Hardt,et al.  The OAuth 2.0 Authorization Framework , 2012, RFC.

[11]  Patrick Fuhrmann,et al.  dCache, Storage System for the Future , 2006, Euro-Par.

[12]  Maarten Litmaath.  The Storage Resource Manager Interface Specification Version 2.2 , 2013 .

[13]  Roy T. Fielding,et al.  Hypertext Transfer Protocol - HTTP/1.1 , 1997, RFC.

[14]  Michael B. Jones,et al.  OAuth 2.0 Authorization Server Metadata , 2018, RFC.

[15]  Steven Tuecke,et al.  GridFTP: Protocol Extensions to FTP for the Grid , 2001 .

[16]  William E. Allcock,et al.  The Globus Striped GridFTP Framework and Server , 2005, ACM/IEEE SC 2005 Conference (SC'05).