View in EDS

An HTTP REST API for Tape-backed Storage.

Saved in:
Bibliographic Details
Title: An HTTP REST API for Tape-backed Storage.
Authors: Afonso, João, Caffy, Cédric, Patrascoiu, Mihai, Leduc, Julien, Davis, Michael, Murray, Steven, Cortes, Pablo
Source: EPJ Web of Conferences; 5/6/2024, Vol. 295, p1-8, 8p
Subject Terms: HTTP (Computer network protocol), DATA management, BIT rate, COMPUTER storage devices, HYPERTEXT systems
Abstract: The goal of the HTTP REST API for Tape project is to provide a simple, minimalistic and uniform interface to manage data transfers between Storage Endpoints (SEs) where the source file is on tape. The project is a collaboration between the developers of WLCG storage systems (EOS+CTA, dCache, StoRM) and data transfer clients (gfal2, FTS). For some years, HTTP has been growing in popularity as the preferred data transfer protocol between many WLCG SEs. However — unlike other protocols such as XRootD and SRM — HTTP does not include a method to stage files from tape to disk prior to transfer, forcing the use of workarounds such as hybrid protocols (different protocols used for the "stage" and "transfer" parts of the operation). The HTTP REST API offers a simple and consistent solution, by extending the HTTP protocol to include staging operations. It provides clients with a common and consistent API across different storage systems to manage and monitor the disk and tape residency of stored files. In this contribution, we present the history and motivation of the HTTP REST API project, the specification of version 1 of the API and implementation details in the various storage and data transfer systems. We also describe our experience of its deployment and use for LHC Run-3 operations. We conclude with a discussion of possible future work. [ABSTRACT FROM AUTHOR]
Copyright of EPJ Web of Conferences is the property of EDP Sciences and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Complementary Index
Description
Abstract:The goal of the HTTP REST API for Tape project is to provide a simple, minimalistic and uniform interface to manage data transfers between Storage Endpoints (SEs) where the source file is on tape. The project is a collaboration between the developers of WLCG storage systems (EOS+CTA, dCache, StoRM) and data transfer clients (gfal2, FTS). For some years, HTTP has been growing in popularity as the preferred data transfer protocol between many WLCG SEs. However — unlike other protocols such as XRootD and SRM — HTTP does not include a method to stage files from tape to disk prior to transfer, forcing the use of workarounds such as hybrid protocols (different protocols used for the "stage" and "transfer" parts of the operation). The HTTP REST API offers a simple and consistent solution, by extending the HTTP protocol to include staging operations. It provides clients with a common and consistent API across different storage systems to manage and monitor the disk and tape residency of stored files. In this contribution, we present the history and motivation of the HTTP REST API project, the specification of version 1 of the API and implementation details in the various storage and data transfer systems. We also describe our experience of its deployment and use for LHC Run-3 operations. We conclude with a discussion of possible future work. [ABSTRACT FROM AUTHOR]
ISSN:21016275
DOI:10.1051/epjconf/202429501008