A Run-time System for Efficient Execution of Scientific Workflows on Distributed Environments

Scientific workflow systems have been introduced in response to the demand of researchers from several domains of science who need to process and analyze increasingly larger datasets. The design of these systems is largely based on the observation that data analysis applications can be composed as p...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:International journal of parallel programming Ročník 36; číslo 2; s. 250 - 266
Hlavní autoři: Teodoro, George, Tavares, Tulio, Ferreira, Renato, Kurc, Tahsin, Meira, Wagner, Guedes, Dorgival, Pan, Tony, Saltz, Joel
Médium: Journal Article
Jazyk:angličtina
Vydáno: Boston Springer US 01.04.2008
Springer Nature B.V
Témata:
Computer Science
Data analysis
Data processing
Datasets
Experiments
Parallel processing
Processor Architectures
Software
Software Engineering/Programming and Operating Systems
Studies
Theory of Computation
Parallel computing
Data-analysis
Scientific workflows
ISSN:0885-7458, 1573-7640
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Scientific workflow systems have been introduced in response to the demand of researchers from several domains of science who need to process and analyze increasingly larger datasets. The design of these systems is largely based on the observation that data analysis applications can be composed as pipelines or networks of computations on data. In this work, we present a run-time support system that is designed to facilitate this type of computation in distributed computing environments. Our system is optimized for data-intensive workflows, in which efficient management and retrieval of data, coordination of data processing and data movement, and check-pointing of intermediate results are critical and challenging issues. Experimental evaluation of our system shows that linear speedups can be achieved for sophisticated applications, which are implemented as a network of multiple data processing components.
Bibliografie:SourceType-Scholarly Journals-1
ObjectType-Feature-1
content type line 14
ObjectType-Article-1
ObjectType-Feature-2
content type line 23
ObjectType-Article-2
ISSN:0885-7458
1573-7640
DOI:10.1007/s10766-007-0068-8