Toward Interlanguage Parallel Scripting for Distributed-Memory Scientific Computing

Scripting languages such as Python and R have been widely adopted as tools for the productive development of scientific software because of the power and expressiveness of the languages and available libraries. However, deploying scripted applications on large-scale parallel computer systems such as...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:arXiv.org
Hlavní autori: Wozniak, Justin M, Armstrong, Timothy G, Maheshwari, Ketan C, Katz, Daniel S, Wilde, Michael, Foster, Ian T
Médium: Paper
Jazyk:English
Vydavateľské údaje: Ithaca Cornell University Library, arXiv.org 06.07.2021
Predmet:
Distributed memory
Interoperability
Interpreters
Libraries
Parallel computers
Programming languages
Supercomputers
ISSN:2331-8422
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Abstract Scripting languages such as Python and R have been widely adopted as tools for the productive development of scientific software because of the power and expressiveness of the languages and available libraries. However, deploying scripted applications on large-scale parallel computer systems such as the IBM Blue Gene/Q or Cray XE6 is a challenge because of issues including operating system limitations, interoperability challenges, parallel filesystem overheads due to the small file system accesses common in scripted approaches, and other issues. We present here a new approach to these problems in which the Swift scripting system is used to integrate high-level scripts written in Python, R, and Tcl, with native code developed in C, C++, and Fortran, by linking Swift to the library interfaces to the script interpreters. In this approach, Swift handles data management, movement, and marshaling among distributed-memory processes without direct user manipulation of low-level communication libraries such as MPI. We present a technique to efficiently launch scripted applications on large-scale supercomputers using a hierarchical programming model.
AbstractList Scripting languages such as Python and R have been widely adopted as tools for the productive development of scientific software because of the power and expressiveness of the languages and available libraries. However, deploying scripted applications on large-scale parallel computer systems such as the IBM Blue Gene/Q or Cray XE6 is a challenge because of issues including operating system limitations, interoperability challenges, parallel filesystem overheads due to the small file system accesses common in scripted approaches, and other issues. We present here a new approach to these problems in which the Swift scripting system is used to integrate high-level scripts written in Python, R, and Tcl, with native code developed in C, C++, and Fortran, by linking Swift to the library interfaces to the script interpreters. In this approach, Swift handles data management, movement, and marshaling among distributed-memory processes without direct user manipulation of low-level communication libraries such as MPI. We present a technique to efficiently launch scripted applications on large-scale supercomputers using a hierarchical programming model.
Author Wozniak, Justin M
Katz, Daniel S
Maheshwari, Ketan C
Foster, Ian T
Armstrong, Timothy G
Wilde, Michael
Author_xml – sequence: 1
  givenname: Justin
  surname: Wozniak
  middlename: M
  fullname: Wozniak, Justin M
– sequence: 2
  givenname: Timothy
  surname: Armstrong
  middlename: G
  fullname: Armstrong, Timothy G
– sequence: 3
  givenname: Ketan
  surname: Maheshwari
  middlename: C
  fullname: Maheshwari, Ketan C
– sequence: 4
  givenname: Daniel
  surname: Katz
  middlename: S
  fullname: Katz, Daniel S
– sequence: 5
  givenname: Michael
  surname: Wilde
  fullname: Wilde, Michael
– sequence: 6
  givenname: Ian
  surname: Foster
  middlename: T
  fullname: Foster, Ian T
BookMark eNotjs1Kw0AYRQdRsNY-gLuA69RvfjNZStRaqCg0-zKZfAlT0pk4Sfx5eyO6unA5nHuvyLkPHgm5obAWWkq4M_HLfawZhWwNTAt6RhaMc5pqwdglWQ3DEQCYypiUfEH2Zfg0sU62fsTYGd9OpsXkzUTTddglextdPzrfJk2IyYMbxuiqacQ6fcFTiN8z4NCPrnE2KcKpn37Za3LRmG7A1X8uSfn0WBbP6e51sy3ud6mRjKbKIlNoADRF2sxNJQUINX82taCZ1JWSkKOwmbFaNKDBcA3a1lVjFctzviS3f9o-hvcJh_FwDFP08-KBSZGrXFJO-Q9aD1Nq
ContentType Paper
Copyright 2021. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Copyright_xml – notice: 2021. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
DBID 8FE
8FG
ABJCF
ABUWG
AFKRA
AZQEC
BENPR
BGLVJ
CCPQU
DWQXO
HCIFZ
L6V
M7S
PHGZM
PHGZT
PIMPY
PKEHL
PQEST
PQGLB
PQQKQ
PQUKI
PRINS
PTHSS
DOI 10.48550/arxiv.2107.02841
DatabaseName ProQuest SciTech Collection
ProQuest Technology Collection
Materials Science & Engineering Collection
ProQuest Central (Alumni)
ProQuest Central UK/Ireland
ProQuest Central Essentials
ProQuest Central
Technology collection
ProQuest One Community College
ProQuest Central
SciTech Premium Collection
ProQuest Engineering Collection
Engineering Database
ProQuest Central Premium
ProQuest One Academic
ProQuest Publicly Available Content
ProQuest One Academic Middle East (New)
ProQuest One Academic Eastern Edition (DO NOT USE)
ProQuest One Applied & Life Sciences
ProQuest One Academic (retired)
ProQuest One Academic UKI Edition
ProQuest Central China
Engineering collection
DatabaseTitle Publicly Available Content Database
Engineering Database
Technology Collection
ProQuest One Academic Middle East (New)
ProQuest Central Essentials
ProQuest One Academic Eastern Edition
ProQuest Central (Alumni Edition)
SciTech Premium Collection
ProQuest One Community College
ProQuest Technology Collection
ProQuest SciTech Collection
ProQuest Central China
ProQuest Central
ProQuest One Applied & Life Sciences
ProQuest Engineering Collection
ProQuest One Academic UKI Edition
ProQuest Central Korea
Materials Science & Engineering Collection
ProQuest Central (New)
ProQuest One Academic
ProQuest One Academic (New)
Engineering Collection
DatabaseTitleList Publicly Available Content Database
Database_xml – sequence: 1
  dbid: PIMPY
  name: ProQuest Publicly Available Content
  url: http://search.proquest.com/publiccontent
  sourceTypes: Aggregation Database
DeliveryMethod fulltext_linktorsrc
Discipline Physics
EISSN 2331-8422
Genre Working Paper/Pre-Print
GroupedDBID 8FE
8FG
ABJCF
ABUWG
AFKRA
ALMA_UNASSIGNED_HOLDINGS
AZQEC
BENPR
BGLVJ
CCPQU
DWQXO
FRJ
HCIFZ
L6V
M7S
M~E
PHGZM
PHGZT
PIMPY
PKEHL
PQEST
PQGLB
PQQKQ
PQUKI
PRINS
PTHSS
ID FETCH-LOGICAL-a521-6ce26ea0081e1f521b54046028ad41758b6509e4c7ac84f080a3808cdbfc62993
IEDL.DBID PIMPY
IngestDate Mon Jun 30 09:10:56 EDT 2025
IsDoiOpenAccess true
IsOpenAccess true
IsPeerReviewed false
IsScholarly false
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-a521-6ce26ea0081e1f521b54046028ad41758b6509e4c7ac84f080a3808cdbfc62993
Notes SourceType-Working Papers-1
ObjectType-Working Paper/Pre-Print-1
content type line 50
OpenAccessLink https://www.proquest.com/publiccontent/docview/2549695131?pq-origsite=%requestingapplication%
PQID 2549695131
PQPubID 2050157
ParticipantIDs proquest_journals_2549695131
PublicationCentury 2000
PublicationDate 20210706
PublicationDateYYYYMMDD 2021-07-06
PublicationDate_xml – month: 07
  year: 2021
  text: 20210706
  day: 06
PublicationDecade 2020
PublicationPlace Ithaca
PublicationPlace_xml – name: Ithaca
PublicationTitle arXiv.org
PublicationYear 2021
Publisher Cornell University Library, arXiv.org
Publisher_xml – name: Cornell University Library, arXiv.org
SSID ssj0002672553
Score 1.7629974
SecondaryResourceType preprint
Snippet Scripting languages such as Python and R have been widely adopted as tools for the productive development of scientific software because of the power and...
SourceID proquest
SourceType Aggregation Database
SubjectTerms Distributed memory
Interoperability
Interpreters
Libraries
Parallel computers
Programming languages
Supercomputers
Title Toward Interlanguage Parallel Scripting for Distributed-Memory Scientific Computing
URI https://www.proquest.com/docview/2549695131
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV1NT8JAEJ0oaOLJ7_iBZA9eC6Ut7XIyUSF6gDTCAU9kPw2JKdgi0X_vzLbowcST17aHzTZ982b29T2A625PGGmN9AJDo5tYaE9ai12K5D2pjDbdQLuwiWQ04tNpL61-jy4qWeUGEx1Ql27PpNtGEG7rhaKJeZvamhjJQdi5Wb55lCFFZ61VoMY21Ml4y69BPX0cps_fM5cgTpBBh-XhprPyaov8Y75uYd-TtLDSRp1fkOzqzGD_f1d4gCsTS5MfwpbJjmDXqT1VcQzjiRPLMjcO3EwsWSpyClZ5ZeMSSbIXhoyW3ZO1LqViGe0NSZf7yRwgOJERK1Mh8NkTmAz6k7sHr0pX8ASWbI-SwGIjiBKYjsUrErlbFOMmCB0hp-CSvPVMpBKheGSRWIqQ-1xpaVWMNSw8hVq2yMwZMN-3xhfKJoHmUUhmA5p895AZykhyG55DY7Nhs-oLKWY_-3Px9-1L2AtIR-Jkeg2orfJ3cwU7ar2aF3kT6rf9UfrUJM3muFm98C8T1b0e
linkProvider ProQuest
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V1LT8JAEJ4gaPTkOz5Q96DHQmmXthyMB5FAeIQEDnhq9mlIDGCLKD_K_-jsQvVg4s2D17Zps53pN4_9Oh_AdbXGFNeKO54yrZuASYdrjVUKj2pcKKmqnrRiE2GvF41GtX4OPrJ_YQytMsNEC9RyKkyPvGwKmQDTAb9yN3txjGqU2V3NJDRWbtFWyzcs2dLbVh3te-N5jYfhfdNZqwo4DEOVYxSwAsVMKFQVjUc45iw0wDDLJMVYGnEzU05RETIRUY0JFfMjNxKSaxEgdvt42w0oUPR1Nw-Ffqvbf_xq6nhBiCm6v9o9tbPCyix5Hy9KWFiFJXwGrfzAfBvIGrv_7BXs4dLZTCX7kFOTA9iyfFWRHsJgaOm-xDY0s54r6bPESMM8k8EKCydPBHNyUjfDgY2ul5JO1zCLl8RCmqVJkZWuBV57BMO_WMYx5CfTiToB4rpauUzo0JMR9c24BGkmB2JuyymPtH8Kxcwi8fobT-Nvc5z9fvoKtpvDbifutHrtc9jxDCvGkg6LkJ8nr-oCNsViPk6Ty7U_EYj_2HyfszUIjQ
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Toward+Interlanguage+Parallel+Scripting+for+Distributed-Memory+Scientific+Computing&rft.jtitle=arXiv.org&rft.au=Wozniak%2C+Justin+M&rft.au=Armstrong%2C+Timothy+G&rft.au=Maheshwari%2C+Ketan+C&rft.au=Katz%2C+Daniel+S&rft.date=2021-07-06&rft.pub=Cornell+University+Library%2C+arXiv.org&rft.eissn=2331-8422&rft_id=info:doi/10.48550%2Farxiv.2107.02841