pPython for Parallel Python Programming

pPython seeks to provide a parallel capability that provides good speed-up without sacrificing the ease of programming in Python by implementing partitioned global array semantics (PGAS) on top of a simple file-based messaging library (PythonMPI) in pure Python. The core data structure in pPython is...

Full description

Saved in:
Bibliographic Details
Published in:IEEE Conference on High Performance Extreme Computing (Online) pp. 1 - 6
Main Authors: Byun, Chansup, Arcand, William, Bestor, David, Bergeron, Bill, Gadepally, Vijay, Houle, Michael, Hubbell, Matthew, Jananthan, Hayden, Jones, Michael, Keville, Kurt, Klein, Anna, Michaleas, Peter, Milechin, Lauren, Morales, Guillermo, Mullen, Julie, Prout, Andrew, Reuther, Albert, Rosa, Antonio, Samsi, Siddharth, Yee, Charles, Kepner, Jeremy
Format: Conference Proceeding
Language:English
Published: IEEE 19.09.2022
Subjects:
Linux
MPI
Operating systems
Parallel Programming
PGAS
Portable computers
Program processors
Programming
Python
Scalability
Semantics
ISSN:2643-1971
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract pPython seeks to provide a parallel capability that provides good speed-up without sacrificing the ease of programming in Python by implementing partitioned global array semantics (PGAS) on top of a simple file-based messaging library (PythonMPI) in pure Python. The core data structure in pPython is a distributed numerical array whose distribution onto multiple processors is specified with a 'map' construct. Communication operations between distributed arrays are abstracted away from the user and pPython transparently supports redistribution between any block-cyclic-overlapped distributions in up to four dimensions. pPython follows a SPMD (single program multiple data) model of computation. pPython runs on any combination of heterogeneous systems that support Python, including Windows, Linux, and MacOS operating systems. In addition to running transparently on single-node (e.g., a laptop), pPython provides a scheduler interface, so that pPython can be executed in a massively parallel computing environment. The initial implementation uses the Slurm scheduler. Performance of pPython on the HPC Challenge benchmark suite demonstrates both ease of programming and scalability.
AbstractList pPython seeks to provide a parallel capability that provides good speed-up without sacrificing the ease of programming in Python by implementing partitioned global array semantics (PGAS) on top of a simple file-based messaging library (PythonMPI) in pure Python. The core data structure in pPython is a distributed numerical array whose distribution onto multiple processors is specified with a 'map' construct. Communication operations between distributed arrays are abstracted away from the user and pPython transparently supports redistribution between any block-cyclic-overlapped distributions in up to four dimensions. pPython follows a SPMD (single program multiple data) model of computation. pPython runs on any combination of heterogeneous systems that support Python, including Windows, Linux, and MacOS operating systems. In addition to running transparently on single-node (e.g., a laptop), pPython provides a scheduler interface, so that pPython can be executed in a massively parallel computing environment. The initial implementation uses the Slurm scheduler. Performance of pPython on the HPC Challenge benchmark suite demonstrates both ease of programming and scalability.
Author Prout, Andrew
Gadepally, Vijay
Jananthan, Hayden
Klein, Anna
Morales, Guillermo
Houle, Michael
Bergeron, Bill
Byun, Chansup
Arcand, William
Hubbell, Matthew
Keville, Kurt
Reuther, Albert
Jones, Michael
Mullen, Julie
Michaleas, Peter
Kepner, Jeremy
Milechin, Lauren
Yee, Charles
Bestor, David
Rosa, Antonio
Samsi, Siddharth
Author_xml – sequence: 1
  givenname: Chansup
  surname: Byun
  fullname: Byun, Chansup
  organization: Massachusetts Institute of Technology
– sequence: 2
  givenname: William
  surname: Arcand
  fullname: Arcand, William
  organization: Massachusetts Institute of Technology
– sequence: 3
  givenname: David
  surname: Bestor
  fullname: Bestor, David
  organization: Massachusetts Institute of Technology
– sequence: 4
  givenname: Bill
  surname: Bergeron
  fullname: Bergeron, Bill
  organization: Massachusetts Institute of Technology
– sequence: 5
  givenname: Vijay
  surname: Gadepally
  fullname: Gadepally, Vijay
  organization: Massachusetts Institute of Technology
– sequence: 6
  givenname: Michael
  surname: Houle
  fullname: Houle, Michael
  organization: Massachusetts Institute of Technology
– sequence: 7
  givenname: Matthew
  surname: Hubbell
  fullname: Hubbell, Matthew
  organization: Massachusetts Institute of Technology
– sequence: 8
  givenname: Hayden
  surname: Jananthan
  fullname: Jananthan, Hayden
  organization: Massachusetts Institute of Technology
– sequence: 9
  givenname: Michael
  surname: Jones
  fullname: Jones, Michael
  organization: Massachusetts Institute of Technology
– sequence: 10
  givenname: Kurt
  surname: Keville
  fullname: Keville, Kurt
  organization: Massachusetts Institute of Technology
– sequence: 11
  givenname: Anna
  surname: Klein
  fullname: Klein, Anna
  organization: Massachusetts Institute of Technology
– sequence: 12
  givenname: Peter
  surname: Michaleas
  fullname: Michaleas, Peter
  organization: Massachusetts Institute of Technology
– sequence: 13
  givenname: Lauren
  surname: Milechin
  fullname: Milechin, Lauren
  organization: Massachusetts Institute of Technology
– sequence: 14
  givenname: Guillermo
  surname: Morales
  fullname: Morales, Guillermo
  organization: Massachusetts Institute of Technology
– sequence: 15
  givenname: Julie
  surname: Mullen
  fullname: Mullen, Julie
  organization: Massachusetts Institute of Technology
– sequence: 16
  givenname: Andrew
  surname: Prout
  fullname: Prout, Andrew
  organization: Massachusetts Institute of Technology
– sequence: 17
  givenname: Albert
  surname: Reuther
  fullname: Reuther, Albert
  organization: Massachusetts Institute of Technology
– sequence: 18
  givenname: Antonio
  surname: Rosa
  fullname: Rosa, Antonio
  organization: Massachusetts Institute of Technology
– sequence: 19
  givenname: Siddharth
  surname: Samsi
  fullname: Samsi, Siddharth
  organization: Massachusetts Institute of Technology
– sequence: 20
  givenname: Charles
  surname: Yee
  fullname: Yee, Charles
  organization: Massachusetts Institute of Technology
– sequence: 21
  givenname: Jeremy
  surname: Kepner
  fullname: Kepner, Jeremy
  organization: Massachusetts Institute of Technology
BookMark eNotj0tLw0AUhUepYFv7CwTJzlXi3Hvn0VlKaK1QMIu6LvNqjeRRJt303xswi8OBb3E434LNur6LjL0ALwC4edtVm1LKNUKBHLEwBhUpeccWoJQURq-VumdzVIJyMBoe2WoYfjnnRMg10Zy9Xqrb9afvslOfssom2zSxySZWpf6cbNvW3fmJPZxsM8TV1Ev2vd0cyl2-__r4LN_3eY2crrk3XkgHfjyjggjOWuE0eQ9AkRwQej2CMeBxFABHFjW5ABAMD0rTkj3_79YxxuMl1a1Nt-PkRX-Nd0LM
ContentType Conference Proceeding
DBID 6IE
6IL
CBEJK
RIE
RIL
DOI 10.1109/HPEC55821.2022.9926365
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Xplore POP ALL
IEEE Xplore All Conference Proceedings
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP All) 1998-Present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Computer Science
EISBN 1665497866
9781665497862
EISSN 2643-1971
EndPage 6
ExternalDocumentID 9926365
Genre orig-research
GroupedDBID 6IE
6IL
6IN
ABLEC
ADZIZ
ALMA_UNASSIGNED_HOLDINGS
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CBEJK
CHZPO
IEGSK
OCL
RIE
RIL
ID FETCH-LOGICAL-i203t-c9c45b1c0226d4dbaa4b73cc113e3b132c74b774b1c21101b3a273bd11d90d673
IEDL.DBID RIE
ISICitedReferencesCount 3
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000890048500057&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
IngestDate Wed Aug 27 02:07:55 EDT 2025
IsPeerReviewed false
IsScholarly false
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-i203t-c9c45b1c0226d4dbaa4b73cc113e3b132c74b774b1c21101b3a273bd11d90d673
PageCount 6
ParticipantIDs ieee_primary_9926365
PublicationCentury 2000
PublicationDate 2022-Sept.-19
PublicationDateYYYYMMDD 2022-09-19
PublicationDate_xml – month: 09
  year: 2022
  text: 2022-Sept.-19
  day: 19
PublicationDecade 2020
PublicationTitle IEEE Conference on High Performance Extreme Computing (Online)
PublicationTitleAbbrev HPEC
PublicationYear 2022
Publisher IEEE
Publisher_xml – name: IEEE
SSID ssj0003320733
Score 1.8698062
Snippet pPython seeks to provide a parallel capability that provides good speed-up without sacrificing the ease of programming in Python by implementing partitioned...
SourceID ieee
SourceType Publisher
StartPage 1
SubjectTerms Linux
MPI
Operating systems
Parallel Programming
PGAS
Portable computers
Program processors
Programming
Python
Scalability
Semantics
Title pPython for Parallel Python Programming
URI https://ieeexplore.ieee.org/document/9926365
WOSCitedRecordID wos000890048500057&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07a8MwED6S0KFT2ialbzwUutSJHpZkzSEhU_DQQraglyHQOsF1Av33lWzXpdClg4U5kMRJiNOd7rsP4DHnLEdEpzFyNokTY02oAcl94yjTmknpkppsQqxW6Xotsx48d1gY51ydfOYm4bd-y7c7cwihsqmUhFPO-tAXgjdYrS6eQikJ_IMtCBgjOV1m8xkLOFDvBRIyaTv_YlGpjchi-L_pz2D8g8aLss7OnEPPFRcw_KZjiNrTOYKnffYZKgFE_h4aZaoMLClvUSvLmjysdz_CGF4X85fZMm55EOItQbSKjTQJ09h4LbhNrFYq0YIag0MEU3t30ggv8B82wZ3Dmip_KdEWYyuR5YJewqDYFe4KIoY0JZjLVFhvvnKjkFNMc6eUP31W5dcwCnpv9k2pi02r8s3f4ls4DUsb0iewvINBVR7cPZyYY7X9KB_q_fkCzLKQVw
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEB5qFfRUtZX63oPgxW3z2GSbc2mpWMseKvRW8loo6LbUVvDfm2zjiuDFw4ZlICGTECYzmW8-gLucsxwR1YuRNUmcaKN9DUjuGkuZUkwIm5RkE-lk0pvNRFaDhwoLY60tk89sx_-Wb_lmqbc-VNYVgnDK2R7se-asgNaqIiqUEs9AGGDAGInuKBv0mUeCOj-QkE7o_otHpTQjw8b_JnAMrR88XpRVluYEarY4hcY3IUMUzmcT7lfZp68FELmbaJTJtedJeY2CLNtlYr25EVrwMhxM-6M4MCHEC4LoJtZCJ0xh7bTgJjFKykSlVGvsY5jKOZQ6dQL3Ye0dOqyodNcSZTA2Ahme0jOoF8vCtiFiSFGCueilxhmwXEtkJVPcSunOn5H5OTS93vPVrtjFPKh88bf4Fg5H0-fxfPw4ebqEI7_MPpkCiyuob9Zbew0H-mOzeF_flHv1BYWzk6A
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=proceeding&rft.title=IEEE+Conference+on+High+Performance+Extreme+Computing+%28Online%29&rft.atitle=pPython+for+Parallel+Python+Programming&rft.au=Byun%2C+Chansup&rft.au=Arcand%2C+William&rft.au=Bestor%2C+David&rft.au=Bergeron%2C+Bill&rft.date=2022-09-19&rft.pub=IEEE&rft.eissn=2643-1971&rft.spage=1&rft.epage=6&rft_id=info:doi/10.1109%2FHPEC55821.2022.9926365&rft.externalDocID=9926365