PyPnetCDF: A high level framework for parallel access to netCDF files
A Python tool for manipulating netCDF files in a parallel infrastructure is proposed. The parallel interface, PyPnetCDF, manages netCDF properties in a similar way to the serial version from ScientificPython, but hiding parallelism to the user. Implementations details and capabilities of the develop...
Saved in:
| Published in: | Advances in engineering software (1992) Vol. 41; no. 1; pp. 92 - 98 |
|---|---|
| Main Authors: | , , , |
| Format: | Journal Article |
| Language: | English |
| Published: |
Elsevier Ltd
2010
|
| Subjects: | |
| ISSN: | 0965-9978 |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | A Python tool for manipulating netCDF files in a parallel infrastructure is proposed. The parallel interface, PyPnetCDF, manages netCDF properties in a similar way to the serial version from ScientificPython, but hiding parallelism to the user. Implementations details and capabilities of the developed interfaces are given. Numerical experiments that show the friendly use of the interfaces and their behaviour compared with the native routines, are presented. |
|---|---|
| AbstractList | A Python tool for manipulating netCDF files in a parallel infrastructure is proposed. The parallel interface, PyPnetCDF, manages netCDF properties in a similar way to the serial version from ScientificPython, but hiding parallelism to the user. Implementations details and capabilities of the developed interfaces are given. Numerical experiments that show the friendly use of the interfaces and their behaviour compared with the native routines, are presented. |
| Author | Galiano, Vicente Migallón, Violeta Penadés, Jose Migallón, Héctor |
| Author_xml | – sequence: 1 givenname: Vicente surname: Galiano fullname: Galiano, Vicente organization: Departamento de Fı´sica y Arquitectura de Computadores, Universidad Miguel Hernández, E-03202 Elche, Alicante, Spain – sequence: 2 givenname: Héctor surname: Migallón fullname: Migallón, Héctor organization: Departamento de Fı´sica y Arquitectura de Computadores, Universidad Miguel Hernández, E-03202 Elche, Alicante, Spain – sequence: 3 givenname: Violeta surname: Migallón fullname: Migallón, Violeta organization: Departamento de Ciencia de la Computación e Inteligencia Artificial, Universidad de Alicante, E-03071 Alicante, Spain – sequence: 4 givenname: Jose surname: Penadés fullname: Penadés, Jose email: jpenades@dccia.ua.es organization: Departamento de Ciencia de la Computación e Inteligencia Artificial, Universidad de Alicante, E-03071 Alicante, Spain |
| BookMark | eNqNkEFLAzEQRnOooFb_Q07euk62u5vEg1Brq4JgD3oOaXaiqemmJmnFf-9KBcGLngaG773DOyaDLnRICGVQMGDN-arQ7Q675xRsLkoAWUBTANQDcgSyqUdScnFIjlNaAbAKSnZEZouPRYd5ej2_oBP64p5fqMcdemqjXuN7iK_Uhkg3Omrv-7c2BlOiOdA9Ra3zmE7IgdU-4en3HZKn-exxeju6f7i5m07uR2ZcyTwyYDgDKVoBy6Zdci2EZSC4FU3NuWVCQ1lWnEms9FLbkum6NWUtOedGMizHQ3K2925ieNtiymrtkkHvdYdhm9S4hrIes6ofiv3QxJBSRKs20a11_FAM1FcrtVI_rdRXKwWN6lv16OUv1Lisswtdjtr5_wiu9gLsS-wcRpWMw85g6yKarNrg_pZ8Au7Kj8E |
| CitedBy_id | crossref_primary_10_3390_app132413230 crossref_primary_10_3390_w10101382 |
| Cites_doi | 10.1007/978-3-540-75755-9_31 10.1145/1048935.1050189 10.1007/978-3-540-71351-7_32 |
| ContentType | Journal Article |
| Copyright | 2009 Civil-Comp Ltd. and Elsevier Ltd |
| Copyright_xml | – notice: 2009 Civil-Comp Ltd. and Elsevier Ltd |
| DBID | AAYXX CITATION 7SC 7TB 8FD FR3 H8D JQ2 KR7 L7M L~C L~D |
| DOI | 10.1016/j.advengsoft.2009.06.005 |
| DatabaseName | CrossRef Computer and Information Systems Abstracts Mechanical & Transportation Engineering Abstracts Technology Research Database Engineering Research Database Aerospace Database ProQuest Computer Science Collection Civil Engineering Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional |
| DatabaseTitle | CrossRef Aerospace Database Civil Engineering Abstracts Technology Research Database Computer and Information Systems Abstracts – Academic Mechanical & Transportation Engineering Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Engineering Research Database Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional |
| DatabaseTitleList | Aerospace Database |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Applied Sciences Engineering Computer Science |
| EndPage | 98 |
| ExternalDocumentID | 10_1016_j_advengsoft_2009_06_005 S0965997809001458 |
| GroupedDBID | --K --M -~X .DC .~1 0R~ 1B1 1~. 1~5 23M 4.4 457 4G. 5GY 5VS 7-5 71M 8P~ 9JN AACTN AAEDT AAEDW AAIAV AAIKJ AAKOC AALRI AAOAW AAQFI AAQXK AAXUO AAYFN ABBOA ABFNM ABJNI ABMAC ABXDB ABYKQ ACDAQ ACGFS ACNNM ACRLP ACZNC ADBBV ADEZE ADJOM ADMUD ADTZH AEBSH AECPX AEKER AENEX AFFNX AFKWA AFTJW AGHFR AGUBO AGYEJ AHHHB AHJVU AHZHX AIALX AIEXJ AIKHN AITUG AJBFU AJOXV ALMA_UNASSIGNED_HOLDINGS AMFUW AMRAJ AOUOD ASPBG AVWKF AXJTR AZFZN BJAXD BKOJK BLXMC CS3 EBS EFJIC EFLBG EJD EO8 EO9 EP2 EP3 FDB FEDTE FGOYB FIRID FNPLU FYGXN G-2 G-Q GBLVA GBOLZ HLZ HVGLF HZ~ IHE J1W JJJVA KOM LG9 LY7 M41 MO0 N9A O-L O9- OAUVE OZT P-8 P-9 P2P PC. Q38 R2- RIG ROL RPZ SBC SDF SDG SDP SES SET SEW SPC SPCBC SST SSV SSZ T5K TN5 WUQ XPP ZMT ~G- 9DU AATTM AAXKI AAYWO AAYXX ABWVN ACLOT ACRPL ACVFH ADCNI ADNMO AEIPS AEUPX AFJKZ AFPUW AGQPQ AIGII AIIUN AKBMS AKRWK AKYEP ANKPU APXCP CITATION EFKBS ~HD 7SC 7TB 8FD FR3 H8D JQ2 KR7 L7M L~C L~D |
| ID | FETCH-LOGICAL-c349t-c0c71098d80b6db7a88f1087f86577f18a0224719e4abaf21a5dc259777c91e23 |
| ISICitedReferencesCount | 6 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000272305200014&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0965-9978 |
| IngestDate | Sun Sep 28 02:44:39 EDT 2025 Tue Nov 18 21:22:30 EST 2025 Sat Nov 29 06:18:32 EST 2025 Fri Feb 23 02:24:23 EST 2024 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 1 |
| Keywords | netCDF Performance Dataset Python interface Parallel distribution MPI |
| Language | English |
| License | https://www.elsevier.com/tdm/userlicense/1.0 |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-c349t-c0c71098d80b6db7a88f1087f86577f18a0224719e4abaf21a5dc259777c91e23 |
| Notes | ObjectType-Article-2 SourceType-Scholarly Journals-1 ObjectType-Feature-1 content type line 23 |
| PQID | 35025314 |
| PQPubID | 23500 |
| PageCount | 7 |
| ParticipantIDs | proquest_miscellaneous_35025314 crossref_primary_10_1016_j_advengsoft_2009_06_005 crossref_citationtrail_10_1016_j_advengsoft_2009_06_005 elsevier_sciencedirect_doi_10_1016_j_advengsoft_2009_06_005 |
| PublicationCentury | 2000 |
| PublicationDate | 2010 2010-1-00 20100101 |
| PublicationDateYYYYMMDD | 2010-01-01 |
| PublicationDate_xml | – year: 2010 text: 2010 |
| PublicationDecade | 2010 |
| PublicationTitle | Advances in engineering software (1992) |
| PublicationYear | 2010 |
| Publisher | Elsevier Ltd |
| Publisher_xml | – name: Elsevier Ltd |
| References | Hinsen (bib4) 2002 Rew R, Davis G, Emmerson S, Davies H. NetCDF user’s guide for C; 1997. van Rossum, Drake (bib3) 2003 Beazley DM. SWIG: an easy to use tool for integrating scripting languages with C and C++. In: Proceedings of the fourth USENIX Tcl/Tk workshop, Monterey, CA; 1996. Peterson P. F2PY users guide and reference manual; 2005. Drummond LA, Marques O. The ACTS collection. Robust and high-performance tools for scientific computing: guidelines for tool inclusion and retirement. Tech. Rep. LBNL/PUB-3175, Computational Research Division, Lawrence Berkeley National Laboratory, Berkeley; 2002. Li J, Liao W, Choudhary A, Ross R, Thakur R, Gropp W, et al. Parallel netCDF: a high-performance scientific I/O interface. In: Proceedings of SC2003: high performance networking and computing, Phoenix, AZ; 2003. Drummond, Galiano, Migallón, Penadés (bib12) 2007; 4699 . Rew R, Davis G. The unidata netCDF: software for scientific data access. In: Proceedings of the sixth international conference on interactive information and processing systems for meteorology, oceanography and hydrology, Anaheim, CA; 2001. Gropp, Lusk, Thakur (bib7) 1999 Unidata software page. Software for manipulating or displaying netCDF data. Blackford, Choi, Cleary, D’Azevedo, Demmel, Dhillon (bib14) 1997 Snir, Otto, Huss-Lederman, Walker, Dongarra (bib6) 1998 Miller PJ. PyMPI – an introduction to parallel Python using MPI. Tech. Rep. UCRL-WEB-150152, Lawrence Livermore National Laboratory, Livermore; 2002. Drummond, Galiano, Marques, Migallón, Penadés (bib11) 2007; 4395 10.1016/j.advengsoft.2009.06.005_bib1 10.1016/j.advengsoft.2009.06.005_bib2 10.1016/j.advengsoft.2009.06.005_bib5 Hinsen (10.1016/j.advengsoft.2009.06.005_bib4) 2002 Drummond (10.1016/j.advengsoft.2009.06.005_bib11) 2007; 4395 Snir (10.1016/j.advengsoft.2009.06.005_bib6) 1998 10.1016/j.advengsoft.2009.06.005_bib15 10.1016/j.advengsoft.2009.06.005_bib13 van Rossum (10.1016/j.advengsoft.2009.06.005_bib3) 2003 Drummond (10.1016/j.advengsoft.2009.06.005_bib12) 2007; 4699 10.1016/j.advengsoft.2009.06.005_bib9 Blackford (10.1016/j.advengsoft.2009.06.005_bib14) 1997 10.1016/j.advengsoft.2009.06.005_bib8 Gropp (10.1016/j.advengsoft.2009.06.005_bib7) 1999 10.1016/j.advengsoft.2009.06.005_bib10 |
| References_xml | – year: 1997 ident: bib14 article-title: ScaLAPACK user’s guide – year: 2002 ident: bib4 article-title: ScientificPython user’s guide – reference: Unidata software page. Software for manipulating or displaying netCDF data. – reference: Rew R, Davis G. The unidata netCDF: software for scientific data access. In: Proceedings of the sixth international conference on interactive information and processing systems for meteorology, oceanography and hydrology, Anaheim, CA; 2001. – reference: Rew R, Davis G, Emmerson S, Davies H. NetCDF user’s guide for C; 1997. – year: 1999 ident: bib7 article-title: Using MPI-2: advanced features of the message passing interface – reference: Beazley DM. SWIG: an easy to use tool for integrating scripting languages with C and C++. In: Proceedings of the fourth USENIX Tcl/Tk workshop, Monterey, CA; 1996. – reference: Drummond LA, Marques O. The ACTS collection. Robust and high-performance tools for scientific computing: guidelines for tool inclusion and retirement. Tech. Rep. LBNL/PUB-3175, Computational Research Division, Lawrence Berkeley National Laboratory, Berkeley; 2002. – reference: Li J, Liao W, Choudhary A, Ross R, Thakur R, Gropp W, et al. Parallel netCDF: a high-performance scientific I/O interface. In: Proceedings of SC2003: high performance networking and computing, Phoenix, AZ; 2003. – year: 1998 ident: bib6 article-title: MPI: the complete reference – volume: 4699 start-page: 251 year: 2007 end-page: 259 ident: bib12 article-title: High-level user interfaces for the DOE ACTS collection publication-title: Lect Notes Comput Sci – year: 2003 ident: bib3 article-title: An introduction to python – reference: Peterson P. F2PY users guide and reference manual; 2005. – volume: 4395 start-page: 417 year: 2007 end-page: 425 ident: bib11 article-title: PyACTS: a high-level framework for fast development of high performance applications publication-title: Lect Notes Comput Sci – reference: . – reference: Miller PJ. PyMPI – an introduction to parallel Python using MPI. Tech. Rep. UCRL-WEB-150152, Lawrence Livermore National Laboratory, Livermore; 2002. – volume: 4699 start-page: 251 year: 2007 ident: 10.1016/j.advengsoft.2009.06.005_bib12 article-title: High-level user interfaces for the DOE ACTS collection publication-title: Lect Notes Comput Sci doi: 10.1007/978-3-540-75755-9_31 – ident: 10.1016/j.advengsoft.2009.06.005_bib5 doi: 10.1145/1048935.1050189 – year: 1997 ident: 10.1016/j.advengsoft.2009.06.005_bib14 – year: 2003 ident: 10.1016/j.advengsoft.2009.06.005_bib3 – ident: 10.1016/j.advengsoft.2009.06.005_bib8 – year: 1999 ident: 10.1016/j.advengsoft.2009.06.005_bib7 – year: 2002 ident: 10.1016/j.advengsoft.2009.06.005_bib4 – volume: 4395 start-page: 417 year: 2007 ident: 10.1016/j.advengsoft.2009.06.005_bib11 article-title: PyACTS: a high-level framework for fast development of high performance applications publication-title: Lect Notes Comput Sci doi: 10.1007/978-3-540-71351-7_32 – ident: 10.1016/j.advengsoft.2009.06.005_bib9 – ident: 10.1016/j.advengsoft.2009.06.005_bib2 – ident: 10.1016/j.advengsoft.2009.06.005_bib1 – ident: 10.1016/j.advengsoft.2009.06.005_bib10 – ident: 10.1016/j.advengsoft.2009.06.005_bib15 – year: 1998 ident: 10.1016/j.advengsoft.2009.06.005_bib6 – ident: 10.1016/j.advengsoft.2009.06.005_bib13 |
| SSID | ssj0014021 |
| Score | 1.8766844 |
| Snippet | A Python tool for manipulating netCDF files in a parallel infrastructure is proposed. The parallel interface, PyPnetCDF, manages netCDF properties in a similar... |
| SourceID | proquest crossref elsevier |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 92 |
| SubjectTerms | Dataset MPI netCDF Parallel distribution Python interface |
| Title | PyPnetCDF: A high level framework for parallel access to netCDF files |
| URI | https://dx.doi.org/10.1016/j.advengsoft.2009.06.005 https://www.proquest.com/docview/35025314 |
| Volume | 41 |
| WOSCitedRecordID | wos000272305200014&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVESC databaseName: Elsevier SD Freedom Collection Journals 2021 issn: 0965-9978 databaseCode: AIEXJ dateStart: 19950101 customDbUrl: isFulltext: true dateEnd: 99991231 titleUrlDefault: https://www.sciencedirect.com omitProxy: false ssIdentifier: ssj0014021 providerName: Elsevier |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1Lb9QwELaWLQc48CggKC8fuK2C8rYNp6VsKQhVK1GqvUWOk1RbRdmqSR_8GP4rM34kCxRpEeISraxM1vJ8Go9nPs8Q8qoIYhkJHnuqKlMvZrH0RJxID73dKPcLnkb6ovBndnDAFwsxH42-u7swFzVrGn51JU7_q6phDJSNV2f_Qt39R2EAfoPS4Qlqh-dGip9_mzdlt_t-z1w6x3rEkxqpQZPKEbE0txCLftc1DEvdMxGdUCOnazW1617r1BAFNHW2HAoYTlqw4ZdIHcNqT0KEa2GFDzp-ouOwR0tkgA4c2yUm-zFB_y7SJm_fJOuVqXt8_TtHyB_o5GDIG4CmFuvzGOvxC8th1eE0d6Vm4C_puGSaeEKYvj7ORJvaWD9B0dhb00fP7tymnfVve4IJT5y8lgVsH8e4MrZMKaahkmEf7NmJX3AOOAVf4Aky4TfIVsjg5DUmW9OPs8WnPk0Fh2_dktHN2VLFDIHw-v_7k__ziyeg3ZvDe-SOPZfQqcHTfTIqm21y155RqN0BWhhybUDc2Da5vVbT8gGZ9fh7Q6cU0Uc1-miPPgroow591KCPditqpKhG30PydW92uLvv2V4dnopi0XnKV8jq5QX387TImeS8CnzOKp4mjFUBl-gsskCUscxlFQYyKVSIxQ-ZEkEZRo_IuFk15WNCq6Ri4LSGVcVVzMGSJEVe5GVYRAxU4idPCHMrmClbyB77qdSZYyyeZMPaY59VkWnyJkgGveSpKeaygcxbp6TMOqXG2cwAXxtIv3R6zcBuYzJONuXqvM2iBE4bURDv_NP3n5Jbhs6CMcFnZNydnZfPyU110S3bsxcWrj8AaLa-bQ |
| linkProvider | Elsevier |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=PyPnetCDF%3A+A+high+level+framework+for+parallel+access+to+netCDF+files&rft.jtitle=Advances+in+engineering+software+%281992%29&rft.au=Galiano%2C+Vicente&rft.au=Migall%C3%B3n%2C+H%C3%A9ctor&rft.au=Migall%C3%B3n%2C+Violeta&rft.au=Penad%C3%A9s%2C+Jose&rft.date=2010&rft.pub=Elsevier+Ltd&rft.issn=0965-9978&rft.volume=41&rft.issue=1&rft.spage=92&rft.epage=98&rft_id=info:doi/10.1016%2Fj.advengsoft.2009.06.005&rft.externalDocID=S0965997809001458 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0965-9978&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0965-9978&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0965-9978&client=summon |