A simple differentiable programming language

Automatic differentiation plays a prominent role in scientific computing and in modern machine learning, often in the context of powerful programming systems. The relation of the various embodiments of automatic differentiation to the mathematical notion of derivative is not always entirely clear---...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Proceedings of ACM on programming languages Ročník 4; číslo POPL; s. 1 - 28
Hlavní autoři: Abadi, Martín, Plotkin, Gordon D.
Médium: Journal Article
Jazyk:angličtina
Vydáno: 01.01.2020
ISSN:2475-1421, 2475-1421
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Abstract Automatic differentiation plays a prominent role in scientific computing and in modern machine learning, often in the context of powerful programming systems. The relation of the various embodiments of automatic differentiation to the mathematical notion of derivative is not always entirely clear---discrepancies can arise, sometimes inadvertently. In order to study automatic differentiation in such programming contexts, we define a small but expressive programming language that includes a construct for reverse-mode differentiation. We give operational and denotational semantics for this language. The operational semantics employs popular implementation techniques, while the denotational semantics employs notions of differentiation familiar from real analysis. We establish that these semantics coincide.
AbstractList Automatic differentiation plays a prominent role in scientific computing and in modern machine learning, often in the context of powerful programming systems. The relation of the various embodiments of automatic differentiation to the mathematical notion of derivative is not always entirely clear---discrepancies can arise, sometimes inadvertently. In order to study automatic differentiation in such programming contexts, we define a small but expressive programming language that includes a construct for reverse-mode differentiation. We give operational and denotational semantics for this language. The operational semantics employs popular implementation techniques, while the denotational semantics employs notions of differentiation familiar from real analysis. We establish that these semantics coincide.
Author Abadi, Martín
Plotkin, Gordon D.
Author_xml – sequence: 1
  givenname: Martín
  surname: Abadi
  fullname: Abadi, Martín
  organization: Google Research, USA
– sequence: 2
  givenname: Gordon D.
  surname: Plotkin
  fullname: Plotkin, Gordon D.
  organization: Google Research, USA
BookMark eNplj8FKxDAURYOM4DgO_kJ3bqy-1yRNsxwGHYUBN7ouafpSIm1akrrw7604C9HVPRcOF-4lW4UxEGPXCHeIQt5zrhChPGPrQiiZoyhw9Ysv2DaldwBAzUXF9Zrd7rLkh6mnrPXOUaQwe9MsdYpjF80w-NBlvQndh-noip070yfannLD3h4fXvdP-fHl8LzfHXNbyGrOrTAF8lIb4FYqKJxTRIJajdXCshWtJUAriaMA4Ugr2Zaq0eAqxZvS8g27-dm1cUwpkqun6AcTP2uE-vtnffq5mPkf0_rZzH4MczS-_-d_AVmzVDQ
CitedBy_id crossref_primary_10_1145_3734192_3734195
crossref_primary_10_1145_3571198
crossref_primary_10_1145_3632897
crossref_primary_10_1145_3617178
crossref_primary_10_1016_j_tcs_2021_09_027
crossref_primary_10_1017_S0956796825100051
crossref_primary_10_1017_S096012952300018X
crossref_primary_10_1145_3527634
crossref_primary_10_1145_3729321
crossref_primary_10_70930_tac_aui3j1jk
crossref_primary_10_1177_29498732251339951
crossref_primary_10_1109_ACCESS_2023_3342136
crossref_primary_10_1145_3571236
crossref_primary_10_1145_3571247
crossref_primary_10_3390_asi5030047
crossref_primary_10_1017_S0960129524000215
Cites_doi 10.25080/Majora-92bf1922-003
10.1145/2450153.2450158
10.1007/s10990-008-9037-1
10.1016/j.entcs.2010.08.013
10.1016/0377-0427(94)90294-1
10.1145/3236765
10.1007/978-3-642-54833-8_3
10.1007/978-3-319-21401-6_26
10.1109/GlobalSIP.2013.6736861
10.1090/surv/185
10.1145/2976749.2978318
10.1145/3190508.3190551
10.1017/S0960129504004359
10.1145/1330017.1330018
10.1016/S0304-3975(03)00392-X
10.1007/978-3-319-89366-2_25
10.1016/j.entcs.2012.08.017
10.1007/10720107
10.1007/3-540-45685-6_17
ContentType Journal Article
DBID AAYXX
CITATION
DOI 10.1145/3371106
DatabaseName CrossRef
DatabaseTitle CrossRef
DatabaseTitleList CrossRef
DeliveryMethod fulltext_linktorsrc
Discipline Computer Science
EISSN 2475-1421
EndPage 28
ExternalDocumentID 10_1145_3371106
GroupedDBID AAKMM
AAYFX
AAYXX
ACM
AEFXT
AEJOY
AIKLT
AKRVB
ALMA_UNASSIGNED_HOLDINGS
CITATION
EBS
GUFHI
LHSKQ
M~E
OK1
ROL
ID FETCH-LOGICAL-c258t-c4a21369a03c5702ff7ee4ed9182ff5d4dce01c5e31404fe975d67b90f873b6c3
ISICitedReferencesCount 30
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000685202400039&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 2475-1421
IngestDate Sat Nov 29 07:47:58 EST 2025
Tue Nov 18 21:02:26 EST 2025
IsDoiOpenAccess false
IsOpenAccess true
IsPeerReviewed true
IsScholarly true
Issue POPL
Language English
LinkModel OpenURL
MergedId FETCHMERGED-LOGICAL-c258t-c4a21369a03c5702ff7ee4ed9182ff5d4dce01c5e31404fe975d67b90f873b6c3
OpenAccessLink https://dl.acm.org/doi/pdf/10.1145/3371106
PageCount 28
ParticipantIDs crossref_primary_10_1145_3371106
crossref_citationtrail_10_1145_3371106
PublicationCentury 2000
PublicationDate 2020-01-00
PublicationDateYYYYMMDD 2020-01-01
PublicationDate_xml – month: 01
  year: 2020
  text: 2020-01-00
PublicationDecade 2020
PublicationTitle Proceedings of ACM on programming languages
PublicationYear 2020
References Cockett J Robin B (e_1_2_1_15_1) 2011; 25
Selsam Daniel (e_1_2_1_35_1) 2017
Maclaurin Dougal (e_1_2_1_30_1) 2015; 238
van Merrienboer Bart (e_1_2_1_44_1) 2018
Bertot Yves (e_1_2_1_9_1)
Griewank Andreas (e_1_2_1_26_1)
Tokui Seiya (e_1_2_1_41_1) 2015; 5
Baydin Atilim Günes (e_1_2_1_6_1) 2016
Blute Richard (e_1_2_1_10_1) 2010
e_1_2_1_20_1
e_1_2_1_40_1
Agrawal Akshay (e_1_2_1_3_1) 2019
Gianantonio Pietro Di (e_1_2_1_17_1)
e_1_2_1_46_1
e_1_2_1_21_1
e_1_2_1_27_1
e_1_2_1_28_1
Goodfellow Ian (e_1_2_1_25_1)
Fischer H. (e_1_2_1_23_1)
Trench W.F. (e_1_2_1_42_1)
Baydin Atilim Günes (e_1_2_1_5_1) 2018; 18
Kriegl Andreas (e_1_2_1_29_1)
Vákár M. (e_1_2_1_43_1)
e_1_2_1_7_1
e_1_2_1_31_1
e_1_2_1_8_1
Frostig Roy (e_1_2_1_24_1) 2018
Wang Fei (e_1_2_1_45_1) 2018
e_1_2_1_12_1
e_1_2_1_4_1
e_1_2_1_13_1
e_1_2_1_34_1
e_1_2_1_1_1
e_1_2_1_33_1
e_1_2_1_32_1
Shaikhha Amir (e_1_2_1_36_1) 2018
e_1_2_1_16_1
e_1_2_1_39_1
Felleisen Matthias (e_1_2_1_22_1) 1987
e_1_2_1_38_1
Abramsky Samson (e_1_2_1_2_1)
Clarke Frank H. (e_1_2_1_14_1)
Blute Richard F (e_1_2_1_11_1) 2009; 22
e_1_2_1_18_1
e_1_2_1_19_1
References_xml – ident: e_1_2_1_8_1
  doi: 10.25080/Majora-92bf1922-003
– volume-title: Interactive theorem proving and program development: Coq’Art: the calculus of inductive constructions
  ident: e_1_2_1_9_1
– ident: e_1_2_1_27_1
  doi: 10.1145/2450153.2450158
– ident: e_1_2_1_38_1
  doi: 10.1007/s10990-008-9037-1
– volume: 5
  volume-title: Proceedings of Workshop on Machine Learning Systems (LearningSys) in the Twenty-Ninth Conference on Neural Information Processing Systems (NIPS)
  year: 2015
  ident: e_1_2_1_41_1
– volume-title: Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018
  year: 2018
  ident: e_1_2_1_44_1
– ident: e_1_2_1_12_1
  doi: 10.1016/j.entcs.2010.08.013
– volume-title: Handbook of Logic in Computer Science (Vol. 3), Samson Abramsky, Dov M
  ident: e_1_2_1_2_1
– volume-title: Evaluating derivatives - principles and techniques of algorithmic differentiation. Frontiers in applied mathematics
  ident: e_1_2_1_26_1
– ident: e_1_2_1_7_1
  doi: 10.1016/0377-0427(94)90294-1
– volume: 238
  volume-title: ICML 2015 AutoML Workshop
  year: 2015
  ident: e_1_2_1_30_1
– volume: 22
  start-page: 622
  year: 2009
  ident: e_1_2_1_11_1
  article-title: Cartesian differential categories
  publication-title: Theory and Applications of Categories
– ident: e_1_2_1_21_1
  doi: 10.1145/3236765
– ident: e_1_2_1_39_1
  doi: 10.1007/978-3-642-54833-8_3
– ident: e_1_2_1_16_1
  doi: 10.1007/978-3-319-21401-6_26
– volume: 18
  start-page: 1
  year: 2018
  ident: e_1_2_1_5_1
  article-title: Automatic differentiation in machine learning: a survey
  publication-title: Journal of Machine Learning Research
– ident: e_1_2_1_40_1
  doi: 10.1109/GlobalSIP.2013.6736861
– ident: e_1_2_1_28_1
  doi: 10.1090/surv/185
– ident: e_1_2_1_1_1
  doi: 10.1145/2976749.2978318
– volume-title: Optimization and nonsmooth analysis. Classics in Applied Mathematics
  ident: e_1_2_1_14_1
– volume-title: Encyclopedia of Optimization
  ident: e_1_2_1_23_1
– volume-title: The convenient setting of global analysis
  ident: e_1_2_1_29_1
– volume-title: A convenient differential category. arXiv preprint arXiv:1006.3140
  year: 2010
  ident: e_1_2_1_10_1
– ident: e_1_2_1_46_1
  doi: 10.1145/3190508.3190551
– volume-title: Foundations of Software Science and Computation Structures
  ident: e_1_2_1_17_1
– volume-title: Introduction to Real Analysis
  ident: e_1_2_1_42_1
– volume-title: Deep Learning
  ident: e_1_2_1_25_1
– volume-title: Simon Peyton Jones, and Christoph Koch
  year: 2018
  ident: e_1_2_1_36_1
– volume-title: Matthew James Johnson, and Chris Leary
  year: 2018
  ident: e_1_2_1_24_1
– volume-title: Domains XIII Workshop.
  ident: e_1_2_1_43_1
– volume-title: Friedman
  year: 1987
  ident: e_1_2_1_22_1
– ident: e_1_2_1_18_1
  doi: 10.1017/S0960129504004359
– volume-title: Proceedings of the 34th International Conference on Machine Learning, ICML 2017
  year: 2017
  ident: e_1_2_1_35_1
– volume-title: Tricks from deep learning. CoRR abs/1611.03777
  year: 2016
  ident: e_1_2_1_6_1
– ident: e_1_2_1_34_1
  doi: 10.1145/1330017.1330018
– ident: e_1_2_1_20_1
  doi: 10.1016/S0304-3975(03)00392-X
– ident: e_1_2_1_33_1
– volume-title: Advances in Neural Information Processing Systems 9, NIPS
  ident: e_1_2_1_4_1
– ident: e_1_2_1_19_1
  doi: 10.1007/978-3-319-89366-2_25
– volume-title: Demystifying differentiable programming: shift/reset the penultimate backpropagator. CoRR abs/1803.10228
  year: 2018
  ident: e_1_2_1_45_1
– volume: 25
  start-page: 537
  year: 2011
  ident: e_1_2_1_15_1
  article-title: Differential restriction categories
  publication-title: Theory and Applications of Categories
– ident: e_1_2_1_31_1
  doi: 10.1016/j.entcs.2012.08.017
– volume-title: Alexandre Passos, Allen Lavoie, Ashish Agarwal, Asim Shankar, Igor Ganichev, Josh Levenberg, Mingsheng Hong, Rajat Monga, et al.
  year: 2019
  ident: e_1_2_1_3_1
– ident: e_1_2_1_13_1
  doi: 10.1007/10720107
– ident: e_1_2_1_32_1
  doi: 10.1007/3-540-45685-6_17
SSID ssj0001934839
Score 2.3946006
Snippet Automatic differentiation plays a prominent role in scientific computing and in modern machine learning, often in the context of powerful programming systems....
SourceID crossref
SourceType Enrichment Source
Index Database
StartPage 1
Title A simple differentiable programming language
Volume 4
WOSCitedRecordID wos000685202400039&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVHPJ
  databaseName: ROAD: Directory of Open Access Scholarly Resources
  customDbUrl:
  eissn: 2475-1421
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0001934839
  issn: 2475-1421
  databaseCode: M~E
  dateStart: 20170101
  isFulltext: true
  titleUrlDefault: https://road.issn.org
  providerName: ISSN International Centre
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV3LT4MwGG_m9ODFt_EdDsaLMoG2lB7J5iPGB4eZ7LZAW5Ilky0Tl538222hYJ1L1IMXQpp-DfArX3_9-j0AOHVij1DiYhsy4thIIGwHCCLlZc656_oe4qgoNkEeH4Nej0aNxl0VCzMdkiwLZjM6_leoZZsEW4XO_gHuelDZIO8l6PIqYZfXXwEfnr8OVMrfuvaJ_IdVeJT2xHpRtoHKSmlS06heygrvjrD9oA4SFknVNDxMYj7QET95ceTeqedaNBzlutTXjdzgyqE6LdPE4DmGiaHQRB4i2HZRGcrcEgvatCpFxoyJnqJ7QzO6xhJbhoN_V95I5bmAkEhKsiA99tyyVTsTlqHVuK8Fl8CyRzBV7n0P74a9jUIUFIXl6icvY6iV7KWWNciJwTK6G2BNbw-ssIR1EzREtgXWq9IbltbE2-AitEqUra8oWwZeVoXXDni-vuq2b21d-MJmHg5ym6HYc6FPYwcyTBwvTYkQSHAqN4NpijniTDguwwKq5EipoARznyTUSQMCE5_BXdDMRpnYA5YUTIQinVyR9zilTgz9RFEUzpnkrvvgrHrjPtNZ4VVxkmF_7rPuA6vuOC4Tocx3Ofi5yyFY_ZxgR6CZT97EMVhh03zwOjkpEPsAby1Mpw
linkProvider ISSN International Centre
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+simple+differentiable+programming+language&rft.jtitle=Proceedings+of+ACM+on+programming+languages&rft.au=Abadi%2C+Mart%C3%ADn&rft.au=Plotkin%2C+Gordon+D.&rft.date=2020-01-01&rft.issn=2475-1421&rft.eissn=2475-1421&rft.volume=4&rft.issue=POPL&rft.spage=1&rft.epage=28&rft_id=info:doi/10.1145%2F3371106&rft.externalDBID=n%2Fa&rft.externalDocID=10_1145_3371106
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2475-1421&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2475-1421&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2475-1421&client=summon