Forward- or reverse-mode automatic differentiation: What's the difference?
Automatic differentiation (AD) has been a topic of interest for researchers in many disciplines, with increased popularity since its application to machine learning and neural networks. Although many researchers appreciate and know how to apply AD, it remains a challenge to truly understand the unde...
Uloženo v:
| Vydáno v: | Science of computer programming Ročník 231; s. 103010 |
|---|---|
| Hlavní autoři: | , , , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
Elsevier B.V
01.01.2024
|
| Témata: | |
| ISSN: | 0167-6423, 1872-7964 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | Automatic differentiation (AD) has been a topic of interest for researchers in many disciplines, with increased popularity since its application to machine learning and neural networks. Although many researchers appreciate and know how to apply AD, it remains a challenge to truly understand the underlying processes. From an algebraic point of view, however, AD appears surprisingly natural: it originates from the differentiation laws. In this work we use Algebra of Programming techniques to reason about different AD variants, leveraging Haskell to illustrate our observations. Our findings stem from three fundamental algebraic abstractions: (1) the notion of semimodule, (2) Nagata's construction of the ‘idealization of a module’, and (3) Kronecker's delta function, that together allow us to write a single-line abstract definition of AD. From this single-line definition, and by instantiating our algebraic structures in various ways, we derive different AD variants, that have the same extensional behaviour, but different intensional properties, mainly in terms of (asymptotic) computational complexity. We show the different variants equivalent by means of Kronecker isomorphisms, a further elaboration of our Haskell infrastructure which guarantees correctness by construction. With this framework in place, this paper seeks to make AD variants more comprehensible, taking an algebraic perspective on the matter.
•Basic forward mode AD is the fusion of two semiring homomorphisms: symbolic differentiation and evaluation.•Three fundamental algebraic abstractions lay the foundations of a single-line definition of AD algorithms.•Different AD algorithms can be obtained using isomorphisms.•Our approach supports common extensions: more primitives, overloading, let-sharing, higher derivatives, non-commutativity. |
|---|---|
| AbstractList | Automatic differentiation (AD) has been a topic of interest for researchers in many disciplines, with increased popularity since its application to machine learning and neural networks. Although many researchers appreciate and know how to apply AD, it remains a challenge to truly understand the underlying processes. From an algebraic point of view, however, AD appears surprisingly natural: it originates from the differentiation laws. In this work we use Algebra of Programming techniques to reason about different AD variants, leveraging Haskell to illustrate our observations. Our findings stem from three fundamental algebraic abstractions: (1) the notion of semimodule, (2) Nagata's construction of the ‘idealization of a module’, and (3) Kronecker's delta function, that together allow us to write a single-line abstract definition of AD. From this single-line definition, and by instantiating our algebraic structures in various ways, we derive different AD variants, that have the same extensional behaviour, but different intensional properties, mainly in terms of (asymptotic) computational complexity. We show the different variants equivalent by means of Kronecker isomorphisms, a further elaboration of our Haskell infrastructure which guarantees correctness by construction. With this framework in place, this paper seeks to make AD variants more comprehensible, taking an algebraic perspective on the matter.
•Basic forward mode AD is the fusion of two semiring homomorphisms: symbolic differentiation and evaluation.•Three fundamental algebraic abstractions lay the foundations of a single-line definition of AD algorithms.•Different AD algorithms can be obtained using isomorphisms.•Our approach supports common extensions: more primitives, overloading, let-sharing, higher derivatives, non-commutativity. |
| ArticleNumber | 103010 |
| Author | van den Berg, Birthe McKinna, James Schrijvers, Tom Vandenbroucke, Alexander |
| Author_xml | – sequence: 1 givenname: Birthe orcidid: 0000-0002-0088-9546 surname: van den Berg fullname: van den Berg, Birthe email: birthe.vandenberg@kuleuven.be organization: KU Leuven, Celestijnenlaan 200A, Leuven, Belgium – sequence: 2 givenname: Tom surname: Schrijvers fullname: Schrijvers, Tom email: tom.schrijvers@kuleuven.be organization: KU Leuven, Celestijnenlaan 200A, Leuven, Belgium – sequence: 3 givenname: James surname: McKinna fullname: McKinna, James email: j.mckinna@hw.ac.uk organization: Heriot-Watt University, Earl Mountbatten Building G.52, Edinburgh EH14 4AS, United Kingdom – sequence: 4 givenname: Alexander surname: Vandenbroucke fullname: Vandenbroucke, Alexander email: alexander.vandenbroucke@sc.com organization: Standard Chartered, 1 Basinghall Avenue, London, EC2V 5DD, United Kingdom |
| BookMark | eNqFkE1LAzEQhoNUsK3-Ai9787Q1X91kBREprR8UvCgeQzaZ0JR2I0ms-O_dbcWDBz0NM_M-w7zvCA3a0AJC5wRPCCbV5XqSjDdhQjFl3YRhgo_QkEhBS1FXfICGnUqUFafsBI1SWmOMKy7IED0uQvzQ0ZZFiEWEHcQE5TZYKPR7DludvSmsdw4itNl3bWiviteVzhepyCv42Rm4OUXHTm8SnH3XMXpZzJ9n9-Xy6e5hdrssDeMylw1zhBJHsXGW10yaaS0wqwlpBHDb1FZqahwXjeQaDDUcLG6c1jWRbEqlYWNUH-6aGFKK4JTxef9ZjtpvFMGqD0Wt1T4U1YeiDqF0LPvFvkW_1fHzH-r6QEFna-ch9pres_URTFY2-D_5L-9HfyI |
| CitedBy_id | crossref_primary_10_4271_03_18_02_0010 crossref_primary_10_1145_3747524 crossref_primary_10_1016_j_cmpb_2025_108826 crossref_primary_10_1145_3632878 crossref_primary_10_1016_j_amc_2025_129677 crossref_primary_10_1016_j_cpc_2025_109832 crossref_primary_10_1109_TAP_2025_3575190 |
| Cites_doi | 10.1145/3236765 10.1162/neco.1994.6.1.147 10.1145/3341700 10.1145/3473583 10.1145/355586.364791 10.1145/3527333 10.1145/1330017.1330018 10.1145/3236779 10.1017/S0956796899003500 10.1007/s11075-015-0067-6 10.1016/0020-0190(86)90059-1 10.1145/3498710 10.1017/S0956796809007175 10.1007/BF00289507 10.1017/S0962492902000132 10.1017/S095679681900008X 10.1216/JCA-2009-1-1-3 |
| ContentType | Journal Article |
| Copyright | 2023 The Author(s) |
| Copyright_xml | – notice: 2023 The Author(s) |
| DBID | 6I. AAFTH AAYXX CITATION |
| DOI | 10.1016/j.scico.2023.103010 |
| DatabaseName | ScienceDirect Open Access Titles Elsevier:ScienceDirect:Open Access CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| EISSN | 1872-7964 |
| ExternalDocumentID | 10_1016_j_scico_2023_103010 S0167642323000928 |
| GroupedDBID | --K --M .DC .~1 0R~ 123 1B1 1RT 1~. 1~5 4.4 457 4G. 5VS 6I. 7-5 71M 8P~ 9JN AACTN AAEDT AAEDW AAFTH AAIAV AAIKJ AAKOC AALRI AAOAW AAQFI AAQXK AAXUO AAYFN ABBOA ABFNM ABJNI ABMAC ABTAH ABVKL ABXDB ABYKQ ACDAQ ACGFS ACNNM ACRLP ACZNC ADBBV ADEZE ADHUB ADMUD AEBSH AEKER AENEX AEXQZ AFFNX AFKWA AFTJW AGHFR AGUBO AGYEJ AHHHB AHZHX AIALX AIEXJ AIKHN AITUG AJBFU AJOXV ALMA_UNASSIGNED_HOLDINGS AMFUW AMRAJ AOUOD ASPBG AVWKF AXJTR AZFZN BKOJK BLXMC CS3 DU5 E.L EBS EFJIC EFLBG EJD EO8 EO9 EP2 EP3 FDB FEDTE FGOYB FIRID FNPLU FYGXN G-2 G-Q GBLVA GBOLZ HVGLF HZ~ IHE IXB J1W KOM LG9 M26 M41 MO0 N9A NCXOZ O-L O9- OAUVE OK1 OZT P-8 P-9 P2P PC. Q38 R2- RIG ROL RPZ SDF SDG SDP SES SEW SPC SPCBC SSV SSZ T5K TN5 WUQ XPP ZMT ZY4 ~G- 9DU AATTM AAXKI AAYWO AAYXX ABWVN ACLOT ACRPL ACVFH ADCNI ADNMO ADVLN AEIPS AEUPX AFJKZ AFPUW AGQPQ AIGII AIIUN AKBMS AKRWK AKYEP ANKPU APXCP CITATION EFKBS ~HD |
| ID | FETCH-LOGICAL-c348t-b3f121f20cfd4938c59703911b7e4db9d8a2cf47b84aec2c4ed0bfaa9183528c3 |
| ISICitedReferencesCount | 7 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001071219400001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0167-6423 |
| IngestDate | Sat Nov 29 07:25:45 EST 2025 Tue Nov 18 22:44:05 EST 2025 Fri Feb 23 02:33:49 EST 2024 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Keywords | Cayley representation Nagata idealization Kronecker Semimodules Automatic differentiation |
| Language | English |
| License | This is an open access article under the CC BY license. |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-c348t-b3f121f20cfd4938c59703911b7e4db9d8a2cf47b84aec2c4ed0bfaa9183528c3 |
| ORCID | 0000-0002-0088-9546 |
| OpenAccessLink | https://dx.doi.org/10.1016/j.scico.2023.103010 |
| ParticipantIDs | crossref_citationtrail_10_1016_j_scico_2023_103010 crossref_primary_10_1016_j_scico_2023_103010 elsevier_sciencedirect_doi_10_1016_j_scico_2023_103010 |
| PublicationCentury | 2000 |
| PublicationDate | January 2024 2024-01-00 |
| PublicationDateYYYYMMDD | 2024-01-01 |
| PublicationDate_xml | – month: 01 year: 2024 text: January 2024 |
| PublicationDecade | 2020 |
| PublicationTitle | Science of computer programming |
| PublicationYear | 2024 |
| Publisher | Elsevier B.V |
| Publisher_xml | – name: Elsevier B.V |
| References | Bird, de Moor (br0050) 1997 Bird (br0460) 2014 von zur Gathen, Gerhard (br0160) 2013 Clifford (br0040) 1873; 4 Meijer (br0110) 1992 Elsman, Henglein, Kaarsgaard, Mathiesen, Schenck (br0380) 2022; vol. 360 Gill, Hutton (br0170) 2009; 19 Huot, Staton, Vákár (br0330) 2022; 18 Boisseau, Gibbons (br0240) 2018; 2 Huot, Staton, Vákár (br0320) 2020; vol. 12077 Wadler, Blott (br0450) 1989 Szirmay-Kalos (br0130) 2021; 65 Hughes (br0260) 1986; 22 Elliott (br0370) 2021; 5 Elliott (br0350) 2009 Wang, Zheng, Decker, Wu, Essertel, Rompf (br0390) 2019; 3 Elliott (br0360) 2018; 2 Reynolds (br0200) 1983 Abadi, Barham, Chen, Chen, Davis, Dean, Devin, Ghemawat, Irving, Isard, Kudlur, Levenberg, Monga, Moore, Murray, Steiner, Tucker, Vasudevan, Warden, Wicke, Yu, Zhang (br0010) 2016 Anderson, Winders (br0430) 2009; 1 Smeding, Vákár (br0340) 2022 Golan (br0150) 1999 Clark, Tärnlund (br0250) 1977; 77 Meijer, Fokkinga, Paterson (br0100) 1991 Gibbons, Wu (br0280) 2014 Hoffmann (br0140) 2016; 72 Griewank (br0440) 2003; 12 Elliott (br0070) March 2009 Jansson, Ionescu, Bernardy (br0080) 2022; vol. 24 Manzyuk, Pearlmutter, Radul, Rush, Siskind (br0420) 2019; 29 Hinze (br0300) 2008 Hutton (br0470) 2016 Wengert (br0030) 1964; 7 Shaikhha, Huot, Smith, Olteanu (br0210) apr 2022; 6 Nagata (br0060) 1962 Kmett (br0400) 2021 Wadler (br0190) 1989 Hoare (br0180) 1972; 1 Gill, Launchbury, Peyton Jones (br0270) 1993 Hutton (br0090) 1999; 9 Krawiec, Peyton Jones, Krishnaswami, Ellis, Eisenberg, Fitzgibbon (br0120) 2022; 6 Atiyah, MacDonald (br0220) 1969 Pearlmutter (br0310) 1994; 6 Karczmarczuk (br0290) ICFP '98, Baltimore, Maryland, USA, September 27–29, 1998 Hinze (br0230) 2012; vol. 7342 Baydin, Pearlmutter, Radul, Siskind (br0020) 2017; 18 Pearlmutter, Siskind (br0410) 2008; 30 Clark (10.1016/j.scico.2023.103010_br0250) 1977; 77 Baydin (10.1016/j.scico.2023.103010_br0020) 2017; 18 Wengert (10.1016/j.scico.2023.103010_br0030) 1964; 7 Hughes (10.1016/j.scico.2023.103010_br0260) 1986; 22 Clifford (10.1016/j.scico.2023.103010_br0040) 1873; 4 Bird (10.1016/j.scico.2023.103010_br0050) 1997 Hoffmann (10.1016/j.scico.2023.103010_br0140) 2016; 72 Boisseau (10.1016/j.scico.2023.103010_br0240) 2018; 2 Gibbons (10.1016/j.scico.2023.103010_br0280) 2014 Shaikhha (10.1016/j.scico.2023.103010_br0210) 2022; 6 Gill (10.1016/j.scico.2023.103010_br0270) 1993 Elliott (10.1016/j.scico.2023.103010_br0350) 2009 Elliott (10.1016/j.scico.2023.103010_br0070) 2009 Hutton (10.1016/j.scico.2023.103010_br0090) 1999; 9 Griewank (10.1016/j.scico.2023.103010_br0440) 2003; 12 Reynolds (10.1016/j.scico.2023.103010_br0200) 1983 Elliott (10.1016/j.scico.2023.103010_br0370) 2021; 5 Pearlmutter (10.1016/j.scico.2023.103010_br0410) 2008; 30 Manzyuk (10.1016/j.scico.2023.103010_br0420) 2019; 29 Karczmarczuk (10.1016/j.scico.2023.103010_br0290) 1998 Szirmay-Kalos (10.1016/j.scico.2023.103010_br0130) 2021; 65 Pearlmutter (10.1016/j.scico.2023.103010_br0310) 1994; 6 Hinze (10.1016/j.scico.2023.103010_br0230) 2012; vol. 7342 Huot (10.1016/j.scico.2023.103010_br0330) 2022; 18 Golan (10.1016/j.scico.2023.103010_br0150) 1999 von zur Gathen (10.1016/j.scico.2023.103010_br0160) 2013 Atiyah (10.1016/j.scico.2023.103010_br0220) 1969 Abadi (10.1016/j.scico.2023.103010_br0010) Wadler (10.1016/j.scico.2023.103010_br0190) 1989 Elliott (10.1016/j.scico.2023.103010_br0360) 2018; 2 Hutton (10.1016/j.scico.2023.103010_br0470) 2016 Wang (10.1016/j.scico.2023.103010_br0390) 2019; 3 Gill (10.1016/j.scico.2023.103010_br0170) 2009; 19 Hinze (10.1016/j.scico.2023.103010_br0300) 2008 Huot (10.1016/j.scico.2023.103010_br0320) 2020; vol. 12077 Smeding (10.1016/j.scico.2023.103010_br0340) Jansson (10.1016/j.scico.2023.103010_br0080) 2022; vol. 24 Anderson (10.1016/j.scico.2023.103010_br0430) 2009; 1 Bird (10.1016/j.scico.2023.103010_br0460) 2014 Krawiec (10.1016/j.scico.2023.103010_br0120) 2022; 6 Elsman (10.1016/j.scico.2023.103010_br0380) 2022; vol. 360 Nagata (10.1016/j.scico.2023.103010_br0060) 1962 Meijer (10.1016/j.scico.2023.103010_br0100) 1991 Wadler (10.1016/j.scico.2023.103010_br0450) 1989 Meijer (10.1016/j.scico.2023.103010_br0110) 1992 Kmett (10.1016/j.scico.2023.103010_br0400) Hoare (10.1016/j.scico.2023.103010_br0180) 1972; 1 |
| References_xml | – volume: 72 start-page: 775 year: 2016 end-page: 811 ident: br0140 article-title: A hitchhiker's guide to automatic differentiation publication-title: Numer. Algorithms – year: 2013 ident: br0160 article-title: Modern Computer Algebra – volume: 2 start-page: 84:1 year: 2018 end-page: 84:27 ident: br0240 article-title: What you needa know about Yoneda: profunctor optics and the Yoneda lemma (Functional Pearl) publication-title: Proc. ACM Program. Lang. – volume: 7 start-page: 463 year: 1964 end-page: 464 ident: br0030 article-title: A simple automatic derivative evaluation program publication-title: Commun. ACM – start-page: 223 year: 1993 end-page: 232 ident: br0270 article-title: A short cut to deforestation publication-title: Proceedings of the Conference on Functional Programming Languages and Computer Architecture – start-page: 149 year: 1999 end-page: 161 ident: br0150 article-title: Semimodules over Semirings – year: 2021 ident: br0400 article-title: ad: automatic differentiation – year: March 2009 ident: br0070 article-title: Denotational design with type class morphisms (extended version) – volume: 3 start-page: 96:1 year: 2019 end-page: 96:31 ident: br0390 article-title: Demystifying differentiable programming: shift/reset the penultimate backpropagator publication-title: Proc. ACM Program. Lang. – volume: 6 year: apr 2022 ident: br0210 article-title: Functional collection programming with semi-ring dictionaries publication-title: Proc. ACM Program. Lang. – volume: 4 start-page: 381 year: 1873 end-page: 395 ident: br0040 article-title: Preliminary sketch of bi-quaternions publication-title: Proc. Lond. Math. Soc. – year: 1969 ident: br0220 article-title: Introduction to Commutative Algebra – volume: vol. 360 start-page: 1 year: 2022 end-page: 26 ident: br0380 article-title: Combinatory adjoints and differentiation publication-title: Proceedings Ninth Workshop on Mathematically Structured Functional Programming – start-page: 124 year: 1991 end-page: 144 ident: br0100 article-title: Functional programming with bananas, lenses, envelopes and barbed wire publication-title: Proceedings of the 5th ACM Conference on Functional Programming Languages and Computer Architecture – volume: vol. 12077 start-page: 319 year: 2020 end-page: 338 ident: br0320 article-title: Correctness of automatic differentiation via diffeologies and categorical gluing publication-title: Foundations of Software Science and Computation Structures – Proceedings of the 23rd International Conference, FOSSACS 2020, Held as Part of the European Joint Conferences on Theory and Practice of Software – volume: 18 start-page: 5595 year: 2017 end-page: 5637 ident: br0020 article-title: Automatic differentiation in machine learning: a survey publication-title: J. Mach. Learn. Res. – year: 1997 ident: br0050 article-title: Algebra of Programming – volume: 18 year: 2022 ident: br0330 article-title: Higher order automatic differentiation of higher order functions publication-title: Log. Methods Comput. Sci. – year: 2022 ident: br0340 article-title: Efficient dual-numbers reverse AD via well-known program transformations – volume: 29 start-page: e12 year: 2019 ident: br0420 article-title: Perturbation confusion in forward automatic differentiation of higher-order functions publication-title: J. Funct. Program. – volume: 6 start-page: 1 year: 2022 end-page: 30 ident: br0120 article-title: Provably correct, asymptotically efficient, higher-order reverse-mode automatic differentiation publication-title: Proc. ACM Program. Lang. – volume: 65 start-page: 1 year: 2021 end-page: 10 ident: br0130 article-title: Higher order automatic differentiation with dual numbers publication-title: Period. Polytech. Electr. Eng. Comput. Sci. – start-page: 195 year: ICFP '98, Baltimore, Maryland, USA, September 27–29, 1998 end-page: 203 ident: br0290 article-title: Functional differentiation of computer programs publication-title: Proceedings of the Third ACM SIGPLAN International Conference on Functional Programming – volume: 30 start-page: 7:1 year: 2008 end-page: 7:36 ident: br0410 article-title: Reverse-mode AD in a functional framework: lambda the ultimate backpropagator publication-title: ACM Trans. Program. Lang. Syst. – volume: vol. 24 year: 2022 ident: br0080 article-title: Domain-Specific Languages of Mathematics publication-title: Texts in Computing – start-page: 347 year: 1989 end-page: 359 ident: br0190 article-title: Theorems for free! publication-title: Proceedings of the Fourth International Conference on Functional Programming Languages and Computer Architecture – start-page: 189 year: 2008 end-page: 200 ident: br0300 article-title: Functional pearl: streams and unique fixed points publication-title: Proceedings of the 13th ACM SIGPLAN International Conference on Functional Programming – year: 2016 ident: br0010 article-title: Tensorflow: a system for large-scale machine learning – volume: 9 start-page: 355 year: 1999 end-page: 372 ident: br0090 article-title: A tutorial on the universality and expressiveness of fold publication-title: J. Funct. Program. – volume: 19 start-page: 227 year: 2009 end-page: 251 ident: br0170 article-title: The worker/wrapper transformation publication-title: J. Funct. Program. – volume: 77 year: 1977 ident: br0250 article-title: A first order theory of data and programs publication-title: Inf. Process. – volume: 12 start-page: 321 year: 2003 end-page: 398 ident: br0440 article-title: A mathematical view of automatic differentiation publication-title: Acta Numer. – year: 2014 ident: br0460 article-title: Thinking Functionally with Haskell – volume: 1 start-page: 271 year: 1972 end-page: 281 ident: br0180 article-title: Proof of correctness of data representations publication-title: Acta Inform. – volume: 6 start-page: 147 year: 1994 end-page: 160 ident: br0310 article-title: Fast exact multiplication by the Hessian publication-title: Neural Comput. – year: 1992 ident: br0110 article-title: Calculating compilers – start-page: 339 year: 2014 end-page: 347 ident: br0280 article-title: Folding domain-specific languages: deep and shallow embeddings (Functional Pearl) publication-title: Proceedings of the 19th ACM SIGPLAN International Conference on Functional Programming – volume: 1 start-page: 3 year: 2009 end-page: 56 ident: br0430 article-title: Idealization of a module publication-title: J. Commut. Algebra – start-page: 60 year: 1989 end-page: 76 ident: br0450 article-title: How to make ad-hoc polymorphism less ad hoc publication-title: Proceedings of the 16th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages – volume: 5 start-page: 1 year: 2021 end-page: 18 ident: br0370 article-title: Symbolic and automatic differentiation of languages publication-title: Proc. ACM Program. Lang. – year: 1962 ident: br0060 article-title: Local Rings – volume: 22 start-page: 141 year: 1986 end-page: 144 ident: br0260 article-title: A novel representation of lists and its application to the function “reverse” publication-title: Inf. Process. Lett. – volume: 2 start-page: 70:1 year: 2018 end-page: 70:29 ident: br0360 article-title: The simple essence of automatic differentiation publication-title: Proc. ACM Program. Lang. – start-page: 191 year: 2009 end-page: 202 ident: br0350 article-title: Beautiful differentiation publication-title: Proceeding of the 14th ACM SIGPLAN International Conference on Functional Programming – start-page: 513 year: 1983 end-page: 523 ident: br0200 article-title: Types, abstraction and parametric polymorphism publication-title: Information Processing 83, Proceedings of the IFIP 9th World Computer Congress – volume: vol. 7342 start-page: 324 year: 2012 end-page: 362 ident: br0230 article-title: Kan extensions for program optimisation or: art and Dan explain an old trick publication-title: Mathematics of Program Construction – Proceedings of the 11th International Conference – year: 2016 ident: br0470 article-title: Programming in Haskell – volume: 2 start-page: 70:1 issue: ICFP year: 2018 ident: 10.1016/j.scico.2023.103010_br0360 article-title: The simple essence of automatic differentiation publication-title: Proc. ACM Program. Lang. doi: 10.1145/3236765 – year: 2009 ident: 10.1016/j.scico.2023.103010_br0070 – start-page: 347 year: 1989 ident: 10.1016/j.scico.2023.103010_br0190 article-title: Theorems for free! – volume: 6 start-page: 147 issue: 1 year: 1994 ident: 10.1016/j.scico.2023.103010_br0310 article-title: Fast exact multiplication by the Hessian publication-title: Neural Comput. doi: 10.1162/neco.1994.6.1.147 – volume: 3 start-page: 96:1 issue: ICFP year: 2019 ident: 10.1016/j.scico.2023.103010_br0390 article-title: Demystifying differentiable programming: shift/reset the penultimate backpropagator publication-title: Proc. ACM Program. Lang. doi: 10.1145/3341700 – ident: 10.1016/j.scico.2023.103010_br0010 – volume: 4 start-page: 381 year: 1873 ident: 10.1016/j.scico.2023.103010_br0040 article-title: Preliminary sketch of bi-quaternions publication-title: Proc. Lond. Math. Soc. – year: 1962 ident: 10.1016/j.scico.2023.103010_br0060 – year: 1969 ident: 10.1016/j.scico.2023.103010_br0220 – volume: 5 start-page: 1 issue: ICFP year: 2021 ident: 10.1016/j.scico.2023.103010_br0370 article-title: Symbolic and automatic differentiation of languages publication-title: Proc. ACM Program. Lang. doi: 10.1145/3473583 – start-page: 191 year: 2009 ident: 10.1016/j.scico.2023.103010_br0350 article-title: Beautiful differentiation – volume: 65 start-page: 1 issue: 1 year: 2021 ident: 10.1016/j.scico.2023.103010_br0130 article-title: Higher order automatic differentiation with dual numbers publication-title: Period. Polytech. Electr. Eng. Comput. Sci. – volume: 7 start-page: 463 issue: 8 year: 1964 ident: 10.1016/j.scico.2023.103010_br0030 article-title: A simple automatic derivative evaluation program publication-title: Commun. ACM doi: 10.1145/355586.364791 – year: 2013 ident: 10.1016/j.scico.2023.103010_br0160 – volume: 6 issue: OOPSLA1 year: 2022 ident: 10.1016/j.scico.2023.103010_br0210 article-title: Functional collection programming with semi-ring dictionaries publication-title: Proc. ACM Program. Lang. doi: 10.1145/3527333 – ident: 10.1016/j.scico.2023.103010_br0340 – volume: 30 start-page: 7:1 issue: 2 year: 2008 ident: 10.1016/j.scico.2023.103010_br0410 article-title: Reverse-mode AD in a functional framework: lambda the ultimate backpropagator publication-title: ACM Trans. Program. Lang. Syst. doi: 10.1145/1330017.1330018 – volume: 2 start-page: 84:1 issue: ICFP year: 2018 ident: 10.1016/j.scico.2023.103010_br0240 article-title: What you needa know about Yoneda: profunctor optics and the Yoneda lemma (Functional Pearl) publication-title: Proc. ACM Program. Lang. doi: 10.1145/3236779 – year: 2014 ident: 10.1016/j.scico.2023.103010_br0460 – volume: vol. 12077 start-page: 319 year: 2020 ident: 10.1016/j.scico.2023.103010_br0320 article-title: Correctness of automatic differentiation via diffeologies and categorical gluing – volume: vol. 24 year: 2022 ident: 10.1016/j.scico.2023.103010_br0080 article-title: Domain-Specific Languages of Mathematics – volume: 9 start-page: 355 issue: 4 year: 1999 ident: 10.1016/j.scico.2023.103010_br0090 article-title: A tutorial on the universality and expressiveness of fold publication-title: J. Funct. Program. doi: 10.1017/S0956796899003500 – volume: 72 start-page: 775 issue: 3 year: 2016 ident: 10.1016/j.scico.2023.103010_br0140 article-title: A hitchhiker's guide to automatic differentiation publication-title: Numer. Algorithms doi: 10.1007/s11075-015-0067-6 – start-page: 189 year: 2008 ident: 10.1016/j.scico.2023.103010_br0300 article-title: Functional pearl: streams and unique fixed points – ident: 10.1016/j.scico.2023.103010_br0400 – volume: 22 start-page: 141 issue: 3 year: 1986 ident: 10.1016/j.scico.2023.103010_br0260 article-title: A novel representation of lists and its application to the function “reverse” publication-title: Inf. Process. Lett. doi: 10.1016/0020-0190(86)90059-1 – volume: vol. 360 start-page: 1 year: 2022 ident: 10.1016/j.scico.2023.103010_br0380 article-title: Combinatory adjoints and differentiation – volume: 6 start-page: 1 issue: POPL year: 2022 ident: 10.1016/j.scico.2023.103010_br0120 article-title: Provably correct, asymptotically efficient, higher-order reverse-mode automatic differentiation publication-title: Proc. ACM Program. Lang. doi: 10.1145/3498710 – volume: 19 start-page: 227 issue: 2 year: 2009 ident: 10.1016/j.scico.2023.103010_br0170 article-title: The worker/wrapper transformation publication-title: J. Funct. Program. doi: 10.1017/S0956796809007175 – volume: 18 start-page: 5595 issue: 1 year: 2017 ident: 10.1016/j.scico.2023.103010_br0020 article-title: Automatic differentiation in machine learning: a survey publication-title: J. Mach. Learn. Res. – start-page: 149 year: 1999 ident: 10.1016/j.scico.2023.103010_br0150 – volume: 18 issue: 1 year: 2022 ident: 10.1016/j.scico.2023.103010_br0330 article-title: Higher order automatic differentiation of higher order functions publication-title: Log. Methods Comput. Sci. – year: 1992 ident: 10.1016/j.scico.2023.103010_br0110 – start-page: 223 year: 1993 ident: 10.1016/j.scico.2023.103010_br0270 article-title: A short cut to deforestation – start-page: 195 year: 1998 ident: 10.1016/j.scico.2023.103010_br0290 article-title: Functional differentiation of computer programs – volume: vol. 7342 start-page: 324 year: 2012 ident: 10.1016/j.scico.2023.103010_br0230 article-title: Kan extensions for program optimisation or: art and Dan explain an old trick – volume: 1 start-page: 271 issue: 4 year: 1972 ident: 10.1016/j.scico.2023.103010_br0180 article-title: Proof of correctness of data representations publication-title: Acta Inform. doi: 10.1007/BF00289507 – year: 1997 ident: 10.1016/j.scico.2023.103010_br0050 – start-page: 60 year: 1989 ident: 10.1016/j.scico.2023.103010_br0450 article-title: How to make ad-hoc polymorphism less ad hoc – start-page: 513 year: 1983 ident: 10.1016/j.scico.2023.103010_br0200 article-title: Types, abstraction and parametric polymorphism – volume: 12 start-page: 321 year: 2003 ident: 10.1016/j.scico.2023.103010_br0440 article-title: A mathematical view of automatic differentiation publication-title: Acta Numer. doi: 10.1017/S0962492902000132 – start-page: 124 year: 1991 ident: 10.1016/j.scico.2023.103010_br0100 article-title: Functional programming with bananas, lenses, envelopes and barbed wire – volume: 29 start-page: e12 year: 2019 ident: 10.1016/j.scico.2023.103010_br0420 article-title: Perturbation confusion in forward automatic differentiation of higher-order functions publication-title: J. Funct. Program. doi: 10.1017/S095679681900008X – year: 2016 ident: 10.1016/j.scico.2023.103010_br0470 – volume: 77 year: 1977 ident: 10.1016/j.scico.2023.103010_br0250 article-title: A first order theory of data and programs publication-title: Inf. Process. – start-page: 339 year: 2014 ident: 10.1016/j.scico.2023.103010_br0280 article-title: Folding domain-specific languages: deep and shallow embeddings (Functional Pearl) – volume: 1 start-page: 3 issue: 1 year: 2009 ident: 10.1016/j.scico.2023.103010_br0430 article-title: Idealization of a module publication-title: J. Commut. Algebra doi: 10.1216/JCA-2009-1-1-3 |
| SSID | ssj0006471 |
| Score | 2.4377468 |
| Snippet | Automatic differentiation (AD) has been a topic of interest for researchers in many disciplines, with increased popularity since its application to machine... |
| SourceID | crossref elsevier |
| SourceType | Enrichment Source Index Database Publisher |
| StartPage | 103010 |
| SubjectTerms | Automatic differentiation Cayley representation Kronecker Nagata idealization Semimodules |
| Title | Forward- or reverse-mode automatic differentiation: What's the difference? |
| URI | https://dx.doi.org/10.1016/j.scico.2023.103010 |
| Volume | 231 |
| WOSCitedRecordID | wos001071219400001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVESC databaseName: ScienceDirect Freedom Collection - Elsevier customDbUrl: eissn: 1872-7964 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0006471 issn: 0167-6423 databaseCode: AIEXJ dateStart: 20211208 isFulltext: true titleUrlDefault: https://www.sciencedirect.com providerName: Elsevier |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1LT9wwELa20EMvtNBWpQ_kA1IPNCjreDd2LxUgEFAJcVjQ3qL4ETUrSFA2rPgN_dUdx3YSlmpVDr1EK28yG3nG45nZz_MhtBtxQUNFeCA5TwOq0zjgTNFgKOJQSD6SsKQason44oJNp_xyMPjtz8IsbuKiYA8P_O6_qhrGQNnm6Owz1N0KhQH4DEqHK6gdrv-k-JOyapCwBmlu-jNVcx0Yvpu99L4ubYNWz4pS5y22w7Twbmr3JhD130u9hPvznsBh0Q0dhEd43fo90J2JAn-2d-iwY4d5BWK7_3x-Vfls4cjZJnYKbG3wZ14UHXjXj1-bQnchqtJgQB6dyunXLAjt1SxcGRPcM2Q-Ud8PE7cdWE9q6M8s4PWJk7f1hhmk_7BY9g3_-3539-OW2ktbXQtA9Ni2WdIISYyQxAp5gdZJPOLgIdcPzo6n5-2-Prbpe_vuvodVgxZ88i5_j3N6scvkDdpwSQc-sMayiQa62EKvPaEHdlp9i8697eCywn3bwa3t4CXb-Y6N5XydY1Aw7uzmxzt0dXI8OToNHNlGICPK6kBE2ZAMMxLKTFEeMQmZpmEPgEWrqRJcsZTIjMaC0VRLIqlWocjSlA9NDM9k9B6tFWWhPyCsTRVhpDgEP5qKUAmISUcRESJiSoox2UbET00iXSd6Q4hyk6xQyzb61j50ZxuxrL597Oc8cbGkjRETsKJVD3583u98Qq86A_-M1urqXn9BL-WizufVjjOhP-jFlpg |
| linkProvider | Elsevier |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Forward-+or+reverse-mode+automatic+differentiation%3A+What%27s+the+difference%3F&rft.jtitle=Science+of+computer+programming&rft.au=van+den+Berg%2C+Birthe&rft.au=Schrijvers%2C+Tom&rft.au=McKinna%2C+James&rft.au=Vandenbroucke%2C+Alexander&rft.date=2024-01-01&rft.issn=0167-6423&rft.volume=231&rft.spage=103010&rft_id=info:doi/10.1016%2Fj.scico.2023.103010&rft.externalDBID=n%2Fa&rft.externalDocID=10_1016_j_scico_2023_103010 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0167-6423&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0167-6423&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0167-6423&client=summon |