Automatic differentiation for ML-family languages: Correctness via logical relations

We give a simple, direct, and reusable logical relations technique for languages with term and type recursion and partially defined differentiable functions. We demonstrate it by working out the case of automatic differentiation (AD) correctness: namely, we present a correctness proof of a dual numb...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Mathematical structures in computer science Ročník 34; číslo 8; s. 747 - 806
Hlavní autoři: Lucatelli Nunes, Fernando, Vákár, Matthijs
Médium: Journal Article
Jazyk:angličtina
Vydáno: Cambridge, UK Cambridge University Press 01.09.2024
Témata:
Algorithms
Differentiation
Functional programming
Machine learning
Neural networks
Programming languages
Semantics
Special Issue: Differential Structures
Software correctness
Program transformations
Recursive types
Differentiable programming languages
Logical relations
Programming language semantics
ISSN:0960-1295, 1469-8072
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:We give a simple, direct, and reusable logical relations technique for languages with term and type recursion and partially defined differentiable functions. We demonstrate it by working out the case of automatic differentiation (AD) correctness: namely, we present a correctness proof of a dual numbers style AD code transformation for realistic functional languages in the ML-family. We also show how this code transformation provides us with correct forward- and reverse-mode AD. The starting point is to interpret a functional programming language as a suitable freely generated categorical structure. In this setting, by the universal property of the syntactic categorical structure, the dual numbers AD code transformation and the basic $\boldsymbol{\omega } \mathbf{Cpo}$ -semantics arise as structure preserving functors. The proof follows, then, by a novel logical relations argument. The key to much of our contribution is a powerful monadic logical relations technique for term recursion and recursive types. It provides us with a semantic correctness proof based on a simple approach for denotational semantics, making use only of the very basic concrete model of $\omega$ -cpos.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0960-1295
1469-8072
DOI:10.1017/S0960129524000215