Guaranteed inference for probabilistic programs: a parallelisable, small-step operational approach - Boreale, Michele; Collodi, Luisa (ACM transactions on probabilistic machine learning, 2025)
In the context of probabilistic programming languages, we put forward an approach to formal semantics and sampling-based inference with guarantees, centered on an action-based language equipped with a small-step operational semantics. We argue that this choice offers benefits in terms of clarity and effective, vectorized implementations. In measure-theoretic terms, a product of Markov kernels is used to formalize the small-step operational semantics. A trace semantics is also introduced based on a probability space of infinite sequences, along with a finite approximation theorem, relating the exact semantics to a truncated-execution semantics. This result directly leads to a sampling algorithm with guarantees, that can be efficiently SIMD-parallelized. Experiments conducted with an implementation based on TensorFlow show that our approach compares very favourably to state-of-the-art tools for probabilistic programming and inference. Keywords : probabilistic programming, operational semantics, measure theory, Monte Carlo simulation, SIMD parallelism.