Refined Kolmogorov complexity of analog, evolving and stochastic recurrent neural networks

Kolmogorov complexity measures the compressibility of real numbers. We provide a refined characterization of the hypercomputational power of analog, evolving, and stochastic neural networks based on the Kolmogorov complexity of their real weights, evolving weights, and real probabilities, respective...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Information sciences Ročník 711; s. 122104
Hlavní autori: Cabessa, Jérémie, Strozecki, Yann
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: Elsevier Inc 01.09.2025
Predmet:
Analog computation
Computability theory
Computational power
Echo state networks
Kolmogorov complexity
Quantization
Recurrent neural networks
Stochastic computation
Recurrent neural networks
Computability theory
Computational power
Echo state networks
Kolmogorov complexity
Quantization
Analog computation
Stochastic computation
ISSN:0020-0255
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:Kolmogorov complexity measures the compressibility of real numbers. We provide a refined characterization of the hypercomputational power of analog, evolving, and stochastic neural networks based on the Kolmogorov complexity of their real weights, evolving weights, and real probabilities, respectively. First, we retrieve the infinite hierarchy of complexity classes of analog networks, defined in terms of the Kolmogorov complexity of their real weights. This hierarchy lies between the complexity classes P and P/poly. Next, using a natural identification between real numbers and infinite sequences of bits, we generalize this result to evolving networks, obtaining a similar hierarchy of complexity classes within the same bounds. Finally, we extend these results to stochastic networks that employ real probabilities as randomness, deriving a new infinite hierarchy of complexity classes situated between BPP and BPP/log⁎. Beyond providing examples of such hierarchies, we describe a generic method for constructing them based on classes of functions of increasing complexity. As a practical application, we show that the predictive capabilities of recurrent neural networks are strongly impacted by the quantization applied to their weights. Overall, these results highlight the relationship between the computational power of neural networks and the intrinsic information contained by their parameters.
ISSN:0020-0255
DOI:10.1016/j.ins.2025.122104