Universal Weak Variable-Length Source Coding on Countably Infinite Alphabets

Motivated from the fact that universal source coding on countably infinite alphabets (<inline-formula> <tex-math notation="LaTeX">\infty </tex-math></inline-formula>-alphabets) is not feasible, this work introduces the notion of "almost lossless source coding&q...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on information theory Jg. 66; H. 1; S. 649 - 668
Hauptverfasser: Silva, Jorge F., Piantanida, Pablo
Format: Journal Article
Sprache:Englisch
Veröffentlicht: New York IEEE 01.01.2020
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Institute of Electrical and Electronics Engineers
Schlagworte:
Alphabets
Channel coding
Codes
Coding
Convergence
countably infinite alphabets (∞-alphabets)
Distortion
Engineering Sciences
Entropy
Entropy (Information theory)
envelope distributions
information radius (i-radius)
metric entropy analysis
Minimax technique
Redundancy
Source code
Source coding
universal source coding
weak source coding
ISSN:0018-9448, 1557-9654
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Motivated from the fact that universal source coding on countably infinite alphabets (<inline-formula> <tex-math notation="LaTeX">\infty </tex-math></inline-formula>-alphabets) is not feasible, this work introduces the notion of "almost lossless source coding". Analog to the weak variable-length source coding problem studied by Han ( IEEE Trans. Inf. Theory , vol. 46, no. 4, pp. 1217-1226, Jul. 2000), almost lossless source coding aims at relaxing the lossless block-wise assumption to allow an average per-letter distortion that vanishes asymptotically as the block-length tends to infinity. In this setup, we show on one hand that Shannon entropy characterizes the minimum achievable rate (similarly to the case of finite alphabet sources) while on the other that almost lossless universal source coding becomes feasible for the family of finite-entropy stationary memoryless sources with <inline-formula> <tex-math notation="LaTeX">\infty </tex-math></inline-formula>-alphabets. Furthermore, we study a stronger notion of almost lossless universality that demands uniform convergence of the average per-letter distortion to zero, where we establish a necessary and sufficient condition for the so-called family of "envelope distributions" to achieve it. Remarkably, this condition is the same necessary and sufficient condition needed for the existence of a strongly minimax (lossless) universal source code for the family of envelope distributions. Finally, we show that an almost lossless coding scheme offers faster rate of convergence for the (minimax) redundancy compared to the well-known information radius developed for the lossless case at the expense of tolerating a non-zero distortion that vanishes to zero as the block-length grows. This shows that even when lossless universality is feasible, an almost lossless scheme can offer different regimes on the rates of convergence of the (worst case) redundancy versus the (worst case) distortion.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0018-9448
1557-9654
DOI:10.1109/TIT.2019.2941895