Competitive Advantage of Huffman and Shannon-Fano Codes

For any finite discrete source, the competitive advantage of prefix code <inline-formula> <tex-math notation="LaTeX">C_{1} </tex-math></inline-formula> over prefix code <inline-formula> <tex-math notation="LaTeX">C_{2} </tex-math></inl...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on information theory Vol. 70; no. 11; pp. 7581 - 7598
Main Authors: Congero, Spencer, Zeger, Kenneth
Format: Journal Article
Language:English
Published: IEEE 01.11.2024
Subjects:
Channel coding
Codes
data compression
Games
Lossless source coding
Portfolios
Source coding
Symbols
Terminology
variable length codes
ISSN:0018-9448, 1557-9654
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:For any finite discrete source, the competitive advantage of prefix code <inline-formula> <tex-math notation="LaTeX">C_{1} </tex-math></inline-formula> over prefix code <inline-formula> <tex-math notation="LaTeX">C_{2} </tex-math></inline-formula> is the probability <inline-formula> <tex-math notation="LaTeX">C_{1} </tex-math></inline-formula> produces a shorter codeword than <inline-formula> <tex-math notation="LaTeX">C_{2} </tex-math></inline-formula>, minus the probability <inline-formula> <tex-math notation="LaTeX">C_{2} </tex-math></inline-formula> produces a shorter codeword than <inline-formula> <tex-math notation="LaTeX">C_{1} </tex-math></inline-formula>. For any source, a prefix code is competitively optimal if it has a nonnegative competitive advantage over all other prefix codes. In 1991, Cover proved that Huffman codes are competitively optimal for all dyadic sources, namely sources whose symbol probabilities are negative integer powers of 2. We prove the following asymptotic converse: As the source size grows, the probability a Huffman code for a randomly chosen non-dyadic source is competitively optimal converges to zero. We also prove: (i) For any non-dyadic source, a Huffman code has a positive competitive advantage over a Shannon-Fano code; (ii) For any source, the competitive advantage of any prefix code over a Huffman code is strictly less than <inline-formula> <tex-math notation="LaTeX">\frac {1}{3} </tex-math></inline-formula>; (iii) For each integer <inline-formula> <tex-math notation="LaTeX">n\gt 3 </tex-math></inline-formula>, there exists a source of size n and some prefix code whose competitive advantage over a Huffman code is arbitrarily close to <inline-formula> <tex-math notation="LaTeX">\frac {1}{3} </tex-math></inline-formula>; and (iv) For each positive integer n, there exists a source of size n and some prefix code whose competitive advantage over a Shannon-Fano code becomes arbitrarily close to 1 as <inline-formula> <tex-math notation="LaTeX">n\to \infty </tex-math></inline-formula>.
ISSN:0018-9448
1557-9654
DOI:10.1109/TIT.2024.3417010