Object of study: An (n,M,Ç«)-code is a block code satisfying ... Practical codes: We can gauge the efficiency of a given k-to-n code by ... PSD at zero of logP. Zj|Z.
Finite Blocklength Results in Channel Coding Yury Polyanskiy, H. Vincent Poor and Sergio Verd´u Princeton University, USA
Introduction
Message W Encoder X n Channel Y n Decoder W ˆ n ˆ |Y n −→ 1 . . . M −→ X (W ) −→ PY n|X n −→ PW
1−τ
• Object of study: An (n, M, ǫ)-code is a block code satisfying
Xj
TX
AWGN channel
1111111 0000000 0000000 1111111 0000000 1111111 BSC(δ1 ) 0000000 1111111 0000000 1111111 0000000 1111111
τ
Yj
111111 000000 000000 111111 000000 111111 BSC(δ2 ) 000000 111111 000000 111111 000000 111111
• Fundamental limit:
Z∼ N (0, 1) ↓ L X −→ −→ Y
RX
τ
ˆ 6= W ] ≤ ǫ P[W
Sj
Power constraint on the codebook {ci} ⊂ Rn is one of the following
1−τ
M ∗(n, ǫ) = max{M : ∃(n, M, ǫ)-code}
• GEC is the first channel with memory for which a finite blocklength analysis is performed. The capacity was found in [MBD89].
• Solution: In [PPV08] it was discovered that for various channels, e.g. discrete memoryless (DMC) and additive white Gaussian noise (AWGN), the following expansion [VS64] is notably tight for n, ǫ of interest: √ ∗ (1) log M (n, ǫ) = nC − nV Q−1(ǫ) + O(log n) ,
• GEC is a channel with dynamics and may serve as a discrete analog of a fading channel.
• GEC can be represented as a binary additive noise channel Yj = Xj + Zj , with the noise process Zj being a hidden Markov chain. Sj known at RX
Sj is not known
C1 = log 2 − 21 [h(δ1) + h(δ2)]
−1 )] C0 = log 2 − E [h(P[Z0 = 1|Z−∞
where the channel capacity C and channel dispersion V are 1 (nC − log M ∗(n, ǫ))2 . V = lim lim sup 1 ǫ→0 n→∞ n 2 ln ǫ
V1 =
• Practical codes: We can gauge the efficiency of a given k-to-n code by defining a normalized rate Rnorm = log Mk∗(n,ǫ) and use (1) to compute it. This enables a (more) fair comparison between the codes with different blocklengths and rates. Normalized rates of code families over AWGN, Pe=0.0001 1
• maximal power : ||ci||2 ≤ nP . P 1 ||ci||2 ≤ nP . • average power : M
Main result: regardless of the power constraint, we have that (1) holds with 1 C(P ) = log(1 + P ) 2 P P +2 2e. log V (P ) = 2 (P + 1)2
V (δ1)+V (δ2) + 2
h(δ1)−h(δ2) 2
2
1 −1 τ
Parallel AWGN channel
V0 = PSD at zero of log PZ |Z j−1 j −∞ Zj ∼ N (0, σj2) ↓ j = 1...L L Xj −→ −→ Yj
where h(δ) = −δ log δ − (1 − δ) log(1 − δ) and V (δ) = δ(1 − δ) log2 1−δ δ is a dispersion of the BSC. • Evaluating the tightness of (1): δ1 = 1/2, δ2 = 0, τ = 0.1
Capacity
0.5
2 and power Main result: Given L parallel AWGN channels with noise levels σ12, . . . σL constraint P we have that (1) holds with
State unknown
State known 0.5
Converse 0.4
Rate R, bit/ch.use
• Importance: Many practical questions regarding M ∗(n, ǫ) can be answered by only knowing a pair of numbers: (C, V ). For example, according to (1) the minimal blocklength needed to achieve a fraction η of capacity is: !2 Q−1(ǫ) V n& . 2 1−η C
• equal-power : ||ci||2 = nP .
Capacity and dispersion:
CL(P ) =
0.4
Rate R, bit/ch.use
• Problem: M ∗(n, ǫ) is impossible to compute exactly even for n ∼ 10.
1 C = lim lim inf log M ∗(n, ǫ) , ǫ→0 n→∞ n
Gaussian channels
Gilbert-Elliott channel (GEC)
Achievability
0.3
Normal approximation
0.3
Capacity
VL(P ) =
Converse
0.2
0.2
0.1
0.1
L X
j=1 L X
C(Wj /σj2) V (Wj /σj2) ,
j=1
where Wj are water-filling powers: Achievability
0.95
Normal approximation 0 0
500
1000
1500
2000 2500 Blocklength, n
3000
3500
4000
0 0
500
1000
1500
2000 2500 Blocklength, n
3000
3500
Wj = |λ − σj2|+ ,
4000
0.9
• Observation: In the state known case the fundamental limit log M ∗(n, ǫ) depends on the channel dynamics τ only in the dispersion term.
Normalized rate
0.85
• New phenomenon: In the state unknown case, when τ ց 0 it is known [MBD89] that C0(τ ) ր C1. In reality, for a fixed n the behavior of n1 log M ∗(n, ǫ) is quite the opposite:
0.8
0.4
0.75
0.65
0.6 2 10
0.3
0.25 Rate, R
0.7
Turbo R=1/3 Turbo R=1/6 Turbo R=1/4 Voyager Galileo HGA Turbo R=1/2 Cassini/Pathfinder Galileo LGA ME LDPC R=1/2 (full BP) 10 Blocklength, n
• Note: the ME-LDPC codes were designed and evaluated by Tom Richardson.
5
10
0 0
j=1
L q X nC(Wj /σj2) − nV (Wj /σj2)Q−1(ǫ) + O(log n) , j=1
√
n term since
PL q j=1
V (Wj /σj2) >
qP
L 2). V (W /σ j j=1 j
VS64 V. Strassen, “Asymptotische Absch¨atzungen in Shannon’s Informationstheorie,” Trans. Third Prague Conf. Information Theory, Czechoslovak Academy of Sciences, Prague, pp. 689-723, 1962.
0.05
10
log M ≈
L X
PPV08 Y. Polyanskiy, H. V. Poor and S. Verd´u, “Channel coding rate in the finite blocklength regime,” submitted to IEEE Trans. Inform. Theory, Nov. 2008.
0.2
0.1
4
j=1
References
0.15
3
Wj = P .
Observation: Note that L parallel codes can achieve at most
which is suboptimal in the
0.35
L X
0.002
0.004
0.006
0.008
0.01 τ
0.012
0.014
0.016
0.018
0.02
• Slow dynamics: When τ → 0 we know that C0(τ ) → C1. In addition, we also have V0(τ ) = V1(τ ) + o(1/τ ), see [PPV09].
MBD89 M. Mushkin and I. Bar-David, “Capacity and coding for the Gilbert- Elliott channels,” IEEE Trans. Inform. Theory, Vol. 35, No. 6, pp. 1277-1290, 1989. PPV09 Y. Polyanskiy, H. V. Poor and S. Verd´u, “Dispersion of the Gilbert-Elliott channel,” submitted to IEEE Trans. Inform. Theory, Jun. 2009.