Finite Blocklength Results in Channel Coding - CiteSeerX

7 downloads 14466 Views 101KB Size Report
Object of study: An (n,M,Ç«)-code is a block code satisfying ... Practical codes: We can gauge the efficiency of a given k-to-n code by ... PSD at zero of logP. Zj|Z.
Finite Blocklength Results in Channel Coding Yury Polyanskiy, H. Vincent Poor and Sergio Verd´u Princeton University, USA

Introduction

Message W Encoder X n Channel Y n Decoder W ˆ n ˆ |Y n −→ 1 . . . M −→ X (W ) −→ PY n|X n −→ PW

1−τ

• Object of study: An (n, M, ǫ)-code is a block code satisfying

Xj

TX

AWGN channel

1111111 0000000 0000000 1111111 0000000 1111111 BSC(δ1 ) 0000000 1111111 0000000 1111111 0000000 1111111

τ

Yj

111111 000000 000000 111111 000000 111111 BSC(δ2 ) 000000 111111 000000 111111 000000 111111

• Fundamental limit:

Z∼ N (0, 1) ↓ L X −→ −→ Y

RX

τ

ˆ 6= W ] ≤ ǫ P[W

Sj

Power constraint on the codebook {ci} ⊂ Rn is one of the following

1−τ

M ∗(n, ǫ) = max{M : ∃(n, M, ǫ)-code}

• GEC is the first channel with memory for which a finite blocklength analysis is performed. The capacity was found in [MBD89].

• Solution: In [PPV08] it was discovered that for various channels, e.g. discrete memoryless (DMC) and additive white Gaussian noise (AWGN), the following expansion [VS64] is notably tight for n, ǫ of interest: √ ∗ (1) log M (n, ǫ) = nC − nV Q−1(ǫ) + O(log n) ,

• GEC is a channel with dynamics and may serve as a discrete analog of a fading channel.

• GEC can be represented as a binary additive noise channel Yj = Xj + Zj , with the noise process Zj being a hidden Markov chain. Sj known at RX

Sj is not known

C1 = log 2 − 21 [h(δ1) + h(δ2)]

−1 )] C0 = log 2 − E [h(P[Z0 = 1|Z−∞

where the channel capacity C and channel dispersion V are 1 (nC − log M ∗(n, ǫ))2 . V = lim lim sup 1 ǫ→0 n→∞ n 2 ln ǫ

V1 =

• Practical codes: We can gauge the efficiency of a given k-to-n code by defining a normalized rate Rnorm = log Mk∗(n,ǫ) and use (1) to compute it. This enables a (more) fair comparison between the codes with different blocklengths and rates. Normalized rates of code families over AWGN, Pe=0.0001 1

• maximal power : ||ci||2 ≤ nP . P 1 ||ci||2 ≤ nP . • average power : M

Main result: regardless of the power constraint, we have that (1) holds with 1 C(P ) = log(1 + P ) 2 P P +2 2e. log V (P ) = 2 (P + 1)2

V (δ1)+V (δ2) + 2



h(δ1)−h(δ2) 2

2 

1 −1 τ



Parallel AWGN channel

V0 = PSD at zero of log PZ |Z j−1 j −∞ Zj ∼ N (0, σj2) ↓ j = 1...L L Xj −→ −→ Yj

where h(δ) = −δ log δ − (1 − δ) log(1 − δ) and V (δ) = δ(1 − δ) log2 1−δ δ is a dispersion of the BSC. • Evaluating the tightness of (1): δ1 = 1/2, δ2 = 0, τ = 0.1

Capacity

0.5

2 and power Main result: Given L parallel AWGN channels with noise levels σ12, . . . σL constraint P we have that (1) holds with

State unknown

State known 0.5

Converse 0.4

Rate R, bit/ch.use

• Importance: Many practical questions regarding M ∗(n, ǫ) can be answered by only knowing a pair of numbers: (C, V ). For example, according to (1) the minimal blocklength needed to achieve a fraction η of capacity is: !2 Q−1(ǫ) V n& . 2 1−η C

• equal-power : ||ci||2 = nP .

Capacity and dispersion:

CL(P ) =

0.4

Rate R, bit/ch.use

• Problem: M ∗(n, ǫ) is impossible to compute exactly even for n ∼ 10.

1 C = lim lim inf log M ∗(n, ǫ) , ǫ→0 n→∞ n

Gaussian channels

Gilbert-Elliott channel (GEC)

Achievability

0.3

Normal approximation

0.3

Capacity

VL(P ) =

Converse

0.2

0.2

0.1

0.1

L X

j=1 L X

C(Wj /σj2) V (Wj /σj2) ,

j=1

where Wj are water-filling powers: Achievability

0.95

Normal approximation 0 0

500

1000

1500

2000 2500 Blocklength, n

3000

3500

4000

0 0

500

1000

1500

2000 2500 Blocklength, n

3000

3500

Wj = |λ − σj2|+ ,

4000

0.9

• Observation: In the state known case the fundamental limit log M ∗(n, ǫ) depends on the channel dynamics τ only in the dispersion term.

Normalized rate

0.85

• New phenomenon: In the state unknown case, when τ ց 0 it is known [MBD89] that C0(τ ) ր C1. In reality, for a fixed n the behavior of n1 log M ∗(n, ǫ) is quite the opposite:

0.8

0.4

0.75

0.65

0.6 2 10

0.3

0.25 Rate, R

0.7

Turbo R=1/3 Turbo R=1/6 Turbo R=1/4 Voyager Galileo HGA Turbo R=1/2 Cassini/Pathfinder Galileo LGA ME LDPC R=1/2 (full BP) 10 Blocklength, n

• Note: the ME-LDPC codes were designed and evaluated by Tom Richardson.

5

10

0 0

j=1

L q X nC(Wj /σj2) − nV (Wj /σj2)Q−1(ǫ) + O(log n) , j=1



n term since

PL q j=1

V (Wj /σj2) >

qP

L 2). V (W /σ j j=1 j

VS64 V. Strassen, “Asymptotische Absch¨atzungen in Shannon’s Informationstheorie,” Trans. Third Prague Conf. Information Theory, Czechoslovak Academy of Sciences, Prague, pp. 689-723, 1962.

0.05

10

log M ≈

L X

PPV08 Y. Polyanskiy, H. V. Poor and S. Verd´u, “Channel coding rate in the finite blocklength regime,” submitted to IEEE Trans. Inform. Theory, Nov. 2008.

0.2

0.1

4

j=1

References

0.15

3

Wj = P .

Observation: Note that L parallel codes can achieve at most

which is suboptimal in the

0.35

L X

0.002

0.004

0.006

0.008

0.01 τ

0.012

0.014

0.016

0.018

0.02

• Slow dynamics: When τ → 0 we know that C0(τ ) → C1. In addition, we also have V0(τ ) = V1(τ ) + o(1/τ ), see [PPV09].

MBD89 M. Mushkin and I. Bar-David, “Capacity and coding for the Gilbert- Elliott channels,” IEEE Trans. Inform. Theory, Vol. 35, No. 6, pp. 1277-1290, 1989. PPV09 Y. Polyanskiy, H. V. Poor and S. Verd´u, “Dispersion of the Gilbert-Elliott channel,” submitted to IEEE Trans. Inform. Theory, Jun. 2009.

Suggest Documents