On the Performance of Turbo Codes over the Binary Erasure Channel

IEEE COMMUNICATIONS LETTERS, VOL. 11, NO. 1, JANUARY 2007

67

On the Performance of Turbo Codes over the Binary Erasure Channel Jeong W. Lee, Member, IEEE, R¨udiger Urbanke, Member, IEEE, and Richard E. Blahut, Fellow, IEEE

Abstract— In this letter, we propose an estimate for the block erasure probability of turbo codes with an iterative decoding in a waterfall region, which is nonlinearly scaled by the information blocklength. This estimate can be used to predict efficiently the block erasure probability of the finite-length turbo codes over a binary erasure channel. Index Terms— Turbo codes, binary erasure channel, block erasure probability.

I. I NTRODUCTION N iterative turbo decoding algorithm can be analyzed by studying two open-loop constituent decoders separately [1]–[5]. Given the statistics of communication channel output, the behavior of each constituent decoder is represented by a transfer function between statistics of extrinsic information. Note that in this letter, the extrinsic information will be called the extrinsic. The widely used statistic of extrinsic is the mutual information with the corresponding information bit [1]. The asymptotic extrinsic information transfer (EXIT) function of a constituent decoder over a binary erasure channel (BEC) can be analytically derived by using the asymptotic analysis [2], [6] under the assumption that the turbo decoding is a stationary process. By using the asymptotic analysis, we can find the threshold erasure probability of a communication channel, at which a waterfall occurs in the block/bit erasure probability curve. The slope of a waterfall, however, depends on the information blocklength, so the relation between the information blocklength and the block/bit erasure probability in a waterfall region needs to be discovered. In this letter, we propose an estimate for the block erasure probability of turbo codes over BEC, which is nonlinearly scaled by the information blocklength, and show that this estimate can be used to predict efficiently the block erasure probability in a waterfall region.

A

II. F RAMEWORK We consider a binary turbo coding scheme, which is composed of two constituent encoders connected parallel through an interleaver [7]. The turbo decoder is composed of two constituent decoders, denoted by DEC1 and DEC2, Manuscript received August 3, 2006. The associate editor coordinating the review of this letter and approving it for publication was Dr. Christina Fragouli. This work has been supported partly by Nano IP/SoC Promotion Group and SUITE of Seoul R&BD Program in 2006 and by EESRI (R-2005B-203) which is funded by MOCIE. J. W. Lee is with the School of Electrical and Electronics Engineering, Chung-Ang University, Seoul 156-756, Korea (e-mail: [email protected]). R. Urbanke is with the Swiss Federal Institute of Technology-Lausanne, CH-1015 Lausanne, Switzerland (e-mail: [email protected]). R. E. Blahut is with the University of Illinois at Urbana-Champaign, Urbana, IL 61801, USA (e-mail: [email protected]). Digital Object Identifier 10.1109/LCOMM.2007.061206.

connected serially through an interleaver and a deinterleaver. Each constituent decoder outputs extrinsics by using channel outputs and priors as inputs. A prior is defined as the logarithmic ratio of the probability that an information bit is 0 and the probability that an information bit is 1, where an extrinsic obtained in the previous constituent decoder is used as a prior. Let us call a BEC with an erasure probability
as a communication channel. An extrinsic obtained at each iteration of a decoding process has ternary values: −∞, 0 or ∞. So, an extrinsic can be considered the output of another BEC which will be called an extrinsic channel [3]. Note that the input of an extrinsic channel is an information bit and the output of an extrinsic channel is an extrinsic or a prior. A zero extrinsic and a zero prior correspond to the erased output of an extrinsic channel. Without loss of generality, we assume an all-zero information block, which results in extrinsics being 0 or ∞. If an information bit is erased by a communication channel and remains erased after a decoding [2], [3], the corresponding extrinsic has the value of 0. Otherwise, the value of extrinsic is ∞. We assume that the turbo decoding is a stationary process [2]. In DEC1, let xI and xO be the probability that a prior is 0 and the probability that an extrinsic is 0, respectively. The asymptotic transfer function from xI to xO with a parameter
is analytically derived [2], which is denoted by xO = g(xI ,
). Since a prior and an extrinsic are outputs of a BEC with an erasure probability xI and xO , respectively, the mutual information between a prior and an information bit is 1 − xI , and the mutual information between an extrinsic and an information bit is 1 − xO . For a given
, the function g(xI ,
) passes through the origin and is monotonically increasing with respect to xI . In DEC2, xO and xI represent the probability that a prior is 0 and the probability that an extrinsic is 0, respectively. The asymptotic transfer function from xO to xI , denoted by xI = h(xO ,
), is also analytically derived, where h is monotonically increasing. By considering statistics of the sequences of priors and extrinsics instead of considering the ensemble statistics of prior and extrinsic, we can obtain more useful insight regarding the probabilistic convergence behavior of the turbo decoding [4], [5]. The turbo decoding process with given iteration, communication channel realization and interleaver can be regarded as a sample path of a stationary process. The randomness of the closed-loop iterative turbo decoding process is reflected in the open-loop constituent decoding as the randomness of the realizations of communication channel and extrinsic channel. Let us consider DEC1. Let bk be the k th bit in a lengthn information block and λO k be the extrinsic corresponding ˆI be the fraction of zero priors in the sequence to bk . Let x of priors, and x ˆO be the fraction of zero extrinsics in the

c 2007 IEEE 1089-7798/07$20.00


68

IEEE COMMUNICATIONS LETTERS, VOL. 11, NO. 1, JANUARY 2007

n=1000 n=200

1

h ( g ( ^x I, ε ,n) , ε ,) n

decoding trajectory

y

start

0.8 converge

0.6 ρ

kj

In case of ε =ε * slope = 1

0.4 0.2 0

O

−0.2 0

50

100 j

150

200

x^ + ( ε )

1

x^ I

Fig. 2. A random function y = h(g(ˆ xI , , n), , n) and a decoding trajectory. The thick curve represents the mean of h(g(ˆ xI , , n), , n) and the dotted bars across the thick curve represent the scattering of h(g(ˆ xI , , n), , n).

Fig. 1. The correlation coefficient ρkj (ˆ xI , , n), k = 100, 1 ≤ j ≤ 200, for n = 200 and n = 1000 with fixed  and x ˆI , where G = (7, 5).

variance given in (2) and (3), respectively. Thus, we can write xI ,
, n) by x ˆO = g(ˆ sequence of extrinsics. We also let g be a transfer function ˆO with a parameter
and n, i.e., x ˆO = g(ˆ xI ,
, n), from x ˆI to x where g is a random function depending on the realizations of communication channel, extrinsic channel and prior sequence. N (ˆ xI ,
, n) is a random variable denoting the number of zero extrinsics in a certain sequence of extrinsics obtained from ˆO = n1 N (ˆ xI ,
, n). Let us DEC1 with x ˆI ,
and n, where x define an index function Ik such that Ik = 1 if λO k = 0 and I = ∞. For a given x ˆ , the random variable Ik has Ik = 0 if λO k xI ,
), where it a Bernoulli distribution with Pr{Ik = 1} = g(ˆ is assumed that Pr{Ik = 1} is independent of k. Then,
 E{Ik } = g(ˆ xI ,
) and var{Ik } = g(ˆ xI ,
) 1 −g(ˆ xI ,
) . (1) n It is clear that N (ˆ xI ,
, n) = k=1 Ik and thus n

1 1 E{ˆ xO } = E{ N (ˆ xI ,
, n)} = E{Ik } = g(ˆ xI ,
). (2) n n k=1

Let ρkj (ˆ xI ,
, n) be a correlation coefficient between Ik and ˆI ,
and n. Under the stationarity assumption Ij for given x of xI ,
, n)
nthe turboI decoding process, we suppose ρ(ˆ x ,
, n) to be constant irrespective of k. Then, j=1 ρkj (ˆ
O var x ˆ =

n n  1  I ρ (ˆ x ,
, n) var{Ik }var{Ij } kj n2 j=1 k=1

=


 1 · ρ(ˆ xI ,
, n) · g(ˆ xI ,
) 1 − g(ˆ xI ,
) . (3) n

It is observed from Fig. 1 that Ik and Ij , k, j = 1, · · · , n, are positively correlated due to a coding constraint if |k − j| xI ,
, n) ≈ is small, but they are almost uncorrelated, i.e., ρkj (ˆ 0, if |k − j| is sufficiently large. It is also observed from xI ,
, n) with given k and j obtained for Fig. 1 that ρkj (ˆ different n are almostthe same. Consequently, we obtain n1 n2 ρ (ˆ xI ,
, n) ≈ j=1 ρkj (ˆ xI ,
, n) for any 1  k  kj j=1 I x ,
, n) is considered constant irrespective n1 < n2 , so that ρ(ˆ of n for given x ˆI and
. Thus, from now on, we will replace xI ,
). For given x ˆI ,
and n, the observation ρ(ˆ xI ,
, n) with ρ(ˆ I of g(ˆ x ,
, n) has a Gaussian distribution with a mean and a

x ˆO = g(ˆ xI ,
, n) = g(ˆ xI ,
) + w1 (ˆ xI ,
, n), (4)   where w1 (ˆ xI ,
, n) ∼ N 0, n1 ρ(ˆ xI ,
)g(ˆ xI ,
)[1 − g(ˆ xI ,
)] is Gaussian with given x ˆI ,
and n. Likewise, we can define O a random function h(ˆ x ,
, n) for DEC2, written by h(ˆ xO ,
, n) = h(ˆ xO ,
) + w2 (ˆ xO ,
, n), (5)   where w2 (ˆ xO ,
, n) ∼ N 0, n1 ρ(ˆ xO ,
)h(ˆ xO ,
)[1 − h(ˆ xO ,
)] is Gaussian with given x ˆO ,
and n. The overall decoding composed of DEC1 and DEC2, denoted by h(g(ˆ xI ,
, n),
, n), can be written by combining (4) and (5) as   I   h g x ˆ ,
, n ,
, n = h(g(ˆ xI ,
) + w1 (ˆ xI ,
, n),
)  I  (6) + w2 g(ˆ x ,
) + w1 (ˆ xI ,
, n),
, n . III. B LOCK E RASURE P ROBABILITY The trajectory of an iterative decoding has a zigzag pattern xI ,
, n),
, n) starting from between y = x ˆI and y = h(g(ˆ h(g(1,
, n),
, n) as shown in Fig. 2. If there exists a nonzero x ˆI satisfying x ˆI = h(g(ˆ xI ,
, n),
, n), then the decoding trajectory gets stuck at such x ˆI . In addition, if at least one information bit has zero extrinsics in both DEC1 and DEC2, then the information block is erased. Note that a zero extrinsic implies that the corresponding information bit is already erased by a communication channel. Thus, the block erasure probability with given
and n, denoted by PB (
, n), is obtained by multiplying ˆI = h(g(ˆ xI ,
, n),
, n)} and Pr{at Pr{∃ x ˆI = 0 satisfying x least one information bit has zero extrinsics in both DEC1 ˆI = h(g(ˆ xI ,
, n),
, n)}. and DEC2 | ∃ x ˆI = 0 satisfying x The nx1 n
−nxterm   n
is obtained by nx2 conditional probability 1 under the t=max{1,n(x1 +x2 −
)} t nx2 −t / nx2 assumption that nx1 , nx2 and n
are integers and x2 ≤ x1 ≤
without loss of generality. The conditional probability is close to 1 for n,
, x1 and x2 of interest. ˆI for a given
at the Let x ˆ† (
) be the value of x I xI ,
, n),
, n)}. It is clear extremum of x ˆ − E{h(g(ˆ † † ˆ (
) is a sufficient condition of that h(g(ˆ x (
),
, n),
, n) ≥ x ˆI = h(g(ˆ xI ,
, n),
, n). the existence of a x ˆI = 0 satisfying x

LEE et al.: ON THE PERFORMANCE OF TURBO CODES OVER THE BINARY ERASURE CHANNEL 0

From (11) and the Taylor expansions around
=
∗ , we have

2 
1  ∂ h (g(x∗ ,
∗ ),
∗ ) · var γ(ˆ x† (
),
, n) = n ∂w1 ∗ ∗ ∗ ∗ ρ(x ,
)·g(x ,
)·[1 − g(x∗ ,
∗ )]+ ρ(g(x∗ ,
∗ ),
∗ )·  x∗ ·(1 − x∗ ) + O (
−
∗ ) + O(1/n2 ). (14)

10

a : n=1024 b : n=2048 c : n=4096

block erasure probability

d : n=8192 −1

10

e : n=16384 a

prediction simulation

b c d e

−2

10

From (12), (13) and (14), the block erasure probability around
=
∗ with a sufficiently large n is approximated by √  (15) PB (
, n) ≈ Q n · α(
∗ ) · (
−
∗ ) ,

−3

10

0.59

69

0.6

0.61

0.62 ε

0.63

0.64

0.65

Fig. 3. The predicted and simulated block erasure probabilities of turbo codes with G = (15, 13), where α(∗ ) = −2.8 is chosen so that the predicted curve fits the simulated curve for n = 1024, where ∗ = 0.636.

Consequently, PB (
, n) ≈ Pr{∃ x ˆI = 0 satisfying x ˆI = x† (
),
, n),
, n) ≥ x ˆ† (
)}, h(g(ˆ xI ,
, n),
, n)}  Pr{h(g(ˆ and we will use the approximate lower bound (the rightmost hand side) as the approximation for PB (
, n), i.e., PB (
, n) ≈ Pr{h(g(ˆ x† (
),
, n),
, n) ≥ x ˆ† (
)}. By the Taylor expansion, (6) is written by   h(g(ˆ xI ,
, n),
, n) = γ(ˆ xI ,
, n) + O w12 (ˆ xI ,
, n) ,

(7)

(8)

where

 I   I  ∂ γ(ˆ xI ,
, n)
h g(ˆ x ,
),
+ h g(ˆ x ,
),
· ∂w1  I  xI ,
, n) + w2 g(ˆ x ,
) + w1 (ˆ xI ,
, n),
, n w1 (ˆ

is Gaussian for given x ˆI ,
and n with a mean
  I  E γ(ˆ xI ,
, n) = h g(ˆ x ,
),


(9)

 ∂ 2 ∂ [ˆ x† (
∗)−h (g(x∗,
∗),
∗)]/{ ∂w h (g(x∗,
∗),
∗) · where α(
∗) = ∂
1 ρ(x∗ ,
∗ ) · g(x∗ ,
∗ ) · [1 − g(x∗ ,
∗ )] + ρ (g(x∗ ,
∗ ),
∗ ) · x∗ · (1 − x∗ )}1/2 and it is independent of n. The equation (15) shows an approximate scaling law for the block erasure probability of turbo codes in a waterfall region around
∗ . A practical application of (15) is a simple and efficient estimation or prediction of a block erasure probability as described below, although the analytical computation of α(
∗ ) is not available. First, we determine
∗ through an asymptotic analysis and choose a small n. For a chosen n, we obtain a block erasure probability curve via a simulation with a relatively low complexity. Then, for a chosen n, we determine α(
∗ ) so that the prediction by (15) fits the simulated curve. Once α(
∗ ) is determined, block erasure probabilities for other n can be predicted from (15) simply by plugging in the values of n. Fig. 3 shows the validity of the proposed estimates. IV. C ONCLUSION

(10)

and a variance
  I  2 1  ∂ var γ(ˆ xI ,
, n) = h g(ˆ x ,
),
· n ∂w1   xI ,
) · [1 − g(ˆ xI ,
)] + ρ g(ˆ xI ,
),
· (11) ρ(ˆ xI ,
) · g(ˆ  I   I  h g(ˆ x ,
),
· [1 − h g(ˆ x ,
),
] + O(1/n2 ). 2 I x ,
, n)} = O (1/n) and var{w12 (ˆ xI ,
, n)} = Since E{w1 (ˆ xI ,
, n),
, n) is approximated O 1/n2 , it is clear that h(g(ˆ by γ(ˆ xI ,
, n) for a sufficiently large n. Then, (7) becomes
 †   PB (
, n) ≈ Pr γ x ˆ (
),
, n ≥ x ˆ† (
)  †   (12) x ˆ† (
) − E{γ x ˆ (
),
, n }

=Q . † var{γ (ˆ x (
),
, n)}

ˆ† (
) = Let
∗ be the value of
for which x † E{h(g(ˆ x (
),
, n),
, n)}. Then, by (8) and (10), x ˆ† (
∗ ) = † ∗ ∗ ∗ † ∗ h(g(ˆ x (
),
),
) + O(1/n), where we let x ˆ (
) = x∗ . ∗ In an asymptotic analysis,
is defined as a threshold. By (10) ˆ† (
) and  order Taylor∗ expansions of x
and† the first E γ x ˆ (
),
, n around
=
, we have 
 †   † x ˆ (
)−E γ x ˆ (
),
, n = O (
−
∗ )2 + O(1/n) (13) ∂ † ∗ x (
)− h (g(x∗ ,
∗ ),
∗)}·(
−
∗) . + {ˆ ∂


The estimate for the block erasure probability of turbo codes over BEC was proposed, which is nonlinearly scaled by the information blocklength for given encoder and erasure probability of a communication channel. The proposed estimate can be used to predict the block erasure probability of turbo codes in a simpler and more efficient way. R EFERENCES [1] S. ten Brink, “Convergence behavior of iteratively decoded parallel concatenated codes,” IEEE Trans. Commun., vol. 49, pp. 1727–1737, Oct. 2001. [2] C. Measson and R. Urbanke, “Asymptotic analysis of turbo codes over the binary erasure channel,” in Proc. 12th Joint Conference on Communications and Coding 2002. [3] A. Ashikhmin, G. Kramer, and S. ten Brink, “Extrinsic information transfer functions: model and erasure channel properties,” IEEE Trans. Inf. Theory, vol. 50, pp. 2657–2673, Nov. 2004. [4] J. W. Lee and R. E. Blahut, “Bit error rate estimate of finite length turbo codes,” in Proc. IEEE 2003 Int. Conf. Communications, pp. 2728–2732. [5] J. W. Lee and R. E. Blahut, “Lower bound on BER of finite-length turbo codes based on EXIT characteristics,” IEEE Commun. Lett., vol. 8, pp. 238–240, Apr. 2004. [6] B. M. Kurkoski, P. H. Siegel, and J. K. Wolf, “Exact probability of erasure and a decoding algorithm for convolutional codes on the binary erasure channel,” in Proc. 2003 IEEE Globecom, pp. 1741–1745. [7] C. Berrou and A. Glavieux, “Near optimum error correcting coding and decoding: turbo-codes,” IEEE Trans. Commun., vol. 44, pp. 1261–1271, Oct. 1996.