Extrinsic information in turbo decoding: a unified view - Global ...

General Conference (Pari A)

EXTRINSIC INFORMATION

IN

TURBO DECODING:

A U N I F I E D VIEW Giulio Colavolpe, Gianluigi Ferrari and Riccardo Raheli Dipartimento di Ingegneria dell’Informazione,Universiti di Parma, Parco Area delle Scienae 181-A, 1-43100 Parma, Italy

Abstract- In this paper, we address the use of the extrinsic information generated by each component decoder in an iterative decoding process. The algorithm proposed by Bahl et al. (BCJR) and the soft-output Viterbi algorithm (SOVA) are considered as component decoders. Numerical results for a classical rate1/2 turbo code transmitted over a memoryless additive white Gaussian noise ( A W G N ) channel are provided. The performance analysis of the considered schemes leads to interesting remarks about the nature of iterative decoding processes.

I. INTRODUCTION N CONJUNCTION with the proposal of “turbo

I

codes”, based on a parallel concatenation of two recursive systematic convolutional (RSC) codes linked together by a n o n u n i f o m interleaver, a suboptimal decoding scheme based on iterative decoding was introduced [l], [2]. The heart of the iterative decoding procedure is the use, in each component decoder, of an algorithm t h a t computes the a posteriori probability (APP) of the information symbols or, more generally, a reliability value for each information symbol. In [l], the original concept of extrinsic i n f o m a t i o n was introduced t o identify the component of the generated reliability value which depends on redundant information introduced by the considered constituent code. A natural reliability value, in the binary case, is the logarithmic likelihood ratio (LLR), defined as A

L ( a k ) = In

P{ak = +Ilinputs} P{ak = -1linputs)

where the word “inputs” refers t o the entire decoder input sequence. The sequence of APPs may be exactly computed by means of the BCJR algorithm [3]. However, its computational complexity is large with respect t o that of the Viterbi algorithm (VA). An alternative is represented by the soft-output Viterbi algorithm (SOVA) [4]-[6], which provides, in addition t o “hard” decisions, a reliability information which can be interpreted as an approximation of the LLR. Whatever the chosen algorithm is, in the literature there exist essentially two approaches t o treat the extrinsic information received by each decoder (and generated by the other one). In a first approach, the exThis work was supported by Minister0 dell’Universitk e della Ricerca Scientifica e Tecnologica (MURST),Italy.

Global Telecommunications Conference - Globecom’99 0-7803-5796-5/99/$10.00 01999 IEEE

trinsic information at the input of a decoder is considered as the output of an additive white Gaussian noise (AWGN) meta-channel [l],[2], [7]. In a second approach, the extrinsic information is used t o update the “a priori” probabilities which are used in the next decoding step [SI, [9]. Therefore, the APPs generated by a decoder become the a priori probabilities for the other one. In this paper, we present a unified interpretation of these two approaches and emphasize their commonalities and differences. More precisely, we show that, either using the BCJR algorithm or the SOVA, the two approaches only differ for a multiplicative factor used in the metric computation. We finally consider the use of a heuristic multiplicative factor and evaluate the performance of the considered decoding schemes for various values of this factor. 11.

U S E OF THE EXTRINSIC INFORMATION

Let us consider a soft-output component decoder in an iterative decoding scheme. In general, this decoder has at its input channel output and extrinsic information sequences, the latter being generated by other component decoders and assumed independent of the channel output. The output of this decoder is a set of sequences of extrinsic information to be passed t o other component decoders. For illustration purposes and without loss of generality, we consider a single channel output sequence { x k } and single input and output extrinsic information sequences { z k } and {wk},respectively. The general case may be handled by an extended vector notation. The channel output may be expressed as xk = ck nk, where { c k } is a code sequence and {nk} is a noise sequence of independent, zero-mean, real Gaussian random variables, with variance 6’. In the original paper on turbo codes and iterative decoding [l],the input sequence { z k } , i.e., the extrinsic information extracted from the reliability values of the information sequence { a k } , is interpreted as the output of a Gaussian meta-channel. Specifically, it is assumed that zk = qzak ni (2)

+

+

where the information symbols { a h } belong t o the binary alphabet {zkl}, {ni} are independent, zero-mean, real Gaussian random variables, with variance CT; and

505

General Conference (Part A)

A

qz = I E { Z k l a k } l . In [I],it is noted that the Gaussian assumption, although not satisfied for the first iterations, is a good approximation when the number of iterations increases. The values of qz and ut are estimated for each data block and each iteration. An alternative approach, which does not require an estimation of vz and uf,is proposed in [8], [9]. In this case, the extrinsic information Zk at the input of the considered decoder is used t o extract a new estimate of the “a priori” probabilities t o be employed in the decoding step. In fact, the considered decoder interprets Zk as an approximation of the LLR of the a priori probabilities according t o (3)

from state m‘ t o state m, P { s k = m l S b - 1 = m‘} is the code transition probability, and the probability density functions a k ( m ) and p k ( m ) may be calculated using forward and backward recursions. In the following subsections, we present the two mentioned approaches to the use of the extrinsic information within a unified interpretation which, to our knowledge, has not been emphasized in the technical literature. A . Extrinsic information as Gaussian-distributed input In this case, the information symbols are assumed independent and identically distributed, i.e., P { a k = +I} = P { U k = -1) = 1/2. Hence, P { s k = m l S k - 1 = m’} = 1/2 for each possible transition. Since R k = ( X k , Z k ) and due to the assumed independence of X k and z k , we may write

which allows t o derive [8]

Therefore, the a posteriori probabilities generated by a decoder are used as a priori probabilities by the other one. 111. BCJR ALGORITHM We begin by summarizing the formulation of the BCJR algorithm [3], as given in [l], . . in order to intraduce the used -notation* Let us denote by the number of states of the considered constituent encoder (C = 2 ” - l , where U is the code constraint length) and s k the state of the encoder at time k. The information bit a k is associated with the transition from state s k - 1 to state s k . The generated LLR may be expressed as

L(Uk)

= In

(+I9 m’,m )ak - 1 ( m ‘ ) p k ( m )

m

k?c’ m’

m

x 7 k (m’

(5)

1,m‘, m ) a k - - l

(m’)pk

(m)

in which the various indices are i E {fl} and m’,m E { O , l , . .. ,C- l}, the probability density functions y k ( i , m’, m ) are defined as

n

7 k ( i , m’,m ) = p ( R k l U k

= i, s

k

where the conditional independence of zg from the transition has been used. Recalling (2) and the Gaussian assumption for Z k , we have p(Zklak

= i)=-

1

506

k

S}. ZZkvz

= m, s k - 1 = m’)

.P{ak = ilSh = m , S ~ - 1= m’} exp

{};”

(9)

where 6 k is a suitable constant, independent of i, m‘ and m. The LLR (5) may be easily expressed as L(ab)

2673,

= 2-

E.

+Wk

(10)

where ‘wk is the generated soft-output defined as

wk

A

=1n

m’ X r ’ k m

R I

= 6 k p ( x k l a k = i,s

’yk(i,m’,m)

(6)

m‘,m ) a k - l ( m ’ ) p k ( m )

( - 1,m’,m ) a k -1

( m ’ ) p k (m)

(11)

m‘

A

= ( x k , % k ) if Lk is interpreted as the outA put of a Gaussian meta-channel, or & = x k if Zk is used t o update the a priori probabilities, P { a k = i l s k = m , S k - 1 = m‘} is either one or zero depending on whether bit i is compatible with the transition where

exp{

I\”/ n)

x7‘k(+19

= m’}

2

Thus, we may express the probability density function (6) as

= m, S k - 1 = m’)

*P{Uk =ilsk =m,Sk-1 P { s k = m l S k - 1 = m’}

.XP{-%-S} 2

in which A

71k(i,m’, m ) = 6kp(xklak = i, sb= m, s ~=-m l ) ~ * P { a k = i ( s k = m, S I - 1 = m‘}. (12)

Global Telecommunications Conference - Globecorn‘99

General Conference (Port A)

B. Extrinsic information used to update the a-priori probabilities In this case, Rk = xk and [8]

A(.) PkexP{T} pk

exp

{ -?}

if uk = 1 if ak = -1

A

and R = { R k } f = l , where RE = ( X k , Z k ) if Z k is interpreted as the output of a Gaussian meta-channel, or RE = 21: if Z k is used to update the a priori probabilities, as in the case of the BCJR algorithm. The maximum likelihood sequence detection (MLSD) strategy corresponds t o the maximization of the following metric {Uk}F=l

= in [p(Rla)P{a}]= lnp(R1a)

+ In P{a} .

(17)

This metric may be arbitrarily multiplied by any constant independent of the information sequence. A . Extrinsic information as Gaussian-distributed input In this case, the information symbols are assumed independent and identically distributed, i.e., P { a k = +1} = P { a k = -1) = 1/2; therefore P{a} = $..Since Re = ( Z k , Z k ) and due t o the assumed Gaussianity and independence of Z k and Z k , we have

A(a) = lnp(x1a)

+ lnp(z1a) + lnP{a} (18)

where A is a suitable constant. Discarding terms independent of the information sequence, we obtain the equivalent metric

C. Discussion and heuristic approach Based on the above results, we may observe that, except for the irrelevant constants 6k and p k , the two approaches in sections 111-A and 111-B differ because the extrinsic information is weighted by different coefficients. This may be noted by comparing the expressions (9) and (14) of the probability density functions 7 k ( i , m ' , m ) ,where this coefficient is v z / u i in the first approach and 112 in the second one, and the relations (10) and (15) which implicitly define the extrinsic information wk, where the coefficient is 2vx/uz in the first approach and 1 in the second one. Based on this interpretation, a heuristic approach may be conceived with the aim of optimizing the receiver performance. To this purpose, the extrinsic information Z k is weighted by a parameter 0 which may be optimized by means of simulation. IV. SOVA Denoting by N the number of samples of each d a t a A A N A block, We define X = {Zk}:='=,, Z = { z k } k = 1 , a =

Global Telecommunications Conference - Globecom'99

which may be recursively computed by the following branch metrics Ak(8)

=xkck + -. zkakvz U2

4

(20)

A SOVA generates soft-outputs by considering only two paths in the trellis diagram-the maximum likelihood path and its strongest competitor. In the case of a binary RSC code, paths terminating in a state S k are associated to different information symbols, i.e., U k = i, i E {fl}.We denote the corresponding code symbols, recumulated and branch metrics by c t ' , A t ) and spectively. Assuming that the winning path includes state sk at time le, an initial reliability value of symbol ak is obtained by considering the absolute value of the difference between the cumulated metrics of the two paths terminating in state SE. Let us consider the case A P ' ) > AL-'). The initial reliability value is [4]-[6]

At),

507

General Conference (Part A)

loo

i approach Second approach

.-. First

1

-

10'' A

CL-*))

3($"

where 2 = + A(+1) k - 1 - A(-') k - 1 ' Similarly, in the case Ai-') > AY'), we have Ai-') - AV') - S. In general, the initial reliability value may be expressed as

This value is then updated at successive time instants (k 1,k 2 , . . .), according t o a suitable rule (41-[6]. Denoting by 'uk the final reliability value derived from ( 2 2 ) , it is related t o the extrinsic information W k of symbol ak by

+

1o-2 U U

m

I0.'

1o4

+

1o

- ~

0

1

2 Eb"0

3

4

Wl

Fig. 1. BER using the BCJR algorithm. The considered numbers of iterations are 1, 3, 6 and 18.

B. Extrinsic information used t o update the a priori probabilities In this case, Rk = 51: and the decoder assumes that P{ak = +1} and P{ak = -1) are given by ( 4 ) . Since P{a} = P{ak} and p ( x 1 a ) is Gaussian, discarding the terms independent of the information sequence, the sequence metric becomes

n,"=,

N

A(.) =

k=l

[y+

In P { a a } ]

.

(24)

cfl[a+ +

Adding the constant ln(1 e z k ) ] ,independent of a, we have the equivalent metric N

A(a) =

[y+ y ]

(25)

k=l

and the corresponding branch metrics

In this case, the final reliability value is

C. Discussion and heuristic approach As in the case of the BCJR algorithm, the two approaches differ for the constant which multiplies the received extrinsic information Z k . Even in this case, a heuristic approach may be conceived by introducing a

508

weighting parameter 0 which is optimized by means of simulation. In 171, the reliability value z k at the input of each component decoder is normalized by multiplying it by the factor 2 q z / u f ,and used t o update the a priori probabilities. Although 171 claims to use the second approach, because of this normalization the approach actually used is the first one. V. NUMERICAL RESULTS The performance of the proposed decoding schemes is assessed for the classical turbo code of rate 1 / 2 , 16state RSC constituent codes with generators G1 = 37, GO= 21 (octal notation), and 256 x 256 interleaver described in [l].The considered component decoders are based on the BCJR algorithm or the soft-output Viterbi decoder architecture proposed in 161. We refer t o the method described in sections 111-A and IV-A, in which the extrinsic information is assumed Gaussian, as first approach; similarly, the method described in sections 111-B and IV-B,in which the extrinsic information is used t o update the a priori probabilities, and that described in sections 111-C and IV-C are referred t o as second approach and third approach, respectively. In Fig. 1, the performance for the BCJR algorithm is shown for various numbers of iterations. It may be observed that the second approach makes a better use of the extrinsic information. Specifically, it entails a BER of for a value of Eb/No of approximately 0.7 dB. Moreover, the third (heuristic) approach does not give any improvement. In fact, for each iteration and each signal-to-noise ratio (SNR), the best value of 0, obtained by trial and error, is 1/2, which corresponds

Global Telecommunications Conference - Globecom'99

General Conference (Part A)

1 oo

-

-

First approach SecondaDDroach

lo-‘

1o-2

I,,,=.

1

+-e E J N O = 0 . 6 dB tiE J N O = 0 . 8 d 0

- E

N - 1 d0

[r

U

m

1 o0

1O 4

I I

0.0

1

4 7 Number of iterations

10

Fig. 2. Estimate of .rlz/uI versus the number of iterations and various values of SNR,for the B C J R algorithm and first approach.

t o the second approach. In the case of the first approach, Fig. 2 shows the estimated value of the ratio .rz/n:, averaged over more than 20 d a t a blocks, for the extrinsic information generated by the second decoder, as a function of the number of iterations and various values of SNR. It may be observed that, independently of the considered iteration, for values of SNR below a convergence threshold (about 0.7 dB), this ratio takes on values greater than 1/2. Therefore, the first approach overweights the extrinsic information at low SNR. This has been observed in [l], in which a heuristic normalization of the extrinsic information has been proposed with the aim of improving the performance at low SNR. Similar simulations have been performed considering component decoders based on SOVA. The performance for the three approaches considered in section N are shown in Fig. 3. Unlike the BCJR algorithm, the optimal value of the weight 8 for SOVA is a function of the number of decoding iterations. More precisely, the optimal value of 8, which is 8 N 0.2 at the first iteration, increases with the number of iterations, and reaches the value 8 N 0.4 from the 10-th iteration ahead. As observed in [7], SOVA overestimates the reliability values-the obtained results are consistent with these references. In fact, the coefficient 8 multiplies the extrinsic information z k generated by the other decoder; hence, a reduced value of 8 scales the sequence {zk}, correcting the overestimation. VI. CONCLUSIONS In this paper, turbo decoding schemes based on the BCJR algorithm and SOVA have been considered. A

Global Telecommunications Conference - Globecom‘99

1o

-~

0

1

2

3

4

E P , [dBl

Fig. 3. BER using SOVA. The numbers of iterations are 1, 3 and 18.

unified interpretation of different approaches to the use of the extrinsic information has been presented. In the

case of the BCJR algorithm, the best approach consists in updating the a priori probabilities. In the case of SOVA, a weighting parameter 8 has to be optimized for each decoding iteration.

REFERENCES [l]C. Berrou and A. Glavieux, “Near optimum error correcting coding and decoding: turbo-codes,” IEEE Trans. Commun., vol. 44, pp.1261-1271, October 1996. [2] B. Sklar, “A primer on turbo code concepts,” IEEE Commun. Magazine, pp. 94-102, December 1997. [3] L. R. Bahl, J. Cocke, F. Jelinek, and R. Flaviv, “Optimal decoding of linear codes for minimizing symbol error rate,” IEEE Dons. Inform. Theory, vol. 20, pp.284-284, March 1974. [4] G. Battail, “PondCration des symboles d6cod6 par l’algorithme de Viterbi,” Ann. Te‘lecornmun.,Fr., 42, N. 1-2, pp. 31-38, January 1987. [5] J. Hagenauer and P. Hoeher, “A Viterbi algorithm with softdecision outputs and its applications,” in Proc. IEEE Global Commun. Conf. (GLOBECOM’BO), Dallas, Texas, November 1989, pp. 1680-1686. [6] C. Berrou, P. Adde, E. Angui, and S. Faudeil, “A low complexity soft-output Viterbi decoder architecture,” Proc. IEEE Int. Conf. on Commun. (ICC’93), Geneve, Switzerland, June 1993, pp. 737-740. [7] L. Papke, P. Robertson and E. Villerbrun “Improved decoding with the SOVA in a parallel concatenated (turbo-code) scheme.” in Proc. Intern. Conf. Comm. (ICC’961. J u n e 1996. pp. 102-106, l8l P. Robertson. “Illuminatinn the structure of code and decoder of parallel concatenated rezursive systematic (turbo) codes,” an Proc. IEEE Global Commun. Conf. (GLOBECOM’94), San Francisco, CA, U.S.A., December 1994, pp. 1298-1303. [9] C. Douillard, M. Jezequel, C. Berrou, A. Picart, P. Didier a n d A. Glavieux, ”Iterative correction of intersymbol interference: turbo-equalization”, Europ. Trans. on Telecommun. ( E T T ) , vol. 6, pp. 507-511, September/October 1995.

..

509