distance spectrum based improved upper bounds for parallel and

DISTANCE SPECTRUM BASED IMPROVED UPPER BOUNDS FOR PARALLEL AND SERIAL CONCATENATED TURBO CODES Igal Sason and Shlomo Shamai (Shitz)

Department of Electrical Engineering Technion|Israel Institute of Technology Haifa 32000, Israel ABSTRACT The ensemble performance of parallel and serial concatenated turbo codes is considered, where the ensemble is generated by a uniform choice of the interleaver and of the component codes taken from the set of time varying recursive systematic convolutional codes. Following the derivation of the ensemble weight enumeration functions of random parallel and serial concatenated codes, improved upper bounds on the bit and message error probabilities of these ensembles of codes are derived and the inuence of the interleaver length N and the memory length of the component codes m are investigated. The improved bounding technique proposed here, which is based on the tangential-sphere bound, is compared to the conventional union bound and to a recent alternative bounding technique by Duman & Salehi which incorporates modied Gallager bounds. The advantage of the derived bounds is demonstrated, especially in the region above the cuto rate, including thus a portion of the rate region where the performance of turbo codes is most appealing.

1. INTRODUCTION The discovery of turbo codes is one of the most recent exciting developments in coding theory. The codes demonstrated near Shannon limit performance on a Gaussian channel with relatively simple component codes and large interleavers. In addition to simulations, theoretical upper bounds for bit error rates of turbo codes were developed. Since explicit results for a particular chosen structure of an interleaver and component codes appear yet intractable, the bounds are developed as averages over certain ensembles, featuring random coding properties. However, since mostly these bounds are based on a union bounding technique, they render apparently useless results for adequately low values of energy per bit to noise power density NEb0 , below the value that corresponds to the cuto rate (R0 ), a region which is of particular interest for the turbo codes operation 6], 7], 8]. An upper bound to the bit error probability of a parallel concatenated coding scheme averaged over the interleavers of a given length was proposed 4]. A probabilistic interleaver called the `uniform interleaver' was introduced, permitting an easy derivation of the weight distribution (WD) function of the parallel concatenated code relying on the WD function of its component codes. A similar union

upper bound to the bit error probability of a serial concatenated coding scheme averaged over the interleavers of a given length was reported in 5]. Simulations in 6] for serial Econcatenation put in evidence an interleaver gain even for Nb0 values below the value that corresponds to the cuto rate but reasonably above the value that corresponds to the channel capacity. The random ensemble WD functions of parallel and serial concatenated codes are derived in 7], 8] respectively. The ensemble is generated by a random uniform choice of the interleaver and also a uniform choice of the component codes taken from the set of time varying recursive convolutional codes with a given memory length. These derivations reduce the performance behavior of random turbo codes to a two parameter family: the interleaving length N and the memory length of the component codes m, assumed here to be the same. Following the derivations of the ensemble WD functions, the union bound was employed to provide upper bounds on the bit error probabilities of the ensembles of random serial and parallel concatenated codes for the AWGN channel based on maximum likelihood decoding 7], 8]. An upper bound on the word error and bit error probabilities of turbo codes with maximum likelihood decoding is derived 9], using a modied version of Gallager bound rather than the standard union bound. This result is a generalization of the transfer function bounds providing a tighter result as compared to the union bound. Thus, the bound in 9] is useful for some range below the channel cuto rate as it does not diverge at the cuto rate like the union bound. Typically, the upper bound on the block error probability is a tight bound for NE0b values 0.5 dB below the cuto rate, and diverges 0.8{1.0 dB below, and the upper bound on the BER is even worse. These bounds on the bit and block error probabilities of turbo codes are not covering thus the full range of their usefulness. The basic idea of the ensemble (average over interleaver and component codes) upper bound here 10] is based on partitioning the code to subcodes, such that each subcode includes all the codewords with the same information weight i or a set of information weights (but not necessarily with the same total Hamming weight d as in 9]) including also the all zero codeword in each subcode. Then the tangential sphere bound devised for block codes 16], is applied on each individual subcode that results in an upper bound on the message error probability for each subcode, given that the

all zero codeword was transmitted. Since all the nonzero codewords are systematic and have the same information weight and the same length, for each subcode the relation between the bounds on the bit error probability and on the message error probability is evident. Since the whole code is linear, then the bit error probability is independent on the transmitted codeword. The upper bound on the bit error probability follows by the union bound for these subcodes (the number of which is reduced as compared to 9]), and using the linearity of the code itself. The partition of the code to subcodes is further optimized under some rules, and this improves the upper bound considerably. A comparison between the bounding techniques here and in 9], demonstrates that our bounding technique is advantageous and it extends the region of NEb0 for which the bounds are useful.

2. PRELIMINARIES In this section, we rst state the underlying assumptions on which our bounds are based, introduce notations and basic relations from 2]{ 10], which apply to parallel and serial concatenated turbo codes, and state further relations and comments useful to our analysis and conclusions.

2.1. Assumptions:

In our analysis, the channel is assumed to be Additive White Gaussian (AWGN) and the signals transmitted through the channel are assumed to be antipodal. The detection is coherent and the decoding is maximum likelihood. The coding structures being considered are of serial and parallel concatenated turbo codes: the component codes are assumed to be systematic and time-varying recursive systematic convolutional (RSC) codes with the same memory length m, and a rate of 12 and also a random interleaving of length N is incorporated in both structures. This type of random (uniform) interleaving is a probabilistic structure that takes into consideration all the possible interleaving permutations, including the option of a non interleaved code as a particular case, where all interleaving congurations are uniformly weighted. A termination with m additional cycles of the shift register is assumed 3], though the results for large values of interleaver length (N m), are insensitive to termination method.

2.2. Notations and relations:

2.2.1. The spectra of parallel and serial concatenated codes: A serial concatenated turbo code cs with component co and ci as outer and inner codes is the rst concatenation structure considered here. The rate of the code cs is R, in units bits , that is also the product of the rates of its compoof symbol nent codes. The uniform interleaver situated between the component codes is operating on bits (and not on symbols) and has a length of N and the common memory length of the component codes is m. The number of codewords of the code cs that are encoded by information bits of Hamming weight w and having a total Hamming weight of h is designated by Acws h . Here, for serially concatenated codes, the number of information

bits is the product of N with the rate of the outer code R(0) , since N is also the length of a codeword of c0 . As the component codes are assumed to be systematic, also the serial concatenated code cs is systematic. Then, it follows that Acws h = 0 if w > h. Similar denitions related to the component codes are derived for Acw0 ` and Ac`ih . As before, since the component codesc are systematic, that implies that for w > ` or ` > h: Aw0 ` = 0 or Ac`ih = 0 respectively. These notations are consistent with those used in papers 5], 6] that deal with serial concatenated codes. However, unlike papers 4], 7] that address parallel concatenated turbo codes, the parameter h of Acws h is the overall Hamming weight of a codeword of cs , and not only of its parity bits as is for parallel concatenation (the parameter h of Acwp h , where cp is a parallel concatenated code, is the Hamming weight of the parity bits of cp ). 2.2.2. The tangential sphere bound The tangential sphere bound is an upper bound on the message error probability, derived in 16]. Suppose that the signals transmitted through an AWGN channel for each message are of the same energy E . It can be shown that the tangential sphere bound is always tighter than the tangential bound 13] and the union bound, especially for moderate values of NE0b 16]. The properties of the tangential-sphere bound 16] (and also the upper bounds in 12], 13]) follow by the central inequality which governs the bound, Prob (A)
Prob (Z 2 B A) + Prob (Z 62 B) : (1) In the case of the tangential sphere bound, A is an event that represents a message decoding error, B is an n-dimensional cone with a half angle  and radius r, and Z is the noise vector added to the received signal by the channel. Since an optimization is carried over  and r, for deriving the tightest upper bound on the message error probability, within this family, it follows that the upper bound does not exceed 1, since in the trivial case that  tends to 4 Prob (A)
1 (as the rst term in the zero, we get pe = right side of (1) becomes zero and the second term becomes 1). On the other hand, the union bound may exceed 1, especially for moderate values of Eb =N0 . The mathematical derivation of this bound is presented in 16]. In 17] the advantage of the tangential sphere bound over the union, tangential 13], sphere 14], Viterbi's 11] and Duman & Salehi 9] upper-bounds is demonstrated for a variety of block codes.

3. MAIN RESULTS Our upper bound on the bit error probability derived here 10], is based on partitioning of the code to subcodes, so that each of the subcodes ci comprises all the non zero codewords with the same information weight i together with the all zero codeword. For a serially concatenated code cs , the spectra fSk(i) g of the subcode ci Sk(i) = Aci sk k = i i + 1  : : : 32N + i  1
i
N2 (2)

For a parallel concatenated code cp , the spectra fSk(i) g of the subcode ci is as follows

Sk(i) = Aci pk;i k = i i + 1 : : : 2N + i 1
i
N : (3) The mathematical derivation of these coecients is presented in 10]. The basic dierence between Eqs. (2),(3) is based on the fact that the parameter k in the notation Aci sk is the whole Hamming weight of the serially concatenated code cs , while the same parameter in the notation Aci pk is the Hamming weight of the parity bits of a codeword received by a systematic parallel concatenated code cp . The second step needed for deriving the upper bound on the bit error probability is to upper bound Pei j0 , that is the message error probability in the subcode ci , given that the all zero codeword was transmitted. By the tangential sphere bound in 16], we nd the following upper bound on Pei j0 for the subcode ci based on its spectra as represented in Eqs. (2),(3) for serial and parallel concatenations respectively. Note that for the case of serial concatenation with component codes of rate 12 and an interleaver of length N : n = 2N is the length of each codeword, while n = KN for the parallel concatenation with K component codes under the same assumptions. The optimal radius ri that corresponds to the spectra of each subcode ci is determined for each subcode separately. Although the subcodes ci are not necessarily linear, the tangential sphere bound is also valid in general as an upper bound on the message error probability given the all zero codeword is transmitted. As the whole code is linear, the message error probability is independent on the transmitted codeword and thus based on the union bound applied to the subcodes, we get the following upper bounds on the bit error probability

8 N > X > 2 Pb
iP > > N > X 1 > P
: b N i Pei j0 2

i=1

for serial concatenation for parallel concatenation :

(4) The derivation of the upper bound here 10] is done by combining the union bound (4) for the subcodes with the tangential sphere bound utilized for each subcode ci . An expected outcome presented in the four gures of this paper is that the standard union bound yields meaningful results only in the rate region below the cuto rate, excluding the portion of the rate region where the performance of turbo codes is most appealing. An improved bounding technique by Duman and Salehi has been published recently 9], when a modied version of Gallager bound replaces the standard union bound. Though the Gallager bound as in 12] achieves the channel capacity, there are a number of places in the derivation of the upper bound in 9], where it is loosened as compared to the actual value for the probability of error. The weakening of the Gallager bound is more dominant in the

derivation of the upper bound on the bit error probability 9] than of the upper bound on the word error probability, since in the rst case the code is partitioned to subcodes ci d that includes all the code words with the same information Hamming weight i and total weight d including also the all zero codeword. The upper bound on the bit error probability presented here 10], is based on partitioning the code to subcodes, each includes all the codewords with the same information weight i (a single constraint on the partitioning of the code). Improved partitions are also addressed 10]. The main dierence between the two methods of the code's partitioning is a result of the fact that for the tangential sphere bound all the signals must only have the same energy (geometrically, all the codewords are on the same sphere), and the constraint on i (being constant for each subcode) is just used for the case of the transition from the upper bound on the message error probability to the upper bound on the bit error probability for each subcode, assuming that the all zero codeword is transmitted. Since all the codewords are of the same length and also the energy per bit is constant, all the signals possess inherently the same energy. The technique in 9] however, even for the derivation of the message error probability implies that the subcodes cd , must have a constant Hamming weight for all their nonzero codewords. This additional constraint results in a partition with an increase number of subcodes as compared to our method here 10] hence due to the union bound used for the subcodes to a looser result. In addition, the tangential sphere bound applied here on each individual subcode, was proved to be in most cases a tighter upper bound on the message error probability for block codes than a variety of other upper bounds 16], 17] giving thus another advantage to the upper bound presented here. Typically, the upper bound on the bit error probability derived in 9] is tight for NEb0 values 0.5 dB below the cuto rate and diverges 0.8{1.0 dB below, when N=500 bits, not covering thus the full range of usefulness of turbo codes. By the improved bounding technique presented here, it was demonstrated (see Fig. 3) that for an interleaver length of N=500 bits, the upper bound on Ethe bit error probability is a reasonably tight bound for N0b values approximately 1E dB below the cuto rate, extending thus the region of b N0 for which the bounds are useful. Further results on the bit and word error probability for dierent constructions of serial and parallel concatenated codes are shown in Figs. 1{ 3. Note that the bounds in Fig. 3 are for xed component parallel concatenated codes. Parallel to the results presented in 7], 8], it is also veried here that increasing the memory length of the component codes m above log2 N aects negligibly the ensemble performance of either parallel or serial concatenated codes, while increasing the decoding complexity of the codes. This result is proved analytically in 8], by showing rst that the average activity of a time-varying recursive convolutional code with a memory length of m is 2m+1 ; 2. Choosing now m = log2 N , results an average activity time that is approximately twice the interleaver length, and this result substantiates the feature that for values of m exceeding log2 N , the weight distribution is dominated by the distribution of the rst non zero input (since with high probability, the encoder once activated will not clear within the observation interval).

4. SUMMARY AND CONCLUSIONS The ensemble performance of serial and parallel concatenated turbo codes is investigated. The ensemble is generated by a uniform choice over all possible interleavers and also a uniform choice over the component codes taken from the set of time varying recursive convolutional codes. We focus on rate 12 component codes and examine also the case of three component codes of rate 12 for the case of parallel concatenation, which gives rise of an overall code of rate 14 (the same overall rate as for serial concatenation with rate 1 2 component codes). In parallel to results reported in 7], 8], the performance behavior of randomly concatenated and interleaved codes is reduced to a three parameter family: the interleaver length N , the memory length m of the component codes (assumed here to be equal) and the number K of component codes in the case of parallel concatenation (for serial concatenation two (K = 2) component codes are assumed: the inner and outer codes). It is also veried here that increasing the memory length of the component codes m above log2 N aects negligibly the ensemble performance of either parallel or serial concatenated codes, while increasing the decoding complexity of the codes. The technique for calculating the weight enumerators of xed convolutional codes 18], is used to compare our improved bounding technique with a recent alternative bounding technique 9] by Duman and Salehi which incorporates a modied Gallager bound. By the improved bounding technique presented here 10], it is demonstrated (see Figs. 1{3), that for an interleaver length of N = 500, the upper bound on the bit error probability is a reasonably tight bound for Eb values, approximately 1 dB below the value of Eb , that N0 N0 corresponds to the cuto rate, extending thus the region of Eb for which the bounds derived in 9] are useful. N0 For the derivation of an upper bound on the bit error probability of turbo codes, the way the codes are partitioned to subcodes is critical. Therefore, the partition of these codes is dependent on their distance spectra, and it is optimized by a computer under some regularity rules. This partition ensures also that the upper bound on the bit error probability is always less than the upper bound on the block error probability, in contrast to alternative bounding techniques reported recently 9]. The reason is that for these bounds and also the union bound, there is no possibility to consider dierent partitions, and also the code is partitioned to subcodes that include all the codewords with the same information Hamming weight (I ) and total Hamming weight (D) and the all-zero codeword, leaving no room for any degree of freedom in the partition.

5. REFERENCES 1] C. Berrou, A. Glavieux and P. Thitimajshima, "Near Shannon limit error correcting coding and decoding: turbo{codes", Proc. of International Conference on Communications , pp. 1064{1070, Geneva, Switzerland, May 23{26, 1993.

2] S. Benedetto and G. Montorsi, \Performance evaluation of turbo codes", Electronic Letters , vol. 31, no. 3, pp. 163{165, February 1995. 3] D. Divsalar and F. Pollara, \Turbo codes for PCS applications", Proceedings of IEEE ICC`95 , pp. 54{59, Seattle, Washington, June 1995. 4] S. Benedetto and G. Montorsi, \Unveiling turbo codes: some results on parallel concatenated coding schemes", IEEE Trans. Information Theory , Vol. 42, No. 2, pp. 409{428, March 1996. 5] S. Benedetto, G. Montorsi, D. Divsalar and F. Pollara, \Serial concatenation of interleaved codes: performance analysis, design, and iterative decoding, JPL TDA Progress Report, 42{126, August 15, 1996. 6] A.J. Viterbi, A.M. Viterbi, J. Nicolas and N.T. Sindushyana, \Perspectives on interleaved concatenated codes with iterative soft-output decoding", Proceedings of the International Symposium on Turbo Codes & Related Topics , pp. 47{54, Brest, France, 3{5 September, 1997. 7] E. Telatar and R. Urbanke, \On the ensemble performance of turbo codes", Proceedings 1997 IEEE Int. Symp. Information Theory (ISIT`97), pp. 105, Ulm Germany, June 29{July 4, 1997. 8] I. Sason and S. Shamai (Shitz), \On union bounds for random serially concatenated codes with maximum likelihood decoding", Technical Report EE{110, Technion, September 1997, Proc. French-Israel Workshop in Coding and Information Integrity , Ein-Boqeq, Dead-Sea, Israel, 27{29, October 1997. 9] T.M. Duman and M. Salehi, \New performance bounds for turbo codes", Proceedings of 1997 Global Communications Conference (GLOBECOM'97), pp. 634{638, USA, Phoenix, Arizona, November 4{8, 1997. 10] I. Sason and S. Shamai, \Improved distance spectrum based performance bounds for parallel and serial concatenated turbo codes", Technical Report, EE Pub. No. 115, Technion, October 1997. 11] A.J. Viterbi, Principles of coherent communication, Chapter 8, pp. 242{244, McGraw-Hill, 1966. 12] R.G. Gallager, Information Theory and Reliable Communications, Wiley, N.Y. 1968. 13] E.R. Berlekamp, \The technology of error correction codes", Proc. of the IEEE , Vol. 68, No. 5, pp. 564{ 593, May 1980. 14] B. Hughes, \On the error probability of signals in additive white Gaussian noise", IEEE Trans. on Information Theory , Vol. 37, No. 1, pp. 151{155, January 1991. 15] L.C. Perez, J. Seghers and D.J. Costello, \A distance spectrum interpretation of turbo codes", IEEE Trans. on Information Theory , vol. 42, no. 6, pp. 1698{1709, November 1996. 16] G. Poltyrev, \Bounds on the decoding error probability of binary linear codes via their spectra", IEEE Trans. on Information Theory , Vol. 40, No. 10,

0

10

−1

m=3

10

Upper bound on the BER

pp. 1261{1271, October 1996. See also H. Herzberg and G. Poltyrev, \The error probability of M-ary PSK block coded modulation schemes", IEEE Trans. on Communications , vol . 44, no. 4, pp. 427{433, April 1996. 17] I. Sason and S. Shamai, \On upper bounds for the block error decoding probability of block codes", in preparation. 18] R.J. McEliece, \How to compute weight enumerators for convolutional codes", conference in honor of P.G. Farrel, Lancaster, England, January 26{28, 1998.

−2

10

m=4

−3

10

Ensemble performance of random serially concatenated turbo codes

0

10

m=5

−4

10

o : N = 100

m=6

+ : N = 200

m=7

upper bounds on the word error probability

x: N = 400

m=8

−5

10 −1

10

0

0.5

1.5 Eb/No [dB]

2

2.5

3

Figure 2: Upper bounds on the bit error probability of serially concatenated random codes with inner and outer codes of rate 12 and a random uniform interleaver of length N = 200, as a function of m = 3 4 5 6 7 8 - the memory length of its components.

−2

10

Upper bounds on the bit error rate for 5/7 code with interl eaver length 500

0

10

−3

10

1

0

0.5

1

1.5

2

2.5

Eb/No [dB] 0

10

−1

Upper bounds on the bit error probability

10

−1

upper bound on the bit error probability

10

−2

10

−3

10

o + * −

−4

N = 50 N = 100 N = 200 − N = 400

tangential sphere bounds

union bound

with some partitions

in Q − form

−3

10

iterative decdoing

−4

10

simulations

−5

10

10

−5

10

−2

10

−6

0

0.5

1

1.5 Eb/ No [dB]

2

2.5

3

Figure 1: A comparison between upper bounds on the ensemble performance of serially concatenated random codes in a Gaussian channel with maximum likelihood decoding. The memory length of the two component codes is m = 5, the overall rate is R = 14 and the interleaver length is N = 50 100 200 400 bits. a. Upper bounds on the word error probability. The improved bound is based on the tangential sphere bound and is compared to the union bound. b. Upper bounds on the bit error probability. The improved bound is based on the tangential sphere bound and a partition of the code with respect to the information weight (I ) and it is compared to the union bound.

10

0.4

0.6

0.8

1

1.2

1.4 Eb/No [dB]

1.6

1.8

2

2.2

2.4

Figure 3: A comparison of upper bounds on the bit error probability of parallel concatenated (turbo) codes in a Gaussian channel with maximum likelihood decoding versus simulation results of iterative decoding. The generators of thehtwo component i codes are the same, G1 (D) = G2 (D) = 1 1+1+DD+D2 2 and the random interleaver is of length N = 500 bits. The simulation results of the iterative decoding are based on 9]. The upper bounds on the bit error probability are based on the tangential sphere bound with three dierent partitions of the code with respect to the information weight I and are also compared with the union bound. The dashed curve is the improved upper bound on the message error probability.