Nonsystematic Turbo Codes

1

Nonsystematic Turbo Codes Adrish Banerjee, Member, IEEE, Francesca Vatta, Member, IEEE, Bartolo Scanavino, Member, IEEE and Daniel J. Costello, Jr. Fellow, IEEE

Abstract— In this paper, we introduce the concept of nonsystematic turbo codes and compare them with classical systematic turbo codes. Nonsystematic turbo codes can achieve lower error floors than systematic turbo codes, because of their superior effective free distance properties. Moreover, they can achieve comparable performance in the waterfall region if the nonsystematic constituent encoder has a low weight feedforward inverse. A uniform interleaver analysis is used to show that rate R = 1/3 turbo codes using nonsystematic constituent encoders have larger effective free distances than when systematic constituent encoders are used. Also, mutual information based transfer characteristics and EXIT charts are used to show that rate R = 1/3 turbo codes with nonsystematic constituent encoders having low weight feedforward inverses achieve convergence thresholds comparable to those achieved with systematic constituent encoders. Catastrophic encoders, which do not possess a feedforward inverse, are shown to be capable of achieving low convergence thresholds by doping the code with a small fraction of systematic bits. Finally, we give tables of good nonsystematic turbo codes and present simulation results comparing the performance of systematic and nonsystematic turbo codes. Index Terms— Nonsystematic turbo codes, iterative decoding convergence thresholds, effective free distance, convolutional encoder feedforward inverses

I. I NTRODUCTION Classical turbo codes, introduced by Berrou et al. [1], are based on the parallel concatenation of two systematic feedback convolutional encoders. We refer to these as systematic turbo codes. By employing systematic encoders, a turbo decoder is able to provide reliable extrinsic information about the information bits in the critical initial stages of the iterative decoding process. For low signal-to-noise ratios (SNRs), large block lengths, and a large number of decoding iterations, this results in a bit error rate (BER) performance curve that is very steep. This is known as the waterfall region of the BER performance curve. On the other hand, for high SNRs, because systematic turbo codes typically contain a few codewords of low weight, there is very little improvement in the BER with increasing SNR, resulting in a flattening of the performance curve. This is known as the error floor region of the BER performance curve. In this paper, we investigate the parallel concatenation of two nonsystematic feedback convolutional encoders. We Adrish Banerjee is with Indian Institute of Technology, Kanpur, India. He was with the University of Notre Dame, Notre Dame, IN, USA. Francesca Vatta is with Universit`a di Trieste, Trieste, Italy, Bartolo Scanavino is with CERCOM, Politecnico di Torino, Torino, Italy and Daniel J. Costello, Jr. is with University of Notre Dame, Notre Dame, USA. This work was supported by NSF Grant CCR02-05310, NASA Grant NAG5-12792, and the State of Indiana 21st Century Science and Technology Fund. Part of this work was presented at the Mathematical Theory of Network Systems Conference, Notre Dame, IN, 2002 and at the IEEE International Symposium on Information Theory, Yokohama, Japan, 2003.

refer to these as nonsystematic turbo codes. Nonsystematic encoders have larger values of d2 , the minimum weight codeword produced by a weight-two information sequence, than systematic encoders of the same complexity [2]. This implies that nonsystematic turbo codes have larger values of effective free distance deff than systematic turbo codes [3]. Thus, for large block lengths, nonsystematic turbo codes will typically exhibit better performance in the error floor region than systematic turbo codes of the same complexity. However, not all nonsystematic encoders provide reliable extrinsic information in the initial stages of iterative decoding. In other words, some nonsystematic turbo codes exhibit weak convergence behavior with iterative decoding, and thus their performance in the waterfall region can be poor. In this paper, we identify a property of nonsystematic encoders that results in good convergence behavior, thus making them attractive choices for use in nonsystematic turbo codes. In section II, we define and give a brief introduction to nonsystematic turbo codes. In section III, we compare the effective free distances of systematic and nonsystematic turbo codes. Then, in section IV, we explore the relationship between the weights of the feedforward inverses of nonsystematic encoders and the convergence behavior of these encoders when used in a turbo configuration. In section V, we use mutual information based transfer characteristics and EXtrinsic Information Transfer (EXIT) charts [4] to analyze the convergence behavior of nonsystematic turbo codes, and tables of good constituent encoders for nonsystematic turbo codes are presented along with simulation results. In section VI, we demonstrate how even catastrophic nonsystematic encoders can be made to exhibit good convergence properties by employing code doping [5], i.e., by replacing some of the parity bits with information bits. In section VII, we conclude and summarize the main results of the paper. II. N ONSYSTEMATIC T URBO C ODES A classical rate R = 1/3 systematic turbo code involves the parallel concatenation of two rate R = 1/2 systematic feedback encoders with generator matrix   convolutional (1)

G1 (D) = 1

g1 (D) (0)

g1 (D)

[1], where the systematic bits from

the second encoder are not transmitted. Thus it is possible to view the second encoder as a rate R = 1 nonsystematic feedback encoder with generator matrix  convolutional  (1)

G2 (D) =

g2 (D) (0)

g2 (D)

. If

(1)

g1 (D) (0)

g1 (D)

(1)

=

g2 (D) (0)

g2 (D)

, the turbo code is

symmetric; otherwise, it is asymmetric [6]. If the rate R = 1/2 systematic feedback encoder is replaced by a rate R = 1/2 nonsystematic feedback encoder, we obtain a nonsystematic turbo code [7]. In Figure 1, a rate R = 1/3 nonsystematic

2

turbo code is shown. The first encoder is a rate R = 1/2 nonsystematic feedback convolutional encoder with generator (1) (2) g (D) g1 (D) matrix G1 (D) = [ 1(0) ], where each of the (0) g1 (D)

g1 (D)

(1)

(0)

numerator and denominator pairs, i.e., (g1 (D), g1 (D)) and (2) (0) (g1 (D), g1 (D)), have no common factors. This encoder has a total of 2ν states, where (0)

(1)

(2)

ν = max(deg(g1 (D)), deg(g1 (D)), deg(g1 (D))) (1) (1)

is the constraint length of the encoder. (Note that if g1 (D) (0) (2) (0) = g1 (D) or g1 (D) = g1 (D) , a rate R = 1/3 systematic turbo code results.) To obtain a symmetric nonsystematic turbo code, one of the first constituent generators is chosen as the second rate R = 1 encoder, i.e., (1)

(1)

g2 (D) (0)

g2 (D)

=

g1 (D) (0)

g1 (D)

(1)

or

g2 (D) (0)

g2 (D)

(2)

=

g1 (D) (0)

g1 (D)

(2)

.

In the asymmetric case, a different generator is chosen for the second rate R = 1 encoder, i.e., (1)

g2 (D) (0)

g2 (D)

(1)

6=

g1 (D) (0)

g1 (D)

(1)

and

g2 (D) (0)

g2 (D)

(2)

6=

g1 (D) (0)

g1 (D)

.

(3)

To decode nonsystematic turbo codes, we can assume received a posteriori probability (APP) log-likelihood ratios (L-values) equal to 0 for the (not transmitted) information bits and apply the same soft-in-soft-out (SISO) turbo decoding procedure as in the case of systematic turbo codes [8], [9]. In the next section, we examine the distance properties of nonsystematic turbo codes that result in better performance in the error floor region compared to systematic turbo codes of the same complexity. III. E FFECTIVE F REE D ISTANCES OF N ONSYSTEMATIC T URBO C ODES Every nonsystematic feedback convolutional encoder has an equivalent systematic feedback encoder, but the encoder mapping between information sequences and codewords is different. For example, the rate R = 1/2 nonsystematic feedback h (1) convolutional i encoder with generator matrix Gns (D) = g (D) g(2) (D) has an equivalent rate R = 1/2 systematic g(0) (D) g(0) (D) feedback convolutional encoder with generator matrix    (1)  g(2) (D) g (D) g(2) (D) Gs (D) = 1 ≡ , (4) g(1) (D) g(0) (D) g(0) (D) but the input sequence g(0) (D) produces the output sequence [g(1) (D) g(2) (D)] for the nonsystematic encoder, whereas the input sequence g(1) (D) produces the same output sequence for the systematic encoder. This has only a minor effect on the performance of a convolutional code, but it can play an important role for a turbo code, due to the action of the interleaver. With regard to the distance spectrum of turbo codes, it is important that information sequences giving low weight output sequences for the first encoder are interleaved so that the second encoder produces a high weight output sequence. For feedback encoders, the output sequence will be of finite

weight if and only if the input sequence terminates the encoder, i.e., it must be divisible by the feedback polynomial g (0) (D). Weight-one input sequences cannot terminate a feedback encoder, so they can be ignored in computing the distance spectrum. However, weight-two input sequences can terminate a feedback encoder, possibly resulting in a low weight output sequence. For any feedback polynomial g (0) (D), there exists a family of weight-two input sequences of the form D j (Dlq + 1), j ≥ 0, l ≥ 1, that terminate the encoder and produce finite weight output sequences, where q is called the cycle length of g(0) (D). When l = 1, D q + 1 is the smallest degree binomial that contains g(0) (D) as a factor, and the resulting output sequence has the minimum possible weight corresponding to a weight-two input sequence. For randomly chosen interleavers, a uniform interleaver analysis shows that higher weight input sequences that terminate one encoder are less likely to terminate the second encoder, especially for large interleaver sizes [10]. Thus, for large block lengths, weight-two input sequences whose ones are separated by the cycle length of the feedback polynomial typically produce the codewords that dominate the low weight portion of the distance spectrum of a turbo code. For this reason, the effective free distance of a turbo code, deff , is defined as the minimum weight codeword corresponding to an input sequence of weight-two [3]. For a rate R = k/n convolutional encoder, denote the minimum weight of the i-th output parity sequence (1 ≤ i ≤ n − k for systematic encoders, and 1 ≤ i ≤ n for nonsystematic encoders) corresponding to a weight-two input sequence as d2 (i). Then the weight-two input minimum distances of systematic and nonsystematic encoders, namely ds2 and dns 2 , can be written as [2] ds2

=

2+

n−k X

d2 (i)

Systematic case

(5)

i=1

dns 2

=

n X

d2 (i).

Nonsystematic case (6)

i=1

The effective free distances of systematic (dseff ) and nonsys1 tematic (dns eff ) turbo codes can now be written as dseff dns eff

(2)

=

ds2 + d2

=

dns 2

+

(2) d2 ,

Systematic case

(7)

Nonsystematic case (8)

where ds2 and dns 2 are, respectively, the weight-two input minimum distances of the rate R = 1/2 systematic and (2) nonsystematic first encoders, while d2 is the weight-two input minimum distance of the rate R = 1 second encoder. Except for trivial generators of the form D j /(Dm + 1), 0 ≤ s j ≤ m, d2 (i) ≥ 2, and hence dns 2 ≥ d2 . This implies that nonsystematic turbo codes can achieve larger effective free distances than systematic turbo codes. To illustrate this point, in Table I we list the maximum effective free distances of rate 1 This assumes a worst case, i.e., that one of the weight-two input sequences from the first encoder that gives output weight ds2 (or dns 2 ) gets interleaved into a weight-two input sequence for the second encoder that gives output weight (2) d2 . This situation can sometimes be avoided by using specially designed interleavers, such as S-random interleavers [11].

3

R = 1/3 systematic and nonsystematic symmetric turbo codes for encoder constraint lengths ν = 2, 3, · · · , 6. The fact that nonsystematic turbo codes can achieve larger effective free distances than systematic turbo codes implies that for large block lengths, the use of nonsystematic turbo codes will typically result in lower error floors. This follows from the facts that a randomly chosen interleaver is most likely to result in a minimum-weight codeword produced by a weight-two input sequence, i.e., the free distance of the code is deff , and that the error floor performance of a turbo code is determined by its free distance [12]. (It may be possible to use S-random interleavers to further lower the error floors of both systematic and nonsystematic turbo codes.) However, for some nonsystematic encoders, the iterative APP decoding algorithm provides poor extrinsic estimates of the information bits after the first iteration of decoding, so that decoding may not converge to the maximum likelihood (ML) solution [7]. To ensure good initial extrinsic estimates, we seek nonsystematic encoders with an “almost” systematic property. Intuitively, we can view the turbo decoding algorithm for nonsystematic encoders as implicitly producing estimates of L-values for the (not transmitted) information bits based on the L-values of the (transmitted) parity bits. These implicit Lvalue estimates of the information bits play the same role in aiding the convergence of iterative decoding for nonsystematic turbo codes as do the explicitly available L-value estimates in the case of systematic turbo codes. It is reasonable to assume that these implicit estimates will be more reliable if fewer parity bits affect the value of a given information bit. Since the number of parity bits that affect the value of an information bit is equal to the weight of (number of nonzero terms in) the feedforward encoder inverse, we will focus our attention on nonsystematic feedback encoders with low weight feedforward inverses. In the next section we study the convergence behavior of nonsystematic turbo codes when the feedforward inverse of the first constituent encoder has low weight. IV. I NVERSES OF N ONSYSTEMATIC F EEDBACK ChONVOLUTIONALi E NCODERS (1)

(2)

g (D) represent a rate R = 1/2 Let G(D) = gg(0) (D) (D) g(0) (D) nonsystematic feedback convolutional encoder. To recover the information sequence from the output of the encoder, we need an inverse matrix, i.e., a 2 × 1 matrix G−1 (D) that satisfies

G(D)G−1 (D) = D l

(9)

for some l ≥ 0. A rate R = 1/2 nonsystematic convolutional encoder G(D) that has a weight-two feedforward inverse G−1 (D) =  T 1 Dβ of delay α, such that G(D)G−1 (D) = D α , where α and β are non-negative integers, is known as a quick-look in (QLI) encoder. For feedback encoders, a QLI encoder satisfies the following equation g(1) (D) g(2) (D) + Dβ (0) = Dα . (0) g (D) g (D) Moreover, if sequence and

u(D) V(D)

(10)

represents  (0) the (1)information  = v (D) v (D) =

h i (1) g(2) (D) u(D) gg(0) (D) represents the encoder output, u(D) (0) (D) g (D) we obtain: V(D)G−1 (D) = v(0) (D) + D β v(1) (D) = D α u(D), (11) where u(D) can be recovered from the encoder output V(D) using an encoder inverse with weight only two. From (11), we see that an error in recovering a particular bit in u(D) can be caused either by an error in v (0) (D) or by an error in v(1) (D). Thus we say that QLI encoders have an error amplification factor of A = 2. In general, for a nonsystematic feedback encoder with feedforward inverse  (1) T G−1 (D) = {˜ g (D)} {˜ g(2) (D)} , the error amplification factor is given by: h i h i g(2) (D)} , (12) g(1) (D)} + wH {˜ A = wH {˜

i.e., QLI encoders have the minimum possible value of A of any nonsystematic feedback encoder. (An example of an 8state QLI feedback encoder is given in Table II.) The QLI property for nonsystematic feedforward convolutional encoders was first defined in [13]. Thirty years later, this property was extended to nonsystematic feedback convolutional encoders, resulting in a class of rate R = 1/2 nonsystematic feedback QLI encoders that exhibit good convergence properties when used as constituent encoders in a turbo coding scheme [7], [14]. Not coincidentally, the decoding bit error probability of convolutional codes at low SNRs was also shown to depend on the weight of the encoder inverse in [15]. In contrast to nonsystematic encoders, systematic feedback encoders have a trivial feedforward inverse G−1 (D) = [1 0] with A = 1. (An example of an 8-state systematic feedback encoder is given in Table II.) Since nonsystematic QLI encoders have A = 2, they can be considered as “almost systematic”. Encoders with feedforward inverses G−1 (D) of weight three are called Easy-Look-In (ELI) [16], and encoders whose inverses have small weight, but weight greater than three, are called Nearly-Quick-Look-In (NQLI) [7]. (In Table II, examples of an 8-state ELI feedback encoder and an 8-state NQLI feedback encoder with feedforward inverses of weight 3 and 4, respectively, are given.) On the other hand, catastrophic encoders do not possess a feedforward inverse, and thus their error amplification factor A is infinite. (An example of an 8state catastrophic feedback encoder is also given in Table II.) Mutual information based transfer characteristics and EXIT charts provide a visualization tool to study the convergence behavior of turbo decoding [4]. We use transfer characteristics to explore how nonsystematic feedback convolutional encoders with different weight inverses compare in terms of their ability to provide reliable output L-values for the (not transmitted) information bits. As an example, we consider a set of 8-state rate R = 1/2 constituent encoders, each having a different weight encoder inverse. Table II lists the different encoders considered, along with their respective encoder inverses. In Figure 2(a) the transfer characteristics of each of the nonsystematic encoders in Table II is shown for an SNR of Eb /N0 = 0 dB. For values of Ia near zero, we see that the lower weight encoder inverses lead to larger values of Ie , i.e., for the 8-state encoders considered, the QLI

4

encoder gives the best initial estimates, then the ELI encoder, which has a weight three inverse, followed by the NQLI encoder, with a weight four inverse. This result supports our previous argument that, when the received channel L-values are unreliable, the (implicit) estimates of the L-values for the (not transmitted) information bits will be more reliable if only a few parity bits are involved in the recovery equation (11). In other words, since the encoder inverses of nonsystematic feedback QLI encoders have the fewest number of terms of any nonsystematic feedback encoder, these encoders exhibit the most effective extrinsic information transfer when the a-priori mutual information is near zero. In contrast, for catastrophic encoders the output mutual information Ie is zero when no a-priori mutual information Ia is available. To design nonsystematic turbo codes with good convergence properties, we not only require first encoders that provide good initial estimates for the (not transmitted) information bits, but also a second encoder whose decoder transfer characteristic complements the first. For this purpose, in the next section we use EXIT charts [4] to search for second encoders for nonsystematic turbo codes that complement first encoders with low weight feedforward inverses. From an EXIT chart analysis we can see that, for two encoders that have a tunnel between their transfer characteristics, increasing or decreasing the channel SNR causes the transfer characteristics to grow further apart or to come closer together. The lowest SNR for which a tunnel still exists is the convergence threshold. Using this idea, we can now search for rate R = 1 second encoders which, when combined with rate R = 1/2 first encoders with low weight feedforward inverses, result in a low convergence threshold. (In this case we refer to the second encoder as being a “good match” for the first encoder.) The result of the search is a list of nonsystematic turbo codes with both good convergence and good effective free distance properties. V. A S EARCH FOR G OOD N ONSYSTEMATIC T URBO C ODES To construct rate R = 1/3 turbo codes using nonsystematic feedback convolutional encoders, we concatenate a rate R = 1/2 nonsystematic encoder with a rate R = 1 nonsystematic encoder (see Figure 1). The search for good nonsystematic turbo codes involves finding constituent encoders with large (2) values of dns and low convergence thresholds. We 2 and d2 considered several different rate R = 1/2 QLI, ELI, and NQLI first encoders and applied the EXIT chart analysis to find the rate R = 1 second encoders best matched to each of the first encoders. In Figure 2(b), the transfer characteristics of several rate R = 1 nonsystematic encoders with 2, 4, and 8 states are shown for an SNR of Eb /N0 = 0 dB. We note that, except for the rate h R i= 1 encoder with generator matrix   1 , all other rate R = 1 encoders are G(D) = 13 = D+1 catastrophic, with infinite weight feedforward inverses, and thus they provide zero a-posteriori extrinsic information for zero a-priori information.h Thei 2-state rate R = 1 encoder   1 with G(D) = 13 = D+1 , known as the accumulator (ACC), is a self-QLI encoder [7] (a special case of a QLI

encoder), since the information sequence can be obtained from the parity sequence using the weight-two feedforward inverse D + 1. (Note, however, that the ACC encoder provides relatively poor estimates for large a-priori information.) Based on Figure 2(b) and the transfer characteristics of other rate R = 1 nonsystematic encoders, we applied the EXIT chart analysis to various combinations of first and second encoders in order to find the best match in each case. Since the effective free distance deff is the most important distance parameter for turbo codes with large block lengths, our selection criteria for constituent encoders involved choos(2) ing nonsystematic encoders with large values of dns 2 and d2 , and low values of N2 , the multiplicity of minimum weight codewords with input weight 2. The effective free distance deff , and the average multiplicity Neff , corresponding to codewords of weight deff in the resulting turbo code, were calculated by averaging with respect to all interleavers of a given length, using the uniform interleaver approach [10]. In Table III several good 8-, 16-, and 32-state nonsystematic turbo codes are listed. The entries in the first column refer to the first rate R = 1/2 encoder, classified on the basis of its feedforward encoder inverse. In the second column, some rate R = 1 encoders that are well matched to the encoder listed in the first column are given. In the third column the effective free distance deff of the nonsystematic turbo code is listed along with its average multiplicity Neff for an interleaver length of N = 4096. Finally, the convergence thresholds (Eb /N0 in dB) are given in the fourth column. The optimum deff systematic encoders with the smallest Neff are also listed for comparison. We see that, as expected, nonsystematic turbo codes have larger values of deff than systematic turbo codes and, due to the “almost systematic” property, their convergence thresholds are similar. For example, in Table III(a), the nonsystematic code using an NQLI first encoder with   13 G1 (D) =  17 13turbo and a second encoder with G (D) = 2 15 15 15 achieves an effective free distance of 18, the maximum achievable deff for an 8-state turbo code, and a convergence threshold of 0.01 dB, only 0.05 dB higher than the systematic turbo code. In Table III(b), the nonsystematic turbo code using an NQLI 37 21 (weight 4 inverse) first encoder with G = 31 1 (D) 31  21  and a second encoder with G2 (D) = 31 has deff = 30, compared to deff = 22 for the best systematic turbo code, and its convergence threshold is only 0.12 dB worse than the systematic code. In Table III(c), the nonsystematic   75 turbo 73 code using an ELI first encoder with G (D) = 67 67  451  and a second encoder with G2 (D) = 67 has deff = 54, compared to deff = 38 for the best systematic turbo code, but its convergence threshold is 0.39 dB worse. As another example, the nonsystematic turbo code using  33 an NQLI (weight 73 4 inverse) first encoder with G (D) = 67 67 and a second   1 has d = 52 and a convergence encoder with G2 (D) = 73 eff 67 threshold only 0.1 dB worse than the systematic turbo code. We also list some nonsystematic turbo codes that have better convergence thresholds than the systematic turbo codes and similar values of deff . For example, in Table III(b), the nonsystematic turbo code using (weight 4 inverse)  37 an 21NQLI  first encoder with G (D) = and a second encoder 31 31   1 with G2 (D) = 37 has a convergence threshold of -0.32 dB

5

compared to the systematic turbo code with a convergence threshold of -0.01 dB, and the values of deff and Neff are the same in both cases. To support the results obtained using the EXIT chart convergence analysis and the uniform interleaver distance analysis, we simulated the BER performance of the 8-state nonsystematic turbo code  the best deff (NQLI first encoder with  17 with 13 G (D) = 1 15 15 and a second encoder with G2 (D) =  13  ) and compared its performance with the systematic turbo 15 code with the best deff for interleaver sizes of N = 4096 bits and N = 16384 bits on an AWGN channel. The BER performance is shown in Figure 3, where the simulation results have been extended using the uniform interleaver analysis.2 As expected from the uniform interleaver and EXIT chart analysis, the error floor performance of the nonsystematic turbo code is better than that of the systematic turbo code, but the waterfall performance is slightly worse. The major benefit of nonsystematic turbo codes is their improved error floor performance compared to systematic turbo codes. In other words, for typical interleavers, their advantage in deff results in a waterfall curve that extends to a lower BER before hitting the error floor than for turbo codes with smaller values of deff . This advantage can amount to several orders of magnitude for constituent encoders with more states and for larger block lengths, but accurate simulation results are difficult to obtain for these parameters. VI. D OPED C ATASTROPHIC E NCODERS Catastrophic constituent encoders can produce noncatastrophic turbo codes with maximum effective free distance [2], and hence they are capable of achieving good error floor performance. However, due to the poor convergence behavior of the constituent decoders, iterative decoding is not effective. In particular, a catastrophic encoder gives zero a-posteriori estimates for zero a-priori information. So, to make it work at all in a turbo configuration, it must be matched with an encoder that gives good a-posteriori estimates for zero apriori information. For rate R  1 nonsystematic encoders,  = the ACC encoder (G2 (D) = 31 ) has this property. However, the ACC encoder provides poor a-posteriori estimates for large a-priori information (see Figure 2(b)), and it does not match well with the 8-state catastrophic first encoder (CAT) given in Table II (see Figure 4(a)). On the other hand, we see that  encoder provides a good match with the the G2 (D) = 15 3 catastrophic encoder for most of the EXIT chart (see Figure 4(b)), except at very low values of a-priori information (see Figure 4(c), an expanded version of Figure 4(b) that focuses on low a-priori values). Thus iterative decoding may succeed if something can be done to start the convergence process in the initial iterations. Since it is known that systematic bits provide good initial extrinsic estimates when little apriori information is available, if some nonsystematic bits are 2 All simulations in this paper were run for a maximum of 100 iterations with a ”genie” stopping rule. In other words, decoding was terminated once the correct codeword was found. This procedure removes simulation artifacts associated with fixing a number of iterations and choosing a particular stopping rule. For all four codes simulated in Figure 3, the average number of iterations was less than 10 at the point where the error floor was reached.

replaced by systematic bits, the convergence behavior will improve. This is demonstrated in Figure 4(d), where every 16th parity bit is replaced by an information bit. The ratio of information (systematic) bits to parity (nonsystematic) bits is called the doping ratio, which in this case is 1:16. This is the idea behind code doping that ten Brink [5] originally introduced for serially concatenated codes. We see that, as a result of doping, there is an “opening of the tunnel” in the EXIT chart at low values of Ia . (In [17], the idea of code doping was also used to design multiple turbo codes with low convergence thresholds.) In Table IV we present some 8-, 16-, and 32-state rate R = 1/3 turbo codes using 1:16 doped catastrophic encoders with good effective free distance and convergence properties. We see that, as a result of doping, deff decreases, but the multiplicities of the minimum weight codewords Neff are very small. Also, we see that iterative decoding converges, as expected from Figure 4(d). Finally, we remark that doping the non-systematic turbo codes in Table III has almost no effect on their convergence thresholds, since they are already “almost systematic”, but it may reduce their effective free distances. We support the convergence analysis results with the simulations shown in Figure 5. There, the BER performance  11 17of the 8-state catastrophic first encoder with G (D) = 1 15 15   second encoder is concatenated with the G2 (D) = 15 3 shown. Results are included for two different block lengths, N = 4096 and N = 16384, and a doping ratio of (1:16).3 As predicted by the EXIT charts analysis, the catastrophic encoder by itself doesn’t converge, but doping results in convergence, since the initial a-posteriori estimates of the doped code help in the initial iterations of decoding. Note that the “catastrophic” 8-state turbo code with doping ratio (1:16) has better waterfall performance than the 8-state codes in Figure 3, due to its low convergence threshold, but that its error floor performance is worse, due to its reduced deff . VII. C ONCLUSIONS In this paper, we have introduced the idea of nonsystematic turbo codes and developed design criteria for constructing good constituent encoders. A uniform interleaver analysis was used to show that nonsystematic turbo codes have larger values of effective free distance deff than systematic turbo codes, and hence they achieve better performance in the error floor region of the BER curve. As the constraint length of the constituent encoder is increased, even larger distance gains can be obtained with nonsystematic turbo codes. EXIT charts were used to provide design guidelines for choosing nonsystematic turbo codes with good waterfall performance. The convergence thresholds of nonsystematic turbo codes for which the first encoder has a low-weight feedforward inverse were shown to be comparable to the convergence thresholds of systematic turbo codes. We also showed that turbo codes with catastrophic encoders can be made to converge at low SNRs by using code doping, i.e., by replacing some nonsystematic bits with systematic bits. 3 The two doped codes simulated in Figure 5 required an average of fewer than 20 iterations at the point where the error floor was reached.

6

Nonsystematic turbo codes are an attractive alternative to systematic turbo codes for applications that require large block lengths, low error floors, and performance close to the Shannon capacity. Nonsystematic convolutional encoders can also be used in serial concatenation to obtain low error floors. Finally, specialized interleavers, such as S-random interleavers, can be designed to further lower the error floors of nonsystematic turbo codes. ACKNOWLEDGMENT The authors would like to thank the reviewers for their helpful comments.

Adrish Banerjee Adrish Banerjee was born in Chandigarh, India in 1974. He got his B.Tech(Hons) degree in Electronics and Electrical Communications engineering from Indian Institute of Technology, PLACE Kharagpur, India in 1996. He received his Masters PHOTO and Ph.D. degrees in electrical engineering from HERE University of Notre Dame, Indiana, USA in 1998 and 2003, respectively. He was a post-doctoral researcher at University of Notre Dame, USA for a year. Since June 2004, he is with Indian Institute of Technology, Kanpur, India where he is an Assistant Professor in the department of electrical engineering. His research interests are in digital communications especially error control coding. Adrish Banerjee is a member of IEEE and Eta Kappa Nu.

R EFERENCES [1] C. Berrou, A. Glavieux, and P. Thitimajshima, “Near Shannon limit error correcting coding and decoding: Turbo codes,” in Proceedings of the IEEE International Conference on Communications, Geneva, Switzerland, May 1993, pp. 1064–1070. [2] D. J. Costello, Jr., H. A. Cabral, and O. Y. Takeshita, “Some thoughts on the equivalence of systematic and nonsystematic convolutional encoders,” in Codes, Graphs, and Systems. Kluwer Academic Publishers, 2002, pp. 57–75. [3] D. Divsalar and R. J. McEliece, “On the effective free distance of turbo codes,” in Electronics Letters, Nov. 1995, pp. 99–121. [4] S. ten Brink, “Convergence of iterative decoding,” in Electronic Letters, vol. 35, May 1999, pp. 806–808. [5] ——, “Code doping for triggering iterative decoding convergence,” in Proceedings of the IEEE International Symposium on Information Theory, Washington D.C., June 2001, p. 235. [6] O. Y. Takeshita, O. M. Collins, P. C. Massey, and D. J. Costello, Jr., “A note on asymmetric turbo-codes,” in IEEE Communications Letters, Mar. 1999, pp. 69–71. [7] P. C. Massey and D. J. Costello, Jr., “Turbo codes with recursive nonsystematic quick-look-in constituent codes,” in Proceedings of the IEEE International Symposium on Information Theory, Washington, D.C., June 2001, p. 141. [8] S. Benedetto, D. Divsalar, G. Montorsi, and F. Pollara, “Soft-input softoutput modules for the construction and distributive iterative decoding of code networks,” in European Transactions on Telecommunications, vol. 9, Mar. 1998, pp. 155–172. [9] O. M. Collins, O. Y. Takeshita, and D. J. Costello, Jr., “Iterative decoding of non-systematic turbo codes,” in Proceedings of the IEEE International Symposium on Information Theory, Sorrento, Italy, June 2000, p. 172. [10] S. Benedetto and G. Montorsi, “Unveiling turbo codes: Some results on parallel concatenated coding schemes,” in IEEE Transactions on Information Theory, vol. IT-42, Mar. 1996, pp. 409–428. [11] S. Dolinar and D. Divsalar, “Weight distributions for turbo codes using random and nonrandom permutations”, in TDA Progress Report 42-122, Jet Propulsion Laboratory, Pasadena, California, Aug. 1995 [12] L. C. Perez, J. Seghers, and D. J. Costello, Jr., “A distance spectrum interpretation of turbo codes,” in IEEE Transactions on Information Theory, vol. IT-42, Nov. 1996, pp. 1698–1709. [13] J. L. Massey and D. J. Costello, Jr., “Nonsystematic convolutional codes for sequential decoding in space applications,” in IEEE Transactions on Communication Technology, vol. COM-19, Oct. 1971, pp. 806–813. [14] D. J. Costello, Jr., A. Banerjee, F. Vatta, and B. Scanavino, “On the convergence of nonsystematic turbo codes,” in Proceedings of the Fifteenth International Symposium on Mathematical Theory of Networks and Systems, Aug. 2002. [15] R. Johannesson, J. Massey, and P. Stahl, “Systematic Bits are Better and No Buts About it” in Codes, Graphs, and Systems. Kluwer Academic Publishers, 2002, pp. 77–89. [16] R. Johannesson and E. Paaske, “Further results on binary convolutional codes with an optimum distance profile,” in IEEE Transactions on Information Theory, vol. IT-24, Mar. 1978, pp. 264–268. [17] S. Huettinger and J. Huber, “Design of Multiple-Turbo-Codes with Transfer Characteristics of Component Codes,’ in Proceedings of the Conference on Information Sciences and Systems (CISS 2002), Princeton, NJ, Mar. 2002.

Francesca Vatta Francesca Vatta received a Laurea in Ingegneria Elettronica in 1992 from University of Trieste, Italy. From 1993 to 1994 she has been with Iachello S.p.A., Olivetti group, Milano, Italy, PLACE as system engineer working on design and implePHOTO mentation of Computer Integrated Building (CIB) HERE architectures. Since 1995 she has been with the Department of Electrical Engineering (DEEI) of the University of Trieste where she received her Ph.D. degree in telecommunications, in 1998, with a Ph.D. thesis concerning the study and design of sourcematched channel coding schemes for mobile communications. In November 1999 she became assistant professor at University of Trieste. In 2002, 2003 and 2005 she spent some months as visiting scholar at University of Notre Dame, Notre Dame, IN, U.S.A. She is an author of more than 50 papers published on international journals and conference proceedings. Her current research interests are in the area of channel coding concerning, in particular, the analysis and design of concatenated coding schemes for wireless applications.

Bartolo Scanavino Bartolo Scanavino was born in 1974. He received the degree in Electronics Engineering in 1999, and the Ph.D. degree in Communications Engineering from Politecnico di Torino, in PLACE 2003. He is currently young researcher at Politecnico PHOTO di Torino. HERE From April 2002 to October 2002, he was a visiting researcher at UCSD (University of California), San Diego, and a consultant engineer in Titan, Corporation, San Diego. His research is mainly in wireless communications, especially modulation and coding, including turbo codes and, more recently,radio resource management. Bartolo Scanavino is a member of IEEE.

7

Daniel J. Costello, Jr. Daniel J. Costello, Jr. was born in Seattle, WA, on August 9, 1942. He received the B.S.E.E. degree from Seattle University, Seattle, WA, in 1964, and the M.S. and Ph.D. degrees in PLACE electrical engineering from the University of Notre PHOTO Dame, Notre Dame, IN, in 1966 and 1969, respecHERE tively. In 1969 he joined the faculty of the Illinois Institute of Technology, Chicago, IL, as an Assistant Professor of Electrical Engineering. He was promoted to Associate Professor in 1973, and to Full Professor in 1980. In 1985 he became Professor of Electrical Engineering at the University of Notre Dame, Notre Dame, IN, and from 1989 to 1998 served as Chair of the Department of Electrical Engineering. In 1991, he was selected as one of 100 Seattle University alumni to receive the Centennial Alumni Award in recognition of alumni who have displayed outstanding service to others, exceptional leadership, or uncommon achievement. In 1999, he received a Humboldt Research Prize from the Alexander von Humboldt Foundation in Germany. In 2000, he was named the Leonard Bettex Professor of Electrical Engineering at Notre Dame. Dr. Costello has been a member of IEEE since 1969 and was elected Fellow in 1985. Since 1983, he has been a member of the Information Theory Society Board of Governors, and in 1986 served as President of the BOG. He has also served as Associate Editor for Communication Theory for the IEEE Transactions on Communications, Associate Editor for Coding Techniques for the IEEE Transactions on Information Theory, and Co-Chair of the IEEE International Symposia on Information Theory in Kobe, Japan (1988), Ulm, Germany (1997), and Chicago, IL (2004). In 2000, he was selected by the IEEE Information Theory Society as a recipient of a Third-Millennium Medal. Dr. Costello’s research interests are in the area of digital communications, with special emphasis on error control coding and coded modulation. He has numerous technical publications in his field, and in 1983 co-authored a textbook entitled ”Error Control Coding: Fundamentals and Applications”, the 2nd edition of which was published in 2004.

8

TABLE I MAXIMUM EFFECTIVE FREE DISTANCES OF RATE R = 1/3 SYMMETRIC TURBO CODES ν 2 3 4 5 6

dseff 10 14 22 38 70

dns eff 12 18 30 54 102

TABLE II INVERSES OF SEVERAL 8-STATE, RATE R = 1/2 CONVOLUTIONAL ENCODERS

Systematic (SYS): Quick-Look-In (QLI): Easy-Look-In (ELI): Nearly-Quick-Look-In (NQLI): Catastrophic (CAT):

h

Encoder i

D 3 +D+1 D 3 +D 2 +1

  Inverse 1 = 0   1 −1 G (D) = 1   D+1 −1 G (D) = D   D2 −1 G (D) = 2 # +1 " D +D

G(D) = 1 i h 2 D 3 +D+1 +D G(D) = D3D+D 2 +1 D 3 +D 2 +1 i h 3 D 3 +D+1 G(D) = D3D+D+1 2 +1 D 3 +D 2 +1 h 3 i 2 +D+1 D 3 +D+1 G(D) = DD+D 3 +D 2 +1 D 3 +D 2 +1

G−1 (D)

G(D) =

G−1 (D) =

h

D 3 +1 D 3 +D 2 +1

D 3 +D 2 +D+1 D 3 +D 2 +1

i

1 D+1 D D+1

TABLE III EFFECTIVE FREE DISTANCE AND CONVERGENCE THRESHOLDS FOR SEVERAL RATE R = 1/3 TURBO CODES (N =4096) I constituent Systematic:  G1 (D) = 1 13/15 QLI:   G1 (D) = 6/15 13/15 ELI:   G1 (D) = 11/15 13/15 NQLI (Weight 4 inv.):   G1 (D) = 17/15 13/15

II constituent G2 (D) = [13/15] G2 (D) = [13/15] G2 (D) = [15/3] G2 (D) = [13/15] G2 (D) = [15/3] G2 (D) = [13/15] G2 (D) = [7/3]

deff , Neff 14, 16, 13, 18, 15, 18, 14,

1.99 1.99 3.99 1.99 3.99 1.99 2.00

Conv. Thr. (Eb /N0 ) -0.06 dB 0.01 dB -0.04 dB 0.01 dB -0.30 dB -0.01 dB -0.20 dB

(a) 8 STATE

I constituent Systematic:  G1 (D) = 1 37/23 QLI:   G1 (D) = 23/31 12/31 ELI:   G1 (D) = 23/31 21/31 NQLI (Weight 4 inv.):   G1 (D) = 37/31 21/31

II constituent G2 (D) = [37/23] G2 (D) = [23/31] G2 (D) = [12/31] G2 (D) = [21/31] G2 (D) = [15/3] G2 (D) = [21/31] G2 (D) = [7/3]

(b) 16 STATE

deff , Neff 22, 28, 26, 30, 23, 30, 22,

1.99 1.99 1.99 1.99 3.98 1.99 1.99

Conv. Thr. (Eb /N0 ) -0.01 dB 0.17 dB 0.06 dB 0.14 dB -0.29 dB 0.11 dB -0.32 dB

9

I constituent Systematic:  G1 (D) = 1 75/57 QLI:  G1 (D) = 31/67 56/67 ELI:  G1 (D) = 75/67 73/67 NQLI (Weight 4 inv.):  G1 (D) = 33/67 73/67 NQLI (Weight 6 inv.):  G1 (D) = 53/67 73/67

II constituent

   

G2 (D) = [75/57] G2 (D) = [31/67] G2 (D) = [13/15] G2 (D) = [45/67] G2 (D) = [7/3] G2 (D) = [73/67] G2 (D) = [33/67] G2 (D) = [45/67] G2 (D) = [5/7]

deff , Nf,eff 38, 48, 38, 54, 38, 52, 50, 54, 40,

1.97 1.97 1.98 1.97 1.98 1.97 1.97 1.97 1.98

Conv. Thr. (Eb /N0 ) 0.09 dB 0.16 dB 0.03 dB 0.48 dB -0.31 dB 0.19 dB 0.10 dB 0.51 dB 0.02 dB

(c) 32 STATE

TABLE IV SEVERAL RATE R = 1/3 TURBO CODES USING 1:16 DOPED AND UNDOPED CATASTROPHIC ENCODERS I constituent Catastrophic:  8-state  G1 (D) = 11/15 17/15 Catastrophic: 8-state Doping ratio 1:16 Catastrophic:  16-state  G1 (D) = 27/31 35/31 Catastrophic: 16-state Doping ratio 1:16 Catastrophic:  32-state  G1 (D) = 55/67 41/67 Catastrophic: 32-state Doping ratio 1:16

u(D)

deff , Neff 18, 1.99 15, 3.99 16, 2.83E-02 13, 5.66E-02 30, 1.99 22, 1.99 28, 1.42E-01 20, 1.42E-01 54, 1.97 38, 1.98 51, 5.77E-02 35, 5.81E-02

Conv. Thr. (Eb /N0 ) — — 0.01 dB -0.32 dB — — 0.34 dB -0.30 dB — — 0.55 dB -0.30 dB

First Encoder (1) g1 (D) g (0)(D) 1

v (D) 0

g(2)(D) 1 g(0)(D) 1

v1 (D)




π

Fig. 1.

II constituent G2 (D) = [17/15] G2 (D) = [15/3] G2 (D) = [17/15] G2 (D) = [15/3] G2 (D) = [27/31] G2 (D) = [7/3] G2 (D) = [27/31] G2 (D) = [7/3] G2 (D) = [55/67] G2 (D) = [7/3] G2 (D) = [55/67] G2 (D) = [7/3]

g(1)(D) 2 g (0)(D) 2 Second Encoder

Rate R = 1/3 Nonsystematic Turbo Code: Encoder Structure.

v2 (D)

10

1 0.9 E /N = 0 dB b

0.8

0

Output Mutual Information, I

e

0.7 0.6 0.5 QLI ELI NQLI CAT Ia=Ie

0.4 0.3 0.2 0.1 0

0

0.1

0.2

0.3

0.4 0.5 0.6 Input Mutual Information, I

0.7

0.8

0.9

1

0.7

0.8

0.9

1

a

(a) Rate R = 1/2

1 0.9 Eb/N0 = 0 dB 0.8 [13/15] [15/3] [1/3] [6/15] [5/7] Ia=Ie

Output Mutual Information, I

e

0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

0

0.1

0.2

0.3

0.4 0.5 0.6 Input Mutual Information, I

a

(b) Rate R = 1 Fig. 2.

Transfer Characteristics for Different Nonsystematic Constituent Encoders Used in a Rate R = 1/3 Turbo Code.

11

0

10

[1 13/15]−[13/15], N=4096, Simulations [1 13/15]−[13/15], N=4096, Bounds [1 13/15]−[13/15], N=16384, Simulations [1 13/15]−[13/15], N=16384, Bounds [17/15 13/15]−[13/15], N=4096, Simulations [17/15 13/15]−[13/15], N=4096, Bounds [17/15 13/15]−[13/15], N=16384, Simulations [17/15 13/15]−[13/15], N=16384, Bounds

−1

10

−2

10

−3

10

−4

BER

10

−5

10

−6

10

−7

10

−8

10

−9

10

−10

10

Fig. 3.

0

0.5

1

1.5

Eb/N0 (dB)

2

2.5

BER Performance for Rate R = 1/3 Turbo Codes: Union Bounds and Simulations, AWGN channel.

3

3.5

12

1

1 Eb/N0 = 0 dB

0.8

0.6 I1e/I2a

I1e/I2a

0.6 0.4

[11/15 17/15] [1/3]

0.2 0

Eb/N0 = 0 dB

0.8

0.4

[11/15 17/15] [15/3]

0.2

0

0.2

0.4

0.6

0.8

0

1

0

0.2

0.4

I1a/I2e (a) Eb/N0 = 0 dB

Eb/N0= 0 dB

0.4

0.3

Doping ratio 1:16

0.3 I1e/I2a

I1e/I2a

1

0.5

0.4

0.2 [11/15 17/15] [15/3]

0.1

Fig. 4.

0.8

(b)

0.5

0

0.6 I1a/I2e

0

0.05

0.1

0.15

0.2 [11/15 17/15] [15/3]

0.1

0.2

0

0

0.05

0.1

I1a/I2e

I1a/I2e

(c)

(d)

Convergence Behavior of Several Rate R = 1/3 Turbo Codes Using a Catastrophic First Encoder.

0.15

0.2

13

0

10

−1

10

−2

10

[11/15 17/15]−[15/3], N=4096 [11/15 17/15]−[15/3], N=16384 Doped (1:16), [11/15 17/15]−[15/3], N=4096 Doped (1:16), [11/15 17/15]−[15/3], N=16384

−3

BER

10

−4

10

−5

10

−6

10

−7

10

−8

10

Fig. 5.

0

0.5

Eb/N0 (dB)

1

BER Performance of Rate R = 1/3 Turbo Codes Using a Catastrophic/Doped Encoder, AWGN channel.

1.5

2