IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 50, NO. 5, MAY 2004
with the MacWilliams–Kløve identities, we are left with about 100 unknowns. There is a chance that this system may be solved completely by extending the techniques presented here, and combining it with all the techniques found in the literature. That will be extensive labour in itself, so we leave it to future work. ACKNOWLEDGMENT This work was inspired by Steven Dougherty’s talk at WCC in Paris, France, in January 2001. Private correspondence with Aaron Gulliver, coauthor of the mentioned talk, has been very valuable during the research. Finally, I wish to thank Torleiv Kløve for his continuous advice as a supervisor, concerning presentation and publication, and also for his help in finding and understanding [5]. REFERENCES [1] S. Dodunekov and J. Simonis, “Codes and projective multisets,” Electron. J. Combin., vol. 5, no. 1, 1998. [2] S. Dougherty, A. Gulliver, and M. Oura, “Higher weights and graded rings for binary self-dual codes,” in Discr. Appl. Math. (Special issue for WCC 2001), vol. 128, 2003, pp. 251–261. [3] T. Helleseth, T. Kløve, and J. Mykkeltveit, “The weight distribution of 1)=n),” Discr. irreducible cyclic codes with block lengths n1 ((q l Math., vol. 18, pp. 179–211, 1977. [4] T. Helleseth, T. Kløve, and Ø. Ytrehus, “Generalized Hamming weights of linear codes,” IEEE Trans. Inform. Theory, vol. 38, pp. 1133–1140, May 1992. [5] T. Kløve, “The weight distribution of linear codes over GF (q l )having generator matrix over GF (q ),” Discr. Math., vol. 23, pp. 159–168, 1978. , “Support weight distribution of linear codes,” Discr. Math., vol. [6] 106/107, pp. 311–316, 1992. [7] O. Milenkovic, “On the generalized Hamming weight enumerators and coset weight distributions of even isodual codes,” in Proc. IEEE Int. Symp. Information Theory, Washington, DC, June 2001, p. 62. [8] H. G. Schaathun, “Duality and greedy weights for linear codes and projective multisets,” in Applied Algebra, Algebraic Algorithms and ErrorCorrecting Codes (Lecture Notes in Computer Science). Berlin, Germany: Springer-Verlag, 2001, vol. 2227, pp. 92–101. [9] J. Simonis, “The effective length of subcodes,” Appl. Algebra Eng. Commun. Comput., vol. 5, no. 6, pp. 371–377, 1994.
0
867
Design and Decoding of Optimal High-Rate Convolutional Codes Alexandre Graell i Amat, Student Member, IEEE, Guido Montorsi, Member, IEEE, and Sergio Benedetto, Fellow, IEEE Abstract—This correspondence deals with the design and decoding 1) of high-rate convolutional codes. After proving that every ( convolutional code can be reduced to a structure that concatenates a block encoder associated to the parallel edges with a convolutional encoder defining the trellis section, the results of an exhaustive search for the 1) convolutional codes is presented through various optimal ( tables of best high-rate codes. The search is also extended to find the “best” recursive systematic convolutional encoders to be used as component encoders of parallel concatenated “turbo” codes. A decoding algorithm working on the dual code is introduced (in both multiplicative and additive form), by showing that changing in a proper way the representation of the soft information passed between constituent decoders in the iterative decoding process, the soft-inpu soft-output (SISO) modules of the decoder based on the dual code become equal to those used for the original code. A new technique to terminate the code trellis that significantly reduces the rate loss induced by the addition of terminating bits is described. Finally, an inverse puncturing technique applied to the highest rate “mother” code to yield a sequence of almost optimal codes with decreasing rates is proposed. Simulation results applied to the case of parallel concatenated codes show the significant advantages of the newly found codes in terms of performance and decoding complexity. Index Terms—Block codes, convolutional codes, dual codes, high-rate codes, inverse puncturing, iterative decoding, puncturing, trellis termination, turbo-like codes.
I. INTRODUCTION As the need for increasingly high data rate communications intensifies, the resources, like bandwidth and energy, become scarce and precious. For instance, magnetic recording and fiber-optic applications require both very high data rates (from one to several tens of gigabits per second) and very low code redundancies, thus, calling for high coding gains and very high code rates simultaneously. Traditionally, algebraic block codes have been preferred for very high coding rates because of the better performance/complexity comparison with respect to convolutional codes. Indeed, to keep the decoding complexity reasonably low for high-rate convolutional codes, one needs to resort to punctured codes [1]–[3], which become rather weak in terms of distance spectrum (or just free distance) for the heavy puncturing required to get very high rates. On the other hand, punctured convolutional codes yield the advantage of flexibility, i.e., they offer a wide range of code rates without modifying the co-decoding algorithm, which remains essentially the same needed to decode the rate-1=2 mother code. With the advent of concatenated codes with interleavers (or turbo-like codes), hard-in hard-out (like those used for algebraic block codes) and soft-in hard-out (as the Viterbi algorithm) decoding algorithms must be replaced by soft-input soft-output (SISO in the following) symbol decoding algorithms to be embedded into the
Manuscript received December 10, 2002; revised November 20, 2003. This work was supported in part by ST Microelectronics and Qualcomm Inc. The material in this correspondence was presented in part at the IEEE Information Theory Workshop, Bangalore, India, October 2002, and at the IEEE Global Communications Conference, San Francisco, CA, December 2003. The authors are with the Politecnico di Torino, 10129 Torino, Italy (e-mail:
[email protected];
[email protected];
[email protected]). Communicated by S. Litsyn, Associate Editor for Coding Theory. Digital Object Identifier 10.1109/TIT.2004.826669 0018-9448/04$20.00 © 2004 IEEE
868
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 50, NO. 5, MAY 2004
iterative decoding strategies adopted for those codes. This has increased the interest for high-rate convolutional codes, since the trellis regularity of convolutional codes (as compared with the time-varying, highly complex trellis structure of block codes) makes a posteriori probability (APP) SISO algorithms more attractive. Still, the trellis complexity of high-rate (n; n 0 1) convolutional codes prevents from applying the SISO algorithms directly to the original trellis. In those cases, a drastic simplification stems from the use of algorithms working on the trellis of the dual code [4]–[7], or syndrome-decoding algorithms [8]. Another advantage of punctured convolutional codes lies in the easiness in searching for optimum puncturing patterns, with respect to the high complexity of exhaustive searches for optimum high-rate convolutional codes. In this correspondence, we address several aspects related to high-rate convolutional codes. • The search for “optimum” codes (optimum here refers to the output weight enumerating function [9]) and encoders (in this case, we refer to the input–output weight enumerating function), based on a particular structure of the encoder. The former can be used as a stand-alone code, or as outer encoder in serially concatenated code structures, whereas the latter are recommended as constituent encoders for parallel concatenated convolutional codes (PCCCs) or as inner code for serially concatenated convolutional codes (SCCCs). Independent work on high-rate convolutional codes has recently appeared in [10], [11], [26].
Fig. 1. The structure of the encoder.
moderate rate convolutional encoder that defines the dynamical part of the trellis. The encoder works as follows. Consider an (n; n01) code C with the structure of Fig. 1. Let us express the information and code sequences as
u = [u0 ; u1 ; u2 ; . . .] =
(0)
(1)
u0 u0
1 1 1 u k0 ( 0
1)
(0)
(1)
1 1 1 u k0
(0)
(1)
1 1 1 x n0
; u1 u1
( 1
1)
;...
(1)
and
x = [x 0 ; x1 ; x2 ; . . .] =
(0)
(1)
x0 x0
1 1 1 x n0 ( 0
1)
; x1 x1
( 1
1)
;...
(2)
where (0) (1) (k01) ui = ui ui 1 1 1 ui
• The decoding algorithm based on the dual code trellis, and its additive form.
is the input vector containing k bits and
• An inverse puncturing technique that adds to the newly found codes the same versatility of punctured convolutional codes, yielding a wide range of code rates starting from the same mother code with very little impact on performance.
is the output vector of n bits. Let us also define the integer number l ranging from 2 to lmax as a parameter in the construction technique.1 The first n 0 l information bits of the vector u i
• A new trellis termination method for high-rate convolutional codes that induces a significantly lower decrease in the code rate, by employing the minimum number of bits for the trellis termination. In the final part of the correspondence, simulation results quantify the advantages of the newly found codes/encoders in a key application. The correspondence is organized as follows. In Section II, we introduce the new structure of a general, high-rate convolutional encoder and prove that every code can be reduced to this structure. The search algorithm based on this structure is discussed in Section III. Tables of the best high-rate convolutional codes and encoders are also reported. In Section IV, we propose a new trellis termination technique and in Section VI, we discuss punctured codes. The decoding algorithm working on the dual code is discussed in Section VI and the proposed inverse puncturing method is treated in Section VII. Finally, several simulation results are reported in Section VIII. Section IX is a concluding summary.
II. THE ENCODER STRUCTURE AND ITS PROPERTIES An (n; k; ) convolutional encoder can be described by a trellis diagram formed by repeated copies of a trellis section, consisting of a set of 2 initial states and 2 final states. Each initial state is connected to k 2 final states by oriented arrows, called edges. In a trellis section there can be multiple (parallel) edges connecting two states. To search for good high-rate (n; n 0 1) convolutional codes, we consider the encoder structure shown in Fig. 1. The basic idea is to build an overall convolutional encoder composed of two constituent encoders: a block encoder associated to the parallel edges, and a low-to-
(0) (1) (n01) x i = xi xi 1 1 1 xi
u0i = ui(0) ui(1) 1 1 1 ui(n0l01)
are encoded by a rate nn0l block encoder, generating a codeword xBC of n bits. At the same time, the last l 0 1 information bits of u i (n0l) (n0l+1) (n02) ui00 = ui ui . . . ui
are encoded by a rate l0l 1 convolutional encoder, generating a codeword xC C of length l. Finally, n 0 l leading zeros are added to xC C and the codeword x is obtained adding (modulo 2) bit-by-bit the two previous codewords xBC and modified xC C . The sequences u and x can also be written as
u (D) = and
x (D ) =
1
i=0
1 i=0
ui Di
2 F k (D )
xi Di
2 F n (D )
where F (D) is the field of Laurent series in D over F and F (D) denotes the field of Laurent series with coefficients from F . Hence, the relationship between u (D) and x(D) can be expressed in a compact form through the polynomial generator matrix G(D) as
x (D) = u (D)G(D):
(3)
With the proposed construction technique, G(D) assumes the following structure: GB C G (D ) = : (4) X G C C (D ) 1As we will see in Theorem I, the parameter defines the rates of the constituent block encoder and constituent convolutional encoder.
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 50, NO. 5, MAY 2004
GBC being the (n 0 l) 2 n generator matrix of the constituent block encoder, X an (l01)2(n0l) all-zero matrix, and GC C (D) identifying the (l 0 1) 2 l generator matrix of the constituent convolutional encoder. The search algorithm described in the following permits to obtain the best high-rate convolutional codes in the form of Fig. 1. For construction, such codes are nonsystematic. On the other hand, Viterbi or maximum a posteriori (MAP) decoding of high-rate convolutional codes is a complex task, which can be highly simplified by applying the MAP algorithm to the trellis of the dual code [4]–[7]. This requires, however, an encoder in systematic form [6]. An equivalent systematic encoder is directly obtained from the generator matrix G (D) of the nonsystematic encoder by permuting and linearly combining matrix rows. Thus, the equivalent systematic code generator matrix can be expressed as the product of G(D) with the inverse matrix of the left (n 0 1) 2 (n 0 1) submatrix A (n011) (D) of G (D ) 1 G sys (D) = A0 1) (D ) 1 G (D ): (n01
(5)
The systematic generator matrix of an encoder with rate (n 0 1)=n can be written as
Gsys (D ) = [ I (n011)
v (D ) ]
(6)
where I n01 is a (n 0 1) 2 (n 0 1) identity matrix and v (D) is a column vector of k elements whose entries may be, in general, rational functions with the same denominator, i.e., the encoder may turn out to be recursive
v (D) = 1=q (D) 1 [v0 (D)v1 (D) 1 1 1 vk01 (D)]T For simplicity, to describe the systematic encoder we will use a single vector z (D)
z (D) = z 0 (D)zq (D) = [v0 (D)v1 (D) 1 1 1 vk01 (D)q (D)]:
The z 0 elements describe the numerator polynomials of v , and zq describes the common denominator polynomial of all elements of v . All these concepts are better clarified by an example. Example 1: Consider the following rate-6=7, eight states, recursive convolutional code described by the nonsystematic polynomial generator matrix (see (4)):
G (D ) =
1
0
0
1
1
1
1
0
1
0
1
1
1
1
0
0
1
1
1
0
0
0
0
1
0
0
0
0
0
0
1
0
0
0
0
0
0
1
1
D+D 1+D +D D+D +D 1+D +D 1+D +D +D 1+D +D
where the upper 3 2 7 submatrix describes the constituent block encoder and the 3 2 4 lower right submatrix defines the constituent recursive systematic convolutional encoder. The equivalent systematic encoder Gsys (D) is easily obtained by multiplying G(D) by the inverse matrix of the left (n 0 1) 2 (n 0 1) submatrix of G(D), An01 (D) 1 G sys (D) = G (D) 1 A 0 n01 (D) = [I (n011) v (D)]
=
1
0
0
0
0
0
0
1
0
0
0
0
0
0
1
0
0
0
0
0
0
1
0
0
0
0
0
0
1
0
0
0
0
0
0
1
D+D 1+D +D 1+D +D 1+D +D 1+D 1+D +D D+D 1+D +D D+D +D 1+D +D 1+D +D +D 1+D +D
869
Fig. 2.
(7; 6) convolutional encoder of Example 1.
Finally, the systematic encoder is described by the single vector z = where the first six entries are the numerator polynomials of the column vector v (D) and the last entry describes the common denominator polynomial of all elements of v (D), i.e, the recursion. Each polynomial in vector z is given in octal form with the lowest degree term on the right. As an example, 15 means 2 3 15 = (1101) = 1 + D + D . The resulting encoder is shown in 5 Fig. 2. [6; 13; 11; 12; 16; 17; 15],
A. Optimal High-Rate Convolutional Codes In a previous work [12], we implemented a search algorithm to find good high-rate (n; n 0 1) convolutional codes with the structure of Fig. 1, starting from suitable known constituent block encoders; as a consequence, the search was not exhaustive. Moreover, the constrained structure of Fig. 1 may seem to be a second cause of search noncompleteness. The following Theorem 1 proves instead that every (n; n01) convolutional code can be reduced to the form of Fig. 1. According to this result, and suitably modifying the previous algorithm, we have been able to obtain the best codes of rate (n 0 1)=n with an affordable computational complexity. Lemma 1: For every linear (n; n 0 1) decodable convolutional code with parallel edges connecting two trellis states there exists an (n; n0l) block code Cp that defines the 2n0l parallel edges joining the zero states. Proof: To prove Lemma 1, we must prove that the set of codewords that label parallel edges between the zero state is closed under component-wise modulo-2 addition. Let us denote this set as Cp . Express the ending state sf of the convolutional encoder and the code bits associated to a single transition x , as a function of the input bits and the initial state
s f = uA 8 s i B
x = uC 8 s i D
(7)
where u is the input vector, si the initial state, A is the (n 0 1 2 ) control matrix, B denote the ( 2 ) state matrix, C is the (n 0 1 2 n) transition matrix, and D is the ( 2 n) observation matrix. Letting si = sf = 0, equations in (7) become
uA = 0 x = uC :
(8)
Thus, the set Cp is defined as
:
uC; 8u : uA = 0g: Cp fx : x = uC
(9)
Now, by using the following equivalence between sets:
Up fu : uA = 0g fu : u = u0Q; 8u0 g
(10)
870
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 50, NO. 5, MAY 2004
where Q is an (n 0 l) 2 (n 0 1) matrix such that QA = 0, definition (9) can be rewritten as
C
fx : x = u QC 8u g: 0
p
0
(11)
Equation (11) is just the definition of a block code of parameters (n; n 0 l), and the lemma is proved. Note that parameter l simply follows from the dimensionality d of the null space of A, i.e., l = n 0 d.2 Also, note that for a decodable code QC must have full rank, i.e., rang n0l . (QC ) = n 0 l , so jCp j = 2 Theorem 1: Every linear (n; n 0 1) convolutional code with parallel edges connecting two trellis states can be expressed as the concatenation (in the form of Fig. 1) of an (n; n 0 l) block code defining the 2n0l parallel edges and an (l; l 0 1) convolutional code that determines its dynamical structure. Proof: Denote the set of parallel edges between two states si and sf as Ci;f . Recalling (7) and (10), and observing that the set
fu : uA = s B 8 s g f
is a coset of Up defined in (10)
fu : uA = s B 8 s g fu : u = u0Q 8 u00K ; 8u0 g (12) where u 00 K is the coset leader and K a suitable (l 0 1) 2 (n 0 1) 00 i
f
matrix. Note that the number of u is equal to the number of cosets of Up , i.e., 2n01 =2n0l = 2l01 . Then, Ci;f can be written as
C
fx : x = uC 8 s D ; 8u : uA = s B 8 s g 0 00 0 = x : x = u Q 8 u K C 8 s D ; 8u 00 0 0 = x : x = u QC 8 u K C 8 s D ; 8u 0 = x : x = u QC 8 x = C 8x 00 = u KC 8 s D. i
i;f
i
f
i
i
i;f
i;f
p
(13)
where x i;f i Therefore, each set of parallel edges between any two states Ci;f is the coset x i;f of Cp . Also, we can obtain the following results, replacing the right-hand side (RHS) of (12) into the first equation of (7):
sf = u 0 Q 8 u00 K A 8 si B = u 00 K A 8 si B
x i;f = u 00 K C 8 si D :
(14)
The two expressions in (14) define a convolutional code of parameters (n; l 0 1). So, it is clear that the dynamical structure of the overall (n; n 0 1) code is completely defined by this simpler convolutional code that generates all coset leaders x i;f . Then, codewords associated to parallel edges between two states i, j are obtained adding x i;j to all codewords in Cp . The generator matrix of the overall code can thus be written as
G 0 (D ) =
a11 .. .
a12 .. .
...
.. .
a1n .. .
a(n0l)1
a(n0l)1
...
a(n0l)n
g11 (D) g12 (D) ... .. .. .. . . . g(l01)1 (D) g(l01)2 (D) . . .
g1n (D) .. . g(l01)n (D)
: (15)
We still have to prove that the dynamical structure is completely defined by a simpler (l; l 0 1) convolutional code. Consider (15), and 2The
null space of min( + 1
has at least dimension ).
(
G (D ) I =
X
Now, we proceed to state and prove Theorem 1.
i
note that without loss of generality, G0 (D) can always be reduced to the following form:
1), such that
a ~1(n0l+1) .. .
...
.. .
a ~1(n01) .. .
a ~(n0l)(n0l+1) . . . a ~(n0l)(n01) 1
...
0
.. .
..
.
.. .
0
...
1
a ~1n .. . a ~(n0l)n g~1n (D) .. . g~(l01)n (D)
(16) where the block code is in systematic form and the left l 2 (n 0 l) submatrix of the constituent convolutional code is reduced to an all-zero matrix X . The matrix G (D) in (16) has exactly the same structure as (4). Therefore, the dynamical part is completely defined by an (l; l 0 1) convolutional code (the lower right matrix in (16)). So, finally the (n; n 0 1) convolutional code can be viewed as the concatenation of an (n; n 0 l) block code and a simpler (l; l 0 1) convolutional code, and Theorem 1 is proved. III. CODE SEARCH In this section, exploiting the code decomposition outlined in Section II, we describe a search for the optimal (n; n 0 1) convolutional codes for various parameters n and . The codes are optimized in terms of the output weight enumerating function T (D) [9]. Such a criterion is suitable for stand-alone convolutional codes and the outer code in serially concatenated codes, whose performance are directly related to the distance properties of the code. In particular, we optimize the first ten pairs fd; ad g of the distance spectrum, i.e., for each pair, distance d is maximized and then its multiplicity ad is minimized. The actions accomplished in the search algorithm are summarized as follows. 1) Fixed n and the constraint length search all rate (n 0 l)=n block codes with dmin greater than or equal to the maximum free distance achievable for an (n; n 0 1; ) code, dof . 2) For each block code Cp satisfying 1), incorporate distance spectrum information of Cp to all (previously) individuated (l; l 0 1) convolutional codes that satisfy dcf dfo , and choose the one optimizing pair fd; ad g.
Note that steps 1) and 2) must be repeated for each l (l min( + 0 l) block codes and 0 and dfc , respectively. This procedure allows us to significantly reduce the search complexity, since it is somewhat split into two simpler searches, that is, those of the constituent block and convolutional codes.3 As an example, the direct search of the best (13; 12; 4) convolutional code based on matrix G sys = [I n01 v (D)] requires to test 25113 = 265 different codes. According to our code decomposition, we should test all combinations of constituent block code/constituent convolutional code resulting from a suitable parameter l. Only for l = 4 and l = 5 there exists a block code with minimum distance 3. For l = 4 there are 1562275 block codes with dmin 3 and 225 possible constituent convolutional codes. For l = 4, a total of 55 block codes and 220 convolutional codes must be tested. The overall complexity is, therefore, less than 246 which represents a factor 219 of improvement with respect 1; n), see Lemma 1) that allows to obtain (n; n (l; l 1) convolutional codes with suitable dmin
3The search on the block code is efficiently accomplished with a search algorithm working on the parity-check matrix For a certain , we must 1) or fewer columns of is linearly simply impose that every set of ( independent.
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 50, NO. 5, MAY 2004
BEST RATE (n
871
TABLE I
0 1)=n RECURSIVE SYSTEMATIC CONVOLUTIONAL CODES AND RELATED WEIGHT SPECTRA
to the direct search based on matrix G sys . Of course, equivalent codes may be avoided, thus simplifying the search. Moreover, the constituent codes that are candidates for a certain rate are, in general, candidates for higher rates, so they can be stored and reused when searching for higher rate codes. An alternative technique to simplify the search for recursive convolutional codes based on combinatorial arguments has been proposed in [11]. Our code decomposition, however, is also applicable to find the best encoders, significantly reducing the search complexity. Moreover, with our technique, also the computation of the distance spectrum is significantly simplified; instead of performing the computation acting on the overall code trellis (characterized with a high number of transitions), we perform it to the simpler constituent convolutional code, with properly added information of the distance spectrum of each coset Ci;f . The optimal recursive convolutional codes obtained from the exhaustive search are reported in Table I. Each code is defined by the vector z (D) (see Section II) in octal form with the lowest degree term on the right. Also, we report in the table the memory , the construction parameter l, the free distance df , and the first six multiplicities of the distance spectrum, ad ; . . . ; ad . As already mentioned, a search for the best (n; n 0 1) recursive codes has been presented in [11] and [26]. Most of the codes in Table I can also be found in [11]. For completeness, a more stringent search to find feedforward convolutional codes that are suitable for decoding by means of the Dual-SISO (DSISO) algorithm, i.e., systematic feedforward codes, has also been performed. The codes are listed in Table II. Notice that
considering only feedforward systematic encoders leads in general to slightly worse codes, in terms of distance spectrum, with respect to those reported in Table I. A. Optimal Encoders Optimizing T (D) may not be sufficient for some specific applications, as it is the case of parallel turbo codes. Indeed, a binary convolutional code admits many encoders, which differ in terms of the input–output weight enumerating function T (W; D) [9] and, consequently, yield different bit error probability performance. Although these differences may be insignificant for the single code, they become crucial when the code is embedded into a turbo-like structure. In fact, it has been proved that the bit error probability of a PCCC depends, in general, on the minimum-weight code sequences of the constituent codes generated by input sequences of low weight [21], whereas for SCCCs this remark concerns the inner encoder. Among the weights of those sequences, the most important one is the effective free distance df ;e , defined in [21] as the minimum weight of code sequences generated by input sequences of weight 2. The optimization criteria for both constituent codes of parallel concatenated codes (and for the inner code of a serial concatenation) are thus quite different from those used for stand-alone convolutional codes. In this subsection, we extend the search to find the best high-rate convolutional encoders to be used in the construction of turbo-like codes. The optimization criterion is based on the optimization of the pairs
872
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 50, NO. 5, MAY 2004
BEST RATE (n
TABLE II
0 1)=n FEEDFORWARD SYSTEMATIC CONVOLUTIONAL CODES AND RELATED WEIGHT SPECTRA
(di ; Ni ) for i = 2 to i = 6, where di is the minimum weight of code sequences generated by input sequences of weight i (notice that d2 = df ;e ), and Ni are their multiplicities. The search criterion looks for recursive encoders4 and sequentially optimizes the pairs (di ; Ni ) for i = 2; . . . ; 6 as it was done in [24] for rates up to 4=5. Tables III and IV report the best encoders using the aforementioned criterion. Table III describes the input–output weight enumerating functions of the best codes. For each code it reports the rate (n 0 1)=n, the memory , and the first six pairs di , Ni . The free distances are made evident as a boldface number. Table IV reports the encoder description for the codes of Table III and the first six terms of their distance spectrum, ad , ad ; . . . ; ad .
IV. ZERO TERMINATION OF THE TRELLIS The standard solution to terminate a convolutional code is to force the encoder back to the all-zero state by appending a block of 1 k tail bits to the N k information bits (zero-termination). The zero termination leads to a block code of rate Nk k (17) < (N + )n n and thus entails a loss due to the 1 n overhead coded bits. Whereas such a loss is limitated for low-to-moderate rates, it may be significant for very-high-rate codes and short block sizes. To avoid this drawback one can resort to tail-biting [14], a solution that keeps the code rate constant with a slight increase in decoding complexity. Indeed, it is 4For feedforward convolutional codes (CC), information sequences with weight = 1 generate error events of finite weight, while, for recursive CC, = 2, as proved in [23]. error events start with
common practice to perform tail-biting termination when dealing with very-high-rate convolutional codes. In this section, we introduce a new method to perform the zero trellis termination that exploits the code structure outlined in Section II to significantly reduce the overhead. Instead of performing some additional steps in the trellis to drive the encoder back to the all-zero state, the technique uses some of the bits in the last trellis sections as tail-biting bits to perform the termination. As pointed out previously, good high-rate convolutional codes with not too large a memory are in general obtained (see also Tables I and II) with a full connectivity of the trellis, i.e., l 0! lmax . Thus, we consider first a fully connected trellis section, and then extend our considerations to every (n 0 1)=n code. Let us consider a trellis diagram representation of the code having N trellis sections. In the following, we consider the trellis section index ranging from 0 to N 0 1. Let us also define the information sequence u = [u 0 ; u1 ; . . . ; uN 01 ] and the sequence of N + 1 states, s = [s 0 ; s1 ; . . . ; sN ], with fsi gi2=1 . For encoders with a full connectivity in the trellis it is easy to fulfill the zero termination condition sN = s0 = 0. From Theorem 2, there exists a ( + 1; ) convolutional code that defines the dynamic of the trellis. Thus, since all states sN 01 at trellis step N 0 1 are connected 0 , only bits of the last k -tuple u with the all-zero state sN N 01 are necessary to drive the encoder back to the all-zero state. In the following, we explain the encoding algorithm by encoding the information block u = [u 0 ; u1 . . . uN 01 ].
• The first N 0 1 k -tuples u = [u 0 ; u 1 . . . u N 02 ] are encoded by the (n; k) encoder initialized to the all-zero state. (N 0 1)n encoded bits are produced.
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 50, NO. 5, MAY 2004
873
To ensure the code zero termination we must force the encoder to reach a state
TABLE III DISTANCE SPECTRUM PERFORMANCE OF THE BEST (n 1)=n SYSTEMATIC ENCODERS (THE BOLDFACE NUMBERS DENOTE THE df )
0
sN 0 +i 2 N 0 +i for each trellis termination step N 0 + i, i = 0; . . . ; 0 1. As seen, for the first 0 1 trellis termination steps, i = 0; . . . ; 0 2, each state sN 0 +i 2 N 0 +i is connected to 2l01 =2 states
sN 0 +i+1 2 N 0 +i+1 l 0 1 i.e., half of the 2 edges leaving each state leads to a suitable ending
state sN 0 +i+1 . So, only 1 bit is necessary in each trellis termination step to report the encoder to a suitable state, i.e., to distinguish the sN 0 +i+1 2 N 0 +i+1 from the other sN 0 +i+1 states. Thus, a total of 0 1 tail-biting bits are necessary to report the encoder to a state sN 01 2 N 01 at trellis step N 0 2. Finally, in the last trellis termination step, N 0 1, l 0 1 additional tail-biting bits are necessary to drive the encoder back to the all-zero state sN = 0 (we must choose one from 2l01 possible edges—the one connected to the all-zero state). The overall number of tail-biting bits is then
0 1 + (l 0 1) = 1 + 0 (l 0 1) 0 1 + (l 0 1) = :
The tail-biting bits values can be calculated generalizing (18) 0=
sN
=
j =1
uN 0j AB j + sN 0 B :
(19)
The new zero-termination solution leads to a block code of rate
Nk 0 Nn
(20)
which causes only a slight rate loss.
•
bits of the last k -tuple are used as tail-biting bits. So, only k 0 bits of the last k -tuple carry information. The value of the tail-
Example 2: Zero termination of a rate–2=3 encoder of memory
= 3.
Consider the (768; 509) terminated code obtained from a rate–2=3 encoder concatenating 256 trellis sections. With the proposed termination technique, p = = 3 tail-biting bits are needed to drive the encoder back to the all-zero state. Let the encoder be described by the systematic generator matrix
biting bits can be calculated through (7)
sN
=
uN 01A 8 sN 01B
(18)
imposing sN = 0. Thus, to drive the encoder back to the all-zero state it is necessary to solve a linear system with equations. This trellis termination procedure can be extended to every (n; n 0 1) code. Consider an (n; n 0 1) code, with a dynamic definedby the (l; l 0 1) constituent convolutional code, with 2 l + 1. It is easy to verify that = 1 + 0 (l 0 1) trellis steps are needed to drive the encoder back to the all-zero state. Indeed, consider the following facts. • All 2 states sN 0 at section index t = N 0 are connected to the all-zero state with a maximum of trellis steps. • A set of 2 =2i states sN 0 +i , at section index t = N 0 + i (0 i 01),areconnectedtotheall-zerostatewithamaximum of 0 i trellis steps. We denote this set N 0 +i . • Each state at section index t = N 0 + i such that
sN 0 +i 2 N 0 +i is connected to exactly 2l01 =2 states sN 0 +i+1 2 N 0 +i+1
1
0
0
1
1+D+D +D 1+D+D 1+D +D 1+D+D
A=
0
1
0
0
1
1
B=
0
1
0
0
0
1
1
0
1
:
The last two sections of the code trellis are shown in Fig. 3. Since each state in the complete trellis section is connected with four states, two steps in the trellis and p = = 3 tail-biting bits are needed to drive the encoder back to the zero state. The values of the tail-biting bits are obtained from the linear system
sN
= u N 01 A + s N 02 B
= u N 01 A + u N 02 AB + s N 02 B
(0; 0; 0) =
u0N 01uN1 01 u1
N 02 N 02 )
0
)
:
By inspection, we obtain the transition and state matrices
0 + (u
of the successive trellis section. • i = 0 1 corresponds to the last trellis termination step. In this case, jN 01 j = 2l01 and each state sN 01 2 N 01 is connected to the all-zero state sN = 0 N by one edge. 5 5Here “edge”means a set of parallel edges defined by the constituent ( block code.
G(D) =
1
2
0
1
0
1
with solutions
0 1
0
0
1
0
+ (sN 02 sN 02 sN 02 )
2
1 0 0
1
1
1
0
1
1
1
1
uN1 02 = sN1 02 + sN2 02 uN0 01 = uN0 02sN0 02 + sN1 02 uN1 01 = uN0 02sN0 02 + sN1 02 + sN2 02 :
874
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 50, NO. 5, MAY 2004
TABLE IV ENCODER DESCRIPTION FOR THE BEST SYSTEMATIC ENCODERS OF TABLE III AND RELATED WEIGHT SPECTRA
Fig. 3.
Trellis termination of the encoder of Example 2.
Note that only 3 tail-biting bits are needed instead of 1 k = 6 bits of 5 the standard zero-termination procedure. V. PUNCTURED HIGH-RATE CONVOLUTIONAL CODES Punctured convolutional codes have been widely used to reduce the decoder complexity of high-rate convolutional codes. A punctured
high-rate convolutional code is obtained from a lower rate mother code by deleting (puncturing) some of the encoded bits to reach the desired rate. Given the original low-rate code, the high-rate punctured code depends on the puncturing pattern. This pattern can be conveniently defined through a puncturing matrix P with binary elements 0; if symbol i of every j th branch is punctured (21) pij = 1; if symbol i of every j th branch is retained. The criterion for choosing the puncturing matrix should be the same addressed for high-rate codes, i.e., the optimization of the distance spectrum. To make our presentation self-contained, but also to compare the performance of the new high-rate convolutional codes presented here with the best punctured codes, we present in this section the results of a search for optimum recursive systematic punctured codes with rates up to 15=16. The best codes found are reported in Table V. The table provides for each punctured code the puncturing matrix, the free distance df , and the first five multiplicities of the distance spectrum. Tables of optimal punctured codes for rates up to 11=12 can also be found in [1]–[3], [27]. However, codes reported in those articles are obtained by puncturing feedforward codes. VI. DECODING HIGH-RATE CONVOLUTIONAL CODES: DUAL-SISO ALGORITHM A straightforward application of APP decoding to high-rate codes becomes quickly impractical as the coding rate increases, since its complexity increases with the number of transitions. Several authors have observed that APPs can be computed on the trellis of the dual code which, for the case of high-rate codes, is inherently
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 50, NO. 5, MAY 2004
BEST RATE (n
875
TABLE V
0 1)=n PUNCTURED CODES AND RELATED WEIGHT SPECTRA
less connected, thus strongly simplifying the decoding. The original idea of dual APP decoding is developed in [4]. Recent works in this area include [7] and [15], which also considers suboptimal APP decoding approaches. In this section, we rederive the APP decoding algorithm in the dual domain to show that with a proper modification of the bit soft information, it is perfectly equal to the one pertaining to the original code. Then, our main contribution is the derivation of the dual algorithm in
the log domain, completing the correspondence between the decoding algorithm in the original and dual domains. Recalling the definition of likelihood ratios (LR) defined as
L0m (O) 0 (I ) Lm
p(rjcm = 1) p(rjcm = 0 p(rm jcm = 1) p ( r m jc m = 0
876
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 50, NO. 5, MAY 2004
with some simple algebra manipulations of the basic results in [4] it is possible to obtain the following result: 1
0 Lm (O) = 0
1 + L (O )
m 0
0
0L (I )
1
n l=1
2C
c
c
1 L (I ) 1+L (I )
n l=1
2C
c
8 c
:
(22)
1+L (I )
Equation (22) has been derived in this form to show that, when considering the dual code as a basis for the computation of soft information it is appropriate to define, instead of the LR used in the SISO module of the original code, the reflection coefficients (RC) defined as follows:6
0 (O ) Rm
1
0 Lm0 (O)
0 (I ) Rm
1
0 Lm0 (I ) :
0 (O ) 1 + Lm 0 (I ) 1 + Lm
Using the RCs (22) simplifies to
0 (O ) Rm
2C
c
2C
c
0 (O)jL Lm
R0 I 8 R0 I
n c l=1 l ( ) n c l=1 l ( )
=
(I )=1
= (r ; b )
:
(ln
jRj; sign(R)) =) R = b exp(r ):
The LRC is represented as a pair of a real number r and a binary quantity b 2 f61g. Using this definition, the products and sums appearing in (26) map into the following operators:
AB =) + (ln jAB j; sign(AB )) = ( r + r ; b b ) A + B =) max3 ( ; ) (ln jA + B j; sign(A + B ))
(23)
(24)
ab ea
+ bb e
b
; sign ab ea
=
ln
=
max(ar ; br ) + ln 1 + ab bb e
+ bb e
(28)
b
0ja 0b j ; max(ar ; br )b : (29)
It is known that the iterative decoder building modules should pro0 (O), an extrinsic information vide at their output, instead of the LR Lm that can be obtained from the total information setting to 1 the corre0 (I ), that is equivalent to divide the total LR Lm 0 (O ) sponding input Lm 0 by the corresponding input LR Lm (I )
L m (O )
but also its sign. Let us consider the following one-to-one mapping that defines the LRCs from the RCs:
0 (O ) Lm 0 (I ) : Lm
The definition of the extrinsic RCs can then be derived observing that letting the LR Lm (I ) to be “1” is equivalent to setting to zero the corresponding RC. To derive an expression for the extrinsic RC from (24) it is suitable to split the dual code in the two subsets that contain the codewords with a “1” or a “0” in the mth position as in (25) at the bottom of the page. The extrinsic RCs are then obtained letting Rm (I ) = 0
The following considerations can be done on the definitions of the operators + and max3 in the log domain (28) and (29). • The addition does not introduce any difficulty when applied to LRCs instead of LLRs since it simply translates into a real addition and a binary addition. • The correcting term of the max3 operator for LRCs can assume two forms, depending on the sign of the binary quantity ab bb . If this term is positive, the correction term is bounded between 0 and log 2 as for the LLR case. However, when negative, it ranges from 01 to 0 and thus cannot be neglected for a suboptimal implementation. A suboptimal solution for the Additive-DSISO is currently under investigation.
(26)
• The binary part of the LRC returned by the max3 operator is equal to the binary part of the LRC with the maximum real part.
Note that (26) is perfectly matched to the input–output relationships of the SISO (see [13] for the derivation)
• The operation that combines two independent LLRs of the same binary quantity is simply its sum. Using LRCs this operation becomes
R m (O ) =
L m (O ) =
2 c2C
R l (I ) c : c l6=m Rl (I ) 6
c C :c =1 :c
2 c2C :c
l=m
=0
c C :c =1 =0
L l (I ) c : c l6=m Ll (I ) 6
l=m
A+B
(27)
So, the DSISO is formally equivalent to the SISO algorithm, provided that we substitute the sequence of LR Lm with the sequence of RC Rm and the code C with its dual C? .
1 + AB
Having defined the two operators “+” and “max3 ” the equations of the Additive-SISO are still applicable to the LRC
m (O) = maxc3:c
=1
For binary codes, the SISO algorithm can be performed more efficiently in the logarithmic domain, where the probabilities are equivalently characterized by the log-likelihood ratios (LLRs). It is also possible to derive an additive version of the DSISO as it was done in [13] for the SISO. However, due to the nature of the RCs, we have to carefully define the operators in the logarithmic domain. Indeed, RCs can be negative quantities, and thus their logarithmic counterpart must be defined accordingly. When defining the log-reflection coefficients (LRCs) it is necessary to take into account not only its absolute value 6A similar derivation is reported in [7] by Fourier transforming the intrinsic likelihood ratios, yielding what we call reflection coefficients.
2 c2C
c C :c =1 :c
=0
6
l=m
A. The Dual SISO in the Log Domain (Additive-DSISO)
0 (O ) = Rm
) max3 ( ; ) 0 max3 (0; + ):
=
Rl0 (I )c 0 c l6=m Rl (I ) 6
l=m
cl l (I ) 0 maxc3:c
=0
6
cl l (I ):
(30)
l=m
VII. VERSATILE HIGH-RATE CONVOLUTIONAL CODES THROUGH INVERSE PUNCTURING A nice property of punctured convolutional codes is their high flexibility. With puncturing, one can obtain several rates from the same mother code, thus simplifying the implementation through a sort of universal encoder [25]. Owing to this implementation, the code rate can be dynamically changed without increasing the decoding complexity, since the decoder works on the trellis of the mother code. This fact has been widely exploited in very large scale integration (VLSI) implementations.
0 0 (I ) + Rm
+ Rm (I )
2 c2C
c C :c =0 :c
=1
Rl0 (I )c 0 c : l6=m Rl (I ) 6
l=m
(25)
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 50, NO. 5, MAY 2004
877
TABLE VI A SEQUENCE OF ALMOST OPTIMAL ENCODERS OBTAINED APPLYING INVERSE PUNCTURING
Fig. 4.
(5; 4; 2) convolutional code of Example 3.
Besides discussions on the decoding complexity, punctured codes are commonly preferred to truly high-rate codes because of their adaptability. Indeed, high-rate codes require, in principle, to change the encoder while changing rate, thus making them not too attractive for practical applications. In this section, we show that a sort of puncturing method can also be applied to high-rate convolutional codes in order to achieve the desired versatility through a sequence of the (almost) best convolutional encoders. We shall refer to this method as inverse puncturing. Indeed, whereas puncturing is applied to obtain higher rate codes starting from a low-rate mother convolutional code, the inverse puncturing method works in the opposite way: starting from an (n; n 0 1) mother convolutional code we obtain the lower rate (m; m 0 1) convolutional code, with m < n, by properly deleting branches of the mother code trellis section. An interesting result is that this shortening technique translates in the dual domain into a simple puncturing of the dual code. This fact can be exploited to use the same encoder–decoder scheme to code/decode truly high-rate codes with different rates. We clarify how the inverse puncturing works with the following example. Example 3: Inverse puncturing of a rate–4=5 convolutional code. Consider the best (5; 4; 2) convolutional code of Table I described by the vector z = [7; 2; 6; 5; 7]. The encoder is reported in Fig. 4. For each four input bits entering the encoder, five bits are sent to the channel. Now, by deleting one of the four input bit lines of the encoder (as an example, we delete the first bit line, marked with 2), we generate a rate–3=4 convolutional code. In a similar way, lower rates can be obtained by increasing the number of deleted bit lines. Note that deleting a bit line of the encoder is equivalent to canceling the corresponding row of the code generator matrix, or, equivalently, the corresponding entry of vector z . Thus, deleting the first bit line in Fig. 4 corresponds to deleting the first entry of vector z , so the inverse punctured (4; 3; 2) convolutional code is described by the vector z i pun = [2; 6; 5; 7]. Also, puncturing in such a way does not affect the encoding process, so that the encoding complexity corresponds to that of the lower rate (4; 3; 2) convolutional code. Let us consider now the decoding algorithm based on the dual code, and how the inverse puncturing affects the decoding process. The dual of an (n; n 0 1; ) convolutional code is an (n; 1; ) feedforward convolutional code whose n polynomials correspond to the entries of the
Fig. 5. Dual trellis section of the
(5; 4; 2) convolutional code of Example 3.
vector z . The trellis section of the dual code of the (5; 4; 2) mother convolutional code is reported in Fig. 5. For each input bit entering the encoder, five output bits are generated, i.e., there are two branches leaving and entering each state, each of them labeled with five binary digits. We have seen that inverse puncturing of the (5; 4; 2) mother code is equivalent to deleting the first entry of the vector z , so the dual code of the punctured (4; 3; 2) code is described by the feedforward polynomials [2; 6; 5; 7]. But this is equivalent to deleting the first bit for each 5-tuple in Fig. 5. Thus, inverse puncturing of the mother code translates, in the dual code domain, to deleting the corresponding bits of the dual trellis. As a consequence, the decoding of the punctured (4; 3; 2) convolutional code can be done over the dual trellis of the (5; 4; 3) mother code by properly deleting bits in the trellis section. Different inverse puncturing patterns could be applied, obtaining different (4; 3; 2) convolutional codes. The best pattern should be chosen 5 in order to optimize the distance properties of the code. To show the potential of the inverse puncturing method, we report in Table VI a sequence of almost best convolutional encoders obtained by inverse puncturing of a rate-8=9, 8 states, mother encoder. The minor differences with respect to the optimal codes of Table III are indicated by boldface numbers. Observe that encoders obtained through inverse puncturing still maintain very good properties. Indeed, the only differences (if there exist) with respect to the optimal ones are observed in the last multiplicities Ni , whereas di remain invariable. VIII. SIMULATION RESULTS To illustrate the potential of the high-rate convolutional codes presented here, the performance of PCCC system based on the use of our high-rate convolutional encoders have been tested through simulations and analytical upper bounds to the bit-error probability.7 The block diagram of a parallel concatenated code is reported in Fig. 6. The encoder consists of the concatenation of two parallel high-rate convolutional encoders of rate (n 0 1)=n (Tables I and II). The overall rate is (n 0 2)=n. The permutation 5(1) is given by an 7The upper bounds are obtained by averaging the union bound with respect to the class of all possible interleavers of a given length [21].
878
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 50, NO. 5, MAY 2004
Fig. 6.
Parallel concatenated system.
Fig. 7.
Performance of various rate-9=11 PCCC with truncated component codes of rate 9=10; sixth iteration; N =4000 bits.
S -random interleaver [20]. The corresponding decoder consists of two SISO decoders working on the dual code (DSISO). As we have shown, the DSISO module is formally equivalent to the SISO module, provided that we substitute the sequence of LRs with the sequence of the so-called “reflection coefficients” (RCs) (see Section VI). Notice the transformation from likelihood ratios to RCs required at the input of the DSISO modules (see (23) in Section VI). The performance of a rate-9=11 PCCC with (10; 9; 4) constituent convolutional codes of Table I and convolutional encoders of Table III are reported in Fig. 7. As a comparison, the performance of the same turbo code based on punctured constituent codes is also reported. Due to their better characteristics, a significant improvement in the floor region is obtained by using our best high-rate codes and encoders. A gain of 1 dB is observed at a bit-error rate (BER) of 1008 . Fig. 8 reports the performance of a rate-5=7 PCCC employing the best (6; 5; 4) encoder of Table III as constituent encoders. As a comparison, the performance for the same turbo code employing punctured convolutional codes are also reported. Due to its better characteristics, a significant improvement in the floor region is obtained by using our best high-rate encoders. Finally, the curves for the same PCCC employing as constituent encoders the best (6; 5; 4) convolutional code of Table II are also reported. The performance of the two PCCC based on high-rate codes are almost identical in the waterfall region. However, the turbo code based on the best rate-5=6 encoder shows a better be-
havior below BER = 1008 , illuminating the role of the T (W; D) on the performance of the PCCC. Finally, we compared the analytical average performance of some turbo codes employing either punctured convolutional codes or our best codes. The results are reported in Fig. 9 for rate-11=13, -9=11, and rate–3=4 turbo codes employing rate-11=12, -9=10, and -6=7 constituent codes, respectively. Significant improvements over punctured codes are obtained if using our high-rate codes. As an example, comparing the two rate-3=4 turbo codes of Fig. 9, we notice a difference of almost 3 dB at a BER of 1007 . Moreover, as explained in the complexity subsection, the use of the dual MAP algorithm to decoding unpunctured codes leads to a slight reduction of the decoding complexity. So, we improve both performance and decoding complexity. Finally, a performance comparison of the zero-termination and tail-biting termination methods is reported in Fig. 10. We considered the termination of the (13; 12; 4) feedforward convolutional code of Table II. Following the considerations addressed in Section IV, only = 4 tail-biting bits are necessary for the zero-terminated code. A direct application of the zero termination would require 1 k = 48 tail-biting bits. In particular, curves in Fig. 10 correspond to a block length of 120 bits (116 for the terminated code using the new termination technique), i.e., 10 trellis sections. The standard zero termination incurs a loss of 1.2 dB at a BER of 1005 with
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 50, NO. 5, MAY 2004
879
Fig. 8. Performance of various rate-5=7 PCCC with truncated component codes of rate 5=6; sixth iteration; N = 4000 bits.
Fig. 9. Average bit-error probability for different rates of PCCC.
respect to the tail-biting codes due to the large overhead. This loss is overcome if we apply the proposed zero termination. Indeed, the new zero-termination technique shows slightly better performance than the tail-biting code. This is due to the fact that tail-biting codes are, in general, slightly worse in terms of error event multiplicities than zero-terminated codes. In our case, the first multiplicities of the distance spectrum of the equivalent block codes obtained by tail-biting and zero-termination of the (13; 12; 4) code yield (ad
; ad ; ad
) = (490; 13330; 291936)
(a d
; ad ; ad
) = (347; 6650; 89955)
and respectively.
IX. CONCLUSION The design and decoding of new (n; n 0 1) high-rate convolutional codes were studied. We proved that every (n; n 0 1) convolutional encoder can be viewed as formed by two constituent encoders: a block encoder associated to parallel edges between states and a low-to-moderate convolutional encoder that defines the nonparallel edges. Following this result, we devised a search algorithm to find the best (n; n 0 1) convolutional codes with an affordable search complexity. The codes were optimized in terms of distance spectrum. An exhaustive search for rates up to 16=17 and various memories was done and tables for both recursive and feedforward codes were reported. As the optimization criteria for constituent encoders of turbo-like concatenated codes are different, and based on input–output (rather
880
Fig. 10.
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 50, NO. 5, MAY 2004
Termination trellis comparison.
than output only) weight enumerating functions, the search was extended to find the best (n; n 0 1) convolutional encoders to be embedded into turbo-like structures. In order to reduce the decoding complexity of such codes we elaborated on a decoding algorithm working on the dual code (Dual-SISO algorithm). We showed that a proper modification of the bit soft information passed in the iterative procedure yields the MAP decoding working on the dual code perfectly equal to the one pertaining to the original code. Moreover, we proposed a new additive version of the Dual-SISO algorithm paralleling the one working on the original code trellis. A new method to terminate the code is derived, which only requires (the code memory) tail-biting bits. The new technique reduces the rate loss to a point that makes the use of tail-biting high-rate codes unnecessary for medium-large block sizes. Finally, an inverse puncturing technique applied to the highest rate mother code to yield a versatile sequence of codes with decreasing rates have been proposed, that induces marginal performance penalties to the code so obtained. Extensive simulation results have shown the significant advantages of the newly found codes. The codes described in this correspondence are good candidates for use in those applications requiring high energy and bandwidth efficiencies, like magnetic recording and fiber optic, and flexibility in the range of bandwidth efficiencies, like digital video broadcasting and others. REFERENCES [1] J. Chang, D. Hwang, and M. Lin, “Some extended results on the search for good convolutional codes,” IEEE Trans. Inform. Theory, vol. 43, pp. 1682–1697, Sept. 1997. [2] G. Bégin, D. Haccoun, and C. Paquin, “Further results on high-rate punctured convolutional codes for viterbi and sequential decoding,” IEEE Trans. Commun., vol. 38, pp. 1922–1928, Nov. 1990. [3] Y. Bian, A. Popplewell, and J. J. O’Reilly, “New very high rate punctured convolutional codes,” Elettron. Lett., vol. 30, no. 14, pp. 1119–1120, July 1994. [4] C. R. Hartmann and L. D. Rudolph, “An optimum symbol-by-symbol decoding rule for linear codes,” IEEE Trans. Inform. Theory, vol. IT-22, pp. 514–517, Mar. 1974. [5] S. Riedel, “MAP decoding of convolutional codes using reciprocal dual codes,” IEEE Trans. Inform. Theory, vol. 44, pp. 1176–1187, May 1998. [6] G. Montorsi and S. Benedetto, “An additive version of the SISO algorithm for the dual code,” in Proc. IEEE Int. Symp. Information Theory, Washington, DC, June 2001, p. 27.
[7] G. D. Forney Jr, “Codes on graphs: Normal realizations,” IEEE Trans. Inform. Theory, vol. 47, pp. 520–548, Feb. 2001. [8] T. Minowa and H. Imai, “Decoding of high-rate turbo codes using a syndrome trellis,” in Proc. IEEE Int. Conf. Communications, vol. 1, 2001, pp. 74–78. [9] S. Benedetto and E. Biglieri, Principles of Digital Transmission With Wireless Applications. New York: Kluwer Academic/Plenum, 1999. [10] H. H. Tang and M. C. Lin, “On (n; n 1) convolutional codes with low trellis complexity,” IEEE Trans. Commun., vol. 50, pp. 37–47, Jan. 2002. 1) convolutional [11] M. Tuechler and A. Dholakia, “New rate-( codes with optimal spectrum,” in Proc. IEEE Int. Symp. Information Theory, Lausanne, Switzerland, June/July 2002, p. 396. [12] A. Graell i Amat, G. Montorsi, and S. Benedetto, “A new approach to the construction of high-rate convolutional codes,” IEEE Commun. Lett., vol. 5, pp. 453–455, Nov. 2001. [13] S. Benedetto et al., “Soft-input soft-output modules for the construction and iterative decoding of code networks,” Europ. Trans. Telecomm., vol. 9, no. 2, pp. 155–172, Mar./Apr. 1998. [14] C. Weiß, C. Bettstetter, and S. Riedel, “Code construction and decoding of parallel concatenated tail-biting codes,” IEEE Trans. Inform. Theory, vol. 47, pp. 366–386, Jan. 2001. [15] J. Berkmann and C. Weiß, “On dualizing trellis-based APP decoding algorithms,” IEEE Trans. Commun., vol. 50, pp. 1743–1757, Nov. 2002. [16] R. Bahl, J. Cocke, F. Jelinek, and J. Raviv, “Optimal decoding of linear codes for minimizing symbol error rate,” IEEE Trans. Inform. Theory, vol. IT-20, pp. 284–287, Mar. 1974. [17] C. Berrou, A. Glavieux, and P. Thitimajshima, “Near Shannon limit error-correcting coding and decoding: Turbo codes,” in Proc. IEEE Int. Conf. Communications, 1993, pp. 1064–1079. [18] A. Glavieux, C. Laot, and J. Labat, “Turbo equalization over a frequency selective channel,” in Proc. Int. Symp. Turbo Codes and Related Topics, 1997, pp. 92–102. [19] A. Graell i Amat, G. Montorsi, and S. Benedetto, “New high-rate convolutional codes for concatenated schemes,” in IEEE Int. Conf. Communications , New York, Apr./May 2002. [20] D. Divsalar and F. Pollara, “Turbo codes for PCS applications,” in Proc. IEEE Int. Conf. Communications, Seattle, WA, June 1995, pp. 54–59. [21] S. Benedetto and G. Montorsi, “Design of parallel concatenated convolutional codes,” IEEE Trans. Commun., vol. 44, pp. 591–600, May 1996. [22] S. Benedetto, D. Divsalar, G. Montorsi, and F. Pollara, “Serial concatenation of interleaved codes: Performance analysis, design, and iterative decoding,” IEEE Trans. Inform. Theory, vol. 44, pp. 909–926, May 1998. [23] S. Benedetto and G. Montorsi, “Unveiling turbo codes: Some results on parallel concatenated coding schemes,” IEEE Trans. Inform. Theory, vol. 42, pp. 409–429, Mar. 1996. [24] S. Benedetto, R. Garello, and G. Montorsi, “A search for good convolutional codes to be used in the construction of turbo codes,” IEEE Trans. Commun., vol. 46, pp. 1101–1105, Sept. 1998.
0
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 50, NO. 5, MAY 2004
[25] J. Hagenauer, “Rate-compatible punctured convolutional codes (RCPC codes) and their applications,” IEEE Trans. Commun., vol. 36, pp. 389–400, Apr. 1988. [26] Ø. Ytrehus, “A note on high rate binary convolutional codes,” Univ. Bergen, Bergen, Norway, Tech. Rep. 68, 1992. [27] I. E. Bocharova and B. D. Kudryashov, “Rational rate punctured convolutional codes for soft-decision viterbi decoding,” IEEE Trans. Inform. Theory, vol. 43, pp. 1305–1313, July 1997.
Two Constructions of Permutation Arrays Fang-Wei Fu and Torleiv Kløve, Fellow, IEEE Abstract—In this correspondence, two new constructions of permutation arrays are given. A number of examples to illustrate the constructions are also provided. Index Terms—Bounds, code constructions, permutation arrays.
I. INTRODUCTION Let Sn denote the set of all n! permutations of Zn = f0; 1; . . . ; n01g. An (n; d) permutation array (PA) is a subset of Sn with the property that the Hamming distance between any two distinct permutations in the array is at least d. An (n; d) PA of size is called an (n; ; d) PA. PAs were somewhat studied in the 1970s, some important papers from that period are [2], [5], and [8]. A recent application by Vinck [14] of PAs to a coding/modulation scheme for communication over power lines has created renewed interest in PAs, see [3], [4], [6], [7], [10], [12], [15]–[17]. In this correspondence, we give a couple of new general constructions of PAs. Let P (n; d) be the maximal size of an (n; d) PA. The constructions give improved lower bounds on P (n; d) in some cases. We say that a PA 5 is balanced if for each position, each element of Zn appears the same number of times in that position of the permutations of 5. In [6], we presented a construction of PAs using balanced PAs as building blocks. Under suitable conditions, the PAs constructed in this correspondence are balanced. II. THE MAIN RESULTS We use Zq to denote the set f0; 1; . . . ; q 0 1g of integers, not the ring of integers modulo q . However, on a number of occasions, we do modular addition. For k , l 2 Zq , k 8q l denotes the unique integer in Zq congruent to k + l modulo q .
881
Ordinary codes will be building blocks in our construction. A code
C of length n over Zq is a subset of Zqn . The minimum and maximum Hamming distances between distinct codewords of C are denoted by dmin (C ) and dmax (C ), respectively. A code of size M and minimum distance at least d is called an (n; M; d; q ) code, and a code of size M , minimum distance at least d1 and maximum distance at most d2 is called an (n; M; d1 ; d2 ; q ) code. The maximal size M for which an (n; M; d; q ) code exists is denoted by Aq (n; d) and the maximal size M for which an (n; M; d1 ; d2 ; q) code exists is denoted by Aq (n; d1 ; d2 ). Clearly, Aq (n; d; n) = Aq (n; d). Tables of bounds on Aq (n; d) are found in [1, pp. 463–498] for q = 2 and in [1, pp. 295–461] for general q (for linear codes). Bounds on A3 (n; d) are given in [13]. We say that C is balanced if for each position, each element of Zq
appears the same number of times in that position of the codewords of C . In particular, a linear code over a finite field is balanced if there is no position where all the codewords are zero. Codes are often defined over a finite field or a finite ring of size q , say. By renaming of the elements, we get a corresponding code over Zq (algebraic properties may not be carried over, but this is not important in our context). In our examples, we therefore assume that the codes considered (e.g., Hamming codes) have elements in Zq . Definition 1: For c = [ci ]0i
