IEEE COMMUNICATIONS LETTERS, VOL. 17, NO. 2, FEBRUARY 2013
373
Spatially Coupled Repeat-Accumulate Codes Sarah Johnson and Gottfried Lechner
Abstract—In this paper we propose a new class of spatially coupled codes based on regular repeat-accumulate codes. We show that these codes have several advantages over spatially coupled regular low-density parity-check codes including simpler encoders and slightly higher code rates when the thresholds and decoding complexity (as measured by the Tanner graph edge density) are similar. Index Terms—Spatial coupling, repeat-accumulate codes, lowdensity parity-check codes, density evolution.
I. I NTRODUCTION
C
ONVOLUTIONAL LDPC codes, otherwise known as spatially coupled LDPC codes (SC-LDPC), were first introduced by Felstr¨om and Zigangirov in the late 90’s [1]. Performance results, generated using either density evolution or decoding simulations, have shown that coupled regular LDPC codes have excellent sum-product decoding thresholds over a range of channels [2]–[4]. Incredibly, and in contrast to standard regular LDPC codes, these thresholds rapidly improve as a function of the average Tanner graph node degree. This enables the design of iterative error correction codes with both excellent thresholds and very low error floors, something not so far achieved with traditional LDPC or turbo codes. Recent exciting developments have shown that the iterative decoding threshold of certain SC-LDPC ensembles converges to their MAP threshold on the binary erasure channel (BEC) [5], [6]. I.e., for spatially coupled codes iterative decoding is actually optimal on the BEC. In this paper we investigate whether the concept of spatial coupling can apply equally well to another class of iterative error correction codes called repeat-accumulate (RA) codes. We consider (q,a)-regular RA codes [7], which are error correction codes formed by the serial concatenation of a rate1 convolutional code, called an 1/q repetition code and a 1+D accumulator, with an interleaver, Π, and (optionally) a ratea combiner between them. Significantly, RA codes can be encoded using a serial concatenation of the constituent encoders, as for serially concatenated turbo codes, and decoded using iterative decoding, as for LDPC codes, thus gaining both the low encoding complexity of turbo codes and the decoding performance of LDPC codes. We will consider spatial coupling of RA codes in such a way as to preserve the inherent advantage of RA codes, most Manuscript received October 23, 2012. The associate editor coordinating the review of this letter and approving it for publication was M. Ardakani. S. Johnson is with the University of Newcastle, Australia (e-mail:
[email protected]). The work of S. Johnson was supported by the ARC Grants FT110100195 and DP1093114. G. Lechner is with the Institute for Telecommunications Research, University of South Australia (e-mail:
[email protected]). The work of G. Lechner was supported by the ARC Grant DP120102123 and by Vienna Science and Technology Fund - WWTF grant ICT10-067. Digital Object Identifier 10.1109/LCOMM.2013.010313.122343
importantly their very simple encoding, while obtaining the threshold advantages promised by the idea of spatial coupling. It is particularly interesting to study the benefit of spatial coupling for RA codes as they are currently employed in a wide range of telecommunications applications including digital video broadcasting (DVB-S2) [8] and wireless local area networks (WiFi and WiMAX) [9], [10]. The method we consider can be seen as a particular constraint on the interleaver design while the accumulator is unchanged. Hence, performance gains due to spatial coupling can be achieved by redefining the interleaver in existing RA coding schemes. Unlike regular LDPC codes, regular RA codes do not have linear minimum distance growth and so very low error floors and asymptotic optimality on the BEC is not actually guaranteed in the current theory. Nevertheless density evolution results show that spatial coupling can indeed improve the performance of RA codes past their belief propagation threshold. Related Work The concept of spatial coupling has been applied to general protograph LDPC codes. In particular, spatial coupling of accumulate-repeat-by-4-jagged-accumulate (AR4JA) codes in [11] which also show minimum distance growth linearly with the block length. While these codes show a faster convergence, as compared to spatially coupled regular RA codes, they do not resemble the original structure of an RA code and hence do not inherit their ease of encoding. We will compare the numerical results for AR4JA codes with spatially coupled regular RA codes in Section IV. II. S PATIALLY C OUPLED RA C ODES Spatially coupled RA (SC-RA) codes can be formed in a similar manner to SC-LDPC codes. We consider two ensembles, the first we will use in practice to construct SCRA codes, and the second is useful to derive density evolution equations. A. The (q,a,L) Ensemble The left hand side of fig. 1 shows the protograph of a standard (q, a)-regular RA code with q = 3 and a = 3. There is one message bit node, shown at the top, a parity bit node, shown at the bottom, and a check node in the middle. A coupled chain of 2L + 1 of these protographs, shown on the right hand side of fig. 1, is formed by connecting each message bit to ˆl = (q − 1)/2 protographs to the left and ˆl protographs to the right1 . We will refer to these codes as the (q, a, L) ensemble. As for coupled LDPC chains we add q − 1 extra check nodes (shown in bold) when forming the coupled chain of protographs. For RA protographs we must also add 1 For
the moment we assume q is odd.
c 2013 IEEE 1089-7798/13$31.00
374
IEEE COMMUNICATIONS LETTERS, VOL. 17, NO. 2, FEBRUARY 2013 −L
−L
+L
+L q
q a
Fig. 1.
Coupled q = 3, a = 3 RA protographs.
Fig. 3.
Fig. 2. Example of a q = 3, a = 3 SC-RA code with L = 1 and M = 2.
q − 1 extra parity bit nodes (shown in bold) to avoid creating any degree-1 check nodes. We could have spatially coupled the parity bit nodes in the same way as the message bit nodes, i.e., by connecting each parity bit node to the check node of the protograph on the right hand side. However, if the parity bit nodes are coupled in this way, the final code will not retain the RA code accumulator structure. Keeping the parity bit nodes uncoupled can be thought of as serially concatenating a spatially coupled low-density generator matrix with a standard accumulator. A particular code from the (q, a, L) ensemble will be formed using copies of the coupled chain to give a total of M message bits per protograph. Our final code will thus consist of (2L + 1)M message bit nodes, (2L + 1 + 2ˆl) aq M parity bit nodes and (2L+1+2ˆl) aq M check nodes. Hence the code rate, assuming every check node results in a linearly independent constraint, is (2L + 1)M rRA = (2L + 1)M + (2L + 1 + 2ˆl) q M (2L + 1)a = . (2L + 1)a + (2L + q)q
a
(1)
When constructing a code from the (q, a, L) ensemble each of the message bit nodes at position i ∈ {−L, · · · + L} is connected to exactly one of the check nodes at positions j ∈ {i − ˆ l, · · · i + ˆl}. The choice of which of the check nodes to connect to at each position can be chosen randomly. For each protograph the M parity bit nodes are connected to the M check nodes in a traditional accumulator pattern. We also connect the final parity bit node in each protograph to the first check node in the following protograph. The main advantage of designing SC-RA codes in this way is that the parity-check matrix retains the format employed to good effect in existing implementations of RA codes, making the switch from traditional RA codes to SC-RA codes in existing applications as simple as a new interleaver specification. Example 1. A (q = 3, a = 3) RA protograph is repeated 2L + 1 times to give the coupled chain in fig. 1. Setting L = 1, M = 2 and randomly choosing an edge permutation for the message bit edges gives the SC-RA Tanner graph in fig. 2. The permutation for the parity bit
Coupled q = 4, a = 4 RA protographs.
edges must be chosen to maintain the accumulator structure. As SCRA codes are systematic we form the codeword using the messages bits first, followed by the parity bits. This gives an SC-RA code with parity-check matrix H = ⎤ ⎡ 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 ⎢ 1 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 ⎥ ⎢ 0 1 0 1 0 0 0 1 1 0 0 0 0 0 0 0 ⎥ ⎥ ⎢ ⎢ 1 0 1 0 0 0 0 0 1 1 0 0 0 0 0 0 ⎥ ⎥ ⎢ ⎢ 0 1 1 0 0 1 0 0 0 1 1 0 0 0 0 0 ⎥ ⎥ ⎢ ⎢ 1 0 0 1 1 0 0 0 0 0 1 1 0 0 0 0 ⎥ ⎥ ⎢ ⎢ 0 0 0 1 0 1 0 0 0 0 0 1 1 0 0 0 ⎥ ⎥ ⎢ ⎢ 0 0 1 0 1 0 0 0 0 0 0 0 1 1 0 0 ⎥ ⎣ 0 0 0 0 1 0 0 0 0 0 0 0 0 1 1 0 ⎦ 0
0
0
0
0
1
0
0
0
0
0
0
0
0
1
1
By slightly re-drawing fig. 1 to push the top row of nodes across to the left immediately shows how to construct SC-RA codes with even values of q. Fig. 3, for example, shows a SC-RA code with q = 4 and a = 4. B. The (q, a, L, w) Ensemble The (q, a, L) ensemble can be modified by adding a “smoothing” parameter w in a similar method to that for LDPC codes [6]. This (q, a, L, w) ensemble is not used in practice but is useful to simplify the derivation of density evolution equations. Considering this ensemble for SC-RA codes will allow a comparison of the asymptotic performance of SC-RA codes with the SC-LDPC ensembles in [6]. As previously, at each position [−L, L] there are M message bit nodes. However, the check nodes are considered to be located at all integer positions [−∞, ∞] and there are aq M check nodes at each position. Only some of these positions actually interact with the message bit nodes. Instead of requiring that each message bit node at position i ∈ {−L, · · · + L} is connected to exactly one of the check nodes at positions j ∈ {i − ˆl, · · · i + ˆl} we assume that each of the q connections of a variable node at position i is uniformly and independently chosen from the range [i, · · · , i+w −1]. Similarly, we assume that each of the a connections of a check node at position i is independently chosen from the range [i−w +1, · · · , i]. q need not be odd. For simplicity we again assume that a parity bit node is associated with every active check node and connected once to that check node and once to the next adjacent active check node on the right. The connections of the parity bit nodes are chosen to maintain the accumulator structure. Using a similar derivation to that for LDPC codes [6] leads to the rate of the (q, a, L, w) ensemble as rRA,w =
2L + 1 +
q a
2L + 1 w a . 2L − w + 2 w + 1 − i=0 wi
JOHNSON and LECHNER: SPATIALLY COUPLED REPEAT-ACCUMULATE CODES
0.505
C. Encoding The motivation for considering SC-RA codes is their low encoding complexity. As for traditional repeat-accumulate codes, SC-RA codes can be encoded with complexity linear in the code length by the serial concatenation of a repetition 1 convolutional encoder or code, interleaver, combiner and 1+D accumulator. RA and SC-RA codes are systematic so that the message bits make up the first K bits in the codeword meaning that codeword bits can be transmitted as soon as message bits are received. The structure of SC-RA codes also has the additional advantage of limiting the number of message bits that must be received before the first parity bit can be encoded. Consider fig. 3. A parity bit in the ith location is a function only of message bits in the ith and previous q − 1 locations. III. D ENSITY E VOLUTION In this section we derive closed form expressions for density evolution for the (q, a, L, w) ensemble on the BEC and show how the multi-edge formulation for LDPC codes can be used to derive thresholds for the (q, a, L) ensemble. A. The (q, a, L, w) Ensemble Following a similar approach to that used for the LDPC wensemble [6] gives density evolution equations for the SC-RA (q, a, L, w) ensemble ⎡ ⎛ w−1
2 1 (+1) () ⎣ 1 − yi+j = ⎝1 − xi w j=0 a−1 ⎤⎞q−1 w−1 1 () ⎦⎠ · 1− xi+j−k (2a) w k=0 a w−1
1 () (+1) () = 1 − 1 − yi xi−k , (2b) yi 1− w k=0
() xi
375
() yi
where and denote the erasure probabilities from message bits and parity bits respectively at position i, at iteration . B. The (q, a, L) Ensemble Density evolution for the (q, a, L) ensembles results in more complicated expressions since the erasure probabilities on edges connected to one protograph cannot be averaged as for the (q, a, L, w) ensemble above. While it is still possible to write the expressions in closed form we instead choose the multi-edge framework to represent the structure of the (q, a, L) ensemble and use multi-edge density evolution to evaluate the decoding thresholds over the erasure channel. For a detailed description of multi-edge density evolution we refer the reader to [12, Sec. 7]. IV. N UMERICAL R ESULTS A. Asymptotic Results Density evolution results are shown in fig. 4 where we compare the decoding thresholds and rates for regular SC-LDPC and regular SC-RA ensembles. A regular LDPC ensemble is denoted by (dl , dr ) where dl and dr denote the variable and
10
0.5
th
6
6
8
10
5
8
5 4
6
4
6
0.495
L = 16
L = 32
0.49
3 0.485 0.42
0.43
3 4
SC−LDPC SC−RA 0.44
0.45
0.46
0.47
0.48
4 0.49
0.5
rate Fig. 4. Thresholds versus rates of spatially coupled LDPC and RA codes. The markers represent the different variable node degrees of the message bits, dl = {3, 4, 5, 6} for the SC-LDPC codes and q = {4, 6, 8, 10} for the SC-RA codes.
check node degree, respectively and the SC-LDPC ensemble is given by (dl , dr , L). We consider codes of asymptotic rate 0.5 and hence q = a for SC-RA codes and dr = 2dl for SCLDPC codes. No smoothing factor w was used and the results were obtained using multi-edge density evolution. Each curve corresponds to a value of L and the markers represent the variable node degree of the message bits. Higher degrees lead to an improved decoding threshold but result in a lower rate due to the increasing number of additional check nodes at the ends of the graph. Note that due to the accumulator (which consists of degree 2 variable nodes), SC-RA codes have a lower average variable node degree than SC-LDPC codes with the same degree for the message bits. To compare regular LDPC and RA codes with the same densities, and hence similar decoding complexities, we compare an LDPC base code with bit degree dl with an RA base code with bit degree q = 1r (dl − 2)+ 2 where r is the asymptotic code rate for large L. Thus fig. 4 shows points for LDPC protographs with dl = {3, 4, 5, 6} and RA protographs with q = {4, 6, 8, 10}. We observe that regular SC-RA codes perform better than regular SC-LDPC codes giving a higher code rate at the same decoding threshold as SC-LDPC codes. Density evolution for the smoothed ensembles shows similar results and is omitted. We also compare the SC-RA codes with a terminated AR4JA code from [11]. For this purpose we present the results in the same way as in [11], i.e., the parameter on the curves is the number of protographs L. Note that L in our paper denotes the one-sided length, i.e., the spatially coupled code consists of 2L + 1 protographs whereas the corresponding parameter in [11] denotes the number of protographs. We will refer to the later as L . The results of this comparison are shown in fig. 5. We see that TAR4JA codes show the best results and converge faster to the Shannon limit with respect to the number of protographs. SC-RA codes also show good results whereas SC-LDPC codes mainly suffer from a significant rate loss for short codes. For comparison we also show the results for (dl , dr ) regular uncoupled LDPC ensembles.
376
IEEE COMMUNICATIONS LETTERS, VOL. 17, NO. 2, FEBRUARY 2013
0.55
0.5
(4, 8)
(3, 6)
50
50
rate
an
25
25
0.45
Sh
does indeed produce codes with excellent iterative decoding performance. We also see that the performance of the SC-RA codes is better than that of the SC-LDPC codes with similar decoding complexity (as measured by the Tanner graph edge density) despite both having the same threshold. We suspect that for finite length codes the structure of the SC-RA codes gives them a further advantage (in addition to the slightly higher rate for the same threshold) over LDPC codes.
SC−RA q=10 SC−LDPC dl=6 TAR4JA e=0 (dl,dr) regular
no
n
lim
it
10 0.4
(3, 5)
10
5
0.35
4
V. D ISCUSSION
3 0.3
2
5 0.25 0.35
0.4
0.45
0.5
0.55
th
0.6
0.65
Fig. 5. Rates versus thresholds of spatially coupled codes. The length L of the TAR4JA code is {5, 6, 7, 8, 9, 10, 20, 50, 100}. 0
10
−1
10
SC−RA q=6, M=100, N = 7100 SC−LDPC dl=4, M = 220, N = 7260 SC−RA q=6, M=300, N = 21300 SC−LDPC dl=4, M = 660, N = 21780 RA threshold SC−RA threshold
−2
Error rate
10
−3
10
−4
10
−5
10
−6
10
In this paper we have proposed a new class of spatially coupled codes based on regular RA protographs. We show that SC-RA codes have several advantages over SC-LDPC codes including simpler encoders and slightly better thresholds than SC-LDPC codes with similar rates and decoding complexity. Simulation results for finite-length SC-RA codes also show improved decoding performances over SC-LDPC codes with the same threshold. Our proposed coupling structure is ideal for incorporating SC-RA codes in existing applications using RA codes. The only change to existing architecture will be in the design of the interleaver, which already varies in different applications. The repeat, interleave, accumulate encoding process is unchanged. Furthermore, the SC-RA codes remain systematic and easily encoded. This method of coupling could also be applied to irregular RA codes with the potential to slightly improve the threshold while maintaining the encoding structure of uncoupled irregular RA codes.
−7
10
0.4
0.41
0.42
0.43
0.44
0.45
0.46
0.47
0.48
0.49
0.5
BEC erasure probability
Fig. 6. Erasure correction performance of SC-LDPC and SC-RA codes with L = 16, for K = 3, 300 and K = 9, 900 using iterative decoding with a maximum of 1, 000 iterations. Solid lines show the word erasure rate and dashed lines show the bit erasure rate.
B. Finite Length Simulations We randomly construct SC-RA codes and compare their decoding performance at finite lengths to SC-LDPC codes. Consider for example the (q, a, L) ensemble with thresholds shown in fig. 4 for L = 16 with q = 6 for the SC-RA code and dl = 4 for the SC-LDPC code. The SC-RA ensemble has an average variable node degree of 3.86 (compared to 4 for the SC-LDPC code), a higher rate and a similar decoding threshold. Fig. 6 shows the erasure correction performance of (6, 6, 16) SC-RA codes with M set to 100 and 300 respectively. Also shown is the performance of (4, 8, 16) SC-LDPC codes with M set to 220 and 660 respectively. (Recall that for SC-LDPC codes M specifies the number of all bit nodes, whereas for SC-RA codes M specifies the number of message bit nodes). For the two shorter codes, each code transmits K = 3, 300 message bits, however the SC-RA code has a slightly higher rate requiring only 7, 100 codeword bits (r = 0.4648) instead of 7, 260 (r = 0.4545). For the two longer codes, each code transmits K = 9, 900 message bits, however the SC-RA code requires only 21, 300 codeword bits instead of 21, 780. In fig. 6 we can see that spatial coupling of RA codes
R EFERENCES [1] A. Jimenez Felstr¨om and K. Zigangirov, “Time-varying periodic convolutional codes with low-density parity-check matrix,” IEEE Trans. Inf. Theory, vol. 45, no. 6, pp. 2181–2191, Sep. 1999. [2] M. Lentmaier, A. Sridharan, K. Zigangirov, and D. Costello, “Terminated LDPC convolutional codes with thresholds close to capacity,” in Proc. 2005 IEEE Int. Symp. Inf. Theory, pp. 1372–1376. [3] R. Tanner, D. Sridhara, A. Sridharan, T. Fuja, and D. Costello, “LDPC block and convolutional codes based on circulant matrices,” IEEE Trans. Inf. Theory, vol. 50, no. 12, pp. 2966–2984, Dec. 2004. [4] M. Lentmaier, A. Sridharan, D. Costello, and K. Zigangirov, “Iterative decoding threshold analysis for LDPC convolutional codes,” IEEE Trans. Inf. Theory, vol. 56, no. 10, pp. 5274–5289, Oct. 2010. [5] M. Lentmaier, G. Fettweis, K. Zigangirov, and D. Costello, “Approaching capacity with asymptotically regular LDPC codes,” in Proc. 2009 Inf. Theory and App. Workshop, pp. 173–177. [6] S. Kudekar, T. Richardson, and R. Urbanke, “Threshold saturation via spatial coupling: why convolutional LDPC ensembles perform so well over the BEC,” IEEE Trans. Inf. Theory, vol. 57, no. 2, Feb. 2011. [7] D. Divsalar, H. Jin, and R. J. McEliece, “Coding theorems for “turbolike” codes,” in Proc. 1998 Allerton Conf. on Communications, Control, and Computing, pp. 201–210. [8] ETSI, “EN 302 307: Digital Video Broadcasting (DVB).” Available: http://www.dvb.org/ , 2009. [9] “The IEEE local and metropolitan area network standards, IEEE Std 802.11.” Available: http://standards.ieee.org/getieee802/ download/802.11n-2009.pdf . [10] “The IEEE local and metropolitan area network standards, part 2: Air interface for broadband wireless access systemsIEEE Std 802.16.” Available: http://standards.ieee.org/getieee802/download/802.16-2009.pdf . [11] D. Mitchell, M. Lentmaier, and D. Costello, “New families of LDPC block codes formed by terminating irregular protograph-based LDPC convolutional codes,” in Proc. 2010 IEEE Int. Symp. Inf. Theory, pp. 824–828. [12] T. Richardson and R. Urbanke, Modern Coding Theory. Cambridge University Press, 2008.
