Design of Variable-rate Irregular LDPC Codes with Low Error Floor Libero Dinoi
Francesco Sottile
Sergio Benedetto
Istituto Superiore “Mario Boella” Torino, Italy, Via P.C. Boggio 61 Email:
[email protected]
Istituto Superiore “Mario Boella” Torino, Italy, Via P.C. Boggio 61 Email:
[email protected]
CERCOM - Politecnico di Torino Torino, Italy, C.so Duca degli Abruzzi 24 Email:
[email protected]
Abstract— In this paper we propose a semi-random technique to design irregular LDPC codes with low error floor. In particular, the suggested technique is related to a popular class of irregular LDPC codes that can be encoded in linear time, and tries to avoid both low-weight codewords and near-codewords, which limit the performance of the Belief Propagation decoder. Besides, an easy and effective rule to design puncturing patterns for this class of LDPC codes is provided. The effectiveness of the new design criteria is proved by comparison with literature results.
I. I NTRODUCTION An LDPC code [1] is a linear block code defined by an mxn sparse parity-check matrix, which represents the relation between the n bits of the codeword and the m parity-check equations that specify the code. Another description of LDPC codes can be provided using Tanner graphs: each of the bits and of the parity-check equations, defined respectively as Variable Nodes (VNs) and Check Nodes (CNs), is represented by a vertex in the graph, while the VN (or CN) degree distribution of a code specifies how the edges are distributed among the variable (or check) nodes of the code. In a regular (c, r) LDPC code all VN have degree c and all CN have degree r in the graph. Irregular LDPC codes have been shown [2] to outperform significantly regular LDPC codes. Basically, there exist two different families of design algorithms for LDPC codes: some papers (e.g. [3][4]) propose deterministic design algorithms based on finite geometries and graph-theoretic algebra, while others [5][6][7] describe pseudo-random design techniques. We will focus on pseudorandom design algorithms, and propose an additional constraint that avoids some deleterious configurations of the parity-check matrix. We want to design LDPC codes intrinsically encodable in linear time: the minimum constraint imposed for this purpose is that the mxm rightmost part of the parity-check matrix has to be in lower triangular form. A more restrictive rule, that yields a simpler encoder, is the one specified in [8], adopted in [5] and justified in [9]: the rightmost part of the parity check matrix is a so-called “zig-zag pattern”. In the latter paper it is shown that such a constraint leads to negligible performance losses and the property of the matrices so obtained are discussed. In the rest of the paper we will focus on this particular structure of the parity-check matrix, known in the literature also as staircase construction.
0-7803-8938-7/05/$20.00 (C) 2005 IEEE
II. LDPC PERFORMANCE AND CYCLES IN TANNER GRAPHS
LDPC codes are decoded using the Belief Propagation (BP) algorithm [10], a two-step technique in which probabilities are exchanged between VNs and CNs. The performance of the code is directly related to the structure of its Tanner graph, as explained in [6] and [11], where the concepts of cycles, stopping sets and linearly dependent sets are introduced and clarified. The set of VNs set to 1 in a codeword corresponds to one or to the union of multiple linearly dependent sets. It can be shown that, at least in LDPC codes without singly connected VNs, all the linearly dependent sets are included in the ensemble of the stopping sets, which, in turn, are included in the ensemble of the cycles of the Tanner graph. Besides, it is known [10] that the BP algorithm does not compute marginal probabilities accurately on factor graphs with cycles. Therefore, some designs aim at maximizing the so-called girth of an LDPC code, i.e., the length of the shortest cycle of its Tanner graph. This is the approach followed by [5]: by bounding the length of cycles the authors indirectly bound the minimum size of linearly dependent sets, which is related to the minimum distance of the code. On the other hand, the authors of [6] aim at a more precise optimization, and indirectly try to bound the size of stopping sets, by avoiding short cycles with low values of the so-called extrinsic message degree (EMD). Actually, since it is difficult to compute the exact EMD, they consider an approximate cycle EMD (ACE), which neglects the effect of variable nodes sharing check nodes outside the cycle taken into account. Another technique aimed at the elimination of small stopping sets is described in [7]. A. Codewords, Near-codewords and Pseudo-codewords The design methods described in the previous paragraph are based on reasonable premises; nonetheless, it can happen that the LDPC codes so obtained, despite identical properties in terms of girth or ACE, show dramatic differences in terms of error-floor. A first analysis of these anomalies proved that, as explained also in [12], often the error floor of LDPC codes is not determined by the spectrum of the code, i.e., by the set of codewords, but by the so-called near-codewords. Near-codewords are binary n-tuples with low-weight (therefore probable error patterns) that violate a very small number of
647
check equations; if such an error event is generated by the channel, it can happen that the LDPC decoder gets stuck on this configuration and is not able to evolve towards a codeword, even if a large number of iterations is performed. Recently, the topic of near-codewords has been linked to the so-called Pseudo-Codewords in a paper [13] that tries to explain the behaviour of the BP algorithm. A more detailed analysis of this topic can be found in [14]. The importance of near-codewords is underlined by the analysis of the case study described in [15]: the authors show that, by simply moving a single edge, the error floor of an (50,30) LDPC code can be lowered dramatically, and attribute this improvement to the fact that the edge modification breaks a cycle of length 6. In Figure 1 we depict the structure of
3
13 6
25
25
16
16
29
30
3
20
29
20
3
16
16 2
(4,1) Near-Codeword
Fig. 1.
1032
1136
452-> 455
454
577
681
909
1033
1137
453
578
682 908
1034
453
578
576-> 579
680-> 682
Systematic bits
Parity-check bits
Codeword (d=10) 367
Codewords and error floor.
is made of few low-degree systematic bits whose ones are in very near rows, few parity check bits are needed to satisfy the related parity-check equations, because of the structure of the zig-zag pattern. If only two systematic bits are involved, for instance, the number of parity-check bits needed to obtain a codeword is exactly equal to the sum of the minimum index row differences between the ones in the two columns. This type of configurations eludes the ACE technique because the zig-zag pattern introduces very long chains in the Tanner graph that do not contribute to increase the EMD of the cycle.
5
13
30
910
1032-> 1136-> 1034 1137
28
28-> 30
6
680
Fig. 2.
nodes 25
5
576
908-> 910
16 20
6 13 Check 16
28
455
367
Check nodes
Variable nodes 3 5
3
(4,3) Near-Codeword
III. T HE DESIGN ALGORITHM
The case study of [15].
the columns of the parity-check matrix corresponding to the four code bits that are involved in the great majority of error events, and the cycles that involve them: the VNs and CNs are represented respectively by circles and squares that contain their indices. Actually, as illustrated in Figure 1, the high error floor is caused by a near-codeword with weight 4 and syndrome weight equal to 1 (the shaded unsatisfied check). By moving the edge that connects the 16-th CN and the 20th VN, the configuration is transformed into a near-codeword with syndrome weight equal to 3, which is significantly less likely to cause the decoder to get stuck on it. This analysis and the one in [12] show that a reasonable LDPC code design algorithm should take into account the minimum distance, i.e., the structure of codewords, and avoid near-codewords. B. Error configurations and zig-zag patterns Another shortcoming of other techniques, at least when they are applied to LDPC codes with the zig-zag pattern, is that they fail to avoid low-weight codewords associated with a low information weight. An example of this is shown in Figure 2. The case depicted in Figure 2 cannot be avoided by the PEG or the ACE algorithm: the codeword is made up of two cycles of length 16 and 14 with ACE values of 2. Since the codeword
Based on the observations of the previous section, we propose a design algorithm that comprises some constraints related to the cycle structure, i.e., to the position of edges in the columns, and eventually another constraint related to the VN degree distribution. A. Constraints on the cycle structure The first constraint is just a simplified version of the design criteria described in [6]: to guarantee a high EMD value for short cycles means fundamentally that short cycles should be composed of high-degree VNs. We do not retain the rather complicated rules of [6], but simply enforce two different values of girth for cycles made only of systematic VNs and for cycles that include also parity-check bits which, having only degree two, do not contribute to an increase of the EMD. These two values of girth will be referred to in the following as Gs and Gm , respectively. We will keep Gm > Gs , to make sure that the cycles with parity-check bits (i.e. with an average lower EMD) are longer than the ones made only of systematic bits. To avoid low-weight codewords, we test each new column added to the parity-check matrix as described also in [9]. We verify that the codewords obtained by encoding the single column and the sum of the tested column and of each of the already assigned columns, achieve a weight at least equal to a
648
threshold denoted by D1,2 . The complexity of this test roughly scales quadratically with the block length. Finally, a third constraint is introduced to get rid of lowweight near-codewords, which are not avoided by the previous test. We impose that at least MNDR ones of a column must be spaced at least of MRD rows with respect to the ones of another column (MNDR and MRD standing respectively for Minimum number of distant rows and Minimum row distance). The choice of the MNDR parameter is strictly dependent on the minimum degree of systematic VNs: if the latter is equal to 4, for instance, a good choice of MNDR could be 2. On the contrary, if systematic VNs have degrees as low as 3, it can be very difficult to successfully achieve a MNDR value of 2, and sometimes we have to choose MNDR equal to 1.
3
MNDR=2 MRD=5
Check nodes
367
K+100-> K+105
K+200-> K+205
K+300-> K+301
100-> 106
200-> 206
v 300-> 302
Systematic bits
Fig. 3.
Parity-check bits
systematic VNs produces on average an increase of the EMD of the shorter cycles. C. Statistical analysis To evaluate the effectiveness of the two classes of constraints described in the previous subsections, we perform a statistical analysis similar to the one described in [16] for turbo codes; we try to estimate the probability that some deleterious configurations (avoided by our constrained design) are picked in a random design of the parity-check matrix and the probability of other harmful configurations that our algorithm is unable to avoid. In the following we will use the symbols K and N to refer respectively to the information block length and to the codeword length, while dV N stands for the minimum degree of the systematic VNs. In the upper part of Figure 4, the typical configuration of a codeword associated with two columns is depicted. Given the configuration in column j, in a random extraction of the parity-check matrix, the probability that a “bad” in column k is chosen can be estimated dVconfiguration N (1 + 2∆m )(dV N !)/(N − K)dV N . Considering that as m=1 we should avoid this configuration not only for the index k but also for the other K columns and for all the possible values of the index j, then the probability of obtaining at least one occurrence of the said configuration can be estimated as being proportional to ((dV N !)/N dV N −2 )(Rc2 /2(1 − dV N dV N Rc ) ) m=1 (1+2∆m ). It must be noted that the minimum dV N distance corresponding to this configuration is 2+ m=1 ∆m . We can observe once again that raising the minimum degree of j
MNDR, MRD and near-codewords.
Systematic bits
The meaning of the MNDR and MRD constraints is exemplified in Figure 3: it can be easily seen that a value of MNDR=2 ensures that all the near-codewords with no more than two check equations failed, and with information weight equal to two, have weight at least equal to MRD+2. On the contrary, if MNDR=1, the protection against low-weight nearcodewords is weaker.
k
D2
D1
D3
Check nodes
p Systematic bits
q r
B. Constraints on the degree distribution
D1
When strict requirements on the error floor must be met, it can be useful to insert additional constraints on the VN degree distribution. As already observed in [9], the error floor can be lowered dramatically if the minimum degree of the systematic VNs is increased; this, on the other hand, causes the average CN degree to be higher, thus worsening the convergence threshold of the obtained code. Nonetheless, this penalty can be limited to 0.1–0.2 dB by a careful constrained design of the degree distributions by means of the Density Evolution method described in [2]. It is worth observing that the improvement due to this constraint can be explained also as a result of the theoretic analysis of [6]: in fact, an increase of the minimum degree of
D2
Fig. 4.
D3
D4 Check nodes
D5
D6
Codewords configuration.
systematic VNs statistically improves the minimum distance; this type of configuration, anyway, can be avoided with the specific constraints that we described in the previous subsections. In the lower part of Figure 4, on the contrary, we depict one of the worst configurations that can be picked up by the semi-random algorithm, and that are not avoided by our design constraints even if MNDR is equal to 2. Fortunately,
649
the probability of obtaining this sequence scales very nicely with the block length. It can be shown that this configuration is chosen with aprobability estimated as proportional to 6 (Rc3 /(1 − Rc )6 )( m=1 (1 + 2∆m ))/(N 3 ). Due to the odd numbers of edges involved, codewords from three systematic VNs of degree 3 can be obtained only if all the last ∆end CNs are set to 1, as shown in Figure 5. With the usual procedure it can be shown that this type of configuration, avoided only if MNDR is equal to 2, is chosen with aprobability estimated as 4 proportional to (Rc3 /(1−Rc )5 )∆end ( m=1 (1+2∆m ))/(N 2 ).
p Systematic bits
q r D2
D1
Fig. 5.
D3
Dend
D4 Check nodes
V. R ESULTS AND C OMPARISONS
Codewords configuration.
Similar considerations can be made for near-codewords; for example, as far as near-codewords with one failed check are concerned, two harmful configurations are depicted in Figure 6, which can be chosen with a probability propordV N −1 (1 + tional respectively to (Rc2 /(1 − Rc )dV N )∆end ( m=1 5 2∆m ))/(N dV N −2 ) and (Rc3 /(1 − Rc )6 )∆end ( m=1 (1 + 2∆m ))/(N 3 ). While the first configuration can be avoided by setting MNDR=2, the second one eludes that control. j Systematic bits
Failed check
k
D2
D1
Dend
Check nodes
in [17], the designer has to avoid that the fraction of check nodes affected by more than one erased bit grows excessively large. In this case, in fact, it is highly likely that the puncturing pattern causes all the bits of a Stopping set to be erased, thus making impossible to reach error-free decoding [11]. For a generic parity-check matrix this rule implies a careful design of the puncturing patterns. On the contrary, the zig-zag pattern permits to obey the aforementioned rule very easily: it is sufficient to puncture the parity bits in an alternate pattern, until the desired rate is achieved. By puncturing only the parity bits, which have degree two, we also achieve the highest flexibility in terms of code-rate. In fact, with this technique, a code of rate Rc can be punctured up to a rate of 2Rc /(1 + Rc ); with only three matrices it is possible to cover every code rate in the range 0.33-0.80. We will show in the next section that puncturing, if the above stated conditions are satisfied, leads to small performance loss.
Failed check
To verify the validity of the design algorithm, we designed a (603,301) LDPC code with parameters Gs = 6, Gm = 8, D1,2 = 10, MNDR=2, MRD=8, and with 2047 edges and minimum systematic VN degree equal to 4, and a (901,301) code, successively punctured to the same code-rate, with parameters Gs = 6, Gm = 8, D1,2 = 12, MNDR=2, MRD=10, with 3059 edges and minimum systematic VN degree equal to 4, and compared them to the codes described in [7]. The VN degree distributions, expressed in terms of node perspective, of our codes and of the ones in [7], are λ(x) = 0.0017 + 0.4992x + 0.3997x3 + 0.0995x7 , λ(x) = 0.0011 + 0.6648x + 0.2431x3 + 0.0910x11 , and λ(x) = 0.4925x + 0.2206x2 + 0.1526x4 + 0.0099x5 + 0.0332x6 + 0.0912x19 , respectively. Our codes were simulated with 80 iterations. We have also drawn the ML bound for the asymptotic BER and FER, based on the minimum distance estimated with the technique described in [18]; this algorithm measured a minimum distance equal to 15 and 14, respectively.
p 0
10
Systematic bits
q
−2
r
10
D2
Fig. 6.
D3
D4 Check nodes
D5
Dend −4
10
FER ; BER
D1
Near-codewords configuration.
−6
10
−8
10
This analysis shows that the configurations eluding our constraints are very unlikely to be chosen by the semi-random design algorithm.
−10
10
−12
IV. VARIABLE - RATE LDPC CODES
10
The rate of LDPC codes can be changed by puncturing; the puncturing patterns, however, have to be designed very carefully, in order to avoid dramatic losses. As explained
650
0
Tian et. al, FER Tian et. al, BER SSC, FER SSC, BER Joint, FER Joint, BER (603,301) 2047 edges, FER (603,301) 2047 edges, BER (901,301) 3059 edges, FER (901,301) 3059 edges, BER (603,301) 2047 edges, bound FER (603,301) 2047 edges, bound BER (901,301) 3059 edges, bound FER (901,301) 3059 edges, bound BER 0.5
1
1.5
2 2.5 E /N [dB] b
Fig. 7.
3
0
Comparison with [7].
3.5
4
4.5
0
10
showed that a great flexibility in terms of code-rate can be achieved by storing only a very limited number of codes. To prove the validity of our approach, results were given in terms of simulated error probabilities compared with codes proposed in the literature.
−1
10
−2
FER
10
ACKNOWLEDGMENT This work was supported by MIUR (Italian Ministry of Education and Research) under the project CERCOM (Center for Multimedia Radio Communications) and PRIMO (Reconfigurable platforms for Wideband Wireless Communications).
−3
10
−4
10
−5
10
[9], (4161,3430), wc=3 [9], (4161,3430), wc=4 (4200,3360), punct. 4088, Ned=16799 (4200,3360), punct. 4088, N =17519 ed
−6
10
2.4
R EFERENCES
2.6
2.8
3
Fig. 8.
3.2 Eb/N0 [dB]
3.4
3.6
3.8
4
Comparison with [9].
Figure 7 shows that our codes exhibit superior performance, especially at high SNR values; at a FER of 10−6 almost 1 dB is gained. It has to be remarked that the punctured code offers almost the same performance of the un-punctured one. Finally, we performed another simulation to assess the importance of the constraints related to the VN degree distribution and of the ones related to the cycle structure. We generated two (4200,3360) codes, punctured to a block length of 4088, with minimum systematic VN degree equal to 3 and 4 and number of edges equal to 16799 and 17519, and compared them to codes of [9] with similar code-rate and block size, with different values of minimum systematic VN degree (referred in the picture as wc ). The VN degree distributions, expressed in terms of node perspective, of the four codes are λ(x) = 0.0002 + 0.1998x + 0.6286x2 + 0.1714x9 , λ(x) = 0.0002 + 0.1998x + 0.7048x3 + 0.0952x9 , λ(x) = 0.0002 + 0.1754x + 0.6794x2 + 0.0904x6 + 0.0546x7 and λ(x) = 0.0002 + 0.1754x + 0.8244x3 , respectively. The parameters of our two codes are Gs = 4, Gm = 6, D1,2 = 10, MNDR=1, MRD=20, and Gs = 4, Gm = 6, D1,2 = 10, MNDR=2, MRD=8, respectively. The simulation results confirm again that increasing the minimum systematic VN degree produces a lower error floor, but, besides, for a given value of the said parameter, it is possible to appreciate a significant difference, in terms of error floor between our codes and the ones from [9], that can be only explained by the constraints related to the cycle structure. VI. C ONCLUSIONS In this paper we described a semi-random design technique for the design of irregular LDPC codes endowed with the zig-zag pattern, based on a constraint on the VN degree distribution and on a set of constraints on the position of the edges in the columns of the parity-check matrix. This technique avoids at the same time low-weight codewords and low-weight near-codewords, the main cause of error floors. Finally, we provided an effective, if trivial, rule for the design of puncturing patterns for this class of codes, and
[1] R. G. Gallager, Low density parity check codes, MIT Press, Cambridge, MA, 1963. [2] T. J. Richardson, M. A. Shokrollahi, and R. L. Urbanke, “Design of capacity-approaching irregular low-density parity-check codes”, IEEE Transactions on Information Theory, vol. 47, no. 2, Feb. 2001, pp. 619– 637. [3] Y. Kou, S. Lin, and M. P. C. Fossorier, “Low-density parity-check codes based on finite geometries: a rediscovery and new results”, IEEE Transactions on Information Theory, vol. 47, Nov. 2001, pp. 2711-2736. [4] J. Rosenthal, and P. O. Vontobel, “Constructions of LDPC Codes Using Ramanujan graphs and Ideas from Margulis”, Proc. of the 38-th Allerton Conference on Communication, Control, and Computing, Monticello, Illinois (USA), Oct. 4-6 2000, pp. 248-257. [5] X.-Y. Hu, E. Eleftheriou, and D. M. Arnold, “Progressive edge-growth Tanner graphs”, Proc. IEEE Global Telecommun. Conf., vol. 2, San Antonio, Texas (USA), November 2001, pp. 995–1001. [6] T. Tian, C. Jones, J. D. Villasenor, and R. D. Wesel, “Construction of Irregular LDPC Codes with Low Error Floors”, Proc. IEEE International Conference on Communications, vol. 5, May 2003, pp. 3125–3129. [7] A. Ramamoorthy, and R. Wesel, “Construction of Short Block Length Irregular Low-Density Parity-Check Codes”, Proc. IEEE International Conference on Communications, vol. 1, June 2004, pp. 410–414. [8] L. Ping, W. K. Leung, and N. Phamdo, “Low density parity check codes with semi-random parity check matrix”, Electronics Letters, vol. 35, no. 1, Jan. 1999, pp. 38–39. [9] M. Yang, W. E. Ryan, and Y. Li, “Design of Efficiently Encodable Moderate-Length High-Rate Irregular LDPC Codes”, IEEE Trans. on Comm., vol. 52, no. 4, April 2004, pp. 564–571. [10] D. J. C. MacKay, “Good error-correcting codes based on very sparse matrices”, IEEE Transactions on Information Theory, vol. 45, Mar. 1999, pp. 399–431. [11] C. Di, D. Proietti, E. Telatar, T. Richardson, and R. Urbanke, “Finite length analysis of low-density parity-check codes on the binary erasure channel”, IEEE Transactions on Information Theory, vol. 48, no. 6, June 2002, pp. 1570–1579. [12] D. MacKay, and M. S. Postol, “Weaknesses of Margulis and RamanujanMargulis Low-Density Parity-Check Codes”, Electronic Notes in Theoretical Computer Science, vol. 74, 2003. [13] R. Koetter, and P. O. Vontobel, “Graph-covers and iterative decoding of finite length codes”, Proc. 3rd International Symposium on Turbo Codes and Related Topics, Brest, France, 2003, pp. 75–82. [14] B. J. Frey, R. Koetter, and A. Vardy, “Signal-Space Characterization of Iterative Decoding”, IEEE Transactions on Information Theory, vol. 47, no. 2, Feb. 2001, pp. 766–781. [15] S. H. Lee, W. H. Lee, S. J. Bae, S. I. Lee, and E. K. Joo, “Bit probability transition characteristics of LDPC code”, 10th International Conference on Telecommunications, Tahiti, French Polynesia, Feb.–Mar. 2003, pp. 553–557. [16] S. Dolinar, and D. Divsalar, “Weight distribution for Turbo Codes Using Random and Non random Permutations”, TDA Progress Report 42-122, August 15 1995. [17] J. Li, and K. Narayanan, “Rate-compatible Low Density Parity Check Codes for Capacity-Approaching ARQ Scheme in Packet Data Communications”, Int. Conf. on Comm. Internet and Info. Tech., Nov. 2002. [18] X.-Y. Hu, M. P. C. Fossorier, and E. Eleftheriou, “On the computation of the Minimum Distance of Low-Density Parity-Check Codes”, Proc. IEEE International Conference on Communications, vol. 2, June 2004, pp. 767–771.
651
