Ensemble Trapping Set Enumerators for Protograph-Based LDPC Codes Shadi Abu-Surra
William E. Ryan
Dariush Divsalar
University of Arizona Email:
[email protected]
University of Arizona Email:
[email protected]
Jet Propulsion Laboratory Email:
[email protected]
Abstract—Protograph-based LDPC codes have the advantages of a simple design (or search) procedure and highly structured encoders and decoders. Recently, techniques for finding asymptotic trapping set enumerators in regular and irregular LDPC code ensembles was published [1]. In this paper, finite-length ensemble trapping set enumerators for protographbased LDPC codes are obtained. The asymptotic results are derived from the finite-length results by letting the block length go to infinity. The asymptotic results allow us to determine whether or not the typical smallest trapping set size grows linearly with codeword length.
I. I NTRODUCTION Graphical configurations associated with so-called Trapping sets in the Tanner graph of an LDPC code are known to create an error-rate floor in the performance curves for iterative decoders [1]–[5]. In [2], Richardson established the notion of trapping sets and suggested a two-phase technique for predicting the performance of LDPC codes in the errorfloor region. The first phase searches for trapping sets and their multiplicities (e.g., by simulation) and the second phase evaluate their contribution to the error floor. The authors of [3], [4] used a similar technique to predict the floor of several LDPC codes on the binary symmetric channels. In [1], the authors derive the average asymptotic behavior of trapping set enumerators over random, regular and irregular, LDPC code ensembles. In this paper, we leverage our earlier work for evaluating weight and stopping set enumerators [6]–[8] to compute the trapping set enumerators for protograph-based LPDC codes. We consider both finite-length ensembles and infinite-length ensembles. Together with Richardson’s performance evaluation technique, these results allow one to estimate the average performance of LDPC codes in an ensemble characterized by a protograph. Moreover, simulation results in [2]–[5] showed that small trapping sets dominates the error-floor level. In this work, we show how to use the asymptotic enumerator results to determine whether or not the typical smallest trapping set size grows linearly with codeword length, i.e., whether or not the ensemble avoids small trapping sets for fairly long codes. Following [1], we derive both general and elementary trapping set enumerators, but [1] does not consider protograph-based LDPC code ensembles. We define general and elementary trapping sets in Sections III and IV, respectively.
Fig. 1.
Vectorized protograph.
The paper is organized as follows: In the next section a review of the method of finding weight enumerators for GLDPC code ensembles is presented. In Section III, general trapping set enumerators is evaluated, and in Section IV enumerators are found for the elementary trapping set case. Numerical results are presented in Section V. Finally, the paper is concluded in Section VI. II. W EIGHT E NUMERATORS FOR P ROTOGRAPH -BASED LDPC C ODE E NSEMBLES The weight enumerators formulas in [7], [8] are used in this paper to compute trapping set enumerators. Consequently, in this section we present a review of our method for finding finite and asymptotic weight enumerators for protograph-based LDPC code ensembles. (As in [7], [8], the results in this section apply to generalized LDPC codes.) Recall that a protograph [9], [10] is a relatively small bipartite graph, containing variable nodes (VNs) and constraint nodes (CNs), from which a larger graph can be obtained by a copy-and-permute procedure: the protograph is copied N times, and then the edges of the individual replicas are permuted among the replicas to obtain a single, large graph. The copy-and-permute process can be simply represented by replacing each node with a vector of nodes of the same type and replacing each edge with a bundle of (permuted) edges of the same type. This “vectorized” protograph is depicted in Fig. 1.
A. Finite-length ensemble weight enumerators Following the notation of [6]–[8], consider an LDPC protograph, G = (V,C,E), where V = {v1 ,v2 ,. . . , vnv } is the set of nv VNs, C = {c1 ,c2 ,. . . ,cnc } is the set of nc CNs, and E is the set of edges. Denote by qvi (qcj ) the degree of variable (constraint) node vi (cj ). Now consider the LDPC code constructed from a protograph G by making N replicas of G and using uniform interleavers, each of size N , to permute the edges among the replicas of the protograph. In order to exploit the results in [11], [12], we treat the VNs and CNs as constituent codes in a concatenated coding scheme. More specifically, the group of N VNs of type vi is considered to be a constituent (repetition) code with a weightdi input of length N and qvi length-N outputs. Also, the group of N CNs of type cj is considered to be a constituent code with qcj inputs, each of length N , and a fictitious output of weight zero. Let A(d) be the average (over the ensemble) number of codewords having weight vector d = [d1 , d2 , . . . , dnv ] corresponding to the nv VN N -groups and satisfying the protograph constraints. A(d) is the vector weight enumerator for the ensemble of codes of length N · nv described by the protograph. Let us further define N v • A i (wi ) = d δdi ,wi,1 · · · δdi ,wi,qv = the vector weight i i enumerator for the type-vi (VN) constituent code for a weight-di input, where wi = wi,1 , wi,2 , ..., wi,qvi is a weight vector describing the constituent code’s output, and c for the type-cij • A j (zj ) = the vector weight enumerator h (CN) constituent code and zj = zj,1 , zj,2 , ..., zj,qcj ,
where zj,l = wi,k if the lth edge of CN cj is the k th edge of VN vi . As shown in [6, Eq. 2], by exploiting the uniform interleaver property, we may write Q c Q v vi X ni=1 Acj (zj ) A (wi ) nj=1 A(d) = Qnv Qqvs N =
s=1 wm,u Qnc cj j=1 A (dj ) Qnv N qvi −1 i=1 di
r=1 ws,r
(1)
where the summation in the first line is over all weights hwm,u , m = 1, i..., nv and u = 1, ..., qvm , and dj = dj1 , dj2 , ..., djqcj is a weight vector which describes the weights of the N -bit words on the edges connected to CN cj , produced by the VNs neighboring cj . The elements of dj comprise a subset of the elements of d. Let St be the set of transmitted VNs and let Sp be the set of punctured VNs. Then the average number of codewords of weight d in the ensemble, denoted by Ad , equals the sum of P A(d) over all d for which {di :vi ∈St } di = d. Notationally, X X Ad = A(d) (2) {di :vi ∈St } {dk :vk ∈Sp }
P under the constraint {di :vi ∈St } di = d. To evaluate Ad in (2), one first needs to compute the vector weight enumerators,
Acj (dj ), for the constraint nodes cj , as seen in (1). Consider the constituent (µ, κ) linear block code C. For standard LDPC codes, C is a single parity-check (SPC) code and µ = κ + 1. Let MC be the K × µ matrix with the codewords of C as its rows, where K = 2κ is the number of codewords in C. In [7], [8] we showed that the vector weight enumerator AC (w) is given by X C (N ; n1 , n2 , . . . , nK ) , (3) AC (w) = {n}
where w = [w1 , w2 , . . . , wµ ] is the weight vector at the input to the constituent code, {n} is the set of integer P solutions to w = n · MC , with n1 , n2 , . . . , nK ≥ 0 and K k=1 nk = N . In (3) C(N ; n1 , n2 , . . . , nK )’s are multinomial coefficients. B. Asymptotic ensemble weight enumerators In the asymptotic case, let us define the normalized logarithmic asymptotic weight enumerator (we will simply call it the asymptotic weight enumerator), r(δ), as ln Ad , (4) n where δ = d/n (recall n is the number of transmitted variable nodes in the code). Following [6], because the formulas in the previous section involve the number of copies, N , instead of n, we define the function r(δ) = lim sup n→∞
˜ = lim sup r˜(δ) N →∞
ln Ad , N
(5)
where δ˜ = d/N . Note that n = |St | · N and so r(δ) =
1 r˜(|St | · δ). |St |
(6)
˜ [7], [8] is where r˜(δ)
˜ = r˜(δ)
max
max
nc X
{δ˜l :vl ∈St } {δ˜k :vk ∈Sp } j=1
−
nv X
acj (δ˜j ) ))
(qvi − 1)H(δ˜i )
i=1
(7) ,
P ˜ In (7), acj (δ˜j ) under the constraint {δ˜i :vi ∈St } δ˜i = δ. is the asymptotic vector weight enumerator of the constraint node cj , H(δ˜i ) = −(1 − δ˜i ) ln(1 − δ˜i ) − δ˜i ln δ˜i , and δ˜j = dj /N . For a generic constituent CN code C, this enumerator is defined as ln AC (w) , N →∞ N where ω = w/N , and is evaluated [7], [8] as aC (ω) = lim sup
(8)
aC (ω) = max {H(Pω )} , (9) {Pω } under the constraint that {Pω } is the Pset of solutions to ω = Pω · MC , p1 , p2 , . . . , pK ≥ 0 and K k=1 pk = 1. These are the asymptotic equivalents of the constraints mentioned
below (3). Note that, Pω = [p1 , p2 , . . . , pK ] is the empirical probability distribution of the codewords in C given a sequence of N such codewords, where pk = nk /N and nk is the number of occurrences of the k th codeword. Let dmin be the minimum distance of a linear block code. In [13] it was shown that nδ nδ X X n Pr{dmin ≤ nδ} ≤ P (l) = Al , (10) l l=1
l=1
where P (l) is the probability that a particular sequence of weight l is a codeword in the ensemble. Let δmin be the second zero crossing of r(δ) (the first crossing is r(0) = 0). Assuming δmin exists, it is called the typical minimum distance if r(δ) < 0 for all 0 < δ < δmin Pnδ min [6]. If Al → Anδmin as n becomes large, then l=1 Pnδ −1 Pr{dmin ≤ nδmin − 1} ≤ l=1min Al → 0, so that with probability near one, dmin ≈ nδmin . That is, the minimum distances of virtually all of the members of the code ensemble increase linearly with n. Selected r(δ) and δmin results for G-LDPC codes may be found in [7], [8]. III. G ENERAL T RAPPING S ET E NUMERATORS Definition 1: An (a, b) general trapping set, Ta,b , is a set of VNs of size a which induce a subgraph with exactly b odddegree check nodes (and an arbitrary number of even-degree check nodes).
Fig. 2.
not distinguish between transmitted and punctured VNs (see Definition 1). However, in evaluating the failure rate of a trapping set, one should make this distinction, but this is not considered in this paper. Based on the above discussion and (2); the trapping set enumerator Aa,b is given by X
Aa,b =
A. Finite-size trapping set enumerators Our method for determining trapping set enumerators for an LDPC code ensemble characterized by a given protograph is similar to our strategy used to find stopping set enumerators in [8]. That is, we create a modified protograph from the original protograph and then determine the weight enumerator for the modified protograph, from which the trapping set enumerators may be obtained. We describe the technique as follows. Assume we are interested in trapping sets of weight a in the graph G in Fig. 1. The value of the companion parameter b depends on which a VNs are of interest, and so we set the values of the a VNs of interest to one and the values of the remaining VNs to zero. With the a VNs so fixed, we are now interested in which CNs “see” odd weight among its neighboring VNs, for the number of such CNs is the corresponding parameter b. Such odd-weight CNs can be identified by the addition of auxiliary “flag” VNs to each CN (see protograph G′ in Fig. 2), where the value of an auxiliary VN equals one exactly when its corresponding original CN sees odd weight. The number of auxiliary VNs equal to one is precisely the parameter b for the set of a VNs that were set to one. Thus, to obtain the trapping set enumerator for the original protograph G, one may apply the weight enumerator technique to G′ , partitioning the set VNs into the original set (St ∪Sp ) and auxiliary flag set (Sf ), much like we had earlier partitioned transmitted and punctured VNs. Note that the set St ∪ Sp (bottom of Fig. 2) accounts for the weight a and the set Sf (top of Fig. 2) accounts for the weight b. Also note that, in counting the number of trapping sets, we do
Modified vectorized protograph.
X
A(d)
(11)
{di :vi ∈St ∪Sp } {dk :vk ∈Sf }
Punder the constraint {di :vi ∈Sf } di = b, where
P
{di :vi ∈St ∪Sp }
Qnc
j=1
A(d) = Q nv
di
=
a and
′
Acj (dj ) . N qvi −1
(12)
i=1 di
c′j
Notice the use of instead of cj in (12) to indicate that the vector weight enumerators in (12) are the CNs in G′ . Those vector weight enumerators can be evaluated using (3). B. Asymptotic trapping set enumerators As in Section II, define the normalized logarithmic asymptotic trapping set enumerator (we will simply call it the asymptotic trapping set enumerator), r(α, β), as r(α, β) = lim sup n→∞
ln Aa,b , n
(13)
where α = a/n and β = b/n (recall n = |St |.N is the code length). The derivation of an expression for (13) from (11) uses the same steps used in deriving (6) from (2) (see [7], [8]). Thus, we omit the derivation and present the final result: r(α, β) = where
1 r˜(|St | · α, |St | · β), |St |
(14)
˜ = r˜(˜ α, β)
max
max
nc X
{δ˜l :vl ∈St ∪Sp } {δ˜k :vk ∈Sf }
−
nv X
j=1
′ acj (δ˜j )
)) (15)
(qvi − 1)H(δ˜i )
i=1
,
P ˜ = α, under the constraint ˜ and {δ˜i :vi ∈St ∪Sp } δi P ˜ ˜ {δ˜i :vi ∈Sf } δi = β. The asymptotic vector weight enumera′ tor, acj (δ˜j ), can be evaluated using (9). (To avoid ambiguity, note the use of ac (.) to refer to the asymptotic vector weight enumerator, and the use of a to refer to the size of the (a, b) trapping set). To establish an analogy to the typical minimum distance in the weight enumerator problem, let ∆ = b/a = β/α, ∆ ∈ [0, ∞). Then classify the trapping sets as T∆ = {Ta,b : b = ∆ · a}
(16)
(ts)
Now, for each ∆, define dmin (∆) to be the ∆-trapping set number, which is the size of the smallest, non-empty, (ts) trapping set in T∆ . Further, let δmin (∆) be the typical ∆trapping set number, which is the value of α at the second zero-crossing of the r(α, ∆ · α) versus α curve (r(0, 0) is the first zero-crossing). Note that, from this definition, as ∆ (ts) (ts) ranges from 0 to ∞, the points (δmin (∆), ∆ · δmin (∆)) trace out what we call the zero-contour curve: {(α, β) : r(α, β) = (ts) 0}. As for the weight enumerator problem, if δmin (∆) exists, then the average ∆-trapping set number over the code ensem(ts) (ts) ble increases linearly with n (or, dmin (∆) ≈ nδmin (∆)). Thus, we are interested in codes with large ∆-trapping set numbers because small trapping sets dominate the iterative decoding performance in the error-floor region. Fig. 3 shows the zero-contour curves for two different hypothetical ensembles. The solid curve and the dashed curve corresponds to Ensemble 1 and Ensemble 2, respectively. We see in Fig. 3 the utility of the ordered pair (α, β) = (α, ∆·α). The line containing these points intersect a given zerocontour curve exactly when the abscissa of the intersection (ts) point is δmin (∆). Note, for any given ∆, the ∆-trapping set number for Ensemble 2 is greater than that for Ensemble 1, a consequence of the fact that the Ensemble 2 zerocontour curve resides above the Ensemble 1 zero-contour curve. Therefore, a code drawn from Ensemble 2 is expected to have a lower floor than a code drawn from Ensemble 1. IV. E LEMENTARY T RAPPING
Fig. 3. The zero-contour curves of the asymptotic trapping set enumerators for two different ensembles. The zero-contour curves correspond to Ensemble 1 and Ensemble 2 are the solid and the dashed curves, respectively
SET ENUMERATORS (e)
Definition 2: An elementary (a, b) trapping set, T(a,b) , is a set of a VNs which induce a subgraph with only degreeone and degree-two check nodes, and exactly b of them has degree one. It was observed that in the error-floor region, after running extensive Monte Carlo simulations [2]–[5], most of the error patterns correspond to elementary trapping sets. Some examples of elementary trapping sets are the (12, 4) trapping
set in the (2640, 1320) Margulis code, and the (4, 4) and (5, 3) trapping sets in the (1008, 504) and (816, 408) MacKay codes [4]. As a result, it is desired to compute the enumerators just for the elementary trapping sets in the code ensembles. To find elementary trapping set enumerators, note that the only difference between the elementary and the general trapping set is the constraints on the degrees of the CNs in the induced subgraph. But this can be taken care of by choosing a proper matrix MC in (3) and (9) when evaluating the vector weight enumerators in (12), and the asymptotic vector weight enumerators in (15), respectively. Consequently, all the discussion regarding general trapping set enumerators is valid for elementary trapping set enumerators after redefining MC . To find MC for the case of elementary trapping sets, note that when the bits of an elementary trapping set are set to 1 with all other bits in G set to 0, the CNs in G can see only certain bit patterns. Specifically, the set of patterns that can be seen by a degree-qc CN c in G are the all-zeros word, the q1c weight-1 words, and the q2c weight-2 words. This implies that the possible patterns for its corresponding degree-(q c +1) CN c′ in G′ are the all-zeros pattern and the qc2+1 patterns of weight two. Therefore, MC contains the all-zeros pattern and all possible combinations of weight-2 patterns. V. N UMERICAL
RESULTS
In this section trapping set enumerator results are presented for several rate-1/2 LDPC code ensembles which have the protographs of Fig. 4. In Fig. 5 through Fig. 8, asymptotic trapping set enumerators for the regular (6, 3) LDPC code are presented. Fig. 5 presents a three-dimensional plot of the asymptotic trapping set enumerator r(α, β). More illuminating is the two-dimensional plot of r(α, β) in Fig. 6 which is parameterized by β. Note that, by our set-up in Section III, the β = 0 curve corresponds to the weight-enumerator for the original protograph (since this implies b = 0). Observe that
0.7 0.6
r(α, β)
0.5
Fig. 4. Protographs for several rate-1/2 LDPC codes: (a) (6, 3) regular LDPC code. (b) Precoded (6, 3) regular LDPC code. (c) RJA LDPC code. (d) ARJA LDPC code. (Note: The shaded VNs represent punctured nodes.)
VI. C ONCLUSION In this paper a method for finding protograph ensemble trapping set enumerators is derived for both the finite and infinite code lengths. The method is conceptually simple and leverages our earlier weight enumeration work by transforming the trapping set enumerator problem into a weight enumerator problem. Moreover, the same method can be easily used to evaluate trapping set enumerators for generalized
0.3 0.2 0.1 0
β
0.2 0.1
0.2
0.1
0
α
0.3
0.4
0.5
Fig. 5. Asymptotic trapping set enumerators for (6, 3) regular LDPC code ensemble.
0.08 0.07 0.06 0.05
r(α, β)
in the figure, with fixed α, r(α, β) increases with increasing β. This indicates that, for this ensemble, the number of near codewords of a given weight exceeds that of the true codewords of the same weight. In Fig. 7, r(α, ∆ · α) is evaluated for several values of ∆. Note in Fig. 7, for a given ∆, the first zero-crossing is at r(0, 0) and, if there is a second zero-crossing, then a typical ∆-trapping set number exists. Observe that the typical ∆-trapping set numbers decrease as ∆ increases. This demonstrates the benefit of partitioning the trapping sets into the sets T∆ . In Fig. 9 we compare the regular (6, 3) protograph and the precoded (6, 3) protograph by drawing the zero-contour curves of their asymptotic trapping set enumerators. The figure shows that the precoded (6, 3) LDPC code ensemble has larger typical ∆-trapping set numbers for the values of ∆ between 0 and 0.1674 (for α > 0.003). The zerocontour curve of the RJA protograph also appears in Fig. 9 to demonstrate the fact that the (6, 3) protograph outperforms the RJA protograph in the sense of having higher typical ∆trapping set numbers for all ∆. The zero-contour curves for the RJA and the ARJA ensembles are presented in Fig. 10. It is observed that the ARJA code ensemble has larger typical ∆-trapping set numbers for the values of ∆ between 0 and 0.05 (for α > 0.0043). Fig. 8 shows the asymptotic elementary trapping set enumerators for the (6, 3) protograph. We found that the zerocontour curves for the asymptotic elementary trapping set enumerators are approximately the same as those for the general trapping set enumerators and so we omit these. The reason they are approximately the same is that, for small values α and β, the majority of the trapping sets are of the elementary type.
0.4
0.04 0.03 0.02
As β increases
0.01 0 −0.01 0
β=[0, 2e−4, 4e−4, 5e−4, 1e−3, 2e−3, 3e−3, 5e−3, 1e−2]
0.02
0.04
α
0.06
0.08
0.1
Fig. 6. Asymptotic trapping set enumerators as a function of α with fixed, several values, β. The ensemble is the (6, 3) regular LDPC code.
LDPC codes. Also in this paper, the parameter typical ∆trapping set number is created and then used to compare various LDPC code ensembles. R EFERENCES [1] O. Milenkovic, E. Soljanin, and P. Whiting, “Asymptotic spectra of trapping sets in regular and irregular LDPC code ensembles,” IEEE Trans. on Inform. Theory, vol. 53, pp. 39–55, January 2007. [2] T. Richardson, “Error-floors of LDPC codes,” in Proc. of the 41th Annual Allerton Conf. on Commun., Control, and Computing, Illinois, September 2003. [3] S. K. Chilappagari, S. Sankaranarayanan, and B. Vasic, “Error floors of LDPC codes on the binary symmetric channel,” in IEEE Int. Conf. on Commun., ICC ’06, June 2006. [4] S. Sankaranarayanan, S. K. Chilappagari, R. Radhakrishnan, and B. Vasic, “Failures of the gallager B decoder: Analysis and applications,” in UCSD Workshop on Information Theory and Its Applications, February 2006. http://ita.ucsd.edu/workshop/06/talks/papers/160.pdf.
−3
0.05
1.4
Regular (6,3) LDPC Precoded (6,3) LDPC RJA
1.2
0.03
1
0.02
0.8
β
r(α, ∆ . α)
0.04
x 10
0.01
0.6
As ∆ increases
0.4
0 ∆=[0, 0.005, 0.025, 0.1, 0.2, 0.5, 1.0]
−0.01 0
0.01
0.02
0.03
α
0.04
0.05
0.2 0.06
0 0
Fig. 7. Asymptotic trapping set enumerators for different T∆ in the (6, 3) regular LDPC ensemble.
0.01
0.02
α
0.03
0.04
0.05
Fig. 9. The zero-contour curves of the asymptotic trapping set enumerators for different protograph-based LDPC code ensembles.
0.45
−4
3
0.4
x 10
RJA ARJA
0.35
2.5 2
0.25 0.2
β
r(e)(α, β)
0.3
0.15 0.1
1.5 1
0.05
0.5
0
β
0.2 0.1
0
0.05
0.1
α
0.15
0.2
0.25
0.3
Fig. 8. Asymptotic elementary trapping set enumerators for the (6, 3) regular LDPC code ensemble.
[5] S. Landner and O. Milenkovic, “Algorithmic and combinatorial analysis of trapping sets in structured LDPC codes,” in IEEE Int. Conf. on Wireless Networks, Communications and Mobile Computing, pp. 630– 635, June 2005. [6] D. Divsalar, “Ensemble weight enumerators for protograph LDPC codes,” in IEEE Int. Symp. on Inform. Theory, pp. 1554–1558, July 2006. [7] S. Abu-Surra, W. E. Ryan, and D. Divsalar, “Ensemble weight enumerators for protograph-based generalized LDPC codes,” in UCSD Workshop on Information Theory and Its Applications, February 2007. http://ita.ucsd.edu/workshop/07/files/paper/paper 218.pdf. [8] S. Abu-Surra, W. E. Ryan, and D. Divsalar, “Ensemble enumerators for protograph-based generalized LDPC codes,” in IEEE Global Telecomm. Conf., GLOBECOM ’07, 2007. submitted to.
0 0
0.005
0.01
α
0.015
0.02
Fig. 10. The zero-contour curves of the asymptotic trapping set enumerators for the RJA and the ARJA code ensembles.
[9] S. Lin, J. Xu, I. Djurdjevic, and H. Tang, “Hybrid construction of LDPC codes,” in Proc. of the 40th Annual Allerton Conf. on Commun., Control, and Computing, Illinois, October 2002. [10] J. Thorpe, “Low-density parity-check (LDPC) codes constructed from protographs,” Tech. Rep. 42-154, IPN Progress Report, August 2003. [11] S. Benedetto, D. Divsalar, G. Montorsi, and F. Pollara, “Serial concatenation of interleaved codes: Performance analysis, design, and iterative decoding,” IEEE Trans. on Inform. Theory, vol. 44, pp. 909–926, May 1998. [12] D. Divsalar, H. Jin, and R. McEliece, “Coding theorems for “turbolike” codes,” in Proc. of 36th Allerton Conf., September 1998. [13] R. G. Gallager, Low-density parity-check codes. Cambridge, MA: MIT Press, 1963.
