A Class of Group-Structured LDPC Codes R. Michael Tanner Computer Science Department University of California, Santa Cruz, CA 95064 U.S.A.
[email protected]
Abstract A class of graphs is designed using subgroups of the multiplicative group of a prime field GF(p). Instances where (p-1) is divisible by 3 and 5 are used to construct quasi-cyclic LDPC codes with bit degree 3 and parity degree 5, among them a [155,64,20] code. While the girth of a graph in this class cannot be greater than twelve, for many practical lengths the graphs have a relatively large girth and a lowdensity of short cycles. Simulation studies show that at short to moderate lengths these codes’ performance meets or surpasses that of randomly generated regular (3,5) LDPC codes when used with sum-product algorithm decoders.
I. Introduction LDPC codes with sparse code constraint (Tanner) graphs with graph message-passing algorithms are now known both theoretically and empirically to achieve decoding near the Shannon limit [1] [2] [3]. The principal method for designing such codes is to generate a random low-density parity check matrix from a suitable ensemble [4]. Such random codes pose implementation challenges due to the unavoidable entropy of the constraint graph connections. Furthermore, any given code must be extensively tested to assure performance, particularly for short to moderate block lengths. In the “Recent Results” session of the 2000 International symposium on Information Theory, Tanner presented a [155,64,20] groupstructured quasi-cyclic (3,5) LDPC code. The code’s parity check matrix is composed of cyclically shifted identity matrices chosen to create a code constraint graph with a large girth, g=8, for the graph’s parameters. Despite the extreme sparseness of the graph, the minimum distance of the code is very respectable. The best code presently known has parameters [155,64,28] [5].
1
This work was supported by NSF grant 99-96222
Deepak Sridhara and Tom Fuja1 Dept. of Electrical Engineering Univ. of Notre Dame, Notre Dame, IN 46556
[email protected],
[email protected]
This paper investigates a class of LDPC block codes that generalize this code construction to larger lengths. The girth of code graphs generated with this prescription can be no greater than twelve. This implies that the graphs will fail to show the logarithmic relationship between girth and block length that is known to be achievable, and, for sufficiently long lengths, random graphs would be expected to outperform these codes. In simulation studies given here, the groupstructured (3,5) LDPC codes tested are as good or better than comparable random codes for lengths up to 1,055. Table 1 lists some codes in this class and gives important parameters for each code and its associated constraint graph. Both improved performance and simplicity of implementation make these group-structured codes and graphs attractive for moderate block length systems.
II. Code Construction A code in this class is specified by its parity check matrix, H, which is a block matrix of circulant matrices. Specifically, H consists of a j × k array of p × p circulant permutation matrices, Hs,t, 1 ≤ s ≤ j, 1 ≤ t ≤k, where p is a prime. Let a and b, a b, be non-zero integers from the field GF(p) with orders k and j, respectively. Thus k and j must be divisors of (p-1). The (s,t) permutation matrix is an identity matrix with its rows left cyclically shifted (modulo p) by a number of positions Ps,t = b(s-1) a(t-1), 1 ≤ s ≤ j, 1 ≤ t ≤k. The [l,m]th entry of Hs,t is Hs,t [l,m] = 1 for l- Ps,t =m mod p, Hs,t [l,m] = 0 elsewhere. With this definition, H is a jp × kp parity check matrix. Each row contains exactly k ones, and each column exactly j ones, and the code is a regular (j,k) LDPC code.
III. Code Rate If all the equations of H were linearly independent, the rate of the code would be R =
1 – (j/k), and this will be called the design rate of the code. The sum of all the p rows in any one of the blocks is the all ones vector, and it is immediately apparent that at least (j-1) equations are linearly dependent. Thus in general R 1 – (j/k); in most instances, the rate is only slightly higher than the design rate. For codes with practical block lengths, the rank of the parity check matrix over GF(2) can be computed directly. Theoretical analysis of rate can be carried out using transforms [6]. Let GF(2x) be the smallest extension field of GF(2) containing a primitive pth root of unity, and let be such a root. An p × p Fourier transform matrix F based on this root has ith row f i = [ 1, ,i 2i,. . ., i(p-1)], 0 i p-1, and fi is an eigenvector for any circulant matrix. In particular, f i Hs,t= i s,t = iP, P= Ps,t; i s,t is the ith eigenvalue of the (s,t) block. The rank of the parity check matrix can be determined by finding the rank ri of the j × k matrix i with (s,t) entry i s,t , for each i, and summing over all 0 i p-1. Furthermore, it is possible to show that the ranks satisfy a conjugacy condition, ri = r2i, so one need only find the rank for one i in each conjugacy class modulo p. Then R = 1 – (( ri )/pk).
the sth position, and • in all other positions: [•,•, . . .,•, xs, •, . . .,•]. A bit is checked by one parity from each of the j blocks, and a bit vertex can be indexed by a j-tuple indicating the parity vertex in each of the j blocks to which it is connected. Bit vertex [x1, x2, . . ., xj] is connected to parity vertexes [x1,•, . . .,•], [•, x2, . . .,•], …, and [•, . . .,•, xj]. Correspondingly, a parity vertex [•,•, . . .,•, xs, •, . . .,•] is connected to all bits [x1, x2, . . ., xj] with xs in the sth position. An edge in the graph is defined by a bit-parity pair of connected vertexes, e.g., [x1, x2, ..., xs,..., xj]–[•,•, ...,•, xs,•, ...,•]. Defining the code as quasi-cyclic mod p forces the graph to have a shift automorphism σ that acts additively on all the j-tuples, mapping parity vertexes to parity vertexes, σ([•,•, ...,•, xs, •, ...,•])=[•,•, ...,•, xs+1,•, ...,•], and bit vertexes to bit vertexes, σ( [x1, x2, . . ., xj]) = [x1+1, x2+1, . . ., xj+1], all indexes computed mod p. This is nothing more than the statement that the H matrix consists of circulants.
The prescription for the codes exploits subgroups of the field GF(p), both the additive shift group visible in the quasi-cyclic form, and the two multiplicative subgroups. By design, the code graph inherits an automorphism group incorporating these subgroups. (The automorphism group for a graph is the group of permutations that map vertices to vertices, edges to edges, and preserve edge-vertex connections.) To describe these permutations, a convenient indexing for the vertices is needed.
A second automorphism π arises from the factor a that appears in the cyclic shifts of the identity matrices in a row of the block matrix H. Specifically, π( [•,•, ...,•, xs,•, ...,•]) = [•,•, ...,•, axs,•, ... ,•], and π( [x1, x2, . . ., xj]) = [ax1, ax2, . . ., axj], all indexes computed mod p. Because the number of positions cyclically shifted, Ps,t = b(s-1) a(t-1), in the successive bit blocks of H are given by orbits of the multiplicative subgroup generated by a, this automorphism π preserves vertices and edges of the graph. The first column of H defines a bit [1, b, b2,. . ., bj-1] that is checked by parities [1,•, . . .,•], [•, b, . . .,•], …, and [•, . . .,•, bj-1]. Automorphism π maps this bit to the first bit of the second block, namely, [a, ab, ab2,. . ., abj-1]. Then t repeated applications of π map this first bit to [at, atb, atb2,. . ., atbj-1], the first bit of the (t+1)th block, 0 ≤ t ≤k-1. Observe that π( σ[x1, x2, ..., xj]) = π([x1+1, x2+1, ..., xj+1]) = ([ax1+a, ax2+a, ..., axj+a])=σa( π[x1, x2, ..., xj]). Thus π maps all bits in a block to the next block, and it clearly preserve edges. Using both σ and π, any bit vertex can be mapped to any other bit vertex, and the automorphism group of the graph is transitive on bit vertexes.
The H matrix consists of j blocks of p parity equations. Let the vertex for the xsth parity, 0 xs p-1, from the sth block, 1 ≤ s ≤ j, be indexed by a j-tuple that has an integer xs in
The definition of H is analogous to the definition of its transpose if b is substituted for a and vice versa, and the direction of the shifts of the circulants is reversed. It should come as
The rate equals the design rate if and only if ri =j for all i. The linear dependency noted above is manifest in 0 , which for these codes is a matrix with ones in all positions. It has rank one, not j, so the rate will always be higher than the design rate. In general, there may be other i for which ri < j, increasing the rate further.
IV. Graph Automorphisms: One transitivity
no surprise that a third automorphism that is the “dual” of π maps a block of parities to the next block of parities. With the indexing as given, the third automorphism, ρ, is defined as follows: ρ([•,•, ...,•, xs, •, ...,•]) = [•,•, ...,•, bxs, •, ...,•], with the bxs component in the s+1th position, cyclically right-shifted in the j-tuple by one position, and ρ( [x1, x2, . . ., xj]) = [bxj, bx1, bx2, . . ., bxj-1], all indexes computed mod p. To see that ρ preserves edges, observe that the bit of the first column, [1, b, b2,. . ., bj-1], is fixed by ρ, and ρ simply permutes the parities checking this bit in an orbit of size j. Furthermore, ρ( σ[x1, x2, ..., xj]) = ρ( [x1+1, x2+1, ..., xj+1]) = ( [bxj+b, bx1+b, bx2+b, . . ., bxj-1+b]) = σb( ρ[x1, x2, . . ., xj]), and it behaves similarly in mapping the parity vertexes, thereby preserving edges. Using both ρ and σ, any parity can be mapped to any other parity. Moreover, we note that ρ( π[x1, x2, ..., xj]) = ρ( [ax1, ax2, ..., axj]) = ( [baxj, bax1, bax2, . . ., baxj-1]) = π(ρ[x1, x2, . . ., xj]), and similarly for the parities. With these three automorphisms, the automorphism group is transitive on edges. The proof is straightforward. Given any edge, the bit vertex of the edge can be permuted to the [1, b, b2,. . ., bj-1] bit of the first column using σ and π. Then the edge can be permuted to the edge [1, b, b2,. . ., bj-1]–[1,•, . . .,•] by repeated application of ρ. Since this is true of any edge, the inverse of this permutation for a second edge can be applied to the permutation for the first edge to map the first edge to the second, which means that any edge can be mapped to any other edge and the automorphism group is transitive on edges.
V. Graph Girth and Diameter The transitivity of the automorphism group implies that these graphs are highly symmetric, but this alone says little about likely performance of the code when the graph G constitutes an iterative graph message-passing sum-product decoder. Two graph parameters that are relevant to performance are the graph girth, gG, the length of the shortest cycle, and the graph diameter, dG, the maximum over all pairs of vertexes of the length of the shortest path between them. The underlying assumption of the sum-product algorithm is that the information gathered by the graph message-passing computations is
statistically independent. This assumption will be valid for a number of iterations equal to (gG -1)/4; one iteration, consisting of a parity computation and a bit computation, propagates information along a path of length two in the graph. For the first (gG -1)/4 iterations, the information cannot have been propagated around the shortest cycle in the graph, and information arriving at any node will indeed be statistically independent. For this reason, it is advantageous for G to have a large girth. It has been observed in empirical studies that the statistical dependence does not greatly impair performance if there is only a low density of short cycles. Similarly, in dG/2 iterations information from any one vertex will have reached any other vertex. The diameter of the graph must satisfy dG ≥ (gG -1)/2, since all vertexes on the path around the minimum cycle cannot be reached in fewer steps. If the diameter is large compared to (gG -1)/2, it means that there are vertexes that are remote from one another, and statistically dependent information will be flowing around the shortest cycles well before all the potentially useful received information has been propagated across the graph. These considerations motivate the design of code graphs with large girth and small diameter for the degrees of the vertexes. The prescription given here is very effective in this regard for graphs that are not too large, but the very form of the H matrix puts an upper limit on the size of the girth.
VI. Girth gG
12
For any j 2 and k 3, a graph in this class cannot have a girth greater than twelve, due to the presence of cycles that arise from a commutativity in the connections. Due to the quasi-cyclic shift-invariance, two paths of length six must meet to form a cycle of length twelve. Let X = [x1, x2, . . .], Y = [y1, y2, . . .], and Z = [z1, z2, . . .] be three bit vertexes, one from each of three different blocks, with edges to the two blocks of parities. Let x2 – x1 = ∆x, y2 – y1 = ∆y, and z2 – z1 = ∆z. For the class of graphs under consideration, these differences are all distinct by the choice of a and b in GF(p): ∆x = at-1 (b1 – b0) for some 1 ≤ t ≤k; ∆y = at’-1 (b1 – b0), t’ t. Since a is an element of order k and b is an element of order j from GF(p), ∆x ≠ ∆y, and by the same reasoning ∆z must be distinct as well.
(Otherwise the graph would have a cycle of length four.)
∆x
−∆y
Parity Node
∆z
Bit Node
Fig. 1 Cycle of length 12 The shift permutation can be applied to these bits to form a path in the graph of length six as follows (see Fig. 1): [0,•, . . .,•]–[0, ∆x, . . .]–[•, ∆x, . . .,•]– [∆x-∆y,∆x, . . .]–[∆x-∆y,•, . . .,•]– [∆x-∆y, ∆x-∆y+∆z, . . .]–[•, ∆x-∆y+∆z, . . .,•]. The path goes from the first parity block to the second via a shift of X, then back to the first parity block via a shift of Y, then to the second parity block via a shift of Z. A second path of length six meets this at both ends: [0,•, . . .,•]–[0, ∆z, . . .]–[•, ∆z, . . .,•]– [∆z-∆y,∆z . . .]–[∆z-∆y,•, . . .,•]– [∆z-∆y, ∆z-∆y+∆x, . . .]–[•, ∆z-∆y+∆x, . . .,•]. The second path goes via a shift of Z, then back via a shift of Y, then again via a shift of X; the order of the Z and X links has been interchanged. Together these constitute a cycle of length twelve. Clearly many cycles of this mathematical provenance will exist, since any pair of parity blocks and any three blocks of bits give rise to multiple cycles of this form. This girth limitation eventually becomes a serious handicap for very long codes. Our class of codes is attractive nonetheless for short to moderate length codes. A projective plane code, for example, has a girth of only six, but projective plane codes are known to perform remarkably well, despite the higher density of their parity check matrices [7]. The question not yet answered is for what parameters j, k, and p, if any, the graphs actually achieve girth twelve. Providing a general answer to that question appears to involve detailed investigation of numbertheoretic properties. For short codes, simple counting bounds can be used to prove that the girth must be less than twelve. Evidence that the multiplicative group automorphisms gives a large number of good graphs is provided in the next section.
VII. Table of R, p, gG, and dG values for j=3, k=5 In this table we present a sequence of codes with design rate 2/5. In each instance the prime p is chosen so that (p-1) is divisible by both 3 and 5. The Magma computational algebra software package [8] was used to generate the graph according to our algebraic prescription, find their girths and diameters, and convert them to codes. Length Prime Rate Girth N; N=5p p R gG 155 31 0.4129 8 305 61 0.4066 10 755 151 0.4424 10 905 181 0.4022 12 1055 211 0.4019 12 1205 241 0.4017 12 1355 271 0.4015 12 2105 421 0.4010 12 3305 661 0.4006 12 5105 1021 0.4004 12 6455 1291 0.4003 12 11555 2311 0.4001 NA Table 1: Graph and code parameters
Diam. dG 6 8 10 8 9 9 9 10 11 13 12 NA
This sequence of graphs for (3,5) regular LDPC codes illustrates nicely the behavior of girth and diameter as the size of the prime p increases. First, for many the diameter is three-fourths the girth, greater than the ideal lower limit of one half. As the prime increases, the girths do attain the maximum of twelve. Thereafter the diameter of the graph increases with no increase in girth. The increase in diameter is not monotonic, depending on the subtle qualities of the primes. The graph quality heuristic of diameter compared to girth points to the 905 code as being stronger than the 305 code. The N=755, p=151 code exhibits interesting atypical behavior in a several respects. First, the code equations are redundant; the code rate is ten percent higher than the design rate. For this quasi-cyclic parity check matrix, there exists a i , i 0,that does not have full rank. Second, the diameter of the graph has increased from the next smaller graph, with no increase girth. Third, a quick test using Magma identified a code word of weight 14, and a graph tree argument gives a lower bound of 10 for the minimum distance. Thus the code’s minimum distance is poor by comparison with the minimum distance of the
shorter [155,64,20] code. With the exception of this one case, girth and diameter measures of these (3,5) codes augur well for sumproduct codes for lengths of at least 2105. The computation of girth and diameter for the 11,555 code was too slow on our machine.
VIII. Simulations Results Figure 2 gives simulation results for six of the codes from Table 1, assuming a BPSKmodulated additive white gaussian noise channel [8] and for comparable randomly generated regular (3,5) LDPC codes. In our tests, the new codes perform as well as (or slightly better than) the randomly generated codes. For the code with blocklength 11,555, the performance of the new code is inferior to that of the best randomly generated code. To obtain a BER of 10-5, the new code requires Eb/N0 ≈ 1.95 dB, while there exists a randomly generated regular code requiring only Eb/N0 ≈ 1.45 dB.
IX. Conclusion These quasi-cyclic group-structured regular LDPC codes have graphs that are highly symmetric and have a relatively simple algebraic description. When the length of the basic cycle p is sufficiently large, the graphs appear to have a relatively large girth for the graph size and vertex degrees, attaining the maximum girth of twelve for many of the (3,5) codes tested. Preliminary simulation studies show that the graph quality is manifest in excellent perfor-mance of the sum-product decoder. The new codes tested perform as well or better than randomly generated (3,5) LDPC codes for lengths of 1055 or less. The group-structure should be advantageous for implementation compared to randomly generated LDPC codes. Figure 2 Simulation Curves
References [1] R. G. Gallager, Low Density Parity Check Codes, MIT Press, Cambridge, MA,1963. [2] D. J. C. MacKay, ``Good error correcting codes based on very sparse matrices," IEEE Trans. on Information Theory, vol. 45, no. 2, March 1999, pp. 399-431. [3] N. Wiberg, Codes and Decoding on General Graphs, Linkoeping Studies in Science and Technology, No. 440, 1996. [4] T. Richardson, A. Shokrollahi, and R. Urbanke, ``Design of capacity-approaching irregular low-density parity check codes," IEEE Trans. on Information Theory, vol. IT47, no. 2, Feb. 2001, pp. 619-637. [5] A.E. Brouwer and T. Verhoeff, “An updated table of minimum-distance bounds for binary linear codes,” IEEE Trans. on Information Theory, vol. IT-39, no. 2, Mar. 1993, pp. 662-677. [6] R. M. Tanner, “A Transform Theory for a Class of Group-Invariant Codes,” IEEE Trans. on Information Theory, vol. 34, no. 4, pp. 752775, July 1988. [7] Y. Kuo, S. Lin, and M. Fossorier, ``Low Density Parity Check Codes Based on Finite Geometries: A Rediscovery and More," submitted to IEEE Trans. on Information Theory, 1999. [8] The Magma Computational Algebra System, University of Sydney, Australia (www.maths.usyd.edu.au:8000/u/magma/) [9] D. Sridhara, T. Fuja, and R. M. Tanner, “Low Density Parity Check Codes from Permutation Matrices,” 2001 Conf. on Info. Sci. & Systems, Johns Hopkins University, March, 2001.
