Anton Blad, Oscar Gustafsson, and Lars Wanhammar. *. Abstract — Low-density parity-check codes have recently received extensive attention as a forward ...
Early Decision Decoding Methods for Low-Density Parity-Check Codes Anton Blad, Oscar Gustafsson, and Lars Wanhammar* Abstract
— Low-density parity-check codes have recently received extensive attention as a forward error correction scheme in a wide area of applications. The decoding algorithm is inherently parallelizable, allowing communication at high speeds. One of the main disadvantages, however, is large memory requirements for interim storing of decoding data. In this paper, we investigate a modification to the decoding algorithm, using early decisions for bits with high reliabilities. Currently, there are two early decision schemes proposed. We compare their theoretical performances and their suitability for hardware implementation. We also propose a new decision method, which we call weak decisions, that offers an increase in performance by a factor of two.
1
INTRODUCTION
Low-density parity-check (LDPC) codes were invented by Robert G. Gallager [1] in the early 60’s. However, despite excellent performance and a practical decoding algorithm, they were largely neglected for over 30 years until they were rediscovered by MacKay and Neal [2] in the mid 90’s. Since then, LDPC codes have been found to have performances comparable to, and in some cases exceeding, those of Turbo codes. The main problem in coding theory is to build a practical and efficient decoder. The decoding algorithm for LDPC codes is inherently parallelizable, and thus suitable for communication at high speeds. One of the main disadvantages, however, is large memory requirements for implementations that are not fully parallel. A typical code with a length of 6000 bits might require up to a million memory accesses for the decoding of a single word. As 95% of the power is consumed in the memories, reducing the amount of memory accesses becomes an important concern, especially in low-power handheld devices. 2
LOW-DENSITY PARITY-CHECK CODES
LDPC codes are defined as the null space of a very sparse parity-check matrix H. H has dimensions M × N , and the set of valid code words are defined by the relation Hx = 0 . Thus code words are of length N, and each code word contain (assuming H is full-rank) M parity symbols. An LDPC code is typically represented by a bipartite graph between N variable nodes and M check nodes, called a Tanner graph. An LDPC code is regular if all variable nodes have a constant degree of j, and all check nodes have a degree of k. *Department of Electrical Engineering, Linköping University SE-581 83 Linköping, Sweden, e-mail: {antonb, oscarg, larsw}@isy.liu.se, tel: +46 13 284059, fax: +46 13 139282
v1
v2
c1
v3
v4
v5
c2
v6
v7
c3
v8
c4
v9
v10
c5
Figure 1: Tanner graph corresponding to the code given by H in (1). The variable nodes v correspond to the columns of H, and the check nodes c correspond to the rows. There is an edge between vn and cm if and only if Hm,n = 1. (1) shows an example code with ( N , j, k ) = ( 10, 2, 4 ) . Thus the number of parity bits is M = N j ⁄ k = 5 , and the code has rate R = 1 ⁄ 2 . However, H actually has one dependent row, which may be removed in order to slightly increase the rate. This always happens when the column weight j is even, as well as with many systematically constructed codes, but becomes a minor issue as the code length grows. 1 1 H = 0 0 0
1 0 1 0 0
1 0 0 1 0
1 0 0 0 1
0 1 1 0 0
0 1 0 1 0
0 1 0 0 1
0 0 1 1 0
0 0 1 0 1
0 0 0 1 1
(1)
Associated with H is the Tanner graph shown in Fig. 1, where the variable nodes have a constant degree of j = 2 , and the check nodes have a constant degree of k = 4 . The girth of a graph is the length of its shortest cycle. This concept is important, as cycles introduce dependencies in the decoding process, thereby reducing the decoder’s performance. 3
SUM-PRODUCT DECODER
The sum-product decoding algorithm works by passing messages representing bit probabilities along the edges of the Tanner graph. As the graph is bipartite, there are two types of messages. The variable-to-check message a q mn, i is meant to represent the likelihood that variable n has the value a, given the information received from check node neighbours other than m in iteration i. The a check-to-variable message r mn, i is meant to denote the
probability that check node m is satisfied, given that variable n is locked at a and m’s other neighbours have separate distributions given by their respective messages to check node m. The probabilities are exact if the Tanner graph has no cycles [3]. However, cycles are needed for the code to be usable [4], and with cycles the messages are accurate only in the first few iterations. Decoding is done in several phases:
15
Reliability (log10 q0n/q1n
10
Initialization a For every variable node, the prior likelihoods p n are initialized to the likelihood that variable n has the value a, given channel information. Then the variable-toa check messages q mn, 0 are set to the prior likelihoods a pn . Horizontal step a The check-to-variable messages r mn, i are computed according to the relation (2), where n1,...nl denotes the neighbours to check node m other than n, and [P] denotes a function which is 1 if P is true and 0 otherwise. A ⋅ B , where
( x n , …, x n ) ∈ {0, 1} 1
−15
1
2
3
4
5
xn
α mn, i 4
l
xn
, i – 1 ⋅ … ⋅ q mn , i – 1
1
7 8 Iteration
9
10
11
12
13
fined as in (4), and the check-to-variable messages β mn, i are defined analogously.
l
B = q mn
6
Figure 2: Example of bit reliabilities for a few variables of a typical decoding.
l
1
l
1
l
Vertical step In the vertical step the variable-to-check messages a q mn, i are computed as a
q mn, i = α n, i ⋅ r m
1 n,
a
i
⋅ … ⋅ r m n, i l
(3)
where m1,...,ml denotes the neighbours to variable node n other than m, and α n, i is a normalizing constant chosen such that q 0mn, i + q 1mn, i = 1 . Decision step a The pseudo-posterior likelihoods q n, i are computed by the same equation (3) as the variable-to-check messages, except that the product is over all the neighbours to n. A tentative decoding xˆ is created by chosing each bit individually to the most likely value, based on the pseua do-posterior likelihoods q n, i . If H xˆ = 0 , xˆ is reported as the decoded word. Otherwise the algorithm continues from the horizontal step until the maximum number of iterations is reached. 3.1
−5
−10
(2)
A = [ a + xn + … + xn = 0 ]
a
0
0
∑
a
r mn, i =
5
Log-likelihood ratio domain
The messages as defined above are redundant, as the sums of the two constituent quantities are one. In an implementation it is therefore beneficial to pass just the ratios of the quantities. Also, to reduce the dynamic range of the messages and to simplify the computations, the messages are defined as the logarithms of the ratios. Thus the variable-to-check messages α mn, i are de-
q mn, i = log ------------1 q mn, i
(4)
EARLY DECISION DECODER
At the end of each iteration the pseudo-posterior a likelihoods q n, i are computed for every variable node a n. q n, i is meant to denote the likelihood that bit n of the sent code word is a, given the information available at 0 1 the ith iteration. Figure 2 shows the ratio q n, i ⁄ q n, i as a function of the current iteration for a few variables of a typical decoding. The value of the ratio has a direct interpretation for the first few iterations: a variable with 0 1 log ( q n, i ⁄ q n, i ) = c has a 10-c chance of causing a decoding error, if it is locked at its most probable value. Generally, as seen in Fig. 2, the variables converge quickly towards either 0 or 1. The idea of early decision decoding is to exploit this fact by deciding variables with high likelihoods, in order to reduce the amount of memory accesses and the switching activity. Currently, there are two types of reliability measures proposed, one by Blad et al. [5], and one by Zimmermann et al. [6]. The first, which we call model one is to define the reliability as 0
q n, i - , c n, i = log -------10 1 q n, i
(5)
a quantity which is already available in a typical decoding process. This is obviously a big advantage, as there is no overhead associated with the computation of the reiability values. The second definition, model two, by Zimmermann, is a
c n, i =
∑ qm'n, i a
m'
(6)
I
E =
∑ i=0
5
di ---N
(7)
SIMULATION RESULTS
The simulations were conducted using a girth-8 (N, j, k) = (1002, 3, 6) integer lattice code [7], with the maximum number of iterations set to 50. Experiments were
0
10
0.4 Prob. prop. BEP Threshold BEP, model one Threshold BEP, model two Efficiency, model one Efficiency, model two
−1
10
0.3
−2
−3
10
0.2 −4
10
Efficiency
Bit error probability
10
−5
10
0.1 −6
10
−7
10
1
1.5
2
2.5 Eb/N0 (dB)
3
3.5
4
0
Figure 3: Performance of the two thresholding algorithms. The thresholds were for model one t=5, and for model two t=2.9992. 0
10
0.4 Prob. prop. BEP Threshold BEP, model two Efficiency, model two
−1
10
0.3
−2
−3
10
0.2 −4
10
Efficiency
10 Bit error probability
as the “confidence” that variable n is in state a. One advantage of this definition is that the reliabilities become more resistant to noise introduced by decisions in earlier iterations. The disadvantage, however, is that the value is not easily determined by the decoder as the values a q m'n, i are not directly available. Also, the values will have dynamic ranges depending on the degrees of the variable nodes, which further complicates implementations. For both schemes a threshold t is defined, and whenever the reliability of a variable is above t, the variable is locked at its most probable value, and its messages are no longer updated. However, there are two ways of locking the messages: either the node is simply deactivated and its outgoing messages retain the values they had at decision time, or the messages attain infinitely reliable values. The first method substantially reduces the potential gains of the algorithm, as the messages from the decided variables still must be read from memory each iteration. The second method, however, introduces noise in the decoding process, thereby reducing performance. We call this method the strong decisions method. The performances of the two methods were investigated by Zimmermann [6], who concluded that forcing the reliabilities resulted in a too big performance drop. Here, a third method is suggested, which we call weak decisions. Obviously, at decision time the reliability of the decided variable is close to the value given by the threshold t. Thus setting the outgoing messages 0 1 a to t, i.e. log ( q mn, i ⁄ q mn, i ) = t , and jq mn, i = t , respectively, should result in reduced noise, while driving of long memory paths is still avoided. Switching activity inside the processing elements remain, though, but as described in Section 6 might be somewhat reduced. In order to evaluate and compare the different schemes, we define a simple measure of the efficiency, which is the number of avoided message computations, as given by (7). The measure should roughly correspond to the power consumption reduction of a hardware implementation. However, the methods proposed have different overheads, which are not captured by the measure. Several other factors are not considered either, such as a small increase in the average number of iterations, and the cost of additional logic required to implement the scheme. I denotes the number of iterations required to decode a block, and di is the (cumulative) number of decided bits in iteration i.
−5
10
0.1 −6
10
−7
10
1
1.5
2
2.5 Eb/N0 (dB)
3
3.5
4
0
Figure 4: Performance of the model two thresholding scheme using simple deactivation of nodes as proposed by Zimmermann. t=2.9992. also conducted using a length-6702 code, with similar results, although these are not included. Figure 3 compares the two threshold models using strong decisions. As seen, the performances are almost equal, with only a slight favor for Zimmermann’s model two. Figure 4 shows model two threshold decoding using simple deactivation of nodes, as proposed by Zimmermann. Unfortunately our results do not support Zimmermann’s conclusion of only a 0.025 dB performance loss at a frame error probability of 10-4 (corresponding to a bit error probability of about 10-6.) Figure 5 compares strong and weak decisions for the model one thresholding scheme. Weak decisions provide a performance increase by a factor of two. Similar results are achieved for model two thresholding. In Fig. 6 the average number of iterations is shown. As can be seen, the number of iterations for the thresholding schemes is insignificantly higher compared to regular probability propagation.
7
0
10
Prob. prop. BEP Threshold BEP, strong decisions Threshold BEP, weak decisions Efficiency, strong decisions Efficiency, weak decisions
−1
10
0.3
−2
−3
10
0.2 −4
10
Efficiency
Bit error probability
10
−5
10
0.1 −6
10
−7
10
1
1.5
2
2.5 Eb/N0 (dB)
3
3.5
4
0
Figure 5: Comparison of strong and weak decisions for the model one thresholding scheme. t=5.
Prob. prop. Model one, strong dec. Model one, weak dec. Model two, strong dec. Model two, weak dec.
45
Average number of iterations
40
In this paper we have compared two methods of early decision decoding for low-density parity-check codes. It is found that the two models, called model one and model two, have similar performance. However, model one is considered more suitable for a hardware implementation, as the values on which decisions are based already are available in the decoding process. We also propose a new decision scheme which we call weak decisions. Weak decisions avoid the memory access problems with Zimmermann’s proposed scheme, while still providing a comparable performance increase. References [1] [2]
50
[3]
35 30
[4]
25 20 15
[5]
10 5 0
1
1.5
2
2.5 E /N (dB) b
3
3.5
4
[6]
0
Figure 6: Average number of iterations for the different decoding schemes. 6
CONCLUSION
0.4
IMPLEMENTATION CONSIDERATIONS
In a hardware implementation of a sum-product decoder, messages are usually passed as signed-magnitude numbers in the log-likelihood domain. Thus the reliabilities of the model one scheme, as defined by (5), are already available. This is in contrast to the model two definition (6), which is not easily evaluated. The weak decision decoding scheme proposed here yields a performance increase similar to the simple deactivation scheme by Zimmermann. The weak decision scheme, however, has not the memory access overheads associated to it. a The check-to-variable messages r mn, i are computed as sums in a domain respresenting uncertainties rather than reliabilities. The “uncertainties” of the incoming messages are summed together to yield the “uncertainty” of the outgoing message. This provides an option for an efficient implementation of the weak decision scheme. By approximating the threshold to a value represented by one of the less significant bits, switching activity inside the processing element can be kept low, while performance is still adequately good.
[7]
R. G. Gallager, “Low-density parity-check codes,” IRE Trans. Inform. Theory, vol. IT-8, pp. 21–28, Jan. 1962. D. J. C. MacKay, “Good error-correcting codes based on very sparse matrices,” IEEE Trans. Inform. Theory, vol. 45, no. 2, pp. 399–431, Mar. 1999. N. Wiberg, Codes and Decoding on General Graphs, PhD thesis, Linköping University, Linköping, Sweden, 1996. T. Etzion, A. Trachtenberg and A. Vardy, “Which codes have cycle-free Tanner graphs?,” IEEE Trans. Inform. Theory, vol. 45, no. 6, pp. 2173–2181, Sept. 1999. A. Blad, Efficient Decoding Algorithms for Low-Density Parity-Check Codes, MSc thesis, Linköping University, Linköping, Sweden, 2005. E. Zimmermann, G. Fettweis, P. Pattisapu and P. K. Bora, “Reduced complexity LDPC decoding using forced convergence,” 7th Int. Symp. on Wireless Personal Multimedia Communications, Sept. 2004. B. Vasic, K. Pedagani and M. Ivkovic, “High-rate girtheight low-density parity-check codes on rectangular integer lattices,” IEEE Trans. Communications, vol. 52, no. 8, pp. 1248–1252, Aug. 2004.
