Low-Density Parity Check Codes for Fading Channels with Memory

To appear in Proceedings of the 36th Annual Allerton Conference on Communication, Control, and Computing, 1998.

Low-Density Parity Check Codes for Fading Channels with Memory Andrew P. Worthen and Wayne E. Stark Electrical Engineering and Computer Science University of Michigan Ann Arbor, MI 48109-2122 fworthena, [email protected]

Abstract

We develop a belief-propagation-based joint decoding and channel state estimation algorithm for low-density parity check codes on fading channels with memory. The excellent empirical performance of LDPC codes on the AWGN channel and the possibility of arranging the code memory relative to the channel correlation suggests that LDPC codes have the potential to perform well on fading channels with memory. We treat the block fading channel, which models a frequency-hopped spread spectrum system. We present performance results for simple codes decoded using our algorithm and compare them with the performance of the belief propagation algorithm without channel state estimation and block-interleaved Reed-Solomon codes.

1 Introduction Low-density parity check (LDPC) codes, rst proposed by Gallager [1, 2], have recently been found to exhibit near Shannon capacity performance on the additive white Gaussian noise (AWGN) channel using a practical decoder [3]. Some LDPC codes over GF (q) with q > 2 have been shown to have performance rivaling that of the best known turbo codes [4, 5]. The codes are speci ed by very sparse random parity check matrices in which each coded symbol participates in no more than j parity check equations and each parity check equation uses no more than k > j coded symbols where j and k are small integers irrespective of the block length n. Uniform column and row weights have typically been used, but probably do not give the best performance. The code is also compactly described by a bipartite graph in which each row or column becomes a vertex, and a 1 in the parity check matrix indicates that the corresponding row and column vertices are connected by an edge. Numerous algorithms have been proposed for generating random matrices (or equivalently graphs) with desired weights and minimal column overlap. Sipser and Spielman [6] give a construction of a more general set of \expander codes" based on expander graphs This research was supported by the Army Research Oce under Grant DAAH04-96-1-0110 and by a National Science Foundation Graduate Research Fellowship. 

1

whose expansion property seems to impart good properties to the resulting codes. They also describe algorithms for generating random graphs likely to have good expansion. The codes can be eciently decoded using an iterative belief propagation algorithm which approximates maximum likelihood decoding. This algorithm, which McEliece, MacKay, and Cheng [7] have shown to be related to the well-known turbo decoding algorithm, successively updates estimates of the a posteriori symbol probabilities using the graph associated with the parity check matrix. Although no convergence proof for this algorithm exists so far in the case of graphs with \loops", numerous empirical results for turbo codes and LDPC codes show that it performs very well. Previous work on belief propagation appears in the arti cial intelligence literature on Bayesian networks [8]. The excellent performance of LDPC codes on the AWGN channel and the proven performance of turbo codes with iterative decoding motivates the extension of these ideas to fading channels with memory. Because LDPC codes are based on random bipartite graphs, carefully chosen constraints on the graph construction could yield codes especially suited to particular fading channel applications. For example, block interleaving can be built directly into the code. Also, when the fading level (or time-varying interference power) is unknown, joint decoding and channel state estimation is optimal. We show that the LDPC decoding algorithm is easily extended to include channel state estimation and characterize its performance on a simpli ed fading channel model for frequencyhopped spread spectrum systems. In section 2, we describe the system model. Section 3 describes the decoding algorithm for fading channels. Section 4 gives the results of some performance simulations. In section 5, we present our conclusions.

2 System Model We investigate iterative decoders for LDPC codes transmitted over a simpli ed, correlated fading channel model. We consider the block fading channel which models frequencyhopped spread spectrum (FH-SS) systems. We model this channel as a pair of symmetric Gaussian channels with dierent signal-to-noise ratios (SNR's) employing binary antipodal signaling with coherent detection. For each block of m coded symbols, we choose one channel with probability p and the other with probability 1 ? p and transmit the symbols over that channel. If the receiver knows which channel was used for each symbol, we say that perfect channel state information (CSI) is available. Otherwise, we assume that the receiver can deduce the channel state only from the received symbols. In both cases, we assume that the receiver knows the block boundaries, channel selection probabilities, and channel SNR's exactly. In a FH-SS system, the \good" and \bad" channels correspond to hops enhanced by constructive multi-path interference or degraded by destructive multipath (or jammers or multiple access interference) respectively, and the pseudo-random hopping pattern ensures that the block states are independent.

3 Decoding Algorithm Consider the symbol-by-symbol maximum a posteriori probability (MAP) decision rule for a symbol vi given some statistically related evidence x

v^i = arg max p(vi jx): v i

2

Ci Vi

Ci

Si Vi

a)

b)

Figure 1: Belief networks for LDPC codes. a) network for AWGN channel. In this example, each variable is the parent of two constraints and each constraint is the child of three variables. b) network for block interference channel. Here each channel state is the parent of four consecutive variables and each variable is the child of exactly one channel state. Previous literature has shown that Pearl's belief propagation algorithm [8] can be used to approximately solve this problem for LDPC codes. We extend the application of Pearl's algorithm to joint decoding and channel state estimation as follows. Figure 1a shows the belief network for an LDPC code on an AWGN channel . Following the nomenclature of Sipser and Spielman [6], \variables" V correspond to unknown coded symbols and \constraints" C correspond to parity check equations. We say that a constraint is satis ed if the sum mod 2 of its parent variables' values is 0. In Figure 1b, we extend this belief network to include channel state estimate nodes S for the block fading channel. In both cases, the observed channel outputs are incorporated into the variable nodes and are either conditionally independent given the transmitted symbol for the AWGN channel, or conditionally independent given the channel state and the transmitted symbol for the block fading channel. Each node Z holds two vectors (z) and (z) corresponding to likelihood contributions for each of its possible values from its parents and children respectively. The algorithm updates these values by combining messages Z (ui) from the node's parents and Y (z) from the node's children. The total likelihood estimate is p(zjx) = (z)(z) for some suitable normalizing constant . In practice, the nodes are updated repeatedly until a \good" estimate of the MAP decision is reached. The channel node values (s) and V (s) are initialized to the a priori probabilities of the channel states p and 1 ? p. The variable node values (v) and C (v) are initialized to the a posteriori probabilities of the coded symbols given the observed channel outputs x and the priors on the channel states p(s): 1

i

i

i

(v) = p(vjx) = 1

X f (xjv; s) p(vjs) p(s) f (xjs)

s

This is analogous to Figure 10 of [7].

3

where

f (xjv; s) = exp(? (s)(x ? (?1)v ) ) is the probability density function of the channel outputs given the transmitted symbol and the channel state for some normalizing constant , and (s) is the coded symbol SNR for channel state s. Also, f (xjs) = f (xjv; s) p(v): 2

X v

This gives

X f (xjV = 0; s) p(V = 0) f (xjV = 1; s) p(V = 1) + f (xjV = 0; s) p(V = 0) p(s) X 1 p(s) = e +1 X e p(s) (1) = 1 ? (0) = (0) =

s

s

?4x (s)

?4x (s)

e? x s + 1 where p(v), the prior on the sent symbol value v 2 f0; 1g, was taken to be 1=2. Note that the channel values x are normalized so that E[xj+1 sent] = 1. We update the variables before the channel states so it is unnecessary to initialize V (s). The constraints are known to be satis ed so we just initialize the C (vi) to be equally likely (i.e. the constraints do not contribute information about the variable states without information about other variables). For the channel state update, we calculate (s) = V (s) for each possible s 4

s

Y

( )

i

i

Y

and the values needed by the variables, which are updated priors on the channel state V (s) = (s) V (s) i

k

k6=i

(see Figure 2a). Note that V (s) is not included since this would be reusing information and result in undesired positive feedback. For the variable update, we calculate (v) = C (v) for each possible v i

Y

i

i

(v) =

X p(vjx; s) (s)

(0) =

X

and so

Vi

s

s

1

e?4x (s) + 1 V (s)

(1) = 1 ? (0) =

X s

4

i

e? x s  (s): e? x s + 1 V 4

4

( )

( )

i

π V (s) 1

λ V (s)

π C (v)

1

π C (v1)

1

π V (s)

π V (s) Si

λ C (v)

i

2

λ V (s)

λ C (v2)

1

λ V (s)

Si

2

π C (v2)

π C (v)

Vi

2

λ (v3)

λ C (v)

π V(s)

Ci

C

2

3

λ C (v1)

π C (v3)

Ci

λ V (s) 3

Vi

Vi

a) b) c) Figure 2: Messages for node updates: a) channel state node, b) variable node, c) constraint node

Y

The messages to be passed to the variable's neighbors are

C (v) = (v) i

k6=i

Ck

(v);

which are updated estimates of p(vjx; s; other constraints), and the updated state estimate contributions

V (s) =

X (v) p(vjs) v

? x s = (0) e? x 1s + 1 + (1) e?e x s + 1 4

4

( )

4

( )

( )

(see Figure 2b). Finally, for the constraint update, we need calculate only the values C (vi) which will be needed in the next variable update. The local values are irrelevant since we know that the constraint must be satis ed. The C (vi) represent the evidence that other variables participating in constraint C contribute to our knowledge of the variable Vi. Thus, from Pearl's formulas,

C (vi ) =

X (c) X p(cjv) Y  (v ): c

v:[v]i =vi

k6=i

C

k

However, to be consistent with the constraint being satis ed (c) must equal 1 only when c = 0. Similarly, we can interpret p(C = 0jv) as being 1 if the parity of v is even and 0 otherwise, so

C (vi) =

X Ifv has even parityg Y  (v ): k6=i

v:[v]i =vi

C

k

v represents the vector of states of C 's parents, I f
g is the indicator function, and the sum is over all possible states such that the i state equals vi (see Figure 2c). th

5

Although for a graph without loops, the update schedule can be nearly arbitrary, update order for belief networks with loops is still not well understood. We report results where we update the constraints, the variables, the channel states, and the variables again in each iteration. After each iteration, we check to see if the maximum likelihood estimates of each of the variables satisfy all the constraints. If so, we declare these estimates to be the decoded word and stop. Otherwise, we continue iterating up to some maximum number of iterations (perhaps 100 or so) and declare a decoding failure if the estimates still do not form a codeword.

4 Results We studied the performance of our decoding algorithm on the block interference channel by simulation. We create the codes by generating a random (c; d)-regular graph following the technique described in Appendix II of [6] and eliminating the multi-edges by random swaps. Graphs created this way are likely to have good expansion and thus may give good performance. This graph becomes the variable-to-constraint graph in the decoding algorithm. We encode a selection of equally-likely random bits using a possibly permuted, systematic generator matrix, pass the resulting codeword through a simulated channel, decode the received channel outputs using our decoding algorithm, and report the number of information bit errors. All the information bit errors we observed were caused by decoding failures. Figure 3 shows our simulation results for a block length 10,240 code with 5,120 information bits (and thus rate 1/2) on a block fading channel. The code was constructed as described above such that each variable participates in 3 constraints and each constraint depends on 6 variables. No attempt was made to optimize the construction. We compare the results for our algorithm with an algorithm that uses only the prior probabilities of the channel states (\no channel estimation") and with a Reed-Solomon code with compatible parameters. The gain of channel estimation over no channel estimation is roughly 0.2 dB. This improvement might be more with other channel parameters. The gain of the LDPC code over the RS code, which has the same block length and rate and is very robust to bursts of errors in a single hop, is about 3 dB. About 2 dB of this gain is because LDPC codes use soft-inputs; the rest is due to improved code performance. Further gains may be possible by using more carefully designed codes or making algorithm modi cations. Figure 4 shows a histogram of the number of iterations required to decode a codeword. This shows that despite the fact that the average number of iterations per word was only 20.9, there were a signi cant fraction of codewords that required more than 50 iterations to decode. Note that the right-most bin includes the words that ran to 100 iterations and failed to decode. These results underscore the importance of running the algorithm until a codeword is found even if it takes 100 or more iterations. This makes real-time implementation more dicult so approaches to either reducing the fraction of words that require a large number of iterations to decode or accelerating the algorithm would be useful. In Figure 5, we compare the performance of LDPC codes with channel estimation with the SINCGARS RS coding system. The SINCGARS code block-interleaves 24 (32,12) extended RS codewords and adds a parity check to each symbol to form a single block of 4608 coded bits with a hop length of 144 bits (one symbol plus its parity check from each codeword). The decoder uses the parity checks to generate erasures for errors and 6

−1

10

−2

bit error rate

10

−3

10

−4

10

−5

10

−6

10

3

4

5 6 average SNR (dB)

7

8

Figure 3: Performance of (10240,5120) LDPC code on the block interference channel.  LDPC code with channel estimation,  LDPC code without channel estimation,  (1024,512) extended RS code. Channel SNR's were separated by 7 dB, the probability of choosing the good channel was 0.5, and the hop length was 10 bits. 0.7

fraction of codewords

0.6 0.5 0.4 0.3 0.2 0.1 0

5

15

25

35 45 55 65 iterations to decode

75

85

95

Figure 4: Iteration histogram for (10240,5120) LDPC code on the block interference channel.  LDPC code with channel estimation. Channel SNR's were separated by 7 dB, the average SNR was 3.8 dB, the probability of choosing the good channel was 0.5, and the hop length was 10 bits. 7

−1

10

−2

10

−3

bit error rate

10

−4

10

−5

10

−6

10

−7

10

3

4

5

6 7 average SNR (dB)

8

9

Figure 5: Performance of (4608,1843) LDPC code on the block interference channel.

 LDPC code with channel estimation,  SINCGARS RS code. Channel SNR's were

separated by 7 dB, the probability of choosing the good channel was 0.5, and the hop length was 144 bits. erasures decoding. Additionally, if more than some threshold number of symbols in a hop are erased, the other symbols in the hop are erased as well. We determined the performance of the Reed-Solomon system by counting simulated bit errors. In codewords that had more than the correctable number of errors and erasures, we counted bit errors in the rst 12 symbols to estimate the information bit error rate. We used a threshold of 20 symbols out of 24 to determine whether to erase the entire hop. The LDPC code was a length 4608 code with 1843 information symbols generated with variable degree 3 and constraint degree 5 and has a total rate higher than the RS-code system. The LDPC code shows a gain of about 3.1 dB relative to the RS-code system. Again, about 2 dB of this might be because the RS code system uses hard decisions and does not perform channel estimation. However, the RS-code system does use techniques to improve its performance in block fading and also uses the burst error correction power of Reed-Solomon codes.

5 Conclusions The belief propagation algorithm for decoding LDPC codes on AWGN channels extends with little added complexity to joint decoding and channel estimation on block interference channels. We have shown that the joint decoding and channel estimation algorithm performs somewhat better than decoding using only the channel state a priori probabilities. Also, LDPC codes perform better than comparable Reed-Solomon code-based systems even though the Reed-Solomon codes have good burst error correction capability.

8

References [1] R. G. Gallager, \Low-density parity-check codes," IEEE Transactions on Information Theory, vol. IT-8, no. 1, pp. 21{28, January 1962. [2] R. G. Gallager, Low-Density Parity-Check Codes, M.I.T. Press, Cambridge, MA, 1963. [3] D. J. C. MacKay and R. M. Neal, \Near shannon limit performance of low density parity check codes," Electronics Letters, vol. 33, pp. 457{458, March 1997. [4] M. C. Davey and D. J. C. MacKay, \Low-density parity check codes over GF(q)," IEEE Communications Letters, vol. 2, no. 6, pp. 165{167, June 1998. [5] M. C. Davey and D. J. C. MacKay, \Low-density parity check codes over GF(q)," in Proceedings of the IEEE Information Theory Workshop, Killarney, Ireland, June 1998, pp. 70{71. [6] M. Sipser and D. A. Spielman, \Expander codes," IEEE Transactions on Information Theory, vol. 42, no. 6, pp. 1710{1722, Nov. 1996. [7] R. J. McEliece, D. J. C. MacKay, and J-F. Cheng, \Turbo decoding as an instance of Pearl's \belief propagation" algorithm," IEEE Journal on Selected Areas in Communications, vol. 16, no. 2, pp. 140{152, February 1998. [8] J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, Morgan Kaufmann, 1988.

9