Development of Rate-Compatible Structured LDPC CODEC ...

Table of C o n t e n t s

List of Abbreviations

Development

of Rate-Compatible Structured LDPC Algorithms and Hardware IP

CODEC

Project Final Report

'»

CHAPTER I : Introduction

1

CHAPTER I I : Background Research

3

2.1 Low-Density Parity-Check Codes

5

2.2 Iterative Decoding Algorithm

9

2.3 Efficient Encoding Method

19

2.4 Irregular Repeat Accumulate Codes

23

2.5 Rate-Compatible Puncturing Algorithm

25

CHAPTER I I I : Effkiently-Encodable Rate-Compatible Codes School of Electrical and Computer Engineering Georgia Institute of Technology December 2006

40

3.1 New Class of Irregular LDPC Codes

41

3.2 Low-Rate Code Design

54

3.3 Efficient Encoder Implementation

57 6 6

3.4 Simulation Results 7

3.5 Conclusions CHAPTER IV :

-*

Rate-Compatible LDPC Codes For Incremental

Redundancy Hybrid ARQ Systems Jaehong Kim Demijan Kline Woonhaing Hur Dr. Aditya Ramamoorthy Dr. Sunghwan Kim Dr. Steven W. McLaughlin

4.1 Incremental Redundancy Hybrid ARQ Systems

75 76

4.2 System Model

7 7

4.3 Simulation Results

7 8

4.4 Conclusions

^

CHAPTER V : Wire-tap Channel Application

84

5.1 Encoding Algorithm for the Wire-tap Channel

84

CHAPTER V I : Optimization of Degree Distributions

92

References

96

L i s t of A b b r e v i a t i o n s

ACK APP ARQ AWGN BEC BER BP BPSK BSC CRC CSI E RC eIRA FEC FER HARQ IC IR IRA LDPC LLR MAP MMSE NACK PEG QC QPSK RCPC V-BLAST VLSI WER 2

ii

Acknowledgement A Posteriori Probability Automatic Repeat reQuest Additive White Gaussian Noise Binary Erasure Channel Bit Error Rate Belief Propagation Binary Phase Shift Keying Binary Symmetric Channel Cyclic Redundancy Check Channel State Information Efficiently-Encodable Rate-Compatible extended IRA Forward Error Correction Frame Error Rate Hybrid Automatic Repeat reQuest Integrated Circuit Incremental Redundancy Irregular Repeat Accumulate Low-Density Parity-Check Log Likelihood Ratio Maximum A Posteriori Minimum Mean Square Error Negative Acknowledgement Progressive Edge Growth Quasi-Cyclic Quadrature Phase Shift Keying Rate-Compatible Punctured Codes Vertical Bell Labs Layered Space-Time Very Large Scale Integration Word Error Rate

CHAPTER I

(QC) LDPC or algebraic constructions that can be encoded by shift registers [9]. Irregular repeat-accumulate (IRA) codes were introduced by Jin el al. [10]. These

INTRODUCTION

codes have a linear-time encoder and their performance is almost as good as irregular LDPC codes. This class of codes was extended, called extended IRA (eIRA) codes, by Yang el al. [11], where they demonstrated high-rate codes with very low error floors. Low-density parity-check (LDPC) codes by Gallager [1] had been forgotten for several

A popular technique for achieving rate adaptation in a system is through the use of

decades in spite of their excellent properties, since the implementation of these codes

rate-compatible puncturing. A rate-compatible punctured code (RCPC) is suitable for

seemed to be impossible at that time. These codes were rediscovered in the middle of the

applying to incremental redundancy (1R) hybrid automatic repeat request (HARQ)

1990s [2] and were shown to achieve Shannon limit within 0.0045dB [3]. LDPC codes

systems, since the parity bit set of a higher rate code is a subset of the parity bit set of a

are now considered good candidates for the next-generation forward error correction

lower rate code [12]. The RCPC scheme has another advantage in that it has the same

(FEC) technique in high throughput wireless and recording applications. Their excellent

encoder and decoder while operating at different rates. The number of parity bits that the

performance and iterative decoder make them appropriate for technologies such as DVB-

transmitter sends depends on the rate requirement. At the decoder end, parity bits that are

S2, IEEE 802.16e [4], and IEEE 802.1 In [5], [6].

not transmitted are treated as erasures. Thus, puncturing provides a low-complexity

While semiconductor technology has progressed to an extent where the

solution to the rate-adaptation problem.

implementation of LDPC codes has become possible, many practical issues still remain.

Motivated by these observations, this reportfirstproposes the puncturing algorithm for

First and foremost, there is a need to reduce complexity without sacrificing performance.

LDPC codes with short block lengths. Based on the puncturing algorithm, a new class of

Second, for applications such as wireless LANs, the system throughput depends upon the

codes is proposed that can be efficiently encoded as well as can be punctured in a rate-

channel conditions and hence the code needs to have the ability to operate at different

compatible fashion. The proposed LDPC codes will be shown to have a linear-time

rates. Third, while the LDPC decoder can operate in linear time, it may be hard to

encoder and have good performance under puncturing for a wide range of rates. Finally,

perform low-complexity encoding of these codes. In particular, the class of irregular

we verify that the proposed codes show good throughput performance when they are

LDPC codes introduced by Richardson el al. [7] may have high memory and processing

applied to 1R-HARQ systems over time-varying channels.

requirements, especially at short block lengths. While the encoding time can be reduced substantially using the techniques presented in [8] at long block lengths, their techniques may be hard to apply at short block lengths. The other option is to resort quasi-cyclic 2

CHAPTER II G =\

BACKGROUND RESEARCH

Channel coding is an essential technique to cope with errors occurring in channels of communication systems and storage systems. Channel coding has flourished in two branches. Channel errors can be corrected with forward error correction (FEC) codes. On the other hand, a receiver may request retransmission of the previous data if it fails to recover them, which is called automatic repeat request (ARQ). FEC codes can be classified into block codes, such as cyclic codes and LDPC codes, and tree codes, such as convolutional codes and Turbo codes. In this chapter, we briefly explain the block codes where LDPC codes are specified. Let us consider linear block codes over the binary field F ! ({0,1}, +, x) Let F be 2

2

the TV-dimensional vector space over F Then, an (N, K) linear block code C is defined 2

as AT-dimensional subspace of F , where K is a data word length and N is a codeword 2

length. Since C is a subspace of dimension K, there are K linearly independent vectors So'S\>"

i

SKI

which span C

.

Let m

v

is thus naturally written as

c = m„g„ + m g [

l

+"

is called code

rate. 1

On the other hand, the null space C of C has dimension N — K and is spanned by N-K

1

linearly independent vectors h„,h,," ,h.-_. Since each h, eC' , we should vk

have for any ceC that h, c^O, V/ This relationship can be represented in the matrix form as H c = 0, where the matrix H is the so-called parity-check matrix defined as T

h

0

i

A

l

c = [c, c,," , c_,] be the corresponding codeword in the code C. The mapping m —> c 0

N

,/n-_,] be the data word and

= [m ,m ," 0

We call the matrix G the generator matrix for C. In fact, C is the row space of G The encoding process can be viewed as an injective mapping that maps vectorsfromthe K -dimensional vector space into vectors from the N -dimensional vector space. The ratio R-- a:

+ m _g _ K

i

K

l

This relationship can be

represented in the matrix form c = mG , where G is a K x N matrix;

-',

. Then, the ;-th check node is connected to the j-

i=2

th variable node if and only if h -\ For example, 1 in column f and row D in the parity-check matrix in Figure 2.1 corresponds to an edge connection between variable

J,

x

x

1

p( )=TiP> " '

where /., is the fraction of edges emanating from variable nodes of degree /, p, is the 6

fraction

of edges

emanating

from

check

nodes

o f degree

m a x i m u m variable node a n dcheck node degrees,

A s a s p e c i a l c a s e w h e n e a c h o f A(x)

/, a n d d

v

and d

c

denote the

^lx{x)dx

respectively.

a n d p(x)

N

is m o n o m i a l , a n L D P C c o d e is said to

lA(x)dx b e regular.

In fact, a n L D P C

code defined with a parity-check

m a t r i x that c o n t a i n s t h e L i k e w i s e , t h e total n u m b e r o f e d g e s f r o m c h e c k n o d e point i s

same number o f Is in each column

(d )

a n d t h e s a m e n u m b e r o f I s i n e a c h r o w (d )

y

c

is

M s a i d t o b e (d , v

d) c

regular L D P C

codes.

T h e n u m b e r o f a l l I s i n / / i s e q u a l t o Md ,

E

and

c

= -

\\p{x)dx a l s o t o Nd . v

H e n c e , t h e c o d e rate R o f a regular L D P C c o d e c a n b e e x p r e s s e d a s T h u s , t h e c o d e rate o f a n irregular L D P C c o d e c a n b e obtained a s

R

=

N - M N

N jp{*)d*

N

\\^{x)dx d

c

S o m e t i m e s it i s c o n v e n i e n t t o h a v e v a r i a b l e a n d c h e c k n o d e d i s t r i b u t i o n s

from

the node

It i s s h o w n i n [1] t h a t t h e r e g u l a r L D P C c o d e s w i t h t h e b e s t p e r f o r m a n c e h a v e d, = 3 . perspective. In g e n e r a l

parity-check

cases,

where

m a t r i x H,

the number

an LDPC

variable nodes and M check nodes.

code

o f Is per column

or r o w is not constant

i s s a i d t o b e irregular.

Assume

That is, thefractions o f variable a n dcheck n o d e s o f each degree are

in the

that there a r e N

Then, thenumber o f variable nodes o f degree / is

P

X. LI

where

A

and p

Irregular

LDPC

t

t

.

,

a r e f r a c t i o n s o f v a r i a b l e a n d c h e c k n o d e s w i t h d e g r e e i, r e s p e c t i v e l y .

H i

a n d t h e total n u m b e r o f e d g e s i n t h eT a n n e r graph f r o m t h e variable n o d e point i s

codes

are more

flexible

in

their

design

because

o f the

constraints a n dhave proved to perform m u c h better then t h eregular L D P C c o d e s

8

relaxed

[7].

2.2

the following APP ratio:

ITERATIVE DECODING ALGORITHM

Pr[c-,=0|,S,] Pr[c,=l|y,i',]'

This section summarizes the iterative decoding of LDPC codes based on the belief propagation (BP) algorithm. The decoding problem consists of finding the most likely vector x such that H • x = 0, where H is a parity-check matrix defining an LDPC code. We consider binary phase shift keying (BPSK) modulated input data over the additive white Gaussian noise (AWGN) channel. Let N -tuple vector x = [x,,! , x ] be a BPSK v

modulated codeword at the transmitter, where x e {-1, 1} . The codeword bits are l

modulated according to *.=(-')" = l-2c-,,

y

where S, is the event that the bits in c satisfy the d parity-check constraints involving c

c,. If c is 0, the remaining (d - 1) bits in a given parity-check equation involving c, must (

contain an even number of Is for S, to occur. On the other hand, if c, is 1, each paritycheck constraint involving c, must contain an odd number of Is. The following Lemma will be helpful for further analysis. Lemma

where c, is the i-th bit of the codeword c = [c,,! ,c ]. Then, the received vector at the

c

2.1

[1]: Consider a sequence of m independent bits a = [«,,! ,o ] with m

Pr[o, = l] = P . The probability that a contains an even number of Is is t

v

receiver can be expressed as y = x + n,

Proof:

where y = [>•,,! , y ], y, e R, and the noise vector n = [«,, ! , n ] is composed of N v

v

independent additive noise n chosen from zero-mean Gaussian distribution with the t

standard deviation cr

We prove this by induction. If a sequence of m independent bits a = [o,,! ,

m

contains an even number of Is, the modulo-2 sum of all bits in a , designated as A„, is 0. For m = 2, we can have Pr[^ =0] = Pr[o, +a =0] 7

=^+0-^.)0-A) = I I(l-2/>)(l-2/>) +

Much like the optimal maximum a posteriori (MAP) symbol-by-symbol decoding of trellis codes, we try to compute the a posteriori probability (APP) that c, equals 1, given the received sequence y and the fact that c must satisfy some constraints. Without loss of generality, we focus on decoding the j-th bit of the codeword. First, let us think about

a]

,I Ino-2/:). +

Assume that the equation holds for m = I, -1 •

10

*K. =0] =I I

Pr[c, = 0|y,S,]_Pr[c, =0\y,] Pr [c, = 11 y, S, ] ~ Pr [c, = 11 y,

!fj(l-2A0

+

1 c, = 0,y]/Pr[S,] Pr [S, \ c, = 1, y ]/Pr [S, ]

Pr[S, ] '

_ Pr[c, = 0 | x ] Pr[5, |c, =0,y]

Then, for m = L , we get

~~ Pr[c, = 11 x ] Pr[i', k, = l , y ] ' P r [ ^ = 0 ] = Pr[^ . +fl =0] i

1

i

From Lemma 2.1, the probability of an odd number of Is in the other d -1 bits of the c

=i

+

I(l-2PrK.,=l])(l-2? )

=i

+

i(l-2(l-Pr[4_,=0]))(l-2/i)

i

y-th parity-check equation is

Since y is assumed to be statistically independent, the probability that all d parityc

check constraints are satisfied is the product of all such probabilities: From the Lemma 2.1, we are ready to get the APP ratio for c, p r

Theorem

2.1

„.

i

n

n

PrfS, |c = l,yl

[1]: Assume that the received samples in the received vector y are

i

+

.

n

o-2^='i^])

„

,

r

statistically independent. Let 5, be the event that the bits in c satisfy the d parityt

check constraints involving c . Then, the APP ratio for c, given y and S, is t

The computation of the APP ratio as given by the formula in the above Theorem 2.1 is n ^ - . . M J

^ = . | , J

i+ n n

i

_

n

(

0 - 2 Prr^=•!>'*, i) l

_

2

P

r

[

s

=

1

|

^

complex. Gallager instead provided an iterative algorithm that is exactly the BP based decoding approach nowadays. Before we give the iterative decoding algorithm, we will

]

}

need the following result. where c and y^ are the £-th bit in they-th parity-check equation involving c, and the i;

received sample corresponding c , respectively.

Lemma

2.2:

Suppose y, = x

:

+ n, l

where

n ~

(0, a

!

t

kj

Proof.

By applying Bayes' rule, we have

= 1/2, then Pr[*,=*|y]= 1

12

+e

1

)

and

Pr[x = t

+1] =

Pr[x

:

= -1]

additional information from the neighboring check nodes is available, they send the

Proof.

message along adjacent edges: P

N

X

,

x

= \ y \=

T-T^ p(y)

1

\ + e~ 2

2

2

l+e •

Subsequently, the messages are iteratively exchanged between check nodes and y

e _

"' +

y

e'

variable nodes. In Figure 2.2, we see a check node connected to d variable nodes. In c

1

each iteration, the check node will receive messages from its neighboring variable nodes, 1

process that information, and pass the updated message back to the neighboring variable •

With these results, we formulate an iterative decoding algorithm for LDPC codes, which is known as the message

passing

algorithm.

nodes: ^(o)44no-2
, , - / > ,

^TANH-'LOG"'

X

LOGTANHFI/^1

=L-2FL.

U S I N G THIS EQUATION, W E HAVE

WHERE W E HAVE DEFINED TANH[IL(R ,)J = Y

T A N H ^ I L O G ^

(x) 2

! -LOG T A N H ^ X J = L O G ^ I Y . 7

I

= '" '-,( ) W E HAVE SHOWN H O W THE UPDATED CHECK NODE M E S S A G E CAN B E CALCULATED IN THE LOG

-

N

0-2**0)) DOMAIN.

= N;

a n h

(^U))-

O N THE OTHER H A N D , THE FORMULA FOR A VARIABLE N O D E M E S S A G E UPDATE I N THE LOG

D O M A I N CAN B E EASILY DERIVED AS

T H U S , THE CHECK NODE UPDATE EQUATION CAN B E

18

COMPLEXITY

IF H

IS BROUGHT TO AN APPROXIMATE LOWER TRIANGULAR

FORM.

ALTERNATIVELY,

ENCODING CAN B E SIMPLIFIED VIA ALGEBRAIC AND COMBINATORIAL CODE CONSTRUCTION METHODS.

SUCH

"STRUCTURED" CODES CAN B E REALIZED WITH S I M P L E ENCODERS BASED ON SHIFT-REGISTER

CIRCUITS. T H I S SECTION BRIEFLY INTRODUCES THE EFFICIENT ENCODING METHOD IN [ 8 ] ,

= '•('.)+ I 4 0

SUPPOSE

A GIVEN PARITY-CHECK MATRIX H

T

SUCH THAT H-c

= 0

IS M

x N

, AND THE ASSOCIATED CODEWORD C

LET US DENOTE THE SIZE O F M E S S A G E

SYMBOLS

K

= N~M

THE

T H E FIRST TERM ON THE RIGHT-HAND SIDE REPRESENTS THE CONTRIBUTION FROM THE /-TH CHANNEL STRAIGHTFORWARD WAY O F CONSTRUCTING AN ENCODER FOR SUCH A CODE IS TO CHANGE H

INTO AN

OUTPUT, WHILE THE SECOND TERM REPRESENTS M E S S A G E S RECEIVED FROM THE NEIGHBORING CHECK EQUIVALENT LOWER TRIANGULAR FORM WITH G A U S S I A N ELIMINATION, AS SHOWN IN FIGURE 2 . 4 . NODES.

H E R E , ALL I N C O M I N G

CHECK NODE M E S S A G E S ARE TAKEN INTO ACCOUNT.

AFTER EACH

ITERATION, THE DECODER HAS TO EVALUATE THE L L R VALUES FOR EACH VARIABLE NODE AND CHECK I F

ALL THE PARITY-CHECK CONSTRAINTS ARE FULFILLED BY VERIFYING I F H • C = 0 , WHERE

^ J L ,

IF£(C,) 0

O V E R A B S C WITH ^ IS

v

1

k

Proof:

W E WILL PROVE THIS B Y INDUCTION.

A RECOVERY-ERROR PROBABILITY O F A VARIABLE NODE

' . ( R > } ( I - * ( R , F ) ) V E V WILL B E t

T H E N FOR V e ^

v 2

1( )={( -*( . 0)

E V,

t

+

1

,

29

30


(v,2^)), ,

where t/ is a degree of the survived check node of v, y 's are the neighbors of the

^(v)=^ fi-n(i-^(^,))

A(r,)

c

survived check node except for v, and the number of unpunctured nodes in the recovery

= R | i - ( i ^ K ) P

(

R

'

tree of v is 5(v) = ^5(^) =^|i-(i-^(m )) i

•

To obtain the recovery-error probability over AWGN channel, we need to define the function (x) in [15].

= «r'(i-y(v,«K"0)), where, ^ is the mean of a log-likelihood ratio message from the channel to 32

UNPUNCTURED VARIABLE NODES, d

c

IS A DEGREE O F A SURVIVING CHECK NODE O F V , y

Proposed Grouping Algorithm

'S ARE THE

d,-\

STEP

NEIGHBORS O F THE SURVIVING CHECK N O D E EXCEPT FOR V , ^ , F ( j ) ;

= S'(V),

AND y

:

e V FOR h

ARE

A

\ CW (

E L R

(T-, R ) , FOR ANY c

eV

k

EXPRESSED AS S T E P 6 I F p > 1 , W E TAKE N O D E S C" SUCH THAT V C E { T , , C

2

, C ^ J ,

DEG(C'^ < DEG(C).

IF

THERE ARE MULTIPLE SUCH NODES, W E PICK ONE FROM THEM ARBITRARY AND CALL IT c .

WHERE r

u

IS THE MOTHER CODE RATE, A ' IS THE BLOCK LENGTH O F THE MOTHER CODE.

SEQUENCE OF DESIGNED RATES, r

0

O F PUNCTURED N O D E S Np

t

< r A / - 1 , but it is hard to find a good degree v

distributions with huge portion of degree-2 nodes. Furthermore, we cannot guarantee

k

the powers of a polynomial in D . Let S = ^ y ( j ) • Thus, S represents the sum of the k

k

;«1

cycle-free among the degree-2 nodes, which is an important design rule that will be size of the submatrix formed by the /-SR, 2-SR,... and £-SR matrices. The j - t h column explained later. When N (2) = M-\, v

there will be no 0-SR nodes in the nonsystematic

43

of £-SR matrix has the following sequence: 44

s

nt)

h =D'+ '

1 for 1 < k

< d

It is obvious that ^(1)>^(2)> !

>y(d)

and

y(d)

= \

from the proof of Fact

3.1. •

/?,= 0, or 1 Then, we have

From the generation sequence in STEP 3, we can notice that £-SR matrix is composed M-r(\)

=

r(\)+R =2- (2)+R i

r

2

of only degree-2 variable nodes except for the last (d^ /)-SR matrix. M -y(\)-

y(2) = y(2) + R =2-

y(3) + R,

2

(a) I

M-y(\)-y(2)-
0

l

THIS MEANS

, IT I S POSSIBLE TO FIND A UNIQUE COLUMN j , BELONGING TO THE / - S R MATRIX

TO WHICH IT IS CONNECTED.

R,

j +S _

F R O M THE PROOF O F FACT 3 . 1 , W E HAVE S, =M

-y(i)-R , l

WHERE

= 0 OR 1 . T H E N , W E HAVE

N O T E THAT THE DEGREE DISTRIBUTION O F THE NONSYSTEMATIC PART IS

y, = A + s . - s , < r ( * ) - i + i ' - ^ t

ALREADY FIXED B Y THE CONSTRUCTION ALGORITHM ( S E E OBSERVATION 3 . 1 AND COROLLARY 3 . 1 ) .

1

w

= F R O M THE ABOVE E X A M P L E 3 . 1 AND 3 . 2 , W E CAN OBSERVE THE RIGHT DEGREE DISTRIBUTIONS O F H2.

< s

EXCEPT FOR THE LAST ROW DEGREE, W E NOTICE THAT THE N U M B E R O F DEGREE-* ROWS IS THE N U M B E R

=

O F COLUMNS IN * - S R MATRIX, WHICH IS EXACTLY y(k).

=

L E M M A 3 . 1 EXPLAINS THE DETAILS.

s

d

- s , - \

k

- s , - \

M-y(d)-R -(M-y(l)-R,)-l d

/ ( / ) - L - / ? , - L



j