The Design of the Maximum-Likelihood Decoding ... - CiteSeerX

The Design of the Maximum-Likelihood Decoding Algorithm of LDPC Codes over BEC. Ki-Moon Lee

Hayder Radha

Dept. of Math, Michigan State University E. Lasing, MI-48824, USA Email: [email protected]

Dept. of ECE, Michigan State University E. Lansing, MI-48824, USA Email: [email protected]

Abstract— We improve the design of the Maximum Likelihood Decoding Algorithm (MLDA) proposed in [1] for the development of LDPC codes. Furthermore, we apply the improved design to the decoding of PEG LDPC codes [3] over Binary Erasure Channels (BEC). We also demonstrate clear improvements over current LDPC decoding algorithms.

I. I NTRODUCTION AND BACKGROUNDS Typically in a Binary Erasure Chanel (BEC) based LowDensity-Parity-Check (LDPC) codes, a code space C is defined as a kernel space Ker(H) = {α ∈ (Fs2 )n |H · αT = 0}, where H is an m × n (0, 1)-matrix over F2 . When α is transmitted over a BEC, the overall code vector α can be represented by α = [αe¯, X] where αe¯ and X represents the received and the lost part of α, respectively. Let ne and ne¯ denote the number of elements in X and αe¯, respectively. Associating the columns of H with [αe¯, X], let [N M ] be the rearrangement of H so that the kernel space constraint HαT = 0 is now modeled as the consistent linear system M X T = βT ,

β T = N αeT¯ .

(1)

When H is designed with a good degree sequence, for examples the sequences in [2], [3], then with high probability system (1) can be solved by the Message Passing Algorithm (MPA) [5] which is equivalent to a lower triangulation of M by means of row and column permutations of it. With short block lengths n, however, the triangulation of M is not guaranteed as p = nne approaches to the ideal limit γ = m n. For LDPC codes decoding, Burshtein and Miller suggest the MLDA [1] that takes full advantage of the efficient encoding algorithms of LDPC codes pioneered by Richardson and Urbanke [2]. The MLDA exploits four routines as following: 1) Approximate Lower Triangulation Algorithm (ALTA): Applying the greedy algorithms [2] on H, obtain a ¯ = permutation matrix pair (P, Q) of M such that M A B ] is in an approximate lower triangular P M QT = [ C D form as shown in Fig 1-(a). Thus system (1) is read as   T  T A B XR βu ¯X ¯ T = P β T . (2) ⇔M T = C D XR βlT ¯ 2) BackhSubstitution i of References (BSR): Multiplying by B −1 0 S = −DB −1 I , transform system (2) into  T  T  A¯ I XR β¯u T ¯ ¯T (3) T = ¯T ⇔ [S M ]X = SP β , C¯ 0 XR β ¯ l

where C¯ = C − DA¯ and A¯ = B −1 A (see Fig. 1-(b)). 3) Gaussian Elimination (GE): Recover XR by GE on ¯ T = β¯T . CX R l T 4) Final Forward Substitution (FFS): Recover XR¯ by XR ¯ = T T ¯ ¯ AXR + βu . In [13, Ex-50.12], Mackay also suggests the encoding algorithm in [2] for LT codes h [4] which i is essentially smae with I 0 the MLDA except S = −DB in the BSR. The MLDA −1 I for Raptor codes [8] can be found in the US paten [9]. Another version of MLDA using pseudo-values on triangulation of M in a bit-level can be found in [10]. The core of the MLDA is in the novel combination of the ALTA and the BSR by S that reduces system (1) into a very ¯ R = β¯T in 3 GE by means of a partial GE small system CX l ¯ over the triangular block [ B ]. Let q×r be the dimension on M D of C¯ where q = (γ − p)n + r. Of particular interest of the ALTA is in the H designed by good degree sequences [2], [3]. Even with short block lengths n, say 2, 000 ≤ n ≤ 20, 000, we show, based on extensive and comprehensive simulations, that the fraction nr is close to 0. Hence, the complexity of the GE on C¯ (O(r2 q)) is very efficient compared to O(n3 ). (see [2, p4]). Simulations also exhibit that C¯ feasibly has its full column rank r for ne very close to m. Even the rank deficient ¯ > 0, the GE on C¯ case, say the deficiency η = r − rank(C) can identify the precise η free variables of XR . Thus in the worst situation, if the retransmission is available on the BEC, the unique solution of (1) is obtainable by the retransmission of the symbols that correspond to the free variables only. In this paper, we transform the linear system via (P, Q), and the implementation of the MLDA is based on the proposed equivalent systems. First of all, we identify system (2) from (1) by looking at ¯ is not necessary. (P, Q), hence the explicit construction of M Permuting columns of M by Q, denote M QT = [MR ; MR¯ ], A ] and P M B ¯ = P T SP . We where P MR = [ C ¯ = [ D ]. Let S R identify S¯ as a series of pivoting process on M over columns ¯ over the columns of [ B ]. Thus, by of MR¯ , as S is on M D ¯ R = SM ¯ R and M ¯ R¯ = SM ¯ R¯ , (3) is interpreted as M ¯ R; M ¯ R¯ )X ¯ T = Sβ ¯ T. [M

(4)

B ] = [ I ], the explicit computation of We note that, since S[ D 0 ¯ R¯ is redundant. M

XR

XR¯

A B C D ¯ by ALTA (a). M

XR

XR¯

XR XR¯

A¯

I

A¯

C¯

0

L

U

0

separated MLDA and the MLDA in [1]. In IV, we report the simulation results tested with 21 -rate PEG codes [3] of block lengths n, 2 · 103 ≤ n ≤ 2 · 104 , for the following scenarios i) the performances of the MLDA and the MPA; ii) the complexities of the MLDA; iii) the number of columns ¯ We summarize the paper in V. of C.

I 0 0

¯ by BSR (c). L\U by GE (b). S M Fig. 1.

¯ The MLDA on M

¯ = r, the GE on C¯ is a factorization Second, when rank(C) process [12, ch.7] that returns a r × r nonsingular sub-matrix C¯r = LU of C¯ such that L and U is in a lower and upper triangular form as shown in Fig. 1-(c), respectively. Hence T from C¯r XR = β¯rT , T XR = (LU )−1 β¯rT ,

(5)

¯ R via P , the GE on C¯ Identifying the rows of C¯ from M ¯ R in such a way that, while can be directly performed on M performing the GE process, entries of L and U in (5) are stored ¯ R . At the end, the GE returns an updated M ¯ R with a set in M of permutation pair (σr , τr ) so that rows of L and U can be ¯ R by looking at (σr , τr ). We interpret the identified from M ¯U ¯ , thus by looking at (σr , τr ), XR LU as an m × m matrix L can be obtained from ¯U ¯ )−1 S¯ · [MR ; MR¯ ]X ¯ T = (L ¯U ¯ )−1 S¯ · β T , (L

(6)

Discarding the equations in (1) that becomes null in (6), we assume that system (1) consists of ne equations. Based on (6), we separate the MLDA into two major steps: 1) pre-decoding: Bring systme (6) as follows: ¯ R by the BSR, a) Obtain (P, Q) by the ALTA on H, M ¯ ¯ ¯ and LU with (σr , τr ) by GE on MR ; b) Discard null equations in (6). 2) post-decoding: ¯ −1 L ¯ −1 S¯ · β T in (6); a) Compute β¯T = U b) Recover each xi of XR from the β¯ via (σr , τr ); c) Recover each xi of XR¯ via (P, Q) by using sparser of the following: T T T xi = β¯i + A¯i XR or Bi XR ¯ = βi + Ai XR .

(7)

The alternative recovery in 2c) is quite reasonable, since A¯ may not be sparse. Particularly, with a large symbol size s T ¯T ¯ T (= |βi |), the recovery of XR by XR ¯ = βl + AXR alone (as in 4 FFS) may result a serious degradation to the efficiency of the MLDA. We will see such degradation in IV. The rest of the paper is organized as follows. In Section II, we clarify systems (4) and (6) of the proposed MLDA in detail, and provide an exemplary algorithm of the routines of the MLDA. In III, we estimate the computational complexity of the MLDA in terms of the number of sign/bit flips of the pre-decoding and symbol additions of the post-decoding. We also compare the number of symbol additions made by the

II. T HE MLDA D ESIGN . In Subsection II-A, the BSR and GE of the proposed MLDA will be clarified in detail to derive system (6). In IIB, we provide an exemplary algorithm for the routines of the proposed MLDA. Let us introduce several notations that are used in the remainder of the paper. For a given (0, 1) matrix K, we denote kij , Ki , and K j as the (i, j)th entry, the ith row, and the k th column of K, respectively. We use the notation 1ij to indicate the (i, j)th 1 in K. We denote |Ki |, |K j |, and |K| as the number of 1’s of Ki , K j , and K, respectively. Corresponding to the (αe¯, X) in (1), E and E¯ denotes the set of indices of symbols in X and αe¯, respectively. A. Basic Linear Algebra on Fn2 As a natural extension of the MPA, the ALTA on H can be designed to obtain a set of successive pairs in [m] × E, (σl , τl ) = (i1 , jr+1 ) ≻ · · · ≻ (il , jr+l ),

r = n − l,

(8)

with which each (s, t)th entry of the triangular block B is identified by reading the (is , jr+t )th entry of H. Once (σl , τl ) ¯ and is obtained , [m] and E is separated into E = (R, R) ¯ [m] = (T , T ) such that ¯ = τl = {jr+1 , . . . , jr+l }, R = E \ τl = {j1 , . . . , jr }, R ¯ T = σl = {i1 , . . . , il }, T = [m] \ σl = {il+1 , . . . , im }. Extending (σl , τl ) into the permutation pair (σ, τ ) such that σ : [m] 7→ [m], σ(ik ) = k and τ : E 7→ E, τ (jk ) = k, P (QT ) is formed by permuting rows of Im×m (columns of ¯ = P M QT Ine ×ne ) in the order of σ (τ respectively). Let M as shown in Fig. 1-(a). Then for each (s, t) ∈ [m] × E, the ¯ is identified from the (is , jt )th element (s, t)th element of M XT T T ¯ of H. Let X = Q X T = [ XR T ] where XR = [xj1 , . . . , xjr ] ¯ R

and XR¯ = [xjr+1 , . . . , xjr+l ]. Then by QT = Q−1 , system P M X T = P β T is now read as (2). B 0 ]. Since it is a lower triangular Let us consider S −1 = [ D Q1 I(k) matrix over F2 , S = k=l S = S (l) · · · S (1) where each (k) ¯ k . With the S is formed by replacing (Im×m )k with M ¯ product form, S M in (4) can be computed by the iteration ¯ := S (k) M ¯ , k = 1, 2, . . . , l. Now with (R, R), ¯ we set M T [MR ; MR¯ ] = M Q such that MR = [H j1 , . . . , H jr ],

MR¯ = [H jr+1 , . . . , H jr+l ]

(9)

Interpreting each S (k) as the elementary matrix S¯(k) = P T S (k) P , which is formed by replacing the (Im×m )ik with the column H jr+k , S is transformed into S¯ as below S¯ = P T SP =

1 Y

k=l

P T S (k) P =

1 Y

k=l

S¯(k) .

(10)

Hence system (3) is transformed into (4) in a product form # " 1 1 1 Y Y Y (k) (k) ¯ ¯ S¯(k) · β T . (11) S · MR ; S · MR¯ X T = k=l

k=l

k=l

¯ R; M ¯ R¯ ] = [SM ¯ R ; SM ¯ R¯ ] as in (4). Notice that since Let [M T I ¯ ¯ R¯ is enough to set up system MR¯ = P [ 0 ], computation of M (4) and is accomplished by the iteration MR := S¯(k) MR ,

k = 1, 2, . . . , l.

(12)

We now construct MR in a ternary form {−1, 0, 1} for the GE on MR later. Iteration (12) is described in Algorithm 2. ¯ = r, the GE on C¯ can be designed Assuming that rank(C) to obtain a r × r nonsingular matrix LU in a product form Q1 Qr (k) −1 (k) product form L−1 = L and U = U , k=1 k=r (k) (k) where each L (U ) is the elementary matrix formed by replacing the row (Ir×r )k with the row Lk (Uk respectively). Therefore, XR can be recovered by XR =

r Y

U (k) ·

k=1

1 Y

L(k) βrT .

(13)

k=r

The GE on C¯ is also naturally shifted on the set of rows ¯ R )i |i ∈ T¯ }, as a form of pivoting iteration on M ¯ R . At {(M ¯ the end, the GE returns an updated MR that consist of −1, 0, 1 with a set of successive pairs (σr , τr ) = (s1 , t1 ) ≻ · · · ≻ (sr , tr ) ⊂ T¯ × R.

(14)

After the GE, an entry lij (uij ) of L (U respectively) can be ¯ R where si ∈ σr identified from the (si , tj )th entry of the M ¯ R is described in Algorithm 3. and tj ∈ τr . The GE on M We observe that, while computing XR by (13), each 1k,j in L(k) or U (k) corresponds to the symbol addition (βl )k := ¯ T . This interpretation is also (βl )k + (βl )j . Let β¯T = Sβ ¯ R via (σr , τr ). In Algorithm 3, each 1kj naturally shifted on M in L(k) (U (k) ) is recoded as the −1sk ,tj (1sk ,tj respectively) ¯ R )s . Therefore, the −1s ,t or 1s ,t corresponds to in (M k j k k j the symbol addition β¯sk := β¯sk + β¯sj via (sk , tk ), (sj , tj ) ¯ (k) (U ¯ (k) ) is the elementary ∈ (σr , τr ). This implies that L matrix formed by replacing those −1sk ,tj (or 1sk ,tj ) as the 1sk ,sj in the row (Im×m )sk . Hence system (6) is verified with ¯ −1 = L

1 Y

¯ (k) , L

k=r

¯ −1 = U

r Y

¯ (k) . U

Algorithm 1: The ALTA over H ¯ E). Output: (σl × τl ) Input: H, (E, /%%/ foreach j ∈ E¯ do Sign-Flip with H j ; /%%/ while E = 6 ∅ do if R = ∅ then do Referencing; else do MPA; return (σl × τl ); exit the ALTA; /%%/ MPA(): while R 6= ∅ do foreach (i, j) ∈ R; do if j ∈ E then E := E \ j; do Sign-Flip with H j ; (σl , τl ) := (σl , τl ) ∪ (i, j); R := R \ (i, j); Referencing(): choose ∃Hi |Hi | = min{|Hs | > 0}; while |Hi | > 1 do choose ∃j such that 1ij ∈ Hi ; do Sign-Flip with H j ; Sign-Flip(): foreach i such that 1ij ∈ H j do flip the 1ij into −1; |Hi | := |Hi | − 1; if |Hi | = 1 then identify the pair (s, t) such that ∃!1st ∈ Hi ; R := R ∪ (s, t); Q1 Algorithm 2: BSR: k=l S¯(k) · MR ¯R Input: MR Output: M foreach (ik , jr+k ) ∈ (σl , τl ) as in the order do /%%/ foreach 1i,jr+k ∈ H jr+k , i 6= ik do (MR )i := (MR )i + (M )ik ; ¯R return MR ; //