This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the IEEE Globecom 2010 proceedings.
Short Erasure Correcting LDPC IRA Codes over GF(q ) Giuliano Garrammone
Bal´azs Matuz
Institute of Communications and Navigation German Aerospace Center (DLR) 82234 Wessling, Germany Email:
[email protected]
Institute of Communications and Navigation German Aerospace Center (DLR) 82234 Wessling, Germany Email:
[email protected]
Abstract—This paper investigates non-binary low-density parity-check (LDPC) erasure correcting codes suitable to guarantee reliable transmission in wireless communications systems. In particular, irregular repeat-accumulate (IRA) codes are considered, characterized by linear-time encoding complexity. The performance of non-binary IRA codes is compared with their binary counterparts on the packet erasure channel (PEC), with considerable advantages for the non-binary construction. Particularly, it is illustrated that the performance of short-block-length erasure correcting IRA codes over Galois fields (GFs) of order q > 2 approaches, under maximum-likelihood (ML) decoding, the performance of ideal maximum distance separable (MDS) codes. This is especially appealing in the context of satellite communications, where efficient codes are required to cope with small link margins. Index Terms—LDPC codes, IRA codes, packet erasure channel, Galois fields, ML decoding, satellite communications.
I. I NTRODUCTION The increasing demand for telecommunication technologies, especially for wireless communications, leads to the need of finding efficient methods to guarantee reliable delivery of data. In particular, the field of packet-level coding, where redundancy is added on higher layers, enjoys increasing popularity [1]. For upper layer coding, the information to be transmitted is divided into k packets, which are encoded into n packets by a packet-level (n, k) encoder. If the encoder is systematic, n − k redundant packets are appended to the first k packets containing the source information. Typically, each packet is further protected on the transmitter side by a physical layer error correcting code followed by an error detecting code (usually a Cyclic Redundancy Check (CRC) is used). On the receiver side error correction takes place on each packet and residual errors are detected by the CRC. Whenever the check fails, the packet is marked as lost (erasure). Thus, the channel can be modeled by a PEC, where a packet is either correctly received or lost with erasure probability ε. If enough packets are collected, the redundancy is exploited to recover the lost packets, without need of retransmissions. Packet-level erasure correcting codes (ECCs) are currently recommended in mobile terrestrial and satellite broadcasting/multicasting communication standards such as DVB-H [2] or DVB-SH [3], which adopt a (255, 191) Reed-Solomon (RS) code built over a Galois field of order 256. RS codes guar-
antee ideal recovery capabilities, at two non-negligible prices: limited flexibility in the choice of the code parameters (n, k) and strongly increasing decoding complexity with growing n. A class of ECCs which is gaining increasing interest is the one based on sparse graph representations (i.e. low-density parity-check codes [4] or Raptor codes [5]), due to their flexibility in the block-size and near-optimum performance for long/moderate block-sizes (n > 1000), at limited complexity. Recently, the capability of LDPC codes have been also demonstrated in DVB-SH field trials campaigns [6], with an efficient ML decoding software implementation sustaining minimum data rates of 1.5 Gbps. However, when moving to short block lengths (i.e.,n < 500) binary LDPC and binary Raptor codes suffer in terms of performance, showing a loss w.r.t. the performance of ideal codes. We will show within this paper that a nearly optimum performance (in terms of erasure recovery capability) can be achieved in the short block size regime by building nonbinary LDPC codes. More specifically, we focus on IRA codes because they are particularly appealing thanks to their simple encoding structure. Furthermore, we will demonstrate that the construction of IRA codes over GF(q), under ML decoding, lowers the error-floor and improves the robustness w.r.t. bursty erasures, without the need of permuting the paritycheck matrix (e.g. [7]). Numerical simulations will compare the performance of the designed codes with the performance of MDS codes, for the memoryless erasure channel. The structure of the paper is the following. First, Section II gives the principle of packet-level coding by means of an example. In Section III we provide an overview of erasurecorrecting binary IRA codes, pointing out their limitations for short block lengths. By focusing on their simple structure, we will recall efficient encoding/decoding algorithms. In Section IV, a simple way to build non-binary IRA codes is presented, together with the modified encoding/decoding schemes over GF(q). Section V provides numerical results, whereas final conclusions are given in Section VI. II. P RINCIPLE OF PACKET-L EVEL C ODING Consider an example of packet-level ECC scheme: the multi-protocol encapsulation (MPE) forward error correction (FEC) protocol [2] adopted in the DVB-SH standard. A
978-1-4244-5637-6/10/$26.00 ©2010 IEEE
This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the IEEE Globecom 2010 proceedings.
(255, 191) RS code is used at link layer to recover lost IP datagrams within a time slice burst. On the encoder side, the so-called application data table (ADT)1 is filled-up columnwise, with u IP datagrams. If required, padding is used (see Fig. 1). Each row of the ADT table (cf. Fig. 1) is then encoded by a (255, 191) RS systematic encoder operating on symbols from GF(256), producing thus 64 redundant bytes that are appended to the respective row, leading to the so-called RS table. Thus, each column of the ADT and RS table is a packet of size W bytes that is a codeword symbol. Once all the rows have been processed, the IP datagrams are encapsulated (as MPE sections) in the transport stream, as well as the columns of the RS tables (as MPE-FEC sections). On the receiver side, the sections containing errors are detected by a CRC and discarded. The RS erasure decoder is then employed to recover the lost sections. (255,191) RS codeword 64 columns (RS table) -
U
6 1 byte ?
6
Usually, a constant check node (CN) degree is fixed, together with variable node (VN) degree distributions constructed according to the differential evolution algorithm [10]. Consider as an example the following degree distributions Λ(x) = 0.55x2 + 0.189x3 +0.004x4 +0.1405x5 +0.0534x10 + +0.0165x11 +0.00189x24 +0.02881x25 +0.0112x100 , P (x) = x10 . (1) The (n − k) × n parity-check matrix H of a systematic IRA code can be written as concatenation of two matrices as H = [Hu |Hp ], where the (n − k) × k matrix Hu corresponds to the systematic part of the codeword2 c, whereas the (n − k) × (n − k) matrix Hp corresponds to the parity part of c and is a double-diagonal matrix. Given the degree distributions, the binary parity-check matrix H is randomly designed according to certain girth optimization techniques [11]. An example of parity-check matrix for a short IRA code of block size n = 256 and code rate R = nk = 0.5, with degree distributions given by (1), is depicted in Fig. 2, where each point represents a non-zero element.
IP u
IP 1
0 W bytes
IP 1 IP 1
IP 2
IP u
20 K
40
IP 2 ?
60
191 columns (ADT )
80 padding
100
Fig. 1.
Encoding following the MPE-FEC protocol adopted in DVB-H [2].
120 0
50
100
150 nz = 1234
200
250
III. B INARY IRA CODES An (n, k) IRA code has a partially-structured parity-check matrix. As every LDPC code, it can be represented by a n−1 bipartite graph [8], with a set of variable nodes V = {Vj }j=0 , n−k−1 a set of check nodes C = {Ci }i=0 , and a set of edges E connecting variable nodes to check nodes. The degree djv (dic ) of the variable node Vj (check node Ci ) is defined as the number of edges emanating from Vj (Ci ). Note that djv corresponds to the Hamming weight of the j-th column of the parity-check matrix H and dic corresponds to the Hamming weight of the i-th row of H. Each edge connection is associated with a non-zero element in the parity-check matrix H, which is the element 1 for the specific case of GF(2). For irregular LDPC codes the variable and check node degrees are described by so-called degree distributions. The degree distributions are usually represented in polynomial form [9]. Let us denote by Λd (Pd ) the fraction of variable (check) nodes of degree d and dv (dc ) the maximum variable (check) node degree. Thus, we denote the variable (check) node degree dv dc Λd xd (P (x) = Pd xd ). distribution polynomial Λ(x) = d=1
d=1
1 The ADT is a W × 191 bytes matrix, where W can be set to 256, 512, 768 or 1024.
Fig. 2. Parity-check matrix of a (256, 128) IRA code. The number of nonzero elements is 1234.
The double-diagonal structure leads to efficient encoding schemes [9]. The encoder structure is shown in Fig. 3. The k u u
-
HT u
k
Fig. 3.
? - mn-k
D p -
Binary IRA encoder.
information symbols u are first multiplied by the k × (n − k) low-density matrix HTu and the resulting n − k symbols are given as input to the accumulator, producing the n − k parity symbols denoted by p. The systematic codeword is obtained by concatenating u and p, thus c = [u|p]. D denotes one symbol delay. 2 Through the paper, we assume that each codeword symbol is a packet of fixed-size W bytes, and the vector notation denotes row vectors.
978-1-4244-5637-6/10/$26.00 ©2010 IEEE
This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the IEEE Globecom 2010 proceedings.
A. Decoding of binary LDPC codes Finite short/moderate length (n, k) LDPC codes show a gain loss w.r.t. the performance of ideal MDS codes (Singleton bound), under iterative (IT) decoding, and the error-floor phenomenon arises. It is well known that there exist sets of VNs representing stopping sets for IT decoding, but not for optimal ML decoding. Thus, given an IT decoder failure, an ML decoder can try to solve the residual stopping. Usually, ML decoding provides optimal performances at moderate speed, whereas IT decoding provides sub-optimal performances at high-speed. In this context, a high-speed hybrid IT/ML decoder, with ML performance approaching the Singleton bound, has been recently developed for binary LDPC codes [12]. ML decoding on the PEC is equivalent to solving the linear system of equations xK HTK = xK HTK ,
(2)
where xK (xK ) is the set of erased (correctly received) encoded symbols and HK (HK ) the submatrix composed of the corresponding columns of the parity-check matrix H. The linear system in (2) can be solved by means of Gaussian elimination (GE) performed on the matrix HTK : the complexity is cubic in the codeword length n. The efficient hybrid IT/ML decoder in [12], with ML performance, is based on structured GE, which exploits the sparseness of the parity-check matrix of the code and relies on advanced pivoting algorithms. The following procedures are performed: 1. IT decoding step. Equations with only 1 unknown are solved by an IT decoder. 2. Triangulation and pivoting step. If IT decoder fails, the remaining matrix HK is put in a lower triangular form by only permuting rows and columns. If the triangulation gets stuck, a pivoting strategy is applied, meaning that columns of the non-triangulized matrix are selected and moved to the side of the matrix. The encoded symbols corresponding to the moved columns are referred to as pivots. 3. Zero-out step. The matrix below the main diagonal of the lower-triangular matrix is transformed into a zero matrix by rows additions. 4. Gaussian elimination. A brute-force GE is applied to solve the system described by the pivot variables only. 5. IT decoding step. If GE succeed, an IT decoding step is sufficient to recover the remaining unknowns, which are a linear combination of pivots. Rows-permutations and rows-additions are also performed on the known term of the system, given by the right hand side of (2). Note that the matrix HK is sparse. Thus, the overall complexity is dominated by Step 4, where the linear system described by the pivot variables is dense, due to the operations in Step 3. In particular, Step 4 has complexity O(α3 ), where α is the number of pivots. Indeed, the algorithm is efficient as long as the number of pivots is kept small. Sophisticated pivoting strategies have been investigated in [12].
Despite advanced decoding techniques/code design, binary codes whose length n is in the order of a few hundred of symbols do not approach the Singleton bound, even under ML decoding, except at very high error rates. Section V will show the gap in performance w.r.t. the Singleton bound for a binary short near-optimum designed IRA code, under ML decoding. IV. N ON - BINARY IRA CODES In order to improve the performance in the short-code-length regime, we move to higher order Galois fields while keeping ML decoding. The non-binary design is aimed to increase the rank of HTK in (2), that raises the probability of successful decoding. By contrary, the performance of IT decoding only depends on the graph structure of the code and not on the order q of the Galois field. In fact, the connections in the bipartite graph of a non-binary code remain invariant w.r.t. those of their binary counterpart3 . As a result, whenever IT decoding fails for GF(2), it also does for GF(q), q > 2. Thus, moving to higher order GFs is only beneficial with ML decoding. Furthermore, binary IRA codes tend to have problems in presence of erasure bursts. The metric that we use to evaluate the robustness w.r.t. a single erasure burst is the Maximum Tolerable Burst Length (MTBL), which represents the maximum length of an erasure burst that can always be recovered, independently of its position within the codeword. The non-binary design increases the MTBL of the code as discussed in Section V. In the remaining part of this section we will illustrate a simple technique to construct, encode and decode non-binary IRA codes able to approach the Singleton bound, even for short code lengths (e.g. n < 500). A. Construction and encoding of IRA codes over GF(q) The construction of non-binary IRA codes goes through the design of their binary counterpart as described in Section III. Once the design of the matrix H is complete, each nonzero element of H in GF(2) is replaced with an element in q−1 q−1 {βi }i=0 \{β0 }, where {βi }i=0 is the set of elements in GF(q) and β0 is the null additive element of the finite field. The MTBL of the non-binary code is measured, and a new paritycheck matrix over GF(q) is built until the maximum achievable MTBL is reached, for the given binary structure. Thus, both matrices Hu and Hp have elements in GF(q). In particular, the double-diagonal matrix Hp is ⎤ ⎡ (0) 0 ··· 0 dm 0 0 ⎥ ⎢ d(1) d(1) 0 0 ··· 0 m ⎥ ⎢ s ⎥ ⎢ (2) (2) ⎥ ⎢ 0 d s dm 0 ··· 0 ⎥ , (3) Hp = ⎢ .. . . .. .. ⎥ ⎢ .. . . ⎥ ⎢ . . . 0 ⎥ ⎢ (n−k−2) ⎦ ⎣ 0 0 · · · d(n−k−2) dm 0 s 0
0
(i)
0
···
(n−k−1)
ds
(n−k−1)
dm
(i)
where dm , i = 0, . . . , n − k − 1 and ds , i = 1, . . . , n − k − 1, are random elements of GF(q)\{0} and belong to the main 3 Note that each incoming message of a CN has to be multiplied by the corresponding non-zero element of H.
978-1-4244-5637-6/10/$26.00 ©2010 IEEE
This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the IEEE Globecom 2010 proceedings.
diagonal and secondary diagonal, respectively. We further (0) define ds 0. The encoding scheme can be simply modified as depicted in Fig. 4 and is summarized within the following steps: 1. The k information symbols u are multiplied by the lowdensity matrix k×(n−k) HTu , generating n−k symbols, denoted u . 2. u enters a rate-1 recursive encoder and its n − k output symbols define the parity vector p. In particular, the first element of u is multiplied by the inversemultiplicative of the first element in the main diagonal of Hp , producing the first encoded parity symbol
−1 (0) . The remaining parity symbols p(0) = u(0) dm p(i) , for i = 1, . . . , n − k − 1, are generated −1 with the (i) (i−1) (i) (i) (i) dm recursion p = u + ds p . 3. The systematic codeword c is obtained by concatenating u and p, that is c = [u|p] as for the binary case. The scheme of Fig. 4 is a straightforward extension of the scheme in Fig. 3, but here each symbol has elements in GF(q) and each operation is carried out in GF(q). Additions can be carried out by simple bit-wise XORs, whereas multiplications can be implemented by lookup tables. More efficient techniques to perform multiplications in GF(q) can be found in [13], where the polynomial representation for the elements of GF(q) is exploited. (i)
ds
u
u -
HT u
u −1 (i) dm
? ? - n
D p -
Fig. 4.
Non-binary IRA encoder.
B. ML decoding of LDPC codes over GF(q) ML decoding of non-binary codes can be performed on the erasure channel by solving the linear system in (2), but each algebraic operation on the non-binary matrix HTK and on the known term has to be carried out in GF(q) instead of GF(2). The efficient algorithm introduced in [12], described in Section III-A can be extended to higher order GFs and goes through the following steps: 1. IT decoding attempt is done to solve parity-check equations with 1 unknown, but before summing up the known terms, each codeword symbol is multiplied by an element in GF(q) according to the general parity-check equation with 1 unknown in GF(q) x(0) β (0) ⊕ · · · ⊕ xβ (x) ⊕ · · · ⊕ x(n−1) β (n−1) = 0, (4)
where x is the erased symbol, β (x) is its known multiplicative term in the parity-check matrix, x(i) is the known codeword symbol which is multiplied by the given β (i) in the parity-check matrix, with i = 0, . . . , n − 1, i = x, and ⊕ denotes the sum operation in GF(q). Note that most of the β (i) coefficients are zero, since we consider LDPC equations. 2. Upon an IT decoding failure, smart GE is invoked through triangulation and pivoting. It is worth to point out that triangulation and pivoting over GF(q) has the same complexity than in GF(2) since only the positions of the elements are involved and no arithmetic operations are performed. 3. Step 3 in Section III-A is modified involving also multiplications in GF(q) in order to put the remaining system in a diagonal form. 4. The system described by the pivots has to be solved applying GE. This step includes arithmetic operations in GF(q). 5. If the pivot VNs are solved then, as in Step 5 in Section III-A, IT decoding over GF(q) is capable to recover the remaining unknown codeword symbols. Note that Step 4 is critical from a complexity perspective, since the system to be solved is dense, as for the binary case. Its complexity is cubic in the number of pivot variables α, that can be kept small with proper pivoting strategies, not dependent on the order q of the field. V. N UMERICAL R ESULTS A binary (256, 128) systematic IRA code has been designed according to Section III with the degree distributions given in (1) (the parity-check matrix is depicted in Fig. 2). The code has been then extended to Galois fields of various order. Monte Carlo simulations were performed on the memoryless erasure channel, collecting 100 decoding failures for each erasure probability. ML decoding for GF(q) has been adopted. Fig. 5 shows the average number of pivots versus the channel erasure rate ε, when smart GE is applied. The results are averaged over 1000 trials, in which IT decoder does not succeed and smart GE is required. Note that, for this short code, the average number of pivots decreases and becomes close to 1 with decreasing ε, and even for values of ε in the order of 0.4. Fig. 6 shows the codeword error rate (CER) vs. channel erasure probability ε for the designed IRA code on various finite field orders q ≥ 2. The performances are compared to the Singleton bound, which represents the performance of an ideal MDS code over the erasure channel. In the region where the error-floor phenomenon does not appear for the binary code (ε ≥ 0.45), a gap between the performance of the optimum IRA and the Singleton bound is evident. However, the gap is filled when moving to higher order GFs, already with an order 4. The error-floor phenomenon appears in the binary case for CER ≤ 10−1 , whereas the code design on higher order GFs significantly lowers it. In particular, the error-floor starts to
978-1-4244-5637-6/10/$26.00 ©2010 IEEE
This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the IEEE Globecom 2010 proceedings.
VI. C ONCLUSIONS
8 7
Average number of pivots
6 5 4 3 2 1 0 0.35
0.4
ε
0.45
0.5
Fig. 5. Average number of pivots vs. erasure rate for near-optimum (256, 128) IRA code.
appear for GF(256) at CER ≤ 10−3 and at e.g. ε = 0.4 one order of magnitude in the CER is gained. The error-floor is lowered under ML decoding, due to the increase in the minimum distance, that also leads to a better robustness w.r.t. bursty erasures. The maximum length of a single erasure burst always recovered by this binary code has been measured to 71, while the design in GF(q), for q ≥ 4 permits to achieve a MTBL of 79. We point out that the comparison among various order GFs is fair, since we apply the erasure correcting code at packetlevel. The amount of data carried in a codeword on the channel is fixed to W × n bytes, but the way how the data is processed changes from the bit-wise approach of GF(2) to, e.g. a bytewise approach in GF(256). 0
10
−1
10
−2
CER
10
−3
10
−4
10
Singleton bound GF(2) GF(4) GF(16) GF(64) GF(256)
−5
10
−6
10
0.35
0.4
0.45 ε
0.5
0.55
Fig. 6. Codeword error rate vs. erasure probability for near-optimum (256, 128) IRA code, on various order GFs.
In this work, the design of small-block-size IRA codes over GF(q), with q ≥ 2 has been considered. Numerical simulations have been carried on the packet erasure channel, under ML decoding. The performance of non-binary IRA codes is significantly improved in comparison with their binary counterparts, for both high and low channel erasure probabilities. Performance of ideal MDS codes is approached in the high erasure probability region, whereas for low erasure probabilities the error-floor is significantly decreased. Moreover, the robustness to bursty erasures (MTBL) is improved. The complexity in decoding is kept limited by using algorithms with advanced pivoting strategies, which do not depend on the order q of the field. The proposed approach might be an appealing solution to add flexibility in wireless broadcasting/multicasting standards such as DVB-H/SH. ACKNOWLEDGEMENT The authors wish to thank Dr. Enrico Paolini from University of Bologna, Italy, for his valuable comments on this manuscript. R EFERENCES [1] J. Byers, M. Luby, and M. Mitzenmacher, “A digital fountain approach to reliable distribution of bulk data,” IEEE Journal on Selected Areas in Communications, vol. 20, no. 8, pp. 1528–1540, Oct. 2002. [2] G. Faria, J. Henriksson, E. Stare, and P. Talmola, “DVB-H: Digital Broadcast Services to Handheld Devices,” Proceedings of the IEEE, vol. 94, no. 1, pp. 194–209, Jan 2006. [3] “Framing structure, channel coding and modulation for Satellite Services to Handheld devices (SH) below 3GHz,” Digital Video Broadcasting (DVB),” Blue Book, 2007. [4] E. Paolini, M. Varrella, M. Chiani, B. Matuz, and G. Liva, “LowComplexity LDPC Codes with Near-Optimum Performance over the BEC,” in Proc. 4th Advanced Satellite Mobile Systems Conference, Bologna, August 2008, pp. 274 – 282. [5] M. Luby, M. Watson, T. Gasiba, T. Stockhammer, and W. Xu, “Raptor codes for reliable download delivery in wireless broadcast systems,” in Proc. of IEEE Consumer Communications and Networking Conf., vol. 1, Jan. 2006, pp. 192–197. [6] A. B. Alamanac, P. Burzigotti, R. D. Gaudenzi, G. Liva, H. N. Pham, and S. Scalise, “In-depth analysis of the satellite component of DVBSH: scenarios, system dimensioning, simulations and field trials results,” International Journal on Satellite Communications and Networks, 2009. [7] E. Paolini, G. Liva, B. Matuz, and M. Chiani, “Generalized IRA erasure correcting codes for hybrid iterative/maximum likelihood decoding,” IEEE Communications Letters, vol. 12, no. 6, pp. 450–452, June 2008. [8] R. Tanner, “A recursive approach to low complexity codes,” IEEE Transactions on Information Theory, vol. 27, pp. 533–547, Sept. 1981. [9] H. Jin, A. Khandekar, and R. McEliece, “Irregular repeat-accumulate codes,” in Proc. International Symposium on Turbo codes and Related Topics, Sept. 2000, pp. 1–8. [10] M. A. Shokrollahi and R. Storn, “Design of efficient erasure codes with differential evolution,” in Proc. IEEE International Symposium on Information Theory, 2000, p. 5. [11] X. Y. Hu, E. Eleftheriou, and D. M. Arnold, “Progressive edge-growth Tanner graphs,” in Proc. IEEE Globecom, San Antonio, Texas, Nov. 2001, pp. 995–1001. [12] G. Liva, B. Matuz, E. Paolini, and M. Chiani, “Pivoting algorithms for maximum likelihood decoding of LDPC codes over erasure channels,” in Proc. IEEE Globecom, Honolulu, HI, USA, pp. 1-6, Nov./Dec. 2009. [13] F. MacWilliams and N. Sloane, The Theory of Error-Correcting Codes, 6th ed., North-Holland, Ed. Elsevier Science Publishers, 1977, vol. 16.
978-1-4244-5637-6/10/$26.00 ©2010 IEEE
