Implementation and performance evaluation of LT and Raptor codes ...

Implementation and performance evaluation of LT and Raptor codes for multimedia applications Pasquale Cataldi, Miquel Pedr`os Shatarski, Marco Grangetto, Enrico Magli Dept. of Electronics, Politecnico di Torino C.so Duca degli Abruzzi, 24 - 10129 Torino, Italy∗ pasquale.cataldi(marco.grangetto,enrico.magli)@polito.it Abstract Digital fountains have changed the standard transmission paradigm. A digital fountain can encode and transmit an unlimited number of data packets until every user gets enough information to guarantee correct decoding. Multimedia broadcasting, emerging peer-to-peer applications are only two examples of many other scenarios where digital fountains can be successfully applied. In this paper practical implementation issues of two class of digital fountain codes, namely LT and Raptor codes, are analyzed. Moreover, the performance in terms of coding and decoding complexity is measured experimentally.

1 Introduction The main idea of a Digital Fountain (DF) is as follows. The encoder is represented by a “fountain” that produces a potentially infinite amount of water drops; every drop represents an encoded packet, which has the same size as a source symbol. At the receiver side, the decoder is represented by a “bucket”; it collects drops that have spurted from the fountain until the bucket is full. Finally, the decoder is able to recover the original information, independently of which drops it has collected [4, 2]. This is very useful for data transmission over networks subject to packet erasures, because the receiver needs to collect a fixed amount of packets, regardless of what packets have been lost. Therefore, DF codes can be classified as erasurecorrecting codes. DFs are random codes that can produce a potentially infinite amount of encoded symbols on-the-fly. It follows that such codes are optimal for any erasure channel because it is only necessary to receive enough symbols to decode with high probability the source information. The number of ∗ This work was partially supported under “MEADOWS” project granted by the Torino-Wireless initiative.

symbols to be collected from the receiver is N = (1 + ε) k, where k is the number of source symbols and ε is called decoding inefficiency or overhead. It can be shown [4] that for k → ∞ these codes have ε → 0, so they are asymptotically optimal. Practical DF codes can be obtained by loosening some of the previous requirements in various ways. For example, the number of packets required to recover the original data could be larger than k, and the number of available encoding packets could be limited. The encoding and decoding times are also very important parameters to be taken into account in a practical implementation. This is especially importante because DF codes have lower complexity that other erasure-correcting codes such as Reed-Solomon codes. Applications of DFs can be found in many areas of digital communications such as data delivery across best effort networks, or reliable data storage on multiple disks. Many multimedia applications, where the use of feedback channels between the sender and the many users is not possible, or it does not represent an efficient solution, can clearly benefit from the DF approach. In this paper the design of two practical DF codes, i.e. LT and Raptor codes, is presented in details. In particular, novel criteria for the selection of the critical parameters of the LT degree distribution are worked out. Moreover, the implementation of various code families allows us to compare the actual performance with the theoretical results available in the literature.

2 Implementation of LT codes As already mentioned, LT codes are rateless and allow on-the-fly random symbol generation. Each encoded packet is obtained from a bitwise exclusive-or of a uniformly random selection of d source symbols, where d is specified by a suitable degree distribution. The features of LT codes make them near-optimal for every erasure channel, but this is only true with a good design of the degree distribution. The used

has been selected as the best solution. In fact, this latter is based on a sum-product algorithm. Moreover, for the erasure channels all messages are either completely certain or completely uncertain. Therefore, all the LT decoders found in literature use the MP algorithm ( [4, 3, 1, 2]).

2

10

1

10

0

c values

10

−1

10

3 Implementation of Raptor codes

−2

10

−3

10

−4

10

δ = 5e−1 δ = 1e−3 δ = 1e−6

−5

10

0

10

1

10

2

10

3

10

source symbols k

4

10

5

10

6

10

Figure 1. Bounds for parameter c as a function of k and δ.

distribution, called Robust Soliton Distribution (RSD) μ(·), is characterized by the two parameters δ and c and is defined in [4]. The first parameter δ is the decoding failure probability after N = (1 + ε)k packets have been received; it also measures the sparsity of an equivalent generator matrix once N has been fixed. Thus, by decreasing δ, the average degree of the encoded symbols increases. The second parameter c is a suitable positive constant, which has been found to affect the LT code performance rather heavily. Analyzing the definition of the RSD, we can determine the values which make the Definition 11 in [4] valid. In particular, by enforcing the consistency of the index ranges in the definition, the following bounds are derived: √ √ 1 k k 1 · ≤ c ≤ · (1) k − 1 ln (k/δ) 2 ln (k/δ) In Fig. 1 the bounds (1) are plotted for some values of δ as a function of k. Observing how this parameter changes the distribution, we have noted that, by decreasing c, both the average degree distribution and the one-degree probability decrease; at the same time, the first peak increases while the second one gets smaller and farther away from the first. Once we have found the bounds for c and how this parameter modifies the degree distribution, we are interested in analyzing the impact on the code performance. In Sect. 4 it is shown that the best performance is achieved for values of c close to the lower bound. Finally, as far as the design of the LT decoder is considered, two different methods are available, namely Gaussian elimination and message passing (MP). Being the Gaussian elimination complexity almost quadratic with k, MP

Raptor codes pre-encode the source symbols using a fixed length block code, and then encode these new symbols with an LT code. The main advantage is that, for correct decoding, it is no longer necessary that the LT decoding succeeds for all the symbols. Thus, it is possible to use a simpler degree distribution that does not recover all the symbols but makes the decoding process faster. The main drawback of Raptor codes is that the total overhead is lower bounded by the overhead of the pre-code. On the contrary, LT codes have no overhead asymptotically. The decoding algorithm is composed of two steps. The inner LT decoder returns a hard bit-reliability vector. This latter is processed by the outer LDPC decoder, based on belief propagation algorithm. Shokrollahi proposed in [1] a new degree distribution that depends only on the overhead and has a maximum degree far lower than k. The average degree, being O (ln (1/ε)), is no longer dependent on the logarithm of k, as in the LT case. In this paper we will call this distribution “Shokrollahi distribution” and the new LT code “weakened LT code” (wLT). LDPC codes with block length 10000 are used as precode. In particular, two types of LDPC with different distributions λ (x), ρ(x) [6], namely Left Irregular (LI) codes and Left Regular (LR) codes, are employed. We recall that the total overhead of Raptor codes, εRaptor , depends on the overheads of the two encoding blocks (pre-coding and wLT encoder): (1 + εRaptor ) = (1 + εLDP C ) · (1 + εwLT )

(2)

From the previous equation it is clear that, given a target total overhead, there are two degrees of freedom for the design. As a consequence, it is important to select the best couple (εLDP C , εwLT ) yielding the best decoding efficiency. Shokrollahi [1] has suggested to set εwLT = εRaptor /2. So, we have a unique expression relating the total overhead εRaptor and the precode’s overhead: εRaptor =

2 · εLDP C 1 − εLDP C

(3)

In Sect. 4 the performance obtained with LI and LR LDPC respectively is shown. LI codes are the LDPC family that is closest to the channel capacity bound. The LI LDPC have been designed according to the generator polynomials λ (x) and ρ (x) reported in Tab. 1 and optimized by [7].

Table 1. Generator polynomials of LI LDPC. 25

5.26% 11.11% 17.65%

λ(x) 0.1614x2 + 0.1254x3 + 0.1253x6 + 0.0466x7 + 0.0932x13 + 0.099x25 + +0.064x26 + 0.2819x100 0.18x2 + 0.12x3 + 0.037x5 + +0.14x6 + 0.05x13 + 0.114x14 + +0.054x33 + 0.11x34 + 0.14x95 + 0.06x96 0.173x2 + 0.11x3 + 0.0694x5 + +0.0834x6 + 0.112x11 + 0.0464x23 +0.12x24 + 0.286x100

ρ(x) x120 x56

Total Overhead

εLDP C

20

15 c = 0.02 Overhead= 5.04 10

x39

5 −2

10

LR LDPC have been proposed as pre-code in [1]. They are characterized by left regular distribution and right Poisson, where check nodes are chosen randomly with a uniform distribution. This corresponds to use  λ (x) = 25 x + 35 x2 (4) ρ (x) = xD where D is a free parameter that has to be set according to the design requirements. In particular, D determines the LDPC overhead 1 ρ (x) dx 5 εLDP C =  1 0 (5) = 2D −3 [λ (x) − ρ (x)] dx 0

4 Experimental results The designed class of DF codes have been simulated for various values of the source length k, in the range between 104 and 106 . This choice permits to validate the code performance in the limit of large k. In all the simulations the LT source symbol is represented by a single bit. Nevertheless, all the results can be extended to longer symbols. As a first approximation, the complexity figures scale with the symbol length since there is a linear dependency between the number of bit ex-or operations and the symbol length. However, this must be reconsidered depending on the actual implementation and adopted hardware.

4.1

LT Optimization

The goal of this first set of experiments is the selection of the values of the parameters c and δ, which minimize the LT total overhead. In Fig. 2 the total overhead is reported as a function of c, for fixed values of δ = 0.5 and k = 104 . The best performance is obtained for c = 0.02. From (1) it turns out that the parameter c may depend also on the number of source symbols k. As a consequence, the experiment reported in Fig. 2 has been repeated for several values of k.

c values

−1

10

Figure 2. Total overhead as a function of c. We have noticed that c ≈ 0.02 yields the minimum overhead in the considered range of k. The effect of δ on the total overhead has been investigated, as well. Having set c = 0.02, the total overhead has been measured as a function of δ. It has turned out that, as expected from LT theory, the overhead does not depend on δ, which represents the decoding failure probability after having received N = (1 + ε) k symbols. As a consequence, the parameter δ can be selected as a trade-off between the sparsity of the equivalent generator matrix and the required failure probability. In particular, larger values of δ yields sparser generation matrix but increase the probability of failure.

4.2

Encoding and Decoding Complexity

In this section the encoding and decoding complexity of the designed code families are analyzed. The results are reported in terms of coding/decoding time, obtained on a Pentium 4 1.7 Ghz processor, 512 MB RAM running Linux OS. All the reported results are evaluated by averaging 1000 independent transmission trials for every source length. In Fig. 3 the encoding times of LT, Raptor LI/LR with the same amount of overhead ε = 0.12 are compared as a function of the source length k. It is worth observing that LT encoding is far slower than Raptor encoding. This happens because LT codes exhibit an average degree that is larger than that of wLT. In the Raptor case, the added complexity due to the pre-code is outweighed by the faster inner wLT stage. Fig. 4 shows the decoding time versus k, when ε = 0.12. It is important to note that the LT decoding complexity is nonlinear with k, and much higher than in the Raptor case. On the contrary, the measured Raptor decoding times are

4 3.5

120 LI Raptor LR Raptor LT LR Rapt. 4.2%

100

LI Raptor LR Raptor LT LR Rapt. 4.2%

80

2.5

time (sec)

time (sec)

3

2 1.5

60

40

1 20

0.5 0 0

0.5

1 source bits

1.5

2

0 0

6

x 10

Figure 3. Raptor and LT encoding time [s] as a function of k; LT, LI/LR Raptor ε = 0.12, LR Raptor ε = 0.042.

almost linear with k, as demonstrated in [1]. Because of the different pre-code complexity, LR Raptor decoding time favorably compares with LI. Finally, in the previous figures the encoding/decoding complexity for LR Raptor with a lower overhead ε = 0.042 is also reported. This amounts to selecting D = 124 for the LR LDPC pre-code. The encoding time of LR Raptor with reduced overhead is not significantly lower than the one measured in the case of ε = 0.12. On the other hand, when ε = 0.042 the decoding process turns out to be slower because there are less checks (or messages) to solve the system.

5 Conclusions In this paper we have presented a complete design for three families of DF codes, namely LT, LR Raptor and LI Raptor. The parameters of the RSD distribution have been optimized by simulations. Moreover, the proposed codes have been analyzed in terms of total overhead, encoding and decoding complexity. This has allowed a better insight into the properties of such codes. In particular, the advantage of Raptor codes over LT codes in terms of complexity has been verified. Future works include the application of the DF paradigm to multimedia applications. Implementation of DF codes for delivery of real-time multimedia contents can be extremely challenging [5]. In particular, real-time audio/video constraints may not be compatible with the latency introduced by DF, such as Raptor codes. In fact, all the proposed

0.5

1 source bits

1.5

2 6

x 10

Figure 4. Raptor and LT decoding time [s] as a function of k; LT, LI/LR Raptor ε = 0.12, LR Raptor ε = 0.042.

codes yields the best performance for long block lengths.

References [1] A. Shokrollahi. Raptor Codes. Transaction on Information Theory, 52(6):2551–2567, 2006. [2] D. J. C. MacKay. Information Theory, Inference, and Learning Algorithms. Cambridge University Press, second edition, December 2003. [3] M. Luby, M. Mitzenmacher, M. A. Shokrollahi and D. A. Spielman. Efficient Error Correcting Codes. IEEE Transaction on Information Theory, 47(2):569–584, February 2001. [4] M. Luby. LT Codes. Proceedings of the ACM Symposium on Foundations of Computer Science (FOCS), 2002. [5] M. Mitzenmacher. Digital Fountains: A Survey and Look Forward. In Information Theory Workshop, pages 271–276. IEEE, October 2004. [6] T. J. Richardson, A. Shokrollahi, R. L. Urbanke. Design of Capacity-Approaching Irregular Low-Density Parity-Check Codes. IEEE Transaction on Information Theory, 47(2):619– 637, February 2001. [7] URL: http://lthcwww.epfl.ch/research/ldpcopt/index.php, Optimal LDPC Polynomials.