Probability of complete decoding of random codes for short messages Zan Kai Chong, Bok-Min Goi, Hiroyuki Ohsaki, Bryan Ng✉ and Hong Tat Ewe A random code is a rateless erasure code with a generator matrix of randomly distributed binary values. It encodes a message of k symbols into a potentially infinite number of coded symbols. For asymptotically large k, the tail bound in Kolchin’s theorem asserts that the high probability of complete decoding (PCD) is attained almost surely with k + 10 coded symbols. However, for small values of k (short messages) it is unclear if such asymptotics are useful. That the random codes achieve a high PCD with k + 10 coded symbols for small k is demonstrated. In particular, a set of lemmas is established and show that the PCD converges to five decimal digits after k = 30. A theorem extending Kolchin’s work is formulated and the theorem is used to explain the complete decoding probabilities of random codes in short messages.
Introduction: Rateless erasure codes for short message transmission: The current Internet architecture relies on individual flow control to manage network congestion. However, today’s Internet traffic is dominated by short-lived flows with small packet sizes. Short-lived flows recovering from the loss of the last packet incur disproportionately high latency with existing flow control mechanisms. Owing to this inefficiency, the Global Environment for Network Innovations envisions future networks without congestion control [1], and erasure codes represent a significant step towards realisation of such networks. Network flows regulated by congestion control are typically modelled as a binary erasure channel whereby a transmitted packet is either received intact by the receiver, or dropped because of the network congestion or packet error. Transport protocols utilising state-of-the-art rateless erasure codes for networking have been proposed in the literature. Namely the variants of Luby transform (LT) codes [2] and Raptor codes [3] such as [4–6] are particularly attractive because of the low decoding complexity. Given a message of k symbols, a ‘rateless’ erasure code generates a potentially infinite number of coded symbols and the receiver reconstructs the original message from any k(1 + ε) coded symbols, where ε denotes the decoding inefficiency. The sender does not need a priori knowledge of the channel condition and it continues sending the coded symbols until the receiver has enough coded symbols to decode. However, the aforementioned rateless erasure codes are only efficient for long messages as reported in [7, 8] whereas the network traffic is dominated by short messages instead [9, 10]. Erasure codes derived from LT codes such as [7, 11, 12] are pushing code design to address the needs of short message transmission. In particular, random code variations such as the windowed code [8] and the stepping-random code [13] have been proposed for this purpose. Decodability of these codes is well explained with Kolchin’s theorem [14], i.e. the probability a random matrix of size (k + 10) × k is nonsingular is 99.9% when k → ∞. However, the theorem does not explain the non-singularity of the random matrix (i.e. complete decoding of the random code) for short messages. In this Letter, we show that random codes achieve a high probability of complete decoding (PCD) with k + 10 coded symbols even for small k. We are aware of similar work in [15] and Shokrollahi [3] whereby the upper bound of the PCD is given as a function of excess (redundant) packets. However, these papers do not explicitly deal with random codes, arguing that the encoding and decoding costs are prohibitive. Our work herein departs from these related works in that we use the exact PCD to dimension Random codes for short messages. Moreover, our work extends Kolchin’s theorem to short messages in support of the random code as a good rateless erasure code. The rest of the Letter is organised as follows. We introduce the random matrix – the generator of random codes in the random matrix. Starting with Kolchin’s theorem, we proceed to show that the random codes achieve high PCD with k + 10 coded symbols for small k (e.g. k = 10). Random matrix: Random codes have a generator matrix of randomly distributed 0 and 1 denoted by G (n×k). A message is also expressed as a matrix S (k×l ) with its dimensions k × l denoted in superscript. A coded symbol x (1×l ) is generated by multiplying a generator row matrix g (1×k) with the message S (k×l ). Therefore n coded symbols are
generated by simple multiplication X (n×l ) = G (n×k)S (k×l ). The original message can be reconstructed from these n coded symbols if there exists k out of n coded symbols, for which their corresponding generator matrix G (k×k) is non-singular. Therefore a full rank matrix implies the complete decoding of the message. Unlike other rateless erasure codes, random codes do not possess any structural properties, apart from the randomness in the generator matrix. Therefore, Gaussian elimination is the only method to reconstruct the original message. Note that we use the terms ‘random code’ and ‘random matrix’ interchangeably in this Letter. All the matrices here have been constructed in binary field GF(2). Kolchin’s theorem: Kolchin’s theorem is used to explain the fixed decoding redundancy of random codes, i.e. a high PCD is achievable with k + 10 coded symbols. Given a random matrix of dimensions (k + m) × k as G ((k + m)×k) whereby m and r are integers such that m + r ≥ 0, r ≥ 0 and m > 0, then, according to Kolchin [14] (Theorem 3.2.1), if k → ∞, then Pr rank G((k+m)×k) = k − r Qm
(1)
where Qm = 2−r(m+r)
1
1−
i=r+1
1 m+r 1 −1 1− i i 2 i=1 2
(2)
Since we are only interested in a full rank matrix, r = 0, (2) can be further simplified as m 1 1 1 1 −1 1 1− i 1− i = 1− i Qm = (1) (3) 2 i=1 2 2 i=1 i=m+1 PCD for short messages: As noted in Kolchin’s theorem, nothing is said about the decodability of random codes for transmitting short messages. In Kolchin’s theorem, we will assume that k is finite and bounded, then study the PCD for a random matrix with small k. Lemma 1: Given that a random matrix G ((n − 1)×k) of dimensions (n − 1) × k has rank (n − 1), where 0 < n ≤ k. Then, the probability of achieving rank n with an extra row is p(n, k) =
n−1
1 − 2i−k
(4)
i=1
Proof: Let g (1×k) be a new row matrix with 2k − 1 possible combinations excluding the row matrix of all zeros. To have rank n in G (n×k), g (1×k) must be independent of the other. Therefore, g (1×k) is limited to τ(n − 1, k) possible combinations, where n−1 n−1 t(n − 1, k) = 2k − 1 − i (5) i=1 = 2k − 1 − (2n−1 − 1) = 2k − 2n−1 . and is the binomial coefficient. Each of the generator row matrix is . generated independently. Correspondingly, the probability of G ((n − 1)×k) attaining rank n with g (1×k) is
t(n − 1, k) 2k t(n − 2, k) t(n − 1, k) = p(n − 2, k) × × 2k 2k t(1, k) t(2, k) t(n − 1, k) = × × ··· × 2k 2k 2k n−1 n−1 t(i, k) = = (1 − 2i−k ) 2k i=1 i=1
p(n, k) = p(n − 1, k) ×
(6)
□
We study Lemma 1 with different matrix dimensions. Surprisingly, we find that the probability of having higher rank numbers (i.e. rank = k − 10, k − 9, …, k) in matrices of different column dimensions
ELECTRONICS LETTERS 5th February 2015 Vol. 51 No. 3 pp. 251–253
have negligibly small difference. For example, the probability of attaining rank 20 (i.e. 30 − 10) in G (20 × 30) is identical to the probability of having rank 30 (i.e. 40 − 10) in G (30 × 40). For different values of k, we tabulate the PCD in Table 1 [using (4) of Lemma 1] and find that the probabilities converge to five decimal digits after k = 30.
Theorem 1: The probability of having rank k in matrix of dimensions (k + m) × k, for m ≥ 0 is Pr rank G((k+m)×k) = k = p(k − m, k) (11)
Table 1: Probabilities of having last 10 rank numbers for k − m = 20 and k − m = 30 and k is fixed as 30
In particular, the probability of achieving full rank (complete decoding) with an extra 10 rows is 0.99902. This is consistent with Kolchin’s theorem (i.e. Q10 = 0.999 in (3)) and extends its applicability for calculating the PCD for short messages as shown in Table 1.
m
p(k − m, k) m p(k − m, k)
10 9 8 7 6
0.99902 0.99805 0.99610 0.99221 0.98446
5
0.96907
4 3 2 1 0
0.93879 0.88012 0.77010 0.57758 0.28879
Conclusion: Random codes achieve a high PCD with 10 extra coded symbols even for short messages. The analysis in this Letter posits random codes as feasible rateless erasure codes for short messages in networking. In practice, short messages appear extensively in the context of network probing, cryptographic key exchange and wireless sensor networks. This Letter shows that the bounded errors in PCD provide adequate reliability for short message transmissions using random codes.
A precise statement of the identical probabilities observed in matrix rank is given in Lemma 2. +
Lemma 2: Let m [ Z and 0 ≤ m < k. Then, the probabilities to have rank k − m in matrices of dimensions (k − m) × k for different k are similar, i.e. |p(k − m + 1, k + 1) − p(k − m, k)| , d
(7)
and δ → 0 exponentially fast with finite and bounded increments of k. Proof: By induction on k, we show that d , (1/2k ) ((2x − 1)/2x ). First, we note that
k−1
|p(k − m + 1, k + 1) − p(k − m, k)| , |p(k − m, k) − p(k − m − 1, k − 1)|
Bryan Ng (Victoria University, New Zealand)
References (8)
= |(1 − 21−k )(1 − 22−k ), . . . , (1 − 2−m−1 )[1 − (1 − 2−k )]| (9)
(10)
x − 1)/2x) < 1, and 0 ≤ m < k for all d [ R+ and Since ((2 k x x inf [1/2 ] k−1 x=m+1 ((2 − 1)/2 ) = 0, by monotone convergence
we have δ → 0 driven by an exponential term 1/2k.
Hiroyuki Ohsaki (Kwansei Gakuin University, Japan) ✉ E-mail:
[email protected]
|p(k − m, k) − p(k − m − 1, k − 1)| k−m−2 k−m−1 i−k i−(k−1) (1 − 2 ) − (1 − 2 ) = i=1 i=1
Let δ denote |p(k − m + 1, k + 1) − p(k − m, k)|. Then, by (9) k−1 x 1 2 − 1 d, k 2 x=m+1 2x
Zan Kai Chong, Bok-Min Goi and Hong Tat Ewe (Universiti Tunku Abdul Rahman, Malaysia)
x=m+1
We may express |p(k − m, k) − p(k − m − 1, k − 1)| as
= |(1 − 21−k )(1 − 22−k ), . . . , (1 − 2−m−1 )[2−k ]| 1 1 1 1 = 1 − k−1 1 − k−2 , . . . , 1 − m+1 2 2 2 2k k−1 1 2x − 1 = k 2 x=m+1 2x
© The Institution of Engineering and Technology 2015 17 November 2014 doi: 10.1049/el.2014.3977
□
To show that the probability of achieving complete decoding with m extra rows is the same as p(k − m, k), we use the well-known result of matrix rank invariance under transposition. We state without proof the following lemma. Lemma 3: The probability of having rank (k − m) in both G ((k−m)×k) and G (k × (k−m)) is identical. Lemma 3 asserts that the rank of a matrix is invariant to matrix transposition. For example, the probability for G ((k − 1)×k) to have rank k − 1 is 0.57758 as stated in Table 1. Then, the probability of having rank k − 1 in G (k × (k − 1)) is also 0.57758. Finally, building upon Lemmas 1 and 3, we propose the following theorem.
1 Bastin, N., Bavier, A., Blaine, J., et al.: ‘The InstaGENI initiative: an architecture for distributed systems and advanced programmable networks’, Comput. Netw., 2014, 61, pp. 24–38 2 Luby, M.: ‘LT codes’. Proc. of 43rd Symp. on Foundations of Computer Science, Washington DC, WA, USA, November 2002, p. 271 3 Shokrollahi, A.: ‘Raptor codes’, IEEE/ACM Trans. Netw., 2006, 14, pp. 2551–2567 4 Ganguly, B., Holzbauer, B., Kar, K., and Battle, K.: ‘Loss-tolerant TCP (LT-TCP): implementation and experimental evaluation’. IEEE Military Communications Conf., Orlando, FL, USA, October 2012, pp. 1–6 5 Botos, A., Polgar, Z.A., and Bota, V.: ‘Analysis of a transport protocol based on rateless erasure correcting codes’. IEEE Int. Conf. on Intelligent Computer Communication and Processing, Cluj-Napoca, Romania, August 2010, pp. 465–471 6 Yuk, S.-W., Kang, M.-G., Shin, B.-C., and Cho, D.-H.: ‘An adaptive redundancy control method for erasure-code-based real-time data transmission over the Internet’, IEEE Trans. Multimedia, 2001, 3, (3), pp. 366–374 7 Wang, X., Willig, A., and Woodward, G.: ‘Improving fountain codes for short message lengths by adding memory’. IEEE 8th Int. Conf. on Intelligent Sensors, Sensor Networks and Information Processing, Melbourne, Australia, April 2013, pp. 189–194 8 Studholme, C., and Blake, I.: ‘Windowed erasure codes’. IEEE Int. Symp. on Information Theory, Seattle, WA, USA, July 2006, pp. 509–513 9 Brownlee, N., and Claffy, K.C.: ‘Understanding internet traffic streams: dragonflies and tortoises’, IEEE Commun. Mag., 2002, 40, (10), pp. 110–117 10 Guo, L., and Matta, I.: ‘The war between mice and elephants’. 9th Int. Conf. on Network Protocols, Riverside, CA, USA, November 2001, pp. 180–188 11 Lu, H., Lu, F., Foh, C., and Cai, J.: ‘LT-W: improving LT decoding with Wiedemann solver’, IEEE Trans. Inf. Theory, 2013, 59, pp. 7887–7897 12 Hyytia, E., Tirronen, T., and Virtamo, J.: ‘Optimal degree distribution for LT codes with small message length’. IEEE Int. Conf. Computer Communications, Anchorage, AK, May 2007, pp. 2576–2580 13 Chong, Z.-K., Goi, B.-M., Ohsaki, H., Ng, B., and Ewe, H.-T.: ‘Stepping-random code: a rateless erasure code for short-length messages’, IEICE Trans. Commun., 2013, E96-B, pp. 1764–1771 14 Kolchin, V.F.: ‘Random graphs (Encyclopedia of Mathematics and its Applications)’, 1998 15 MacKay, D.J.: ‘Fountain codes’, IEE Proc., Commun., 2005, 152, (6), pp. 1062–1068
ELECTRONICS LETTERS 5th February 2015 Vol. 51 No. 3 pp. 251–253
