This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the ICC 2008 proceedings.
Network Coding with Multi-Generation Mixing: Analysis and Applications for Video Communication Mohammed Halloush, Member, IEEE, and Hayder Radha, Senior Member, IEEE
Abstract—Network Coding (NC) is an emerging networking approach that improves overall throughput over packet networks. Meanwhile, traditional NC approaches have limited advantages under certain network conditions, such as sparse connectivity and high losses; this is especially true for real-time applications. In this paper, we propose and analyze a generalized approach of network coding, which is based on Multi-Generation Mixing (MGM). As we demonstrate in this paper, MGM-based NC improves the performance of real-time data communications under scenarios of sparse connectivity and high loss rates. Under such scenarios, practical network coding not only fail to achieve any improvements; on the contrary it may lead to performance degradations. The analytical as well as the simulation studies we present in this paper show major improvements that can be achieved in situations where practical network coding is not a viable option. In particular, we demonstrate major gains in PSNR video quality under MGM-based network coding.
I. INTRODUCTION Practical network coding requires grouping sender packets into chunks of packets called generations, where encoding is done among packets belonging to one generation [4]. This is necessary for controlling the overhead and computational complexity. It has been shown that applying network coding increases the robustness and reliability of communicating sender generations [6, 8]. The main benefits of network coding are improved resilience to losses and increased throughput. Network coding may not perform efficiently in some scenarios of sparsely connected networks with high loss rates. Inefficiency is due to the possibility of having intermediate nodes not being able to generate independent (useful) encodings; these are encodings that carry new information to neighbors. Inefficient encoding at intermediate nodes causes a situation where receiver(s) are unable to receive what is sufficient to decode. Communicating video contents may benefit from the deployment of network coding. Meanwhile, the sensitivity of video to losses and related real-time constraints have to be
M. Halloush is with the Department of Electrical and Computer Engineering, Michigan State University, East Lansing, MI 48824, (E-mail:
[email protected] ). H. Radha is with the Department of Electrical and Computer Engineering, Michigan State University, East Lansing, MI 48824, (E-mail:
[email protected] ). This work was supported in part by NSF Awards CNS-0721550, CCF0515253, and CNS-0430436.
taken into consideration when applying network coding. Coded video frames have dependency; loss of one frame may spread to multiple frames. With network coding applied to non video contents, the inability to recover a generation causes the loss of information included in that generation only; on the other hand for video contents the loss of one generation may cause a situation where multiple generations are lost having each generation spans one or more video frames. In this paper we will apply the generalized approach of network coding with Multi-Generation Mixing (MGM) [3] in networks communicating video contents. First, we evaluate the performance of MGM network coding through an analytical study as well as simulations. We show that MGM improves the performance of network coding in situations where traditional Generation by Generation (G-by-G) [4, 5] may not be a viable option. The rest of the paper is organized as follows. In Section II we briefly outline key issues related to the efficiency of NC for video communication. In Section III we describe in detail the MGM approach and how encoding/decoding is done at transmitter(s), intermediate nodes, and receiver(s). In Section IV we show the advantages of MGM network coding through analytical modeling using a simple canonical structure. In Section V we illustrate the performance of MGM-based NC when applied to H.264 video and compare it with traditional NC. Finally we conclude the paper in Section VI. II. NC FOR VIDEO It is well known that there is a level of dependency among Group-of-Pictures (GOP) video frames; loss of one frame may cause other frames to be useless and undecodable. Figure 1 shows a traditional sequence of encoded video frames of one GOP and the dependency among them; the loss of one frame spreads to other video frames.
Figure 1: Spread of frame loss, loss of a P frame causes the loss of multiple dependent frames.
In general, video frames are packetized, and the resulting packets are grouped in generations. With traditional NC, i.e. Generation-by-Generation (G-by-G) network coding, by grouping packets in generations, we create dependency among packets that belong to the same generation. This means if
978-1-4244-2075-9/08/$25.00 ©2008 IEEE
This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the ICC 2008 proceedings.
insufficient number of packets was received to recover a generation, the whole generation can not be recovered and all of the received generation packets are useless. In other words either the whole generation packets are recovered or none. The sensitivity of applying G-by-G network coding to video comes from the fact that the loss of one generation can cause the loss of other generations when these generations span multiple video frames. Although one can align NC generations with video GOPs, this simple approach could present severe constraints either on video coding rates/GOP structure or on NC generation sizes. An ideal solution, which is based on the natural separation between the networking and application layers, would be to network code the video packets without being constraints by the GOP structure or other video coding parameters. Under traditional NC, such ideal separation, however, implies that the loss of one generation can be very expensive in terms of video quality especially if the NC generation size is large. Furthermore, the environment where NC is applied has a direct effect on its performance. High losses and sparse connectivity are the main degrading factors on the performance of network coding. Finally, any proposed solution should not require any additional network resources such as additional buffers at intermediate nodes. We will show next how MGM network coding can address the above issues while maintaining the same level of buffer sizes within the network. In particular, we will analyze MGM-based NC and demonstrate how it can improve the performance of NC to a degree where we can consider it as a viable option for enhancing video communication. III. MULTI-GENERATION MIXING III.1 ENCODING MGM network coding employs the randomized Linear Network Coding (LNC) [2] approach for generating independently coded packets. With G-by-G network coding where the generation size is k packets, a vector of coefficients of size k is generated randomly from a finite field GF (2 s ) . The encoding vector associated with encoded packet E x is c x = {c x,0 , ..., c x,k −1} . Encoding vectors are
attached with the transmitted packets. Sender generates independent linear combinations by employing such randomly generated encoding vectors. Sender s generates the encoded packet E x from input packets (p0,…, pk-1) of a generation as follows: k −1
E x = ∑ c x, j . p j j =0
Therefore, one node can generate k independent packets of a generation of size k as follows: ⎡ E 0 ⎤ ⎡ c0,0 K c 0,k −1 ⎤ ⎡ p 0 ⎤ ⎥⎢ ⎢ ⎥ ⎢ ⎥ O M ⎥ ⎢M ⎢M ⎥=⎢ M ⎥ ⎢⎣ E k −1 ⎥⎦ ⎢c k −1,0 K c k −1,k −1 ⎥ ⎢⎣ p k −1 ⎥⎦ ⎣ ⎦
The encoding operation is shown in Figure 2. As shown in Figure 2 each generation is encoded separately of other generations, which means that each generation will be decoded separately at a receiver node after receiving sufficient number of independent encoded packets of that generation.
Figure 2: G-by-G mixing where generations are encoded separately.
With MGM network coding, generations are grouped into Mixing Sets (MS) where the size of a mixing set is m generations. Each generation is assigned a sequence number i , and each mixing set has an index M . Generation i belongs to mixing set with index M = ⎣i / m⎦ . Each generation in the mixing set has a position index; the position index of generation i in a mixing set of size m is l = (i %m) . The position index of a generation indicates its relative position within the mixing set. MGM is a generalized approach; G-by-G network coding is a special case of MGM where the mixing set size is one ( m = 1 ). Under MGM encoding, when a node (either a sender or an intermediate node) sends a packet that belongs to generation i with position index l in a mixing set with index M , that node encodes all the packets it has that are associated with the generations in the same mixing set and have position indices less than or equal to l . For packets in generation with position index l where 0 ≤ l < m the maximum size of the encoding vector used to generate a packet associated with that generation is (l + 1) ⋅ k since the maximum number of packets that can be randomly mixed (encoded) equals (l + 1) ⋅ k . Sender generates encoded packet E x associated with a generation of position index l and encoding vector C x = {c x ,0 ,K, c x ,((l +1)⋅k )−1} as follows: Ex =
((l +1).k )−1
∑
j =0
c x, j . p j
Therefore, one node can generate (l + 1) ⋅ k independent packets, when encoding packets associated with a generation of position index l as follows: ⎡ E0 ⎤ ⎡ c0,0 ⎤ c0,((l +1)⋅k )−1 ⎤ ⎡ p0 K ⎢ ⎥⎢ ⎥ ⎥ ⎢ M O M ⎢M ⎥=⎢ ⎥ ⎢M ⎥ ⎢ E((l +1)⋅k )−1 ⎥ ⎢c((l +1)⋅k )−1,0 K c((l +1)⋅k )−1,((l +1)⋅k )−1 ⎥ ⎢ p((l +1)⋅k )−1 ⎥ ⎣ ⎦ ⎣ ⎦⎣ ⎦
This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the ICC 2008 proceedings.
Although (l + 1) ⋅ k useful packets of a generation with position index l can be generated, it is sufficient for a receiver node to receive k useful packets of that generation. This is true if all generations in the mixing set that have lower position indices than l were recovered. In general, a sender node can generate k + Rl packets associated with the generation of position index l where Rl packets are extra independent encoded packets that can protect against losses that may occur to any generation in the mixing set that have a position index j where 0 ≤ j ≤ l . Figure 3 shows how MGM encoding is performed within a mixing set. Packets associated with generation of position index l , is encoded with packets associated with all generations that have lower position indices in the same mixing set.
can be positive or negative. A positive λl means that λl more packets are needed so that the total number of linearly independent packets received of generation l is k . On the other hand a negative λl means that λl packets have been received over k independent packets associated with generation l , and these packets can protect all generations with position indices less than or equal to l . Under MGM a receiver node will be able to decode a subset of (l + 1) generations of the mixing set m generations, where this subset starts from the first generation of the mixing set and ends at the generation with position index l such that 0 ≤ l ≤ (m − 1) , if the receiver have received independent packets with vectors that form a decoding matrix of the form. [0] (k −λ0 ),(l −0)k ⎤ ⎡ [G1] ( k −λ0 ), k ⎢ [G 2] [0] (k −λ1 ),(l −1)k ⎥⎥ ( k − λ1 ), 2 k ⎢ ⎥ ⎢ : : ⎥ ⎢ ⎢⎣[Gl ] ( k −λl ),(l +1) k [0] ( k −λl ),(l − L ) k ⎥⎦ (l +1) k ×(l +1) k
In the decoding matrix shown above, [Gi ]( k −λi ),(i +1) k is a submatrix of vectors that are received with the useful packets of the generation with position index i and have rank k − λi . [0] ( k −λi ),(l −i ) k is a sub-matrix of zeros that have the same number of rows as [Gi ]( k −λi ),(i +1) k and completes its number of Figure 3: MGM, each generation is encoded with previous generations in the MS.
If a receiver node needs more packets in order to decode the generation with position index l , then under MGM, neighbor nodes of that receiver can forward packets of different generations (with higher position indices). In other words, a receiver node can use whatever useful (independent) packets it receives (to decode unrecovered generations with positions ( l , l − 1 , …, 0 ) in the same mixing set. With G-by-G network coding it will not be possible to decode an old unrecovered generation using other (future) generation packets; and whatever has been received of that old generation is useless (it was not sufficient to decode that generation). On the other hand with MGM network coding this unrecovered generation can still be recovered from the information carried in packets associated with generations of higher position indices in the same mixing set. III.2 DECODING With G-by-G network coding, a receiver will be able to decode the received generation of size k after receiving k useful (linearly independent) packets of that generation. The decoding matrix which is the matrix formed by the encoding vectors included in the received useful packets must have a rank of k . Let’s say that a receiver received k − λl linearly independent encoded packets of a generation with position index l , λl is an integer that represents a particular number of packets. λl
columns to (l + 1) ⋅ k . With MGM network coding, packets associated with generation of position index l where 0 ≤ l p m can be successfully decoded in one of two possible scenarios: -Incremental Decoding: All generations with position indices less than l in the mixing set have already been decoded and the decoder receives at least k independent MGM packets associated with generation of position index l . -Collective Decoding: The number of independent packets received (collectively) of all (l + 1) generations (i.e., generations with position indices zero to l ) is at least (l + 1) ⋅ k packets. Collective decoding of MGM encoded packets of the (l + 1) generations is based on the following necessary condition. Out of the (l + 1) ⋅ k independent packets received, and for any consecutive subset of l ' generations (that starts at the first generation to the generation with position index l ' where l ' ≤ l ) regardless of the number of independent encodings available of these (l '+1) generations, no more than (l '+1) ⋅ k independent encodings can be used in decoding packets associated with generation l . This condition ensures that the encoding vectors included in the (l + 1) ⋅ k independent packets can form a decoding matrix of rank (l + 1) ⋅ k assuming that all encoding vectors are padded so they have the same length which is (l + 1) ⋅ k .
This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the ICC 2008 proceedings.
IV. MODEL In this section we will study the performance of MGM network coding through an analytical model. Consider the network shown in Figure 4, which consists of sender A, a set of intermediate nodes i = {i1 , i2 ,..., iN } and receiver B. Node A can communicate with all nodes in the set i directly. At the same time each node in i can communicate with node B directly.
this can be seen in Figure 5. As the mixing set size m increases, we note that the probability of successfully delivering all sender generations increases.
Figure 4: Nodes within a circle can communicate directly.
Let’s investigate the probability of successfully delivering packets from node A to node B with MGM network coding. Assume A has h generations to be delivered to B where the generation size is k packets. Let p be the probability of packet loss while being transmitted. Let Pi ,mk be the probability that set i receives at least mk useful packets sufficient to recover a mixing set of size m. Having sender A sent k independent packets associated with each generation of the mixing set and R extra packets associated with the last generation of the mixing set to protect against losses that may occur in the mixing set, we have: mk −1 mk + R ⎛ ⎞ ⎟⎟ ⋅ (1 − p) i ⋅ p ( mk + R −i ) Pi ,mk = 1 − ∑ ⎜⎜ i ⎝ ⎠ i =0
Figure 5: Successful delivery as packet loss changes. k=50, 300 packet, 10% extra packets per generation
Another improvement that is achieved with MGM network coding is the amount of redundancy needed to protect sender data to achieve successful delivery is lesser as the mixing set size increases. Figure 6 shows that by increasing the size of a mixing set the average nmber of extra packets per generation associated with the last generation of the mixing set to achieve 95% reliability decreases.
Let PB , mk be the probability that node B receives at least a number of useful packets that are sufficient to recover the m mixing set generations: k +R ⎛ n ⎞ ⎟⎟ ⋅ (1 − p) k +λ ' ⋅ p ( n−k −λ ') PB ,mk = Pi ,mk ⋅ ∑ ⎜⎜ n=k +λ ' ⎝ k + λ ' ⎠ The probability of successfully delivering the h generations from A to B is: PS = ( p B,mk ) h / m
λ ' is the minimum number of independent packets that are still needed to recover the m mixing set generations. λl ' is the number of independent packets that are associated with the generation of position index l and are still needed to recover that generation. Hence: m− 2
λ ' = ∑ λl ' = (m − 1) ⋅ λl ' l =0
λl ' is computed from: λl ⎛ k ⎞ pλ ' = ∑ ⎜⎜ ⎟⎟ ⋅ p λl ' ⋅ (1 − p ) ( k −λl ') l j = 0 ⎝ λl ' ⎠ Where pλ ' is very high. l
It is worth mentioning that the case of MGM with m = 1 represents the traditional G-by-G network coding. As the size of the mixing set ( m ) increases, there will be an improvement in the reliability of delivering sender generations to receiver;
Figure 6: Percent of extra packets to achieve 0.95 reliability, k=50.
Figure 7 shows that for a packet loss rate of 0.05 as the number of extra packets per generation associated with the last generation of the mixing set increases, better improvement in the reliability of successful delivery is achieved, and its better with increased MGM mixing set size. V. SIMULATION Our simulation is done in a wireless setting [7]. Nodes are distributed in a grid topology. The grid is 20 by 20 units area. Within each one-by-one unit area there is only one node that is randomly located. Each node can communicate packets to all nodes within a radius r. The sender is located on the corner of the grid. Our results show that for r=1.5, traditional G-byG network coding ( m = 1 ) is not efficient to achieve sufficient distribution of sender data to other nodes. While with MGM network coding, larger number of nodes receives more useful packets. Our simulation is done on different video traces [9, 10, 11]. For space limitations we will show the results for 270
This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the ICC 2008 proceedings.
of nodes losing at most the corresponding number of generations g i not counting nodes lost at most g i −1 . It is clear that with an increased mixing set size more nodes lose less generations, which ensures that MGM is an improving approach for network coding.
Figure 7: Improvement in successful delivery as the percent of extra independent packets per generation increases. k=50, 300 packets.
frames Forman video sample, 30 fps. Frames are packetized in 1 KB packets where k=10. As shown in Figure 8 the average number of generations recovered by all nodes is higher as the size of the mixing set increases.
Figure 10: Portion of nodes loosing at most the corresponding number of generations.
VI. CONCLUSION MGM network coding provides the flexibility for decoding generations incrementally or collectively. This approach improves the reliability of recovering sender data. MGM improves the ability of intermediate nodes to propagate useful information in a way that makes network coding a viable option for enhancing the quality of video communication even in extreme situations of losses and sparseness. REFERENCES [1]
Figure 8: Number of recovered generations for all nodes over time.
Figure 9 shows a major enhancement in the average video quality received by all nodes of the topology, with the increased mixing set size. More than 10 dB of PSNR improvements can be achieved under MGM, which is quite significant quality gain.
[2] [3]
[4]
[6]
[7] [8]
[9]
Figure 10 shows the distribution of generations lost among receiving nodes in the topology. This figure shows the percent
Mohammed Halloush and Hayder Radha, "Network Coding with Multi-generation Mixing," Conference of Information Sciences
[5]
Figure 9: Average PSNR for all nodes as a function of the number of decodable packets received.
R. Ahlswede, N. Cai, S. Li and R. Yeung, "Network information flow", IEEE Transactions in Information Theory, July 2000. S.-Y. R. Li, R. W. Yeung, and N. Cai. “Linear network coding”. IEEE Transactions on Information Theory , Februray, 2003.
[10] [11]
and Systems (CISS08), Princeton University, NJ, USA, 2008 P Chou. Y Wu K Jain, "Practical network coding", Allerton Conference on Communication, Control, and Computing, Monticello, IL, October 20, 2003. C. Gkantsidis and P. Rodriguez. Network coding for large scale content distribution. Infocom, Miami, FL, Mar. 2005. C.Fragouli, J Le Boudec, J. Widmer, "Network coding: an instant primer", Computer Communication Review, ACM SIGCOMM, Jan 2006. A. Al-Hamra, C. Barakat and T. Turletti, "Network coding for wireless mesh networks: A case study", wowmom, 2006. P. Seeling, M. Reisslein and B. Kulapala. “Network performance evaluation using frame size and Quality Traces of single layer and two layer video: A tutorial, “IEEE Communications Surveys and Tutorials. Vol. 6, no. 3, pp. 58-78, Third Quarter 2004 P. Seeling and M. Reisslein, "Offset distortion traces for tracebased evaluation of video quality after network transport (extended version)," Apr. 2005, availabe from http://trace.eas.asu.edu Video traces for network performance evaluarion. Available at: http://trace.eas.asu.edu/ Jvt reference software version 12.4, Available at: http://iphome.hhi.de/suehring/tml/
