1
A Survey on Randomized Algorithms in Networking Design: Random Network Coding Azadeh Khaleghi Department of Electrical and Computer Engineering University of Toronto
[email protected]
Abstract—This survey attempts to provide an overview of the distributed and randomized schemes used in networking. This concept by itself is very diverse and the problems discussed in this paper are by no means exhaustive. We restrict our review to the class of randomized schemes which are based on the concept of random network coding. In particular we describe the existing schemes for randomized coding in multicast. Furthermore, we review the practical challenges and the existing strategies to mitigate them in real networks. We also survey a decentralized random-gossip protocol which utilizes random network coding to simultaneously disseminate multiple messages to all the nodes in the network.
I.
INTRODUCTION
Practical network operations are currently based on traditional routing or store-and-forward schemes which allow encoding to be performed only at the source nodes while the internal nodes within the network are simply to transport unmodified packets supplied by the source nodes. Network coding generalizes this operation beyond traditional routing, allowing interior nodes to perform arbitrary mathematical operations on information received at their incoming links. In random network coding the operations performed at the interior nodes are in some form of uncoordinated random linear mappings from inputs on outputs over some field. In this paper we survey recent developments in random network coding. In particular we review existing random network coding protocols for multicast. Moreover we describe the existing practical challenges in the design of network schemes which utilize random network coding for packet transmission. This paper is organized as follows: in section II we provide a rough background on network coding; section III describes an algebraic framework for network coding based on [1]; in section IV we survey the developments, (mainly based on [9]) in the distributed random network coding approach to multi-source multicast; in section V
we describe the practical scheme for random network coding proposed by Chou et al. in [2] which uses buffering to synchronize arbitrary packet arrivals and departures at a node; section VI presents a random gossip protocol presented by Deb and M´edard [3], which makes use of random linear coding to simultaneously disseminate a set of k messages in a large network of n nodes in a distributed setting. II. N ETWORK C ODING BACKGROUND The usefulness of network coding is best viewed through the following classical example: consider a network as shown in Fig.1 with directed unit capacity links in which a single source node s multicasts information to receiver nodes y and z . Multicasting refers to transmitting common information from one source node to a set of destination nodes. Here the interior nodes, transfer the packets received at their input links to their output links with no modification. Clearly, the link w −x is the bottleneck of the transmission since w has two input links and a single output link. Since the capacity of w − x is 1, it is impossible for w to simultaneously deliver b and b0 to x by routing, i.e. without combining b and b0 . This results in a transmission rate of 1/2 bits per unit time at w and an overall transmission rate of 2/3 bits per unit time for the network. Fig.2 shows the same network but with interior nodes broadcasting the modulo2 addition of the information received at their incoming links. In particular w sends b ⊕ b0 to x on w − x. Therefore y receives both b0 and b ⊕ b0 and is therefore able to recover both b and b0 . Similarly z is able to recover both b and b0 . This scheme therefore allows the network to achieve an overall rate of 2 bits per unit time, which is in fact the capacity of the network. As shown from this example, network coding generalizes routing by allowing information to be combined. This example was first introduced by Ahlswede et al. in [4] to demonstrate that coding at the interior nodes is necessary in order for a network to achieve the maximum
2
Fig. 1.
Example of a network using a routing scheme.
unit capacity and have the same delay. Edges with larger capacities are viewed as multiple unit capacity edges in parallel. Similarly links with longer delay are viewed as multiple links in series. A set X = {X1 , X2 , ..., Xr } of discrete random processes is observable at one or more source nodes. There are d ≥ 1 receiver nodes. The output process at a receiver β is denoted Z(β, i). Each edge l carries a random process Y (l) which is a linear combination of the processes Y (j) carried by its incident incoming links and processes Xi generated at the node where l originates from. For convenience it is usually useful to distinguish between the source and non-source nodes. Thus for a link j originating from v with a set of links Lv = {l : l is incident incoming link of v} we obtain the following formulation: X ai,j Xi if v is a source node; i X Y (j) = (1) fl,j Y (l) if v is an interior node; l:l∈Lv
Fig. 2.
Example of a network using a network coding.
possible multicast transmission rate as the coding symbol size approaches infinity. Later Li et al. extended this result in [5] to show that linear coding with finite symbol size is in fact sufficient for multicast connections. Ho et al. studied random linear network coding and showed that the multicast capacity can be achieved by performing random linear network coding over a sufficiently large finite field. This randomized approach is particularly useful in situations where coordination among nodes within the network is infeasible, since it allows the network to achieve, without coordination among nodes, the same transmission rates as with centralized planning. III. A N A LGEBRAIC M ODEL FOR N ETWORK C ODING In this section we describe an algebraic model for network coding, mainly based on Koetter and M´edard’s as presented in [1]. A communication network is represented in the form of a directed graph G(V, E) with a vertex set V and an edge set E . There exist ν ≥ 1 source nodes and d ≥ 1 receiver nodes in the network. All edges are of
Fig. 3. Illustration of linear network coding for a network with two source nodes, s1 and s2 , and one receiver node β.
By definition, a source node in a network does not have any incident incoming processes. However, we can view a source node as an interior node with artificial incident incoming links, each of which carries one of the input processes Xi . This view allows for a uniformity of notation in the sense that for any node within the network, be it a source node or an interior node the
3
outgoing signal is a linear combination of its incident incoming processes. An output process Z(β, i) at the receiver node β is a linear combination of signals on its terminal links 1 represented as: X Z(β, i) = bβ,i Y (l) (2) {l:l∈Lβ }
Fig.3 illustrates an example of a network coding scheme for a network with two source nodes s1 and s2 and a single output node β . The output of the source nodes Y (e1 ), Y (e4 ) and Y (e2 ) is a linear combination of the input processes X1 , X2 and X3 available at the source nodes. With X2 available at s1 and X1 and X3 available at s2 . The output Y (e3 ) at the interior node v is a linear combination of the incident incoming links Y (e1 ) and Y (e2 ), with linear coefficients according to fei ,e3 , i = 1, 2. Finally the output node β is a linear combination of the processes on its terminal links Y (e3 ) and Y (e4 ), with coefficients according to bβ,i , i = 1, 4. The random processes Xi , Y (l) and Z(β, i) are represented as sequences Xi = {Xi,0 , Xi,1 , · · ·}, Y (l) = {Y0 (l), Y1 (l), · · ·}, and Z(β, i) = {Z0 (β, i), Z1 (β, i), · · ·} of symbols from F2u . Hence all operations, such as the linear combinations in 1 and 2 are performed in the field F2u . Considering an F2u linear network allows for the input/output relationship of the network to be described by a single transfer matrix. Therefore a transfer matrix Mβ can be obtained to describe the relationship between a vector of source values [X1 , X2 , · · · , Xr ] and the corresponding output vector [Zβ,1 , Zβ,2 , · · · , Zβ,r ] at a receiver node β : [X1 , X2 , · · · , Xr ]Mβ = [Zβ,1 , Zβ,2 , · · · , Zβ,r ]
(3)
The input signals undergo three main operations before they appear as an output signal at the receiver nodes. 1) The source processes are transferred to signals on source nodes’ outgoing links. 2) Signals at interior nodes are combined and transmitted between incident links based on the network structure, which in turn depends on the interior connections within the network. 3) The signals on terminal links are transferred to output processes. Each of the above operations can be performed using a corresponding transfer matrix: Coefficients ai,j ∈ F2u can be collected into a transfer matrix A = (ai,j ) from the input processes to signals on 1 Incident incoming links of a receiver node are regarded as terminal links.
the source nodes’ outgoing links. Similarly the coefficients bi,j ∈ F2u at each receiver β can be collected into a transfer matrix Bβ = (bi,j ) from signals on the terminal links to the output processes. Finally, the structure of network can be encapsulated in an |E| × |E| matrix F where, fi,j if edge i feeds into edge j; Fij = (4) 0 otherwise; Where an edge i is said to feed into an edge j if i is an incident incoming edge at the same node where edge j originates from. Following these observations, the transfer matrix Mβ can be viewed as a function of the triplet (A, F, Bβ ). In fact as shown in [1] for each receiver β we have: Mβ (A, F, Bβ ) = A(I − F )−1 BβT
(5)
For proof of 5 we refer the reader to [1]. Thus Mβ (A, F, Bβ ) describes a linear network code for a network G(V, E). A network code is feasible, if all receivers are able to reconstruct the source processes. This means that Mβ (A, F, Bβ ) is a valid transfer matrix if its determinant does not vanish. IV. R ANDOMIZED N ETWORK C ODING Consider a network with a large number of nodes and a dynamically varying topology that requires online operations. For such a system it may be expensive or infeasible to reliably maintain coordination within the network nodes. This type of networking scenario calls for a randomized scheme that obviates the need for centralized knowledge of the graph topology and/or synchronous communication. In [6] a distributed randomized routing algorithm is proposed to mitigate this issue in wireless sensor networks. In [7] Qin et al. present what they regard as a ”probability random routing” scheme addressing the issue of high frequency topological changes that usually occur in wireless networks. As discussed earlier, and as shown in [8], network coding is more beneficial than routing, thus here we focus on a randomized network coding scheme discussed in [9] rather than randomized routing algorithms. Consider a network with input vector X = [X1 , X2 , ..., Xr ] and output vector [Zβ,1 , Zβ,2 , · · · , Zβ,r ]. As described in III, we can obtain an input/output relationship Zβ = XMβ (A, F, Bβ ) where Mβ (A, F, Bβ ) = A(I − F )−1 BβT defines a linear network code. Thus far, we have not discussed any approaches for determining the coefficients ai,j and fi,j that define the network transfer matrix. As shown in [1], any set of coefficients in Fu2 for which the determinant
4
of Mβ does not vanish can potentially be chosen to give a feasible network code. Therefore these coefficients need not necessarily be deterministic. A randomized network coding approach such as that analyzed in [9] allows for some or all nodes other than the receiver nodes to independently and randomly select linear mappings from inputs to outputs in the field F2u . This is done with the constraint that the randomly chosen coefficients together with the fixed coefficient values preserve feasibility.
encode their outgoing messages differently, but they may also communicate their outgoing messages to different nodes within the matrix. Fig.4 shows a snapshot of a network in operation that uses a randomized coding scheme with randomly chosen encoding coefficients ai,j and fi,j . Fig.5 shows another snapshot of the exact same network taking the same input sequences and encoding them with some 0 . v possibly different encoding coefficients a0i,j and fi,j 1 in Fig.4 generates a message Y (e1 ) and transmits it to v2 along e1 . In Fig.5 the same node v1 generates a different message Y˜ (e1 ) 6= Y (e1 )(based on a different set of randomly chosen coefficients) and transmits it to v3 along e1 . Clearly, in this randomized setting, the receivers cannot use the same coefficients ai,j and fi,j to decode the received signals in Fig.5. Therefore, since the linear coefficients have been chosen uniformly at random, the overall linear combinations of the source processes must be communicated to the receivers in every network usage. For example with each block, a vector of finite field coefficients each corresponding to a source process can be sent so that at each node, the same linear mapping is applied to the coefficient vectors as to the information packets.
Fig. 4. Snapshot of a network with a random network coding scheme
Although Ho et al.[9] provide a sketch of a potential distributed random network coding scheme, they do not present a detailed practical algorithm. Instead they analyze the asymptotic bound on the probability of error in randomized scheme for multicast. In particular they show that for a feasible multicast connection and a random network code, the decoding error probability decreases exponentially with the number of codeword bits. For a detailed derivation of this bound we refer the reader to .[9].
V. P RACTICAL R ANDOM N ETWORK C ODING
Fig. 5. Second Snapshot of the same network as in Fig.4. As can be observed, the nodes have remained the same, but the network structure as well as the messages passed along edges have changed from the previous snapshot.
As shown in 4 the coefficients fi,j describe the internal node connectivity of the network. Thus in a randomized network coding scheme the structure of the network may change at each usage. In fact every time a randomized transfer matrix is generated, not only will the nodes
Let β be a receiver node in a network with some linear network coding scheme. Assume that a set of r input processes X = {X1 , · · · , Xr } is available at the source nodes of the network, where each process Xi is represented as a sequence Xi = {Xi,0 , Xi,1 , · · · Xi,n }. β observes the output sequence Z(β, i) = {Z0 (β, i), Z1 (β, i), · · · Zn (β, i)} and the input/output relationship of the network is given according to 3. We can expand this equation, replacing the processes with their corresponding sequence elements to
5
obtain:
Z(β, 1) Z(β, 2) = .. . Z(β, r) X1 X2 MβT . = . . Xr X1,1 X1,2 · · · X1,n X2,1 X2,2 · · · X2,n MβT . .. .. = .. .. . . . Xr,1 Xr,2 · · · Xr,n Z1 (β, 1) Z2 (β, 1) · · · Zn (β, 1) Z1 (β, 2) Z2 (β, 2) · · · Zn (β, 2) .. .. .. .. . . . .
(6)
Z1 (β, r) Z2 (β, r) · · · Zn (β, r)
In a randomized setting Mβ is not fixed, hence given Z(β) the receiver is unable to recover the input sequences. Now assume that instead of the Z(β) as given by (6), β receives the matrix Z 0 (β) created by augmenting Z(β) with Mβ = (mi,j ), i.e. m1,1 · · · m1,r Z1 (β, 1) · · · Zn (β, 1) m2,1 · · · m2,r Z1 (β, 2) · · · Zn (β, 2) 0 Z (β) = . .. .. .. .. .. .. . . . . . mr,1 · · · mr,r
Z1 (β, r) · · · Zn (β, r)
Z 0 (β) allows for the direct recovery of the input sequences, since it makes the network transfer matrix Mβ available to the receiver. It is relatively simple to have the network produce Z 0 (β) instead of Z(β), since 1 · · · 0 X1,1 · · · X1,n .. .. (7) .. .. Z 0 (β) = MβT . . . . 0 · · · 1 Xr,1 · · · Xr,n
This scheme can be easily accomplished by concatenating the ith unit vector of length r to the beginning of each input sequence and have the source nodes transmit this extended vector instead. Therefore at a cost of sending r extra symbols, the transfer matrix becomes available to the receivers. This idea was presented by Chou et al. in [2] who also show that this overhead cost is negligible as compared to the practical network sizes. Assume that a set of r input sequences become available at the source nodes every t seconds. Denote the input sequences corresponding to the gith set, X(gi )
of the gith generation. The encoding scheme presented in (7) will be sufficient for random network coding only if the sequences from the same generation are carried in synchrony. In practical scenarios this may not be the case. For example packets on different edges maybe subject to different propagation and queuing delays. Therefore synchronization of the packets related to the same set of source sequences must be enforced in a practical random encoding scheme. In [2] a buffering model is proposed to synchronize the packet arrivals and departures at each node. In this model, packets that arrive at a node are sorted by their generation number and stored in a buffer. The node will then transmit (whenever possible), a random linear combination of all the packets of the same generation already in the buffer. Periodically, the current generation is advanced and the old generation is flushed from the buffer. If a packet arrives for a generation that has been flushed out of the buffer, it will be discarded. A packet of a certain generation arriving at a node can be viewed as a vector that may or may not be in the span of the packets of the same generation already in the buffer. The latter case is undesirable since in this situation the dimensionality of the subspace from which the outgoing vectors are generated will not be changed. Such packet can be safely discarded from the buffer. Whether or not a packet presents new information to its corresponding coset within the buffer is determined at the destination node. This allows the system to be completely distributed, eliminating the need for any central authority within the network. This buffering algorithm is best described through an example. Fig.6 demonstrates a snapshot of the nodes v , v 0 and v” within a larger network which follows the buffered-based networking scheme described above. Three generations of random messages are being passed within the network, each represented a different color. Each link in Fig.6 is numbered according to the corresponding generation of the message it transmits. Consider node v which has received two messages from generation 2, one from generation 1 and another from generation 3. Note that v contains within its buffer vectors from all three generations. Hence this node accepts all of its incoming packets but generates its outgoing messages from generation 1 which has a higher priority in the buffer. The rest of the incident incoming packets at v are stored to be used in priority sequence. The corresponding buffer at v 0 contains only messages from generation 3 while the preceding generations have been flushed out of the buffer. Therefore v 0 accepts the incoming message of generation 3, rejecting messages of any other generation. Similarly sequences of generation
6
Fig. 7. A snapshot of the first round of operation in a gossip-based communication in a network. Fig. 6. Snapshot of three nodes within a network that operates a practical randomized network coding scheme with buffering as proposed by Chou et al. in [2]
1 have been flushed out from the buffer at v”, therefore the packet corresponding to this generation is rejected. Since generation 2 has priority over generation 3 in the buffer, the incident incoming packet of generation 3 is accepted and stored in the buffer at v” while the output messages are created as linear combinations of generation 2 packets, previously stored.
VI. A LGEBRAIC G OSSIP Consider a scenario where there exist n nodes in the network, a subset of which have an overall knowledge about k messages. For example node i may have message m1 and m2 , and is interested in the rest of the k − 2 messages. Similarly node j may have m1 ,m3 and m7 and is interested in the other k − 3 messages. However, not all k message are with all the n nodes to start with. The objective is to design the message transmission between the nodes in order to ensure that all the n nodes receive all the k messages as fast as possible. This problem has been posed by Deb and M´edard in [3]. They propose a gossip-based protocol where random linear coding is used to disseminate a set of k messages, M , in a large network of n nodes. The protocol is decentralized and distributed by design. A gossip-based protocol works in discrete-time steps or rounds, and in every round, a node selects a communication partner from other nodes in the network. Therefore all the nodes in the network are receivers and transmitters at the same time. Fig.7 and Fig.8 illustrate this scheme in two successive time steps. There are eight nodes in the network and each of the nodes has a distinct message to start with, and the goal is to disseminate all the messages
Fig. 8. A snapshot of the second round of operation in a gossipbased communication in a network.
among all the nodes. In the first time-step, each node picks up a node at random and transmits the message it has. In the second round, some of the nodes already have two distinct messages. For transmitting a message in the second round, these nodes have to decide which message to transmit without the knowledge of the contents of its communication partner. Any gossip protocol has two layers of design aspects: 1) a gossip algorithm by which in every round, every node decides its communication partner, using a deterministic randomized scheme. 2) a protocol by which any node, upon deciding the communication partner decides the content of the message to send to the communication partner. The proposed dissemination scheme operated as follows: A. Message Communication Model At the beginning of each round, each node u has a communication partner v selected uniformly at random
7
from within the n − 1 possible nodes. Once a communication partner is chosen, there will be two possible transmission directions, and hence two different models for message exchange: 1) Having chosen a called node v , the caller node u transmits a message to v 2) Having been chosen by the caller u, the called node v transmits to the caller node, allowing the communication process to be initiated by the transmitting node B. Message Transmission The messages are assumed to vectors over the finite field Fq of size q ≥ k . The key idea is to have the nodes collect several linear combinations of the messages in M , where the linear combinations are generated according to the practical scheme of random linear coding discussed in V. Each node can recover all the messages successfully once it collects k independent linear combinations. In other words, once the dimension of the subspace spanned by the received ”code vectors” at a node becomes k , then the node can recover all the messages. The proposed message transmission is based on the practical scheme of random linear coding discussed in V, in which augmenting the random coding vectors as overhead with each packet as described in (7) will only add an additional klog2 q bits with each message. Therefore this scheme disseminates all messages to all nodes in time √ ck + O( kln(k)ln(n)) The asymptotic bounds on the number of transmission rounds required for all the n nodes to recover all k messages with high probability, as provided by Deb and M´edard [3] is as follows: R=
ck √ Ω( k/lnk)
if k ln(n)3 ; if k ln(n)2 ;
(8)
For detailed derivation of the above bound we refer the reader to the original paper. VII. D ISCUSSION The goal of this survey is to introduce the recent research achievements in randomized algorithms in networking design. It has been shown that network coding is particularly beneficial over routing schemes in networks [8]. Therefore the emphasis of our study has been mainly on the theory of random network coding and the practical implementation of such protocols.
Throughout this survey we studied a distributed randomized network coding approach presented by Ho et al. [9], which asymptotically achieves optimal capacity in multi-source multicast networks. Moreover we reviewed a gossip-based dissemination mechanism proposed by Deb and M´edard in [3]. This scheme utilizes a random linear coding protocol, which has proven to be beneficial in spreading messages across the network in optimal time. Moreover, it has been shown that in context of multicasting correlated data, network coding plays two roles: compression and efficient information relaying. Random mixing is theoretically optimal asymptotically. However, with random mixing, the decoding complexity is very high, because the destination node may receive a linear system that has fewer equations than unknowns. The solution proposed by Chou et al. in [2] achieves low complexity by a careful combination of random mixing. Thus far, the best understood scenario where random network coding offers unique advantages is multicasting in a communication network. Thus future work in this area would be extensions of distributed randomized network coding to non-uniform code distributions. R EFERENCES [1] R. Koetter and M. M´edard, “An algebraic approach to network coding,” IEEE/ACM Transactions on Network Coding, vol. 11(5), pp. 782 – 795, 2003. [2] P. Chou, Y. Wu, and K. Jain, “Practical network coding,” 51st Allerton Conference in Communication, Control and Computing., 2003. [3] S. Deb, M. M´edard, and C. Choute, “Algebraic gossip: a network coding approach to optimal multiple rumor mongering,” IEEE Transactions on Information Theory, vol. 52, pp. 2486–2507, 2006. [4] R. Ahlswede, N. Cai, S.-Y.R. Li, and Yeung.R.W., “Network information flow,” IEEE Trans. Inf. Theory, vol. 46, pp. 1204– 1216, 2000. [5] S.Y.R. Li, R.W. Yeung, and N. Cai, “Linear network coding,” Proceedings of 2003 IEEE Transactions on Information Theory, vol. 49, pp. 371–381, 2003. [6] S.D. Servetto, “Constrained random walks on random graphs: Routing algorithms for large scale wireless sensor networks.,” Proceedings of the 1st ACM International Workshop on Wireless Sensor Networks and Applications, 2002. [7] Y. Qin and C.Y. Lee, “A probability random scheme of distrobuted routing algorithm for manets.,” Proceedings of the ninth internation conference on communication systems, ICCS, vol. 7, pp. 65–69, 2004. [8] T. Ho, R. Koetter, M. M´edard, D.R. Karger, and M. Effros, “The benefits of coding over routing in a randomized setting,” Proceedings of 2003 IEEE International Symposium on Information Theory, p. 442, 2003. [9] T. Ho, M. M´edard, J. Shi, M. Effros, and D.R. Karger, “On randomized network coding,” Proceedings of 41st Annual Allerton Conference in Communication, Control, and Computing, 2003.
