Multi-Commodity Network Flows over Multipaths with ... - CiteSeerX

This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the IEEE ICC 2010 proceedings

Multi-Commodity Network Flows over Multipaths with Bounded Buffers Tricha Anjali, Alexander Fortin

Sanjiv Kapoor

Department of Electrical and Computer Engineering Illinois Institute of Technology Chicago, Illinois Email: {anjali, afortin}@iit.edu

Department of Computer Science Illinois Institute of Technology Chicago, Illinois Email: [email protected]

Abstract—In this paper we address the issue of designing multipath routing algorithms. Multi-path routing has the potential of improving the throughput but requires buffers at the destination. Our model assumes a network with capacitated edges and a delay function associated with the network links (edges). We consider the problem of establishing a specified throughput from multiple source to destination pairs in the network, given bounds on the buffer sizes available at the network intermediate nodes and a bound on the maximum delay that the paths are allowed to have. We formulate the problem using a linear programming model. We also present practical heuristics and present the experimental results on an existing network topology. The results are promising and demonstrate the effectiveness of simultaneous multipaths in the multi-commodity scenario.

I. I NTRODUCTION Traditional algorithms for establishing network connectivity between a source-sink pair find a single path of shortest length (delay) between the source and destination. However, this path may be plagued by congestion problems since this is the path that will always be chosen to forward the total traffic between the node pair. On the other hand, splitting the traffic among multiple paths utilizes the paths simultaneously and reduces the congestion on any given path, while increasing the overall network transmission capacity. Multipath routing protocols work on the principle that higher performance can be achieved by utilizing more than one feasible path [1]. Multipath routing can be effectively used for maximum utilization of network resources by giving the node a choice of next hops for the same destination. Multipath routing has been proposed to take advantage of network redundancy, reduce congestion, and address QoS issues [2], [3]. Traffic engineering, lower delay, increased fault tolerance, and higher security are other compelling reasons that exist for discovering and utilizing multiple paths. The main disadvantage associated with the multipath routing manifests as the packet disordering at the receiver, since the traffic is split into these multiple paths with different latencies creating jitter. Solutions to this problem for the TCP protocol have been proposed in the literature such as [4], [5]. The problem of different latencies on multiple paths can be resolved via either bounding the jitter or by the use of buffers at the destination. The choice of multipaths entails that, to synchronize across various paths, buffers need to be established at the destination so that the data packets

can be stored and sequenced appropriately. We contend that we can utilize the buffers at the intermediate network nodes to increase the throughput and decrease the buffer requirement at the destination. In this paper, we formulate optimization problems to model the routing issues in multi-source-sink multipath routing. We want to establish a specified throughput from multiple sources to respective destinations in the network, given bounds on the buffer size available at the network nodes. The paths to be established are required to be of bounded delay. This formulation is general enough and can be applied to networks with different approaches to implement multipath routing. Our model assumes a network with capacitated edges and a delay function associated with the network links (edges). We formulate the optimal Fixed Buffer Multi-Commodity Multipath Flow problem (FBMCMPF) to determine the set of multipaths between the source-destination pairs given a fixed buffer size at the network nodes. We design a minimum cost network flow problem which models the bounded-buffer problem. This leads to a pseudo-polynomial algorithm, based on linear programming. The solution of the linear program brings out an interesting feature associated with the MCMPF problem. It may be beneficial to use long paths in the network in order to satisfy the buffer bounds, since paths which have similar delays would require smaller buffer sizes. Long paths can be obtained by accruing the delay in a cycle one or multiple times. Thus, the artificial construct of choosing non-simple paths (paths with repeating nodes, or in other words, with cycles) can lead to smaller buffer sizes. The explicit routing capability provided by virtual circuit based technologies can be used to establish these non-simple paths in a network. Before the actual data transfer, a virtual circuit is established between the end-hosts. During the establishment of the virtual circuit, identifier translation tables are created at each intermediate hop. The data is then forwarded on the virtual circuit by performing an identifier translation. Related problems have been studied, but we are not aware of any work dealing explicitly with buffer size constraints. In the seminal work [6], the authors address multipath routing issues as a non-linear optimization problem. They are concerned with delay and loss rate in an adaptive setting, and their cost per

978-1-4244-6404-3/10/$26.00 ©2010 IEEE

This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the IEEE ICC 2010 proceedings

link is a function of the traffic on that link, an approach which ignores the buffers at the receiver. A fixed number (k) of actual paths are constructed in [7] to maximize the total flow (throughput). We also maximize flow, but we consider together the buffer constraint, and allow softer (dictated by total cost) constraint on the paths. The paper is organized as follows. In Section II, we introduce the mathematical model for the Fixed Buffer MultiCommodity MultiPath Flow (FBMCMPF) problem. In Section III, we provide the model for the FBMCPF problem. In Section IV, we discuss the heuristic solution followed by the experimental results in Section V, implementation issues in Section VI and conclusions in Section VII. II. T HE B OUNDED B UFFER M ULTI -S OURCE -S INK M ULTIPATH P ROBLEMS Following the terminology in [8], we define the multisource-sink multipath bounded-latency network flow problem: Suppose we are given a digraph G = (V, E), with n nodes and m edges (links), a capacity function c(u, v) : E → R+ , a length (delay) function l(u, v) : E → R+ , and a set of distinct source-sink pairs, (S-T) such that required flow is to be transmitted between each source-sink pair. Note that we do not assume that delays or capacities across the edges to be equal. Given a flow path P from source s to sink t, the delay on the path, L(P ), is the sum of the delays on each edge of the path, i.e.  l(u, v) L(P ) = e=(u,v)∈P

The multi-commodity multipath flow problem (MCMPF), is to generate a set of s-t paths, (s, t) ∈ ST , P1 (s, t), P2 (s, t), ..., Pk (s, t) with corresponding flow values f1 (s, t), f2 (s, t), ..., fk (s, t) such that the following conditions are satisfied k i=1 fi (s, t) = γ(s, t) (demand constraint) i,e=(u,v)∈Pi (s,t) fi (s, t) ≤ c(u, v); ∀e = (u, v) ∈ E (capacity constraints) L(Pi (s, t)) ≤ L; ∀Pi (s, t) (path length (delay) constraint) where γ ∈ Z + . An instance of the multipath flow problem is denoted as M CM P F (N, γ, L), where N is the input network characterized by N = (G, c, l, S − T ). Because of jitter, multi-paths will require constraints on the buffer size so as to reduce the impact of the jitter. We define the following problem: Fixed Buffer Multi-Commodity MultiPath Flow (FBMCMPF): Suppose paths P1 (s, t), P2 (s, t) . . . Pk (s, t) carry flow f1 , f2 . . . fk , where P1 (s, t) is the path of maximum length (or delay), in order to achieve the desired throughput. The flow transmission schedule require that data packets transmitted at the rate fi on paths Pi (s, t), 1 ≤ i ≤ k would accumulate at the destination within this time frame. The number of packets arriving between times t + L(Pi (s, t))

and t + L(P1 (s, t)) on path Pi is given by fi · (L(P1 (s, t)) − L(Pi (s, t))). These packets could be out of sequence and will be buffered Assuming that the buffer size at each node in the network is given as a constant, we achieve the following constraints for the the various nodes in the network: For the sink we have the constraint:  fi · (L(P1 (s, t)) − L(Pi (s, t))) ≤ B. (1) (s,t)

i

Here B is the buffer size at sink, and P1 is the path with maximum length (delay). For every intermediate node, we can model the buffer as a self-loop at node v with capacity B and a delay of 1. Thus a packet that stays in the buffer for T time steps is modeled by a path that cycles around the loop T times. In fact, we can add a loop at each sink to model the buffer at the sinks also. The delay function is given as l : E → {1, 2, . . . , k}. III. L INEAR P ROGRAM M ODEL As in [8] we provide a solution using Linear programming. The solution involves fixing a value of L(s, t) as a guess that the longest path used for flows from s to t has length (delay) exactly L(s, t). A range of values of L(s, t) are tried. We denote by P(L) the sets of such paths. With the length (delay) restriction above, the problem becomes: Given a network N = (V, E, c, l, S − T ) and integers L(s, t), γ(s, t), B, we are to find a set of paths Pi (s, t), i ∈ {1, 2, . . . , q}, carrying a flow fi such that: l(P1 (s, t)) = L(s, t) ∀(s, t) q  fi = γ(s, t)∀(s, t) q 

(2) (3)

i=1

fi (s, t)m(i( s, t), e) ≤ c(e),

(4)

(s,t) i=1

for all e ∈ E, where m(i(s,t) , e) is the number of times path Pi (s, t) uses edge e. We model the above problem by a network flow model. Similar constructions appeared in [9], [7] for variations of max-flow. Given a choice of L(s, t) let L = max(s,t) L(s, t). Construct ˆ as follows. Vˆ = V (N ˆ ) = ∪L V1 We call ui a network N i=0 the copy of node u from Vi . For an edge e ∈ E with tail u ˆ ˆ )), for each i = 0, 1, . . . , L − l(e), and head v put in E(E( N an edge ei with tail ui and head vi+l(e) . Also add edges gj , for j = 0, 1, 2, L(s, t) − 1, with tail tj and head tL (s, t); these are the only edges with cost: cost(gj ) = L(s, t) − j. The goal is to ship at minimum cost γ(s,t) units of flow from each pair ˆ , subject to joint (s, t) ∈ ST a flow from s0 to tL(s,t) in N capacity constraints as follows. For each edge e ∈ E, and each i = 0, 1, 2, . . . , L−l(e), we have f(s,t) (ei ) as the flow on edge ˆ shipping flow corresponding to the (s, t) pairs. We ei of N use the following notation: for a node u, δ + (u) is the set of edges with tail u. and δ − (u) is the set of edges with head u.

This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the IEEE ICC 2010 proceedings

The linear feasibility program modeling this problem is to determine the feasible solution to: L(s,t) −1



 eˆ∈δ − (ˆ v)

(L(s,t) − j)f(s,t) (gj ) ≤ B, ∀(s, t)

j=0



f(s,t) (ˆ e) =

f(s,t) (ˆ e) ∀(s, t), ∀ˆ v ∈ Vˆ \ {s0 , tL }

eˆ∈δ + (ˆ v)



f(s,t) (ˆ e) = γ(s,t) , ∀(s, t)

eˆ∈δ + (s0 ) L−l(e)

  i=0

f(s,t) (ei ) ≤ c(e) , ∀e ∈ E

(s,t)

ˆ f(s,) (ˆ e) ≥ 0 , ∀ˆ e∈E This feasibility can be computed for each possible vector of lengths (L(s,t) , ∀(s, t)) to provide the feasible minimal vector. Note that this would require Rk invocations of the linear program where R is the bound on the range of values that L(s,t) can take. This cost may be prohibitive and so we consider simpler heuristic algorithms. IV. H EURISTIC S OLUTION Because of the NP-complete nature of the FBMCMPF problem, we propose a heuristic algorithm to find feasible paths that satisfy the constraints. The interesting aspect of heuristics is that they allow us to get faster, real-time solutions with results close to the optimal, besides the fact that we can work with larger network topologies and larger parameters than the optimal program. Our heuristics aim to maximize the routed flow subject to either jitter or buffer constraint - in fact it can handle both constraints if needed. The proposed heuristics are greedy in nature. Given the network topology, we use Shier’s algorithm [10] to find the k shortest paths in the network for each source-sink node pair. Repeated nodes are allowed in each of the paths. This algorithm sorts the paths in the increasing order of their costs. The link delays are used as the cost for the links. So, the paths obtained by this algorithm are in increasing order of their delays. These paths are then used in the determination of the usable paths using two approaches. In multiple user algorithms, allocation of the network resources is the main issue. Two different methods can solve this problem. The first one is the fair allocation algorithm in which the network is fairly shared among the users according to the users requirements. The second solution is using chunk allocation. In this case, overall network capacity is divided into chunks and those chunks are allocated one by one to users. Heuristic 1: Fair Allocation: In this case, two main metrics are used: network capacity and number of users. The network capacity is the overall network capacity. The number of users is the number of users who are currently asking for bandwidth, i.e. the number of still unsatisfied users. The algorithm can be outlined by the following steps:

1) All link capacity are divided by the number of users. 2) This network with divided capacity is then allocated to one user at a time. 3) When the program has gone through the users list, it computes the remaining network. 4) The program goes back to the first step (division of network capacity by the number of unsatisfied users) until the network is saturated (no more bandwidth to allocate) or all users are satisfied. This algorithm guarantees that all users are initially granted at least some flow in the network. As far as possible, the network is shared fairly among the users. Heuristic 2: Chunk Allocation: With this algorithm, allocation is done in bandwidth chunks. It means that the overall network capacity is allocated using chunks. Then, the algorithm can be outlined as follows: 1) Users requirements are divided into chunks. 2) These requirements (composed of chunks) are now fulfilled one chunk at a time. 3) After one round, the remained network is computed (update of links capacity) and the algorithm goes back to step 2 until all users are satisfied or buffer at receiver side overflows or the network is saturated. This algorithm is designed to assign bandwidth chunk by chunk. Chunk size is a critical issue because processing time depends on chunk size. However, this algorithm is not fair since there is no guarantee that each user will be allocated even a single chunk. The pseudo-code for the first heuristic is given next. The second heuristic is similar. Algorithm 1: Heuristic 1 1: for Each user do 2: Obtain a pseudo-network by dividing the link capacities by the number of users 3: Find the k shortest paths in the pseudo-network using length as cost 4: Pick the first path from list of shortest paths 5: if Length of this path Length constraint then 16: break 17: end if 18: if Buffer constraint is violated then 19: break 20: end if 21: Add this path to list of chosen paths

This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the IEEE ICC 2010 proceedings

Fig. 1.

GEANT topology

Reduce the demand to be routed by the amount send on this path 23: Decrease every links capacity by this amount as well 24: Increase counter 25: end while 26: end for An extension to these heuristics also provides the solution for the distributed buffer scenario. As mentioned earlier, the advantages of using buffers at intermediate nodes include higher throughput and reduced buffer requirement at the destination, along with the potential for the reuse of the same buffer for many flows. 22:

V. S IMULATION R ESULTS We have implemented the two heuristics on a Java platform. We have used two topologies to demonstrate the results. The first one is a simple 6 node topology for easy verification of the results and second one is a real topology, obtained from [11]. This network called GEANT is a pan-European backbone which connects Europe’s national research and education networks. The figure 1 shows the topology. The network has 33 nodes and 94 links. The link delays are estimated based on the length of the links and are assigned between 1 and 30 msec. The reason for using this network topology is that it represents the kind of backbone for which the proposed heuristic schemes are designed. Especially, connections between nodes are highly static and there is a high level of availability. Moreover, user requirements do not vary so much. In the following, we will present results for the GEANT topology. We will describe the results in the case where the buffer at the destination node is used to account for the delay. We will then also briefly discuss the case of using distributed buffers located at intermediate nodes. Fair allocation algorithm is composed of two parts. The first one allocates fairly network resources among users. The

second part is the wastage bandwidth avoidance algorithm which allocates the remaining bandwidth to still unsatisfied users (according to their requirements). Similarly, the chunk allocation algorithm is composed of two parts. The first one allocates chunk resources to the users. The second part is the wastage bandwidth avoidance algorithm which allocates the remaining bandwidth to still unsatisfied users (according to their requirements). In both the algorithms, the first part consists of various iterations until either the network capacity is exhausted or all the user requests are met. We considered 6 commodities for the GEANT network. In figure 2, we show the total flow that can be sent between all the sources and destinations as a function of the buffer size at the receivers, obtained with the two heuristics. The first data point where buffer is 0 corresponds to the least cost single path flow, currently being used in TCP/IP protocols. We notice that for 0 buffer, the two algorithms do not have the same output. This is because of the difference in their allocation methods. Also, increasing the buffer size has substantial impact on the flow in the network. Using a buffer of approximately 52Mb, we are able to achieve throughput of 35Gb/s which is about twice as compared to the single shortest path throughput of 20Gb/s. We expect that the optimal program, on the other hand, will be able to achieve a higher throughput with no buffer. This is because the optimal program allows cycles in the paths which can adjust the delay of the paths such that all paths have similar delay. In the figure 3, we show the congestion in the network for the two heuristics as the maximum flow through the network is increased. As we can see, as the flow increases, the network congestion increases, as expected. However, overall the congestion of the fair allocation scheme is higher than the chunk allocation, since the fair allocation utilizes the network resources faster. Note that more flow can be sent through the network even after the congestion has reached the value of 1, since congestion value is defined as maximum among all the links and does not imply that all the links are fully utilized. The heuristic program typically runs in less than 1 sec whereas the optimal program will take much longer to execute. The fair allocation algorithm is slow to reach the saturation level because all the users are able to send small fraction of their flows and so larger buffers will be needed. Also noticeable is the difference in performance of the chunk allocation algorithm for different chunk sizes. This is due to the fact that large chunk size implies faster allocation of the network and so more throughput for smaller buffers. Another metric of comparison among the two heuristics is the processing time. We used a computer with 3 GHz Pentium 4 processor and 1 GB RAM for the simulations. As the number of commodities increases, the trend in increase in processing time is similar for the two algorithms. However, the processing time for the chunk allocation is highly dependent on the chunk size. For the same commodities, as the chunk size increases, the processing time for the algorithm reduces exponentially. This is due to the fact that larger chunks allocate the network resources sooner and the number of iterations required will

This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the IEEE ICC 2010 proceedings

FairallocationversusChunkallocationalgorithms 40,00 35,00 30,00

Throughput inGbit/s 25,00

Fairallocationalgorithm

20,00

Chunkallocationalgorithm (sizeof5)

15,00 10,00

Chunkallocationalgorithm (sizeof50)

5,00 0,00

BuffersizeinMbits

Fig. 2.



Heuristic results for GEANT topology

1

09 0.9

0.8

0.7

0.6

Congestion 0.5

Fair Heuristic Heuristic4 4 Chunk Heuristic4

0.4

0.3

0.2

0.1

are included for redundancy and efficiency, and there is no bottleneck link. Currently, the algorithms are designed to be executed by a central entity responsible for the network. If there is a distributed way of propagating information about the network state to all the nodes, then the proposed algorithms can be deployed in a distributed manner. Also, one assumption is that the network delays are not dynamic. In other words, during the calculation of the paths, we assume that the delay values do not change. We can incorporate slight variability depending on the update frequency. As new requests arrive in the network, updated link delay values can be used to calculate the current paths for the flows. Other issues noted during the simulations and implementation of the proposed heuristic algorithms can be summarized as follows. In the case of fair allocation algorithm, the network capacity is divided by the number of commodities. In the core of the network, the number of commodities may be very large. Also, this division may result in allocations which are not feasible. For the chunk allocation algorithm, the chunk size is a critical factor to evaluate the performance of the scheme. VII. C ONCLUSIONS

0 4

11

20

33

35

40

45

50

55

MaximumFlow

Fig. 3.

Network Congestion

be less. In the figure 4, we show the processing time for the chunk allocation heuristic as a function of the chunk size. For the case of the distributed buffers, we can apply similar fair and chunk allocation algorithms for determination of the buffer size at the intermediate nodes. 90000 80000 70000 60000 Chunksize1

50000 Processingtime inms 40000

Chunk size 2 Chunksize2 Chunksize5 Chunksize10

30000 20000 10000 0

BufferSize

Fig. 4.

Processing time for the chunk allocation heuristic

VI. I MPLEMENTATION I SSUES We expect that the flow problems mentioned in this paper will be applied to the core of the network. Typically, the access nodes have only single connection to the network and multipaths are not possible. Also, typically the access links are the bottlenecks in the end-to-end paths. So, these algorithms are only applied to the network core where multiple paths

In this paper, we have considered multi-commodity multipath routing problems with buffer constraints. We have shown optimal theoretical results to obtain optimal buffer size for a desired total throughput. We have also presented two heuristic solutions which have been experimentally shown to substantially improve the throughput as compared to the current single-path solutions. Our theoretical solutions give polynomial-time approximations for the problems when we have a fixed number of source-sink pairs. R EFERENCES [1] N. F. Maxemchuk, “Dispersity Routing”, in Proceedings of IEEE ICC, 1975. [2] H. Suzuki and F. A. Tobagi, “Fast Bandwidth Reservation with Multiline and Multi-path Routing in ATM Networks”, in Proceedings of IEEE Infocom, Florence, Italy, 1992, pp. 2233–2240. [3] I. Cidon, R. Rom, and Y. Shavitt, “Analysis of Multi-Path Routing”, IEEE/ACM Trans. on Networking, vol. 7, pp. 885–896, December 1999. [4] S. Bohacek, J. Hespanha, J. Lee, C. Lim, and K. Obraczka., “A New TCP for Persistent Packet Reordering–TCP-PR”, IEEE/ACM Trans. on Networking, to appear in April 2006. [5] H. Han, S. Shakkottai, V. Hallot C, R. Srikant, and D. Towsley, “MultiPath TCP: A Joint Congestion Control and Routing Scheme to Exploit Path Diversity in the Internet”, vol. 14, pp. 1260–1271, December 2006. [6] A. Elwalid, C. Jin, S. Low, and I. Widjaja, “MATE: MPLS Adaptive Traffic Engineering”, in Proceedings of IEEE INFOCOM’01, Anchorage, USA, April 2001, pp. 1300–1309. [7] R. Banner and A. Orda, “Multipath routing algorithms for congestion minimization”, in Proc. IFIP Networking 2005, Waterloo, Ontario, Canada, May 2005. [8] T. Anjali et al., “Multipath Network Flows with Bounded Buffers”, in Accepted for publication in IEEE INFOCOM, 2010. [9] S.T. McCormick, “The complexity of max flow with bounded-length paths”, Working paper, 1006. [10] D. R. Shier, “Iterative Methods for Determining the k Shortest Paths in a Network”, Networks, vol. 6, pp. 205–229, 1976. [11] GEANT network pan-European backbone, http://www.geant.net/upload/pdf/Topology Oct 2004.pdf.