Global Stability with Averaged Feedback and Network Delay

1

Global Stability with Averaged Feedback and Network Delay Richard J. La and Priya Ranjan

Abstract— We analyze a variant of rate control framework where feedback signal is generated based on the averaged load and is delayed. We derive a sufficient condition for global stability in the presence of arbitrary communication delay and gain, and show that a sufficient condition derived earlier for a system with no averaging suffices. We validate our result using numerical examples with a family of popular utility and price functions. Index Terms— Rate control, global stability, delay.

I. I NTRODUCTION With emerging networked control systems, control of a distributed system in the presence of varying delays is emerging as an interesting and important issue. Examples of such systems include a sensor network consisting of arrays of sensors that collect and provide feedback information for a control system, and a communication network with many end users that individually adjust their transmission rates based on the feedback information provided by active queue management (AQM) schemes [1], [2], [10]. In this article we study a variant of a rate control system proposed in [8], which is first studied in [13]. In this rate control system the feedback provided to the end user is based on the averaged or low pass filtered version of the user rate (as opposed to the instantaneous rate as in [8]) and is delayed due to the presence of network delays. This resembles the spirit of Random Early Detection (RED) [2] or Proportional controller [4] type schemes with one or more stages of low pass filtering. Stability results based on linear analysis are derived in [13], and are shown to depend on both delay and gain. In [12] we study the stability of a rate control system based on an optimization framework proposed in [7]. We derive global stability criteria in the presence of arbitrary network delays. These stability conditions are derived using the invariance-based global stability results for nonlinear delay differential equations [5], [6]. In this article we show that the same basic approach we adopt in [12] can be extended to study the stability of a system where congestion/feedback signal is based on filtered load as done in [13] and is delayed due to a network delay, and derive global stability criteria on utility and resource price functions. The rest of this article is organized as follows: We present our main results on global stability in Section II. Numerical examples are presented in III. We conclude in Section IV. The authors are with the Department of Electrical and Computer Engineering and the Institute for Systems Research, University of Maryland, College Park, MD 20742 USA. Email: {hyongla,priya}@isr.umd.edu.

II. R ATE C ONTROL

WITH

F EEDBACK D ELAY

We consider a variant of Kelly’s model [8] with a single resource and a single flow. Although here we consider only a single user, the model and the results presented in the paper can be easily extended to any finite number of users. Unlike in the Kelly’s model in which the feedback signal is a function of instantaneous rate, we assume that the feedback signal depends on the average rate of the flow over a period. Furthermore, the feedback signal from the resource is delayed before the sender receives it due to the network delay. In practice estimating the instantaneous rate through a resource is difficult, if not impossible, and can be very noisy. Thus, in order to obtain a better estimate of the aggregate rate, a resource may need to use an average over some period. As will be shown shortly, we model this averaging by introducing a low pass filter at the resource [13]. We first show the existence of a unique solution of the system and then establish the conditions under which the system is stable regardless of the feedback delay. A. Background In the optimization framework for rate control proposed by Kelly [7] a user receives utility U (x) when it gets a rate of x. This utility function could represent either the user’s true utility or some function assigned to the user through the selected end user algorithm. We take the latter view and assume that the utility functions of the users are used to select the desired rate allocation among the users. The utility U (x) is an increasing, strictly concave and continuously differentiable function of x over the range x ≥ 0. We let x(t) denote the user’s rate at time t and µ(t) := p(ω(t)) the price charged by the resource at time t as a function of the low pass filtered estimate of the load ω(t) at time t. We assume that the price function p(ω) is non-negative, non-decreasing and continuously differentiable. Since a user’s rate is bounded in practice, we assume that the rate belongs to a compact set [Xmin , Xmax ] ⊂ IR+ := (0, ∞). Here the lower (resp. upper) bound Xmin (resp. Xmax ) can be arbitrarily close to zero (resp. arbitrarily large). B. Feedback signal based on low pass filtered load In our model, unlike in the original Kelly’s primal algorithm, the congestion/feedback signal generated at the resource is a function of a low pass filtered version of load. We assume that the feedback signal generated by the resource is returned to the user after a fixed round trip time T ∈ IR + . In

2

the presence of low pass filtering and a feedback delay the interaction between the user and the resource is given by the following delay differential equation [8], [12]. d x(t) = κ (w(t) − x(t − T )µ(t − T )) , dt = κ (x(t)U 0 (x(t)) − x(t − T )p(ω(t − T ))) , d α>0, (1) α ω(t) = −ω(t) + x(t) , dt where κ > 0, and w(t) := x(t)U 0 (x(t)) denotes the user’s willingness to pay at time t. It is clear from (1) that the user always attempts to reach an equilibrium where its willingness to pay equals its total price. In this article we are interested in finding conditions on user’s utility and resource price functions for the asymptotic global stability of the system in (1). First, for notational simplicity let us define g(x) and f (x) to be user’s willingness to pay and total price, respectively, as a function of its rate as follows: 0

g(x) := x · U (x) and f (x) := x · p(x)

(3)

This allows us the following change of coordinate. x(t) = g −1 (y(t)) ⇒ x(t) ˙ =

y(t) ˙ g 0 (g −1 (y(t)))

(4)

where the inverse g −1 (·) exists from Assumption 1. Let κ(y(t)) := −κg 0 (g −1 (y(t))). Clearly, κ(y(t)) > 0 under Assumption 1. We write (1) in terms of y(t).
d y(t) = κ(y(t)) g −1 (y(t − T ))p(ω(t − T )) − y(t) dt d (5) α ω(t) = −ω(t) + g −1 (y(t)) dt We study the system in (5) and show that there is a close correspondence between the invariance and global stability properties of a discrete-time map yn+1 = f (g −1 (yn )) := F (yn )

Since the functions in (5) are Lipschitz continuous by assumption, a unique solution exists for all t ≥ 0 for any initial function [φy ; φω ] ∈ YI × Yg−1 (I) . Furthermore, the invariance property of the solutions stated below (Theorem 1) ensures that it stays positive and bounded by the initial set they start in, which is assumed to be invariant under map F . Theorem 1: If initial function [φy ; φω ] ∈ YI × Yg−1 (I) , then the corresponding solution (y(t), ω(t); φy , φω ) satisfies y(t) ∈ I and ω(t) ∈ g −1 (I) for all t ≥ 0. Proof: Before proving this theorem we first state two lemmas that will be used in the proof of the theorem. Lemma 1: Suppose that I ? := [a? , b? ] ⊂ IR is a compact interval and η : [0, ∞] → IR is a continuous function with values in I ? . If σ : [0, ∞] → IR+ is a continuous, strictly positive, and bounded function, and u(t) is a solution of following equation σ(t)

(2)

We make the following assumptions on the functions g(x) and f (x). Assumption 1: (i) The function g(x) is strictly decreasing with g 0 (x) < 0 for all x > 0, (ii) the function f (x) is strictly increasing for all x > 0, and (iii) both g(x) and f (x) are Lipschitz continuous on [Xmin , Xmax ]. An example of a family of utility and resource price functions that satisfies Assumption 1 is used in numerical examples in Section III. We rewrite the system given in (1) in a form more amenable to analysis: Define y(t) := g(x(t)) .

C. Existence of a unique solution & global stability

(6)

and those of (5) for fixed orbits as done in [12] with no averaging. Assumption 2: Suppose that I := [a, b] ⊂ g([Xmin , Xmax ]) is a compact interval invariant under the map F . Let Y := C([−T, 0], IR+ ) be the Banach space of continuous functions mapping the interval [−T, 0] to IR + , and for all A ⊂ IR+ , YA := {φ ∈ Y | φ(s) ∈ A for all s ∈ [−T, 0]}.

d u(t) + u(t) = η(t) dt

(7)

with u(0) ∈ I ? , then u(t) ∈ I ? for all t ≥ 0. Proof: We prove this lemma by contradiction. Suppose that the lemma is not true. Then, let t0 ≥ 0 be the first time at which the solution leaves I ? . In particular, suppose u(t0 ) = b? and every interval (t0 , t0 + δ), δ > 0, contains a point τ such that u(τ ) > b? and u(τ ˙ ) > 0. However, if u(τ ) > b? , from (7) we must have u(τ ˙ ) < 0 because η(t) ≤ b? , which is a contradiction. The case u(t0 ) = a? is handled similarly. This completes the proof. This lemma is essentially based on [3]. A similar lemma can be also found in [11]. Lemma 2: Suppose that I ? := [a? , b? ] ⊂ g([Xmin , Xmax ]) is a closed interval invariant under map F . If y(t) ∈ I ? and w(t) ∈ g −1 (I ? ) for all t ∈ [t1 , t1 + T ], then y(t) ∈ I ? for all t ∈ [t1 + T, t1 + 2T ]. Proof: We first rewrite the first equation in (5): d y(t) + y(t) = g −1 (y(t − T ))p(ω(t − T )) dt Then, from Lemma 1 one can see that in order to prove the lemma, it suffices to show that the right-hand side, i.e., g −1 (y(t − T ))p(ω(t − T )), lies in I ? for all t ∈ [t1 + T, t1 +2T ]. This follows directly from the assumed invariance property of I ? under map F and monotonicity properties of the functions g and f as follows. First, note that the map F (y) = f ◦ g −1 (y) = g −1 (y) · p(g −1 (y)). Since the price function is assumed to be non-decreasing and the function g is monotonically decreasing, one can see that, for all t ∈ [t1 + T, t1 + 2T ], κ(y(t))−1

g −1 (b? )p(g −1 (b? )) ≤ g −1 (y(t − T ))p(ω(t − T )) ≤ g −1 (a? )p(g −1 (a? )) because y(t − T ) ∈ I ? and ω(t − T ) ∈ g −1 (I ? ). From the invariance property of I ? , we have F (I ? ) = [g −1 (b? )p(g −1 (b? )), g −1 (a? )p(g −1 (a? ))] ⊂ I ? , and the claim follows.

3

We now proceed with the proof of the theorem. From Lemma 2, if φω ∈ Yg−1 (I) and φy ∈ I then y(t) ∈ I for 0 ≤ t ≤ T . Using this and applying Lemma 1 to the second equation in (5) it is plain to see that ω(t) belongs to g −1 (I) for 0 ≤ t ≤ T because y(t) lies in I for 0 ≤ t ≤ T . Now the claim follows from an induction argument on time (called step method). We now consider the case where the map F has an attracting fixed point y ? with immediate basin of attraction I0 : F n y0 → y ? for any y0 ∈ I0 . Assumption 1: Suppose that there is a sequence of compact intervals Ik , k = 0, 1, 2, . . ., such that F (Ik ) ⊂ int(Ik+1 ) ⊂ Ik+1 ⊂ int(Ik ) for all k = 0, 1, . . ., where int(Ik ) denotes ? the interior of Ik , and ∩∞ k=0 Ik = {y }. ? It is clear that the fixed point y is the user’s willingness to pay at the equilibrium. In other words, if x? is the equilibrium rate that satisfies U 0 (x? ) = p(x? ), then y ? = g(x? ) = x? · U 0 (x? ). Theorem 2: If initial function φy ∈ YI0 and φω ∈ Yg−1 (I0 ) , then limt→∞ (y(t), ω(t); φy , φω ) = (y ? , g −1 (y ? )). Proof: Before proving the theorem, we introduce the following lemma. Lemma 3: Fix k = 0, 1, . . .. Let L = (a, b) be any open interval that satisfies F (Ik ) ⊂ L ⊂ Ik . Suppose that y(s) ∈ Ik and ω(s) ∈ g −1 (Ik ) for all s ∈ [−T, 0]. Then, there exists finite time t? = t? (L) such that y(t) ∈ L and ω(t) ∈ g −1 (L) for all t ≥ t? . Proof: We first show that there exists some t0 such that y(t) ∈ L for all t ≥ t0 . Let L0 = (a0 , b0 ) be an open interval 0 containing F (Ik ) and whose closure L is contained in L, 0 i.e., L ⊂ L. We prove that there exists finite time t0 such 0 0 that y(t0 ) ∈ L . Suppose this is not true, i.e., y(t) 6∈ L for 0 all t ≥ 0. First, assume that y(t) ≥ sup L for all t ≥ 0. d y(t) ≤ − This implies that there exists  > 0 such that dt 0 −1 because g (y(t − T ))p(ω(t − T )) ≤ sup F (Ik ) < sup L , and hence y(t) ↓ −∞. This contradicts the assumption that 0 y(t) ≥ sup L for all t ≥ 0, and there exists finite time t0 such 0 0 that y(t0 ) ∈ L . The case where y(t) ≤ inf L for all t ≥ 0 can be handled similarly. Now, following the same argument 0 in the proof of Lemma 2 one can show that y(t) ∈ L for all 0 t ≥ t0 . Since L ⊂ L, clearly y(t) ∈ L for all t ≥ t0 . Thus, there exists some finite t0 such that y(t) ∈ L for all t ≥ t0 . We now show that there exists finite t1 ≥ t0 such that for all t ≥ t1 , ω(t) ∈ g −1 (L). Let L? = (a? , b? ) be an open interval ? such that a < a? < a0 < b0 < b? < b and L its closure. 0 From above we know that y(t) ∈ L for all t ≥ t0 . Following the same argument above we can show that there exists some ? finite t1 ≥ t0 such that ω(t1 ) ∈ g −1 (L ) and, by applying ? Lemma 1, it follows that ω(t) ∈ g −1 (L ) for all t ≥ t1 . Since ? g −1 (L ) ⊂ g −1 (L), this implies ω(t) ∈ L for all t ≥ t1 . Thus, there exists some finite t1 such that ω(t) ∈ g −1 (L) for all t ≥ t1 . Now, letting t? = t1 , the lemma follows. By repeatedly applying Theorem 1 and Lemma 3 we can construct an increasing sequence {tk , k = 1, 2, . . .} such that, for all t ≥ tk , y(t) ∈ Ik . Then, the theorem follows directly ? from the assumption ∩∞ k=1 Ik = {y }.

A comparison of Theorem 2 with Theorem 2 in [12] with no averaging shows that the addition of a low pass filter to the control loop does not tighten its stability criterion, and the stability of the map F is a still sufficient condition. We have also shown in [12] that in the case of no filtering, when the map F is unstable, the delay differential system loses its stability for sufficiently large delays. In Section III we show using a numerical example that even with filtering, when the map is not stable, the system (5) is unstable when the delay T is sufficiently large. D. Multiple stage filtering In this subsection we extend our results in the previous subsection to the case where more than one low pass filter is added to the system. An example is Random Early Detection [2] the packet drop probability of which depends on the averaged queue size. We show that a similar stability condition suffices even with multiple stages of filtering. Suppose that there are N , N ≥ 1, stages of low pass filtering. This scenario is modeled by the following set of differential equations: d x(t) = κ (x(t)U 0 (x(t)) − x(t − T )p(ωN (t − T ))) , dt d α1 ω1 (t) = −ω1 (t) + x(t) , (8) dt d αi ωi (t) = −ωi (t) + ωi−1 (t) , i = 2, . . . , N dt where ωi (t), i = 1, . . . , N , is the output of the i-th stage low pass filter, and αi are positive constants. Corollary 1: If initial function φy ∈ YI0 and φω i ∈ Yg−1 (I0 ) for all i = 1, . . . , N , then limt→∞ (y(t), ω1 (t), . . . , ωN (t); φy , φω1 , . . . , φωN ) = (y ? , g −1 (y ? ), . . . , g −1 (y ? )). Corollary 1 states that the introduction of low pass filters in the rate control scheme does not tighten the stability condition of the system when considering an arbitrary network delay. III. N UMERICAL E XAMPLE In this section we present numerical examples using a family of popular utility and price functions. We take a flow with utility function U (x) = − a1 x1a with a = 2.0 and price function p(x) = (x/C)b with b = 0.7 and the link capacity C = 5. In particular, this type of utility function with a = 1 has been found useful for modeling the utility function of Transmission Control Protocol (TCP) algorithms [9]. In addition, it is shown [12] that the price elasticity of demand of the flow decreases with increasing a. We show in [12] that with these utility and price functions, the map F is stable if and only if a > 1 + b with IR+ as the region of attraction. Thus, Theorem 2 tells us that for the selected values of a and b the user rate will converge to the equilibrium rate, starting from any positive, continuous initial function since 0.7 + 1 < 2.0. The equilibrium user rate is x? = C b/(1+a+b) = 1.3559. The reverse delay T from the resource to the sender is set to 100 in the simulation, and κ = 1. Low pass filter parameter α−1 is set to 0.05. Initial

4

functions are set to x(s) = 3 and ω(s) = 3 for all s ∈ [100, 0]. Fig. 1(a) shows the evolution of the instantaneous rate x(t) and average rate ω(t). Clearly, the system exhibits stable behavior, and both the instantaneous and averaged rates approach the equilibrium value x? = 1.3559 as t goes to ∞. Plot of x(t) and ω(t) (a = 2.0, b = 0.7)

R EFERENCES

4.5 x(t) ω(t)

4 3.5

Rate

3 2.5 2 1.5 1 0.5 0 0

500

1000

1500 Time (t)

2000

2500

3000

(a) Plot of x(t) and ω(t) (a = 1.4, b = 0.7) 8 x(t) ω(t)

7 6

Rate

5 4 3 2 1 0 0

500

1000

1500 Time (t)

2000

2500

interaction of underlying utility and price functions, and in the presence of one of more stages of stable low pass filters in the feedback loop, the same (sufficient) stability conditions derived with no filtering suffice. In other words, the addition of the filters does not require more restrictive stability conditions.

3000

(b) Fig. 1. Plot of rate x(t) and filtered load ω(t). (a) a = 2.0 and b = 0.7, (b) . a = 1.4 and b = 0.7

In Fig. 1(b) we also show the rate x(t) with a = 1.4, which violates our stability condition (1.4 < 1 + 0.7). In this case, the system is unstable and the user rate shows oscillatory behavior. These numerical examples demonstrate that providing feedback based on low pass filtered version of the user rate in the control loop does not affect the stability condition of the system with an arbitrary delay as long as system with feedback based on instantaneous rate is stable. IV. C ONCLUSION We studied the global stability issue of a rate control scheme where the feedback signal is based on an average rate of the user as opposed to instantaneous value. We showed that the dynamical stability of the system is determined by the

[1] S. Athuraliya, V. H. Li, S. H. Low, and Q. Yin. Rem: Active queue management. IEEE Network, 15(3):48–53, May/June 2001. [2] S. Floyd and V. Jacobson. Random early detection gateways for congestion avoidance. IEEE Trans. on Networking, 1(7):397–413, 1993. [3] Jack K. Hale and A. F. Ivanov. On a high order differential delay equation. Journal of Mathematical Analysis and Applications, 173:505– 514, 1993. [4] C. V. Hollot, V. Misra, D. Towsley, and W. Gong. On designing improved controllers for aqm routers supporting TCP flows. In Proc. of IEEE INFOCOM, Anchorage AK, 2001. [5] A. F. Ivanov, E. Liz, and S. I. Trofimchuk. Global stability of a class of scalar nonlinear delay differential equations. Accepted for Publication, Preprint from Author, 2003. [6] A. F. Ivanov, M. A. Pinto, and S. I. Trofimchuk. Global behavior in nonlinear systems with delayed feedback. Proc. Conference of Decison and Control, Sydney, 2000. [7] F. Kelly. Charging and rate control for elastic traffic. European Transactions on Telecommunications, 8(1):33–7, January 1997. [8] F. Kelly, A. Maulloo, and D. Tan. Rate control for communication networks: shadow prices, proportional fairness and stability. Journal of the Operational Research Society, 49(3):237–252, March 1998. [9] S. Kunniyur and R. Srikant. End-to-end congestion control schemes: Utility functions, Random losses and ECN marks. In Proc. of IEEE INFOCOM, Tel-Aviv, Israel, 2000. [10] S. Kunniyur and R. Srikant. Analysis and design of an adaptive virtual queue algorithm for active queue management. In Proc. of ACM SIGCOMM, 2001. [11] J. Mallet-Paret and R. D. Nussbaum. Global continuation and asymptotic behaviour for periodic solutions of a differential-delay equation. Annali di Matematica Pura ed Applicata, 145(4):33–128, 1986. [12] Priya Ranjan, Richard J. La, and Eyad H. Abed. Global stability conditions for rate control with arbitrary communication delays. accepted for publication in IEEE/ACM Trans. on Networking, 2005. [13] Glenn Vinnicombe. On the stability of networks operating tcp-like congestion control. Proc. IFAC, 2002.