Redesigning an active queue management system - CiteSeerX

Redesigning an Active Queue Management System Deepak Agrawal, Fabrizio Granelli Dept. of Information and Communication Technology – University of Trento Via Sommarive, 14 – 38050 Trento, ITALY Email: [deepak,granelli]@dit.unitn.it Abstract– In this paper a robust Proportional-Integral-Derivative (PID) controller is proposed for Active Queue Management (AQM). Linear Quadratic Regulator (LQR) method is used to design controller, therefore named LQR-PID. LQR is a robust design technique as compared to classical Gain & Phase Margin and Dominant Pole Placement methods. The LQR-PID controller marks the packets according to queue length with a probability and notifies congestion to sources; in turn sources adjust their send rate, thus maintaining queue length at desired level in bottleneck routers. By maintaining queue length at desired level, delay can be predicted and quality of service can be provided. Simulation results demonstrate the robustness and superiority of LQR-PID AQM as compared with other AQM schemes in the literature. Keywords– Active Queue Management, Congestion Proportional-Integral-Differential Controller, TCP, LQR

control,

I. INTRODUCTION Active Queue Management (AQM) mechanism is implemented at routers to improve the end-to-end congestion control and to provide low delay and low loss in best-effort networks by active congestion notification in advance [1]. Random Early Detection (RED) [13] is the best known AQM scheme, which provides fairness among sources and also controls queue length. A detailed performance analysis of RED in real networks [7] concludes that RED is quite sensitive to parameter settings. Parameter setting details are illustrated in “gentle RED” [14], ARED [15], SRED [9] etc. For the purpose of improving the fairness of RED, BRED [5], FRED [6], REDPD [12] and CHOKe [10] are also studied. All these methods for changing RED configurations have their own limitations and advantages. The design of an AQM mechanism has also been considered as an optimization standpoint [4]. Control theory application for AQM design is an alternative approach. Linearization of TCP, modeling of TCP & RED as a control system, Proportional & PI controller for AQM by calculating packet drop probability based on the current queue length, has been discussed in [2, 3]. The results of various simulations show that PI outperforms RED in regulating steady state queue length to a desired reference value with changing levels of congestion. In both cases, time taken to achieve stable queue length is quite long. Also at steady state the fluctuations are high. Sliding mode variable structure control (SMVS) [11] is another recently proposed AQM controller, which applies a

IEEE Communications Society Globecom 2004

702

sliding mode variable structure control and shares PI’s goal of maintaining a stable average queue length. PD-RED [17] is compared with adaptive RED [15] and found to be better in most of the cases, even if variance of the queue length at steady state is large. In this paper we have implemented a combined PID controller to fulfill the objective of AQM i.e. responsiveness and minimum steady state error means stable queue length. ‘D’ improves the stability and responsiveness of the system and reduces overshoot of queue length. The rest of paper is organized as follows. In section II we introduce the linear control system model and describe the LQRPID algorithm with its mathematical formulation. The performance of LQR-PID and its comparison with PD and PI is discussed in section III. Finally in section IV, conclusions are drawn. II. TCP MODEL AND LQR-PID ALGORITHM II.A TCP Control System Model TCP source sends packets and TCP receiver replies with acknowledgement (ACK) as feedback for successful packet delivery. Packets are stored in buffer at routers, when the total sending rate of the TCP sources is greater than the capacity of the link between the router and the TCP receivers. If the arrival rate into the buffer is higher than its service rate, the buffer may overflow and some packets may be dropped. In order to achieve high utilization and to meet the required QoS, the AQM controller should stabilize the buffer occupancy of the router at a given target level such that link underutilization and buffer overflow are significantly reduced and a predictable QoS can be provided (e.g., delays with upper bounds). To this aim, we consider the TCP congestion control mechanism as a feedback control system, where controlled variable is the instantaneous queue length and the target is level of buffer occupancy. Nonlinear ordinary differential equations, which describe the transient behavior of networks supporting TCP flows, are developed in [16]. These equations are linearized in [2] and the linear TCP/AQM system model is shown in Figure 1, where, QT is the target queue size, GC(s) is the AQM controller, and GP(s) is the TCP window-control mechanism and queue dynamics. The transfer function of plant GP(s) is

0-7803-8794-5/04/$20.00 © 2004 IEEE

C 2 −sR0 ( )*e 2N GP(s) = 2N 1 (s + 2 )(s + ) R0 R0 C

AQM controller marks packets with a probability. The marking probability is calculated according to the LQR-PID controller and it is a function of the difference between the instantaneous queue length and the reference queue length. Hence, the cooperation of AQM and TCP can be viewed as a discrete-time control system. Let ’T’ be the sampling time interval (inverse of sampling frequency ‘w’), ‘i’ the time index, q (i ) the instantaneous queue length at time ‘i’, and QT the target buffer occupancy. Our aim is to determine marking probability p(i) such that it will maintain the target queue length independent of the traffic load and disturbance such as non-responsive UDP flows and varying no. of TCP sources. In order to achieve this objective, we propose to adapt p(i ) to the actual queue dynamics using a simple control mechanism. At time ‘i’, if the actual queue length q (i ) is smaller

(1)

where R0 = Round Trip Time (RTT), N = no. of active TCP sessions, C = link capacity (packets/sec)

QT

p(i)

e(i)

+

GC

q(i)

GP

Fig.1. Linearized TCP/AQM control system.

II.B LQR-PID Controller P-and-I controller operates on the past errors and does not attempt to predict future errors, a characteristic that limits the performance of the PI controller. This shortcoming is overcome by using ‘D’ controller, which predicts future error and takes corrective measures, resulting in faster response. In fig. 1 ‘GC’ represents the LQR-PID controller. PID is a widely used industrial control despite availability of new and advanced controllers. Transfer function of LQR-PID controller is given by K (2) GC ( s ) = K p + i + K d s s where Kp, Ki and Kd are Proportional, Integral and Derivative gains, respectively. Most important part in designing LQR-PID controller is to determine its parameters, i.e. determining gains Kp, Ki, and Kd for optimum performance, because of interdependency of controllers when used together. There are numerous methods described in control theory e.g. Dominant pole placement, Gain margin & Phase margin, Linear Quadratic Regulators (LQR) etc. We have used robust LQR [18] design method for determining controller parameters. LQR system provides at least a phase margin of 60 degree and a gain margin of infinity [18]. For determination of controller gains, we have considered LQR parameter ωn*R0 = 1.3 & ξ = 0.75, where ωn is natural frequency and ξ is damping ratio. Other parameters: round trip time R0 = 0.246 sec, No. of active TCP connections N = 60, link capacity C = 15Mbps and sampling frequency w = 160Hz, are set same as in [2, 3]. The design procedure is not illustrated here because of space limitation.

II.C. Control Action The objective is to maintain the queue length very close to the target value by active notification of congestion to sources. The


703

than the target buffer occupancy level QT, p(i ) is decreased to increase the buffer occupancy, and if the actual queue length is higher than QT, p(i ) is increased to decrease the buffer occupancy. Objective of controller [3] is to adapt p(i ) so that amplitude of error signal e(i ) is kept as small as possible. Where e(i ) is given by e(i ) = q(i ) − QT (3) A discrete form of LQR-PID controller of Eq. (3) is described as Kd ) * e (i ) − T (4) 2 * Kd Kd (K p + ) * e(i − 1) + * e (i − 2 ) T T For maintaining 0 ≤ p(i ) ≤ 1 we set p(i ) = 0 if p(i ) < 0 and p (i ) = p (i − 1) + ( K p + T * K i +

p(i ) =1 if p(i ) >1. An additional parameter ‘L’ is used for maintaining high link utilization and also not to penalize sources which are in the process of backing off in response to previous packet drops. When q (i ) < L no packet is dropped. There is always a time lag between the packet drop and source response to the packet drop. Even if there is no packet drop i.e. q (i ) < L, computation of probability continues, thus avoiding the need to initialize the process. The marking probability is updated at fixed interval of time as per the criteria described in [3], but performance of controller is poor with low value of sampling frequency w=3-6Hz, therefore in our simulations also high sampling frequency, 160Hz, is used. The algorithm of LQR-PID control process can be summarized as follows: • Sampling of the queue length q (i ) . • Computation of error signal by (3). • Computation of mark/drop probability p(i) by (4). • Use of p(i ) until new value ‘p’ is calculated at time i+1.

0-7803-8794-5/04/$20.00 © 2004 IEEE

• Store e(i ) , e(i − 1) and p(i ) to be used at time ‘i+1’. Following parameters are important for LQR-PID control system: • Sampling frequency/time interval ‘w/T’ – it sets the sampling rate for controller. Its value is 160/0.00625s. • Target queue level ‘QT’ – this parameter decides the steadystate queue length, which will affect the utilization and the average queuing delay. A high target will improve link utilization, but will increase the queuing delay while low target leads to underutilization of resources. • Proportional gain ‘Kp’ – this parameter affects the reaction and stability of the control system. A large proportional gain will speed up system reaction but may cause system instability. Its value is Kp = 3.482*10-10. • Derivative gain ‘Kd’ – This parameter predicts the future error and affects the reaction and stability of the control system also, but not so much as the proportional gain Kp. The value used is Kd =8.695*10-11. • Integral gain ‘Ki’ – this parameter affects fluctuations of queue level at steady state condition. The value used is Ki = 1.563*10-12. • No drop threshold ‘L’ – An extra parameter with value slightly less than the target queue size, below which no packets are dropped. It also limits fluctuation in steady state conditions. L = 0.9 * QT.

ns-2.27. The TCP connections are modeled as always busy FTP connections, transmission is made whenever the congestion window permits. Large receiver’s advertised window size is assumed, so that transmission is not limited by the receiver. Every packet is acknowledged individually by the receivers. All sources send data from 0.0sec to 100.0secs. Parameters for PI are: a = 1.822*10-5, b = 1.816*10-5, sampling frequency = 160Hz and for PD; kp = 0.0002, kd = 0.08, sampling interval = 0.01sec. Figs. 3, 4 and 5 show the instantaneous queue length q (i ) of LQR-PID, PI and PD controller respectively for 100 TCP connections. It is clear from the graph that LQR-PID response is fast and queue is more stable, Moreover overshoot is negligible in case of LQR-PID. We can see that LQR-PID-Controller is effective at stabilizing and keeping the queue length around target QT. In order to adequately show ability of LQR-PIDController, we achieve stable queue length, thus controlling queuing delay for meeting specified QoS requirements. In comparison with the presented scheme, the PI controller takes long time to settle and also occupies buffer for longer time, resulting in variable delay and jitter, while the PD controller queue evolution have larger fluctuations as compared to LQRPID in steady state condition. Moreover, initial overshoot, i.e. high buffer occupancy before steady state, is almost negligible for LQR-PID as compared to PD. 1200

III. SIMULATIONS AND DISCUSSION We have performed LQR-PID algorithm simulation in the ns-2 environment [8]. For simulations, a typical bottleneck configuration as shown in Fig 2 is used. In our simulation experiments, we test whether LQR-PID-Controller meets the AQM objective of stabilizing the queue length at an arbitrary chosen value for different loads and link capacities. All links connected between different nodes, except bottleneck links, have a capacity of 16Mb/s and delay of 10ms, while the bottleneck link is 45Mb/s and 10ms. All sources are TCP/Reno/FTP. Explicit Congestion Notification bit is set true.

1000 800 600 400 200 0 0

20

40

60

80

100

Fig.3. Queue length (packets) v/s Time (sec), LQR-PID controller. 1400 1200 1000 800 600

D1

S1

400 200 0 0

S2

B1

B2

D2

40

60

80

100

120

Fig.4. Queue length (packets) v/s Time (sec), PI controller.

Dn

Sn

20

1200 1000 800

Fig.2. Bottleneck topology used for simulations.

600 400

In the following simulations, parameters are set as follows: packet size = 500 bytes, round trip time = 100ms, buffer size = 1125 packets and target queue = 630 packets. The TCP tick timer is set to 300ms and other parameters are set at default values of


704

200 0 0

20

40

60

80

100

120

Fig.5. Queue length (packets) v/s Time (sec), PD controller.

0-7803-8794-5/04/$20.00 © 2004 IEEE

Figs. 6-7 show the marking probability of LQR-PID & PI control for 100 TCP connections, which is very low and nearly constant in case of LQR-PID, while it is higher for PI and even more (with fluctuations) for PD (not shown here). This verifies that the system is quite stable and all our previous conclusions apply. Figs. 8-9 show the evolution of instantaneous queue length with LQR-PID for different bottleneck links, i.e. 15Mb/s and 100Mb/s respectively. This also proves the effectiveness of LQR-PID under variable network conditions. The target queue level is reached quickly and there is small fluctuation in queue length at steady state. Effects of round trip time (RTT=1000ms) on queue evolution are shown in Fig. 10, underlining the robustness of the proposed method. 0.025 0.02 0.015 0.01 0.005 0 0

20

40

60

80

100

Fig.6. Marking probability vs. Time (sec), LQR-PID controller.

1200 1000 800 600 400 200 0 0

20

40

60

80

100

Fig.10. Queue length (packets) v/s Time (sec), RTT= 1000ms.

Effect of continuously changing load on evolution of queue is shown in Fig. 11. Initially there are 100 active TCP sources and 900 sources have start time uniformly distributed over 100sec. It is clear from the figure that LQR-PID effectively accommodates load and stabilizes the queue at target level. Fig. 12 shows the evolution of queue length for 100 TCP connections having variable round trip time. The RTT is uniformly distributed between 100 and 400ms. We have set a wide range of RTTs, from 100 (R100) to 400ms (R400), which include the crucial transition R100 ≤ R0 ≤ R400. The results indicate that LQR-PID is still effective at stabilizing the queue even when TCP connections have different RTTs. The results in Figs. 11 and 12 show the robustness of LQR-PID.

0.12 0.1

1200

0.08

1000

0.06

800

0.04

600

0.02

400

0 0

20

40

60

80

100

200

120

0 0

20

40

60

80

100

120

Fig.7. Marking probability vs. Time (sec), PI controller.

Fig.11. Queue length (packets) v/s time, 100 - 1000 TCP connections. 1200 1000

1200

800

1000

600

800

400

600

200

400

0 0

20

40

60

80

200

100

0 0

20

40

60

80

100

120

Fig.8. Queue length (packets) v/s Time (sec), C= 15Mb/sec.

Fig.12. Marking probability v/s Time (sec), 1000 TCP connections

1200 1000

For further justification of robustness of proposed method, the simulation results in Figure 13 & 14 show the ability of LQRPID controller to accommodate the disturbances caused by unresponsive UDP connections. We have considered 250 TCP connections and 250 UDP flows with uniform propagation delay. Each of the UDP sources sends 200bytes packets at a rate of 64kbps with an interval of 0.005secs. The bottleneck link is 45Mb/s. Figs. 13 and 14 provide the result of queue length and

800 600 400 200 0 0

20

40

60

80

100

Fig.9. Queue length (packets) v/s Time (sec), C= 100Mb/sec.


705

0-7803-8794-5/04/$20.00 © 2004 IEEE

platforms for high-performance computational grids oriented to scalable virtual organization (GRID.IT)”, (contract n. RBNE01KNFP)

marking probability. Queue stabilization is very fast as well as marking probability is very low with very small fluctuations. This confirms the effectiveness of LQR-PID controller under various network conditions and different traffic scenarios.

REFERENCES 1200

[1]

1000 800 600 400 200

[2]

0 0

20

40

60

80

100

[3]

Fig.13. Queue length v/s Time (sec) with TCP/FTP and UDP/CBR. 0.06

[4]

0.05 0.04 0.03

[5]

0.02

[6]

0.01 0 0

20

40

60

80

100

[7]

Fig.14. Marking probability of TCP/FTP and UDP/CBR v/s Time (sec).

IV. CONCLUSIONS Under varying network and load conditions, no available controller is suitable is suitable for achieving responsiveness and stability in steady state condition; therefore, the paper proposes to use a combination of different controllers in order to overcome the limitations. To this aim, a different AQM algorithm named LQR-PID has been designed and evaluated. LQR-PID combines the advantages of P, I, and D controllers with additional robustness in comparison to conventional methods. We have demonstrated with simulation results that a LQRPID-Controller is able to maintain the queue length around the given target under different conditions e.g. different bottleneck link capacities and different types of traffic. The simulation results have also demonstrated that LQR-PID Controller is also robust to non-responsive UDP connections. Comparison with other queue-based AQMs (PD & PI) has demonstrated the superiority of LQR-PID Controller in achieving faster convergence to target queue occupancy, and maintaining the queue length closest to the target with minimum fluctuations. These properties of LQR-PID Controller indicate that LQR-PID can achieve high throughput, predictable delay and low jitter.

[10]

[11]

[12]

[13] [14] [15] [16]

[17]

V. ACKNOWLEDGMENT

[18]

This work has been partially supported by Italian Ministry of Education and University (MIUR) under FIRB project “Enabling


[8] [9]

706

B. Braden, D. Clark, J. Crowcroft, B. Davie, S. Deering, D. Estrin, S. Floyd, V. Jacobson, G. Minshall, C. Partridge,L. Peterson, K. Ramakrishnan,S. Shenker, J.Wroclawski, & L. Zhang, “Recomedations on Queue Management and Congestion Avoidance in the Internet”, RFC2309, April 1998. C. Hollot, V. Misra, D. Towsley, and W. Gong, “A Control Theoretic Analysis of RED”, in Proceedings of IEEE INFOCOM’01, Anchorage, Alaska, USA, April 2001, pp1510-1519. C. Hollot, V. Misra, D. Towsley, and W. Gong, “On designing Improved Controllers for AQM Routers Supporting TCP Flows” in Proceedings of IEEE INFOCOM’01, Anchorage, Alaska, USA, April 2001, pp17261734. David Lapsley and Steven Low, “Random Early Marking for Internet Congestion Control”, in Proceedings of IEEE Globecom’99,Rio de Janeiro, Brazil, December 1999,pp1747-1752. F.Anjum and L.Tassiulas, Balanced-RED, “An Algorithm to Achieve Fairness in Internet”. http://www.isr.umd.edu/CSHCN/ Lin, D. and Morris, R., “Dynamics of random early detection”, Proceedings of ACM, SIGCOMM’97, Cannes, France, October 1997, pp127-137. M. May, J. Bolot, C. Diot, and B. Lyles,” Reasons Not to Deploy RED”, Proceedings of 7th. International Workshop on Quality of Service (IWQoS’99), London, UK, June 1999, pp260-262. ns-2 http://www.isi.edu/nsnam/ns/ Ottomon, T., Lakshman, T. and Wong, L., SRED, “Stabilized RED”, in Proceedings of INFOCOM’99, New York, NY, March 1999, pp13461355. Pan, R., Prabhakar, B. and Psounis, K., “CHOKe - A Stateless Active Queue Management Scheme For Approximating Fair Bandwidth Allocation”, Proceedings of INFOCOM’00, Tel-Aviv, Israel, April 2000, pp942-951. Ren Fengyuan, Lin Chuang, Ying Xunhe, Shan Xiuming, Wang Fubao, “A Robust Active Queue Management Algorithm Based on Sliding Mode Variable Structure Control”, in Proceedings of INFOCOM’02, New York, USA, June 2002, pp64-79. R. Mahajan, S. Floyd, D. Wetheral, “Controlling High-Bandwidth Flows at Congested Router”, in Proceedings of ACM 9th International Conference on Network Protocols, Mission Inn, Riverside, California, November 2001, pp192-202. S. Floyd, and V. Jacobson, “Random Early Detection Gateways for Congestion Avoidance”, IEEE/ACM Transactions on Networking, vol. 1 no. 4, August 1993, pp397-413. S. Floyd, “Recommendations on using the gentle variant of RED,” http://www.aciri.org/floyd/red/gentle.html. S. Floyd, R. Gummadi, and S. Shenker, “Adaptive RED: an algorithm for increasing the robustness of RED’s Active Queue Management,” http://www.icir.org/˜ floyd. Vishal Misra, Wei-Bo Gong, and Don Towsley, “Fluid-based Analysis of a Network of AQM Routers Supporting TCP Flows with an Application to RED”, in Proceedings of ACM SIGCOMM’00, Stockholm, Sweden, August 2000, pp151-160. J. Sun et al, “PD-RED: To improve the performance pf RED”, IEEE Communication letters, vol. 7 No. 8 August 2003. T.K. Kiong, W. Quing-Guo and H.C.Chieh,”Advances in PID Control”, Springer-Verlag London 1999.

0-7803-8794-5/04/$20.00 © 2004 IEEE