A CONTROL FRAMEWORK FOR ONLINE ERROR ...

A CONTROL FRAMEWORK FOR ONLINE ERROR CONTROL ADAPTATION IN NETWORKED APPLICATIONS Oscar Fl¨ ardh ∗,1 Carlo Fischione ∗,1 Karl H. Johansson ∗,1 Mikael Johansson ∗,1

Automatic Control, School of Electrical Engineering, kth, SE-100 44, Stockholm, Sweden {oscar.flardh | carlofi | kallej | mikael.johansson}@ee.kth.se ∗

Abstract: For many real-time applications running over packet-switched networks, it is important to maintain delivered data quality using a limited amount of network resources. It is therefore natural to employ cost functions that allow online trade-off between the experienced application quality and the resource usage. However, minimizing such cost functions requires perfect knowledge of the network state at the transmission side, while, in general, such information is only partially available. In this paper, we introduce a new adaptive error correction algorithm that optimizes the amount of redundancy based on the available information from the application and the network. An extremum-seeking control algorithm is employed to deal with the high level of uncertainty in the network models. The validity of our approach is illustrated in simulations with varying network loads and loss correlation. Keywords: Error control, adaptive redundancy, extremum seeking control

1. INTRODUCTION There are several different types of real time applications running over networks, including multimedia streaming, control applications and industrial automation. The need to communicate information over unreliable networks, which introduce delays and information loss, imposes difficulties for these applications. For real time applications, reliability cannot be achieved by retransmissions, as is done in tcp, and other methods are needed. One approach is to modify each application to handle the effects of network communication. 1 This work is partially supported by the European Commission via the Integrated Project RUNES and the Network of Excellence HYCON, by the Swedish Research Council, and by the Swedish Foundation for Strategic Research through an Individual Grant for the Advancement of Research Leaders

Within the area of estimation and control there has been much work recently in this direction, resulting in several novel control and estimation algorithms with performance guarantees in presence of delays and packet loss (Sinopoli et al., 2005), (Ling and Lemmon, 2004). Another way to cope with network imperfections is developing interfaces between the network and the application, and thereby handling the uncertainty introduced by the network when data is lost or delayed in the communication. This can be done by introducing coding, increasing the probability that the message is transmitted without error. This will, in turn, increase the reliability or quality of the application. If reduced quality is associated with a certain cost, i.e. the higher the quality reduction the higher the cost, this cost will decrease when redundancy is increased.

From this point of view, the redundancy should u PSfrag replacements Sender Receiver be high to have a low cost. In contrast, increasing Network redundancy will increase the network load, due p,d to the extra redundant packets that are sent. From this perspective, the redundancy should be Fig. 1. The receiver sends feedback information low to reduce the cost of network load. Putting about the quality of the received message. these two effects together, there will be a tradeoff This is used by the sender to adapt the between quality and performance when adjusting amount of redundancy used in communicathe redundancy. This raises the need to adapt tion. the amount of redundancy to the current network error correction process, there will be a different conditions. The goal of adaptive schemes should amount of distortion d of the original message. then be to find this optimal tradeoff given the This distortion is affected by the network loss procurrent application demands and network state. cess p and the amount of redundancy used. The Previous work includes both analysis of coding control objective is to keep the distortion small with different redundancy amounts (Barakat and without using excessive redundancy. To capture Fawal, 2004), (Lundqvist, 2005) and algorithms this tradeoff a cost function c is introduced that for adapting the coding (Lundqvist, 2005), (Park takes the redundancy u and the distortion d as and Wang, 1997). Most adaptation schemes, howinput, i.e. ever, relies on a model of the network loss process c = h(d, u) to determine the redundancy amount. Based on The cost function should penalize both high rethis model, which includes parameters such as dundancy, which decreases the throughput, and the packet loss probability, the optimal amount high distortion. For small u, the distortion d is of redundancy can be found given some objective likely to be high and hence the cost is high. For function. The drawback with this model reliance is high u, the distortion will be low but the cost will that the network loss process can never be exactly be high since u is high. The best tradeoff will then known, possibly causing suboptimal performance. be the one that minimizes the expected cost. The The purpose of this paper is, therefore, to present coding scheme and the cost function is further a new approach to error control that aims to described in section 4. provide good adaptation without relying on a The problem considered in this paper is finding a loss model. This extends the work in (Fl¨ardh et controller that uses both information on the netal., 2005), where it was shown that by considering work loss process and the distortion after decoding the outcome of the error correction process, it is to adapt the redundancy amount; see Figure 1. possible to adapt the redundancy amount even in The sender transmits coded messages over the presence of model errors. The main contribution network. These messages will be received with compared to (Fl¨ardh et al., 2005) is the introducsome distortion, which can be communicated back tion of the cost function, the hybrid control law to the sender. The receiver may also estimate the that switches between feedforward and feedback loss process p and provide the sender with that control actions and the consideration of systeminformation. Based on the information about d atic coding. To illustrate our approach, we present and p, the sender adjusts the redundancy u via an error controller that combines the extremum seeking feedback mechanism presented in (Fl¨ardh u = f (d, p) et al., 2005), tracking the optimum of the cost Thus, the function f represents the control algofunction, with a change detection filter detecting rithm used by the sender, while the information p sudden changes in network conditions. Simulaand d is produced by the receiver. In this paper tions show that this approach allows for efficient we only consider the choice of f , the receiver (low distortion with low redundancy) communialgorithms to get d and p will not be discussed. cation over a wide range of network conditions. The feedback mechanism provides a robust way of finding the optimum without model reliance, 3. CONTROL ARCHITECTURE while the feedforward change detection filter improves the transient behavior. Even though both p and d in Figure 1 are sent back from the receiver to the sender, they represent different types of information from a control 2. PROBLEM FORMULATION point of view. The system to be controlled is the error correction process, with redundancy u as input and the cost c as output. While u has a direct We consider error correction on packet-level where influence on d, it has little or no influence on the N packets are collected into a block, and each loss process. Thus, the signal p can be considered block b consist of ub redundancy packets and N − as a disturbance acting on the system, and the ub data packets. Depending on the outcome of the

cost can be seen as a function of u parameterized ˜ p). by p, i.e., c = h(u; Since the sender knows u, it only needs information on d to find c. Hence basing the control signal on the information given by p results in a feedforward controller, while basing the control signal on d results in a feedback controller. Combining this two components, the error controller uses the feedforward information for fast transient behavior and the feedback information to get good performance during steady state.

find the current value of the cost function, why in this case u = f (d) Since the sender knows u, it can calculate c given d. The purpose of the controller is to find the optimum of the function c, which is unknown since no loss model is assumed. To find the optimum, we utilize an extremum-seeking controller. Such a controller aims to maximize the output of a system, without the need for a model. The proposed controller for this application is given by ub+1 = ub − β sgn(udb cdb )

3.1 Feedforward Control Basing the control signal only on the current estimated packet loss probability gives that u = f (p) The estimation is made as the average loss probability over a time window w2 . Based on the this estimate, the applied control signal is given by an online table look-up from the optimal redundancy. The optimal redundancy can be found offline given an assumption of the loss process. In this case, independent losses are assumed. This strategy is combined with a change detection filter, to detect sudden changes in packet loss probability. The filter is a sequential probability ratio test (sprt) and works in the following way (Gustafsson, 2001). At each time step (i.e. for each packet) a residual is calculated as εt = pt − pˆt−1 , where pt is the estimated packet loss probability over a (shorter) time window w1 , and pˆt−1 is the packet loss probability over a longer window w2 . Then the filter state gt is updated according to gt = gt−1 + εt − ν The parameter ν is used to compensate for slow drifts. If gt exceeds a certain threshold, there will be an alarm and a new packet loss probability is estimated. The internal state of the estimator is also reset, including both windows. Using only feedforward information p to determine the control u has two major drawbacks, the need for accurate estimation of the packet loss probability and the reliance on a loss model. If the estimate or the model is incorrect the applied redundancy will not be optimal, resulting in suboptimal performance.

(1) udb

where β is the control parameter and and cdb denote derivative estimates at block b of the signals u and c. This controller is inspired by the switching controllers (Sternby, 1979) and aims to drive the system towards the optimum. The basic principle is to estimate on which side of the peak the system is, and then move towards that peak. If the relation between c and u is given by a unimodal static function, the system will always converge to the optimum. A more thorough analysis can be found in (Fl¨ardh et al., 2005). 3.3 Error Controller The error controller uses the feedback algorithm during steady state, but also utilizes the change detection filter that the feedforward controller uses. Since both information on the network and the distortion determines the control signal, u = f (p, d) If a sudden change in packet loss probability is detected, the redundancy is reset to what is optimal given the new estimate. The optimality here is with respect to independent losses. The extremum seeking algorithm 1 then continue seeking for the optimal steady state value. By combining these ideas it is possible to achieve both fast responses during transients and stabilizing on the optimal value during steady state. The control algorithm will then be ( u∗ (pb ) if change detected ub+1 = ub − β sgn(udb cdb ) otherwise (2) where u∗ (pb ) is the optimal redundancy amount for packet loss probability pb , when the losses are assumed to be independent. 4. PERFORMANCE ANALYSIS

3.2 Feedback Control To address the drawbacks with the feedforward controller, a feedback control algorithm is studied. It uses feedback information on the distortion to

To illustrate the behavior and performance of the controllers, we consider an error correction example. For a general coding setup, we present a simple cost function and evaluate the characteristics of the different control architectures.

1

2

3

.....

number of recovered data packets. The distortion will then be expressed as

N−u

Coding

d = (N − u) − z

Block 1

2

3

.....

N−u

.....

N−1

N

.....

N−1

N

Network

1

2

3

.....

N−u

≥ N−u ? Decoding

PSfrag replacements 1

2

3

.....

N−u

Fig. 2. Error correction. If at most u out of N sent packets in a block are lost, then it is possible to recover the original N − u data packets of the block.

Note that this is always a positive number since its not possible to recover more data packets than has been sent. This distortion measure captures the amount of lost data after the error correction, and thereby the rate of distortion the application on the receiving side experiences. Based on that increasing u decreases the throughput linearly, and that a large d is not desirable, the following cost function will be studied  2 d(u) u c(u) = (4) +ρ N N where ρ is a positive constant.

The objective is then to minimize the expected value of c. If the packet loss process p is known, the cost function can be found as a function of the redundancy u and p.

4.1 Coding To protect the data sent over a network from losses, error correcting codes can be used. The considered setup here is when packets are collected into a block of fixed size N . In each block b, there are ub redundancy packets and N − ub packets containing application data, see figure 2. The minimum distance for such a setup is ub + 1, so if the code is maximum distance separable, for example Reed Solomon codes, up to ub packet losses can be corrected for (Blahut, 1983). In other words, if at least any N − ub out of the original N packets are received, all the original information can be recovered. 4.1.1. Systematic Coding When systematic coding is used, all the original data is not encoded but sent in the N − u data packets. Hence some information is transmitted even if more than u packets are lost. This amount will then depend on how many of the original data packets are received. Let z be the number of of recovered data packets per block and X1 ∈ [0, N −u], X2 ∈ [0, u] be stochastic variables denoting the number of received data and redundancy packets respectively. Then z can be expressed as ( N − u if X1 + X2 ≥ N − u z= (3) X1 if X1 + X2 < N − u

4.2 Cost function In this example we use a simple, non-application dependent distortion measure as the difference between the number of sent data packets and the

4.3 Simulations To investigate the performance of the three controllers described in section 3, two simulation scenarios are studied. In both cases, the packet loss probability increases from 3% to 8%. In the first simulation, the losses are independent while in the second the losses are generated by the Gilbert model. These simulations aim to illustrate the different properties of the control algorithms. The step changes in packet loss probability illustrates both the transient and steady state performance, while the different loss models used reveals the robustness to model errors. For the simulation with independent losses, the feedforward controller naturally performs best. The transient behavior is fast and since there are no model errors, also the steady state performance is good. The feedback controller is much slower in the transients but finds the optimal value during steady state. The oscillations, though, give a slight decrease in performance. The error controller performs well during the transients, but the steady state oscillations causes the performance to be a little worse than for the feedforward. For the simulation with the Gilbert loss model, the cumulative cost is shown in the table below. The error controller is compared with the feedforward and feedback control algorithms, and also with a controller using the optimal redundancy amount. From Table 1 it is seen that the error controller performs best followed by the feedback controller, while the feedforward controller now performs worst. The redundancy amount for the different control algorithms are shown in Figures 3 and 4. The error and feedback controllers still

250

24

Feedforward control Feedback control Error control Optimal control

Feedforward control Optimal control Feedback control

22

200

20 18

150

16

u

Cumulative cost

b

14

100

12 10 50

8 6 0

4

0

1

2

3

4

5

Block no

6

7

8

9

10

1

2

3

4

5

Block no

6

7

8

9

10 4

x 10

x 10

Fig. 3. Control signal during a simulation with Gilbert losses. 25

Fig. 5. Cost function during a simulation with Gilbert losses. very well during steady state and diverges only marginally from the optimal strategy.

REFERENCES

20

15

u

b

10

5

0

0

4

Optimal control Error control 0

1

2

3

4

5

Block no

6

7

8

9

10 4

x 10

Fig. 4. Control signal during a simulation with Gilbert losses. Table 1. The combination of fast transient behavior and small steady state error results in very good overall performance for the error controller. Controller Optimal control Error control Feedback control Feedforward control

Cumulative Cost 190 198 202 245

come close to the optimal steady state value, but the oscillations are larger. The feedforward controller, assuming independent losses, results in too low redundancy and incurs a much higher cost; see the cumulative cost plot in Figure 5. The rate of divergence is particularly high in the second part, where the network performance is worse. The feedback controller again lacks in the transient, but together with the error controller it performs

Barakat, Chadi and Alaeddine Al Fawal (2004). Analysis of link-level hybrid fec/arq-sr for wireless links and long-lived tcp traffic. Performance Evaluation Journal 57(4), 43–500. Blahut, Richard E. (1983). Theory and Practice of Error Control Codes. Addison-Wesley. Fl¨ardh, Oscar, Karl H Johansson and Mikael Johansson (2005). A new feedback control mechanism for error correction in packetswitched networks. In: 44th IEEE CDC ECC. Gustafsson, Fredrik (2001). Adaptive Filtering and Change Detection. John Wiley & Sons. Ling, Q. and M.D. Lemmon (2004). Power spectral analysis of networked control systems with data dropouts. IEEE Transactions on Automatic Control 49(6), 955–959. Lundqvist, Henrik (2005). Error and Traffic Control for High-Speed Networks. PhD thesis. Royal Institute of Technology (KTH). Park, K. and W. Wang (1997). AFEC: an adaptive forward error-correction protocol and its analysis. Technical report. Technical Report CSD-TR97-038, Department of Computer Sciences, Purdue University. Sinopoli, B., L. Schenato, M. Franceschetti, K. Poolla and S. Sastry (2005). An lqg optimal linear controller for control systems with packet losses. In: 44th IEEE Conference on Decision and Control. Sternby, Jan (1979). A review of extremum control. Technical report. Department of Automatic Control, Lund Institute of Technology.