An engineering approach to dynamic prediction of network ...

INTERNATIONAL JOURNAL OF NETWORK MANAGEMENT Int. J. Network Mgmt 2005; 15: 151–162 Published online 28 February 2005 in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/nem.554

An engineering approach to dynamic prediction of network performance from application logs By Zalal Uddin Mohammad Abusina*,†, Salahuddin Muhammad Salim Zabir, Ahmed Ashir, Debasish Chakraborty, Takuo Suganuma and Norio Shiratori Network measurement traces contain information regarding network behavior over the period of observation. Research carried out from different contexts shows predictions of network behavior can be made depending on network past history. Existing works on network performance prediction use a complicated stochastic modeling approach that extrapolates past data to yield a rough estimate of long-term future network performance. However, prediction of network performance in the immediate future is still an unresolved problem. In this paper, we address network performance prediction as an engineering problem. The main contribution of this paper is to predict network performance dynamically for the immediate future. Our proposal also considers the practical implication of prediction. Therefore, instead of following the conventional approach to predict one single value, we predict a range within which network performance may lie. This range is bounded by our two newly proposed indices, namely, Optimistic Network Performance Index (ONPI) and Robust Network Performance Index (RNPI). Experiments carried out using one-year-long traffic traces between several pairs of real-life networks validate the usefulness of our model. Copyright © 2005 John Wiley & Sons, Ltd.

Zalal Uddin Mohammad Abusina is with the National Institute of Information and Communications Technology (NICT), Japan. He works for Japan Gigabit Network-II (JGN-II) Project at NICT’s Tohoku University Office housed in its Research Institute of Electrical Communications (RIEC). Salahuddin Muhammad Salim Zabir joined the Department of Computer Science and Engineering of Bangladesh University of Engineering and Technology in 1995. At present, he is with RIEC, Tohoku University. He is a member of the IEEE, BCS and BAAS. Ahmed Ashir received his PhD in 1999 from Tohoku University, Japan. He was with Japan Gigabit Network (JGN) Project of the Telecommunication Advancement Organization (TAO), Tohoku University Office. Debasish Chakraborty received his PhD in 1999 from Tohoku University, Japan. He is currently with Research Institute of Electrical Communications, Tohoku University, Sendai, Japan. Takuo Suganuma is currently with Research Institute of Electrical Communications (RIEC), Tohoku University, Sendai, Japan. He is a member of the IEEE. Norio Shiratori is a professor at the Research Institute of Electrical Communication (RIEC), Tohoku University. He has been engaged in research on distributed processing systems and flexible intelligent networks. He is a Fellow of the IEEE, IEICE, IPSJ. *Correspondence to: Zalal Uddin Mohammad Abusina, Research Institute of Electrical Communication, Tohoku University 2-1-1, Katahira Aoba-ku, Sendai 980-8577, Japan. † E-mail: [email protected]

Copyright © 2005 John Wiley & Sons, Ltd.

152

Z. U. M. ABUSINA ET AL.

1. Introduction

T

he Internet is becoming an increasingly important component of modern day communication and information exchange. Yet, the applications which use the Internet in general do not have much information about the underlying path(s) from the source to the destination, far less the characteristics of the paths. The layering concept has isolated applications from the network related information with the result that network applications such as FTP, WWW, Mirroring etc. are currently operated with little or no knowledge about the routes and their characteristics.1,2 It is clear that these applications could operate more efficiently if the routes and their characteristics are known and/or are made available to the concerned application.3 To attain these goals, there are two separate issues to be dealt with. First, a better way to quantify network performances should be defined so that the information is suitable for the users or applications. The second question is how to estimate, or predict, the future network performance parameters based on past observations. This is added to the question of how to make them available to users and/or applications. The IP Performance Metric working group (IETF-IPPM-WG)4,5 is working on developing a set of metrics that will characterize quality, performance, and reliability of Internet data delivery services (networks). Several tools6–9 exist to measure different parameters of network performances. However, estimation or prediction of network performance has been a challenging issue because of the inherent uncertainty in its behavior influenced by several factors1 as follows. Dynamic Behavior. The network resources as well as the utilities are dynamic, thus the network characteristics will have a dynamic component too. For example, a link may be down for a short while and this will show up as a very low data transfer rate, which is not the true general characteristic of the network. Burstiness. The network traffic traversing the network does not maintain a consistent pattern. At times, when a session of an application starts, the traffic pattern suddenly goes high. On the other hand, during the inter-session period, network

Copyright © 2005 John Wiley & Sons, Ltd.

resources remain idle. For these reasons, sudden impulses are common phenomena on the network performance (e.g. throughput, delay, latency, traffic size etc). Human Factor. Network behavior is greatly influenced by human working hours. The human working hours may be the official working hours (excluding the holidays) and the time span when network usages are relatively cheap. In our previous works,10,11 we have shown this periodic behavior. RFC 3432,12 also deals with such characteristics. In a proper prediction model, all these features should be taken into consideration. The existing literature, as will be outlined in the next section, describes works carried out from different perspectives. Despite differences in the addressed domains, these works have one point in common. Most of them aim at developing complex, and to some extent complete, mathematical models to predict future semi-static network traffic. These may work well in providing the management with necessary information regarding networking requirements. However, they fail to predict the dynamic network behavior in the immediate future. In this paper, we address network performance prediction as an engineering problem. The basic contribution of this paper is to predict network performance (e.g., throughput), dynamically for the immediate future. Therefore, rather than concentrating on a complex mathematical model, we develop a relatively simple heuristic-based approach that yields a reasonably accurate estimate of network throughput in the near future with moderately low computational requirement. Our proposal also considers the practical implications of prediction. Therefore, instead of following the conventional approach to predict one single value, we predict a range within which the network performance may lie. This range is bounded by our two newly proposed indices, namely, Optimistic Network Performance Index (ONPI) and Robust Network Performance Index (RNPI). ONPI corresponds to the best expected network performance. RNPI, on the other hand, corresponds to the region of lowest expected network performance. Most of the time, network performance is expected to lie between ONPI and RNPI. This approach has practical implications for various applications.

Int. J. Network Mgmt 2005; 15: 151–162

DYNAMIC PREDICTION OF NETWORK PERFORMANCE

W

e address network performance prediction as an engineering problem.

We examine our model through a one-year-long data transaction between several pairs of networks. It is found that our model performs quite well in predicting network throughput dynamically. Besides forecasting network throughput, this model can also be a candidate for prediction of other network performance parameters such as round trip delay etc. The rest of this paper is organized as follows. In Section 2, we provide a brief outline of related works on network performance prediction. We then present our proposed model for prediction of network performance in Section 3. We present the experiments and results along with evaluation of our model in Section 4. Finally we make conclusions in Section 5.

2. Related Works With the development and deployment of measurement tools,13–15 prediction of network performance based on network history has started gaining pace.16 In particular, network throughput prediction has attained maximum attention among researchers.17 One particular trend that is being observed is the development of complex mathematical models for network performance prediction. These models, in general, make forecasts of semistatic network throughput off-line. For example, in Reference 18, the seasonal form of the Auto Regressive Integrated Moving Average (ARIMA) model is employed to perform long-term predictions (two or more years into the future). Again, in Reference 19, a simple linear extrapolation algorithm is used to predict hotspots (sudden increases of traffic) which are frequently seen around large world-wide events such as the Olympic games or the World Cup. Some prediction models based on neural networks have also been proposed. However, in general, they focus on application-specific performance prediction. In Reference 20, single and multiple frame video throughput prediction using neural network models has been proposed.

Copyright © 2005 John Wiley & Sons, Ltd.

153

It has been observed10,11 that network past history may be an excellent source for estimating network behavior in the future. However, except for presenting a gross measurement, that is, a normal statistical average over different time slices of the 24 hour day and inferring that obtainable network performance has some relation with the period at which it operates, no further analysis has been made. Since mere statistical averages are prone to error induced by non-representative data (that is, outliers) this information is not enough for our purposes. These previous efforts have predicted the probable value of a future performance parameter as a single curve, hence neglecting the intrinsic bursty nature of the traffic. Also, they predict network performance off-line, ignoring the dynamic change in the network operating environment. Predicting a single curve with a view to claiming that network throughput will attain that value in the future does not have significant meaning from the practical application point of view. Also, offline information may prove to be of little use and only to applications, if at all. However, in our approach presented in this paper, we propose a range for expected network performance (throughput) as a representative performance parameter. Again, our model makes a dynamic prediction of network throughput, taking immediate past network throughput information to forecast the immediate future. These two characteristics make our approach a practical engineering solution for meeting the requirements of various applications.

3. Our Estimation Model Our estimation model21 follows the observations made in Reference 10. As discussed in Section 2, the authors show that network past history may be an excellent source for estimating network behavior in the future. However, except for presenting a gross measurement, that is, a normal statistical average over different time slices of the 24 hour day and inferring that network throughput has some relation with the period at which it operates, no further analysis has been made. Since mere statistical averages are prone to error induced by non-representative data (the outliers),

Int. J. Network Mgmt 2005; 15: 151–162

154

Z. U. M. ABUSINA ET AL.

260 Average 240

Network Throughput [kbps]

220

Median

200 180 160 140 120 100 80 60 00:00

03:00

06:00

09:00 12:00 15:00 Hours of day (GMT)

18:00

21:00

00:00

Figure 1. Comparison of average and median of network traffic

this information is not enough for deployment purposes.22 This is revealed clearly in Figure 1. Here, we plot the average and median of network throughput between two networks (RIEC-net and goo-net) at different times of day computed over a month (March 1998). This clearly indicates that at least in 50% of the cases, the average overestimated the network throughput. In this paper we therefore propose a model for the prediction of network performance that considers median as the base statistics. We then use standard statistical tools like percentiles, quartiles, SIQR (Semi Inter Quartile Range) and their aggregates. Since we propose an engineering approach to network performance prediction based on network measurements, rather than a complex mathematical framework, our model introduces some new heuristic-based operators similar to the basic Genetic Algorithm (GA) operators. These new operators and associated actions will be referred to by the same name as their GA counterparts in this text. As stated before, the essential requirement of a measurement-based dynamic network performance prediction mechanism is to have practical significance. Internet traffic has a continuously

Copyright © 2005 John Wiley & Sons, Ltd.

varying pattern. Therefore, predicting a single value to represent network throughput in the immediate future is almost meaningless. This fact leads us to propose a range-based prediction model. The idea is to make a prediction of a range within which we wish our network to operate. At the two ends of this lie Optimistic Network Performance Index (ONPI) and Robust Network Performance Index (RNPI) which we describe later. The reason for using the term performance index is that, in addition to predicting network throughput, this model can be used for other network performance parameters like round trip delay etc. In this paper we, however, focus on network throughput only and the term performance index would correspond to throughput in a network. Our model essentially depends on a continuous measurement of network throughput. We consider both the historical network traffic information as well as the immediate past network traffic information to forecast a meaningful range for network throughput in the immediate future. This type of prediction can enhance the performance of various applications over the Internet quite significantly.

Int. J. Network Mgmt 2005; 15: 151–162

DYNAMIC PREDICTION OF NETWORK PERFORMANCE

155

240 Mon Tue

220

Wed Thu

Network Throughput [kbps]

200

Fri

180 160 140 120 100 80 60 00:00

03:00

06:00

09:00

12:00

15:00

18:00

21:00

00:00

Hours of day

Figure 2. Medians of network traffic at different times on different days of a week

—3.1. Definitions and Assumptions— We are using median network performance indices as our base statistics. This gives us an estimate of the central tendency of the observed data. At the same time, it helps us to remove the outliers from active consideration. Once we have chosen the measure of central tendency, we need some way to characterize the measure of dispersion. There are many ways such as range, variance/standard deviation, mean absolute deviation, semi-interquartile range etc. for the purpose.22 Among them, semi-interquartile range or SIQR is very much resistant to outliers. It is similarly true for percentile ranges. We have therefore used percentile ranges in our mathematical framework. Figure 2 shows the medians of network throughput on different working days of a week at different times of each day. From this figure we easily observe that at a particular time of each day, network throughput follows similar patterns for different working days of a week. Therefore, as in Reference 10, we also infer that the operating period is one of the parameters governing network performance. We consider this time dependence dividing them into discrete time slices, Dt. The

Copyright © 2005 John Wiley & Sons, Ltd.

length of these time slices may vary depending on the requirements. We may also consider variable time slices Dt for some applications. For the sake of easy representations, from now on, we shall refer to different time slices Dt as t, t + 1, t + 2, and so on. We can have many observations l, l - 1, l - 2, etc. at the past for the same set of time slices. Similarly, we can have many observations l + 1, l + 2, l + 3 etc. for the present and also the future for the same set of time slices. In this model we consider two types of time dependence. One is the historical dependence of network throughput for a particular time slice t. The other one is the dependence on the current state of network throughput. In our mathematical model, we have taken the first type of dependence into account through the historical factor ht,l+1. The idea is that at time slice t in observation l + 1, network throughput should show a behavior close to some derivation from several past observed behaviors of the network throughput at the same time slice, t. The second category of time dependence stands for the effect of currently obtained network throughput. The idea is, if at time slices t - 1, t - 2, t - 3 etc. in observation l + 1 we have some amount of network throughput, the probable

Int. J. Network Mgmt 2005; 15: 151–162

156

Z. U. M. ABUSINA ET AL.

throughput at time t may be partly characterized by those observations. This is a sort of rolling over of network performance from one time slice to another. Therefore, we call the effect the roll over factor. In our current approach, the roll over factor has been considered to be determined by a roll over function. The roll over function itself is considered to be dependent on the characteristic function, gt(l + 1) and the roll over ratio, rt(l + 1). In this model, we have assumed a simple characteristic function considering dependence on only one time slice t - 1 at observation l + 1 as follows: 0 if there is no dependence gt (l + 1) = ÏÌ Ó1 if there is some dependence

Generally, there is some dependence between two different time slices. However, we have noted that at a certain point in time, there is no relation between the performance index at some time slice with the next time slice. It is also observed that at times the network performance shows some abrupt changes. We have introduced a ‘dynamic cross over operator’, described later in the paper, to adjust to the abrupt change in network behavior. In our model, instead of estimating some single value to be the expected network performance, we have introduced a new concept of predicting a range within which we would wish the network throughput to probably be. At the two ends of this range lie the ONPI and the RNPI. If the network operates at nearly the highest efficiency, the throughput would be near ONPI. Of course, the fraction of time when throughput can be this high would be small. On the other hand, if the network operates nearly at its normal efficiency, i.e. at least at an efficiency which is expected in most of the cases, then the network throughput would be near RNPI. Most of the time the network is expected to operate near this value. We may consider ONPI and RNPI to be analogous to the first and third quartiles or 10 and 90-percentiles, respectively, and so on. ONPI and RNPI vary with time and load on the network. Therefore, if we need to know what is the least we should expect the network throughput to be at an instance of time, we should consider RNPI as our index. However, if some applications are too sensitive to network bandwidth availability, ONPI should be used as the predicted network throughput.

Copyright © 2005 John Wiley & Sons, Ltd.

W

e propose a range bounded by two performance indices rather than predicting one single value.

—3.2. Mathematical Modeling— Using the mathematical framework and assumptions described in the previous subsection, we employ techniques having some resemblence to those of a Genetic Algorithm to predict our performance indices. In our model, we consider the statistical parameters of interest to be genes of a chromosome. We may look upon a chromosome, ct,i corresponding to the statistical parameters of interest at time slice t in observation i, to be a vector as follows: ct , i = (mt , i , s ot , i , s rt , i , nt , i )

Here, mt,i is the median of performance index at time t in observation i sot,i is the optimistic performance index at time t in observation i srt,i is the robust performance index at time t in observation i nt,i is the number of accesses at time t in observation i sot,i and srt,i may correspond to 10 and 90percentiles or the first and third quartiles. In this paper, we consider the former pair, i.e., the percentiles. The number of accesses, nt,i, indicates the representativeness of the data in consideration. A higher value of nt,i assures a higher weight for corresponding historical performance data. Once we have our chromosomes defined, we can describe the alive population as a matrix: Èmt ,1 Ím t,2 Pt = Í Í M Í Îmt , l

s ot ,1

s rt ,1

s ot , 2

s rt , 2

M

M

s ot , l

s rt , l

nt ,1 ˘ nt , 2˙ ˙ M˙ ˙ nt , l ˚

Therefore, at time t, we have l chromosomes in the alive population. That is, each row i of Pt corresponds to an observation i for the time slice t.

Int. J. Network Mgmt 2005; 15: 151–162

DYNAMIC PREDICTION OF NETWORK PERFORMANCE

The ages of different chromosomes differ. We are considering a model of fixed alive population. Then, in order to accommodate places for newly born offspring, the old ones have to die. Here, a new offspring, for example ct,l+1, corresponds to a new observation l + 1 for time slice t. The chromosome that dies due to the inclusion of a new offspring normally goes beyond our active consideration. In order to obtain the prediction of network performance for observation l + 1 at time slice t, we normally employ a subset of all these observations or chromosomes. Therefore, we will use a submatrix of Pt to be our learning set or learning window. The learning window can be defined as: Èmt , l -w +1 Ím t , l -w + 2 Wt = Í Í M Í Îmt , l

s ot , l -w +1 s ot , l -w + 2 M s ot , l

s rt , l -w +1 nt , l -w +1 ˘ s rt , l -w + 2 nt , l -w + 2˙ ˙ ˙ M M ˙ s rt , l nt , l ˚

In order to account for the first type of time dependence, we use a strategy that the nearest past bears the closest resemblance with the immediate future. As such, while generating the statistics indicating historical time dependence, we assign highest preference on the chromosome corresponding to the nearest past for a fixed time slot. In doing so, we use the following normalized weight vector p, p = (pl -w +1, pl -w + 2 ,L pl ) We then compute the historical time dependence, ht,l+1 as the product of normalized weight vector p and the learning set, Wt. h t,1+ 1 = p . Wt

The second type of time dependence, i.e. the roll over factor, rt,l+1 may be computed with the aid of roll over function, which itself depends on several parameters. These are: the roll over ratio, rt,l+1 defined as: l

l

s ot , i * pi - Âi = l -w +1s rt , i * pi  rt ,1+1 = l i = l -w +1 , l Âi = l -w +1so t -1 ,i * pi - Âi = l -w +1s r t -1 ,i * pi (

)

(

)

the characteristic function gt(l + 1) and the chromosome ct-1,l+1. The vector roll over factor, rt,l+1, can then be defined as rt,l + 1 = rt , l +1 * gt (l + 1) * c t - 1,l + 1

Copyright © 2005 John Wiley & Sons, Ltd.

157

The roll over factor, however influences our prediction model depending on its importance determined by the following parameter: n A, l +1 = nt -1, l +1

which will later be normalized for actual prediction. Here nt-1,l+1 corresponds to the chromosome ct-1,l+1. The implication of taking this number of accesses into account is that the roll over factor would be more meaningful if there were a greater number of accesses and less meaningful if there were fewer accesses. The other factor involved in this process is, n A¢ , l +1 = median{nt , i 1 £ i £ l }

We then define the roll over weight as follows: at , l +1 =

n A, l +1 n A, l +1 + n A¢ , l +1

Mutation. Once equipped with the above tools, we describe prediction as a mutation of all genes, in this case in the chromosomes of the learning set, Wt and ct-1,l+1. That is, all chromosomes concerned in this case change to some extent and thus we have the prediction vector: v t,l + 1 = (1 - at , l +1) * h t,l + 1 + at ,l +1 * rt,l + 1

It is worth noting that we have employed a somewhat new type of mutation to have an estimate for prediction. In the ideal case, the prediction vector vt,l+1 should be close to the vector corresponding to the chromosome for actual observed performance, i.e. ct,l+1. Birth and death. Using mutation, we have an estimate or prediction of how the network is expected to behave. Then, as we observe the actual performance, we have a new chromosome ct,l+1. As mentioned before, similar to the natural process of life and death, the oldest chromosome dies in order to make room for the new chromosome in the alive set. The implication is to emphasize the latest set of network performance information for use in the future. The matrix corresponding to the alive population set after one such birth–death process is as follows: Èmt , 2 Ím t,3 Pt = Í Í M Í Îmt , l +1

s ot , 2

s rt , 2

s ot , 3

s rt , 3

M

M

s ot , l +1

s rt , l +1

nt , 2 ˘ nt , 3 ˙ ˙ M ˙ ˙ nt , l +1˚

Int. J. Network Mgmt 2005; 15: 151–162

158

Z. U. M. ABUSINA ET AL.

The learning set after a birth and death process would also change accordingly as: Èmt , l -w + 2 Ím t , l -w + 3 Wt = Í Í M Í Îmt , l +1

s ot , l -w + 2

s rt , l -w + 2

s ot , l -w + 3

s rt , l -w + 3

M

M

s ot , l +1

s rt , l +1

nt , l -w + 2˘ nt , l -w + 3˙ ˙ M ˙ ˙ nt , l +1 ˚

Cross-over. Sometimes network performance changes abruptly. Then it is likely that the prediction model would not be able to provide so good an estimate of the throughput. Mathematically, we shall call an estimate to be not so good if the difference between the predicted and actually observed chromosomes: d t , l +1 = c t,l + 1 - v t,l + 1

exceeds some threshold T, i.e., d t , l +1 > T

In such cases, if dt,l+1 remains high for a considerably long time, a cross over would take place. The idea is to replicate all the genes of the latest chromosome in the oldest ones in the learning set excluding the one corresponding to the number of accesses. Then the system is tested for determining whether dt,l+1 is below the threshold value. If not then the process is repeated. However, the number of such iterations has a limit at the window size. The value of T could be some a per cent of vt,l+1. The matrix for the learning set after one birth and death and one cross over would be something like: Èmt , l -w + 3 Í M Wt = Í Ímt , l +1 Í Îmt , l +1

s ot , l -w + 3

s rt , l -w + 3

M

M

s ot , l +1

s rt , l +1

s ot , l +1

s rt , l +1

nt , l -w + 3˘ M ˙ ˙ nt , l +1 ˙ ˙ nt , l -w + 2˚

network throughput dynamically. In the following text, we therefore focus on only one such pair. This particular pair consists of inbound traffic to RIECnet from goo-net. RIEC-net (riec.tohoku.ac.jp) is an academic network of Tohoku University Research Institute of Electrical Communication. Goo-net (goo.ne.jp) is a popular commercial network in Japan providing a variety of services including free web mail accounts, important news, advertisements etc. The data traffic log for downstream traffic between RIEC-net and goo-net for a period of one year was analyzed. To maintain conformance with observations and characterization efforts in our previous works, we used the same data set as in References 1, 21 and 23 for evaluation purposes. We considered some portion of the network trace data as the learning set and tried to make predictions, using our model, for the the remainder of it. This is done with a view to imitate real-time dynamic prediction of network traffic. We then compare the degree to which the predictions and the actual values are in agreement. We also analyze the effects of varying learning set sizes. Furthermore, the behavior of the prediction model facing some abrupt changes in network performance is also analyzed. As described earlier, no existing work predicts network throughput dynamically like our one. At the same time, we propose a range bounded by two performance indices rather than predicting one single value. This is completely new and practical in addressing the problem. Therefore validation and performance comparison of our approach can be made solely with actual data which we present in the following subsections. We compare our predicted ONPI with the 10th percentile and RNPI with the 90th percentile of the descending network throughput data.

—4.2. Accuracy of Prediction—

4. Experiments and Evaluation —4.1. Experimental Set-up— Since we have proposed an engineering approach rather than a complex mathematical one, experimental verification is essential for its validation. We therefore tested our model with long time data between several networks. Our model performed equally well in these cases to predict

Copyright © 2005 John Wiley & Sons, Ltd.

We first consider how accurately our prediction model could perform. In presenting our results, we naturally emphasize the characteristics of our model for the period of the day when the demand for network bandwidth is most crucial. A correct prediction for this period is likely to be more important than for any other time. Also, we show the accuracy for a particular day in Figure 3. Other days follow similar convergence characteristics.

Int. J. Network Mgmt 2005; 15: 151–162

DYNAMIC PREDICTION OF NETWORK PERFORMANCE

159

100000 ONPI(predicted)

90000 80000

ONPI(Actual)

Network Throughput [kbps]

70000 60000 50000 40000

RNPI(Predicted)

30000

RNPI(Actual)

20000 10000 0 13

14

15

16

17

18

19

Hours of Day

Figure 3. Accuracy of prediction Figure 3 shows our predicted RNPI and ONPI network throughput for a particular learning set size of 8, with broken lines and the corresponding actual values using solid lines. We can easily notice that both ONPI and RNPI predictions by our model fit quite well with the actual values. For all the cases, except some abrupt change in network traffic, both ONPI and RNPI match within 10% of the actual value. Therefore we can infer that under normal network behavior, our model works well enough. One interesting point to note would be that, in the case of ONPI, there is a higher difference between what we expected to see and what we observed actually. However, in the case of RNPI, both the prediction and the observation are quite close. This is important because at RNPI, we want a robust estimate of performance index. The significance is that the reliability of the model increases with the increase in requirement.

—4.3. Effect of Learning Set Size— In our experiments, we have observed that the size of the learning set often influences the accuracy of the prediction. More interestingly, although a

Copyright © 2005 John Wiley & Sons, Ltd.

reasonably large learning set size is essential for optimum prediction, a much larger learning set size may not always be something we should opt for. In Figure 4 we observe that in comparison with a reasonable learning set size, a learning set with higher cardinality (here l = 12) performs worse in predicting both the ONPI and RNPI. It should still be noted that the downgrading effect caused by too large a learning set is more prominent for ONPI than for RNPI. This is good for the users of this model as the error in the predicted RNPI is still near the range of acceptability.

—4.4. Effect of Abrupt Changes— Any model that learns through experience is not supposed to predict well at the occurrence of an impulse behavior. Therefore, it is quite natural that the accuracy of prediction of our model suffers when some abrupt changes occur in network performance. Figure 5 shows one such example. One interesting point to note in this context is that here, also, the predicted ONPI differs to a great extent from actual behavior. But on the other hand, the predicted RNPI does not differ that much from the

Int. J. Network Mgmt 2005; 15: 151–162

160

Z. U. M. ABUSINA ET AL.

100000

Network Throughput [kbps]

90000 80000 ONPI(Actual) 70000 60000 ONPI(Predicted)

50000 40000

RNPI(Predicted) 30000 RNPI(Actual) 20000 10000 0 13

14

15

16

17

18

19

Hours of Day

Figure 4. Effect of learning set size

350000

300000

Network Throughput [kbps]

ONPI(Actual) 250000

200000

ONPI(Predicted)

150000

100000 RNPI(Actual) 50000 RNPI(Predicted) 0 13

14

15

16

17

18

19

Hours of Day

Figure 5. Erroneous prediction in case of abrupt change

Copyright © 2005 John Wiley & Sons, Ltd.

Int. J. Network Mgmt 2005; 15: 151–162

DYNAMIC PREDICTION OF NETWORK PERFORMANCE

161

200000 180000 160000 ONPI(Actual) Network Throughput [kbps]

140000 ONPI(Predicted)

120000 100000 80000 60000

RNPI(Actual)

40000

RNPI(Predicted)

20000 0 13

14

15

16

17

18

19

Hours of Day

Figure 6. Abrupt change not affecting the prediction considerably

actual value. This, once again reaffirms that our model has been working well where it is needed. However, the model shows some much better characteristics in some other occurrences of abrupt behavior (Figure 6) when learning about the abrupt change is achieved fast.

5. Conclusions Network performance prediction based on network measurement traces appears to be a daunting challenge to network researchers. Several works in different contexts have been done in the past. These contributions, however, mostly focus on devising complex mathematical formulations to predict mostly pseudo-static network behavior in the future. In this paper, we address network throughput prediction as an engineering problem. The main contribution of this paper is to predict network throughput dynamically for the immediate future. Our proposal also considers the practical implication of prediction. Therefore, instead of following the conventional approach to predict one single value, we predict a range within which network performance may lie. This range is bounded by our two newly proposed indices,

Copyright © 2005 John Wiley & Sons, Ltd.

namely, Optimistic Network Performance Index (ONPI) and Robust Network Performance Index (RNPI). This approach has practical implications for various applications. We examine our model through a one-year-long data transaction between several pairs of networks. It is found that our model performs quite well in predicting network performance dynamically. Besides forecasting network throughput, this model can also be a candidate for prediction of other network performance parameters like round trip delay, unidirectional delay etc.

References 1. Chakraborty D, Ashir A, Suganuma T, Keeni GM, Roy TK, Shiratori N. Self-similar and fractal nature of Internet traffic. International Journal of Network Management 2004;14:119–129. DOI: 10.1002/nem.512. 2. Ahmed A, Mansfield G, Shiratori N, A meeting scheduling system for global events on the Internet. Proceedings of the Internet Society’s 8th Annual Networking Conference, INET’98, July 1998, Geneva, http://www.isoc.org/inet98/proceedings/1b/1b_ 1.html. 3. Claffy KC. Internet measurement: Myths about internet data, in Proceedings of the 24th North

Int. J. Network Mgmt 2005; 15: 151–162

162

4.

5.

6.

7.

8.

9.

10.

11.

12.

13. 14.

15.

Z. U. M. ABUSINA ET AL.

American Network Operators’ Group (NANOG24), February 2002. IETF-IPPM-WG: Framework for IP performance metrics, http://www.ietf.org/html.charters/ippmcharter.html. Paxson V, Almes G, Mahdavi J, Mathis M. Framework for IP performance metrics. ftp://ftp.isi.edu/ innotes/rfc2330.txt May 1998. Koide K, Keeni GM, Kitagata G, Shiratori N. DCAA: a dynamic constrained adaptive aggregation method for effective network traffic information summarization. IEICE Transactions on Communications 2004;E87-B:413–420. Paxson V, Almes G, Mahdavi J, Mathis M. An architecture for large-scale Internet measurement. IEEE Communications 1998;36(8):48–54. Graham ID. Non-intrusive and accurate measurement of unidirectional delay and delay variation on the Internet. Proceedings of the Internet Society’s 8th Annual Networking Conference, INET’98, July 1998, Geneva, http://www.isoc.org/inet98/proceedings/ 6g/6g_2.html. Murayama Y, Yamaguchi S. DBS: a powerful tool for TCP performance evaluations. Proceedings of SPIE Volume 3231, November 1997. Ashir A, Mansfield G, Shiratori N. Estimation of network characteristics and its use in improving performance of network applications. IEICE Transactions 1999;E82-D:747–755. Ashir A, Mansfield G, Shiratori N. Network processes: Scheduling and server selection for efficient operation. Proceedings of Internet Conference 98: IC98. December 1998, Kyoto, Japan. Raisanen V, Grotefeld G, Morton A. Network performance measurement with periodic streams. RFC 3432, 2002. Squid Internet Object Cache, http://www.nlanr.net. Kamiya H, Ohta K, Kato N, Mansfield G, Nemoto Y. Improving efficiency of network services. AsiaPacific Network Operations and Management Symposium, Sep. 1998. PTOPOMIB Working Group (concluded), Physical Topology MIB, http://www.ietf.org/html.charters/ ptopomib-charter.html.

Copyright © 2005 John Wiley & Sons, Ltd.

16. Seshan S, Stemm M, Katz RH. SPAND: Shared Passive Network Performance Discovery. http:// www.cs.berkley.edu:80/ss/papers/usits97/html/ photo.html. 17. Mansfield G, Jayanthi K, Ashir A, Shiratori N. Network maps: Synthesis and applications. International Conference, APSITT99, August 1999, Mongolia. 18. Groschwitz NK, Polyzos GC. A time series model of long-term NSFNET backbone traffic. Proceedings of IEEE ICC’94. 19. Baryshnikov Y, Coffman E, Rubenstein D, Yimwadsana B. Traffic prediction on the Internet. Technical Report EE200514-1, Computer Networking Research Center, Columbia University, May 2002. 20. Drossu R, Lakshman TV, Obradovi’c Z, Raghavendra C. Single and multiple frame video traffic prediction using neural network models. Computer Networks, Architecture and Applications, Raghavan SV, Jain BN (eds), Chapman and Hall, 1995, pp. 146–158. 21. Zabir SMS, Ashir A, Shiratori N. Estimation of network performance: an approach based on network experience. Proceedings of IEEE ICOIN-15, January 2001. 22. Jain R. The Art of Computer Systems Performance Analysis, Wiley: U.S.A., 1991. 23. Ashir A, Suganuma T, Kinoshita T, Roy TK, Mansfield G, Shiratori N. Network traffic characterization and network information services-R and D on JGN. Computer Communications 2001;24: 1734–1743. 

If you wish to order reprints for this or any other articles in the International Journal of Network Management, please see the Special Reprint instructions inside the front cover.

Int. J. Network Mgmt 2005; 15: 151–162