Sep 24, 1997 - enables us to dynamically plan and consider uncertainty when decisions are ... such as the processing capacity at the network nodes. ..... If by our allocation strategy or simply through over-demand the node capacity is not.
Baltzer Journals
24.09.97
Modelling Aspects of Distributed Processing in Telecommunication Networks Asgeir Tomasgard1 , Jan A. Audestad2 , Shane Dye1 Leen Stougie3 , Maarten H. van der Vlerk4 , Stein W. Wallace
1
Department of Economics and Technology Management, Norwegian University of Science and Technology, N-7034 Trondheim, Norway 2Telenor AS, P.O. Box 6701 St. Olavs plass, N-0130 Oslo, Norway 3 Faculty of Economics and Econometrics, University of Amsterdam, Roetersstraat 11, 1018 WB Amsterdam, The Netherlands 4 Dept. of Econometrics, University of Groningen, P.O. Box 800, 9700 AV Groningen, The Netherlands 1
The purpose of this paper is to formally describe new optimization models for telecommunication networks with distributed processing. Modern distributed networks put more focus on the processing of information and less on the actual transportation of data than we are traditionally used to in telecommunications. This paper introduces new approaches for modelling decision support at operational, tactical and strategic levels. One of the main advantages of the technological framework we are working within is its inherent exibility, which enables us to dynamically plan and consider uncertainty when decisions are made. When we present the models, emphasis is placed on the modelling discussions around the shift of focus towards processing, the new technological aspects, and how to utilize exibility to cope with uncertainty.
Keywords: distributed networks, distributed processing, telecommunication, stochastic (integer) programming, stochastic modelling, optimization. 1 Introduction In this paper we present several models for telecommunication networks with distributed processing. In particular, we consider possibilities for handling uncertainty which is present in operational as well as investment problems. Several factors are important in the technological push that has opened the possibilities for distributing the processing in telecommunication networks. Examples of technological factors are digital technology, modern high-speed network architectures like BISDN and packet switched data transmission concepts like ATM (see for example [8, 31]).
Work partially funded by Telenor
A. Tomasgard et al. /Modelling Aspects of Distributed Processing. ..
2
This allows for provision of more and more complex services. Another technological factor is the use of bre optics for the network wiring, which yields cheap transportation capacity. In addition several standardization initiatives for distributed telecommunication architectures have been developed. The standardization concerns software, equipment, and interfaces between the `objects' present in the network. One of these initiatives is the Telecommunications Information Networking Architecture Consortium (TINA-C) [6, 33] scheduled to be completed in 1997. One of the main di erences between new networks based on distributed processing and traditional telecommunication networks (with localized processing) is the increased exibility in resource allocation. The impact of the above developments is that an enormous number of services can be provided on a telecommunication network. In this paper the term service is used to encompass a set of software applications together with the set of computing and network resources it uses. Typical examples of resources in a distributed network are: switches, servers, transmission options, routers, and computers. Services capable of processing information are o ered through computers, or computing nodes, by the software applications running on these computers. While traditional services tend to use more of the transportation resources in the network, we believe that the extensive growth in newer services will come from those requiring more resources on the computers running the applications used by the services. At the same time the investment cost of transportation capacities has decreased as bre optic technology has become standard. This means that the limited resource in the distributed networks will often be the computing resources, such as the processing capacity at the network nodes. If there is a shortage of computing resources, then some of the requested services can not be o ered. Another development is that deregulation all over the world introduces competition in telecom markets. New players enter the telecommunications scene and the roles of old players change. For example, the di erences between distributed computer networks and telecommunication networks are getting smaller as a consequence of the development discussed above. Deregulation and new technology together create a rapidly changing industry: traditional telecommunication companies lose their monopolistic positions and will therefore have to become competitive. New technology and deregulation mean that there is a need to introduce optimization models at a higher layer in the communications network. This paper presents models with operational and tactical/strategic time horizons. We concentrate on models in which services and distributed processing of information at computing nodes is treated rather than transportation and routing of information. The main purpose of the paper is to present examples of how the change of focus in service provision towards processing of information will lead to interesting new optimization models. We do not attempt, at this stage, to provide the precision necessary for real applications. As far as the literature is concerned, a reference on the technological aspects of distributed networks is [27]. However, this reference is neither telecommunications nor optimization oriented. A large number of optimization models with applications within telecommunications exist, see for example [15, 16, 32]. There are also papers dealing with aspects of speci c new services introduced into modern telecommunication networks, like video on demand [1, 5]. The theory of queuing networks has been applied to transportation aspects of communications networks, see for example [28]. Then, stochastic linear programming models have been used to address uncertainty in demand when investing in components of the transportation structure [13, 35]. When it comes to robustness
A. Tomasgard et al. /Modelling Aspects of Distributed Processing. ..
3
and reliable routing an example is found in [14]. A common feature of all of the above work is that the main focus is on transportation of data and the resources related to transportation. In [20] distribution of the workload in a network of computers is discussed with whole jobs being processed on a single machine. In fact we are not aware of any literature dealing with optimization models for distributed networks that focus on distributed processing aspects and distributed services' use of processing capacity at the network nodes. Clearly, there is a great deal of uncertainty found in both investments in and the operation of these modern distributed networks. At the same time, as we will see, we have excellent mechanisms for dealing with this uncertainty because of greater exibility in investment, con guration, and resource allocation. In this paper we will deal with uncertainty using stochastic programmingmodels. For the basic concepts and terminology of stochastic programming we refer to [19]. The paper is organized as follows. Section 2 de nes the necessary technological terms and the framework we operate within. At the operational level, consideration is given to the problem of how to utilize network resources in order to meet demand for services both now and in the near future (Section 3). As an example of a problem that appears at the tactical/strategic level, we treat the question of where to place computer resources in a region in the distributed network (Section 4). This section closes by introducing some notation used in the rest of the paper. Stochastic parameters are always denoted by Greek letters accented with a tilde, e.g. ~, whereas is a realization of the same stochastic parameter. Furthermore parameters related to demand for processing capacity are denoted with d or everywhere in the paper. Parameter s is used to denote capacity and variables x are always concerned with processing capacity. Their interpretation and units vary between the models, as the models operate in di erent timeframes and concentrate on di erent aspects.
2 The technological framework Aspects of distribution constitute the basis for the optimization models that are presented below in Sections 3 and 4. Therefore we will brie y describe a telecommunications model of the distributed framework. This section is based on the TINA-C documentation for which [3, 7, 30] are introductions. The basis for parts of TINA-C has been the Information Networking Architecture (INA) of Bellcore[2, 34]. 2.1 A distributed network model
We may describe the relationship between transportation networks, computers with processing resources and the applications used to build services by the three planes shown in Figure 1. In the upper plane we have a set of software applications. The computers, or computing nodes, where the applications reside, can be seen as the \glue" which binds the distributed application architecture and the underlying network architecture together. Examples of interaction between applications are shown by the solid lines between them. This interaction usually creates information ows through the computers and the underlying networks. The solid lines in the computing nodes plane describe virtual links
A. Tomasgard et al. /Modelling Aspects of Distributed Processing. ..
4
Applications
Computing nodes
Networks
Figure 1: The relationship between the transportation networks, the computing nodes and service applications. between pairs of computers. We use the term virtual to indicate that there does not necessarily exist a corresponding direct physical link between the two nodes but that it is possible to transport information between them. The real transportation of information happens in physical links in the networks described in the network plane. In the network plane the nodes describe physical networks and the solid lines indicate which networks are connected to each other (without saying how they are connected). If all the networks are connected, we have virtual all-to-all links in the computing nodes plane. Mappings, possibly many-to-many, between planes are shown by the broken lines. One computing node may accommodate several applications and several identical instances of an application may reside at di erent nodes. In the same way one network may contain several computing nodes and one computing node may reside in several networks (providing processing for all these networks while not necessarily allowing transportation between them). Applications may be moved or copied between computing nodes with no limitation on how often this may occur. The mapping between applications and computing nodes is therefore dynamic. From the above description we understand that several instances of an application can exist at di erent computing nodes mapped to the same or to di erent networks. In many contexts the mapping between computing nodes and the networks will be stable. However, systems are emerging where even the mapping between nodes and
A. Tomasgard et al. /Modelling Aspects of Distributed Processing. ..
5
networks is dynamic. This is, for example, the case in mobile telephony, where the computing node in the mobile station can be mapped to several physical networks during execution of a service. Another example is low orbit satellites where the computing node is actually the satellite. For modelling purposes we will assume in the rest of this paper that the mapping between networks and computing nodes is static. Note that the distributed network model described above is a useful model for the Internet. The upper plane corresponding to the Web; the middle plane representing the servers; and the bottom plane being the international structure of telephone and data networks. 2.2 Distribution transparencies
In order to formalize some of the requirements for a distributed network, the TINA-C documentation [3, 17] describes a set of features, called transparencies, that a distributed telecommunication network must have. These transparencies are important with respect to the ability to adapt to changes, the availability of services and the exibility in service deployment, provision and management.
Access transparency. Every application reveals a standardized interface to other
applications, independent of where co-operating applications are placed. Applications which together constitute a service can thus be placed on di erent computers, without this a ecting the interaction between them and the access they have to each other. Location transparency hides the location of applications from each other. Applications can interact while their locations change over time, without the other applications noticing it. Migration transparency is an extension to the location and access transparencies above which allows dynamic relocation of application instances. Relocation can take place while services are interacting with each other. This transparency gives us the possibility to completely move an application while it is in use and interacting with other applications. Concurrency transparency allows several applications to interact with the same instance of another application at the same time. The fact that other applications use the same instance is hidden. Failure transparency ensures that errors generated by one instance of the application are hidden from other applications and users. Replication transparency hides the replication of one application to other computers, from other applications. This means that it may not be possible to know which instance of an application we are actually using.
These transparencies imply that applications can be distributed freely over the computing nodes of the network.
A. Tomasgard et al. /Modelling Aspects of Distributed Processing. ..
6
2.3 Abstract infrastructure
The engineering model [17] from TINA-C is used to describe any distributed system as a set of objects interacting through an abstract infrastructure, the Distributed Processing Environment (DPE). This abstract infrastructure exists on every computer in the network and provides a layer between the computer's native computing and communication environment and the services composed of applications. The mechanisms necessary to implement the transparencies above are hidden in the implementation of the DPE. The infrastructure and functionality provided by the DPE are the same for all hardware platforms and all services. In this abstract infrastructure traders ll an important role. A trader is a repository which contains information about available interfaces and application instances. Each trader has a domain where it is working and a domain can have several traders. Traders interact with each other to make sure that requests for applications can be met independently of where both the application instances and the users reside. Hence the traders make it possible to assume that our applications can interact to make a service without knowing each other's location or physical hardware platform. 2.4 Roles
We identify some of the roles of agents that are found in a distributed network. We only include the roles that have an impact on the models addressed in this paper. The customers are people or software requesting services. A service provider o ers services to customers. A service can be provided by anyone who has a computer which runs a DPE and is connected to a network. A network provider provides a transportation network with routing and switching capabilities to customers and service providers. Some or all of the network nodes are computing nodes with a DPE, they correspond to computing nodes in Figure 1. The network providers thereby act as hosts for the service providers' applications, both when it comes to computing resources like processing capacity and when it comes to transportation resources. Deregulation all over the world opens the possibility of many network and service providers operating in the same market. It is important to note that a network provider does not necessarily act as a service provider, and that service providers do not necessarily own any of the processing or transmission equipment they use.
3 Service provision The rst model we will consider is that of dynamically allocating processing resources at the computing nodes. From Section 2 we know that applications are connected to each other virtually through the underlying computing nodes and physically through the network structure, as described in Figure 1. The service provider allocates node resources in order to meet customer requests for services. The resource allocation should be made in order to utilize the resources in the best possible way, and may include an option to reject requests. We assume that processing capacity is the limiting resource. Before we completely de ne the problem, we must take a closer look at the services,
A. Tomasgard et al. /Modelling Aspects of Distributed Processing. ..
video
services composed of subservices
service B
service A
sound
7
data
subservices
computing nodes plane
Figure 2: Services are composed of subservices. Services can be distributed. how they use resources, and how the distribution transparencies from Section 2.2 in uence resource allocation. 3.1 Services and subservices
A service has already been de ned as a set of applications and their required resources. As a result of the distribution transparencies described in Section 2.2 the applications constituting a service may in principle be allocated arbitrarily to computing nodes of the network. However, from a practical point of view, it may be useful to group applications together into units that always run on the same node. We de ne a subservice as such a collection of applications that are to be located together to perform their assigned task. Thus, in the rest of this paper we will consider the subservices as the smallest indivisible entities of computer code that are the building blocks for the construction of services. Thus, all services are built from subservices, as shown in Figure 2. Here service A is built from subservices \sound" and \video". Service B is built from subservices \sound", \data" and \video". The distribution transparencies mean that services can be distributed, with their subservice instances spread over several nodes. Figure 2 illustrates this for service B. Also several instances of the same subservice may exist at di erent nodes, as for subservice \sound" in the illustrated example. The arcs in the lower half of the gure represent the di erent instances. A subservice instance can be used by several services and a service usually consists of several subservices. The number of customer requests for a service describes the demand from the customer locations. When a subservice is installed at a node, we assume that the only limit on the number of requests that can be served by it is the node's processing capacity. Concurrency and failure transparencies hide the requests from each other. We distinguish between two types of computer capacity use by a subservice. The rst one is a xed resource requirement induced by having the software running and ready to meet requests in real time. A subservice uses a xed amount of the node resources whenever it is installed
A. Tomasgard et al. /Modelling Aspects of Distributed Processing. ..
8
on the node, even when it is not satisfying a single request. The second one emerges from accessing the subservice. In this paper we assume that this second processing capacity requirement of a subservice increases linearly with the number of simultaneous requests for a service having the subservice as a constituting part. The linear factor depends on the speci c service as the processing intensity di ers from service to service. When a decision is made to install a subservice at a node, the subservice cannot usually be used to meet the demand immediately. There is normally a setup time in which the subservice can be collected from a database, loaded into the memory and initialized, before it can be used to serve requests. There can also be a delay between the time the service provider decides to remove a subservice and the time its xed resource use is released. 3.2 Quality of service, transparencies and location of demand
The customer's requirements from a service are covered in the concept Quality of Service (QoS) [18]. QoS for applications or subservices can be looked at as a set of user perceivable attributes de ning the desired behavior of the subservice when it comes to, for example, delay, reliability, image quality, image size, security etc. At a lower level one also de nes QoS parameters for the networks: bandwidth, delay, blocking probabilities, loss rates, etc. We are only concerned with QoS for subservices here, because we are examining the problem facing service providers. To be as general as possible, we will not specify the speci c parameters. In light of the distribution transparencies we assume that the QoS of a subservice is not a ected by the speci c computing node on which it is installed. The distribution transparencies from Section 2.2 indicate that the location of the subservice instances used to meet requests is not important from either the point of view of the interacting subservices or that of the customers. Customers have no preferences when it comes to the service instances they use, except through QoS, which we assumed not to be a ected by di erences in the nodes. The distribution transparencies therefore give rise to a lot of exibility when it comes to allocating resources in the network, and are crucial to the problem described in this section. The location of subservices clearly does matter when information generated or processed by services is to be transported in the underlying transportation networks. Even if we assume that all nodes have the same capabilities to run subservices, transportation may a ect the QoS experienced by the customer, for example, through time delay as a consequence of congested networks. It is natural to assume that the network provider can interact with the service provider to deploy services to nodes nearby the customers. This makes it possible to reduce the overhead from transportation of data when information
ows depends on the allocation of services to nodes. The important question is then: Is it, in our setting and given our assumptions, sensible to integrate into our decision models the representation of the actual underlying co-operation and the information ows? Even if the inclusion of information ows in service provision models may at rst sight seem necessary, there are several reasons why the opposite can be true. Firstly, in cases where the service provider is not the network provider, an agent o ering services may not know the physical location of his own subservices, because of migration and replication transparencies. It is also reasonable to believe that service providers, who rent computer capacity from network providers, can use this capacity to set up any service they like. Network providers responsible for the transportation network therefore cannot necessarily
A. Tomasgard et al. /Modelling Aspects of Distributed Processing. ..
9
control which services are running at which locations. Secondly, a service provider may not know the location of all of the users or even subservices of his service because this information can be hidden by the transparencies and the use of traders de ned in Section 2.3. Thirdly, we have indicated that there is a shift in focus, where services move from being mainly transportation oriented to being processing oriented. In keeping with the focus of this paper we assume that the processing capacity is the limiting factor, not the transportation. Under the fundamental assumptions that we have a distributed network where distribution transparencies are valid, and processing capacity is the limiting resource, we have the following interpretation of the e ect of distribution transparencies on resource allocation at an operational level: A1 : In regions of the networks the demand for processing capacity can be served at any computing node in the region, independent of the demand's and the node's location and without a ecting the subservices' QoS.
Based on the above discussion, this would imply the following underlying assumption: A2 : Within regions of the global network, the transportation networks are
designed to keep the time delay acceptable on transportation of generated information and to ensure the correct QoS for subservices, independent of the allocation of requests for subservices to nodes.
If processing capacity is the limited resource, the interpretation above is necessary to fully utilize the possibilities created by the distributed environment for dynamic resource allocation. It is important to note that the size of the regions above does not depend on physical distance, but rather on the underlying transportation networks of the network providers in question and their alliances. Possibly only one such region could exist, namely the whole world. If the regions are non-overlapping we can model the allocation of node resources for each region independently and without regard to customers' QoS requirements arising from transportation. In this way, it is possible to (heuristically) decompose any network into regions in which transportation has no e ect on the customers QoS. As we see the distributed framework from Section 2 strongly in uences our decision problem. The main e ect on modelling the resource allocation is that we do not have to model any information ows generated by a request, only the demand for the services. Under our assumptions, the actual locations of customers are thus irrelevant to the problem of allocating node resources. This means that the demand for processing capacity through subservices can be aggregated over the customers, services and all the locations in at least a region in the network providing the service. 3.3 A deterministic service provision model
We are now able to specify in more detail the service provider's problem. The service provider has a set of computers available where he can install subservices. He wants to allocate subservices to computing nodes in order to meet the demand for services. Installing too many subservices at a node implies that there will be no processing capacity
A. Tomasgard et al. /Modelling Aspects of Distributed Processing. ..
10
left to serve the demand for them, because of their xed resource need. Providing too few may imply that there is the capacity to serve demand, but not the ability. If by our allocation strategy or simply through over-demand the node capacity is not sucient to meet all demand, we will have to reject customers. There are several ways of interpreting rejections. Because of deregulation we know that there is a competitive market for subservices. In view of the distribution transparencies described in Section 2.2 and the DPE functionality described in Section 2.3, it is reasonable to assume that traders will be able to perform the task of identifying other instances of the subservice that the service provider is not able to o er. Subservices that are rejected are then either hired from other providers, or served in another region of our provider's operating area. The rejection cost can be interpreted as the cost of obtaining the subservice elsewhere. It is also possible that when a customer is rejected, he chooses to nd another service provider, or that a trader performs this task for him. Under the additional assumption that the service provider is not able to change the market price with his decisions, the rejection cost is equal to the market price for the rejected (sub)service. When there is a market for subservices and we utilize the functionality of traders, we can treat hiring subservices from elsewhere as partial rejection of a service. We do not have to di erentiate between the di erent activities above that would be taken for individual customer rejections. In all cases the rejection cost is equal to the market price for the rejected subservice. This allows us to have services appearing only implicitly in the model. In Section 3.6 we will brie y discuss the model in which services should be explicitly modelled. In that case rejection costs for a service are introduced. Let us rst describe a static model assuming deterministic demand and no lead time for installing subservices. Although demand uctuates over time and is not known in advance, there are two reasons for considering this model. First it gives us insight in the underlying allocation problem that may be used in solving the more realistic but also more complicated non-deterministic problem. Secondly, taking average values for the demand may be useful to nd a sort of default con guration of the system, i.e., a con guration that is used as long as no strange demand patterns occur. The above problem can be modelled as a linear mixed integer programming problem (MIP) [29]. The aim is then to allocate the subservices over our computing nodes so as to minimize the cost of rejection. Let I be the set of computing nodes and J the set of subservices. The zero-one variable z (i; j ) indicates whether or not node i has subservice j installed on it. The variable x(i; j ) denotes the amount of demand for processing capacity generated by subservice j and met at node i. In practice, the x(i; j ) will take on integer values, however relaxing the integrality requirements does not a ect the optimal solution much since the optimal values will, typically, be either 0 or relatively large. Let the demand for processing capacity generated by subservice j be d(j ). The cost of meeting a request for subservice j by hiring it from elsewhere is denoted by (j ). The xed resource usage for subservice j at node i is denoted by r(i; j ). The total capacity of node i is s(i). The feasible region of the deterministic subservice provision problem without rejections is then formulated in the following mathematical programming model: min
X (j)(d(j) ? X x(i; j))
j 2J
i2I
A. Tomasgard et al. /Modelling Aspects of Distributed Processing. .. s.t.
X r(i; j)z(i; j) +
j 2J
X x(i; j) jX x(i; j ) 2J
s(i);
11
i 2 I;
d(j ); j 2 J ; M (i; j )z (i; j ) ? x(i; j ) 0; (i; j ) 2 I J ; z (i; j ) 2 f0; 1g; x(i; j ) 0; (i; j ) 2 I J : P The term d(j ) ? i2I x(i; j ) in the objective is that part of the demand for subservice j i2I
that we have to reject. The rst set of constraints makes sure for each node that the sum of the xed resource use induced by installing subservices at the node and the demand met at the node do not exceed its capacity. The second set of constraints says that we can only process the subservice demands that we see. The third set of constraints ensures that the processing capacity at a node cannot be used to meet demand for a subservice, unless the subservice is installed there. The constants M (i; j ) in these constraints are upper bounds on the values for the corresponding variables x(i; j ), and the lowest value M (i; j ) can take is minfs(i) ? r(i; j ); d(j )g. For this deterministic mixed-integer problem a large class of facets of the feasible region is known [9]. The related feasibility problem (i.e., the problem of nding a feasible solution that satis es all demand, under the assumption that such a solution exists) has the structure of a Transportation Network with Supply Eating Arcs (TSEA) [10]. It is interesting to note that the deterministic feasibility problem of TSEA is NP -complete in the strong sense, see [10]. In [10] an algorithm is also given that solves the TSEA feasibility problem when r(i; j ) = r(j ). 3.4 Uncertainty
One of the main advantages o ered by the distribution transparencies of Section 2.2 is the possibility of dynamically changing the resource allocation. This ability to change is even more important when we try to consider uncertainty of demand in our models. In the treatment of uncertainty here and in the rest of the paper, we use terminology from stochastic programming. In a stochastic programming setting we assume that uncertainty in the demand for services can be modelled by making them random parameters with known probability distributions. In Section 3.1 we described how demand for services can be mapped to demand for processing capacity through their use of subservices. Thus, from the demand distributions for services we derive distributions for resource use generated by the subservices. Further discussion on this subject can be found in [38, 39]. At any time the state of the system of computers running services can be described by the current demand for processing capacity, a list of the running subservices, and a probability distribution of resource use generated by the di erent subservices at some future point in time. A very useful observation is that under most (ordinary) demand patterns for the services there will be no trouble meeting demand (otherwise, the investment problem is not properly solved). We are not interested in these cases. Instead, we will study
requests
A. Tomasgard et al. /Modelling Aspects of Distributed Processing. ..
12
time required to install a subservice
ev even cu ent t b rre id eg nt en ins co tifi nf ab ig u le in rati ad on eq ua ev te en te nd s
time required to shut down any subservice
time
Figure 3: Example of a peak in service demand situations in which unexpected peaks in demand for a single service occur. We assume that the time between these peaks is so large that we need only consider one service peaking at a time. After a peak has occurred, there is time to return to a robust state before a new peak might arrive. The conditional probability distribution concerning which service, if any, is going to peak in the next time period and the height of the peak, is based on observed trac patterns. The progress of a peak is depicted in Figure 3. Here we can see the relations between uncertainty and decisions as the peak develops. At some point in time an event begins and the service provider soon becomes aware of it (but not which service is peaking). As the peak evolves the service provider gains more information until he can identify the peak. We assume that at this point, due to relatively high setup times of installing subservices and release times of the xed capacity use after shutting down subservices, there is not enough time to change the con guration in order to meet the peak. The service provider must then rely on his prior decisions made under uncertainty. Under the above assumptions for the duration and distributions of peaks and for subservice lead times, the decision process can be modeled as a two-stage process. In the rst stage we decide which subservices to o er (install) on which computing nodes, having only probabilistic information on which service is going to peak. In the second stage uncertainty is resolved, and our only possible recourse action concerns what demand should be met using the subservices we installed in the rst stage. For an example see Figure 4 of Section 3.5. The time frame of the dynamic decision process will typically be
A. Tomasgard et al. /Modelling Aspects of Distributed Processing. ..
13
from milliseconds to minutes, depending on the lead time for setting up services and the duration of demand uctuations. In this paper we will model the case of discrete distributions, described in terms of scenarios [19]. The above discussion indicates that the individual scenarios typically only show peaks in a few of the subservices, depending on which service peaks in that scenario. Some of the subservices may peak in all scenarios, but some of them will also only be signi cantly a ected if one speci c service peaks. So, in general, our distribution is such that the di erences between scenarios can be large. If not, uncertainty would probably not be important. 3.5 Subservice arrangement under uncertainty
Now we turn to the mathematical model incorporating the two-stage dynamic decision process described above. We base the problem formulation on the deterministic model in Section 3.3 and the above discussion. In the stochastic case prices can be uncertain as well as demand. Let ~ (j ) be the random cost of rejecting demand for the resource created by subservice j . Random demand for subservice j is denoted by ~(j ). Here we give an example of a two-stage formulation of the stochastic service provision problem. Minimize Q(z ) X s:t: r(i; j )z (i; j ) s(i); i 2 I ; j 2J
z (i; j ) 2 f0; 1g; (i; j ) 2 I J ; where the constraint indicates that the node capacity limits how many subservices can be installed in a node, and Q(z ) = E [q(z; ; )]: The expectation is taken over the stochastic parameters ~ and ~, and q(z; ; ) = min s.t.
X (j)[(j) ? X x(i; j)]
j 2J
i2I
X x(i; j) s(i) ? X r(i; j)z(i; j) = s (i); z jX j x(i; j ) (j );
2J
2J
i 2 I;
j 2 J; x(i; j ) M (i; j )z (i; j ); (i; j ) 2 I J ; x(i; j ) 0; (i; j ) 2 I J : The second-stage problem is a bipartite transportation problem with cost coecients only on the arcs used to reject demand. The constraints mean the same here as in the deterministic problem in Section 3.3, but the z -variables are now xed and moved to the right-hand side to indicate this. The structure of the objective function can be utilized when solving the model, for example as in [36]. The rst stage is a binary knapsack problem [26] with several independent knapsacks and a convex objective. i2I
A. Tomasgard et al. /Modelling Aspects of Distributed Processing. ..
14
One should also note that the rst-stage constraints are redundant, since they are represented in a stronger form in the second stage. On the other hand, since the rststage constraints imply sz (i) 0 8i 2 I , the model can be regarded as a stochastic integer programming model with relatively complete continuous recourse (see for example [19, 37]). The following example indicates why uncertainty, rejections and dynamics must be included in the model. Example 1: The e ect of uncertainty on decisions
Assume that we have only one node with a capacity of 200 resource units. We have three subservices (A; B; C ), with deterministic rejection costs = (40; 31; 30) and xed resource usage r = (40; 20; 10). Uncertain demand is described by two possible future scenarios, S 1 with S 1 = (80; 0; 70) and S 2 with S 2 = (0; 150; 70). The two scenarios have equal probability of occurring. Let us rst try to look at this from the point of view of deterministic analysis. The optimal solution if we know ahead of time that scenario S 1 will occur, is z 1 = (1; 0; 1) and x1 = (80; 0; 70). The rejection cost is 0. The optimal solution if we know that S 2 will occur is z 2 = (0; 1; 1) and x2 = (0; 150; 20). The rejection cost is 1500, due to partial rejection of subservice C . Remember that in reality the demands are not known when subservices are installed. The two solutions above therefore give no idea about the solution to this stochastic dynamic problem, because in one case subservice A is installed and in the other subservice B . Next, we consider using expected demand = (40; 75; 70) as data, thus treating the problem as if it was deterministic. The optimal deterministic solution (z ; x ) is then to install subservices B and C , z = (0; 1; 1), and set x = (0; 75; 70) with a rejection cost of 4040 = 1600. It follows from Jensen's inequality (see for example [19]) that this rejection cost gives a lower bound to the expected rejection cost of our actual stochastic problem. The interesting part of this solution is only its rst-stage decision z . In practice, we will minimize the actual rejection costs of the scenario that occurs. This decision process is shown in Figure 4. Thus, to nd the expected cost of the rst-stage solution z , for each scenario we must nd the minimum rejection cost given that only subservices B and C are installed. If scenario S 1 occurs we meet the entire demand for subservice C , hence xS 1 = (0; 0; 70), and if S 2 occurs we choose to meet the entire demand for subservice B and as much as possible for subservice C , so that xS 2 = (0; 150; 20). It follows that the true expected cost resulting from the rst-stage solution of the deterministic model is 0:5 80 40 + 0:5 50 30 = 2350. Obviously, this is an upper bound for the expected cost of the optimal solution of the stochastic model. All of the choices the service provider has, together with rejection costs are given in Table 1. It can easily be seen that the best expected rejection cost is given by installing all three subservices, leading to an optimal cost of 1660. As we can see the solution we found from deterministic analysis in this case is not optimal. Its expected rejection cost is 42% higher than the optimal one. Here the possibility of a peak in demand for subservice A and the resulting high rejection cost in this scenario, more than outweigh the expected marginal value of any alternative use of the resource units involved. This was not discovered through deterministic analysis.
A. Tomasgard et al. /Modelling Aspects of Distributed Processing. ..
S1 z
xS1
~ δ S2
set up subservices
15
demand revealed
xS2
meet demand
time
Figure 4: The two-stage decision process of our example.
Table 1: All possible rst-stage solutions with their expected rejection costs.
xS 1 ( 0; 0; 0) A (80; 0; 0) B ( 0; 0; 0) C ( 0; 0; 70) A; B (80; 0; 0) A; C (80; 0; 70) B; C ( 0; 0; 70) A; B; C (80; 0; 50)
installed
( ( ( ( ( ( ( (
expected xS 2 rejection cost 0; 0; 0) 6025 0; 0; 0) 4425 0; 150; 0) 3700 0; 0; 70) 3925 0; 140; 0) 2255 0; 0; 70) 2325 0; 150; 20) 2350 0; 130; 0) 1660
A. Tomasgard et al. /Modelling Aspects of Distributed Processing. ..
16
3.6 Extensions of the models
In the previous models we implicitly assumed that the service provider had full knowledge of the sizes of all the available computing nodes and the ability to utilize this. We can imagine that, in some cases, the network provider who owns the computers on which the subservices are running, is free to replicate and migrate the subservice instances running on his infrastructure. In hiding the complete infrastructure from the service provider using migration and replication transparencies, he is completely free to utilize his resources. The service provider sees the infrastructure only as one single node. Note that the models presented in this paper then describes the network provider's problem. The corresponding service provider's problem will have only one computing node, and is discussed in [38, 39]. It may also be useful to model rejection of requests for services. In other words we then model a situation where the service provider is, or wants to be, the only provider of all or some of the subservices constituting a service. In case there are no customer rejections, there is no di erence from the earlier models. Models representing the problem with service rejections can be found in [38], where also multi-stage models and other variants of the service provision problem are discussed.
4 Node location problem At the operational level we have seen that the distributed framework of Section 2 enables us to both dynamically reallocate resources at the nodes and to distribute the processing in a region. This clearly in uences the discussion of the network provider's strategic investments in the underlying infrastructure on which we run the services, namely the computing nodes. We will assume that processing capacity is the limited resource, that distribution transparencies are valid and that our networks are divided into nonoverlapping regions of the type discussed in Section 3.2. As an example of a problem at a strategic level we treat the problem of locating computing nodes (computers) in the network. In the node location problem we consider a xed set of candidate sites inside a region in our network. The purpose of the computers is to meet demand for processing capacity generated by the requests for services within the region. Each site has a computer of a given capacity, and there is an investment cost associated with installing it. In practice there may be several \sites" with the same geographical location, but with computers of di erent size, type and cost. Each site also has costs connected to o ering (sub)services to customers located at di erent places in the region. The network provider wants to locate the processing capacity within the region so as to minimize the cost of installing enough processing and link capacity to meet the overall demand. Obviously di erent choices of node sites can lead to di erent costs for meeting service demand, both as a consequence of investments made in the transportation infrastructure and as a consequence of di erent operating costs.
A. Tomasgard et al. /Modelling Aspects of Distributed Processing. ..
17
possible site e1
e5
i1
i2
e10
e8
e2
virtual link
e6 e9
i4 e4
e7
i3
e3
Figure 5: Possible computing node sites and locations of demand in a region of the network. 4.1 Transparencies, link capacities and demand
First we need to de ne customers and their locations in the region. Imagine that the locations of customers correspond to the possible sites for the computing nodes in Figure 5. Subservices are often requested as an interaction between two or more customers, possibly at di erent locations. In Section 2.1 we explained that all pairs of computing nodes are connected by virtual links. We let these virtual links between the possible sites of the computing nodes as well as the sites themselves correspond to the \locations" of the requests for subservices. This is shown in Figure 5 where demand locations are indicated with the letter e and possible sites with i. Note that many subservices bring about interactions between more than two locations. In this case, the request would contribute to demand for the subservice over all links included in the request. It is important to see how the technological framework we de ned in Section 2 in uences investments. We are free to install the processing capacity where we want in the region, because assumption A1 in Section 3.2 says that the location of a subservice inside a region should not in uence the QoS experienced by the customer. The distribution transparencies hence indicate that it is possible to deliberately invest in processing capacity in such a way that there is a shortage of capacity in one part of the region to be met by a surplus in other parts. Because demand is uncertain this is one of the most important issues when we are discussing investments in distributed networks. In general the uncertain demand is not equally distributed over the region and not necessarily correlated in di erent parts of the region. The exibility we experienced in the distributed framework at the operational level is now carried along to deal with uncertainty at the tactical/strategic level when discussing investments. Although we are free to install processing capacity wherever we want, this does not mean that the operational and investment costs are independent of these locations. The
exibility at the operational level when it comes to resource allocation at the nodes as described in assumption A1, induces a requirement for investments in transportation infrastructure in assumption A2. Over- or under-investing in node capacity in parts of the region implies extra information ows between the demand locations and the computing nodes. The e ect of this must be considered when the network provider dimensions the
A. Tomasgard et al. /Modelling Aspects of Distributed Processing. ..
18
underlying transportation infrastructure. We will now at the tactical/strategic level try to minimize this location dependent cost of transportation infrastructure together with the investment cost in nodes. There are at least two di erent ways of including location dependency in the model:
We can explicitly model the links between customer locations and their capacity
and instalment costs. In the price of meeting demand for processing capacity at a location from a given site, we include the cost of the necessary transportation infrastructure generated by this choice.
The last approach is selected because there is an enormous amount of e ort already put into models for transportation network design. Given the location of sites, their sizes, and the type/amount of demand we expect at the sites, the problem of designing the underlying transportation network is extensively treated in the literature, see for example [13, 15, 16, 28, 32, 35]. The extra investment costs of the local transportation infrastructure coming from installing a node at a speci c site, can be split in two. A xed cost depending on whether a node is built or not and \operating costs" depending on the actual usage of the node| the demand met there. The xed part of investments in local transportation infrastructure is best included in the investment cost for the node. For every computing node the price of meeting demand for processing capacity generated at a given location contains operating costs of both the node and the underlying transportation infrastructure. 4.2 Uncertainty
Demand for processing capacity is uncertain. At the di erent demand locations we consider aggregated demand for processing capacity over customers and subservices. It is not feasible to retain the identity of the subservices generating the demand when we allocate it to nodes at the tactical/strategic level. One reason is the uncertainty about which subservices will actually exist. Another equally important reason is, as indicated in Section 3, that at the operational level distribution transparencies allow us to allocate subservices to the computing nodes dynamically. At the tactical/strategic level it is therefore the amount of demand for processing capacity that is allocated to a node that matters, not which subservices that generated it. Estimates for the capacity used just to install and provide di erent subservices must be contained in the demand for processing capacity. This is in addition to customer demand. Other uncertain factors are the prices of meeting demand at di erent locations from di erent computing nodes and rejection costs. We assume that all these uncertain parameters are only known in distribution when investment decisions for processing capacity are made. It is natural to think that over time, when new insights in demand patterns become available, the investments can be modi ed utilizing this new information. The decision process is then typically multi-stage. In other cases the investment decision can be of the type \now or never", and the only recourse action is to utilize the equipment when it is installed. Then a two-stage model will capture the decision process. In this paper we treat the two-stage model. It better
A. Tomasgard et al. /Modelling Aspects of Distributed Processing. ..
19
illustrates all the technological aspects discussed above since it is less complex than the multi-stage model. Earlier we showed how demand uncertainty in uences decisions in the service provision problem. In particular we stressed the negative e ect of not utilizing exibility when decisions are made. When it comes to dimensioning processing capacity and estimating the need of link capacity at a tactical/strategic level, long term demand-uncertainty is important. Flexibility from utilizing the distributed framework, cannot be captured by deterministic analysis. In recognition of the importance of uncertainty for these types of decisions, we will therefore go directly to a stochastic model. 4.3 The node location model
The problem can be formulated as a sort of capacitated facility location model. [12, 21] give overviews of di erent formulations for this problem and contain a lot of references. In our model the computers are the facilities, the virtual links between them are the demand locations, and demand for the subservices on the links is aggregated to nd the corresponding demand for processing capacity. The demand should be met to minimize cost subject to capacity constraints on the nodes. For simplicity of notation we assume that no nodes exist already. Let y(i) be 1 if a computer i is installed and 0 otherwise. The corresponding xed investment cost is f (i). It is natural to assume that the capacity of the computers is limited. The constant s(i) is the capacity of computer i. Also here we will assume that it is possible to reject requests for (sub)services. Therefore, it is reasonable to assume that the network provider will set a minimum total capacity of his computers in a region, which we denote by sl . He might also like to set a maximum capacity in a region as argued above, which we denote by su be the maximum. The variable x(i; e) is the demand for processing capacity at demand location e which we decide to meet at node i. The set of demand locations (links) is E . The demand for processing capacity at link e is ~(e). We denote the possibly stochastic cost of meeting the demand from location e at node i as ~(i; e). This parameter should include the operating costs from installing or renting infrastructure in the transportation network. In this formulation we have assumed that the costs are linear in the use of processing capacity. The variable t(e) describes how many resource units demanded at location e we choose not to meet. The rejection cost is given by ~ (e). The corresponding stochastic two-stage model can be written as follows. First stage: X Minimize f (i)y(i) + Q(y) s.t.
i2I
sl
X s(i)y(i) s ; i2I
u
y(i) 2 f0; 1g; 8i 2 I : where the constraint limits how much capacity can be installed in the region, and Q(y) = E [q(y; ; ; )]; where the expectation is taken over the stochastic parameters ~ ; ~ and ~.
A. Tomasgard et al. /Modelling Aspects of Distributed Processing. ..
20
The second stage problem is then
q(y; ; ; ) = min s.t.
X X (i; e)x(i; e) + X (e)t(e) i2I e2E
X x(i; e) + t(e) = (e); i X x(i; e) s(i)y(i); 2I
e2E
e2E
8e 2 E ; 8i 2 I ; 8(i; e) 2 I E :
x(i; e); t(e) 0; Here, the rst set of constraints requires that demand at each location is either met or rejected. The second set of constraints ensures that demand is only met at a node if a computer was previously installed there and that the level of demand met is limited by the capacity of that computer. Some references for the stochastic facility location problem are [4, 11, 23, 24, 25, 22]. The formulation above is a stochastic integer programming model with binary rst stage and relatively complete continuous recourse, see for example [19, 37]. 4.4 Related models
The ideas presented here can be extended to strategic models where the purpose is to simultaneously plan the computing resources in all the di erent regions of the network provider's domain. The distributed framework presented in this paper may be used to deliberately plan capacities in such a way that there is a shortage in one region and a surplus in another. Trading of capacities between regions can be considered one of the main issues of investment planning in distributed networks. Strategic models including these aspects may for example give bounds on the minimal capacity to be installed in the individual regions. Such an approach is presented in [38, 39]. Another approach to a similar problem is presented in [20].
5 Conclusions and future work The models that we present in this paper are examples of how to deal with important decisions in telecommunication networks with distributed processing capabilities. They are based on a distributed telecommunications framework de ned by TINA-C[6, 33]. The distribution transparencies and the abstract infrastructure de ned here are crucial for models treating service provision and investment in a distributed processing setting. Under the assumption that processing capacity is the limited resource in our telecommunication network, we conclude that, in regions of the network, allocation of computer resources to services at an operational level may be made independently of the location of the demand. In requiring this, we utilize eciently the increased exibility for dynamically allocating resources in a distributed network. The models presented in this paper are above the transportation level of the network. They do not treat detailed design of the transportation network underneath, but rather include the relations between the distributed computing nodes providing services and the
A. Tomasgard et al. /Modelling Aspects of Distributed Processing. ..
21
actual data transported, through the prices of transportation capacity. Increased exibility in service provision clearly also in uences the decisions to be made for investments in infrastructure, both when it comes to processing and transportation capacities. Strategic and tactical decision models for distributed telecommunication networks therefore cannot be studied in isolation from models concerning dynamic resource allocation and the assumptions they are based on. A shift in focus at the operational level from transportation of data to processing of information is also bound to present a need for modelling new aspects when it comes to investments. There is no ecient general method for solving stochastic integer programs. As for deterministic integer programming, tailored algorithms and solution methods are necessary to solve the models. This is going to be the main area of work on the above models in near future. Also it is clear that a lot of work remains when it comes to describing the underlying stochastic features of our problems. Because these models also describe a reality and a functionality of networks that are not present today, there are, as yet, only limited data available. Work remains to be done both when it comes to de ning what data we should assume are available and how to collect the data.
References [1] S. Aggarwal, J. A. Garay, and A. Herzberg. Adaptive video on demand. Manuscript. [2] E.C. Anderson and N. Natarajan. A reference architecture for telecommunications operations applications. Journal of Network and Systems Management, 1(3), 1993. [3] J.A. Audestad. Lecture notes. Course 45356 Communication in distributed systems, Department of Computer Systems and Telematics, UNIT/NTH, 7034 Trondheim, Norway, 10.10.1995. [4] J. Balachandran and S. Jain. Optimal facility location under random demand with general cost structure. Naval Research Logistics Quarterly, 23:421{436, 1976. [5] A. Bar-Noy, J.A. Garay, and A. Herzberg. Sharing video on demand-constant competitive ratio with long noti cation time. In 3rd Annual European Symp. On Algorithms, LNCS (979), pages 538{553, Corfu, Greece, September 1995. Springer-Verlag. [6] W.J. Barr, T. Boyd, and Y. Inoue. The TINA initiative. IEEE Communications Magazine, pages 70{76, March 1993. [7] N. Chapman and S. Montesi. Overall concepts and principles of TINA. Document no. TB MDC.018 1.0 94, TINA-C, February 1995. [8] M. De Prycker. Asynchronous Transfer Mode, Solution for Broadband ISDN. Prentice Hall, New Jersey, 1995. [9] S. Dye, L. Stougie, and A. Tomasgard. Relaxations and valid inequalities for the service provision problem. In preparation. [10] S. Dye, A. Tomasgard, and S.W. Wallace. Feasibility in transportation networks with supply eating arcs. Networks, to appear. [11] P.M. Franca and H.P.L. Luna. Solving stochastic transportation-location problems by generalized Benders decomposition. Transportation Science, 16:113{126, 1982. [12] R.L. Francis, L.F. McGinnis, and J.A. White. Locational analysis. European Journal of Operational Research, 12(3):220{252, 1983. [13] A.A. Gaivoronski. Robust access planning for ATM network in the presence of uncertain multiservice demand. In NETWORKS`94 \Planning for a customer-responsive network",
A. Tomasgard et al. /Modelling Aspects of Distributed Processing. ..
22
September 1994. [14] B. Gavish and I. Neuman. Routing in a network with unreliable components. IEEE Transactions on Communications, 40(7):1248{1258, 1992. [15] B. Gavish, editor. Operations research in telecommunications. Annals of Operations Research, 36:402 pages, 1992. [16] B. Gavish, editor. Special issue on telecommunications systems: Modeling, analysis and design. Operations research, 43(1):29{191, 1995. [17] P. Graubmann, W. Hwang, M. Kudela, K. MacKinnon, N. Mercouro , and N. Watanebe. Engineering modelling concepts (DPE architecture). Document no. TB NS.005 2.0 94, TINA-C, 1994. [18] F. Halsall. Data Communications, Computer Networks and Open Systems. Addison-Wesley, New York, 1992. [19] P. Kall and S.W. Wallace. Stochastic Programming. Wiley, Chichester, 1994. [20] A. Kreidi and B. Sanso. Optimization of a geographically distributed air-ground airline telecommunication system. Technical report, GERAD publication G-94-47, 1994. [21] M. Labbe and F.V. Louveaux. Location problems. Discussion paper, Institut de Statistique, Universite Libre de Bruxelles, Belgium, July 1996. [22] G. Laporte, F.V. Louveaux, and L. Van Hamme. Exact solution of a stochastic location problem by the L-shaped algorithm. Transportation Science, 28:95{103, 1994. [23] L.J. Le Blanc. A heuristic approach for large scale discrete stochastic transportation-location problems. Computers and Mathematics with Applications, 3:87{94, 1977. [24] F.V. Louveaux. Discrete stochastic location models. Annals of Operations research, 6:23{34, 1986. [25] F.V. Louveaux and D. Peeters. A dual-based procedure for stochastic facility location. Operations research, 40:564{573, 1992. [26] S. Martello and P. Toth. Knapsack Problems: Algorithms and Computer Implementations. John Wiley & sons, 1990. [27] S. Mullender, editor. Distributed Systems. Addison-Wesley, New York, 1993. [28] P. Nain and K.W. Ross, editors. Stochastic modelling of telecommunication systems. Annals of Operations Research, 35:357 pages, 1992. [29] G.L. Nemhauser and L.A. Wolsey. Integer and Combinatorial optimization. WileyInterscience series in discrete mathematics and optimization. Wiley, New York, 1988. [30] G. Nilsson, F. Dupuy, and M. Chapman. An overview of the Telecommunications Information Networking Architecture. In TINA 95 Conference, Melbourne, Australia, P1, 1995. [31] R. Onvural. Asynchronous Transfer Mode Networks: Performance Issues. Artech House, Boston, 1994. [32] R.O. Onvural, H.G. Perros, and G. Pujolle, editors. Methodologies for high speed networks. Annals of Operations Research, 49:373 pages, 1994. [33] T. Rowbotham. The TINA Consortium { A Collaborative Way Forward. In TELECOM 95, Geneva, 1995. [34] H. Rubin and N. Natarajan. A distributed software architecture for telecommunication networks. IEEE Network Magazine of Computer Communication, January/February 1994. [35] Suvrajeet Sen, Robert D. Doverspike, and Steve Cosares. Network planning with random demand. Telecommunications Systems, 3:11{30, 1994. [36] L. Stougie, A. Tomasgard, and M.H. van der Vlerk. Some approaches for solving the stochastic service provision problem. In preparation. [37] L. Stougie and M.H. van der Vlerk. Stochastic integer programming. In M. Dell'Amico, F. Maoli, and S. Martello, editors, Annotated Bibliographies in Combinatorial Optimization, chapter 9, pages 127{141. Wiley, 1997. [38] A. Tomasgard, S. Dye, S.W. Wallace, J.A. Audestad, L. Stougie, and M.H. van der Vlerk. Stochastic optimization models for distributed communication networks. Working paper #3,
A. Tomasgard et al. /Modelling Aspects of Distributed Processing. ..
23
Department of Economics and Technology Management, Norwegian University of Science and Technology,7034 Trondheim, Norway, 1997. [39] A. Tomasgard, S. Dye, S.W. Wallace, J.A. Audestad, L. Stougie, and M.H. van der Vlerk. Modelling in distributed telecommunications networks. Tinbergen Institute Discussion Paper TI 97-030/4, 1997.
