Building Adaptive, Reflective Network Services By ... - Semantic Scholar

-1-

Middleware For Building Adaptive Systems Via Configuration Sanjai Narain Ravichander Vaidyanathan

Stanley Moyer William Stephens

Kirthika Parameswaran Abdul-Rahim Shareef

Telcordia Technologies 445 South Street Morristown, NJ 07960 {narain, vravi, stanm, wes, kirthika, shareef}@research.telcordia.com February 12, 2001

Abstract1 COTS (commercial off-the-shelf) devices are capable of executing powerful, distributed algorithms. Very large, adaptive systems can be created by simply integrating these devices, not by creating new devices, algorithms or external control systems. The principal integration method is configuration: every device is designed to have a finite set of configuration parameters that can be set to definite values. These parameters are static in that, unlike the device’s dynamic state, these do not change during the device’s normal operation. Inspite of the ubiquity of the configuration method, there is no “science” for it. There are no tools which let one specify global, end-to-end system functionality requirements at a high level, reason about these, and have these compiled into device configurations. On the other hand, it is inherently difficult to develop this science because of the large conceptual gap between global, end-to-end requirements and device configurations. A middleware is needed to bridge this gap in the form of specification, reasoning, compilation and diagnostic tools. This position paper sketches a technique called Service Grammar for developing such middleware for a given domain. It is based on the idea that algorithms for implementing end-to-end functional requirements can be expressed as the enforcement of a set of fundamental requirements. These are upon the existence of definite high-level relationships between configuration parameters of various devices. We can collect these into a Requirements Library. By mixing and matching items from it we can define algorithms for implementing a very wide range of end-to-end functional requirements. This Library can also simplify design of the reasoning, compilation and diagnostic tools. This plan is illustrated by analyzing how realistic service requirements are implemented in a Virtual Private Network via configuration steps. First, a network of “resilient”, secure tunnels is set up by composing capabilities of protocols OSPF, GRE and IPSec. Then, quality of service guarantees are established by composing capabilities of MPLS, RSVP and DiffServ. Finally, as an example of configuring middleware components themselves, an example is presented from CORBA security. 

Copyright 2001 Telcordia Technologies, Inc. This paper appeared in Proceedings of ACM SIGPLAN Workshop on Optimizing Middleware, Salt Lake City, UT, June 18, 2001.

1

-2-

1. Introduction COTS devices, including middleware components, are capable of executing powerful, distributed algorithms. Very large, complex systems can be created by integrating these devices. The principal integration method is configuration: every device is designed to have a finite set of configuration parameters that can be set to definite values. These parameters are static in that, unlike the device’s dynamic state, these do not change during the device’s normal operation. No additional distributed algorithm or control software is developed. Rather, the capability of existing algorithms is harnessed. For example, a financial services company’s network can have thousands of routers, yet complex, end-to-end connectivity, security and performance services are created on this network purely via configuration steps2 [C]. Interestingly enough, the configuration method also allows one to build adaptive systems by composing adaptive capabilities of constituent distributed algorithms. It is not necessary to build an external control system. COTS devices executing such algorithms monitor their behavior and if they detect errors, take corrective action. Examples are SONET’s self-healing rings, MPLS’ fast restoration of Labeled Switched Paths, TCP’s reliable delivery of IP packets, and OSPF’s updating of routing tables based on network link states. Inspite of the ubiquity of the configuration method, there is no “science” around it. Systems designers develop definite plans for creating global, end-to-end functionality. These plans contain high level abstractions associated with distributed algorithms, relationships between these, and operations upon these. However, there are no tools to let designers formally express such plans, reason about and refine them, and have these be executed automatically to generate device configurations. If for some reason the end-toend functionality becomes unavailable there are no tools to automatically diagnose configuration errors. Instead, designers write down such plans on paper, and manually perform such reasoning, refinement, configuration generation and diagnosis. In the absence of such a science of configuration, practice of the configuration method is highly inefficient and costly. For example, a Yankee Group [YG 98] report estimates that 45% of the operations cost of a WAN is due to device configuration. A government report [LSN01] estimates that 20% of personnel in a rapid insertion force are engaged in setting up, operating and maintaining the network. If we make the assumption that router administration costs are of the same order of magnitude as router capital costs, annual router configuration costs are many billions of dollars. However, it is inherently difficult to develop such a “science” of configuration. There is a large conceptual gap between global, end-to-end functional requirements and device configurations. Realistic systems can have a large number of devices each with many configuration parameters and possible values. Even visualizing dependencies between requirements and device configurations, at and across multiple layers of abstraction is a 2

Note that network administrators are typically not software developers, yet the configuration method enables them to create large, complex services.

-3-

formidable task. Configuration error diagnosis is difficult because it requires global reasoning. In the presence of configuration errors, devices continue to function correctly in isolation, but they don’t work together. Diagnosing why the devices don’t work together involves global reasoning which is inherently difficult. This paper sketches a technique called Service Grammar for developing such middleware for a given domain. It is based on the idea that algorithms for implementing end-to-end functional requirements can be expressed as the enforcement of a set of fundamental requirements. These are upon the existence of definite high-level relationships between configuration parameters of various devices. We can collect these into a Requirements Library. By mixing and matching items from it we can define algorithms for implementing a very wide range of end-to-end functional requirements. This Library can also simplify design of the reasoning, compilation and diagnostic tools. This plan is illustrated by analyzing how realistic service requirements are implemented in a Virtual Private Network via configuration steps. Note that these problems are not those of standardizing device interfaces. What is needed is a middleware for bridging the large conceptual gap between global, end-to-end functional requirements and device configurations. Section 2.1 presents an example of a network of “resilient”, secure tunnels created by composing the capabilities of protocols OSPF, IPSec and GRE. Traffic flows over this secure network, but if a tunnel fails, traffic is redirected to ensure that it continues to flow over this network. Section 2.2 shows how to add quality of service assurance to this network by composing capabilities of MPLS, RSVP and DiffServ. Section 2.3 presents a similar example from CORBA security service. Section 3 sketches the Service Grammar technique and Section 4 contains a summary.

2. Creating Adaptive Systems via Configuration We now provide several examples of how complex adaptive services can be created purely via configuration steps. We show that definite algorithms are designed using highlevel abstractions and operations upon these. Yet, there is no system which can take these algorithms as input and compile them down into device configurations. This compilation is manually performed today.

2.1 A Network of resilient, secure tunnels. This example shows how to integrate capabilities of OSPF, IPSec and GRE protocols to create a network of “resilient”, secure tunnels. All data flows only over these tunnels, but if some tunnels fail, data is rerouted in such a way that it still flows only over secure tunnels. The most important point is that all this can be ensured purely via configuration of routers. In particular, the “adaptiveness” of the resulting system is obtained by exploiting the adaptiveness built into the constituent protocols. No new software, protocols or centralized control system are created.

-4-

Dedicated Line 3

OSPF

OSPF

CE 1

CE 2

CE: Customer Edge Router Dedicated Line 1 Dedicated Line 2

CE 3 OSPF

Figure 1. Gateway Routers Connected Via Dedicated Lines As shown in the above figure, suppose a company has multiple locations around the world and gateway routers at each location are linked via private, circuit-switched lines. The network of these lines need not form a full mesh (although this network is). Since these are dedicated, a certain level of security is ensured. Also, a routing protocol such as OSPF can be made to run at the gateway routers which would ensure that traffic is routed from any gateway to any gateway if a path exists. OSPF will also ensure resiliency. If a link were to fail, a new path from every source to every destination would be calculated.

-5-

IPSec Tunnel 3 OSPF OSPF

CE 1

CE 2

IPSec Tunnel 1

ISP Network

OSPF does not work over IPSec tunnels. These are not point to point links.

IPSec Tunnel 2

CE 3 OSPF

Figure 2. Simple Replacement of Dedicated Lines with IPSec Tunnels Confuses OSPF Now suppose the company were to “modernize” and decide to replace its private lines with IPSec tunnels over their ISP’s shared infrastructure. The tunnels would ensure strong security. However, routing and resiliency would be seriously affected. IPSec tunnel end points would no longer belong to the same IP subnet so they would no longer be recognized as immediate neighbors by OSPF. Thus, traffic would no longer be automatically routed from one tunnel to the next. To “hook” the tunnels together would require some form of static routing. But static routing neither scales with the number of gateways, nor does it ensure resiliency. If a link fails, alternative routes are not automatically computed.

-6-

IPSec over GRE tunnel 3

OSPF

OSPF

CE 1

CE 2

IPSec over GRE tunnel 1

CE: Customer Edge Router ISP Network

IPSec over GRE tunnel 2 OSPF works fine. GRE tunnels are point to point links. These are secured with IPSec.

CE 3 OSPF

Figure 3. IPSec over GRE is Required To Make OSPF Work Again Fortunately, an interesting solution exists. One first replaces the dedicated lines between gateway routers with GRE tunnels. On each router, new GRE interfaces are declared. Both end points of a GRE tunnel do belong to the same IP subnet. Then, one sets up an IPSec tunnel over the GRE tunnel. Now, OSPF again discovers GRE tunnel end points as immediate neighbors and thus works unchanged. In particular, it ensures routing and resiliency. Conceptually, all traffic between gateways is routed only over the secure network of GRE tunnels. Care has to be taken to avoid routing loops. GRE tunnels are abstractions which are implemented on top of the ISP’s physical infrastructure, so the physical path from one GRE tunnel end point to another should not be the tunnel itself. This is done by keeping the overlay network of GRE/IPSec tunnels strictly separate from the ISP’s network. A separate OSPF process is used to allow gateway routers to communicate with their ISPs. This process is unaware of the overlay network. This “algorithm” can be summarized as the enforcement of the following high-level requirements:

-7-

Requirements To Enforce To Build Above System 1. There exists a GRE tunnel between each pair of gateway routers linked earlier via a dedicated line. 2. Each GRE tunnel is protected via IPSec. 3. A new OSPF process is defined on the gateway routers for discovering routes only in the network of GRE tunnels and in the LANs sitting behind customer routers. The ISP network is invisible to this process.

However, this algorithm cannot today be input into any system today and be “compiled” into device configurations. The compilation is manually performed by experienced network administrators. The table below contains sample router configuration parameters which are set to definite values to enforce the above requirements:

Sample Router Configuration Parameters Category IP Subnetting OSPF

IPSec

GRE

Configuration Parameter 1. 2. 1. 2. 3. 4. 1. 2. 3. 4. 5. 6. 7. 1. 2.

IP address Subnet mask Area Stub identifier Process ID Interfaces on which active Encryption algorithms Hash algorithms Tunnel peers Traffic filters Authentication mode Authentication peers Preshared keys Tunnel interfaces Tunnel peers

-8-

2.2. Adding Quality of Service Assurance We now examine how to satisfy end-to-end Quality of Service requirements for the above company’s Virtual Private Network. We show how to do this by exploiting the capabilities of the DiffServ [BDCDZW98] and MPLS [DMAOM99] [RVC01] protocols, via configuration steps. Again, the adaptiveness of the resulting system is obtained by composing the adaptiveness built into these protocols. The service contract with each customer specifies bounds on the traffic rates in each class of service that the customer is allowed: voice, premium data and best-effort data. From knowledge of the aggregate capacity in the network between any two points of customer attachment, the Service Provider can then provision for the customer demand for voice and premium data services. Over-subscription of network capacity is made possible by the best-effort class. For instance, if the aggregate capacity between two points of customer attachment is 2 Mbps, the service provider may enter into a service contract of 2.2 Mbps with the customer (say 0.5 Mbps for voice, 1 Mbps for premium data and 0.7 Mbps for best-effort), thus hoping for a statistical multiplexing gain of 10%. Traffic belonging to different classes is marked at customer routers before entering the Service Provider’s network. These classes are defined using packet filters that use the five-tuple of fields in an IP packet header. These filters are configured on every customer edge router (CE)+. To ensure that the customer stays within the specified bounds, policing mechanisms such as Cisco’s Committed Access Rate (CAR) are configured at the ingress to the Service Provider network (PE routers). Policing is accomplished on a per-class, per-customer basis, and needs to be implemented in a consistent fashion with the marking and classification functions. Appropriate scheduling and packet discard mechanisms are then configured on the routers in the Service Provider network (PE and R) in order to provide the requisite class of service for voice, premium data and best-effort data, e.g. Weighted Fair Queuing (WFQ) and Random Early Detection (RED).

+

An underlying assumption in the depicted model is that the Service Provider has management control over the CE router. This corresponds to a common business model where the management of the CE router is also outsourced by the Enterprise customer to the Service Provider

-9-

OSPF OSPF

CE 1

CE 2 R

R

Demand: 0.5 Mbps

PE

ISP Network

PE

Capacity: 1 Mbps Demand: 1 Mbps

R

CE: Customer Edge Router PE: Provider Edge Router P: Provider Core Router

CE 3 OSPF

Entire Demand Routed Over Shortest Path IGP route. Customer’s service contract is violated.

Figure 4: Customer’s service contract is violated even with DifServ Now assume that in Figure 4 the instantaneous aggregate demands on customer sites 1 and 3 towards customer site 2 is 1.5 Mbps as shown. Further the capacity of all physical links in the network is 1 Mbps. Thus, the aggregate capacity of the network between the points of customer attachment is 2 Mbps. However, due to shortest path IGP routing, all the traffic is routed over the lower half of the ISP’s network, resulting in overloading of the links and violation of the customer’s service contract. Such an imbalance in the routing of traffic can be corrected by the deployment of an explicitly routed MPLS TE tunnel between the points of customer attachment over the upper half of the ISP’s network. Part of the customer demand can then be routed statically into the MPLS tunnel, load balancing the ISP network. In the depicted example, customer 1’s traffic is directed via the MPLS tunnel, while customer 3’s traffic follows the shortest IGP path between the customer’s points of attachment. In this fashion, MPLS traffic engineering can be used to optimize the use of network bandwidth resources. Resources can also be reserved in MPLS tunnels by means of RSVP-TE to assure a lower bound on the quality of service. Backup MPLS tunnels can be configured, with or without resource reservation, in case the original tunnel fails. Thus, the quality of service can automatically degrade in stages from guaranteed service to best effort in case of network anomalies.

- 10 -

OSPF OSPF

CE 1

CE 2

0.5 Mbps

R

R

Demand: 0.5 Mbps

PE

ISP Network

PE

MPLS Tunnel

Demand: 1 Mbps 1 Mbps

R

CE 3 OSPF

Part of the demand is routed over the explicitly configured MPLS tunnel. Customer’s service contract is met.

Figure 5: MPLS TE allows the ISP to meet the customer’s service contract We now summarize the algorithm for provisioning this complex, network service as enforcement of the following high-level requirements: 1. Customer traffic at every CE router is classified, marked, and policed according to service contracts with end customers. 2. Scheduling and discard algorithms on every router in the service provider network with relative weights/priorities between the various classes of service. 3. MPLS traffic engineering tunnels for load-balancing are established between every pair of provider edge routers. Again, there is no system which can take this algorithm as input and compile it into device configurations. Rather, the compilation is performed manually.

- 11 -

A sample of the parameters required to setup the service is depicted below: Category Classification and Marking

Policing

Scheduling and Discard

MPLS Traffic Engineering

Configuration Parameter 1. 2. 3. 1. 2. 3. 4. 1. 2. 3. 4. 1. 2. 3. 4.

IP traffic filters DiffServ codepoints Association of codepoints to filters Mean rate Allowed burst size Non-conformance action (discard/remarking) Re-marking codepoint Scheduling weights per class Buffer dimension per class Discard thresholds Discard probability Tunnel interface Tunnel IP address Explicit path for tunnel Static route for traffic

2.3. CORBA Security This example shows how the configuration method is used to integrate “software” components as well. CORBA applications requiring security (e.g., authentication, authorization, message integrity, and privacy) can have their requirements met at the link layer, network layer, and/or application layer (i.e., middleware). For example, wireless link layer security could be available in a wireless network, IPsec tunnels could be created at the network (IP) layer, and security middleware like the CORBA Security Service can provide application layer security. At all these layers, the specific quality of protection" for the security is configurable. The CORBA Security Service is typically implemented as a distributed service that consists of an authentication server, an authorization server (i.e., policy manager), and CORBA interceptors on both the client object and server object platforms — see Figure 4 below for a depiction of these (and other) components. The desired quality of protection for the method call on a CORBA object is specified in the policy manager (e.g., authorization server). The link, network, and application layers must then all be configured to provide the appropriate levels and kinds of security.

- 12 -

CORBA Security Service Components

Authentication Server

Client Application ORB Core interceptor authentication integrity privacy IIOP SSL/GSSAPI TCP IP IPsec link layer

Authorization Server

CORBA Name Service

Server Application ORB Core interceptor authentication integrity privacy IIOP SSL/GSSAPI TCP IP IPsec link layer

CORBA Security Service Components

Figure 5. Configuring Components To Provide CORBA Security Service The above security services can be implemented, via configuration steps, by enforcing the following constraints: 1. the client interceptor must be able to communicate securely with the authentication servers (so it must have the right security "keys" for authentication and encryption and it must know the "address" of the authentication server 2. the server interceptor must be able to communicate with the authorization server (like the bullet above it must have the correct "key" and know the address) 3. The integrity and privacy functions must have the right encryption and "signature" keys 4. The SSL/GSSAPI layer must be configured correctly or communication will fail. 5. The IPsec layer (i.e., hosts and any participating routers) must be configured correctly to enable communications 6. Link layer security must be properly configured. 7. The access policy in the authorization server must be correct (enable access to the object and method for the calling client) or access will be denied. 8. Different authentication mechanisms for client object authentication are supported (e.g., public key, username/password, Kerberos) and if any information here is configured incorrectly (e.g., the wrong certificate) then the client will not have access to the desired service 9. the client object has to locate (the address) of the intended server object through a name server (CORBA Naming Service) or some other mechanism (e.g., a config file).

- 13 -

3. Overview of Service Grammar

End-To-End Functional Requirements

Middleware Decompose Proof Engine

Configure (Big Leap!)

Requirements Library

Diagnosis Engine

Requirements Compiler

Devices

Figure 5. Service Grammar Middleware Architecture As shown in the Figure above, there is a large conceptual gap between end-to-end functional requirements and device configurations. To bridge this gap for a particular application domain, a middleware is created using the Service Grammar technique with the following modules: 1. A Requirements Library containing definite relationships between configuration parameters of devices. An algorithm for implementing an end-to-end service can be defined as the enforcement of requirements in the Library. 2. A Diagnosis Engine for checking if a requirement in the Library is true given particular configuration settings of devices in a system. 3. A Requirements Compiler for changing configuration settings of devices in a system to enforce a given requirement in the Library. 4. A Proof Engine for checking if a conjunction of Library Requirements satisfies an end-to-end requirement.

- 14 -

We now outline how these modules can be developed. 3.1 Requirements Library We have shown in the previous examples that definite high-level algorithms are created on paper to implement end-to-end requirements. These algorithms take the form of requirements that need to be enforced via configuration steps. If we can create a fundamental Library of such requirements, these can be mixed and matched to create algorithms to implement a very wide range of end-to-end requirements. The question is what is this Library? 3.2 Diagnosis Engine The Diagnosis Engine checks whether a Library requirement is true given particular device configurations. Now, to check whether a system wide requirement is true given particular device configurations, express it as a collection of Library requirements, then call the Diagnosis Engine recursively. 3.3 Requirements Compiler The Requirements Compiler compiles requirements into device configurations. 3.4 Proof Engine No matter how good the specification language, there is always the question of whether the specifications themselves are correct. For example, program correctness is an important question even when programming languages are highly expressive such as Prolog or Lisp. The Proof Engine is fundamentally different from the Diagnosis Engine. The latter checks if a collection of Library requirements are true, given device configurations. The former checks if the collection itself has useful properties. Examples of questions the Proof Engine could answer are: 1. Is connectivity between A and B possible? The Diagnosis Engine could check that a routing protocol architecture has been set up via device configurations. However, the architecture itself may be flawed, leaving out A’s subnet entirely, for example. 2. Assuming there is connectivity between A and B, is every segment in the path between A and B protected? Different segments may be protected at different layers e.g., SSL at the application layer, IPSec at the network layer, and WEP at the wireless link layer. 3. Assuming the entire path between A and B is protected, is a certain QoS assured? QoS requirements at the application layer may be inconsistent with those at the network (Diffserv) layer and those at the link layer. Furthermore, security protocols may add substantial overhead.

- 15 -

4. Summary COTS devices are capable of executing powerful, distributed algorithms. Very large, adaptive systems can be created by simply integrating these devices, not by creating new devices, algorithms or external control systems. The principal integration method is configuration. Inspite of the ubiquity of the configuration method, there is no “science” for it. Developing this science is inherently difficult because of the large conceptual gap between end-to-end functionality requirements and low-level device configuration steps. A middleware is needed to bridge this gap in the form of specification, reasoning, compilation and diagnostic tools. This paper sketches a technique called Service Grammar for developing such middleware for a given domain. It is based on the idea that algorithms for implementing end-to-end requirements can be expressed as the enforcement of a set of fundamental requirements in a Requirements Library. These requirements are upon the existence of definite highlevel relationships between configuration parameters of various devices. This Library simplifies the design of the reasoning, compilation and diagnostic tools. This plan is illustrated in the context of a Virtual Private Network with complex end-to-end requirements on connectivity, security, performance and adaptiveness.

References [BDCDZW98] S. Blake, D. Black, M. Carlson, E. Davies, Z. Wang, W. Weiss, An Architecture for Differentiated Services, RFC 2475, December 1998. [C] Cisco Systems. A Primer for Implementing a Cisco Virtual Private Network. http://www.cisco.com/warp/public/cc/so/neso/vpn/vpne/vpn21_rg.htm. [CN99] Cohen, E., Narain, S. Temporal logic. Encyclopedia of Electrical And Electronics Engineering, volume 21, ed. John Webster. John Wiley & Sons, 1999. [DH99] Doraswamy, N., Harkins, D. IPSec: The New Security Standard For The Internet, Intranets and Virtual Private Networks. Prentice Hall, 1999. [DMAOM99] D. Awduche, J. Malcolm, J. Agobua, M. O'Dell, J. McManus, Requirements for Traffic Engineering for Traffic Engineering for MPLS, RFC 2702, September 1999. [H95] Huitema, C. Routing in the Internet. Prentice Hall, Englewood Cliffs, New Jersey, 1995. [LSN01] Workshop on New Visions for Large-Scale Networks: Research and Applications. Interagency Working Group for Information Technology Research and Development, Virginia, VA, March 12-14, 2001. http://www.ccic.gov/iwg/pca/lsn/lsnworkshop-12mar01/

- 16 -

[NSR01] Narain, S., Shareef, A., Rangadurai, M. Diagnosing Configuration Errors in Virtual Private Networks. Proceedings of IEEE International Communications Conference, Helsinki, Finland, 2001. [RVC01] E. Rosen, A. Viswanathan & R. Callon, Multiprotocol Label Switching Architecture, RFC 3031, January 2001 [SKKK00] Schmidt, D., Kachroo, V., Krishnamurthy, Y., Kuhns, F. Developing Next Generation Distributed Applications With QoS-Enabled DPE Middleware, IEEE Communications Magazine, Vol. 38, No. 10, October 2000. [WBPMRCY98] Whetten, B., Basavaiah, M., Paul, S., Montgomery, T., Rastogi, N., Conlan, J., Yeh, T. “The RMTP-II Protocol,” Internet Draft, draft-whetten-rmtp-ii00.txt, dated April 1998. [YG98] IVPN Service Backbones: The Operations Cost Angle. Data Communications Report Vol. 13, No. 19, Yankee Group, December 1998