High Performance Web-server Systems

High Performance Web-server Systems V. Cardellini, M. Colajanni

University of Rome - Tor Vergata Roma, I-00133 fcardellini, [email protected]

Philip S. Yu

IBM T.J. Watson Research Center Yorktown Heights, NY 10598 [email protected]

Abstract. Popular Web sites cannot rely on a single powerful server nor on independent mirrored-servers to support ever increasing request load. Scalability and availability can be provided by distributed Webserver architectures that distribute client requests among the multiple servers in a user-transparent way. In this paper we will review the state of the art in distributed Web-server systems. We will analyze the eciency and limitations of the various approaches and their tradeo. 1 Introduction The explosive growth of trac on the World Wide Web is causing a rapid increase in the request rate to popular Web sites. These sites can suer from severe congestion, especially in conjunction with special events. One approach to handle popular Web sites is based on the replication of information across an independent mirrored-server architecture. This solution has a number of disadvantages, including the not usertransparent architecture, and the lack of control on the request distribution. A more promising solution is to rely on a distributed architecture that can route incoming requests among several server nodes in a user-transparent way. This approach can potentially improve throughput performance and provide Web-server systems with high scalability and availability. However, a number of challenges must be addressed to make a distributed server cluster function eciently as a single server. We consider as a distributed Web-server system any architecture consisting of multiple Web-servers with some mechanism to spread the client requests among the servers. The multiple servers may be locally or geographically distributed (namely, LAN and WAN Web-server systems, respectively). In this paper we describe and discuss the various solutions for distributed Web-server systems. We refer to client as a program, typically a Web browser, that establishes connections to Internet for satisfying user requests. Clients are connected to the network through gateways; we will refer to the network sub-domain behind these local gateways as domain. To request a document from a Web-server, each client rst needs to resolve the mapping of the host-name contained in the URL to an IP address. The client obtains the IP address through an address mapping request to the Domain Name System (DNS) server, which is responsible for the address resolution process. A distributed Web-server system needs to appear as a single node to the outside world, so that users need not be concerned about the names or locations of the replicated servers. Other factors considered include scalability, load balancing, availabilSupported by the Italian Ministry of University and Scienti c Research in the framework of the project High Performance Systems for Distributed Applications. 

ity/dependability, applicability to existing Web standards, and geographical distribution (i.e. solutions applicable to both LAN and WAN distributed systems). We classify the existing distributed Web-server architectures by focusing on the entity which distributes the client requests among the servers. In such a way, we identify four classes of methods, the rst of which (client-based) requires modi cation on the client side, and the remaining three (DNS-based, dispatcher-based, server-based) act on one or more components of the distributed Web-server system.

2 Client-based approach We rst consider the routing of the requests from the client side. (Hereafter, unless otherwise speci ed, requests mean data requests, not address mapping requests.) The Web clients play an active role in the request routing, since it is assumed that the clients (not the users) know the existence of the replicated servers of the Web-server system. The way of accessing the Netscape Communication site through the Netscape Navigator1 browser is the rst example of a client-based solution [14]. The mechanism for spreading the load across the multiple Netscape Web-servers is as follows. When the user requests access to the Netscape home page (located at the URL www.netscape.com), Navigator selects at random a number i and directs the user request to the server wwwi.netscape.com. Such approach is not generally applicable, as the number of Web sites that have the privilege to own a browser to distribute the load among their servers is extremely limited. Furthermore, this architecture is not scalable unless the client browser is re-installed. Moreover, the random selection scheme, which is used for spreading the requests, does not guarantee load balancing among the servers. Another solution is the Smart Clients approach [16], that aims at migrating some server functionality to the client machine. The client request routing is achieved through a Java applet which is executed at the client side each time a user requests an access to the distributed Web-server system. As the applet knows the IP addresses of all servers in the cluster, it can get information on the server state and network delays. This received information is analyzed by the applet to select the most appropriate server that will receive the user requests. A major drawback of the Smart Clients approach is the increase in network trac due to the message exchanges to monitor server load and network delay. Moreover, although this approach provides scalability and availability, it lacks portability because it requires modi cation of the client code.

3 DNS-based approach The rst distributed Web-server architecture used a request routing mechanism on the server side. The responsibility of spreading the requests among the servers is delegated to the DNS that provides a single virtual (URL) interface to the outside world. This approach implements load balancing in the logical URL name to IP address translation process, where the DNS can use various scheduling policies to select the appropriate server. The main diculty arises in the limited control that the DNS has on the client requests. In fact, there are many entities in Internet that can cache the URL name to IP address mapping to reduce network trac. Moreover, every Web client browser typically caches some address resolution. Besides providing the IP address, the DNS also speci es a validity period, known as Time-To-Live (TTL), for caching the address 1

Netscape Navigator is a registered Trademark

mapping. After the expiration of the TTL, the address mapping request is forwarded to the DNS for assignment to a Web server in the cluster. Otherwise, the address mapping request does not reach the DNS and is handled by the intermediate name servers. These alternative ways of address resolution are shown in Figure 1. Document request Document response Server 1 (IP address 1) URL User

Client A Internet

(IP 1, TTL)

DNS

URL User

Client B

Server N

(IP N, TTL)

(IP address N)

Document response Document request

Figure 1: DNS-based approach

We distinguish the DNS-based architectures focusing on the algorithm that the DNS uses to select a Web-server and to choose the TTL value. 3.1 Round-robin DNS

The round-robin DNS approach [13] is the rst implemented for distributed homogeneous Web servers by the National Center for Supercomputing Applications (NCSA). The DNS for the NCSA domain has been modi ed in order to implement a roundrobin algorithm (namely, DNS-RR) for address mapping. The load distribution under the DNS-RR is not very balanced because of address caching mechanisms and uneven distribution of client requests among dierent domains [6, 9]. Additional drawbacks of this approach are related to the fact that the DNS-RR ignores both server capacity and availability. The poor performance of the RR-DNS policy motivates the study of alternative DNS routing schemes that require additional state information. 3.2 DNS algorithms using system state information

We next consider various alternatives to DNS-RR using system state information (e.g., client information, server information, or combination) under a xed TTL value for all address mapping requests. Some approaches (e.g., lbmnamed [15]) base the scheduling decision on the current load on the Web-servers. When an address request reaches the DNS, it selects the server with the lightest load. In [6] it was shown that the DNS policies based on server state information only are not eective in balancing the load across the servers. In fact, due to address caching, each address mapping can cause a burst of future requests to the selected server and make the current load information poorly correlate with future load. An alternative is to use client information. Based on the client request load, various DNS scheduling policies were proposed in [6] for a homogenous Web server cluster. The hidden load weight is chosen as measure of the average number of requests sent from

each connected domain to a Web site during the TTL caching period. These DNS policies aim at identifying the requesting domain and the hidden load weight that the request imposes on the server so as to assign it to the most appropriate server. One example is the multi-tier round-robin policy, where dierent round-robin chains are used for requests in dierent ranges of hidden load weight. DNS can also take into account the geographical location of the client to make server selection. This is one of the two alternative modes of the Cisco DistributedDirector [5]. In this mode, DistributedDirector acts as a primary DNS that takes client-to-server topological proximities and link latency into account to determine the most suitable server. However, this solution requires the TTL be set to zero to prevent the address mapping from being cached by other name servers. This has serious limitations on applicability because it makes DNS a potential bottleneck. Furthermore, it does not consider that the TTL choice has no eect on client caching and that most intermediate name servers do not accept very low TTL values. Since the client or server information alone may not always be sucient to select the most appropriate Web-server, it was found in [6] that the best results are achieved by the policies that combine these kinds of information. The hidden load weight information is combined with alarm feedback messages from critically loaded servers to let the DNS exclude them from further assignments during the overloaded period. Similarly, the DistributedDirector chooses the Web-server combining client proximity and server availability. 3.3 Adaptive TTL algorithms

Scheduling algorithms based on the hidden load weight and alarms from critically loaded servers can lead to much better load balancing than RR-DNS and maintain high Web site availability. However, they give less satisfactory results when generalized to a heterogeneous Web-server system. For this reason, a new class of adaptive TTL scheduling policies was proposed in [7]. In these algorithms, the TTL value is adaptively adjusted to each address mapping request based on the request rate of the source domain and the capacity of the selected server. This approach aims at smoothing the impact of bursts of requests originated from non-uniform domains to each server and can take server availability into account by using feedback mechanisms from the servers. Since the adaptive TTL algorithms only require information available at the DNS, they can be applied to both locally and geographically distributed environment.

4 Dispatcher-based approach Another class of solutions, that aim to achieve full control of client request load and mask the request routing among multiple servers, extends the address virtualization, done by the DNS-based approaches at the URL level, to the IP level. This approach provides the Web-server cluster with a single virtual IP address (IP-SVA). In fact, this is the IP address of a dispatcher that acts as a centralized scheduler and, in contrast to the DNS, has complete control on the routing of all client requests. The dispatcher identi es each Web-server through a private address. We dierentiate the techniques of this class through the mechanism used to reroute the packets reaching the dispatcher to the selected Web-server.

4.1 Packet single-rewriting by the dispatcher

We rst consider architectures where the dispatcher reroutes client-to-server packets by rewriting their IP address. An example of this solution is the TCP router mechanism described in [9]. The distributed Web-server cluster is made up by a group of nodes and a TCP router that acts as a dispatcher. All client requests reach the TCP router because the IP-SVA is the only public address. The dispatcher uses simple algorithms for the selection of the Web-server (e.g., round-robin, server load). For each client-to-server packet, the dispatcher replaces its IP-SVA with the IP address of the selected server. Before sending the response packets to the client, the Web-server needs to replace its IP private address with the IP-SVA. Although this solution is transparent to the client, the TCP router architecture does require modi cation of the kernel code of the servers, since the IP address substitution occurs at the TCP/IP level. On the other hand, this approach provides high system availability, because upon failure of a front-end node, its address can be easily removed from the dispatcher table of Web-servers. 4.2 Packet double-rewriting by the dispatcher

This class of distributed Web-server systems also relies on the presence of a centralized dispatcher that controls all client requests. The dierence from the TCP router is in the modi cation of the source address of the response packets. Now, the modi cation of all addresses, including that in the server-to-client packets, are carried out by the dispatcher itself. This packet double-rewriting mechanism is based on the Network Address Translation (NAT) de ned in [10] and shown in Figure 2. (Response, IP-SVA)

Server 1 (IP address 1)

Document request User

Client A

(Response, IP 1)

URL

Address dispatcher (IP-SVA)

Internet URL User

Client B

(IP 1) (IP N) (Response, IP N)

Document request Server N (IP address N) (Response, IP-SVA)

Figure 2: Packet double-rewriting by the dispatcher.

Two architectures that are based on this approach with a server fault detection mechanism are the Magicrouter [1] and the LocalDirector [4]. Magicrouter uses a mechanism of fast packet interposing, where a user level process intercepts network packets and modi es them by changing the addresses and checksum elds. Three algorithms can be used to select the Web-servers: round-robin, random and incremental load which is similar to selecting the lowest load server based on the current load estimate and the per TCP connection load adjustment. The LocalDirector from Cisco rewrites the IP information header of each incoming packet according to a dynamic mapping table between each session and the server to which it has been redirected. The routing policy selects the server with the least number of connections.

4.3 Packet forwarding by the dispatcher

The solutions based on the packet double-rewriting by the dispatcher have two major disadvantages: they do not work with the protocols that carry and use IP address inside the application [10], and furthermore, they require that the dispatcher rewrites not only client-to-server but also server-to-client packets (which are typically much more than client-to-server packets). The packet single-rewriting by the dispatcher mechanism is more ecient, because the more numerous server-to-client packets are rewritten by the Web-servers. However, packet rewriting remains a big overhead that it is preferable to avoid. Packet forwarding mechanisms, such as Network Dispatcher [12] and ONE-IP address [8], aim at addressing this issue. The IBM Network Dispatcher is an extension of the basic TCP router mechanism. Lets distinguish its LAN implementation from WAN implementation. The LAN Network Dispatcher solution assumes that the dispatcher and the servers are on the same local network. Hence, the dispatcher can forward the client packets to the selected server using its physical address on the LAN without modifying the IP header. This solution is transparent to both the client and server because it does not require packet rewriting. Furthermore, the scheduling policy used by the dispatcher can be dynamically based on server load and availability. The extension of the Network Dispatcher to a WAN distributed architecture requires a dierent architecture based on two levels of Network Dispatchers. The rst level acts similarly to a packet single-rewriting mechanism, because the dispatcher replaces the IP-SVA address with the IP address of the chosen sub-cluster that is coordinated by a second level Network Dispatcher. The second level Network Dispatcher changes this IP address back to the original IP-SVA. As the second level Network Dispatcher and the Web-servers of this sub-cluster are on the same local network, the packet forwarding mechanism can be done at the MAC address level. This mechanism limits the rewriting to the client-to-server packets. The ONE-IP address architecture uses a dierent forwarding approach based on the ifconfig alias option. This solution publicizes the same secondary IP address of all Web-servers as IP-SVA (here called ONE-IP). Starting from this basic approach, two techniques are described in [8]. The routing-based dispatching requires that all the packets addressed to the ONE-IP address are rst rerouted by the subnetwork router to the dispatcher of the distributed Web-server system. The dispatcher selects the destination server by applying a hash function to the client IP address and then rerouting the packet to the selected server. In the broadcast-based dispatching solution the subnetwork router broadcasts the packets having the ONE-IP address as destination to every server in the Web-server cluster. Each of them evaluates whether it is the actual destination by applying a hash function to the client IP address and comparing the result with its assigned identi er. Problems of the ONE-IP approach can arise from augmented trac overhead and the static scheduling algorithm which does not take the server state and heterogeneity into account for the routing decision. The LAN Network Dispatcher architecture avoids most of the network trac problems of the ONE-IP solutions. However, it requires that the dispatcher and the Web-servers are directly connected by the same network segment. 4.4 HTTP redirection by the dispatcher

In the HTTP redirection approach a centralized dispatcher receives all incoming requests and reroutes them among the Web-servers through the redirection mechanism

provided by HTTP. Redirection is done by specifying the appropriate status code in the response and indicating in its header the Web-server address. The Web-server selection can be based on server state information and/or client location information. The Distributed Server Groups approach [11] adds new methods to HTTP protocol to administer the cluster and exchange messages between the dispatcher and the servers. Since the dispatcher needs to be aware of the server load, each server periodically reports the number of processes in its run queue and the number of received requests per second. Based on this information, the dispatcher selects the least loaded server. The Cisco DistributedDirector [5] (already discussed in Section 3.2 as a DNS approach) uses some metric to estimate the proximity of a client to the servers and the server availability, determines the most suitable server for the request and redirects the client to the selected server. The HTTP redirection approach can be nely applied to LAN and WAN distributed Web-server systems. Unlike the other dispatcher-based solutions, the HTTP redirection does not require the modi cation of the packet addresses. However, this architecture duplicates the number of necessary TCP connections. Moreover, this mechanism can be used for distributed Web-server systems that provide HTTP services only.

5 Server-based approach Unlike the DNS-based and dispatcher-based solutions, the server-based techniques use a distributed scheduling policy, in which all servers in the cluster can participate in the request (re-)assignment. The architecture of server-based approaches uses a two-level distributed scheduler. Client requests are initially assigned from the DNS in the roundrobin manner and then possibly redirected. The two proposals of this class dier in the way the redirection decision is taken and implemented. Scalable server World Wide Web (SWEB) system [2] redirects requests through the HTTP protocol. Each server may reassign a received request to any other server of the cluster. The decision to serve locally or to redirect a request is based on the minimization of the response time. Such an assignment mechanism implies an overhead of communication among the servers, as every server needs to periodically broadcast its current load. Moreover, the HTTP redirection may increase response time and network trac because each redirected request requires two connections. Distributed Packet Rewriting (DPR) uses the packet rewriting mechanism to reroute requests [3]. Each server can redirect a connection to another server in a client transparent manner. Two algorithms can be used to select the Web server. The former policy is stateless, since the destination server is determined through a hash function applied to the client IP address and the port number. The latter takes into account the current server load. DPR can be applied to both LAN and WAN distributed Webserver systems. However, the packet rewriting and redirecting cause a delay that can be considerable in WAN distributed Web-server systems.

6 Summary In this paper, we propose a classi cation of existing high performance Web-server system based on the entity that dispatches the client requests among the distributed servers. A summary of the tradeo among these approaches is given below. The client-based approach reduces the load on Web servers by implementing the routing service at the client side. However, this solution lacks portability since the

client must be aware that the Web site is distributed. DNS-based approach routes client requests through address mapping. However, due to address caching, DNS can control only a very small fraction of address requests. So the scheduler needs to use additional state information to achieve better performance. By adaptively setting the TTL value for each request, the performance can be greatly improved and the DNS architecture can be applied even to heterogeneous servers. The main drawback of dispatcher-based solutions comes from the presence of the dispatcher, which can become a bottleneck. However, the centralized scheduler and the single IP address prevents DNS problems arising from address caching. The solutions that adopt the packet rewriting/rerouting mechanism are most applicable to server clusters on a LAN or high speed Intranet. The distributed scheduling provided by the server-based architecture guarantees scalability and does not introduce a potential bottleneck in the system. However, the redirection mechanism that characterizes this approach can increase the latency time perceived by the users, especially in a geographically distributed system.

References [1] E. Anderson, D. Patterson, E. Brewer, \The Magicrouter, an application of fast packet interposing", University of California, Berkeley, May 1996. [2] D. Andresen, T. Yang, V. Holmedahl, O.H. Ibarra, \SWEB: Toward a scalable World Wide Web server on multicomputers", Proc. of 10th Int. Symp. on Parallel Processing, Honolulu, pp. 850{856, April 1996. [3] A. Bestavros, M. E. Crovella, J. Liu, D. Martin, \Distributed Packet Rewriting and its application to scalable server architectures", T.R. BUCS-TR-98-003, Boston University, Dec. 1997. [4] Cisco's LocalDirector. Available online at http://www.cisco.com/warp/public/751/lodir/index.shtml [5] Cisco's DistributedDirector. Available online at http://www.cisco.com/warp/public/751/distdir/dd wp.htm [6] M. Colajanni, P.S. Yu, D.M. Dias, \Analysis of task assignment policies in scalable distributed Web-server system", IEEE Trans. on Parallel and Distributed Systems, vol. 9, no. 6, June 1998. [7] M. Colajanni, P.S. Yu, V. Cardellini, \Dynamic load balancing in geographically distributed heterogeneous Web-servers", Proc. of IEEE 18th Int. Conf. on Distributed Computing Systems, Amsterdam, May 1998. [8] O. Damani, P. Chung, Y.Huang, C. Kintala, Y. Wang, \ONE-IP: techniques for hosting a service on a cluster of machines", Proc. of 6th Int. World Wide Web Conf., Santa Clara, Apr. 1997. [9] D.M. Dias, W. Kish, R. Mukherjee, R. Tewari, \A scalable and highly available Web server", Proc. of 41st IEEE Computer Society Int. Conf., pp. 85{92, Feb. 1996. [10] K. Egevang, P.Francis, \The IP Network Address Translator (NAT)", RFC 1631, May 1994. [11] M. Garland, S. Grassia, R. Monroe, S. Puri, \Implementing Distributed Server Groups for the World Wide Web", T.R. CMU-CS-95-114, Carnegie Mellon University, Jan. 1995. [12] G. Hunt, G. Goldszmidt, R. King, R. Mukherjee, \Network Dispatcher: A connection router for scalable Internet services", Proc. of 7th Int. World Wide Web Conf., Brisbane, Apr. 1998. [13] T.T. Kwan, R.E. McGrath, D.A. Reed, \NCSA's World Wide Web server: Design and performance", IEEE Computer, no. 11, pp. 68{74, Nov. 1995. [14] D. Mosedale, W. Foss, R. McCool, \Lesson learned administering Netscape's Internet site", IEEE Internet Computing, vol. 1, no. 2, pp. 28{35, Mar.-Apr. 1997. [15] R.J. Schemers, \lbmnamed: A load balancing name server in Perl", Proc. of the 9th Systems Administration Conf., Monterey, Sep. 1995. [16] C. Yoshikawa, B. Chun, P. Eastham, A. Vahdat, T. Anderson, D. Culler, \Using Smart Clients to build scalable services", Proc. of Usenix 1997, Jan. 1997.