A TPC-W-based Tool for Benchmarking E-commerce Programming ...

A TPC-W-based Tool for Benchmarking E-commerce Programming Technologies Yussuf Abu Shaaban∗

Jane Hillston†

June 14, 2002

Abstract E-commerce systems are an important new application area in which maintaining good performance under scaling workloads is crucial to business success. The TPC-W benchmark is a benchmark designed to exercise a web server and associated transaction processing system in representative e-commerce scenarios. Whilst the benchmark specifies the architecture of the system, and the form of the interactions between users, web server and underlying database, it does not stipulate the supporting technology used to communicate between the web server and the database. In this paper we describe a tool which has been developed to allow comparative benchmarking of different technologies with minimal re-implementation. The tool implements the TPC-W specification in a module fashion which allows different application servers to be inserted and experimented upon.

1

Introduction

In recent years e-commerce systems have become one of the most noticeable manifestations of the Internet phenomenon. Although there are no widely accepted definitions, such systems are generally deemed to provide facilities for commercial transactions to take place between remote participants. In particular [3] identifies three categories of e-commerce systems: Electronic Markets: the use of information and communication technology to present offers available in a market segment so that the purchaser can compare the offers and make a purchase decision. An example of an electronic market is an airline booking system. Electronic Data Interchange (EDI): provides a standardised way for coding trade transactions, so that they can be communicated from one computer to another. A market segment where EDI is used is large supermarket chains which use EDI for transactions with their suppliers. Internet Commerce: the commercial use of the Internet. Information and communication technologies can be used to allow the consumer purchasing over the Internet. Goods can be delivered by post or delivery can be electronic in the case of electronic materials. For the remainder of this paper we will treat “e-commerce” as synonymous with Internet commerce. In many e-commerce systems, timely behaviour is crucial in order to maintain the site owner’s competitive edge: poor performance can quite literally translate into lost revenue [1]. However, little systematic work has been done on analysing the performance of such systems and their supporting technologies. In this paper we describe a tool which aims to provide a framework in which different e-commerce programming technologies can be easily benchmarked against the standard TPC-W benchmark [11]. The TPC-W benchmark has been developed by the Transaction Processing Performance Council (TPC) as a response to the rise of e-commerce systems. It specifies the behaviour of an on-line bookstore, including ∗ supported by EPSRC studentship 00317428, ICSA, Division of Informatics, [email protected] † ICSA, Division of Informatics, University of Edinburgh. Email: [email protected]

1

University of Edinburgh.Email:

many of the elements commonly found in e-commerce applications: a web-site supported by a web serving component which can present both static and dynamic web pages; a relational database which is accessed from the web server to provide transaction processing and decision support. Moreover the benchmark also specifies emulated remote browsers for different classes of customers, providing the workload on the system. Our tool implements the TPC-W benchmark, in Java, as a framework for the investigation of the application server technologies. The application server sits between the web server and the database and provides the business logic as well as the interface between the manipulation and the presentation of the data. Whilst the benchmark places various constraints on the application server it does not stipulate how it should be implemented. Available technologies for the this module of the system include CGI, ASP, Java Servlets, JSP and PHP. (These technologies are presented in more detail in the following section.) Furthermore, the tool incorporates support for experiment design, allowing the user to replicate runs of the benchmark and compare the results obtained with different workload mixes. The remainder of the paper is structured as follows. In Section 2 we review the role of the application server within an e-commerce system and the different technologies currently available. In Section 3 we describe the TPC-W benchmark in some detail and in the following section we show how this is captured within the design of our tool. Section 5 gives details of the implementation of the tool. The use of the tool in evaluating the scalability of the Java Servlets application server technology is described in Section 6. Finally in Section 7 we give some conclusions and discuss our further work.

2

Application Server Technologies

The application server is a vital component of an e-commerce system: it is the software component that contains the business logic of the system. It is responsible for receiving HTTP requests via the web server from clients. The business logic associated with HTTP requests is executed. This can involve interacting with database servers and/or transaction servers. The application server also has the responsibility of dynamically building web pages formatting database query results in HTML to be sent back to clients. Today, many programming environments exist for implementing application servers, fulfilling the need for dynamic web-site generation and allowing the connection between web front-ends and databases. In this section we give brief descriptions of the major programming technologies that are used to build e-commerce applications. These include the Common Gateway Interface (CGI), Microsoft’s Active Server Pages (ASP), Personal Home Pages (PHP), Java Servlets and Java Server Pages (JSP). Common Gateway Interface The Common Gateway Interface (CGI) [4] is a standard, not a programming language. It is used to create an interface between HTTP user requests and back-end databases. CGI specifies how to pass arguments to the executing program as part of the HTTP request. It allows the returned HTML document to depend in any arbitrary way on the request, e.g. a CGI program can access information in a database and format the result as HTML. A CGI program can be any program that can accept command line arguments. Perl is a common choice for writing CGI scripts, but they can also be written in other languages such as C/C++, FORTRAN, TCL and UNIX shell scripts. One advantage of CGI scripting is that it is a well-tested approach with many existing libraries available for use. Other advantage is that the developer is not limited to a particular programming language. However, performance is the main disadvantage of CGI. This is due to the fact that CGI is not multithreaded. A CGI program needs to create a new process for each user request, and this process will be terminated as soon as the data transfer is complete. Thus, on each request, the operating system has to load the program, allocate memory for it and then de-allocate and unload the program from memory. This is inefficient, especially if there is a large number of client requests to be handled. Active Server Pages Active server pages (ASP) [5] was introduced in 1996 by Microsoft and forms a server-based technology. It was developed for the Windows NT platform along with Microsoft Internet Information Server and modified for Microsoft’s .Net e-commerce servers. ASP can be used to connect a web-site with a database using its database access technology, ActiveX Data Objects. With ASP, it is also possible to design dynamic, interactive HTML documents in response to user requests.

2

ASPs are written with VBScript, which is a subset of Visual Basic for Applications(VBA). VBAs are used by Microsoft office product and are based on Visual Basic, making ASP a popular choice for Windows developers. An ASP file is HTML with ASP as an extension. When a request is received for an ASP file, the final HTML document will be generated along with the data being sent from the database server. ASP is inherently multi-threaded making it a good choice for large e-commerce sites. A disadvantage of ASP is its lack of portability: ASP programs can only run on ActiveX-supporting platforms. Personal Home Pages Personal Home Pages (PHP) [10] is a server-side, cross-platform, HTML embedded scripting language. It was developed late in 1994 by Rasmus Lerdorf to keep track of visitors to his on-line resume. Since then, it has undergone several changes with two versions released, PHP3 and PHP4. Currently, PHP is shipped with a number of commercial products such as Stronghold web server and RedHat Linux. PHP provides a programming approach similar to ASP, but its broad support for databases gives it an edge over VBScript. Its code can be embedded directly into a HTML page and executes on the server. PHP modules are lightweight and speedy. Without any process creation overhead, it can return very quickly. Java Servlets Java Servlets [7] are server side Java code that runs on a server to answer client HTTP requests. Servlets make use of the Java standard extension classes in the packages java.servlet and javax.servlet.http. Since servlets are written in the highly portable Java language and follow a standard framework, they provide a means to create sophisticated server extensions in a server and OS independent way. When a servlet program is called for the first time, it is loaded into memory. After the request is processed, the servlet remains in memory and will not be unloaded from memory until the server is shutted down. Servlets offer excellent connectivity with many databases through the use of Java’s JDBC package. The output of HTML in a servlet environment is an issue, as the servlet is required to output all HTML internally. This requires complicated output statements to handle the output of the entire HTML content, as well as the code for the rest of the application. Java Server Pages Java Server Pages (JSP) [6] was developed as a collaboration between Sun Microsystems and a number of web server, application server and development tool vendors as a solution for webdatabase applications. Similarly to ASP, JSP code resides alongside HTML and is parsed, compiled and executed when a page is requested. JSP is an extension of the servlet technology. When a JSP page is first called, it is compiled into a Java servlet class and stored in the server memory. This enables very fast responses for subsequent calls to that page. Anything that can be achieved through JSP can in principle be achieved through Java servlets, however, JSP allows the placement of code generating dynamic HTML alongside static content. This is in contrast to Java servlets, where numerous print statements are needed to output dynamic HTML as well as static content. Currently in the tool we offer implementations of Java Servlets and PHP.

3

TPC-W Benchmark

The TPC-W benchmark has been developed by the Transaction Processing Performance Council (TPC), a consortium of system and database vendors[11]. Historically, TPC has specified standard benchmarks (e.g. TPC-C, TPC-H) for evaluating the performance of both transaction processing and decision support database systems. One of its latest benchmarks is TPC-W, an e-commerce-specific benchmark. It specifies the behaviour of an on-line bookstore, including the three main components of an e-commerce application: remote browsing, web server and database server. TPC-W’s remote browsing specifications are described next. This is followed by an overview of the web server component and finally TPC-W’s database server specifications are discussed.

3

3.1

Remote Browser Emulator

A main component of the TPC-W benchmark is the Remote Browser Emulator (RBE), which is a specification for a set of Emulated Browsers (EBs). EBs simulate the activities of concurrent web browsing e-commerce users, each autonomously traversing the bookstore web pages by making requests to a web server. Each EB can represent one of three classes of users: a customer, a new user or a site administrator. As described in Section 3.2, TPC-W defines 14 web interactions which can be requested by an EB. During its lifetime, an EB requests a sequence of these web interactions moving from one interaction to the next, in the same way a web browsing user navigates a site clicking one hypertext link after another. TPC-W specifies the next possible navigation options that can be requested by an EB on completion of each of the web interactions defined in the benchmark. Threshold integer values between 1 and 9999 are specified for each navigation option (one for each measurement interval, see Section 3.4). To select its next request, the EB generates a random number, from a uniform distribution between 1 and 9999. It then selects the navigation option for which the threshold is equal to or most immediately greater than the random number. The EB spends a random period of time (Think Time) sleeping between subsequent web interactions. This emulates the user’s think time and is generated from an exponential distribution specified in TPC-W. User-specific information must be maintained in an EB, possibly including session tracking details and customer identification.

3.2

Web Server

The TPC-W benchmark defines 14 web interactions to be supported by a web server component which are: Home, Shopping Cart, Customer Registration, Buy Request, Buy Confirm, Order Inquiry, Order Display, Search Request, Search Result, New Product, Best Seller, Product Detail, Admin Request and Admin Confirm. These interactions vary in the amount of server-side processing they need. Some require dynamically generated HTML pages and one or more connections to a database. Others are relatively lightweight, requiring only web serving of static HTML pages and images. For each web interaction, TPC-W specifies its input requirements, processing definition, response page definition and EB navigation options which are the set of web interaction that can be selected by the EB on completion of the interaction. Secure Socket Layer (SSL) [9] connections are required for payment-related web interactions. The TPCW benchmark also defines a payment gateway emulator which authorises the payment in a Buy Confirm web interaction. Session tracking is vital to any e-commerce application. Some method is needed to retain information such as shopping carts and customer IDs from one HTTP request to another. TPC-W suggests two techniques for session tracking which are URL-rewriting and cookies [2].

3.3

Database Server

The TPC-W benchmark defines the exact schema used for the online bookstore database, together with a scale factor (with possible values of 1,000; 10,000; 100,000; 1,000,000; 10,000,000). The schema consists of eight tables: customer, address, order, order line, credit card transaction, item, author and country. Additional tables may be added to the schema. The scaling rule for the database depends on two factors: the number of EBs that will be used as a workload and the scale factor.

3.4

Performance Metrics and Measurement Intervals

TPC-W defines one primary performance metric which is throughput, measured as the number of completed web interactions per second (WIPS). Three distinct measurement intervals are specified by TPC-W: shopping interval, browsing interval and ordering interval. They are distinguished by the ratio of browsing related web pages visited to ordering related web pages visited during the measurement interval. The shopping interval is intended to reflect a shopping scenario, in which 80% of the pages the user visits are related to browsing and 20% are related to ordering. In a browsing interval, ordering pages visited go down to 5% whereas in an ordering interval the ratio of browsing and ordering is even.

4

4

Design of the Tool

Our tool allows comparative benchmarking of e-commerce programming technologies. In designing the tool, a number of design criteria were taken into account. These are discussed in the next section. This is followed by a detailed overview of the design.

4.1

Design Criteria

As will be seen in Section 4.2, a set of design criteria were met in the design of the tool. These include the following: • The tool should provide experiment design features allowing the user to specify a number of experiments. For each experiment, factor level combinations such as size of workload driving the system, size of store and measurement interval structure can be specified. The number of replications for each experiment can also be specified. • The tool should provide a set of performance metrics necessary for assisting e-commerce technology performance evaluators. Metrics such as response time frequency distributions of different web interactions, throughput, revenue throughput, CPU utilisation and memory utilisation should be analysed and presented graphically. • Ensuring that the analysis and presentation of experimental results have minimal overhead was an objective from early stages in the design, thus preserving the realistic nature of the simulation. This resulted in a local data collection strategy in the design. Experimental data results are maintained locally by different parts of the system. Results are gathered at the end of each experiment run from different parts of the system for analysis and presentation. • Realistic e-commerce modelling was an important design criterion. This was ensured in the design by adopting TPC-W as an e-commerce model. Not only are different e-commerce site components represented faithfully; the design also includes realistic workload generation capabilities based on TPCW’s e-commerce access patterns. • Developing a flexible, extendable tool was a major concern. This led to a modular design where incorporating a new programming technology for benchmarking just involves adding a new module to the tool. In addition, modular design allows for the easy extension of the tool to benchmark other e-commerce components such as security protocols, session tracking techniques, etc.

4.2

Design Overview

Figure 1 illustrates the main modules of the tool. The RBE module is responsible for generating and maintaining the workload on the System Under Test (SUT). Its design is based on the TPC-W benchmark as described in Section 3.1. The Web Server contains an Application Server module which represents the application code for implementing the 14 web interactions specified in the TPC-W benchmark (see Section 3.2). The Database Server module represents the persistent storage component of the TPC-W e-commerce model. The final main module in the design is the Control Unit which provides features for experimental design, data gathering and analysis. It is also responsible for controlling the setup and maintenance of experiments on various parts of the tool. The Control Unit module is described next, this is followed by a description of RBE, Web Server and finally the Database Server module. 4.2.1

Control Unit

The Control Unit is the central component of the benchmark. In addition to providing a GUI to the user, it is responsible for setting up and maintaining the running of experiments. It is also responsible for gathering experiment data results from different parts of the system for analysis and presentation. 5

System Under Test (SUT) Web Server HTTP RBE TCP/IP

Application Server

Database Server

Control Unit

Figure 1: Tool Overall Design

Figure 2 illustrates the main sub-modules of the Control Unit. The ControllerGUI provides a GUI allowing the user to input experimental design details and to monitor the progress of experiments. In addition, ControllerGUI allows the user to specify the e-commerce technology to be benchmarked. ControllerGUI interacts with the Control Unit sending it user input and receiving from it data to be displayed. The Experimental Design component is responsible for holding the experimental plan of the user. It informs the Control Unit of the experiment design details needed at each stage of the simulation. As the user is allowed to specify a number of experiments which are run sequentially, the Experimental Design component consists of a set of Experiment objects, each holding the design details for one experiment which include: • Number of replications. • Number of EBs to be instantiated for steady state (workload size). • Measurement interval structure which can be any combination of shopping, browsing and ordering intervals (see Section 3.4). The time period of each interval must be specified. • Ramp up period specifying the time period that must be elapsed before measurements are recorded. • Size of store which is a scale factor (with possible values of 1,000; 10,000; 100,000; 1,000,000; 10,000,000) contributing to the determination of the bookstore database size (see Section 3.3). Experimental data gathering and analysis is the responsibility of the Result Analysis Unit. It consists of a set of Experiment Results objects holding the results for each experiment executed in the simulation. Each Experiment Results object is linked with an Experiment object and contains a set of Run objects holding the results of each experiment run. During an experiment run, experimental data is recorded locally by different parts of the system minimising data recording overhead. On completion of the run, the Result Analysis Unit receives a request from the Control Unit to gather the experimental data which are then stored in a 6

1

Control Unit

1

1

1

Controller GUI

1

1

Result Analysis Unit

Experimental Design

1

Time Control

RBE Controller

1

Web Server Controller

1

Database Server Controller

1

1

1...*

Experiment

1

1..* 1

1

Experiment Results

1

1..*

Run

Figure 2: Control Unit

Run object. Data analysis is done after the completion of each experiment. Performance metrics provided include the frequency distribution of the response time for all types of web interactions, Throughput, Revenue Throughput (dollars/sec), CPU Utilisation and Memory Utilisation. Time Control is the central timing component of the tool. It distributes timing information to all parts of the system upon request. The Control Unit also includes three controller components, RBE Controller, Web Server Controller and Database Server Controller for controlling the RBE, Web Server and Database Server respectively. Instructed by the Control Unit, these components configure and control the running of experiments on the parts of the tool they are responsible for. Controlling a new extension to the tool requires only adding a new controller to the Control Unit. 4.2.2

RBE

RBE is the component responsible for driving the tool workload. As shown in Figure 3, it includes a set of EBs which emulate web browsing e-commerce users requesting web interactions from a web server as specified in the TPC-W benchmark (see Section 3.1). RBE also contains a Browsing Control module responsible for managing the creation and maintenance of EBs with the guide of the RBE Controller module of the Control Unit. An EB is designed to emulate a web browser used by an e-commerce user which can be a customer, a new user or a site administrator. Its Navigation Unit emulates the user’s navigational behaviour. The decision on which navigation option to select next is based on the current web interaction just completed and its EB navigation option thresholds, specified by TPC-W (see Section 3.1). The Navigation Unit generates a random Think Time period which is spent sleeping between web interaction requests. The EB User Session sub-module is designed to emulate the maintenance of e-commerce user-specific information in a web browser including session tracking details and customer identification. The sending and receiving of HTML content via HTTP and TCP/IP over a network connection is emulated by the Request Unit sub-module. It is responsible for forming HTTP requests for the different web interactions specified in the TPC-W benchmark. It is also responsible for maintaining HTTP/TCP/IP connections with the web server and associating these connections with User Sessions. In addition, the Request Unit log data statistics about web interactions requested and completed successfully including the starting and finishing time of a web interaction, its type and how many times it has been requested in an 7

1

RBE

1

1..*

Browsing Control

EB

1

Navigation Unit

1

User Session

1

Request Unit

Figure 3: RBE

experiment run. EBs are created and maintained by the Browsing Control component. It receives instructions from the Control Unit (via its RBE Controller) on the measurement interval type and the number of EBs required for each experiment. When creating an EB, Browsing Control provides it with the following: • The type of user the EB represents: a customer, a new user or a site administrator. The choice of the type of user is decided randomly by the Browsing Control component, based on a distribution selected during experiment design. • The type of measurement interval(s) for the experiment. Other tasks of Browsing Control include informing EBs of the start/end of a measurement interval and/or an experiment run (upon orders from the Control Unit). Browsing Control also collects data recorded by the Request Unit to be passed to the Control Unit. 4.2.3

Web Server

The Web Server component is responsible for serving EB HTTP requests for the 14 web interactions specified in TPC-W as described in Section 3.2. It provides support for multiple concurrent secure SSL and/or insecure connections with RBE. Figure 4 illustrates the two main sub-modules comprising Web Server: Application Server and Web Server Control. The Application Server includes the application code implemented by the programming technology currently benchmarked by the tool. A static HTML page and a set of images or an application program accessing the Database Server reside in the Application Server for each web interaction specified by TPC-W. The Web Server Control sub-module is responsible for setting up the Web Server for a whole simulation. It receives instructions from the Web Server Controller of the Control Unit on the programming technology to be benchmarked. It is the responsibility of Web Server Control to load and setup the required modules specific for that programming technology. In addition, Web Server Control collects experimental data results recorded by the Web Server including CPU utilisation and pass it to the Control Unit on request.

8

Web Server

Web Server Control

Application Server

Figure 4: Web Server

4.2.4

Database Server

Database Server contains two parts, Database and Database Server Control as illustrated in Figure 5. The Database schema follows exactly the bookstore schema specified in TPC-W (see Section 3.3). Database

Database Server

Database Server Control

Database

Figure 5: Database Server Server Control is responsible for populating/de-populating the Database. Through the Database Server Controller, the Control Unit informs Database Server Control of the number of EBs and a scale factor. This determines the scale at which Database Server Control populates the Database. In addition, Database Server Control collects experimental data results recorded by the Database Server including memory utilisation and pass it to the Control Unit on request.

5

Implementation

In this section we discuss the tool implementation issues. First, the decision to implement the tool in the Java language is discussed. The use of Apache as the web server of the tool is then described and the choice of MySQL to manage the bookstore database is discussed. This is followed by a description of the support provided for e-commerce programming technologies. 9

Java (SDK Version 1.3.0) is used to implement the tool. Being a pure object-oriented programming language, Java naturally ensured the realisation of the tool’s modular design. The implementation in a portable programming language as Java resulted in a platform-independent tool which can be deployed on machines of different architectures across a network. Java’s java.net package, with its URL and socket classes, provided a neat implementation of the interactions between the EBs (see Section 4.2.2) and the Web Server (see Section 4.2.3) components of the system. The reliable System.currentTimeMillis() method, which is part of Java’s java.lang package is used to get timestamps from various parts of the system needed to produce performance metrics. System.currentTimeMillis() is quick with almost no overhead, thus enforcing the realistic nature of the simulation. A web server is needed to server HTTP client requests and host the application server (see Section 4.2.3). The Apache web server (Version 1.3) is used, a popular high performance freeware web server and is the choice of the majority of active site developers. A survey by Netcraft [8] on October 2001 concluded that 61% of the active sites they monitor use Apache. The way Apache is designed was another reason for choosing it. It is built around an API which allows third-party programmers to add new server functionality. Everything in Apache is implemented as one or several modules, using the same extension API available to third parties. Thus, extending the tool to provide support for a new e-commerce programming technology requires implementing a new Apache module and incorporating it using Apache’s API. Cookies was the method adopted for session tracking. A Database Management System (DBMS) is needed to host the bookstore database (see Section 4.2.4). It was decided to use MySQL (Version 3.23), an open source DBMS. MySQL is a popular DBMS and works well with Apache for different e-commerce programming technologies.

Currently, the tool provides application server implementations in two ecommerce programming technologies; Java Servlets [7] (using mod jserv apache module) and PHP [10] (using PHP apache module). Providing benchmarking support for another e-commerce programming technology requires only finding/implementing a supporting Apache module and implementing the application code specific to that technology.

6

Preliminary Results

An initial experiment was designed and implemented using the tool to evaluate the scalability of the application server Java Servlet implementation. The effect of varying the size of the workload on average throughput was examined. The input factor in the experiment was the number of Emulated Browsers (EBs) making simultaneous requests to the tool’s Web Server. The number of EBs was varied from 1 to 1000 in a set of experiment runs. In each run, the average throughput was calculated by dividing the number of requests completed successfully by 900 seconds (15 minutes), the length of the measurement period. A shopping scenario was adopted where 80% of the web pages requested by EBs were shopping-related pages and 20% were ordering-related pages. In each run, the bookstore database was initially populated as follows (according to TPC-W specification): • Customer table: 2880 * Number of EBs rows. • Item table: 1000 rows. • Country table: 92 rows. • Address table: 2 * Number of customers rows. • Orders table : 0.9 * Number of customers rows. • Order line: 3 * Number of orders rows. • Author: 0.25 * Number of items rows. • cc xacts: 1 * Number of orders rows.

10

Figure 6: Number of Emulated Browsers VS. Average Throughput

In all experiment runs, the RBE (workload generator), the web server, application server and the database server were all running on a GenuineIntel Pentium III, 1 GHz, 265MB Linux Dell Machine. Figure 6 summerises the results obtained from running the experiments. It can be seen that average throughput increases as the workload size approaches 100 EBs before it starts to fall sharply. Further experiments need to be performed on the PHP application server implementation to compare the scalability of the two technologies.

7

Conclusions and Further Work

A tool to allow the comparative benchmarking of e-commerce programming technologies has been developed. It is based on the TPC-W benchmark, ensuring the realistic modelling of e-commerce components. It provides a set of experiment support features allowing the user to design a set of experiments to be run sequentially by the tool. For each experiment, a number of replications can also be specified. A set of e-commerce relevant performance metrics are analysed and presented with minimal overhead on the simulation. This was achieved by adopting a local experimental data recording strategy during experiment runs. Data is gathered and analysed on completion. The modular design of the tool widens the scope of further work. New modules can be incorporated to benchmark other programming languages/technologies. The RBE module can be enhanced to provide robots workload [12] in addition to remote browsers. In addition to benchmarking e-commerce programming technologies, the tool can even be extended further with very small modifications to allow for the benchmarking of other e-commerce components such as security protocols, payment protocols and session tracking mechanisms. 11

References [1] Menasce D. A. and Almeida V. A. Scaling for E-Business: Technologies, Models, Performance, and Capacity Planning. Prentice Hall PTR, 2000. [2] Ince D. Developing Distributed and E-commerce Applications. Addison Wesley, 2002. [3] Whiteley D. e-Commerce: Strategy, Technologies and Applications. McGraw Hill, 2000. [4] Kruse M. http://mkruse.netexpress.net/info/cgi. Technical report. [5] Microsoft. http://msdn.microsoft.com/library/psdk/iisref/aspguide.htm. Technical report, Microsoft. [6] Sun Microsystems. http://java.sun.com/products/jsp/whitepaper.html. Technical report, Sun Microsystems. [7] Sun Microsystems. http://java.sun.com/products/servlet/whitepaper.html. Technical report, Sun Microsystems. [8] Netcraft. http://www.netcraft.com/survey. Technical report, Netcraft. [9] Freier A. O. and Kocher P. C. http://home.netscape.com/eng/ssl3/draft302.txt. Technical report, Netscape Communications. [10] Bakken S. S., Aulbach A., Schmid E., Winstead J., Wilson L. T., Lerdorf R., Zmievski A., and Ahto J. http://www.php.net/manual.en. Technical report, The PHP Group. [11] TPC. http://www.tpc.org/tpcw. Technical report, Transaction Processing Performance Council. [12] Almeida V., Menasce D., Riedi R., Peligrinelli, Fonseca R., and Meira W. Analyzing robot behavior in e-business sites. In Proc. 2001 ACM Sigmetrics Conference, Cambridge, MA, June 16-20, 2001.

12