Behavioral-Based Performability Modeling and ...

Accepted Manuscript Behavioral-Based Performability Modeling and Evaluation of e-Commerce Systems Pece J. Mitrevski, Ilija S. Hristoski PII: DOI: Reference:

S1567-4223(14)00033-7 http://dx.doi.org/10.1016/j.elerap.2014.06.005 ELERAP 565

To appear in:

Electronic Commerce Research and Applications

Received Date: Revised Date: Accepted Date:

22 November 2013 18 June 2014 18 June 2014

Please cite this article as: P.J. Mitrevski, I.S. Hristoski, Behavioral-Based Performability Modeling and Evaluation of e-Commerce Systems, Electronic Commerce Research and Applications (2014), doi: http://dx.doi.org/10.1016/ j.elerap.2014.06.005

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

© 2014. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/

Behavioral-Based Performability Modeling and Evaluation of e-Commerce Systems Pece J. Mitrevski (corresponding author) Department of Computer Science and Engineering Faculty of Technical Sciences “St. Clement of Ohrid” University Ivo Lola Ribar St, 7000 Bitola Republic of Macedonia [email protected] tel: +389 47 207 702; fax: +389 47 203 370 Ilija S. Hristoski Department of Business Informatics Faculty of Economics “St. Clement of Ohrid” University Gjorche Petrov St, 7500 Prilep Republic of Macedonia [email protected]

Last revised: June 18, 2014 (edited by Robert J. Kauffman) __________________________________________________________________________________ Abstract. Assuring high quality of web services, especially regarding service reliability, performance and availability of e-commerce systems (unified under the term performability), has turned into an imperative of the contemporary way of doing business on the Internet. Recognizing the fact that customers’ online shopping behavior is largely affecting the conduct of e-commerce systems, the paper promotes a customer–centric, holistic approach: customers are identified as the most essential “subsystem” with a number of important, but less well-understood behavioral factors. The proposed taxonomy of customers and the specification of operational profiles is a basis to building predictive models, usable for evaluating a range of performability measures. The hierarchical composition of sub-models utilizes the semantic power of deterministic and stochastic Petri nets, in conjunction with discreteevent simulation. A handful of variables are identified in order to turn performability measures into business-oriented performance metrics, as a cornerstone for conducting relevant server sizing activities. Keywords: Consumer behavior, discrete-event simulation, performability, service reliability, stochastic Petri nets, system availability, service reliability, workload characteristics __________________________________________________________________________________________

1

1. INTRODUCTION As e-commerce becomes mainstream, achieving and retaining customers’ satisfaction regarding the quality of web services being delivered becomes an ultimate goal: the principal element of the contemporary way of doing business on the Internet, a crucial factor that affects both the existence and the progress of Internet companies. The complexity to face and cope with this challenge is even bigger, knowing the fact that e-commerce web services rely heavily on large-scale systems, consisting of thousands of computers, networks, software components, and users. Large systems are inherently complex, whilst the randomness in the way customers demand those web services initiate the problem of managing and planning the capacity of hardware resources. In fact, the true challenge is to achieve an optimal balance among: (1) implementation investments, (2) the costs needed to continually upgrade and maintain a particular e-commerce web site, (3) the performability of the underlying system, (4) the achieved level of customers’ satisfaction, all related to (5) the delivered quality of web services. It can be accomplished solely through a proactive, continuous and scientifically-based capacity planning procedure, not by the application of ad hoc, nonregular procedures and rules of thumb, based on one’s subjective opinion and experience. In that context, Menascé and Almeida (2002) define the capacity planning as being “the process of predicting when future load levels will saturate the system and determining the most cost-effective way of delaying system saturation, as much as possible,” taking into account “the evolution of the workload due to existing and new applications, and the desired service levels.” The absence of such methodology can easily push a company into a “no-win situation:” the inappropriate capacity of hardware resources implies poor performance (long response times), reduced service reliability and system availability (unexpected and frequent crashes, long-lasting downtimes). These, in turn, imply poor QoS levels being delivered to customers, who, due to their increased dissatisfaction, abandon e-commerce web sites massively and prematurely. This can be devastating for the Internet companies, imposing financial and sales losses, decreased productivity, customers’ disloyalty and endangered reputation. The average downtime cost per hour may range from thousands to millions of dollars, depending on the type of industry (Patterson, 2002). Based on a Ponemon Institute (2011) study, Emerson Network Power (2011) released a report that reveals and analyzes the financial impact of infrastructure vulnerability. For instance, downtime can be particularly costly for e-commerce companies with revenue models that depend on the data center’s ability to deliver IT and networking services to customers, with the highest cost of a single event topping US$1 million (more than US$11,000 per minute). According to the Information Technology and Intelligence Corp. (2011), their high availability survey and trends for 2010 revealed that while companies cannot achieve zero downtime, one out of ten companies said it needs greater than 99.999% availability. Moreover, according to Dun & Bradstreet, Inc., which licenses information on businesses and cor-

2

porations for use in business-to-business marketing and supply chain management, 59% of Fortune 500 companies experience a minimum of 1.6 hours of downtime per week. If one takes the average Fortune 500 company (at least 10,000 employees, paid an average of US$56 per hour (US$40 per hour salary plus US$16 per hour in benefits)), the labor part of downtime costs for an organization this size would be US$896,000 weekly, translating into more than US$46 million per year (Arnold, 2010). Faced with ever-increasing financial problems, Internet companies cannot keep pace with everincreasing needs for upgrading the existing hardware infrastructure due to steadily growing service demands, which closes the vicious circle. In order to avoid such an unpleasant scenario, predictive models have to be built and evaluated on a regular basis. To investigate the impact of the inherent performance characteristics of the hardware infrastructure on the quality of service levels being delivered by the e-commerce system, it is inevitable to take into account the unexpected and stochastic changes in the workload, causing service reliability degradation and traffic bursts (or peak rates) that can easily exceed the average traffic levels by the order of ten (Banga and Druschel, 1999; Jiang and Dovrolis, 2005). This is crucial, since even small increases in the traffic intensity can significantly degrade system performance, whereas large ones can even cause functional breakdowns, making the e-commerce systems completely unavailable to customers’ service requests. Analyzing the genesis of the workload itself, a single fact emerges: customers’ online shopping behavior is the main underlying factor affecting the behavior of e-commerce systems’ hardware infrastructure. Actually, customers’ behavior during the course of an online shopping session is the true generator of all performability-related problems that arise at the server-side. It is, to a certain extent, irrational and stochastic, as being a reflection of people’s intrinsically different mindsets, resulting in rather different and unexpected decision-making processes when shopping online. As a result, customers exhibit heterogeneous, yet predictable navigational patterns, when surfing throughout the ecommerce web site for the duration of online shopping sessions. Capturing those navigational patterns and defining them quantitatively, in line with a relevant taxonomy of customers’ behaviors, is a basic premise to performing both modeling and evaluation of a wide gamut of performability measures. Buying decisions have costs and benefits and qualitative rationale for doing them. But, this also involves people making these decisions – people with their own backgrounds, incentives and experiences. The last couple of years have seen an upswing of researchers who are trying to explain these kinds of unexpected behaviors and why consumers don’t make purchasing decisions purely based on micro-economic principles. This blend of the economics and emotions has been subject to investigation of a separate branch in economics, known as behavioral economics (Ariely, 2009; Curran, 2010). What can we learn from behavioral economics to improve the way we make business decisions and improve business successes? According to Latino (2000): “reliability is a combination of two trains of thought. The first is what one calls the hard side of reliability. The hard side consists of all the tools that you physically go and measure reliability with ... The other train of thought is the behavioral side.” 3

What causes people to perform tasks in such a way that it causes performability setbacks? Scarcity is the basic economic problem that ascends because people have unlimited wants, but limited resources. Because of scarcity, various economic decisions must be made to allocate resources in an efficient manner. Standard economics assumes that we are rational – that we know all the relevant information about our decisions, that we can estimate the value of the different options, and that we are unobstructed in assessing the implications of each possible choice (Ariely, 2009). The outcome is that we are supposed to be making rational and pragmatic decisions. On the basis of these assumptions, economists draw in-depth conclusions about shopping trends. We are all far less rational in our decision-making than standard economic theory assumes. Moreover, our irrational behaviors are neither random, nor meaningless – they are systematic and predictable: some people tend to spend more when shopping online than when they count out the cash; automated bill pay, another favorite of retailers and service providers, makes use of a similar lack of “tangible” payment; many decisions involve weighing the value of money now versus its more unclear future value; and, in many situationsconsumers are driven by emotion, not thought (Dratch, 2014). A survey by University of Southern California's Center for the Digital Future (2013) says that 76% of adult Internet users buy online, and for 59% of the Internet users “online purchasing has reduced their buying in traditional retail stores” (the most popular online purchases are for books, clothes, and travel, each reported by 66 percent of online buyers). Nevertheless, 48% of the Internet users are “very concerned or extremely concerned about the privacy of personal information when or if buying online,” whereas 44% are “very concerned or extremely concerned about the security of credit card information when buying online.” To put it differently, some customers know exactly what they want (they are focused and passionate) and frequently place an order, whereas others browse or search repetitively and almost never place an order (they are just curious or reluctant), being affected by those security related issues, anxious about the privacy of their personal information when purchasing from their favorite online stores. In conventional economics, the assumption that we are all rational implies that, in everyday life, we compute the value of all the options we face and then follow the best possible pathway. But, we do not always base these judgments on our essential values – our likes and dislikes. In our holistic approach, we identify customers as the most essential “subsystem” of an ecommerce system, with a number of important, but less well-understood behavioral factors. The modeling of customers’ classes and the specification of the operational profiles is a basis to building predictive models, suitable for evaluation of a number of performability measures. Performance and dependability are often modeled separately, based on the assumption that the faults of an individual subsystem do not necessarily affect performance. Conjointly, performability assessment is defined as “quantifying how well the object system performs in the presence of faults over a specified period of time” (Meyer and Sanders, 2001). In this study, the performability evaluation strategy is based on a hierarchical composition ap4

proach that combines results from: (1) dependability (continuity of and readiness for correct service, based on system availability and service reliability); and (2) performance models. The proposition is that “the faults” in the system are the behavioral factors at play here, that can undermine one’s business goals, and “a failure” is an instance in time when a customer displays behavior that is contrary to some micro-economic principles. Dependable behavior, in our context, refers to human behavior that is situation-adequate, as well as goal-oriented. We combine the best practices of modeling systems’ behavior with stochastic Petri nets and the flexibility coming out from building and executing discreteevent simulations. Performability measures can then be turned into business-oriented performance metrics – a valuable resource for e-commerce web site’s top management teams in creating their policies, including the processes of decision making, both on an operational and strategic level. Nonetheless, these analyses also represent a cornerstone for conducting relevant capacity planning activities – improving e-commerce systems’ performance by horizontal and vertical scaling. The remainder of this paper is organized as follows: Section 2 focuses on related work and identifies most of the pro and con arguments. The rationale behind the introduction of the novel, customercentric, holistic approach is explained in Section 3. Section 4 is devoted to workload characterization, which can be decomposed into two basic parts: a qualitative aspect – modeling the operational environment, and a quantitative aspect – modeling the arrival intensity. A model of the server-side is given in Section 5. In Section, 6 we shift our attention to performability evaluation. We discuss the hierarchical composition approach, identify inter-dependencies and information exchange among various sub-models, perform discrete-event simulations and analyze a range of behavioral-based service reliability, performance and performability measures. Finally, Section 7 concludes the paper with a discussion on the theoretical and practical contributions, limitations and implications for future research. 2. MOTIVATION AND RELATED WORK The assessment of performance, service reliability and system availability as a function of the system’s workload intensity is a key step in the design, analysis and tuning of computer systems, including e-commerce systems. Having defined the measures needed, several general approaches, mutually not exclusive, can be undertaken when it comes to predicting their values, including the following: (1) making an educated guess based on experience with previous, similar systems; (2) building one or more prototypes and taking measurements; (3) using discrete-event simulation to model and evaluate the system; (4) constructing and solving analytical models of the system (Sahner et al., 1996). The current research activities affecting e-commerce systems originate from the period between 1994 and 2002 and are very heterogeneous in relation to the ranges, depths, approaches, techniques, and methodologies being employed. In most cases, the up-to-date research activities have been focused mainly on the performance analysis of e-commerce systems, while dependability issues, encompassing both the service reliability and system availability considerations, have been taken into account either partially, separately and narrowly, or not mentioned at all. This is quite reasonable, since performance-

5

related issues occur more frequently than those affecting dependability, and they are subject to rigorous service-level agreements that impose high quality of service levels of web services being delivered to end users. The most complete and profound work on the performance of e-commerce systems has been concluded over time by Menascé and Almeida (2002, 2000b) and Menascé et al. (1994), through their comprehensive approach encompassing the development of analytical performance evaluation models, primarily based on the probability theory and the theory of queuing networks. By applying the basic operational laws, including the Utilization Law, the Forced Flow Law, the Service Demand Law, Little’s Law, as well as the Response Time Law, they have developed analytical models both on a macro level (whole systems), including finite and infinite queue lengths and/or users, and also on the micro level (particular hardware components), in the case of open and closed systems, taking into account one or more workload classes. In addition, their analytical models have been implemented in Microsoft Excel using Visual Basic for Applications (VBA) programming language. Recently, various classes of stochastic Petri nets (PNs) have been also employed, as well. For instance, Christensen et al. (1999) have developed a predictive model for capacity planning of web servers, based on the usage of the class of timed hierarchical colored Petri nets. Kihl et al. (2002) have proposed computer simulations for performance modeling and evaluation of e-commerce systems, whilst Brosso et al. (2002) have proposed the usage of XML Petri nets – the Petri Net Modeling Language (PNML) – to represent SPNs for web based systems. Additionally, research effort regarding ecommerce systems’ performance has been also made by Menascé et al. (2002b), Kounev and Buchmann (2003), Urgaonkar et al. (2005), as well as Ferrari et al. (2006). Besides the existence of a relatively big and ever-increasing number of e-commerce systems worldwide, there is a considerably smaller number of research endeavors that are entirely and exclusively dedicated to modeling and evaluation of performability measures of such systems, integrating both performance and dependability issues, in spite of the on-going, high dynamics in development and exceptional attractiveness of this substance. One of the rare attempts to address performability of systems that are similar to e-commerce systems on electronic funds transfer systems was by Araújo et al. (2011). The dependability evaluation, encompassing reliability and availability issues have been investigated in a rather limited number of isolated and specific cases (Callou et al., 2012; Hecht, 2001; Microsoft Corp., 2013; Wei et al., 2011). Speaking strictly about the modeling and evaluation techniques for addressing performability, one can state that they are based mostly on a variety of probabilistic, discrete-state models, including: (1) combinatorial reliability models: series-parallel reliability block diagrams (RDBs), fault trees (FTs) and reliability graphs (RGs); (2) directed, series-parallel acyclic task-precedence graphs; (3) Markov and semi-Markov models, including Markov chains, semi-Markov reward models and Markov reward models; (4) product-form queuing networks (PFQNs); and (5) various classes of stochastic Petri nets. For instance, Callou et al. (2012) have used a Petri net-based approach for quantification of a data 6

center’s dependability by applying the hierarchical composition approach, while, yet again, Araújo et al. (2011) have addressed performability of EFTs using stochastic Petri nets. Recently, Robidoux et al. (2010) have proposed extensions in the form of dynamic reliability block diagrams (DRBDs) for addressing dependability issues. These can be converted into colored Petri nets (CPNs), in order to perform dynamic analysis of their behavioral features, along with an analysis of the correctness of the model (Xu et al., 2009). In addition, Wei et al. (2011) have presented a hierarchical methodology that combines the advantages of the reliability block diagrams and the class of generalized stochastic Petri nets (GSPNs) to quantify both reliability and availability. Besides these, in order to address performability issues, attempts have been made to employ Markov reward models (Donatiello and Grassi, 2001; Pattipati et al., 2001), the semi-Markov reward models (Ferrari et al., 2006), the Markov regenerative reward models (Logothetis and Trivedi, 1997), as well as combinatorial multistate models (Veerarghavan and Trivedi, 1994). Each particular approach has its own advantages and drawbacks in terms of user-friendliness, ease of construction and use, efficiency and accuracy of solution algorithms, and availability of software tools for obtaining a numerical solution. No single approach is the best, or even inevitably suitable for every system and every measure of interest. For instance, analytical models require relatively high level of abstraction of the real system in order to be computationally tractable, that is, mathematically solvable, which imposes the problem of inaccuracy. However, once an analytical model is set up and either a closed-form solution or a numerical solution is obtained, it is both easy and fast to perform predictions, trade-off studies, what-if and sensitivity analyses, and compare design alternatives regarding the capacity planning issues. Discrete-event simulation seems to be a flexible approach, since it allows capturing system’s behavior in an arbitrary level of details or abstraction. Nevertheless, building up a simulation model can be a very time-demanding task, due to its complexity. Yet another drawback is the fact that simulation runs are, in general, long-lasting activities in order to assure statistically significant results. The combinatorial reliability models, including reliability block diagrams, fault trees and reliability graphs are straightforward and easy to understand, but they cannot represent neither the dependent behavior of components, nor they can address performance considerations. Alternatively, the wide gamut of Markov processes’ classes offer great flexibility for modeling all performability components at once, including performance, reliability and availability, but they are not always intuitive and perceptive. In addition, the size of their state-space grows much faster than the number of systems and components involved in the analysis, thus making the model specification and analysis exceptionally difficult. Finally, queuing networks are extremely intuitive to be developed, but only those having a product-form yield an efficient solution method. Besides, they cannot capture the behavior of systems exhibiting features like parallelism, simultaneity, synchronicity, blocking, mutual exclusion, etc. All of these features can be successfully addressed by various classes of stochastic timed Petri nets, which 7

are graphical and mathematical modeling tools that possess semantic power for modeling any kind of systems with an arbitrary complexity. Besides the already known methodologies for obtaining analytical solution (both steady-state and transient solution) of certain classes of stochastic Petri Nets, there is a plethora of dedicated software tools that support obtaining a numerical solution, and are also usable for addressing various performability measures, like SHARPE (Sahner et al., 1996), DSPNexpress (Lindemann et al., 1999), and TimeNET (Zimmermann and Knoke, 2007). A comprehensive survey of such software tools can be found in Haverkort and Trivedi (2001). 3. THE RATIONALE BEHIND THE CUSTOMER-CENTRIC HOLISTIC APPROACH Several significant conclusions can be drawn from the survey in the previous section, including the following: (1) Addressing e-commerce systems’ performability is a recent, yet not enough investigated and elaborated issue, in spite of the ever increasing number of such systems worldwide. (2) In most cases, the focus has been put solely on obtaining performance metrics, whilst dependability considerations either have not been taken into account at all, or they have been elaborated partially and isolated from performance. (3) The performability research has been carried out very rarely, affecting systems that are, in a way, similar to e-commerce systems. (4) Besides the existence of a variety of approaches and techniques for addressing performability issues, none of them is exclusively customer-centric: none of them is based purely and solely on the customers’ dynamic and stochastic behavior during online shopping sessions. Despite the fact that e-commerce systems are actually computing systems, they possess many unique characteristics, reflecting the e-commerce client-server paradigm, the specifics of the online interaction with customers via Internet, as well as the internal multi-tier architecture of web sites. In addition, e-commerce systems include many communication components that cannot be seen with ordinary computing systems, whereas e-commerce system users exhibit a dynamic and stochastic behavior, causing service reliability degradation and Internet traffic bursts. Since e-commerce systems present a particular class of computing systems, they have to be treated in a unique manner, in order to be modeled and evaluated properly. In this context, we expand the meaning of the term “e-commerce system” by including customers as its fundamental and inevitable ingredient, besides the hardware resources residing at the ecommerce web site. The notion of an “e-commerce system” now refers to the unity between the humans and the machine via Internet, and their mutual interaction, as well as interoperability. This is quite natural and understandable reasoning, since e-commerce web site’s hardware infrastructure cannot perform without customers, and, vice versa, they cannot exist or perform without e-commerce hardware infrastructure. The resulting synergy that comes out from the interaction between these two elements drives the modern e-commerce paradigm. By including customers, the notion of e-commerce systems complies entirely with the Blanchard’s (1991) definition of a system, as being “a collection of multiple elements that are interrelated and work in cooperation, functioning together to achieve a de-

8

sired goal or specific objectives.” We employ a somewhat hybrid evaluation methodology that combines the best practices of modeling systems’ behavior with stochastic Petri nets and the flexibility coming out from building and executing discrete-event simulations. To be more precise: (1) we utilize the semantic expressing power of the classes of generalized stochastic Petri nets (GSPNs) (Ajmone-Marsan et al., 1984) and deterministic and stochastic Petri nets (DSPNs) (Choi et al., 1993; Ciardo and Lindemann, 1993) to capture the crucial features of a system’s behavior, using the basic graphical syntax primitives inherent to these two classes as basic building blocks. However, instead of solving the resulting models analytically, (2) we turn to SimPy/Python simulation programming environment (Matloff, 2008; Müller and Vignaux, 2003) to obtain a numerical solution. 4. WORKLOAD CHARACTERIZATION Throughout all segments within this research, the underlying idea is that the dynamic and stochastic online behavior of customers is the main reason for overloading the hardware infrastructure of ecommerce systems, which results in series of phenomena, including the unexpectedly prolonged response time, as being one of the most prominent performance indicators. Therefore, it is crucial to precisely describe the global workload of an e-commerce web site in terms of its main components, a process known as workload characterization (Menascé, 2003; Menascé and Almeida, 2002, 2000a, 2000b; Menascé et al. 2002, 1999). In fact, the workload characterization is the key aspect that defines the service demand: the demand for various hardware resources (CPU, HDD, RAM) of an e-commerce system. The workload characterization can be decomposed into two basic components: modeling the operational environment (a qualitative aspect), and modeling the arrival rates (a quantitative aspect). 4.1. Modeling the Operational Environment The “operational environment” concept, in a general sense, could embrace different dimensions – in this particular case, we partition the “input space” of the client-side of an e-commerce system by grouping customers that exhibit as nearly as possible homogenous online shopping behavior. Existing taxonomies. A number of authors have independently tried to identify some basic classes of customers, given in Table 1. Two behavioral aspects are important: buying behavior (purchasing behavior), and online behavior. Nevertheless, most of the analyses have been performed by taking into account solely the economic motivations for online shopping. A segmentation of online consumers provides a valuable understanding of to whom online marketers might direct their offers. But, what usually is known about these segments is the hours active/month, unique domains accessed/month, pages accessed/month, percent buying, that is, not much about their online behavior.

9

Table 1. Some existing taxonomies of customers Kung (2011) Window shoppers Comparers Dedicated shoppers

Shergill and Chen (2005) Trial Occasional Frequent Regular

Davis (2007) Ready-to-buy Potential Repeat Discount Impulse

Hunter (2009) Loyal customers Discount customers Impulse customers Need-based customers Wandering customers

Williams (2008) Tire-kickers Shopping around Comparison shoppers Desperate customers Non-customers

Nam and Chung (2000) Simplifiers Surfers Connectors Bargain shoppers Routine followers Sportsters

Cuthbert (2000) Web Newbies Reluctant shoppers Bargain shoppers Surgical shoppers Enthusiast shoppers Power shoppers

Kau et al. (2003) On-off shoppers Comparison shoppers Traditional shoppers Dual ehoppers E-laggards Information surfers

Our taxonomy. Menascé (2003) points out that workload characterization can be done at multiple levels and time scales, for example, at the business, session, function and HTTP request layer. Our workload characterization encompasses the second and the third layer, since we assess the customers’ behavior during online shopping sessions, while they invoke various specific e-commerce functions (login, search, browse, add-to-cart, and alike). Despite the fact that the number of sessions submitted to an e-commerce site is huge, and regardless of the different e-shoppers’ mindsets and purchasing behaviors, it is still possible to identify specific groups of mutually similar sessions. This can be done by analyzing web server’s HTTP logs by using clustering algorithms (Menascé et al., 2002, 1999). In such a way, different customer’s classes can be defined as being sets of customers exhibiting similar navigational patterns, a specific and unique online shopping behavior during web sessions. Each identified class can be graphically visualized, in most convenient way, by its corresponding customer behavior model graph (CBMG) (Menascé and Almeida, 2002, 2000b). To gain insight into these segments, we analyzed online behavior using a sample of active online consumers of a regional e-commerce online bookstore. Based on the analytical data from 2,000 registered byers for 10,000 offered items, we draw conclusions on the behavior of the average e-commerce user in the region, by using parallel threshold non-hierarchical clustering method (Reddy and Acharyulu, 2009). Complementary findings of the six months research on the strategies applied on the regional e-commerce online bookstore Kupikniga.mk have been published by Antovski and Armenski (2012). Customers differ, and various customers behave in a rather different manner while shopping online. So a qualitative analysis of customers’ typical behaviors has been carried out, regardless of their socio-economic, demographic, ethnic, cultural and other characteristics, their innate and gained habits to go for shopping, as well as regardless of their immediate motivation to shop online. As a result, we have identified and qualitatively described five generic customer’s classes (simultaneously determined as several cluster centers), which include the following: (1) passionate, (2) fo-

10

cused, (3) reluctant, (4) curious, and (5) selective customers. Each particular class represents a single typical online shopping behavior pattern. Minimizing intracluster variance of the average distance of all points of a cluster, while maximizing intercluster variance of the distance between clusters, is the purpose of clustering (Menascé et al., 1999). To achieve this goal, the number of clusters could easily be made equal to the number of points. However, one needs to select a relatively small number of clusters in order to get a compact representation of the workload: the ratio between the intra and intercluster variance, and the ratio between the intra and intercluster coefficient of variation, are important in determining the quality of the clustering process. Our taxonomy of customers is quite general and fine-grained enough, at the same time – however, it might be closely related to the bookstore case. As e-commerce sites become more sophisticated, they can process their logs to identify user profiles based on the navigation and buying patterns, resulting in an expanded or reduced number of clusters. Yet, the proposed taxonomy is still simple and compact to capture the most typical online behaviors that can be encountered in any B2C e-commerce web site, be it an existing one or still in a design phase. In order to quantitatively capture different online behavior patterns, a suitable predictive model has to be constructed, as well. The DSPN model of the online shopping behavior, originally proposed by Mitrevski and Hristoski (2011), is used as a fundamental building block during the construction of a series of stochastic Petri net models for the evaluation of a plethora of performability measures. This stochastic model (Fig. 1), which reflects the decision-making process during an online shopping session, depicts the complexity of the interaction between a typical customer and an e-commerce web site in a quite simplified and clear way. For simplicity, while still preserving generality, it has been supposed that an customer can invoke some e-commerce functions where he/she resides for a certain time, the think time (e.g. “search” and “checkout” with exponentially distributed think time with rates λ and µ, respectively). For the sake of clarity, we briefly turn to the dynamics of the DSPN model: the token in place PSEARCH denotes that a customer is about to make a new search through the products catalogue, which is a time-consuming activity. The firing time of transition TEND_SEARCH is exponentially distributed with rate λ. The product is either found (firing of transition TFOUND with probability p FOUND), or not found (firing of transition TNOT_FOUND with probability 1-pFOUND). In addition, it is the customer’s decision to add the product to the shopping cart (firing of transition TADD with probability pADD) or not (firing of transition TNOT_ADD with probability 1-pADD). The firing of immediate transition TADD leaves a token in place PIN_BASKET (removes all (zero or one) tokens and puts back one), just to indicate that the shopping cart is not empty (the number of products is irrelevant). Regardless of the search outcome, the customer can either (a) make a new search (probabilities pCONTINUE_1, pCONTINUE_2 and pCONTINUE_3, respectively), (b) end shopping without placing an order (probabilities pEND_1, pEND_2 and pEND_3, respectively), or (c) proceed to checkout provided that the shopping cart is not empty (probabilities pCHECKOUT_1, pCHECKOUT_2 and pCHECKOUT_3, respectively). One can expect the probability of a new search to be the 11

highest when no product was found, and the probability of ending without placing an order to be the lowest when the shopping cart is not empty. On the other hand, checking-out is also a time-consuming activity. The firing time of transition TReview-Submit-Verify is exponentially distributed with rate µ. The consumer reviews and submits delivery address and credit card information, credit card authorization is verified, and order is placed (token in place PORDER_PLACED). Right away, the system sends the customer an e-mail confirmation regarding the purchase (firing of transition TE-MAIL) and the lifecycle of the ecommerce application ends (token in place PEND). While the customer stays in any of the tangible states, a timeout mechanism can be activated due to his/her inactivity (firing of transition TTIMEOUT), meaning that a deterministic time τ (with resampling policy) has run out and the session has been terminated forcibly and prematurely by the server-side of the e-commerce system. Otherwise, the session terminates regularly (in a timely manner), with two possible outcomes: either successfully (with an order placed), or unsuccessfully (without placing an order). However, in the latter case, two situations are possible, too: either (1) the customer has previously put one or more products into the shopping cart, but he/she has not bought them; or (2) the customer has left the virtual store without putting anything into the shopping cart. The DSPN model is suitable for obtaining various client-side related performance metrics, since it is also based on a CBMG of a particular e-commerce web site, but includes a stochastic temporal specification of the think times (Fig. 2). One should note that the search/browse behavior varies over time, so this process is not stationary over large timescales. It is anticipated that a better understanding of how individuals use the web could lead to improved design of web sites and web interfaces, but then again online merchants are interested in click rates because they contain information about an individual’s interest in the content of a web page (Scott and Smyth, 2003). A user browses the products catalogue when he or she is looking around to see what is there, usually moving with a mind open to discover new things. In contrast, a user searches when he or she needs to locate a specific product. These events appear according to a two-state Markov modulated Poisson process (MMPP) – a doubly stochastic Poisson process where the rate process is determined by a continuous-time Markov chain. A two-state MMPP means that the Markov with intensity r1, and transits back with intensity r2. When the MMPP is in state “search,” the stochastic process is a Poisson process with rate λ1, and when the MMPP is in state “browse,” rate λ2 is used. Nevertheless, one can still deal with the mean rate in a two-state MMPP (Heffes, 1980), given as follows:

λ=

λ1r2 + λ2 r1 r1 + r2

(1

. )

Consequently, the semantics of the timed transition TEND_SEARCH (Fig. 1) resembles, more or less, the meaning of the verb “to search” – “try to find something by looking or otherwise seeking carefully and thoroughly” (“browse” and/or “search by keywords”), and does not jeopardize the correctness of the results obtained. Based on the qualitative, narrative description of each particular customer class, a quantitative def12

inition of each of them has been made, using the parameters of the DSPN model (the firing rates of the exponential transitions and the weights of the immediate transitions). Table 2 summarizes the qualitative and quantitative definition of customers’ classes, by setting the intrinsic parameters of the DSPN model. This way, one can simulate the online behavior of an individual customer belonging to a specific class (by varying the numeric parameters within the DSPN model). For instance, the probability pEND_2 of ending an online session without placing an order, after putting an item in the shopping cart, is considerably bigger with reluctant, and even more with selective customers, while the probability pCHECKOUT_2 of ending an online session with placing an order is noticeably lower with selective, and even more with curious customers (to name a few).

Fig. 1. DSPN model of customer’s online shopping behavior (Mitrevski and Hristoski, 2011)

13

Fig. 2. CBMG corresponding to the DSPN model of customer’s online shopping behavior Table 2. Defining customers’ classes using the probabilities within the DSPN model

Curious

DSPN model parameters

Qualitative Description λ µ pFOUND pNOT_FOUND pADD pNOT_ADD pCONTINUE_1 pEND_1 pCHECKOUT_1 pCONTINUE_2 pEND_2 pCHECKOUT_2 pCONTINUE_3 pEND_3 pCHECKOUT_3

Focused

Customer Classes Passionate

Reluctant

Selective

Rarely initiate a Browse and search Know exactly what Multiple searches / Moderate number of session / repetitively / long they want / short longer session durasearches / short session durasession duration / session duration / tion / not sure about the tion / not sure about almost certainly place choice / rarely place almost never place place an order the choice / very an order frequently an order an order rarely place an order

1/60 1/180 0.61 0.39 0.04 0.96 0.89 0.05 0.06 0.89 0.05 0.06 0.89 0.05 0.06

1/60 1/180 0.79 0.21 0.84 0.16 0.32 0.17 0.51 0.32 0.17 0.51 0.32 0.17 0.51

1/60 1/180 0.91 0.09 0.96 0.04 0.36 0.01 0.63 0.36 0.01 0.63 0.36 0.01 0.63

1/60 1/180 0.48 0.52 0.51 0.49 0.26 0.25 0.49 0.26 0.25 0.49 0.26 0.25 0.49

1/60 1/180 0.24 0.76 0.28 0.72 0.14 0.60 0.26 0.14 0.60 0.26 0.14 0.60 0.26

Given the customers’ classes being defined previously, one has to specify how they comprise the workload. An operational profile (OP) defines, in percentages, the participation of each particular ecommerce class within the workload mix during a certain period of time. Let N denote the total number of distinctive customers’ classes being identified. Intuitively, the operational profile can be specified by a corresponding row-vector P = [p1, p2, p3, …, pN], having N elements, representing the probabilities of each particular customer class to participate within the overall workload mix, given that p1 + p2 + p3 + … + pN = 1.

14

Within each specific operational profile, customers, belonging to specific classes, participate in various proportions, whilst each particular class is characterized by a specific frequency (intensity) and a specific sequence of invoked specific e-commerce functions (login, register, search, add-to-cart, checkout, etc.). Each invocation of any of these functions results in generating and sending a corresponding HTTP request from the customer’s browser towards the server(s) of the e-commerce system. The system has to accept and process all of these HTTP requests, and each of them has to be answered by sending a corresponding HTML code back to the client’s computer. By generating a unique service demand, each operational profile poses a characteristic and rather different workload to the ecommerce system, as it is its own fingerprint. To rationalize the needs of our research, eight characteristic operational profiles have been defined (Table 3), including the following: OP#1(100%; 0%; 0%; 0%; 0%), OP#5(20%; 20%; 20%; 20%; 20%), whilst the rest six of them can be represented by a vector (80%; a%; b%; c%; d%), where abcd are combinations of the values 0 and 10, of class 4, given that a + b + c + d = 20. The underlying idea is to carry out various evaluations for operational profiles in which the percentage of the most desirable class of customers (the class of passionate customers) is 20% less than that one in the OP#1, and also to investigate the impact of the equal distribution of the customers’ classes within the workload mix. Table 3. Definition of operational profiles for the GSPN model of the operational environment

Transition TCLASS_1 (Passionate) TCLASS_2 (Focused) TCLASS_3 (Reluctant) TCLASS_4 (Curious) TCLASS_5 (Selective)

OP#1 Weight 1.0 0 0 0 0

Transition TCLASS_1 (Passionate) TCLASS_2 (Focused) TCLASS_3 (Reluctant) TCLASS_4 (Curious) TCLASS_5 (Selective)

OP#5 Weight 0.2 0.2 0.2 0.2 0.2

Characteristic Operational Profiles OP#2 OP#3 Weight Weight 0.8 0.8 0.1 0 0.1 0 0 0.1 0 0.1 OP#6 Weight 0.8 0.1 0 0.1 0

15

OP#7 Weight 0.8 0 0.1 0.1 0

OP#4 Weight 0.8 0 0.1 0 0.1 OP#8 Weight 0.8 0.1 0 0 0.1

Operational profile Transition

Weight

TCLASS_1 TCLASS_2 … TCLASS_N

w1 w2 … wN

N

∑w

i

=1

i =1

Fig. 3. GSPN model of the operational environment Fig. 3 shows the formal GSPN model of the operational environment. A customer, represented by a token in the place PSTART, is going to start his/her online session, by firing the immediate transition TSTART. The selection of the corresponding class is done when the token arrives at the place PSELECT, when N immediate transitions TCLASS_1 , TCLASS_2, …, TCLASS_N become concurrently enabled, each having specific firing weight w1 , w2, …, wN. The probability of firing the immediate transition TCLASS_i , i = 1, …, N, represents the probability of occurrence of a class i customer, given by the following expression:

P (TCLASS _ i ) =

w (TCLASS _ i ) N

∑ w (T

CLASS _ j

j =1

=

wi

.

N

) ∑w

(2)

j

j =1

When a transition TCLASS_i fires, a single token appears in the place PEXECUTE (the “Class i” customer can carry out his/her online session) and i tokens appear in the place PCLASS (due to multiplicities of the outgoing arcs originating from the transition TCLASS_i towards the place PCLASS), thus identifying the corresponding class that the new customer belongs to. Strictly adhering to the formal definition of SPNs, the row-vector [w1, w2, …, wN] represents the operational profile, timed transitions have marking-dependent firing rates, and immediate transitions have marking-dependent weights (they depend on the number of tokens in PCLASS in all the models that include a sub-model of the operational environment: Figs. 10, 12 and 13). 4.2. Modeling Arrival Intensity Besides the identification of basic customer classes and the specification of various operational profiles, the process of workload characterization also includes the rates at which particular shopping sessions have been initiated at the e-commerce web site (customers’ arrival rates). The process describing customers’ arrivals is the Poisson process, since: (1) there is a zero probability of two arrivals at exactly the same instant of time; (2) the number of arrivals in the future is independent of what have happened in the past; and (3) the number of arrivals in the future is independent and identically distributed (i.i.d.) random variable over time, so the process is stationary. The time elapsed between two 16

consecutive arrivals is exponentially distributed, with a mean 1/α, where α denotes the intensity of arrivals of customers in the system. As a matter of fact, α is the arrival rate in a general context that applies to all customers. If there are N classes of customers being defined in total, and p1, p2, …, pN are the probabilities that correspond to the participation of each particular class within a specific operational profile, given that p1 + p2 + ... + pN = 1, then customers belonging to class i (i = 1, …, N) arrive with an intensity αi = pi * α (Bolch et al., 2006). 5. MODEL OF THE SERVER SIDE E-commerce sites, especially those of the leading Internet retailers, have multi-tier hardware architecture, consisting of multiple servers, distributed throughout two or more LAN segments. In such a way, the site’s architecture becomes more scalable, more flexible, more reliable and highly available (Schroeder et al., 2000). We keep in mind the simplest possible two-tier hardware architecture of a

medium-to-large scale e-commerce system, consisting of a front-end server (FES), a web server (WS), a database server (DbS), an application Server (ApS) and an authentication server (AuS), distributed into two high-speed LAN segments (Fig. 4).

Fig. 4. A schematic representation of e-commerce server-side hardware architecture Each HTTP request may need several operations, based on the types of processing by some of the back-end servers, before completing. Moreover, a request may have to be processed more than once in a particular back-end server. Table 4 summarizes the different back-end servers (Kihl et al., 2002), along with the mean processing time and standard deviation (σ). We assume that the processing time has a normal distribution and can vary up to ±10% (±3σ) of its mean. In addition, the propagation time (transmission delay) via Internet WANs has been considered as a random variable with a Normal distribution and parameters N(µ = 0.5 s; σ = 0.13 s), which yields 99.7% of the values within the interval [0.1, …, 0.9] s. At the server-side, the transmission delays between the servers have been neglected, due to the usage of high-speed LAN segments, as well as the propagation times in routers and firewalls.

Table 4. Servers’ stochastic processing time parameters

17

Server Front-end server (FES) Web server (WS) Database server (DbS) Application werver (ApS) Authentication server (AuS)

Mean processing time (ms) 1.0 10.0 5.0 10.0 10.0

Standard deviation (ms) 3.33E-05 3.33E-04 1.66E-04 3.33E-04 3.33E-04

Range of values (ms) [0.9, …, 1.1] [9.0, …, 11.0] [4.5, …, 5.5] [9.0, …, 11.0] [9.0, …, 11.0]

6. PERFORMABILITY MODELING AND EVALUATION In this section, we will show how the complex concept of performability can be assembled in a bottom-up fashion, using a hierarchical composition approach (Fig. 5). Note that the hierarchical composition is a means for defining how a model is composed from other (sub-)models, and should not be confused with hierarchical Petri nets; nor should sub-models be confused with subnets (generally speaking, they do not even have to be “nets” of any kind).

Fig. 5. Inter-dependencies and information exchange among various sub-models The modeling has been done by applying stochastic Petri Nets (used in a CBMG manner, but relying on the use of random think times and their distribution functions, instead of using a matrix that represents the average think times between states of the CBMG), whereas the evaluation of the corresponding metrics has been performed by discrete-event simulation in SimPy/Python – an objectoriented, process-based, general-purpose discrete-event simulation package based on standard Python programming language. Discrete-event simulation (DES) utilizes a mathematical/logical model of a physical system that portrays state changes at precise points in simulation time. Both the nature of the state changes and the time at which the changes occur, require precise description. Within DES, time advances not at equal size time steps, but rather until the next event can occur, so that the duration of activities determines how much the clock advances. All of the following aspects of the e-commerce paradigm have been successfully modeled in SimPy: the client-side (the customer’s online shopping behavior); the server-side (the hardware configuration of a typical ecommerce web site, on a system level); various classes of customers (the qualitative component of the workload specification); the workload intensity (the quantitative component of the workload specification); the HTTP requests generation for the main functions invoked by the customer; the propagation delays of the HTTP requests being forwarded from clients’ browser towards e-commerce servers and the delays of the corresponding responses, via Internet; and evaluation of plethora of metrics for both

18

client and server-side. Our SimPy implementation internally consists of three processes (named Source, Customer, and Request), along with their corresponding process execution methods (PEMs) (Fig. 6).

Fig. 6. A schematic representation of the internal structure of the SimPy simulation model The Source process implements the generation of customers according to the given arrival rate, and according to the probability distribution of their type. It also defines the parameters of the simulation runs. The Customer process implements the client-side, the customer’s online behavior during the online session. Finally, the Request process implements the server-side, with propagation of each particular HTTP request through the e-commerce web site’s hardware infrastructure, being already specified. Each web site’s server has been modeled on a system level, rather than on a component level. Since HTTP requests are processed by a particular server in a FIFO (FCFS) manner, each of them has been modeled as a resource facility with an infinite queue length, for simplicity reasons. Random variates are provided by the standard Python random module, which implements pseudo-random number generators for various distributions. 6.1. Behavioral-Based Service Reliability Wang and Trivedi (2009) interpret the concept of user-perceived service reliability as follows: “During the user interaction (session) with the system, the user issues multiple tasks (or requests) at different time points for different services in the system. The user-perceived service reliability is the probability that all tasks in the user session are successfully completed.” In our case, we look at it in the opposite direction, from a system’s perspective, using the “language” of business-oriented performance metrics: if not all tasks in an customer’s session are successfully completed, both the buy-tovisit ratio and the revenue throughput decrease. Moreover, during a session, all those multiple requests/tasks consume system resources, possibly reducing service accessibility – the ability of other customers to initiate a transaction in the service when desired (load levels saturate the system). Yet again, in Wang and Trivedi’s own words, “longer user think time can decrease the service reliability.” These are the main reasons why we adopt the probability for a successful outcome of a session as a measure of the behavioral-based service reliability. Having defined customers’ classes, a GSPN model of a single customer’s online shopping session (Fig. 7) has been derived directly from the DSPN model. It does not include the timeout mechanism, and contains two absorbing places: PORDER_PLACED (a successful session) and PEND (an unsuccessful 19

session). Therefore, it is suitable for assessing the conditional probability for a successful outcome, by customers’ classes.

Fig. 7. GSPN model of a single customer’s online shopping session The Bowman-Shelton test of normality has shown that the conditional probability for a successful outcome of sessions follows the normal probability distribution for each particular class of customers (Fig. 8) (200 simulation runs were performed per class, each simulating a time interval of 2 hours, with α = 1.0 customers/s). It has been concluded that the highest average successful rate (0.853 s−1) is evident for the class of passionate customers, followed by the classes of focused (0.568 s−1) and reluctant (0.198 s−1) customers. Contrary to some initial expectations, the class of curious e-shoppers (0.099 s−1) has exhibited an insignificant, yet a higher average successful rate than the class of selective eshoppers’ (0.025 s−1), a fact that confirms the sorting of preferable customers’ classes in a descending order, as follows: (1) passionate; (2) focused; (3) reluctant; (4) curious; and (5) selective ones. Such arrangement of customer classes is necessary when characteristic operational profiles are being defined.

20

34

34

35

32

~ N (µ; σ)

Relative frequency

30

25 21

21

20

18 15

15 9

10

5

8

3 1

2 0

0

0

1

1

0 0,97600,9767

0,97730,9780

0,97870,9794

0,98010,9808

0,98150,9822

0,98290,9836

0,98430,9850

0,98570,9864

0,98710,9878

Probability range

Fig. 8. Approximation of conditional probability for successful outcome of session, for the class of passionate customers The analysis of the partial coefficients of elasticity, obtained by series of simulation runs in which the conditional probability for a successful outcome of sessions (dependent variable) has been evaluated against the values of the weights of immediate transitions (independent variables) within the GSPN model in Fig. 7, has shown that the successfulness of sessions is a result of the complex process of decision-making, and is least dependent on the probability of putting a product into the shopping cart (pADD), and the probabilities of initiating an order (pCHECKOUT_1, pCHECKOUT_2, and pCHECKOUT_3) representing the weights of immediate transitions within the GSPN model. The box and whiskers plot (Fig. 9) confirms the definitive sorting order of customers’ classes. Classes of e-Customers

Curious Selective Reluctant Focused Passionate 0,0

0,1

0,2

0,3

0,4

0,5

0,6

0,7

0,8

0,9

1,0

Conditional probability

Fig. 9. Distributions of conditional probability for session outcome, by customer classes Next, using the GSPN behavioral-based service reliability model (Fig. 10), it is possible to evaluate the total probability for a successful outcome of a session, for various operational profiles. This model combines the GSPN model of the operational environment in Fig. 3 and the GSPN model of a single customer’s online shopping session in Fig. 7. Again, as in the case of the conditional probability, the application of the Bowman-Shelton test of normality has shown that the distribution of the total probability follows the Normal distribution. As expected, the OP#1, which is 100% comprised of the class of passionate customers, has resulted with

21

the highest total probability ( PSUCCESS = 98.27%) for a successful outcome of sessions, while OP#5, which defines a mix of uniformly distributed classes of customers, has resulted with the lowest total probability ( PSUCCESS = 50.48%) for a successful outcome of sessions. Within operational profiles where the participation of the passionate customers’ is 20% less than in OP#1, the average value of the total probability for a successful outcome of sessions varies between 84.45% and 91.97% (Fig. 11).

Fig. 10. GSPN behavioral-based service reliability model

22

OP #8

Operating profiles

OP #7 OP #6 OP #5 OP #4 OP #3 OP #2 OP #1 0,4

0,5

0,6

0,7

0,8

0,9

1,0

Total probability

Fig. 11. Distributions of probability for successful outcome of a session, by operational profiles 6.2. Performance The performance sub-model considers only the execution behavior of a shopping session, independently of the outcome result. Therefore, the complexity of the decision-making process during an online shopping session is “hidden” by refinement of the Petri net-based system specification and including a number of timed transitions – TSEARCH_WITHOUT_ITEMS_ADDED, TSEARCH_WITH_ITEMS_ADDED, TEND_ITEMS_NOT_ADDED, TEND_ITEMS_ADDED, TFOUND_SELECT_ADD and TCHECKOUT (with exponentially distributed firing times with rates λ, λ, η, η, ψ and φ, respectively), leading to increased number of tangible states with nonzero sojourn times. The success of the user session depends not only on the number of requests during the session, but also on the user think times. In addition, the mechanism of timing failure (firing of deterministic transition TTIMEOUT) comes into play – a timing failure happens when a customer’s session has been aborted prematurely and forcibly by the e-commerce system, due to his/her inactivity. Such an irregular behavior can be put and analyzed in a broader context, as well. Namely, the customer is directly responsible for triggering a timing failure of the shopping session, due to his/her inattentive behavior. Since the online shopping session is the basic interaction that occurs between the client and the server processes within the e-commerce system, the customer is directly inducing performability degradation (i.e. “faded” perception that the system performs “well” over a specified period of time) by causing a timing failure. In other words, “shorter user think times result in shorter session duration, and the system can stay up during the session and serve all the requests successfully” (Wang and Trivedi, 2009). Since the model includes a deterministic time (τ) that implements the timeout mechanism, the corresponding stochastic Petri Net model necessarily belongs to the class of DSPNs. The DSPN representation in Fig. 12 is suitable for evaluating the mean session length, regardless of the outcome (successful/unsuccessful), in the presence of timing failures (multiple customers and concurrent session execution was used during 250 simulation runs, each simulating a time interval of 2 hours, with τ = 600 s 23

and marking-dependent firing rates η, ψ and φ (so they depend on the number of tokens in PCLASS)). The duration of each individual shopping session is accumulated, regardless of its outcome (with or without timing failure). At the end of the simulation, it is divided by the total number of sessions in order to calculate the mean session length. In a session, the remaining firing time (RFT) of the deterministic transition TTIMEOUT is resampled (reset to τ) in each new tangible marking of the Petri Net (possibly the same). The firing of TTIMEOUT indicates that the server is unable to process further customer’s requests because the maximum allowed time between requests elapsed (token in place PTIMEOUT_EXPIRED).

Fig. 12. DSPN performance model (evaluation mean session length with timing failures) Slightly modified, the DSPN representation in Fig. 13 assumes rescheduling a new session at the end of each regularly terminated session (sequential session execution was used during 1,000 simulation runs, each simulating a time interval of 2 hours, with τ = 600 s). Such an assumption is suitable for

24

evaluating two performance metrics, including the mean time to timing failure, as well as the mean number of sessions until timing failure.

Fig. 13. DSPN performance model (mean time and mean number of sessions to timing failure) OP #8

OP #7

Operating profiles

OP #6

OP #5

OP #4

OP #3

OP #2

OP #1 240

260

280

300

320

340

360

Duration [s]

Fig. 14. Mean session length with timing failures, by different operational profiles A propos to the performance measures, simulations have shown that the mean session length is shortest with operational profiles OP#4 (254.30 s) and OP#8 (254.87 s), while OP#5 exhibits the longest mean session length (324.00 s) (Fig. 14).

25

Further on, an evaluation of the mean time to timing failure has been performed. It is highest with OP#5 (311.80 min), whereas OP#1 (122.07 min) and OP#2 (124.55 min) exhibit lowest mean time to timing failure (Fig. 15). Finally, the mean number of sessions until timing failure has been evaluated, too. It is highest with OP#5 (77.92), which is almost as twice as large a value than with other operational profiles (33.40 – 40.40) (Fig. 16).

Fig. 15. Distributions of the mean time to timing failure, by operational profiles

Fig. 16. Distributions of mean number of sessions to timing failure, by operational profiles 6.3. Performability The behavioral-based service reliability, the performance, and the e-commerce system’s availability, are all brought together under the performability umbrella. The analysis of performability measures is carried out in two specific cases: (1) when the e-commerce system is comprised of a single module (a standard configuration); (2) when, besides the main module, there is an additional, redun26

dant (spare) and non-active module within the e-commerce system, waiting to be activated in the case of a failure of the main module (a cold standby configuration). The corresponding GSPN models of both configurations are shown in Fig. 17 and Fig. 18, respectively. One should note that both models incorporate GSPN sub-models of e-commerce system’s availability. The evaluation of performability measures has been carried out taking into account the mean time to failure (MTTF) and the mean time to repair (MTTR) parameters of the e-commerce system. In the case of the cold standby configuration, besides the previously mentioned ones, the mean activation time (MAT) of the spare module has been taken into account, as well. The actual values of all of these parameters correspond to the specification of a “well managed system” (Araújo et al., 2011): MTTF_main = 360 hours, MTTR_main = 8 hours, MTTF_spare = 140 hours, MTTR_spare = 10 hours, MAT = 0.125 hours. Both of the performability models comply with the hierarchical composition approach, since there is an information exchange between them and the previously introduced behavioral-based service reliability and performance models. The firing rates of the exponential transitions TSESSION (Fig. 17) and TSESSION_MAIN, TSESSION_SPARE (Fig. 18) are based on the mean session length metrics being evaluated with the performance sub-model, whilst the firing weight of the immediate transition TORDER_PLACED (Fig. 17, Fig. 18) is, in fact, equivalent to the total probability for a successful outcome of a session, being evaluated with the behavioral-based service reliability sub-model. Borrowing the queuing terminology, the exponential transitions are said to be of infinite server type (Ajmone-Marsan, 1990). For these two configurations, the following performability measures have been evaluated: (1) mean time to an unsuccessful session; (2) mean number of successful sessions until an unsuccessful session. Customers with different online shopping behavior arrive randomly and independently, according to the operational profile (multiple customers and concurrent session execution was used during 250 simulation runs, each simulating a time interval of 30 days, with arrival rates α = 0.01, 0.05, 0.1, 0.5, 1, 5, 10, 15, 20, 25 and 30 s−1).

27

Fig. 17. GSPN performability model with a single e-commerce system’s module

Fig. 18. GSPN performability model for the cold standby configuration Fig. 19 shows that the mean time to an unsuccessful session is inversely proportional to customers’ arrival rate, for all operational profiles. Its function decreases monotonously from ∞ (for α = 0 customers/s), and asymptotically approaches a limiting value, which is different and characteristic for each operational profile (for α → ∞ customers/s). The operational profile OP#1, comprised of 100% passionate customers, exhibits the smallest, while the operational profile OP#5, where all classes of customers are being included equally, exhibits the largest value of the mean time to an unsuccessful session.

28

1000

OP #1

Operating profile #5

OP #2 OP #3 OP #4 OP #5

100

OP #6

Time [s]

OP #7 OP #8

10

Operating profile #1

1 0.01

0.05

0.1

0.5

1

5

10

15

20

25

30

Arrival rate [e-Customers/s]

Fig. 19. Mean time to an unsuccessful session, for various operational profiles and customers’ arrival rates (0 ≤ α ≤ 30 customers/s) In order to detect the existence of a statistically significant correlation among the dependent variable (the mean time to an unsuccessful session) and the independent variables (the total probability for a successful outcome of sessions and the mean session length), a correlation analysis has been performed and the value of the Pearson’s correlation coefficient has been evaluated, for each pair of variables. It has been concluded that there is a statistically significant correlation (both positive and strong) only between the mean time to an unsuccessful session and the total probability for a successful outcome of sessions (correlation = 0.987). The functional relationship between the mean number of successful sessions to an unsuccessful session and the customers’ arrival rate is graphically presented in Fig. 20, for all operational profiles. It can be described by a logarithmic function, for each particular operational profile. For a given arrival rate α, the operational profile OP#1, including 100% of passionate customers, yields the highest value, while the operational profile OP#5, where customers’ classes have been equally distributed, yields the lowest value of this performability measure. 900

OP #1 800

OP #2 OP #3

Successful sessions

700

OP #4 600

OP #5

Operating profile #1

OP #6

500

OP #7 400

OP #8

300 200

Operating profile #5

100 0 0

1

5

10

15

20

25

30

Arrival rate [e-Customers/s]

Fig. 20. Mean number of successful sessions to an unsuccessful one, for various operational profiles and customers’ arrival rates (0 ≤ α ≤ 30 customers/s)

29

The evaluation of both the mean time to an unsuccessful session and the mean number of successful sessions to an unsuccessful one, for the cold standby configuration, performed under the same operational conditions as with the standard configuration, has shown that there is no significant change, due to the fact that the extra operational time of the whole system, being gained by activation of the spare module, has been still small enough to significantly influence the results. Table 5 depicts the descriptive statistics of the availability for both configurations being investigated (16,000 simulation runs were used, each simulating a time interval of 30 days). Nevertheless, the absolute increase of system availability of only 1.02 percentage points could easily be converted to 3.7 days, or 89.1 hours of additional operational time per year (i.e. 1.7 hours per week). These figures are in line with the findings that “59% of Fortune 500 companies experience a minimum of 1.6 hours of downtime per week” (Arnold, 2010), as discussed in the beginning of this article – and this is how it might be masked. Knowing the total probability for a successful outcome of sessions, and assuming some percentage participation of various operational profiles in the mix of customers, it is easy to estimate the gain in the number of successful sessions per year, given a specific arrival rate α. An example scenario is presented in Fig. 21 (the percentage participation of operational profiles OP#1–8 is 1%, 2%, 15%, 10%, 55%, 3%, 6% and 8%, respectively). If one supposes that each successful session yields a profit of just $1, the total extra profit per year can be easily estimated, as well. In that context, the analyses show that if the system availability increased by roughly 1%, than the Internet company would make an additional profit of more than $1 million per year, at an arrival rate of just give customers/s. Table 5. A comparison of the descriptive statistics underlying the probability distributions of ecommerce systems’ availability (standard vs. cold standby configuration) Parameters of the descriptive statistics Number of observations Arithmetic mean Standard deviation Minimum First quartile Median Third quartile Maximum

Standard configuration

Cold standby configuration

16,000 0.960 0.060 0.016 0.948 0.974 0.992 1.000

16,000 0.970 0.032 0.181 0.968 0.976 0.983 1.000

30

Fig. 21. Estimated additional successful sessions per year, from cold standby configuration 7. CONCLUSION As e-commerce systems and their web services have become more complex and mission-critical, it becomes an imperative to develop relevant tools and methods, as well as approaches for assessing system’s metrics regarding its performance, dependability, service reliability, and system availability. A general conclusion that can be drawn from this study is that the overall performability degradation is more pronounced when customers with unfavorable online shopping behavior appear more frequently, or at least equally distributed within the workload mix (OP#5). Only sporadic appearance of customers with either (1) low probability of finding a product (Reluctant and/or Selective), or (2) long session duration due to repetitive searches (Curious), or (3) low probability of placing an order (Curious and/or Selective) (OP#2–4; OP#6–8), has a lower impact on performability – however, the differences become more pronounced with increasing customers’ arrival rates. Or, more concisely: the mix of customers that initiate online shopping sessions has a considerable influence on the perception “how well the system performs over a specified period of time.” From the user’s perspective, a more sophisticated and highly available e-commerce system will not always lead to a higher perceived quality of web services as compared to a system with apparently more modest characteristics (i.e. insufficient capacity of hardware resources and/or reduced system availability), but more favorable operational profile. 7.1. Theoretical Contributions The main contributions of this study can be summarized as follows. (1) We have proposed a customer-centric, holistic approach, based on building and evaluating stochastic predictive models, through the usage of corresponding graphical and mathematical formalisms, and solving them by using computer simulations with discrete events. (2) The applied concepts and methodologies rely entirely 31

on the dynamic and stochastic behavior of customers during their online shopping sessions, since: (a) a stochastic model of customer’s online shopping behavior has been used; (b) a qualitative and a quantitative classification of customers have been carried out, taking into account their similar online shopping behavior patterns; (c) a workload characterization of a “general-in-nature” e-commerce system has been defined, through the specification of characteristic operational profiles, that define the participation of various classes of customers in the mix, in various proportions. (3) Hierarchical composition has been applied in studying the complex concept of performability of e-commerce systems, for developing stochastic models that implement its particular components, including performance and dependability (behavioral-based service reliability and system availability). (4) Complex e-commerce systems are viewed as a unity among customers, the Internet, and the e-commerce web sites. (5) A new approach has been promoted to solving classes of stochastic Petri nets using a programming language for discrete-event simulation, based on active processes (SimPy/Python). In the following subsection, we discuss how these concepts can be applied in practice. 7.2. Turning Performability Measures into Business-Oriented Performance Metrics Generally speaking, performability measures are a way for measuring particular e-commerce web site’s efficiency in attaining its goals. A variety of measures have already been defined and used to assess the success of e-commerce web sites in terms of popularity and revenue generated, for example, hits/second, page views/day, click-throughs, and unique visitors (Menascé and Almeida, 2000b). In addition, performance models are used to compute traditional performance metrics, such as response time, throughput, utilization, and mean queue length for systems comprising the server-side. The specification of a variety of interesting performance and dependability measures has been facilitated with the inclusion of reward structures as an integrated part of modeling in “a class of SPNs where the model definition includes the structural and timing behavior of the system, as well as specification of the measures to be computed from the analysis of the model,” known as stochastic reward nets (SRNs) (Ciardo and Trivedi, 1993). A reward structure and business-oriented performance metrics. Underlying an SRN is an independent semi-Markov reward process with reward rates associated to the states and reward impulses associated to the transitions between states. These two components specify a reward structure over the underlying stochastic process: ρ(µ) is the rate at which reward is accumulated when the marking is µ, and rТ(µ) is the instantaneous reward gained when firing transition T while in marking µ. Once a timing representation is assumed for a Petri Net model, the usual performance indices can be defined in “net terms” – for instance, “the mean number of firings of a transition per time unit” (throughput) as a productivity measure. The throughput of a transition is defined as the expected number of firings per time unit and is normally considered in the stationary limit. In our case, all reward rates are zero and the reward accumulated by the SRN up to time t is:

32

max{ n:t [ n ] ≤ t }

Y (t ) =

∑r

T[ j]

( µ [ j −1] )

(3)

j =1

where, for j > 0, T[j] is the j-th transition to fire, its firing leads to µ[j], the j-th marking encountered, and t[j] is the time at which it fires. Then, the expected number of firings of transition T up to time t is simply the expected accumulated reward when only the reward impulse for transition T equals one (all the other reward impulses are zero) and is denoted by E[YT(t)]. In order to exploit the notion of a reward structure, we assume that in an observed time period [0, …, t] customers of various classes (as defined by a specific scenario) start their online sessions with an e-commerce web site, with an arrival rate α. During the sessions, they invoke specific e-commerce functions, like Search, Browse, Select, Add to Cart, etc. By following the “token game” along the net in Fig. 12, majority of the customers (SCompleted) finish and complete regularly their visits to the virtual store, while others ( S Expired = E[Y T

(t )] ) do not, due to inactivity, which happens when a determin-

TIMEOUT

istic timeout period expires after invoking and staying in a specific state/function. Both of these comprise the total number of customers (STotal = SCompleted+ SExpired). Out of those that have finished regularly (SCompleted), a number ( SSuccess = E[Y T

REVIEW − SUBMIT −VERIFY

(t )] ) have bought something, resulting in successful

sessions. Nonetheless, there are also a certain number of customers (SUnsuccess) that have left the virtual store without buying anything, thus resulting in unsuccessful sessions. This group can be further divided into two disjoint subsets, including a subset which consists of customers who have put something in the shopping basket and have left the store without paying ( S Put − and − Not − Paid = E[Y T

END _ ITEMS _ ADDED

(t )] ),

and a subset of customers that have not put anything at all into the shopping basket and have left the store ( S Left = E[Y T

(t )] ). All these values are functionally dependent on the customers’ END _ ITEMS _ NOT _ ADDED

arrival rate α. When simulating the GSPN model for the evaluation of the total probability for a successful outcome of a session (Fig. 10), the following variables can be introduced, as well: •

I Put − and − Paid = E  lim YT ADD (t ) ⋅ P (# P ORDER _ PLACED =1): Expected number of items that have t → ∞ 

been put in the shopping cart and have been paid for, per session, which is the product of the expected accumulated reward in steady-state by repetitively firing transition TADD and the probability that the marking process has transitioned in an absorbing state with a token in place PORDER_PLACED. •

I Put − and − Paid = I Put − and − Paid ⋅ SSuccess : Total number of items that have been put in a shopping cart and have been paid for, during successful sessions in the time period [0, …, t].

•

I Put − and − Not − Paid = E  lim YT ADD (t ) ⋅ P (# P END =1) : Expected number of items that have been t →∞ 

33

put in a shopping cart and have not been paid for, per session, which is the product of the expected accumulated reward in steady-state by repetitively firing transition TADD and the probability that the marking process has transitioned in an absorbing state with a token in place PEND. •

I Put − and − Not − Paid = I Put − and − Not − Paid ⋅ S Put − and − Not − Paid . Total number of items that have been put in a shopping cart and have not been paid for, during sessions when customers have left the store without placing an order, in the time period [0, …, t].

We are especially interested in assessing innovative business-oriented performance metrics (Menascé et. al, 2000), which include the following: •

BV: Buy-to-visit ratio (%). A business-oriented metric that measures the average number of

sale transactions per visit to a particular e-commerce web site; in other words, it shows how often customers buy something each time they visit an e-commerce web site:

BV = •

S Success S Success = STotal S Success + S Put − and − Not − Paid + S Left + S Expired

(4)

RT: Revenue throughput ($/sec). A business-oriented metric that measures how much mon-

ey per time unit have been derived from sales, for a particular e-commerce web site, by taking into account individual prices of each of the sold items; this measure implicitly represents both the customer’s and e-commerce web site’s behavior, since its value increases whenever an customer is satisfied/happy with the quality-of-service issues, such as response time, as well as with the site’s internal look, structure, organization and prices, and will continue to visit it for shopping, over and over.

1 K RT = ⋅ ∑ Ci ⋅ I Put − and − Paid ( i ) ⋅ pi t i =1

(5)

where, for i > 0, t is the observed time interval, K is the total number of product types being sold online, Ci is the unit price of an i-type product, pi is the probability of buying an item of itype product (based on its popularity, either by experience or by means of some statistical data already available) and IPut-and-Paid(i) is the estimated number of sold i-type product items. •

PLT: Potential loss throughput or lost revenue throughput ($/sec). Another business-

oriented metric, that measures the amount of money in customers’ shopping carts that is not converted into sales because customers have prematurely left the web site due to poor performance or other reasons, such as system unavailability, service inaccessibility, etc. Similarly,

1 K PLT = ⋅ ∑ Ci ⋅ I Put − and − Not − Paid ( i ) ⋅ pi t i =1

(6)

where IPut-and-Not-Paid(i) is the estimated number of i-type product items that have been put into shopping carts, but have not been paid for. Such an analysis, a research endeavor per se, is a valuable resource for e-commerce web site’s top 34

management teams in creating their policies, including the processes of decision making, both on an operational and strategic level. Nonetheless, these analyses also represent a cornerstone for conducting relevant server sizing activities, since pertinent and prompt capacity plan is a fundamental premise to achieve customers’ satisfaction, which, in turn, always results in higher revenues. The business-oriented performance metrics and an average company. Is there a simple, clear

statement of what this means to an average company running an e-commerce web site? To put it differently, people making information systems decisions should rethink their approach to server sizing – the discussion at the end of Section 6 shows that the increase of system availability with a cold standby configuration could easily be converted into millions of dollars of additional profit per year. To do so, investments in high-availability and fault-tolerance solutions are necessary (Information Technology and Intelligence Corp., 2011). Likewise, “click rates can be combined with other information about the content which is being browsed in order to make inference about a user’s state of mind” (Scott and Smyth, 2003). In our case, very rapid click rates, many repetitive searches, and lower probability of placing an order, suggest that a customer (e.g., curious, selective, or reluctant) is searching for something that he has not found on the current page. At times, when a customer’s click rate falls to zero, his session has to be aborted prematurely and forcibly by the e-commerce system, due to his inactivity, so resources allocated to him should be directed elsewhere. In either case, both the buy-to-visit ratio (BV) and the revenue throughput (RT) decrease. Moderate click rates, however, suggest that the customer (passionate or focused) is absorbing content, followed by higher probability of placing an order. Therefore, business managers should, in some way, put all the efforts into shaping the mix of customers that come to their web site towards operational profiles in which the percentage participation of the most desirable classes (passionate and focused customers) is as high as possible. As a result, the average number of sale transactions per visit (BV) will be maximized, whereas the amount of money in customers’ shopping carts that is not converted into sales (PLT), will be minimized. Business managers must determine the variety of reasons for purchasing a product or service and tailor their marketing and sales efforts to specifically reach the segment of population that will most likely buy their product or service online (MaGee, 2008). The main impediment here is the fact that some technology adoption concerns are more psychological than technical. Chang et al. (2005) identify several social psychological variables that are critical determinants for the adoption of online shopping: attitude toward online shopping, perceived behavioral control, subjective norm, intention, risk aversion, self-confidence, intention to use the Internet for information search, and site commitment. Yet, they pinpoint areas that would aid in developing a better understanding of the dynamics of an customer’s decision to shop online and derive two reference models: the antecedents of the “intention to use it” or “the usage” of online shopping (perceived characteristics of the web as a sale channel, web site and product characteristics, and consumer characteristics), as well as the relationships amongst these antecedents. In this context, their findings may be useful for online marketers, compa-

35

nies or retailers who seek to offer their products or services over the Internet and are eager to understand the peculiarities of customers’ motivation and behavior. 7.3. Limitations and Implications for Future Research

The proposed methodology and the predictive models can be additionally expanded in several ways, including the following: (1) enrichment of the models that originate from the DSPN model of customer’s online behavior (Mitrevski and Hristoski, 2011) with a gamut of new e-commerce functions (login, register, browse, select-item, remove-from-cart, etc.), in order to bring the behavior model to another level of abstraction; this step might be counter-productive if the resulting stochastic Petri models are going to be solved analytically, or even numerically by a dedicated software, due to the inevitable state-space explosion, but is very useful if the solution of those models is going to be made by discrete-event simulations, since the evaluation results would be more accurate; (2) enrichment of the comparative analyses by including new characteristic operational profiles; (3) stochastic modeling of the workload changes dynamics, through a dynamic change of the structure of involved operational profiles during the observed time period, in order to model the Internet traffic burst periods (e.g., selfsimilar processes for modeling Web servers: N-Burst/ME/1, N-Burst/G/1, etc.) (Nossenson and Attiya, 2004). The models this research relies on are based on the DSPN and GSPN classes of stochastic Petri nets, where tokens represent particular customers. Because all the tokens have the same “color”, they are semantically equal to each other, so they cannot be distinguished between. Since we propose taxonomy of customers based on their different online behavior patterns during the course of a shopping session, a natural extension of our approach is to employ the class of colored stochastic Petri nets (CPSNs) for modeling purposes, which is suitable for representing customers belonging to different classes by tokens with different color. This opens many other questions in terms of the complexity of such models and the possibilities for their solving, either analytically or numerically by utilizing a corresponding software tool. The performability measures have been evaluated by taking into account two common ecommerce hardware configurations: the standard one and the cold standby configuration. In both cases the working parameters correspond to the class of a “well managed system.” So, it is necessary to carry out series of simulations in order to evaluate performability measures for at least two additional classes of e-commerce systems, including the class of fault-tolerant systems, as well as the class of highly-available systems (Araújo et al. 2011). For these classes of systems, the value of the MTTF parameter increases both for the main and the spare system, whilst the values of the MTTR and the MAT parameter remain the same. The inclusion of higher criteria with these two classes has been done in order to improve system availability. The mutual comparison of the simulation results for all of these three classes of systems will give a significant contribution to the evaluation of performability measures.

36

A propos to the cold standby configuration, a single possible research course of action might be the case of upgrading the model of the existing e-commerce system by adding up a second, third, etc. redundant module, besides the existing one. In this case, it would be interesting to make an economic feasibility study of such upgraded e-commerce systems, through a relevant cost-benefit analysis and an analysis of the functional dependence of the achieved availability level versus the total number of added redundant modules. Inspired by the new evidence that the increase of system availability with the cold standby configuration (although unnoticeable at first sight) could easily be converted into dozens of hours of additional operational time and hundreds of thousands of customers’ sessions per year, we identify two ways of improving the performance of e-commerce systems towards near-zero downtime: horizontal scaling (scaling out) and vertical scaling (scaling up). By scaling out, the total capacity is increased and the whole system becomes highly available, whereas scaling up means resource maximization of an existing unit within a given system. Cresttechglobal.com (2013) has announced that in 38% of all cases, the application server has been identified as a bottleneck, whilst in 24% the web server has been recognized as the main contributor to e-commerce web site’s poor performance. The database server and the storage system have been identified as a bottleneck by Zhang et al. (2004). Under the assumption that the resulting speedup of the server can been acquired according to the Amdahl’s Law, the following metrics can be assessed: (1) the average overall end-to-end response time, (2) end-to-end response time versus CPU speedup, (3) servers’ throughput, (4) the utilization of the web server, (5) web server’s average service demand, (6) the maximum queue length at the web server, (7) the average processing time by e-commerce function (add-to-cart, browse, login, logout, pay, register, search), etc. In this research effort of its own, simulations of the e-commerce clientserver interaction should show whether the scaling techniques, being applied on a single server, can significantly increase both its own performances and those of the e-commerce system as a whole. As a final point, this study raises some new unanswered questions and a handful of research avenues have been pinpointed and discussed. All these impose the need for further and long-term exploration of not only e-commerce systems and customers’ online shopping behavior, but also the tools and techniques for addressing the wide spectrum of issues related to performability modeling and evaluation of systems the e-commerce paradigm relies on. REFERENCES

Ajmone-Marsan, M., 1990. Stochastic Petri Nets: An Elementary Introduction. Advances in Petri Nets 1989, Lecture Notes in Computer Science (LNCS), 424, 1-29. Ajmone-Marsan, M., Conte, G., Balbo, G., 1984. A Class of Generalized Stochastic Petri Nets for the Performance Evaluation of Multiprocessor Systems. ACM Transactions on Computer Systems, 2(2), 93-122. Antovski, Lj., Armenski, G., 2012. KupiKniga.mk: Transforming a Website into a Profitable ECommerce System Using Assisted Conversions Funnel. In Markovski, S., Gusev, M. (Eds.), ICT Innovations 2012 Web Proceedings (pp. 321-329). Skopje: ICT ACT.

37

Araújo, J., Maciel, P., Zimmermann, A., Andrade, E., Sousa, E., Callou, G., Cunha, P., 2011. Performability Modeling of Electronic Funds Transfer Systems. Computing, 91(4), 315-334. Ariely, D., 2009. Predictably Irrational: The Hidden Forces That Shape Our Decisions (Revised and Expanded Edn.). HarperCollins e-books. Arnold, A., 2010. Assessing the Financial Impact of Downtime. Available at www.businesscomputingworld.co.uk/assessing-the-financial-impact-of-downtime/. Last accessed March 2014. Banga, G., Druschel, P., 1999. Measuring the Capacity of a Web-Server Under Realistic Loads. World Wide Web, 2(1-2), 69-83. Blanchard, B. S., 1991. System Engineering Management. New York: John Wiley and Sons. Bolch, G., Greiner, S., de Meer, H., Trivedi, K. S., 2006. Queuing Networks and Markov Chains: Modeling and Performance Evaluation with Computer Science Applications (2nd Edn.). Hoboken, New Jersey: John Wiley Sons, Inc. Brosso, I., Bressan, G., Ruggiero, W. V., 2002. Simulation E-Business Applications Using PNML. In Proceedings of The 2nd WSEAS International Conference on Simulation, Modeling and Optimization (ICOSMO 2002), Athens, Greece. WSEAS Press [AEC074], 1:3571-3576. Callou, G., Maciel, P., Tutsch, D., Araújo, J., Ferreira, J., Souza, R., 2012. A Petri Net-Based Approach to the Quantification of Data Center Dependability. In Pawlewski, P. (Ed.), Petri Nets Manufacturing and Computer Science (pp. 313-336). InTech. Chang, M. K., Cheung, W., Lai, V. S., 2005. Literature derived reference models for the adoption of online shopping. Information & Management, 42(4), 543-559. Choi, H., Kulkarni, V. G., Trivedi, K. S., 1993. Transient Analysis of Deterministic and Stochastic Petri Nets. In Marsan, M. A. (Ed.), Application and Theory of Petri Nets 1993, Lecture Notes in Computer Science (LNCS), 691, 166-185. Christensen, S., Kristensen, L. M., Mortensen, K. H., Thomasen, J. S., 1999. Capacity Planning of Web Servers Using Timed Hierarchical Coloured Petri Nets. In Proceedings of The HP Openview University Association (HP-OVUA ’99) 6th Plenary Workshop, Bologna, Italy. Ciardo, G., Trivedi, K. S., 1993. A Decomposition Approach for Stochastic Reward Net Models. Performance Evaluation, 18(1), 37-59. Ciardo, G., Lindemann, C., 1993. Analysis of Deterministic and Stochastic Petri Nets. In Proceedings of the 5th International Workshop on Petri Nets and Performance Models, Toulouse, France, 160169. crestechglobal.com., 2013. Top Performance Bottlenecks in Web Application, White Paper. CresTech Software Systems Pvt. Ltd. Available at www.crestechglobal.com/wpcontent/uploads/2013/08/Whitepaper_PerformanceTesting.pdf. Last accessed November 2013. Curran, C., 2010. Behavioral Economics and Project Failure. Available at www.ciodashboard.com/project-and-program-management/behavioral-economics-projectmanagement/. Last accessed January 2013. Donatiello, L., Grassi, V., 2001. Closed-Form Solutions for Performability. In Haverkort, B. R., Marie, R., Rubino, G., Trivedi, K. S. (Eds.), Performability Modelling: Techniques and Tools (pp. 59-82). Chichester: John Wiley Sons Ltd. Dratch, D., 2014. How Behavioral Economics Explains 6 Common Money Mistakes. Available at www.creditcards.com/credit-card-news/behavioral-economics-6-money-mistakes-1267.php. Last accessed March 2014. Emerson Network Power, 2011. Understanding the Cost of Data Center Downtime: An Analysis of the Financial Impact of Infrastructure Vulnerability. Liebert Corporation, Columbus, OH, USA.

38

Ferrari, G., Ezhilchelvan, P., Mitrani, I., 2006. Performance Modeling and Evaluation of E-Business System. In Proceedings of The 39th Annual Simulation Symposium (ANSS ’06), Huntsville, Alabama, USA, 135-142. Haverkort, B. R., Trivedi, K. S., 2001. A Survey of Performability Modelling Tools. In Haverkort, B. R., Marie, R., Rubino, G., Trivedi, K. S. (Eds.), Performability Modelling: Techniques and Tools (pp. 223-261). Chichester: John Wiley Sons Ltd. Hecht, M., 2001. Reliability/Availability Modeling and Prediction for e-Commerce and Other Internet Information Systems. In Proceedings of The 2001 Annual Reliability and Maintainability Symposium. Philadelphia, PA, USA, 176-182. Heffes, H., 1980. A Class of Data Traffic Processes – Covariance Function Characterization and Related Queuing Results, The Bell System Technical Journal, 59(6). Information Technology and Intelligence Corp., 2011. Trends In High Availability And Fault Tolerance. Available at searchcio.techtarget.com/podcast/Trends-in-high-availability-and-faulttolerance. Last accessed March 2014. Jiang, H., Dovrolis, C., 2005. Why is the Internet Traffic Bursty in Short Time Scales?. ACM SIGMETRICS Performance Evaluation Review, 33(1), 241-252. Kihl, M., Widell, N., Nyberg, C., 2002. Performance Modeling of Distributed E-commerce Sites. In Widell, N. (Ed.), Performance of Distributed Information Systems (pp. 65-77). Department of Communication Systems, Lund University, Lund, Sweden. Kounev, S., Buchmann, A., 2003. Performance Modeling and Evaluation of Large-Scale J2EE Applications. In Proceedings of The 29th International Conference on Resource Management and Performance Evaluation of Enterprise Computing Systems (CMG 2003), 273-283. Latino, M. A., 2000. Behavioral Based Reliability, 2000 Machinery Reliability Conference. Available at www.reliability.com/industry/articles/article36.pdf. Last accessed November 2013. Lindemann, C., Lohmann, M., Thümmler, A., Waldhorst, O., 1999. The DSPNexpress 2000 Performance and Dependability Modeling Environment. In Proceedings of The 29th International Symposium on Fault-Tolerant Computing, Madison, Wisconsin, 228-231. Logothetis, D., Trivedi, K., 1997. The Effect of Detection and Restoration Times for a Error Recovery in Communication Networks. Journal of Network and Systems Management, 5(2), 173-195. MaGee, S., 2008. How to Identify a Target Market and Prepare a Customer Profile. Edward Lowe Foundation. Available at edwardlowe.org/erc/?ercID=6378. Last accessed June 2014. Matloff, N., 2008. Introduction to Discrete-Event Simulation and the SimPy Language. Available at heather.cs.ucdavis.edu/~matloff/156/PLN/DESimIntro.pdf. Last accessed May 2013. Menascé, D. A., 2003. Workload Characterization. IEEE Internet Computing, 7(5), 89-92. Menascé, D. A., Abrahao, B. D., Barbará, D., Almeida, V. A. F., Ribeiro, F., 2002a. Characterizing EBusiness Workloads Using Fractal Methods. Journal of Web Engineering, 1(1), 74-92. Menascé, D. A., Almeida, V. A. F., 2000a. Challenges in Scaling e-Business Sites. In Proceedings of the 26th International Computer Measurement Group (CMG) Conference, Orlando, FL, USA, 329-336 Menascé, D. A., Almeida, V. A. F., 2000b. Scaling for E-Business: Technologies, Models, Performance, and Capacity Planning. Prentice Hall PTR, New Jersey, USA. Menascé, D. A., Almeida, V. A. F., 2002. Capacity Planning for Web Services: Metrics, Models, and Methods. Prentice Hall PTR, New Jersey, USA. Menascé, D. A., Almeida, V. A. F., Dowdy, L. W., 1994. Capacity Planning and Performance Modeling: From Mainframes to Client-Server Systems. Prentice Hall PTR, New Jersey, USA.

39

Menascé, D. A., Almeida, V. A. F., Fonseca, R. C., Mendes, M. A., 1999. A Methodology for Workload Characterization for E-Commerce Sites. In Proceedings of The 1st ACM Conference on Electronic Commerce (EC ‘99), Denver, CO, USA, 119–128. Menascé, D. A., Almeida, V. A. F., Fonseca, R., Mendes, M. A., 2000. Business-Oriented Resource Management Policies for e-Commerce Servers. Performance Evaluation: Special Issue on Internet Performance Modeling, 42(2–3), 223–239. Menascé, D. A., Barbara, D., Dodge, R., 2002b. Preserving QoS of E-commerce Sites Through SelfTunning: a Performance Model Approach. In Proceedings of The 3rd ACM Conference on Electronic Commerce, 224-234. Meyer, J. F., Sanders, W. H., 2001. Specification and Construction of Performability Models. In Haverkort, B. R., Marie, R., Rubino, G., Trivedi, K. S. (Eds.), Performability Modelling: Techniques and Tools (pp. 179-222), John Wiley Sons. Microsoft Corp., 2001. Planning for Reliability and High Availability. Microsoft Commerce Server 2000 Technical Article. Available at msdn.microsoft.com/en-us/library/ee797245 (v=cs.10).aspx. Last accessed June 2013. Mitrevski, P., Hristoski, I., 2011, e-Consumer Online Behavior: A Basis for Obtaining e-Commerce Performance Metrics. In Gušev, M., Mitrevski, P. (Eds.), ICT Innovations 2010 (pp. 142-151), Berlin Heidelberg: Springer Verlag. Müller, K., Vignaux, T., 2003. SimPy: Simulating Systems in Python. O’Reilly ONLamp.com Python DevCenter. Available at onlamp.com/pub/a/python/2003/02/27/simpy.html. Last accessed May 2013. Nossenson, R., Attiya, H., 2004. Evaluating Self-Similar Processes for Modeling Web Servers. In International Symposium on Performance Evaluation of Computer and Telecommunication Systems (SPECTS), 805-812. Patterson, D., 2002. A Simple Way to Estimate the Cost of Downtime. In Proceedings of The 16th USENIX Conference on System Administration, LISA ’02. USENIX Association, Berkeley, CA, USA, 185-188. Pattipati, K. R., Mallubhatla, R., Gopalakrishna, V., Viswanatham, N., 2001. Markov-Reward Models and Hyperbolic Systems. In Haverkort, B. R., Marie, R., Rubino, G., Trivedi, K. S. (Eds.), Performability Modelling: Techniques and Tools (pp. 83-106). Chichester: John Wiley Sons Ltd. Ponemon Institute, 2011. Calculating the Cost of Data Center Outages: Benchmark Study of 41 US Data Centers. Ponemon Institute, Traverse City, MI, USA. Reddy, N., Acharyulu, G.V.R.K., 2009. Marketing Research. Excel Books India. Robidoux, R., Xu, H., Xing, L., Zhou, M., 2010. Automated Modeling of Dynamic Reliability Block Diagrams Using Colored Petri Nets. IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans, 40(2), 337-351. Sahner, R. A., Trivedi, K. S., Puliafito, A., 1996. Performance and Reliability Analysis of Computer Systems: An Example-Based Approach Using the SHARPE Software Package. Kluwer Academic Publishers, Norwell, MA, USA. Schroeder, T., Goddard, S., Ramamurthy, B., 2000. Scalable Web Server Clustering Technologies. IEEE Network, 38–45. Scott, S. L., Smyth, P., 2003. The Markov modulated Poisson process and Markov Poisson cascade with applications to web traffic data. Bayesian Statistics, 7, 671–680. Urgaonkar, B., Pacifici, G., Shenoy, P., Spreitzer, M., Tantawi, A., 2005. An Analytical Model for Multi-Tier Internet Services and Its Applications. In Proceedings of The ACM International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS), 291-302.

40

Veerarghavan, M., Trivedi, K. S., 1994. Composite Performance and Reliability Analysis Using Combinatorial Multistate Models. IEEE Transactions on Computers, 43(2), 229-234. Wang, D., Trivedi, K. S., 2009. Modeling User-Perceived Reliability Based on User Behavior Graphs. International Journal of Reliability, Quality & Safety Engineering, 16(4), 303-330. Wei, B., Lin, C., Kong, X., 2011. Dependability Modeling and Analysis for the Virtual Data Center of Cloud Computing. In Proceedings of The IEEE 13th International Conference on High Performance Computing and Communications (HPCC 2011), Banff, Alberta, Canada, 784-789. Xu, H., Xing, L., Robidoux, R., 2009. DRBD: Dynamic Reliability Block Diagrams for System Reliability Modeling, International Journal of Computers and Applications, 31(2), 132. Zhang, Q., Riska, A., Riedel, E. Smirni, E., 2004. Bottlenecks and Their Performance Implications in E-Commerce Systems. In Proceedings of The 9th International Workshop on Web Content Caching and Distribution (WCW 2004), Beijing, China, 273−282. Zimmermann, A., Knoke, M., 2007. TimeNET 4.0: A Software Tool for the Performability Evaluation with Stochastic and Colored Petri Nets. User Manual. Technische Universität Berlin, Real-Time Systems and Robotics Group, Faculty of EE&CS Technical Report 2007-13, 1-32.

• • • • •

We model eCustomers’ shopping behavior with Deterministic and Stochastic Petri Nets We employ hierarchical composition of sub-models to build a performability model We propose taxonomy, specify operational profiles and use discrete-event simulation Highly available eCommerce system will not always lead to higher user-perceived QoS Performability measures are turned into business-oriented metrics for server sizing

41

Page 16, Equation in Fig.3: N

∑w

i

=1

i =1

Page 16, Equation (2): wT P TCLASS _ i = N CLASS _ i = w TCLASS _ j

(

)

(

∑ ( j =1

)

wi N

) ∑w

j

j =1

Page 34, Equation (5): 1 K RT = ⋅ Ci ⋅ I Put −and −Paid (i ) ⋅ pi t i =1

∑

Page 34, Equation (6): 1 K PLT = ⋅ Ci ⋅ I Put −and −Not −Paid (i )⋅ pi t i =1

∑

42