Measuring the Performance of eCommerce Websites - Pacific Asia

Measuring the Performance of eCommerce Websites – An Owner's Perspective / Ghandour et al.

Measuring the Performance of eCommerce Websites – An Owner’s Perspective Ahmad Ghandour Dept. of Information Science, University of Otago, New Zealand [email protected] George Benwell School of Business, University of Otago, New Zealand, [email protected] Kenneth R Deans Dept. of Marketing, University of Otago, New Zealand [email protected]

Abstract Measurements that capture performance have long enabled businesses to monitor and improve their strategic and operational goals. For eCommerce websites, there has been a limited understanding of how to measure performance even though it has been researched in many ways and in various contexts over the past decade. The authors of this study adopted an owner perspective to develop and empirically test a model of website performance measurement. The results suggest that performance is a second order factor model. The first order factors of the model are usage, financial benefits and owner satisfaction. The model is useful as a tool for benchmarking the performance of the website as well as a foundation for operationalising performance. Keywords: Website, Performance, eCommerce, Operationalising.

Pacific Asia Journal of the Association for Information Systems Vol. 3 No. 1, pp.1-27 / March 2011

1


Introduction Effective web-based selling requires businesses to have a sound knowledge of how to measure the performance of their website as part of their management practices. Clear, useful measurements that capture website performance have enabled businesses to improve strategies and operations as part of their evaluation effort. The website is an increasingly important sales channel (sometimes a company’s sole interface) linking the business and its customers. Consequently there is a growing demand to see returns on internet-related investments and a greater focus on performance is becoming critical for internet–based commerce. As a result of the proliferation of commercial websites, and the many calls from businesses for their use, there is a need to investigate their effectiveness. However, this is not possible without an appropriate performance measurement tool. Measuring the performance of a website has been researched in a variety ways and contexts over the past decade. In many instances, a single or a collection of financial measures was used to represent website performance. However, the measurement of website performance is neither simple nor straight forward as it is multidimensional in nature. The different perspectives, website user, owner and designer may add other layers of complexity to the construct measurements. Whilst the user and the designer perspectives are well advanced in the literature, there is a relative dearth of scholarly studies that address the owner’s needs. The provision of such a perspective may enhance an owner’s ability to increase returns and benefits from their online activity. The current study addresses the owner’s perspective to develop a performance measure that can be used to understand what constitutes an effective website. This article is organised in seven sections. The background to the study is introduced in the next section where the theoretical perspectives for measuring performance and the literature are examined. This is followed by defining a theoretical domain for the owner’s perspective of

2

website performance measures. An operational definition of website performance is then described in section four followed by data collection and analysis strategy in section five. Section six discusses model analysis and results. The final section discusses the results and describes the implications of the study.

Background The importance of website performance measurements has been established in the literature (Auger 2005; Bremser and Chung 2005; Hahn and Kauffman 2001; Huizingh 2002; Jonathan 2002; Straub et al. 2002a; Wade and Nevo 2005; Welling and White 2006). More specifically, websites that sell good/services (eCommerce websites) implement such measurements but there are often controversies over the measurements to be used. Past studies indicate that issues pertaining to website management are of great importance to owners worldwide (Adam and Deans 2001; Ho 1997; Kim et al. 2003). Different measurements have been proposed in many ways and various contexts over the past decade. In general however, managers have no structured set of criteria upon which to gauge the performance of their eCommerce websites. In many instances, simplistic and rather varied measures of performance have been utilised (DeLone and McLean 2004; Molla and Licker 2001). This inconsistency may have prevented theory development on website performance, effectiveness and management. The measurement of website performance has proven to be a difficult task as it depends on which stakeholder perspective (the user, the designer or the owner) is adopted. It is further complicated by the fact (Palmer 2002) that can it be assessed at different levels (individual and organisational) using different interrelated criteria (Molla and Licker 2001). Multiple, interrelated success dimensions are more likely to capture changes in performance than a single item or even a set of financial measures (Segars and Grover 1998) as they only portray one facet of performance (Auger 2005; Barua et al. 2001; Barua et al.



1995; Quaddus and Achjari 2005). An examination of the literature revealed three perspectives when assessing the effectiveness of a website, user, designer, and owner. The extant literature reveals that the user perception of a website can strongly affect its performance and that the user is affected by a positive experience with the interface. These studies addressed website quality and user satisfaction as measures of performance. Several instruments have been developed to define, measure, and manage the quality of websites (Aladwani and Palvia 2002; Barnes and Vidgen 2003; Katerattanakul and Siau 1999; Loiacono et al. 2002; Mich et al. 2003; Sharkey et al. 2007; Webb and Webb 2004; Zhang and Von Dran 2002). Similarly, instruments have been developed to define and measure user satisfaction of a website (Bailey and Pearson 1983; Baroudi and Orlikowski 1988; Bharati and Chaudhury 2004; Doll and Torkzadeh 1988; McKinney et al. 2002; Muylle et al. 2004; Wang et al. 2001; Zviran et al. 2006). The underlying premise of these studies is that the performance of the website is measured by how satisfied the users are and consequently how willing they would be to revisit the site. The more frequently visitors visit a website, the greater the probability they will make a purchase. Udo and Marquis (2001) found that maintaining a high quality website affects the number of people visiting the website and subsequently doing business with it. The various studies served to improve website designs in terms of “what to include” but have done little to inform businesses on success or profitability. The designer perspective literature reveals what technical features are imperative for websites (Auger 2005; Day 1997; Drèze and Zufryden 1997; Kim et al. 2003; Kohavi and Parekh 2003; Mich et al. 2003; Olsina et al. 2000; Palmer 2002; Spiliopoulou 2000; Turban and Gehrke 2000; Udo and Marquis 2001; Zhang et al. 1999). The information gained from these studies is important to designers who face the task of planning future website designs.

The third perspective is that of the owners. The recent literature reports that owners are able to measure the performance of their websites based on their own perceptions (Alpar 2001; Huizingh 2002; Quaddus and Achjari 2005). Although user and designer perspectives represent a legitimate approach for assessing website performance, it is critical to identify website effectiveness from the owner perspective. The owner’s perceptions help to pinpoint areas within the business where eCommerce is creating value; they are the ones who need to know the return on their information technology (IT) investment (Tallon and Kraemer 2002).

A Theoretical Domain of Owner’s Perspective eCommerce Websites Performance There have been a number of attempts to include the owner perspective (Huizingh et al. 2007; Huizingh 2002; Lu 2003; Quaddus and Achjari 2005; Stockdale et al. 2006; Teo and Pian 2004; Zhu and Kraemer 2003). These have either been incomplete attempts to measure website performance, (e.g. Lu (2003) included only one item, “company satisfaction”), or used established user perspective tools that evaluate websites. Stokedale et al (2006), guided by the instrument eQual, developed by Barnes and Vigden (2002), tried to identify critical success factors for effective website usage by small businesses from the owner’s perspective. Although eQual was developed for a “user voice” perspective, owners were still unable to recognise benefits. Schaupp et al (2006) defines the organisation’s perspective of a website success as its ability to create an on-going relationship with the end consumer (user), which will either immediately or eventually lead to a transaction. They further posited that metrics for site effectiveness need to be tied not only to the website navigation patterns of its users, but also to the website specific goals. The literature also provides examples of how companies develop metrics for eCommerce and their websites’ effectiveness along differ-


3


Table1 - Literature summary on labels of performance Authors Hoffman and Novak 1996 Gomory et al 1999 Alpar 2001 Barua et al 2001 Barua et al 2001 Udo and Marquis 2001 Chen et al 2002 Goldfarb 2002 Huizingh 2002 Olson and Boyer 2003 Lu 2003 Lu 2003 Zhuang and Lederer 2003 Zhu and Kraemer 2003 Lii et al 2004 DeLone MacLean 2004 Epstien 2004 Scharl et al 2004 Quddas and Achjari 2005 Auger, 2005 Bremser and Chung 2005 Pujani and Xu 2006 Pujani and Xu 2006 Huizingh et al 2007 Fisher, 2007 Hong 2007 Tang and Huang 2008

Performance labels Website Usage Web Usage Traffic Operational Excellence Financial Performance Website Effectiveness Usage Performance Website Performance Organisational Performance Company Satisfaction Core Benefit Business Benefit Firm Performance Website Effectiveness Net Benefits Outputs/Outcome Website Effectiveness eCommerce Success Overall Performance Performance Website Use Organisational Benefits Website Success Website Strategy Website Success Firm Performance

ent dimensions to capture the change in website performance (Barua et al. 2001; Lu 2003; Pujani and Xu 2005; Straub et al. 2002b). However, there is very little evidence in the literature that an owner’s perspective of performance has been modelled adequately. Such modelling would enable an owner to better realise the potential of their website. As noted by DeLone and McLean (1992), the performance of any information system (IS) is a complex phenomeno likely to only be captured by multiple, interrelated dimensions. According to Churchill (1979), a theoretical domain of a complex variables can be formed through an extensive literature review coupled with expert opinion. An operational

4

No. of items 7 6 5 5 10 5 2 6 3 3 1 8 5 4 4 9 11 7 6 8 8 6 14 5 3 12 4

basis for the complex phenomenon can then be defined. Focussing on the owner perspective as a theoretical underpinning, an extensive review of the literature was conducted to identify the different measures used to assess the performance of a website. Different labels of performance were used such as effectiveness, success and benefit. A summary of these labels is shown in Table 1. Items for performance measurement from these studies were collated to generate a pool of 161 items. A detailed list of items identified in the various studies is included in Appendix A. Analysing the collated items for redundancies was undertaken by the first au-



Table 2 - Performance measures in their respective dimension Website usage may be measured by 1 Number of visitors 2 Repeat visitors 3 Visitor to customer conversion 4 Page views 5 Time spent on website 6 Navigation behaviour tracking 7 Customer profile 8 User environment 9 Referring website 10 Reach 11 Bounce rate Profitability of website may be measured by 1 Return On Investment (ROI) 2 Website sales 3 Sales growth 4 Profit from the website 5 Cost reduction 6 Market share increase Owner Satisfaction may be measured by; 1 Extent to which the website has strengthened competitive position 2 Number of customers who visit the website 3 Search engine ranking

thor to remove duplicates items and to identify measures that may be identical after which a set of 20 items remained. Next, “expert opinions” were sought from a panel of five academics (two with their own online businesses). The panel were asked to classify the 161 items into the 20 items specified by the first author. These classifications were virtually identical between panel members, and the small inconsistencies were resolved through discussion with the authors. In general, it was agreed among the panel members that the 20 items are relevant and relatively distinct performance measures. The intercoder reliability using Cohen’s (1960) coefficient Kappa as a function of observed agreement between the panel members was then estimated. For all items, the average Kappa was greater than 0.75, which exceeds the suggested minimum of 0.70 (Landis and Koch 1977). To create a theoretical structure for the performance measurements, the panel members were also asked independently and individually to group the items based on similarity.

The composite result was three groups providing structure for the identified performance measurements items of three dimensions. Table 2 shows these measurement items in their respective performance dimensions. Each dimension is a distinct concept used previously as a performance measure. These are website usage used previously by (Alpar 2001; Quaddus and Achjari 2005), financial benefits used by (Auger 2005; Zhu 2004), and owner satisfaction used by (Huizingh 2002).

Website Usage IS researchers have demonstrated that usage is a key variable in explaining the performance impact of information technology. Seddon (1997) pointed out that system use is a good proxy for IS success when the use is not mandatory. In eCommerce, website users are customers and so their use is most often voluntary. The nature of the use and the amount of usage are both important indica-


5


tors of success and will not only impact on the organisation but will also assist an organisation in improving the quality of its website (DeLone and McLean 2003). Hence traffic measures should be determined with reference to the number of new or repeat visitors, the conversion rates and the patterns of navigation (DeLone and McLean 2004). A company should gather clickstream data from site traffic and may care to make inferences regarding the site’s effectiveness. Schuap (2006, p2) stated that "Gathering clickstream data is the predominant way of determinig success from the firm’s perspective". Quaddus and Achjari (2005) framed eCommerce success in terms of operational and strategic measures and argued behavioural statistics of visitors to the website are an operational measures. According to Epstein (2004), increased website usage leads to increased sales, improvement in sales and cost savings, all ultimately leading to profitability. Huizingh (2002) argues that the number of visitors is a more convincing measure of website performance than website sales as customers might be informed online and complete the purchase offline. Online technology however, enables the collection of large amounts of detailed data on visitor traffic and activities on websites. Such data offers a plethora of metrics from which companies should carefully choose measures for different purposes (Phippen et al. 2004). There are tools that provide various statistics about website usage (for example, Google analytics, LiveSTATS.XSP, and CMS400.NET). However such tools provide only raw metrics and are of little benefit to businesses (Hong 2007; Phippen et al. 2004; Spiliopoulou 2000). In addition, even though web metrics may be capable of extracting interesting website usage patterns and could indicate how users are actually using the website, a link to business performance is still lacking (Hahn and Kauffman 2001). Website owners need to convert these raw metrics into meaningful information that can be used in evaluating the commercial performance of their websites.

6

For the purpose of this research, website usage was captured by the metrics available to owners who utilise clickstream data. These metrics are critical to assess website activities and user behaviour. From the owners’ perspective, such metrics may suggest where improvements can be made with regard to design, layout, and navigation (Schonberg et al. 2000). Despite the limitations of clickstream data (Weischedel and Huizingh 2006), detailed and concrete data on users’ behaviour can be collected to reveal trends rather than provide descriptive data/statistics on website usage. Indeed, a reasonable measure could be determined by assessing whether the full functionality of a website is being used for its intended purposes (Welling and White 2006). For example, “pageviews” is a widely used metric and refers to the numbers of pages being viewed by website visitors. A website is said to perform well if all pages have been viewed. This is being referred to as Website relevance (Phippen et al. 2004) and it captures the navigation pattern of visitors within the Website. In another example, analytical tools provide information about the length of time visitors stay on the website. The more time visitors stay on the website the "stickier" is said to be. Hence the time spent on the website metric is better known by the term "stickiness". However, businesses that monitor their website do not necessarily use all available metrics, either because they don’t know they exist, because they don’t know how to use them or they do not have any faith in, or use of, the metric. A comprehensive set of metrics captures the extent to which the company retrieves information about its visitors in terms of numbers, navigation patterns within the site, and the visitor’s profile.

Financial benefits For websites used to sell goods and services online, the financial benefits are of particular interest. Owners are usually responsible for determining the appropriate level of investment in their website to ensure adequate benefits. Owners will usually only invest if they see their website as creating value ra-



ther than as a cost centre (Epstein 2004). According to Zhu (2004), the business value of eCommerce is best measured by gains in financial benefits. However, evaluating websites in terms of the financial gain is a daunting task as there is no single measure that can be used. Researchers have used different measures in different contexts to uncover different dimensions of financial gain. Giaglis et al (1999) observed that the most common methods of evaluating information technology investments is by way of established accounting techniques, such as Return on Investment (ROI). One layer of complication as indicated by Zhu and Kraemer (2003) is that financial performance is a multi-dimensional concept that can be measured along three dimensions: profitability, cost reduction, and inventory efficiency. Another complication is that such a construct includes measures that reflect performance in the market, such as market share and market growth. Much of the literature has focused on issues surrounding website financial benefits (Auger 2005; Barua et al. 2001; DeLone and McLean 2004; Epstein 2004; Hong 2007; Olson and Boyer 2003; Pujani and Xu 2005; Thelwall 2001; Zhu and Kraemer 2003). In essence, however, the financial measures remains essential in the performance measurement reflecting the fulfillment of economic goals (Venkatraman and Ramanujam 1986).

Owner Satisfaction The non-financial or the intangible benefits of a website have also been reported as measurements of performance. According to Kim et al (2003), the main reason businesses establish a presence on the web is to harness tangible and intangible benefits. Huizingh (Huizingh et al. 2007) argues that such benefits can be realised in terms of owner satisfaction. Owners also determine their IT expenditure according to their satisfaction with the website (Huizingh 2002). Hong (2007) defines satisfaction as conforming to individual’s expectations. Consequently, when the website drives traffic, communicates certain features that enhance the user’s experience, generates trust and strengthens the competitive

position of the business, the owners are inclined to be satisfied.

An Operational Definition Website Performance

of

Developing the Website Performance Instrument As suggested by Churchill (1979), the next step to follow the development of the theoretical domain is to formally convert the list of collated items into a measurable scales. The primary purpose of this step is to ensure that the meaning associated by the researcher is the same as that of the targeted respondents (Segars and Grover 1998). Given the multidimensionality of this concept, the three factors identified (website usage, financial benefits, and owner’s satisfaction) represent an a priori model of website performance. While each factor measures one dimension, the system of factors measures a second order factor of website performance. Working within this context, a formal conversion of the 20 performance measures shown in Table 2 onto measurable scales is undertaken. Each needs to be measurable and understood equally by respondents and the researcher. The rewordings of items in the instrument were guided by the, the length of the items, level of difficulty and clarity (DeVellis 2003).

Subjective vs. objective measure Whether for competitive or other reasons, businesses are reluctant to divulge objective measures of performance, in particular the financial benefit from their websites (Epstein 2004) preferring to communicate more subjective evaluations. According to Chen et al.(2002), the subjective measures are more convenient to obtain and free of specific targets, contexts, or time frames. Moreover, studies have shown that subjective measures of performance are closely correlated with various objective measures of financial benefits (Dess and Robinson 1984; Venkatraman and Ramanujam 1987). Consequently, asking owners for subjective self-reported opinions was adequate alternative to giving direct observations objective figures on the perfor-


7


mance of their websites. The usage metrics performance used by the targeted respondents regardless of the tool being used.

Formative vs. reflective measures Care should be taken as a construct can be measured formatively or reflectively (Diamantopoulos and Siguaw 2006). According to Hardin et al. (2008), this is important to decide early in the measurement development process as it affects the content of the measure. The primary difference between a reflective and a formative measurement is that while the construct causes variance in its reflective indicators, the direction of causality is reversed such that the formative indicators cause variance in the construct (Cenfetelli and Bassellier 2009). According to Chin (1998b), the choice between measuring latent constructs with formative or reflective indicators should be based on the research objectives, the substantive theory for the latent construct, and the empirical conditions as shown in Table 3. The focus of this study was to develop a website performance measuring instrument from the owner’s perspective.The observed variation and covariation of the observed variances are of interest. Moreover, for reasons mentioned above, items of performance were chosen to be measured subjectively i.e. by attitude of the respondents. In such case reflective formulation of construct is appropriate (Diamantopoulos and Siguaw 2006; Hardin et al. 2008; Marakas et al. 2008).

were reworded to capture the specific metric

The Format and the pre-test of the Instrument A Likert scaling is often used when developing an instrument which measures opinion, belief and attitudes (DeVellis 2003). Since this study examined the belief of the participants towards what counts towards the performance of their websites, a Likert scale was deemed suitable. However, as argued by Venkatraman and Ramanujam (1986) and because the study is concerned with the performance of websites from the owners’ perspective, their response is expected to be biased in favour of their website. Therefore, asking the importance of an item alongside their perceived success on that item becomes more meaningful in capturing the actual performance. Hence, each performance measure (Table 2) was formatted to ask the respondents to indicate how important each item was in the performance of their website and then indicate their perceived success for the same item. A resultant composite measure (that is used in the analysis which is “importance” * “perceived success”) is referred to as the “effective” performance measure as it takes into account the importance of each item in the evaluation process (Auger 2005). The option of not applicable/available (NA) was also provided in order to classify missing values as a ’user missing value’ when NA is selected or a ’system missing value’ when value was not recorded.

Table 3 - Chen’s (1998b) criteria for determing whether a constructs is reflective or formative Criterion 1. Research Objective Address the researcher’s purpose for employing a particular model

Formative

Reflective

Account for the unobserved variance (Abstract level)

Account for the observed variance (measurements level)

2. Substantive theory Addresses the underlying conceptual properties of constructs

Constructs determined by an explanatory combination of variables

Psychological constructs in which people respond according to their belief

3. Empirical condition Addresses covariation among the indicators

Covariation can adversely affect the stability of indicator coefficients

Covariance among indicators is expected

8



A panel of academics and a panel of business owners reviewed the selected items for contents, wording, clarity, format, ease of use, and appropriateness. The feedback led to some changes as some items were slightly refined. The resulting questionnaire consisted of 20 items measuring the three factors are shown in Appendix B.

Data Collection Strategy

and

Analysis

An online survey was chosen because: •

The targeted businesses had a website in place, were assumed to be computerliterate and contactable by email (Saunders et al. 2003).

•

Respondents can complete the questionnaire at their own convenience (Saunders et al. 2003) and with greater anonymity (Sekaran 2003).

•

It offers the ability to survey a wider geographical area (Sekaran 2003) with different demographic profiles (McDonald and Adam 2003).

•

It is the least expensive means of gathering data; has less pressure for immediate response, has no interviewer bias; and data may be automated (McDonald and Adam 2003; Sekaran 2003).

The population for this study was online businesses within New Zealand. A link to the online survey was sent out by email to businesses engaged in eCommerce. A total of 1093 emails were sent, and 344 responded giving a 31.47 % response rate. However, there were only 225 (20%) businesses that were actively engaged in web analytics. This was considered to be appropriate for the growing field of eCommerce (Couper 2000; Sheehan and McMillan 1999). 93.1% of respondents were owners of businesses. This is highly relevant to this study as the owner perspective is the focus. Other respondents include internet sales, marketing manager, eCommerce manager, web manager, customer service manager, and operation manager. The respondents indicated that

their website has been active for an average of 7 years (ranged from 1 month to 120 months). The percentage of online business sales compared with offline business ranged from 1 to 100%. The sample has: 100% online business (22%), more than 90% (36%), less than 10% (23%), with the remaining 19% between 10% and 90%. Companies in the sample sold to both individual customers (42.7%), and companies (57.1%). 21.8% of companies had no employees and are run solely by their owners; 14.7% have 5-19 employees and 8% have 20-50. The respondents were asked about the goal(s) of their website and were given seven pre-defined goals to choose from. Respondents could select more than one goal. The selling products goal was the highest (96.0%), which adheres to the objective of the present study, the marketing goal was the next most important goal. The data was analysed using factor analysis to uncover the underlying factors of all listed items. This was complemented by fitting a confirmatory model in AMOS based on factors identified in EFA. This additional step was taken to assess the dimenionality of identified performance factors. With AMOS it is also possible to provide empirical evidence for trait validity and contstruct reliability for items and factors of website performance. The following fit indices CFA, GFA, and RMSEA with the values >0.9, >0.9 (Byrne 2001a), and < 0.08 (Newcomb 1994) respectively and the CMIN/DF, are the indices chosen to determine the model fit in this study. Prior to any statistical examination the dataset was evaluated for applicability to the model (Hair et al. 1998). Data were analysed for non-response bias, missing data and data normality. Nonresponse bias was tested by assessing the differences between the early and late respondents with regard to the means of all the variables for both samples (Armstrong and Overton 1977). Early respondents were defined as the first 60% of the returned questionnaires, and the last 40% were considered late respondents. These proportions approx-


9


imate the actual way in which questionnaires were returned. No significant differences between the early and late respondents were found, suggesting that response bias was not an issue. Another important issue is how to deal with missing data (Hair et al. 1998; Kline 2005). Although there is no clear guideline, 5% or 10% missing data on a particular variable is usually acceptable (Cohen et al. 2003). All missing data were imputed for SEM and avoided excessive deletions in the dataset. It was noted that while missing data were randomly distributed across the sample, Little’s MCAR test indicated that data were not Missing Completely At Random (MCAR). Importance rated items have chi-square = 5236.247, p = 0.032. Perceived success rated items have chi-square = 7179.198, p = 0.035. According to Little and Rubin (1989), if data are not MCAR, missing values should be imputed. While there are different ways to impute data, the missing values were replaced with the mean value of the variable based on all valid responses (Hair et al. 1998). Finally, the data were checked for normality using the Mardia statistic (a measure of multivariate kurtosis). A critical ratio above 1.96 signifies departure from multivariate normality with 95% confidence. AMOS provided the Mahalanobis distance for each case in the dataset, where the greater the Mahalanobis distance, the greater the contribution to the departure from multivariate normality (AMOS user Manual). Upon checking the Normality portion of AMOS output, the joint multivariate kurtosis was 39.03 and the associated critical ratio was 12.96 indicating non-normality data (AMOS user Manual). Outliers were related to the question of normality (Byrne 2001b) which can also be reported from the normality portion of AMOS output through the Mahalanobis distance. The small gap in Mahalanobis d2 values between cases suggested no outliers, and all cases were included in the analysis. Bollen-Stine p-value is usually used when data is not normal. The Bollen-Stine p-

10

value was 0.39 indicating that the model was an acceptable fit for the data.

Model Analysis and Results Exploratory Factor Analysis (EFA) In order to determine factors associated with performance, an exploratory factor analysis was arried out. Varimax rotation was specified to identify items that might indicate potential factor. An Eigen value greater than 1 and percentage of variance extracted that accounts for at least 5% of the common variance were the criteria used to obtain a meaningful factor of the items. Each item loadings were examined at 0.5 and above on each potential factor (Chin 1998a; Hair et al. 1998). As shown in Table 4, some items were deleted due to cross loading. 18 items and three factors were extracted. Each scale items captured a significant amount of variation for each factor and a total variance was 61.63%. Internal consistency appeared to be high indicated by Cronbach’s alpha.

Confirmatory Factor Analysis (CFA) A CFA using AMOS 17 was used to justify the underlying factor structure revealed by the EFA and assess the reliability and validity of factors and items. A first order and a second order CFA were conducted.

A first-order CFA model A first-order CFA specifies the pattern by which each measure loads on a particular factor given more than one latent factor which is then tested for validity using CFA procedures (Byrne 2001b). The factors (Usage, Financial and Satisfaction) were tested as a collective network. The results showed that these factors were distinct but correlated. In other words, while these dimensions were independent in their prediction of website performance, the change of performance along one implies changes along the others. This is expected results for reflective measurements. All measures showed loadings of more than 0.5 the usage of the website; six measures represent the financial benefits and three measures represent owner satisfaction. Data



Table 4 - Exploratory Factor Analysis 1

Loadings 2

Number of Visitors Repeat Visitors Conversion Rate Website Relevance Website Stickiness Navigation Tracking Customer Profile User Environment Referring Website Reach Bounce Rate

0.542 0.546 0.562 0.616 0.688 0.655 0.689 0.592 0.607 0.668 0.633

0.162 0.151 0.552 0.242 0.363 0.383 0.495 0.358 0.590 0.355 0.356

Financial

Return on Investment Online Sales Sales Growth Profit from Website Cost Reduction Market Share Increase

0.384 0.354 0.388 0.343 0.105 0.115

Satisfaction

Competitive Position No. of Customer Search Engine Ranking

0.234 0.306 0.290

Factor

Usage

Item

Eigen value

%Variance

Alpha

0.288 -0.028 0.450 0.053 -0.161 0.098 0.292 0.218 0.340 -0.025 0.381

7.932

44.068

0.855

0.804 0.838 0.836 0.850 0.632 0.667

-0.178 -0.206 -0.206 -0.222 -0.271 -0.187

1.916

10.647

0.920

0.499 0.452 0.388

0.683 0.759 0.726

1.245

6.915

0.825

3

Table 5 - Goodness-of-fit statistic for model in Figure 1 Model

χ2

df

P

χ2 /df

RMSEA

TLI

CFI

GFI

Default model

223.83

116

0.00

1.93

0.06

0.94

0.95

0.90

were fitted satisfactorily as shown in Figure 1 and Table 5.

used to test the efficacy of such structure (Bollen 1989; Joreskog and Sorbom 1993).

A second-order CFA model

As shown in Figure 2 the performance construct is a second-order factor governing the correlations among Usage, Financial Benefits, and Owner Satisfaction. The fit statistics in Table 6 support the existence of such a structure (Bollen-Stine p-value also indicated model fit). These statistics indicate not only a good model fit but also suggest empirical acceptance of the model. The factor estimates revealed strong significant relationships for website performance acting as a function of Usage (0.75), Financial Benefits (0.87), and Owner Satisfaction (0.83). All standardised

As theorised, a website performance is evidenced across multiple dimensions. The correlations among the three dimensions shown in Figure 1 suggest that the website performance is an aggregate of Usage, Financial Benefits, and Owner Satisfaction. While the first-order CFA facilitated the inspection of how well the scale items measures each dimension and captured the correlations among them, the second-order CFA can be


11


estimates were substantively reasonable and statistically significant at p