developing a navigation evaluation framework for ...

Proceedings of the 11th Annual Conference of Asia Pacific Decision Sciences Institute Hong Kong, June 14-18, 2006, pp. 274-280.

DEVELOPING A NAVIGATION EVALUATION FRAMEWORK FOR BUSINESS WEB SITES: BASED ON WAYFINDING THEORY Benjamin Lum, Byounggu Choi, Simon K. Poon University of Sydney, Madsen Building F09, University of Sydney, NSW 2006, Australia EMAIL: [email protected], [email protected], [email protected]

ABSTRACT Internet usage across the world has been growing rapidly and as a result, the uptake of e-commerce by consumers has been rising. However, the conversion rate of visitor-to-purchasers is still quite low due to the lack of sophisticated web navigation design. Although a large amount of research has been conducted to solve this problem, most of this research still suffers from limitations such as lack of theoretical foundation, generalized criteria, and lack of consideration for specific domain characteristics. To fill this gap, this research proposes a framework for evaluating navigation in electronic commerce. Wayfinding theory is adopted as the theoretical foundation. The process of cognitive mapping in wayfinding is used to categorize web site navigation design into four dimensions: (I) orientation; (ii) route decision; (iii) route monitoring; and (iv) destination recognition, and for each dimension, more specific hands-on evaluation criteria is developed. To provide preliminary validation for the proposed framework, a latent content analysis was performed by applying the framework to existing electronic commerce web sites and observing how well the framework discriminate the well designed web sites from the poorly designed sites. This research has some implications for managers and academics. The proposed framework may be used by managers by applying it to their electronic commerce web site and assessing their supports for navigation. Furthermore, it can be used as a starting point for extending the evaluation framework or expanding the concept to create similar frameworks for other web site aspects and domains. Keywords: e-commerce, website design, wayfinding theory

INTRODUCTION The growth of Internet usage has been rapidly rising over the past few years. This has resulted in an increasing number of commercial web sites available and an increase in the acceptance of e-commerce by customers. This growth can be seen in the continuously escalating revenue from online retailing. It is expected that online retail sales will grow up to US$329 billion by 2010 [5]. However, despite the huge number of sales through e-commerce, the average rate of business-to-consumer web site visitors converting into customers was a mere 4.9 percent [11]. Prior research has found that web interface design can have an effect on consumer purchasing behavior and is consistently mentioned as critical to the success of e-commerce. Among web interface design, the navigation component is crucial in the success of an ecommerce web site, as it makes information easier to find and allows users to acquire more of the information they are seeking [13]. It has been shown that 60 percent of experienced web shoppers gave up purchasing an item from an ecommerce web site due to navigation problems [20]. Although numerous researchers have explored the area of web site design evaluation, relatively little attention has been paid to navigation. Furthermore, most of this research is often limited because it is very difficult to carry out in-depth analysis to cover all aspects of the web site design in a comprehensive way. In addition, other research lacks consideration for domain specific characteristics and requirements. Above all, much of the research lacked a strong theoretical foundation. To overcome these problems, this study proposes a web site evaluation framework on the basis of wayfinding theory. More specifically, this research adopts wayfinding theory to attempt to understand the navigational behavior of users and their requirements in the e-commerce web site environment.

DEVELOPING A NAVIGATION EVALUATION FRAMEWORK FOR BUSINESS WEB SITES

275

This remainder of this paper is organized as follows. Section 2 reviews and synthesizes the related previous research in web site design. Section 3 details the development of the proposed navigation evaluation framework for e-commerce web sites based on wayfinding theory. Section 4 shows the preliminary validation undertaken and the implications are discussed. Section 5 presents the current status and contribution of this research. REVIEW OF LITERATURE Web Site Design and Evaluation

Aspect-specific

Domain- and aspect-specific

General

Domainspecific

Aspect driven

Domain driven Figure 1: The four categories of web site design research Previous research which focused on web site design can be broadly classified into four categories: (1) general web site design; (2) domain-specific web site design; (3) aspect-specific web site design; and (4) domain- and aspect-specific web site design (see Figure 1). This can be represented along two axes: aspect driven, where the research is driven by a particular aspect of web site design such as usability and trust; and domain driven, where the research is driven by a particular web site domain such as e-commerce or financial domains. Studies classified as general web site design focused generically on both domain and design aspects. Therefore, they only concentrated on very general web site design aspects applied them to general web site domains. For example, Hong et al. [8] developed a generic set of evaluation criterion and guidelines with both papers employing the use of the survey method for the validation of their criteria and guidelines covering aspects such as reliability, security, navigation and attractiveness, and applied it to a generic domain. The second category focused on developing criteria that is specific to a particular domain. Similar to general web site design research, research in this category also focused on general aspects in web site design, however, developed their design guidelines specific to a particular domain. For example, Fang et al. [4] developed a set of nineteen design rules for e-commerce web sites. The third category covers research that focuses specifically on an aspect of web site design for a general domain. For example, Tsygankov [21] uses the SAFT-model to develop a set of evaluation points for assessing trustworthiness from web site design. Studies in the forth category focused on a specific domain and also a specific aspect of web site design. The advantage of taking this approach is that it allows researchers to delve deeper into the specifics of that aspect in a certain domain and able to provide insights into how particular web site design features affects their users. For example, Hong et al. [8] explored the effects of information format in online retail catalogues on shopping behavior by comparing the list format with matrix format. Category Study Hong et al. (2004) General Zhang et al. (2001) Zhang et al. (2000) Fang et al. (2003) Domain Kim et al. (2002) specific Shchiglik et al. (2004) Aspect Katerattanakul et al. (1999) specific Palmer (2002)

Theory or Model Architectural metrics Kano's Model of Quality Two Factor Theory None Architectural metrics WebQual 3.0 None None

Focus Web site evaluation Web site quality Web site design Customer-centered design rules Internet business quality Web site quality Information quality Metrics for usability, design and performance

Validation Survey Survey Survey Interviews Survey Survey Survey Content analysis

276

BENJAMIN LUM, BYOUNGGU CHOI, SIMON K. POON

Domainand aspectspecific

Tsygankov (2004) Agarwal et al. (2001)

SAFT Web site evaluation – trust Microsoft Usability Usability Guidelines Haubl et al. (2000) None Decision aids Hong et al. (2004) Cognitive fit perspective Information format Park et al. (2000) None Navigation Table 1: Comparison of previous studies in web site design

None Survey and experiment Experiment Experiment Experiment

Synthesis of Previous Research Synthesis of previous research revealed some interesting implications. First, it was found that much of the past research lacked a strong theoretical foundation. Because a strong theoretical foundation is important in research to ensure that the criteria defined are comprehensive and do not overlook any aspects [12], research lacking in theoretical foundation cannot guarantee a result that is comprehensive and valid. Second, much of the past research lacked consideration for the generic requirements and characteristics of web sites in different domains. This is evident in research that builds its set of criteria in a particular domain may not be applicable for other domains, or sometimes entirely omitted in the design evaluation criteria. Finally, it was found that numerous studies provided only general criteria with adequate depth to evaluate web site design. This usually results from studies which attempt to broadly cover a large number of web site design aspects. These general criteria are insufficient for managers to evaluate their business web site as they can overlook important specific web site design criterion for enhancing a customer’s shopping experience. DEVELOPING THE EVALUATION FRAMEWORK Theoretical Background Wayfinding is a general theory that originated from the field of architecture that attempts to explain the human process of navigating physical environments, such as a building or a city. Wayfinding can be referred to as how well people are able to find their way to their particular destination with minimal delay and anxiety [18]. Wayfinding theory proposes that cognitive mapping is essential when relocating from one place to another. Cognitive mapping can be defined as processing and organizing information from past and current experience, in order to generate a series of plans to navigate the physical environment and complete their task [17]. Cognitive mapping in wayfinding involves four distinct stages [3]: 1. 2. 3. 4.

Orientation: Determining the current location in respect to nearby objects and the destination. Route Decision: Choosing a route that will get to the desired destination. Route Monitoring: Monitoring the route taken to confirm that the route is in the correct direction. Destination Recognition: Recognizing that the correct destination, or at least a point nearby, has been reached.

Developing the Framework Even though wayfinding was developed for the physical environment, several researchers have successfully applied wayfinding theory to understand navigation in virtual reality [2] and the World Wide Web [7]. To further enhance the existing research, this research proposes a navigation evaluation framework on the basis of the cognitive mapping process performed in human wayfinding. Therefore, the proposed evaluation framework will be based on four dimensions: (1) orientation; (2) route decision; (3) route monitoring; and (4) destination recognition (see Table 2). Orientation Dimension Orientation in wayfinding involves determining the current location in respect to nearby objects and the destination. From the perspective of web site design, orientation in web design involves supporting the user by providing features that allow the user to determine where they are currently located and also allow the user to see what web pages are around them. The orientation dimension consists of two web site evaluation categories: (I) spatial awareness; and (ii) visual appearance. Spatial awareness assists users in developing a mental model of the relationship between the different sections of the web site and determining their location with respects to nearby pages and their destination. A site map and the use of menus can assist users in gaining spatial awareness. The visual appearance of a web site is what provides the users with their first impressions of the web site and tends to set the tone and theme of the web site. It also serves as a means for a web site to create a unique identity [23]. This identity can be used to help a visitor identify their location on the web. Visual appearance is supported by features such as eye catching images or titles and consistency.

DEVELOPING A NAVIGATION EVALUATION FRAMEWORK FOR BUSINESS WEB SITES

277

Route Decision Dimension Route decision involves a user choosing a route that will get them to their desired destination. From the perspective of web site design, the dimension of route decision can be adapted to provide users with features that will assist them in choosing their route. The route decision dimension consists of three categories: decision aids to assist the user in choosing a product [6]; personalization to attempt to push links to the user which may be useful to them [1]; and page linking and structure to ensure links are visible and shortcuts are available to users. The three categories defined for route decision can assist the user in deciding which route to take, and also reduces the amount of effort required in choosing a route. In an e-commerce environment, the route decision can also involve the decision of which product to view. The criterion defined in each of these categories also supports this form of route decision. Wayfinding Process

Evaluation Category

Feature

Orientation

Spatial awareness

Menus Site map

Visual appearance

Consistency Color use Eye catching images or titles Attractiveness

Route decision

Decision aids

Recommendation agent Comparison matrix

Personalization

Adjusts contents to match customer’s interests

Page linking and structure

Links visibility Shortcuts availability Alternative paths Minimize external hyperlinks

Route monitoring

Location identification

Indicator of webpage location within the web site Label of current position

Path identification

Indicator of path Provide clear navigational cues

Destination Recognition Information quality

Information accuracy Up-to-date information Completeness of information

Relevant information Table 2: The proposed evaluation framework Route Monitoring Dimension Route monitoring in wayfinding involves monitoring the route taken to confirm that the route is in the correct direction. In web site design, route monitoring provides features which will allow users to track the path they have taken, identify where they are, and also assist in deciding which path to take next. The route monitoring dimension provides support for users to identify their current location and cues to assist users in deciding which path to take next [14]. Destination Recognition Dimension Wayfinding defines destination recognition as recognizing that the correct destination, or at least a point nearby, has been reached. From the perspective of web site design, the destination recognition dimension evaluates the information quality [24] of the web site, as users will know they have reached their destination when they find the information they

278

BENJAMIN LUM, BYOUNGGU CHOI, SIMON K. POON

require. The quality of the information provided at the destination is important, as few consumers make their purchasing decisions based on the name of the product alone. PRELIMINARY VALIDATION To evaluate the validity and usefulness of the proposed evaluation framework, a latent content analysis was performed by applying the framework to existing e-commerce web sites and observing how well the framework discriminate the well designed web sites from the poorly designed ones. Four samples were selected with the assistance of BizRate. BizRate is a third party web site that keeps track of the reputations of electronic commerce web sites and ranks them according to information collected from actual consumers. To compile the sample, the top (ClubMac and OneCall) and bottom two (PriceRite Photo and MyTaxFree) electronic commerce web sites listed in BizRate’s “Computers and Software” category were selected. This category was selected due to the ever-changing nature of technology and the vast array of products available in the industry. Table 3 shows the evaluation result.

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

MyTaxFree

Y

Completeness of information

Y

Y Y

Relevant information

Y

Y

Up-to-date information

Y

Y

Information accuracy

Y

Y

Indicator of path

Alternative paths

Y

Y

Provide clear navigational cues

Shortcuts availability

Y

Y

Label of current position

Links visibility

Y

Y

Destination recognition

Indicator of web page location within the web site

Minimise external hyperlinks

Y

OneCall PriceRite Photo

Adjusts contents to match customer's interests

ClubMac

Comparison matrix

Recommendation agent

Y

Y

Consistency

Y

Y

Attractiveness

Y

Colour use

Low

Y Y

Eye catching images or titles

High

Website

Menus

Rank

Route monitoring

Route decision

Site map

Orientation

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Table 3: Summary of comparison

Comparison of Real-World Web Sites Based on the Proposed Framework Orientation Dimension The top two web sites provide spatial awareness by using a well defined menu layout on its web pages. For example, to assist users in orientation, ClubMac and OneCall provides a comprehensive menu system on the top navigation bar providing access to the product categories with rollover functionality to allow users to see in detail all the product types in each category. This allows users to gain awareness of the pages around them and where their destination is. In addition, visual appearance has also been used by them to support orientation on their web site. Eye catching images and titles have been placed throughout the site to inform users of where they currently are. In contrast, the bottom two web sites do not support spatial awareness and visual appearance. For example, the overuse of MyTaxFree’s business colors of red, gold and black contribute to the overall unattractiveness of the web site, as well as making it difficult for a user to identify the different sections of a page. Route Decision Dimension Both high ranked web sites provide decision aids by using a recommendation agent or comparison matrix. For example, ClubMac utilizes a recommendation agent on the product listing pages and category pages to provide shortcuts to best selling and featured products. OneCall provides a sophisticated search feature utilizing natural language processing that allows users to input questions to assist users in deciding their route. However, the route decision dimension is poorly supported by the bottom two web sites. For example, the use of an image map on the MyTaxFree home page to provide

Y

DEVELOPING A NAVIGATION EVALUATION FRAMEWORK FOR BUSINESS WEB SITES

279

the list of all brands available in each of the categories makes it difficult for a user to be sure where each of the links are. In addition to this, a number of the brands listed in the image map are dead links and clicking on simply refresh the home page. Route Monitoring Dimension ClubMac provides location identification by using tabbed navigation to allow users to identify which section of the site they are currently in, which is a consistent feature on all their pages. Both ClubMac and OneCall utilize a breadcrumb navigation feature on the top of every page which provides users with a label of their current position in order to support path identification. In contrast, route monitoring is another dimension poorly supported by MyTaxFree and PriceRite Photo. While navigating through the web site, there is no indication of where a user is on the web site, nor is there an indicator of the path taken by the user to get to where they are. For example, when a user is viewing the Sony digital cameras page in MyTaxFree, there is no clear title or heading indicating that it is the digital camera section, or the Sony brand. Destination Recognition Dimension The two top web sites provide high quality information to assist users that are in the process of making a purchase decision. They provide complete information regarding the product on the product pages by sourcing information direct from the manufacturer’s web site. However, there is a great deal of irrelevant information presented to the two bottom web sites user while navigating to the product page, a user may not be able to recognize they are close to their destination. For example, MyTaxFree includes on each of the product listing pages a list of top sellers which is located predominantly on the left hand site, listing items which are not relevant as they are not of the brand or the type that the user wished to view, which may cause confusion for the user. FINDINGS AND IMPLICATIONS While they do not support some criteria such as comparison matrix (shown in Table 3), the two top web sites support almost all evaluation criteria proposed by our framework. In contrast, many evaluation criterions are not supported by the bottom two web sites. On the basis of our framework, a manager can find the limitations of their web sites and a way of supporting their customers effectively. For example, ClubMac is lacking a feature that allowed their potential customers to compare products in matrix, which can assist users in making their purchasing decision. Providing these features can improve a visitor’s ability to navigate through their website. The emphasis of such comparison is to show how the proposed evaluation criteria adequately discriminates between well designed and poorly designed web sites. The exploration according to the latent content analysis indicates the potential applicability of the framework. Although the four websites selected are not a true representation of the whole population, this preliminary exploration has illustrated how managers can develop well designed web sites to support potential consumers’ navigation activities. CURRENT STATUS AND CONTRIBUTIONS The motivation of this research is to improve the navigation support in e-commerce web sites. The contribution made by this research is a navigation evaluation framework based on wayfinding theory. This framework can be used for assessing the supports of navigation on e-commerce web sites. Based on the framework proposed in this paper, an experiment will be performed in the next phase. A sample of participants with reasonable homogeneous level of IT skills will play the role as online consumers to further validate out framework. The results will provide theoretical and managerial insights towards the navigation requirements of consumers in e-commerce web site design. In addition, it will also assist managers who wish to establish effective business web sites.

REFERENCES [1] Agarwal, R. and Venkatesh, V. "Assessing a firm's web presence: A heuristic evaluation procedure for the measurement of usability," Information Systems Research, 2001, 13(2), 168-186. [2] Darken, R. P. and Siebert, J. L. "A toolset for navigation in virtual environments," UIST Proceedings, 1993. [3] Downs, R. M. and Stea, D. "Cognitive maps and spatial behavior: Process and products," Image and Environment, 1973, 8-26. [4] Fang, X. and Salvendy, G. "Customer-centered rules for design of e-commerce web sites," Communications of the ACM, 2003, 46(12), 332-336.

280

BENJAMIN LUM, BYOUNGGU CHOI, SIMON K. POON

[5] Forrester. Forrester research us ecommerce forecast: Online retail sales to reach $329 billion by 2010, Forrester Research Press Release. 2005. [6] Häubl, G. and Trifts, V. "Consumer decision making in online shopping environments: The effects of interactive decision aids," Marketing Science, 2000, 19(1), 4-21. [7] Hodkinson, C., Kiel, G., and McColl-Kennedy JR. "Consumer web search behavior: Diagrammatic illustration of wayfinding on the web." International Journal of Human-Computer Studies, 2000, 52(5), 805-830. [8] Hong, S. and Kim, J. "Architectural criteria for website evaluation - conceptual framework and empirical validation," Behavior & Information Technology, 2004, 23(5), 337-357. [9] Hong, W., Thong, J. Y. L., Wong, W., and Tam, K. Y. "The effects of information format and shopping task on consumers' online shopping behavior: A cognitive fit perspective," Journal of Management Information Systems, 2004, 21(3), 149-184. [10] Katerattanakul, P. and Siau, K. (1999). "Measuring information quality of web sites: Development of an instrument," Proceeding of the 20th international conference on Information Systems (ICIS 1999), Charlotte, North Carolina, United States, Association for Information Systems. [11] Kerner, S. M. Bricks-and-mortar retailers see lower online conversion rates, Clickz Network. 2005. [12] Kim, J., Lee, J., Han, K., and Lee, M. "Businesses as buildings: Metrics for the architectural quality of internet businesses," Information Systems Research, 2002, 13(3), 239-254. [13] Machlis, S. "Site redesigns keep it simple," Computerworld, 1998, 32(43), 43-44. [14] Olsina, L., Lafuente, G., et al. "E-commerce site evaluation: A case study," Lecture Notes in Computer Science, 2000, 1875, 239-252. [15] Palmer, J. W. "Web site usability, design and performance metrics," Information Systems Research, 2002, 13(2), 151-167. [16] Park, J. and Kim, J. "Effects of contextual navigation aids on browsing diverse web systems," ACM Conference on Human Factors in Computing Systems, 2000, Hague, Amsterdam. [17] Passini, R. Wayfinding in architecture, NY, Van Nostrand Reinhold, 1992. [18] Peponis, J., Zimring, C., and Choi, Y.K. "Finding the building in wayfinding," Environment and Behavior, 1990, 22(5), 555-590. [19] Shchiglik, C. and Barnes, S. J. "Evaluating website quality in the airline industry." Journal of Computer Information Systems, 2004, 44(3), 17-25. [20] Spool, J. M., Scanlon, T., Schroeder, W., Snyder, C., DeAngelo, T. Web site usability: A designer's guide, San Diego, CA: Academic Press, 1998 [21] Tsygankov, V. A. "Evaluation of website trustworthiness from customer perspective, a framework," Proceedings of the 6th international conference on Electronic commerce (ICEC2004), Delft, The Netherlands, ACM Press, 2004. [22] Zhang, P., Small, R. V., von Dran, G.M., and Barcellos, S. "A two factor theory for website design," Proceedings of the 33rd Hawaii International Conference on System Sciences (HICSS 2000), Maui, HI, IEEE, 2000. [23] Zhang, P. and von Dran, G. M. "Expectations and rankings of web site quality features: Results of two studies on user perceptions," Proceedings of the 34th Annual Hawaii International Conference on System Sciences (HICSS 2001), Maui, HI, IEEE, 2001. [24] Zhang, X., Keeling, K. B., Pavur, R.J., "Information quality of commercial web site home pages: An explorative analysis," Proceedings of the Twenty First international Conference on Information Systems (ICIS 2000), Brisbane, Queensland, Australia, Association for Information Systems, 2000.