Web Site Evaluation Index: A systematic methodology and ... - CiteSeerX

Proceedings of the 8th WSEAS International Conference on Telecommunications and Informatics

Web Site Evaluation Index: A systematic methodology and a metric system for the assessment of the quality of web sites DIMITRIOS XANTHIDIS, PARIS ARGYRIDES and DAVID NICHOLAS University College London [email protected]

Abstract: - In a previous paper the authors introduced a template for the evaluation of web sites. This was tested using a sample of a number of Greek companies’ web sites and it proved to be useful enough as a general evaluation tool. Soon thereafter the authors realized the need to come up with a measurement and a metric system as a step ahead of that evaluation template. The WSEI (Web Site Evaluation Index) can be used not just to evaluate if a website is good or not, but also, how good it is especially when compared with other of similar quality. Key-Words: - Evaluation template, web sites, measurement, metric, internet, eCommerce This study is divided into two parts. In this first part, the WSEI is introduced and explained thoroughly both in terms of how it was designed and how it can be used. In the second part (a paper following) the WSEI is used to evaluate a sample of international Web sites in an attempt to test whether its application is feasible and its results interpretable.

1 Introduction The exponential increase of the number of web sites currently available on the Net together with the rapid growth of eCommerce activity especially in the developing digital economies lead the authors to design a template that could be used to evaluate the quality characteristics of these web sites. This evaluation template was introduced in a previous paper, tested using a sample of 232 Greek companies’ web sites of a considerable size and proved that a systematic methodology can be formed, to allow website designers to create more attractive and functional websites [1]. The evolution of the internet technologies, however, brings higher connection speed, improved security protocols, advanced content delivery mechanisms and more attractive design tools. Hence, the authors decided it was necessary to update the suggested Web Site Evaluation Template (WSET), by including new features and possibly, omitting obsolete elements. The updated Web Site Evaluation Index, from now on WSEI, attempts to improve the rating system of certain aspects of websites, which are not easily measurable, such as attractiveness, which depend more on personal opinion rather than scientific assessment. More important than this, however, by utilizing the Web Site Evaluation Index, we are proposing a metric that will not only evaluate if a website is good or not, a suggestion rather than a scientific evaluation, but also how good it is when compared to other similar web sites. Having a metric rather than just a measurement is important because it provides feedback that the designer can use to improve the current website elements. Suggesting that a website is good or bad does not help pinpoint the elements that need to be added or modified and to what extent.

ISSN: 1790-5117

2 Aims and Objectives The aim of this first part of the study is to introduce the Web Site Evaluation Index (WSEI) and its use in assessing the quality of a web site. For this purpose the following steps were followed:  Update the previous Web Site Evaluation Template to encompass newly introduced internet technologies such as updated security features, content delivery mechanisms and compatibility features with other than desktop PC devices.  Check existing evaluation measures to ensure whether they are still applicable by today’s standards.  Omit those vague issues such as a website being “appropriate and appealing” which may not be objectively evaluated.  Attach a “value” tag on each of the measurement points that will help the assessment process of the web site using this methodology.

3 Background Previous research showed that the website structure could be separated into four key dimensions (i.e. stickiness, customization and globalization, accessibility/availability and hard/software requirements and security and privacy) each containing several inter-related elements [1]. The

194

ISBN: 978-960-474-084-0


The first step to correctly evaluate any number of web sites is to separate them in different stages of complexity in order to avoid comparing highly complex eCommerce websites with simple informational ones. The five stages follow [8]:  Stage 1: No internet representation.  Stage 2: Marketing: The website contains static, not frequently updated content, which serves to promote the company’s profile and some basic product/service or other information.  Stage 3: Catalogue Provision: The web site may include online product catalogues, either downloadable or through active content.  Stage 4: Transactional stage: Bi-directional communication is established between the website and the user. It may include online chat or helpdesk, monetary handling through 3rd party Credit card processing systems. Web sites at this stage go beyond the limits of software since human-to-human interaction might be possible through the product delivery.  Stage 5: Interactive stage: The web site, the company’s online system rather, basically supports all company’s internal processes to appear as the company itself. Physical company location might just be a warehouse for reserving product stock and an office for online support. All transactions are conducted online through sophisticated managing systems. Such systems may include credit card handling, automatic notification for pickup place and destination of product, automatic stock management, etc.

evolution of technology, though, allowed IT professionals to incorporate more elements, enhance existing and remove others [2]. Thus, as an example of a rather obsolete element, one could point to the “best viewed in 640x480 resolution”, where a website is designed to be optimally viewed in that low resolution, typically reserved for personal computers over a decade old [3]. On the other hand, the wide use of streaming media websites, nowadays, such as youtube.com, introduced new technologies in content delivery and media compression leading several website designers to adopt higher standards [4, 5]. This is the reason why it was decided to review the existing Web Site Evaluation Template (WSET) through further research to meet the new standards. The previous evaluation template used the four basic dimensions mentioned before to assess websites treating them as equal. However, treating all websites the same way, experience shows, is not fair and could produce biased results for some websites. For example, nobody expects from a simple informational website to implement antivirus scanners, intranet access, list of products not designated for export, SSL protocols, etc. [1]. Additionally, some elements, such as the attractiveness of the website, or the annoying elements, are based on the reviewer’s subjective opinion rather than systematic and proven assessment making the respective measurement results debatable to say the least. Similarly, as to the term “appropriate and appealing” used in WSET as a separate element it would be preferred to combine the results of the “stickiness” dimension to determine whether a website is appealing-attractive or not thus eliminating the possible evaluator’s bias [6]. Another pointed weakness of the template was that some of its features address rather similar issues. For example, the presence of floating hyperlinks is only applicable when a scrolling mechanism is used. Since a “proper” website should avoid using scrolling mechanisms two of the related questions could be combined in a form of “reduced usage of scrolling mechanisms and presence of floating hyperlinks”. Several websites have tens of hyperlinks in their home page which may induce problems when checking each and every link for validation. Commercial applications exist, fortunately, for such a purpose like Inspyder’s InSite Web Link Validator. Furthermore, having online surveys to receive user feedback can become an overload sometimes, especially on lower stage websites. Instead it could be suggested that a website has at least two different communication methods.

ISSN: 1790-5117

The second step is to select the elements and features to be implemented aiming to make it attractive which are collectively labelled “stickiness” in the reviewed original WSET template [1]. Then, the next step is to decide on the scope of the web site providing relative information, a concept often referred to as globalization, as well as determine the level of customization, if any, that could help the user [1, 9]. This decision, when implemented could cause positive first impressions to some populations or culture shock to others. Therefore, the strategy followed in designing the site should address whole populations [1]. Since the websites will be categorized depending on their level, it would be wiser to associate a suitable language pack with each level. Level 1-3 websites should include at least the English language along with the company’s native language (Other Language section). Premium websites that target audience outside the country’s boundaries should include other globally spoken

195

ISBN: 978-960-474-084-0


Security protocols are required when a transactional (L3) website handles sensitive information. A secure protocol uses a cipher to encrypt data, so it would be better to merge the two questions (Protocol used & Cipher strength) in a single question in the form of “If transactional/interactive, is a secure protocol used?” Regarding an anti-virus embedded in the website, we believe that this option is limited to only websites that handle attachments, such as gmail, hotmail, yahoo, ISP webmails etc [13]. To provide a check of a virus in a website, it will be better to use the computer’s own antivirus scanner and scan the temporary internet files which are automatically stored on the computer’s hard drive. Relevant to the above is the element, often overlooked, of web site expiration which applies to transactional websites. In order to prevent a third party from tracing the browser’s history or back button to retrieve sensitive personal and/or banking information just entered by an online user, many websites will “expire” thus preventing spoofing [14]. This question should be placed in a category with optional components, which if incorporated, will yield a large variety of possible outcomes that will make the use of the WSEI somewhat impractical. Spyware was, also, removed from WSEI as the antivirus/antispyware scanner installed on the personal computer performing the evaluation will warn the user for any potential dangers and the browser will alert if any certificate is suspicious or expired making the implementation of such a feature in the web site obsolete and possibly redundant [15].

languages too. Aspects relating to globalization should be solely related to L3 websites since L1 and L2 websites do not deal with eCommerce transactions. Next, questions relating to “Level of Customization the Web site achieves” are also removed from the updated WSEI for the same reason as above; websites are already separated before the evaluation. Since taxation issues deal with importing/exporting goods from/to other countries and having a list of countries that shipping restrictions apply seem to be interconnected, it is reasonable to combine them with the WSEI question “applicable restrictions and/or taxation issues listed for a commodity to be exported/imported to/from certain countries”. The question above belongs to L3 websites only. Finally, all the “shipping/billing” questions are valid for L3 websites as well for obvious reasons (only transactional) [10]. A very important point is to address accessibility/ availability and hardware/ Software Requirements. “The introduction of new types of electronic devices such as personal digital assistants (PDAs), new generation mobile phones with embedded internet capabilities etc, in addition to the personal computers and laptops (notebooks) all having different abilities of presenting content to their user has lead the companies to find ways to make their web sites available to different platforms and operating systems. The advent of mobile commerce in several digital economies worldwide has stressed this need even further” [11]. Another element closely related to the hardware requirements is the time required to load any web site. This is a rather complex hardware issue affected by many variables such as hardware speed, internet connection speed, website hosting server speed, current network utilization, current worldwide network utilization, website server load, etc, and was, thus, removed. Finally, the option to download 3rd party components should also be removed since the actual (modern) browsers will automatically detect if a plug-in is missing and will offer the option to download from the respective company. Last but of the most important, security, privacy, legalities and ethics are probably the most discussed technologies issues, nowadays. Several studies were conducted to clarify how they affect large corporations’ successful or failed strategies to attract digital consumers. Currently, the most effective way to tackle this problem is to apply available mechanisms, in the form of software packages, aiming to protecting and securing valuable and sensitive data and restricting access to vulnerable systems [12].

ISSN: 1790-5117

4 Methodology The first decision, towards designing the metric evaluation system, was to completely remove the case of companies not having online presence for which the use of the WSEI would not be applicable. Likewise it is not feasible to successfully evaluate a website that belongs to such a fifth stage as mentioned in the background section. Areas outside the scope of designing and developing a web site such as the data exchange between other third party websites, total company control and transactions through online systems - basically a website completely integrated with every business activity of the company – would not be possible to evaluate. The second decision before forming the actual measurement/metric system, was to determine the questions’ weight factor; how important is each issue being addressed. Since no previous professional or academic work was found during the literature review on the matter, decisions were based on

196

ISBN: 978-960-474-084-0


Finally, the last column shows the question’s value (strength).

personal professional and academic experience. It was decided to follow a numeric style of assigning questions’ value which would start from 1 and double the value of every next question, i.e. q11, q2 2, q3 4, q4 8, q5 16, ..., q8 128, q10 256. The rationale behind this scheme was to assign a unique value to each question that would not be “lost” in the total value but would distinctly appear in that value. Furthermore, an important feature/question would receive more points than all those less important put together suggesting, in effect, a clear priority of elements to be implemented. The decision of what would be the question with higher value was based on how important is the question for the overall quality of the web site and how expensive is it in terms of time and work effort for an expert to implement the respective feature. Then, to keep the process in reasonable and time limits and help the users of the methodology become comfortable with it, so as to make the procedure as practical as possible, the authors decided to limit the maximum attainable points to 256, i.e. 9 questions. This was achieved by combining WSEI questions into fewer ones but at the same time retaining the meaning and purpose. Dimensions such as Security and Privacy received on the other hand more questions to balance with the remaining dimensions. The resulting WSEI methodology serves quite well the novice and the expert evaluators of web sites. The former may understand what should definitely be implemented and, of course, avoided just by taking a quick look and the methodology’s template and the points attained (or should be). The expert could, by just getting the scores of the web site, understand the strengths and weaknesses and, thus, advice the web developers accordingly.

B.

Evaluation Scheme‐Example To further explain how the WSEI (see table 1 in appendix) is to be used to analyze websites, the following example is provided. Suppose a web site with the score 45(255).56(114).99(371).64(4).L1 Given that this is the score of a Level 1 (L1) web site then it can be easily translated. The first part, i.e. 45(255) represents the assessment of the quality of Dimension 1 of the WSEI. The number inside the parenthesis indicates the maximum points necessary for this Level of web site concerning this dimension. The score outside is that received after the assessment of the site. This score can only be the sum of the values 32, 8, 4 and 1 from the WSEI; there is no other way. This means only elements 1, 3i, 3ii and 4ii of Dimension 1 where successfully implemented whereas the rest for that dimension either were not properly or not implemented at all. The second part, likewise, suggests a maximum necessary score of 114 of which 56 points were given. Again, these points could be given only by addressing successfully the elements 1ii, 1iii and 1iv. What is interesting here is that the evaluator of the site may easily and quickly find that not only other important elements were not implemented but the “internal search engine available” element which is not a requirement for web sites of this level was implemented without particular need. The third part suggests a maximum score of 371 yielded from successfully implementing the necessary elements of this level of web site. The score 99 means that elements 1iii, 2, 4ii and 4iii were implemented correctly. The fourth part illustrates probably the most interesting, and somewhat extreme, case possible which, however, is addressed by this methodology. The maximum score for this dimension at this level is 4 meaning just one element is required to be implemented that of “the web site displayed properly”. However, the attained score 64 indicates that instead of this element the “cipher strength” element is implemented without a particular need.

A.

Suggested working WSEI The reader may see the WSEI at the appendix (table 1). This methodology and template was derived from the research done, the previous WSET and the author’s expertise gained in the website eCommerce field. Each dimension/structure is a section by itself in the WSEI and has a title and the maximum attainable points for the corresponding dimension. The first column indicates the question number (1, 2, 3) and whether it is divided to subcategories (i, ii, iii). The second column is dedicated to the actual evaluation question. The third column shows the applicable website levels since there are some questions that are not valid for Level 1 websites but rather for those at Level 2 or Level 3.

ISSN: 1790-5117

C.

Possible issues ‐ Limitations From a preliminary check, two possible problems can be identified. The abundance of Internet browsers (Microsoft IE, Opera, Firefox, Safari, Camino, Netscape, Google’s Chrome, Mobile IE, Mobile Opera, etc) renders the WSEI section of browser compatibility check inefficient. Therefore, it

197

ISBN: 978-960-474-084-0


[3]

W3School.com. (2009) Internet Browser Statistics www.w3schools.com/browsers/browsers_ display.asp [4] Montalbano, E., IDG News Service. (2009) Adobe's New Flash Can Stream Internet Content to TVs. tech.yahoo.com/ news/ pcworld/ 20090420/ tc_pcworld/ adobesnewflashcan streaminternet contenttotvs [5] Rodriquez-Rosello, L., Ballesteros, I. L. (2007), Networked Media of the Future, Directorate General Information Society and Media, European Commision, Brussels. [6] Website Standards Association. (2008) Business Website Usability Guidelines. (PDF) [12] ConneCT Management Advisory Committee (CMAC). (2000) Website Standards Definitions. www.state.ct.us/cmac/stds/defin.htm [8] Abid Warsi. (2008). Customizable Websites – The definite Guide www.webcredible.co.uk/userfriendly-resources/ web-usability/ customisation. shtml [9] MarketingSherpa. (2007) Website Globalization Report marketingsherpa.com/ Reports/ WebsiteGlobalization.pdf [10] eCommerceOptimization.com. (2007) eCommerce and Shopping Cart Usability: 21 Best Practices www.ecommerceoptimization.com/articles/ecom merce-shopping-cart-usability-21-best-practices/ [11] ITU – International Telecommunications Union. (2008) Mobile Internet Users. www.itu.int/ITU-D/ict/newslog/ Mobile+Internet+Users+To+Top+17+Billion+By +2013.aspx [12] Verisign. (2009) Secure Sockets Layer: How It Works. www.verisign.com/ssl/ssl-informationcenter/how-ssl-security-works/index.html [13] McAfee.com. (2008) Yahoo! and McAfee Partner To Make Searching The Web More Secure For User. www.mcafee.com/us/about/ press/corporate/2008/20080507_080000_t.html [14] Web Application Security Consortium. (2005) Insufficient Session Expiration. www.webappsec.org/projects/threat/classes/insuf ficient_session_expiration.shtml [15] Microsoft Small Business Center – Kim Komando. (2009) Is your Web site usable? www.microsoft.com/smallbusiness/resources/mar keting/online-marketing/is-your-web-siteusable.aspx#IsyourWebsiteusable

was decided to just consider the 2 most popular browsers (1 for desktops and 1 for PDAs). The second issue identified is the hyperlink check whether the link is valid or not. Surely nobody can afford to sit and manually check every hyperlink in a website if it leads somewhere or not especially when given a specific and usually limited time frame. Specialized tools are available for this kind of testing from 3rd party companies. It is yet to be decided what tool to use because of the price tag and its ability to search in different website levels.

5 Further research - Conclusion The WSEI methodology is an attempt by the authors to set the foundations for a comprehensive yet practical metric and measurement system for the assessment of websites. It is the outcome of 5 years of research and study which has proved that such a systematic methodology is practical in a field which is one of the most dynamic and rapidly changing in the computer science and business information systems. There is one key point that should be noted. The reason the authors consider this methodology, metric and measurement system is important is not because of the types of issues, features and/or elements that it incorporates at this particular time or their effect in determining whether a web site is good or not and to what extent. Indeed these elements will probably change in time. It’s significance is that it can be used as an instrument to quickly decide what should be done while developing a web site, realizing the weaknesses of an existing one and, even, roughly but reasonably accurately estimate either the cost of the ideal web site based on the designers’ decision or the best possible web site based on the investors budget. As already mentioned in the beginning of this paper, this is the first part of the study that just presented the methodology. In the second part, a working paper at the moment, this methodology is tested and the result will be presented as well as the authors’ feelings and experience of using it on a significant number of web sites from various types of companies worldwide. References: [1] Xanthidis, D., Nicholas D., Argyrides, P. (2009), Emerging Markets and E-Commerce in Developing Economies, IGI Global, pp. 293-317. [2] Theofanos, Mary Frances. (2003). Guidelines for Accessible and Usable Web Sites: Observing Users Who Work With Screen Readers, www.redish.net/content/papers/interactions.html.

ISSN: 1790-5117

198

ISBN: 978-960-474-084-0


Appendix Dimension 1: Stickiness (L1, L2, L3: 255 MAX Points) Question Strength Reduced usage of scrolling mechanisms and presence of floating hyperlinks LEVEL 1,2,3 32 Hyperlink placement/style i. Hyperlinks easily accessible at a glance (Font properties such as name, size, LEVEL 1,2,3 64 bold/no bold, color of the text hyperlinks distinguishing them from the rest of the text)? iii. Icons used in graphical type hyperlinks intuitively identifiable, i.e. do they LEVEL 1,2,3 2 represent the target object or are they misleading? 3. Hyperlink target/content i. Tendency NOT to have dead hyperlinks in the site (use home page)? LEVEL 1,2,3 8 ii. Hyperlinks lead to relevant pages? LEVEL 1,2,3 1 4. Site maps i. Presence of any type of site map, i.e. site tree diagram, drop-down menus, etc.? LEVEL 1,2,3 16 ii. Mapping mechanisms informative as to the actual depth in which the user LEVEL 1,2,3 4 navigates? 5. Web site user interface attractiveness i. Lack of distracting and annoying elements? LEVEL 1,2,3 128 Dimension 2: Information Quality and Feedback (L1: 114 MAX Points, L2 & L3: 127 MAX Points) 1. Information quality and completeness i. Any “read more” hyperlinks available clarifying possibly broad, unclear or LEVEL 2 & 3 4 unknown topics to the reader? ii. Is the information provided in the Web site signed and, thus, credible? LEVEL 1,2,3 32 iii. Is the information provided updated on a reasonably expected timeliness? LEVEL 1,2,3 16 iv. Any internal search engine available? LEVEL 2 & 3 8 2. Visitor’s feedback enabled and online help available i. Email links available to the visitors? LEVEL 1,2,3 64 ii. Feedback forms available? LEVEL 1,2,3 2 iii. On line help available (e.g. FAQs, etc.)? LEVEL 2 , 3 1 Dimension 3: Customization and Globalization (L1, L2: 371 MAX Points, L3: 511 Max Points) 1. Languages supported (cocacola.com is worldwide) i. English LEVEL 1 , 2 , 3 256 ii. Spanish, Chinese, French, German LEVEL 3 128 iii. Other (Greek, Turkish, Arabic, Hebrew, Japanese, etc.) LEVEL 1 , 2, 3 32 Colors used: Is Web site color related with the cultural background (Western, 2. LEVEL 1 , 2 , 3 64 Asian, etc) of the targeted population? 3. Issues related to globalization i. Any applicable restrictions and/or taxation issues listed for a commodity to be LEVEL 3 4 exported/imported to/from certain countries? ii. Any information provided about available shipping/ delivery options? LEVEL 3 8 4. Payment – shipping/billing options i. List and Description of different payment options available? LEVEL 1 , 2 , 3 16 ii. Currency converter available? LEVEL 1 , 2 , 3 1 iii. Use of the universal “postal code” instead of the regional “zip code”? LEVEL 1 , 2 , 3 2 Dimension 4: Accessibility, Security and Software Requirements (L1: 4 MAX Points, L2: 15 MAX Points, L3: 127 MAX Points) Is the Web site accessible (Platform Compatibility) – PDA / PC 1. LEVEL 2,3 1 Is the Web site optimized for users with a mental or physical handicap? 2. LEVEL 2,3 2 Web site displayed properly, i.e. no horizontal scrolling mechanisms, no twisting of 3. LEVEL 1,2,3 4 objects, etc., in different display resolutions? 4. Security / Privacy i. If transactional/interactive, is a secure protocol used? LEVEL 3 32 ii. Cipher strength higher than 64bits (128 is Industry Standard) LEVEL 3 64 5. Privacy i. Privacy statement? LEVEL 2 , 3 8 ii. Masked e-mail addresses through scripts, forms, buttons, etc? LEVEL 3 16 1. 2.

Table 1: Web Site Evaluation Index

ISSN: 1790-5117

199

ISBN: 978-960-474-084-0