1
Usability of the Academic Websites of Jordan's Universities An Evaluation Study
Suleiman H. Mustafa
[email protected]
Loai F. Al-Zoua’bi
[email protected]
Faculty of Information Technology, Yarmouk University, Irbid, JORDAN
Abstract The aim of this research study was to evaluate the websites of Jordan's universities from the usability perspective. Two online automated tools, namely: html toolbox and web page analyze were used along with a questionnaire directed towards users of these websites. Tools were used to measure the websites internal attributes which can not be perceived by users, such as html code error, download time, and size of html page. The questionnaire was developed and designed based on 23 usability criteria divided into 5 categories. Each category deals with one usability aspect. The results showed that the overall usability level of the studied Websites is acceptable. However, there are some weaknesses in some aspects of the design, interface, and performances. Suggestions are provided in the study to enhance the usability of these websites. Keywords Website Usability, Website Evaluation, University Websites. Jordanian Universities.
1. INTRODUCTION The Web is playing a main role in diverse application domains such as business, education, industry and entertainment. As a result, there are increasing concerns about the ways in which websites are developed and the degree of quality delivered. Developing a website should be passed through several design guidelines to ensure that the website can achieve the purposes and goals intended to be accomplished. Additionally, an organization’s website is a gateway to its information, products and services. As such, it should ideally be a reflection of the needs of the clients it serves. Unfortunately, website design is often driven by technology, organizational structure or business objectives, rather than by users needs.
However, in recent years website owners and developers have gradually begun to acknowledge and address the issue of usability [1]. Usability is considered as one of the major factors that determine the successfulness of a website. As defined by ISO 9241-11, it is “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” [2]. Several evaluation methods have been proposed to assess the usability of websites in order to suggest enhancements to the design of websites. Some methods are address experts, while others are directed toward users. Such evaluation methods include • Heuristics used by a group of experts with reference to established guidelines or design principles [3, 4, 5, 6, 7], • Prototyping based on developing a mock site that can be shown to users before the real site is launched. Users can also participate in focus user groups or controlled laboratory sessions to provide usability feedback [7], • Checklists of usability items [8], • Cognitive walkthroughs involving one or a group of evaluators inspecting a user interface by going through a set of tasks to evaluate website understandability and ease of learning [9, 10], • Questionnaires for extracting, recording, and collecting information to measure the user satisfaction with website usability [11, 12]. The usability of any website can be evaluated and determined using usability evaluation methods and techniques.
2 Generally, any website should meet the needs of its various stakeholders [13]. Educational Website users are mainly concerned with the following two major questions: 1. Can I find the information I am looking for in my website easily? 2. Can I find the information in timely manner? The Websites of Jordan's universities aim at providing up-to-date information and services to students, instructors, academic administrators and other users in efficient ways. There are, at present, twenty three universities in Jordan, each having its own website. Generally, usability studies of Arabic Websites and in particular academic Websites are almost nonexistent. The research reported in this paper was intended to point out the strengths and weaknesses of the usability aspects in the design of these academic websites, including interface, performance and effectiveness, content and organization, and the educational purposes for these websites. The results are supposed to draw the attention of webmasters to overcome the limitations of these websites and improve their efficiency.
2. RELATED WORK Evaluations of the usability of websites have been conducted over the years and for many domains. For example, Akoglu [14] developed a special tool for assessing the usability of architectural department websites in the University of Istanbul. The tool consisted of two parts; the first part can be accessed by users where they visit and answer the questions about the website, and the second part was constructed for the use of administrators where he/she can manage the content of the usability evaluation. The evaluation was based on two environments: traditional laboratory environment and Internet environment. Kirakowski et al. [15,16] evaluated the user satisfaction with usability of 5 websites based on a questionnaire method. The authors developed a new questionnaire ( named WAMMI) for the evaluation. For more information about WAMMI the reader is referred to (http://www.ucc.ie/hfrg/questionnaires/wammi). The questionnaire showed that the evaluation of user satisfaction contributes to the successful development of websites. The research by Chiew and Salim [17] focused on developing a web-based tool (called WEBUSE) which consists of 24-questions for evaluating the usability of
websites. The report generated by the tool indicates how good is the website with respect to usability. The researchers claim that WEBUSE is suitable for the evaluation of all types of websites and for any domain. The tool can assist webmasters to improve their websites based on the response provided by the visitors of the intended websites. In another research conducted by Harms and Schweibenz [18], the researchers used a combination of two methods of usability engineering for the web, namely the heuristics method and the laboratory-test method with actual users involved in evaluating the usability of the Saarland Museum's Website. The authors claim that this combination of methods provides good results, but it is costly in terms of time and resources. They also pointed out that one of the major drawbacks of the heuristics is that it cannot be applied successfully by novice evaluators since they need some background knowledge in Web design and evaluation. The results obtained from the evaluation led to the redesign of the Saarland Museum’s Website. With regard to academic websites, the research conducted by a group from the University of Maryland [3] performed a usability evaluation for the Master of Information Management (MIM) (part of the College of Information Studies at the University of Maryland, College Park) program website to know whether their website meets the needs of its users. The methods used focused on conducting user testing tasks and performed heuristics evaluation for these tasks. A set of recommendations were provided to guide future redesign of the website. The researchers proposed a new heuristic evaluation technique based on modifications of two widely used heuristics evaluation techniques [4] for the evaluation process. As a result, the researchers suggested that the MIM website does not meet the needs of many of its users. Although the website does a good job of presenting information to prospective students, it does not fulfill the needs of current students and faculty. A similar case study was conducted by the Website Usability Testing Center at Wisconsin-Stout University [19] to evaluate the usability of their University’s website www.uwstout.edu. The researchers used qualitative testing criteria such as navigation times through subject evaluations to assess the usability of the site. The study showed that the website suffers from several problems including website specific jargon, unorganized link patterns, confusing search engines, and poorly
3 emphasized information. Based on the results of the evaluation the authors proposed a number of recommendations in order to improve and unify the university website. A study by Gullikson [20] looked at assessing how the information architecture affected user navigation of an academic institution’s Website. Participants answered a series of six questions using the site under a 3-minute time constraint for each question. User interaction with the site was recorded, and the subjects were asked to elaborate on the decisions they made during a playback of the testing session. The study tracked the number of answers found correctly, the pages viewed in the process, and the time between mouse clicks. Corry [21] conducted a usability evaluation of an existing Midwestern University Website. An analysis was conducted to restructure the information contained in the current Website; a prototype was developed and tested against the existing site. Usability was based on the ability of subjects (such as students, parents, and faculty) to quickly and accurately locates answers to a set of questions. While the study worked well, the metrics used to measure usability were limited to task completion time and the number of user errors.
3. QUESTIONNAIRE-BASED EVALUATION METHOD In this study, two website evaluation methods were used: the questionnaire and automated tools In order to select the appropriate website evaluation usability criteria for the questionnaire-based evaluation method, we identified and analyzed the usability aspects in terms of user satisfaction, readability, navigation and others related to the websites of Jordanian universities. Two forms were used to conduct our questionnaire: a paper form and an online form. The questionnaire was divided into two sections. The first section addressed the characteristics of participants, including: university name, internet and computer usage, age of participant, and the access frequency of participant university website. The second section included thirty questions that were used to evaluate the usability of Jordanian academic websites. The questions were classified into five categories (See Appendix 1): Content, organization, and readability (CAT 1), Navigation and links (CAT 2), User interface design (CAT 3), Performance and effectiveness (CAT 4), and Educational information (CAT 5).
After an extensive study of related resources [16, 21, 22, 23, 24, 25], twenty three website usability evaluation criteria were identified as shown in Table 1. Each criterion was assessed, in terms of the five categories given above (as applicable), on a scale of five items as shown in Table 2. The responses were evaluated according to the merit points adopted by Chiew and Salim [17]. Table 1: The 23-Usability Evaluation Criteria No
Usability Criteria
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
Display space Scroll left and right Accessibility Distracting or irritating elements Orphan page Placement and content of site map Information search Link colors Up-to-date information Download time Back button Open new browser windows Respond according to users’ expectations Web advertising Follow real world conventions Hyperlink description Consistent design Use of color Organization of information Navigational aids Registration information Faculties information Instructors information
Table 2: Options for question and corresponding merits Option Merit Strongly Disagree (S.D)
0.00
Disagree (Dis)
0.25
Fair
0.50
Agree (Ag)
0.75
Strongly Agree (S.A.)
1.00
The response to each question is assigned a merit value. Merit points are then accumulated according to the five usability categories shown in Table 3. Mean value for
4 each category is considered as the usability index for that category. The usability index for a category, x, is defined as: x = [∑(Merit for each question of the category)] / [number of questions] Table 3: Usability Categories and their Points Category
Point
Usability Level
Content, Organization and Readability (CAT1) Navigation and Links (CAT2) User Interface Design (CAT3) Performance & Effectiveness (CAT4) Educational Information (CAT5)
External attributes depend on the website and its usage, while the internal attributes of the website only depend on how the website has been designed and developed [27]. These internal attributes of the website can be assessed and evaluated using automated tools. In this part of the evaluation, two automated online tools were used: Web Page Analyzer [28] and HTML Toolbox [29]). The internal attributes that can be measured include: • Total number of HTML files on the page. • The total number of objects on the page. • The total number of images. • Total size of images. • Browsers compatibility. • Total number of broken and bad links.
Overall Usability The usability level of each category is determined by the usability points. Table 4 shows the usability points and the corresponding usability level. The overall website usability index is the mean value of usability points for the five categories. Table 4: Usability Points and the Corresponding Usability Levels Points (x) Usability Level 0
