W3C Mobile Web Best Practices Evaluation of an Educational Website .... server. Most of the information in the mobile website is dynamically generated, to keep.
W3C Mobile Web Best Practices Evaluation of an Educational Website Prof. F. Javier Diaz, B.S. Ivana Harari, B.S. Paola Amadeo Faculty of Computer Science, National University of La Plata. La Plata, 1900, ARGENTINA [javierd, iharari, pamadeo] @info.unlp.edu.ar Abstract. This article presents the experience of a heuristic evaluation on the mobile website of the National University of La Plata, UNLP, based on the recommendations of the World Wide Web Consortium, W3C, for this type of interface. It also includes a brief reference to the methodology used during the development process, in which prototyping and evaluations had a leading role, taking into account the specific characteristics for being both an educational and a mobile website. The evaluation took place through two mechanisms, a manual or human revision and an automatized validation. This concluded with endless problems and errors discovered through both mechanisms. They are being analyzed and solved in order to achieve the quality of use required in the final version of the mobile website. Keywords. Mobile user interface, heuristic evaluation, mobile usability, rapid prototyping.
1. Introduction Mobile technologies have become very popular in Argentina, especially regarding the use of cell phones. It is considered that in our country, the number of active mobile phones in April of the past year had reached 37 million, 35.5% more than the same month of 2005. [5] This ongoing tendency in the youth population and even in low-income citizens makes it necessary to have educational websites which allow access from wireless devices. Such is the case of Indiana University [12], the University of Baltimore [13] and Cornell University [1], all located in the United States. In Europe, The Wireless University Forum [15] encompasses all the European universities
interested in the development of software and wireless technology. For this reason we began working in a project to provide mobile access to the website of the National University of La Plata [14], by means of the realization of an adequate user interface for mobile phones. The website of the UNLP is second in position in the national ranking of universities on the Internet, after the website of the University of Buenos Aires. It is also 22nd in the Latin American ranking, led by the National University of San Pablo [16]. The National University of La Plata has over 90 thousand students, 10 thousand teachers and over 40 thousand graduates. It is constituted by a university system comprising 17 faculties, 5 preuniversity schools and 9 museums, with over 100 degree courses and over 100 postgraduate courses. All are dependencies which are scattered geographically in different sectors of the city [17]. According to the Google search engine, there are around 292.000 web pages under the domain unlp.edu.ar. These data illustrate the immensurable amount of information the website handles, as well as its level of heterogeneity and dispersion. They also indicate the many different user profiles which access the site, with different queries, interests and tasks in mind. The completion of the mobile website of the UNLP will allow our university community, administration staff, students, teachers, nonteaching staff, potential students and other visitors to access essential information regarding the University, using their mobile phones with access to the Internet. All this complexity implies a great design challenge, as the version of the website must be more reduced, as required in mobile interfaces, with a more restricted content, more succinct information, with a simpler browsing, due to the great limitations of the screen. This article will explain the general characteristic of the mobile website, its
development process, its design stage and aspects of its implementation, and will analyze the heuristic evaluation on its advanced prototypic versions in depth. During this evaluation, the Mobile Web Best Practices recommended by the W3C were considered and analyzed by means of two mechanisms, a manual and exhaustive revision and an automatized revision with a validator. Finally, the article will explain the results obtained in the evaluation which will reopen and improve the design of the mobile website. All this framed within a re-engineering and prototyping cycle essential to the convergence towards the final version of the product with guaranteed use quality.
2. Characteristics of the development process of the mobile website In early 2007, once the project was passed by the University, we planned a schedule of activities corresponding to a re-engineering process to suit the mobile version of the website.
The planned stages were: Phase Length State Pre-design 3 months Finished Design 2 months Finished Development of 3 months Finished the Prototype Usability Tests 1 month Finished Heuristic 1 month Finished Evaluation Re-design 1 month Started Production 1 month Table 2. Schedule of activities
-The pre-design stage: The portal was analyzed from different perspectives -most significant content, most visited pages, periods of most visits, most requested information, most popular searches, among others. We also focused on the development of mobile interfaces for educational websites, investigating mobile-specific design, examples on the market, trends in the most common information and contents visualized in the mobile interface. During this phase, we analyzed the diversification of mobile phones on the market and their incidence in the design of the mobile interface. We also studied development alternatives for the website, as well as appropriate mechanisms to automatically support both its fixed and mobile Internet form.
-The design phase: Considering all the analyzed in the previous phase, we worked on the content of the website, selecting, filtering and dismissing information. The content and the browsing structure were reorganized, focusing on interaction objects appropriate for mobile phones. The prototyping technique was used so as to capture the design and test its interaction with users. -Development of the prototype: At this stage we developed a high level prototype in order to perform an evaluation on end users and before HCI experts. It is an evolutionary prototype, which means any necessary modifications will be introduced to the prototype itself, developing into the final product. The mobile interface can be represented faithfully through the prototyping versions, allowing its modeling, testing and redesign.
-Usability test: We organized a usability test for the mobile interface on 30 individuals with different backgrounds in the use of cell phones and the website of the university in its fixed form [3]. The participants had to perform 5 tasks of different degrees of complexity. Performance aspects, such as error rates, completion times and difficulty levels were recorded and analyzed by means of direct observation, and subjective aspects such as opinions, criticism and suggestions were gathered by means of interviews and polls. -Development of the heuristic evaluation: The designers of the website verified the compliance of the prototype to W3C norms for mobile websites [18]. Each one of the suggestions was considered for correcting the prototype in any necessary aspect. This stage is going to be described throughout the article. -Re-design: In this phase the prototype will be improved, solving the problems detected during the usability test and related to the heuristic evaluation. Hence, the mobile website will be launched with all assessments and usability tests completed. -Production stage: It is estimated that by the end of the current year the website will be accessible via mobile phone to the general public.
3. Design aspects
The design of the mobile website consisted of a series activities contemplated in the design recommendations for this type of interface [8][7][11]. This article will present a summary of these activities, together with the most important screen captions, describing the navigation system and the most relevant visualization aspects. The design process in its entirety can be found in the reference [2]. This article presents the design considerations applied based on an exhaustive study of the traditional web site, use statistics, most relevant sectors and site user profiles. Applying the said design considerations regarding the content (filtering, dismissal and reorganization), browsing (by content, forward and backward, relocation of hyperlinks at the end of each page), mechanisms of interaction (minimizing the entrance of data, providing shortcuts, searches, hierarchical menus, radio buttons, lists), vertical scrolling and queries, the resulting main page is as follows:
the user can press the number; select it with the navigation buttons of the cell phone or with a stylus, depending on the device. The heading of each page includes contextual information about the current section, the hyperlinks visited to get there and an icon to return to the main page.
Figure 5. Contextual menu
The original design considered the inclusion of an iconic bar for backward and forward browsing at the end of each page. However, in most mobile phones it did not display correctly. Also in the heading, the logo of the UNLP is always visible in dark green. This logo will be modified to show only the initials, given the fact that in such reduced space the complete words do not display correctly. The original design included a contextual menu which would display in the lower left corner of the screen, with many options. However, WML does not support Javascript. We are currently analyzing alternatives to implement it. The News section, one of the most visited of the traditional web site, was redesigned in order to show the information in a brief, concise and consistent way. A hyperlink, leading to a page with the news headings and their date, was included in the main page. Clicking on one of the news headings will display a brief comment on it. The display is as follows:
Figure 4. Prototype
The hyperlinks located in section A of the traditional web site were located as hyperlinks in the main page, together with the most popular hyperlinks of the other sections, as indicated by the statistics of visits to the web site. With the aim of providing shortcuts to facilitate browsing, each option was assigned a number. In this way, in order to access a section,
Figure 6. News headings and details
During the development stage we used the simulator OpenWave SDK 6.2.2 [10]. Physical cell phones were used for the usability test.
4. Implementation aspects The site of the UNLP is developed in PHP & Perl on the side of the server, and on the side of the client uses Javascript, Flash and Applets written in Java. The database engine is PostgreSQL. Regarding the mobile site, the prototype was programmed in WML, PHP4 and the editor of choice was VIM. The site itself is over a thousand lines of code and is located in a computer with Linux, is running an Apache server. Most of the information in the mobile website is dynamically generated, to keep consistency and facilitate maintenance. Cascading Style Sheets were used but, based on the different usability tests performed on different cell phone models, during which sometimes it was impossible to browse the site, their convenience will be further analyzed.
5. The heuristic evaluation The evaluation has a very important role in the interface engineering, in all stages of its life cycle. [6] In the specific case of a mobile website, the complexity is even bigger because of the limitations of the mobile devices and their heterogeneity, rendering the evaluation with both end users and experts in HCI crucial for achieving use quality in the product. For the mobile website we have considered two main types of methods of evaluation, which have emerged in the HCI area – the methods of investigation and inspection. They differ in whether the end users participate in the evaluation or not. As an investigation method, we developed usability tests in which the final product was tested on over thirty users with different profiles, which performed the evaluation process. [4] As an inspection method, we performed a heuristic evaluation on the mobile website, which is the object of study of this article. Heuristic evaluations have proven to be an excellent technique in the detection of interaction problems. [9] In order to organize this evaluation we had to have a sufficiently advanced prototype of the mobile website, thereby the evaluation was performed late in the year. Two people who work in the area of HCI participated in this heuristic evaluation, testing
the website independently and based on the Mobile Web Best Practices, recommended by the W3C. The evaluation process consisted mainly of looking for the parts of the mobile interface which did not follow some of the recommendations proposed, taking into account different contexts and situation. The validator of the W3C was used as well to obtain an automatized checking apart from the manual revision. The process ended with a report which specified all the sectors of the interface which presented flaws on a usability level, detailing the norms infringed and the possible solutions or depuration alternatives to be considered in the final version of the product.
5.1. The recommendations considered in the evaluation The norms considered in the evaluation were those recommended by the W3C, called Mobile Web Best Practices version 1.0. Basically, the phase consists of analyzing the product taking into account the following issues: Navigability: analyzes the URI, the navigation mechanisms, the links to external resources which should be minimized and access keys taking into account restrictions on the use of image maps and pop-ups. User control: verifies that the pages do not reload or redirect automatically. Also, keystrokes should be reduced to the minimum amount possible. Content: verifies that the language used is clear and simple, that the content is appropriate for its use in a mobile context and that it is consistent when accessed from different devices. Also, that the content is organized hierarchically, i.e. that the priority information is displayed before and first. Images: verifies avoiding the use of images to position elements or create spaces, restricting large images or high resolution details and checks that they are legible and do not hide textual information. Color: analyzes that the information transmitted through color is still available without it. It also includes the verification of the adequacy of the contrasts. Design: this point is related to the layout of the pages (their measures, font, capacities) so they facilitate their reading, loading and use. It
suggests performing tests both on real devices and emulators. Technologies and markup: Here the W3C suggests creating documents which are valid under the standards and technologies of the W3C. This stage must evaluate the structure, tables, frames, objects, scripts, applets and plugins. It should also consider error messages. Style sheets: the W3C recommends the use of simple style sheets to control the presentation and prove the legibility of the document in case they are disabled. Forms: in this stage the elements which must be analyzed are specific to forms, such as input, default values, tabulation order, controls position and tagging to facilitate the access to the form from any mobile device.
User control
Content
Images Color Design
5.2. The validator used The previous section detailed the recommendations of the W3C for mobile user interfaces. As can be seen, many of them require manual checking, i.e. they demand the direct intervention of a human evaluator. Despite the W3C offers a product which validates best practices, it performs an important validation which is incomplete in the current valid version. The validator used is the W3C Mobile Web Best Practices checker (Beta)
6. The results of the evaluation The evaluation of the portal employed two mechanisms, human revision and automatized validation. The results of both processes will be further addressed in the following sections.
6.1. Results of the human revision or manual evaluation of the portal At this point, two evaluators analyzed exhaustively the prototype of the mobile website, considering each recommendation of the W3C and revising al the pages of the site to check compliance. The results thrown by this analysis were as follows: Principle Navigabilit y
Errors detected Links to unnecessary external resources were found There are pop-ups when a phone
Technologi es and markup
Style sheets Forms
number is selected by mistake and it begins to dial. Extensive keystrokes required to achieve the option of going back from the end. Lack of direct access to the items of the menus. There is extensive and unnecessary content when accessing the courses of a faculty. The content is not displayed correctly in some cellular devices. The image of the University and the arrows are not clear. --There are pages which are too long such as the list of courses, information about them and news. There were tests with Openwave emulator, Nokia and Motorola physical devices. Tries to access and download documents in PDF format, unrecognizable by the cellphone. Presents scripts. Inadequate use of tables to define structure. The style sheets present problems from different cellular devices. ----
6.2. Results of the automatized evaluation The validator found 5 major errors, shown below.
The validator passed the following points:
It also raised 3 warnings:
The validator did not apply a verification on forms for not being used in the mobile website.
7. Conclusions The development of the mobile version of an educational website as is the case of the UNLP, required the completion of the necessary methodological stages to guarantee its correct structure & functionality. Taking into account user-centered design principles and by means of prototyping, the evaluations with and without the intervention of users helped us prevent error & enhance usability of the site.
8. References [1] Cornell University Unplugged. http://enterprise.usa.siemens.com/products/ [2] Diaz, Harari, Amadeo (2006) Design Considerations for an Educational Website. Argentinean Congress of Computer Science. http://www.linti.unlp.edu.ar/tikiread_article122f.html?articleId=339 [3] Diaz, Harari, Amadeo (2007) Evaluating the Usability of the Mobile Interface of an
Educational Website. International Joint Conferences on Computer, Information, and Systems Sciences, and Engineering. (CIS2E 07) [4] Giacoppo, Sasha. (2001) Development Methods: User Needs Assessment & Task Analyses. University of Maryland. http://www.otal.umd.edu/hcirm/dvlpmeth.html [5] Instituto Nacional de Estadísticas y Censos. Estadísticas de Servicios Públicos. August 2007. http://www.indec.gov.ar/ [6] Jakob Nielsen Heuristic Evaluation http://www.useit.com/papers/heuristic/ [7] Lasen, Hamill. Mobile world : Past, present and future. Ed. Springer (2005) ISBN 9781-85233-825-1 [8] Love, S. Understanding mobile humancomputer interaction. Ed. Elsevier Butterworth-Heinmann (2005) ISBN 07506-6352-9 [9] Nielsen, Mack (1994) Usability Inspection Methods. Ed. John Wiley & Sons. ISBN 0471-01877-5 [10] Openwave SDK http://www.openwave.com [11] Seven usability guidelines for websites on mobile devices (2007) http://www.webcredible.co.uk/user-friendlyresources/web-usability/mobileguidelines.shtml [12] Site of Indiana University. www.indiana.edu. [13] Site of the University of Baltimore. www.ubalt.edu. solutions/hipathwireless/mainColumn Paragraphs/03/document/Cornell.pdf [14] Site of the National University of La Plata. http:/ /www.unlp.edu.ar. [15] The Wireless University Forum. www.twuf.com [16] The website of the UNLP is amongst the best of the country. http://www.universia. com.ar/portada/actualidad/noticia_actualidad .jsp?noticia=18367 [17] The book of the Hundredth Anniversary. Editorial de la Universidad Nacional de La Plata (2005) [18] W3C Mobile Web Best Practices checker (Beta). http://validator.w3.org/mobile/
