There is an increasing need for tools to support cheaper and faster usability evaluation of Web applications. In this paper, we describe an approach to remote ...
Short Talks
CHI 2001 • 31 MARCH - 5 APRIL
Remote Automatic Evaluation of Web Sites Based on Task Models and Browser Monitoring Fabio Patern6 & Lafla Paganelli CNUCE-C.N.R. Via V.Alfieri 1
56010 Ghezzano-Pisa, Italy +39 050 3153066 { fabio.paterno, laila.paganelli}@cnuce.cnr.it ABSTRACT
OUR APPROACH
There is an increasing need for tools to support cheaper and faster usability evaluation of Web applications. In this paper, we describe an approach to remote automatic evaluation of web sites. The novelty lies in automatic analysis based on a combination of web browser monitoring and task models which aims to capture and analyse users actual behaviour.
The first issue to address was what kind of logs we should consider. Many approaches to the analysis and evaluation of Web applications focus on web server logs (see for example [6]). We discarded this approach because it suffers from many limitations [1]. Firstly, servers cannot collect data on some potentially crucial user interactions that occur on the client-side only (e.g., within-page anchor links, form element interactions, etc.). A second related point is that the validity of the data is highly suspect because of caching by proxy servers and browsers and dynamic IP addressing. Caching refers to the built-in functionality of browsers and proxy servers that allows frequently accessed web pages to be stored locally thus preventing servers from knowing when a cached page has been requested again. Dynamic IP addressing is the practice of assigning potentially different IP to a client each time it accesses the Internet. Thus, we have implemented a tool in Java script that is able to log all user interactions with a Web browser. These logs contain a wide variety of user actions (such as mouse clicks, text typing, link selections ...) as well as browsing behaviour such as start and finish of page loading.
Keywords
Tools for Usability Evaluation, Web Interfaces, Task Models. INTRODUCTION
The advent of the Internet has made the development of Web applications quite easy. However, obtaining effective WWW user interfaces is still a difficult task, especially when the number of pages composing the web site considered is high. ORen people with low expertise in usability are faced with the problem of improving their web sites and do not want to devote a lot of resources to this. To this end, automatic evaluation can prove to be helpful by providing useful information that allow designers to identify the problematic parts of the user interface. Remote evaluation aims to give such support when evaluators and users are widely separated in time and/or space [2]. RemUSINE [4] is a task model-based tool for supporting usability evaluation, which however suffers from the limitation of not being designed for or applicable to web applications. This approach is important because it proves that it is possible to use models to support analysis of real user behaviour whereas usually empirical testing and model-based evaluation have been considered separately. We have developed a new approach that takes into consideration previous experiences in the area of automatic usability [5, 7] evaluation and provides support for the evaluation of web applications.
During the display of graphical user interfaces there is often dynamic enabling and disabling of interaction techniques. This was used in the first version of RemUSINE to identify errors (actions not suitable for the current task) during user sessions. The basic concept was that if users perform a certain action (disabled), it means that they probably want to perform a related task. However, the error reveals that the user has not understood that something must first be done in order to correctly perform the desired task (at least according to the system task model). This is moot in web interfaces: a link is always enabled. Therefore, it is more difficult to identify the user intentions. We thus provide the possibility of representing in the user side the structure of the high level tasks that users can perform through the web application. Users are then asked to explicitly indicate what task they want to perform and, when they change tasks, they must indicate their wishes so that when the analysis is performed the tool knows how to consider the interaction log and is able to identify user errors. We develop the task model using the ConcurTaskTrees notation [3]. It gives the possibility of using different icons
© Copyright on this material is held by the Author(s).
anyone, anywhere.
283
Short Talks
to indicate how tasks are allocated. In case of task models for web applications, a link selection is an interaction task whereas presenting a web page is a system task. Tasks are hierarchically structured until we reach basic tasks, tasks associated with an action that is considered atomic. Also basic tasks can be associated with multiple web pages. For example, if we consider a system task associated with showing information concerning a work of art, this task is associated with all the pages associated with a work of art (in a museum web site they can be thousands of pages). This one-to-many relationship allows designers to limit the size of the task model when large web sites are considered.
CHI 2001 • 31 MARCH - 5 APRIL
pages never accessed can indicate either pages with poor content or pages difficult to be reached. It is possible to analyse the behaviour of the users during the sessions to know whether their performance improves (showing a user interface easy to learn) or not. In addition, user actions (such as scrollbar movements and windows resize) are considered by the automatic analysis to see wheth(x the dimensions of the page are optimal or too much infomlation is conveyed. The tool is able to analyse not only single sessions but also groups of them thus allowing the evaluator to understand whether a problem occurs often across many users or is only a problem limited to certain users and circumstances. CONCLUSIONS
In this work we have shown how task models combined with web browsers monitoring can be used to support automatic evaluation of web sites. This can be helpful also for large web sites and it provides information more useful than that obtained by tools based on the analysis of web server logs. The approach is more effective when goaloriented web applications are considered and it shotdd be complemented with other techniques (such as remote questionnaires) to analyse in-depth some dimensions of usability such as user satisfaction or amusement. We plan to extend the tool in order to provide automatic suggestions for possible improvements of the pages found problematic. REFERENCES
Figure l: The part of the tool providing task-related results. The tool provides designers with a number of results that are related to both tasks and web pages and it also shows their relationships (for each task, the list of pages associated with them and viceversa). This allows designers to perform analysis from both point of views, for example it is possible to know how long it takes users to perform some tasks and similarly how long it takes to download some pages. Thus designers can, for example, first see whether there are tasks particularly long and if some of them exist they can identify which pages create problems and whether they occur because of an excessive downloading time. The tool also displays the sequences of tasks performed and the associated sequences of web pages accessed and" provides lists of tasks performed (and pages accessed) ordered by time duration or frequencies of performance (in ease of tasks) or number of accesses (in case of pages). Patterns of tasks, actions and pages accessed are also automatically detected and displayed. The tool provides automatic support for analysing whether the sequence of tasks performed is the optimal one for achieving the current goal and to count the number of useless actions performed and pages accessed. Other features provide the indication of tasks completed, those started and not completed, and those never tried (see Figure 1). The last information is given also for web pages:
284
1. Etgen, M., Cantor, J., What does getting WET (Web Event-logging Tool) Mean for Web Usability? Proceedings HFWeb'99. http://zing.ncsl.nist.gov/ hfweb/proceedings/etgen-cantor/index.html 2. Hartson, R., Castillo, J., Kelso, J., Kamler, J., Neale, W., The Network as an extension of the Usability Laboratory, Proceedings of CHI'96, 1996, pp.228-235. 3. Patem6, F., Model-Based Design and Evaluation of Interactive Application. Springer Verlag, ISBN 185233-155-0, 1999. 4. Patem6, F., Ballardin, G., Model-Aided Remote Usability Evaluation, Proceedings INTERAC, T'99, pp.434-442, lOS Press, ISBN 0 9673355 0 7, Edinburgh, September'99. 5. Scapin, D., Vanderdonckt, J., Mariage, C., Bastie~, C., Farene, C., Palanque, P., Bastide, R., A Framework for Organizing Web Usability Guidelines, Proceedings HFWeb'00, http://www.tri.sbe.com/hfweb/ 6. Winckler M., Freitas, C., Lima, J., Usability Remote Evaluation for WWW, Extended Abstracts CHF2000, pp.131-132. 7. Yvory, M., Hearst, M. (1999), State of the Art in
Automated Usability Evaluation of User Interfaces. 2000. Report available at http://www.cs.berkeley.edu/ -ivory/research/web/papers/survey/survey.htrnl
CH12001
