case study: mobile commerce application usability

3 downloads 546 Views 160KB Size Report
usability of mobile application designed for sales. Application functionality and user interface quality of design has been tested using methods of cognitive walk-.
CASE STUDY: MOBILE COMMERCE APPLICATION USABILITY 1

Miroslav Minović1, Velimir Štavljanin1 Faculty of Organizational Sciences, Belgrade University

Abstract: This paper presents research results on usability of mobile application designed for sales. Application functionality and user interface quality of design has been tested using methods of cognitive walktroughs and focus group with end users in lab environment. Further research in this areas are desireable in order to improve usability of mobile commerce applications. Key words: MOBILITY, MCOMMERCE, PDA, USER INTERFACE, USABILITY, FOCUS GROUPS. 1.

INTRODUCTION

Complex application development requires different problems solution related to reliability, security, speed, system maintenance and other user requirements. Recent years significant attention is dedicated to user interface. Term user interface has been always implied only as nice designed layout of menus, buttons and other elements, but functionality, ease of performing individual tasks, speed of learning or everything pooled as usability are always been forgotten. When we talk about overall system acceptability, it is combination of system social acceptability and system practical acceptability [5]. Practical acceptability has many aspects, and we can analyze it within various categories such as cost, support, reliability, etc., as well as the category of usefulness. Usefulness can be further broken down into two categories of utility and usability. Usability gives answer to question how well users can use functionality of the system [9]. Usability applies to all aspects of the system with which a human might interact.

with end users in final phase of application development after testing and before final release. 2. USABILITY EVALUATION APPLICATION FOR SALES

OF

MOBILE

2.1. System prototype Application we tested was designed for salesmen and was realized as PDA application under the Palm operating system. Figure 1. shows forms of the application with the arrows showing directions of using user interface. After starting application system shows logging form (1) to the user. Then user enters username and password, and selects warehouse from the list. Finally he clicks the Login button. After successful logging system shows form in which user selects place for which he wants to see customers (2). Place selects by click on the value in the list. After selecting the right place, system shows customers for that place in the list (3). User can scroll and lookup customers by input only part or whole customer name. After selection of the customer, system shows creating new invoice form (4). Invoice header is automatically filled, and user selects payment type and priority. Then user clicks Create invoice button, and then Create Invoice Item button. After New Invoice Item forms opens (6), user selects product and quantity. Product selection goes through list box and quantity is entered by hand. After click Save button, opens Selected items form (8). User can click on New item button or Exit button. Prototype we will test is without forms (5) and (7), because it is interface after improvement.

In this paper we will present results of the usability research for PDA (Personal Digital Assistant) application. This application is part of the company information system, and is designed for sales force. Application is designed to improve salesman work in different aspects. Salesman must have all information about products and stock quantities up to date. By using printed lists as a source of information, quantities were up to date only few moments after printing. To check state of the stock for a particular product, salesman calls company warehouse to update lists. Ordering of products was by telephone, too. This process is time consuming, subjected to errors, and stressful for managers. Using mobile technology some previously stated problems could be eliminated. Usability is primary connected with quality of application design and includes: design of user interface, development methodology, testing methodology and installation procedure. In the first phase we used cognitive walkthroughs method without end users, for early phase of application development (application concept). In second phase we used focus group technique

Figure 1. User interface of application prototype for selling

1

2.2. Task description a. logging on the system b. customer selection (by location) c. invoice creation d. invoice item creation (by selecting products)

Action C: input of the specific amount of the product System answer C: showing of the product amount Action D: click on the button Save item System answer D: system writes message about saving successfulness and redirects user to Selected items form

2.3. List of actions by task a. Logging on the system 1. username input 2. password input 3. pop-up trigger click 4. selection of the value from the list 5. login button click b. Customer selection 1. selection of list box with customers location 2. partial customer name input 3. click on the value from the list c. Invoice creation 1. pop-up trigger click 2. checkbox click 3. create new invoice button click 4. create new invoice item button click d. Invoice item creation 1. selection of the product from the list 2. add product button click 3. add new item button click 4. delete item button click 5. item overview button click 2.4. Categories of users Users are experts in the field of selling, but they are not familiar with using PDA. 2.5. Cognitive walkthroughs method In the next part we will present questionnaire created by the cognitive walkthroughs method, which we used for usability evaluation of the user interface for creating invoice item task. This is only one example we used in usability evaluation of mobile application.

Figure 2. Creating new invoice item form – before testing Action A: Create Invoice Item button click System answer A: New invoice item form is shown Action B: click on a specific product from the list System answer B: selection of the product for invoice item

Action E: click button Invoice Item Creating System answer E: system opens input form for new invoice item Action F: click on the Exit button System answer F: system opens form Invoice with the items in the invoice Next questions are evaluation questions which must be answered by the evaluator, for every action. Action A: Creating Invoice Item button click Question 1: Will users be able to produce whatever effect action has? Answer 1: Yes, it's clear from the New Invoice Form context and button label Question 2: Will users be able to notice that correct action is available? Answer 2: Probably, but it will be better to open New Invoice Item Form immediately after new invoice was created. Question 3: Will users be sure that selected action is correct, and that it is right one for effect they want to produce? Answer 3: Yes, new form will be opened, with 'New Invoice Item' label; users can be confused about which action to choose Question 4: Will users understand the effect action produced? Answer 4: Yes, confirmation dialog will be shown after new invoice item was created. Also all created items will be listed. Action B: click on a specific product from the list Question 1: Will users be able to produce whatever effect action has? Answer 1: Choosing article for new invoice is a basic task; user can be confused with too many elements on the form; main problem could be finding right article from list with thousands of products (limited display, no search and filter feature); Question 2: Will users be able to notice that correct action is available? Answer 2: Yes, with minimum knowledge of using PalmOS; Question 3: Will users be sure that selected action is correct, and that it is right one for effect they want to produce? Answer 3: Yes, selected article will be shown in different color. Question 4: Will users understand the effect action produced? Answer 4: Yes, it's obvious that article is chosen for new invoice. Action C: input of the specific amount of the product Question 1: Will users be able to produce whatever effect action has? Answer 1: If user makes wrong input, error message will be shown. Question 2: Will users be able to notice that correct action is available? Answer 2: Depending on user experience with PalmOS applications. Question 3: Will users be sure that selected action is correct, and that it is right one for effect they want to produce?

2

Answer 3: They could easily make a mistake, because they have to use Graffiti writing recognition system for input. Question 4: Will users understand the effect action produced? Answer 4: Yes. Action D: click on the button Save item Question 1: Will users be able to produce whatever effect action has? Answer 1: Yes. Question 2: Will users be able to notice that correct action is available? Answer 2: Yes, Save button is in focus on the form. Question 3: Will users be sure that selected action is correct, and that it is right one for effect they want to produce? Answer 3: Yes, save confirmation message will be shown. Question 4: Will users understand the effect action produced? Answer 4: Yes, after successful invoice item creation list with all created invoice items will be shown. Action E: click button Invoice Item Creating Question 1: Will users be able to produce whatever effect action has? Answer 1: Yes Question 2: Will users be able to notice that correct action is available? Answer 2: Yes, but user will be more efficient if New Invoice Item Form will be opened immediately after previous item was created. List with already created items could be available on demand. Question 3: Will users be sure that selected action is correct, and that it is right one for effect they want to produce? Answer 3: Yes. Question 4: Will users understand the effect action produced? Answer 4: Yes.

usability, for task invoice item creation, can be performed by following actions: Action A: Invoice Item Creation button click Problem analysis: Because invoice without items is no purpose invoice, user can be confused by button New Invoice Item; otherwise it can be tiresome. Suggested corrections: By click on the Invoice Creation button, invoice item creation form will be opened immediately, without necessity for another button click. Action B: click on a specific product from the list Problem analysis: Potential problem is large number of products presented in a small list. Because of poorly utilized PDA display and without selection and filtering capabilities only way of product access is by scrolling, and it is very complicated when several thousand of products is presented. Suggested corrections: Making of a separate form, for product picked from the list of available, with capability of exploring products by name, filtering view by categories and product sorting by some criteria. Action C: input of the specific amount of the product Problem analysis: Because of reduced text input capabilities of PDAs without keyboard, but with character recognition system Graffiti 2, which is time consuming for learning. Suggested corrections: Solution can be elimination of product quantity input as a series of characters, and we proposed system of 6 buttons, with predefined amount addition respectively 1, 2, 5, 10, 20 and 50. Action E: click button Invoice Item Creating Problem analysis: Preview of all selected items and New Invoice Item button click after every item creation can be very tiresome for end user, but is not necessary for efficient task completing. Suggested corrections: After saving new invoice item, immediately continue with next new item input, without previewing all items. Only after all items were added by the user, list with all items will be shown on the New Invoice form.

Action F: click on the Exit button Question 1: Will users be able to produce whatever effect action has? Answer 1: Yes. Question 2: Will users be able to notice that correct action is available? Answer 2: Yes, it's obvious from the context Question 3: Will users be sure that selected action is correct, and that it is right one for effect they want to produce? Answer 3: Yes, Invoice Form will be shown, with all invoice data on it. Question 4: Will users understand the effect action produced? Answer 4: Yes, Invoice Form will be shown, with all invoice data on it.

After applying suggested corrections of user interface, forms for creating new invoice items are shown on Figure 3.

2.6. Research results and suggested improvements

2.7. Focus group technique

On the basis of results obtained by cognitive walkthroughs method accomplished in laboratory environment, without participation of final users, we can conclude that potential problems of user interface

Moderator has set the following goals for the session: Ease of learning, ease of using the application, application performance, design.

Figure 3. Forms for creating new invoice items – improved after I cycle testing

3

Six people were participating in session, four of them were future end users, one developer who will be in charge for application maintenance and as a sixth participant, senior IT executive. PDA (Personal Digital Assistant) Tungsten T3 with PalmOS 5.21 with preinstalled mobile sales application was given to all participants. At the beginning of session moderator has demonstrated briefly application and it’s features (prototype description), and then asked participants to create new invoice. Time limit has been set to 30 minutes maximum. Complete session has been video recorded, while the moderator was writing down his observations. When session was completed, moderator has been asking questions related to set goals and discussion took place. Typical answers were noted. Finally, text and video material recorded during the session was analyzed. Report, with typical answers of each user

User 1

User 2

User 3

User 4

Develo per

Learning to work with the system I was not able to understand the meaning of certain buttons. I was not able to log in, and in attempt to log in with wrong password system did not provided any feedback. It is complicated to work with PDA using the pen.

Using the application

Very difficult to use, I would rather use paper and pen as by now. It takes some time to learn.

Completely unusable.

is Senior Training mandatory for IT executi new users. ve

Better navigation would be of great help. I easily get lost in the system, it’s difficult to find to desired article. I’m not always sure in which phase I am at the moment.

After a bit of practice it is very usable. Save confirmation should be added.

Effectiveness of using the application It is very difficult to find article to be added to invoice. Satisfactory, except adding article to invoice.

Required click on a confirm button (after adding each item on invoice) is slowing me down. Very slow

After a bit of practice seems all right. Some things can be speed up.

Table 1. Design

• •



used by trained professionals, learning curve is not that critical issue. Bad navigation is a limiting factor for ease-of-use. Search and choosing articles segment in application was identified as a slowest part in system. Improvements can be made with adding category filters for articles. From video session analysis can be concluded that the most critical part of user action is writing article name with stylus pen (Grafitti2). Users who were unable to write article name correctly had to scroll trough the list with thousands of articles which was very time consuming. One of the possible improvements would be a search system which does not rely on using stylus pen to write article name. Most comments made regarding design are related to lack of colors which creates monotone look of application. Adding color to user interface would improve ease-of-use.

2.9. Implemented improvements Monotone

Colors are missing.

I dislike it.

Terrible

Forms contain too many elements. Could be better.

2.8. Analysis and improvement suggestions Upon session analysis, moderator, along with the team, has concluded the following: User 4 comments can be excluded from the analysis, because from the beginning until the end of the session user had a very negative attitude and an unconstructive approach to testing; We will not discuss motives of such behavior. Moderator has tried it’s best to include user 4 to participate, however he failed. Further analysis has distinguished the following comments and suggestions as per focus group goals: • Learning to use the system requires certain effort, however since the system is meant to be

With very simple changes there has been significant improvement in application functionality. Major problem for focus group has been searching for article in the list. Implemented improvement was changed in the following way: user select first letter of article with navigation buttons. As a next step only selected range of existing letters in article name can be chosen. In every further step list of articles is narrowing. This way user can avoid using stylus pen. Navigation is improved with adding menu’s and changing button names and it’s functions so that using the application becomes more intuitive. Situation of having too many items per form was resolved with visual enhancements such as adding colors and polishing user interface. This way application has become more user friendly, intuitive and useable. 3. CONCLUSION Modeling of mobile application for PDA user interface is a great challenge, because it will be a key factor in accepting them in every day use. This will further influence to using new possibilities offered by new mobile technologies. Paper presents usability testing during two cycles. First cycle uses method of cognitive walk-troughs whereas the second one uses focus group technique. Cognitive walk-troughs method without great complexity provides very good results improving application design and usability. This way number of errors is decreased resulting in more user self confidence leading to shorter training time, comfortable work and finally, positive attitude of end users. Negative side of this method is the fact that is been applied in lab conditions by application designer who was directly involved in design process. Focus group technique includes end users, correcting the negative effect of previous method. Collected attitudes and opinions of end users represent valuable source of information about possible problems and improvements related to usability of mobile application. Given that usability is a key factor in mobile application success, further usability testing should be continued. Next

4

steps could involve field research, as well as gathering end user feedback after certain period of using the application. Analysis of these data should reveal further possible usability improvements.

4. LITERATURE [1] P. Polson, C. Lewis, J. Rieman, C. Wharton, "Cognitive walkthroughs: A method for theory-based evaluation of user interfaces", International Journal of Man-Machine Studies, 36 (1992) 741-773. [2] C. Wharton, J. Rieman, C. Lewis, P. Polson, “Cognitive walkthroughs: A practicioner’s guide.”, Usability Inspection , John Wiley, New York, 1994. [3] Lewis C., Polson P., Wharton C., Rieman J., ’’Testing a walk – throught methodology for theorybased design of walk.up.and-use interfaces’’, Proceed. ACM CHI 90 Conference (Seattle, WA), 1990 [4] Alan J. Dix, Janet E. Finlay, Gregory D. Abowd, Russell Beale, “Human-Computer Interaction”, Prentice-Hall, 2000 [5] Jean Osterm, “Palm OS User Interface Guidelines”, PalmSource Inc., 2003 [6] John Krogstie, Petter Bae Brnadtzæg, Jan Heimm Andreas L. Opdahl, “Usable M-Commerce Systems: The Need for Model-Based Approaches”, Idea Group Inc, 2003 [7] Susy S. Chan and Xiaowen Fang, “Mobile Commerce and Usability”, Idea Group Inc, 2003 [8] Jacob Eisenstein, Jean Vanderdonckt, and Angel Puerta, “Applying Model-Based Techniques to the Development of UIs for Mobile Computers” [9] Jakob Nielsen, “Usability Engineering”, Morgan Kaufmann, 1993

5