35th International Conference on Computers and Industrial ...

MEASURING USABILITY OF ICONIC BASED GUIs OF MOBILE EMERGENCY SERVICE SOFTWARE BY USING HCI Y.Batu Salman, Adem Karahoca Bahcesehir University, Engineering Faculty, Computer Engineering Department Bahcesehir, Istanbul, TURKEY, (+90)212-6696523, [email protected], [email protected]

Abstract: The cause of having many dimensions and factors, the term “quality” has many different related definitions yet. In this paper, the approaches are settled on the software quality. From a general point of view, the usability of the software product is the important point for the software quality. In other words, the combination of all of the quality metrics can describe usability and the effectiveness of the product. This paper provides a detailed information about the software process improvement methodologies; capability maturity model (CMM) and ISO 9000 standards which are integrated to a software, used in the emergency service of Acibadem Hospital in Istanbul, Turkey. The aim of this paper is to determine the comparison of the CMM and the ISO 9000 standards by only using the one of the software quality factors which is usability. Because of the reason of considering about usability factor, the ISO 9126 and ISO 9241 standards are used in the study. Keywords: Software process improvement, the capability maturity model, ISO 9126, ISO 9241, quality factors, software quality assurance, usability, HCI, Cognitive Science

1. Introduction When it is based on the usability definitions as “quality in use” (ISO/IEC 14598-1,1999), it can be clearly understood that usability has recently become a key software quality factor. From the view of (Abran et al.,2003) usability refers to a set of multiple concepts, such as execution time, performance, user satisfaction and ease of learning. Also, usability has some categories such as product effectiveness which includes output and satisfaction at the time of use of the product, product attributes such as interfaces, capability of the organization and process used to develop the product (Abran et al.,2003). Such as (Schneidermann, 1998), (Nielsen,1993) defines usability by giving five quality components; learnability, effeciency, memorability, errors and satisfaction. In addition to these, (Jones,1997) defines usability like the total effort required to learn, operate and use software or hardware. Designing a well-organized system for every fields of the industry is very important for effective productivity and quality assurance to provide long term solutions(Ashrafi, 2003). Organizations understand the importance of integrating new software technologies and methodologies to increase and improve the productivity, quality, security for fundamental problems in managing the software process (Anderson et.al,2001). Software companies must use the software process improvement (SPI) methodologies such as capability maturity model (CMM) which is mostly used in USA and ISO 9000 standards which is especially used in Europe and Japan (Paulk et al.,1991), (Paulk et al.,1993). In the study of (Xenos,2001), the purpose is deciding the perspectives of ISO standards and capability maturity model (CMM) based on the software usability factor, according to the study, the experienced engineer will get that usability is present in both software quality methodologies and the need for setting up usability goals, design software usability and working under a usability engineering method is not only encouraged, but required by ISO standards and CMM in some situations. In the study of (Abran et al.,2003), the distinctions between ISO 9241 and ISO 9126 are studied, the differences are occurred because both are designed by different perspectives; one is by human-computer interaction experts and the other is by software engineering experts. This project first stage includes the user and organization requirements analysis, rapid prototyping, design and usability evaluation of the software that is used in emergency service at Acibadem Hospital in Istanbul, Turkey (Karahoca et al., 2005). Additionally in this study, iconic based user interfaces are designed for electronic healthcare recording (EHRs) systems by using Human Computer Interaction (HCI) techniques with its focus on cognitive science to determine what and how users understand graphical user interfaces (GUIs). For the iconic interfaces, the icons are decided to be used for the system are selected by a test which

35th International Conference on Computers and Industrial Engineering

1663

is completed by the healthcare staff in Acibadem Hospital. The detailed information discussed in the test results section of the paper. The purpose of this paper is to evaluate the distinctions and similarities between the software process improvement (SPI) methodologies, specially capability maturity model, ISO 9126 and ISO 9241 standards from the concept of software quality factor; usability, quality in use. The evaluated results and conclusion are gained by integrating these methodologies into the software which is designed for the emergency service of the Acibadem Hospital in Istanbul, Turkey. The tested software is based on iconic GUIs to find out the effectiveness and efficiency of it. Interfaces are used to test the usability of the product on the healthcare staff. In Section 2 of this paper, software usability factor and its role in capability maturity model and ISO 9126 & ISO 9241 are defined, emphasized their weaknesses and strengths. In Section 3, the role of the iconic based product is given from the perspectives of capability maturity model and ISO standards. Test results and observations are given in Section 4 and finally the discussions, further recommendations are in Section 5. 2. Background 2.1. Software Quality Factors It is impossible for all the projects developed to become within the estimated cost or budget and reaching the maximum number of software quality factors, because the nature of the system under development will determine the weighting of quality factors to be achieved in the delivered software. Here are the software quality factors with their short explanations (Galin, 2004). Table 1. Software Quality Factors Correctness: matching the user’s objectives. Reliability: the specified time period of a software performing Efficiency: more work with less system resources Integrity: the control of accessing to the software by unauthorized users. Usability: to learn and operate the software. Maintainability: degree of ease to fix errors.

Testability: effort to test any function of the software product. Flexibility: ease of modifications. Portability: efforts to change the environment of the software. Reusability: ease of effort to use the software to another environment. Interoperability: combining one system with another.

The most important thing to remain competitive in the sector for the software organizations is to develop quality software products, on time and within the estimated budget (Ashrafi,2003). Software quality factors are the metrics for a product to develop it in required and expected characteristics. As it is known, each software system is unique with its basic characteristics such as (1) long life cycle which includes maintainability, flexibility, portability; (2) human factors which includes reliability, correctness, testability, (3) real-time applications which includes efficiency, reliability, correctness; (4) classified information processed which includes integrity. Each system must be analyzed for its fundamental characteristics. 2.2. Usability The usability of a software product recently became one of the most important software quality factors. The usability term is coming from “user friendly” and its target is reaching the quality in use for the end users. Not like the cost and time parameters, because of having a variety of dimensions, it is a hard task to measure the quality in usability. Usability is defined as the ease of use and acceptability of a system for a particular class of users carrying out specific tasks in a specific environment. Ease of use affects the users’ performance and satisfaction, while accepting the effects whether the product is used. As it is related with the effort of learning, operating and dealing with the input and output of the program, the criterions; training, input/output rate, input/output volume and communicativeness are included by the usability software quality factor. In other words, it depends on the end user’s experience and effort with the software product in the parallel of the combination of the all factors such as learnability, satisfaction, effectiveness, usefulness of the product and efficiency. By using given factors of the product, the usability can be measured. Table 2. Usability factors Learnability Satisfaction Effectiveness Usefulness Efficieny

1664

is the measure of time period passed for a user to become productive and re-learning period after not using the product. is the rate of the users who are becoming more adaptable with the product. the rate of the user’s success while performing the product the measure which the product enables a user to reach the estimated values and to do the related work. the measure of preventing the chance of error by the user while working with the software product.


2.3.2. Capability Maturity Model Levels and Key Process Areas The capability maturity model is designed for the software organizations to organize a plan of process improvement for their practices. In CMM, there are some activities which must be targeted and achieved one by one. By focusing on these activities, the capabilities of software process improvement can be gained by the organizations. As it is proposed, there is a leveled architecture like the other methodologies, developed for the quality management and improvement (Galin, 2004). Table 3. CMM key process areas Level 1 – Initial

Level 2 – Managed Process Level 3 – Defined Process

- Ad Hoc Processes

- Requirement Management - Software Project Planning - Software Project Tracking and Control Software Subcontract Management Supplier Agreement Management - Measurement and Analysis - Software Quality Assurance - Software Configuration Management

Integrated Software Management - Technical Solution Organization Process Focus Organization Process Definition - Organization Training - Risk Management Software Product Engineering - Integrated Teaming

Level 4 – Quantitatively Managed Process

Level 5 – Optimizing Process

- Quantitative Software Management Software Quality Management - Organizational Process Performance

- Casual Analysis and Resolution Technology and Process Change Management - Defect Prevention

Each key process area is described in terms of key practices which support to satisfy the objectives of the key process areas when implemented. There are some categories of the process which are management, organizational and engineering. The software usability factor is presented in the level 3 of capability maturity model in “Software Product Engineering” key process area (Xenos,2001). In capability maturity model, usability is not strongly presented. In ISO 9000 standards, usability has a greater role than in CMM. 2.4. ISO 9241 on Software Usability In ISO 9241, the term “usability” is defined as the achieving goals and objectives with effectiveness, efficiency and satisfaction of a product for the chosen users. According to the ISO 9241, the measurable characteristics of usability factor are depends on efficiency; resources such as financial, human, satisfaction; comfort and acceptance and effectiveness; completeness, accuracy. In ISO 9241, the main point is how to identify the information which is necessary to take into account while evaluating usability in terms of measures of user performance and satisfaction are explained and the way should be followed is told. Table 4. Usability Evaluation Parameters in ISO 9241-10 1. 2. 3. 4. 5. 6. 7. 8. 9. 10.

Concluding time of work Concluding rate of work The percentage of completing work per unit time The rate of success over failure Time spent for errors The number or percentage of errors The rate of the completing without any error in estimated time The number of commands used The frequency of help and documentation The percentage of positive or negative user conditions

11. The repeating number of unsuccessful commands 12. The number of successful and unsuccessful users 13. The number of interfaces which are causing unwanted directions of the users 14. The number of unused commands 15. The number of users preferencing the system 16. The duration of users to solve a problem when occurred 17. The amount of clicking to complete the task 18. The number of losing the control of system by the users 19. The parts of the system that the users are satisfied or disappointed

According to the results of the given factors in above, the software usability test is concluded. In addition to these, measurement of system usability has three characteristics. How well do the users achieve their goals using the system?(Abran et al.,2003) What resources are consumed in order to achieve their goals? (Abran et al.,2003)How do the users feel about their use of system? (Wixon&Wilson, 1997) 2.4.2. ISO 9126 on Software Usability ISO 9126 states a quality model of six different factors which are usability, functionality, reliability, efficiency, maintainability and portability for the development process of the software product. In this model, the software factor usability has two different roles, product and process oriented. In product oriented role, usability is a part of a detailed software development activity and in process oriented role it provides a matching between the product and the user expectations. In ISO 9126, the definition of the term software usability is the capability of the software product to be understood, learned, used and attractive to the user, when used under specified conditions. The measurable characteristics of usability factor are depends on understandability of the user for the software product,


1665

learnability which is the effort of the user to learn the applications, operability and finally attractiveness is related with the graphical interface of the products. 2.4.3. Differences between ISO 9126 & ISO 9241 on Usability Approach Here is a table of the distinctions between these two standards based the measurable characteristics that each of them has. (Abran et al.,2003) Table 5. Differences between ISO 9126 and ISO 9241 for usability ISO 9126 Effectiveness Efficiency Satisfaction

ISO 9241 Understandability Learnability Operability Attractiveness

According to (Bevan & Schoeffel, 2001), ISO 9241 can provide to get in which context particular attributes specified in ISO 9126 are required. 2.5. CMM vs ISO 9000 From the general view of the methodologies, the following table includes the distinctions and degrees of the software process improvement methodologies capability maturity model and ISO 9000. For each of the methodologies, a score has been given from 1 to 3 for the given requirements. (Pijl et al.,1997) Table 6. Differences between CMM & ISO 9001 standards CMM

ISO 9000

3 2 3 2 2 2 3 1 18

3 2 2 3 2 3 2 3 20

Requirement Stability Scope of the model Area of application Ease of application Availability of tasks Availability of experience Ease of presenting results Degree of acceptance TOTAL

3. Materials & Methods Cognitive walkthrough methodology used to inspect the simplest way to follow the process for the users. A defined task, break into the sub task hierarchy to create an action sequence tree (Dix et al.,2004). In the cognitive walkthrough evaluation methodology, we tested the developed software according to the listed items. 1. Prototyping of the emergency software is prepared for interface usability testing, 2. A description for tasks is listed for nurses, 3. A scenario is prepared for the needed actions, 4. Experienced and novice users are determined, Every test performed on the locally worked emergency software on the PC and every stage has fully running GUIs with local databases. Therefore, users are tested with near to fully implemented version of emergency software. Emergency service nurses follow up related forms according to the sequence of the S1={Tn1,Tn2,Tn3,Tn4,Tn5,Tn6,Tn7,Tn8,Tn9,Tn10,Tn11,Tn12,Tn13,Tn14,Tn15,Tn16,Tn17,Tn18, Tn19,Tn20,Tn21,Tn22,Tn23,Tn24} Each S1 task sequence starts from T1 for the new arrival patients and when the nurses finish the Tn1 system automatically passes to Tn2 until reaching the task Tn24. Nielsen’s(1993) discount usability techniques provide a benefit to implement tests only five users to gain 75% success. Table 7. Task sequence for the nurses General Info

Health History

Arrival Info

Treatment Information

Observations

Tn1: Patient History Tn2: Family History Tn3: Contact Tn4: Religious Belief Tn5: Translator

Tn6: Previous Diseases Tn7: Previous Surgeries Tn8: Addictions Tn9: Still Used Medicine Tn10: Nutrition Tn11: Allergy:

Tn12: Arrival Way Tn13: Triage Tn14: Judical Event Tn15:Arrival Complaints Tn16:Vital Symptoms Tn17:Functional Efficiency Tn18:Pain Tn19:Mood

Tn 20: Required Inspections Tn21: Requests and Aplications

Tn22:Vital Symptom Observation Tn23: Nurse Observation Tn24: Discharge Information

1666


Figure shows the interface of the software, for the usage of healthcare staff in the hospital to keep the track of patients who came to the emergency service. As it can be clearly seen from the system, it is an iconic based study which is testing and measuring the usability of it by the given metrics. The modules are grouped according to the related fields. It starts from the top to the end until the data entered is saved for each form.

Figure 1. Nurse Menu System on Arrival Way form

In here, the key point is the decision of the icons which will be used for the system. The used icons are decided after making some tests on the healthcare staff in Acibadem Hospital. These tests are including the terms related with the forms of the software product, and each term is representing each form. It is asked to healthcare staff to draw a figure for each medical term. Mostly drawn figures are considered as the icons for the GUIs. These pictures are drawn by 43 doctors and 34 nurses. 4. Test Results While evaluating the success of usability software factor for this product, the performance and motivation of user’s interaction are included. The user’s efforts are measured by taking time parameter for each form to be completed. Also people who are expert in the iconic-based GUIs are grouped as their familiarities in computer usage. On the other hand, without considering about time, the correctness ratio of the entered information is 100% in the forms. In other words, although using different styled interfaces for the users, the first view of iconic based GUIs did not cause any problems for the healthcare staff in first tests. Table 8. Average Time Rate of Nurses to complete all tasks. Login Form General Info Health History Arrival Info Treatment Info Observations Discharge Info

Nurse 1 20 115 135 125 30 55 10

Nurse 2 5 90 120 130 25 50 10

Nurse 3 5 95 105 145 20 65 15

Nurse 4 15 110 155 190 45 65 10

Nurse 5 5 85 140 115 30 45 15

Average Time(sec) 10 99 131 141 30 56 12

The result of the test which is experimented on five nurses is given in Table 8. It includes the completion time for a group of tasks which are grouped in the software. It can be seen from Figure 1. The nurses that are familiar with the computer usage have success ratios between 89% and 93.2%. However the nurses that do not use computer much like the others has a less success ratio, 74.5%. As a result, it is observed that the ratio of familiarity with the computer systems of the end users are directly effected the success ratio of completing the forms. In addition to these researches, to evaluate the usability of the product which is designed by using iconic based GUIs clearer, 10 chosen questions are asked to the users. Table 9 includes the questions and the ratio of answering “yes” for them.


1667

Table 9. User Evaluations for iconic-based GUIs Do you use computers in your daily life? The use of icons in the interfaces helped me. Grouping of the forms helped me. Information entry is easy to use. The order of the data entry forms is similar with the existing printed forms. The correct selection of icons for each term effects the success ratio. The interfaces helped while entering data to the forms. Icons made me feel more comfortable for data entry. The product satisfied me. The product is easy to learn.

83% YES 91% YES 93% YES 74% YES 87% YES 96% YES 92% YES 88% YES 90& YES 92% YES

5. Conclusion In this study, health sector is selected to observe the experiments and results according to the usability software factor of the new product. Although users complete their required tasks by using iconic based GUIs, the factor of ease of learnability has an important role of using the product in high success ratio. On the other hand, while analyzing the icons for each form in the product, the demands of the end-users are considered. Selecting the icons by getting feedback from the users increased the usability rate of the software product that developed for this study. References 1. Abran, A., Khelifi, A. and Suryn,W. (2003). Usability Meanings and Interpretations in ISO Standards, Software Quality Journal, 11, 315-338. 2. Anderson, J., Fleck,F., Garrity,K. and Drake, F. (2001). Integrating Usability Techniques into Software Development, IEEE Software, vol.18, no.1,pp. 46-53, January/February. 3. Ashrafi , N. (2003). The impact of software process improvement on quality: in theory and practice, Information and Management 40, 667-690. 4. Bevan, N. and Schoeffel, R.(2001). A proposed standard for consumer product usability, UAHCI, New Orleans 5. Dix, A.,Finley, J.,Abown, G., Beale, A. (2004). Human-Computer Interaction, Prentice Hall. 6. Galin, D. (2004). Software Quality Assurance from Theory to Implementation, Pearson Education 7. ISO 9241-11 (1997) Draft International Standard on Ergonomics Requirements for office work with visual display terminals (VDT), Part 11: Guidance on Usability, ISO, 1997. 8. ISO/IEC 9126 (1998) Information Technology – Software product quality, Part 1: Quality model, ISO 9. ISO/IEC 14598-1 (1999). Software product evaluation – General Overview, International Organisation for Standardisation, Geneva 10. Jones, C.,(1997). Software quality analysis and guidelines for success, Int. Thomson Computer Press, 492. 11. Karahoca, A., Karahoca D., Yalçin, S.(2005). Usability Solution in Mobile Software for Hospital Emergency Services by using HCI, Submitted to Journal of IEEE TRANSACTIONS ON INFORMATION TECHNOLOGY IN BIOMEDICINE 12. Nielsen, J. (1993). Usability Engineering, San Diego: Academic Press, Inc. 13. Paulk, M.C., Curtis, B. and Chrissis, M.B.(1991). Capability Maturity Model for Software, Software Institute, CMU/SEI-91-TR-24, DTIC Number AD24063. 14. Paulk, M.C., Curtis,B., Chrissis, M.B. and Weber, C.V.(1993). Capability Maturity Model for Software, Version 1.1. Software Engineering Institute, CMU/SE-93-TR-24. 15. Pijl, G.J.V., Swinkels, P. and Verrijdt, J.G.(1997). ISO 9000 versus CMM: Standardization and certification of IS development, Information and Management 32, 267-274. 16. Schneiderman, B. (1998). Designing the user interface, Addison Wesley. 17. Wixon, D. and Wilson, C. (1997). The usability engineering framework for product design and evaluation, In Handbook of Human-Computer Interaction, ed. M.Helander, Amsterdam, 665. 18. Xenos, M., Usability perspective in software quality, (2001), Proceedings of 8th Panhellenic Conference on Informatics with international participation, Vol.2, 523-529.

1668
