Usability Perspective in Software Quality - Semantic Scholar

M. Xenos, “Usability Perspective in Software Quality”

Usability Perspective in Software Quality Michalis Xenos * Computer Technology Institute, 61 Riga Feraiou Str, GR-26110, Patras, Greece, e-mail: [email protected]

Abstract. This paper presents usability perspective in the most frequently used models of software quality characteristics, namely FCM, CSQ and ISO 9126. Furthermore, it discuses the requirements, in terms of usability, in ISO 9001 quality assurance international standard (using the ISO 9000-3 guidelines for software as well), in Malcolm Baldrige National Quality Awards and in Capability Maturity Model. The aim of the paper is to demonstrate that usability forms part of the overall framework of software quality and to underline the importance of usability into software quality models and standards.

1

Introduction

The concept of ‘software quality’ can be described from five different perspectives [1]: transcendental view, user view, manufacturing view, product view and value-based view. The user view (quality as fitness for purpose) is the most common view about quality in general –not only software quality– according to the definitions of quality given by quality pioneers, such as Crosby [2]: “conformance to user requirements” and Juran [3]: “fitness for use”. As far as the other four perspectives are concerned, the transcendental view describes quality as something that can be recognised but not defined, while the value-based view describes quality in relation to the value that the customer is willing to pay for the software. Software engineers consider quality as conformance to specifications (manufacturing view) and as an abstract concept that can be divided into specific product characteristics (product view). Regardless of the way one views software quality, usability plays an important role in all quality perspectives: in recognising the quality of a software product (transcendental view), in determining if it is fit for purpose (user view), in designing from early stages how the user will use it (manufacturing view), in determining its quality characteristics based –among others– on the usability factor (product view), and in affecting the value that the user will be willing to pay for a usable –or less usable– product (value-based view). Based on usability definition as “quality in use” [4], it is easily assumed that usability is an important factor of software quality. Of course, it is not the only factor related to software quality. Section 2 discusses usability’s relation to software quality from the perspective of product view and focuses on usability’s role in the various proposed models *

Research supported by the Network of Excellence on Software Usability funded by the Greek Secretariat for Research and Technology (GSRT), in which Computer Technology Institute participates.

Usability Engineering Workshop, Proceedings of the 8th Panhellenic Conference on Informatics with international participation, Vol. 2, pp. 523-529, Cyprus, 2001.

1


for determining product characteristics related to software quality. Section 3 presents usability’s role in the development process (as part of an underlying software quality methodology), how it is related to the process maturity and the process capability to assure product quality and consequently usability. Finally, section 4 presents conclusions about the usability perspective in software quality.

2

Usability as a Software Quality Factor

As previously mentioned, usability is an important factor of software quality. Usability has always been present, even in the very first models of software quality. Of course, usability is not the only factor affecting software quality, but one of many. In the following discussion, some of the most well known models of software quality are presented and the role of usability among the other quality factors in each model is discussed. Moreover, the specific characteristics into which usability can be divided (according to each model presented) and what each characteristic means, in practical terms, are also presented. The basic idea in all software quality models is to define software quality, which is a term that is too abstract to be studied directly, by dividing it into attributes (usually called quality factors). One of these factors is usability, a concept that is also abstract and therefore is usually divided into more specific attributes (usually called characteristics). In some models, these characteristics are further divided into sub-characteristics and so on, always with the same concept in mind: to divide an abstract term into better defined terms (terms that can be measured objectively or with a reduced level of subjectivity).

Operability

Usability

Training

Communicativeness Fig. 1. Usability perspective in FCM model

Perhaps the earliest quality model is the FCM [5] model, proposed by McCall in 1977 (also known as McCall’s model). This model divides quality into 11 factors: Correctness, Reliability, Efficiency, Integrity, Usability, Maintainability, Testability, Flexibility, Portability, Reusability, Interoperability. Each one of these 11 factors is related to a set of criteria. A quality criterion can be related to more than one quality factors and vice versa. Each criterion can be associated with one or more quality metrics (a metric is a measure that captures some aspect of the criterion). The name of the model (FCM) derives from


2


the three terms: Factors – Criteria – Metrics. The usability factor in FCM model is shown in figure 1. The three criteria into which usability is divided according to the FCM model are: Operability, Training and Communicativeness. These criteria are associated only with the usability factor. Operability is associated to the user’s effort for operation and operation control (for example mouse support, macro-commands, etc). Training is associated to the effort required to teach the use of software to the user, while communicativeness is associated to how well the software communicates to the user the purpose for which it has been developed and the method to use it. Based on the FCM measurement method, each criterion could be associated with metrics in the form of questions allowing subjective “yes” or “no” answers. At about the same time that FCM was proposed, Boehm proposed CSQ [6] in 1978 (also known as Boehm’s model). This model mainly focuses on software quality from the developer’s perspective and divides quality into 7 quality factors (called intermediate constructs): Portability, Reliability, Efficiency, Human Engineering, Testability, Understandability, Modifiability. Each of these intermediate constructs is further divided into primitive constructs, which could be measured using metrics. Given the fact that understandability in CSQ is considered as a sub-characteristic of maintainability (i.e., how easily the developer can understand what a part of the code does), it is evident that usability does not exist in the prime factors of the model. However, this is not a surprise, since in these days usability was not considered as something that could be engineered into software and CSQ mostly focuses on developers. Of course, this belief has changed over the years and nowadays usability is part of the engineering process of software development [7,8].

Understandability

Usability

Learnability

Operability Fig. 2. Usability perspective in ISO 9126

FCM and CSQ became very popular quality models, with FCM being the one most commonly used by many software developers to divide quality into sub-characteristics and set measurable goals. One of the probable reasons why FCM was used more than CSQ was the increased recognition of usability’s contribution to software quality. Based on these models, international efforts lead to the development of the international standard ISO 9126 [9] for software quality. ISO 9126 comprises of a basic set of 6 independent quality characteristics: Functionality, Reliability, Usability, Efficiency, Maintainability, Portability. Unlike FCM and CSQ, ISO 9126 is completely hierarchical and relates each


3


sub-characteristic to only one of the basic characteristics. The usability characteristic in ISO 9126 is shown in figure 2. According to ISO9126, usability can be divided into understandability (which is the user’s effort for recognising the underlying concept of the software), learnability (which is the user’s effort for learning how to use the software) and operability (which is the user’s effort for operation and operation control – like mouse support, macro-commands, etc). ISO 9126 is currently used by many software developers to define quality goals and usability is always a prime concern in software quality. At the same time with ISO 9126 publication, Nielsen [10] focusing only on usability, divided it into 5 major attributes: Learnability, Efficiency, User retention over time, Error rate, Satisfaction. Learnability is related to how easy it is to learn the software functionality. Efficiency is related to how fast the user can perform certain tasks. User retention over time is related to how well the user remembers how the system works after having not used it for a period of time. Error rate is related to the number of errors that the user makes while using the software and is negatively related to usability. Finally, satisfaction is related to the user’s subjective impression of the system. Summarising, in most software quality models usability is considered as a major factor that affects the overall quality of the software. The major attributes that are associated with usability are (regardless of the model): how easily the user understands the concept of the software, how easily, efficiently and without making errors the user operates the software, how easily the user learns how to work with the software and how well this knowledge is retained.

3

Usability into a Software Quality Assurance System

In order to describe the usability perspective in the manufacturer view of software quality, we present the role of usability in ISO 9001 international standard [11] and ISO 9000-3 [12] guideline for the application of ISO 9000 into software. Then, we present the usability perspective in the Malcolm Baldrige National Quality Awards [13], which are used mostly in USA to assess the overall level of an organisation’s quality process. Finally, we present the usability perspective in the CMM Key Process Areas [14], which are more and more used by many software developers.

3.1 Usability in ISO 9001 ISO 9001 is a model for quality assurance in design-development, production, installation and servicing, where a contract between two parties requires the demonstration of the supplier’s capability to design and supply product. The requirements specified in this International Standard are aimed primarily at preventing nonconformity at all stages from design through to servicing. ISO 9000-3 provides guidelines for the application of ISO 9001 specifically to the development, supply and maintenance of software. In such a generic standard such as ISO 9001, usability is not explicitly mentioned, but something like this is expected since such standards only express general principles, so as to be interpreted by the developers of diverse products [15]. The requirements of ISO 9001 are divided into 20 sections. Of these sections (and according to ISO 9000-3 guide-


4


lines), the section under the heading “Design control” encourages the designing of software usability, in such a way, “…so as to allow validation during product acceptance…” allowing in this way the developer to freely apply any engineering method and place strong or minimum emphasis on usability design. The section under the heading “Inspection and Testing” of ISO 9001 requires that “…the supplier should validate its (software’s) operation as a complete product, when possible under conditions similar to the application environment…”. This statement requires the application of acceptance tests simulating the user working conditions, but also encourages the use of usability labs. Such labs could be used to measure software usability, as well as to run regular software alpha testing with user participation within the manufacturer labs –and before the final product delivery. Moreover, in the section under the heading “Statistical Techniques” of ISO 9001 it is required that techniques for “…verifying the acceptability of product characteristics…” are used by software developers. This is interpreted as requirement for user acceptance and user satisfaction surveys. In such surveys, usability is always an important factor that determines user acceptability of the delivered product. Summarising, usability perspective does not appear very strongly in ISO 9001 software quality assurance international standard. However, this standard encourages (although not requires) usability engineering and the use of usability labs (and this becomes even more evident after studying ISO 9000-3 guidelines). On the other hand, ISO 9001 requires user satisfaction measurements, where usability plays a very important role compared to all other factors.

3.2 Usability in Baldrige Awards Malcolm Bardrige National Quality Awards are very popular in USA. They are a measure of an organisation’s overall level of quality and can be used to assess a quality process. They consist of criteria, on a 1000-point scale, measuring the conformance of the quality process used by the organisation to specific requirements. The methodology proposed in order to achieve maximum conformance to most of these requirements constitutes a quality model that can be successfully applied to software [16]. Many software development companies have scored highly in Baldrige awards and –most important– have improved the quality of the development process in an effort to conform to the awards’ requirements. The requirements of the Baldrige awards are divided into 7 major categories. Each category consists of a number of criteria (with a number of questions associated to each criterion). Usability is related to the 7th category which comprises of the following criteria (the points corresponding to the questions associated to each criterion are in parenthesis): 7.1 Customer expectations (35), 7.2 Customer relationship management (65), 7.3 Commitment to customers (15), 7.4 Customer satisfaction determination (30), 7.5 Customer satisfaction results (85), 7.6 Customer satisfaction comparison (70). All the above criteria add up to a total of 300 points, which is 30% of the entire Bardrige award points. Usability is highly related to 7.4 Customer satisfaction determination and to 7.5 Customer satisfaction results, where the following are required: “…a robust system for determining customer satisfaction…”, “…improve the mechanisms for determining customer satisfaction…” and “…key indicators (e.g., survey data, customer assessments) that reflect


5


high customer satisfaction…”. In brief, requirements related to usability play a very important role when a software development company bases the development process on the requirements of Malcolm Baldrige National Quality Awards. Usability issues are very important in order to achieve high scores in Baldrige awards and a company is required to apply measurements of customer (user) satisfaction, using either usability labs or surveys, and use the results of such measurements as indicators of improvement.

3.3 Usability in CMM The Capability Maturity Model for Software (CMM) [17] provides software organisations with guidance on how to gain control on their processes for developing and maintaining software and how to evolve toward a culture of software engineering and management excellence. CMM is based on continuous process improvement and provides a framework for organising evolutionary steps into 5 maturity levels (1. initial, 2. repeatable, 3. defined, 4. managed, 5. optimising) that lay successive foundations for continuous process improvement. Each maturity level of CMM is composed of key process areas and each key process area identifies a cluster of related activities that, when performed collectively, achieve a set of goals considered important for establishing process capability in that maturity level. Each key process area is described in terms of key practices that, when implemented, help to satisfy the goals of that key process area. Process categories are: Management, Organisational and Engineering. Usability issues are related to the engineering key process area, of “Software Product Engineering” of CMM level 3. In this third level key process area, user involvement into final software inspection for conformance to the required software characteristics is required. Furthermore, the use of tools and method to ensure conformance to these characteristics is encouraged. In such a way, given the fact that usability is an important software characteristic strongly related to user satisfaction, usability engineering is strongly encouraged and measurements of user satisfaction (and of course usability measurements) are considered important in the engineering process. Summarising, in Capability Maturity Model, usability does not appear to be very strongly present and perhaps is less present than in ISO 9001. This is due to the fact that usability is considered as a software quality characteristic for the achievement of which specific requirements can be set, designed and engineered and therefore, does not require special attention (more than the other software quality characteristics).

4

Conclusions

As demonstrated in this paper (and despite the reference to Boehm’s model), nowadays, usability is considered as one of the major factors contributing to software quality. It is not important (and certainly not within the scope of this paper) whether usability is one of 6 (as in the case of ISO 9126) or 11 (as in the case of FCM model) or any number of software quality factors. What is important is that what we perceive and call ‘software quality’ cannot be achieved without achieving what we perceive and call ‘software usability’, a fact stressed (more or less strongly) by most software quality models.


6


Usability is a factor present (directly or indirectly) in almost all software quality assurance models. It is usually divided in more specific characteristics so that it can be measured. The amount of emphasis given on usability, within software quality, depends on the type of application or the model itself. Models focusing on developers usually put less emphasis on usability (compared to other quality factors) than models focusing on the product itself. As far as certain types of software applications are concerned, usability is the most important quality factor, however, this should not lead to the false assumption that usability is the only quality factor: the quality of a usable but unreliable or nonfunctional application is just as low as the quality of an application of low usability. On the other hand, the assumption that in some applications usability is not important is just as misleading. For example, one may consider usability as a very important factor for educational software, but not as important for real time control software. This is a false assumption. It is argued that regardless of the great differences in the usability requirements of various software systems (deriving from the various categories of users), usability is always important. To use the aforementioned example, in the case of a real time control software, reliability certainly is a very important factor related to software quality, however, usability is important as well, given the fact that user errors is such systems must be eliminated and usability is strongly related to user error rate. In the so-called software quality assurance models (such as ISO 9001 and Capability Maturity Model) it may seem as if usability is not a factor on which emphasis is given. It is a fact that the Malcolm Bardrige National Quality Awards place a lot of emphasis on usability in comparison to ISO 9001 and CMM, since, in Baldrige awards, usability (related to customer satisfaction) is very important for the achievement of a high score. On the other hand, the experienced engineer will recognise that usability is present in all three software quality assurance models mentioned above and that the need for setting up usability goals, design software usability and working under a usability engineering method is not only encouraged, but in some aspects required by ISO 9001 and CMM as well.

References 1. Kitchenham, B., Pfleeger S.: Software quality – The elusive target. IEEE Software, January (1996) 12–21. 2. Crosby, P.: Quality is free, New-York. McGraw-Hill. 1979. 3. Juran, J., Gryna, F.: Quality Planning and Analysis. 2nd ed. New-York. McGraw-Hill. 1980. 4. ISO/IEC 14598-1.: Software product evaluation – General Overview. International Organisation for Standardisation. Geneva. 1999. 5. McCall, J. et al.: Factors in software quality. Vol. 1,2,3. AD/A-049-014/015/055. Nat. Tech. Inf. Service. Springfield. 1977. 6. Boehm, B. et al.: Characteristics of software quality. North Holland. 1978. 7. Ferre, X. et al.: Usability basics for software developers. IEEE Software, January (2001) 22–29. 8. Anderson, J. et al.: Integrating usability techniques into software development. IEEE Software, January (2001) 46–53.


7


9. ISO 9126.: Information technology – Software product evaluation – Quality characteristics and guidelines for their use. International Organisation for Standardisation. Geneva. 1992. 10. Nielsen, J.: Usability Engineering. AP Professional. Boston. Mass. 1993. 11. ISO/IEC 9001.: Quality management and quality assurance standards. International Organisation for Standardisation. Geneva. 1991. 12. ISO 9000-3.: Quality systems – Guide to the application of ISO 9001 to the development, supply and maintenance of software. International Organisation for Standardisation. Geneva. 1991. 13. Steeples, M.: The corporate guide to the Malcom Baldrige National Quality Award. WI: ASQC Quality Press. 1993. 14. Paulk, M. et al.: Key practices of the Capability Maturity Model. Version 1.1. Software Engineering Institute. Technical Report. CMU/SEI-93-TR-025. 1993. 15. Ince, D.: ISO 9001 and Software quality assurance. Quality Forum. McGraw-Hill. 1994. 16. Kaplan, C. et al.: Secrets of software quality. McGraw Hill. 1995. 17. Paulk, M. et al.: Capability Maturity Model for software. Software Engineering Institute. Technical Report. CMU/SEI-91-TR-024. 1991.


8