Usability Evaluation of the Agile Software Process

Usability Evaluation of the Agile Software Process Wan Fatimah Wan Ahmad, Saad Masood Butt, and Lukman Rahim Computer and Information Sciences Department Universiti Teknologi PETRONAS Tronoh, Perak, Malaysia {fatimhd,lukmanrahim}@petronas.com.my, [email protected]

Abstract. Agile development methods are most flexible approach for software development where the development team keeps on improving the software with ongoing involvement of user. Despite its flexibility approach in software development, agile methods are still lacking usability evaluation approaches in their development, and integration of usability evaluation into agile development methods is not adequately addressed. In Agile software development, emphasis on requirement gathering and development of comprehensive feasibility reports is not considered in detail. Traditional software development considered this documentation as a quality attribute for the success of the project. Rapid solutions provided by agile software methods leads to the deficiency of good design and architecture which renders the project as expensive. As a remedy, a proposed life cycle for agile software development has been designed. The proposed life cycle outlined in this paper integrates usability evaluation concepts and agile software methodologies for the development of interactive software. To achieve the results, a survey form was formulated and carried out. The results show a greater interest of usability at the initial stage of software development along with user participation and involvement at each stage. The experiment was conducted in the software company by developing an interactive desktop-based application to evaluate the proposed life cycle and IEEE Std. 12207-2008, ISO 9241:210 was used to validate the proposed software model. Keywords: software development, usability evaluation, agile software.

1

Introduction

A high-level of usability is recognized as an important feature for software products. The software products are not successful in the market due to a very common failure in effective usability design. Thus, one of the complications involved in software development is to observe users and analyse the user participation in the design and development stages, their behaviour and efficiency, then gather ideas for the ensuing development. It is quite apparent that the development of better systems needs the collaboration of different subject matter experts (SME) from the Human Computer Interaction H. Badioze Zaman et al. (Eds.): IVIC 2013, LNCS 8237, pp. 640–651, 2013. © Springer International Publishing Switzerland 2013

Usability Evaluation of the Agile Software Process

641

(HCI), Software Engineering (SE), Stakeholder, Usability experts and User experience experts. Software Engineers, HCI experts and end-users are the most important key players of the software development life cycle. Involving users in the software development is an important thought-provoking task. Neglecting HCI approaches from software development will affect the role of usability in the software and make the software difficult to learn and to use which results in the dissatisfaction of the users. Software engineers, HCI experts and users need to cooperate with each other to create a software product that is usable and useful for the target audience. Unfortunately, in reality, these three key players (Software Engineers, HCI experts and Users) do not cooperate as smoothly as they should. By the time this lack of cooperation is released, big software projects may fail to deliver what they promised; therefore, such projects often fall short [1]. Their failure can be in various forms; such as (a) the delivered system is not able to offer considerable specifications to the conventional system which is top quality that would make it possible to actually set up the system, (b) involvement of the user is introduced too late which produces a high impact on the software efficiency , (c) the cost, time and resources are exceeded by large factors and (d) the product is difficult to understand due to the lack of usability role in the development. The agreed upon factors for which software projects fail are many with variety of types [2]. In this paper, a new agile software process is proposed where usability evaluation has been integrated into an agile software development method.

2

Literature Review

Symon [3] the author claimed that user contribution at the design stage has a smaller footprint size because it contains specific or functional matters. Due to this purpose, software designers normally avoid user’s participation in the level. On the contrary, Olsson [4], in his research, claimed that users should be engaged in the design and development of systems. User’s engagement in the application development process is an important responsibility towards the result of the application item [5]. Furthermore, Carrol [6] strengthened that user participation is an important factor in the success of a project and is the best choice for many projects. Pessagno [7] elaborated further on the importance of usability in developing interactive software. The results of the survey and usability test mentioned in his study has shown that design is an important factor in determining a site’s success because it creates its identity while simultaneously facilitating its usability. As mentioned in [8], the modified heuristics are more efficient and capture more defects then the one proposed by Nielsen 10 heuristics. Also the research [9] shows that the pitfalls of the Heuristics Evaluation (HE) and it seems useful to follow more than one method for software usability evaluation. The methodology mentioned in [10] is to improve the problems with the agile process and focus on the software design approach. Agile software development is the most widely used software models in the software Industry. Its efficiency to handle rapid change in the requirement handling and involving user at every stage of development is valued. However, agile based projects fail due to insufficient quality attributes.

642

W.F.W. Ahmad, S.M. Butt, and L. Rahim

The word agile means fast, light and more nimble. Agile process is also considered as a light weight process [11]. One major catastrophe in many software projects was the time it took for development exceeded the deadline. To develop systems faster with its limited time, agile software methods were developed. These methods develop systems faster by less focus on analysis and designing [12]. U-SCRUM methodology [13] is proposed by Singh to improve the usability. Unlike SCRUM having one product owner, U-SRUM has two product owners, one focused on the functionality and other focused on the usability. Result shows USRUM improve usability in the product then traditional SCRUM. McInerney and Maurer mentioned in [14], the possibility of integrating software engineers and usability experts in agile software development. As both are from different backgrounds having different domain knowledge that helps to solve the problem that mostly occurred in software development in their own ways and most importantly agile methods only focus in functional requirements and neglect usability issues.

3

Current Gaps in Software Models

At present various software projects are being developed using different software models. However, there are some loopholes which persist in the software development such as documentation, total cost of the project, time to complete the project, resources required for the project and usability of the software. Research and market report shows 70 to 80% effort is done for the development of feasibility report and SRS (software requirements specification) document of the software project. The requirements of the software keeps on changing with time which greatly impact the development of software. Usability does not focus on software projects as the role of usability experts are not defined properly in the development of software. In order to handle this issue huge amount is spent on users in the training of software. Still many software fails due to lack of user understanding and poor software interface. Table 1 shows the various factor leads to the failure of software. Table 1. Major factors persist in software models Factors Software Documentation Cost Time Resources Usability

Details A lot of paperwork is done to make SRS (software requirements specification) document and gather the requirement before the development starts. Non agile software project sometimes over exceed the project cost due to the unclear gathered requirement. If the SRS (software requirements specification) document keeps on changing it takes more time to complete the project before the deadline. Resources can be hardware or software. So depends on the requirement client is giving you. Many companies spend a lot of money on the training of particular software to train their users. Because real users were not involved in the design phase hence software designer design interface which is not easy for the user to understand or sometime make the whole software failed. The role of usability expert is also not clear at any stage of software development

Usability Evaluation of the Agile Software Process

643

Hedberg discussed about the Integration of HCI specialist in Open Source Software (OSS) Development [15]. Typical OSS development projects are organized around developers whose interaction is based on specialized technical aspects and source code. It seems very difficult to communicate with end users who have no technical knowledge. Hedberg proposed a model (as shown in Fig.1) that intergrade HCI in OSS and makes the existence of HCI professionals noticeable in the projects, and encourages connections between designers and the HCI specialists in the course of a project.

Fig. 1. The proposed model with the technical level and the human level roles

In 2007, Memmel et al. from Human-Computer Interaction Lab Germany proposed a software development life cycle in his paper “Agile Human Centered Software Engineering” later published by British Computer Society [16]. The authors of this paper confess that they did not implement this development lifecycle practically. But the proposed lifecycle is based on the facts of industry and research experience. His proposed lifecycle CRUISER shown in Fig.2, helps to bridge HCI and SE based on common features of both fields. CRUISER is very close to XP but don’t have agile aspects. The author tried to integrate the important discipline in one lifecycle and increased the involvement of user and stakeholder by using prototype and scenarios.

Fig. 2. Phases of CRUISER

644

W.F.W. Ahmad, S.M. Butt, and L. Rahim

Pardha proposed a model as shown in Fig.3, which provides a development infrastructure which accommodate Usability Engineering and SDLC. The proposed model have some potential downfall such as resource overhead and need expert for documentation entry into the design representation model.

Fig. 3. Proposed Model-Based Framework for Integrating Usability and Software Engineering Life Cycles

4

Proposed Lifecycle

Figure 4 shows the agile life cycle starts from Flex REQ [17] and ends at a final product. Passing through various processes helps agile software experts, usability experts to work together. Flex REQ [17] is a process to develop product feasibility documents in a small amount of time unlike traditional soft model spends a considerable amount of time in documentation to achieve product quality at the end.

Fig. 4. Agile Usability Software Engineering Lifecycle

Interactive User Prototyping consists of two further process user interface prototyping and architectural prototyping as shown in Fig. 4. The final requirements gathered from the Flex REQ phase are now in the phase of designing (IUP) where interface and architecture prototypes are refined according to the specified requirement.

Usability Evaluation of the Agile Software Process

645

The resulted prototypes from the IUP phase will further tested for the Usability by using CASI [19]. CASI is a Usability evaluation method helps to improve the usability of software interfaces. Involvement of user and usability expert is highly important in this phase to find the usability defects. CASI keeps on the evaluating the interface until user fully satisfied. After CASI phase the coding starts and later unit and acceptance testing will be conducted to check final product satisfies all specifications and useful for the customer.

5

Survey and Experiment

For this analysis, a survey has been conducted randomly with 45 selected IT professionals. The purpose of the study is to create an “Agile Usability Software Engineering Life Cycle” that could comprehend the influence of the users in the software development process. The purpose is to make software development process reliable and finally integrate the Usability Evaluation to make the software more usable. The distribution of the survey targeted IT experts, researchers, software users and stakeholders. The questionnaire is divided into four sections. Section A is about demographic information of all those people who will answer the survey questions. Section B is on the software process particularly focusing agile process in software industry. Section C looks into the Usability Evaluation in the agile software process. Section D is focused on developing a process that able to do the things that normally require human intelligence to perform that task in software development. All questions mentioned in every section were rated using the scale of 1 to 5 (1= strongly disagree, 2= disagree, 3= fair, 4= agree, and 5= strongly agree). Fig.5 shows 81% respondents agreed on the active participation of users in software development. Software interfaces also play a magnificent role in product success and failure but also prefer less documentation in software development process. Rest 5% disagree with the active participation of users in software development, role of software interfaces in product quality and also disagree on less document in software development.

Fig. 5. Active participation of users in software development

646

W.F.W. Ahmad, S.M. Butt, and L. Rahim

From Fig.6, 78% of the respondents agreed on the Usability Evaluation methods to be considered in software development. Some prefer to consider it in evaluation methods in agile software development. Remaining 6% disagree to consider in agile software development.

Fig. 6. Usability Evaluation in the agile software process

Fig.7 shows 82% respondents agreed on a development of process that is the part of the software model to make the development faster. The other 4% disagrees with such model used in software process.

Fig. 7. Use various processes in an Agile Software Model for faster development

The focus of this paper is mainly to analyze the role of usability and users in the software model. From the research, it has been illustrated that in the software development, the parts played by HCI and users are important. In addition, role of helping process in software model that makes development faster and effective. Table 2 shows the comparison of various software models based on different features.

Usability Evaluation of the Agile Software Process

647

Table 2. Comparison with other Models Features User Involvement Adaptation of Usability Development and Integration of helping process Interface Evaluation using UE Cost

SDLC Beginning No

Prototyping Model High Intermediate

Agile Model High No

Spiral Model High No

No

No

No

No

No

No

No

No

Low

High

Intermediate

Expensive

After analysis of various software models through numerous factors and keeping survey analysis report that was discussed, it has been found that all models except agile model to be expensive to use (in term of cost, time and resources), used for big projects and having lack of Usability approaches. Whereas agile software model is a renowned model and is followed by many companies for small medium and large project. Hence introducing usability approaches in agile model increase the efficiency and usability of software. The project subsequently was developed using the most popular methodologies such as Scrum, Waterfall Model and OSSD and was rated using the scale of 0 to 4 (0= No, 1= fair, 2= Normal 3= Good, 4= Excellent). The results shown in Fig.8 are based on three important features; Usability, User Involvement in software development and Time taken to meet the deadline of the project deployment. From the outcomes it indicates that Usability concentrated more in AUSEL as compared to other models. On the other hand, “User Involvements” was observed more in AUSEL and Scrum. The “Time” for completion outcome shows that by using OSSD the time of completion will be lesser as compared to other software models.

Fig. 8. Comparison with other Models

648

6

W.F.W. Ahmad, S.M. Butt, and L. Rahim

Validation

There are some essential features that are mostly considered in validation of the software model. In the proposed AUSEL, Industry Standards should be followed to validate every process and make the processes of AUSEL standardize. To validate AUSEL, ISO 9241:210 [21] (Usability standards) and IEEE Std. 12207-2008 (System Context Processes) are followed. The International Standards (ISO 9241:210 and IEEE Std 12207-2008) determine a common model for software life cycle process, having a well-defined terminology that can be recommended by the software industry [20]. Fig.9 shows the most common processes of system set by the International Standards Group that may be performed during the lifecycle of software system. The outcomes mentioned in each process need to be achieved to standardize the process. Few standard processes are used in order to validate and standardize the Agile Usability Software Engineering Lifecycle (AUSEL).

Fig. 9. System Context Processes [20]

The Table 3 shows the validation of AUSEL processes using ISO Usability Standards. The outcome of each process shown in Figure 9 has its own benchmark. The model shall be validated only when it meets the benchmark.

Usability Evaluation of the Agile Software Process

649

Table 3. ISO 9241-210 (Usability Standards)

ISO 9241-210 (Usability Standards)

AUSEL

The design is based upon an explicit understanding of users, tasks and environments. The design is driven and refined by user-centered evaluation. Users are involved throughout design and development. The process is iterative. The design addresses the whole user experience. The design team includes multidisciplinary skills and perspectives.

AUSEL understand user, user requirements and design such interface that help user to perform its task easily. In AUSEL the interface usability evaluation is carried out by CASI. Active user participation is observed in whole lifecycle. AUSEL have iteration between every process. IUP process of AUSEL focus on Usability and User Experience at software designing stage. AUSEL involves software developers, designer and Usability expert in software development.

7

Future Work

In this paper various models are discussed in each section; with literature review and gaps within software models inclusive. Several models which are discussed and proposed, are either having weakness or not yet practically tested. Nevertheless, all of these models do have their significance in terms of usability in the software development. At present Open Source Software Development is a most widely used technique in software development. However, as per discussion, Open Source Software Development fails when user involvement is considered. In few of the proposed models, it has been indicated that the importance of HCI for software development is not only just for improvement in product usability but also to increase the market value of their product. The standards used to validate the proposed model will be endorsed by the software companies. A survey technique will be carried out in order to be validated by the software companies.

8

Conclusion

Agile development methods are most flexible approach for software development where the development team keeps on improving the software with ongoing involvement of user. Despite its flexibility approach in software development, agile methods are not integrated with usability approaches, whereas it is crucial to integrate in order to achieve software usability. It is essential to incorporate usability process in the agile software method. This paper has produced a variety of contributions: literature review, current gaps in software models, survey, proposed agile software

650

W.F.W. Ahmad, S.M. Butt, and L. Rahim

model, experiments and results. From the literature and proposed life cycle we derived that there are many benefits that can be achieved by integrating usability in the Agile software model. A few major loopholes were succinctly explained under the section of current gaps in software models. A survey was conducted among IT professionals to analyse Usability Evaluation in Agile software development. After getting the survey results, a proposed Agile model i.e. Agile Usability Software Engineering Lifecycle is proposed. The proposed model is then tested with a software company to evaluate it. Meanwhile the AUSE life cycle was validated by IEEE Std 12207-2008 and ISO 9241-210 (Usability standards).

References 1. Kikuno: Why do software projects fail? Reasons and a solution using a Bayesian classifier to predict potential risk. In: 11th IEEE Pacific Rim International Symposium (2005) 2. Belengueret, J.: HCI Designers and Engineers: It is possible to work together, http://citeseerx.ist.psu.edu/viewdoc/ summary?doi=10.1.1.11.1728 3. Symon, G.: The work of IT system developers in context: an organizational case study. Human–Computer Interaction 13(1), 37–71 (2007) 4. Gasson: User involvement in decision-making in information systems development. In: Conference Proceeding of 18th IRIS. IRIS Association, Gjern Denmark (2005) 5. Ollson, E.: What active users and designers contribute in the design process. Interacting with Computers 16, 377–400 (2004), http://www.elsevierComputerScience.com 6. Carroll, M.: Introduction: The Scenario Perspective on System Development in ScenarioBased Design: Envisioning Work and Technology in System Development. IEEE Transactions on Software Engineering 24(12), 1156–1170 (1995) 7. Pessagno, R.: Design and usability of social networking web sites. In: California Polytechnic State University (2011), doi:digitalcommons.calpoly.edu/grcsp/23 8. Ivory, M.Y., Hearst, M.A.: The state of the art in automating usability evaluation of user interfaces. ACM Comput. Surv. 33, 470–516 (2007) 9. Polson, P.G., Lewis, C., Rieman, J., Wharton, C.: Cognitive walk- throughs: a method for theory-based evaluation of user interfaces. J. Man-Mach. Stud. 36, 741–773 (2002) 10. Akbari, F.: A Review to the Usage of Concepts of Software Architecture in Agile methods. In: International Symposium on Instrumentation & Measurement, Sensor Network and Automation, IEEE Software, vol. 2, pp. 389–392 (2012) 11. What is agile software development (May 14, 2008), http://www.agilealliance.org/show/2 12. Koskela: Software configuration management in agile methods. VTT publication 5 14, ESPOO, pp –54 (2003) 13. Singh, M.: U-SCRUM: An agile methodology for promoting usability. In: Proc. AGILE 2009 Conference (Agile 2009), pp. 555–560. IEEE Press (2009) 14. Mcinerney, P., Maurer, F.: UCD in agile projects: dream team or odd couple? Interactions 12(6), 19–23 (2007) 15. Hedberg, H., Iivari, N.: Integrating HCI Specialists into Open Source Software Development Projects. In: Boldyreff, C., Crowston, K., Lundell, B., Wasserman, A.I. (eds.) OSS 2009. IFIP AICT, vol. 299, pp. 251–263. Springer, Heidelberg (2009)

Usability Evaluation of the Agile Software Process

651

16. Memmel, T., Gundelsweiler, F., Reiterer, H.: CRUISER: A Cross-Discipline User Interface and Software Engineering Lifecycle. In: Jacko, J.A. (ed.) HCI 2007. LNCS, vol. 4550, pp. 174–183. Springer, Heidelberg (2007) 17. Pyla, P.S., Pérez-Quiñones, M.A., Arthur, J.D., Hartson, H.R.: Towards a model-based framework for integrating usability and software engineering life cycles. Arxiv Preprint cs/0402036 (2008) 18. Masood Butt, S., Ahmad, W.F.W.: Handling requirements using FlexREQ model. In: 2012 IEEE 3rd International Conference on Software Engineering and Service Science (ICSESS), pp. 661–664. IEEE (2012) 19. Butt, S.M., Ahmad, W.F.W.: Analysis and Evaluation of Cognitive Behavior in Software Interfaces using an Expert System. International Journal 5 (2012) 20. IEEE Standards Association, 12207-2008-IEEE System and Software EngineeringSoftware life cycle processes, https://standards.ieee.org/findstds/standard/12207-2008.html 21. ISO 9241-210:2010 Ergonomics of human-system interaction -Part 210: Human-centred design for interactive systems, http://www.iso.org/iso/catalogue_detail.htm?csnumber=52075