Classification Model of the Software Quality Level

2018 International Conference on the Quality of Information and Communications Technology

Classification Model of the Software Quality Level according to the user’s perception Jˆonatas Mendonc¸a

Ana Cristina Fernandes Lima

Vin´ıcius Di Oliveira

University of Bras´ılia - UnB Master Program of Applied Computing Bras´ılia, DF [email protected]

University of Bras´ılia - UnB Master Program of Applied Computing Bras´ılia, DF [email protected]

University of Bras´ılia - UnB Master Program of Applied Computing Bras´ılia, DF [email protected]

Ana Carla Bittencourt Reis

Simone Borges Sim˜ao Monteiro

University of Bras´ılia - UnB Master Program of Applied Computing Bras´ılia, DF [email protected]

University of Bras´ılia - UnB Master Program of Applied Computing Bras´ılia, DF [email protected]

Abstract—The quality of software has become indispensable today, so approaches to ensure the quality of software products is needed. This work consists in the use of the ELECTRE TRI method applying the MCDA methodology to classify the software quality according to the end-users’ perception. It is a case study, characterized as exploratory of practical nature and follows a semi-structured qualitative-quantitative approach with users of a distance education software.This method allowed the identification and structuring of the problem, besides the measurement and classification of the software quality level. Index Terms—software quality classification MCDA ELECTRE TRI user perception questionnaire

Applied Computing. The objective is to extract criteria of quality in use and the user’s perception, for the development of an action plan for the continuous improvement of software. II. S OFTWARE Q UALITY AND C ONTEXT According to SWEBOK 3.0 [3], software quality is an area of software engineering knowledge that can refer to: “the desired characteristics of software products, the extent to which a particular software product has these characteristics and processes, tools and techniques that are used to ensure these characteristics.” The method presented addresses two dimensions of software quality similar to ISO 9126 [4], however with a more comprehensive view regarding the quality criteria. In this new version of ISO25010 [6] the quality in use specifies quality characteristics related to human iteration with the software and refers to the ability to meet the requirements to achieve specified goals with productivity, effectiveness, safety and user satisfaction [2], [5]. The Quality of Use approach intend also to ensure functionality and appropriate support for real-life use. It is considered not only the end-user’s view, but the context of the work environment being represented by the following characteristics: Effectiveness, Efficiency, Satisfaction, Absence of Risk and Context Coverage [2], [6]. Virtual learning environments are softwares that aid in the assembly of courses accessible through the Internet and designed to help students and teachers in content management. Thus, the environment needs to develop usability standards, which comprise the degree of ease in which the system enables user interaction with tools and other users [7]. The Moodle Modular Object-Oriented Dynamic Learning Environment is a distance learning platform based on free software. In this case, the @aprender UnB Environment is used at the University of Bras´ılia with the Moodle platform as a tool to support students and teachers in the learning activities.

I. I NTRODUCTION Softwares today support us in our daily activities, from a daily communication software to the virtual environments of learning. In this way, quality becomes a great motivator, and receiving quality products and services is a goal that covers the area of software development. On the other hand, after the completion phase of software development, it is available for use in the so-called productive environment. Most of the time, feedback is only collected from the end users when there is a serious problem in its use and an intervention is necessary. Multi-criteria methods can support decisions regarding software quality analysis. Multiple-criterion decision-making methods are suitable for problems, and decisions and assumptions are based on multiple criteria, which are sometimes conflicting, and require an axiomatic structure to provide decision support [1], [2]. The proposed method is applied through a case study, in a Virtual Learning Environment, the Moodle 1 , widely used in universities in the world, with 10,723 universities registered in the United States and 5,325 Brazilian universities. The case study was carried out at the University of Bras´ılia - UnB, with the support of the undergraduate and graduate students in Production Engineering and Master Program of 1 Moodle:

www.moodle.org

978-1-5386-5841-3/18/$31.00 ©2018 IEEE DOI 10.1109/QUATIC.2018.00054

297

result of the 19 sub-criteria analysis was: i) There was a preponderance of the learning environment classification as Class A, that is, Very Good; ii) Users a19 and a37, based on the results applied with the ELECTRE-TRI, rated the quality level of the learning environment as being Regular; iii) There were no ratings like Bad (D) and Very Bad (E); iv) The level of quality classification of the UnB @aprender environment, using the 19 quality criteria approach with their respective weights, was “Very Good”.

III. M ETHODOLOGY In order to describe the situation addressed by the present study, the investigative approach based on quantitative and qualitative data of the case study was carried out with users of the @aprender environment. A structured questionnaire was applied to identify the quality of use by following the criteria of Effectiveness, Efficiency, Satisfaction, Safety and Context. As the problem in which subjectivity is strongly present, the proposal was based on the concepts of multi-criteria decision analysis (MCDA) and used as the MCDA method, ELECTRE TRI [41]. The research was structured in the following steps according to the MCDA approach [8]: Problem Identification and Structuring, Construction and model use, and Develop Action Plan.

VI. C ONCLUSION This work aimed to present a multi-criteria analysis model, which was applied in a case study on an UnB learning platform, to evaluate the level of quality of a software, according to users feedback. The classification method applied in the article, regarding problems of classification of the quality level in use of a software, are still few explored. It is observed a great use in the analysis of services quality, regarding the level of user satisfaction, but still little used to analyze the quality of a software. The case study using the ELECTRE-TRI method in the learning environment revealed that through evaluators’ judgment on software quality, obeying 19 criteria and their respective weights, the learning environment quality was classified as Very Good from the end-user’s perspective. Nevertheless, it is important to note that some evaluators rated the software as “Regular” and “Good”, which demonstrates a need to analyze the problems that caused the low evaluation. For future work, new questions must be answered. It is intended to cover applied research if different evaluators profiles impact the classification result. Further studies could cover other MCDA methods considering and other dimensions of quality, such as internal and external quality, in order to minimize possible impacts presented in the quality in use.

IV. C ASE STUDY Identifying the Study Object: The development of this study was aimed at the learning environment, developed with the Moodle Platform at the University of Brasilia. Specify the Criteria: The criteria were defined according to a literature review on the quality criteria proposed by the quality models in use. The following quality dimensions were considered (Effectiveness, Efficiency, Satisfaction, Safety, Context). For each quality dimension, criteria were defined. Evaluation: Application of a questionnaire in which value judgments regarding the quality in use of the learning platform were collected, using the multi-criteria analysis. Equivalence Class: Identify the equivalence classes together with their respective limits (lower and upper). 5 classification categories or classes have been defined and their lower and upper boundaries defined: Very good (5 − 4.5), Good (4.5 − 3.5), Regular (3.5 − 2.5), Bad (2.5 − 1.5) and Very bad (1.5 − 1.0). Scale of Judgments: The following scale was used: Totally Agree - 5; Partially Agree - 4, Neither Agree, Neither Disagree - 3; Partially Disagree - 2; Totally Disagree - 1. Value Judgment: Collected with the university’s teachers and students who completed the on-line filling. The results obtained from these judgments were record in quantitative data. Classification in the ELECTRE Method: In this stage the classification of students’ and teachers’ perceptions regarding the quality in use was obtained using ELECTRE TRI classification procedures. The ELECTRE TRI method allows the classification of alternatives according to two procedures: optimistic procedure (less demanding) and pessimistic procedure (more demanding).

R EFERENCES [1] Trendowicz and S. Kopczynska, “Adapting multi-criteria decision analysis for assessing the quality of software products. current approaches and future perspectives.” Advances in COMPUTERS, vol. 93, pp. 153–226, 2014. [2] M. H. B. M. Moraes and F. R. Lima Junior, “Proposic¸a˜ o e aplicac¸a˜ o de uma me- todologia baseada no ahp e na iso/iec 25000 para apoiar a avaliac¸a˜ o da qualidade de softwares de gest˜ao de projetos,” Revista GEPROS, vol. 12, no. 2, p. 239, 2017. [3] P. Bourque and R. E. Fairley, Guide to the Software Engineering Body of Knowledge (SWEBOK(R)): Version 3.0, 3rd ed. Los Alamitos, CA, USA: IEEE Computer Society Press, 2014. [4] ISO/IEC, ISO/IEC 9126. Software engineering – Product quality. ISO/IEC 9126, 2001. [5] I. 25000, “Iec 25000 software and system engineering–software product quality requirements and evaluation (square)–guide to square,” International Organization for Standarization, 2005. [6] I. 25010, “Iec 25010,” International Organization for Standarization, 2006. [7] L. M. d. Brito, J. R. Giuberti J´unior, S. G. S. Gomes, and J. B. Mota, “Ambientes virtuais de aprendizagem como ferramentas de apoio em cursos presenciais e a distˆancia.” 2013. [8] V. Belton and T. Stewart, Multiple criteria decision analysis: an integrated approach. Springer Science & Business Media, 2002. [9] W. Yu, “ELECTRE TRI (aspects m´ethodologiques et manuel d’utilisation),” Document- Universit´e de Paris-Dauphine, LAMSADE, 1992.

V. D ISCUSSION AND R ESULTS The results were obtained by the ELECTRE TRI classification algorithm using the J-ELECTRE tool. The quality scores were defined in the five classes and was pointed between a more optimistic and a pessimistic analysis [9]. This problem had 54 alternatives (a1 to a54), the answers of each evaluator, and 19 criteria (g1 to g19). The judgment data were processed through the ELECTRE-TRI algorithm classification order. It could be verified that there was no incomparability between the results found in the research using this approach. The

298