Abstract â Software flexibility has been viewed as one of the software quality ... Keywords: Software quality assessment, flexibility measurement, extensibility, ...
International Review on Computers and Software (I.RE.CO.S.), Vol. xx, n. x
Software Quality Assessment using Flexibility: A Systematic Literature Review Hema Subramaniam1, Hazura Zulzalil 2,
Abstract – Software flexibility has been viewed as one of the software quality determinant factors apart from usability, reliability and maintainability. Software expected to achieve high level of flexibility in order to comply with current dynamic nature of user requirement. However, there is no effort on systematically collect, review and synthesize article that relate to this topic. This trigger a need for a systematic literature review whereby it covers software flexibility concept in the context of software quality assessment and various perspective of quantifying it. Literature search related to software quality and flexibility reveal 360 papers, and after further reading, 29 articles identified as primary paper. Traceability metrics and mind map used to locate the current research and resolved problem in the field of software flexibility. Information regarding software flexibility measurement in assessing software quality also been extracted through literature review. This paper serves as tool for software developer to anticipate the available software flexibility measurement method and tools in the process of assessing software quality. Keywords: Software quality assessment, flexibility measurement, extensibility, maintenance evaluation, systematic literature review
I.
Introduction
Software quality is one of the knowledge areas in Software Engineering body of knowledge that got a wide attention among software development industries nowadays. Due to increasing complexity of software requirement, software quality becomes a crucial thing to evaluate which motivate researcher to produce software quality models. Software quality assessment model define as analytical models that provide quantitative assessment of selected quality characteristic or subcharacteristic based on measurement data form software projects [1]. Software quality models play important role in assessing software quality from various perspectives. Most of the software quality models are created based on software quality determinant factors. Among the listed quality factors, there are number of quality factors which overlapping across the model. This indicates the importance of the characteristic in determining the software quality. Even though, each software quality models using different terms to describe the quality characteristics, it’s do refer to same characteristics. These review aim to investigate the usage of flexibility and its affiliated terms in assessing software quality. The various definition and understanding of flexibility covered in background and motivation section. Meanwhile, research methodology section outlines the searching strategy and ways to conduct systematic literature review. The outcome of the systematic literature review presented in traceability matrix format in results and discussion section.
Manuscript received January 2007, revised January 2007
II.
Background and Motivation
Flexibility is one of quality determinant factors as specified in the McCall quality model which is grouped under product revision perspective [1]. Bug fixes and complies with newly added requirement are among the reasons for a software to be released by versions. Versions control is also influenced by the flexibility of software. According to IEEE standard glossary, flexibility means “the ease with which a system or component can be modified for use in applications or environments other than those for which it was specifically designed” [2]. The word flexibility also referring to ability to modify the section/part that needs to be changed while the unchanged sections being remained [3]. Software flexibility can be referred in two perspectives; Software process flexibility and software product flexibility. Software process flexibility involve from the requirement phase throughout to implementation phase always need to adhere to the environment/business process changes. Software process flexibility becomes a mediator between software process standardization and project performance [4]. Meanwhile software product flexibility can be achieved by designing software that able to change from one state to another state. Software product flexibility always refers to requirements modification and software internal structure changes. There are wide varieties of affiliated terms associated with flexibility in software measurement and information system terminologies [5]. Even though the terms refer to
Copyright © 2007 Praise Worthy Prize S.r.l. - All rights reserved
S.Hema, Z.Hazura
software measurement, this terms also closely related to software quality assessment as well. Portability, extensibility, extendibility, adaptability are among the terms that identified by author that considered as affiliated terms of flexibility. Affiliated terms of flexibility help in refining the results for our literature search strategies. Different author refer to flexibility in different terms which give same meaning.
adaptability term combined with measurement, metrics and evaluation. TABLE I SEARCH TERM USED Section Affiliated Terms /Keyword
Section ID 1
Software Flexibility
2
Measurement
III. Research Method The method used to conduct this systematic literature review based on the approach proposed by Barbara Kitchenham in their article title systematic literature review in software engineering [6]. There are few steps proposed by author in this article that manage the literature in systematic manner to facilitate the finding of the most relevant information in the field of research. Steps to conduct systematic literature review including developing reviewing protocol, identification and selection of primary studies, data extraction and lastly reporting the results. The reported result is the main activity that leads to the knowledge gap tracking. Knowledge gap is important in research to identify the researchable problem. Research always starts with a problem and questions. Software flexibility research relies on the following research questions that need to be answered during the literature review process. Q1: What are the inter relation between available software quality models and what are the overlapping characteristic between the software quality models? Q2: What are the available measurements models on measuring software flexibility? Q3: Where is the flexibility point relying on software/application? Q4: Flexibility had been covered by previous research in how many perspective and what are they? Paper article searching strategy started with finding journal names that relate to software quality assessment and the relation of software flexibility. The identification of Journal name and their associated publisher is the initial step that helps in developing review protocol as mention by Barbara Kitchenham [6]. Article searching strategies become cumbersome as the improper searching strategies reveal many searching results and give difficulties in identifying and selecting primary studies. Adaptability, expendability, extendibility, extensibility and portability have been identified as affiliated terms of flexibility [5]. Searching strategy divided into 2 sections; flexibility and measurement. Error! Reference source not found. shows the search terms sections and it’s affiliated terms. Affiliated terms play an important role in locating the paper, as different author using different term but pointing to same meaning. Searching has been conducted in cross section manner. For example
Copyright © 2007 Praise Worthy Prize S.r.l. - All rights reserved
Flexibility Adaptability Expandability Extendibility Extensibility Portability Measurement Metrics Evaluation
Our searching strategies have revealed 360 papers that associated with software flexibility and software quality assessment. There is certain filtration process used in the process of finding the primary paper. Error! Reference source not found. shows the stage by stage filtration process that help in displaying primary result that closely related to software flexibility measurement towards achieving software quality assessment. As an initial stage article searched by keyword in a cross section manner using available electronic databases such as IEEE, ACM, Wiley and etc. 360 articles have been identified using these searching strategies which later refined by article publication year. Article which is published in latest 5 years used to locate the latest finding on the field of research. Search result displayed 100 articles from various journals and conference proceeding which then refine to dedicated journal and conference proceeding as stated in Table 1. First level reading has been conducted in order to obtain primary article that reflects the idea of software flexibility measurement in measuring the software quality assessment.
Fig. 1. Primary paper filtration process flowchart
IV.
Result and Discussion
Twenty nine primary articles or also known as anchor papers used in identifying the knowledge gap in software quality assessment. The result of reading represented in two illustration format; mind map and traceability matrix. Both representation help in identifying the area of research discussed in the paper. Based on literature review, roadmap as illustrated in Error! Reference
International Review on Computers and Software, Vol. xx, n. x
S.Hema, Z.Hazura
source not found. produced which consist of area that related to software flexibility. The area includes the discussion or interested area on software flexibility process, measurement method, concept and terminology used in software flexibility, management and evaluation and assessment method of software flexibility. Anchor papers that identified through filtration process, become the major reference to produce the following roadmap. Due to the author limitation, article filtration activity and reviewing process only involve authors of the paper instead of group of reviewers. Research questions that identified earlier have been answered after the implementation of review process. Interrelation between quality models and their overlapping characteristic been identified by anticipating the meaning of affiliated terms. There is variety of software flexibility models has been identified using traceability matrix. However, the flexibility point that accommodate in the software/application is less discussed by authors. There are 5 perspectives (as shown in Error! Reference source not found.) that related to flexibility measurement in relation with software quality assessment being derived from the literature review activity.
[4]
J.Y. Liu, V.J. Chen, C. Chan, and T. Lie, "The impact of software process standardization on software flexibility and project management performance : Control theory perspective," Journal of Systems and Software, vol. 50, 2008, pp. 889-896.
[5]
O.T. Pusatli and S. Misra, "A Discussion on IS and Software Measurement Terminology Flexibility as an Example," Internation Conference on Computation Science and Its Applications, IEEE, 2010, pp. 250-254.
[6]
B. Kitchenham, O.P. Brereton, D. Budgen, M. Turner, J. Bailey, and S. Linkman, "Systematic literature reviews in software engineering – A systematic literature review," Information and Software Technology, vol. 51, 2009, pp. 715.
[7]
J. Niu, L. Shen, S. Peng, and F. Li, "A Measurement Method of Software Flexibility Based on BP Network," International Workshop on Intelligent Systems and Applications, 2009, pp. 1-4.
[8]
D.D. Zeng and J.L. Zhao, "Achieving Software Flexibility via Intelligent Workflow Techniques," Proceeding of the 35th Annual Hawaii International Conference on System Sciences, IEEE, 2002, pp. 606- 615, 7-10.
[9]
S. Peng, L. Shen, H. Liu, and F. Li, "User-Oriented Measurement of Software Flexibility," 2009 WRI World Congress on Computer Science and Information Engineering, 2009, pp. 629-633.
[10]
V. Stoicu-Tivadar and L. Stoicu-Tivadar, "Software Flexibility Improvement for Document Oriented Applications," Health (San Francisco), 2009, pp. 21-26.
[11]
K. Rautiainen, P.O. Box, L. Vuornos, and C. Lassenius, "An Experience in Combining Flexibility and Control in a Small Company ’ s Software Product Development Process," Empirical Software Engineering, 2003.
[12]
B.P. Barbosa, E.A. Schmitz, and A.J. Alencar, "The Case for Managerial Flexibility in MMF-based Software Development Projects," 2008, pp. 114-115.
[13]
Y. Wang, M. Jia, J. Guo, and B. Zhang, "Evaluating Model of Software Flexibility of Domestic Foundational Software," International Conference on Electrical and Control Engineering, 2011, pp. 5906-5909.
[14]
N.M. Villegas, G. Tamura, L. Duchien, and R. Casallas, "A Framework for Evaluating Quality-Driven Self-Adaptive Software Systems," Proceedings of the 6th International Symposium on Software Engineering for Adaptive and SelfManaging Systems (SEAMS '11), New York: ACM, 2011, pp. 80-89.
[15]
A. Kumar, U. Pradesh, and P.S. Grover, "A Quantitative Evaluation of Aspect-Oriented Software Quality Model ( AOSQUAMO ) ACM SIGSOFT Software Engineering Notes," Software Engineering Notes, vol. 34, 2009, pp. 1-9.
[16]
S.K. Dubey and A. Rana, "Assessment of Maintainability Metrics for Object-Oriented Software System," Software Engineering Notes, vol. 36, 2011, pp. 1-7.
[17]
C. Babu, R. Vijayalakshmi, S.S. College, and O. Engineering, "Metrics- Based Design Selection tool for Aspect Oriented Software Development," Software Engineering Notes, vol. 33, 2008.
Fig. 2. Software Flexibility Area
Appendix Refer to appendix A for traceability metrics on software quality assessment using flexibility
References [1]
D. Samadhiya, S. Wang, and D. Chen, "Quality Models : Role and Value in Software Engineering," Technology Management, 2010, pp. 320-324.
[2]
A. September, "IEEE Standard Glossary of Software Engineering Terminology," Office, vol. 121990, 1990.
[3]
R. Martinho and D. Domingos, "A Two-Step Approach for Modelling Flexibility in Software Processes," 23rd IEEE/ACM International Conference on Automated Software Engineering, 2008, pp. 427-430.
Copyright © 2007 Praise Worthy Prize S.r.l. - All rights reserved
International Review on Computers and Software, Vol. xx, n. x
S.Hema, Z.Hazura
[18]
W. Royce, "Software Development Models of Code ( SLOC ).," Development.
[19]
S. Barrett and Y. Khattak, "Primitive Components : Towards more flexible Black Box AOP," Proceedings of the 1st International Workshop on Context-Aware Middleware and Services: affiliated with the 4th International Conference on Communication System Software and Middleware, ACM, 2009, pp. 24-30.
[20]
D. Perez-palacin, P. Milano, and D. Elettronica, "Software Architecture Adaptability Metrics for QoS-based," Proceedings of the joint ACM SIGSOFT conference -- QoSA and ACM SIGSOFT symposium -- ISARCS on Quality of software architectures -- QoSA and architecting critical systems -- ISARCS (QoSA-ISARCS '11), New York: ACM, , pp. 171-175.
[21]
Y. Singh, P.K. Bhatia, and O. Sangwan, "Software reusability assessment using soft computing techniques," Software Engineering Notes, vol. 36, 2011.
[22]
E. Reisner, C. Song, K. Ma, J.S. Foster, and A. Porter, "Using Symbolic Evaluation to Understand Behavior in Configurable Software Systems," Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering, New York: ACM, , pp. 445-454.
[23]
Q.P. Thi, D.T. Quang, and T.H. Quyet, "A Complexity Measure for Web Service," 2009, pp. 226-231.
[24]
S. Wang and X. Liu, "A Study on Flexibility of ERP System Based on Grey Evaluation Model," Technology, 2010, pp. 36.
[25]
M. Rauterberg, "An Empirical Validation of Four Different Measures to Quantify User Interface Characteristics Based on a General Descriptive Concept for Interaction Points," 1996.
[26]
J.H. Yahaya, "Measuring Unmeasurable Attributes of Software Quality Using Pragmatic Quality Factor," Methodology.
[27]
J. Osbeck and P. Roden, "INVESTIGATION OF AUTOMATIC PREDICTION OF SOFTWARE QUALITY," Meta, 2011.
[28]
X. Wang, Y. Bai, and C. Cai, "A Production Rule-based Knowledge System for Software Quality Evaluation," Engineering and Technology, 2010, pp. 208-211.
[29]
W. Xiao-jun, "Metrics for Evaluating Coupling and Service Granularity in Service Oriented Architecture," Measurement, 2009.
[30]
F. Mårtensson, H. Grahn, M. Mattsson, P.O. Box, and S. Ronneby, "Evaluating Software Quality Attributes of Communication Components in an Automated Guided Vehicle System," Engineering, 2005.
[31]
P. Qualitative and A. Using, "Predicting Qualitative Assessments Using Fuzzy Aggregation," Computer Engineering, pp. 267-272.
[32]
P. Warren, C. Gaskell, and C. Boldyreff, "Preparing the Ground for Website Metrics Research," World Wide Web Internet And Web Information Systems, 2001.
Copyright © 2007 Praise Worthy Prize S.r.l. - All rights reserved
Authors’ information Hema Subramaniam is a PhD student at the faculty of Computer Science and Information Technology, Putra University of Malaysia. She holds a Master Degree in Software Engineering from Selangor University (UNISEL). Her research interest includes software quality and software measurement.
Hazura Zulzalil holds a PhD from Putra University of Malaysia. She holds a Master Degree in Software Engineering from UPM. Her research interest includes software quality and quality for web based application.
International Review on Computers and Software, Vol. xx, n. x
S.Hema, Z.Hazura
Copyright © 2007 Praise Worthy Prize S.r.l. - All rights reserved
International Review on Computers and Software, Vol. xx, n. x
International Review on Computers and Software (I.RE.CO.S.), Vol. xx, n. x
Appendix A: Traceability matrix on software quality assessment using software flexibility Paper Reference Process Model Measurement metrics Measurement Model Evaluation / Assessment Management and concept Paper Reference Process Model Measurement metrics Measurement Model Evaluation / Assessment Management and concept
1: [3] ×
2: [7]
3: [8] ×
4: [9]
5: [10]
6: [4]
7: [11] ×
8: [5]
9: [12] ×
10: [13]
11: [14]
12: [15]
× ×
20: [23]
×
21: [24]
22: [25]
23: [26]
×
24: [27]
×
×
28: [31]
29: [32]
15: [18]
16: [19]
×
×
17: [20]
18: [21]
× ×
×
×
× 25: [28]
×
26: [29]
27: [30]
×
×
× ×
14: [17]
×
× 19: [22]
13: [16]
× ×
×
×
×
Manuscript received January 2007, revised January 2007
Copyright © 2007 Praise Worthy Prize S.r.l. - All rights reserved
