Available Available online online at at www.sciencedirect.com www.sciencedirect.com
ScienceDirect
Available online at www.sciencedirect.com Procedia Procedia Computer Computer Science Science 00 00 (2016) (2016) 000–000 000–000
ScienceDirect
www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia
Procedia Computer Science 109C (2017) 725–730
The 8th International Conference on Ambient Systems, Networks and Technologies (ANT 2017)
Towards a New Framework of Software Reliability Measurement Based on Software Metrics Dalila Amara, Latifa Ben Arfa Rabai** SMART SMART Laboratory, Laboratory, University University of of Tunis, Tunis, Institut Institut Supérieur Supérieur De De Gestion, Gestion, Tunis, Tunis, Tunisia Tunisia
Abstract Abstract Measuring Measuring software software reliability reliability is is one one of of the the most most important important concerns concerns for for software software engineers engineers and and decisions decisions makers makers since since if if we we can’t can’t measure measure it it we we can’t can’t master master it. it. For For engineers, engineers, the the system system reliability reliability level level can can be be used used to to measure measure the the progress progress of of system system testing testing by by comparing comparing the the current current software software failure failure intensity intensity with with the the desired desired one. one. For For decision decision makers,the makers,the system system reliability reliability level level Guarantee Guarantee better better exploitation exploitation by by the the users users therefore therefore better better customer customer satisfaction. satisfaction. A A literature literature review review shows shows aa lack lack of of studies studies focusing focusing on on the the reliability reliability measurement measurement processes processes related related to to software software development development life life cycle cycle (SDLC). (SDLC). In In this this paper, paper, our our intention intention is is to to propose propose aa thoroughly thoroughly analysis analysis of of the the software software reliability reliability measurement measurement processes. processes. Since Since the the current current software software measurement measurement trends trends are are focusing focusing on on software software metrics, metrics, we we will will propose propose aa new new framework framework of of reliability reliability measurement measurement based based on on software software metrics. metrics. This software reliability reliability and and by by decision decision makers makers to to make make appropriate appropriate This new new framework framework is is required required by by developers developers to to assess assess their their software quality quality improvements. improvements. © 2016 The Published by Elsevier B.V. © 2016 The©Authors. Authors. Published Elsevier by B.V. 1877-0509 2017 The Authors.by Published Elsevier B.V. Peer-review under responsibility of the Conference Program Chairs. Peer-review under responsibility of the Conference Peer-review under responsibility of the Conference Program Program Chairs. Chairs. Keywords: Keywords: reliability reliability measurement; measurement; reliability reliability metrics; metrics; reliability reliability models; models; reliability reliability techniques; techniques; reliability reliability measurement measurement process. process.
1. 1. Introduction Introduction Software Software measurement measurement is is an an emerging emerging field field of of software software engineering engineering that that helps helps with with assessing assessing the the quality quality of of software systems. Quality in these systems is described by different attributes that may be internal such software systems. Quality in these systems is described by different attributes that may be internal such as as size size 1, 2 3, 5 6 and or external external such such as as reliability reliability and and portability portability3, 5.. Among Among these these attributes, attributes, Farooq Farooq et et al. al.6 show show and complexity complexity1, 2 or that that reliability reliability is is one one of of the the most most important important quality quality factors factors and and it it is is the the only only common common factor factor of of different different quality quality 4 models models i.e i.e McCall’s, McCall’s, Boehm’s, Boehm’s, FURPS FURPS and and ISO ISO 25010 25010 that that replaced replaced the the ISO ISO 9126 9126 4.. * * Latifa Latifa Ben Ben Arfa Arfa Rabai. Rabai. Tel.: Tel.: +216-98-385-392 +216-98-385-392 ;; fax: fax: +0-000-000-0000 +0-000-000-0000 .. E-mail address: E-mail address:
[email protected] [email protected] 1877-0509 1877-0509 © © 2016 2016 The The Authors. Authors. Published Published by by Elsevier Elsevier B.V. B.V. Peer-review Peer-review under under responsibility responsibility of of the the Conference Conference Program Program Chairs. Chairs.
1877-0509 © 2017 The Authors. Published by Elsevier B.V. Peer-review under responsibility of the Conference Program Chairs. 10.1016/j.procs.2017.05.428
726 2
Dalila Amara et al. / Procedia Computer Science 109C (2017) 725–730 Author name / Procedia Computer Science 00 (2015) 000–000
Reliability measurement covers different elements like reliability techniques, models and metrics. These elements are widely discussed in the literature. For example, Mansour and Mustafa7 present numerous syntactic metrics that may be indirectly used for reliability measurement. Moreover, Nikora and Lyu8 present different reliability models and syntactic metrics for quality assessment. Additionally, Etzkorn and Delugach9, Stein et al.10 and Mili et al.11 propose different semantic metrics to assess software reliability. However, to the best of our knowledge, little research has been focused on the reliability measurement processes. The main process based only on reliability models, was firstly proposed in 1996 by Nikora and Lyu8 and then discussed in 2012 by Farooq et al.6. This process ignored software metrics through the software development life cycle (SDLC). Hence, our intention in this paper is to propose a more exhaustive reliability measurement process covering not only the reliability models steps but also reliability techniques and metrics. This leads researchers to a better view of software reliability measurement. This paper is structured as Follows. We introduce in section 2 the reliability as a quality factor. In section 3, we present the fundamentals of reliability measurement. In section 4, we describe the basic reliability measurement framework and we give its advantages and shortcomings. In section 5, we propose a new framework and we discuss its advantages regarding to the basic one. Finally, we conclude and give some perspectives. 2. Reliability as a Quality Factor Software quality is described through a number of attributes and aims to identify their capability to achieve the required system specifications31. These attributes are classified into two internal and external categories3, 31. Internal attributes concern the developers of the system and can be directly measured from the program syntax5 and include size1, 14, complexity1, cohesion1, 15, reuse1, coupling7, etc. External attributes are visible to the users of the system and include reliability, maintainability, etc5, 13. The distinction between internal and external qualities is not salient since internal qualities help developers to achieve external ones 31. External attributes are organized by different quality models including McCall’s, Boehm’s, FURPS, Dromey and ISO 250102, 4. Reliability is an external software quality attribute defined by the ISO/IEC 25010:2011 4 and by the ISO/IEC/IEEE 24765:201017 as the degree to which a system, product or component performs specified functions under specified conditions for a specified period of time. In terms of statistical behavior, reliability is the probability that the software product operates for a given amount of time without violating its specification 30, 31 . In terms of the severity of failures, software reliability is different from other engineering disciplines products i.e manufacturing since we can deliver reliable software despite design errors 31. 3. Software reliability measurement fundamentals Reliability measurement consists on using mathematical and statistical models, techniques and metrics to quantitatively assess and to predict the software reliability6, 18, 20. Reliability modeling identifies the dependency between the failure process and the principal factors affecting it using statistical methods in the operation and testing phases6, 13. These models estimate the system reliability level and are divided into two main categories which are time between failure models and Fault-Count Models8, 21. Reliability techniques are used to minimize the fault occurrence and the negative effects of software failures. They are classified into four categories which are fault prevention, fault removal, fault tolerance and fault forecasting 8, 12, 23. Reliability metrics help to estimate the current reliability and to predict its future behaviour. These metrics are classified on syntactic and semantic groups. Syntactic metrics are proposed as a quantitative measure to assess the quality of the source code and to give information about their attributes such as size, complexity and cohesion3, 5, 11. Data used to compute these metrics are collected from the code such as the number of methods in the class, the number of attributes, etc. To collect these data different tools can be used24. Examples of syntactic metrics Chidamber and Kemerer (C&K) suite9. Semantic metrics are proposed as a quantitative measure to assess the semantic aspect of the source code from for example its design specification independently of its syntax11, 25.
Dalila Amara et al. / Procedia Computer Science 109C (2017) 725–730 Author name / Procedia Computer Science 00 (2015) 000–000
727 3
They are used to assess different quality attributes like cohesion and complexity15. To compute these metrics, techniques like knowledge based program understanding may be used26. The most important semantic metrics proposed in the literature are Etzkorn and Delugach suite 9, Stein et al. suite 25 and Mili et al. suite 11. 4. Focus on software reliability process Nikora and Lyu in 1996 proposed a process for reliability measurement. It consists on a set of iterative activities during system test and post delivery until achieving a certain reliability level8. 4.1. Major phases of the basic process Nikora and Lyu process consists of four major steps presented in Fig. 1. These steps are: Define reliability objective: the objective is to quantify the required level of the product’s reliability by comparing estimated resources with available ones6. It can be defined based on existing products having the same customer base, or by defining failures and their severities, the cost and the delivery date in order to help users and developers identifying unreliable components and increasing the productivity8. Develop operational profile: operational profile is a quantitative characterization of how a system will be used. It is a set of disjoint operations and their associated probabilities of occurrence 6. It helps to manage resources and to guide each step of SDLC. This step consists of four other steps. First, determining the customer profile which is the probability of using the system by the customer that can be a person, a group or an institution27. Second, defining the user profile which is the probability occurrence of the user group which is a set of users that employ the system. A user may be a person, group, or institution that employs the system27. Third, defining the system-mode profile that consists of a set of functions or operations in order to analyze the execution behavior. Different modes can exist simultaneously and only one mode can be in effect at a time8, 27. Fourth, determining the functional profile step consists on identifying functions needed for each system mode, creating a function list and then determining their probability of occurrence. Model and measure reliability: this step covers perform system testing, collect failure data, apply software reliability tools, select appropriate reliability models, use reliability models to estimate current reliability, verifying if the reliability objective is made and continue testing or start to deploy steps as presented in Fig.1. R eliability modeling is based on failure data collected during tests and operation that include times between successive failures (execution or testing times) and the number of failures during each test interval8. Validate Reliability in the field: once reliability estimates are obtained, the projected reliability must be compared to the observed one in order to verify if the required level is achieved. This step helps providing feedbacks that may enhance the process, establish the validation of the model and determine the growth of reliability 27.
Fig. 1. The basic reliability measurement process.
728 4
Dalila Amara et al. / Procedia Computer Science 109C (2017) 725–730 Author name / Procedia Computer Science 00 (2015) 000–000
4.2. Advantages and limits of the basic reliability measurement process The basic framework presented above is an iterative approach used for reliability measurement. It defines the reliability objective by identifying the failures’ severities, the cost and the delivery date and cost. It helps developers and users to identify components that have the most effect on the system. The development of an operational profile helps to manage resources and to guide software design, implementation and testing. Additionally, reliability models are needed to verify if the reliability objective is achieved. Finally reliability validation encourages developers to enhance the development of their software using resulted feedback. Based on what has been presented, we observed that the basic process measurement focus only on reliability models to estimate reliability in testing and operation phases of SDLC. However, the key elements of reliability measurement cover models, techniques and metrics that can be applied from the requirement phase. 5. The Proposed Realiability Measurement Framework Based on shortcomings presented above, we found that the basic process needs to be updated in order to represent how reliability techniques and metrics can be used in this process throughout different phases of SDLC. The objective is to produce reliable software rather than be limited to assess the reliability of the completed product. To this end, we propose in this section a new process in which reliability techniques and metrics are integrated from requirement to testing phase in a such SDLC. 5.1. Major phases of the proposed process The new added steps are designed by colored states in Fig. 2 and subdivided into two main phases: The Requirement, Design and Coding phase: this phase include different steps as presented in Fig. 2. The design activity that this activity decomposes a system into modules and it is a fundamental phase of software development process and it is required by engineers to achieve the desired qualities including reliability 31 . In this phase different reliability techniques and metrics can be applied. For example, techniques as fault prevention can be used in the requirement phase to develop and to maintain the required specifications 8,12 . For software metrics, since faults have widely varying impacts on product reliability30 , Kumar and Misra32 in 2008 developed a model to predict reliability in the early SDLC phases based on software engineering metrics. Moreover, in 2009, Pandey and Goyal33 have developed an early fault prediction model based on software metrics that predicts the number of faults present before testing. Once reliability techniques and metrics are applied, designers have to check if the required reliability level defined in the reliability objective and the operational profile steps is achieved. If it is so, it will be possible to move to the implementation and the testing phases. Otherwise, this activity may be repeated until obtain the required reliability level. The implementation and testing phase: this phase consists on using reliability models, semantic and syntactic metrics in the testing phase as in the basic process. Testing can be used to estimate the fault density and the reliability of the program 30 . It is also an effective method for estimating and predicting reliability 34 . In this step different reliability models 22, 21 and techniques 8,12 can be applied as in the basic process. Moreover, software syntactic and semantic metrics and reliability techniques can be used to quantify some aspects of software testing. For example, fault tolerance techniques i.e N -version programming reference and recovery block programming 23 are needed in testing phase. Other new testing techniques proposed to assess and to improve software reliability are discussed by Lv et al 35 . For reliability metric, the most commonly used metric to quantify reliability in testing phase is the mean time to failure (MTTF) 8, 30 . Other metrics that can be used are discussed by Mili and Tchier 30 . Reliability validation: this step consists on verifying if the reliability objective is achived. If that’s the case, the product is released, otherwise, these steps are repeated until achieving appropriate reliability level. 5.2. Advantages and limits of the proposed process The proposed framework is an exhaustive description of how and when using the key elements of reliability
Dalila Amara et al. / Procedia Computer Science 109C (2017) 725–730 Author name / Procedia Computer Science 00 (2015) 000–000
729 5
measurement throughout the SDLC. The use of reliability models and techniques from the requirement phase will improve reliability by developing required specification and avoiding unneeded tests. Hence, appropriate corrections and improvements will be done with lesser cost and time. Additionally, in testing phase, the parallel use of software metrics, reliability models and techniques enhance the measurement of reliability and minimize the cost and effort needed for corrections and improvements. Finally, rather focusing only on describing fundamentals of reliability measurement i.e models, techniques and metrics, we have to identify their use in each phase of the SDLC. This will help developers to ensure the reliability level of their systems in each phase of SDLC. However, the validation of this process using data from real projects is required.
Fig. 2. The proposed software reliability measurement process.
6. Conclusion and Perspectives In this paper, we have illustrated fundamentals of reliability measurement in the software engineering context. We stated that reliability measurement covers techniques, models and metrics. We also observed that the existing reliability measurement process focused only on the use of reliability models in the measurement step in the testing phase. However, reliability measurement as described above includes models, tools, techniques and metrics which can be applied earlier in the SDLC. Hence, the proposed framework covers all of these elements and shows their possible use in the different phases of SDLC. Based on what has been presented, we target to more focus on the possible use of this process in real applications, we intend to use some benchmarks in order to evaluate and validate our framework. References 1. Fenton NE, Pfleeger SL. Software metrics: a rigorous & practical approach. International Thomson Computer Press. 2nd ed 1996. 2. Helali RGM. A Comparison between semantic and syntactic software metrics. International Journal of Advanced Research in Computer and Communication Engineering 2015. 4:1–5.
730 6
Dalila Amara et al. / Procedia Computer Science 109C (2017) 725–730 Author name / Procedia Computer Science 00 (2015) 000–000
3. Saini N, Kharwar S, Agrawal A. A Study of significant software metrics. International Journal of Engineering Inventions 2014; 3: 1-7. 4. ISO/IEC CD 25010 : Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — System and software quality models, 2011. 5. Harrison R, Counsell, Nithi RV. An evaluation of the MOOD set of object-oriented software metrics. IEEE Trans. Softw. Eng, 1998. 24: p. 491–496. 6. Farooq SU, Quadri SMK., Ahmad N. Metrics, models and measurements in software reliability. IEEE 10th International Symposium on Applied Machine Intelligence and Informatics (SAMI). Slovakia; 2012. p. 441–449. 7. Mansour YI, Mustafa SH. Assessing internal software quality attributes of the object-oriented and service-oriented software development paradigms: a comparative study. Journal of Software Engineering and Applications, 2011, 4:p. 244–252. 8. Lyu MR. Handbook of software reliability engineering. Computer Society Press and McGraw-Hill Book Company. New York, USA; 1996. 9. Etzkom L, Delugach H. Towards a semantic metrics suite for object-oriented design. 34th International Conference on Technology of ObjectOriented Languages and Systems. California, USA; 2000. p. 71–80. 10. Stein C, Etzkorn LH., Cox G, Farrington Ph.A, Gholston S, Utley DR., Fortune J. A new suite of metrics for object-oriented software. Proceedings of the 1st International Workshop on Software Audits and Metrics. Portugal; 2004. p. 49–58,. 11. Mili A, Jaoua A, Frias A, Helali RGM. Semantic metrics for software products. Innovations in Systems Software Engineering. 2014, 10: p.203–217. 12. Lyu M.R. Software reliability engineering: a roadmap. 29th International Conference On Software Engineering, Future of Software Engineering. Minneapolis; 2007. p. 153–170. 13. Fenton N.E. Software measurement: a necessary scientific basis. IEEE Trans. Softw. Eng. 1994: 20, p. 199–206. 14. Rosenberg L., Hammer T., and Shaw J. Software metrics and reliability. (ISSRE 1998 Best Paper), 9th International Symposium on Software Reliability. Germany; 1998. p. 1–8,. 15. Gall C.S., Lukins S., Etzkorn L.H., Gholston S., Farrington Ph.A., Utley D.R., Fortune J. and Virani S. Semantic software metrics computed from natural language design specifications. IET Software. 2008: 2, p. 17–26. 16. Kaur G., and Bahl K. Software reliability, metrics, reliability improvement using Agile process. International Journal of Innovative Science Engineering & Technology. 2014: 1, p. 143–147. 17. ISO/IEC/IEEE 24765: Systems and software engineering, 2010. 18. Tiwari V., and Pandey P.K. Open Source Software and Reliability Metrics. International Journal of Advanced Research in Computer and Communication Engineering. 2012: 1, p. 808–815. 19. Hecht H. Measurement, estimation and prediction of software reliability. National Chapter Conf., in Software Engineering Technology. England; 1977: 2, pp. 209–224. 20. Wohlin C., Host M., Runeson P., and Wesslén A. Software reliability. Encyclopedia of Physical Sciences and Technology (third edition), Academic Press, 2001. 21. Goel A.L. Software reliability models: assumptions, limitations, and applicability. IEEE Trans. Softw. Eng. 1985: 12, p. 1411–1423. 22. Jatain A., Mehta Y. Metrics and models for software reliability: a systematic review. International Conference on Issues and Challenges in Intelligent Computing Techniques (ICICT). India; 2014. p. 210–214. 23. Jaoua A., and Mili A. The use of executable assertions for error detection and damage assessment. Journal of Systems and Software. 1990: 12, p. 15–37. 24. Lincke R., Lundberg J., and Löwe W. Comparing software metrics tools. Proceedings of the 2008 international symposium on Software testing and analysis. New York; 2008. p. 131–142. 25. Stein C., Etzkorn L.H., Gholston S., Farrington Ph.A., Utley D.R., Cox G., and Fortune J. Semantic metrics: metrics based on semantic aspects of software. Applied Artificial Intelligence: An International Journal. 2009: 23, p.44–77. 26. Cox G.W., Gholston S.E., Utley D.R,, Etzkorn L.H., Gall C.S., Farrington Ph.A., and Fortune J. Empirical validation of the RCDC and RCDE semantic complexity metrics for object-oriented software. Journal of Computing and Information Technology. 2007: 15. p. 151–160. 27. Musa J.D. Operational Profiles in Software-Reliability Engineering. IEEE Software. 1993: 10, p. 14–31. 30. Mili A., Tchier F. Software Testing: Concepts and Operations, 2015. 31. Ghezzi C, Jazayeri M, Mandrioli D, Fundamentals of Software Engineering, 2nd Edition 32. Kumar, K. S., & Misra, R. B. (2008). An enhanced model for early software reliability prediction using software engineering metrics. In Proceedings of 2nd International Conference on Secure System Integration and Reliability Improvement (pp. 177–178) 33. Pandey, A. K., & Goyal, N. K. (2009). A fuzzy model for early software fault prediction using process maturity and software metrics.International Journal of Electronics Engineering, 1(2), 239–245. 34 Farooq S.U, Quadri S.M.K. Evaluating Effectiveness of Software Testing Techniques With Emphasis on Enhancing Software Reliability. Journal of Emerging Trends in Computing and Information Sciences VOL. 2, NO. 12, December 2011 35. Lv J, Yin B, Cai KY. On the asymptotic behavior of adaptive testing strategy for software reliability assessment. IEEE transactions on Software Engineering 40(4), 2014
