Implementation of Software Testing Practices in Pakistan’s Software Industry Bilal Maqbool, Fateh Ur Rehman, Muhammad Abbas, Saad Rehman Department of Computer & Software Engineering, College of E&ME, National University of Sciences and Technology (NUST), Islamabad, Pakistan +92-343-5642265, +92-332-6736292, +92-332-9244006, +92-51-9278050
[email protected],
[email protected],
[email protected],
[email protected]
but guaranteeing the quality of the product is important and crucial part as well as it is a tough task. Quality should be assured of the product that is developed using agile methodology [5]. For the effective software engineering, the transfer of great quality software is required. Primarily the client satisfaction level is considered as the quality attribute [6]. Software testing identifies problems in software and assures that delivered software is error, bug, and fault free [7].
ABSTRACT Agile methodology uses the incremental and iterative method and is commonly utilized in the Pakistan’s industry projects as they can accommodate changes in requirements. Product distribution is accomplished by using small iterations/repetitions, but guaranteeing the quality of the product is important and crucial part as well as it is a tough task. Quality should be assured of the product that is developed using agile methodology. The study centers on the five key parts of software testing, explicitly software testing methods, software testing metrics, practices and techniques, testing standards, automated testing tools, and testing education & training. Grounded on survey outcomes, research paper evaluates the implementation of existing practices in the software testing, provide some recommendations and observations for the software testing future in Pakistan IT industry & also suggested the solution that how quality is assured in agile software development using different factors.
Reasons for doing this survey are: Firstly, the cost of the software testing is flattering a large ratio of the cost of software product construction budget [8]. Furthermore, according to the researchers’ view, in future quality of software will become a vital factor in IT industry. Software testing policies and strategies become more significant. An effective examination will help in identifying the best practices and strategies [9].
CCS Concepts • Software and its engineering → debugging
Thirdly, the survey can help to indicate forthcoming research area and guidelines [10].
Software testing and
Fourthly, comparison and assessment of studies will contribute to many companies to recognize & classify optimal software testing approaches and advance software quality [11].
Keywords Software Quality Assurance (SQA); Quality Control; Software Testing; Software Engineering (SE)
Finally, the outcomes will offer direction for the training and education of software engineers, testers and developers [12].
1. INTRODUCTION
The finding described in the research study are established on 21 effectively responded questionnaire. The outcomes from exploring these 21 responses show the trends of usage of software testing practices in Pakistan’s IT industry. Regardless of the small number of sample data in the analysis, the reliability and constancy of the information got increased our self-assurance to account these findings in the paper.
Software Quality Assurance (SQA) is consist of tasks and actions for certifying and confirming the quality of a software product [1]. Software Quality Control is consist of different tasks designed and developed to assess a developed working product [2]. Software testing is a practice of running applications or programs to find defects in it [3]. The agile methodology uses the incremental and iterative method and is commonly utilized in the Pakistan’s industry projects as they can accommodate changes in requirements [4]. Product distribution is accomplished by using small iterations/repetitions,
2. LITERATURE REVIEW A number of studies and researchers have been carried out in Pakistan to find out software testing trends in IT industries. Following works has been studied and reviewed:
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from
[email protected]. ICMSS 2018, January 13–15, 2018, Wuhan, China © 2018 Copyright is held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 978-1-4503-5431-8/18/01…$15.00
Hafsa Shareef Dar et al. (2014) explains that static testing will improve the developer output and efficiency in agile methodology and facilitate users to improve confidence in quality-end product [13]. Another research paper identified SQA problems and their resolutions are proposed to handle those difficulties & enhance the quality of a software product [14]. To avail, a reputable place in International Marketplace software companies, need to essence on
DOI: https://doi.org/10.1145/3180374.3181340 147
the quality of software products. Software Quality Assurance has a very significant part because it can increase customer satisfaction.
Figure 2. Iterative Testing Model Agile-style software testing model excludes the part of the predetermined flow from the software test cycle. E.g., while examining the outcomes of a test, the tester may recognize that the test was defective and want to go back to planning and designing phases of tests. In other testing models, it is not possible. Agilestyle software testing can be applied as a compliment or as an overall approach to any other software testing approach. Agilestyle model is showed in Figure 3.
According to Nuzhat Naqvi et al., they discussed a model for SQA to decrease problems & to use this model to enhance the quality of software products [15]. About all software companies admitted that they are lacking software quality in their products. Ming-Chang Lee mentioned software quality metrics effort on software quality aspects of product, process, and project [16] in his research paper. He group into six categories in accordance with the software life cycle. To understand the association of standards of software quality factor, he has discussed software quality standard, model, factors, criteria, and metrics. Asim Iftikhar and Sheikh Muhammad Ali investigate and highlights the role and importance of software quality management techniques and practices in the software IT industry of Pakistan [17]. They discussed the evaluation of the more-expert and less-expert companies in the area of quality management critical factors. Results explain that the “use of software” & “age of quality” had limited effect on the quality management of software products.
3. FRAMEWORK Figure 3. Agile Testing Model
Software testers can use several types of software testing models when testing a software product. There are many software-testing models: iterative, waterfall and agile styles models [18].
4. RESEARCH METHODOLOGY Survey results reveal that the corporations are implementing agile methodologies, especially Scrum. Questionnaires indicated there are some issues. The advantages and issues of effective implementation of agile practices are explained in detail in this research paper.
Testing waterfall-style is same as developing a software product using the waterfall model; it is a linear process excepting the least feedback loop. Waterfall style model is showed in Figure 1.
Participants involved in the research have different roles and responsibilities in the organization; the majority were Senior Software Engineers; details are presented below in Figure 1. This research aims to determine different kinds of software testing tools, techniques, standards, certification and metrics that companies use in Pakistan to perform software-testing activities and to give a brief overview about the best practices used in IT industry [19], [20]. Another aim of this research is to find whether present training in software testing trained in the offices fulfill the kinds of testing skills and methodologies that the currently followed in the industry [12]. Established on these goals, few suppositions are engaged to develop the questions & outline the path of the study. Figure 1. Waterfall Testing Model
The study focused experienced staffs engaged with software testing activities in IT companies. Total Five key testing correlated areas practices were explored by the examination. In addition, organization structure and size, organization’s history and overall software development and testing processes with were assessed. Considering these areas and objectives, the survey was organized into six following segments:
Testing iterative-style is same as developing a software product using the iterative software development model, it is an iterative process accepting the most feedback loop and having a release after each iteration. Iterative style model is showed in Figure 2.
• • • • • •
148
Organization Data Software Testing Standards Software Testing Tools Software Testing Methodologies and Techniques Software Testing Standards and Metrics Software Testing Education and Teaching
Survey encompassed on both open-end and closed-end type questions. Survey meetings were done by telephonic interview, face to face and through e-mail attachments. For respondent’s flexibility, some were requested to fill the questionnaire online at Google Forms. Privacy and confidentiality were guaranteed to all participants.
2 6 10
In result, total 21 companies/individuals take part in the study. Many companies were asked to partake in this examination, a comparatively low rate of organizations takes part in this survey. However, the significance of survey was measured to be very essential, above 65% of participants had team leadership/managerial positions.
3
Banking, finance & insurance Research & development Software house & IT consultancy Telecommunications
5. EXPERIMENTAL DATA In the experiment of this survey, we assessed the accuracy of the software testing strategies and how much they are implemented in the Pakistan’s IT industries. We drew samples data from 21 organizations by doing a survey. We then analyzed the answers to see the differences in the gathered data.
Figure 5. Respondents grouped by industry
6. RESULTS & ANALYSIS
As stated earlier, research paper includes many research questions to get their answers in software testing area. The questionnaire was organized into six segments to get data.
The 3 famous approaches are static analysis, test case selection, and dynamic analysis. Black box testing is commonly used than white-box testing. May organizations adopted data flow analysis techniques. Nobody stated the use of symbolic analysis and rare companies described the use of mutation analysis.
21 organizations answered to our survey questionnaire, more than 57% are “overseas based private” organization area. 14.3% are from the government and 28.6% were local private organizations (Table 1). In addition, details are illustrated in Figure 4.
Researchers mainly concerned in respondents’ opinions to show the hurdles to implementing “software testing practices” in their organizations. Results after the questionnaires are showed in Table 3. In addition, details are illustrated in Figure 6.
Table 1. Respondents grouped by sector Sector Type Response %age Government
3
14.28571
Local private organization
6
28.57143
Overseas-based private organization
12
57.14286
Total
21
100
15 10 5 0
Table 3.Testing methodology adaption barriers
12 6
3 Government
Local private organization
Overseas-based private organization
Barrier
Response
Lacking expertise Do not think there is any barrier Lack of support tools Usage Difficulty Usage Cost Time-consuming to use Unaware of testing methodologies Do not think it is cost effective or useful Other Total
5 3 2 3 1 2 1 1 3 21
Figure 4. Respondents grouped by sector Do not think it is cost… Unaware of testing… Time-consuming to use Usage Cost Usage Difficulty Lack of support tools Do not think there is any… Lacking expertise
The popular business type of the companies was IT consultancies and software house (47.6%). Further businesses comprised insurance and finance, R&D, & telecommunication (Table 2). In addition, details are illustrated in Figure 5. Table 2. Respondents grouped by type of industry Type of Industry
Responses
%age
Finance, insurance & Banking
6
28.57142857
Research & development
3
14.28571429
Software house & IT consultancy
10
47.61904762
Telecommunications
2
9.523809524
Total
21
100
0
1 1 2 1 3 2 3 5 1
2
3
4
5
6
Figure 6. Testing methodology barriers In the previous 4 years, many organizations have automated their software testing tasks. A large number of companies have tools
149
for buying already developed tools, although only a few about 3 organizations developed their own tools. Among organizations, surveyed, computerized testing tools for regression testing, test execution, & test results reporting and examination are placed the top 3 places for automatic testing tasks. Additional accomplishments such as test planning & management and developing test scripts or cases also used by the responder organizations.
Costly to use
2
Do not think there is any barrier
2
Usage Difficulty
2
Time inefficient
6
Think it is not useful
1
Don't know of any software testing metrics
3
21 respondents described that budget was the main hurdle to using automatic testing tools in their companies. A moderate number of respondents regarding time and problems to use tools as reasons that stopped them from using these tools. The result is presented in Table 4. Details are also illustrated in Figure 7.
No information resource available
3
Other
2
Total
21
Standards that are developed in-house were engaged by 13 companies, whereas 8 companies uses blend of commercial & “inhouse standards” for software products testing. Of the 21 companies answered to the questionnaires, 8 are certified quality organizations. Remarkably, out of the 8 accredited organizations, only 5 assumed that their development practices were enhanced by attaining the certification, whereas 2 did not consider so & 1 was not assured.
It is encouraging to see that 11 (>50%) of the 21 companies using metrics. However, only a few organizations said, the worth of the software products was enhanced by using software principles. Three organizations were not confident, & 2 companies differed about the positive effects of metrics on software quality. Table 4. Testing tools adaption barriers Barrier
Response
Difficult to use
3
Costly to use
6
Do not think there is any barrier
2
No information resource available
2
Do not think it is useful
1
Do not know of any software testing tool
2
Time-consuming to use
4
Other
1
Total
21
Figure 8. Testing metrics adoption barriers Table 6 sum up the opinions on hurdles while implementing the standards for testing software products and certification in companies. Many companies (5 companies) tell that there is no hurdle. Major amounts of replies representing that the cost (2 companies) & the time (3 companies) are the core restrictions on the use of testing standards. Details are also illustrated in Figure 9. Table 6. Standards adoption barriers Barrier Cost inefficient Think there is no barrier Think it is not useful Time-consuming to use Difficult to use Unaware of software testing standards No information resource available Other Total
Figure 7. Testing tools adaption barriers As shown in Table 5, about 2 of the participants (21 companies) stated that there is no hurdle or drawback in the use of the software testing metrics. There was a larger number of participants (6 companies) which find that by using the “software testing metrics” is not efficient it consumes a lot of time. Details are also illustrated in Figure 8. Table 5. Testing metrics adoption barriers Barrier
Response
150
Response 2 5 2 3 4 3 0 2 21
It was promising that 14 companies gave more chances to their software testing teams to get education and training to do activities regarding testing of Softwares. Training courses (External) were also provided to their staff. Also internal courses and self-study were done. On the needs basis, only 8 companies gave training to their employees. It is regret to account that only 4 companies provided regular education and training sessions to their staffs. Barriers to giving education and training, the budget is still an important aspect (7 companies), schedule constraints stated by 5 companies. Only 4 organizations did not consider that there is any hurdle to give education to their employees for the testing of software systems (Table 7). Details are also illustrated in Figure 10.
Figure 9. Barriers to adoption of standards
edge from others organizations in terms of software product quality.
Table 7. Training barriers for software testing employees Barrier Time Think there is no barrier Course Money Others Total
Response 5 4 3 7 2 21
White-box and dynamic testing will improve the coder efficiency and the output in agile development and allow users to develop confidence in the quality end software product. In future, we expect practitioners and academic to evaluate and implement software-testing frameworks in various Pakistan software organizations.
8. REFERENCES [1] Softwaretestingfundamentals.com, Software Quality Assurance – Software Testing Fundamentals. http://softwaretestingfundamentals.com/software-qualityassurance/ [2] Mosaicinc.com. Difference Between QA, QC and Testing. http://www.mosaicinc.com/mosaicinc/rmThisMonth.asp [3] Myers, Glenford J., The art of software testing, New York: Wiley, c1979. ISBN: 0471043281 Physical descriptions: xi, 177 p.: ill.; 24 cm. [4] Muhammad Amir, Khalid Khan et al., An Appraisal of Agile Software Development Process, International Journal of Advanced Science and Technology, Vol.58, (2013), pp.75-86 [5] Sajid Ibrahim Hashmi and Jongmoon Baik, Software Quality Assurance in XP and Spiral - A Comparative Study, IEEE Fifth International Conference on Computational Science and Applications, April 1, 2009
Figure 10. Training barriers to software testing staff
[6] Dr Rashid Saeed, Muzamil Safdar et al., Service Quality Attributes and Customer Satisfaction: An Empirical Study on Microfinance Banks- a case from Pakistan, J. Basic. Appl. Sci. Res., 3(6)723-731, 2013
7. CONCLUSION & FUTURE WORK The research paper described and examined the outcomes of the testing practices and procedures. Many companies were asked to partake in this examination, a comparatively low rate of organizations takes part in this survey. However, the significance of survey was measured to be very essential, above 65% of participants had team leadership/managerial positions.
[7] Jovanović, Irena, Software Testing Methods and Techniques, Transactions on Internet Research, January 2009. [8] Claude Y. Laporte, Nabil Berrhouma et al., Measuring the Cost of Software Quality of a Large Software Project at Bombardier Transportation: A Case Study, 14 SQP VOL. 14, NO. 3/© 2012, ASQ.
The survey explored the trends of software testing practices that are followed in Pakistan IT industry. The necessity for this research paper is to establish the ideal connection between software quality & testing; guaranteeing that strategies for testing software systems will produce the software that is best in quality, as software are becoming an important part of our everyday lives.
[9] Alessandro Orso and Gregg Rothermel, Software Testing: A Research Travelogue (2000–2014), FOSE’14, May 31–June 7, 2014, Hyderabad, India. [10] D. Bhatt, A Survey of Effective and Efficient Software Testing, IREJOURNALS | Volume 1 Issue 1, May 2017
From this survey results, it is clear that training and education in software testing and software quality assurance is not satisfactorily addressed in formal software industries of Pakistan. Software organizations in Pakistan need to make enhancements in their software testing programs so that they can get a competitive
[11] Akalanka Mailewa and Jayantha Herath, A Survey of Effective and Efficient Software Testing, mics, 2015
151
[12] Ing. Gustavo Lopez, Ing. Francisco Cocozza, and Prof. Marcelo Jenkins, Design and Implementation of a Software Testing Training Course, ASEE, June 2015
[16] Ming-Chang Lee, Software Quality Factors and Software Quality Metrics to Enhance Software Quality Assurance, British Journal of Applied Science & Technology, 2014.
[13] Hafsa Shareef Dar etal., Influence of static testing in agile development: a case from Pakistan software market. Lahore, PAK: Sci.In.t, 2014
[17] Asim Iftikhar and Sheikh Muhammad Ali, Software Quality Assurance a Study Based on Pakistan’s Software Industry, PJETS, Volume 1, No 2, 2011, 65-73
[14] Ali Javed, Muazzam Maqsood et a., How to improve software quality assurance in developing countries, ACIJ, Vol.3, No.2, March 2012
[18] Scott Barber. “The A-B-C's of software testing models”. Search Software Quality, December 2007 [19] Rashmi.V, Charumathi.M et al., Study of Software Testing Techniques and Tools, ijesc, 2017
[15] Nuzhat Naqvi et al., Survey to Improve Software Quality Assurance in Developing Countries, IJTNR, 2012.
[20] Randall W. Rice, The Value of ISTQB Certification and Training. Rice Consulting Services, Inc, 2011
152
