The Application of Automated Tools to Improve the Software ... - SERSC

International Journal of Software Engineering and Its Applications Vol. 8, No. 10 (2014), pp. 95-114 http://dx.doi.org/10.14257/ijseia.2014.8.10.10

The Application of Automated Tools to Improve the Software Testing Process Amin Moradbeiky1, Vahid Khatibi Bardsiri2, Sepideh Aghajani3 and Roghaye Heydari4 1,4

Kerman Branch, Islamic Azad University, Kerman, Iran Bardsir Branch, Islamic Azad University, Kerman, Iran 3 Tarbiat Modares University, Tehran, Iran [email protected], [email protected], [email protected], [email protected] 2

Abstract Software testing is the major challenge of complex software systems. In this field, accuracy and running time are two major performance factors frequently employed to evaluate the reliability of software testing. Automated software testing is a technique which significantly improves these performance factors. There are various automated testing tools dealing with design features however this paper focuses on the comparison of the testing tools exclusively used for .Net applications. We attempted to search for high performance testing tools including popular software testing environments so that the obtained results can be extended to a wide domain of software developers. The main goal of this paper is to analyze and compare the automated software testing tools using practical examples and review the performance of these tools in terms of software reliability determination. Keywords: software engineering, automated software testing, testing tools, reliability

1. Introduction Testing tools are suitable for perception, observation, tracking, controlling, predicting, and examining software design projects. The prediction and artificial intelligence (AI) algorithms are successful to produce reasonable results especially when input range is wide. In this regards, Catala et al. eliminated the requirements for considering all modules using Naive Bayes algorithm [1]. Pachauri et al. incorporate genetic algorithm to cover all parts of software and test it accurately [2]. Proposing systems that explicitly focus on testing and measuring quality of software products is a key issue in software quality control. Software testing process usually plans as an advanced project to cover diverse or sometimes controversial needs. Catlani et al. proposed an approach to test software based on its fundamental role to increase the software development lifecycle [3]. The remainder of the paper is organized as follows: the formal techniques, advantages, drawbacks as well as the common principles of automated testing process are explained in Section 2. In Section 3, we focus on testing some of the given testing tools in different conditions using various examples. The specification and performance of the given testing tools are compared in Section 4. The conclusions are drawn in Section 5.

ISSN: 1738-9984 IJSEIA Copyright ⓒ 2014 SERSC

International Journal of Software Engineering and Its Applications Vol. 8, No. 10 (2014)

2. Automated Software Testing Process The logical structure of software engineering such as object-oriented and modular design are the two common principles to enhance the quality of products. Consequently, new tools encourage software developers to follow these principles effectively. They provide possibility of building integrated tools in order to achieve purpose of testing process automation [4]. Beizer proposed methods to automate minor and major software tests base on the generated graph by algorithm. Although software testing process is a time-consuming process, it is possible to automate the process before design of software and extend it as a part of software development [5]. Moreover, it can reduce human problems in debugging as Shneiderman referred in [6]. Thus, the greatest benefit of an accurate and suitable test is software quality assurance [7] and they are closely interrelated.

3. Automation Tools In this paper, we introduce and consider different testing tools. First, the applicability and the features of tools are considered. Next, we tried to choose broad spectrum including various types of tests. Finally, we look for high performance testing tools including popular software testing environments to extend our results into a wide group of software developers. Investigation of tools begins with the introduction of each tool. In the next stage, the necessary setting of the tools to use them effectively will be illustrated. Furthermore, the result of testing process will be presented and the features of each tool will be challenged. The scripts of each example are written as simple as possible to avoid deviations from the main objective of the investigation of the tools. 3.1. MbUnit Tool MbUnit is a developed open source testing tool for the .Net framework and is used generally for unit testing. The tool adds to the project through some libraries and uses labeled classes and methods to run it in the predicted condition. MbUnit utilizes an executive interface namely Icarus GUI Test Runner to include the results in the labeled points in order to observe the progress of running. MbUnit has a distribution called MbUnit-White to test the user interface [8]. Section 3-1-1 presents some examples of testing followed by discussions on test results. 3.1.1. Examine MbUnit Tool: Sample 1. In this example, we intend to test the sum of two numbers function in MbUnit tool using a testing function shown in Fig. 3.1. For this purpose, we used an automation platform called Gallio that provides command line and Windows GUI test runner for a number of test frameworks including MbUnit, Nunit, MSTest, and xUnit.net. The class of Assert is used to ensure the accuracy of test. When we call the Add function, the expected result is returned to Assert Class. Then, the Gallio considers it and show error message if it is in contrast with the expected value. 1 [Test] 2 public void new_test() 3 { 4 Class1 objcls = new Class1(); 5 int res = int.parse(objcls.add_(2,3).trim()); 6 Assert.AreEqual(5, res); 7 }

Figure 1. Testing Sum of Two Numbers Function

96

Copyright ⓒ 2014 SERSC


The process is started by clicking on “Start” button and the output is published on test report window tab. As illustrated in Figure 3.2 the corrected results are asserted by the check mark sign. Notice that a class will pass the test, provided that all methods return correct results. Moreover, the constructor method of a class will be considered as a common method and can pass or fail the test. In “Execution Log” section, the results of consideration are published in more detail including duration of test, number of received assert during assertion process, duration of processing each class or method and some more information, e.g. number of execution threads or number of passed or failed methods.

Figure 2. Gallio Eexecution Environment Graphical format of test results are shown on the so-called “Test Result” window tab (see Figure 2), which indicates important points of test results describing as follows: (1) Steps of processing (2) Test kind at each step (3) Time spent on each step (4) Number of received asserts at each step At top of “Test Result” window tab, the percentage of error and success of test are shown in red and green. Red indicates error rate and green displays success rate.

Figure 3. Result of Testing Add Function Notice that received asserts is determined by assert number of subclasses. For instance, as shown in Figure 1, new_test method has one assert and it is also taken into account in upper class. Sample 2.We try again to review the result of Gallio with this assumption that according to input parameters, the applied method or class run into a runtime error.


97


1 public string ID2SelCol(string ID, string tblName, string ColID, string ConStr, string ColRes) 2 { 3 string Result = ""; 4 SqlCommand com = new SqlCommand(); 5 SqlConnection con = new SqlConnection(); 6 SqlDataReader sqlread = null; 7 con.ConnectionString = ConStr; 8 com.CommandText = "SELECT * FROM " + tblName + " WHERE show=1 AND " + ColID + "='" + ID + "'"; 9 com.Connection = con; 10 con.Open(); 12 sqlread = com.ExecuteReader(); 13 while (sqlread.Read()) 14 { 15 Result = sqlread[ColRes].ToString(); 16 } 17 con.Close(); 18 return Result; 19 } Figure 4. Database Connectivity Algorithm The method for connection to database is shown in Figure 4. In addition, Figure 5 represents the code to handle the method. 1 [Test] 2 public void sample3() 3 { 4 Class1 objcls = new Class1(); 5 int a_ = 1; 6 int b_ = 1; 7 string res = objcls.ID2SelCol("24", "tbl_res", "num", "this text is bad connection string!", 8 "name"); 9 Assert.AreEqual(a_, b_); 10 } Figure 5. Testing Database Connectivity Algorithm As Figure 6 shows, through testing the algorithm, an error occurred before receiving the assert point. Hence, the number of asserts is set to 0. Moreover, when an algorithm has run into an error, the reason and duration of error will be also displayed, and it is possible to compare the error with other error types. Therefore we can deal with the errors more accurately in order to handle the decrease in system speed.

98



Figure 6. Result of Testing Database Connectivity Algorithm

Figure 7. Result of Testing Database Connectivity Algorithm 3.2. WCAT Tool Stress testing is a general software testing tool hence in this section we perform this test using Web Capacity Analysis Tool (WCAT) that is primarily designed for ASP.NET applications. WCAT tool has been developed by Microsoft and it is one of the modules that are associated to the Internet Information Services (IIS). The tool performs stress test only for web applications. However, it is not limited to software that is developed by a particular developer. In this experiment, we apply the WCAT tool to assign a workload on software and get detailed reports of waiting capacity (request/second). Furthermore, some other substantial information is applied to make performance more effective. 3.2.1. WCAT Tool Settings: The tool includes different setting files with arbitrary names and each file had its own configuration. At the next stage, these files should be introduced to the tools [9]. -Scripts.txt: requests are defined in this file. In other words, in this file the requested pages and how to apply is defined. Figure 8 shows a sample of the file [9].

NEW TRANSACTION classId = 1 NEW REQUEST HTTP Verb = "GET" URL = "http://localhost/Stest/Default.aspx" Figure 8. Sample of script.txt File A different value can be determined for the parameter called “classId” in order to generate appreciate traffic when it is needed. -Distribution.txt: weight of different requests for pages is defined in this file [9]. For instance, it is possible to increase the number of request for server 1 as much as two fold of requests for server 2 using this file. When you are testing one page, this file is not necessary.


99


Figure 9 represents sample of stress distribution. (1 refers to “classId” that is defined in script.txt file and 50, which are written next to number 1, mean 50 % of traffic is assigned to server 1). 1 50 2 25 3 25 Figure 9. Sample of distribution.txt File -Config.txt: The test time and the number of virtual clients in order to assign traffic are determined in this file. Figure 10 indicates a sample of the file [9]. Warmuptime 5s Duration 30s CooldownTime 5s NumClientMachines 1 NumClientThreads 20 Figure 10. Sample of Config.txt File 3.2.2 Test Results: A page is generated to examine this tool as some requests run into runtime errors. The script of tested webpage is depicted in Figure 11.

1 2 3 4 5 6 7 8 9 10 11 12 13 14

if (Session[ses_n] == null) Session[ses_n] = "1"; else { int ppp = int.Parse(Session[ses_n].ToString().Trim()); if (ppp % 2 == 0) { int ddddddd = 0; int.Parse("dfsd"); } ppp++; Session[ses_n] = ppp.ToString(); Response.Write(Session[ses_n].ToString()); } Figure 11. The Tested Algorithm using WCAT

The generated page is located in a specific address and pre-defined settings according to Section 3-2-1 are applied in WCAT tool. The results are classified as overall results and disaggregated results. Disaggregated results are the result of each individual request and response. For all categories of results, error types as well as request or response specifications in terms of time and volume are represented. An important point is that status of received responses is in the form of http codes. Other noticeable points is the specifications of network connection, traffic of connection, and input or output features. At the end of report, all component parts are concluded with a final summary (see Figure 12).

100



Figure 12. The Resulting Output of WCAT 3.3. Performance Test in VS 2008 published by Team System Visual Studio Team System develops a performance testing tool to test the software performance as an important aspect of software test. VSTS create a test and control agent for every website in order to test performance of the website. In order to perform the test, firstly, we designed a sample website includes a main page and a login page that used an authentication tool namely SSO . Next, when different users are trying to login the website simultaneously, the performance of the login page and also memory usage by SSO tool are tested. This test determines or verifies how a system performs in terms of speed, scalability, and stability. The performance is concerned with different measures such as achieving response times, throughput, and resource-utilization levels that meet the performance objectives for the project or product [10]. 3.3.1. Review a Practical Example: In this example, a website is created using VS2008 includes two pages named Login.aspx and Home.aspx. When a user inputs username and password on the login page and clicks on login button, the information is forwarded to the server in order to check. Then user will be directed to the home page, provided the account has been verified. 1 2 3 4 5 6 7

if (Membership.ValidateUser("SuperUser", "superuserPwd")) { authMemory = Application["SuperUser"] as AuthInMemory; if (authMemory != null) { Session["AUTHCOMPONENT"] = authMemory.AuthComponent;


101


8 bool hasRights = authMemory.AuthComponent.HasEntityAccess("LoginPage", Rights.Read); 9 if(hasRights) 10 Response.Redirect("~/public/home.aspx"); 11 Else 12 labelMessage.Text = "can't log-in!"; 13 } 14 } 15 else 16 { 17 labelMessage.Text = "can't log-in!"; 18 }

Figure 13. Authentication Algorithm Sample A tool namely SSO is used for user login. As can be seen in the Figure 13, login status stored in a session. So some part of memory is used by users who are signing in. In other words, the memory usage to store logged in user information and also the response time of login page should be measured. For instance, in this section, the configuration of Load Test is accomplished. As is shown in Figure 15, the test are carried out under the condition that 100 users start signing in, In addition, 100 users are added at each minute and the maximum number of users is limited to 1000 users using the login page simultaneously.

Figure 14. Test Configurations Next, the test will be carried out in order to produce the results. (See Figure 15)

102



Figure 15. Test Results We should measure memory usage and response time for login page in order to specify the server threshold. Thus, the .NET CLR memory counters is required to measure the website memory. The counters are available: 1. # Bytes in all Heaps 2. # Gen 0 Collections 3. # Gen 1 Collections 4. # Gen 2 Collections 5. Gen 0 heap size 6. Gen 1 heap size 7. Gen 2 heap size 8. Large Object Heap size Moreover, the available memory and users workload graphs can be represented to determine the number of simultaneous users that can be supported by software. Then average response time and page response time are calculated to measure the performance of the login page. Notice that average response time indicates the required time to response web request, however, page response time includes the time that requester should wait to load all dependent requests such as loading pictures and styles in the webpage. Through conducted experiments, we found that using the global.asax leads to generate unrealistic results. Thus, we concluded to not use the file.


103


3.4. Microsoft Testing Tool Microsoft testing tool is not limited to a specific software environment and it is carried out on source codes generally [10]. Based on our comparative study this tool has more functionality than the others. Figure 16 shows Form1Test class and its methods that are created by Code Generator. As can be seen, the classes should be labeled to test as the label named TestClass. Labeling is used to indicate which sources such as classes or methods of projects should run to test by the tool. Similarly, selected methods to test should also be indicated by label as TestMethod in the figure. 1 [TestClass()] 2 public class Form1Test 3 { 4 5 [TestMethod()] 6 public void add1Test() 7 { 8 Form1 target = new Form1(); // TODO: Initialize to an appropriate value 9 int a_ = 0; // TODO: Initialize to an appropriate value 10 int b_ = 0; // TODO: Initialize to an appropriate value 11 int expected = 0; // TODO: Initialize to an appropriate value 12 int actual; 13 actual = target.add1(a_, b_); 14 Assert.AreEqual(expected, actual); 15 } 16 }

Figure 16. Sum of Two Numbers Algorithm For testing, first an object of the class including the selected method is constructed. Then based on determined data structure a sample of the method will be created and finally the result of sample method is compared with expected value using line 15 that is shown in Figure 16. A successful result will be reported if the value is true, otherwise the error will be reported. 3.4.1. Review Tool using Examples: Example 1: We edited the code shown in Figure 16, and performed the test project as a common project in a VS environment. 1 2 3 4 5 6 7 8 9 10 11

[TestMethod()] public void add1Test() { Form1 target = new Form1(); // TODO: Initialize to an appropriate value int a_ = 4; // TODO: Initialize to an appropriate value int b_ = 5; // TODO: Initialize to an appropriate value int expected = 9; // TODO: Initialize to an appropriate value int actual; actual = target.add1(a_, b_); Assert.AreEqual(expected, actual); }

Figure 17. Testing Sum of Two Numbers Algorithm with the Condition that the Result is True

104



After running test, since the expected result is equal with actual result, the result is reported successfully. (see Figure 18)

Figure 18. Result of Testing Sum of Two Numbers Algorithm with the Condition that the Result is True As shown in Figure 18, the window displays test project name, test method name and error message if any error occurred. Moreover, in this window it is possible to select and run several methods separately as well as grouping methods when the given classes or methods are many. It is also possible to select desired column to be displayed in the results window.

Figure 19. Test Result Settings Example 2: In this example we intend to consider the result of testing method with the condition that the result is different from what is expected. Consequently, we modified the multiply method as the division value to be returned instead of the result of multiplying that is expected. 1 public int mul(int a_, int b_) 2 { 3 return a_ / b_; 4 } Fig.20. Multiplication algorithm 1 public void mulTest() 2 { 3 Form1 target = new Form1(); // TODO: Initialize to an appropriate value 4 int a_ = 6; // TODO: Initialize to an appropriate value 5 int b_ = 2; // TODO: Initialize to an appropriate value 6 int expected = 12; // TODO: Initialize to an appropriate value 7 int actual; 8 actual = target.mul(a_, b_); 9 Assert.AreEqual(expected, actual); 10 }

Figure 21. Testing Multiplication Algorithm


105


After running the testing algorithm as shown in Figure 22, the result of the test shows a false result and an error displayed that the expected result is 12 but the output result is 3. Finally it is concluded that an error occurred.

Figure 22. Result of Testing Multiplication Algorithm Example 3: Sometimes, when calculating an expression, the exact answer is unclear, however it is possible to give an estimate of the answer range. For instance, when multiplying two integers greater than one, the result should be greater than the maximum number. In VS testing tool, we can define a range of possible correct answers. As shown in Figure 23. This possibility is provided in line 13. Commands execute respectively, and when the testing tool runs the command at line 13, the execution is stopped, and then shows the results as a message. Consequently, the running program terminates regardless of the continuing codes if there are. 1 2 3 4 5 6 10 11 12 13 14

[TestMethod()] public void mulTest() { Form1 target = new Form1(); // TODO: Initialize to an appropriate value int a_ = 6; // TODO: Initialize to an appropriate value int b_ = 2; // TODO: Initialize to an appropriate value int actual; actual = target.mul(a_, b_); if (System.Math.Max(a_, b_) > actual) Assert.Inconclusive("‫عدد بدست آمده منطقی نیست‬."); }

Figure 23. Estimating Possible Correct Answers Algorithm

Figure 24. Result of Estimating Possible Correct Answers Algorithm As illustrated in Figure 24, the program displays this massage after execution. The result is published together with a question mark. It indicates that the final action is dependent on the decision of user himself. Example 4: If the code under test contains an exception error, it will be shown in error messages part. For instance, the code in the next figure is designed to access database and retrieve the specific information from the database. However, it fails and this is due to the fact that the connection information is false.

106



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

public string ID2SelCol(string ID, string tblName, string ColID, string ConStr, string ColRes) { string Result = ""; SqlCommand com = new SqlCommand(); SqlConnection con = new SqlConnection(); SqlDataReader sqlread = null; con.ConnectionString = ConStr; com.CommandText = "SELECT * FROM " + tblName + " WHERE " + ColID + "='" + ID + "'"; com.Connection = con; con.Open(); sqlread = com.ExecuteReader(); while (sqlread.Read()) { Result = sqlread[ColRes].ToString(); } con.Close(); return Result; }

Figure 25. A Sample Algorithm to Access Database 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

TestMethod()] public void ID2SelColTest() { Form1 target = new Form1(); string ID = "24"; string tblName = "tbl_user"; string ColID = "id_col"; string ConStr = "this text is not valid!"; string ColRes = "name"; string expected = "hasan"; string actual; actual = target.ID2SelCol(ID, tblName, ColID, ConStr, ColRes); Assert.AreEqual(expected, actual); Assert.Inconclusive("Verify the correctness of this test method."); }

Figure 26. Result of Testing the Sample Algorithm to Access Database

Figure 27. Test Result As can be seen in Figure 27, the reason of error is displayed in the error message. An interesting feature of this tool is that it enables you to define an exception for a method in order to verify the method if the exception condition happens. Otherwise the method does not meet the condition and consequently contains error. 1 [TestMethod] 2 [ExpectedException(typeof(ArgumentException),"A allowed.")] 3 public void EmptyUserIdInConstructor() 4 {


empty

userId

was

inappropriately

107


5 6 7

Class1 logonInfo = new Class1(); logonInfo.minus(String.Empty, "123321"); }

Figure 28. Testing Method with an Exception In above method, the login functionality of an application is tested. The evaluation criterion is to check whether the empty value is acceptable as an input. Therefore, a new label is added to the method to declare its accuracy. If the defined exception happens, the result will be marked as a true method, and otherwise it will be marked as an unsuccessful method. Furthermore, the source of failure will be displayed in the label.

Figure 29. Test Result 3.5. WatiN Tool WatiN tool is a specific tool that is designed for testing web applications. It includes interesting features such as software compatibility with different web browsers and access to the various controls [11]. WatiN is not limited to a specific web applications environment however its configuration is compatible with .Net framework. WatiN can provide an easy way to automate web application testing and also checking their compatibility with Internet Explorer and FireFox. It supports Microsoft Internet Explorer 6 to 9 as well as Mozilla FireFox 2.X and 3.X. The tool consists of an open source library in order to provide the possibility for further development [11]. WatiN communicates with Internet Explorer using COM. Communication with Firefox is established by JSSH (a plug-in for FireFox) in WatiN.Core.dll file [11]. JSSH is a plug-in to send commands automatically to FireFox via telnet [11]. The tool allows user to open Internet Explorer and work with elements of forms. WatiN can read form elements or update the value of these elements. In addition, it can activate the existing events in a web page [11]. Today is not hard to test code. Many tools are developed to test coded. Nevertheless, for web applications an important layer entitled User Interface Layer should not be disregarded. Control user interface is so important to automating the testing process and sometimes our test fails because automated test cannot find a control in the form. Despite the fact, developers do use the tool and know it as a good way to automated testing. WatiN requires an environment that is compatible with .Net engine. In addition, it needs external software such as Nunit or Gallio to run testing process and review results. WatiN includes variety features, i.e. graphical user interface, performance test and regression test. The advantages of WatiN are listed as following: 1. Automatic access to all the elements 2. find all the elements containing specific features 3. Support AJAX appropriately 4. Utilize object oriented syntax so it is initiative for developers 5. Support creating snapshot of a webpage during processing 6. Easy integrate with other Unit Testing Tools

108



7. Can be used with all .net compatible languages 8. Easy to customize by user because of its open source nature In spite of the mentioned benefits, WatiN tool has the following drawbacks that can resolved due to the nature of open source software. 1. Ajax handling and Silverlight automation ? 2. Limitation on COM automation in Internet Explorer because HTTP status codes are not exposed 3. During accessing the web page elements, a feature called Protected Mode (a security feature on IE 7 and upwards) causes problems. Thus, the feature should be inactivated when WatiN attempt to access the web page. However, inactivating the feature can lead to security problems 3.5.1. Review an Example: The checking property algorithm is illustrated in Figure 30. It evaluates a property of given element in a Webpage to check out whether it is equal to reference value. Firstly, a sample of IE browser is created in the script and pointed to a specific address. Next, an instance of TextField class with all the properties is created using WatiN library. Finally, virtualization of click on the control is done and then using line 14 the value of MaxLength property is compared with 10. Provided that the value is equal to 10, it will be verified, otherwise it will be rejected and the page will be closed. 1 [TestFixture] 2 public class Test1 3 { 4 [Test] 5 Public static void Test_MaxLength() 6 { 7 WatiN.Core.IE window= BAP.login(); 8 Link lnk_BAP_Request=window.link(Find.ByUrl(“http://127.0.0.1/w1/Default.aspx”)); 9 10 TextField txt_Artwork=window.TextField(Find.ByName(“ct100_Main_ddlartwork_text”)); 11 Lnk_BAP_request.Click(); 12 txt_Artwork.Click(); 13 //Verify the maximum Length of the TextBox 14 Assert.AreEqual(10,txt_Artwork.MaxLength); 15 //Calling logout Method 16 BAP.Logout(window); 17 } 18 }

Figure 30. A Sample Checking Property Algorithm Test results using Gallio platform are shown in Figure 31 where the red color indicates that the test has been failed.


109


Figure 31. Gallio Test Report to Check Out a Property An overview of Gallio Test Explorer results for each given object is depicted in Figure 32.

Figure 32. Results of Each Object in Gallio Test Explorer In the next step, we changed the value of property in order to match with the preset value and compiled the test procedure again.

110



Figure 33. The Second Gallio Test Report to Check Out a Property It is possible to test a web page in other web browsers, e.g. FireFox using the following code. 1 2 4

using (var brrr = new FireFox("‫))"تست آدرس صفحه‬ { //‫محل نوشتن کدها برای بررسی صفحه‬ 3 }

Figure 34. Compatibility Test with FireFox Another noticeable point about WatiN is its ability to archive the test results. In this respect, it is possible to create a snapshot of results in a specific web browser using the following script. Moreover, creating snapshot can be subject to certain conditions, i.e., occurrence of an error or a predefined state and etc. 1 2 3 4 5 6

using (var browser = new IE("http://www.google.com")) { browser.TextField(Find.ByName("q")).TypeText("‫;)"امین مرادبیکی‬ browser.Button(Find.ByName("btnG")).Click(); browser.CaptureWebPageToFile("screen.png"); }

Figure 35. Creating Result Snapshot Algorithm


111


Figure 3.36. Result Snapshot

4. Comparison of Tools A wide variety of software testing tools are available. With an overall review of these tools, it can be perceived that some of them are released as an open source tool and some are free. Having open source license is a great privilege because of ability to customize the toolbox. On the other hand, we have the flexibility to combine open source environments to take the advantage of more features. Development of an appropriate test case is an important step in testing process that can improve outcomes [12, 13]. Open source tools can be customized to generate appropriate test case for different application purposes. Configuration of tools is another important point. Some of the existing tools use customized user interface code to simplify the process. However, it limits the configuration flexibility and is not preferable by professional designers. This paper reviewed more flexible tools in configuration, including scripting platform that provides the possibility to extend by coding as a framework. Eventually, we used the testing tools for practical tests to cover different technologies. Moreover, combining open source environments was considered as a contributing factor to overcome limitations of the tool's features. We summarized our review on tools in Table 1. Table 1. The Review of Testing Tools Tools name

Test type

Language Support

MbUnit

Unit testing

.Net

WCAT

Load testing Stress testing

Web based

Web app

Independent

No

VSTS Performance Test

Performance testing

Web based in .Net

Web app

Independent

No

Microsoft

Unit testing

.Net

Desktop

Independent

No

112

Technology Operational Support independence Desktop Independent app

Open Source Yes

Variety reports Text report Text report Text, statistical, graphical report Text,



Project Test

WatiN

app

Compatibility test Integration testing

Web based

Web app

statistical, graphical report Dependent

Yes

Depends on handler

5. Conclusion Using testing tools can effectively make software testing process simpler and more accurate. Variation of the output results of testing is a key parameter in this field. Therefore, the testing tools with large variety of result are more efficient because they can provide more detailed analysis. The possibility to use a given testing tool in combination with other testing tools is a great privilege. Thereby, we are not limited to the features of a specific tool and we can effectively use additional advantages of new tools. As explained, WatiN has this feature to be used in combination with other environments. In addition, we should pay a special attention to the flexibility of open source testing tools which can be consistent with the project requirements. It may sound that the mentioned testing tools is complicated to be used due to the strict requirement for programming knowledge, however it can provide the possibility of customizing test or design test in parallel with software development or even before this phase. Moreover, open source tools can be improved to achieve the best results and only some tools can design test based on the specific application of the software. In the testing process, generate appreciate test case is a contributing factor in successful results, however, none of testing tools mentioned in this paper can generate it automatically. Therefore, testing tool with the ability of generating automated test case can significantly improve the testing process.

References [1] C. Catala, U. Sevima and B. Dirib, “Practical development of an Eclipse-based software fault prediction tool using Naive Bayes algorithm”, vol. 38, no. 3, (2011), pp. 2347–2353. [2] A. Pachauri and G. Srivastava, “Automated test data generation for branch testing using genetic algorithm: An improved approach using branch ordering", memory and elitism”, vol. 86, no. 5, (2013), pp. 1191–1208. [3] M. Catelani, L. Ciani, V. Scarano and A. Bacioccola, “Software automated testing: A solution to maximize the test plan coverage and to increase software reliability and quality in use ”, Computer Standards & Interfaces, vol. 33, (2011), pp. 152–158. [4] B. Beizer, “Software Testing Techniques”, 2d ed., Van Nostrand-Reinhold, (1990). [5] R. S. Pressman, “Software Engineering: a Practitioner's Approach”, McGraw-Hill, (2005). [6] B. Shneiderman, “Software Psychology”, Winthrop Publishers, (1980). [7] E. Miller, “The Philosophy of Testing” in Program Testing Techniques, IEEE Com puter, Society Press, (1977), pp. 1–3. [8] Gallio, www.gallio.org [9] IIS Tools, www.support.microsoft.com/kb/231282 [10] VS tools, www.msdn.microsoft.com/en-US [11] Watin, www.watin.org [12] Y. Zhan and J. A. Clark, “Search-based mutation testing for simulink models”, In: H.-G. Beyer and U.-M. O’Reilly, editors, Proc. of theGenetic and Evolutionary Computation Conference (GECCO’05), (2005), pp. 1061–1068. [13] A. Bertolino, “Software testing research: achievements, challenges, dreams”, In: Proc. of the 1st Workshop on Future of Software Engineering (FOSE ’07) at ICSEpp. (2007), pp. 85 – 103.


113


Authors Amin Moradbeiky, he received M.Sc in Information Systems From Islamic Azad University of Kerman in 2014, He is researcher of CyberSpace Research Institute From 2013, He is a Software Project Manager in Pars Kian Co From 2007 to now, His main research interests include Data Mining, Web Mining, software effort estimation, software test, etc..

Vahid Khatibi Bardsiri, he is a lecturer at the Department of Computer Science, Islamic Azad University, Bardsir Branch, Iran. He holds B.Sc. and M.Sc. Degrees in software engineering from Ferdowsi University of Mashhad,Iran (2002) and Science and Research Branch of Islamic Azad University,Iran (2004), respectively. He received his Ph.D. in the area of software development effort estimation at Universiti Teknologi Malaysia (UTM),2013. He is a senior member of International Association of Computer Science and Information Technology (IACSIT). His research interests are agile software development methods, soft computing techniques and software measurement.

Sepideh Aghajani, she studied M.Sc in Information Systems at Tarbiat Modares University in 2011. From 2008 until now she has involved in different research projects on Information Systems, Data Mining and Web Mining fields. She is a Consultant Member in Iranian Computer Science Organization.

Roghayeh Heidary, she received her BA degree of Information Technology engineering from Islamic Azad University of Sience and Reaserch of Fars in (2010), she is a student in MA of Information Technology engineering (Software design and production trends) of Islamic Azad University of Kerman, her main research interests include software effort estimation, software project management, etc.

114


The Application of Automated Tools to Improve the Software ... - SERSC

The Application of Automated Tools to Improve the Software ... - SERSC

Suggest Documents

MultiRefactor: Automated Refactoring To Improve Software Quality

The Application of Software Tools to Complex Distributed ... - CiteSeerX

The Application of New Software Tools to Quantitative Protein Profiling ...

Taxonomy of Automated Software Testing Tools - CiteSeerX

Software Risk Management: Using the Automated Tools - CEUR-WS.org

Evaluation of Software Quality to Improve Application ...

A Gamification Approach to Improve the Software

Application of Automated Analysis Software to Eddy ...

Device Application to Improve

Model to Evaluate and Improve the Quality of the Software ...

Decision support tools to improve the effectiveness of ... - CiteSeerX

YEASTRACT-DISCOVERER: new tools to improve the analysis of ...

THE PERCEIVED USABILITY OF AUTOMATED TESTING TOOLS ...

THE PERCEIVED USABILITY OF AUTOMATED TESTING TOOLS

Tools to Improve the Process of Engineering Design - Virginia Tech

Novel Tools to Improve the Management of Spatial ...

Automated Numbers of Cluster Determination Using the ... - sersc

Automated linear regression tools improve RSSI WSN localization in

An Intelligent Technique to Improve Quality of Service (QoS) - SERSC

Comparison Between the Rules Of Data Storage Tools - SERSC

application of enzymatic treatment to improve the concentration of

Application of Design of Experiments Box-Behnken to improve the ...

Application of STAMP to Improve the Evaluation of Safety ...

Application of infrared image restoration to improve the accuracy of