Applying Techniques For Searching In Complex Domains ... - CiteSeerX

Applying Techniques For Searching In Complex Domains To Bug Localization Yosi Ben-Asher

Eitan Farchi y

October 6, 1997 Abstract In [1] we address the problem of \searching in domains" in order to locate a buggy element. The domain itself is structured such that we can query parts of it, and, if the queried part contains a buggy element, continue searching in that part alone. If the answer to the query is negative, then the part has no buggy elements, and the searching process continues checking the complementary part of the domain. We ilisturate how automatic bug elimination can be done by mapping program sematics and strucutre to a search in domain problem. Speci cally, we apply pre-postconditions to the ow graph and dirent other control graphs and program slicing to the D-U graph to obtain a search in domain problem. When the search in domain problem thus obtained is `simple' e.g., a tree search in domain problem, we apply eeicent search in domain algorithms to localize the bug. If partial sematics of the program is not availble, e.g., when prepostconditions are not avaible, the search in domain algorithms can still be applied to direct a human searching process.

1 Introduction In [1] we address the problem of \searching in domains" in order to locate a buggy element. The domain itself is structured such that we can query parts of it, and, if the queried part contains a buggy element, continue searching  y

Mathematics and Comp. Sci. dep. Haifa University, Haifa. I.B.M Research Center ,Haifa, Israel

1

in that part alone. If the answer to the query is negative, then the part has no buggy elements, and the searching process continues checking the complementary part of the domain. In [2] we present a polynomial (in the size of jDj = (O(n4 log3 n) steps) algorithm for searching in tree-like (or Forest like) search domains. Bug localization is the process of identifying the part or parts of the program responsible for a given bug. Previous work semi-automaized the process of bug localization. In [3] category partition and program slicing is used to search for the part or parts of the program responsible for a given bug. Program speci cation and a given test DB is used to minimize human intervention.

 ???? Obtain [3]  ???? What is the exact relation between our applications and previous work

Pre-postconditions are used to express program semantics and prove its correctness [6]. An invariant is a condition true after each statement execution. In software engineering program pre-postconditions and invariants are usually used to specify programs semantics. In addition, tools have been implemented to implement pre-postconditions especially in the context of OO [7]. Implementing pre-postconditions gives the advantage that when an execution (at a customer cite or in lab testing) fails a log can be written to a a log le. In general, failure of various pre-postconditions does not localize the bug and the programmer should manually localize the bug. In this paper we propose several methods of automating bug localization using the tree search algorithm presented in [2] and other search in domain algorithms. The various search domains are

 Using pre-postconditions on the program ow-graph and other graphs,

e.g., the program syntatic tree, the call-graph and others.  Program slicing applied to the de ne use graph.

For each domain we de ne the possible queries and show that the domain thus de ned meets the abstract de nition of a search domain. Searching 2

the program ow graph gives natural program constructs in which the bug may have occurred. The calling graph is suitable for bug elimination in OO languages were the call-graph express the control ow (The McCabe measure of a method under the OO paradigm is expected to be small). Thus, our approach gives a exible method of automatic bug eliminations that gives results which are meaningful to a human. If autmoatic queries can not be construcuted, e.g., when pre-postconditions are not available search in domain algorithms can still be applied to direct a human searching process. Based on a serach in domain algorithm, the program recomends where to set the next break point during the process of bug-localization. This can result in a more eecient bug-localization process. ?????We relate our search to binary search. This work is organized as follows. The `Search in Domain' section de nes the search in domain problem and its special case search in DAGs. The `Pre-postcondition' section gives examples of bug-elimination where pre-postconditions are used to de ne the domain queries. The `Program Slicing' section give examples of bug-elimination where program-slicing is used to de ne the queries.

2 Search In Domain Problem We address the problem of \searching in domains" in order to locate a buggy element. The domain itself is structured such that we can query parts of it, and, if the queried part contains a buggy element, continue searching in that part alone. If the answer to the query is negative, then the part has no buggy elements, and the searching process continues checking the complementary part of the domain. A classical example is the search for a \marked" number in the set [1; : : :; n]. A query i 2 [1; : : :; n] returns `yes' if the number is smaller than i, and the search continues with the range [1; : : :; i ? 1]. Otherwise, it continues with the range [i; : : :; n]. In this case the optimal strategy is to use a binary search and query i = n2 . We use set of sets to describe the structure of general search domains. Denote the set of elements of the search domain by D. For any subset of the search domain i  D, such that there is a history of the search process that reaches i , we postulate a set S = f 1i ; :::; ii g, ji  i of subsets of i. The ji s can be used by the searcher to search for the buggy element in i

3

k

i when i is reached during the search process.

Somtimes Compact representation of the search domain can be used to automatize the search process. A compact representation satis es that in every stage of the search the set of allowed queries can be computed directly from the representation. In contrast, in the case of set representation, the set of allowed queries is explicitly listed in advance. One such compact reresntation is a DAG. The tree special case is treated in [2] and a polynomial alogorithm in the size of D is obtained for nding the buggy elment. In [1] we de ne a search in a DAG and show the equivalent representation as set of sets. The de ntion of a search in a DAG follows. Note that this also de nes a search in a tree. De nition 2.1 Let G =< V; E > be a DAG, CG (u) the connected com-

ponent starting at node u 2 V and G ? CG (u) its complement graph. A search in G for a buggy node v 2 V involves querying a node u 2 V , and if v 2 CG(u) then the search continues with G0 = CG (u); otherwise, it continues with the complement G ? CG (u) (until jGj = 1).

Figure 1 describes all the possible histories of a search in a DAG. The letters leabeling the edeges represent queries at a DAG node. If a query is answered by `no', the search continues at the sub DAG reached by following the edege labeled by `no'. Figure 2 is an example of a search in a Tree. If the node x is queried and the answer is `yes' the search continues in T1, otherwise, the search continues at T ? T1 .

3 Pre-postcondition During the program design cycle program speci cation is introduced. Program speci cation varies greatly, from informal design documents to formal mathematically oriented speci cation, e.g., axiomatic or algebraic approaches [9]. Formal speci cation methods are usually applied to special software that have outstanding safety and correctness requirements. In contrast, assertions and their special case, pre-postconditions are commonly used in the industry to support the design and testing phases. For example, the C pro4

a b

c d

c

no

yes

a

b

c

no

yes

a

b c

d b

d

yes

no

b

c

yes

b

a

yes

no

b

d yes

no

c

b

c d

b

b

b

d

yes

no

b

yes

no

c no

b c

b

yes

c d

c

c

b

d

a

d d

c

no no

d

b

b

yes

d

Figure 1: Search In a DAG

r T1

T3

T2

x

y g

c a

z

b

e

Figure 2: Search In a Tree 5

f

gramming language supports implementing pre-postconditions through the

assertion facility. Other examples are dierent works on class assertions in

C++, (see [7] and others). Applying the pre-postcondition approach to a program segment S , we de ne a relation fQgS fP g. Q and P are assertions, i.e., sentences, in rst order language referring to the program state. The pre-postcondition relation fQgS fP g requires that if Q holds for the program state before the execution of the program segment S then P should hold for the program state after S nishes executing. For example, the following pre-postcondition relation does not hold.

 fsum = 0g  for(i = 1; i < n; i++) { sum = sum + i*i;

 fsum = 1  1 + ::: + n  ng A basic block is a maximal sequence of consecutive program statements (s1 ; :::; sn) for which any ow of execution that reaches program statement s1, continues executing s2 , s3 , and so on until sn is executed. In other words, none of the si is a control statement or halt statement. Next we de ne a program's P ow-graph. The vertices of the owgraph are the basic blocks of the program P . There is an edge from a basic block B1 to a basic block B2 if there is an execution of P that executes B1 immediately followed by B2 . A program ow graph is clearly a directed graph. It is not an acyclic graph as loops create cycles in the ow-graph. Typical program control constructs, e.g., if-then, if-then-else, repeat and case of the Pascal language, are represented in the ow-graph as sub-graphs (of vertices) that are:

 A connected sub-graph.  Have only one point of entry and one point of exit. I.e., one point

where the ow of any execution can enter the sub-graph, and one point where the ow of execution can exit the sub-graph. 6

 Any path from a point of entry to a point of exit in the sub-graph has all its vertices in the sub-graph.

Any sub-graph meeting the above conditions is called a program construct sub-graph. For a given program P , we denote its ow-graph by Gp. A program construct sub-graph is typically denoted by S . The entry point for a program construct S is denoted by Si and the exit point is denoted by So . Given a program P , and a program construct S , we could have pre and postcoditions associated with Si and So respectively. In the running example, the while is a program construct with the pre-condition fsum = 0g and the post-condition fsum = 1  1 + ::: + n  ng. If S has pre and postconditions associated with its entry and exit points we say that S is a quesry set. Given a test suite T that failed (i.e., at least one of the tests t 2 T failed), we attempt at localizing a bug. This is done by systematically analyzing the pre-postconditions failures along the various executions of T. We thus apply search in domain methods to automatically localize a bug in P . To further develop this approach, we next de ne the search in domain problem associated with a program P and a test suite that fails T . The search domain D is the set of all basic blocks of the program P . The queries at D are all the query sets of P , i.e., all program constructs that have pre and postconditions associated with them. For a given query set S , we run the entire test suite T and check whether the pre and postconditions at S are met. If a test t 2 T is identi ed for which the pre-condition at Si succeeds while the post condition at So fails, we declare S to have a bug and continue the search at S . Otherwise, we declare S to be `correct' and continue the search in the complementary graph Gp ? S . At the next step0 of the search, the possible queries at S are all the program constructs S of P that have both their Si0 and So0 in S . In the0 same way, the possible0 queries at Gp ? s are all the program constructs S of P that have both Si and So0 in P ? S . This de nitions is continued, in the same way, for S and Gp ? S . If the transformational methodology [4] is used, a detailed pre-postcondition annotation of the program can be achieved up to the level of the basic blocks. In addition, the pre-postconditions represents the program semantics. In this case, the search will probably be eective in localization of the bug. We next give a concrete example of a program and its search domain. For 7

01

12

05

08

16

40

20

35

20

35

29

24

Figure 3: Search Tree an integer n, de ne mod max(n) to be the maximal i s.t., 2i jn. The relation mon max(n) is used in the program below to de ne pre-postconditions. The pre-postcoditions in the program below are de ned at the entry and exit points of the various programs' `loops'. If the pre-postcondition for the `loop' at line 01 fails, we can proceed to check if either the prepostconditions for the `loops' at lines 05 or 08 fail. If on the other hand, the pre-postcondition at line 01 succeeds, we can proceed to check the prepostcoditions for the `loops' at lines 12, 16, 20, 24, 29, 35, 41. Continuing in this maner we de ne a search in domain problem. It is easly ver ed that this search in domain problem can be compactly represented using a search in domain tree. The program's search in domain tree is given in gure 3. We have inserted a bug at line 26. Line 26 should read: if ((o mod 2) == 0), but we have omitted mod 2. As o is either 1 or 2, line 27 is never executed; thus, any intial value for sum will demonstrate this bug. Using the tree compact representation of the search domain the algorithm described in [2] is applied. The algorithm rst checks the pre-postcondition for the `loop' at line 16, which fails, and the pre-postcondition for the `loop' at line 20, which also fails. The localization process proceeds to check the pre-postconditions 8

for the `loops' at lines 29 and 24. The bug is nally localized in the `loop' at line 24. Note that manually we would probably have checked the prepostcondition for the `loops' at lines 01 and 12, which is a sub-optimal search. This is the case particularly if we hide the `loops' at lines 12, 16, 20, 24, 29, 35, 41 inside another function call.

9

fpre1 : mod max(sum) = rg 01 for(i = 1; i  2; i + +) 02 f 03 if ((i mod 2) == 0) 04 sum = i  sum; fpre2 : mod max(sum) = rg 05 for(j = 1; j  2; j + +) 06 if ((j mod 2) == 0) 07 sum = j  sum; fpos2 : mod max(sum) = r + 1g fpre3 : mod max(sum) = rg 08 for(k = 1; k  2; k + +) 09 if ((k mod 2) == 0) 10 sum = k  sum; fpre3 : mod max(sum) = r + 1g 11 g fpos1 : mod max(sum) = r + 3g fpre4 : mod max(sum) = rg 12 for(i = 1; i  2; i + +) 13 f 14 if ((i mod 2) == 0) 15 sum = i  sum; fpre5 : mod max(sum) = rg 16 for(j = 1; j  2; j + +) 17 f 18 if ((j mod 2) == 0) 19 sum = j  sum; fpre6 : mod max(sum) = rg 20 for(l = 1; l  2; l + +) 21 f 22 if (l mod 2) == 0) 23 sum = l  sum; fpre8 : mod max(sum) = rg 24 for(o = 1; o  2; o + +) 25 f 26 if (o) == 0) bugisinsertedhere 27 28

g

sum = o  sum;

fpos8 : mod max(sum) = r + 1g fpre9 : mod max(sum) = rg 10 29 for(p = 1; p  2; p + +) 30 f 31 if (p mod 2) == 0) 32 sum = p  sum; 33 g fpos9 : mod max(sum) = r + 1g 34 g fpos6 : mod max(sum) = r + 3g

Whenever pre-postcondition are availble other graphs can be used for bug elinimation with good results. One example is the program syntactic tree which is contained in the ow-graph. The program sysntactic tree is probably a better choice for automatization as it is easly calculated. In the context of OO programing the control ow is largly substituded by inheritance and polymorphism. A clear indication of this is that the McCabe measure of an OO method is expected to be small. We can thus de ne the search domain on inheritance hirarchy or call graphs. Pre-postconditions are not the only mean of creating queries that support bug elinimation. Other means are progam invariants and abstract interpetation [8]. One example is expresion parity. Assume an expresion de ned by

Ge E ? > T ?> E + T T ?> F and an abstract interpetation par(E ) that returns ?> T  F F ? > c constant true i E has the desinged parity. If the parity of an expresion E is wrong,

we can de ne a search domain by recursively checking that sub-expresions of E have the right parity. For example, if E ? > E + T , we check if parity of T on the right is correct. If the parity of T is correct, the search for the mistake in the expresion is continued in E , otherwise, the search for the mistake in the expresion is continued in T .

4 Program Slicing De ne-use pairs are traditionally used to de ne test adequacy criteria[5]. A de ne-use pair is an ordered pair of two program statement (s1; s2 ) that meet the following criteria:

 The statement s1 assigns a value to the variable X and the statement s2 is either a control statement containing X or an assignment statement with the variable X appearing in the right side of the assignment.  There is no assignment statement to the variable X on any path leading from s1 to s2 in the program's control graph. The de ne-use, DU , test adequacy criteria is met for a program P and a test suite T if for each de ne-use pair, (s1 ; s2), there is a test t 2 T that 11

exercises (s1 ; s2). To give a concrete example, consider the following program segment:

 int i, j, k; //statement 0  read(i); //statement 1  if (i  3) //statement 2 { j = i + 5; //statement 3

else { read(i); //statement 4 { k = i - 5; // statement 5

j = k; // statement 6 The de ne-use pairs in the above program segments are:

   

(1; 2), here statement 2 is a control statement. (1; 3), here statement 3 is an assignment statement. (4; 5) (5; 6)

Note that (1; 5) is not a de ne use pair as statement 4 is a re-de nition of i along the path starting at statement 1 and ending at statement 5. In addition, reading input data into variable X is considered an assignment statement for X . Finally, variable de nitions are considered an assignment statement (statement 0 in the code segment). There are many variation de ned in the literature for De ne-use test adequacy criteria, e.g., a `de ne-use control' test adequacy criteria is met if every pair of de ne-use (s1; s2) for which s2 is a control statement is exercised by the test suite. For simplicity, we concentrate on the de ne-use test adequacy criteria given above. Our bug elimination technique can be applied to other de ne-use criteria. For a given program P , the set of de ne-use pairs de nes a directed graph DUp . In any directed graph G, a parent of a vertex v is a vertex w 12

that has a path from w to v . A child of a vertex v is a vertex u for which there is a path from v to u. For a given vertex v 2 DUp, the set of all parents and childs of v , including v itself, form a program slice Pv associated with v. In the running example, statement 5 and 6 are childs of statement 4 and statement 4 has no parents, thus, statement 4, 5 and 6 form the program slice P4 . In the context of regression testing, Program slicing minimizes the number of test runs required when a code modi cation is made to maintain a de ne-use coverage level[5]. We next apply program slicing to bug elimination.

5 Program Slicing and Bug Elimination For simplicity, we limit our discussion to program errors that result from an erroneous program statement. The discussion can be easily generalized to erroneous control graph edges or erroneous de ne-use relations, etc. Bug elimination refers to the brode process of identifying a `bug' in a program P given a set of failing tests T . We restrict the concept of bug elimination as follows. Bug elimination, in our context, is the process of setting a break point s11 along a trace of t1 2 T , running the test t11 until it hits the break point s11 and then examining the state of the program at s11. Based on examining the program's state at s11 the user decides either to progress the program to a break point s21 along the trace of t11 2 T or to halt the run and run a dierent test t2 2 T until it reaches a break point s12 along the trace of t2 2 T . Our objective is to support the bug elimination process by automatically recommending, at each stage of the bug elimination, the next test t 2 T to run and the next break point s along t's trace. Thus, the algorithm, using a query in domain representation of the program P automatically identi es a test t 2 T to run and a break point s along the trace of t 2 T . The query in domain problem is de ned using the dierent possible program slicing of the program P , When the break point s is reached, the user examines the program state and reviews the code, thus determining whether the statement s is erroneous. Her resolution is fed back into the algorithm and is used to automatically determine the next test t0 2 T to run and the next break point 13

s0 along t0 's trace. We next de ne the query in domain problem enabling

the automatic support just described. A trace trt associated with a test t 2 T is the sequence of statements executed by P when t is used in the order of their execution. A trace trt exercises a de ne-use pair (s1 ; s2) of a variable X , if s1 appears before s2 in trt and there is no rede nition of the variable X in the trace trt between s1 and s2 . The set of all de ne-use pair exercised by a test t, or a trace trt , is the set of all de ne-use pairs exercised by trt . We denote this set by DUt The domain D of the `query in domain' problem is the set of all de neuse pairs of P . A query associated with a statement s and a test t 2 T is the set of all de ne-use pairs obtained by intersecting the program slice at s and the de ne-use pairs exercised by t 2 T , i.e., Ps intersection DUt . We denote this set by Qst . The query structure at D is the set of all queries at any statement s and for any test t 2 T . I.e., QD = fQst jt 2 Tands is a statement of P g. For any subset of de ne-use pairs B the query structure at B , QB . is de ned by the intersection of B with the query structure at D. I.e., the query structure at B , QB = fB intersection QD g. The rationale behind this de nition is as follows. If the statement s is erroneous then the error might be related to wrong de ne-use relationships in the program slice Ps . The test t exercises the de ne-use pairs Qst , thus, potinally revealing the error at s through exercising de ne-use pairs belonging to the program slice Ps . The size of Qst represents the percentage of the search domain D that might have caused a bug in s and is exercised by t. The set of all de ne-use pairs D represents all the possible errors at any program statement s. We thus attempt to eectively search for an erroneous de ne-use pair in D by using the query structure Qst . We next give a concrete example. Consider the following program that calculates the square root of a positive number:

   

read(X); //statement 1 A1 = 0.00001; //statement 2 X2 = X; //statement 3 while (X2 - X1)  0.00002 //statement 4 14



{ X3 = (X1 + X2)/2; //statement 5 { if (X3 * X3 - X) * ( X1 * X1 - X)  0 //statement 6

 X2 = X3; //statement 8

{ else {  X1 = X2; //statement 7

Statement 7 is an error and should have been X1 = X3. For the only test with input three, the program fails with an in nite loop. In addition, the path exercised by the input three is (1, 2, 3, 4, 5, 6, 7, 4,...). The following program slicing are available:

     

statement 1: f(1; 3); (1; 4); (1; 6); (1; 8); (1; 5)g statement 2: f(2; 4); (2; 6); (2; 7); (2; 8); (2; 5)g statement 3: f(3; 4); (3; 6); (3; 7); (3; 8); (3; 5)g statement 5: f(1; 5); (2; 5); (3; 5); (7; 5); (8; 5)g statement 7: f(7; 5); (7; 4); (7; 6); (7; 8)g statement 8: f(8; 5); (8; 6); (8; 4); (8; 7)g

The algorithm will recommend the setting the following break points: statement 1, statement 2, statement 3 and statement 7. At this stage the user will nd the bug.

6 conclusion TBC

References [1] Y. Ben-Asher and E. Farchi. Compact representations of search in complex domains. In not-published, 1997. 15

[2] Y. Ben-Asher, E. Farchi, and I. Newman. Optimal search in trees. In 8'th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA97), New Orleans, 1997. [3] Peter Fritzson et. el. Generalized algorithmic debugging and testing. ACM Letters on Programming Languages and testing, 1:303{322:, YEAR =. [4] R. Balzer et. el. Software technology in the 1990s: using a new paragdigm. IEEE Computer, 1983. [5] Rajiv Gupta et. el. Program slicing-based regression testing techniques. In Proceedings of the Conference on Software Maintenance, 1992. [6] R.W. Floyd. ssigining meaning to programs. Mathmatical Aspects Of Computer Science, pages 19{31, 1967. [7] Sara Porat and Paul Fertig. Class assertions in c++. JOOP, 1995. [8] Dieter Maurer Reinhard Wilhelm. Compiler Design. Addison Wesley, 1995. [9] Hans van Vliet. Software Engineering Principles and Practice. John Wiley and Sons, 1993.

16