Detection of Redundant Code using RD

Detection of Redundant Code using R2 D2 Ant´onio Menezes Leit˜ao INESC-ID/Technical University of Lisbon Rua Alves Redol n. 9 1000–029 Lisboa Portugal [email protected]

Abstract We present the R2 D2 redundancy detector. R2 D2 identifies redundant code fragments in large software systems written in Lisp. For each pair of code fragments, R 2 D2 uses a combination of techniques ranging from syntaxbased analysis to semantics-based analysis, that detects positive and negative evidences regarding the redundancy of the analyzed code fragments. These evidences are combined according to a well-defined model and sufficiently redundant fragments are reported to the user. R 2 D2 explores several techniques and heuristics to operate within reasonable time and space bounds and is designed to be extensible.

1. Introduction One of the most used editing operations of our time is copy&paste. Whenever a programmer uses copy&paste he/she introduces redundant code in the program, in the sense that it is code that didn’t need to be there if the programmer decided instead to refactor the original code. Redundant code is, thus, code that can be eliminated using an appropriate abstraction or reuse technique. It is well known that redundant code severely impacts program maintenance and should be minimized. We will now look at the causes and consequences of redundancy.

1.1. Causes of Redundancy There are several causes for the existence of redundant code including duplication, idioms, design patterns and coincidence. Duplication Code duplication occurs when the programmer prefers to copy, paste and modify some code fragment instead of writing a new one. This is a very frequent phenomena in software development. The duplicates are also known as clones [BYM + 98]. Code duplication occurs because: • It’s an extremely simple form of reuse. • When a single version of the code must satisfy several purposes it might be preferable to create duplicates to allow for its independent evolution. • When programmer productivity is measured by the amount of written code, duplication increases income.

• The shortening of development times cuts the time spent in library creation and evolution, forcing the replication and adaptation of existent code. • Some efficiency problems might require code duplication, in particular in languages that do not support inlining or macros. • Large software systems usually promote code duplication because it is generally difficult for a programmer to understand the entire system. As a result, the programmer will employ a kind of exampleoriented-programming where he copies already developed code fragments to adapt them to different purposes. Idioms Some algorithms and data-structures are already so well-known that there are established idioms for its use. These idioms are also designated as clich´es [Wat94]. Mapping a function over a list of values is one example of an idiom frequently used in functional languages. Idioms constitute a source of redundancy because, due to limitations of the most used programming languages, they usually aren’t sufficiently abstracted. Mapping a function, for example, requires first-class functions and higher-order functions and not every programming language fulfils these requirements, forcing the programmer to repeatedly write the same idiom instead of an appropriate abstraction. Design Patterns Design patterns [GHJV96] are a categorization of usual programming and design practices extracted from real software systems. The advantage of design patterns is that they are tested solutions for some already known problems. In this sense they are very helpful for the design phase because they avoid the costly reinvention of solutions. Unfortunately, most current programming languages do not provide support for the direct use of design patterns. As a result, the programmer might use the pattern as a guide for writing the code but hardly will he be able to write on the same abstract level. In this case, as was already acknowledged on [GHJV96], the use of the pattern entails the detailed description of its mechanics, as well as the detailed description of its specific use. This causes the “expansion” of the pattern on all the points of the program where it is used, contributing to the increase of the redundancy. Some design patterns also reflect limitations of the programming languages [BLR98], such as, lack of automatic garbage collection [Wil95], lack of multi-methods [GWB91], etc. Even worse is the situation of some design patterns that, when not directly supported by the programming languages, imply explicit code duplication. The lack of support for multiple inheritance and mixins [Moo86, Kee89] is a well-known example [BC90]. Repetition of design patterns causes redundancy at the system design level as it promotes independent and unrelated solutions that could benefit from an integrated view of the problems that triggered those patterns. Instead, changes to the problems will cause repeated changes to the design. Coincidence This kind of redundancy arises when different programmers (or the same programmer at different times) need to implement identical functionality. In this case, it is possible that the different versions become very similar, although they were created independently. This problem also occurs when the programmer doesn’t have enough knowledge of the programming language libraries. In this case, the effort needed to search a library for the required functionality might seem bigger than the effort needed to create the functionality from scratch. The result is that programmers are constantly re-inventing the wheel. As an example, [vD98] presents a survey conducted on two relatively small software systems (containing around 100000 lines of code each) that showed that each system contained 4 different (and buggy) procedures to determine leap years. When we extrapolate this to very large systems containing millions of lines of code we can expect to find hundreds of different procedures developed for very similar purposes.

1.2. Consequences of Redundancy The existence of redundancy in a software system is a problem with severe consequences: On the size of the system All studies point out that a significant fraction of the source code of a system is composed of duplicates. [LPM+ 97] and [Bak95] estimate that fraction between 5 and 10% while [MLM96] documents experiments where it goes as high as 20%. Given the fact the existent tools for duplicate detection are not sophisticated enough to detect all the relevant cases, we suspect that the real value might be even higher. Besides being a burden on the efficiency of a software system, duplicate code is a serious problem for system maintenance as it forces programmers to deal with more code than the strictly necessary. On debugging The biggest problem of code duplication is the error propagation that occurs at each copy. Every undetected error is multiplied by all the copies made from the fragment where it occurred. 1 In this case, the costs (in time, money and image) of the error correction multiply for all the duplicates as the most probable situation is the one where the same error is reported on different occasions, in different places, by different users. The different corrections tend to diminish the system integrity and make the problem even more serious as they make the duplicate recognition harder. On maintenance Even when the copy does not propagate errors, it has the serious consequence of spreading important algorithms or data structures all over the code. If the programmer needs to modify those algorithms and data structures it will be necessary to locate them and repeat the changes on all occurrences. This location is generally not easy to achieve. On the design of the system Many cases of code duplication represent problems in the design of the system, for example, bad use of procedural abstraction, abstract data types, or inheritance mechanisms. Redundancy is not always a bad thing. Some efficiency problems might suggest a certain amount of code duplication. In some other cases, introducing abstraction just to reduce a small redundancy might reduce code legibility. In all cases, however, it is necessary to be able to detect redundancy. This leads us to the analysis of current redundancy detection approaches.

1.3. Approaches to Redundancy Detection A lot of effort has been spent on redundancy detection. There are several approaches, including textual analysis [Bak95, DRD99, Joh94, WKJ+ 95], syntactical analysis [Wis92, Wis96, BYM + 98], metric-based analysis [Ott76, FR87, VW96], or semantic-based analysis [Kri01, KH01]. We will now discuss these approaches. 1.3.1

Textual Analysis

Given the fact that programs are usually accessible as text files, a very simple way of redundancy detection is to textually compare all the lines of code of a program. 1

Note that even copies made from algorithms or data structures described on reference textbooks might contain errors that are propagated each time the code is copied from the book. One analysis [Nor91] of several well-known Artificial Intelligence books showed that several of them, namely [ASS85, CM85, Hen89, Wil86, WH88], contained one error in their description of the unification algorithm. Most probably, the error started on one book and “propagated” to the others. It is still propagating to the programs that are being written using those books as references.

This kind of analysis is specially oriented for detection of redundancies caused by copy and modification of the text of a program using a text editor. In this case, the copies will largely be identical line by line or will have some systematic difference caused, e.g., by a change on a variable name or constant value. Another possible difference is caused by the indentation which is automatically adjusted by modern text editors or by user comments added to explain the changes. The Dup system [Bak95] explores the above mentioned properties to identify redundant code. Dup analyzes the program text comparing lines of source code. Two lines of code match if they contain the same sequence of characters modulo white space, comments, and a systematic change of identifiers and constants. To avoid including false positives that arise due to syntactical and pragmatical properties of the programming language (such as lines containing only a closing curly brace that are common in C programs or begin/end pairs that are common in Pascal programs), Dup only considers a matching when the code section is bigger than some pre-established threshold. Usually, Dup results are shown graphically in dotplots [ML81]. Dotplots are bi-dimensional graphics where each axis represents a sequence (of characters, lines, ADN nucleotides, etc) and a dot on (i, j) means that the ith symbol of the sequence equals the jth. The human brain is specially adapted to identify redundancies on a dotplot as these will show up as visual patterns such as diagonal lines or rectangles. The approach used in [Joh94] is designed to deal with larger volumes of source code. This is achieved using a sampling method and then a digital fingerprint [Joh92] for each sampling element. These digital fingerprints have the property that: (1) when two samples are identical, its fingerprints are the same; (2) when two samples are different, there is a high probability that its fingerprints are different. A well-designed fingerprint function will then allow for quick detection of duplicate code sections and only rarely will it produce false positives. Note that before applying this function, the source code might be normalized just like in Dup to allow detection of duplications that were further modified. This approach is used in the ART system [WKJ + 95]. Because there is no need for syntactic or semantic analysis, text-based approaches have the advantage of being programming language independent. Moreover, the comparison process is also independent of the correctness of the analyzed program. The biggest advantage, however, is its efficiency: this approach can take advantage of the efficient algorithms already available for string comparison and, as a result, is extremely fast and can deal with millions of lines of code. Unfortunately, there are some shortcomings: only very simple syntactical elements are compared, which excludes detection of semantically equal code fragments that are syntactically different. Moreover, the size threshold that must be used to avoid false positives limits the detection to duplications of relatively big sections of code. 1.3.2

Syntactical Analysis

Syntactical analysis methods are based on the comparison of abstract syntax trees (AST). Abstract syntax trees have two nice properties for the detection process: (1) comments and white space are automatically eliminated; (2) each identifier is recognized according to its context. The abstract syntax trees are then compared using several possible techniques. YAP [Wis92, Wis96] is a system that operates on a very simplified (and normalized) abstract syntax tree. This tree is produced by canonicalizing the program identifiers, reordering of the procedures according to invocation order, expansion of each procedure on the first invocation point and replacement of the remaining invocation by distinct markers, and deletion of all syntactical elements that do not belong to the language, leaving only reserved words and names of pre-defined procedures. The result is a linearized sequence of syntactical elements which is then compared using an algorithm that can deal with transposed subsequences. Note that YAP was developed to deal primarily with the problem of software plagiarism where it is common to find transposed elements. Clone Doctor [BYM+ 98] is a tool that finds redundant code fragments via the comparison of all subtrees of the abstract syntax tree of a program. This allows for fine grained comparison between code fragments but is

computationally demanding. The process is O(n 3 ) on the number of nodes of the abstract syntax tree (an abstract syntax tree typically has ten times more nodes than the number of lines of the represented program). There are two solutions to make the process more efficient: (1) diminish the number of compared trees; (2) compare each tree with only some of the other trees. Clone Doctor explores both solutions. Each tree is stored in an hash table, thus dividing the tree space into several different independent subspaces. Only the trees in a given subspace are compared. Moreover, the indexing function ignores very small trees, largely reducing the number of trees and making the process less sensitive to small code variations. Each subspace will then contain all subtrees that have a similar structure, only differing on the leaves. Each pair of trees on a subspace is then analyzed using a similarity function that takes into account both the number of common nodes and the number of different nodes. This approach can deal with cases where the code was structurally changed (by adding or removing code). 1.3.3

Metric-based Analysis

Program metric is a measure used to characterize quantitatively essential features of a program in order to allow its classification, comparison and mathematical analysis [CDS86]. One of the first metric-based duplication detectors was presented in [Ott76] and explored four metrics, namely the number of unique operators, the number of unique operands, the number of operators and the number of operands. Programs with identical values of these four metrics are considered sufficiently similar to deserve a closer look. Other systems have been developed that use other, more sophisticated metrics, such as the one described in [FR87]. Note that some metrics might be very expensive to compute. According to [VW96], metric-based detectors are very effective for almost perfect duplications but less effective when the duplication was further modified. 1.3.4

Semantic-based Analysis

This type of analysis explores abstractions of the program semantics to guide the clone detection process. Examples of such abstractions include control- or data-flow graphs or program dependence graphs. Detection of clones in these graphs amounts to the identification of identical subgraphs. In the general case, this problem is NP-complete so several techniques have been developed to make the problem feasible. The approach taken in [Kri01] only considers subgraphs starting on predicate vertices, which allows for finding duplicates within procedures. In [KH01], the clones are found by starting with two matching nodes and slicing backwards from those nodes in parallel while there are matching predecessors. In the end, two isomorphic subgraphs are identified, representing the clones. This approach can find non-contiguous clones, clones with reordered statements, and even clones that are intertwined with each other.

2. R2 D2 We now present R2 D2 —Retargetable Redundancy and Deceit Detector. R 2 D2 is a tool that detects redundancies in large software systems. R2 D2 compares abstract syntax trees of code fragments, searching for similarities and differences that confirm or refute the belief that there is redundancy. Moreover, R 2 D2 has extensive (and extensible) knowledge of the programming language semantics and also explores simplified forms of semantic-based analysis. R2 D2 is composed of four components: • A syntactical analyzer that builds the abstract syntax trees of the programs. This overcomes variations in the textual forms of the programs.

(defun square-root (x) (square-root-iter 1.0 x))

(defun sqrt (x) (iter-sqrt 1.0 x))

(defun square-root-iter (guess x) (cond ((good-enough? guess x) guess) (t (square-root-iter (improve guess x) x))))

(defun iter-sqrt (g x) (if (solution-p g x) g (iter-sqrt (approach-solution g x) x)))

≈ (defun solution-p (g x) ?

(> 0.0001 (abs (- (square g) x))))

(defun good-enough? (guess x) (< (abs (- (* guess guess) x)) 0.0001))

(defun approach-solution (g x) (/ (+ (/ x g) g) 2))

(defun improve (guess x) (/ (+ guess (/ x guess)) 2))

(defun square (x) (* x x))

Figure 1. Computing square roots by Newton’s method.

• A set of comparing functions, each one designed to analyze particular characteristics of the code, ranging from some text-based properties to semantic properties. This set can be easily changed, even in run time. • A data-driven matcher that identifies code fragments and dispatches them to the appropriate comparing function. • A combination function that combines the information produced by the comparing functions. Different combination functions can be used, e.g., to produce a global similarity metric or an explanation of the differences. Although R2 D2 can be easily adapted for other programming languages, its main focus is, for the moment, on programs written in Lisp-like languages such as Common Lisp, Scheme, ELisp and older dialects such as FranzLisp, LeLisp or ZetaLisp. Our examples will be presented using the Common Lisp language.

3. The Comparison Process R2 D2 ’s detection process is guided by the abstract syntax trees of the code fragments under analysis but it also understands, to a large extent, its semantics. However, many semantic features are excluded from the analysis either due to the excessive computational effort they require or due to their reduced relevance for the comparison process. To better understand what is included and what is excluded from the analysis, we will explain R 2 D2 modus operandi when it compares the functions square-root and sqrt of the programs presented in Figure 1.

3.1. Analyzing Identical Forms Each time R2 D2 compares two code fragments it considers several features of the fragments. In the specific case of the first fragments named square-root and sqrt, R 2 D2 knows that it is comparing two defun forms. In the Common Lisp language, a defun form is used to define functions and this means that R 2 D2 can use a specific comparison function that knows how to compare function definitions by decomposing the comparison process into several sub-processes that compare, for both forms, the name, the parameter list, the documentation string (if it is present or absent and the text itself) and finally, the body. Each of these comparisons can confirm or refute the presence of redundant code. Let us now consider the procedures body. Given the fact that both expressions are function calls, this is (a very small) evidence that we face a duplicate. Analyzing the call arguments (same number, same sub-expressions) again (slightly) suggests a duplication. However, this is not enough, because the way the arguments are used is much

more relevant. When we face primitive procedures, the name of the procedures is generally sufficient to establish similitude or difference. When we face user-defined procedures, the difference is not so clear and the solution is to recursively compare the called procedures. This way, the comparison of the square-root and sqrt procedures depends, among other things, on the comparison of the called procedures square-root-iter and iter-sqrt. This process continues until we reach the language primitive procedures. As a result, the comparison of two programs entails a comparison of the call graphs.

3.2. Form Equivalence When we compare the bodies of the procedures square-root-iter and iter-sqrt, we notice that the first one is composed by a cond form and the second one is composed by an if form. The cond and the if forms are two of the large set of conditional forms provided in the Common Lisp language. The fact that they are both conditional forms is a small evidence that the functions to which they belong are similar. However, to continue the comparison, R2 D2 needs to know that there is a strong relation between both forms, expressed by the following equivalence: (cond (test-form then-form) (t else-form))

≡

(if test-form then-form else-form)

The equivalence expresses a bi-directional semantics-preserving transformation [Gri91] that allows one of the forms to be converted into the other without changing the program semantics. For R 2 D2 , this equivalence shows that the comparison between a cond form and an if form should be decomposed into three sub-comparisons: one for the test-forms, another for the then-forms and, finally, a third one for the else-forms. The results of these comparisons are then combined with all the other evidences found. When we continue the process for the first pair of expressions, the call graphs analysis leads us to the comparison between the good-enough? and solution-p procedures. Again, for the correct comparison it is necessary for R2 D2 to know that the relational operators < and > are related by the following equation: 2 (< expr1 expr2 ) ≡ (> expr2 expr1 ) This way, we can include, not only an evidence for the fact that both expressions use a relational (and numerical) operator but also the resemblance between the corresponding sub-expressions. It is important to note that the chosen sub-expressions are determined by the equivalence between the code fragments to which they belong. For this reason, it is necessary to have a sufficiently large set of equivalences between code patterns that allows us to deal with most syntactical variations of each programming language. One simple way to obtain such equivalences in Lisp languages is to acknowledge that, in almost all cases, the syntactical variations are implemented using macros [Bak92] and it is sufficient to expand all macro calls to reduce both code fragments to a common form. In fact, when R 2 D2 doesn’t have an explicitly defined equivalence between forms it explores the macro-expansion approach. In non-Lisp languages we could take advantage of the large number of semantics-preserving transformations that are available [BG98, PP96].

3.3. Primitive and User-Defined Procedures The equivalence between the relational operator suggests that R 2 D2 should proceed by comparing both numeric constants 0.0001 and also by comparing the expression (abs (- (* guess guess) x)) vs the expression (abs (- (square g) x)). This process will continue until it needs to compare the expression (* guess guess) vs (square g). Although the first expression uses a primitive procedure (with two arguments), the second uses a user-defined procedure with one argument. At first glance, this indicates that there 2

This equation is not valid in the presence of side-effects. Usually, this is not a problem because, due to the unspecified evaluation order that is used in several programming languages (including Scheme, Pascal, and C), programmers avoid mixing side-effects that depend on a specific evaluation order.

is some evidence against the duplication verdict. However, a deeper analysis might show some resemblances. In fact, when we inline the square function, we get (* g g) whose similarity with (* guess guess) is clear. This way, when R2 D2 needs to compare a primitive procedure call with a non-primitive procedure call, it can refine the comparison using the inline expansion of the non-primitive procedure. Unfortunately, inlining is not a trivial transformation. Besides the necessary care to avoid variable capture, there are additional complications that make inlining a computationally expensive transformation. R 2 D2 overcomes these problems via a simplified but very efficient form of inlining [Lei01] which is sufficient for the analysis. Basically, R2 D2 replaces the abstract syntax tree node that represents the user-defined procedure call by the procedure’s abstract syntax tree, only obeying the calling conventions when it is computationally feasible to do so. Our tests demonstrate that the lost precision is insignificant.

3.4. Order-independent Comparisons Continuing the comparison process leads us to the analysis of the bodies of the procedures improve and approach-solution and, in particular, to the analysis of the sub-expressions (+ guess (/ x guess)) and (+ (/ x g) g). Although the syntactical trees are different, the difference can be eliminated if we swap the arguments of the addition. We can obviously do this because addition is a commutative operation (modulo side-effects on the evaluation of the arguments). However, it is not always obvious whether we should swap arguments. In the previous example, if the arguments were already on a different order we wouldn’t need to swap them. To deal with this problem, R2 D2 explores permutations of arguments that maximize the confidence that it faces similar code fragments. This approach is also used in many other contexts such as when comparing let bindings or do iteration steps. Unfortunately, the exploration of permutations grows with the factorial of the number of elements. For instance, in the following example (let ((x (+ a b)) (y (* (+ c d e) (max f g h)))) (+ x y)) there are 2! · 2! · 2! · 2! · 3! · 3! = 576 possible permutations. Clearly, it is not feasible to explore all alternatives. We solve this problem using an heuristic approximation justified by the fact that, in most cases, each element in one form only matches correctly a unique element in the other form. In this case, we can maximize the global evidence by first comparing every element in one form with every element in the other form and then we select the pair with maximum evidence and remove all the other pairs containing one of those elements. The process then proceeds with the remaining pairs. This way, the process becomes quadratic but is a good approximation to the correct (factorial) process.

3.5. Analyzing the Call Graph When we continue the comparison process we reach a point where we need again to compare the procedures square-root-iter and iter-sqrt. This happens because they are both recursive. Obviously, to avoid infinite regress, we cannot continue as before but we can include evidence for the fact that they are both recursive. R2 D2 can deal with this situation and it also discriminates between directly and indirectly recursive procedures and even the number of intermediate procedures between indirect recursive calls is accounted as another evidence. This is part of the results that R2 D2 obtains from the call-graph analysis. For our example, the call graphs are represented in Figure 2. Although the call-graphs show obvious resemblances, they don’t discriminate the call points for each procedure. From the call-graph it is impossible to know whether some procedure was called from the test expression of an if or from the consequent. This entails that it

square-root

sqrt

square-root-iter

iter-sqrt

good-enough? *

-

abs

improve


/

+

*

Figure 2. Call graphs for procedures square-root and sqrt.

is perfectly possible to have similar call-graphs that correspond to completely different procedures. Moreover, the lack of discriminating power increases the computational effort needed to compare the graphs. R2 D2 deals with these problems by guiding the comparison of the call graphs with the comparison of the abstract syntax trees of the procedures, taking into account the most common syntactical variations. In fact, the comparison of the syntax trees is intertwined with the comparison of the call graphs. Exploring syntactical variations allows call graphs to be matched in several different possible ways and also allows matching apparently incompatible nodes such as the ones related to the primitive procedures < and >.

4. Extending R2 D2 We presented some techniques used by R 2 D2 to compare programs. It should be clear that the more detailed the analysis, the more complex and computationally expensive is the process. R 2 D2 ’s approach is to position itself on a middle ground: its analysis is not as simple as a mere textual analysis nor as complex as program dependence graph-based analysis. This allows R 2 D2 to deal with large programs and, at the same time, account for the syntactical variations and different organization of the code fragments. R2 D2 ’s development followed four different criteria: • R2 D2 should contain a set of comparison function sufficiently generic to deal with all possible programs, even if it cannot perfectly discriminate all the relevant situations. This allows R 2 D2 to be used even with unknown languages as long as the programs written in those languages can be presented to R 2 D2 in the form of abstract syntax trees. • R2 D2 should be extended by its users, via specialized comparison functions that explore detailed knowledge of the programming language and its most relevant syntactical variations. This implies that the description of these functions should be as simple as possible. • Comparison functions should be easy to install, remove, and weight so that R 2 D2 can be easily adapted to different kinds of duplicate detection (e.g., redundancy or plagiarism). • R2 D2 should be as fast as possible in order for its use in an interactive environment. We now discuss R2 D2 ’s solution to satisfy these criteria.

4.1. Patterns To allow for easy extension, R2 D2 explores a simple pattern-based language to describe the programming language and its syntactic variations. These patterns will then match the abstract syntax trees that we want to compare. The pattern language is described by the following grammar:



::= | | |( . ) ::=(?is ) ::=? In the previous grammar, represents any Lisp symbol and represents any Lisp atom that is not a . A is a Lisp function. A allows arbitrary predicates over the matched elements. The containing pattern will only match when the variable value satisfies the predicate. Note also that the dotted-pair notation can be written using the same conventions used in Lisp for lists. For example, let’s consider the when form that is provided by the Common Lisp language. This form accepts a test expression and any number of consequent expressions that are sequentially evaluated only when the test expression evaluates to true. To match a Common Lisp when form, we can use the following pattern: (when ?test . ?expressions) In the pattern, names starting with a question mark are pattern variables that will be bound with the corresponding subtree of the matched abstract syntax tree. All the other names must match exactly. For example, the match between the previous pattern and the following when form: (when (> a b) (print a) (print ’is-bigger)) will bind ?test to (> a b) and ?expressions to the sequence ((print a) (print ’is-bigger)). To improve matching speed, the pattern language is restricted so that backtracking is never needed. Moreover, all patterns are pre-compiled using partial evaluation and indexed according to the syntax form to which they apply. This process is detailed in [Lei01].

4.2. Syntax The pattern language is used to define the syntax of the language of the analyzed programs. For example, the following expression (defform (if ?test ?consequent ?alternative)) defines a function that compares two if forms and that decomposes the comparison into three independent comparisons between the tests, the consequents and the alternatives of both forms. Whenever there is the need for more specialized comparison functions, the pattern variables can be typified with the name of the appropriate comparison form. For example, to compare two defun forms, we can use the following definition: (defform (defun (?the function-name ?name) (?the arglist ?arglist) . ?body)) In this case, the ?name and the ?arglist of two given procedures will be compared using specialized compare functions named, respectively, compare-function-name and compare-arglist. The ?body will be compared using the generic compare function.

4.3. Equivalences The user can add code equivalences to R 2 D2 using the defequivs form. For example, to establish the equivalence between the Common Lisp forms if and cond, he can write: (defequivs ((if ?test ?consequent ?alternative) (cond (?test ?consequent) (t ?alternative)))) The previous equivalence is used by R 2 D2 not only to decompose the comparison of an if form and a cond form (or vice-versa) into three independent comparisons for the test, consequent and alternative, but also to include some evidence that he is facing similar code. Note that when the analyzed trees do not match the patterns (e.g., because the if form doesn’t contain the optional alternative or because the cond form has more clauses), this particular decomposition will not be used. This turns the equivalence definition into a specialized comparison function. Although in most cases the equivalences are established between two forms, it is possible to include more forms. For an example, consider the next equivalence between list accessors: (defequivs ((cadr ?x) (car (cdr ?x)) (second ?x) (nth 1 ?x))) In this case, each of the 12 two-element combinations of the patterns (e.g. (second ?x) and (cadr ?x)) constitutes a specialized comparison function.

4.4. Canonical Forms Another approach to deal with syntactical variations is to transform the code into a canonical form prior to the comparison process. For example, the Common Lisp form (1+ x) is equivalent to the form (+ x 1). Instead of defining both forms as equivalent, we can instead instruct R 2 D2 to always translate any occurrence of the first form into the second and only then apply the comparison. This is achieved with the following syntax: (defconversion (1+ ?expr) (+ ?expr 1))

5. Evidences So far we presented the decomposition of the comparison between two code fragments into several comparisons between sub-fragments. We now discuss the generation and combination of the results of each comparison. During the comparison process, we can find some code fragments that are very similar but we can also find other fragments that are very different. When the same code fragment contains sub fragments that are similar and sub fragments that are different we must have some way of combining this conflicting information to ascertain the similarity of the containing forms. This information constitutes what we call evidences for and against the verdict of code redundancy. R2 D2 does not impose any specific model for the representation and combination of evidences. R 2 D2 provides, however, one possible solution: certainty factors [Sho74]. The certainty factors theory is not as accurate as more recent models but has the very nice property of being the fastest to compute a good approximation to the correct mathematical combination of information from multiple sources.

(defuna add-elemsb (listc ) (ifd (endpe listf ) 0g (+h (firsti listj ) (add-elemsk (restl listm )))))

≈ ?

(defuna list-copyb (listc ) (ifd (endpe listf ) ()g (consh (firsti listj ) (list-copyk (restl listm )))))

Figure 3. Two definitions and relevant evidences.

According to the Certainty Factors Theory, each collected evidence should be represented by a numeric value in the range [−1, 1]. This value is called the certainty factor. The factor 1.0 means absolute certainty that it is true. The factor −1.0 means absolute certainty that it is false. The factor 0.0 means ignorance. The combination function for certainty factors is as follows:   x + y − xy C(x, y) = x + y + xy   x+y

1−min(|x|,|y|)

if x > 0 and y > 0, if x < 0 and y < 0, otherwise.

The above combination function was designed to satisfy a set of desired properties, namely that the combination of ignorance does not affect the result, the combination of an absolute certainty is an absolute certainty, the combination of symmetric (conflicting) non-absolute certainties is ignorance and the combination of truth and false values (a contradiction) is an error. These properties are very useful for detecting duplicates because: • Sometimes, we find evidences so strong that, independently of the remaining evidences, we want to treat this situation specially. A high certainty factor is an assurance that the resulting combination will also have a high certainty factor. In the same way, a certainty factor closer to −1.0 allows for the quick assertion that the compared fragments are not duplicates even if we collect some evidence of the contrary. • Some evidences make sense for redundancy detection but not for plagiarism detection and vice-versa. The properties of the certainty factor 0 allow us to use it to switch off certain evidences depending on the comparison context. • When a code fragment was duplicated and then modified, the gathering of evidences will combine some positive evidences (related to the similar parts) but also some negative evidences (related to the modifications). The combination will produce a certainty factor that reflects the relative weight of the modifications versus the original size of the fragments. Therefore, we get the nice behavior that a large code fragment with a small number of changes will be reported, but a small fragment with many modifications will not. • Being a numeric value, the certainty factor allows for the selection, among different choices, of the one that maximizes or minimizes the certainty that we face redundancy. This is useful, for instance, when we compare sequences of forms or commutative and associative operations. It also allows for the easy use of cutoff thresholds that eliminate duplicates whose certainty is bellow some confidence level. The use of the certainty factors is done by including them on each compare function. To give an example, consider the definitions presented in Figure 3. The comparison process discovers the following (simplified) set of evidences:

a b c d e, i, l f, j, m g h k

both-defuns not-equal-function-names equal-arg-names both-ifs equal-primitive-functions equal-symbols not-equal-atoms not-equal-primitive-functions both-direct-recursive

As we can see, there are only three negative evidences, namely, not-equal-function-names (related to the names of the procedures), not-equal-atoms (related to the returned value on the base case) and not-equal-primitive-functions (related to the combination done in the recursive case). It should be clear that different function names is not very relevant. On the other hand, the remaining two negative evidences are related to the fundamental elements for the computation of the procedures. All the remaining evidences being positive, these two are the ones that can influence the combined evidence. The values assigned to them must depend on the purpose of the comparison: if we are interested in detecting plagiarism they should have strong negative values because they must contradict the remaining positive evidence in order for the procedures not to be reported as plagiarism; if we are interested in detecting redundancy, then they shouldn’t contradict the remaining evidences because, in fact, both procedures are particular cases of the folding of a list using a combinator and an initial value and they can be rewritten in the much simpler form: (defun add-elms (list) (fold #’+ 0 list)) (defun list-copy (list) (fold #’cons () list)) Another very good property of the use of certainty factors is that they allow an optimization that greatly reduces the complexity of the process. This optimization is based on the fact that, in the usual situations, the number of redundant code fragments is much smaller than the total number of code fragments. This implies that only a small number of comparisons will show evidence for redundancy and, in most cases, they will collect a lot of negative evidence. Heuristically, this means that when we have already collected significant negative evidence, it is probable that we will collect even more. This allows us to speed up the comparison process by cutting off comparisons as soon as we have collected enough negative evidence.

6. Results In this section we discuss the application of R 2 D2 to SNePS [ST95]. SNePS is a legacy system composed of 65000 lines of Common Lisp code divided along 240 files. Each procedure was compared with all the others and the similar ones were grouped into clusters. Each cluster represents a pattern of code that is repeated on each procedure belonging to the cluster. All results were obtained on a 500 MHz Pentium III computer with 128 MB of RAM. To analyze the execution time evolution we test increasing fractions of the SNePS system. The results are shown in Figure 4. As we can see, in spite of the quadratic growth, it is possible to analyze the entire SNePS system in 5 minutes. More modern hardware should shorten this time, making it perfectly acceptable for interactive use. Regarding the number of procedures that are considered similar, this obviously depends on the minimum certainty factor considered relevant. In Figure 5 we present the relation between these two parameters. The horizontal axis represents the certainty factor and the vertical axis represents the number of procedures considered similar with an equal or larger certainty factor. Decreasing the certainty factor allows for more redundancy to be detected,

300

Execution Time (s)

250 200 150 100 50 0 0

200

400

600

800

1000

1200

Number of Procedures

1400

1600

1800

Number of similar procedures

Figure 4. R2 D2 execution time when applied to increasing fractions of SNePS. 300 250 200 150 100 50 0 0.9

0.99

0.999

0.9999

0.99999

0.999999

0.9999999

1.0

Certainty Factor

Figure 5. Number of similar procedures versus certainty factor.

but the acceptable differences also become bigger. For very small certainty factors even vaguely related procedures are reported. Selecting only the procedures that were found redundant with a certainty factor no smaller than 0.99 causes 2 R D2 to group them into 41 clusters containing from 2 to 7 procedures each for a total of 113 procedures. We manually checked all these clusters and we could confirm that all the procedures were obvious examples of redundant code (in most cases, the direct result of copy and paste operations followed by small modifications). We could conclude that no false positives were found. We now present some examples of these clusters found by R 2 D2 . Our first example (presented in Figure 6) shows two Common Lisp macros whose body differs only in the expressions surrounded by rectangles. This is an obvious situation of copy and paste followed by a slight modification. Another interesting case is presented in Figure 7. It shows a situation where one of the functions was modified (defmacro addr (register &rest forms) ‘(let* ((regs (get-registers)) (forms ’,(setup-forms (mapcar #’evaluate-form forms))) (found nil) (newregs nil)) (dolist (r regs) (when (packageless-equal ’,register (car r)) (setq r (if (null (cdr r)) (cons (car r) forms) (cons (car r) (append (flistify (cdr r)) (flistify forms)))) found t)) (lisp::push r newregs)) (if (not found) (lisp::push (cons ’,register forms) newregs)) (set-registers newregs)))

≈

0.99

(defmacro addl (register &rest forms) ‘(let* ((regs (get-registers)) (forms ’,(setup-forms (mapcar #’evaluate-form forms))) (found nil) (newregs nil)) (dolist (r regs) (when (packageless-equal ’,register (car r)) (setq r (if (null (cdr r)) (cons (car r) forms) (cons (car r) (append (flistify forms) (flistify (cdr r))))) found t)) (lisp::push r newregs)) (if (not found) (lisp::push (cons ’,register forms) newregs)) (set-registers newregs)))

Figure 6. Copy&Paste

(defun compute-new-support.num-quant (ch old-rui rui rui-set n sign) (let* ((crntct (context.ch ch)) (newsupport (new.sup)) (freevars (freevars.n *NODE*)) (f-sbst (restrict.sbst (subst.rui rui) freevars)) (inst-num-quant (select-instances.num-quant f-sbst (if (eq sign ’POS) *NUM-QUANT-NEG-INSTANCES* *NUM-QUANT-POS-INSTANCES*))) (fns (get-fns f-sbst rui-set n sign))) (setq fns (union.Set (fns.rui old-rui) (union.Set (fns.rui rui) fns))) (when (isassert.n *NODE*) (setq newsupport (compute-new-support1.num-quant fns (sneps:node-asupport *NODE*)))) (when *KNOWN-INSTANCES* (do.set (inst *KNOWN-INSTANCES*) (let* ((instnode (applysubst *NODE* (subst.inst inst))) (supinstnode (filter.sup (sneps:node-asupport instnode) crntct))) (when (and (not (isnew.sup supinstnode)) (issubset.sbst (restrict.sbst (subst.rui rui) freevars) (restrict.sbst (subst.inst inst) freevars))) (setq newsupport (merge.sup newsupport (compute-new-support1.num-quant fns supinstnode))))))) newsupport))

≈

0.99

(defun compute-new-support.and-or (ch fns subst) (let ((crntct (context.ch ch)) (newsupport (new.sup)) (freevars (freevars.n *NODE*))) (when (isassert.n *NODE*) (setq newsupport (compute-new-support1.and-or fns (sneps:node-asupport *NODE*)))) (when *KNOWN-INSTANCES* (do.set (inst *KNOWN-INSTANCES*) (let* ((instnode (applysubst *NODE* (subst.inst inst))) (supinstnode (filter.sup (sneps:node-asupport instnode) crntct))) (when (and (not (isnew.sup supinstnode)) (issubset.sbst (restrict.sbst subst freevars) (restrict.sbst (subst.inst inst) freevars))) (setq newsupport (merge.sup newsupport (compute-new-support1.and-or fns supinstnode))))))) newsupport))

Figure 7. Copy&Paste plus Interleaving (defun vfix-prsp3 (vb) (fixsuffix-s vb))

≈

0.99

(defun fixsuffix-s (word) (prog ((vowels ...) (len (length word)) ...)

Figure 8. Redundant code.

after a copy and paste process. The example demonstrates R 2 D2 ’s ability to deal with interleaved code. Finally, we present a last example in Figure 8. The function on the right has, in fact, 25 lines of deeply nested tests. Note that, in this case, the function on the left is nothing more than an alias for the function on the right. Although the functions body is completely different, both textually, syntactically and semantically, R 2 D2 reports that one of those two functions is obviously redundant.

7. Related work There are several other systems that identify redundant code. Most systems, however, only deal with duplicate code, i.e., code that was developed using copy and paste editing operations. In this case, the copies will largely be identical line by line or will have some systematic difference caused, e.g., by a change on a variable name or constant value. The systems in the area of textual analysis explore the above mentioned properties to identify redundant code. Dup [Bak95] analyzes the program text comparing lines of source code and presents the results in dotplots. The Duploc system [DRD99] uses a very similar approach but with improved user interaction, as it is possible to zoom in on the dotplot and then point to a particular dot and immediately see the corresponding sections of redundant code. Art [WKJ+ 95] is another system that explores digital fingerprints to achieve essentially the same effect but with much larger volumes of source code. All these systems have the advantage of being language-independent. The approach taken in CCFinder [KOK + 01] is also textually-based but the programs are previously tokenized (in a programming language-dependent way) in order to remove indentation and small syntactical variations. Syntactical analysis methods are based on the comparison of abstract syntax trees. YAP [Wis92, Wis96] is a system that operates on a simplified abstract syntax tree. Clone Doctor [BYM + 98] finds duplicate code fragments via the comparison of subtrees of the abstract syntax tree of a program. Both these systems only use syntactical information. R2 D2 , on the other hand, besides the abstract syntax trees, explores a large body of knowledge related to the semantics of the language and program. Moreover, the use of certainty factors allows the distinction between detecting redundancy and detecting plagiarism.

In Clone Doctor, the abstract syntax trees comparison depends on a sufficiently sophisticated hashing function that eliminates syntactical variations. In YAP, these variations are eliminated during a pre-processing phase. In R2 D2 we deal explicitly and independently with the syntactical variations and it is possible to dynamically change the set of equivalences used. Several syntactical variations are tested and each one can have a different contribution. In Clone Doctor, the similarity is quantified by the number of identical nodes in the abstract syntax trees. In YAP, that quantification is done according to the largest common sequence between canonical forms. In R 2 D2 the quantification is done using a large variety of factors that include not only the similarity between abstract syntax trees but also the similarity of the call graphs, the difference between primitive and user-defined procedures, and also some of the metrics used in other, metric-based, approaches such as [FR87]. We currently see some more sophisticated tools [Kri01, KH01] that explore program dependence graphs. Although allowing for a more precise analysis, these are much more expensive to compute, limiting its applicability to small software systems and making it hard to use the tools in an interactive setting.

8. Summary and Future Work We presented R2 D2 , a tool designed to detect redundancy (and plagiarism) on software using a mixture of syntax-based and semantic-based analysis. This mixture is obtained by a fine-grained decomposition of the process into specialized comparing functions, each one exploring different features. The information gathered by these functions is represented in the form of evidences and combined using the certainty factor model. R2 D2 has the advantage that each of its parts is replaceable. It is possible to add new languages by the specification of their syntaxes, it is possible to explore other language features by including new specialized comparison functions and it is possible to replace the evidences and their combination with other models. For the moment, R2 D2 is being used for the detection of redundancies and plagiarism in software written in Lisp-like languages, but we plan to adapt it to detect redundancies in programs written in other languages such as C, C++, or Java. This will allow for a comparison between R 2 D2 and other tools such as CCFinder [KOK + 01], YAP [Wis92, Wis96], and Clone Doctor [BYM + 98]. Adapting R2 D2 to other languages raises some interesting questions: • Parsing programs written in Lisp languages is usually a simple task that is solved by the Lisp environment itself. Other languages with more complex syntaxes such as C++ or Java require specific parsers. We experimented a bit with the Zebu parser generator [Lau92] to parse Java programs and produce the abstract syntax trees needed for the comparison process. • The need for equivalences between code fragments that we explained in section 3.2 is justified in Lisp languages because usually there are several syntactically different forms with equivalent semantics. These different forms result mainly from macro-capabilities that, outside the Lisp family, are either inexistent (Java) or comparatively poor (C, C++). This means that the need for equivalences between code fragments is mitigated in these languages and we can just avoid defining them. It is relevant to note that, without form equivalence, R2 D2 behaves more or less like Clone Doctor, comparing just the abstract syntax trees. • Call-graph analysis is relatively straightforward in procedural languages but becomes a challenge in objectoriented languages as it is in general impossible to statically determine which method is called at a given call site. It is possible, however, to identify all methods that can possibly be called at a given call site and use them as different alternatives to guide the comparison process. If this becomes computationally demanding, we can simply drop the call-graph analysis and just depend on the abstract syntax tree comparison. • Given R2 D2 ’s modularity, it is easy to include other comparison functions, e.g. the comparison of program dependence graphs. However, more complex analysis should be balanced with the interactiveness requirements.

We also plan to attach R2 D2 to a CVS repository so that each time code is committed, it is compared with all the remaining code in the repository to identify possible redundancies and inform the programmer of that fact so that he can take better advantage of the already developed code. This coupling between R 2 D2 and CVS will provide a permanent indicator of the amount of redundancy in the repository.

References [ASS85]

Harold Abelson, Gerald Jay Sussman, and Julie Sussman, Structure and interpretation of computer programs, 6 ed., MIT Press, USA, 1985.

[Bak92]

Henry G. Baker, Metacircular semantics for common lisp special forms, ACM LISP Pointers 5 (1992), no. 4, 11–20.

[Bak95]

Brenda S. Baker, On finding duplication and near-duplication in large software systems, Proceedings: Second Working Conference on Reverse Engineering (Linda M. Wills, Philip Newcomb, and Elliot J. Chikofsky, eds.), IEEE Computer Society Press, 1995, pp. 86–95.

[BC90]

Gilad Bracha and William Cook, Mixin-based inheritance, OOPSLA/ECOOP ’90 Proceedings (Norman Meyrowitz, ed.), ACM SIGPLAN, 1990, pp. 303–311.

[BG98]

Robert W. Bowdidge and William G. Griswold, Supporting the restructuring of data abstractions through manipulation of a program visualization0, ACM Transactions on Software Engineering and Methodology 7 (1998), no. 2, 109–157.

[BLR98]

Gerald Baumgartner, Konstantin L¨aufer, and Vincent F. Russo, On the interaction of object-oriented design patterns and programming languages, Tech. Report CSD-TR-96-020, Department of Computer Science, Purdue University, February 1998.

[BYM+ 98] Ira D. Baxter, Andrew Yahin, Leonardo Moura, Marcelo Sant’Anna, and Lorraine Bier, Clone detection using abstract syntax trees, Proceedings; International Conference on Software Maintenance (T. M. Koshgoftaar and K. Bennett, eds.), IEEE Computer Society Press, 1998, pp. 368–378. [CDS86]

S. D. Conte, H. E. Dunsmore, and V. Y. Shen, Software engineering metrics and models, Benjamin / Cummings Publishing Company, Menlo Park, CA, 1986.

[CM85]

E. Charniak and D. McDermott, Introduction to artificial intelligence, Addison-Wesley Publishing Company, 1985.

[DRD99]

St´ephane Ducasse, Matthias Rieger, and Serge Demeyer, A language independent approach for detecting duplicated code, Proceedings ICSM’99 (International Conference on Software Maintenance) (Hongji Yang and Lee White, eds.), IEEE, September 1999, pp. 109–118.

[FR87]

J. A. W. Faidhi and S. K. Robinson, An empirical approach for detecting program similarity within a university programming environment, Computers and Education 11 (1987), no. 1, 11–19.

[GHJV96] E. Gamma, R. Helm, R. Johnson, and J. Vlissides, Design patterns: Elements of reusable objectoriented software, Addison Wesley, Reading, 1996. [Gri91]

William G. Griswold, Program restructuring as an aid to software maintenance, Ph.D. thesis, University of Washington, 1991.

[GWB91]

Richard P. Gabriel, John L. White, and Daniel G. Bobrow, CLOS: Integrating object-oriented and functional programming, Communications of the ACM 34 (1991), no. 9, 29–38.

[Hen89]

Wade L. Hennessey, COMMON LISP, McGraw-Hill, New York, NY, USA, 1989.

[Joh92]

J. Howard Johnson, Identifying redundancy in source code using fingerprints, Proceedings of CASCON ’92, (Toronto, Ontario; November 9-11, 1992), November, 1992, pp. 171–183.

[Joh94]

J. H. Johnson, Substring matching for clone detection and change tracking, International Conference on Software Maintenance: proceedings, September 19–23, 1994, Victoria, British Columbia, Canada (1109 Spring Street, Suite 300, Silver Spring, MD 20910, USA) (Hausi A. Muller and Mari Georges, eds.), IEEE Computer Society Press, 1994, pp. 120–126.

[Kee89]

Sonya E. Keene, Object-oriented programming in Common Lisp: A programmer’s guide to CLOS, Addison-Wesley Publishing Company, Cambridge, MA, 1989.

[KH01]

Raghavan Komondoor and Susan Horwitz, Tool demonstration: Finding duplicated code using program dependences, Lecture Notes in Computer Science 2028 (2001), 383–??

[KOK+ 01] T. Kamiya, F. Ohata, K. Kondou, S. Kusumoto, and K. Inoue, Maintenance support tools for java programs: CCFinder and JAAT, Proceedings of the 23rd International Conference on Software Engeneering (ICSE-01) (Los Alamitos, California), IEEE Computer Society, May12–19 2001, pp. 837– 838. [Kri01]

J. Krinke, Identifying similar code with program dependence graphs, Proc. Eight Working Conference On Reverse Engineering (WCRE 2001) (Stuttgart, Germany), October 2001, pp. 301–309.

[Lau92]

Joachim Laubsch, Zebu: A tool for specifying reversible LALR(1) parsers, Tech. Report HPL-92-147, Hewlett Packard Laboratories, December 01 1992.

[Lei01]

A. Menezes Leit˜ao, Reengenharia de programas, Ph.D. thesis, Dep. de Eng. Inform´atica, Instituto Superior T´ecnico, Universidade T´ecnica de Lisboa, October 2001.

[LPM+ 97] Bruno Lagu¨e, Daniel Proulx, Ettore M. Merlo, Jean Mayrand, and John Hudepohl, Assessing the benefits of incorporating function clone detection in a development process, Proceedings: 1997 International Conference on Software Maintenance, IEEE Computer Society Press, 1997, pp. 314–321. [ML81]

J. Maizel and R. Lenk, Enhanced Graphic Matrix Analysis of Nucleic Acid and Protein Sequences, Proceedings of the National Academy of Sciences, Genetics, vol. 78, 1981, pp. 7665–7669.

[MLM96]

J. Mayrand, C. Leblanc, and E. Merlo, Experiment on the automatic detection of function clones in a software system using metrics, Proceedings of the International Conference on Software Maintenance (Washington), IEEE Computer Society Press, November 4–8 1996, pp. 244–254.

[Moo86]

David A. Moon, Object-oriented programming with Flavors, ACM SIGPLAN Notices 21 (1986), no. 11, 1–8, OOPSLA ’86 Conference Proceedings, Norman Meyrowitz (editor), September 1986, Portland, Oregon.

[Nor91]

Peter Norvig, Correcting a widespread error in unification algorithms, Software Practice and Experience 21 (1991), no. 2, 231–233.

[Ott76]

K. Ottenstein, An algorithmic approach to the detection and prevention of plagiarism, ACM SIGSCE Bulletin 8 (1976), no. 4, 30–41.

[PP96]

Alberto Pettorossi and Maurizio Proietti, Rules and strategies for transforming functional and logic programs, ACM Computing Surveys 28 (1996), no. 2, 360–414.

[Sho74]

E. H. Shortliffe, MYCIN: A rule-based computer program for advising physicians regarding antimicrobial therapy selection, Ph.D. thesis, Stanford Artificial Intelligence Laboratory, Stanford, CA, October 1974.

[ST95]

S. C. Shapiro and The SNePS Implementation Group, SNePS 2.3 user’s manual, Department of Computer Science, SUNY at Buffalo, Buffalo, NY, 1995.

[vD98]

Arie van Deursen, The leap year problem, 33, Centrum voor Wiskunde en Informatica (CWI), ISSN 1386-369X, December 31 1998, SEN-R9807, p. 7.

[VW96]

K. L. Verco and M. J. Wise, Plagiarism a` la mode: a comparison of automated systems for detecting suspected plagiarism, The Computer Journal 39 (1996), no. 9, 741–750.

[Wat94]

R. C. Waters, Cliche-based program editors, ACM Transactions on Programming Languages and Systems 16 (1994), no. 1, 102–150.

[WH88]

Patrick Henry Winston and Berthold Klaus Paul Horn, Lisp, third ed., Addison-Wesley, Reading, MA, USA, 1988, Prepared with TEX.

[Wil86]

Robert Wilensky, Common LISPcraft, W. W. Norton & Co., New York, NY, USA, 1986.

[Wil95]

Paul R. Wilson, Garbage collection, Computing Surveys (1995).

[Wis92]

Wise, Detection of similarities in student programs: YAP’ing may be preferable to plague’ing, SIGCSEB: SIGCSE Bulletin (ACM Special Interest Group on Computer Science Education) 24 (1992).

[Wis96]

, YAP3: Improved detection of similarities in computer program and other texts, SIGCSEB: SIGCSE Bulletin (ACM Special Interest Group on Computer Science Education) 28 (1996).

[WKJ+ 95] Michael Whitney, Kostas Kontogiannis, J. Howard Johnson, Morris Bernstein, Brian Corrie, Ettore Merlo, James G. McDaniel, Renato De Mori, Hausi A. M¨uller, John Mylopoulos, Martin Stanley, Scott R. Tilley, , and Kenny Wong, Using an integrated toolset for program understanding, Proceedings of the CASCON ’95, November 1995, pp. 262–274.