The Code Orb – Supporting Contextualized Coding via At - CiteSeerX

The Code Orb – Supporting Contextualized Coding via Ata-Glance Views (NIER Track) Nicolas Lopez and André van der Hoek Department of Informatics University of California, Irvine Irvine CA, 92627, USA

{nlopezgi,andre}@ics.uci.edu ABSTRACT While code is typically presented as a flat file to a developer who must change it, this flat file exists within a context that can drastically influence how a developer approaches changing it. While the developer clearly must be careful changing any code, they probably should be yet more careful in changing code that recently saw major changes, is barely covered by test cases, and was the source of a number of bugs. Contextualized coding refers to the ability of the developer to effectively use such contextual information while they work on some changes. In this paper, we introduce the Code Orb, a contextualized coding tool that builds upon existing mining and analysis techniques to warn developers on a line-by-line basis of the volatility of the code they are working on. The key insight underneath the Code Orb is that it is neither desired nor possible to always present a code’s context in its entirety; instead, it is necessary to provide an abstracted view of the context that informs the developer of which parts of the code they need to pay more attention to. This paper discusses the principles of and rationale behind contextualized coding, introduces the Code Orb, and illustrates its function with example code and context drawn from the Mylyn [11] project.

Categories and Subject Descriptors D.2.3 [Software]: Software Engineering – Coding Tools and Techniques. D.2.7 [Software]: Software Engineering – Distribution, Maintenance, and Enhancement.

General Terms Measurement, Documentation

Keywords Contextualized coding, Code Orb, recommendation systems, software visualization

1. INTRODUCTION When a developer must edit code to maintain or evolve it, they are presented with a flat file: the current version of the code as it has emerged over time. Although this is a good starting point, as after all it is the present version of the code that must be modified, it ignores contextual information related to the code’s history, and Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. ICSE'11, May 21-28, 2011, Waikiki, Honolulu, HI, USA Copyright 2011 ACM 978-1-4503-0445-0/11/05.

even its current state. Was it part of any bugs that were resolved in the recent past? Is it well covered by test cases? How many developers have touched it? Are there high levels of code churn? Having this kind of information available could drastically influence how the developer looks at and eventually modifies the code. Code that has been buggy over the past two weeks, has undergone twelve changes during that time, was edited by a single developer, and is not covered by any test cases must in all likelihood be approached quite differently from code that five developers touched over the past months, has had just a few bugs, and is well-covered by test cases. While, of course, every change should be made with care, certain changes sure must be made with extra care. Other than becoming familiar with the historical and present context of a piece of code informally, for example based on previous interactions with the code or through exchanges with other developers, a developer faced with changing it has no means of actually being provided with this context in a structured and automated manner. Most of the information can certainly be obtained from a variety of individual tools, such as test tools, configuration management systems, or issue trackers, but it must be explicitly requested and is presented in a variety of different ways by different tools. No active feedback exists that continuously guides developers. In this paper, we use the term contextualized coding to refer to the ability of the developer to effectively use contextual information while they work on some changes. The key insight underneath our work is that it is neither desired nor possible to continuously present a code’s context in its entirety; instead, it is necessary to provide an abstracted view of the context that informs the developer of which parts of the code they need to pay more attention to. Four principles guide the realization of contextualized coding in specific tools: (1) automatically and continuously collect relevant information from the systems and analysis tools available, (2) summarize the information at the level of individual lines of code, (3) visualize the information it in a unified, reasonably unobtrusive manner, and (4) allow the developer to obtain further details on-demand. We have implemented a prototype contextualized coding tool, the Code Orb, as a new Eclipse [5] plug-in. The Code Orb uses three dimensions of contextual information to warn the user of the volatility of the code they are working on: (a) code churn, (b) implication with past bugs, and (c) number of different authors. For each of the dimensions, the Code Orb calculates a relative, normalized level of warning, with high code churn [12], high implication with past bugs [15], and low numbers of different authors [14] all indicating increased levels of volatility and attention that the developer probably should pay. The Code Orb then combines these levels

of warning into a single overall level, and presents this context in a visualization directly in the IDE. The levels of warning change with each line of code as the developer navigates and modifies the code. The rest of this paper introduces the principles behind contextualized coding, presents our prototype Code Orb, illustrates its function with example code and context drawn from Mylyn, and then concludes with related work and an outlook at our future work.

2. CONTEXTUALIZED CODING Contextualized coding tools can be seen as recommendation systems [4, 8, 9, 10 and 15] that focus on providing a relevant, yet passive context. Whereas most recommendation systems seek to provide specific advice regarding a certain course of action that a developer should undertake, contextualized coding tools simply provide developers with access to relevant additional information based on the code’s past and present context. To build tools that do so effectively, we have identified four principles that such tools ideally adhere to. First, information must be collected automatically and continuously. This information can pertain to all sorts of different context, be it code churn, amount of parallel development having taken place, coverage by test cases, overt connectivity to other modules, and so on – as long as each of these represents a dimension along which it has been shown (or is suspected) that the volatility of the code depends on it. Our work, thus, builds upon the numerous empirical studies that are now available and show the impact of these different dimensions (e.g., [2, 4, 8, 10, 12 and 14]). Second, contextualized coding relies on the summarization of the information at the level of individual lines of code. Not every line of code in a file is as volatile as other lines of code; some are historically buggier that others, less covered by test cases, or simply have been edited by just one person over many months of time as compared to other lines of code in the same file that were touched by multiple developers recently (and thus underwent the scrutiny of multiple eyes). Different parts of a file, thus, may have a very different context. Third, information must be visualized in a unified and reasonably unobtrusive manner. The developer should be able to quickly and without losing focus from their main activity, programming, take in the context of a certain piece of code. Given the different kinds of information that are available and the many different ways in which this information could be presented, it is important to settle on a unified presentation mechanism that conveys its meaning ata-glance. Finally, the developer should be able to obtain further details ondemand. While a summary view is crucial, a developer has to also be able to drill down into specifics, for example examining which kinds of bugs certain lines of code have been involved in, inspecting the test cases covering the code at hand, or finding how often a piece of code has been changed over the past few days. It should require minimal effort for them to do so.

a.

b. Figure 1. The Code Orb.

3. THE CODE ORB The Code Orb is a plug-in for Eclipse that has as its main display a circular view that is divided into multiple slices, with each slice covering one contextual dimension. The color ranges from green, indicating a low level of warning for that dimension, to red, indicating a high level of warning level for that dimension. The center of the Code Orb provides a number that summarizes all dimensions. The Code Orb in Figure 1a, thus, indicates an overall level of warning that is low, communicating to the developer less volatility for the code than the Code Orb in Figure 1b. The Code Orb is extensible and can accommodate many different dimensions, simply by inserting more slices in the pie. Its current incarnation includes three: code churn, implication with past bugs (bugginess), and number of different authors – each calculated per line of code. Each of these factors has been shown to influence the volatility of code, and each of them therefore provides an indication of the extra scrutiny a developer may need to pay to the code they are working on. Specifically, higher levels of code churn are known to be associated with the higher occurrence of post-release defects [12]; the presence of past bugs is known to be an indicator of higher probability of future bugs [15]; and the lower the number of authors, the higher the possibility of defects [14]. To request more information regarding a particular dimension, the developer can simply hover the mouse over its corresponding part of the Code Orb.

3.1 Data Sources The Code Orb is an extensible tool that calculates data for each dimension based on information stored in several types of support systems. In its current prototype we are mining information collected from the versioning system and the issue tracking system. However, the Code Orb can be easily extended to include information collected from static and dynamic analysis tools such as test coverage analysis tools like EclEmma [4] or fault localization tools like Tarantula [9]. From the versioning system, the Code Orb obtains the history of each line of code, specifically by analyzing all of the diffs in the archive and determining how often each line of code has changed. It then mines the commit logs to associate the changes, and thereby indirectly individual lines of code, to specific bugs and to identify the various developers who have edited each line of code. Our strategies are similar to that of [17] for identifying changes made from file version to file version, and to [15] for deducing the associations between bugs and changes. Clearly, there are no absolute bounds on the “maximum” volatility that we could encounter. Some code files may have exhibited only a few bugs, others hundreds. Some may have been edited by just a single person, others by numerous developers. These numbers will also differ per system under development. To map these different ranges to a uniform green to red spectrum, we therefore normalize the values per file that the Code Orb is used on. For each of the dimensions, the Code Orb calculates a warning level between 0 and 1. To do this, it first calculates a raw value for each dimension for a line of code, and compares it with the values for the rest of the class. The Code Orb then calculates ‫݌‬ሺ‫ ݔ‬൑ ‫ݒݎ‬ሻ, the probability that another random line of code in the same class would have a raw value ‫ ݔ‬smaller than the ‫ ݒݎ‬for this line (this is also known as the cumulative probability), using a Poisson distribution with λ= 1. Finally, it maps this probability to an RGB color between green and red.

This kind of normalizing within a class represents an initial design choice, one that we expect to fine-tune by comparing to other possible approaches, such as normalizing across the entire system, a package, or across certain slices of time. In addition, we clearly will need to validate whether our assumption regarding the distribution of green versus red following a Poisson distribution indeed is true, or whether another distribution is present.

a link to the full issue description. For the ownership dimension the Code Orb shows the number of owners and the id and email of each owner. Finally for the code churn dimension, the Code Orb shows the file version numbers for each change made to this line of code and a link to open a diff between the current version and each previous version.

Note that we do recognize that different bugs have different impact. We therefore capture the priority and severity of each issue in the issue tracker, and weigh bugs with higher priority and severity more than bugs with lower priority and severity.

To understand the behavior of the Code Orb, we selected a project in which we had full access to its code, version history, and issue tracking history, Mylyn [11]. Mylyn is a non-trivial project, and is particularly useful since each commit log has an associated issue number. We then used CVSScan [16] to visualize Mylyn’s history to find classes with an interesting context. Specifically, we looked for classes that: (1) were relatively large (over 1000 lines of code), (2) had been modified by at least four developers over the past six months, (3) had over 100 versions total, and (4) involved at least 20 bugs. Out of the resulting set, we selected two classes, TaskEditor.java and AbstractTextEditorPane.java, and loaded them into the Code Orb.

3.2 Presentation To allow for unobtrusive presentation of the context and scalability for multiple dimensions, we use peripheral displays around and anchored to the code editor. The Code Orb is inspired by outlying displays of videogames; orbs and radars in videogames are circular or spherical views that, in the periphery of the actual window of action, present information to the player by using various color and shape patterns. The Code Orb includes a main view and an editor column (Figure 2). As discussed earlier, the main view consists of a circular graph that shows the contextual information for the current line(s) of code selected in the editor. The graph is divided into slices, each with a color representing the warning level for a single dimension. In the center, the orb includes an overall warning level from 0 to 100, calculated using the weighted average of the dimensions in the slices. The main view of the Code Orb is presented as part of the IDE at the periphery of the main editor, as shown in Figure 2a. The location can be changed simply by moving the view.

4. DISCUSSION

Figure 2 shows the Code Orb when we applied it to TaskEditor. This class has 195 versions, was modified by five developers over the last six months, and was associated with over 50 bugs. The second class, AbstractTextEditorPane.java, has over 160 versions, was modified by five developers as well, and had only been associated with slightly over 20 bugs. By examining the output of the Code Orb in detail, we learned a number of initial lessons. First, we found that much of the code was green, and has not actively changed in quite some time. Though this is unsurprising, it confirms that the Code Orb does not inadvertently draw attention to where it should not. Code that has been relatively stable for quite some time was classified as non-volatile. Interestingly, this included many of the attribute and field declarations, as well as method headers, with the exception of a small number of them which had not been changed often, but when they were changed, were changed to fix bugs. Second, we found some parts of code that were relatively yellow overall, but when we looked in detail had one dimension particularly high. For example, the orb in Figure 1a was for a line of code that had been involved in several changes, all for enhancements, so it had a high code churn but no warning for bugginess or ownership. This is an interesting pattern, and shows the usefulness of presenting all dimensions together in the Code Orb.

Figure 2. The Code Orb. The editor column is included because the main view of the Code Orb shows the warning levels only for the selected lines of code, a behavior that is desired, but does not provide the full picture. It particularly has the limitation that it requires a developer to move up and down the editor to fully understand the context for an entire file. The code editor column preempts this manual activity by showing the overall warning level for each line as a color gradient from green to red (Figure 2b); that is, the number in the center of the main view is translated to the spectrum of green to red to be shown in the editor column. Although a warning level is a useful mechanism to quickly and unobtrusively give a developer an idea of what the volatility of the code is, they must have access to further information that presents the details underlying the warning. The Code Orb allows developers to hover over a slice to obtain this detail. In the case of the bugginess dimension, the Code Orb shows, for each bug associated with this line of code, its id in the tracking system, its title and

Third, we also found some code with higher warning levels (Figure 1b showing the highest we found). Again, this is unsurprising, though it is interesting to examine the code where this happens in more detail. In some cases, the warnings concerned methods with large and complex structures, which gave rise to a large number of changes because of bugs in the code. In other cases, the lines with high warning were neither complicated nor particularly interesting from a first look. The fact that these lines of code also received high warnings because they, too, had been changed and involved in multiple bugs in the past, indicates that the Code Orb adds to what a developer may glean just casually examining the code. It is precisely in these kinds of situation that the Code Orb can assist the developer in avoiding unpleasant surprises later.

5. RELATED WORK Contextualized coding and the Code Orb are related to previous work in risk awareness and code smells, recommendation systems, code visualization, and mining software repositories.

Zeller and Zimmerman’s work [15] on risk awareness is probably the closest idea to ours. Similarly to the Code Orb, their plug-in Hatari highlights code that has been related to bug fixes by inserting a column in the Eclipse editor column. Other systems that also show such warnings at the line of code level include EclEmma [4] and Tarantula [9]. While each of these systems is very useful in its own right, their combination highlights why the Code Orb is needed: since each adds a column to the Eclipse editor column, they together (especially if yet other systems are involved) quickly overload this view and the cognitive ability of the developer to interpret it. Finally, the Stench Blossom [18] provides a visualization of code smells at a fine-grained level in an unobtrusive manner. Compared to the Code Orb its focus is only on showing indicators obtained through code analysis and not on the wider spectrum of dimensions we define as code context. As already highlighted in Section 2, contextualized coding tools are a kind of recommendation system. Indeed, Happel and Maalej [7] would classify the Code Orb as a recommendation system that “assists with provision of artifacts and quality measures.” For the former, it supports linking code to other artifacts such as issues, bug descriptions, and commit logs. For the latter, it provides basic quality measures in the form of warning levels. Other recommendation systems (e.g., Mylyn [11], Dhruv [1], Lighthouse [13]) also aim to assist developers while coding. The Code Orb is different because its context is fine-grained and anchored to individual lines of code, not to proxies such as emergent design in Lighthouse, or issues and bug reports in Mylyn and Dhruv. Previous work in software visualization is also related to the Code Orb. Tools such as SeeSoft [6] operate at the level of individual lines of code much like our approach. However, unlike the Code Orb, the objective of most of these pixel metaphor visualizations is to present information for as many lines of code as possible in a single screen. It is therefore difficult for the developer to assess an individual line of code, and, even though they support developers in understanding the code, we believe these visualizations are illsuited to effectively supporting developers while they are changing the code. Finally, our work is related to studies that mine software repositories. First, as we have highlighted, the Code Orb builds upon the results of previous studies, including [2, 8, 10, 12, and 14], which have as objective predicting defects based on project history. We also find commonalities with other work that provides visualizations of code history and its evolution such as [16] and [3]. However, unlike these previous works that aim at predicting faults or understanding the history of the whole project, our objective is to use this information to support the developer who is actively involved in changing code.

6. CONCLUSIONS & FUTURE WORK The Code Orb is a contextualized coding tool that builds upon the idea that a summarized, line-by-line sensitive context could well influence how a developer approaches and changes code. In this paper, we have introduced a first prototype of the Code Orb, given an initial impression of how it works with Mylyn as the example, and anchored the discussion in four principles of contextualized coding. The elegance and minimal IDE footprint of the Code Orb are its strengths, as it easily integrates in the work environment. Clearly, this is merely a first step. First and foremost, our future work involves bringing the Code Orb to actual developers to understand if and how they use the Code Orb in their programming activities. Second, we wish to expand the set of dimensions that it

presents so that the basis upon which developers can make decisions is richer. Third we want to explore the impact of changing how metrics are converted into different color gradients, and how factors such as different weighting strategies impact the developer’s perception of code volatility. It is important, however, to complement this expansion with an analysis as to how independent or dependent the various measures are; we do not want to inadvertently amplify one cluster of dimensions that are strongly correlated over others that are independent but could be as important.

7. REFERENCES [1] A. Ankolekar, K. Sycara, et al. 2006. “Supporting online problem-solving communities with the semantic web”. WWW '06, USA, pp. 575-584 [2] C. Bird, N. Nagappan, et al. 2010. An analysis of the effect of code ownership on software quality across windows, eclipse, and Firefox. Technical report, UC Davis. [3] F. Chevalier, D. Auber and A. Telea. 2007. “Structural Analysis and Visualization of C++ Code Evolution using Syntax Trees”. In IWPSE '07, pp. 90-97. [4] EclEmma Project: EclEmma – Java Code Coverage for Eclipse, Ret. Dec. 2010, from: http://www.eclemma.org/ [5] Eclipse project: The Eclipse Foundation open source community website, the Eclipse Foundation, 2010. Retrieved Dec. 09, 2010, from: http://www.eclipse.org/ [6] S. Eick, J. Steffen, and E. Sumner. “Seesoft a tool for visualizing line-oriented software statistics”. IEEE Transactions Software Engineering, 18(11), pp. 957-968, 1992. [7] H. Happel, and W. Maalej. 2008. “Potentials and challenges of recommendation systems for software development”. RSSE ’08, USA, pp. 11-15. [8] H. Hata, O. Mizuno, and T. Kikuno. “Fault-prone module detection using large-scale text features based on spam filtering”. Empirical Software Engineering. 15(2),p147-165,2010 [9] J. Jones, and M.J. Harrold. 2005. “Empirical Evaluation of the Tarantula Automatic Fault-Localization Technique”. In ASE '05, USA, 2005, pp. 273-282. [10] S. Kim, T. Zimmermann, et al. 2007. “Predicting faults from cached history”. ICSE '07, pp. 489-498. [11] Mylyn Project: Eclipse Mylyn Open Source Project, the Eclipse Foundation, 2010. Retrieved Dec. 09, 2010, from: http://www.eclipse.org/mylyn/ [12] N. Nagappan and T. Ball. 2005. “Use of Relative Code Churn Measures to Predict System Defect Density”. ICSE '05, USA, pp. 284-292. [13] T. Proença, N. Moura, and A. van der Hoek. 2010. “On the Use of Emerging Design as a Basis for Knowledge Collaboration”. New Frontiers in Artificial Intelligence: JSAI-isAI 2009 Workshops. pp. 124-134. [14] F. Rahman, and P. Devanbu. 2011.“Ownership, Experience Defects:A fine-grained Study of Authorship”.ICSE'11,USA. [15] J. Śliwerski, T. Zimmermann and A. Zeller. “HATARI: raising risk awareness”. ACM SIGSOFT Software Engineering Notes, 30(5), pp.107-110, 2005. [16] L. Voinea, A. Telea and J. van Wijk. 2005. “CVSscan: visualization of code evolution”. SOFTVIS '05, pp. 47–56. [17] G. Canfora, L. Cerulo and M. Di Penta. 2007. Identifying Changed Source Code Lines from Version Repositories. In MSR '07. 14 pages [18] E. Murphy-Hill and A. Black. 2010. An interactive ambient visualization for code smells. SOFTVIS '10, USA, pp. 5-14.