Submitted to ACM TOSEM. Earlier version: Working Paper 94/4, Department of Computer Science, University of Waikato. Please do not copy or pass on to anyone else without authors’ permission.
Constructing Integrated Software Development Environments with Dependency Graphs John C. Grundy and John G. Hosking
Abstract A new approach to building Integrated Software Development Environments (ISDEs) is described. Graphbased representations of software system data, and multiple views of this data, are kept consistent by having graph components respond to descriptions of changes made to related graph components. This technique supports integrated, bi-directionally consistent graphical (interactively-edited) and textual (freeedited) views of software development, a generic undo/redo facility, incremental attribute recalculation, and version control and collaborative software development facilities. An object-oriented framework is specialised to construct new environments, and experience using this framework to build several ISDEs is discussed.
1. Introduction 1.1.
ISDEs
Integrated Software Development Environments (ISDEs) provide tools for various software management tasks, such as analysis, design, implementation, debugging, maintenance and version control [23, 7]. ISDEs usually support both graphical and textual representations, or views, of parts of a software system [29, 26]. Support for many views and view types is usually required, together with some form of consistency management between views that share information [23]. ISDEs also require a mechanism for integrating new or existing tools into the environment and for extending tools within the environment. 1.2. Constructing ISDEs Traditional programming environments support text-based program development with loosely integrated development tools [7]. Purpose-built environments, such as THINK C [33], are tightly integrated but very difficult to extend [30, 23]. Language-based environments, such as Smalltalk [9], are relatively easy to extend, but lack specific abstractions for environment extension and integration. Smalltalk’s MVC model is very general and, while able to support multiple views of software development, offers few abstractions for modelling software system data or for maintaining consistency between textual and graphical views. CASE tools, such as Software thru Pictures [35] and the OOATool [6], provide graphical view consistency, but usually support only simple generation of program fragments. This results in a lack of consistency management and traceability between design and implementation views. Reverse-engineering of design diagrams from program code partially solves this problem, but does not support fuzzy change propagation (changes that can not be directly translated from design to implementation and vice-versa). For example, consider the addition of a method calling relationship between two classes at the design level. An environment can not automatically add the code-level method call to the text of one class, as it does not know where in the class’s method code the call should go, nor the correct arguments to the call. Recent research in ISDEs has been concerned with abstract specification of software system structure and semantics, managing the trade-off between integration and extensibility, providing integrated textual and graphical views of software development, and supporting version control and collaborative software development. Declarative specification and generation of language-based environments has usually been based on the abstract syntax of a language’s structure, together with attribute grammars for specifying and
Authors’ addresses: J. Grundy, Department of Computer Science, University of Waikato, Private Bag 3105, Hamilton, New Zealand,
[email protected]; J. Hosking, Department of Computer Science, University of Auckland, Private Bag, Auckland, New Zealand,
[email protected].
-1-
recalculating semantic information. Examples are the Cornell Program Synthesizer [32] and Mjølner environments [21]. These environments are, however, text-based, and thus require additional graphical tools for software analysis and design, which introduces problems for environment integration. MELD [17] uses abstract grammars to provide flexible multiple views of a program, but these are restricted to structure-edited, textual views. LOGGIE [4], Dora [26], PECAN [27], and GLIDE [19] utilise structureoriented editing of views of shared, graph-based program representations. These systems support graphical and textual view consistency by propagating editing changes between views. Weaknesses are that fuzzy change propagation can not be indicated, such as changes to design views affecting code views in a nonspecific way, and restrictive structure-editing techniques have to be employed for all program representations [36]. Semantic Program Graphs (SPGs) [24] use a hypergraph-based notation to represent both the executable and unexecutable aspects of software systems for ISDEs. While a flexible approach to software system data specification, SPGs do not directly describe how multiple textual and graphical views of a software system are defined, manipulated and kept consistent. FormsVBT [3], built using the Zeus framework [5], supports interactive editing of graphical views and free-editing of textual views. View consistency is via token substitution in textual views and incremental redisplay of graphical views. Graphical view updates must be locked out when a textual view is edited, however, and only a simple S-expression language can be supported for the textual “program”. FIELD environments [30] give the appearance of tight integration with extensibility via selective broadcasting of events between Unix tools. A lot of effort is required to build “wrappers” around tools in order to integrate them, however, and the definition of broadcast events is complex and somewhat ad hoc [23]. It is unclear whether multiple textual and graphical views of software development can be integrated as effectively in FIELD as in Dora, PECAN or FormsVBT. FIELD does not yet support version control or collaborative software development. Dora supports a limited form of collaborative software development but not version control. Mercury [18], [25], and Mjolner [22] environments support collaborative development via module interface broadcasting or multiple, shared versions. These environments currently support only textual program implementation. 1.3. The MViews Approach We describe a new model for constructing ISDEs, called MViews, which represents software systems using a graph-based form. The syntax, semantic attribute values and multiple views of a software system are all represented as graph components, allowing a wide range of programs to be described. View graph components are rendered in either graphical or textual forms, and are modified by user operations. Updates on these view graph components are translated into operations on the underlying software system graph components, and multiple views of this underlying graph are updated to maintain consistency. Consistency management between updated graph components is supported by a novel change propagation mechanism. Whenever a graph component is modified, a description of the change the component has just undergone is propagated to all related graph components. These related components then respond to the change descriptions by updating their own state to maintain consistency. This response mechanism is often abstracted into the relationships between graph components, resulting in a flexible, reusable consistency management mechanism. The mechanism also supports a novel way of keeping textual and graphical views consistent. Storage of change descriptions by graph components supports a wide range of other ISDE consistency management functions. These include a generic undo/redo facility, modification histories, version control, tool integration, and collaborative software development. Flexible environment extensibility is supported by this approach. Components and views can be added or modified without affecting existing structures, as change descriptions are propagated to all related components with components often having no direct knowledge of the other components they are related to. New environments are constructed by specialising an object-oriented framework, which provides a consistent, integrated user interface. To provide a context, the following section briefly describes an example ISDE constructed using MViews. An overview of MViews concepts is then given by showing how the example environment is modelled and developed. Constructing MViews-based ISDEs using a reusable object-oriented framework of classes is described, along with the implementation of this framework using an object-oriented Prolog. Experience
-2-
with building several practical ISDEs using the framework is compared and contrasted to similar ISDEs built using other approaches. Current and future research is then summarised.
2. An Example MViews Environment SPE (the Snart Programming Environment) is an ISDE for Snart [10], an object-oriented extension to Prolog. SPE demonstrates the kind of environment that can be built using MViews. A more detailed description of SPE may be found in [12]. Figure 1 shows a screen dump from SPE during the development of a drawing editor program. Several windows are shown, each corresponding to different views of the program. The ‘window-root class’ graphical view shows an analysis-level diagram representing important generalisation (inheritance) and aggregation (attribute type) structures for drawing program classes; the ‘figure-Drawing’ view shows a design-level diagram describing method calling protocols for rendering figures in windows. The ‘drawing_window-Class Defn’ textual view shows a detailed class interface; the ‘drawing_window::add_figure-Method’ a method implementation. Graphical and textual debugging views, and textual documentation views, are also provided. Graphical views use an icon and connector representation. They include a palette, used for selecting tools for interactive manipulation of the view contents. Textual views are free-edited and parsed. Inter-view navigation is either via menus or via an automatically constructed point and click hyper-link system, which allows access from any view component to any other view also containing a representation of the component. There is no restriction on the number of views able to be constructed and displayed, with the contents and layout of each view under user control.
Fig. 1
A screen dump from the Snart Programming Environment (SPE).
As any given item of information may be represented in several views, SPE must keep these different views consistent under change. To achieve this it ensures that: • changes are propagated to all affected views, so no inconsistent information is manipulated by or presented to the programmer • where appropriate, views are updated automatically, so the programmer need not do this manually • changes made to a view that can't be automatically applied to other views (eg propagation of a design view change to a code view) are indicated in some way, so that the programmer is immediately aware of potential inconsistencies After a view change, other graphical views are usually updated automatically by SPE. For example, when a class feature is renamed in one view, any other graphical representations of this feature are automatically updated. Many changes can be automatically applied to textual views, such as renaming classes and features, adding or deleting features, and changing the type of a feature. Some changes cannot be -3-
automatically applied, for example when a client-supplier relationship (to be implemented by a method call) is added between classes in a graphical design view, the corresponding change to the textual view cannot be automatically inferred. In this case, a description of the change is inserted into the view’s text, to indicate that it is affected by the change. An example of this is shown in figure 2. These change descriptions act as a visual cue for the programmer to manually implement the change (for example, by adding an appropriate method call to the view). No other ISDEs, to our knowledge, provide this level of flexible graphical and textual view integration and consistency management.
Change descriptions inserted into view's text
Fig. 2. Textual view consistency in SPE.
SPE provides modification histories for classes which show all of the changes that a class has undergone. Some recent extensions to SPE include multiple versions of classes, features and views with automatic merging support. Collaborative software development is supported by sharing multiple class and view versions, and by broadcasting change descriptions to other developer’s environments [11].
3. An Overview of MViews 3.1. Data Representation We have developed a new model for designing and constructing ISDEs like SPE, called MViews. MViews represents ISDE data as a collection of (possibly disjoint) directed graphs, similar to LOGGIE, Dora and GLIDE, so that it can support both hierarchial, tree-based textual languages and graph-based graphical languages. Software system data is represented as components (nodes) connected by relationships (labelled edges). Each component has attributes (name/value pairs) associated with it. Figure 3 shows an example of an MViews graph for part of the drawing editor program of Figure 1 modelled in SPE. Relationships are specialised forms of components that can be connected by other relationships and have attributes, and thus we collectively refer to all graph nodes and edges as components. MViews graphs are used to represent both the structure of a software system and some of the semantic values associated with these structures, similar to the calculated attributes of decorated abstract syntax trees used in attribute grammar systems [15, 32]. For example, the full interface (signature) of a class in SPE must be calculated from attributes and methods the class itself defines, and those it inherits via its generalisation relationships with other classes. This interface list can be represented by MViews as a list attribute of the class or a relationship from the class to components representing each calculated interface feature. The calculation of such values is described shortly. MViews supports both fine-grained software system representation, using the graph-based structure, and coarse-grained representation using text components. The latter are graph nodes containing a sequence of textual characters. For example, in Figure 4, an extension of the graph of Figure 3, the drawing_window class and drawing_window::clicked method have text components which equate to the formatted text of their concrete representations. This approach allows aspects of programs which need to be individually edited and have semantic values attached to them to be represented with individual graph components. Aspects of -4-
the system which are free-edited and used only as a single component are represented in a coarse-grained fashion, for much greater efficiency. In SPE, for example, method code, such as the drawing_window::clicked method of Figure 4, and textual documentation, are represented only as text components. Other approaches, such as Dora, GLIDE, Mjolner and LOGGIE are restricted to fine-grained representations, as all of their views are structure-edited and thus require fine-grained program storage. Key:
Attribute(Value)
name(window)
class
Component
parent(s) generalisations
Relationship child(ren)
name(drawing_window) ...
class
features
name(figures)
name(figure)
name(clicked)
feature class
feature
type(attribute)
type(method)
class(window, features([ ... ])). class(drawing_window, parents([window]), features([ figures:list(figure), clicked, ... ])). class(figure, features([ ... ])).
calls
type type(list) Abstract MViews Representation
Concrete Snart Representation
Fig. 3. Representing SPE data using MViews program graphs.
Aspects of the system can be represented in both coarse and fine grained manners simultaneously. For example, in SPE, class interfaces are represented as both text components, such as the one for the drawing_window class in Figure 4, and as fine-grained graph components. This allows the format of the class definition, including comments and user-defined layout, to be retained. Consistency is maintained between the two representations via the view consistency mechanism described below. name(window)
class
generalisations
class(drawing_window, parents([window]), features([ figures:list(figure), clicked, ... ])).
name(drawing_window) class text features
text forms
name(figures)
name(clicked) feature
feature
type(attribute)
type(method) calls
drawing_window::clicked(...Args...) :...
text forms text
DrawingWindow@figures(FigureList), ...
Fig. 4. Coarse and fine-grained representations in MViews program graphs.
3.2.
Multiple Views
MViews uses a three-layer architecture to provide multiple views. Figure 5 shows an example of multiple views as used in SPE. A base layer provides a shared representation of the software system as a graph. View layers are graphs representing the information needed for each view, i.e. base layer components have corresponding view layer components. In Figure 5, three such view layers are shown. Display layers act as both renderers and interactive editors of the view layer components with which they are associated. External views provide an interface mechanism for environment tools not implemented using MViews.
-5-
External Tool
Display/ External Layers
External Interface (Data/Event interchange)
...
class_icon
gen_glue
View Layers
drawing_window class_text
class_icon
view rel.
external_class
feature_text
window
view rel.
view rel.
view rel.
drawing_window
view rel.
text
class text forms
generalisations features
Base Layer
text forms
class window
feature
text
Fig. 5. Multiple views of an SPE program in MViews.
View layer components (view components) are connected to base layer components (base components) via view relationships. These allow view components to access base component data, and permit changes to be propagated bi-directionally between a base component and its view components. Base components typically have several view components in different views, with each view component representing a subset of its corresponding base component's attributes and relationships. View components may be chained together to abstract away from specific kinds of base components, and then rendered, forming a flexible view abstraction mechanism. View components may also act as filters on base component data, similar to MELD’s dynamic (database) views [17]. MELD dynamic views provide multiple views via a database-like filtering mechanism, with base information updated via structure-editing and then dynamic views are recalculated to reflect this change. MELD’s dynamic views are often read-only, however, due to the view update problem where base component information is lost by the filter. MViews filters can often retain view relationships back to the base components, allowing aspects of them to be updated. The display layer supplies both a rendering and an editing mechanism for view components. MViews utilises free form editing of textual views, allowing existing editors to be used for textual programming and because this interaction style is generally preferred by programmers [36]. MViews utilises interactive editing for graphical views, with view components modified by interactive edits on their renderings, which is generally preferred by programmers [2]. This is in contrast with most other ISDEs, such as Dora [26] and Mjølner [21], which usitlise restrictive structure-oriented editing approaches for all views. MViews does support structure-edited views, however, if desired. 3.3.
Graph Operations and Update Records
Software system structures are modified by applying operations to graph components. MViews components support a number of fundamental graph operations, shown in Table 1. Operations are of the form C.O(Arg1, Arg2, ...) where component C has operation O applied with parameters Arg1, Arg2, etc. New environments can define more complex operations made up of sequences of fundamental graph operations. Display layer edits are translated into operations on view layer components by the view’s editing tools or parser. Graphical view editor modifications are translated into view component operations by generic editing tools supplied by MViews, or by environment-specific editors. Free-edited textual views render text components obtained from base components, and parse these to update any of the fine-grained aspects of their base components.
-6-
Operation
Update record generated
Comp.create(Type) Comp.delete Rel.establish(Type,Comp1,Comp2)
add_component(Comp) delete_component(Comp) establish(Rel,Comp1,comp2)
Rel.dissolve(Type,Comp1,Comp2) Comp.update(Name,NewVal)
dissolve(Rel,Comp1,Comp2) update(Comp,Name,OldVal,NewVal)
Table 1.
Description Create a new component Comp of type Type (e.g. base_class) Delete component Comp (and dissolves all relationships to it) Establishes a relationship using Rel between Comp1 (parent) and Comp2 (child) Dissolve a relationship between Comp1 and Comp2 Update value of attribute Name of Comp from OldVal to NewVal
Fundamental operations on MViews graph components.
When an MViews component is updated by an operation, other components dependent on its state must be updated appropriately to maintain inter-component consistency. When a component is modified, all affected views must be updated to reflect the change [27, 34] and affected semantic attributes must be recalculated [4,32]. This change propagation process should be as efficient and automatic as possible [15]. To allow for flexible environment extensibility and integration, MViews components broadcast a description of an operation’s effect to all the relationships they participate in. These relationships respond to this change description by updating their own state, or related components’ states, to maintain consistency. Graph relationships are thus not only used for structural information, but also used to describe intercomponent dependency for change propagation. Whenever an operation is applied to a graph component, the operation generates a description of the change it has made, called an update record. The update records of fundamental graph operations are also shown in table 1. An update record is a tuple of the form UpdateKind(Component,Value1,Value2, ...), where: • UpdateKind indicates the operation that was applied to the component (update attribute, establish relationship, delete component, etc.) • Component is the updated component which generated the upate record • Value1 etc are operation-specific values describing exactly what change the component underwent For example a Comp.update(Name,NewValue) operation (i.e. update of attribute Name of component Comp) generates the update record update(Comp,Name,OldValue,NewValue), where OldValue is the previous value of Name for Comp. Generated update records are propagated, via the graph relationships, to components which need to modify their state in response to the original graph update. Components receiving update records can interpret them in any way and apply further operations to maintain consistency with the changed component. Update records are first propagated to the relationships that an updated component participates in, called the dependents of the updated component. These dependent relationships respond to the update record by either: updating their own states, or those of other connected components, by applying operations; propagating the update record to all other components participating in the relationship; or ignoring the change in the updated component. 3.4.
Consistency Management
Figure 6 shows an example of change propagation between components in SPE: 1) an SPE user edits a view component causing operations to be applied to the view component; 2) update records are generated and propagated to the view component’s dependents, including its view relationship; 3) the view relationship translates the view component update into operations on its base component (if necessary) 1; 4) the base component operations generate update record(s); 5) these update records are propagated to the base component’s dependents, including all of its view relationships; 6) these view relationships translate the base component update records into operations on their view components (if necessary) 2; 7) the updated view components re-render their displays. An external view component sends change messages to an external tool or translates the changed data into changed external tool data.
1If the update is view component-specific, for example changing an icon’s location or the font of some text, then the base
component is not affected by the change, and the view relationship does not propagate it to the base component. 2 If the view component is not affected by the base component change, for example the view component does not use an
updated base component attribute, the view relationship does not propagate the change to the view component.
-7-
7. 2.
1.
class_icon
7.
gen_glue
7.
2.
View Layers
external_class
class_text
6. class_icon
6.
6. 3.
view rel.
view rel.
view rel.
view rel.
4. 5.
class
generalisations
5. 5. features
Base Layer
class
Fig. 6. Change propagation between components and their views.
MViews' generic view relationship translates updates of base component attributes to updates of view component attributes of the same name (and vice-versa). This behaviour can be modified by defining new view relationships which support, for example, automatic addition of view components when base components are added. Filtering view relationships can be developed in the same way. Generic aggregation relationships propagate update records on child components to the parent of the relationship. For example, updates on feature components in SPE are propagated to the owning class component. Generic attribute dependency relationships between components use update records for efficient incremental attribute recalculation. For example, addition of a new feature to a class results in the recalculation of a class interface, which is dependent on changes to the class’s relationship with its features. MViews thus provides the speed and flexibility of an operational approach to the inter-component consistency problem, yet allows generic, relationship-type-specific constraints to be easily specified and reused. 3.5.
Other ISDE Facilities
The update record propagation mechanism supports many other ISDE facilities, including: component modification histories, undo/redo, lazy consistency management, version control, and collaborative software development. These applications are described in more detail in Section 4.5. As ISDEs are typically made up of several tools, including different views, environment integration should be at both the user interface and tool data levels [23, 30]. MViews supports ISDE integration via the base layer and integrated, consistent user interfaces through display layers. MViews environments are extensible, as new views or components can be connected to existing components via relationships, without changing the old components. External tools can be interfaced via external views, with update records translated to or from external tool events, or MViews data exported and imported using a translator. MViews environments may also have more than one base layer, with base components in different base layers linked via relationships. This allows independently developed environments to be integrated via relationships between base components, with these relationships providing incremental translation of data changes and events between different tools. An example of using this technique to integrate SPE and an independently developed Entity-Relationship modeller is described in [13].
4. Constructing ISDEs with MViews New MViews environments are constructed by reusing and refining classes from an object-oriented framework. Once a new ISDE has been designed using the MViews model, classes to represent base and view components for the new environment are specialised from MViews framework classes. 4.1. Base and View Component Representation Figure 7 shows an inheritance hierarchy for the major MViews component classes, together with specialisations of the classes used in the implementation of SPE. Some important attributes and methods of -8-
each class are also shown. The mv_component class defines data and functionality common to all MViews components. Specialisations of this class define view, relationship, and layer components. Further specialisations define specific kinds of views, relationships and layer components. Application classes, such as those shown for SPE, are then specialised from the bottom-level (leaf) MViews classes. Additional component characteristics also require specification. For example, the rendering of a graphic icon, such as an SPE class icon, is carried out by the picture method of mv_graphic_icon. This method must be redefined for each kind of graphical view component. Similarly, the parse and unparse methods of textual views carry out incremental parsing and unparsing of text components within the view and require appropriate redefinition. User interaction on views and their components can also be specified by overriding the MViews methods clicked, double_clicked, drag_icon, parse, etc. These then translate user edits into operations on the view components by calling operation methods, such as update , establish , and delete_component.
mv_view_layer
mv_display_layer display
mv_graphic_view
spe_class_icon_view
edit_tools
edit_tools
mv_text_view
hide
spe_class_text_view parse unparse
parse unparse
mv_layer components mv_base_layer
spe_class_icon_view_rel mv_component add_component attributes delete_component
mv_relationship children
relationhips type update
dissolve establish parents
mv_view_rel
mv_many_to_many
spe_features_glue_view_rel
mv_one_to_many
spe_base_features mv_base_comp
mv_one_to_one
spe_base_gen
spe_feature_glue feature_name feature_kind mv_graphic_glue spe_gen_glue
mv_graphic_comp picture
mv_view_comp view
mv_display_comp display
mv_graphic_icon
spe_class_icon class_name class_kind
hide mv_text_comp text_form
spe_class_view_text
mv_layer_comp spe_base_class class_name class_kind mv_base_comp spe_base_feature feature_name feature_kind
mv_text_form unique_id
spe_class_text
spe_feature_text
Fig. 7. Reusable MViews classes (icons with dashed borders) and some specialised SPE classes (icons with solid borders)
4.2. Operations and Update Records The graph operations of Table 1 are implemented as methods in the MViews framework. Some additional operations are implemented in the framework classes which provide management functions for change propagation, efficient list attributes, and simple link (component-to-component) relationships. Whenever an operation method is called, the caller supplies additional information as arguments, for example to indicate the attribute to change, the new attribute value, the kind of relationship to establish, and so on. After the component’s state has been changed by the operation, an update record is automatically generated and propagated to the component’s relationships. The record_update method is called after every operation and is given information to generate an appropriate update record. This update record is then broadcast to all of the relationship components the generating component participates in. These relationships are informed -9-
of the change by having their update_from method called, with the update record and generating component as arguments. The update_from method is over-ridden by components to determine their response to different kinds of update records and broadcasting components. New ISDE component classes can define new operation methods which are built on top of the fundamental operations. These “macro” operations can call record_update to generate an abstract update record for the operation as a whole, although the fundamental operations they use will still generate low-level update records. class
1.
update(class_name, 'dwindow')
update(Name,NewValue) 2. record_update(Update) 4.
4.
3.
4.
features
Update=update(class,class_name, 'window','dwindow')
5.
feature
5.
update_from(Update, FromComp)
feature
5.
...
update_from(Update, FromComp)
Fig. 8. Example of the interaction between MViews consistency management methods.
Figure 8 shows an example of the propagation of update records among SPE component objects in the MViews framework: 1) an update operation is applied to a class by calling the class component’s update method; 2) the update method changes class’ s class_name attribute value and then calls class ’s record_update method with the update record update(class,class_name,’window’,’dwindow’) ; 3) record_update determines the components dependent on class’s state. These are all of the relationships class participates in and any components class is linked to by simple link relationships. This list of components can be cached for efficiency; 4) record_update calls the update_from method of each dependent component with the update record value and class ; 5) dependent components respond by applying operations to themselves or other related components, based on the update record they receive. This update record propagation mechanism is built into the MViews framework, and the ISDE implementer need only define appropriately specialised update_from response methods. Application classes can, however, override any method to specify additional functionality. For example, in SPE the add_component method of spe_base_class is specialised to initialise attribute dependencies for the class’s interface list attribute. This then means that the attribute is recalculated when the class’s feature list or any of its generalisations’ interfaces change. Similarly, the update_from method is redefined to ensure a class has unique feature names, generating an “error” update record if this constraint is violated. Cycles can occur during update propagation. This is currently handled by allowing the fundamental operations, such as update attribute, establish/dissolve relationship and delete component, to occur only once on the same component for each distinct MViews view editing operation. Thus if two attempted updates of the same attribute for a component are made during the same editing operation, only the first actually succeeds and generates an update record, thus breaking the propagation cycle. 4.3. View Consistency The translation of changes between view graph and base graph components is done via view relationhips. View relationships have a default update_from behaviour which translates attribute modifications bidirectionally between base and view components. ISDE developers specify attribute mappings and the type of view relationship to be used to connect view components to their base components. Different types of view components are connected to the same base component using different types of view relationship. This allows different functionality to be supported by the view relationship depending on what the view component requires. For example, automatically adding a view component when a new base component is added can be supported in this way, as can filtering view relationships. Figure 9 shows an example of graphical and textual view consistency in SPE: 1) Feature ‘click’ of a base class is renamed to ‘clicked’ (this update could originate from another view that is not shown). The base - 10 -
aggregation relationship passes this update record up to the base class component which owns the feature, and the base class propagates the update record to its view relationships; 2) One view relationship translates this feature rename into modifications to the feature_names attribute of a class_icon view component; 3) the class icon is re-rendered to reflect this change; 4) the base class feature rename is also propagated to a textual view component; 5) an update record description is expanded into the textual view to inform the SPE programmer of the class change. features
/* updates_start(10) update(2). % rename feature click to clicked update(3). % add feature figures:figure Display */
figure window click name ...
figures
class(window, features( a:string, ...
3.
class_icon
feature_names
5., 8.
class_icon
8.
class_text
feature_glue
View
2. 4., 7.
7. ...
view_rel
view_rel
1., 6.
Base 1., 6.
class 1., 6.
features 1.
feature
Fig. 9.
6. feature
Examples of view consistency in SPE.
If a new feature figures of type figure is added to the base class: 6) a new base features relationship is established between the base class and new feature and an update record is propagated to the view relationships; 7) The class icon view relationship adds a new class icon for ‘figure’ and feature glue for ‘figures’ to the graphical view, and connects these to base classes via view relationships. The class text view relationship propagates the update record to the textual view component; 8) the new icon and glue are rendered while an update record is inserted into the textual view. In this example, the MViews framework classes propagate the update records and the view components handle the view component updates automatically. The ISDE implementer must specify the action to take when expanding the class icon and feature glue, but methods are provided by MViews to do this easily. The ISDE developer can specify the readable description of update records inserted by MViews in textual views, although a default format is built into the framework mv_text_comp class. 4.4 Textual View Consistency MViews framework classes automatically handle most graphical view updates, as the structure of rendered graphical view components is held in the view’s subset layer. For structure-edited textual views, this is also the case. In fact, a structure-edited textual view can be thought of as a graphical view with view components rendered using text rather than icons and glue. It is much more difficult to keep free-edited textual views consistent with changes to other views, as the structure of the view’s components must be recovered via parsing. All update records potentially affecting the text of a view are automatically added to the view’s text in a special header section. Programmers are thus always informed of potential inconsistencies within the view’s text via these messages. MViews can, however, automatically apply some updates to textual views, i.e. update the view’s text, rather than insert a readable description of the update record in the view header. To do this, the ISDE implementer must specify how to parse the view's text to find where to make the modification and how to unparse the update record into changes to the affected text component.
- 11 -
/* updates_start(10). update(2). % rename feature figures to gfigures updates_end. */ class(drawing_window, inherits(window), features( update attribute declaration visible:boolean, buttons:list(drawing_button), figures:list(figure), clicked, display, .... (a) )). replace OldName by NewName in 'class' '(' Name ',' inherits(Parents), 'features' '(' , [ Method | Attribute : Type ',' ]* , NewName ':' Type
1. class(drawing_window, 2. inherits(window), 3. features( 4. visible:boolean, buttons:list(drawing_button) 5. figures:list(figure) (c)
(b)
Fig. 10.
/* updates_start(10). updates_end. */ class(drawing_window, inherits(window), features( visible:boolean, buttons:list(drawing_button), gfigures:list(figure), clicked, display, .... )).
Example of incrementally unparsing updates in SPE.
Figure 10 (a) shows the modification which needs to be made to a view’s text when renaming the attribute figures to gfigures. Figure 10 (b) shows a regular expression which describes the modification that must be made to the textual view in response to an update record of the form update(feature, feature_name, figures, gfigures). Figure 10 (c) illustrates the incremental unparsing process: 1) the class definition header is parsed; 2) then the inherits definition; 3) then the features definition header; 4) a sequence of declarations; 5) until the declaration to modify is found. The value, start position and length of the each lexical token are returned by the lexical analyser, and these are modified appropriately in the text to affect the change. This incremental unparsing process works well when the elements in the text to be updated are uniquely identifiable (such as class and feature names, feature types and method arguments in SPE) or when the elements to update can be identified by context i.e. by the tokens that surround them. For aspects not uniquely identifiable, the text component can be annotated with tokens which appear as comments in the language being processed but have meaning to the lexical analyser. For example, lines with method calls on them can be suffixed (annotated) by the comment % method(ID). The ID value can be used by MViews to locate an appropriate inter-class relationship corresponding to the method call. After editing a textual view, the text component stored in the base graph is updated by copying the new text for the component from the view. The view must then be parsed so that any overlapping, fine-grained information stored in the base graph is updated to reflect the change to the text component. To parse a textual view, a yacc-style parsing grammar is specified by the ISDE implementer. The parser constructs a parse graph of an updated textual view. The ISDE implementer also specifies mapping functions between parse graph components and any base components represented in the texual view. These functions are used to determine appropriate operations to apply to update base components which overlap with the text component. MViews is designed to use existing compilers for code generation and execution. After a textual view has been parsed, MViews may pass the updated view text to an existing compiler for the language under development. MViews can, however, be used to implement a code generator, or can be used to specify dynamic language semantics so base component graphs can be “run” within the environment. This is achieved by specifying additional methods associated with base components which carry out the code generation and/or execution-time semantics. The base components can be augmented with extra attributes and relationships to hold code generation or run-time semantic values in the same way as static semantic values are represented and recalculated. 4.5. Other Applications of Update Records In addition to view consistency, MViews environments make use of update records to implement many other ISDE facilities. As MViews update records are automatically generated by components after modification, they can be used to record changes to view components for an undo/redo facility, can be stored in versions for use in version control, and can be broadcast to other users’ environments to help facilitate collaborative development. They can also be used to drive an incremental attribute recalculation - 12 -
algorithm and their propagation can be delayed to support lazy conistency management. Figure 11 illustrates these other uses of update records in MViews environments. 2.
undo/redo list
class_icon
View view rel.
lazy processing list
3. class 4.
version records
generalisation 1. attr dep.
class
5. class
Another Developer's Base Layer
1. Base
Fig. 11.
Example of other uses of update records in MViews environments.
1. Incremental attribute recalculation is driven by update records propagated along attribute dependency relationships. For example, if a new feature is added to a class, the class interface is incrementally recalculated by adding the new feature name and type to the class interface attribute list. The attr_dependent(Name,DepComp,DepName,RecalcFunc) method is used to establish such attribute dependency relationships, and RecalcFunc is used to recalculate the attribute’s value when necessary. 2. A generic undo/redo mechanism is supported by storing update records generated by modified view components in a modification history list. Undo is supported by sending stored update records back to their generating components for reversal, by calling the undo_op(Update) method. Redo is accomplished by calling redo_op(Update). A transaction roll-back mechanism is also supported by calling abort_op, which reverses the effects of all operations associated with an aborted view edit. 3. Lazy consistency management is useful when consistency management needs to be performed after a sequence of related operations, rather than after each individual operation. For example, performing view consistency lazily allows for more efficient view redisplay when several aspects of a view component are modified by one editing operation. Lazy consistency management is supported by storing update records for propagation to dependents at the end of each MViews view editing operation. This supports lazy view consistency and lazy incremental attribute recalculation. The add_lazy_update(Update) and process_lazy_updates methods are used to manage this lazy update record processing. Lazy processing also allows low-level update records to be composed into higher-level update records, reducing the number of update records propagated and stored. For example, graphical icon update records representing changes to the icon's depth and width could be combined into a single ChangeSize update record. This record can then be expanded, stored and automatically applied as one update record. Dependent components may respond to this composed update record or either one of its composite Update records. 4. Stored update records can be used to create a modification history for components. Grouping these stored update records into version records supports version control, as groups of updates can be undone and redone to produce different component versions. We are currently experimenting with version control facilities in MViews environments based on update records and non-sequential undo/redo of these records (to support version merging and partial version reversal) [11]. 5. Collaborative software development is supported by both sharing versions (to support asynchronous collaboration) and broadcasting update records, as they are generated, to other developer’s environments (to support synchronous collaboration) [11].
5. Framework Implementation The MViews framework has been implemented using Snart, itself implemented in LPA MacProlog on the Macintosh. Snart classes implement component and relationship types, class attributes implement component attributes and methods implement operations. Link relationships are implemented using object list attributes for efficiency. Update records are represented by Prolog terms, and the update_from, undo_op, - 13 -
etc. Methods used to process update record terms are written in a declarative style, by specifying update record patterns to respond to and a sequence of methods to call in response. Graphical component renderings are defined using LPA’s Graphics Description Language (GDL). MViews provides an object-oriented interface to GDL, and building-blocks for constructing graphical editors. These include: icons and interconnection glue; icons and sub-icons; manipulation mechanisms, such as click and drag and tool palettes; hypertext-like click-points on icons; and commonly used graphical component constraints. Textual view components define how update records are applied to their text components using methods similar to update_from. The regular expressions used in the incremental unparsing algorithm are interpreted by Snart methods. Definite Clause Grammars (DCGs) are used to provide the yacc-like facilities for parsing textual view changes into operations on base components. MViews uses a standard Macintosh text window editor for displaying and editing textual views, augmented with menus to assist view navigation and update record management. Base layer and view layer component persistency is supported via persistent Snart objects. Snart objects may be dynamically saved to and loaded from a high-perfomance, single-user persistent object store, requiring no programming by ISDE implementers. Text component data is saved as complex object attributes, and an in-core caching scheme means the performance of persistent Snart objects is almost the same as dynamic objects [10]. Environment evolution and extensibility are supported as Snart provides an automatic translation mechanism for old objects when their class definitions have been modified (for example, when enhancing or extending an MViews environment). This converts old object states into new object states, and is augmented by a MViews base layer class method which converts old MViews graph structures into their new forms. ISDE implementers can specialise this base layer method to allow MViews environments to dynamically translate old environment data into its new form. A C++ prototype of MViews has also been implemented which passes update records as stack-allocated C structures and converts them to objects for storage. This prototype runs much faster than the Snart framework but its functionality is as yet less developed, particularly its user interface and component persistency.
6. Experience with MViews In addition to SPE, the authors have developed several other ISDEs using the MViews model and framework. MViewsER provides integrated Entity-Relationship diagrams and textual relational schema, kept bi-directionally consistent. MViewsDP provides a graphical drag-and-drop interface builder for dialog boxes, with the dialog interface and validation rules being defined in textual views, with both views kept bidirectionally consistent. The dialog specification can be run and tested from within the environment. MViews has also been used by other researchers in the following applications. Cerno-II [8] is a graphical debugger complementing SPE for visualising a running Snart program. Cerno provides not only graphical object representations but abstract visualisation of collection structures, method calls and timing diagrams. EPE is an environment for constructing EXPRESS specifications and corresponding EXPRESS-G diagrams3 [1]. EPE also provides facilities for prototyping an executable implementation of the EXPRESS specifications, and visualising execution of the prototype. Hyper-Pascal [20] is a visual Pascal-like language which provides a variety of graphical programming views together with textual views for input/output format specification. Finally, the Skin programming environment provides a visual/textual functional language for constructing flexible user interface components and prototyping their execution [14]. The largest of these ISDEs, SPE and EPE, provide reasonable performance for developing quite large applications. For example, the largest systems so far modelled in SPE have been MViews and SPE themselves, which together consist of over 60 Snart classes, 1100 class attributes and methods, and almost 400K of source code. 25 graphical views and nearly 200 textual views make up the complete system definition in SPE. SPE’s performance when working with this system is generally good, although reloading and redisplaying complex graphical views from persistent storage can take up to half a minute.
3 EXPRESS and EXPRESS-G are object-oriented specification languages [16].
- 14 -
EPE has been used to model similar sized systems (60-70 classes in 20-30 views representing a generic model of buildings). SPE and the MViews Snart framework were originally developed as proof-of-concept prototypes. Despite this, both systems have been used to develop substantial, useful, and practical environments and software systems. EPE was constructed by specialising both SPE and Cerno-II. This involved the addition of several relationships between classes not modelled in SPE and changes to the rendering of both graphical and textual forms of classes. The developer of EPE did not need to modify the SPE framework, but only needed to specialise SPE classes to redefine renderings (particularly the parsing and unparsing of textual views to suit the EXPRESS language, which is very different to Snart) and add extra attributes and relationships. We have prototyped a collaborative form of SPE, called C-SPE, which allows several software developers to collaborate on software construction [11]. A distributed form of Snart object stores has been prototyped to support optimistic sharing of software system data for collaborative software development. C-SPE requires improvement to the MViews version control and synchronous editing facilities before its usefulness and scalability can be established. Feedback from users of SPE and other MViews ISDEs indicates they find the degree of integration in MViews environments very useful when developing software. They also like the way views are kept consistent, particularly the way potential effects on textual view components are displayed as update record decriptions. The storage of update records in version records allows users to trace back through the editing history of views and modification history of base components, which has also proved useful. Several other researchers have reused MViews to produce some of the environments described above, and have all found the framework concepts simple to understand and reasonably straightforward to reuse.
7 . Comparison to Other Approaches A framework approach is less abstract, and generally requires more effort, than ISDE generation from a formal grammar. However, frameworks provide not only a reusable model but also much greater flexibility. Using a framework, environment implementers are able to code data management and user interface mechanisms differing in style from those originally envisaged by the framework's implementers. In contrast, generated environments are restricted to the capabilities offered by the generator language. These are usually considerably less powerful than framework programming languages. For example, more flexible, practical systems can be developed using Unidraw, Zeus, Garden and FIELD (all based on framework specialisation), than can be developed with MELD, LOGGIE, Dora, Mjolner and GLIDE (all generated from formal grammars) [28, 34]. As MViews environments are constructed by framework reuse, they are very flexible and have a large amount of functionality abstracted into the framework. Integration is supported by the framework’s consistent user interface building blocks and by a shared data repository. Extensibility is supported as new structures can be added without affecting existing components. Environment evolution is also supported: the modification of existing structures generally has a limited effect on related structures; and the framework and Snart automatically handle conversion between class versions. Using external tool integration, MViews environments can support extensibility to the degree of FIELD, but retain more integrated data and user interfaces via multiple views. As noted by Meyers [23], FIELD’s selective broadcasting requires considerable effort to integrate tools and the messages used are somewhat ad hoc. MViews environments use a consistent set of messages, which can be augmented by composing abstract messages from fundamental ones. Dora, constructed using Interviews, supports graphical and textual views of object-oriented program development, but the usefulness and scalability of Dora appears considerably less than that of SPE, as Dora supports less analysis and design facilities and very limited propagation of change between design and implementation views. SPE recently took 3 person weeks to fully reimplement using an updated version of MViews. This is considerably less than the time taken to develop similar environments, such as [25], Dora [26], and FIELD [31]. The MViewsDP dialog box constructor has a similar level of functionality to FormsVBT, written in Zeus. The MViewsDP textual view consistency is more powerful, however, as information which does not overlap with graphical view information can be represented. MViewsDP took less than a person-week to develop with MViews, whereas FormsVBT took several weeks to develop using Zeus [3]. A similar - 15 -
interface builder in Unidraw has additional functionality, but can only generate C code and is unable to keep that code mutually consistent with graphical view changes [34]. Environments can be specified and generated more quickly using formal grammar approaches, as used by CPS, LOGGIE, Mjolner and Hudson’s attribute grammars, than by framework specialisation. ISDE developers must define Snart classes and appropriate methods to build MViews environments. CPS and Mjolner do not support multiple views, however, and LOGGIE supports only structure-edited views. While Hudson’s algorithm is very efficient, and more abstract than MView’s attribute dependencies for static semantic attributes, MViews can be used to more efficiently recalculate complex attribute values. Mjolner grammars can be modified by users to support, for example, a different concrete language syntax, whereas MViews environment classes must be modified to achieve the same affect. Thus more work must be done to adapt MViews environments to individual developer’s requirements. Also, experience with MViews has shown it takes new ISDE implementers longer to understand how to reuse the MViews framework than to use environment generators. However, MViews does allow more flexible, powerful and practical environments to be built. The persistency mechanism provided by Snart performs well for single-user environments, and is less effort to maintain than those of Garden and Dora, which require database translation mechanisms to be employed to support environment evolution. We have built a prototype distributed object store to help support collaborative software development, which provides similar facilities to those of [25, 22]. It is as yet unclear how well this persistency mechanism will perform with large numbers of component version records and many collaborating software developers.
7. Summary MViews provides a new approach to constructing integrated, extensible ISDEs with multiple textual and graphical views of software development. MViews represents software system data and multiple views of this data as dependency graphs. Discrete change descriptions to graph components are propagated to related components which then respond to these changes to maintain consistency. This mechanism supports environment integration and extensibility, as views (tools) share a common base data representation and new views and base components can be added without affecting existing components. Integrated, multiple textual and graphical views of software development are supported with bi-directional consistency management. A generic undo/redo facility, efficient incremental attribute recalculation, lazy consistency management, version control and collaborative software development facilities are also supported. Reuse of the object-oriented MViews framework allows flexible, practical ISDEs to be quickly constructed and maintained. MViews is currently being extended to support multi-user, distributed software development using update records for version control, configuration management, and broadcasting to other developer’s environments [11]. The MViews framework is being extended to provide more abstract support for attribute dependency specification and recalculation, specification of update record composition, and support for improved nonsequential undo/redo for version control. Generation of framework classes from a more abstract visual and textual specification of MViews environments is planned, which will allow further specialisation of generated classes, combining the advantage of a framework with the advantages of (partial) ISDE generation. SPE is being extended to support collaborative, distributed object-oriented software development using these new MViews facilities.
Acknowledgments John Grundy has been supported by an IBM Postgraduate Scholarship, a William Georgetti Scholarship and a New Zealand Universities Postgraduate Scholarship. We also gratefully acknowledge the financial support of the University of Auckland Research Committee and the many helpful comments provided by our colleague Rick Mugridge and the anonymous referees in the preparation of this paper.
References 1. 2. 3.
Amor, R., Augenbroe, G., Hosking, J.G., Rombouts, W., and Grundy, J.C., “Directions in modelling environments,” , Dept of Civil Engineering working paper, 1993. Arefi, F., Hughes, C.E., and Workman, D.A. Automatically Generating Visual Syntax-Directed Editors. Communications of the ACM 33, 3 (March 1990), 349-360. Avrahami, G., Brooks, K.P., and Brown, M.H. A Two-View Approach to Constructing User Interfaces. ACM Computer Graphics 23, 3 (1990), 137-146.
- 16 -
4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35. 36.
Backlund, B., Hagsand, O., and Pherson, B. Generation of Visual Language-oriented Design Environments. Journal of Visual Languages and Computing 1, 4 (1990), 333-354. Brown, M.H. Zeus: A System for Algorithm Animation and Multi-View Editing. In Proceedings of the 1991 IEEE Symposium on Visual Languages, IEEE Computer Society Press, 1991, pp. 4-9. Coad, P. and Yourdon, E. Object-Oriented Analysis, Yourdon Press , Second Edition(1991). Dart, S.A., R.J., E., Feiler, P.H., and Habermann, A.N. Software Development Environments. COMPUTER 20, 11 (November 1987), 18-27. Fenwick, S., Hosking, J.G., and Mugridge, W.B. Visual debugging of object-oriented systems. In Proceedings of TOOLS Pacific 94, 1994. Goldberg, A. and Robson, D. Smalltalk-80: The Language and its Environment, Addison-Wesley, Reading, MA (1984). Grundy, J.C. Multiple textual and graphical views for Interactive Software Development Environments, Ph.D. dissertation, Department of Computer Science, University of Auckland, June 1993. Grundy, J.C., Mugridge, W.B., Hosking, J.G., and Amor, R., “Collaborative, Integrated Software Development with Multiple Views,” , Working paper, 1993. Grundy, J.C., Hosking, J.G., Fenwick, S., and Mugridge, W.B. Visual Object-Oriented Programming, Prentice-Hall (1994), Chapter 11. Grundy, J.C. and Venable, J., “Integrating CASE tools via hierarchical data model integration,” , Working Paper, 1994. Hosking, J.G., Fenwick, S., Mugridge, W.B., and Grundy, J.C., “Cover yourself with Skin,” , Technical Report, 1994. Hudson, S.E. Incremental Attribute Evaluation: A Flexible Algorithm for Lazy Update. ACM TOPLAS 13, 3 (July 1991), 315-341. ISO, “Part 11: The EXPRESS Language Reference Manual in Industrial automation systems and integration - Product data representation and exchange”, ISO DIS 10303-11, August 1992. Kaiser, G.E. and Garlan, D. Melding Software Systems from Reusable Blocks. IEEE Software 4, 4 (July 1987), 17-24. Kaiser, G.E., Kaplan, S.M., and Micallef, J., Multiuser, Distributed Language-Based Environments. IEEE Software (November 1987), 58-67. Kleyn, M.F. and Browne, J.C. A High Level Language for Specifying Graph Based Languages and their Programming Environments. In Proceedings of the 1993 International Conference on Software Engineering, May 1993, pp. 324-334. Lyons, P., Simmons, C., and Apperley, M. HyperPascal: Using visual programming to model the idea space. In Proceedings of the13th New Zealand Computer Society Conference, Auckland, August 1993, pp. 492-508. Magnusson, B., Bengtsson, M., Dahlin, L. An Overview of the Mjølner/ORM Environment: Incremental Language and Software Development. In Proceedings of TOOLS ‘90, Prentice-Hall, 1990, pp. 635-646. Magnusson, B.A.U.M.S. and M.S., M.S. Fine-grained Revision Control for Collaborative Software Development. In Proceedings of the1993 ACM SIGSOFT Conference on Foundations of Software Engineering, Los Angeles CA, December 1993, pp. 7-10. Meyers, S. Difficulties in Integrating Multiview Editing Environments. IEEE Software 8, 1 (January 1991), 49-57. Meyers, S. Representing Software Systems in Multiple-View Development Environments, Ph.D. dissertation, Department of Computer Science, Brown University, 1993. Nascimento, C. and Dollimore, J. A model for co-operative object-oriented programming. IEE Software Engineering Journa 8, 1 (1993), 41-48. Ratcliffe, M., Wang, C., Gautier, R.J., and B.R., W. Dora - a structure oriented environment generator. IEE Software Engineering Journal 7, 3 (1992), 184-190. Reiss, S.P. PECAN: Program Development Systems that Support Multiple Views. IEEE Transactions on Software Engineering 11, 3 (1985), 276-285. Reiss, S.P. GARDEN Tools: Support for Graphical Programming. In Proceedings of Advanced Programming Environments, Lecture Notes in Computer Science #244, Springer-Verlag, Trondheim, Norway, June 1986, pp. 5972. Reiss, S.P. Working in the GARDEN Environment for Conceptual Programming. IEEE Software 4, 11 (November 1987), 16-26. Reiss, S.P. Connecting Tools Using Message Passing in the Field Environment. IEEE Software 7, 7 (July 1990), 57-66. Reiss, S.P. Interacting with the Field environment. Software practice and Experience 20, S1 (June 1990), S1/89S1/115. Reps, T. and Teitelbaum, T. Language Processing in Program Editors. COMPUTER 20, 11 (November 1987), 2940. THINK C Reference Manual, SymantecCorporation Ltd, 1990. Vlissides, J. Generalized Graphical Object Editing, Ph.D. dissertation, Stanford University, 1990. Wasserman, A.I. and Pircher, P.A. A Graphical, Extensible, Integrated Environment for Software Development. SIGPLAN Notices 22, 1 (January 1987), 131-142. Welsh, J., Broom, B., and Kiong, D. A Design Rationale for a Language-based Editor. Software - Practice and Experience 21, 9 (1991), 923-948.
- 17 -
