A graph based framework for the implementation of visual environments J. Rekers Department of Computer Science, Leiden University Niels Bohrweg 1, 2333 CA Leiden, The Netherlands email:
[email protected] http://www.wi.leidenuniv.nl/~rekers/
Abstract We propose an architecture for storing visual expressions within an environment which supports various forms of visual expression editing, like syntax directed editing, free editing, and layout editing. The proposed architecture is based on two related graph data structures: an abstract syntax graph represents the logical structure of a visual expression in terms of its visual language syntax; a spatial relations graph represents the pictorial structure of a visual expression with graphical objects as nodes and spatial relations as edges. We explain the distinction between the two ways of structuring in detail, we show how coupled graph grammars can be used to define, build and relate the two structures, and we explain how an editor for a visual language can be defined on top of these data structures. We use the visual language Message Sequence Charts (MSC) as running example. Keywords & Phrases: Structure of visual expressions, editing visual sentences, application of graph grammars.
1. Introduction An important class of visual languages are the so-called diagrammatic visual languages: each sentence is composed of graphical objects which have certain spatial relations among each other, and the interpretation of such a sentence is fully determined by these visible aspects. Examples of frequently used diagrammatic visual languages are EntityRelationship models of data, State Transition diagrams, SDL descriptions of processes, data flow programs in Show&Tell, and Message Sequence Charts. For a user of a visual language, the two most important aspects of a visual program are the physical layout of the diagram (what the user sees and manipulates), and the meaning of the diagram (what the user expresses with it). Any implementation of the language has to maintain the correspondence between these two aspects. We propose to use two intermediate graphs to connect the physical layout and the meaning of a diagram: • The spatial relations graph (SRG) describes the structure of the diagram seen as a picture; this representation is geared towards the visualization of the diagram. • The abstract syntax graph (ASG) describes the structure of the diagram according to its visual language; this representation is geared towards the interpretation of the diagram.
A. Schürr Lehrstuhl für Informatik III, RWTH Aachen Ahornstr. 55, D-52074 Aachen, Germany email:
[email protected] http://www-i3.informatik.rwth-aachen.de/people/andy/ Abstract Syntax Graph (ASG) represented by
represents
Supports activity: interpretation syntax directed editing
Spatial Relations Graph (SRG) constraint solving
graphical scanning
Physical Layout
Figure 1:
layout editing representation oriented editing low level graphics editing
Our three levels of representation
An implementation of a visual language in which diagrams can be edited, analysed, and executed will benefit greatly if this distinction is kept as clear as possible. We present graph grammars as single formalism for describing the structure of visual sentences, and for describing all kinds of operations on them, all based on the above representations. The choice to use distinct data structures for the different operations has been inspired by the Model-ViewController concept of Smalltalk. In this paper, we will start with an explanation of the proposed data structure and its benefits, and we will show how graph grammars may be used to specify and implement major portions of visual environment tools on top of such a data structure. Furthermore, we will discuss the relationships between nowadays existing visual environments and environment generators and the presented graph grammar based approach of this paper. Finally, we propose a general architecture for specialized visual language environments.
2. The proposed data structure We describe the proposed framework (as depicted in Fig. 1) in more detail and use the visual language Message Sequence Charts (MSC) as running example. MSC [22] is a language for the description of interaction between entities. A diagram in MSC describes which messages are interchanged between process instances, and what internal actions they perform. Fig. 2(a) shows a (very simple) diagram in MSC, consisting of process instances i1 and i2. Process instance i1 first sends a message m0 to the environment, next a message m1 to instance i2, and then dies. Instance i2 starts by receiving message m1 from i1, and next performs an internal action a. Time proceeds down-
wards within a process instance, but there is no time relationship between instances, other than that a message has to be send before it can be received. The abstract syntax graph (ASG) representation of a diagram reflects the internal (logical) structure according to the corresponding visual language. Nodes and edges in this graph representation correspond to language constructs. For EER diagrams such constructs are entity, relationship and attribute; for Message Sequence Charts process instance and message; for finite state automata state and transition. This level describes the high-level structure of the graphical sentence, but does not state what the constructs look like. For instance, the graph of Fig. 2(b) is an appropriate ASG for the MSC diagram of Fig. 2(a). A process instance in the MSC diagram is modelled as an instance edge between the enclosing MSC node and a start node. The next edges denote the time dependencies within a process instance. Messages are independent from the instances, but might form connections between them. This ASG representation contains just everything necessary to interpret the MSC diagram. However, the MSC diagram has to be represented on screen to the user, and in order to do so a physical representation of the diagram must be created. The physical layout of a visual program consists of a collection of graphical objects (line, circle, rectangle, text fragment) with properties such as their location, size, colour, and pen kind. This level defines what the user sees and manipulates. For example, part of the physical layout of the MSC diagram of Fig. 2(a) is shown in Fig. 2(d).
msc simple i1
i2
m0 m1 a
(a) An MSC diagram simple
MSC
m0
The spatial relations graph (SRG) abstracts from the physical layout: instead of the individual logical properties of objects, it represents the higher level spatial relations which hold between the objects (touches, contains, left of, labels). Furthermore, it may contain so-called emergent objects, such as the overlapping area of two circles, or the point at which two lines intersect. An example of a SRG is given in Fig. 2(c). The nodes in this representations are the graphical objects of the physical representation (but for the Pos nodes which are emergent objects necessary to formulate the constraints on the attachment points of the Arrow’s end on the Line); the edges define spatial constraints which should hold between the graphical objects. Note how much more detail the SRG contains compared to the corresponding ASG.
Env
from i1 next
Start
instance
Out
next
next
Out
End
from to
i2 next
Start
instance
m1
Message next
In
next
Action
End
a
(b) The Abstract Syntax Graph of the MSC diagram
Box
contains
simple
Text
i2
white
m0
Text
labels Text
labels
i1
white
Box
bot touch
Box
Text
to
labels
Pos has
above
Arrow
Pos m1
from
above
bot touch has
has
in
has
Line
in
has
Pos
Pos
Text
Line has
above
labels
above to
In order to relate these two, quite different, representations of a diagram some intermediate representation is necessary. This should represent the pictorial structure of the diagram which has to be met by any physical representation of the abstractly represented diagram.
to
Message
has
vertical
Pos
from
Arrow
black
Box
Pos top touch
Pos on
above white
Box
contains a
Text
vertical
has
above
Pos
top touch
black
Box
(c) The Spatial Relations Graph of the MSC diagram Line(start=(245, 455), end=(245, 379)) Text(topleft=(197, 472), strval='i1') Text(topleft=(242, 472), strval='i2') Rect(fill=none, topleft=(188, 462), botright=(213, 455)) Rect(fill=black, topleft=(188, 379), botright=(213, 372)) Rect(fill=none, topleft=(232, 462), botright=(258, 455)) Rect(fill=black, topleft=(232, 379), botright=(258, 372)) Line(arrowatend=1, fill=black, start=(200, 436), end=(168, 436)) ...
(d) The graphical objects which form the Physical Layout of the above MSC diagram. Figure 2:
An MSC and its three representations
{Start, In, Out, Action, End} is a Event
λ
::=
MSC
::=
contains
Box
Text
string val
/* x is a formal parameter of the production */
name
λ
x
x
1)
1)
env
Env
string val
x Box
::=
Text
Box
labels
x
2)
instance
MSC
::=
MSC
2)
name next
Start
in
has
Pos
has
Pos
color white
top touch Box
color black
above
Line
End
bot touch Box
orientation
vertical
x
3)
name
Event
next 1
Event
next 1
4)
Event
::=
2
next
Event
Event
3
next
Event
next 1
2
Pos
Event
1
Event
Event
Action
::= next
Out
next
2
3)
has
Event
string
x
Text val
1
1 above
Line
has
contains
above
::=
has
Line
on
Pos
color
Box
white
has
2
from
Message
Pos
Pos
above
has
2
name x
Pos
2
to 4
Event
next 3
In
next
Event
Pos
has
4
Figure 3: The graph grammar productions for MSC abstract syntax graphs
2.1. Maintaining the correspondences These three different representations of a diagram need to be connected: • A spatial relations graph in fact forms a constraint system on basis of which a constraint solver can generate an appropriate physical layout, and, vice versa, graphical scanning deals with retrieving a spatial relations graph out of the physical layout of a diagram. • Each construct in the SRG represents a construct in the ASG, and each construct in the ASG is represented by construct(s) in the SRG. These relations need to be created and maintained by the environment which supports the graphical editing. These correspondence relations rigorously connect the physical layout of the diagram with its structure, and in fact makes it a visual sentence. The diagrams which are correct according to a visual language L all have abstract syntax graphs. These ASG’s form a graph language which can conveniently be defined by means of a graph grammar [13, 14]. One can imagine a graph grammar GG(ASGL) which defines the language of correct ASG’s for the visual language L. Similarly, one can describe the language of correct Spatial Relation Graphs by the graph grammar GG(SRGL). The productions in these two grammars have a close correspondence: where the ASG rule introduces a certain abstract notion, the corresponding SRG rule will introduce its visualization.
2.2. The graph grammars for the MSC case Fig. 3 shows the graph grammar GG(ASGMSC) which defines the language of correct abstract syntax graphs for Message Sequence Charts. Production 1 in this grammar creates an MSC node with name x and an ENV node out of the blue. Production 2 extends an existing MSC abstract syntax graph with a new production instance, which ini-
Pos
1 above
Line
3 has
above
::=
Line
has
Pos
Pos
has
2
4
x
4)
string val
Pos
Pos
Text
1
3 has above
has
above
Line
has
has
labels from
Pos
Arrow
above
Pos
to
has
Pos
above
2
Pos
Line
has
4
Figure 4: The graph grammar productions for MSC spatial relations graphs
tially consists of the events Start and End only. The grey colour for the MSC node indicates that it is a common context node of the production’s left- and right-hand side. It has to be present, but remains unmodified except the fact that it becomes the source of a new instance edge. The next two productions provide the chain of events with its contents. Both make use of the subclass relation {Start, In, Out, Action, End} is a Event, i.e. their grey Event context nodes may match nodes with any of the above mentioned labels. Production 3 inserts an Action by making room between two Events already present in a chain of events; production 4 creates a Message and inserts its start and end point into two (not necessarily distinct) event chains. In order to deal with messages to and from the environment in MSC, we need two additional productions which are quite similar to production 4. Every graph that can be obtained by repeatedly applying these productions is a correct MSC abstract syntax graph. The graph grammar for correct MSC spatial relation graphs GG(SRGMSC) works in a similar manner and is show in Fig. 4. For example, here production 4 states what a Message looks like (an Arrow labelled by a Text), and what spatial relations these new objects should have with the existing graphical objects. Similarly, every graph that
can be obtained by repeatedly applying these productions is a correct MSC spatial relations graph. A graph parser, such as described in [13,14], derives the structure of a graph according to its graph grammar. This means that if one has a SRG graph for an L diagram, and parses this graph according to GG(SRGL), this will result in a derivation D. This derivation D can be used to generate the corresponding ASG graph. The same holds for the reverse direction of generating a SRG graph out of an ASG. One can maintain this correspondence as the two grammars are closely related: where the production GG(ASGL)[i] introduces a language construct, production GG(SRGL)[i] introduces its physical representation. In addition to this one also needs to know which graph element in the ASG is represented by which element in the SRG. This is achieved by introducing repr edges which connect the two graphs. This information is used and extended by each production application. These repr edges make the two productions i into a single one. For example, here is the combined production 3 for the MSC case: x name
Event
next
Event
Event
1
repr
next
2
1
repr
repr
next
Action
Event
repr
2
repr
3.1. Syntax directed editing Syntax directed editing commands allow the user to change the diagram in terms of entire language constructs. For the MSC case, this would for example be the insertion of a new process instance, the deletion of a message, or the shuffling of the order of events on a process instance. Using the graph grammar approach of section 2.2 it is quite straightforward to implement insertion of language constructs by applying a selected production in both the SRG and the ASG, and by using the constraint solver to update the display on basis of the changes in the SRG. This keeps all representations up to date with each other, does not require graph parsing, and guarantees syntactically correct diagrams at all times. We are afraid though that this approach will lead to an awkward user-interface: consider the insertion of a message between two process instances according to the productions 3 of section 2.2. The user has to select two process instances and the four surrounding points on the time lines, in order to tell where to insert the message in the diagram. The user interface would be far less forcing if the user can insert an unconnected message, and is free to connect it later to process instances. In order to realize such a behaviour, we need a more lenient SRG grammar with, for example, such productions: x
::= Pos has
Line has
Pos
1 above
Pos
2
has
Line has
!
1
string
Text val
x
1)
on
Pos
Box
color
labels
::=
!
Arrow
Text
white
Pos
above
Pos
λ
contains
above has
string val
"
has
Pos
2
For further details concerning the definition of these relationships between ASG’s and SRG’s by means of coupled graph grammars the reader is referred to [2, 15].
3. Syntax directed graphical editing We now describe how the suggested general framework for the representation of a visual sentence can be used to implement a syntax directed graphical editor for a visual language. Such an editor supports drawing of diagrams by providing language specific editing commands. Ideally, such an editor should support the user in creating syntactically correct diagrams, but should not hinder the user by requiring syntactically correct diagrams at all times. We consider the following classes of editing commands essential for a language specific graphical editor: (1) syntax directed editing, (2) layout editing, and (3) free editing. We will explain what we mean by each of these classes and will sketch how they can be implemented on basis of our framework.
2)
Line
1 above
has
Pos
has
::=
Line
has
1
above has
Pos
above
2
Pos
2
3)
Pos
4)
Pos
Arrow
::=
Pos
Arrow
::=
Pos
from
to
Arrow
Arrow
Production 1 inserts a yet unconnected labelled arrow; production 2 creates a connection point on the time-line of a process instance, and production 3 and 4 connect the end points of the arrow to the time lines. Execution of production 1 can be offered via a menu-item, productions 3 and 4 can be activated through special drag actions which also execute production 2 automatically. Note that these lenient productions describe a language of SRG graphs also, and that the original language of SRG graphs is a subset of this language only. This improves the user-interface, but has as consequence that the diagram will most of the time be unconnected, and thus incorrect according to the visual language. By definition, an incorrect diagram does not have an ASG
either. However, in case the constructed SRG graph can be parsed according to the original strict SRG grammar, then the diagram is correct, and an ASG graph can be generated for it. This makes sense as the user will only be interested in execution of correct diagrams. By executing productions in reverse order (right to left) the editor can simply provide syntax directed delete commands. The need for syntax directed change commands will in most cases be satisfied by the add and delete commands according to the above small-step productions, but some commonly requested complex change commands can be specified by graph rewriting productions, too.
3.2. Layout editing The constraint solver chooses a particular layout within the limits specified by the SRG, but this choice might not be the best according to the user. Layout editing commands allow the user to change the physical layout by moving graphical objects, but these moves are restricted such that all constraints remain satisfied. This ensures that the layout only is changed, and not the interpretation of the diagram. For example, if the user moves a MSC process instance, all message arrows will stretch in order to remain connected. If the user moves a message arrow vertically, the move is restricted such that the above relations remain satisfied. Layout editing commands can be realized in two fashions: 1. If the user moves an object, then the constraint solver is asked for the area within which it may move without violating the constraints. The allowed moves are then restricted to this area. 2. If the user moves an object, then the move request is translated to an temporary high-priority constraint, and the constraint solver is asked to assign new positions to all objects such that the constraints remain satisfied. This provides more freedom to the user, but requires fast and incremental constraint solving.
3.3. Free editing Given the framework as we have till now, it is reasonably straightforward to offer the possibility of free editing to the user. In free editing mode the user is free to insert, delete, change, or move graphical objects without any interference by the syntax directed nature of the editor. This will sometimes be the easiest way to make complex changes to the diagram. At the moment the user indicates that (s)he is done, the positions and sizes of the graphical objects need to be analysed in order to extract the spatial relations which hold between them. This is called Graphical Scanning and results in a new version of the SR graph. All updates of the SRG can then be propagated to the related ASG as in section 3.1. Afterwards, syntax directed editing commands can be performed again.
4. Related work The idea to keep different representations of a visual program to support the editing of it, and to use a declarative mapping mechanism between the levels is not new. Kamada et al. [7,10] describe a general framework for bidirectional translation between abstract and pictorial data. They advocate to use the same levels as we do, and have mapping rules expressed in Prolog to keep the SRG and the ASG up to date with each other. Early work only dealt with visualization of abstract data via such mappings, but by introducing a second set of, more loose, reverse mapping rules, the pictorial representation can also be edited in order to update the abstract representation. The difference between this work and ours is that their mapping rules only state how a given SRG data structure is to be translated into an ASG structure, and vice versa, but they do not define how correct SRG or ASG structures are to be obtained. We also take care of that aspect with our production oriented approach. Furthermore, the use of different representational levels is an instance of the Model-View-Controller (MVC) concept of Smalltalk, or more specific, of the Observer design pattern as described by Gamma et al. [4]. This pattern separates the presentational aspects of a user interface from the underlying application data, and lets the Subject and the Observer communicate via notifications. Our graph based framework proposal complements the MVC concept, which is mainly useful as a program design concept. It advocates the usage of a graph data model and (coupled) graph grammars as a specification formalism, i.e. it is more concrete and offers new means to create an unambigous description of a specific Observer pattern instance. Furthermore, we allow the Subject and Observer to exchange their roles if necessary. Until now, we have discussed the benefits of storing visual language sentences as a pair of coupled ASG and SRG graphs on an abstract level, without relating this idea to the architectural concepts of already implemented concrete visual language environments. In the sequel, we will select some typical visual environments and discuss, whether or not they are built around similar concepts, and to which extent they do support the activities, syntax directed editing, layout editing, and free editing of section 3. It is of course not possible to cover all important visual environments which were realized within the last years. Therefore, we will focus our interest onto some of them which are graph based and/or generated with the aid of a meta environment. The selected environments and their most important properties are summarized in Table 1. The first two columns deal with hand-coded visual programming environment for a fixed language. The following columns present the properties of generated environments. Please note that they do not show the characteristics of their environment
generators, which are in all cases except PROGRES more or less traditional compilers which take a textual description as input. Table 1 is divided into two blocks of rows. The first block discusses to which extent the selected environment supports certain activities and what kind of formalism is used to generate the corresponding environment components. The second block deals with the underlying data structures for supported activities: • free editing manipulates directly a physical layout, i.e. the graphical objects of a low-level graphics editor, • layout editing deals with higher-level visual structures (also called SRG’s or views), • syntax directed editing and analysis work usually on more abstract structures (also called ASG’s or application models), • and execution/compilation activities use often abstract structures as input and produce new runtime structures as output. The first column of Table 1 deals with MViews, a generic framework which has been used to implement a partly visual programming environment for the object-oriented logic programming language Snart. It is as far as we know the first environment presented in the visual language literature which is graph based, built around the well-known Model-View-Controller concept of Smalltalk, and propagates changes in the model (abstract structure) to a number of related graphical and/or textual views (visual structures) in an incremental fashion. The next column summarizes the properties of tools generated with the PROGRES meta environment, which is itself a kind of visual programming environment. The PROGRES system uses a single formalism PROgrammed Graph REwriting Systems to specify a class of directed attributed graphs together with all needed operations for syntax directed editing, layout editing, analysis,
DiaGen is the only other presented environment generator which is based on (hyper-)graph grammars (graph rewriting systems). It offers formalisms for almost all required activities, but the expressiveness of its transformation rules is rather limited. As a consequence, it seems to be rather difficult to specify complex analysis or execution activities. Table 1 shows that DiaGen uses a single data structure for all purposes, a hypergraph with its superimposed derivation tree and additional (in-)equality constraints. Therefore, it cannot be used to generate environments in which quite different views for a single abstract structure have to be maintained. Replacing hypergraphs and hypergraph grammars by nary relations and constraint multiset grammars (CMG) we get (more or less) the entries of the following column. Again a single data structure is used to deal with free editing, layout editing, and syntax directed editing activities at
PROGRES [16, 17]
DiaGen [11]
CMG [1, 9]
SPARGEN [6]
fixed text language
under development gen. editor from hypergraph grammar (+ constraint solver)
generated parser from picture layout grammar
visual debugger
generated from programmed graph grammar with derived attributes and relationships
generated parser from CMG (constraint solver)
generated from derivation tree/graph transformation rules
realizable with C++ parsing actions
physical layout
display view
Tk/Tcl Widgets
graphical objects
graphical objects
C++ objects
visual structure
subset view graph
TkLayout graph
base program graph
n-ary relations + linear constraints
attr. derivation tree
abstract structure
Attributed hypergraph + derivation tree with constraints
free editing Operation
Furthermore, we should mention that derived node attributes may be used to realize incrementally working layout algorithms. Unfortunately, directed equations, which define derived attributes, are considerably less powerful than undirected linear inequalities in DiaGen [11] or undirected equations in DeltaBlue [3]. Therefore, the PROGRES system’s support for defining layout computation and editing activities is still insufficient. Finally, there is no support for free editing and parsing until now.
MViews (SPE) [5]
Properties
layout. editing fixed language syntax dir. editing analysis execution
Data Structure
and execution activities. These attributed graphs play the role of ASG’s and may be different within different generated prototypes. The representational data structures, which play the role of SRG’s, do not differ among generated prototypes. They are realized once and for all using the package TkLayout on top of the user interface toolkit Tk/Tcl. This means that we have a separation of ASG’s and SRG’s as needed, but neither specific classes of SRG’s nor tool specific editing activities on SRG’s may be specified.
Attributed graph runtime structure
?
any data structure generated with C++ parsing actions
Table 1: Some representative visual programming environments and environment generators
the same time. The corresponding specification deals with all these aspects at the same time, i.e. a single CMG production defines the physical layout as well as the abstract syntax of a language construct. This has the same drawbacks as mentioned above for DiaGen. Finally note that CMG’s do not support as far as we know analysis or execution of visual language sentences, but suitable extensions of the underlying logic-oriented formalism should be feasible. The last column of Table 1 presents SPARGEN, a kind of visual language compiler generator. It uses picture layout grammars to describe the syntax and appearance of visual language sentences. The generated parsers allows for additional action routines written in C++, which are called during parsing. SPARGEN represents, therefore, in our opinion the most pragmatic approach for the realization of visual language compilers. These compilers take the data structure of a low-level graphics editor as input and produce any desired C++ data structure as output. Tools for layout editing or syntax directed editing are outside the scope of SPARGEN.
5. A Graph Based System Architecture Summarizing the discussion above we can savely state that none of the presented systems fulfills all requirements of the preceding sections 2 and 3 of this paper. A lot of work is still necessary to implement all sketched ideas within a single visual language environment generator. This is the major goal of our research groups at Leiden University and RWTH Aachen. One intermediate result of these activities is the already mentioned graph grammar based environment generator PROGRES at the RWTH Aachen [17]. Another one was built by the participants of the 1995 Visual Language course at Leiden University. It is a syntax directed editor for MSC diagrams which uses Fresco [21] as user-interface builder and DeltaBlue [3] as constraint Tool Integration Layer LayoutManager
User-Interface-Control GraphicsEditor
Displayer [Scanner]
User Interface Toolkit
X-Window Invariant Part
SRGManager
ASGManager
Compiler, Interpreter
View-Control ConstraintSolver
Unparser & Parser
Execution Machinery
SR-Graphs
AR-Graphs
Exec.Graphs
Graph-oriented Database System Adapted Part
Generated Part
Figure 5: Proposed architecture for a visual language environment
Figure 6:
A syntax directed MSC diagram editor
solver. A screendump of one of the editors built is shown in Fig. 6. Currently, master thesis students are extending this editor by allowing it to be parameterized by a visual syntax definition. The ideas behind this editor are already quite similar to the principles sketched in section 3, but for the coupling of language constructs which is offered via typed male and female plugs. Based on our experiences with the development of these systems, we believe that a graph (grammar) based visual environment’s architecture should have about the following form (cf. Fig. 5): • Abstract data types for spatial relations graphs (SRgraphs) and abstract syntax graphs (AR-graphs) are the system’s central components. Their internal structure and interface operations are generated from graph grammar specifications. • These graph data types should be persistent and are, therefore, either stored within a graph-oriented database system like GRAS [8] or an object-oriented database system with a “thin” graph-oriented layer on top of it. • Additional graph data types and tools, specified by means of graph grammars, may be added for analysis or execution activities (dashed dark grey boxes). • It is the task of (incrementally working) Unparser and Parser components, which are generated from coupled graph grammars, to keep SRG’s and ASG’s in a consistent state, whenever they are modified via their corresponding syntax directed SRG- and ASG-Managers. • A ConstraintSolver, like the one of EDGE [12], which combines constraint solving techniques for systems of inequalities with standard layout algorithms, is responsible for maintaining SRG layouts in a consistent state. • Displayer and Scanner components are responsible for keeping the “widgets” of a User Interface Toolkit like Fresco [21] and the corresponding SRG in a consistent state (a Scanner is not needed as long as the system does not contain a low level Graphics Editor).
Two integration layers are necessary in the general case as a kind of glue between the above listed components. The first one, named View-Control and User-InterfaceControl, control unparsing/parsing activities between multiple SRG views and their underlying ASG’s as well as displaying/scanning activities between presentations of SRG’s on the screen iand their SRG’s. The second one, the Tool Integration Layer, coordinates any necessary interaction between offered tools. It is either just a simple main program or a broadcast message server in the general case.
6. Conclusions Within this paper we have presented a graph (grammar) based framework for the implementation of visual (programming) environments. Relying on experiences with the Model-View-Controller concept of Smalltalk we postulated the needs for maintaining separate data structures for free editing and layout editing on one hand, and for syntax directed editing and execution activities on the other hand. These activities were discussed in more detail with a main emphasis on editing operations. Afterwards, a number of already existing visual environments and environment generators were studied with respect to the activities they do support and their internal data structures. The discussion showed that all presented environments compromise either the required separation of physical layout, spatial relation graphs, and abstract syntax graphs, or support only a subrange of the functionality for manipulating and executing visual language sentences. The ideal system would use a single formalism to model separate data structures, their local operations, and the relationships between them. It would also combine facilities for manual layout, semi-automatic layout with constraints as well as full automatic layout by means of incrementally working standard (graph) layout algorithms. The last part of the paper sketched our ideas how an “ideal” graph based architecture for visual language environments might look like, and which language specific components of such an architecture might be specified and generated by means of graph grammars. The realization of visual environment generators following this paradigm should be feasible within the next years. All needed techniques are already developed, but not yet combined within a single system (cf. Table 1). Once existent, such a generator will reduce the amount of work for implementing visual languages dramatically, and it will offer the chance to develop lots of special purpose visual languages for certain niches, where text-oriented languages are used nowadays.
7. References [1] [2]
S.S. Chok and K. Marriott. Automatic construction of user interfaces from constaint multiset grammars. In [20], pp 242-249. G. Engels, M. Andries, and J. Rekers. How to represent a visual program? Workshop on the Theory of Visual Languages, Gubbio, Italy, (1996).
[3]
B.N. Freeman-Benson, J. Maloney, and A. Borning. An incremental constraint solver. In CACM, vol 33, no 1, ACM Press (1990), pp 54-63.
[4]
E. Gamma, R. Helm, R. Johnson, and J. Vlissides. Design Patterns - Elements of Reusable Object-Oriented Software. Addison-Wesley professional computing series (1995).
[5]
J.C. Grundy and J.G. Hosking. Constructing Multi-View Editing Environments Using MViews. In [18], pp 220-224.
[6]
E.J. Golin and T. Magliery. A compiler generator for visual languages. In [18], pp 314- 323.
[7]
T. Kamada, and S. Kawai. A General Framework for Visualizing Abstract Objects and Relations. In Transactions on Graphics, vol 10, no 1, ACM Press (1991), pp 1-39.
[8]
N. Kiesel, A. Schürr, and B. Westfechtel. GRAS, a GraphOriented Database System for (Software) Engineering Applications. In Information Systems, vol 20, no 1, Pergamon Press (1995), pp 21-51.
[9]
K. Marriott. Constraint Multiset Grammars. In [19], pp 118-125.
[10] S. Matsuoka, S. Takahashi, T. Kamada, and A. Yonezawa. A General Framework for Bidirectional Translation between Abstract and Pictorial Data. In Transactions on Information Systems, vol 10, no 4, ACM Press (1992), pp 408-437. [11] M. Minas and G. Viehstaedt. A generator for diagram editors providing direct manipulation and execution of diagrams. In [20], pp 203-210. [12] F. Newbery Paulisch. The Design of an Extendible Graph Editor, LNCS 704, Springer Verlag (1991). [13] J. Rekers and A. Schürr. A graph grammar approach to graphical parsing. In [20], pp 195-202. [14] J. Rekers and A. Schürr. Defining and Parsing Visual Languages with Layered Graph Grammars. Will appear in the september 1996 issue of the JVLC. Also available from www.wi.LeidenUniv.nl/TechRep/tr96-09.html. [15] A. Schürr. Specification of Graph Translators with Triple Graph Grammars. In Proc. WG’94, LNCS 904, Springer Verlag (1994), pp 151-163. [16] A. Schürr, A. Zündorf, and A. Winter. Visual programming with graph rewriting systems. In [20], pp 326-333. [17] A. Schürr, A. Winter, and A. Zündorf. Graph Grammar Engineering with PROGRES, in: A. Schäfer, P. Botella (eds.): Proc. ESEC ’95, LNCS 989, Springer Verlag, pp. 219-234 [18] E.P. Glinert, K.A. Olsen (eds). Proc. 9th IEEE Symp. on Visual Languages - VL'93. IEEE Computer Society Press (1993). [19] A.L. Ambler, T.D. Kimura (eds). Proc. 10th IEEE Symp. on Visual Languages - VL'94. IEEE Computer Society Press (1994). [20] V. Haarslev (ed). Proceedings 11th IEEE Symp. on Visual Languages - VL'95. IEEE Computer Society Press (1995). [21] Fresco, an object-oriented user interface builder. Public domain software, www.faslab.com/fresco/HomePage.html. [22] ITU-T, Geneva. Recommendation Z.120: Message Sequ. Chart (MSC), www.win.tue.nl/win/cs/fm/sjouke/msc.html.
