Integrating Data/Program Structure and their Visual ... - CiteSeerX

Integrating Data/Program Structure and their Visual Expressions in the Visual Programming System Hiromichi ENDOH Jiro TANAKA University of Tsukuba, 1-1-1 Tennodai Tsukuba Ibaraki 305-0006 Japan Phone: +81-298-53-5165, Fax: +81-298-53-5206 e-mail:{endoo,jiro}@softlab.is.tsukuba.ac.jp

Abstract

up a node and adding it to a graph, the model of the program is also modified like that. This is referred by “direct manipulation[1][10],” and is an important concept in VPS.

“The direct manipulation” has already become a popular keyword in the field of visual programming. In many visual programming system(VPS), the user can manipulate the program from its visual expression in the editor subsystem. But the execution process is still displayed in another window and user have to move the mouse cursor frequently. So we noticed the importance of the manipulation interface where user can do neccessary operations for programming in one field. We have implemented a prototype of such VPS using the IntelligentPad architecture. The elements of the data/program structures are expressed as pads. When user manipulate them, the program is also changed and is automatically evaluated.

2 Considering VPS architecture related to the manipulation interface 2.1 The problems Present VPSs often consist of multiple subsystems such as like a program editor and a program browser/tracer. User will edit a program with the editor, save it into file, then run a tracer and watch the execution process of the program. But there is two problems with such VPS architecture: • Since the functional parts of VPS such as editor and tracer are separated, user may have to switch the window focus of these subsystems and make great effort to manipulate programs. During debugging, for example, a user may frequently modify the program and try to execute it. Since conventional VPS consists of editor and tracer as individual programs, user will have to move mouse from one window to another on every time user finishes editing or executing the program (See Figure 1).

1 Introduction Visual programming system(VPS) is the environment that visualizes the model of programs in two(or more)dimentional forms with visual expressions such as pictures or animations[5][8]. VPS converts the data elements into pictorial ones, language model such as data structures or program structures into graphs, and often program behaviors into animation patterns. User can understand the data/program structure more intuitively than conventional text-based programming language, because visualized programs look simpler and more appealing. VPS also means the environment where user can manipulate data/program structures with those visual expressions, using a pointing device such as a mouse. The model of program is associated with its visual expression, and a change to the one is also applied to the another. When user manipulates the visual expression of program such as picking

• While user edits the data/program on the editor window, the changes will not be reflected immediately to the tracer. Such system might not be enough saying “based on the direct manipulation,” and it is confusing to user that there is difference between two program views on both of the editor and the tracer.

2.2 What's desired in the manipulating interface If user can do all necessary operations to edit and execute program in one field, user will have no need to move mouse 1

User−interface part Model part Hello, World!

program tracer User’s manipulation

A pad

program editor

Screen

UI part: visible to user Model part Slot

Figure 1. On a separated-architecture VPS, user have to switch the window focus frequently.

From other pads

To other pads Data/messages

Figure 2. The architecture of a pad cursor so frequently. So implementing such manipulation interface to VPS will be good for reducing user's effort and will make programming process smooth. As well as the point mentioned above, we noticed the following points which should be desired when building manipulation interface on VPS. • The visual expressions of data/program structure uses familiar metaphor to user or are converted by simple visualization rules. • The data structure and program structure can be converted to similar visual expressions (it is the best if we can apply the same expression to them). The visual expressions of them affect the style of manipulation interface when they are implemented on a VPS. If the gap between those expressions is large, it will be difficult to provide consistent manipulation interface to both of data and program. • The model of data/program structure can be manipulated from their visual expressions on the screen. And when the model is modified during program execution, the field will also be updated at once. These points are important for the models to be seen as if they appear directly on the field.

3 Designing the new VPS 3.1 IntelligentPad IntelligentPad[11] is a visual toolkit system developed by Yuzuru Tanaka at Hokkaido University. Unlike the ordi-

nary Widget-based toolkits such as X toolkit or Motif, IntelligentPad can make application systems on top of itself. There are primitive components called “pads.” Each pad is a visible object on the screen and user can manipulate it with pointing devices. User assembles or pastes them each other to make applications. It can be treated as primitive pad and reusable when making new applecations. A pad consists of two major subsystems. One is the User-Interface(UI) part, which handles user' s manipulation and graphical output such as drawing itself. The another is the model part, which holds user data and the internal state of the pad (See Figure 2). When user manipulates a pad, the control events are passed to the UI part of the pad. The UI part works according to the events and outputs the results to its canvas. The model part manages data exchanges of the pad. It has some objects named “slots,” through which the pad exchanges data or control messages with other pads. Pads should connect their own slots each other to exchange data among them. With the two subsystems combined, a pad can change its appearance as its internal state changes and vice versa.

3.2 Defining visual expressions from the language model Considering the characteristic of the pad, we have designed the data structures as sets of pads. When one of them receives user' s manipulation events, it changes its own state(See figure 3). In this way a change to the data structure

User’s manipulation screen UI part

Model part Cluster of pads

Figure 3. VPS architecture with integrated data/ program structure

is directly reflected to its visual expression, and the VPS with such data/program structure and their visual expression can satisfy the user's needs mentioned at section 2.2. It seems that logic programming model, object-oriented programming model and data flow model are suitable for the programming model of such VPS. It is because such programming models are usually defined by some primitive objects and their relations; it is easy to define visual expressions from such programming models. There are some VPS implementations based on these programming models[6][4][9]. We have designed this pad-based VPS using GHC[7] data/program structure as the language model of this VPS. Each of primitive objects of GHC, atom, functor, and goal predicate is represented by pad, and the relations among them such as unifications or argument bindings are displayed as lines(See Figure 4).

4 Stageless programming: editing and execution By integrating data/program structure and their visual expressions using IntelligentPad architecture, the system now can know the when and how user manipulated the visual expressions on the screen. when user manipulates the visual expressions, an event about the manipulation is notified to the UI part of the corresponding pad. We had implemented the VPS so that it can start execution of the program when user manipulates it. When the user makes a program on the screen and it is ready to be evaluated, the program is automatically executed. The user can check the behavior of the program interactively since the editing process and executions process are together. Let us see the example to append a list structure to another one executed on our system: 1. First you create a goal predicate named “append” (See Figure 5). It takes two lists from its inputs, and appends one list to the tail of the another. Then you create cons cells and atoms which are the elements of two list (Figure 5). They can be created by selecting “atoms and functors” and then typing their names. You will need another goal “result,” which is a dummy goal to hold the output of “append.” 2. Connect the atoms and the arguments of cons cells by clicking the mouse button on them. Now you have two lists on the screen (Figure 6). 3. Connect the two lists to “append” on its input and “result” to the output (Figure 7). At the moment “Append” is activated and expands its function body to the field (Figure 8). The program execution starts by this action and the input list is being appended to another list (Figure 8, 9).

goal predicates

predicate

4. Finally you have the appended list connected to ”result” (Figure 10). a

cons

atoms / functors

a arguments of "cons" are bound to "a" and nil

cons []

Figure 4. Visual expression of GHC using IntelligentPad

In these examples, user just put the goal and atoms, and connected them. There is no conventional programming stages like editing, compilation, and execution. The program on the system is always ready to be activated by user.

Figure 5. Create the goal “append”

Figure 6. Creating atoms and cons cels and making lists with them

Figure 7. Connect the lists to “append”

Figure 8. “Append” is activated (its function body has appeared in the center of this figure)

Figure 9. First unification has done ∼ “Append” has reduced

Figure 10. Final state; second unification has done

5 Related Work There are some work related with our VPS; both in closecoupled program view and its execution, and in direct manipulation. Forms/3[2] is called a form-based VPS. User pastes functions cells on base just like defining cell functions of spreradsheets. The results of a program can be displayed animated, and the changes on the program form are reflected immediately. Unlike our VPS, definitions on each cell function are written by text. Leogo[3] is a Logo-like iconic programming environment. It also supports programming by direct manipulation. The window where the programs are defined is separated from the execution window.

6 Conclusions To reduce user's effort during programming with VPS, the manipulation interface where user can mainpulate the data/program structures intuitively from their visual expressions is neccessary. We noticed that the model of the data/program structure and their expressions should be integrated. So we have designed and implemented a prototype of such VPS. We have integrated the data/program structure model and their visual expressions using IntelligentPad. Our system can handle the data/program strctures and their visual expressions at the same time, so the user can maniplate programs more intuitively than the VPSs with conventional editor-tracer architecture.

References [1] B.Schneiderman. Direct manipulation: A step beyond programming languages. IEEE Computer, 19(8):pp.57–69, aug 1983. [2] P. Carlson, M. Burnett, and J. Cadiz. A seamless integration of algorithm animation into a declarative visual programming language. In Proceedings Advanced Visual Interfaces (AVI' 96), 5 1996. [3] A. Cockburn and A. Bryant. Leogo: An equal opportunity user interface for programming. Journal of Visual Languages and Computing, 8(5-6):601–619, 1997. [4] F. R. Giles, P. T. Cox, and T. Pietrzykowski. Prograph: a step towards liberating programming from textual conditioning. In Proc. of 1989 IEEE Workshop on Visual Languages, 1989. [5] E. J. Golin and S. P. Reiss. The specification of visual language syntax. J. Visual Languages and Computing, Vol.1(No.2):pp.141–157, 1990. [6] K. M. Kahn and V. A. Saraswat. Complete visualizations of concurrent programs and their execution. In Proc. of 1990 IEEE Workshop on Visual Languages, pages pp.7–15, 1990. [7] K.Ueda. Guarded Horn Clauses. ICOT tech. report TR-103, Institute for New Generation Computer Technology, 1985.

[8] B. A. Myers. Taxonomies of visual programming and program visualization. J. Visual Languages and Computing, Vol.1(No.1):97–123, 1990. [9] B. S. S. Toyoda and S. Matsuoka. Supproting deign patterns in a visual parallel data-flow programming environment. In 1997 IEEE Symposium on Visual Languages, sep 1997. [10] B. Schneiderman. Designing the User Interface, chapter 5. Addison-Wesley, 2nd edition, 1992. [11] Y. Tanaka and T. Imaktaki. Intelligentpad: A hypermedia system allowing functional composition of active media objects through direct manipulatios. In Proc. of IFIP 11th World Computer Congress, pages pp.541–546, 1989.