QUICK: a tool for graphical user-interface construction ... - Springer Link

1 Introduction

QUICK a tool for graphical user-interface construction by non-programmers Sarah Douglas, Eckehard Doerry, and David Novick* Department of Computer and Information Science, University of Oregon, Eugene, OR 97403-1202, USA e-mail: douglas @cs.uoregon.edu

The last decade has placed the superiority of graphical user interfaces over traditional text-based approaches beyond dispute. In almost all contexts, users have found graphical interfaces easier to learn, faster to use, and less error-prone. However, it has been shown that the creation of powerful graphical interfaces takes up to 80% of the time required to develop an application. In our work, we seek to extend the benefits of graphical interaction to the next layer of computer user - the interface designer. Our work in this area is distinguished from other efforts by two important differences. First, while other efforts focus primarily on the design of "standard" user interfaces, our approach emphasizes the creation of unique and innovative interfaces by supporting, among other things, arbitrary user-designed graphical representations, direct specification of animation, and digitized sound. Second, our goal is to cater to the nonprogrammer. Thus, we address a challenging trade-off: maximizing power and flexibility in an extremely simple environment. We explore the utility of the prototype object-oriented paradigm, a high-level userinterface language, and a direct-manipulation programming environment in this context. Key werds: User interface- T o o l k i t - Object-oriented languages * Present address: Department of Computer Science and Engineering, Oregon Graduate Institute of Science and Technology, USA Offprint requests to: S. Douglas The Visual Computer (1992) 8:11%133 9 Springer-Verlag 1992

The concept of an interface design toolkit has received considerable attention in recent years for several reasons. First, a toolkit allows constructing interfaces with relative ease, abstracting the interface construction task, and providing an efficient means of exploring large numbers of interface designs. The commercial applications of such toolkits are obvious. However, we are more interested in a research-oriented perspective, in which the toolkit becomes a valuable exploratory tool. By comparing and formally evaluating a number of interfaces constructed with the toolkit, we can begin to identify regularities in the domain (i.e., what interface features are "good" in various contexts). More importantly, by examining the composition of the toolkit itself and evaluating its effectiveness, we can formulate some notion of interface construction primitives, both in terms of static entities and functionality. The QUICK (Quick User Interface Construction Kit) system represents our attempt to explore the above issues with respect to a particular goal: providing an interface design tool for non-programmers to build graphical direct manipulation interfaces that go beyond specifying standard interface objects, such as windows, menus, and buttons, to allow the easy construction of complex user and object interactions within windows. In this age, where computer literacy is essential for many groups, providing tools to help the casual user unlock the creative potential of the computer is of high priority. We explore the utility of several important concepts in this context: High-level user-interface programming language, - Object-oriented paradigm, Direct-manipulation programming environment. We believe that the combination of these features has resulted in a uniquely powerful blend of flexibility and simplicity, allowing non-programmers to construct a wide range of complex interfaces.

2 Related work We initially began work on this research with a great deal of inspiration from a system called Programming by Rehearsal (Gould and Finzer 1984). Programming by Rehearsal used a subset of Smalltalk classes directly related to the user interface to create a simplified programming system for

] ]7

teachers. It also used the metaphor of the theatre to create a novel programming environment. To support the concept of "rehearsing" (programming) the "actors" (graphic objects) on the "stage" (window), Gould and Finzer invented the programming-by-demonstration paradigm. Several systems have been developed as object-oriented construction kits for interface design. Trillium (Henderson 1986), written in LOOPS, was limited to the design of copiers but used object composition. Fischer and Lemke's WLISP (1988) was a construction kit for windows based on an object-oriented class inheritance LISP called ObjTalk. Fischer and Lemke's F R A M E R (1988) added design critics to the WLISP construction kit and had more of a direct manipulation environment for programming. Barth's GROW (1976) was an inheritance object-oriented system for creating graphic displays as objects (grid, text, box, icon). It supported both composition of objects and dependency between object attributes. It did not have a direct manipulation interface and did not integrate user actions. Although Cardelli's (1987) direct-manipulation interface designer was not explicitly object-oriented, its dialog box components acted like objects. Myers' early work on Peridot and later work on Garnet (1990) combines programming by demonstration with active value objects. Recent work of Stasko (1991) in the Dance system uses direct-manipulation techniques to build algorithm animations by demonstration.

3 The QUICK system Many interface-construction toolkits are designed to assist professional industrial programmers in prototyping standardized interfaces for commerical applications. This focus is reflected in their commitment to predefined classes of interface objects, relatively complex conceptual frameworks, and intricate (though powerful) approaches to specifying interface functionality. In contrast, Q U I C K is designed specifically to be used by non-programmers to create unique (and often non-standard) applications. The initial impetus for Q U I C K was the need for a tool to allow teachers to construct simple specialized applications for classroom use. This unusual requirement raised a challenging problem: how to provide a means of specifying interface layout and functionality without requiring the user to know how to

118

program, while at the same time providing sufficient power to allow the user to accomplish diverse interface-construction goals. While this is clearly a tradeoff situation, we believe that QUICK represents an acceptable middle ground. Q U I C K is similar to many of the systems named above, exploring the integration of key concepts from each: direct manipulation, programming by demonstration, and use of a high-level language to specify user interface functionality. However, QUICK differs from other toolkits in several important ways.

Object-oriented. The notion of an object refers to the logical and conceptual grouping of both state and functionality. While a number of toolkits are implemented in an object-oriented language, they seldom fully extend this commitment into the domain of the end-user. In QUICK, all interface entities are viewed uniformly as objects by both the system and the end user. A key feature of this approach is that, as objects, the interface components can refer to each other and communicate using the message passing paradigm. No predefined class structure. Most toolkits provide a predefined set of classes (e.g., windows, buttons), each with a particular graphic representation and functionality that the user can instantiate, creating components to use in constructing a new interface. We find this overly restrictive 1 in that the user is constrained to constructing only those sorts of interfaces supported by these specific classes (i.e., interfaces that consist solely of windows, buttons, and other predefined entities). There is no way to create interfaces in which arbitrary graphical entities respond to user events in unique ways including, for instance, sound and animation. In QUICK, we explore a less structured approach. Every functioning interface object is an object in the programming sense. Unlike most object-oriented languages, QUICK's objects are prototypical rather than classed (Borning 1979; Borning 1986; Ungar and Smith 1987). That is, every object is created with all the characteristics of the basic prototypical object. There is no notion of inheritance or class-based specialization. This design choice reflects a decision to support the initial bottom-up design of interface objects from a set of fundameni Note that in the commercial world, where standardization is important, it may actually be advantageous to have a firmly established library of interface objects. Thus, it may be desirable to sacrifice flexibility in favor of uniformity in these contexts

Fig. 1. The "lifeboat" interface

tal parts as a construction kit (Fischer and Lemke 1987), rather than as a top-down taxonomic specialization using inheritance. We feel that this approach lends itself to cognitive simplification. At the same time, we provide strong support for the two primary advantages of the class-based approach, abstraction, and reuse. Abstraction is supported in that the user may define new "classes" by aggregating groups of objects, thereby defining a new type of object. Reuse is supported in that objects (simple or aggregate) may be duplicated. In sum, our approach represents a commitment to a prototypical (vs class-based) view of objects, to user-defined object abstraction, and to reuse via a copy-edit approach. In the remainder of this section we briefly describe the Q U I C K system. We begin with an example to serve as a firm foundation for our discussion and to illustrate the highly interactive direct manipulation interfaces that Q U I C K is designed to produce.

3.1 Example: the "'lifeboat"

in QUICK. That is, there is no need for additional application programming and thus the interface constructed in Q U I C K is the application. The scenario is as follows. The application user (student) is asked to imagine that she or he is aboard a sinking ship. The task is to stock the lifeboat with various items, such as life vest, map, and water. The application starts by asking the user, in whatever language is being taught, to place one such object in the lifeboat. If the user succeeds, the system announces success and proceeds to ask the user to place the next object in the lifeboat. If, however, the user puts the wrong object in the lifeboat or fails to place the object in the lifeboat, the object moves back to its initial position, the system announces failure, and the request to put the appropriate object in the lifeboat is repeated. When the last item is in the lifeboat, the application is finished. To help the user learn the names of the objects, each object announces its name when it is clicked on with the mouse. Figure 1 shows a view of the running application, which was created in approximately one-half hour (not counting time to digitize sounds and draw the images).

The following example illustrates the power and simplicity of the Q U I C K system and will serve as a descriptive vehicle throughout the paper. The application was developed as a beginning-level tool for teaching languages 2 and was created entirely

3.2 Programming

2 The example presented is not the complete application. The application was designed under a joint project with the linguistics department and is currently used in research exploring the learning of Japanese as a second language

A major distinction between Q U I C K and other approaches to interface design lies in our more abstract and flexible view of interface components.

3.2.1 The prototypical object

119

As noted previously, most toolkits provide a predefined set of interface entities (e.g., windows, buttons), each with certain behavior and attributes inherently associated with it. That is, the definition of a particular class of entity, say, a button, tightly constrains how instances of the class look (i.e., their screen representation), what kinds of interface events they respond to, and how they respond. Thus, an instance of the button class might be able to respond to single-clicking, but not to dragging, double-clicking, or arbitrary user-defined events. Under this approach, the interface designer chooses from a predefined set of interface components those components that provide the desired functionality. Constructing an interface is therefore primarily a composition task. Unfortunately, this approach severely constrains interface design. One can only build interfaces specifically supported by the predefined abstractions. Q U I C K takes a more abstract view of interface construction. Instead of providing predefined classes, we provide a single, fully general interface entity, allowing the user to define its attributes and behavior. In other words, the Q U I C K object may be viewed as a sort of shell, a basic framework that the user elaborates into a functioning interface component. This creative elaboration-based approach distinguishes QUICK from class inheritance systems, which embody the compositionbased perspective. Abstractly, Q U I C K objects consist of four components: graphic representation, spatiality, functionality, and attributes.

Graphic representation. In QUICK, the notion of an object's representation on the screen is disjoint from the object itself. That is, the object is viewed as a logical entity that can be represented in a number of ways. Two representations are predefined. The image of the object is an arbitrary picture created by the user in a drawing program and the text of an object is an arbitrary string, initially the name of the object. Not only may the user switch between representations of the object, but both the image and the text of an object may be changed at any time. Indeed, one way to display values in Q U I C K is to create an object, ask it to display its text representation, and then simply change that text at run time. Spatiality. Our commitment to direct manipulation and the object-oriented viewpoint has forced us

120

to develop a clear framework for expressing spatial orientation and relations in the QUICK system. Each Q U I C K object incorporates an explicit representation of its position in the interface, recording both its Cartesian position in the workspace and its plane, allowing the system to provide a simulated three-dimensional environment in which the designer can manipulate the relative "depth" of objects in the workspace. The explicit representation of position provides the designer with a means for expressing spatial relations among objects in the interface as well: it is possible to test spatial relationships 3 (e.g., in, above, left of) between objects programmatically. The semantics of the spatial relation functions has been validated by us in empirical studies with people and is an unusual secondary contribution of our research (see Douglas et al. 1987). This feature has proved invaluable in designing the direct manipulation interfaces that are the target of our work. In the lifeboat example, for instance, the ability to test whether an object is "in" the lifeboat is crucial. The final aspect of spatiality in QUICK is the incorporation of animation. Each object may have associated with it a path, which the user records in a "by-rehearsal" style (Gould and Finzer 1984) by asking the system to watch (record) and then dragging the object through the desired motions. The object can be asked to run its animation, moving itself along the predefined path. We have invested considerable effort in exploring how best to support the user in specifying animation in the interface, an area that has received little attention up until now except in the area of algorithm animation (Stasko 1991). For instance, it should be possible to specify that a given motion is to be repetitive, cyclic, or relative to the current position of the object in the interface or to some fixed point in the Cartesian plane. QUICK accomplishes these goals by allowing the user to edit the recorded path, specify the number of repetitions, and select animation characteristics along three orthogonal dimensions: mode, continuity, and orien-

tation. Briefly, mode is either Jump or Drag and specifies whether the object moves between adjacent path coordinates in a single leap or whether it moves 3 In general, spatial relations (e.g., in, above, left of) have proven very difficult to define concretely due largely to a strong dependence on the context and nature of the objects being related. We base our algorithms on past research in this area by Douglas et al. (1987)

Movement for rope

:.:.:.:.:.:.:.:

:::::::::::::::

(188, 179) (189, 179) (191, (193, 179) (194, 179) (196, 179) (196, 179) (201, 177) (202, 176) (203, 176)

"--] V:

--

~ I

Movement Params O Drag

~ ~ ~ ~ ~

@Jump (~ l-way O WrapAround O 2-way (~ Relative 0 Fixed

"epea"l' ]

Drag

ncrement: I

i

more slowly along an interpolated line between them. The continuity may be one-way, cyclic, and two-way, specifying what happens when the object reaches the end of its path with one or more repetitions left to go. In the first case, it runs its course 4 again, starting where it left off; in the second case, it first jumps back to where the animation originally started; in the third case, it simply reverses its path. Finally, the orientation may be either fixed or relative, specifying whether animation should start from the object's current position (at the time it is asked to animate itself) or from some userspecified fixed point every time. Taken together, these features provide a powerful and flexible means of specifying how an object should behave when asked to animate itself. Note that it is possible to ask several objects to simultaneously animate themselves. While this is very nice in that it allows multiple interface components to be in motion at once, it raises a new challenge: How can we allow the user to specify timing constraints to synchronize the activity? For example, the user may want to express "animate objects Foo and Fum, but arrange for them to reach the intersection of their paths at the same time." We briefly discuss our current work in this area in Sect. 7.

Functionality. The issue of how to specify interface functionality is central to the design of an interface. Each object has associated with it a set of actions, specified in the Q U I C K user interface language, that define how it will respond to various userinitiated events. Four built-in events are one-click 4 A path is represented by an array of offsets. Hence, movement can easily be initiated from any point

Fig. 2. The Q U I C K animation editor. The Q U I C K user can edit the recorded animation as well as specify how it is to be executed

event, two-clicks event, landing event, and dragging event, which specify what should occur when that object is single-clicked, double-clicked, dropped, and dragged, respectively. For instance, in the lifeboat application, the one-click event of each object causes the system to announce the name of that object. In addition, the user may define new actions for any object along with the conditions under which they should be activated. Thus, each of the objects in the lifeboat application (with the exception of the lifeboat itself) has a special speech action defined to announce "Put the (object's name) in the lifeboat," which is programmatically triggered when it becomes that object's turn to be placed in the lifeboat. Attributes. Just as the functionality component defines an object's behavior, the attributes component defines its static characteristics. Built-in attributes specify whether the object is visible, draggable, and/or highlighted and establishes a number of user-level characteristics. For instance, in the lifeboat example, the lifeboat's "draggable" attribute is set to false, making it a stationary component of the interface, while all other objects have the "draggable" attribute set to true, allowing them to be moved about. Naturally, the user may define new attributes (of any data type) and test and retrieve all object attributes programmatically.

3.2.2 The passive background One other Q U I C K component deserves brief mention. The background is simply a bitmapped image (as created by any of the popular painting pro-

121

grams) that serves as the passive backdrop for all of the action in the interface. This approach is much more efficient than defining the various lines, boxes, and other passive background elements in the interface as individual QUICK objects.

4 QUICK language and structure editor Q U I C K language is extremely simple when compared with other approaches, such as programming with data flow diagrams, forcing the user to program in the underlying application language, and even other high-level interface languages. We reiterate that while perhaps appropriate for professional programmers, all of these approaches are too complex for users with very little programming experience.Thus, QUICK language is a simple, linearly organized, declarative language consisting of a small number of high-level commands. To further simplify QUICK, the language is embodied in a structure editor, which guarantees that user programs will be free of syntax errors. The price we pay for this simple controlled environment is a loss of expressive power. However, it is not our goal to allow the user to express everything, but rather to discover and provide a minimal set of primitives that allow our unsophisticated user population to design a wide variety of interfaces. The following list outlines the QUICK language: Animate (list of objects) Flash (object) Highlight (object) Unhighlight (object) If (test) ( t h e n ) (else) For-each (list of objects) (code) Set-attribute (object) (attribute) (value) Say (list of sounds) Move (object) (location) Trigger-action (object) (action) Brief descriptions of the most important commands follow: Animate allows the user to programmatically initiate animation. The argument is a list of objects that all simultaneously trace their user-defined animation paths.

122

If is perhaps the most powerful command in the QUICK language, allowing the user to incorporate conditional branching into an otherwise linear language. Much of the power of the QUICK conditional stems from the variety of options available to fill the (test) argument. Among other things, the user may construct arbitrarily complex Boolean statements by nesting AND, OR, and NOT constructs, test Boolean attributes of any object, and test spatial relations between objects. Then (then) and (else) arguments contain arbitrary code including, potentially, further If statements. For-each is a simplified looping construct. Each object in the list is in turn bound to the self variable, after which the code in the (code) argument is executed. The user may select an all-things option to fill the (list of objects) argument, causing the loop to iterate over all interface objects extant at the time the statement is encountered.

Set-attribute allows the user to adjust any object's attributes programmatically. Note that the (value) slot may be filled by a constant or a computed expression. Move causes the named object to be moved to the specified location. By default, the movement is along an interpolated path between the current position and the specified position. However, the user may change the speed and nature of the movement, just as with animation.

Trigger-action allows the user to trigger programmatically any actions (either built-in or user-defined) listed for the named object. In other words, this command provides an alternative means (other than interface events) of triggering actions defined for an object. This has proved especially useful when used in conjunction with user-defined actions: for example, the speech action discussed earlier is activated in this way. Trigger-action, in conjunction with the ability to create new user-defined actions for an object, fulfills our commitment to bring the power of object-oriented programming to the novice interface designer. In object-oriented terminology, the actions associated with an object can be viewed as methods, while the Trigger-action command implements a message-passing mechanism to activate those methods. As previously noted, we extend this paradigm with a unique twist: actions may also be activated by interface events.

Iboat Interface .' 9",".". . . . . .

.:.:.:;-:.:.:.:,:.:.:.:.~:.:.:.:.:-:.:.:.:~.1.:.::::

Actions for rope [speech 1 click action Name:

Code:[

,,

I {View}

I click action

sa~

r ~

[ e-rope-subj ]

~

{""n3

say Name:

SOy

Parameters:

[ e-rope-subj

]

sounds Name: sounds PoPametePS:

new sound

-.=.

[] L

e-good e-goodbye e-in-the-lifeboat e-lifeboat-sob e-lifevest-subj e-map-sobj e-no

I Can(

Fig. 3. A look at the structure editor; programming the one-click action of the Rope object to say its name (digitized sound)

~-nlll

4.1 Structure editor

5 Designer interface to QUICK

Despite the simple nature of QUICK language, programming would still be error-prone for novice users in a free-form environment. Thus, we provide a hierarchically structured editor that at each stage presents the user with only legal options from which to choose. In fact, the only time the user is actually asked to type anything is when an object or a file needs to be named. Figure 3 gives the flavor of the structure editor. In this figure, we are in the process of programming the one-click action of the Rope object of the lifeboat application to announce its name. The stacking of the dialogs illustrates the hierarchical nature of the editor. The user selects the one-click action to edit, is present with a code dialog, selects say from a popup menu (no longer visible in the figure), is presented with a specialized dialog for say, and selects the (initially empty) list of sounds to work on further. At the moment of this snapshot, the user has just depressed the mouse to display a popup menu of available sounds and has selected the digitized recording of "The rope." Thus, the structure editor combines the fill-in-theblanks and multiple-choice paradigms to create a highly constrained programming environment.

Control of the QUICK system is primarily accomplished via a hierarchically structured menu system. The structure of the menus is shown in Fig. 4. In general, the items in the Interface menu are concerned with the QUICK construction kit as a whole, while the items in the Objeet menu control

MQnq~

Menu Items

Submenu Item~

/ New

//Open / ~ Close

Interface ~

Save

~\~. Saveas... ~\ \\ \ Run

/ Local Actions / / Global Actions

//

\\Stop

\

\~Lexicon Quit New

// / Set Init Position // //Lower Plane / / / / / H i g h e r Plane / ~ , / Edit

/Duplicate

/.,,~ Actions ~/-'~Attributes Object ~ - - ~

f

~

/

Use Image Use Text Set Image - Set Text

~ -

Attributes ~ Record

~

Representation~

'~-Animation,,,~__~_ \\\'~ Group \ \ \ UnGroup ~'~ Save

Object Load Object

~

Run

Fig. 4. Overview of QUICK menu/submenu structure

123

w

laterlace

lUll['.lil

Wlfll]ows

foji Interface

Image

!

Mask

-

iiiiiiiiiiiF I;!L! ::::::::::::::::::::::::::: ,:.:.:.:.:.:.;.:.:.:.:.:.: :.:.:.:.:.:.:.:.:.:.:.:.:.

iliiiiii:!:i:i:i:i:i:~:::

q~I~iiiiJii!iN!Nii!iiiiiiiiiiNiiii!iNiNN

I[ .cce.t )

[ Change file)

[ Cancel

] Fig. 5. User is asked to select an image (and mask) for the new object. The image may be changed at any time

operations on individual objects or groups of objects. Although detailed discussion is beyond the scope of this paper, 5 several items warrant emphasis:

grammed. All other objects are very similar, and were, in fact, created by applying the copy-edit approach to the Rope object. The Rope object is created and programmed in three easy steps:

Run/Stop. These items control the mode of the sys-

1. Create the object. We simply select New from the object menu. The user is prompted for a name and asked to select an image for the new object.

tem and implement the rapid prototyping feature of QUICK. At any time during programming, the user may select Run to move the interface to the run mode instantly. In this mode, the interface behaves just as the run-time product will. Selecting Stop causes the interface to move back into programming mode. In this way, the user can quickly make changes, evaluate them, and make further modifications. Thus, Q U I C K supports a very tight modify-test iteration, a key feature of the rapidprototyping paradigm.

Duplicate. This important feature implements reuse in QUICK. The user can completely program a given object and then use the copy-edit approach to create a similar object.

6 Lifeboat revisited In this section, we take a more detailed look at how a Q U I C K interface is actually constructed by showing how the lifeboat interface described earlier was implemented. For the sake of brevity, we will show only how the Rope is created and pro5 See Douglas et al. (1990) for in-depth description of the Q U I C K system

124

2. Set the attributes. We want the Rope to be draggable at run time so that the user can move it into the lifeboat. Because the default value is non-draggable, we select the rope by clicking on it, and then select Edit from the submenu of the Attributes item in the Object menu. This invokes an attribute editor for the attributes of Rope, allowing us to toggle the value of the draggable? attribute. At this time, we also add a user-defined, Boolean-valued attribute called active, which marks an object as the current one asking to be placed in the lifeboat. 3. Program the new object. According to the specifications above, we must provide programs for two of the built-in interface events and one special event for each object: One-click action. The object should simply announce its name, in this case, "The rope." Landing action. When the object is dragged and then released by the user, it should determine whether or not it is in the lifeboat, and whether it is, in fact, the object that has asked to be moved to the lifeboat. If so, the object should announce

~'he

success (say "good"). If not, it should announce failure (say "wrong") and move back to its initial position.

Speech action. Ask to be moved into the lifeboat. The code for the Rope eventually generated via the structure editor is:

Landing action: IF [the value of ROPE's "active" attribute is TRUE] THEN IF [Rope is IN Lifeboat] THEN SAY [" Good"] SAY [" Put the lifevest in the lifevest in the lifeboat"] SET-ATTRIBUTE "active" of Rope to False SET-ATTRIBUTE "active" of Lifevest to True TRIGGER-ACTION Lifevest Speech ELSE MOVE Rope Initialposition SAY ["Wrong. The rope is not in the lifeboat"] T R I G G E R - A C T I O N Rope Speech ELSE Say ["You have moved the wrong object."] MOVE Rope Initialposition TRIGGER-ACTION Active_object Speech

One-click action: SAY [" The rope"] Speech action: SAY [" Put the rope in the lifeboat"]

7 Discussion and future work 7.1 Stretching limitations: how to have your cake and eat it Interface design toolkits can be characterized by the level of abstraction of the predefined entities provided by the toolkit (Douglas et al. 1990). At the lowest level of abstraction, toolkits like XWINDOWS and the Macintosh toolbox provide a large number of very low-level construction primitives. While this approach forces the interface programmer to explicitly construct all higher-level function-

Te '~

If Rope is current object to be moved to lifeboat

Make Lifevest the current object to be moved. Ask Lifevest to request to be moved to lifeboat. User moved rope but did not drop it in the lifeboat. Ask Rope to repeat request to move to lifeboat. The user moved the wrong object to the lifeboat (i.e. Rope did not ask to be moved). Ask the currently active object to repeat its request to be moved to the lifeboat.

ality from such primitives, it does provide unparalleled flexibility. At the other extreme, toolkits like Hypercard and Prototyper provide a relatively small palette of high-level interface entities (e.g., windows and buttons), allowing rapid construction, but rigidly constraining the types of interfaces that can be created. By settling on a middle ground, a median level of abstraction, we have necessarily struck a compromise between expressive power and abstraction. Is there a way that a toolkit can still maintain access to these features? We believe so and have worked to incorporate features that allow the designer to "stretch" the functionality of the QUICK system towards the extremes, as shown in Fig. 8. By providing mechanisms for aggregation, we support the assemblage of primitive interface entities into more abstract ones. Strong support for extensibility allows the user to easily incorporate fundamental level code in an evolving design. These as-

125

Interface mmnlnn~aiimWindows " New NN ojilnterface Duplicate Nil

'

~

f

======================== :::::.:,:.:.:.;.;.:.:.:.:~

Delete

NX ",....,.....................,.. :':':':'::':':':':::':': ,...........

Actions Bl/|ii~ n ~ l ~ | ~ Image Animation

O ............

:!:: _6 _ :

:

.

,..............,

..

.

), l m"~TnilPrunu ~-n~nauu],u TI, Higher Plane F Lower Plane

}/.'gl'e/i

I : :! = ~ 1 : ! 1

.

::

~ i ............... ~;~........................................................ ti~i;i;i:i:i:i:i:i:i:i:i:~:i:i Save Object ii:;i:;!:;!?ii?iiii!i!iiii:,!' Load Object !ii?ii!}!i!i!!iiiiiiiiii!

'

:::::::::::::::::::::::::::::::::::::::::::::::::

Interface Em31pllm Windows

click 17 click 2?

~ i i::!:! i iiii ............ ili'~:::::il ~"i' i i ' ::~ : : : : [ :::: .......i '

--

raggable~ raggable': m visible? highlighted?

[.Replace Current Value] (Replace Initial Value) [New Attribute] [.Delete Attribute )

plane

Current Value: True Initial Value:

False

Type:



pects are discussed in more detail in the following sections.

7.1.1

Creating

entities through

high level interface aggregation

A major premise of this work is that it is important to support not only the construction of a narrow class of standardized interfaces, but also the creation of innovative and unusual interfaces. In general, we work to be beyond specifying simply the shape of a window, button, or menu to support specification of what goes on inside the window as well. However, it is important to point out that

126

Fig. 6. a The user moves the object to where it should appear at run time and its initial position is set; b The object's attributes are set and new attributes defined; here, the Rope's "draggable?" attribute is being set to "true" to indicate that it can be dragged at run time

Q U I C K can be used to easily model standard-interface entities, such as windows, menus, buttons, and other " s t a n d a r d " interface entities. In this section, we present a short example that illustrates how Q U I C K can be used to create a simple "pull-down" menu. The approach is as follows: view the menu in terms of its component pieces; create each piece as a separate Q U I C K object, specifying the appropriate behavior; and juxtapose the components via direct manipulation. Importantly, the set of component pieces may then be aggregated (partonomic abstraction) into a new user-defined interface entity that can be stored and reused in later designs. We decompose a pull-down menu into the following components:

~. Interface I[tl~lltl'ill W i n d o w s New Duplicate

L

~N roji ~D

Interface

;;i;i; ...........:::::::::::::::::::::::::::::::::::::::::::: 0

Attributes

~

--

:

:~

)'

Global Action~

~ '~

I}rl)llp Ih~ ..,~poup

:~: ~.,,..........: .:-: .:

iiiiiiiiiii}iiiiiiiii!i!7Load:; O b j e c t ............liii!}ii}iii!i!iiiiiiiii ""'""'""'"'''

~:::::::+:

I n t e r f a c e llt],'II=Jlll Windows foji Interface

~

~

1

1

[ draggi.g action click action I click action

1

12

v

'

I click action Na~e:

"

'

[ ~

click action

Code:

animate cond flash

highlight move attribute? t r i g g e r local a c t i o n un highlight set

Fig. 7. a Programming the Rope object ... Entering the structure editor; b Programming the Rope object ... Programming the oneclick action (see also Fig. 3)

A menu title: tJ ~,~

I Low-level

Aggregation ~ " ~ ~.~)

Extensibility

I

I

I

I

1

I

I

I

I

Median

i

I

I

Representation: The descriptive title that appears in the menubar. Behavior: When single-clicked, highlights itself and causes the menu background and items to become visible.

High-lever

Mechanisms for extensibilly and aggregation provide access to the best features of all worlds.

Fig. 8. Stretching capabilities through aggregation and extensibility

A Menu background: Representation: A blank oblong box large enough to contain all menu items. Behavior: None.

127

Menu items:

Representation: Text items representing the choices in the menu. Behavior: Single-clicking cuases item text to flash and then initiates the action defined for that item. Figure 9 shows how the primitive menu components are composed to create a more abstract interface entity. Now suppose the designer aggregates the five component objects into a new compound object under the name menu-object. If at a later point (in the same design or during a new one) the designer requires another menu, the stored m e n u - o b j e c t can be loaded in and trivially modified to fit the new task. Modification would primarily involve changing the text strings of the menu title and items and assigning appropriate actions to each item. In sum, the less-structure approach to interface design taken in QUICK in no way impedes designers of "standard" interfaces. By way of partonomic abstraction, designers may compose an unlimited variety of higher-level interface entities, including widely used entities, such as windows, menus, and buttons. In this way, Q U I C K provides all of the time-saving power associated with the composition-based approach to interface design, in which predefined interface components are selected and composed, while at the same time maintaining flexibility and the ability to design unique interfaces. Thus, Q U I C K is well-suited to the needs of both Paste Copy

Edit

Cut

a) Juxtapose the components to create the menu object

Edit

Click Edit

Click

Copg

Edit Click Edit

b) The new Menu object in action

Fig. 9. Creating a more abstract, specialized interface entity by aggregation

128

designers interested in creating uniform, standardized interfaces as well as designers wanting to explore new concepts in interface design.

7.1.2 Extensibility for greater expressive power For some programs that are interface intensive, with very little underlying computation to be done, the interface generated by Q U I C K is the finished application. That is, there is no further programming to be done after the interface is created. The lifeboat example presented earlier is a prime example of such an application. However, in the case of more complex software, which may involve intricate data structures and elaborate computations, the sophisticated design-kit user will, at some point, need to implement functionality, which simply is not supported by the abstractions embodied in the design toolkit. At this point, the designer must resort to a less-abstract specification language, most commonly the low-level language on which the toolkit is based. In these cases, the interface construction kit must become a tool to streamline the mundane aspects of interface construction, freeing human programmers to focus on unusual interface effects and computational asepcts beyond the scope of the interface toolkit. The Q U I C K system strives to support this goal through a variety of features: - Q U I C K generates fully functional Lisp code, which can be arbitrarily extended and modified by the designer. - Unlike many other toolkits, code that has been modified/extended by the user may still be loaded by the Q U I C K system, allowing further changes to be made. The QUICK language provides a mechanism for referencing values produced by arbitrary Lisp objects. Requests for such values are simply incorporated into the generated code and are resolved at run time. In this way, interface code generated by QUICK may easily reference values produced by another application code. - Q U I C K language is fully extensible. The user may create a library of personalized interface functions to extend the primitives provided by the QUICK language. - QUICK prototypical object is extensible (as discussed previously) - the user may arbitrarily add

r~

new attributes and define new user-interface events. The purpose of this section has been to emphasize that, while the simplicity of Q U I C K makes it an especially promising tool for unsophisticated designers, this feature in no way compromises its power as an interface design tool. By aggregating simple interface entities, it is possible to model more complex entities, such as buttons and menus. Although the median level of abstraction adopted in Q U I C K allows a wide range of interfaces to be created without leaving the Q U I C K language, the system provides a variety of mechanisms to support low-level elaborations.

7.2 Work in p r o g r e s s animation Q U I C K systems use a version of what has become known as the path-transition paradigm (Stasko 1990). Currently, the Q U I C K system allows the user to record an animation for an object in a bydemonstration fashion, directly moving the object through the desired motion, while the system "watches," recording the motion. This results in a vector of points, the path, which is stored by the object. As discussed earlier, the user is able to specify certain characteristics that determine how the system uses this vector to animate the object at run time. While this level of support for the specification of animation is unusual, we have found that there is room for improvement. Specifically, there are two areas in which we are working to address:

7g

simultaneously, it becomes desirable to support synchronization of animations. One might, for instance, want to specifiy that two objects with intersecting paths arrive at the intersection at the same time. In short, we would like to be able to specify some set of constraints that govern the interaction of simultaneous animations. Based on our observations of users, we are more interested in qualitative constraints. For example, users often express the desire to specify that some (animation) event occurs before another - but they rarely need to specify that it occurs, say, 5,4 s before the other. Our current efforts have focused on a solution that promises to address both of the problems above: a graphical path editor, which diagrams out the paths recorded for one or more objects and allows the user to directly manipulate the paths, to place landmarks on the paths, and to specify constraints between landmarks. Figure 10 shows our first prototype. As indicated in this figure, the path is illustrated as a line through all of the points in the path, which show up as small black squares. The user may "grab" any square and move it to readjust the path. 6 In addition, the user may place landmarks, indicated by larger diamonds, at any point on a path. Landmarks may then be related by simple constraints. In this way, we provide both a means to easily edit an animation path and a mechanism to synchronize multiple animations.

i~

- -

Swoop

Interface

I

Path editing. Often, it becomes desirable to alter the recorded path in some way, either because the user was not able to record it accurately (i.e., lack of precise motor control) or because later developments in the design process call for a change. As indicated in Sect. 3, a path editor is currently provided. However, this editor is quite primitive - the user is simply allowed to edit the vector of points that comprise the path. Not only is this time-consuming, it is clearly incompatible with the direct visual style of interface programming embodied by QUICK. The end result is that users generally tend to simply re-record a path, instead of editing an existing one.

Synchronizing animations. Given that every interface object may have an animation recorded for it and that any number of animations may occur

~

~

~:!i;8::'4~..... I : 4::,

Fig. 10. Animation Path Editor. Recorded paths are indicated by lines. The two diamonds connected by the dashed line are landmarks that have been related by a constraint; the equal sign indicates that the landmarks must be reached at the same time 6 Much of the inspiration for this approach was gained from the Bezier reshaping features now available in current drawing programs

129

7.3 Work in progress: specifying functionality Another shortcoming we have identified lies in the structure editor, the designer's interface to the QUICK language. Preliminary evaluative studies (see next section) have revealed that, while the structure editor does, in fact, preclude all syntactic and many semantic errors, it is too unwieldy and complex - the Q U I C K programmer is required to move through a (potentially deep) hierarchy of specialized dialogs in order to specify the functionality of interface objects. Thus, programming can be viewed largely as textual selection. Again, this appears to be in conflict with the direct visual style we aspire to in QUICK. On the other hand, our initial decision to use a structure editor was based on the conclusion that purely visual approaches (Serius 1989) quickly become indecipherable for all but the smallest of programs. Thus, we are currently exploring several compromise approaches which maintain the robustness of the structure editor, but do away with most of the stacked editing dialogs.

8 Evaluation We believe empirical evaluation of software has been greatly neglected in recent work and have a strong commitment to reversing this trend. Thus, we are currently engaged in two distinct evaluation efforts. Our first study works to evaluate the utility of Q U I C K as an interface design tool for programmers. We plan to gauge utility by combining several metrics: coding time, coding correctness, and attitude. Importantly, we are interested in gaining insight into the contribution of the direct manipulation and object-oriented aspects of QUICK to its overall success as an interface toolkit. For instance, we are curious about whether subjects will reuse objects and how they fare with the prototype model of copy and edit (as opposed to the standard object-oriented model of class-hierarchy specialization). A study by Lange and Moher (1986) will give us classifications of coding strategies that pay particular attention to reuse. The study involves ten pairs of subjects working in pairs. We have found this technique, known as constructive interaction (Miyake 1986), to be much more productive than the well-known "thinkaloud" approach (Newell and Simon 1972) in pro-

130

viding insight into a subject's problem solving process. Each pair is shown a 30-min videotape of how to use QUICK and then asked to implement three distinct interface specifications. While apparently quite different, the three problems are designed to provide opportunities for reuse. The resulting videotapes are then analyzed to give the above performance measures. Preliminary results indicate that, even with minimal training, programmers are able to produce simple interfaces using QUICK. However, several disturbing trends are beginning to emerge as well. First, although subjects have the opportunity for reuse, both within (via copy-edit) and between interface design efforts, they generally fail to do so. We feel that part of this is due to a failure to detect functional similarity between objects in the same as well as different designs. This is an area where a more interesting study could be done based on existing cognitive psychology research in analogy. We also note that a major problem with reuse is that frequently designers fail to predict what might be useful in a future design and consequently do not save objects either individually or as aggregate objects. Our second study is more general and addresses the question of how useful Q U I C K is when compared a) to low level interface programming and b) to other interface design toolkits. A problem in addressing the first objective is that the only expert QUICK programmers available are the four designers, only two of whom are still at the university. Therefore, we are able to report a very preliminary result. We asked a QUICK expert to use the system to prototype a direct manipulation cardiovascular simulation interface developed on another project. The programmer of the original interface, working in Allegro Common Lisp, took approximately 500 h (six months) to complete it. Our QUICK expert produced a prototype in approximately six hours. We believe that much of this improvement is due to the direct manipulation techniques and the high-level object-oriented Q U I C K language. With object-oriented programming, Schmucker (1986) estimates that development time can be reduced by a factor of four or five.

9 Conclusions The concept of interface-construction kits has become become firmly established and well-accepted in recent years. QUICK system allows novice inter-

face designers to rapidly construct sophisticated graphical user interfaces by direct manipulation and use of a high-level user-interface programming language. Interfaces that non-programmers are able to construct include standard-interface objects, such as menus and buttons, as well as complex user-object interactions that occur within windows. In designing QUICK, we have placed special emphasis on a highly interactive construction environment and incremental development - the user is able to rapidly move between programming and testing the developing interface. Our work has made three major contributions to user-interface-design toolkits by demonstrating the feasibility and improved programming productivity offered by 1) prototype object-oriented programming, 2) high-level user interface programming language, and 3) direct manipulation environment.

Prototype object-oriented programming. We have demonstrated the feasibility of a non-inheritancebased object-oriented approach, namely the prototype. Designers apparently find the object-oriented viewpoint very natural for interface design. The prototypical view of an object taken in QUICK seems particularly suited for non-programmers. Instead of having to first become familiar with the various predefined types of objects available in a class-inheritance system and then facing the nontrivial task of deciding how instances of those classes can be composed to implement the specification, the novice simply creates a generic QUICK object and programs it to behave appropriately.

High-level user-interface programming language. Our goal has been to identify a minimal set of representational and functional abstractions that are conceptually simple, but powerful enough to allow construction of a wide variety of user interfaces. The QUICK language is an initial effort at describing such a high-level language. Because it is fully extensible, the user may add more elements to the language as demanded. We have included in the language a set of spatial-relation functions, which allow the user to specify graphic positions without using an xy coordinate system. These functions are based on empirical psychological experiments that we have conducted and are thus an unusual feature of the system and contribution of the research. Direct manipulation environment. We have combined the previous two approaches into a programming environment based on direct manipulation.

The primary advantage here appears to be avoidance of symbolic programming. Visual programming and programming by demonstration allow both novice and expert designers to construct an interface in much the same way they construct realworld artifacts (e.g., sandwiches, interior design) by directly moving components about and monitoring results visually. Related to direct manipulation is rapid prototyping. Novice designers have difficulty visualizing the entire solution to an interface specification and often have little confidence in their work. Experts often must decide between several alternative approaches to implementing a given feature. Hence, rapid prototyping allows designers to "chip away" at a design, implementing and testing various aspects of the interface individually. Because QUICK provides a rapid switch between programming and using the created interface, it supports rapid prototyping. By taking a different, slightly more abstract view of the interface design process, we have created a system with some very unusual characteristics. Our strong commitment to a interactive concepts, such as direct manipulation and programming by demonstration, transform the interface-construction process from a static, textual process into a dynamic, visual process. By avoiding the limitations associated with a fixed, predefined library of interface entities often found in other kits, and, instead, allowing the user to define new abstractions, we have created a construction kit that is extremely flexible. Not only can we emulate the entities provided as primitives (e.g., buttons, pull-down menus) in other construction kits, we also are able to incorporate unusual features, such as user-defined graphics, sound, and animation. The QUICK system is an interpreter implemented on the Macintosh II computer in Allegro Common LISP using Allegro's Object LISP system. It generates LISP code that can be supplemented by additional LISP programming if necessary. Both graphic images and digitized sounds are created by other pieces of software and are easily imported into QUICK. The source code is roughly 700 K in length. Previous publications on QUICK (Douglas et al. 1990, 1991) have discussed in more detail the Lifeboat example used in this paper as well as another extensive example, which is an icon-based construction kit for a cardio-vascular simulation interface. Readers seeking more details on the system are directed to those sources.

131

Acknowledgements. This research has been funded in part by a grant from the U.S. Department of Education (FIPSE) and by a contract from US WEST Advanced Technologies Lab. We would also like to acknowledge the work of Gary Smithrud, who programmed the first versions of QUICK.

References Apple (1987) Hypercard user's guide. Apple Computer Barth PS (1986) An object-oriented approach to graphical interfaces. ACM Trans Graph 5(2): 142-172 Borning A (1979) ThinkLab - a constraint-oriented simulation laboratory. Unpublished doctoral dissertation. Stanford University Borning A (1986) Classes versus prototypes in object-oriented languages. Proc ACM/IEEE Fall Joint Computer Conference, Dallas Cardelli L (1987) Building user interfaces by direct manipulation. DEC Systems Research Center Technical Report No. 22 Douglas SA, Novick DG, Tomlin RS (1987) Consistency and variation in spatial reference. Proc Ninth International Conference on Cognitive Science, Seattle Douglas SA, Doerry E, Novick DG (1990) QUICK: a user interface design kit for non-programmers. Proc SIGGRAPH Symposium on User Interface Software and Technology, Snowbird Douglas SA, Doerry E, Novick D (1991) QUICK: exploring the middle ground in user interface design tools. Proc 24th Hawaii International Conference on Systems Science, Kailua-Kona Fischer G, Lemke AC (1988) Construction kits and design environments: steps toward human problem-domain communication. Hum-Comput Interact 3(3): 179-222 Fischer G, Lemke AC (1989) Knowledge-based design environments for user interface design. Submitted to the 1lth International Conference on Software Engineering Gould L, Finzer W (1984) Programming by rehearsal. Technical Report No. SCL-84-1, Xerox PARC, Palo Alto Henderson DA (1986) The trillium user interface design environment. Proc Human Factors in Computing, CHI'86, Boston, pp 221-227 Lange B, Moher T (1986) Some strategies for reuse in a objectoriented programming environment. Proc Human Factors in Computing, CHI'89, Boston, pp 221-227 Miyake N (1986) Constructive interaction and the interative process of understanding. Cognitive Sci 10:151-177 Myers B (1986) Visual programming, programming by example, and program visualization: a taxonomy. Proc Human Factors in Computing systems Conference~ CHI'86, Boston Myers B (1990) Creating user interfaces using programming by example, visual programming and constraints. ACM Trans Prog Lang Syst 12(2): 143-177 Newell A, Simon H (1972) Human problem solving. PrenticeHall, Englewood Cliffs Schmucker K (1986) MacApp: an application framework. BYTE 11 (8) Serius89 (1989) Serius89 Documentation Release 1.1. Serius Corporation Shaw M (1986) An input-output model for interactive systems. Proc Hum Fact Comput. CHI'86, Boston, pp 261-273

132

Shneiderman B (1983) Direct manipulation: a step beyond programming languages. IEEE Comput 16(8): 57-69 SmethersBarnes (1989) Prototyper user's manual. Smethers Barnes Publishing Division, Portland Smith RG (1984) On the development of commercial expert systems. AI Magazine 5(3):61-73 Stasko JT (1990) The path-transition paradigm: a practical methodology for adding animation to program interfaces. J Visual Lang Comput 1(3):213-236 Stasko JT (1991) Using direct manipulation to build algorithm animations by demonstration. Proc Human Factors in Computing. CHI'91, New Orleans, pp 307-314 Ungar D, Smith RB (1987) SELF: the power of simplicity. Proc Object-Oriented Programming Systems, Languages and Applications Conference, Orlando

SARAH DOUGLAS is associate professor in the Department of Computer and Information Science and a member of the Cognitive and Decision Sciences Institute at the University of Oregon. She received her PhD from Stanford University in 1983 and completed a research internship at Xerox Palo Alto Research Center. Dr. Douglas is particularly interested in research on user modeling in both standard and AI interfaces, natural and artificial language interfaces, and the process of user interface design. Dr. Douglas is currently doing research on high-level object-oriented languages for user interfaces, simulation-based tutoring systems, and representation of spatial and causal relations in interfaces. ECKEHARD DOERRY is currently a PhD student in the Department of Computer and Information Science at the University of Oregon, after having completed his MS there in 1989. He graduated summa cum laude from Northern Arizona University in 1987 with a BA in mathematics and a BS in computer science and engineering. Doerry's current research falls in the area of user interfaces, with special emphasis on user-interface design and interfaces for intelligent tutoring systems.

DAVID NOVICK is assistant professor of Computer Science and Engineering at the Oregon Graduate Institute. He earned his BA (Honors College, political science and economics), MS, and PhD (computer and information science) at the University of Oregon, and JD from Harvard University. Dr. Novick's research interests include interactive systems, including technologically mediated interaction and control of interaction among intelligent agents, and human-computer interaction, including methods of interface design and implementation. A central theme of this research is the role of meta-acts in control of interaction.

133