Temporal Scripting using TEMPO Laurent Dami Eugene Fiume1 Oscar Nierstrasz Dennis Tsichritzis Centre Universitaire d'Informatique Universite de Geneve 12 rue du Lac CH-1207, Geneve SWITZERLAND email:
[email protected] (csnet), dami@cgeuge51 (bitnet) January 12, 1994 Abstract A language for scheduling temporal activities in an object-oriented environment is proposed. Particular emphasis is put on reusing temporal speci cations in various contexts, which is achieved through a mechanism of parameterized sampling based on a notion of virtual time. A small set of temporal operators are used to build parallel and sequential compositions of activities. Complex activities can be encapsulated as object methods, thereby making them more easily reusable. The system, called TEMPO, has been implemented within C++; it has been used for generating computer animations, and is easily extendable to other time-driven applications. Keywords: temporal speci cation, active objects, scheduling, virtual time, concurrency, computer animation 1
currently at Department of Computer Science, University of Toronto
1
1 Introduction Digital computers only work with discrete instructions, so when a computer is to be used for producing output in a continuous space, some sampling decisions have to be taken: for example, mapping a curve to a nite set of vectors or points, or mapping a robot motion to a discrete sequence of commands to its motors. Conversion from discrete to continuous space can then occur either explicitly, like a series of sound samples being translated into a continuous signal for the loudspeakers, or implicitly, like a sequence of images being perceived as an animated scene. Deciding a which stage the sampling should occur is a dicult problem: early sampling can make implementations more ecient, whereas late sampling is more appropriate for supporting high-level modi cations and combinations. The problem is particularly well illustrated by the competing windowing systems on the market: some systems, like Suntools or X windows, adopt an early mapping from windows in the programs to bitmaps on the screen, while systems based on the Postscript language (NeWS, DisplayPostscript) leave that mapping to the window server. The rst approach is fast for low-level operations, like drawing a vector, moving a window or painting an area in reverse video, but is far less exible than the second approach, that for example allows a user to resize any window and automatically get its content to be recomputed according to the new size. In the time domain, the sampling problem is particularly crucial. In order to get a computer to perform an activity that spans over time, the programmer must usually nd a way to divide that global activity into a discrete sequence of small steps that can be performed by the program. Consequently, the structure of the program re ects the sampling scheme adopted by the programmer; that is, the sampling scheme is \hard-coded" in the program. Now consider the following situations:
we want to execute the activity at a di erent speed we want to execute the same activity on another machine, with a di erent computing
power we want to combine the activity with another one, in order to get a more complex behaviour
In all those cases, although we want something very similar to what we had before, it will be dicult to reuse the same code, since the speci cation of a conceptually continuous activity cannot be dissociated from the way it was implemented in a program in terms of a discrete sequence of instructions. 2
We were confronted soon to that problem after we started working on an object-oriented environment for computer animation. Our purpose was to build a library of objects - for example the object Human - with sets of prede ned methods for motions - like a walk method - and to be able to specify complex animations by combining several objects and motions from the library. In other words, our main concern was to support reusable temporal speci cations, so that the design of an animation does not have to start always from scratch again. In order to reach that goal, we had to nd a way for describing motions independently from their sampling, in a manner such that the same walk method could be applied for a character walking during 10 seconds and for another walking during 50 seconds, and that the same animation could be recorded either for lm at 24 images per second or for video at 30 images per second. We choose to handle the problem with an approach based on late sampling. The TEMPO system described in this paper - TEMPO stands for `Temporal Expressions Managing Parallel Objects' - has a model of continuous time, in which temporal activities can be speci ed and combined into more complex con gurations. Once a whole temporal con guration is built, it can be sampled into a sequence of global states of the system. The sampling strategy is chosen at run-time, according to the user's needs, which provides an interesting degree of exibility. TEMPO has been designed as a scripting language. By scripting we mean that it is a small language, implemented on top of a target language, and especially tuned to make certain kinds of bindings easier to specify. General ideas about scripting languages and software reuse are presented in [Kapp89]; their application in the case of TEMPO resulted in a language of temporal expressions that uses a small set of temporal operators to combine activities together. Common temporal situations can be scripted as simple temporal expressions; unusual combinations, if they are not expressible as temporal expressions, can always be programmed within the target language. The paper rst exposes the general execution model of TEMPO, and then gives the syntax and semantics of temporal expressions. Then the current implementation is described, together with some examples of its use. Examples throughout the paper will be mainly inspired from our primary application domain, namely computer animation. We are convinced, however, and shall try to illustrate, that the same principles can be applied successfully into various other areas, like robotics, simulation, process control, computer music.
3
2 Basic concepts of TEMPO TEMPO is designed to work in an object-oriented environment. Object-oriented programming is now suciently widespread so that we do not need to present it here in detail; good descriptions can be found in [Nier86], [Cox86] or [Meyer88]. At run-time, a TEMPO program is a collection of objects; objects have private instance variables to hold state information, and have sets of methods available to the other objects through which their state can be seen and/or modi ed. Objects are associated to classes, that describe their structure and the list of methods. Objects evolve under the control of temporal expressions, that de ne trajectories in the state space ( gure 1). Unlike method invocations, that provoke immediate state modi cations, temporal expressions specify changes that take place over a speci c period of time. For example, we might specify that variable x is to be linearly interpolated between values 10 and 100 over the duration 4.5. This does not mean that every intermediate value will be computed; it merely states that the program is able to compute a new value of x for any advance in time. So if time advances regularly by units of 1.5 (as will be explained later), x will successively take the values 10, 40, 70, 100, 100, 100, 100, : : : . Temporal expressions that control the evolution of only one object may be encapsulated in one of its methods; for instance our animation library contains a Human object whose interface has a method called walk, which is a temporal expression. The temporal units of the above example are not real-time units. We use a notion of virtual time within the program, in order to synchronize and combine various temporal expressions. Virtual time is continuous, so durations and delays may take any positive real value. Virtual time units are used to specify the relative durations and starting times of expressions; they do not say anything about their actual computation time. So the example would yield exactly the same output if the duration were 30 and the advances in time 10 units. Choosing a duration is therefore arbitrary, and makes only sense as compared to other durations in the same program. The execution of a program proceeds by discrete jumps in virtual time ( gure 2). After a jump the new state of the object world may be used by the program to produce some output: when we use TEMPO for computer animation, a virtual camera generates an image after each jump, looking at the new positions and orientations of the objects in the scene. From a given state, the system can jump to any state in the future; the amount of virtual time to skip must be speci ed at each jump. Therefore each application can choose the most appropriate sampling policy. For example when we design an animation, we frequently 4
6
5
3 4 1 2
0
Fig 1: A temporal expression is a trajectory in the state space Numbers are temporal units. The trajectory need not be continuous, but a state is defined for every real t.
+=1.2
6
•
3
+=2
•
• 4
+=.8
0
•
5
+=1.2
1 2
•
Fig 2: The expression has consumed the temporal steps Although the trajectory conceptually defines a state for every real t, the expression only went through discrete jumps.The amount of time that is consumed at each jump is decided by an external sampler. From its current point, the expression can go to any suceeding point on the time line.
5
generate a rst draft with very large jumps, just to check if the positions and movements are correct; the resulting animation is of poor quality, because of undersampling, but it is quick and cheap to produce. If the draft corresponds to our expectations, we then generate a higher-quality animation, using exactly the same speci cation, but with a higher sampling frequency. While a jump is being computed, objects are updated to their new state in a random order; the system only guarantees that at the end of the jump all objects that were performing temporal activities have been updated. Therefore the updating process can be done in parallel - our current implementation uses a coroutine package to do pseudo-concurrency. The whole purpose of having a continuous time model was to be able to reuse temporal speci cations in various contexts. The binding between a speci cation and its context can be done by giving parameters to the free variables in the speci cation, and by using temporal operators to combine it with other temporal speci cations. For example, assuming that we have an interpolate function in our library, that brings a variable from an initial value to a nal value over a speci c period of virtual time, we can write the expression: interpolate(x, 10, 100, 4.5) & (3*interpolate(y, 10, 50, 3.2))
that starts an interpolation on variable x for 4.5 units of virtual time and at the same time starts a sequence of 3 interpolations on y, each of which will last 3.2 units. Any temporal expression that uses temporal operators can in turn be encapsulated as a library function, and then become integrated into more complex expressions. For example the walk method mentioned earlier involves a complex combination of motions of the arms and legs; nevertheless, it is easy to combine several walking characters and other moving objects in an animation, because the complexity of the walking motion has been hidden under the walk method abstraction, under the assumption that the motions are context independent. Since the speci cation of activities is independent from sampling, various sampling policies can be explored. We already mentioned that a same animation program can generate either a limited number of pictures for testing the animation, or a larger number for high-quality output. Another interesting approach is to bind the sampling to real-time: following that path, we designed a sampler that dynamically chooses the amount of virtual 6
time to jump according to the real-time elapsed during the previous jump. In other words, if the process of updating the objects from one state in virtual time to another takes time t to compute, then the next jumps in virtual time will be adjusted to t that value t. The result of such sampling behaviour is that the application always runs at the same apparent speed, but constantly adapts its rendering quality to the processing power of the machine. If it is an animation, we will see fewer images when the machine is heavily loaded, and more images when the processor has more time for our application; but in both cases the apparent motions of the objects will be at the same speed. This may be quite useful for example for writing a monitoring program capable of graceful degradation of performances.
3 Activities and aging methods When a temporal expression is activated, it gets bound to a particular environment, and then the objects in that environment can start to evolve. We shall use the word activity to talk about an activation of a temporal expression; domain(A) denotes the set of objects controlled by activity A, and state(A) denotes the state of the objects in domain(A). The domains of activities should be disjoint, even though our current implementation does not check it: requesting a single object to perform multiple activities concurrently will yield unpredictable behaviours. For example the variable x cannot perform two di erent interpolations at the same time. It is perfectly possible, however, to partition an object on its instance variables, like asking a Human object to perform an activity with its arms and move its legs at the same time - this is exactly what is done in the walk method. Any activity has a method for jumping in the future by any amount of virtual time, thereby modifying the state of objects in its domain. The jumping method, also called aging method, is written `+='. For example the instruction: A += 0.8;
lets activity A jump 0.8 units of virtual time in future. State(A) will be modi ed accordingly. From that notation it is obvious that the amount of virtual time to be skipped is a parameter that can vary at each jump. Since the parameter to the aging method can be any positive real value, it may well happen that it is larger than the amount of time actually needed by the activity to ful ll its job. Consider the following example: 7
Activity A = interpolate(x, 10, 100, 4.5); A += 10;
Here activity A, which is supposed to last 4.5 units of virtual time, receives the instruction to jump 10 units in future. What will happen is that A will set the variable x to its nal value, namely 100, and will remember that some virtual time has not been \consumed". The amount of time unused by an activity can be retrieved with the timeleft function; at the end of our example, timeleft(A) is equal to 5.5. An activity for which timeleft is non-zero is a terminated activity, that will no longer a ect its environment. More formally, if A has terminated, then ( (A+= t) = state(A) 8t 0 : state timeleft(A+= t) = timeleft(A) + t Each activity decides for itself when it terminates (if ever). It is perfectly possible to write temporal expressions that have no xed duration, but will terminate when a particular condition is met. For example an expression could increase the value of x until it gets higher than 200; the duration of such an expression would depend on the initial value of x.
Continuity property: The model of continuous time in TEMPO assumes that a state of the object world is de ned for every time t 2 R+, and that this state does not depend
on the speci c history through which time t was reached. Therefore it is very important that all aging methods comply with the following constraint: 8t1; t2 2 R+; state((A+= t1)+= t2) = state(A+= (t1 + t2)) Temporal expressions are either so-called \basic expressions", in which the programmer directly supplies the aging method, or compound expressions involving temporal operators. In the latter case, it is easy to check, through the semantics of the temporal operators, that the constraint is respected. In the former case, however, it is up to the programmer to comply with the continuity constraint; the constraint cannot be checked by the program. Therefore \basic expressions" should be written with care, and are usually hidden in libraries; \normal users" are likely to use only the temporal operators for combining prepackaged expressions.
4 Temporal expressions The syntax of temporal expressions is quite simple: 8
temp_expr : | | | | | | ;
basic_temp_expr '(' temp_expr ')' temp_expr '>>' temp_expr temp_expr '&' temp_expr positive_real '>>' temp_expr positive_integer '*' temp_expr temp_expr '[' speed_modulator ']'
// // // // //
sequential execution parallel execution delay repeated execution speed modulation
Basic expressions will be discussed later, since they require some low-level implementation details. We shall concentrate for the moment on the semantics of temporal operators. The semantics is given in terms of the two operations that are de ned on activities, namely the aging and timeleft functions. Proofs that all operators comply with the continuity constraint are given in the appendix.
4.1 Sequential execution Sequential composition of two expressions E1 and E2 is written E1 E2. The notation E1; E2, that would probably have been more intuitive, has been rejected for implementation reasons (operator ` ' can be overloaded in C++; operator `;' cannot). The compound expression (E1 E2) will rst redirect all aging instructions to the sub-expression E1. When E1 terminates, its un-used virtual time is transmitted to E2. Subsequent temporal steps received by (E1 E2) are forwarded to E2. This can be stated formally: 1. aging function: ((E1 E2)+= t) = ((E1+= t) E2) 2. timeleft function: timeleft(E1 E2) = timeleft(E2) 3. reduction rule: if timeleft(E1 ) 0, i.e. if E1 has terminated, then (E1 E2) = (E2+= timeleft(E1))
9
4.2 Delayed execution The expression (x E ) states that E has to wait x units of time before starting. Its obvious semantics can be written as follows: (
x ? t) E ) if x t 1. aging function: ((x E )+= t) = (( (E += (t ? x)) if x < t 2. timeleft function: timeleft(x E ) = timeleft(E )
4.3 Parallel execution Parallel execution of two expressions E1 and E2 is written (E1&E2). The aging method for the compound expression merely transmits jumps in virtual time to the sub-expressions, until both have terminated. 1. aging function: ((E1&E2)+= t) = ((E1+= t)&(E2+= t)). 2. timeleft function: timeleft(E1 &E2) = min(timeleft(E1); timeleft(E2)).
4.4 Repeated execution An activity can be repeated in expressions of the form (n E ), where n is a positive integer value. The de nition can be given recursively: (
E if n = 1 nE = E ((n ? 1) E ) otherwise
4.5 Speed modulation Speed modulation provides external control over the apparent speed of an activity, while the activity internally always behaves at constant speed. Aging instructions are passed through a lter that modi es the values of jumps in virtual time. Therefore a di erent 10
temporal behaviour can be obtained without any modi cation to the original code of the activity. For example, if we have an animation expression in which a bird is apping wings, and we now want the bird to accelerate, we need not program a new ap motion: it suces to use an accelerate lter to modulate the speed of the existing motion. Modulating lters can perform a variety of interesting changes to a temporal behaviour, like combinations of accelerations and deccelerations, or transforming a continuous motion into a discrete one. They perform the mapping between external time and internal time. They must be monotonic increasing functions from R+ to R+ , and they must have an inverse. For example a possible accelerating lter could be: f (x) 0:2x2 We write E [f ] for \expression E is modulated by lter f ". In order to describe its semantics, we need an intermediate function, derived from f , that re ects the fact that we must support discrete jumps in virtual time. So for any lter f , we de ne: fc0(t) f (c + t) ? f (c) , where c stores the `current time' of the expression. Then E [f ] is de ned as: E [f ] = E [[f00 ]] where the intermediate operator E [[fc0]] has the following semantics: 1. aging function: (E [[fc0]]+= t) = (E += fc0(t))[[fc0+t]] 2. timeleft function: timeleft(E [[fc0 ]]) = c ? f ?1(f (c) ? timeleft(E )) To illustrate the use of speed modulation, consider a lter generator accelerate that, for a given initial speed and acceleration factor, yields an accelerating lter: accelerate(a; v0) x:ax2 + v0x Now if we apply it to an activity A, as in the expression: Activity B = A[accelerate(0.2, 0)];
then the sequence: 11
B += 1; B += 1; B += 1; B += 1; B += 1; B+= 1;
is equivalent to: A += 0.2; A += 0.6; A += 1.; A += 1.4; A += 1.8; A += 2.2;
so that, when B 's local time is 6, A's local time is already 7.2. As a matter of fact, 7.2 is the result of applying the function accelerate(0:2; 0) to the value 6. Therefore the relationship between external time and internal time has been preserved, independently from the number of jumps in virtual time (a single instruction B += 6 would also have been equivalent to A += 7.2). Hence, the speed modulation operator also complies with the continuity constraint. A proof is given in the appendix.
4.6 Other operators We have limited ourselves until now to the operators that seemed most generally useful. Nothing, however, should prevent a user from de ning new operators to suit particular needs. The only requirement is to provide de nitions of the `+=' and timeleft functions that respect the continuity constraint discussed above. We could imagine, for example, an alternate operator that would alternatively send aging requests to several activities. In a previous version of this work, activities always had a xed duration, so that we could implement other temporal operators like E1 o E2 (simultaneous termination), or E1[t1] E2[t2] (general synchronization of E1's local time t1 with E2's local time t2. These operators must know the ending time of an activity; they are described in detail in [Fium87]. They have been abandoned in our current version of TEMPO, because we found too limiting the constraint of having only activities with a xed duration.
5 Implementation of basic expressions Using the temporal operators, complex hierarchies of expressions can be speci ed. At some level, however, these hierarchies have to rely on a set of so-called \basic expressions" that directly implement the aging behaviour. The current chapter will tackle some delicate 12
questions about writing such basic expressions; it may be skipped by readers who do not want to go into that level of detail. The way basic expressions are written is highly dependent on the environment chosen for the implementation. We currently use the C++ programming language [Strous86], mainly because temporal expressions can be easily integrated into the language by means of its operator overloading feature. Additionally, C++ is one of the few object-oriented languages that are strongly typed, which is an important advantage for managing large libraries of objects. We de ned a C++ class Script for encapsulating temporal expressions, and we implemented the operators `&', ' ', `*' and `[]' on objects of that class, according to the semantics given in the previous chapter. The implementation dynamically creates a parse tree for each temporal expression met in the program. Intermediate nodes in the tree correspond to temporal operators; they are used to transmit aging requests down to the leaves. The leaves are associated to basic activities (i.e. activations of basic expressions). An example of a basic expression is the interpolate function that was used in earlier examples. Since there are no lower-level functions that could be combined with temporal operators to get the interpolating behaviour, this has to be implemented as a basic expression. Its declaration looks like: Script interpolate(float &x, float a, float b, float d);
where x is a reference to the variable that will be interpolated, a and b are the starting and ending values for the interpolation, and d speci es the duration in virtual time units. The function is of type Script, which means that it will return an object that complies with the protocol for temporal expressions, namely that has a method for aging itself. The object returned by interpolate will in fact encapsulate a coroutine. The reason for using coroutines is that a basic expression cannot be executed immediately, as would occur with any `normal' function call. Basic expressions can only proceed when they receive instructions to jump in virtual time. Between jumps, the binding between the function and its parameters must be kept, as well as some local information like for example the current time (sum of all temporal jumps that have been executed up to now). Coroutines precisely give us the possibility of keeping a local state while suspending execution, and are easier to implement than more sophisticated models of concurrency. Within coroutines, aging behaviours are speci ed as sequences of \time-consuming loops". Whenever aging requests are received by the coroutine, the rst loop is executed. 13
That loop has to modify the state of the script's objects according to the amount of virtual time which is to be jumped. At some point an instruction in the loop may decide that its action is terminated; that decision is usually taken according to the total amount of virtual time that has been \consumed" up to now by the loop. Control then goes to the next timeconsuming loop. After the last loop in the body of the function, the activity terminates, and the coroutine disappears together with its bindings and local variables. Between successive loops, the programmer is free to insert any sequence of C++ instructions; they will be executed at the appropriate time, but will be considered by the temporal system as having a virtual duration which is zero. C++ code can also be inserted before the rst time-consuming loop: that code will be executed when the function is activated, i.e. before it receives any aging request. Typically some initialisation instructions are executed at that moment. The syntax of a time-consuming loop in our implementation is of the form: CONSUME(var) TIMELEFT(value) Two macros enclose the instructions that implement the aging behaviour. The rst macro, CONSUME(), will transmit the value of the last jump in virtual time to a local variable in the coroutine. Then the instructions written by the programmer are supposed to use that value to modify the state of the objects in the activity's domain. The macro TIMELEFT() allows the programmer to tell to the temporal system which amount of virtual time has not been \consumed" by the loop. If the value is non-negative, which means that the loop received enough virtual time to ful ll its job, the loop terminates; the value passed to TIMELEFT() is automatically transmitted to the next loop. Otherwise control goes back to the previous CONSUME() macro. Let us look now at the implementation of our interpolate function, which contains one single time-consuming loop: Script interpolate(float &x, float a, float b, float duration) { float time = 0; float timeleft = -1; float jump;
14
CONSUME(jump);
// receive in variable "jump" the // amount of virtual time to skip
time = time + jump; if (time >= duration) { // check if the jump is too big timeleft = time - duration; time = duration; } x = (b - a) * (time / duration); // update state of "x" TIMELEFT(timeleft);
}
// if "timeleft" is negative, go // back to the last "CONSUME", // otherwise terminate the loop //
terminate the activity
The most important part of that code is the instruction that assigns a new value to x, according to the virtual time that has elapsed. It is not dicult to see how the same technique could use any other function than linear interpolation to control the evolution of x over time. Several instructions in the example are common for activities with a xed duration: the jumps in virtual time are summed up in a local variable, until their total equals or exceeds the duration. Since activities with a xed duration are frequently needed, we de ned a higher-level macro that makes them easier to write. Its syntax is: DURING(, ); and it is implemented in terms of the earlier macros: #define DURING(dur, C_INSTRUCTION) { double Duration = (dur); double Time = 0.; double Jump = 0.; double TimeLeft = -1.; CONSUME(Jump); if ((Time += Jump) >= Duration) {
15
\ \ \ \ \ \ \ \
Jump -= (TimeLeft= (Time - Duration)); Time = Duration; }; {C_INSTRUCTION} TIMELEFT(TimeLeft); }
\ \ \ \ \
Within the DURING statement, the programmer can use the variables Time,Jump and
Duration to implement the aging behaviour.
Our interpolate function is now much easier to write: Script interpolate(float &x, float a, float b, float dur) { DURING(dur, { x = (b - a) * (Time / Duration); } ); }
The code within time-consuming loops has to ful ll certain constraints if we want to avoid unpredictable results. In particular, it has to respect our continuity requirement, namely the fact that two executions of a loop with jumps t1 and t2 are identical to a single execution of the loop with jump (t1 + t2). It can be easily seen that this is true in the interpolate example. Unfortunately no automatic checking can be performed, because the compiler or interpreter does not have enough information about the semantics of the instructions in the loop. Hence, basic temporal expressions have to be written with care.
6 Some Examples In this section we shall present some examples of TEMPO programs. We hope to demonstrate with them that our approach leads to concise speci cations that clearly exhibit the global behaviour of a system. Unfortunately the outputs of these programs are dicult to render in a paper, since they are temporal behaviours. Hopefully some images extracted from an animation will allow the reader to get an idea about the motions speci ed in the program. 16
6.1 De ning an object for the animation library Our current animation library has a collection of prede ned objects like Human, Bird, Dog, Helicopter. The most complex is the Human object and would probably be the most interesting to describe; however a detailed presentation would be too lengthy in the scope of this paper, and we shall therefore present the simpler Bird object. The C++ interface for class Bird is: #include "graphObj3d.h" class Bird : public GraphObj { GraphObj *lwing, *rwing; float cur_angle; // wings rotation angle public: Bird(GraphObjNode *p = GraphObjRoot); float max_up_angle; // default = 40 float max_down_angle; // default = -60 float angle() {return cur_angle;} void set_angle(double); float flap_speed; // nb flaps per tick, default 0.1 SCRIPT flap(double = ETERNITY); };
Bird is a subclass from GraphObj, a class that implements common graphical motions like Translate or Rotate. It has two pointers to other graphical objects that will be bound to the wings of the bird. Another private variable, cur angle, stores the angle between the body and the wings. Two methods in the public interface permit to set and retrieve the value of that angle; they check that the value stays within the bounds of max up angle and max down angle, which are two public variables (i.e. they can be written from outside the bird environment). Another public variable controls the speed for
apping wings. The class has a constructor, a method that is automatically called when a new instance of Bird is created, and that performs several initialisations like loading the graphical commands representing the bird and creating the two wings objects. The only temporal method in the interface is ap. It takes a duration parameter which by default is ETERNITY (a constant bound to the largest oating-point value). It 17
is implemented with a basic temporal expression: SCRIPT Bird::flap(double d) { double a; DURING(d, { a = max_down_angle + (max_up_angle - max_down_angle) * (sin(PI2*flap_speed*Time) + 1) / 2; set_angle(a); } ); }
Using the DURING loop that was introduced earlier, the ap motion modi es the angle of the wings according to a sinusoidal function between the minimum and maximum values. At this point our example still looks rather low-level programming. We acknowledge the fact that programming library objects, especially if they use basic temporal expressions, is a complex task. But the interest of TEMPO shows up when we start using the temporal operators to combine library objects. This is where the idea of scripting comes in: whenever sets of similar applications have to be programmed, it becomes worthwhile to provide an environment in which not only some pre-packaged components are made available, but also some binding mechanisms are especially tuned for that class of applications. Next section tries to illustrate that approach.
6.2 An animation script We shall now design a small animation using the Bird object class. It creates three birds that y together, but with di erent speeds and behaviours. #include "bird.h" main() { Camera camera(cout);
// declare a camera that sends graphi-
18
// cal commands to standard output GraphObjNode flock; // abstract graphical object Bird bird1(&flock), bird2(&flock), bird3(&flock); // three birds, members of flock; bird1.flap_speed = 0.15; bird2.flap_speed = 0.8;
// nb flaps per unit of time
// initial positioning bird1.RotateZ(PI*2/3); bird1.Translate(Vect3d(0, 150, 0)); bird2.RotateZ(PI*6/5); bird2.Translate(Vect3d(0, 150, -20)); bird3.Translate(Vect3d(0, 150, 30)); camera.lookFrom.Translate(Vect3d(-400));
ScriptVar scene = bird1.flap() & bird2.flap() & bird3.flap() & flock.RotateZ(40, PI*6) & ( bird1.Translate(15, Vect3d(0, 0, 20)) >> bird1.Translate(25, Vect3d(0, 0, -20)) ) & ( 20 >> camera.lookFrom.Translate(15, Vect3d(-100, 0, 300)) ); camera.film(scene); }
The rst lines are declarations that create the various objects in the scene, including an instance of Camera. Graphical objects are put in a global hierarchy, where each object's position and orientation is relative to its parent. For example the wings of a bird move in the space of the bird's body, and the bird in turn moves in the space of the ock node. Unlike other nodes in that example, ock is not directly associated to a graphical rendering; it 19
Figure 3: Some images from the 3birds animation
20
merely acts as a structuring device, through which the three birds can be moved together. So we will actually see three birds rotating when the command ock.RotateZ is executed. After the declarations, a collection of commands are issued to bring the objects to appropriate initial positions. Then the description of the motion is stored in variable scene: the three birds start apping wings; at the same time the rst bird starts an upwards motion during 15 units of virtual time, followed by a downwards motion during 25 units; 20 units after the beginning of the scene, the camera starts to move away from the birds. Once the scene is de ned, the camera can start taking pictures of it. Film is a library method that iteratively prompts the user for an amount of time to jump, executes the jump and then generates graphical commands according to the new positions of the objects in the scene. If the programmer does not want the jumps in virtual time to be interactively selected by the user, the sampling can be directly controlled with instructions of the form: scene += ;
6.3 Example: a script to control a sound synthesizer Musical composition and synthesis is another area where multiple parameters have to be controlled in parallel, yet with a global notion of time. For example the evolution of a sound may be a combination of variations of pitch, amplitude, or spectrum. The FORMES object-oriented system developed at IRCAM1 [Coin87] provides an explicit notion of time, with mechanisms for building hierarchical temporal processes . Processes are linked through monitors, whose function is somewhat similar to our temporal operators (but they have nothing to do with the monitors presented in [Hoare74]). Local clocks associated with processes simulate time ow in the system. We give a short example to show how TEMPO programs could provide functionalities similar to the FORMES system, such as generating sound envelopes, for example. Only the amplitude parameter of sounds is considered in the example: class Sound{ 1
Institut de Recherche et Coordination Acoustique/Musique, Paris
21
double amplitude; ... public: ... }
/* other sound parameters (pitch, etc.) */
First a method to perform crescendo and decrescendo can be de ned by using the
interpolate function described earlier:
SCRIPT Sound::changeAmpl(double duration, double newAmpl) { return interpolate(amplitude, amplitude, newAmpl, duration); }
Using the sequential and delay operators, we can now de ne a method to generate envelopes for sounds. An envelope is a sequence of three stages: an attack, during which the sound augments up to a given amplitude, a sustain where the amplitude is stationary, and a decay to let the sound disappear: SCRIPT Sound::envelope(double attack, double sustain, double decay, double ampl) { return (changeAmpl(attack, ampl) >> sustain >> changeAmpl(decay, 0)); }
The rst three parameters are the duration for the three stages, and the last parameter is the amplitude at which the sound will be sustained. The operator ` ' indicates both sequential execution and delay; thus, the rst activation of the changeAmpl method is followed by a delay (sustain), followed again by a second activation of changeAmpl that goes through a decrescendo to perform the decay. Similarly, higher-level musical functions can be de ned by specifying sequential or parallel execution of several envelopes. Musical parameters like legato or staccato can be introduced by choosing appropriate delays between the notes, or by slightly overlapping successive envelopes. Temporal expressions controlling other parameters (pitch, timbre) can be executed concurrently. 22
7 Future Research TEMPO provides a exible environment for working with temporal speci cations. It has been successfully implemented and used on a network of Sun/3 workstations running the Unix2 operating system. To our knowledge, few other attempts have been made at providing a general framework for specifying temporal activities. An intense research activity is being developed on temporal logic and temporal reasoning, like the excellent work of [Allen83], but few actually tackle the problem of scheduling long-term activities in a program. Some systems have been developed for speci c areas, like FORMES in music, ASAS for animation [Rey82], or CINEMIRA, another animation environment that has a notion of animated types to control variations over time [Mag85]. TEMPO seems more general and more exible, however, despite several restrictions that we hope to relieve in future versions. Research will proceed in three main directions. First temporal expressions still look very much like programming, and could be made more user-friendly. A project for graphical rendering of temporal operators has been started, in order to allow the user to interactively build and modify an expression by manipulating boxes on the screen. We already have a prototype running, but it does not support all TEMPO operators. Figure 4 shows an example of our current prototype with the script `3birds.C'. A hierarchy of rectangles on the screen re ects the hierarchy of expressions in the script; when they can be deduced from the script, the durations are pictured by the widths of the rectangles. Rectangles can be moved and stretched, and temporal activities are delayed or scaled in time accordingly. The resulting con guration can then be downloaded into a le as a temporal expression. Another problem is that the TEMPO model says nothing about the order in which objects are updated during a jump in virtual time. Therefore it is unsafe to let objects communicate during jumps. For example if object x needs to know the position of y to update its state, it cannot know, when it receives the request to jump from virtual time t1 to t2, if y is still at t1 or already at t2. The current approach, that considers that all updates occur concurrently, can probably be re ned with some ordering constraints; it shall be quite dicult, however, to ensure that those constraints do not interfere with the semantics of temporal operators. A possible solution would be to use the TimeWarp mechanism of [Je 85], in which activities proceed independently on their local time lines, until an ordering con ict occurs: in that case the con icting activities go back in virtual time in order to restore the system into a previous coherent state. 2
UNIX is a trademark of ATT Bell Laboratories
23
Figure 4: Example screen of the visual script editor Finally we have to nd a way to check that domains of concurrent activities do not overlap, so that an object is not updated by two activities at the same time. A mechanism for getting exclusive control over an object has to be implemented, together with an algorithm for resolving con icts, like for example a queue of pending activities on an object.
Acknowledgements
We would like to thank Xavier Pintado and Gerti Kappel for their useful comments on an earlier draft of this paper, and Bruno Leemann and Alec Rimensberger for their work on the graphical interface.
A Proofs of continuity for temporal operators A.1 Sequential execution ((E1 E2)+= t1)+= t2 = ((E1+= t1) E2)+= t2 24
= ((E1+= t1)+= t2) E2 = (E1+= (t1 + t2)) E2 = (E1 E2)+= (t1 + t2)
A.2 Delayed execution (
((x ? t1) E )+= t2 if x t1 (E += (t1 ? x))+= t2 otherwise 8( > < ((x ? t1 ? t2) E )+= t2 if (x ? t1) t2 if t1 + t2 > x t1 = > E += (t2 ? (x ? t1)) : E += (t1 ? x + t2 ) otherwise ( ? (t1 + t2)) E if x (t1 + t2) = (Ex+= ((t1 + t2) ? x) otherwise = (x E )+= (t1 + t2)
((x E )+= t1)+= t2 =
A.3 Parallel execution ((E1&E2)+= t1)+= t2 = = = =
((E1+= t1)&(E2+= t1))+= t2 ((E1+= t1)+= t2)&((E2+= t1)+= t2) (E1+= (t1 + t2))&(E2+= (t1 + t2)) (E1&E2)+= (t1 + t2)
A.4 Repeated execution Since repeated execution is de ned recursively on sequential execution, and sequential execution has been proven to follow the continuity constraint, repeated execution necessarily follows also the constraint. 25
A.5 Speed modulation (E [f ]+= t1)+= t2 = = = = = = =
(E [[f00 ]]+= t1)+= t2 ((E += f00 (t1))[[ft01 ]]+= t2 ((E += f00 (t1))+= (ft01 (t2)))[[ft01+t2 ]] (E += (f00 (t1) + ft01 (t2)))[[ft01+t2 ]] (E += ((f (t1) ? f (0)) + (f (t1 + t2) ? f (t1))))[[ft01+t2 ]] (E += f00 (t1 + t2))[[ft01+t2 ]] E [f ]+= (t1 + t2)
References [Allen83] James F. Allen, \Maintaining Knowledge about Temporal Intervals", CACM, Vol. 26 No 11, p. 832 ss, 1983. [Andr83] Gregory R. Andrews and Fred B. Schneider, \Concepts and Notations for Concurrent Programming", ACM Computing Surveys, Vol. 15 No 1, 1983. [Bez87] Jean Bezivin, Some Experiments in Object-Oriented Simulation, OOPSLA'87 Conference Proceedings, Special Issue of SIGPLAN Notices, Vol 22 No 12, December 1987. [Coin87] Pierre Cointe, Jean-Pierre Briot, Bernard Serpette, \The Formes System: a Musical Application of Object-Oriented Concurrent Programming", in ObjectOriented Concurrent Programming, ed. by A. Yonezawa and M. Tokoro, MIT Press, 1987. [Cox86] B.J. Cox, \Object Oriented Programming { An Evolutionary Approach", Addison-Wesley, Reading, Mass., 1986. [Fium87] Eugene Fiume, Dennis Tsichritzis and Laurent Dami, \A Temporal Scripting Language for Object-Oriented Animation", in Proceedings of Eurographics 1987, Elsevier Science Publishers (North-Holland), Amsterdam. [Hoare74] C.A.R. Hoare, \Monitors: an Operating System Structuring Concept", CACM, Vol. 17, No 10, pp 549-557, Oct 1974. [Je 85] David R. Je erson, \Virtual Time", in ACM TOPLAS, Vol 7. No 3, July 1985. 26
[Kapp89] G. Kappel, J. Vitek, O. Nierstrasz, S. Gibbs, B. Junod, M. Stadelmann, D. Tsichritzis, \An Object-Based Visual Scripting Environment", in ObjectOriented Development, ed. D. Tsichritzis, Universite de Geneve, 1989. [Mag85] Nadia Magnenat-Thalmann and Daniel Thalmann, \Computer Animation: Theory and Practice", Springer-Verlag, 1985. [Meyer88] B. Meyer, \Object-oriented Software Construction", Prentice-Hall, 1988. [Misra86] Jayadev Misra, \Distributed Discrete-Event Simulation", in ACM Computing Surveys, Vol 18 No 1, March 1986. [Nier86] Oscar M. Nierstrasz, \What is the \Object" in Object-oriented Programming?", in Proceedings of the CERN School of Computing, Renesse, The Netherlands, 1986. [Rey82] Craig W. Reynolds, \Computer Animation with scripts and actors", in ACM SIGGRAPH 1982 Conference Proceedings (July 1982) [Rey87] Craig W. Reynolds, \Flocks, Herds, and Schools: A Distributed Behavioral Model", in ACM Computer Graphics, Vol. 21, No 4, July 1987. [Strous86] Bjarne Stroustrup, \The C++ Programming Language", Addison-Wesley, Reading, Massachusetts, 1986. [Weg87] Peter Wegner, \Dimensions of Object-Based Language Design", OOPSLA'87 Conference Proceedings, Special Issue of SIGPLAN Notices, Vol 22 No 12, December 1987.
27
