A Foundation for Programming Environments ... - ACM Digital Library

A Foundation for Programming Environments E. R. Gunsner

AT&T Bell Laboratories Murray Hill, New Jersey07974 ABSTRACT

Recent work on generating tools and techniques have significantly eased the design and construction of programming environments(PEs). Concurrently, advancesin workstation hardware have enhancedboth the potential capabilities and performanceof PEs. We argue that, to capitalize on these two developments,various deficienciesin traditional languagesand operating systemsmust be avoided. To this end, we have developedPegssup,a systemto serveas a software foundation for the construction and execution of PEs. It consists of a concurrent functional programming language, the run-time support for this language,and a core library of standard tools and utilities. We presenta discussionof Pegasus,its current state, and directions for future development. I. introduction We are developinga system, called Pegasus,to serve as a foundation for the construction, generation and specificationof interactive programming environments (PEs)‘. This project is motivated by two trends in the field of PEs: the new potential for highly interactive user interfaces as a result of improvementsin workstation hardware, and the ongoing researchinto models and generation tools to easethe construction of PEs. A state-of-the-art personalworkstation is sufficient to support highly interactive PEs with features such as: large collection of services,multiple views and contexts (both textual and graphical), mouse/menu-based input, and fast user response. In such an environment,it is important that execution of a user command in one view should not impair other aspectsof system response.For example, an editing operation that results in a lengthy attribute reevaluationshould not block user interaction in another context. Our experience with constructing window-baseduser interfaces has led us to conclude that concurrency is the natural model for supporting multiple views. An additional advantageof concurrency is the use of background processesfor bookkeepingtasks; this allows the idle time betweenuser actions to be usedconstructively.

t

Consultant.

I.

We use the term “Programming

Environment” to refer to the whole range of possible application environments.

Permissionto copy without fee all or part of this material is grantedprovided that the copies are not made or distributed for direct commercialadvantage, the ACM copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Association for Computing Machinery. To copy otherwise, or to republish, requires a fee and/ or specific permission. Q I986 ACM O-8979 I-2 I 2-g/86/00 I2/2 18 75C 218

While workstation hardware has improved significantly, workstation operating systems have not advanced much beyond the UNIX’ system and its clones, particularly in the areas of the user interface and interprocess communication’. WC believe that current workstation optrating systems do not provide an adequate foundation for the implementation of PEs. Our principal reason for rejecting them is that the concurrency needed to support multiple views and prompt interactive response cannot be provided by the standard system processes. There are several problems: process creation and management incur a high overhead, largely becauseeach processhas its own virtual address space; interprocess communication is often slow and cumbersome; the number of processesallowed to coexist is often limited; and the built-in process scheduling algorithm may not be suitable for the highly interactive nature of PEs. Because the various processesof a PE are cooperating and essentially friendly’, there is no need for the full security of UNIXtype processes. The other trend of interest is the use of formal models, particularly attribute grammars, as a basis for specifying and generating parts of PEs. This has been an active area of research for some timeIDRTBlIIRspr8411RT841[JF8511DR~SI[RMT861 . While this research has concentrated on the language-based editor aspects of PEs, generation tools can also be used for other aspects such as program executionfRK841 programming-in-the-large issues, and the user interface. A typical generation tool translates a “high-level” specification into a program in some target language (often C or Pascal). This program is compiled and linked with a core library and possibly some additional code supplied by the PE author. Although these specification languages tend to be declarative, there is usually a computational aspect; for example, the implementation of semantic functions in an attribute grammar. In a PE construction system that has more than one generation tool, it is obviously desirable that the specification languages of the various tools share as much notation and semantics as is practical. Particularly, the notation and semantics of computation and data types should be identical. Thus, we believe that a common base language for the generation tools of a PE construction system is necessary. The semantics of such a base language and of the derived specification languages are, as a rule, highlevel and applicative (e.g., [RT841), Low-level issues such as storage allocation are handled by the underlying run-time support for the language. This support is provided as part of a core library. Because the expressivenessof a specification language is often limited, many systems provide a way for a PE author to supplement her specifications with code in a system language such as C. In writing such code, the author must be cognizant of low-level details such as storage management conventions. Thus, many of the advantages of using a high-level specification based tool are lost. The solution to this is to extend the common base language to provide greater expressiveness;in effect, to be a high-level system language. Furthermore, we believe that this language should also be used as the target language for the generation tools, thus providing a unified semantic model for all aspects of the generated PEs. Such a language is also 2. Registered trademark of AT&T. 3, We ignore here such esoteric systems as SmaUtalk-80 (Trademark

of Xerox Corporation).

4. There are occasional casea when security is necessary. such as during the execution of a piece of a PE user’s code for debugging purposes.

suitable for ad hoc implementationsof PEs. The lines of thought presentedabove have motivated our design of Pegassuas a foundation for PE construction. Pegasusconsistsof a concurrent functional programming language,which servesas a common base language for specificationtools and as a target language, and the run-time support for this language. In addition, there is a library of default implementationsof useful system utilities, such as a windowing system.Pegasusprovidesa high-level semantic framework in which the builder of PE construction tools can concentrateon the problemsof PE construction insteadof the low-level problemsof systemimplementation.

II. Requirements for a PE Foundation In addition to the fundamental design goals elucidated above, there are a number of ancillary design principles and issuesthat are important to the practicality of a PE foundation. Our thinking on these issues affects both the design and the implementationof Pegasus. II.1 Eflkiency

Efficiency is a very important issuein the construction of practical PEs. Many researchershave explored we, on the efficient algorithms for handling incremental updating causedby editing operationsIDRT8lltRIUTW, other hand, are primarily concernedwith insuring the efficiencyof our foundation. The principal issuesare: i)

Concurrerrcy.In a system with many cooperating concurrent processes,it is necessaryto have a low-overhead context switch and fast interprocess communication. This suggests the use of “lightweight” processes.

ii)

Memory monugement. The allocation and deletion of objects constitutesa significant run-time cost for PES. Thus the design of the memory managementsystem can have a decided effect on the overall performance of the PEs. Fortunately, this has been an area of extensiveresearch (e.g., lBaker781, Hh0ks841, LH831, and IUngar8411.

iii) Compiler optimization. The base languageshould be compiled rather than interpreted. This allows

the use of modern optimization techniques,which will improve systemperformancein all areas. II.2 Portabiity An important role of a PE foundation is to provide a system independentmodel for system dependent features. The hardware and systemdifferencesbetween machinesshouid be hidden or minimized by the PE foundation, enchancing the portability of PEs built on that foundation. To fulfill this goal, the foundation itself must be designed to be portable over a wide collection of workstations, reflecting the common semanticsamong them. II.3 Other Design Issues

There are a few other issuesthat need to be discussed: i)

No built-in assumptions. Becausewe cannot anticipate the full range of future developmentsin

the field of PEs, it is important that we minimize built-in assumptionsabout the tools and PEs based on Pegasus. In those caseswhere such assumptionsare necessary,we need to provide an

220

escapemechanism, ii)

ModuZarify. PEs are large and complex software systems. As with any such system, structuring and organizing the components is a difficult task. Thus, it is important that our language support modularity and programming in the large.

iii) Concurrency vs. cofoulines. In implementing concurrency on a single processor, an important design issue is how to multiplex the CPU. We reject explicit coroutine switches as placing an unacceptable burden on the PE author. This leaves us with three possible techniques: coroutine switches generated by the compiler at suitable points; embedded coroutine switches built into language features such as the interprocess communication mechanisms; and pre-emptive scheduling. We reject the first of these for two reasons. It is unclear how one could insure a reasonably fair distribution of CPU access without incurring the cost of excessive numbers of coroutine switches. The other problem is that these coroutine switches cannot be inserted into code compiled by other compilers. We also reject embedded coroutine switches becausethere is no guarantee that a process will execute a coroutine switch in any finite amount of time. This plays havoc with any real-time sensitive processes. So we are left with pre-emptivc scheduling. iv) Process scheduling. To insure interactive response,the asynchronous context switches need to occur frequently (at least 25 per second). Process scheduling should be designed to guarantee real-time sensitive processesquick accessto the CPU. v)

Hardware ossutnprions. We make certain assumptions about the hardware of the target system. We see the machine as a personal workstation with a fast processor (2 1 MIPS), a large physical memory (3 4M bytes), and a large high resolution bitmap. Some form of pointing device, such as a mouse, must be available. These requirements are easily met by a wide variety of commercial systems.

III. An Overviewof Pegasus In this section, we discuss, in some detail, the design and organization of Pegasus. There are three logical components that compose the Pegasus system: the Pegasus Meta-Language (PML), a concurrent functional programming language; the Pegasus Virtual Machine (PVM), which supplies the run-time support for PML, and the Pegasus core library. We discuss each of these components below, in addition to the relation between Pegasusand PE construction tools. III.1 PML The original motivation for PML was the need for a common base language for the various PE construction tools. As Pegasushas evolved, PML has come to occupy the role of the unifying semantic model for the whole system. PML is a general purpose concurrent functional programming language derived from MLIMihe’ss’, We chose ML as the basis of our language design for several reasons: ML’s polymorphic type system, first class functions, and expression oriented syntax and semantics make it a suitable notation for expressing semantic definitions (for example, see IWattg51); because ML is statically typed it is possible to

221

generate fairly good code for itic8~e”i841;and ML is a high-level language providing a powerful programming notation. PML also has a module mechanism based on the one developed for MLt”8cQ85’. PML programs are organized as collections of module8 which can be separately compiled. Data abstraction is supported by separating module interfaces from implementations. In addition to providing the vehicle for organizing the components of a Pegasus based FE, the module mechanism provides a clean way for PE authors to make modifications to the core library. We elaborate on this below. Although there are some minor policy and syntactic difference8 between PML and ML, the major difference is that PML is concurrent. New processesmay be created dynamically. Processescommunicate by typed channels, which may also be created dynamically. III.2 PVM Because PML is a “high-level” language, a fairly elaborate run-time support environment is necessary. The PVM provides this support as well as an interface to the bitmap display, mouse, and keyboard. It is organized into three levels. Level-O servesthe role of an operating system nucleus. It is invoked by service requests, run-time faults, or timer interrupts. When invoked, it saves the state of the running process, services the request/fault, checks for mouse and keyboard input, and then starts a new process. All processesrun in the same address space. Processes can request a variety of services including: process creation/termination, change of priority, establishment of a fault handler, processsynchronization, and mouse and keyboard input. Run-time faults such as dividing by zero are passed to the process’s fault handler. Process scheduling is done via a priority queue of ready processes. Scheduling is based on the last time a process ran and its base priority (settable by a service request). The algorithm is fair in the sense that any process that is ready will eventually get to run. While level-0 provides low-level primitives to support concurrency, level-l builds on this to provide the PML run-time environment. Level-l contains a collection of abstract data types and built-in functions. The more significant of these include: i)

Channels - Level-l implements a generic channel data type for interprocess communication as well as operations to support a non-deterministic “select” operation on multiple channels.

ii)

Memory management - The primitives for allocating new objects and managing the heap are an important part of level-l. The current &ses8,prganizcs the heap into semi-space&&“‘*’ with objects accessed via forwarding pointers Storage reclamation is done by a separate garbage collection process,described below.

iii) Graphical data types - Low-level bitmap graphical data types and operations are implemented in level-l. These include such objects as points, fonts, and bitmaps, and such operations as bitblt, line drawing, and polygon filling. This provides a standard view of bitmap graphics that is independent of the underlying system’s conventions. In addition, other predefined PML data types such as String and BitVector are implemented by level-l.

222

Level-2 is a collection of system processesthat provide support for the PVM; these processesare hidden from the user code’s view. Our current design has four level-2 processes: i)

The root process is responsible for initializing level-l, creating the other system processes,and starting the user’s code.

ii)

The input monitoring process translates the input events provided by level-0 into a stream of messageson the predefined input data channel.

iii)

The time-delay processis used to implement a sleep mechanism.

id

The garbage eolleetor (GC) process is responsible for reclaiming discarded objects. As noted above, we organize the heap in a manner similar to Brooks’ version of Baker’s real-time semi-space mcth~~tbkCr78~hOOk184~ Unlike these real-time methods, however, we do not place any garbage collection overhead on object allocation; instead, the GC process concurrently scavengesobjects. By running the GC process at a low-priority (i.e., in the background) we can shift garbage collection activity to otherwise idle periods. Fortunately our early results suggest that highly interactive programs, which are the most sensitive to real-time performance, have many idle periods.

III.3 The PegasusCore Library The eorc library is a collection of PML modules providing useful processes,data types and services, such as a math library or window management. Because our choice of semantics for these utilities may differ with the philosophy of some PE author or may, in fact, be insufficient for a given PE, it is desirable that logical units of the eorc bc replaceable. Since the core is composed of modules with clear separation between interfaces and implementations, it is feasible to make such replacements. All that is required is that the new implementation satisfy the interface specification. This is in keeping with our design goal of no built-in assumptions. III.4 Programming Environment Construction Tools We envision a collection of tools targctcd to Pegasus to facilitate the construction of PEs. Attribute grammar based tools for handling static semantics arc obvious examples, but WCalso plan to explore tools for constructing user interfaecs and for organizing the relationships and communication between the different components of a PE. Pegasushas been designed to avoid any assumptions about the models that might be used as bases for such tools. We do assume that PML (or a subset) will bc used as a basis for the specification language in most eases,and that PML will bc the target language. Eventually, we plan to provide direct accessto the PML compiler’s type checker and internal representation so that PE construction tools will be able to generate a type chcckcd intermediate form instead of text. The module facility of PML will bc used to structure and organize the various pieces produced by the construction tools.

22:!

IV.

Implementation

We have been building a prototype implementation of Pegasuson the Apollo’ workstation. As of this writing (August 1986), we have implemented the PVM on the Apollo. Except for a small amount of assembler code in the process scheduler the system is written entirely in C+ +tSuoursC1We have completed a preliminary design of PMLIRGNW1and have almost finished a prototype PML compiler. The compiler design isolates all target machine dependencies to the final code generation phase, this should ease the porting of the compiler to new machines. There are some interesting technical details to our PVM implementation. A key problem was how to implement the interrupts needed for prc-cmptivc scheduling. WC organized our system as three Apollo processes. One of these, called the PVM proecss, executes PVM and PE code. The timer process sends a UNIX signal to the PVM process at fixed intervals (the default is 35 milliseconds). This signal gets caught as a fault by the level-0 proecss scheduler, which suspendsthe current PVM process and starts a new one. The third Apollo process is responsible for starting and stopping the system and for doing postmortems. On top of this, we have built an experimental system in C + +. This system is being used to investigate issues such as the design of a window package and system performance. The results of these investigations will be used to guide us in our design and implementation of the eorc library in PML, and to improve system performance. Our experimintal system includes a display system supporting such graphical objects as windows, menus, scroll bars, icons, and other analogue controls. The architecture of the display system is based on the server-client model, similar to that used in the Andrew window systembbrris861 , with windows implemented as virtual bitmaps using the layers model of Pikettikc83’and all graphic operations performed by a server process called the display manager (DM). Graphical operations arc implemented by an RPC-like protocol over channels, with the actual channel operations hidden at the user level. The DM process also monitors the input data channel, mentioned above, tracking the mouse and directing input to the appropriate client processes. Preliminary performance tests show that the overhead of the prc-cmptive scheduling consumes about 8.1% of the CPU time. Of this, about 3.9% is used by the timer process and the balance is used by level-O. The base overhead of a level-0 service rquest is 20-25 times the cost of a procedure ca116.This overhead is tolerable for a prototype, and we find that highly interactive features such as mouse tracking work smoothly.

V. Portability 5. Product of Apollo Computer Inc.. Chelmsford, Mass. 6. These measurements were made on an Apollo DN560 with a l2MHz affect the results, they should be taken with a grain of salt.

224

MC68020

processor.

Because the masurement

techniques

An important goal of the Pegasus effort is to make the porting of PEs routine. Thus, an important question is the portability of Pegasns itself. Our current list of target systems all have the same processor family (MC680007). Therefore, the problem of retargeting the PML compiler will be minor in the near term. These systems do differ, however, in the kinds of graphic primitives provided, in their models of mouse and keyboard input, and in their run-time environments. It is the role of the PVM to provide a common interface to these systems. Thus, porting Pegasusis principally a problem of reimplementing the PVM. We spent 2-3 person-months implementing our prototype PVM. Since many of the internal structures are machine independent, we expect to be able to port the PVM in about one person-month.

VI. Future Work In addition to continuing work on our prototype, there are a number of areas and issues we plan to explore: i)

Distributed environmenrs. Pegasus in its current state employs a single user, single workstation model. We recognize the need to extend this to a multiple user, multiple workstation model supporting the interactive sharing of data and objects residing in a distributed environment.

ii)

Operuting system hooks. The technique we use to implement pre-emptive scheduling has several drawbacks. Not the least of these is that two heavy-weight context switches are required for each timer interrupt. We plan to explore modifications to the underlying operating system to directly support pre-emptive scheduling of lightweight processes.

iii) Garbage collection performance. Memory management places a significant overhead on PEs. We plan to conduct some empirical tests to compare different strategies for memory management. Of particular interest are generation based methods of heap organizationtLH*311Uns’r*41. iv) Color bitmap graphics. We see color playing an increasingly important role in user interfaces. Color capabilities need to be integrated into the Pagaauagraphical model in a fashion consistent with features (such as texture filling) of the monochrome model. Id

Persistent objects. Currently, we rely on the underlying file system for permanent storage. We would like to introduce a notion of persistenr objecrs as part of the memory management system.

vi) Incremental loading. We plan to add a loader to Pegaans,possibly as a part of the PVM model. This will allow us to separate the PVM from the PE load modules, and will allow the exploration of dynamic reconfiguration of PEs. v)

Languuge design issues. Although the design of PML has solidified for the time being, there are some areas of concern. In particular, the concurrency mechanisms of PML are fairly low-level. Once we have a better understanding of the requirements of our system we plan to rc-examine this

7. Product of Motorola

Inc.

225

aspect of the language design. And, of course, work on PE design and implementation.

VII. Related Work Most PE research can be divided into work on specific environments and work on PE generation tools and models. Pegasus does not fit into either of these categories, although it has certainly been affected by work from both. The importance and utility of multiple views has been demonstrated by PECAN’Re’U841 Pegasus can support the paradigm of PECAN, but can also be used to explore other paradigms. The Syntbezizer Generrtor’RT841has a specification language that unifies syntactic and semantic domains of attribute grammars. While this approach has merit with respect to defining the semantics of language-based editors, it does not have the power or flexibility for dealing with the other aspects of PEs. Cedrr~““n85JtszH851 is a PE that shares several features with Pegasus. It also is structured around a concurrent programming language, with lightweight processes,dynamic storage management, and the use of modules/interfaces for structuring system components.

VIII.

Conclusion

We have argued that conventional workstation operating systems and system languages do not provide an adequate foundation for PEs. We have described a system, Pegasus, that provides a machineindependent semantic framework that serves as a foundation tailored for PEs. Pegasus does not impose any particular philosophy, but instead provides a basis for the exploration of different PE tools, models, and applications. We have built a prototype that demonstrates that the model provided by Pegasus, and in particular the concurrency features, can be implemented efficiently.

References (Baker781 Baker. H.G. “List Processingin Real Time on a Serial Computer,” Communicofionr of the ACM. V. 21, Nr. 4, April 1978, pp. 280-294. [Brooks841 Brooks, R.A. “Trading Data Spacefor ReducedTime and CodeSpacein Real Time GarbageCollection on Stock Hardware,” Conference Record of the 1984 ACM Symposium on LISP and Functional Programming, Aug. 6-8. 1984, pp. 256-262. [Cardclli841 Cardelli, L. “Compiling a functional language,” Conference Record of the 1984 ACM Symposium on Lisp and Functional Programming, August 6-8, 1984,pp. 208-217.

[DRTI 11

Demers,A., Reps,T., Teitelbaum,T. “Incremental Evaluation for Attribute Grammars with Applications to Syntax-directed Editors,” Conference Record of the 8th ACM Symposium on Principles of Programming Languages, January 26-28, 1981, pp. 105-116.

[DRZ851

Demcrs, A., Rogers, A., Zadeck. F.K. “Attribute Propagation by Message Passing,” Proceedings of the ACM SIGPLAN 85 Symposium on Language Issues in Programming Environments, June 25-28, 1985,

pp. 43-59.

[Don851

Donahue,J. “Integration Methods in Cedar,” Proceedings of the ACM SIGPLAN 85 Symposium on Language Issues in Programming Environments, June 25.28, 1985,pp. 245-251.

[JF85l

Johnson, G.F., Fischer, C.N. “A Mcta-Language and System for Nonlocal Incremental Attribute Evaluation in Language-Based Editors,” Conference Record of the 12th ACM Symposium on Principles

226

o/Programming Languuges. January 14-i6. 1985, pp. 141-151. 1~~831

Lieberman, H.. Hewitt, C. “A Real-time Garbage Collector Based on the Lifetime of Objects,” Communications of the ACM, V. 26, Nr. 6, June 1983, pp. 419-429.

(MacQ851

MacQuccn. D. “Modules for Standard ML,” AT&T Bell Laboratories Memorandum, 1985. An earlier version of this appeared in the Conference Record of the 1984 ACM Symposium on LISP and Functional Programming, Aug. 6-8, 1984.

[Milner851

Milner, R. “The Standard ML Core Language,” Polymorphism, V. 2, Nr. 2, October, 1985, pp. l-28. An earlier version of this appeared as “A Proposal for Standard ML,” Conference Record of the 1984 ACM Symposium on Lisp and Functional Programming, August 6-8, 1984.

[Morris861

Morris, J.H., et al. “Andrew: A Distributed Personal Computing Environment,” Communications of the ACM, V. 29, Nr. 3, March 1986, pp. 184-201.

[Pike831

Pike, R. “Graphics in Overlapping Bitmap Layers,” ACM Transactions on Graphics, V. 2. Nr. 2, 1983. pp. 135-160.

[R&s841

Reiss, S. “Graphical Program Development with PECAN Program Development Systems,” Proceedings of the ACM SIGSOFTXSIGPLAN Software Symposium on Practical Software Development Environments, April 23-25, 1984, pp. 30-41.

[Reps841

Reps, T. Generating

[RGN861

Reppy, J.H., Gansncr, E.R., North, S.C. “The Pegasus Mcta-language - Preliminary Definition,” Unpublished internal Memorandum, 1986.

tRK841

Reppy, J.H. and Kintala, C.M.R. “Generating Execution Facilities for Integrated Programming Environments,” AT&T Bell Laboratories Memorandum, 1984.

[RMT86]

Reps, T., Marceau, C., and Teitelbaum, T. “Remote Attribute Updating for Language-Based Editors,” Conference Record of the 13th ACM Symposium on Principles of Programming Languages, January 1315, 1986, pp. 1-13.

[RT84)

Reps, T., Teitelbaum, T. “The Synthesizer Generator,” Proceedings of the ACM SIGSOFT/SIGPLAN Software Engineering Symposium on Practical Software Development Environments, April 23-25, 1984, pp. 42-48.

[StrousBLI

Stroustrup, B. The C+ + Programming Language, Addison-Wesley, Reading, Mass., 1986.

[SZHSSI

Swinchart, D.C., Zcllwcger, P.T., Hagmann, R.B. “The Structure of Cedar”, Proceedings of the ACM SIGPLAN 85 Symposium on Language Issues in Programming Environments, June 25-28, 1985. pp.230244.

(Ungar841

Ungar, D. “Generation Scavenging: A Non-Disruptive High Performance Storage Reclamation Algorithm,” Proceedings of the ACM SIGSOFT/SIGPLAN Software Engineering Symposium on Practical Software Development Environments, April 23-25, 1984, pp. 157-167.

(Watt861

Watt, D.A. “Executable Semantic Descriptions,” Software January 1986. pp. 13-43.

Language-based Environments, The MIT Press,Cambridge, Mass., 1984.

227

- Practice and Experience,” V. 16, Nr. 1,