1 Macintosh, MacOS and A/UX are registered trademarks of Apple Computers Inc. ... the Apple Desktop Bus (ADB), which handles communications with the ...
ChorusToolbox : MacOS running on top of Chorus* Christian Bac, Hong Quang Nguyen, Institut National des Télécommunications, 9 rue Charles Fourier, 91000 Evry, France.
Abstract This paper addresses the problem of cohabitation and cross-usage between a micro kernel and an event driven operating system. The aim of this work was to use the graphical power and the musical capabilities of the Macintosh hardware and software directly from Chorus active entities (called actors) with the minimum work done in the field of devices' drivers and hardware interfaces. To achieve it, we have reused the Macintosh Operating System1, and encapsulated it in a subsystem on top of the Chorus2 micro kernel. The subsystem allows Chorus actors to use the MacOS operating system interface. The resulting subsystem is composed of two actors: a Supervisor actor and a Server that executes at the user level. The supervisor actor handles MacOS system calls and interrupts. For each of these interrupts or system calls, it sends a message to the server. The server is a multiplexed actor with two specialized threads handling the interrupt messages and one thread waiting for messages describing system calls. This thread creates a new one for each system call, so there are as many threads as there are uncompleted system calls. The memory mapping of the server allows it direct access to the hardware and recreates, as far as it is possible, the MacOS environment. This paper describes how the subsystem works, how the application and the server communicate, what choices we have made during the design and what lessons we have learned during the implementation.
Keywords: subsystem, micro kernel, co operation, cohabitation, reuse. 1. Introduction The technology of distributed systems using micro kernels [1] [2] is accepted as the basis to construct new operating systems called subsystems. Many of these subsystems emulate the UNIX3 [3] [4] operating system but others emulate different systems like DOS and MacOS [5]. This work takes a new approach which uses the binary part of an existing system (MacOS) and encapsulates it in a subsystem over a micro kernel (Chorus). We previously have made the Chorus micro kernel and MacOS cohabit and cooperate [6]. This previous work on cooperation and studies of the AU/X Toolbox[11], which allows applications running under AU/X to access to MacOS ROM, lead us to build a MacOS subsystem over Chorus. This subsystem uses the ROM and RAM parts of the MacOS system with very little changes. It allows Chorus actors to use the MacOS system calls and the MacOS programming technique, known as events programming, and to deal with the Macintosh graphical display.
* In Proceeding of International Conference SUUG'94: Open Systems: Solution for Open World, April
1994, Moscow, Russia. 1 Macintosh, MacOS and A/UX are registered trademarks of Apple Computers Inc. 2 Chorus is a registered trademark of Chorus Systèmes 3 UNIX is a registered trademark of AT&T USL. 42
Today, the subsystem has been built and we are testing its functionalities and responsiveness with Chorus actors using MacOS system calls. We have not yet tried to design an interface allowing MacOS applications to run over the subsystem. We are supporting source portability.
2. Macintosh and MacOS This section describes briefly the Macintosh hardware, the Macintosh software and investigates the interfaces between operating system software and the application.
2.1. Macintosh hardware As shown in Figure 1 below, the Macintosh II CX hardware [9] that was used, is composed of: – a Motorola MC68030 microprocessor with an integrated Paged Memory Management Unit equivalent to the MC68851, – and many added chips of which the following are of interest for this paper: • the Apple Desktop Bus (ADB), which handles communications with the keyboard and the mouse, • two Versatile Interface Adapters (VIA), based on the Rockwell 6522 chip, which support the ADB, the Real Time Clock, timers and other I/O devices,
NUBUS BIU
BIU Slots Interrupts
RAM 8M MC68030
ROM 256K ints
ADB RTC
VIA1 MC68881
vbl
VIA2
Slots
SCSI HD
SCC IWM FD
ASC Figure 1 - Macintosh II hardware
2.2. Overview of MacOS The Macintosh's user interface is supported by the routines available in the User Interface Toolbox. These routines are logically divided into groups usually known as managers or packages. Applications call these routines to implement the user interface and keep it consistent across all applications. The User Interface Toolbox contains the higher-level software routines that implement the interface between the operating system and user applications. The operating system contains the lower-level software routines that directly 43
control the hardware. Figure 2 shows the relationship between application programs, the User Interface Toolbox, the operating system, and the Macintosh hardware [7].
A MACINTOSH APPLICATION PROGRAM
USER INTERFACE TOOLBOX (In ROM) Resource Manager QuickDraw Font Manager Toolbox Event Manager Window Manager Control Manager Menu Manager
OTHER HIGH LEVEL SOFTWARE (In RAM)
TextEdit Dialog Manager Desk Manager Scrap Manager Toolbox Utilities Package Manager
OPERATING SYSTEM (in ROM) Memory Manager Segment Loader Operating System Event Manager File Manager Device Manager Disk Driver Sound Driver ROM Serial Driver Vertical Retrace Manager System Error Handler Operating System Utilities
Binary-Decimal Conversion Package International Utilities Package Standard File Package
OTHER LOW LEVEL SOFTWARE (in RAM) RAM Serial Driver Printing Manager Printer Driver AppleTalk Manager Disk Initialization Package Floating-Point Arithmetic Package Transcendental Functions Package
MACINTOSH HARDWARE Figure 2 - Overview of the Macintosh Software
2.3. System calls in MacOS The Toolbox and Operating System reside mainly in ROM. However, to allow flexibility and future development, application code must be kept free of any specific ROM addresses. For this reason all references to Toolbox or Operating System routines are made indirectly through the trap dispatcher which looks up the addresses of the routines in the trap dispatch tables in RAM. There are two dispatch tables, one for the Toolbox which is called the ToolTable and one for the Operating System which is called the OSTable. The trap dispatch mechanism allows the routines themselves to be moved to different locations in ROM, or to be replaced altogether by RAM-based routines, without disturbing the operation of programs that depend on them. Calls to the Toolbox and Operating System are implemented by means of the MC68000's 1010 emulator trap. In the MC680004' instruction set, instruction words that begin with binary 1010 (hexadecimal 0xA, 4 And newer microprocessors of the family.
44
the so-called A-line instructions) don't correspond to any valid machine language instruction and are known as unimplemented instructions. Figures 3 and 4 show the structure of trap words for the calls to the Toolbox and Operating System.
15 14 13 12 11 10 9 1
0
1
0
8
0
1
trap number reserved auto-pop bit
Figure 3 - Macintosh Toolbox trap word
15 14 13 12 11 10 9 1
0
1
0
0
8
7
flags
0 trap number keep A0 bit
Figure 4 - Macintosh Operating System trap word Each user level call to a Toolbox or to the Operating System is translated to a trap word. The routine is specified by a the trap number. So there can be 1024 different calls to the Toolbox and 256 different calls to the Operating System.
3. Chorus A Chorus System is composed of a small-sized Nucleus and of possibly several System Servers that cooperate in the context of subsystems to provide a coherent set of services and user interface. The interfaces [10] to the subsystem and to the Nucleus are described in Figure 5.
Subsystem Application
Subsystem Interface Subsystem Actor
Subsystem Actor
Subsystem Actor
Kernel Interface Subsystem Actor
CHORUS Kernel
User Address Space
System Address Space
Figure 5 -Chorus Subsystem Interfaces A Chorus domain consists of a set of Chorus sites. Each site hosts the Chorus Kernel and some actors. A detailed description of the Chorus system and of the Chorus/MiX UNIX subsystem can be found in [1].
3.1. Basic Abstractions The Chorus kernel provides the following basic abstractions (see Figure 6): 45
•
• • •
The actor defines an address space that can be either in user space or in supervisor space. In the latter case, the actor has access to the privileged execution mode of the hardware. User actors have a protected address space. One or more threads can run simultaneously within the same actor. They can communicate using the memory space they share. Threads from different actors communicate through the Chorus IPC that enables them to exchange messages through ports named by global unique identifiers. The Chorus IPC provides the ability for one thread to communicate transparently with another thread, regardless of the nature or location of the two threads.
Site Messages Threads
Port
Actor
Chorus Kernel
Chorus Kernel
Figure 6 -Chorus Basic Abstractions
3.3. Kernel Services to the Subsystem The Chorus Kernel allows user or system threads to take control of exceptional events within an actor. For supervisor actors, the kernel provides facilities to connect hardware interrupts to interrupt handlers within the actors. When an interrupt occurs, the kernel sequentially calls a prioritized sequence of user routines associated with a given interrupt. Any of the individual routines may initiate a break in the sequence, if necessary. The supervisor supports a similar facility, through which actor routines may be associated with hardware traps or exceptions. This facility allows subsystem managers to provide efficient and protected subsystem interfaces.
4. Toolbox Subsystem Design The toolbox subsystem reuses the MacOS software providing a Macintosh interface to Chorus actors.
4.1 Chorus Toolbox design As shown in Figure 7, the Chorus Toolbox consists of two actors. The supervisor actor which is called ChorusTbxSup (later called Supervisor) handles A-line traps and interrupts. It provides the MacOS interface to user level actors. The server actor which is called ChorusTbxServer (later called Server) runs at the user level. It does most of the work, by calling the MacOS routines in RAM or in ROM.
46
ChorusTbx Server
Applications
User Mode
Supervisor Mode Kernel Interface
Subsystem Interface
Chorus traps A-line traps
CHORUS Kernel
ChorusTbxSup interrupts
IPC
Figure 7 - ChorusToolbox subsystem The Supervisor and the Server actors communicate by means of the Chorus IPC. Exceptions resulting from A-line traps in the application or in the Server are caught by the Supervisor. The Supervisor builds a message and uses the Chorus IPC to send it to the Server. There is an initial negotiation between the Supervisor and the application. This negotiation is used to map the application memory space into the server address space allowing a greater flexibility to exchange parameters and results between the application and the server.
4.2. Supervisor Design This actor executes at the supervisor level. It handles the low level part of the A-line traps and the interrupts. It contains the following threads: -
-
The main thread initializes the subsystem when it starts. It connects exceptions handlers to the A-line traps and to other exceptions handled by the subsystem. It also connects interrupt handlers to the interrupts caused by VIA1 and VIA2. It then waits for an application to start a MacOS session. When the application asks for MacOS operation, the main thread sets the server memory mapping so that the server can use the memory segments of the application directly. The other thread is called TbxSupIt: it relays the interrupt handler's messages to the server. In fact, to avoid deadlocks, the interrupt handlers are not allowed to use the Chorus IPC. They can just use a restricted form of IPC named mini-messages. Mini-messages are allocated in the actor memory and can be sent to a thread in the same actor. So the interrupt handlers send mini-messages to the TbxSupIt thread and this thread sends a message to the Server to service the interrupt.
4.3 Server Design This actor executes at the user level. It handles any request concerning the Toolbox or the Macintosh Operating System. It also encapsulates the MacOS interrupt handlers into separated threads. It contains the following threads: -
-
One thread named TbxServerInit: it executes once when the Server starts its execution. It creates the communication ports, sends identifiers to the Supervisor actor allowing it to modify the Server address space and to send messages to the Server ports. It then launches the other threads (described below), and finally ends allowing them to execute. The TbxStdAline thread waits for messages on a specific port named AlinePort. It performs the same work as the MacOS trap dispatcher, i.e. analyzes the trap numbers and then launches a new thread at the address of the MacOS routine. As will be explained later, this new thread executes on the 47
-
-
application stack avoiding any copy of parameters and results. When the routine ends, the TbxStdAline sends results back to the supervisor which resumes the initiating thread. The TbxStdIt thread waits for messages describing interrupts on a specific port named ItPort. It selects the interrupt handler corresponding to the interrupt number and awakens it. The handler analyzes the interrupt context and if the interrupt service is potentially long running, it creates a thread to handle the interrupt. The TbxVBL, TbxADB and TbxSlot threads are created by TbxStdIt. They respectively service the VBL5 , ADB and the slot interrupts, by calling the VBL Manager, ADB Manager or the Slot Manager, and they may use many A-line traps.
5. ChorusToolbox Implementation This section describes in greater details how the subsystem uses the MacOS system to service the A-line traps from the applications or from the server.
5.1. Physical memory In order to use the RAM part of the MacOS system, we kept the memory sharing that was used in our previous work on cohabitation [6]. The physical memory is divided in three areas (see Figure 8 below): – the memory reserved for the MacOS system (composed of two physical parts), – the memory reserved for the Chorus system (where the kernel and actors are placed), – and a third area that allows a correct alignment of Chorus memory space, and that we call "No Man's Land".
0x7FFFFF 0x630000
MacOS
Chorus 0x080000 0x050000
No Man's land
MacOS
0x000000 Figure 8 - Sharing of physical memory space
5.2. Server address space The Server needs to access the physical memory initialized by MacOS. Its memory space begins in the first page at address 0. This page contains global variables, the switch table used by the dispatcher, etc. It also needs access to: • • •
the ROM address space to use the ROM part of MacOS, the I/O space to control the internal disc, the keyboard and mouse, and the sound chip, and the extension bus (the NuBus).
The Server memory mapping, as shown in Figure 8, is done by the Supervisor because only an actor running in supervisor mode is able to map physical memory space into the virtual address space in an actor. 5 The Vertical Blank Line interrupt occurs every 1/60ths of a second. It is used to trigger actions in
MacOS. 48
Physical Addresses
??
Chorus Kernel
??
ChorusTbxSup
??
Application
Virtual Addresses 0xffff ffff System 0x0900 0000 Space 0x0800 0000
0x0100 0000 I/O Space
0x50f0 0000 0xf090 0000 0x4000 0000 0x0063 0000
?? ?? 0x0000 0000
0x00f0 0000
NuBus (video Board) ROM (256 KO) MacOS
ChorusTbxServer Application Stack
0x0090 0000 0x0080 0000 0x0063 0000
User Space
0x0009 0000 0x0008 0000
MacOS (Globals) 0x0000 0000 Figure 9 - Virtual address space in Chorus Toolbox
When an application connects to the Supervisor asking for MacOS services, the Supervisor maps the application address space into the Server address space. This allows the Server to use parameters on the application stack and avoids copying parameters and results. This mapping also solves the problem of using pointers in system calls. In Mac/OS there are many system calls that use pointers and also "handles". A handle is a pointer that contain the address of another pointer that contain the address of the object. Handles are used to describe memory locations that can change. The overall memory mapping is shown in Figure 9 ordered by virtual addresses.
5.3 A-line traps execution The subsystem must conform to the MacOS interface described in Inside Macintosh [7]. This raises two fundamental questions: • •
Which part of the application context is useful to the Server ? How must we restore the application context, once the A-line trap has been executed ?
In fact both questions can be rephrased differently: • •
The first one means: does an A-line trap require some CPU register to contain any information ? The second one signifies: what does an A-line trap leave in the CPU registers that is useful to the application after the A-line trap completion ?
For the application, a call to the Toolbox or to MacOS looks like a function call. These functions are implemented in a library. The application calls the library function corresponding to the system call as it calls any other function. The application follows the language conventions to pass arguments and recover results. Our applications are written in C language and so they stack parameters, call the function and the result of the function is returned in one register 6. The library functions set correctly the registers in respect to the Toolbox or MacOS convention. As the Macintosh basic language is Pascal, the library function must convert Pascal arguments to C arguments and back. Each library function contains an A-line trap. 6 The data register 0 on the MC68030.
49
5.3.1. Catching the trap The exception caused by the A-line trap is caught by the Chorus Kernel which detects that this exception is connected to a handler in the Supervisor. The Kernel calls the exception handler from the Supervisor. This exception handler decodes the exception and calls the handler connected to A-line traps (AlineHdl). The AlineHdl composes a message containing the service code and the application context (registers). It sends the message7 to the AlinePort of the Server and waits for a reply.
5.3.2. Executing the MacOS code The Server thread TbxStdAline which waits on the AlinePort, receives the message, saves it in a message pool to be able to receive new messages and to properly reply later. It then calls a function (TbxAline) that analyses the trap word and starts a new thread at the address of the routine corresponding to the A-line in MacOS. The new thread stack pointer is set at the application stack top. In order to keep track of the message number, we use the software register [8] associated with the started thread. When the routine ends, a function is called that creates a new message containing the "may be" modified registers and the software register number. This message is sent to the AlinePort and the thread ends.
5.3.3. Resuming the application The Server thread TbxStdAline receives the return message, and calls a function (AckAline) to acknowledge the A-line trap. This function retrieves the saved message corresponding to the number found in the return message. It modifies the context in respect to the "auto-pop " and "don't-preserve a0 " bits corresponding to the A-line trap [12], and it replies to the supervisor thread AlineHdl. The AlineHdl restores the context of the faulting thread according to the reply message content and resumes it. The faulting thread program counter is set behind the trap word address.
5.4 Interrupt handling Implementation In our prototype, we are only concerned with the level 1 and 2 interrupts. These two levels are used to control most of MacOS when it is running. To each level correspond six different sources of interrupt and these sources are mastered by circuits called the Versatile Interface Adapter (VIA). Each source of interrupt is controlled by its own handler. Like the trap handlers, interrupt handlers are triggered through tables called respectively Lvl1DT and VIA2DT8 . Both tables are in the first page of memory (global variables of MacOS). In its standard operation the system uses the tables to call four handlers, vital to MacOS: • • • •
The One Second interrupt handler that is used to control the accurate time management. The Vertical Blanking interrupt handler which is used to perform recurrent tasks every sixtieth of a second. The Apple Desktop Bus interrupt handler that controls the mouse and keyboard. The Slot interrupt handler that is used to control the extension bus and the video board.
5.4.1. Two step acknowledgement In our model, the MacOS interrupt handlers are called by the server at the user level. There are two supervisor handlers that catch the interrupt at the system level. These supervisor handlers acknowledge the
7 In fact the message contains a type field, which describes whether the message corresponds to an A-line
trap or to a reply to an A-line trap. 8 Lvl1DT means Level 1 Dispatch Table, and VIA2DT means Versatile Interface Adaptor number 2 Dispatch Table. 50
interrupt at the hardware level, and keep a copy of the hardware registers necessary to select the MacOS handler at the server level. The supervisor handlers send a mini-message containing the type of interrupt and the copy of the hardware registers to the TbxSupIt thread. This thread relays the messages to the server.
5.4.2. Interrupt threads At the server level, there are two functions called VIA1Hdl and VIA2Hdl that respectively handle VIA1 and VIA2 interrupt messages. We have two types of interrupt: those that are short to service and those that need a longer time. Short interrupts are directly handled by the VIA1Hdl or VIA2Hdl functions. Interrupts of the other type are handled by specialized threads that call the appropriate MacOS handler. The Server guarantees that there is only one thread executing at a time in each MacOS handler.
5.5. Execution priority The different actors are assigned priorities which ensure that: • •
Every thread in the supervisor has priority over the other threads. Every thread in the server has priority over other user level threads.
The server threads are assigned fixed priority. The interrupt handling threads have priority to the A-line handling threads. When an interrupt handler thread does an A-line trap, the server detects what kind of interrupt handler it is and assigns the same priority to the thread servicing the A-line.
6. Experience We were able to use the Chorus kernel features to integrate MacOS functions in a subsystem over Chorus and to give a MacOS interface to user level applications. This subsystem is able to support distributed applications that use a well known and user friendly interface to communicate with the applications users. In order to allow MacOS interrupt handlers to execute at the user level, the subsystem catches the privilege violations and if it comes from an interrupt handler the supervisor emulates the instruction. The memory mapping of the application in the server actor gives a vision similar to the one given by Macintosh Operating System to its applications. This mapping allows quick and efficient exchange of data between the application and the server. Our first implementation made a heavy use of messages and threads creation, making the overall system slow. We have redesigned parts of the system to use semaphores and permanent threads to speed it up.
7. Conclusion We have demonstrated that system reuse is possible and that it is possible to build a subsystem over a micro kernel by using an existing system in RAM or in ROM. There are many tricky parts in the re utilisation of a system, for example, we had not enough space, in this paper, to investigate the integration of resources initialized by MacOS in our subsystem. These tricky parts make the global work somewhat tough and sometimes it is harder to try and use things already done than rewrite them directly; but reuse seems to be a promise for the future: if we can reuse existing things, we will be able to use new things nearly in the same way. This would allow an evolving operating system which takes advantage of future developments in software.
51
References [1]
[2]
[3]
[4] [5] [6]
[7] [8] [9] [10]
[11] [12]
M. Rozier, V. Abrossimov, F. Armand, I. Boule, M. Gien, M. Guillemont, F. Herrmann, C. Kaiser, S. Langlois, P. Léonard and W. Neuhauser. Chorus Distributed Operating Systems. Computing Systems, 1(4), 1988. Accetta, M.J., Baron, R.V., Bolosky, W., Golub, D.B., Rashid, R.F., Tevanian, A., and Young, M.W. Mach: A New Kernel Foundation for UNIX development. In Proc Usenix Summer, July, 1986. F. Hermann, F. Armand, M. Rozier, M. Gien, P. Léonard, S. Langlois and W. Neuhauser. Chorus, a new technologie for building UNIX systems. In Proc. EUUG Autumn '88 Conference, Cascais (Portugal) 1988. Guide to OSF/1: A Technical Synopsis. O'Reilly & Associates, Inc. 1991. G. Malan, R. Rashid, D. Golub, and R. Baron, DOS as a Mach 3.0 Application. C. Bac, E. Garnier, Cohabitation and Cooperation of Chorus and MacOS. In Proc. Usenix symposium on Microkernel and Other Kernels Architecture, San Diego (Ca), September 1993, pp. 61-71 Inside Macintosh : Overview. Apple Computer Inc. Reading, MA: Addison Wesley, 1992. Chorus Systèmes, Overview of Chorus Distributed Operating Systems, from Chorus v.3 Programmer's reference Manual. Apple MacIntosh Family Hardware Reference, Addison Wesley ref. 19255, 1988. J. Walpole, J. Inouye and R. Konuru, Modularity and interfaces in Micro-Kernel Design and Implementation: A Case Study of Chorus on the HP PA-RISC, In Proc. Usenix symposium on Microkernel and Other Kernels Architecture, San Diego (Ca), Spring 1992, pp. 71-82 Apple A/UX Toolbox Users Manual, from Apple A/UX Documentation. Inside Macintosh Volume 1: Copyright Apple Computer Inc. Reading, MA: Addison Wesley, 1985.
52
