A \Bare-Machine" Implementation of Ada Multi-Tasking ... - CiteSeerX

A \Bare-Machine" Implementation of Ada Multi-Tasking Beneath the Linux Kernel (extended abstract)

Hongfeng Shen, T.P. Baker Department of Computer Science, Florida State University Tallahassee FL, USA 32306-4019 [email protected], [email protected]

Abstract. An Ada tasking kernel is implemented as a layer beneath the

Linux operating system. This provides low overhead and precise control of execution timing for real-time Ada tasks, which execute within the Linux kernel address space, while allowing the Linux operating system to execute non-real-time tasks in the background. The Ada tasking kernel is derived from Yodaiken's Real-Time Linux kernel, with new scheduling and synchronization primitives speci cally to support the GNAT Ada runtime system. Implementing the Ada tasking primitives directly on the hardware does not just lower execution overhead and improve control over execution timing; it also opens the door for a simple restrictedtasking runtime system that could be certi ed for safety-critical applications.

1 Introduction This paper describes how the GNAT multi-tasking runtime system (GNARL) has been ported to execute directly over the hardware of a generic Intel PCcompatible architecture, as a layer underneath the Linux[6] operating system. The GNAT Ada compilation system[8] has been very successful, and has been ported to many processor architectures and operating systems, but until recently its multi-tasking runtime system (GNARL) has not been implemented on a bare machine. The tasking model of the Ada 95 programming language was intended to permit a light-weight bare-machine implementation, and so was the GNARL. The components of GNARL that are dependent on a particular machine and operating system are isolated by a low-level tasking interface, called GNULLI. Previously, GNULLI has always been implemented as \glue code" to the services of an existing thread library, which in turn is layered over an underlying operating system. The performance of Ada tasking has been limited by the threads library and operating system, which in no case were designed to support Ada, and in most cases were not intended for real-time applications. Thus, it has remained an open question how well the tasking implementation would work if it were supported by a small, simple, and highly predictable implementation of the GNULLI, executing directly on the hardware.

The rest of this paper tells more about this project. Section 2 explains the motivation and background, including R-T Linux. Section 3 explains the design of the implementation. Section 4 reports on the outcomes of functional and performance tests.

2 Background There are two independent motivations for doing a bare- machine implementation of the GNULLI. The rst motivation is to achieve the kind of eciency and predictable execution timing needed for hard real-time applications. In versions of the GNULLI where Ada tasks are implemented using the concurrent programming primitives of a commercial operating system, the activities of the operating system can compete with the tasks in the application, and cause unpredicable variations in their execution timing. The second motivation is to lay the groundwork for the kind of small, certi ably-safe implementation of a restricted subset of Ada tasking that was proposed at the 1997 International Real-Time Ada Workshop at Ravenscar [3]. A simple bare-machine implementation is needed for this because the added complexity of a full o-the-shelf operating system would be an obstacle to certi cation. The target hardware architecture for our bare-machine implementation of GNULLI is a generic PC-compatible machine with an Intel 486/586 processor. This was chosen for its wide availability and low cost. A tasking kernel by itself is not be very useful. Interesting applications do input and output, and that requires device drivers. The complexity of a single device driver can exceed that of a tasking kernel. Moreover, hardware device interfaces are often poorly documented or even secret, and subject to frequent changes. Therefore, development and maintenance of an adequate collection of hardware device drivers is very daunting. One way to avoid putting a lot of new work into device drivers is to reuse the device drivers of an existing operating system, such as DOS or Linux. A problem is that these operating systems were not designed for real-time use, and their devices drivers are sources of timing unpredictability. Reuse of non-realtime device drivers is possible, but only if they are run in the background, at low enough priority that they cannot impede the progress of hard-real-time tasks. Support for background-foreground separation is generally a useful thing, since all but the simplest real-time systems include a combination of hard and soft real-time or non-real-time processing. For example, a system might have hard-real-time periodic tasks with tight jitter constraints to serve sensors and actuators, and soft-real-time or non-real-time tasks to perform actions that have inherently variable time delays, such as communicating over a network and logging data to a disk. In such a situation, the I/O drivers for the sensors and actuators would need to meet hard-real-time constraints, but the I/O drivers for the network and disk I/O might be reused from a non-real-time operating system.

A traditional way to implement foreground and background processing is for the foreground task scheduler to have single background task, which in turn executes the background task scheduler and all the background tasks. This model was followed by a series of low-level Ada kernels developed at The Florida State University during the late 1980's and early 1990's, in which the DOS operating system ran as the background task. More recently, this model has been followed by Yodaiken's Real-Time Linux[5]. Linux has many advantages over DOS for the role of background operating system, including support for concurrent threads of control and separate virtual address spaces. Real-Time Linux is a layer of software that is added to the Linux kernel, as a dynamically loaded module. R-T Linux catches hardware interrupts before they get to the regular Linux interrupt handlers. This allows the R-T Linux layer to not only preempt regular Linux tasks, even when they are executing in the Linux kernel, but also to postpone the execution of the hardware interrupt handlers installed by the Linux device drivers. The R-T Linux kernel provides for the creation and scheduling of very light-weight foreground tasks, which execute entirely in the kernel address space. The regular Linux operating system kernel and all the tasks that it schedules run in the background, at a lower priority than all of the R-T Linux tasks. The background tasks have access to the full services of the Linux operating system, but do not have predictable timing. The foreground processes have predictable timing, but cannot use the conventional Linux I/O or make other conventional OS service calls, since they execute below the level of the OS. Communication between the foreground and background is possible only using the bounded FIFO buer structures supported by the R-T Linux kernel. R-T Linux also provides ne-grained (0.8 s) clock and timer services, which can be used to precisely schedule time delays and periodic execution.

3 Design and Implementation Issues 3.1 Basis in R-T Linux For this project, we chose to use Linux as the background operating system, and reuse as much code as possible from R-T Linux. R-T Linux does not support multiple processors, but we had already decided to restrict our attention to single-processor systems, since our objective was a very simple Ada kernel. All the machine-dependent and OS-dependent code of RT-Linux could be reused, including the mechanisms for inserting R-T Linux into the foreground, the context switching code, and the timer driver. The FIFO's could also be reused directly. Only the task scheduler and synchronization primitives needed rewriting, to support the Ada dispatching and locking policies. We chose to rewrite them in Ada, but to simply de ne Ada interface to import the original R-T Linux C-language code for the timer driver and FIFO buers.

3.2 Living Inside the Kernel

Execution within and beneath the Linux kernel imposes some restrictions on what an \application" can do. In particular, it cannot perform operations, like calling the standard C malloc to allocate memory or the standard C libraries to do input and output, that are implemented using kernel traps. Lower-level substitutes must be found for these services, or they must be done without. Dynamic memory allocation can be done within the kernel address space via calls to kmalloc, but for predictable performance these should be limited to a few calls done during initialization of each kernel module. Input and output can be done directly, by means of custom low-level device drivers, or else must be routed through buers to and from background servers, which can use the full I/O capabilities of the Linux operating system. Conceptually, an application that includes both foreground and background tasks has multiple partitions. The foreground tasks exist in a real-time partition that executes in the kernel address space using the bare-machine runtime system. The background tasks exist in one or more non-real-time partitions, each of which corresponds to a Linux process, and which each use a runtime system layered over the Linux operating system.

3.3 Ada 95 Rationale Implementation Model

The Ada 95 Rationale[1] describes a very simple model for implementation of task scheduling and protected object locking on a single processor, based on the notion of priority ceiling locking[2]. Our objective was to reimplement that GNAT tasking primitives according to this model. The model is based on a few simple principles: 1. Scheduling is strictly preemptive, according to task active priority. A task can only execute if it is the task at the head of the list of highest-priority ready tasks. 2. Only ready tasks can hold locks. A task that is holding a lock must give up the lock before it goes to sleep. 3. A task is not permitted to obtain a lock if its active priority is higher than the ceiling of the lock. 4. A task that is holding a lock inherits the priority ceiling of the lock, (only) while it holds the lock. It follows that whenever a task executes an operation to obtain a lock there will be no other task holding that lock. (Otherwise, either the priority ceiling would be violated or the task that is holding the lock should be executing.) Thus, for the implementation of the GNAT tasking primitive Write_Lock it is sucient to simply update the active priority of the current task to the ceiling priority of the lock object referenced as parameter. For the implementation of the primitive Unlock, the active priority of the current task needs to be restored, and a check may need to be done to see whether some other task should preempt the current task.

3.4 Task Control Blocks

In the GNARL each task is represented by a record object called a task control block (ATCB). The ATCB contains a target-dependent component LL. which is a

record containing private data used by the GNULLI implementation. When the runtime system is layered over a thread library, this target-speci c data includes a thread ID. A thread ID amounts to a reference to another record object in the underlying thread implementation, which might be called a thread control block. For our bare-machine implementation, the task implementation is done directly, so there is no separate thread ID and no thread control block; all the necessary information about a task is stored directly in the ATCB.

3.5 Lock Operations

ATCB's, and other GNARL objects for which mutual exclusion must be enforced, are protected by locks. When the runtime system is layered over a thread library, locks are implemented using the mutex objects of the thread library. For example, consider the code in Figure 1. In the code, L.L is a reference to a POSIX mutex object, and pthread_mutex_lock and pthread_mutex_unlock are imported C functions that that lock and unlock a mutex. procedure Write_Lock (L : access Lock; Ceiling_Violation : out Boolean) is Result : Interfaces.C.int; begin Result := pthread_mutex_lock (L.L'Access); Ceiling_Violation := Result /= 0; end Write_Lock; procedure Unlock (L : access Lock) is Result : Interfaces.C.int; begin Result := pthread_mutex_unlock (L.L'Access); end Unlock;

Fig. 1. Locking operations using POSIX thread operations. In the bare-machine runtime system, this is replaced by code that directly implements the operations, as sketched in Figure 2. The code assumes that there is a single ready queue, ordered by priority and linked via the ATCB component Succ. All queues are circular and doubly linked. (Some type conversions have been omitted from the original code, for brevity.) Due to the priority ceiling policy, as explained further above, there is no need to record the owner of the lock, or to use any looping or special atomic hardware operation (like test-and-set) to obtain the lock. That makes these operations much simpler to implement than the POSIX thread mutex operations.

3.6 Sleep and Wakeup Operations

The GNAT runtime system uses primitives Sleep and Wakeup to block and unblock a task, respectively. Sleep is only allowed to be called while the current

procedure Write_Lock (L : access Lock; Ceiling_Violation : out Boolean) is Prio : constant System.Any_Priority := Current_Task.LL.Active_Priority; begin Ceiling_Violation := False; if Prio > L.Ceiling_Priority then Ceiling_Violation := True; return; end if; L.Pre_Locking_Priority := Prio; Current_Task.LL.Active_Priority := L.Ceiling_Priority; if Current_Task.LL.Outer_Lock = null then -- If this lock is not nested, record a pointer to it. Current_Task.LL.Outer_Lock := L.all'Unchecked_Access; end if; end Write_Lock; procedure Unlock (L : access Lock) is Flags : Integer; begin -- Now that the lock is released, lower our own priority. if Current_Task.LL.Outer_Lock = L.all'Unchecked_Access then -- This lock is the outer-most, -- so reset our priority to Current_Prio. Current_Task.LL.Active_Priority := Current_Task.LL.Current_Priority; Current_Task.LL.Outer_Lock := null; else -- If this lock is nested, pop the old active priority. Current_Task.LL.Active_Priority := L.Pre_Locking_Priority; end if; -- Reschedule the task if necessary. -- We only need to reschedule the task if its Active_Priority -- is less then the one following it. -- The check depends on the fact that the background task -- (which is always at the tail of the ready queue) -- has the lowest Active_Priority. if Current_Task.LL.Active_Priority < Current_Task.LL.Succ.LL.Active_Priority then Save_Flags (Flags); -- Saves interrupt mask. Cli; -- Masks interrupts Delete_From_Ready_Queue (Current_Task); Insert_In_Ready_Queue (Current_Task); Restore_Flags (Flags); Call_Scheduler; end if; end Unlock;

Fig. 2. Direct implementation of locking operations. task is holding the lock of its own ATCB. The eect of the operation is to release the lock and put the current task to sleep until another task wakes it up, by calling Wakeup. When the task wakes up, it reacquires the lock before returning from Sleep. When the runtime system is layered over a thread library, these operations are implemented as shown in Figure 3. The ATCB component LL.CV is a POSIX condition variable, and LL.L is a POSIX mutex. The operation Sleep corre-

sponds directly to the POSIX operation pthread_cond_wait, and the Wakeup operation corresponds directly to the POSIX operation pthread_cond_signal. procedure Sleep (Self_ID : Task_ID; Reason : Task_States) is Result : Interfaces.C.int; begin if Self_ID.Pending_Priority_Change then Self_ID.Pending_Priority_Change := False; Self_ID.Base_Priority := Self_ID.New_Base_Priority; Set_Priority (Self_ID, Self_ID.Base_Priority); end if; Result := pthread_cond_wait (Self_ID.LL.CV'Access, Self_ID.LL.L.L'Access); pragma Assert (Result = 0 or else Result = EINTR); end Sleep; procedure Wakeup (T : Task_ID; Reason : Task_States) is Result : Interfaces.C.int; begin Result := pthread_cond_signal (T.LL.CV'Access); pragma Assert (Result = 0); end Wakeup;

Fig. 3. Sleep and wakeup using POSIX thread primitives. In the bare-machine runtime system, these operations are implemented directly, as shown in Figure 4. Wakeup is a little bit more complicated than Sleep, because the task being awakened could be on a Timed_Sleep call (like a Sleep call, but with a timeout). Sleeping tasks with timeouts put on the timer queue, which is ordered by wakeup time. When a timeout expires, the timer interrupt handler moves the timed-out task from the timer queue to the ready queue. Otherwise, if a task is awakened before the timeout expires, it is the responsibility of Wakeup to remove it from the timer queue and possibly disarm the timer. Note that Timer_Queue is implemented as a dummy ATCB, so Timer_Queue.LL.Succ is the rst true task in the timer queue.

3.7 Dynamic Priorities

When GNULLI is layered over a thread library, priority changes are done via calls to the thread library. For POSIX threads, these calls are very general and so are necessarily rather heavy weight, since the position of the thread in the ready queue may need to be changed, and the scheduler may need to be called. However, the way these are used in the GNAT runtime, the normal case can be much lighter weight. It is a policy if the GNAT runtime system that whenever the priority of a task is changed the current task must hold the ATCB lock of the aected task. This means the current task is inheriting the priority ceiling of that lock. ATCB locks have the highest (non-interrupt priority) ceiling. Thus, the current task will be inheriting sucient priority that it cannot be preempted (unless the new priority is an interrupt priority). It follows that unless we are raising the priority of a task otehr than the current task, and raising it to an interrupt priority, there

procedure Sleep (Self_ID : Task_ID; Reason : System.Tasking.Task_States) is Flags : Integer; begin -- Self_ID is actually Current_Task, that is, only the -- task that is running can put itself into sleep. To preserve -- consistency, we use Self_ID throughout the code here. Self_ID.State := Reason; Save_Flags (Flags); Cli; Delete_From_Ready_Queue (Self_ID); -- Arrange to unlock Self_ID's ATCB lock. if Self_ID.LL.Outer_Lock = Self_ID.LL.L'Access then Self_ID.LL.Active_Priority := Self_ID.LL.Current_Priority; Self_ID.LL.Outer_Lock := null; else Self_ID.LL.Active_Priority := Self_ID.LL.L.Pre_Locking_Priority; end if; Restore_Flags (Flags); Call_Scheduler; -- Before leaving, regain the lock. Write_Lock (Self_ID); end Sleep; procedure Wakeup (T : Task_ID; Reason : System.Tasking.Task_States) is Flags : Integer; begin T.State := Reason; Save_Flags (Flags); Cli; -- Disable interrupts. if Timer_Queue.LL.Succ = T then -- T is the first task in Timer_Queue, further check. if T.LL.Succ = Timer_Queue then -- T is the only task in Timer_Queue, so deactivate timer. No_Timer; else -- T is the first task in Timer_Queue, so set timer to T's -- successor's Resume_Time. Set_Timer (T.LL.Succ.LL.Resume_Time); end if; end if; Delete_From_Timer_Queue (T); -- If T is in Timer_Queue, T is removed. If not, nothing happened. Insert_In_Ready_Queue (T); Restore_Flags (Flags); Call_Scheduler; end Wakeup;

Fig. 4. Direct implementation of sleep and wakeup. is no need to move the aected task in the ready queue, or call the scheduler; changing the priority of a task becomes a simple assignment statement.

4 Testing and Performance This section is to be provided in the full paper.

Conclusion We have coded an implementation of the task primitives, and run tests of the primitives by themselves. As of the time this extended abstract is written, we have still to measure the performance of the primitives, and then test them in the context of the rest of the GNARL runtime system. We expect to have performance data in time to include in the full paper by February, and to have additional experience using the bare-machine GNULLI in a restricted Ada asking implementation to report at the Ada-Europe meeting.

References 1. Ada 9X Mapping/Revision Team, Annex D of the Ada 95 Rationale, Intermetrics, Inc. (January 1995). 2. T.P. Baker, Stack-based scheduling of real-time processes, in Advances in Real-Time Systems, IEEE Computer Society Press (1993) 64-96. 3. A. Burns, T. Baker, T. Vardanega, Session Summary: Tasking Pro les, Proceedings for the 8th International Real-Time Ada Workshop, Ada Letters XVII, 5 (September/October 1997) 5-7. 4. ISO/IEC: ISO/IEC 8652: 1995 (E) Information Technology { Programming Languages { Ada. (1995) 5. V. Yodaiken, The RT-Linux approach to hard real-time, paper available at http://rtlinux.cs.nmt.edu/~rtlinuxwhitepaper/short.html. 6. Linux operating system web page, http://www.linux.org. 7. Real-Time Linux operating system web page,http://luz.cs.nmt.edu/~rtlinux/. 8. Ada Core Technologies, Inc., GNAT web page, http://www.gnat.com.

This article was processed using the LaTEX macro package with LLNCS style