Applying Meta-Programming Techniques to Reconcile Heterogeneous Scheduling Policies in Open Distributed Real-Time Systems Angelo Corsaro, Chris Gill, and Ron Cytron
Douglas C. Schmidt
corsaro,cdgill,cytron @cs.wustl.edu Department of Computer Science Washington University, St. Louis, MO 63130, USA
[email protected] Electrical and Computer Engineering Dept. University of California, Irvine, CA 92697, USA
This paper was submitted to the 3rd International Sympo- tributed real-time and embedded (DRE) systems with comsium on Distributed Objects and Applications, September 8- plex QoS requirements, such as stringent latency, jitter, and 10, 2001, Rome, Italy. dependability needs. For example, future combat systems will involve heterogeneous collections of mobile autonomous vehicles [10] that must collaborate to perform coordinated maneuvers in support of time-critical missions, such as reconnaissance, perimeter defense, and suppression of enemy Abstract air defenses. Likewise, QoS-enabled DOC middleware will In open distributed real-time and embedded (DRE) systems, also benefit commercial DRE systems, such as distributed virdifferent ORB endsystems may use different scheduling poli- tual reality applications, distributed multimedia collaboration cies. To ensure appropriate end-to-end application behavior systems, and massively-multiplayer online persistent-world in an open architecture, however, DRE systems must enforce games. an ordering on activities originating in an endsystem and acKey challenges arising in these types of DRE systems intivities that migrate there, based on the relative importance of volve communicating and enforcing the relative importance of these activities. This paper describes the meta-programming various competitors (such as threads or operations on CORBA techniques applied in Juno, which is an extension to Real-time objects) to ensure appropriate scheduling of system resources CORBA that enhances the openness of DRE systems with re- (such as memory, CPU time, and network bandwidth) at a spect to their scheduling policies by enabling dynamic order- given point in time. Resolving these challenges is essential ing of priority equivalence classes. We use the forthcoming to build DRE systems that are simultaneously: OMG Dynamic Scheduling Real-time Specification of CORBA 1. Open, i.e., system components can connect and interop(DSRT-CORBA) as a case study to illustrate our techniques. erate in a flexible manner without having to be preconfigKeywords: Real-Time, Distributed System, CORBA, Dyured statically; and namic Scheduling, Meta Architecture. 2. Dependable, i.e., the system can preserve key end-toend QoS properties, such as timeliness and resource constraints. 1 Introduction For example, mobile autonomous vehicles [10] should be able to collaborate in a dependable and efficient manner, despite the fact that their scheduling policies and implementations may be heterogeneous. The forthcoming Dynamic Scheduling Real-Time CORBA Joint Revised Submission (DSRT-CORBA JRS) [11] addresses some aspects of the challenges outlined above. For example, the DSRT-CORBA JRS defines a distributable thread mechanism that has the following properties:
Emerging challenges: Distributed object computing (DOC) middleware, such as CORBA [1], COM+ [2], [3], and Java RMI [4], shields developers from many complexities associated with developing distributed systems. For example, DOC middleware allows applications to invoke operations on distributed objects without concern for object location, programming language, OS platform, communication protocols and interconnects, and hardware [5]. The maturation of DOC middleware specifications and implementations over the past decade has greatly simplified the development of open distributed systems with complex functional requirements. More recently, the emergence of quality of service (QoS)enabled DOC middleware, such as Real-time CORBA (RTCORBA) [6], Real-Time Java [7], [8] and Distributed Realtime Java [9], is leading to the simplification of open dis-
1
It can extend and retract its locus of execution to transition from ORB-to-ORB while servicing an operation request. It contends with other competitors for the use of different resources (such as CPU time, memory, or network bandwidth) in the various ORBs it traverses in a dynamic call graph.
Scheduling parameter mapping: The DSRT-CORBA JRS does not define the mapping of scheduling parameters when distributable threads pass through ORBs that are configured with (1) heterogeneous scheduling policies or (2) different scheduling parameters for the same scheduling policy. For example, one ORB visited in a request’s dynamic call For example, Figure 1 illustrates a representative DRE sys- graph traversal could be configured using an earliest deadline tem in which three endsystems are running three ORBs con- first (EDF) [12] scheduling policy, where the scheduler orders competitors based only on the propinquity of their deadlines. Another ORB in the traversal could use a valued-based End System (B) scheduling policy [13], where every competitor is characterRunning an RM Scheduler ized by a time-dependent function that describes the value associated with the competitor at a given point in time—the scheduler tries to maximize the value gained by the system using information provided by this function. In the DSRT-CORBA JRS, when a distributable thread traverses endsystems, its corresponding scheduling information End System (A) must be understood at each endsystem, i.e., the composition of Running an EDF Scheduler scheduling policies used along the chain of endsystems must be semantically coherent, even if the result is non-optimal. At this point, however, there is no standard that specifies how Competitor Created at End-System (A) to provide interoperability between heterogeneous (but still End System (C) Competitor Created at End-System (B) Running a Generic composable) schedulers. This omission limits the openness Competitor Created at End-System (C) Value Based Scheduler of DRE systems using DSRT-CORBA JRS middleware. It contains certain scheduling information that is embedded with other fields in a GIOP request and used by the ORBs visited by the distributable thread to ensure the thread is processed with the appropriate priority end-toend.
Computer
Workstation
Minicomputer
Satellite dish
Server
Workstation
Satellite
Comm. Tower
Figure 1: DRE Systems with Competitors that Migrate from Scheduling information propagation: Another relevant System to System. issue that neither the RT-CORBA nor DSRT-CORBA JRS specification addresses is whether scheduling information figured with three different scheduling policies. Threads are propagated on a hop-by-hop basis through a distributed call distributed across endsystems as a result of remote operation graph should be updated. Although this issue is not directly invocations or distributable thread migration.1 As a result, related to interoperability, the solution described in this paper competitors originating on different endsystems contend for to enable interoperability can be used to propagate and update the same set of resources on each ORB endsystem. To adjudi- scheduling parameters end-to-end. cate this competition, some type of scheduling is required. In this paper, we present a solution to the problems outlined RT-CORBA 1.0 specifies a Scheduling Service to relieve above by application programmers of the tedious and error-prone task Formalizing the problem of interoperability in the context of configuring scheduling properties on each end-system. This of open DRE systems; service is an optional part of the RT-CORBA, specification, however, so that it may not be available for all RT-CORBA Defining formalisms to represent different instances of ORBs. Moreover, the RT-CORBA 1.0 Scheduling Service the problem; and deals only with priorities, so that mappings of more complex Providing a meta-programming architecture [14, 15] that scheduling properties into an order of competitors execution maps the formalized abstractions to an RT-CORBAelegibility remains under-specified. based software architecture. The DSRT-CORBA JRS–and the RT-CORBA specification upon which it builds–are the most advanced open standards Figure 1 outlines our solution approach in the context of that address static and dynamic scheduling in the context CORBA. As shown in this figure, three endsystems are conof open, QoS-enabled middleware for DRE systems. Nei- figured with three different scheduling policies. The competither specification, however, fully addresses the interoperabil- tors initiated at endsystem (A) are square, those initiated at ity aspect of the challenges outlined above, due to under- endsystem (B) are circular, and those initiated at endsystem specification in the areas of: (C) are triangular. To preserve the QoS properties requested 1 Threads at each endsystem are shown with a different shape, depending by the competitors–while simultaneously enabling an open aron the endsystem on which each originated. chitecture in which competitors can traverse endsystems with-
2
out concern for how QoS requirement are expressed–we apply a technique that reconciles 1. The properties used by each scheduler to enforce QoS; and 2. The properties used by each competitor to express its QoS requirements.
Schedulers grant competitors access to shared resources. The order in which competitors can access a resource depends on scheduler policies and competitor properties. Scheduling policies are formulated in terms of the properties they use to determine the ordering of competitors. Thus, these properties can be viewed as an abstraction of the competitors for the purpose of scheduling. Since we focus on dynamic systems, all our schedulers operate on-line [18], rather than off-line [19].
Our approach allows each ORB endsystem to schedule competitors—including those initiated remotely—by adapting Next, we present a formal model for properties, competithe competitors’ properties for use by the ORB’s local scheduler. We use a meta-programming architecture based on a two- tors, and schedulers. The advantage of creating a formal model level reflective middleware model [14, 16, 17] to implement is to enable heterogeneous ORB endsystems to exchange precise information about the properties associated with individthe solution presented in this paper. ual competitors and schedulers. This information allows each Paper organization: The remainder of this paper is orga- endsystem to transform competitors’ properties and reconcile nized as follows: Section 2 defines the formalisms and as- them for each ORB endsystem’s scheduler. sumptions used throughout the paper; Section 3 presents Juno, which is our meta-programming architecture for enhancing the openness of DRE middleware and illustrates how Juno is be- 2.1.1 Properties ing implemented to address problems that arise in the context Definition 2.1 Let be the Universe of properties. A generic of the DSRT-CORBA JRS and the TAO RT-CORBA ORB; element of is denoted by and is called a property type, or Section 4 compares our approach with related work; and Sec- simply a property. Each property is associated with the tion 5 summarizes the contributions of this paper and outlines following tuple: our future research directions.
is the domain of the property. is the default value for the property. 2. That is, given any property we denote its associated , and its associated default value by . domain by Moreover, given a property we define a Tagged Domain ( ) of as: "! $#&%(')# Given any * + we denote the property of the tagged element # by * and its value by * ,"-". * . 1.
This section defines the terminology used throughout the paper and motivates the assumptions that underly our work. The formalisms presented in this section are applicable to any open DRE system. For concreteness, however, we focus the examples in the context of RT-CORBA [6] and the DSRT-CORBA JRS [11].
2.1 Properties, Competitors, and Schedulers We model an open DRE system as consisting of properties, competitors, and schedulers, which are defined informally as follows:
Where:
2 Terminology and Formalisms
In the context of RT-CORBA2 an example of a property is the deadline of a distributable thread. In this case, the domain of the property is the time, which in RT-CORBA is represented as the integral type TimeBase::TimeT. Other examples of properties in RT-CORBA include criticality (which distinguishes classes of real-time competitors) and the periodCompetitors denote entities that can contend for common icity of activities. system resources. Competitors expose properties that describe the features or QoS requirements of a particular Definition 2.2 Given a non-empty set of properties: 2 Henceforth, our use of the term “RT-CORBA” connotes both static and competitor, such as its deadline or worst-case execution dynamic scheduling capabilities. time.
A property describes a feature, such as a criticality level, a deadline, or a constraint on jitter. We do not restrict the domain of the properties, i.e., a property can be a function, which allows utility and/or quality functions to be expressed as properties.
3
/0 1 32 45 367
2.1.3 Schedulers
8 /9 :
