Application of the QuO Quality-of-Service Framework to a Distributed Video Application David A. Karr, Craig Rodrigues, Joseph P. Loyall and Richard E. Schantz
Yamuna Krishnamurthy and Irfan Pyarali fyamuna,
[email protected] Department of Computer Science Washington University St. Louis, MO, USA
dkarr, crodrigu, jloyall,
[email protected] BBN Technologies Cambridge, MA, USA
f
Douglas C. Schmidt
[email protected] Electrical & Computer Engineering Department University of California Irvine, CA, USA
Abstract
ment and implementation of a wide variety of software applications in a wide variety of environments. The application of DOC middleware to real-time, embedded software (RES) has resulted in the emergence of middleware support for the strict quality of service (QoS) requirements of RES use cases. For example, the Minimum CORBA specification [OMG00], the Real-time CORBA 1.0 specification [OMG00], and the Real-Time Specification for Java (RTSJ) [Mic] are examples of extensions and services that have grown out of a need to support embedded and realtime applications. Adaptation of distributed software to maintain the best possible application performance in the face of changes in available resources is an increasingly important and complex problem for RES applications. We have been developing QuO, a middleware framework supporting adaptive distributed-object applications.
Adaptation of distributed software to maintain the best possible application performance in the face of changes in available resources is an increasingly important and complex problem. In this paper, we discuss the application of the QuO adaptive middleware framework and the CORBA A/V Streaming Service to the development of real-time embedded applications. We demonstrate a standards-based middleware platform for developing adaptive applications that are better architected and easier to modify and that can adapt to changes in resource availability to meet QoS requirements. These are presented in the context of an Unmanned Aerial Vehicle (UAV) video distribution application. The UAV application is developed using QuO and the A/V Streaming Service, and uses adaptive behavior to meet timeliness requirements in In this paper, we apply QuO to the development of an the face of restrictions in processing power and network Unmanned Aerial Vehicle (UAV) video distribution applibandwidth. We also present some experimental results we cation, in which an MPEG video flow adapts to meet its have gathered for this application. mission QoS requirements, such as timeliness. We discuss three distinct behaviors that adapt to restrictions in processing power and network bandwidth: reduction of 1 Introduction the video flow volume by dropping frames, relocation of Middleware for Distributed Object Computing (DOC) is a software component for load balancing, and bandwidth an emerging and increasingly accepted tool for develop- reservation to guarantee a level of network bandwidth. We 1
2
have developed a prototype application which uses QuO, the TAO real-time ORB, and the TAO A/V Streaming Service to establish and adaptively control video transmission from a live video camera via a distribution process to viewers on computer displays. The TAO A/V Streaming Service is an implementation of the CORBA A/V Streams specification [OMG98], which grew out of the need to transmit multimedia data among distributed objects.
2.1
An Adaptive Framework for DOC The Benefits of Adaptation
Except in systems whose deployment environment is extremely stable, operational DOC systems will encounter more-or-less temporary conditions that impinge on their available computing resources (e.g., processing power and network bandwidth) and have a consequent effect on the ability of the system to deliver the quality of service needed by users. For example, other applications may require some resources, or hardware may fail, or the network may even be reconfigured. It is also possible in many cases that the desired quality of service may change depending on the usage patterns at any given time, e.g., some use cases may require a great volume of data (high precision) even if this comes at the expense of latency (timeliness); other use cases may reverse these priorities. It is highly desirable, therefore, that software systems be able to adapt to these varying resources and needs. Because of this, many existing systems already incorporate specialized adaptations (typically ad-hoc and local to a subsystem) to adapt to at least some of these variations. In other cases, adaptive behaviors are encoded in more general-purpose software; for example, the TCP protocol will adjust its data rate upward as bandwidth becomes available to support transmission of more data, and downward when not enough bandwidth is available to transmit data at the current rate. Such adaptations tend to be “one size fits all,” however, and prove to be poor behaviors for certain applications.
The application demonstrates a standards-based middleware platform for RES applications and that adaptation can be effectively controlled by a superimposed QuO contract to regulate performance problems in the prototype that are induced by heavy processor load on the host of a critical component. Our experience also shows that the use of the QuO framework, in contrast to typical ad-hoc performance-optimization techniques, which become entangled with basic functionality, leads to a form of aspect-oriented programming with a beneficial separation of concerns. This results in code that is clearer and easier to modify, and promotes re-usability of software under different sets of requirements.
The rest of this paper is organized as follows. Section 2 provides a brief introduction to QuO. Section 3 describes the UAV implementation. Section 4 describes the way in which the UAV prototype can adapt to resource constraints using QuO while still delivering its live video feed in a timely manner using the A/V Streaming Service. Section 5 describes some of the domain-specific issues that we encountered in implementing the adaptive strategies, which are illustrative of corresponding issues that we would expect to encounter in other application domains. In Section 6, we present empirical results showing that while system loads can dramatically degrade software performance metrics of the application (in this case, timeliness), a QuO-controlled adaptation restores the metric to the needed value. In this section we also compare our experiences with modifying adaptive behavior through QuO with the more difficult task of modifying ad-hoc adaptations entangled with functional code. Section 7 discusses related work. Section 8 projects future work on this research. Finally, Section 9 presents some concluding remarks.
2.2
The Benefits of a Framework
Adaptation is made more complicated by applicationlevel issues. For example, consider a video-editing system in which the user can fast-forward to a desired section of the video and then copy that section frame by frame to a new file. During the fast-forward mode, the most important performance characteristic may be that the position in the video (measured in seconds since the beginning) advance at a constant rate. The number of frames actually transmitted during any given period of real time is less critical, provided the user is shown enough frames to detect what scene or action is being shown. Once the 2
proxy (i.e., a skeleton) then unmarshals the data and delivers it to the remote servant. Upon method return, the process is reversed. A method call in the QuO framework is a superset of a traditional DOC call, and includes the following components:
“copy” mode is entered, however, it is critical to copy every frame, even if it is not possible to do so at the normal speed of motion of the video. Protocols such as TCP and UDP can relatively easily be selected and configured to adapt within a reasonable set of parameters in either one of the two application modes described above. The difficulty is that this application must be able to switch between one mode and the other during its execution. This complicates the application’s interface to network facilities; the code that implements the “fast forward” and “copy” functions must be tangled up with code to achieve the needed QoS at any instant in the communications protocols. The complexity of this greatly increases when other considerations (e.g., running in different computing environments, or other user preferences) are taken into account. It is therefore desirable to separate the concerns of the program’s functional specification and these condition-dependent optimizations of QoS. Separation of concerns is the primary objective of Aspect-Oriented Programming [KIL+ 96]. The use of an appropriate framework to handle these QoS concerns alongside the functional code created by the application developer, enables a separation of concerns. As we will see, this results in code that is much clearer, is developed at a greater speed, and is easily modifiable to meet new performance requirements.
Contracts specify the level of service desired by a client, the level of service an object expects to provide, operating regions indicating possible measured QoS, and actions to take when the level of QoS changes.
Delegates act as local proxies for remote objects. Each delegate provides an interface similar to that of the remote object stub, but adds locally adaptive behavior based upon the current state of QoS in the system, as measured by the contract.
System condition objects provide interfaces to resources, mechanisms, objects, and ORBs in the system that need to be measured and controlled by QuO contracts.
In addition, QuO applications may use property managers and specialized ORBs. Property managers are responsible for managing a given QoS property (such as the availability property via replication management [CRS+ 98] or controlled throughput via RSVP reservation management [BBN98]) for a set of QuO-enabled server objects on behalf of the QuO clients using those server objects. In some cases, the managed property requires mechanisms at lower levels in the protocol stack. To support this, QuO includes a gateway mechanism [SZK+ 99], which enables special purpose transport protocols and adaptation below the ORB. In addition to traditional application developers (who develop the client and object implementations) and mechanism developers (who develop the ORBs, property managers, and other distributed resource control infrastructure), QuO applications involve another group of developers, namely QoS developers. QoS developers are responsible for defining QuO contracts, system condition objects, callback mechanisms, and object delegate behavior. To support the added role of QoS developer, we
2.3 The Benefits of QuO Quality Objects (QuO) is a distributed object computing (DOC) framework designed to develop distributed applications that can specify (1) their QoS requirements, (2) the system elements that must be monitored and controlled to measure and provide QoS, and (3) the behavior for adapting to QoS variations that occur at run-time. By providing these features, QuO opens up distributed object implementations [Kic96] so that the implementation strategies of the application’s functions can be controlled in an adaptive manner. In a client-to-object logical method call over a typical DOC toolkit, a client makes a logical method call to a remote object. In a traditional CORBA application, the client does this by invoking the method on a local ORB proxy. The proxy marshals the argument data, which the local ORB then transmits across the network. The ORB on the server side receives the message call, and a remote 3
uav.navair.navy.mil/home.htm
are developing a QuO toolkit, described in earlier papers [LBS+ 98, LSZB98, VZL+ 98], and consisting of the following components:
Quality Description Languages (QDL) for describing the QoS aspects of QuO applications, such as QoS contracts (specified by the Contract Description Language, CDL) and the adaptive behavior of objects and delegates (specified by the Structure Description Language, SDL). CDL and SDL are described in [LBS+ 98, LSZB98].
The QuO runtime kernel, which coordinates evaluation of contracts and monitoring of system condition objects. The QuO kernel and its runtime architecture are described in detail in [VZL+ 98].
Figure 1: Artist’s depiction of UAVs
can receive remote-control commands from a ship in orCode generators that weave together QDL descrip- der to perform such actions as changing its direction of tions, the QuO kernel code, and client code to pro- flight or directing a laser at a target. Several UAVs might duce a single application program. Runtime integra- be active during an engagement, as depicted in Figure 1. tion of QDL specifications is discussed in [LBS+ 98]. The prototype supports the UAV concept of operation by disseminating data from a UAV throughout a remotely The QuO contract offers a number of powerful abstrac- located ship. As shown in Figure 2, there are several steps tions for programming QoS in a DOC application. These to this process: include regions, which abstract the notion of regions of operation which may depend on momentary user prefer1. Video feed from off-board source (UAV). ences or on the condition of the computing environment (as reflected by the system conditions). The QuO contract 2. Distributor sends video to hosts on ship’s network. may also contain states, an abstraction on which one can program a state machine whose inputs are the changing 3. Users’ hosts receive video and display it. system conditions. We currently have implementations of QuO to support 4. Users analyze the data and (optionally) send comCORBA applications over C++ and Java, as well as Java mands to UAV to control it. RMI applications.
Our prototype simulates the first three of these steps. The command phase of the fourth step is observed as a requirement to be able control the timeliness of data displayed on a user’s video monitor: if the data is too stale, it will not represent the current situation of the physical UAV and the scene it is observing, and the user cannot control the UAV appropriately. Hence, for example, for such uses it is not acceptable to suspend the display during a period of network congestion and resume the display from the same point in the video flow when bandwidth is restored.
3 The Unmanned Air Vehicle application As part of an activity for the US Navy at the Naval Surface Warfare Center in Dahlgren, Virginia, USA, we have been developing a prototype concept application for use with an Unmanned Air Vehicle (UAV). A UAV is a remotecontrolled aircraft that is launched in order to obtain a view of an engagement, performing such functions as spotting enemy movements or locating targets. A UAV 4
1
2
3
4
Figure 2: Sequence of events in typical UAV operation
Video Source Process
Video Distributor Process
Video Forwarding Code
Throughput UAV Video File
UAV SIMULATION HOST
Display Proxy Process
Instrumentation
Video Forwarding Code
Video Forwarding Code
Output Rate Control Contract
Video Display Process
Frame Rate Priority
QuO Adaptive Control Process
VIDEO DISPLAY HOST
VIDEO DISTRIBUTION HOST
Figure 3: UAV Prototype Architecture
5
Video Display Code
3.1 Prototype architecture
the viewers through the network, even when the network is congested.
Figure 3 illustrates the initial architecture of the demonstration. It is a three-stage pipeline, with an off-board UAV sending MPEG video to an on-board video distribution process. The off-board UAV is simulated by a process that continually reads an MPEG file and sends it to the distribution process. The video distribution process sends the video frames to video display processes throughout the ship, each with their own mission requirements. All remote method calls in this architecture are made via TAO, the real-time ORB developed by the Distributed Object Computing Group at Washington University in St. Louis [SLM98]. However, in early versions of the prototype, ad-hoc TCP connections were made between the processes in order to transmit video data. This made reconfiguration of the system processes (e.g., changing the number of processes or their locations) difficult, as it was necessary for each process to know the specific hosts and ports that would be used to establish each connection. In current versions, we have replaced this flow connection setup between the various processes with TAO’s implementation of the CORBA A/V Streaming Service stream setup. This service is discussed further in section 3.2. QuO adaptation is used as part of an overall system concept to provide load-invariant performance. Some of the video displays located throughout the ship must display the current images observed by the UAV with acceptable fidelity, regardless of the network and host load, in order for the shipboard operators to achieve their missions (e.g., flying the UAV or tracking a target). There are several ways to achieve this goal by appropriate adaptations to various conditions of the system. Among the possible adaptive strategies are:
Send a reduced amount of data, for example by dropping frames of the video. The resultant video appears as if the camera had simply captured fewer images per second, without affecting the speed at which objects in the scene move.
Move the distributor from an overloaded host to a different host where more performance is available.
We discuss these adaptations in more detail in Section 4.
3.2
A/V Streams transport
As described in Section 3.1, we are connecting the components of the UAV example using the TAO A/V Streaming Service [MSS99]. This is an implementation of the CORBA A/V Streaming Service [Obj97], which is intended to support multimedia applications, such as videoon-demand. The TAO A/V Streaming Service is layered over TAO and ACE [SS94], which handle flow control processing and media transfer, respectively. The CORBA A/V Streaming Service controls and manages the creation of streams between two or more media devices. Although the original intent of this service was to transmit audio and video streams, it can be used to send any type of data. Applications control and manage A/V streams using the A/V Streaming Service. Streams are terminated by endpoints that can be distributed across networks and are controlled by a stream control interface, which manages the behavior of each stream. The CORBA A/V Streaming Service combines (1) the flexibility and portability of the CORBA object-oriented programming model with (2) the efficiency of lower-level transport protocols. The stream connection establishment and management is performed via conventional CORBA operations. In contrast, data transfer can be performed directly via more efficient lower-level protocols, such as ATM, UDP, TCP, and RTP. This separation of concerns addresses the needs of developers who want to leverage the language and platform flexibility of CORBA, without incurring the overhead of transferring data via the standard CORBA inter-operable inter-ORB protocol (IIOP) operation path through the ORB.
4
Adaptation in UAV
In this section, we discuss some performance issues in Use a bandwidth reservation protocol to ensure that our UAV concept application, and adaptive behaviors that the distributor is able to send the necessary data to address these issues. 6
A bottleneck may occur in the application because at some point along the video transport path there are not enough resources to send the entire video to the viewers in real time. For example, the distributor host may not have enough CPU power available to dispatch video frames to all viewers at that rate, or there may be insufficient bandwidth in the network path to one or more viewers. In either of these cases, one of the following methods can be used to detect the bottleneck, depending on the system architecture:
Track the number of frames received by the distributor and the number of frames displayed by each viewer, and compare the numbers. This can detect bottlenecks that “back up” the distribution paths as well as overload conditions that slow down the distributor or viewer or that cause frames to be lost.
If the transport provides some form of back pressure, for example if TCP is used, measure the rate at which the distributor is able to send frames to viewers. This will detect not only overload conditions at the distributor, but also network congestion downstream and overload at viewers. When those conditions prevent the viewer from reading frames at the requested rate, back pressure will force the distributor’s sending rate (the measured condition) to decrease.
DATA FILTERING Condition: excessive network or CPU load Action: drop selective frames
Figure 4: Adaptation by filtering frames attained at frame rates of 24 frames or more per second. Figure 4 shows a mechanism for reducing the number of frames in the video stream. For example, if the distributor receives a video at 30 frames per second, it can reduce resource usage by deleting two of every three frames to produce video output at 10 frames per second. This particular example was selected as one of the adaptations in our UAV prototype, for reasons that are described in detail in Section 4.2. Alternatively to reducing the number of frames, the amount of data per frame might be reduced. This would typically reduce the image quality of each frame. A second adaptation is to move the distributor to a host that does not suffer the bottleneck, either because of better network location or because of greater available CPU resources. Figure 5 illustrates this adaptation. A new instance of the distributor must be started on the new host, and new communication paths must be formed between the UAV, the new distributor instance, and the viewer to replace the corresponding paths that led through the old distributor instance. The old distributor can then be halted and its paths torn down. A third adaptation applies when the bottleneck is due to competing data flows that take up some of the network bandwidth needed by the UAV. This adaptation reserves a certain amount of network bandwidth (through an Integrated Services (IntServ) protocol) for the distributor’s communication paths so that a sufficient rate of data can be transmitted. Figure 6 illustrates this adaptation. It is also possible to combine these adaptations in various ways. For example, it might be necessary not only to move the distributor to a new host, but also to send a reduced video flow (e.g., fewer frames) to certain viewers through RSVP-enabled links. Which adaptations should be used can depend on the
In our current implementations of the UAV application, we used the TCP transport protocol, and so were able to control adaptation using the second detection method (measured only at the distributor). We are in the process of creating a new implementation over a UDP transport layer; this implementation requires the first detection method (comparing the frames at distributor and viewer). We now present three examples of adaptation that are used in our current UAV application. One adaptation to these conditions is simply to reduce the amount of data being sent. Depending on user requirements, it may be possible to omit some frames of the video entirely, resulting in an end-user video that displays the motion of the scene in real time (that is, objects that move across the real-life scene at constant speed appear to move at constant speed in the video), but without the total illusion of continuously displayed motion that can be 7
r
mission-critical tasks that each user of the system has to perform. There may be many simultaneous uses of a given video flow that passes through a given distributor, each with a unique set of requirements. For example, the video might be used simultaneously to guide the UAV’s flight, to control on-board systems (such as a laser to “paint” a target), or to collect reconnaissance data for further analysis. Some of these uses may require only a few images per second, others depend critically on getting all video frames sent by the UAV; some may require a high-resolution, wide-angle view, while others need only a part of the image, or lower resolution. Some tasks may have higher priority than others, and so should be the last to suffer degradation of service, and these priorities may change from minute to minute during a mission. QuO provides the flexibility to accommodate such a diverse set of uses: different sets of requirements can be embodied in different contracts. QuO also provides an interface for a resource manager to control the use of resources by the application; for example, the resource manager can set the value of a special system condition that causes the application to relinquish resources that are needed by other applications in the system.
r
LOAD BALANCING Condition: excessive CPU load Action: move distributor to more lightly loaded host Figure 5: Adaptation by moving distributor
4.1
Video transmission concepts
In order to understand the adaptive behavior of the concept application, one must first understand the nature of the underlying video transmission format. In this section we briefly review the relevant issues as they pertain to controlling the adaptive behavior. The Motion Pictures Expert Group (MPEG) format is a widely-used format for the encoding of multimedia data [Gal91]. Several versions of the standard have been defined to deal with various multimedia concerns. The MPEG-1 standard was selected for this project because relevant video footage is readily available in this format, as are various open-source software tools for analyzing, displaying, and modifying videos in this format. MPEG-1 allows multiple synchronized data flows (e.g., video and audio) to be part of the same stream, but the prototype UAV application has only video data, and so we are not (yet) concerned with synchronization of audio with video. More recent MPEG formats offer additional features, such as support for using greater bandwidth and for sending additional types of data separately from the video flow, but
NETWORK RESERVATION Condition: excessive network load Action: Use IntServ to reserve bandwidth BBBBBBB B BBBBB IBPBIBB B PB IPB BBB BIBBB P IP PBP BPIBBBBBBBBBBBBBBBBBBBBBBBBBBBB BPBIP P BP P P P P P P P B I B B B B B B B B B B B B B B BBBI P P B B B B BB
Figure 6: Adaptation by reserving bandwidth
8
in this paper we use only MPEG-1 and will refer to it simply as “MPEG.” The MPEG video format renders a sequence of images (frames of a video) into a highly compressed string of bytes. It was designed to be able to transmit broadcastquality video over a link of 1.5 Mbit bandwidth. A typical application of MPEG-1 is to store a video in a file in this format, and to play it back by retrieving the file, decoding it incrementally and displaying it on a monitor. The format is, however, not limited to retrieved files; for example, devices exist that take in a direct video feed and produce MPEG-encoded output. In our prototype, a live video feed such as this was not available, so it was simulated by reading pre-recorded data from a file at a fixed rate. A video flow consists primarily of a sequence of frames that was (presumably) recorded by a camera at some fixed frame rate (selected from a small set of standard rates used by modern film or video equipment); when displayed at this same rate, the frames accurately show the images and motion of the recorded scene. The video flows we used in testing the UAV application were typically captured at the rate of 30 frames per second. The remainder of the video data consists of various control blocks interspersed among the frames. In order to increase the compression of the video flow, MPEG uses three distinct types of frame. The first and most essential type of frame is the I-frame (standing for “intraframe”). An I-frame is a compressed image of a single frame. For our purposes here, the most important feature of the I-frame is that it is independent of other frames, that is, it can be displayed correctly without knowing anything about the image contained in any other frame of the video. The second type of frame is the P-frame (standing for “predictive”). A P-frame contains only the data necessary to correctly extrapolate the image in question from the image of a previously-displayed frame, (which may have been encoded as an I-frame or as another P-frame, and which may or may not have been the immediately preceding frame) as depicted in Figure 7. In this figure, frames of a video are shown in increasing time order from left to right; the P-frame is shown as a grid, and the arrow indicates the dependence of the displayed image on the previous I- or P-frame. Various algorithms such as motiondetection are used to express the extrapolation concisely.
P Figure 7: Predictive (P) Frame The third type of frame is the B-frame (standing for “bidirectional”). To correctly decode a B-frame requires the images from two other frames (each of which may have been encoded either as an I-frame or as a P-frame), one displayed prior to the B-frame and one displayed after the B-frame, as shown in Figure 8. In this figure, the B-
B Figure 8: Bidirectional (B) Frame frame is shown as a grid, and arrows in both directions indicate the dependence of the B-frame’s image on the images of frames displayed before and after. The image of the B-frame is then constructed by a combination of data interpolated between the two other frames and data encoded in the B-frame itself. Because the format allows different parameters for the compression algorithms, and because the compression ratio can be dependent on the contents of frames, the number of bytes in a frame can vary substantially from one video file to another and even among frames of the same type in the same video. But in general, I frames require by far the most bytes to transmit, because they cannot make use of the data in other frames; P frames are much smaller because most of the necessary data is contained in a prior frame; and B frames are typically the smallest of all, because they need only record the deviations from a straight-line interpolation between two other frames. (In most of the MPEG files we have examined, a few B frames were slightly larger than the smallest P frames in the same video, so size is not absolutely correlated with frame type.) 9
Sequence header The MPEG format groups frames into two higher-level GOP header GOP header structures: a group of pictures (GOP) consists of a header followed by one or more frames, and a sequence consists I B B P B B P B B P B B P B B of a header followed by one or more groups of pictures. 2 0 1 5 3 4 8 6 7 11 9 10 14 12 13 In a typical 30-frame-per-second MPEG encoding, each GOP consists of a single I-frame, four P-frames, and ten Figure 10: Sequence of frames in MPEG file B-frames. These frames are to be displayed in the order matching Figure 9, that is, every fifteenth frame is an Iframe, every third frame after the I-frame is a P-frame, network reservation can be implemented without regard and each I- or P-frame is immediately followed by two for the details of the video encoding; an alternative enB-frames. The arrows in the figure show the dependen- coding can be employed without needing to change these behaviors. The implementation of data filtering to reduce the volume of video data, however, is highly dependent on the video encoding format itself. P B B I B B P B B P In order to perform data filtering in the UAV prototype, we employ the technique of reducing the frame rate transmitted from the distributor to the viewer. Similar Figure 9: Sequence of display of frames in MPEG techniques can be applied elsewhere in the data path, of course, in particular between the UAV itself and the discies of P-frames and B-frames on other frames. Note that tributor. (To reduce the quality of the individual frames P-frames may depend on other P-frames. Hence to dis- displayed, it is necessary to trans-code the contents of the play the image of a P-frame, it is first necessary to have frames themselves; tools to perform this task exist but are received the previous I-frame and all other P-frames (af- not yet used in our application). But the frame rate must ter that I-frame) on which the current P-frame depends. not be reduced in such a way as to create a “slow motion To display a B-frame, it in necessary to have received the effect”; that is, a vehicle that crossed the field of view of I- or P-frame preceding the B-frame in display order and the UAV camera in 2.5 seconds should cross the applicathe I- or P-frame following the B-frame in display order, tion display in 2.5 seconds, and so forth for all other action as well as all other I- and P-frames on which those two in the video, in order that the display continue to present frames depend. a true and up-to-the-moment view from the UAV itself. Within a GOP, the frames can be encoded in a different For the purposes of experiments performed on our protoorder from the order in which they are to be displayed. type, therefore, we assumed that the UAV transmits video The first frame is always an I-frame. Typically, the first data at the rate of 30 frames per second, which is received two B-frames are to be displayed before this I-frame, even by the distributor at that rate (when system resources perthough they occur later in the encoding. (An obvious ra- mit), but the distributor implements an adaptive behavior tionale for this is that the I-frame can be decoded by itself that sends out a smaller number of frames representing but the B-frames can be decoded only when the I-frame the action that occurs during each second. The subset to is known.) Similarly, the two B-frames that are displayed be sent is selected by dropping (eliminating) some frames before each P-frame are encoded after the P-frame. So the from the video, and sending out the remaining frames at a 15 frames of a GOP are encoded as shown in Figure 10, reduced rate. where the numbers under the frame indicate the order in Our options for efficiently dropping frames are limwhich these frames are to be displayed. ited by the MPEG encoding format, however, and so we must follow a set of application-specific adaptation strategies. For example, if we could simply drop every second 4.2 Adaptation in a video domain frame, we would be left with 15 frames out of every 30, At the beginning of Section 4, we presented three adap- and could send the video at the rate of 15 frames per sective behaviors. Of these behaviors, load balancing and ond without affecting the apparent speed of motion of the
10
scene. (At this rate the human eye would be able to detect a slight flickering or stroboscopic effect as one image was replaced by the next, because the 1/15 second interval between images is a little longer than the threshold for distinguishing successive still images from true continuous change in the scene.) This particular example is impractical in the videos we worked with, however, because some of the frames dropped would have been I-frames, but 16 other consecutive frames (all other frames in the same GOP, and the first two B-frames in the next GOP) depend directly or indirectly on each I-frame, so dropping a single I-frame results in more than 0.5 second of the video being lost. Further, in support of the application, one of whose requirements is to track moving images as continuously as possible, it is highly desirable to minimize any intervals in the video during which the image remains still. Hence it is not desirable to display, say, the I-frame and nine subsequent frames in a GOP, and drop the remaining six frames. While this scheme incurs the load entailed in transmitting 18 frames per second, it suffers intervals of 7/30 second during which no motion is seen; this is a longer interval than if the video were displayed at at a steady rate of only 5 frames per second (6/30 second between frames). From the human-factors point of view, it is desirable that the intervals between the correct display times of frames be as uniform as possible. Because of these issues and the dependencies between frames, the best frame-dropping protocols drop B-frames when only a few frames are to be dropped (because a missing P-frame implies an interval of at least 1/5 second —six frames— between the correct times of displayed frames). There are 20 B-frames in each second of video, so this technique can bring the sending rate down to 10 frames per second. To drop more frames, P-frames can then be dropped. I-frames should be dropped only if intervals of 1 second or more between images are acceptable. If each frame in the MPEG format contained an exact timestamp showing the time at which it was supposed to be displayed (e.g., expressed in milliseconds since the instant when the video started to be displayed), it would be relatively easy to delete frames from the video one at a time without causing jitter (frames displayed sooner or later than the correct time). But the frames do not include such a timestamp, so there are fewer good options for dropping frames. For example, if we were to drop one
frame out of every three, we would be left with a video in which, to minimize jitter, some images in a sequence should be displayed 1/30 second after the previous image, and some 1/15 second after. But the typical MPEG video player is designed to read a single frame rate from the sequence header and to display all frames in the sequence at that constant rate; indeed the MPEG format is designed to support nothing more sophisticated. A technique for “dropping” frames without incurring this display-timing difficulty is to replace B- or P-frames with “dummy” frames rather than dropping them entirely [HSG00]. The dummy frame contains only the minimal information to allow the viewer to display an image similar to the preceding (or following) image, but omits all the new image information that the B- or P-frame itself would have contained; hence this frame is very small, and the bandwidth required to transmit the video is reduced, although not as much as if the frame were eliminated entirely. For our current implementation we chose to drop frames entirely in such a way that the remaining frames were to be displayed at a constant rate. This implementation provided us with three significantly different levels of QoS among which to adapt the application, as determined by the frame rate:
30 frames per second. This is done by transmitting the video intact. When this rate is achieved it represents the highest level of QoS.
10 frames per second. This is done by dropping all B-frames from the video, and transmitting all the Iand P-frames. At this level of QoS, most perception of motion in the video scene is preserved, but a careful human observer can detect by eye the transitions from one image to the next.
2 frames per second. This is done by dropping all P- and B-frames from the video, and transmitting all I-frames. At this level of QoS, the image changes frequently enough for some motion to be judged, but finer details of motion (and some very short-lived actions) can be lost entirely.
It is then possible to adaptively switch among these three frame rates by assigning each frame rate to a dif-
11
ferent region of a QuO contract, and setting the framedropping protocol at any given time according to the current region. (We also implemented frame rates of 1 frame per second and slower, but due to their extremely low levels of fidelity we excluded these from our adaptive behaviors on human-factors grounds.) In actual MPEG videos we examined, mean sizes of I frames were in the range between 11500 and 14000 bytes, of P frames in the range between 4900 and 7000 bytes, and of B frames in the range between 2900 and 3400 bytes. In the sample video provided by the Navy and selected (for reasons unrelated to our UAV implementation or adaptations) as the test case for the UAV application, I-frames averaged approximately 13800 bytes, P-frames approximately 5000 bytes, and B-frames approximately 2900 bytes. The approximate size in bits of two average GOPs is therefore (2(13800) + 8(5000) + 20(2900)) 8 = 1004800
consequently the encoded sizes of these dropped frames are relatively small. On the other hand, reduction in the perceived value of the reduced-frame-rate display to a human viewer also is not proportional to the reduction in frame rate, judging from the informal reactions of people who have watched demonstrations of the application adapting.
4.3
QuO Contract
Figure 11 shows a QuO contract that adapts the distributor to available resources by increasing or decreasing the rate at which frames are transmitted to viewers. We show it to illustrate the high-level adaptation-oriented abstraction that QuO provides, and to illustrate the isolation of these adaptive behavior aspects from the rest of the code. This contract is substantially similar to the one used in the prototype, but with certain details of syntax and non-essential functionality elided in order to fit it legibly within a single column of text. The contract divides the operational conditions of the distributor into three QuO regions:
(i.e., near the capacity of a 1.5 Mbit link). This is the bandwidth requirement of sending one second of the video at the full rate of 30 frames per second. NormalLoad This region is entered when resources are If we drop the rate to 10 frames per second by elimiadequate to transmit the video to the viewers at the nating the B-frames, the bandwidth required, in bits per full bit rate. In this region, the distributor sends 30 second, falls to approximately frames per second.
HighLoad This region is entered when there are not adequate resources to transmit the video at the full bit and if we drop the rate to 2 frames per second by elimrate. In this region, the distributor sends 10 frames inating the P-frames as well, the required bandwidth in per second. Intermediate frames of the video are bits per second falls to approximately dropped so that the remaining frames, displayed at the rate of 10 per second, depict normal-speed mo2(13800) 8 = 220800: tion. (2(13800) + 8(5000)) 8 = 540800
That is, reducing the frame rate from 30 to 10 (a 67 percent reduction) reduces the bit rate by 46 percent, and reducing the frame rate from 30 to 2 (a 93 percent reduction) reduces the bit rate by 78 percent. These are substantial reductions of bandwidth and other system requirements, so it is not hard to find system conditions under which the full bandwidth is not supportable, but one of the reduced-bandwidth adaptations is. The reduction in bit rate is not proportional to the reduction in frame rate, because the frames that must be dropped first are precisely those frames that have the greatest dependency on other frames (and the fewest frames depending on them), and
ExcessLoad This region is entered when there are not adequate resources to transmit the video even at the reduced bit rate required for 10 frames per second. In this region, the distributor sends 2 frames per second, again dropping intermediate frames in order to preserve the speed of motion. This contract communicates with the rest of the system in several ways. First, the system condition object actualFrameRate is set periodically by the distributor; its value is the actual number of frames sent in the previous second. The contract uses this system condition to
12
contract UAVdistrib ( syscond quo::ValueSC timeInRegion, syscond quo::ValueSC actualFrameRate, callback InstrCallback instrControl, callback SourceCtrlCallback sourceControl) { region NormalLoad (actualFrameRate >= 27) { } region HighLoad ((actualFrameRate < 27 and actualFrameRate >= 8)) { state Duty until (timeInRegion >= 3) (timeInRegion >= 30 -> Test) { } state Test until (timeInRegion >= 3) (true -> Duty) { } transition any->Duty { sourceControl.setFrameRate(10); timeInRegion.longValue(0); } transition any->Test { sourceControl.setFrameRate(30); timeInRegion.longValue(0); } } region ExcessLoad (actualFrameRate < 8) { state Duty until (timeInRegion >= 3) (timeInRegion >= 30 -> Test) { } state Test until (timeInRegion >= 3) (true -> Duty) { } transition any->Duty { sourceControl.setFrameRate(2); timeInRegion.longValue(0); } transition any->Test { sourceControl.setFrameRate(10); timeInRegion.longValue(0); } } transition any->NormalLoad { instrControl.setRegion("NormalLoad"); sourceControl.setFrameRate(30); } transition any->HighLoad { instrControl.setRegion ("HighLoad"); } transition any->ExcessLoad { instrControl.setRegion ("ExcessLoad"); } };
Figure 11: QuO Contract for UAV
gauge the amount of data that the distributor has resources to send. This particular measurement is quite generalpurpose; whether the restricted resource is bandwidth, an I/O device, or CPU time, to the extent that this restriction affects the ability of the distributor to transmit video at its desired frame rate, the deficiency will be detected in the form of a reduced actual frame rate. The prototype implementation averages the rate over a period of one second; a shorter period may be practical, but the rate must be averaged over some period in order to be measurable and accurate. An alternative that might provide a shorter reaction time would be to monitor more basic system conditions such as processor load and network load, and to predict when the achievable frame rate is likely to be reduced rather than merely observing it, but such schemes entail tradeoffs, such as the greater complexity of calibrating the predictions, and the failure to detect performance problems caused by conditions that are not measured. A disadvantage of estimating the capacity to send frames by measuring only the frames actually sent is that it is difficult to detect when there is excess capacity (and when the frame rate might safely be increased). This contract addresses this problem by occasionally attempting to send the video flow at the next higher frame rate from its current setting. The frequency and duration of these “tests” is controlled by a state machine within the current region, which alternates between the Duty and Test states at certain intervals of time. The time in any state is measured by virtue of the system condition timeInRegion, which is set to zero every time there is a state transition and is thereafter incremented once per second. Based on the value of actualFrameRate, then, the contract selects the correct region from among NormalLoad, HighLoad, and ExcessLoad, which in turn controls the frame rate via execution of the sourceControl.setFrameRate callback, which is called on transition to the NormalLoad region or to the Duty states of the other two regions. Then, if the contract is in a Duty state, after the value of timeInRegion passes a predetermined threshold the contract will transition to the Test state for a few seconds, at which time it sets a higher frame rate. The until clause of each state prevents any transitions out of that state for a few seconds, ensuring that a stable measurement of the new achievable frame rate is made; at the end of this time, if the test succeeds (that is, if the actual frame rate is observed to be at the 13
requested rate) a contract reevaluation results in a change of regions. Otherwise, the contract begins a new Duty cycle in the same region (unless, of course, insufficient resources for the HighLoad region force a transition down to the ExcessLoad region). The instrControl callback enables the contract to communicate with a resource manager that monitors and controls the resource usage and location of application processes. We installed the prototype in an environment controlled by such a resource manager. The transition into the ExcessLoad region caused the contract to execute the code in transition any->ExcessLoad, which in turn transmitted an indicator of the region to the resource manager. The resource manager then restarted the distributor on a different host where more resources were available.
5 Domain-specific issues We encountered various issues with the video format and components of our UAV concept prototype that affected the ability to build an application with the functional and adaptive properties we describe. We discuss two of the main issues below. These domain-specific issues are illustrative of the issues of the multiple levels at which adaptation interacts with specific features of an application domain, and the difficulty of composing current off-the-shelf software.
5.1 Designed-in latency The interpolation of B-frames between the other frames of an MPEG video significantly increases the compression of the video compared to what is practically possible using only I- and P-frames. This is simply because the average B-frame is smaller than the average P-frame (the next best candidate for replacement). This is why typical MPEG videos use the format described in Section 4.1. The price of this compression is the fact that some video frames are dependent on other frames, not just frames that are displayed earlier but also frames that must be displayed later. Figure 12 shows how this affects the latency of an application such as the UAV. A sequence of images are de-
Scene
Encoding P B B I B B P B B P
Display time
Figure 12: Timing of Recording and Display of Frames
tected by the camera mounted on the UAV, one frame every 1/30 second. Suppose these frames are recorded by appropriate hardware and software into an MPEG video flow of the kind we have used, containing I-, P-, and Bframes. If the UAV had used a traditional television broadcast format such as NTSC or PAL to transmit the video images to a television receiver on the ship, each pixel (or, rather, each pixel-sized segment of a line on the TV monitor) would be displayed at very nearly the exact time it was detected; the only latency would be that entailed by the time taken for a radio wave to travel from UAV to ship, plus some delays in the hardware for detecting and reproducing the image. Of course this format requires extremely high bandwidth (due to total lack of compression) and it relies on the allocation of physical wires or broadcast frequencies to transmit this image data exclusively; that is why our prototype sends a compressed encoding over a packet-oriented network. But using the typical MPEG encoding, a pixel that is to be encoded in the first B-frame in Figure 12 cannot be transmitted at all until the following I-frame (on which the encoding of the B-frame depends) has been observed. This fact alone delays the encoding (and transmission) of the B-frame by at least 1/15 of a second (two interframe intervals) after the detection of the B-frame’s image. And that estimate assumes that I-frame detection, encoding, and transmission can all occur instantaneously. A more realistic assumption is that the B-frame is delayed by at least 1/10 second, assuming hardware and software that can completely record, en-
14
code, and transmit an I-frame in just 1/30 second. When one frame is delayed 1/10 second—and in fact when this happens to one frame out of every three—the entire video display must be delayed by 1/10 second in order to avoid jitter. And this is merely the delay imposed by the video format itself, not including any latency that inevitably occurs in the actual transmission of the frames from one stage of the system to the next. 5.1.1 Partial solution One approach that would reduce the inherent latency of the MPEG format is to encode the MPEG format using only I- and P-frames, and no B-frames. That is, where currently we are using MPEG video flows in which each 1/2 second of time that passes in the scene is encoded as shown in Figure 10, instead we might encode the same 15 images as one I-frame followed by 14 P-frames. Simply replacing all B-frames by P-frames has the disadvantage, however, that adaptation by frame-dropping becomes harder. We cannot simply delete two of every three P-frames, for example, because all P-frames after the first P-frame in a GOP are dependent on that first P-frame. So in order to for the distributor to drop frames, either it must trans-code all the P-frames it sends (i.e., recompute P-frames by decoding and re-encoding the video, or otherwise composing the sequences of Pframes received), or we must send it more I-frames. For example, by sending the original 30-frames-per-second video as a repeating sequence of IPP, we could adapt to 10 frames per second simply by dropping P-frames, and to lower rates by dropping some I-frames as well. Either of the suggested re-encodings carries a bandwidth and I/O penalty due to the replacement of more compactly encoded frames by ones that require a longer encoding. Specifically, consider the frame sizes of our sample video (described in Section 4.2) and assume that the average P-frame size remains the same. Then the number of bits in the encoding of one second of video, consisting of 2 I-frames and 28 P-frames (i.e., merely replacing all B-frames in the existing encoding by Pframes), is (2(13800) + 28(5000)) 8 = 1340800
and the cost of sending 10 I-frames and 20 P-frames (a
repeating IPP sequence) is (10(13800) + 20(5000)) 8 = 1904000
which figures are, respectively, 33 percent and 89 percent higher than the cost of the encoding that we used.
5.2
Viewer timing idiosyncrasies
Because the MPEG format is often used for applications with very different performance characteristics than the UAV prototype, there are issues that arise with respect to the MPEG viewer. The viewer that we used to obtain our current results (a program derived originally from Berkeley mpeg play) presented difficulty in showing frames at the desired frame rates without jitter. First, the viewer design included a relatively complex built-in scheme for optimizing the display time of frames. This posed a difficulty for us because the optimizations addressed concerns very different from the ones in which we were interested. In particular, we found that while our MPEG encoding required buffering of approximately 0.1 second of video before display, the viewer buffered frames for a considerably longer time (typically about 0.6 to 0.7 seconds, estimated by empirical observations) before displaying them. Such an amount of buffering is not sufficient for smoothing out latency variations when retrieving video from, say, a typical World Wide Web site, but might have been useful for retrieving the same images from a particularly bursty disk drive. The viewer code is also complicated by a scheme for dropping frames in the viewer itself, presumably because some display devices might not be fast enough to display all frames at the rate specified in the MPEG flow. Neither of these performance conditions applies in our target application, and the buffering in particular defeats our goal of minimal end-toend latency. In fact, this special code in the viewer, which was presumably intended to reduce jitter in the display, actually increased the display jitter in our prototype. This was because rather than displaying frames at the rate they were received, the viewer would attempt to display frames at the rate its internal variables indicated, and secondarily would attempt to keep its buffer filled. If we simply started delivering 10 frames per second instead of 30, the viewer would display the new frames in bursts on
15
the order of 20 frames with 1/30 second interval between frames, and a much longer interval between bursts. A second difficulty contributing to this bursty display was that the program set its frame rate at the beginning of each run, immediately on reading the first frame rate indicator in the MPEG flow, and never updating it thereafter. This is a natural design for the typical MPEG application, in which a file really contains video recorded at one and only one frame rate, and these frames are all to be displayed. Our application, however, required special code to be inserted in order to modify certain internal variables of the viewer at any time during a run. In the latest versions of the prototype based on this viewer, we smoothed out the burstiness of the display by continually modifying the requested display rate to match the rate at which the viewer proxy forwarded frames to the viewer. This workaround was not entirely satisfactory, however. In particular, the difficulty of ascertaining how many frames were buffered at any given instant caused difficulty when the frame rate changed adaptively: for example, if frames sent at 30 frames per second are still in the buffer when the viewer is told to reduce the rate to 10 frames per second, the viewer displays these frames in slow motion; on the other hand, if we wait too long to tell the viewer to change the frame rate, some of the 10frame-per-second frames will be displayed in fast motion. These difficulties are in sharp contrast to the experiences in modifying program behaviors that were designed to be controlled by the QuO framework. We revisit this point in Section 6.2. For ongoing work, we were induced finally to make what might be termed a radical “compile-time adaptation”: we have completely replaced our MPEG viewer with another implementation, the DVDview player [Far]. Preliminary observations of this player indicated that it contributes a relatively small amount of buffer-induced latency, and it appears to display frames at the rate it receives them. That is, because it does not implement a fixed set of complex adaptive strategies within the basic functional code itself, this viewer is more reusable, and in particular is suitable as a QuO component in order to achieve better end-to-end behavior in a variety of circumstances. While further testing is required, this viewer appears promising for the purposes of our prototype application.
6 6.1
Results Adaptation controls latency
We performed experiments to test the effectiveness of our adaptive behavior in the UAV application. The three stages were run on three Linux boxes, each with a 200MHz processor and 128MB of memory. The video transport was TCP sockets. At time t = 0, the distributor started. Shortly after this, the video began to flow. At t = 60 seconds, t = 62 seconds, and t = 64 seconds, three special load-simulating processes were started on the same host as the distributor, each attempting to use 20 percent of the maximum processing load (a total of 60 percent additional processing load). This reduced the distributor’s share of processing power below what it needed to transmit video at 30 frames per second. At t = 124 seconds, the load was removed. At time t = 300 seconds (approximately), the experiment terminated. The basic premise is that the full load was applied for a duration of one minute, starting after the pipeline had had time to “settle in,” and ending a few minutes before the end of measurement so we could observe any trailing effects. This scenario was run twice, once without QuO attached and without any adaptation (the control case) and once with a QuO contract causing adaptation (the experimental case). For the purposes of this experiment, the only adaptation enabled was to reduce bandwidth by dropping frames. Figure 13 shows the effect of the increased load on the latency of the video stream. In this graph, the x-axis represents the passage of time in the scene being viewed by the UAV, and the y -axis represents the “lateness” of each image, that is, the additional latency (in delivery to the viewer) caused by the system load. That is, if all images were delivered with the same latency, the graph would be a constant zero. The label “Load” indicates the period of time during which there was contention for the processor; without QuO adaptation, the video images fall progressively further behind starting when the contention first occurs, and the video does not fully recover until some time after the contention disappears. Figure 14 summarizes these results. The lateness values in all these figures are based on the timing of the I frames, which occur 2 times per second and (in the ideal)
16
35
Concern Display time of frames Transmission load
30
9
25 20
Load
^
15 10
with QuO
)
5 0 0
50
100
without QuO
150
200
250
QuO
Hours
300
Figure 13: Lateness of frames received by viewer
No Yes
Ad hoc
Time to develop Months (or never)
Figure 15: Impact of QuO framework on development time
6.2
Adaptation
Coding Method
Lateness (sec) Mean Maximum 5.400 32.696 0.067 1.930
Figure 14: Mean and maximum lateness of frames
Software engineering with QuO
The effectiveness of QuO as a software engineering framework is exemplified in Figure 15. In this table, the concern of “display time of frames” refers to the issues described in Section 5.2 concerning the rate at which frames are displayed. These issues arose because the viewer implementation mixed QoS concerns in an ad-hoc fashion with application function code. Because of this manual tangling of concerns, the code was unnecessarily complex and it was very difficult to modify or even fully understand its behavior. In other words, adaptation of the program to new performance criteria (i.e., fast local decoding and display hardware but variable quality of video input, as opposed to its original domain in which the video input was of uniform quality but decoding and display could suffer delays) required an unacceptably high investment of effort by programmers. On the other hand, changing the parameters of behaviors controlled by QuO often took only minutes (for a simple change in, say, a threshold value) to a day (for more substantial changes in behaviors exhibited).
are not biased between non-adaptive and adaptive cases since our adaptations never drop I frames. (We also obtained numbers for all frames, and they are substantially similar to these.) Lateness for each frame was calculated by counting the I frames seen so far and computing how long it should have taken to get to the nth frame. The maximum lateness, then, is the greatest delay between the time we expected a frame to be displayed and the time that frame actually was displayed. The mean lateness, of course, is highly dependent on the period over which it 7 Related work was averaged (in this case, an arbitrarily chosen period of five minutes), but assuming the same period is used in Hemy et al. present an adaptive MPEG transmission apall cases, the mean lateness is a legitimate figure of merit, plication that also does frame-dropping, but in a slightly with lower values representing better performance. different way [HSG00]. Where we delete frames enThe outcome of this experiment demonstrated that tirely, they insert “dummy” frames into the MPEG flow adaptation leads to improved performance of the applica- in order to replace the dropped frames. In this way they tion. The added latency caused by adverse system condi- achieve most of the possible reduction in bandwidth withtions (in this case, excessive CPU load) occurs in a sharply out changing the frame rate used by the viewer (although reduced magnitude and duration when adaptation is en- the rate of sending new images is reduced by the same abled, and the video image is continuously usable for its factor as in the UAV prototype). The Agilos middleware project implements a hierarchiintended real-time purpose despite the fluctuation. 17
cal adaptive QoS control architecture [LN00], paralleling QuO in some ways, and differing in others. For example, while QuO requires no direct knowledge of other applications that share common resources (though it is possible to link these together via common system condition objects), Agilos incorporates all applications in an environment, including possibly unrelated applications, into a common framework of application-neutral resource control.
8 Future work A number of extensions and additions to the current work are in progress and are being planned. The results in Section 6 were obtained from a version of the UAV prototype that used TCP as its communication protocol. We are currently developing a prototype in which A/V Streams uses the UDP protocol instead. We are also in the process of implementing bandwidth reservation for the MPEG flows over A/V Streams. This will enable QuO contracts for the UAV to use a richer set of adaptations, as described in Section 4, and will afford opportunities to test UAV adaptation under conditions in which multiple system conditions (such as CPU load and network congestion) vary independently, and in which multiple adaptive strategies (such as bandwidth reservation and frame dropping) may be combined. In future versions of the UAV application, the distributor will be able to support multiple viewers, dynamically created and connected at run time, and independently adaptable. Moreover, we plan to support multiple distinct video flows in the system (possibly involving multiple sources and distributors). These features will allow the investigation of the interaction of multiple adapting entities in a complex system. QuO itself is still an evolving framework —for example, new syntax for states and for “locking down” regions was added to the contract language in order to more clearly express contract features used by the contract of the UAV prototype— and the development of the UAV application will continue interacting with the evolution of QuO, both as a testing ground for improvements to the framework and as a driver of more innovations.
9
Conclusion
In this paper, we described a prototype multimedia application, the UAV video distribution system, and the standards (MPEG video and CORBA A/V Streams streaming transport) over which this prototype is implemented. We describe QuO, a framework for distributed object computing designed to enable adaptive optimization of distributed software systems. We demonstrated how QuO can interact with a distributed-object (CORBA) application —even one in which the communication path to be adapted is not a client-server method call and return— in order to implement application- and implementationspecific adaptations to system performance issues. Further, we present empirical results that show that QuO can perform these adaptations effectively and efficiently. Moreover, our experience was that the use of the QuO framework made the implementation and redesign of adaptive behaviors easier for developers than ad-hoc methods typically used. These methods involve extensive manual entanglement of code to perform basic functions with code to optimize performance under specific conditions, resulting in code that is difficult to develop, understand, and maintain. In contrast, the QuO-based adaptive behaviors are separated from the basic video functions, are easy to understand by inspection, and are easily modifiable.
Acknowledgements This work is sponsored by DARPA and US AFRL under contract nos. F30602-98-C-0187 and F3361500-C-1694. The authors would like to gratefully acknowledge the support of Dr. Gary Koob, Thomas Lawrence, and Mike Masters for this research. The authors would also like to acknowledge the contributions of Michael Atighetchi, Tom Mitchell, John Zinky, and James Megquier, the Naval Surface Warfare Center (NSWC) Dahlgren, VA (in particular Mike Masters, Paul Werme, Karen O’Donoghue, David Alexander, Wayne Mills, and Steve Brannan), and the DOC group at Washington University, St. Louis, and University of California, Irvine, to the research described in this paper.
18
References
[LSZB98] Joseph P. Loyall, Richard E. Schantz, John A. Zinky, and David E. Bakken. Specify[BBN98] BBN Distributed Systems Research Group, ing and measuring quality of service in disDIRM project team. Dirm technical tributed object systems. In Proceedings of overview. Internet URL http://www.distthe 1st IEEE International Symposium on systems.bbn.com/projects/DIRM, 1998. Object-oriented Real-time distributed Computing (ISORC), April 1998. + [CRS 98] M. Cukier, J. Ren, C. Sabnis, D. Henke, J. Pistole, , W. Sanders, D. Bakken, M. Berman, [Mic] Sun Microsystems. The real time D. Karr, and R. E. Schantz. AQuA: An adapspecification for java. Internet URL tive architecture that provides dependable dishttp://java.sun.com/aboutJava/ communitributed objects. In Proceedings of the 17th typrocess/first/jsr001/. IEEE Symposium on Reliable Distributed Systems, pages 245–253, October 1998. [MSS99] Sumedh Mungee, Nagarajan Surendran, and Douglas C. Schmidt. The Design and Perfor[Far] Dirk Farin. DVDview: Softwaremance of a CORBA Audio/Video Streaming only MPEG-1/2 video decoder. Service. In Proceedings of the Hawaiian InterInternet URL http://webrum.uninational Conference on System Sciences, Janmannheim.de/math/farin/dvdview.html. uary 1999. [Gal91] Didier Le Gall. MPEG: a video compression standard for multimedia applications. Commu- [Obj97] Object Management Group. Control and Management of Audio/Video Streams: OMG RFP nications of the ACM, April 1991. Submission, 1.2 edition, March 1997. [HSG00] Michael Hemy, Peter Steenkiste, and Thomas Gross. Evaluation of adaptive filtering of [OMG98] OMG. Control and Management of Audio/Video Streams, OMG RFP Submission (ReMPEG system streams in IP networks. In IEEE vised), OMG Technical Document 98-10-05. International Conference on Multimedia and Object Management Group, Framingham. MA, Expo 2000, New York, New York, 2000. Oct 1998. [Kic96] Gregor Kiczales. Beyond the black box: Open implementation. IEEE Software, 1996. [OMG00] OMG. CORBA 2.4 Specification, OMG Technical Document 00-10-33. Object Management [KIL+ 96] Gregor Kiczales, John Irwin, John Lamping, Group, Framingham. MA, Oct 2000. Jean-Marc Loingtier, Cristina Videria Lopes, Chris Maeda, and Anurag Mendhekar. Aspect[SLM98] Douglas C. Schmidt, David L. Levine, and oriented programming. ACM Computing SurSumedh Mungee. The Design and Perforveys, 28(4es), 1996. mance of Real-Time Object Request Brokers. Computer Communications, 21(4):294– [LBS+ 98] Joseph P. Loyall, David E. Bakken, Richard E. 324, April 1998. Schantz, John A. Zinky, David Karr, Rodrigo Vanegas, and Ken R. Anderson. QoS Aspect Languages and Their Runtime Integration. [SS94] Douglas C. Schmidt and Tatsuya Suda. An Object-Oriented Framework for Dynamically Springer-Verlag, 1998. Configuring Extensible Distributed Communi[LN00] Baochun Li and Clara Nahrstedt. QualProbes: cation Systems. IEE/BCS Distributed Systems Middleware QoS Profiling Services for ConfigEngineering Journal (Special Issue on Configuring Adaptive Applications. Springer-Verlag, urable Distributed Systems), 2:280–293, De2000. cember 1994. 19
[SZK+ 99] Richard E. Schantz, John A. Zinky, David A. Karr, David E. Bakken, James Megquier, and Joseph P. Loyall. An object-level gateway supporting integrated-property quality of service. In Proceedings of the 2nd IEEE International Symposium on Object-oriented Real-time distributed Computing (ISORC), May 1999. [VZL+ 98] Rodrigo Vanegas, John A. Zinky, Joseph P. Loyall, David Karr, Richard E. Schantz, and David E. Bakken. QuO’s runtime support for quality of service in distributed objects. Proceedings of Middleware 98, the IFIP International Conference on Distributed Systems Platform and Open Distributed Processing, September 1998.
20
