Dynamic Network Adaptation Framework employing Layered Relative Priority Index for Adaptive Video Delivery JongWon Kim1 and Jitae Shin2 1
2
Networked Media Lab. Department of Information and Communications, Kwang-Ju Institute of Science and Technology (K-JIST), Gwangju, 500-712, Korea,
[email protected] School of Information and Communication Engineering, Sungkyunkwan University, Suwon, 440-746, Korea,
[email protected]
Abstract. Delivering high-quality media contents robustly over a wide range of network conditions of the wired/wireless Internet is a highly challenging task. To address this challenging task, the continuous media applications at the edge of network have become more and more adaptive while the best-effort Internet is slowly progressing towards improved networking services. Thus, the role of network adaptation, which dynamically links the quality demand of application contents to the underlying networking services, has become more crucial. In this paper, we present a novel network adaptation approach that efficiently leverages its awareness of both media contents and underlying networks. Detailed discussion on the framework will be covered with deployment scenarios such as the adaptive coordination of source-channel error control for wireless video and the mapping of prioritized video packet to the available network QoS (quality of service) services.
1
Introduction
Realization of high-quality media streaming over the IP Internet faces lots of challenges [1]. Streaming media applications in general have very strict requirements on the network service, thus making the current best-effort (BE) Internet model less than sufficient. They require stable networks and systems, feasible signaling/transport protocols, and network-adaptive applications to achieve acceptable-quality media distribution. From the network side, upcoming QoS (quality of service) networks can alleviates several complications of current besteffort Internet. That is, networks are slowly evolving toward improved QoS services to guarantee loss, delay, and throughput. Enhanced systems are also emerging to support reliable media streaming better. From the protocol side, real-time transport protocol pair (RTP/RTCP) can provide inter-operable/monitored realtime transport channel over the IP network. RTP/RTCP in itself does not guarantee QoS to the streaming applications but acts as a supporter. The key for the successful media streaming, however, is still with the applications at the edge. Recent years the streaming media applications have become more and more
2
JongWon Kim and Jitae Shin
adaptive to address the limitation of best-effort Internet [2, 3]. Media applications at the server and client systems are required to response to the dynamic fluctuation of underlying networks. Either in a proactive or in a reactive manner, they are controlling sending rate, applying different error controls, and adjusting end-to-end latency. With this network adaptation, they are satisfying the requirements of both application itself and involved systems in face of diverse network fluctuations and heterogeneities. The role of network adaptation is to link the quality demand of application contents to the underlying networking services. For a successful network adaptation, it is highly important to leverage the awareness of both media contents and underlying networks. In our opinion, well-established prioritization (or layering in coarse adaptation case) can play an important role for the efficient network adaptation. That is, via the prioritized media contents, coordination between different priority packets and network service levels can be established efficiently. In this paper, with the layered-RPI (relative priority index), coordinated delivery of packetized video over the given wireless/QoS-enabled networks is investigated under the dynamic network adaptation framework. The proposed framework includes the following components: 1) relative priority generation based on the so-called corruption model and indexing/categorization of streaming video content at the sender, 2) available network adaptation tools to match the fluctuation of the underlying networks, and 3) forward/backward interaction mechanisms assisting the dynamic network adaptation. We introduce two kinds of deployment cases for the proposed framework that focus on the packet-level unequal error protection (UEP): the adaptive coordination of source-channel error control for wireless video [4] and the mapping of prioritized packet to the available QoS service of the differentiated service (DiffServ) network [5].
2
Dynamic Network Adaptation Framework
The proposed dynamic network adaptation framework for the packetized video delivery is illustrated in Fig. 1. The video contents are first pre-processed and layer-encoded. Following the target (albeit assumed) constraints on the bandwidth and buffer, constant quality rate control manipulates the rate composition among the base and enhancement layers in the layered encoding. At the same time they are analyzed with R-D (rate-distortion) and corruption model [6]. Then the encoded video stream with the associated R-D/corruption model parameters is passed to the network adaptation/prioritized packetization module to wait for the delivery. In this module, the encoded stream is first tailored (or transcoded) in the source rate/error-resilience sense to match the given estimated available bandwidth/loss/delay of the underlying network. Then it is packetized with priority (i.e., the layered RPI) before going through the network adaptation at the sending application. Based only on the priority, they are adapted to the network condition in rate/loss/delay sense. That is, the packets are selectively discarded and protected with differentiation. Note that various types of feedbacks are available to guide the required network adaptation. Guided by both
Lecture Notes in Computer Science
3
end-to-end network feedback (e.g., end-to-end congestion control3 ) and network feedback (e.g., direct congestion indication), the sender application may control the network adaptation. Application-level feedback can also notify about the playout status at the corresponding party and may request the speedup of slowdown of transmission for synchronized playback. Once sent to the network, it may go through network-initiated adaptation, which is also called as network filtering (e.g., with schemes such as priority-based packet dropping and receiverbased layer selection). Finally, the delivered packets are adaptively processed at the receiver to match the receiver capability and user preference.
Layered Video Encoding
Network Adaptation & Prioritized Packetization
R-D / Corruption Model
Video Preprocessing
Frame Complexity (Quality Constraint)
Layered encoding
Constant Quality Rate Control
R-D Analysis Corrupti on Analysis
Target Minimum Bandwidth (Network Constraint) Target Minimum Buffer Size (Receiver Constraint)
Layered RPI
Network Adaptation (Source Rate/Error Resilience) (only if applicable)
Estimated Available Bandwidth, Loss/Delay
Wired/Wireless Networks
Network Adaptation Prioritized Packetization
(Network Rate/Loss/ Delay)
Network Adaptation Prioritized Layered Packets
Network Feedback
Network Monitoring & Feedback Handling
Receiver/ User
(Network Filtering)
Network Adaptation
Re c Se eive (Receiver / lec r tio User n Adaptation)
Network Feedback (end-to-end) Application Feedback
Fig. 1. Packetized video delivery with the dynamic network adaptation.
The proposed network adaptation framework basically assumes the existence of a network or other equivalent options that support prioritized variable-rate delivery of video stream and the associated end-to-end performance and cost (e.g., pricing) models. Since a video codec has several options to trade the compression efficiency for flexible delay manipulation, error resilience, and network friendliness, the coordination (i.e., network adaptation) framework has to provide an simplified interaction process between the video application and the target network. Note that the interaction is taking place at multiple junctions as the media stream is delivered from the sending application, via the underlying network, and to the receiving application. Note that the main purpose of introducing the layered-RPI is to abstract and isolate the coding details from the network adaptation task. By assigning layered-RPI to each video stream in an appropriate manner, the proposed framework can accommodate the demand of each stream to achieve the best end-to-end performance in adapting to the fluctuating networks. Given the prioritization of video stream, the proposed network adaptation can be controlled in both feedforward and feedback sense. 3
TCP-friendly congestion control is required in order to avoid network congestion collapse and share network resources evenly with TCP traffic.
4
JongWon Kim and Jitae Shin
They need to accommodate the fluctuation of the given network in addition to the inherent variability of video stream and receiver/user heterogeneity. Thus, the adaptation should focus on how to dynamically react to sudden changes in the application and network. 2.1
Required Source Prioritization and Layered-RPI
For the streaming video applications, the layered-RPI assignment should reflect the influence of each stream (or down to packet) to the end-to-end quality. Packets will be marked by the content-aware applications in the granularity of session, flow, layer, and/or packet. Among three key parameters for QoS (rate, error, and delay), it is important to associate priority for loss and delay, respectively. Note that the rate (or bandwidth) is linked with the layering itself and the extreme case of this is MPEG-4 FGS (fine-grained scalability) layering. Most of existing prioritization schemes are in coarse granularities of session, flow, and layer. The per-flow prioritization is promoted under the name of userbased allocation within access networks [7]. Also, lots of prioritization for the UEP is best matched with layer-based differentiation as done in [8] with objectbased scalability. For delay, the session-based granularity to account for the delay effect of the source seems a first choice. Since the video application context (e.g., video-conferencing vs video e-mail) plays a crucial role in delay prioritization, RDI (relative delay index) is kept constant for whole session (e.g., session (A) and (B) in Fig. 2).
el a I(D RD
sit en yS
Adjust mapping level
ty) ivi
pricing
mechanism
X
(A) Categorizer (B)
Vide
o P ac
Sample equivalent-cost line from ISP
ket N umbe rs
Video source: RPI
QoS Mapping
RPI → k k→q (at end-system) (at VG with SLA)
Relative Delay
RLI(Loss Sensitivity)
Feedback from receiver
X X X
X
X X
X X
X
BE AFn
AF2
AF1
Relative Loss Rate
Network DS Level: q
Fig. 2. Source prioritization using RPI and network adaptation (or QoS mapping) for loss and delay.
However, the prioritization can be differentiated down to each packet to enable a fine-grained differentiation. Packet-based prioritization may be adopted to accurately account for the impact of each packet to the end-to-end quality. Especially the loss impact, quantified by RLI (relative loss index), is dependent on the employed video coding scheme. In this work, we consider a motion-
Lecture Notes in Computer Science
5
compensated prediction-based video compression scheme of MPEG-4 and H.263. At the packet-level, there are dependency relations such as semantic and prediction dependency. The semantic packet-level dependency exists between a packet that includes header parameters and other dependent packets that needs them for decoding. Also, linked by spatial or temporal prediction, the corruption caused by a packet loss can affect the decoding of following packets. This is called the prediction packet-level dependency and we have developed a corruption model to quantify this dependency (i.e., loss impact of a packet) [6]. With the proposed corruption model, the loss impact of each macroblock is explained by taking into account the error concealment, the temporal dependency, and the loop filtering effect. The corruption of macroblocks in a packet is then merged to explain the RLI. For more detailed discussion of this topic, we refer to [6].
2.2
Required Behavior of Network Adaptation
Basically, well-implemented network adaptation can bring benefit to both applications at end and networks by providing better service match at the willingness to pay more complexity. For the network-adaptive applications, the dynamic network adaptation should consider interests of both. That is, the application should get benefit from its network adaptation capability while the network enjoys the benefit of different charging and maximizes the end-user satisfaction. Under a given cost constraint, an efficient network adaptation is trying to match the layered-RPI to the underlying network service level. In this process, the adaptation granularity has to be manipulated intelligibly. The effectiveness is dependent on the accurate association of layered-RPI to each packet, which is one of key investigation issues. In general, the issue of solving all these network adaptation for rate/loss/delay at the same time is too challenging to be solved simultaneously. Thus, depending on the situation, one may focus on the error and delay control separating the rate control issue. For example, one can simply enforce maximum to the allowed transmitting rate by token bucket (TB) policing in either per-flow or aggregate-flow sense. Then, for QoS 2-tuple {delay, loss}, which is actually the major concern of the proposed layered-RPI, appropriate network adaptation is requested in different degrees by user applications, anticipating different levels of guarantee (or assurance) according to the price paid. That is, each application will demand its loss rate/delay preference by marking the layered-RPI, which is further divided into loss and delay part, respectively. In the proposed network adaptation, each video stream can demand different loss and delay treatment as shown in the left side of Fig. 2. Underlying network will meet these demands with its service provisioning capability. It may provide several differentiated delay and loss levels as shown in the right side of Fig. 2 like the DiffServ network [9]. Or similarly we can envision the effect of FEC/ARQ error controls for the above differentiation.
6
3
JongWon Kim and Jitae Shin
Network Adaptation at End-systems: Channel-adaptive Protection for Wireless Video
Priority Assignment RPI calculation Channel Condition Different size GOB
Joint source channel design for packetizing video packet for wireless transmission
Wireless channel variation
GOB packet Channel packet overhead Header
Syn
RS code ratio
Optimal Allocation rate to RS/RCPC
CF Packetization scheme
RCPC code ratio Variation of size channel packets relying on RPI value and size of GOB,CRC as well as RS parity
GOB packet and CRC with RS parity
RCPC parity
GOB packet and CRC with RS parity
RCPC parity
GOB packet and CRC with RS parity
RCPC parity
Changeable RCPC parity to fit with fixed size of channel packet
Fig. 3. Feedback-based adaptive network adaptation (error control) for wireless video.
Reliable video transmission over fading wireless channels relies heavily on a dynamic and coordinated protection effort in response to time-varying channel and source variations. Several adaptive feedback-based systems for rate adaptation, modulation, power control, handoff management, and diversity antenna have been studied recently. Channel adaptation depends on the accuracy of estimated channel status and the underlying time-scale of the estimate [10]. Given the layered-RPI and the estimated channel condition, a proactive network adaptation can be developed as in [4]. A product FEC, which is composed of both Reed-Solomon (RS) and rate-compatible punctured convolutional (RCPC) error correcting codes, is utilized to protect video packets of varying priorities and sizes as well as under instantaneously varying channel. Based on the layeredRPI and the size of each video packet along with the feedback channel state information, network adaptation by the UEP is conducted with the RS/RCPC product code. The resulting channel packet is modulated and transmitted over the underlying wireless channel. At the receiver, the received signal is decoded by the maximum likelihood-based scheme and the long-term fading parameter is estimated. The calculated fading parameter is then fed back via a reliable channel to the sender. To achieve realistic joint source-channel adaptation, special attention will be paid to the channel status feedback in terms of accuracy and delay, the product code tradeoff, and the involved packetization efficiency. As depicted in Fig. 3, source video packets of a variable size from a group of video frames are re-organized into a group of channel packets of a fixed size, which is called the channel slot. Since each channel slot consists of fixed-size channel packets, the payload and the redundancy should be negotiated to maintain a fixed total amount. To be more specific, the level of protection is chosen to give the maximum protection to the video packet stream with both bandwidth and packetization constraints. In order to provide the best network adaptation under
Lecture Notes in Computer Science
7
these constraints, a solution based on dynamic programming is devised and its gain is explored. Please refer [4] for simulation results.
4
Network Adaptation at Networks: QoS-mapping for Video over DiffServ Network
FEEDBACK CONTROL (adjust RPI® k mapping)
RPI from application ’s ®K priority then thenRPI RPI® k
IN
Customer Network
Video Gateway
DiffServ Domain
IN
IN IN
Pricing SLA mechanism TB Packets with k Rate-shaper or dropper
TB
Per-flow
Per-class
TM Class-queues
TM
Packet forwarding mechanism
TM
Per-flowQoS mapping (k à q )
Class-based re-mapping
FEEDFORWARD CONTROL
Legend RPI : relative priority index IN : interior node SLA: service level agreement TB: token bucket TM: TB-based marker
Fig. 4. Feedback and feedforward QoS mapping control for video over DiffServ.
The emerging DiffServ scheme in IP-QoS methods enables to provide service differentiation in a simple and scalable manner. Especially, the relative DiffServ architecture [11] is an attractive approach that does not require admission control, resource reservations, or signaling. It can provide higher classes receive better services (e.g., lower delay/jitter and lower loss rate) than lower classes. Our approach focuses on the QoS mapping between network-aware streaming applications having k priority categories and q DS levels to achieve efficient network adaptation. We designate the term ‘video gateway (VG)’ for the traffic conditioning entity at boundary and the VG is responsible for realizing the specialized traffic managements. The QoS mapping covers both layered-RPI prioritization and feedforward/feedback QoS mapping. Each packet is categorized into layer k by RPI at end-systems, without knowing about other applications. An assigned RDI limits the range/ of k → q mapping level to meet a certain statistical delay range. We can extract an effective mapping set (k → q) under total cost constraint in flow/packet granularity [5]. The traffic conditioning is performed via TB-based re-marking by degrading k → q mapping level when a flow traffic volume exceed allowed bandwidth in the feedforward control. Feedback-based network adaptation enable the fine-tuned refinement on top of coarse feedforward mapping. Receiver sends a report of delay/packet loss to sender whenever necessary and can ask the adjustment of QoS mapping when it is not satisfied with the received quality or the current quality demands too much price. This feedback mechanism enables the whole network adaptation to be adjusted dy-
8
JongWon Kim and Jitae Shin
namically and to stabilize the end-to-end QoS within an acceptable range. Please refer [5] for the performance results.
5
Conclusion
We proposed the dynamic network adaptation between categorized packet video and underlying networks. The interaction can be categorized in a variety of ways: feedforward/feedback control and prioritization granularity. We believe this framework can give insight to the desired interaction between the prioritized video applications and the underlying networks.
Acknowledgement This work is funded in part by the University Research Program supported by Ministry of Information & Communication in Republic of Korea.
References 1. D. Wu et al., “Streaming video over the internet: approaches and directions,” IEEE Trans. on Circuits and Systems for Video Technology, vol. 11, Mar. 2001. 2. R. Rejaie, M. Handley, and D. Estrin, “Quality adaptation for unicast audio and video,” in Proc. ACM SIGCOMM, 1999. 3. J. Kim et al., “TCP-friendly internet video streaming employing variable framerate encoding and interpolation,” IEEE Trans. on Circuits and Systems for Video Technology, vol. 10, Oct. 2000. 4. W. Kumwilaisak, J. Kim, and C.-C. J. Kuo, “Reliable wireless video transmission via fading channel estimation and adaptation,” in Proc. IEEE Wireless Communication and Networking Conference (WCNC), 2000. 5. J. Shin, J.-G. Kim, J. Kim, and C.-C. J. Kuo, “Dynamic QoS mapping framework for relative service differentiation-aware video streaming,” European Transaction on Telecommunication, vol. 12, pp. 217–230, May/June 2001. 6. J.-G. Kim, J. Kim, J. Shin, and C.-C. J. Kuo, “Coordinated packet-level protection with a corruption model for robust video transmission,” in Proc. SPIE Visual Communication Image Processing (VCIP), Jan. 2001. 7. A. Banchs and R. Denda, “A scalable share differentiation architecture for elastive and real-time traffic,” in Proceedings of 8th IWQoS, June 2000. 8. Y. T. Hou, D. Wu, B. Li, T. Hamada, I. Ahmad, and H. J. Chao, “A differentiated services architecture for multimedia streaming in next generation Internet,” Computer Networks, vol. 32, pp. 185–209, Feb 2000. 9. S. Blake, D. Black, M. Carlson, E. Davies, Z. Wang, and W. Weiss, “An architecture for differentiated services,” RFC 2475, IETF, Dec. 1998. 10. L. Hanzo, C. H. Wong, and P. Cherriman, “Channel-adaptive wideband wireless video telephony,” IEEE Signal Processing Magazine, vol. 17, July 2000. 11. C. Dovrolis, D. Stiliadis, and P. Ramanathan, “Proportional differentiated services: Delay differentiation and packet scheduling,” in Proc. ACM SIGCOMM, (Boston, MA), September 1999.
