network congestion and achieves higher visual quality compared to non-adaptive .... We define the total number of packet
➠
➡ NETWORK-ADAPTIVE VIDEO STREAMING USING MULTIPLE DESCRIPTION CODING AND PATH DIVERSITY Joohee Kim, Russell M. Mersereau, and Yucel Altunbasak School of Electrical and Computer Engineering Georgia Institute of Technology Atlanta, GA 30332 ABSTRACT This paper presents a TCP-friendly rate allocation algorithm for multiple description coding combined with path diversity. A 3D SPIHT-based multiple description coding is used to generate two independent substreams and each substream is protected with an FEC-based unequal error protection algorithm. In the proposed video streaming framework, two senders simultaneously stream complementary substreams to the receiver over different paths. The proposed rate allocation algorithm coordinates distributed video streaming from multiple senders to a single receiver to minimize the overall distortion. Simulation results show that the proposed algorithm increases tolerance to packet loss due to network congestion and achieves higher visual quality compared to non-adaptive methods. 1. INTRODUCTION In order for video streaming to become widespread over errorprone networks, there are many challenging technical problems that need to be solved. One of the challenges is that most current video coding algorithms were developed for relatively benign, nearly error-free environments, yet wireless channels can have high error rates and highly variable capacity. Furthermore, video applications are highly sensitive to end-to-end delay, and this stringent delay requirement constrains any rate control and error control strategy. To provide a reasonable quality of service at high loss rates, it is important that the source coders be both errorresilient and network-adaptive. The set partitioning in hierarchical trees (SPIHT) [1] algorithm generates an embedded (or progressive) bitstream for either images or video that is convenient for bit-rate adaptation in timevarying network conditions, e.g., wireless networks. Since the bitstream is progressive, rate changes can be accommodated easily and instantaneously by dropping the appropriate packets either at the source or at the intermediate nodes. However, SPIHT-coded bitstreams are very sensitive to data losses because of the dependence among wavelet coefficients in constructing a significance map. The most common way to protect data from packet losses is to request retransmission of any lost packets to the sender. However, this may not be acceptable for video streaming because of the delay constraints imposed by the need for continuous playback at the receiver. Instead, forward error correction (FEC) is a more This work was supported in part by the National Science Foundation (NSF) under grant SPS-0105654 and the State of Georgia under Yamacraw Initiative.
0-7803-7965-9/03/$17.00 ©2003 IEEE
appropriate error control method. Furthermore, progressive bitstreams provide a natural basis for unequal error protection (UEP), by which perceptually more important parts of the bitstream can be assigned a greater level of error protection. As a result, a number of researchers have identified FEC-based unequal error protection methods as promising error control solutions for video streaming over unreliable channels [2, 3]. Recently, several methods that divide the original bitstream into multiple correlated substreams, which are transmitted independently, have been proposed for wavelet-based coders to get additional error-resilience at high loss rates [4, 5]. The advantage of this multiple-substream approach is that errors in one substream do not affect the others because each substream is independently decodable. This provides some robustness against packet losses. The multiple-substream approach is similar in spirit to multiple description coding (MDC). The principle behind MDC is to generate multiple correlated descriptions of the source such that each description approximates the source with a certain level of fidelity. When more than one description is received, they can be combined to further enhance the reconstructed image quality. The benefits of using a multiple-substream approach in video streaming can be further amplified when the multiple-substream approach is combined with path diversity [6]. In this approach, each substream is explicitly transmitted over an independent path to a client. Path diversity exploits the fact that the probability of having all the paths simultaneously congested is relatively low. As a result, multiple-substream coding combined with path diversity can achieve higher throughput and increase tolerance to packet loss and delay due to network congestion. Path diversity was also used in distributed video streaming frameworks for streaming video from multiple senders simultaneously to a single receiver [7, 8]. In [7], a rate allocation algorithm for distributed video streaming in which two senders stream complementary packets from a single description coded bitstream to a single receiver was proposed. In this scheme, all packets are protected by fixed FEC codes and the rate allocation algorithm adjusts the transmission rates of all senders in order to minimize the probability of lost packets. In this paper, we propose a rate allocation algorithm for multiple-substream coding combined with path diversity. We assume a distributed video streaming framework in which two senders simultaneously stream complementary substreams to a single receiver. Since each substream is transmitted over a different path to the receiver, each sender will have different amount of available bandwidth and experience different degree of packet loss. The proposed algorithm estimates the available bandwidth and the packet loss rates for both senders and adjusts the transmission rates and the amount of channel coding for both senders in a coordinated
II - 653
ICME 2003
➡
➡ S1 substream−1 3−D SPIHT
str−1
R S2 substream−2 1 GOP (after wavelet decomposition)
3−D SPIHT
Fig. 2. A distributed video streaming framework in which two senders simultaneously stream complementary substreams to a single receiver.
str−2
Table 1. Notations used to formulate the proposed rate allocation ( algorithm.
Fig. 1. Frequency-domain partitioning.
(
fashion to minimize the overall distortion at the receiver. Simulation results show that the proposed algorithm increases tolerance to packet loss due to network congestion and achieves higher visual quality compared to non-adaptive methods. 2. RATE CONTROL FOR DISTRIBUTED MULTIPLE-SUBSTREAM VIDEO STREAMING
In this paper, a 3-D SPIHT-based multiple-substream coding is used to generate two independent substreams. After a three-level wavelet decomposition, the 3-D wavelet coefficients in a GOP (a group of frames) are divided into two groups according to their spatial and temporal relationships as in Fig. 1 and each group is independently encoded using a 3-D SPIHT algorithm [9]. Then, each substream is protected with an FEC-based unequal error pro- tection algorithm proposed in [3]. As shown in Fig. 2, senderand sender- simultaneously transmit substream- and substream , respectively, to the receiver. Since each substream is transmitted over a different path to the receiver, each sender will have different amount of available bandwidth and experience different degree of packet loss. In addition, each sender should share the available bandwidth fairly with other competing TCP connections. To minimize the overall distortion under time-varying network conditions for distributed multiplesubstream video streaming, we propose a rate allocation algorithm that estimates the available bandwidth and the packet loss rates for two senders and adjusts the transmission rates and the amount of channel coding for both senders in a coordinated fashion. 2.2. Bandwidth Estimation In our video streaming framework, the receiver estimates the available bandwidth for each sender based on the TCP-friendly rate control (TFRC) algorithm proposed in [10]. Each sender estimates and sends its estimated round-trip time (RTT) to the receiver. The receiver estimates the loss event rate of each sender and calculates the available bandwidth using the estimated RTT and loss rate according to the following equation for sender- :
"! #% $ ! '$ &
Total number of packets to be sent for all senders Number of packets in an FEC block for sender- Number of data packets in bit-plane , for sender- The last bit-plane to be sent for sender- Estimated TCP-friendly bandwidth for sender- in packets per GOP period
2.1. Multiple-Substream Video Coding and Path Diversity
)+* - . /
(1)
where
is the packet size, is the round-trip time for , and sender- , is the steady-state loss event rate for sender is the TCP retransmit timeout value for sender- . 0 gives an upper bound on the transmission rate for sender- in bytes/sec. The receiver calculates the transmission rates for both senders and sends identical control packets to each sender that contain the time stamp when the transmission rate is calculated, the estimated loss rates for both senders, and the TCP-friendly transmission rates as calculated by the receiver for both senders using (1). 2.3. Rate Allocation Algorithm The proposed rate allocation ( algorithm is performed on a GOP basis. We define the total number of packets to be sent from all senders for a GOP period, , as
(
6AG ) 21'3'4 75 6
98;? :=576A B5 6
H (2) DCFE & 4 5B6 (bytes/sec) is the total source coding rate for where the combination of data and redundancy for all senders and
is the number of frames in a GOP. In our video stream:= ing framework, are two senders and sender- ( we assume (JI that there ( transmits substream- to the receiver. ( Then, the proposed algo$ and $ so as to minimize the overrithm divides into $ represents all distortion at the receiver, where the number ( of packets transmitted by sender- at GOP period for . & $ should satTo be fair with the competing TCP connections, isfy the following condition:
(
$LK 0
1'3'
6+G ) $ 8 ? :=5B6A 75
6
H DCFE & (
(3)
$ is the TCP-friendly estimated bandwidth where for sender $ at GOP period calculated by (1) for . Given and the estimated loss rate for sender- , , the& optimal amount of redundancy is applied to each bit-plane for substream- using
II - 654
➡
➡ c BP−1
BP−2
BP−3 c
k c k BP−Is k s1
ls ls2 ls3 1
packet
1
s
VS1
sender−1 stream−1
FEC
L (Packet Size)
3
s
. . .
2
45Mbps, 45ms TCP1
stream−2
...
. . .
1
TCPSINK
sender−2
s
s
ls
VS2
Is
Is
Is
TCP20
...
3
45Mbps, 40ms
receiver
FEC
s
...
TCP1
FEC
TCP15
Fig. 4. Simulation topology.
Ns
Packet Number
Fig. 3.. Multiple-substream bit-plane-wise unequal error protec is the size of bit-plane for substream- in bytes. The tion. ( ) bits belonging ) are filled into packets and the reG to bit-plane packets are filled with channel coding maining redundancy.
a bit-plane-wise UEP algorithm proposed in [3]. Fig. 3 shows an example of applying this UEP ( pacektization to substream) . In this are filled into packets scheme, the bits belonging to bit-plane
( G ) and the remaining packets are filled with channel coding redundancy, where is the total ( number of packets to be sent for a GOP (Fig. 3). In other words, ) the $ . source data for bit-plane is protected by an FEC code & Now, we can( state our rate allocation algorithm as follows ( (The notation used inI our rate allocation algorithm is defined in Ta $ $ ble 1.): Given and , the proposed ) $ $ algorithm finds $ ) $ ) $ and for that mini& & mize the overall distortion at& the receiver given by: &
! # "%$&('*) I 0/ ( $ $213 + #",$ .& - > & *
I I $ *
subject (JI to
(
(
$ (
0/
$
(
(JI
$ K .
&
I
/
( $ &
$K . $
(4)
&
(5)
$ is the probability that packets are lost out of where > packets& sent by sender- and
$ is the distortion when the receiver decodes up to bit-plane for substream- . We use the two-state Markov model to simulate the bursty packet loss behav@ ior. Then, the probability of lost packets within the block of packets is given by [11]: I
5476
8 ",: 9 I-
@ = $ & = $ & (6) = is a gap length where is the average packet loss probability, > ; (the event that after a lost packet = packets are received and then a packet greater @ : >=$ $ is the probability of@ a gap , andis lost), length than = is the probability of packet & packets following a lost packet. losses within the next >
@
&
$
( I (
packets to transmit for a GOP period. The proposed algorithm tries ) that satisfies the constrains all possible combinations of ( & in (5) and chooses the one that minimizes the overall distortion de( fined in (4). The proposed algorithm is fast enough to be used for real-time applications, since the number of comparisons is only for two senders.
>
Since the receiver sends identical control packets to each sender that contain the time stamp, the estimated loss rates, and the TCPfriendly transmission rates, each sender independently runs the proposed rate allocation algorithm to get its optimal number of
3. SIMULATION RESULTS In this section, we present the simulation results of the proposed rate allocation algorithm for multiple-substream coding combined with path diversity using the ns-2 network simulator [12]. For these experiments, we used the ! 8= M OTHER AND DAUGH TER video sequence. We encoded the video sequence using the 3-D SPIHT-based multiple-substream coding [3] with GOP size of 16 frames and generated two independent substreams using frequency-domain partitioning. We assume that sender- and sender- simultaneously stream stream- and stream- , respectively, to the receiver over different paths. Fig. 4 shows the network topology used in our simulation. At time , 20 TCP sources that share the same path with sender- (VS1) and 15 TCP sources that share the same path with sender- (VS2) start transmitting data. At , 10 new TCP sources that share the same route with senderstart transmitting data and stop at . Figs. 5(a) and (b) show the estimated TCP-friendly bandwidth and the estimated loss rate of each path, respectively. Fig. 5(c) shows the optimal number of packets that are sent from each sender obtained by the proposed rate allocation algorithm. As shown in Fig. 5(c), our algorithm adjusts the transmission rates for both senders according to time-varying network conditions. Figs. 5(d) compares the PSNR (Peak Signal to Noise Ratio) at the receiver relative to the original video sequence between ADAPTIVE and NON-ADAPTIVE methods. In ADAPTIVE method, both senders adjust their transmission rates and the amount of channel coding using our proposed rate allocation algorithm in order to be TCP-friendly and to minimize the overall distortion at the receiver. However, in NON-ADAPTIVE method, both senders use fixed amount of channel coding and do not adjust their transmission rates according to time-varying network conditions. Thus, they are not TCP-friendly since sometimes they transmit more than the average bandwidth of other TCP sources that share the same path. As seen from Fig. 5(d), ADAPTIVE method achieves higher and less variable visual quality than NONADAPTIVE method. The quality drops drastically when the number of lost packets exceeds the error correction capability of the fixed FEC codes in NON-ADAPTIVE method.
II - 655
@? BA@C
C
?C
CDC
➡
➠ 4. CONCLUSIONS In this paper, we present a TCP-friendly rate allocation algorithm for multiple-substream coding combined with path diversity. In our work, multiple senders simultaneously stream complementary substreams to the receiver over different paths. The proposed algorithm estimates the available bandwidth and the packet loss rate for each sender and adjusts the transmission rate and the amount of channel coding for all senders in a coordinated fashion to minimize the overall distortion at the receiver. Simulation results show that the proposed algorithm increases tolerance to packet loss due to network congestion and achieves higher visual quality compared to non-adaptive methods.
1200 Transmission Rate (sender-1) Transmission Rate (sender-2)
TX Rate (Kbits/s)
1000
800
600
400
200
0 60
80
100
120
140
160
180
5. REFERENCES
GOP Number
(a) 0.05 Estimated loss rate (sender-1) Estimated loss rate (sender-2)
Loss Rate
0.04
0.03
0.02
0.01
0 60
80
100
120
140
160
180
GOP Number
(b) 200
Transmitted Packets per GOP
N1 (sender-1) N2 (sender-2) N (combined) 150
100
50
0 60
80
100
120
140
160
180
GOP Number
(c) Expected PSNR (Adaptive) Expected PSNR (Non-adaptive)
42
Expected PSNR (dB)
40 38 36 34 32 30 28 26 60
80
100
120
140
160
180
200
GOP Number
(d) Fig. 5. (a) TCP-friendly transmission rate of two senders, (b) Estimated loss rate of two senders, (c) Optimal number of packets to be sent from each sender in ADAPTIVE method, and (d) PSNR comparison between ADAPTIVE and NON-ADAPTIVE method.
[1] A. Said and W. Pearlman, “A new, fast, and efficient image codec based on set partitioning in hierarchical trees,” IEEE Trans. on Circuits Syst. Video Technol., vol. 6, pp. 243–250, June 1996. [2] A. Mohr, E. Riskin, and R. Ladner, “Unequal loss protection: graceful degradation of image quality over packet erasure channel through forward error correction,” IEEE Journal on Sel. Areas in Commun., vol. 18, no. 6, pp. 818–828, June 2000. [3] J. Kim, R. M. Mersereau, and Y. Altunbasak, “An integrated multiple-substream unequal error protection and error concealment algorithm for internet video applications,” in Proceedings of ICIP ’02, Sept. 2002. [4] C. Creusere, “A new method of robust image compression based on the embedded zerotree wavelet algorithm,” IEEE Trans. on Image Processing, vol. 6, no. 10, pp. 1436–1442, Oct. 1997. [5] S. Cho and W. Pearlman, “Error resilient compression and transmission of scalable video,” in Proceedings of SPIE, Applications of Digital Processing, vol. 4115, pp. 396–405, 2000. [6] J. G. Apostolopoulos, “On multiple description streaming with content delivery networks,” in Proceedings of INFOCOM ’02, July 2002. [7] T. Nguyen and Z. Zakhor, “Distributed video streaming with forward error correction,” in Proceedings of Packet Video Workshop ’02, Apr. 2002. [8] A. Majumdar, R. Puri, and K. Ramchandran, “Distributed multimedia transmission from multiple servers,” in Proceedings of ICIP ’02, Sept. 2002. [9] B. J. Kim, Z. Xiong, and W. A. Pearlman, “Low bit-rate scalable video coding with 3-d set partitioning in hierarchical trees (3-d spiht),” IEEE Trans. on Circuits Syst. Video Technol., vol. 10, no. 8, pp. 1374–1387, Dec. 2000. [10] S. Floyd, M. Handley, J. Padhye, and J. Widmer, “Equationbased congestion control for unicast applications,” in Proceedings of SIGCOMM ’00, Aug. 2000. [11] U. Horn, K. Stuhlmuller, M. Link, and B. Girod, “Robust internet video transmission based on scalable coding and unequal error protection,” Image Communication, special issue on real-time video over the Internet, vol. 15, pp. 77–94, Sept. 1999. [12] S. McCanne and S. Floyd, “The lbl/ucb network simulator,” newblock Lawrence Berkeley Laboratory, University of California , Berkeley (http://www.isi.edu/nsnam/ns/).
II - 656
