Adaptive QoS Framework for Multiview 3D Streaming Jin Ryong Kim1 , Youjip Won2 , and Yuichi Iwadate3 1
3
Digital Contents Research Division, Electronics and Telecommunications Research Institute, Daejeon, Korea
[email protected] 2 Div. of Electrical and Computer Engineering, Hanyang University, Seoul, Korea
[email protected] NHK Science & Technical Research Laboratories, Japan Broadcasting Corporation, Tokyo, Japan
[email protected]
Abstract. We present the adaptive QoS framework for multi-view 3D streaming to deliver the media in time and at the same time, we provide an optimal solution to minimize the quality variation. We dynamically adjust the number of polygons in 3D model so that it can support constant frame rate. We also propose to minimize the frequencies in QoS transition to provide better user perceptive streaming. As a result, the stable frame transmission rate is guaranteed and the quality fluctuation becomes smoother. Keywords: QoS, Virtual Studio, Interactive TV, Multimedia Streaming, 3D Streaming
1
Introduction
When 3D media move downstream to viewers, transmission rate in the network will be varied depending on the network traffic condition. We propose an adaptive QoS management scheme to efficiently stream time-critical media and optimal quality adaptation scheduling algorithm for multiview 3D streaming. NHK[3] developed a 3D model generation system using multiple cameras with multi-baseline stereo algorithm and the volume intersection method. It is designed to generate 3D model media contents and support any viewpoint. Salehi[5] proposed an optimal rate smoothing algorithm based on the traffic smoothing technique to achieve minimum variability of the transmission rate. Cuetos et al.[2] proposed to find a shortest path to minimize variability. Nelakuditi et al.[4] accomplished the maximum reduction of quality variability for layered CBR video using bidirectional layer selection. This paper extends NHK’s multiple camera system by developing the QoS architecture to develop 3D streaming system for immersive environment. It is designed for delivering visual contents from the studio to the consumer platform via the Internet.
This work is in part funded by KOSEF through Statistical Research Center for Complex System at Seoul National University.
M. Bubak et al. (Eds.): ICCS 2004, LNCS 3036, pp. 519–522, 2004. c Springer-Verlag Berlin Heidelberg 2004
520
2
J.R. Kim, Y. Won, and Y. Iwadate
Rate Adaptive Transmission
The idea of our scheme is to monitor the network bandwidth availability and send the 3D video having appropriate bit rate. Each time slot represents the time unit for playing a video. Let k be the time slot at tk . C is the number of frames in a time slot and it is set to C=20 for every time slot to have a fixed frame rate for every time slot. We define Wk as a quantitative amount of the available network bandwidth at each time slot k. We assume that the current Wk is known. Given a maximum network bandwidth, Wk is fluctuating over a wide range. The available network bandwidth curve is divided into time slots and quantified under δ ≥ φ for k = 1...N where N is the total number of time slots, φk is inner portion of network bandwidth curve, and δk is outer portion of network bandwidth curve. Wk is an optimal value at each time slot k from the network available bandwidth curve. QoS level at time slot k is determined as follow: if available bandwidth is greater than rE , then QoSk is assigned as ’E’, where QoSk is QoS level at k and rlv is bit rate for QoS level lv. If available bandwidth is between rE and rG , then QoSk is assigned as ’G’. If available bandwidth is between rG and rF , QoSk is assigned as ’F’. If available bandwidth is between rF and rP , then QoSk is assigned as ’P’. If available bandwidth is lower than rP , then QoSk is assigned as ’B’. Qosk of ’E’, ’G’, ’F’, ’P’, and ’B’ can be mapped into quality scale Qk of 5, 4, 3, 2, and 1 for simplicity.
3
Optimizing Quality Variation
Fig. 1(a) illustrates the unstable video transmission if the server transmits only one quality of the video sequence and Fig. 1(b) illustrates how the video sequence is adaptively transmitted under variable network bandwidth using our existing scheme. It also illustrates the quality fluctuation in Fig. 1 (b). Each colored time slot represents the distinct quality. The basic idea of our extended scheme is that instead of changing the quality of the time slot at each point of time, we keep the same quality and raise the quality level at some point. We accomplish this scheme by prefetching some portions of the next time slot as shown in Fig. 1(c). This enables us to maximize the usage of available network bandwidth and minimize the quality variation.
Available Network Bandwidth (Mbits/sec)
Available Network Bandwidth (Mbits/sec)
Available Network Bandwidth (Mbits/sec)
(a)
Time (b)
Time
Fig. 1. Example of quality optimization
(c)
Time
Adaptive QoS Framework for Multiview 3D Streaming
521
We forecast the future network bandwidth availability using double exponential smoothing based predictor (DESP)[1]. Using DESP, the future available network bandwidth can be forecasted as P (k) = αW (k)+(1−α)(W (k−1)+b(k−1)) and b(k) = γ(P (k) − P (k − 1)) + (1 − γ)b(k − 1) where P (k) is smoothed value at k and b(k) is a trend equation. α and γ are smoothing and trend constants, respectively, and α ∈ [0, 1] and γ ∈ [0, 1]. Forecast equation Z(k) is defined as Z(k + u) = P (k) + ub(k) where u denotes u − period ahead forecast. We model the quality adaptation for CBR video by replacing re-scheduled time slots to maintain a uniform quality scale. In formulating optimal quality adaptation, we consider a discrete-time model. We assume that there are 5 seconds of buffers in the client. The time slots are scheduled in the server based on DESP. We set u as 4 to have 4 time slots in the server. We consider the future available network bandwidth trend as a reference quality scale for every point of time. Let i be the index and N be the number of time slots in the server. Then, the mean quality scale of the time slots in the server N +k
Qi
is Qavg = i=k+1 . We define a majority quality scale in the system for N selecting optimal quality scale. Let fR (r, k) denotes a set of frequencies of quality scales where r ∈ {E, G, F, P, B}. Then, the majority quality scale is Qmajority = max{fR (r, k)}. We now introduce our prefetching algorithm. The time slots are divided into layers with the same size and it is represented as granules. When the server transmits the time slot with rescheduled quality scale, some vacant granules become available. Then, we take advantage of using these empty spots to prefetch some granules from the next time slot. We keep track of residual bandwidth RB(k) and it is defined as RB(k) = RB(k − 1) + Wk − rlv . L(k) denotes the depth of layer at k and it is defined as L(k) = rlv l where l denotes the size of granule. e(k) denotes the number of empty granules and it is e(k) = RB(k) . Using L(k) and e(k), we prefetch granules in k + 1 as l L(k+1) P ref etch = j=L(k+1)−e(k)+1 Qjk+1 where j denotes the layer index.
4
Performance Experiment
We examine the effectiveness of rate adaptive transmission. To measure the smoothness of the quality scale, we use average run length(ARL) metric proposed by [4]. ARL is the metric to measure a sequence of consecutive frames in a layer. This metric attempts to measure the smoothness kiin the perceived quality of a nj L 1 where ki is the number layered video. It is defined by ARL = L i=1 j=1 ki of runs in the ith layer, and nj is the length of the jth run. Fig. 2 illustrates the performance of rate adaptation transmission scheme between normal and optimal transmission. Fig. 2(a) illustrates the quality fluctuation with the traffic interferences at 5 Mbps in normal rate adaptive transmission. In Fig. 2(a), the majority quality scale in this figure is ’F’ and some transitions are occurred between ’G’ and ’F’. ARL is 1.263, which is very small run length. As a result, the quality fluctuation and degradation of perceptual quality is highly occurred.
J.R. Kim, Y. Won, and Y. Iwadate
5
5
4
4
3
3
QoS Level
QoS Level
522
2
1
2
1
0
0 0
20 40 60 80 Frame Sequence (5Mbits/sec UDP Traffic)
100
(a) UDP Traffic = 5 Mbits/sec
0
20 40 60 80 Frame Sequence (5Mbits/sec UDP Traffic)
100
(b) UDP Traffic = 5 Mbits/sec
Fig. 2. Performance of Normal and Optimal Rate Adaptive Transmission
Fig. 2(b) is the performance of optimal rate transmission with the traffic interferences at 5 Mbps. In Fig. 2(b), ARL is 14.286. Note that ARL is longer in optimal rate transmission than normal rate adaptation scheme. It is also noticed that only two quality scales are used and an extremely smoothed result is accomplished. Overall, the results show that the optimal rate transmission algorithm exhibits smoother quality fluctuation compared to the normal rate adaptive transmission scheme.
5
Conclusion
In this paper, we proposed adaptive QoS management to reduce time delay and guarantee the constant playback rate in delivering high-quality, 3D contents. We also provided optimal quality adaptation scheme to minimize the quality fluctuation. Our results show that proposed QoS architecture can effectively utilize the available network bandwidth and minimize the quality variation.
References 1. C. Chatfield. Time-series forecasting. Chapman and Hall/CRC, 2001. 2. P. de Cuetos and K. Ross. Adaptive rate control for streaming stored fine grained scalable video. In Proceedings of NOSSDAV 2002. ACM, May 2002. 3. Y. Iwadate, M. Katayama, K. Tomiyama, and H. Imaizumi. Vrml animation from multi-view images. In ICME2002 IEEE International Conference on Multimedia and Expo, pages 881–884. IEEE, August 2002. 4. S. Nelakuditi, R. Harinath, E. Kusmierek, and Z.-L. Zhang. Providing smoother quality layered video stream. In Proceedings of NOSSDAV 2000. ACM, June 2000. 5. J. Salehi, Z.-L. Zhang, J. Kurose, and D. Towsley. Supporting stored video: Reducing rate variability and end-to-end resource requirements through optimal smoothing. IEEE/ACM Trans. Networking, 6(4):397–410, August 1998.
