Memory and Disk Bandwidth Management in Video-on-Demand Servers Fu-Ching Wang, Chih-Yuan Cheng, Meng-Huang Lee, Chun-Hung Wen Tzu-How Lin, Szu-Chi Wang, and Yen-Jen Oyang Department of Computer Science and Information Engineering National Taiwan University Taipei, Taiwan, ROC e-mail:
[email protected]
Abstract
In this paper, we propose the so-called generalized relay mechanism to exploit semiconductor memory to upgrade the service capacity of video-on-demand servers. The design of the generalized relay mechanism is aimed at optimizing allocation and tradeo of the memory resource and the disk bandwidth resource. It is expected that exploiting semiconductor memory to upgrade the service capacity of video-on-demand servers will become a favorite approach in the future as advances in semiconductor technologies continue to increase the density of memory chips and cut the costs at a fast pace.
1 Introduction
In recent years, several studies concerning exploiting semiconductor memory to upgrade the data retrieval bandwidth capacity of video-on-demand(VOD) servers have been reported [1, 2]. These studies were motivated by observing continuously rapid increase of memory chip density. Actually, the basic techniques used in the generalized interval caching policy[1] and the relay mechanism[2] are very similar. However, the generalized interval caching policy do not address run-time allocation and tradeo between the memory resource and the disk bandwidth resource. It only deals with allocation of the memory resource among competing streams but does not discuss the situations in which disk bandwidth become limited. The relay mechanism does address run-time allocation and tradeo between the memory resource and the disk bandwidth resource but there is room for further improvement. One of the main de ciency of the relay mechanism proposed in [2] is that it does not take account of the popularity characteristic of video programs when carrying out run-time resource allocation. Our study reveals that this factor plays a signi cant role. Therefore, we proposed a more comprehensive resource allocation algorithm called the generalized relay mechanism. The generalized relay mechanism not only takes account of the popularity characteristic of video programs but also employs a practice that specula-
tively allocates memory. The simulation results reveal that the generalized relay mechanism achieves significant improvement over the original relay mechanism in terms of service capacity of the system.
2 The generalized relay mechanism
In the following discussion, A relay chain is a group of streams that relay data from one to another. A relay chain consists of a leading stream and a number of followers. The leading stream retrieves the desired data from the disk system and the followers are served by the data relayed through memory. A relay chain uses up one unit of disk bandwidth and some amount of memory. The amount of memory used is determined by the elapsing time between the leading stream and the tailing stream of the chain. To facilitate further discussion, the elapsing time is measured in numbers of service cycles and the amount of memory is measured in numbers of bu ers. Figure 1 illustrates the two primitive operations, namely splitting and merging, in the generalized relay mechanism. A splitting operation breaks a relay chain into two and releases a number of bu ers but takes one more unit of disk bandwidth. A merging operation merges two relay chains into one and releases one unit of disk bandwidth but takes a few more bu ers. The release of disk bandwidth due to a merging operation is delayed by a few cycles. In Figure 1, the release of the disk bandwidth originally used by the relay chain containing I3 and I4 will be delayed by 3 service cycles. Figure 2 depicts the 5 data structures of the resource allocation algorithm in the generalized relay mechanism. The functions of these 5 data structures are as follows: 1. The relay pairs list contains all the access instances pairs whose two access instances form two consecutive entities in one of the relay chains. A relay pair is a point where the splitting operation may be applied. The list is sorted in the descending order according to the number of memory bu ers each pair occupies. 2. The candidate pairs list contains access instance
I1 I2 I3 I4 T0
T1
T2
T3
T4
T5
I1 and I2 form a relay chain and I3 and I4 form the second relay chain. In total, this arrangement takes 2 units of disk bandwidth and 2 buffers.
T6
Time, in period of service cycles
Split
I1 I2 I3 I4 T0
T1
T2
T3
T4
T5
Merge
I1, I2, I3 and I4 form a relay chain. This arrangement takes 1 unit of disk bandwidth and 5 buffers.
T6
Time, in period of service cycles
Figure 1: Example of the merging and splitting operations Relay Pairs List Head
1
Head
5 6
Tail
......
Tail
bu ers associated with the chain. 4. The free bu er pool contains unallocated memory bu ers. 5. The free disk bandwidth pool keeps a record of unallocated disk bandwidth. Figure 3 shows the pseudo-code of the resource allocation algorithm in the generalized relay mechanism. This algorithm is executed every service cycle to optimize allocation of the memory resource and the disk bandwidth resource.
4 Free Disk Bandwidth Pool
Candidate Pairs List
2
Free Buffer Pool
......
3
Speculative Relay Chains List Head
......
Tail
Figure 2: The data structures used in the generalized relay mechanism.
3 An evaluation of the generalized relay mechanism
pairs whose two access instances are not included in a relay chain but may be merged into a relay chain if memory bu ers are available. The list is sorted in the ascending order according to the number of memory bu ers each candidate pair requires in case merging is to be applied. The order de nes which pair has a higher priority to be merged if memory is limited. 3. The speculative relay chains list contains all the speculative relay chains. A speculative relay chain is headed by a leading access instance and contains one or more memory bu ers but no following access instance. A newly admitted access instance will head a speculative relay chain if free memory bu ers are available. The data that the leading access instance retrieves from the disks is temporarily stored in the associated memory bu ers with the hope that the data will soon be accessed by a second access instance that starts later. This list is sorted in the ascending order according to the bu er utilization factor of each speculative relay chain. The bu er utilization factor of a speculative relay chain is the popularity of the program that the leading access instance is accessing divided by the number of memory
This section reports an evaluation of the generalized relay mechanism based on simulation runs. The simulation environment assumes a Karaoke system with four 1-GBytes hard disks and one juke-box. The Karaoke programs are compressed by the MPEG-1 standard. The system stores a total of 5000 programs. The most popular 100 programs, which account for 43% of the requests that the system serves, are stored in the hard disks. The remaining 4900 programs are stored in the juke-box. Only the hard disk subsystem contains semiconductor memory for implementing the generalized relay mechanism because the programs stored in the juke-box are less frequently accessed and, therefore, there is no advantage to implement the relay mechanism. The programs are stored in hard disks by a block interleaved scheme [3]. The streams that access the hard disks are divided into four groups depending on their starting times. The four groups of streams access the disk subsystem in an interleaved and rotatory manner. During each service cycle, the disk heads scan across the disk surfaces and retrieve one block of data for each stream. Table 1 summarizes the parameters of the simulation run that the following discussion is 2
while (Candidate pairs list is not empty) begin if (free buffer pool is not empty) then begin allocate one buffer to each unmarked speculative relay chain using the buffer utilization factor as the priority function; /* i.e. The one with a higher buffer utilization factor has a higher priority. */ mark the speculative relay chains to which a buffer is just allocated; end; if (the number of free buffers is not enough for merging the head of the candidate pairs list) then begin release all buffers of the speculative relay chains whose buffer utilization factor is smaller than the buffer utilization factor of the head of the candidate pairs list; /* This step is labeled as 6 in Figure 3. */ while (free disk bandwidth is available and the number of buffers associated with the head of the relay pairs chain is greater than two times the number of buffers required by the head of the candidate pairs list) begin split the head of the relay pairs list; /* This step will release a few buffers, labeled as 1 in Figure 3, and take one unit of disk bandwidth, labeled as 4 in Figure 3. */ end; if (the number of free buffers is not sufficient for merging the head of the candidate pairs list) then break from the while loop; end; else begin merge the head of the candidate pairs list; /* This step will release a unit of disk bandwidth, labeled as 3 in Figure 3, and take a few buffers, labeled as 2 in Figure 3. */ end; end; while (the number of free buffers available is less than the number of unmarked speculative relay chains) begin remove the head of the speculative relay chain and release the buffers associated with it; /* This step will put a number of buffers to the free buffer pool, labeled as 6 in Figure 3. */ end; allocate a new buffer to each speculative relay chain and remove the mark;
Figure 3: Pseudo-code of the resource allocation algorithm tem from users' viewpoint. The simulation is conducted as follows. In every service cycle, each idle client, a client that is not accessing data, independently decides whether an access request is to be issued based on a pre-determined issuing probability. If an access request is to be issued, the client, independently of the other clients, selects a program to access based on a popularity distribution function. The popularity distribution function speci es how likely each of the programs in the system will be selected by a client. In this study, we used a popularity distribution collected from a Karaoke facility, which is a prevailing entertainment in many Asia countries. Figure 4 depicts the cumulated popularity distribution curve starting from the most popular program. Figure 5 depicts the simulation results. It shows the rejection rates with the generalized relay mechanism, the original relay mechanism, and the intuitive approach versus the size of the memory incorporated. The intuitive approach always lls up the memory with the most popular programs, i.e. always loads the
No. of clients 400 No. of Programs 5000 Duration of one service cycle 1.8 sec Duration of each program 3 min Simulation time 2 hours Max No. of streams in a stream group 20 No. of disks 4 Issuing probability 1% Table 1: The simulation environment based on. We conducted a large number of simulation runs with various parameter values. The results presented in the following is just a typical case. We measured the rejection rate of the system, i.e. the rate that the requests are rejected during the simulation run due to limited capacity of the system. The rejection rate represents the service quality of the sys3
Memory size 0.0MB 28.8MB 57.6MB 115.2MB 230.4MB 460.8MB 921.6MB Rejection rate of IA 36.5% 36.5% 33.6% 28.3% 20.9% 11.2% 0.29% Rejection rate of RM 36.5% 27.8% 23.5% 18.1% 10.2% 4.0% 2.9% Rejection rate of GRM 36.5% 26.6% 22.2% 16.5% 8.1% 1.3% 0.08% (IA-GRM)/IA 0.0 27.3% 34.0% 42.0% 61.1% 88.1% 71.4% (RM-GRM)/RM 0.0 4.4% 5.5% 8.9% 20.5% 66.7% 97.2% Table 2: The improvement of the generalized relay mechanism (GRM) over the intuitive approach (IA) and the original relay mechanism (RM). 1 40% Intuitive Approach Relay Mechanism Gen. relay mech 30% Rejection rate
Cumulated popularity
0.8
0.6
0.4
20%
10%
0.2
3% 0 0
0
0
500 1000 1500 2000 2500 3000 3500 4000 4500 5000 Programs
28.8
57.6 115.2 230.4 Memory size(MBytes)
460.8
921.6
Figure 5: The simulation results with issuing probability = 1%
Figure 4: The cumulated popularity distribution curve of Karaoke programs used in the simulation.
with the most frequently accessed programs.
most popular programs in entirety into the memory. Table 2 shows the improvement of the generalized relay mechanism over the original relay mechanism and the intuitive approach. The results reveal that the generalized relay mechanism always yields better results than the original relay mechanism and the intuitive approach. The improvement is particularly signi cant when a good size of memory is incorporated to reduce the rejection rate to a reasonable level. If a memory of 921.6MBytes is incorporated, then the rejection rates due to implementing the generalized relay mechanism, the original relay mechanism, and the intuitive approach are 0.08%, 2.9%, and 0.29%, respectively. If we use 3% rejection rate as the acceptable design criterion, then the sizes of memory required by the generalized relay mechanism, the original relay mechanism, and the intuitive approach are 378MBytes, 921MBytes, and 684MBytes, respectively.
References
[1] A. Dan and D. Sitaram. \Generalized interval caching policy for mixed interactive and long video workloads," In Proceedings of 1996 IS&T/SPIE Symposium on Electronic Imaging: Science & Technology, Conference on Multimedia Computing and Networking, Feb. 1996.
[2] C.H. Wen C.Y. Cheng M.H. Lee F.C. Wang, and Y.J. Oyang. \A study of bu er cache management in video-on-demand systems," In Proceedings of 1995 International Workshop on HDTV and the Evolution of Television, Taipei, Taiwan, November 1995. [3] Y.J. Oyang, M.H. Lee, C.H. Wen, and C.Y. Cheng. \Design of Multimedia Storage Systems for ondemand playback," In The Proceedings of 11th International Conference on Data Engineering, March 1995.
4 Conclusion
In this paper, we propose the so-called generalized relay mechanism to exploit semiconductor memory to upgrade the service capacity of video-on-demand servers. The simulation-based study shows that the generalized relay mechanism enjoys a signi cant advantage over the original relay mechanism and the intuitive approach, which always lls up the memory 4
