The Design and Implementation of Media Distribution System In 3Tnet VOD Quan Zheng, Zhong-Kui Zeng, Wei Luo Department of automation University of science and technology of China HefeiˈChina
[email protected],
[email protected] Abstract—According to the characteristics of 3Tnet VOD, we propose a design and implementation of Media Distribution System for streaming media, which utilizes the bandwidth of 3Tnet and improves the media distribution speed, while also providing a more scalable distribution scheme. Simulation results show that the proposed scheme could not only support VOD service on the 3Tnet in a scalable way, but also reduce the hardware cost of the network and server load. Keywords-3Tnet, VOD, P2P, Content Distribution
I.
INTRODUCTION
With the rapid development of computer network and communication technology, Video-on-Demand(VOD) is getting more and more popularity as a new trend of service mode in high-speed network. Typically a VOD system which is geographically distributed through high-speed communication networks stores a large number of video programs in one or more servers and provides video streams on clients’ demand. However, providing large-scale VOD services is still a challenging problem[1]. Tbps transmission, Tbps routing, and Tbps switching (3TNet) is the highperformance broadband information network key project of the national 863 Program of China, constructed to meet the requirements of Web interactive TV. 3Tnet is the nextgeneration operable high-performance broadband information demonstration network that supports large-scale concurrent streaming services and interactive multimedia services. In traditional VOD systems, caching proxy server is applied between the central server and the users to improve the quality of services. Researchers have done lots of research on this. Currently there are two main directions: The first one is proposing new scheduling strategy of proxy caching to improve the local hit rate and reduce the central server load, such as the scheme caching the whole streaming media[3], and the segment-based proxy caching scheme[4]. However, the problems stay unsolved in real-time and reaction between clients or congestion in residential network. The second direction is to improve the system architecture. There are two alternative (and competing) technologies for delivering live video streaming to end-users: Content Delivery Networks (CDN) and Peer-to-Peer (P2P) systems. We can build a hybrid CDN-P2P architecture that incorporates the superiority of both
978-1-61284-459-6/11/$26.00 ©2011 IEEE
1791
technologies and mutually oơsets each others’ de¿ciencies. Several researchers have hypothesized and analyzed the potential bene¿ts of such an approach[6,7]. In a multi-level cache system[8], content distribution platform is applied between the central server and the proxy caching server, which can significantly reduce the central server load, but it requires a large amount of additional hardware investments. In this paper, we propose a Media Distribution System(MDS) on residential network over the China's high performance broadband information network. MDS fully utilize the bandwidth of 3Tnet and can be built on existing servers without significant additional hardware investment. At the same time, the system could not only support true VOD service on the 3Tnet in a scalable way, and also reduce the hardware cost of the edge network and the upload burden of residential network, respectively. II.
RELATED TECHNOLOGIES
A. The traditional VOD system The traditional VOD architecture based on the proxy caching is shown in Figure 1. In a VOD system, a number of central servers store all the video resources. Between the central servers and user, we add a hierarchy of servers which cache the movies delivered from the central servers to the users. The caching servers could be placed close to user to reduce the transmission cost, and reduce the burden on the central servers, The system is able to achieve scalable storage and streaming
Fig 1 . architecture of traditional proxy caching
systems are usually huge and the cache nodes have limited capacity, the cache servers can only storage the hot video, and cache other video partially. When user request arriving, The cache servers will directly deliver the video to the user if the request data is stored in it, Otherwise, it'll request data from the central server. Since some movies are popular while others are not, meanwhile the popularity may change over time, there is a demand to decide which video and how much of the video should be cached locally depends on the delivery strategy to minimize the system cost. B. Dynamic-Link Library(DLL) Technology Library is a collection of small software programs any of which are called up by a larger program that is running on your computer. Each small program is compiled in advance produces object file, called library. The advantage of the library is that the function in the library files can be connected with the application by link program, so that we do not have to compile these programs every time when developing the procedure. Library can be divided into two types, static-link and dynamic-link. Static object files must be linked into an executable program when compiled, and dynamic linking is performed dynamically by the operating system while the program is running, such type of linking can improve upon memory usage because there only needs to be one copy of the DLL in memory for multiple processes to have access to it. III.
THE DESIGN AND IMPLEMENTATION OF MEDIA DISTRIBUTION SYSTEM
A. The architecture of MDS The overlay of the MDS architecture is illustrated in Figure 2. In the architecture, PN is responsible for P2P data transmission as the most basic P2P node; DS is the directory server, which provides P2P resource query information; SM is the distribution and scheduling module, which manages the scheduling and distributing strategy of the entire system. Details of each module are as follows: PN(P2P Node) P2P nodes of MDS is the P2P data interfaces of edge caching proxy, caching proxy send and receive P2P data
Fig 2 . the overall architecture of system
1792
through the PN, PN is responsible for the P2P resource query and data transmission of the entire system. Each PN is corresponds to an edge caching proxy, it also regularly collect the User-demand information of all the video from the caching proxy, and forwards the information to the strategy manager. DS(Directory Server) DS record the resources distribution information of the entire P2P system, and provide P2P resources inquiries service for PN. Before downloading the data, PN send a request to DS to find the caching proxy that the request data are stored in, then launch data request to PN corresponding to the caching proxy. SM(Strategy Manager) Distribution and scheduling strategy management module manages and executes the distribution and scheduling strategy of the entire system. The manager chooses an appropriate distribution and scheduling strategy according to the specific network after collecting user-demanding information of all the video in every caching proxy, then calculates the distribution and adjust instruction of video data and send the instruction to PN. The instruction decided which video and how much of the video should be cached locally at a given request rate in order to minimize the system cost. The distribution process of P2P media data flow is as follow: 1) SM send the video distributed instruction to PN corresponding to the edge caching proxy . 2) PN send a request to DS to find the caching proxy that stores the requested data. 3) According to the result returned from the DS, PN requests data from the PN, or forwards the request to the central content server if the requested data are not stored at the caching proxy. 4) PN stream the data to its corresponding caching proxy. 5) After the data transmission, PN report the result of instruction execution to SM and the data change of corresponding caching proxy to DS. B. System module implementation The media distribution system presented in this paper program in C++, using g + + 3.4.6 as compiled tool, runtime environment is Redhat_AS4, The following is the implementation details of the module. DS DS maintain the P2P resource information of the entire system by keeps a list which keep tracks of the available PN’s information, it map PN to matching corresponding video resource. Initially only the central content server’s information is kept in the list, as the video data were streamed to each caching proxy, DS will gradually add the PN corresponding to these caching proxy into the list. The available information list is shown in Table. I. This list records related available PN information with the current four different videos (video 1,2,3,4 ).When PN contact corresponding PN information consisted of PN's IP and storage resources
TABLE I AVAIABLE PN LIST
Video
Available PN
video1
Central content server
PN02
PN05
video2
Central content server
PN01
PN03
video3
Central content server
PN06
video4
Central content server
The available PN information table synchronously update when a video was distributed to a new caching proxy or deleted, as PN would send the modification to DS. DS is also responsible for monitoring the PN by regularly test its network status and processing capabilities of PN. The PN will be removed from the available list if there is an exception and be rejoined to it after a normal test. PN To improve the performance and scalability of PN’s implementation, we cache the P2P data source list in memory for the P2P data source list occupied small memory to speed up the P2P resource query speed. And for the large volumes of video data, PN use a block-type reception and transmission method, that is, PN will not receive the next piece of data until the current data has been sent to caching proxy or stored into hard disk. Although the utilization rate of bandwidth will partly decline, it will significantly reduce the memory footprint for the video data could not accumulate in internal storage. 1) Internal implementation of PN and the major work processes. PN maintain its data source cache list which keep the data sources recently queried from the DS. PN draws data from these data sources after receiving the data distribution instruction, then forwards the received data to caching proxy, if the current data source is insufficient to complete the task, a re-query to DS is required to obtain the latest data source. After completed the task, report the updated PN’s information to DS, and send the results of instruction execution to SM. 2) P2P data source selection. The selection of data sources is very important when PN downloading data from P2P networks, it is preferential to get video data from other PN to alleviate the burden on central content server. We sort these PN based on the PN’s distance from others, available bandwidth and current load, then choose the lowest-cost PN for downloading. In the stage of data source selection, W = B×b - D×d - L×l, where the network distance D between PN is set to d, the available bandwidth B is set to b, respectively, and the weight of central content server is set the minimum. When receiving data request, the system select PNs with the largest number W and transport the data concurrently, and adjust the weight according to its operational aspect to achieve the optical value. SM The implementation framework of SM is as follows. First, distribution and scheduling strategy is compiled to produce the corresponding dynamic-link library files. The strategy management module play two roles: collecting Video-on-
1793
demand information and network information of the entire system and choosing the most appropriate distribution strategy. After these, the main program of SM load the dynamic-library files corresponding to the strategy into memory, call the distribution and scheduling function in the library, compute the distribution and scheduling instruction of all the PN, finally send instruction to PN to complete scheduling operation. The main program of load the dynamic library in SM is as follows: /*SM distribution and scheduling function, input is the name of dynamic-link library file corresponding to distribution strategy*/ int SM_Scheduler::schedule(string strategyFileName) { //getting current information of all the PN vector pns = SM_Scheduler::getAllPnInfo(); /*according to the input dynamic-link library file name, load the specified scheduling strategy*/ SM_DeployStrategy*strategyPtr= SM_DeployStrategy::getCurrentstrategyPtr(strategyFileName); /*input the information of all the PN, use the scheduling strategy loaded in last step to calculate the data distribution instruction of each PN*/ vector pnCommands = strategyPtr->generateCommand(pns); //send the distribution instruction to all PN SM_Scheduler::snedCommandToPn(pnCommands); }
If you want to add or change the existed distribution and scheduling strategy, what you need is to compile the distribution strategy to produce dynamic-link library files, then add them into the library or replace the original library files, no need to modify the SM process. IV.
EVALUATION
To evaluate the performance of MDS system, we used NS2 (Network Simulator, version 2) software to do the media distribution speed test. NS2 is a widely used network simulation software, developed by UC Berkeley, with a virtual clock which could produce a variety of network simulation events. The server of testing platform is Dell PowerEdge 2650, Intel Xeon 2.4 GHz CPU dual-core, 4 GB memory, the operating system is RedHat_AS4DŽSpecifically, we varied the cache proxy number from 1 to 500, separately tested the distribution latency between the central server and the proxy caching in traditional structure compared to in the MDS. The central server’s bandwidth is set to 1Gbps, the export bandwidth of caching proxy is set to 100Mbps, the size of distributed video is set to 500MB, and recording the time the first caching proxy start receiving data until the last caching proxy finish receiving data, experimental results is shown in Figure 3. From the experimental results, we can see that when the number of caching proxy is less than 50, the traditional structure and MDS have a little difference in distribution latency. However, there is a linearly increase with large fluctuation in the latency of the traditional structure when the
lower than the DS which means a common server can run SM, SM and DS can even run on the same server if the number of server is less than 100. From all above we get the conclusion that the MDS can run in the distribution architecture of the traditional video-ondemand system, only one or two more general servers are need to run DS and SM, no need investment of additional hardware. V.
Fig 3.
compare of distribution time
number of caching proxy increasing, caused by the heavy burden of the central server and network instability, while distribution latency of the MDS increase slowly, and the fluctuation is small. The network is stabilized, distribution speed is significantly higher than the traditional distribution structure. At the same time, we have tested the consumption of the single machine hardware as shown in Table 2: TABLE II SINGLE MACHINE HARDWARE CONSUMPTION TEST Data transmission CPU utilization Memory Required speed 10MBps 18MB 1ˊ8% 100MBps 34MB 5ˊ2% 600MBps
8ˊ9%
CONCLUSION
In this paper we proposed a media distribution system (MDS) based on the characteristics of 3Tnet, which can be built on the distribution architecture of traditional video-ondemand system and doesn’t need large scale of hardware investment. It can also significantly reduce the video content distribution latency and support the distribution and scheduling strategy in a scalable way. We can select the most appropriate strategy according to concrete network condition and users’ VOD behavior. Currently the development and testing of the system are complete, master control server has been deployed in the Shanghai Broadband Technology and Application Engineering Research Center, trial operation on the 3TNet. The MDS System is able to support distributing and scheduling 8000 videos at present, and the distribution speed is significantly faster than the traditional C/S video distribution model. Future research can proceed along several avenues. First, a new design of distribution and scheduling strategy for the 3Tnet VOD is a important direction. Second, some other design issues could be considered to improve the system performance, such as the popularity of the programs. Finally, larger real system deployment over the 3Tnet will validate and strengthen our proposals.
114MB
REFERENCES
We can judge from the test results that even in the case that the transmitted data amounts is 600Mbps, the CPU consumption is low and memory occupied by PN is 114MB,only 2.8% of the 4GB,so it can by all means be ran on the existing caching proxy server without affecting its normal operation. Test of the single machine service ability is as follows: DS can offer 13000 times P2P resource queries per second, if there’re 500 servers, each server can perform 2.5 queries per second, while in the case that 500 servers distribute concurrently, each PN execute 0.12 query per second on average, much more less than the service capability of DS. The DS can run on a common dual-core server for the number of proxy server is usually less than 500 in practical applications, Hardware requirement of SM depends on the specific distribution strategy, The memory footprint wouldn't exceed 50M on account of the usage of dynamic-link library with which the strategy not to be used could withdraw from the memory. And the data adjusting frequency of proxy server is much more less than the P2P resource querying frequency. So SM’s consumption of hardware resources is significantly
1794
[1]
[2]
[3]
[4]
[5]
[6]
[7] [8]
M Hofmann, TSE Ng, K Guo, S Paul, and H Zhang, “Caching techniques for streaming multimedia over the Internet”. Bell Laboratories Technique Report, BL01, 1345-990409-O4TM, 1999. Chiu M.Y.M and Yeung K.-H.A, “Partial video sequence caching scheme for VOD systems with heterogeneous clients,” IEEE Trans, Industrial Electronics, Vol. 45, pp 44 – 51, Feb 1998 JM Almeida, DL Eager, M Ferris, MK Vernon, “Provisioning content distribution networks for streaming media”. INFOCOM ,vol. 3, pp 17461755, 2002 Kun-Lung Wu, Philip S. Yu K and Joel L. Wolf, “Segment-based proxy caching of multimedia streams,” International World Wide Web Conference, Proceedings of the 10th international conference on World Wide Web, pp. 36-44, 2001 Kyung Oh Lee, Jeong Bae Lee and Kee Wook Rim, “A dynamic scheduling algorithm for video-on-demand servers,” IEEE Trans, Consumer Electronics, Vol. 50, pp 1113-1118, Nov. 2004 Chuan-Dong Yang, Zheng-Wei Yu, Xing-Gang Wang, “Research on a Hybrid Architecture for Media Streaming Distribution”, Journal of Computer Applications, vol. 25, pp 2204-2207,2005 Liang Deng㧘 “combine P2P and CDN to realize the IPTV on-demand service”, Network applications, vol. 1, pp 39-44, 2007 Wei Xiang,, Jun Li, Gang Wu, Qin Chen, “Service Scheme for videoon-demand in 3Tnet Environment” , Journal of University of Science and Technology of China, vol. 37, pp 189-194, 2007
