SIP-Based Adaptive Multimedia Transmissions for Wired and Wireless Networks* Weijia Jia and Man-Ching Yuen Department of Computer Science, City University of Hong Kong, 83 Tat Chee Avenue, Kowloon, Hong Kong, SAR China
[email protected]
Abstract. SIP (Session Initiation Protocol) is a signaling protocol standardized by IETF, aiming to manage the multimedia transmission sessions among different parties. This paper illustrates an adaptive multimedia transmission system for wired and wireless networks based on SIP with protocol selection mechanism for a certain level of QoS guarantee. In our system, SIP is not only used for call setup signaling but also for carrying the information in the protocol selection. Using Agent Server, our system can select the most suitable protocol for adapting different situations intelligently during the connections and data buffering service is also provided for various media data flows between the end users with acceptable QoS level without any interruption and disconnection regardless of types of devices, platforms and protocols used.
1 Introduction Mobile multimedia transmissions such as online movies, live TV, network radio and audiovisual conversation require good quality of service (QoS) anytime and anywhere. However, it is difficult to attain the targets over the integrated wired wireless networks because the wireless devices typically have the limited resources of processing power or memory. Little work is done to enable dynamic multimedia communication, especially end-to-end multimedia transmission across wired and wireless networks. Compared with wired Internet, there are several obstacles for endto-end wireless multimedia transmission: (1) Low capability and limited resource of terminals: A wireless terminal typically has a small display with low resolution, slow processor and small memory space. However, multimedia applications usually require high capability of graphic processing, large size of memory space and also a big screen to display pictures and videos. Obviously, wireless terminals can only support limited multimedia applications. (2) Diversity of wireless terminals: Wireless terminals often support only a limited set of data formats due to their low capabilities and limited resources. When two wireless terminals of different types communicate with each other, their supported data formats may not be compatible and thus communication cannot succeed. Although it is possible that the data sent from the sender can be *
This effort is partially sponsored by City University of Hong Kong strategic grants 7001587, and 7001709 and the National Basic Research Program (973) MOST of China under Grant No. 2003CB317003.
J. Cao, W. Nejdl, and M. Xu (Eds.): APPT 2005, LNCS 3756, pp. 505 – 514, 2005. © Springer-Verlag Berlin Heidelberg 2005
506
W. Jia and M.-C. Yuen
converted into the format supported by the receiver, these conversion often costs much and is not practical or acceptable to resource-limited wireless terminals. (3) Low bandwidth of wireless networks: Due to low capability of wireless devices, multimedia data streams created by wireless devices can not be compressed very much and require high bit rate for real-time transmission. Therefore, it is difficult to develop dynamic real-time multimedia communication protocols for audiovisual conversation and videoconferencing. (4) Fluctuated bandwidth and blockouts of wireless connections: Multimedia data transmission, especially real-time transmission, requires steady high bit rate and is intolerant of package delay. On the contrary, wireless networks have fluctuated bandwidth and high probability of traffic congestion. Usually, the transmission time of multimedia data in a session is quite long, but the blackout of wireless connection may cause frequent session reconnection and data retransmission. SIP (Session Initialization Protocol) [12] is a signaling protocol of Application Layer which is standardized by IETF (Internet Engineering Task Force), and it aims to manage multimedia sessions among different parties. The principle of SIP is to set up sessions or associations between two or more end users. SIP is not used for transmitting data, but sessions initiated with SIP can exchange various types of media data using appropriate protocols such as RTP, RSTP and so on. It can carry out bidirectional authentication and capability negotiation. SIP is simple and extensible. It accepts complementary information inserted as SIP payload for other applications. Currently, SIP is able to set up a call for a multimedia session of complex requirements by carrying more detailed information using protocols such as the Session Description Protocol (SDP) [13]. This paper proposes an adaptive protocol selection mechanism for integrated wired-wireless multimedia transmission mechanism using SIP to maintain a certain level of QoS guarantees. SIP is not only used for call setup signaling, but also carries information for protocol selection. SIP usually carries an SDP packet describing an audio or video session, indicating supported communication protocols, end terminals capabilities, QoS requirements of applications and session ID which is used for user identification of multi-parties communication. Besides selecting the most suitable protocol for adapting different situations intelligently during connection, data buffering service is provided with Agent Server such that media data flows must transmit between end users. In this way, the end users can communicate among the others at their best acceptable QoS level regardless of types of devices, platforms and protocols they are using. The rest of this paper is organized as follows. Section 2 introduces some related works. Section 3, describes the adaptive protocol selection mechanism. Section 4 presents the multimedia transmission connections on integrated wired and wireless networks by using SIP technology. Section 5 concludes the paper.
2 Related Work WMSTFP [2] is an end-to-end TCP-friendly multimedia streaming protocol, which is used to detect the status of the wired and wireless parties in the wireless Internet. By accurately distinguishing the packet losses due to transmission errors from the congestive losses and smoothing out the pathologic round-trip-time values caused by the
SIP-Based Adaptive Multimedia Transmissions for Wired and Wireless Networks
507
highly dynamic wireless environment, higher throughput in wireless Internet can be achieved and transmission rate can be adjusted in a smooth and TCP-friendly manner. UPnP™ Forum [3] is an industry initiative designed to enable simple and robust connectivity among stand-alone devices and PCs from different vendors. The forum members are engaged in producing standards to describe device specifications usually in XML format. iMobile [4] is a proxy-based mobile service platform designed to provide personalized services. iMobile provides a modular architecture that supports accesses from various mobile devices to various information spaces. However iMobile does not support communications between devices. Transcoding service plays a very important roll in the design of wireless multimedia system [5-8]. A video transcoding technology using intermediate data processor is proposed in [5] to enhance the quality of transcoded data. A video transcoding proxy for 3G wireless mobile Internet access and a video transcoding gateway for wireless access are proposed in [6] and [7] respectively. End-to-End Wireless Multimedia Transmission system (EEWMT) is developed and designed to transcode end-to-end data flows [8]. A lot of other related works have been engaged in wireless access of multimedia data, but most of them just consider data transmissions between wireless terminals and servers. In our proposed system, data transmission between wireless terminals is also considered using Agent Server as the intermediate party to provide a certain level of QoS guarantees. Moreover, to maintain the best acceptable QoS level, SIP is used for both call setup signaling and carrying information for processing an adaptive protocol mechanism, such that the most suitable transmission protocol is selected for transmission of various multimedia data flow adaptively during connection without interruption or disconnection.
3 Adaptive Protocol Selection The adaptive protocol selection mechanism is used to ease the balance of transmission performance and communication interoperability among various clients and servers in wired networks and wireless mobile networks. We briefly describe its mechanism. Based on features and popularities of different existing protocols, the transmission protocols used in our system are classified as: TCP (Transmission Control Protocol) or UDP (User Datagram Protocol), 1. Pure HTTP (HyperText Transfer Protocol) [10], and 2. A web services protocol, SOAP (Simple Object Access Protocol). TCP/UDP may be used to provide efficient and fast data transmission. Especially, it enables real-time data transmission using UDP with the support of certain protocols like RTP (Real Time Protocol). However, it is difficult to implement using TCP/UDP because they may not be interpretable between different OS platforms. As a result, TCP/UDP implementations in a specific OS platform may not be reused in another OS platform. Pure HTTP enables communications across different platforms, and also allows communications penetrating some firewalls. However, it is not flexible for developers to support new services due to its limited number of services and commands provided. As web services protocols work based on XML [11], they have similar functionalities as HTTP. The only difference is that the web services protocols can
508
W. Jia and M.-C. Yuen
provide a consistent and simple interface for developers to support connection services in a uniform way. Since both pure HTTP and web services protocols are used in the application layer, their data transmission rate is low and may not be suitable for the real-time multimedia transmission. As these popular transmission protocols have different advantages and disadvantages, they are used under different situations. Table 1 shows performance comparison of different natures of data with four types of network protocols (TCP, UDP, HTTP and web services). There are many ways to select the most suitable protocol for different situations. One of the examples is to apply fuzzy membership functions [16]. There is a unique membership function associated with each input parameter. The membership functions associate with weighting factors with each input and the effective rules. These weighting factors determine the degree of influence or degree of membership (DOM) for each active rule. By computing the logical product of the membership weights for each active rule, a set of fuzzy output response magnitudes are produced. All that remains is to combine and specify these output responses. Table 1. Performance Comparison of Different Natures of Data with Four Types of Network Protocols (TCP, UDP, pure HTTP and web services)
Text (Data size: very small)
Nonreal time
Audio (Data size: small)
Nonreal time
Real time
Real time
Image (Data size: medium or large)
Nonreal time
Video (Data size: large or very large)
Nonreal time
Real time
Real time
TCP/IP Suitable (Fast data transmission rate) Suitable (Fast data transmission rate) Suitable (Fast data transmission rate) Suitable (Fast data transmission rate) Suitable (Fast data transmission rate) Suitable (Fast data transmission rate) Suitable (require more network resource) Suitable (require more network resource)
UDP/IP Suitable (Fast data transmission rate) Suitable (Fast data transmission rate) Suitable (Fast data transmission rate) Suitable (Fast data transmission rate) Suitable (Fast data transmission rate) Suitable (Fast data transmission rate) Most suitable (Fast data transmission rate) Most suitable (Fast data transmission rate)
Pure HTTP Most suitable (relatively slow data transmission rate) Most suitable (relatively slow data transmission rate) Most suitable (relatively low data transmission rate)
Web Services Most suitable (relatively slow data transmission rate) Most suitable (relatively low data transmission rate)
Most suitable (relatively low data transmission rate)
Most suitable (relatively low data transmission rate)
Most suitable (relatively low data transmission rate)
Most suitable (relatively low data transmission rate)
Most suitable (relatively low data transmission rate)
Most suitable (relatively low data transmission rate)
Not suitable (Very low data transmission rate)
Not suitable (Very low data transmission rate)
Not suitable (Very low data transmission rate)
Not suitable (Very low data transmission rate)
Most suitable (relatively low data transmission rate)
SIP-Based Adaptive Multimedia Transmissions for Wired and Wireless Networks
509
To effectively communicate among the different platforms with varies devices, our adaptive protocol selection mechanism is designed possessing the following functionalities: (1) Allowing different kinds of client devices to communicate in the integrated wired and wireless networks while the communication performance is monitored at an acceptable level most of the time; (2) Allowing clients to dynamically select a suitable protocol for adapting different situations intelligently without any interruption and disconnection; (3) Providing consistent APIs for different protocols thus reducing development overhead of service modules regardless of platforms and devices, and (4) Providing simple APIs for different service modules so that the APIs are reusable and extensible for supporting new services. We here briefly describe the implementation and design of the adaptive protocol selection system for handling the interactive communications among integrated networks and various client devices. The design issues are categorized into two parts: connection establishment and transmission protocol selection below: Connection Establishment consists of two major steps: (1) Initialization of communication session between client devices through agent servers. A set of agent servers in the networks are responsible to provide data buffering and QoS guaranteed services. All communications between end users must pass through the agent servers. The communication session between agent server and client device can be initialized using SIP for agent server or client device. By considering different protocols supported by client devices and characteristics of transmission sessions, the way of communication between agent servers and client devices is defined during the communication initialization stage (see Section 4). (2) Notification of both protocol and platform of all communication parties to agent servers. Whenever communication is initialized either by agent servers or client devices, agent servers have to know the type of protocol of client devices, platform of client devices and QoS requirements of applications. It is because negotiations between agent servers and client devices may be required for having services of the best performance. It is also useful for selecting the most adequate transmission protocol in later step. Transmission Protocol Selection is defined in two respects: (1) Common APIs (Application Program Interfaces) of supported transmission protocols. To support most of the services provided by transmission protocols, we have devised the common APIs of the transmission protocols available in agent servers. Each service module has a set of APIs for providing its service to client devices where the APIs are able to support different protocols, platforms and client devices. (2) Selection of a suitable transmission protocol in different situations. There exist a number of data type, such as non real-time text data and real-time video data. To balance the performance (user’s point of view) and system overhead (developer’s point of view), different transmission protocols are suitable for different data transmissions. To select the most suitable transmission protocol according to different situations, we have to define data transmission, the QoS requirements of session, the available protocols supported by client devices and the protocols supported by agent servers. Once agent servers have the information, they determine the most adequate transmission protocol for specific data transmission and inform client devices by providing an appropriate data transmission process using the most adequate protocol. Our system uses SIP to select the most suitable transmission protocol adaptively depending on the nature of media flows and the transmission capability [9] as detailed in next section.
510
W. Jia and M.-C. Yuen
4 SIP Based Multimedia Transmission Control Our system is designed to provide services to end-to-end multimedia transmission, including both real time and non-real time transmission, with certain level of QoS guarantees. Fig. 1 shows the framework of our system for wired and wireless networks. Four main parts of our system are User Agent in client device, SIP Proxy Server, Database Server and Agent Server. We here illustrate them as below: 1. User agents (UA) are SIP endpoints that send or receive signaling messages residing on client devices and help client devices to communicate with servers. UA collect device profile of client devices, capabilities of client devices and QoS requirements of sessions to be requested, and then sends these information to the servers. UA also convert user commands and application signaling into formats that can be read by servers and also translate server responses for users and applications. Device profile is used to describe the technical specifications and capabilities of the device such as multimedia processing capability and network transmission capability. Some other device information such as manufacturer name and device model are also presented in the profile. Device profile must be in a universal format so that the servers are able to recognize all kinds of devices and provide appropriate services to them. The device profile is given as a XML formatted file listing device specifications and capabilities. Fig. 2 presents a simple example of device profile. 2. SIP proxy servers store the information of all the major SIP proxy servers and provide DNS services. Each major SIP proxy server further connects to a set of proxy servers within its network domain. The control data flow through SIP proxy servers, while all media data in communication between end users flow through agent servers only. 3. Database servers store the updated information of all agent servers in their domains and respond to the requests from either SIP proxy servers or agent servers. Examples of the information in database server are user profiles and device profiles. 4. Agent servers are application-layer routers and receive call requests from UA or another proxy, try to locate the receivers via the selected route paths defined by SIP initially, and forward the requests to another location until the given address is reached. They execute the adaptive protocol selection mechanism and keep track the change of situations during data transmission. To achieve the best level of QoS guarantees, an agent server provides many categories of services to client devices: the data buffering service prevents transmitters retransfer lost data due to network congestion or disconnection and the data transcoding service helps the heterogeneous terminals to communicate with each other seamlessly. All data flows in communications must go through agent servers until receiving terminals are reached. Basically, SIP is a control protocol for establishing media sessions and it is used for both call setup signaling and session transmission for adaptive protocol selection mechanism. Five functionalities that support the establishment and termination of multimedia communications, for the adaptive protocol selection mechanism, are called:
SIP-Based Adaptive Multimedia Transmissions for Wired and Wireless Networks
511
1. User location detection determinates the end system to be used for communication. 2. User capability detection defines the media and media parameters to be used. 3. User availability detection decides the willingness of the called party to engage in communications. 4. Call setup establishes call parameters at both called and calling party during “ringing”. 5. Call handling handles many management operations including transfer and termination of calls. The sessions’ addresses to be established are carried out in the body of the application layer message. It has two types of messages: request and response. SIP messages carry the descriptions of media sessions in their payload/header using Session Description Protocol (SDP) [13]. Some additional mechanism is needed for payload modification is defined for the servers below: 1. 2. 3. 4. 5.
INVITE: This message is used to invite another participant to a session. ACK: This message confirms session establishment. BYE: This message is used to close a session. CANCEL: This message cancels a pending INVITE message. REGISTER: This message is used to register the current address of a potential participant. 6. RESPONSE: This message is used to give response to request and indicates success or failures and progress updates.
Fig. 1. Framework of SIP based end-to-exnd multimedia transmission for integrated wired and wireless networks
W. Jia and M.-C. Yuen
512 User
SIP Proxy
Agent
Database
Database
Agent
SIP Proxy
User
Agent
Server
Server
Server
Server
Server
Server
Agent
(Client 1)
(Client 1)
(Client 1)
(Client 1)
(Client 2)
(Client 2)
(Client 2)
(Client 2)
1. SIP INVITE 2. Request 3. Reply 4. SIP INVITE 5. SIP INVITE 6. SIP OK 7. SIP OK 8. SIP OK 9. ACK 10. Connection request 11. Connection request 12. Connection request 13. Connection reply 14. Connection reply 15. Connection reply 16. Media flow
Fig. 2. Signaling scenario for connection establishment in call setup stage
To establish a connection session, a UA sends a SIP INVITE request to SIP proxy server. As mentioned before, the SIP proxy server only stores the information of all other SIP proxy servers and provides DNS services. SIP proxy server sends a request to database server which searches the information of agent servers among the end users. Once the connection path is established, the media data flows through agent servers between end users directly rather than through proxy servers. During communication, agent servers keep track the status of connection, and deploy the adaptive protocol selection mechanism. During the connection, the agent server is responsible to select the most suitable transmission protocol for efficient communications. Note that for agent server, each communication protocol will reserve a specified port which is randomly generated. Figs 2-4 illustrate the adaptive connection establishment signaling protocols used in our system for call setup stage, change of transmission protocols or connection path between end users. Thus, our system has three advantages that support large varieties of devices and transmission environments, and also provide certain level of QoS guarantees: (1) Heterogeneous communication: Due to the diversity of wireless terminals, data formats supported by different terminals may be not compatible with each other. In our system, data flows go through agent servers that may cause the incompatible data formats into acceptable formats according to the device profiles of the receiving terminals. As a result, terminals can send and receive data in preferred formats without concerning data format incompatibility problems. (2) Low cost of terminal resource: Instead of terminals, agent servers are responsible to convert data formats, retransmission for lost packages, and reconnection to lost packets and saves resources for terminals more effectively and efficiently. Moreover, a certain level of QoS can be guaranteed.
SIP-Based Adaptive Multimedia Transmissions for Wired and Wireless Networks
513
(3) Faster recovery from disconnection: Connection loss is quite often for wireless connection due to signal fading, interference and path blackout. Once the connection between the receiver and its agent server is lost, the transmitter has to wait until it reconnects and retransmits the lost packages. In our system, since the terminals are connected to the specified agent servers, and data buffering in agent servers can keep receiving the data from the sender even if the receiver is disconnected. Once the receiver is reconnected, the buffered data will be delivered. User
SIP Proxy
Agent
Database
Database
Agent
SIP Proxy
User
Agent
Server
Server
Server
Server
Server
Server
Agent
(Client 1)
(Client 1)
(Client 1)
(Client 1)
(Client 2)
(Client 2)
(Client 2)
(Client 2)
1. SIP INVITE 2. SIP OK 3. ACK 4. Notice 5. New connection request 6. New connection request 7. New connection request 8. New connection reply 9. New connection reply 10. New connection reply 11. Media flow in new connection
Fig. 3. Signaling scenario for adaptive change of transmission protocols during connection
User
SIP Proxy
Agent
Database
Newly
Database
Agent
SIP Proxy
User
Agent
Server
Server
Server
Invited
Server
Server
Server
Agent
(Client 1)
(Client 1)
(Client 1)
(Client 1)
Agent Server
(Client 2)
(Client 2)
(Client 2)
(Client 2)
1. Request 2. Reply 3. SIP INVITE 4. SIP INVITE 5. SIP OK 6. SIP OK 7. ACK 8. New connection request 9. New connection request 10. New connection reply 11. New connection reply 12. Media flow in new connection
Fig. 4. Signaling scenario for change of connection path between end users during connection
514
W. Jia and M.-C. Yuen
5 Conclusions We have proposed an adaptive multimedia transmission system for wired and wireless networks using SIP technology for call setup signaling, information carrying via agent servers. Based on SIP, our system can be used in various communication environments and it is extensible. Through the adaptive protocols, the most suitable connection mechanism can be selected adaptively based on the dynamic change of traffic or flows.
References 1. 3GPP, “Open services architecture”, Application Programming Interface, 3G TR 29.998, http://www.3gpp.org. 2. Yang F., Zhang Q., Zhu W., Zhang Y.-Q., “Bit Allocation for Scalable Video Streaming over Mobile Wireless Internet”, Proceeding of INFOCOM'2004, Hong Kong, March 2004. 3. Universal Plug and Play Forum, Understanding Universal Plug and Play, 2000, http://www.upnp.org/download/UPNP_UnderstandingUPNP.doc. 4. Chen Y.-F., Huang H., Jana R., John S., Jora S., Reibman A., Wei B., “Personalized Multimedia Services Using A Mobile Service Platform”, Wireless Communications and Networking Conference, 17-21 March 2002. vol.2, pp. 918 - 925. 5. Iwasak O., Uenoyama T., Ando A., Nishitoba T., Yukitake T., Etoh M., “Video Transcoding Technology for Wireless Communication Systems”, IEEE Vehicular Technology Conference Proceedings, Tokyo, 15-18 May 2000. vol.2, pp. 1577 - 1580. 6. Warabino A., Ota S., Morikawa D., Ohashi M., Nakamura H., Iwashita H., Watanabe, F., “Video Transcoding Proxy for 3Gwireless Mobile Internet Access”, IEEE Communications Magazine, October 2000, Vol.: 38 , Issue: 10, pp. 66 - 71. 7. Lei Z., Georganas N. D., “Video Transcoding Gateway for Wireless Video Access”, Electrical and Computer Engineering, Canadian,4-7 May 2003. vol. 3, pp. 1775 - 1778. 8. Shen J., Han B., Yuen M.-C., Jia W., “End-to-End Wireless Multimedia Transmission System”, IEEE Vehicular Technology Conference Fall 2004. 9. Yuen M.-C., Cheng L., Au P.-O., Jia W., “Adaptive Generic Communications for Integrated Mobile and Internet Web-Services”, The 5th International Conference on Web-Age Information Mangement 2004. 10. Berners-Lee T., Fielding R., Frystyk H., “Hypertext Transfer Protocol -- HTTP/1.0”, RFC1945, May 1996. 11. Hollenbeck S., Rose M., Masinter L., “Guidelines for the Use of Extensible Markup Language (XML) within IETF Protocols”, RFC3470, Mar 2002. 12. Handley M., Schulzrinne H., Schrooler E., Rosenberg J., “Session Initiation Protocol”, RFC2543, IETF., March 1999. 13. Handly M. and Jacobson V., “SDP: session description protocol”, RFC2327, IETF, April 1998. 14. “H.323 - Packet-based multimedia communications systems”, ITU standard. 15. Schulzrinne H. and Rosenberg J., “A Comparison of SIP and H.323 for Internet Telephony”, Network and Operating System Support for Digital Audio and Video (NOSSDAV), Cambridge, England, July 1998. 16. Chen G., Pham T. T., “Introduction to Fuzzy Sets, Fuzzy Logic, and Fuzzy Control Systems” CRC Press, 2000.
