In UMA applications, it would be desirable (to avoid the extraction on-the-fly) to have available descriptions of the parts that have to be matched/bridged (the ...
An End-to-End Framework for Mobile Multimedia Applications
João Magalhães, Agostinho David Siemens, SA Rua Irmãos Siemens 1 2720-093 Amadora - Portugal {joao.magalhaes, agostinho.david}@lis2.siemens.pt
Abstract: Future mobile communications will bring a terminal diversity that until today has never happened before. 3GPP services will have to be developed taking into account every possible client. This paper describes an end-to-end framework able of delivering multimedia content that is tailored for each type of terminal. The framework is composed by three applications: a content authoring/analyzer application, a content customization engine and a multi-characteristics terminal simulator. The framework uses MPEG-7 as the content description standard and WAP UAProf as the user context description standard. Keywords: universal access, mobile mult imedia, MPEG-7, WAP UAProf. 1
Introduction
The exploding variety of multimedia information is nowadays a reality since everyone has a camera, a scanner or another device that almost instantly generates multimedia content. Most often the content author wishes to share its masterpiece with everyone,
but the variety of terminals and networks may be a problem if he wants everyone to see his work with the best possible quality. This scenario describes today’s situation of accessing multimedia content from any terminal as illustrated in Figure 1. Typically, when a terminal accesses content to which it was not designed for, the user experience is very poor. The user may experience long delays if the content is adapted ignoring the characteristics of the involved parts making its visual presentation less impacting, etc. The scenario described misses a bridging element between all the components involved, which should take into account their characteristics and assure an efficient and consistent inter-working. In other words, an efficient way to "access any information from any terminal", allowing the delivery of any content (or an adaptation of it) to the user should be provided. The access to multimedia information by any terminal through any network is a new concept referred in this paper as Universal Multimedia
Smartphone Content Author
Multimedia content Multimedia content authoring
PDA
Hand Held Computer
Figure 1 – Mobile information services chain.
Access (UMA). The objective of UMA technology is to make available different presentations of the same information, more or less complex e.g. in terms of media types, suiting different terminals, networks and user preferences. In UMA applications, it would be desirable (to avoid the extraction on-the-fly) to have available descriptions of the parts that have to be matched/bridged (the user characteristics, and the content characteristics) in order to more easily customize the desired content: − Description of the content characteristics: information on the content features, which are instrumental to perform an efficie nt customization of that content. Provided by XML/XSL for the text and content structure and MPEG-7 for audio and video. − Description of the user environment: information on the user conditions which are useful to make available a customized variation of the pretended content. We use a generic user environment description (UED) format, which can be translated from/to, other formats (currently WAP User Agent Profile [4], and PocketPC). This paper describes an end-to-end system capable of delivering multimedia content to heterogeneous terminals and access conditions. 1.1
Multimedia Content Description
Describing content is the act of gathering special characteristics about the content that may be used in a later process to easily retrieve, filter or adapt the content in question. Thus a content description is intended to facilitate the access to the pretended content (finding, filtering and universal access). Descriptions focusing retrieval and filtering applications are quite different from descriptions targeting content adaptation applications. The content descriptions, which may help universal access systems, focus on the content structure and encoding characteristics: encoding parameters, region of interests, content object importance, video transcoding hints, and video summarization. Universal access descriptions must provide information that enables the content rendering in any user environment, eventually with transformations in terms of resolution, quality or even modality. There are several frameworks to describe content in order to enable more efficient search and filtering of
content (through indexed descriptions). In W3C there are several projects addressing this topic [3]; the Dublin Core is another work on the topic [3]; and also the Society Motion Pictures Television Engineers (SMPTE), the Digital Video Broadcast group (DVB) [6], and Moving Picture Experts Group (MPEG) [7] are organizations working on this type of technology. Each of these organizations is focusing on their natural technology or application environment, e.g. W3C is targeting Web applications and MPEG audiovisual content. MPEG-7 is a new standard developed by MPEG, which is concerned with multimedia content description at very different levels: it offers a wide range of description tools that range from low-level features such as color and pitch to high-level features such as the name of some characters in a scene. One of the MPEG-7 target applications is universal access to multimedia content, which makes MPEG-7 particularly interesting for the present problem.
2
A Universal Multimedia Access System
The content customization can be implemented in three different places: 1) at the content server, 2) at a proxy server, and 3) at the user terminal. This framework implements the customization engine at a content server and at a proxy server. A complete approach reaching from a high level viewpoint of the UMA problem, down to the low level of the adaptation algorithms, enabled to have a very complete framework to test multimedia adaptation theories. Consequently, this framework has studied the universal access problem at three different levels: − UMA System: at the highest level the endto-end problem was analyzed and a complete framework were designed and tested. See Figure 2. − UMA Platform: this is the network element that performs the content customization. This element aggregates a content customization engine and the required surrounding interfaces. − UMA Engine: this is the lowest level of the problem: the content customization engine that implements the multimedia content adaptation / selection algorithms.
UED server
MPEG-7 description server
UMA Platform Content side network
User side network UMA browser
UMA authoring tool Content server
Figure 2 – Main elements of a UMA System. Figure 2. illustrates the UMA System and its major elements. The function of each element in the UMA System is: − Content server: this element will act as the content source. − UMA authoring tool: this element is used to create/analyze the multimedia content available at the content server and generate an MPEG-7 description that will be saved locally or posted into an MPEG-7 description server. − MPEG-7 description server: this element stores the MPEG-7 descriptions received from the UMA authoring tool. In this database, one MPEG-7 description exists per each URI. This element provides the UMA Platform with the MPEG-7 description for the pretended piece of content. − UED server: this element stores the user environment description (UED) received from the UMA browser. This element provides the UMA Platform with the UED for the pretended user. It implements the same functionality as a WAP UAProf server. − UMA Platform: this element corresponds to the application implementing the content customization required to provide the best experience to the user for the content he/she asked. It will act as a content customization server. − UMA browser: this element includes a Web browser used to access content and allows the user to manage his/her environment description.
The content server, the MPEG-7 description server and the UED server are mere Web servers (Apache Web server), which implement the HTTP POST command allowing other applications to store content in the server. The UMA browser, the UMA Platform and the MPEG-7 description tool (marked with bold) are applications that were implemented in the context of this framework. The UED server contains the base user environment description, which is a static description that never changes; it is the description of a certain terminal model published by the terminal manufacturer. For every content request, the terminal (in this case the UMA browser) sends only the differences relatively to the base UED and a reference of his base UED. 2.1
UMA Authoring Tool
This application enables the user to create a service that can target multiple terminals types (currently WAP and PocketPC). The UMA Platform does not customize the HTML and WML content, instead the UMA authoring tool, Figure 3, generates one variation for each terminal. It implements a content analyzer that scans image and video content, and builds an MPEG-7 description that will be consumed later by the UMA Platform. The content analyzer functionality is quite useful for creating descriptions of existing multimedia content, since it is able to scan the content server local disk and generate the corresponding MPEG-7 description. The generated description can be saved locally or posted (through an HTTP command POST) to an MPEG-7 description server. We are still improving the application for the cases where the user whishes to deliver already existing content for any terminal. For that task we are
Figure 3 – The UMA authoring tool interface. network).
implementing an algorithm, which analyzes the content and allows the author to “say” the importance that each part of the content has when the content is being rendered in each type of terminal. 2.2
Figure 4 shows the main modules of the UMA Platform: − Graphical User Interface (GUI): this module monitors and administrates the interface to the Platform. It is highly platform dependent and only the platform administrator should have access to it.
UMA Platform
As was mentioned before, the UMA Platform implements the content customization engine plus the required modules to interface with a network. The implemented UMA Platform can be configured as a content server or as a proxy server. Several software modules compose the UMA Platform (see, Figure 4): the UMA Engine, the content and descriptions processing modules and the interfacing modules (with the Administrator and with the
− Network Interface Manager (NIM): the Network Interface Ma nager is responsible for the network communications between the UMA Engine, and the other UMA System elements. It is also responsible for retrieving the descriptions. Provides all the UMA Platform
MPEG-7/UED Lib
Video Lib
XML Lib
Image Lib
UMA Engine
GUI
Network Interface Manager
Network Administrator
Figure 4 – UMA Platform building modules.
functions to interface with the network, e.g. an HTTP network, and enables the Platform to be configured as a content server or as a proxy server. − UMA Engine: this is the content customization module; several modules running in parallel compose it. − MPEG-7 / UED Lib: it is a software library that offers an MPEG-7/ UED API to parse and serialize MPEG-7/ UED descriptions. Because the UED and MPEG-7 use the same base technology for description representation (XML Schema), this library is the same for both MPEG-7 and UED APIs. − XML Lib: this is the module responsible for parsing XML data. − Image Lib: this module provides a set of tools to process images. − Video Lib: this module provides a set of tools to process video. 2.3
UMA Browser
The UMA browser allows the user to manage and create his UED. Its primary function is to simulate several terminals and access conditions and test the overall system. The applic ation permits the user to access any characte ristic of the UED and modify the pretended values. The application has the following functionality: whenever the user changes his base UED, the UMA browser sends it to the UED server in a HTTP POST command. When the UMA Platform needs the base UED it requests the UED server. With each user request (an HTTP GET command) goes the differences of the actual user
environment relatively to its base UED. 3
Tests
In order to test the UMA System, the UMA Platform can be configured as a proxy server or as a content server. Figure 5 presents the test-bed used for the content server configuration. The content server based configuration has the advantage to ease the dynamic content adaptation (on-the-fly) because the content is locally available. The content server config uration provides the best solution for a secure environment, where content is usually encrypted. This architecture also benefits real-time streaming environments where the adaptation is done while the converted content is being delivered. In addition to the technical issues, it is necessary to consider the rights management and business implications of adaptive content delivery. In the proxy server based configuration the content is retrieved from many servers: there is content with widely varying appearances, created with many different authoring tools (e.g. Web pages). In these cases the content customization may give results that are quite unpredictable due to the variety of content appearance. The issue of copyright infringement becomes critical in a proxy-based system, since an author may have little control of the content adaptation to be performed (in the case where there is no coordination between the content owner (or provider) and the UMA Platform in proxy configuration). Several types of images and video with the corresponding MPEG-7 descriptions were used for testing the overall system. Four categories of images were selected in order to evaluate the performance of the system under different conditions. See Table 1.
UMA Platform as content server
User terminal 3GPP network
multimedia content
content Content author
MPEG-7 descriptions by URI
WAP UAProf Server
content descriptions
UMA authoring tool
WAP User Agent Profiles
Figure 5 – Universal Multimedia Access test-bed.
Content characteristics Width Height
Image Category A
Image Category B
Image Category C
Image Category D
Video Category A
Video Category B
Video Category C
1600
1024
640
320
352
240
240
1200
768
480
240
288
192
192
Bits per pixel
24 bits
16 bits
8 bits
8 bits
24 bits
24 bits
24 bits
Format
JPEG
JPEG
JPEG
JPEG
MPEG-1
MPEG-1
MPEG-1
Bit rate
-
-
-
-
256 kbit/s
128 kbit/s
64 kbit/s
Frame rate
-
-
-
-
25 fps
15 fps
10 fps
Table 1 – Characteristics of tested content.
User environ. characteristic
WAP Terminal 1
WAP Terminal 2
Pocket PC 1
Pocket PC 2
Hand Held PC 1
Hand Held PC 2
Width
176
176
240
240
640
640
Height
144
144
320
320
240
240
Color Domain
Gray
Color
Gray
Color
Color
Color
Bits per pixel
1
8
4
8
8
16
Min bit rate
64 kbit/s
64 kbit/s
64 kbit/s
64 kbit/s
64 kbit/s
64 kbit/s
Max bit rate
64 kbit/s
128 kbit/s
64 kbit/s
128 kbit/s
64 kbit/s
128 kbit/s
Table 2 – User environment tested conditions. Images range from a high resolution to a low resolution. The goal was to evaluate the influence of large content in the content customization process. For video, three categories were selected, ranging from CIF resolution and 256Mbit/s to QCIF resolutions and 64kbit/s. See Table 2. On the user side, seven descriptions were used to simulate WAP terminals, Pocket PCs and Hand Held PCs with different characteristics and different network connections. See Table 2. 4
Conclusions
From the results it is concluded that in order to achieve a good performance it is vital that previous adaptations can be re-used. Also the diversity of content characteristics that were used to test the system confirmed that the adaptation processes are the main cause of the server load. Therefore, adaptation algorithms optimization is essential. The system also showed that maintenance costs could decrease thanks to the adaptation on the fly and reduction of storage space. When the content source had a very high quality and the terminal limited capabilities the server resources were rapidly consumed. In order to improve these situations, strategic content variations with lower quality that ease the customization reduced the required
adaptation efforts. The presented framework studied the universal access to multimedia content problem: the key factors of a good universal access system concern the user environment description, the content description, the decision algorithms and the adaptation algorithms. UMA technology will not have an answer from one single research area; the answer will come from all these research topics since they are all interdependent. Universal access is a very recent research topic that still holds many interesting aspects which needs to be researched and solved. 5
References
[1]
João Magalhães, “Universal Access to Multimedia Content based on the MPEG-7 Standard”, M.Sc. Thesis, 2002.
[2]
3GPP Home Page http://www.3gpp.org
[3]
World Wide Web Consortium W3C http://www.w3c.org
[4]
Wireless Application Protocol http://www.wapforum.org
[5]
Dublin Core Metadata Initiative http://dublincore.org
[6]
Digital Video Broadcasting http://www.dvb.org
[7]
Moving Pictures Expert Group – MPEG http://mpeg.telecomitalialab.com/
6
Glossary
CIF
Common Intermediate Format
HTTP Hyper Text Transfer Protocol QCIF Quarter-CIF UED User Environment Description UMA Universal Multimedia Access XML eXtensible Meta Language
