Metadata Management Hannu J¨arvinen 51768P
[email protected] January 8, 2007 Abstract This paper is written as a seminar work to course T-111.5550 Seminar on Multimedia at Helsinki University of Technology. The focus of this paper will be kept on multimedia metadata. In addition to general information and necessary facts the paper also discusses about some real life use cases, different standards, downsides and alternatives to metadata.
1
Contents 1 Introduction
3
2 Metadata in multimedia 2.1 Storing metadata . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Metadata lifecycle . . . . . . . . . . . . . . . . . . . . . . .
3 4 4
3 Use 3.1 3.2 3.3 3.4
cases Creating a content description manually . . . Creating a content description automatically Modifying existing metadata . . . . . . . . . Searching content using metadata . . . . . . .
4 MPEG-7 4.1 Introduction . . . . . . 4.2 Components . . . . . . 4.3 Competing standards . 4.4 Critics . . . . . . . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
5 5 6 6 6
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
6 6 7 7 7
5 Downsides of metadata
8
6 Metadata versus content interpretation
8
7 Future visions
9
2
1
Introduction
Generally metadata means data about data. In multimedia it means data about media content. Most common types of metadata in multimedia are for example author, title, format, date and location. There is an actual need for better multimedia metadata management systems. World Wide Web is converting towards Semantic Web containing all kinds of multimedia contents. This rise of the amount of multimedia content on the web as well as local storages requires us to develope better means to search, filter and locate media content in the same way it is now possible to execute search of the textual content using search engines like Google or Yahoo. Metadata is not a new thing, but its meaning and importance will be noticed more in the future because of the movement toward machineprocessable and semantic-based content description on the web.[5] Nowadays there are some de facto standards which support different kinds of metadata. Some examples are TIFF images and MP3 files with ID3 tags. These metadata types are very simple and concentrates on relavely small area of multimedia domain.[3]
2
Metadata in multimedia
Never before have there been so wide use of digital cameras, personal computers and Internet connections around the world. The ways of producing and distributing digital content – multimedia have became generally available. The amount of the multimedia content in personal and public databases is only growing. Somehow all this should be managed so that the content would be easily at hand. We need new kind of metadata standards that could cover this need.[8] Problem seems to be very simple but it is not. Simple metadata structures like ID3 tags are a good answer to a problem when concerning only very narrow area like MP3 audio music files. Nowadays we are looking for an answer to a bigger problem and trying to find a good way to use metadata generally, no matter what kind of multimedia content we are handling. The problem occurs because the domain is too wide for any single standard to give an ultimate solution to it. In a figure 1, there are four main multimedia domains and their subcategories. A general human way to categorize media content is to use content types such as video and audio. Also an other usual way is to use industry domains like music or film. Again user can think content domains to be like news or sports. The fourth domain arranges content by processes and workflows for example creation and distribution.[3]
3
Multimedia Domains Industries
music
film
Processes and creation Workflows Content types Content domains
images
production
video
news
TV
audio
sport
enterprise
management graphics
distribution text
entertainment
Figure 1: Multimedia Domains.
2.1
Storing metadata
Metadata is also data as well as the content that it describes and it has to be stored somewhere. In internal storing the metadata is stored in the same file as the data and is thus also available if the data itself is available. If we have the actual data we can easily manipulate also the metadata. Downside is that we cannot hold only the metadata together but also has to have the actual data which usually means too massive rise of required storage space. In some cases metadata records are not at the start of the file, which also leads to some problems when considering streaming capabilities. In external storing metadata is stored in a separate file from the actual data content. This allows better use of metadata in dedicated metadata management servers. These database servers can then search and store metadata information and only have references to the actual data. Downside is that the multimedia content is not guaranteed to exist and this easily leads to the situation where metadata server contains lots of ghostrecords, that does not have the corresponding content anywhere on the Web available or at least the record can not help to find it. Nowadays many new standards like MPEG-7 and TV-Anytime use XML language as a storing format for metadata. That helps us to manipulate it easier and makes it also human readable without using any dedicated software for the purpose. Old metadata standards were storing it binary like the content.
2.2
Metadata lifecycle
Metadata lifecycle conciders all phases when metadata is been somehow processed. This covers stages from producing to consuming metadata. These different stages have not been given so much attention yet but will most probably get it in the future.[4] Creating metadata is the first phase. Metadata should be created as early as possible because later it can be impossible or very hard to produce it. For example the digital camera should record date and location to the record. If location is not recorded, doing it manually later will be much bigger task to do and it would require human to do it.
4
Figure 2: Multimedia Metadata lifecycle. After the creation the content can be manipulated in many ways. In the manipulation process the metadata should also be changed to correspond to the new data. Manipulation can be combining or attaching two or more multimedia elements together. These elements do not have to represent the same content type. We can build one multimedia presentation from videos, pictures, songs and texts. It would be great if somehow the metadata of all used content could still be available when managing the produced content. We can also manipulate the content without mixing it with other contents. We would for example want to convert videos or pictures to other encoding formats without losing the metadata. This kind of solutions are not in wide use yet. Final phase is the deletion of the metadata. In some cases it would be better to keep the metadata even if the actual data is already deleted. This issue also relates to Digital Rights Management which makes it a hot topic nowadays. Every step between these phases is critical if we are not using compatible standards. Metadata has great value and we can not afford to lose it on the way. In this sector there is a great challenge to things work right. Nowadays if you listen a normal audio CD with a PC you can not normally see even the names of the songs, this is not the way it should be.[4]
3
Use cases
The use cases described in this section are made to correspond the current technical situation with the metadata management. These are daily life usecases which concern normal non-technical people.
3.1
Creating a content description manually
Lets concider a very simple situation about adding a picture to Flickr. A seller in a company has just got new logo versions of their product. Now he wants to add these pictures to Flickr so that others can comment on them before adding to official website. First he browses the pictures and gives common tags for all of the them. After that he writes descriptions for pictures and saves them. Now pictures are available for other users to
5
comment. Pictures can be found using the tags as search strings. Other way is to search first the Flickr member and then browse his pictures.
3.2
Creating a content description automatically
Automatic creation of the metadata is must when it is possible. Here is very simple use case of this when taking a photo with digital camera. A man is on a vacation abroad and having a tourist tour. He wants to have some pictures taken from the sights and uses his digital camera for the purpose. Back home he connects the camera to his PC which retrieves the pictures automatically. After some years he wants to show the pictures to his friends. Now he can find the pictures in subdirectories named by dates corresponding to the shooting dates. One picture looks very nice and some friends are wondering how long was the exposure time. After some guesses the exposure time is checked up from the image properties. Most digital cameras use Exchangeable image file format (Exif) format to store metadata.
3.3
Modifying existing metadata
After watching a good movie a man surfs to IMDb (Internet Movie Database) to read other peoples comments about it and to give a vote. He searches the movie by name and checks the rating. Then after a quick concideration he decides his vote and just clicks the corresponding number of stars. Now his vote will be taken into account when counting the average rate for the movie.
3.4
Searching content using metadata
A man spent his holiday visiting Barcelona and its sights. Back home he decided to build up a multimedia presentation about the trip. He starts to go through the photographs and videos that he took there. Soon he realizes that he does not have any footage from the most important sight in the city, the Sagrada Familia. To build up the presentation he needs to find some footage of this church. To find video material he visits the Youtube service and writes the search string ’barcelona sagrada familia’. He is lucky to find lots of videos and is capable to choose the best videos for his presentation.
4 4.1
MPEG-7 Introduction
One of the promising multimedia metadata standards is MPEG-7 which is also known as Multimedia Content Description Interface. It is the next ISO/IEC standard under development by the MPEG working group. It follows the succesfull development of the MPEG-1, MPEG-2 and MPEG-4 standards. These previous standards were focused on coding and representation of AV content but the MPEG-7 focuses on description of multimedia content. The standard is aiming to provide interoperability to different systems and applications used in all process phases from producing to consuming the audio-visual content discription. Hopefully it becomes a flexible and scalable framework which helps us to use multimedia content with large variety of devices from PCs to mobile terminals.[12]
6
4.2
Components
The MPEG-7 standard is numbered as ISO 15938, and has 7 subparts. These parts are Systems, Description Definition Language (DDL), Visual, Audio, Multimedia DSs, Reference Software and Comformance. The Systems part includes system level functions such as converting descriptions for storing and transportation. The MPEG-7 DDL is a language for creating, modifying and extending Description Schemes and audio-visual Descriptors. Next two parts, Visual and Audio specifies a set of standardized Visual and Audio Description Schemes and audio-visual Descriptors. The Multimedia DS part offers generic description for all kinds of multimedia content, it allows you to define same kind of DSs and Ds as Visual and Audio parts offers by default. Defining own DS or Ds is not necessary because MPEG-7 defines more than 450 metadata types itself[3]. The Reference Software part offers implementation of important parts of the standard such as DDL parser, visual and audio Ds and multimedia DSs. The last part, MPEG-7 Conformance offers guidelines for testing the conformity of the implementation of MPEG-7.[12]
4.3
Competing standards
There are ofcourse many other metadata standards developed and being developed by other communities. Society of Motion Picture and Television Engineers (SMPTE), European Broadcasting Union (EBU) and TV-Anytime are just couple of those. SMPTE/EBU are much alike, they focuse on metadata about the production, broadcasting and distribution of different multimedia contents. Their main focus is on distribution over broadcasting channels. TV-Anytime specification enables AV -services based on users local storage. Like MPEG-7, TV-Anytime also uses XML language for representing the metadata. Its metadata is based on Description Scheme framework and is specified in MPEG-7 Description Definition Language. Some of the tools of MPEG-7 has been adopted to TV-Anytime and there is an effort to harmonize the Electronic Program Guide DS. Still TV-Anytime will contain also non-MPEG-7 tools. Main thing is that TV-Anytime offers solution to much more narrow domain than MPEG-7. It focuses on applications such as personal video recorders (PVR) in an efficient way but is not suitable for wider use. TV-Anytime forum is not active anymore but finished its work in 2005.[12][3]
4.4
Critics
It looks like MPEG-7 is the standard to be used in the future. Still some criticism has been leveled against it. It is said that the freedom in terms of structures and parameters is such that generically understanding MPEG-7 produced by others is quite difficult[8]. Another thing is also that MPEG-7 tries to be the cure to the problem only by itself but it has been thought that the multimedia domain is too wide for any single standard to address it[3].
7
5
Downsides of metadata
It is easy to see the good side of the existance of the metadata but there are also some disadvantages. Where should we draw the line what to include in the metadata and what to keep out. Ofcourse it is good to have a name of director of the movie in a metadata record but what about the actors. One might say that ofcourse it should include the names of the main actors but then what about other actors in smaller parts, what if we have thousends of characters in the movie. The second problem also relates to the selection of the metadata to store. Large amount of metadata makes it very expensive. It is not only the data size that grows up, but the work that has to be made do generate all the metadata. Not all the multimedia producers are willing to make metadata records without an extra cost. Some people also say that metadata is useless because of the search engines that we have. Searching text works fine today and tomorrow will bring us techniques to search pictures, video and audio the same way, by giving an example. This is discussed in more detail at the next chapter.
6
Metadata versus content interpretation
Although the amount of the multimedia available is rising up we are not necessary forced to use any kind of metadata. An alternative approach is to use content interpretation. With a textual content this certainly is not anything new, actually it is just the thing that generally used search engines do. Development of the same kind of search engines for images and audio has begun. In some cases there already exist some protype applications for such a purposes. ImageRover for example is a content-based image browser for the World Wide Web.[2] In the same way that textual search engines do, the indexing of the images around the web is done by many robots that are connected together. This means that we do not need a super computers to do calculations but we are using a distributed architecture. Content interpretation is a very interesting and evolving domain but is hardly competitive to metadata management. Both of these techniques will be needed in the future and should not be forgotten. Still it is much easier for human beings to write textual input to search field than try to hum or draw the target content. Instead of that, machine to machine communication on the web can benefit more from these kind of applications.
8
7
Future visions
Metadata management will be on a key role to lead the way to the Semantic Web. Future vision is that it gives users and machines possibility to have much better understanding of the semantics of the data contents in the Web. Important is that this allows machines to communicate with each other without any additional help from humans. This scenario has been one factor of givin birth to a new term – Web 2.0 which has been under a lot of critics whether it is only a buzzword or has actual meaning and value. Some of the developed standards presented in this paper will be used and some of them will be discarded. Which one of those will be in use, that we can not predict. That will be decided by the developers and the users.
9
References [1]
Milind M. Buddhikot , Guru M. Parulkar , Jerome R. Cox, Jr., Design of a large scale multimedia storage server, Selected papers of the annual conference on Internet Society/5th joint European networking conference, p.503-517, December 1994, Prague, Czech Republic
[2]
ImageRover: a content-based image browser for the World Wide Web Sclaroff, S.; Taycher, L.; La Cascia, M.; Proceedings IEEE Workshop on Content-Based Access of Image and Video Libraries 20 June 1997 Page(s):2 - 9
[3]
Metadata standards roundup Smith, J.R.; Schirling, P.; Multimedia, IEEE Volume 13, Issue 2, April-June 2006 Page(s):84 - 88
[4]
The life cycle of multimedia metadata Kosch, H.; Boszormenyi, L.; Doller, M.; Libsie, M.; Schojer, P.; Kofler, A.; Multimedia, IEEE Volume 12, Issue 1, Jan.-March 2005 Page(s):80 - 86
[5]
That obscure object of desire: multimedia metadata on the Web, Part-1 van Ossenbruggen, J.; Nack, F.; Hardman, L.; Multimedia, IEEE Volume 11, Issue 4, Oct.-Dec. 2004 Page(s):38 - 48
[6]
That obscure object of desire: multimedia metadata on the Web, part 2 Nack, F.; van Ossenbruggen, J.; Hardman, L.; Multimedia, IEEE Volume 12, Issue 1, Jan.-March 2005 Page(s):54 - 63
[7]
Multimedia annotations on the semantic Web Stamou, G. van Ossenbruggen, J. Pan, J.Z. Schreiber, G. Smith, J.R. National Tech. Univ. of Athens, Greece Appears in: Multimedia, IEEE Jan.-March 2006 Volume: 13 , Issue: 1 On page(s): 86 - 90
[8]
Toward a scalable multimedia metadata infrastructure using distributed computing and semantic web technologies Asirellil, P.; Grazia Di Bono, M.; Martinelli, M.; Salvettil, O.; Signore, O.; Catasta, M.; Morbidoni, C.; Piazza, F.; Tummarello, G.; Integration of Knowledge, Semantics and Digital Media Technology, 2005. EWIMT 2005. The 2nd European Workshop on the (Ref. No. 2005/11099) 30 Nov. - 1 Dec. 2005 Page(s):153 - 156
[9]
Study on Personalized Data Broadcasting Service using TV-Anytime Metadata Yong Ho Kim; Han-kyu Lee; Jin Soo Choi; Jin Woo Hong; Consumer Electronics, 2006. ISCE ’06. 2006 IEEE Tenth International Symposium on 28-01 June 2006 Page(s):1 - 6
[10] Metadata - the role of the TV-Anytime specification Morecraft, C.; Storage and Home Networks Seminar, 2004. The IEE 3 Nov. 2004 Page(s):32 - 40 [11] MPEG-7 metadata for video-based GIS applications Tae-Hyun Hwang; Kyoung-Ho Choi; In-Hak Joo; Jong-Hun Lee; Geoscience and Remote Sensing Symposium, 2003. IGARSS ’03. Proceedings. 2003 IEEE International Volume 6, 21-25 July 2003 Page(s):3641 3643 vol.6cd [12] Overview of the MPEG-7 standard Shih-Fu Chang Sikora, T. Purl, A. Dept. of Electr. Eng., Columbia Univ., New York, NY; Appears in: Circuits and Systems for Video Technology, IEEE Transactions on Publication Date: Jun 2001 Volume: 11, Issue: 6 On page(s): 688-695
10
