The Future is ‘Ambient’ Artur Lugmayr 1 New AMbient MUltimedia Group (NAMU) Institute for Signal Processing, Tampere University of Technology POB. 553, Korkeankoulunkatu 1, FIN-33101 Tampere, FINLAND ABSTRACT The research field of ambient media starts to spread rapidly and first applications for consumer homes are on the way. Ambient media is the logical continuation of research around media. Media has been evolving from old media (e.g. print media), to integrated presentation in one form (multimedia – or new media), to generating a synthetic world (virtual reality), to the natural environment is the user-interface (ambient media), and will be evolving towards real/synthetic undistinguishable media (bio-media or bio-multimedia). After the IT bubble was bursting, multimedia was lacking a vision of potential future scenarios and applications. Within this research paper the potentials, applications, and market available solutions of mobile ambient multimedia are studied. The different features of ambient mobile multimedia are manifold and include wearable computers, adaptive software, context awareness, ubiquitous computers, middleware, and wireless networks. The paper especially focuses on algorithms and methods that can be utilized to realize modern mobile ambient systems. Keywords: bio-media, ambient media, ubiquitous computation, pervasive computation
1. INTRODUCTION Ambient intelligence deals with the embedment of media in the natural environment of people. Technology should be hidden from the people’s eyes and intuitively to use. Complex user-interfaces and interaction modalities are envisioned to be part of the past. However, we are still a few years away from the realization of systems with predominant speech interfaces, gesture recognition, and smart technology that knows the consumers’ desires. But the direction to create media systems, where the central question is ‘How can systems smartly know what the consumer desires and act autonomously on his behalf?’ is the right development. Today’s technology is by far to complex to use and dominated by the consumer question ‘How can I command the technology that I can do or get what I like?’. The European Union expert group for the definition of EU wide research goals (ISTAG) defined therefore technology for ambient intelligence as research goal until 2010. European wide several research projects have this as a common vision [1, 2]. C2000
2000
2010
'2020
Figure 1: Evolution of media. Old media where predominant till 2000, but digitalization let a new media form emerge: new media. New media descended is dominated by pervasion the natural environment of people with media associated with natural interaction. Ambient media will be dominating around 2010, but processification (the introduction of biological processes in media technology) will lead to a new form of media: bio media. 1
Artur Lugmar,
[email protected], mobile: +358 40 821 0558, www.lugy.org, www.digitalbroadcastitem.tv Multimedia on Mobile Devices II, edited by Reiner Creutzburg, Jarmo H. Takala, Chang Wen Chen, Proc. of SPIE-IS&T Electronic Imaging, SPIE Vol. 6074, 60740J, © 2006 SPIE-IS&T · 0277-786X/06/$15
SPIE-IS&T/ Vol. 6074 60740J-1
1.1. The Need for Creating a New Form of Media: Ambient Media One thing has to be considered – embedding media naturally into the peoples’ environment indicates the need for the creation of a new form of media. A form of media is determined by its underlying technology. The form of printing media has been made possible by the invention of typesetting by Johann Guttenberg in 1448 [3]. The form of new media has been made possible by the developments of information technology during recent decades (including e.g. the Internet, the Web, and electronic media). Ambient media are implicating as well a new form of media. These media are not anymore based on single objects compiled together (such as e.g. images and text on a web-page) from a central database system. Ambient media create a mix of real-world objects in the natural environment of the consumer and digital objects which are of any arbitrary form.
PULL
The consumer explicitly selects the content that he would like to enjoy
AMBIENT
The consumer configures either explicitly or implicitly devices that automated aggregate the content for him
PUSH
The consumer obtains a filtered list of content from his content aggregator
Figure 2: Ambient media differ from pull and push media. In pull media the consumer explicitly selects the content that he would like to enjoy (e.g. the Internet). In push media a filtered list of content is delivered from his content aggregator (e.g. digital television). In ambient media technology automatically aggregates the content for the consumer in a smart way (e.g. intelligent fridge).
The topology of ambient media is distribution, rather than storage of media in a central database system. In ambient media, the analogue world mixes with the digital world. The central point is that the technology of these media is more and more hidden and naturally to use. One example is the intelligent fridge that is communicating to the consumer in the supermarket which food is needed to cook a certain dish. The media is a mix of services: an online shop offers cooking recipes, networking enables communication between different entities, RFID readers in the fridge read the food that is available, RFID tags in shops guide the consumer to the right products, the consumer interacts with his mobile phone – or simply with a monitor on his shopping car. This scenario is already becoming true for some departments stores, as e.g. in the Metro FutureStore [4]. 1.2. Central Questions of this Research Paper This research work is attempting to answer the following research questions: o o o o o o
What is the media? What distinguishes ambient media from new media? Which form of media are ambient media? How do their basic technical components look like? What might be after ambient media? What are a few ambient example services?
1.3. The Business Viewpoint A media business has two products for selling: “its content (to readers and viewers); and its audience (to advertisers)” [5]. Before elaborating about new forms of media, let us consider a few facts currently happening in the entertainment industry:
SPIE-IS&T/ Vol. 6074 60740J-2
o
o
o
o
o
Lack of risk-taking: old media producers to not understand the world of new media. As far they are still convinced that the new media bubble was bursting in 2000. They do not understand that a convergence between old media and new media has to take place to be competitive. New media do not understand old media – however – they are quicker in catching up (e.g. AOL bought Time Warner) and stuff up their media departments; New media are like silent-films: new media are mostly unexplored – they are simply like black and white silent films a century ago. Many new paradigms enriched media towards ambient media: interactivity, natural interaction, making technology smart, sociable technology, etc. What can be made out of these media forms is still part of the work of media artists and creative content creators; Media sociability: media are not anymore from single sources. Sociable media, where many contribute to create the content are gaining importance. Examples of this trend are web-blogs, collaborative databases (e.g. IMDb [6]), online gaming, etc.; The myth of piracy: lack of understand how new media work is the cause for falling shares of old media companies, rather than piracy. It is simply the comfort to download files from the PC and copy it to multimedia devices that made ‘illegal downloading’ a success. The too late introduction of legal downloading (thus creating comfort for the consumer for buying digital content) such as Apple’s iPod [7] services was the major mistake of the content industry. Nevertheless, illegal downloading is still stealing; Service spaces rather than single products: one movie, one music record, one book created revenues in the times before new media. But consumers expect more, they expect a portfolio of products that are relating to the actual product. A service space consisting of web-portals for fans, active consumer participation in creation processes, email lists, variability of the product (e.g. iPod), fast access to digital products, mobile phone ringtones, etc.;
1.4. Bio-Media – Predictable Descended of a New Form of Media after Ambient Media Digitalization of media dealt with the processing of media with information technology. Pervasion embeds media into the natural environment of humans. Processification of media deals with finding methods that let media transform and effect a biological object. In more general context processification deals with the finding of “any method […] that results in a transformation in a […] biological object, […] or an organism” [3]. The processification of media will enable the creation of synthetic worlds, which are fully undistinguishable from the real world. The human would be fully immersed with his senses into the media experience – the media integrates human capacity. One example is brain interfaces [8], whose development is nowadays in a premature stadium. These interfaces allow people with special needs to overcome their handicaps partially. But why not using this technology for creating a new form of media? Why not using this technology in entertainment? 1.5. Related Works There are many related works available covering the topic ‘New Media’. One of the most comprehensive works has been undertaken by LEV MANOVICH in his book ‘The Language of New Media’ [9]. However, his viewpoint on new media is more from the film side, which represents the current dilemma of the content industry: the content industry (e.g. movie producers) start slowly to understand new media and the new media industry (e.g. Yahoo!) slowly starts to converge with old media. The most comprehensive description of ambient media – or ambient intelligence – can be found in the ISTAG reports published by the European Union in [1, 2]. Bio media have been introduced in [10], and more about the future of broadcast media can be found in [11].
2. WHAT IS AMBIENT INTELLIGENCE? – WHAT MAKES MEDIA AMBIENT? Ambient intelligence is a further development of the field of ubiquitous computation, pervasive computation, and multimedia. Ambient intelligence is a more systematic approach towards the mentioned research fields. Ambient intelligence research covers complete platforms, development methods, sociological aspects, society developments, among many other features. Ambient intelligence is “a condition of an environment that we are in the process of creating” [1, 2]. The place for ambient media is an ambient intelligent space that is “ […] composed of collaborative (location or social based) subspaces of devices (including sensor and actuator systems), services (including their interfaces) and the connecting networks.” [1, 2]. Ubiquitous computation as well as pervasive computation is focusing on
SPIE-IS&T/ Vol. 6074 60740J-3
single features in hiding technology from people and making it smart. Ambient intelligence is more a top-down approach. Def. Ubiquitous Computation: “Ubiquitous computing has as its goal the enhancing computer use by making many computers available throughout the physical environment, but making them effectively invisible to the user. A number of researchers around the world are now working in the ubiquitous computing framework. Their work impacts all areas of computer science, including hardware components (e.g. chips), network protocols, interaction substrates (e.g. software for screens and pens), applications, privacy, and computational methods.” [12] Def. Pervasive Computation: “ […] computers and sensors ‘everywhere’ in devices, appliances, equipment, in homes, workplaces and factories, and in clothing. Another element involves pervasive communication - a high degree of communication among devices and sensors through a ubiquitous and secure network infrastructure with a wired core and wireless adjuncts that communicate with the core […]” [13] A form of media describes their style, representation, and presentation. The form of media is determined by the underlying technology. Ambient media are therefore a form of media that is based on ambient intelligent technology. 2.1. Five Principles of Ambient Media In the following the five principles of ambient media as new form for media are enlisted: 1.
2. 3. 4. 5.
ambience: a media object is part of the natural environment and must not essentially be numerically represented – the media object is a service oriented asset space consisting of single services, content, networks, and pervasive technology; intelligence: the media is aggregated by technology in an automated way. It is less a questions of push/pull media, it is more a question of smart technology for content aggregation; distribution: the media itself is distributed across a wide area of networks. A central database can be used for storage, but it is not essentially the single source for the media; self-organization: the media is aggregated from many various distributed sources either automatically (via smart technology) or collaboratively (sociability) of knowledge); pervasiveness: technology becomes invisible part of our natural environment and media are accessible anytime, anyhow, anywhere, and when the consumer desires;
2.2. Differences between Ambient Media and New Media The five principles of new media are [9]: numerical representation, modularity, automation, variability, and transcoding. Ambient media go a slightly different approach. A numerical representation is not essential – media can be also represented analogue, as one principle of ambient media is ‘ambience’. One example is the Stock Orb developed by Ambient Devices Inc.[14]. The Stock Orb glows in different colors according the ‘health’ of the stock portfolio (red for low share prices; green for high share prices). Modularity – media are composed from different objects, where single objects keep their independence (see [9]) – is only limited valid. Ambient media and the components of ambient media are not essentially digitally representable. This restricts the possibility of composing new media objects out of single media objects. Ambient media are focusing on an asset space, which is a space consisting of services, content, networks, and pervasive technology. One example for such an asset space is a TV broadcast, which is distributed over many different channels, accompanied by an internet chat forum, additional broadcast description in an online movie database, SMS shopping services, etc. All these components contribute to the media. Ambient media has therefore more a service oriented viewpoint. A high degree of automation, where media can be altered either through computer systems or human interaction is present in ambient media as well as new media. In ambient media systems two additional features are predominant: selforganization, and intelligence. Self-organization also extends towards collaborative media aggregation besides the introduction of smart technology for media aggregation. Both additional features contribute to the asset space. The principle of variability that media exist in many different versions is also valid for ambient media. But there is one major difference: ambient media do not rely on a database approach for compiling media. Ambient media are distributed and self-organizing. One example for distribution are knowledge databases such as Wikipaedia [3], which is accessed to complete one service in the asset space (e.g. the consumer watches TV and would like to get additional
SPIE-IS&T/ Vol. 6074 60740J-4
information about some keywords). One example for self-organization are collaborations that are contributing to online dictionaries. With the last principle of new media, transcoding, LEV MANOVICH was mostly addressing society changes due to the introduction of computers into media systems. However, ambient media substantially change this paradigm and add much more discussion around societal changes. These include privacy issues, sociability of information technology, pervasiveness, etc. 2.3. Technology Determining the Form of Ambient Media Ambient intelligent system consist of two essential components: the ambient component addresses hardware technology as part of the natural environment of people; and the intelligence component defining software technology and the algorithmic component.
Components
Ambient
Intelligence
• Smart materials
. .
Micro/Nano-electronics/systems Smart materials • MEMS technology & sensor technology • Embedded systems . Ubiquitous Communication • I/O device technology . Adaptive software
• • • • •
Media management & handling Natural interaction Computational intelligence Contextual awareness Emotional computing
Platform design Software design, engineering and integration Experience prototyping Figure 3: The different components of ambient intelligence according ISTAG. The ambient component addresses hardware technology part of the natural environment; the intelligence component defines software technology and the algorithmic component. Besides ambient intelligence as well includes platform design; software design, engineering, and integration; and experience prototyping.
The ambient components defined by ISTAG as “a system/environment/background view, including sensor technology obtaining data about the physical world, real-time embedded systems, ubiquitous communication ranging from pico-networks to mobile broadband access, I/O devices supporting touch-less control, and adaptive software” [2]. The intelligence component is defined by ISTAG as “user/person/foreground view, that is including media management and handling, natural interaction, computational intelligence, contextual awareness, and emotional computation” [2].
3. EXAMPLE APPLICATIONS OF AMBIENT MEDIA Within the scope of this chapter a few examples of ambient media are stated. Firstly, the goal of the NAMU research group – to develop the ‘Digital Aura’, a mobile ambient entertainment platform – is presented; Second, a few ambient example project from other institutions are collected and their key-features listed. 3.1. Development of the ‘Digital Aura’ The research efforts around the NAMU research group evolves around the question: ”what can be done with signaling information obtained from sensor networks around the mobile user in his natural environment to enable technology to act on the consumers’ benefit?”.
SPIE-IS&T/ Vol. 6074 60740J-5
The consumer shall be supported in his natural environment by smart wireless devices knowing the surrounding environment, knowing the consumer behavior, and knowing possible sources for media. The consumer shall be enabled to interact naturally with his mobile device (e.g. via speech), as well with his natural environment in a natural way. The technology required to enable such a form of media services, is smart middleware technology, metadata, and adaptive services. The requirements to a digital aura enabled service are [15]: o o o o o o o o o o o
non-intrusive body parameter monitoring; natural interaction models between humans, and humans and devices; mobile phone technology as enabler; freedom of unavailability; need for the definition of an abstract reference architecture; metadata shall act as catalyst for integration, adaptation, media management, media tagging; need to develop a set of methods and techniques for algorithms, intelligence, awareness (location & content); development of suitable new media forms for deployment on ambient systems; smart signal analysis methods development; development of computational demanding algorithms for distributed ambient nodes; proof of the ambient platform on an example service scenario; • •
Intelligant metadata Processing Distributed compurabonal intelligence
• Smart media algorithms
I
Signal data analysis
Figure 4: The ‘Digital Aura’ is a mobile ambient platform. The platform shall enable several ambient features such as natural interaction, mobile smart media, or pervasiveness of media. Many various different components have to be researched and developed. This figure gives an overview of soft-components, hardware technology, and methods & algorithms that require substantial research.
3.2. Compilation of Ambient Example Projects Table 1: Example projects (compiled in [16], where also Ref. to projects can be also found)
Project Home care Wall-screen display Virtual museum guide
Key-features Smart monitoring of body parameters and interaction with health care personnel Interaction between humans via wall-screen displays over wireless channels Location based intelligent museum guide leading the consumer through a museum
SPIE-IS&T/ Vol. 6074 60740J-6
The Rover Maria Road Warrior
Dimitrios and the digital me Carmen Smart space Selective availability Oxygen AceMedia Amigo
Digital assistant keeping track of the visit of a tourist in another place by creating a digital visit album Personalized communication device acting as substitute for PC, laptop, PDA and other electronic devices during travels as well as environment personalization and identifier device Digital-Me as device performing communication between the consumer and other consumers via a virtual avatar Automatic Debiting System (ADS) Triggered emergency messages by a emergency management agency Freedom for being unavailable for some sort of communication Pervasive, embedded, nomadic, adaptable, efficient, intentional, and eternal human-centered computer system Integration of knowledge, semantics and content for human centered media by a three layered approach (content, metadata, and intelligence) Device interoperability in a home-entertainment centre
4. CONCLUSION While reading this paper there will be a lot of controversy around the addressed topics. As already my first paper around bio media [10] was causing huge discussions around the ‘creation of an artificial human’, also this paper will cause many discussions. But I am glad, that it is still possible to create emotional discussions around scientific topics. Many previous conferences ended with just a few ‘polite questions’ by the conference chair. I saw also only a few conferences where hot discussions around research topics took place. I hope that this paper creates discussion – discussion about the future of media! The more topic related conclusion of this paper is: o Ambient media is the future of multimedia; o Ambient media unifies pervasive computation and ubiquitous computations; o Ambient media faces the challenge of hiding technology from the consumer and making it simple to use; o Ambient media are between push/pull content, where technology aggregates the content FOR the consumer; o Ambient media allow natural interaction and easy access to content anytime, anywhere, and anyhow; o After ambient media, there will be a new form of media – and it will be biological aspired (bio media);
REFERENCES 1. 2. 3. 4. 5. 6. 7. 8. 9. 10.
11.
12.
ISTAG, Scenarios for Ambient Intelligence in 2010 - Final Report (Feb. 2001). 2001. http://www.cordis.lu/ist/istag.htm. ISTAG, Ambient Intelligence: From Vision to Reality. 2003, European Union: IST Advisory Group. Wikipedia, Wikipedia - The Free Encyclopedia. http://www.wikipedia.org. Metro Group, Metro Future Store. http://www.future-store.org/servlet/PB/s/4gnx9217np1885p55rx1el05lc15rn1i5/menu/1007054/index.html. The Economist, King content, in The Economist. 2006. IMDb, Internet Movie Database. http://www.imdb.com/. Apple Inc., iPod. http://www.apple.com/ipod/. TU Graz, Laboratory of Brain-Computer Interfaces. http://www.bci.tu-graz.ac.at/. Manovich, L., The language of new media. 2001, Cambridge, Mass.: MIT Press. xxxix, 354 p. Lugmayr, A. From Ambient Multimedia to Bio-Multimedia (BiMu). in EUROPRIX SCHOLARS CONFERENCE. 2003. Tampere, Finland. http://www.acten.net/cgibin/WebGUI/www/index.pl/newsletter11?wid=509&func=viewSubmission&sid=792, www.lugy.org. Lugmayr, A., S. Niiranen, and S. Kalli, Digital Interactive TV and Metadata - Future Broadcast Multimedia. 2004: Springer New York. http://www.springeronline.com/sgw/cda/frontpage/0,10735,5-175-72-265939880,00.html. Weiser, M., Hot topics-ubiquitous computing. Computer, 1993. 26(10): p. 71-72.
SPIE-IS&T/ Vol. 6074 60740J-7
13. 14. 15. 16.
NIST, Pervasive Computation, Definition, http://www.itl.nist.gov/pervasivecomputing.html. http://www.itl.nist.gov/pervasivecomputing.html. Ambient Devices Inc. http://www.ambientdevices.com/cat/index.html. A. Lugmayr, T. Saarinen, and J. Tournut, The Digital Aura - Ambient Mobile Computer Systems, in to be published at the PDP2006 in Montbéliard-Sochaux - France. 2006. Holmlid, S. and A. Bjorklind, Ambient Intelligence to Go - AmIGo White Paper on mobile intelligent ambience. 2003, Linkopings Universitet, Santa Anna IT Research Institute, Prevas AB, Technology Nexus AB, and Ericsson. www.santaanna.se/projectbriefs/AmIGo_WhitePaper.pdf.
SPIE-IS&T/ Vol. 6074 60740J-8
