Implicit User-Generated Content in the Age of ...

1

Bits of Personality Everywhere: Implicit User-Generated Content in the Age of Ambient Media Artur Lugmayr1, Simon Reymann1, Stefan Kemper1, Tillmann Dorsch1, and Pablo Roman1 Member, IEEE, ACM 1

Tampere University of Technology (TUT), NAMU (New AMbient MUltimedia) Lab., Tampere, FINLAND http://namu.cs.tut.fi, [email protected]

ABSTRACT. This article approaches the way how to think about collaboration with pervasive technology in a new way. It introduces ambient media as new form of media, where the media as such is linked to physical worlds, and smartly aggregated rather than generated with a push or pull model. Collaboration between human man and technology becomes a issue of collaboration, rather than a question of human-computer interaction. As far user contributed content was mostly seen as process, where consumers contribute to online Web forums or create their own videos and upload it to YouTube. The user actively contributes and collaborates to create the content of the media. However, pervasive technology enables much more: pervasive technology allows implicit collaboration, thus the collection of consumer data passively and aggregate the content of the media. One example is the open source platform Portable Personality (P2), whose goal is to generate a personality profile of the implicit collected contextual information. P2 will offer the possibility of personality profiles, rather than simple usage history information or sensor data input. Within the context of this article, the notion of ambient media is introduced. As a practical example to underline the new way of thinking about media, P2 is introduced and the underlying principles explained. Index Terms— Computer applications, Multimedia computing, User interfaces, Intelligent systems

I. INTRODUCTION

I

MAGINE you are standing in the middle of New York’s Time Square. Imagine all the mesmerizing advertisement media delivering their messages to you – penetrating you with their strong colors, blinking, huge letters, sound effects or animations. Now imagine a mobile device, permanently sending you messages about the nearby restaurant special offers; that friends that you added years ago to your messenger account are in the approximate of your mobile, thus you; and the ubiquitous body-sensors sending you messages to your mobile computer, that your heart-rate is too high due to your stress level. Now, imagine the promises of pervasive computation that you just heard a few years ago: sensor networks in your mobile collect implicitly data about you, your location and environment, and offer you services that are in the proximity of you. Everything implicit, thus the machine knows your desires and personalizes the collected data for your benefit. The machine collaborates with you actively, rather than relying on human-computer interaction strategies. Thus user-contributed content becomes a matter of collaboration with other humans, between devices, and between device and human. Especially implicit usercontributions, where the system collects data about the consumer without his active participation to aggregate content in his behalf shall ease the collaboration with the machine. The waiter in the restaurant that you are just now visiting on New York’s Time Square reads your nutrition preferences from your mobile with a nutrition reading device and offers you a personalized meal. And now, imagine, you won’t have such a pervasive sensor data hungry device and are sitting in the middle of a forest in Scandinavia: silence, peacefulness, and no disturbances. Your nutrition is determined by the catch from this afternoon’s fishing trip. Which future you prefer? Nevertheless, let’s focus on the topic of the article, and elaborate societal and philosophical issues in the conclusion

section. II. AMBIENT MEDIA User-generated content is nowadays a buzz word, as well as Web 2.0 is. To further elaborate the key thesis of this article, it is important firstly to understand the concept of ‘content’. Many use content as synonym for ‘digital entities’ such as video or audio streams. But the idea goes far further – content “is the information and knowledge contained in something […] distinct from its form or style” [1]. Content therefore is the information rather than the bits and bytes that contain this information. And collaborative exchange of information exists since human man-kind exists. Without the collaborative exchange of information and the relevant technology therefore, the building of social structures and first settlements would purely be possible. It’s not the video stream or digital object which is the content. The video stream is only the media for the information delivered to the consumer. And the media evolved with the emergence of new media technologies.

FIG. 1 Ambient Media and the Relation between Digital Overlay and Physical World In times of pervasive computation, where computational technology is seamlessly hidden in the natural environment of

2 the consumer, the definition for media altered towards “conveying collaborative experiences in the natural environment enveloping the human as either a mode of artistic expression or factual communication under certain aesthetics rules” [2]. For within the scope of this article the term ambient media is used as synonym for pervasive media or ubiquitous media, as ‘ambient’ characterizes the embedment of the digital overlays within the natural environment in a better way. Ambient media are not purely technology focused, but also include theories coming from aesthetics, art, user experience, and even architecture. The goal is to provide a multidisciplinary viewpoint towards the realization of our future living environments. One example are smart homes, considering besides technological aspects also well-being and comfortable living experiences. However, the technology enabling the new forms of media – let them be referred to as ambient media, pervasive media, or ubiquitous media – is based on pervasive and ubiquitous technology. The technology is based on miniaturized electronics, smart sensor networks, and machine learning algorithms in processing units. The intelligence of the digital system becomes part of a distributed mesh of computational nodes collaborating with each other and with the consumer to exchange information. We even can go one step further, and describe ambient media as a new form of media, which are the “particular way in which [entities] exist or manifest themselves, morph the natural environment entities with the synthetic artificial created world, collaborate with each other, and intelligence of arrangements (composition) and their contextualization and their sub-components as an artistic or factual genre for the generation of human experience”. Without further elaborating the complex definition of ambient media as new form, let’s compile the principles of ambient media, before explaining the details within the scope of this article. The principles of ambient media are: • morphing • manifestation • intelligence • experience • collaboration

GPS location information. This principle can be called morphing. Morphing describes the connectivity – the mapping process between the real-world and the digital overlay network. In the case of tagging location information to photographs, it’s the GPS location. The location information maps the physical world with the information contained on a web-site. Some might argue, that the principle is similar to Tangible User-Interfaces are another example for the mapping between both worlds – TUIs map physical objects to the digital world [3].

A. Linking the Physical World with the Digital Overlay At first we need to connect worlds – the digital overlay world and the physical world. Until now, media where part as single entities in real physical worlds. In the future, anything is a connected and collaborating media – the smart light-bulb, the fridge, the wallpaper, or the coffee cup. Thus, there exist two levels: the physical real-world entities, and the digital world as overlay. It is of importance to interconnect these two worlds that they seamlessly integrate. There exist nowadays many examples for this overlay relationship as described above: the consumer can take photographs with his mobile phone, and upload these to a web-site. Each of them photographs is tagged e.g. with GPS location information. Thus, the connectivity between the physical world and the digital overlay world is mapped via

D. Implicit User Generated Content as User Experience User-generated content is a matter of collaboration. Consumers collaborate to generate new content in any imaginable form. Thus in this context, collaboration means to work in cooperation to achieve a specific goal, thus the process of collaboration can simply described as the “actions of working with [something] to produce something” [1]. In the case of ambient media, collaboration addresses inter-device communication, inter-human communication, as well as computer mediated collaboration. Devices become ‘collaborators’ rather than tools and support the human in achieving his goals in an intelligent matter. However, also devices need to collaborate with each other and integrate seamlessly to be able to cope with given challenges and ease the life of humans.

B. Stimulation of Human Senses by Pervasive Technology However, the digital space in times of pervasive computation becomes an overlay of virtual world onto the real world. Even better, it becomes to a relation between realworld objects and its digital overlay. The connectivity between both layers is one of the principles of ambient media – manifestation. One very simple example for manifestation is the ambient orb, developed at the MIT [4]. The orb turns red, yellow, or green – depending on the status of its own stock portfolio. Digital numbers turn into easy human graspable information. For a person only interested in the status of its portfolio, rather than being interested in playing with shares this visualization (manifestation) of the digital world is sufficient. C. Intelligent Objects and their Smart Aggregation One very important change in ambient media is the change towards media aggregation, rather than push/pull schemes. In pull schemes, the consumer has to search explicitly for content – e.g. by surfing the web. In the case of television, the content is pre-edited by an editor and broadcasted to the consumer. The consumer can only anticipate indirectly in the creation process in push schema content aggregative models, e.g. via customer polls or voting. In the case of ambient media, the aggregation shifts towards the system. The system compiles content in behalf of the consumer – the content is contextualized to situation, environment, personal preferences, and importance. One simple example are reminder messages which are displayed, when while watching a TV show the smart home PC discovers an exciting web-site contributing to the watching experience.

3 IIII. PORTABLEE PERSONALITTY (P2): THE DIGITIZ ZED PERSONAL LITY User-generatted content is a matterr of collabo oration. mans, collabooration takess place bassed on Between hum t to the medium m film. In I film, peersonality. Leet’s compare this thhe key challeenge for the director is to create em motional biinding between the audieence and the story he teells [5]. Em motional bindding is based on o the principple, that the auudience ‘ffeels’ with the major characcter. As far devicees are tools, raather than coooperating partnners, as huumans wouldd be. The keyy-idea is to create collabboration baased on persoonality by emootionally bindding the consuumer to deevices. Thus devices d need to emotionallly respond to human innputs – they need n to have ppersonality! In n the case of ambient a m media, where devices d becom me smart, and active collaboorators, it is a question of develooping the riight algorithm ms and techniques. Hoowever, theyy will be em merging soonn. One i a cat exxample is the iCat [6]. Thee iCat is a deevice, which is thhat can be conntrolled by a consumer syystem. Based on the im mplementation n, it creates emotional e feeddback and binnds the coonsumer makiing use of thiss toy. The basic daata-structure needed, is a sort of digitalizaation of peersonality. Feeatures such as personalitty traits, the human m mind, activities of persons, the spectruum of understanding neeed to be ‘ddigitalized’. One O approachh which is cuurrently unndertaken by the NAMU Lab. at thee Tampere Univ. U of Teechnology (T TUT) is thee open source forum Portable P Peersonality (P P2) and the Emotional Ambient A Respponsive Ch Character (EA ARC). Withinn this and thhe next sectioon, the unnderlying ideaas are presenteed.

surrrounding. In other words, based on con ntextual inform mation, thee personality profile, and other data existing e as a priori knnowledge abouut the consum mer or the env vironment, actions in thee real world caan be triggereed.

FIG. 3 Porrtable Personaality (P2) as layered l system m of provider, con p nsumers, and daemons to bridge the diigital and thee real-world One examplee is, that the coonsumer just paid p a bus ticket via thee RFID tag innside the mobbile phone, an nd the mobile phone acccelerometer sensors measuure movementt, the state woould be ‘inn-bus’. This sttate can be deelegated to vaarious mobile phone appplications, leaading e.g. to a reduction of ring-tone voluume or inccrease of louddspeaker volum me in case of an a incoming call. c

P ass basis for colllaboration, thus FIG. 2 Personality imp plicit consum mer-generated d content A. Portable Personality A P (P P2) System To understannd the conceptt of Portable Personality P (P P2) [7], inn the digital world, w P2 acts as personal innformation haarvester, coollecting dataa about its ennvironment via v sensor neetworks, ussage informattion about thee activities of the customerr. Thus, P22 collects dataa from both, the t physical and a digital woorld. P2 accts as conneccting point bbetween both levels. To observe o haappenings from m the physicaal world, conteextual inform mation is accquired by muultiple sensorss. To achieve a personality profile, thhis data is maatched to creaate a personal content item m – the peersonality proofile. The colllection of perrsonal contennt items caan be from a contemporarry personalityy state which can be foorwarded to effectuators on o the mobile phone or in the

nality (P2) forr location bassed FIG. 4 Poortable Person servvices and as context collecttion tool The collectioon of personnal content ittems could form f a p s state which can be appllied to contemporary personality o got evaluaated by personal agentss on your moobile phone or surrrounding servvice providerss. P22 is the core novel technology contribu uted by this project, p annd is the Aggrregative Conteextual Media Engine (ACM ME) for thee collection of o consumer context inforrmation. P2 collects c annd interprets contextual innformation. P2 P brings toogether moobility and peersonalized m multimedia serrvices for peo ople on thee move. P2 paves the w way for a more m pervasivve and enntertaining lifee and performss true personaalization for ennabling eassy access to mobile m multim media.

4 P2 digitizes your personality! Currently consumer data is collected separately by each application. P2 technology collects user information in a uniform way - from any type of application, such as video players, social networks, instant messengers, or music players. This information is compiled into a digital personality profile stored on a mobile device. The digital personality profile can be exchanged with any trusted service provider in the consumer’s environment helping to increase media experience. Within the scope of this project the current version of P2 shall be extended towards a capability to work with generic context information coming from any source (e.g. sensors, media players, media usage). B. The Philosophy of P2 One question still remains – what is the representation of one’s personality in the digital overlay network? Is it an avatar a consumer can select and design graphically? Can the consumer alter the personality of his digital counterpart – or shall the consumer do psychological tests to find the fully compliant representation of his digital counterpart? One of the core thesis of this paper is, that ambient media evolve implicit, thus – except while talking about concrete multimedia services – the ambient system aggregates the media in behalf of the consumer. However, without adequate representation of the consumer’s psyche, his behavior, his attitude, potential actions, and activities it would become a random process. Within this section we describe our approach for a natural collaboration, where consumer data is implicitly collected from the consumer. In fact, based on a personality profile, information exchange becomes to a condition of collaboration, rather than explicitly exchanging information. 1) Manifestation P2 manifests in many arbitrary forms. Services accessing the personality profile provide the consumer with the essential information at the right time, at the right place, and in the right form. In practical cases, the consumer could enter a music store, and an interactive banner advertisement displays a new album fitting to his music taste. Via the mobile he can interact with the advertisement system, and browse e.g. through similar albums. 1) Morphing Morphing in P2 takes place on many different levels. Firstly, the digitalization of personality; Secondly, the creation of the link between the physical and the digital world; Third, mapping of environmental context information with the data present on other layers. Morphing however, also takes place on service level, which represent current states either of the real-world or of the digital world. These sates are interconnected. 2) Intelligence The intelligence of the P2 system lies in algorithm

performing the data-mining task to achieve a personality profile. Currently P2 offers simple mechanisms for matching metadata based profiles. However, the future P2 system will also provide functionalities for gaining knowledge about the personality of the person. 3) Experience The consumer experience of the system is depending on the service. It’s difficult to predict what it will be. However, considering gaming, where the consumer can play location based games, the game and team work is experience. It’s also an experience when two different P2 personality profiles of two different persons meet. They are aware of their physical existence – each of the two personalities could change and adapt towards the other needs. However, it gets more complex with a bunch of personalities, which cluster based on their skills or interests. User-generated content would happen implicit – thus based on the system decisions. The consumer would be observer of his own creation. Different personalities in different clusters could overlap or represent a completely new cluster. This would even organize work in a new way. 4) Collaboration The intelligence of the P2 system lies in algorithm performing the data-mining task to achieve a personality profile. Currently P2 offers simple mechanisms for matching metadata based profiles. However, the future P2 system will also provide functionalities for gaining knowledge about the personality of the person. IV. EMOTIONAL AMBIENT RESPONSIVE CHARACTER You still remember the movie The Golden Compass by Chris Weitz? Each human could control a little spirit, as a digital representation. The spirit could take any form depending on phantasm: dog, cat, and bird. The spirit in the film could be seen as the digital representation of personality. P2 acts as teaching book for avatar learning. On the long run, the personality profile evolves towards a pretty sophisticating digital portrait of you, but in digital form. Implicit usergenerated content, where the consumer is passively observed is the key for the realization of such a model. However, we are not so far yet, but instead we can present an artistic installation – the emotional ambient responsive character. The idea is to let a digital character respond to human emotions. In principle it is a game concept, where a dog reacts on the mood of the player. The virtual world responds emotionally by being sad, angry or happy.

5 meedia form as implicit i user-ggenerated con ntent. It shoulld have preesented, that simplicity is the key for the key for gaining g knnowledge abou ut the consum mer. The agg gregation of content c becomes a mattter of smart syystems aggreg gating contentt based hered informattion about the consumer. Th hus the onn implicit gath consumer data e.g. in form m of a person nality profile is the fo the future of user-geneerated conten nt. And driiving force for wiith user-generaated content w we do not meaan people disccussing onn web-sites or placing conteent for collabo oration onto th he web. W state, that consumer geneerated contentt will happen hidden We an nd in the backg ground. Still, for both h presented appplications theere is still a far f way to go to make th hese really sopphisticated. Bu ut they should d act as refference of how w ambient media might dev velop in the fu uture. ACKNOW WLEDGMENT FIG. 5 Emotional E Am mbient Responsive Characcter uld be the man nifestation off the system. The T dog The dog wou he narrative, thus t the ass virtual avataar is the maniffestation of th reesponse of hu uman emotionns. Morphing, thus the binding of th he real-world with the diggital world iss performed via the ob bservation off emotions, an nd letting thee dog based on the em motional inpu ut. The intelliggence of the system s lays fiirstly in th he interpretatiion of humann emotions, and a secondly in the sim mulation of th he emotions of o the virtual avatar. a Collab boration ass interconnecttion between the digital an nd the real-w world is cllearly visible. The digital becomes b a cou unterpart of th he realw world and acttive contributtor. Howeverr, the system m is an ex xperience as su uch. V. CO ONCLUSIONS P2 is still under u develop pment, and faar away from m being reeality to provide a personality based, imp plicit, user-geenerated co ontent personaalization. How wever, the opeen source foru um is in grrowth and mo ore and moree people are getting g intereested. If yo ou would likee to join the development team, please do not heesitate and visit the website w on http://www.po h ortablepeersonality.org and contribuute. The goal is to let user--data in th he hands of th he consumer. And this is only o ensured with w an op pen source plaatform, rather than proprietaary software firms. f The overall concept off ambient media m is developing raapidly! More and a more app plications emeerge on markeets. We allready know a few of them: GPS locaation based seervices; G Geo-caching, w where people hide treasurees that other people caan search for; RFID technollogy in food products p to dettermine caalories of food from thee grocery sto ore shopping;; smart su urveillance sy ystems detectting conspicu uous movemeents on strreets; smart fridges; f and emergency e wrristwatches no otifying am mbulances in case c of emerggencies. However, many m search fo for the killer application, and a it’s stiill to come. But B one thing is clear, ‘am mbient media’ are the neext developmeent after ‘digittal media’. Am mbient media will be a completely new n media ennvironment, with w its own ambient a m media languag ge, ambient media m form, and a aestheticcs. This arrticle should have h shaded some light on this newly em merging

This work was supported s by the Academy of Finland, project No.. 213462 (Finnnish Centre of Excellence E Prograam (2006 - 2011)).

FIRST AUTH HOR SHORT BIO Dr..-Techn. Artur Lugmayr L describees himself as a creative thinker and his scieentific work is situated betweeen art and scien nce. His vision can be exppressed as to create c media exxperiences on future emergingg media techhnology platform ms. He is the hhead and foundeer of the New AMbient A MU Ultimedia (NAM MU) research ggroup at the Tampere T University of Tecchnology (Finlannd) which is paart of the Finniish Academy Centre of Exccellence of Signaal Processing from m 2006 to 2011 (http://namu.cs.tu ( ut.fi). He is holding h a Dr.-Teechn. degree from m the Tampere University U of Tecchnology (TU UT, Finland), andd is currently enggaged in Dr.-Artss studies at the School S of Mootion Pictures, TV V and Productionn Design (UIAH, Helsinki). He chaaired the ISO O/IEC ad-hoc grooup "MPEG-21 iin broadcasting"; won the NOKIA A Award of 22003 with the texxt book "Digital interactive TV and Metadata" published p by Springer-Verlag in 2004; represeentative of the Sw wan Lake Movin ng Image & Music Award (http://www.swaan-lake-award.orrg/); board mem mber of MinndTrek (http://ww ww.mindtrek.orgg), EU project prooposal reviewerr; invited keyy-note speaker for conferencess; organizer an nd reviewer of several connferences; and has h contributed oone book chapter and written over 25 scieentific publicatioons. His passion in private life is to be a notoriouus digital film m-maker. He is founder of thee production coompany lugYmeedia Inc. (htttp://www.lugy-m media.tv). More abbout him on Goog gle.

REFEERENCES [1] [2] [3] [4] [5] [6]

[7]

[8]

Oxford, "Oxfordd Refereence ON NLINE," http://ww ww.oxfordreferennce.com/. A. Lugm mayr, "Ambient M Media," Novatica,, vol. 33, pp. 35-339, JulyAugust 2007 2 http://www..ati.es/novatica/in nfonovatica_eng.h html. B. Ullmer and H. Ishii, "Emerging fram meworks for tangiible user ms Journal. vol. 39, 3 2000 interfacees," in IBM System Ambientt Devices Inc., htttp://www.ambien ntdevices.com/. K. Reiszz and G. Millar, "T The Technique off Film Editing," 1978 1 "iCat," Philips, ww.research.philiips.com/newscentter/pictures/041209http://ww icat.htmll, http://ww ww.research.philiips.com/technologies/syst_softw/rrobotics/ index.htm ml. S. Reym mann, V. Bruns, annd A. Lugmayr, "P2 " - Portable Perrsonality a Middleeware Solution ffor Smart User Profile P Managem ment and Distributtion," in Interacctive TV: A Shaared Experience,, TISCP Adjunct Proceedings P of E EuroITV 2007, Am msterdam, 2007 G. J. Suussman, "Amorphhous computation n," [Cambridge,, Mass.]: MIT Cennter for Advancedd Education Servvices,, 1998