Tangible Jukebox: Back to Palpable Music Daniel Gallardo Universitat Pompeu Fabra, MTG
[email protected]
Sergi Jordà Universitat Pompeu Fabra, MTG
[email protected]
ABSTRACT
disadvantages:
Since commercial musical recordings became available about a century ago and until very recently, they had always been distributed by means of a physical support. Nowadays that record companies have ultimately started to distribute music over the Internet, “music” does not use physical space anymore, and we can now store amazingly large music collection in our computers or portable music players. But this etherealness does not come without some drawbacks, such as browsing by the covers or organizing and grouping elements or collections.
• They occupy large amounts of space. • Their capacity is limited. • By the fact that the data is contained in the support, loosing the support implies loosing the data. Physical musical supports have indeed evolved in terms of size and capacity, but this has not recently prevented the traditional “physical” market model from being ultimately beaten by big online stores like ITunes or Amazon [3]. Apart from the ease and speed of the purchasing process and the storage space savings we have already mentioned, an additional benefit provided by Internet digital music purchases, is the ability to store the acquired music files in a remote server, thus allowing access to the music from almost anywhere. But in this process, “ubiquity” also entails the loss of all physical references to these data. As a result, accessing any of these songs may not be as instantaneous or at least as natural as picking a physical object and inserting it in the playback system. More generally, it seems that as flexible as working with virtual references can be, the lack of physicality exhibits its weakness in all processes that typically involve manipulation and spatial operations, such as grouping and organizing albums. In this project we propose to overcome physical and spatial restrictions by allowing the management of songs (virtual data) using tangible cards within a tabletop interface. Cards play the role of “song containers”, permitting the user to organize and manipulate, store, share and transport virtual data in a tangible way.
In this paper we propose a system in which files (intangible) become physical objects again, thus recovering part of the mysticism related with vinyls or compact discs, without loosing most of the benefits inherent to digital data, and applying at the same time some advantages that tangible and tabletop interfaces can offer. Complementarily, and from a more agnostic perspective, this paper also starts to explore on the tangible manipulation of sets or collections of items. Author Keywords
Tangible user interface, tabletop, music collections, playlist, player files. ACM Classification Keywords
E.2 Data Storage Representation: object representation. General Terms
Design, Languages, Management. INTRODUCTION
Commercial music reproduction and distribution started with the first phonograph cylinders at the beginning of the 20th century and took a huge step forward, with the much better sounding gramophone records of the mid 1920s. Since then, and for almost one century, recorded music has been distributed using different physical supports, such as vinyl disks, magnetic tapes or digital compact disks. Without disregarding their countless benefits and without minimizing the fact that recorded music has probably been the most influential single development in the whole history of music [2], we can however recall that physical musical supports do also share some common
PLAYLISTS: THE NEW MUSIC UNIT
Audio CDs have a maximum playing time of 74 minutes typically storing between ten to twenty songs, although quite often, consumers buy albums just because a few of the particular songs included in a disc. For this reason, Internet music stores do often allow the acquisition of individual songs. This shift from the album unit towards the song unit, together with the immense storage capabilities of current devices, and the added possibilities that digital classifications permits, is changing radically the hierarchies and the structures with which many users classify their music.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. TEI 2010, January 25–27, 2010, Cambridge, Massachusetts, USA. Copyright 2010 ACM 978-1-60558-841-4/10/01...$10.00.
Current music player devices have the capacity for storing thousands of songs, which implies many albums, a whole music collection indeed. Quite often, instead of being structured into nested folders, these thousands of songs are uasually tagged with different labels (album, artist, genre,
199
year, etc.), and thus do not show a clear hierarchical structure. As a result, playing entire albums is not such a common practice anymore, and new users prefer to manage playlists constructed according to many different personal criteria.
the relative complement of A in B). These are the operations we will start with, adding also the remaining operations specific to musical playlists and not available in sets, such as browsing content, sorting elements and playing them. Music collection
The music collection used to test TJ’s functionalities is a subset of Magnatune [10], Creative Commons-licensed with more than six thousand songs from all genres and styles, which is used in many research projects because of its license type. In our database, each song is accompanied with the habitual metadata provided by Magnatune (title, author, album…), the “cover art”, plus some high level descriptors provided by an in-house music annotation library
MUSIC CARDS
Our system proposes the use of tangible cards for representing playlists, where this term is understood as a list of songs and is implemented as a data structure that contains a collection of references to several locally or remotely stored musical files. By means of these tangible cards, which can be stored, manipulated and shared between different users, playlists become as physical as albums where. Our tangibles are made with two-sided cardboard cards. One side shows the information for the users, such as the “cover art” or any other useful graphical information, which can even be even handwritten or drawn. The other side shows the fiducial codes meant for the visual tracking system and does thus not include any user-friendly information.
developed in our research group (Laurier). This library
preprocesses all the music files for extracting several music content parameters, which will be used by our system in some classification and sorting operations. SYSTEM OVERVIEW
TJ uses a tangible multitouch surface, which is compounded by a projector and a camera underneath. The camera captures all the activity on the surface, identifying and detecting the positions of tags and fingers, while the projector is in charge of the visual feedback of the system. The whole system runs as two separated processes: one for the visual tracking, which is done by reacTIVision [9], the open-source tracking software originally designed for the reactable [8], and the other for the own TJ application.
Several methods for representing computer readable labels exist, which can be roughly separated into two groups: the ones that contain an invariable ID, like RFID systems where the information is referenced dynamically to an unique ID [12]; and the ones that generate new labels for each modified data, like the QR-codes used in iCandy [5]. In Tangible Jukebox (TJ) we use printed fiducial tags, each representing an invariable ID to which variable virtual data can be dynamically assigned. By means of some tag reader device, users can access and modify the virtual data, like in the mediaBlocs [11], where a server maintains a table that joins each ID with it corresponding data.
ReacTIVision is the program in charge of tracking both the fingers and the tagged objects that touch the surface at any time. While reacTIVision cannot distinguish between different fingers, giving only the positions (x,y) for each of them, objects are always identified by its id, position (x,y) and angle on the surface.
These playlists cards represent the main component of TJ; all processes and actions described in this paper will be focused on how to manage, modify, combine and playback the information contained in these cards.
The second process is TJ’s main thread. It generates queries to the database, draws the visual feedback and resolves all interaction metaphors related with playlists.
WHICH ACTIONS COULD BE APPLIED TO A TANGIBLE PLAYLIST?
Tangible Jukebox Objects
TJ objects are plastic or paper squares (different materials imply different functionalities) of about 7x7cm. Most of them are hybrid objects [6] that are compounded by a real and a virtual part. They can be classified into three main groups: Tools, Tags and Playlists.
The term ‘playlist’ came about in the 1950s when, during the early days of top 40 radio formats, stations would devise a limited list of songs to be played. Nowadays it is mostly employed in the digital domain, where it has become ubiquitous parallel to the increase of musical storage and playback capabilities of personal computers. As a result, playlists, and by extension the actions they tend to permit, are nowadays quite influenced by the GUI paradigm, and it would probably constitute a design error trying to mimic on our tangible playlists, all and only the actions allowed in “traditional” graphical playlists.
Playlist cards are the main objects of this application. They are made of paper board or any printable paper, with the back side (the one in contact with the table surface) showing a unique reacTIVision ID, and the other side is ready to be printed, drawn or handwritten with any useful information, such as cover art, song titles or special events (e.g. “Paul’s birthday party list”). Playlist cards are the only TJ objects that “contain” data. In that sense they could clearly be classified as containers, following Holmquist taxonomy [7], although we will later see that when used in combination, these cards can also become operators.
Playlists are just sets of songs (that can be ordered or not), and the first that comes to mind when dealing with sets of any items, are the basic set operations: union or addition, intersection, and subtraction (where B-A is understood as
200
Tools objects are made of ticker plastic. They do not contain data and their functions include changing the system behavior and the modification of other objects, such as the data included in the playlist cards. They can thus be considered tools according to (Holmquist et al. 99). In TJ. current iteration, there is only one tool, which is in charge of modifying the reproduction order of each playlist.
levels (categories) on the tree. Virtual wheels also allow for more operations, such as eliminating songs, selecting several songs for dropping them into existing or new playlists.
Tags are a group of plasticized cards, one for each of the high-level descriptors employed (e.g. danceability, sad, happy, party, acoustic, aggressive, relaxed). They work in combination with the sorting tools, and their purpose is to help classifying and sorting the data stored into the playlists, according to different criteria. Not tools and definitely not containers, considering their role on defining how to undertake actions, they could be classified as ‘adverbs’, if we wished to extend Fishkin’s basic distinction between ‘noun metaphor’ and ‘verb metaphor’ tangibles [4].
Figure 1 Detail of wheel menu at Tangible Jukebox.
Operating with playlists
In the physical world we often create several CD stacks for organizing the discs or for selecting the ones we plan to play in different situations. We may combine several stacks into a bigger one, divide a big one into several smaller, or browse different stacks for selecting a few discs with which to create a new stack. This type of frequent operations inspired us for implementing the basic set operations in our tangible playlists.
TANGIBLE PLAYLIST ACTIONS
TJ playlists are cards that users can print and easily create using any printable paper or paper board. Their total number is limited only by the size of the reacTIVision symbol dictionary employed (typically several hundred IDs). Most of the operations that can be done with playlists - and which result in the modification of their contents - can be applied and controlled without the need of any additional object, interacting directly with these playlist objects.
The operations implemented are three: addition, subtraction and intersection. These operations were chosen from some hypothetic scenarios: Addition: User A wants to get a new playlist that contains his two preferred playlists.
Playlists as containers
Playlist cards do not store data by themselves; they hold references to data structures stored in some computers. Apart from having an “unlimited” storage capacity, these cards have the advantage of hiding all the lower level data storage information to the final user, who does not need to know where the songs are physically stored. This transparency also allows users to share or exchange playlists by just lending or exchanging paper board cards.
Subtraction: User A wants to play a playlist but she does not want to listen again the listened songs last time. Intersection: User A and B have playlists with their preferred songs and they want to create another one that contains their preferred songs. In TJ, operations between playlists become available at any time that two or more tangible playlist cards are placed near each other. When this happens, a link between each two neighboring tangibles appears on the table surface. The order of the operation chain is based on temporal conditions; as new tangibles are placed on the surface or moved to the immediacy of the operation zone, they are added to the end of the operations. As shown in Figure 2, the first operand (i.e. the tangible that was placed first) is illuminated with an aura, and a translucid virtual playlist that represents the result of all the operation chain is represented in its proximity. In that sense, TJ operations do not modify the operands involved, creating instead a virtual list that will contain the result of the chain. This virtual list can later be “physicalized” by placing an “empty” playlist card on it.
Playlists as controllers
For each tangible, apart from its ID, reacTIVision delivers three additional parameters (X position, Y position and angle). We use this information in combination with the detection of fingers, for controlling most of the actions that playlists support, without the need of additional objects. Browsing playlists
Browsing the songs of a playlist is one of the most basic functionalities any playlist system must support. For navigating across a playlist, users only have to place its tangible on the table and press an expand button that appears next to it. This deploys a wheel menu with the following tree structure: genre, author, year, album and songs [1]. Navigating through leaves it as easy as spinning the virtual wheel [Figure 1] using the fingers, while dragging out a cell or spinning the tangible, users can navigate through menu
201
arrangements on a 2D tabletop surface (e.g. the order of the operands in operation chains) and the coexistence of real and virtual entities (e.g. the temporary playlists that temporarily hold the operations results). For all that, the various iterations of TJ have proven to be an excellent test bed for our own experiments. However, we have not yet pursued any serious user tests specific to the TJ interaction. TJ is also ready for multiuser interaction. It was designed for being accessed from anywhere of the table (that is because of the wheel menus) and the platform allows multiple objects and finger gestures at the same time. Future investigations could be centered on that and how a multiuser playlist manager could be useful for instance in parties.
Figure 2 Operating with different playlists. ACTIONS USING ADDITIONAL TANGIBLES
The sorting process is the only one that requires an additional tangible. When it is placed near a playlist, appears a representation of the sorted elements and a button pad [Figure 3]. By default the sorting parameter is alphabetical order. When Essentia tags are linked to the sorter, the sorting parameter switches to the parameters contained by the tags sorting elements according their musical parameters. It also allows sorting elements by its metadata by taping the sorter buttons.
REFERENCES
1.
Andric,A, et AL. Music Mood Wheel: Improving browsing experience on digital content through an audio interface. Automated Production of Cross Media Content for Multi-Channel Distribution, 2006. AXMEDIS'06. Second International Conference on. Pages 251—257. 2. Attali, J. Bruits : essai sur l’économie politique de la musique, 1977. 3. Benderoff, E. Apple's iTunes the No. 1 music seller, besting Wal-Mart. Chicaho Tribune, April 3, 2008. 4. Fishkin, K.P., A taxonomy for and analysis of tangible interfaces, Pers Ubiquit Comput (2004) 8: 347–358. 5. Graham, J. and Hull, J.J. Icandy: a tangible user interface for itunes. 2008, ACM New York, NY, USA 6. Habakuk,J. Noncommand-Based Interaction in tangible virtual environments. CHI 2006. Fraunhofer Institute for production systems and design technology. 7. Holmquist L, Redstro m J, Ljungstrand P (1999) Token-based access to digital information. In: Proceedings of the 1st inter- national symposium on handheld and ubiquitous computing (HUC’99), Karlsruhe, Germany, September 1999, pp 234–245. 8. Jordà,S, et AL. The reactable*. In Proceedings of the international music conference (ICMC 2005), Barcelona, Spain. 579--582 9. Kaltenbrunner,M, et AL. Improved topological fiducial tracking in the reactivision system. Proceedings of the IEEE International Workshop on Projector-Camera Systems (Procams 2005), San Diego (USA). 10. Magnatune collection: http://www.magnatune.com/ 11. Ullmer,B, et AL. mediaBlocks: Physical Containers, Transports, and controls for online media. MIT Media Lab. Proceedings of the 25th annual conference on Computer graphics and interactive techniques 12. Zhang, N, et AL. Nomadic Tangible Music Player with RF-enabled Sticker. 2002. Proc. ICAT’02 pages 184— 185
Figure 3 Sorting process
Play playlists
During all the execution process, a floating speaker icon is placed on the table. When a playlist card is placed on it, the player is deployed [Figure 4]. It has two finger sliders: One for modifying the volume and the other for playing-time navigation. It also shows the sorted playlist with the current element highlighted which can be changed by spinning the tangible.
Figure 4 Screen capture of the player. CONCLUSIONS AND FUTURE WORK
This project started as a proof of concept for the development of a tangible media player which would allow to manipulate physical references (in opposition to virtual references or physical files), as well as a tool for exploring some complex interaction patterns in a tabletop interface. In particular we were interested in the distinctions between containers and tools [7] and nouns and verbs [4]. We also wanted to explore the combinations of spatial and temporal
202
