Interactive Gallery

Gallery ONE: Cleveland Art Exhibition Interacting With Art Benan Demir

Carlos Joaquin Padial Lopez

Abstract

Pompeu Fabra University

Pompeu Fabra University

Barcelona,Spain 08003

Barcelona,Spain, 08013

[email protected]

[email protected]

This paper is a critical report that analyses Gallery One’s ArtLens application in terms of its virtual subjectiveness and how it fits under the category of Augmented Reality. The report analyses Gallery One and its application ArtLens from an interactive perspective and gives solutions to the different problems that can be faced when using this application. The report also gives a detailed description in terms of physical and logical interfaces, which conveys the data and metadata necessary to understand and develop the project itself. To summarize, the paper assess the validity of application under the term Augmented reality, also considering ways to improve its efficiency in the aim of engaging users with the variety of artworks in Gallery One.

Author Keywords augmented reality; gallery one; artlens; virtual subjectiveness; interactive systems; user experience.

ACM Classification Keywords B.4.1. Data Communications Devices; H.5.2 User Interface; H.5.1 Multimedia Information Systems

Introduction

1.Augmented Reality

Gallery One has been named one of the most interactive galleries around the world. With 10 interactive features under its roof, Gallery One offers the user a variety of interactive experiences with multimedia platforms, and it has been able to make use of Augmented Reality in its ArtLens application, Studio Play and Lenses.

One of the earliest definitions of AR is by Tom Caudell in 1990: “The interaction of superimposed graphics, audio and other sense enhancements over a real-world environment that’s displayed in real-time”(Casella, 2009). ArtLens sustains this with the camera it uses within the iPad. Some artworks in the gallery have an icon next to the compositions indicating the art piece can be looked up and explored using ArtLens (Image 1). The iPad scans the real time image in the gallery of the artworks and as the camera is held up, the information bars appear pointing out from the specific points of the painting. In another definition, Augmented Reality “is the art of superimposing a computer generated content over a live view of the world” (Craig, 2013). Hence, Artlens can be classified as Augmented Reality due to the use of the iPad as the physical interface to interact with the live environment.

The ArtLens application lets the user understand and interact with the details of the art pieces. The user, together with an iPad borrowed from the gallery or your own, can visualize and interact with metadata of the paintings and other types of art. The interaction between the user and the new experience through an iPad help user to see and think of things that they wouldn’t be able to without the application. ArtLens makes use of interactive displays which are both flexible and of low maintenance (Alexander, 2014). It proposes the user a new way of interacting with art through a multi-layered content. Fully developed with all types of media systems, the iPad is the bridge in the mapping in this interaction. The Scanning feature allows the information reflected from the piece of art to the iPad and user engagement. Through the cloud base data storage, the user is able to reach a database of information for them to examine (Alexander & Cleveland, 2014). ArtLens curator find augmented reality as the most suitable system in order to engage user with the art work while keeping them immersed in real-time and informed with augmented metadata. The paper will be analysing Augmented Reality and its features accordingly.

Augmented Reality applications are based on a very accurate type of vision. Hence, one of the needs of the user in Gallery One is the requirement to have space to visualize through the device the piece of art fully (Milgram, Takemura, Utsumi, & Kishino, 1994). The visualization of the piece of art for a future interaction has to be completely done through the iPad´s camera. As Craig (2013) defined Augmented Reality, “The key part of AR is that you need to place a layer of virtual information over your view of the real world and, in order to do that, there must be a device in between to display that information upon”. Hence, it can be said that ArtLens, together with the Scanning feature, is a perfect example of this definition for the use of information displayed in a device, in this case the iPad, which becomes a funnel for the user to understand the different pieces of art.

Authors such as Roy Kalawsky (N. Parés, lecture, January 15, 2015) define Mixed and Virtual Reality as the interaction with synthetic environments by means of a Virtual Reality in which users are summoned into. Therefore, ArtLens cannot be classified as neither mixed nor Virtual Reality due to lack of a synthetic or virtual environment in which users interact.

2. Virtual Subjectiveness Figure 1. Simple diagram of a VR system (a), virtual environment (b), user (c), interactive communication (d).

Image 1. Visitor uses ArtLens to get information about the painting (http://mw2014.museumsan dtheweb.com/bow/artlens/).

Full potential of AMVR devices are being defined in different spectrums. Each scientist has a different definition and approaches it from different aspects. This paper analyzes ArtLens through the approach of Roc Pares and Narcis Pares, evaluating the system by considering its virtual subjectiveness. First, the definition of VR will be analyzed, then, Augmented Reality will be defined through; ‘what VR is’ and ‘what AR isn’t’ explanations. Elis states that environments in AR are composed by three parts: dynamics, content and geometry. However, this approach doesn’t consider one of the biggest elements in a Virtual Reality, which is the user. He only relates to an element called “self” where he tries to express the user’s relation to the environment (Ellis, 1991, p. 322). User is crucial to argue while addressing a system, as the product’s success relies on the perception of the user. This is built on the relation between the user and the environment, in specific terms, the interface and the mapping. In terms of Pares, this is the user and interactive communication (see Figure 1).This communication allows the exchange of information, which is the base and focus point in many AMVR systems. As Pares’ focus on VR, this paper bends and modifies the analysis points accordingly, fitting to an Augmented Reality system as Gallery One’s

curators define ArtLens as. The adapted understanding of VR in this paper is defined as an “interactive communication medium that is generated in real time” (Pares & Pares, 2006, p. 525). We would like to break down this definition in order to critically assess how ArtLens fits and doesn’t fit to this definition. ArtLens is an interactive communication between the art piece and the user that gives information in the specific artwork that user directs their camera at. The communication in this medium is based on information transfer. On the other hand real-time generation is a term that is specific for VR and cannot be applied to Augmented Reality systems. There is real-time image, however, this is not generated, but rather just seen through a display and a camera. This is where ArtLens is separated from the VR systems and accepted as AR as the authors of the app also describe it.

2.1 The Experience The interface is usually mistaken to be considered under one label: The physical product that allows the engagement between the environment and the user. Here ArtLens’ ‘Scanning’ feature will be analyzed in the two-way system of the interface: physical and logical. The report will not go in depth of the physical interface rather argue the mediation between the experience and the user in the logical interface. ArtLens uses a seethrough camera based system allowing a view of the real world. In the Reality-Virtuality continuum, it is complex to argue where ArtLens stands as the app merges real environment and augmented metadata in one display however there is no movemental interaction between the two, and there is no virtual environment that the user is immersed in. Therefore, it can be said that ArtLens stands a step closer to reality regarding the continuum principles.

2.2 Physical Interface

2.3 Logical Interface

The physical interface is only composed by the iPad, which interacts with the real-image of the artwork. The scanning feature incorporates the camera and Qualcomm’s Vuforia image-recognition SDK, which provides the Augmented Reality experience. When the user scans the artwork, the ArtLens recognizes the object and brings in metadata to inform the user. This content is attached to the app screen, also connected to the specific regions of the artwork on the AR display. (see Figure 3). There is a data structure that enables users to explore and learn about the CMA’s collection indistinctly in their own ways. The artwork is dynamic; the descriptions and images for artwork flow automatically from the asset management systems (Alexander, 2014).

The user sees the real image through the camera’s lenses. The technology uses image-recognition to select the art-piece from its attributes. The user, merging with the augmented reality, directs the camera to the art piece hand-held. However, this depends on their viewpoint as the user can also avoid the screen and just look at the artwork itself. According to Ellis, the logical interface is the point of view where the environment is constructed (Ellis, 1991, p. 322), which would be through the camera lens in this example. ArtLens uses a first person point of view, which allows the user to point at the painting, covering the whole image, and when the user directs the iPad to another view, the augmented data disappears. The augmented reality dimensions fits to the same scale of what the user sees in real life. The only possibility is the first person point of view. However, as the device doesn’t require any output system other than the iPad screen itself, the experience is non-immersive. The user can still interact with the real-environment. Considering the experience is non-immersive, the presence factor does not need to be included as it is not a valid concept for this specific Augmented Reality experience.

2.4 Potential Mapping

Figure 3. Example Augmented Reality app architecture (http://www.csc.ncsu.edu/faculty/mpsingh/papers/columns/IC -NWI-17-06-AR-13.pdf).

The metadata also moves along as the iPad is rotated and moved. As the user goes 1 step back, the augmented data also reshapes itself in the same ratio. This is because the AR cannot be as distorted as a VR, and ArtLens uses the camera and real time image augmented with metadata. The possible mapping is 1:1 as the image sizes don’t change, and the ratio is kept fixed of the reflected image. One problem could be that the application is dependent of the iPad’s camera lens

and when the gallery is crowded, the space that a user has will be limited.

3. Level of Interaction and Behavior After the mapping and interface is discussed, it is important to link the logical interface with the real environment augmented with metadata. The user is exposed to explorative medium in order to access data and metadata. It needs to be considered that this is only in the ‘Scanning’ feature that explorative interaction is offered. The other features of the app are multimedia systems and offer collaborative interaction. The app invites the visitors to engage with each other in the ‘Tour’ feature where they can create their own tour. There, private tours are created according to the user’s own taste and can be made visible for other visitors, allowing a collaborative interaction. However, this is a multimedia platform. The AR in ArtLens offers only explorative interaction for user to understand and analyze the artwork more. The degree of the eyesight is fixed to the artwork, as it needs to be held up so the iPad plays the role of a frame that keeps the point of view fixed. Even if the user steps back and forth, the points of the augmented metadata stays fixed.

4. Interaction Design The starting point of the project was to engage the user with the variety of art pieces in the gallery. The main goal in the design is to give information to the users, hence is why the method on ‘how to do so’ has been developed accordingly. The application used by Gallery One to scan and give information can be defined as content-driven due to the importance of content among its interaction in contrast to interaction-driven applications, which aim to primarily give importance to interaction itself.

The application is an innovative way of supplying metadata about the art pieces to those interested in art and its features. Users, who have basic or no knowledge of the pieces of art, are able to enrich their knowledge with broad information about the displayed art pieces. In this explorative content-driven interaction visitors are summoned to know more about the compositions displayed in an interactive way. For this, Gallery One provides the visitors with the necessary apparatus to submerge themselves in the most detailed information that can be found in the gallery´s database. Also, it has been decided that facial recognition was the best choice with 1:1 mapping. The logical interface, along with the physical interface, has been chosen as suitable input and output systems that serve to the purpose of giving information.

5. Further Critical View ArtLens can be considered as a valid AR system when analysed under the previous terms and definitions. The necessity of a device to separate the user from the real environment is accomplished through the use of an iPad, which displays the metadata that is delivered to the user for further information about the different pieces of art. Specifically on what a colour symbolizes, time period, brush strokes, theme and style of the artist and many more. In terms of improving the system to achieve a better experience; there are specific things that could be considered. The design used by the authors of this project is innovative and creative, however inefficient when certain situations are faced within Gallery One. The main problems of the apparatus used are: its lack of compatibility with other users that are not able to

borrow the iPads due to reasons such as battery insufficiency. Moreover, the lack of sufficient devices for a big number of visitors makes the interaction with the variety of features difficult due to the compatibility of this application with Android or other types of devices. The application can be downloaded in devices such as iPads or iPhones. However, the large amount of data that has to be downloaded takes up to 10 minutes. Hence, visitors will have a hard time using the interactive systems if the gallery does not provide a device. It also needs to be considered that in order for the Scanning feature to work, there cannot be any other objects or possible visitors between the iPad and the artwork. This is an environment that is hard to create in a crowded day at a gallery, even though if in alignment, multiple users can scan the same image. Overall, the fact is that this application has been built with a content-driven strategy, supports it to fit its’ purpose; as the goal of the innovation has been set before the interface design. CMA’s strategy is to engage the audience with the artwork and help them see what wasn’t visible before applying AR. The system is explorative and content-driven, and these two strategies serve each other. The user is directed to learn, and content-driven design focuses on it. In terms of usability, the application has a high standard in reaching the user’s needs as it is simple and efficient. The mapping is fixed yet enough for the purpose of the application. Improvements such as a collaborative interaction could be reached through the help of each user by sharing their opinion on a specific artwork and, this way, enrich the metadata.

Acknowledgements We thank all the researchers, theorists, publications, lecturers, authors Cleveland Museum of Art who wrote and provided helpful information on Gallery One and Augmented Reality Systems.

References 1.

2.

3.

4.

5.

6.

7.

Alexander, J. (2014, April 2). Gallery One, the First Year: Sustainability, Evaluation Process, and a New Smart Phone App. Retrieved March 10, 2015, from MW2014: Museums and the Web 2014 website: http://mw2014.museumsandtheweb.com/paper/ga llery-one-the-first-year-sustainability-evaluationprocess-and-a-new-smart-phone-app/ Alexander, J., & Cleveland, T. (2014). MW2014 : Museums and the Web 2014 Gallery One , the First Year : Sustainability , Evaluation Process , and a New Smart Phone App. Casella, D. (2009, November 3). What is Augmented Reality. Retrieved March 9, 2015, from Digital Trends website: http://www.digitaltrends.com/mobile/what-isaugmented-reality-iphone-apps-games-flash-yelpandroid-ar-software-and-more/ Craig, A. B. (2013). Understanding Augmented Reality. Understanding Augmented Reality. doi:10.1016/B978-0-240-82408-6.00008-4 Ellis, S. R. (1991). Nature and origins of virtual environ- ments: A bibliographical essay. Computing Systems in Engineering, 2(4), 321–347. Helmrelch, A., Jones, J., & Stevens, J. J. (2006). Gallery One , Cleveland Museum of Art. Exhibition Critiques, 10. Milgram, P., Takemura, H., Utsumi, a, & Kishino, F. (1994). Mixed Reality ( MR ) Reality-Virtuality ( RV ) Continuum. Systems Research,

8.

9.

2351(Telemanipulator and Telepresence Technologies), 282–292. doi:10.1.1.83.6861 Pares, N. (Presenter). (2015, January 15). Interaction Models. Lecture presented at Pompeu Fabra University, Barcelona, Spain. Pares, R., & Pares, N. (2006). Towards a Model for a Virtual Reality Experience: The Virtual Subjectiveness. Presence, 15(5), 524-538.