In Proceedings of HCI International 2003. Lawrence Erlbaum Associates, Vol.1, 1143-1147.
Developing a mixed reality co-visiting experience for local and remote museum companions C. Randell A. Galani, M. Chalmers B. Brown, I. MacColl Computer Science University of Bristol Computing Science Bristol, UK University of Glasgow
[email protected] Glasgow, UK {areti, matthew, barry, ianm} @dcs.gla.ac.uk
A. Steed Computer Science University College London London, UK
[email protected]
Abstract This paper focuses on the first stage of the City project that concerns the design of a mixed reality system that may support co-visiting for local and remote museum visitors. We discuss the initial visitor studies, the prototype system and the user trials, with a focus on the role direct interaction and peripheral awareness have in the shaping of the visitor experience. The paper concludes with reflections on the use of the system and future plans.
1
Introduction
Social interaction among museum visitors shapes their engagement with the displays and influences their overall museum experience (McManus, 1987). Interaction can be intentional or unintentional, and may happen among members of the same group or among ‘strangers’. Museological studies have emphasised the learning arising from social interaction during the visit (Falk & Dierking, 2000) and more recently on how visitors use the social activity around them in making sense of exhibits (vom Lehn, 2002), but social aspects of the visit remain unsupported for remote visitors visiting a museum’s web site. Museum web sites often resemble digital brochures or databases, neglecting the need of the remote visitor for deeper engagement with collections and the museum experience (Galani, 2003). Our research partially responds to the increasing number of remote visitors, which often outstrips the number of visitors to the traditional museum premises (Lord, 1999). It also adds a new dimension to museological research by bridging between local and remote visitors, and exploring the combination of traditional and digital media in the visitor experience. Our project, City, is concerned with how sociality may be achieved across different media, and explores new types of social interaction between on-site and off-site visitors. We develop digital spaces and artefacts that correspond to a traditional building or urban space, and the artefacts within it, and support spatial forms of interaction, such as positional awareness and gesture, as well as verbal forms such as talk over a shared audio channel. Our work is set within Equator, an interdisciplinary research collaboration (www.equator.ac.uk), and extends existing mixed reality (MR) (Koleva, Schnadelbach, Benford, & Greenhalgh, 2000) and augmented reality (AR) (Billinghurst, Karo, &
1
Poupyrev, 2001) research. Our work is also informed by museum–based research such as PARC’s SottoVoce (Aoki et al., 2002). In the next section, we relate our studies of social interaction in two cultural institutions in Glasgow to our system that supports a shared museum visit for a group of local and remote visitors. We then outline the user trials of the system, outline ongoing technological developments, and finally reflect on issues of social participation and engagement in museum MR experiences.
2
Elements of co-visiting
To initially investigate the activity of co-visiting in traditional museum settings, we used ethnographic methods such as unobtrusive observations and videotaping. Approximately 60 visitors in friend and family groups were observed in two cultural institutions in Glasgow, The Lighthouse, Centre of Architecture, Design and the City (www.thelighthouse.co.uk) and the House for an Art Lover (www.houseforanartlover.co.uk). The studies focused on visitors’ social interaction with their companions and how this shaped their visiting behaviour. The studies suggested that co-visiting involves the constant collaboration between co–visitors, in that they continually engage with each other and the exhibition, for example by highlighting elements of a display for each other and by mutually managing the pace of the visit. Visitors engage with the displays in the gallery by interacting with them individually and by seeing their friends doing so; their friends’ activity informs their future engagement with displays. Similarly, they engage with each other in direct interactions that involve conversation and explicit collaboration, as well as subtle interactions that can be characterised as peripheral awareness. Covisiting is constituted by continuous and harmonious interweaving of interactions of various degrees of engagement. In that respect, flexibility in the transition between and combination of such interactions is essential for a good experience (Cheverst et al. 2000). Co-visitors both generate and use a range of resources when engaging with their friends and with displays. These resources are, on one hand, visual and verbal cues, such as orientation, gestural behaviour etc., and on the other hand, shared content. Visual and verbal cues may support both explicit communication and implicit personal engagement. The use of the resources does not follow a repetitive pattern but is contextually dependent on the situation or the activity in hand. Often the same resource, for example a pointing gesture, acquires different communicative roles in the course of a single visit. Shared content includes elements of the exhibition, e.g. labels, audio guides, displays, as well as people’s previous experience. Shared content facilitates peoples’ discussions and interpretations of the artefacts. In our studies, museum content was presented via a range of media, such as touch screens, audio guides, leaflets and also objects in collections. Covisitors used all media inseparably and often made associations between content delivered in different media, e.g. audio narration and label text. They also used shared previous experience— built up during the visit, for example by jointly watching an introductory video, or which had been acquired before the visit in other museums, institutions and interactions.
3
The ‘City’ system
The design of the City system was informed by the visitor studies described above, as well as by technical and interactional goals. The prototype explored co-visiting by people who know each other and share an interest in museum visiting, but who may not always be able to visit together because of geographical separation. The City system was designed for visits to a specific
2
exhibition: the Mackintosh Interpretation Centre (hereinafter ‘Mack Room’) in The Lighthouse. The Mack Room is devoted to the life and work of C.R. Mackintosh (1868-1928), Glasgow architect and artist. The exhibition combines textual and graphical displays with authentic artefacts, and over 20 screens presenting video and interactive material to visitors.
Figure 1
Figure 2
Figure 3
The City co-visiting system combines virtual environments (VE), hypermedia technology, handheld devices and ultrasound tracking technology, coordinated through Equator’s shared tuple space infrastructure, EQUIP (MacColl, Millard, Randell, & Steed, 2002) to allow three visitors, one on-site and two remote, to visit the Mack Room simultaneously. An ultrasound positioning system and a wireless communications network is installed in the Mack Room and the Lighthouse respectively. The on-site visitor carries a PDA that is location–aware and displays the ongoing positions of all three visitors on a map of the gallery (Figure 1). The two off-site visitors use two different environments: a web-only environment and a VE. The web visitor uses a standard web browser with an applet that displays another gallery map (Figure 2). The VE visitor uses a first person, 3D display with avatars representing the other visitors (Figure 3). All visitors share an audio channel, and wear headphones and microphones. The system also supports multimedia information for the off-site visitors in the form of web pages that are dynamically presented upon movement in the map or VE. This automatic presentation schematically follows the spatial organisation of the exhibition, so that all visitors may ‘look’ at the same display in the same location. In that respect the system supports interaction around corresponding exhibits in the Mack Room and in digital form: ‘hybrid exhibits’ (Brown et al., 2003).
4
Evaluation and Discussion
A set of user trials of the prototype system was carried out in the Mack Room. The trials aimed to offer us an initial understanding of users’ experience of the system in a museum setting, and how this might compare with traditional on-site co-visiting. Qualitative methods such as video and audio recordings and semi-structured interviews were used to analyse the users’ experience. The experience was highly interactive and retained many of the elements of traditional museum covisiting (Brown et al., 2003). The users of the system took advantage of the available shared resources, for example the audio channel and the map/VE, in order to effectively engage with their friends and the exhibition, but also contributed individual information to their interaction.
3
The trials also showed that direct interaction among the visitors was mainly achieved through verbal communication. Peripheral awareness was mainly facilitated by shared location and orientation, although sometimes one visitor would be on the periphery of talk between two others i.e. overhearing people discussing about an exhibit other than the one he or she was looking at. In that respect, the shared audio was used to compensate for the difficulty or unfamiliarity of gesture or deixis in this hybrid medium. The spatial arrangement of the information informed both the discussions between companions and individual assumptions of what others were viewing. Offsite visitors’ direct engagement with the exhibits was achieved through both spatial (map– and VE–based) and informational (link-based) navigation, with a slight preference for the former. Reflecting on the trials, as well as our design process, we aimed to balance a respect for the behavioural ecology of the setting with our awareness that our technologies cannot and need not replicate the traditional interactions between museum visitors. Our discussions about future design and development can be seen as involving three interlinked aspects of system design and user experience: the duration of use, the media used and the setting. Firstly, we suggest that future trials run for a longer period of time in order to offer greater insight into the process of appropriation: the development of new forms of interaction to compensate for the lack of traditional ones or to take advantage of new affordances. Each set of trial participants only used the system for one hour, on average, and became relatively familiar with the system’s resources for interaction but just began to appropriate them. Secondly, the use of media— especially audio—compensated for the coarse grain of spatial awareness and unfamiliar resources for gesture and deixis, but did not conform to traditional museum ‘etiquette’. The PDA visitor spoke more loudly and often than is traditional in museums, and was relatively unaware of the reactions of visitors to the Mack Room who were not part of our trial. Even though the accepted behaviour in a setting often changes over time, we do not wish to promote a style of interaction akin to the (apparently increasing) use of mobile phones inside museums and classrooms. To better fit with this setting of use, we are considering ways to support communicative achievements such as deixis and peripheral awareness of activity through slightly different media. Even simple changes in the type of headphones used by the PDA visitor may make him or her more aware of his or her own audibility, and of other people’s conversations and reactions. Lastly, we are aware that we could experiment with the same technology in a different setting. Slight modifications to our system now support its use outdoors in the city streets as well as inside the Mack Room. This lets us compare user experience in the current setting with other settings that may demand slightly different forms of peripheral awareness and engaged interaction. From a museum studies perspective, MR systems such as the one presented in this paper may have immediate benefits in two interrelated areas: accessibility of collections and educational activities. In the system trials, the combination of location–specific information with participants’ talk and motion proved to support visitors’ engagement with the collections and each other. The remote visitors’ interest in the exhibition was raised by this real–time feedback, and they expressed their desire to visit the traditional gallery space. On-site visitors also benefit from interaction with their on-line friends, e.g. they were often prompted to look at artifacts that they had missed at a first glance. In that respect, both local and remote visitors take advantage of the different perspectives imposed by their different media, and we suggest that maintaining a balance of diverse perspectives is essential. Moreover, the City system can support on-site or off-site guides in offering tours that address the needs and expectations of a range of local, remote and mixed audiences. More generally, MR systems may contribute to communication and collaboration between school and other educational groups. They may also support a rich contextualisation of
4
collections, for example in the use of ethnographic material, by fostering direct communication between visitors and communities of origin.
5
Conclusion
This paper has presented the implementation and study of a mixed reality system that allows onsite and on-line visitors to share a museum visit. Informed by studies of visitors to traditional museum settings, we have aimed to support direct or engaged interaction between visitors, mutual and peripheral awareness of activity, and location–specific display of multimedia information. In particular, our work suggests that rich user experiences that span on–site and off–site visitors can be based on mixtures of ubiquitous computing, hypermedia, and map– and VE–based interaction. In our ongoing work, we continue to explore novel combinations of media that support traditional goals such as the mutual reinforcement between visitors’ social interaction and their interpretation of cultural institutions, exhibitions and collections.
Acknowledgements We acknowledge the contribution made by our Equator colleagues C. Greenhalgh (U. Nottingham) and D. Millard (U. Southampton); the assistance of D. McKay (House for an Art Lover), L. Bennett and S. MacDonald (Lighthouse); the participation of the visitors who took part in our studies; the funding of the UK EPSRC; and Hewlett–Packard’s ‘Art & Science’ donation.
References Aoki, P. M., Grinter, R. E., Hurst, A., Szymanski, M. H., Thornton, J. D., & Woodruff, A. (2002). Sotto Voce: Exploring the Interplay of Conversation and Mobile Audio Spaces. Proc. ACM CHI 2002. Billinghurst, M., Karo, H., & Poupyrev, I. (2001). The MagicBook: A transitional AR interface. Computer Graphics, 25, 745-753. Brown, B., MacColl, I., Chalmers, M., Galani, A., Randell, C., & Steed, A. (2003). Lessons from the Lighthouse: Collaboration in a shared mixed reality system. To appear in Proc. ACM CHI 2003. Cheverst, K., Davies, N., Mitchell, K., Friday, A., & Efstratiou, C. (2000). Developing a Contextaware Electronic Tourist Guide: Some Issues and Experiences. Proc. ACM CHI 2000. Falk, J. H., & Dierking, L. D. (2000). Learning from Museums. Walnut Creek: Altamira Press. Galani, A. (2003). Mixed Reality Museum Visits: Using new technologies to support co-visiting for local and remote visitors. Museological Review, 10, (to appear). Koleva, B., Schnadelbach, H., Benford, S., & Greenhalgh, C. (2000). Traversable Interfaces Between Real and Virtual Worlds. Proc. ACM CHI 2000. Lord, M. (1999). Editorial. Museum International, 51(4), 3. MacColl, I., Millard, D., Randell, C., & Steed, A. (2002). Shared visiting in EQUATOR City. Proc. ACM Collaborative Virtual Environments 2002, Bonn. McManus, P. (1987). It's the Company You Keep...: The social determination of learning-related behaviour in a science museum. Intl. J. Museum Management & Curatorship, 6, 263-270. vom Lehn, D. (2002). Exhibiting Interaction: Conduct and Participation in Museums and Galleries. Ph.D. thesis, King’s College, University of London, UK.
5
