Augmented Space: Bringing the Physical Dimension into Play Elena Not, Daniela Petrelli, Oliviero Stock, Carlo Strapparava, Massimo Zancanaro Istituto per la Ricerca Scienti ca e Tecnologica, I-38050 Povo/Trento, Italy e-mail: fnot j petrelli j stock j strappa j
[email protected]
1 Introduction In an \ideal guided visit" to a space (as an exhibition, a museum, an archaeological site, a city, and so on) what a visitor would like to have is a exible companion that helps him in visiting the museum or town as an information space. Of course museums or exhibitions (or towns) are physically organized in some rigid way. When organizing the physical layout of the exposition areas a speci c criterion for items positioning has to be chosen, according to a \default" perspective of information presentation, either chosen by an architect or imposed by geographical constraints. It may happen that the default physical organization does not meet directly the visitor's expectations, possibly making it dicult to build a personal route. In the case of a museum, paper maps, provided at the entrance, may help visitors navigate through the di erent rooms to nd those items that meet their own interests. Written and tape recorded guides can provide useful description of the objects displayed but they do not guarantee exibility, either because of technological constraints (i.e. audio tapes force a prede ned path) or because the descriptions are not coherently related to each other. On the other hand, virtual museums may o er a more exible object display determined by the visitor's individual preferences and history of interaction (as happens for example in the ILEX system [Knott et al., 1996]). New hardware technology allows the fruition of virtual repositories of information while enjoying the physical space: for example, kiosks or portable devices may allow the access to a portion of the virtual space relevant for the object in front of the visitor. Adaptive and dynamic hypertext technology may be exploited to tailor the presentation to the visitor interests, the actual context of the visit and so on. Figure 1 illustrates the situation in which a hyperspace is connected to the physical space according to a given ontology. 1
Virtual Space Semantic organization of domain concepts
Physical Space
Figure 1: Information (virtual space) might be directly related to physical space as in phone or CD-ROM audio guides, or might be connected through an ontology as in distributed kiosks.
2 Physical+Virtual vs. Augmented Anyway, new hardware technology and adaptive and dynamic hypertext technology may be exploited to go beyond the simple juxtaposition of the two spaces. In fact, the peculiarity of inserting a hypertextual structure in a physical space generates new ways of navigating information: 1. moving around the physical space, approaching the various cases, the visitor implicitly \clicks" on meaningful points of the hypertext (the system is able to track the visitor's position by means of sensors); 2. as in solely virtual spaces, the visitor may explore the sub-network of the hypertext nodes related to the physical object he is standing in front of; in addition he can proceed with the exploration within the physical dimension: 3. after getting information about the object, the visitor may decide to move in a direction that was explicitly or implicitly suggested by the message (for example because a comparative description was heard that introduces a new interesting and related object). But the visitor may also decide to suddenly change the suggested tour thread. Physical hints may attract his interest more than the proposed hypertextual links: he may be distracted by interesting objects close by or he may have personal intuitions about semantic relations between objects that make him stray from the undertaken path. The system can try to cope with movements that drop out 2
from the hypertextual structure applying techniques coming from the area of dialogue modeling: for example, tracking topic shifting or modifying its assumptions on the user's preferences. Exploiting the features emerging from the integration of a physical space with a virtual space we can build an augmented space where the system can play a new role in the information-providing task: not only can the system provide the visitor with information tailored on his own interests and interaction history, but it can also support the visitor in his own exploration of the physical space, helping him to nd what he is looking for and suggesting new interesting physical locations.
Figure 2: How a virtual space is dynamically created in an augmented space Figure 2 illustrates a scenario in which a exible hyperspace is created taking into consideration both the semantic relations, the physical constraints as well as the user model and the interaction history. As an example of how an augmented museum would assist a personalized visit, let us suppose that the visitor is in front of a case containing a stu ed squirrel. The system may propose a description of the animal, suggesting additional explorable information about its predators. If the visitor asks for this type of proposed elaboration, the system may provide a description of animals like eagles, foxes, etc. If the cases containing the predators are close by, the system could suggest to have a look at them, explaining how to reach the new location. However, if the predators are shown very far away and the visitor has displayed interest just for rodents, the system may decide not to propose the site. We are currently exploring the potentialities of augmented spaces inside a project called HyperAudio. HyperAudio aims at developing a portable system 3
(running on a palmtop equipped with an infrared sensor and headphones) which is expected to guide a visitor throughout an exposition space, providing information by means of audio messages: the visitor can get instructions for nding the objects of interest, hear descriptions with references to what already seen and what to be seen shortly, ask for additional information and receive suggestions on alternative routes. Therefore, he is provided with a personalised guide for exploring the augmented physical space. Special research emphasis will be put on the investigation of the new problems arising from the interaction in a physical space.
2.1 A new notion of Path
In any information space (e.g. a museum setting) the notion of path is twofold:
it has a semantic dimension because the purpose of the movement is to get information and not simply to move around; it has the usual spatial dimension in which the visitor goes from one point to another. The spatial dimension itself is twofold: - the movement can be in the physical space, involving perceptual experiences with real objects and physical tiredness, or - the movement can be in the virtual space, i.e. browsing and jumping in the hypertext (as soon as the hypertext is exible the topology of this space can be dynamically reorganized).
In an information space there may exist prede ned paths, i.e. sequences of physical/virtual locations preselected by the designer, that o er particular perspectives on the content. In any case the visitor is not compelled to follow a prede ned path, but may take diversions, both in physical and virtual space: we will call the actual sequence of undertaken steps the actual path. In an augmented information space, alternative steps -dynamicallydeterminedcan be suggested at any physical/virtual location, according to the visitor interests, the previous steps and possibly educational strategies. In evaluating concepts' relevance and vicinity, the system has also to take into consideration the peculiarities of the two dimensions of the information space (semantic and spatial). One key element is the di erent notion of distance: there is a semantic distance as a measure of how much two concepts are related to each other and there is a spatial distance as a measure of the length of the path and the e ort required to follow it. Distances in the physical and in the virtual space in uence the system's decisions when suggesting new paths.
4
3 Architecture for a Portable Presentation System The most suitable hardware for a system helping the visitor in the exploration of an augmented space is a mobile platform of reduced size (e.g. palmtop computer) together with some devices used for position localization, from the simple con rmation of the visitor's presence in front of the object, up to the determination of his distance and orientation in the room.
3.1 Implicit and Explicit Input
The visitor explicitly interacts with the system: in the case of a palmtop, the person uses a pen on the palmtop screen to get suggested follow-up messages (for link) selection or control commands activation. Interaction is also implicit as the system monitors the visitor's movements by means of sensors. It is up to the system to decide if the movement has to be intended as a relevant input or not. For example, if the system has suggested to go to a given location, sensors activated along the route will not trigger presentations (anyway the information that the visitor has passed through other locations can be used for other purposes).
3.2 Output
System output includes either content presentation and next steps suggestion. In our project HyperAudio, the content will be presented in linguistic form through an audio device. Commands, museum maps and follow up links will be presented as clickable icons or text.
3.3 Modules and Knowledge Sources 3.3.1 Macronodes and Ontology
In the HyperAudio project, the virtual space is designed so that its contents and in its structure can be used in an adaptive way. Information is organized in a network of macronodes (see Fig. 3). Each macronode includes a set of linkable audio les (some will be used for the actual presentation), pointers to other macronodes (including the particular rhetorical relations among them), the type of text (e.g. general description, statistical data, and so on), a pointer to a relevant semantic concept in the ontology, and possibly a link to a physical location.
3.3.2 User Model and Interaction History
The system maintains a model of the particular user with whom it is currently interacting. Our user model has two kinds of information: what the user has 5
Semantic organization of domain concepts
Virtual Space
Concept Macronode ... Macronode Type Location
Concept Macronode ... Macronode Type Location
Concept Macronode ... Macronode Type Location
Physical Space
Figure 3: The macronode structure been exposed to or is assumed to know, and what the user seems to be interested in. The user's knowledge model is based on an initialization (depending on a user pro le, e.g. expert or novice) and on a modelization of what the user has become aware. The user's interest model provides a model of the potential interest of the user and consists on an activation/inhibition weighted network whose nodes are associated with ordered sets of individual concepts. The user model is exploited during the process of content generation in order to avoid repetition and false implicatures, and to promote other objects potentially relevant from the user point of view. The updating of the user model depends on both what the user really chose and what he rejected (i.e. stopping an audio play, not clicking a particular choice, unexpectedly changing the physical path, and so on). The system also maintains a history of the interaction that can a ect further output (e.g. introduction of comparisons, reminding preceding presentation sequences, avoiding repetitions).
3.3.3 Generator/assembler
The role of the generator is to build the content presentation and the suggested paths. Di erent degrees of exibility could be envisaged for the generation task. In Hyperaudio, a simple temporary solution is adopted: the generator is an intelligent assembler that, rst, chooses the most appropriate subset of audio les from the current macronode and plays them. In the second place, it decides which rhetorical relations to other macronodes are relevant in the present con6
text and displays them. Finally, if needed, it integrates the presentation with maps and spatial directions using some limited capacity of reasoning about spatial relationships. For example, this can be exploited for providing instructions on how to reach a certain location or relevant rooms.
4 Enhancing adaptivity in an Augmented Space The system adaptivity might emerge in di erent forms, both in the information provided and in the further steps suggested. The adaptivity in content presentation may emerge from:
choice of language style: the context of interaction may suggests di erent linguistic choices, for example the selection of deictic referring expressions (instead of inde nite or de nite ones) in case an audio message describing an object is heard when the visitor is in front of the object (possibly touching it); presentation preparation: the content of the message can be produced with di erent degrees of complexity, obtaining from less to more exible presentations ([Dale et al., 1997]). The presentation could be obtained by:
- simply selecting and concatenating pre-existing audio/text les; - modifying existing audio/text les through the introduction of cohesive
devices that improve the uency of the nal message (anaphora introduction, introduction of linking sentences that signal comparisons, prosody adjustments, choice of cue-phrases, and so on); - generating from scratch, with a exible content selection and organization, starting from the knowledge represented in the ontology. Note that not always this is the most appropriate solution even if you do not consider the technical diculty: pre-existing texts prepared by critics or domain experts can be good material to exploit.
The simplest form of adaptivity in the selection of the next steps to suggest consists in the identi cation of \locally" interesting links, according to user model and interaction:
new virtual locations to visit may be suggested depending on rhetorical structure and text typology; new physical locations to visit may be suggested depending on ontology relevance and physical constraints.
Adaptivity can be further enhanced when the system pursues more ambitious goals concerning educational strategies and has to negotiate between the visitor's 7
initiative and its own promotion contraints. A simple example of this is that a system attracts the visitor toward a prede ned path when this is compatible with the visitor's interests.
References
[Brusilovsky, 1996] Peter Brusilovsky. Methods and techniques of adaptive hypermedia. User Modeling and User Adapted Interaction (Special Issue on adaptive hypertext and hypermedia), 6(2-3), 1996. [CAC, 1993] Communication of the ACM (Special Issue on ComputerAugmented Environment), volume 36, 1993. [Dale et al., 1997] Robert Dale, Maria Milosavljevic, and Jon Oberlander. The web as dialogue: the role of natural language generation in hypertext. In
Proceedings of the AAAI Spring Symposium on Natural Language Processing for the World Wide Web, 1997. [Knott et al., 1996] Alistair Knott, Chris Mellish, Jon Oberlander, and Mick O'Donnell. Sources of exibility in dynamic hypertext generation. In Proceedings of the 8th International Workshop on Natural Language Generation,
Herstmonceux Castle, UK, june 1996. [McNamara et al., 1992] Timothy P. McNamara, John Halpin, and James Hardy. The representation and integration in memory of the spatial and nonspatial information. Memory and Cognition, 20(5):519{532, 1992.
8
