Presence, the sense of being in a virtual environment (VE), is analysed in an embodied ... user of a VE lead to what we call immersion. Immersion can be ...
Embodied Presence in Virtual Environments Thomas Schubert Frank Friedmann Friedrich–Schiller–University Jena, Germany Holger Regenbrecht Bauhaus University Weimar, Germany Abstract Presence, the sense of being in a virtual environment (VE), is analysed in an embodied cognition framework. We propose that VEs are mentally represented as meshed sets of patterns of actions and that presence is experienced when these actions include the perceived possibility to navigate and move the own body in the VE. A factor analyses of survey data shows 3 different presence components: spatial presence, involvement, and judgement of realness. A path analysis shows that spatial presence is mostly determined by sources of meshed patterns of actions: interaction with the VE, understanding of dynamics, and perception of dramatic meaning.
1 The role of psycholog y in virtual environments When we use virtual environments (VEs), we often experience presence, the subjective sense of being in the virtual place. Presence is observable when people interact in and with a virtual world as if they were there, when they grasp for virtual objects or develop fear of virtual cliffs [1]. The presence phenomenon is at the centre of psychological research in VEs. Applications of VEs for psychological research and applications of psychological findings for hardware and software design need an understanding of what presence is, how it develops, and how we can measure it. It has been said that psychology "determines what is and what is not virtually 'real''' [2, p. 244] If this is so, what can psychological theories tell us about when and how we experience presence and construct virtual environments as our reality? In fact, already in the 60's the radical constructivist Heinz von Foerster used an early stereoscopic virtual reality system to display four–dimensional virtual objects
We thank Gabi and Hartmut for their hospitality during our stay at Pfuhlsborn, where we developed the ideas for this paper.
[3]. He showed in his experiment that only those participants who were allowed to interact with the objects via two joysticks were able to understand the forth dimension, as opposed to those who merely watched the interaction. Thus, acting in an environment led to a perceived reality of this environment . In this vein, we want to present an approach which frames presence as embodied cognition – as the outcome of the active interpretation of a virtual environment. We argue that presence emerges when possibilities of bodily action in the virtual world are mentally represented. The interpretation of the environment through action and how it leads to the attribution of reality and Wirklichkeit is the topic of this paper.
2 Psychological concep ts of presence Virtual environments are computer–based three–dimensional spaces presented via various media, such as pictures on head–mounted displays or monitors. They can also be presented via text only, as it is the case in text–based virtual environments. The stimulus field depicting a three–dimensional space and its coupling with the user of a VE lead to what we call immersion. Immersion can be described objectively and should be distinguished from presence. Presence, in contrast, is a psychological phenomenon. It has been defined as the "the participant's sense of 'being there' in the virtual environment." [4, p. 131]. Biocca [5, sect. 5.1.1.] has noted that "users experiencing presence report having a compelling sense of being in a mediated space other than where their physical body is located...". Presence is such a central element of virtual reality that it is seen as a part of its definition [6]. Immersion and the content which is presented by it on the one side and presence on the other side describe a dichotomy between presentation of stimuli and psychological experience. It is tempting to see these two in an uni–directional causal relationship, where stimuli govern the psychological experience. We think however that the bodily and cognitive activity of the user – his interaction with the virtual world on various levels – is the true source of presence [6]. It is the interplay of bodily and cognitive processes that needs to be described. Interestingly, the user's body – although seemingly disappearing in most VE applications – has been at the centre of theoretical concerns since the beginning of presence research, mostly as a result of the application of J. J. Gibson's [7] ideas to the conceptualisation of presence. The emphasis of bodily action and motor processes for perception is an important point of this work and is compatible with new proposals in cognitive science such as that from Varela, Thompson & Rosch [8]: "Sensory and motor processes, perception and action, are fundamentally inseparable in lived cognition.... Perception consists in perceptually guided action." (p. 173). Modern cognitive theories like the mental model theory look promising [1] for an extension of these ideas. Such an approach has been suggested by Biocca [5], who assumes that "users are ... constructing a mental model of the virtual space and responding to and attending to cues in the virtual mediated environment." (sect. 6.1.2). The combination of both ideas, the emphasis on the body and the concept of mental models, is possible in the embodied cognition framework [9–11]. The main focus of this approach is memory and its function. To approach this goal,
a theory of situational conceptualisation and meaning grounded in embodied cognition has been developed. We will focus on this aspect of the proposal, ignoring the aspect of memory for the moment.
3 Glenberg's framewor k for embodied cognition Glenberg [11] argues that only a part of the interaction with the environment can be managed without a representational system. Tasks like differentiation, however, require more than is available from an optical flow field. With virtual environments gaining in complexity and richness of possible actions, those tasks will become more and more important. Glenberg proposes that in the service of those needs, a conceptualisation of the situation is created by representing meshed patterns of actions possible in this situation. Patterns of action derived from the projectable properties of the environment are combined (or meshed...) with patterns of interaction based on memory. The two patterns can combine because they are both embodied, that is, both are constrained by how one's body can move itself and manipulate objects. The resulting pattern of possible actions is a conceptualization: the possible actions for that person in that situation. Thus meaning of an object or a situation is a pattern of possible action. (p. 4)
The cognitive representation of an environment consists of possible patterns of actions. It thus captures the relation between our body and the objects in our environment, forming the meaning of the situation. At least two steps of interpretation are inherent in the understanding. First, projectable properties are actively created by the individual agent. Second, non–projectable properties retrieved from the memory are meshed with the representation in the search for an understanding of the situation. Since the conceptualisation is driven by the need to survive in a dangerous environment, the projectable stimuli from the environment have priority for the understanding. This priority is provided by the process of clamping. "Clamping projectable properties ensures that experiences are individuated or situated" [11, p. 6f]. To explain the ability of conscious remembering and understanding of language, Glenberg introduces the concept of suppression, which is central for our own application of the framework on presence: "In the service of prediction, we have developed the ability to, if not ignore, at least to suppress the overriding contribution of the current environment to conceptualization." [11, p. 7] This suppression is easy when the to be represented transformations follow embodied constraints. When however brute force manipulation of perceptual symbols is necessary, the construction of an embodied understanding will be much harder. As the last point in this overview of some of Glenberg's ideas we want to mention the special feel of memory described by him. He explains that this feel of memory
is rooted in the fact that we remember our own actions which we have performed with our own bodies.
4 The understanding o f virtual worlds: suppression and construction Understanding of a virtual environment is, just like the understanding of language, the processing of mediated information. The understanding will also result in a meshed set of patterns of actions possible in this situation. "We understand language (that is, it takes on meaning) in the same way we understand a physical situation – in terms of possibilities for action." [11, p. 41] Just like the understanding of language, the understanding of a virtual environment will need the suppression of conflicting projectable features from the real environment – namely the stimuli from the head–mounted display (HMD) or screen, cables, sound from the environment, the constraints of the field of view and so on. Thus, the understanding of a VE is the process of an active suppression of stimuli from the real world and the construction of meshed sets of patterns of action on the basis of mediated stimuli – the visual representation of the VE and, if available, stimuli perceived with other senses. The resulting meaning of the virtual world consists of the possible actions in it. We assume that the central interaction with an environment is the navigation of the body or at least body parts in it. Further kinds of interaction are the manipulation of objects and the influence on agents (objects in the virtual world which have their own intentions and which perform actions themselves). We can now propose an interpretation of presence as embodied presence: Presence develops from the representation of navigation (movement) of the own body (or body parts) as a possible action in the virtual world. Presence is the outcome of media perception. In the process of developing presence, a mental model of the virtual three–dimensional space is constructed, consisting of the possible actions in this space. The possible actions of the body are central in this model. Stimuli from the real environment must be suppressed for presence to emerge. The more the mediated stimuli follow embodied constraints (e.g., coupling with body movement), the easier is the construction. Because the virtual environment is perceived in terms of embodied action, a feeling equivalent to the feel of memory mentioned above develops: This is what we call the sense of presence.
5 Evidence for embodi ed presence in self reports One way to test this model would be to experimentally manipulate the involved processes suppression and construction, probably using a sophisticated VR system. Indeed, most of the presence research took place in VR installations with HMD environments. While this research has told us a lot about the technical factors which contribute to presence, we want to argue for a supplementary research approach. This approach emphasizes two points: First, we concentrate on how
users experience their interactions with a VE. Second, in order to do this, we investigate a wide range of rich and complex VEs and a wide spectrum of subjective experiences with survey methods. The research questions are then: Which different experiences will be reported by the users, and which presence and immersion components can be found? On the basis of the presented theoretical model, the following predictions can be made. Presence should involve at least two components: One component related to the suppression of the actual environment and the focusing on the VE, and a second component related to the mental construction of a space out of the VE in which the body can be moved. These two factors were described above as central in the understanding of the VE as being the own environment. Furthermore, the interaction experiences should include those that relate to interactions between body and VE which are the basis for and the topic of meshed patterns of actions. Data which confirm these predictions and describe additional experiences were presented by Schubert, Regenbrecht and Friedmann [12]. In this study, we surveyed 246 users of different VEs, including VEs using HMDs and CAVEs, text–based MOOs and MUDs, but mainly users of screen–based 3D games. Today's video and PC 3D games present highly developed, complex VEs, incorporating sophisticated visual, aural, spatial and dramatic content. The participants answered a 75 item questionnaire. The questionnaire consisted of items drawn from various presence and immersion scales, including almost all published presence items from the last years. Additionally, we developed items which specifically asked about body–VE relations and construction of the VE as being the own environment.* The data were analysed in factor analyses. Details are presented elsewhere [12], but we want to describe here the factors and then present additional path analysis on these data which relate to the framework presented above.
5.1 Immersion and Presence Factors The data were first factor analysed using oblique rotation. Eight factors emerged in the factor analysis. Three of them describe components of presence: (1) the relation between the VE as a space and the own body (spatial presence), (2) the awareness devoted to the VE (involvement) and (3) the sense of reality attributed to the VE (realness). These factors were categorised as presence factors because they included only subjective reports of how the users experienced the environment, rather than descriptions of interactions between user and VE or descriptions of the technical side of the VE. Factor (1) describes exactly what is commonly included in the presence definition. Interestingly, the formulation "sense of being in a place" is actually the item with the highest loading on this factor. This factor confirms that the construction of the VE as being the own environment involves the construction of meshed patterns of
*
See [12] for a description of the survey.
navigational actions. Factor (2) isolates the attention side of presence, the concentration and focus on the VE and the suppression and forgetting of the real environment, relating to the suppression process described above. Factor (3) was not predicted by us. It combines items which involve the comparison between VE and real world concerning their "realness". We think that this is some kind of judgement elicited by our questions and that it relates closely to presence, but is probably not a part of the actual presence experience itself. Five additional factors contain items which assess the stimuli presentation and properties of the interaction between user and VE: (4) the sensory quality, describing richness and consistency of the multimodal presentation (quality of immersion), (5) perception of dramatic content and structures (drama), (6) awareness of interfaces that distract from the VE experience (interface awareness), (7) the possiblity to explore and actively search the VE (exploration), and (8) the ability to predict and anticipate what will happen next (predictability). Table 1. Factors of experiences in VEs Presence
Immersion
Spatial Presence SP Involvement Realness
INV REAL
Stimuli presentation Quality of Immersion Drama Interface Awareness
Interaction QI DRA IA
Exploration EXPL of VE Predictability PRED
These findings are highly consistent with presence factors described and predicted by Held and Durlach [13], Sheridan [14] and Witmer and Singer [15]. However, we can extend their analyses in the following way: First, the factor analysis shows that the presence components and the immersion factors are indeed separate constructs. Secondly, presence itself is not a unitary construct, but consists of at least 3 components. Thirdly, using additional exploratory path analysis we can investigate how well the various immersion factors can predict the presence components. This will be the topic of the remainder of this paper.
5.2 Modelling causal relatio ns between Immersion and Presence Path analyses are commonly used for confirmatory tests of models of causal relations between variables. However, they can also be used for exploratory analyses. In the analyses described here, we investigate if the presence factors can be significantly predicted by the immersion factors in a path model. It is important to note however that the direction of the causal flow implied by our model is an a priori assumption. That is, the direction of the relations can not be tested with data like ours but must be tested in a controlled experiment.
QI 1
SP
DRA
1
REAL
IA
EXPL
INV
eSP
1
eREAL
eINV
PRED Figure 1. Initial path model.
We started with a model which incorporated the following assumptions: First, the immersion factors determine spatial presence, involvement and judgement of reality. Second, spatial presence and involvement also have effects on judgement of reality. This model is depicted in Figure 1. The variables used in this model were computed as the mean scores of those items clearly loading on each factor (loadings higher .40). We used the same data set as in the factor analyses (see comment below). This model was tested with the statistical package AMOS. The model has one degree of freedom; therefore fit and modification indices can be computed. The model does not fit the data (χ²(1, n=246)=39.348, p
