Physiological Linkage of Dyadic Gaming Experience

513080

research-article2013

SAG45110.1177/1046878113513080Simulation & GamingJärvelä et al.

Physiological Linkage of Dyadic Gaming Experience

Simulation & Gaming 2014, Vol. 45(1) 24–40 © 2013 SAGE Publications Reprints and permissions: sagepub.com/journalsPermissions.nav DOI: 10.1177/1046878113513080 sag.sagepub.com

Simo Järvelä1, J. Matias Kivikangas1, Jari Kätsyri1, and Niklas Ravaja1,2

Abstract Dyadic gaming experience was studied in a psychophysiological experiment where conflict structure and the presence of an artificial intelligence (AI) agent in a turnbased game were varied in four different conditions. Electrocardiographic and electrodermal activity signals of 41 same-gender dyads were recorded to study joint changes in their physiological signals. A strong physiological linkage was found within dyads in all conditions, but the linkage scores did not differentiate between conflict modes. The only significant difference in linkage between conditions was an increase when the AI agents were not present. In addition, linkage was associated with different self-report scales assessing social presence. These results suggest that social presence and physiological linkage within dyads are higher when dyads can focus on each others’ actions without distractions. Keywords AI agents, digital games, gaming experience, physiological linkage, psychophysiology

For many people, gaming is a fundamentally social activity by nature. Several studies have shown that playing with other people constitutes a large motivational factor for gaming (Kallio, Mäyrä, & Kaipainen, 2011; McGee, Merritt, & Ong, 2011; Sherry, Lucas, Greenberg, & Lachlan, 2006); in fact, recent neuroimaging evidence has shown that playing against real people enhances reward responses to game events at the neural level (Kätsyri, Hari, Ravaja, & Nummenmaa, 2012). A vast amount of social 1Aalto

University, Helsinki, Finland of Helsinki, Finland

2University

This article is a part of the symposium: Development of a Finnish Community of Game Scholars Corresponding Author: Simo Järvelä, Aalto School of Business, Department of Information and Service Economy, Aalto University, P.O. Box 21220, Helsinki, FI-00076 Aalto, Finland. Email: [email protected]

Downloaded from sag.sagepub.com by guest on January 15, 2016

25

Järvelä et al.

dynamics is involved in digital gaming even in games that are superficially singleplayer and non-social, and naturally even more in co-located or mediated multiplayer games (de Kort & Ijsselsteijn, 2008; Stenros, Paavilainen, & Mäyrä, 2011). Social presence (Biocca & Harms, 2003) has been defined as the sense of being with another and it is responsive to the numerous social characteristics of the socio-spatial context in which the gameplay takes place (de Kort & Ijsselsteijn, 2008) which makes it particularly relevant when examining social interactions in gaming. Social interactions are challenging to quantify in experimental settings—for example, self-reports are influenced by cultural factors and not all emotional reactions are consciously accessible (Mauss & Robinson, 2009; Paulhus & Vazire, 2008; Robinson & Clore, 2002). We have previously suggested physiological linkage as a method for assessing social interaction and presence in dyadic or multiplayer games (Ekman et al., 2012). Physiological linkage refers to the synchronization (i.e., mutual variation) of physiological activity between two or more people, such as a mutual increase in heart rate (HR) during an intense shared experience. A more in-depth account of the use of psychophysiological linkage in the context of gaming can be found from Ekman and colleagues (2012; see also Mandryk, 2008, on the psychophysiological method in general). The physiological activity in question can be any quantifiable signal with enough variability during the experiment; such as heart rate, skin conductance (electrodermal activity [EDA]), contractions in facial muscles (measured by facial electromyography [EMG]), or brain activity (e.g., as measured with electroencephalography [EEG]). Outside the linkage approach, these signals have been measured increasingly often during the recent years also in game research (for a review, see Kivikangas et al., 2011). The mirror neuron system (Rizzolatti & Craighero, 2004; Rizzolatti, Fadiga, Gallese, & Fogassi, 1996), which is involved in imitating the perceived movements of other humans, is a plausible neurophysiological mechanism underlying physiological linkage. According to the embodied cognition theory (see, for example, Barsalou, 2008; Mahon & Caramazza, 2008; Niedenthal, Barsalou, Winkielman, Krauth-Gruber, & Ric, 2005), in order to understand, for example, the facial expressions of others, the same brain areas of the observer must be activated that are used in producing them (Niedenthal, Brauer, Halberstadt, & Innes-Ker, 2001). That is, for example, the perception of a smile causes similar brain activation as when the person would be smiling himself or herself (Dimberg & Öhman, 1996). This in essence means that the perception and mimicry of other persons’ emotional expressions and movements lead to similar emotional states in the observer (Hatfield, Cacioppo, & Rapson, 1993; Rizzolatti & Craighero, 2004). This in turn may lead to the phenomenon of emotional contagion where emotional states are transferred from a person to another (Barsade, 2002; Hatfield et al., 1993). These same mechanics also explain empathy, that is, how the emotions of others are perceived (Levenson & Ruef, 1997). Physiological linkage was first used in analysis of marital interactions, where several linkage indices were associated with conflicting conversations (Levenson & Gottman, 1983). More recent studies have shown that physiological linkage is also related to empathy (Levenson & Ruef, 1992) and performance (Henning, Boucsein, & Gil, 2001), which highlights the fact that linkage is associated with also other than


26

Simulation & Gaming 45(1)

negative interactions. A recent suggestion is that linkage captures the intensity of social interactions that is elevated in, but not specific to, interpersonal conflicts (Chanel, Kivikangas, & Ravaja, 2012). Consequently, physiological linkage could be a viable method for evaluating social interactions also during digital game playing (cf. Ekman et al., 2012). However, given that physiological linkage has received little research attention, particularly so in the gaming context (for exceptions, see Chanel et al., 2012; Spapé et al., in press), solid established methods for calculating physiological linkage still do not exist. Consequently, our approach in the present investigation is largely exploratory and pioneering in nature. Different game modes with varying levels of cooperation and competition are one of the central features in digital multiplayer games that affect social interaction. To test these measures, we set out to investigate how various modes of competition and the presence of an artificial intelligence (AI) agent (i.e., computer-controlled game characters) affect the physiological linkage between two players when playing a digital turn-based game. Switching from cooperative to competitive mode (e.g., playing on the same side vs. different sides) fundamentally changes the players’ social relationship with each other, yet at the same time the actual game content may remain largely the same. Division between competition and cooperation, of course, is not a natural dichotomy, but rather a simplification of how the structure of conflict defines the goals of individual players (e.g., coinciding and conflicting goals in cooperation and competition, respectively), that may be partly or wholly overlapping. Furthermore, the players do not necessarily always play “correctly”: Even if the conflict structure defines a competition between the players, the players may have non-competitive attitudes when the situation provides other entertaining elements (Ekman et al., 2012; Salen & Zimmerman, 2004; Stenros et al., 2011). In our previous studies, we have found some evidence for arousal and experienced social presence differences that depend on the conflict structure (Kivikangas, Kätsyri, Järvelä, & Ravaja, 2013). Previous studies have also shown that competing against human players is more arousing and elicits more positive emotions than competing against AI controlled opponents (Ravaja et al., 2006; see also Mandryk, Inkpen, & Calvert, 2006). Another potential factor in social interaction is the mere presence or absence of AI agents in addition to its role as an opponent or collaborator. The concept of social presence has from the start covered also AI agents that are experienced to some extent as genuine social actors (Lee, 2004; Nass & Moon, 2000), although it has also been recognized that differences exist (Lim & Reeves, 2010). Whether the social interaction between humans is affected by the presence of AI agents remains unclear. In the present study, we investigate whether social interaction within player dyads, as well as their physiological linkage, is affected by (a) various modes of competition and cooperation, and (b) the presence of AI agents. We expect that varying the level of competition affects the level of physiological linkage so that competitive modes increase the linkage between players through more intensive social context. Cardiovascular reactivity to stress is a well-known phenomenon (Brownley, Hurwitz, & Schneiderman, 2000), and along with the established effects of arousal on electrodermal activity (Dawson, Schell, & Filion, 2000), the changes in the intensity of social


27

Järvelä et al.

interaction are likely to be most evident on physiological signals that are related to bodily activation or arousal; such as ECG (electrocardiogram) and EDA (electrodermal activity). We also expect that physiological linkage is correlated positively with social presence self-reports, reflecting its association with social interaction.

Method Participants The participants were 100 Finnish university students recruited in 50 dyads. The dyads were always of same gender with 29 male and 21 female dyads with age ranging from 18 to 32 (M = 22.9) years. The participants in the same dyad had volunteered for the experiment together as friends. Due to technical difficulties, 9 of the dyads had to be removed from the physiological data set, which resulted in 82 participants in 41 dyads.

Stimulus The participants played HEDGEWARS (http://hedgewars.org), an open-source clone of a popular commercial game WORMS by Team 17 (http://worms.team17.com). HEDGEWARS is a turn-based artillery game with a two-dimensional map and ballistic shooting (Figure 1). In a typical multiplayer setting, each player controls a team of several hedgehogs. Teams may also be controlled by AI players, and it is possible to group any teams together so that they are playing on the same side (e.g., two human players against one AI player). This inherent flexibility allowed us to set up various levels of cooperation and competition in our experimental setting (see “Procedure” section). We chose a turn-based game (cf. Chanel et al., 2012) so the stimulus would be identical to both players while the playing activity cycle would switch between actively playing when their own turn and audience mode on the other player’s and the computers turn. When the game dynamic is changed from repetitive cycles of simultaneous action to much slower cycles of one player acting at the time while the other is only watching, simultaneous movements and action is not a potential source of linkage. The aim of the game is to be the last team on the map, by reducing the health of the other sides’ hedgehogs to zero by shooting at them, typically several times, or by managing to knock them in to the water. During each turn, one of the players has 45 seconds to move one hedgehog in the player’s team, choose one of the available weapons, and shoot. More complicated weapons were disabled from the game to decrease the plausible advantage of the more experienced players. Turn order was randomized. The game was run on KUBUNTU LINUX desktop computer and projected to a white screen with a video projector. All game events were annotated automatically and saved to a log file.

Procedure The participants arrived in dyads and signed informed consent forms before the experiment begun. They could practice the game while the electrodes were attached to them,


28


Figure 1. Screenshot of HEDGEWARS.

after which they took part in a 5-minute resting period. The participants sit in front of the screen and shared the same mouse and keyboard as game controllers, using them in turns. To enable comfortable and equal use of controllers within dyads, the participants could move the keyboard and mouse freely to suit their own positioning at the start of their own playing turn. The participants played HEDGEWARS in four different conditions in a randomized order; the four conditions were as follows: 1. Full cooperation: The participants were playing in a single team (consisting of six hedgehogs) against one AI team (six hedgehogs). 2. Competitive cooperation: The participants were playing in two teams (three hedgehogs) on the same side against two AI teams (three hedgehogs). The condition was otherwise similar to Condition 1, except that dividing the players’ side into two teams made the kills of each team separately visible, allowing us to instruct the players to competing against each other for number of kills. 3. Competition, with AI allies: Both participants were playing one team (three hedgehogs) against each other, both with one AI team (three hedgehogs) on their side.


29

Järvelä et al.

4. Competition, without AI allies: The participants were playing in their own team (both with three hedgehogs) against each other, without AI teams in the game. Conditions 1 through 3 represent different aspects of cooperation and competition: In Condition 1, only cooperation, in Condition 3 (and 4) only competition, and in Condition 2 cooperation and competition at the same time. The fourth condition was included to test the effect of presence/absence of AI agents in the game. The number of hedgehogs in each condition was decided so that the amount of controllable characters remains the same throughout the experiment. In the first condition, both participants are in the single team so the number of hedgehogs is six instead of three. The alternative was to keep the amount of opponents constant; this would have meant six hedgehogs in condition four. However, ultimately we chose the former option in order to keep the playing as an activity as similar as possible from condition to condition. Before the experiment, the participants had filled out a background questionnaire. Before and after each condition, the participants filled out a series of self-report questionnaires on separate computers next to them while their psychophysiological data were recorded for the whole duration of the experiment.

Data Collection Physiological data acquisition. The physiological signals were recorded from participants with the Varioport-B portable recorder systems (Becker Meditec, Karlsruhe, Germany). Electrodermal activity (EDA) was recorded with Varioport 16-bit digital skin conductance amplifier (input range = 0-70 microsiemens, μS), that applied a constant 0.5 V across Ag/AgCl electrodes with a contact area of 4 mm diameter (Becker Meditec) at 32 Hz sampling rate. Electrodes were filled with TD-246 skin conductance electrode paste (Med Associates Inc.) and attached to the middle phalanges of the ring and little fingers of the subject’s left hand after hands were washed with soap and water (the ring and little fingers were used to reduce the interference between gaming and EDA recording). EDA signal was downsampled to 4 Hz and smoothed using LEDALAB (V.3.2.5) toolbox for MATLAB, and divided into phasic and tonic components using the nonnegative deconvolution method (Benedek & Kaernbach, 2010). Instead of using the basic signal that is a composite of momentary responses and longterm level, this technique allows a more accurate representation of the neurological drivers of the two components. Electrocardiogram (ECG) was recorded with a modified Lead II configuration (electrodes on low rib on the left and the right collar bone), and sampled at 512 Hz. ECG signal was analyzed using the ECGLAB toolbox for MATLAB (de Carvalho, da Rocha, de Oliveira Nascimento, Neto, & Junqueira, 2002). R-peaks were identified from the original 512 Hz series and corrected for ectopic beats. Interbeat interval (IBI) time series was obtained by interpolating with cubic splines at 4 Hz. Square root of the mean squared difference of successive IBIs (RMSSD) and HF component of spectral IBI were extracted for heart rate variability measures.


30


Abdominal respiration, electromyography (EMG), electroencephalography (EEG), and acceleration with a three-dimensional motion sensor were also recorded; however, these results have already been reported elsewhere (Kivikangas et al., 2013; Spapé et al., in press) and are not replicated here. Self-reports. The pre-experiment questionnaires included several trait questionnaires; however, we report here only Emotional Empathy results (Balanced Emotional Empathy Scale; Mehrabian, 1996), which are related to our prior predictions on the association between empathy and physiological linkage. This questionnaire is aimed at evaluating the tendency to empathize with fictional and non-fictional stories and characters. In addition, we asked our participants to report their previous experience with both HEDGEWARS and WORMS games, of which the former is a version of the more popular latter one. Later, we found that the WORMS experience was the more useful of these items, and subsequently we used it as the only index of previous experience. Post-condition questionnaires reported here include state-questionnaires related to the players’ experiences during the preceding playing session. We gathered responses to the Social Presence in Gaming Questionnaire (de Kort, Ijsselsteijn, & Poels, 2007), that is similar to the Social Presence Inventory of Biocca and Harms (2003) except that it has been adjusted specifically for gameplay context. The questionnaire includes the Behavioral Involvement (with items such as “My actions depended on the actions of my partner”), Empathy (e.g., “When I was happy, my partner was happy”), and Negative Feelings (e.g., “I felt revengeful”) subscales. Additionally, we selected the bi-directional Perceived Comprehension scale from the Social Presence Inventory (Biocca & Harms, 2003) that was missing from the Social Presence in Gaming Questionnaire. This questionnaire taps into the feeling of mutual understanding between one player and another; bi-directionality here means that each participant evaluated both how he or she felt toward the other player, and how the other player felt toward him or her. All questionnaires were presented in Finnish.

Data Analysis Calculations of linkage indices. Linkage scores for IBI, HF, RMSSD, and tonic and phasic EDA were calculated similarly as in Henning and coworkers (2001). Simply put, coherence represents the similarity in frequency of two signals. To obtain weighted coherence scores (Porges et al., 1980), the frequencies below approximately 0.04 Hz were filtered out using the detrending method specified by Tarvainen, Ranta-aho, and Karjalainen (2002). Mean square coherence was calculated between 0.05 and 1.25 Hz using 256 point Hann windows with 75% overlap, weighted by both participants’ series power spectral values at the specified frequencies. To study the slow-term variation during the game sessions, time series for both participants were calculated using 10-second mean values of IBI and tonic and phasic EDA. The resulting series were differenced by subtracting the preceding value from each data point. Cross-correlation refers to whether the two signals are changing to the same (or opposite) direction from moment to moment. Cross-correlation coefficients


31

Järvelä et al.

at zero lag were obtained using standard procedures. The cross-correlations were further processed by transforming the scores using Fisher’s z-transformation in order to stabilize the variance of correlation coefficients (not to be confused with standardized z scores). This resulted in transformed z scores. Statistical analyses. All physiological linkage indices were averaged over the whole playing period (i.e., one condition), that varied from 3 to 20 minutes, with a mean of 10 minutes (SD = 3.9 minutes). As the analyses were carried out on dyadic physiological data, all data were aggregated to one unit per row format before using in analyses. The aggregations used were mean (representing average level of the dyad) and difference (representing how much the levels of individuals differed within the dyad). Because z scores are normally distributed, we were able to use regular t tests to investigate whether the scores differ from zero, revealing the presence of physiological linkage during the experiment. Because of the hierarchical structure of the data, we used a separate data set where the four play sessions were aggregated into one value per dyad for the t tests (similar results were obtained from non-aggregated values). The changes between conditions and the associations between questionnaire and physiological data were analyzed with the Linear Mixed Models procedure in SPSS 21, with maximum likelihood estimation for comparing the fit of the models, and the dyad defined as the unit of analysis. Although compound symmetry covariance structure for the residuals has been recommended for dyadic analysis (Kenny, Kashy, & Cook, 2006), we chose to use first-order autoregressive covariance structure instead because it produced the best fit for our physiological data, as measured with Hurvich and Tsai’s criterion (AICC). This may have occurred because our repeated design consisting of four successive experimental periods deviated from the basic dyadic design without repeated measures variables. The analyses for associations between physiological linkage and other variables (i.e., playing condition and self-reports) were conducted with two different models. The effect of condition was investigated with a basic model with previous experience of the game, condition (a four-level categorical variable), Condition × Previous Experience interaction, and period length as predictive variables. Period length was included, as according to previous findings it may have an effect on some linkage scores (Henning et al., 2001). Gender and period order were also examined, but they were omitted from the final analyses because they did not have a consistent effect on the variables. The association between self-reports and physiological linkage was studied with a covariate model, that included condition and period length as predictive variables in addition to the post-period state questionnaire scores. Previous experience was dropped from the covariate models because they did not show significant effects in the basic model analyses. No established effect size estimates exist for specific contrasts in Linear Mixed Model analyses. However, our selected linkage measures, coherence and (cross-)correlation, are analogous to squared correlation coefficients, that are conventionally used


32


Table 1. Physiological Linkage Effect of Different Physiological z Indices (N = 41). Source variable Interbeat interval RMSSD High frequency band Tonic electrodermal activity Phasic electrodermal activity

M

SD

t

p

1.06 1.16 0.27 0.87 1.33

0.63 0.73 0.54 0.79 1.17

10.70 10.22 3.28 7.07 7.33

.5).


33

Järvelä et al. Table 2. Associations Between Physiological Linkage Measures and Self-Reports. Source variable Perceived comprehension HF cross-correlation SPGQ empathy RMSSD cross-correlationa SPGQ behavioral involvement IBI coherence HF cross-correlation RMSSD cross-correlationa SPGQ negative feelings IBI coherencea Balanced Emotional Empathy Scale IBI cross-correlation

Estimate

SE

F

df

p

0.091

0.031

8.589

1,114.00

.004

−0.065

0.023

8.076

1,136.72

.005

−0.016 −0.068 0.033

0.005 0.003 0.016

9.025 5.119 4.391

1,139.69 1,132.12 1,118.87

.003 .025 .038

−0.006

0.003

5.611

1,115.73

.020

0.026

0.009

8.641

1,64.05

.005

Note. Only significant associations are reported. aReported result is calculated with difference between participants, not mean.

Analyses investigating the association between physiological linkage scores and self-reports are reported in Table 2. It was found that HF cross-correlation was positively associated with mean perceived comprehension (p = .004) and negatively with behavioral involvement (p = .025). Furthermore, IBI coherence was negatively associated with mean behavioral involvement (p = .003), difference of participants’ negative feelings (p = .020), and period length, F(1, 145.66) = 18.992, p < .001, and RMSSD cross-correlation was negatively associated with difference of participants’ empathy (p = .005) and positively with difference of participants’ behavioral involvement (p = .038). Finally, IBI cross-correlation was positively associated with mean balanced emotional empathy (p = .005). Parameter estimates indicated that an increase of one in the self-reports would result in a change ranging from .09 (in IBI cross-correlation by Perceived Comprehension association) to .006 (in IBI coherence by Negative Feelings), the upper limit being a sizable effect in an effect that is defined only between zero and one. Tonic psychophysiological and self-report results are presented elsewhere (Kivikangas et al., 2013; Spapé et al., in press). Event-based analysis will be conducted and the results reported at a later time.

Discussion We investigated how different levels of structural competition were reflected in physiological linkage within dyads and also whether the presence of AI agent had an effect. The amount of physiological linkage and its connection to various self-reports in different conditions was analyzed in order to determine their relation.


34


Our analyses show that, on average, the physiological signals were strongly positively correlated within dyads. This means that during the experimental setup, the changes in heart rate, heart rate variability, phasic skin conductance responses, and tonic skin conductance level often occurred together within the dyads. At minimum, this illustrates how gaming as an activity is such that the physiological signals within dyads are strongly connected to each other. In other words, for example, when one player’s heart rate increased, so did the other one’s. It is important to note that unlike in the other study with linkage in gaming context (Chanel et al., 2012), our stimulus was a turn-based game with different activity cycles between players and did not have simultaneous movements as a potential source for physiological linkage. So, mere simultaneous similar activity is not sufficient to explain the results. This implies that the social situation of playing a digital game with a partner is reflected in the physiological signals of those involved and their linkage to each other. We found that IBI weighted coherence, representing the synchronous changes in heart rate in the frequency domain, was associated with the condition. Furthermore, an investigation of the predictive variables showed that the period length had a strong effect on this result. As a reason to believe that coherence is affected by the period length (longer periods should have lower coherence) exists, and considering that one of the conditions (Condition 4, competitive mode without the computer players) was on average somewhat shorter than others, including this variable in the model seems justified. As a result, we found more linkage (indexed by IBI weighted coherence) during Condition 4 than in other conditions, after taking the period length into account. The first three conditions did not differ from each other, so contrary to our predictions, the differences in competition modes did not increase or decrease the physiological linkage within dyads. It would seem that the difference between Condition 4 and other conditions—that is, the presence of AI agents in the game—is the cause for this result. This can interpreted as a sign that the AI agents in this type of game disturb the players from focusing solely on each other’s actions which in turn diminishes physiological linkage between the two. In a sense, the amount of socially relevant events occurring during AI turns is smaller than during player turns and consequently without AI presence, the linkage between players is higher. The larger proportion of less socially relevant information in other conditions could also prevent the gaming situation from being socially and experientially competitive even though it would structurally be so (Conditions 2 and 3). When no AI agents are present in the game, the players automatically focus more on each other which might turn the game more competitive also experientially that can also be reflected in the increased linkage scores in the no AI condition. It is possible, however, that the lack of AI agents increased the pace of the game in Condition 4, as unlike in other conditions the players had no pauses during the AI turns. This lead to more arousing game experience and higher HR scores (see Kivikangas et al., 2013), that could in theory be reflected in linkage scores as well. However, increased arousal levels do not automatically mean higher linkage between participants; it would require very high arousal levels to peak the reactions so that high levels alone would cause increased linkage. So, it is unlikely that the increased linkage


35

Järvelä et al.

scores would be due to higher pace of the game alone, but more likely due to AI presence and the degree of meaningfulness in AI actions. In any case, this exemplifies how linkage indices are sensitive to changes in the social context of the game. On the other hand, neither index of skin conductance was associated with the condition, indicating that although linkage existed in all conditions between players, the competitiveness or cooperativeness, or the presence of AI agents in the game, was not related to it. It was interesting that although phasic and tonic skin conductance showed overall positive correlations, they were not associated with conditions or any of the self-reports. If the linkage is associated with social presence as we have suggested, this implies that it might not be relevant for arousal-based signals. This replicates the finding in another work (Chanel et al., 2012), where linkage was associated with heart indices (such as IBI, HF), but not skin conductance (tonic electrodermal activity). When investigating the associations between self-reports and linkage scores, we found that they were present with cardiac indices, but not with skin conductance indices. Cardiac measures (IBI coherence, and HF, RMSSD, and IBI cross-correlation) were connected to various self-report scales. Several associations between cardiac measures and self-reported empathy related scales (perceived comprehension, the difference in negative emotions, the difference in empathy scores and balanced emotional empathy) suggest that physiological linkage is indeed associated with personality traits and situational social dynamics. In this case, the sense of empathy and understanding between players was connected to increased physiological linkage, that is, when the participants felt the same way, rated similarly in empathy or felt that they could comprehend each other, this was also shown as increased physiological linkage. This finding supports the idea of embodied processing where mirror neuron system based mimicry of others constitutes to emotional contagion and empathy. That is, in order to understand the other person, we simulate similar responses in our own bodies and consequently exhibit those same reactions and this is shown as synchronization of physiological signals. The other self-report that was connected to physiological linkage was Behavioral Involvement. Curiously, unlike other indices it was negatively associated with heart rate variability measures traditionally linked to, for example, attentive processes and parasympathetic activation (Ravaja, 2004). In other words, when the players felt that their actions influenced the other player’s action the cross-correlation of their HF indices was lower. This can be interpreted as a shift of focus from the game as a stimulus to the social consequences of game events that lowers the scores of attention related indices and parasympathetic activation through relaxation. That is, the when the social consequences of the gameplay become more relevant, the participants focus less intensively on the gameplay activity itself. At first glance, this seems to be contradictory to the result where RMSSD cross-correlation was higher when their experience of mutual dependency of their actions differed from each other. In a turn-based artillery game, the sense of behavioral involvement is not symmetrical, that is, the players are affecting each other’s actions asynchronously at different times. In addition, in a typical game the most dramatic and consequential actions are highly asymmetrical, that is, one player is affecting the outcome of the game considerably more than the other (e.g., killing several hedgehogs in the opposing team with one skillful or lucky shot). The game state


36


might also change considerably between player turns due to the actions of AI agents, resulting in decreased behavioral involvement, but possibly increasing physiological linkage. Yet despite the asymmetrical behavior the understanding of the consequences of the actions are understood by both players and the most effective player actions in the game are often also the most relevant socially. This dynamic is shown in increased physiological linkage when behavioral involvement is rated differently by the players.

Limitations As physiological linkage is a relatively little researched phenomenon and we are still lacking an established singular method of calculating the indices, the current work is based on our best knowledge on the subject. Therefore, it is possible that other methods of calculating the indices or using other physiological signals would ultimately produce more valid results. The interpretation of the results is also challenging due to the lack of solid background theory and the previous work has been conducted on quite diverse field of topics. In the current experiment, only various gaming modes were compared. Although typically a comparison to baseline would be made, this was not possible with linkage scores, as the purpose of baseline, calming the participants, would have driven their physiological activities to similar levels if they followed the instructions. This would show as synchronous physiological activities, but it would not constitute linkage in the sense we use the term in this article. Similar to the speed of two cars on the opposite sides of the city would show a perfect synchrony when they were stopped in lights: not due to any connection between them, but because they were not actually doing anything. Therefore an additional experimental period, during which the participants would have been playing the game on separate computers and without a connection between the two, would have been needed to provide a suitable comparison. For further studies on physiological linkage, whether gaming related or otherwise, we suggest that a control condition for comparison is arranged. The relatively high number of analyses conducted casts some doubt on the validity of weakest associations (cf. Kruschke, 2010). However, the effect sizes for the t tests suggest a particularly large effect, and although the differences between conditions and the associations with self-reports were much smaller, the largest of them cover almost the quarter of the variance within the values reported in this experiment. Yet although our findings are in line with the existing literature on the topic, naturally we cannot completely rule out the possibility that some of the findings are purely a result of chance. For comparison purposes, similar studies should be conducted with other games as well. However, identical condition structure with the same cooperation and competition modes is impossible to arrange in most games, as typically the game content changes considerably between different modes (e.g., single and multiplayer modes in first person shooters). The similarity of results in our study and Chanel et al.’s (2012) is encouraging for future directions. As such, these results apply for co-located games only; generalizing them to mediated forms of gaming, MMOGs, for example, would also require further testing.


37

Järvelä et al.

Conclusion In summary, the results show that physiological linkage exists between players when playing a multiplayer game, and also that it is a measure that is sensitive to changes in social interaction and context. Our results support the notion that the physiological signals of players are linked together and affecting each other leading to emotional contagion. This emphasizes the methods usability in assessing the social dynamics of various dyadic or small group gaming situations. In our results, the various competition and cooperation modes did not have an effect on physiological linkage and on that regard the modes can be considered equivalent in social relevance. However, this is probably true only in light-hearted games such as HEDGEWARS. Our results further suggest that the presence of AI agents in a multiplayer game can disturb the sense of social presence between players and engagement in the game, and that the actions of AI agent are not considered as socially relevant as the other player’s actions. On a more general level, the association of empathy and social presence related self-reports and physiological linkage in the results give support to the theory of embodied cognition, mimicry, and emotional contagion. These results contribute not only to games research in the form of basic understanding on the effects of different game modes and presence of AI agents, but also the development of physiological linkage as measure for social interaction and presence. Declaration of Conflicting Interests The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The research was funded by Play Society (Tekes, Next Media program) and Graduate School in User-Centered Information Technology (UCIT).

References Barsade, S. G. (2002). The ripple effect: Emotional contagion and its influence on group behavior. Administrative Science Quarterly, 47, 644-675. doi:10.2307/3094912 Barsalou, L. (2008). Grounded cognition. Annual Review of Psychology, 59, 617-645. doi:10.1146/annurev.psych.59.103006.093639 Benedek, M., & Kaernbach, C. (2010). Decomposition of skin conductance data by means of nonnegative deconvolution. Psychophysiology, 47, 647-658. doi:10.1111/j.14698986.2009.00972.x Biocca, F., & Harms, C. (2003). Guide to the Networked Minds Social Presence Inventory v. 1.2: Measures of co-presence, social presence, subjective symmetry, and intersubjective symmetry. East Lansing: Michigan State University. Brownley, K., Hurwitz, B., & Schneiderman, N. (2000). Cardiovascular psychophysiology. In J. T. Cacioppo, L. G. Tassinary, & G. G. Berntson (Eds.), Handbook of psychophysiology (2nd ed., pp. 224-263). New York, NY: Cambridge University Press.


38


Chanel, G., Kivikangas, J. M., & Ravaja, N. (2012). Physiological compliance for social gaming analysis: Cooperative versus competitive play. Interacting With Computers, 24, 306-316. doi:10.1016/j.intcom.2012.04.012 Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155-159. doi:10.1037/00332909.112.1.155 Dawson, M. E., Schell, A. M., & Filion, D. L. (2000). The electrodermal system. In J. T. Cacioppo, L. G. Tassinary, & G. G. Berntson (Eds.), Handbook of psychophysiology (2nd ed., pp. 200-223). New York, NY: Cambridge University Press. de Carvalho, J., da Rocha, A., de Oliveira Nascimento, F., Neto, J., & Junqueira, L. J. (2002). Development of a Matlab software for analysis of heart rate variability. In B. Yuan & X. Tang (Eds.), 6th International Conference on Signal Processing (pp. 1488-1491). Beijing, China: Institute of Electrical and Electronics Engineers Inc., People’s Posts & Telecommunications Publishing House. de Kort, Y. A. W., & IJsselsteijn, W. A. (2008). People, places, and play: Player experience in a socio-spatial context. Computers in Entertainment, 6(2). Article 18. doi:10.1145/ 1371216.1371221 de Kort, Y. A. W., Ijsselsteijn, W. A., & Poels, K. (2007). Digital games as social presence technology: Development of the Social Presence in Gaming Questionnaire (SPGQ). In L. Moreno (Ed.), Presence 2007 conference (pp. 195-203). Starlab Barcelona S.L., Barcelona, Spain. Retrieved from http://www.temple.edu/ispr/prev_conferences/proceedings/2007/de%20Kort,%20IJsselsteijn,%20and%20Poels.pdf Dimberg, U., & Öhman, A. (1996). Behold the wrath: Psychophysiological responses to facial stimuli. Motivation and Emotion, 20, 149-182. doi:10.1007/BF02253869 Ekman, I., Chanel, G., Järvelä, S., Kivikangas, J. M., Salminen, M., & Ravaja, N. (2012). Social interaction in games: Measuring physiological linkage and social presence. Simulation & Gaming, 43, 321-338. doi:10.1177/1046878111422121 Hatfield, E., Cacioppo, J. T., & Rapson, R. L. (1993). Emotional contagion. Current Directions in Psychological Science, 2, 96-100. Henning, R., Boucsein, W., & Gil, M. C. (2001). Social-physiological compliance as a determinant of team performance. International Journal of Psychophysiology, 40, 221-232. doi:10.1016/S0167-8760(00)00190-2 Kallio, K. P., Mäyrä, F., & Kaipainen, K. (2011). At least nine ways to play: Approaching game mentalities. Games and Culture, 6, 327-353. doi:10.1177/1555412010391089 Kätsyri, J., Hari, R., Ravaja, N., & Nummenmaa, L. (2012). The opponent matters: Elevated fMRI reward responses to winning against a human versus a computer opponent during interactive video game playing. Cerebral Cortex. Advance online publication. doi:10.1093/ cercor/bhs259 Kenny, D. A., Kashy, D., & Cook, W. (2006). Dyadic data analysis: Methodology in the social sciences. New York, NY: Guilford Press. Kivikangas, J. M., Chanel, G., Cowley, B., Ekman, I., Salminen, M., Järvelä, S., & Ravaja, N. (2011). A review of the use of psychophysiological methods in game research. Journal of Gaming and Virtual Worlds, 3, 181-199. doi:10.1386/jgvw.3.3.181_1 Kivikangas, J. M., Kätsyri, J., Järvelä, S., & Ravaja, N. (2013). Gender differences in emotional responses to competitive and cooperative game play. Manuscript submitted for publication. Kruschke, J. K. (2010). What to believe: Bayesian methods for data analysis. Trends in Cognitive Sciences, 14, 293-300. doi:10.1016/j.tics.2010.05.001 Lee, K. (2004). Presence, explicated. Communication Theory, 14, 27-50. doi:10.1111/j.14682885.2004.tb00302.x


39

Järvelä et al.

Levenson, R. W., & Gottman, J. M. (1983). Marital interaction: Physiological linkage and affective exchange. Journal of Personality and Social Psychology, 45, 587-597. doi:10.1037/0022-3514.45.3.587 Levenson, R. W., & Ruef, A. M. (1992). Empathy: A physiological substrate. Journal of Personality and Social Psychology, 63, 234-246. doi:10.1037/0022-3514.63.2.234 Levenson, R. W., & Ruef, A. M. (1997). Physiological aspects of emotional knowledge and rapport. In W. Ickes (Ed.), Empathic accuracy (pp. 44-72). New York, NY: Guilford Press. Lim, S., & Reeves, B. (2010). Computer agents versus avatars: Responses to interactive game characters controlled by a computer or other player. International Journal of HumanComputer Studies, 68, 57-68. doi:10.1016/j.ijhcs.2009.09.008 Mahon, B. Z., & Caramazza, A. (2008). A critical look at the embodied cognition hypothesis and a new proposal for grounding conceptual content. Journal of Physiology-Paris, 102, 59-70. doi:10.1016/j.jphysparis.2008.03.004 Mandryk, R. L. (2008). Physiological measures for game evaluation. In K. Isbister & N. Schaffer (Eds.), Game usability: Advancing the player experience (pp. 207-235). Burlington, MA: Morgan Kaufmann Publishers. Mandryk, R. L., Inkpen, K. M., & Calvert, T. W. (2006). Using psychophysiological techniques to measure user experience with entertainment technology. Behavior & Information Technology, 25, 141-158. doi:10.1080/01449290500331156 Mauss, I. B., & Robinson, M. D. (2009). Measures of emotion: A review. Cognition & Emotion, 23, 209-237. doi:10.1080/02699930802204677 McGee, K., Merritt, T., & Ong, C. (2011). What we have here is a failure of companionship. In C. Paris, N. Colineau, V. Farrell, G. Farrell, & W. Huang (Eds.), Proceedings of the 23rd Australian Computer-Human Interaction Conference on—OzCHI ’11 (pp. 198-201). New York, NY: ACM Press. doi:10.1145/2071536.2071568 Mehrabian, A. (1996). Manual for the Balanced Emotional Empathy Scale (BEES). Available from Albert Mehrabian, 1130 Alta Mesa Road, Monterey, CA 93940. Nass, C., & Moon, Y. (2000). Machines and mindlessness: Social responses to computers. Journal of Social Issues, 56, 81-103. doi:10.1111/0022-4537.00153 Niedenthal, P. M., Barsalou, L. W., Winkielman, P., Krauth-Gruber, S., & Ric, F. (2005). Embodiment in attitudes, social perception, and emotion. Personality and Social Psychology Review, 9, 184-211. doi:10.1207/s15327957pspr0903_1 Niedenthal, P. M., Brauer, M., Halberstadt, J. B., & Innes-Ker, Å. H. (2001). When did her smile drop? Facial mimicry and the influences of emotional state on the detection of change in emotional expression. Cognition & Emotion, 15, 853-864. doi:10.1080/02699930143000194 Paulhus, D. L., & Vazire, S. (2008). The self-report method. In R. Robins, R. Fraley, & R. Krueger (Eds.), Handbook of research methods in personality psychology (pp. 224-239). New York, NY: Guilford Press. Porges, S. W., Bohrer, R. E., Cheung, M. N., Drasgow, F., McCabe, P. M., & Keren, G. (1980). New time-series statistic for detecting rhythmic co-occurrence in the frequency domain: The weighted coherence and its application to psychophysiological research. Psychological Bulletin, 88, 580-587. doi:10.1037/0033-2909.88.3.580 Ravaja, N. (2004). Contributions of psychophysiology to media research: Review and recommendations. Media Psychology, 6, 193-235. doi:10.1207/s1532785xmep0602_4 Ravaja, N., Saari, T., Turpeinen, M., Laarni, J., Salminen, M., & Kivikangas, M. (2006). Spatial presence and emotions during video game playing: Does it matter with whom you play? Presence: Teleoperators and Virtual Environments, 15, 381-392. doi:10.1162/pres.15.4.381 Rizzolatti, G., & Craighero, L. (2004). The mirror-neuron system. Annual Review of Neuroscience, 27, 169-192. doi:10.1146/annurev.neuro.27.070203.14423 Downloaded from sag.sagepub.com by guest on January 15, 2016

40


Rizzolatti, G., Fadiga, L., Gallese, V., & Fogassi, L. (1996). Premotor cortex and the recognition of motor actions. Cognitive Brain Research, 3, 131-141. doi:10.1016/0926-6410(95)00038-0 Robinson, M. D., & Clore, G. L. (2002). Belief and feeling: Evidence for an accessibility model of emotional self-report. Psychological Bulletin, 128, 934-960. doi:10.1037//00332909.128.6.934 Salen, K., & Zimmerman, E. (2004). Rules of play: Game design fundamentals. Cambridge, MA: MIT Press. Sherry, J. L., Lucas, K., Greenberg, B. S., & Lachlan, K. (2006). Video game uses and gratifications as predictors of use and game preference. In P. Vorderer & J. Bryant (Eds.), Playing video games: Motives, responses, and consequences (pp. 213-224). Mahwah, NJ: Lawrence Erlbaum. Spapé, M. M., Kivikangas, J. M., Järvelä, S., Kosunen, I., Jacucci, G., & Ravaja, N. (2013). Keeping your opponents close: Social context affects EEG and fEMG linkage in a turnbased computer game. PLoS ONE, 8(11), e78795. doi:10.1371/journal.pone.0078795 Stenros, J., Paavilainen, J., & Mäyrä, F. (2011). Social interaction in games. International Journal of Arts and Technology, 4, 342-358. doi:10.1504/IJART.2011.041486 Tarvainen, M. P., Ranta-aho, P. O., & Karjalainen, P. A. (2002). An advanced detrending method with application to HRV analysis. IEEE Transactions on Biomedical Engineering, 49, 172-175. doi:10.1109/10.979357

Author Biographies Simo Järvelä, BBA, has a background in cognitive science and human resources management. His current research interests include physiological linkage, embodied cognition, psychophysiology, media experience, games in various forms, and social interaction. Contact: [email protected]. J. Matias Kivikangas (MA in psychology, University of Helsinki, 2006) is a researcher at EMOID group and a graduate student at University of Helsinki. His dissertation is focused on the psychophysiology of the (local) social game experiences: how the different aspects of social interaction shape the personal experience of playing digital games. Contact: [email protected]. Jari Kätsyri (PhD, Cognitive Science, Helsinki University of Technology) is a postdoctoral researcher at Aalto University. He is interested in studying emotions in both human-human and human-computer interactions. His recently finished projects include studies utilizing simple video games in functional MRI brain imaging settings. Contact: [email protected]. Niklas Ravaja, PhD, is a professor at the Department of Social Research, University of Helsinki, and director of research at the Department of Information and Service Economy, Aalto School of Business. His areas of research interest and expertise include the psychophysiology of attention, emotion, and temperament and media psychology. He is an associate editor of Journal of Media Psychology. Contact: [email protected].
