WEARABLE CAMERAS IN PSYCHOLOGICAL SCIENCE 1 Accepted ...

WEARABLE CAMERAS IN PSYCHOLOGICAL SCIENCE

1

Accepted Manuscript at Social Psychological and Personality Science 2/24/2017

A snapshot of the life as lived: Wearable cameras in social and personality psychological science Nicolas A. Brown, Andrew B. Blake, & Ryne A. Sherman Florida Atlantic University

Authors’ Note This research was supported by National Science Foundation grant #1420105 to Ryne A. Sherman, Principal Investigator. Any opinions, findings, conclusions, or recommendations expressed in this article are those of the individual researchers and do not necessarily reflect the views of the National Science Foundation. All statistical analyses were conducted using R (R Core, Team, 2016). Correspondence regarding this article may be sent to Ryne A. Sherman, Department of Psychology, Florida Atlantic University, Boca Raton, FL 33431. Email: [email protected]

WEARABLE CAMERAS IN PSYCHOLOGICAL SCIENCE

2

Abstract Recently advances in mobile sensing technology provide innovative methods for understanding how individuals think, feel, and behave in vivo. Despite the utility of these new methods, as of yet researchers have not been able to see the environments people encounter in their daily lives. In this article, we introduce wearable cameras as a new tool for sampling from participants’ everyday situations. Wearable cameras are small devices that capture pictures on a fixed interval (e.g., 30 s). This article discusses the benefits and disadvantages of incorporating wearable cameras into personality and social psychological research. Drawing on our experiences using wearable cameras in research, we provide insights into ethical and legal considerations when designing studies using these devices. Lastly, we report data on situation change and perceived obtrusiveness from, to our knowledge, the first wearable camera study in personality and social psychology.

WEARABLE CAMERAS IN PSYCHOLOGICAL SCIENCE

3

“…lives are lived day by day, one day at a time, from day to day, day after day, day in and day out.” (Craik, 2000, p. 234) A fundamental goal of psychological science is the description and explanation of human behavior. For social psychology, the laboratory experiment has proven to be an indispensable method for advancing knowledge of human behavior. For personality psychology, surveys and personality assessments have been equally indispensable. Despite their utility for psychological science, laboratory experiments and personality assessments offer, at best, an incomplete and impoverished view of lives as they are lived. The purpose of this article is to introduce a new method for studying lives as they are lived: wearable cameras. To do so, we (1) provide a brief historical overview of lived day analyses in social/personality psychology, (2) describe wearable camera technology with attention to its advantages, limitations, and ethical considerations, and (3) present some empirical results from a large wearable camera study. A Brief History of Measuring Lives as Lived In perhaps the most well-known study of an individual’s daily life, Barker and Wright (1951) sent observers to meticulously document the day of a 7-year-old boy in the Midwestern United States. Each observer followed this boy every place he went, keeping notes about every situation that he experienced (e.g., getting dressed, going to school) over the course of his entire day. Despite such pioneering efforts to overcome the long-identified limitations of traditional psychological research methods (Reis, 2012), it is only recently that a large wave of research focused on studying psychological processes as they naturally occur began (Bolger, Davis, & Rafaeli, 2003). One reason for this is undoubtedly that recent advances in technology have made studying psychology outside the lab more feasible than ever.

WEARABLE CAMERAS IN PSYCHOLOGICAL SCIENCE

4

Prior to the 1980s, the most state-of-the-art method for studying life as it is lived was the daily diary method (Wilhelm, Perrez, & Pawlik, 2012). Since then, a number of methods have emerged that allow researchers to collect ecological/environmental/contextual data. These methods include experience sampling1 (ESM; Conner, Tennen, Fleeson, Barrett, 2009; Scollon, Kim-Prieto, & Diener, 2003; Shiffman et al., 2008), the Electronically Activated Recorder (EAR; Mehl, Pennebaker, Crow, Dabbs, & Price, 2001), and mobile sensing using smartphones (Dufau et al., 2011; Miller, 2012; Sandstrom, Neal, Mascolo, & Rentfrow, in press). Table 1 provides a brief description of these methods and their relative advantages and disadvantages. While all of these methods have enriched our view of individuals in their everyday environments, none provide a window for researchers to actually see the environments people experience. Being able to (objectively) see another person’s natural environment is advantageous for many reasons. While methods such as the EAR or ESM are excellent, neither captures objective information about the physical world (e.g., Is it dark? How many people are present?) very well. For example, while the EAR can be used to estimate how much time a person spends reading (i.e., by listening to pages turn; Mehl et al., 2001), this method cannot inform the researcher about what the person is reading. In our view, reading a textbook on thermodynamics seems psychologically different from reading Teen Magazine. While ESM provides some ability to combat this issue (e.g., you can ask a person what they were reading), typical ESM studies limit the amount of information a participant can provide because long surveys increase participant fatigue. Moreover, ESM requires the participant to first perceive and filter environmental information before responding, making the information they provide more subjective and 1

Other similar terms include ecological momentary assessment (EMA; Shiffman et al., 2008) and intensive repeated measures in natural settings (Moskowitz & Sadikaj, 2012).

WEARABLE CAMERAS IN PSYCHOLOGICAL SCIENCE

5

potentially biased. Wearable cameras provide raw, unfiltered, and objective information about the visual environment. No other method offers this. Wearable cameras may be useful for many research endeavors. Close relationships researchers may ask partners to simultaneously wear camera to investigate time spent together, talking to each other, facing one another, etc. Such dyadic designs could permit researchers to understand interdependent behavior (Kelley & Thibaut, 1978), processes underlying phenomena such as perceived partner responsiveness, or conflict resolution. Likewise, aggression researchers may equip participants with cameras to better understand the antecedents of aggressive behavior in real-world contexts. Cameras may be particularly adept at picking up signs of implicit discrimination (e.g., lack of eye contact) in everyday situations. In our view, wearable cameras are useful for answering an innumerable number of research questions because they extensively and conveniently record the daily situations a person encounters. As situations are a key driver of behavior, wearable cameras provide a powerful tool for measuring them. Wearable Cameras in Psychological Science Until recently, studies that permit researchers to see the environments that people experience have been quite rare. One ambitious study hired a professional video-recording team to follow a few volunteers around for a 24-hour period (Craik, 2000). While this study demonstrated the value of capturing every moment in a person’s day for understanding that person, it is not feasible to conduct such a study on a large group of participants (e.g., N=200 or more). Further, the size and obvious presence of the recording team and equipment increases the likelihood of measurement reactivity (e.g., biased data due to the act of being measured; Barta, Tennen, & Litt, 2012). Such reactivity could sacrifice the ecological validity supposedly gained by conducting research in the real-world (Reis, 2012).

WEARABLE CAMERAS IN PSYCHOLOGICAL SCIENCE

6

Table 1. Methods for collecting ecological data Method Daily diary

Description Assessment of construct of interest on a once-a-day interval

Experience Brief surveys sent to participants sampling (ESM) at set intervals via smartphone app, text messaging, etc.

Advantages •

• •

Disadvantages

Suitable for behavior that occurs relatively infrequently (e.g., about once a day)

•

Can capture psychological processes in vivo Relatively inexpensive to implement

•

•

• •

Electronically Activated Recorder (EAR) Mobile sensing

Wearable cameras

Device that records brief audio • clips throughout the day at semiregular intervals Uses participant’s smartphone to • collect data such as physical location, responses to brief surveys • • Small cameras that take images, and collect information about participant’s location

•

Can hear behavior via language expression as it occurs in real time Do not have to buy new equipment (use participant’s existing smartphone) Relatively inexpensive Access to numerous smartphone sensors (e.g., GPS, light, accelerometer) Permits researcher to see situations that participants experience

• • • •

• •

Relies on retrospective reporting which may be prone to memory biases Does not capture phenomenon of interest in real time Can cause survey fatigue through repeated assessments Unable to glean precise information about participant location Requires participant to own a mobile phone (or borrow similar device) Legal concerns in certain states regarding privacy laws Sound files must be manually transcribed Assumes participants own a mobile phone (or must borrow one from lab) Must develop an application that can work across different smartphones (e.g., iPhone, Android) Legal concerns in certain states regarding privacy laws Images may be unusable in certain conditions (e.g., at night, in poor lighting conditions)

WEARABLE CAMERAS IN PSYCHOLOGICAL SCIENCE

7

Modern Wearable Cameras In an effort to better know themselves, millions of individuals now track a variety of physiological and psychological data such as sleep quality, number of steps walked, and daily emotion (Swan, 2013). Wearable cameras that capture pictures of daily life are one of the latest devices added to this Quantified Self movement. For example, the Narrative Clip is a small, lightweight camera that automatically captures pictures according to preset intervals (see Figure 1).2 The device contains many features including an accelerometer, a wide-angle lens, GPS, and Bluetooth connectivity. Although there are many sensors available on the Narrative Clip, our focus here is on the mobile camera feature. We began using the Narrative Clip as part of a largescale investigation of person-situation transactions (Sherman, 2014). In what follows, we draw on our experiences from this investigation to describe the advantages and limitations of the Narrative Clip (simply referred to as “wearable camera” hereafter). Advantages. First, as Figure 1 demonstrates, the wearable camera is small and lightweight—participants can easily wear the device on their shirt or blouse (or on a lanyard around their neck). The camera is small enough to be unobtrusive to both the participant wearing the device, and to casual observers. This feature is particularly desirable due to possible problems introduced by reactivity (but see Limitations). Second, the wearable camera is simple to operate. The device captures images at regular intervals while it detects light, putting itself in sleep mode when no light is detected. As a result, participants can simply place the device in their pocket when they do not wish to record.

2

For a full description of the Narrative Clip, see www.getnarrative.com. Although other wearable cameras may be available, because our experience only pertains to the Narrative Clip, we emphasize its functionality here. There may be specific advantages and drawbacks associated with other cameras, but the ethical, legal, and logistical concerns discussed here apply to wearable cameras in general.

WEARABLE CAMERAS IN PSYCHOLOGICAL SCIENCE

8

Third, the wearable camera possesses adequate battery life. In our most recent study, the device was capable of holding a charge for at least a participant’s waking day (approximately 16 h). However, studies spanning multiple days are also possible as the device can be easily charged with a USB cable. Figure 1. Volunteers wearing the wearable camera on shirt collars and with necklace.

WEARABLE CAMERAS IN PSYCHOLOGICAL SCIENCE

9

Fourth, the wearable camera software and hardware are customizable. In terms of software, users may customize the intervals between capturing images from as short as every 10 seconds to as long as every two minutes.3 This is ideal for studies that may require less (or more) temporal resolution. Hardware may also be customized by exchanging wearable mounts and lanyard-type accessories. Finally, the cost for the camera is on par with other ambulatory assessment devices such as iPods or Palm Pilots (approximately USD$200). Depending on the duration of the proposed study, researchers do not need to purchase many cameras. Limitations. While new methodological and/or technological innovations present promising ways to collect ecological data, there are often some accompanying limitations. Wearable cameras are no exception. First, when participants wear the cameras, there is the possibility for reactivity. That is, participants may behave differently than they would during their normal day. This might include avoiding certain situations due to privacy concerns (e.g., going to the gym locker room).4 Furthermore, other individuals may behave differently around the participant as a result of the participating wearing the camera. Second, because participants can temporarily stop wearing the camera due to privacy concerns (e.g., in a restroom) or when it is not physically feasible to do so (e.g., swimming in a pool), there is the possibility of forgetting to continue wearing the camera (e.g., leaving it in one’s pocket). The resulting impact on data collection could vary from relatively minor (only a few situations are not recorded) to major (most of the participant’s day is missed). To counteract these potential issues, researchers could periodically send messages (via e-mail or text-messaging) to remind participants when to start wearing the device, and throughout the day(s) to ensure compliance. Nonetheless, because each 3

Altering the time between intervals impacts battery life—shorter intervals lead to decreased battery life, whereas longer intervals may increase battery life. 4 This can be mitigated by properly instructing the participant that the wearable camera can be temporarily removed.

WEARABLE CAMERAS IN PSYCHOLOGICAL SCIENCE

10

photo is date and time stamped, researchers can estimate the total number of photographs “missing” from the record. Third, the quality of captured images may vary significantly as a result of three key factors: time of day, how the camera is worn, and general movement (see Figure 2). Because the camera does not possess a flash, images captured in low-light conditions are generally of poor quality. While the images are still useful for participants to recall what they were doing, they may be unusable for certain data analyses such as object recognition (see Future Directions). Wearing the camera on a shirt or blouse may cause the camera angle to capture images of the ceiling and so forth, instead of a straightforward, egocentric perspective. In these circumstances, wearing the camera on a lanyard or relatively tight fitting necklace is preferred (see Figure 1). General movement may also affect the quality of camera images—abrupt or sudden movements can cause image blurring, although the device does have an accelerometer to help mitigate this issue. As with all technologies, there is the possibility for researcher to encounter technical difficulties with the cameras. In our experience, these problems were infrequent, but resulted in the loss of participant images, or failure of the camera to capture any images on occasion. Finally, relying on any third party software for research comes with risks associated with that party’s existence. During the revision stage of this manuscript, the company producing Narrative Clip announced it had filed for dissolution (Narrative, 2016). Although the company has now been acquired and not shut down service, it is presently unclear how long the Narrative Clips currently in existence will continue to be supported and if production of new Narrative Clips will resume. Despite this, we believe that wearable cameras (as a research tool) are unlikely to disappear from the market.

WEARABLE CAMERAS IN PSYCHOLOGICAL SCIENCE

11

Figure 2. Sample images captured with the wearable camera. Image quality exemplars Good

Poor

WEARABLE CAMERAS IN PSYCHOLOGICAL SCIENCE

12

Ethical and Legal Considerations using Wearable Cameras Researchers hoping to incorporate wearable cameras into their research program might understandably be concerned about both legal issues and securing approval from their institutional review board (IRB) or local ethics committee. Having gone through the process ourselves, we understand that significant ethical concerns may arise. Privacy concerns using wearable cameras generally fall into one of two categories—the loss of privacy for the participant, and for that of bystanders. Although IRBs have their own idiosyncrasies and might interpret federal regulations differently, it is our hope that the following information will help researchers anticipate and address these ethical concerns. Protecting participant privacy. Because participants receive and sign informed consent about the research, and are voluntarily capturing images of their daily lives and sharing them with the research team, the potential risk to participant privacy is relatively easy to maintain. Nonetheless, protocols should be put into place to ensure that participants may screen all of their images prior to sharing them with the research team. This is as simple as downloading all of the photos and allowing the participant to privately delete any photos that they do not wish to share before any member of the research team sees them. Such a procedure is similar to those used by EAR researchers (Mehl et al., 2001). Of course, this screening procedure alone does not absolve the researcher of their responsibility to safeguard participants’ identity once the remaining images are stored. To do so effectively, there are two primary strategies. First, all images should be stored on devices that are password protected, and if possible, encrypted. In the event these devices (e.g., external hard drives) are ever stolen or lost, the possibility of the images being accessed is greatly diminished. Second, access to the repository of participant images should be tightly controlled. Consent documents should describe who will

WEARABLE CAMERAS IN PSYCHOLOGICAL SCIENCE

13

have access to the images – typically only members of the research team, which may include research assistants, graduate students, and collaborators. Any individual that access the images (e.g., research assistants in the coding process) should sign a confidentiality agreement agreeing not to disclose any information regarding the content of the images. Protecting the privacy of others. Although participants wearing the device are agreeing to share parts of their public or private lives, it is inevitable that images including bystanders will be captured. Thus, the following question arises: is it legal to take a picture of another individual without their consent? Unlike wiretapping laws which govern audio recording, there are few laws that prohibit visual recording (i.e., video or photography).5 In general, visual recording is considered unlawful only if there is a reasonable expectation of privacy. For example, a person outdoors at a public park, or on a public beach, has no reasonable expectation of privacy. However, it could be deemed illegal, and is certainly unethical, to wear the camera in a private residence, such as a friend’s dorm room, the restroom, or in a locker room. In these instances, participants should take off the device until they return to a public area, or ensure that it is okay with all other parties at risk to be photographed to continue wearing the device (e.g., a friend’s dorm room). Additionally, we instruct participants that, should anyone in the situation feel uncomfortable with them wearing the camera, it is best to remove the device temporarily. Lastly, there may be some concerns that the camera could be used for surveillance purposes (i.e., spying on others). Although the camera is a small device and may go unnoticed to the casual observer, the camera is a poor surveillance device. With a battery life insufficient to sustain longer than 24 hours, poor performance under low lighting conditions, and relatively unpredictable photograph timing, there are far better choices for surveillance activities on the market. In sum, the legal and

5

Although we use the term visual recording here, we refer specifically to photographing images.

WEARABLE CAMERAS IN PSYCHOLOGICAL SCIENCE

14

ethical obstacles to wearing the device are surmountable, so long as reasonable precautions and instructions to participants are provided. An Application of Wearable Cameras in Social/Personality Psychological Science Thus far we have introduced wearable cameras as a novel method for researchers to see participants’ everyday lives and environment. In what follows we describe some results from a recently completed, large scale study of person-situation transactions using wearable cameras. Wearable cameras are particularly useful for person-situation transaction research because they allow researchers to photographically capture virtually every (public) situation a person encounters. Thus, one impetus for the study was a basic understanding of how people transact from one situation to another. Here we present results of that investigation as well as data regarding the perceived obtrusiveness of the wearable camera. Method Participants Two-hundred and ninety-eight participants (Mage=21.11, SD=6.21) recruited from the Florida Atlantic University community participated in exchange for up to USD$160.6 Participants were compensated USD$150 for completing all portions of the study (see Procedure section below). Further, a USD$10 bonus was awarded to participants that recruited two of their peer acquaintances to complete a personality survey regarding the target participant. The gender breakdown of the sample was 40.6% female, 58.7% male, and 0.7% did not indicate. The sample was comprised of 17% African-American, 6% Asian, 40% Caucasian, 27% Latino/a-Hispanic,

6

The target sample size for this study was initially 200 to detect effect sizes of approximately r=.20 with at least 80% power. Furthermore, effect sizes in the typical range for personality and social psychology tend to stabilize around N=200 (Schönbrodt & Perugini, 2013). In an effort to match the demographics of the local community, the total N was increased to 300. The project ended at N=299 when the final participant failed to complete the protocol after long delay.

WEARABLE CAMERAS IN PSYCHOLOGICAL SCIENCE

15

8% Other, and 2% did not indicate, closely matching the ethnic diversity of the broader Florida Atlantic University community. Measures7 Situation change questionnaire. To assess situation change, we developed a categorical forced-choice measure with five response options based in part on Buss’ (1987) theory of person-environment interaction (see also Rauthmann & Sherman, 2016a). These items tapped selection (“I left the situation for another situation”), manipulation (“I did something to intentionally change the situation”), evocation (“The situation change was beyond my control”), and construal (“My perception or goals in the situation changed”). If none of the four choices accurately represented why their situation changed, participants were asked to select “Other.” Obtrusiveness questionnaire. To assess the obtrusiveness of the camera, we adapted an obtrusiveness questionnaire developed for EAR research (see Mehl & Holleran, 2007). Participants rated the degree to which they felt the camera was obtrusive, both personally and to bystanders, using a 5-point Likert-type scale ranging from 1 (Not at all) to 5 (A great deal). The eight obtrusiveness questionnaire items are provided in Table 1. Participants also indicated if they chose not to record any situations (yes or no) and were provided with an open-ended textbox for further explanation. Lastly, participants were asked whether others asked them to take off the camera (yes or no). Procedure This study was comprised of three parts. In Part 1, participants visited the laboratory and were interviewed during a brief “getting-to-know-you” style interview. Afterward, participants 7

Because this is part of a large-scale project on person-situation transactions, participants completed many other measures including personality assessments, situation ratings, emotions, and goal checklists. Because this article is concerned only with the wearable camera methodology aspect of the project, we do not discuss them here.

WEARABLE CAMERAS IN PSYCHOLOGICAL SCIENCE

16

completed a number of personality inventories, and were provided detailed instructions on how to operate the wearable camera. For Part 2, participants were instructed to wear the camera on the next day—from the time they wake up until they fall asleep—in all situations in which it was feasible to do so. In Part 3 of the study, participants returned to the laboratory where their images were downloaded. Participants were allowed to privately screen their pictures and asked to delete those that they did not wish to share with the research team (e.g., inadvertently wore camera in a private location, etc.) Next, participants were provided the following instructions: “For the next part of the study we are going to have you divide the pictures into different episodes. The goal is for you to indicate each time your situation changed. This might include each time you changed locations, someone else came into the situation, or the topic of conversation changed. Use your own judgment to decide when one situation ends and another begins.” After completing this task, participants indicated the reason why each situation changed using the situation change questionnaire (see Measures) and provided ratings of the situational characteristics, momentary goals, and behavior. After rating all of their situations, participants completed the obtrusiveness questionnaire. Results Participants provided data on a total of 5,280 situations, with an average of 18.46 situations per participant (SD=11.50; Median=16). A total of 254,208 images were gathered from this study, an average of 889 images per participant (SD=405; Median=886.5; Min=18, Max=1707). The average situation contained 48.15 images (SD=69.82; Median=23; Min=0; Max=995). Participants indicated that their situations primarily changed due to selection (46.6%

WEARABLE CAMERAS IN PSYCHOLOGICAL SCIENCE

17

of total situations), followed by manipulation (23.9%), construal (13.8%), and evocation (11.0%). Only 4.7% indicated “Other” as a reason why their situation changed (see Figure 3), suggesting that the four substantive categories capture the near totality of reasons for situation change. The descriptive statistics from the obtrusiveness measure are presented in Table 1 and are visually depicted in Figure 4. Overall, participants were generally aware that they were wearing the camera, however most did not find it uncomfortable to wear. Furthermore, participants reported that bystanders were generally aware of the camera, and it was often talked about. To some degree, the camera may have influenced the behavior of others, but judging by the median (2), this was infrequent. It is important to note that the present study took place over only one day. As other studies employing similar technology, such as the EAR, have demonstrated, participants stop talking about the device after several days (Mehl & Holleran, 2007). We suspect a longer study would find a similar result for wearable cameras. Importantly, participants reported that the camera did not impede their activities, nor affect their behavior (including places they went, and things they did). Lastly, nearly all of the participants (84.5%) reported that they chose not to record certain situations (e.g., using the restroom), however only small proportion of participants stated that bystanders asked them to temporarily stop wearing the camera (11.3%). Sample reasons for not recording situations include, “Personal situations involving hygiene,” “I didn’t record them because they may have been deemed illegal or inappropriate,” and “I went on a date, and I took it off for that. I wanted to keep it personal.” However, the vast majority involved being in private places (e.g., restrooms).

WEARABLE CAMERAS IN PSYCHOLOGICAL SCIENCE

18

Table 2. Descriptive statistics for the obtrusiveness questionnaire Obtrusiveness for participant To what degree… …were you generally aware of the Narrative? …did you feel uncomfortable wearing the Narrative? …did the Narrative impede your daily activities? …did the Narrative change your actual behavior? …did the Narrative influence places you went?

M

SD

Med

Min

Max

3.03 1.86 1.42 1.34 1.42

1.30 1.31 0.82 0.74 0.87

3 1 1 1 1

1 1 1 1 1

5 5 5 4 5

Obtrusiveness for others To what degree… …were people around you aware of the Narrative? 3.14 1.30 3 1 5 …did you talk to people around you about the Narrative? 2.94 1.29 3 1 5 …did the Narrative influence the behavior of the people 1.99 1.18 2 1 5 around you? Note. N = 281. The average intercorrelation among the item is .17 (SD = .13, median = .15, min = -.03, max = .55). Figure 3. Frequency distribution of reasons for situation change.

WEARABLE CAMERAS IN PSYCHOLOGICAL SCIENCE

19

To estimate the number of pictures “missing” from the record (i.e., either not recorded or deleted by participants), we first assumed a 16 hour waking day for each participant. Thus, on average, each participant would be expected to yield 1,920 pictures if they never removed the camera or deleted any photos. As noted above, the mean number of pictures per participant was 889, indicating that, on average, 1,031 photos (out of the total possible) were not provided to us. While this may seem like a large number missing, it is important to keep in mind that the 1,920 figure is an unreasonable expectation based on a 16 hour waking day, never removing the camera, and deleting zero photos. Indeed, no participant actually reached this hypothetical total. Flipping these numbers on their head, our cameras captured visual information for nearly 7.5 hours of participants’ daily lives. Discussion When developing a new research method, it is incumbent upon the researchers to demonstrate that this new method provides insight or advantages over prior methods. That is, mere existence of a method is not enough reason to use it. In the case of wearable cameras, the results for situation change just described provide preliminary evidence that the wearable camera method can provide new insights for social scientists. Although theory about situation change has existed for nearly 30 years (Buss, 1987), only this approach to measuring situation change provided us with an accurate description what kinds and how much of situation change occur. We also believe that wearable camera methods are superior to the day reconstruction method (Kahneman et al., 2004), because the photo-stream produced by the cameras provides stronger memory cues for people attempting to recall their day than time-period prompts. Although it seems rather obvious that people would recall more when prompted by photographic streams than day reconstruction methods, one small study (n=14)

WEARABLE CAMERAS IN PSYCHOLOGICAL SCIENCE

20

investigated this empirically and found that the wearable camera method identified approximately twice as many activities as the day reconstruction method (Kelly et al., 2015). At a broader level, the preliminary evidence from (to our knowledge) the first wearable camera study in personality and social psychology, indicates that participants felt generally comfortable wearing a small, wearable camera throughout their day. The feasibility of this methodology for research is at least partially dependent on the willingness of participants to wear the device and continue wearing it throughout the study. We acknowledge that the participants in the current study are younger, are likely to be technologically savvy (so-called “digital natives”), and are generally accepting of newer technology. However, it is important to note that wearable cameras have been used previously in research, with both younger (e.g., Pauly-Takacs, Moulin, Estlin, 2011), and older participants (e.g., Hodges, Berry, & Wood, 2011). Nevertheless, because the cameras are relatively easy to use, they appear to be a novel, yet promising method for measuring lives as they are lived. In what follows, we discuss future directions for designing new wearable camera studies and possible solutions for analyzing camera data. Study design. Although the study presented here took place in multiple phases in the lab, it is important to note that studies do not need to be overly complicated. For example, in a hypothetical study, participants could complete a number of surveys before coming to the lab to pick up a wearable camera. After wearing the camera for a specified period of time (e.g., a few hours, an entire day), the participant could download the images on their own computer and complete any necessary ratings via an online survey. For studies lasting longer than a day, participants could charge the device on their own at home. Furthermore, instead of relying on retrospective reports, researchers could integrate wearable cameras with an ESM to collect in vivo assessments. These ESM reports could be matched to the date and time of the images.

WEARABLE CAMERAS IN PSYCHOLOGICAL SCIENCE

21

Lastly, wearable cameras could be coupled with the EAR or other mobile sensing devices to provide information about the raw, physical stimuli in the participant’s environment as well as any auditory stimuli or geolocation data. Indeed, we believe that the future of real-time, real-life studies will combine several of these methods. Coding data. A major advantage of using wearable cameras to study daily life is the large number of photographs that provide a visual record of the person’s encounters. These photographs can be coded and analyzed in a variety of ways. First, as described previously, participants can provide their own ratings of the photos. These ratings are aided by the memory cues the photographs provide. Second, trained or naïve coders can later view these photos themselves and code/rate them for a wide-variety of cues or characteristics. Coding schemes will undoubtedly vary depending on the specific questions of interest to the researchers. Analyzing data. Advances in computer vision have made manually analyzing each and every single image unnecessary. For example, commercial computer software such as Clarifai (https://www.clarifai.com/) uses machine vision to detect objects in images. In the study presented here, these objects—or situation cues (Rauthmann, Sherman, & Funder, 2015) —can be connected to the psychological characteristics of the situation. Consider a situation where a participant is spending time with a significant other. This situation might be described by its psychological characteristics—it is likely to positive and contains some social aspect (Rauthmann, et al. 2014)—but also by its physical properties (another person is present, a glass of wine, food, etc.). One goal in this research area is to predict situational characteristics from the physical aspects gleaned through object recognition technology such as Clarifai (see also Aghaei, Dimiccoli, & Radeva, 2016).

WEARABLE CAMERAS IN PSYCHOLOGICAL SCIENCE

22

Figure 4. Plot of obtrusiveness questionnaire means with 95% confidence intervals and medians. N = 28

WEARABLE CAMERAS IN PSYCHOLOGICAL SCIENCE

23

Furthermore, as described in this article, one of the primary goals of this research was to understand the processes of situation change. Research in computer science has already begun using convolution neural networks to automatically segment images captured by wearable cameras (Bolanos, Mestre, Talavera, Giro-i-Nieto, Radeva, 2015; Doherty et al., 2011). That is, given a set of images, a computer algorithm can segment the images into situations roughly similar to the manual process conducted by participants in our study. While no single algorithm can be expected to perfectly identify objects, or segment situations, the algorithms grow more precise over time with each additional analysis. Conclusion The laboratory experiment and the psychological questionnaire are essential and indispensable tools in the social/personality scientist’s toolbox. However, standing on the shoulders of recent technological advances, the time is now right for social/personality scientists to study people in their natural environments. Experience sampling, unobtrusive eavesdropping, and mobile sensing have made it easier than ever to understand how people act and interact in their everyday situations. To these we now add the wearable camera method, which provides researchers with a window into the natural environments people encounter in their daily lives. We hope that this article serves as a guide for other social scientists to use wearable camera methods in their own research.

WEARABLE CAMERAS IN PSYCHOLOGICAL SCIENCE

24

References Aghaei, M., Dimiccoli, M., & Radeva, P. (2016). With whom do I interact? Detecting social interactions in egocentric photo-streams. Presented at the International Conference on Pattern Recognition. December. Cancun, Mexico. Barker, R. G., & Wright, H. F. (1951). One boy’s day. A specimen record of behavior. New York: Harper & Row. Barta, W. D., Tennen, H., & Litt, M. D. (2012). Measurement reactivity in dairy research. In M. R. Mehl & T. S. Conner (Eds.), Handbook of Research Methods for Studying Daily Life (pp. 108-123). New York: Guilford Press. Bolanos, M., Mestre, R., Talavera, E., Giro-i-Nieto, X., & Radeva, P. (2015). Visual summary of egocentric photostreams by representative keyframes. Paper presented at the 2015 IEEE International Conference on Multimedia & Expo Workshops (ICMEW). DOI: http://doi.ieeecomputersociety.org/10.1109/ICMEW.2015.7169863 Bolger, N., Davis, A., & Rafaeli, E. (2003). Diary methods: Capturing life as it is lived. Annual Review of Psychology, 54, 579-616. Buss, D. M. (1987). Selection, evocation, and manipulation. Journal of Personality and Social Psychology, 53, 1214-1221. Conner, T. S., Tennen, H., Fleeson, W., Barrett, L. (2009). Experience sampling methods: A modern idiographic approach to personality research. Social Personality Psychology Compass, 3, 292-313. Craik, K. (2000). The lived day of an individual: A person-environment perspective. In. W.B., Walsh, K.H. Craik, R. H. Price (Eds). Person-Environment Psychology: New Directions and Perspectives. (pp. 233-266). Mahwah: Lawrence Erlbaum Associates.

WEARABLE CAMERAS IN PSYCHOLOGICAL SCIENCE

25

Doherty, A. R., Caprani, N., Conaire, C., Kalnikaite, V., Gurrin, C., Smeaton, A. F., & O’Connor, N. E. (2011). Passively recognising human activities through lifelogging. Computers in Human Behavior, 27, 1948-1958. Dufau, S., Duñabeitia, J. A., Moret-Tatay, C., McGonigal, A., Peeters, D., Alario, F. X., … Grainger, J. (2011). Smart phone, smart science: How the use of smartphones can revolutionize research in cognitive science. PLoS ONE, 6, 1-3. Gurrin, C., Smeaton, A. F., & Doherty, A. R. (2014). Lifelogging: Personal big data. Foundations and Trends in Information Retrieval, 8, 1-107. Hodges, S., Berry, E., & Wood, K. (2011). SenseCam: A wearable camera that stimulates and rehabilitates autobiographical memory. Memory, 19, 685-696. Kahneman, D., Krueger, A. B., Schkade, D. A., Schwarz, N., & Stone, A. A. (2004). A survey method for characterizing daily life experience: The day reconstruction method. Science, 306(5702), 1776-1780. Kelly, P., Thomas, E., Doherty, A., Harms, T., Burke, Ó., Gershuny, J., & Foster, C. (2015). Developing a method to test the validity of 24 hour time use diaries using wearable cameras: A feability pilot. PLoS ONE, 10: e0142198. doi:10.1371/journal. pone.0142198 Kelley, H. H., & Thibaut, J. W. (1978). Interpersonal relations: A theory of interdependence. New York, NY: Wiley-Interscience. Mehl, M. R., & Holleran, S. E. (2007). An empirical analysis of the obtrusiveness of and participants’ compliance with the Electronically Activated Recorder (EAR). European Journal of Psychological Assessment, 23, 248-257. Mehl, M. R., Pennebaker, J. W., Crow, M. D., Dabbs, J., & Price, J. H. (2001). The electronically activated recorder (EAR): A device for sampling naturalistic daily

WEARABLE CAMERAS IN PSYCHOLOGICAL SCIENCE

26

activities and conversations. Behavior Research Methods, Instruments, and Computers, 33, 517–523. Microsoft Corporation (2016). Microsoft Research SenseCam. Retrieved from http://research.microsoft.com/en-us/um/cambridge/projects/sensecam/ Miller, G. (2012). The smartphone psychology manifesto. Perspective on Psychological Science, 7, 221-237. Narrative (2016, October 10). Narrative ending sales and support of Narrative Clip [Blog post]. Retrieved from http://blog.getnarrative.com/2016/10/an-open-letter/ Pauly-Takacs, K., Moulin, C. J. A., & Estlin, E. J. (2011). SenseCam as a rehabilitation tool in a child with anterograde amnesia. Memory, 19, 705-712. Rauthmann, J. F. (2015). Structuring situation information: A road map of the multiple pathways to different situational taxonomies. European Psychologist, 20(3), 176-189. Rauthmann, J. F., Gallardo-Pujol, D., Guillaume, E. M., Todd, E., Nave, C. S., Sherman, R. A., Ziegler, M., Jones, A. B., & Funder, D. C. (2014). The situational eight DIAMONDS: A taxonomy of major dimensions of situation characteristics. Journal of Personality and Social Psychology, 107, 677-718. Rauthmann, J. F., Sherman, R. A., & Funder, D. C. (2015). Principles of situation research: Towards a better understanding of psychological situations. European Journal of Personality, 29, 363-381. Reis, H. T. (2012). Why researchers should think “real-world”: A conceptual rationale. In M. R. Mehl & T. S. Connor (Eds.), Handbook of Research Methods for Studying Daily Life (pp. 3-21). New York: Guilford.

WEARABLE CAMERAS IN PSYCHOLOGICAL SCIENCE

27

Sandstrom, G. M., Neal, L., Mascolo, C., & Rentfrow, P. J. (in press). Putting mood in context: Using smartphones to examine how people feel in different locations. Journal of Research in Personality. Schlotz, W. (2012). Ambulatory psychoneuroendocrinology: Assessing salivary cortisol and other hormones in daily life. In M. R. Mehl & T. S. Connor (Eds.), Handbook of Research Methods for Studying Daily Life. New York: Guilford. Schönbrodt, F. D., & Perugini, M. (2013). At what sample sizes do correlations stabilize? Journal of Research in Personality, 47, 609-612. Scollon, C. S., Kim-Prieto, C., & Diener, E. (2003). Experience sampling: Promises and pitfalls, strengths and weaknesses. Journal of Happiness Studies, 4, 5-34. Sherman, R. A. (2014). Situation change: Understanding person-environment transactions in real life. Grant proposal submitted to the National Science Foundation. Shiffman, S., Stone, A. A., Hufford, M. R. (2008). Ecological momentary assessment. Annual Review of Clinical Psychology, 4, 1-32. Wilhelm, F. H., Grossman, P., & Muller, M. I. (2012). Bridging the gap between the laboratory and the real world: Integrative ambulatory psychophysiology. In M. R. Mehl & T. S. Conner (Eds.), Handbook of Research Methods for Studying Daily Life (pp. 210-234). New York: Guilford Press. Wilhelm, P., Perrez, M., & Pawlik, K. (2012). Conducting research in daily life: A historical review. In M. R. Mehl & T. S. Conner (Eds.), Handbook of Research Methods for Studying Daily Life (pp. 62-86). New York: Guilford Press. Wolf, P. S. A. & Jacobs, W. J. (2009). GPS technology and human psychological research: A methodological proposal. Methods and Measurement in the Social Sciences, 1, 1-5.