Virtual reality as a screening tool for sports ... - Semantic Scholar

Brain Injury, December 2012; 26(13–14): 1564–1573

ORIGINAL ARTICLE

Brain Inj Downloaded from informahealthcare.com by Universite Du Quebec A Trois - Rivieres on 03/17/13 For personal use only.

Virtual reality as a screening tool for sports concussion in adolescents

PIERRE NOLIN1, ANNIE STIPANICIC1, MYLE`NE HENRY1, CHRISTIAN C. JOYAL1, & PHILIPPE ALLAIN1,2 1

Laboratoire de Recherche Interdisciplinaire en Reálite´ Virtuelle (LARI-RV), De´partement de psychologie, Universite´ du Que´bec a` Trois-Rivie`res, Que´bec, Canada, and 2Laboratoire de Psychologie ‘Processus de penseé et interventions’, Universite´ d’Angers, Angers, France (Received 9 September 2011; revised 8 May 2012; accepted 23 May 2012)

Abstract Primary objective: There is controversy surrounding the cognitive effects of sports concussion. This study aimed to verify whether the technique of virtual reality could aid in the identification of attention and inhibition deficits in adolescents. Study design: A prospective design was used to assess 25 sports-concussed and 25 non-sports-concussed adolescents enrolled in a sport and education programme. Methods and procedures: Participants were evaluated in immersive virtual reality via ClinicaVR: Classroom-CPT and in real life via the traditional VIGIL-CPT. Main outcomes and results: The neuropsychological assessment using virtual reality showed greater sensitivity to the subtle effects of sports concussion compared to the traditional test, which showed no difference between groups. The results also demonstrated that the sports concussion group reported more symptoms of cybersickness and more intense cybersickness than the control group. Conclusions: Sports concussion was associated with subtle deficits in attention and inhibition. However, further studies are needed to support these results. Keywords: Virtual reality, virtual classroom, ClinicaVR: Classroom, sports concussion, mTBI, mild traumatic brain injury, adolescents

Introduction The term sports concussion refers to a case of mild traumatic brain injury (mTBI) resulting from a sport accident. Sports concussion is defined as change in mental status resulting from trauma or rotational forces to the brain, with or without loss of consciousness [1]. Concussions represent 8.9% of all high school athletic injuries and 5.8% of all collegiate athletic injuries and they occur in sports such as hockey and football, as well as soccer, basketball, lacrosse and wrestling [2]. According to Sosin et al.(1996) approximately 300,000 sport-related

concussion occur annually in the United States [3]. Given that the direct and indirect costs of sports concussion are estimated at nearly US$17 billion [4], continued research in this area is a worthwhile endeavour. Research on mTBI, both sports-related and nonsports-related, has been gaining prominence in the literature. Empirical research conducted in the 1980s by Bart and colleagues [5, 6] has served as the foundation for subsequent studies. While certain people experience at least some of the cognitive effects of mTBI for the rest of their lives, others

Correspondence: Pierre Nolin, PhD, Laboratoire de Recherche Interdisciplinaire en Reálite´ Virtuelle, De´partement de psychologie, Universite´ du Que´bec a` Trois-Rivie`res, C.P. 500, Trois-Rivie`res, Que´bec, G9A 5H7, Canada. Tel: (819) 376-5011 ext. 3544. Fax: (819) 376-5195. E-mail: [email protected] ISSN 0269–9052 print/ISSN 1362–301X online ß 2012 Informa UK Ltd. DOI: 10.3109/02699052.2012.698359


ClinicaVR: Classroom-CPT and sports concussion recover from it; the nature and the duration of the post-acute recovery process are still a matter of debate [7]. While most people with non-sportsrelated mTBI tend to show strong recovery over the 3 months after their accident [8–14], some experience persistent symptoms beyond this period. The prevalence of post-acute symptoms is highly variable from study-to-study: Binder et al. (1997) [15] reported post-acute symptoms in 7–8% of cases, while Rimel et al. [6] reported 33%. These symptoms have been known to last several months or even years [8, 16–20]. Two meta-analyses conducted by Be´langer and Vanderploeg [7] and Broglio and Puetz (2008) [21] on studies dealing with sports concussion identified the impact of sports concussion on a number of neurocognitive functions in the initial hours, days and, in some cases, weeks post-injury. The studies pointed out deficits in four areas: (1) attention, (2) memory and learning, (3) overall cognitive function and (4) executive function. Generally, most participants appear to recover fully during the months following the concussion. Some individuals, however, experience long-term effects and/or slower recovery. While most of the existing research has focused on the acute phase post-concussion, recent studies conducted using neuroimaging techniques support the idea that certain athletes experience an impact over the long-term. One of the key considerations in studying sports concussion is the tool used to assess the impaired cognitive functions. Standard neuropsychological assessment is considered sensitive to the immediate effects of concussion in certain athletes [22]; it is recognized as being more sensitive than regular neurological or radiological testing [23]. If neuropsychological assessment is to detect the effects of sports concussion, it must be sensitive to the symptoms of mild brain injury [24–26]. Recently, Broglio et al. [27] noted that event-related potentials (ERPs) were useful in identifying areas of cognition that remained dysfunctional for an extended period of time post-concussion and that would otherwise go undetected by standard neuropsychological tests. The short- and long-term effects of sports concussion have been observed in several related studies that have been conducted using sophisticated neuroimaging techniques such as functional magnetic resonance imaging (fMRI) [28–31] and ERPs [27, 32–34]. While these technologies are essential in demonstrating the effects of sports concussion in the brain, they are difficult to use and are not readily transported into ‘everyday’ environments, such as schools with sports and education programmes (SEPs). There is still a need, therefore, to further develop tools for assessing the cognitive effects of

1565

sports concussion that are both user-friendly and sufficiently sensitive. Virtual reality By means of immersive virtual reality, users can interact in real time with a 3-dimensional, computersimulated environment that mirrors daily life [35–38]. This makes it a revolutionary tool. Users enter the immersion by wearing a head-mounted display (HMD) onto which the virtual environment is projected. The user’s head movements are sensed by a detection system integrated into the HMD and converted into computerized data. Studies using immersive neuropsychological assessments to study people with mTBI are few. The advantages offered by virtual reality, such as greater sensitivity and ecological validity, make research in this area a promising endeavour. Testing in virtual reality gauges the true functioning of subjects in environments where they are left to their own devices and allowed to behave naturally. Additionally, automatic scoring reduces the number of errors that might skew the results. By its very nature, testing in virtual reality detects subtle deficits, which are often imperceptible in traditional assessments [36, 37, 39–43]. The use of virtual reality techniques in attention and inhibition testing is therefore a promising avenue for improved identification of cognitive deficits following sports concussion. Only recently has virtual reality become a tool for studying people with traumatic brain injury (TBI). Most studies on this topic have been conducted on adults and most have focused on a limited number of cognitive domains, such as motor and balance rehabilitation [44–51], social problem-solving [52], community living skills [53] and car driving [54]. These studies have demonstrated the advantages of virtual reality on adults with moderate or severe TBI. As for studies on children with TBI, those using virtual reality as an assessment tool are relatively scarce [55, 56]. Some studies have demonstrated the utility of virtual reality in cases of moderate or severe TBI [39, 40], but not the mild type (mTBI). (The only two studies to have used virtual reality to assess people with mTBI were both conducted on adults.). Slobounov et al. (2006) [57] showed that advanced virtual reality technology, used in combination with balance and motion tracking technologies, was able to detect residual symptoms of concussion symptoms in university athletes. More recently the same research team (Slobounov et al. 2010) [58] studied the effects of concussion using fMRI in combination with a virtual reality paradigm in adult athletes. In summary, while virtual reality has met with success in detecting subtle cognitive deficits resulting

1566

P. Nolin et al.


from sports concussion, further studies are needed, both on adults and on children. To the best of the authors’ knowledge, the present study is the first to focus on adolescents. The objective of this study is to compare performance in attention and inhibition tests (both traditional and virtual) between two groups of adolescents enrolled in an SEP: those who have sustained a concussion and those who have not.

Method Participants Group 1 (sports concussion) was composed of 25 adolescents enrolled in an SEP at the Acade´mie les Estacades in Trois-Rivie`res, Que´bec, Canada. All participants suffered a concussion while playing their sport at some time in the 2 years preceding the study. The time between concussion and evaluation ranged from 1–24 months (mean ¼ 11.71 months, SD ¼ 8.80 months). The most common sports in which the participants had been injured were hockey, basketball and soccer. The sports concussion group was made up of 15 boys and 10 girls, with a mean age of 13.64 years (SD ¼ 1.11 years). All participants in the group suffered a Grade 1 concussion (Cantu Concussion Grading Guideline, 2001 [59]). By this definition, they experienced no loss of consciousness (LOC) due to the injury and any post-traumatic amnesia lasted under 30 minutes. Concussions were of the simple type, which is defined by the Summary and Agreement Statement of the 2nd International Conference on Concussion in Sport (McCrory et al. 2005 [60]) as a concussion for which associated symptoms resolved within 7–10 days without complications. Inclusion criteria also included commonly accepted clinical symptoms of mTBI taken from SCAT2 (http://www.cces.ca/files/ pdfs/SCAT2[1].pdf) such as headache, ‘pressure in head’, neck pain, nausea or vomiting, dizziness, difficulty concentrating, irritability, etc. [14]. Group 2 (control) consisted of 25 adolescents who were also enrolled in the SEP but who did not sustain a sports concussion in the 2 years preceding the study. This group was also made up of 15 boys and 10 girls, this time with a mean age of 13.76 years (SD ¼ 1.13 years). All participants followed the normal SEP curriculum and had no history of neurological, psychological or learning disabilities. The two groups were equivalent with respect to age [t(48) ¼ 0.38, p < 0.05] and gender. All participants in the study were francophone.

Measures Sense of presence and cybersickness questionnaires. In order to verify the quality of the immersion during the virtual neuropsychological test, we administered two questionnaires to each participant. The questionnaires covered two important factors in virtualreality studies: sense of presence and cybersickness. Sense of presence refers to the propensity to respond to virtually generated sensory data as if they were real [61]. Cybersickness denotes symptoms that may be felt during or after the participant’s experience in virtual reality, such as nausea or eye strain. The first questionnaire, the Presence Questionnaire, was developed by Witmer and Singer (1998) [62] and revised by the UQO Cyberpsychology Laboratory, 2004 [63]. It consists of 18 questions answered on a Likert-type scale ranging from 0 (‘not at all’) to 7 (‘completely’). For the purposes of this study, a customized version of the questionnaire was used with eight items expressly adapted for this adolescent study population. The psychometric validity of the questionnaire is supported by the Cronbach’s alpha of 0.78 for this study sample. The second questionnaire, the Post-Exposure Symptom Checklist, was developed by Kennedy et al. (1993) [64] and later translated into French and revised by the UQO Cyberpsychology Laboratory (2002) [65]. It consists of 16 items answered on a Likert-type scale ranging from 0 (‘none’) to 3 (‘severe’). The variables measured by this questionnaire are the number of cybersickness and the mean intensity of cybersickness as experienced by participants. The psychometric validity of the questionnaire is supported by a Cronbach’s alpha of 0.87 for this study sample. The complete list of symptoms can be found in Table I. Neuropsychological test: Traditional To test attention and inhibition functions in a traditional setting, the VIGIL-CPT (Vigil Continuous Performance Test, Cegalis & Bowlin, 1991 [66]) was used. In this computerized test, letters appear one at a time in the centre of a screen, changing at an interval that is kept constant throughout the test. The participant is required to click the mouse each time the letter K appears after being immediately preceded by the letter A. The 6minute test presents a total of 300 stimuli, 60 of which require a response. In clinic and in research, the VIGIL-CPT is a recognized measure of sustained attention, vigilance, impulsivity and reaction time [67]. The three variables measured in the traditional test were (1) the number of omissions (i.e. failure to respond to the letter K when immediately preceded by the letter A), (2) the number

ClinicaVR: Classroom-CPT and sports concussion

1567

Table I. Frequency of cybersickness symptoms experienced during the ClinicaVR: Classroom test and results from group comparisons. Frequency

Symptoms General discomfort Fatigue Headache


Eye strain Difficulty focusing Increased salivation Sweating Nausea Difficulty concentrating ‘Fullness of the head’ Blurred vision Dizziness with eyes open Dizziness with eyes closed Vertigo ‘Stomach awareness’ Burping

Gr Gr Gr Gr Gr Gr Gr Gr Gr Gr Gr Gr Gr Gr Gr Gr Gr Gr Gr Gr Gr Gr Gr Gr Gr Gr Gr Gr Gr Gr Gr Gr

1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2

0 None

1 Slight

2 Moderate

3 Severe

12 21 4 10 14 18 4 8 12 17 14 22 24 22 20 25 11 21 10 16 15 20 18 24 17 22 22 24 23 22 24 25

10 3 13 14 6 7 10 12 8 8 9 3 1 3 4 0 11 4 8 9 5 5 5 1 4 3 3 1 2 3 1 0

3 1 6 1 3 0 8 4 2 0 2 0 0 0 1 0 3 0 7 0 3 0 1 0 2 0 0 0 0 0 0 0

0 0 2 0 2 0 3 1 3 0 0 0 0 0 0 0 0 0 0 0 2 0 1 0 2 0 0 0 0 0 0 0

Chi square 7.22* 8.18* 5.58 3.85 5.86 6.78* 1.09 5.56 9.39** 8.44* 5.71 5.52 4.78 1.09 0.22 1.02

* p < 0.05; ** p < 0.01.

of commissions (i.e. responding to the letter K when not preceded by the letter A or responding to another letter) and (3) the mean reaction time in milliseconds. For each of the three variables, performance can be tracked over time by splitting the 6-minute test into three blocks of 2 minutes each. A comparison of the scores in the three blocks serves as a measure of the effects of fatigue as the test progresses. Neuropsychological test: Virtual The immersive test of attention and inhibition was first developed by Rizzo et al. [39]. It was revised by the Digital MediaWorks team (http://www.dmw.ca/) under the name ClinicaVR: Classroom-CPT. The test is identical to the traditional VIGIL-CPT except for the environment in which it is administered: instead of being presented on a computer screen,

the stimuli appear on a whiteboard situated in a virtual classroom. The virtual classroom features objects and people commonly found in real classrooms, such as the blackboard, desks, a teacher and students. Participants were immersed in the virtual environment by wearing an Emagin Z800 HMD with the ability to monitor the wearer’s head movements. Participants were able to look 360 around themselves as well as up and down in the virtual environment. Typical classroom sounds were played to the participant through headphones integrated into the HMD. Throughout the duration of the virtual test, the wearer experienced auditory and visual distractions typical of a real classroom, such as a knock at the door, a bell announcing the end of class, children laughing outside and a visit from the principal. The four variables measured in the virtual test were (1) the number of omissions, (2) the number of commissions, (3) the mean reaction time

1568

P. Nolin et al.

in milliseconds and (4) the number of left–right head movements (i.e. those made along the horizontal axis). As in the traditional VIGIL-CPT, it was possible to obtain performance scores for the 6-minute test in three blocks of 2 minutes, thereby tracking performance over the course of the test.

There was no correlation between cybersickness score and virtual test scores (omissions r ¼ 0.14, p > 0.05; commissions r ¼ 0.15, p > 0.05; reaction time r ¼ 0.22, p > 0.05; left–right head movements, r ¼ 0.08, p > 0.05). The cybersickness intensity scores showed that all participants, on average, experienced ‘mild’ cybersickness. Table I shows the frequency of reported cybersickness by group. The sports concussion group reported more symptoms of cybersickness [F(1,48) ¼ 12.73, p < 0.001] than the control group (see Table II). The intensity of the cybersickness was higher for the sports concussion group [F(1,48) ¼ 23.50, p < 0.001] than for the control group (see Table II). The score from the cybersickness questionnaire was then co-varied in comparative analysis of the groups for the virtual test scores.


Procedure Students participated individually in testing sessions during regular class hours. The order of the traditional and virtual tests was counterbalanced across participants to prevent skewing of the results due to practice or fatigue effects. The data were collected over a period of 5 weeks. The study procedure was approved by the Human Research Ethics Committee of the University of Que´bec at Trois-Rivie`res.

Group comparisons: Traditional vs immersive neuropsychological test

Results Sense of presence and cybersickness questionnaires

This section presents the results of group comparisons performed on the traditional and virtual neuropsychological test scores. First, results are given for ANOVAs performed on the three variables measured in the traditional VIGIL-CPT: omissions, commissions and reaction time. Secondly, results are given for the ANCOVAs performed on the four variables measured in the virtual version of the CPT: omissions, commissions, reaction time and left–right head movements; scores from the cybersickness questionnaire were co-varied in these analyses. Table II shows the means and standard deviations for all variables as well as the results from the statistical analysis. Table II shows that there was no significant difference between the two groups for any of the three variables in the traditional VIGIL-CPT.

This section presents the results for sense of presence and cybersickness experienced by participants in the virtual test. On average, individuals in the sports concussion group experienced roughly the same sense of presence as those in the control group (see Table II); the ANOVA showed no difference between the groups for this variable [F(1,48) ¼ 1.61, p > 0.05]. All participants felt ‘moderately’ present during the virtual test. Since there was no correlation between sense of presence score and the virtual test scores (omissions r ¼ 0.03, p > 0.05; commissions r ¼ 0.04, p > 0.05; reaction time r ¼ 0.06, p > 0.05; left–right head movements, r ¼ 0.06, p > 0.05), this study was able to compare performance between the two groups for the immersive test.

Table II. Scores for traditional VIGIL-CPT and ClinicaVR: Classroom-CPT and results from group comparisons. Group 1 Sports concussion (n ¼ 251)

Sense of presence Cybersickness (number of symptoms) Cybersickness (intensity) Traditional VIGIL-CPT Omissions Traditional VIGIL-CPT Commissions Traditional VIGIL-CPT Reaction time ClinicaVR: Classroom Omissions ClinicaVR: Classroom Commissions ClinicaVR: Classroom Reaction Time ClinicaVR: Classroom Left–right head movements

Group 2 Control (n ¼ 25)

M

SD

M

SD

3.44 6.24 1.57 4.79 4.54 324.49 3.36 5.96 377.23 72.28

0.94 3.63 0.44 7.52 3.59 58.38 3.26 6.47 42.54 69.98

3.42 3.32 1.23 3.12 6.40 320.33 1.96 3.88 380.08 35.04

0.73 1.89 0.14 3.26 4.53 39.00 2.17 2.65 46.67 21.29

*p < 0.05; **p < 0.01; ***p < 0.001; t ¼ statistical tendency between 0.06–0.09.

F Co-variable (Cybersickness) — — — — — 0.02 4.10* 3.64 t 0.53

F

Partial Eta squared

0.01 12.73*** 13.79*** 1.04 2.52 0.09 2.23 5.34* 1.22 6.64**

0.00 0.21 0.22 0.02 0.05 0.00 0.05 0.10 0.03 0.12


1569

2.5

2.05

2

1.89 1.57

1.5

Group 1

1.38

Group 2 1

0.71 0.5

0 Block 1

Block 2

Block 3

Figure 1. Number of commission errors in ClinicaVR: Classroom-CPT by time block.

35 Number of left-right head movements


Number of commission errors

2.21

30

30.24

25

23.43 20

Group 1

18.56 15 10

14.16

Group 2

11.08 9.8

5 0 Block 1

Block 2

Block 3

Figure 2. Number of left–right head movements in ClinicaVR: Classroom-CPT by time block.

However, significant differences did exist between the groups for two of the four variables in the virtual version of the CPT: number of commission errors and number of left–right head movements. (These differences were observed after the effect of cybersickness was removed.) Since significant differences were observed for commission errors and head movements in the virtual test, supplementary analyses were conducted on the respective groups’ test scores for these variables over the three 2-minute blocks. (Recall that scores from the VIGIL-CPT, in both the traditional and the immersive versions, can be examined by dividing the 6-minute test into three blocks of 2 minutes each, thus obtaining a measure of performance over time.) Figure 1 shows the number of commission errors by group for the three time blocks. Since there were not enough participants for a multivariate analysis, an ANOVA was used to compare performance

between the groups in each time block. A Bonferroni correction was applied with a significance threshold of 0.017 to avoid Type I errors. The analyses showed no significant difference between the two groups; the significance threshold of 0.04 was reached only in Block 3. In summary, the following trend emerged from the statistical analysis: the control group tended to make fewer commission errors over time, whereas the sports concussion group made the same number of errors all throughout the test. Figure 2 shows the number of left-right head movements for the three time blocks by group, using the same statistical approach as above. The analyses showed a significant difference between the groups for left–right head movements in Block 1 [F(1,48) ¼ 4.27, p ¼ 0.01] and in Block 3 [F(1,48) ¼ 8.84, p ¼ 0.019] and a statistical trend was observed in Block 2 [F(1,48) ¼ , p ¼ 0.06]. It can be concluded that the sports concussion participants

1570

P. Nolin et al.

made more head movements than control participants throughout the duration of the test.


Discussion The goal of the present study was to determine if an attention and inhibition test administered in virtual reality (ClinicaVR: Classroom-CPT) was sensitive enough to detect the subtle effects of sports concussion in a group of adolescents enrolled in an SEP. The virtual environment represents a revolutionary assessment tool for neuropsychological research, yet it remains under-exploited, especially in studies on children with mTBI. These results support the hypothesis that ClinicaVR: Classroom-CPT is sensitive enough to detect subtle effects in the sports concussion group when compared to the control group. The virtual test specifically revealed a difference between the groups in the number of commission errors and the number of left–right head movements; both these variables may be interpreted as deficits of inhibition. The data are consistent with previous studies that have demonstrated a link between sports concussion and cognitive deficits of this type [26, 68–70]. Given that subtle cognitive deficits typically improve within 3–6 months of the accident [7, 71, 72], it is somewhat surprising that they were detected in these sports concussion participants. Few studies have dealt with the long-term effects of sports concussion. Further research into this topic is needed and the sensitivity of the tool will be a key consideration in this. A handful of studies using fMRI [30] and event-related potentials [27, 34] point to the existence of long-term cognitive effects of sport concussion in adults. Sports-concussion group experienced more cybersickness than the control group, both in the number of cybersickness and in the intensity thereof. Specifically, the sports concussion group reported the following five symptoms more frequently: general discomfort, fatigue, increased salivation, difficulty concentrating and ‘fullness of the head’. Generally speaking, the nature of these subjectively reported symptoms is consistent with the kind of cognitive deficits detected by the virtual test. Attention and inhibition problems among adolescents were found to be greater in the virtual CPT than in the traditional version; in fact, the traditional version showed no difference between the groups for these deficits. The immersive assessment approach thus appears more sensitive to the effects of sports concussion than the traditional approach. It is possible that the superior sensitivity of ClinicaVR: Classroom-CPT may be due to the ‘ecological’ nature

of the tool; this means that it closely represents reallife experiences, thus placing greater demands on the participant’s capacities of attention and inhibition. The question of whether the ecological representativeness of the tool accounts for these results is one that must be addressed by future research. Such research should illustrate, first, the relationship between the two assessment approaches (traditional and virtual) and, secondly, the relationships between these approaches and existing ecological measures such as the Behavior Rating Inventory of Executive Function [73] and observed behaviours in real-life classrooms. Perhaps it is not the case that the virtual CPT is more ecological than the traditional version, but that it is simply more complex given that it requires participants to process more information. At the very least, it can be said that ClinicaVR: Classroom-CPT seems to respond well to sensitivity criteria for detecting the subtle effects of sports concussion. The results contribute to a body of existing literature that has demonstrated the utility of the virtual classroom in studying children and adolescents with ADHD [74–79]. Importantly, the virtual classroom also has the benefit of being enjoyable to participants [78, 80]. The use of ClinicaVR: Classroom-CPT need not be limited to the population in our study: it would be equally advantageous in future clinical neuropsychological research on other populations. The results generated in the present study could serve as the basis for a subsequent prospective study. The study would be designed in such a way that the sensitivity of ClinicaVR: Classroom-CPT in detecting the subtle effects of sports concussion would be verified by assessing, at the beginning of the school year, basic functions in all students enrolled in the SEP, and by subsequently performing short- and long-term reassessments on those students who sustain concussions in the future. This proposed study design is consistent with recommendations made in related studies [23] and could aid in preventing faulty attribution, i.e. the phenomenon of injured students being viewed as having other diagnosis such as hyperactivity or conduct disorders [81].

Conclusion Neuropsychological assessment, used as a tool for detecting cognitive deficits resulting from sports concussion, is recommdended by experts in the field as part of a sports concussion protocol [82–84]. Assessments based on computerized tests are recommended over traditional ones (e.g. paper and pencil tests) because they present a number of



advantages [23, 85]). One such advantage, easy reproducibility, makes it possible to conduct multiple studies at the same time or in different locations; another is the fact that no specialized training is required to operate the equipment. ClinicaVR: Classroom-CPT meets all these criteria and is also much more affordable and user-friendly than neuroradiological technologies. For all these reasons, ClinicaVR: Classroom-CPT is a compelling choice for the study of sports concussion.

Acknowledgements The authors would like to thank Albert Rizzo, from University of Southern California - Institute for creative technologies, and Roman Mitura and Dean Klimchuk, from Digital MediaWorks, for allowing them to use the Virtual Classroom and ClinicaVR: Classroom. They would also like to thank the following people for their valuable contributions to this research project: the staff of Acade´mie les Estacades de Trois-Rivieres, particularly Rosemarie Boucher, Luce Mongrain and Michel Boutin; Dr David Fecteau of the Centre Hospitalier Re´gional de Trois-Rivie`res; Fernand Bouchard of the St. Maurice Physiotherapy Clinic; and all research assistants from our laboratory (Julie Baillargeon, Caroline Cliche, Rosalie Coderre Se´vigny, Jessica Couture, Anne Drouin-Germain, Andreé-Anne Durocher, Isabelle Fournier, Nicolas Leblond, Vale´rie Noel, Roxan Noel, Pacifieé Rodrigue, Mikako Tsuchigahata, Emanuelle Valliomeé). Decleration of Interest: The authors report no conflicts of interest. This research was made possible thanks to the Fond de De´veloppement Acade´mique du Re´seau de l’Universite´ du Que´bec (FODAR) and Fonds Institutionnel de Recherche (FIR) de l’Universite´ du Que´bec a` Trois-Rivie`res (UQTR).

References 1. American Academy of Neurology Practice Parameter. The management of concussion in sports (summary statement). Report of the Quality Standards Subcommittee. Neurology 1997;48:581–585. 2. Gessel LM, Fields SK, Collins CL, Dick RW, Comstock RD. Concussions among United States High School and Collegiate athletes. Journal of Athletic Training Med Sports Systems 2007;42:495–503. 3. Sosin DM, Sniezek JE, Thurman DJ. Incidence of mild and moderate brain injury in the United States, 1991. Brain Injury 1996;10:47–54. 4. Thurman DJ, Alverson C, Dunn KA, Guerrero J, Sniezek JE. Traumatic brain injury in the United States: A public health perspective. Journal of Head Trauma Rehabilitation 1999;14: 602–615.

1571

5. Barth JM, Stephen N, Giordani B, Rimel RN, Jane JA, Boll TJ. Neuropsychological sequelae of minor head injury. Neurosurgery 1983;13:529–533. 6. Rimel RW, Giordani B, Bruno Barth JT, Boll TJ, Jane JA, Disability caused by minor head injury. Neurosurgery 1981;9:221–228. 7. Belanger HG, Vanderploeg RD. The neuropsychological impact of sports-related concussion: A meta-analysis. Journal of the International Neuropsychological Society 2005;11: 345–357. 8. Dikmen S, McLean A, Temkin N. Neuropsychological and psychosocial consequences of minor head injury. Journal of Neurology, Neurosurgery & Psychiatry 1986;49:1227–1232. 9. Dikmen SS, Machamer JE, Winn HR, Temkin NR. Neuropsychological outcome at 1-year post head injury. Neuropsychology 1995;9:80–90. 10. Dikmen SS, Ross BL, Machamer JE, Temkin NR. One year psychosocial outcome in head injury. Journal of the International Neuropsychological Society 1995;1:67–77. 11. Gentilini M. Neuropsychological evaluation of mild head injury. Journal of Neurology, Neurosurgery & Psychiatry 1985;48:137–140. 12. Gronwall DM, Sampson H. The psychological effects of concussion. Oxford, England: Auckland U Press; 1974. 13. Levin HS, Gary Jr HE, High Jr WM, Mattis S, Ruff RM, Eisenberg HM, Marshall LF, Tabaddor K. Minor head injury and the postconcussional syndrome: Methodological issues in outcome studies. New York: Oxford University Press; 1987. 14. Levin HS, Grafman J, Eisenberg HM. Neurobehavioral recovery from head injury. New York, NY: Oxford University Press; 1987. 15. Binder LM, Rohling ML, Larrabee GJ. A review of mild head trauma. Part I: Meta-analytic review of neuropsychological studies. Journal of Clinical and Experimental Neuropsychology 1997;19:421–431. 16. Alves W, Macciocchi SN, Barth JT. Postconcussive symptoms after uncomplicated mild head injury. The Journal of Head Trauma Rehabilitation 1993;3:48–59. 17. Hartlage LC, Durant-Wilson D, Patch PC. Persistent neurobehavioral problems following mild traumatic brain injury. Archives of Clinical Neuropsychology 2001;16: 561–570. 18. Powell TJ, Collin C, Sutton K. A follow-up study of patients hospitalized after minor head injury. Disability and Rehabilitation: An International, Multidisciplinary Journal 1996;18:231–237. 19. Alexander GE. Factors related to patient report of cognitive difficulties following head injury. Dissertation Abstracts International 1992;52(12-B, Pt1):6651. 20. Deb S, Lyons I, Koutzoukis C. Neurobehavioural symptoms one year after a head injury. British Journal of Psychiatry 1999;174:360–365. 21. Broglio SP, Puetz TW. The effect of sport concussion on neurocognitive function, self-report symptoms and postural control: A meta-analysis. Sports Medicine, 2008;38:53–67. 22. Iverson GL. Evidence-based neuropsychological assessment in sport-related concussion. In: FM Webbe, editor. The Handbook of Sport Neuropsychology (pp. 131–153) Springer Publishing Co.; 2011. 23. Schatz P. Computerized neuropsychological assessment in sport. In: FM Webbe, editor. The Handbook of Sport Neuropsychology (pp. 173–186). Springer Publishing Co.; 2011. 24. Gronwall D, Wrightson P. Delayed recovery of intellectual function after minor head injury. The Lancet 1974;14:605–609.


1572

P. Nolin et al.

25. Middleton J. Annotation: Thinking about head injury in children. Journal of Child Psychology and Psychiatry 1989;30:663–670. 26. Nolin P, Mathieu F. L’importance de la sensibilite´ des mesures neuropsychologiques dans l’identification des de´ficits de l’attention chez des enfants ayant subi un traumatisme cranioce´re´bral le´ger. Revue de Neuropsychologie 2001;11: 23–38. 27. Broglio SP, Moore RD, Hillman CH. A history of sportrelated concussion on event-related brain potential correlates of cognition. International Journal of Psychophysiology 2011;82:16–23. 28. McAllister TW, Saykin AJ, Flashman LA, Sparling MB, Johnson SC, Guerin SJ, Mamourian AC, Weaver JB, Yanofsky N. Brain activation during working memory 1 month after mild traumatic brain injury: A functional MRI study. Neurology 1999;53:1300–1308. 29. McAllister TW, Sparling MB, Flashman LA, Saykin AJ. Neuroimaging findings in mild traumatic brain injury. Journal of Clinical and Experimental Neuropsychology 2001;23:775–791. 30. Chen CJ, Toh SC, Fauzy WM. The theoretical framework for designing desktop virtual reality-based learning environments. Journal of Interactive Learning Research 2004;15: 147–176. 31. Ptito A, Chen J-K, Johnston KM. Contributions of functional magnetic resonance imaging (fMRI) to sport concussion evaluation. NeuroRehabilitation 2007;22:217–227. 32. Dupuis F, Johnston KM, Lavoie M, Lepore F, Lassonde M. Concussions in athletes produce brain dysfunction as revealed by event-related potentials. NeuroReport: For Rapid Communication of Neuroscience Research 2000;11: 4087–4092. 33. Leblanc J, De Guise E, Gosselin N, Feyz M. Comparison of functional outcome following acute care in young, middleaged and elderly patients with traumatic brain injury. Brain Injury 2006;20:779–790. 34. De Beaumont L. Long-term effects of sports concussion. Psychologie - recherche et intervention. Montreál: Universite´ de Montreál; 2010. 35. Pratt DR, Zyda M, Kelleher K. Virtual reality: In the mind of the beholder. IEEE Computer Society Press 1995;28:17–19. 36. Tarr MJ, Warren WH. Virtual reality in behavioral neurosciences and beyond. Nature Neuroscience 2002;5:1089–092. 37. Trepagnier CG. Virtual environments for the investigation and rehabilitation of cognitive and perceptual impairments. NeuroRehabilitation 1999;12:63–72. 38. Zhang J, Sun Y, Wang X, Wu G. How do learners utilize the course package and learning support services in distance learning: A survey on the learning processes. International Journal on E-Learning 2003;2:17–23. 39. Rizzo AA, Buckwalter JG, Bowerly T, van der Zaag C, Humphrey L, Neumann U, Chua C, Kyriakakis C, van Rooyen A, Sisemore D. The virtual classroom: A virtual reality environment for the assessment and rehabilitation of attention deficits. CyberPsychology & Behavior 2000;3: 483–499. 40. Rizzo AA, Schultheis M, Kerns KA, Mateer C. Analysis of assets for virtual reality applications in neuropsychology. Neuropsychological Rehabilitation 2004;14:207–239. 41. Theriault M, De Beaumont L, Tremblay S, Lassonde M, Jolicoeur P. Cumulative effects of concussion in athletes revealed by electrophysiological abnormalities on visual working memory. Journal of Clinical and Experimental Neuropsychology 2011;33:30–41. 42. Nolin P, Martin C, Bouchard S. Assessment of inhibition deficits with the virtual classroom in children with traumatic

43.

44.

45.

46.

47.

48.

49.

50.

51.

52.

53.

54.

55.

56.

57.

58.

59.

brain injury: A pilot-study. Annual Review of CyberTherapy and Telemedicine 2009;7:240–242. Martin C, Nolin P. La reálite´ virtuelle comme nouvelle approche e´valuative en neuropsychologie: L’exemple de la classe virtuelle avec des enfants ayant subi un traumatisme cranio-ce´re´bral. A.N.A.E. Approche Neuropsychologique des Apprentissages chez l’Enfant 2009;21:28–32. Rose FD, Brooks BM, Rizzo AA. Virtual reality in brain damage rehabilitation: Review. CyberPsychology & Behavior 2005;8:241–262. Larson EB, Ramaiya M, Zollman FS, Pacini S, Hsu N, Patton JL, Dvorkin AY. Tolerance of a virtual reality intervention for attention remediation in persons with severe TBI. Brain Injury 2011;25:274–281. Levin MF. Can virtual reality offer enriched environments for rehabilitation? Expert Review of Neurotherapeutics 2011;11: 153–155. Mumford N, Wilson PH. Virtual reality in acquired brain injury upper limb rehabilitation: Evidence-based evaluation of clinical research. Brain Injury 2009;23:179–191. Mumford N, Duckworth J, Thomas PR, Shum D, Williams G, Wilson PH. Upper limb virtual rehabilitation for traumatic brain injury: Initial evaluation of the elements system. Brain Injury 2010;24:780–791. Sveistrup H, McComas J, Thornton M, Marshall S, Finestone H, McCormick A, Babulic K, Mayhew A. Experimental studies of virtual reality-delivered compared to conventional exercise programs for rehabilitation. CyberPsychology & Behavior 2003;6:245–249. Thornton KE, Carmody DP. Electroencephalogram biofeedback for reading disability and traumatic brain injury. Child and Adolescent Psychiatric Clinics of North America 2005;14:137–162. Thornton M, Marshall S, McComas J, Finestone H, McCormick A, Sveistrup H. Benefits of activity and virtual reality based balance exercise programmes for adults with traumatic brain injury: Perceptions of participants and their caregivers. Brain Injury 2005;989–1000. Hanten G, Cook L, Orsten K, Chapman SB, Li X, Wilde EA, Schnelle KP, Levin HS. Effects of traumatic brain injury on a virtual reality social problem solving task and relations to cortical thickness in adolescence. Neuropsychologia 2011;49:486–497. Yip BCB, Man DWK. Virtual reality (VR)-based community living skills training for people with acquired brain injury: A pilot study. Brain Injury 2009;23:1017–1026. Wald JL, Liu L, Reil S. Concurrent validity of a virtual reality driving assessment for persons with brain injury. CyberPsychology & Behavior 2000;3:643–654. Penn PR, Rose FD, Johnson DA. Virtual enriched environments in paediatric neuropsychological rehabilitation following traumatic brain injury: Feasibility, benefits and challenges. Developmental Neurorehabilitation 2009;12: 32–43. Yen Hwee-Ling WJT. Rehabilitation for traumatic brain injury in children and adolescents. Annals Academy of Medicine 2007;36:62–66. Slobounov S, Slobounov E, Newell K. Application of virtual reality graphics in assessment of concussion. CyberPsychology & Behavior 2006;9:188–191. Slobounov SM, Zhang KD, Pennell D, Ray W, Johnson P, Sebatianelli W. Functional abnormalities in normally appearing athletes following mild traumatic brain injury: A functional MRI study. Experimental Brain Research 2010;202:341–354. Cantu RC. Posttraumatic retrograde and anterograde amnesia: pathophysiology and implications in grading and safe return to play. Journal of Athletic Training 2001;36:244–248.


ClinicaVR: Classroom-CPT and sports concussion 60. McCrory P, Johnston K, Meevwisse W, Aubry M, Cantu R, Dvorak J, et al. Summary and agreement statement of the 2nd international conference on concussion in sport, Prague 2004. Clinical Journal of Sport Medicine 2005;15:48–55. 61. Sanchez-Vives MV, Slater M. From presence to consciousness through virtual reality. Nature Reviews Neuroscience 2005;6:332–339. 62. Witmer BG, Singer MJ. Measuring presence in virtual environments: A presence questionnaire. Presence 1998;7:225–240. 63. UQO Cyberpsychology Laboratory. Traduction du Presence Questionnaire de Witmer & Singer. Gatineau :Universite´ du Que´bec en Outaouais; 2004. 64. Kenedy RS, Lane NE, Berbaum KS, Lilienthal MG. Simulator sickness questionnaire : An enhaced method for quantifying simulator sickness. International Journal of Aviation Psychology 1993;3:203–220. 65. UQO Cyberpsychology Laboratory. Traduction du PostExposure Symptom Checklist. de Kenedy, et al. Gatineau: Universite´ du Que´bec en Outaouais; 2002. 66. Cegalis J, Bowlin J. Vigil : software for the assessment of attention. Nashua, NH : Forthought; 1991. 67. Egeland J, Kovalik-Gran I. Measuring several aspects of attention in one test: The factor structure of Conners’s Continuous performance test. Journal of Attention Disorders 2010;13:339–346. 68. Fazio VC, Lovell MR, Pardini JE, Collins MW. The relation between post concussion symptoms and neurocognitive performance in concussed athletes. NeuroRehabilitation 2007;22:207–216. 69. McAllister WR, McAllister DE. Recovery of conditioned fear by a single postextinction shock: Effect of similarity of shock contexts and of time following extinction. Learning & Behavior 2006;34:44–49. 70. Belanger HG, Vanderploeg RD, Curtiss G, Warden DL. Recent neuroimaging techniques in mild traumatic brain injury. The Journal of Neuropsychiatry and Clinical Neurosciences 2007;19:5–20. 71. Schretlen DJ, Shapiro AM. A quantitative review of the effects of traumatic brain injury on cognitive functioning. International Review of Psychiatry 2003;15:341–349. 72. Belanger HG, Curtiss G, Demery JA, Lebowitz BK, Vanderploeg RD. Factors moderating neuropsychological outcomes following mild traumatic brain injury: A metaanalysis. Journal of the International Neuropsychological Society 2005;11:215–227. 73. Gioia GA, Isquith PK. Ecological assessment of executive function in traumatic brain injury. Developmental Neuropsychology 2004;25:135–158. 74. Adams R, Finn P, Moes E, Flannery K, Rizzo AS. Distractibility in attention/deficit/hyperactivity disorder (ADHD): The virtual reality classroom. Child Neuropsychology 2009;15:120–135.

1573

75. Parsons TD, Bowerly T, Buckwalter JG, Rizzo AA. A controlled clinical comparison of attention performance in children with ADHA in a virtual reality classroom compared to standard neuropsychological methods. Child Neuropsychology 2007;13:363–381. 76. Moreau G, Guay MC, Achim A, Rizzo A, Lageix P. The virtual classroom: An ecological version of the Continuous Performance Test–A pilot study. Annual Review of CyberTherapy and Telemedicine 2006;4:59–66. 77. Parsons TD, Bowerly T, Buckwalter JG, Rizzo AA. A controlled clinical comparison of attention performance in children with ADHD in a virtual reality classroom compared to standard neuropsychological methods. Child Neuropsychology 2007;13:363–381. 78. Pollak Y, Shomaly HB, Weiss PL, Rizzo AA, Gross-Tsur V. Methylphenidate effect in children with ADHD can be measured by an ecologically valid continuous performance test embedded in virtual reality. CNS Spectrums 2010;15: 125–130. 79. Rizzo AA, Bowerly T, Buckwalter JG, Klimchuk D, Mitura R, Parsons TD. A virtual reality scenario for all seasons: The virtual classroom. CNS Spectrums 2006;11: 35–44. 80. Pollak Y, Weiss PL, Rizzo AA, Weizer M, Shriki L, Shalev RS, Gross-Tsur V. The utility of a continuous performance test embedded in virtual reality in measuring ADHD-related deficits. Journal of Developmental and Behavioral Pediatrics 2009;30:2–6. 81. Mittenberg W, Henry G, Heilbronner R, Enders C, Stanczak S. Comparison of the Lees-Haley Fake Bad Scale, Henry-Heilbronner Index, and Restructured Clinical Scale 1 in identifying noncredible symptom reporting. The Clinical Neuropsychologist Neuropsychology, Development and Cognition 2007;22:919–929. 82. Aubry M, Cantu R, Dvorak J, Graf-Baumann T, Johnston KM, Kelly J, Lovell M, McCrory P, Meeuwisse WH, Schamasch P. Summary and agreement statement of the 1st International Symposium on Concussion in Sport, Vienna, 2001. Clinical Journal of Sport Medicine: Statement 2002;12:6–11. 83. McCrory PM. Preparticipation assessment for head injury. Clinical Journal of Sport Medicine 2004;14:139–144. 84. De Brito SA, Hodgins S, McCrory EJP, Mechelli A, Wilke M, Jones AP, Viding E. Structural neuroimaging and the antisocial brain: Main findings and methodological challenges. Criminal Justice and Behavior 2009;36:1173–1186. 85. Segalowitz SJ, Mahaney P, Santesso DL, MacGregor L, Dywan J, Willer B. Retest reliability in adolescents of a computerized neuropsychological battery used to assess recovery from concussion. NeuroRehabilitation 2007;22: 243–251.