AJSLP
Predicting Story Goodness Performance From Cognitive Measures Following Traumatic Brain Injury Karen Lê,a Carl Coelho,a Jennifer Mozeiko,a Frank Krueger,b and Jordan Grafmanc
Purpose: This study examined the prediction of performance on measures of the Story Goodness Index (SGI; Lê, Coelho, Mozeiko, & Grafman, 2011) from executive function (EF) and memory measures following traumatic brain injury (TBI). It was hypothesized that EF and memory measures would significantly predict SGI outcomes. Method: One hundred sixty-seven individuals with TBI participated in the study. Story retellings were analyzed using the SGI protocol. Three cognitive measures—Delis-Kaplan Executive Function System (D-KEFS; Delis, Kaplan, & Kramer, 2001) Sorting Test, Wechsler Memory Scale—Third Edition (WMS–III; Wechsler, 1997) Working Memory Primary Index (WMI), and WMS–III Immediate Memory Primary Index (IMI)— were entered into a multiple linear regression model for each discourse measure. Two sets of regression analyses were performed, the first with the Sorting Test as the first predictor and the second with it as the last.
S
ituated at the nexus of cognition and communication, discourse abilities are common and persistent sequelae of traumatic brain injury (TBI). Discourse refers to connected utterances that holistically have meaning (Cherney, Coelho, & Shadden, 1998). It is the purposeful operation of complex and contextualized language. Relative to discourse, individuals with TBI consistently demonstrate difficulty with global measures of content and organiza-
a
University of Connecticut, Storrs George Mason University, Fairfax, VA c Kessler Foundation Research Center, West Orange, NJ Correspondence to Jordan Grafman:
[email protected] Editor: Swathi Kiran Associate Editor: Jacqueline Hinckley Received August 23, 2011 Revision received December 17, 2011 Accepted January 17, 2012 DOI: 10.1044/1058-0360(2012/11-0114) b
Results: The first set of regression analyses identified the Sorting Test and IMI as the only significant predictors of performance on measures of the SGI. The second set identified all measures as significant predictors when evaluating each step of the regression function. Conclusion: The cognitive variables predicted performance on the SGI measures, although there were differences in the amount of explained variance. The results (a) suggest that storytelling ability draws on a number of underlying skills and (b) underscore the importance of using discrete cognitive tasks rather than broad cognitive indices to investigate the cognitive substrates of discourse.
Key Words: discourse analysis, cognition, brain injury, narratives, executive function
tion (Coelho, 2002; Jorgensen & Togher, 2009; Moran & Gillon, 2010). Global measures, sometimes referred to as macrolinguistic measures, examine discourse beyond the level of words and sentences. For example, in a study by Jorgensen and Togher (2009) involving monologic story retelling, individuals with TBI were found to produce fewer story grammar elements and fewer essential information units than non brain–injured (NBI) individuals. Although the participants with TBI performed comparably to the NBI comparison group when they were paired with communication partners who provided scaffolding, the participants with TBI told less structured and less informative stories when they were on their own. In addition to omitting critical content, individuals with TBI demonstrate difficulty formulating the central idea of the story (Chapman et al., 2004; Hay & Moran, 2005). Story grammar measures, which assess the framework used to structure semantic content within stories (e.g., episodes), have consistently revealed a reduced ability of individuals with TBI to organize narrative discourse (Coelho, 2002; Coelho, Youse, Lê, & Feinn, 2003). A recently developed measure of story goodness, called the Story Goodness Index (SGI; Lê, Coelho, Mozeiko, &
American Journal of Speech-Language Pathology • Vol. 21 • S115–S125 • May 2012 • A American Speech-Language-Hearing Association Supplement: Select Papers From the 41st Clinical Aphasiology Conference
Downloaded From: http://ajslp.pubs.asha.org/ by University of Connecticut - Health Center, Carl Coelho on 05/01/2014
S115
Grafman, 2011), combines content and organizational measures and provides a way to understand performance on one measure in relation to the other. The SGI examines story retelling ability, specifically assessing narrative content and organization through story completeness and story grammar analyses, respectively. The story completeness measure involves a tally of the critical components of the story; the story grammar measure examines the proportion of utterances that are organized into episodes. In a validation study of 171 individuals with TBI, the SGI demonstrated sensitivity and reliability in examining narrative discourse and afforded the identification of subgroups of storytelling ability (Figure 1; Lê, Coelho, Mozeiko, Krueger, & Grafman, 2011). Storytellers fell into one of four quadrants of ability: those whose stories were (1) organized and incomplete, (2) organized and complete, (3) disorganized and incomplete, or (4) disorganized and complete. Narrative discourse performance of the group with TBI was distributed across all quadrants, whereas the NBI comparison group clustered distinctly in Quadrant 2 (organized, complete). A logical follow-up to the validation study is to understand the processes involved in producing narrative content and narrative organization. Given the absence of aphasia in the TBI group and the infrequency of aphasia within the TBI population, the explanation is likely to rest not on linguistic factors but on cognitive factors. However, the exact identity and nature of the cognitive processes underlying discourse ability have yet to be determined. Investigations of the cognitive substrates of discourse require careful consideration
FIGURE 1. Goodness of story narratives plotted as a function of story grammar and story completeness for the participants with traumatic brain injury (TBI). Quadrants were defined by cutoff points at 1 SD (dashed line) and 2 SD (solid line) below the mean for both story grammar and story completeness measures. Using 1-SD values, the distribution of scores was 21% in Quadrant 1, 54% in Quadrant 2, 19% in Quadrant 3, and 6% in Quadrant 4. 2-SD cutoff points resulted in 15%, 73%, 9%, and 3% in the respective quadrants.
Note. From “Measuring Goodness of Story Narratives: Implications for Traumatic Brain Injury,” by K. Lê, C. A. Coelho, J. Mozeiko, F. Krueger, and J. Grafman, 2011, Aphasiology, 25, p. 755. Copyright 2011 by Taylor & Francis Ltd. Reprinted with permission.
of the cognitive skills necessary for specific discourse tasks. Germane to understanding narrative discourse performance on the SGI then is the examination of the cognitive prerequisites for story retelling and, in particular, the cognitive demands associated with story completeness and story grammar. A potential model for explaining the relationship between cognitive processes and narrative discourse production is the structure building framework (SBF; Gernsbacher, 1990). The SBF is a cognitive model of discourse processing that involves the construction of mental representations through three fundamental processes: (a) laying a foundation, (b) mapping relevant information onto the foundation, and (c) shifting to build a new substructure when incoming information does not cohere with the substructure that is currently activated. Central to the SBF is the assumption that the cognitive processes and mechanisms deployed for discourse are domain general rather than language specific. The mental representations are formed via two general mechanisms: (a) enhancement of relevant information and (b) suppression of irrelevant information. Incoming stimuli activate memory nodes that give rise to inchoative mental structures. For example, initial words, sentences, or pictures (i.e., in a picture story) lay the foundation for representing sentences, paragraphs, and episodes (Gernsbacher, 1995). If the information coheres with the current mental structure, subsequent incoming information is mapped onto that foundation. When new stimuli do not correspond to the existing structure, currently activated memory nodes are suppressed, and a new set of memory nodes is activated to establish another framework to bind the novel stimuli. The SBF proposes that most mental representations are arborizations of multiple substructures. Although the SBF was originally developed to explain the processes and mechanisms of discourse comprehension, the same processes and mechanisms are purported to support discourse production (Gernsbacher, Tallent, & Bolliger, 1999). Therefore, the SBF may provide insights into the cognitive bases of narrative discourse production deficits as measured by the SGI following TBI. The organization of narrative discourse through story grammar components may be thought of as laying a foundation. The foundation then is the episode structure. The mapping of relevant information may be interpreted as the process by which a thematically unified narrative emerges. Stimuli that cohere with the episode structure are mapped onto it; stimuli that do not cohere will form the basis for a new episode. The ability to shift to build new episode substructures implicates executive attentional skills, which are often impaired in individuals with TBI. Control mechanisms of enhancement and suppression are central to effective structure building, suggesting key roles for aspects of executive functions (EFs), such as the involvement of working memory (WM) in temporarily storing story information for mapping onto episode structures. Applied to story retelling, the SBF implicates other related memory systems. In particular, encoding and recalling the narrative stimuli (e.g., words, pictures) and the incorporation of real-world knowledge would be a function of immediate and declarative memory (Edgin, Pennington, & Mervis, 2010; Eichenbaum, 2004). Indeed, emerging evidence points to EFs, WM, and immediate memory as potentially fruitful avenues to elucidating
S116 American Journal of Speech-Language Pathology • Vol. 21 • S115–S125 • May 2012 Downloaded From: http://ajslp.pubs.asha.org/ by University of Connecticut - Health Center, Carl Coelho on 05/01/2014
the relationship between narrative discourse and cognition. EFs may be broadly defined as cognitive control processes that are critical for carrying out goal-directed and self-regulatory behavior (Kennedy & Coelho, 2005; Pickens, Ostwald, Murphy-Pace, & Bergstrom, 2010). WM, often considered a component process of EF, has been widely operationalized as a short-term memory system of simultaneous storage and processing (Baddeley, 1986; Cowan 2010; Oberauer, 2002). Models of the WM system differ in structure. For example, Baddeley and Hitch (1974) conceptualized WM as a multicomponent model with separate and dedicated component “work spaces” for phonological and visuospatial information. An episode buffer was later added to the model to account for the integration of verbal and spatial information as well as the abstraction of ideas from verbal and spatial information (Baddeley, 2000). Other accounts depict WM as temporarily activated aspects of long-term memory (LTM), eliminating the need for separate phonological and visuospatial modules (Cowan, 1988; Oberauer, 2002). These embeddedprocess models of WM further propose a focus of attention, comprised of some of the activated LTM information. Information held in the focus of attention is selected for greater processing. Fundamental to most WM models are central executive processes that function to control the deployed cognitive processes, such as the allocation of attention. In addition to EF and WM, immediate memory and declarative memory are also likely to have a role in story retelling. Immediate memory is a short-term representational store. Unlike WM, information in immediate memory is held for retention without further manipulation or processing (Edgin et al., 2010). Declarative memory refers to knowledge that is explicitly learned and recalled and includes episodic and semantic memory (Eichenbaum & Cohen, 2001; Tulving, 1995). Declarative memory has not been as closely related as WM to EF in the literature but is likely to play a role in narrative discourse in forming representations that connect characters and events across space and time. The narrative discourse elicitation task of story retelling, which is frequently used by researchers in discourse and is the subject of the current study, often requires an immediate recounting of the stimulus story from memory. Such a task would draw on immediate and declarative memory domains. Global aspects of narrative discourse production have demonstrated sensitivity to EF impairments. Story narratives are often elicited either through story retelling, in which the story stimuli is typically no longer present, or through story generation, in which the stimuli typically remain in front of the participant for the duration of discourse elicitation. Performance on the Wisconsin Card Sorting Task (WCST; Healon, Chelune, Talley, Kay, & Curtiss, 1993), which is a commonly used EF measure, has correlated moderately with story grammar measures (e.g., total episodes) in studies of individuals with TBI on story retelling and story generation tasks (Coelho, 2002; Coelho, Liles, & Duffy, 1995). Another story retelling study identified lowto-moderate correlations between WCST scores and measures of content (e.g., gist) and organization (e.g., story grammar) and moderate correlations between word fluency, which is another EF measure, and informational measures of
core propositions and gist (Brookshire, Chapman, Song, & Levin, 2000). However, a recent examination of story retelling and story generation in individuals with TBI found no correlations between a number of EF measures (e.g., WCST, semantic fluency, trail making tests) and content analyses of global coherence and thematic density (Marini et al., 2011). The use of different narrative discourse measures and a small sample size (N = 14) may have contributed to the nonsignificant findings. By contrast, a study of the TBI cohort (N = 167) in the SGI validation study found moderate correlations between story grammar measures based on a story retelling task and performance on a card sorting test that was analogous to the WCST (Mozeiko, Lê, Coelho, Krueger, & Grafman, 2011). Overall, the TBI literature provides support for a relationship between story retelling and story generation and EFs. Memory impairments, like discourse impairments, are a hallmark of TBI (Jennekens, de Casterlé, & Dobbels, 2010; Vakil, 2005). Yet, the relationship between memory and discourse continues to be defined. In one study examining story retelling and story generation in individuals with TBI, story grammar measures correlated moderately with associative learning performance, which is a measure of immediate declarative memory, but did not correlate significantly with WM scores (e.g., digit span; Youse & Coelho, 2005). Another study of children with TBI indicated that “n-back” task performance, tapping WM, varied with content measures, including summarization ability (Chapman et al., 2006). However, there was no correlation between immediate declarative memory measures and discourse content. Investigations of individuals with hippocampal lesions may provide further insights on the role of declarative memory in discourse performance. Amnesic individuals with dense declarative memory deficits have been found to have poorer local coherence within their stories than NBI participants, although global coherence was comparable (Kurczek & Duff, 2011). These findings suggest the possibility of different underlying cognitive processes for local and global coherence and make a case for examining declarative memory, which has been less extensively studied than other memory domains for its role in discourse production. Conflicting results, such as those presented by these studies, suggest a tentative relationship between particular memory processes and narrative discourse content and organization that warrants further scrutiny. Much of the research on discourse and cognition is correlational, examining how one specific cognitive measure relates to a particular discourse task. No study to date has examined how well potential cognitive substrates together predict discourse performance. The present study of 167 individuals with TBI investigated the contribution of measures of EF, WM, and immediate declarative memory in predicting outcomes on story goodness measures of story completeness and story grammar using story retelling for discourse elicitation. This study was an initial step toward modeling discourse performance based on cognitive measures using the SBF as a potential framework. It was hypothesized that all three cognitive factors would be significant predictors for both story completeness and story grammar in separate multiple regression analyses. Lê et al.: Predicting Story Goodness From Cognitive Measures
Downloaded From: http://ajslp.pubs.asha.org/ by University of Connecticut - Health Center, Carl Coelho on 05/01/2014
S117
Method Participants Participants included 167 of the 171 individuals with TBI from the SGI validation study (Lê, Coelho, Mozeiko, Krueger, & Grafman, 2011). Four individuals did not complete neuropsychological testing and were excluded from the current study. All participants were native English-speaking male Vietnam War veterans, 52 to 70 years of age, who were recruited through the Vietnam Head Injury Study (VHIS), which is a longitudinal investigation of the consequences of head injury that is sponsored by the U.S. Army Medical Research and Materiel Command. The last author is one of the principal investigators for the VHIS. The VHIS was in Phase III of investigation when the data for this study were collected. The major goals of the VHIS addressed questions related to the following topics: (a) prefrontal cortex function, (b) neuroplasticity and aging, (c) memory, (d) behavioral and psychosocial outcomes, and (e) neurological studies (Raymont, Salazar, Krueger, & Grafman, 2011; VHIS— Phase III, 2011). Participants had sustained severe penetrating head wounds due to projectile shrapnel fragments during the Vietnam War between 1967 and 1970. The injuries resulted in diffusely distributed multifocal lesions in the brain. Mechanisms of penetrating-head injury are different from that of closed-head injury (Grafman & Salazar, 1987). However, discourse production deficits in this population have paralleled those observed in populations with closedhead injury (Coelho, 2002; Lê, Coelho, Mozeiko, Krueger, & Grafman, 2011). At the time of concomitant neuropsychological and language testing and discourse elicitation, time post injury ranged from 34 to 37 years. In the prior validation, the participants with TBI (n = 171) were compared to NBI comparison participants on a number of demographic variables. The NBI group included 46 English-speaking male Vietnam War veterans, 55 to 76 years of age, with no history of neurologic disease or injury. In addition to age and years of education, scores on three tests were obtained for each participant: the Boston Naming Test (BNT; Kaplan, Goodglass, & Weintraub, 1983), the Token Test (DeRenzi & Vignolo, 1962), and the Armed Forces Qualification Test (AFQT; U.S. Department of Defense, 1984). The AFQT is a measure of aptitude that is
used by the military to determine enlistment eligibility and to place recruits in military occupation corresponding to their level of skill. Because scores are obtained before military service, the AFQT provides an estimate of premorbid functioning (Plag & Goffman, 1967). Independent-samples t tests were performed for the demographic variables with a Bonferroni adjustment for multiple comparisons (Table 1). Using the adjusted alpha level of .01 (.05/5), the two groups of participants were comparable on all five variables. Notably, although a few individuals with TBI performed in the lower range on the BNT, the TBI and NBI groups did not differ significantly on the language measures. This finding underscores the absence of frank aphasia and the relative preservation of basic linguistic abilities in the participants with TBI, which is consistent with the infrequency of aphasia in the TBI literature (Heilman, Safran, & Geschwind, 1971; Sarno, 1984).
Procedure Discourse elicitation task. As part of our study, we showed participants a 16-frame picture story, Old McDonald Had an Apartment House (Barrett, 1998), with no soundtrack on a computer screen. The story depicts the adaptation of a rural farmer to urban life and the conflict that arises with his fellow tenants and landlord when he begins growing vegetables indoors. Upon completion of viewing the story, each participant was instructed, “Tell me that story you just watched.” Each retelling was digitally video-recorded. Discourse analyses. Recordings were transcribed verbatim by 10 undergraduate and graduate students majoring in communication disorders who were not the same individuals who performed the neuropsychological testing on, or conducted the discourse task with, the participants. Transcripts were then segmented into T-units. A T-unit, or minimal terminal unit, is identified as a main clause and any subordinate clauses that are attached to or embedded within it (Hunt, 1965). Using the protocol detailed for the SGI, story narratives were analyzed for content and organization (Lê, Coelho, Mozeiko, & Grafman, 2011). Story completeness, the content measure, involved tallying the number of critical story components (events and characters) that were mentioned by the storyteller out of five. The five components were considered critical
TABLE 1. Demographic data for the two groups of study participants: the non brain–injured (NBI) group and the group with traumatic brain injury (TBI). NBI group
Group with TBI
Measure
M
SD
Range
M
SD
Age (in years) Education (in years) AFQT BNT Token Test
59.07 15.09 67.17 55.67 98.74
3.52 2.39 22.17 3.70 1.57
55–76 12–20 14–85 46–60 94–100
58.09 14.82 60.09 54.01 98.20
2.60 2.48 25.51 6.13 2.54
Range 52–70 8–22 1–99 25–60 87–100
df
p
215 204 189 214 210
.04 .51 .16 .08 .17
Note. AFQT = Armed Forces Qualification Test (U.S. Department of Defense, 1984); BNT = Boston Naming Test (Kaplan, Goodglass, & Weintraub, 1983); Token Test (DeRenzi & Vignolo, 1962). From “Measuring Goodness of Story Narratives: Implications for Traumatic Brain Injury,” by K. Lê, C. A. Coelho, J. Mozeiko, F. Krueger, and J. Grafman, 2011, Aphasiology, 25, p. 750. Copyright 2011 by Taylor & Francis Ltd. Reprinted with permission.
S118 American Journal of Speech-Language Pathology • Vol. 21 • S115–S125 • May 2012 Downloaded From: http://ajslp.pubs.asha.org/ by University of Connecticut - Health Center, Carl Coelho on 05/01/2014
because they were present in 80% or more of the stories that were produced by the NBI participants in the development of the SGI (Lê, Coelho, Mozeiko, & Grafman, 2011). Story grammar analysis was used to quantify organization. Story grammar guides the comprehension and expression of logical relationships (temporal and causal) between people and events (Merritt & Liles, 1987). Most story grammar models establish the episode as the essential component (e.g., Frederiksen, 1975; Mandler & Johnson, 1977; Stein & Glenn, 1979). Episodes are cognitive structures that serve to organize story content, binding story components together through temporal and causal links. The protocol for story grammar analysis is based on that of Merritt and Liles (1987). The main story grammar measure was the proportion of T-units within each episode structure, which is a more sensitive measure than a count of the total episodes (Coelho, 2002). The analysis involves two steps. The first step identifies the number of episodes in the story. A complete episode has three components: (a) an initiating event that provides the character’s motivation for action, (b) the character’s attempt, and (c) a direct consequence that marks either attainment or nonattainment of the character’s goal. An incomplete episode contains two of the three story components and is included in the total episode count as it represents a partial organizing structure. However, a single story component that is not linked to another story component (e.g., an attempt without an initiating event or direct consequence) does not comprise an episode. In the second step, the number of T-units within episodes is divided by the total number of T-units in the story to arrive at the proportion measure, which indicates how much of the narrative is organized according to story grammar rules. Examples of transcripts coded for both story completeness and story grammar may be found in the appendices of the initial development study for the SGI (Lê, Coelho, Mozeiko, & Grafman, 2011). Approximately 10% of the transcripts were randomly selected for point-to-point reliability assessment for the two discourse measures. The first author coded transcripts for story completeness and story grammar. Intrarater reliability was a comparison of the consistency of coding by a single individual. The third author provided comparisons for interrater reliability. Intrarater reliability and interrater reliability for the completeness measure were both 100%. Intrarater and interrater reliability for the story grammar measure was 90% and 84%, respectively. Cognitive measures. EFs were indexed using the Sorting Test composite scaled score from the Delis-Kaplan Executive Function System (D-KEFS; Delis, Kaplan, & Kramer, 2001). The Sorting Test is analogous to the WCST. The first condition of the Sorting Test, entitled free sorting, requires the participant to sort six cards into two groups of three in as many ways as possible (up to a maximum of eight). Cards may be sorted according to verbal–semantic features related to the words or according to visual–spatial features displayed on the cards. The second condition, sort recognition, requires the participant to identify the sorting or categorization concept as the examiner sorts the same six cards in two groups of three. Performance on this test draws on a number of constituents of EFs, including concept formation, cognitive
flexibility, and regulation of behavior (Dimitrov, Grafman, Soares, & Clark, 1999). The Working Memory Primary Index (WMI) from the Wechsler Memory Scale—Third Edition (WMS–III; Wechsler, 1997) provided the WM metric. The WM score reflects performance on verbal and nonverbal tasks. The verbal WM task, letter–number sequencing, involves ordering increasing strings of alternating letters and numbers. The participant is asked to repeat the string with the letters in alphabetical order and the numbers in ascending order. In the nonverbal WM task, spatial span, the examiner points to a series of blocks. The participant is required to replicate the pointing pattern in the same exact order as the examiner in the first condition while, in the second condition, the pattern must be produced in backwards order. The Immediate Memory Primary Index (IMI), also from the WMS–III, served as a measure of immediate declarative memory. The immediate memory score reflects the ability to remember verbal and nonverbal information across four tasks. The two verbal immediate declarative memory tasks involve retelling an orally presented stimulus story and associative word learning. The nonverbal tasks entail identification of visually presented photographs of faces and recall of information from a series of visually presented scenes. The subtests that comprise the IMI require immediate recall. Because the story retelling task also involves immediate recall of the stimulus story, the IMI was selected for its potential similarity in cognitive demands as opposed to the General (Delayed) Memory Primary Index (GMI). Additionally, the GMI on the WMS–III has not been found to be more sensitive to memory deficits than the IMI across a variety of clinical populations, including TBI, the correlation between the means on each index being .98 (Hawkins, 1998). Although delayed memory scores may be more reflective of declarative memory in general, the GMI does not appear to contribute significantly more than the IMI on the WMS–III. Data analyses. Pearson product–moment correlation coefficients were calculated for the cognitive measures and each discourse measure (i.e., story completeness, or number of critical components, and story grammar, or proportion of T-units in episode structure). Specifically, to examine the relationship between cognitive performance and discourse performance as measured by the SGI, correlation analyses were performed between scores on the Sorting Test, WMI, and IMI and scores on story completeness and story grammar. Correlational analyses were also performed to examine the relationships between the cognitive measures themselves. Two multiple linear regression analyses, one predicting story completeness and the other story grammar, were performed with Sorting Test, WMI, and IMI scores entered as predictors in that order. The order was based on the strength and consistency of findings in the literature regarding the relationship between cognition and discourse ability. For example, of the cognitive tasks examined, card sorting tests, which are purported to measure aspects of EF, have demonstrated the strongest and most consistent associations with discourse measures similar to the ones used for this study. As such, the Sorting Test was entered first. Post hoc regression analyses were performed entering the Sorting Test last to examine the possibility that placement of a broad EF measure first may obscure the effect of the memory measures. Lê et al.: Predicting Story Goodness From Cognitive Measures
Downloaded From: http://ajslp.pubs.asha.org/ by University of Connecticut - Health Center, Carl Coelho on 05/01/2014
S119
TABLE 3. Pearson product–moment correlation matrix among the Story Goodness Index (SGI; Lê, Coelho, Mozeiko, & Grafman, 2011) and the cognitive measures.
Results Descriptive Statistics Discourse measures. For story completeness, the mean score was 3.59 (SD = 1.55) of the five critical components referenced. For story grammar, the mean was a proportion of .61 (SD = .25) T-units in episode structure (Table 2). Cognitive measures. The mean Sorting Test score was 10.32 (SD = 3.31). The scoring range for the Sorting Test is 1 to 19, representing the lowest to highest possible scores. The mean WMI was 99.41 (SD = 13.5). The mean IMI was 96.02 (SD = 15.68). The scoring range of all possible scores for the WMI is 49 to 155, and that for the IMI is 45 to 155.
Correlations The story completeness measure correlated moderately to moderately high with the cognitive measures: r = .43 for the Sorting Test, r = .31 for the WMI, and r = .51 for the IMI, with p < .001 for all correlations (Table 3). The story grammar measure had low-to-moderate correlations: r = .32, p < .001 for the Sorting Test, r = .15, p < .05 for the WMI, and r = .32, p < .001 for the IMI (Table 3). There were moderately high correlations (r = .55, p < .001) between the Sorting Test and the WMI and between the Sorting Test and the IMI (r = .51, p < .001) and moderate correlations between the WMI and the IMI (r = .47, p < .001; Table 4).
Multiple Regression Analysis The multiple regression model with the three cognitive measures significantly predicted È30% of the variance in story completeness: R2 = .296, F(3, 163) = 22.81, p ≤ .001. Results indicated that only the Sorting Test (b = .23, p ≤ .05) and the IMI (b = .39, p ≤ .001) were significant predictors. The Sorting Test was a significant predictor in each of the three regression models—by itself, when the WMI was present in the second model, and when both the WMI and the IMI were present in the last model. The Sorting Test predicted 18.4% of the variance; the IMI contributed an additional 10.5% (Table 5).
TABLE 2. Descriptive statistics for the discourse and cognitive measures. Measure
M
SD
Story completeness (number of critical components) Story grammar (proportion of T-units in episode structure) Sorting Test Working Memory Primary Index Immediate Memory Primary Index
3.59
1.55
.61
.25
10.32 99.41 96.02
3.31 13.50 15.68
Note. The Sorting Test is from the Delis-Kaplan Executive Function System (Delis, Kaplan, & Kramer, 2001) and the Working Memory Primary Index and Immediate Memory Primary Index are from the Wechsler Memory Scale—Third Edition (Wechsler, 1997).
Story completeness
Story grammar
.43** .31** .51**
.32** .15* .32**
Sorting Test Working Memory Primary Index Immediate Memory Primary Index *p < .05; **p < .001.
The multiple regression model with the three cognitive measures accounted for 14.2% of the variance in story grammar: R2 = .142, F(3, 163) = 8.99, p ≤ .001. Again, only the Sorting Test (b = .26, p ≤ .01) and the IMI (b = .23, p ≤ .01) were significant predictors. The Sorting Test remained a significant predictor across the three models. The Sorting Test predicted 10.5% of the variance; the IMI contributed an additional 3.6% (Table 6). As expected, the second set of regression analyses (placing the WMI as the first predictor and the Sorting Test as the last) did not differ from the first set of regression analyses when all three cognitive predictors were entered in terms of the amount of variance predicted. However, the WMI was a significant predictor for both story completeness (b = .31, p ≤ .001) and story grammar (b = .26, p ≤ .05) in the first model by itself (Tables 7 and 8). When the IMI was entered in the second models for each of the discourse measures, the WMI became nonsignificant and the IMI was found to be significant (b = .46, p ≤ .001 for story completeness; b = .31, p ≤ .001 for story grammar). Paralleling results of the first set of regression analyses as expected with the same standardized beta values and significance levels, the WMI remained nonsignificant and the IMI maintained significance when the Sorting Test was added as the last predictor.
Discussion The aim of the present study was to examine the contribution of cognitive processes to individuals’ narrative discourse production following TBI. The cognitive domains of interest were EFs, WM, and immediate declarative memory. Specifically, this study attempted to identify the factors underlying the scattered performance of individuals with TBI on measures of story completeness and story grammar in the SGI. Results of the correlational analyses substantiate TABLE 4. Pearson product–moment correlation matrix among the cognitive measures. Sorting Working Memory Immediate Memory Test Primary Index Primary Index Sorting Test Working Memory Primary Index **p < .001.
S120 American Journal of Speech-Language Pathology • Vol. 21 • S115–S125 • May 2012 Downloaded From: http://ajslp.pubs.asha.org/ by University of Connecticut - Health Center, Carl Coelho on 05/01/2014
.55**
.51** .47**
TABLE 5. Summary of the multiple regression analysis for the variables predicting story completeness. Model 1
Model 2
Model 3
Variable
B
SE B
b
B
SE B
b
B
SE B
Constant Sorting Test Working Memory Primary Index Immediate Memory Primary Index R2 F for change in R 2
1.53 .20
.36 .03
.43**
.64 .17 .01
.82 .04 .01
.37** .10
–1.16 .11 –.001 .04
.85 .04 .01 .01 .30 24.22**
.18 37.17**
.19 1.45
b .23* –.01 .39**
*p ≤ .05; **p ≤ .001.
the relationship between particular cognitive abilities and story retelling. The moderate correlations between EF (i.e., Sorting Test) and performance on the SGI measures is consistent with findings in the literature that EF figures importantly in the assessments of narrative discourse content and organization at global levels. The Sorting Test taps a number of skills, including those considered to be separate component functions of EF: shifting, updating, and inhibition (Miyake, Emerson, & Friedman, 2000). These functions are analogous to some of the processes and mechanisms of the SBF and have been used to explain aspects of narrative discourse production (Mozeiko et al., 2011). New episode structures are constructed by shifting to new representations to accommodate novel story stimuli. Updating is similar to mapping new information to elaborate the currently activated episode structure. Inhibition is akin to the SBF’s suppression mechanisms and likely plays a role in managing extraneous or irrelevant story information and in dampening activation of current episode structures to enhance new ones. The moderate correlation between WM and story completeness and the weaker correlation between WM and story grammar suggest that WM contributes differently to informational aspects of discourse than to organizational aspects. These findings shed light on the conflicting studies of WM and discourse. Grouping these results with those in the literature (e.g., Chapman et al., 2006; Youse & Coelho, 2005), a pattern of dissociation emerges in which WM appears to vary with narrative content but not with narrative structure, suggesting that greater attention be given to the selection of the type of discourse analysis used in studying the role of
WM. Analyses involving discourse gist (Chapman et al., 2006) and story completeness involve global processing of the characters and events within the story. In both content analyses, the information within the story must be temporarily represented and manipulated, implicating WM processes. Discourse gist requires a distillation of the whole text into a succinct summary. Story completeness, as examined in the SGI, involves the processing of visual story information and the abstraction of the critical story components into an oral retelling. The moderate to moderately high correlations between immediate declarative memory and the narrative discourse measures suggest that immediate recall of explicit information in declarative memory is important for retelling stories that are complete and organized in content. Declarative memory has not been well examined in studies of discourse production, particularly in individuals with TBI. Findings from other neurologic populations may provide insights. For example, a study of the effect of hippocampal amnesia on a collaborative learning task found that participants with amnesia demonstrated deficits in flexibility for referential expressions, which were used to facilitate the task (Duff et al., 2008). The authors of the study suggested that although the inability to access richer stores of expressions did not necessarily affect the exchange of necessary information, it could potentially affect the quality of everyday social interactions. Likewise, it is possible that demands on declarative memory could affect the use and organization of story information in the TBI population. Within the SBF, problems in declarative memory may be reflected as memory nodes that may not be activated or may be inaccessible to activation for the
TABLE 6. Summary of the multiple regression analysis for the variables predicting story grammar. Model 1
Model 2
Model 3
Variable
B
SE B
b
B
SE B
b
B
SE B
Constant Sorting Test Working Memory Primary Index Immediate Memory Primary Index R2 F for change in R 2
.36 .02
.06 .01
.32**
.41 .03 –.001
.14 .01 .002
.35** –.04
.25 .02 –.002 .004
.15 .01 .002 .001 .14 6.84*
.11 19.34**
.11 .20
b .26* –.10 .23*
*p ≤ .05; **p ≤ .001.
Lê et al.: Predicting Story Goodness From Cognitive Measures
Downloaded From: http://ajslp.pubs.asha.org/ by University of Connecticut - Health Center, Carl Coelho on 05/01/2014
S121
TABLE 7. Summary of the post hoc multiple regression analysis for the variables predicting story completeness. Model 1
Model 2
Model 3
Variable
B
SE B
b
B
SE B
b
B
SE B
Constant Working Memory Primary Index Immediate Memory Primary Index Sorting Test R2 F for change in R 2
.097 .04
.85 .01
.31**
–1.81 .01 .05
.83 .01 .01
.09 .46**
–1.16 –.001 .038 .11
.85 .01 .01 .04 .30 7.78*
.10 17.24**
.26 37.21**
b –.01 .39** .23*
*p ≤ .05, **p ≤ .001.
formation of episode structures. Underlying impairments in explicit memory systems may then be reflected as stories that are missing essential content or episode components that organize the semantic information or perhaps as stories that lack personal connections or integration of knowledge beyond the boundaries of the stimuli presented. Findings from studies of amnesia and the current study suggest that more in-depth investigations of declarative memory and discourse production ability may be productive endeavors. The correlational analyses indicated moderate to moderately high correlations between all three cognitive measures. Without further analysis, it would be difficult to determine which, if any, of the cognitive factors predicts performance on measures of the SGI above and beyond the others. As such, the regression analyses afforded further insights into the relationship between cognition and discourse. Overall, results of the multiple regression analyses demonstrated that the cognitive measures significantly predicted performance on both discourse content and organizational measures, providing support for the involvement of these particular cognitive processes in story retelling. The first set of regression analyses in which EF was entered as the first predictor revealed the Sorting Test and immediate memory as the only significant predictors for story completeness and story grammar. A potential assumption from the first pair of regression models is that WM is not involved in processing or organizing information within stories. Given the research on WM and discourse, the scenario that WM is not an important factor is unlikely. A more plausible interpretation is that entering EF into the regression models first
cloaks the effect of WM. The Sorting Test, like many other EF measures, draws on several disparate cognitive skills (e.g., initiation of problem-solving behavior, abstract thinking, inhibition of previous responses; Miyake et al., 2000; Pickens et al., 2010). Therefore, it is a tenable hypothesis that the Sorting Test subsumes WM as well. Indeed, the second set of regression analyses, with EF entered last, bore out the claim that WM is a significant predictor for story completeness and story grammar as demonstrated by Model 1 for each measure. In Model 2, the presence of immediate memory washes out the significance of WM. This is not a surprising finding given that immediate memory is assumed to be contingent on WM. For example, to remember people and events in episode memory, knowledge must be simultaneously manipulated, processed, and stored to bind the features into a cohesive episode (Eichenbaum & Cohen, 2001). Spatial short-term memory tasks, in particular, have correlated highly and consistently with established WM measures, leading some to argue that spatial short-term memory tasks are in fact good WM tasks themselves (Oberauer, 2005). Model 3, in both the first and second set of regression models, demonstrated that declarative memory is an important substrate for discourse content and organization above and beyond the processes captured by the Sorting Test. Although the pattern of results within the models was similar between content and organizational measures, the amount of variance explained by each model was quite different. The Sorting Test, WM, and immediate memory accounted for a moderately large portion of the variance
TABLE 8. Summary of the post hoc multiple regression analysis for the variables predicting story grammar. Model 1
Model 2
Variable
B
SE B
b
B
Constant Working Memory Primary Index Immediate Memory Primary Index Sorting Test R2 F for change in R 2
.33 .003
.14 .001
.15*
.13
