Sensitivity and specificity of abstraction using gist reasoning measure ...

DOI: 10.1111/jabr.12073

ORIGINAL ARTICLE

Sensitivity and specificity of abstraction using gist reasoning measure in adults with traumatic brain injury Asha K. Vas1,* | Jeffrey S. Spence2,* | Benjamin Eschler2 |  Sandra B. Chapman2 1

Texas Woman’s University, Dallas, TX, USA

2

Center for BrainHealth-University of Texas, Dallas, TX, USA

Abstraction, a unique aspect of human reasoning, is affected by TBI. Lack of sensitive metrics to characterize abstraction in TBI recovery may impede detection of cognitive deficits

Correspondence Asha K. Vas, Department of Occupational Therapy, Texas Woman’s University, Dallas, TX, USA. Email: avas @twu.edu

and impact daily function. Recent findings found lower performance in abstracting meaning from complex information, referred to as gist reasoning, in adults with TBI. This study extends those findings to (a) compare abstract thinking between adults with TBI and controls on gist reasoning, similarities, and proverbs and (b) examine and compare sensitivity and specificity of abstraction measures. Participants included adults with moderate to severe chronic TBI (n = 30) and controls (n = 40), all between 25 and 55 years. ANOVA examined group differences on abstract thinking performance. A 10-­fold cross-­validation analysis examined sensitivity and specificity of each measure, and all possible combinations of measures. The TBI group performed significantly lower on gist reasoning and similarities. Results demonstrated higher sensitivity of gist reasoning (84.7%) as compared to similarities and proverbs. Specificity of gist reasoning (71.1%) was second highest, with proverbs multiple choice being the highest (85.2%). Combination of measures did not yield significant gains in sensitivity beyond gist reasoning alone. Gist reasoning could provide an ­additional diagnostic tool to detect impaired abstraction deficits in adults with chronic TBI. Furthermore, gist reasoning performance has implications to guide cognitive training.

*Equally contributing authors

216  |  wileyonlinelibrary.com/journal/jabr © 2016 Wiley Periodicals, Inc.

J Appl Behav Res 2016; 21: 216–224

Sensitivity, Specificity of Gist Reasoning in TBI

  |  217

1 | INTRODUC TION Capacity to abstract meanings from information that we encounter every day is one of the distinguishing characteristics of human cognition (Gabrieli, 2004). Humans draw upon this dynamic skill on a daily basis for optimal performance in vocational, social, and home activities. Impaired capacity to engage in abstract thinking as a result of disease or injury including traumatic brain injury (TBI) could have deleterious effects on daily functioning. Abstract thinking deficits following a TBI have been documented since World War-­I. Goldstein’s influential papers elaborated on abstraction deficits in adults with frontal lobe injuries, including TBI (Goldstein, 1936). In a case study of an adult with frontal lobe injuries, Goldstein states: In illustrations which contain a number of things the patient may pick out some details, but is unable to understand the picture as a whole and fails in acting in response to the whole. A precise examination reveals that the patient’s real understanding does not depend on the greater or smaller number of components in a picture, but on whether the components hang together concretely, and in ways familiar to the patient’s experience, or whether understanding of their connection requires a more abstract synthesis on his part. In one case, the patient may apprehend pictures with many details. In the second, he may lack understanding even if there are only few details. The patient’s deficiency lies in his inability to find out the essence of the picture (Scherzer, Charbonneau, Solomon, & Lepore, 1993). This case study eloquently describes the effects of TBI on abstract thinking, especially the disparity between understanding the components versus the whole/bigger picture. Since Goldstein’s reports, researchers have developed tools to examine the impact of TBI on abstraction abilities in acute stages of recovery. For example, abstraction is measured by an individual’s ability to generate categories for word pairs, word lists and/or recognizing implied meanings of single sentences, metaphors, proverbs, or segments of a paragraph (Scherzer, Charbonneau, Solomon, & Lepore, 1993; Holliday, Hamilton, Luthra, Oddy, & Weekes, 2005; Nicholas & Brookshire, 1995; Ferstl, Walther, Guthke, & von Cramon, 2005). Results indicated varying ­degrees of impairments on these word-­ and sentence-­level abstraction measures. Whereas abstracting categories for word pairs (e.g., in what way are an orange and banana similar?) showed minimal difficulty following a TBI, generating novel categories beyond commonly known ways to group words or objects was reported to be significantly impaired (Scherzer, Charbonneau, Solomon, & Lepore, 1993). Researchers also reported impact of TBI on recognizing implied meanings of story segments or familiar life situations also requiring abstraction to fill in gaps of literally expressed meanings (e.g., John and Mary laid a towel over the sand. Question: Where were John and Mary?) (Holliday, Hamilton, Luthra, Oddy, & Weekes, 2005). In daily life, one is typically called upon to assimilate, synthesize, and abstract meanings from large amounts of information, referred to as gist reasoning (Chapman & Mudar, 2014). Healthy adults engage in gist reasoning when understanding and extracting the essence and meaning whether from newspaper articles, movies, lectures, Internet stories, information from job interviews, or personalized medical information from medical professionals. Prior findings demonstrated gist reasoning deficits in adults with TBI in chronic stages of recovery (Vas, Spence, & Chapman, 2015). Results also identified positive associations between gist reasoning and daily-­life functionality. This study extends the findings of this previous work to accomplish three goals: (a) compare abstract thinking performance between adults with TBI and healthy controls on gist reasoning and commonly used abstraction measures of “similarities” and “proverbs tests” (free-­inquiry, multiple-­choice formats), (b) examine sensitivity and specificity of gist reasoning in classifying TBI versus healthy adults, and (c) compare sensitivity and specificity across gist reasoning and commonly used abstraction measures. Additionally, we examined combinations of abstraction measures that yielded highest sensitivity and specificity. We predicted lower performance in the TBI group as compared with control group on gist reasoning and similarities tests. We also hypothesized gist reasoning to demonstrate higher sensitivity and specificity as compared to similarities and proverbs in TBI versus healthy controls.

   | 

218

Vas et al.

2 | METHODS 2.1 | Participants This study included 70 adult participants: 30 adults with moderate to severe TBI and 40 healthy adults (control group). The TBI group consisted of 18 male and 12 female participants. The control group consisted of 20 male and 20 female participants. All participants aged between 25 and 55  years at testing. The TBI group included participants who sustained a moderate to severe TBI (Glasgow Coma Scale, GCS ≤12 at the time of injury) (Teasdale, Jennett, Murray, & Murray, 1983) at or after the age of 18, and were at least 1  year post injury. The current data are a subset of a larger study that examined the impact of TBI on gist reasoning, executive functions, and daily functionality in the same population. All participants were native English speakers with (a) minimum high school education, at least 8th-­grade equivalency on vocabulary and reading comprehension (as measured on Wide Range Achievement Test, WRAT) (Wilkinson & Robertson, 2006); as gist reasoning measure involves reading lengthy information, (b) estimated premorbid and current IQ of 85 or above (as measured on Wechsler Test of Adult Reading WTAR (Wechsler, 2001) and Wechsler Abbreviated Scale of Intelligence WASI (Wechsler, 1999), as lower intellectual abilities could contribute to variance in processing information in gist reasoning task (Table  1). Exclusion criteria for both groups (TBI and control) included previous history of stroke, learning disability, substance abuse, and major psychiatric disorders as determined by phone screening with the participant. Participants with significant vision and hearing problems were not included in the study as the experimental procedures involved reading and processing auditorily presented information. This study was conducted in accordance with the Institutional Review Board regulations of University of Texas at Dallas and University of Texas Southwestern Medical center.

2.2 | Measures Participants were tested on measures of gist reasoning, (Chapman, Gamino, & Cook, unpublished data) similarities (Wechsler, 2008), and proverbs tests (Delis, 2001). Gist reasoning (examined on Test of Strategic Learning, TOSL) assesses participant’s ability to synthesize complex/lengthy information into generalized ideas. The TOSL consists of three texts that vary in length (from 291 to 575 words) and complexity. The test requires the participant to provide a synopsis of each of the three texts. An example T A B L E   1   Demographic and clinical characteristics of TBI (n = 30) and control group (n = 40) Variables

TBI (n = 30)

Control (n = 40)

Age at testing

38.53 ± 9.91

37.67 ± 10.58

0.18

0.34 ns

Premorbid estimate verbal IQ (WTAR)

113.4 ± 8.72

116.2 ± 7.94

0.34

−1.43 ns

112.9 ± 11.39

−0.25

−1.46 ns

15.52 ± 1.79

−0.05

−0.21 ns

Current IQ (WASI) (Wechsler, 1999)

109 ± 13.62

ES

t(68)

Education

15.43 ± 1.75

SES (BSMSS) (Barratt, 2005)

43.39 ± 5.66

45.4 ± 8.26

−0.28

−0.92 ns

Reading grade (WRAT)

12.28 ± 1.17

12.51 ± 1.05

−0.2

−0.86 ns

Additional characteristics of TBI group GCS (Teasdale et al., 1983) Age at TBI Years since TBI

5.3 ± 3.08

–

–

–

28.8 ± 9.43

–

–

–

9.8 ± 8.83

–

–

–

TBI, traumatic brain injury; CI, confidence interval; ES, effect size; ns, group differences were not significant; WTAR, Wechsler Test of Adult Reading; WASI, Wechsler Abbreviated Scale of Intelligence; SES, socio-­economic status; BSMSS, Barratt Simplified Measure of Social Status; WRAT, Wide Range Achievement Test; GCS, Glasgow Coma Scale.

Sensitivity, Specificity of Gist Reasoning in TBI

  |  219

of a synopsis is provided to clarify that the task entails combining and synthesizing the explicit details in the text to construct generalized ideas. The participant is provided with a copy of the text to follow along as the examiner reads each text aloud. The TOSL measure has a manualized objective scoring system wherein only abstracted ideas are accounted for gist reasoning performance. The TOSL has demonstrated test–retest reliability and is validated as a measure of ability to abstract meaning from complex information in typically developing youth (Gamino, Chapman, & Cook, 2009), in healthy adults (Anand, Chapman, Rackley, Keebler, Zientz, & Hart, 2010), in children with TBI (Cook, Chapman, & Gamino, 2007), and in adults with TBI (Vas, Chapman, Cook, Elliott, & Keebler, 2011). The WAIS-­IV test of similarities consists of 20 word pairs. The examinee is asked to describe a common concept that is similar to each word pair (e.g., “In what way are cookies and candy alike?”). Test–retest reliability of WAIS-­IV ranges from 0.70 to 0.90. Inter-­scorer coefficients are also reported to be high (~0.90) (Wechsler, 2008). The Delis-­ Kaplan Executive Function System (D-­KEFS) Proverbs test consists of eight sayings that are presented both in free-­inquiry and multiple-­choice format (e.g., what does the saying “A small leak will sink a large ship” mean?). The free-­inquiry subtest is administered first followed by the multiple-­choice subtest. In the free-­inquiry format, the participant is asked to interpret the proverb and convey the generalized meaning. In the multiple-­choice format, the same eight proverbs are presented (one at a time) with four alternative interpretations for each saying. The participant is asked to identify the alternative that best explains the generalized meaning of the proverb. Performance on D-­KEFS subtests including proverbs tests, in different age groups, showed moderate to high range reliability. The validity of D-­KEFS instruments has been demonstrated in numerous neuropsychological studies (Delis, 2001; Delis, Kramer, Kaplan, & Holdnack, 2004).

2.3 | Procedures Each participant was tested in a quiet well-­ventilated room. All participants were first administered characterization measures (in no specific order), including IQ, SES, reading comprehension, and so on. Following the administration of the characterization measures, participants were tested on abstract thinking performance (in no specific order). Participants were offered rest breaks as needed. Two trained examiners blinded to the participants group status scored the performance on gist reasoning, similarities, and proverbs tests. Interrater reliability of scores assessed on intraclass correlation coefficients in both groups for all measures was over 90% (Cronbach’s α range 0.86–0.99, CI 0.76–0.98).

2.4 | Data analyses Descriptive analyses examined distribution patterns of participant characterization variables of age at testing, IQ, education, socioeconomic status, and reading proficiency, as these variables could influence abstract thinking performance. Analysis of variance examined group differences between TBI and control group on performance on abstract thinking measures. A linear discriminant analysis was utilized to predict class membership (i.e., TBI patient or healthy control) based on four abstraction measures-­gist reasoning, proverbs (free-­inquiry and multiple-­choice formats), and similarities. We allowed all possible combinations of abstraction measures to obtain maximum sensitivity (correctly identifying a TBI patient) and specificity (correctly identifying a healthy control). Classification rates were estimated by 10-­fold cross-­validation.

3 | RESULTS Both TBI and control groups were comparable on characterization measures, including age, education, IQ, and socioeconomic status (Table  1). Analysis of variance demonstrated significantly lower performance in the TBI group as compared with the control group on similarities and gist reasoning tests (Table  2). The

   | 

220

Vas et al.

T A B L E   2   Abstraction abilities in TBI group (n = 30) and control group (n = 40) Measures

Group

Mean

SD

Gist reasoning

TBI

11.43

7.65

Range

Control

21.10

Similarities

TBI

54.37

Control

57.62

Proverbs free inquiry

TBI

10.37

Control

11.42

Proverbs multiple choice

TBI

30.33

Control

31.00

2.43

95% CI

0–31

8.57–14.29

7.04

7–34

18.84–23.35

4.86

48–69

52.54–56.18

4.94

47–70

56.04–59.21

2.68

4–14

9.36–11.37

2.63

4–14

10.58–12.27

2.57

22–32

29.33–31.46

22–32

30.22–31.78

F(1, 68)

p

30.00