Neuropsychological Assessment and Employment Outcome After ...

The Clinical Neuropsychologist 2002, Vol. 16, No. 2, pp. 157±178

1385-4046/02/1602-157$16.00 # Swets & Zeitlinger

Neuropsychological Assessment and Employment Outcome After Traumatic Brain Injury: A Review 1

Mark Sherer1, Thomas A. Novack2, Angelle M. Sander3, Margaret A. Struchen4, Amy Alderson5, and Risa Nakase Thompson6

Methodist Rehabilitation Center, University of Mississippi Medical Center, Jackson, MS, USA, 2Spain Rehabilitation Center, University of Alabama-Birmingham, Birmingham, AL, USA, 3Baylor College of Medicine, The Institute for Rehabilitation and Research, Houston, TX, USA, 4Baylor College of Medicine, The Institute for Rehabilitation and Research, Houston, TX, USA, 5Shepard Center, Atlanta, GA, USA, and 6Methodist Rehabilitation Center, Jackson, MS, USA

ABSTRACT While there may be many reasons for obtaining neuropsychological assessment after traumatic brain injury (TBI), prediction of real world functioning is generally a key goal. The present paper reviews 23 studies concerning the relationship between neuropsychological test results and employment outcome after TBI. The review was conducted in accordance with guidelines developed by the Committee on Empirically Supported Practice of Division 40 (Neuropsychology) of the American Psychological Association. Results of the review support a Category A (strongly supported) recommendation for the use of early neuropsychological assessment to predict late employment outcome. Studies of late neuropsychological assessment and subsequent employment outcome and studies of concurrent neuropsychological assessment and employment outcome were inconclusive regarding either support or contraindication for neuropsychological assessment to predict employment outcome. Almost all studies conducted at these late or concurrent time points had signi®cant limitations with regard to study type or adequacy of methodology. However, there is no conceptual basis for believing that neuropsychological ®ndings obtained closer in time to assessment of employment outcome should be less predictive of this outcome than neuropsychological ®ndings obtained at an earlier time.

While traumatic brain injury (TBI) can cause a variety of physical and neurobehavioral impairments (Horn & Sherer, 1999), there is general agreement that cognitive and behavioral impairments are of the greatest signi®cance for patient and family adjustment and eventual functional outcome (Brooks, Campsie, Symington, Beattie, & McKinlay, 1986; Sherer, Madison, & Hannay, 2000). Given the importance of neurobehavioral impairments after TBI, neuropsychological assessment to document these impairments is a common aspect of care after TBI. The timing of neuropsychological assessments after TBI as well as the procedures used to assess cogni-

tive function may vary widely from one practitioner to another. In an effort to determine common practice for timing of neuropsychological assessments after TBI, Sherer and Novack (in press) surveyed 33 neuropsychologists who had extensive experience with patients with TBI. Survey results indicated that neuropsychological assessments were recommended at resolution of posttraumatic amnesia (PTA), 3- and 6-months postinjury, and 1 and 2 years postinjury for patients with moderate or severe TBI. In the same survey, neuropsychologists identi®ed 11 goals of neuropsychological assessment after TBI. These goals were: (1) documentation of

Address correspondence to: Mark Sherer, Ph.D., ABPP-Cn, Neuropsychology, Methodist Rehabilitation Center, 1350 E. Woodrow Wilson, Jackson, MS 39216, USA. Tel.: 1-601-364-3448. E-mail: [email protected] Accepted for publication: April 4, 2002.

158

MARK SHERER ET AL.

cognitive, behavioral, and emotional status, (2) feedback to the patient regarding strengths and weaknesses, (3) treatment implications, (4) feedback to the patient's family/signi®cant other, (5) recommendations regarding supervision needs, (6) recommendations regarding timing of return to work and/or school, (7) prediction of long-term functional outcome, (8) determination of competence, (9) recommendations regarding return to driving, (10) detection of late complications, and (11) determination of effectiveness of drug trials. The predominant themes of these 11 goals are documentation of de®cits, feedback to the patient, family, and clinicians, and prediction of functioning. While prediction of functioning is one primary goal of neuropsychological assessment after TBI, there has been criticism of the relevance of neuropsychological test ®ndings for prediction of ``real world functioning'' (Hart & Hayden, 1986; Sbordone & Long, 1996; Silver, 2000). Ecological validity has been de®ned as ``the functional and predictive relationship between the patient's performance on a set of neuropsychological tests and the patient's behavior in a variety of realworld settings (e.g., at home, work, school, community)'' (Sbordone, 1996, p. 16). The ecological validity of neuropsychological assessment has been questioned on several grounds, including: (1) lack of similarity of the testing environment to real world environments, (2) failure of neuropsychological tests to adequately sample cognitive abilities needed for real world functioning, and (3) suppression or enhancement of patient performance during neuropsychological assessment by nonneurologic factors (Sbordone, 1996; Silver, 2000). The emphasis on prediction of real world functioning after TBI found in the survey conducted by Sherer and Novack (in press) is in con¯ict with the concerns about the relevance of neuropsychological assessment for real world functioning described above. The goal of the present paper was to address this con¯ict by providing a comprehensive review of investigations of the relevance (ecological validity) of neuropsychological assessment to employment outcome after TBI. Return to work has been described as a key outcome after TBI because of its high value

within western society and its signi®cance for personal identity (Prigatano, 1999). Return to work has been the focus of many TBI outcome studies as this outcome may be assessed more reliably than other possible outcomes of interest (Sherer et al., 2000). For the present paper, the authors sought to review all investigations of the relevance of comprehensive neuropsychological assessment in predicting return to work after TBI. Studies were selected on the basis of exclusive focus on a TBI sample (thus excluding studies of mixed neurological populations) and actual measurement of return to work. Electronic searches using Medline and PsychInfo were conducted to identify articles for review. Additional articles were added through review of reference lists from previously identi®ed articles and through personal knowledge of the authors. A total of 23 articles were identi®ed for review. These articles could be placed in three categories based on the timing of neuropsychological assessment and evaluation of employment outcome: (1) early neuropsychological assessment (at resolution of PTA or 1 month postinjury) with late evaluation of employment outcome, (2) late neuropsychological assessment at 6 months postinjury) with later evaluation of employment outcome, and (3) concurrent neuropsychological assessment and evaluation of employment outcome. While the present review focused speci®cally on the relationship of neuropsychological test ®ndings to employment in persons with TBI, there is a much larger literature that has addressed the relationship of neuropsychological test ®ndings to a variety of functional outcomes in a variety of patient populations (see Heaton & Pendleton, 1981, for a review). PROCEDURE The present review was conducted in accord with guidelines developed by the Committee on Empirically Supported Practice (COESP) of Division 40 (Neuropsychology) of the American Psychological Association (Heaton, Barth, Crosson, Larrabee, & Reynolds, 2002). The goal of the COESP is to ``promote the evaluation and documentation of empirically supported neuropsychological practice and clinical application of neuropsychological principles to patient care.''

NEUROPSYCHOLOGICAL ASSESSMENT AND EMPLOYMENT OUTCOME

The COESP guidelines for review include several steps. Articles are reviewed to determine the type of study and the adequacy of methodology. Interrater agreement for type of study and adequacy of methodology is calculated and reported in the review. Based on the quality of empirical support for the neuropsychological practice being reviewed, a recommendation regarding empirically supported practice is made. COESP provides guidelines for determination of study type, adequacy of methodology, and recommendations regarding empirically supported practice. Based on preliminary experience with the guidelines for determination of study type and adequacy of methodology, the current authors revised these guidelines to make them more speci®c to the types of articles to be reviewed for this paper. Our revisions also resulted in guidelines that are more detailed with more speci®c criteria for determining study type and adequacy of methodology. The goal of these revisions was to improve interrater agreement. The authors practiced use of these guidelines on articles on other topics not included in this review. The guidelines for determination of study type and adequacy of methodology used in the current review are presented in Table 1 and Table 2, respectively. The COESP guidelines for recommenda-

159

tions regarding empirically supported practice are presented in Table 3. Review of the three types of studies was divided among the authors with the same pair of authors rating all articles in one of the three categories of studies. Each rater independently rated each article. When a pair of raters disagreed on a rating, a consensus, ®nal rating was reached through discussion.

FINDINGS Interrater Agreement Interrater agreement was acceptable. The three pairs of raters had perfect agreement on study type for 17 of 23 articles (74%) and gave ratings no more than one category different on 22 of the 23 articles (96%). Weighted Kappa was 0.76 ( p < :001) indicating excellent agreement (Landis & Koch, 1977). The pairs of raters had perfect agreement on adequacy of methodology for 15 of the 23 articles (65%) and gave ratings no more than one category different on all 23 articles

Table 1. Type of Study. I. High Generalizability and Reliability. Prospective study of a consecutive series of patients (trauma center admits); sampling is adequate to ensure generalizability ( 10 subjects per predictor variable) to the population of interest (e.g., persons with TBI); may be a multicenter or well done, large, single center study; subjects lost to follow-up are described and reasons for failure to return to follow-up are mentioned; analyses permit speci®c conclusions about predictive power. II. Moderate Generalizability and Reliability. (1) Similar to I, but is a single center study with less careful description of the initial series and/or failure to describe subjects not available for follow-up; or (2) Prospective study of a series of patients admitted for inpatient rehabilitation following TBI; sampling is adequate to ensure generalizability ( 10 subjects per predictor variable) to the population of interest; may be a multicenter or well done, large, single center study; subjects lost to follow-up are described and reasons for failure to return to followup are mentioned; analyses permit speci®c conclusions about predictive power. III. Limited Generalizability and Reliability. (1) Similar to I, but is a single center study with marginal sample size (5±9 subjects per predictor variable); or (2) Similar to II-2 with regard to 10 subjects per predictor variable, but is a single center study with less careful description of the initial series and/or failure to describe subjects not available for follow-up; or (3) Prospective study of a series of patients admitted for postacute rehabilitation following TBI; sampling is adequate to ensure generalizability ( 10 subjects per predictor variable) to the population of interest; subjects lost to follow-up are described and reasons for failure to return to follow-up are mentioned. IV. Very Limited Generalizability and Reliability. (1) Similar to II-2, but is a single center study with marginal sample size (5±9 subjects per predictor variable); or (2) Similar to III-3, but with less careful description of the initial series and/or failure to describe subjects not available for follow-up. V. Unknown Generalizability and Reliablity. Series of patients not meeting I, II, III, or IV above. Such series are likely to be so idiosyncratic that generalizability to other patients groups cannot be estimated reliably. VI. Unknown Generalizability and Reliablity. Single case report. Note. Adapted from Heaton et al. (2002).

160

MARK SHERER ET AL.

Table 2. Adequacy of Methodology. Subjects

Measures

Methodology

Analysis

Commendable 1) Method of recruitment is well described. 2) Subjects are representative of larger TBI population (consecutive trauma admits or consecutive rehab admits). 3) Sample is well described ± age, education, cause of injury, severity of injury, time from injury to assessment of predictors, time from injury to assessment of outcome, etc. 4) Analyses are done comparing patients lost to follow-up to those who were available for follow-up.

1) Appropriate range of neuropsychological areas is assessed. 2) Norms are used as appropriate. 3) The measure of outcome is clearly related to the outcome of interest (Does the study claim to measure one thing [employment], when it really measures another [employability]?).

1) Outcome ratings are collected independently of assessment of predictors. 2) Predictive power of neuropsychological measures is adjusted for more than 1 other relevant predictors (e.g., severity, age, education, etc.).

1) Sample size is adequate ( 10 subjects per predictor). 2) Analysis is not based on a stepwise entry of predictors unless a clear a priori rationale for the order of entry is presented. 3) Adjustment is made for multiple tests (correlations or between group comparisons).

Acceptable

1) and 3) from above but subjects may be somewhat less representative of the general TBI population. An example would be patients referred to a postacute program; the ®ndings may generalize to patients treated in other postacute programs but will not generalize to the overall TBI population. Patients lost to follow-up and reasons for loss to follow-up are described but analyses comparing lost patients to retained patients are not presented.

At least 2 of the 3 above are present.

1) Possible confounding of outcome ratings cannot be ruled out, but did not clearly occur. 2) Some attempt is made to account for or adjust for at least one mediating variable.

1) Sample size is adequate ( 10 subjects per predictor). 2) Stepwise regression is acceptable. (3) Numbers of comparisons is excessive, but comparisons are planned.

Marginal

Marked restrictions in the population sampled, such as only those patients who were available for followup. Limited description of sample such as missing severity of injury or missing time from injury to follow-up.

At least 1 of the above is present.

Some degree of confound seems likely, but does not necessarily invalidate the ®ndings.

1) Marginally inadequate sample size (5±9 subjects per predictor). 2) Number of correlations or comparisons is excessive and unplanned (not speci®cally mentioned in introduction, hypotheses, or planned analysis section).

161


Table 2. (Continued). Flawed

Sample is speci®c to one setting and would not generalize to others. Description of sample is too poor to allow determination of generalizability.

None of the above is present.

Outcome ratings are clearly confounded with assessment of predictors.

1) Fewer than 5 subjects per predictor in a regression analysis. 2) Number of correlations or comparisons is clearly excessive (as many comparisons as subjects).

Note. Adapted from Heaton et al. (2002).

Table 3. Recommendations Regarding Empirically Supported Practice. Category A (strongly supported) ± The practice is supported by at least one de®nitive, multicenter Level I study (or single center with very broad population) with internally consistent ®ndings, or more than one study with more restricted samples. The supporting studies have no ``fatal ¯aws.'' Category B (supported) ± The practice is supported by only one (single center, nonreplicated) Level I study or by more than one Level II study with no fatal ¯aws and clear, positive results. Category C (tentatively supported) ± Either of the above, but the studies have serious ¯aws and/or equivocal results. Or, the practice is supported by the preponderance of evidence from Level III or lower studies ± more/better research is needed. Category D (insuf®ciently or inconclusively supported) ± Available evidence is insuf®cient in amount, quality and/ or consistency to warrant conclusions at the present time. Category E (contraindicated) ± Preponderance of adequate evidence indicates that the practice is not useful. Note. From Heaton et al. (2002).

(100%). Weighted Kappa was 0.55 ( p < :01) indicating fair to good agreement (Landis & Koch, 1977). STUDIES WITH EARLY NEUROPSYCHOLOGICAL EVALUATION AND LATE ASSESSMENT OF EMPLOYMENT OUTCOME Nine studies that examined the ability of early neuropsychological assessment to predict late employment outcome were reviewed. Neuropsychological assessment was considered ``early'' if it was obtained at resolution of PTA or within 1 month postinjury. Some articles did not specify the timing of initial neuropsychological evaluation and these were included if it appeared that

the neuropsychological evaluation was completed during inpatient rehabilitation. Employment outcome was considered ``late'' if employment status was assessed 1 year or more postinjury. Summaries of the nine studies and Study Type and Methodology ratings are presented in Table 4. Najenson, Groswasser, Mendelson, and Hackett (1980) studied 147 persons with TBI (125 with blunt injuries and 22 with penetrating injuries) who were treated in an inpatient rehabilitation program and/or day treatment program. The strength of the study is that it provides an early demonstration of the relationship of neuropsychological assessment to employment outcome. Cognitive status was assessed with a battery of neuropsychological measures, but speci®c ®ndings are only reported for the Wisconsin Card

162

Table 4. Studies of Early Neuropsychological Assessment and Late Productivity Outcome. Subjects

Timing of cognitive evaluation

Timing of follow-up

Findings

Study type ratings

Methodology ratings

Najenson et al., 1980

147 consecutive patients with TBI (125 blunt injuries, 22 penetrating injuries) who were seen for inpatient rehabilitation and/or day treatment. 102 consecutive patients with TBI seen at a trauma center.

During inpatient rehabilitation or day treatment.

Greater than 6 months postdischarge.

Degree of overall cognitive de®cit was predictive of employment outcome at follow-up.

R1 ± III R2 ± III Final ± III

R1 ± Marginal R2 ± Marginal Final ± Marginal

1 month postinjury.

1 year postinjury.

Patients who were employed at follow-up were less impaired on several neuropsychological measures than those who were not employed at follow-up.



Fraser et al., 1988

Dikmen et al., 1994

366 consecutive patients with TBI (93 severe, 56 moderate, and 213 mild) seen at a trauma center who were working at time of injury.

1 month postinjury.

1 month, 6 months, 1 year, 2 years postinjury.

Neuropsychological variables (testability at 1 month postinjury, the Halstead Impairment Index, and Name Writing speed) were predictive of employment status at follow-up.

R1 ± II R2 ± II Final ± II

R1 ± Acceptable R2 ± Acceptable Final ± Acceptable

Vilkki et al., 1994

53 patients with TBI from a series of 109 consecutive patients with TBI seen at a trauma center.

1 to 8 months postinjury.

1 year postinjury.

A principal component analysis with varimax rotation of 11 cognitive tests resulted in 2 factors. Both factors were predictive of work status at follow-up.

R1 ± III R2 ± IV Final ± IV

R1 ± Marginal R2 ± Flawed Final ± Flawed

MARK SHERER ET AL.

Study

Table 4. (Continued). Fabiano & Crewe, 1995

Fleming et al., 1999

Boake et al., 2001

Sherer et al., 2002

R1 ± Marginal R2 ± Flawed Final ± Flawed

An average of 58 months (SD 42.6) postinjury.

At resolution of PTA.

1 year postinjury.


2±5 years postinjury (mean 44.2 months, SD 16.9).

Scores from a cognitive screening test were predictive of employment status at follow-up.


R1 ± Acceptable R2 ± Marginal Final ± Acceptable


1±4 years postinjury.

10 of 15 neuropsychological measures were signi®cantly predictive of employment outcome at follow-up.


R1 ± Marginal R2 ± Acceptable Final ± Marginal


1 year postinjury.

Overall cognitive status was predictive of productivity outcome even when adjusted for age, education, initial GCS, PTA, and preinjury employment status.




163

Note. R1 Rater 1, R2 Rater 2.

R1 ± IV A discriminant analysis revealed that various measures R2 ± V Final ± V from the Wechsler Adult Intelligence Scale ± Revised contributed to prediction of employment status at follow-up. R1 ± IV Patients who were employed R2 ± IV at follow-up differed from Final ± IV those not employed on delayed recall but not on initial recall, ®ne motor speed, or visual scanning speed.

During inpatient rehabilitation.


Cifu et al., 1997

94 patients from a consecutive series of 126 patients with severe TBI seen at 1 of 3 rehabilitation centers. 132 consecutive patients with TBI seen at 1 of 4 rehabilitation centers who were employed at time of injury and for whom 1 year followup data were available. 209 patients with TBI from a series of 478 consecutive patients with TBI admitted to a rehabilitation unit. 293 patients with nonpenetrating TBI (176 severe, 55 moderate, 56 mild, 6 unknown) who were seen for inpatient rehabilitation at 1 of 4 centers, testing was available for 230 patients. 388 patients with TBI from a series of 667 eligible patients from the TBI Model Systems data base.

164

MARK SHERER ET AL.

Sorting Test and the Raven Matrices Test. Results indicated that while 27 (73%) of 37 patients, whose cognitive functioning at initial neuropsychological evaluation was considered normal, were working competitively at follow-up and 27 (43.5%) of 62 patients with slight cognitive impairment were competitively employed, only 9 (18.8%) of 48 patients with severe impairment were competitively employed. Weaknesses of the study include limited description of the sample and inadequate description of the protocol for assessment and follow-up. While the initial neuropsychological assessment was completed during active rehabilitation, and follow-up information regarding employment status was obtained at least 6 months after discharge from rehabilitation, the exact times postinjury for the neuropsychological evaluation and the follow-up are not reported. While most subjects had coma of greater than 1 day, 21 subjects were described as having coma of 1 day or less. Unfortunately, durations of loss of consciousness and PTA were not reported for these 21 subjects so that the possibility that some of them may not have sustained brain injuries cannot be ruled out. While the study appears to have been well done, the failure to report adequate information regarding subjects' characteristics and multiple aspects of the methodology severely limits the conclusions that can be drawn from this investigation. Fraser, Dikmen, McLean, and Miller (1988) studied 102 consecutive patients hospitalized with TBI (60 with initial GCS 12, 12 with GCS 9± 11, and 30 with GCS 3±8). Neuropsychological assessments were obtained at 1 month postinjury using several measures from the Halstead±Reitan Neuropsychological Battery. Patients who were employed at 1 year postinjury were signi®cantly less impaired on all neuropsychological measures than those who were not employed at follow-up. This study provides important ®ndings indicating that early cognitive impairment is associated with poorer employment outcome. Weaknesses include failure to control for multiple comparisons, failure to statistically control for other predictors of employment outcome (such as age and injury severity), and restriction of discussion to only those patients who were employed at the time of injury.

Dikmen et al. (1994) studied 514 consecutive patients with TBI who were hospitalized at a trauma center. The 514 patients studied by these authors include the 102 patients previously reported on by Fraser et al. (1988). Analyses focused on the 366 patients who were employed at the time of injury. Of these patients, 93 had severe TBI (GCS 8), 56 had moderate TBI (GCS 9±12), and 213 had mild TBI (GCS 13±15). Follow-up at 1 year postinjury was excellent with only 6% of subjects lost to follow-up. Univariable analyses revealed that a number of factors such as education level, premorbid job stability, injury severity, and neuropsychological test ®ndings were predictive of employment status at follow-up periods including 1 and 2 years postinjury. Neuropsychological measures studied were the Halstead Impairment Index and name writing speed. Multivariable regression models indicated that neuropsychological test performance made an additional contribution to prediction of employment outcome above that made by demographic and injury severity variables. The exact improvement in predictive power made by neuropsychological variables was not reported. This study is very well done and provides excellent evidence of the relationship of early cognitive status to late employment outcome. Vilkki et al. (1994) studied 53 patients with TBI taken from a consecutive series of 109 TBI patients seen at a trauma center. Injury severity was not reported and exact time to neuropsychological testing also was not reported. Work status at follow-up was considered normal if the patient was working at or above his/her preinjury level. Thus, a patient who was competitively employed at a level below his/her preinjury level would not have coded as having ``normal'' work status. Initial analyses were based on the authors' conceptual groupings of 13 cognitive measures. Seven of the 13 scores differed between subjects with normal versus impaired work status. In a secondary analysis, two cognitive factors (Operational Resources and Mental Programming) were derived from 11 of the cognitive tests using principal components analysis followed by varimax rotation. Both cognitive factors contributed to prediction of work status at follow-up. The signi®cance of the ®ndings from this study is


severely limited by inadequate characterization of the sample, an excessive number of analyses given the sample size, and the use of stepwise regression with no apparent a priori determination of order of variable entry. Fabiano and Crewe (1995) studied 94 patients from a series of 126 patients with severe TBI who were seen for inpatient rehabilitation at one of three centers. Initial neuropsychological assessment was done during inpatient rehabilitation, but the exact time from injury to evaluation was not reported. Follow-up determination of employment outcome was obtained at an average of about 5 years postinjury. Several measures from the Wechsler Adult Intelligence Scale ± Revised (WAIS±R) contributed to prediction of employment outcome. Due to missing WAIS±R data, only 66 subjects were included in the discriminant analysis while 12 predictors were utilized. With fewer than 6 subjects per predictor, the resulting prediction model is likely quite unstable and this constitutes a major limitation for this study. Cifu et al. (1997) studied 132 patients who were employed at the time of TBI and were taken from a larger series of 245 patients for whom 1year follow-up data were available. Subjects were studied as part of the TBI Model Systems program funded by the National Institute on Disability and Rehabilitation Research. Fortynine patients who were employed at followup were compared to 83 patients who were not employed at follow-up on 14 functional ratings and ®ve neuropsychological test scores using t tests with no correction for multiple comparisons. Those employed at follow-up differed from those not employed on 6 functional ratings and on delayed verbal recall (Wechsler Memory Scale Logical Memory). No differences were seen for immediate recall (Wechsler Memory Scale Logical Memory), ®ne motor speed (Grooved Pegboard Test), or speed of visual scanning (Trail Making Test). The strength of the study is the excellent characterization of the sample studied, but the signi®cance of the ®ndings is limited by the failure to evaluate the possible moderating effects of neuropsychological test results on functional ratings and the large number of comparisons done on a sample of limited size with no correction for multiple comparisons.

165

Fleming, Tooth, Hassell, and Chan (1999) investigated early cognitive status and delayed employment outcome in a sample of 209 patients with TBI who responded to a follow-up questionnaire sent by mail to a series of 446 TBI patients seen at an inpatient rehabilitation unit. No comparison of the 209 patients who returned questionnaires and the 237 patients who did not was presented. Findings indicated that scores from a cognitive screening test (the Barry Rehabilitation Inpatient Screening of Cognition ± BRISC; Barry, Clark, Yaguda, Higgins, & Mangel, 1989) were predictive of employment status at follow-up. A weakness of the study is the failure to obtain follow-up data on over 50% of the sample and failure to compare the 209 subjects for whom follow-up data were available to the 237 for whom follow-up data were not available. Obviously, the generalizability of the ®ndings is limited by the high percentage of subjects lost to follow-up. Otherwise, the study is well done and provides strong evidence for the predictive value of early cognitive status. Boake et al. (2001) studied 293 patients with nonpenetrating TBI from the TBI Model Systems data base. Only patients for whom employment follow-up data were available were studied and there was no description of patients lost to followup. Two hundred and thirty patients received at least partial neuropsychological assessments at resolution of PTA. Univariate risk ratios indicated that 10 of 15 neuropsychological measures were predictive of employment outcome assessed at 1 to 4 years postinjury. This study is generally well done except for the failure to provide any information about patients lost to follow-up. A strength of this study is the examination of the predictive power of individual neuropsychological measures. These ®ndings may provide guidance to practitioners regarding test selection for evaluation of patients with TBI. A weakness is the failure to adjust these measures for other predictors of outcome such as education level, preinjury employment status, and injury severity. Sherer et al. (2002) studied 667 patients with TBI seen for inpatient rehabilitation at one of six TBI Model System sites. Analyses were based on 388 subjects for whom 1 year follow-up data were available. A comprehensive neuropsychological

166

MARK SHERER ET AL.

assessment was done at resolution of PTA. In order to reduce the number of predictors submitted to multivariable logistic regression analysis, the 16 available neuropsychological tests scores were submitted to principal component analysis (PCA). PCA resulted in three components that were interpreted as overall indices of overall cognitive status. The authors did not attempt to interpret the speci®c underlying cognitive abilities represented by each of the three components. Multivariable logistic regression analysis revealed that cognitive status at initial assessment was predictive of productivity status at 1-year postinjury even when adjusted for other predictors including age, education level, preinjury employment status, and severity of injury as indicated by the initial Glasgow Coma Scale (GCS) score and duration of PTA. Strengths of this study include the large sample size and sophisticated approach to analysis. A weakness is the fact that 279 subjects (42%) were lost to follow-up. While the authors reported the number of subjects lost to follow-up, no comparison was provided between subjects available for follow-up and those not available for follow-up. STUDIES WITH LATE NEUROPSYCHOLOGICAL EVALUATION AND SUBSEQUENT ASSESSMENT OF EMPLOYMENT OUTCOME Nine studies that provided information regarding the relationship of late neuropsychological assessment to subsequent employment outcome were reviewed. Summaries of these nine studies and Study Type and Methodology ratings are presented in Table 5. Many of the studies in this section focused on establishing the ef®cacy of postacute programs. The association of neuropsychological test performance and return to employment was a secondary issue. As a result, important information was sometimes lacking and independent analysis of the neuropsychological test performance was limited. In an often-cited pioneering study, Prigatano et al. (1984) examined the effectiveness of postacute rehabilitation on cognitive recovery and return to employment. Eighteen participants in a

postacute rehabilitation program completed neuropsychological evaluation an average of 21.6 months after injury (range 6±81 months) and again after completion of the 6-month program. Although not speci®cally stated, return to work was apparently assessed 2±33 months following program completion. The neuropsychological test results obtained at the conclusion of the program did not differentiate the employed from the unemployed subjects. However, participants did demonstrate signi®cant cognitive improvements during the 6-month program (WAIS, PIQ, Block Design, and Wechsler Memory Quotient) in comparison to a matched control group also tested on two occasions. Based on multiple analyses of covariance with initial test scores serving as covariates, program participants returning to competitive employment exhibited greater improvements on measures of psychomotor speed and memory (Digit Symbol, Wechsler Memory Quotient, Visual Reproduction, and Associative Learning). Although this study is commendable in several respects, the number of statistical analyses is excessive for the small number of subjects. In addition, the small and highly selective group of individuals included in this study limits the potential generalization of the ®ndings to other individuals with TBI. The study by Burke, Wesolowski, and Guth (1988) represents another attempt to demonstrate the effectiveness of postacute rehabilitation. Thirty-nine individuals were administered a battery of neuropsychological tests, presumably at the initiation of a postacute rehabilitation program lasting an average of 8 months. At scheduled evaluations, occurring 3±12 months after program completion, 50% of program participants were employed. One-way analysis of variance with employment status as a betweensubjects variable yielded no signi®cant differences in WAIS±R score (presumably the FSIQ) and Wechsler Memory score (presumably the MQ), the only cognitive tests mentioned. Unfortunately, the study is weakened by problems with the statistical analysis and methodology. The authors did not employ multivariate statistics despite considering multiple variables in examining outcome and the sample was small and biased in selection. In addition, limitations in the descrip-

Table 5. Studies of Late Neuropsychological Assessment and Subsequent Employment Outcome. Subjects

Timing of cognitive evaluation

Timing of follow-up

Findings

Study type ratings

Methodology ratings

Prigatano et al., 1984

18 participants in a postacute program; all had severe injuries based on coma duration > 24 hours.

22 months for initial assessment; 28 months for follow-up assessment.

Between 2 and 33 months following completion of the program.

Those returning to work evidenced greater change in tests of memory, visuo-motor speed, and psychopathology.

R3 ± IV R4 ± V Final ± V

R3 ± Acceptable R4 ± Commendable Final ± Acceptable

Burke et al., 1988

39 participants in a postacute program; 6 weeks of coma on average, but no range provided.

At least 4 years after injury.

3±12 months following program completion.

WAIS±R FSIQ and WMS MQ R3 ± V R4 ± V did not differentiate between Final ± V those returning and not returning to competitive employment.

Ezrachi et al., 1991

59 participants in a postacute program; described as moderate to severe injuries.

35 months for initial assessment; 40 months for follow-up assessment.

6 months after completion of the program.

Visual processing skills, verbal reasoning, and verbal aptitude were associated with employment.

Lam et al., 1991

40 participants in a postacute program; described as having severe TBI.

Unstated.

23 months postinjury.

R3 ± IV The competitively employed group exhibited better WAIS±R R4 ± IV Final ± IV PIQ, visual memory, and perceptual-motor skills.


Ryan et al., 1992

28 months on Unstated. 80 consecutive average (SD 3.4). admissions for a prevocational training program; no description of injury severity.

R3 ± V Test results loaded on three factors associated with referral R4 ± V for vocational evaluation, but Final ± V no relationship was found with actual employment.


Ruff et al., 1993

93 subjects initially, falling to 55 at follow-up; described as severely injured.

Initial testing presumably on clearing of PTA, follow-up testing at 6 months after injury.

Employment at 6 and 12 months after injury onset.

Verbal intellectual skills and speed of processing were strong predictors of return to work.

R3 ± V R4 ± V Final ± V

R3 ± II R4 ± III Final ± III



R3 ± Commendable R4 ± Acceptable Final ± Acceptable


Study

167

168

Table 5. (Continued). Timing of cognitive evaluation

Timing of follow-up

Findings

Study type ratings

Study

Subjects

Malec et al., 1993

R3 ± V 12 months following There was no correlation Time between 29 subjects (20 with program completion. between 12 neuropsychological R4 ± V TBI) participating in a injury and measures and return to work. Final ± V assessment varied postacute program; from 38 days to 25 severity of injury is years. unclear. Employment status determined at the same time as cognitive assessment.

Average of 23 months for the employed group; 30 months for the unemployed group.

O'Connell, 2000

Not indicated. Median of 9.5 months, but greater than 10 years in 2 cases.

43 consecutive admissions to a postacute program; no severity information data base.


R3 ± Commendable R4 ± Commendable Final ± Commendable

A test of verbal reasoning and two tests of auditory processing correctly classi®ed 85% of the subjects regarding employment status.



Verbal memory and Performance IQ were signi®cantly related to employment.

R3 ± IV R4 ± IV Final ± IV


MARK SHERER ET AL.

Isaki & Turkstra, 2000 10 employed subjects, 10 unemployed subjects; moderate TBI based on GCS score.

Methodology ratings


tion of the subject sample, such as providing only vague information regarding time since injury and time of program participation, limit the potential generalization of these results. Ezrachi, Ben-Yishay, Kay, Diller, and Rattok (1991) evaluated the relationship between neuropsychological performance and employment for 59 patients in the context of another pioneering postacute program. Because a large number of neuropsychological tests were administered, factor analysis of test performances was utilized to reduce the total number of neuropsychological predictors to 12. Stepwise multiple regression revealed an initial Verbal Aptitude factor (de®ned by WAIS Comprehension, Similarities, and Vocabulary subtests completed at initiation of the program at an average of 34 months after injury) as a signi®cant predictor of return to work, although both duration of coma and community reintegration contributed more strongly to prediction of vocational outcome. At follow-up evaluation (about 5 months after the ®rst evaluation), both Verbal Aptitude and Psychomotor Dexterity (de®ned by Purdue Pegboard performance) were predictive of employment; however, other variables again contributed more strongly to predicting vocational outcome. As with the Prigatano et al.'s (1984) study, the generalization of this study's results is questionable due to small sample size and the selective inclusion. The small subject to predictor ratio (even with the data reduction procedures employed) also limits the statistical reliability of this investigation. The study by Lam, Priddy, and Johnson (1991) represents a retrospective analysis of data obtained during postacute rehabilitation with follow-up evaluation of the 40 subjects taking place at an average of 23 months postinjury (the time from program completion to follow-up was not provided, nor was the time from injury to program admission). From an eclectic group of neuropsychological tests administered sometime during the course of the treatment program (at an unspeci®ed interval following injury), the authors chose to investigate a select number of tests commonly administered. Curiously, their choice of tasks did not include a measure of verbal memory functioning. At follow-up evaluation, 19 participants (48%) were employed, although only 6 were

169

considered to be competitively employed (making $200 or more per week). A total of eight scores derived from the neuropsychological tests were subjected to one-way analyses of variance with employment status serving as a between-subjects factor. Cell sizes ranged from as low as 4 subjects. Analysis of variance revealed that the competitively employed group exhibited signi®cantly better performance than the marginally employed and unemployed groups on the WAIS±R PIQ, the Benton Revised Visual Retention Test, and the Distractibility component of the Gordon Diagnostic System. Unfortunately, the predictive capacity of these variables was not addressed using multivariate techniques. The authors also noted that WAIS±R FSIQ was signi®cantly correlated with postinjury weekly wage. This study suffers from the same ¯aws already identi®ed in other studies, particularly the reliance on a very small sample size. In addition, there is insuf®cient description of the sample and selection criteria to allow reliable and valid generalization of outcome results. Ryan, Sautter, Capps, Meneese, and Barth (1992) studied the relationship between neuropsychological test performance and success of prevocational training in a sample of 80 consecutive admissions to a prevocational training program at a state-supported rehabilitation center. Due to attrition, however, data from only 54 subjects were available for analysis. In contrast to other studies reviewed, the outcome measures utilized are not a re¯ection of employment per se, but rather represent referrals for vocational training (74%) and employment status at the end of such training (15%). The impact of the neuropsychological test results on the referral process was not described. Several neuropsychological tests (Peabody Individual Achievement Test Reading Comprehension, Rey Auditory Verbal Learning Test, Aphasia Screening Test), and the Beck Depression Inventory discriminated between those individuals receiving vocational referrals and those not, with correct classi®cation of 78% of individuals. Apparently, a similar analysis was not performed with respect to the outcome of vocational training. However, the authors did subject the neuropsychological test scores to a factor analysis, identifying factors

170

MARK SHERER ET AL.

labeled as overall cerebral integrity, verbal mnemonic functioning, and depressive symptomatology. Based on an unspeci®ed analysis the authors concluded that the factor scores were not predictive of vocational training outcome. Unfortunately, the small sample size, particularly given the analyses employed, is again a major limitation that undermines this study. In addition, the severity of injury of the sample is not adequately described and it is questionable whether this study actually assesses return to employment as opposed to participation in vocational training. The study by Ruff et al. (1993) had the potential to generate a large sample size given their reliance on the Traumatic Coma Data Bank. Unfortunately, of 242 eligible subjects only 93 had full data at initial assessment, dropping to 55 at 6- and 12-months follow-up. Given the criteria for entry into the data bank, it appears that all subjects were severely injured, although speci®c severity data were not provided in the article. Neuropsychological evaluations took place 6 months after injury, while employment outcome was assessed at 1-year postinjury. The authors focused on 11 variables, 7 of them neuropsychological test scores, as predictors of return to employment at 6 months (18%) and 12 months (31%) postinjury. Although not clearly stated, it appears that students returning to school were included in the employment group. Pearson correlations re¯ected signi®cant relationships between return to work and several neuropsychological test scores, including the Trail Making Test, the Block Design subtest of the WAIS±R, the Finger Tapping Test, and the speed component of the 2 and 7 Test. A logistic regression model employing the WAIS±R Vocabulary subtest and the 2 and 7 speed score correctly predicted those unable to return to work, but lacked sensitivity in identifying those who did return to competitive employment. While this study has a number of commendable features, sampling processes are confusing, and the authors fail to evaluate those individuals lost to follow-up. Nonetheless, the other commendable aspects of this study support more reliable and valid generalization of the ®ndings than other studies already discussed. The effectiveness of a postacute program modeled after Prigatano et al. (1984), including

employment evaluation at 1 year postcompletion, was examined by Malec, Smigielski, DePompolo, and Thompson (1993). Twenty-nine subjects ®nished the program, but only 21 were eligible for the designated 12-month followup. Neuropsychological evaluation consisting of 12 tests was completed during the preprogram screening. At follow-up 10 subjects (48%) were employed. None of the neuropsychological test measures correlated signi®cantly with vocational outcome. Although commendable for the measures employed and the methodology (such as not losing a single case eligible for follow-up), the impact of this study is attenuated by small sample size and heterogeneous sampling practices, such as the wide variation in time between injury and assessment. Isaki and Turkstra (2000) examined the relationship between language skills and employment status in 10 employed individuals versus 10 unemployed individuals. Injury severity was described as moderate to severe, but selection criteria were not described. All subjects were between 1 and 4 years postonset of injury. Using a combination of t tests and discriminant function analyses, the authors found that a measure of verbal reasoning and two tests of auditory discrimination correctly classi®ed 80% of the employed subjects and 90% of the unemployed subjects. The reliability of this ®nding, and the generalization of the results overall, is questionable given the small sample size and the failure to adequately describe the sampling procedure. In the context of a postacute rehabilitation program, O'Connell (2000) administered the WAIS±R and WMS±R. Summary scores from these instruments, along with demographic information (age, gender, race, occupation prior to injury, and education) were entered into a logistic regression to predict return to work for a group of 43 people with TBI. Unfortunately, the severity of injury was not speci®ed and the interval between injury and neuropsychological assessment was very broad (up to 10 years postinjury). The WMS±R Verbal Memory quotient, and to a lesser extent the WAIS±R PIQ, accounted for a signi®cant proportion of the variance in employment status, with higher scores associated with a return to employment. The reliability and generalization


of these results is questionable based on the small subject: predictor ratio, inadequate information regarding study methodology, and limitations in sampling and sample description. STUDIES WITH CONCURRENT ASSESSMENT OF NEUROPSYCHOLOGICAL STATUS AND EMPLOYMENT OUTCOME Only ®ve studies have investigated the relationship between neuropsychological test performance and concurrent employment status. Of these, only one study had this relationship as its major focus. The remaining four studies included information on the relationship between cognition and employment status as a secondary issue, but were not primarily designed to determine the ecological validity of neuropsychological assessment. Summaries of these ®ve studies and Study Type and Methodology ratings are presented in Table 6. In an early study, Weddell, Oddy, and Jenkins (1980) investigated broad social adjustment for a group of patients with severe closed TBI. Fortyfour patients, who were consecutively admitted to an inpatient rehabilitation facility and met the inclusion criteria, were seen for follow-up assessment at approximately 2 years postinjury. Inclusion criteria limited the sample to patients aged 16±39 with greater than 7 days of PTA. Semistructured interviews regarding social recovery were conducted with relatives of the patients, and patients completed a neurological evaluation and the Ravens Standard Progressive Matrices test. While not the primary focus of the study, the investigators found that nonworking patients had signi®cantly poorer scores on the Matrices test than did those in full-time employment. Strengths of this study include its status as an early demonstration of the relationship between neuropsychological test performance and employment outcome, adequate characterization of the subject sample and inclusion criteria, an attempt to account for other relevant predictors (restricting sample by age and injury severity), and employment of appropriate nonparametric statistical procedures. Weaknesses included inadequate description of patients excluded for

171

follow-up, which limits understanding of the representativeness of the study sample, and use of only one neuropsychological measure. Melamed, Stern, Rahmani, Groswasser, and Najenson (1985) studied performance on an experimental neuropsychological task and concurrent employment status for 51 patients with TBI seen for follow-up evaluation 2±3 years after discharge from a rehabilitation hospital. Patients were administered a psychiatric examination, dual task performance testing (simultaneous performance of rotary pursuit and a delayed digit recall task), and an occupational interview. Nine patients who were unemployed and characterized as ``living a passive life'' were unable to perform the dual task test, and were not included in the overall analyses. One-way analyses of variance with employment status as a between-subjects variable found signi®cant differences among the groups on the rotary pursuit task across all time periods assessed. Patients working in the open market performed signi®cantly better than did those in a protected employment setting, who performed better than unemployed but active individuals. Weaknesses of the study include inadequate description of subject inclusion criteria and limited characterization of the subject sample (such as lack of information regarding injury severity and vague information about the timing of inpatient rehabilitation). These weaknesses limit the generalizability of the ®ndings to other individuals with TBI. Furthermore, the study failed to adjust or account for other possible relevant predictors of employment outcome, included only one experimental measure of neuropsychological functioning, and provided limited information regarding statistical analyses performed. Brooks et al. (1987) studied a sample of 134 persons with severe TBI who had been admitted to a neurosurgical unit in Glasgow and were available for follow-up between 2 and 7 years following injury. Subjects were characterized as having sustained severe injury if they had a coma duration of 6 or more hours, posttraumatic amnesia of at least 2 hours, or had undergone surgery for evacuation of an intracerebral hematoma. No description was given of the subjects who were lost to follow-up, and it is not clear whether the

172

Table 6. Studies of Concurrent Neuropsychological Assessment and Employment Outcome. Subjects

Timing of cognitive and outcome assessments

Weddell et al., 1980

2 years ( 3 months) 44 patients (16 full-time employed; 28 < full-time postinjury. employed) with severe TBI who had received inpatient rehabilitation.

Melamed et al., 1985

51 patients with TBI of unknown severity who were seen for inpatient rehabilitation.

Brooks et al., 1987

2±7 years postinjury. 98 patients with severe TBI who were admitted to a trauma center, all of whom were working at least part-time prior to injury.

Bayless et al., 1989

50 patients (25 employed, 25 unemployed) with nonpenetrating TBI of unknown severity who had been medically cleared for RTW, and 25 noninjured control subjects.

2±6 years postinjury (Median 4 years).

At least 2 years after being medically released to work.

Findings

Study type ratings

Methodology ratings

R5 ± III Performance on a single R6 ± III neuropsychological test was signi®cantly poorer in nonworking Final ± III patients as compared to patients with full-time employment.


Dual task performance differentiated between patients working in the open market, patients working in protected settings, and unemployed but active patients. Performance on the dual task for patients working in the open market resembled that of normal controls.



6 of 9 neuropsychological measures were signi®cantly poorer for patients who failed to return to work postinjury. 2 of these 6 measures were signi®cantly predictive of return to work.

R5 ± II R6 ± III Final ± II

R5 ± Acceptable R6 ± Marginal Final ± Marginal

R5 ± V Performance on a single R6 ± V neuropsychological test was signi®cantly poorer in unemployed Final ± V head-injured patients as compared to both employed head-injured patients and noninjured controls.

R5 ± Flawed R6 ± Flawed Final ± Flawed

MARK SHERER ET AL.

Study

Table 6. (Continued). Hanlon et al., 1999

100 patients with mild TBI who were consecutive referrals to a concussion clinic.

While age and acute injury predictors were not signi®cantly predictive, performance on two memory measures was associated with increased probability of good vocational outcome. However, neuropsychological test measures did not show an incremental advantage over acute injury variables in predicting poor vocational outcome.

R5 ± III R6 ± V Final ± IV

R5 ± Acceptable R6 ± Marginal Final ± Acceptable



Cognitive assessment at Median 177 days postinjury (3±40 months); Outcome data at 1-year postinjury.

173

174

MARK SHERER ET AL.

134 persons included were consecutive admissions or were merely those available for followup. A subset of 101 subjects who had been employed prior to injury were included in the analyses. The neuropsychological tests administered included Progressive Matrices, Mill Hill Vocabulary Scale, word ¯uency and Token Test, WMS Logical Memory, Buschke Selective Reminding Test, learning of three verbal paired associates, Rey Complex Figure, and Paced Auditory Serial Addition Test (PASAT). Persons were considered to be employed if they were working at least part-time in the week prior to testing. The t tests revealed that compared to unemployed persons, employed persons had signi®cantly higher scores on Progressive Matrices, Logical Memory, Buschke long-term storage, associate learning, Rey Copy, and PASAT. These six scores were then entered into a stepwise linear regression model predicting employment, and only WMS Logical Memory and PASAT remained in the model. While this study provided early evidence that neuropsychological performance can provide information on current employment status, generalizability of the conclusions was limited by unclear sample recruitment procedures and failure to describe subjects lost to follow-up. The study was also limited by not adjusting for other variables that could contribute to employment, such as age and injury severity, as well as by the use of uncrossvalidated stepwise regression with no apparent a priori determination of entry order. While the sample was limited to persons with severe injury, there was still a range within that category, with duration of PTA ranging from 2 days to greater than 28 days. Bayless, Varney, and Roberts (1989) compared 25 persons with TBI who had returned to work with 25 persons who had failed to return and 25 normal controls. The origin of these patients and the recruitment procedures were not described. Injury severity was not detailed, but inclusion criteria of at least 30 min of combined loss of consciousness and posttraumatic amnesia makes it likely that some subjects had experienced mild injury. All subjects had been medically cleared to return to work for at least 2 years prior to testing. The range of time postinjury was not described. Subjects were assessed using the Lezak's Tinker

Toy Test, a putative measure of executive functioning. Employment was de®ned as having been working for at least 75% of the time since injury. Analyses were poorly described. As best can be determined, one-way ANOVA was conducted, revealing that the unemployed TBI group performed worse than the employed TBI group, which performed at the level of the control group. A subsequent ANCOVA was conducted, but the variables used as covariates were not speci®ed. Reportedly, the results did not differ from that of the ANOVA. Conclusions based on this study are seriously limited due to the small sample size, unknown generalizability of the sample based on inadequate description, use of only one neuropsychological measure, and poor description of the statistical analyses. In one of the more impressive studies, Hanlon, Demery, Martinovich, and Kelly (1999) evaluated a sample of 100 persons with mild TBI who were consecutive referrals to an outpatient concussion clinic. Neuropsychological assessment was conducted at 3±40 months postinjury (Median 177 days). The neuropsychological battery included Trail Making Test, WAIS±R Digit Span, WMS±R Logical Memory, WMS±R Visual Reproduction, California Verbal Learning Test, Boston Naming Test, Controlled Oral Word Association, Judgment of Line Orientation, Finger Tapping Test, Grooved Pegboard, Wisconsin Card Sorting Test, and Beck Depression Inventory. A weakness of the study was that employment status was assessed at 1 year postinjury, regardless of when neuropsychological assessment was conducted. Outcome was rated as good (return to work in previous capacity), modi®ed (return in a different capacity), or poor (failure to return to work). Thirty-three percent of the sample was classi®ed as having good vocational outcome, while 22% was classi®ed as having poor outcome. Two hierarchical stepwise logistic regression models were constructed. In the ®rst, employment status was dichotimized as either good or modi®ed/poor, while in the second it was categorized as good/ modi®ed versus poor. Variables were entered in blocks, with demographic variables entered ®rst, followed by injury mechanism and type, then neuropsychological test scores. Only variables that showed a linear relationship to the categorical


vocational outcome variable were included in the model. The method for determining this linear relationship was not described. The variables included in the model were: age; injury mechanism (acceleration/deceleration with head hitting object, acceleration/deceleration with head not hitting object, head struck by falling object); type of injury (motor vehicle collision, fall, assault, motor vehicle-pedestrian, falling object, sports/recreation); four neuropsychological test scores (Logical Memory Immediate; Visual Reproduction Delayed; Judgment of Line Orientation; Trail Making B); and the Beck Depression Inventory. The results showed that scores on Logical Memory Immediate and Visual Reproduction Delayed distinguished between persons with good and modi®ed/poor outcome. Neither age nor injury characteristics showed predictive value. In contrast, none of the neuropsychological test scores distinguished between good/modi®ed and poor outcome. Only mechanism of injury showed predictive value, with those who were hit by a falling object having worse outcome. This study makes a strong case for the value of neuropsychological performance in predicting employment status, after controlling for other relevant variables. The results are only generalizable to those with mild TBI who were referred for evaluation at an outpatient clinic. Since only selected persons with mild TBI are typically referred for assessment, the generalizability to the larger population of persons with mild TBI is limited. However, it is noteworthy that the authors checked for differences in vocational outcome between persons involved in litigation and those not involved in litigation. No difference in vocational outcome was found. DISCUSSION AND RECOMMENDATIONS Our review of these articles reveals that the majority have signi®cant weaknesses that limit the conclusions that can be drawn. The most common weaknesses were small sample size, an excessive number of analyses (or inappropriate analyses) given the sample size, inadequate characterization of the study sample, failure to describe subjects lost to follow-up, and failure to

175

adjust neuropsychological test scores for other predictors of employment outcome. In some cases, these apparent weaknesses may result from different conventions for reporting research ®ndings that were used in the earlier years when several of the investigations were conducted. We often found that characterization of the sample was inadequate to allow the reader to determine to which population the ®ndings might generalize. In many cases, the possible association of neuropsychological test scores with employment outcome was only an incidental aspect of a study conducted for some other purpose. This was frequently the case for studies involving late neuropsychological assessment with subsequent assessment of employment outcome and concurrent neuropsychological assessment and employment outcome. In these studies, less care may have been given to describing analyses of neuropsychological assessment and employment outcome as these were not the primary focus of the article. Finally, small sample sizes may be unavoidable in many instances when examining the predictive value of neuropsychological assessment distant from the time of injury since recruitment of subjects is based on participation in postacute programs that work with only a few patients at a time. There is a strong need for larger sample studies, such as with data from the TBI Model Systems, that examine the value of neuropsychological evaluation completed a year or more after injury. Our review of studies of early neuropsychological assessment and late employment outcome revealed three level II studies with adequate methodology that found that early neuropsychological ®ndings were predictive of employment outcome. There were no level II studies or studies with acceptable methodology that produced negative ®ndings. Accordingly, a Category A (strongly supported) recommendation is given for the use of early neuropsychological assessment to predict late employment outcome. These studies exhibited acceptable methodology that allows generalization of the results, in part, due to the larger sample sizes involved. Generation of larger sample sizes was possible because of the early identi®cation of patients with TBI, often while still involved in acute care or rehabilitation. These

176

MARK SHERER ET AL.

studies accounted for other predictors of employment outcome such as initial severity of injury, indicating that neuropsychological assessment makes a contribution to outcome prediction that is not redundant to that made by medical indices of injury severity. Our review of studies of late neuropsychological assessment and subsequent employment outcome, and studies of concurrent neuropsychological assessment and employment outcome revealed no basis for support or contraindication for neuropsychological assessment to predict employment outcome. Almost all studies in these groups had serious limitations with regard to study type or adequacy of methodology. Accordingly, a Category D (insuf®ciently or inconclusively supported) recommendation is given for the ability of late neuropsychological assessment to predict subsequent employment outcome, and for the concurrent validity of neuropsychological assessment in predicting employment outcome. As was noted above, almost no studies have been conducted with investigation of these possible relationships as a primary goal. Additional investigation is needed in these areas. However, there is no conceptual basis for believing that neuropsychological ®ndings obtained closer in time to assessment of employment outcome should be less predictive of this outcome than neuropsychological ®ndings obtained at an earlier time. Even if subsequent studies do not indicate that neuropsychological assessment facilitates prediction of employment in studies of these types, neuropsychological assessment of patients with TBI in the postacute period may still be justi®ed for other reasons such as feedback to the patient and family and guidance for therapies. In spite of the weaknesses of many of the studies reviewed, we believe that the present review provides strong support for the relationship of neuropsychological test results to employment outcome after TBI. Employment after TBI is a complex outcome that is in¯uenced by a number of injury-related and noninjury-related factors. Previous studies have shown that preinjury education level, preinjury employment, accuracy of self-awareness, and preinjury alcohol use are all important factors in determining employment outcome. Other, as yet understudied factors,

such as family support, access to public transportation, access to appropriate postacute rehabilitation services, and general economic conditions are also likely to be signi®cant factors in employment outcome. Given the complexity of these relationships, positive ®ndings relating early neuropsychological assessment to employment outcome are quite impressive. Our ®ndings support neuropsychological assessment as a routine practice for persons with TBI. In view of the wide range of neuropsychological measures used in various studies and the minimal overlap in measures between studies, the present authors did not attempt to determine which neuropsychological measures may be most predictive of employment outcome. This question is worthy of additional investigation, but such an investigation would require a very large sample so that adjustments could be made for the substantial intercorrelations among neuropsychological test scores. Additional investigation is needed to determine the predictive ability of neuropsychological assessment for other outcomes such as school success, ability to manage personal ®nances, ability to safely live independently, and others. The relative lack of studies of these outcomes at this time likely re¯ects the greater dif®culty of valid and reliable measurement of these outcomes as compared to employment outcome. The authors are sensitive to the weaknesses of this review. While interrater agreement for our study type was excellent, agreement for adequacy of methodology ratings was only fair to good. Other reviewers may have developed different criteria for commendable, adequate, marginal, and ¯awed methodology. In view of recent ®ndings that effort has substantial impact on neuropsychological test performance in persons seeking compensation after reported brain injuries (Green, Rohling, Lees-Haley, & Allen, 2001), it is unfortunate that only one of the reviewed studies reported on whether subjects were in litigation or seeking compensation for disability due to their injuries. This study (Hanlon, Demery, Martinovich, & Kelly, 1999) failed to ®nd an effect of litigation status on vocational outcome. We would note that it seems likely that effort may be less of a concern in studies of consecutive TBI cases or studies of patients who are voluntarily participat-


ing in treatment as compared to persons who were referred for evaluations as part of their litigation or compensation seeking. Further, we would expect that poor effort among some subjects (particularly mildly impaired subjects) would diminish the relationship between neuropsychological test results and employment outcome. To the extent that poor effort may have affected the scores of subjects in the reviewed studies, the ®ndings may represent a conservative estimate of the true relevance of valid neuropsychological ®ndings for employment outcome. The process of writing this review served to remind all the authors of the need for more reliable and valid procedures for reviewing articles as part of critical reviews, as was done in the current paper, or for consideration for publication in scholarly journals.

ACKNOWLEDGMENTS This project was partially supported by funding from the National Institute on Disability and Rehabilitation Research TBI Model Systems program (Grant numbers: H133A980035 (Methodist Rehabilitation Center), H133A980010 (University of Alabama-Birmingham), and H133A970015 (The Institute for Rehabilitation and Research).

REFERENCES Bayless, J.D., Varney, N.R., & Roberts, R.J. (1989). Tinker toy test performance and vocational outcome in patients with closed-head injuries. Journal of Clinical and Experimental Neuropsychology, 11, 913±917. Boake, C., Millis, S.R., High, W.M., Jr., Delmonico, R.L., Kreutzer, J.S., Rosenthal, M., Sherer, M., & Ivanhoe, C.B. (2001). Using early neuropsychological testing to predict long-term productivity outcome from traumatic brain injury. Archives of Physical Medicine and Rehabilitation, 82, 761± 768. Brooks, N., Campsie, L., Symington, C., Beattie, A., & McKinlay, W. (1986). The ®ve-year outcome of severe blunt head injury: A relative's view. Journal of Neurology, Neurosurgery, and Psychiatry, 49, 764±770. Brooks, N., McKinlay, W., Symington, C., Beattie, A., & Campsie, L. (1987). Return to work within the

177

®rst seven years of severe head injury. Brain Injury, 1, 5±19. Burke, W.H., Wesolowski, M.D., & Guth, M.L. (1988). Comprehensive head injury rehabilitation: An outcome evaluation. Brain Injury, 2, 313±322. Cifu, D.X., Keyser-Marcus, L., Lopez, E., Wehman, P., Kreutzer, J.S., Englander, J., & High, W. (1997). Acute predictors of successful return to work 1 year after traumatic brain injury: A multicenter analysis. Archives of Physical Medicine and Rehabilitation, 78, 125±131. Dikmen, S.S., Temkin, N.R., Machamer, J.E., Holubkov, A.L., Fraser, R.T., & Winn, R. (1994). Employment following traumatic head injuries. Archives of Neurology, 51, 177±186. Ezrachi, O., Ben-Yishay, Y., Kay, T., Diller, L., & Rattok, J. (1991). Predicting employment in traumatic brain injury following neuropsychological rehabilitation. Journal of Head Trauma Rehabilitation, 6, 71±84. Fabiano, R.J., & Crewe, N. (1995). Variables associated with employment following severe traumatic brain injury. Rehabilitation Psychology, 40, 223±231. Fleming, J., Tooth, L., Hassell, M., & Chan, W. (1999). Prediction of community integration and vocational outcome 2±5 years after traumatic brain injury rehabilitation in Australia. Brain Injury, 13, 417± 431. Fraser, R., Dikmen, S., McLean, A., & Miller, B. (1988). Employability of head injury survivors: First year postinjury. Rehabilitation Counseling Bulletin, 31, 276±288. Green, P., Rohling, M.L., Lees-Haley, P.R., & Allen, L.M. (2001). Effort has a greater effect on test scores than severe brain injury in compensation claimants. Brain Injury, 12, 1045±1060. Hanlon, R.E., Demery, J.A., Martinovich, Z., & Kelly, J.P. (1999). Effects of acute injury characteristics on neurophysical status and vocational outcome following mild traumatic brain injury. Brain Injury, 13, 873±887. Hart, T., & Hayden, M.E. (1986). The ecological validity of neuropsychological assessment and remediation. In B.P. Uzzell & Y. Gross (Eds.), Clinical neuropsychology of intervention (pp. 21± 50). Boston: Martinus-Nijhoff. Heaton, R.K., Barth, J.T., Crosson, B.A., Larrabee, G.J., & Reynolds, C.R. (2002). Request for review papers on empirical support for areas of neuropsychological practice. Newsletter 40 (Newsletter of the Division of Clinical Neuropsychology of the American Psychological Association), 20, 36±41. Heaton, R.K., & Pendleton, M.G. (1981). Use of neuropsychological tests to predict adult patients' everyday functioning. Journal of Clinical and Consulting Psychology, 49, 807±821.

178

MARK SHERER ET AL.

Horn, L.J., & Sherer, M. (1999). Rehabilitation of traumatic brain injury. In M. Grabois, S.J. Garrison, K.A. Hart, & L.D. Lehmkuhl (Eds.), Physical medicine and rehabilitation: The complete approach (pp. 1281±1304). Cambridge, MA: Blackwell Science. Isaki, E., & Turkstra, L. (2000). Communication abilities and work re-entry following traumatic brain injury. Brain Injury, 14, 441±453. Lam, C.S., Priddy, D.A., & Johnson, P. (1991). Neuropsychological indicators of employability following traumatic brain injury. Rehabilitation Counseling Bulletin, 35, 68±74. Landis, J.R., & Koch, G.G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159±174. Malec, J.F., Smigielski, J.S., DePompolo, R.W., & Thompson, J.M. (1993). Outcome evaluation and prediction in a comprehensive-integrated post-acute outpatient brain injury rehabilitation programme. Brain Injury, 7, 15±29. Melamed, S., Stern, M., Rahmani, L., Groswasser, Z., & Najenson, T. (1985). Attention capacity limitation, psychiatric parameters and their impact on work involvement following brain injury. Scandinavian Journal of Rehabilitative Medicine, 12, 21±26. Najenson, T., Groswasser, Z., Mendelson, L., & Hackett, P. (1980). Rehabilitation outcome of brain damaged patients after severe head injury. International Rehabilitative Medicine, 2, 17±22. O'Connell, M.J. (2000). Prediction of return to work following traumatic brain injury: Intellectual, memory, and demographic variables. Rehabilitation Psychology, 45, 212±217. Prigatano, G.P. (1999). Principles of neuropsychological rehabilitation. New York: Oxford University Press. Prigatano, G.P., Fordyce, D.J., Zeiner, H.K., Roueche, J.R., Pepping, M., & Wood, B.C. (1984). Neuropsychological rehabilitation after closed head injury in young adults. Journal of Neurology, Neurosurgery, and Psychiatry, 47, 505±513. Ruff, R.M., Marshall, L.F., Crouch, J., Klauber, M.R., Levin, H.S., Barth, J.T., Kreutzer, J.S., Blunt, B.A., Foulkes, M.A., Eisenberg, H.M., Jane, J.A., & Marmarou, A. (1993). Predictors of outcome

following severe head trauma: Follow-up data from the Traumatic Coma Data Bank. Brain Injury, 7, 101±111. Ryan, T.V., Sautter, S.W., Capps, C.F., Meneese, W., & Barth, J.T. (1992). Utilizing neuropsychological measures to predict vocational outcome in a head trauma population. Brain Injury, 6, 175±182. Sbordone, R.J. (1996). Ecological validity: Some critical issues for the neuropsychologist. In R.J. Sbordone & C.J. Long (Eds.), Ecological validity of neuropsychological testing (pp. 15±41). New York: St. Lucie Press. Sbordone, R.J., & Long, C.J. (Eds.). (1996). Ecological validity of neuropsychological testing. New York: St. Lucie Press. Sherer, M., Madison, C.F., & Hannay, H.J. (2000). A review of outcome after moderate and severe closed head injury with an introduction to life care planning. Journal of Head Trauma Rehabililitation, 15, 767±782. Sherer, M., & Novack, T.A. (in press). Neuropsychological assessment after traumatic brain injury in adults. In G. Prigatano, C.M. Cullum, & N. Pliskin (Eds.), Demonstrating utility and cost-effectiveness in clinical neuropsychology. Philadelphia: Psychology Press. Sherer, M., Sander, A.M., Nick, T.G., High, W.M., Jr., Malec, J.F., & Rosenthal, M. (2002). Early cognitive status and productivity outcome following traumatic brain injury: Findings from the TBI Model Systems. Archives of Physical Medicine and Rehabilitation, 83, 183±192. Silver, C.H. (2000). Ecological validity of neuropsychological assessment in childhood traumatic brain injury. Journal of Head Trauma Rehabililitation, 15, 973±988. Vilkki, J., Ahola, K., Holst, P., Ohman, J., Servo, A., & Heiskanen, O. (1994). Prediction of psychosocial recovery after head injury with cognitive tests and neurobehavioral ratings. Journal of Clinical and Experimental Neuropsychology, 16, 325±338. Weddell, R., Oddy, M., & Jenkins, D. (1980). Social adjustment after rehabilitation: A two-year followup of patients with severe head injury. Psychological Medicine, 10, 257±263.