Developmental Neuropsychology Predictive Accuracy ...

This article was downloaded by: [University of Calgary] On: 31 August 2015, At: 20:16 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: 5 Howick Place, London, SW1P 1WG

Developmental Neuropsychology Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/hdvn20

Predictive Accuracy of the Wide Range Assessment of Memory and Learning in Children With Attention Deficit Hyperactivity Disorder and Reading Difficulties Deborah Dewey , Bonnie J. Kaplan , Susan G. Crawford & Geoffrey C. Fisher Published online: 08 Jun 2010.

To cite this article: Deborah Dewey , Bonnie J. Kaplan , Susan G. Crawford & Geoffrey C. Fisher (2001) Predictive Accuracy of the Wide Range Assessment of Memory and Learning in Children With Attention Deficit Hyperactivity Disorder and Reading Difficulties, Developmental Neuropsychology, 19:2, 173-189, DOI: 10.1207/ S15326942DN1902_3 To link to this article: http://dx.doi.org/10.1207/S15326942DN1902_3

PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages,

and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content.

Downloaded by [University of Calgary] at 20:16 31 August 2015

This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions


DEVELOPMENTAL NEUROPSYCHOLOGY, 19(2), 173–189 Copyright © 1998, Lawrence Erlbaum Associates, Inc.

Predictive Accuracy of the Wide Range Assessment of Memory and Learning in Children With Attention Deficit Hyperactivity Disorder and Reading Difficulties Deborah Dewey Departments of Pediatrics and Psychology University of Calgary Alberta Children’s Hospital

Bonnie J. Kaplan Department of Pediatrics University of Calgary Alberta Children’s Hospital

Susan G. Crawford Alberta Children’s Hospital

Geoffrey C. Fisher Department of Psychiatry University of Calgary Alberta Children’s Hospital

The predictive accuracy of the Wide Range Assessment of Memory and Learning (WRAML; Sheslow & Adams, 1990) over and above more standardized diagnostic tools in children with attention deficit hyperactivity disorder (ADHD) and reading disabilities (RD) was examined. Fifty-three children with ADHD, 63 with RD, 63 with ADHD–RD, and 112 normal comparison children were administered the WRAML, the Wechsler Intelligence Scale for Children–Third Edition (WISC–III; Wechsler, 1991), the Achenbach (1991) Child Behavior Checklist (CBCL), and the Requests for reprints should be sent to Deborah Dewey, Behavioural Research Unit, Alberta Children’s Hospital, 1820 Richmond Road SW, Calgary, Alberta, T2T 5C7, Canada. E-mail: [email protected]

174

DEWEY, KAPLAN, CRAWFORD, FISHER


Woodcock–Johnson Psycho-Educational Battery–Revised (WJ–R; Woodcock & Johnson, 1989). Results of a series of discriminant function analyses revealed that the academic, intellectual, and behavioral measures could correctly classify 73.1% of children, but the WRAML subtests alone were able to correctly classify only 58.5% of participants. Combining all of the memory, academic, intellectual, and behavioral measures resulted in 77.5% of cases being correctly classified. These results suggest that the use of a measure of memory functioning such as the WRAML did not significantly improve the predictive accuracy of a diagnosis of ADHD, RD, or both over and above more standard diagnostic academic, intellectual, and behavioral measures.

Inattention is one of the major characteristics of attention deficit hyperactivity disorder (ADHD; American Psychiatric Association, 1994). Children who are affected are unable to sustain attention and are more disorganized and distracted than children of the same age (Zentall, Harper, & Stormont-Spurgin, 1993). Memory deficits may be an associated feature of the inattention characteristic. In fact, memory occupies a central place in Barkley’s (1996) recent theoretical model of ADHD. Assessment tools recently developed to evaluate people for the characteristics of ADHD also typically include an item devoted to memory (Brown, 1996). The results of studies that have investigated memory in children with ADHD are, however, inconsistent, and there is insufficient data in the literature to support the idea that memory deficits are a central feature of ADHD. Ott and Lyman (1993) found no memory deficits relative to controls in children with ADHD when they looked at memory for spatial location. On the other hand, children with ADHD were clearly deficient when tested for free recall. A recent study that investigated the performance of children with ADHD on the Wide Range Assessment of Memory and Learning (WRAML; Sheslow & Adams, 1990) found that rote memory measures (Number–Letter Memory and Finger Windows) and a paired associate task between relatively meaningless sounds and shapes (Sound Symbol) discriminated between children with ADHD and normal comparison children (Adams, Sheslow, Robins, Payne, & Wilkinson, 1991). As the memory tasks increased in semantic (Verbal Learning, Story Memory) and spatial (Picture Learning) meaningfulness, however, the performance of the children with ADHD was similar to that of normal comparison children. Research that has investigated the memory abilities of children with learning disabilities (LD) has suggested that they display a more generalized memory deficit; however, whether their performance on memory tasks differs from children with ADHD is still open to debate. Robins (1992) did not find any difference in short-term verbal memory or verbal learning over trials in children with ADHD compared to children with LD. Phelps (1996) reported that children’s scores on the Verbal, Visual, Learning, and General Memory indexes of the WRAML failed to adequately distinguish between children with ADHD and children with LD. Other researchers have, however, reported that impairments in verbal memory did discriminate be-


PREDICTIVE ACCURACY OF THE WRAML

175

tween children with LD and children with ADHD (August & Garfinkel, 1990; Felton & Wood, 1989). Korkman and Pesonen (1994) reported double dissociations in the neuropsychological test profiles of children with ADHD, children with LD, and children with both disorders. The children with ADHD performed relatively well overall but had specific difficulty on tests evaluating inhibition and control. In contrast, the children with LD had difficulties with the auditory analysis of speech, digit span, and storytelling. All children had difficulties with name retrieval. Some of the inconsistent results cited previously could be due to the fact that most studieshavenotdistinguishedbetweenchildrenwithcomorbidADHDandLDandchildren with ADHD without LD (Seidman, Biederman, & Faraone, 1995). A second reason for the inconsistent findings is that the investigators failed to distinguish between initial learning and memory; however, Cahn and Marcotte (1995) demonstrated normal memory of learned material in children with ADHD but impaired initial learning. Kaplan, Dewey, Crawford, and Fisher (1998) recently used the WRAML to evaluate memory skills in three groups of children who met rigorous research criteria for ADHD, reading disability (RD), or ADHD + RD. Results showed no evidence of rapid forgetting of information across a delay in children with ADHD compared to normal comparison children; however, children with verbal skills deficits (either RD alone or RD in combination with ADHD) forgot significantly more information from the stories at delayed recall compared to normal comparison children and children with ADHD. Thus, once children with ADHD encoded information into long-term memory, they appeared to have no difficulties in retaining and retrieving that information. Because children with ADHD performed poorly on the three WRAML subtests that were sensitive to attention and concentration, Kaplan et al. concluded that ADHD was associated with encoding difficulties (probably in large part due to attention problems) but not with difficulties in the long-term retention or retrieval of learned material. Although groups of children with different diagnoses may perform differently on some neuropsychological tests, this does not mean that these tests will be useful for the diagnosis or classification of children for clinical purposes (Elwood, 1993). Given the findings of Kaplan et al. (1998) and Cahn and Marcotte (1995), it would be useful to determine if statistically significant differences in memory skills are in fact clinically useful. In other words, does performance on the WRAML assist in the classification of children as ADHD, RD, or combined ADHD + RD? Trying to distinguish among children with ADHD, RD, and ADHD + RD closely approximates the situation encountered in clinical settings where the differential diagnosis of these children is not always clear cut. It would be beneficial to clinicians if the utility of the WRAML within the context of a full clinical assessment and within the domain of differential diagnosis could be established. No studies, however, have examined the extent to which its use improves the predictive accuracy of a diagnosis of ADHD, RD, or both over and above more standard diagnostic tools.

176



If memory problems are an important component of both ADHD and RD, it may be important for all neuropsychological assessments of children suspected of evidencing these disorders to include a measure of memory functioning in addition to the tests of cognitive ability, achievement, and behavioral measures that are typically included in a standardized diagnostic assessment. Thus, the purpose of this study was to determine if the WRAML would be a useful addition to a neuropsychological assessment of children suspected of having ADHD, RD, or both.

METHOD Participants Children with attention or learning problems, or both, and normal comparison children participated in a comprehensive assessment of memory as well as academic, cognitive, perceptual, and motor skills. A fixed ratio sampling method was employed in several special schools and clinics (e.g., every second or fifth record, depending on the size of the facility) to telephone families and invite them to participate in a study of learning and attention disorders. The normal comparison group was selected by matching every second child in the attention or learning problems, or both groups on the basis of age, sex, and neighborhood school. In total, 179 children identified with attention or learning problems, or both participated in the study and met the criteria for assignment to one of three attention and learning problems groups as follows: • ADHD (n = 53; 46 boys, 7 girls) in which the child met the Diagnostic and Statistical Manual of Mental Disorders (3rd ed., rev. [DSM–III–R]; American Psychiatric Association, 1987) criteria on the parent version of the Diagnostic Interview Schedule for Children (DISC; Costello, Edelbrock, & Costello, 1985). • RD (n = 63; 42 boys, 21 girls) in which the child had evidence of phonological deficits (scored at or below the 24th percentile on Woodcock–Johnson Psycho-Educational Battery– Revised [WJ–R] Word Attack subtest; Woodcock & Johnson, 1989), and scored at or below the 16th percentile on the Wide Range Achievement Test–Revised (WRAT–R) Spelling subtest (Jastak & Wilkinson, 1984) or the WJ–R Spelling subtest, and scored less than 17 on the Auditory Analysis Test (AAT; Rosner & Simon, 1971); or in which the child showed evidence of deficits in basic reading skills (scored at or below the 16th percentile on the WJ–R Basic Reading); or if they showed deficits in reading comprehension (scored at or below the 16th percentile on the WJ–R Reading Comprehension). • ADHD + RD (n = 63; 48 boys, 15 girls) in which the child met both types of criteria.



177

The short form of the Wechsler Intelligence Scale for Children–Third Edition (WISC–III) Vocabulary and Block Design (Wechsler, 1991) was used to estimate Full Scale IQ (FSIQ) from the WISC–Revised (WISC–R) norms (Sattler, 1988). All of these children obtained an estimated FSIQ greater than 75. Information about comorbid LD other than ADHD and RD was not available; however, we did obtain information about conduct disorder and oppositional defiant disorder from the DISC. Significantly more children in the ADHD or ADHD + RD groups had oppositional defiant disorder, c 2(2, N = 179) = 21.44, p < .001, than did children with RD. No significant association between conduct disorder and group membership emerged, c 2(2, N = 179) = 3.44, ns. The normal comparison group included 112 children (82 boys, 30 girls). None of the children in this group met the preceding criteria for ADHD or RD. All children in this group obtained an estimated FSIQ above 75. Although the DISC was not administered to the parents of the normal comparison children, the scores of all of these children on the Child Behavior Checklist (CBCL; Achenbach, 1991) were below the cutoff T score of 60 for the attention subscale (M = 51.49, SD = 2.77). There was a trend toward a significant association between sex and group membership, c 2(3, N = 291) = 6.49, p < .10, with fewer girls in the ADHD group compared to the other three groups. There was a significant group difference for age, F(3, 287) = 3.47, p < .05. The children in the normal comparison group were significantly younger than the children in the ADHD group. No adjustment was made for age in the following analyses because each analysis incorporated standard scores from the particular test in question, all of which had already been adjusted for the participant’s age. Measures A scaled score (i.e., M = 10, SD = 3) was computed for each of the nine subtests of the WRAML (Sheslow & Adams, 1990). The nine subtests fall into one of three indexes. The Verbal Memory index includes the Story Memory, Sentence Memory, and Number–Letter Memory subtests; the Visual Memory index consists of Picture Memory, Design Memory, and Finger Windows; and the Learning index encompasses Verbal Learning, Sound Symbol, and Visual Learning. The three indexes are summed to produce a General Memory index, which supposedly reflects the child’s overall memory abilities. All nine of the subtests involve immediate recall; however, four subtests (Story Memory, Verbal Learning, Sound Symbol, and Visual Learning) also have a delayed recall component in which the child is asked to remember the same material after a delay, during which time other memory tasks are completed. Savings scores, which take into account the rate of forgetting across a delay, can be calculated for these four subtests. Savings scores were calculated for the Story Memory subtest (delayed recall divided by initial recall) as well as for the Sound Symbol, Verbal Learning, and Visual Learning subtests (delayed recall divided by recall on the fourth trial).

178


The Broad Written Language scale of the WJ–R (Woodcock & Johnson, 1989), which includes the Dictation and Writing Samples subtests, was administered according to standardized procedures to all children. Parents were also asked to complete the CBCL (Achenbach, 1991).


Procedure Each child was evaluated individually by testers who were blind to the diagnosis of the child. Participants who were taking stimulant medications (e.g., Ritalin®) were tested while off their medication. The tests were presented in the following order: WRAML, WISC–III, WJ–R, WRAT–R, and AAT. Analyses When examining the predictive accuracy of diagnosis of an assessment instrument, one must take into account its validity. A number of recent investigators (Aylward, Gioia, Verhulst, & Bell, 1995; Burton, Donders, & Mittenberg, 1996; Dewey, Kaplan, & Crawford, 1997; Phelps, 1995) have raised concerns regarding the validity of the factor structure of the WRAML, in particular the validity of the Learning index, and have gone on to suggest that an attention and concentration factor may exist. This reorganization of the WRAML subtests may increase the clinical utility of this instrument, and using an Attention Concentration index may allow clinicians to examine the extent to which memory performance is affected by difficulties attending to the task at hand. Investigators have also reported that the individual subtests of the WRAML had significant amounts of unique variance (Callahan, Haut, Haut, & Franzen, 1993; Gioia, 1991). The WRAML also includes three delayed recall tests that are not factored into the child’s overall memory scores. Therefore, when investigating the predictive accuracy of the WRAML in children with suspected ADHD, RD, or both, it may be more useful and valid to combine the subtests into Verbal, Visual, and Attention Concentration indexes, as opposed to using the standard indexes (i.e., Verbal Memory, Visual Memory, and Learning). The usefulness of all of the individual subtests and the delayed recall tests should also be examined. It is possible that rates of forgetting in populations of children with learning or attention problems, or both, could assist in discriminating among different types of developmental learning or attention problems, or both. For the aforementioned reasons, a series of discriminant function analyses were conducted. The first analysis used the Verbal Memory, Visual Memory, Learning, and General Memory indexes as predictors of group membership, which is consistent with the procedure followed by Phelps (1996); the second analysis used the Verbal (i.e., Story Memory, Verbal Learning, Sound Symbol), Visual (Design Memory, Picture Memory, Visual Learning), and Attention Concentration indexes (i.e., Finger Windows, Sentence Memory, Number–Letter) as predictors; the third


179

analysis used scores from each individual subtest as well as the four savings scores as predictors; the fourth analysis used FSIQ, Broad Written Language from the WJ–R, and the T score for the Attention subscale from the CBCL; and the final analysis used each of the predictors mentioned in the third and in the fourth analysis. For each discriminant function analysis, the assumptions of linearity, normality, and homogeneity of variance–covariance matrices were examined. No outliers or threats to multivariate analysis were found.


RESULTS Comparison of Group Means The means and standard deviations for the WISC–III, WJ–R Broad Written Language, CBCL, and WRAML are given in Table 1. Results of between-group comparisons on the WRAML variables are reported elsewhere (Kaplan et al., 1998). Group differences for reading skills as measured by the WJ–R were not considered because reading subtests from the WJ–R were used as part of the research criteria to define RD in this study. As significant group differences were found for FSIQ (control group was higher), F(3, 287) = 22.12, p < .001, and socioeconomic status (SES; control group was higher), c 2(6, N = 285) = 13.45, p < .05, both of these variables were used as covariates in subsequent between-group comparisons. An analysis of covariance (ANCOVA) on the Broad Written Language scores of the WJ–R with FSIQ and SES as covariates revealed a significant group difference, F(3, 300) = 96.89, p < .001. Post hoc group comparisons using the Scheffe test indicated that the three diagnostic groups scored significantly lower than the normal comparison children, and that the children with RD and ADHD + RD scored significantly lower than the children with ADHD. Group differences on the Attention Problems subscale of the CBCL were also examined using an ANCOVA with FSIQ and SES as covariates. Results revealed a significant group difference, F(3, 226) = 92.14, p < .001. Post hoc comparisons indicated that the three diagnostic groups scored significantly higher than the normal comparison group, and that the RD group scored significantly lower (indicating fewer attention problems) than either the ADHD group or the ADHD + RD group. Discriminant Function Analysis Consistent with Phelps (1996), the first discriminant function analysis used the indexes for Verbal Memory, Visual Memory, Learning, and General Memory as the predictors of group membership. Three discriminant functions were identified with a combined c 2(9, N = 291) = 111.03, p < .001. After removal of the first discriminant function, no significant association remained. As shown in Table 2,


180 TABLE 1 Means and Standard Deviations (Unadjusted for Covariates) ADHDa Variable Age Design Memory Finger Windows Number–Letter Picture Memory Sentence Memory Story Memory Sound Symbol Visual Learning Verbal Learning General Memory index Verbal Memory index Visual Memory index Learning index FSIQ estimate Broad Written Language Attention problems T score (CBCL)

RDb

ADHD + RDc

Controlsd

M

SD

M

SD

M

SD

M

SD

12.37 10.15 8.96 7.85 9.83 8.45 10.58 9.75 9.90 10.92 97.02 93.40 97.74 101.38 104.00 89.12 68.28

2.37 2.65 3.25 2.36 2.56 2.10 2.40 2.58 2.73 2.56 9.95 11.01 12.57 11.48 12.64 11.70 9.21

11.98 8.92 8.79 6.51 9.05 7.00 10.02 6.87 9.60 9.43 87.19 86.16 92.32 90.81 96.46 75.67 54.33

1.94 2.86 2.85 2.57 2.64 2.71 2.37 2.27 3.02 3.01 12.60 12.20 12.19 14.84 11.61 8.87 5.60

11.72 9.79 8.65 6.71 9.89 7.11 9.95 7.56 10.41 10.46 91.37 86.84 96.14 96.57 98.40 77.25 66.46

2.20 3.17 3.00 2.14 2.43 2.46 2.57 2.27 2.68 2.45 11.25 11.20 14.41 11.82 13.47 11.97 8.54

11.27 10.83 11.10 9.15 9.13 9.93 11.03 10.38 11.22 12.00 104.56 100.21 102.46 108.18 110.74 104.12 51.16

2.21 2.88 2.69 2.37 2.41 2.57 2.58 3.04 2.57 2.67 11.19 10.59 12.43 14.17 12.75 9.64 2.53

Note. ADHD = attention deficit hyperactivity disorder; RD = reading disability; FSIQ = Full Scale IQ; CBCL = Child Behavior Checklist (Achenbach, 1991). an = 53. bn = 63. cn = 63. dn = 112.



181

49.5% of the total cases were correctly classified. The correct classification rates for the ADHD, RD, ADHD + RD, and normal comparison were 24.5%, 52.4%, 36.5%, and 67%, respectively. The General Memory, Verbal Memory, and Learning indexes contributed most to the first discriminant function. Children in the ADHD group were most commonly misclassified as normal comparison children; children in the RD group were most commonly misclassified as ADHD + RD; children in the ADHD + RD group were most commonly misclassified as RD; and the normal comparison children were most commonly misclassified as ADHD. The Verbal, Visual, and Attention Concentration indexes were used as predictors of group membership in the second discriminant function analysis (see Table 3). Three functions were identified with a combined c 2(9, N = 290) = 142.58, p < .001. After the first discriminant function was removed, a trend toward an association between groups and predictors remained; however, no significant association remained once the first two functions were removed. In total, 51% of the cases were correctly classified: 35.8% of the ADHD group, 51.6% of the RD group, 34.9% of the ADHD + RD group, and 67% of the normal comparison group. The Attention Concentration index and the Verbal Memory index contributed most to the first discriminant function. Children in the ADHD group were most commonly misclassified as having ADHD + RD or as being in the normal comparison group. Children with RD were most comTABLE 2 WRAML Memory and Learning Indexes: Discriminant Function Analysis Dependent Variables WRAML General Memory index WRAML Verbal Memory index WRAML Learning index WRAML Visual Memory index Percentage of variance accounted for

Function 1

Function 2

Function 3

.934 .828 .759 .450 96.2

.244 –.527 .555 .470 3.0

–.002 .190 –.257 .268 0.7

Note. Percentage of grouped cases correctly classified = 49.5%. WRAML = Wide Range Assessment of Memory and Learning (Sheslow & Adams, 1990). TABLE 3 WRAML Attention Concentration Index, Verbal Memory Index, and Visual Memory Index: Discriminant Function Analysis Dependent Variables WRAML Attention Concentration index WRAML Verbal Memory index WRAML Nonverbal Memory index Percentage of variance accounted for

Function 1

Function 2

Function 3

.851 .654 .243 94.9

–.311 .627 .803 3.4

.424 –.423 .545 1.8

Note. Percentage of grouped cases correctly classified = 51.0%. WRAML = Wide Range Assessment of Memory and Learning (Sheslow & Adams, 1990).


182


monly misclassified as having ADHD + RD, whereas children with ADHD + RD were most commonly misclassified as having RD. Normal comparison children were most commonly misclassified as having ADHD. The third discriminant function analysis incorporated the nine subscales of the WRAML and the four computed savings scores as the predictors of group membership (see Table 4). Three discriminant functions were identified with a combined c 2(39, N = 284) = 196.22, p < .001. After removal of the first discriminant function, there was a trend toward a significant association between groups and predictors. Correct classification rates for the ADHD, RD, ADHD + RD, and the normal comparison participants were 57.7%, 45.9%, 49.2%, and 70.9%, respectively, with the overall correct classification rate being 58.5%. Variables contributing significantly to the first discriminant function were Sentence Memory, Sound Symbol, Number–Letter, and Story Memory savings scores. Children in the ADHD group were most often misclassified as members of the normal comparison group. Children in the RD group were most often misclassified as members of the ADHD + RD group, whereas children in the ADHD + RD group were most often misclassified as members of the RD group. Finally, children in the normal comparison group were most often misclassified as belonging to the ADHD group. The fourth discriminant function analysis incorporated the child’s FSIQ, the child’s achievement on Broad Written Language, and the child’s T score on the Attention Problems subscale of the CBCL (see Table 5). Three discriminant functions were identified with a combined c 2(9, N = 234) = 385.67, p < .001. Once the first function was removed, a significant association among groups remained. TABLE 4 WRAML Subtests and Savings Scores: Discriminant Function Analysis Dependent Variables

Function 1

WRAML Sound Symbol WRAML Sentence Memory WRAML Number–Letter WRAML Story Memory savings score WRAML Story Memory WRAML Finger Windows WRAML Visual Learning WRAML Verbal Learning WRAML Picture Memory WRAML Visual Learning savings score WRAML Design Memory WRAML Verbal Learning savings score WRAML Sound Symbol savings score Percentage of variance accounted for

.603 .574 .526 .376 .199 .397 .196 .378 –.074 .048 .237 –.028 .113 85.6

Function 2 .333 –.136 –.108 .304 .031 –.548 –.464 –.107 .298 .227 –.060 –.061 .133 8.3

Function 3 –.339 –.049 –.107 .314 .090 .070 –.449 –.481 –.479 .439 –.357 .319 –.213 6.1

Note. Percentage of grouped cases correctly classified = 58.5%. WRAML = Wide Range Assessment of Memory and Learning (Sheslow & Adams, 1990).


183

TABLE 5 WISC–III FSIQ, Broad Written Language, and Attention Problems: Discriminant Function Analysis Dependent Variables


Broad Written Language T score for attention problems (CBCL) WISC–III FSIQ estimate Percentage of variance accounted for

Function 1 .765 –.658 .283 76.6

Function 2 .627 .750 .291 23.3

Function 3 –.147 –.072 .914 0.1

Note. Percentage of grouped cases corrected classified = 73.1%. WISC–III = Wechsler Intelligence Scale for Children–Third Edition (Wechsler, 1991); FSIQ = Full Scale IQ; CBCL = Child Behavior Checklist (Achenbach, 1991).

When the first two functions were removed, no significant association was found. Broad Written Language contributed the most heavily to the first discriminant function; the T score for Attention Problems contributed most heavily to the second discriminant function. The correct overall classification rate was 73.1%. For the ADHD, RD, ADHD + RD, and normal comparison children, the correct classification rates were 63.4%, 75.9%, 50%, and 90.6%, respectively. Children with ADHD were most often misclassified as having ADHD + RD. Children with RD were most often misclassified as having both RD and ADHD, whereas children with ADHD + RD were most often misclassified as having only ADHD. Normal comparison children were most commonly misclassified as having RD. To determine if the inclusion of scores on the WRAML would assist in discriminating among these groups above and beyond the discrimination provided by a standard assessment of intellectual and academic skills as well as attention problems, the children’s scores on the WISC–III, WJ–R Broad Written Language, CBCL Attention Problems subscale, as well as the WRAML subtests and savings scores were included in a final discriminant function analysis (see Table 6). Reading scores from the WJ–R were not used for this analysis because these scores had been used to define RD. Three functions were identified with a combined c 2(48, N = 227) = 412.73, p < .001. A significant association was found among groups after removal of the first function; however, no significant association was found after the first and second functions were removed. This analysis resulted in 77.5% of the children being correctly classified: 72.5% of the ADHD, 75% of the RD, 61.5% of the ADHD + RD, and 91.6% of the normal comparison children. Children with ADHD were most commonly misclassified as having RD or ADHD + RD; children with RD were most often misclassified as having ADHD + RD; children with ADHD + RD were most commonly misclassified as having solely RD; and normal comparison children were most often misclassified as having RD. The variables that contributed most significantly to the first function were Broad Written Language and Sentence Memory, whereas the T score for Attention Problems and

184


TABLE 6 WRAML, WISC–III, Broad Written Language, and Attention Problems: Discriminant Function Analysis


Dependent Variables WJ–R Broad Written Language WRAML Sentence Memory WRAML Number–Letter Attention Problems T score (CBCL) WRAML Sound Symbol WRAML Picture Memory WRAML Story Memory WRAML Visual Learning WRAML Finger Windows WRAML Visual Learning savings score WRAML Story Memory savings score WRAML Verbal Learning WISC–R FSIQ estimate WRAML Design Memory WRAML Sound Symbol savings score WRAML Verbal Memory savings score Percentage of variance accounted for

Function 1

Function 2

Function 3

.715 .303 .255 –.607 .255 –.084 .098 .116 .246 .064 .140 .150 .263 .099 .028 .002 75.6

.546 .245 .205 .697 .420 .147 .103 .124 .027 .019 .192 .232 .246 .176 .098 –.050 22.6

–.089 –.145 –.173 –.073 .140 –.099 .030 –.564 –.503 .470 .402 –.386 –.306 –.254 .164 .094 1.8

Note. Percentage of grouped cases correctly classified = 77.5%. WRAML = Wide Range Assessment of Memory and Learning (Sheslow & Adams, 1990); WISC–III = Wechsler Intelligence Scale for Children–Third Edition (Wechsler, 1991); WJ–R = Woodcock–Johnson Psycho-Educational Battery–Revised (Woodcock & Johnson, 1989); CBCL = Child Behavior Checklist (Achenbach, 1991); WISC–R = Wechsler Intelligence Scale for Children–Revised norms (Sattler, 1988); FSIQ = Full Scale IQ.

Sound Symbol contributed significantly to the second discriminant function (see Table 6). Summary In summary, this series of discriminant function analyses showed that using the Verbal, Visual, Learning, and General Memory indexes of the WRAML resulted in only 49.5% of the children in the various groups being correctly classified. Examining the performance of these children on the Verbal, Visual, and Attention Concentration indexes did not substantially improve the clinical utility of the scale because only 51% of the cases were correctly classified. The overall correct classification rate was 73.1% when standardized cognitive, academic, and behavioral measures were used. This rate increased slightly to 77.5% overall with the addition of the WRAML subtests and savings scores to the battery. Thus, it appears that using the WRAML did not substantially improve predictive accuracy over more standard diagnostic tools.



185

A closer inspection of the classification rates for each of the three diagnostic groups does, however, warrant a slightly different conclusion. For the ADHD group, 57.7% were correctly classified when only the WRAML subtest and savings scores were used in a discriminant function analysis; 63.4% were correctly classified when cognitive, academic, and behavioral measures were used; and 72.5% were correctly classified when all cognitive, academic, behavioral, and memory measures were used to discriminate among the groups. The pattern for children with ADHD + RD was similar to that for children with ADHD: 49.2% were correctly classified using only the WRAML subtest and savings scores; 50% were correctly classified using the cognitive, academic, and behavioral scores; and 61.5% were correctly classified using the entire battery of cognitive, academic, behavioral, and memory measures. Thus, for children with ADHD and ADHD + RD, the addition of a measure of memory functioning (i.e., WRAML) to the standard test battery decreased the number of misclassifications by approximately 10%. For children with RD, however, this was not the case. They were correctly classified 45.9% of the time when only the WRAML subtest and savings scores were used in a discriminant function analysis; 75.9% were correctly classified when cognitive, academic, and behavioral measures were used; and 75% were correctly classified when all cognitive, academic, behavioral, and memory measures were used. Hence, using the WRAML to assess memory functioning in children with RD did not add anything substantial to the discriminating power of the standard test battery.

DISCUSSION It has been suggested that the WRAML may be a useful instrument for evaluating memory impairments and learning problems in children with ADHD and LD. No previous studies, however, have examined the extent to which using the WRAML improves the predictive accuracy of a diagnosis of ADHD, RD, or both over and above more standard diagnostic tools. In this study, we investigated whether the inclusion of the WRAML in a psychometric assessment improved the predictive accuracy of classifying children with ADHD, RD, or both over more standard diagnostic tools that assess cognition, achievement, and behavior. This type of distinction closely approximates the situation encountered in clinical settings where the differential diagnosis of these children is not always clear cut. Further, as in the clinic, comorbidities of childhood developmental disorders may muddy the diagnostic picture. We addressed this, however, by having both “pure” and comorbid groups. Although statistically significant group differences among the children with ADHD, RD, combined ADHD + RD, and children with no attention or learning problems on the WRAML were reported by Kaplan et al. (1998), the findings of


186


the present study indicate that the use of the WRAML does little to improve predictive accuracy over more standard diagnostic tools. When the Verbal, Visual, and Learning indexes of the WRAML were used, only 49.5% of the children in the ADHD, RD, ADHD + RD, and normal comparison groups were correctly classified. Further, even though a number of researchers have suggested that a potentially more useful way of examining the performance of these children on the WRAML may be to use the Verbal, Visual, and Attention Concentration indexes, the results of this study indicated that this reorganization of the WRAML indexes did not improve the clinical utility of the scale (i.e., 51% of the cases were correctly classified). These findings indicate that the indexes of the WRAML are not particularly useful for the differential diagnosis of children with learning or attention problems or both. When we investigated the predictive accuracy of the WRAML subtests and savings scores over and above standardized cognitive, achievement, and behavioral measures, the overall correct classification rate for all four groups rose very little (i.e., from 73.1%–77.5%). When the classification rates for each of the three diagnostic groups were examined individually, the addition of a measure of memory functioning (i.e., WRAML) to the standard test battery decreased the number of misclassifications of children with ADHD and ADHD + RD by approximately 10%. This was not true for children with RD. Whether or not a 10% improvement in the identification rate of children with ADHD is clinically important is open to discussion. These findings do suggest, however, that for children with ADHD or combined ADHD and RD, the inclusion of an assessment of memory skills may assist in more accurate identification. Kaplan et al. (1998) reported that children with only ADHD did not display difficulties on any of the memory tasks compared to the children in the normal comparison group, with the exception of Number–Letter Memory, Sentence Memory, and Finger Windows. The children with ADHD + RD, however, displayed difficulties on the preceeding subtests as well as Sound Symbol, and they had significantly poorer Story Memory savings scores compared to the normal comparison children. Further, the children with ADHD + RD performed significantly poorer on Sound Symbol and had poorer Story Memory savings scores than children with only ADHD. Thus, the performance of children with ADHD on certain subtests of the WRAML may be useful in differentiating between those children with ADHD who display concomitant RD and those who do not. Instead of using the WRAML to increase the predictive accuracy of a diagnosis of ADHD, RD, or both, above a standard clinical assessment, one might ask if a clinician’s time might be better spent in administering tests developed specifically for the assessment of attention, such as the Continuous Performance Test (CPT; Barkley, 1997). Clinicians may also consider administering formal neuropsychological assessment batteries such as the Halstead–Reitan (H–R) or the Luria–Nebraska Neuropsychological Battery (LNNB), which consist of various subtests that assess a



187

broad range of neuropsychological functions. It is thought that if neurobiological factors, particularly frontal lobe dysfunction, contribute to ADHD, then neuropsychological testing would be useful in determining the presence of this disorder. However, there are concerns associated with the use of these approaches to the assessment of ADHD. For example, although the CPT has been proven reliable in discriminating children with ADHD from children without ADHD, its predictive validity has been questioned (Barkley, 1997). In regard to the use of neuropsychological batteries with this population, a review of the literature by Gordon and Barkley (1998) did not establish a basis for supporting the routine administration of these batteries within a typical ADHD evaluation. They reported that no single subtest or combination of subtests within the H–R or LNNB displayed sufficient predictive validity. Thus, at this time, the research evidence is not definitive regarding the optimal assessment for diagnosing children suspected of having ADHD and differentiating them from children with RD. If there are any questions about learning difficulties in a child suspected of having ADHD, it is essential that the evaluation include a comprehensive assessment of cognitive and academic skills to determine if the child is comorbid for ADHD and RD, as this will influence the number and types of interventions that are initiated with the child and family. In summary, the results indicated that the WRAML does little to improve predictive accuracy of diagnosis over standard psychometric measures of cognition, achievement, and behavior for children with problems in learning, attention, or both. Examination of children’s performance on the WRAML, however, may assist in identifying those children with ADHD who also have RD and those who do not. Future research that investigates the usefulness of delayed recall tests of the WRAML is needed to determine if assessing rates of forgetting in populations of children with learning or attention problems or both can assist in discriminating among different types of developmental learning problems, attention problems, or both. Research that investigates whether the WRAML has clinical utility outside the domain of differential diagnosis, such as for predicting or monitoring the child’s response to treatment intervention, is also needed.

ACKNOWLEDGMENT Support for this research was provided by an Alberta Mental Health Research Unit Award and the Alberta Children’s Hospital Foundation.

REFERENCES Achenbach, T. M. (1991). Manual for the Child Behavior Checklist/4–18 and 1991 profile. Burlington: University of Vermont.


188


Adams, W., Sheslow, D., Robins, P., Payne, H., & Wilkinson, G. (1991, August). Memory abilities in children with attention deficit hyperactivity disorder. Paper presented at the meeting of the American Psychological Association, San Francisco, CA. American Psychiatric Association. (1987). Diagnostic and statistical manual of mental disorders (3rd ed., rev.). Washington, DC: Author. American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author. August, G. J., & Garfinkel, B. D. (1990). Comorbidity of ADHD and reading disability among clinic-referred children. Journal of Abnormal Child Psychology, 18, 29–45. Aylward, G. P., Gioia, S. J., Verhulst, S. J., & Bell, S. (1995). Factor structure of the Wide Range Assessment of Memory and Learning in clinical populations. Journal of Psychoeducational Assessment, 13, 132–142. Barkley, R. A. (1996). Attention deficit hyperactivity disorder. In E. J. Mash & R. A. Barkley (Eds.), Child psychopathology (pp. 63–112). New York: Guilford. Barkley, R. A. (1997). Attention-deficit/hyperactivity disorder. In E. J. Mash & L. G. Terdal (Eds.), Assessment of childhood disorders (3rd ed., pp. 71–129). New York: Guilford. Brown, T. E. (1996). Brown Attention-Deficit Disorder scales: The manual. San Antonio, TX: Psychological Corporation. Burton, D. B., Donders, J., & Mittenberg, W. (1996). A structural equation analysis of the Wide Range Assessment of Memory and Learning in the standardization sample. Child Neuropsychology, 2, 39–47. Cahn, D. A., & Marcotte, A. C. (1995). Rates of forgetting in attention deficit hyperactivity disorder. Child Neuropsychology, 1, 158–163. Callahan, T. S., Haut, J. S., Haut, M. W., & Franzen, M. D. (1993, MONTH). Confirmatory factor analysis of the WRAML. Paper presented at the meeting of the American Psychological Association, Toronto, ON. Costello, E. J., Edelbrock, C. S., & Costello, A. J. (1985). Validity of the NIMH Diagnostic Interview Schedule for Children: A comparison between psychiatric and pediatric referrals. Journal of Abnormal Child Psychology, 13, 579–595. Dewey, D., Kaplan, B. J., & Crawford, S. (1997). Factor structure of the WRAML in children with ADHD or reading disabilities: Further evidence of an attention/concentration factor. Developmental Neuropsychology, 13, 501–506. Elwood, R. W. (1993). Clinical discriminations and neuropsychological tests: An appeal to Bayes’ Theorem. The Clinical Neuropsychologist, 7, 224–233. Felton, R. H., & Wood, F. B. (1989). Cognitive deficits in reading disability and attention deficit disorder. Journal of Learning Disabilities, 22, 3–13. Gioia, G. A. (1991, February). Re-analysis of the factor structure of the Wide Range Assessment of Memory and Learning: Implications for clinical interpretation. Paper presented at the meeting of the International Neuropsychological Society, San Antonio, TX. Gordon, M., & Barkley, R. A. (1998). Tests and observational measures. In R. A. Barkley (Ed.), Attention-deficit hyperactivity disorder: A handbook for diagnosis and treatment (2nd ed., pp. 294–311). New York: Guilford. Jastak, S., & Wilkinson, G. S. (1984). The Wide Range Achievement Test–Revised. Wilmington, DE: Jastak Associates. Kaplan, B. J., Dewey, D., Crawford, S., & Fisher, G. (1998). Deficits in long-term memory are not characteristic of ADHD. Journal of Clinical and Experimental Neuropsychology, 20, 518–528. Korkman, M., & Pesonen, A.-E. (1994). A comparison of neuropsychological test profiles of children with attention deficit-hyperactivity disorder and/or learning disorder. Journal of Learning Disabilities, 27, 383–392. Ott, D. A., & Lyman, R. D. (1993). Automatic and effortful memory in children exhibiting attention-deficit hyperactivity disorders. Journal of Clinical Child Psychology, 22, 420–427. Phelps, L. (1995). Exploratory factor analysis of the WRAML with academically at-risk students. Journal of Psychoeducational Assessment, 13, 384–390.



189

Phelps, L. (1996). Discriminative validity of the WRAML with ADHD and LD children. Psychology in the Schools, 33, 5–12. Robins, P. M. (1992). A comparison of behavioral and attentional functioning in children diagnosed as hyperactive or learning-disabled. Journal of Abnormal Child Psychology, 20, 65–82. Rosner, J., & Simon, D. P. (1971). The auditory analysis test: An initial report. Journal of Learning Disabilities, 4, 384–392. Sattler, J. M. (1988). Assessment of children. San Diego, CA: Author. Seidman, L. J., Biederman, J., & Faraone, S. V. (1995). Effects of family history and comorbidity on the neuropsychological performance of children with ADHD: Preliminary findings. Journal of the American Academy of Child and Adolescent Psychiatry, 34, 1015–1024. Sheslow, D., & Adams, W. (1990). Wide Range Assessment of Memory and Learning. Wilmington, DE: Jastak Associates. Wechsler, D. (1991). Manual for the Wechsler Intelligence Scale for Children–Third Edition. New York: Psychological Corporation. Woodcock, R. W., & Johnson, M. B. (1989). Woodcock–Johnson Psycho-Educational Battery–Revised. Allen, TX: DLM. Zentall, S. S., Harper, G. W., & Stormont-Spurgin, M. (1993). Children with hyperactivity and their organizational abilities. Journal of Educational Research, 87, 112–117.