Assessment of effort in children: A systematic review

This article was downloaded by: [The Am Academy of Clinical Neuro ] On: 18 December 2013, At: 04:58 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Child Neuropsychology: A Journal on Normal and Abnormal Development in Childhood and Adolescence Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/ncny20

Assessment of effort in children: A systematic review a

Jonathan DeRight & Dominic A. Carone a

b

Syracuse University, Psychology, Syracuse, NY, USA

b

SUNY Upstate Medical University, Syracuse, NY, USA Published online: 17 Dec 2013.

To cite this article: Jonathan DeRight & Dominic A. Carone , Child Neuropsychology (2013): Assessment of effort in children: A systematic review, Child Neuropsychology: A Journal on Normal and Abnormal Development in Childhood and Adolescence, DOI: 10.1080/09297049.2013.864383 To link to this article: http://dx.doi.org/10.1080/09297049.2013.864383

PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/termsand-conditions

Child Neuropsychology, 2013 http://dx.doi.org/10.1080/09297049.2013.864383

Assessment of effort in children: A systematic review

Downloaded by [The Am Academy of Clinical Neuro ] at 04:58 18 December 2013

Jonathan DeRight1 and Dominic A. Carone2 1

Syracuse University, Psychology, Syracuse, NY, USA SUNY Upstate Medical University, Syracuse, NY, USA

2

The assessment of response validity is now considered an important and necessary component of neuropsychological evaluations. One way for assessing response validity is with performance validity tests (PVTs), which measure the degree of effort applied to testing to achieve optimal performance. Numerous studies have shown that normal and neurologically impaired children are capable of passing certain free-standing PVTs using adult cutoffs. Despite this, PVT use appears to be more common in adults compared to children. The overall purpose of this systematic review is to provide the reader with a general overview of the existing literature on PVTs in children. As part of this review, goals are to inform the reader why PVT use is not as prevalent in children compared to adults, to discuss why PVTs and related methods are important in pediatric cognitive evaluations, and to discuss practical limitations and future directions. Keywords: Symptom validity testing; Effort testing; Malingering; Children; Performance validity testing.

Performance validity testing (PVT) is now regarded as an important and necessary component of neuropsychological evaluations according to the National Academy of Neuropsychology (NAN; Bush et al., 2005) and the American Academy of Clinical Neuropsychology (AACN; Heilbronner et al., 2009). Although the NAN position paper is silent on the use of PVTs with children, the AACN consensus conference statement on PVTs noted that the use of such tests in children is worthy of future scientific investigation. The overall purpose of this systematic review is to provide the reader with a general overview of the existing literature on PVTs in children. As part of this review, goals are to inform the reader why PVT use is not as prevalent in children compared to adults, to discuss why PVTs and related methods are important in pediatric cognitive evaluations, and to discuss practical limitations and future directions. Valid interpretation of neuropsychological test results relies on the assumption that examinees put forth their best effort to do well (Kirkwood, Kirk, Blaha, & Wilson, 2010). Although the term “symptom validity testing” is often used in research and clinical practice, this is actually a broad term that has traditionally referred to assessing the validly of reported symptoms and test performance. However, this term tends to be used mostly Address correspondence to Jonathan DeRight, 430 Huntington Hall, Syracuse University, Syracuse, NY 13244, USA. E-mail: [email protected]

© 2013 Taylor & Francis


2

J. DERIGHT & D. A. CARONE

in reference to the latter (often referred to as effort testing). Some researchers have recently distinguished between performance validity tests (PVTs) and reserve the term “SVTs” for symptom reporting validity measures (Larrabee, 2012; Van Dyke, Millis, Axelrod, & Hanks, 2013). For the purposes of this review, the term “PVT” is used synonymously with effort testing. Effort testing involves the determination of whether an individual is fully engaged in and/or complying with the demands of test-taking in order to do well (Brooks, 2012; Carone, 2008). Scoring below empirical cutoffs on PVTs means that other obtained test results cannot be assumed to be a reliable and valid reflection of the patient’s capabilities. As a result, the “floor-effect” model is used in which extremely low areas of test performance are identified in patients without significant neurological conditions when compared to patients with significant neurological conditions. This is consistent with the approach of analyzing atypical performance patterns on neuropsychological testing to identify poor effort and malingering (Larrabee, 2003). However, scores below PVT cutoffs do not automatically provide the reason for poor effort. This determination relies on an integration of test result findings, behavioral observations, records review, and clinical reasoning. While some patients may not put forth their best effort because of apathy or disinterest (e.g., children compelled by their parents to go to an evaluation that they do not want to attend), sometimes poor effort is due to malingering (i.e., intentional exaggeration motivated by external incentives; American Psychiatric Association, 2000). Boone (2007) discusses literature concerning the notions of “self-deception” and “other-deception” with regard to symptom production in which the former is conceptualized as based on unconscious and involuntary motivation whereas the latter is conceptualized as conscious and voluntary motivation. At the extremes, malingering would involve otherdeception whereas a somatoform disorder would involve self-deception. However, Boone notes that the distinction may not be so simplistic and that some patients may blend otherand self-deception, such that malingering and somatoform disorders can be present in the same patient. Thus, in Boone’s view, conscious and unconscious deception lies on a continuum. While one can theorize about supposed “unconscious” motivations regarding symptom production, clinicians should always be careful not to invent noncredible explanations for noncredible performance on PVTs. For example, theorizing that a chronic mild traumatic brain injury (mTBI) pediatric patient with external gains (e.g., continued absence from school, pursuing extensive academic accommodations) failed a PVT due to impaired memory would ignore the evaluation context and the fact that these tests are extremely easy and less often failed by groups of patients with severe TBIs (Green & Merten, 2013). Reasons for Pediatric PVT Use and Disuse Although national survey data indicate that the majority of neuropsychologists often or always administer effort tests (Sharland & Gfeller, 2007), to our knowledge no data have specifically been published on the frequency of PVT use with children. Although anecdotal, our consultations with colleagues and personal experience reviewing case files of children tested by other psychologists indicate that pediatric PVT use is far less frequent than adult PVT use. The scant use of measures of response validity in clinical and research settings has been noted by others as well (Constantinou & McCaffrey, 2003; Donders, 2005; Greve et al., 2007; Kirk et al., 2011; McCann, 1998). A recent survey of forensic psychologists (Archer, Buffington-Vollum, Stredny, & Handel, 2006) showed that standalone validity measures are often utilized with adults (e.g., Structured Interview of


EFFORT TESTING IN CHILDREN

3

Reported Symptoms [SIRS], Test of Memory Malingering [TOMM; Tombaugh, 1996], Rey 15-Item Test, and Portland Digit Recognition Test) but only nonperformance-based (i.e., symptom reporting) validity measures were discussed for forensic cases involving children and adolescents (e.g., Minnesota Multiphasic Personality Inventory-Adolescent [MMPI-A]). Clinically, our experience is that pediatric PVT use is virtually nonexistent among school psychologists where testing is commonplace for special education placement and continuation decisions. The dearth of PVT use in school settings appears to be due to a lack of awareness of the importance of symptom validity assessment among school and educational psychologists, as evidenced by a near absence of publications on this topic in the major school and educational psychology journals. By contrast, there are hundreds of articles on symptom validity assessment in fewer comparative neuropsychology journals, although the publication rates vary significantly by journal and not much is published on the topic in specific child neuropsychology journals. One reason that psychologists may be less apt to use PVTs with children is because of a false assumption that children are unable to purposely “outsmart” the examination or the examiner due to the complex nature of the assessment or that any such attempts would be easily detected due to an unsophisticated approach (Rohling, 2004; Walker, 2011). However, it has been clearly established that relying on subjective impressions alone to determine the adequacy of patient effort is fraught with limitations and inaccuracies (Guilmette, 2013). Moreover, studies have shown that children possess the ability to engage in deception by the time they are 2- to 3-years-old despite using unsophisticated approaches at that age (Kirkwood et al., 2010). However, the sophistication of children and their ability to deceive improves with age (see the section “Development of Deception in Children”). Another reason that psychologists may be less apt to use PVTs with children is because, while some may acknowledge that children are capable of sophisticated attempts at purposeful deception, it may be assumed that children do not have an incentive toward negative response bias because children are not always referred for neuropsychological evaluations for the same reasons as adults (e.g., to determine vocational disability). While there are different referral reasons for adult and pediatric neuropsychological assessments, potential incentives for negative response bias exist in many pediatric cases, although the child’s awareness of such incentives will differ depending on the case. In some cases, the parent/guardian or family attorney may pressure the child to perform poorly on the examination because of such incentives that can sometimes benefit the parent (Rohling, 2004). In these instances, the child may or may not be aware of the true external incentive to do poorly but may perform poorly because of being told not to try to do well. Examples in which such situations may occur include children who have suffered an established or alleged mTBI resulting in personal injury litigation or other compensation-based claims (e.g., no-fault insurance benefits). In some cases, the compensation-seeking process may have already been initiated by the time the evaluation has begun but in other cases the parent/guardian may wait until the results of the neuropsychological evaluation to form the basis of a claim (and may acknowledge this during interview if asked). Not all forms of compensation seeking require a personal injury in pediatric cases. For example, some parents may apply for disability funds (e.g., Social Security Supplemental Security Income [SSI]) for children from birth to 18 years of age based on the results of a neuropsychological evaluation and records review in children with known or suspected neurodevelopmental disorders. In some cases, other individuals (usually parents) have been noted to encourage the deliberate production of poor


4


performance. This typically occurs in pediatric cases involving compensation seeking or other external incentives and when it occurs it is referred to as “malingering by proxy.” While the concept of assessing for suboptimal performance in neuropsychological testing is relatively new, Kompanje (2007) described a possible case of malingering by proxy in 1593. A more recent example is a case of malingering by proxy in a 9-year-old child during a Social Security Disability (SSD) psychological evaluation by Chafetz and Prentkowski (2011), which was partly detected via egregious PVT failure. In a study by Chafetz (2008), the base rate of PVT failure in children during SSD exams was 27–40%, depending on the PVT used. Even in pediatric mTBI cases (ages 8–17) classified as nonlitigating at the time of the assessment, PVT failure rates have been documented at 17% (Kirkwood & Kirk, 2010). In such cases, children may simply not put forth good effort because they do not want to be at the exam because of oppositional defiant tendencies (particularly with authority figures), boredom, immaturity, attention, and approval (e.g., from parents or health care providers), to avoid perceived task failure, or due to a perceived lack of control such as believing they are incapable of good cognitive performance due to their injury (Constantinou & McCaffrey, 2003; Donders, 2005; Flaro, Green, & Blaskewitz, 2007; Mantynen, Poikkeus, Ahonen, Aro, & Korkman, 2001; Schmitz & Skinner, 1993). While some of these cases would not meet criteria for malingering, PVT failure in such cases reflects that the data obtained are not reliable or valid and can only be said to reflect the child’s minimum level of ability (Bush et al., 2005). In some cases, malingering may still be a possibility even if no financial compensation seeking is present if the child is trying to appear impaired to avoid responsibility. Examples include being excused from school to stay home and to watch television and/or play video games most of the day or removal from a sport that the child does not like but feeling pressure to play from a parent(s). In an increasingly competitive society, parents and children are sometimes motivated to obtain academic accommodations such as extra time on tests for an alleged disability in order to improve test scores or an exemption from a required class (e.g., a foreign language; Rohling, 2004). As a result, such children are often administered a battery of cognitive tests by a school psychologist, occupational therapist, and speech language pathologist prior to a neuropsychological evaluation—all without the use of PVTs. This adds an additional layer of complexity to the assessment of response validity in pediatric neuropsychological evaluations because the examiner also must consider the reliability and validity of prior cognitive testing results when making comparisons to current test results. While some prior examination results may be invalid due to poor effort, others may be invalid due to cheating, as has been documented in instances such as the Atlanta school cheating scandal (Carter, 2013; Copeland, 2013). For adolescents, pharmacological treatment for attention deficits has strong abuse and resale potential, which provides motivation through secondary gain to obtain this medication to use recreationally, as an “educational steroid,” or to sell it for profit (Frazier, Frazier, Busch, Kerwood, & Demaree, 2008; Harrison, 2012; Rabiner, 2013; Rohling, 2004). These factors make it all the more important for developmentally appropriate PVT use in pediatric cognitive evaluations. Development of Deception in Children Numerous single-case studies have provided evidence that children can feign cognitive impairment during neuropsychological examinations (Chafetz & Prentkowski, 2011; Flaro & Boone, 2009; Flaro et al., 2007; Henry, 2005; Heubrock,



5

2001; Kirkwood et al., 2010; Lu & Boone, 2002; McCaffrey & Lynch, 2009). Furthermore, Stouthamer-Loeber (1986) estimated that approximately 19% of normal children engaged in some type of lying or deception according to parent reports. Additionally, Walker (2011) contended that any parent is aware that children have the ability to deceive and that there has been somewhat of a historical naiveté from clinicians regarding the ability of children to malinger illness. Children develop the ability to deceive over time in different stages. Experimental studies suggest that children develop a marked increase in the deployment and understanding of deceptive strategies between the ages of 3 and 5 years (Blaskewitz, Merten, & Kathmann, 2008; Constantinou & McCaffrey, 2003; Johnson, 1997; Lewis, Stanger, & Sullivan, 1989; Nagle, Everhart, Durham, McCammon, & Walker, 2006; Newton, Reddy, & Bull, 2000; Polak & Harris, 1999). Lewis and colleagues studied deception in children by asking 3-year-olds not to look at a toy when left alone in a room and later questioned them about whether they had looked. Their results showed that 29 out of 33 children in the study looked at the toy and 11 children falsely denied looking while another 7 children did not give a response. These findings were replicated in an adapted experimental paradigm by Polak and Harris, who observed that preschoolers often deny misconduct if they did not intend to carry out the action in question but that they are not very effective at sustaining deception by feigning ignorance when questioned about the behavior later (Polak & Harris, 1999). Although children have the capacity to deceive around the age of 2 or 3 years, the ability to intentionally deceive and sustain a false belief generally does not arise until approximately the age of 6 and improves with age due to improved cognitive sophistication, inhibitory control, and theory of mind (Johnson, 1997; Talwar, Gordon, & Lee, 2007). Theory of mind is the ability to attribute mental states to oneself and others and to realize the difference between the two. With regards to deception, the recipient must carefully construe false statements in order to not arouse suspicion and, optimally, will be able to produce verbal and nonverbal actions consistent with the false statement (Talwar et al., 2007). The development of deceptive abilities parallels the gradual development of executive functioning. As executive functioning continues to develop, the multidimensional “skill” that is involved with deception, such as remembering previously divulged information, managing information, and emotional and behavioral control, is improved (Spence, 2004). In the earliest documented study on pediatric malingering on cognitive tests, Faust, Hart, and Guilmette (1988) showed that three children (9 to 12 years old) were able to knowingly alter their neuropsychological test results with minimal coaching in a manner that convinced most practitioners that brain dysfunction was present. Specifically, 93% of the 42 clinicians in this study attributed the children’s poor performance to a diagnosed abnormality (87% of which was attributed to cortical dysfunction) and 0% suspected malingering. Faust, Hart, Guilmette, and Arkes (1988) also provided evidence that three adolescents who purposely performed poorly on a neuropsychological evaluation convinced 78% of the clinicians that they had true impairment, and again 0% detected malingering when no forewarning of malingering was provided to the clinicians. When clinicians were forewarned about the possibility of malingering in the protocols they were reviewing and provided information on the base rates of malingering, the clinicians’ accuracy level did not surpass chance. With the above in mind, this review seeks to identify the frequency of PVT use in pediatric samples in the existing literature, to offer an analysis of the types, indications, and performance on PVTs in those samples, and to provide suggestions for future research in this area.

6


METHODS


Literature Search and Inclusion Criteria An initial literature search was completed using the Pubmed and PsycInfo databases. Book chapters were excluded from analysis because we sought to focus on studies that went through a formal peer review process. The following search criteria were used in both databases: ([pediatric OR children] AND [“symptom validity test*” OR malingering OR “effort test*” OR “validity test*” OR “childhood deception”]). These searches returned 245 initial results in Pubmed and 209 overall results in PsycInfo when the option for “peer reviewed” was selected. The PsycInfo results only yielded one additional unique result that was not in the Pubmed results and fit the inclusion criteria listed below. In addition, reference sections were reviewed to identify additional relevant studies, as suggested by Wright, Brand, Dunn, and Spindler (2007). In order to be included in this review, studies had to meet the following inclusion criteria: (a) an original empirical study (including case studies) from a peer-reviewed journal; (b) use of a formal and/or embedded PVT; (c) included a sample of children (average age less than 18); and (d) primarily focused on the assessment of effort. Using these criteria, 32 studies were included in this systematic review. RESULTS The 32 studies that were included in this review are summarized in Table 1. Studies that met inclusion criteria for this review ranged from year 2002 to 2013. Six out of the 32 studies were experimental in nature (i.e., included experimental and control groups), while the remaining 26 studies were derived from clinical samples. A slight majority (19 out of 32) of the studies were case control studies, in which individuals who failed PVTs were generally compared to those who did not. There were four prospective comparison studies, in which a newer measure of effort was compared to a more established PVT. Finally, there were three randomized controlled trials and six case reports. The most popular sample type was a heterogeneous sample (10 out of 32), consisting of a possible mix of neurological and psychiatric diagnoses. Other sample types included school testing (7 out of 32), purely neurological (5 out of 32), mTBI (5 out of 32), Disability Determinations Service (DDS; 3 out of 32), legal (1 out of 32), and purely experimental (1 out of 32). The average sample size was 113.56 (case reports excluded). The average age (for studies that provided such information) was 11.50 years and ranged from 5 to 19. PVTs were used 63 times across these 32 studies. Eighty-one percent (51) of the PVTs in the review were free-standing measures of effort, with the remaining 19% being embedded measures of effort (12). The most often used PVT used was the Test of Memory Malingering (TOMM; n = 20), followed by the Medical Symptom Validity Test (MSVT; n = 10), the Word Memory Test (WMT; n = 10), Reliable Digit Span (RDS; n = 6), and the Rey Fifteen-Item Test (FIT; n = 4). All remaining tests were only used once and included the Victoria Symptom Validity Test (VSVT), Amsterdam Short Term Memory Test (ASTM), Word Completion Memory Test (WCMT), Nonverbal Medical Symptom Validity Test (NMSVT), composite T-score for List A of the California Verbal Learning Test—Children’s Version (CVLT-C), Computerized Assessment of Response Bias (CARB), Raven’s Standard Progressive Matrices, Dot Counting Test (DCT), Vocabulary subtest minus digit-span subtest (VOC-DS), and the Symptom Validity

Harrison and Armstrong

Carone et al.

Kirkwood et al.

Brooks

Brooks et al.

Harrison et al.

Larochette and Harrison

Loughan et al.

Perna and Loughlan Welsh et al.

Chafetz and Prentkowski

1

2

3

4

5

6

7

8

9 10

11

Author

2011

2013 2012

2012

2012

2012

2012

2012

2013

In press

In press

Year

Case Report

Case Control PC

PC

Case Control

Case Report

Case Control

Case Control

PC

Case Report

Case Control

Study design

DDS

Heterogen Neuro

Heterogen

School

Heterogen

Neuro

Neuro

mTBI

Neuro

School

Sample type

1

75 54

51

63

1 (sub)

100 63 (sub) 53

452

86 79 (sub) 1 (sub)

86

Sample size

(12.2) 6–18 (11.8) 6–18 6–17 (13) 9

11–14

6–19 (14.5) 6–19 (12.4) 17

8–16 (14.7)

15

(12.2)

Age range (mean)

Table 1 A Compilation of Studies that Explicitly Focused on Performance Validity Testing in Children.

SA SA EMB SA SA EMB

SVS RDS

SA

SA SA SA

SA SA EMB EMB SA SA SA

EMB SA SA

EMB

PVT type

TOMM TOMM RDS TOMM

TOMM

WMT VSVT WMT

MSVT TOMM DSS AST VSVT TOMM TOMM

VOC-DS WMT WMT

RDS

PVTs used


0/100 0/100 90.5/9.5

— — —

88/12 90/10 65/35 0/100 0/100 0/100

— — — — — —

(Continued )

86/14

—

—

84/16* 87/13* 87/13* ** 100/0 97/3 94/6

86/14 87/13 100/0

— — — 100/0 100/0 100/0 ** —

99/1

Exp Group % Valid/% Invalid

—

Control % Valid/% Invalid

EFFORT TESTING IN CHILDREN 7

Kirk et al.

Kirkwood et al.

Kirkwood, Yeates, Randolph, and Kirk

Gast and Hart

Gunn, Batchelor, and Jones

Kirkwood and Kirk

Kirkwood et al.

Rienstra et al.

McAllister et al.

Blaskewitz et al.

12

13

14

15

16

17

18

19

20

21

Author

Table 1 (Continued).

2008

2009

2010

2010

2010

2010

2010

2012

2011

2011

Year

RCT***

Case Control

Case Control

Case Report

Case Control

RCT***

Case Control

Case Control

PC

Case Control

Study design

School

Neuro

School

Heterogen

mTBI

School

Legal

mTBI

mTBI

Heterogen

Sample type

70

60

48

6

193

90

107

276

50 (sub)

101

Sample size

6–17 (12.1) 6–11

7–12 (9.9)

MSVT

TOMM WMT ASTM WCMT TOMM

MSVT TOMM RDS WMT FIT

WMT MSVT

(8.7) 8–17 8–16 (13.2)

TOMM

TOMM

MSVT TOMM RDS DSS MSVT

TOMM

PVTs used

(14.2) 12–17 (15.4) 6–11

8–16

5–16 (10.6) (14.2)

Age range (mean)

SA

SA SA SA SA SA

SA SA EMB SA SA

SA SA

SA

SA

SA SA EMB EMB SA

SA

PVT type


100/0 100/0 90/10 65/35 90/10

—

100/0

(N (N (N (N (N

= = = = =

6) 6) 2) 1) 1)

(Continued )

16/84

83/17 83/17 0/100 100/0 0/100

—

—

90/10 83/17

95/5

99/1

82/18 26/74 92/8 96/4 81/19

96/4


98/2 —

98/2

—

100/0 100/0 — — —

—


8 J. DERIGHT & D. A. CARONE

Carone

Chafetz

Chafetz et al.

Flaro et al.

Nagle et al.

Donders

Constantinou and McCaffrey

Courtney et al.

22

23

24

25

26

27

28

29

Author


2003

2003

2005

2006

2007

2007

2008

2008

Year

Case Control

Case Control

Case Control

RCT***

Case Report

Case Control

Case Control

Case Control

Study design

Heterogen

School

Heterogen

Exp

Heterogen

DDS

DDS

Heterogen

Sample type

111

138

100

35

S1: 1 S2: 2 S3: 1

TOMM FIT WMT CARB

(8.1) 6–17 (11.2)

TOMM CVLT-C

TOMM

WMT MSVT NMSVT

SA SA SA

SA

SA EMB

SA

SA SA SA

SA EMB EMB SA EMB EMB SA SA

SA SA EMB SA

TOMM FIT RDS MSVT TOMM A-Test RDS MSVT A-Test RDS TOMM MSVT

PVT type

PVTs used

5–12

6–12 (8.9) 6–16

7–16 (11)

(10.6) (11.5)

(11.5)

S2: 27

S1: 96 S2: 27

(10.6)

(11.8)

S1: 96

38

Sample size

Age range (mean)


40/60 **** **** 52/48 **** **** 72/28 63/37 100/0 0/100 100/0

—

— — —

97/3 100/0 100/0 nr 95/5 88/12

— — —

(Continued )

100/0

100/0

— —

32/68 90/10 10/90 95/5


100/0 100/0 41/59 —


EFFORT TESTING IN CHILDREN 9

McKinzey et al.

Lu and Boone

31

32

2002

2003

2003

Year

Case Report

Case Control

Case Control

Study design

mTBI

School

Heterogen

Sample type

1

44

135

Sample size 7–18 (12.6) 7–17 (12.5) 9

Age range (mean)

FIT DCT

RSPM

WMT

PVTs used

SA SA

EMB

SA

PVT type

17/83 0/100 0/100

— —

86/14


98/2

—


Notes. Percentages were rounded up when necessary and subgroups were combined when possible. AST = automatized sequences task; ASTM = Amsterdam Short Term Memory Test; CARB = Computerized Assessment of Response Bias; CVLT = California Verbal Learning Test; DCT = Dot Counting Test; DDS = Disability Determinations Service; DSS = Digit Span Scaled Score; Emb = Embedded; Exp = Experimental; FS = Free-Standing; Heterogen = Heterogeneous mixtures of affective disorders, mild uncomplicated TBIs, ADHD, LDs, Pervasive Developmental Disabilities, and/or Intellectual Disabilities; MSVT = Medical Symptom Validity Test; mTBI = Mild Traumatic Brain Injury; Neuro = Neurological Referral; NMSVT = Nonverbal Medical Symptom Validity Test; nr = exact failure rate not reported; PC = Prospective comparison; RCT = randomized controlled trial; RSPM = Raven’s Standard Progressive Matrices; Sub = subsample; SVS = Symptom Validity Scale for Low-Functioning Individuals; TOMM = Test of Memory Malingering; VOC-DS = Vocabulary subtest of the WISC-IV minus the Digit Span subtest scaled scores; WCMT = Word Completion Memory Test; WMT = Word Memory Test. *In this study, MSVT failure was used as the gold standard and from there patients who scored below cutoffs of the TOMM and DSS were included. Thus, these classification accuracy scores do not reflect the percent of the total sample who scored below the TOMM and DSS cutoffs. The study also excluded 15 patients who failed the MSVT but passed the TOMM and Digit Span measures in an effort to decrease false positives but this likely resulted in false negatives for the TOMM and DSS cutoffs. **The AST is composed of four times the automated sequencing tasks. The optimal cut scores were as follows: Time to recite the alphabet (≥8 seconds; 50% sensitivity, 91% specificity); Time to count to 20 (≥6 seconds; 50% sensitivity, 92% specificity); Time to recite the days of the week (≥4 seconds; 31% sensitivity, 96% specificity); Time to say the months of the Year (≥10 seconds; 36% sensitivity, 90% specificity); and total time (≥27 seconds; 55% sensitivity, 90% specificity). ***In these studies, individuals were either coached on how to fail a PVT and/or were instructed to feign impairment. ****The A-test and RDS were used in this study but were not presented in a manner that allowed for a determination of the percent who passed or failed.

Green and Flaro

30

Author



10 J. DERIGHT & D. A. CARONE


11

Histogram of PVT Failure rates 18 16 Number of Reports


14 12 10 8 6 4 2 0

0

5

10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 Failure Rate (%) Figure 1 Histogram of PVT failure rates in children.

Note. Each report refers to studies from Table 1, including both experimental and control groups of studies that were included in this systematic review, excluding case reports.

Scale (SVS) for Low-Functioning Individuals. When relevant in the study, control groups performed perfectly to near perfectly in 12 out of 13 PVTs used. The only exception was RDS, in which only 41% of controls earned a passing score (note that this was only done in Blaskewitz et al., 2008). The other 12 PVTs that were passed were all free-standing measures of effort. Excluding case reports, the weighted average PVT failure rate in experimental groups was 15.51%. Figure 1 provides a histogram of the PVT failure rates included in Table 1. As can be seen in Figure 1, the failure rate is relatively skewed toward outliers that are abnormally high failure rates and a majority of the values are below a 15% failure rate. Results of this review show that children are capable of passing free-standing PVTs with adult cutoffs. This is true for control samples, which obtained perfect to near-perfect scores, and experimental groups with moderate-to-severe neurological conditions. The following subsections present a brief synopsis of findings from the most often utilized and validated PVTs that are used with children (TOMM, MSVT, WMT, and RDS).

TOMM The TOMM was the most frequently used PVT and was often passed by healthy children and children with medical or neurological diagnoses (Brooks, Sherman, & Krol, 2012; Nagle et al., 2006; Perna & Loughan, 2013). Children between the ages of 5 and 12 have been shown to be more than 98% accurate on Trial 2 of the TOMM and all but 2 out of 61 children were able to pass the TOMM using adult cutoffs (Constantinou & McCaffrey, 2003). Trial 1 performance on the TOMM is highly predictive of Trial 2 performance in children as in adults (Perna & Loughan, 2013). Nagle et al. showed that children performed comparably to adults on the TOMM, regardless of whether they were told to do their best or told to “fake” brain injury, scoring above 45 on Trials 1 and 2 on average. These authors reported that children appeared to have difficulty “faking”


12


cognitive impairment on the TOMM, as their performance was comparable to adult norms and to their peers “best” attempts. However, the instructions for “faking” in this study, when administered to 6- to 12-year-olds, may have led to the children performing well regardless of the instructional set provided. Specifically, whereas one group was told to perform the best they can, the instructional set for the other group made no specific mention of faking or performing poorly. Instead, the instructional set in the “fake” condition told children to pretend that they were in a car accident, hit their head and woke up with a brain injury and to try and perform the way someone with brain damage would. Although speculative, it is possible that the children may have interpreted this as assuming that patients with brain injury would also try their best, leading to the excellent results on the TOMM. In a study by Rienstra, Spaan, and Schmand (2010), all 48 children scored above the established adult cutoffs on the TOMM and the WMT. MacAllister, Nakhutina, Bender, Karantzoulis, and Carlson (2009) administered the TOMM to 60 children aged 6 to 17 years old who were diagnosed with epilepsy. Six out of the 60 children failed the TOMM (pass rate of 90%) and resulting scores were not correlated with age but were correlated with intelligence estimates (MacAllister et al., 2009). In another study, 107 boys in the juvenile court system and between the ages of 12 and 17 were administered the TOMM and only 2 of the children did not pass using adult cutoffs. In that study, Full Scale IQ only correlated with the retention trial (Gast & Hart, 2010). Kirk et al. (2011) showed that 96% of the children (ages 5–16) from a large clinically referred sample with heterogeneous neurological and/or psychiatric diagnoses were able to pass the TOMM using recommended adult cutoffs on both Trial 2 and the Retention Trial, which is consistent with the results from the Donders study. Donders only found one possible false positive on the TOMM out of a sample of 100 children (ages 6–16) with mixed neurological diagnoses. MSVT As many of the studies in this review demonstrate, the MSVT is a good freestanding measure of effort in children (Kirkwood & Kirk, 2010). Children with diagnosed “clinical” disorders, children tested in a foreign language, and children with a mean Full Scale IQ of 65 have been found to pass the MSVT (Green & Flaro, 2003). Blaskewitz et al. (2008) administered the MSVT to 51 children and all but 1 passed using the standardized effort cutoffs (due to scoring just at the cutoff mark on one of the MSVT effort subtests). Kirkwood and colleagues (2012) provided evidence that the MSVT’s validity indices primarily measure effort and are not affected by ability. In their study, pass or fail performance in pediatric mTBI patients was not affected by demographic, developmental, or neurological factors. MSVT performance was a robust unique predictor of ability-based test performance in nearly all neuropsychological domains, even after controlling for demographic and diagnostic variables such as age, attention deficit/hyperactivity disorder (ADHD), learning disability, special education, and time since injury. The study highlights the powerful mediating effect that effort level has on neuropsychological test results. Carone (2008) evaluated the validity of the MSVT by administering the test to a clinically referred sample of children with a history of moderate-to-severe brain damage, dysfunction, and/or developmental disabilities. Results showed that 95% of children in the study were able to pass the MSVT. The two children who failed had shown behavioral evidence of noncooperativeness and thus were not considered false positives. Also


13

noteworthy, the children in this study performed considerably better on the MSVT and rated the test as much easier than adult patients with mild traumatic brain/head injury.


WMT Green and Flaro (2003) used the WMT with 135 children between the ages of 7 and 18 with heterogeneous neurological and/or psychiatric diagnoses and found that 86% of the participants passed the WMT using adult cutoffs. All 6 children that initially failed the WMT were informed that their results suggested poor effort and they all freely admitted that they had not tried their best. When they were retested and encouraged by a small incentive (a choice of sweets), all but one of them passed, indicating that they had the ability to pass the test but did not pass because of suboptimal effort (the one child who failed twice had oppositional defiant disorder). Recently, Carone, Green, and Drane (in press) reported on the case of a 15-year-old child with epilepsy, surgical removal of the left anterior hippocampus and hippocampal gyrus, and a postoperative stroke who passed the WMT. Courtney, Dinkins, Allen, and Kuroski (2003) found that the use of adult norms for the WMT without regard for a child’s developmental status and other contextual factors (e.g., reading level) appeared ill advised for children under age 11. This is why the WMT (and MSVT) manual now contains explicit instructions for modifying the test administration based on the child’s reading level. Specifically, this involves asking the patient to read each word aloud as it appears on the screen and the examiner immediately to correct any reading errors.

Wechsler Intelligence Scale for Children-IV (WISC-IV) Digit Span and Reliable Digit Span Blaskewitz et al. (2008) showed that a majority of children putting forth full effort (59%) failed an RDS cutoff of ≤7 (which is commonly used with adults) and concluded that RDS results appeared to be closely linked to cognitive abilities in children. Kirkwood, Hargrave, and Kirk (2011) showed that a lower RDS cutoff score of ≤6 yielded a sensitivity of 61% and a specificity of 92% in children with mTBI. However, Welsh, Bender, Whitman, Vasserman, and Macallister (2012) concluded that RDS yielded a large number (35%) of false positives with a cutoff of ≤6, which limits its utility in detecting poor effort as an embedded effort measure in a pediatric epilepsy population and likely other children with a significant cognitive loss, which would not include mTBI. However, Loughan, Perna, and Hertza (2012) reported that a WISC-IV Digit Span scaled score of ≤4 yielded a specificity of 91% and sensitivity of 43% for detecting poor effort in children with heterogeneous dual diagnoses. In a pediatric mTBI sample, a Digit Span scaled score of ≤5 yielded a sensitivity of 51% and a specificity of 96% (Kirkwood et al., 2011). Thus, when using Digit Span as an effort measure in children, these results indicate that the scaled score cutoff is more appropriate than applying adult RDS cutoff to children. However, in a recent study with adolescents, Harrison and Armstrong (in press) found that RDS was insensitive to impairments associated with learning disabilities and that the Digit Span Scaled Score (DSSS) and the difference between Vocabulary minus Digit Span on the WISC-IV were associated with an excessively high false positive rate.

14



DISCUSSION All studies reviewed in Table 1 utilized at least one free-standing PVT based on cutoffs derived from adults. Recognition memory paradigms were often used as a basis for effort testing in children because performance on these measures is relatively resistant to brain damage/dysfunction (Broadbent, Squire, & Clark, 2004). Recognition-memorybased paradigms include those in the verbal domain (e.g., MSVT, WMT) and the visual-spatial domain (e.g., TOMM, VSVT). While there may be occasions when a child specifically puts forth poor effort in a domain other than memory (e.g., slowed speed to gain extra time on tests), the effect of poor effort as measured by recognitionmemory tests has generally been found to have a pervasive influence across all neuropsychological domains (e.g., processing speed, motor functioning, general intellectual functioning, academic achievement) such that scores in those domains are lower in children who fail the recognition-based memory test compared to those who pass (Green, 2007). Harrison, Green, and Flaro (2012) found that that recognition-based memory tests (e.g., WMT, TOMM) were able to effectively identify select cases of a patient seeking accommodations for reading disorders and attention deficit disorder. Because verbal ability (e.g., reading accuracy) is highly variable in children, a visualspatial PVT may be more suitable for younger children (e.g., ages 5–9) unless special care is taken to reduce the possibility of verbal confounds on verbally based PVTs. Our review also showed that nine studies (28%) utilized embedded measures with adult-based cutoffs. Embedded indicators of effort are useful because they do not increase assessment time, are relatively resistant to coaching and allow for monitoring of effort in tests that were not originally designed to assess symptom validity (Kirkwood, Connery, Kirk, & Baker, 2013). For example, RDS is one of the few embedded measures investigated in both children and adults because the Digit Span subtest is used on the child and adult versions of the Wechsler Intelligence Scales (Kirkwood et al., 2011; Welsh et al., 2012; Whitney, Shepard, & Davis, 2013). As a general rule of thumb, embedded PVTs are much more prone to false positives in children compared to free-standing PVTs. For example, whereas 100% of pediatric controls in Grades 2 through 4 scored above the cutoffs on three free-standing PVTs, 59% scored below the adult-based RDS cutoff (Blaskewitz et al., 2008). This may occur because most embedded PVTs are based on total quantitative performance from tests that were originally designed as cognitive ability tests whereas free-standing PVTs were not. Thus, at a low enough age (e.g., ages 6–11) and/or with a moderate-to-severe neurological condition (e.g., severe TBI, mental retardation), significant specificity problems emerge with embedded PVTs (e.g., unacceptably high false positive rates). A noteworthy unique exception to traditional embedded or free-standing effort measures is the Symptom Validity Scale (SVS; Chafetz, 2008; Chafetz, Abrahams, & Kohlmaier, 2007) because it is actually a rating scale that combines performance on traditional quantitatively based embedded measures (e.g., RDS) with atypical qualitatively derived aspects of test performance (e.g., Ganser responses, not knowing one’s birthday). The SVS was validated against the TOMM and MSVT in children and adults in the context of SSD psychological evaluations. The authors showed that the SVS composite score was more predictive of effort than any individual parameter on the SVS. Chafetz and Abhrahams (2006) also developed another embedded measure known as the A-test. This is a simple measure of auditory vigilance that was developed for use in SSD evaluations and can be a helpful component in identifying malingering in some children



15

in the SSD context (Chafetz, 2008). Although the SVS and the A-test show promise, more information is needed regarding specificity of the SVS in severely impaired children (e.g., children with mental retardation) outside of a disability-seeking examination context in order to further determine its utility in other types of pediatric cases. A recent embedded effort measure developed for use with children is an automatized sequencing task created by Kirkwood et al. (2013). The authors found that this task can have value in detecting invalid performance in high-functioning children and adolescents but noted that the classification accuracy of embedded measures can be expected to be worse in more severely impaired children. Although developing new pediatric embedded PVTs for children is encouraged, the same fundamental problems with specificity will likely emerge when embedded effort measures from tests designed specifically for adults are used with younger and moderately to severely injured children. Lowering the cutoffs from embedded measures designed for adults can improve specificity in some pediatric samples, as Kirkwood et al. (2011) showed when using an RDS cutoff of 6 in mTBI. However, with more severely impaired samples, that same cutoff yielded a high false positive rate in a more significantly neurologically compromised pediatric sample (Welsh et al., 2012). With severely impaired neurological samples, cutoffs on embedded measures may need to be set so low to yield high specificity rates that the resulting sensitivity rates will be significantly compromised when applied to groups where mild-to-no-significant cognitive impairment would be expected. While aggregation of failed SVTs has been used with adults as a way to decrease false positives with adults (Larrabee, 2008), such a strategy may not work to reduce false positive rates with embedded measures in very young children or children with moderateto-severe brain damage/dysfunction. This is because an aggregate is only as good as the sum of its parts and children in the aforementioned groups may genuinely perform poorly on multiple embedded measures for the reasons already discussed. However, the study of aggregating failed PVTs in children remains an avenue for future research. The optimal PVT is one that achieves the balance of high specificity in groups with severe brain damage/dysfunction yet high sensitivity in groups with mild-to-no brain damage/dysfunction. In our view, the best approach in selecting PVTs with children is to utilize free-standing measures that have been proven to be easily passed by groups of healthy young children and young children with moderate-to-severe neurological conditions, which has been shown with tests such as the MSVT, WMT, and, to some extent, the TOMM (Blaskewitz et al., 2008; Carone, 2008; Donders, 2005; Green & Flaro, 2003; MacAllister et al., 2009). Ideally, classification accuracy statistics of PVTs would be judged in head-to-head comparisons within the same sample by defining groups independent of the test being studied. However, any PVT used to define the groups should not have previously demonstrated higher failure rates in very young children or patients with severe brain damage/dysfunction compared to the PVTs used in the head-to-head comparisons. The TOMM, being a visual-spatial test, has the advantage of eliminating reading demands for very young children. However, a disadvantage of the TOMM is that it has been found to be less sensitive with adults (Armistead-Jehle & Gervais, 2011; Gervais, Rohling, Green, & Ford, 2004; Lindstrom, Lindstrom, Coleman, Nelson, & Gregg, 2009) and children (Chafetz, 2008; Chafetz et al., 2007) when compared to more verbally based free-standing PVTs, likely because patients tend to report more problems with verbal memory than visual-spatial memory. This difference is likely because most daily memory


16


complaints tend to be in the verbal domain (e.g., forgetting information from conversations or reading materials) given that verbal expression is the primary manner by which people communicate (Gervais et al., 2004; Green & Allen, 2000). Although verbally based PVTs are more sensitive than visual-spatial PVTs, clinicians must take care to ensure that reading skills are appropriately developed to administer the test in accordance with published data. This would include at least a Grade 3 reading level for the WMT (Green, 2003) and, by extension, the MSVT (Green, 2004). The child’s reading level can easily be checked with a standardized reading academic achievement test. For children who have a reading level lower than Grade 3, or in which it is suspected prior to the administration of a word-reading test, MSVT and WMT manuals specifically allow for the examiner to have the patient read each word aloud and to correct reading errors. If the child is unable to read, oral versions of each test are available from the publisher. Courtney et al. (2003) showed that children in the 10–17 age range (reading level = 6.8) had an average passing score on the WMT, whereas children in the 6–9 age range with an average reading score of 1.9 had an average score below the cutoff score for poor effort. They suggested cautions in WMT use with younger children, but it is noteworthy that the study did not use the corrective method discussed above for children with a low reading level (although the authors would not have been aware of this at the time the study was performed). While many free-standing effort tests with adults can be passed by children with adult cutoffs, the possibility exists that a child with a severe enough neurological condition may genuinely score below established cutoffs. When this happens, it is important for the clinician to have some way to determine whether the result is a false positive or was due to poor effort. One validated way to do this is by computing a discrepancy score on PVTs that incorporates the difference between scores on traditional effort (easy) subtests and ability (hard) subtests. In adults, this discrepancy profile, known as the severe impairment profile (Carone, 2009), has been shown to reduce the false positive rate in adult patients with severe neurological conditions (Green, Montijo, & Borckhaus, 2011; Howe, Anderson, Kaufman, Sachs, & Loring, 2007; Howe & Loring, 2009; Singhal, Green, Ashaye, Shankar, & Gill, 2009) on the WMT and MSVT. Clinicians can also apply profile analysis to severely impaired children to test the hypothesis that a score below the cutoffs may have been due to genuine impairment, thus reducing false positives. For example, when using profile analysis with the WMT, Larochette and Harrison (2012) found that, whereas 9.5% of adolescents with severe reading or learning problems scored below the WMT effort cutoffs, profile analysis reduced the false-positive rate to less than 1%. The severe impairment profile was also recently used by Harrison and Armstrong (in press) in adolescents with significant reading impairment and reduced false positives from 11% to 1%. Green, Flaro, Brockhaus, and Montijo (2013) also applied the severe impairment profile to children with heterogeneous neurological and psychiatric disorders to reduce false positives on the WMT, MSVT, and nonverbal MSVT (NV-MSVT; Green, 2008). While some simulators will fail the WMT, MSVT, and/or NV-MSVT and obtain a discrepancy score consistent with a severe impairment profile (Chafetz & Biondolillo, 2013; Singhal et al., 2009), it is important to remember that profile analysis involves a combination of psychometric data and sound clinical reasoning. That is, psychometric criteria alone are used to determine whether a patient meets criteria for a possible severe impairment profile but then clinical reasoning based on the known or suspected condition of the patient is used to determine whether it is plausible that a very severe memory impairment is present



17

from dementia or another severe neurological condition that can explain the finding (Singhal et al., 2009). If so, the neuropsychologist would be conservative and assume that effort was good and that severe cognitive impairment caused the score(s) to fall at or below established cutoffs. If not (e.g., mTBI), the neuropsychologist would conclude that effort was poor. Such an approach is consistent with the application of diagnostic criteria for malingering established by Slick, Sherman, and Iverson (1999) and Slick and Sherman (2013), which states that clinicians must rule out whether effort test failure (and other behaviors consistent with malingering) is due to psychiatric, neurologic, or developmental factors. In most cases, however, failure on PVTs will occur in patients without severe neurological conditions (Mittenberg, Patton, Canyock, & Condit, 2002). Other limitations in pediatric PVT research include poor generalization with simulator studies and poor comprehension of the intentions of vignettes because they may not generalize to real-world scenarios that would result in poor effort (Blaskewitz et al., 2008). One example of the latter was explained earlier regarding the possible misinterpretation of the vignette in the study by Nagle et al. (2006). As an example of the former, a child who is asked to perform poorly on tests in a research setting may take an unsophisticated approach to the assessment (e.g., answering most questions wrong, providing grossly incorrect answers). This is contrasted with a real-world setting in which a significant financial settlement is dependent on the examination results and the child may have the advantage of being coached by one’s parents and/or attorney regarding more subtle attempts to suppress test scores and to avoid detection. In general, no simulator using healthy controls can replicate the real-life experiences, psychiatric histories, psychosocial stressors (e.g., financial pressures, family dysfunction, abuse histories, academic/work stress), antisocial personality characteristics, and other factors that may contribute to compensation seeking, avoidance of responsibility, and ultimately poor effort/malingering. However, such studies can be useful in determining the strategies that simulators employ, which primarily involved feigning memory impairment (Tan, Slick, Strauss, & Hultsch, 2002). In the latter study, which used undergraduate students, the WMT most accurately classified patients into their respective groups whereas the TOMM achieved the lowest classification rate. Clinicians also need to be aware of specific limitations relating to the particular diagnosis that the child has when using PVTs. For example, MacAllister et al. (2009) advised caution in the interpretation of the TOMM in young epilepsy patients with a very low IQ (especially if behavioral problems are also evident) and with ongoing interictal epileptiform activity that may disrupt attention. As with any research, appropriate cautions should also be used for studies that involve low sample sizes or are in need of further validation studies, as is the case with a simulator study performed by McKinzey, Prieler, and Raven (2003) with Standard Progressive Matrices (Raven, 1958) in 44 children aged 7–17 years of age. Use of Multiple PVTs As is the case with adults, a child’s effort may wax and wane throughout the assessment. Consequently, it has often been suggested to use multiple PVTs and/or multiple methods to assess symptom validity (Bush et al., 2005; Heilbronner et al., 2009). To adequately assess effort throughout the evaluation such tests and methods should be employed near the beginning and middle to end of the assessment. Examples of this being done in the literature include Blaskewitz et al. (2008) administering a test battery consisting of a mixture of PVTs (TOMM, MSVT, MSVT, RDS) to children

18


between the Second and Fourth grades. The SVS contains numerous embedded quantitative and qualitative effort measures for both adults and children. More recently, Kirkwood et al. (2013) administered the TOMM, MSVT, RDS, and a newly created embedded PVT (an automatized sequencing task) to children as examples of using multiple PVTs during a neuropsychological assessment.


Feedback on Failed PVTs With regard to clinical casework, the question will inevitably arise as to what a clinician should do during and/or after the evaluation if a young child or adolescent fails a PVT. In that situation, we recommend following the symptom validity feedback model (Carone, Bush, & Iverson, 2013; Carone, Iverson, & Bush, 2010) with a slight variation to account for a discussion about the child or adolescent and to direct the discussion to the parent or guardian. In short, this feedback model suggests that clinicians use the “good news/bad news” approach (e.g., “the bad news is that your scores are very low, but the good news is that there is evidence that your child is capable of much better performance”). More specifics about how to do this and to manage other possible scenarios encountered when discussing this topic are provided in the referenced feedback model. Overall, clinicians should attempt to be forthcoming about PVT failure, to explain results in a clinical context, to debrief the parents about possible emotions that they may feel, and to try to instil motivation and hope for improvement (Carone et al., 2010). Recent Political Developments Due to misconceptions about PVT use, psychologists have been officially discouraged from using formal PVTs during SSD evaluations (Chafetz et al., 2007) and the Social Security Administration (SSA) decided to no longer fund their use in a ruling on September 13, 2012. However, after consultation with the National Academy of Neuropsychology and the American Academy of Clinical Neuropsychology (including Dr. Chafetz), U.S. Senator Tom Coburn wrote a letter to the SSA urging reconsideration of this policy based on the weight of the current scientific evidence (Association of Administrative Law Judges, 2013). In response, the SSA recently indicated that it plans to seek external expertise to evaluate its policy on PVTs in determining disability (Congressional Report No. A-08-13-23094, 2013). In the interim, however, psychologists performing such evaluations (which often involve children) have to either utilize nonfunded time to use free-standing PVTs or include embedded measures. However, the latter approach is problematic given that embedded measures have much lower specificity than free-standing PVTs. For further discussion of this topic, see Chafetz (2010). SUMMARY, CONCLUSIONS, AND FUTURE DIRECTIONS The conclusions from this review have implications for both research and clinical practice. First, research in the area of pediatric PVTs remains in its infancy compared to its adult counterpart but has increased significantly in recent years. Our review shows that the majority of pediatric PVT studies have employed PVTs with children using adult cutoffs and that, in most cases, children are able to pass these tests when they are free-standing measures. Some studies have used embedded effort measures, which have greater specificity problems compared to free-standing effort tests due to their original use as an ability



19

measure. Being aware of cautions, limitations, and modifications of PVT use with children will allow clinicians to utilize well-validated PVTs with children down to age 5, depending on the type of patient. However, there is only one study supporting the use of PVTs with 5-year-olds (Kirk et al., 2011) and no studies examining PVT use in children younger than 5 years of age. For very young children particularly, behavioral observations will play an important role in conclusions about effort and compliance with test procedures, while also being aware of the limitations of behavioral observation methods when used in isolation. In both research and clinical practice, a combination of multitest and multimethod approaches is the current gold standard in the evaluation of test-taking effort with children. Furthermore, as always, it is important not to rely solely on PVTs in isolation in determining symptom validity but to also take them into consideration with data from the interview, the mental status examination, behavioral testing, and performance on other neuropsychological and psychological testing (Walker, 2011). In the future, it is highly recommended that the development of future PVTs include separate ability subtests so that a discrepancy model can be calculated analogous to the severe impairment profile on the MSVT, WMT, and NV-MSVT to reduce false positives in severely impaired children. While corrective methods to verbally based PVT test administration can be helpful in children with low reading scores, we encourage the development of more visual-spatial effort tests for use with children, particularly in cases when reading and/ or language barriers are prominent. Examples include children tested in a foreign language in which there is no verbal test translation and children who are literally unable to read the tests stimuli (e.g., children lower than age 7). For children younger than age 5, PVTs would likely need to be in the visual-spatial domain to ensure that the child can comprehend the test stimuli. In research using vignettes during PVTs with children, it will be important to more clearly specify that the child is to try to perform poorly on the test (as someone with a brain injury might) to obtain some external goal and to try to avoid detection. Without being this specific, children may unwittingly comply with the vignette in an unintended manner, as was discussed earlier with regards to the study by Nagle et al. (2006). Lastly, while embedded PVTs currently have limitations in certain pediatric samples, it is important to continue pursuing research in this area so that more embedded measures are available to clinicians with an optimal balance of sensitivity and specificity. Rather than applying adult cutoffs on embedded measures to children, a more promising avenue may be to specifically develop embedded PVTs for children that are not also used with adults. Overall, while the study of pediatric PVT use remains in its relative infancy, it is clearly growing fast, and we expect continued rapid growth in the future. Original manuscript received August 12, 2013 Revised manuscript accepted November 6, 2013 First published online December 16, 2013

REFERENCES Archer, R. P., Buffington-Vollum, J. K., Stredny, R. V., & Handel, R. W. (2006). A survey of psychological test use patterns among forensic psychologists. Journal of Personality Assessment, 87(1), 84–94. Armistead-Jehle, P., & Gervais, R. O. (2011). Sensitivity of the test of memory malingering and the nonverbal medical symptom validity test: A replication study. Applied Neuropsychology, 18(4), 284–290. doi:10.1080/09084282.2011.595455


20


Association of Administrative Law Judges. (2013). Retrieved from http://aalj.org/system/files/ documents/01.30.13lettertossaresvts.pdf Blaskewitz, N., Merten, T., & Kathmann, N. (2008). Performance of children on symptom validity tests: TOMM, MSVT, and FIT. Archives of Clinical Neuropsychology, 23, 379–391. Boone, K. B. (2007). Commentary on “Cogniform disorder and cogniform condition: Proposed diagnoses for excessive cognitive symptoms” by Dean C. Delis and Spencer R. Wetter. Archives of Clinical Neuropsychology, 22, 675–679. Broadbent, N. J., Squire, L. R., & Clark, R. E. (2004). Spatial memory, recognition memory, and the hippocampus. Proceedings of the National Academy of Sciences, 101(40), 14515–14520. Brooks, B. L. (2012). Victoria symptom validity test performance in children and adolescents with neurological disorders. Archives of Clinical Neuropsychology, 27, 858–868. Brooks, B. L., Sherman, E. M. S., & Krol, A. L. (2012). Utility of TOMM Trial 1 as an indicator of effort in children and adolescents. Archives of Clinical Neuropsychology, 27, 23–29. Bush, S. S., Ruff, R. M., Troster, A. I., Barth, J. T., Koffler, S. P., Pliskin, N. H., & Silver, C. H. (2005). Symptom validity assessment: Practice issues and medical necessity NAN policy & planning committee. Archives of Clinical Neuropsychology, 20, 419–426. Carone, D. A. (2008). Children with moderate/severe brain damage/dysfunction outperform adults with mild-to-no brain damage on the medical symptom validity test. Brain Injury, 22(12), 960–971. Carone, D. A. (2009). Test review of the medical symptom validity test. Applied Neuropsychology, 16(4), 309–311. doi:10.1080/09084280903297883 Carone, D. A., Bush, S. S., & Iverson, G. L. (2013). Providing feedback on symptom validity, mental health, and treatment in mild traumatic brain injury. In D. C. Carone & S. S. Bush (Eds.), Mild traumatic brain injury: Symptom validity assessment and malingering (pp. 101–117). New York, NY: Springer. Carone, D. A., Green, P., & Drane, D. L. (in press). Word memory test profiles in two cases with surgical removal of the left anterior hippocampus and parahippocampal gyrus. Applied Neuropsychology: Adult, 1–6. Advance online publication. doi:10.1080/09084282.2012.755533 Carone, D. A., Iverson, G. L., & Bush, S. S. (2010). A model to approaching and providing feedback to patients regarding invalid test performance in clinical neuropsychological evaluations. The Clinical Neuropsychologist, 24(5), 759–778. doi:10.1080/13854041003712951 Carter, C. J. (2013, March 29). Grand jury indicts 35 in Georgia school cheating scandal. Retrieved from http://www.cnn.com/2013/03/29/us/georgia-cheating-scandal/index.html Chafetz, M. D. (2008). Malingering on the social security disability consultative exam: Predictors and base rates. The Clinical Neuropsychologist, 22, 1–18. Chafetz, M. D. (2010). Symptom validity issues in the psychological consultative examination for social security disability. The Clinical Neuropsychologist, 24(6), 1045–1063. doi:10.1080/ 13854046.2010.481637 Chafetz, M. D., & Abrahams, J. (2006). “A’’ random letter test predicts effort in the disability exam. New Orleans, LA: American Psychological Association. Chafetz, M. D., Abrahams, J. P., & Kohlmaier, J. (2007). Malingering on the social security disability consultative exam: A new rating scale. Archives of Clinical Neuropsychology, 22, 1–14. Chafetz, M. D., & Biondolillo, A. M. (2013). Feigning a severe impairment profile. Archives of Clinical Neuropsychology, 28(3), 205–212. doi:10.1093/arclin/act015 Chafetz, M. D., & Prentkowski, E. (2011). A case of malingering by proxy in a social security disability psychological consultative examination. Applied Neuropsychology, 18, 143–149. doi:10.1080/09084282.2011.570619 Congressional Report. (2013). The social security administration’s policy on symptom validity tests in determining disability claims A-08-13-23094. Retrieved from http://oig.ssa.gov/sites/default/ files/audit/full/pdf/A-08-13-23094.pdf



21

Constantinou, M., & McCaffrey, R. J. (2003). Using the TOMM for evaluating children’s effort to perform optimally on neuropsychological measures. Child Neuropsychology, 9(2), 81–90. Copeland, L. (2013, April 14). School cheating scandal shakes up Atlanta. Retrieved from http:// www.usatoday.com/story/news/nation/2013/04/13/atlanta-school-cheatring-race/2079327/ Courtney, J. C., Dinkins, J. P., Allen III, L. M., & Kuroski, K. (2003). Age related effects in children taking the computerized assessment of response bias and word memory test. Child Neuropsychology, 9(2), 109–116. Donders, J. (2005). Performance on the test of memory malingering in a mixed pediatric sample. Child Neuropsychology, 11, 221–227. doi:10.1080/09297040490917298 Faust, D., Hart, K., & Guilmette, T. J. (1988). Pediatric malingering: The capacity of children to fake believable deficits on neuropsychological testing. Journal of Consulting and Clinical Psychology, 56(4), 578–582. Faust, D., Hart, K., Guilmette, T. J., & Arkes, H. R. (1988). Neuropsychologists’ capacity to detect adolescent malingerers. Professional Psychology: Research and Practice, 19(5), 508–515. Flaro, L., & Boone, K. (2009). Using objective effort measures to detect noncredible cognitive test performance in children and adolescents. In J. E. Morgan & J. J. Sweet (Eds.), Neuropsychology of malingering casebook (pp. 369–376). New York, NY: Psychology Press. Flaro, L., Green, P., & Blaskewitz, N. (2007). The importance of symptom validity testing with children: WMT and MSVT. Praxis Der Rechtspsychologie, 17, 125–139. Frazier, T. W., Frazier, A. R., Busch, R. M., Kerwood, M. A., & Demaree, H. A. (2008). Detection of simulated ADHD and reading disorder using symptom validity measures. Archives of Clinical Neuropsychology, 23, 501–509. Gast, J., & Hart, K. J. (2010). The performance of juvenile offenders on the test of memory malingering. Journal of Forensic Psychology Practice, 10(1), 53–68. doi:10.1080/ 15228930903173062 Gervais, R. O., Rohling, M. L., Green, P., & Ford, W. (2004). A comparison of WMT, CARB, and TOMM failure rates in non-head injury disability claimants. Archives of Clinical Neuropsychology, 19(4), 475–487. Green, P. (2003). Greens’ Word Memory Test user’s manual. Edmonton: Green’s Publishing. Green, P. (2004). Medical symptom validity test. Edmonton: Green’s Publishing. Green, P. (2007). The pervasive influence of effort on neuropsychological tests. Physical Medicine and Rehabilitation Clinics of North America, 18, 43–68. Green, P. (2008). Green’s non-verbal medical symptom validity test (NV-MSVT) for Microsoft Windows: User’s manual 1.0. Edmonton: Green’s Publishing. Green, P., & Allen III, L. M. (2000). Patterns of memory complaints in 577 consecutive patients passing or failing symptom validity tests. Archives of Clinical Neuropsychology, 15(8), 844–845. Green, P., & Flaro, L. (2003). Word memory test performance in children. Child Neuropsychology, 9(3), 189–207. Green, P., Flaro, L., Brockhaus, R., & Montijo, J. A. (2013). Performance on the WMT, MSVT, & NV-MSVT in children with developmental disabilities and in adults with mild traumatic brain injury. In C. R. Reynolds & A. M. Horton (Eds.), Detection of malingering during head injury litigation (2nd ed., pp. 201–219). New York, NY: Plenum Press. Green, P., & Merten, T. (2013). Noncredible explanations of noncredible performance on symptom validity tests. In D. C. Carone & S. S. Bush (Eds.), Mild traumatic brain injury: Symptom validity assessment and malingering (pp. 73–100). New York, NY: Springer. Green, P., Montijo, J., & Brockhaus, R. High specificity of the word memory test and medical symptom validity test in groups with severe verbal memory impairment. Applied Neuropsychology, 18(2), 86–94. doi:10.1080/09084282.2010.523389 Greve, K. W., Bianchini, K. J., Stickle, T. R., Love, J. M., Doane, B. M., & Thompson, M. D. (2007). Effects of a community toxic release on the psychological status of children. Child Psychiatry and Human Development, 37, 307–323.


22


Guilmette, T. J. (2013). The role of clinical judgment in symptom validity assessment. In D. C. Carone & S. S. Bush (Eds.), Mild traumatic brain injury: Symptom validity assessment and malingering (pp. 31–41). New York, NY: Springer. Gunn, D., Batchelor, J., & Jones, M. (2010). Detection of simulated memory impairment in 6- to 11-year-old children. Child Neuropsychology, 16(2), 105–118. Harrison, A. G. (2012). Assessment and accommodation of students with learning disabilities at the postsecondary level in Canada: A special issue for psychologists in education. Canadian Journal of School Psychology, 27(1), 3–11. Harrison, A. G., & Armstrong, I. (in press). WISC-IV unusual digit span performance in a sample of adolescents with learning disabilities. Applied Neuropsychology: Child, 1–9. Advance online publication. doi:10.1080/21622965.2012.753570 Harrison, A. G., Green, P., & Flaro, L. (2012). The importance of symptom validity testing in adolescents and young adults undergoing assessments for learning or attention difficulties. Canadian Journal of School Psychology, 27(1), 98–113. doi:10.1177/0829573512437024 Heilbronner, R. L., Sweet, J. J., Morgan, J. E., Larrabee, G. J., Millis, S. R., & Conference Participants. (2009). American academy of clinical neuropsychology consensus conference statement on the neuropsychological assessment of effort, response bias, and malingering. The Clinical Neuropsychologist, 23(7), 1093–1129. Henry, G. K. (2005). Childhood malingering: Faking neuropsychological impairment in an 8-yearold. In R. L. Heilbronner (Ed.), Forensic neuropsychology casebook (pp. 205–217). New York, NY: Guilford Press. Heubrock, D. (2001). Münchhausen by proxy syndrome in clinical child neuropsychology: A case presenting with neuropsychological symptoms. Child Neuropsychology, 7(4), 273–285. Howe, L. L., & Loring, D. W. (2009). Classification accuracy and predictive ability of the medical symptom validity test’s dementia profile and general memory impairment profile. The Clinical Neuropsychologist, 23(2), 329–342. Howe, L. L. S., Anderson, A. M., Kaufman, D. A. S., Sachs, B. C., & Loring, D. W. (2007). Characterization of the medical symptom validity test in evaluation of clinically referred memory disorders clinic patients. Archives of Clinical Neuropsychology, 22, 753–761. Johnson, E. A. (1997). Children’s understanding of epistemic conduct in self-deception and other false beliefs stories. Child Development, 68, 1117–1132. Kirk, J. W., Harris, B., Hutaff-Lee, C. F., Koelmay, S. W., Dinkins, J. P., & Kirkwood, M. W. (2011). Performance on the test of memory malingering (TOMM) among a large clinic-referred pediatric sample. Child Neuropsychology, 17(3), 242–254. doi:10.1080/ 09297049.2010.533166 Kirkwood, M. W., Connery, A. K., Kirk, J. W., & Baker, D. A. (2013). Detecting performance invalidity in children: Not quite as easy as A, B, C, 1, 2, 3 but automatized sequences appears promising. Child Neuropsychology, 1–8. Advance online publication. doi:10.1080/ 09297049.2012.759553 Kirkwood, M. W., Hargrave, D. D., & Kirk, J. W. (2011). The value of the WISC-IV digit span subtest in detecting noncredible performance during pediatric neuropsychological examinations. Archives of Clinical Neuropsychology, 26, 377–384. Kirkwood, M. W., & Kirk, J. W. (2010). The base rate of suboptimal effort in a pediatric mild TBI sample: Performance on the medical symptom validity test. The Clinical Neuropsychologist, 24(5), 860–872. doi:10.1080/13854040903527287 Kirkwood, M. W., Kirk, J. W., Blaha, R. Z., & Wilson, P. (2010). Noncredible effort during pediatric neuropsychological exam: A case series and literature review. Child Neuropsychology, 16, 604–618. doi:10.1080/09297049.2010.495059 Kirkwood, M. W., Yeates, K. O., Randolph, C., & Kirk, J. W. (2012). The implications of symptom validity test failure for ability-based test performance in a pediatric sample. Psychological Assessment, 24(1), 36–45. doi:10.1037/a0024628



23

Kompanje, E. J. O. (2007). A case of malingering by proxy described in 1593. Child Abuse & Neglect, 31, 1013–1017. Larochette, A.-C., & Harrison, A. G. (2012). Word memory test performance in Canadian adolescents with learning disabilities: A preliminary study. Applied Neuropsychology, 1, 38–47. doi:10.1080/21622965.2012.665777 Larrabee, G. J. (2003). Detection of malingering using atypical performance patterns on standard neuropsychological tests. The Clinical Neuropsychologist, 17(3), 410–425. Larrabee, G. J. (2008). Aggregation across multiple indicators improves the detection of malingering: Relationship to likelihood ratios. The Clinical Neuropsychologist, 22, 666–679. Larrabee, G. J. (2012). Performance validity and symptom validity in neuropsychological assessment. Journal of the International Neuropsychological Society, 18(4), 625–630. Lewis, M., Stanger, C., & Sullivan, M. W. (1989). Deception in 3-year-olds. Developmental Psychology, 25, 439–443. Lindstrom, W. A., Lindstrom, J. H., Coleman, C., Nelson, J., & Gregg, N. (2009). The diagnostic accuracy of symptom validity tets when used with postsecondary students with learning disabilities: A preliminary investigation. Archives of Clinical Neuropsychology, 24, 659–669. Loughan, A. R., Perna, R., & Hertza, J. (2012). The value of the Wechsler intelligence scale for children-fourth edition digit span as an embedded measure of effort: An investigation into children with dual diagnoses. Archives of Clinical Neuropsychology, 27, 716–724. Lu, P. H., & Boone, K. B. (2002). Suspect cognitive symptoms in a 9-year-old child: Malingering by proxy? The Clinical Neuropsychologist, 16(1), 90–96. MacAllister, W. S., Nakhutina, L., Bender, H. A., Karantzoulis, S., & Carlson, C. (2009). Assessing effort during neuropsychological evaluation with TOMM in children and adolescents with epilepsy. Child Neuropsychology, 15(6), 521–531. doi:10.1080/09297040902748226 Mantynen, H., Poikkeus, A. M., Ahonen, T., Aro, T., & Korkman, M. (2001). Clinical significance of test refusal among young children. Child Neuropsychology, 7, 241–250. McCaffrey, R. J., & Lynch, J. K. (2009). Malingering following documented brain injury: Neuropsychological evaluation of children in a forensic setting. In J. E. Morgan & J. J. Sweet (Eds.), Neuropsychology of malingering casebook (pp. 377–385). New York, NY: Psychology Press. McCann, J. T. (1998). Malingering and deception in adolescents: Assessing credibility in clinical and forensic settings. Washington, DC: American Psychological Association. McKinzey, R. K., Prieler, J., & Raven, J. (2003). Detection of children’s malingering on Raven’s standard progressive matrices. The British Journal of Clinical Psychology, 42, 95–99. Mittenberg, W., Patton, C., Canyock, E. M., & Condit, D. C. (2002). Base rates of malingering and symptom exaggeration. Journal of Clinical and Experimental Neuropsychology, 24(8), 1094–1102. Nagle, A. M., Everhart, D. E., Durham, T. W., McCammon, S. L., & Walker, M. (2006). Deception strategies in children: Examination of forced choice recognition and verbal learning and memory techniques. Archives of Clinical Neuropsychology, 21, 777–785. Newton, P., Reddy, V., & Bull, R. (2000). Children’s everyday deception and performance on falsebelief tasks. British Journal of Developmental Psychology, 18, 297–317. Perna, R. B., & Loughan, A. R. (2013). Children and the test of memory malingering: Is one trial enough? Child Neuropsychology, 19(4), 438–447. doi:10.1080/09297049.2012.731500 Polak, A., & Harris, P. L. (1999). Deception by young children following noncompliance. Developmental Psychology, 35(2), 561–568. Rabiner, D. L. (2013). Stimulant prescription cautions: Addressing misuse, diversion and malingering. Current Psychiatry Reports, 15, 375–383. Raven, J. C. (1958). Standard progressive matrices, 1956 revision. San Antonio, TX: Oxford Psychologists’ Press/The Psychological Corp.


24


Rienstra, A., Spaan, P. E. J., & Schmand, B. (2010). Validation of symptom validity tests using a “child-model” of adult cognitive impairments. Archives of Clinical Neuropsychology, 25(5), 371–382. Rohling, M. L. (2004). Who do they think they’re kidding: A review of the use of symptom validity tests with children. Newsletter 40, 22(2), 1, 21–26. Schmitz, B., & Skinner, E. (1993). Perceived control, effort, and academic performance: Interindividual, intraindividual, and multivariate time series analysis. Journal of Personality and Social Psychology, 64, 1010–1028. Sharland, M. J., & Gfeller, J. D. (2007). A survey of neuropsychologists’ beliefs and practices with respect to the assessment of effort. Archives of Clinical Neuropsychology, 22(2), 213–223. Singhal, A., Green, P., Ashaye, K., Shankar, K., & Gill, D. (2009). High specificity of the medical symptom validity test in patients with very severe memory impairment. Archives of Clinical Neuropsychology, 24(8), 721–728. doi:10.1093/arclin/acp074 Slick, D. J., Sherman, E. M., & Iverson, G. L. (1999). Diagnostic criteria for malingered neurocognitive dysfunction: Proposed standards for clinical practice and research. The Clinical Neuropsychologist, 13(4), 545–561. Slick, D. J., & Sherman, E. M. S. (2013). Differential diagnosis of malingering. In D. C. Carone & S. S. Bush (Eds.), Mild traumatic brain injury: Symptom validity assessment and malingering (pp. 57–72). New York, NY: Springer. Spence, S. A. (2004). The deceptive brain. Journal of the Royal Society of Medicine, 97(1), 6–9. Stouthamer-Loeber, M. (1986). Lying as a problem behavior in children: A review. Clinical Psychology Review, 6, 267–289. Talwar, V., Gordon, H. M., & Lee, K. (2007). Lying in the elementary school years: Verbal deception and its relation to second-order belief understanding. Developmental Psychology, 43(3), 804–810. doi:10.1037/0012–1649.43.3.804 Tan, J. E., Slick, D. J., Strauss, E., & Hultsch, D. F. (2002). How’d they do it? Malingering strategies on symptom validity tests. The Clinical Neuropsychologist, 16(4), 495–505. Tombaugh, T. N. (1996). TOMM: Test of memory malingering. North Tonawanda, NY: Multi-Health Systems. Van Dyke, S. A., Millis, S. R., Axelrod, B. N., & Hanks, R. A. (2013). Assessing effort: Differentiating performance and symptom validity. The Clinical Neuropsychologist. Advance online publication. doi:10.1080/13854046.2013.835447 Walker, J. S. (2011). Malingering in children: Fibs and faking. Child & Adolescent Psychiatric Clinics in North America, 20, 547–556. doi:10.1016/j.chc.2011.03.013 Welsh, A. J., Bender, H. A., Whitman, L. A., Vasserman, M., & Macallister, W. S. (2012). Clinical utility of reliable digit span in assessing effort in children and adolescents with epilepsy. Archives of Clinical Neuropsychology, 27(7), 735–741. doi:10.1093/arclin/acs063 Whitney, K. A., Shepard, P. H., & Davis, J. J. (2013). WAIS-IV digit span variables: Are they valuable for use in predicting TOMM and MSVT failure? Applied Neuropsychology—Adult, 20(2), 83–94. doi:10.1080/09084282.2012.670167 Wright, R. W., Brand, R. A., Dunn, W., & Spindler, K. P. (2007). How to write a systematic review. Clinical Orthopaedics and Related Research, 455, 23–29.