Burgess, P. W. (1997) Theory and methodology in executive function research. In: P. Rabbitt (Ed.) Theory and Methodology of Frontal and Executive Function.
CHAPTER FOUR
Theory and Methodology in Executive Function Research Paul W. Burgess Department of Psychology, University College London, U.K.
Please reference this paper as: Burgess, P. W. (1997) Theory and methodology in executive function research. In: P. Rabbitt (Ed.) Theory and Methodology of Frontal and Executive Function (pp. 81-116). Hove, U.K.: Psychology press: -
-
INTRODUCTION The term "executive functions" is a relatively new one to neuropsychology (see, for example, Baddeley & Wilson, 1986, 1988; Lezak, 1983; Stuss & Benson, 1986). It is used to describe a range of poorly defined processes which are putatively involved in activities such as "problem solving" (Levin, Goldstein, Williams, & Eisenberg, 1991), "planning" (Shallice, 1982), "initiation" of activity (Burgess & Shallice, 1996c; Shimamura, Janowsky, & Squire, 1991), "cognitive estimation" (Shallice & Evans, 1978) and "prospective memory" (e.g. Cockburn, 1995; Shimamura, 1989). Historically these processes have always been linked to the frontal lobes (e.g. Benton, 1991) and damage to them has resulted in the range of symptoms previously known as the "frontal lobe syndrome" (see Stuss & Benson, 1986, for review). It is probably fair to say that until recently the study of executive function has been a Cinderella area of (cognitive) neuropsychology. For instance, one of the standard textbooks in the area (Ellis & Young, 1988) has no mention at all of the area. Another (McCarthy & Warrington, 1990) has only 22 pages devoted to what they cautiously call "problem solving," when memory as an equally broad topic commands approximately three times as much space, and the neuropsychology of language commands approximately seven times the space! This state of affairs exists despite the generally accepted view that executive functions are those processes which "enable a
82
BURGESS
person to engage successfully in independent, purposive, self-serving behaviour" (Lezak, 1995, p. 42). Given this characterisation, it is hardly surprising that those texts which concern themselves with more general aspects of the practice of neuropsychology (e.g. Kolb & Whishaw, 1990; Lezak, 1995) have more to say on the topic of executive/frontal lobe function, and the subject is one of primary concern to those working in rehabilitation settings, since the range of behavioural and cognitive disturbances which these so-called "dysexecutive" (Baddeley & Wilson, 1988) patients often show suggest poor outcome and difficult management (e.g. Alderman & Burgess, 1994; Alderman, Fry, & Youngson, 1995; Burgess & Wood, 1990; Thomsen, 1987; Von Cramon & Von Cramon, 1994). For these reasons it would seem that the recent increase in research in the area has been to some extent driven by a practical need to understand executive function, rather than for pure theoretical reasons alone. A further aspect of the study of executive functions has been the prevalence of group study methodology. Most investigators in the area of executive function have produced group studies at one time or another (e.g. Kopelman, 1991; Milner, 1963, 1982; Petrides, 1985; Schacter, 1987; Shallice, 1982; Shallice & Evans, 1978; Janowsky, Shimamura, & Squire, 1989; Stuss and his colleagues, 1982, 1991; and all chapters in this volume, to mention but a very few). Moreover, most of these large group studies have sought specifically to link certain types of behavioural manifestation with damage to certain brain structures. In large part these two aspects of executive function research (group studies and brain structure links) do not sit easily with the creed of "ultra-cognitive" neurpsychology, where results from group studies are regarded as unsuited to theory development, and questions regarding the neurological basis of behaviour are irrelevant (Shallice, 1988, p. 203). For researchers within this tradition, the primary and hard fought-for development over the last 10 or 20 years has been rejection of exactly these methodologies and concerns (e.g. Caramazza, 1986; Caramazza & Badecker, 1991; Ellis, 1987; McCloskey, 1993). Thus those who might be described as "ultra-cognitive neuropsychologists" have little option but to disregard much of executive function research. Perhaps this is the reason for the complete omission of the subject in Ellis and Young's otherwise excellent textbook. However, the possibility I want to consider here is that for both practical and theoretical reasons the exclusive study of either groups or single cases is unlikely to be sufficient a methodology in executive function research. The strategy most likely to succeed is one where both approaches are admitted. In particular I want to argue that within the area of executive functions, the key finding of theoretical interest to the ultra-cognitive neuropsychologist-
4. EXECUTIVE FUNCTION RESEARCH
83
the double dissociation--can often be misleading. The primary reasons, which are all interrelated, are as follows: 1. Executive tasks are less pure measures than non-executive ones. 2. The relative lack of correspondence between behaviour and putative cognitive processes in executive functions may lead to apparent dissociation~ between behaviours without this necessarily reflecting fractionation of underlying processes. 3. Serial dependency in sequences of complex behaviour may lead to misleading dissociations, on the one hand, and preclude the discovery of functional independence, on the other. 4. The nature of executive control is that it serves to adapt behavioural routines. This means that no two behavioural sequences which stress these functions to an identical degree will be exactly the same. Consequently, in the absence of necessary associations between observed behaviours, simple dissociations may be uninstructive. These points will be discussed in detail, followed by a brief examination of the implications of measuring cognitive processing in novel situations using single-case design. I must stress that this chapter is not an attack on the utility of single-case methods. Far from it. It is intended as an appeal for principled discussion about the unique methodological demands made by the nature of executive system functioning.
THE NATURE OF EXECUTIVE FUNCTION Current theoretical interpretations of the neuropsychological data from dysexecutive patients (i.e. patients who show problems with executive control) owe much to the distinction between automatic and controlled behaviour in traditional cognitive psychology (see, for example, Atkinson & Shiffrin, 1968; Schneider & Shiffrin, 1977); current consensus regards the executive system as a process or set of processes whose primary purpose is to facilitate adaptation to novel situations (see, for example, Burgess & Cooper, 1996; Karnath, Wallesch, & Zimmerman, 1991; Shallice, 1988, p. 345; Sirigu et al., 1995). This system works by "modulation and control of more fundamental or routine cognitive skills" (McCarthy & Warrington, 1990, p. 343; see also for related accounts Damasio, 1985; Duncan, 1986; Fuster, 1980; Luria, 1973; Shallice, 1982; Teuber, 1972). These routine cognitive skills are generally regarded as those which have been overlearned by practice or repetition, and thus can include anything from motor, reading, or language skills to semantic memory (Burgess & Shallice, 1996b). However most real-life situations require at least some adaptation of these
84
BURGESS
skills (since no two situations are exactly alike), and Norman and Shallice (1980, 1986; see also Shallice & Burgess, 1991b) outline five types of situation where routine, automatic activation of behaviour would not be sufficient for optimal performance: 1. Those that involve planning or decision making. 2. Those involving error correction or troubleshooting. 3. Situations where responses are not well-learned or contain novel sequences of actions. 4. Dangerous or technically difficult situations. 5. Situations which require the overcoming of a strong habitual response or resisting temptation. That these characterisations were necessary reveals a key aspect of current views on the role of executive processes in cognition: Neuropsychologists would hardly feel it necessary to define the circumstances under which speech production processes are in use; or, taking an example which is more theoretically oriented, situations which might stress the graphemic buffer (e.g. Ellis, 1982). In these examples, there is presumed to be a high degree of correspondence between the putative process and a specific behaviour. In other words, the cognitive system underlying the behaviour is supposed to be dedicated to it, or at least to a range of highly similar behaviours. Thus there are thought to be dedicated cognitive processes underlying reading, spelling, object recognition, and so forth. Neuropsychological (double dissociation) evidence of specific impairments has led to the current view of these cognitive systems as relatively "informationally encapsulated", and some have argued that they are examples of modularity in functional architecture beyond only the input level that Fodor (1983) envisaged, or the output level as Marshall (1984) suggests, (see Shallice, 1988, pp. 269-273; Shallice, 1984). Thus if Fodor's "central processes" now exist, they are probably synonymous with what neuropsychologists call executive processes. These are processes which guide behaviour in many different situations and, critically, work either by action upon the output from routine processing resources, or by direct input to them. Fodor was gloomy about the prospects for the study of central processes, maintaining that if they had the properties which he ascribed to them (e.g. non-modular, and not informationally encapsulated) they would be "bad candidates for scientific study" (Fodor, 1983, p. 127). With an unintended neuropsychological pun, he maintained that ". . . enthusiasm for a frontal assault on central processes . .. seems to have considerably abated" (p. 126). The latter statement hardly applies to neuropsychology (Fodor's point was made in reference to AI work). But what about the former? Are executive processes non-modular and, if they are, does this mean that they are bad candidates for study?
4. EXECUTIVE FUNCTION RESEARCH
85
EXECUTIVE FUNCTIONS AND THE PROCESS-BEHAVIOUR' DISTINCTION As an example of the assumed process-behaviour correspondence in the cognitive neuropsychology of routine processing, consider for instance Bruce and Young's (1986) model of face recognition. This model characterises well the neuropsychological (and cognitive psychological) findings from prosopagnosic patients; it does so largely by defining stages in face processing and recognition (e.g. "expression analysis") and assuming that, at least to a large degree, different cognitive processes support these stages. Thus a double dissociation between performance on tests of facial expression and identity (e.g. Etkoff, 1984) suggests that recognition of faces and expression analysis share little in terms of processing resources (see also McNeil & Warrington, 1993). The model (like others in the area) is based on studies involving close putative process-behaviour correspondence: the evidence for preserved "facial expression processing" comes from tests where, for instance, the participant is asked to judge facial emotion from photographs. The finding of preserved ability to recognise faces (and therefore intact "face recognition units," in Bruce and Young's terms) typically might come from asking the participant to recognise people from pictures of their faces. Here the processes thought to be tapped when the participant performs the behaviour of, for instance, responding to the question "tell me who this person is" are assumed to be highly dedicated to that situation: one does not suppose that they are tapped to nearly the same degree if the participant is shown a word and asked to define it. Thus we have a process (or a set of them) which is very closely dedicated to supporting behaviour in one narrow type of situation. This is fundamental to the notions of "informational encapsulation," and "routine processing" (specialisation of cognitive systems through repetition). By contrast, executive processes theoretically manifest themselves in a range of quite different situations, the only unifying feature of which might be the involvement of that process. This is the rationale behind, say, Duncan's (Duncan, Burgess, & Emslie, 1995; Duncan, in preparation) contention that executive processes are synonymous with Spearman's g. If we
'
Just in case there is any doubt, definitions of "process" and "behaviour" as used in this chapter are as follows: Cognitive processes are: (1) Event independent. They are not necessarily tied to any one behaviour and may he used in many. (2) Are not related, performance-wise,to the output of previous processes. (3) Are only assessed through measurement of a behavioural operation. They not directly observable. (4) The possibility of infinite reduction is a pragmatic and theoretical danger. (5) The possibility of parallel processing exists. Behaviours are: (I) Directly observable, event dependent. (2) Behaviours relevant to one goal are performed serially. (3) Even the simplest behaviour may require the activation of many processes. (4) Behaviours have an endpoint which is observerdefined.
86
BURGESS
accept this theoretical standpoint, a number of methodological and practical considerations follow.
Methodological Consequences of Low Process-Behaviour Correspondence Increased "Measurement Error." Let us assume that any neuropsychological measure is subject to two primary sources of measurement error: the first is random error arising from factors which change from person to person and from one occasion to another (e.g. fluctuations in arousal or fatigue, external noise, interuptions, and so forth); the second is non-random involvement of cognitive processes which are not intended to be measured. This second source might occur, for instance, with the involvement of reading systems in performing Warrington's Recognition Memory Test for words, where the intention of the test is to measure mnestic functions, hut where variations in reading ability may affect performance (Burgess & Shallice, 1994) in subtle and poorly understood ways. The first source of error is what is traditionally referred to in psychometrics as measurement error, but from the point of view of theorising in neuropsychology the second source is probably more relevant because it is nonrandom, and thus more troublesome for statistical interpretation. This second type of measurement error in many areas of neuropsychological enquiry (e.g. semantic memory deficits) is generally ignored. It is assumed that a patient's ability, for instance, to give information about animals identified from pictorial representation is so highly indicative of some internal state that the possibility of measurement error can be safely disregarded. However, measurement of a control process, when that process only receives input from some slave process and outputs through another, will always be contaminated by effects attributable to these slave processes, quite apart from the error attributable to other sources such as processbehaviour translation, pure observation error and so forth. Empirical support for this viewpoint is provided by data taken from the study in this volume by Wilson, Alderman, Burgess, Emslie and Evans. If executive tasks are contaminated by the influence of slave processes to a greater extent than slave processes are contaminated by each other, then performance on executive tasks should generally show greater correlations with indicators of slave processes than can be found between measures which are good indicators of supposedly "informationally encapsulated" resources, or at least between resources which are psychologically supposed to be quite distinct. As part of the Wilson et al. study, certain traditional executive tests were administered to a group (N= 94) of mixed-aetiology neurological patients,
4.
EXECUTIVE FUNCTION RESEARCH
87
alongside a number of putatively non-executive tasks of memory, language, and perception. As an example of an "executive task", let us consider performance on the Wisconsin Card-Sorting Test or WCST (number of categories achieved on the Nelson version). Two "non-executive" tasks one might choose which theoretically should share few direct processing resources with the WCST, and with each other, might be Warrington's (1984) recognition memory test (RMT) for words, and the Position Discrimination subtest from the VOSP (Warrington & James, 1991). The RMT for words requires the participant to make a judgment about 50 words presented one-by-one. They are, without delay, then asked to identify the target words when presented with a foil. The test is generally considered to measure memory ability for verbal material. Performance on this task should,prima facie, share little in the way of processing resources with the Position Discrimination subtest of the VOSP. Here the participants are presented with two squares which each have a dot marked in them. They are asked to identify the square which has the dot in the centre, rather than just offset from it. This task has no obvious verbal or memory component and is generally regarded as a test of visuospatial or perceptual ability. The assumption that verbal memory and perceptual abilities share little in common should therefore be enough to predict strongly that there will be little relationship between these tasks in an unselected neurological population. And indeed this appears to be the case: the correlation between these tasks was found to be ,141, which is not significantly different from zero. Moreover, neither test was significantly correlated with participants' age (RMT words, -.IS; Position Discrimination, -.13). But what about their relationship with the WCST? Performance on the WCST, unlike the other two tests, was significantly related to age (r = -.28, p < .01), to position discrimination (r = .36, p < .002) and was very strongly related to performance on RMT words (a surprisingly high correlation of .59, p
