Expertise and Expert Performance-based Training - Old City Publishing

© 2009 Old City Publishing, Inc. Published by license under the OCP Science imprint, a member of the Old City Publishing Group

Tech., Inst., Cognition and Learning, Vol. 7, pp. 121–146 Reprints available directly from the publisher Photocopying permitted by license only

Expertise and Expert Performance-based Training (ExPerT) in Complex Domains Paul Ward*, Joel Suss and Itay Basevitch Department of Psychology, and Learning Systems Institute, Florida State University, 1107 W. Call Street, Tallahassee, FL 32306-4301, Tel: 850-645-7424. Fax. 850-644-7739

The aim of this article is to provide an overview of research on expertise and training in complex domains. First, we present a summary of the origins of the research on expertise and the development of current theory in expert performance. Then we present a synopsis of the literature on expert performance in sport and perceptual-cognitive skills training. We expand on this section by summarizing some of the literature on expert performance and training in other complex domains, such as law enforcement, nursing, and the military. In each section, we examine the evidence-based approaches to training that have been developed across domains. The research suggests that experts develop superior anticipation, situational assessment and decision making skills and strategies that are supported by cognitive representations consistent with the acquisition of long-term working memory skill. We argue that the perceptualcognitive strategies of experts and the way in which they were acquired may be an informative basis for training future experts. This has been termed an evidence-based approach to training. We refer to this approach here as the Expert Performance-based Training (ExPerT) method. Such training is typically based on a situation-specific expert model, is dynamic and context-sensitive, and can be objectively validated with known outcomes. Evidence-based training of this type mimics the real-world demands faced by individuals within the domain, provides opportunities for deliberate practice on challenging and low frequency cases, and has been shown to contribute to the development of the type of skills and representations that underpin expert performance in complex domains. Keywords: Perceptual-cognitive skill; Representative task; Long term working memory; Expert model; Situation-specific training.

*Corresponding author: [email protected]

121

121-146 pp TICL_004_Ward.indd 121

11/23/2009 3:19:14 PM

122

Ward et al.

Expertise and Expert Performance-based Training (ExPert) in Complex Domains Our goal in this article is to describe how researchers that work predominantly in domains outside of education have measured expertise and expert performance, and developed training programs to improve the performance of future experts. We emphasize the shift toward evidence-based approaches to training that have used information gleaned from the measurement of expert performance as a basis to inform instructional design. We refer to this approach in this article as the Expert Performance-based Training (ExPerT) method. To provide some context to this literature we first provide an overview of the origins of the research on expertise and the subsequent development of current theories of expert performance in traditional and non-traditional domains. In their classic theory of expertise, Simon and Chase (1973) proposed that skill was primarily acquired through extensive domain-specific experience. Rather than differ from novices in basic capacities or elementary information processes (Newell & Simon, 1972), experts were assumed to be able to circumvent information-processing limits by encoding pieces of information in terms of pre-existing patterns (i.e., chunks) in short-term memory (STM). Simon and Chase focused their efforts on skill-based differences in pattern-recognition-type processes primarily because grandmaster and world-class chess players did not differ in the structure of their planning and evaluation of future move sequences (see de Groot, 1965). They concluded that it was the process of recognizing meaningful patterns that drove any subsequent search in which experts engaged and, ultimately, their ability to select the next best move in any given chess match situation. To test their theory, Chase and Simon (1973a, 1973b) extended de Groot’s (1965) original work by asking players to recall arrangements of briefly presented chess pieces. Their analysis showed that complex pattern matching mechanisms (i.e., chunking), and storage of chunks in short-term memory (STM), adequately explained the superior recall of experts for meaningfully structured trials. However, memory capacity, i.e., the number of different chunks in STM, did not reliably differ between experts and less-skilled players for material that was assumed to be devoid of patterns, such as unstructured game configurations (c.f. Gobet & Simon, 1996). A number of researchers have questioned the assumptions upon which this research was based. Charness (1976) adapted the original recall paradigm to include a working memory-demanding task that was interpolated between viewing and recalling the chess position, such that any information stored in STM about the chess position would be irretrievably lost. Expert chess players


11/23/2009 3:19:14 PM

Expert Performance and Training

123

were still able to accurately recall the position of chess pieces after the interruption indicating that performance must rely upon storage in long-term memory (LTM) rather than STM as originally assumed (see also Frey & Adesman, 1976). Similar results have been reported in the comprehension of sentences in a text. Glanzer and colleagues (e.g., Fischer & Glanzer, 1986; Glanzer, Dorfman, & Kaplan, 1981; Glanzer, Fischer, & Dorfman, 1984) found no decrement in comprehension when skilled readers were interrupted between consecutive sentences while reading the text, by asking them to read an additional unrelated sentence. In line with this conflicting evidence, in a substantive review of the available empirical evidence, Ericsson and Kintsch (1995) proposed an alternative theory—Long-Term Working Memory (LTWM) theory. Their review demonstrated that more complex mechanisms were responsible for superior expert performance; namely that experts are able to generate memory representations that described the semantic relations in the current situation that are stored and remain accessible in LTM. Research from a range of domains indicates that such integrative representations permit future retrieval demands of the situation to be anticipated, and performance to be effectively executed, monitored and controlled (see Ericsson & Kintsch, 1995). In line with this theory, the research in text comprehension suggests that situational representations of the text (e.g., situation models) facilitate and support encoding of predictive and other types of inferences about the current situation (Ericsson, Patel, & Kintsch, 2000; Kintsch, 1998; Zwaan & Radvansky, 1998). Only by eliciting the true properties of expertise and considering how the mechanics of the underlying representation bring about superior performance will we increase our understanding of expertise and move beyond simple descriptions of experts as superior recognizers or intuitive decision makers. Much of the evidence amassed to date that would allow us to examine the nature of the processing underlying expert performance has been collected in relatively self-paced domains and/or on tasks that afford the opportunity to reason and plan in vivo about the current situation, such as text comprehension, medical diagnosis, and standard chess playing. While the use of tasks that are more representative of real-world performance, such as move selection in chess, has increased, few researchers have made scientifically rigorous attempts to examine expert performance in complex domains (e.g., dynamic, stressful, temporally constrained, etc.) using representative tasks. Although effective instructional methods have been developed, without such investigations into the nature of expertise in complex domains, it is difficult to see how the evidence necessary to develop effective training procedures could be accrued.


11/23/2009 3:19:14 PM

124

Ward et al.

In the following sections we provide an overview of some of the research on expertise in complex domains. We focus on sport and provide a more cursory review of other complex domains, such as law enforcement, health-care and the military. Each of these sections ends with a summary of how this research has been used to inform training and instructional practices in complex domains.

Perceptual-Cognitive Expertise and Training in Sport Chase and Simon’s (1973a, 1973b) pioneering work on the study of expertise in chess motivated some of the original work examining sports expertise. Inspired by the assumption that the cognitive mechanisms underpinning expertise were largely based on differences in recognition-based, rather than search-based processing, sports researchers adapted the original recall paradigm to the domain of sport, and specifically to team sports such as basketball and volleyball (Allard, Graham, & Paarsalu, 1980; Allard & Starkes, 1980). Consistent with the literature on chess expertise, the research generally indicated that skilled players could recall the positions of substantially more players when viewing meaningful game configurations (i.e., structured patterns of play), particularly when these were of complex patterns of play that dynamically unfolded over time rather than when presented with snapshot (i.e., still) images of the game (Borgeaud & Abernethy, 1987; Williams, Davids, Burwitz & Williams, 1993). When participants viewed unstructured patterns of play, the results corroborated the original findings in chess; the expert advantage was diminished and the ability of skilled players to recall the position of players was reduced to novice levels. This research lent support to the notion that recognition-based memory skills supporting the ability to recall patterns of play were primarily responsible for the expert advantage. Subsequent research, however, indicated that superior memory for game positions was an incidental by-product of the experts’ substantial and deliberate engagement in their domain of expertise (Allard, Parker, Deakin, & Rodgers, 1993; Williams & Davids, 1995). Improvement in memory for gamerelated information is likely to have occurred as a consequence of developing more domain-specific skills, such as the ability to anticipate an opponent’s intention to act based on environmental and postural information. A parallel stream of research in chess attempted to train novice participants’ memory for game positions in an effort to determine whether acquiring pattern recognition skills increased performance in chess (see Ericsson & Staszewski, 1989). Although participants improved their memory for positions within a relatively short timeframe, such skills did not improve game performance. These studies suggest that while


11/23/2009 3:19:14 PM


125

memory recall for patterns of meaningful information may be reflective of expertise, other skills may play more of a causal role in skilled performance. Shortly after the introduction of the recall paradigm in sport, Abernethy and colleagues, followed by Williams and colleagues (e.g., Abernethy, 1987; Abernethy & Russell, 1984, 1987; Williams & Burwitz, 1993) began to assess other aspects of performance that were, arguably, more representative of in-game decision making. For instance, Abernethy and Russell (1987) used the temporal1 and spatial2 occlusion paradigms, originally introduced by Haskins (1965) as a training tool, to determine whether skilled badminton players could anticipate the direction and force (i.e., depth) of their opponent’s strokes (e.g., forehand, backhand, smash and drop-shots). These paradigms allowed researchers to determine the temporal and spatial locations of perceptual cues in the environment that skilled players used to anticipate their opponent’s intentions. Specifically, researchers examined whether skilled players could base their anticipatory decision on subtle, yet deterministic, postural information that was available prior to the appearance of more obvious cues, such as the racket swing after contact and subsequent ball flight. Findings indicated that skilled players used advance cues occurring early in the opponent’s action sequence to predict depth (i.e., whether the shuttlecock would land closer or further away) more accurately than novices. Cues from the initial part of the flight of the shuttlecock, on the other hand, allowed them to better anticipate stroke direction. The spatial occlusion data also revealed differences in the location of cues used for deciding upon the outcome of the shot. Unskilled players relied mostly on the opponent’s racket action, while skilled players utilized information from the racket and racket arm to predict the outcome. In a similar example, Williams and Burwitz (1993) measured soccer players’ anticipation skill in a video simulation task using the temporal occlusion paradigm. In this study, goalkeepers were required to predict the direction of a series of penalty shots. Results showed that skilled goalkeepers could anticipate the In the temporal occlusion paradigm, typically, the participant views a video of a dynamic action sequence (e.g., an opposing tennis player hitting a groundstroke) filmed from a first-person perspective giving the impression of immersion within the game. The test film is edited to unexpectedly end at a crucial point in the dynamic action sequence (e.g., 120 ms before the opponent’s tennis racket makes contact with the ball) to remove all perceptual information from the screen. The participant’s task is usually to anticipate the outcome of the situation (e.g., what the opposing player/team will do next) based on the available information prior to occlusion. 2 In the spatial occlusion paradigm, the participant views a similar action sequence and performs a similar task to that in the temporal occlusion paradigm. However, rather than the test film ending unexpectedly, part of the information in the display is obstructed (i.e., using a dynamic mask) or removed and replaced with background information via digital software. 1


11/23/2009 3:19:15 PM

126

Ward et al.

future trajectory of the ball earlier and more accurately than novices. The data indicated that skilled goalkeepers used an anticipation strategy that relied on the extraction of perceptual information prior to the penalty taker’s foot contacting the ball (Williams & Burwitz, 1993). To gain some insight into the nature of expert cognitive processing as well as to more carefully identify task-relevant cues used by expert athletes, a number of researchers began to study the eye-movement behavior and visual search characteristics of experts across a range of sports. The first studies investigating search patterns in sport were conducted by Bard and colleagues (Bard and Fleury, 1976; Bard, Fleury, Carriere, & Hall, 1980). For example, Bard et al., (1980) analyzed the visual search patterns of expert and novice judges in gymnastics. Participants were shown films of gymnasts performing various routines and were asked to judge the performances and detect errors. Expert judges fixated on the gymnasts’ upper body, while novice judges located most of their fixations on the legs of the gymnasts suggesting that their decision about the quality of movement execution were based on different information. In two examples from a team sport, Williams and colleagues (Williams, Davids, Burwitz, & Williams, 1994; Williams & Davids, 1998) examined the relationship between search strategies, attention and skill level in experienced and less-experienced soccer players. Players watched large simulations of dynamic film sequences showing unfolding patterns of soccer play in either 11-versus-11-player, 3-versus-3-player or 1-versus-1-player situations. Players were asked to anticipate what happened next by physically responding in the direction of the anticipated play. In the 11-versus-11 and 1-versus-1 conditions experienced players employed a high frequency of fixations of short duration. In the 11-versus-11 condition, experienced players scanned between the player in possession of the ball and players ‘off the ball’ (i.e., in locations of the pitch not occupied by the player in possession) to aid in their tactical decision making. In the 1-versus-1 situation players scanned between the different regions of their opponent’s body with experienced players extracting information from more informative postural cues such as the hip, lower leg and ball regions. However, in the 3-versus-3 condition, there were no differences in search strategies between skill groups. A subsequent spatial occlusion paradigm demonstrated that experienced players used a peripheral information extraction strategy. Experienced players in this condition picked up information ‘off the ball’ without shifting their gaze via the use of peripheral vision suggesting that experienced players were able to flexibly adapt their processing strategy to the changing constraints of the situation. Although the collection of eye-movement behavior and the use of different search strategies provide some useful information about the nature of


11/23/2009 3:19:15 PM


127

skill-based differences in performance, there is still some ambiguity as to the relationship between visual behavior, the utility of fixated information, and the underlying cognitive processing supporting expert performance. For instance, when individuals fixate on the same stimulus there is no guarantee that they will see the same information, interpret it in the same way, or see or use that information at all. This highlights a key difference between the behavioral (i.e., looking—fixations) and perceptual-cognitive characteristics (i.e., seeing—processing) of the visual system (Abernethy, 1988) (for a review, see Williams, Davids, & Williams, 1999). In a series of studies, Ward, Williams and colleagues examined the development of perceptual-cognitive skills in soccer (Ward & Williams, 2003), the nature of the cognitive mechanisms responsible for superior performance (Ward, Ericsson, & Williams, 2009) and how they were acquired (Ford, Ward, Hodges, & Williams, 2009; Ward, Hodges, Starkes, & Williams, 2007). Elite and sub-elite, 9 to 18-year-old players from National-level academy and recreational soccer teams were assessed on their ability to ‘read the game’ via a battery of visual function and perceptual-cognitive skills tests (for a similar approach, see Helsen & Starkes, 1999). Tests included assessments of static and dynamic visual acuity, peripheral vision, stereoscopic depth sensitivity, pattern recall (i.e., of meaningful and random patterns of play), anticipation (i.e., of the outcome of a soccer situation via temporal occlusion), and situational assessment (in which players had to generate and prioritize the available options on the pitch). The results indicated that skill groups could not be reliably differentiated on tests of visual function and that situation assessment and anticipation skills were most predictive of skill level (Ward & Williams, 2003). Ward et al. (2007) further demonstrated that elite players acquired these skills primarily through more substantial investment in deliberate team practice activities, including increased time spent during practice sessions in tactical and strategic decision making. Recreational players, on the other hand, invested more time in playful activities. Ironically, however, data from a recent follow-up study showed that those who were selected from the elite group to receive a professional contract with the soccer club, four years after the original data collection, were not those who had solely accrued more deliberate team practice during their early years of participation. Only those who balanced play and practice, rather than independently pursued one of these activities at the expense of the other, progressed to the professional level (Ford et al., 2009). In a subsequent study, Ward et al. (2009) elicited concurrent verbal reports of participants’ thinking as they performed anticipation and situational assessment tasks. In the Ward and Williams (2003; see also, Ward et al., 2009, Exp. 1)


11/23/2009 3:19:15 PM

128

Ward et al.

paper, participants performed each task in separate paradigms using different stimuli. Also, the anticipation task used a temporal occlusion paradigm where the situational assessment task did not. In the Ward et al. (2009, Exp. 2) study, both anticipation and situational assessment tasks were performed simultaneously in response to one test film. Both experimental tasks were performed, first, using a temporal occlusion paradigm (i.e., without any visual-perceptual cues) requiring participants to rely upon their memory encoding of the stimulus prior to occlusion. The last frame of the dynamic action sequence was then presented back on screen as a single slide which acted as a perceptual cue and participants completed the experimental tasks a second time. This study explicitly tested predictions from LTWM theory (Ericsson & Kintsch, 1995) and more traditional views of expertise. Contrary to prior research, this study demonstrated that the accuracy of a participant’s anticipatory decision was positively related to the number of options generated (cf. Johnson & Raab, 2003; Klein, Wolf, Militello, & Zsambok, 1995) and to the accessibility and quality of the underlying cognitive representation. Other research in sport that has also recorded verbal protocols as a means to examine the underlying cognitive representation supporting successful performance suggests that skilled performance is supported by the ability to update initial plans by integrating contextual information on the fly. Where experts used both action plan (i.e., prior knowledge and plans about what to do in a given situation) and current event profiles (i.e., modifications of existing plans based on current environmental information), novices and less-skilled athletes primarily relied on less well-developed action plan profiles (e.g., McPherson, 1999a, 199b). The research by Ward and colleagues and McPherson is consistent with the development of LTWM skills that support superior domain-specific performance. A recent meta-analysis on perceptual-cognitive expertise in sport (Mann, Williams, Ward, & Janelle, 2007) summarized the findings of the sports research. Experts were more accurate in their decision making and were faster in anticipating opponents’ intentions, compared to less-skilled athletes. Moreover, the differences in decision accuracy observed between experts and novices were similar across sport type, irrespective of whether the participants engaged in interceptive, strategic or other types of sport. However, the nature of the stimuli and research paradigms used resulted in skill-based differences in the magnitude of effect; effect sizes increased as the mode of presentation (i.e., static, video simulation, and field) and research paradigm became more representative of the real-word task and environment in which athletes typically engaged. The majority of the research in the domain of sport has empirically demonstrated that skilled and expert athletes possess enhanced perceptual-cognitive


11/23/2009 3:19:15 PM


129

skills. Moreover, the related literature on deliberate practice in sport has also demonstrated that these skills are consistent with being acquired via sustained engagement in diligent efforts to improve (e.g., Ward, Hodges, Williams, & Starkes, 2004). This research suggests that training has an important role in producing superior performance and deliberate practice is the best predictor of individual differences in athletic performance (Ericsson & Ward, 2007). The obvious question to ask at this point, specifically if one is interested in acquiring the kind of perceptual-cognitive skills described above, is whether certain training methods and strategies can aid in the efficient development of expertise. A fundamental assumption of training research is that humans have the capacity to improve their performance and that any limitations that nature may impose to developing excellence in healthy individuals (e.g., non-clinical populations) can be dwarfed by sustained engagement in well-designed training and practice over a lifetime (Ward et al., 2004). Although we have accumulated a great deal of knowledge about expert-novice differences in sport and on the nature of processes and mechanisms that mediate superior performance, this alone does not provide sufficient information on how to develop and train future experts. Fortunately, a wealth of research has been amassed in the domain of sport using various approaches to training perceptual-cognitive skills that can inform practice in other domains. This is particularly timely because the training of athletes by sports coaches, as in other instructional domains, has frequently been based on the coach’s intuition about how athletes learn and/or what they need to improve, methods inherited from their own mentors, or on some fashionable approach advocated by their peers. Although such methods may or may not have led to success in their respective athletes and teams, in the main, methods used by coaches in the real world are rarely based on objective and/or empirical evidence. Moreover, training methods are often geared towards improving skills that are directly observable, such as technique, rather than on less-tangible, perceptual-cognitive skills that might improve one’s ability to ‘read the game’. Similarly, emphasis is often placed on immediate performance gains as opposed to long-term learning. The early research on training perceptual-cognitive skills in sport can be traced back to the 50s and 60s. Although this research coincided with classic research on chess expertise (e.g., deGroot, 1965) and focused on the acquisition of similar cognitive mechanisms (e.g., recall and recognition), it preceded the popularization of research on sports expertise by roughly two decades! For instance, Damron (1955) trained American Football high school players to memorize patterns of play using slide simulations of game scenarios. After training, which also included discussion of the advantages conveyed in each scenario/pattern, players recognized about 75% of the simulated plays. In addition, when tested on a


11/23/2009 3:19:15 PM

130

Ward et al.

tachistoscopic presentation of real game plays during a match, they were able to recognize 95% of the defensive patterns, providing some indication of the transferability of training. A similar study with high school American Football players examined differences between training with the use of film simulation and flash cards (Londerlee, 1967). Results revealed that simulation training was more effective and that players in the film simulation group were able to recognize patterns of plays significantly faster than the flash-card training group. The studies demonstrated that such training can improve players’ ability to recognize patterns of play and increase the speed with which they are recognized. In a similar vein to the research on sports expertise, the assumption was that such memory skills have an important role in developing superior performance. Although recall and recognition skills may be important for athletes, they do not capture the full spectrum of perceptual-cognitive skills (e.g., anticipation, decision making and planning) required for superior performance in actual real game situations. In order to train more central components of expert performance that would likely result in more effective transfer to the field, the research transitioned toward investigating training of game-related behaviors. These included the ability to anticipate their opponents’ intentions and to make optimal and timely decisions. One of the first studies that examined the ability to train more representative perceptual-cognitive skills, such as anticipation, was conducted in tennis (Haskins, 1965). Haskins used the temporal occlusion paradigm to train tennis players’ ability to anticipate the direction of an opponent’s return shot. Players were trained with the use of film simulation, in which sequences of tennis actions were presented from the player’s on-court perspective. The players viewed the film until the screen went blank, either one frame or eight frames after the opponent hit the ball with their racket. They were then prompted to predict the direction of the shot. Players significantly improved their response time to predict direction of the opponent’s shot. In a similar study, baseball players were trained to improve their batting skills using the occlusion paradigm (Burroughs, 1984). During training, players were shown partial film of pitchers throwing the ball to the plate. Each pitch was occluded five frames after the pitcher released the ball and participants had to predict the final location of the pitch. Feedback was provided at the end of each trial. After training, players were tested on real pitches and compared to a control group. Trained participants anticipated the ball’s final location more accurately than the controls. Burroughs made use of an innovative “occlusion helmet”, called the Visual Interruption System, in the post-test measurement—a real-world temporal occlusion device. The pitcher’s foot triggered a delayed signal to the helmet, which immediately occluded the batter’s vision and, therefore, denied the


11/23/2009 3:19:15 PM


131

batter visual information of the ball’s trajectory during the final part of the pitch. In a subsequent lab-based study involving American Football players, Christina, Baressi and Shaffner (1990) examined linebackers’ ability to anticipate offensive plays. Participants were shown video clips of offensive tight ends’ moves/actions from a linebacker’s perspective. The participants were required to react to the play using a joystick, and accuracy and speed of response were recorded. Players were able to improve response accuracy without increasing response time. While many of these training studies were pioneering and paved the way for subsequent research, there were a number of concerns about the quality of experimental procedures that bring into question the validity of the findings. Some studies, for instance, did not include a baseline measurement, thus making it difficult to conclude that improvements were reliably attributable to the intervention employed. Others did not include a control and/or placebo group making it difficult to determine whether improvement was the result of the training or, for instance, increased task familiarity. Furthermore, the majority of the studies did not assess if the observed improvements in performance transferred to the real world—a major hurdle if one is to assess the true utility of this research in improving actual game performance. More recently, researchers interested in training perceptual-cognitive skill in sport began to address these concerns. One of the first studies employing a more rigorous design was conducted by Farrow, Chivers, Hardingham, and Sacuse, (1998), who attempted to improve the anticipation ability of novice tennis players. Players in the training group physically responded to a series of tennis serves presented on film. They were trained to attend to physical cues and associate these with shot type and direction. They also received feedback throughout their training. The training group significantly improved the time it took to respond to each shot from pre- to post-test, whereas the placebo (i.e., who watched professional tennis on video and were asked questions on match content) and control groups did not show any signs of improvement. A similar study was conducted with field hockey players (Williams, Ward, & Chapman, 2003). Experienced outfield players were trained and tested on goalkeeping skills. Baseline performance was measured using film simulation, in which players attempted to save penalty flicks. The training group engaged in video simulation training in which they viewed video of players taking a penalty flick using a progressive temporal occlusion technique (where more or less information was eventually revealed to the trainee using temporally edited video film). Important cues and relevant information were identified in the training and participants were given feedback on their performance. The placebo group viewed a different instructional video, which focused on technical goalkeeping skills.


11/23/2009 3:19:15 PM

132

Ward et al.

The control group was only required to complete the baseline measurement and post-test. Only the training group significantly improved their performance, showing improved response times for both simulated and field conditions. These studies on perceptual-cognitive training in sport provide some evidence for the trainability of such processes and the causal relationship between training and performance. Although prior training studies have led to improvements in performance for the trained participants, many studies did not explicitly elucidate an expert model of the specific task to be trained that could act as the basis for subsequent training. Instead, participants were trained largely based on the identification of relatively deterministic biomechanical cues (e.g., hip rotation) and their association with the outcomes of action (e.g., ball landing location on court). A review of the extant research, however, indicates that the perceptual-cognitive strategies of experts and the way in which they were acquired may be an informative basis for training future experts. This extends the expert-performance approach (for an overview, see Ericsson & Ward, 2007) and has been termed an evidence-based approach to training (see Ward, et al., 2008). We refer to this method, henceforth, as Expert Performance-based Training (ExPerT). Williams, Ward, Knowles, and Smeeton (2002) conducted the first sports study to use the ExPerT method. They explicitly elicited the strategy used by skilled players on a given task, and then used that strategy as a basis for training less-skilled players on the same task. Their study was constructed in two parts: the first recorded expert-novice differences in tennis performance and process (i.e., eye-movement behavior) data while anticipating an opponent’s return shot. Expert tennis players were significantly quicker in responding to the opponent’s stroke (i.e., 140 ms faster than novices). Additionally, differences in gaze behavior were observed between the groups, with the skilled participants focusing longer on the central body areas and less on the racket and ball, as was the focus of the novices. Results supported previous studies in which experts demonstrated superior anticipatory skills and more effective search strategies compared to novices. The second part of the study provided training to the players, based upon an informal expert model derived primarily from the findings of Experiment 1. As in the first part of the study, players were initially measured on their ability to anticipate an opponent’s return shot via video simulation. Anticipation skill was also assessed in the same way in the real world on a tennis court (the simulation task was a replication of this real task in the laboratory). Two training groups were provided with an hour of training that included video simulation, occlusion training and on-court training. Furthermore, to examine the effect of instructional delivery method, one group was provided with explicit instructions and feedback,


11/23/2009 3:19:15 PM


133

while another group was required to learn by themselves with the use of highlighted relevant information. The placebo group did not receive this training. Instead, they watched a traditional coach-led instructional training video highlighting technique and strategy, while the control group was only measured on the pre- and post-tests. Results revealed that both training groups significantly improved their response accuracy on both the simulated and field task from preto post-test; no differences were found between the two training groups. Moreover, both the placebo and control groups did not show any signs of improvement. Importantly, participants in the training groups were able to transfer the acquired skills from the lab to the field. The ExPerT method—where an expert model is elicited first and subsequently used as the basis for training—has also been applied to training other perceptual-cognitive skills in sport, such as situational assessment (see Williams, Heron, Ward, & Smeeton, 2005). Although researchers have successfully used the ExPerT method to train novice and low-skilled athletes, much less research has been conducted on training more advanced and skilled athletes. Research of elite and sub-elite athletes in volleyball, basketball and tennis (Adolphe, Vickers, & Laplante, 1997; Farrow & Fournier, 2005; Smeeton, Williams, Hodges, & Ward, 2005) provide initial evidence that skilled athletes can improve their performance by training perceptualcognitive skills such as anticipation and decision making in an evidence-based manner consistent with the methods described above. For instance, Fadde (2006) trained collegiate baseball batters on baseball pitch recognition using temporal occlusion and video-based methods. In addition, Fadde examined far transfer of training gains via batting statistics from subsequent competitive games. Results indicated that players in the treatment group (i.e., interactive video training) performed significantly better in post-test real-game situations than players in the control group. Numerous questions, however, remain in relation to training skilled athletes, such as which method of delivery (e.g., implicit, explicit, analogy-based, etc.) is more efficient, the optimal length of training, and the specific benefits to performance when transferring to the field (Ward et al., 2008). To summarize, a number of exemplar studies were reviewed in this section to highlight the way in which expertise has been measured and trained in sport. The research suggests that skilled athletes possess superior anticipation and situation assessment skills and employ qualitatively different eye-movement behaviors that allow them to more efficiently and flexibly extract task-relevant cues from the environment to aid their tactical decision making. Moreover, research using verbal reports indicates that expert performance in sport is consistent with predictions from LTWM theory. The training literature and, in particular, research adopting the ExPerT method, has demonstrated that use of an expert model


11/23/2009 3:19:15 PM

134

Ward et al.

based on actual expert process and performance data can aid in the improvement and the training of athletes. We now provide an overview of research that has examined expertise and expert performance in other complex domains and highlight the development and use of similar evidence-based practices in training and instructional design.

Expertise and Training in Other Complex Domains To date, the scientific study of expert and superior performance has shown that it is possible to investigate the acquisition and structure of skilled performance in complex and challenging domains using representative tasks and translate our understanding into evidence-based training methods (e.g. Ericsson & Ward, 2007; Ward et al., 2008). Although this approach has been applied most comprehensively in the domain of sport, there are other non-game domains that are equally, if not more complex, arguably more challenging, stressful, and/or life-threatening, and that would benefit from the adoption of similar training methods. These include societally relevant fields such as law enforcement, health-care, and the military. These fields, amongst others, are similar in that they require intensive training prior to successful qualification, yet successful qualification does not always guarantee superior, or even adequate real-world performance. Furthermore, training levels are often not maintained after qualification and can result in stagnation, performance decline and skill decay. The main benefits of applying the ExPerT method in these domains are that researchers and instructional designers are forced to: (a) identify real-world tasks where performance is integral to successful operation with the domain; (b) specify the perceptual-cognitive skills, strategies, knowledge and associated cognitive representations that support superior performance on these tasks; (c) outline the training progression necessary to attain high levels of performance; and (d) create opportunities for ongoing deliberate practice activities that will facilitate skill maintenance and continued improvement over a person’s career. In many domains of expertise, it is difficult to define what constitutes, and therefore measure, expert performance. For instance, in law enforcement, officers engage in many tasks from report writing, to resolving domestic disputes, recognizing deadly threats, and discharging their firearm, the frequency and saliency of which drastically differ. Despite such difficulties, researchers have attempted to capture and measure superior performance in a number of complex domains. For instance, Ward and colleagues (Harris et al, 2006; Ward et al, 2007) developed a simulated task environment (STE) to assess performance on representative


11/23/2009 3:19:15 PM


135

law enforcement tasks. Novice (i.e., police recruits) and expert (i.e., SWAT qualified police officers) participants, equipped with a blank-firing handgun, were asked to talk aloud while interacting with 20, near life-size, video scenarios of incidents ranging from domestic disputes to a terrorist attack. Participants subsequently gave retrospective reports after some of the trials. In some of the trials the perpetrator gave up peacefully or the event did not require a lethal force response and in others the perpetrator drew and fired a lethal weapon. Performance on the latter ‘shoot trials’ was assessed by measuring the time taken to respond by firing a first shot. Three scenarios best discriminated between novices and experts and contributed similarly to the skill model. Experts were faster at responding on these trials, sometimes by a matter of seconds. The authors, however, were particularly interested in the behavioral and cognitive processes that supported successful performance and so groups were subsequently reclassified based on a within-task criterion (i.e., trial success). When the perpetrator was non-compliant and the situation continued to escalate, successful participants fired a shot. In contrast, unsuccessful participants either saw the threat early but did not pursue a similar course of action or saw the threat late and subsequently ordered participants to stand down. Both options, however, resulted in an unsuccessful outcome. The retrospective verbal report data indicated that successful participants attempted to predict the outcome prior to its occurrence based upon the evolving context and/ or evaluated the negative outcome of the threat. These data are consistent with LTWM theory (Ericsson & Kintsch, 1995). Superior performers, particularly successful experts, showed evidence that they had developed mechanisms that allowed them to create on-line representations of unfolding events, which they then used to plan, monitor, and make predictions about the occurrence and consequence of future events. These data could be used to create an expert model that guided instruction using the ExPerT method. In another law enforcement study—the Survival Scores Research Project (FLETC, 2004)—researchers investigated the performance of officers in a realistic scenario using live role players and simunition equipment (i.e., paintball). Trainees were initially paired with a senior officer who was also a driving instructor and confederate in the experiment. First, stress was induced by having participants complete routine then emergency high-speed driving maneuvers both as driver and passenger. When the senior officer was driving and the trainee was a passenger, the confederate officer deliberately drove erratically and appeared as if to lose control of the vehicle. Participants then immediately received a radio call to respond to a staged domestic dispute. Upon arrival at the scene, the senior and trainee officer were met by a sergeant, who instructed them to enter the house and take a report. After entering the house,


11/23/2009 3:19:15 PM

136

Ward et al.

the senior partner made a scripted mistake and his handgun was taken by a suspect. The suspect was one of two unarmed role players—the other was a cooperative complainant. Following the script, the suspect shot the senior partner and took the complainant hostage. The remainder of the event constituted the measurable component of the study: The trainees had to resolve the situation, which inevitably resulted in engaging the suspect in a gun battle inside the house. During the exchange, trainees encountered a scripted firearm malfunction that required them to clear an inert round of ammunition from the firearm before they could resume shooting. The gun battle typically continued for several minutes until the suspect terminated the scenario by collapsing on the floor. The results showed that the majority of participants failed to perform each of the assessed tasks adequately. For example, although the goal for each trainee was to hit the suspect with no less than 70% of shots fired, only 3% of participants were successful in meeting this criterion. Furthermore, only 31% of the participants dealt adequately with the firearm malfunction, and 19% shot the hostage. Based on trainee performance in the study, the authors recommended that highstress scenario-based training be introduced earlier in a recruit’s training, and that recruits be given more opportunities to participate. In another high-stakes domain, researchers interested in critical-care nursing expertise have attempted to address factors such as the relationship between content knowledge and the ability to implement and adapt such knowledge based on the availability of situational information. This is particularly important in domains were competency testing to assess entry into a domain is primarily conducted via the use of paper-and-pencil knowledge-based testing. Whyte, Ward, and Eccles (in press) examined differences in the knowledge and performance of novice and experienced critical-care nurses in a STE. The STE featured a lifesize, human patient simulator and a fully equipped, true-to-life, intensive care unit suite similar to that found in any hospital. Participants first completed a knowledge test related to protocols for dealing with respiratory events and then responded to four simulated respiratory-based scenarios. The results showed that although experienced nurses possessed highly superior knowledge when compared to novice nurses, this did not translate into reliable differences in actual clinical performance. Differences in performance in the simulated scenarios were observed only after nurses were reclassified into groups based on whether they had attained board certification. Board certified nurses were able to maintain the simulated patient’s physiology within acceptable bounds during the four challenging respiratory events. The superior performance demonstrated by these nurses was attributed to their engagement in highly demanding, solitary deliberate practice activities of the type required to achieve board certification.


11/23/2009 3:19:16 PM


137

In a review of the research on nursing expertise, Ericsson, Whyte, and Ward (2007) concluded that challenging scenarios utilizing sophisticated patient simulators provide ideal training opportunities for student nurses as well as experienced nurses. Old cases of varying complexity that have known correct actions (e.g., as demonstrated by expert performers on trials with successful outcomes that can be objectively verified) can be used as a basis for training unstable patient simulations, for instance, via the ExPerT method. Furthermore, the incorporation of human patient simulators into such methods allows low frequency events to be recreated and repeated with higher frequency in the laboratory. This offers opportunities for trainees to observe and monitor vital signs, administer treatments and medications, and permit errors to be made—which are necessary for learning— without catastrophic effects. In a training method introduced by Larew, Lessans, Spunt, Foster, and Covington (2006), nurses engaging with a patient simulator were provided with a series of increasingly informative hints that eventually allowed them to make the correct diagnosis and perform the actions necessary to bring about a successful resolution for the patient. Feedback and reflection on training scenarios of this type lead to clear and reproducible performance differences on subsequent training scenarios. Similarly, extended (i.e., 2–3 hours) individual training using nursing simulators has been associated with improved performance on the 15-station Objective Structured Clinical Examination when compared to a nontreatment control group (Alinier, Hunt, Gordon, & Harwood, 2006). Researchers in other real-world domains have also investigated complex performance using simulated tasks. For instance, Omodei, McLennan, Elliott, Wearing, and Clancy (2005) used a simulated scaled world program to create a complex firefighting environment in which command and control performance in wildland firefighting could be investigated. Participants received preliminary training in the functioning of the scaled world, which allowed for detailed creation of landscapes of varying composition, flammability characteristics, fire spread rates, and asset protection priorities. Participants were assigned the role of fireground commander and were in charge of two subordinate sector commanders (confederates). Their task was to control two operational sectors of wildland and minimize destruction due to fire. The confederate sector commanders were instructed to carry out only the actions (e.g., deploy firefighting resources to protect property, construct firebreaks, etc.) ordered by the fireground commander. The fireground commander sat in an isolated room and viewed the fire development on a screen while communicating with the sector commanders via two-way radios. The commander’s performance was measured by the percentage of landscape assets (e.g., forest, grasslands, and houses) saved from destruction at the end of each scenario.


11/23/2009 3:19:16 PM

138

Ward et al.

The authors found that the commanders performed worse when provided with complete and reliable information, suggesting that they were less able to prioritize the information presented to them, and that they spent too much time evaluating the information instead of formulating actions. Additionally, the commanders tended to make use of all available firefighting assets, which resulted in cognitive overload and, ultimately, performance degradation. These findings support Klein, Calderwood and Clinton-Cirocco’s (1985) observation that expert fireground commanders relied on their experience to identify suitable courses of action, rather than engaging in deliberate evaluation of all available options that might interfere with making a temporally constrained decision. These and similar studies have been used to inform instructional design and form the basis of subsequent training programs. A prime example of how this research has informed training can be seen in the creation of a law enforcement training regimen as part of the Survival Scores Research Project (FLETC, 2004). One of the key outcomes of this study was the development of the ‘Student Centered Feedback’ model for use during scenario training and assessment (Wollert, 2008). This model involves the instructor eliciting retrospective verbal reports at the conclusion of the each scenario in order to establish the cues perceived by the trainee, the thought processes underlying the decisions they made and the consequent actions taken. Trainees are then asked to identify the positive aspects of their performance, and any alternative courses of action that may have been preferable in that situation. Finally, trainees are given the opportunity to correct any performance deficiencies by repeating the same type of scenario—a classic example of deliberate practice through the use of repetition and feedback provided by a coach. Similar methods have been implemented in military contexts (for a review see Ward et al., 2008). For example, Klein (1997) developed an approach to training military personnel, termed decision skills training, which is designed to increase a person’s experience base. According to Klein, a broader experience base should facilitate more accurate decision making. Thirty squad leaders from the US Marine Corps received decision skills training for a few hours a week over three and half months. Typically, trainees were presented with a text-based description of a situation and made time-limited, action-based decisions, which were subsequently disseminated to subordinate role players. A series of reflective, post hoc exercises were used to critique, probe and evaluate the decisions made. For example, in a decision making critique activity, trainees reflected on the decisions they faced during training scenarios by asking questions such as, ‘what made this decision tough?’, ‘what one piece of information would have helped you the most?’, and ‘what other actions did you consider and why didn’t you choose


11/23/2009 3:19:16 PM


139

them?’ Changes in decision-making behavior were not assessed directly, although subjective ratings indicated that trainees and their supervisors felt that the training was beneficial. Similarly, Freeman and Cohen (1997a, 1997b) developed critical thinking skills training for military officers, which focused on metacognitive skills such as the ability to critique and reason about decision options. Two experiments were conducted with Navy officers using computer simulations. The impact of the training on decision-making processes and decision outcomes was measured. The officers improved their ability to critique assessments and made more accurate situation assessments without any negative impact on their confidence. While the training developed by these studies to improve decision making and critical thinking skills is creative and intuitively appealing, the assessment of the validity and reliability of some of these methods is still in its infancy. Examples of training methodologies that have explicitly adopted approaches that are compatible with ExPerT, in other complex, non-sport domains include Cognitive Engineering Based on Expert Skill (CEBES; Staszewski & Davidson, 2000) and the Event-based Approach to Training (EBAT; Dwyer, Oser, & Fowlkes, 1995; Fowlkes, Dwyer, Oser, & Salas, 1998; Stout, Salas, & Fowlkes, 1997). The CEBES method, grounded in the theory and principles of cognitive science, first derives an expert model from empirical evidence, which then serves as a blueprint for training. Following this approach, Staszewski (1999) developed an expert model of mine-detection clearance operations that would subsequently be used as the basis for training. An information processing analysis of expert performance on a mine detection task identified the specific equipment manipulations and performance-specific perceptual information, knowledge, and thought processes required for successful task performance. Using this information, Staszewski and Davidson (2000) conducted a study to determine whether training based on this information would improve the performance of trainees that were at an intermediate level of skill. Twenty-two soldiers with standard mine detection training were randomly assigned to training and control groups. A special training and testing facility was constructed for this study using simulated mines buried beneath the ground surface. Participants used a hand-held metal detector to locate the mines, and placed marking chips on the ground to indicate the location of detected mines. Both the training and control groups completed a pre-test to determine mine detection probability (PD) and were found to be equivalent (PD = 0.58 and 0.55 respectively), although their performance was well below that required by the military standard (PD = 0.92). Following the pre-test, the experimental group received five days of training developed from the prior analysis of an expert’s mine detection skill (see Staszewski,


11/23/2009 3:19:16 PM

140

Ward et al.

1999). Training included correct use of detection equipment, identification of perceptual cues used to guide detection, and training in the necessary domain-specific knowledge, thinking, and temporal organization of skills employed by the expert mine detector. Computerized and video feedback was given to participants immediately after each training drill. At the conclusion of the training program, both the training and control group completed a post-test on an unfamiliar test area. The training group improved their performance (PD = 0.94), exceeding the military standard, while the control group showed no improvement. Three weeks later, eight soldiers from the experimental training group completed a transfer experiment at a different facility using real, deactivated mines. The soldiers had not practiced their detection skills in the interim but, prior to the transfer test, completed a series of refresher drills designed to familiarize them with the new environment. The soldiers’ performance in the transfer test (PD = 0.97) was comparable to their initial post-training test performance. It is difficult to ascertain whether the sustained improvement was accounted for by the additional familiarization training received or whether the results truly reflected learning. Regardless, use of this method does appear to be an effective way to improve soldiers’ ability to detect mines and, most importantly, improve their ability to detect the most threatening types of mines (see also Staszewski, 1999). In the EBAT, representative events from the real world are first identified such that specific behavioral responses that should ideally occur in each instance can also be identified. This allows the researcher to standardize stimuli and scenario presentation across individuals, and specify when certain scenarios should be introduced in training. The requirement for performance outcomes to be specified a priori relative to a highly specific event means that it would be relatively straightforward to provide trainees with more objective measurements of their performance, rather than subjective ratings. In an evaluation of the EBAT, Stout et al. (1997) trained a group of student Navy pilots. The training was designed to improve teamwork in the cockpit, and focused on the pilots’ knowledge, attitude and context-specific skill competencies. Forty-two, similarly experienced, student pilots were randomly assigned to two-person teams either in a training or control condition. Specific situated problem scenarios were developed (e.g., loss of aircraft on a radar, engine malfunction, etc.) to train the pilots’ communication, assertiveness and situational awareness skills, and behavioral responses specific to each event within the training scenarios were specified a priori. Feedback on performance in these scenarios was given if the desired behavior was not exhibited at the appropriate point during training. Training was conducted over two days and also included classroom-based lectures and demonstrations, and roleplay and simulated exercises in which the aforementioned skills were practiced.


11/23/2009 3:19:16 PM


141

The content of the training program was based on a prior task/needs analysis of teamwork behaviors and skills (Prince & Salas, 1993). The control group did not complete any training. At the conclusion of training, participants completed an evaluation flight scenario in their pairs. Trained teams correctly performed between 8 and 13% more of the scenario events than the control group. In addition to the behavioral evaluation data, participants also completed an attitudinal questionnaire either before and after training (i.e., training group) or before and after evaluation (i.e., control group). Both groups also completed a knowledge test prior to evaluation. Results showed a small, positive post-training increase (e.g., approximately 0.27 increase on a 5-point scale across 4 subscales) in the training group’s attitudes to teamwork in the cockpit, while no change was found for the control group. The trained group scored significantly higher on the knowledge test, and rated the training as very, if not extremely, useful to their job performance. Many of these methods appear to hold much promise for improving the performance of future experts in complex domains. The CEBES method and EBAT, in particular, are similar to the ExPerT method—primarily used in sport to develop perceptual-cognitive skill (e.g., Williams et al., 2002). Collectively these approaches provide subtle variations of context-sensitive, perceptual-cognitive skills training, many of which are evidence-based, that is consistent with the type of diligent and deliberate practice activities in which experts engage. Moreover, the situational and dynamic nature of this type of training is consistent with the development of integrated cognitive representations that can be updated in real time and that have been shown to be responsible for successful and superior performance in complex domains (Ericsson & Kintsch, 1995; McPherson, 1999a, 1999b; Ward et al., 2009).

Summary In this article we presented a brief summary of the theoretical development in the broader expertise literature followed by an overview of the research on expertise, expert performance, and perceptual-cognitive and decision skills training in sport and other complex domains, such as law enforcement, nursing and the military. The sports literature used numerous paradigms, including temporal and spatial occlusion, and eye-movement and verbal report recordings to demonstrate that experts develop superior anticipation and situational assessment skills that are supported by cognitive representations consistent with the acquisition of long-term working memory skills. We outlined the ExPerT method


11/23/2009 3:19:16 PM

142

Ward et al.

and showed how expert data collected from performance during representative and simulated tasks have been used to train perceptual-cognitive skills in an evidence-based manner. The data suggest that the skills that are characteristic of the expert advantage can be trained, at least in part, using this method. The literature reviewed on expertise in other complex domains, primarily using simulated task environments, showed that the way in which experts succeed is qualitatively different from their less skilled counterparts, and the skills and strategies necessary for success are dependent on similar cognitive representations and deliberate practice regimens to those identified in sport. Although the focus and content of training has been more varied in non-sport, complex domains, many of the approaches used are consistent with the ExPerT method. This method is characteristically based on a situation-specific, expert model or data and the training is dynamic, context-sensitive, and can be objectively validated with known outcomes. Training programs using the ExPerT method are created not only to mimic the real-world demands faced by individuals within the domain, and to provide opportunities for deliberate practice on both challenging and low frequency cases, but, importantly, to develop the type of skills and integrated cognitive representations that underpin expert performance in complex domains.

References Abernethy, B. (1987). Review: Selective attention in fast ball sports II: Expert-novice differences. Australian Journal of Science and Medicine in Sport, 19(4), 7–16. Abernethy, B. (1988). Visual search in sport and ergonomics: Its relationship to selective attention and performer expertise. Human Performance, 4, 205–235. Abernethy, B., & Russell, D. G. (1984). Advance cue utilisation by skilled cricket batsmen. Australian Journal of Science and Medicine in Sport, 16(2), 2–10. Abernethy, B., & Russell, D. G. (1987). Expert-novice differences in an applied selective attention task. Journal of Sport Psychology, 9, 326–345. Adolphe, R., Vickers, J., & Laplante, G. (1997). The effects of training visual attention on gaze behavior and accuracy: A pilot study. International Journal of Sports Vision, 4(1), 28–33. Alinier, G., Hunt, B., Gordon, R., & Harwood, C. (2006). Effectiveness of intermediate-fidelity simulation training technology in undergraduate nursing education. Journal of Advanced Nursing, 54, 359–369. Allard, F., & Starkes, J. L. (1980). Perception in sport: Volleyball. Journal of Sport Psychology, 2, 22–53. Allard, F., Graham, S., & Paarsalu, M. L. (1980). Perception in sport: Baskeball. Journal of Sport Psychology, 2, 14–21. Allard, F., Parker, S., Deakin, J., & Rodgers, W. (1993). Declarative knowledge in skilled motor performance: Byproduct or constituent? In J. L. Starkes & F. Allard (Eds.), Cognitive issues in motor expertise (pp. 95–108). Amsterdam: Elsevier.


11/23/2009 3:19:16 PM


143

Bard, C., & Fleury, M. (1976). Analysis of visual search activity during sport problem situations. Journal of Human Movement Studies, 3, 214–222. Bard, C., Fleury, M., Carriere, L., & Halle, M. (1980). Analysis of gymnastics judges visual search. Research Quarterly for Eexercise and Sport, 51, 267–273. Borgeaud, P., & Abernethy, B. (1987). Skilled performance in volleyball defense. Journal of Sport Psychology, 9, 400–406. Burroughs, W. A. (1984). Visual simulation training of baseball batters. International Journal of Sports Psychology, 15, 117–126. Charness, N. (1976). Memory for chess positions: Resistance to interference. Journal of Experimental Psychology: Human Learning and Memory, 2, 641–653. Chase, W. G., & Simon, H. A. (1973a). Perception in chess. Cognitive Psychology, 4, 55–81. Chase, W. G., & Simon, H. A. (1973b). The mind’s eye in chess. In W. G. Chase (Ed.) Visual information processing (pp. 215–281). New York: Academic Press. Christina, R. W., Barresi, J. V., & Shaffner, P. (1990). The development of response selection accuracy in a football linebacker using video training. The Sport Psychologist, 4, 11–17. Damron, C. F. (1955). Two and three dimensional slide images used with tachistoscopic training techniques in instructing high school football players in defenses. Research Quarterly, 26, 26–43. de Groot, A. (1965). Thought and choice in chess. The Hague: Mouton. Dwyer, D. J., Oser, R. L., & Fowlkes, J. E. (1995). A case study of distributed training and training performance. Proceeding of the Human Factors and Ergonomics Society 39th Annual Meeting, Santa Monica, CA, 1316–1320. Ericsson, K. A., & Kintsch, W. (1995). Long term working memory. Psychological Review, 102, 211–245. Ericsson, K. A., & Staszewski, J. (1989). Skilled memory and expertise: Mechanisms of exceptional performance. In D. Klahr & K. Kotovsky (Eds.), Complex information processing: The impact of Herbert A. Simon (pp. 235–267). Hillsdale, NJ: Erlbaum. Ericsson, K. A., Whyte IV, J., & Ward, P. (2007). Expert performance in nursing: Reviewing research on expertise in nursing within the framework of the expert-performance approach. Advances in Nursing Science, 30, E58–E71. Ericsson, K. A., & Ward, P. (2007). Capturing the naturally-occurring superior performance of experts in the laboratory: Toward a science of expert and exceptional performance. Current Directions in Psychological Science, 16 (6), 346–350. Ericsson, K. A., Patel, V. L., & Kintsch, W. (2000). How experts’ adaptations to representative task demands account for the expertise effect in memory recall: Comment on Vicente and Wang (1998). Psychological Review, 107, 578–592. Fadde, P. J. (2006). Interactive video training of perceptual decision-making in the sport of baseball. Technology, Instruction, Cognition, and Learning, 4(3), 265–285. Farrow, D., & Fournier, J. (2005). Training perceptual skill in basketball: Does it benefit the highly skilled? In Proceedings of the 11th World Congress of Sport Psychology: Promoting Health & Performance for Life. Sydney, Australia: International Society of Sport Psychology. Farrow, D., Chivers, P., Hardingham, C., & Sacuse, S. (1998). The effect of video-based perceptual training on the tennis return of serve. International Journal of Sport Psychology, 29, 231–242. Federal Law Enforcement Training Center (FLETC). (2004). Survival scores research project. Glynco, GA: Author. Fischer, B., & Glanzer, M. (1986). Short-term storage and the processing of cohesion during reading. The Quarterly Journal of Experimental Psychology A: Human Experimental Psychology, 38, 431–460. Ford, P., Ward, P., Hodges, N. J., & Williams, A. M. (2009). The role of delibrate practice and play in career progression in sport: The early engagement hypothesis. High Ability Studies. 20(1), 65–75


11/23/2009 3:19:17 PM

144

Ward et al.

Fowlkes, J., Dwyer, D. L., Oser, R. L., & Salas, E. (1998). Event-based approach to training (EBAT). International Journal of Aviation Psychology, 8, 209–221. Freeman, J. T., & Cohen, M. S. (1997a). Training for complex decision making: A test of instruction based on the recognition/metacognition model (Tech. Rep.). Naval Air Warfare Center Training Systems Division. Orlando, FL. Freeman, J. T. & Cohen, M. S. (1997b). Training for complex decision-making: A test of instruction based on the recognition/metacognition model. In Proceedings of the 3rd International Command and Control Research and Technology Symposium 1996. Monterey, CA. Frey, P. W., & Adesman, P. (1976). Recall memory for visually presented chess positions. Memory & Cognition, 4, 541–547. Glanzer, M., Dorfman, D., & Kaplan, B. (1981). Short-term storage in the processing of text. Journal of Verbal Learning & Verbal Behavior, 20, 656–670. Glanzer, M., Fischer, B., & Dorfman, D. (1984). Short-term storage in reading. Journal of Verbal Learning & Verbal Behavior, 23, 467–486. Gobet, F. & Simon, H. A. (1996). The roles of recognition processes and look-ahead search in timeconstrained expert problem solving: Evidence from grandmaster level chess. Psychological Science, 7(1), 52–55. Goulet, C. Bard, C., & Fleury, M. (1989). Expertise differences in preparing to return a service serve: A visual information processing approach. Journal of Sport and Exercise Psychology, 11, 382–398. Harris, K. R., Tashman, L., J., Ward, P., Ericsson, K. A., Eccles, D. W., Williams, A. M., Ramrattan, J. & Hassler, L. (2006). Planning, evaluation, and cognition: Exploring the structure and mechanisms of expert performance in a representative dynamic task. In Proceedings of the 28th Annual Conference of the Cognitive Science Society, 26–29th July (pp.327–332). Vancouver, BC, Canada: CSS. Haskins, M. J. (1965). Development of a response-recognition training film in tennis. Perceptual and Motor Skills, 21, 207–211. Helsen, W. F., & Starkes, J. L. (1999). A multidimensional approach to skilled perception and performance in sport. Applied Cognitive Psychology, 13, 1–27. Johnson, J. G., & Raab, M. (2003). Take the first: Option-generation and resulting choices. Organizational Behavior and Human Decision Processes, 91, 215–229. Kintsch, W. (1998). Comprehension: A paradigm for cognition. New York: Cambridge University Press. Klein, G. A. (1997). Developing expertise and decision making. Thinking and Reasoning, 3, 337–352. Klein, G. A., Calderwood, R., & Clinton-Cirocco, A. (1985). Rapid decision making on the fire ground (KA-TR-84-41-7). Yellow Springs, OH: Klein Associates Inc. Klein, G., Wolf, S., Militello, L., & Zsambok, C. (1995). Characteristics of skilled option generation in chess. Organizational Behavior and Human Decision Processes, 62, 63–69. Larew, C., Lessans, S., Spunt, D., Foster, D., and Covington, B., (2006). Innovations in clinical simulation: Application of Benner’s Theory in an interactive patient care simulation. Nursing Education Perspectives, 27, 16–21. Londerlee, B. R. (1967). Effect of training with motion pictures versus flash cards upon football play recognition. Research Quarterly, 38, 202–207. Mann, D., Williams, A. M., Ward, P., & Janelle, C. M. (2007). Perceptual-cognitive expertise in sport: A meta-analysis. Journal of Sport & Exercise Psychology, 29, 457–478. McPherson, S. L. (1999a). Expert-novice differences in performance skills and problem representations of youth and adults during tennis competition. Research Quarterly for Exercise and Sport, 70, 233–251. McPherson, S. L. (1999b). Tactical differences in problem representations and solutions in collegiate varsity and beginner female tennis players. Research Quarterly for Exercise and Sport, 70, 369–384.


11/23/2009 3:19:17 PM


145

Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice-Hall. Omodei, M. M., McLennan, J., Elliott, G. C., Wearing, A. J., & Clancy, J. M. (2005), ‘More is better?’: A Bias Toward Over-use of Resources in Naturalistic Decision Making Settings’, in Montgomery, H., Lipshitz, R. and Brehmer, B. (Eds.), How Professionals Make Decisions (pp. 29–41). Mahwah, NJ: Lawrence Erlbaum. Prince, C., & Salas, E. (1993). Training and research for teamwork in the military aircrew. In E. L. Weiner, B. G. Kanki, & R. L. Helmreich (Eds.). Cockpit resource management. (pp. 337–366). Orlando, FL: Academic Press. Simon, H. A., & Chase, W. G. (1973). Skill in chess. American Scientist, 61, 394–403. Smeeton, N. J., Williams, A. M., Hodges, N. J., & Ward, P. (2005). The relative effectiveness of various instructional approaches in developing anticipation skill. Journal of Experimental Psychology: Applied, 11, 98–110. Staszewski, J. (1999). Information processing analysis of human land mine detection skill. In T. Broach, A. C. Dubey, R. E. Dugan, & J. Harvey (Eds.), Detection and remediation technologies for mines and mine-like targets IV, Proceedings of the Society for Photo-Optical Instrumentation Engineers 13th Annual Meeting, SPIE, 3710, 766–777. Staszewski, J., & Davison, A. (2000). Mine detection training based on expert skill. In A. C. Dubey, J. F. Harvey, J. T. Broach, & R. E. Dugan (Eds.), Detection and remediation technologies for mines and mine-like targets V, Proceedings of Society of Photo-Optical Instrumentation Engineers 14th Annual Meeting, SPIE, 4038, 90–101. Stout, R. J., Salas, E., & Fowlkes, J. E. (1997). Enhancing teamwork in complex environments through team training. Group Dynamics: Theory, Research, and Practice, 1, 169–182. Vickers, J. N. (1996). Control of visual attention during the basketball free throw. American Journal of Sports Medicine, 24, S93–S97. Ward, P., Ericsson, K. A., & Williams, A. M. (2009). Complex perceptual-cognitive expertise in a simulated task environment. Manuscript submitted for publication. Ward, P., Farrow, D., Harris, K. R., Williams, A. M., Eccles, D. W., & Ericsson, K. A. (2008). Training perceptual-cognitive skills: Can sport psychology research inform military decision training? Military Psychology, 20, S71–S102. Ward, P., Harris, K. R., Ericsson, K. A., Eccles, D. W., Tashman, L., & Lang, L. (2007). Cognitive basis for expert and superior performance in law enforcement. In D. S. McNamara & J. G. Trafton (Eds.), Proceedings of the 29th Annual Meeting of the Cognitive Science Society (p. 1884). Austin, TX: Cognitive Science Society. Ward, P., Hodges, N. J., Starkes, J. L., & Williams, A. M. (2007). The road to excellence: Deliberate practice and the development of expertise. High Ability Studies, 18(2), 119–153. Ward, P., Hodges, N. J., Williams, A. M., & Starkes, J. L. (2004). Deliberate practice and expert performance: Defining the path to excellence. In A.M. Williams and N.J. Hodges (Eds.), Skill acquisition in sport: Research, theory and practice (pp. 232–258). London: Routledge. Ward, P., & Williams, A. M. (2003). Perceptual and cognitive skill development in soccer: The multidimensional nature of expert performance. Journal of Sport and Exercise Psychology, 25, 93–111. Whyte, J., Ward, P., & Eccles, D. W. (in press). The relationship between knowledge and clinical performance in novice and experienced critical care nurses: An application of the expert performance approach. Heart and Lung: The Journal of Acute and Critical Care. Williams, A. M., & Burwitz, L. (1993). Advance cue utilization in soccer. In T. Reilly, J. Clarys, & A. Stibbe (Eds.), Science and football (Vol II, pp. 239–244). London: E & FN Spon. Williams, A. M., & Davids, K. (1995). Declarative knowledge in sport: A by-product of experience or a characteristic of expertise?. Journal of Sport and Exercise Psychology, 7, 259–275. Williams, A. M., Davids, K., Burwitz, L., & Williams, J. G. (1993). Cognitive knowledge and soccer performance. Perceptual and Motor Skills, 76, 579–593.


11/23/2009 3:19:17 PM

146

Ward et al.

Williams, A. M., Davids, K., Burwitz, L., & Williams, J. G. (1994). Visual search strategies in experienced and inexperienced soccer players. Research Quarterly for Exercise and Sport, 65, 127–135. Williams, A. M., Davids, K., & Williams, J. G. (1999). Visual perception and action in sport. London: E & FN Spon. Williams, A. M., Heron, K.,Ward, P., & Smeeton, N. J. (2005). Using situational probabilities to train perceptual and cognitive skill in novice soccer players. In T. Reilly, J. Cabri, & D. Arajuo (Eds.), Science and football V (pp. 337–340). Taylor & Francis: London. Williams, A. M., Ward, P., & Chapman C. (2003). Training perceptual skill in field hockey: Is there transfer from the laboratory to the field? Research Quarterly for Exercise and Sport, 74, 98–103. Williams, A. M., Ward, P., Knowles, J., & Smeeton, N. J. (2002). Anticipation skill in a real-world task: Measurement, training, and transfer in tennis. Journal of Experimental Psychology: Applied, 8, 259–270. Wollert, T. (2008). The right kind of feedback. FLETC Journal, 6(1), 44–50. Zwaan, R. A., & Radvansky, G. A. (1998). Situation models in language comprehension and memory. Psychological Bulletin, 123, 162–185.


11/23/2009 3:19:17 PM