Proceedings of HF 2002, Nov. 25-27, 2002, Melbourne, Australia
Learning and Transfer in an Applied Visual Spatial Task Shayne Loft, Andrew Neal and Michael Humphreys
Key Centre for Human Factors and Applied Cognitive Psychology
University of Queensland
[email protected]
Keywords: Learning, transfer, air traffic control, training, context Abstract This paper presents a new dynamic visual spatial task for use in applied cognition research. The aim of the experiment reported is to illustrate a major limitation of learning from individualized examples - the inability to transfer across different contexts. Instance-based models of learning emphasize the role that memory for previous examples plays in subsequent task performance and the predictions were based on this framework. The task required participants to decide as quickly and as accurately as possible whether pairs of aircraft moving on the screen would come within 1 cm of each other (conflict). During training the surface features of the items were held constant, and during transfer they were changed. Changing the spatial configuration of the aircraft had the largest negative impact on performance, followed by orientation and then position. The results illustrate some key ways in which episodic memory influences performance in a dynamic visual spatial environment. The results reported raise some avenues for further enquiry. Design implications are discussed.
1. Introduction This paper presents a new visual spatial task, based on air traffic control, for use in applied cognition research. The experiment reported in this paper is the first in a series of studies concerning how people learn from individualized examples.
1.1. Learning from examples Performance improvements usually occur after repeated performance on the same task. Practice involves the presentation of new and old examples of problems that need to be classified or solved. There are two general classes of theories concerning how individuals learn from examples; abstraction and instance models of learning. These two distinct approaches carry different assumptions regarding the casual mechanisms responsible for improvements in performance associated with task practice (Estes, 1986; Logan, 1988; Anderson, 1983). Abstraction models of learning claim that performance improvements result from abstract representations of statistical regularities within presented practice examples (Anderson, 1983; Rumelhart & Ortony, 1977). That is, individuals develop rules based on information abstracted from each successive learning example. Instance models of learning emphasize knowledge derived from individualized learning episodes (Estes, 1986). A basic premise present in all instance models of learning is that individuals retrieve previously encountered examples, and use this information for subsequent decisions (e.g., Hinztman, 1986; Logan,
Proceedings of HF 2002, Nov. 25-27, 2002, Melbourne, Australia
1988). That is, individuals use memories for past examples to solve new examples, rather than abstracting and applying general rules to new examples. A wide variety of models share the assumption that the retrieval of previous examples is essential for learning and these models have been applied to cognitive task domains such as judgment (Kahneman & Miller, 1986), problem solving (Ross, 1984), and categorization (Allen & Brooks, 1991).
1.2 Transfer of cognitive skill Kimball and Holyoak’s (2000) taxonomy of skill transfer and expertise draws clear distinctions between structural and surface features of practice examples. To illustrate, consider the following air traffic control example. In this example, a pair of converging aircraft is flying at the same altitude toward a 105-degree common angle of intersection. One of the aircraft is traveling at 1000km per hour and the other at 500km per hour. Controllers would use the speeds and relative positions of the aircraft to make a conflict status decision, also taking into account the angle at the point of intersection. These are structural features of the item, because they are functional and goal relevant. The item also contains non-functional and goal irrelevant features considered surface in nature. Surface features are commonly referred to in the learning and transfer literature as context. Surface features include the position of the aircraft event on the radar screen, the orientation of the angle of intersection and the configuration of the aircraft. The key point regarding surface features is that they make absolutely no difference to conflict status, constituting only the context in which the aircraft are presented. Instance models of learning emphasize the role of memory for previously presented examples. According to such models (e.g. Hinztman, 1986; Logan, 1988), each time a training example is presented, the entire processing episode is stored as a separate memory trace. This trace contains information concerning the presenting structural conditions of the example (speeds, relative position of the aircraft, and angle of intersection) and associated conflict status (conflict/non-conflict). When presented with a new example, individuals can retrieve traces of previously seen examples. The speeds, relative position and angle of intersection of the current pair of aircraft can be compared to the retrieved traces to make an accurate conflict status decision. An assumption present in all episodic learning models is that each trace includes the particular context in which the processing operations were carried out. Instance learning is inherently dependent on memory, and context changes can have a powerful effect on memory. When presented with a new example, the retrieval of previous examples is dependent on the match between the current context and previous contexts (Tulving & Thomson, 1973). Task performance should be sensitive to the surface features of items, even if these features are irrelevant to the outcome of items. While transfer should be very good to new items presented in the same or similar context as training, learning will most likely not transfer well beyond the original context. With sufficient contextual variation, individuals might not use the knowledge derived from training because they think they are looking at a completely new type of item, based on superficially dissimilar features.
1.3. The air traffic control task The task required participants to predict the conflict status of pairs of aircraft, based on their speeds, relative position, and angle of intersection. The training conditions in this experiment were optimal for memory retrieval and instance learning. That is, we used low variability examples, in terms of both structural and surface features, during training. Participants were expected to benefit from the repeated presentation and quickly establish strong memory representations for training examples. Training conditions were not considered diverse enough to allow generalization mechanisms to abstract rules from the examples presented.
Proceedings of HF 2002, Nov. 25-27, 2002, Melbourne, Australia
During training, we presented participants with two pairs of aircraft at set speeds and angles of intersection. Participants were required to decide whether each pair of aircraft would conflict, as quickly and as accurately as possible. Figure 1 presents an example of a training trial. On each of the 14 training trials, items with the same structural (speeds, angle of intersection) and surface features (position of the item as a whole on the screen, orientation, and configuration) were presented. The relative positions of the aircraft differed on every training trial. This determined conflict status, making the outcome of the item either a conflict or a non-conflict. On the test trial, we presented them with a series of items containing different surface features than the trained items. The test items were identical to the training examples structurally (speeds, angle of intersection). Surface features in the task were superficial in nature and had no impact on the conflict status of items. Surface features simply provide the context in which the structural and goal-relevant features of the item take place. There were four types of transfer items presented on the transfer trials
Figure 1: The training task
Trained items were exactly the same as the item presented during training in terms of both structural and surface features. For position items, the position of the item was moved. The aircraft speeds and angle of intersection are the same as training and the item is simply moved to a new location on the screen. For orientation items, the rotation of the angle was altered by 180 degrees. For configuration items, the configuration of the item was altered. For example, for item B in Figure 1, the 1000 km/hr aircraft is moved to the upper flight path and the 500 km/hr aircraft is moved to the lower flight path. It was predicted that the participants would become both faster and more accurate at making conflict status decisions over training. For the transfer trials, it was predicted that the configuration surface change would have the largest negative impact on accuracy and reaction time of conflict status decisions, followed by the orientation change and then the position change. These transfer trial predictions were based on the reasoning that changing the configuration of the item represents the largest surface level change, followed by changing the orientation, followed by changing the position.
2. Method 2.1. Task and apparatus Figure 1 illustrates the task interface that was used during training. Small green circles symbolize aircraft. Each aircraft has a letter and a speed shown on a green flight strip. Aircraft fly on set flight paths. Many of
Proceedings of HF 2002, Nov. 25-27, 2002, Melbourne, Australia
the flight paths cross at some point on the screen. These intersection points provide a set of angles. Angles are replicated at different points on the screen at different degrees of rotation. Aircraft are in conflict when a five nautical mile separation standard (1 cm on the computer screen) is violated. When in conflict, aircraft symbols and flight strips turn yellow, and they turn green again once the five nautical mile separation standard is re-established. All items consist of a pair of aircraft traveling along different flight paths that intersect. Conflicts occur if the two aircraft reach the intersection at approximately the same time and violate the 1cm or five nautical mile separation rule. The development of conflicts and non-conflicts was standardized across all speed and angle of intersection combinations using an automated application tool. Participants were presented with two sets of training trials. There were 14 trials in each training set and the duration of each trial was one minute and fifty seconds. The two training sets are labeled AB and CD. Participants were also presented with two sets of analogous transfer trials. There were 16 trials in each transfer set and the duration of each trial was 30 seconds. The two transfer sets are labeled AB and CD and correspond to the training sets. Half the participants were presented with training set AB, transfer set AB, training set CD and then transfer set CD. The other half of participants were presented with training set CD, transfer set CD, training set AB and then transfer set AB. Training sets AB and CD consisted of two pairs of aircraft. Each pair of aircraft was heading toward a common point of intersection. The structural and surface features of the pairs of aircraft remained constant on each of the 14 trials in the training set. The details of the structural features of the four training items (A, B, C & D) are presented below in Table 1. The relative starting positions of the pairs of aircraft in each training trial were altered to produce the conflicts and non-conflicts. The term range refers to how long, in seconds, the pair of aircraft takes to conflict or pass each other safely. The ranges used during training were 85, 90, 95 and 100 seconds. This ensured that participants could not search for patterns in starting positions of aircraft to determine conflict status. Item Type Item A Item B Item C Item D
Speed of Pair km/hr 1200 and 300 1000 and 500 1100 and 400 900 and 600
Angle of Intersection 120 degrees 75 degrees 105 degrees 60 degrees
Range (seconds) 85, 90, 95, 100 85, 90, 95, 100 85, 90, 95, 100 85, 90, 95, 100
Table 1: Details of the four training items
For the two transfer sets AB and CD, single pairs of aircraft were presented to participants. Participants had 30 seconds to determine the conflict status of each item presented before the aircraft disappeared and the next item was presented. Each transfer item had a range of 85 seconds. They were run for 30 seconds, from a range of 85 seconds to 55 seconds. The 16 items contained in transfer sets AB and CD were presented in blocks of 4; with a 20 second break between each block. In transfer set AB, for example, the 16 items consisted of trained items A and B, position item A and B, orientation items A and B and configuration items A and B. On both the training and transfer tasks, the scoring system encouraged both fast and accurate performance. Participants responding early were rewarded with more points, but if incorrect, were penalized with the same score.
2.2. Participants and procedure Thirty-five first year psychology students volunteered to participate in return for course credit. Firstly, participants were given instructions on how to complete the training trials. They then completed the
Proceedings of HF 2002, Nov. 25-27, 2002, Melbourne, Australia
training set. Participants were then given instructions for how to complete the transfer set and completed it. This was followed by a 10-minute break. After the break participants completed the second set of training and test materials.
3. Results 3.1 Training Figures 2 presents accuracy over training, collapsed over the 4 different types of training items (A, B, C and D). All training data analyses were carried out within a repeated measures analysis of variance. The effect of training was significant, [F (1,34) = 4.74, MSE = .134, p
