Considering Perception and Cognition in the Design of an ... - CiteSeerX

Beichner, R. (1995). Considering perception and cognition in the design of an instructional software package. Multimedia Tools and Applications, 1, 173-184.

Considering Perception and Cognition in the Design of an Instructional Software Package

Robert J. Beichner North Carolina State University Physics Department Raleigh, NC 27695-8202

Abstract An instructional multimedia software package was developed for use by students taking introductory physics courses. Studies have indicated that many of these students possess a set of common misunderstandings of graphs describing the motion of objects. The software described here was constructed as a student tool which would specifically address these difficulties. The impact of educational psychology, cognitive science, and human factors research on software design and user interface development are described. This work was supported by NSF grant MDR-9154127 with additional support from the RasterOps Corporation, Sony Corporation of America, and Apple Computer, Inc.

Considering Perception and Cognition in the Design of an Instructional Software Package

Introduction This paper describes the influence of educational psychology, cognitive science, and human factors research in the development of an instructional software package for use by introductory physics students. Certain previously well-defined student difficulties were targeted for remediation. The prevalence of these problems and their resistance to traditional instruction motivated the entire development effort. Current understanding of how people use computers in instructional settings was supplemented with observations of users interacting with pilot versions of the program. This had a significant impact on screen appearance and suggested the types of user interactions that were finally built into the software. The mechanism for the package’s basic functionality stems from a desire to take advantage of the strong response of the visual cortex to objects moving in the visual field. People naturally pay attention to motion and are able to perceive slight changes in the position of very small objects, even against complex backgrounds. This input from the field of neurology, coupled with a recognition of the effectiveness of temporally correlated stimuli in promoting cognitive linkages, resulted in a unique design [1] which is now being replicated by other instructional software developers [2,3]. The goal was to approach what Rutkowski [17] calls the principle of transparency-when the tool seems to disappear and concentration is focused on the task itself.

The Problem to be Addressed Introductory physics students need to understand how graphs can be used to describe the motion of objects. Graphs provide a concise method of presenting data that can reveal trends in data while not obscuring the details. They are an especially important part of kinematics, the study of motion. Physics teachers, assuming students can gather meaning from graphs, often use them as a means of instruction. Unfortunately, students’ graph interpretation skills are generally poor. So instruction based on this “language” often results in a lack of understanding of kinematics variables (position, velocity, and acceleration) and the graphs representing them [5]. The problems students have in understanding kinematics graphs have been described in McDermott, Rosenquist, & van Zee [10] and van Zee & McDermott [22]. These studies clearly demonstrated that students taking introductory high school and college

level physics classes understand graph construction techniques, but fare poorly when asked to explain the concepts conveyed by the graphs. Clement et al. [8], Mokros & Tinker [15], and Bell et. al. [6] indicate that two graphing errors are very common. The first of these might occur when a student is asked to draw a velocity versus time graph of an object rolling down a hill. Many students produce incorrect graphs which look like the hill traversed by the object. It is easy to see how the path of the object is mistakenly taken as a cue in drawing the graph. This type of error may indicate that students view graphs as something concrete rather than as an indicator of abstract trends. It is commonly referred to as the “graph as picture” error. The second misinterpretation commonly seen is a confusion of the slope of a line and a point on the line. For example, students asked to find the place of maximum change in a graph (i.e. where the slope is steepest) sometimes indicate the point of largest value, where the change can actually be zero. A more recent study replicated these findings and also found that students have great difficulty determining slopes of lines that do not pass through the origin as well as interpreting the meaning of areas under graphed kinematics curves [5]. Microcomputer-based labs, abbreviated MBLs, have been quite successful in addressing many of these problems [21]. Brasell [7] discovered that it was the immediacy of real-time collection and display of data in graphical form that resulted in a significant impact on learning. Real-time graphing lets the student process the event and its graph simultaneously rather than sequentially. Since working memory has a limited capacity and retention time [12], the simultaneous presentation of event and graph “makes the most” of the cognitive facilities available. This should make it easier to transfer the event-graph unit (already linked together) into long-term memory as a single entity. Perry & Obenauf [16] suggest that this temporal alignment is important to reasoning about motion. More generically, Shuell [19] notes that “Contiguity (the proximity of two events) is well established as one of the fundamental variables affecting traditional types of learning” (pg. 426). Fleming [9] notes that “Side-by-side placement invites comparison. Criterial information is contrasted between the two, increasing its saliency” (p. 245). Since one of the major misinterpretations being addressed is the belief that a kinematics graph is essentially a picture of the event, displaying the two next to each other allows students to see that this is actually not the case. While examining real-time graphing on a perceptual level, Brasell [7] points out that movement in a computer display tends to capture students’ attention, thus causing them to selectively attend to the important parts of the graph, where changes in the physical situation cause changes in the graph. The initial stages of motion perception actually take place in the retina and do not share general mental resources like memory and attention

[10]. Because of this they are “cognitively impenetrable” and aren’t greatly influenced by higher-level processes. Thus it is very difficult to ignore something that is moving or somehow changing. Early examples of instructional software would often have a word repeatedly flashing on the screen–literally to the point of distraction. VideoGraph was designed to take advantage of this attention-focusing mechanism to direct student attention to important information.

Description of the software These ideas from the psychology of perception suggest that it might be possible to present the motion event on the computer screen beside the relevant graphs and help students make the cognitive linkages between the two. That, in fact, is the major function of the software to be discussed in this article. As is suggested by the name “VideoGraph,” the package displays a video of a motion event alongside a relevant kinematics graph. Students begin by clicking on the locations of up to twelve objects as they move from frame to frame in the video window. This allows the software to record the position of each object to within the resolution available on the screen. In this case, the monitor displays 72 pixels per inch. Because a small window enlarges the area around the cursor, students are able to easily mark object locations to the nearest pixel. Once this is done the video must be calibrated to actual dimensions and times. This is done by dragging the cursor through a known distance on the video. Typically, a meterstick is placed so that its image is recorded in at least one frame. After the user moves the cursor from one end to the other of the standardizing object in the video, its size is entered via a dialog box. As long as the object of known size is in the same two dimensional plane as the motion being studied, it permits a determination of the scaling factor between the event that was recorded and its computerized display. The “realworld” equivalent of a pixel is determined by the focal length of the videocamera lens and the distance from the lens to the moving object(s). The time between frames is also typed into the calibration dialog box. This is only a function of the number of frames skipped in transferring the video from tape to the computer since videocameras (at least in the US) capture a frame every 30th of a second. This is discussed later in more detail. Once the moving objects have been marked and the video calibrated, the video and graph windows are automatically synchronized so that as the video plays, a marker on the graph indicates the point associated with the video’s currently displayed motion. In other words, as an object moves in the video window, a small circle jumps from point to point on the graph. For students having difficulty attending to both video and graph simultaneously, replaying the video multiple times, stopping and starting its playback,

and moving forwards and backwards a frame at a time enables them to make the cognitive linkages between what the object is doing and how the graph represents that. Equally important as going from video to graph is the reverse–linking a graphed point to the relevant video frame. The user interface accomplishes this by being sensitive to mouse clicks directly on the graphed datapoints. Clicking on any given point causes a nearly instantaneous change in the video window so that it displays the frame that generated that data. Actually it is a bit more complex than this since velocity graph points are average values determined by the change in position between two adjacent video frames. The software causes the second of the pair of frames to be displayed. Although the time axis of the graph indicates that the calculated velocity happens at the midpoint (in time) between the two frames, this is a subtle point which students have not noticed without it being pointed out to them. This particular approximation, that the average velocity happens at the mid-time, is only true when the acceleration of the moving object is constant. For the small time interframe intervals normally studied with this software, this is a reasonable simplification. VideoGraph allows students to see both motion and graph without the overhead of microcomputer-based lab interfacing equipment and setting up a satisfactory motioninvolving situation. Although this is sometimes an advantage, VideoGraph was not intended to replace MBL experiences for students, only supplement them. It is no substitute for real world experience. Nonetheless, it is difficult to categorize VideoGraph since, in addition to opening up the possibility of examining complex prerecorded situations, it can also be used by students to analyze motion events which they themselves have videotaped. It is not real-time like other MBL hardware and software packages since there is a delay between taping the event and analyzing it. But working with images transferred to computer disk does allow students to work on projects outside the lab classroom. Not only does this open up a much larger set of possible motions to study (videodisc scenes of bungee jumping, televised ballet dancing, videotapes of athletic competitions, etc.) it also permits more time for reviewing these situations. Laboratory time is often too short to allow examination of a wide variety of motion events.

Insert Figure 1 near here

Figure 1 makes it clear that the central task of the software is to help students link the motion event with its graph. Data representing the vertical velocity of the athlete’s center of mass has been graphed. (The position of the center of mass is calculated by the software once the locations of various parts of the athlete’s body have been marked by the process described earlier.) The user has set up the video window to display several different objects at once. Notice also that the video is indicating a series of markers for the objects being displayed. It is actually showing object positions for all the frames in the entire video sequence. This provides an easy mechanism for viewing the complete path of each object, getting around the limitation that a single video frame can only show objects at one instant of time. The current position of the object being graphed is surrounded by a bright circle on the video. This capability can be toggled on and off so that users can view a single marker for each object, corresponding to the particular frame actually being displayed at any given time. Normally it is turned on since students report that being able to see where the object is along its path is quite helpful. This combination of ways to travel back and forth between visual and graphical representations has its foundation in Sternberg’s [20] ideas of knowledge acquisition, three components of which are relevant here: (1) selective encoding–picking out relevant information for further processing, (2) selective combination–putting information together in a way which has meaning for the learner, and (3) selective comparison–noting relationships between new information and old information. Letting students select the technique(s) they prefer for linking video and graph should promote all three ways of acquiring knowledge. We have reason to expect [8] that the unusual parts of the graphs and motion events will draw the attention of the students. Since changes in motion are typically keys to understanding the causes of motion, the software takes advantage of students’ visual perception to direct them to exactly what they need to see to help them build their own (correct) ideas about kinematics and related graphs. Kolers & Brison [13] comment that design features such as color and contrast not only serve to bring content topics to mind, but are themselves retained for the long term like the semantic or linguistic features of words describing the content. In general, visual stimuli have been found to be superior to verbal stimuli in children’s retention and recall: learners recall more information when responding in pictorial form than in verbal [11]. So being able to quickly jump back and forth between a video presentation of the motion and its graphical representation should be an effective way to help students understand the relationships between the two. The software includes several additional features which encourage comparisons between different parts of the motion and the graphs and between different graphs. For

example, it is possible to measure distances in the video and relate this data to position graphs. This can also be compared to areas under velocity graphs as another way of conveying the same information. Similarly, it is simple to have the software determine the slope of the position graph for comparison to a velocity graph which itself is only a single mouse click away. Student users are guided by the software and accompanying handouts to examine important parts of the graphs (maxima and minima, intercepts, places of uniform or changing slope, etc.). This, combined with the synchronization between graph and motion, helps them locate the “action” in the kinematics event that causes those particular aspects of the graphs. It might be instructive to examine how the software can be used to address a specific graph misinterpretation that students bring to the classroom. The graph as picture error noted earlier is a prime candidate for correction using the videographic technique. Students are asked to examine a graph of y position vs. x position. This is actually a direct representation of the object’s path and holds no surprises for the student since that is what they were expecting. They are then asked to compare this to a y position versus time kinematics graph. This can be quite different in appearance, as can be seen in figure 2. Students are then asked to find more situations where the two graphs look different and other situations where they are similar in appearance. For example, the center of mass of the jumper in figure 1 follows a parabolic path. The y position kinematics graph is also parabolic. This type of exploration gives students an opportunity to develop a deep understanding of what determines the shapes of kinematics graphs. So not only is the original misinterpretation directly challenged, but corrected conceptions of kinematics are encouraged.

Insert Figure 2 near here

Software Response to User Input During the design phase it was decided that a nearly instantaneous response to user control was important so that students could grasp the impact of their actions. For example, students can click the mouse to change their marking of object location and instantly see the modification in the graph. Of course, this makes it simple for them to correct marking errors but more importantly, it also lets them see how object location is

intimately involved in shaping the graphs. Setting a new coordinate origin while watching the graphs gives insight into which kinematics variables depend on this value (position) and which don’t (velocity and acceleration). Rotating coordinate axes not only makes it easier to analyze some types of motion, but also lets students see how the x and y components of the kinematics variables relate to the coordinate system which has been established. Similarly, a change in their calibration indicates the sensitivity of the software to marking errors. Shneiderman [18] and others have noted that direct manipulation has positive benefits in interface design, so this feature was incorporated into the software. For example, the contents of the graph window which can be changed are responsive to mouse clicks. Figure 3 indicates the different controls built into the graph window. (1) Popup menus are available when pressing the mouse down on the labels of the graph axes. Students can select from x or y position, x or y velocity, or time for either graph axis. (2) A popup menu also appears when the student presses the mouse down on the graph title. This changes the data to be graphed to the student’s choice of any one of twelve objects or the system’s center of mass. (3) Clicking the mouse on a number along one of the axes brings up a small box for typing in a new value for that number. This lets students set the smallest and largest values to be graphed and also set the spacing of grid lines on the graph. (4) As noted earlier, clicking on a graphed datapoint causes an instant updating of the video window to reflect that instant of time. (5) A small button in the lower left corner allows students to smooth the data. (6) The entire graph window can be resized following normal operating system interface techniques. This may seem trivial, but since students often do not recognize that the appearance of a graphed line depends on the horizontal and vertical scaling factors, this is actually something worth having them explore.

Insert Figure 3 near here

The menubar was created after considering the order of tasks taken up during normal operation of the software. Names of menus and items were carefully chosen to reflect the logical purpose of the commands and were kept as concise as possible. Shneiderman [18] cites several researchers who showed that simpler menus made for greater ease of use. The software also automatically enables and disables menu items, depending on whether they can appropriately be selected. For example, if object locations have not

been marked or calibration information has not been set, then it is not possible to view the data or a graph and the relevant menu items are disabled. Similarly, if a graph window is not open, students cannot select the slope or area items under the Measure menu. In some cases the name of the menu item changes to fit the situation. For example, the Print item changes from “Print This Video Frame” to “Print Graph” to “Print Data” depending on which window is on top of the others. By adjusting the menubar in this limited manner, students are helped to remember what each item does.

Video Capture Originally the video analysis software also had the capability to capture video images from videotape or videodisc and store them onto magnetic computer disks. This added considerably to the complexity of the package, so this portion was moved to a separate utility program called VideoGrab. The goal for this piece of software was to let students quickly acquire video for analysis with VideoGraph. The program was made as simple as possible. There are only two menus, File and Edit. The video controller window was made to look like the familiar VCR control panel, as can be seen in figure 4.

Insert Figure 4 near here

To use VideoGrab, students first have to name the movie file they are creating. Then a dialog box lets them configure automatic image capture, specifying not only how many frames will be “grabbed,” but also how many frames of the original video source are to be skipped between captures. For example, if a long sequence is being captured, it might be useful to sample every third frame. This would make the time between frames 0.10 seconds and would reduce the number of frames that would need to be marked for analysis. Then the video controller is used to advance to the beginning of the desired sequence and the record button is pressed. The video source is automatically advanced through the frames while the software directs the video board to capture images, compress them, and then store them on disk. Operation is the same whether students are collecting images from a videodisc player or from a computer-controlled VCR. This has proven to be quite successful. Tests with students show that they spend approximately five minutes using VideoGrab to capture a single set of images. Earlier non-automated

software required considerably more time and effort and generally distracted from the analysis tasks.

Coding Techniques VideoGraph and VideoGrab were both written using Symantec’s Think C for the Macintosh. As mentioned above, they were originally a single application, but students were getting confused about the VCR controls for capturing and the movie controls for playing back an already recorded movie. Splitting the programs separated the two tasks and greatly simplified things for the students. QuickTime was used to handle display and control of the video sequences. This addition to the Macintosh operating system has a large assortment of built-in features and capabilities. Unfortunately (for this project, at least) it is not frame oriented. It records information that changes with time. Although this is a more general technique, it led to complications in working with video which changes regularly every thirtieth of a second. For example, the user interface aspect that allows clicking on a graphed data point to jump to the relevant video frame was not trivial programming. An earlier calculation, done during calibration, was done to determine the conversion factor between time in the QuickTime movie and video frame number. Once that was done it was a fairly simple task to instruct QuickTime to display any desired frame. When playing a movie it works in the opposite direction. As QuickTime displays a particular part of the movie, it can be asked to report the current video time. This was converted to a frame number which was used to determine the location of the corresponding marker on the graph. (These coordinates were stored in an indexed array of points which is updated any time a change is made to the graph window’s appearance). The time required to erase the old marker and display the new one is done so quickly that there is no noticeable reduction in movie playback speed. Although this particular aspect of QuickTime made for more complex coding, other capabilities more than made up for the programming difficulties. For example, high quality video compression comes automatically with QuickTime. Even the dialog boxes that allow users to set the type and amount of compression are built in. Other standard dialog boxes allow control of video capture boards, making adjustment of contrast, brightness, and color saturation of the images easy for the user. This allows VideoGrab, via QuickTime, to be hardware independent. Any video capture board that follows the QuickTime VDIG specifications can be controlled. The only code that was hardware dependent were those routines controlling the video sources. Luckily, those are also somewhat standardized. Commands are sent out one serial port to control any Pioneercompatible videodisc player. A different set of commands, sent through the other serial

port, are used to operate any ViSCA-capable video device. (ViSCA is Sony Corporation’s acronym for their “Video System Control Architecture.”) It should also be noted that there was actually two way communication through the serial ports with video source devices since frame number or elapsed tape time was displayed in the video controller window.

Summary As is necessary with any piece of software, field trials were undertaken to see where users ran into problems and to listen for suggestions to improve the interface. This was done in three high schools and two college classrooms. Input from several dozen teachers was also solicited as they worked with the software in workshop settings. Other teachers and graduate students and support staff helped develop curricular materials to supplement the software. Earlier testing of similar software [4] indicated that it must be embedded in a curriculum designed to take advantage of it in order for it to be successful. A preliminary analysis of specialized testing indicates that students using VideoGraph and VideoGrab in enriched curricular settings were able to perform better on a standardized test of kinematics graph interpretation [5]. Although details of this analysis are not appropriate for this article and will be reported in another publication, the results are germane. Students using the software in a variety of classrooms with different teachers had significantly better kinematics graph interpretation skills than students exposed to more traditional curricula. Software that takes advantage of student perceptual and cognitive abilities was able to help students change their ideas about a targeted content area. Physics education research presents a rich source of challenges to be addressed by designers of instructional software. These suggestions from research can be coupled with ideas from the psychology of learning and perception to suggest guidelines for presenting information to students, ranging from the overall functionality of programs to specifics about how users should interact with data. This project’s overall goal of helping students make the cognitive linkages between actual motion events and the abstract kinematics graphs which represent them has been realized. This appears to be a fruitful way to develop software and is recommended as a model for instructional designers.

References 1. Beichner, R., DeMarco, M., Ettestad, D., & Gleason, E. (1989). VideoGraph: A new way to study kinematics. In E. Redish & J. Risley (Ed.), Computers in Physics Instruction (pp. 244-245). Raleigh, NC: Addison-Wesley.

2. Personal communication with Mark Luetzelschwab, Dickinson College, June 14, 1994. 3. Personal communication with Jack Wilson, Rennsalaer Polytechnic Institute, September 12, 1994. 4. R. Beichner, “The Effect of Simultaneous Motion Presentation and Graph Generation in a Kinematics Lab,” Journal of Research in Science Teaching, 27, 803-815, 1990. 5. R. Beichner, “Testing student interpretation of kinematics graphs,” American Journal of Physics, in press. 6. A. Bell, G. Brekke, and M. Swan, “Misconceptions, Conflict, and Discussion in the Teaching of Graphical Interpretation,” Paper presented at the Second International Seminar in Misconceptions and Educational Strategies in Science and Mathematics, Cornell University, Ithaca, NY, 1987. 7. H. Brasell, “The effect of real-time laboratory graphing on learning graphic representations of distance and velocity,” Journal of Research in Science Teaching, 24, 385-395, 1987. 8. J. Clement, J. Mokros, and K. Schultz, “Adolescents’ graphing skills: A descriptive analysis,” Paper presented at the annual meeting of the American Educational Research Association, 1986. 9. Fleming, M. (1987). Displays and communication. In R. Gagné (Ed.), Instructional Technology: Foundations (pp. 233-260). Hillsdale, NJ: Lawrence Erlbaum. 10. W. Hendee and P. Wells (Eds.), The Perception of Visual Information, SpringerVerlag: New York, 1993. 11. A. Howe and E. Vasu, “The Effect of Two Modes of Sensory Input on Children's Retention and Recall,” Paper presented at the annual meeting of the National Association for Research in Science Teaching, Atlanta, Georgia, 1990. 12. S. Hulse, H. Egeth, and J. Deese, The Psychology of Learning (5th ed.), McGrawHill: New York, 1980. 13. P. Kolers and S. Brison, “Commentary: On Pictures, Words, and Their Mental Representation,” Journal of Verbal Learning and Verbal Behavior, 23, 105-113, 1984. 14. L. McDermott, M. Rosenquist, and E. van Zee, “Student difficulties in connecting graphs and physics: Examples from kinematics,” American Journal of Physics, 55, 503-513, 1987. 15. J. Mokros and R. Tinker, “The impact of microcomputer-based labs on children’s ability to interpret graphs,” Journal of Research in Science Teaching, 24, 369-383, 1987.

16. B. Perry and P. Obenauf, “The acquisition of notions of qualitative speed: The importance of spatial and temporal alignment,” Journal of Research in Science Teaching, 24, 553-565, 1987. 17. C. Rutkowski, “An introdution to the human applications standard computer interface, part 1: Theory and principles,” Byte, 7(11), 291-310, 1982. 18. B. Shneiderman, Designing the User Interface: Strategies for Effective HumanComputer Interaction, Addison-Wesley: Reading, MA, 1992. 19. T. Shuell, “Cognitive Conceptions of Learning,” Review of Educational Research, 56, 411-436, 1986. 20. R. Sternberg, Mechanisms of cognitive development, Freeman: New York, 1984. 21. R. Thornton and D. Sokoloff, “Learning motion concepts using real-time microcomputer-based laboratory tools,” American Journal of Physics, 58, 858-867, 1990. 22. E. van Zee and L. McDermott, “Investigation of student difficulties with graphical representations in physics,” Paper presented at the Second International Seminar in Misconceptions and Educational Strategies in Science and Mathematics, Cornell University, Ithaca, NY, 1987.

List of Figures

Figure 1. This screen shows VideoGraph tracking an athlete’s right forearm, right thigh, and overall center of mass. The video and the graph are linked. Playing the video moves the black circle along the graph. Clicking on a graphed datapoint instantly advances the video to that specific frame. Figure 2. Here we see VideoGraph displaying the path of a bungee jumper’s fall, along with a pair of graphs. One replicates the path and the other, a true kinematics graph, does not. Figure 3. The mouse-sensitive control areas of the graph window are indicated by the numbers. (These numbers are for illustrative purposes only and do not appear during student use of the software.) The circle at the 1 second mark along the curve indicates the graphed data point which is related to the currently displayed video frame. This circle moves along the curve as the video is played. Figure 4. VideoGrab’s Video Controller allows computer operation of a videodisc player or ViSCA–capable VCR.

❸

❺