118
IEEE TRANSACTIONS ON EDUCATION, VOL. 50, NO. 2, MAY 2007
Impact Assessment of a Microprocessor Animation on Student Learning and Motivation in Computer Engineering Ken Ferens, Member, IEEE, Marcia Friesen, and Sandra Ingram, Member, IEEE
Abstract—This paper reports on the impact of using custom animation software to teach second/third year computer/electrical engineering students in the microprocessing systems course at the University of Manitoba, Winnipeg, MB, Canada. Over the span of 13 years’ experience with teaching the course, the difficulties and limitations with conventional lectures and visual aids led to the development of custom animation of the course material to provide an additional teaching modality to teach the complex and abstract subject matter more effectively. The custom animation software consists of the ability to create and/or modify microinstructions, create and/or modify macroinstructions, and animate the execution of instructions (using the “water-flowing through pipes” analogy) by showing address and data transmission juxtaposed against an animated clock. A postunit, mixed method survey administered to students reveals substantial cognitive gains and modest motivational outcomes, reinforcing the animation’s effectiveness and sparking motivation for further research. Index Terms—Computer animation, courseware, macroinstruction, microinstruction, microprocessor, microprogrammability, mixed method survey, postunit.
I. INTRODUCTION
C
OURSEWARE is receiving increasing attention in engineering education for its potential to enhance teaching of complex subject matter. Defined as computer-based educational material used to assist students in their learning process and instructors in their pedagogy [1], courseware can take many forms, including software-based tutorials, intelligent tutoring systems, self-assessment tools, interactive design tools, virtual labs and simulations of physical lab equipment and/or experiments, and interactive animations of physical processes. When implemented, courseware generally complements (rather than replaces) existing learning activities in the engineering classroom and lab. Conventional teaching modalities, such as narration (lecture), visual images (pictures in books, in class), or written
Manuscript received October 6, 2005; revised November 16, 2006. This work was supported by a University of Manitoba TLT Grant from University Teaching Services. K. Ferens is with the Department of Electrical and Computer Engineering, University of Manitoba, Winnipeg, MB R3T 5V6, Canada (e-mail: ferens@ee. umanitoba.ca). M. Friesen and S. Ingram are with Design Engineering, University of Manitoba, University of Manitoba, Winnipeg, MB R3T 5V6, Canada (e-mail:
[email protected];
[email protected]). Digital Object Identifier 10.1109/TE.2007.893172
text (reading books) are becoming increasingly inadequate to handle the rising complexity of computer engineering subject matter. In response, many instructors have been turning to animation. Animation is an additional and complementary modality that specifically addresses the complexity of the course material. Benefits include coherence with teaching and learning theory which commonly indicates that multiple teaching modalities support the multiple learning styles of students in the classroom. Furthermore, the interactive nature of animation, in which students input data and parameters and immediately see their effects in the animation, support both active and constructivist theories of learning, in which students are actively engaged in creating and integrating their own new knowledge with their existing knowledge, and where they receive instant feedback on their engagement. This study was a formative evaluation of students’ experiences with animation courseware used in a third-year computer engineering course in Microprocessing Systems. Specific objectives were: 1) to develop an animation videographically showing the internal workings of a microprocessor as an additional teaching modality by which to enhance students’ conceptual understanding of underlying principles; 2) to develop the animation in ways that are supported by teaching and learning theory; and 3) to assess the cognitive and motivational outcomes in students as a result of using the animation in learning. As an initial study, the results will guide further development of both the animation and the course curriculum. In computer engineering, physical components are generally too small and run too fast for students to observe physically their actual working. Processes and components are also often too complex to put into one or two dimensions (visual images and written text). Consequently, the modeling, simulation, and/or animation of these complex processes in space and time are important. In this project, the animation addressed students’ fundamental understanding of a microinstruction, how a microprocessor executes microinstructions, how microinstructions are sequenced to form macroinstructions, and the effect of changing or adding new microinstructions to form new macroinstructions. The animation brings dynamic visualization to this subject matter which printed material lacks. This paper is organized in the following manner. First, a literature review discusses key findings of related research on the use of animation in teaching and learning complex subject matter in engineering. This section also provides a brief description of the available animation tools and their limitations. Second, the features and structure of the animation software are described.
0018-9359/$25.00 © 2007 IEEE
FERENS et al.: IMPACT ASSESSMENT OF A MICROPROCESSOR ANIMATION
This description includes functional, architectural, and interactivity aspects. Third, a typical laboratory experiment is presented, which shows how the animation facilitates the teaching and learning of new instructions. This procedure is followed by a description of the application of a survey, which serves to assess the efficacy of the animation software. Following, the results are presented and analyzed. Finally, conclusions are drawn and future work is discussed.
II. LITERATURE REVIEW The standard approach to reinforcing classroom lectures on microprocessing systems is to provide laboratories with development hardware and software that offer register and memory view and modification, breakpoints, and tracing capability [2]. The major limitation with that approach is that the system is not capable of tracing through microcode and thus leaves students with the assumption that microcode is unimportant or does not exist. Another approach is to use a software simulator, [3], [4]. Some of these simulators have a graphical user interface (GUI), providing considerable control over many attributes, such as the ability to assemble source code, modify registers and memory locations, trace through macroinstructions, and set conditional breakpoints. However, electrical and computer engineering students need to understand the deeper concepts of microoperations, microcode, control vectors, and control memory. Furthermore, most of these simulators do not provide a graphical depiction (i.e., animation) of address and dataflow, relying on the student’s imagination. Yurcik has provided useful simulation surveys [5]–[7], which discuss additional simulation tools, some of which provide animation. However, these animation tools seem to lack the ability to show intuitively and graphically the execution of a processor’s microoperations. In particular, what is lacking in these tools, and what is important for student comprehension, is the ability to create microoperations, to configure microoperations by establishing control vectors, to add new macroinstructions by defining a sequence of new microinstructions, to view the flow of address and data through the components of the microprocessor, and to provide an animation of the clock showing the synchronization of address and dataflows with clock events. This paper develops animation software that addresses these shortcomings in existing available tools. The animation developed in this study was designed taking into account principles of multimedia learning developed by Mayer and Moreno [8], [9], who define animation as “a simulated motion picture depicting movement of drawn or simulated objects” [8, p. 88] and whose main features are 1) pictorial representation; 2) motion or apparent movement; and 3) simulation, or objects that are artificially created through drawing or some other simulation method. Using a cognitive science approach to multimedia learning, Mayer and Moreno’s design principles (not including principles related to animation with spoken narration, which this animation did not use) include 1) multiple representation, which states that an explanation in words accompanied with pictures communicates ideas better than words alone; 2) spatial contiguity, which states that deeper learning happens when corresponding text and animation are presented
119
near rather than far from each other on the screen; 3) coherence, which states that deeper learning happens when extraneous visual and auditory effects are excluded; and 4) personalization, which states that deeper learning occurs when on-screen text is conversational rather than formal. The animation also reflects many of the best practices in instructional design, software design, and courseware-use extracted from Premier Award courseware led by NEEDS—A Digital Library for Engineering Education [10]. These best practices in the area of instructional design relate to clear statement of learning objectives, active learner involvement, applicable and transferable learning, well structured content, and multimedia components. Best practices in software design relate to learner engagement, effective interface and navigation, and technical reliability. Finally, best practices in use and assessment call for use in appropriate settings and attention to assessment and evaluation through valid and appropriate methods. The literature review reveals consistent motivations by instructors who add courseware as a teaching tool. Motivations for simulations and animations of laboratory experiments often address the need to be cost-effective when increasing student access to laboratory experience, otherwise hampered by limited equipment and high equipment-to-student ratios [11]–[13]. Instructors are also motivated to capitalize on specific features of courseware that are limited in traditional lecture and textbook-oriented teaching, such as active and interactive elements, instantaneous feedback on performance, focused guidance on corrective actions, dynamic visual representations of processes and phenomena, and the user’s ability to modify input parameters or operating conditions and immediately observe the effects in a “safe” environment (i.e., without jeopardizing costly laboratory equipment) [14]–[17]. Underlying these motivations are instructors’ goals for enhanced student outcomes: greater and lasting fundamental and conceptual understanding of complex, abstract subject matter (cognitive outcomes); enhanced abilities to assess their own knowledge, its applications and shortcomings (metacognitive outcomes); and enhanced motivation and interest in the subject matter (affective outcomes) [14], [18], [19]. These motivations and goals are consistent with those behind the animation developed in this study. While Campbell et al. [13] highlight multiple and broad potential evaluation measures for courseware including learners’ ratings, learners’ performance, job performance and impact on the organization in which they work, and return on investment, most instructors limit their evaluation to the first two measures. The literature contains fairly consistent evaluation measures used to determine the effectiveness of a courseware tool and generally combines quantitative and qualitative tools. Commonly, instructors evaluate student learning outcomes using pretests and posttests, comparing experimental and control groups on pretests and posttests, gathering data on questionnaires with either closed-ended responses (for example, agree/disagree or Likert-type scales) or freeform comments [11]–[17], [19], [20]. Less frequently, in-depth student interviews or focus group interviews may be used to gather qualitative data [14], [21]. In these studies, data analysis
120
IEEE TRANSACTIONS ON EDUCATION, VOL. 50, NO. 2, MAY 2007
Fig. 1. Screen snapshot of the custom software showing the animation of address flow and the beginning of dataflow, juxtaposed against an animation of the clock. (Note: the animation tool may be obtained by contacting K. Ferens at
[email protected].)
is generally limited to basic statistics (means, standard deviations) on quantitative data; statistical significance tests are less frequently included. Freeform comments are typically analyzed qualitatively and coded thematically for reoccurring patterns and trends. While assessment and evaluation designs vary in their scope and complexity, most instructors are addressing the calls of Hall et al. [22] for iterative assessment with ongoing feedback, triangulation of multiple outcomes, and progressive application of multiple experimental methodologies. The purpose of the evaluation is generally to assess cognitive and affective student outcomes and to assess features of the courseware itself, such as quality, design, features, ease of use, and usage patterns [12], [14], [15], [20], [23]. Instructors generally use “understanding” and subject matter mastery (linked to assignment and test grades), problem-solving skills, and learning gains as indicators of cognitive outcomes [11], [12], [14], [17], [19]. Affective outcomes are generally indicated by students’ self-assessment of their motivation to engage in the material and their sustained interest in the subject matter as reflected, for example, in their use of the courseware beyond the mandatory requirements for the course [11], [14], [17], [19]. Both the evaluation purpose and measures implemented in this study are consistent with the literature. III. DESCRIPTION OF THE ANIMATION The animation developed in this study addresses the inadequacy of conventional teaching methods — such as oral lectures
complemented with notes, figures, and other graphical aides — to explain the complex internal operation of a microprocessor. Furthermore, this study addresses the lack of tools for the animation or simulation of the internal operation of a microprocessor, in particular, the control, configuration, and execution of microoperations. The major features of the animation software include providing students with the ability to: 1) construct a microoperation by entering the states of control signals for each component; 2) construct a macroinstruction by specifying information such as an operation code and a sequence of previously defined microoperations; 3) change the contents of registers, enter data into memory, and enter binary encoded instructions into memory; 4) animate the execution of each microinstruction of a given macroinstruction; 5) show the address and dataflow by using the “water flowing through pipes” analogy; and 6) show the temporal correlation of address and dataflow by animating the clock. As shown in Fig. 1, the animation software’s main page consists of three sections: the top section displays a facility to configure microoperations and macrooperations, and controls the animation and execution of instructions. The bottom section shows the attached memory. The center section shows the basic microprocessor and its components. The microprocessor is physically split into three major parts: the address generator unit (AGU), which is used to generate addresses to address the memory; the data path unit (DPU), which provides multiple paths for data propagation; and the computer control
FERENS et al.: IMPACT ASSESSMENT OF A MICROPROCESSOR ANIMATION
unit (CCU), which stores the control signals for each microoperation, definitions of macroinstructions, and sequencing information. Each part consists of smaller components, such as the multiplexers (or data switches), registers, adder, and arithmetic logic unit (ALU). Each component is a multifunctional device, whose function is controlled by its control signals. The first tab in the configuration section, named “Configure OPs,” provides a facility to create microoperations. For each unique microoperation, the set of 77 control signals (called a control vector) must be configured to implement the desired operation. The mapping of the states of the control signals and the associated function for each device is displayed by moving the mouse pointer over the name of the device, causing a text pop-up window to appear (provided the help is switched on). Users establish the desired state of a control signal by clicking on the cell, causing the cell to change value in rotation (x, 0, 1) for each click on the cell. Because of the animation software’s representation of each component in the microprocessor as a multifunction device, whose particular function may be controlled by specification of its control signals, students realize that very many microoperations are possible. In theory, because there are 77 binary-valued control signals, there are a total of 2 possible microoperations, but only a small fraction of these are unique and practical. Furthermore, by providing the ability to control any device through specification of the device’s control signals, the animation software enables students to learn that they can create microoperations that implement a desired function. In this way, the animation software provides an ideal platform for the creation and design of microoperations. The second tab, named “Set OP Paths” is reserved for future options. The third tab, named “Build Instructions,” is used to construct macroinstructions. To build a macroinstruction, the user must enter a binary encoded operation code (Opcode), mnemonic, source, destination, addressing mode, number of microoperations, ordered sequence of microoperations, and condition code register configuration. In being required to enter an 8-bit operation-code, students realize that the operation-code is used to identify a macroinstruction uniquely, and to link all of the configuration information connected with that macroinstruction, including the number of microoperations, the sequence of microoperations, and the control vector for each microoperation. Furthermore, students realize this information is stored in computer control memory in the CCU part of the microprocessor. Moreover, students comprehend that the operation-code is used as a “key” to retrieve execution information for a macroinstruction. In addition, since 8-bits are used to specify the operation-code, students begin to understand that a finite macroinstruction set is possible: . Finally, in being required to enter the number and ordered sequence of microoperations, students realize that different macroinstructions may be defined by specifying different numbers of different microoperations, which they have full control of creating, as discussed above. Consequently, for each of the 256 possible macroinstructions, an arbitrary ordered sequence of microoperations is possible,
121
giving rise to a theoretically infinite number of different macroinstructions. However, students come to the end realization that the size of the control memory limits the number of macroinstructions, microoperations, and control vectors. The fourth tab, named “Animate,” is used to show temporally and graphically the execution of each microoperation of a macroinstruction. The “water flowing through pipes” analogy is used to show the paths the address and data take throughout the microprocessor. Furthermore, the propagation of address and data is correlated with the animation of the clock, providing visual cues on exactly when addresses and data pass into and through the various components of the microprocessor and memory. By providing them with the “water flowing through pipes” analogy of the address and dataflow, the animation software enables students to notice that the address and data need a certain amount of time to propagate to their destination: hence, the animation visibly clarifies the concept of “address and data settling time.” Furthermore, students realize that the clock has an inherent speed limit, which cannot exceed the required address and data propagation times. Also, the “water flowing through pipes” depiction helps in debugging the configuration. During an animation, students may identify configuration errors by noting the device at which the flow stops. The juxtaposition of the address and dataflow with the animation of the clock visually depicts the sequential and concurrent nature of the microprocessor. For example, students realize that the memory address flows in the first half of the clock and is sequentially followed by the dataflow in the second half of the clock. In addition, because of the juxtaposition, the processor’s ability to perform concurrent operations becomes strikingly apparent. For example, the Program Counter (PC) may be incremented while the memory is being addressed; the Temporary Holding Register (THR) may be reset while data is moved from memory to the processor; and a byte of an AGU register may be transferred into the Accumulator A while data from memory is moved into the Memory Data Buffer (MDR). The animation software visually depicts its concurrent behavior by showing multiple dataflows and/or toggling the color of active devices, as appropriate (Fig. 2). IV. INTERACTIVITY The animation software provides several interactive aspects to support debugging and “what if” analysis. First, for debugging purposes, during the “water flowing through pipes” analogy of data and/or address flow, or before the animation begins, the user may change the value of a destination register or memory location to verify visually that the correct value is being written to the destination object at the correct time. In addition, if a destination object obtains its input from a source object, then, by changing the value of the source, the user may perform “what if” analysis and discovery. For example, during the execution of a conditional branch instruction, the user may change the value of the condition code register (CCR) to perform “what if” analysis. Also, in addition to allowing the change of registers and memory locations, the animation software also allows changing the microcode of any microinstruction. For
122
IEEE TRANSACTIONS ON EDUCATION, VOL. 50, NO. 2, MAY 2007
Fig. 2. Screen snapshot showing an example of the animation’s method of depicting a concurrent operation in the microprocessor. The animation shows “water”, starting from the output of the least significant byte (LSB) of the Index Register (IX), flowing onto the RBUS, through MUX2, and almost reaching its destination, the Accumulator A. While the IX is flowing, data from memory flows onto the DBUS, through MUX0, and almost reaching its destination, the MDR.
TABLE I ASSIGNMENT: IMPROVING LOOP EFFICIENCY
example, during the execution of a microinstruction, the animation may provide feedback by indicating one or more errors in the control signal configuration. In response, the user may pause the animation, correct the control signal configuration, and resume or restart the animation.
V. DESCRIPTION OF A LABORATORY ASSIGNMENT One of the animation-based assignments given to students worked with improving the simple loop shown on the left in Table I.
FERENS et al.: IMPACT ASSESSMENT OF A MICROPROCESSOR ANIMATION
123
Fig. 3. Configuration information for the ADDA instruction utilizing the new addressing mode, IND
Fig. 4. Configuration of the microoperations for the ADDA $00, components are shown because of space constraints.
X+.
X+ instruction. Note: the control vector for a microoperation has 77 components, and only 12
Students were asked to design a more efficient instruction to improve the speed at which the loop is executed. An interesting solution was to create a new addressing mode, called “Indexed ). The IND addressing X with post increment” (IND mode is the same as the “Indexed X” (IND X) addressing mode, except that the former increments the Index X register while the adinstruction is performing its main operation. The IND dressing mode was utilized in the “ADDA” instruction, thereby, removing the need for the “INX” instruction, and improving the loop efficiency by two clock cycles. To verify their solution, the students implemented the ” instruction in the animation software by “ADDA $00, following three main steps. (Step 1) The students built the macroinstruction and entered configuration information into the fields, as shown in Fig. 3. (Step 2) The students created the microoperations, which they specified in the Instruction Set Builder (Fig. 3). Through an analysis of the problem, students conjectured that the ADDA could have the same microoperations as the ADDA $00, $00,X, except they needed to find out which microoperation could be used to concurrently increment the X register. They determined that the only microoperation in which X can be incremented while maintaining the integrity of the instruction was the microoperation #4, “adda to a.” They correctly determined that X cannot be incremented during the second or third microoperations because the effective address of the second operand has not been calculated yet, and the present value of X cannot change during these microoperations. In the fourth microoperation, the effective address has already been generated and stored in the THR, and the value of the X register may be changed without affecting the desired “ADD” operation of the macroinstruction. As a result, students created four microoperations: “fetch,” “getoffset,” “addoffset,” and “add to a,” as shown in Fig. 4. (Step 3) The students animated the macroinstruction for verification. Fig. 5 shows the animation of the fourth microoperation, in which the X register is incremented concurrently while the memory data is added to Accumulator A.
VI. METHODOLOGY To perform a preliminary exploration of the cognitive and motivational impacts of the microprocessor animation, this study used a mixed-method, postunit student survey (Table II) administered to students in the microprocessor systems course, who voluntarily and anonymously agreed to participate. The survey was administered in two consecutive years in which the course was taught: first to 67 students enrolled in 2005, and then to 33 students enrolled in 2006. The survey combined 13 closed-ended questions (agree/disagree/neutral or five-point Likert scales) and five areas of qualitative feedback, and was assessed by other faculty members for face validity prior to implementation. The survey questions were constructed to explore elements of learner satisfaction including application-specific self-efficacy, curiosity, and enjoyment [24] and elements of the learner interface, the content of the animation software, and the personalization aspect [25]. These elements are identified as being influential in students’ decisions to use an electronic learning system. They include the ability to easily access, understand, and work with the tool within a framework of personal choice and control (self-efficacy [24], learner interface, and personalization [25]), enjoying the time spent using the tool (enjoyment [24]) and finding curiosity in the subject matter and imagination aroused as a result of using the tool (curiosity [24]), and evaluating the tool as providing up-to-date and useful content [25]. While significant work is being undertaken to develop empirically validated models for measuring students’ experiences along clearly delineated dimensions, the survey used in this exploratory study combined several dimensions so that questions one through seven focused on content, application-specific selfefficacy, and learner interface, while questions eight through 13 focused on personalization, enjoyment, and curiosity. Specifically, students were asked to self-assess cognitive and affective outcomes as a result of using the microprocessor animation, focusing on conceptual understanding of microcode architecture, implementation, execution, and modification, and students’ perceptions of their own competency in microprocessing systems,
124
IEEE TRANSACTIONS ON EDUCATION, VOL. 50, NO. 2, MAY 2007
X+
Fig. 5. Animation showing the execution of the “add to a” microoperation of the “ADDA $00, ” macroinstruction in the first iteration of the loop. Note that during the animation, the “clock bar” moves from left to right. Interesting actions occur as the clock bar crosses an edge or a level of the clock. In this example, the X register was incremented at the falling edge of clock -1 in the fourth clock cycle.
learning material preferences, and ongoing subject matter interest. In addition, a basic statistical analysis of aggregate unit and course grades from prior years to the study year, controlling for trends in prior years, was conducted. VII. RESULTS The results provide an initial understanding of students’ experiences with the microprocessor animation. The survey, administered in the last week of classes in both the 2005 and 2006 spring terms, revealed a substantial increase in cognitive outcomes. First, with respect to the impact of animation on showing how a microprocessor executes a macroinstruction through sequencing of microinstructions, an overwhelming 86.6% of 2005 student respondents indicated that animation accomplished this objective, while another 76.1% said animation showed this process either well or very well. Interestingly, by 2006, these numbers swelled to 100% of students indicating animation accomplished the objective, with 94% saying the animation showed the process either well or very well. Furthermore, 60% of 2005 respondents and 72.7% of 2006 respondents said animation was either more or much more effective compared to labs/notes in showing the process. When asked to elaborate, the “flow of water” metaphor used in the animation to visualize dataflow overwhelmingly emerged as a key component which enhanced their understanding of the microcode sequencing and execution process. Comments included “looking at the clock cycles and time delays of water
flow allowed me to visualize voltage flow. It [water flow analogy] gives a visual on how it [instruction execution] actually happens” and “the layout in which the microinstructions were put into the animation software helped me to see how they work together.” The survey contained a number of questions relating to the animation of the clock. With conventional lecture notes, a teaching challenge has always been to explain the abstract and complex events that occur within the microprocessor in relation to certain edges and levels of the clock. With the addition of the animation, students’ mastery of the knowledge was noticeably enhanced (Fig. 6). Students’ qualitative responses on how animation increased their overall understanding of course material focused on dataflow visualization and event timing in relationship with a “moving clock” and broader-based comments, including “We can build instruction by ourselves, we can create new instructions and, from building the instructions, we can learn and understand more quickly” and “[the animation] allowed me to put everything together to see how all the microops flow together. [The animation] also showed the importance of FETCH.” As an additional measure of overall cognitive outcomes, students were asked if they thought their academic performance in the course had been positively affected by the animation component. These numbers differed somewhat between the two years with 64% of the 2005 respondents indicating yes, compared to 57.6% in 2006.
FERENS et al.: IMPACT ASSESSMENT OF A MICROPROCESSOR ANIMATION
TABLE II MICROPROCESSOR ANIMATION SURVEY
125
126
IEEE TRANSACTIONS ON EDUCATION, VOL. 50, NO. 2, MAY 2007
“it is important to receive the facts through the notes and lectures, but the animation increases comfort level with the material and improves conceptualization;” “Assembler language was easier to learn in labs/lectures but the animation helped visualize the instruction sequence;” and “Both are important. The animation does the better job of demonstrating microops than do the notes, but I cannot imagine trying to learn [how to interface peripheral devices to the microprocessor] from it [the animation software].” When asked how much time they had spent with the animation above and beyond the requirements of the course, the majority of 2005 and 2006 respondents indicated that little or no time was spent, while a minority of respondents spent up to 25 additional hours experimenting with the animation software. VIII. DISCUSSION AND FUTURE WORK Fig. 6. 2005 survey results showing the impact of computer animation on learning complex concepts of a microprocessor’s internal operation.
Fig. 7. Results of past years’ test scores.
Taken overall, the results on the impact of the animation on student learning gains was mixed. Although student survey responses indicated self-assessed learning gains, the overall comparison of unit grades between previous years’ classes and the current class using the animation did not indicate statistically significant increases (Fig. 7). The inconclusive nature of the results is compounded by slightly different test content from year to year, differing student abilities year over year, and the complexity of comparing students using an animation to students not having the animation—a comparison widely recognized to be inherently “apples-to-oranges.” The survey also addressed motivational outcomes, including whether students thought their performance in subsequent related courses would be enhanced because of their experiences with animation in the course. Perhaps because of the hypothetical nature of the question, results were mixed. While 40.3% of 2005 respondents indicated yes, another 40.3% were uncertain and 19.4% said no. In 2006, the number of respondents indicating yes dropped to 24.2%, and there was an increase in undecided (57.6%). Whereas, an impressive 79% of 2005 respondents and 82% of 2006 respondents said that if they were to take the course again, the best method of learning would be a combination of notes/ labs/computer animation, as opposed to each alone. Their qualitative responses highlighted the role of animation as a reinforcement or complement to traditional theory which significantly increased their understanding. A sampling of comments include:
The “water flowing through pipes” animation method, showing data and address flow, was perhaps the most commented-on feature of the animation software. The “water flowing through pipes” feature has shown to be a very powerful component of the animation, and provides an element of visual learning that many students are finding to be critical in their ability to understand. Students acquired a deeper understanding of how the microprocessor controls memory devices by issuing an address, which is followed by dataflow. In addition, by juxtaposing the animation of the clock with the “water flowing through pipes” animation, students gained a realization that a simple task, such as loading data into a register, requires a time-ordered sequence of events, such as fetching an operation-code. Furthermore, students are able to correlate address and dataflow with certain clock edges and levels. Some comments reflected students’ engagement in higher order cognitive processes (according to Bloom’s Taxonomy, where lower-level cognitive processes are “identifying” and “describing” and higher-level processes are “analyzing,” “synthesizing,” and “evaluating”). The students mentioned the ability to do things such as “build and execute”, how changes are constructed, problem solving, cause and effect relationships, how parts interact, and “negative case analysis” (i.e., the ability to debug or understand where their design went wrong). IX. SUGGESTED ASSIGNMENTS OR LABORATORIES USING THE ANIMATION TOOL As a formative study, the results will be used to guide future development of the microprocessor animation and the course curriculum. Some students suggested optimizing the branch instruction to run faster on average. Currently, all conditional branch instructions require three microoperations: 1) The “fetch” microoperation obtains the operation code of the instruction. This operation code indicates the exact condition to be evaluated. 2) The “get offset” microoperation obtains the offset of the instruction. The signed offset is used to determine the number of locations to branch in the case the condition is true. 3) The “make decision” microoperation evaluates the condition and determines whether a branch should be taken. If the condition evaluates true, then the signed offset is added to the current value of the PC; otherwise, no branch is taken, and the PC is not altered. The suggested improvement is to evaluate
FERENS et al.: IMPACT ASSESSMENT OF A MICROPROCESSOR ANIMATION
the condition in the second microoperation instead of the third. The advantage is that if the condition is false, then executing the third microoperation is unnecessary, and thus the branch instruction would run faster on average. This particular suggestion of improving the performance of the branch instruction is actually one example of a larger concept, called branch prediction. While branch prediction is not within the scope of this introductory course, this particular student’s feedback shows how the flexibility of the animation tool provides a good learning environment for learning advanced concepts, such as branch prediction. Also, an end-of-course project is planned, in which students will design an instruction set of a small microprocessor. Based on their experience with the instruction set of the 68HC11 microprocessor used in the laboratories, students found several instances where a desired instruction was not available in the instruction set — a typical problem most programmers face when developing software to solve a problem. In addition to creating new instructions by defining new or using existing microoperations, students will compare the performance of their new instruction set with the existing set by comparing the number of microoperations and execution times required to solve the same problem. The comparison will also take into consideration the additional hardware real-estate required to add new microoperations and instructions, and to consider the tradeoffs of creating a new instruction versus using existing nonoptimal instruction. The authors plan to add peripheral devices to the animation, such as analog-to-digital and digital-to-analog converters, parallel and serial interface adapters, and seven segment and other displays. Furthermore, animation of interrupts and I/O programming is planned. X. SUGGESTED IMPROVEMENTS TO THE ANIMATION TOOL A number of improvements to the Animation’s interface were suggested by student survey respondents. 1) In addition to water flowing along the bus, display the value of the address or data while the data moves from source to destination. This step would help verification and debugging. 2) The memory device should have load and store functionality so that users may reload previous memory configurations, reducing the time required to reenter data into memory. 3) To prevent accidental control state, register, or memory modification, a “lock” and “unlock” functionality should be in place. Although the method for modifying registers and memory is quite simple (just click on the cell), accidental clicking may cause an inadvertent change. The problem is exacerbated because an accidental change may not be easily detected because of the complexity and large number (i.e., 77) of control signals. When the item is locked, clicking on the cell has no affect. Unlocking the item enables modification of the item. 4) Entering the states of all 77 control signals for a microoperation is cumbersome and time consuming. The definition of a device’s functionality is displayed by moving the mouse pointer over the name of the device. This action causes a text popup window to display a “help table” which contains
127
several rows, each row containing the mapping between the function and the required states of the control signals. A better method would be for the software automatically to set the control signals in the “Configure OPs” table upon the user clicking on a row in the help table that displays the desired functionality. 5) Dynamic “undo” ability would be a good feature of the animation software. For example, users found that they needed to execute all preceding microoperations each time they wanted to correct configuration errors in the fourth microoperation. Instead of restarting the macroinstruction from the beginning, a user could slide the clock bar back to the desired time position, and then begin the animation from that point in time. To implement this feature correctly, as the clock bar is slid back by the user, the software needs to undo each event that occurred, like reversing the flow of water and restoring previous register and memory values. The impact of the animation on student learning and attitudes will continue to be assessed in future years. The survey used in this initial study will be reviewed and revised to ensure that a valid and expanded understanding of the use and impact of the computer animation in the course is captured. Additional assessment measures will also be implemented to provide multiple measures and more triangulated data. ACKNOWLEDGMENT Dr. H. Poskar assisted in the development of the animation software. Note: the animation tool may be obtained by contacting K. Ferens at
[email protected]. REFERENCES [1] P. Eibeck, “Criteria for peer-review of engineering courseware on the NEEDS database,” IEEE Trans. Educ., vol. 39, no. 3, pp. 381–387, Aug. 1996. [2] Freescale Semiconductor Technical Staff, M68EVB912B32 Evaluation Board: M68EVB912B32/D Rev. 1, Freescale Semiconductor, Inc., 1999. [3] W. Bilderbeek, H. Broeders, and A. van Rooijen, “The THRSim11 68HC11 Simulator,” Dr. Dobb’s J., Jul. 22, 2001. [4] G. Wolffe, W. Yurcik, H. Osborne, and M. Holliday, “Teaching Computer Organization/Architecture With Limited Resources Using Simulators,” in Proc. 33rd Tech. Symp. Computer Science Education (SIGCSE), Covington, KY, Feb./Mar. 2002, pp. 176–180. [5] C. Yehezkel, W. Yurcik, and M. Pearson, “Teaching computer architecture with a computer-aided learning environment: State of the art simulators,” in Proc. Int. Conf. Simulation and Multimedia in Engineering Education (ICSEE), Phoenix, AZ, Jan. 2001. [6] W. Yurcik, G. Wolffe, and M. Holliday, “A Survey of simulators used in computer organization/architecture courses,” in Proc. Summer Computer Simulation Conf. (SCSC), Orlando, FL, Jul. 2001. [7] W. Yurcik, “The simulation education homepage,” Simulation, pp. 202–206, Apr. 2001. [8] R. E. Mayer and R. Moreno, “Animation as an aid to multimedia learning,” Educ. Psych. Rev., vol. 14, no. 1, pp. 87–99, 2002. [9] R. E. Mayer and R. Moreno, “Aids to computer-based multimedia learning,” Learning and Instruction, vol. 12, pp. 107–119, 2002. [10] X. Teng, J. G. Tront, B. Muramatsu, and A. Agogino, “Best practices in the design, development and use of courseware in engineering education,” in Proc. American Society for Engineering Education/IEEE Frontiers in Education Conf., Indianapolis, IN, 2005, pp. F2E–1-7. [11] M. Ogot, G. Elliott, and N. Glumac, “An assessment of in-person and remotely operated laboratories,” J. Eng. Educ., vol. 92, no. 1, pp. 57–64, 2003. [12] S. Hsieh and P. Y. Hsieh, “Integrated virtual learning system for programmable logic controller,” J. Eng. Educ., vol. 93, no. 2, pp. 169–178, 2004.
128
IEEE TRANSACTIONS ON EDUCATION, VOL. 50, NO. 2, MAY 2007
[13] J. O. Campbell, J. R. Bourne, P. J. Mosterman, and A. J. Brodersen, “The effectiveness of learning simulations for electronic laboratories,” J. Eng. Educ., vol. 91, no. 1, pp. 81–87, 2002. [14] P. S. Steif and L. M. Naples, “Design and evaluation of problem solving courseware modules for mechanics of materials,” J. Eng. Educ., vol. 92, no. 3, pp. 239–247, 2003. [15] D. M. Tilbury and W. C. Messner, “Control tutorials for software instruction over the World Wide Web,” IEEE Trans. Educ., vol. 42, no. 4, pp. 237–246, Nov. 1999. [16] M. J. Moure, M. D. Valdes, A. Salaverria, and E. Mandado, “Virtual laboratory as a tool to improve the effectiveness of actual laboratories,” Int. J. Eng. Educ., vol. 20, no. 2, pp. 188–192, 2004. [17] C. Elmas and M. A. Akcayol, “Virtual electrical machinery laboratory: A Fuzzy Logic Controller for Induction Motor Drives,” Int. J. Eng. Educ., vol. 20, no. 2, pp. 226–233, 2004. [18] S. A. Boothroyd, L. Chan, and W. M. Robertson, “Visualizing coherent light with an electromagnetic wave simulator,” IEEE Trans. Educ., vol. 39, no. 1, pp. 29–39, Feb. 1996. [19] M. Mernik and V. Zumer, “An educational tool for teaching compiler construction,” IEEE Trans. Educ., vol. 46, no. 1, pp. 61–68, Feb. 2003. [20] J. V. Ringwood, “Computer-aided learning in artificial neural networks,” IEEE Trans. Educ., vol. 45, no. 4, pp. 380–387, Nov. 2002. [21] A. Palipana, P. Barker, S. Rothberg, and F. Lamb, “Developing a standardized evaluation methodology for computer based learning materials,” in Proc. American Society for Engineering Education Annu. Conf. and Exposition, Montreal, QC, Canada, 2002. [22] R. H. Hall, “A model for the evaluation of innovative engineering courseware: Engineering an assessment program,” in Proc. American Society for Engineering Education Annu. Conf. and Exposition, Montreal, QC, Canada, 2002. [23] F. de Coulon, E. Forte, and J. M. Rivera, “KIRCHHOFF: An educational software for learning the basic principles and methodology in electrical circuit modelling,” IEEE Trans. Educ., vol. 36, no. 1, pp. 19–22, Feb. 1993. [24] S. L. Toral Marin, F. J. Barrero Garcia, R. Martinez Torres, S. G. Vasquez, and A. J. Lillo Moreno, “Implementation of a web-based educational tool for digital signal processing teaching using the technological acceptance model,” IEEE Trans. Educ., vol. 48, no. 4, pp. 632–641, Nov. 2005.
[25] Y.-S. Wang, “Assessment of learner satisfaction with asynchronous electronic learning systems,” Inf. Manage., vol. 41, pp. 75–86, 2003.
Ken Ferens (M’96) received the B.Sc., M.Sc. and Ph.D. degrees from the University of Manitoba, Winnigpeg, MB, Canada, in 1989, 1991 and 1996, respectively. He is an Assistant Professor in Electrical and Computer Engineering at the University of Manitoba. His research interests include wavelets and fractals, artificial neural networks, audio and video compression, internet protocols, satellite telecommunications, microprocessing systems and microcontrollers, and software engineering.
Marcia Friesen received the B.Sc. and M.Ed degrees from the University of Manitoba, Winnipeg, MB, Canada, in 1995 and 2003, respectively. She is currently working towards the Ph.D. degree at the University of Manitoba, where she is Program Director for the Internationally Educated Engineers Qualification Pilot Program. She works in faculty development for the Faculty of Engineering. Her research and professional interests include curriculum design and instructional methods in engineering, discipline culture, and educational administration.
Sandra Ingram (M’02) received the B.A.(Hons) degree from the University of Winnipeg, Winnipeg, MB, Canada in 1983, the M.Ed. degree from the University of Manitoba, Winnipeg, MB, Canada, in 1985, and Ph.D. degree from the University of Toronto, Toronto, ON, Canada, in 1993. She is an Assistant Professor in technical communication in the Faculty of Engineering, University of Manitoba, Winnipeg, MB, Canada. Her research interests include integrated technical communication approaches in the undergraduate curriculum, engineering education, women in engineering, and design education.
