Innovating Adaptive and Neural Systems Instruction with ... - CiteSeerX

To be published in the Proceedings of the IEEE, January 2000

Innovating Adaptive and Neural Systems Instruction with Interactive Electronic Books Jose C. Principe, Neil R. Euliano and W. Curt Lefebvre Computational NeuroEngineering Laboratory NEB 486, Electrical and Computer Engineering Department University of Florida, Gainesville, FL 32611, USA {principe}@cnel.ufl.edu ABSTRACT

This paper describes an integrated strategy to innovate teaching in the undergraduate classroom by appropriately utilizing information technologies. The innovation is an interactive learning environment built around an interactive electronic book (i-book). The i-book is a tight integration of a hypertext document with a simulator. The hypertext specifies the presentation order of the theoretical topics, and the simulator is an essential piece of learning. The lectures exploit the ibook, transforming the classroom into an interactive teaching laboratory. An electronic white board is used in every lecture to teach from the text and run the simulations. Our goal is to teach adaptive systems to undergraduate Electrical Engineering students. Undergraduates do not have the mathematical background to comprehend the equation-based approach utilized in graduate level adaptive systems courses. We have used the i-book for undergraduate instruction with very positive results. We are now ready to extend the format for distance learning. This new teaching methodology transcends adaptive systems and can be applied to other engineering and science courses which have access to simulators.

Dr. Jose C. Principe BellSouth Professor EB 451, Bldg #33, Electrical Engineering Department University of FLorida Gainesville, FL 32611 phone 352-392-2662 fax 352-392-0044 [email protected]

1

To be published in the Proceedings of the IEEE, January 2000

1. Introduction Four years ago we posed the innocent question of what would be required to teach adaptive systems to undergraduates. Since the undergraduate student does not have the required mathematical background to fully comprehend the theory of adaptive systems, we formulated a data-driven model of instruction which explains the course material by blending theory with a software simulator. We went beyond the step of using a software simulator in the classroom. In our instructional model, the simulation becomes an essential piece of information delivery, deeply affecting the organization of the textbook material, the teaching methodology and the required classroom equipment. The new interactive teaching methodology allows the student to actively reinforce the concepts encapsulated in equations with the behavior of the adaptive system observed through the simulations. This vision lead to an integrated strategy to revise the presentation of the subject material in a sequence of 200 concepts with demonstrations, to merge the electronic text with a software simulator, to develop a new teaching style and to create an Interactive Teaching Laboratory (ITL) at the University of Florida. Our experience shows that: The flow of topics found in paper textbooks has to be reversed for use with our interactive teaching methodology. In a typical engineering course or textbook, the subject material is first presented in a theoretically general sense and it is followed by applications to specific engineering examples. The proposed interactive teaching methodology flows more naturally when each key application is broken down into individual concepts which are illustrated sequentially with a live simulation to enhance understanding and to motivate the student. Hence the simplest case must be treated first, with more realistic models presented later. The conventional teaching format has to be modified. We settled on a studio format that begins with the presentation of a concept using simple equations and an example/simulation to reinforce the concept and explain the equations. The students then experiment with the simulator by changing the parameters and observing the changes in the behavior of the system. The instructor then summarizes and generalizes the topic before proceeding to the next concept. The conventional classroom equipment has to be improved. With a grant from the National Science Foundation we created the Interactive Teaching Laboratory (ITL) where each student has a

2

To be published in the Proceedings of the IEEE, January 2000

PC workstation which is connected to a server running the electronic book. We use an LCD projector and an electronic white board in every lecture. This gives us an intimate contact with the material (circling and writing electronically over the projected text). Our course can easily be modified to a WEB based course, but thus far we have kept the professor in the students’ learning loop (i.e. synchronous learning). This paper describes our efforts, experiences and conclusions to innovate undergraduate instruction in the area of adaptive systems. Digital signal processing is our area of expertise, so comparisons and examples will be drawn from this area. However, we realize that the reformulation of instruction based on i-books transcends adaptive systems and the original motivation of teaching challenging topics. i-books have a wide applicability in Electrical and Computer Engineering (ECE) due to the trend of teaching ECE topics based on computer simulators such as Spice, Simulink, Matlab, Mathcad, Java and digital simulators. We foresee professors in electronics, signals and systems, computers and communications developing and utilizing similar teaching methodologies to enhance the delivery of information to ECE undergraduate students. The relevance of computer enhanced education has been recognized in the signal processing community where its main conference, the International Conference on Acoustics, Speech and Signal Processing (ICASSP) has, for the last 7 years, had one session devoted to DSP Education (see also the November 1995 issue of the IEEE DSP Magazine, and the special issue on DSP instruction of the IEEE Transactions on Education (May, 1996)). The paper is organized as follows: section 2 presents our model for the integration of text with a simulator for interactive learning with i-books. Section 3 explains how we put together the course material and the presentation. Section 4 discusses the syllabus for the adaptive systems course. Section 5 discusses the classroom format, and the teaching experience is presented in section 6. Section 7 provides our vision for the future.

3

To be published in the Proceedings of the IEEE, January 2000

2. A model to develop and use interactive electronic books 2.1 Computers for engineering instruction In order to address the usefulness of i-books for engineering instruction, we have to ponder both the advantages and disadvantages of computer enhanced learning. The four major advantages in computer enhanced engineering instruction are: 1. The power of simulation that comes from the universal nature of digital computation. Digital computers are in fact formal systems [18], so they can simulate any mathematical solution to a problem. Thus, a computer program is practically and theoretically as powerful a representation as an equation. The formal nature of simulators is particularly appealing in engineering. Moreover, if the computer program is modular (object-oriented) with iconic interfaces, then engineering design can also be taught. 2. Multimedia simulations with sound and/or graphics utilize the learner’s sensory inputs. For reasons that are not fully understood, the human brain is able to extract much more information from images (and sounds) than from lists of numbers, effectively facilitating the understanding of difficult concepts and making learning more natural [5]. 3. When a concept is learned through a simulation the real world context is immediately conveyed about the importance and the use of the concept, blending explanation with engineering design. The ubiquitous access to the word wide web tremendously simplifies the search for appropriate demonstration data. 4. Finally, with a simulator the knowledge that the student gains is immediately reusable to new problems. This is very different from the static information available in books where several steps are required to go from the design to the implementation of a solution.

The disadvantages that we consider relevant are: 1. The computer is still not as easy to use as a book. Some contributing factors include: small displays which limit the amount of information that can be conveyed at one time, a lack of content based search, a lack of efficient browsing, and possibly the fact that people aren’t comfortable using computer based books. 2. We still do not know how to exploit computer based instruction as effectively as expository lectures. Most of the experiences we know of have been a substitution of the media (from paper to multimedia) without rethinking the teaching methodology. 3. There is a potential danger in the utilization of simulators as recipes without fully understanding the underlying concepts. Although it can be argued that the role of the engineer is to ade-

4

To be published in the Proceedings of the IEEE, January 2000

quately utilize tools to solve problems, conceptual understanding provides a more lasting and upgradeable form of knowledge.

Our efforts seek not only the substitution of the presentation media from paper to electronic form, but more importantly, a modification of the way the material is organized and presented to the learner. We believed from the onset that computer-based presentations alone would not greatly improve the learning environment for instruction. In fact multimedia materials tend to speed up the presentations and to increase the distance between the student and the professor -- often putting the student in a passive mode which hinders conceptual understanding and retention. Complementing signal processing textbooks with computer programs (mostly Matlab m files) in appendices is becoming common place but the organization of the subject material has not changed significantly with the addition of the software. Two notable exceptions are the recent textbooks by McClellan et al [25] and Strum and Kirk [37], where multimedia is used more effectively in explaining the theoretical concepts of signal processing and linear systems.

2.2. The laboratory model Digital signal processing education has flirted with computer enhanced education for many years. Three major types of efforts have been pursued: distance learning, educational simulators and classroom teaching using computers. In distant learning, the goals have been to enable virtual collaboration across the web [27], [38], [9] and asynchronous learning networks [13] [2]. Most of the reported work has addressed the creation of multimedia simulators linked to the DSP laboratory. Normally one of the mathematical simulators (Matlab, Mathcad, Mathematica, Khoros, Simulink) is used either to illustrate each of the fundamental concepts of DSP (discrete linear systems-FIR and IIR, digital filter design, Discrete Fourier Series and FFTs) [39], [6], [3] or to address integrated DSP applications (speech, image processing) [8], [42]. Simulators for instruction across the web, based on Java [7] [33], have been recently developed and many web sites provide specific interactive demonstrations of DSP algorithms. Teaching complete DSP courses enhanced by computers has been reported primarily by McClellan’s group at Georgia Tech [22], [36], [24], [23] and Harger [11], [12] at Maryland. McClellan’s

5

To be published in the Proceedings of the IEEE, January 2000

group uses Matlab to supplement every traditional lecture with a computer demonstration to relate the theory with real world phenomena or applications, very much inspired by Steiglitz course taught at Princeton in the 1970s. A recent book was published illustrating their methodology [25]. Harger utilizes Mathcad simulations enhanced with explanatory text to exemplify all the fundamental topics covered in a traditional DSP course. He also modified the classroom and brought computers to each student’s desk to allow interaction with the course material. These are important experiences that helped mold our own efforts to develop an interactive teaching methodology for adaptive systems. Our model for the learning environment was borrowed from the everyday life of the research laboratory where the professor interacts with the graduate student at a high conceptual level. Instead of learning through lectures, the students learn through the results of their research and experimentation, with guidance from their advisor. However, graduate students already have the necessary knowledge that makes them autonomous agents, capable of pursuing and validating alternatives. This environment tends to be loosely structured, but is highly effective (we learn much more when we are active players [5]). When applied to teaching, this implies that the classroom model should be substituted by a laboratory model. Software simulators are excellent at recreating real world situations, but it is important that undergraduate students be guided closely because they do not have a theoretical understanding of the material. This has been the real bottleneck in engineering teaching: how does one bootstrap students’ knowledge until they can solve real world problems? Traditionally, engineering teaching has been divided into expository lectures where the theoretical material is covered, and laboratories where application of the concepts is illustrated. This division is arbitrary and counter productive. Theory in engineering appears as a necessity to solve a practical problem with a model based approach, so engineering by definition lives in the interface of first principles with experimentation. As such engineering is vitally dependent upon the synergisms created between theory and practice, which are often destroyed when we divide teaching between lectures and laboratories. This is one of the reasons why our schools tend to either graduate technologists or theoreticians. The previous attempts ([25] and [11]) were not able to adequately resolve the lecture-laboratory duality. McClellan complements traditional lectures with computer based laboratories to enhance

6

To be published in the Proceedings of the IEEE, January 2000

understanding. The course structure still has a list of chapters with the corresponding laboratories. However, this strategy is powerful because it allowed the group to teach DSP ahead of the traditional analog circuits and systems course, which is a landmark achievement. Harger’s approach is diametrically opposed: he takes a problem (the laboratory) and wraps it with the appropriate theory, which means that the course becomes a summation of loosely coupled laboratories. Although there is no lecture/laboratory distinction as in [25], the systematization of concepts suffers because there is no explicit logical flow of topics.

2.3. Implementing the laboratory model with i-books The central piece of our strategy is the creation of an instructional medium where text co-exists with a functional simulator and which allows visualization and student interaction, very much as proposed by [11]. In our approach however, we do not wrap examples with text, but we re-organize the theory in fine-grain conceptual modules each of which is illustrated with a simple simulation to enhance conceptual understanding throughout the text. The level of integration between the text and the simulator is therefore at a different level as can be observed from the number of simulations which are 200 in our case versus 32 in [11] (although one-to-one comparison is not possible because the subject material is different). At the end of each chapter there is always at least one full simulation that can be used for a project, but it is incrementally built throughout the chapter. In our case we preserve the flow of concept presentation inherent in traditional textbooks, but we are able to bring the advantages of the simulation to the concept level, instead of the application level. This was made possible by integrating an hypertext document with a software simulator, which we call an interactive electronic book (i-book). We submit that i-books are posed to innovate undergraduate engineering teaching because they make the laboratory model of instruction possible. But in order for this to happen three major aspects must be designed into ibooks: seamless integration of theory and practice, encapsulation of mathematical detail, and reordering of topics according to the bottom-up style of the simulator. 2.3.1. Seamless integration of theory and practice. One of the major concerns during the conceptual development of the i-book format was how to intertwine theory and practice throughout the presentation of the material. We devised a way to

7

To be published in the Proceedings of the IEEE, January 2000

present the theory of adaptive systems by blending short explanations (including the more representative equations) with a software simulator that runs “live” examples illustrating the equation at work with data. The student learns the concepts through both the text/equations and the simulator within a very short cycle. Each reinforces the other, so the student is lead to utilize both during the learning experience. The goal is to create synergistic relations between the theory explained in the text/equations and the application of these concepts. The interface should therefore be as seamless as possible, such that the simulation becomes a natural extension of the text. We used a commercial simulator called NeuroSolutions [26] specially designed for adaptive and neural simulation. We explicitly create in the text an element called the NeuroSolutions Example (Figure 1).

Figure 1: The example element included in the text. The font color and type are different. The example ends with the icon that will launch the simulator.

The example text describes what the student is going to encounter in the simulation, and ends with an icon that when clicked will launch the simulator and run the example. The example itself is a model of an adaptive system (called the NeuroSolutions breadboard), already prepared to run the simulation at the touch of a button (Figure 2). The student does not need to write code or to spend

8

To be published in the Proceedings of the IEEE, January 2000

a lot of time before running the simulation, that is, the student has instant feedback from the experiment. An example normally includes several parts called panels and the last panel is a summary. In each panel there is a brief explanation of what the students will see and also an indication of things to try and to watch. The student goes back to the text (the place immediately after the NeuroSolutions example) at the touch of a button. Students will thus be able to execute the simulation and visualize the example effortlessly. All the examples include dynamic visualization probes that show numbers changing, curves being plotted on the fly, or audio outputs. We expect that the action visualized in the simulations will motivate the learners to try things of their own, such as changing the parameters of the system, changing data files, or even changing the network topology. An illustrative example of the integration achieved in our text is the following: In adaptive systems, the stepsize of adaptation (also called the learning rate) affects the stability of adaptation. In a linear system, the solution can be analytically determined from the eigendecomposition of the input autocorrelation matrix. We develop the stability of adaptation concept by asking the student to change the learning rate in the simulator and plot the learning curve. Soon they realize that for some values of the stepsize the error decreases with the iteration number, while in other cases the error explodes. Naturally the question arises how to determine the stepsize for convergence, and then mathematics appear as the saving grace. This style is apparent through out the presentation, not only in the simple case illustrated, but also in more complex concepts such as robustness of the adaptive solution, difference between regression and classification, or the role of this input layer in time lagged feedforward neural networks (just to name a few). The goal is to stimulate “what if” questions and allow for a direct way to check the answers. There is a fine line between a challenge that elicits a positive action from the student (try things out) and one that leads to withdrawal. For the present generation of students, which is used to instant gratification, we believe that this way of structuring the examples in the simulator will be conducive to self-study. The role of the professor in class is to push for action. 2.3.2 Encapsulation of mathematical detail Mathematical equations are useful in science and engineering because they are a universal lan-

9

To be published in the Proceedings of the IEEE, January 2000

guage to precisely condense and manipulate knowledge. However, from a practitioners point of view, the equations do not need to be derived at the same time the concepts are taught. Therefore we profusely utilize hyperlinks with “know-more boxes” to hide away the full mathematical description when the concept is addressed in the text (Figure 2). In this figure, the underlined text “derivation of the time constant of adaptation”, contains a link to a “know more box” with the derivation also shown in the figure. We also created pop-up boxes with the equations referenced in the text to help follow derivations or improve understanding. This organization encapsulating detail makes the presentation light and appealing the first time around, but contains the full detail for a more in depth study.

Figure 2: A panel of one of the examples showing the explanatory text, the adaptive system block diagram, the learning rate box, the regression line, the learning curve, the system parameters, and the final mean square error.

There is a second crucial point about mathematical formalisms that we wish to address. Mathematics has been used for encapsulating knowledge in the physical sciences and engineering due to the accuracy and universality of the description language. The problem in learning mathematics is similar to the difficulty experienced in learning a “foreign language” which represents concepts, rules and actions very differently from every day experiences. This is at the core of the “with-

10

To be published in the Proceedings of the IEEE, January 2000

draw” from mathematics facing the present generation of students, and it tremendously hampers professors in their quest to improve the curriculum. When writing an equation on the board we sometimes feel like a street artist painting “graffiti on a wall,” because the equation means different things to each undergraduate (including the empty set). This means that the power of the mathematical description language is being lost. Until this aspect is corrected, university professors have to find alternate formalisms to convey concepts precisely and effectively to students. We submit that simulators are a powerful alternative due to the formal nature of programming. The issue is to come up with sufficiently general simulators to span the different uses of mathematics in the physical sciences and engineering. At least we can use simulators for topical areas, as we have done here for adaptive systems. 2.3.3 Ordering of topics Very quickly we realized that we had to re-order the conventional presentation of the topics if we wanted a fine-grain integration between the simulator and the text. The reason is that a simulation is built bottom-up with the basic building blocks, while in the traditional mathematical approach we tend to present the most general case first and then particularize to each situation (top-down). The two methods have contradictory requirements. Maintaining the traditional top-down flow results in a textbook where the simulator is used only at the end of each chapter, as we find in most present books with computer based problems. We let the bottom-up approach dictate the order of the topic presentation, similar to [23], but in our case there is no distinction between concepts and laboratory topics. The integration is made at a deeper level, where the laboratory topic is spread throughout the chapter, each piece being incrementally added and blended with the theory. Figure 2represents an ADALINE (adaptive linear element) with the corresponding learning components and controllers to run the simulations. Each icon in the breadboard encapsulates a functional block of the adaptive system. If we start with a fully functional adaptive system the breadboard becomes very complex and difficult to explain, or has to be presented in a laboratory type experiment. Alternatively, we introduce in the text each piece of the ADALINE independently, and show its function. For instance, the first simulation constructs the parameteric mapper (the sum-of-products with the bias), then we show that changing the parameters modify the posi-

11

To be published in the Proceedings of the IEEE, January 2000

tion of the regression line. Next, we introduce the concept of the error between the input and the desired response and how to measure it. We can make the system “aware” of the error through feedback. We show that changing the system parameters affects the error, and then we present a systematic way of doing this using gradient descent (the learning algorithm). Only then do we start analyzing the properties of adaptation which require mathematically sophisticated concepts. The student naturally learns that an adaptive system is built from the mapping sub-system, the criterion of optimality and the adaptive algorithm. Moreover, the building blocks are reusable throughout the simulations which also helps create a functional definition for the adaptive systems families. This is one of the great advantages of an iconic user interface for instruction: Ultimately, building blocks exemplify the power of the divide-and-conquer strategy so useful in science and engineering and show the modularization in engineering design. The need for a different flow in the material presentation and a different teaching style are probably the most valuable lessons learned from our experience.

Figure 3: Example of the presentation of a concept (time constant of adaptation) and the corresponding “know-more box” with the derivation.

12

To be published in the Proceedings of the IEEE, January 2000

2.4. Teaching Style with i-books Alternating between the theory and the simulations is deemed crucial in our teaching methodology and it is mapped into the text flow and book organization. It enables the presentation of mathematically difficult concepts by complementing the equations with behavior in the adaptive system. The method not only cements concepts while they are still fresh but also keeps the interest level high. Moreover, this integration emphasizes the nature of engineering, where we build systems to solve practical problems. Therefore students do not learn the concept in a vacuum as is often the case, but they keep a grounding to reality. The examples illustrate the key aspects of the solution using both fabricated data to elucidate the concepts and real data to assess the applicability of the method. Unfortunately this “paper” article severely limits an illustration of this technique, but an example may still be useful. After we cover the least means square (LMS) algorithm three weeks into the semester, we illustrate the practical use of regression in class. We visit the NOAA (National Oceanographic and Atmospheric Agency) Web site to look at climatological data. Climatological data is a good example because every student has an idea of the variables and their interrelationship. Each student downloads a pair of time series from the site onto their classroom computer, and solves the regression problem utilizing one of the supplied breadboards. It is important to point out that the examples are not just “demonstrations”, they are interactive simulations which include experimentation by the student. Each student runs the simulator, must set the parameters for proper convergence, and is asked to provide the equation that “explains” the data, the final mean square error and the correlation coefficient. Going from a set of data to an equation that describes the relationship is something that amazes the student and clearly illustrates the idea of model building from data, so important in science and engineering. After teaching the course twice, we settled on the following teaching format: each topic is explained in about 10 minutes, the corresponding simulation is run by the instructor and the behavior explained bridging the theory with what is visualized. Then the students are allowed at least 10 minutes to run the same example by themselves, change parameters or data files and ask questions. The professor concludes the topic and moves to the next topic. Therefore, there is always a very short and alternating cycle between theory and application.

13

To be published in the Proceedings of the IEEE, January 2000

Most often students voice their questions, but frequently the professor needs to bootstrap the discussions by asking “what if” questions, and letting the student verify the answers by running the simulator. This means that the professor has to be intimately familiar with the simulator and with the topic material to effectively exploit the electronic environment. The students are encouraged to read the text before coming to class.

14

To be published in the Proceedings of the IEEE, January 2000

3. Constructing the i-book 3.1. Hypertext Writing and integrating a hypertext document with a simulator requires many skills that typically do not reside in a single individual. The knowledge and practice of teaching the subject material has to be complemented with a good simulator and an intimate knowledge of computers and programming to create the communications between the programs. Other practical aspects can not be forgotten such as availability of the software, and universality of the operating system/computer platform. We chose the Windows operating system, the defacto standard in the PC world. We decided to use the Windows Help format for the hypertext document since it can be used on any Windows platform without additional support files. We created the text in Microsoft Word and converted it to a hypertext document using RoboHelp [34], the most widely used software to create help files in Windows. We believe that new languages such as Java may become an appealing alternative to create the hypertext document, but they were strategically too risky when we started developing the electronic book concept in 1995. The organization of the text fits the hypertext document model [20]. The book is built from a collection of fine-grain concepts each with its own simulations (there are 200 simulations throughout the book). The concepts are divided into chapters and sequentially presented according to our vision of adaptive systems, very much like a traditional textbook (except for their ordering). Using Bloom’s taxonomy this organization corresponds to the comprehension level [4]. This organization simplifies the navigation throughout the electronic book for the novice learner, but it can not be the only way to navigate through the text. Browsing in a book is a key step to become familiar with a subject, but unfortunately we can not (yet) flip pages in an electronic book. Therefore we include at the end of each chapter a “concept” map of the chapter with a flowchart of the topics (tree design) and related concepts (Figure 4). Each box has a hyperlink that transfers control to the particular section of the text that covers the concept. This presents the flow of concepts in a structured manner, and although not as effective as browsing, it provides a goal oriented

15

To be published in the Proceedings of the IEEE, January 2000

access to the subject material. It corresponds to the analysis level of Bloom’s taxonomy.

Figure 4: An example of the concept map for Chapter 7. Each circle is a hyperlink that once clicked transfers control to the begining of the correponding section.

Although the i-book does not implement by hyperlinks any of the higher levels of Bloom’s taxonomy (synthesis and evaluation), these aspects are an integral part of the simulation examples. The simulation is where synthesis and evaluation of the learned material take place, by allowing the learner total freedom to change the topology, the parameters, and the data. This seems a much more natural way to achieve integration of knowledge than by allowing freedom of search in hypertext documents as proposed by [35]. We therefore believe that our i-book spans all the cognitive levels for instruction of the subject material.

3.2 The simulator Due to our proposed bottom-up approach to text writing, the choice of the simulator is critical in developing the i-book. Although Matlab is widely used for Digital Signal Processing instruction it is not an open software system, it is slow for adaptive systems (in particular neural networks), and it is too detailed when the goal is functional simulation. One of the top requirements for the i-book

16

To be published in the Proceedings of the IEEE, January 2000

is the seamless integration of the text with the simulator, which in the Windows environment means compatibility with protocols of parameter passing between different programs (as implemented by the Object Linking and Embedding standard - OLE). Matlab is weak in this respect. Speed is also crucial. If we have to wait 10 minutes to adapt a neural network, the student’s curiosity will erode and the impact will be lost. Matlab and Mathcad are programming languages, which are of real practical value for the practitioner engineer, but have steep learning curves. Although they can and have been used for introductory DSP courses (see [25] and [12] respectively), the complexity of the modules quickly gets out of hand for adaptive systems (adaptive filters and neural networks) instruction. A functional simulator such as Simulink or the icon interface Cantata for Khoros [8] makes more sense to teach adaptive systems to undergraduates, because it raps the complexity of the mathematics in interacting functional modules that can be visualized with icons. The problem with Simulink or Cantata is that training even the simplest adaptive system can be very slow. A simulator for adaptive distributed processing systems was jointly developed by the third and first authors of this paper, embedding our style, systematization, and experience of adaptive systems in the program structure. This package has since evolved into the commercial product NeuroSolutions. Undoubtedly, this intimate knowledge of the package and the common philosophy between our view of the subject matter and the package internal structure was very beneficial for text writing. For our simulation approach to text writing, the internal structure of the software simulator greatly influences the order of presentation of topics, and in this case we had no difficulty. Frequently we needed to write dynamic link libraries (DLLs) to illustrate special aspects of the theory not included in the package. Almost always we were able to implement the idea in an elegant way. Conversely, many of the features of the package were already designed with adaptive systems instruction in mind. NeuroSolutions has an icon-based user interface similar to Simulink or Cantata which is very appealing pedagogically. Systems are constructed Lego style by dropping components on a white screen called the breadboard. This resembles circuit construction familiar to students, and is also reminiscent of Papert’s block worlds. The fundamental components of the package exist in palettes, are logically organized into functional families, and are mapped into icons. The mathemat-

17

To be published in the Proceedings of the IEEE, January 2000

ics for each family are well documented in the Help facility. NeuroSolutions contains a complete set of algorithms necessary for adaptive system instruction and has an object-oriented engine tailored for efficient adaptive systems simulations. Additionally, NeuroSolutions has the following useful features: it is OLE compatible which allows for tight integration with other windows programs (including the hypertext document); it allows for the creation of custom components (DLLs) which enable unique methods of demonstrating concepts; it includes a macro language, which greatly facilitates the creation of the examples; and it has many flexible visualization tools. For instance, the idea of probing the neural network to find out how the system is achieving the input-output map is very important for teaching (and in general for quantifying the solution). The probe family in NeuroSolutions provides versatility and powerful visualization capabilities. In Figure 5 we are visualizing the discriminant function of the system, the filter outputs, the discrimination space, and the actual output of the system during learning. This provides a unique way to understand what goes on inside the system when it is adapting.

Figure 5: This breadboard shows a TLFN (time lagged feedforward network) interpreted as the cascade of a linear filter stage and a static linear mapper. The displays show (left to right): the desired response, the two filter outputs, and the input signal; the two filter outputs in signal space; the output of the filters as a discriminant space; and the discriminant function discovered by the MLP to achieve the desired response.

18

To be published in the Proceedings of the IEEE, January 2000

Due to the integration with a simulator, writing an electronic textbook is far different from writing a typical textbook. An extensive knowledge of the simulator is necessary to find ways to best illustrate the concepts. The professor has to develop a “feel” for the simulator to decide what can be demonstrated better by equations or by simulation. Painstaking detail is required to create the simulation examples, find the parameters that work best, and then integrate them with the text. The seamless integration of hypertext and simulations in the i-book was implemented through a macro language which effectively allows the package to be controlled from an external program by sending macro-commands. A NeuroSolutions example is therefore nothing but a sequence of macros that construct the breadboard, set parameters, open files and probes as required for the illustration of the concept. Our conclusion is that due to the additional work and skills required, a team of authors is typically required to write an interactive electronic book.

19

To be published in the Proceedings of the IEEE, January 2000

4. Materials Adaptive systems (adaptive filters and neural networks) are becoming ubiquitous in engineering applications [15], as the new advances in object recognition [21], speech recognition [32], intelligent control [29], and neural networks [14] clearly demonstrate. We anticipate that the next generation of major engineering systems will contain embedded adaptive sub-systems to better exploit the increasing amount of available data or to seek optimal operating conditions. Adaptive systems are becoming an integral part of electrical engineering systems, so if we do not innovate our curricula, new products embedding adaptive algorithms will be delayed because there is an ill-prepared work force to design, implement and test adaptive systems. There are other advantages in teaching adaptive systems to ECE undergraduates. The idea of a data model is critical in experimental science, and unfortunately it has been diluted in the engineering degree because it is traditionally dressed and taught as statistics which is “universally unappreciated” by undergraduates. Adaptive systems courses will give back the importance of data modeling and the “learning from example” design metaphor, but now with an application specific focus which is more useful and appealing to the undergraduate. However, adaptive systems are not traditionally taught to ECE undergraduates due to the sophisticated mathematics required to explain the material. There is an enormous body of knowledge and many books to teach adaptive systems at the graduate level [16], [17], [41], [19], but very little at the undergraduate level. Similar to McClellan’s approach to teaching DSP before analog circuits and systems by using computer enhanced lectures, we also propose to innovate the undergraduate curriculum by teaching adaptive systems with an i-book. The major challenge in teaching adaptive systems to undergraduates is their lack of mature and sophisticated mathematical knowledge. The i-book we describe here will be published by Wiley [31] with the title “Neural and Adaptive Systems: Fundamentals Through Simulations”. The book is a comprehensive treatment of adaptive and neural systems with over 600 printed pages and 200 simulations. The topics start with regression and gradient descent learning, elements of statistical pattern recognition, and cover single and multilayer perceptrons (MLPs), and radial basis functions (RBFs). Data representation (principal component analysis) and associative memories are also presented, along with clustering algorithms. Time processing is reviewed and blended with regression to provide optimal filtering and its nonlinear extensions by

20

To be published in the Proceedings of the IEEE, January 2000

substituting the linear processing by MLPs and RBFs. Hence, the book unifies adaptive filtering and pattern recognition, two related topics that have seldom been presented in the same textbook.

21

To be published in the Proceedings of the IEEE, January 2000

5. Classroom Format From the start we envisaged students using the computer during every lecture very much as proposed in [11]. All ECE departments have computer laboratories, but few are set up for classroom instruction. Normally a computer Lab is a large room full of computers for indiscriminate student use. Many courses have their own laboratories but in our institution computers are setup in benches, cluttered with other instrumentation and they do not face the instructor. We were fortunate to have secured an NSF Instrumentation and Laboratory Improvement (ILI) grant that enabled the purchase of PC equipment to set up a classroom specifically designed to meet our needs. This classroom was named the Interactive Teaching Laboratory (ITL). The ITL is laid out in the traditional style of desks facing the instructor. However on each desk we placed one PC with the corresponding internet connection, creating an intranet. A multimedia Pentium 133 MHz with SVGA performs adequately. The instructor workstation is a more powerful PC and operates as a Windows NT server. The student stations are NT clients, with read permission on the electronic book directory. The NT server also operates as the WEB server for the class assignments. This arrangement allows quick access to the internet during classes to search and download data. Additionally we use an LCD projector and an electronic white board (Smart board) to allow projection of the computer display onto the white board. This arrangement is very effective for classroom presentation, since the instructor’s finger (or supplied pens) work as the mouse. The instructor can thus navigate through the hypertext or execute the simulations by simply touching the electronic white board. The instructor stands in front of the class and teaches by selecting the material, circling formulas, underlying text, doing short derivations or illustrating concepts with figures. This provides a very intimate contact with the material that is normally lacking with conventional projection on a screen. The electronic white board is also very handy during the simulations because almost all the functions in the iconic interface can be actuated with the mouse, i.e. with the finger. All the aspects of demonstrating the simulator can be accomplished very naturally by standing in front of the class, not sitting hidden behind a computer screen (Figure 6). Similar, but more sophisticated, experiences have been reported in the literature such as the classroom 2000 at Georgia Tech [1], where remote access is provided to the classroom, and students use pen-

22

To be published in the Proceedings of the IEEE, January 2000

based technology.

Figure 6: The smart board with the projected computer screen and the conventional white board.

In this new learning environment, the professor’s role is more like the conductor of an orchestra during a rehearsal: highlighting the importance of topics, providing the timing for instruction, and answering questions. These are the undisputed benefits of having the professor in the loop for instruction, and in our classroom format we are fully utilizing them. We even dare to say that the availability of the simulator frees the professor from the nitty-gritty details of making sure that the formulas are correct, or that the calculation is done right, and allows total concentration on the three important aspects of teaching named above. The price paid as we stated above is the time required to gain an intimate knowledge of the material and of the simulator. One of the aspects to consider is the utilization of facilities required by this teaching methodology. Since the textbook and the examples are on the computer, students must have access to the classroom beyond the lectures. We have been scheduling daily 2 hour periods of access to the classroom. Office hours are also conducted in the classroom. Students who own computers (as has been recently required at our university) do not need laboratory time since they can access the NT server through the internet for assignments and purchase the electronic book CDROM. Up to now we have put the electronic book on the ITL server for download to the registered students (with the permission of the publisher).

23

To be published in the Proceedings of the IEEE, January 2000

6. Teaching Experience The course was taught on an experimental basis as a senior elective in the Spring of ‘97 in one of the Department’s computer laboratories, and in the ITL during the Spring ‘98 and ‘99. Ten students were enrolled each year. During the first year the teaching conditions were far from satisfactory because the room was too big, with many computers and was not prepared for instruction. It was difficult to create the atmosphere conducive to learning by experimentation which we seek with the simulator. The response of the students and our own assessment by teaching in the ITL is overwhelmingly positive. We are conveying difficult concepts not only by mathematics but also by interacting with the simulator and observing its behavior. This reaches a far larger student population than the more traditional equation based learning. The students are caught off-guard with the “high-tech” classroom and this typically motivates them and makes them naturally curious about the course content as well. In the beginning of the course there is a steep learning curve, since the students must learn not only the material but also the simulator. However, once this is achieved, they reap immediate benefits for learning. Lectures tend to be lively with our format of sandwiching explanations with examples. When a live simulator is given to the students, their tendency is to “break it”. So the professor receives many questions of “why do I get this strange behavior”. We think that this is very positive, because in the process of trying to break something, the student normally learns the normal operating range and the limits of the method. An example is the largest stepsize for convergence. Even before this topic is covered in class, we always have a couple of questions about overflow. Of course many times we can not explain what happened, so there is the danger of sterile discussions, as is the case of laboratory instruction. One of the highlights of the course is that it is really a blend of theory with design. The integration of the hypertext and the simulator allows us to teach problem solving rather than just theory. As we explained before, three weeks into the semester we are browsing the WEB in search of real data to download to the simulator and performing linear regression. We setup the problem formulation in an adaptive system framework and found excellent examples that highlighted the power of the technique. Of course the student’s knowledge is mediated by the existence of the simulator,

24

To be published in the Proceedings of the IEEE, January 2000

so they need to have this tool available when in the future they apply the learned material. We do not know how to compensate for this aspect, but it is common throughout the use of technology, irrespective if the tools are books, instruments, operating systems or simulators. The class assignments go far beyond what we are used to in undergraduate education where the examples have to be cleverly selected to obtain simple workable answers. In this course, the first project was an “open” assignment on data modeling. The students were requested to select a problem that captured their attention, formulate a hypothesis, and validate their hypothesis using regression. We received excellent projects from most student. Since we covered climatology in class, we always receive a couple of weather related data projects. But the majority of projects address very different topics such as: correlation between crime rate and law enforcement, between longevity and GNP, between SAT scores and performance in university studies, correlation between professor salaries and tuition, etc. The second project deals with the topic of classification of a given data set. We use a handwritten optical character recognition data set with segmented digits. Students enjoy the project because they realize the power of the tool for their own engineering careers. The third project is an adaptive filtering application where the data is supplied by the instructor. We use audio files such that the students can attach meaning to the outcome of the processing (e.g. cancelling washing machine noise from a voice segment). Students can propose their own topics for the projects, but they have to discuss the topic with us in advance. A final exam was given testing the conceptual understanding of the material. Here we found that the student performance tends to be clearly clustered in two groups. The majority of students is able to master the simulator and the application of the technology to practical problem solving, and are also able to thoroughly understand the fundamental concepts of the material. The rest mastered the use of adaptive systems (and the simulator) but did not absorb the concepts and theory as well as expected. We believe that one of the dangers of interactive learning environments is an excessive differentiation among the students. If the student is interested and has a good background, he/she can very rapidly absorb large amounts of information, well beyond what we might expect from an undergraduate student. Last year one of the undergraduate students proposed a topic that lead to a pub-

25

To be published in the Proceedings of the IEEE, January 2000

lication of a conference paper, which is unusual. On the other hand, average students may struggle with the extra level of difficulty and never reap the benefits of their effort. Hence the “dynamic range” of student progress should be constantly monitored. The lab periods and the office hours are very important because there the professor has an opportunity to coach students in a one-onone basis and “equalize” progress. We have not yet initiated a pedagogical evaluation of the course material nor of the teaching format. We started this year, under a NSF CRCD grant, collaborating with the Education Department toward this goal, and there is a rich literature to help us evaluate both aspects of interest [2], [10]. We still would like to present, as a mere indication, last year’s course review by the ECE Department Peer Teaching Review Committee. The committee praised the effectiveness of the electronic white board and the i-book for in-class teaching, and the availability of the simulator for hands-on participation. Table I summarizes the lectures and course evaluations done by the students. There was also a teacher evaluation which is not shown here. Table 1: Student Reviews (1 best, 5 worst) student

Effectiveness of class lectures

Effectiveness of the course

1

1

1

2

1

3

3

1

1

4

1

2

5

4

1

6

1

1

7

3

1

8

1

1

9

1

1

10

1

1

Mean

1.5

1.3

St. Dev.

1.08

0.675

The transcribed comments include the beneficial home use of the simulator to repeat lectures, the importance of the applications, the quality of the examples to illustrate the concepts, the effectiveness of the computers and electronic white board in class. Concerns expressed by the students

26

To be published in the Proceedings of the IEEE, January 2000

were centered on the speed of the delivery of information and the lack of conventional homework assignments. We are correcting this aspect by providing more homework that include not only problem solving but also confirmation of answers with the simulator.

27

To be published in the Proceedings of the IEEE, January 2000

7. The future The printed book has been finely perfected through the ages and today it is the undisputed medium for the repository of human knowledge. The printed book has wonderful advantages but also limitations because it is a static medium. Information technologies may overcome this limitation but the computer technology only recently has crossed the threshold of usability [28]. Therefore we have a long road to tune the electronic book format. We believe that information technology is posed to revolutionize textbooks as deeply as the invention of the printing press in 1440. But we must be as bold as Gutenberg to imagine a general purpose, flexible, effective format which will allow a professor to efficiently cast his/her knowledge into a lively, appealing format that students will enjoy using. We are still very far away from this goal. First, building an i-book is still a rare group effort because there are no general purpose software tools that would allow a professor to create all the facets of the medium (simulation, text, figures, multimedia, and browsing control). Second, we still do not know what actually works, so we have to create many examples before conclusions are drawn. Third, we have to conquer the hearts and minds of the instructors still very much used to the old ways of teaching. There are four major paths for furthering the work described here: improving the i-book format, usability of the i-book for teaching, distance learning with the ibook, and spreading the teaching format to ECE disciplines with i-books.

7.1 Improving the i-book format The present i-book format has many advantages over the textbook for home studying. Students have the capability to duplicate exactly what has been done in class, and take the time to learn every step at their own pace. We believe that for professionals the lay-out and interactiveness of CDROM books lead wonderfully to self-study, even if they are not attending a course. We have only scratched the surface for writing i-books. Three aspects that we would like to pursue are: adding the capability for student annotations, allowing for the customization of electronic books, and performing testing on the student’s knowledge. The addition of student notes to the CDROM text is just a technological problem of synchronizing the displayed text with a transpar-

28

To be published in the Proceedings of the IEEE, January 2000

ent layer accepting handwritten notes. These pages will be saved into a different file and overlaid when the student reads the text at home. Xerox PARC already has such software available [40]. Customizing i-books is an exciting possibility. In principle, an electronic book for a topic can contain all the information from the overview to the most detailed and advanced research paper. The problem is one of hierarchically organizing the knowledge, mapping it to a database structure and allowing the configuration of the i-book according to the reader’s level of knowledge, i.e. from novice to the researcher. Once these difficulties are conquered, the i-book will be a dynamic entity, with the text being compiled on the fly depending on the readers’ expertise. Testing the performance of learners is also a way to provide a quick feedback to students on how efficient their study was. We envisage at the end of each book section a simple questionnaire with multiple answers that students have to work out before they are allowed to go to the next section (when the test mode of the i-book has been selected). The i-book can automatically link to the material where the concepts of the incorrect answers reside. This drill is essential for learning and can be enforced easily. Likewise, the simulator can test for mouse clicks to configure a given breadboard and provide feedback on how reasonable the student configuration is.

7.2. Usability of i-books for teaching We have developed a methodology to effectively use i-books for adaptive systems instruction. There are a few things that will make teaching more efficient. During classroom teaching we would like to have the ability to copy equations or definitions to a second screen that could be tiled with the normal presentation. This is important because sometimes we need to refer to key concepts described earlier in the lecture. Presently, we use a conventional white board, which is fine for in-class instruction, but destroys the distance learning capability. An intermediate solution is a set of page marks which we could quickly go to. A better but more expensive solution is to use two electronic white boards. Presently our students use the i-book for study and during the classroom lectures. But we believe that there are several degrees in the usage of i-books for instruction. The instructor can use the ibook for class delivery, instead of writing notes on a blackboard or using transparencies, while the

29

To be published in the Proceedings of the IEEE, January 2000

students attend a normal classroom (without computers). The instructor benefits from the simulator to illustrate concepts. The availability of the CDROM for student self study is already considered an improvement versus the current level of paper books due to the simulator. The fundamental aspect that needs to be tested is how comfortable other instructors will be with the i-book we created. The reason is that the i-book format being less static will interact with the instructor, so his/her reaction has to be carefully evaluated. We plan to evaluate this aspect during the team teaching experiments to be conducted next year under the CRCD grant (see below).

7.3 Distance learning with i-books The recent interest in distance learning will accelerate the innovation of the textbook in spite of the fact that the present thrust in WEB-based instruction has been centered on the enabling electronic technologies rather than on rethinking the textbook for the information age. We too often see million dollar investments for information delivery to find out that the instructional material is simply a scanned version of a conventional textbook or “slide shows”, while the actual lectures are video distorted versions of the real scenes due to bandwidth limitations. These are great experiences, but we submit that the use of the internet and computers for instruction requires a paradigm shift in the way we interact with and create textbooks. After all, electronic delivery of information requires generation of information in a format that fully exploits the electronic medium. Our group at the University of Florida is in a favorable position to innovate distance learning methodologies in the area of adaptive systems since the source of information (the book) is already in electronic form. Our present goal is centered on improving the interaction of remote students with the instructor and the rest of the class. Presently, distance learning requires large bandwidths and remote student interaction is limited to an off-line mode (chat rooms, downloads etc.). Some may argue that the next generation of fiber optical cables and asynchronous protocols (internet II) will solve the access problem. However, the short history of the WEB tells us that the demand will increase faster than the technology innovation, so we have to intelligently think about how to save bandwidth. Therefore we submit that in order to be practical, remote student access should be done through modems or wireless networks for price and load performance rea-

30

To be published in the Proceedings of the IEEE, January 2000

sons. We recently proposed to the National Science Foundation (CRCD program) a scheme to allow remote real-time distant learning interaction using low-bandwidth modems or wireless networks. This is possible provided that the remote student has a computer running the CDROM book used in class. We plan to achieve remote teaching by synchronizing the student computer with the professor’s computer during the explanation portion of the topic and provide a real-time audio feed with the professor’s voice. During the simulation period we will allow the remote student to interact with the class as any other in-class student through a real-time audio feed. We will queue the requests, and accept one student at a time. One computer in class will be controlled by the remote student such that he/she can point to the text in question, or to a result of the simulator that he/she does not understand. This can be done with low bandwidth communications because once again it is just a matter of synchronization between computers and mouse clicks. These actions will be synchronized with the student voice. Video will be auxiliary in the scheme we discussed, because all the information is available locally in digital form. We will include video when the bandwidth is available, but it is not the major vehicle for transmission of information as in most distance learning experiences. An interesting topic is team teaching across the web using the interactive electronic book. Our CRCD grant includes the MIT and McMaster Universities as joint sites for the team teaching experiments. When we solve the remote student interaction as discussed above, we have also opened the door to allow a remote professor to teach our in-class students. Imagine a team teaching environment where each topic is taught by the expert in the area. This could not only tremendously improve education in general but help smaller universities offer high-tech courses. Many times, the synchronous learning modality that we advocate has to be substituted by asynchronous learning. The availability of the CD-ROM enormously facilitates the study, since students already have in their own computers the material in high quality format. They only need to receive the audio feed and the synchronization commands from the WEB site, and post their questions in chat rooms.

31

To be published in the Proceedings of the IEEE, January 2000

7.4 Spreading the i-book format for ECE instruction Although the described classroom and i-book formats are very flexible, they have been developed solely from our experience in teaching adaptive systems. Other ECE areas may require distinct combinations of text and simulators that we have not considered. But the flexibility of the computer will very likely provide appropriate solutions, given enough time and expertise to specify and program the solutions. We are aware that our simulator limits the instruction to adaptive filters and neural networks. The general purpose simulators mentioned in the literature review have the potential to be integrated with hypertext documents, so we foresee similar experiences in other ECE areas such as electronic circuits, telecommunications and signals and systems. Our recommendation is that the extra step of seamlessly integrating the text with the simulator is crucial for interactive teaching, in spite of the fact that it is the most time consuming step. Until the software industry gets interested in the educational market and develops computer-based authoring tools that allow a single professor to write i-books, innovation will probably require team efforts. But our experience shows that a small team is capable of developing an i-book and take the challenge of rethinking the textbook and classroom for the next Millennium.

Acknowledgments: This work was partially supported by NSF grants DUE-9751290 and EEC-9872526.

References [1] Abowd G., C. Atkeson, A. Feinstein, C. Hmelo, R. Kooper, S. Long, N. Sawhney, M. Tani, Teaching and Learning as multimedia atuthoring: the classroom 2000 project”, Proc. ACM Multimedia’96, 1-12, 1996. [2] Atman C., Bursic K., “Documenting a process: the use of verbal protocol analysis to study engineering student design”, J. of Engineering Education, Special issue on Assessement, 121-132, 1998. [3] Bamberger R., B. Evans, E. Lee, J. McClellan, M. Yoder, “Integrating analysis, simulation and implementation tools in electronic courseware for teaching signal processing”, Proc.

32

To be published in the Proceedings of the IEEE, January 2000

IEEE Int. Conf. Acoustics Speech and Signal Proc., 2873-2876, 1995. [4] Bloom B., M. Engelhart, D. Furst, W. Hill, D. Krathwohl, Taxonomy of Educational Objectives: The Classifications of Educational Goals, David McKay, New York, 1956. [5] Ca C-L., J. Kulich, “effectiveness of computer based intruction: an update analysis”, in Computers in Human Behavior 7, 75-94, 1991. [6] Chiang K., Evans B., Huang W., Kovac F., “Real time DSP for sophomores”, Proc. IEEE Int. Conf. Acoustics Speech and Signal Proc., 1097-1100, 1996 [7] Clausen A., A. Spanias, A. Xavier, M. Tampi, “A Java signal analysis tool for signal processing experiments”, Proc. IEEE Int. Conf. Acoustics Speech and Signal Proc., 1849-1852, 1998. [8] Donohoe G., P. Valdez, “Teaching digital image processing with Khoros”, IEEE Trans. Education, vol 39, #2, 137-142, 1996. [9] Etter D., Orsak G., Johnson D., “Distance teaming experiments in undergraduate DSP”, Proc. IEEE Int. Conf. Acoustics Speech and Signal Proc., 1109-1112, 1996. [10] Guzdial M., J. Kolodner, C. Hmel, H., Narayanan, D. Carlson, N. Rappin, R. Husbscher, and J. Turns, “Computer support for learning through computer problem solving”, Comm. ACM vol 39, #4, 43-45, 1996. [11] Harger R., “Introducing DSP with electronic book in a computer classroom”, IEEE Trans. Education, vol 39, #2, 173-179, 1996. [12] Harger R., “Teaching in a computer classroom with a hyperlinked, interactive book”, IEEE Trans. Educatioin, vol 39, #3, 327-335, 1996. [13] Harris D. and A. DiPaolo, “Advancing asynchronous distance education using high speed networks”, IEEE Trans. Education, vol 39, #3, 444-449, 1996. [14] Hwang J., Kung S., Niranjan M., Principe J., “The past, Present and Future of Neural Networks for Signal Processing”, IEEE Signal Proc. Magazine, 28-48, November 1997. [15] Haykin, S., "Neural networks expand Signal Processing horizons", IEEE Signal Processing Magazine, vol. 13, No. 2, 24-49, March 1996. [16] Haykin S., Neural Networks: A Comprehensive Foundation, McMillan, 1995. [17] Haykin S., Adaptive filter theory, Prentice Hall, 1996. [18] Haugeland J., Artificial Intelligence: The Very Idea, MIT Press, 1985. [19] Honig M., Messerchmitt D., Adaptive Filters: Structures, Algorithms and Applications, Kluwer1984. [20] Jonassen D., Hypertext/hypermedia, Educational Technology Publications, Englewood Cliffs, 1989.

33

To be published in the Proceedings of the IEEE, January 2000

[21] LeCun, Y., L. Bottou, Y. Bengio, P. Haffner, "Gradient based learning applied to document recognition”, Special issue on Intelligent Signal Processing (Eds. Haykin and Kosko), vol 86, #11, 2278-2324, 1998. [22] Madisetti V., J. McClellan, T. Barnwell, “DSP design education at Georgia Tech”, Proc. IEEE Int. Conf. Acoustics Speech and Signal Proc., 2869-2872, 1995. [23] McClellan J., R. Schafer, M. Yoder, “Experiences in teaching DSP first in the ECE curriculum”, Proc. IEEE Int. Conf. Acoustics Speech and Signal Proc., 19-22, 1997. [24] McClellan J., R. Schafer, J. Schoforf, M. Yoder, “Multimedia and world wide web resources for teaching DSP”, Proc. IEEE Int. Conf. Acoustics Speech and Signal Proc., 1101-1104, 1996. [25] McClellan J., Schafer R., Yoder M., DSP First, Prentice Hall, 1998. [26] NeuroSolutions User’s Manual, NeuroDimension Inc., Gainesville, Florida, 1995. [27] Orsak G. and D. Etter, “Connecting the engineer to the 21st century through virtual teaming”, IEEE Trans. on Education, vol 39, #2, 165-172, 1996. [28] Proceedings Electronic Book ‘98 Workshop, NIST Oct 8-9, Gaithersburg, 1998. [29] Puskurius G., Feldkamp L., "Neurocontrol of nonlinear dynamical systems with Kalman fitler trained recurrent networks", IEEE Trans. Neural networks, vol 5, #2, 279-297, 1994. [30] Penfield P., and R. Larson, Education via advanced technologies”, IEEE Trans. Education, vol 39, #3, 436-443, 1996. [31] Principe J., Euliano N., Lefebvre C., Neural and Adaptive Systems: Foundations through Simulations, Wiley, in press. [32] Rabiner L. and Juang B-H. “Fundamentals of Speech Recogniton”, Prentice Hall, 1993.5.Intelligent Signal Processing, Special Issue on Proc. IEEE, November 1998. [33] Rahkila M., M. Karjalainen,”An experimental architecture for interactive web-bsaed DSP education”, Proc. IEEE Int. Conf. Acoustics Speech and Signal Proc., 1857-1860, 1998. [34] RobotHelp Manual, Blue Sky Software, La Jolla, CA, 1995. [35] Ross T., “, Bloom and Hypertext: Parallel taxonomies?”, ED-Tech Review, 11-16, 1993. [36] Schodorf J., Yoder M., J. McClellan, R. Schaffer, “Using multimedia to teach the theory of digital multimedia signals”, IEEE Trans. Education, vol 39, #3, 336-341, 1996. [37] Strum R., D. Kirk, Contemporary Linear Systems, PWS, Boston, 1996. [38] Sun C., and C. Chou, Experiencing CORAL: design and implementation of distance cooperative learning”, IEEE Trans. Education vol 39, #3, 357-366, 1996. [39] Shaffer J., J. Hamaker, J. Picone, “Visualizaiton of signal processing concepts”, Proc. IEEE Int. Conf. Acoustics Speech and Signal Proc., 1853-1856, 1998.

34

To be published in the Proceedings of the IEEE, January 2000

[40] Weber K., A. Poon, “A tool for real timevideo logging”, in Proc. ACM-CHI’94 Conference, pp 58-64, 1994. [41] Widrow B., Adaptive Signal Processing, Prentice Hall, 1985 [42] Zoltowski M., J. Allebach, C. Bouman, “Digital signal processing with applications” a new and successful approach to undergraduate DSP education”, IEEE Trans. Education, vol 39, #2, 120-126, 1996.

35