Session 13d4 Learning Science by Doing Science on the ... - CiteSeerX

Session 13d4 Learning Science by Doing Science on the Web Suzanne Keilson, Elliot King, and Megan Sapnar Electrical Eng. and Eng. Science, Writing and Media Loyola College Baltimore, MD 21210 Abstract - An interdisciplinary team from Loyola College in Maryland's departments of Engineering, Physics, Computer Science, and Writing and Media has created the Internet Science Institute; a project to use the Web to make lab-based learning attractive and accessible. Part of the project's core philosophy is to teach science by having students participate in the scientific method. The first of a series of planned modules has been developed to test such fundamental issues as Web usability, the incorporation of multimedia, overall module and site structure, and learning effectiveness. The topic of the first module is the motion of the pendulum and the structure that we used guides students through three levels of experience. The first is a "hands-on" lab that utilizes readily available materials. A more "refined" lab experience then layers on more detail to the phenomena under discussion, and focuses on data acquisition, test and measurement issues. The final section of the module is based on mathematical models and simulations (JAVA applets) and conceptually focuses on the most sophisticated aspects of the phenomena or their broader application. These modules are part of a larger Web site, the Internet Science Institute, which is planned to incorporate many other Web and local resources, such as electronic bulletin boards and cataloged resources. With this "layered" approach to the learning modules we hope to achieve maximal effectiveness and usability across a broad spectrum of a potential audience both at Loyola College and in the wider education community. For example, the approach is flexible enough so that a middle school student could learn what parameters govern a pendulum's period and a college or even graduate student could explore the Web materials and links on non-linear dynamics, differential equations, and chaos. We are especially interested in bringing such experiences in science learning to non-science college majors. Funded by NSF grant 2-7321-28128.

effectiveness to guide development suitable for this new medium. This project aims at taking the first steps to address that need. One aim of the Internet Science Institute is to ground scientific theory and learning in a physical experience. One comes to the Internet Science Institute Web page (www.isi.loyola.edu - currently requires password authorization) and is greeted by a choice of disciplines computational physics, physics, and computer science. Only one detailed module has been created, under the physics "wing"- physics lab section. Once the pendulum experiment site is entered all of its "submodules" are laid out visually on a single page. It is intended that the ideal navigation is through pendulum notes (background) to introductory experiment, data analysis (lab notebook), laboratory experience (parts I and II) and finally to mathematically modeling and simulations. The main page for navigation through these sections is shown in Figure 1. The structure should be thought of as consisting of three major sections the "hands-on" element, the "refined" lab with images and video of a lab experiment from set-up and data acquisition to data analysis, and, finally, the simulations section. An important element of all three conceptual sections of the learning module is the use of linked application packages (word processing and spreadsheet - Microsoft's Word and Excel ) to create lab notebooks, which the student can print and save. Figure 2 is an image screen capture of the Web page (with scrolling) that details the more sophisticated laboratory experience, data acquisition and analysis. This same information is provided as a "high bandwidth" option using RealPlayer Video. Both the relatively static "text and pictures" approach and the more dynamic approach of "plugins" such as the video have their drawbacks.

Introduction

A small group of student volunteers agreed to test this first module. The volunteers were drawn from a first year Introduction to Engineering class and a sophomore Engineering Mechanics class. This was originally intended as a required exercise for those classes, but when problems cropped up with the computer platforms having the appropriate capabilities, the implementation of a "hands on" experience, and the general readiness of the computer labs it became an extra credit incentive. The lesson learned, not a new one for educators, is that one can never be too careful

The use of hypertext and multimedia in science education has grown rapidly in the 1990s. The emergence of the Web in the mid-1990s has added new opportunities for using text, audio and video material in education. Although early experience has indicated great potential, most efforts are still exploratory and rudimentary, tied to individual courses [13]. Designing educational material for Web-based learning first requires careful experiments of usability and

Web Usability Study

0-7803-5643-8/99/$10.00 © 1999 IEEE November 10 - 13, 1999 San Juan, Puerto Rico 29th ASEE/IEEE Frontiers in Education Conference 13d4-7

Session 13d4 when planning the details of any kind of lab or demonstration. Observers were drawn from a class in the writing and media department on the methodology of surveys and other measures used in the social sciences. They observed and recorded the activities of the engineering student testers as they performed the Web lab for the first time. Thus, objective measures of time-on-task, time-to-completion, and difficulties encountered were obtained. Lab notebook computer files were collected to assess learning effectiveness. Figure 1: Screen Shot of Pendulum Experiment Entry Page.

Evaluation of the student testers consisted of (1) a usability survey filled out by observers, (2) a navigation questionnaire filled out by the student testers, (3) a conceptual survey to test learning effectiveness, (4) lab reports from the student testers, (5) and a summary from the student observers. The usability survey consisted of three basic questions with some additional annotations: 1. How easy was the site to navigate? Observe. a. Is the student doing the lab in logical order? b. Was the student stumped anywhere within the site? Why? Technological problems/waits and pauses? Where? Rate of purposeful behavior (steady?) Difficulty understanding directions/content? Where? Ask a friend or the surveyor for help? Why? c. Was the lab completed correctly? Observe Did the student do the lab according to directions? Did the student save three files as they were supposed to?

2. Could the student complete the lab in the tie allotted? (90 minutes, uninterrupted) 3. Demographic information How often do you use the Internet in a given week? Have you ever used the Internet in a classroom setting before? What is your major? How much computer experience do you have on a scale of 1-10 with 1 being least and 10 being most experienced? Not surprisingly the usability survey was generally favorable to the site design, given the fact that the testers were people who were already somewhat familiar with computers and the Internet. The navigation questionnaire covered similar topics as the survey, but rather than from the viewpoint of a neutral observer, the questions are asked of the explicit perceptions of the user on whether specific elements of the sequence where confusing or didn't work. There were eight questions such as was it obvious where to go next, or were the pop-up windows confusing when navigating through the site. 1. How often did you feel lost or confused navigating the structure of this web site? 2. Was it obvious to know where to go next? 3. Did the Microsoft Word lab documents open the first time you tried them without any difficulty? 4. Did you use the stopwatch provided 5. If you did try the stopwatch, did it work, was it helpful, or did it get in your way? 6. Did the first Pendulum experiment introduction page adequately introduce you to what was to come? 7. Did you find the lab notebook pop-up windows confusing when navigating through the steps of the experiments? 8. In the course of participating in the experiments, did you ever click on the Experiment Menu on the bottom of the screen? Why? The summary from the observers detailed what they saw as difficulties in the particulars of the lab set up or other issues that cropped up during the sessions. The conceptual survey consisted of a four question "quiz" to see if students were getting the gist of the physical concepts of the pendulum lesson without looking back at any of the materials. These were short answer "fill-in-the-blank" type questions. In addition students had to demonstrate understanding by sketching the motion of the pendulum versus time and labeling some information on the graph. 1. The period of a pendulum depends on ______. It does not depend on _______. 2. Pendulums come to rest because of ________. 3. Sketch the motion of a pendulum versus time. On the sketch indicate how you would calculate the period dot the pendulum

0-7803-5643-8/99/$10.00 © 1999 IEEE November 10 - 13, 1999 San Juan, Puerto Rico 29th ASEE/IEEE Frontiers in Education Conference 13d4-8

Session 13d4 Figure 2: Screen shot of Pendulum module laboratory experience page ("low bandwidth" version - text and figures only).

4.

A mathematical model is an ________. A mathematical model of the motion of a pendulum can include the effects of _____________ and _______.

Finally, the lab reports from the student testers were also collected. These consisted of word processing and spread sheet documents that were linked to the web site, were completed by the students and then saved to diskette for submission. An example of some of the figures, data, and answers from the three sections of the pendulum module that one student submitted are displayed in Figures 3-5, at the end of this paper. In Figure 3, the student had to supply the data in the number of swings column, by actually performing an experiment with string and washers. In Figure 4, the student was provided with the data, but had to do some elementary data analysis and explain their calculations. Finally, in Figure 5, the student was just provided with column headings and a reference to a JAVA applet located at http://www.lightlink.com/sergey/java/java/pend1/index.htm and had to respond to the assignment in a more open-ended manner. These lab reports, as just described, were guided exercises, with a number of questions for analysis after each section. The convenience of incorporating the word processing file and spreadsheet into the Web-based module can not be overstated. It was easy to link, easy for the students to use, and alleviated a lot of the frustration that students may feel about writing up lab reports.

Results To summarize the results from these sources of information about web usability and efficacy we found that the best source for information was the observer's report. This tended to provide a more frank portrayal of what occurs in an interactive session. Some of the key findings from the observer's reports ranged from mundane details that were troublesome to the users to insights into the nature of the interaction of students with the computer, each other, and their engagement in the learning and discovery process. Some key findings can be listed as follows: • Pop-up windows can be confusing even to this relatively experienced group. • The hands-on experimental portion had numerous problems, confusions, questions and stumbling blocks that are more troublesome for the novice than the experienced user. For example, some of the student comments were - "what is a period? (to be measured half swing or full)" "On line stopwatch was problematic." • Students often want to "wing it" not spending time reading text or instructions 0-7803-5643-8/99/$10.00 © 1999 IEEE November 10 - 13, 1999 San Juan, Puerto Rico 29th ASEE/IEEE Frontiers in Education Conference 13d4-9

Session 13d4 • People are often impatient with computers (e.g. when the plug-in Real Player was loading). This can lead to further difficulties if impatience leads to behaviors such as too many mouse clicks, which can freeze the system. The more experienced users were more patient with the machines, but this indicates that the human is adapting to the machine interface, rather than that there is a welldesigned interface. • Students generally preferred watching pictures/video to reading. • Students aimlessly "played" with the simulations without developing insight or coherent exploration. They are not learning what you think they are learning, just the techniques to do the task at hand, but not larger concepts. • Students don’t come to class prepared - (little things e.g. formatted disks) - teachers must end up providing this or the lesson gets subverted from the main task at hand. • At least one group did not understand and follow directions, which were clearly printed, namely to use the simulations to do the third lab component, al though none of this appeared in the survey information, it did become evident in observer's notes and in the materials that the students handed in. • Even though it is preferred, video on-line can be too long and boring - again not mentioned in survey but observer comments that "group shuts off video". Our original video ran approximately nine minutes. Two minutes is probably more optimal. • Students did not always get the big picture that was trying to be conveyed, namely the nature of mathematical models and approximations to forces in the real world. They also did not fully understand the forces at play and the effects of friction and starting angle on the motion of the pendulum. Again, many did not bother to thoroughly read the text or follow directions carefully and so missed out on the insights that they were supposed to achieve. For example, most of the student tests said that pendulums come to rest because of gravity (rather than friction). • These comments indicate a problem of engagement with the assignment and discovery process. Some of that can be attributed to the fact that this was an extra credit assignment for these student-testers. This qualitative self-assessment is purposefully critical to provide insight for improving the educational module.

Summary The lessons learned from this first module of the Internet Science Institute are both simple and profound. 1. Portability and platform issues still plague the system. Real advances in Web-based learning won't occur until applications, "plug-ins", and browsers are more

2. 3.

4.

5.

seamless in their usage and attuned to human expectation and practices. The Web-site developer can not anticipate every browser problem or lack of plugin/application. Even something like the word processor or spreadsheet program might not be available. This will clearly limit how universal a site such as the Internet Science Institute can be. One can never be too over prepared in any kind of hands-on learning situation. The Web-site developer needs to work in close proximity to the content provider, to allow crossfertilization of ideas. Less can be more - bells and whistles and moving pictures do not necessarily enhance a Web page or learning. For example, the multimedia video of a lab experiment didn't not enhance student learning compared to a "low bandwidth" version with still pictures and textual explanations. The mathematical simulations were meaningless to the students unless their actions were guided. Self-exploration without guidance usually results in limited discovery. The synergy of student-teacher or student-student interaction is great. Students learn from each other. Those who worked on the lab in teams (of two) were more efficient in completing the lab and seemed to have retained more. The flip-side is to avoid the passive-observer problem of teamwork.

More modules are being developed, particularly one in the area of robotics, and more students (non-science majors) will help in our assessment of the current module in the spring term of 1999.

References 1. Kulik, Janes and C. Kulik, "Effectiveness of Computerbased Instruction: An Updated Analysis", Computers in Human Behavior, 7(1-2) 75-04 (1991). 2. McArthur, David and Matthew Lewis, "Untangling the Web: Applications of the Internet and other Information Technologies to Higher Education" A Report for the California Education Roundtable (RAND corporation: Santa Monica, CA 1996). 3. Nott, Matthew, et al., "Enhancing Traditional University Science Teaching Using the World Wide Web", in Proceedings of the Sixth IFIP World Conference on Computers in Education, Tinsley, J. David and van Weert, Jom J. (eds.) (Chapman & Hall: London 1995) pp235-243.

Acknowledgement Thanks to Duane Shelton who spearheaded this project, Bernie Weigman who provided the basic computer data acquisition, pendulum lab and was the video star, and Randy

0-7803-5643-8/99/$10.00 © 1999 IEEE November 10 - 13, 1999 San Juan, Puerto Rico 29th ASEE/IEEE Frontiers in Education Conference 13d4-10

Session 13d4 Jones, creator of another module/element of the ISI. All

provided invaluable help as participants in the ISI project.

Figure 3: Example of student tester lab notebook - first part of pendulum module. Instructions: Here is a chart followed by a series of questions. After you have entered your data into the field titled “number of swings” and answered the questions, go to FILE and SAVE AS onto your floppy disk. You can email the file as an attachment to your professor, or print out the data. To enter your data in the chart just below, you must double click on the chart to select it. length 6 6 6 12 12 12 24 24 24 48 48 48

number of swings number of washers 38 1 39 2 39.5 3 27 1 27 2 27 3 19 1 19 2 19 3 13.5 1 13.5 2 13.5 3

60 50 40

length

30

number of washers

20 10 0 0

20

40

60

Figure 4: Example of student tester lab notebook - second part of pendulum module. Instructions: … .. tim e 0 0 .0 5 0 .1 0 .1 5 0 .2 0 .2 5 0 .3 0 .3 5 0 .4 0 .4 5 0 .5 0 .5 5 0 .6 0 .6 5 0 .7 0 .7 5 0 .8 0 .8 5 0 .9 0 .9 5 1 1 .0 5 1 .1 1 .1 5 1 .2 1 .2 5 1 .3 1 .3 5 1 .4 1 .4 5 1 .5 1 .5 5 1 .6 1 .6 5 1 .7 1 .7 5 1 .8 1 .8 5

a m plitu d e ( d e g r e e s ) 4 .6 6 -6.26 - 1 2 .8 3 - 1 9 .1 2 - 2 5 .2 2 - 3 0 .3 9 - 3 4 .9 4 - 3 8 .3 4 - 4 0 .7 8 - 4 2 .4 7 - 4 3 .0 4 - 4 2 .1 7 - 4 0 .4 5 - 3 7 .5 3 - 3 3 .8 2 - 2 9 .1 4 - 2 3 .8 8 - 1 8 .0 6 - 1 1 .9 3 -5.43 1 .2 1 7 .8 4 14 20.1 2 5 .7 3 3 0 .7 8 3 4 .8 5 3 8 .0 1 4 0 .5 8 4 1 .9 5 4 2 .2 8 4 1 .6 4 4 0 .1 7 3 7 .4 4 3 4 .0 8 29.6 2 4 .8 7 1 9 .5 1

M o tio n o f a p e n d u lu m Tim e (sec)

0

1

2

3

4

5

6

7

8

9

1 0

5 0 4 0 3 0 2 0 1 0 0 -1 0 -2 0 -3 0 -4 0 -5 0

0-7803-5643-8/99/$10.00 © 1999 IEEE November 10 - 13, 1999 San Juan, Puerto Rico 29th ASEE/IEEE Frontiers in Education Conference 13d4-11

Session 13d4 1.

Calculate an experimental estimate of period of this pendulum. Explain your procedure.

SAMPLE ANSWER FROM STUDENT: T=(approx.) 2sec, I viewed the crests of the amplitude v time graph and each was about 2sec apart. 2.

Calculate an experimental estimate of the rate of decay (degrees per second) of the amplitude of this pendulum. Explain your procedure.

SAMPLE ANSWER FROM STUDENT: rate of decay = 3.45 degrees/sec, I took the amplitude at time=0sec and subtracted the amplitude at time=1sec… 4.66-1.21

Figure 5: Example of student tester lab notebook - third part of pendulum module. Instructions: … From simulation number 1 ( http://www.lightlink.com/sergey/java/java/pend1/index.html ) you will fill out the chart below. You can experiment in this simulation with different lengths and starting angles of the pendulum. Fill in the chart for at least three different lengths and three different starting angles. (That’s nine trials!) Make sure your lengths and starting angles cover an appropriate range of values. The simulation provides the theoretical period and the experimental period. Make sure you record those results below. length 4.99 4.99 5 2.51 2.51 2.51 1 1 1

1.

starting angle period (theory) period ("experiment") error (difference of periods) 90 4.484 5.301 0.817 60 4.482 4.814 0.332 45 4.491 4.686 0.195 90 3.18 3.756 0.576 60 3.177 3.432 0.255 45 3.175 3.31 0.135 90 2.0058 2.387 0.3812 60 2.01 2.166 0.156 45 2.013 2.1 0.087

What conclusions can you draw from this simulation? SAMPLE ANSWER FROM STUDENT: The theoretical approximation of the period does not take in account the other outside forces of resistence. The period will always be off by some error because the exact period cannot be predicted. This experiment shows that the actual does, in fact, vary from the theoretical.

0-7803-5643-8/99/$10.00 © 1999 IEEE November 10 - 13, 1999 San Juan, Puerto Rico 29th ASEE/IEEE Frontiers in Education Conference 13d4-12