Session T1F TEAM LEARNING IN AN ONLINE LAB Bernardo Wagner1 and Jörg Tuttas 2 Abstract This contribution deals with the implementation and subsequent evaluation of a distance-learning experiment as part of a laboratory for process control for students of electrical engineers in the main section of their studies. In one of our control laboratory experiments students have to design, implement and test a discrete controller for a process engineering plant. Last winter term we have started to investigate students motivation, acceptance and learning results in conjunction with online laboratory access. To compare local and online lab access a summary evaluation has been performed with twenty students grouped into eight small teams. Four teams were not allowed to enter the room with the process engineering plant inside, so they had to operate and control the whole experiment through the Internet. The other four teams have worked on the laboratory experiment in the traditional way while sitting in the same room as the process to be controlled. Index Terms Evaluation, team learning, remote control laboratory, problem-based learning.
based on exercises (see Figure 1). Together with the problem statement and the actual distance-learning experiment as well as metacognitive sequences they form the pedagogical articulation scheme of a complete action [5]. Based on conventional technologies (HTML, Java etc.), it is thus possible to observe and control our distancelearning experiment from the institute as well as from anywhere in the world. In our learning scenario the laboratory devices can be observed through a web camera and can be operated by a remote controller connected to the Internet. The control program is designed, implemented and tested interactively by small teams during the lab exercises. To do that they use a graphical Petri net editor and some other software tools which were developed at our institute [6]. All tools are programmed in Java. The necessary information regarding the lab experiment (theoretical background, user manuals, etc.) is provided on our institute's web pages.
DISTANCE-LEARNING EXPERIMENT In contrast to current purely receptive uses of the Internet for obtaining information, in a distance-learning experiment the Internet becomes a constructivist medium where knowledge is build up by the student in the learning situation (learning by doing). On the condition that the experiments include design and construction exercises with a high degree of interactivity real learning spaces are formed. Conception The notion of learning behind the conception of our teaching and learning environment is based on the model of the reflexive subject as described in [1]. In line with the constructivist paradigm, students dealing with the teaching and learning environment are confronted with a problematic situation, in this case with a technical problem. The distancelearning experiment creates for the learner a greater proximity to professional reality due to its concreteness as an object which is inherent in the system, in contrast to pure software simulation. This concreteness also serves to greatly increase the learners’ motivation, since they see directly the physical effects of their actions and they are given responsibility for a materially represented process. Purely instructional sequences based on a course of lectures (here: the learning materials in our course “Design of Discrete Controllers”) alternate with example sequences 1 2
FIGURE. 1 DISTANCE-LEARNING EXPERIMENT WITH INSTRUCTIONAL AND CONSTRUCTIVIST ELEMENTS.
Laboratory Experiment The lab experiment starts with the description of a fault in the process engineering plant. The students have to work one after the other on four subtasks which have an increasing intellectual demand.
Bernardo Wagner, Learning Lab Lower Saxony, University of Hannover, 30159 Hannover, Lange Laube 32, Germany,
[email protected] Jörg Tuttas, Center for Engineering Education, University of Hannover, 30159 Hannover, Lange Laube 32, Germany,
[email protected]
0-7803-6669-7/01/$10.00 © 2001 IEEE October 10 - 13, 2001 Reno, NV 31th ASEE/IEEE Frontiers in Education Conference T1F-18
Session T1F In accordance with the nature of the task the students have to analyze a given Petri net to detect and to repair the fault (temperature exceeds a maximum). To avoid a renewed appearance of the fault the students have to expand the Petri net in a way that the fault is now reported through a web page (implementation in HTML), where an operator has to confirm the new error message. Afterwards the graphical control program has to be optimized using opening and synchronizing of parallel Petri net structures. In the last subtask it is requested to design a cyclic operation of a sequence of different heating phases.
EVALUATION After a formative evaluation a summary evaluation has followed which was intended to provide information as to the suitability of distance-learning experiments in professional education. The results will allow us to discuss questions of whether distance-learning experiments in contrast to pure simulations or other computer-based learning arrangements are more suited to the promotion of professional competence. Based on Kirkpatrick’s levels approach [2], we have the following levels of evaluation which feed into a needs analysis. TABLE I EVALUATION QUESTIONS ON DIFFERENT EVALUATION LEVELS [3]
Evaluation level
Central evaluation question
E1: product level
What result do experts arrive at on evaluating the program? How do learners react to the program? What learning successes were achieved? Could the learning content be transferred? What effect did the learning unit have? How should one judge the investment in continuing education?
system is divided into four components (learner, learning task, learning environment and reference framework) and three processes (knowledge use, knowledge acquisition and knowledge storage). Within the evaluation, suitable methods are used to examine the learning requirements with respect to the professional, method and social competencies of the learners. In addition to this, it documents their motivation and acceptance with regard to the distance-learning experiment. The evaluation of the component “learning task” is supported by its future and present significance, the exemplary nature of the topic and the question of whether the topic contributes to structuring in terms of content. Also, problematic aspects of the topic with regard to different learning groups need to be evaluated. After the distancelearning experiment has been carried out, we need to check what extent the learning objectives formulated within the methodological planning of the teaching/learning unit have been reached. It should also be related to the tests of prior knowledge carried out at the beginning. The evaluation of the learning environment relates to the entire instruction process and has an influence on the instructional design of the teaching/learning environment. By instructional design we mean the entire process of designing the learning environment. Evaluation of the learning environment thus takes place on the visual, content and structural level. Evaluation methods
E2: reaction level E3: learning level E4: action level E5: success level E6: return-oninvestment level
The evaluation approach As success in learning is dependent both on situational as well as constituted parameters, the intention is to carry out the evaluation of the distance-learning experiment using a theory-based holistic approach. UCIT (Universal Constructive Instructional Theory) [4], developed since 1991, provides a suitable, subtle and systematic theoretical framework. According to UCIT, the teaching and learning
A suitable way of evaluating a distance-learning experiment is a comparative investigation of the process which varies of the reference framework. Three scenarios are conceivable (see Figure 2 and Table II). The network-based infrastructure enables methods which are not possible in conventional teaching/learning arrangements. Thus synchronic recording of the learners and their activities in the teaching and learning environment, database-based “tracking” or “screen capturing” allow us to obtain information on the learning strategies of the learners. Because of the big amount of data in this first evaluation we have restricted the recording of the screen content, mouse movements and clicks to one group. The obtained data will serve us as a basis for the development of new analysis methods for such amounts of digital data. For evaluation of our distance-learning experiment we have developed an online questionnaire. It is divided into the categories "existing knowledge", "preparation", "lab taking" and "transfer". The online questionnaire was filled in by the students immediate after finishing the lab exercises. The type of possible answers include simple yes/no, multiple choice, grading and free text.
0-7803-6669-7/01/$10.00 © 2001 IEEE October 10 - 13, 2001 Reno, NV 31th ASEE/IEEE Frontiers in Education Conference T1F-19
Session T1F
Laboratory L
Classroom
T
L
L
L
Classroom
Scenario 1: conventional form of teaching (real group work)
Laboratory
L
Classroom
Classroom L
L
L
Laboratory T
L
Classroom
T
Scenario 2: conventional form of teaching, but with a “distance-learning experiment” L
learner
Scenario 3: distributed, virtual groups T
tutor
FIGURE. 2 Evaluation Scenarios TABLE II TEAM LEARNING SCENARIOS
Scenario One Scenario 1 is a conventional scenario of a laboratory experiment. Tutor, learners and the test object (in this case the process which is to be controlled) are in one room. Although the technological basis is identical with scenarios 2 and 3 (the process is controlled via an Internet connection), learners can see the result of their actions directly in front of them.
Scenario Two In contrast to scenario 1, the learners are now in a different room, at a distance to the actual object. They receive visual impressions by a video transmission as well as some software tools (images of process states, graphic process visualization). Tutorial supervision is direct here too.
Scenario Three The idea is altered here to the extent that the groups, the experiment and the tutorial supervision are spread out over different rooms.
the internet. Others will follow in the near future. Two or three students have formed a lab team which ones collaborated during the whole lab course. Four teams have worked remotely on the Petri net experiment (see Figure 3) via Internet. We call this teams the online teams. The other four teams worked in the traditional way with direct, local connection to the lab devices. Our small database will not deliver valid statements. Nevertheless this first evaluation brings up interesting results.
This scenario will probably be carried out in winter term 2001/2002.
THE EXPERIMENT This winter term twenty students in electrical engineering have taken part on the lab course in discrete control. The complete course consists of eight individual experiments. Themes dealt with are Petri net and state machine modeling, PLC and robot programming and distributed control using field busses. Currently, only one experiment is connected to
FIGURE. 3 REMOTE TEAM
0-7803-6669-7/01/$10.00 © 2001 IEEE October 10 - 13, 2001 Reno, NV 31th ASEE/IEEE Frontiers in Education Conference T1F-20
Session T1F Most students had prepared the lab exercises in working groups. In average they spent two hours for preparing themselves. Main focus of preparation was on understanding the problem statements and how to operate the laboratory tools and devices (75%) and to work on additional exercises given in the lab description to prepare the students (70%). Almost all students (95%) have used the delivered hypertext-based information system (delivered on CDROM). 25% used the course materials and 15% other reading books for additional reference. TABLE IV FREE-TEXT ANSWERS TO RELEVANCE (EXCERPT) Question Answers (selection)
I like to work in a remote laboratory, because...
FIGURE. 4 PETRI NET EDITOR WITH AN INTERNET RUNTIME SYSTEM
The resulting Petri net of one team is shown in Figure 4. A medium sized Petri net has been implemented interactively on screen. In the pop-up window one can see the dialog for connecting the remote engineering and runtime system with the lab server.
RESULTS The online and the local teams have slight difference in existing knowledge. 25% of all students have taken the corresponding course of lectures in "Design of Discrete Controllers", 40% of them belong to the local group. This course of lectures introduces to the theoretical foundations of Petri nets and their application in automatic control. All teams from both groups have finished successfully the lab experiment within the given four hours. No difference in the achieved results has been observed between local and online teams. TABLE III RELEVANCE AND ACCEPTANCE (EXCERPT)
1: Will remote maintenance and remote engineering have any practical value to you in the future? (1=very important ... 6=no value) 2: Does this lab experiment (Petri net design and programming for controllers) has any practical value for electrical engineers? (1=very important ... 6=no value) 3: Please assign a grading level to the whole lab experiment: (1=EXCELLENT ... 6=VERY BAD)
remote
1,7
local
2,1 2,4
remote
remote
2,3 1,7
local
2,6
local
... you don't have to go to the university. You can do all the lab work very comfortable from home. I liked the remote experiment, even though I think the team and the tutor need to be at one place. Otherwise it will be difficult to communicate and this will increase the time necessary to complete the lab experiment. ... it was fun. The remote experiment was a new and challenging experience, therefore I enjoyed it. ... I could do it from home at any time. ... I think that this new way of working will become more and more important in the near future.
The remote group has given the lab experiment a significant better grade (average 1.7 in comparison to 2.1, see Table III). Both groups graded the technology value of the Petri net experiment similarly (2.3 and 2.4). The most significant result was that the remote teams graded the whole lab experiment much better than the local group (1.7 and 2.6, see Table III, question 3). As a result of our experiment one could see that possibly remote lab experiments will have a positive impact on acceptance and motivation. Almost all students could imagine to accept further remote laboratories (95%). As one could see in Table IV student especially recognize the additional value of online labs for their future engineering work.
CONCLUSION Despite the small database our evaluation shows that distance-learning experiments can lead to the same learning results than traditional local lab experiments. We have investigated two scenarios. In both cases teams have shown
0-7803-6669-7/01/$10.00 © 2001 IEEE October 10 - 13, 2001 Reno, NV 31th ASEE/IEEE Frontiers in Education Conference T1F-21
Session T1F that they are able to transfer theoretical knowledge to concrete problem situations. Additionally the online teams have noticed that remote diagnosis and maintenance could be important for them in the near future. Because of limited perception (computer screen) the online teams have to plan and operate more carefully the lab experiments than the local teams. It is indicated that online experiments could possibly be used to teach planning and analyzing competencies in a more effective and inherent way. Almost all students do accept online lab experimenting and understand that net-based forms of learning have several advantages in comparison to presence teaching. As with most new ways of teaching and learning students motivation is increased at first. This has been measured by asking the students to grade the whole experiment. It is plainly visible the online teams have been more satisfied than the local teams.
REFERENCES [1]
Smith, K.A. and Waller, A.A.: "New Paradigms for Engineering Education"; FIE Conference, 1997, Pittsburgh, Pennsylvania
[2]
Kirkpatrick, D.L.: "Evaluating Training Programs. The Four Levels"; Berrett-Koehler; San Francisco 1994.
[3]
Latchman, H. A.: "Salzmann, Ch.; Gillet, Denis: „Information Technology Enhanced Learning in Distance and Conventional Education"; IEEE Transactions on Education, Vol. 42, No. 4; November 1999; Seite 247-254.
[4]
Schott, F.; Kemter, S.; Seidl, P.: "Theoretical aspects of instruction of multimedia learning environments"; In L. Issing & P. Klimsa (Hrsg.); “Information und Lernen mit Multimedia”; Psychologie Verlags Union; Weinheim 1997; Seite 179-192.
[5]
Tuttas, J.: „Fernexperimente in der ingenieurswissenschaftlichen Ausbildung – von rezeptiver Informationsvermittlung zum konstruktivistischen Lernen im Internet”; GI FG 1.1.5/7.0.1 Intelligente Lehr- / Lernsysteme, Proceedings; 1. – 2. Dezember; Hamburg 2000.
[6]
Wagner, B.: „From Computer-Based Teaching to Virtual Laboratories in Automatic Control"; FIE Conference, November 1999, San Juan, Puerto Rico
0-7803-6669-7/01/$10.00 © 2001 IEEE October 10 - 13, 2001 Reno, NV 31th ASEE/IEEE Frontiers in Education Conference T1F-22
