Session T4B COOPERATIVE LEARNING IN COMPUTER ARCHITECTURE: AN EDUCATIONAL PROJECT AND ITS NETWORK SUPPORT Yannis A. Dimitriadis1, Alejandra Martínez 2, Bartolomé Rubia3, and Maria J. Gallego4 Abstract This paper describes the design, setup and evaluation of an educational project in Computer Architecture studies. Its design through the educationaltelematic framework DELFOS, proposed by our research group, aims to employ Computer Supported Collaborative Learning in computer engineering studies. Its main objective is to integrate lectures, simple assignments, and laboratory work through projects and case studies in computer architecture design and evaluation. Experimental studies were carried out during two academic years with very encouraging results. At least three student groups in each session dealt with one out of 5 distinct case studies. Students assumed the roles of consulting firm and computer manufacturer, who try to give an answer to a specific customer/market sector. Cooperative learning was carried out through synchronous guided discussions within the class in key dates, asynchronous document sharing and discussions, as well as collaborative edition of the final project report. Analysis of the experimental work was performed on both ethnographic data and computer transcriptions, while current studies deal with student interactions with Social Network Analysis methods. Index Terms Computer Supported Collaborative Learning, Computer Architecture education, Debate organizer, social constructivism
INTRODUCTION An important shift in the dominant educational paradigm has been observed during the last decade. This change can be expressed as a student-based teaching/learning process, where students construct their own knowledge through active and cooperative methods. This new model, heavily influenced by the pedagogical theories of social constructivism [1], implies a radical change in curriculum design and practice in all educational levels. Although higher education studies should have been pioneers in this shift, due to their close relation to modern research studies, such a change has been produced rather slowly. Common problems that may explain this phenomenon, include overcrowded classes, low emphasis on pedagogical formation of university professors as opposed to
research work, as well as the conception that knowledge transmission is the fundamental issue in higher education. Engineering studies can be regarded as one of the most adequate candidates for the use of the new constructivist model. Design, implementation and evaluation of systems provide a very interesting field for analysis of valid alternatives, teamwork, project-based and peer learning, debates, etc. More concretely, typical electrical, computer or telecommunications engineering studies are supported by two additional factors. First of all, the IEEE/ACM Computing Curricula 1991 [2] emphasize the use of recurrent conceptsand system design/evaluation through projects and case studies. On the other hand, the most commonly required qualifications of a professional in this field, as expressed by the company managers, include efficient teamwork, autonomy and initiative, as well as abilities for presentations and technical writing. Information and Communications Technologies (ICT) have been strongly involved in supporting the educational paradigm of cooperative learning, constituting an important and growing research field, known as Computer Supported Collaborative Learning (CSCL) [3]. The advances in computer networks, distributed systems and software engineering have allowed a rapid and efficient development of platforms and applications that support the fundamental techniques of the constructivist paradigm. An essential advantage for the use of CSCL in ICT-related studies is that technology forms at the same time the means and the objective of the teaching/learning process. In this paper, we describe an educational project in Computer Architecture studies and its CSCL support. A clear methodology for curriculum design and telematic support coverage is necessary for an efficient and effective project. Our proposal is based on the educational-telematic framework DELFOS (a Description of tele-Educational Layer-Framework Oriented to learning Situations) [4], that has been developed by our research group and validated in several applications of different educational levels [5]. This experience consists in project-based and case-study techniques that allow students to go through the principal tasks of design and evaluation of computer systems (mainly at the conventional machine level). Its main objective is to test the validity for this specific field of the methods inspired
1
Yannis A. Dimitriadis, Univ. of Valladolid, School of Telecom. Engineering, Camino del Cementerio s/n, 47011 Valladolid, Spain,
[email protected] Alejandra Martínez, Univ. of Valladolid, School of Computer Engineering, Camino del Cementerio s/n, 47011 Valladolid, Spain,
[email protected] Bartolomé Rubia, Univ. of Valladolid, Faculty of Education, Calle Hernández Pacheco 1, 47014 Valladolid, Spain,
[email protected] 4 Maria J. Gallego, Univ. of Valladolid, School of Telecom. Engineering, Camino del Cementerio s/n, 47011 Valladolid, Spain,
[email protected] Partial financial support for this research project was given by the Autonomous Region of Castilla and León, Spain (VA18/99, VA117/01), the Ministry of Science and Technology, Spain (TIC2000-1054) and the European Union Leonardo Programme. 2 3
0-7803-6669-7/01/$10.00 © 2001 IEEE October 10 - 13, 2001 Reno, NV 31st ASEE/IEEE Frontiers in Education Conference T4B-13
Session T4B in the constructivist paradigm, providing at the same time some insight on the adequate design, development and evaluation of CSCL systems. The rest of this paper is organizes as follows: in Section II we provide a description of the objectives, methodology and design of the proposed educational project. We also present the specific context of Computer Architecture as well as the employed telematic tools. In the following section, we describe the experimental work developed along the last two academic years, together with the discussion of relative merits and problems derived from the evaluation process. Finally, we draw the main conclusions and show the current research work that focuses on the integration of ethnographic and quantitative data for the evaluation of group interactions.
THEORETICAL FOUNDATIONS AND DESIGN In this section, we begin with a brief description of the theoretical background and the objectives to be achieved. Then, we present the specifics of the educational project, its organization, and relate it to the context of Computer Architecture studies in a Spanish university. Finally, we will describe the telematic tools (existing or newly developed) that were found necessary for an efficient technological support of the project. The whole description will be based on the methodology proposed in the framework DELFOS. Background and Objectives As already stated in the introductory section, the work presented here is based on the principles of social constructivism for the teaching/learning process. Therefore, individual learning should be complemented by cooperative techniques, in which students enhance their previous knowledge taking advantage of zones of proximal development. Individual and group work with peers, as well as the support of the mediator (professor) may then allow student to construct their own knowledge in a social context. This model, called the constructivist spiral (see Figure 1), forms the core of the framework DELFOS and guides the design of pedagogical objectives and telematic tools. In our project, the main objective consists in bringing together the three existing types of engineering courses, i.e. lectures, simple assignments and laboratory work. In their current format, lectures promote student passivity and the premise that theory is a rigid construction isolated from practice. On the other hand, laboratory work may restrict meaningfulness and autonomy for the students, when it is (they are) based on closed-form guides. Finally, simple assignments typically follow the lecture format and give a non-realistic idea of the engineering practice. Following the principles of the educational model as well as the directives of the IEEE/ACM Computing Curricula, the main objective was to provide contextualized, integrated and meaningful knowledge.
FIGURE. 1 THE CONSTRUCTIVIST SPIRAL AS SUPPORTING EDUCATIONAL MODEL OF THE FRAMEWORK DELFOS.
The modality of project-based learning applied to certain case studies was found to be especially appropriate for the fulfilment of the above objective. It was necessary to integrate the whole process in a single project, in which lectures provide the introductory basis, exercises take into account realistic conditions, and lab work serves for seeking evidence in the form of information or experimental results. All these modalities can contribute to the achievement of the final project goal. Then, the project work should be structured in a sequence of educational situations, that in turn should be decomposed in specific activities. Such a structure, proposed in DELFOS, allows a clear identification of objectives and means for every activity and situation. On the other hand, each project is based on a specific case study, with a limited scope and real-world data. However, one single case study for all students provides limited knowledge and does not show that different premises/restrictions lead to different solutions. Therefore, a well-thought selection of several case studies may broaden the learning horizon, produce conflicts and give rise to debates among various student teams. The last main objective of our project deals with the use of ICT technologies and and in particular those related with CSCL. According to our model, technology should be a supporting element for the constructivist spiral, instead of being an objective on its own. Such an integration of telematic and educational aspects was the main goal to be achieved within DELFOS. Following DELFOS methodology, the educational objectives are collected in templates that lead to an iterative process of prototype development, using participatory analysis and design techniques. Thus, we either use existing tools or develop new ones, according to the needs discovered in this process. This way, we are able to incorporate telematic tools that are robust, efficient and adequate for the learning situations and activities. In particular, we had to provide technology that
0-7803-6669-7/01/$10.00 © 2001 IEEE October 10 - 13, 2001 Reno, NV 31st ASEE/IEEE Frontiers in Education Conference T4B-14
Session T4B promotes active, intentional, articulative, collaborative and
conversational learning.
FIGURE. 2 THE BSCW SHARED WORKSPACE FOR DOCUMENT SHARING AND THREADED DISCUSSIONS.
The Computer Architecture Educational Project Taking into account the general conditions described in the previous subsection, we are now able to present the specific project. Computer Architecture is an appropriate field for the implementation of the ideas expressed above. Several research studies have already been reported in the literature, confirming such a statement [6]. Design and evaluation of computer systems is a task in which different opinions/solutions can be valid, teamwork is necessary for such a complex issue, and technicalcommercial information as well as simulators are available in public domain. All these elements are especially suitable to a quantitative analysis, as the one proposed in the very popular textbook by Hennessy and Patterson [7]. Students must take roles within a project whose goal is to design and evaluate computer systems for different market sectors. The following roles are then present in the project: • Computer manufacturer (producer). • Customers of computer products (consumers).
•
Consulting firms, that study market and technology aspects, in order to assist customers, manufacturers, or simply increase their know-how.
Initially, the customer or the producer CEO (Chief Executive Officer) formulates the functional and nonfunctional requirements, that correspond to a market sector. Then, the consulting firm produces the equivalent technical requirements, together with a workload description. Later, the director of the engineering Department can choose existing benchmarks or design new ones, analyze the cost of various solutions with respect to subsystems and complete computer systems, and evaluate the global solution. Finally, producer CEO and/or customer make up their decision, establishing appropriate marketing and financial strategies. Case studies, used in the experimental work, correspond to different market sectors that require different solutions, such as weather prediction institutes (supercomputing), airline ticket reservation companies (on-line transaction processing), Internet Service Providers (multiprocessing high-end servers), etc. The project is divided in 3 subprojects (situations, using DELFOS terminology), where different subsystems (CPU, memory hierarchy, etc.), or techniques (analytic 0-7803-6669-7/01/$10.00 © 2001 IEEE October 10 - 13, 2001 Reno, NV 31st ASEE/IEEE Frontiers in Education Conference T4B-15
Session T4B models, real machine benchmarking or simulations) are studied. Each subproject presents two milestones: in the intermediate one basic decisions are made, collected through questionnaires and used in a synchronous debate, while in the final milestone of each subproject, a formal technical report has to be produced and delivered to the CEO/director/customer. By the end of the whole project, a technical report is collaboratively produced among all groups that deal with the same case study. With respect to the roles, professor acts as customer/producer CEO/Engineering Dept director, while students assume the work of engineers in the consulting form and manufacturing company. From the brief description of the course organization, we can see how the initial design objectives were reflected in the project structure. In this sense, the process of knowledge construction fulfils the properties of complexity/context (project method and case studies), intentionality (problem formulation and information search with the simple mediation of the professor), reflection and articulation (technical report writing, questionnaires and reviews), as well as conversation and collaboration (role-playing, debates, collaborative writing of final report) [8].
Telematic tools The strategy within our transdisciplinary group EMIC (Education, Media, Informatics and Culture) is to employ only the necessary technology and try to reuse existing tools, developed by us or by third party teams. In any case, using the methodology and modelling proposed in DELFOS, we are able to detect the needs and then reuse any software components. The main tools employed in the educational project are: • BSCW (Basic Support for Cooperative Work): A robust software package [9], licence-free for educational uses, managed by the German Institute GMD and developed through several European Union projects. Its role consists in serving for asynchronous document sharing, threaded discussions and as a workspace for the rest of the tools. A snapshot of its use in our project is shown in Figure 2, where debates and shared documents corresponding to each case study can be observed. We should point out the important group awareness mechanism provided by BSCW, in the form of an activity report that is generated and sent by e-mail every night.
FIGURE. 3 A SNAPSHOT OF THE TABLE PRODUCED BY THE SYNCHRONOUS DEBATE ORGANIZER.
•
Synchronous Debate Organizer: One of the basic needs detected during the first experimental phase was to contribute in a fluid preparation of synchronous debates, that would take place during the project milestones (reviews). A design module allows the professor to design the questionnaire with both closed and open-form questions, that is automatically generated by the generator module. Using the presentation module, students can answer the questions, and finally the organizer module
presents a table with the different opinions. The final table is used during the synchronous debates, taking into account that alarm signals denoting inconsistencies, discrepancies, etc. give rise to specific debates. Such alarm situations are designed by the professor during the initial phase, using an artificial intelligence component of the Design module. Two versions have been made available; one heavy-load version based on Java technologies, and another one based on simple mail and scripting
0-7803-6669-7/01/$10.00 © 2001 IEEE October 10 - 13, 2001 Reno, NV 31st ASEE/IEEE Frontiers in Education Conference T4B-16
Session T4B
•
• •
technologies. A snapshot of the tables used in the synchronous debates can be seen in Figure 3. SunForum: This official Sun product was used as an application sharing tool for and collaborative problem solving. The latest version of 1999 was found to be rather unstable for effective classroom use. Electronic mail: Extensively used through the BSCW shared workspace, in which senders and recipients assume the roles of the project. No tools were used for collaborative edition, although tools produced by the EMIC group, such as PENCACOLAS [10] (synchronous) and CECI (asynchronous) are planned to be reused in the following experimental phase. At this stage we found more convenient not overwhelm students with tools, placing their attention in the project content.
DISCUSSION OF THE EXPERIMENTAL WORK In this section, we explain the specific experimental setup and discuss some of the main findings. This work took place in the 4th (out of 5) year of the Telecommunications Engineering School, University of Valladolid, Spain. The complete class of 100-120 students is divided in 3 sessions of 40 students (maximum), in which the elementary unit consists of groups of 2 students. Given that they are faced with 5 different case studies, 3-4 student groups assume the same case study within each session. The 13 week-long semester corresponds to 3 subprojects of 4 weeks each, where the reviews (synchronous debates) take place every 2 weeks. Elaboration of the final report started in the 6th week, in which all groups corresponding to the same case study in the same session have regular meetings to discuss the respective solutions and the different versions of the final report (schemes, drafts, etc.) among them and with the professor. The computing environment consists of 20 Sun/Solaris workstations connected through a Fast Ethernet network, in which students have access from any point in campus or residencies. In fact, the work done can be described as semi-presential. The basic tools of Computer Architecture correspond to simulators and compilers associated to the Hennessy and Patterson textbook, as well as benchmarks (synthetic and SPEC). Additional literature is also available in the laboratory in paper and electronic format (most of the latter is collaboratively generated in the BSCW shared workspace). The experimental work took place in two phases during the fall semester (September to February) of the academic years 1999-2000 and 2000-2001. During the first year, the initial design and setup was evaluated,
while in the second year the revised project was extensively and systematically evaluated. In fact, evaluation is a critical point for the analysis of the effectiveness/efficiency of the educational projects, besides being a tool for the assessment of the teaching/learning process. Then, monitorization and evaluation are essential in order that professor provide feedback, support student work and serve as a means that the students reflect about the whole procedure of knowledge construction (metacognitive level). In our case, the following methods were used: • Computer logs, through the databases of BSCW, mail, SunForum and Debate Organizer tools. • Technical reports and regular student questionnaires evaluating the educational project, that were deposited in BSCW shared workspace. • Systematic ethnographic observations collected in the diary of an external observer, as well as voice transcriptions of the activities of a selected set of student groups. • Grades and observations made by the professor. All these data sources are heterogeneous in form and nature, since they are both qualitative and quantitative, ethnographic and objective. This information richness is very useful for the whole process, but a method has to be established in order to combine them and draw conclusions, especially on the effects of student interaction and collaboration. Currently, a modelling process of the interactions is being generated, that presents data in XML format and feeds the NUD*IST [11] and SNA (Social Network Analysis) [12] tools for further analysis. Although formal processing methods are currently in progress, we provide several conclusions drawn from the evaluation process: • The net outcome is clearly positive. Among students, 64% vs. 21% sustain that they learn more and better, as compared to the traditional methods. The same fact is reflected in the grades provided by the professor. • Such an enhancement of learning required a considerably increased effort for students and professor. This aspect has already been observed by our team in similar situations of student-based learning. Our recommendation consists in balancing additional student effort with reduced core contents. • The periodic in-class reviews and debates, supported by the Debate Organizer, was the method found to be most effective. More than 60% of the students mentioned that this process allowed them to rethink and formulate partial decisions, situate them in the context of all case studies and guide the rest of the work. • The asynchronous BSCW shared workspace was mainly exploited for URL and document sharing after
0-7803-6669-7/01/$10.00 © 2001 IEEE October 10 - 13, 2001 Reno, NV 31st ASEE/IEEE Frontiers in Education Conference T4B-17
Session T4B
•
•
•
student inquiries. On the other hand, information overloading and poorly organized structure of the emerged workspace were criticized. Attitude on collaboration, teamwork and student contribution in class were significantly enhanced. Among the factors that mostly influenced this fact, we include collaborative writing of final report, use of lab as a unique open space for class activities, the awareness mechanism provided by BSCW, the Debate Organizer, as well as the fact that student groups of the same case study were consistently situated in the same class area. Students’ maturity and content assimilation were demonstrated, among others, by their capacity to extrapolate to a new case study that was presented to them in the final written exam. The process of designing the educational project, development and use of telematic tools, and evaluation within the DELFOS framework were found to be satisfactory. However, we detected a need for simplification of the process together with the development of reusable software components and design tools.
ACKNOWLEDGMENT The authors would like to acknowledge the contributions of Dr. César Osuna (DELFOS framework), Dr. Eduardo Gómez (second version of Debate Organizer), Raquel Díaz (setup of the initial experimental environment), Inma Garrachón and José Antonio Marcos (modelling of learning interactions), the rest of the transdisciplinary group EMIC, and last but not least all the students that eagerly participated in this academic program.
REFERENCES [1]
Vygotski L., Mind in Society, Cambridge, MA, USA: Harvard University Press, 1978.
[2]
Turner A., J., "A summary of the ACM/IEEE-CS Joint Curriculum Task Force report: Computing Curricula 1991", Communications of the ACM, Vol 34, No 6., June 1991, pp. 69-84.
[3]
Koschmann T. (editor), CSCL: Theory and Practice of an Emerging Paradigm, Mahwah, JJ: USA: Lawrence Erlbaum Associates, 1996.
[4]
Osuna C., and Dimitriadis Y., A.., "A Framework for the Development of Educational-Collaborative Applications Based on Social Constructivism", Proceedings of CYTED RITOS International Workshop on Groupware, Cancún, México, September 1999, IEEE Computer Society Press.
[5]
Osuna C., and Dimitriadis Y., A., "Using a Theoretical Framework for the Evaluation of Sequentiability, Reusability and Complexity of Development in CSCL Applications", Proceedings of the European Computer Supported Collaborative Learning Conference, Maastricht, the Netherlands, March 2001.
[6]
Gehringer E., (editor), "Special issue on Computer Architecture education", IEEE Computer Society Technical Committee on Computer Architecture Newsletter, September 2000.
[7]
Hennessy, J., .L., and Patterson D., A., Computer Architecture: A Quantitative Approach, San Francisco, CA, USA: Morgan Kaufmann, 2nd edition, 1995.
[8]
Jonassen D., H., Pek K., and Wilson B., F., Learning with Technology. A Constructivist Approach, Upper Saddle River, NJ, USA: Prentice Hall, 1999
[9]
GMD-FIT, Basic Support http://bscw.gmd.de, 2001.
CONCLUSIONS AND FUTURE WORK In this paper, we presented the design, development and evaluation of an educational project in Computer Architecture. This project follows the constructivist spiral model for the teaching/learning process, as well as the methodology proposed within the DELFOS educationaltelematic framework. The evaluation process showed that students learn better and more, with the cost of increased effort. The project-based methodology that uses different case studies allowed a contextualized and meaningful work for the students. At the same time, the course organization and the telematic tools that support it enhanced maturity and capacity for extrapolation. Among all telematic tools, the Debate Organizer (developed by our group) and the BSCW shared workspace were found to be the most effective for this project. In this sense, we plan to detect the software components that are necessary for building a consistent CSCL environment within DELFOS framework. Finally, we would like to insist on the need for a transdisciplinary group, composed by education specialists and engineers, for the success of any CSCL project. At the same time, this type of projects should be integrated in a global plan within the same degree. Current work consists in the development of a formal analysis method for the evaluation of the group interactions, using ethnographic and quantitative data. Then, all available data from the last experimental phase could be fully exploited, in order to draw more solid conclusions.
for
Cooperative
Work
3.3.1,
[10] Blasco M.T., Barrio J.L., Dimitriadis Y.A., et al. "From cooperative learning towards the virtual class. An experience in composition techniques", The Ultibase Journal http://ultibase.rmit.edu.au/Articles/blasco1, December 1999. [11] QSR, NUD*IST. Software for qualitative data analysis, Thousand Oaks, CA, USA:Scolari, 1997. [12] Wasserman S. and Fauster K., Social Network Analysis: Methods and Applications, Cambridge, UK: Cambridge University Press, 1994.
0-7803-6669-7/01/$10.00 © 2001 IEEE October 10 - 13, 2001 Reno, NV 31st ASEE/IEEE Frontiers in Education Conference T4B-18
