Session T1B KNOWLEDGE BY USER DEMAND AND SELF-REFLECTION: NEW MODELS FOR TEACHING AND ASSESSMENT IN EDUTAINMENT SOFTWARE DESIGN Torben Svane1, Christina Aderklou2, Lotta Fritzdorf3, and Jo Hamilton-Jones4 Abstract This paper reports on experiences from applying Knowledge by User Demand (KBUD) to teaching at the Edutainment Software Design programme at Halmstad University, Sweden. 'Edutainment' denotes educational and recreational systems for homes, schools and work. Upon graduation, students may find careers in game design, project management, systems development etc. KBUD emphasizes participation in selection and creation of course contents, to foster a continuous review process of how to attain competencies needed to accomplish a task. Continuous self-reflection, on how and why competencies develop, and how personal learning can be refined, is highlighted in discussions and through using projects from previous courses as case studies. KBUD classes offer a core of approximately 50% of course content. Students select another 25% from a range of teacher-prepared themes and develop themselves the remaining themes under teacher supervision. KBUD modules combine subject adherence with encouraging student interest and participation, whilst also ensuring content vitality. Index Terms Competence, Course Contents, Metalearning, Self-reflection, Students.
INTRODUCTION With many academic subjects expanding their knowledge base at exponential pace - and with sectors such as IT and Information Systems being included - it is a constant challenge to find methods that will provide a sustained learning arena. 'Sustained' in this context indicates a scenario where students will acquire skills and competencies useful for the duration of their professional life to come, rather than solely for obtaining satisfactory assessment grades. This paper presents a new pedagogic model entitled Knowledge by User Demand (KBUD) and reports on experiences from its application to teaching at the Edutainment Software Design programme at Halmstad University, Sweden. Lacking wellestablished definitions, 'edutainment' is here seen as computer-based systems for learning and leisure, in homes, schools, and at work. Applications may include 'any-level' educational software, e-learning packages, recreational systems, instructional technology toolkits, computer games etc. The rationale behind the proposed model draws its ideas from the notion that learning is a life-long process. Although this notion is not innovative in itself, a focus on the process
as such may however be more accentuated today. One reason for this could be that the need to update knowledge and skills is more frequent and revolutionary, and thus more difficult to maintain. There is a need to cover not only content, but also methods for acquiring competence. To be frank and perhaps also a little harsh: what we teach our students during their undergraduate years may well turn out to be almost 'worthless' as up-to-date, working knowledge when they graduate! Many parameters in society and technology change quickly. The influences of the Internet, peoples' new work patterns, computer use in homes, schools and at work, or the rapid evolution of hardware and communication technologies are only to mention a few. For the edutainment software industry, emerging media patterns of 'less TV, more IT' among young people [1] are also part of an important shift in society, typical of its times and well worth attention. With this scenario in mind, the teaching team decided to try a new approach to accentuate students' awareness of their role in the learning process. Our idea was to integrate into every course, lecture and seminar a situation whereby the students themselves would have to handle greater responsibility for keeping up with change. The aim was to develop an understanding, which would encourage students to continously evaluate their own competence profile within various contexts. Understanding not only how to be assessed but also how to self-assess and to do so repeatedly would force students to reflect upon what new knowledge, skills and insights that were needed to solve a given problem - and hopefully, also how to attain the competences in question. Such awareness, we argued, would probably be far more important in a long-term perspective than merely knowing menial facts about systems development (soon to be replaced by newer models), hardware principles (often outdated before books even came from print) or Web programming languages (which seem to undergo an evolutionary phase every two years or so, anyway). The work with developing the KBUD model is still in progress. The teaching team has engaged in several activities that relate to the basic notion of supporting and triggering self-reflection and -assessment. During the academic year 2000-2001, we have also had a great deal of student input with regard to the design and possible activities that can support KBUD. Finding out about the pro's and con's is a learning process in itself. The results so far, and most students' enthusiasm, have motivated us to pursue our work further.
1
Torben Svane, Halmstad University, School of Info Science, Comp & Electr Engineering, S-301 18 Halmstad, Sweden,
[email protected] Christina Aderklou, Halmstad University, School of Society and Health Science, S-301 18 Halmstad, Sweden,
[email protected] 3 Lotta Fritzdorf, Halmstad University, School of Society and Health Science, S-301 18 Halmstad, Sweden,
[email protected] 4 Jo Hamilton-Jones, Halmstad University, School of Information Sci, Comp & Electr Engineering, S-301 18 Halmstad, Sweden,
[email protected] 2
0-7803-6669-7/01/$10.00 © 2001 IEEE October 10 - 13, 2001 Reno, NV 31st ASEE/IEEE Frontiers in Education Conference T1B-1
Session T1B THEORETICAL FRAMEWORK Fostering a reflective and emancipatory approach to learning is important both for students and teachers [2]. For the latter group, starting points in self, students, colleagues and theory [3] can be a working paradigm. Teachers will often have substantial experience of being in a teaching environment, and will in most cases add less new knowledge per week to their existing knowledge base than will students. Moreover, when it comes to students, experience profiles can be quite diverse and hence difficult to generalize. For a majority, the bulk of course content will be new knowledge. Digesting all they are exposed to will in most cases keep students busy many times, so busy that they forget to reflect upon newlygained knowledge per se and its value, context, relevance, and pathway to the individual. Alas, teaching may actually keep attention away from the learning process itself! Liberating one's self from 'only' focusing the specific paragraphs in the modular outline takes some training and courage. Not all students manage to climb the ladders of expertise [4] or those of the learning process [5] - [6]…but they must still be respected for individuality and ambition. Keeping students active is however also a good ting - of course. An interactive lecture will bring more to the students, because such an approach will make them think - and give them the time needed for thinking [7]. Activating thought on subject - and supporting a reflection on the individual's own learning [8] - will enhance the overall learning capability. Much of this activating and focusing has however emphasized
Teacher(s) Student(s)
Brainstorming session
To give students a first experience in awareness - of KBUD, and of our mission to have them interact and influence, we held a full-day workshop on their second day of classes. We named the event Competent Edutainment Developers in 2003. The conference included a number of activities and started with creativity exercises and emancipation from clichés about university education. By the end of the day, the group worked with conscious and reflective considerations on what (academic and professional) concepts, themes, activities etc. they would like to see covered during their three-year education. A schematic model of a day is presented in Figure 1.
Theme C Conc ept I Conc ept II Conc ept III --Added keywords from group B + A
D Theme D Conc ept I Conc ept II Conc ept III --Added keywords from most groups
Theme A Conc ept I Conc ept II Conc ept III --Theme B --Theme C etc.
A
A
Thematic keyword clustering
C
Theme B Conc ept I Conc ept II Conc ept III --Added keywords from group A
STARTING UP KBUD AWARENESS
Theme A Keyword I Keyword II Keyword III --Theme B --Theme C etc.
A
B
course contents, rather than on the process itself - the need to address this issue is not a new observation. What is sometimes called meta-reflection or thought about thought is indeed also a relevant, but broader concept than the KBUD approach - our main concern is not only reflection. Rather, we strive to bring about a process that will engage each student in continuous review of the currently percieved problem situation and to take responsibility for their own learning. A desirable scenario would find students engaged in individual review of knowledge, skills and insights, a reasoning about competence misfit, an investigation of available resources (including action to create your own resources, if none are found sufficient) and finally, a proactive approach to obliterate the percieved misfit.
Group discussion on theme
Theme A Conc ept I Conc ept II Conc ept III --Added keywords from all groups
A
A
1) Revision, summary and suggestions 2) Presentation of theme + action plan
Voting procedure and consensus on priorities in action plan
”Circus” workshops (group A ending at their own theme station)
Generated theme concepts
Activity 1 Activity 2 Activity 3 Lecture 1 Lecture 2 Concept 1 Concept 2 Concept 3
IIIII III IIIII IIIII IIII I IIII II IIIII IIIII I
FIGURE. 1 Schematic model of a one-day, student-centered competence awareness conference. 0-7803-6669-7/01/$10.00 © 2001 IEEE October 10 - 13, 2001 Reno, NV 31st ASEE/IEEE Frontiers in Education Conference T1B-2
Session T1B After a short introduction, the workshop started with a brainstorming session on trends in society and edutainment, and competences necessary to be successful on that arena. This activity lasted 45 minutes and generated nearly 100 concepts, covering a broad range of perspectives on the proposed theme. Students were then given a longer (30 minute) break, during which the teacher team grouped the generated keywords into thematic clusters. At this specific occasion, clusters dealt with education, hardware and technology, man and society, the entertainment industry, the professional role, and a few more. Following this came group discussions each covering one theme, with staff members as facilitators, in case a group needed input to start their discussion. The outcome of each session was a list of concepts that the specific group deemed relevant, important and interesting to cover in a more focused discussion, which would take place in the afternoon. During lunch, participating teachers used the time to inform colleagues about the discussion that had taken place in their group. By doing so, we felt better prepared for the coming sessions. In the afternoon, the student groups visited the different thematic stations, starting with the one following their own. Group A started at station B, group B took on theme C etc. and at each station, a second discussion on the specific theme was held, again with a staff member as facilitator. New keywords were added to the list and upon return to their 'home' station (being the last), each group faced an extended set of perspectives and viewpoint on 'their' theme. After this, the groups had another extended break during which they should revise and sum up the emerged (and broadened) view. Each group should also suggest an action plan for activities, lectures, workshops, field trips etc. that would make possible to achieve the thematic competence goals the group had found most important. Each list was limited to ten suggestions with seven groups came many suggestions, anyway. After the presentations, all suggestions were put on display and the students were given three stickers each, to use as votes for the activities which they (individually) felt were the most interesting. The three votes could all be used for one activity, distributed among different suggestions - or retained and not used at all, if none of the alternatives were interesting enough, in the student's private opinion. The voting results were reviewed and a priorities list was set up for the ESD class of 2000, as the group saw the future when beginning their education. A similar activity will be repeated at the end of their first and second year, to enable a revision of the list. Some of the activities are presented in the results section, and many have already been incorporated into the modular contents, the topic of the next section.
THE KBUD COURSE CONTENT MODEL During our implementation of KBUD we have learned that giving less is giving more (!) - it has been through not filling the modules that we have seen self-going, competenceaware students develop and blossom. This section will give a description on how a general model for KBUD could be im-
plemented - and it can be so in many different subjects. We also point to some of the benefits of adopting this model. The Edutainment Software Design programme is one of the newest undergraduate programs at Halmstad University. The first cohort of students started their education in the fall of 2000. The program addresses a special but growing niche. Traditional systems development do's and don'ts may not readily apply to edutainment systems. Whereas we in many other design classes tell students to ensure conformity with existing standards - to save the users from surprises - games and edutainment applications should surprise and challenge their users, in some way. As it is, the most successful games are indeed best-selling because of their non-conformity...they represent something new, rather than something familiar. With this radically different paradigm in mind, all courses for this programme are new developments. A lot of traditional subject content is of course included, but it has often been set in a new perspective. With much course development ahead, the teaching team decided to try a new approach, as the potential benefits of new thoughts in new courses were so obvious. Our model has been adopted in seven courses so far, in informatics (programming, systems development, and software engineering classes), in pedagogy (for courses in development psychology and creative environments), and in media-related courses (authoring tools and media analysis). When developing a new course module, we have adopted the procedure presented in Figure 2. After a discussion on aims and goals for the proposed module, how it will fit with the overall purpose of the programme, the potential for multidisciplinary integration etc., the teaching team will devise a content skeleton for the entire course (activities, which would fill 100% of available contact time). In a second phase, a selective process is initiated, in which a critical review filters out the absolute core of the course - what must be included, when delivering a course with the proposed title. After filtering, subject areas that are deemed close to core, important or upcoming trends, or plain and simply interesting are packaged into themes. Usually, a theme will cover a lecture, a workshop and a seminar. Themes may be offered as possible units in the module when the course is presented to the students, but not all packages need to be revealed from start. The presentation will however include more packages than what can be covered within the given course. When the students attend their first session, the module is presented, with core themes/contents and the aims, goals, etc. as they stand at start. In addition, some of the teacherproduced packages that the team think could be interesting and relevant supplements to the core are described. As not all packages can be chosen, a student-centered discussion on the competence profile of the specific group, on their personal interests, desired learning outcomes, etc. will take place. This discussion helps the specific course to recognize students as individuals rather than as just another group, one of many to take the class that year. The discussion will also bring up (student) suggestions on themes, not covered in the core or by proposed supplements - packages to be developed.
0-7803-6669-7/01/$10.00 © 2001 IEEE October 10 - 13, 2001 Reno, NV 31st ASEE/IEEE Frontiers in Education Conference T1B-3
Session T1B
Thematic Student package package ”A” 0.5 cr ”1” 0. 5 cr
Thematic packages and activities developed by teacher/experts team
Thematic package ”B” 0.5 cr Thematic package ”C” 0.5 cr
Core course content, not more than 2.5 of (total) 5 cr Thematic package ”D” 0.5 cr
Student package ”2” 0. 5 cr
Student-produced, novel, course-specific thematic packages and activities
Student package ”3” 0. 5 cr
Student package ”4” 0. 5 cr
FIGURE. 2 A KBUD-packaged course of 5 credit points It is then up to the students, in cooperation with staff mentors, to create the new additional packages. Staff participation is important to ensure coherence with subject and a relevant level of difficulty for the course as such. The process will start with students 'selling' their concept to the class, and a decision on which packages to develop further. Resources to cover costs such as books, travel expenses etc. will be made available. In the first (introductory) class, all groups had to develop a package and deliver it at a seminar as a 20 minute presentation, followed by a 20 minute activity, in which all other attending students should be actived and engaged in some way. This 'total' approach was used to get all students acquainted and familiar with the methodology. Suggestions for themes and activities included Your body - the new interface, Role-Playing Games on the Internet, and a short theatric dramatization of how to play Quake, to name a few.
When classes are finished for the term, the teaching team (and student representatives) will revise the outcomes and go through what core and package suggestions that should be presented the next time the class is given. Here, the KBUD approach has one of its strengths. The revision will automatically update the course, and will also ensure vitality and relevance with current topics because of the student input. It will never be the same course twice. Certain themes will feel outdated after some time, and will then be moved out of the core, or taken off the course entirely. Other, student-created packages may be so well prepared and professional in content and form that they can be adopted into the teacher-side package offerings (perhaps after some modification, but most of the work is already done). When the new course starts, there is again room for new, student-triggered package creation. The process is described in Figure 3.
B
A
B
Thematic Student package package ”A” 0.5 cr ”1” 0. 5 cr
Teacher package, now seen as part of the core
Student package ”2” 0. 5 cr
Thematic A package ”B” 0.5 cr
C
Top-quality student package, turned into a ”teacher-side” thematic offering for next course/year’s students
C Previous core concept, still being relevant (and offered as a package) or considered so outdated that it is taken off the course entirely
Thematic package ”C” 0.5 cr
Student package ”3” 0. 5 cr Thematic package ”D” 0.5 cr
Student package ”4” 0. 5 cr
To come: next course/ year’s student packages
FIGURE. 3 Course revision using KBUD 0-7803-6669-7/01/$10.00 © 2001 IEEE October 10 - 13, 2001 Reno, NV 31st ASEE/IEEE Frontiers in Education Conference T1B-4
Session T1B AN EXAMPLE: MULTIMEDIA DEVELOPMENT I The class is the second of totally twelve in Informatics. Students have finished the introduction, and are half-way through their first classes in pedagogics and media. MD-I is their first programming class, and holds a major project. Assignments are handed out, as is group affiliation. Students will not choose their own projects or team-mates until the last project of the year. In real life, one has little influence on selection of projects and colleagues. It is important to learn to function well with many personalities, and to quickly get a project up and running, regardless of how fascinating (or not) you think it is. In an eight-week first level class, students have 36 contact hours, including lectures, workshops and tutoring, so there is little room for dissipation. The greater part of getting familiar with the developing environment and the programming language is therefore the students' own responsibility. The core for MD-I was decided to include an introduction to programming methods, the specific syntax and structure of the language (Director Lingo), an overview of the most common commands and their use, and an introduction to interface design. Systems design was not covered purposely; the class after MD-I would deal with such isses, and projects from this class would be used as input/case studies there. Initial suggestions and final packages are presented in Figure 4. Cover briefly Do not cover as part of the OOP, in Games databases in course int roDirector created in this course, duction ”speak ” Director save for DataUsing the next base conProgram- sprites for nections ming methods, animation in Director (workshop) Using sounds, images and video Cover only graphics, as students will cover sound
Lingo syntax, structure and commands, i/f design
More on behaviors and shortcuts
Product packaging (setup, etc).
Extracting and controlling sound files
FIGURE. 4 The KBUD Workpackage for MD-I, winter 2000 Being their first programming class, students decided to exclude database interaction, to spend more time on basics. Moreover, it was decided to decrease time spent on sounds, as this was partially covered in media lectures, and because students offered to hold a workshop on the subject. A suggestion for a seminar to look at games created in Director and hence learn more about features was turned into part of the introduction.
SOME RESULTS OF KBUD ACTIVITIES Firstly we must say that initially, many students felt frustrated with the approach. Few were used to reflecting on competence
profiles. Many hesitated when they were not only asked but also urged to participate, to form an opinion, and influence their own learning. It took an entire semester to get to a point where the majority felt comfortable with handling their competence development. Once accustomed however, student response to activities initiated (where many have been conducted outside ordinary class hours) has been positive. Activities of general interest have included seminars on ethics, a guest lecture on entrepreneurship, a series of guest lectures by pedagogy professors (primarily aimed at teacher students, but also open for ESD students), a full-day conference on electronic learning, and a field trip to the Swedish Army Ground Battle Facilities, to get a more thorough understanding for simulator software (battle tank simulators) and for the architecture and rationale behind war game software/simulations. Other activities, more related to specific classes have included a three-day clinic on web programming so that, in a later-coming class, the group could start at a higher level of difficulty. Other activities have included workshops on using different toolkits and applications for computer graphics, 3D design, etc. Participation in activities has averaged 80-85% of the group, counting 36 students in total. Students have also formed workgroups for marketing communications, to better promote their programme. These workgroups are entirely selfgoverned by the students but resources as a computer to host their web site, costs covering printing, travel expenses for a the 'road show' to recruit new students etc. are covered within the programme budget. For many activities, students will however also act themselves to engage national/regional sponsors. KBUD strives to incorporate self-reflection into both the design, realization and evaluation elements, and into student behavior. An example of the latter is the first semester diary assignment, in which students must capture and reflect upon their own learning process. In the review of the diaries it was found learning was seen as an everyday process with little reflection, except in the context of formal education. This new insight encouraged students to suggest a different approach for the next module (dealing with creativity) rather than that planned. Instead of studying creative environments and direct the main attention on the environment, students suggested a focus on personal and group creativity, and on approaches to improve one's own creativity (and so the course was revised). On a final note regarding activities, many projects receive points in all major disciplines, to emphasize the need for a holistic view on competence. Also, projects done in one class are used as input in following classes where they present excellent cases for improvement. Reviewing their own work in a new perspective will also let students reflect on knowledge gained.
DISCUSSION The bulk of this paper has presented an approach for designing courses and content that will urge students to self-assess. Assessment promises to be quite a contentious issue. As we encourage our students to become reflective practitioners, we must also take reflection into account in our assessment.
0-7803-6669-7/01/$10.00 © 2001 IEEE October 10 - 13, 2001 Reno, NV 31st ASEE/IEEE Frontiers in Education Conference T1B-5
Session T1B At first, let us consider structures already available. Styles of learning provide a framework of skills, which can be used for aligning assessment tasks against what we wish the students to learn. One of the most common taxonomies [10] provides a quick and easy method to check the cognitive skill level that is required by an assessment task. Reference [11] provides a more detailed classification of those skills expected of graduates. Indeed, there are many more that could be used. However, what is important in our particular situation is to reflect on skills and capabilities pertinent for our particular students to develop - it is a necessary first step in developing an efficient assessment system [12]. Some practices will be appropriate although we anticipate a need to constantly review, redefine and probably also adjust our position. Another element to consider is the effect that assessment will have on students. The frameworks in [13] exhorts us to be cautious. We must encourage students to be reflective not only on the course but also in their assessment. All too often, the reward of learning is seen to be the result of the assessment, whereas the true reward of the learning is in the allround growth of the individual. As we recognize that students will have life-long learning, we must also make them aware that preparations for that learning also are important, although not necessarily part of the grade. This needs to be conveyed to each student prior to the assessment process. Further, we need to consider the many tools available for assessment. Self-, peer-, and group assessment as well as groupbased projects are vehicles that readily spring to mind using learning journals or problem-based case studies, and are also used in ESD. There are however many more and it is important to carefully research those that we think are most suitable and that students also view as the most suitable. We anticipate that this design of assessment methods will be an on-going process as we gain new insight from the students themselves. In a programme with an innovative profile, we anticipate that the feedback supplied with each individual assessment will be more demanding of us as professional tutors. If our students are becoming more reflective, undertaking 'deep' rather than only 'surface' learning, then teachers too need to be part of a more substantial learning process. This would call for more involvement and commitment from us (involving time, deeper thought processes, research) and also, providing a more worthwhile and rewarding all-round environment for the student to continue their learning. We will need to reflect this through stimulating and supportive feedback. Finally, it is fundamentally important that assessments allow students to demonstrate the reflective nature that has been introduced, nurtured and developed in their classes. If this is not accomplished, we may instead have provided them with another obstacle to 'deal with' in their learning process. Handled properly however, students will be able to demonstrate more freely personal and professional skills acquired and developed. Over time, this will help us to become more confident in knowing that these students are indeed the good edutainment software design professionals we wish them to be.
A KBUD drawback: initially, higher costs KBUD style course development has been a rewarding experience as it has brought vitality and participation into our planning. The approach has however also drawn more resources than what an ordinary class would do. If student outcome is what counts however, money is well spent. Besides increased consciousness about learning and individuals with an ability to self-reflect, the approach has also fostered students with a high GPA. This fact should of course not only be attributed to KBUD; favourable applicant numbers must also be considered. Out of nearly 800 applying, a group of 38 were selected to ESD. Admission to Swedish universities are handled through a national ranking system, where applicants with the highest scores will get their selections first.
CONCLUSIONS The teaching and planning practice presented aims at ensuring student reflection and continuous course revision. It directs focus to students, as individuals and as participating reviewers of course design. The approach draws more resources than traditional courses but will also refine students' ability to reflect on required problem-solving competences, and to take action to attain desired competences. This outcome may be one of the most valuable insights they can achieve during their university years, when seen in a life-long perspective.
REFERENCES [1]
Woodard IV, E H, Gridina, N, "Media in the home 2000", The Annenberg Public Policy Center (University of Pennsylvania), Survey series No. 7, 2000, www.appcpenn.org
[2]
Metzirow, J, "Fostering Critical Reflection in Adulthood", Jossey-Bass Higher Education Series, 1990.
[3]
Brookfield, S, "Becoming a Critically Reflective Teacher", Jossey-Bass Higher and Adult Education Series, 1995.
[4]
Dreyfus, H, Dreyfus, S, "Mind over Machine", Free Press, 1986.
[5]
Smith, P D, "A Process Education Approach to Teaching Computer Science", ASCUE'96 Proc, www.saintmarys.edu/~psmith/
[6]
Kolb, D A, "Experiential Learning - Experience as a Source for Learning and Development", Prentice-Hall, 1984.
[7]
Rodger, S, "An Interactive Lecture Approach to Teaching Computer Science", Proceedings of the 26th SIGCSE Technical Symposium on Computer Science Education, March 1995, pp 278 - 282.
[8]
Fekete, A, Kay, J, Kingston, J, Wimalaratne, K, "Supporting Reflection in Introductory Computer Science", Proceedings of the 31st SIGCSE Technical Symposium on Computer Science Education, March 2000, pp 144 - 148.
[9]
Bloom, B S, "A Taxonomy of Educational Objectives Handbook I: Cognitive Domain, 2nd Ed.", McKay, 1965.
[10] Brown, G, Bull, J, Pendlebury, M. "Assessing Student Learning in Higher Education", Routledge, 1997. [11] Rowntree, D, "Assessing Students: How Shall We Know Them?", Kogan Page, 1987.
0-7803-6669-7/01/$10.00 © 2001 IEEE October 10 - 13, 2001 Reno, NV 31st ASEE/IEEE Frontiers in Education Conference T1B-6
