A Framework for Describing and Comparing Courses and Curricula

A Framework for Describing and Comparing Courses and Curricula Michela Pedroni

Manuel Oriol

Bertrand Meyer

Chair of Software Engineering ETH Zurich 8092 Zurich, Switzerland +41 44 632 76 84



[email protected]

[email protected]

[email protected]

ABSTRACT Curriculum and course planning is a key step in developing quality educational programs, but current practices very often lack a systematic approach. This article addresses this issue by refining and expanding the concept of Testable, Reusable Unit of Cognition (Truc). The methodology allows modeling courses and verifying compliance of a given course to a given description. It also makes it possible to describe precisely what students have previously learned and, as a result, adapt the teaching to their specific needs. The article presents a case study of comparing a subset of two introductory programming textbooks and introduces the application TrucStudio that supports the methodology.

Categories and Subject Descriptors K.3.2 [Computers and Education]: Computer and Information Science Education – curriculum, computer science education.

General Terms Theory, Standardization.

Keywords Curriculum design, knowledge modeling, course comparison.

1. INTRODUCTION Designing courses and curricula presents many challenges. In particular, it is difficult to ensure both that the course covers the entire subject area and that the students’ background meets the prerequisites. There is a clear need for a methodology and tools for curriculum design that help with: •

Comparing curricula and courses.

•

Assessing students’ background (particularly important in Europe due to the “Bologna” guidelines [7] which promote student mobility).

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Conference’04, Month 1–2, 2004, City, State, Country. Copyright 2004 ACM 1-58113-000-0/00/0004…$5.00.

•

Evaluating textbook coverage or how a textbook satisfies a curriculum recommendation or a standard (such as the ACM/IEEE standards [8] for computing education).

•

Avoiding unwanted repetitions of topics through a curriculum.

•

Defining the prerequisites needed for a course.

•

Checking that course material exhibits a sound sequence of topics and is at all times building on prior knowledge.

The tasks listed above are mainly contained in two activities: (1) extracting the knowledge taught in courses/textbooks/curricula as a list of units of knowledge and comparing it to the intended results or knowledge acquired from another source, and (2) defining the dependencies that occur between units of knowledge and verifying that they are met. For both of these tasks, a more formal understanding of curricula is essential. This article discusses a technique to design, describe, and compare curricula based on an extension of the notion of “Truc” [5], and applies it to the comparison of two course structures for introductory programming. The main contributions are: •

A systematic approach to modeling courses based on a refinement and extension of the notion of Truc.

•

Objective means to compare courses and curricula on the basis of coverage of topics.

•

The application to a case study comparing two specific textbook extracts in the same area (introductory programming).

•

A prototype application that supports the methodology.

Roadmap. Section 2 presents an overview of the approach. Section 3 presents the notion of Truc and analyzes its limitations. Section 4 introduces additional useful concepts to cope with these limitations: notions and relations. Section 5 is the case study and section 6 gives a summary on the current state of the supporting software, TrucStudio. Section 7 reviews related work. Section 8 describes future work and concludes.

2. OVERVIEW The key step towards systematic curriculum description is to provide a model of the units of knowledge that make up the educational material. The actual course or curriculum is then a path through these units, with possible repetitions. This defines an objective mechanism to compare two courses by analyzing their paths according to their contents and the order of visit.

cover all of the above Trucs. A similar course taught with Java probably will not teach the Truc “contract” (see Figure 1). This difference can easily be captured by comparing the covered Trucs. In this respect, then, Trucs are useful.

feature call

Such an approach immediately raises several issues: granularity, relations between units, and domain-specificity. Granularity. What granularity should the units have if the ultimate goal is to compare courses? Should there be a single level of granularity, or several? The present work defines units of knowledge at two levels of granularity: “Trucs” and “notions”. At the higher level, a “Truc” is a set of concepts and operational skills emerging from one central idea; at the lower level, a “notion” is a single concept, or facet of a concept. Relations between units. What kinds of relations may occur between units of knowledge? The answer presented here involves two relations between notions, “is a” and “requires”, and one between Trucs, “dependency”. Domain-specificity. How should we handle domain-specific aspects such as the role of the programming languages? This article proposes to handle domain-specific aspects through special tags added to Trucs, notions, and relations. The following sections develop these answers, starting from the definition of Truc and extending it with notions and domainspecific items.

3. TRUCS AND THEIR LIMITATIONS 3.1 Basic properties A Truc (Testable, Reusable Unit of Cognition) is “a collection of concepts, operational skills and assessment criteria” [5] and satisfies the following properties: (1) all components of the Truc follow from a central, clearly identified idea; (2) the Truc possesses a clear, unambiguous description; (3) the Truc includes assessment criteria to judge whether a student has mastered the concepts and skills; (4) included topics span a scope of general interest; and (5) the scope is small enough to be covered in one or a small number of lectures or in a textbook chapter or a few sections. Trucs are described in a standardized scheme containing sections on the Truc’s name, any alternative names, its dependencies, and a summary. The standard scheme also includes sections on the role of the Truc in the field, when and where it can be applied, its benefits; it gives examples, states the common confusions that might occur when mastering the concepts represented by the Truc, and the disadvantages of applying the concepts. The Truc’s specification concludes with a set of typical tests for assessing understanding of the Truc’s concepts.

3.2 Assessment To assess how Trucs can help with comparing curricula, consider Trucs covering a subset of the concepts for object-oriented programming provided in the ACM/IEEE Curriculum standard [8] “object”, “argument passing”, “class”, and “feature call” (method in the syllabus). An additional Truc is “contract”. Obviously, a course that teaches introductory programming with Eiffel will

object

class

argument passing Eiffel Eiffel

contract

Figure 1: Example of Trucs When considering smaller-grained differences, Trucs are too coarse grained (e.g. to adapt the course to an audience). For example, Java uses the main method as entry point for an application; this concept is unknown in Eiffel. These detailed differences are particularly interesting for courses that build on prior knowledge acquired in other courses. As an example, the instructor of a course for Eiffel programmers on design patterns that uses Java as a support will be able to spend very little time on language details. Thus, it requires that he considers what exactly the students have already learnt in the Eiffel courses and where this knowledge differs from the required Java knowledge. In most cases the differences are very small, but still need to be understood to comprehend the course. Checking whether a textbook or a course covers all the required topics constitutes another task where details are of great importance. Coverage verification entails comparing existing educational material to some knowledge to teach. It is easy to verify that the big topics (the Trucs) are covered; instructors or textbook writers rarely forget to treat a whole Truc. But the devil is in the details and they may forget some notions. As an example, the case study in section 5 identifies several small parts of otherwise covered Trucs that are missing in the two selected textbooks. These observations suggest that Trucs need some refinement to address the demands of course/textbook comparison.

4. ADDITIONAL CONCEPTS 4.1 Notions To address the need for a more fine-grained kind of knowledge unit, the framework proposes a unit called notion. While a Truc represents a collection of concepts and operational skills a notion represents a single concept or operational skill or a specific facet of a concept, subject to the following properties: 1.

A notion is part of at least one Truc (it may be part of several).

2.

It represents a particular view or usage of the Trucs to which it belongs.

3.

It may have the same name as the Truc to which it belongs, in which case it is said to be the Truc’s Central notion.

The description of Notions uses the following components: its name, a short summary, and a list of alternative names.

Examples of notions that are part of the above mentioned Trucs are “simple qualified feature call” (method calls of the form x.f with x a query returning an object and f a procedure), “chain of qualified calls” (method call chains of the form x1.x2.x3.f), or “return value”, all belonging to the Truc “feature call”.

4.2 Relations Two types of relations between notions impose a structure on a set of notions in a Truc: the “is a” relation and the “requires” relation. “a requires b” defines that understanding notion a requires to know notion b. This relation is comparable to the client relationship between classes in object-oriented systems. It may appear across Trucs boundaries and as such it defines the Trucs dependencies. “a is a b” expresses that a notion b refines notion a; it is comparable to the inheritance mechanism in object-oriented systems. A refined notion implicitly inherits all the “requires” links from its ancestor, but may also introduce additional ones. For simplicity, the methodology prohibits “is a” links across Truc boundaries. The “requires” and the “is a” links between notions define a graph where the nodes are notions and the edges are links. We call this graph a notions graph. Since each Truc contains a set of notions, the Trucs can be seen as clusters that group notions in the notion graph. This is then called a clustered notions graph. Figure 2 shows an example of a clustered notions graph containing the notions of the Trucs “feature call”, “object”, “class”, “argument passing”, and “contract”. As an example, the notions for the Truc “class” are “class”, “class declaration”, “type”, and “creation declaration” (shown as grey boxes in the figure). The solid black arrowed lines represent “is a” links while the dashed lines mark “requires” links.

chain of qualified feature calls

feature call

feature

simple qualified feature call

unqualified feature call

argument passing

object

argument passing

object

Truc

class

Eiffel

Domain-specific Truc Notion

contract Eiffel

Eiffel

class

class declaration

type

creation declaration

Domain-specific notion

contract “requires” “is a”

Eiffel Eiffel

class invariant

While the introduction of notions is important for course/textbook comparison, the relations (both on notions and on Truc level) are not needed for this task. They constitute, however, the basic instrument to deal with the second main activity in curriculum and course design: identifying dependencies between units of knowledge and verifying that educational material meets them. While the framework shows the constructs here, this paper refrains from detailing further on their application.

4.3 Domain-specific notions and relations The scope of Trucs should be “of general interest, beyond a specific course” [5]. The practice of teaching, however, involves a specific course and specific tools, which we must take into account. For example, a course on object-oriented programming is quite sensitive to the choice of programming language; this choice should not influence the concepts that the students learn, but it will influence the teaching by introducing additional links and notions. Comparing such domain-specific dependencies is per se interesting to assess the differences between a course taught and a prerequisite. It is thus possible to define domain-specific notions, Trucs and relations.

4.4 Curricula, Courses, and Trucs

argument

instance

The “dependency” links between Trucs can directly be used in the Trucs graph (see Figure 1) that can be abstracted from the clustered notions graph (see Figure 2). The inter-Truc links in Figure 2 show the following Trucs dependencies: “feature call” requires “object” and “argument passing”, “object” requires “class”, and “argument passing” requires “object”.

Figure 2 shows the additional Trucs considering the domainspecific extension for OOP taught with the programming language Eiffel. Domain-specific notions and links show in lighter color and are tagged with “Eiffel”. As an example, the Truc “contract” with its notions “contract” and “class invariant” is added when making the Eiffel-specific extension of OOP. Note that domain-specific notions may also be added to non-domain specific Trucs (e.g. to the Truc “feature call”). Domain-specific extensions are mere additions of notions, links, and Trucs to the clustered notions graph.

feature call

return value

A student is said to have mastered a particular Truc, if she has mastered all the notions belonging to its clustered notions graph. The dependencies of a Truc are a direct consequence of the “requires” links that pass across Truc boundaries. In short, a Truc A depends on a Truc B if at least one notion in Truc A exists that requires a notion that is part of Truc B.

Domain-specific “requires” Domain-specific “is a”

Figure 2 Clustered notions graph with domain specific notions and Trucs.

A Curriculum is a set of Trucs. The set of all the notions of all the Trucs composing a Curriculum is called the set of underlying notions of the curriculum. A course is a list of notions taught one after the other. The set of all the notions in a course is called the set of underlying notions of the course. Comparing two curricula or two courses is to compare their underlying notions: one is compliant with another if its set of underlying notions includes the other’s set of underlying notions. Besides compliance verification, it may also be interesting to compare two sets of underlying notions and find both their intersection (describing the common knowledge they cover) and their relative complements (describing what has to be taught to

complete knowledge or what has to be mentioned as a difference from previous knowledge).

5. A CASE STUDY: COMPARING TWO PROGRAMMING TEXTBOOKS The clustered notions graph enables us to model courses’ educational contents by inspecting what notions of the graph are covered one after the other. This results in a list of notions capturing both the sequence and the coverage of notions. In general, the comparison of courses or textbooks consists of two actions: (1) developing the clustered notions graph and extending it with domain-specific elements, and (2) mapping the content of the two courses to these entities. The case study considers two textbooks (Touch of Class by Bertrand Meyer [6], using Eiffel, and Programming and Problem Solving with Java by Nell Dale et. al. [2]) offering introductions to Object-Oriented Programming (OOP). Both books are intended for students with no prior programming experience. While Touch of Class uses the programming language Eiffel, the second book uses Java.

contract class invariant argument passing argument return value chain of feature calls simple qual. feature call unqualified feature call feature call feature creation declaration class declaration type class instance object

Chapter 3

Besides highlighting the differences in the topic sequences, Figure 3 also allows to detect topics that are entirely missing or only covered by one of the textbooks. As an example, the notions “creation declaration” (denoting the concept of constructor for Java) and “unqualified feature call” are missing in the Eiffel book extract, but covered in the Java book. As expected, only the Eiffel book covers the Truc “contracts”. The notion “chain of feature calls” (denoting calls of the form o1.o2.o3.f where o1, o2, and o3 are queries returning an object and f is a procedure) is missing in both books. The above analysis suggests two conclusions about using notions: (1) They allow the detection of common and distinct topics; (2) they allow the detection of missing topics.

6. TRUCSTUDIO

Touch of Class Chapter 2

notions of the Trucs “object” and “class” and moves on to “feature call” in the second half of Chapter 2. The Eiffel book takes a different approach by starting with mostly notions of the Truc “feature call” and “object” (touching “class” shortly), then goes on with a more detailed discussion of notions of the Truc “class” and concludes with a discussion of “contract”.

Chapter 4

contract arg. passing

feature call

To alleviate the amount of work requested by educators to use Trucs, it is important to provide software support allowing the creation, maintenance, and application of Trucs. Figure 4 shows the alpha version of TrucStudio that supports the methodology associated to Trucs (a downloadable beta version is scheduled for April 2007). To enable further scalability, TrucStudio defines three layers for domain descriptions: Clusters (knowledge areas, e.g. OOP), Trucs (e.g. class), and Notions (e.g. class declaration). The dependency relations for Trucs and Clusters are deduced from the underlying layers.

class object Chapter 1

Chapter 2

Programming and Problem Solving with Java Touch of Class Programming and Problem Solving with Java

Figure 3 Comparison of textbook extracts. For brevity, the study considers only the first two or three chapters of the textbooks and only the notions and Trucs showcased in Figure 2. Textbooks (just as lessons) are a succession of topics (notions taught). Figure 3 shows in what order the notions from Figure 2 appear in the two textbook extracts. The x-axis of the diagram is the timeline (showing the division into single chapters for both books), while the y-axis shows the various notions sorted by the Truc to which they belong. The analysis of data point clusters for each of the textbooks shows that while the general scope of the textbooks is similar, the order of topic introduction varies. The Java book starts covering mostly

Figure 4 Screenshot of TrucStudio prototype. Currently (January 2007), the tool already supports loading and saving of domain descriptions and course descriptions in xml format, exporting the clustered notions graph to dot (the file format for Graphviz [3]), and automatic checking for soundness of sequences (verifying that the dependencies for teaching a particular Truc or Notion are met). Further work includes support for ontology file formats, implementation of several graphical

views, support for domain-specificity, and export of Truc descriptions to HTML for online access by students and teachers.

7. RELATED WORK The original work on Trucs [5] prohibits both cycles and transient subgraphs in the dependency structure. The refined methodology presented here refrains from putting such restrictions on the “requires” relations between notions and the resulting deduced Truc dependencies. Relaxing the requirement of non-transient subgraphs enables a higher expressiveness for Trucs and notions. Identification of cycles in the dependency structure of Trucs or notions is by itself an interesting task since it shows concepts that are probably difficult to teach one without another and require a cyclic teaching approach. The required software support to automate cycle detection will be integrated in TrucStudio. Mead et. al. [4] developed the idea of Anchoring Concepts (AC) and Anchoring Graphs. An AC is defined as a concept that is either foundational, or both integrative and transformative. The definition of AC in connection with Cognitive Load Theory supports the construction of directed graphs (Anchoring Graphs) where an edge from one AC to another shows that the source node carries part of the cognitive load to learn the target node. While the concept of Trucs+Notions and AC seem to be very similar, they differ significantly in their intents. The Anchoring Graph approach identifies the foundational concepts that should be taught before another AC in order to reduce the cognitive load while the present work identifies concepts and their relations. This has the consequence that to build an AC graph, one has to consider the teaching sequence while a notion graph is only concerned with the material. On one side, our approach is closer to the way teachers view the material they teach and thus it enables a domain specialist to design notion graphs in an intuitive way. The actual teaching is left to the teacher. On the other side, the Anchoring Graph approach requires to take teaching-related decisions at design-time. In the future we hope that the two approaches can interact and complement one another. The MOT model [9] includes graphical knowledge representation for designing tele-education systems. Similar to the approach proposed in this paper, it defines knowledge graphs containing “knowledge objects” as nodes. MOT proposes six types of knowledge objects and six types of links accounting for the edges of the graph. Exact rules define for each link type what knowledge object types it may connect. The complexity of the MOT may be needed for building an online education system, but is an obstacle for comparing curricula. The extended Trucs take a simpler approach to modeling both knowledge and dependencies. The Curriculum Initiative CC2001 [8] defines the body of knowledge for Computer Science by listing the core concepts belonging to a specific area. An ongoing recent effort is to capture the computing knowledge in an ontology [1]. In the future it is likely that standard Trucs would be defined according to these specifications. While these efforts have a great impact on curricular planning, they don’t specify how compliance to the course definitions (as reported in CC2001) can be assessed. The present approach would then benefit such efforts by providing the needed infrastructure.

8. FUTURE WORK AND CONCLUSIONS The work so far has included defining curricula for OOP and adding domain-specific extensions for Eiffel. We plan to change all the course descriptions into Truc descriptions, thus building a strong basis of examples. It is unlikely that we can achieve this for all the computer science courses by ourselves. To enable a collaborative effort, we plan to extend TrucStudio with the functionality of storing curricula and Trucs in a publicly accessible repository. This might make the framework attractive to associations such as the ACM that committed to providing standard syllabi. This article has presented a new way of defining curricula, domain-specific extensions and courses. The methodology is graphical and conjectured to be suited for an intuitive understanding by both teachers and students. The central point is to allow easy comparison of similar domains and teaching. This brings both normative and descriptive support to the definition of curricula, and yields clear views of previous knowledge and course prerequisites. The approach seems particularly useful in teaching environments characterized by a wide diversity of student backgrounds.

9. REFERENCES [1] Lillian (Boots) Cassel, Andrew McGettrick, and Robert H. Sloan. A comprehensive representation of the computing and information disciplines. In SIGCSE ’06: Proceedings of the 37th SIGCSE technical symposium on Computer science education, pages 199–200, New York, NY, USA, 2006. ACM Press. [2] Nell Dale, Chip Weems, and Mark Headington. Programming and Problem Solving in Java. Jones and Bartlett Publishers, 2003. [3] Graphviz, Graph Visualization Software. Online at http://www.graphviz.org [4] Jerry Mead, Simon Gray, John Hamer, Richard James, Juha Sorva, Caroline St. Clair, and Lynda Thomas. A cognitive approach to identifying measurable milestones for programming skill acquisition. In ITiCSE-WGR ’06: Working group reports from ITiCSE on Innovation and technology in computer science education, pages 182-194, Bologna, Italy, 2006. ACM Press. [5] Bertrand Meyer. Testable, reusable units of cognition. Computer, 39(4):20–24, 2006. IEEE Computer Society Press. [6] Bertrand Meyer. Touch of class - learning to program well with object technology and design by contract. Available online under: http://se.inf.ethz.ch/touch. [7] Ministry of Education and Research, Norway, The BolognaBergen website. Available online at: http://www.bolognabergen2005.no [8] The Joint Task Force on Computing Curricula. Computing curricula 2001 (final report). ACM and IEEE, December 2001. Online at: http://www.acm.org/sigcse/cc2001. [9] Gilbert Paquette. Meta-knowledge representation for learning scenarios engineering. In S. Lajoie and M. Vivet, editors, Proceedings of AI-Ed99 AI and Education, open learning environments. Amsterdam, June 1999. IOS Press.