Model-Based Reasoning for Domain Modeling in a Web-Based Intelligent Tutoring System to Help Students Learn to Debug C++ Programs Amruth N. Kumar Ramapo College of New Jersey 505 Ramapo Valley Road Mahwah, NJ 07430-1680
[email protected]
Abstract. The benefits of using Model-Based Reasoning for domain modeling are several-fold. We analyze these benefits and illustrate them in the context of a Web-based Intelligent Tutoring System. The system is designed to teach students to analyze and debug C++ programs for semantic and run-time errors. We have evaluated one instance of the Model-Based tutor, which deals with debugging pointers in C++, in several sections of Computer Science II course. We will present the results of these evaluations, which confirm the learnability of Model-Based tutors.
1
Introduction
We are developing an Intelligent Tutoring System to help students learn the C++ programming language by analyzing and debugging C++ code segments. Among the six levels of abstraction of educational objectives proposed by Bloom [4], we target application (use methods in new situations, solve problems using knowledge) in our ITS, as opposed to program synthesis, which has been the focus of many earlier works (e.g., LISP Tutor [15], PROUST [10], BRIDGE [6], ELM-ART [7] and Assert [3]). Our work focuses on tutoring programming constructs rather than the entire programming enterprise. It focuses on semantic and run-time errors in C++ programs as opposed to syntax errors that a compiler would detect. We have been using Model-Based Reasoning [8] to model the domain for our tutoring system. In Model-Based Reasoning, a model of the domain is first constructed, consisting of the structure and behavior of the domain. In our case, this would be a model of the C++ language, consisting of objects in the language and their mechanisms of interaction. This model is used to simulate the correct behavior of an artifact in the domain. In our case, the model is used to simulate the expected behavior of some particular C++ language construct, e.g., C++ pointers. The correct behavior is compared with the behavior predicted by
the student for that artifact. The discrepancies between these two behaviors are used to hypothesize structural discrepancies in the (mental model of the student for that) artifact. In our case, the behavioral discrepancies are used to generate feedback to tutor the student. Figure 1 illustrates the architecture of an Intelligent Tutoring System that uses Model-Based Reasoning for domain modeling (figure is adapted from [9]).
Program Model
Simulation Behavior
Structural Discrepancies
Behavioral Discrepancies
Student Model
Predicted Behavior
Fig. 1. Architecture of Model-Based Reasoning for Tutoring Program Debugging
Insofar as the domain model is complete, Model-based reasoning is comprehensive in its coverage of possible behavioral (and hence, structural) discrepancies. This is not necessarily true of Rule-Based systems (e.g., production rules used in ACT-R theory [2]), which cannot address behavioral discrepancies unless they have been explicitly encoded into the tutoring system. Similarly, CaseBased Reasoning systems are primarily constrained to the types of cases already entered into the knowledge base [16] In the next section, we will discuss and analyze the benefits of using ModelBased Reasoning in our tutoring system. In Section 3, we will describe the currently implemented features of the tutoring system. In Section 4, we will present the results of evaluating the tutor in several sections of our Computer Science courses. Finally, we will discuss conclusions and future work in Section 5.
2
Model Based Reasoning for Domain Modeling in an Intelligent Tutoring System
There are several advantages to using Model-Based Reasoning for domain modeling in Intelligent Tutoring Systems. Domain Model is the Expert Module: We need not include the answers to problems in the ITS. The model knows the correct answer, i.e., it is capable of solving each problem to obtain the correct answer. Therefore, the domain model doubles as the runnable expert module. Constraint-Based Modeling [14] does not require the inclusion of a runnable expert module either. However, it targets the knowledge that prescribes user’s actions whereas Model-Based reasoning targets the knowledge that describes the domain’s behavior. In this sense, the two could
co-exist in an Intelligent Tutoring System. There have been more recent efforts to extend Constraint Based Modeling to include a runnable expert module [12]. Dynamic Generation of Problems: Limited problem set has been recently recognized as a potential drawback of encoding a finite number of problems into a tutor [12]. Using Model-Based Reasoning for domain modeling can easily address this drawback. Since domain models based on Model-Based Reasoning are capable of solving problems on their own without being told the correct solution, a tutor using such models need not be restricted to administering only the problems that have been encoded into it. When coupled with a scheme for generating problems, such a tutor can potentially administer an unlimited number of problems to the learner. One scheme used in literature to dynamically generate problems is by using BNF-like grammar, e.g., [11]. In this scheme, problems are generated by randomly instantiating the grammar. Each rule of grammar can be carefully designed with specific pedagogical objectives in mind. We have used such a generative scheme with our Model-Based tutor to be able to generate an unlimited number of problems. (Please See Figure 2, where templates are BNF-like grammar rules).
Problem Code
Problem Model Template
Expert Model Domain Model
Solution
Domain Objects
Fig. 2. Generative Architecture of a Model-Based Tutor
Randomizing a template to generate problems is acceptable in the domain of program analysis and debugging, since the context of the problem is fully captured in the templates. However, this is not necessarily true in all the domains, e.g., in the legal domain [13], the context is much richer and harder to capture in templates. One criticism that may be leveled at the process of randomizing a template is that it could generate only trivial variations of a ”typical problem.” In the programming domain, this is not true. Randomizing can yield sufficiently interesting and non-trivial variations of a problem. E.g., consider the following snippet of correct C++ code: {
int *variablePointer; int count = 32; variablePointer = &count;
}
cout
