An Environment for Self-Assessing Java Programming Skills in Undergraduate First Programming Courses [Extended abstract] Lorenzo Bettini1 1 Dip.
Leonardo Cecchi2
Pilu Crescenzi1
Gaia Innocenti1
Michele Loreti1
di Sistemi e Informatica, Universita` di Firenze, Via C. Lombroso, 6/17, 50134 Firenze, Italy {bettini,piluc,innocent,loreti}@dsi.unifi.it
2 Axiom
Consulting, Via Panciatichi, 26/3, 50127 Firenze, Italy.
[email protected]
ABSTRACT In this paper we propose a new environment for allowing students of a first programming undergraduate course to test their Java code. This environment allows the student to learn the basics of the Java language without necessarily knowing the object-oriented features of the language itself, and the teacher to propose new tests by making use of a graphical test editor. Moreover, the client-server architecture of the web-based version of the environment is designed so that the student does not even need a Java virtual machine on its computing device, but only a web browser. This latter feature makes our environment a useful tool for ubiquitous testing of Java programming skills.
Categories and Subject Descriptors D.3.2 [Language Classifications]: Object-oriented languages; K.3.1 [Computer Uses in Education]: Distance learning; K.3.2 [Computer and Information Science Education]: Computer science education
Keywords Java, distance education, client-server, automatic test
1.
INTRODUCTION
A continuously growing number of undergraduate first programming courses use Java as the first programming language, recognizing this language’s role in the development of web-based systems and the emerging prominence of the object-oriented approach to software development (see, for example, [4]). There is no doubt that the principal features of object-oriented programming are of great benefit when it comes to constructing large software systems, but it is a long journey from the first programming course to the construction of such systems. For this reason, several educators argued that it is better to introduce students to structured programming before turning to object-oriented programming and to other approaches that are beneficial for programming in the large (see, for example, [2]).
ICALT 2004 August 30-September 1, 2004, Joensuu, Finland
Motivated by the apparently contrasting desire of using Java as the first programming language and of following the “structured programming before object-oriented programming” approach, a simple application, called JavaMM (or, equivalently, Java--), has been developed at the Department of Computer Science of the University of Florence. This application allows the students to familiarize themselves with the basic structures of the Java programming language and with the basic notions of structured programming, without having to bother with issues related to object-oriented programming from the very beginning of the course. A detailed description of Java-- (along with a survey of existing tools for teaching Java) can be found in [1]: The application itself is freely available starting from the Java-- home page (see [9]). Java-- has been successfully used since 2002 within the first programming course of the degree program in Computer Science: In particular, the tool has been used during the first month of the course, while successively the students have been introduced to a more sophisticated IDE (such as JCreator LE [10]). According to the survey compiled at the end of the course, the students seemed to appreciate the tool: Only one student over almost one hundred complained about Java--, without specifying the reasons of the complaint. Moreover, the number of students that passed the exam at the end of the first edition of the course, in which Java-- was used, increased by more than 40% with respect to the previous edition. In this paper, we propose an extension of Java-- that includes a self-test exercise component, which allows students to test their knowledge of the Java language by writing and checking the correctness of a method performing a specific required task. The basic idea of this component consists of specifying the characteristics of a test in an XML file, which is interpreted every time a user submits its solution of the test (that is, the implementation of a Java method). This file contains the specification of the method to be tested, the specification of a correct implementation of the method, and the specification of how the test input has to be chosen. The checker interprets the XML file in order to compare the result of the submitted method with the correct method on the specified input. Subsequently, it generates either an error message specifying the input that caused the error or a message stating that the test has been passed. From the teacher point of view, we have developed a graphical tool that allows the user to easily generate new tests, that is, to create the XML file associated with a test (of course, the teacher has to write and compile the Java code containing the correct implementation of the method to be tested). Most of the textbooks on Java programming make use of several different kinds of self-test exercises, such as, multiple-choice and
short-answer quizzes, jumbled sentence exercises, crossword puzzles, matching and fill-in-the-blanks exercises: The most recent (and enjoying) example of such utilization of these kinds of exercises is [6]. Moreover, several authoring programs enables teachers to make (Web-based) interactive teaching materials based on these exercise-types: An example of such programs, available both for Windows-based and for MacOS-based computers, is the set of Hot Potatoes applications (see [8]). However, as far as the authors know, no tool is available to generate Java code-writing exercises that allows the students to verify that their solutions is semantically (not only syntactically) correct. In [7] a test-first approach to programming and to learning is described: To this aim the author first uses an ad hoc framework, and then uses JUnit. His extreme teaching approach, however, is based on the assumption that objects really need to come first, while, as we said before, our approach assumes that structured programming comes before object-oriented programming. For this reason, we decided to integrate the Java-with the self-test exercise component (of course, our tool can be also used within a Java course where objects are taught first). In the paper, we also propose a web-based client-server version of Java-- (including the self-test exercise component) that allows the students to make use of this tool by means of basically any web browser, without even needing a Java virtual machine. In particular, the proposed environment consists of two main ingredients. From the client point of view, there are static HTML pages, which allows the user to write Java code, to submit the code to a Java compiler, to see either the compilation errors or the standard output messages, and to test the implementation of a specified method. From the server point of view, there is a collection of servlets (for an introduction to servlets, see [3]), which receive and compile Java codes, and generate either error or standard output HTML pages. The main advantage of this approach is the fact that the client software requirements are extremely light, and basically any currently available mobile computing device can be used in order to utilize the Java-- tool (of course, standard computers can also be used. . . ). In other words, this environment turns out to be a useful tool for ubiquitous programming and learning Java. The paper is structured as follows. In Section 2, we describe the structure of the XML file that specifies a test, and we briefly describe the graphical tool that allows to generate this XML file. In Section 3, instead, we describe the component of Java-- that allows the student to execute one test, while in Section 4 we introduce the software architecture of the web-based client-server version of Java--. Finally, in Section 5 we present some preliminary evaluation results regarding the effectiveness of the Java-- tool, while in Section 6 we propose some directions for future research and developments.
2.
THE TEST SPECIFICATION
Each test in our framework consists of checking the correctness of a method implementation. Thus, a test is described by specifying the signature of the method, that is, the name of the method, the type of its returned value, and the list of the types of its parameters. Since our target is to deploy a self-test tool for undergraduate students of a first programming course based on the “structured programming before object-oriented programming” approach, in the first version of our program we have decided to allow only a subset of primitive data types, the String class type, and arrays of these types. In particular, the returned values and the input parameter can be int, double, char, boolean, String, or an array of any of the previous types.
The document type definition (DTD) associated with the XML file containing a method specification is the following one:
Note that, besides the signature of a method, the above DTD file contains some more elements. In particular, a test description has to specify the name of the class that contains the correct implementation of the method, which has been coded and compiled by the teacher. Moreover, the test can be performed on fixed values of the parameters, on random values of the parameters, or on a combination of fixed and random values: In the case of randomly chosen input values, the test description has to specify the range within which the random values have to be chosen. Finally, in the case of randomly chosen values, the test can specify how many times it has to be performed. As an example, let us suppose that we want to test the implementation of a method maxBetweenTwo, which is supposed to return the maximum between two input integer numbers: −10 and 10. Moreover, suppose that the correct implementation of this method is contained in the class Maximum. Then, the XML file specifying such a test is the following one: 1 Maximum INT INT −10 INT 10
The XML files containing the specification of a test can be manually created by the teacher: However, a simple graphical interface has been developed in order to support this task. Once the test has been specified, the teacher has to write and compile the class containing the correct implementation of the method. In our example, this class might be the following one: public class Maximum { public int maxBetweenTwo( int a, int b ) { return a>b?a:b; } }
Finally, the teacher can associate with the test a natural lan-
guage description of the test itself (to be included in the file description.txt) and a code template of the method that the student has to code (to be included in the file template.txt). In the example of the maximum between two integers, the description might be “Write a method, called maxBetweenTwo, that, given two integers, returns the maximum between them”, while the code template might be the following one: int maxBetweenTwo( int a, int b ) { }
3.
THE TEST EXECUTION
Figure 1 shows the interface through which the student can execute a test (all the other functionalities of the JavaMM program are described in [1]). As shown in the first picture, the student can choose one test among all those that have been specified by the teacher (in our case, there are three tests available). Once the student has chosen a test, the corresponding code template and description text are shown in the code and in the test description panel, respectively (as shown in the second and in the third figure). At that point, the student has to write the code that implements the method: This code can finally be tested by clicking on the Test button. If the implementation is correct (or at least passes the test), a confirmation message is shown in the Output panel, otherwise a failure message is shown in this panel (we are assuming that the code written by the student is syntactically correct, since otherwise a compilation error would appear in the Errors panel). The failure message also specifies the name of the method, the value of the input parameter that caused the failure, the expected result, and the obtained result (see, for example, the last picture of Figure 1 where the test failure is due to the fact that the body of the method maxBetweenTwo contained the instruction return a
