A system to help teaching and learning algorithms - IME-USP

A system to help teaching and learning algorithms Leônidas de Oliveira Brandão, Romenig da Silva Ribeiro, Anarosa A. F. Brandão University of São Paulo - USP São Paulo, Brazil {leo,romenig}@ime.usp.br, [email protected] teaching and learning aspects. It can run in any desktop, web browser, or integrated to a web-based Learning Management System, such as Moodle. Nevertheless, the paper main contribution is related to the results of some experiments conducted with iVProg, in order to evaluate possible gains while teaching and learning algorithms and programming using it.

Abstract— One of the big challenges that teachers and students face at early stages of CSET (Computer Science, Engineering, and Technology) courses are the difficulties related to teaching and learning algorithms. Usually, such knowledge is presented during introductory programming related disciplines, when students are also presented to some programming language to describe simple algorithms. Therefore, they need to learn not only the new language but also a formal way to describe the solution of problems using that language. This is much different from any formalism they were presented in their previous student’s experience. This paper presents iVProg, a system conceived to overcome some of these difficulties by using visual programming. Some experiments were conducted during three years to evaluate students’ performance without the system support and with it using different didactic approaches.

II. RELATED WORK Many researchers dedicated efforts to develop pedagogical methods and educational software to support processes of teaching and learning algorithms and programming. Hazzan et al. [5] made an overview on the main teaching methods based on available educational software, different forms of class organization, and mentoring software development projects. Existing educational software that support teaching and learning algorithms and programming differ in several ways, e.g. some of them work offline, other work online and a few in both modes [3][4][6][7][8]. Since the number and approaches adopted for educational software is considerable, Rongas et al. [9] proposed four different categories to classify them: (i) integrated development interfaces; (ii) visualizing systems; (iii) virtual learning environments, and (iv) submission systems. Since iVProg is based on the Visual Programming paradigm, our interest relies on visualizing systems. Among them, there are two systems that present many commonalities with iVProg: Alice [3] and Scratch [4]. In fact, both are visual programming software, with open source licenses, that allow online testing as iVProg. However, only iVProg presents the possiblitity of online edition and integration to Learning Management Systems, such as Moodle. Scratch was conceived at Massachussets Institute of Tecnology and it allows students to learn some important mathematical and computacional concepts through an easy and friendly interface [10]. The system use the mouse as interaction instrument where students drag-and-drop some code components to build the algorithm. Students are able to create animations, simple games, musics, interactive histories, and more. It is possible to watch animations and play Scratch games through the Internet, there is also an interesting online repository of Scratch projects, but algorithms edition is only available offline. Furthermore, it is not integrated into a LMS. Alice was developed at Carnegie Mellon University. The system uses the creation of 3D animation and interactive histories to motivate learners during the processes of developing skills in algorithms design and acquiring computacional concepts. Alice also uses drag-and-drop as interaction method, and, such as Scratch, it is not integrated into a LMS.

Index Terms — algorithms; visual programming; learning system; introductory programming; free software.

I. INTRODUCTION One of the big challenges that teachers and students face at early stages of CSET (Computer Science, Engineering, and Technology) courses are the difficulties related to teaching and learning algorithms [1][2]. Usually, such knowledge is presented during introductory programming related disciplines, when students are also presented to some programming language to describe simple algorithms. Therefore, they need to learn not only the new language but also a formal way to describe the solution of problems using that language. This is much different from any formalism they were presented in their previous student’s experience. Moreover, students are also presented to some programming environment in order to edit, debug, compile and run programs they built and that describe algorithms to solve some proposed problems. Such an overload of new information may decrease student’s motivation and compromise their performance, occasionally leading to course abandon. In order to overcome this problem some attempts of using visual systems to support the learning of introductory programming were proposed [3][4]. Such systems adopt the drag-and-drop approach to guide students on building programs without any concern related to the programming language syntax and semantics. They also offer the possibility of compiling and running programs within them, which avoid the use of programming environments at the initial stage of learning algorithms and programming. This paper presents a free system called iVProg that offers visual programming features to support both, introduction to algorithms and the principles of programming, considering

42nd ASEE/IEEE Frontiers in Education Conference

978-1-4673-1352-0/12/$31.00 ©2012 IEEE

1103

Management Systems (LMS), such as Moodle [14]. By being integrated into LMS, it can foster the development of digital content in a collaborative manner and the use of several LMS resources combined with iVProg in order to visualize the results of activities that were performed by students and sent to the LMS. iVProg has been used in the last editions of an introductory programming discipline with success, and some experiments were conducted to evaluate how does the system influence the learning of algorithms and programming. Two experiments were described in the next section. Currently the system is being rebuilt in order to be compliant with a framework our research group developed to serve as basis for any iLM and facilitate their maintenance and evolution [15]. The version used in the experiments is available at http://www.matematica.br/ivprog/iVprog2011.jar.

Initially, Scratch was designed for K12 and high school students and Alice for undergraduate students of introductory courses in Computer Science. Both of them focus the engagement of the students and try to let their first contact with programming easier with a lower cognitive load during the process [11]. In the next session, we are going to explain how iVProg was built and justify our focus on lowering the cognitive load for the student. III. IVPROG OVERVIEW iVProg [12] is a system that derives from the Alice system in a way it was built as a simplified version of Alice to run in web browsers and to be used by Brazilian students that, generally, don´t speak English. Therefore, it follows Alice license as an open source code and is also coded in Java. It's important to know that iVProg was adapted to teach algorithms using the procedural programming approach while Alice adopts an object-oriented approach. Moreover, in order to make possible the use of iVProg in web browsers it was needed an important reduction of the Alice´s original code, which is more than 140Mb heavy. Currently, iVProg code is under 10% of Alice´s one. Nevertheless, it was built as an iLM (Interactive Learning Module) [13].

IV. EXPERIMENTS The experiments took place during two semesters of an introductory programming discipline (MAC110), in an undergraduate course of Mathematics. A blended learning approach [16][17] was adopted, since besides being a face-toface course, there were several assignments to be done at distance weekly. The idea was to investigate if the use of iVProg could improve the understanding of how to solve problems algorithmically and how visual programming would impact the ability of using an imperative programming language (such as C) in a traditional programming environment. The first experiment involved students of MAC110, in an undergraduate course of Mathematics during the first semester of 2010. It was a controlled experiment, in a laboratory with observers. The second experiment involved classes T1, T2 and T3 during whole semesters of 2005, 2010 and 2011, respectively. In the following subsections the experiments were described and their results were presented. Discussion about the results is provided in a section dedicated to it. A. Observed Exercises The first experiment was conducted with 6 students, volunteered, randomly divided in two groups, G1 and G2. Each one was asked to solve two exercises in a period of one hour. An observer, that would annotate the number of mistakes related to syntax and the number of times the program was run until the student evaluate it as correct, was aside of each student. Students from group G1 would do the exercises using iVProg at the beginning and using a C traditional programming environment. Students from group G2 would do the same exercises using the C language in a traditional programming environment at the beginning and using iVProg after. In both cases the exercises would be done individually without advice of anyone. The experiment goal was to analyze the initial hypothesis that the system would simplify the programming task for newcomers in programming. Participants answered a questionnaire composed of two questions: (Q1) Do you know the C language? If yes, how experienced are you? and (Q2) Did you already attend MAC110? If yes, what was the programming language and

Figure 1 Screenshot of iVProg.

The main functionalities available in iVProg are the ones related to visual programming, such as the possibility of using drag-and-drop to build functions and methods in a program that is built within the system. In fact, there are pre-defined buttons for comparison between variables, for the most used mathematical functions, and for the main programming constructs, such as if-then-else and loops. Figure 1 shows the screenshot of iVProg. In the left side there are the aforementioned buttons, in the right side is the workspace, where the program is built, the horizontal menu bar presents buttons for running the program, undo/redo functionalities and a wastebasket for discarding program components. Another important functionality that iVProg maintains from Alice is that the system prevents some common mistakes students use to do, such as variable attribution instead of equality between variables content. Altough iVProg is a simple version of Alice, it also presents the additional functionality of communicating with web servers, which allows it to be integrated to Learning

42nd ASEE/IEEE Frontiers in Education Conference

1104

Students 5 and 6 couldn´t finish exercise E2 using both C and iVProg. They had focused on solving debugging problems and forgot the main goal that was to give an algorithm solution to the problem. This was not observed in G1, since an algorithm was defined and tested using iVProg before solving the same problem in C. For E1, the time spent for all students to solve it using C was almost 10 times longer than the used to solve it using iVProg. For exercise E2 we cannot analyze such a discrepancy since only one student did it until the end of the experiment schedule.

environment used? and start doing the exercises. After solving each exercise, the student would send it to the LMS. TABLE I RESULTS OF OBSERVATION – EXPERIMENT 1 - G1 GROUP 1 Q1 Q2 a) E1 b) iVProg c) A) B) E1 C C) D) E2 a) iVProg b) c) A) B) E2 C C) D)

student 1 student 2 student 3 average no no no yes yes no 2 1 3 2,0 1 2 3 2,0 0 0 0 0,0 9 5 15 9,7 2 1 5 2,7 2 1 2 1,7 1 0 1 0,7 8 1 0 16 4 4 4

2 1 0 5 5 1 5

14 2 0 21 5 2 2

TABLE II RESULTS OF OBSERVATION – EXPERIMENT 1 – G2 GROUP 2

8,0 1,3 0,0 14,0 4,7 2,3 3,7

E1 C

E1 iVProg

The exercises are: E1- Build a program that has two numbers as input parameters and returning their average. E2 – Build a program that ask for a natural number N and a sequence of N numbers as input parameters and print the large and the smaller numbers of the sequence in the screen. Each observer would annotate the following items for exercises done using iVProg: a) time in minutes to solve the exercise; b) number of times that use the RUN button; c) number of times that syntax or logical mistakes were discovered. Each observer would annotate the following items for exercises done using C: A) time in minutes to solve the exercise; B) number of times the program was compiled; C) number of times the program run; D) number of times that syntax or logical mistakes were discovered. Only one student of Group 2 answered Q1 positively and only one student of each group hadn´t attended MAC110 previously. Nevertheless, as we can see in Table I and Table II, it seems that had attended MAC110 before did not influenced in the experiment result. Table I shows the results of observations with G1. The time spent to solve E1 using C was 5 times longer than using iVProg, even had been solved it immediately before using iVProg. For exercise E2 such a discrepancy was smaller, just 2 times longer, indicating that as students hadn´t previous knowledge about the C language (answer Q1) they learned about it while solving exercise E1. This is also visualized in the average column, lines a) and A). Nevertheless, all students did both exercises using iVProg and C, in an interval from 13 minutes (student 2) to 53 minutes (student 3). Table II shows the results of observations with G2. This group started doing E1 using C and, having solved it, did the same using iVProg. Same steps where followed to exercise E2. As it can be depicted from Table II, only student 4 did both exercises using C and iVProg, spending 68 minutes to finish them, even having previous knowledge about the C language (answer Q1) and had attended MAC110 before (answer Q2).

E2 C

E2 iVProg

Q1 Q2 A) B) C) D) a) b) c) A) B) C) D) a) b) c)

student 4 student 5 student 6 yes no no yes yes no 23 50 35 4 10 11 2 1 11 3 6 10 2 5,0 2,0 1 1,0 1,0 0 0,0 0,0 30 didn't finish didn't finish 5 4 4 13 didn't finish didn't finish 5 3

average 36 8,33333333 4,66666667 6,33333333 3,0 1,0 0,0 30 5 4 4 13,0 5,0 3,0

A. Experiment II – using iVProg in formal courses The main goal was to analyze iVProg influence in the learning process of algorithms and programming in an undergraduate course of Mathematics. It was conducted during 3 years during one academic semester a year (18 weeks) of an introductory programming discipline (MAC110). The experiment involved 3 classes, T1 (without using iVProg), T2 and T3 (using iVProg with different didactic approach), all of them from the same course and discipline, and having the same lecturer. In T1, algorithms and programming were introduced using a system that simulates a computer (iCG) and adopted an assembly-like programming language during 5 weeks, in a laboratory. High-level commands in C were inferred from the low-level model. This system had a compiler to its machine language and is freely available at http://www.matematica.br/icg/iCG.jar. From week 6 to 18 only the traditional programming environment was used, but classes took place in classroom. For this reason, only 5 exercises were solved online and other 3 exercises, more complex, involving modeling the problem solution before going to code, were left as homework. Learning assessment was made through the exercises and two theoretical exams, in classroom, at weeks 11 and 18. In T2, algorithms and programming were introduced using only iVProg during 6 weeks. From week 7 to 11 both, iVProg and a traditional C programming environment were used to solve the proposed exercises. From week 12 to 18 only the traditional C programming environment was used. Classes took place in a laboratory with at most two students by computer. The exercises proposed were related to: average of n numbers; algorithm of Archimedes for GCD; evaluate the n term of Fibonacci´s; tests involving types, such as char and int; calculate cos(x) using Taylor´s series; vectors: sorting using an

42nd ASEE/IEEE Frontiers in Education Conference

1105

very beginning benefited students understandability about the C language.

auxiliary function, binary search, representing sets and verifying when a vector is a permutation one. An amount of 33 exercises/programs were done using iVProg and 20 using the C language, most of them solved online. It is important to observe that during the weeks where iVProg and C were used in parallel, exercises were first solved using iVProg and after using C. Learning assessment was made through the exercises and two exams: the first online in the laboratory, one student per computer, at week 11, and the second was theoretical, in classroom, at week 18.

V. USABILITY ISSUES The first time iVProg was used in a formal course was with T2. In order to evaluate some usability issues, the following question was made to students at the very beginning, in order to evaluate their familiarity with computers and its influence on the usability assessment: Q1) How many hours a day do you spend using a computer? The results show 3.31 hours/day as average time, and let us considered that they are familiar enough with computers and iVProg usability would not be influenced by difficulties related to computer misuse. A questionnaire composed of quantitative (Q2-Q4) and qualitative questions (Q5-Q7) was applied to T2 at the end of semester, in order to assess iVProg usability. For better understanding and organization, we present each question associated with its motivation and results.

TABLE III EXPERIMENT 2 – T1, T2 AND T3 LEARNING ASSESSMENT

30 25

In T3, algorithms and programming were introduced using the same approach of the one adopted in experiment 1 for G1: the use of iVProg and a traditional C programming environment in parallel. From the very first lecture, students solved proposed problems using iVProg and after that, the same problem was solved using C. Therefore, the idea was using iVProg to find the algorithm that solved the problem and C to learn some programming language. Nevertheless, since some programming concepts such as file manipulation, strings and pointers are not implemented in iVProg, the last 4 weeks of the course were whole made using just C. Thus, 25 problems were solved using both iVProg and C and 04 problems using only C. Learning assessment was made through the exercises and two exams. The first exam occurred at week 11 and was composed of 2 theoretical questions and 3 problems to be solved in the laboratory, individually, using iVProg or C. The second exam occurred at week 18 and was composed exclusively of theoretical questions, to be solved in classroom. In order to compare the influence of iVProg to support the learning of algorithm and programming, we evaluate the averages of exams 1 and 2, non-C (iCG -T1 or iVProg - T2 and T3) and C exercises and the students´ frequency at classes for T1, T2 and T3. Table III shows these results. Considering the frequency at lectures, T2 and T3 presented higher attendance average than T1. This may represent that T2 and T3 was more motivated than T1, but there is not any conclusion. Concerning the grades students obtained in exam 1, average was increasing from T1 to T3, but from T1 to T2 the growth was bigger. In this case the result was strongly related to the use of iVProg, since students did the exam using it. Nevertheless, average grades of exam 2 indicate that the contact with the C language from the very beginning had influence on it, which is theoretical and C-based. This could be explained by comparing the grades of T2 (where C was introduced after 7 weeks of lectures) with T1 and T3. Moreover, comparing the performance of students from T2 and T3 with T1 we may infer that the use of iVProg improved understandability related to algorithms. Also, the adoption of iVProg combined to a C programming environment from the

20 15 10 5 0 1

2

3

4

5

Figure 2 Answers of Q2

Q2) iVProg can be classified as ___, considering its visual aspect. 1. Uncomprehensible; 2. Hard to comprehend; 3. Neutral; 4. Comprehensible; 5. Very Comprehensible. The goal of this question was verifying if iVProg user interface was comprehensible. Figure 2 shows that 75% answered that iVProg is comprehensible or very comprehensible. Q3) In your opinion, iVProg is, in its first use: 1. Very hard; 2. Hard; 3. Neutral; 4. Easy; 5. Very easy. The goal of this question was discover how comfortable students felt at the first sight with iVProg. Figure 3 shows that 42% answered that iVProg was neutral or easy to use in their first sight, but a significant number of answers (38%) considered iVProg hard to use at the first sight.

42nd ASEE/IEEE Frontiers in Education Conference

1106

programs, the need of organization and method to comprehend the language syntax in order to use it correctly. Answers to these questions indicate that iVProg, and visual programming in general, may benefit newcomers in programming. Nevertheless, answers to the last question give some hints about improving iVProg, since they suggested that iVProg could generate an executable file from an iVProg program; the building of an area for storing programmed functions to be reused, among others.

16 14 12 10 8 6 4 2 0 1

2

3

4

VI. DISCUSSION

5

The experiment 1 and the results obtained in 2010 lead us to adopt, in 2011, an approach to smooth the introduction of the C language and its associated programming environment based on the use of both: iVProg and C from the very beginning. iVProg was used to find algorithmic solution to problems and then solve the same problems using C. Comparing the results from 2010 and 2011, we realized that students do improve their understanding about both algorithms and programming. In general, students had considerable difficulties on dealing with the C language compiler and finding the syntax errors. Therefore, their attention was deviated from the real problem, that is to find an algorithm solution to a problem, to these debugging issues. This fact could significantly decrease their motivation during the course. In the other hand, we believe that iVProg, by reducing the cognitive load during the process, could partially solve the motivational problem.

Figure 3 Answers of Q3

Q4) In general, solving exercises using iVProg is: 1. Very unpleasant; 2. Unpleasant; 3. Neutral; 4. Pleasant; 5. Very pleasant. The goal of this question was discover how comfortable students felt using iVProg while solving exercises. Figure 4 shows that 46.8% answered that iVProg is neutral, pleasant or very pleasant, but it was classified as unpleasant or very unpleasant by 53.2%. This was explained in the qualitative part of the questionnaire, where students reported some bugs that occurred while using iVProg. Qualitative questions are given next, and after some answers will be commented. Q5) Would you please specify the main advantages of programming using iVProg compared with C?; Q6) Would you please specify the main advantages of programming using C compared with iVProg?; Q7) Is there any resource you would like to find in iVProg?.

VII. CONCLUSION AND FUTURE WORK In this paper we presented iVProg, a visual programming system, as a tool that may enhance the understanding of algorithms and programming during introductory programming disciplines of CSET courses. Experiments indicated that it does improve understanding about algorithms and may improve the understanding of a programming language such as C when used in combination of it from the very beginning. The system is being fully redeveloped using a Software Product Line approach [15] in order to simplify its architecture and clean its code with the aim of facilitating its maintenance and evolution. This new development will turn possible the configuration of iVProg as a tool to support the learning of programming for several audiences: from kids to adults. Also, we intend to implement the automatic assessment support to students' assignments, which will provide immediate feedback for students and reduce teacher's workload. Therefore, teachers may focus their attention to the teaching process and programming techniques. Moreover, since iVProg is an iLM, it can be integrated to Moodle using iAssign [18] and all the functionalities related to assignments reports may be benefited from the automatic assessment feature.

14 12 10 8 6 4 2 0 1

2

3

4

5

Figure 4 Answers of Q4

In general, advantages of iVProg compared with C are related to the fact that in iVProg the programmer doesn´t need to be concerned with the syntax of the language, which is something very important while using C or any other programming language. Also, the use of the drag-and-drop functionality for the main programming commands, instead of typing all of them in C, kept users away of typos. Several answers stated that by using iVProg they could focus on the solution of the problem as an algorithm, instead of issues related to the language. Concerning the advantages of using C compared with iVProg, answers were related to the time to compile complex

ACKNOWLEDGMENT Leonidas O. Brandão is partially supported by FAPESP grant 2011/10926-2 and CNPq grant 550449/2011-6. REFERENCES

42nd ASEE/IEEE Frontiers in Education Conference

1107

[1]

[2]

[3] [4]

[5]

[6]

[7]

[8]

[9]

[10] Malan, D., J., Leitner, H., H.,"Scratch for budding computer scientists", Proceedings of the 38th SIGCSE technical symposium on Computer science education, Volume 39 Issue 1, March 2007, pp. 223-227 [11] Utting, I., Cooper, S., Kölling, M., Maloney, J., Resnick, M., “Alice, Greenfoot, and Scratch - A Discussion”, ACM Transactions on Computing Education (TOCE), Vol. 10 Issue 4, No. 17, November 2010 [12] Kamiya, R.R; Brandão, L. O. . iVProg - um sistema para introdução à Programação através de um modelo Visual na Internet. In: Anais do Simpósio Brasileiro de Informática na Educação, 2009. [13] Moura, J., Brandão, L.O. and Brandão, A.A.F. “A web-based learning management system with automatic assessment resources” Proceedings of ASEE/IEEE Frontiers and Education Conference, 2007, F2D_ 1-6. [14] Moodle: Modular Object-Oriented Dynamic Learning Environment http://moodle.org/ (acessed at 04/14/2012) [15] Dalmon, D. L., Brandão, A. F. F., Isotani, S., and Brandão, L. O., Work in Progress - A Framework for Building Interactive Learning Modules, Proceedings of ASEE/IEEE Frontiers in Education Conference, 2011, S3E-1 - S3E-2. [16] Alonso, F., Manrique, D., Martínez, L. and Viñes, J.M. "How Blended Learning Reduces Underachievement in Higher Education: An Experience in Teaching Computer Sciences", IEEE Transactions on Education, Vol. 54, No. 3, Aug. 2011. [17] Djenic, S., Krneta, R. and Mitic, J., "Blended Learning of Programming in the Internet Age", IEEE Transactions on Education, Vol. 54, No. 2, May 2011. [18] Rodrigues, P. A., Brandão, L. O., Brandão, A. A. F., “Interactive Assignment: a Moodle component to enrich the learning process”, Proceedings of Frontiers in Education, 2010, pp. T4F-1 - T4F-6.

Lahtinen, E., Ala-Mutka, K., Järvinen, H., "A Study of the Difficulties of Novice Programmers", ITiCSE '05 Proceedings of the 10th annual SIGCSE conference on Innovation and technology in computer science education, Volume 37 Issue 3, September 2005, pp. 14-18. Bennedsen, J., Caspersen, M., E.,"Failure Rates in Introductory Programming", ACM SIGCSE Bulletin, Volume 39, Issue 2, June 2007, pp. 32-36. Conway, M., J., Pausch, R., "Alice: Easy to Learn Interactive 3D Graphics", Computer & Graphics, Vol. 31, Issue 3, 1997, pp. 58-59. Maloney, J., Resnick, M., Rusk, N., Silverman, B., Eastmond, E. "The Scratch Programming Language and Environment", ACM Transactions on Computing Education, Vol. 10 Issue 4, No. 16, November 2010 Hazzan, O, Lapidot, T, Ragonis, N, "Teaching Methods in Computer Science Education", Guide To Teaching Computer Science: An ActivityBased Approach, 2011, pp. 91-118 Kölling, M., Introduction to Programming With Greenfoot: Objectoriented Programming in Java With Games and Simulations, Prentice Hall, 2009, pp. 190 Levy, R., B.,Ben-Ari, M., Uronen, P. A., "The Jeliot 2000 program animation system", Journal of Visual Languages & Computing, Volume 22, Issue 5, October 2011, pp. 375-384 Barnes, D., J.,Kölling, M., Objects First with Java - A Practical Introduction using BlueJ, Prentice Hall / Pearson Education, 2012, pp. 546 Rongas, T., Kaarna, A., Kalviainen, H., "Classification of computerized learning tools for introductory programming courses: learning approach," Advanced Learning Technologies, 2004. Proceedings. IEEE International Conference on , Vol, No #, 30 Aug.-1 Sept. 2004, pp. 678680

42nd ASEE/IEEE Frontiers in Education Conference

1108