Enhancing practice and achievement in introductory programming using an ADRI editor Dr. Sohail Iqbal Malik Department of Computer Science Buraimi University College Buraimi, Oman
[email protected]
Abstract— Practice is considered an important step in grasping the precise concepts of computer programming for novices. An editor plays an integral part in promoting practice and acquiring programming skills. Most of the programming editors available in the market promote programming shortcut (to convert the problem statement directly into a computer program). In this study, a new editor was introduced in the introductory programming (IP) course based on the ADRI approach which discourages programming shortcut. Moreover, it focuses on paying equal attention to programming knowledge (syntax & syntax) and problem solving strategies. The results show improvement in achieving students’ learning outcomes and positive impact on attrition rate. Students (treatment group) who used the ADRI editor show more satisfaction on using the program development environment compared to the control group.
problem statement (Problem StatementCode) as shown below in Fig. 1.
Keywords—introductory programming; ADRI approach; novice programmers; problem solving
Fig. 1: Programming Shortcuts (adapted from reference [3])
I.
INTRODUCTION
Learning to program is one of the challenges for a substantial number of novices in computer science and related disciplines. The key challenge for novices in learning programming is to acquire different sets of skills (syntax, semantics and problem solving) at the same time. Moreover, in programming courses, students are required to study theoretical concepts of programming domain and practice these concepts while designing and developing programs from a given problem statements. As a result, the whole process pushes the cognitive processes of many novice programmers into overload [1]. Reference [2] stated that ‘the old saw that practice makes perfect has solid psychological basis’ (p. 18). Practice is an important component in learning process of programming. If novices want to become expert programmers then they should practice, practice and practice programming problems [2]. It is generally accepted that novice programmers need 10 years of experience to become an expert programmer [2]. Reference [3] discussed that in most cases novice programmers attempt to write code directly to solve the
Problem solving phase Problem statement
Algorithm
Shortcut? Code Implementation phase
The traditional approach used in the learning process of the introductory programming (IP) course also promotes this practice. All the programming examples discussed during the lectures and exercises solved in the lab sessions are based on this practice. The task in the examples and exercises is to convert the problem statement directly into a computer program which promotes students taking programming shortcuts. Most programming editors encourage this practice and promote programming shortcuts [4]. Reference [2] suggested that, in teaching programming, it is important to cover models of program design, data representation, data structure, and problem domain. If the instructor, during the teaching process, does not cover any one of these models, ‘the students will make up their own models of dubious quality’ (p. 21). The instructor should cover initial concepts, models and rules of programming at a simple level and ‘only expand and refine them as the student gains experience’ (p. 21). Moreover, he explained that practice is an important component in programming courses and designing practice questions for problem solving is difficult as compared to designing programming syntax questions.
II.
RELATED WORK
Reference [5] reviewed supporting software tools used in teaching programming. He categorized supporting tools into four groups as follows: Tools with a reduced development environment Example-based environments Tools based on visualization Simulation environments A. Tools with a reduced development environment help novices to overcome the obstacles of understanding the commercial development environments. These commercial environments are developed for professional programmers. BlueJ [6], DrJava [7], AnimPascal [8] and THETIS [9] are a few examples of reduced development environments. B. Example-based environment helps programmers to use previously written or learned programs to solve new problems. This approach is also reflected in the software reuse area. The four examples of example-based environment are ELM-PE (Episodic Learner Model—the programming environment) [10], ELM-ART (Episodic Learner Model—the Adaptive Remote Tutor) [11], WebEx (short for Web Examples) [12] and Javy [13]. C. Tools based on visualization help students in achieving learning because ‘human beings are good in processing visual information’ (p.135) [14]. Visualizations have been used in learning for a long time. It helps students to learn abstract and complex concepts of programming [15]. ANIMAL (A New Interactive Modeler for Animations in Lectures) [16] and LEONARDO [17] are two examples of visualization tools used for teaching programming.
Research questions 1, 2 and 3 are explored in sections IV. and VI. respectively. IV.
ADRI BASED COURSE DESIGN
The ADRI based approach is introduced in a new design of the introductory programming course. A. The ADRI Model The ADRI (Approach, Deployment, Result, and Improvement) model is a well-known quality assurance model which is also used extensively in the education and business sectors [20]. The ADRI model is used by Australian and New Zealand universities for quality audit processes. Moreover, it is used by the Australian Business Excellence Framework and New Zealand Business Excellence Foundation in their Business Excellence Awards to evaluate quality [21].The ADRI model was based on the Plan-Do-Check-Act (PDCA) model developed by Walter Shewhart which was then updated by Deming into the Plan-Do-Study-Act (PDSA) model [22]. The PDSA cycle was used to improve a new online health policy course [23]. The ADRI model was used in a ‘360degree evidence-based approach for curriculum enhancement’ by the Australian Universities Quality Agency (AUQA) for underpinning the quality audit process [24]. Reference [25] explained the four approaches of ADRI (Fig. 2) as follows: (1) ‘Approach - Thinking and planning (2) Deployment - Implementing and doing (3) Results - Monitoring and evaluating (4) Improvement - Learning and adapting’
D. Simulation environments help students to understand programs and their effects by means of visualizations in an imaginary world. ‘Students observe (and sometimes take active part in) this imaginary world, whose inhabitants behave according to the execution of program instructions. So program actions are reflected in inhabitants’ actions’ (pp. 138139) [14]. Alice [18] and Karel the Robot [19] are well known examples of simulation environments. III.
RESEARCH QUESTIONS
The study focuses on determine the impact of the ADRI editor on students learning process in the introductory programming. The research questions being addressed in this study are: RQ1: How the ADRI editor promotes practice in the introductory programming? RQ2: what is the impact of the ADRI editor on the students learning outcomes in the introductory programming? RQ3: what is the impact of the ADRI editor on the students’ satisfaction in the introductory programming?
Fig. 2: Four Stages of the ADRI Model (adapted from [26])
The first stage (approach) of the ADRI model generally consists of thinking and planning about tasks [27][28][29]. For the IP course, this stage focuses on problem solving strategies such as flowchart and pseudo-code [4].
The second stage (deployment) provides a platform to execute or implement tasks [27][28][29]. For the IP course, this stage focuses on syntax and semantics and the novices write a code for a given problem statement by using any programming language [4]. The third stage (result) refers to the output or findings as consequences of the approach and deployment stages [27][28][29]. For the IP course, this stage shows the input, output and the process used to solve the problem statement. Moreover, it provides basic information about the common syntax and semantic errors occur in the programming language [4]. The fourth stage (improvement) refers to the conclusions drawn from the result stage [27][28][29]. For the IP course, this stage focuses on modifying the problem statement so that novices can get exposure to different programming constructs [4]. B. ADRI based introductory programmin course materials The course materials used in the IP course were redesigned to take advantage of the benefits of using the ADRI model. Learning resources were developed based on the four stages of the ADRI model. The ADRI editor was prepared to facilitate the learning process. The changes to the programming examples used in lectures and the programming problems set as laboratory exercises are depicted in an appendix A. C. ADRI based Editor An editor was developed to support novices using the ADRI based approach. It has some distinguishing features compared to other generally available programming editors. It has been specifically designed to emphasize the four stages of the ADRI approach with separate interfaces for each stage. The exercise questions are embedded in the editor. Moreover the exercise questions can be accessed directly related to particular topics of the introductory programming course. The ADRI-based editor is a simple editor based on the Java language. Java is a platform independent language [30] so it is compatible with different operating systems.
Fig. 3 shows one of the user interfaces of the ADRI-based editor. It consists of six menus (File, Edit, ADRI, Question, Topic and Execute).Each menu has further sub-menus which provide further functionality for the novices. There is a separate interface for each stage of the ADRI approach. Each interface shows the stage name (e.g. in fig. 3 ADRI: Approach (Problem solving strategies)) at the top of the page. Then it shows a problem statement along with the information of the topic to which it belongs in the IP course. For example, in fig. 3, the problem statement belongs to Algorithms & Pseudocode topic. A text area is provided for the students to write the solution of a given problem statement which can be written in either pseudo-code or a programming language. The file menu has four sub-menu options which are New, Open, Save and Exit. The edit menu provides three sub-menu which are cut, copy and paste. These menus provide basic functionality to the users. The ADRI menu has four sub-menu options reflecting each stage, Approach, Deploy, Result and Improvement. The first sub-menu option, Approach, deals with problem solving strategies. In this sub-menu, novices focus on writing pseudocode and drawing a flowchart for a given problem statement. The interface of the Approach sub-menu shows title, flowchart button, topic, problem statement and text area. The Edraw software is connected with the editor to assist with drawing the flowchart; it is invoked by pressing the flowchart button as shown in fig. 4. The second sub-menu option Deploy focuses on programming knowledge (syntax and semantics). In this sub-menu, novices practice how to convert pseudo-code or flowchart into a computer program by using a programming language. This stage emphasizes on syntax and semantics of the programming language. The interface of the Deploy submenu displays title, topic, problem statement and text area. The third sub-menu option, Review, displays the expected output of the problem statement. The interface of the Review sub-menu depicts title and expected output. The last sub-menu option, Improve, also deals with programming knowledge (syntax and semantics). The problem statement of the Approach stage is updated or modified in such a way that it provides more practice of different programming constructs to the novices. The interface of the Improve sub-menu displays title, modified problem statement and text area.
Four sub-menu options for ADRI approach Fig. 4: Edraw interface for drawing flowcharts Fig. 3: Four stages of the ADRI editor
The Question menu consists of two sub-menus which are Next and Previous as shown below in fig. 5. The novices can use these sub-menus to move back and forth easily from one exercise question to another. When the question is changed by using the sub-menus options, it is updated for all four stages of the ADRI approach.
Next and Previous sub-menus for changing questions
Fig. 5: Question Menu with Next and Previous sub-menus
The Topic menu has seven sub-menu options which are the topics included in the IP course (e.g. Basic elements of C++) as shown below in fig. 6. Students can go directly to the exercise questions related to a particular topic of the IP course by using these sub-menus. It saves their time and they can quickly search the exercise questions.
Topic menu showing all introductory programming topics
Fig. 6: Topic menu with introductory programming topics as sub-menu
The Execute menu provides one sub menu option which is the DOS prompt. The novices can use this option to compile their programs. The main purpose of the editor is to facilitate the learning process based on the four stages of the ADRI approach. Its separate views for each of the four stages of the ADRI approach discourages students for taking programming shortcuts as shown in fig. 1. In contrast, it encourages novices to practice a set of different programming skills for each problem statement. It provides some unique features, in particular all the questions are embedded in it and students can choose questions by topic (RQ1). D. Implementation of ADRI based approach The IP course offered in semesters 1 & 2, 2014-15 included the ADRI approach. The new approach was introduced in lectures and lab sessions. All the programming examples and problems were presented using this new approach (see appendix A). The programming examples that
were discussed in lectures and programming problems that were presented in lab sessions were described in terms of the ADRI approach. Students used the ADRI based editor in lab sessions to solve the programming problems. The students can easily access all the programming problems for all the introductory programming topics through the Topic and Question menus of the editor. Moreover the editor’s separate view for each of the four stages of the ADRI approach helped them in practicing and acquiring different skills required for the programming domain. The new approach was also incorporated into assessment tools. The assignment tasks were based on four stages of the ADRI approach. Students had to write pseudo-code, draw flowcharts, write an underlying process to solve the problem statement and produce a program (using C++) for the given problem statement. In the exam, different types of questions were given which covers programming knowledge (syntax and semantics) and problem solving skills such as write pseudo-code, draw flowchart, write a program by using C++ language for a given problem statement. V. METHOD A. Data Collection The final grades of the IP course for the last four semesters (1 & 2, 2013-14 and 1 & 2, 2014-15) were collected and analyzed. Out of these four semesters, in two semesters (1 & 2, 2014-15), the course was offered with the ADRI approach and the remaining two semesters (1 & 2, 2013-14), the course was offered with the traditional approach. The course was offered in a normal setting to the treatment groups without any special arrangements, it means the students did not know that it was an experimental study. The population being surveyed consists of IP students in semester 1, 2013-14 (control group) and semester 1, 2014-15 (treatment group) at Buraimi University College, Oman. The control group consisted of 99 students including 82 female (83 %) and 17 male (17%) students. The treatment group consisted of 118 students including 89 female (75 %) and 29 male (25%) students. Buraimi University College offers degree programs in two shifts, morning (8am – 2pm) and evening (2pm-9pm). The morning shift is exclusively offered for female students whereas the evening shift is offered for both male and female students. This practice is not very common in other higher education providers in Oman. Consequently, we attract more female students compared to male students. This affords us a unique opportunity to undertake this investigation with the majority of participants being female, a rare situation in IT. Surveys The questions in both surveys were the same. Each survey consists of 38 questions which comprises of closed and open ended questions. This study is a part of my PhD work and
analyzed only subset of questions from both surveys to explore the third research question. Other questions of the surveys will be analyzed in other research study. The survey has three parts. The first part covers demographic questions related to students’ prior experience in programming, their study major, degree level and gender. The second part covers questions which are mainly related to the course content. All the topics covered in this course have been included. A five-point Likert scale is used, from very difficult to learn (1) to very easy to learn (5). There is one open-ended question in this section which allows respondents to offer any other relevant comments on the course content. This part of the survey is based on an existing survey developed by reference [31]. The third part of the survey focuses on questions related to the teaching materials of the course. Again, five-point Likert scales are used; questions related to the learning situation use a scale of never(1) to always (5) used in this mode of study, and those relating to the teaching materials use a scale of useless material (1) to very useful material (5). Again, there is one open-ended question which gives respondents the option to comment on any other relevant issues related to teaching and learning aspects of the course which are not otherwise covered by the survey instrument. All the closed ended questions are taken from the survey developed by reference [31]. The surveys data was analyzed and compared by using mean and Mann-Whitney U test methods. Mann-Whitney U test is a non-parametric test which is used to compare the difference between two independent groups. VI. RESULTS A. Failure, dropout and attrition rates (RQ2) Fig. 7 shows the failure, dropout and attrition rates for the IP course for four semesters. As mentioned above, out of these four semesters, in two semesters (1 & 2, 2014-15), the course was offered with the ADRI approach and the remaining two semesters (1 & 2, 2013-14), the course was offered with the traditional approach.
The failure rates for two semesters (semester 1 & 2, 201314) offered with the traditional approach are 13.1% and 15.2%. On the other hand, the failure rates for other two semesters (semester 1 & 2, 2014-15) offered with the ADRI approach are 11.1% and 9.1%. It shows that the ADRI approach has had a positive impact on the performance of the students and failure rates have decreased. Likewise, the dropout rates are 12.7% and 11.4% for those semesters offered with the traditional approach, and are 1.5% and 3.0% for those semesters offered with the ADRI approach. It clearly shows that the ADRI approach has had a significant impact on the dropout rates. The overall impact of the attrition rate on the IP course by summing the failure and dropout rates for each semester are also calculated and examined. For semesters offered with the traditional approach, the attrition is 25.8% and 26.6%. On the other hand, the attrition rate in the semesters offered with the ADRI approach is 12.6% and 12.1%. Overall, the attrition rates are less than half in the semesters offered with the ADRI approach compared than the semesters offered with the traditional approach. It clearly shows the ADRI approach has a significant impact on not only the performance of the students (as represented by the improved failure rate) but also the persistence of students (represented by the significantly reduced dropout rate) compared to the traditional approach. B. Analysis of Surveys responses (RQ3) Table 1 shows the comparison of means of control and treatment groups for the set of programming concepts questions. Table 1: Mean comparison of programming concepts questions
Programming Concepts Questions
Using the program development environment Understanding problem solving strategies Understanding programming structures Learning the programming language syntax Designing a program to solve a certain task Dividing functionality into procedures Compiling and executing programs Finding bugs in my own program
Mean Values Control Treatment Group Group 2.8
3.17
2.55 2.53 2.75 2.78 2.85 2.93 2.77
3.02 2.85 3.09 2.86 3.16 3.15 3.19
It is clearly evident from table 1 that there is an improvement in the mean values of the treatment group compared to the control group. In general, the treatment group finds it easier to learn the programming concepts. The Mann-Whitney U test reveals that there are some areas such as Using the program development environment (p=.029), Understanding problem solving strategies (p=.001), Understanding programming structures (p=.018), Learning the programming syntax (p=.019), and Finding bugs in my own program (p=.017) which have significant differences at Fig. 7: Failure, dropout and attrition rates
the level of p
