An Investigation into Student Characteristics Affecting Novice Programming Performance Nelishia Pillay School of Computer Science, Pietermaritzburg Campus, University of KwaZulu-Natal, Pietermaritzburg, KwaZulu-Natal, South Africa
[email protected]
Abstract Novice programmers usually experience difficulties when programming for the first time. The main aim of the study presented in this paper is to identify those characteristics that negatively effect procedural programming performance, so that additional support can be provided in the instruction of programming courses for students possessing these characters. Investigations were conducted at two South African tertiary institutions. At both institutions a first course in Java programming, focussing on procedural programming aspects was used for purposes of the study. The characteristics investigated were the student’s problem solving ability, gender, learning style, first language and previous computer experience. The study revealed that a student’s problem solving ability and first language definitely have an impact on his or her programming performance. Keywords Novice programming performance, student characteristics, procedural programming. 1. Introduction Students usually experience difficulties when learning to program for the first time. This paper reports on a study conducted to identify the relationship between student characteristics and performance in a first course in procedural programming. Once the characteristics that negatively effect programming performance have been determined future extensions of this study will address developing methodologies to assist students possessing these characteristics to overcome their learning difficulties. Section 2 summarises previous work in this area. The methodology employed is presented in Section 3. Section 4 discusses the results obtained.
Vikash R. Jugoo Department of Computer Science, Mangosuthu Technikon, Umlazi, KwaZulu-Natal, South Africa
[email protected]
2. Previous Work Previous studies relating student characteristics to programming performance have examined the affect of gender, previous computer experience, the students’ mother tongue, learning style and the students’ performance in other problem solving courses, such as Mathematics and Science, on programming performance. Studies conducted by Byrne et al. [2] and Chumra [4] indicated that gender has no effect on programming performance. Similarly, Allert [1] and Byrne et al. [2] have found that previous computer experience did not in any way influence programming performance. However, students who had previous programming experience tended to perform better than other student in the study reported in [2]. The study conducted by Byrne et al. revealed that a student’s programming performance is not in any way effected by the fact that his/her mother tongue is different from the language of instruction, i.e. English. On the other hand results reported by Chumra [4] indicted that students with poor reading comprehension performed poorly in the programming course. According to both Byrne et al. [2] and Chumra [4] there is a strong positive correlation between students’ performance in a programming course and their performance in other problem solving courses such as Mathematics or Science. The Kolb learning inventory was employed by Byrne et al. to ascertain student learning styles and the Felder-Silverman inventory was used by Allert [1] and Thomas et al. [6] for this purpose.
While the study conducted by Byrne et al. [2] revealed that there was no correlation between programming performance and learning style, both Allert [1] and Thomas et al. [6] reported that reflective and verbal learners were found to perform better in programming courses than active and visual learners.
A large percentage of students in both the studies were not first language English speakers, hence tutors were available to help students whose did not understand the questionnaire. In the second study checks were built into the questionnaire to determine whether students experienced difficulties in interpreting the questions.
3. Methodology This section describes the methodology employed to determine which characteristics negatively effect programming performance in a first course in procedural programming at South African tertiary institutions. The characteristics investigated are gender, students’ mother tongue, previous computer experience, learning style and problem solving ability. In order to determine the relationship between these characteristics and programming performance two studies were conducted, the first at University of KwaZulu-Natal (Study 1) and the second at Mangosuthu Technikon (Study 2). In both studies the programming course used was a first course in Java programming covering procedural programming aspects. The language of instruction in both studies was English. The class size in the first study was 67 and in the second 30. Students involved in both studies were from a variety of cultural backgrounds and English was not the first language for 46% of the students in Study 1 and all of the students in Study 2. Thus, only the first study focussed on determining whether there is a correlation between a student’s first language and programming performance. All of the students in the first study were computer literate as they completed a three week course in Computer Literacy prior to the programming component of the module. As all the students in the first study were computer literate the relationship between previous computer experience and programming performance was only examined in the second study. In Study 2 70% of the students were not computer literate. Sixty seven percent of the class in the first study and 43% of the students in the second study were male. In the first study the students’ performance in Mathematics IS1 (a first year Mathematics course) was used as an indicator of the students’ problem solving ability and in the second study Development Software 1 (a course focussing on problem solving and the development of algorithms) was used for this purpose. A questionnaire consisting of 18 statements presented by Clark [3] as the Kolb Learning Style Indicator was administered to students.
These checks indicated that the interpretation difficulties experienced by students were not significant and thus were not taken into consideration when analysing student feedback. An analysis of the responses to the questionnaire revealed that in Study 1 40% of the students were Accomodators, 30% Assimilators, 20% Divergers, and 10% Convergers while in Study 2 40% were Accomodators, 10% Assimilators, 30% Divergers and 20% Convergers. 4. Results and Discussion In both Study 1 and Study 2 two marks were used to determine the relationship between the characteristics outlined in the previous section and programming performance, namely, a class mark (Mark 1), i.e. a mark reflecting programming performance during the semester, and an exam mark (Mark 2). Table 4.1 and Table 4.2 list the statistical data for the gender comparison in the first and second studies respectively. Mark 1
Mean
Mark 2
M
F
M
F
46
36
41
33
0.2544 0.3894 p Table 4.1: Statistical Data for the Gender Comparison for Study 1 Mark 1
Mean
Mark 2
M
F
M
F
60
56
58
54
0.4114 0.3872 p Table 4.2: Statistical Data for the Gender Comparison for Study 2 In both studies the male students appear to have performed slightly better than the female students. However, t-tests conducted with significance levels of 0.05 and 0.1 have indicated that this difference is not statistically significant.
This is consistent with the results reported by Byrne et al. [2] and Chumra [3]. For South African students, the results obtained with respect to the mother tongue of students being different from that of the instruction of the course are different from those presented by Byrne et al. [2] and similar to those reported by Chumra [4]. From Table 4.3 it is evident that students whose first language is the same as the language of instruction, namely, English, have performed better than those students whose mother tongue is not English. In order to test the significance of this result t-tests were conducted. As illustrated in Table 4.3 this result is statistically significant at both levels 0.1 and 0.05 for the course mark, and 0.1 for the examination mark. Mark 1
Mark 2
English 1st Mean
49
English 2nd
English 1st
English 2nd
36
43
32
Rho
Rho
Mark 1
Mark 2
0.4085
0.28562
0.0020 0.0329 p Table 4.4: Correlation Coefficients for Study 1 There is a positive correlation between the students’ problem solving ability and programming performance for the course mark for Study 1. The correlation between the student’s exam mark and problem solving ability is slightly weaker for this study. Both correlations were found to be significant at the 0.05 and 0.1 levels of significance. The correlation coefficients and corresponding p values listed in Table 4.5 indicate that a stronger correlation between students’ programming performance and problem solving ability was found for Study 2. The correlation for both these marks are statistically significant at a 0.05 and a 0.1 level of significance.
Mark 2
0.61842
0.70072
0.0010 0.0001 p Table 4.5: Correlation Coefficients for Study 2 As in the studies conducted by Byrne et al. [2] and Chumra [4], this study has also revealed that there is a positive correlation between students’ problem solving ability and programming performance. Table 4.6 tabulates the statistical data for the comparison between previous computer experience and programming performance for Study 2. There is a very small difference in the mean marks for computer literate students and those students who have not used a computer prior to the course. The ttests conducted have indicated that these differences are not significant. This is consistent with the results obtained by Allert [1] and Byrne et al. [2]. Mark1
0.0039 0.1061 p Table 4.3: Statistical Data for the First Language Comparison for Study 1 Table 4.4 lists the Pearson correlation coefficients and corresponding p values for the relationship between Mathematics IS1 and programming performance for the first study.
Mark 1
Mean
Mark 2
Comp. Exp.
No Comp. Exp.
Comp. Exp.
No Comp. Exp.
60
54
59
54
0.3438 0.3304 p Table 4.6: Statistical Data for the Previous Computer Experience Comparison for Study 2 In both Study 1 and Study 2 the majority of the students were found to be Accomodators. Table 4.7 displays the mean course mark and exam mark for each learning style category for Study 1 and Table 4.8 lists this data for Study 2. Mark 1
Mark 2
Accomodators
49
49
Assimilators
50
52
Convergers
51
47
35 33 Divergers Table 4.7: Mean marks for each learning style for Study 1
Mark 1
Mark 2
Accomodators
59
57
Assimilators
62
45
Convergers
57
54
59 56 Divergers Table 4.8: Mean marks for each learning style for Study 2 The Duncan Multiple Range Test was applied to the means in both studies. In Study 1 this test revealed that there is a difference in programming performance between Assimilators and Divergers. From Table 4.7 it is evident that Assimilators have performed better than Divergers. Furthermore, the highest mark obtained for both the course mark and the exam mark (96% and 95% respectively), was obtained by an Assimilator. The better performance by Assimilators could possibly be attributed to the fact that there lecturer taking the course is an Assimilator. According to Kuri et al. [5] the teachers learning style usually influences his/her method of instruction. In Study 2, however there appeared to be no correlation between learning style and programming performance. This study has revealed that while students with a particular learning style may perform well in a programming course, like in Study 1 and the studies conducted by Allert [1] and Thomas et al. [5], students’ learning style cannot be used as a general indicator of programming performance. 5. Conclusion and Future Work This study indicated that performance in a procedural programming course is gender independent. Similarly, computer literate students did not perform significantly better than those students that did not use a computer prior to the programming course. Students whose first language is not the same as that used in the instruction of the course did not perform as well as the rest of the class. Methodologies for providing these students with additional support will be investigated. The study revealed a positive correlation between the students’ problem solving ability and programming performance. For purposes of the study presented in the paper students’ performance in Mathematics and other problem solving courses was used as a measure of their problem solving ability.
Future extensions of this study will include deriving a formal measure to ascertain problem solving ability. Furthermore, methodologies for assisting students to improve their problem solving abilities will also be developed. In Study 1 Assimilators tended to perform better than Divergers while in Study 2 students’ learning style appeared to have no effect on their programming performance. Thus, a general conclusion with respect to learning style cannot be drawn from this study. Additional experiments will be conducted to perform reliability tests for the learning style inventory as employed in[1] and using different learning style inventories such as the Felder-Silverman Inventory. Instructional strategies to cater for the different learning styles will also be investigated. 6. References [1]
Allert J., Learning Style and Factors Contributing to Success in an Introductory Computer Science Course, in Proceedings of IEEE International Conference on Advanced Learning Technologies, 2004, pp.385 -389, IEEE Computer Society, September 2004.
[2]
Byrne P., Lyons G., The Effect of Student Attributes on Success in Programming, in SIGCSE Bulletin - inroads - Proceedings of ITiCSE 2001, Vol. 6, No. 1, pp. 49 - 52, ACM Press, 2001.
[3]
Clark D., Learning Styles - Or, How We Go From the Unknown to the Known, http://www.nwlink.com/~donclark/hrd/styles.ht ml, 2000.
[4]
Chumra G.A., What Abilities are Necessary for Success in Computer Science? In SIGCSE Bulletin - inroads, Vol. 30, No. 4, pp. 55a - 58a, ACM Press, December 1998.
[5]
Kuri N. P., Truzzi O. M. S., Learning Styles of Freshman Engineering Students, http://www.ineer.org/Events/ICEE2002/Proce edings/Papers/Index/o001-006/o003.pdf, 2002.
[6]
Thomas L., Ratcliffe M., Woodbury J., Jarman E., Learning Styles and Performance in the Introductory Programming Sequence, in SIGCSE Bulletin - inroads, Vol. 34, No. 1, pp. 33-37, ACM Press, 2002.
