A Method to Describe Student Learning Status for ... - IEEE Xplore

Download PDF
5 downloads 747 Views 987KB Size Report
Comment
help instructors to know student individual's learning status in C programming. SKP-based SLSD focus on the syntactic knowledge called Syntax Knowledge ...
2016 IEEE 30th International Conference on Advanced Information Networking and Applications

A Method to Describe Student Learning Status for Personalized Computer Programming e-learning Environment Yu Yan, Kohei Hara, Hiroto Nakano, Takenobu Kazuma and Aiguo He Department of Computer and Information Systems The University of Aizu Aizu-wakamatsu City, Fukushima, Japan Email:{d8171106,m5191112, m5181110, d8142103, aiguo}@u-aizu.ac.jp

Abstract—Personalized e-learning environment is desirable in computer programming education. An important issue on personalized e-learning environment is to know the learning status of each student. This article proposes a method, SKPbased Student Learning Status Description(SKP-based SLSD), to help instructors to know student individual’s learning status in C programming. SKP-based SLSD focus on the syntactic knowledge called Syntax Knowledge Point(SKP) extracted from program source code. Firstly, it gathers all syntactic knowledge that should be learned by the students by extracting SKP from the source code in teaching materials or exercises’ model answers. Then, for each student, it collects his learning activities on each SKP by extracting SKP from the source code the student have read or taught at lectures and wrote at exercises or tests. Finally, for each student, his understanding of each SKP is estimated based on the collected data. Student learning status can be described by his understanding of all SKPs. By SKP-based SLSD, the information used to describe student learning status can be more detail, be better-defined and better-handled by computer systems. We have also conducted experiments and proved that SKP-based SLSD is effective and feasible. Index Terms—personalized, e-learning, programming, education, syntax knowledge point

programming exercise to each student based on his inferred score and attitude toward the programming exercise. However, in above studies, there are following issues about the information used to describe student learning status: • It should be detail enough. Those collected in exercise problem level or programming concept level is too indistinct to obtain an overall and standardized measurement. • It should be well-defined to give more understandable learning status for objective analysis. • It should be well-designed to be treated by computer systems. This article proposes a method, SKP-based Student Learning Status Description (SKP-based SLSD for short), to give solutions to above issues. Here, we focus on the syntactic knowledge in the program source codes the students have read, wrote or been tested. All those syntactic knowledge which students should understand is extracted from the source code of sample programs in teaching materials or exercises’model answers. The syntactic knowledge is called Syntax Knowledge Point(SKP). Student learning activities on each SKP are collected and his understanding of this SKP is estimated based on the collected data. Then finally, student learning status is described by his understanding of all SKPs. By SKP-based SLSD, the information used to describe student learning status can be more detail, be better-defined and better-handled by computer systems. The remaining sections of this article describes the definition of SKP, the method for extracting SKP from C source code, the algorithm for SKP understanding estimation, the implementation of prototype system and experiment for verifying the correctness of SKP-based SLSD.

I. I NTRODUCTION E-learning environment is popular as an effective way to help students or instructors in learning or instructing[9]. Furthermore, personalized e-learning environment providing an dynamic and optimal learning process or adaptive learn contents to student according to his learning status. And the aim of personalized e-learning is enhancing learning ability of individual student[3], [6], [10]. With computer programming more important[8], personalized e-learning environment has also been applied to programming education. For example, Sasithorn Chookaew’s system[11] aimed at promoting students’conceptual learning on basic computer programming. This system offers a learning plan, a sequence of learning programming concept, for each student according to his percentage of incorrect answers(PIA) to each programming concept and relationship among programming concepts. Hiroshi Taguchi’s method[12] has also proposed a method to recommend an appropriate C-language 1550-445X/16 $31.00 © 2016 IEEE DOI 10.1109/AINA.2016.38

II. SKP- BASED S TUDENT L EARNING S TATUS D ESCRIPTION A. Syntax Knowledge Point(SKP) A computer program source code can be expressed by a tree structure, called syntax tree. For example, Figure 1 shows the syntax tree of the C source code below. #include 231

int sum(int a,int b){ return a+b; } void main(){ int a=1,b,m; b=a+2; m=sum(a,b); printf("m=%d",m); }

TABLE I E XAMPLES OF LEAF NODE TYPE NAME

symbol int float main void ( { , ; + user-defined variable name user-defined function name library function name

•

inner node function declaration function header declaration variable declaration compose statement expression statement jump statement assignment expression binomial expression function expression

Every node in the tree has its type (a symbol, an expression or a statement etc.) and performs a special syntactic role for its parent node. The combination of node n’s type Tn , its syntactic role R and its parent’s type Tp represents the syntactic meaning of the program code part corresponding to the node. That is, {Tn , R, Tp } represents a syntactic knowledge which needs to be learned by students. Here, we call the meaning expressed by the combination {Tn , R, Tp } Syntax Knowledge Point (SKP). For example, in Figure1, the pairs of nodes surrounded by red color circlets are SKPs corresponding to a variable declaration as below:

•

Variable declaration in compound statement Data type in variable declaration Variable name in variable declaration “,” in variable declaration “;” in variable declaration

type name DCF DCFH DCVM ST CMP ST EX ST JMP EX ASS EX BIN EX F

Since the role of node is only cared by its parent, node role can be decided by node parent as a local definition and a serial number is enough. Table III shows examples of node role used in function head declaration. TABLE III E XAMPLES OF NODE ROLE NAME

role return-type function-name ( )

B. SKP extraction algorithm SKP in a given C source code can be extracted by following steps. 1) Getting the syntax tree of the source code: The syntax tree of the C source code can be constructed by using an application program of C parser. 2) Naming node type and role: Node type T and node role R are only used for computer to automatically distinguish nodes with different syntactic meaning in syntax tree. Therefore, the names of T and R are given as below: •

function name and variable name have same type name. Table I shows examples of leaf node type name. For the inner nodes in the syntax tree, the type name is given based on the name of the source code part the subtree with the inner node as its root corresponds to. Table II shows examples of inner node type name. TABLE II E XAMPLES OF INNER NODE TYPE NAME

Fig. 1. Syntax tree of a C program

1. 2. 3. 4. 5.

type name int float main void ( { , ; + ID ID the name of the function

node role 01 02 03 04

3) Naming SKP: Every SKP is given by a node and its parent. SKP name has the format of {Tp R Tn }. Table IV shows examples of SKP function head declaration. C. SKP based student learning status Here we discuss a method to get the learning status of the given student by estimating his understanding of SKPs from his learning activity history.

For the leaf node in the syntax tree, its type name is the symbol which the node corresponds to in the source code. However, leaf nodes corresponding to user-defined

232

n = max(1, max((nca − nwa ), ncac ) + nr ∗ 0.6)

TABLE IV E XAMPLES OF SKP IN FUNCTION HEAD DECLARATION

Tp DCFH DCFH DCFH DCFH

R 01 02 03 04

Tn int ID ( )

SKP DCFH01int DCFH02ID DCFH03( DCFH04)

Y (k)i = 1.84/[(log (te /n))1.25 + 1.84]

Meaning return-type user-defined function name symbol ( in function head symbol ) in function head

te is the elapsed time since the last positive learning activity recorded. • Get A(k) and Y (k) A(k) = max(A(k)i |i = 1, NT ) Y (k) = max(Y (k)i |i = 1, NT ) Formula (3) comes from Takehiko Fukami’s model of forgetting[5] based on Ebbinghaus curve[4]. This model reveals the change of retention of memory on a certain event with time(in minutes) increasing from the happening of that event. Figure 2 shows the value of Y (k) with different n over time. The value of Y (k) is higher when n is bigger. And n is used to represent the retention of memory in the same time.

1) SKP set: Let K be the set of SKP that should be learned by the given student. K = {ki | i = 1, Nk }

(3)

(1)

Nk is the number of SKPs in K. K can be given by extracting SKPs from the source code of all sample programs in the teaching materials and programs as model-answer for exercises. 2) Student learning activity and learning history: For each given k in K, the student has following kinds of learning activity: • Read: A positive activity that a sample program source code which includes k has been explained by the instructor at a lecture or an exercise. • Correct-Answer: A positive activity that a program source code which should include k has been submitted as the answer of an exercise or examination problem, and k was used correctly. • Wrong-Answer: A negative activity that a program source code which should include k has been submitted as the answer of an exercise or examination problem, and k was not used or used incorrectly. Then student learning history for k is a list of learning activities recorded by a real-time learning support environment with time stamps. 3) Estimating understanding of SKP: Based on student learning history described above, for any k in K, the estimated understanding value of k at current time t, u(k, t), is given by formula (2). u(k, t) = (A(k) + Y (k))/2 (2)

Fig. 2. Value of Y (k) with different n

4) Student learning status: Based on above discussion, the current learning status S of the given student is his understanding of K. S = U (K, t) = {u(ki , t) | i = 1, Nk }

Here A(k) is obtained based on the accuracy rate of student’s answer and Y (k) is obtained based on the elapsed time since the last positive learning activity recorded, to consider the forgetfulness about k. The algorithm to get A(k) and Y (k) is as follows: • Give a set of time period T = {Ti |i = 1, NT } • For each Ti in T , get the following numbers of learning activities recorded in last Ti days: – nr : number of Read activities – nca : number of Correct-Answer activities – nwa : number of Wrong-Answer activities – ncac : number of last successive Correct-Answer activities • Get A(k)i for Ti A(k)i = nca /(nca + nwa ) • Get Y (k)i for Ti

(4)

III. I MPLEMENTATION OF P ROTOTYPE S YSTEM Based on an educational purpose C virtual machine[7], an prototype system was developed for the purposes below: • Verification of the algorithms discussed in Section II • Offering test tools for future SKP related study • Offering basic functions for personalized C programming learning support environment This system is developed by using Java technology to be used in Internet based application systems. Figure 3 is the architecture of this system. A Sample source code is a C program source codes in teaching materials or the model answers given by the instructors. A Submitted source code is a C program source code submitted by a student as an answer to a problem of exercise or test. In each submitted source code, code parts which cause compile error

233

CVStatement IterationStatement() : { CVExpressionEx e_i, e_c, e_r; CVStatementEx s, s_b; CVMSymbol ptl; CVMSymbol ptr; CVMSymbol sc1, sc2; CVMSymbol si; } { ( {si = new CVMSymbol(token);} "(" {ptl = new CVMSymbol(token);} e_c = Expression() ")" {ptr = new CVMSymbol(token);} s_b = Statement() {s = new CVWhileStatementEx(si, ptl, e_c, ptr, s_b);} | {si = new CVMSymbol(token);} "(" {ptl = new CVMSymbol(token);} [ e_i = Expression() ] ";" {sc1 = new CVMSymbol(token);} [ e_c = Expression() ] ";" {sc2 = new CVMSymbol(token);} [ e_r = Expression() ] ")" {ptr = new CVMSymbol(token);} s_b = Statement() {s = new CVForStatementEx(si, ptl, e_i,sc1, e_c, sc2, e_r, ptr, s_b);} ) {return s;} }

Fig. 3. Experimental system architecture

were removed so that the source code may not be a correct answer but has no compile error. SKP Extraction function extracts SKP information from the source codes and gives the results as Sample SKP and Submitted SKP, which then are used by SKP Learning Activity Recording function to create Learning History, which is used by SKP Understanding Estimation to give the learning status of the student.

As shown above, the extended definition adds Java codes to Doug South’s definition to create the syntax tree. In the syntax tree, every node is an object of Java class. In above example, CVMSymbol represents a symbol; CVExpressionEx represents an expression; CVStatementEx represents a statement; Especially, CVWhileStatementEx represents a WHILE statement and CVForStatementEx represents a FOR statement. The nodes of syntax tree are all subclass of CVMNode. For extracting SKP, every node implements following methods: • getNodeType() gives its node type name. • getChildRole(CVMNode c) gives the role of its child c • getSKPID() gives its SKP ID For each input of source code, this function gives the number of occurrences of each SKP type in the source code.

A. SKP Extraction The core part of SKP Extraction is a C-parser. The Java code of the C-paser is generated by using JavaCC (Java Compiler Compiler)[1] which compiles the C grammar definition file extended from Doug South’s contribution[2]. The following is an example of extending Doug South’s definition for WHILE statment and FOR statement. (Doug South’s definition) void IterationStatement() : {} { ( "(" Expression() ")" Statement() | "(" [ Expression() ] ";" [ Expression() ] ";" [ Expression() ] ")" Statement() ) } iExtended definitionj

B. SKP Learning Activity Recording This function works when the following events occurred to any student: • When a set of sample codes is shown to the student at a lecture or exercise, it creates a Read record for each SKP type name in the code set.

234

•

•

When the student had not submitted his answer for a given problem in a test or exercise after the submission deadline, it creates a Wrong-Answer record for each SKP type name in the model answer code of the problem. When the student has submitted his answer for a given problem in a test or exercise, it creates a Correct-Answer record for each SKP type name which appeared in both submitted code and its model answer and the number of occurrences in both code were almost same; and a Wrong-Answer record for each SKP type name which only appeared in the model answer or the number of occurrences in both code were too different.

C. SKP Understanding Estimation Based on the Learning History, this function can give an estimated understanding of each SKP at any given time. IV. E XPERIMENT In order to verify the correctness of SKP-based SLSD, experiments were carried out by using the prototype system. In author’s university, C programming is instructed for firstyear students as the most basic programming learning by a 15 weeks course named Introduction of Programming. This course was given from Mar. to Jun. in 2015. It includes 15 lectures and exercises in total and 12 lectures of them offer sample programs and 13 exercises that the students need to submit their answer program codes. These experiments were based the source codes of Introduction of Programming: sample programs, answer model programs and students answer programs. A. SKP extraction Figure 4 shows all types of SKP extracted from all C sample program source codes given at each lecture in the Introduction of Programming course. In Figure 4, the top row shows the number of lecture and the leftmost column shows the type name of SKP extracted and the colored cells show the appearance of each SKP type in each lecture. 216 kinds of SKP were successfully extracted from all the sample source codes. That means 216 kinds of SKP can represent all syntactic C programming knowledge which need to be learned by C beginners like first-year university students. This number of SKP is not so large to be handled by computer system. Figure 4 can give an overview of syntactic knowledge the students learn in each lecture. For example, from Figure 4 it can be found that the basic structure of C program and function printf() were taught at first lecture(lec01); Integer type variable declaration, assignment statement and function scanf() were taught at lec02; Double type variable declaration were introduced at lec03; IF statement, conditional expression and SWITCH statement were taught both at lec04 and lec05; FOR statement was taught at lec06; Array was taught at lec07; User defined function was taught at lec13; Therefore, the concepts in C program can be expressed by combination of SKP;

Fig. 4. SKP distribution diagram

235

Figure 4 also give a way to check the validity of teaching materials. For example figure 4 shows a problem in the course that Type Casting was taught in lecture 3 and had never been touched again in further lectures. B. Estimation of Understanding of SKP We selected exercise answer source codes submitted by 4 student individuals, s1 , s2 , s3 , and s4 . The term-end test scores of this course was s1 > s2 > s3 > s4 . By using the prototype experimental system, we extracted the submitted SKP from those source codes, created learning history for the 4 students and estimated their understanding of SKP over the whole term of the course. Here the set of time period T used in the estimation process was T = {21days, 120days}. Figure.5, Figure.6 and Figure.7 illustrate the understanding curve of several typical SKP of the 4 selected students over the course time. In each figure, the x-axis is the course time and y-axis is the understading value the given SKP. Figure.5 is

Fig. 6. Understanding curve of ST CMP04ST IF

application of IF statement and their understanding values are a little low than s1 . Figure.7 is the understanding curve

Fig. 7. Understanding curve of ST JMP01continue Fig. 5. Understanding curve of DCFH02main

of SKP ST JMP01continue, the continue statement in FOR and WHILE statement. ST JMP01continue was introduced at lec06 and, after 7 weeks, appears in sample programs of lec13 and lec14. Therefore, student are lack of learning time for this SKP. The understanding of this SKP is far smaller than the ones in Figure.5 and Figure.6. Figure.8 shows the learning status of the selected students on the 110th day, the final learning day of this course. The status is described by the understanding value of all the SKP needed to be learned by the students. The x-axis of Figure.8 is all kinds of SKP shown in the leftmost column of figure.fig:skpdistributiondiagram, the leftmost SKP is introduced at the first lecture and the rightmost SKP is introduced at the last lecture, and the y-axis is the understanding value of each SKP. It can be seen that s1 is the best learner of the 4 students, and the order of understanding

the understanding curve of SKP DCFH02main, the name of main function in function head declaration. DCFH02main is the most basic syntax element for constructing a C program. Because this SKP is included in every C source code and is touched in every lecture and exercise, it is a well understood SKP. Student understandings of this SKP show high value over the course period time. The reason that the understanding curve of s1 is low at the beginning of the course is that in one submitted source code of s1 , the name of main function was wrong. The reason that the understanding curve of s4 is low over the period time of the course is that s4 had not submitted all his answer source code for exercise. Figure.6 is the understanding curve of SKP ST CMP04ST IF, the declaration of IF statement. This SKP also appeared at every lecture and exercise since it firstly introduced at lec04. s1 ’s understanding of this SKP is very high. Other students had not completed or not submitted their answer source codes for some exercise problems which is needed to be solved by complicate

236

TABLE V L OW UNDERSTANDING SKP IN s4 ’ S LEARNING STATUS # 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36

Fig. 8. SKP-based student learning status

value is s1 > s2 > s3 > s4 , the same with the order of termend test scores. It also turned out from Figure.8 that if the SKP is introduced too late or is not repeated enough, its understanding value is lower. C. Estimation of Student’s weak point From student’s learning status, his weak points in program syntax can be estimated. For example, from s4 ’s learning status shown in Figure.8, the kinds of SKP which understanding value is less then 0.3 are shown in Table V. Since every SKP can be considered as a relation between a node and its parent in the syntax tree, the SKPs in Table V can be grouped based on the relation as shown in Figure 9. From Figure 9, it turned out that s4 has lack of understanding toward following concepts: • (as shown in a) of Figure 9): User defined function, return statements, double and int type arguments, type casting of variable, and variable array declaration with initializers. • (as shown in b) of Figure 9): Application of FOR statement and IF statement, library function sqrt(), type casting of expression. • (as shown in c) of Figure 9): Assignment expression used as the variable update part of FOR statement, and used in conditional expression.

SKP type EX BIN61( EX BIN62) EX V81( EX V82double EX V83) EX V01ID EX URY01 EX URY02EX CST DCV07EX URY EX BIN81( EX BIN82double EX BIN83) EX ASS03EX URY ST IF07ST EX EX BIN01EX F EX BIN01EX PDF ST FOR09ST EX EX ASS61( EX ASS62) EX CND01EX ASS DCFH01double DCAG01double ST JMP03EX V DCAG01int EX ASS02¯ DCFH01void EX BIN03EX F DCFH05, EX F01sqrt ST JMP03EX F ST FOR07EX ASS EX BIN03EX PDF EX BIN82int DCV07EX PDF ST EX01EX BIN DCV03EX BIN

Meaning ( in binomial exp. ) in binomial exp. ( in variable type casting double type casting of variable ) in variable type casting variable in exp. - ahead of unary exp. constant in unary exp. unary exp. as variable initializer ( in binomial exp. type casting double type casting in exp. ) in binomial exp. type casting unary exp. in assignment exp. Expression statement in IF statement Function call in binomial exp. Predefine in left side of binomial exp. Statement in FOR statement ( in assignment exp. ) in assignment exp. assignment exp. in conditional exp. Return data type (double) of function double type argument Variable in return statement int type argument ¯in assignment exp. Return data type (void) of function Function call in binomial exp. , in function decalaration Function call of sqrt() Function call in return statement Variable update in FOR statement Predefine in right side of binomial exp. int type in binomial exp. type casting Predefine as variable initializer Binomial exp. in exp. statement Binomial exp. in array declaration

be objective and obtained easily and easy to further application systems. In traditional personalized computer programming elearning environment, to get learning status of each student, instructors need to prepare quiz or test and score the submission of students. The proposed method can give learning status automatically and save the time of the instructors. Though the proposed method is only implemented and tested for C language, it can be used for other computer languages. We have the plan as below to solve the issues of the proposed method in future research:

V. C ONCLUSION This article proposed a Syntax Knowledge Point(SKP) based method to describe learning status of individual students for personalized computer programming e-learning environment. Because SKP is the minimum concept unit in computer programs that should be learned, it can be used to show student learning status in detailed level then concept-based description[11] and exercise-problem-based description[12]. SKP is extracted from program source code so that SKP extraction and SKP based learning status estimation can be realized by computer system. Therefore, the learning status can

•

•

237

SKP can only be used to check student’s understanding of knowledge of syntax. We are going to improve the parser in our prototype system to extract high level information for checking student’s understanding of algorithm implementation. SKP extraction is impossible for source codes with compile error. In current implementation of the proposed method, it needs to remove all parts which caused compile error from submitted source code to accomplish SKP extraction. This is a simple way to deal with uncompleted

[5] Takehiko Fukami. ebbinghauss forgetting curveand its model function. In The 78th Annual Convention of the Japanese Psychological Association, 2014. [6] Chin-Chung Tsai Judy C.R. Tseng Gwo-Jen Hwang, Pei-Shan Tsai. A novel approach for assisting teachers in analyzing student web-searching behaviors. Computers & Education, 51(2):926–938, Sep. 2008. [7] Aiguo He. C virtual machine for educational program visualization for beginners. THE IEICE TRANSACTION ON INFORMATION AND SYSTEMS, J198-D(10):1292–1300, October 2015. [8] C. M. Toyoshiro Nakashima. Analysis of source codes created by beginners in programming education. In Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing, volume 2, pages 774–781. Eighth ACIS International Conference, July 30-August 1 2007. [9] Deng S. Using learning styles to implementing personalized e-learning system. International Conference on Management and Service Science, 2011. [10] A. Dix S. Manning. Identifying students mathematical skills from a multiple-choice diagnostic test using an iterative technique to minimise false positives. Computers & Education, 51(3):1154–1171, Nov. 2008. [11] Dechawut Wanichsan Parames Laosinchai Sasithorn Chookaewa, Patcharin Panjaburee. A personalized e-learning environment to promote student’s conceptual learning on basic computer programming. Procedia - Social and Behavioral Sciences, 116:815–819, Feb. 2014. [12] SHIMAKAWA HIROMITSU ITOGA HIROYA YAMAMOTO TETSUO TAGUCHI HIROSHI, MOURI KOICHI. Programming training of students according to individual understanding and attitude. Information Processing Society of Japan (IPSJ), 48(2), 2007.

Fig. 9. Groups of s4 ’s low understanding SKP

•

•

source codes but will lower the accuracy of learning status estimation. A new way is needed to extract SKP from uncompleted source code for getting learning status with high accuracy. The usefulness of the proposed method needs to be confirmed in real e-learning environment with more number of students. We are going to develop an experimental system for test this method and find further issues. The proposed method, can be implemented for decreasing instructors‘job, such as, indicating the next contents that students need to learn according to their own situation. We will work on a personalized programming learning support to reveal each students‘learning situation for instructors and help individual student learn programming by smart indication. R EFERENCES

[1] JavaCC is a parser/scanner generator ..., https://java.net/projects/javacc [Online; accessed 10-August-2015]. [2] Doug South: C grammar definition for use with JavaCC, http://java.net/downloads/javacc/contrib/ grammars/C.jj, [Online; accessed 10-August-2013]. [3] Campo M. Casamayor A, Amandi A. Intelligent assistance for teachers in collaborative e-learning environments[j]. Computers & Education, 53(4):1147–1154, 2009. [4] H. Ebbinghaus. Memory: A Contribution to Experimental Psychology. New York by Teachers College, Columbia University, 1885.

238