A Computerized Approach to Diagnosing Student Learning Problems ...

H. C. Chu et al. / Asian Journal of Health and Information Sciences, Vol. 1, No. 1, pp. 43-60, 2006

A Computerized Approach to Diagnosing Student Learning Problems in Health Education HUI-CHUN CHU1, GWO-JEN HWANG1,*, JUDY C. R. TSENG2, AND GWO-HAUR HWANG3 2

1 Department of Information and Learning Technology, National University of Tainan, Taiwan Department of Computer Science and Information Engineering, Chung Hua University, Taiwan 3 Information Management Department, Ling Tung University, Taiwan

ABSTRACT With the rapid progress of computer technology during recent years, researchers have attempted to develop more effective programs for testing and improving student learning performance. However, in conventional testing systems, students merely obtain a score based on their test results, and are given no direction regarding how to improve their learning performance. Students would benefit from ways of analyzing test results and being provided with learning suggestions. This investigation presents a learning diagnosis approach for providing students with personalized learning suggestions by analyzing their test results and test item related concepts. Based on this approach, a testing and diagnosis system is implemented on computer networks. Experimental results on a nutrition course have demonstrated the feasibility of this approach in enhancing students in their learning performance, making it highly promising for further study. Key words: learning diagnosis, health education, nutrition course, concept-effect relationship.

1. INTRODUCTION In recent years, the issues concerning children's concepts of health have attracted researchers from several fields, including medicine, nutrition and education. It was indicated in (Piko & Bak, 2006) that, children of 8 to 11 years old have considerable knowledge about health, illness and disease risks. In addition, they found that children hold positive attitudes toward health and health promotion, which implies the need of proper guidance while teaching health-related courses. With recent rapid advances in computers and communications technology, researchers have attempted to utilize computer network technology for research on education, including the development of computer-aided tutoring and testing systems (Antao, Brodersen, Bourne, & Cantwell, 2000; Chou, 2000). In conventional testing systems, students are assigned a score or grade as a test result to represent their learning status. This approach allows students to know their scores (or grades) in reference to their learning status, but means that students might be unable to improve their learning status without further guidance. Therefore, researchers have proposed a concept-effect model to represent the prerequisite relationships among concepts in a course (Hwang, 2003; Hwang, Hsiao, & Tseng, 2003). The model has been demonstrated to be useful for helping teachers *

Corresponding author. E-mail: [email protected]

43

H. C. Chu et al. / Asian Journal of Health and Information Sciences, Vol. 1, No. 1, pp. 43-60, 2006

to detect student learning problems; however, previous experiences also showed the difficulty of applying the model without the assistance of a fully automatic system. This investigation proposes a fully computerized approach to diagnosing student learning problems for courses containing a large number of concepts. Furthermore, a testing and diagnosis system is implemented based on the new approach. To evaluate the efficiency of the new approach, some experimental results on a nutrition course are provided to demonstrate how the new approach can help students to improve their learning performance.

2. BACKGROUNDS AND RELEVANT RESEARCHES In the past decade, many computer-assisted tutoring systems have been developed. For example, a system which can assist in organizing system knowledge and operational information to enhance operation performance was proposed by Vasandani and Govindaraj (1991, 1995); moreover, a system that can automatically determine exercise progression and remediation during a training session based on past student performance was presented by Gonzalez and Ingraham (1994). Additionally, various techniques and tools for developing intelligent tutoring systems have been proposed. For example, the neural networks technique has been employed to model student behaviors in the context of an intelligent tutoring system (Harp, Samad, & Villano, 1995). Furthermore, planning methods, consistency enforcement, objects and structured menu tools have been employed to construct intelligent simulation-based tutors for procedural skills (Rowe & Galvin 1998). Additionally, a method for detecting the on-line status of students was proposed by Hwang (1998). Finally, a multi-agent systems (MAS) approach has been proposed to establish an interactive intelligent tutoring system (Giraffa, Mora, & Viccari, 1999). Clearly, the development of intelligent tutoring systems and learning environments has become a key issue in both computer science and education recently (Ozdemir & Alpaslan, 2000). In the meantime, researchers have shown that paper-administered and computer-administered tests are equivalent in terms of testing quality, encouraging the development of computer-based testing systems and relevant techniques (Olsen, Maynes, Slawson, & Ho, 1986). For example, Wainer (1990) proposed a form of computerized adaptive testing, which applies some forecasting methodologies to shorten the test without compromising accuracy; Fan, Tina, and Shue (1996) presented a system that is capable of changing the numeric part of test items during testing to prevent students from memorizing the answers. Furthermore, an investigation for assessing students based on their group discussions and portfolio was reported by Rasmussen, Northrup, and Lee (1997), and an attempt for creating facilities to allow students to submit comments on courseware design and delivery was depicted by Khan (1997). Although researchers have identified testing and assessment as important issues in computer-based instruction and have suggested appropriate design

44

H. C. Chu et al. / Asian Journal of Health and Information Sciences, Vol. 1, No. 1, pp. 43-60, 2006

strategies and techniques, few systems have attempted to diagnose student learning problems. Most conventional testing systems assign a score or status indicator to each student after testing, thus determining the learning status of that student, but do not consider how to improve it. To cope with this problem, the ITED III (Intelligent Tutoring, Evaluation and Diagnosis) project was initiated by the National Science Council of Taiwan, Republic of China (Hwang, Lin, Tseng, & Lin, 2005). The learning diagnosis model employed in ITED III originated from the concept that learning information, including facts, names, labels, or paired associations, is frequently a prerequisite to the efficient performance of a more complex skill of higher level, particularly in science courses (Salisbury, 1998). For example, thorough knowledge of the names and abbreviations of chemical elements and their atomic weights is required to comprehend scientific writing or chemical formula. Hence, learning a scientific concept-effectively generally requires knowledge of some basic concepts. Researchers have defined such relationships among concepts as concept-effect model (Hwang, 2003); moreover, they also developed tools to assist teachers in defining such relationships (Hwang, 2005). Consider two concepts or skills, Ci and Cj. If Ci is a prerequisite for the efficient performance of the more complex and higher-level concept Cj, then a concept-effect relationship Ci  Cj is said to exist. Notably, a concept may have multiple prerequisite concepts, and a given concept can also be a prerequisite concept of multiple concepts. Fore x a mpl e ,t ol e a r nt h ec on c e pt“ Ex e r c i s e ,”on e migh tf i r s tn e e dt ol e a r n“ Ex e r c i s e& me t a bol i s m,”whi l el e a r n i n g“ Nu t r i t i on ” mi g h tr e qu i r ef i r s tl e a r n i ng“ Vi t a mi ns ,”“ Ca l or i e ”a n d“ Pr ot e i n ”( a ss h owni n Figure 1).

Basic concept of health

Nutrition

Vitamins

Vitamins in Foods

Calorie

Calorie in foods

Protein

Protein in foods

Exercise

Exercise & metabolism

Figure 1. Illustrative example of health education concept-effect relationship diagram.

Following the construction of concept-effect relationships, the main problem is how to diagnose student learning problems. Notably, previous investigations employ a manual approach to diagnosing student learning problems that foresees the following steps:

45

H. C. Chu et al. / Asian Journal of Health and Information Sciences, Vol. 1, No. 1, pp. 43-60, 2006

Step 1: Calculate the incorrect answer rate for each student and each concept. Step 2: Define a threshold for identifying poorly learned concepts. Step 3: Record the poorly learned concepts and give feedback to the students. Although the manual approach has been applied to several subject units and has achieved desirable results, researchers found that the use of the approach is difficult and time consuming, and hence a fully computerized approach is needed. To deal with these problems, the following section proposes a computerized learning diagnosis approach. Moreover, some experimental results are also given to evaluate the performance of the new approach.

3. COMPUTERIZED MODEL FOR LEARNING DIAGNOSIS To diagnose the learning status of a student, a concept-effect propagation approach is employed to detect student learning problems. In the following , we shall present the approach in details. 3.1. Constructing Concept-effect Propagation Table A concept-effect propagation table (CEPT) is used to model the concept-effect propagation relationships among concepts. The CEPT records all concepts that may be influenced by each concept in the student learning process. For example, in Figure 2, the concepts affected by C1 are C1, C2…C10; the concepts affected by C3 are C3, C5, C6…C10; and the concepts affected by C7 are C7 and C10.

C1 Basic concept of health C2

C3

Nutrition C4

Vitam ins in Foods

C6

C5 Calorie

V itam ins C8

Exercise

Protein

C7 Exercise & m etabolism

C10

C9 Calorie in foods

Protein in foods

Figure 2. Illustrative example of a concept-effect relationship diagram.

46

H. C. Chu et al. / Asian Journal of Health and Information Sciences, Vol. 1, No. 1, pp. 43-60, 2006

The CEPT for the concept-effect relationships from Figure 2 is listed in Table 1, where Cj represents the concepts affected by Ci. If CEPT(Ci ,Cj) = 1, we say that “ Cj is one of the concepts affected by Ci during the student learning process.”A situation in which a student fails to learn Ci well implies that he/she will not be able to learn Cj well.

Table 1. Illustrative example of a concept-effect propagation table (CEPT) Propagated prerequisite relationships

Prerequisite concept Ci

Propagated effect concept Cj C1

C2

C3

C4

C5

C6

C7

C8

C9

C10

C1

1

1

1

1

1

1

1

1

1

1

C2 C3 C4 C5 C6 C7 C8 C9 C10

0 0 0 0 0 0 0 0 0

1 0 0 0 0 0 0 0 0

0 1 0 0 0 0 0 0 0

1 0 1 0 0 0 0 0 0

0 1 0 1 0 0 0 0 0

0 1 0 0 1 0 0 0 0

0 1 0 0 0 1 0 0 0

0 1 0 1 0 0 1 0 0

0 1 0 0 1 0 0 1 0

0 1 0 0 0 1 0 0 1

3.2. Constructing Test item-Concept Relationships Given a learning unit, comprising ten concepts (C1, C2, …, C10), and a test sheet, containing eight test items (Q1, Q2, Q3, …, Q8), a test item relationship table (TIRT) can be created, as listed in Table 2.

Table 2. Illustrative example of a test item relationship table (TIRT)

Test item Qn

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8

C1 1 0 0 0 0 0.2 0 0

C2 0.2 0.8 0 0 0 0 0 0

C3 0 0.4 0.6 0 0 0 0 0

C4 0 0 0.2 1 0 0 0 0

47

Concept Cj C5 C6 0 0 0 0 0.4 0 0 0 1 0 0 0.8 0 0 0 0

C7 0 0 0 0 0 0.2 1 0

C8 0 0 0 0 0 0 0 0.6

C9 0 0 0 0 0 0 0 0.2

C10 0 0 0 0 0 0 0 0.4

H. C. Chu et al. / Asian Journal of Health and Information Sciences, Vol. 1, No. 1, pp. 43-60, 2006

Each TIRT (Qn, Ci) entry represents the degree of association between test item Qn and concept Ci. When an educational expert provides a new test item, the item bank management module will ask that expert to specify the degree of relevance d(Qn, Ci) using a value ranging from 0 to r, where r was suggested to be an integer ranging from 3 to 7, so that the expert can determine the value without difficulty (Hwang, 2002). 3.3. Diagnosing Student Learning Problems with Matrix Composition After performing a test, the answers of the students are recorded in an answer sheet table (AST). Table 3 lists an illustrative example of an AST, where each entry AST(Sk, Qn) is a value ranging from 0 to 1; 0 indicates that student Sk answered test item Qn correctly, 1 indicates that Sk failed to answer Qn correctly, and a value between 0 and 1 indicates a partially correct answer. Notably, for true/false questions and multiple-choice questions, the value of AST(Sk, Qn) is either 0 or 1 while for short-answer questions, the value of AST(Sk, Qn) can range from 0 to 1.

Table 3. Illustrative example of an answer sheet table (AST) Test item

Q1

Q2

Q3

Q4

Q5

Q6

Q7

Q8

S1

0

0

1

0

0

1

0

1

S2

0

1

1

0

0

1

1

0

S3

0

0

0

1

0

1

1

0

S4

0

1

1

1

0

0

1

0

S5

0

0

1

0

0

0

1

1

To diagnose student learning problems, the relationships in AST, TIRT and CEPT must be composed. Since the traditional matrix multiplication operation requires complex computations and the computation results are not in [0, 1], further analysis of student learning status is difficult. Thus, this study adopts the max-min composition method. Let R1={(x, y)|(x, y)X×Y} and R2 = {(y, z)|(y, z)Y×Z} be two fuzzy relations. The max-min composition of R1 and R2 is R1 R2={(x, z)|(x, z)=Max{Min{μR1( x ,y ) ,μR2( y ,z ) } } f orxX, yY and zZ } whe r eμR1。R2 is again the membership function of a fuzzy relation on fuzzy sets (Catelani & Fort 2002). An illustrative example of the max-min composition method is given as follows:

48

H. C. Chu et al. / Asian Journal of Health and Information Sciences, Vol. 1, No. 1, pp. 43-60, 2006

Y1 Y2 Y3 Y4 Y5 X1  0.1 0.2 0 1 0.7    R1(X, Y)= X2  0.3 0.5 0 0.2 1  ;  X3  0.8 0 1 0.4 0.3   Z1 Z2 Y1  0.9 0  Y2  0.2 1 R2(X, Y)= Y3  0.8 0  Y4  0.4 0.2 Y5  1 0

Z3 Z4 0.3 0.4   0.8 0  ; 0.7 1   0.3 0  0 0.8 

(X1, Z1) = Max{Min[(X1, Y1), (Y1, Z1)], Min[(X1, Y2), (Y2, Z1)], Min[(X1, Y3), (Y3, Z1)], Min[(X1, Y4), (Y4, Z1)], Min[(X1, Y5), (Y5, Z1)]} = Max {Min[0.1, 0.9], Min[0.2, 0.2], Min[0, 0.8], Min[1.0, 0.4], Min[0.7, 0]} = Max {0.1, 0.2, 0, 0.4, 0} = 0.4. By repeatedly performing the max-min composition operations, the relationships of X's and Z's can be derived as follows: Z1 Z2 X1  0.4 0.7  R1 R2 = X2  0.3 1 X3  0.8 0.3 

Z3 Z4 0.3 0.7   0.5 0.8  . 0.7 1  

Note that a crisp relation such as the AST derived from the answers of Truth/False questions or the CEPT can be considered as a restricted case of the fuzzy relation; therefore, although max-min composition is defined normally on the basis of fuzzy relations, it can be applied to the composition of the AST (students-to-test item), TIRT (test item-to-concept) and CEPT (concept-to-concept) to derive the relationship between each student and each concept. Namely, applying the following fuzzy relation composition operations can derive the error degree for each student regarding each concept: Error_Degree (Sk, Cj) = AST(Sk, Qn)

TIRT(Qn, Ci) CEPT(Ci, Cj).

Performing the fuzzy relation composition operations for the illustrative example produces Error_Degree (Sk, Cj) = AST TIRT CEPT

49

H. C. Chu et al. / Asian Journal of Health and Information Sciences, Vol. 1, No. 1, pp. 43-60, 2006

0   0   0  0   0 

0 1 0 0 1 0 1 1 0 0 1 1 0 0 1 0 1 1 1 1 1 0 0 1 0 1 0 0 0 1

C1  S1  0 .2 S  0 .2  2 S3  0 .2  S4 0  S5  0

1  0 0 0 0 0 0 0  0 1 0.2 0  0 0.8 0.4 0 0 0 0 0 0 0  0  1    0 0 0 . 6 0 . 2 0 . 4 0 0 0 0 0 0  0    0 0 0 1 0 0 0 0 0 0  0   0 0 0 0 1 0 0 0 0 0  0  0 0   0. 2 0 0 0 0 0.8 0.2 0 0 0  0    1  0 0 0 0 0 0 1 0 0 0  0   0 0 0 0 0 0 0.6 0.2 0.4 0 0    0 

1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0  0 1 0 1 1 1 1 1 1  0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 1 0 0  0 0 0 0 1 0 0 1 0 0 0 0 0 0 1 0 0 1  0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0  0 0 0 0 0 0 0 0 1 

C 2 C 3 C 4 C 5 C 6 C 7 C8 C 9 C10  0 . 2 0 . 6 0 . 2 0 . 6 0 . 8 0 . 6 0 . 6 0. 8 0 . 6   0.8 0.6 0.8 0.6 0.8 1 0.6 0.8 1 .  0 . 2 0 . 2 1 0 . 2 0 . 8 1 0 . 2 0. 8 1  0 . 8 0 . 6 1 0 . 6 0 . 6 1 0 . 6 0. 6 1   0 0 . 6 0 . 2 0 . 6 0 . 6 1 0 . 6 0. 6 1  

For each student, a vector is obtained to describe the fail-to-correctly-answer degree for each concept based on the calculation results. For example, Error_Degree (S1) = [0.2, 0.2, 0.6, 0.2, 0.6, 0.8, 0.6, 0.6, 0.8, 0.6]. By labeling the values in the vector on the corresponding concepts of Figure 2, a labeled concept-effect relationship diagram for S1 is obtained and displayed in Figure 3.

0.2 C1 Basic concept of health C2

0.6

C3

Nutrition C4

0.6

Vitamins C8

0.6

Vitamins in Foods

C5

Exercise

0.8

C6

Calorie C9

0.2

0.6

Protein

0.8

C10

Calorie in foods

C7

Exercise & metabolism

0.8

Protein in foods

Figure 3. Labeled concept-effect relationship diagram for S1.

50

0.2

H. C. Chu et al. / Asian Journal of Health and Information Sciences, Vol. 1, No. 1, pp. 43-60, 2006

3.4. Generating Learning Guidance for Individual Student To generate learning guidance for each individual student, the technique of fuzzy inference is employed, which has been widely discussed and applied in a variety of areas, such as system control, modeling, signal processing and expert systems (Zeng, Zhang, & Xu, 2000; Liu, 2002). A fuzzy inference procedure involves three primary processes: fuzzification, implication, and defuzzification. Fuzzification operations are used to combine a real-time input value (e.g. temperature and speed) with stored membership function information to produce fuzzy input values. Fuzzy implication attempts to identify areas of correspondence between the fuzzified input facts and the antecedent part (the IF part) of a fuzzy rule with IF/THEN structure and outputs the value of the consequent part (the THEN part). Meanwhile, defuzzification combines all fuzzy outputs into a specific composite outcome. In particular, membership functions provide a method of translation between linguistic expressions such as "Tom is old" and numerical input facts such as "Tom is 67 years old" (see Figure 4).

degree 1.0 0.92

OLD

0.46 0

YOUNG

0

67

100

age

Figure 4. Membership function of age.

To provide learning guidance for each student, the membership functions for LOW, AVERAGE and HIGH Error_Degree are defined as follows:

1   x  2  1 2( )  LOW(x) =   x  2  2( )    0 

51

for x  for x  for x  for x 

H. C. Chu et al. / Asian Journal of Health and Information Sciences, Vol. 1, No. 1, pp. 43-60, 2006

0   x  2( )2    x  2 1 2( ) AVERAGE(x)=      x  2  1 2( )    2( x ) 2    0 

for x 

0   x  2( )2 HIGH (x) =    x  2  1 2( )    1 

for x 

(+ ) for x  2 (+ ) for x  2 ( ) for x  2 ( ) for x  2 for x 

for x  for x  for x 

Degree of membership

where x = Error_Degree (Sk, Cj) and , , and are the parameters that define the curves of the functions. If one assumes that  = 0,  = 0.5, and = 1.0, the membership functions in Figure 5 can be derived.

1.0

LOW

AVERAGE

0

0.4 0.5

HIGH

0.92 0.68

0.32

1.0

Error_degree (Sk, Cj) Figure 5. Membership function for describing Error_Degree.

52

H. C. Chu et al. / Asian Journal of Health and Information Sciences, Vol. 1, No. 1, pp. 43-60, 2006

The corresponding fuzzy rules for determining Learning_Status are given as follows: IF THEN IF THEN IF THEN

Error_Degree (Sk, Cj) is HIGH Learning_Status (Sk, Cj) is Poorly-Learned Error_Degree (Sk, Cj) is AVERAGE Learning_Status (Sk, Cj) is Partially-Learned Error_Degree (Sk, Cj) is LOW Learning_Status (Sk, Cj) is Well-Learned

For example, assuming original Error_Degree (Sk, Cj) = 0.4, by applying the fuzzification operations, the following fuzzy input values will be produced: Fact 1: Error_Degree (Sk, Cj) is HIGH with degree 0.32; Fact 2: Error_Degree (Sk, Cj) is AVERAGE with degree 0.92; Fact 3: Error_Degree (Sk, Cj) is LOW with degree 0.68. By applying the fuzzy implication operation, the outputs will be: Output 1: Learning_Status (Sk, Cj) is Poorly-Learned with degree 0.32; Output 2: Learning_Status (Sk, Cj) is Partially-Learned with degree 0.92; Output 3: Learning_Status (Sk, Cj) is Well-Learned with degree 0.68. Consequently, by applying the maximum-membership defuzzification, the final outcome is “ Le a r n i ng _St a t u s(Sk, Cj) is Partially-Le a r n e d”wi t hde g r ee 0.92. Based on the Learning_Status (Sk, Cj), a learning guidance is generated for each student, and a tutoring suggestion is provided to the personalized tutoring system as feedback by applying the following rules: IF THEN IF THEN IF THEN

Learning_Status (Sk, Cj) is Poorly-Learned Arrange for Student Sk to re-learn the unit containing Concept Cj Learning_Status (Sk, Cj) is Partially-Learned Arrange more practice related to Concept Cj for Student Sk Learning_Status (Sk, Cj) is Well-Learned Record that Student Sk has completed the study of Concept Cj

Since the concept-effect propagation relationships have been considered in the construction of the CEPT, the critical learning problem is easily solved by identifying the partially-learned concept which is closest to the root of the concept-effect relationship diagram. In the example given in Figure 3, the critical learning problem is “ combinational circuit,”and the learning guidance is presented in Figure 6.

4. DEVELOPMENT OF THE PROPOSED LEARNING DIAGNOSIS APPROACH

53

H. C. Chu et al. / Asian Journal of Health and Information Sciences, Vol. 1, No. 1, pp. 43-60, 2006

Based on the proposed learning diagnosis approach, a computer-assisted testing system has been implemented on the Windows NT environment using the JAVA language. A graphical user interface is provided to assist teachers in defining the prerequisite relationships among concepts. For a new subject unit, the teacher needs to define a Root concept, and then defines prerequisite relationships using the format. Notably, each individual concept can be a parent or child of multiple concepts. A test interface is provided to allow students to perform self-assessment or participate in group tests. For self-assessment, a discussion group window is available for students to conduct on-line discussions and answer the test items together. However, the discussion window can be disabled during group tests if desired. To conduct a group test for a class of students, the teacher merely needs to define some parameters, such as course identification, class identification, average difficulty of the test items, test time, and so on, and the testing system then generates test and answer sheets accordingly. The students undergoing testing are asked to log on to computers located in dedicated computer rooms, and during the test period their access is restricted to the test items only. Following the submission of student answers to the test items, the system gathers the answers from clients and generates learning guidance for individual students (see Figure 7), as well as a summarized test report for the teacher. Concept Learning status of the concept C1 Basic concept of health You have completed the study of this concept. C2 Nutrition You have completed the study of this concept. C3 Exercise You need to re-learn this concept. C4 Vitamins You have completed the study of this concept. C5 Calorie You need to do more practices related to this concept. C6 Protein You need to re-learn this concept. C7 Exercise & metabolism You need to do more practices related to this concept. C8 Vitamins in Foods You need to do more practices related to this concept. C9 Calorie in foods You need to re-learn this concept. C10 Protein in foods You need to do more practices related to this concept. To-be-enhanced learning paths: PATH1: Nutrition  Vitamins  Vitamins in Foods PATH2: Nutrition  Calorie  Calorie in foods PATH3: Nutrition  Protein  Protein in foods Learning guidance for the student: 1. According to the diagnosis from the system, we found that you did not learn the concepts “ Nutrition,”“ Exercise,”“ Vitamins,”“ Calorie,”“ Pr ot e i n,”“ Vitamins in Foods,”“ Calorie in Foods,”“ Protein in Foods,”a n d“ Ex e r c i s e& me t a bol i s m”well. 2. The critical l e a r ni ngpr o bl e mi st hemi s unde r s t a ndi ngofc onc e pt“ Nut r i t i o n,”which affects the learning of other concepts; therefore, you need to re-learn this concept prior to other concepts. Critical learning path: Nutrition  Protein  Protein in foods Figure 6. Illustrative example of a learning guidance.

54

H. C. Chu et al. / Asian Journal of Health and Information Sciences, Vol. 1, No. 1, pp. 43-60, 2006

Figure 7. Illustrative example of a learning guidance.

5. EXPERIMENT ON A NUTRITION COURSE To evaluate the efficiency of the presented approach, an experiment was conducted involving seventy k-3 students enrolled in a health education course, namely, “ Nu t r i t i onofFoods .”Th ee x pe r i me n ti nv ol v e dseventeen concepts and twenty-six concept-effect relationships, all of which were new to the students. The students were randomly separated into two groups, Group-A (control group) and Group-B (experimental group), each containing thirty-five students. The students in Group-A (V1) received the learning guidance and relevant homework given by the teacher, while those in Group-B (V2) received learning suggestions given by ITED III and relevant homework given by the teacher following each on-line test. Within a semester, all sixty students took two group tests (i.e. a pre-test and a post-test) and three self-assessments. The following presents the statistical results obtained from applying the SPSS to analyze the group tests.

55

H. C. Chu et al. / Asian Journal of Health and Information Sciences, Vol. 1, No. 1, pp. 43-60, 2006

(1) Pre-test The pre-test aimed to ensure that both groups of students had the equivalent knowledge required for learning the nutrition course. The test sheet of the pre-test contained twenty true/false questions, thirty multiple-choice questions and ten short-answer questions for testing the basic notations related to the study of nutrition. Table 4 presents the t-test results of the pre-test. Notably, the mean and standard deviation of the pre-test was 55.86 and 11.62 for Group-A (V1), and 57.14 and 10.77 for Group-B (V2). As the p-value (Significant level) = 0.633 > 0.05 and t =  0.48, we can infer that in the pre-test, Groups A and B do not significantly differ at a confidence interval 95%. From the above it was evident that the two groups of students have equivalent abilities in learning the ASP course. Table 4. t-test of the pre-test results N 35 35

V1 V2

Mean 55.86 57.14

SD 11.62 10.77

t

p

-0.48

0.633

(2) Post-test The post-test was intended to compare the basic nutrition knowledge of the two groups of students after learning the course. The test sheet of the post-test contained twenty true/false questions, thirty multiple-choice questions and ten short-answer questions. Table 5 lists the t-test values for the post-test results. Notably, the mean and standard deviation of the post-test were 77 and 14.83 for Group-A (V1), and 93.57 and 6.89 for Group-B (V2). From the mean of the post-test, Group-B seems to achieve better performance than Group-A. As the p-value = 0 < 0.05 and t =  5.996, we can conclude that Group-B achieved significantly better performance than Group-A after implementing the subject approach. Table 5. t-test of the post-test results V1 V2

N 35 35

Mean 77 93.57

SD 14.83 6.89

t

p

-5.996

0

6. CONCLUSIONS This investigation presents a web-based intelligent testing and diagnostic system in a networked environment. A fuzzy approach has been employed to diagnose poorly-learned, partially-learned and well-learned concepts, and to provide learning suggestions. An experiment on a science course was conducted to

56

H. C. Chu et al. / Asian Journal of Health and Information Sciences, Vol. 1, No. 1, pp. 43-60, 2006

evaluate the performance of this approach. Based on the results of the pre-test, the two groups of students had equivalent basic knowledge before learning the nutrition course. The students in both groups then received the same subject materials, an equivalent amount of homework, the same tests, and were taught by identical teachers. The two groups differed only in terms of the source of learning suggestions. For the control group, the learning suggestions and the relevant homework were given by the teacher. Meanwhile, for the experimental group, the learning suggestions were given by applying the novel approach, and the teacher was asked to assign homeworks accordingly. From the results of the post-test, the students in the experimental group has progressed more significantly than the students in the control group. This study thus concludes that the proposed approach can diagnose student learning problems and provide helpful suggestions to students. Besides the new applications, several relevant studies are in process. In developing an intelligent tutoring system, various factors need to be considered, including the item bank maintenance, the concept relationship construction, the generation of feasible test sheets, the implementation of adaptive tutoring strategies and the design of subject materials. Among the research issues concerning the identification of student learning problems, the effective generation of a feasible test sheet may be the most important. Currently, a research group is working on developing test sheet generating algorithms using different approaches, including dynamic programming, heuristic algorithms and genetic algorithms.

REFERENCES Antao, B. A. A., Brodersen, A. J., Bourne, J. R. & Cantwell, J. R. (2000). Building intelligent tutorial systems for teaching simulation in engineering education. IEEE Transactions on Education, 35(3), 222-225. Catelani, M. & Fort, A. (2002). Soft fault detection and isolation in analog circuits: Some results and a comparison between a fuzzy approach and radial basis function networks. IEEE Transactions on Instrumentation and Measurement, 51(2), 196-202. Chou, C. (2000). Constructing a computer-assisted testing and evaluation system on the world wide web - the CATES experience. IEEE Transactions on Education, 43(3), 266-272. Fan, J. P., Tina, K. M., & Shue, L. Y. (1996). Development of knowledge-based computer-assisted instruction system. Proceedings of International Conference on Software Engineering: Education and Practice, Dunedin, New Zealand. Giraffa, L. M. M., Mora, M. C., & Viccari, R. M. (1999). Modeling an interactive ITS using a MAS approach: from design to pedagogical evaluation. Proceedings of 3rd International Conference on Computational Intelligence and Multimedia Applications, New Delhi, India, 153-158. Gonzalez, A. V., & Ingraham, L. R. (1994). Automated exercise progression in

57

H. C. Chu et al. / Asian Journal of Health and Information Sciences, Vol. 1, No. 1, pp. 43-60, 2006

simulation-based training. IEEE Transactions on Systems, Man and Cybernetics, 24(6), 863-874. Harp, S. A., Samad, T., & Villano, M. (1995). Modeling student knowledge with self-organizing feature maps. IEEE Transactions on Systems, Man and Cybernetics, 25(5), 727-737. Hwang, G. J. (1998). A tutoring strategy supporting system for distance learning on computer networks. IEEE Transactions on Education, 41(4), 1-19. Hwang, G. J. (2002). On the Development of a Cooperative Tutoring Environment on Computer Networks. IEEE Transactions on System, Man and Cybernetic Part C, 32(3), 272-278. Hwang, G. J (2003). A concept map model for developing intelligent tutoring systems. Computers & Education, 40(3), 217-235. Hwang, G. J., Hsiao, J. L., & Tseng, J. C. R. (2003). A computer-assisted approach for diagnosing student learning problems in engineering courses. Journal of Inofrmation Science and Engineering, 19(2), 229-248. Hwang, G. J., Lin, Bertrand M. T., Tseng, H. H., & Lin, T. L. (2005). On the Development of a Computer-Assisted Testing System with Genetic Test Sheet-Generating Approach. IEEE Transactions on Systems, Man, and Cybernetics: Part C, 35(4), 590-594. Hwang, G. J. (2005). A Data Mining Algorithm for Diagnosing Student Learning Problems in Science Courses. International Journal of Distance Education Technology, 3(4), 35-50. Jouffe, L. (1998). Fuzzy inference system learning by reinforcement methods. IEEE Transactions on Systems, Man and Cybernetics –Part C, 28(3), 338-355. Khan, B. H. (1997). Web-based instruction (WBI): What is it and why is it?. In B. H. Khan (Ed.), Web-Based Instruction (pp. 5-18). Englewood Cliffs, NJ, USA: Educational Technology Publication. Liu, P. (2002). Mamdani fuzzy systems: universal approximator to a class of random processes. IEEE Transactions on Fuzzy Systems, 10(6), 756-766. Olsen, J. B., Maynes, D. D., Slawson, D., & Ho, K. (1986, April). Comparison and equating of paper-administered, computer-administered and computerized adaptive tests of achievement. Paper presented at the Annual Meeting of American Educational Research Association, San Francisco, California. Ozdemir, B., & Alpaslan, F. N. (2000). An intelligent tutoring system for student guidance in Web-based courses. Proceedings of 4th Int. Conference on Knowledge-Based Intelligent Eng. Systems and Allied Technologies, 2, Brighton, UK, 835-839. Piko, B. F., & Bak, J. (2006). Children's perceptions of health and illness: Images and lay concepts in preadolescence. Health Education Research, 21(5), 643-653. Rasmussen, K., Northrup, P., & Lee, R. (1997). Implementing Web-based instruction. In B. H. Khan (Ed.) Web-Based Instruction (pp. 341-346). Englewood Cliffs, NJ, USA: Educational Technology Publication. Rowe, N. C., & Galvin, T. P. (1998). An authoring system for intelligent procedural-skill tutors. IEEE Intelligent Systems, 13(3), 61-698.

58

H. C. Chu et al. / Asian Journal of Health and Information Sciences, Vol. 1, No. 1, pp. 43-60, 2006

Salisbury, D. F. (1998). Effect drill and practice strategies. In David H. Jonassen (Ed.), Instructional Designs for Microcomputer Courseware (pp. 103-124). Lawrence Erlbaum Associates. Vasandani, V., & Govindaraj, T. (1991). Intelligent diagnostic problem solving tutor: an experimental evaluation. IEEE Transactions on Systems, Man and Cybernetics, 3, 1739-1744. Vasandani, V., & Govindaraj, T. (1995). Knowledge organization in intelligent tutoring systems for diagnostic problem solving in complex dynamic domains. IEEE Transactions on Systems, Man and Cybernetics, 25(7), 1076-1096. Wainer, H. (1990). Computerized adaptive testing: A primer. Hillsdale, NJ, USA: Lawrence Erlbaum Associates. Zeng, K., Zhang, N. Y., & Xu, W. L. (2000). A comparative study on conditions for Takgi-Sugeno fuzzy systems as universal approximators. IEEE Transactions on Fuzzy Systems, 8(6), 773-780.

Hui-Chun Chu is currently a Ph.D. student of the Department of Information and Learning Technology at National University of Tainan in Taiwan, Republic of China. Her research interests include e-learning, information technology-applied instructions, and medical expert systems.

Gwo-Jen Hwang is currently a Professor of the Department of Information and Learning Technology and the Dean of the College of Science and Engineering at National University of Tainan in Taiwan, Republic of China. Dr. Hwang received his Ph.D. degree from the Department of Computer Science and Information Engineering at National Chiao Tung University in Taiwan. Dr . Hwa ng’ sr e s e a r c h i nt e r e s t si n c l u de e-learning, computer-assisted testing, expert systems and mobile computing. He has published over 200 academic papers in those fields.

59

H. C. Chu et al. / Asian Journal of Health and Information Sciences, Vol. 1, No. 1, pp. 43-60, 2006

Judy C. R. Tseng is currently an Associate Professor of the Department of Computer Science and Information engineering at Chung-Hua University in Hsinchu, Taiwan, Republic of China. She is also the Chairman of that department. Her research interests include database systems, e-commerce, data mining and e-learning.

Gwo-Haur Hwang is currently an Assistant Professor of the Information Management Department at Ling Tung University in Taichung, Taiwan, Republic of China. He is also the Chairman of that department. His research interests include e-learning, e-commerce and medical expert systems.

60