English course E-learning system based on relative ...

Multimed Tools Appl DOI 10.1007/s11042-010-0708-7

English course E-learning system based on relative item difficulty using web component composition Hwa-Young Jeong & Bong-Hwa Hong & Bhanu Shrestha & Seongsoo Cho

# Springer Science+Business Media, LLC 2011

Abstract Many researches about e-learning system have been applied item difficulty to increase learning effectiveness. And development environment was changed the internet based learning media contents into the more various technology such as component, web 2.0, service oriented development and so on. Especially, service-oriented development is one of new trend in web based system and has become mainstream in software development. In the development, web components aims at providing support to serviceoriented technique by enabling automatic discovery, composition, invocation and interoperation of the services. In this paper, we aimed the implementation of English elearning system including the item guessing parameter and considering the relative correction of item difficulty. In the system, a learner was given to choose the learning step by the relative difficulty. In order to process and combine, all the learning contents are based on Sharable Content Object Reference Model (SCORM) with Learning Management System (LMS). Also, each learning contents are belong to Sharable Content Objects (SCOs).

H.-Y. Jeong Department of General Education, Kyunghee University, 1, Hoegi-dong Dongdaemun-gu Seoul 130-701, Korea e-mail: [email protected] B.-H. Hong Department of Information Communication, Kyunghee Cyber University, Dongdaemun-gu Seoul 130-701, Korea e-mail: [email protected] B. Shrestha : S. Cho (*) Department of Electronic Engineering, Kwangwoon University, Seoul 139-701, Korea e-mail: [email protected] B. Shrestha e-mail: [email protected]

Multimed Tools Appl

Keywords E-learning system . SCORM . Item analysis . Web component . Relative item difficulty

1 Introduction Technological advances in information and network technology have made the transition from conventional classroom-based to virtual learning space. On-line learning is a new learning domain, while Web-based learning has been predicted as the main distance learning approach in the future [7]. Web-based learning offers many advantages to those seeking to advance their education [17]. Moreover, as numerous web-based tutoring systems have been developed, a great quantity of hypermedia in courseware has created information, and cognitive overload and disorientation [6]. Courseware specification is the process of identifying the aims and objectives of the material, placing this in the context of the student population and their assumed previous knowledge and describing the detailed syllabus [4]. Recently, the subject of education form changes teacher to learners and learners can enhance their own problem-solving ability and learning ability through solving the problems that is appropriate to learners by themselves. Although an item often appears easy or difficult based on personal judgment, the scientific method of calibration often assigns difficulties higher or lower than one would expect [13]. That is, students learn in different ways, and training techniques and technologies are being designed to accommodate individual learning needs [19]. But, most of web based educational course don’t have interactions that support meaningful learning and are providing one-sided learning contents simply and repeatedly. Also, individualization learning that presents learning contents and method separately according to learning ability of individuals is deficient. And existing setting question methods in question bank system having learning contents are mostly fixed or randomized setting-question method then they cannot be providing more efficient learning contents. Learning objects are often regarded as traditional documents, but it is possible to reuse learning objects in a much more sophisticated way, if we can access the components of a learning object and repurpose them on-the-fly. However, this requires a more innovative and flexible underlying model of learning object components [18]. SCROM has solved this problem in integrating the standards of e-learning, in the hope that future e-learning materials can meet the goal of being reusable, accessible, durable and interoperable. Fine teaching methods and creative learning activities are key to effective learning [15]. We present web component based English course e-learning system considering the relative correction of item difficulty. For the precise learning results of learner and the application of setting-question according to selected learning step, the item guess parameter that contained correct answer by guessing was considered in the calculation of the relative correction of item difficulty. We used learning object by the SCORM for an efficient applications. For efficient management and development the system, we implement and use the components were embodied with web. Because component oriented programming is about assembling systems from prefabricated parts, designed to be modular and re-usable, using common communication protocols, are user configurable, and easily composable [3]. We embodied the setting-question parts and the calculation and application parts of the relative correction of item difficulty that there are main business logics with web component. And then we showed that this system is applicable by assembling and operating the web component which was embodied in the learning courses

Multimed Tools Appl

2 Related work 2.1 The relative item difficulty Birnbaum [5] modified the two-parameter logistic model to include a parameter that represents the contribution of guessing to the probability of correct response. Nevertheless the resulting model has become known as the three-parameter logistic model, even though it technically is no longer a logistic model. The equation for the three-parameter model is item difficulty parameter, item discrimination parameter, item guessing parameter [10]. The item difficulty is simply the percentage of students taking the test who answered the item correctly and calculation formula is as follows [14]: P¼

R N

ð1Þ

where N is the number of total learners and R is the number of learners who answer correctly. The item guessing parameter represents the value of learners who gave an answer correctly by guess among all learners who answer correctly in truth. That is, when G indicates the number of learners who don’t know an answer but guess the answer, the number of learners who guess the answer is as follows: GR ¼ G»

1 Q

ð2Þ

where GR is the number of learners who answer correctly by guess, G is the number of learners who guess the answer, and Q, the number of answers. Also, the calculation formula for the number of learners who guess an answer but cannot give a correct answer is given in Equation (3) and (4). GW ¼ G»

Q1 ¼W Q

ð3Þ

WQ Q1

ð4Þ

G¼

where W is the number of learners who guess an answer but cannot give a correct answer. Therefore, GR is calculated as follows: GR ¼

W Q1

ð5Þ

The item guessing parameter, the ratio of learners who don’t know the answer but give the correct answer by guess, is presumed by the following formula: PGR

GR ¼ ¼ N

where N is the number of total learners.



 W =N Q1

ð6Þ

Multimed Tools Appl

The relative correction of the item difficulty represents the relative difficulty that excludes item guessing parameter, the ratio of learners who give the correct answer by guess, and is calculated as calculation formula is as follows: PC ¼ P  PGR where PC is the relative correction of item difficulty. 2.2 Sharable content object reference model (SCORM) The SCORM was developed to address the need for interoperability of learning objects between Learning Management System (LMS) [16], and it has become the most widely used international standard for e-learning [8]. The SCORM also created the possibility of sharing content and creating new content by assembling existing elements [2]. The SCORM content aggregation model contains the following components: Assets, Sharable Content Object (SCO) and Content Aggregations (CA). Assets are an electronic representation of media, text, images, audio, web pages or other data that can be presented in a web client [18]. The SCO represents a collection of one or more assets that include a specific launchable asset that utilizes the SCORM run-time environment to communicate with LMS [1]. A Content Aggregation is a map (content structure) that can be used to aggregate learning resources in a well integrated unit of education (for example course, chapter, module, etc.). So, all the teaching materials components can be categorized as SCO or Asset and are placed in teaching materials components database [15]. The SCORM focuses on the interface between content (SCO or Asset) and the LMS. In other words: how learning content gets into an LMS system, how it gets presented to learners, and how the learner’s progress within that content is communicated to the LMS [11]. 2.3 Web component Web components are a service-oriented software unit with explicit goal-directed and contractually specified interfaces based on the internet. Web components can not only be published independently but also be subjected to composition by third parties [9]. Karim [20] presented web components as below: a)

Web components must be self-descriptive black-box entities that encapsulate services, which are accessible only via well-defined interfaces. b) Web components must provide one or several interfaces to be used and require one or several interfaces to communicate with other components. c) Web components must be configurable in order to be used in different contexts. d) Web components must be accessible through the Web (interfaces based on HTTP, SMTP, etc.) e) Web components must be language and platform independent. A web Component Based Development (CBD) view is shown in Fig. 1. These systems are probably distributed and located in different geographical places, being communicated with distributed models like Common Object Request Broker Architecture (CORBA), Enterprise Java beans (EJB) and/or Distributed Component Object Model (DCOM), and making use of important rules, security methods and XML/XMI techniques for the intermediate representation of the software components information[12].

Multimed Tools Appl Organization CBD View

Web Service View Select and selection

Software Component Archiecture

WSDL Specification + Implementation

Third Parts UDDI

Organization component repositories COTS Commercial Off-The-Shelf

Fig. 1 Web Component Based Development (CBD) view

3 Design and analysis E-learning system 3.1 Design of the relative item difficulty In this research, we applied the relative difficulty to each question. That is, when learner selects learning level and the number of questions to study before studying then pertinent questions produced according to the relative correction of item difficulty are provided as many as the number of selected Items in multiple-choice type followed by select one of five examples. When learner finishes the studying then learning result is displayed according to correct or incorrect answers for presented problems and the relative difficulty is calculated

Table 1 Evaluation standard by the relative correction of item difficulty Learning step

Relative correction of item difficulty

Means

High

0~0.2

Very difficult

Normal-high

0.21~0.40

Difficult

Normal

0.41~0.60

Normal

Easy-normal

0.61~0.80

Easy

Easy

0.80~1

Very easy

Multimed Tools Appl Table 2 Calculation of the relative correction of item difficulty No

N

R

G

GR

PGR

P

PC

1

20

12

10

2

0.1

0.6

0.5

:

:

:

:

:

:

:

:

and stored to next application for studying. When item is registered for the first time, there is not enough learning results to calculate the relative correction of item difficulty. Therefore, examiner sets the relative correction of item difficulty at discretion. And then, as learning proceeds, the relative correction of item difficulty set at the first time is calculated and reapplied by learning result of learner. Table 1 shows the application standard of the relative correction of item difficulty containing the item guessing parameter. Relative correction item difficulty is calculated every learning and re-applied in the next step using the calculated values, then presented the learning step to the learners which are matched with those calculated values. E-Learning system EJB Server Course Construction

Account management Teacher Account database

Courseware Database

Connector

Question result

Item difficulty

Question Question Bank

Learner Account database

Service course management and calculate the relative difficulty SCORM

Testing/ Assessment Service

Local Content Repository

Selection Course Administration Service

SCORM Content Packages

Sequencing Service

Content Management Service

Delivery Service Launch

Learner Profiles Service

Tracking Service

SCORM Tracking Data

Remote Content Repositories

API Adapter

SCORM API

SCORM Content Browser

Fig. 2 Development view of the SCORM based e-learning system considering the relative difficulty

Multimed Tools Appl Fig. 3 Class diagram of question result component

QuestionResult QuestionNumber QuestionContent CorrectAnswer

DifficultyReo mteEJB

CalculateResult() DifficultyHo me

The relative correction of item difficulty was calculated as PC =P - PGR by the relative item difficulty and item guess parameter of item analysis method. For example, when total learners are 20 for the first question in Table 2 and learners who gave a correct answer are 12 in multiple-choice question followed by five options, the relative correction of item difficulty is calculated as 0.6. The relative item difficulty (P) becomes as 12/20=0.6, the number of learners guessing answer (G) is as 8(5)/(5–1)=10, the number of learners giving a collect answer by guessing (GR) is as 10(1/5)=2 and item guessing parameter becomes PGR =GR/N=2/20=0.1. Therefore, the relative correction of item difficulty (PC) is as 0.6–0.1=0.5 then the first question is the question with medial relative difficulty corresponding to ‘medium’ in learning levels. where No. is the number of item, is the number of total learners, R is the number of learner giving a correct answer, G, the number of learners guessing an answer, GR,, the number of learner giving a correct answer by guessing, PGR, the item guessing parameter, P, the relative item difficulty, and PC,the relative correction of the item difficulty. 3.2 Design of SCORM based E-learning web component considering the relative difficulty The system was designed with UML and constituted with dispersed server having EJB web component server. EJB parts were embodied with stateless session bean. SetDifficulty DifficultyHo me

SetDiff()

ItemDifficulty RequestDifficultyLevel QuestionNumber Score

DifficultyReo mteEJB

Result Question Number Content Example 1 Example 2 Example 3 Example 4 Example 5 Correct Answer

SetDiff() GetDiff() UpdateDiff() CalculateDifficulty()

Fig. 4 Class diagram of item difficulty component

GetDifficulty GetDiff()

UpdateDifficulty UpdateDiff()

Multimed Tools Appl

The development view of this system is as shown in Fig. 2. There are five components in this system; account management, course construction, question result, item difficulty, and question. To composite each components, it used the connector between them. In this system, learner can use learning level choice, question-learning and analysis of learning results after log-in and examiner can set setting-question and the relative difficulty at the very beginning of setting-question. After that, as learning proceeds by lots of learners, learning results of learners are reflected on the relative correction of item difficulty and learning level of each presented Items by the relative item difficulty is reset. To analyze the score and calculate the relative item difficulty, main components will be QuestionResult and item difficulty. As shown in Fig. 3, QuestionResult class has question number which learner solved and a number of an examination paper and verifies answers.

Account management

Courseware

Question

Question result

Item Difficulty

: Learner

1. Log in 2. Check ID and Password

3. Request course information 4. Check learning contents

5. Display learning contents 6. Study

7. Set study step by item difficulty

8. Request the question

9. Select the question according to study step

10. Provide the question

11. Answer the question

12. Request to check correct answer 13. Check the answer 14. Request to analyze relative correction of item difficulty 15. Get each parameter data 16. Calculate relative correction of item difficulty and update the data

17. Send analysis data of relative item difficulty 18. Update item difficulty of the each question

19. Support learning result of the course

Fig. 5 Sequence diagram of learner’s mode

Multimed Tools Appl

DifficultyHome is home interface class and DifficultyRemoteEJB is remote interface class of EJB component. That is, Question that is application class request and process item difficulty via EJB’s interface. DifficultyRemoteEJB class calculates the relative correction of item difficulty and sets questions classified by learning level. QuestionResult can get process result of Question class. Figure 4 shows internal class of item difficulty component that handles the relative correction of item difficulty and sets questions. DifficultyHome is class to register component from the outside. The item difficulty class is deal with analysis and calculation of the relative item difficulty. So, it has actual business logic to set questions, Pcr class to calculate item guess parameter and Pc class to calculate the relative correction of item difficulty. Questions corresponding to learning level chosen by learner are extracted randomly according to the relative item difficulty collection from question information database in CheckQuestion, internal method. Extracted questions have information of each items set in result class and provide learners questions. After log-in, examiner can register new learning question with setting up the relative item difficulty in the part of question registration and can modify and delete

: (Question)

: DifficultyHome

: DifficultyReomteEJB

: ItemDifficulty

: SetDifficulty

: GetDifficulty

: UpdateDifficulty

1.Registration component and create object

2.Create remote object

3. Call remote method

4.Request to calculate relative correction of item difficulty

5.Request to calculate item difficulty 6. Result of item difficulty

7.Request to calculate item guessing parameter

8. Result of Item guessing parameter

9.Calculate relative correction of item difficulty 10.Result of relative correction of item difficulty

11.Setup study step again

12. Request to select question

15. Calling result of remote method

13. Request question according to study step

14. Provide question

Fig. 6 Sequence diagram of EJB component for the relative correction of item difficulty

: Result

Multimed Tools Appl

existing questions. In new question registration, the relative item difficulty set by examiner is recalculated and reapplied according to the learning result of learner for the use of learning system and next learner reflect this when he sets learning level. Figure 5 shows sequence diagram of learner’s mode. After log-in, learner sets learning level before entering learning then questions selected by the relative correction of item difficulty corresponding to learning level are extracted randomly. In case of presenting extracted questions then learning proceeds and learning result by examination paper of learner is represented after progression of learning. By learning result, the relative correction of item difficulty is calculated and reset for presented each item and learning level which each item belongs to is reset in accordance with that result. Figure 6 shows sequence diagram of the inner part of item difficulty component. Registration of component and creation of object come through DifficultyHome (i. e. home interface) then internal methods of the component can be called out through DifficultyRemote (i. e. remote interface). Calculation of the relative correction of item difficulty is requested according to learning result of learner and then, in the component, the relative item difficulty is calculated first through ItemDifficulty. And item guess parameter by guessing factor is calculated through Pcr then the relative correction of item difficulty which subtracts item guess parameter from the relative item difficulty is calculated through Pc. In accordance with the result of the relative correction of item difficulty, learning level of item is reset by the relative difficulty. When setting question is requested, items which are selected by learner and correspond to learning level are extracted. Setting question is provided to learner through result which has the information of extracted items. 3.3 Application of English course E-learning system In this research, we make English course e-learning model system and Fig. 7 shows the course ontology. Course Ontology

Relative Item Difficulty - Very difficult - Very difficult - Difficult - Normal - Easy - Very easy

Learning Units

Unit 1

- The subject - Verbs - Pronouns

Unit 2

- A preposition - A conjunction

Unit 3

- Noun clauses - Nouns and S-V agreement

Unit 4

- Tense - The Passive and the active voice

Unit 5

Fig. 7 Course ontology

Exercises

- The comparative - The subjunctive mood

Unit 1

- Question No. 1 - Example 1 - Example 2 - Example 3 - Example 4 - Solution :

Unit 2

- Question No. 1 - Example 1 - Example 2 - Example 3 - Example 4 - Solution :

: : Unit 5

Multimed Tools Appl

(a) List of question contents

(b) Update and delete screen of question Fig. 8 English e-learning system screen by the relative item difficulty a List of question contents b Update and delete screen of question c Selection screen of learning level d Question screen e Learning result screen

Multimed Tools Appl

(c) Selection screen of learning level

(d) Question screen Fig. 8 (continued)

Registered questions are shown in the list of Fig. 8(a). When examiner clicks learning item’s contents then they can see detailed contents as Fig. 8(b) and can modify and delete relevant item. The learner can access learning system through log-in. After log-in, he/she can select the relative difficulty of presented question to study in learning level selecting screen as Fig. 8(c). When system selects ten questions with the relative difficulty relevant to selected learning level and present them to learner as Fig. 8(d), then he/she can precede learning. The learning result is shown in Fig. 8(e).

Multimed Tools Appl

(e) Learning result screen Fig. 8 (continued)

4 Results and analysis The initial ten times application results of this research system to learners is shown in Fig. 9. Since questions are presented according to learning level selected by learner among total questions, the number of learners (N) and the number of learners who give a right answer (R) is different by each Item. Therefore, the whole values become different. As learning proceeds, 30 times application results are as Fig. 10. That is, according to variation of the number of learners and the number of learners who give a right answer, the relative item difficulty (P), item guess parameter (PGR) and the relative correction of item difficulty (PC) are calculated and learning level (or step) is readjusted on the basis of Table 1.

Fig. 9 Initial ten times application results

Multimed Tools Appl

Fig. 10 Application results for 30 times

From these results, this research shows that learning level for each question is reset by reflecting learning result according to increase the number of times of learning of various learners on the relative correction of item difficulty continuously. Therefore, more relevant items can be presented in learning level selected by learners. 5 Conclusion In this research, we designed and embodied web component based e-learning system considering the relative correction of item difficulty and applied English course learning system. We let learners to select the item difficulty by themselves according to five learning levels (Very Difficult, Difficult, Normal, Easy, and Very easy) and can expect more appropriate learning effect. Especially, we calculated and applied the relative correction of item difficulty considering guessing factor of learners who gave a correct answer. Getting out of existing development process in system development, we embodied core business logic as web component. In the result of application, we could know that the learner was able to study efficiently by select relative item difficulty. It also can change the learning level in the next study by calculate and update item difficulty on the relative correction continuously. As a further research subject, it is necessary to prepare the theoretical basis to verify objectivity about the relative difficulty which is set at the first question registration by examiner. Also, with running parallel with detailed classification works such as learning chapter and so forth, the learner should be able to choose not only learning level but also learning chapter, and subjective questions should be applied in company with objective multiple-choice questions.

References 1. Advanced Distributed Learning Sharable Content Object Reference Model Version 1.2, ADLNet, Advaced Distributed Learning Initiative, 2001 2. Barr A (2006) Revisiting the -ilities: adjusting the distributed learning marketplace. Again Learn Technol Newsl IEEE Comput Soc 8:3–4

Multimed Tools Appl 3. Battle S (2002) Web components: services that wear state on their sleeve. OMG Web-services workshop, Hewlett-Packard 4. Benyon D, Stone D, Woodroffe M (1997) Experience with developing multimedia courseware for the World Wide Web: the need for better tools and clear pedagogy. Int J Hum Comput Stud 47:197–218 5. Birnbaum A (1968) Some latent trait models and their use in inferring an examinee’s ability, Addison-Wesley 6. Chen C-M (2008) Intelligent web-based learning system with personalized learning path guidance. Comput Educ 51:787–814 7. Chen L-H (2010) Web-based learning programs: use by learners with various cognitive styles. Comput Educ 54:1028–1035 8. David FD, Hu X, Eric C, Mathews, Susarla S (2006) Intelligent Delivery of Sharable Content Objects—An Integrated Solution to Enhance Learning, Learning Technology Newsletter, IEEE Computer Society 8 9. Fei YK, Wang ZJ (2004) A Concept Model of Web Components, Proceedings of the 2004 IEEE International Conference on Services Computing 10. Frank Baker B (2001) The basics of Item response theory, ERIC Clearinghouse on Assessment and Evaluation 11. Gord Mackenzie, SCORM 2004 Primer A (Mostly) Painless Introduction to SCORM Version 1.0, McGill 2004 12. Iribarne L (2004) Web components: a comparison between web services and software components. Colombian J Comput 5(1) 13. Jeong HY (2005) Learner’s tailoring E-learning system on the item revision difficulty using petrinet VSMM 2006. LNCS 4270:318–327 14. Liu YC, Chien HW, Huang SH (2004) An Novel Data Management System of Teaching Material Conforming with SCORM, Proceedings of the IEEE International Conference on Advanced Learning Technologies, IEEE computer society 15. Ozkan S, Koseler R (2009) Multi-dimensional students’ evaluation of e-learning systems in the higher education context: an empirical investigation. Comput Educ 53:1285–1296 16. Perrin KM, Mayhew D (2000) The reality of designing and implementing an internet-based course. Online Journal of Distance Learning Administration 13 (4) 17. STAR Early Literacy and Item Difficulty, STAR Early Literacy Application Note pp. 103, March 2002 18. Verbert K, Jovanovic J, Gaševic D, Duval E, Meire M (2005) Towards a global component architecture for learning objects: a comparative analysis of learning object content models, proceedings of world conference on educational multimedia, hypermedia and telecommunications 19. Windsor JA, Diener S, Zoha F (2008) Learning style and laparoscopic experience in psychomotor skill performance using a virtual reality surgical simulator. Am J Surg 195:837–842 20. Zeramdin K, Rekik Y, Gillet D (2004) Enhanced Web Components and Connectors Description for Authoring e-Learning Environments, 5th Int. Conf. on Information Technology Based Higher Education and Training

Hwa-Young Jeong received the M.S. and Ph.D. degrees from Kyunghee University, Seoul, Korea, in Software Engineering in 1994. He has been working as an Assistant Professor from March 2005. He has working experiences in R&D center of Aju System Co., Ltd. (related to developing the FA machine, IC Test Handler of Semiconductor) as a programmer and software engineer from 1994 to 1998 and he also worked as the same position in CAN Research Co., Ltd. in 1998 to 1999. His research interests include Web Engineering, Multimedia Application, A methodology of Software Development, Networks and so on.

Multimed Tools Appl

Bong-Hwa Hong received the M.S. and Ph.D. (major in Computer Architecture) degrees in Electronic Engineering from the Kyunghee University in 1992 and 2001 and Ph.D. degrees in Education from Comberland University, North Carolina, in 2009. From 1997 to 2004, he had been an Assistant Professor in Dept. of computer science at Semyung University, Chechon, Korea. He also had been an Associate Professor in Dept. of Information and Communication of Kyunghee Cyber University, Seoul, Korea from 2004 to 2009. Currently, he is working as an Associate Professor in the Information and Communication, Kyunghee Cyber University, Seoul, Korea. His research interests include Computer Networks, Cyber Education, Digital Contents, Ubiquitous computing.

Bhanu Shrestha is born in Nepal and he received the B.S. M.S. and Ph.D. degrees in Electronic Engineering from Kwangwoon University, Seoul, Korea in 1998, 2004 and 2008 respectively. He is currently working as a Professor and a researcher in the RFIC Center, Electronic Engineering Department at the same University. His main research interests include RFIC/MMIC/HMIC, Computer Networks, and Ubiquitous Computing.

Multimed Tools Appl

Seongsoo Cho received the B.S. degree from Gwangju University in 1993, Gwangju, Korea, and the M.S. degree in Publishing & Magazine from Kyunghee University, Seoul, Korea, in 1996 and Ph.D. degrees in Electronic Engineering (major in Multimedia Contents) from Kwangwoon University, Seoul, Korea, in 2010. Currently, he is working at Kwangwoon University as an Adjunct Professor. His main research interests include Computer Networks, Digital Contents, Ubiquitous computing, Multimedia Editing & 2D, 3D, Designing, and U-electronic Editing & Publishing Contents.