Application of Paraconsistent Logic in an Intelligent

Application of Paraconsistent Logic in an Intelligent Tutoring System Sylvia Encheva1 , Sharil Tumin2 and Maryna Z. Solesvik1 1

Stord/Haugesund University College, Bjørnsonsg. 45, 5528 Haugesund, Norway [email protected], [email protected] 2 University of Bergen, IT-Dept., P. O. Box 7800, 5020 Bergen, Norway [email protected]

Abstract. This paper describes application of six-valued logic in an assessment sub-system of an intelligent tutoring system. This allows the system to handle situations with inconsistent and incomplete input. Special attention is paid to the decision making rules an intelligent agent is applying for assessing students’ understanding of new terms. If an assessment is not in anyway connected with grading, i.e. a student will neither loose nor gain by providing honest answers, we assume that there is no need for penalty rules deterring system misuse. Instead we implement rules that distinguish between students’ hesitation in the process of giving an answer and lack of knowledge.

Keywords: intelligent infrastructures and automated methods, logic

1

Introduction

Most automated assessment tests are based on binary logic, i.e. an answer chosen by a student is either correct or incorrect and that implies possesion or lack of knowledge. The assigned score in a case of an incorrect answer can be either a zero, which does not distinguish an incorrect answer from a missing answer, or a negative score. Penalty marking is applied for preventing guessing. However, according to [8] one should ask whether ’educated’ guess is something to be discouraged since professional people use educated guesses in all fields of activity and by [17] 30% of the students choose the correct answer for wrong reasons. Some systems reward partially correct answers in a sense that a part of an answer is correct and another part is either missing or incorrect. The assigned score is related to the significance of missing or incorrect part of the answer. However different, the above mentioned approaches have one thing in common, they all use binary logic while making conclusions about student’ knowledge. Thus any of the above mentioned approaches does not treat incomplete or inconsistent information. In this paper we discuss how to assess students’ understanding of new terms, shortly after they have been introduced in a subject. Application of six-valued logic allows the system to handle situations with inconsistent and/or incomplete input. Special attention is paid to the decision making rules an intelligent

agent is applying for assessing students’ understanding of new terms. If an assessment is not in anyway connected with grading, i.e. a student will neither loose nor gain by providing honest answers, we assume that there is no need for penalty rules deterring system misuse. Instead we implement rules that distinguish between students’ hesitation in the process of giving an answer and lack of knowledge. Such rules are used only for the intelligent assessment sub-system. This work presents futher development a response-driven Web-based assessment system enhancing learning [3]. The original system contains tests assessing recall of facts, high-level thinking, asking students to evaluate consequences and draw conclusions, and various quizzes. The system responds to students’ needs as they progress through the system. This is incorporated by providing a structured learning system environment, suitable for offering automatic help through logic-based and qualitative reasoning mechanisms and guiding students in their navigation. The rest of the paper is organized as follows. Related work and statements from many-valued logic may be found in Section 2 and Section 3 respectively. The main results of the paper are placed in Section 4. The system architecture is described in Section 5 and our experience using the system in Section 6. The paper ends with a conclusion in Section 7.

2

Related Work

Inspired by the Aristotle writing on propositions about the future - namely those about events that are not already predetermined, Lukasiewicz has devised a three-valued calculus whose third value, 12 , is attached to propositions referring to future contingencies [10]. The third truth value can be construed as ’intermediate’ or ’neutral’ or ’indeterminate’ [16]. Another three-valued logic, known as Kleene’s logic is developed in [9] and has three truth values, truth, unknown and false, where unknown indicates a state of partial vagueness. These truth values represent the states of a world that does not change. A brief overview of a six-valued logic, which is a generalized Kleene’s logic, has been first presented in [11]. The six-valued logic was described in more detail in [6]. In [4] this logic is further developed by assigning probability estimates to formulas instead of non-classical truth values. Two kinds of negation, weak and strong negation are discussed in [18]. Weak negation or negation-as-failure refers to cases when it cannot be proved that a sentence is true. Strong negation or constructable falsity is used when the falsity of a sentence is directly established. A level-based instruction model is proposed in [12]. A model for student knowledge diagnosis through adaptive testing is presented in [7]. An approach for integrating intelligent agents, user models, and automatic content categorization in a virtual environment is presented in [15].

3

Preliminaries

For describing six-valued logic we use notations as in [6]. Thus – t denotes true - it is possible to prove the truth of the formula (but not its falsity) – f denotes false - it is possible to prove the falsity of the formula (but not its truth) – ⊥ denotes unknown - it is not possible to prove the truth or the falsity of the formula (there is not enough information) – ⊥t denotes unknownt - intermediate level of truth between ⊥ and t – ⊥f denotes unknownf - intermediate level of truth between ⊥ and f – ⊤ denotes contradiction - it is possible to prove both the truth and the falsity of the formula In other words the six-valued logic distinguishes two types of unknown knowledge values - permanently or eternally unknown value ⊤ and a value ⊥ representing current lack of knowledge about a state [5]. The epistemic value of formula when it is known that the formula may take on the truth value t is denoted by ⊥t and by ⊥f when it is known that the formula may take on the truth value f.

knowledge

false

true

contradiction

unknown

unknown t

f

unknown

truth

Fig. 1. Knowledge lattice

A lattice [2] showing a partial ordering of the elements f, ⊥f , ⊤, ⊥t , ⊤, t by degree of knowledge is presented in Fig. 1. The knowledge lattice illustrates how the truth value of a formula that has a temporary truth value can be changed as more knowledge becomes available. Suppose a sentence has a truth value ⊥f at one point of time and f at another. Its truth value is then determined as f, i.e. the system allows belief revision as long as the revision takes place in an incremental knowledge fashion. Below is a truth table for the six-valued logic as shown in [5].

∧ t f ⊤ ⊥t ⊥f ⊥

4

t t f ⊤ ⊥t ⊥f ⊥

f ⊤ f ⊤ f f f ⊤ f ⊤ f ⊥f f ⊥f

⊥t ⊥t f ⊤ ⊥t ⊥f ⊥

⊥f ⊥f f ⊥f ⊥f ⊥f ⊥f

⊥ ⊥ f ⊥f ⊥ ⊥f ⊥

Application of Six-Valued Logic

In this scenario we consider a multiple-choice test (MCT) assessing student’s understanding of new terms. The test consists of two questions. According to the result of a MCT, understanding of a term is achieved if a student gives a correct answer to each of the two questions about that term. Such tests are placed after a new term has been introduced in the theoretical part of the system. Modus ponens (P,PQ→Q) can be applied if there is no doubt about the truthstatus of P . Most tests are based on the understanding that a ’correct answer to one question about a term’ (P ) implies ’understanding of that term’ (Q). In our view a single question is not enough, since a student can get a correct answer by guessing. To minimize this we propose use of two questions. What should an intelligent agent do in a situation where one of the questions receives a correct answer and the other one receives a wrong answer or is not answered? The binary logic does not provide a model for sets like {correct answer, incorrect answer} or {correct answer, no answer is provided}. Such a situation can be resolved by applying six-valued logic. Based on the truth table for six-valued logic in Section 3 we propose the following: – Two correct answers imply understanding of that particular term. The assigned truth-value is t. The process of questioning is terminated. – One correct answer and one unanswered question imply some doubt about the student’s understanding of that particular term. The assigned truthvalue is ⊥t . The system first provides additional explanations and then suggests to the student to answer one new question taken from the database. – One correct answer and one incorrect answer imply doubt about the student’s understanding of that particular term. The assigned truth-value is ⊤. The system first provides additional explanations and then suggests to the student to answer two questions - one new question taken from the database and the question from the first trial that received an incorrect answer. – Two unanswered questions imply doubt about the student’s understanding of that particular term. The assigned truth-value is ⊥. The system first provides additional explanations and then suggests two new questions taken from the database. – One incorrect answer and one unanswered question imply doubt about the student’s understanding of that particular term. The assigned truth-value is

⊥f . The system first provides additional explanations and then suggests to the student to answer the same questions. – Two incorrect answers imply lack of understanding of that particular term. The assigned truth-value is f. The system first provides additional explanations and then suggests to the student to answer the same questions plus one new question taken from the database. If the second set of responces contains an incorrect answer and/or unanswered questions the system advises the student to work more with the originally provided learning materials and terminates the automated questioning process. We believe that several rounds of questioning would make the learning process time consuming for the student and thus disturb the learning flow. However, the student can start a new assessment of his/her understanding of that particular term at any time he/she wants.

5

System Architecture

In our earlier work [3] relations between wrong answers and understanding status of a student were implemented in tables of MCTs’ distracters and their implications. These static bindings were done early at the implementation stage by the expert human tutor based on her knowledge and teaching experience of the subject matter. Each MCT was expertly crafted as to build tables of inferences to be used by intelligent diagnostic agents. By employing six-valued logic in an assessment sub-system, we are able to postpone these binding between students’ responses to tests and their understanding status at a later stage. This greatly reduces the difficulty of making/choosing appropriate distracters for the MCT. Instead of a wrong answer leading to a consequence that was hard coded in an inference table as in [3], truth value/consequence relations among the six truth values as discussed in Section 4 are used to control the next step in the process of a student learning activities. The complexity of managing inference tables is reduced to only managing six inferences for each test. The system is implemented using the so-called LAMP Web server infrastructure and deployment paradigm. It is a combination of free software tools of an Apache Web server, a database server and a scripting programming platform on a Linux operating environment. The three-tiers Web deployment in our system is 1) an Apache front end Web server, 2) a Python based application middleware for dynamic content, data integration and users’ software agents and 3) a back end PostgreSQL database server for data store of both static and dynamic data. Behind this traditional three-tiers Web deployment is a service support subsystem. Communication framework based on XML-RPC is used to connect the Web application middle-ware and the intelligent assessment/diagnostic system together. The separation of these two units made it possible to modularly design and implement the system as loosely couple independent sub-systems. An authenticated user receives a unique session key that is used to identify user in the system for that particular session. This session key is saved in the

Web Server

Web Clients

Application middleware

Datastores

Intelligent Diagnostics

Users Authenticator

Internet

Users Stack Profiler

Student

Software agents

Database

Intelligent assessment

XML-RPC

Dynamic Page Publisher

Learning Units

Fig. 2. System architecture

user Web browser cookie. All sessions’ dynamic data stored in the database are indexed using this session key. A session key is used to index state variables assigned to a user for that particular session. The session key cookie and dynamic data stored are used together to keep user’s states of interaction with the system within otherwise a stateless HTTP protocol. The dynamic page publisher compiles a page to be presented to the user from a template file in relation to the user response, current state variables and activities history. A template file contains the static declarations of a document. The variables in a particular template files are given values by the dynamic page publisher module during the production of an HTML document. The resulting HTML document is sent back to the user Web browser. This module also acts as a handler when a user requests a page or sends a form back to the Web server. Each learning unit is atomic, self-contained and reusable. The dynamic page publisher makes use of these learning units to provide students with a dynamic and personalized learning material as a direct reaction to students’ interaction to the system. The user authenticator authenticates a user during login and creates initial session contact in the system if the user provides correct credentials. It is realized by creating a unique session key and saving that key in the database of the server and a cookie in the client. The module also provides user authorization during an active user’s session. The user authenticator module is responsible for session cleanup at user’s logoff. The users stack profiler keeps track of user activities history in a stack like data structure in the database. Each event, like for example response/result of a test or a change of learning flow after following a hint given by the system, is stored in the database. This module provides the percepts to the intelligent mod-

ules of the software agents sub-system. The users stack profiler communicates directly with the agents by sending messages over the XML-RPC communication channel. By using some common data stored in the database, the users stack profiler indirectly affects the behavior of the user’s agents and vise verse. The application middleware and the software agents run independently of each other. As such, they can be situated on different servers. The middleware implement the Web side of the system while the software agents implement the decision side of users learning process. Given a certain response to a particular test at a particular user state, what best action can be taken to increase the probability that the user will learn a particular unit of knowledge? This decision is done by the intelligent diagnostics agent. The intelligent assessment agent is a special agent that implements six-valued logic table of inference for testing students’ understanding of a particular basic term by using a pair of questions after the term has been introduced. The agent does an early diagnostic about absorption of knowledge. A response given by a particular student from a test will give the system an indication about the state of learning of a particular term. Each of the six different truth values of a response triggers different rule-based reaction as discussed in Section 4. This agent helps to implement a part of an intelligent tutoring system, which differ from the intelligent diagnostics agent. The intelligent assessment agent facilitates students’ early absorption/assimilation of new terms.

6

Experience Using the Intelligent Assessment Sub-System

The intelligent assessment sub-system was introduced in the beginning of 2006. A group of 114 undergraduate engineering students enrolled in a calculus course have been using the original version of the system during the Fall semester 2005 and the extended with an intelligent assessment sub-system version during the Spring semester 2006. Another group of 111 undergraduate engineering students enrolled again in a calculus course have been using the extended version during the Fall semester 2006. Both groups expressed clear satisfaction using the intelligent assessment sub-system. The first group was comparing the two versions and stated that the extended version was very helpful for self-evaluation of recently introduced terms which in turn contributed to a faster learning process.

7

Conclusion

This paper presents an intelligent sub-system assessing students initial understanding of new terms. The decision making process is based on a six-valued logic. Our motivation for employing six-valued logic is that this way the system will provide better personalized help to the students and course builders will receive more detailed information about the effectiveness of their learning materials. We believe that assessment rules for students’ understanding of new

terms, shortly after they have been introduced, should differ from the ones used for marking students’ achievements in a subject.

References 1. Belnap, N.J.: How a computer should think. In Contemporary Aspects of Philosophy. Proceedings of the Oxford International Symposia, Oxford, GB, (1975) 30–56 2. Davey, B. A. and Priestley, H. A.: Introduction to lattices and order. Cambridge University Press, Cambridge (2005) 3. Encheva, S., Tumin, S.: Responce-Driven Web-Based Assessment System. Proceedings of KES 2005, Australia, Lecture Notes in Computer Science, 3683, (2005) 749–755 4. Fitting, M.: Kleene’s Logic, Generalized. Journal of Logic and Computation, 1(6), (1991) 797–810 5. Garcia, O. N., Moussavi, M.: A Six-Valued Logic for Representing Incomplete Knowledge. Proceedings of ISMVL, (1990) 110–114 6. Garca-Duque, J., Lpez-Nores, M., Pazos-Arias, J., Fernndez-Vilas A., Daz-Redondo, R., Alberto Gil-Solla, Blanco-Fernndez, Y., Ramos-Cabrer, M.: A Six-valued Logic to Reason about Uncertainty and Inconsistency in Requirements Specifications. Journal of Logic and Computation, 16(2), (2006) 227–255 7. Guzm` an, E., Conejo, R.: A model for student knowledge diagnosis through adaptive testing. Lecture Notes in Computer Science, 3220, Springer-Verlag, Berlin Heidelberg New York, (2004) 12–21 8. Johnstone, A.: Effective Practice in Objective Assessment. www.physsci.heacademy.ac.uk/Publications/PracticeGuide/ 9. Kleene, S.: Introduction to Metamathematics. D. Van Nostrand Co., Inc., New York, N. Y., (1952) 10. Lukasiewicz, J.: On Three-Valued Logic . Ruch Filozoficzny, 5, (1920), English translation in Borkowski, L. (ed.) 1970. Jan Lukasiewicz: Selected Works. Amsterdam: North Holland, (1920) 11. Moussavi, M., Garcia, O. N.,: A Six-Valued Logic and its application to artificial intelligence. Proceedings of the Fift Southeastern Logic Symposium, (1989) 12. Park, C., Kim, M.: Development of a Level-Based Instruction Model in Web-Based Education. Lecture Notes in Artificial Intelligence, 3190, Springer-Verlag, Berlin Heidelberg New York, (2003) 215–221 13. Priest, G., An Introduction to Non-Classical Logic, Cambridge, (2001) 14. Prior, A.N.: A Statement of Temporal Realism. In Copeland, B.J. (ed.) Logic and Reality: Essays on the Legacy of Arthur Prior. Oxford: Clarendon Press, (1996) 15. Santos, C.T., Os` orio, F.S.: Integrating intelligent agents, user models, and automatic content categorization in virtual environment. Lecture Notes in Computer Science, Vol. 3220. Springer-Verlag, Berlin Heidelberg New York, (2004) 128–139 16. Sim, K.M.: Bilattices and Reasoning in Artificial Intelligence: Concepts and Foundations. Artificial Intelligence Review, 15(3), (2001) 219–240 17. Tamir, P.: Justifying the selection of answers in multiple-choice items. International Journal of Science Education, 12, (1990) 563–573 18. Wagner, G.,: Vivid Logic: Knowledge Based reasoning with two kinds of negation. Lecture Notes in Artificial Intelligence, 764, Springer-Verlag, Berlin Heidelberg New York (1994)