Domain Knowledge Text. An arithmetic logic unit (ALU) is a digital electronic circuit that performs arithmetic and bitwise logical operations on integer binary.
Adaptive Courseware based on Natural Language Processing (AC & NL Tutor) Ani Grubišić, PI Faculty of Science, University of Split, Croatia
Slavomir Stankov* and Branko Žitko, Co-PIs Faculty of Science, University of Split, Croatia *retired full professor from Faculty of Science, University of Split, Croatia
ONR Cognitive Science of Learning Program Review October 2015 1
OBJECTIVE • Develop Adaptive Courseware & Natural Language Tutor (AC & NL Tutor) • Develop cost effective tutor • semiautomatic authoring tool for knowledge extraction from variety of sources
• Develop tutor that adapts • build mechanisms that enable tutors to modify their pedagogy based on learning about learners’ prior knowledge
• Develop tutor that communicates on natural language • support natural language processing between learner and tutor
• Develop pedagogical framework and learning scenario for approximate solution of challenging 2-sigma problem • learning analytics and educational data mining enables identification and quantification of successful components that promote learning 2
TECHNICAL APPROACH
The AC & NL Tutor Overview Authoring Tool
Domain Knowledge Text
ITS
Stereotype + Learning Records
Domain Knowledge Graph
Expert / Teacher
Learning Analytics
Learner
3
TECHNICAL APPROACH
Build an authoring tool • Text preprocessing • Text analytics • Knowledge extraction
Domain Knowledge Text An arithmetic logic unit (ALU) is a digital electronic circuit that performs arithmetic and bitwise logical operations on integer binary numbers. This is in contrast to a floating-point unit (FPU), which operates on floating point numbers. An ALU is a fundamental building block of many types of computing circuits, including the central processing unit (CPU) of computers, FPUs, and graphics processing units (GPUs). A single CPU, FPU or GPU may contain multiple ALUs.
ALU | is | digital electronic circut ALU | performs | arithmetic operations ALU | performs | bitwise logical operations ALU | operates on | integer binary numbers FPU | operates on | floating point numbers CPU | is | computing circuit FPU | is | computing circuit GPU | is | computing circuit CPU | contains | ALU FPU | contains | ALU GPU | contains | ALU
4
TECHNICAL APPROACH
Design Learner Model • Combination of: • Stereotype learner model • Bayesian learner model • Overlay model
5
• Stereotypes + Bloom’s knowledge taxonomy Knowledge stereotypes
Grade
Stereotype
Recall
Comprehension
Application
Analysis, synthesis and evaluation
1/E,F
Novice
maybe
none
none
none
2/D
Beginner
mostly
maybe
none
none
3/C
Intermediate
mostly
mostly
maybe
none
4/B
Advanced
mostly
mostly
mostly
maybe
5/A
Expert
mostly
mostly
mostly
mostly
• Bayesian learner model
• Set evidence after test: CPU =T, arithmetic operation=T, integer binary number=F, FPU=T
• Exclude from learning all relations where student knows both concepts with probability ≥ 75% ALU | is | digital electronic circut ALU | performs | arithmetic operations ALU | performs | bitwise logical operations ALU | operates on | integer binary numbers FPU | operates on | floating point numbers CPU | is | computing circuit FPU | is | computing circuit GPU | is | computing circuit CPU | contains | ALU FPU | contains | ALU GPU | contains | ALU
7
• Overlay missing conceptions
Student knowledge knowledge –– Student beforelearning learning after
Expert knowledge
8
Student knowledge – Student knowledge – after before learning learning
Expert knowledge
misconceptions
9
TECHNICAL APPROACH
Develop Adaptive and Dynamic Courseware • Courseware elements generation • Courseware elements selection and sequencing • Courseware elements presentation
10
Automatic and dynamic Graph unit label u
LCE level l
ALU 1 FPU 2
Lesson 3 Lesson 3
CPU
CPU 3
GPU
GPU 4
LCE root
ALU FPU
Length of path Concepts from included in central the LCE root p
of courseware elements
Rank
0
5
(1,0,3)
0
7
(2,0,3)
Lesson 3
0
6
(3,0,3)
Lesson 3
0
6
(4,0,3)
11
Adaptive and dynamic and of courseware elements LCE root
Graph unit label u
LCE level l
Length of path from central root p
ALU
ALU 1
Lesson 3
0
FPU
FPU 1
Lesson 3
0
CPU
CPU 3
Lesson 3
0
Test
Test
Concepts included in the LCE
ALU digital electronic circle arithmetic operation bitwise logical operation integer binary operation FPU floating point number computing circle ALU digital electronic circle arithmetic operation bitwise logical operation CPU computing circle ALU digital electronic circle arithmetic operation bitwise logical operation GPU
Rank
(1,0,3)
(2,0,3)
(3,0,3)
computing circle GPU
GPU 4
Lesson 3
0
ALU digital electronic circle arithmetic operation bitwise logical operation
…
(4,0,3)
Adaptive and dynamic
of courseware elements
Develop Natural Language Communication Natural Language Generation An arithmetic logic unit (ALU) is a digital electronic circuit that performs arithmetic and bitwise logical operations on integer binary numbers
Natural Language Understanding TUTOR: What is Arithmetic Logic Unit (ALU)? USER: ALU is digital electronic circuit. TUTOR: Yes, ALU is digital electronic circuit which performs? USER: arithmetic operations and logical operations TUTOR: What kind of logical operations ALU performs? USER: bitwise TUTOR: Does ALU performs operations on floating point numbers? USER: no
13
Tutoring frame in AC & NL Tutor Tutor plans and asks auxiliary question S4
S1
S2
S3
S5
Tutor asks main question
Student answers
Tutor evaluates answer and states about correctness
Tutor estimates student's understanding
14
TECHNICAL APPROACH
Learning and teaching process in the AC & NL Tutor
15
TECHNICAL APPROACH
Learning cycles in the AC & NL Tutor
Final stereotype
Final stereotype expert
Learning and teaching
Initial test
Stereotype determination
Test
16
AC&NL Tutor
Initial Testing
Learner Teacher Cycle 1
Stereotype
Learning
Testing
Learning ……………… ……………… ……………… ……………… ……………… ……………… ……………… .. ……………..
Determination
Testing ……………… ……………… ……………… ……………… ……………… ……………… ……………… .. ……………..
Learning Analytics
Cycle 2 Learning
Testing
Learning ……………… ……………… ……………… ……………… ……………… ……………… ……………… .. ……………..
Testing ……………… ……………… ……………… ……………… ……………… ……………… ……………… .. ……………..
... Cycle N
17
TECHNICAL APPROACH
Detail process model Learning Object
Domain Knowledge Text
Domain Knowledge Graph
AUTH
INIT
NL Understanding
Representative Sample Algorithm
LEARN, TEST
NL Generation
Testing Object
NL Generation
Dialog Object
LEARN Domain Knowledge Subgraph
INIT, TEST
Courseware Generation
Evaluation Learner's Answers
NL Understanding
Test Result
Dialog Result
Stereotype + Bayesian Learner Model
Diagnosis
Learning Record
18
PEDAGOGICAL ASPECT What is being taught?
How is being taught?
Who is being taught?
Learner Knowledge State
Learner Knowledge State
Didactic phase Learner’s actual knowledge level Perception and diagnostic phase
Level of learner’s knowledge (Bloom taxonomy) Misconceptions and missing conceptions
Adaptation phase
Expert knowledge state Minimize diferences between learner’s and the domain knowledge
19
EMPIRICAL ASSESMENT • Develop a better understanding of typical sentence forms in relation with the Bloom’s knowledge taxonomy. • Develop a better understanding of typical question forms in relation with the Bloom’s knowledge taxonomy. • Examine the effects and benefits of AC-ware Tutor, CoLaB Tutor and AC & NL Tutor for learning and teaching process.
20
CONCLUSIONS & NAVAL RELEVANCE
• We will develop an effective and efficient intelligent tutoring system that targets learners from all educational levels: elementary school, high school, undergraduate, graduate and postgraduate university learners, as well as, navy recruits. • This tutor will allow cost-effective deployment and support of intelligent tutoring systems and improve effectiveness of learning and teaching process • This tutor will improve the training effectiveness for the Navy 21
CURRENT AND FUTURE WORK Year 1 1
2
Year 2 3
4
5
6
7
8
Year 3
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36
Prep Project setup Design software model Task A. Build the AC & NL Tutor Sub-task 1: Build an authoring tool Integrate tools for text preprocessing Design/Build text classifier Design Domain Knowledge Graph Design/Build knowledge extraction Design/Build domain knowledge sampler Design/Build domain knowledge reasoner Test Sub-task 2: Design Learner Model Design Learner model (overlay + stereotype + Bayes) Design/Build Learners model diagnosys Test Sub-task 3: Develop Adaptive and Dynamic Courseware Design Courseware object Design Learning Record Design/Build Courseware generation Test Sub-task 4: Develop Natural Language Communication Design/Build NL generation Design/Build NL understanding Test Task B. Pedagogical aspect with learning analytic Instructional Design Model Scenario for learning and teaching Task C. Empirical assessment CoLab Tutor AC ware tutor AC&NL tutor Task D. Creating reports
22
