The present thesis aim at making contributions to the fields of computer support collaborative learning and work (CSCL, CSCW) on three main levels. First, as an ...
Peer Knowledge Modeling in Computer Supported Collaborative Learning
THÈSE NO 4366 (2009) PRÉSENTÉE le 17 avril 2009 INFORMATIQUE ET COMMUNICATIONS Recherche et appui pour la formation et ses technologies SECTION D'INFORMATIQUE
ÉCOLE POLYTECHNIQUE FÉDÉRALE DE LAUSANNE POUR L'OBTENTION DU GRADE DE DOCTEUR ÈS SCIENCES
PAR
Mirweis Sangin
acceptée sur proposition du jury: Prof. S. Süsstunk, président du jury Prof. P. Dillenbourg, Dr G. Molinari, directeurs de thèse Prof. F. Fischer, rapporteur Dr D. Gillet, rapporteur Prof. S. Teasley, rapporteur
Lausanne, EPFL 2009
3
Abstract Learners benefit from collaboration because it triggers effective interaction processes such as externalization, elicitation and negotiation of knowledge. In order to communicate effectively, learners need to have a certain representation of their peers’ knowledge. We refer to the process of building and maintaining a representation of the peers’ knowledge as peer knowledge modeling. The present thesis aim at making contributions to the fields of computer support collaborative learning and work (CSCL, CSCW) on three main levels. First, as an empirical contribution, we investigate the process of peer knowledge modeling in the context of CSCL. Our main research question inquires the effects of a socio-cognitive support, providing co-learners with cues about their peer’s prior knowledge, on collaborative learning outcomes and processes. In an empirical study (the KAT experiment), university students (N=64) participated in a remote computer-mediated dyadic learning scenario. They were provided (or not) with a visual representation of their partner’s prior-knowledge level through a Knowledge Awareness Tool (KAT). Results showed that the KAT enhances co-learners’ collaborative learning gain. This effect appears to be mediated by the positive effect of the KAT on participants’ accuracy in estimating their peer’s knowledge. Analyses on the process level showed that participants of the KAT condition produce more elaborated utterances. KAT condition dyads’ interactions are more focused on knowledge negotiation, whereas the control condition dyads are mainly focused on task completion. The KAT seems to provide a sensitizing metacognitive support, structuring and regulating the collaboration by helping co-learners to cope with their knowledge gaps and discrepancies. Second, as a methodological contribution, we examine the affordance of dual eye tracking techniques as an innovative methodology to investigate, on a deeper level, the socio-cognitive processes underlying collaboration. We introduce DUET (DUal Eye-Tracking), a method using a multimodal technique to collect rich data featuring peers’ synchronized gaze patterns, verbal interaction and potentially activities. We examine the main research applications of DUET and exemplify them with analyses conducted in the context of the KAT experiment. DUET method appears to be a promising technique to investigate collaborative processes on a deep level. Finally, as a third computational contribution, we built upon micro-level analyses of the verbal referencing process to introduce and test REGARD, a computational model allowing to automatically detect verbal references and locate the specific object of reference. The results of the test show a reasonably good accuracy of the REGARD algorithm to detect and associate verbal references to objects of reference. We discuss the design and research application of the REGARD model for the field of CSCL, CSCW and more generally HCI. Keywords: CSCL, CSCW, HCI, eye tracking technologies, peer knowledge modeling, mutual modeling, dual eye tracking, gaze-aware interface.
5
Résumé Les apprenants bénéficient de la collaboration car elle induit des processus d’interaction comme l’externalisation, l’élicitation et la négociation de connaissance. Pour communiquer de manière effective, les apprenants doivent avoir une certaine représentation de la connaissance de leur partenaire. Nous nous référons au processus de création et maintien de cette représentation de la connaissance du partenaire par la terminologie modélisation de la connaissance du partenaire. Cette thèse contribue aux champs de recherche et d’application du travail et de l’apprentissage collaboratif médiatisés par l’ordinateur (CSCL, CSCW) à trois différents niveaux. Premièrement, comme contribution empirique, nous examinons le processus de modélisation de la connaissance du partenaire dans le contexte de CSCL. Notre question de recherche principale s’enquière de l’effet, d’un support socio-cognitive fournissant des indices quant au niveau de connaissance du partenaire, sur les performances et les processus d’apprentissage collaboratif. Dans une investigation empirique, des étudiants universitaires (N=64) ont participé à un scénario d’apprentissage dyadique médiatisé par l’ordinateur. Ils ont eu à leur disposition (ou non), une représentation graphique du niveau de connaissance préalable de leur partenaire, à travers un outil de conscience de connaissance (Knowledge Awareness Tool, KAT). Les résultats démontrent que le KAT augmente le gain d’apprentissage collaboratif. Cet effet est médiatisé par le fait que le KAT a un impact positif sur la précision des apprenants quant à l’estimation de la connaissance de leur partenaire. Les analyses au niveau des processus démontrent que les participants de la condition avec KAT produisent plus d’énoncés élaborés et sont plus focalisés sur la négociation de connaissance, tandis que les apprenants de la condition contrôle sont principalement focalisés sur la complétion de la tâche. Ainsi, le KAT semble avoir un effet de sensibilisation metacognitive qui structure et régule la collaboration en aidant les apprenants à mieux faire face aux déficits et aux différences de leur connaissance préalables. Deuxièmement, comme contribution méthodologique, nous examinons l’affordance d’une méthode innovatrice d’oculométrie synchronisée (dual eye tracking) pour l’étude approfondie des processus socio-cognitive sous-jacents à la collaboration. Nous introduisons DUET, une méthode basée sur des techniques multimodales permettant de collecter des données sur les mouvements de regard synchronisés, les interactions verbales et potentiellement les activités des collaborants. Nous discutons les principaux axes d’applications de la méthode DUET pour la recherche, exemplifiés par des analyses portant sur les données récoltées dans le cadre de l’expérience KAT. Les investigations démontrent que DUET semble être une méthode prometteuse dans le cadre de l’investigation approfondie des processus collaboratifs. Finalement, comme troisième contribution d’ordre computationnelle, basé sur la micro-analyse du processus de référenciation verbale, nous introduisons REGARD, un modèle computationnel permettant de détecter automatiquement des références verbales et la localisation des objets de
6
référence. Le test du modèle démontre une performance prometteuse en terme de détection de références verbales et leur association aux objets de référence correspondants. Nous discutons le potentiel d’application de ce modèle dans le cadre du CSCL et CSCW. Mots-clés : CSCL, CSCW, IHM, technologies d’oculométrie, modélisation de la connaissance du partenaire, modélisation mutuelle, interface gaze-aware.
9
Table of content INTRODUCTION ......................................................................................................................... 15 CHAPTER 1
COMPUTER SUPPORTED COLLABORATIVE LEARNING ............................. 19
1.1 Collaborative Knowledge Construction ............................................................................ 20 1.1.1 Socio-constructivist approach of collaborative learning ............................................... 20 1.1.2 Cultural and historical approaches of collaborative learning ...................................... 21 1.1.3 Distributed cognition....................................................................................................... 23 1.2 Computer Supported Collaborative Learning.................................................................. 24 1.2.1 A brief history of CSCL ................................................................................................... 24 1.2.2 The interaction paradigm................................................................................................ 26 1.2.3 The process variables: verbal interactions .................................................................... 28 1.3 Structuring and regulating interaction in CSCL ............................................................. 34 1.3.1 Structuring approaches ................................................................................................... 34 1.3.2 Regulating approaches.................................................................................................... 36 1.4 Summary ............................................................................................................................... 38
CHAPTER 2
MUTUAL KNOWLEDGE MODELING .............................................................. 41
2.1 Social and cognitive approaches to mutual modeling ...................................................... 42 2.1.1 Social model: the attribution theory ............................................................................... 42 2.1.2 Cognitive model: theory of mind.................................................................................... 43 2.2 Psycholinguistic approach to mutual modeling ................................................................ 47 2.2.1 The common ground and the grounding process ........................................................... 47 2.2.2 Audience design and perspective taking ......................................................................... 50 2.3 Mutual modeling in CSCL .................................................................................................. 55 2.3.1 Intersubjectivity ............................................................................................................... 55 2.3.2 Mutual modeling.............................................................................................................. 56 2.3.3 Peer knowledge modeling ............................................................................................... 57 2.4 Mutual knowledge modeling: a functional model ........................................................... 59 2.4.1 What is mutual modeling?............................................................................................... 59 2.4.2 Why do we model others? ............................................................................................... 60 2.4.3 How do we model others? ............................................................................................... 60 2.4.4 Functional model of peer knowledge modeling.............................................................. 61
CHAPTER 3
KNOWLEDGE AWARENESS TOOLS .............................................................. 63
3.1 Awareness Tools in CSCW and CSCL .............................................................................. 63 3.1.1 Definition of Awareness .................................................................................................. 63 3.1.2 Awareness tools in CSCW and CSCL ............................................................................. 66 3.2 Knowledge awareness in CSCL .......................................................................................... 69 3.2.1 Activity-based knowledge awareness.............................................................................. 69
10
3.2.2 Subjective knowledge awareness .................................................................................... 70 3.2.3 Objective assessment-based awareness tools ................................................................. 74 3.3 Conclusions............................................................................................................................ 75
CHAPTER 4
EYE-TRACKING TECHNOLOGIES IN HCI..................................................... 77
4.1 Visual Attention and eye-movements ................................................................................. 77 4.2 Eye-tracking technologies .................................................................................................... 78 4.3 Eye-tracking application in cognitive science and HCI ................................................... 79 4.3.1 Gaze as dependent measure ............................................................................................ 80 4.3.2 Gaze as a interactive input.............................................................................................. 80 4.4 Joint attention and dual eye tracking ................................................................................ 81 4.4.1 Coordination of attention ................................................................................................ 81 4.4.2 DUET: DUal Eye-Tracking method ............................................................................... 83 4.5 Summary ............................................................................................................................... 86
CHAPTER 5
THEORETICAL FRAMEWORK AND RESEARCH QUESTIONS ........................ 87
5.1 Summary of theoretical considerations ............................................................................. 87 5.1.1 Definition of collaborative learning ............................................................................... 87 5.1.2 The “interactions pattern” model................................................................................... 87 5.1.3 Structuring and regulating .............................................................................................. 90 5.1.4 The importance of prior-knowledge ............................................................................... 90 5.1.5 The importance of mutual knowledge modeling............................................................. 91 5.2 The theoretical framework .................................................................................................. 92 5.3 Objectives of the empirical research .................................................................................. 92 5.4 Research questions and hypotheses.................................................................................... 93 5.4.1 The main effect of the KAT on learning outcomes ......................................................... 93 5.4.2 Differential effect of the KAT on asymmetric pairs ........................................................ 94 5.4.3 The effects of the KAT on the verbal interaction process variables .............................. 95 5.4.4 Relation between outcomes and verbal interaction ....................................................... 96 5.4.5 Prior knowledge asymmetry and verbal interactions..................................................... 97 5.4.6 The effects of KAT on knowledge construction actions.................................................. 97 5.4.7 Dual gaze analyses .......................................................................................................... 97
CHAPTER 6
METHODS OF THE EMPIRICAL STUDY ....................................................... 101
6.1 Participants and design ...................................................................................................... 101 6.2 Instructional material and technical setup ...................................................................... 102 6.2.1 Instructional Material ................................................................................................... 102 6.2.2 Technical setup .............................................................................................................. 102 6.3 Procedure............................................................................................................................. 102 6.3.1 Prior knowledge verification ........................................................................................ 103 6.3.2 Individual learning phase ............................................................................................. 103 6.3.3 Pretest ............................................................................................................................ 104
11
6.3.4 Collaborative concept-mapping phase ......................................................................... 104 6.3.5 Posttest........................................................................................................................... 104 6.3.6 Knowledge estimation test ............................................................................................ 105 6.4 Experimental conditions and dependant variables ........................................................ 105 6.4.1 Experimental conditions................................................................................................ 105 6.4.2 Dependant variables ..................................................................................................... 105 6.5 Coding of verbal interaction ............................................................................................. 108 6.5.1 Domain knowledge categories ...................................................................................... 108 6.5.2 Mutual knowledge modeling categories ....................................................................... 112 6.5.3 Collaboration management categories......................................................................... 114 6.5.4 Non-coded categories .................................................................................................... 114 6.5.5 Inter-judge reliability .................................................................................................... 114 6.6 Summary ............................................................................................................................. 114
CHAPTER 7
THE EFFECTS OF KAT ON LEARNING OUTCOMES.................................. 117
7.1 Introduction ........................................................................................................................ 117 7.2 Effects of the KAT on the dependent variables .............................................................. 118 7.2.1 Descriptive analyses and treatment checks .................................................................. 118 7.2.2 Unit of analysis .............................................................................................................. 121 7.2.3 Effects of the KAT on learning outcomes ..................................................................... 122 7.2.4 Effect of the KAT on Mutual Modeling Accuracy ........................................................ 122 7.2.5 Relation between the learning gain and Mutual Modeling Accuracy ......................... 122 7.2.6 Mediation effect of the Mutual Modeling Accuracy ..................................................... 123 7.3 KAT and the prior knowledge asymmetry...................................................................... 123 7.3.1 Interaction between KAT and pair level prior knowledge asymmetry ........................ 124 7.3.2 The differential effect of KAT on less and more knowledgeable peers ........................ 125 7.4 Discussions ........................................................................................................................... 126 7.5 Summary ............................................................................................................................. 128
CHAPTER 8
THE EFFECTS OF KAT ON VERBAL INTERACTIONS ................................ 131
8.1 Effect of KAT on the verbal interaction .......................................................................... 131 8.1.1 Domain knowledge ........................................................................................................ 132 8.1.2 Mutual modeling............................................................................................................ 136 8.1.3 Collaboration management........................................................................................... 136 8.2 Relation between outcomes and verbal interactions ...................................................... 137 8.3 Prior knowledge asymmetry and verbal interactions .................................................... 138 8.4 Hedges and uncertainty-markers ..................................................................................... 143 8.4.1 Hedges and epistemic markers ..................................................................................... 143 8.4.2 Uncertainty markers ...................................................................................................... 145 8.4.3 Method and coding process .......................................................................................... 145 8.4.4 Results ............................................................................................................................ 146 8.5 Case studies ......................................................................................................................... 148
12
8.5.1 Case study 1 ................................................................................................................... 148 8.5.2 Case study 2 ................................................................................................................... 150 8.5.3 Case study 3 ................................................................................................................... 152 8.5.4 Case study 4 ................................................................................................................... 153 8.6 Summary and discussions.................................................................................................. 156 8.6.1 Effects of KAT on verbal interaction ............................................................................ 156 8.6.2 Relation between outcomes and verbal interaction ..................................................... 157 8.6.3 Knowledgeability and verbal interactions .................................................................... 158 8.6.4 Hedges and uncertainty markers .................................................................................. 159
CHAPTER 9
ANALYSES OF CONCEPT-MAPPING ACTIONS ........................................... 161
9.1 Concept-map action types ................................................................................................. 161 9.1.1 Definition of a concept-map.......................................................................................... 161 9.1.2 Concept-map actions ..................................................................................................... 161 9.2 The effects of KAT on the production of actions ............................................................ 162 9.3 Relations between performance and production of actions .......................................... 163 9.4 Prior knowledge asymmetry and action production ...................................................... 163 9.5 The effects of KAT on action transactivity...................................................................... 164 9.6 Discussions ........................................................................................................................... 165
CHAPTER 10
ANALYSES OF THE DUAL EYE TRACKING DATA.................................... 167
10.1 Introduction ...................................................................................................................... 167 10.2 Did users actually look at the KAT? .............................................................................. 168 10.2.1 Data preparation ......................................................................................................... 168 10.2.2 The effective use of the KAT during collaboration..................................................... 169 10.2.3 Discussions .................................................................................................................. 173 10.3 Joint-attention in collaborative learning ....................................................................... 174 10.3.1 Gaze-coupling measure with cross-recurrence analysis ........................................... 174 10.3.2 The effect of the KAT on the gaze-coupling................................................................ 177 10.3.3 Gaze-coupling and verbal interaction ........................................................................ 177 10.3.4 Discussions .................................................................................................................. 178 10.4 Dual gaze data and socio-cognitive processes ............................................................... 179 10.4.1 Dual gaze features ....................................................................................................... 179 10.4.2 Dual gaze features and socio-cognitive processes ..................................................... 181 10.4.3 Discussions .................................................................................................................. 185 10.5 Summary ........................................................................................................................... 186
CHAPTER 11
REGARD: REMOTE GAZE-AWARE REFERENCE DETECTOR ............. 189
11.1 Eye movements and explicit references ......................................................................... 189 11.1.1 Method ......................................................................................................................... 190 11.1.2 Results .......................................................................................................................... 192 11.1.3 Discussion.................................................................................................................... 195
13
11.2 REGARD: a computational model of reference detection .......................................... 196 11.2.1 Introduction to the REGARD model ........................................................................... 196 11.2.2 Conceptualization and implementation of REGARD ................................................. 197 11.2.3 Testing dataset............................................................................................................. 203 11.2.4 Optimization and testing of the REGARD model ....................................................... 203 11.2.5 Performance of the REGARD model with optimized parameters .............................. 207 11.2.6 Discussion.................................................................................................................... 209 11.3 Summary ........................................................................................................................... 210
CHAPTER 12
GENERAL DISCUSSION.............................................................................. 213
12.1 Summary of the main results .......................................................................................... 213 12.1.1 Mutual modeling in computer supported collaborative learning .............................. 213 12.1.2 Dual gaze patterns and socio-cognitive processes..................................................... 215 12.1.3 REGARD: REmote Gaze-Aware Reference Detector ................................................ 215 12.2 Discussion of the main contributions ............................................................................. 216 12.2.1 Empirical contributions: peer knowledge modeling in CSCL ................................... 216 12.2.2 Methodological contribution: dual eye tracking ........................................................ 219 12.3 Design implications and further work ........................................................................... 221 12.3.1 Knowledge Awareness Tools to foster collaborative learning .................................. 221 12.3.2 Design and research implications of the DUET method ........................................... 222 12.3.3 Design implications of the REGARD model ............................................................... 223 12.4 Final words ........................................................................................................................ 225
REFERENCES ........................................................................................................................... 227 APPENDIX A ............................................................................................................................ 251 APPENDIX B............................................................................................................................. 255
