Interactivity in Graphical Representations: assessing

Interactivity in Graphical Representations: assessing its benefits for learning

Nuno Ricardo Palmeiro Otero

Thesis submitted for the degree of Doctor of Philosophy

October 2002

School of Cognitive and Computing Sciences University of Sussex Brighton

ii

I hereby declare that this thesis has not been submitted, either in the same or different form, to this or any other University for a degree.

Signed:

iii

Acknowledgements Firstly, I would like to express my sincere gratitude to Dr. Yvonne Rogers and Professor Benedict du Boulay for supervising my work. Since my arrival at Cogs, Dr. Yvonne Rogers was patient and supportive, enriching my thinking with judicious comments and guiding my investigation with careful considerations about which types of problems are solvable and which are not. Professor Benedict du Boulay was initially a member of my research committee and later become also supervisor. He always gave me sound support, thoughtful observations and, at several points, boosted my confidence. I will never forget both! I would like to express my thanks to Dr. Mike Scaife who was part of my research committee and always gave constructive comments on my work both in the research committee meetings and whenever I searched him for additional advice. I am truly sad I will not be able to enjoy is company anymore. I also thank the Human Centred Technology Group members for providing a friendly environment where one can discuss his research ideas. Dr. Rose Luckin, Dr. Richard Cox and Dr. Pablo Romero were kind to discuss with me specific topics of my investigation giving valuable feedback. I dedicate this thesis to my wife Monica. Her support, love and affection were unconditional. I do not think one can find more for a life-companion. To my little daughter Mariana I thank for re-teaching me that the world is really and always full of surprises and wonders. To my parents, sister and all family I thank having been able to get along with my temper... it is not easy and surely demonstrates their affection. I consider myself lucky to have been able to find such good friends. To name a few who were more directly involved in this time of my life: Ann, Claudia, Fabrice, Gabriella, Guillaume, Hilan, John, Lucas, Marco, Nuno, Pablo, Paul, Raquel, Silvia, Tania, Tiago, Youli. I would like to sincerely thank to Dr. Antonio Baptista for is medical care since it enable me to write the rest of my thesis with comfort. I also remember with gratitude

iv the dedication that the people of the Neurosurgery unit of Hospital dos Capuchos, Lisbon, show to their patients. I would like to thank the support given for the main experiment by the Departamento de Matemática, Faculdade de Ciências da Universidade de Lisboa and Escola Superior de Comunicação Social (Portugal) and Monica Dias, Maria Manuel Torres and Miguel Gaspar for their advice on building the ILEs. My research was supported by a grant from Fundação para a Ciência e a Tecnologia, Praxis XXI, BD/15717/98, which is gratefully acknowledged.

v University of Sussex Nuno Ricardo Palmeiro Otero Doctor of Philosophy (CSAI, COGS)

Interactivity in Graphical Representations: assessing its benefits for learning

Summary This thesis investigates the usefulness of interactive graphical representations for learning. Although interactive graphical representations have been used and promoted to facilitate learning, there are still open questions regarding what kinds of graphical representations and forms of interactivity are effective and under what circumstances. Particular questions addressed by this thesis include: how the use of distinct types of graphical representations' manipulation improve learning and how learners' previous knowledge and cognitive abilities influence their interaction and consequent performance with these type of representations. A study assessed the benefits of interactive diagrams to teach a geometry concept. Two variables were manipulated: (1) the possibility (or absence) to manipulate the whole diagrams (WD) and (2) the possibility (or absence) to manipulate elements of the diagrams (ED). From the combination of the two variables referred to, four interactive learning environments (ILEs) were built with different graphical representations and levels of interactivity: one with WD plus ED (WD+ED), another with WD (WD), a third with ED (ED) and a fourth without interactive graphical components in the diagrams (BASIC). Eighty first year undergraduate students participated in the study: 38 from a mathematics degree and 42 from a geology degree. Evaluation of the learning performance used a multiple-choice test (MCT) and a posttest (PT). The MCT analysis showed that background knowledge and spatial ability affected how much a learner benefited from each ILE. PT results suggested that learners with lower levels of spatial ability and geometry knowledge appeared to benefit more

vi from the ILEs with the ED interactivity property. However, the learners with higher levels of spatial ability and geometry knowledge seem to have their performance hindered by the WD interactivity property. The interaction of eighteen participants from a sub-sample of forty of the initial eighty participants was video-recorded. Video analysis of learners' interactions revealed that: the more participants explored the content the better the MCT scores; the interactivity provided was used but learners tended not to manipulate the diagrams' elements much; the WD+ED group seemed to have explored the textual representations less than the other groups; attentional transitions between representations occurring before starting to answer the MCT positively affected performance in the MCT. Based on these results, design issues and topics for further experiments are considered. A congruence hypothesis regarding an effect over the translation between representations knowledge/ability is proposed.

vii

Table of Contents Table of Contents ............................................................................................................vii List of Figures .................................................................................................................xii List of Tables .................................................................................................................xiv 1 Introduction ....................................................................................................................1 1.1 Claims about the benefits of virtual environments for learning..............................1 1.2 External representations as an alternative approach ...............................................3 1.2.1 Defining some terms ........................................................................................3 1.2.2 Initial considerations on the use of ERs frameworks to study learning...........5 1.3 Overview of the thesis.............................................................................................7 2 External representations ...............................................................................................10 2.1 Roles of external representations: reviewing some influential papers..................10 2.2 External representations and learning ...................................................................15 2.2.1 External representations and interactivity: what are the assumed benefits of interactivity?............................................................................................................17 2.2.2 Designing representations for learning: do novices need different types of representations and what is the importance of learners' differences? .....................18 2.2.3 Designing for an optimal level of computational offloading? .......................20 2.2.4 Externalisation and external representations: interacting factors and design implications .............................................................................................................21 2.3 Summary ...............................................................................................................23 3 Multiple external representations and Interactive Learning Environments .................25 3.1 How to characterise interactive learning environments that use multiple representations?...........................................................................................................26 3.2 Some key empirical findings concerning the use of multiple representations in interactive learning environments ...............................................................................30 3.3 Summary ...............................................................................................................31 4 A pilot study on design issues related to the integration of virtual environments in ILEs .................................................................................................................................33 4.1 Description of the Vari House system ..................................................................34 4.1.1 Description of the Vari House package .........................................................34 4.1.2 Brief description of CosmoPlayer 2.1............................................................36 4.2 The study...............................................................................................................38 4.3 The results and some design suggestions..............................................................39 4.4 Conclusions: design issues to inform the design of ILE and topics for further investigation ................................................................................................................46 4.4.1 Findings..........................................................................................................46 4.4.2 Lessons learned that informed the design of the main experiment's ILEs and topics of research ....................................................................................................47 5 The design of the interactive learning environments ...................................................51 5.1 A learners' activity model for understanding how learners integrate and use multiple representations ..............................................................................................51 5.1.1 Brief introduction to the domain ....................................................................51 5.1.2 The interactivity properties under investigation ............................................51 5.1.3 The theoretical framework .............................................................................53 5.2 General considerations about learning with interactive graphical representations .....................................................................................................................................58 5.3 Designing the interactive learning environments..................................................61

viii 5.3.1 Step one..........................................................................................................62 5.3.2 Step two..........................................................................................................65 5.3.3 Step three........................................................................................................67 5.3.4 Step four .........................................................................................................71 5.3.5 Step five .........................................................................................................74 5.3.6 Step six ...........................................................................................................76 5.4 Sketching some normative models of learner' use of the interactive learning environments ...............................................................................................................78 5.4.1 Before reaching the questions' part in the interactive learning environment .78 5.4.2 After reaching the questions' part in the interactive learning environment ...79 6 The hypotheses, the experimental design and the procedure.......................................82 6.1. The hypotheses.....................................................................................................82 6.2 The participants.....................................................................................................84 6.3 The apparatus ........................................................................................................86 6.4 The procedure........................................................................................................88 7 Background knowledge, spatial ability and interactive properties: a quantitative analysis............................................................................................................................91 7.1 The distribution of spatial ability and geometry knowledge.................................91 7.2 The impact of background knowledge, spatial ability and interactivity properties on immediate learning.................................................................................................94 7.2.1 Testing Hypothesis 1: The impact of the ILEs interactivity properties .........94 7.2.2 Testing Hypothesis 2: The impact of geometry knowledge and spatial ability ...............................................................................................................................102 7.2.3 The impact of the type of interactive learning environment on performance by type of questions ..............................................................................................106 7.2.4 The impact of spatial ability: separating text and diagrams' questions........108 7.2.5 The impact of geometry knowledge: separating text (type T) and diagram (type D) questions .................................................................................................112 7.3 The importance of background knowledge, spatial ability and interactive properties for performance in the post-test ...............................................................114 7.3.1 Testing Hypothesis 3: The impact of the ILEs interactivity properties on the PT ..........................................................................................................................115 7.3.2 Testing Hypothesis 4: The impact of geometry knowledge and spatial ability ...............................................................................................................................119 7.4 Concluding remarks regarding the quantitative analysis. ...................................121 8 The learners' interaction patterns with the ILEs.........................................................123 8.1 Framing the analysis: some questions.................................................................123 8.2 The methodology for the video analysis .............................................................124 8.2.1 Organising the data from the video recordings ............................................124 8.2.2 The coding categories and coding scheme...................................................125 8.2.3 Building a diagrammatic representation of learners' activity.......................130 8.2.4 Methodological considerations: How to write the learners' "stories"? ........131 8.2.5 The sample ...................................................................................................133 8.3 Assessing the learner's interaction patterns with the different interactive learning environments. ............................................................................................................134 8.3.1 Question 1: Can we identify differences in the number of actions and transitions between representations performed under the different interactive learning environments? .........................................................................................137 8.3.2 Question 2: What is the relationship between the number of actions and the scores in the performance tests? What is the relationship between number of

ix transitions between representations performed while interacting with the interactive learning environments and the scores in the performance tests? ..........................140 8.3.3 Question 3: How did learners distribute the overall time spent with the learning activity between time spent exploring the interactive learning environments explanation part and actually answering the questions? Can we identify a relationship with the scores on the performance tests?.........................143 8.4 Analysing the activity in each explanation step..................................................145 8.4.1 Some additional interaction patterns............................................................146 8.4.2 The questions ...............................................................................................146 8.4.3 General description of the six learners.........................................................147 8.4.4 Comparing the ED multiple-choice test high and low score learners..........148 8.4.4.1 Can the highest and lowest scorers be distinguished based on their transitions between representations?.................................................................149 8.4.4.2 How did the learners distributed their time interacting with the interactive learning environment?.....................................................................150 8.4.4.3 Do the dyads of interaction patterns (rushing/reflecting, switching/focusing and equally distributed/concentrated) distinguish the lowest and the highest scorers? ....................................................................................151 8.4.4.4 Which representations did the learners chose to support the answering? ...........................................................................................................................151 8.4.4.5 Additional comments ............................................................................152 8.4.5 Comparing the WD multiple-choice test high and low score learners.........154 8.4.5.1 Can we distinguish the highest and lowest scorers based on their transitions between representations?.................................................................155 8.4.5.2 How did the learners distributed their time interacting with the interactive learning environment?.....................................................................155 8.4.5.3 Do the dyads of interaction patterns (rushing/reflecting, switching/focusing and equally distributed/concentrated) distinguish the lowest and the highest scorers? ....................................................................................156 8.4.5.4 Which representations did the learners chose to support the answering? ...........................................................................................................................157 8.4.5.5 Additional comments ............................................................................157 8.4.6 Comparing the WD+ED multiple-choice test high and low score learners. 157 8.4.6.1 Can we distinguish the highest and lowest scorers based on their transitions between representations?.................................................................158 8.4.6.2 How did the learners distributed their time interacting with the interactive learning environment?.....................................................................159 8.4.6.3 Do the dyads of interaction patterns (rushing/reflecting, switching/focusing and equally distributed/concentrated) distinguish the lowest and the highest scorers? ....................................................................................160 8.4.6.4 Which representations did the learners chose to support the answering? ...........................................................................................................................160 8.4.6.5 Additional comments ............................................................................161 8.5 General overview of the qualitative analysis performed.....................................161 9 General discussion .....................................................................................................166 9.1 Revisiting the learning activity model ................................................................167 9.1.1 Understanding the diagrams and understanding the text .............................168 9.1.1.1 Individual differences: who benefits from which interactive learning environment?.....................................................................................................168

x 9.1.1.2 Did the different interactive learning environment s promote distinct comprehension for text and diagrams? .............................................................173 9.1.1.3 Can a pattern be identified relating the results obtained with the Paper Folding Test and geometry test and learners' levels of understanding of text and diagrams with the different interactive learning environments?.......................174 9.1.2 The integration of text and diagrams in understanding and learning...........175 9.1.3 Facilitating and inhibiting factors: what can be said about the learning assumptions ...........................................................................................................180 9.1.3.1 Did the BASIC ILE overloaded the learners?.......................................180 9.1.3.2 Did the BASIC ILE promoted a content exploration strategy that was beneficial? .........................................................................................................182 9.1.3.3 Did the more interactive interactive learning environments offload the need to understand the relationships between the diagrams' elements and the 3D nature of the diagrams? .....................................................................................183 9.1.3.4 Were the learners distracted by the interactive properties of the diagrams? ..........................................................................................................184 9.2 Future work .........................................................................................................185 9.3 Summary .............................................................................................................189 10 Conclusions ..............................................................................................................191 10 References ................................................................................................................194 11 Appendixes...............................................................................................................203 Appendix I.................................................................................................................204 Appendix I - A ......................................................................................................204 Appendix I - B.......................................................................................................205 Appendix II ...............................................................................................................207 Appendix II - A .....................................................................................................207 Appendix II - B .....................................................................................................208 Appendix II - C .....................................................................................................209 Appendix II - D .....................................................................................................210 Appendix II - E .....................................................................................................211 Appendix II - F......................................................................................................212 Appendix III (MCT)..................................................................................................213 Appendix IV (PT) .....................................................................................................221 Appendix V (GT) ......................................................................................................224 Appendix VI (PFT) ...................................................................................................225 Appendix VII ............................................................................................................226 Appendix VII - A ..................................................................................................226 Appendix VII - B ..................................................................................................229 Appendix VII - C ..................................................................................................231 Appendix VIII ...........................................................................................................233 Appendix VIII - A.................................................................................................245 Appendix VIII - B .................................................................................................247 Appendix VIII - C .................................................................................................248 Appendix VIII - D.................................................................................................249 Appendix VIII - E .................................................................................................251 The high scorer of the 2DI ILE. ........................................................................251 The low scorer of the 2DI ILE.. ........................................................................262 The high scorer of the 3D ILE. .........................................................................269 The low scorer of the 3D ILE. ..........................................................................275 The high scorer of the WD+ED ILE. ................................................................281

xi The low scorer of the WD+ED ILE. .................................................................288 Appendix VIII - F .................................................................................................293 Comparing the 2DI multiple-choice test high and low score learners. .............293 Comparing the 3D multiple-choice test high and low score learners. ..............297 Comparing the WD+ED multiple-choice test high and low score learners. .....300 Appendix VIII - G.................................................................................................305 Appendix IX..............................................................................................................306

xii

List of Figures Figure 1 - Snapshot of the Vari House system displaying the more frequent layout of three different frames ..............................................................................................35 Figure 2 - Figure displaying the common table with the hyperlinks to the different content units ............................................................................................................36 Figure 3 - Plot displaying the interaction pattern of participant 1 in relation to the time spent in each window and the sequence of transitions between the different windows ..................................................................................................................40 Figure 4 - Plot displaying the interaction pattern of participant 2 in relation to the time spent in each window and the sequence of transitions between the different windows ..................................................................................................................40 Figure 5 - Plot displaying the interaction pattern of participant 3 in relation to the time spent in each window and the sequence of transitions between the different windows ..................................................................................................................41 Figure 6 - The learning activity model............................................................................54 Figure 7 - Diagrams displaying the basic principle of the stereographic projection (similar to the ones present in textbooks) and explanation of the available interactive properties...............................................................................................60 Figure 8 - Snapshot of the WD+ED ILE, showing the different frames and its content 62 Figure 9 - Diagram of step one showing the basic principle of the stereographic projection ................................................................................................................63 Figure 10 - Screenshots of the step one diagram (see Figure 9) with the exemplification of the interactivity properties available for the WD+ED ILE.................................64 Figure 11 - Diagram showing the principles of the spherical projection........................66 Figure 12 - Stereographic projection of the hypothetical crystal shown in Figure 11....67 Figure 13 - Screenshots showing the interactivity property provided for the diagram displayed on Figure 11 for the WD and WD+ED ILEs ..........................................67 Figure 14 - Diagram displaying a great circle and a meridian........................................68 Figure 15 - Diagram showing the projection of a meridian............................................69 Figure 16 - Diagram presenting the projection of a great circle .....................................70 Figure 17 - Screenshots of diagram displayed in Figure 16 with the exemplification of the interactivity properties available for the WD+ED ILE .....................................70 Figure 18 - Diagram showing the projection of a great circle using points N and S as projection points......................................................................................................71 Figure 19 - Diagram displaying a small circle and a parallel .........................................72 Figure 20 - Diagram showing the projection of a parallel ..............................................73 Figure 21 - Diagram depicting the projection of a small circle ......................................73 Figure 22 - Screenshots of diagram displayed in Figure 21 with the exemplification of the interactivity properties available for the WD+ED ILE .....................................74 Figure 23 - Diagram showing the projection of points using two different points of projection and a representation of a stereographic net............................................75 Figure 24 - Screenshots of diagram displayed in Figure 23 with the exemplification of the interactivity properties available for the WD+ED ILE .....................................75 Figure 25 - The Wullf Net...............................................................................................76 Figure 26 - Diagram showing the projection of all points at a fixed angular distance from point A............................................................................................................77

xiii Figure 27 - Screenshots of diagram displayed in Figure 26 with the exemplification of the interactivity properties available for the WD and WD+ED ILEs .....................77 Figure 28 - The relationship between the PFT and the MCT scores for the geology students by type of ILE with fitted regression lines and corresponding R2 ............96 Figure 29 - The relationship between the PFT and the MCT scores for the maths students by type of ILE with fitted regression lines and corresponding R2 ............97 Figure 30 - Relationship between the PFT and the MCT for the geology students by type of ILE ............................................................................................................101 Figure 31 - Relationship between the PFT and the MCT for the maths students by type of ILE ....................................................................................................................101 Figure 32 - Means of the MCT for the Low-Low and High-High groups by ILE........104 Figure 33 - The learning activity model........................................................................167

xiv

List of Tables Table 1 - The different diagrams versions ......................................................................53 Table 2 - The questions of each explanatory step with a summary of the topic covered and the type (T, D or TD) of each part....................................................................81 Table 3 - Distribution of participants by type of ILE and degree ...................................85 Table 4 - Distribution of participants that completed the PT by type of ILE and degree. The missing cases are the ones who did the MCT but not the PT. .........................86 Table 5 - Descriptive statistics regarding the PFT and the GT .......................................92 Table 6 - Number of observations by group ...................................................................94 Table 7 - Descriptive statistics for the MCT...................................................................94 Table 8 - Tests of between subject effects for the ANOVA model. Dependent variable: MCT ........................................................................................................................95 Table 9 - Tests of between subject effects for the ANCOVA model with different linear relationships between the MCT and PFT by main factors. Dependent variable: MCT ........................................................................................................................97 Table 10 - Regression analysis testing the adequacy of the linear and quadratic models to express the relationship between PFT scores and MCT scores by groups .........98 Table 11 - Tests of between subject effects for the ANCOVA model with different relationships between the MCT and PFT by type of ILE and Degree. Dependent variable: MCT .......................................................................................................100 Table 12 - Parameter estimates for the ANCOVA model with different relationships between the MCT and PFT by type of ILE and Degree. Dependent variable: MCT ...............................................................................................................................100 Table 13 - Distribution of the participants by groups of scores in the PFT and GT by type of ILE ............................................................................................................102 Table 14 - Descriptive statistics for the MCT, regarding the High-High and Low-Low groups....................................................................................................................103 Table 15 - Composition of the different groups by degree ...........................................105 Table 16 - Most significant regressions by ILE and type of question for the geology students..................................................................................................................109 Table 17 - Most significant regressions by ILE and type of question for the maths students..................................................................................................................110 Table 18 - Distribution of participants who scored medium in the MCT by type of ILE and degree .............................................................................................................111 Table 19 - Correlations between the GT and type of questions, all participants ..........112 Table 20 - Best fit regression models by ILE and type of question for the geology students..................................................................................................................113 Table 21 - Best fit regression models by ILE and type of question for the maths students ...............................................................................................................................113 Table 22 - Distribution of participants in stage 3 by ILE and degree ...........................114 Table 23 - Distribution of the participants that completed the PT, by groups of scores in the PFT, GT, type of ILE and degree....................................................................115 Table 24 - Descriptive statistics regarding the PT (all participants together)...............115 Table 25 - Tests of between subject effects for the ANCOVA model by main factors ED, WD and degree. Dependent variable: PT ......................................................116 Table 26 - Tests of Between-Subjects Effects, Dependent Variable: PT; model including the distinct regression lines for each group regarding the PFT.............................117

xv Table 27 - Parameter estimates of the ANCOVA model; model including the distinct regression lines for each group regarding the PFT ...............................................118 Table 28 - Tests of Between-Subjects Effects, Dependent Variable: PT; model including the distinct regression lines for each groups by ED variable regarding the PFT..118 Table 29 - Parameter estimates of the ANCOVA model comparing the BASIC/WD and the ED/WD+ED ILEs. ..........................................................................................119 Table 30 - Descriptive statistics of the two models run concerning the two groups: High-High and Low-Low; Dependent variable: PT..............................................120 Table 31 - Tests of Between-Subjects Effects for the two groups: High-High and LowLow, Dependent Variable: PT ..............................................................................121 Table 32 - Possible transitions between the representations or relevant areas of the ILEs. ...............................................................................................................................128 Table 33 - Considered aggregations of the possible transitions between representations. ...............................................................................................................................128 Table 34 - Example of the description of the first three minutes of a learner interacting with the WD+ED ILE ...........................................................................................129 Table 35 - Scores in the MCT and the PT for the sample considered for the video analysis, as well as the same scores grouped by low, medium and high scores. ..134 Table 36 - Descriptive statitics by degree regarding the average of all the steps of the actions registered on the video analysisfor the BASIC (moving and rotating the diagrams and scrolling the text before and after reaching the questions) .............135 Table 37 - Descriptive statitics by degree regarding the average of all the steps of the actions registered on the video analysisfor the ED (moving and rotating the diagrams and scrolling the text before and after reaching the questions) .............136 Table 38 - Descriptive statitics by degree regarding the average of all the steps of the actions registered on the video analysisfor the WD (moving and rotating the diagrams and scrolling the text before and after reaching the questions) .............136 Table 39 - Descriptive statitics by degree regarding the average of all the steps of the actions registered on the video analysisfor the WD+ED (moving and rotating the diagrams and scrolling the text before and after reaching the questions) .............137 Table 40 - Descriptive statistics of the four variables corresponding to the "move" and "rotate" before and after reaching the questions. ..................................................138 Table 41 - Set of t tests (means) comparing the variables "move" and "rotate" before and after reaching the questions against a baseline of 0. .............................................138 Table 42 - Correlations (one tailed) between TIME BEFORE QUESTIONS and RELATIVE TIME BEFORE QUESTIONS with MCT and PT...........................143 Table 43 - General description of the participants of the sub-sample chosen for the stepby-step analysis .....................................................................................................147 Table 44 - Values of the two learners in relation to the percentages of points obtained and number of correct answers in each step..........................................................148 Table 45 - Percentage of correct answers for each type of question on each explanatory step ........................................................................................................................148 Table 46 - Learners' classification in the transitions related dimensions and corresponding absolute values for the variables involved. ...................................149 Table 47 - Learners' classification in the time related dimensions and corresponding absolute values for the variables involved. ...........................................................150 Table 48 - Percentage of points obtained and number of correct answers in each step for the two learners. ....................................................................................................154 Table 49 - Percentage of correct answers for each type of question on each explanatory step for the two learners. .......................................................................................154

xvi Table 50 - Learners' classification in the transitions related dimensions and corresponding absolute values for the variables involved. ...................................155 Table 51 - Learners' classification in the time related dimensions and corresponding absolute values for the variables involved. ...........................................................156 Table 52 - Percentages of points obtained and number of correct answers in each step for the two learners................................................................................................158 Table 53 - Percentage of correct answers for each type of question on each explanatory step for the two learners. .......................................................................................158 Table 54 - Learners' classification in the transitions related dimensions and corresponding absolute values for the variables involved. ...................................159 Table 55 - Learners' classification in the time related dimensions and corresponding absolute values for the variables involved. ...........................................................159 Table 56 - Normality for the PFT and GT scores .........................................................226 Table 57 - Summary of statistics for the Kruskal Wallis test concerning the GT and PFT scores.....................................................................................................................227 Table 58 - Kruskal Wallis test for the differences between the distinct groups concerning the GT and PFT scores .......................................................................227 Table 59 - Summary of statistics concerning the GT comparing the geology and maths students..................................................................................................................227 Table 60 - Tests comparing the geology and maths students in relation to the scores on the GT ...................................................................................................................227 Table 61 - Test of Normality for the MCT. ..................................................................229 Table 62 - Descriptive statistics concerning the 2X2X2 factorial ANOVA to test hypothesis 1. Dependent variable: MCT...............................................................229 Table 63 - Tests the null hypothesis that the error variance of the dependent variable is equal across groups. Design: Intercept+ED+WD+Degree+ED*WD+ED*Degree+ WD * Degree+ED * WD * Degree.......................................................................230 Table 64 - Distribution of participants by type of ILE and degree ...............................230 Table 65 - Levene's Test of Equality of Error Variances regarding the MCT..............230 Table 66 - Tests of normality for the different types of questions................................230 Table 67 - Test of normality concerning the PT ...........................................................231 Table 68 - Descriptive statistics regarding the ANCOVA model to test hypothesis 3. Dependent variable: PT.........................................................................................231 Table 69 - Tests the null hypothesis that the error variance of the dependent variable is equal across groups. Design: Intercept+Delay+ED+WD+Degree+ED* WD+ ED * Degree+ WD * Degree+ED * WD * Degree........................................................232 Table 70 - Descriptive statistics regarding the ANCOVA model that includes the PFT as co-variate. Dependent variable: PT.......................................................................232 Table 71 - Levene test: Tests the null hypothesis that the error variance of the dependent variable is equal across groups. Design: Intercept+ED * WD * Degree+ED * WD * Degree * PFT+Delay .........................................................................................232 Table 72 - Descriptive statistics concerning variables registered for the video analysis by degree regarding step 1 and BASIC ILE..........................................................233 Table 73 - Descriptive statistics concerning variables registered for the video analysis by degree regarding step 1 and ED ILE ................................................................233 Table 74 - Descriptive statistics concerning variables registered for the video analysis by degree regarding step 1 and WD ILE...............................................................234 Table 75 - Descriptive statistics concerning variables registered for the video analysis by degree regarding step 1 and WD+ED ILE .......................................................234

xvii Table 76 - Descriptive statistics concerning variables registered for the video analysis by degree regarding step 2 and BASIC ILE..........................................................235 Table 77 - Descriptive statistics concerning variables registered for the video analysis by degree regarding step 2 and ED ILE ................................................................235 Table 78 - Descriptive statistics concerning variables registered for the video analysis by degree regarding step 2 and WD ILE...............................................................236 Table 79 - Descriptive statistics concerning variables registered for the video analysis by of degree regarding step 2 and WD+ED ILE...................................................236 Table 80 - Descriptive statistics concerning variables registered for the video analysis by degree regarding step 3 and BASIC ILE..........................................................237 Table 81 - Descriptive statistics concerning variables registered for the video analysis by degree regarding step 3 and ED ILE ................................................................237 Table 82 - Descriptive statistics concerning variables registered for the video analysis by degree regarding step 3 and WD ILE...............................................................238 Table 83 - Descriptive statistics concerning variables registered for the video analysis by type of degree regarding step 3 and WD+ED ILE...........................................238 Table 84 - Descriptive statistics concerning variables registered for the video analysis by degree regarding step 4 and BASIC ILE..........................................................239 Table 85 - Descriptive statistics concerning variables registered for the video analysis by degree regarding step 4 and ED ILE ................................................................239 Table 86 - Descriptive statistics concerning variables registered for the video analysis by degree regarding step 4 and WD ILE...............................................................240 Table 87 - Descriptive statistics concerning variables registered for the video analysis by type of degree regarding step 4 and WD+ED ILE...........................................240 Table 88 - Descriptive statistics concerning variables registered for the video analysis by degree regarding step 5 and BASIC ILE..........................................................241 Table 89 - Descriptive statistics concerning variables registered for the video analysis by degree regarding step 5 and ED ILE ................................................................241 Table 90 - Descriptive statistics concerning variables registered for the video analysis by degree regarding step 5 and WD ILE...............................................................242 Table 91 - Descriptive statistics concerning variables registered for the video analysis by type of degree regarding step 5 and WD+ED ILE...........................................242 Table 92 - Descriptive statistics concerning variables registered for the video analysis by degree regarding step 6 and BASIC ILE..........................................................243 Table 93 - Descriptive statistics concerning variables registered for the video analysis by degree regarding step 6 and ED ILE ................................................................243 Table 94 - Descriptive statistics concerning variables registered for the video analysis by degree regarding step 6 and WD ILE...............................................................243 Table 95 - Descriptive statistics concerning variables registered for the video analysis by degree regarding step 6 and the WD+ED ........................................................244 Table 96 - Frequency of observations by group............................................................247 Table 97 - Learner's classification in the dimensions related to time measures. ..........251 Table 98 - Learner's classification in the dimensions related to frequency of transitions. ...............................................................................................................................252 Table 99 - Learner's scores in the different types of measures. ....................................253 Table 100 - Learner's classification in the dimensions related to time measures. ........262 Table 101 - Learner's classification in the dimensions related to frequency of transitions. ...............................................................................................................................263 Table 102 - Learner's scores in the different types of measures. ..................................263 Table 103 - Learner's classification in the dimensions related to time measures. ........269

xviii Table 104 - Learner's classification in the dimensions related to frequency of transitions. ...............................................................................................................................270 Table 105 - Learner's scores in the different types of measures. ..................................271 Table 106 - Learner's classification in the dimensions related to time measures. ........275 Table 107 - Learner's classification in the dimensions related to frequency of transitions. ...............................................................................................................................275 Table 108 - Learner's scores in the different types of measures. ..................................276 Table 109 - Learner's classification in the dimensions related to time measures. ........281 Table 110 - Learner's classification in the dimensions related to frequency of transitions. ...............................................................................................................................282 Table 111 - Learner's scores in the different types of measures. ..................................283 Table 112 - Learner's classification in the dimensions related to time measures. ........288 Table 113 - Learner's classification in the dimensions related to frequency of transitions. ...............................................................................................................................288 Table 114 - Learner's scores in the different types of measures. ..................................289

1

1 Introduction This thesis investigates how interactivity in graphical representations aids learning. However, the initial starting point of the research was broader; the intention was to investigate the learning benefits of 3D virtual environments (VEs). The reason being that many claims have been made about the benefits of using VEs to support learning. Before continuing, two terms need to be defined: virtual environment (VE) and interactive learning environment (ILE). Slater and Usoh (1993) consider that a VE is an environment produced by the dynamic interaction of a human user with a world displayed by the computer. Zeltzer (1992) identifies three key components of a virtual environment: "(1) a set of computational models of objects and processes to be simulated, (2) some means of modifying the states of these models over time course of the simulation, and, finally, (3) communication channels that allow the participant to experience the simulated events and processes through one or more sensory modalities. " (pp. 127). A VE in which the computational models of objects and processes simulated as well as the "view" presented to the user follow a three dimensional geometry can be considered a 3D VE. An ILE is considered to be any system composed of one or more artefacts (computer based or not) built to aid learning that displays a certain level of interactivity. Interactivity is defined as the degree to which a certain system responds to user initiated activities. Considering this broad definition of ILE a 3D VE built for learning is a special type of ILE. The rest of the chapter goes as follows. Section 1.1 presents a brief overview of some of the claims made about the learning benefits of VEs and empirical studies. Section 1.2 provides a critique of the general approach taken by the reviewed studies. It also defines the author's position regarding the investigation of this topic. Finally, section 1.3 presents an overview of the approach taken in the thesis.

1.1 Claims about the benefits of virtual environments for learning The definitions of learning are numerous and concurrent theories proliferate. Wild and Quinn (1998) distinguish between academic type of learning or, in another

2 way, the learning of others' descriptions of the world, and experiential learning, that involves one's very own learning events. One of the main claims about of VEs and learning is that they promote experiential learning (Osberg et al., 1997; Winn, 1993, 1997; Winn et al., 1999). Another claim is that VEs allow the integration of different types representations and ways of interacting with information (Dede et al., 1994; Dede, Salzman, & Loftin, 1996b; Salzman, Dede, Loftin, & Chen, 1999). Particularly, it has been argued that such a strategy will enable the learner to build bridges between the academic and experiential types of knowledge. Another benefit is that VEs can promote learning and retention by encouraging learners to be active explorers of the information, navigating through the VE and making decisions about positioning and alternative perspectives (Wickens, 1992). There are several examples of VE systems built for conceptual learning, for example: biology teaching (Mikropoulos, Chalkidis, & Kossivaki, 1997), physics (Brelsford, 1993; Dede et al., 1994; Dede, Salzman, & Loftin, 1996a; Dede et al., 1996b), science in general (Osberg et al., 1997; Winn et al., 1999) (for an extensive review of research projects concerning virtual environment systems applied to education see Youngblut, 1998). However, the main findings have not really supported the claims. For example, Osberg et al. (1997) conducted an evaluation program to understand the impact of their VE systems in comparison with other presentation modes. Assuming a constructivist approach, the authors proposed to test the following hypothesis: that learning about a wetland cycle using constructivist principles (student-directed information retrieval and compilation, collaborative VE design and construction) promote better learning of the domain than learning about wetlands cycle through traditional means (teacher-directed classroom lecture, single textbook readings, worksheet completion) (Osberg et al., 1997). The results show that the constructivist approach did not produced significant improved results compared with the traditional approach. Furthermore, the authors consider that what was more important was being able to build the VE rather than experiencing it. This suggests that immersive VE alone does not produce positive results. Another study examined how VE immersive technology could be used as a component of a pedagogical strategy to teach children that the Earth is spherical (Moher, Johnson, Ohlsson, & Gillingham, 1999). The authors propose that research

3 regarding the use of VEs to support learning should be focused in problems that are: important (represented in recognised curricula standards), difficult by themselves or resistant to traditional teaching methods, questionably enhanced by VE technologies, and theoretically informed (educational, psychology, and cognitive sciences) (Moher et al., 1999). Teaching children that the earth is spherical implicates a fundamental conceptual change (Moher et al., 1999). The learning task with the VE involved collaboration between two children. The findings were disappointing insofar as the VE did not support the learning as expected. The children were too engaged on the task and did not reflect on the conceptual issues and might have had difficulties co-ordinating the several external representations (ERs) utilised (Moher et al., 1999). Why have many studies not found that learning is facilitated by VEs? What design issues need to be improved? From the two studies clear answers as to what are the factors at play are not easy to find. For example, from the first study one does not know where the problems came from: the usability issues, the types of representations used, or if there is too much focus on one of the tasks that composed the VE experience (planning, building programming and experiencing the VE). The second study acknowledges the need to study further the types of representations used and how they can be combined, something that was missing from their initial conceptual framework. Furthermore, some kind of interaction between the collaborative nature of the task and the type of representations used can be expected, and this topic was not mentioned. This indicates that using VEs to support learning is not easy and suggests to us the need to study particular variables. One of those is the concept of ERs and their specific role in supporting learning.

1.2 External representations as an alternative approach 1.2.1 Defining some terms In the previous section the expression external representation (ER) was introduced. It seems necessary to make explicit in this thesis what are considered to be the meanings of the following terms: representation, external representation and graphical representation.

4 According to Larkin and Simon (1987) "A representation consists of both data structures and programs operating on them to make new inferences" (pp. 67). Palmer (1977) considers that any definition of the concept of representation needs to take into account two distinct entities and their correspondence/relationship: the representing world and the represented world. In fact, the author identifies five entities that are involved in the specification of the concept: (1) the represented world, (2) the representing world, (3) what aspects of the represented world are being represented, (4) what aspects of the representing world are doing the representing and (5) the correspondence between the two worlds. However, Mason (1987) assuming a constructivist stance, refers that the term representation might not be the correct one to be used when describing what goes on inside a person. A person's inner experiences are his/hers subjective world and not a re-presentation of the external world. What a person communicates to the external world are presentations of his/her inner world (Mason, 1987). Scaife and Rogers (1996) consider that the term representation has different meanings depending on the contextual use. The authors point out that a frequent distinction is between representation as process and representation as a product. The former relates to the transformations and preservations that happen when obtaining the actual representation from the world/objected to be represented, which seems to approach the concerns expressed by Palmer (1977). The latter is concerned with the structural characteristics of the representation and this sense resembles the position taken by Larkin and Simon (1987). Scaife and Rogers (1996) assume in their discussion the use of the two meanings. This thesis follows the same approach. In fact, the term representation and its precise definition and importance to the conceptualisation of cognition are themes of on-going debate which goes beyond the scope of this thesis. The second term to be defined in this section is of external representation (ER). According to Zhang (2000) external representations are the structures and knowledge in the environment, not only as physical symbols but also as external rules, constraints or relations implanted in material configurations. Cox (1996) states that: …an adequate definition of the term 'external representation' must distinguish between modalities (graphical versus linguistic) and must acknowledge

the

interaction

between

internal

and

external

representations and the role of ERs in helping to disambiguate internal

5 representations. It must acknowledge a distinction between formal and 'everyday' external representational activity. (pp. 7). Finally, the attention is drawn to the definition of graphical representations. The following representations are examples of the graphical representations set: graphs, diagrams, maps, plans, animations and virtual environments (Scaife & Rogers, 1996). Several authors would agree that graphical representations are distinct from sentential ones (for example, Cox, 1996; Larkin & Simon, 1987; Scaife & Rogers, 1996). Larkin and Simon (1987) argue that the principal distinction between diagrammatic (an example of a graphical representation) and sentential representations is that the former maintains unequivocally the information concerning the topological and geometric relations of the components of a certain problem. Cheng, Lowe and Scaife (2001) identically state that diagrammatic representations make use of space and spatial properties like topology, location, geometry, etc. However, Cheng, Lowe and Scaife (2001) propose that diagrams are not that different from other types of representations. For example, propositions also use, in a distinct degree, space and spatial properties to convey information. Conversely, diagrams usually incorporate propositions. Hence, "strong claims about the difference between diagrams and other representations should be treated with caution." (Cheng et al., 2001, pp. 84). Cox (1996) similarly considers that pure forms of graphical versus linguistic representations are difficult to find.

1.2.2 Initial considerations on the use of ERs frameworks to study learning Several authors when reviewing conceptual and methodological problems that affect the design of computer systems for learning, consider the importance of the ERs framework (Cheng, 1999b; Scaife & Rogers, 1996; Tergan, 1997). Tergan (1997) stresses the need for a comprehensive theoretical framework that takes into account the interaction of codes, modes, and processing capabilities of ERs, content aspects, instructional methods, learner variables, and contextual conditions. Nevertheless, many of the studies concerning VEs and learning have not adopted a framework that clearly acknowledges the possible effects that distinct ERs might have on learning. Instead, a number of assumptions driving the research are observed (Rogers & Scaife, 1998; Scaife & Rogers, 1996, 2001). Tergan (1997) also indicates the necessity to think about a fine grained task analysis of the activities that a learner has to perform when

6 interacting with an ER and, at the same time, consider the assembling of the several different elements/representations, according to a rationale that assures the proper matching of the task, the cognitive processing and learning. Rogers and Scaife (1998) refer to the need for a different approach that would allow distinguishing the different effects at play when designing and analysing a external graphical representation for learning. It is argued that system design and theoretically driven research can and should be mixed (Scaife & Rogers, 2001). Scaife and Rogers (2001) present "...a user-centred design approach for developing interactive software environments that combines general methods with specific concerns, aimed at supporting users in their understanding and learning of a given domain" (Scaife & Rogers, 2001, pp.116). The design methodology they propose involves five stages: •

Stage one - operationalising high-level requirements - this is considered to be a preliminary stage where the high level requirements are defined in terms of "what" and "why" a particular system is to be built. According to the authors the outcome of this stage is an initial identification of the range of the problem space and a collection of ideas about which problems are to be addressed and how.

•

Stage two - exploratory studies and informant design - in this stage the main goal is to investigate about practices in the domain space identified in the previous stage. Two components are considered: informant design (see Scaife & Rogers, 1999; see Scaife, Rogers, Aldrich, & Davies, 1997, for a description of the methodology) and exploratory studies. The result is the identification of constraints concerning the set of possible design ideas through the detection of needs, preferences and existing practices.

•

Stage three - prototyping and user testing - this third stage sees the beginning of the actual design through the building of low and mid tech prototypes, as well as scenarios, storyboards and scripts of interface components and interaction styles. One of the goals of this stage is to test and validate assumptions or solve specific problems. "The outcome from this phase are a set of specific design implications, which are intended to inform directly the conceptual model for the application." (Scaife and Rogers, 2001, pp. 120).

7 •

Stage four - specifying the conceptual model - from the information gathered in the previous stages, it is now possible to be clear about a set of design specifications. These design specifications frame the functionality to be provided as well as the connections with the range of activities to be supported. The level of detail and clarification of goals are important since these are the basis for further discussion of the design problems to be solved with programmers.

•

Stage five - implementation and evaluation - in this stage the process of implementation starts and the evaluations of the different iterations of the system are also done. The evaluation is processed against a set of goals established for the system.

The authors describe a case study related to their experience developing a 3D virtual theatre for children, to let them act out, script, edit and direct their own plays. The design approach just referred to above blends practical system's development issues with theory testing. Summarising, many studies that assess the learning benefits of VEs do not seem to be detailed enough to give clear design guidance. Part of the problem comes from a lack of interest on the study of the cognitive properties of graphical representations, and their effects on learning processes. The approach taken in this thesis acknowledges the problem referred to and puts a particular emphasis on the study of ERs and levels of interactivity, and their effects on the learning.

1.3 Overview of the thesis The previous sections gave a general overview of relevant research regarding the learning gains of VEs. It was noted how much of the research has not been theoretically driven. In contrast, this research is concerned with how cognitive theory can be used to inform the design of VEs to support learning. Specifically, this investigation shows a strong interest in how ERs can be designed as interactive learning environments (ILEs). To this end, a review of some influential papers concerning the study of ERs and their roles in different tasks, from problem solving to learning is provided in chapter 2. This chapter then progresses with a special focus on learning tasks highlighting what are considered to be key aspects of the domain. Chapter 3 extends the theoretical basis

8 covered initially and is dedicated to issues regarding the use of multiple-representations in learning systems. A way to characterise multi-representational ILEs is discussed as well as some especially important empirical findings concerning these types of learning systems. Two empirical studies were carried out. First, an initial pilot study explored a specific type of VEs technology, VRML1, and related design issues concerning ways to support learning (chapter 4). The pilot study not only helped focusing the research scope of the main experiment but also shaped the ideas regarding general design issues of VRML technology use for learning. In fact, these design related findings were useful on the development of the ILEs used in the main experiment. Considering the initial goal of investigating the learning benefits of VEs, the main experiment was designed to enquire whether providing interactive graphical representations with distinct levels of interactivity was beneficial to teach a certain geometry concept. The main dependent variables were: (a) the possibility (or not) to rotate the whole diagram through direct manipulation and (b) the possibility (or not) to manipulate certain elements of the diagram, in order to simulate the relationship between the elements. Chapter five introduces the design and development of the four ILEs used for the main experiment, covering not only interface issues but also the explanation of the domain and the distinct representations used. Chapter six presents the general hypotheses drawn, the sample used, the tests utilised and the procedure followed in the main experiment. Chapter seven concerns the quantitative analyses of the data collected and testing of the hypotheses. The findings suggest a complex interaction between learners' background knowledge, learners' levels of spatial ability, type of ILE, and type of learning performance task. Nevertheless, the quantitative analysis did not give an insight on the actual use of the distinct ILEs. In order to investigate the issue a qualitative analysis was done regarding video recording data of learners' interactions with the ILEs. Chapter 8 presents and discusses in detail the methodology followed and the results obtained. The analysis enabled the identification of different patterns of interaction.

1

VRML technology is commonly used to implement web-based desktop VEs.

9 Chapter nine presents the general discussion of the results obtained while chapter ten summarises the main issues found.

10

2 External representations The previous chapter reviewed some of the research on the use of virtual environments (VEs) for learning (a type of interactive learning environment - ILE) and argued about the need to investigate in detail the interactivity properties of the graphical representations used. Furthermore, the chapter stressed the importance of studying this problem using a conceptual framework and findings from research on external representations (ERs). This chapter will discuss the major studies done in the ERs area and introduce the external cognition framework.

2.1 Roles of external representations: reviewing some influential papers From the research in this area two issues are made salient: •

A shift from a focus on the internal mechanisms that act on the ERs to the recognition of the importance of the ERs characteristics and the interplay between the internal and the external (Scaife & Rogers, 1996).

•

The revision goes from frameworks more suited to well-defined problem spaces to the external cognition framework that explicitly acknowledges the need to extend the analysis to more ill structured domains (Rogers, 1999; Rogers & Scaife, 1998; Scaife & Rogers, 1996). Moreover, the framework provides concepts regarding the roles of ERs and their relationship with internal ones (the cognitive properties of ERs) and also design principles that can guide and frame empirical research.

A seminal work is Larkin and Simon's (1987) study which investigated why, sometimes, it is advantageous to use diagrams to solve a problem. The authors consider that the main advantage of diagrammatic representations over sentential ones is the ability of the former to explicitly present the information about the topological and geometric relations among the components of the problem. Larkin and Simon (1987) apply the distinction between informational and computational equivalences in order to compare the effectiveness of sentential and diagrammatic representations. Informational equivalence is defined by the exact informational correspondence between two given

11 representations, meaning that all the information in one representation can be inferred from the other. On the other hand, in order for two representations to be computationally equivalent they need to be informationally equivalent but also the operators required for each must be the same and work with similar efficiency. The authors are able to point out that on the two examples used, a pulley system and geometry problems, the diagrammatic representation demands less computational effort by easing search and recognition. Furthermore, the diagrams were also able to offload memory due to the particular characteristics of its labelling system. Larkin and Simon (1987) conclude that: (a) diagrams ease search by grouping relevant information, (b) the need to match symbolic labels is reduced in diagrams which decreases computational effort, and (c) diagrams support perceptual inferences. Larkin (1989) extends the analysis of the previous paper by focusing on the role of some external displays in some specific tasks. More specifically, she considered a class of very easy problems that people solve routinely to the point of not thinking about them as problems. The extension referred to makes the importance of the external more salient by showing how physical constraints of the external display are perceived and how that affects task performance. Koedinger and Anderson (1990) starting point is the study and modelling of how experts solve some geometry problems. The observation of experts lead them to indicate that: •

Experts seem to initially plan the proof for the problem in a abstract way (the abstract problem space). The planning involves making key inferences but skipping minor ones, especially algebraic.

•

Experts use macro-operators which means chunking problem solving steps. Furthermore, these macro-operators show a certain degree of regularity among the subjects observed.

•

The inferences were coupled to the diagrams as the regularity in experts' step skipping was explained by knowledge structures cued by images in the problem diagram. The diagram not only serves to facilitate the search for applicable knowledge but also to guide and cue abstract planning operators.

12 Based on their findings, the authors designed a system (the DC model) that could account for the experts' strategies by organising their knowledge in diagrammatic schemas, the so called diagram configuration schemas. These are understood as sets of geometry information associated with a single geometric image. Furthermore, experts tend to do forward search, which means solving a problem sometimes without using a goal statement. This forward inferencing is closely tied to the use of diagram configuration schemas, since without the latter only very simple geometry problems could be solved with an inference strategy. Moreover, the authors refer that the macrooperators are not derived from "simple" execution operators in a traditional sense: "Rather, they derive from perceptual chunking of domain objects and they merely bear a macro-operator relation with execution space operators." (Koedinger and Anderson, 1990, p. 545). The model built is clearly influenced by Larkin (1989). In terms of the application of the findings to geometry instruction, they suggest that there is space for introducing the teaching of diagram configuration schemas, capitalising on the benefits of using diagrammatic representations. Koedinger and Anderson (1993) apply and extend the findings of their previous study to the modelling of an intelligent tutoring system. Interestingly, Laborde (1996) distinguishes two types of invariants needed to learn geometry: (a) spatial invariants that involve perceptual processes and permit the recognition of diagrammatic features and (b) geometrical invariants that foster reasoning about theoretical objects of geometry. The author refers that learning geometry implies the building of links between these two kinds of invariants, resembling the framing described by Koedinger and Anderson (1990, 1993). Another early seminal work was by Zhang and Norman (1994), who propose a theoretical framework and methodology to analyse tasks that use resources in the internal mind and resources in the external environment. The authors consider the possibility of characterising a task at an abstract level, the abstract structure of the task, composed of external and internal representations. The representational analysis and the methodology put forward assume that the tasks under study can be hierarchically decomposed into different levels and that at each level different isomorphic representations can be employed. This decomposition strategy allows for a better identification of representational properties and their effects. In their study, the problem under investigation is the Tower of Hanoi. The authors apply the same conceptual framework to the analysis of numeration systems (Zhang & Norman, 1995). Zhang

13 (1997) using the Tic Tac Toe game explores the effect of congruence between perceptual biases and different characteristics of the ERs used in this study. Zhang's research is important because it starts unveiling the relationship between perceptual inferences and working memory in problem solving. In fact, though previous studies like Larkin and Simon (1987) and Larkin (1989) talk about the existence of perceptual inferences and the relative ease that humans sometimes have with this class of inferences, they do not specify the mechanisms under which the phenomena occur. More recently, Stenning and Oberlander (1995) propose a theory that intends to explain why graphical representations can aid inferencing: the specificity of graphical representations. The central idea is that the way a graphical representation is able to represent a certain problem space limits abstraction, or in another way, restricts the number of alternative interpretations, and this aids cognitive processing. Non-graphical representations, in their turn, allow the expression of abstraction and indeterminacy. Cox (1996) considers the best representation to be one with just the required level of abstraction-expressing power for the specific task being solved, not more. Many studies have tested the Stenning's et al. theory and found support for its postulates (Cox, Stenning, & Oberlander, 1995; Dobson, 1999; Oberlander, Stenning, & Cox, 1999; Stenning, Cox, & Oberlander, 1995). However, Dobson (1999) casts doubt on the usefulness of the specificity principle stated by the theory when the differences between the representations regarding the variable are not that big. According to Scaife and Rogers (1996) there has been a focus on the internal mechanisms that act upon the ERs and not enough consideration has been paid to the dynamics between the external and the internal. Their external cognition framework seeks to fill this gap and focuses on the explanation of the dynamical relationship between internal and ERs. The paper addresses the question of determining which representations to use in learning environments. The authors' strategy to tackle the problem of conceptualising the relationship between the internal and the external involves the identification of the relevant cognitive properties of the ERs. They propose the following properties/notions: •

Computational offloading - relates to the phenomena of how informationally equivalent representations might require different degrees of cognitive effort.

14 •

Re-representation - refers to the fact that problem solving can be eased by different ERs that share the same abstract structure.

•

Graphical constraining - points out that some graphical components of a graphical representation are able to constrain the number and types of inferences made about the represented concept.

•

Temporal and spatial constraining - relates to the possibility of making salient certain processes of a concept through the use of specific representational elements or strategies distributed over time and space.

Additionally, the authors propose the following design principles that can also frame specific research questions (Rogers & Scaife, 1998; Scaife & Rogers, 1996): •

Explicitness and visibility - different types of graphical ERs can make distinct aspects of the information encoded more relevant and/or are able to express more vividly underlying phenomena of complex processes.

•

Cognitive tracing and interactivity - the issue here is the possible benefit of a user/learner being able to leave visible traces on the representation. The authors consider that this can imply a trade off between more or less interactive representations. More interactive representations although, for example, allowing testing and feedback, might make tracing more difficult.

•

Ease of production - this design principle suggests the possible link between the ability to produce a certain type of representation and the ability to comprehend it more fully.

•

Combinability - puts the emphasis on the possibility of combining different types of ERs. In this case, the designer must know not only the strengths and weaknesses of the different types of representations taken separately, but also the advantages and disadvantages of the combination of representations. The overall effect might not result from a simple additive process.

15 •

Distributed graphical representations - in this case the issue lies on the possible differences that different contexts and social environments can impose, demand or facilitate on the use of ERs.

Cheng, Lowe and Scaife (2001) review the field of diagrams' use and notice that simple claims about the benefits of using diagrams are not generally easy to make due to the complexity of the phenomena. For example, the authors draw attention to the fact that propositions also employ properties that are commonly attributed to diagrams, namely the use of location, topology and geometry to encode information. Thus, it is probably best to consider a continuum regarding the utilisation of the properties referred to above, making it possible to position a particular representation in a specific place of the continuum. Furthermore, diagrams can come in a wide variety of types, making swift general claims about the benefits of their use difficult. The authors claim that the study of diagrams' use should take into consideration: (a) the importance of the background knowledge that the users bring to the interaction with the diagram, (b) since diagrams are not commonly employed in isolation, one has to acknowledge the issue of using multiple representations and how this affects the task and (c) the importance of being able to modify and construct ERs. Finally, according to Cheng, Lowe and Scaife (2001) there is a need for a more general conceptual/theoretical framework instead of a case-based approach, which they argue has been dominating the research. Such framework should be able to cover not only the range of topics that emerge when one studies diagrams' use but also the additional issues that come with the introduction of multimedia use.

2.2 External representations and learning This section covers studies that focus on the use of ERs in learning tasks. Considering that the adequacy of a representation is linked to the type of task and subjects’ differences (for example, levels of background knowledge and cognitive abilities), some issues that are specific to learning tasks can be found. The following come to mind:

16 •

The main research topic of this thesis is the investigation of the added value of making graphical representations interactive, when considering learning situations.

•

Prior knowledge clearly affects the ability to interact and take advantage of ERs (Cheng et al., 2001). The most efficient representation for an expert to solve a problem might not be equally as good for a novice who needs to learn a domain. An expert might be skilled using and reasoning with a certain type of representation to solve a problem, while a novice might need another type of representation to start acquiring the required background knowledge and operators in order to be able to utilise the most efficient one later on. Other factors like cognitive abilities might also influence the way a learner interacts and benefits from a certain representation. Furthermore, the ability to use a certain representational formalism might require the teaching of “intermediate” representations.

•

Another issue concerns the possibility that in order to achieve learning efficiency a representation might need to have a "balanced" cognitive load. On the one hand, a too easy representation, in terms of computational offloading (Brna, Cox, & Good, 2001; Kirschner, 2002; Pollock, Chandler, & Sweller, 2002; Rogers & Scaife, 1998), might not foster the necessary cognitive effort and compromise retention. On the other hand, a too difficult representation might exhaust the cognitive resources needed for learning.

•

Another point relates to the problem of the externalisation process and the difference between using pre-fabricated representations and constructing one's own (Cheng et al., 2001; Cox, 1997, 1999). The study of the needed functionalities, probably tailored to each type of representation, that take advantage of the externalisation process is clearly necessary. The design property cognitive tracing (Scaife and Rogers, 1996; and Rogers and Scaife, 1998) gives an account of this issue.

•

Finally, it is worth noting that the first point clearly introduces the issue of the use of multiple-representations and related problems in learning

17 environments, the topic of the last chapter of the literature review (chapter 3).

2.2.1 External representations and interactivity: what are the assumed benefits of interactivity? A main issue in the present research regards the potential benefits of interactive graphical representations for learning. Interactive graphical representations have been used and promoted to facilitate learning (Cheng, 1999a). However, not much is known about what kinds of graphical representations and what forms of interactivity are effective (Rogers, 1999). Kirsh (1997) discusses the term interactivity and its importance in learning environments. The author proposes an extension of Norman's action cycle model to include three classes of actions that give a finer account of a person's embodied activity. The classes are: preparation, maintenance and complementary actions. However, as the author also admits, the paper does not give clear guidance on how these actions affect the learning or how the actions can be supported in an ILE. Rogers and Scaife (1998) present a critique on the overly optimistic use of different media and modalities in learning environments. The design decisions are, most of the times, not guided by solid conceptual frameworks, basing the inclusion and mix of different types of representations in naive assumptions of their benefits. Rogers (1999) states the need for a better understanding of how can interactive graphical representations facilitate problem solving and, in doing so, what are the learning benefits of making the task easier. The main problem lies, therefore, in determining the optimal balance between how much of the solution is readily available from the representation and how much is left for the learner to do. The author discusses two general claims regarding the benefits of making graphical representations interactive: •

One claim states that adding interactivity can reduce the learner's effort when constructing and/or manipulating a graphical representation for problem solving. One way to achieve this effort reduction is by constraining the problem space, making use of perceptual tasks instead of

18 complex chains of deductions. However, the actual benefit for learning can be questioned since the learner might skip the conceptual understanding and focus on the perceptual task itself. •

The second claim is that interactive graphical representations can offload low-level cognitive activities allowing the learner to concentrate on highlevel ones. Nevertheless, some of the low level cognitive activities might perform an important role. These activities might promote learning by enforcing the learner to externalise his initial mental representation. The process of externalisation might, in turn, make the learner explicitly represent information and draw his/her attention to less understood issues of the topic to be learned improving his understanding.

According to Rogers (1999), the main benefit of interactive graphical representations is, through the use of the extra interactivity provided, to explicitly represent the connections between different representations and thus facilitate the integration of information.

2.2.2 Designing representations for learning: do novices need different types of representations and what is the importance of learners' differences? This thesis does not intend to make an extensive review concerning the differences between experts and novices uses of ERs or all cognitive abilities and their possible influences on graphical representations interaction. However, some examples are given to emphasise the point covered by the question that frames this section. Koedinger and Anderson (1990) study the way experts solve geometry problems and highlight the different problem solving strategies they possess in comparison with novices. Kalyuga, Chandler and Sweller (1998), applying the cognitive load theory (Kirschner, 2002; Pollock et al., 2002), found that depending on their levels of expertise subjects were showing better performances with different displays. However, the cognitive load theory departs somewhat from the general conceptual approach discussed so far. It focuses on the study of instructional design principles to guide, for example, the inclusion and general spatial arrangement of representations in relation to human processing capabilities and not the cognitive properties of the representations

19 themselves as did the previous studies considered. Nevertheless, from the study referred to, the relevance for the understanding of the influence of expertise in problem solving performance with ERs is clear. Lowe (1996) found that novices processing a static weather map showed a dominant perceptual effect while experts attended to domain relevant elements. Lowe (1999) extends the previous research referred to and investigates the possible benefits of using animations to help novices build better models of meteorological phenomena in relation to the reading of weather maps. The findings were not optimistic and the author considers that static diagrams might be (sometimes) preferable to animations. Furthermore, Wright, Milroy and Lickorish (1999) suggest that different cognitive skills are needed to integrate text with static or animated graphics. Good and Brna (1996), analysing a case study of teaching recursion to novice programmers, consider that representations for different purposes, i.e. for understanding versus for practical use, are not necessarily one and the same. Rogers and Scaife (1998) illustrated the need to build the understanding of the formalisms underlying food webs through the use of intermediate representations that reflect different levels of abstraction. One would expect that experts would be comfortable with more abstract representations while novices, in some domains, would profit by being introduced first to more concrete representations. The authors propose the notion of dynalinking, a design solution to help learners integrate different representations. Summarising, Cox (1999) suggests that three factors and their interaction explain the effectiveness of a particular ER in a particular context. The factors include the properties of the representations, the task's demands and other within-subjects factors such as prior knowledge and cognitive style. The relevance of this section's questions is better understood if the following is pointed out at this stage: a learning task is not the same as a work environment task; a novice's prior knowledge is not, by definition, the same as an expert's one.

20

2.2.3 Designing for an optimal level of computational offloading? Rogers and Scaife (1998) and Rogers (1999) stress the possible existence of an optimal level of computational offloading to promote efficient learning, meaning that the design of the system's representation should reflect this issue. Brna, Cox and Good (2001) also focus on this. The authors comment that "making things easier" and helping students to learn are not always compatible goals. The cognitive load theory (Kirschner, 2002; Pollock et al., 2002), as its name indicates, also addresses the same issue. However, as considered above, the focus is not on the cognitive properties of the representations themselves. The theory generally considers that the cognitive load associated with the apprehension of the instructional material should be reduced in order to let the resources be allocated to the acquisition of the needed schemas. A case of the design and use of diagrams to offload learning tasks and promote understanding of complex domains is Cheng's work on law encoding diagrams (LEDs) (Cheng, 1999a, 1999b, 2002). LEDs are a specific type of diagrammatic representation able to encode the relationships between the components of a scientific law through the use of geometric, topological or spatial constraints. The instantiation of a diagram is a case of the captured law or instance of the modelled phenomena. The use of LEDs have been proved beneficial for conceptual learning in several domains (Cheng, 1999b). An example of this approach is the utilisation of AVOW diagrams to teach electricity (Cheng, 2002). These diagrams not only encode important laws of rule the domain but are also able to provide different crucial aspects and perspectives of the domain making them an efficient representational system (for an in-depth description of AVOW diagrams and their representational power see Cheng, 2002). Empirical testing showed that the AVOW diagrams were superior in increasing understanding than conventional methods using algebra (Cheng, 1999b). Nevertheless, Cheng (1999a) discusses the addition of computer supported interactivity to enhance LEDs. The reason being that computer supported interactive instantiations of LEDs can overcome practical difficulties identified with the use of static versions. Cheng (1999a) refers: (a) in some cases reasoning with LEDs require considerable drawing and re-drawing of the representations, (b) some geometric and diagrammatic reasoning ability is needed to construct LEDs and this may impede less able learners, (c) LEDs may become awkward to manipulate when complex phenomena are involved, even more than the equivalent

21 algebraic expressions, and (d) designing effective LEDs for new domains is not easy. Instructors may find it simpler to generate appropriate graphs or mathematical models. Cheng (1999a) considers, though, that interactive LEDs can ease problems a), b) and c), since the computer can be designed to keep the constraints of LEDs and be called to do part of the drawing and re-redrawing. Furthermore, the systems can be designed to focus learners' attention on the important aspects of the domain. However, as already pointed out in sub-section 2.2.1 following Rogers (1999), it is still not clear how to balance the offloading between the computer and the learner. The concept of computational offloading is central to this thesis since the two variables manipulated in the main study are supposed to ease the cognitive load concerning the recognition of 3D elements and the need to mentally simulate the relationship between the diagrams' elements. However, the question is how beneficial is this particular case of cognitive offloading.

2.2.4 Externalisation and external representations: interacting factors and design implications Reisberg (1987) considers that the limits of internal representations are one of the principal reasons for the importance of external ones. The author focuses on the benefits of externalising one's thoughts and presents three ways of how cognitive performance can be augmented by externalisation: •

Perceptual knowledge - Externalising one's thoughts allows us to take advantage of perceptual knowledge. The author considers the separation between conscious and perceptual knowledge. Reisberg (1989) suggests that by externalising the internal representations we make them inspectable by perceptual skills and this might facilitate problem solving in some tasks.

•

The non-intentionality of ERs - External representations can be reinterpreted, and, according to Reisberg (1987), are non-intentional. The process of externalising one's ideas allows for a more detached analysis of the core issues of the idea itself, permitting the emergence of different perspectives. The confrontation of perspectives improves thought.

22 •

Discovering omissions - Internal representations seem to represent details unequally. Important elements might be skipped but the person is not aware of this. The externalisation enables the detection of existing gaps.

Cox (1997) found that not all diagrams' interpretation errors were associated with errors in the construction of representations. Cox (1999) discusses in detail the relations between self-constructed representations, externalisation and individual differences. The following points are especially important for the present section: •

Learners need to have knowledge about which type of representation to choose in order to solve a problem. This applies to self-constructed as well as pre-fabricated representations. In Cox's terms, choosing an ER modality is less important than picking up the one that fits the level of abstraction of the information in the problem.

•

Learners might show a preference for a certain modality. Many studies classify learners according to one dimension: visualisers and verbalisers. Visualisers tended to choose graphical representations while verbalisers choose sentential ones. However, Cox (1999) considers this distinction too simplistic. Instead another dimension might be the ability to translate between different modalities or to stick to just one.

•

Finally, Cox (1999) suggests the existence of between subjects' differences in the extent to which they externalise their reasoning and the way they do so.

The implications of the second issue are particularly relevant to ILEs that employ more than one type of representation and that rely on this characteristic to optimise learning. For example, ILEs that use different modalities of ERs as a complementary strategy could be making life more difficult to "low translators". Clearly, additional supporting strategies are needed to cater to these types of learners. The

distinction

between

pre-constructed

and

learners'

self-constructed

representation is important and the framing situates the present study: the experiment was designed to provide pre-fabricated ERs. Another aspect concerns a methodological issue: if externalisation is important what is its influence in distinct methodologies and levels of interactivity? One could argue that as long as the same method is used to

23 evaluate the different ILEs under study then no problem is present. However, it could be the case that, for example, the thinking aloud technique can promote a specific type of externalisation. In turn, that process might interfere differently with the learning process depending on distinct types of ILEs. If this is the case then surely there is a methodological problem that has not been given due importance in many studies.

2.3 Summary The literature clearly establishes that ERs are important in cognitive activities like problem solving and learning. Their distinct characteristics constrain how tasks are performed and how successful is the performance. However, the adequacy of an ER is also linked to subjects' individual differences, for example, background knowledge and cognitive abilities. A framework that acknowledges the interplay between internal and ERs need to take into account the complex interactions between the factors considered. Focusing on learning tasks, and considering that our goal of assessing the benefits of interactive graphical representations, the following research issues also emerge from the review: •

What is the value of adding interactivity to graphical representations? Is the aim offloading the learner? If yes, does this really promotes learning?

•

What is the effect of learners' background knowledge and cognitive abilities on learning tasks using distinct ERs?

•

How important is the externalisation process in pre-constructed ERs? What factors should one be aware of concerning the research methodologies used?

The two first research topics mentioned above directly informed this research. The following variables considered in the main experiment reflect such care: •

Learners' levels of background knowledge and cognitive abilities, judged to be important in the domain applied. The goal was to see how different levels of these two variables affected performance and learning with different ERs.

24 •

The different types and levels of interactivity. Included to assess the benefits of interactive graphical representation.

This thesis contribution to the field is the study of how the two previous points are related. Basically, the general questions are: Are interactive graphical representations able to support learning better? To what extend does learners' background knowledge and cognitive abilities affect the learning with ERs with distinct characteristics? Do the same benefits of using a certain ER extend to all types of learners? The third point regarding the importance of the externalisation process was not subject to experimental manipulation. However, the reflection about it affected the methodology used and is a topic of discussion in chapter 9.

25

3 Multiple external representations and Interactive Learning Environments The previous chapter discussed the use of different representations for supporting learning. Rogers (1999), analysing the benefits of interactive graphical representations for learning, suggests that combining different representations and dynamically linking them is a key issue. In this chapter the focus is on the way multiple external representations (MERs) can aid learning and their use in interactive learning environments (ILEs). Section 3.1 addresses the problem of how to characterise interactive learning environments that employ MERs. It starts by presenting de Jong et al.'s (1998) proposed dimensions to classify ERs. These dimensions reflect basic empirical research done investigating the properties and benefits of different types of ERs, some of which was referred to in the previous chapter. The section then follows with Ainsworth's (1999a; 1999b) taxonomy that aims to classify the functions of MERs. Ainsworth's work specifically deals with the relationships between the distinct ERs that compose an ILE. The importance of these two classificatory systems comes from the need to disambiguate the nature of the distinct ERs used in the ILEs and the relationships they establish between themselves, which is a crucial issue in order to be able to evaluate the actual cognitive and learning benefits of their employment. Section 3.2 presents what are considered key empirical findings in the domain. An initial comment regarding the distinction between virtual environments (VEs) and ILEs is needed. VEs for learning can be considered to be a specific type of ILEs (as already stated in chapter 1). In chapter 1 the generalist approaches concerning the assessment of VEs learning benefits was criticised. Interestingly, although the term ILE stands for a broader class of learning systems, the research using it has generally been more careful with the specification of the systems' characteristics and the evaluation of their learning benefits. In particular, such literature often uses external representations (ERs) frameworks to the evaluation of the ILEs.

26

3.1 How to characterise interactive learning environments that use multiple representations? Given the multitude of ILEs that employ distinct ERs, an initial issue is to consider how this type of systems can be characterised (de Jong et al., 1998). The authors propose a set of dimensions to describe ERs. The dimensions considered somehow reflect basic research on each particular topic and form an extensive set that is judged to be useful in disambiguating the nature of distinct ILEs that use MERs. Furthermore, the set also provides a framework that can guide the description and naming of variables to be manipulated in experiments that investigate the learning benefits of ERs in ILEs. The dimensions are: •

Perspective - many times, concepts, problems or, more generally, domains can be inspected from different perspectives. As a simple example, the authors distinguish between the functional and the topological perspectives to describe the parts of an engine, the former focusing on the function of each part and the latter on its location. The authors go on describing the perspectives of other systems covering different domains.

•

Precision - a representation might vary on the level of accuracy of the information it contains: from a very precise quantitative approach to a purely qualitative one. The authors consider that, in principle, qualitative and quantitative types of reasoning, supported by their respective representations with different levels of precision, are complementary. The former helps the student to understand the concept and to select the appropriate quantitative tools of analysis. The latter helps to clarify, verify and extend the outcomes of the qualitative reasoning.

•

Modality - a dimension describing the particular form of expression used to display information. It seems that the term here has a different meaning from the one given by Stenning and Oberlander (1995). Stenning and Oberlander (1995) make a finer distinction between the perceptual basis of the representation apprehension and the possibility to express information in radically distinct ways using the same perceptual channel (modalities).

27 •

Specificity - this dimension is related to the fact that different representations might have different computational properties. Given the broad character of the definition, the authors refer to two approaches: (a) the first, more general, seems to include the different studies that cover the problem of finding the cognitive properties of representations (for example, Larkin, 1989; for example, Larkin & Simon, 1987; Zhang & Norman, 1994, 1995) and (b) the second is related to Stenning and Oberlander (1995) theory. Considering this framing, Scaife and Rogers (1996) description of the cognitive properties of ERs (covered in section 2.1) capture the essence of the whole issue with the added benefit of being more explicit about the actual properties to be studied. In fact, Scaife and Rogers (1996) refer that: (a) a good example of a study that addresses the issue of computational offloading is Larkin's and Simon work; (b) rerepresentation is nicely illustrated by Zhang's and Norman research; (c) graphical constraining is related to Stenning's et al. investigations. However, it should be noted that graphical constraining is a sub-set of the scope of the implications that the specificity theory covers, as Stenning et al. conceptualise it. Nevertheless, since this thesis addresses the problem of interactive graphical representations, it is reasonable to substitute this particular dimension, specificity (in this case following de Jong et al.'s specification, and safeguarding against the abusive use of the term specificity), with the cognitive properties proposed by Scaife and Rogers (1996).

•

Complexity - this dimension acknowledges that different amounts of information can be "included" in a representation. Given that many systems have more than one representation, an important research topic relates to the effect of informational redundancy between representations in learning systems. However, the authors comment that from fully redundant multiple representations the same information is derivable. This affirmation resembles the notion of informational equivalence proposed by Larkin and Simon (1987).

Taking into account the discussion about optimal levels of computational offloading and the present focus on multi-representational ILEs, it is clear that these

28 dimensions can be manipulated in order to match the desired level of offloading throughout the learning activity. In particular, the interactions between precision, cognitive properties of ERs, modality and complexity seem worth further investigation. Ainsworth (1999a; 1999b) goes one step further on the need to focus on the specificities of the use of MERs by providing a taxonomy of their functions2. In fact, the author's taxonomy addresses the important issue of the distinct types of relationships that different ERs might establish between themselves in an ILE and how these might be used to foster learning. The author considers the following: •

MERs in complementary roles - this functional role decomposes into two functions: o MERs to support complementary processes - in which case the representations

are

informationally

equivalent

but

not

computationally. The idea is to take advantage of the distinct cognitive mechanisms involved in the processing of each representation and optimise learning. The author refers three classes of reasons to exploit this path: the diversity of preferences exhibited by learners, the diversity of tasks and the possibility of improving performance by using more than one strategy. o MERs to support complementary information - the two possibilities are:

either

each

representation

has

different

information or there is a certain degree of redundancy between the information that the representations encode. For the first case, Ainsworth considers that the major reason for this design decision is to make representations more usable. For the second case, the partial redundancy of information might support distinct interpretations of the domain. •

MERs to constrain interpretation - use of the learner's understanding of one representation to constrain the interpretation of another one. Two particular cases are suggested: the use of a familiar representation that

2

The dimensions proposed by de Jon et al. (1998) can be studied separately in systems that do not have multiple representations while Ainsworth theme is clearly directed to these multi-representational systems.

29 helps interpreting a less familiar or more abstract one, and the use of a representation whose properties may constraint the interpretation of another one. •

MERs to construct deeper understanding - this function is decomposed into three components corresponding to three sub-specifications of the meaning "deeper understanding": o MERs to support abstraction - the idea is that by giving learners the possibility to explore different domain representations, they will build commonalities across these representations. This knowledge about shared references across representations can be used to elicit higher-level cognitive structures. o MERs to support extension - in this instance the goal is to encourage generalisation to new situations. The two possibilities advanced are to extend the domain of use of a representation and to extend the way the domain knowledge is embodied to include other representations. o MERs to teach the relationships between representations - the main focus is to promote the understanding of the existing links between the representations. The process of translating between representations is a core issue.

Ainsworth (1999a; 1999b) acknowledges that, in most situations, the built ILEs do not fit nicely onto one of the functions proposed; the norm is that more than one of these functions are at play. Hence, a crucial part of the evaluation of an ILE that uses MERs is to clearly understand which functions are present and how they fit with the learning goals. Summarising, de Jong et al.'s (1998) set of dimensions, with the corresponding amendments referred to, serve to typify the ERs that are used in an ILE while Ainsworth's (1999a; 1999b) further enhances the characterisation of the ILE by specifying the relationships between the ERs. These two tools complement each other. Its use seems to contribute to the clarification of the research topics that a particular experiment studying one or more specific ILE(s) address.

30

3.2 Some key empirical findings concerning the use of multiple representations in interactive learning environments Cox (1996) presents a series of studies regarding the employment of ERs to teach first order logic. In relation to the serial utilisation of MERs involving switching between them (some other studies focus on the concurrent instead of serial use), the following issues seem to be particularly relevant: •

The identification of two distinct patterns of interaction: (i) the "judicious switching", when learners systematically take advantage of the expressive powers of each type of representation and (ii) "thrashing", that characterises learners who just jumped from wrongly built representations and attempted something different with other types.

•

The finding about the difficulties learners had with translation between representations.

In fact, learners difficulty in co-ordinating and translating between different representations is central to the problem of using multiple representations in learning systems (Ainsworth, 1999a, 1999b; Ainsworth, Bibby, & Wood, 1998; Ainsworth, Wood, & Bibby, 1996; Cox, 1996, 1999; Janvier, 1987; Tabachneck-Schijf, Leonardo, & Simon, 1997; Tabachneck-Schijf & Simon, 1998; Tabachneck, Koedinger, & Nathan, 1994). Rogers and Scaife (1998) propose the notion of dynalinking. The idea was to find a way to support children's learning and consequent use of abstract forms of ERs (in this particular study the case was food webs). The authors' approach involved thinking about how to design different interactive representations, each expressing distinct levels of abstraction of information. The next step was to interlink the representations making the relationships between the distinct ERs explicit. This strategy should enable children to reach the meaning of more abstract forms. However, a quite difficult challenge is to find these intermediate representations, especially in domains already very abstract by their own nature (for example: mathematics, geometry or some non-intuitive physics phenomena). In these cases, one has to avoid falling into the trap of developing representations that elicit wrong analogies and misconceptions.

31 Ainsworth (1999a; 1999b) hypothesises about the different ways translation between representations can be implemented to match the learning goals and the distinct functions that MERs can accommodate. Furthermore, the author also addresses the question of how to evaluate the benefits of using MERs, concentrating on the need to analyse learners' comprehension of each representation taken separately and/or to judge the level of translation between representations achieved.

3.3 Summary The previous chapter (chapter 2) contains several references to MERs. In fact, in real world contexts, it is not usual to observe the use one type of ER in isolation. More often than not, diverse ERs appear together. The chapter pointed out that a possible advantage of interactive graphical representations for learning is the chance to use the extra interactivity to explicitly represent the existence and nature of the relationships between ERs. This present chapter focused on the use of MERs for learning. It started by describing a set of dimensions that characterise ERs and a taxonomy of the functions of MERs. The aim was to discuss a conceptual tool characterising the ERs built for the main experiment in this thesis and, in this way, help place this present research in the field. Some key empirical findings regarding the use of MERs for learning were also presented. These are: •

The existence of distinct patterns of learners' interactions with computer systems that employ MERs.

•

The observation that learners have difficulty co-ordinating and translating between different ERs.

This research incorporated the topics reviewed by: •

Using the classification tools, dimensions and functional taxonomy to describe in detail the multi representational system built for the main experiment (see chapter 5).

•

Analysing how learners used the different ERs provided. Furthermore, we looked at the way the learners switched between different types of ERs

32 with the intent of identifying interaction patterns and relate them to performance (see chapter 7).

33

4 A pilot study on design issues related to the integration of virtual environments in ILEs This chapter reports on a pilot study undertaken with the primary goal of evaluating design issues related to the integration of desktop VEs technology in ILEs. In fact, although some research has been conducted regarding usability issues and corresponding design solutions for VEs (for example, Ford, 2000; Kaur, 1998; Ruddle, Howes, Payne, & Jones, 2000; Wickens & Baker, 1995), the way that this type of graphical representation and its associated technology integrates in ILEs that make use of other types of ERs, say text, as not been sufficiently covered. For example, it is not clear how the interactivity provided by VEs affects the way learners explore other types of ERs present in the same ILE. Furthermore, it is also uncertain how co-ordination of VEs and other types of ERs is achieved or can be promoted through design solutions. A second goal of this pilot study was to use the findings to start the development of a methodology to assess the efficiency of multi-representational ILEs that use VEs for conceptual learning. In order to start exploring these issues, this usability study was conducted with an ILE that uses 3D VE technology but also displays a separate frame with text. The ILE under investigation, the Vari House3 system, aims to teach basic principles of archaeology. More specifically, the Vari House ILE makes use of Web technology (HTML and VRML) and is composed of three frames: one frame presents a virtual archaeological site for learners to explore -the Vari House VE-, another frame displays virtual objects that can be manipulated and the third frame shows text. Some parts of the text are directly related to activities to be performed on the VE or objects frames but other parts concern information about archaeology (the design is described in more detail in section 4.1.1). Thus, the general characteristics of the Vari House ILE, including the employment of VRML technology, seemed appropriate to be used for this pilot study. The remaining of the chapter goes as follows. Section 4.1 describes the system under study: the Vari-House system. Section 4.2 describes the methods and procedure 3

The Vari House system used was a beta version and it is property of Learning Sites Inc. The author would like to express his sincere thanks for the permission granted to use the system on his research.

34 used. Section 4.3 concerns the findings from the study. Section 4.4 gives an overview of the study stating some research topics that seem to deserve further attention as well as the lessons learned for the development of the ILEs to be utilised in the main experiment. In relation to the ILEs developed for the main experiment, it should be noted that the applied domain changes from one used on the pilot study - the domain on the pilot study is archaeology while on the main experiment it is geometry. Although the decision regarding the changing of the domain was made after the completion of the pilot study analysis, the reasons for such can be put forward in advance. These were: •

In order to pursue further the investigation topics identified some alterations to the Vari House ILE design were needed. However, since the ILE is a proprietary system changing it would have had to involve negotiation with the proprietor.

•

Nevertheless, another option could have been to build new ILEs but not to change the domain. However, it should be pointed out that developing an ILE for archaeology definitely involves considerable effort on computer graphics programming. Such effort would have been certainly a major problem considering the available computer graphics programming skills.

•

At the time it was also thought that the geometry domain allows a better grasp of the problem space boundaries than archaeology. Such characteristic seemed to be a plus for pursuing the analysis of learning benefits.

4.1 Description of the Vari House system This section will cover a brief description of CosmoPlayer 2.1, the VRML viewer used for the interaction with Vari House ILE and the description of the system itself.

4.1.1 Description of the Vari House package The Vari House ILE aims to teach archaeology to grades 6-12, through the use of virtual worlds (VRML files), text (HTML files) and pictures/diagrams (usually JPEG files). The content is very precise and accurate in respect to the archaeological information, either in the text, pictures/diagrams and virtual worlds. The system was

35 constructed to supplement classroom teaching, considering that the teacher should have an active role on the organisation of the learning sessions. The teacher can, in a certain degree, organise the sequence of the exploration of the links and so establish the connections between different learning units. Two manuals are provided: one for the teacher, with suggestions about planning the lessons, and one for the students, with questions about the content. Figure 1 shows that the system employs frames, giving the possibility to display the virtual worlds and text information at the same time. In fact, in most of the cases, the system displays three different windows: one with the virtual world (restricting the learner to the use the movement mode controls, see next section), one that shows the objects available in the virtual world (restricting the learner to utilise only the examine mode controls, see next section) and one with text.

Objects window

VE window

Text Window

Figure 1 - Snapshot of the Vari House system displaying the more frequent layout of three different frames

In terms of the navigation through the content, the learner is initially presented with a simple HTML page containing hyperlinks to the different themes covered and also to the general instructions on how to use the ILE. The hyperlinks concerning the themes in this initial step are presented in a table. For example, by clicking on the top left one the learner is taken to a description of the archaeological site (see Figure 2). In this case the corresponding files are just one text (HTML) file and some pictures. However, the more common layout is the three frames one referred to above. In some sense, the themes are organised into learning units that usually contain text, a virtual

36 world and probably some objects. The learner navigates from learning unit to learning unit by using the hyperlinks presented in a table similar to the one described earlier (see Figure 2). The table is located at the end of the text of each learning unit. Furthermore, the virtual worlds, in the VE-window, also have some objects and by clicking on them the user is able to interact with the corresponding virtual object in the object-window. Moreover, all the objects shown in the object-window have a corresponding HTML document to be displayed in the text-window with some explanation about it. One point concerning the content navigation clearly emerges, the text window can be "updated" by two different methods: clicking on the hyperlinks present in the units' table or by clicking the objects in the objects-window. This design poses some problems concerning learners' awareness of theirs content location.

Figure 2 - Figure displaying the common table with the hyperlinks to the different content units

It should be pointed out that the learner does not have a clear indication from the ILE of the content exploration sequence. Considering that the authors of the ILE envisioned its use in classroom teaching, guiding through the sessions was thought to be conducted by the teacher. This means that, at least partially, the sequence of the exploration was to be determined by the teacher.

4.1.2 Brief description of CosmoPlayer 2.1 The recommended VRML viewer to use with the Vari House ILE is CosmoPlayer 2.X. This VRML viewer has two basic different modes for interaction: the movement mode and the examine mode. The two modes are set by pushing a small virtual handle and each one has a different set of controls that are displayed in the central part of the console. The movement mode has: •

Go - allows the moving in the world (right, left, forward and backward). This control turns the view of the user in the direction of the travel.

37 •

Slide - permits the sliding straight up or down, right or left and it does not turn the view in the direction of travel.

•

Tilt - can be used to look up and down, right or left but without moving.

•

Gravity/float - this controls the possibility to fly when moving in the world, or in other words, the possibility of changing the height of the moving.

The examine controls are: •

Rotate - allows the user to spin the object and is the default control for the examine mode.

•

Pan - is the counterpart of slide in the examine mode.

•

Zoom - permits the user to zoom in and out the object.

Furthermore, there is a group of key combinations that permit, to more experienced users, faster ways of choosing and switching controls instead of using the console. Finally, there are some controls available to both modes of interaction: •

Seek - this control allows the user to click an object and go directly to it turning off immediately after. There is also a Continuous Seek mode that acts as if the button would be continuously pressed but without being able to interact with the objects.

•

Straighten - straight up the user's view and level in the VE.

•

Undo/Redo - with similar use to other systems, these buttons keep track of the places where the user stopped through his path going back or forward to previous places.

•

Viewpoints area - the controls related to viewpoints are found here: information about the current viewpoint, a list of the viewpoints available, two buttons that permit the user to go the previous or next viewpoint. This facility allows the navigation by Viewpoints. In fact, alternatively to using

38 the movement controls referred to the user can also utilise predefined navigation paths by selecting specific pre-programmed viewpoints from a list. In other words, the user can select a certain location in the VE and the VRML viewer "takes" him there. •

Change of controls handle - this handle controls the mode of interaction of CosmoPlayer. The user has to drag the handle in order to change between modes.

CosmoPlayer has quite a few different controls. The trade-off is between allowing a greater flexibility in the interaction with the virtual worlds for expert users and some possible initial confusion to novice users. It seems that the functionalities offered in the viewer cover the major interaction needs for desktop type VEs. However, there is always the possibility for the virtual world designer to restrict the user to a specific mode of interaction (and this way restrict the number of controls available) by defining it in the VRML file.

4.2 The study To test the software, three Sussex University PhD students with some knowledge about HCI acted as participants. Whilst not children, they acted as good testers of the software's usability. Each subject was given an introduction about the goals of the Vari House package and a CosmoPlayer demonstration. They were also presented with five questions that they should answer, all of them taken from the students' manual of the package (see Appendix I- A). After this initial overview and brief instructions, they were told to browse freely through the system as long as they wanted and asked, if possible, to think aloud. The session was videotaped through two different sources: a video camera that recorded the actions that users produced ''outside'' the system and a direct recording of the activity inside the system using the out connection of the computer to a video recorder. For the analysis of the videos a detailed notation that covers the functionality of the system was developed (see Appendix I - B). This strategy allowed the construction of a profile of the overall activity of each participant, which helped to formulate the criteria/questions to evaluate the system. The main questions asked were:

39 •

How were the interface controls used? This involves understanding both if the participants have difficulties comprehending the function that each control performs and how do they integrate the different controls present in the VRML browser into a sequence of actions.

•

Which worlds and documents were browsed? It is important to understand if the participants are able to explore all the representations available. It could be the case that, since they did not have an overview of all the worlds and documents of the system; they would not explore all the information. If this is true, why did this happen? Were they unable to recall which worlds and documents they visited?

•

How and when did the participants move between the different windows/representations? This could tell something about preferences of browsing and perceived linking between windows.

After the session each participant was asked general questions about their experience with the system, covering the points and criteria stated above and possible suggestions for improving the system. All the participants were highly co-operative.

4.3 The results and some design suggestions In order to have a first impression of the interaction patterns Figure 3, Figure 4 and Figure 5 plot the time that each participant spent in each type of window, the sequencing and overall time spent interacting with the Vari House system4.

4

Please note that the width of the bars on the plots do not mean time spent on the window. The information about the time spent on the window is given by the Y axis.

40

Minutes spent on the activity

Participant 1 10 9 8 7

Text Objects World

6 5 4 3 2 1 0 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

Actions performed (the numbering corresponds to the actual sequence)

Figure 3 - Plot displaying the interaction pattern of participant 1 in relation to the time spent in each window and the sequence of transitions between the different windows

Minutes spent on the activity

Participant 2 10 9 8 7 6

Text Objects World

5 4 3 2 1 0 1

2

3 4

5

6

7

8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33

Actions performed (the numbering corresponds to the actual sequence)

Figure 4 - Plot displaying the interaction pattern of participant 2 in relation to the time spent in each window and the sequence of transitions between the different windows

41

Participant 3 10 9

Minutes spent on the activity

8 7 6

Text Objects World

5 4 3 2 1 0 1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

Actions performed (the numbering corresponds to the actual sequence)

Figure 5 - Plot displaying the interaction pattern of participant 3 in relation to the time spent in each window and the sequence of transitions between the different windows

Figure 3, Figure 4 and Figure 5 show that the three participants display different patterns of interaction: •

Different frequency of windows use and consequently the number of transitions between the

distinct

windows,

especially

comparing

participants 1 and 2 with participant 3. •

Different amount of time spent on average in the different windows.

In general, the three participants did not have many difficulties understanding the functionalities of the VRML controls. However, the integration of the different controls of the moving or examining modes into navigation or manipulation activity was not always easy. This means that although the functionality of each control was clear, sometimes choosing the controls or sequence of controls in order to perform a structured activity was difficult. As examples: (a) sometimes the participants seemed to be trying different controls in small steps until deciding which one was more suitable for the intended action and in what sequence should they be used or (b) just utilise one or two controls in a efficacious but not efficient way. In relation to the use of controls for navigation in the VE, one way to ease the task is to provide pre-specified viewpoints

42 and associated pre-programmed navigation paths to travel to the viewpoints. Although this functionality was provided in the Vari House ILE, the participants did not use it nor did they enquire about its possibilities. However, it should be noted that the experimenter did not explicitly explained this particular control to the participants. Considering the amount of controls explained in the introduction and available for use in the interface, maybe the participants did not pay attention to this particular feature. Further research is needed to investigate the benefits of such functionality in comparison with user "driven" navigation. For example, are pre-specified viewpoints and navigation helpful for less experienced users or other specific group of users? Are pre-specified viewpoints and navigation more useful in small scale or large scale VEs? This later topic is relevant taking into consideration that pre-defined navigation paths might not promote exploration of the environment which in turn might influence negatively the ability to represent and memorise spatial properties of the VE. As an overview of the participants' general interaction patterns: •

Participant 1 (see Figure 3) was highly interested in the moving world window. However the text window was browsed in a quite quick way, which puts some doubt about the level of attention spent on reading the text. The learner chose to answer the questions at the end of the interaction with the ILE.

•

Participant 2 used the controls the least of the three participants in the moving and examine windows. This is probably due to a lack of experience with interacting with 3D and the choice of a cautious mode of interacting. The participant clearly stated that the questions given acted as activity goals. She concentrated her efforts on the documents, searching for the information to answer the questions. Some breakdowns of the interaction occurred due to difficulties of understanding how the Internet browser's controls work with the frames and hyperlinks implemented in the Vari House ILE. The interaction plot indicates that (see Figure 4): (a) the preferred window was the text one, the participant seemed to read the texts more carefully than participant one and showed a more structured search for the information in the text to answer the questions, (b) the

43 questions were started to be answered during interaction with the ILE, peaking in the end. •

Participant 3 (see Figure 5) also preferred the text window and also said that he had adopted the questions as goals for the activity. Comparing to participant 2, he apparently performed a very intense search for information on the documents at the beginning of the interaction while participant 2 seems to have done it more at the end. Moreover, participant 3 did not seem to have big difficulties in using the virtual worlds and so he utilised more controls. He also answered the questions as the interaction went on, and not at the end as participant 1.

In relation to the answers given by the three participants, although fairly correct they were not very detailed, making comparison and discrimination between them more difficult. Nevertheless, the answers of participants 2 and 3 were more detailed than those of participant 1, since the responses of the later had fewer references to specific information that could be found in the documents. Participants 2 and 3 spent considerably more time in the text window, without switching to other windows, which might have given them a better sense of the information to be collected. Moreover, answering while interacting might also have improved their performance. If correct, this finding raises some issues about the level of integration of the information in the documents and the worlds. If the more relevant information is not in the world and certain participants tend to focus in them then a learning objective might be lost. Reviewing the general problems found: •

The navigation in the VE - There were no major problems to register. However a minor problem occurred when the participants collided with an element of the world, especially inside buildings: there was no feedback about the situation, and sometimes that proved to be confusing.

•

The manipulation of virtual objects - This functionality registered some difficulties related to the understanding of how to grab the objects and manipulate them. The intuitive choice is to go directly to the object and not to its boundaries, as was required in some cases. Furthermore, one of the users suggested that object and information about it could be

44 displayed at the same time, avoiding the extra step of clicking the object to display the information in the text window. Finally, the identification of the clickable virtual objects was sometimes problematical. Some of the confusion comes from their similar shape and proximity to other objects that did not show the same behaviour. Similar objects should present the same type of behaviour. Moreover, if the user does not seem to understand that a certain element of the virtual world is an object, then feedback should be provided. For example, if the user passes by a certain object two or three times without clicking then the object could change colour. •

The use of frames and the understanding of how the different file structures worked seemed problematic. For example, in two cases one user felt lost because of continuous clicking on the BACK button of the Internet browser. He was searching for a previous HTML document but did not remember to use the history facility of the browser. Furthermore, the users did not explore all the worlds and HTML documents which indicates additional support of content exploration problems. However, considering that in the classroom context envisioned by the authors the teachers would provide structure to the explorations this problem might not be so urgent. Nevertheless, it clearly draws attention to the possible difficulties of learners' understanding of the documents and worlds tree structure. The design problems involving the use of frames are well documented (Nielsen, 2000).

•

Another issue is to understand how the participants switched between the virtual worlds and documents. This addresses the question of how the integration between the two different representations was being made. The observation and analysis of participants' behaviour suggests that while they were performing a structured browse either on the text or in virtual world, the preference was not to switch between windows. They seemed to favour exploring the world or reading the text in more detail and then switch. For example, at one point a user said, while browsing the text: ''OK! This gives me things to look for. Now I have to go to the house!'' However, when they were exploring the virtual worlds and found

45 an object they tended to click on it and, most of the time, would then read the information provided. Therefore this design appears to promote the changing of the activity. Furthermore, since the examination of a virtual object might imply the changing of texts in the text window and since the structure of the files was not well understood then the whole sequence might be disrupting the possibility of a structured and planned information search. Finally, one of the users complained about the length of some of the HTML documents, considering in particular that some of the questions inscribed in the text did not seem highly related to the overall subject. So a possible question is: in what way do these questions in the text prompt actions in the virtual world? Furthermore, it is also interesting to know how such actions might help the understanding of the text and content. Finally, considering that a user might act just after the suggestion for a particular action, there is also the issue of the effect of switching between representations and the possible cognitive processes involved in keeping track of the text and related implications for the learning process. Concerning this particular problem some design solutions could involve better supporting, for example, the building of an image of the available worlds and documents, stating clearly the relationships between them. This could be achieved by: o Providing a representation that clearly displays names of the

virtual worlds and their associated documents. For example, a possible design solution related to the correspondence between virtual objects and accompanying text would be to use a common background colour for each object and related document for an easier recognition of the link. o Providing a Back button in each document that would display the

last document browsed. o Developing a history register of the browsed worlds and

documents (in fact this was suggested by one of the users) or specify the possibility of using the one provided by the browser.

46 o Thinking about the feasibility of a different chunking of the

information in the pages and how to guide the learner through it. Maybe the teachers' guidance is a key issue to explore further.

4.4 Conclusions: design issues to inform the design of ILE and topics for further investigation The pilot study and the qualitative analysis performed uncovered some design issues related to multiple-frames' ILEs as well as the use of VEs in combination with other types of external representations. The findings informed the design of the ILEs used in the main experiment as well as promoted the framing of some research topics.

4.4.1 Findings •

In general, it seems fair to say that, on the activity of these three users, the virtual worlds acted as a support representation of the information displayed in the HTML pages. A point for further investigation is to know how the questions placed in the HTML pages, with the objective of providing suggestions for active exploration, sometimes in the VE, fulfil the expectations and shape the learning and integration of information and learners' experiences. It seems to be a question of knowing how the coordination of information and learning activity emerges when learners need to explore 3D worlds and text.

•

One main difficulty with the Vari House system seems to be learners' perception of the files' organisation and structure. In particular, the problem of knowing how to link the documents to the virtual worlds is crucial.

•

The participants were able to learn some of the content in the ILE and answer the questions in their 45 minutes (on average) of activity: they answered the questions in a fairly accurate way although they were not very detailed or elaborated. They reported having acquired information about archaeology in a novel and interesting way. Answering the questions basically needed a good search through the information available in the HTML pages. It can also be observed that the subject who

47 spent more time just browsing the virtual world and not exploring the text in more detail gave the less rich answers, because the key information for answering was in the text. But this observation would need more data to be able to generate a more substantiated comment. •

Concerning the interaction, the two subjects who had more experience with 3D interacted more with the virtual worlds and engaged more in exploration. Furthermore, the switching between different types of representations was sometimes slightly confusing and the two subjects who showed better answers tended to use the questions as goals for framing the activity.

•

VEs seem to promote exploratory activities. A crucial design topic is to understand what amount of structure and direction should be embedded in the VE as an aid to the learning process and also how to implement it. Considering the use of other ERs in relation to the VE, it should be understood which are the types that best fit together and the best sequencing for their presentation to maximize the overall efficiency of learning.

4.4.2 Lessons learned that informed the design of the main experiment's ILEs and topics of research In relation to the "lessons" learned for the design of the ILEs to be used on the main experiment and research topics to be pursued further, the following can be stated (the actual design solutions implemented are underlined): •

Design issues: o The participants in the pilot study seemed to have difficulties understanding the structure of the files tree provided - The design

of the ILEs must take into consideration the need to provide a clear view of the structure of the learning units, probably suggesting a sequence for the exploration.

48 The content of the developed ILEs is broken into 6 units. A navigational bar is provided with a numbering system that intends to structure and help the content exploration. o The participants were sometimes confused with the way objects could be manipulated - The objects to be developed must afford

their manipulation in a clear way and all interactive elements should have common features that distinguish them from noninteractive elements. The functionality of rotating the whole diagram present in some of the versions of the ILEs was designed in such a way that turning the virtual object was possible whenever the mouse was over it. Furthermore, all diagrams' interactive elements that are displayed on the ILEs flashed and changed colour for signalling. o The participants followed suggestions present in the text to perform activities in the VE - The textual explanations of the

concepts to be presented in the ILE should point unambiguously to the VEs or other virtual objects for further clarification and some kind of activity upon the latter could be suggested. However, the amount of suggested activity should not be too big and should be carefully selected according to relevance for the overall explanation, since too much encouragement to interact with the VE might distract the learner from exploring more fully the other available ERs. The textual part of the ILEs built displays a table that explicitly makes reference to the elements of the diagrams to be attended to and also to the way the geometric objects are built. This later information can aid the learner tracking the underlying principle of the geometric construction and might also implicitly suggest exploratory actions. The combination of text information, implicit suggestions for activity and actual actions might encourage the integration of information from text and diagrams.

49 •

Research topics: o The way participants in the pilot study distinctively switched

between the ERs suggests that this can be related to content understanding and acquisition in ways not really discernible in the study. In fact such phenomena might be related to different strategies to tackle co-ordination between multiple ERs (see chapter 3 for a literature review on the topic). The qualitative analysis performed on the main experiment regarding the video recordings of learners' interactions clearly addressed the issue by focusing on the learners' transitions between the available ERs. o The general methodology used in the pilot study proved to be

valuable and was further utilised with some adaptations in the main experiment. Finally, from the pilot study some reflection upon the best way to start studying the properties of the graphical representations more often used in VEs and their effects on the learning was initiated. The idea was to begin with a basic investigation regarding the properties of interactive graphical representations and establish a research baseline. The baseline principle involved the study of each property at a time, whenever feasible. The research results obtained with the baseline could, then, be extended with further research in a step-by-step manner. However, the Vari House system did not seem to allow such approach. The design problems found would have required some changes and since this was a proprietary system the project would, probably, have been difficult. Thus, for the main experiment some ILEs were purposely built. Furthermore, a decision was made to focus on the study of the possibility to manipulate computer generated graphical objects and his effect on the learning. Moreover, one basic question kept coming to mind: why use 3D technology to teach anyway? To reassure ourselves of the need to use 3D in an ILE a decision was made to look for a domain where people were unequivocally trying to depict 3D properties of an object. Furthermore, considering the available resources regarding graphical programming skills the choice of domain would have to fall on a problem that would not be to demanding on the programming side. The search led to geometry and the study of one type of projection: the stereographic

50 projection. These are the basics of the main investigation and, definitely, the topic of the rest of the thesis.

51

5 The design of the interactive learning environments This chapter starts (section 5.1) by presenting an initial model of the learning activity, bearing in mind the general goal of investigating the benefits of using interactive diagrams to explain a concept that requires the depiction of 3D properties. It continues in section 5.2 with the description of some general assumptions about learning, considering the provided external representations (ERs). Section 5.3 provides a general description of the interactive learning environments (ILEs) designed and goes into detail about the diagrams and their interactivity. Section 5.4 presents a normative model of the learners' interaction patterns.

5.1 A learners' activity model for understanding how learners integrate and use multiple representations 5.1.1 Brief introduction to the domain As noted before the main goal of this research is to investigate whether providing interactive graphical representations for the explanation of an abstract, visual concept, that requires the depiction of objects with 3D properties, is beneficial for learning. To test this, the stereographic projection was chosen as the application domain, a one-toone transformation from the surface of the sphere (the surface of a 3D geometric entity), less one point, onto a plane (by definition a 2D geometric entity). In general terms, the concept belongs to the geometry domain, but in the case of this study, there is an emphasis on one of its particular applications: the problem of studying symmetry relationships in crystallography. The use of the stereographic projection in crystallography involves explaining the process of representing the angular relationships between the faces of a crystal using a 2D representation called a stereogram. Some of the graphical representations in textbooks explaining the concept are 2D diagrammatic representations that, through the use of pictorial cues are intended to show 3D "objects" (see for example Borchardt-Ott, 1995).

5.1.2 The interactivity properties under investigation The interactivity of the graphical representations designed can be decomposed into two different properties/functionalities: (a) the possibility (or not) to rotate the

52 whole object and (b) the possibility (or not) to move elements of the object in order to apprehend the relationships between them. The first property stated above is related to the depiction of structural properties of 3D objects using motion or perspective cues. The perception of a 3D object using a 2D display is inherently ambiguous (Proffitt & Kaiser, 1991). The visual cues that allow the human visual system to perceive depth have been investigated thoroughly and can be divided into two classes: (a) the primary cues that comprise binocular disparity, convergence and accommodation and (b) the secondary cues that contain, as examples, perspective and elevation, size, texture, shading and shadow, motion, reference frames (Hubona, Shirah, & Fout, 1997). According to Proffitt and Kaiser (1991) the successful use of pictorial conventions for representing 3D objects is questionable since several studies show consistent errors when humans try to infer 3D structure from a single 2D projection. However, Proffitt and Kaiser (1991) argue that motion is a minimally sufficient condition for enhancing the perception of three-dimensional forms. In a classic study, Wallach and O'Connell (1953) demonstrated that people can effectively recover 3D form when presented with 2D projections of rotating objects. This effect was called by the authors the Kinetic Depth Effect - KDE. Furthermore, user-controlled manipulation of objects rotation seems to be preferable than user un-controlled rotations (Hubona et al., 1997). In the ILEs developed, the interactive property being provided in some of the diagrams is the chance to directly manipulate the representation through the use of a peripheral (the mouse). Such facility aim at facilitating the perception of the objects' 3D properties instead of having to apprehend the 3D form through the decoding of perspective cues. Considering the arguments put forward concerning the perception of 3D form using a 2D display, one can think that providing whole diagram manipulation is definitely a plus for recognising the 3D properties of the diagrams built to explain the stereographic projection. However, and contrary to the experimental settings of most of the studies that address the problem of objection recognition, in the case of the built ILEs, there is explicit reference and naming of some of the elements that compose the diagrams which should aid the objects' identification. Hence, maybe providing the possibility to manipulate the whole diagram is not that decisive. The second interactivity property investigates whether letting the learner manipulate elements of the graphical representation to be able to simulate the

53 relationship between them is beneficial. Further discussion concerning the possible benefits of the two interactivity properties implemented can be found in sub-section 5.2. In order to try to assess the benefits of these two properties four different versions of the diagrams and corresponding ILEs were created. Table 1 displays the combinations of the two properties and related diagrams' versions:

Manipulation of diagrams' elements - ED

Yes No

Manipulation of diagrams - WD Yes No WD+ED ED WD BASIC

Table 1 - The different diagrams versions

The previous section briefly described the domain and this present section introduced the main diagram design variables to be investigated. However, to describe in detail the space problem of our investigation and keep track of the underlying conceptual questions the consideration of a theoretical framework was found to be useful. This is the theme of the next section.

5.1.3 The theoretical framework The following diagram establishes the general research space of the present work, showing an initial model of the learning process that occurs when a learner interacts with an ILE with the characteristics of the ones developed for this research. The model helped to set the investigation and guide the construction of relevant conceptual questions that are thought to underlie this specific learning activity. Furthermore, the description of the ILEs was enhanced using de Jong's et al. (1998) dimensions: perspective, precision, modality, specificity and complexity (for a more detailed description and discussion of these see section 3.1). This approach can help compare the characteristics of the ILEs built with the ones used in other studies. However, as discussed in section 3.1 regarding the characterisation of multi-representational ILEs, Scaife's and Rogers (1996) cognitive properties of ERs were integrated (computational offloading, re-representation, graphical constraining and temporal and spatial constraining) since it can be argued that they are more precise than de Jong's et al. specificity dimension. Finally, the use of Ainsworth's taxonomy (also covered in more detail in section 3.1) of the functions that multiple ERs play in ILEs can frame the type

54 of relationships that the ERs utilised in the ILEs have. In fact, the author considers that a crucial part of the evaluation of an ILE that uses multiple external representations (MERs) is to clearly understand which functions are present and how these fit with the learning goals. Learning Activity Types of Diagrams

WD+ED

BASIC ED

WD

Recognizing the elements of the diagrams and their relationships

A) Facilitating effect B) Inhibiting effect

Integrating diagrams and text

Understanding the text

Text

Figure 6 - The learning activity model

All ILEs contain two types of representations, text and diagrams. The ILEs will be described following de Jong et al. (1998) proposed "dimensions" to portray representations, Scaife and Rogers's cognitive properties of ERs and design principles (Scaife & Rogers, 1996) and Ainsworth's functional taxonomy of MERs (Ainsworth, 1999a, 1999b). In relation to perspective (de Jong et al., 1998) there is just one perspective available since the textual explanation and accompanying diagrammatic representations in all the ILEs do not offer radically different views of the concept to be explained (being able to rotate the diagram does not seem to be enough in order to think of different perspectives). Considering precision (de Jong et al., 1998), the explanation provided is qualitative since no analytical or algebraic expressions are employed. The decision about using this level of precision took into account the fact that the ILEs were intended to introduce the participants to the concept of stereographic projection and not to provide a fully-fledged and meticulous explanation (see du Boulay, 2000; Sime,

55 1996, in the domain of teaching engineering, who discusses the benefit of promoting first a qualitative understanding, before immersing in detailed quantitative approaches). In terms of modality (de Jong et al., 1998), all the ILEs employ both textual and graphical representations. Furthermore, the graphical representations used and integrated in the ILEs are more or less interactive, including or not the possibility to rotate the whole diagram or manipulate its elements. As referred to in the literature review, the specificity dimension (de Jong et al., 1998) was substituted by the cognitive properties and design principles proposed by Scaife and Rogers (1996). In principle, if one focuses on the different versions of diagrams provided, then the more interactive diagrams should off-load (Scaife & Rogers, 1996) cognitive and computational effort. These diagrams provide the necessary information to capture the nature of the objects to be apprehended in a more "direct" way through the use of manipulation, either to see the 3D nature of the diagrams, the explicitness and visibility design principle (Scaife & Rogers, 1996), or giving the possibility to simulate the relationship between its elements, the interactivity design principle (Scaife & Rogers, 1996). However, as discussed, some interaction effects between the format of the diagrams, learners' ability to use them and the co-ordination between text and diagrams might alter the balance of benefits of using the interactive versions. The complexity "dimension" (de Jong et al., 1998) is difficult to judge since this research does not possess a methodology to assess possible different levels of complexity for the diagrams built (the reference concerns the diagrammatic representations and not the textual ones since the experimental conditions relate to the former). The diagrams should be informationally equivalent (following Larkin and Simon's 1987 definition) and the differences between the versions concern the balance of performing some tasks to extract information internally or externally. In terms of Ainsworth's functional taxonomy (Ainsworth, 1999a, 1999b), the two types of representations, sentential and diagrammatic, seem to serve a complementary role with partial redundancy of information between the text and diagrams, as will be explained in more detail in the following paragraph. As shown in Figure 6, the learning activity model considers the need for a certain degree of integration between text and diagrams. In fact, in the ILEs developed, there is a certain degree of redundancy concerning the information displayed in the diagrams and in the text. In some cases, though, the overlapping is not complete and one representation may contain unique information. It should also be pointed out that there

56 is a dependency relationship between diagrams and text. It does not seem plausible that some of the diagrams can be understood without referring to the text, especially if one is not familiar with the concept and its implications for the problem at hand. The inclusion of the integration element in the model also relies on findings from previous studies in multiple representation learning systems that acknowledge the importance of this factor and the difficulties that people seem to have with it (Ainsworth, 1999a, 1999b; Ainsworth et al., 1998; Ainsworth et al., 1996; Cox, 1996, 1999; Janvier, 1987; Tabachneck-Schijf et al., 1997; Tabachneck et al., 1994). The model also establishes that this process of integration is dependent, to some extent, on the level of the learner's understanding of the textual and graphical representations considered separately. The issue concerning the influence that learners' levels of text and diagrams understanding considered separately might have on the learning activity is related to the question of knowing which abilities and previous knowledge affect it. In the present case, the choice was to investigate the influence of spatial ability and geometry knowledge levels (for a discussion of the various components of spatial ability and its influence on reasoning tasks with diagrams in the physics domain see Kozhevnikov, Hegarty, & Mayer, 2002). In particular, previous geometry knowledge and level of spatial ability seem to be important variables in the prediction of geometry test scores (Kirby & Boulter, 1999). It is important to point out that one is not claiming that geometry knowledge only relates to the textual part of information present in the ILE, since this type of knowledge also influences the understanding of geometry diagrams. However, the text in the ILE is about geometry and the geometry test (GT - described in more detail in section 6.3) used to assess the geometry knowledge is composed of problems stated in a sentential form sometimes with accompanying diagrams. Hence, it seems reasonable to think that the GT can give information about the learners' levels of geometry knowledge related to text comprehension. The combining of sentential, algebraic and diagrammatic representations is an important characteristic of geometry. In relation to the use of spatial ability5 and geometry tests, the additional implications on the research can be added. First, considering that learners levels' of

5

The spatial ability test utilised was the Paper Folding Test (PFT). This test is described in more detail in section 6.3.

57 spatial ability and geometry knowledge appear to be good predictors of geometry test scores (for example, Kirby & Boulter, 1999) it seemed important to include these variables to allow the control for differences and degree of comparability of the groups formed for the main experiment (see chapter 6 for further details regarding the design of the experiment). Second, it can also be the case that given the specificities of the distinct diagrams developed and corresponding ILEs the effects of learners' levels of spatial ability and geometry knowledge on theirs performance can be different for each type of system. It should be noted that, in order to ensure the capture of different levels of geometry knowledge, the sample under study on the main experiment is composed of two distinct groups of first year undergraduate students: one group comes from a geology course and the other from a mathematics course. It is expected that the mathematics students might show better performance scores on the GT. However, it can also be the case that other variables not controlled by the GT or spatial ability test and related to each group specific background knowledge characteristics might have an impact on the way learners interact with the distinct ILEs and/or learning rates measured by performance tests. The learners' performance on the learning task was assessed using two distinct tests: a multiple-choice test (MCT) done while interacting with the ILEs and a post-test (PT) done after the interaction with the ILEs. The MCT is described in more detail in sub-section 5.4.2. The PT is presented in sub-section 6.3. Finally, the model indicates that the different types of interactivity of the graphical representations might have different facilitating and/or inhibiting effects on the process of integration and concept apprehension. This issue is more extensively covered in section 5.2 when dealing with the general assumptions about learning. The presented model is intended as a framework to help understand the empirical findings obtained in the main experiment and also to suggest further topics for investigation. The current experimental design does not allow to fully test the model. However, one can hypothesise two stages in the understanding of the content, (a) the understanding of the representations in isolation and (b) the integration of information from both representations. A difficulty is that the very nature of a multi-representational learning system that tries to explain a concept like the stereographic projection, is defining what interactions are causing the learning effects. However, the model suggests that the systems' diagrams help to disambiguate the text and facilitate interpretation. The text frames assist the interpretation of the diagrams probably supporting the perceptual

58 search and understanding. Some hypotheses concerning parts of the relationships envisioned in the model are formulated and tested in a quantitative way (chapter seven), while other parts of the learning activity model are covered and informed, looking for trends and further research issues, when presenting the qualitative analysis (chapter eight).

5.2 General considerations about learning with interactive graphical representations The model considered in the previous section refers to possible inhibiting and facilitating effects bound to the different diagrams' versions and ILEs. Here the different, sometimes even concurrent, learning assumptions/considerations related to learning with interactive diagrams and corresponding ILEs are presented: 1. The BASIC ILE, the less interactive of the ILEs developed, might overload the learners with the task of having to decode the pictorial cues that represent the 3D properties of the diagrams and to mentally simulate the relationships between the diagram's elements. Specifically, the BASIC ILE presents the learner with static diagrams. In this format, the learner needs to interpret the pictorial cues provided in order to extract the 3D nature of the elements being depicted. For example, the learner has to understand that the apparent circle is indeed a sphere or that the trapezium is really standing for a plane. Furthermore, the relationship between the diagrams' elements is shown in a static way. For example, if the learner wants to simulate the projection for other points than the ones given, he must to do it mentally. In other words, the task of simulating the projection of other points has to be done internally, unless the learner chooses to extend the diagram provided externalising the inferences made about the "workings" of the whole concept, for example by writing down other possibilities 2. In contrast with assumption number one, by having less interactive properties the BASIC ILE might: (a) force the learner to take a step by step approach to the understanding of the ideas present in the ILE and

59 promote a structured view of the concepts; (b) make the learner focus on the text and not just "play" with the interactive diagrams. 3. The ED, WD and WD+ED ILEs are intended to offload the need: (a) to simulate the relationships between diagrams' elements (ED); (b) to understand the 3D nature of the diagrams (WD); (c) both tasks (WD+ED). This offloading might benefit the learning process by freeing cognitive resources that were being allocated to the understanding of the diagrams properties, and thus allowing a focus on the conceptual content. In more detail: o The ED ILE differs from the BASIC version by allowing the

learner to simulate changing the position of the diagrams' elements over a fixed plane (see Figure 7 for a brief description of one of the diagrams). Hence, it is making the task of simulating partly an external one (because the learner's simulations are constrained to the plane provided). The learner can read off directly from the diagram how altering a certain parameter affects other parameters. Moreover, by providing such a facility it was hoped that this would enhance the learners' ability to simulate the relationships between diagrams' elements internally. o The WD version differs from the BASIC and ED versions insofar

it offloads the need to interpret the pictorial cues. The learner is able to perceive the 3D nature of the objects being represented by directly manipulating the whole diagram. However, the learner is not able to simulate different positioning for the diagrams' elements (contrary to the ED ILE), thus this task relies on internal mental resources. o The final condition - WD+ED - differs from the WD alone by

acting like an extended version of the ED (see Figure 7). The learner can manipulate the whole diagram and, at the same time, the diagrams' elements. Moreover, in some diagrams the manipulation of the diagrams' elements is not restricted to one plane, the learner can move the elements in three directions (x, y

60 and z). The benefit of this is that by offloading the need to interpret the pictorial cues and internally simulate the dynamics of the relationships between the diagrams' elements, the learner might be able to concentrate on experimenting with the diagrams and thus acquire the key ideas and concepts involved by actively testing "hypotheses". 4. However, contrary to assumption 3, the learners might get distracted with the interactive properties and/or not pay due attention to the textual part of the ILE. Furthermore, it should be interesting to investigate which of the offloading strategies proves to be more efficient, taking into consideration the inhibiting factor referred to and also the interaction skills that each type of diagram requires.

Legend - In the ED and WD+ED versions of the diagrams, the projection line (AS) is a manipulable line that can be rotated along the fixed point S. The (A) dot corresponds to the point on the sphere to be projected and moves accordingly whenever the line AS is manipulated. The (a) dot corresponds to the point projected in the projection plane (r) and moves accordingly whenever the line AS is manipulated. Figure 7 - Diagrams displaying the basic principle of the stereographic projection (similar to the ones present in textbooks) and explanation of the available interactive properties

61 In general, it is considered that the interactive versions of the diagrams offload (a) the need to interpret pictorial cues or (b) simulate the relationships between the diagrams' elements or (c) both. However, there is a cost since the learner needs to understand how to work with the functionalities provided (i.e. the moving elements in the diagram).

5.3 Designing the interactive learning environments Considering the general design of the developed ILEs, the concept of the stereographic projection was broken into six explanatory steps, each comprising a certain piece of text and the corresponding elucidative diagram(s). The ILEs were developed as HTML pages built using three different frames (see Figure 8 for a snapshot of the WD+ED ILE): •

A navigational frame, where the user can choose which step to explore, although a sequence is implicit. By clicking on the corresponding button the text and initial diagram of the step are displayed.

•

A text frame, where the text is presented and hyperlinks can provide additional diagrams (where they exist).

•

A diagram frame, where the diagrams are presented. In the WD and WD+ED ILEs an additional bar is displayed corresponding to the commands available in the VRML browser used - Cosmo Player.

62

Figure 8 - Snapshot of the WD+ED ILE, showing the different frames and its content

In relation to the text frame, the steps generally begin with the definitions of concepts, following a two-column table where one side displays the central elements of the diagram to be understood and the other side provides the description of the process followed to construct the corresponding geometrical representation. At the end of each explanatory text/step there are multiple choice questions that the learner is required to answer on a separate sheet of paper. The design of the ILEs and writing of the content was informed by a maths teacher, a geology teacher and a maths post-graduate. Prior to the experiment, a pilot test with 3 subjects was run focusing on possible design and usability issues. The data collected fed into a re-design of the content layout and navigability of the ILEs. In the next sub-sections the goals and general content of each explanatory step as well as the diagrams and their interactivity are introduced.

5.3.1 Step one The general goal of step one is to present the concept of the stereographic projection in broad terms. Some of its properties that are particularly important for the remaining explanations are mentioned, and a specific reference is made focusing on its application to crystallography.

63 The main definitions to be understood in this step are the stereographic projection, the equatorial plane and the primitive circle. To fully understand the definitions referred to, it is useful to identify the following elements: the sphere, the equatorial plane, points A and a, that correspond to the point to be projected and its projection, respectively; and finally, the line AS, which is the projection line of point A. The main relationship between elements to be understood is the definition of the stereographic projection itself, making a link between a point in the surface of the sphere (A), the projection line (AS), the projection plane and the corresponding projected point (a). Knowledge of how to articulate these elements enables one to simulate the projection and acquire the basic principle of the stereographic projection. Another important relationship introduced at this stage, and required for the overall comprehension of the applications of the stereographic projection, is that between the group of points in the surface of the sphere and the centre of the sphere. It provides a complete reference for the directions in 3D space, where each direction is defined by the line that contains the centre of the sphere and one of the points in the sphere's surface. There is just one diagram in this step and this depicts the basic principle of the stereographic projection, showing the projection of a point in the surface of the sphere onto the projection plane (Figure 9).

Figure 9 - Diagram of step one showing the basic principle of the stereographic projection Legend: N - north pole; S - south pole; O - centre of the sphere; r - equatorial plane; e - equatorial line; p - space on the surface of the sphere delimited by e, .A - point to be projected; a stereographic projection of A; AS - projection line when projection is to S.

64 Figure 10 exemplifies the available interactivity for the more interactive ILE, the WD+ED. From Figure 10 A to 10 B the manipulation of the diagram's element is shown while on Figure 10 C the whole diagram is rotated from the position of 10 B.

A - the diagram of step 1

B - the manipulable line is

C - the whole diagram is rotated

moved to the right simulating the

to see where the points lie

projection of points Figure 10 - Screenshots of the step one diagram (see Figure 9) with the exemplification of the interactivity properties available for the WD+ED ILE. Legend: see legend to Figure 9.

In the BASIC version of the diagram, a static picture displaying the projection of one point is presented. The ED version differs from the BASIC one by allowing the learner to simulate the projection of points in a fixed plane. The learner can read off directly from the diagram where the projected points will lie in the projection plane. However, in order to take advantage of the functionality provided, one has to learn how to manipulate the projection line, dragging it to the intended place and recognising the related movement of the point to be projected and the projected point. The WD version simply allows the learner to rotate the whole diagram, offloading the need to interpret the pictorial cues. To rotate the diagram, the learner just needs to click and drag in any direction. This rotation task demands that one understands the mapping between dragging the mouse in a bi-dimensional space (the display) and the rotation of the whole diagram. The WD+ED version combines the functionalities of the ED and the WD, and adds the possibility to simulate the projections of all the points in the surface of the sphere. Consequently there is a need to coordinate the rotations with the moving of the elements and to understand the mappings related to the provided functionalities.

65

5.3.2 Step two The goal of step two is to explain the usefulness of using the stereographic projection in crystallography. The problem is to create a method to classify crystals with respect to the symmetry relations between their faces. The method involves the analysis of the angular relationships between the crystal's faces. The general idea requires a twostage process where the directions of the faces of the crystal are projected onto the sphere - the spherical projection - and then the corresponding points are projected onto the projection plane - the stereographic projection. In the end, one has a flat representation of the directions of the faces of the crystal - the stereogram. The step begins by presenting the reasons why the stereographic projection is used in crystallography. It then proceeds by showing the procedure to build a spherical projection of the faces of the crystal and establishes the connection with the stereographic projection. Some references are made concerning the resulting 2D representation, the stereogram, and examples are given. In this step, the learner acquires an understanding of the following "new" concepts: definition of the normal to the face, which establishes the direction of the face of the crystal; definition of the spherical pole of the face, which is the point where the normal of the crystal face meets the surface of the sphere; definition of the angle between spherical poles, which is the angle between the corresponding normals; definition of the spherical projection. In addition, the learner needs to recall the notion of equatorial plane and stereographic projection, both introduced in the previous step. Regarding the main elements to be identified, one can name: the normal of the face of the crystal, the spherical poles, the angles and arcs between the represented spherical poles and, obviously, the hypothetical crystal whose faces are to be projected. In terms of the relationships between the diagram's elements in this step, it is important to attend to the mappings between them: the faces of the crystal and their corresponding normals, and the resultant spherical poles. Also, the learner has to picture how the spherical poles are projected onto the projection plane, making the connections from the crystal to be represented to the final stereogram, representing the directions of its faces. The step contains four different diagrams: (a) two diagrams displaying two different hypothetical crystals and the necessary elements for the depiction of the

66 spherical projections of its faces, and (b) the two corresponding stereograms of the hypothetical crystals, which, by its own nature, are 2D representations for all the ILEs. The diagrams provided do not have interactive elements in all the different versions. The BASIC and ED versions are equal, just displaying a static picture and the WD and the WD+ED ones are also similar since the learner is only able to rotate the whole diagram in both versions. Overall the step contains four different diagrams: (a) two diagrams displaying two different hypothetical crystals and the necessary elements for the depiction of the spherical projections of its faces, and (b) the two corresponding stereograms of the hypothetical crystals, which, by their own nature, are 2D representations for all the ILEs (for example, see Figure 11, Figure 12 and Figure 13). Hence, in this step, the differences between all the versions are restricted to the possibility or not to offload the need to interpret pictorial cues.

Figure 11 - Diagram showing the principles of the spherical projection Legend: N - north pole; S - south pole; O - centre of the sphere; r - equatorial plane; e - equatorial line; OA and OB - examples of normals to the face of the crystal if drawn from O; A and B examples of spherical poles of the crystal's faces; u - angle between the spherical poles; v - dihedral angle between two faces of the crystal; AB -arc between two sperical poles.

67

Figure 12 - Stereographic projection of the hypothetical crystal shown in Figure 11

A - one of the diagrams of step 2 displaying the

B - the same diagram rotated on the x, y and z axis.

principles of the spherical projection Figure 13 - Screenshots showing the interactivity property provided for the diagram displayed on Figure 11 for the WD and WD+ED ILEs Legend: see legend to Figure 11.

5.3.3 Step three The goal of step three is to introduce the concepts of great circles and to demonstrate its stereographic projections. The explanation begins with some general comments regarding great circles and their projections. It follows with the definitions of the great circle and meridian, plus the description of their projections and accompanying diagrams. Some comments related to the specificities of each projection are also drawn as well as additional references concerning the link between what was presented in step two and the content of step three. The "new" definitions to get hold of are: the great circle, the meridian, the south and north hemispheres and the concept of two symmetric pieces of a circle. Additionally, the concept of the stereographic projection is widely used and, by now,

68 one would expect the learner to feel more comfortable with it. Considering the elements to be identified in this step, one should note that four different diagrams are employed. In all the diagrams, the learner has to identify the now common sphere and projection plane as well as the following specific elements: meridian and great circle for diagram 1 and the projected lines, projected and to be projected points of the corresponding figure in the remaining three diagrams. In terms of the more relevant relationships between elements, in the first diagram one has to understand the difference between the meridian and great circle while diagrams 2, 3 and 4 require the learner to comprehend the relationship between the figures to be projected and the projected outcomes. The first diagram (Figure 14) displays a great circle and a meridian within the sphere and projection plane. It is intended to illustrate the difference between a great circle and a meridian. In the BASIC version the usual static picture is presented. Considering the ED version, the learner can drag the meridian and simulate all possible meridians or drag the great circle and rotate it along the z-axis simulating the possible great circles. The WD version only allows the learner to rotate the whole diagram while with the WD+ED version the meridian can be dragged along the y-axis simulating all possible meridians and the great circle can be rotated in all directions simulating all possible great circles.

Figure 14 - Diagram displaying a great circle and a meridian Legend - N - north pole; S - south pole; r - equatorial plane; e - equatorial line; O - centre of the sphere; M - meridian; GC - great circle.

69 The second diagram (Figure 15) depicts a meridian and the projection of half of its points. It shows that the projection of a meridian is always a circumference with infinite radius. The ED diagram allows the learners to drag the meridian simulating its rotation along the y-axis6 and showing the corresponding projections, in contrast with the BASIC static diagram. The WD+ED allows the rotation of the meridian similarly to the ED but also gives the possibility of rotating the whole diagram while the WD diagram only permits the latter.

Figure 15 - Diagram showing the projection of a meridian Legend - N - north pole; S - south pole; O - centre of the sphere; r - equatorial plane; e - equatorial line; M - meridian; m - stereographic projection of the meridian's points lying in the north hemisphere when projection is to S.

The third diagram (see Figure 16) shows the projection of a great circle. Nothing is new about the BASIC or WD diagram, its properties and functionalities remain identical to the previous diagrams described. The ED and WD+ED versions allow the learner to drag the great circle rotating it along the z-axis and simulating the corresponding stereographic projection.

6

By definition the meridians are a sub-group of great circles that make a 90 degree angle with the equatorial circle; thus the simulation of all meridians can be done by rotating one meridian along the y axis.

70

Figure 16 - Diagram presenting the projection of a great circle Legend - N - north pole; S - south pole; O - centre of the sphere; r - equatorial plane; e - equatorial line; GC - great circle; gc - stereographic projection of GC when projection is to S.

A - the diagram showing the

B - the great circle is moved to

projection of a great circle

the

right

showing

C - The whole diagram is rotated

the

corresponding projection Figure 17 - Screenshots of diagram displayed in Figure 16 with the exemplification of the interactivity properties available for the WD+ED ILE Legend - see legend to Figure 16.

The fourth diagram (see Figure 18) is similar to the third one also showing the projection of a great circle. However, the projection uses the two poles for the projection thus confining the projected points to the equatorial circle.

71

Figure 18 - Diagram showing the projection of a great circle using points N and S as projection points Legend - N - north pole; S- south pole; O - centre of the sphere; r - equatorial plane; e - equatorial line; GC - great circle; gc - stereographic projection of GC when projection is to S or N for points lying in the north or south hemispheres, respectively.

5.3.4 Step four The major goal of step four is to explain the process of projecting a small circle and note its importance for the building of the stereographic projection of the faces of a crystal. The explanation begins by referring to the concept of the projection of a small circle, following the importance of this notion to the projection of the crystals' faces. After this introduction, the usual table with the major definitions, elements and examples is displayed. In this step there are no additional comments. The "new" definitions are: (a) the definition of a circle in the sphere surface centred in a point in the sphere surface, (b) the definition of small circle, and (c) the definition of parallel. The step also makes use of the crucial definitions of the equatorial plane and stereographic projection. The most important elements to be identified, taking into account the existence of three different diagrams, are: in diagram 1 - parallel, small circle, angle U and V; in diagram 2 - the line of projection, the specific points on the circle to be projected (parallel) and the corresponding projection of the points; in diagram 3 - the line of projection, the specific points on the circle to be projected (small circle), centre of the small circle, projection of the small circle, centre of the projection

72 of the small circle. In terms of the relationships between elements: the initial basic issue relates to the understanding of the difference between a parallel and a small circle and is exemplified in the diagram 1. Another issue regards the relationship between a parallel and its projection, noting that the point corresponding to the centre of the projected points is also the projected centre of the parallel (covered by diagram 2). An additional point to consider involves the need to relate the small circle with its projection and acknowledge that the point lying in the centre of the projected points is different from the projected centre of the small circle (diagram 3). The learner should acknowledge the differences concerning the projections of a parallel and a small circle. The first diagram of step four displays a small circle and a parallel (see Figure 19). No interactive elements are available. Hence, the differences between all the versions have to do with the possibility to rotate the whole diagram.

Figure 19 - Diagram displaying a small circle and a parallel Legend - N - north pole and centre of PR; S - south pole; O - centre of the sphere; r - equatorial plane; e - equatorial line; PR - parallel; SC - small circle; A - centre of SC; u - angular distance between any spherical pole in PR and N; v - angular diatance between any spherical pole in SC and A.

The second diagram concerns the projection of a parallel (see Figure 20). The BASIC and WD versions maintain its general properties. The ED and WD+ED versions allow the learner to drag the projection line along the z-axis by 180 degrees simulating all possible parallels and its stereographic projections.

73

Figure 20 - Diagram showing the projection of a parallel Legend - N - north pole; S - south pole; O - centre of the sphere; r - equatorial plane; e - equatorial line; PR - parallel; pr - steregraphic projection of PR when projection is to S.

The third diagram depicts the projection of a small circle (see Figure 21 and Figure 22). Once more there is nothing new to comment on in the BASIC and WD versions. The ED and WD+ED provide the functionality of dragging a small circle and rotating it along the z-axis and simulating the corresponding projections.

Figure 21 - Diagram depicting the projection of a small circle Legend - N - north pole; S - south pole; O - centre of the sphere; r - equatorial plane; e - equatorial line; SC - small circle; sc - steregraphic projection of SC when projection is to S; A - centre of SC; a - stereographic projection of A when projection is to S; b - centre of sc.

74

A - the diagram showing the

B

-

the

small

diagram

is

projection of a small circle

manipulated to the right showing

C - the whole diagram is rotate

the corresponding projection Figure 22 - Screenshots of diagram displayed in Figure 21 with the exemplification of the interactivity properties available for the WD+ED ILE Legend - see legend to Figure 21.

5.3.5 Step five This step introduces the concepts underlying the construction and use of stereographic nets and in this particular case the Wullf Net. The whole explanation starts by referring to the use of the stereographic nets and the concepts behind its construction. The table exemplifies the construction of the Wullf Net, identifying its main elements, procedures and the representation itself by clicking on a hyperlink. In relation to new definitions to be understood the only one in this step is the definition of Wullf Net. The most important elements to be identified are: the meridians and parallels to be projected and their corresponding projections, the points of projection, and the line of projection. The main relationship between elements to be considered in this step is the mapping connecting the parallels and meridians and its projections and also the reference points drawn in the net. This mapping involves understanding the first diagram where the building of the projection is demonstrated and the outcome, the Wullf Net itself. Hence the step has two diagrams the one demonstrating the building of the projection and another concerning the Wullf Net. The first diagram of step five shows a rough representation of a Wullf Net embedded in a sphere and gives the possibility to simulate the stereographic projection of points (see Figure 23 and Figure 24). In the interactive versions, the ED and WD+ED, the learner is able to drag a line and simulate the projection of points, taking

75 into account the location of the points to be projected: north hemisphere points are projected using the south pole and vice-versa. In fact the functionality is quite similar to the one provided in the diagram of step one. The added issues are the insertion of the Wullf Net and the projections using both poles.

Figure 23 - Diagram showing the projection of points using two different points of projection and a representation of a stereographic net Legend - N*, S*, E* and W* - directions in the Wulff Net, represent north, south, east and west, respectively; N and S - points where the spherical poles are to be projected; O - centre of the sphere; r - equatorial plane; e or N*S*E*W* - equatorial line and its stereographic projection; A sperical pole to be projected; a - stereographic projection of A when projection is to S; b stereographic projection of A when projection is to N.

A

-

Diagram

projection

of

showing points

and

the a

representation of a stereographic

B - The projection line is

C- The whole diagram is

manipulated

manipulated

simulating

the

projection of alternative points

net Figure 24 - Screenshots of diagram displayed in Figure 23 with the exemplification of the interactivity properties available for the WD+ED ILE Legend - see legend to Figure 23.

76

The second diagram just displays the Wullf Net, which is by definition a 2D representation, see Figure 25.

Figure 25 - The Wullf Net

5.3.6 Step six The main goal of step six is to demonstrate how to project all the points that are at a fixed angular distance from a given point. Such a demonstration is intended to promote reasoning about how to work with angular distances and stereographic nets. In this step there are no "new" definitions to be acquired and only one diagram is available. In fact, it is a kind of corollary of all the previous steps. Considering the key elements, the learner has to identify three different points in the surface of the sphere and its corresponding projections in the projection plane as well as the small circle to be projected and their actual projection. In terms of the relationships between elements, the learner has to attend to several the points to be projected and their respective projections, as well as the relationship between the surface of the sphere, parallels and meridians and the Wullf Net. The diagram available in step six does not have any interactive elements (see Figure 26 and Figure 27). Hence, similarly to other cases already described the difference between the versions concerns the possibility or not to manipulate the whole diagram.

77

Figure 26 - Diagram showing the projection of all points at a fixed angular distance from point A Legend - N*, S*, E* and W* - directions in the Wulff Net, represent north, south, east and west, respectively; N and S - points where the spherical poles are to be projected; O - centre of the sphere; r - equatorial plane; e - equatorial line; A - sperical pole to be projected; a - stereographic projection of A; N*AS* - meridian containing A; N*aS*- stereographic projection of N*AS*; C circle centred in A; c - stereographic projection of c; u - angular distance between any point in C and A; .X and Y - two sperical poles in C; x and y - stereographic projection of X and Y; b - middle point of xy.

A - Diagram showing the projection of all points at

B - The whole diagram rotated to a new position

a fixed angular distance from certain point Figure 27 - Screenshots of diagram displayed in Figure 26 with the exemplification of the interactivity properties available for the WD and WD+ED ILEs Legend - see legend to Figure 26.

78

5.4 Sketching some normative models of learner' use of the interactive learning environments This section deals with the expectations and hypotheses about learners' activity when interacting with the different ILEs developed. It has been divided into two distinct parts: before and after reaching the questions set in the ILE. The aim is to come up with a series of expected behaviour related to the available interactivity of the ILEs and their explanatory steps. In turn, this analysis will help to frame the investigation of the learners' activity patterns.

5.4.1 Before reaching the questions' part in the interactive learning environment As regards to the learners' exploration of the content before reaching the questions' part in the ILE, a general strategy can be considered that applies to all the designed ILEs in a similar way. It was decided to reflect upon a generic four-step model: 1. Start by reading the initial considerations regarding the concept. 2. Progress to the identification of the key elements and the diagrams provided.

This

task

should

involve

some

switching

between

representations - text and diagrams. 3. Consider the step-by-step examples of how to build the projections, simulating alternative points, or inspect the demonstrations of particular concepts. 4. Read the final comments and disambiguate the meaning, if necessary, by re-reading parts of the text or examining the diagram. However, step two of the explanation (see section 5.3.2) requires an extra comment regarding the display of some stereograms. In this case, considering the need to understand the mapping between the hypothetical crystals displayed and the stereograms, it seems quite useful to switch between the diagrams displaying the crystals and the stereograms. Attention should be paid not only to how one

79 representation is converted to the other but also to identifying the differences between the available representations and the represented objects.

5.4.2 After reaching the questions' part in the interactive learning environment This sub-section presents a summarised description of the questions in each step. Note that the questions can be divided into three different groups according to where the relevant information to answer can be found, which in some sense corresponds to the learners' expected activity during the interaction with the ILEs. The questions' types are: •

Questions type T - where the relevant information is in the textual part.

•

Questions type D - where the relevant information is in the diagrams.

•

Questions type TD - where the information can be found in either text or diagrams.

One of the collaborators who worked with the author on the content of the ILEs as well as the author himself classified the multiple-choice test (MCT) questions into the three types described above. After, the individual classifications were compared and the discrepancies were solved by mutual agreement. The above categories of question will be used to classify the questions present in each step in the next sub-sections and the classification will be utilised in the qualitative analysis of the major experiment run. The three noted characteristics allow to reflect on two different questions when focusing on the qualitative analysis: •

Did the learners choose the correct representation to look for information? This question regards efficiency of choice and can be answered through the analysis of the learners' choices in questions of type T and D (above).

•

What was their preference in questions where the information could be equally found in text or diagrams? This question focuses on learners' preferences and can be resolved by analysing the choice of type TD questions.

For the following analysis it is assumed that search in the ER demands less effort than retrieving the information from memory. Hence the learners will tend to double-

80 check their answers to the questions by re-inspecting the available diagrams or text. Table 2 presents the different questions posed. The rationale for the decision regarding the classification of each question and questions' parts can be found in Appendix II.

Steps 1

Questions and their covered topics

Type T

Question 1 (1 part): Definition of stereographic

X

Type D

Type TD

a, b, c, d

e, f, g

projection. Question 2 (7 parts): Simulation of the projection of some points. Question 3 (5 parts): Simulation of the projection

a, b, c, d, e

of some points. 2

Question 1 (1 part): Definition of spherical

X

projection. Question 2 (4 parts): Identification of points'

a, b, c, d

location on the surface of the sphere when projecting faces of a crystal. Question 3 (4 parts): Identification of the

a, b, c, d

stereographic projection of some faces of a crystal. 3

Question 1 (3 parts): Identification of the shape of

a

b, c

great circles' projections. Question 2 (1 part): Comparison of the radius size

X

of the stereographic projection of a meridian and a great circle. 4

Question 1 (1 part): Identification of the shape of

X

the projection of a small circle. Question 2 (1 part): Definition of small circle. Question 3 (2 parts): Identification of the centre of the projections of a parallel and a small circle.

X a, b

81 Steps 5

Questions and their covered topics

Type T

Question 1 (1 part): Identification of what is being

X

Type D

Type TD

represented in a Wullf Net. Question 2 (1 part): Understanding of the

X

usefulness of the Wullf Net. 6

Question 1 (1 part): Identification of the shape of

X

the figure in the Wullf Net that corresponds to the group of points that distance themselves 30 degrees from a given point. Question 2 (1 part): Understand the process of

X

constructing the figure referred to in the previous question. Table 2 - The questions of each explanatory step with a summary of the topic covered and the type (T, D or TD) of each part

Please note - whenever the question just has one part an X will be placed in the corresponding type, for the cases of questions with several parts their numbering will be inserted in the respective box.

82

6 The hypotheses, the experimental design and the procedure This chapter presents the hypotheses, the variables, the sample used, the design of the experiment and the procedure followed on the main experiment. Section 6.1 covers the main hypotheses of the study. Section 6.2 concerns the description of the sample used for the experiment. Section 6.3 describes the tests used7. Section 6.4 concerns the procedure followed.

6.1. The hypotheses The hypotheses are concerned with the putative benefits of the more interactive diagrams and with the assessment of the effects of individual differences on performance. Cheng (1999a) points out some possible benefits of adding interactivity to diagrammatic representations. However, Rogers (1999) highlights the need for further research in order to understand how can interactive graphical representations facilitate problem solving and, in doing so, what are the benefits of making the task easier for learning. The problem seems to be related to a lack of clear indication on how to balance the offloading between the computer and the learner. Summarising, in the literature review one cannot find definite answers regarding the conditions under which adding interactivity to graphical representations for learning is clearly beneficial. This thesis and its hypotheses seek to address the problem. Considering the panorama, probably the more cautious approach would be to formulate un-directional hypotheses concerning learners' performance using the distinct ILEs. However, hypotheses 1 and 3 stated below assume a positive directional approach towards the benefits of the more interactive versions by expecting that learners using the WD+ED ILE will show better scores on the performance tests. Hypotheses 2 and 4 add another issue to the previous expectation by considering that these benefits should be even more relevant for low ability learners. Thus hypotheses 2 and 4 intend to account the impact that learners' prior knowledge and cognitive abilities might have on way they 7

Please note that the multiple-choice test (MCT) was been already covered in detail in the previous chapter.

83 interact with ERs and solve problems (for example, Cheng et al., 2001; Cox, 1999; Scaife & Rogers, 1996). The reasons considered for optimistic stance regarding the benefits of the more interactive diagrams on learning were: •

Considering the built versions of the ILEs and their functionality it was thought that the interactivity added to the diagrams could offload the cognitive effort and free resources for the understanding. More specifically, being able to rotate the whole diagram could aid the recognition of the diagrams structural properties while manipulating the diagrams' elements would disambiguate the relationships between them. The offloading provided by these two functionalities, given the topic to be learned and its characteristics, would be crucial to the learning and overcome possible problems related to learners getting distracted with the interactive properties and/or not pay due attention to the textual part of the ILE.

•

The beneficial effect of the added interactivity should be particularly important for less able learners since they would be the ones with probable overstretched cognitive load. Hence, diminishing the cognitive load associated with the two functionalities provided would free critical resources for the learning. It should be noted that it was thought that the two critical resources that would influence performance on this particular problem would be geometry knowledge and spatial ability (following, for example, Kirby & Boulter, 1999).

•

The pilot study findings decisively informed the design of the built ILEs and enable avoiding design pitfalls that could hinder the benefits of the interactivity effect. The following topics emphasize the lessons learned and implemented issues: (a) the organisation of the learning units and related navigational issues; (b) the design of the diagrams' rotation functionality; (c) the design of cues to highlight the diagrams' interactive elements; and (d) the provision of reference elements in the text that pointed to the diagrams and the implicit suggestions of learning activities in order to promote the understanding of both text and diagrams.

84 Furthermore, the participants of the pilot study appeared to be enthusiastic with the technology. The main hypotheses can be stated as follows: •

Hypothesis 1 - The more interactive ILEs provide better immediate learning results than the less interactive ILEs. By immediate learning results is meant the results from answering the questions in the MCT.

•

Hypothesis 2 - The participants with less knowledge of geometry and lower spatial skills will benefit more from the use of the WD+ED ILE, achieving better scores in the MCT than those using the other three systems.

•

Hypothesis 3 - The participants who use the more interactive ILEs will have better scores in a delayed post-test (PT).

•

Hypothesis 4 - The participants with less knowledge of geometry and lower spatial skills will benefit more from the use of the WD+ED ILE, achieving better scores in the post-test (PT) than those using the other three systems.

6.2 The participants The participants were drawn from two different groups of students, namely geology and mathematics undergraduates. The different groups of students, maths and geology students, were included to explore the possibility that coming from a different course might reveal different interaction patterns and/or responses to the MCT and PT, which in turn could lead to exposing effects of prior knowledge, in particular, related to domain knowledge (see sub-section 5.1.3). For example, the mathematics students might show better performance scores on the GT as an indication of their better grasp of geometry. This higher level of general geometry knowledge, assessed by the GT, might influence their performance on the MCT and PT. However, it can also be the case that other variables not controlled by the GT or spatial ability test and related to each group specific background knowledge characteristics might have an impact on the way learners interact with the distinct ILEs and answered the MCT and PT. Furthermore, a

85 complex interaction between background knowledge, levels of geometry knowledge, levels of spatial ability and type of ILEs, might show up. In this study, eighty participants, 42 first year undergraduate geology students and 38 first year undergraduate mathematics students started the experiment and were randomly assigned to one of the 4 ILEs. Table 3 displays the frequencies of the total number of participants that started the experiment in each comparison group, showing that subjects were roughly evenly distributed. Degree Geology

Maths

Type of ILE BASIC ED WD WD+ED Total BASIC ED WD WD+ED Total Overall Total

Frequency 11 11 10 10 42 8 10 10 10 38 80

Table 3 - Distribution of participants by type of ILE and degree

Table 4 displays the number of subjects that did the MCT and also completed the PT.

86

Degree

Type of ILE

Geology

BASIC ED WD WD+ED

Maths

BASIC ED WD WD+ED

Frequency Valid Missing Valid Missing Valid Missing Valid Missing

11 0 11 0 8 2 8 2

Valid Missing Valid Missing Valid Missing Valid Missing

7 1 6 4 4 6 5 5

Table 4 - Distribution of participants that completed the PT by type of ILE and degree. The missing cases are the ones who did the MCT but not the PT.

Table 4 shows that some participants did not complete the experiment by missing the PT. The missing observations are especially concentrated among maths students in the ED, WD and WD+ED conditions.

6.3 The apparatus The main materials used in this experiment consisted of four different ILEs, which have already been described in detail (see chapter five), and four different tests: •

The Paper Folding Test (PFT) consists of twenty items, divided into two trials of ten (see Appendix VI). In each item, the participant sees a sequence of five drawings that depict a sheet of paper being folded and punched. The task is to choose from five alternatives the corresponding unfolded sheet. The scores for the PFT were obtained by: adding up the correct answers and subtracting one fourth of the wrong answers (for a

87 complete description of the test and its psychometric properties see Ekstrom, French and Harman, 1979). •

The geometry test (GT) is an adaptation of a sample of the GRE geometry sub-tests (Stewart & O'Toole, 1999). The test consists of 10 multiplechoice questions to be answered in 20 minutes (see Appendix V). It was marked in a similar way to the PFT. The subtracting term, however, depends on the number of parts present in the question, which is not constant.

•

The MCT consists of six parts, each included at the end of the corresponding step in the explanation of the stereographic projection as presented in the ILEs developed. The test has fifteen questions, some of them with more than just one item (see 5.5.2 for a more complete description of the test and Appendix III). The marks were obtained by adding up the number of correct answers. Each question was worth one point and each item of the question was worth one divided by the number of parts of the corresponding question. The score of the question was the sum of the parts.

•

The PT consists of 16 true/false questions about the geographic projection. The test does not replicate the MCT since the questions and the questions' format are different. Nevertheless, the PT questions, similarly to the MCT, cover issues related to concept definitions and the simulation of points' projections (see Appendix IV). The PT was administered, on average, 10 days after the interaction took place. It should be noted, however, that due to participants' time availability the lag between interacting with the ILEs and answering the PT is not equal for all the participants. The marks for the test correspond to the sum of points obtained over all the questions, each valued one point if answered correctly and minus one point if answered incorrectly.

88

6.4 The procedure The experiment has a between-subjects design, where each participant was allocated to one of the ILEs. In terms of the experimental groups formed, there were two distinct groups of students (maths or geology students) times four different ILEs that equals eight groups in total (2 X 4 = 8). The experimental design involved three different stages. First (stage 1), participants were tested on their spatial ability and knowledge of geometry. The spatial ability was tested through the PFT and the geometry knowledge was tested using an analogue to the GRE geometry test module, the GT. The procedure for the application of the PFT is standardised (Ekstrom et al., 1979) and was followed accordingly. In relation to the procedure followed for the application of the GT, the instructions for answering the questions were read including: time available to answer, the particular issues regarding the figures and diagrams presented and the questions' formats. No practice questions were available. Second (stage 2), the participants were randomly assigned to one of the different ILEs. The sessions were run in a computers' room and usually involved four participants each on his/her respective machine. The participants were told that the session would involve the exploration of the ILEs in order to investigate the geometry concepts presented and answer a few questions displayed at the end of a text frame on a separate sheet - the MCT. At start, the experimenter pointed to the participants the initial screen of the ILEs where the general instructions for the session could be found. Note that the instructions were differentiated depending on the type of ILE since they included considerations regarding the diagrams interactivity. The experimenter would then read the instructions for the session. The instructions included reference to: •

Time available - no constraints were imposed on the time available.

•

Order/sequencing of the steps' exploration - no order or sequencing of the exploration was forced, although there was an implicit sequence given by the numbered hyperlinks connecting each step.

89 •

Interactivity of the diagrams - the interactivity available was explained, including the cues that would enable the participants to identify interactive objects, and briefly demonstrated by showing the example of step one. Only a small practice period with the diagram of step one was allowed (approximately 2 minutes). No free exploration session was provided.

•

Questions' format - the questions' format was explained.

•

Asking questions during the session - the participants were told that they could ask questions related to general issues of the interaction whenever they thought needed. However, they were instructed not to exchange ideas between themselves.

Participants' interaction with the ILEs was video recorded. The video recording of learners' activity was done using a video camera with open microphone aimed at the computer screen without catching the learners' "outside system activity". However, although the video camera had the microphone open and thus was able to capture sound, no particular episodes were registered. One possible explanation for the fact is that maybe subjects assumed the sessions to be of a individual examination type maintaining silence and avoided self comments. This procedure is different from the one utilised for the pilot study insofar has it did not use a graphics board video-out facility to record the on-screen activity directly to a video recorder. The reason for this difference in procedure concerning the video recording regards the constraints due to sample size and the accessible resources. In order to be able to use a larger sample than the one pursued in the pilot study, it was thought that at least a four computers session would be appropriate. Fewer computers would have meant an extended period for the experiment and that would have clashed with participants' availability due to course examinations. In relation to video recording using a video recorder, the computers available in the computers' room did not have a video-out interface so this source could not be utilised. Furthermore, a log file of learners' activity could not be generated also due to specificities of the computers' network. Nevertheless, the time that participants took to explore the ILEs and answer the questions were recorded by the experimenter in a sheet

90 of paper. This information was taken into account when modelling the interaction with the ILEs in the quantitative analysis. Finally (stage 3), the participants were tested on the knowledge acquired in stage 2 through a questionnaire containing 16 sentences to be answered true or false - the PT. The sessions for the PT were run in groups and the dates for the testing were subject to participants' availability. Due to this fact there were different time lags between stages 2 and 3 for different participants. The average interval between the two stages measured 10 days. The analysis of the results obtained at this stage will take the time lag between the two stages into account by conditioning on it.

91

7 Background knowledge, spatial ability and interactive properties: a quantitative analysis Section 7.1 analyses the spatial visualisation and geometry knowledge variables. Section 7.2 focuses on the testing of the hypotheses concerning the multiple-choice test (MCT) and section 7.3 on the hypotheses regarding the post-test (PT). Finally, in section 7.4, some concluding remarks are drawn.

7.1 The distribution of spatial ability and geometry knowledge The analysis of the spatial ability and geometry knowledge variables investigated the comparability of the different groups. Table 5 presents some descriptive statistics. Table 5 shows that: •

In relation to the PFT, individual scores vary between 3.75 and 18.75 in a total maximum possible of 20 points. The geology students in the BASIC ILE condition achieved the highest mean score of all the groups tested and the maths students in the WD+ED ILE condition achieved the lowest. In general, there is no noticeable specific trend concerning the comparisons between geology and maths students in the spatial ability test.

•

Individual scores in the GT vary between 0 and 7 in a total maximum possible of 10 points. The geology students in the ED ILE condition achieved the lowest average score and the maths students in the WD ILE condition achieved the highest average score. In general, the maths students scored better in the GT than the geology students.

92 ILE

Degree

Tests

N

Minimum

Maximum

Mean

Std. Deviation

BASIC

Geology

GT PFT GT PFT

10 11 8 8

2.00 7.75 2.00 5.25

7.00 17.50 8.00 16.70

3.80 12.82 4.25 10.22

1.62 3.60 2.05 3.92

GT PFT GT PFT

11 11 10 10

1.00 6.50 2.00 6.00

6.00 18.75 6.00 16.25

2.36 11.61 4.40 12.03

1.43 3.86 1.35 3.25

GT PFT GT PFT

10 10 9 10

.00 5.50 3.00 5.50

7.00 15.75 7.00 14.25

2.90 10.45 5.22 10.28

2.02 3.42 1.09 2.86

GT PFT GT PFT

9 10 10 10

1.00 5.75 1.00 3.75

5.00 15.50 6.00 15.75

3.11 10.50 3.40 9.10

1.167 3.11 1.65 3.84

Maths ED

Geology Maths

WD

Geology Maths

WD+ED

Geology Maths

Table 5 - Descriptive statistics regarding the PFT and the GT

In regard to the normality of the two variables, the tests performed showed (see Table 56, in Appendix VII -A): •

Scores in the GT do not follow a normal distribution for any of the considered groups.

•

Scores in the PFT comply with a normal distribution when students were classified by degree but not when taken all together.

It is surprising that the PFT scores exhibit different distributions by degree. Unfortunately no plausible explanation for the fact can be drawn. It should be pointed out, though, that these results do not have any implications in the general models to be considered. The similarity between groups by degree and ILE in what regards to the GT and the PFT scores was tested. The normality tests' results discussed above led the choice of non-parametric tests. On the one hand, no significant differences among the groups concerning the PFT were found, suggesting some homogeneity regarding the distribution of this variable by type of ILE and degree (see Table 57 and Table 58 in Appendix VII - A). On the other hand, the GT scores seem to be differently distributed

93 by degree, with the maths students showing better scores (see Table 57 to Table 60 in Appendix VII - A). This finding is not surprising since it seems reasonable to assume that the maths students have a deeper knowledge of geometry. In summary, the spatial ability as measured by the PFT seems to be distributed evenly across the comparison groups while the geometry knowledge as measured by the GT seems to be higher among maths students. Considering that the concept to be taught, the stereographic projection, covers the same general domain as the GT, it was useful to investigate the relationship between the PFT and the GT scores. The goal is to investigate whether answering the multiplechoice questionnaire and the post-test, on the one hand, and the GT on the other, require common abilities. The actual comparison between these tests is delayed for later on, the focus now brings on the relationship between the GT and the PFT. The correlation between the two variables is significant for the geology students (r = .429; n = 42; sig. = .005) but not for the maths students (r = .220; n = 38; sig. = .184). A significant correlation between the PFT scores and the GT scores when taking all the participants together was also found (r = .264; N = 80; sig. = .019). Taking into account the correlations by degree, it seems that the relationship between the two variables observed for all participants should come from the geology students' sub-sample. The estimated relationship between the PFT and the GT scores indicates that spatial ability may play a more important role in solving geometry problems for the geology students than for maths students. This result is surprising since other studies have found a significant and positive relationship between levels of spatial ability and performance in geometry tests (Hegarty & Kozhevnikov, 1999b; Kirby & Boulter, 1999). However, it is conceivable that maths students possess some additional expertise in the domain that enables them to solve the geometry problems without making so much use of spatial ability. The question to be answered now is: •

Will this pattern of dependency be repeated with the ILEs and corresponding tests (MCT and PT)? The confirmation of existing different relationships between the PFT and the MCT or PT for the distinct groups has important consequences on the choice of the statistical procedures to test Hypothesis 1.

94

7.2 The impact of background knowledge, spatial ability and interactivity properties on immediate learning The data about immediate learning concerns stage 2 or when participants had to solve a test while interacting with the ILE - the MCT. ILE BASIC ED WD WD+ED Total

Geology 11 11 10 10 42

Maths 8 10 10 10 38

Total 19 21 20 20 80

Table 6 - Number of observations by group

Mean 5% Trimmed Mean Median Std. Deviation Minimum Maximum

Statistic 8.86 8.92 8.63 2.37 3.15 13.36

Table 7 - Descriptive statistics for the MCT

The size of the sample by degree and ILE is displayed in Table 6. Table 7 shows that scores in the MCT range between 3 and 13 in a total maximum possible of 16 points, with a mean of 8.86. The distribution appears to be quite symmetric since the mean, the 5% trimmed mean and the median are very close together. In relation to the normality of the MCT variable, the test performed reveals that it cannot be rejected, supporting the use of parametric tests (see Table 61 in Appendix VII - B).

7.2.1 Testing Hypothesis 1: The impact of the ILEs interactivity properties Hypothesis 1 - The more interactive ILEs provide better immediate learning results than the less interactive ILEs. By immediate learning results is meant the results from answering the questions in the MCT.

95 The assessment of differences between comparison groups to test Hypothesis 1 involved a (2 X 2 X 2) full factorial ANOVA model comparing the groups by degree, ED, WD and the corresponding combinations of interactions between the main factors. This analysis is followed by an investigation about the possibility of using a factorial ANCOVA including the PFT scores as co-variate. It should be pointed out that alternative models including the GT as a second co-variate were also investigated. However, the GT did not reveal a significant effect upon the general scores of MCT and also the PT, contrasting with Kirby's and Boulter (1999) results. Nevertheless, the analysis of the GT was useful as it showed some differences related to background knowledge of the geology and maths students. The aim is to see whether such model reduces the noise due to individual differences and, thus, increases the accuracy of the testing. In relation to the initial (2 X 2 X 2) full factorial ANOVA, no significant differences were found regarding the main effects (ED - (F (1, 72) = 1.44; sig. = .24); WD - (F (1, 72) = 1.48; sig. = .23; Degree - (F (1, 72) = .179; sig. = .67) or interaction effects between them (ED X WD - (F (1, 72) = .33; sig. = .57; ED X Degree - (F (1, 72) = .52; sig. = .47; WD X Degree - (F (1, 72) = 1.25; sig. = .27; ED X WD X Degree - (F (1, 72) = 2.51; sig. = .12). These results suggest that the two types of interactivity provided have no special impact on the learning. Furthermore, the two groups of learners, maths and geology students, do not differentiate from each other neither on general performance nor when using any of the specific types of interactivity. Source

Type IV df Mean Sum of Square Squares Corrected Model 44.877 7 6.411 Intercept 6182.89 1 6182.89 ED 7.99 1 7.99 WD 8.26 1 8.26 Degree .99 1 .99 ED*WD 1.83 1 1.83 ED*Degree 2.89 1 2.89 WD*Degree 6.98 1 6.98 ED*WD*Degree 13.94 1 13.94 Error 400.71 72 5.57 Total 6725.80 80 Corrected Total 445.58 79 R Squared = .101 (Adjusted R Squared = .013)

F

Sig.

1.152 1110.96 1.44 1.48 .18 .33 .52 1.25 2.51

.341 .00 .24 .23 .67 .57 .47 .27 .12

Table 8 - Tests of between subject effects for the ANOVA model. Dependent variable: MCT

96 As indicated above, the next stage of testing Hypothesis 1 used an ANCOVA model to check whether additional co-variates reduce noise and allow for potential differences between groups to show up. Previous analysis showed that the GT scores were not good predictors of the MCT scores. Contrary to the findings regarding the GT, however, the PFT scores showed to be good predictors of the outcome. Thus the ANCOVA models to be tested include this variable. Before presenting ANCOVA models, though, its assumptions need to checked. In fact, one of the central assumptions of such model is the homogeneity of the regression slopes. This assumption means that all comparison groups are similarly affected by the co-variate, no interaction between the co-variate and the independent variables being allowed (Howell, 1997; Tabachnick & Fidell, 2001). Hence, to evaluate the appropriateness of an ANCOVA model the relationship between the PFT and the MCT by group is investigated.

Degree: Geology 14

12

ILEs

10

WD+ED Rsq = 0.2107

Multiple-choice test

8

WD Rsq = 0.1735

6

ED Rsq = 0.4612

4

BASIC 2

Rsq = 0.4382 4

6

8

10

12

14

16

18

20

Spatial ability

Figure 28 - The relationship between the PFT and the MCT scores for the geology students by type of ILE with fitted regression lines and corresponding R2

97

Degree: Maths 14

12

ILEs 10

WD+ED

Multiple-choice test

Rsq = 0.0044 WD

8

Rsq = 0.1073 ED

6

Rsq = 0.0679 BASIC

4

Rsq = 0.0015 2

4

6

8

10

12

14

16

18

Spatial ability

Figure 29 - The relationship between the PFT and the MCT scores for the maths students by type of ILE with fitted regression lines and corresponding R2

Figure 28 and Figure 29 reveal that the relationship between the PFT scores and the MCT is not the same for all the groups. A more structured analysis of the relationship between the PFT and the MCT scores was pursued using an ANCOVA model allowing for different effects of the PFT by group. Table 9 displays the results suggesting the existence of different impacts of the PFT on the MCT scores in what concerns to the slope of the straight lines, which approach significance at the 5% level. Source

Type IV df Sum of Squares Corrected Model 123.827 15 Intercept 235.407 1 Degree * ED * WD 17.944 7 Degree * ED * WD * PFT 78.950 8 Error 321.755 64 Total 6725.803 80 Corrected Total 445.582 79 R Squared = .278 (Adjusted R Squared = .109)

Mean Square

F

Sig.

8.255 235.407 2.563 9.869 5.027

1.642 46.824 .510 1.963

.087 .000 .824 .066

Table 9 - Tests of between subject effects for the ANCOVA model with different linear relationships between the MCT and PFT by main factors. Dependent variable: MCT

98 The finding was further investigated through a simple regression analysis by group trying to uncover the best fit for the relationship between the two variables. Both the linear and the quadratic specifications were tried out. It is important to bear in mind that the present results are exploratory given the small number of observations per group and the main goal of the analysis, namely to check the assumption about the homogeneity of the effect of spatial ability as measured by the PFT scores. Table 10 shows that geology students seem to be more dependent on spatial ability to solve the MCT than maths students: among geology students, significance were achieved for the BASIC and ED ILES, and almost significance was achieved for the WD+ED ILEs; among maths students, only the BASIC ILE showed a significant relationship. This finding is similar to what was observed when analysing the relationship between the PFT and the GT. The second comment concerns the comparison between the BASIC and ED and the WD and WD+ED. Table 10 suggests that the systems without WD are more dependent on the subjects' spatial ability than the systems that have this interactivity property: significance was achieved for the BASIC and ED ILEs, except in the case of the maths students interacting with the ED ILE; but no significant relationship was found for the WD and WD+ED ILEs, although geology students interacting with the WD+ED ILE present an almost significant result.

BASIC Linear Regression Anova of the Model

Quadratic Regression Anova of the Model

Geology ED WD

WD+ED BASIC

ED

Maths WD

WD+ED

R Square Adj. R Sq. St Error DF Regre. DF Residual F Sig. F

.43821 .37579 1.84373 1 9 7.2034 .0265

.46124 .40138 1.71565 1 9 7.70517 .0215

.17351 .07020 2.29571 1 8 1.67948 .2311

.21070 .11203 2.53537 1 8 2.13552 .1821

.00148 -.16494 2.69378 1 6 .00888 .9280

.06791 -.04860 2.32554 1 8 .58285 .4671

.10731 -.00428 2.59492 1 8 .96168 .3555

.00441 -.12004 1.94969 1 8 .03545 .8553

R Square Adj. R Sq. St. Error DF Regr. DF Residual F Sig. F

.47949 .34936 1.88236 2 8 3.68474 .0734

.46526 .33158 1.81292 2 8 3.48034 .0818

.18251 -.05105 2.4408 2 7 .78142 .4939

.47441 .32425 2.21176 2 7 3.15922 .1053

.72945 .62123 1.53602 2 5 6.74048 .0381

.13885 -.10719 2.38963 2 7 .56433 .5926

.10731 -.14774 2.77409 2 7 .42074 .6721

.10486 -.15089 1.97636 2 7 .41002 .6786

Legend - The numbers in bold and underlined correspond to the best significant results. The numbers in italic and underlined correspond to important almost significant results in terms of comparisons between groups. Table 10 - Regression analysis testing the adequacy of the linear and quadratic models to express the relationship between PFT scores and MCT scores by groups

99 Summarising the findings so far, it seems that the spatial ability variable is important to explain performance in the MCT. However, a model assuming that spatial ability affects all groups in a similar way does not fit the data. Instead, the relationship between spatial ability and the performance on the MCT needs to be group-specific. Furthermore, a straightforward causal relationship between spatial ability and the MCT is not claimed, and such an issue is beyond the scope of this work. To model an ANCOVA that accounts for these between groups differences in regression lines, a group-specific specification was established. In other words, the model includes 8 different regression lines to account the relationship between the PFT scores and the MCT by group. The main results to report concern the effect of the co-variate spatial ability on the MCT. The effect of the variable PFT is significant (F(2,62) = 3.352; sig. = .003). Among the geology students, the ILE-specific parameter estimates show significant effects of the spatial ability for the BASIC, ED and WD+ED ILEs (a linear positive effect was found for the BASIC and ED ILES and a quadratic concave relationship characterises the WD+ED ILE). For the math students significance is only achieved for the BASIC ILE, where a positive and convex shape seems to characterise the relationship between spatial ability and the performance in the MCT (a detailed view of the statistics is available in Table 11 to Table 12). These results reinforce the exploratory findings referred to above, revealing the different effects of the spatial ability on the performance of the MCT by type of ILE and degree.

100 Source

Type IV Sum df of Squares Corrected Model 172.75 17 Intercept 71.85 1 ED * WD * Degree 66.30 7 ED*WD* Degree * PFT 118.00 8 PFQD2DM 31.74 1 PFQD3DIG 17.18 1 Error 272.83 62 Total 6725.80 80 Corrected Total 445.58 79 R Squared = .388 (Adjusted R Squared = .220)

Mean Square F

Sig.

10.16 71.85 9.47 14.75 31.74 17.18 4.40

.009 .000 .051 .003 .009 .053

2.31 16.33 2.15 3.35 7.21 3.90

Table 11 - Tests of between subject effects for the ANCOVA model with different relationships between the MCT and PFT by type of ILE and Degree. Dependent variable: MCT Note - The almost significant value obtained for the interaction between the type of ILE and degree does not mean a significant difference between the groups. Part of the effect comes from the specification of the ANCOVA model considered especially taking into consideration the inclusion of the quadratic elements for the BASIC and WD+ED (PFQD2DM and PFQD3DIG- variables introduced to model the quadratic relationship between the PFT and the MCT for the maths and geology students using the BASIC ILE and WD+ED ILE).

B Std. Error Parameter Intercept 7.703 1.787 Basic Geology -2.875 3.031 Basic Maths 18.674 7.216 ED Geology -3.445 2.751 ED Maths -.705 3.213 WD Geology -2.175 2.865 WD Maths -1.358 3.151 WD+ED Geology -19.784 8.643 WD+ED Maths 0 . PFT Basic Geology .430 .185 PFT Basic Maths -3.714 1.389 PFT ED Geology .390 .172 PFT ED Maths .182 .215 PFT WD Geology .290 .205 PFT WD Maths .296 .244 PFT WD+ED Geology 3.550 1.611 PFT WD+ED Maths 3.190E-02 .182 PFT Quad. Basic Maths .169 .063 PFT Quad. WD+ED Geo. -.144 .073

t

Sig.

4.311 -.949 2.588 -1.252 -.220 -.759 -.431 -2.289 . 2.329 -2.675 2.270 .846 1.418 1.213 2.203 .175 2.686 -1.976

.000 .347 .012 .215 .827 .451 .668 .025 . .023 .010 .027 .401 .161 .230 .031 .862 .009 .053

95% Confidence Interval Lower Bound Upper Bound 4.131 11.274 -8.933 3.184 4.249 33.099 -8.944 2.054 -7.129 5.718 -7.903 3.553 -7.656 4.940 -37.062 -2.507 . . 6.085E-02 .799 -6.490 -.938 4.657E-02 .733 -.248 .612 -.119 .700 -.192 .784 .329 6.771 -.333 .396 4.311E-02 .294 -.289 1.674E-03

Table 12 - Parameter estimates for the ANCOVA model with different relationships between the MCT and PFT by type of ILE and Degree. Dependent variable: MCT

The following graphs show the relationships estimated (Figure 30 and Figure 31):

101 Geology students: relationship between PFT and the MCT scores 18

multiple choice test score

16 14 12 10 8 6

BASIC ED WD WD+ED

4

19

18

17

16

15

14

13

12

11

10

9

8

7

6

5

2

spatial ability

Figure 30 - Relationship between the PFT and the MCT for the geology students by type of ILE

Maths students: relationship between PFT and the MCT scores 18

multiple choice test score

16 14 12 10 8 6

BASIC ED WD WD+ED

4

19

18

17

16

15

14

13

12

11

10

9

8

7

6

5

2

spatial ability

Figure 31 - Relationship between the PFT and the MCT for the maths students by type of ILE

According to Figure 30 and Figure 31, the WD+ED ILE does not seem to benefit any type of learner, independently of the degree and level of spatial ability as measured

102 by the PFT. In fact, average scores in the MCT by level of spatial ability seem to be always lower among individuals interacting with the WD+ED ILE. Only averagespatial-ability geology students and low-spatial-ability maths students appear not to lose (or gain) from the use of the WD+ED ILE. Nevertheless, and summarising, the fact that learners with different background knowledge and levels of spatial ability might benefit differently from distinct ILEs should be retained.

7.2.2 Testing Hypothesis 2: The impact of geometry knowledge and spatial ability Hypothesis 2 - The participants with less knowledge of geometry and less spatial skills benefit more from the use of the WD+ED ILE, achieving better scores in the MCT than those using the other three systems.

To test hypothesis 2 we began by splitting the sample into high and low scorers in the PFT and the GT. To do so, subjects were ranked according to both scores. The groups were formed based on the percentiles of the PFT and GT empirical distributions with the groups containing approximately the same number of subjects. Participants scoring below the 50th percentile were allocated to a low group in the corresponding tests while participants scoring above the 50th percentile were placed on a high group. With this procedure a group of students that scored badly on both the PFT and the GT was separated from a group of students that scored highly on both tests. Table 13 describes the groups:

ILE

Total

BASIC ED WD WD+ED

GROUPS Low-Low High-High 4 5 6 7 5 7 9 4 24 23

Total 9 13 12 13 47

Legend - Low-Low corresponds to subjects that scored low in both tests and High-High to subjects that scored high in both tests.

Table 13 - Distribution of the participants by groups of scores in the PFT and GT by type of ILE

103 For the present purposes, only the Low-Low and High-High groups were analysed, as the comparison between the two extreme combinations is the most informative about the relative advantages from the use of the interactive properties of the ILEs. Groups characterised by combinations of Low-High were too small and less informative for the purpose of this analysis. This explains their exclusion from the present analysis. Separate ANOVAS were run for the two different groups8. The orthogonal planned contrasts compare: •

The BASIC ILE with the rest of the ILEs (contrast 1).

•

The ED ILE with the WD and WD+ED ILEs (contrast 2).

•

The WD ILE with the WD+ED ILE (contrast 3). Groups Low-Low BASIC ED WD WD+ED Total

N 4 6 5 9 24

Mean 8.22 7.42 7.44 8.12 7.82

Std. Deviation 2.08 2.55 2.38 2.52 2.3

High-High

5 7 7 4 23

11.2 8.95 9.1 6.98 9.14

2.15 1.95 2.8 1.15 2.45

BASIC ED WD WD+ED Total

Table 14 - Descriptive statistics for the MCT, regarding the High-High and Low-Low groups.

The obtained estimates do not validate our Hypothesis 2. Participants in the LowLow group did not seem to benefit more from interacting with the WD+ED ILE given that no significant effect was found between the groups (F(20) = .177, sig. = .911). In fact, there was not one ILE that clearly helps them obtaining better scores in the MCT (the results of the planned contrasts were all not significant, see Figure 32 for a visual display of the group's averages and Table 14 for the actual values). However, the story seems to be different for the subjects in the High-High group. Firstly, the results of the groups comparison almost reached significance (F(19) = 2.760, sig. = .071) Secondly, 8

Doing separate analysis, although not statistically as powerful as just one, does not compromise the specific testing of this hypothesis since it is clearly targeted to the Low-Low group.

104 the contrast 1 was significant (t(19) = -2.546, sig. = .020), indicating that the BASIC ILE was better than the others ILEs taken together. Contrasts 2 and 3 were not significant, raising suspicions about the difference between the BASIC and the WD+ED ILES. The post-hoc tests performed (Gabriel, Hochberg's GT2 tests) provided further indications of this effect by showing a significance estimate at p