PROCEEDINGS OF IADIS WWW/INTERNET 2003 – PP. 326 - 332
Knowledge Acquisition Strategies and Navigation in Hypermedia Learning Environments: The Influence of Instructional Design Properties Mattias Steinke Learning Lab Lower Saxony L3S Expo Plaza 1 30159 Hannover Germany
[email protected]
Thomas Huk Learning Lab Lower Saxony L3S Expo Plaza 1 30159 Hannover Germany
[email protected]
Christian Floto Technical University of Braunschweig Institute of Social Sciences Bienroder Weg 97 38106 Braunschweig Germany c.floto@tu -bs.de
ABSTRACT In order to understand and enhance the value of new media in education it is necessary to develop criteria for the evaluation of the effectiveness of learning with hypermedia environments. From the point of view of a multimedia producer it is an important goal to support the process of learning and maximize the learning outcome by a proper design of the whole system as well as of the content presentation within the system. But how is content presentation linked to navigation and knowledge acquisition strategies when learning with hypermedia devices? The process of learning may be studied investigating navigation strategies and user behaviour when tackling a task. In this context a glossary is considered to be an important tool in the navigational environment to better understand the topics presented in the content modules e.g. animations. Against this background the study presented aims to investigate the use of a glossary in dependence of varying instructional design of animations. 300 students at college level were asked to work with different versions of a hypermedia learning environment on cell biology. While testing students’ log files were tracked and subsequently analyzed applying a variety of statistical methods. The results show that the instructional design of animations indeed exhibit a significant influence on learners’ behaviour, i.e. on the use of the glossary. KEYWORDS Knowledge acquisition, navigation, content design, animations
1. INTRODUCTION Online and offline working hypermedia learning environments increasingly play a role in the area of education. To enhance the value of these new media for studying and learning purposes a growing body of research deals with the influence of learner characteristics, the goal of the learning task, and system design on the navigation performance and knowledge acquisition of the learner (e.g. Barab et al, 1997; Land and
326
PROCEEDINGS OF IADIS WWW/INTERNET 2003 – PP. 326 - 332
Hannafin, 1997). In the past decade, many studies have shown that individual differences exhibit significant effects on student learning with hypermedia systems (Chen and Macredie, 2002). In addition to the respective learner`s abilities and experience the system design is likely to influence the learning performance (e.g. Barab et al, 1997). According to MacGregor (1999) research on the design of hypermedia systems is in its infancy, therefore few design principles have been established. Research focuses on the effect of different navigation tools (e.g. Farrell and Moore, 2000-2001; Chiu and Wang, 2000; Moeller and Mueller-Kalthoff, 2000) and different navigational or content designs (e.g. Stiller, 2001) on learners` achievement and navigation behaviour. Besides navigation tools and the navigational design, the value of a hypermedia learning environment depends on the value of its single components and their instructional, e.g. graphical and didactical presentation. These components might be defined as content modules which represent the nodes within a hypermedia system or learning objects within the internet and often contain a broad bandwidth of media, e. g. text, graphics, animations, simulations, video, narration and sound. It seems advisable for multimedia producers and users to reflect intensively on the instructional design of these modules. However, little empirically based information exists regarding what characteristics of an instructional animation are potentially beneficial to learners and how learners deal with these materials when they encounter them (Lowe, 2001). Empirical research about the impact of the design of pictures and animations on learning often is done in artificial experimental settings with single modules. The project CRIMP (Criteria for the Evaluation of Audiovisuals in Multimedia Productions), presented here, tries to bridge the gap by testing single modules with different instructional design and by embedding these modules into a comprehensive professional learning environment and testing these versions at schools and universities in realistic learning scenarios (Huk et al, 2002). The project is carried out by a working group of the Learning Lab Lower Saxony L3S (see http://www.learninglab.de), together with the Swedish Learning Lab and the Stanford Learning Lab constituting the Wallenberg Global Learning Network (WGLN). CRIMP uses an experimental approach to answer questions on the effectiveness and efficiency of learning with hypermedia devices by testing a CDROM on cell biology with students. The software is commercially available in the languages English and German and presents a considerable amount of content offering very different possibilities for navigation. The CD-ROM “The Cell 2 – the Powerhouse” thereby should enable users to learn and acquire knowledge in an explorative and autonomous way. It was produced by the company IWF Knowledge and Media in Goettingen, Germany. The product is the winner of the “digita 2002”, one of the most important prices for multimedia products in Germany. Furthermore, a website on cell biology is offered to the users (see http://www.cells.de) to provide them with ideas and multimedia material. In general online users are more or less anonymous and it is difficult to check user data provided via internet. Therefore it seems necessary to investigate the relationship between software design and learners` needs under controlled conditions . Offline media can easily be transformed for online use by Shockwave technology. Often online and offline media are developed in parallel as it was done in a project on cell biology of the IWF. Accordingly, the results gained by testing an offline learning environment such as a complex CD-ROM can be used for the development of online content modules or learning objects for different databases in the web. This paper will focus on the following question: Does the instructional and graphical design of content modules or learning objects exhibit an impact on the whole process of knowledge acquisition and navigation of the user within a hypermedia learning environment? When watching an animation or a movie on a CDROM or other hypermedia devices an important part in the process of gaining knowledge and understanding is the navigation and search for related material or further explanations to better understand the subject. In this process the glossary can be considered a valuable part of the navigational environment offered in many hypermedia systems. It may be described as an external navigation tool that enables fast access to explanations of difficult terms found in the content modules or learning objects within the hypermedia environment. By this the process of learning or handling a specific task is supported and the learner can better understand and consolidate the content. From the point of a multimedia producer it is an important goal to sustain the process of learning and maximize the learning outcome by a proper design of the system as well as the content presentation within the system. In the study presented the use of the glossary in a hypermedia environment and its dependence on instructional and graphical design shall serve as an example to have a first insight into the connection of content design and learning process. Two hypotheses on the impact of the varying didactical and graphical design of an animation on the use of the glossary of the CD are tested in the present paper:
327
PROCEEDINGS OF IADIS WWW/INTERNET 2003 – PP. 326 - 332
1. The presence of visual signals leads to an increase in the use of the glossary. 2. A three- dimensional graphical design leads to an increase in the use of the glossary
2. MATERIAL AND METHODS To investigate the question raised in the introduction four evaluation variants of the CD-ROM on cell biology offering different didactical and graphical designs but the same content were produced. One part of the CD, an animation that deals with the enzyme ATP synthase was modified in a 2x2 factorial design pattern: -
One factor represents the degree of reality (two-dimensional vs. three-dimensional) The other factor represents the presence or absence of visual signals.
300 students of 19 school classes from different parts of Germany participated in the study. Students were introduced to work for 20 minutes with the hypermedia software on a certain task related to the modified animation on the biomolecule ATP synthase. Each student used one of the four variants of the software. The individual’s navigation processes got documented by log files. Number of clicks on the glossary served as an indicator for the usage of the glossary. For further comparison of the intensity of usage of the glossary, only log files were analyzed that revealed at least one click on the glossary. Students who never used the glossary were not incorporated in the ANOVA (see below). Usage of the glossary of students clicking on the glossary at least once revealed a skewed distribution. Hence logarithmic transformation was applied to the data. Transformed data did not deviate significantly from normal distribution (Kolmogorov-Smirnov-Test, p=0.197). Therefore, differences in usage of the glossary were analyzed by a 2-way-ANOVA with graphical quality (2D vs. 3D) and presence/absence of signals as factors. In order to account for any differences in conditions between school classes (e.g. teacher, time of the day the test was done, classroom surrounding) school class membership information served as a blocking factor in statistical analyses. Students were informed about the presence of different versions of the animation only at the end of the lesson after filling out a questionnaire evaluating their impression about the animation. On this questionnaire students were asked whether they agreed or disagreed with each of the following five statements in an ordinal five-point rating scale: 1. The movie “ATP synthase” is easy to understand. 2. The representation of the movie is clear. 3. I would like to watch such movies more often. 4. The length of the movie is appropriate for the content. 5. The movie is attractive. Due to the ordinal scale property, nonparametric statistical methods (Kruskal-Wallis-ANOVA and MannWhitney-U-Test) were used for analyzing possible differences in student´s impression. Statistical analyzes was realized with GenStat for Windows 6th Edition and SPSS 11 for Windows.
3. RESULTS The number of students using the glossary at all was rather low in all four hypermedia variants since less then 50% of the participants used the glossary at least once in each hypermedia version (Table 1). Usage of the glossary did not differ significantly between the 19 school classes participating in the study (Table 2, Class stratum, p>0.05) and was quite similar within signalled and non-signalled animation if graphical quality was the same (Table 1). The implementation of signals led to no significant differences in user behaviour regarding to usage of the glossary (Table 2, Class.*Units* stratum, signals, p=0.138). On the other hand, mean usage of glossary was higher in both hypermedia versions containing the 3D-animation as compared to the hypermedia variants with the 2D animation embedded (Figure 1). These results support the hypothesis that graphical quality has a significant impact on user navigation since usage of the glossary was
328
PROCEEDINGS OF IADIS WWW/INTERNET 2003 – PP. 326 - 332
enhanced in case students used a hypermedia software with a 3D-animation included (Table 2, Class.*Units* stratum, graphical quality, p=0.049). Table 1. Descriptive statistics displaying usage of the glossary with respect to instructional design properties of the animation (signals/no signals, 3D/2D) within the four hypermedia versions. (N = number of participants, Usage (%) = percentage of participants using the glossary at least once)
Signals N Usage (%)
No signals
3D
2D
3D
2D
77 37,6
78 37,2
78 43,6
67 32,8
Table 2. ANOVA-table displaying the statistical results of the usage of the glossary in dependence of instructional design properties. “Class” takes into account differences between the 19 school classes participating. (d.f. = degrees of freedom, F = F-value, p = significance probability)
Source of variation
d.f
Sum of Squares
Mean Squares
F
p
Class stratum Signals Graphical quality (3D-2D) Interaction term Residual
1 1 1 17
0.04 0.04 0.27 3.37
0.04 0.04 0.27 0.20
0.19 0.20 1.35 0.50
0.672 0.657 0.261
Class.*Units* stratum Signals Graphical quality (3D-2D) Interaction term Residual
1 1 1 84
0.89 1.58 0.32 33.27
0.89 1.58 0.32 0.40
2.24 3.98 0.80
0.138 0.049 0.373
107
39.78
Total
Figure 1. Frequency of the use of the glossary with respect to graphical quality (2D/3D) of the animation. The grey box indicates the interquartile range, the horizontal black line shows the median.
329
PROCEEDINGS OF IADIS WWW/INTERNET 2003 – PP. 326 - 332
Agreement to the five statements evaluating the animation indicated a rather positive impression of the students to all software variants (Table 3). Non-parametric ANOVA, however, revealed significant differences between the four variants only in statement 3 (“I would like to watch such movies more often”, Table 3, Kruskal Wallis ANOVA, p=0.05). A closer inspection on that statement showed that students working with a variant that contained a 3D-animation of the ATP -Synthase liked the animation more than students working with the hypermedia software containing a 2D-animation (Mann Whitney U-Test, p=0.031). The presence of signals had no significant impact on agreement to statement 3 (Mann Whitney UTest, p=0.16). Table 3. Level of agreement (mean ± SD) to five statements (see Material and Methods) considering the impression the students had about the animation they worked with. Positive numbers indicate a positive judgment since rating ranged from -2 (“I disagree”) to +2 (“I agree with the statement”). Results of Kruskal Wallis ANOVA on differences in judgement are given. Number of students involved is given in parentheses for each group separately. Signals
No signals
ANOVA
statement
3D
2D
3D
2D
1
0.70±1.14 (71) 1.01±1.17 (71) 0.90±1.14 (71) 1.12±0.82 (68) 0.93±0.99 (71)
0.76±1.10 (70) 1.01±0.96 (70) 0.77±1.08 (70) 0.80±1.11 (69) 0.69±0.97 (71)
0.67±1.08 (72) 0.99±0.93 (71) 0.90±0.97 (69) 0.70±1.064 (71) 0.83±0.82 (69)
0.37±1.09 (59) 0.76±0.92 (59) 0.41±1.18 (59) 0.64±1.17 (59) 0.55±0.98 (58)
2 3 4 5
Chi square
p
5.510
0.138
5.018
0.171
7.824
0.050
6.193
0.103
6.391
0.094
4. DISCUSSION When developing a hypermedia learning environment important issues to reflect on refer to the tools to use for navigation, e.g. index, site map, search machines. A high variation of navigation tools and access possibilities might lead to a better acceptance by the users. This view is supported by various studies reviewed by Chen and Macredie (2002) recommending to take different learning styles into account when designing navigation support for hypermedia systems. An alphabetical index may for example better support learners who tend to be analytical, a site map may better support learners who prefer to process information in a global fashion. Therefore, many hypermedia systems today provide multiple navigation tools to allow users to structure their navigation strategies according to individual taste and need. But what about the influence of content design on the process of learning? An overview on research regarding the role of learner characteristics and other internal, e.g. motivational factors, as well as software design and other external factors, e.g. type of exercise on navigation when learning with hypertext, is provided in Unz (2000). Stiller (2001) provides an overview on research investigating the impact of bimodal (visual and auditive) presentation of text when learning with hypermedia devices. The influence of content modules that contain animated pictorial information on the navigation process so far has not been investigated intensively. CRIMP studies the relation of the didactical and graphical design and the resulting log files. Hereby the log files serve as a window to the navigation and learning processes. The use of the glossary and its dependence on the design of single content modules was investigated as the glossary is considered an important tool in the process of knowledge gain. Two hypotheses were tested: 1) The authors would expect that the presence of signals leads to an increase in the use of the glossary. This expectation rises from results of the study of the isolated animation on ATP synthase. An improved learning success of the students using the signaled version of the movie could be shown (Huk et al, 2003).
330
PROCEEDINGS OF IADIS WWW/INTERNET 2003 – PP. 326 - 332
The bimodal presentation of the content in the animation (moving pictures and narration) seems to sustain understanding even more when important terms and complicated spatial processes explained in the accompanying narration are shown additionally on the screen by the integration of signals. One could expect that visual perception of difficult terms increases the use of the glossary to look up such terms again for a better understanding when using the whole learning environment. However, such an increase could not be found in this study. One explanation for that might be that the integration of signals might be effective enough to grasp the important information without looking into further details by using external tools such as the glossary. 2) The authors would expect that a three-dimensional graphical design leads to an increase in the use of the glossary. This expectation rises from various results drawn from neuropsychological research which provide evidence that a positive mood can systematically affect cognitive processing and hence improves creative problem solving (e.g. Ashby et al, 2002). The results of the log file analysis reveal a relationship between the graphical design of the animation and the frequency of the use of the glossary. It is possible that the presentation of a well-designed three-dimensional animation compared to a simpler two-dimensional animation leads to such a positive affect. These results are enforced by the analysis of the questionnaires since students working with the hypermedia version that contained the three-dimensional animation significantly stronger supported the statement that they wanted to see such animations more often.
5. CONCLUSION Much remains to be learned regarding the issue how different learners perceive hypermedia systems with different design properties. To develop effective multimedia material for education it is necessary to understand the underlying learning processes when using the material. The educational approach depends among others on goals, target groups, content and learner characteristics. According to Alessi and Trollip (2001) educators should use a variety of multimedia materials and approaches to provide flexible learning environments that meet the needs of the greatest number of their learners. However, professional e-learning software normally is available in one version only, namely the version the producers consider to be the best. The results of this paper show that a variation of the graphical and instructional design with the same content presented may lead to a different use of software tools for knowledge acquisition; in this case the use of a glossary. Presently intuition and artistic judgement still seem to be guiding forces when designing animations (Lowe, 2001). Against this background investigations such as performed by the CRIMP project can create an important empirical data basis for future design principles which take the users needs into account and maximize the learning outcome.
6. REFERENCES Alessi, S.M. and Trollip, S.R., 2001. Multimedia for learning: methods and development. Allyn and Bacon, Boston, London, Toronto, Sydney, Tokyo, Singapore. Ashby, F.G. et al, 2002. The effects of positive affect and arousal on working memory and executive attention. In Moore, S., & Oaksford, M. (Eds.) Emotional Cognition: From Brain to Behaviour. Amsterdam: John Benjamins, pp. 245287. Barab, S.A. et al, 1997. Hypermedia Navigation: Profiles of Hypermedia Users. Educational Technology Research and Development, Vol. 45, No. 3, pp. 23-41. Chen, S.Y. and Macredie, R.D., 2002. Cognitive styles and hypermedia navigation: Development of a learning model. Journal of the American Society for Information Science and Technology, Vol. 53, pp. 3-15. Chiu, C.-H. and Wang, F.-U., 2000. The influence of navigation map scope on disorientation of elementary students in learning a web-based hypermedia course. Journal of Educational Computing Research, Vol. 22, No. 2, pp. 135-144. Farrell, I.H. and Moore, D.M. , 2000-2001. The effect of navigation tools on learners` achievement and attitude in a hypermedia environment. Journal of Educational Technology Systems, Vol. 29, No. 2, pp. 169-181. Huk, T. et al, 2003. Helping teachers developing computer animations for improving learning in science education. – In: Crawford, C. et al. (Eds.), Proceedings of the Society for Information Technology and Teacher Education. AACE, Albuquerque, USA, pp. 3022-3025.
331
PROCEEDINGS OF IADIS WWW/INTERNET 2003 – PP. 326 - 332
Huk, T. et al, 2002. The role of navigation and motivation in e-learning - the CRIMP-approach within a Swedish-German research cooperation. In: Wagner, E. and Szuecs, A. (Eds.) Proceedings of the 2002 EDEN Annual Conference, EDEN Secretariat, Granada, Spain, pp. 364-369. Land, S.M. and Hannafin , M.J., 1997. Patterns of Understanding with Open-Ended Learning Environments: A Qualitative Study. Educational Technology Research and Development, Vol. 45, No. 2, pp. 47-73. Lowe, R., 2001. Understanding information presented by complex animated diagrams. In J.-F. Rouet & J. Levonen & A. Biardeau (Eds.), Multimedia learning - Cognitive and instructional issues. Amsterdam: Elsevier Science Ltd, pp. 6574. MacGregor, S.K., 1999. Hypermedia Navigation Profiles: Cognitive Characteristics and Information Processing Strategies. Journal of Educational Computing Research, Vol. 20, No. 2, pp. 189-207. Moeller, J.-M. and Mueller-Kalthoff, T., 2000. Lernen mit Hypertext: Effekte von Navigationshilfen und Vorwissen (Learning with hypertext: the impact of navigational aids and prior knowledge). Zeitschrift fuer Paedagogische Psychologie, Vol. 14, No. 2/3, pp. 116-123. Stiller, K., 2001. Navigation ueber Bilder und bimodale Textdarbietung beim computerbasierten Lernen (Navigation through pictures and audio-visual text presentation in computer-based learning). Zeitschrift fuer Medienpsychologie, Vol. 13, pp. 177-187. Unz, D., 2000. Lernen mit Hypertext: Informationssuche und Navigation. Waxmann, Muenster, New York, Muenchen, Berlin.
332
