Kounin J. S. (1970). Discipline and group management in classrooms. New York: Holt, Rinehart and Winston. Lau^'.^v^-(^2000^); AnalysinS teaching and ...
Optimising Instruction and Learning by ICTs: Multilevel Transformation Theory and an Empirical Pilot Study
Ton Mooij
University ofNijmegen, ITS Toemooiveld 5
NL 6525 ED Nijmegen The Netherlands
Tel. 003124 3653558 Fax 003124 3653599
Corresponding author. Tel.: ++31-243653558; fax: ++31-243653599. E-mail address: t.mooii(%its.kun.nl
Optimising Instruction and Learning by ICTs: Multilevel Transformation Theory and an Empirical Pilot Study Abstract
Specific combinations of educational and ICT conditions may optimise learning processes, in particular for learners at risk. The research question asks which curricular, instruction, and TCT
characteristics can theoretically be expected to optimise learning processes and outcomes, and how this optimising can be developed in educational practice. In answering this question, a theoretical multilevel framework is developed to specify instmction, learning, and ICT conditions that may optimise both teaching and learning. By means of a participatory, use-oriented pilot study in secondary education it is explored how teachers can develop and practice instruction and learning processes that are qualitatively more transparent, more flexible, and more sensitive to
differences between learners, than before. At the same time, a multilevel software prototype is developed and tried out in school practice to support the instruction and learning management of teachers, learners, and school management. The results are evaluated positively and reflect further multilevel conditions in and outside school to optimise instruction and learning processes and outcomes. Also, software characteristics to construct more gsneralised 'diagnostic, instmction, and management systems' (DIMS) are suggested and illustrated.
Keywords: Architectures for educational technology system; Improving classroom teaching; Interactive learning environments; Secondary education; Teaching / learning strategies 1. Introduction
In many coimtries computers are becoming more and more important to designing,
stimulating, and controlling teaching and learning processes and effects (Sinko & Lehtinen, 1999; Smeets et al., 1999). A broad array of Information and Communication Technologies (ICTs) is recognised and implemented, both theoretically and in practice (e.g., Blumenfeld, Fishman, Krajcik, Marx, & Soloway, 2000; Lally, 2000). It seems that ICTs are gaining more and more influence in supporting instructional management at different school levels, at different places, and at different times. Within schools, for example, teachers, school leaders, and ICT coordinators pave the ways to involve other actors like learners, parents, and external professionals in ICT practices. Often some learners are best in 'handling ICTs' and may support other learners and the teachers.
Those 'lower level' educational actors usually provide very useful infonnation about possibilities to improve ICT uses and effects at both lower and higher levels. Mooij and Smeets
(2001) carried out qualitative modelling research into ICT uses in secondary schools with varying backgrounds. Their empirical results suggested five successive phases and corresponding models of ICT implementation. After the initial acquisition of hardware and software and some isolated classroom use in phases one - three, in the fourth phase the educational emphasis in school shifts towards ICTs to support learning processes in more integrated and flexible ways. Examples of school practice characteristics in phase 4 are: consideration of differences between learners'
competencies, use of indicators of learning progress to differentiate next learning processes, organisation of learning and learning itself become more independent of time and place, and stimulation ofICT professionalisation of the school team.
Phase 5 was not found in school practice yet and was sketched by some theoretical features. This phase can be characterised by co-ordinated deployment of software, in particular by using applications that bring about school subject integration and continuously integrated instruction and organisational flexibility extra to phase 4. More specifically, in phase 5 education is 1
supposed to be restructured from the perspectives of the learners. This process requires a coordinated transformation of teaching and learning processes by changing curriculum structuring and instruction management, in close relationship to supporting ICT conditions. Moreover, from the perspectives of pedagogy and learning psychology, transfonnation of education in itself does not seem to be enough. Even as early as the very beginning of kindergarten, differences in development between children may already be very large (Byme, 1998; Jewett et al., 1998; Mooij, 2000). A mismatch between a child's characteristics on the one
hand, and actual pedagogical or instruction characteristics on the other, may create behavioural, emotional, social, and cognitive problems for the child (Jones, Gullo, Burton-Maxwell, & Stoiber, 1998; Walker et al., 1998). Characteristics and processes comparable to those in primary education also occur from the beginning of secondary education and onwards (Schuyt, 1995). For example, in a national survey, Dutch teachers and school leaders in secondary education state that they have too few differentiated curriculum levels and insufficiently differentiated learning materials, to properly and adequately adapt learning processes to the competencies and learning differences between learners (Mooij, 2001). Basing learning sensitively upon learners' initial competences and characteristics, and providing extra or other kinds of stimulation to learners who are functioning either low or high compared to the other learners in class, may therefore optimise instruction and learning processes and effects. Curriculum, instruction, and ICT management characteristics could act as conditions to improve school practice at different levels, in co-ordinated and empirically controlled ways. This optimisation or quality enhancement is expected to demonstrate itself in qualitative and quantitative processes, as well as in more positive learning processes and effects for in particular the learners who are deviating relatively most from the other learners in class (Jones, Rasmussen, & Moffitt, 1997; Kemp, 2000; Wegerif, Mercer, & Dawes, 1998).
Integrating educational differentiation or 'inequality' according to the actual differences between learners, could be the best way to work on 'equality of educational opportunities and outcomes' (Collier, 1994). ICTs seem to be very promising to realise such a pedagogical or educational goal, and in checking whether it has been achieved. Realisation of this 'equality by inequality' goal requires a consistent, theoretically based, multilevel restmcturing approach in
operationalising and implementing relevant optimisation conditions in practice, required ICTs, and their mutually integration processes. The developmental processes in constructing phase 5 therefore need to be based in both theory and practice, probably by various cycles of qualitative and quantitave research. The research question to guide a first cyclus of these developments can be formulated as which curricular, instruction, and ICT characteristics can theoretically be
expected to optimise learning processes and outcomes, and how can this optimising be developed in educational practice?
In answering this question, I will first constmct a theoretical multilevel framework relevant to instruction and learning processes. Then I concentrate on instruction, learning, and ICT conditions to optimise instruction and management of learning processes and outcomes.
Information from a participatory, user-oriented, developmental pilot study in secondary education provides a concrete example of curricular, learning, and ICT conditions in practice. Experiences with a newly developed software prototype lead towards a more general design of software
characteristics integrating different user functions in a more generalised, multilevel organisational context. 2. Theoretical aspects and conditions
2.1. Education as a multilevel system
In designing and evaluating educational processes or effects, the multilevel structure of
education can assist to reveal nested or cross-level relationships between many educational characteristics (Cronbach, 1983; Mooij, Terwel, & Huber, 2000). For example, a learner can be
part of a small group of learners, which in turn is part of a whole class. This structuring allows theoretical multilevel modelling of learning by variables like e.g., the learner's initial competencies and motivation, and instructional characteristics, at the learner level. At the same
time, at the small group level, prior learning characteristics of the small group of which the learner is a member may be relevant. Also, again at the same time, characteristics of the learner's
class, the class' instructional environment, and variables concerning teacher quality, may be at stake at the class level (cf. Blumenfeld, Fishman, Krajcik, Marx, & Soloway, 2000; Crook, 1998). A schematic illustration is given in the bottom part of Table 1. Multilevel modelling provides for organising and checking of individual, small group, and class conditions that may differentially stimulate learning processes of the learners in a class. Moreover, evaluation of learning achievement of the individual, the small group, and the class, can be related to the learning progress of the specific actor at a specific level, but also to cross-
level processes or effects (Goldstein, 1995; Kemp, 2000). If evaluation of individual learning
achievement is based on the learner's own progress, the evaluation may keep its motivating effects for the learner. However, if the individual evaluation is based on e.g., the class' mean,
learners deviating from this mean may come to experience motivation and learning problems because of non-fitting instruction conditions.
Yet, learning processes and effects are situated in a still more complex environment. Several classes make up an educational unit (for example, all first-year classes, classes 1-3, or a specific
combination of school subject departments). Some units represent a location or a separate part of a school building, and different locations combine to constitute a school: see Table 1. The lower-
level instmction and learning processes at the individual, small group, and group level, may for instance be affected also by collaboration processes between the teachers and the pedagogicaldidactic climate at the unit, location, or school level. Other potentially relevant variables can be the location management's strictness in getting along with behavioural rules and order, or the kind of educational priorities set by the overall school management. Characteristics or processes at these higher levels may thus interfere with motivational, social, or achievement processes and effects at the learner, small group, or class levels (cf. Collier, 1994; Mooij, 1999a, 1999b). At higher levels of the educational system, different schools may be governed by the same school board. This relationship can for example be related to specific policy aspects or financial
allocation resources. Different school boards can be found within the same municipality or region which is characterised by e.g., specific social or economic features, and by specific regional institutions for care, health, well-being, police, or justice. A group of regions make up a whole
country and build the national level with, for example, specific curricular, cultural, or linguistic reqmrements or consequences compared to other countries. For example, some countries have a
national curriculum with specific achievement conditions (e.g., the United Kingdom). In other counb-ies, however, curricular specifications are left to educational publishers and schools or
teachers, while assessment of learning achievement is left to national tests which do not directly measure curricular progress (e.g., The Netherlands). If evaluation of individual learning achievement is based on such a national representative test, the motivation and learning problems for learners deviating from this norm may be stimulated again.
Finally, different countries constitute the international level with specific collaborative and competition processes in public or private areas. For example, at the international level, national
or governmental ICT policies combine with those of the European Union to strengthen ICT development and implementation in the EU (cf. Ministry of Education, Culture, and Science,
1999).' ^ ^ - ---. - -.--, ^.^-, ^^ ^^^,
The level-specific characteristics in Table 1 thus illustrate that many educational features can be relevant in instruction and learning and achievement processes. Moreover, characteristics at or
between different levels may interact over the course of time, resulting in rather complex
processes and effects which may influence instruction and learning processes and outcomes
directly or indirectly (see also Goldstein, 1995; Tymms, Merrell, & Henderson, 2000). ICTs could be used to handle this complexity and, at the same time, optimise instruction and learning processes and results.
2.2. Multilevel curricular, learning, and ICT concepts
A curriculum is usually characterised by a set of curricular themes representing contentbased^ concepts and subconcepts related to specific knowledge, skills or competencies, and attitudes, at specified difficulty levels. A theme and its corresponding learner instouction should
inspire, motivate, and challenge each learner to act as a learner by a careful selection of instructional activities. Usually, instmctional activities are based in'one or more cumcular
themes and are developed or selected by a teacher, the team, or a publisher or curriculum
developer. A theme can also contain/ree activities which are chosen or developed by the learners
and evaluated by the learners themselves, or evaluated in co-operation with the teacher.
An instructional line denotes a hierarchical arrangement of curricular concepts and
subconcepts corresponding with specific curricular, instruction, and diagnostic or evaluative
leaming materials representing specific learning activities at a specified difficulty level (cf. Mooij, 2000). Different kinds oi quality indicators need to be integrated into instmctional lines, in order to diagnose and evaluate learning processes and their outcomes on each learner since the
learner's start in school In practice different kinds of instructional lines may be assigned to different learners, small groups of learners, classes, or units, depending on the learners'
characteristics and the kind of grouping processes in a class, unit, location, or school. In practice, location level and school level may be identical. Teachers, coaches, or other professionals from inside or outside school work with the set of
instructional lines and diagnostics on the one hand, and the groupings of learners on the other.
They may support learning processes in differentiated ways, to integrate and optimise teaching and learning. That is, given the initial competencies of an individual learner, and the mean initial
competencies or deviation m initial competencies within a small group, class, or unit, learning progress may take place differently at these different levels because of different arrangements in instructional features. Moreover, variables like degree of selfevaluation of learning achievement
may stimulate learning processes and outcomes, and have positive but different "effects on the appreciation and next use of level-specific instructional features. Teachers or coaches will also
fulfill an important pedagogical role here. Furthermore, ICTs could help to design, mtegrate, and registrate the multilevel mstmction and learning processes and effects, and assist in assigning
learners into groups and to specific instmctional lines. By doing this, ICTs could support
possibilities to self-evaluate learning processes and outcomes in more responsible ways than before, and assist to learning in or outside school. A schematic overview of these possibilities at the combined levels of the individual, small group, class, and unit, is shown in the lower part of Table 2.
Furthermore the location level and school level are characterised by continuous instruction features and middle- and short term curriculum and organisation development. At these levels
comparable learning aspects as v/ith the lower levels may be in the focus, but now at a higher level of aggregation. ICTs can be used to design curriculum and instruction features for the
school or location; to design and registrate grouping of learners; and to design, registrate, and evaluate learning processes and effects in and out of school: see Table 2.
Next, the school board has - among other things - tasks with respect to supervising longtenn curriculum and organisation development, and in supporting institutional interests within the context of the region. The school board itself may experience a learning process on these issues
and could use ICTs e.g., to assist in evaluating and supervising instruction and learning in and out of school, from long-temi school and regional perspectives. At the regional or municipal level, collaboration strategies between different kinds of
institutions may be relevant, resulting in a regional learning process. ICTs could help to design, monitor, and evaluate instruction and learning, from a within and between regional point of view. Higher up, at the national level, national or governmental policies or criteria may stimulate or restrict specific curricular or instructional features. At this level, a pedagogical-didactic kernel structure can be defined as a set of normed referents representing diagnostic and evaluative indicators or benchmarks of instructional lines used in or outside schools. Integrated research on curriculum, instruction, and test development is needed to provide for a reliable and valid pedagogical-didactic kernel structure and its consistent relationships with instmctional lines. In
this respect, a national learning process is at stake when instruction and learning are integrated in a process of continuous improvement. ICTs can also support such a national learning process, including the evaluation of regional and institutional or school specifics.
Finally, at the international level, the instructional focus may be directed at linking and funding research and development projects, resulting also in an international learning process. Of course, ICTs can play an important role in the relevant registration, communication, and evaluation processes: see Table 2.
2.3. Multilevel ICT features potentially optimising instruction and learning
ICTs can fulfill different roles to optimise multilevel curricular, instruction, learning, and management processes (cf. Ely, 1999). For example, ICTs can order and present curricular
themes, concepts and subconcepts; present instmctional lines to different learners or groups of learners at different places at different times; and assist in including or evaluating quality indicators in these lines. ICTs can also help to assess each learner's initial levels of competence; provide for stimulating learning experiences in individual and social respects; registrate and evaluate quality and progress in both individual and group-normed ways; and assist timely availability of specific instructional lines for marginal learners or learners at risk.
Within a school, input or change of instructional lines, integration of external professionalists in school processes, adequate grouping of learners, and coaching in and out of school, may result in improving learning for learners, in particular for learners at risk. Registration and analysis of relevant data, features like planning and logging of learning activities and materials by a learner or a small group of learners, can be carried out automatically. If necessary, teachers in school can easily communicate or collaborate with external specialists (psychologists, health or care specialists) on actual learning problems, prevention issues, treatments, or evaluation of treatments
in school: see the lower part of Table 3. In these ways, educational quality and learning optimisation for each learner can be supported from the beginning in school onwards. At the regional level and below, regional or municipal collaboration between different kinds
of institutions can result in different kinds of collaboration and learning data or benchmarks between and within institutions. Multilevel analysis can produce relevant learning and evaluation data and norms, also about processes and outcomes between and within regions (Goldstein, 1995): see the middle part of Table 3.
Finally, at the national and international level, ICTs are of use in constructing the pedagogical-didactic kernel structure and relevant indicators or benchmarks in the educational
system. A pedagogical-didactic kernel stmcture is for example necessary to make educational
evaluation processes independent of the commonly used age-based groupings or age-based achievement norms. The evaluation alternatives to foster motivation and achievement are
individual evaluation of progress, in the context of considering small group and class or unit mean achievements. Norm based evaluation of achievement produces extra information about the comparison of the individual's achievement with the achievement of the category ofage-mates or
educational mates, but should generally not function as a kind of feed-back in actual learning processes. This independence is essential to prevent phenomena like continuous school
demotivation, forced failure of learners at risk, youth criminality, and drop-out (Collier, 1994; Mooij, 1999b, 2001). 2.4. General hypothesis
The theoretical modelling of multilevel instmction, learning, and ICT features in Tables 2 and 3 is expected to enable an optimising fa-ansformation of the educational system. In this transformation, in particular the learning motivation and achievement of learners at risk will be
promoted. Learners at risk are those learners who are deviating much below or above the general mean of competencies of learners in the own class, unit, location, or school, or an age-based achievement norm.
Two main arguments can underline this hypothesis. First, multilevel competence-based
learning in flexible groupings, as sketched in the above, generally results in qualitatively more supportive and controlled learning processes and effects (cf. Mooij, Terwel, & Huber, 2000). Second, in the flexible optimisation system, teachers and other professionals can improve their support of slower or less adequate but also of the haighly able or gifted learners because of the more self-regulative possibilities in the optimisation situation compared with the traditional situation.
3. Method
2.1. Participatory, use-oriented design
Recent methodology supports a research strategy in which users, for example teachers and school staff, collaborate with researchers and other specialists to secure validity of innovation processes (cf. Crosier, Cobb, & Wilson, 2002; Kensing, Simonsen, & Bodker, 1998). Wilson (1999) expects that 'use-oriented' strategies '(...) increase the likelihood of successful
implementation because they take the end use into account at the beginning design stages' (p. 13). dark and Estes (1999) address the development of authentic educational technologies by
collaboration between for example practitioners, technologists, scientists, craftspeople, and artists. They illustrate a developmental cycle of a science-based technology beginning with
descriptive and empirical research, followed by constmctmg generic technology, then by contextualising the technology v/hich generates new issues for research, etcetera.
According to a participatory use-oriented design, I chose to explore the relevance of the above multilevel theorising in a pilot study in educational practice. It was possible to collaborate with secondary-school teachers and school management to create part of the conditions of Tables 2 and 3, in particular at the lower levels. Comparable multilevel developmental research has been
described by Blumenfeld et al. (2000) and Mooij (1999a, 1999b). In line with those studies,
development of curriculum, instruction, learning, and ICT conditions in practice was expected to start a learning process for all persons involved (cf. Remillard, 2000; Van den Akker, 1999). During this iimovation process the transformation and optimising goals and procedures would
become concrete and be checked qualitatively by the empirical circumstances. The explorative research and development took place during a three year period (July 1999 - June 2002). 3.2. Developmental pilot study: process characteristics
The pilot study is situated in one location (510 learners) out of four locations of a secondary school, in the Eastern part of The Netherlands. The location was already familiar to the
researcher. Teachers and school management had some experience in experimenting with educational innovation. The location is characterised by a relatively liberal climate for both learners and teachers, with no serious school problems or unusual characteristics. Most lessons
are whole-group and rather traditional, while the actual ICT resources are comparable to those in other locations or schools.
During the research period, a steering committee and a development team were in function.
The steering committee (management, researcher, software specialist) met about once a month to discuss planning, progress, and next steps. The development team met each week (three teachers, one researcher, and the software specialist). The teachers were facilitated for about two extra
hours per week, in addition to their lesson hours. First meetings were devoted to explaining the goal and functioning of instmctional lines and possibilities of ICTs. Concrete examples were taken from comparable research in primary and secondary education (cf. Mooij, 2000). Next, the teachers developed instructional lines within the framework of a self-chosen project on 'Water and environment'. Different uses of ICT were integrated in the curriculum activities for learners, for example multimedia and the Internet. The activities were designed for small groups of learners (2-4). Occasionally, a member of the school management joined the meeting. Once in every three months, the location's team received updates and new information from the three teachers.
After one year of developments, results were tried out with learners in the first school-year. The 50 learners were from two classes (aged 12-13). Some of these lessons were observed by the researcher, and instruction and learning processes were discussed with teachers and learners.
Explicit attention was given to integrated registration, evaluation, management, and controlling functions of software that could support teaching and learning processes in flexible ways. 4.RESULTS
4.1. Instructional lines and activity worksheets An overview of the instructional lines as developed by the teachers is given in Table 4.
Individual levels of initial competencies in Mathematics and Reading comprehension are the first learner competencies to be tested (see the bottom part of Table 4). The second row from the
bottom lists the school subjects to which the curricular activities refer (Biology, Physics / Chemistry, Mathematics, Technology, Care / Well-being). 'Own choice' in the last column means that learners can create an activity of their own, which should be related to the kind of activities in the same row.
'Use of water' in the third row from the bottom is a first instructional line represented by five activities on 'living environment' and five activities on 'water house-keeping'. Each activity should be carried out by a small group of learners. In the same vein, subsequent instructional lines were developed with respect to 'Water as a substance', 'Water and Technology', and 'Water in the environment', respectively. From bottom to top in Table 4, the activities listed in the columns have relationships with the respective school subjects.
To aid the self-instruction and self-management of learners, the teachers also created activity worksheets for each activity within the insti^ictional lines. The worksheets contain aspects of preparatory instructional specifics, how these should be carried out, their evaluation, and of
related project activities. Curricular and learning materials themselves are present in or around the classroom, school library, or the Internet. The worksheets thus provide detailed instructional
information about the learning processes, materials and sources, appliances that can or should be used, and an evaluation. An example of a worksheet appears in Table 5 and concerns 'Aquarium as a living enviromnent' (see the respective cell in Table 4) 4.2. First software characteristics
Teachers and learners are the most important 'primary' categories of users, so the development of the software first of all concentrated on facilities that were relevant to these users
at the lower levels of the educational system. It was decided that the software should support the insfa-uction and self-management of the learners, but in general the instructional activities
themselves were not done on the computer (cf. Mooij, 2002). This means that working on the activities themselves could usually be done independent of computer use, which promoted easy computer access and differentiation in didactic working procedures. In interaction with teachers and learners, the software was designed to e.g., conceptualise and create instructional lines, create
groupings of learners, work with instructional lines by teachers and learners, and registrate and check learning processes automatically. After this first specification phase in practice, a software prototype named LINE ('Learning In Networked Environments': www.scholen.net/line) was
developed. An overview of most relevant software characteristics in the prototype media 2002, and the potential users, is given in Table 6.
The software characteristics listed in Table 6 refer to administering, registering, organising, and evaluating everyday school information. It is about curriculum content aspects with respect to instructional lines and diagnostics (individual, small group, class, unit, age-normed), flexible
grouping of learners, planning of combinations of curriculum contents and learners, support of specific learners, and detailed specifications of learners' progress in specific periods of time.
Such administrative and evaluative activities require a lot of teachers' attention each day, and automatic logging and using digital databases seems to be of great assistance to them. Moreover,
digital availability of this information may assist users at all levels of the educational system. 4.3. Evaluation
The teachers worked hard and enthusiastically, also in their spare time which was not
facilitated. They soon grasped the idea of working with instmctional lines and they used these as their global orientation in curriculum and instruction development. Important to them was that the new activities could be carried out by small groups of learners, during the lessons. During an evaluation meeting with learners participating in the development research in June 2001, the developing teachers emphasised in particular the importance of collaboration between learners,
the fact that learners were able to choose their own friends in the team, the relevancy of integrating instructional lines within the regular curriculum, and the urgency of developing the software further.
Also in June 2001, ten of the participating learners volunteered to express their opinion on working with instructional lines. They generally liked this way of working, but they also made remarks on potential improvements by spending about the same amount of time on activities, necessary teacher support, occasional confusion about sources required or places where to find them, learning to collaborate within the small group without whole-group instruction, and the
importance of getting school marks to experience the experimental activities as real school activities.
A year later, still more progress in curricular and instruction development was achieved, but the working with instructional lines and activity worksheets clarified one main educational shortcoming. Though the teachers tried to overcome traditional evaluation of the learners' work
and outcomes, this was not easily realised because of the lack of adequate evaluation procedures or tests. Teachers made evaluation and judgement of learning achievement already more individualised, and more influenced by collaboration processes within small groups. However, because of the general lack ofnormed achievement or progress indicators, it was not yet possible to estimate in reliable and valid ways in how far a learner's progress was meaningful from
normed points of view. In other terms, evaluation could only be done on a local basis, and not by using a pedagogical-didactic kernel structure.
4.4. Next developmental steps
As a first consequence of this evaluation, integrated educational and software development was concentrated on the further operationalisation of diagnostic evaluation and judgement of learner's achievement. The teachers provided criteria related to different aspects of learning e.g., social, knowledge-related, skill-related, or outcome-related. An instructional activity, or part of this activity, can be evaluated according to a scale ranging from 0 to 100 % correct. This evaluation is done by both learners and teachers, which allows a detailed systematic evaluation over specific insfa^ictional activities or parts of these activities, and also of the agreement or
discrepancies between teacher(s) and leamer(s). This evaluation procedure already seems to be very worthwhile in practice and can - in the future, with more learners - assist in the construction
of psychometrically adequate evaluation or test procedures, also in the context of developing an adequate pedagogical-didactic kernel structure.
Furthermore, observations during the experimental lessons and discussions with teachers, remedial staff, management, and learners resulted in suggestions about more generalised 'diagnostic, instmction, and management systems' (DIMS) to optimise functions of instructional
lines and ICTs in organisations (see also Mooij, 2002). From this generalised point of view, the content of instructional lines can refer to specific characteristics or items in a specific order e.g., a
specific issue or subject in a questionnaire or observation list. Also, generalising the software features means that ICTs can registrate or evaluate progress - either locally or normed - in psychological, instruction, work, and management processes; for various types of users, in
different kinds of institutions, at different levels. For example, a learner or a group of learners, teachers, parents, and coaches in or outside school. But also, at the same time, be relevant for or used in research and development, national and international management, regional institutions, institutional boards, units in institutions, and local or institutional management. These different
functions of DIMS are schematised in the ellipse in the middle of Figure 1 and in the margins of this Figure, respectively. In the upper part of Figure 1, research and development are functionally related to the creation or evaluation of normed instructional lines or benchmarks, and qualitative and
quantitative analysis of data. Moreover, (international management can directly or indirectly use ICTs to collect data and provide feedback about normed and other data results with respect to psychological, instruction, work, or management processes. Furthermore, in the lower part of Figure 1, the creation and evaluation of local instructional
lines and benchmarks is a feature characterising the local or total management, the unit, and the teacher or coach. Assignment of normed and local instructional lines or benchmarks is a
characteristic of a teacher or coach, the school management, and the school administration: see Figure 1.
The more general meaning and function of the concept of 'instructional line' in the software
lnt^reta,tion ?fDIMS^C^n^support.the longitudinal integration, regisfa-ation, optimising, and
multilevel evaluation of different kinds of processes, from different kinds of mstitutions. at
different times, and at different places. The multilevel processes may refer to e.g, diagnostic
processes and individual treatments at a very young age, the diagnostics and evaluation of
multilevel intra- and mter-institutional interventions in primary or secondary education, or the
evaluation of combined in- and out-school efforts to prevent school demotivation or development
^Z°^..antis0??' crimmalbehaviour (cf. Mooij, 1999a, 1999b). The software design characteristics of Figure 1 will be worked out in greater detail, in close coifaboration'wTth institutional developments in and outside schools (see: www.dims.nl). 5. DISCUSSION
^Theorising about and modelling of the relevance ofICTs to educational transformation and optimising is still rare, in particular with respect to the optimising functions ofICTs to learners
who traditionally are at risk. Therefore, the focus was concentrated on theoretical possibiiitie7to
optimise instruction and learning by adequate multilevel integration of ICTs in^ education for
^a?ers.m genera1' ^ leamers at risk in particular. Using the results of a former modeiling
study as a point of departure, the research question was formulated as which cumcular.
instruction, and ICT characteristics can theoretically be expected to optimise learning processes and outcomes, and how can this optimising be developed in educational practice? In answering this question, at first the theoretical relevance of a multilevel educational
framework to learning was elucidated (see Table 1). Then concentration was" directed" at "level
specific instruction^ learning, and ICT conditions (Table 2), which in combination with specific " -T_functi^ns/Table,3^ are exPected to enable the optimising of instruction and leaming processes. Next, in a three year empirical pilot study in secondary education, co-operation with
teachers and school management led to the development of four instructional lines on 'Water and
environment' (Table 4) including a format of specific activity worksheets (Table 5) to assist the self-^regulation and self-instruction of learners. These development results allow more flexible
t_TTCh.ingan? leaynin^ Proce3ses' ^Particular for small groups^ of learners. Also, a first" prototype oflntemet-based software designed to support registration, evaluation, management, and control of integrated instruction and learning processes was developed and tried out°in"the'sam~e'school practice. Characteristics of the software prototype are described in Table 6.
The evaluation of both practice development and software use by teachers, management, and
learners, was positive. However, though evaluation of learning achievement could be°camed out
in individual, small group, class, and unit ways, evaluation on the basis of a pedagogicai-didactic
k!rnel^.structu,r,e c?u1^ n?t be carried out- This situation is a general phenomenon In secondary
education in The Netherlands. As a next step, more detailed ICT evaluation characterisnc's"have
.e!n^-!nte^aled in thesoftware- Moreover, suggestions from observations during the experimental lessons and from the teachers' management, and learners led to the developmmt'of
^rc^Sleral ?sign ofdia§nostic' instruction, and management systems (DIMS: see Figure'1)^ These software features are implemented in a new version of the software.
All in all, the results of the multilevel theorising, educational innovation, and software
development seem very promising. Though a lot of work still has to be done, "teachers "and
learners now ^already experience real changes in their everyday work. The present results furthennore allow next concentration on more specific diagnostic and remedial processes, for example the integration of instruction and learning specifics for learners with specific problems 10
in e.g., reading comprehension. With the aid of such results it will become possible to test the hypothesis about the positive motivation and learning achievement of learners at risk. Finally, the outlined multilevel transformation and optimising of education still has a long way to go. A very careful, long-term, gradually broadening collaboration between persons and institutions is needed, with new educational practices and instruments, at an increasing number of levels within the whole educational system. Here only a first step was taken, to learn also that the integrative and supportive role of ICTs will become crucial (see also Griffin & Beagles, 2000). However, given the increasing interest in the present development activities, shown from different sides, more participation in these activities may lead towards the optimising of the educational system, in particular for the learners at risk. Ackn owledgements
The empirical part of the project was financed by the Dutch ministry for Education, Culture, and Science, and by the secondary school involved. I want to thank Michael Polman (Antenna Foundation Nijmegen) and Michael Sperber (ITS Nijmegen) for their collaboration in modelling software features ofLFNE and DIMS, respectively References
Akker, J. van den (1999). Principles and methods of development research. In J. van den Akker, R. Maribe Branch, K. Gustafson, N. Nieveen, and Tj. Plomp (Eds.), Design approaches and tools in education and training (pp. 1-14). Dordrecht, The Netherlands: Kluwer. Blumenfeld, Ph., Fishman, B. J., Krajcik, J., Marx, R. W., & Soloway, E. (2000). Creating usable innovations in systemic reform: scaling up technology-enibedded project-based science in urban schools. Educational Psychologist, 35 (3), 149-164. Borko, H., Davinroy, K. H., Bliem, C. L., & Cumbo, K. B. (2000). Exploring and supporting
teacher change: two third-grade teachers' experiences in a mathematics and literacy staff development project. The Elementary School Journal, 700 (4), 273-306.
Byme, B. (1998). The foundation of literacy. The child's acquisition of the alphabetic principle. Hove, UK: Psychology Press.
dark, R. E., & Estes, F. (1999). The development of authentic educational technologies. Educational Technology, JP (2), 5-16.
Crook, Ch. (1998). Children as computer users: The case of collaborative learning. Computers &. Education, 30 (3/4), 237-247. Cronbach, L. J. (1983). Designing evaluations of educational and social programs. San Francisco: Jossey-Bass.
Crosier, J. K., Cobb, S., & Wilson, J. R. (2002). Key lessons for the design and integration of virtual environments in secondary science. Computers & Education, 38, 77-94. Ely, D. P. (1999). Conditions that facilitate the implementation of educational technology innovations. Educational Technology, 39 (6), 23-27.
Goldstein, H. (1995). Multilevel statistical models. London: Arnold. Griffin, S. L., & Beagles, Ch. A. (2000). Training and performance support systems (TPSS): a case study from needs assessment to return on investment. Educational Technology, 40 (5), 34-42.
Jewett, J., Tertell, L, King-Taylor, M., Parker, D., Tertell, L, & Orr, M. (1998). Four early childhood teachers reflect on helping children with special needs make the transition to kindergarten. The Elementary School Journal, 98 (4), 329-338. 11
Jones^ B. F, Rasmussen,^ C. M., & Moffitt, M. C. (1997). Real-life problem solving. A
collaborative approach to interdisciplinary learning. Washington DC: American Psychological Association.
Jones, I, Gullo, D. F, Burton-Maxwell, C, & Stoiber, K. (1998). Social and academic effects of varymg^types of early schooling experiences. Early Child Development and Care, 146, 1-11.
Kensing, F Simonsen J, & Bodker K (1998). MUST: A method for participatory'desYgn. Human-Computer Interaction, 13, 167-198.
King^E:'olshea'A-A" Joy Patyk, L. I, Popp, L. A., Runions, T, Shearer, J, & Hendren, R. T. (1985). Programming for the gifted. Ontario: Ministry of Education.
Kounin J. S. (1970). Discipline and group management in classrooms. New York: Holt, Rinehart and Winston.
Lau^'.^v^-(^2000^); AnalysinS teaching and learning in networked collaborative learning environments^ issues in work and progress. Paper presented at the European Conference on
oiSheffield. u^ati,°,nalResearch' umversityofEdinburgh, September 2000. Sheffield, UK: University of Ministry of Education, Culture, and_Science (1999). Onderwijs Online. Verbindingen naar de toekomst. [Education On Line. Connections to the Future.] Zoetermeer, The Netherlands: Author.
Mooij'.T- _(1999a)- Promoting prosocial learner behaviour: 1: A multilevel theoretical model. British Journal of Educational Psychology, 69 (4), 469-478.
Mooij, T. (1999b)^ Promoting prosocial learner behaviour: 2: Secondary school intervention and learner effects British Journal of Educational Psychology, 69 (4), 479-504
Mooij, T. (1999a) Preventing antisocial behaviour of young children at risk. Risk Management: An International Journal, 7,49-61.
Mooij, T. (199%). ^Integrating gifted children into kindergarten by improving educational processes. Gifted Child Quarterly, 43, 63-74.
TJ'T'^2,OOO^JFT to °Ptimise didactic management in early education. In C. Day and D. van ^een (Eds.) Educational Research in Europe. EERA Yearbook 2000 (pp. 59-71). Leuven7 Apeldoom, Belgium/ The Netherlands: Garant.
Mooii;.I^(2?0?^ Desi^ng, a.digital mstructional management system to optimize early education Educational Technology Research & Development, 50(4), 11-23.
Mooy, T., & Smeets, E. ^2001). Modelling and supporting ICT implementation in schools. Computers & Education, J(? (3), 265-281^
Mooij, T, Terwel, J, &_Huber, G. (2000). A social perspective on new lean-ung. In R.-J. Simons, J.
van der Linden, and T. Duffy (Eds.), New Learning (pp. 191-208). Dordrecht, The Netherlands: Kluwer.
Re^ul^rd'_^ T" ^000^ Can curriculum materials support teachers' learning? Two fourth-grade teachers' use of a new mathematics text. The Elementary School JournaU 00 (4), 331-350.' Schuyt, C.J. M.^1995). Kwetsbare jongeren en hun toekomst [Vulnerable youth and their
future.] Ryswijk, The Netherlands: Ministerie van Volksgezonuheid, Welzijn en Sport.
Sinko,^ M, & Lehtinen, E. (1999). The Challenges of ICT in Finnish Education'~^s^\a.
Finland: Atena. - " ---------^ .-^~,
Smeets,^ E, & Mooij, T. (2001). Pupil-centred learning, ICT, and teacher behaviour: observations
in educational practice. British Journal of Educational Technology, 32 (4), 403-417^
smeets'.E^MOOij' T-' Bamps' H-' Bart()lome, A., Lowyck, J, Redmond, D, & Steffens, K. ^9?.Jhe.Im^ac!^{I1^0rmcltionand communication Technology on the TeacAer. Nijmegen, The Netherlands. ITS. [http://webdoc.ubn.kun.nl/anon/i/impaofina.pdf]
T5Tn^p^Merre11! c~: &Henderson. B (2000) Baseline assessment and progress during the first three years at school. Educational Research and Evaluation, 6 (2), 105-129. 12
Walker, H. M., Kavanagh, K, Stiller, B, Golly, A, Severson, H. H, & Fell, E. G. (1998). First step to success: An early intervention approach for preventing school antisocial behavior. Journal of Emotional and Behavioral Disorders, 6 (2), 66-80.
Wegerif, R., Mercer, N., & Dawes, L. (1998). Software design to support discussion in the primary curriculum. Journal of Computer Assisted Learning, 14, 199-211.
Wilson, B. G. (1999). Evolution of learning technologies: From instructional design to performance support to network systems. Educational Technology, 39 (2), 32-35
13
Table 1
Multilevel Educational Stmcture and Level Specific Characteristics related to Learning ^evelofeducational system Educational variables or characteristics related to learning, e.g. 10. International
International collaboration and competition, policies, and projects - System, organisational, and didactic characteristics across countries
9. National
Guidelines / mles for curricular, cultural, or language characteristics - System, organisational, and didactic characteristics at national level
8. Regional / municipal 7.School board
- Social or economic characteristics, willingness to collaborate - Organisational or didactic characteristics at regional / municipal level
Policy or allocation characteristics, quality of long-term supervision Organisational or didactic characteristics at school board level
6.School
School policy and educational priorities of management Organisational or didactic characteristics at school level
5 Location / part of school
- Behavioural mles and order, aggregated results of learning processes - Organisational or didactic characteristics at location level
4. Unit of specific location
- Collaboration between unit teachers, aggregated learning results - Instruction or organisational characteristics at unit level
3. Class / group / form
- Teacher-leamer interactions, group learning process and results Instruction or organisational characteristics at the group or class level
2. Small group of learners
- Leamer-leamer characteristics and interactions, small group learning style - Instruction or organisational characteristics specific to small group level
1. Individual learner
- Initial competencies, motivation, learning styles, learning results Instruction or organisational characteristics specific to the level of learner
14
Table 2
Multilevel Educational Stmcture and Curricular, Learning, and ICT Aspects Educational level
Instmction
Level specific characteristics, e.g. Learning
10. Inter- - Linking and funding research national and development projects
9. National - Curricular policy / criteria - Pedagogical-didactic kernel
- International learning and progress
ICTs
Design / monitor / evaluate instruction and learning processes and outcomes in and between (sets of) projects or countries
National learning and progress - Design / monitor / evaluate insb-uction and learning processes
structure
and outcomes in and between
- Research and development
(regional sets of) schools / health / care institutions
8. Regional / - Collaboration strategy between - Regional learning and progress municipal e.g., educational and health / care
instruction and learning processes and outcomes in and between schools / health / care institutions
institutions
7. School - Supervising long-term board curriculum and organisation development - Supporting institutional interests
- Design / monitor / evaluate
School board learning and progress
- Evaluating / supervising instruction and learning processes and outcomes in and out of school
- Evaluating / supervising longtemi school and regional development
Initiating / supporting regional collaboration 6. School - Continuous instructional lines
- (Mean) initial competencies
5. Location and evaluation / grouping
- School/location / unit / class /
for school / location / units
small group / individual aspects of learning processes and effects or
- Design / registrate grouping at
- Location / unit assignment and
grouping of learners
- Middle- / short-term curriculum outcomes / results
and organisation development - Integrating external professions in school
4. Unit - Instmctional lines / diagnostics 3. Class - Class / small group / individual
- (Self)evaluating instructional
- Design curriculum / instruction
school - unit level
- Design / registrate / evaluate learning processes and effects in
effects at school - individual level and out of school at the school/ location level
- Design / log curriculum,
2. Small assignment of learners
- (Mean) initial competencies Unit / class / smai! group / individual learning processes and
group - Integrating teaching / learning 1. Individual - Integrating external professions learner - Coaching in and out of school
effects or outcomes / results
insiTuction, grouping at each level - Registrate / support learning processes, outcomes, evaluations,
- (SelQevaluating instructional effects at unit - individual level
- Support selfregulation and
in and out of school selfevaluation
15
Table 3 Multilevel ICT-features to Optimise Instruction and Learning Processes and Outcomes Educational level
Level specific ICT characteristics with respect to, e.g. Input or change features of Registration or use of Analysis or output of
10/9-1:
- Pedagogical-didactic kernel
International / structure national - Educational system individual - Grouping of learners
- Pedagogical-didactic kernel structure data / benchmarks
Educational system data / nomis
within
- Pedagogical-didactic kernel stmcture evalution data / norms
- Educational system evaluation data / norms
educational system
- Comparing data (longitudinally) between and within nations
8/7-1: - Regional / municipal Regional / collaboration between different school board - kinds of institutions
- Regional / municipal
- Regional / municipal evaluation
collaboration data / benchmarks between different kinds of
- Comparing data (longitudinally)
individual
institutions
between and within institutions
6. - 1: - Instmctional lines (for learners
- Instructional lines data / norms
- Evaluative data (longitudinally)
School, at risk)
Grouping data of learners
location, unit, - Integrating external professions class, small - Grouping of learners
data / norms
between and within schools / locations / units / classes / small
groups / individual learners -Comparing data in and out of
group, learner - Coaching in and out of school
school
16
Table 4 Four Instructional Lines on 'Water and Environment' Cells with prescribed assignments, related to respective school subjects
Own/free
Insto-uctional line
First/second part of line
4. Water in the
4.2. Too less/ Adaptation Cycle of water High/low
Water as Nurturing
Own
environment
much water water levels
energy source world-wide
completion
4.1. Evaporation Per cent of Earn by water in body wasting Organisation ofwater
Cooling water Water as an Own element completion
completion
3. Water and 3.2. Water Biological Sewer/ Make an installation to clean or purify Own
Technology purification purification drinking water (drinking) water completion purification
3.1. At home Pollution of Separation of Water bill, I Supply and Washing Own
drinking substances en II discharge, II machine completion water 2. Water as a substance
2.2. Charac- Surface Density, Fomiulae teristics tension Temperature
Preparing food Own completion
2.1. Sweet and Water mixing Relationships Drinking Phenomena salt / separation water
Ventilation
1. Use of water 1.2. Water
Water in - Transport; I Costs of water Supply and organisms; I and II discharge, I
management and II and control
1.1. Living environment
Aquarium as a living environment
Uwn
completion Water Own
recreation and completion safety
Aquarium, II Designing an Hygiene; if so Own (including aquarium required, completion diagnostics) reading diagnostics
School subjects
1. Biology 2. Physics / ^~ 4. Technology 5. Care / Well- 6. Own Chemistry Mathematics being choice
Initial Mathematics
Infonnation from primary education and a special test developed by the teachers (see cell
competencies
Aquarium, II)
Reading Information from primary education and from special tests (spelling, vocabulary, speed, comprehen- studying, reading comprehension)
17
Table 5
Activity Worksheet concerning: 'Aquarium as a Living Enviromnent' 1. Goal: What will you discover? How can you make a living environment in a small fresh-water aquarium? 2. Prior knowledge Concepts What do you know?
General knowledge about an aquarium Aquarium; Living environment;
Temperature; Light; Oxygen / Carbon dioxide; Food chain Ventilation; Filtering
Skills What can you do?
Set up an aquarium Carry out an investigation Use sources
Understanding
Transfer to an aquarium with salt water
3. What do you need?
The aquarium book; Internet site; Practice material
4. With whom do you collaborate? Work in a group of two 1. 5. What has to be done?
2.
1. Read The aquarium as a living environment" chapter in the aquarium book carefully and make notes.
2. Draft the set-up for a fresh-water aquarium and explain the functions of each piece of equipment. 3. Describe a chain of food in an aquarium. 2.
6. Who will do what?
1.
7. What do you investigate?
Design an investigation in which you can prove the relationship between the amount of light and algal growth. After checking the plan, you carry out the investigation.
"8. WTio will do what? 1. 2. 9. How is the time schedule?
Before the activity......................minutes
During the activity......................minutes After the activity.........................minutes
10. Did you complete all tasks?
Study of theory
Draft the set-up for creating and installing an aquarium Describe the chain of food
Carry out the investigation 11. Evaluation
How was the collaboration?
What is your opinion of ine result? What could be done better with this activity?
Do you have an idea for another activity in this area?
18
Table 6 First Features of Software supporting Multilevel Instruction Management Educational
Instmctional features
Potential users
level 9. National
pedagogical-didactic kernel stmcture: age-
schools, external professionals, parents,
normed test means, heterogeneities
policy instances, research schools, parents, policy instances, external professionals, research & development, (specialist) teachers
8. - 6: Regional or selection out of kernel stmcture group(s) of schools
schools, (specialist) teachers, parents, external professionals, policy instances,
6.-5: School or
a. conceptualizing instmctional lines
kindergarten /
b. assigning materials and activities
location
c. assigning diagnostic tools and evaluation research & development d. groupings of learners / teachers e. assigning learners to lines
4. Unit within the school / some
d. smaller groupings of learners / teachers e. assigning learners to lines
classes and teachers
teachers, specialist teachers, learners, parents, external professionals, research & development
teachers, learners, parents, external professionals, research & development e. assigning learners to lines f. matching materials to individual learners/
3. Teacher or class d. smaller groupings of learners / teachers of learners
(small) groups of learners
g. planning instructional lines for individual learners / (small) groups h. checking, coaching and evaluation 2. Small group of
i. networked learning and evaluation
learners
j. start again with learning according to f
1. Individual learner i. networked learning and evaluation
j. start again with learning according to f
19
learners, teachers, parents, external professionals, research & development learners, teachers, parents, external professionals, research & development
Figure 1 First Design of General "Diagnostic, Instmction, and Management Systems" (DIMS) DIMS
'Create/evaluate' normed instructional lines / benchmarks
group of learners /
research and
development
workers
Collect/ feedback normed and other results
management national / international
child / learner / worker
Use and register progress in normed and local psychological / instructional / work /
management (multilevel) regional set of
processes
institutions
Create / evaluate local instructional external coach
lines / benchmarks
institutional board
"Assign normed' and local instructional lines / benchmarks teacher/coach
unit
Input data./ grouping of learners and teachers / workers administration
management local / total
20
