Int. J. Technology Enhanced Learning, Vol. 3, No. 6, 2011
Orchestrating technology enhanced learning: a literature review and a conceptual framework Luis P. Prieto* GSIC-EMIC Group, University of Valladolid, Cno. del Cementerio, s/n, 47011 Valladolid, Spain E-mail:
[email protected] *Corresponding author
Martina Holenko Dlab University of Rijeka, Omladinska 14, 51000 Rijeka, Croatia E-mail:
[email protected]
Israel Gutiérrez University Carlos III of Madrid, Av. Universidad 30, 28911 Leganés, Spain E-mail:
[email protected]
Mahmoud Abdulwahed College of Engineering, Qatar University, P.O. Box 2713, Doha, Qatar E-mail:
[email protected]
Walid Balid Automatic Control and Automation Department, Faculty of Electrical and Electronic Engineering, University of Aleppo, P.O. Box 13462, Aleppo, Syria E-mail:
[email protected] Abstract: The notion of ‘orchestrating learning’ has gained acceptance within the TEL research community in recent years. However, there is little consensus about what ‘orchestration’ means, and what orchestrating learning in a concrete educational context entails. This paper aims to address these two concerns. Through a literature review focused especially on the field of TEL, we provide definitions of this orchestration, and gather the most commonly cited aspects of orchestration into a unified conceptual framework. This emergent framework is then used as an analytical lens to structure data from an existing case study in order to illustrate its usefulness as a tool to understand and propose new Copyright © 2011 Inderscience Enterprises Ltd.
583
584
L.P. Prieto et al. solutions to aid orchestration in complex, real-world TEL situations. Although further theorisation and modelling of orchestration is still needed, the presented framework provides a first step, backed up by a serious review of the field. Keywords: orchestrating learning; literature review; case study; conceptual framework. Reference to this paper should be made as follows: Prieto, L.P., Holenko Dlab, M., Gutiérrez, I., Abdulwahed, M. and Balid, W. (2011) ‘Orchestrating technology enhanced learning: a literature review and a conceptual framework’, Int. J. Technology Enhanced Learning, Vol. 3, No. 6, pp.583–598. Biographical notes: Luis P. Prieto holds an MSc in Telecommunications Engineering at the University of Valladolid, and is currently working towards his PhD dissertation at the GSIC-EMIC interdisciplinary research group, which specialises in computer-supported collaborative learning (CSCL). After seven years working as a Software Developer at Telefonica I+D, he turned to research, first about grid computing, and now in the field of technology enhanced learning. His main research interests include developing technological and conceptual tools to support orchestration of learning, especially collaborative learning in authentic primary and higher education contexts. Martina Holenko Dlab is a PhD student at Faculty of Electrical Engineering and Computing in Zagreb. She holds a Master in Education of Mathematics and Computer Science. She works as a Teaching Assistant at the Department of Informatics at University of Rijeka where she is currently involved in several undergraduate and graduate computer science courses. Her research is mainly in the area of adaptive hypermedia and recommender systems. She is interested in influences of Web 2.0 and social software on the teaching and learning processes and on the development of web-based learning environments. Israel Gutiérrez is Research Assistant, Associate Teacher and PhD student in the Telematics Department of the University Carlos III of Madrid. He obtained his Telecommunication Engineer degree from the University Carlos III of Madrid in 2007 and Interuniversitary Master in Telematic Engineering from the same university and the Polytechnic University of Catalonia in 2009. After working in the European ICOPER project (eContent plus programme) about the usage of competence-oriented standards in the assessment process, his research is focused on how to improve the efficiency of the provision of formative feedback in class and the orchestration of these interactions. Mahmoud Abdulwahed holds a BSc, MSc, and PhD in Engineering. His main research focus has been on technology enhanced learning, science, technology, engineering and mathematics (STEM) education, and in mathematical modelling of pedagogical processes. He worked as a Researcher at Loughborough University, UK after completing his PhD in Summer 2010. In Fall 2011/2012, he has been appointed as an Assistant Professor in a joint position between the College of Engineering and the College of Education at Qatar University, Qatar. Walid Balid received his BS in Electronic Engineering and MS from the University of Aleppo, Syria, in 2006 and 2011, respectively. He is currently a PhD student and he has lectured as a teaching associate in the Department of Automatic Control and Automation at the Faculty of Electrical and Electronic Engineering, University of Aleppo. His main research interests include
Orchestrating technology enhanced learning
585
embedded systems development, artificial intelligence, and novel constructivist pedagogical methods for engineering education. He is a Principal Developer, Senior Research Engineer, and has been a Manager of the R&D department at AL-AWAIL Co. for electronic engineering in Aleppo, Syria since 2006.
1
Introduction
The increasing dependence on information and communication technology (ICT) for enhancing learning and teaching has introduced a complexity factor into learning environments of all kinds. This technological complexity, combined with the natural pedagogical diversity of educational settings (ranging from traditional initiation-response-evaluation sequences to inquiry-based or collaborative learning), gives raise to a variety of challenges for practitioners and students alike. This is especially true in authentic educational settings, where additional constraints (logistics, curriculum, time constraints, etc) apply. In the field of technology enhanced learning (TEL, see Balacheff et al., 2009), the metaphor of ‘orchestrating learning’ has been frequently used to address the challenges of coordinating the complexity of conducting learning activities in these settings. However, orchestration has been variously defined by researchers, and few concrete definitions of the concept exist. Consequently, there is confusion among researchers about what ‘orchestrating’ comprises, which hampers the accumulation of knowledge and the reusability of results from different studies related to this topic. This paper aims to address this concern through a literature review of the existing definitions of the orchestration concept, with the emphasis on its use in the field of TEL (Section 2). This section also reveals a number of trends that highlight the importance of several aspects in describing, analysing and designing ‘orchestrated learning’. The review concludes with the proposal of a conceptual framework which includes eight aspects regarding the questions “what is orchestrating learning?” and “how should this orchestration be done?” The usefulness of this emergent conceptual framework as an analytical lens to understand and propose new solutions to the orchestration challenges of authentic settings is illustrated through the analysis of existing data from an authentic TEL setting (a technology-enhanced classroom in a primary school in Spain). By using the proposed framework, the authors are able to detect challenges to the orchestration of learning in such an authentic setting (Section 3). Finally, several concluding remarks are given in Section 4.
2
Orchestrating learning in TEL literature
2.1 What is orchestration? Definitions One of the first questions that arise when the word ‘orchestration’ is brought up in conversation among TEL researchers is: “what do you mean by orchestration?” In our literature review on the concept of orchestration we have found few formal definitions of the word, and fewer still which are specific to TEL situations.
586
L.P. Prieto et al.
A general English dictionary tells us that orchestrating is “to arrange or combine so as to achieve a desired or maximum effect” (Merriam Webster, Inc., 2011). Apart from its obvious musical definition, the metaphor of orchestration has also been used in purely technological terms. In the area of service-oriented architecture, orchestration has been used to denote the (often automated) combination of services to create higher level services and processes (Peltz, 2003). It is no wonder that this concept has resonated as an adequate metaphor for teaching practice, if we consider that teachers have certain (pedagogical) goals in mind, decide what the flow of the lesson will be, and also decide when the lesson is over (Kovalainen et al., 2001; Forman and Ansell, 2002), hence the assertion of teachers ‘orchestrating classroom discourse’ (Jurow and Creighton, 2005). In this sense of ‘classroom interactions management’, orchestration has been used on numerous occasions by educational researchers, who sometimes refer to conventional pedagogical practices: Chamberlain et al. (2001) define orchestration as a process that leads to knowledge construction in a student-centred approach under teacher guidance; Watts (2003) refers to orchestrating learning as a way of knowing and understanding students who have diverse preferences and appreciate and create a variety of approaches to support their learning; other authors (McCutchen et al., 1994; McCutchen, 2000; Graham and Harris, 2000) use the term ‘orchestration’ to emphasise that writing processes must be activated and coordinated by a control structure, such as the monitor in the Hayes and Flower (1980) model. However, in the field of TEL and related areas, the use of the word ‘orchestration’ is relatively recent and is still a rather fuzzy term. Fischer and Dillenbourg (2006) provide a definition in the field of computer-supported collaborative learning (CSCL): “the process of productively coordinating supportive interventions across multiple learning activities occurring at multiple social levels”. Later, Dillenbourg et al. (2009) expanded on this definition by stating that learning environments “are integrated with activities occurring at various social levels (e.g., individual, group, class), across different contexts (classroom, home, laboratory, field trips, etc.) and media (with or without computers, video, etc.)”. Apart from these definitions in the field of CSCL, orchestration is also increasingly used in the rest of the TEL research community. As an example, the STELLAR Network of Excellence (an EU-funded project) proposed that ‘orchestrating learning’ is one of TEL’s ‘grand challenges’. Apart from references to the aforementioned works by Dillenbourg and others, their materials highlight other definitions such as “the challenge of the actual implementation of all the interactions needed for a successful scenario” (Niramitranon, 2006), and the importance of making orchestration ‘adaptive’, so that it “takes into account the needs and flow of the learning moment” (Balacheff et al., 2010). Drawing from these and other sources, there have been attempts to provide a more complete definition of orchestration as: “coordinating a teaching/learning situation, from the point of view of the teacher. Orchestration aims to manage (or subtly guide) the different activities occurring at different educational contexts and social levels, using different resources and tools in a synergic way. Orchestration is […] often guided by a design (in the form of a script or not), that may be flexibly modified during the enactment (automated or not) of the activity, in response to emergent occurrences” (Prieto et al., 2011a). Before we plunge into the different aspects and dimensions of the metaphor of orchestration in TEL literature, we should note a couple of things about its context of use.
Orchestrating technology enhanced learning
587
Even if the concept of orchestration can be stretched to apply to any learning situation, the word ‘orchestration’ is almost invariably used in the context of formal learning, tied very often with the concept of a ‘classroom’ where there is a teacher or ‘more knowledgeable other’. This includes a main body of work in the context of face-to-face lessons in a (physical) classroom (e.g., Beauchamp and Kennewell, 2010; Dillenbourg and Jermann, 2010), but also in distance and blended learning (Dillenbourg and Jermann, 2007; Carell and Schaller, 2009; de la Fuente-Valentín et al., 2010). The workplace is another possible context for orchestrated learning experiences (Balacheff et al., 2010).
2.2 Aspects of orchestration There have been few attempts to provide conceptual frameworks that account for the different aspects and characteristics of orchestration of learning activities, especially those involving new technologies. Those sources that do provide conceptual frameworks either lack extensive literature reviews to back them up [being more based on their authors’ own research and personal experience, see Dillenbourg and Jermann (2007, 2010) for example], or provide mere lists of issues and research challenges (Balacheff et al., 2010). Moreover, there are several TEL research works that mention orchestration and delve into some particular aspect of it, but do not give explicit definitions or conceptual frameworks for it. We have tried to address these deficiencies in this paper by reviewing extensively the TEL literature related to the concept of ‘orchestrating learning’, and trying to provide a cohesive framework that includes the aspects most often mentioned by TEL researchers in relation to this concept. We believe that this kind of framework can be useful as an analytical lens when researching TEL settings (especially authentic, complex classroom settings, as mentioned above) as a way to structure the information available to the researcher and to detect challenges and eventual solutions to aid orchestrating learning in such settings (please refer to Section 3 for an example of this). Rather than presenting a list of all the reviewed literature here, and for brevity’s sake, we have chosen to cluster the main aspects mentioned in the available literature, presenting these in thematic groups, along with the literature sources that mention these themes. •
Design/planning: As can be seen from the definitions of orchestration provided above, an important component of orchestration is planning the learning activities that will be coordinated. This planning ahead, often referred to as learning design (Koper and Tattersall, 2005) or simply design, can be related to the areas of instructional planning and instructional design, which have a long tradition in education. This aspect also relates to the idea of orchestration coming from music (which musical instruments will play which parts of a musical composition). In education, it has been argued that orchestrating can be better achieved via designing collaborative lesson plans that are rich enough to accommodate the nine elements of Gardner’s (2006) model of intelligence (Moran et al., 2006). There is an even greater need in TEL for adequate planning of the activities and technological tools that will be used to enact them, so that the objectives of a learning activity can be achieved. In this sense, we can see that the design and enactment of flow control in e-learning environments (also known as ‘scripting’) is very closely related to orchestration, including the work in the IMS Learning Design specification (IMS Global Learning
588
L.P. Prieto et al. Consortium, 2003) and other scripting frameworks (e.g., Kobbe et al., 2007). Consequently, the importance of design and scripting is mentioned in several orchestration frameworks (Dillenbourg et al., 2009; Dillenbourg and Jermann, 2010). However, it is important to note that the preparation and enactment of orchestrated activities do not necessarily follow a linear process; for example, Berggren et al. (2005) depict a ‘bricoleur’ teacher that designs activities and re-designs them as the course progresses. This adaptation is one of the hallmarks of orchestrated TEL activities, as we will see below.
•
Regulation/management: The idea of ‘orchestrating learning’ trades upon the virtues of managing the processes of learning and teaching in order to maximise outcomes on a variety of fronts (Watts, 2003). According to Watts, well-orchestrated learning takes place when ‘it all comes together’, when a teacher stages personally satisfying sessions that ‘chime’ with learners’ favoured modes of learning which, at the same time, are modulated by the demands and characteristics of the subject under consideration. Thus, an important aspect of many works on orchestration is the regulation of learning activities [either external regulation or self-regulation, as mentioned by Dillenbourg et al. (2009)]. Issues related to class, time, workflow and group management, which appear in numerous orchestration works (such as Dillenbourg and Jermann, 2007, 2010; Dillenbourg, 2008; Niramitranon et al., 2010), can also be gathered under this theme. This regulation and management of learning processes and their constraints can be done either ‘manually’ through social interactions (Dimitriadis et al., 2007; Alavi et al., 2009; Jermann et al., 2009) or mediated and/or automated by technological means. In the latter case, this is normally done by specifying the learning activities through computer-interpretable scripts (Weinberger et al., 2009; Gruber et al., 2010; de la Fuente-Valentín et al., 2010). Also related to this theme are the works of Wecker and Fischer (2007) on the ‘fading’ of scripts as a way in which this regulation/management may be modified over time as learners progress.
•
Adaptation/flexibility/intervention: Another aspect that can be derived from the definitions of orchestration provided above, as well as from many other sources dealing with orchestration (Dillenbourg and Jermann, 2007, 2010; Dillenbourg et al., 2009), is that of orchestration as intervention; that is, the act of changing and adapting the design/plan to both the local context of the classroom and the emergent occurrences during the enactment of learning activities. This often requires the management of learning activities, either through social mechanisms (see above) which are naturally flexible, or through technological systems that are flexible enough to handle those adaptations (Dillenbourg and Tchounikine, 2007; Karakostas and Demetriadis, 2009).
•
Awareness/assessment: Since the concept of orchestration revolves around making interventions in response to classroom context and emergent occurrences, awareness of what is happening in the classroom and within the learners’ minds is crucially important in any well-orchestrated learning scenario (Dillenbourg et al., 2009; Balacheff et al., 2010). Alavi et al. (2009) demonstrate how simple awareness mechanisms can enhance the orchestration of a certain type of classroom, and the concept is further developed by Dillenbourg and Jermann (2010). Assessment (either formative or summative) can provide insight into the progress towards the intended
Orchestrating technology enhanced learning
589
learning outcomes being gradually achieved by the learners, thus allowing for adequate adaptations of the learning design (see above). For example, the formative e-assessment case studies analysed in Pachler et al. (2010) show that defining the roles of the key players (teacher, learner as an individual, and peers) is crucial in order to orchestrate formative assessment in the teaching process. Although the use of technology in the formative assessment process has a socio-technical impact in orchestration, its high current and future potential is shown by Pellegrino (2010). In fact, the awareness provided by formative assessment is as beneficial for learners (to improve their learning), as it is for teachers [see, e.g., Watts (2003), where the orchestration process is defined as adapting the teacher’s teaching style to the learner’s learning style in a concrete context]. •
Roles of the teacher and other actors: The body of research work around orchestration focuses primarily on the perspective of the teacher [as expressed explicitly by Dillenbourg and Jermann (2010) and Balacheff et al. (2010)], where teacher presence is essential to achieve orchestration (Kennewell et al., 2008). This has prompted many advocates of learner-centred approaches to dismiss the concept of orchestration altogether. However, nothing precludes the concepts presented here from being used by learners themselves, shifting the load of orchestrating the activities from the shoulders of teachers. In fact, some authors argue that teachers in orchestrated learning are ‘guiders, not a knowledge source’ (Chamberlain et al., 2001; Natriello, 2007), and that there is a correlation between concepts of orchestrating learning and principles of constructivist pedagogy (Chamberlain et al., 2001; Richardson, 2003) as a means of facilitating authentic learning (Richardson, 2003; Natriello, 2007). There is a range of possibilities for more learner-driven orchestration, from approaches where learners directly affect the awareness mechanisms (Alavi et al., 2009), to scripted environments where scripts are progressively faded (Wecker and Fischer, 2007), or scenarios where only the widest goals and activities are set by the teacher, and the learning tasks and their coordination are handled by the students.
So far, we have identified five main aspects of the literature regarding what orchestration means in TEL. However, there is a dearth of advice about how to design and support well-orchestrated learning experiences. The main exception to this is the work of Dillenbourg and Jermann (2010), which takes a more normative approach to the orchestration of face-to-face classrooms. In this and other related works we can find three aspects that are not concerned so much with what orchestration is, but rather how it should be done: •
Pragmatism/practice: Dillenbourg and others have noted that many of the ideas behind the concept of orchestration deal with making TEL research results available to average (as opposed to TEL-expert) teachers, and catering for the constraints of authentic classroom settings especially (Dillenbourg, 2009; Dillenbourg and Jermann, 2010). This emphasis on pragmatic research efforts and changing everyday teaching practice in authentic settings (as opposed to controlled experiments) seems to be a common concern of current TEL research (hinted at by the fact that the latest international CSCL and EC-TEL conferences have had ‘practice’ as one of their main themes).
590
L.P. Prieto et al.
•
Alignment/synergy: Probably the single most often cited characteristic of a well-orchestrated learning experience is the coordination of the elements to be orchestrated (learning activities at various social levels, tools and scaffoldings used, including teacher and peer actions) so that they provide what Tabak (2004) labelled ‘synergistic scaffolding’ (Dillenbourg et al., 2009; Dillenbourg and Jermann, 2010; Prieto et al., 2011b). The alignment of as many of these elements as possible, through the changing conditions of a learning situation, in order to attain the learning goals that are desired at different levels, is one of the few clues given to the TEL practitioner who intends to better orchestrate a learning scenario. It has already been pointed out that to align the intended learning outcomes of a course with teaching and learning is a big challenge (Biggs and Tang, 1999). Watts (2003) introduces new factors in this equation, such as the engagement of students to meet the intended outcomes and the learners’ method of learning, which is influenced by innate issues like gender or culture.
•
Models/theories: The increasing complexity of current and future learning scenarios urges the development of more robust theories and models of orchestrating learning (Natriello, 2007). A number of models of orchestration are proposed in the literature, but they are strongly heterogeneous and very context specific. Weinberger et al.’s (2007) model of orchestration for CSCL environments is implemented via guiding scripts. Chamberlain et al. (2001) implemented the orchestration of an engineering course via a cyclic learning pedagogical model. Orchestrating the process of listening in language learning courses is implemented via cyclic pedagogical models and activating meta-cognition and cognitive skills (Vandergrift, 2003). Sha and Aalst (2009) describe a 2 × 3 model to orchestrate assessment in collaborative work, emerging from the interaction of three elements (knowledge cognition, collaboration, and regulation) on two levels (individual and group). Another model of five practices is proposed to orchestrate mathematics learning (Stein et al., 2008). Less explicitly, other researchers propose what can be considered, to a large extent, orchestrating learning models. For instance, Abdulwahed et al. (2008b) show mathematically the advantages of an orchestrated lecturing process as opposed to passive lectures. Abdulwahed et al. (2008a) view the process of orchestrating project-based learning from a Cybernetics perspective. Abdulwahed and Nagy (2009) describe a pedagogical model for orchestrating laboratory education based on Kolb’s experiential learning and by utilising virtual and remote experimental technologies. Prieto et al. (2011b) hint at using Activity Theory (Engeström et al., 1999) as a theoretical lens to analyse the orchestration of learning experiences and the alignment of the mediational means used towards the learning objectives at different granularities. Nevertheless, a deeper theoretical analysis backed up with empirical data is needed to alleviate the lack of general theories and models that can guide researchers and practitioners in orchestrating learning.
The reviewed literature on orchestration has also spotted, although not studied in detail, the influence of other aspects such as the motivational/emotional ones, or the role of evaluation and other after-the-experience processes common in learning scenarios. Other authors point at wider-scope issues, and argue that educational institutions should be restructured to enable orchestrated learning that meets the needs of future education (Natriello, 2007). Further research is needed in these aspects in order to provide a more complete view of the orchestration of TEL scenarios.
Orchestrating technology enhanced learning
591
The ‘five + three aspects’ presented so far can be considered our contribution to a more general and cohesive model of orchestrating learning, especially aimed at TEL. However, the authors reckon that the usefulness of this frame-word for TEL researchers has to be further grounded in empirical data. This conceptual framework is represented graphically in Figure 1. Figure 1
3
Graphical representation of the orchestration conceptual framework emerging from the literature review
Orchestrating TEL in practice: analysis of a case study data
This section describes a case study of TEL research and practice, analysing its data from the point of view of the orchestration conceptual framework presented in the previous section. Even if this case is the object of past and ongoing research efforts with its own goals and methodologies, this analysis illustrates how to use the proposed conceptual framework to understand how orchestration happens in such an authentic setting, also providing readers with a clearer view of what the abstract concepts mentioned so far mean in real-world TEL situations. Moreover, this analysis can also serve to highlight how each of these elements may have contributed or hampered the success of orchestration in that concrete case and the unsolved orchestration challenges that remain ahead. First, we describe briefly the educational setting and the context of the research
592
L.P. Prieto et al.
study. Then, the orchestration of learning in that concrete setting is characterised around the framework’s most relevant aspects. Finally, these aspects are used to detect current challenges and new solutions to help orchestration in the TEL setting under study.
3.1 Context This section describes a study that took place in the context of one primary school in a rural setting in Spain. The goal of the study was to examine the integration of a new collaborative tool (SRI, 2011) into the school’s everyday classroom practice and curriculum, which already involved other technologies like digital whiteboards or tablet PCs. Group Scribbles is a simple shared whiteboard tool, based on the metaphor of shared virtual stickers where students and teachers can draw, write and improve upon each other’s ideas, which also allows for flexibility and improvisation (Roschelle et al., 2007). An example of its interface can be seen in Figure 2. For six months, and following a qualitative case study methodology (Stake, 2005), the researchers helped five teachers to co-design and deploy 32 learning activities involving Group Scribbles and other tools, and observed as the teachers enacted those activities. Figure 2
Graphical interface of the group scribbles collaborative application (see online version for colours)
The five classrooms had between 18 and 25 students, aged from six to eight years old. The background and experience of the five participant teachers was heterogeneous, with relatively novel (less than four years’ experience) and also very experienced teachers (more than 20 years of experience). Each classroom had an interactive whiteboard, a PC for the teacher, and up to 20 tablet PCs for students (which were rotated between the different classrooms). The school management encouraged teachers to make full use of the available ICT resources, and the use of Group Scribbles was one of its initiatives to achieve this aim. Apart from the observation data (including observer notes, audio and screen recordings), the researchers used other data sources, such as focus groups, interviews and school documentation to contextualise and triangulate the study results. Overall, the
Orchestrating technology enhanced learning
593
integration of Group Scribbles into the studied classroom’s everyday practice was considered a success, although a number of challenges and points for improvement were detected. A more detailed description of the data and findings of this case study can be found in Prieto (2009). Here, we will only analyse the data and findings from the perspective of the orchestration conceptual framework presented above.
3.2 Characterising the situation through orchestration aspects Regarding the design of the learning experiences, it was found that teachers’ designs were invariably high-level, coarse skeletons of action that were completed on-the-fly by teachers while they enacted them, even improvising in the face of classroom occurrences. This was mostly due to the extremely tight time constraints of the setting, which did not allow for long design and co-design sessions with teachers. An analysis of the teachers’ designs revealed that these skeletons were made up of a limited set of elements [that the researchers later called ‘design routines’; see Prieto et al. (2010)], which embodied some of the tool affordances and teaching style. Also related to the design aspect was the fact that the deployment of their activity ideas was sometimes problematic for teachers, and often required the intervention of researchers. The management and regulation aspects of the observed experiences were typical of this kind of face-to-face context: most coordination was done socially (e.g., the teacher giving instructions orally) in a natural and flexible manner. However, when analysing the observation data, the researchers noted that this social coordination of the learning process, including both ICT and non-ICT tools, was also highly routinised (i.e., the task distribution or the assessment of an activity were normally managed in the same way by each teacher), a fact that led the researchers to propose the concept of ‘enactment routines’, which also embodied aspects of tool affordances and teaching style (Prieto et al., 2011c). It is also interesting to note how the adaptations and interventions were made during the enactment of the designed activities. Group Scribbles’ simplicity allowed for the flexible adaptation of the activities to unexpected classroom events (e.g., the creation of new, fully improvised tasks). It is also worth noting that the adaptations and improvised tasks followed the same sets of design and enactment routines that have been mentioned. All the observed activities contained assessment tasks for children to complete, and in fact most of the Group Scribbles activities had a general aim to assess the progress of the class after a few more ‘traditional’ lessons. Yet, the most important assessment aspect was the discovery by teachers of the on-the-fly assessment capabilities of Group Scribbles. Since in many cases communication happened through Group Scribbles’ shared whiteboard, the teacher could easily and immediately assess the students’ knowledge and problematic concepts just by observing the evolution of the whiteboards. Regarding the role of the teacher and other actors, the studied classrooms presented a teacher exerting his role in a classical way as the main source of knowledge. However, within this general trend, great differences could be found in teaching styles, with more experienced teachers acting more as facilitators of student collaboration and knowledge construction (which, in the study, was marked by the use of a wider range of the aforementioned ‘routines’ and greater levels of improvisation, to keep students engaged). Also of note is the role of the school principal, not only as a manager but also as the main technological advocate in the school, an often overlooked but very important contextual factor in the integration of new technologies in the classroom.
594
L.P. Prieto et al.
3.3 Successes, challenges, and insights In general, the integration of the Group Scribbles tool into the everyday classroom practice of the primary school was considered a success, being well regarded by the school management, teachers, and students themselves. The study highlighted the importance of addressing contextual constraints pragmatically (e.g., the aforementioned lack of time, the low average technical knowledge of teachers) in the success of real-world TEL experiences. In this concrete case, the simplicity and flexibility of Group Scribbles for on-the-fly assessment and adaptations have been highlighted as a success factor (Prieto, 2009). Also, the researchers’ bottom-up approach of co-designing activities integrated in the usual curriculum may have contributed importantly to this success. However, important challenges were also encountered to the successful orchestration of the activities: the deployment of activities (i.e., progressing from the teachers’ ideas to the ICT infrastructure that embodied them) was difficult for teachers, even in the case of a simple application like Group Scribbles. This factor alone, which required the researchers’ intervention before most of the activity enactments, hinders the scalability and sustainability of this kind of intervention. This points out that (semi-)automated methods of deploying learning designs (expressed in a language usable by teachers) might be a useful orchestration aid to keep design and the rest of the orchestration aspects (which mostly happen during enactment) connected. The discovery of the design and enactment routines in the orchestration by the teachers in the school highlights the usefulness of these routines as a conceptual tool to help teachers to orchestrate their learning activities [e.g., to be used in professional development efforts; see Prieto et al. (2011c)]. This could be seen as the first step in a theory (even if it is a humble one) or model about how orchestration of multiple activities and tools happens in this kind of primary classroom. Also, this study and related studies in the US point out that an alignment of this conceptual and technological scaffolding for teacher practice is necessary for really orchestrated learning (Prieto et al., 2011b). However, it is clear that the researchers still lack a complete, reliable model or theory about what is the best way to orchestrate learning in this kind of classroom. Further research work is needed in that regard.
4
Conclusions
So far, we have provided a review of the concept of ‘orchestrating learning’, especially focussed in the area of TEL, showing the variety of contexts in which the term has been used. Emerging from this literature review, a novel conceptual framework of orchestration has been proposed, composed of ‘five + three’ thematic groups: design/planning, regulation/management, adaptation/flexibility/intervention and awareness/assessment are relevant to the question “what is orchestrating learning?”, while pragmatism/practice, alignment/synergy, and models/theories are more related to the question “how can orchestration be achieved?”. A way of applying the framework in practice as an analytical lens to understand complex, authentic TEL settings has been illustrated through a small case study. We believe that, regardless of the accuracy of the orchestration metaphor to model TEL approaches, many of the works reviewed here share a common concern about the
Orchestrating technology enhanced learning
595
increasing complexity of educational practice in authentic TEL settings. Tools like the conceptual framework presented here may help TEL researchers to understand better these complexities, so that the solutions they propose, however focussed or minimalistic, can be more easily integrable with the rest of the classroom ecosystem and its constraints, where they will have to be used. This will undoubtedly lead to a greater (and more sustainable) impact of TEL research across our educational system.
References Abdulwahed, M. and Nagy, Z.K. (2009) ‘Applying Kolb’s experiential learning cycle for laboratory education’, Journal of Engineering Education, Vol. 98, No. 3, pp.283–294. Abdulwahed, M., Nagy, Z.K. and Blanchard, R. (2008a) ‘Constructivist project based learning design, a cybernetics approach’, Journal of Education, Informatics and Cybernetics, Vol. 1, No. 2, pp.1–8, ISSN 1943-7978. Abdulwahed, M., Nagy, Z.K. and Blanchard, R. (2008b) ‘Using feedback control engineering for analyzing and designing an effective lecturing model’, 38th Annual Frontiers in Education Conference (FIE 2008), IEEE, pp.1–6. Alavi, H.S., Dillenbourg, P. and Kaplan, F. (2009) ‘Distributed awareness for class orchestration’, Learning in the Synergy of Multiple Disciplines – Proceedings of the 4th European Conference on Technology Enhanced Learning (EC-TEL 2009), Springer, pp.211–225. Balacheff, N., Bottino, R. and Fischer, F. (2010) ‘D1. 1 The STELLAR vision and strategy statement’, Deliverable, STELLAR NoE, available at http://dspace.ou.nl/handle/1820/2823 (accessed on 19 November 2011). Balacheff, N., Ludvigsen, S., de Jong, T., Lazonder, A. and Barnes, S. (Eds.) (2009) Technology-Enhanced Learning: Principles and Products, Springer, New York. Beauchamp, G. and Kennewell, S. (2010) ‘Interactivity in the classroom and its impact on learning’, Computers & Education, Vol. 54, No. 3, pp.759–766, ISSN 03601315, available at http://linkinghub.elsevier.com/retrieve/pii/S0360131509002760 (accessed on 19 November 2011). Berggren, A., Burgos, D., Fontana, J.M., Hinkelman, D., Hung, V., Hursh, A. and Tielemans, G. (2005) ‘Practical and pedagogical issues for teacher adoption of IMS Learning Design standards in Moodle LMS’, Journal of Interactive Media in Education, Vol. 2, No. 2, pp.1–24. Biggs, J.B. and Tang, C. (1999) Teaching for Quality Learning at University. ISBN 0335201717, Open University Press, Buckingham. Carell, A. and Schaller, I. (2009) ‘Scenario-based orchestration of Web 2.0 applications in university teaching and learning processes: a case study’, International Journal of Web Based Communities, Vol. 5, No. 4, pp.501–514. Chamberlain, M.K., Williams, C.B., Cowan, F.S. and Mistree, F. (2001) ‘Orchestrating learning in a graduate engineering design course’, 13th International Conference on Design Theory and Methodology. de la Fuente-Valentín, L., Pérez-Sanagustín, M., Santos, P., Hernández-Leo, D., Pardo, A., Kloos, C.D. and Blat, J. (2010) ‘System orchestration support for a flow of blended collaborative activities’, 2010 International Conference on Intelligent Networking and Collaborative Systems, IEEE , pp.415–420. Dillenbourg, P. (2008) ‘Integrating technologies into educational ecosystems’, Distance Education, Vol. 29, No. 2, pp.127–140, ISSN 0158-7919. Dillenbourg, P. (2009) ‘Exploring neglected planes: social signs and class orchestration’, Community Events – Proceedings of the International Conference of Computer-supported Collaborative Learning (CSCL2009), pp.6–7.
596
L.P. Prieto et al.
Dillenbourg, P. and Jermann, P. (2007) ‘Designing integrative scripts’, in Fischer, F., Kollar, I., Mandl, H. and Haake, J. (Eds.): Scripting Computer-supported Collaborative Learning: Cognitive, Computational and Educational Perspectives, Springer, New York. Dillenbourg, P. and Jermann, P. (2010) ‘Technology for classroom orchestration’, in Khine, M.S. and Saleh, I.M. (Eds.): New Science of Learning: Cognition, Computers and Collaboration in Education, Springer, New York. Dillenbourg, P. and Tchounikine, P. (2007) ‘Flexibility in macro-scripts for computer-supported collaborative learning’, Journal of Computer Assisted Learning, Vol. 23, No. 1, pp.1–13, ISSN 0266-4909, available at http://www.blackwell-synergy.com/doi/abs/10.1111/j.13652729.2007.00191.x (accessed on 19 November 2011). Dillenbourg, P., Järvelä, S. and Fischer, F. (2009) ‘The evolution of research in computer-supported collaborative learning: from design to orchestration’, in Balacheff, N., Ludvigsen, S., de Jong, T., Lazonder, A. and Barnes, S. (Eds.): Technology-Enhanced Learning, Springer, New York. Dimitriadis, Y., Asensio-Pérez, J.I., Hernández-Leo, D., Roschelle, J., Brecht, J., Tatar, D., Chaudhur, S., DiGiano, C. and Patton, C.M. (2007) ‘From socially-mediated to technologymediated coordination: a study of design tensions using Group Scribbles’, Proceedings of the Computer Supported Collaborative Learning 2007 Conference, CSCL 2007. Engeström, Y., Miettinen, R. and Punamäki-Gitai, R-L. (Eds.) (1999) Perspectives on Activity Theory, Cambridge University Press, Cambridge, UK. Fischer, F. and Dillenbourg, P. (2006) ‘Challenges of orchestrating computer-supported collaborative learning’, Paper presented at the 87th Annual Meeting of the American Educational Research Association (AERA). Forman, E.A. and Ansell, E. (2002) ‘Orchestrating the multiple voices and inscriptions of a mathematics classroom’, Journal of the Learning Sciences, Vol. 11, Nos. 2/3, pp.251–274. Gardner, H. (2006) Multiple Intelligences: New Horizons, ISBN 0465047688, Basic Books, New York. Graham, S. and Harris, K.R. (2000) ‘The role of self-regulation and transcription skills in writing and writing development’, Educational Psychologist, Vol. 35, No. 1, pp.3–12, ISSN 0046-1520. Gruber, M., Glahn, C., Specht, M. and Koper, R. (2010) ‘Orchestrating learning using adaptive educational designs in IMS Learning Design’, Sustaining TEL: From Innovation to Learning and Practice, pp.123–138, Springer, New York. Hayes, J.R. and Flower, L.S. (1980) ‘Identifying the organization of writing processes’, Cognitive Processes in Writing, pp.3–30, Lawrence Erlbaum Associates, Hillsdale, NJ. IMS Global Learning Consortium (2003) IMS Learning Design v1.0 Final Specification. Jermann, P., Zufferey, G., Schneider, B., Lucci, A., Lépine, S. and Dillenbourgh, P. (2009) ‘Physical space and division of labor around a tabletop tangible simulation’, Proceedings of the 9th International Conference on Computer Supported Collaborative Learning, pp.345–349. Jurow, A. and Creighton, L. (2005) ‘Improvisational science discourse: teaching science in two K-1 classrooms’, Linguistics and Education, Vol. 16, No. 3, pp.275–297, ISSN 08985898, available at http://linkinghub.elsevier.com/retrieve/pii/S0898589806000179 (accessed on 19 November 2011). Karakostas, A. and Demetriadis, S. (2009) ‘Adaptation patterns in systems for scripted collaboration’, Proceedings of the 9th International Conference on Computer Supported Collaborative Learning, pp.477–481. Kennewell, S., Tanner, H., Jones, S. and Beauchamp, G. (2008) ‘Analysing the use of interactive technology to implement interactive teaching’, Journal of Computer Assisted Learning, Vol. 24, No. 1, pp.61–73, ISSN 1365-2729.
Orchestrating technology enhanced learning
597
Kobbe, L., Weinberger, A., Dillenbourg, P., Harrer, A., Hämäläinen, R., Häkkinen, P. and Fischer, F. (2007) ‘Specifying computer-supported collaboration scripts’, International Journal of Computer-supported Collaborative Learning, Vol. 2, Nos. 2/3, pp.211–224. Koper, R. and Tattersall, C. (2005) Learning Design: A Handbook on Modelling and Delivering Networked Education and Training, Springer, New York. Kovalainen, M., Kumpulainen, K. and Satu, V. (2001) ‘Orchestrating classroom interaction in a community of inquiry: modes of teacher participation’, Journal of Classroom Interaction, Vol. 36, No. 2, pp.17–28. McCutchen, D. (2000) ‘Knowledge, processing, and working memory: implications for a theory of writing’, Educational Psychologist, Vol. 35, No. 1, pp.13–23, ISSN 0046-1520. McCutchen, D., Covill, A., Hoyne, S.H. and Mildes, K. (1994) ‘Individual differences in writing: implications of translating fluency’, Journal of Educational Psychology, Vol. 86, No. 2, p.256, ISSN 1939-2176. Merriam Webster, Inc. (2011) Merriam-Webster online dictionary, April, available at http://www.merriam-webster.com/dictionary/orchestrate (accessed on 19 November 2011). Moran, S., Kornhaber, M. and Gardner, H. (2006) ‘Orchestrating multiple intelligences’, Educational Leadership, Vol. 64, No. 1, pp.22–27, ISSN 0013-1784. Natriello, G. (2007) ‘Imagining, seeking, inventing: the future of learning and the emerging discovery networks’, Learning Inquiry, Vol. 1, No. 1, pp.7–18, ISSN 1558-2973, available at http://www.springerlink.com/index/10.1007/s11519-007-0005-1 (accessed on 19 November 2011). Niramitranon, J. (2006) ‘COML (Classroom Orchestration Modelling Language) and scenarios designer: toolsets to facilitate collaborative learning in a one-to-one technology classroom’, available at http://halshs.archives-ouvertes.fr/hal-00190507/ (accessed on 19 November 2011). Niramitranon, J., Sharples, M., Greenhalgh, C. and Lin, C-P. (2010) ‘Orchestrating learning in a one-to-one technology classroom’, in Sharples, M., Greenhalgh, C. and Lin, C-P. (Eds.): 6th IEEE International Conference on Wireless Mobile and Ubiquitous Technologies in Education, IEEE, pp.96–103, ISBN 9781424464272, available at http://ieeexplore.ieee.org/ lpdocs/epic03/wrapper.htm?arnumber=5476543. Pachler, N., Daly, C., Mor, Y. and Mellar, H. (2010) ‘Formative e-assessment: practitioner cases’, Computers & Education, Vol. 54, No. 3, pp.715–721, ISSN 0360-1315. Pellegrino, J.W. (2010) ‘Perspectives on the integration of technology and assessment’, Journal of Research on Technology in Education, Vol. 43, No. 2, pp.119–134. Peltz, C. (2003) ‘Web services orchestration and choreography’, IEEE Computer, Vol. 36, No. 10, pp.46–52. Prieto, L.P. (2009) ‘An exploration of teacher enactment of CSCL activities in computer-integrated classrooms’, Master’s thesis, Escuela Técnica Superior de Ingenieros en Telecomunicaciones, University of Valladolid. Prieto, L.P., Villagrá-Sobrino, S., Dimitriadis, Y., Jorrín-Abellán, I.M., Martínez-Monés, A. and Anguita-Martínez, R. (2010) ‘Recurrent routines in the classroom madness: pushing patterns past the design phase’, 7th International Conference on Networked Learning. Prieto, L.P., Dimitriadis, Y., Villagrá-Sobrino, S., Jorrín-Abellán, I.M. and Martínez-Monés, A. (2011a) ‘Orchestrating CSCL in primary classrooms: One vision of orchestration and the role of routines’, Proceedings of the 9th international conference on Computer-Supported Collaborative Learning (CSCL 2011), Paper presented at the Workshop on How to Integrate CSCL in Classroom Life: Orchestration, available at http://www.gsic.uva.es/lprisan/ CSCL2011_WSOrchestration_Prieto_submission.pdf. Prieto, L.P., Villagrá-Sobrino, S., Dimitriadis, Y., Schank, P., Penuel, W. and DeBarger, A.H. (2011b) ‘Mind the gaps: using patterns to change everyday classroom practice towards contingent CSCL teaching’, Proceedings of the 9th International Conference on Computer-Supported Collaborative Learning (CSCL 2011).
598
L.P. Prieto et al.
Prieto, L.P., Villagrá-Sobrino, S., Jorrín-Abellán, I.M., Martínez-Monés, A. and Dimitriadis, Y. (2011c) ‘Recurrent routines: analyzing and supporting orchestration in technology-enhanced primary classrooms’, Computers & Education, Vol. 57, No. 1, pp.1214–1227. Richardson, V. (2003) ‘Constructivist pedagogy’, The Teachers College Record, Vol. 105, No. 9, pp.1623–1640, ISSN 1467-9620. Roschelle, J., Tatar, D., Chaudbury, S.R., Dimitriadis, Y., Patton, C. and DiGiano, C. (2007) ‘Ink, improvisation, and interactive engagement: learning with tablets’, Computer, Vol. 40, No. 9, pp.42–48, ISSN 0018-9162. Sha, L. and van Aalst, J. (2009) ‘A 2x3 model of student-directed formative assessment in collaborative knowledge creation’, Computers in Education, pp.205–212. SRI (2011) Group Scribbles official website, available at http://groupscribbles.sri.com (accessed on 19 November 2011). Stake, R. (2005) ‘Qualitative case studies’, in Denzin, N. and Lincoln, Y. (Eds.): The SAGE Handbook of Qualitative Research, pp.443–466, Sage Publications Inc., Thousand Oaks, CA. Stein, M.K., Engle, R., Smith, M. and Hughes, E. (2008) ‘Orchestrating productive mathematical discussions: five practices for helping teachers move beyond show and tell’, Mathematical Thinking and Learning, Vol. 10, No. 4, pp.313–340, ISSN 1098-6065. Tabak, I. (2004) ‘Synergy: a complement to emerging patterns of distributed scaffolding’, Journal of the Learning Sciences, Vol. 13, No. 3, pp.305–335. Vandergrift, L. (2003) ‘Orchestrating strategy use: toward a model of the skilled second language listener’, Language Learning, Vol. 53, No. 3, pp.463–496, September, ISSN 0023-8333, available at http://doi.wiley.com/10.1111/1467-9922.00232. Watts, M. (2003) ‘The orchestration of learning and teaching methods in science education’, Canadian Journal of Science Mathematics and Technology Education of Science, Mathematics and Technology Education, available at http://www.informaworld.com/ index/918899916.pdf. Wecker, C. and Fischer, F. (2007) ‘Fading scripts in computer-supported collaborative learning: the role of distributed monitoring’, pp.764–772, available at http://portal.acm.org/citation. cfm?id=1599600.1599739. Weinberger, A., Clark, D., Dillenbourg, P., Diziol, D., Sampson, V., Stegmann, K., Rummel, N., Hong, F., Spada, H., Mclaren, B., Brahm, T. and Fischer, F. (2007) ‘Orchestrating learning activities on the social and the cognitive level to foster CSCL’, Proceedings of the 8th International Conference on Computer Supported Collaborative Learning, pp.38–47, International Society of the Learning Sciences. Weinberger, A., Kollar, I., Dimitriadis, Y., Mäkitalo-Siegl, K. and Fischer, F. (2009) ‘Computer-supported collaboration scripts: perspectives from educational psychology and computer science’, in Balacheff, N., Ludvigsen, S., de Jong, T., Lazonder, A. and Barnes, S. (Eds.): Technology-Enhanced Learning: Principles and Products, Amsterdam University Press, Springer, New York.
