Responsive Design: principles for effective technological ... - CiteSeerX

Responsive Design: principles for effective technological support of collaborative education research Paper Presented at the Conference of the American Educational Research Association San Francisco, April 5th-11th, 2006

Richard Procter and Patrick Carmichael Centre for Applied Research in Educational Technologies, Cambridge University

Abstract The ‘Learning how to Learn’ project of the Economic and Social Research Council (ESRC) Teaching and Learning Research Programme in the UK has, over its four-year lifespan, employed a range of electronic tools in support of its work developing formative assessment practices in primary and secondary schools through a programme of development and research. These have been developed through an iterative process of ‘use-case analysis’ involving university-based researchers and teachers, and made possible by the involvement of software developers within the research team. This has allowed the development of a number of specialized tools (for data collection, analysis and user engagement in and with the work of the project). These have reflected the design and needs of all project members at different stages in its development, but have also been informed by a set of underlying design principles which have had application to other projects and research contexts.

Introduction: The Learning How to Learn Project1 ‘Learning How to Learn – in classrooms, schools and networks’ was a four year (2001-2005) development and research project within the ESRC Teaching and Learning Research Programme (TLRP). It sought to develop understandings of the organizational conditions of schools, and across networks, in which teachers are successful in developing pupils' knowledge and practices in learning and in learning how to learn through assessment for learning. This reflected our aspiration to bring insights from classroom-level and school-level research into closer alignment with one another. James, Pedder, Black, Wiliam and McCormick (2003) argue that such alignment offers our best chance of furthering understanding of effective learning, its nature, the teaching practices that promote it, and the kinds of professional learning and institutional conditions that help teachers to adopt new practices. These avenues of investigation constitute an enormous range of inquiry and, since earlier reviews of research (Black and Wiliam, 1998) indicate a ‘poverty of practice’, there was a perceived need to stimulate and support innovation through development. For these reasons, it was necessary to bring together a large team of academic staff and researchers with different areas of substantive and methodological expertise and willingness to take on a variety of roles. Previous research on assessment for learning had been relatively small scale, so another area of interest was to investigate the sustainability of innovations, which were developed in authentic settings with modest levels of support, as they were adapted and spread across teachers and schools. This required the team to recruit a sizeable sample of schools and to work in partnership with head teachers, school coordinators, ‘focal’ teachers who were identified for detailed study, and Local Authority (LA) advisers. By the time of its conclusion, the project had involved 17 academics and researchers from five Higher Education Institutions and 40 infant, primary and secondary schools in five LAs and one 'Virtual Education Action Zone' (VEAZ - a novel organizational form, based on the 'Education Action Zones' established in the UK but involving geographically distributed schools linked by electronic networks). An advisory group, including primary and secondary teachers and LA representatives, also supported the project. This number and range of participants, and the scope and locations of project activities, presented the project team with a number of challenges, not least the need to develop and maintain internal and external communications for various purposes Not only was it important to ensure that team members were aware of project progress in all its dimensions, in order to avoid fragmentation, omission and duplication of effort, but the highly integrated design required that different researchers in different sites for different purposes would need access to a centralised ‘data pool’. Alongside these internal requirements there were also the usual demands for information about the project, and its 1

The 'Learning how to Learn' project (ref: L139 25 1020) was directed by Mary James (University of Cambridge until December 2004, then at the Institute of Education, University of London) and co-directed, from 2002, by Robert McCormick (Open University). Other members of the research team were Patrick Carmichael, Mary-Jane Drummond, John MacBeath, David Pedder, Richard Procter and Sue Swaffield (University of Cambridge), Paul Black and Bethan Marshall (King’s College London), Leslie Honour (University of Reading) and Alison Fox (Open University). Past members of the team were Geoff Southworth, University of Reading (until March 2002), Colin Conner and David Frost, University of Cambridge (until April 2003 and April 2004 respectively) and Dylan Wiliam and Joanna Swann, King’s College London (until August 2003 and January 2005 respectively). Carmel Casey-Morley and Nichola Daily were project administrators. Further details are available at http://www.learntolearn.ac.uk.

2

outputs, from external audiences. The project team never underestimated the value of face-toface contact in distributed teams in order to develop both intellectual and social capital so monthly whole team meetings, twice yearly advisory group meetings and regular sub-group meetings were the norm. As time progressed so dissemination talks to potential ‘user’ groups also increased. However, the sheer number of communications and the quantity of the data meant that this was a prime case for use of electronic tools. It is important to stress that, from the outset, members of the project team saw the role of electronic tools not only as a means of supporting the management of the research activities, but also as a means of establishing links with teachers in participating schools, LA representatives and other 'publics' interested in the conduct and findings of project. The Project Schools: Some Important Early Findings As we set about selecting and developing the electronic tools to support the project, we had to take into account a set of early findings from schools involved in the project, including those that were members of networks and of the VEAZ. On the whole, these schools were well equipped with computers (the vast majority of which were connected to the internet) and teachers reported that they regularly used new technologies in their planning or teaching (nearly 80% of primary and secondary teachers reporting using the internet to find lesson resources and about half complete 'schemes of work'). However, in contrast, only about 20% reported using the World Wide Web to find information about development of teaching practices: clearly an important finding since this was the focus of project activities! In addition, there was little evidence of teachers contributing to networked resources (either at regional or local level) and few schools (even those in the Virtual Education Action Zone) reported establishing or using school-to-school electronic links. Another general pattern was that, even in those schools that had extensive electronic networks, electronic means of communication were generally used to supplement or reinforce face-to-face meetings, paper communications and telephone conversations. In one secondary school with nearly 650 computers connected to a school network, the ratio of pupils to computers was less than 2 and all staff had wireless laptops for planning, administration and communication; despite this, staff still met for daily 'briefings' and announcements made by email were also disseminated via newsletters, memos and notice-boards (these findings will be reported in more detail in a forthcoming paper). In the light of these findings, we realized that we could make no assumptions about: • • •

The ability or willingness of participating teachers to see an online resource such as a project website as an appropriate source of information about the development of teaching practices – the issue with which the project was most concerned! The likelihood of teachers contributing to a central 'resource bank', 'knowledge exchange' or other central resource, even if they had found it a useful source of information The likelihood of schools – even those in existing networks – sharing practice solely via electronic means

As such, we recognized the importance of employing a 'blended' approach to supporting and researching development activities in schools. This would need to employ online environments and email alongside telephone and face-to-face meetings, and, more importantly, to be sufficiently responsive and flexible that it could adapt to different school

3

contexts, staff preferences and changing circumstances. To have built a significant element of our research design or our plans for user engagement around a specific (and possibly a novel) technology might have 'raised the bar' for participation to the extent that individual teachers or whole schools might be precluded from participation in the project as a whole. So with this twin set of concerns (the needs of the project team and the need to engage with school-based participants) in mind, we engaged in a series of ‘use case analysis’ activities involving both the project team and the project advisory group, with its teacher and LA representatives (see Carmichael, 2003, for a full account of this process). Our central roles in the project, and our involvement in the full range of project activity (rather than, for example, being employed to develop software on a consultancy or contract basis) allowed a much closer association between the project design and its realization, on the one hand, and our development of software to support it. Rather than working to develop software in response to a specification document or from a project plan, we were party to and frequently involved in the discussions that generated them; when we encountered problems or decisions had to be made, we were able to consult project members, either individually or collectively. As the role of electronic tools became more central to the project, they – and the process by which they had been developed - were also subject to the scrutiny of the project’s advisory group. While participatory approaches are now becoming widespread in software design (Gottesdiener, 2002; Carmichael, 2003), this combination of iteration, flexibility and accountability goes further than most participatory design processes. It allowed us to develop a process of what we now characterize as ‘responsive design’ – responsive not only to immediate needs of the ‘users’ but also to their often domain-specific concerns, approaches and theoretical frameworks. Wenger, in his survey of software supportive of communityoriented behaviour suggests that solutions generally arise through a process of 'combination and convergence' (2001, 5); while we would not suggest that (even at the end of the project) we had reached a 'solution', the iterative and responsive process in which we were involved probably matched the needs of project participants more than any pre-determined strategy might have done. Responsive Design: our Principles The most obvious manifestation of the electronic tools we developed to support the project is its website (http://www.learntolearn.ac.uk). This has evolved over the life of the project to fulfil a number of roles including: • • • • •

the provision of information about the nature and scope of the project to both participants and the wider public; provision of classroom and staff development materials for use by LA and school coordinators involved in the project; data collection, storage and analysis provision of project management and data analysis tools for use by the research team; sharing exemplification of developing practice in classroom assessment.

Central to these roles is an innovative content management system that builds web pages and other types of outputs from data components held on a web server: a typical page viewed by a user might actually represent many small components embedded in a template. These components represent the basic units of communication within the project; the largest are characteristically documents constructed in other applications (such as Microsoft Word

4

documents and PowerPoint presentations) with the smallest being single bibliographical references, dates and times or email addresses. Underpinning our development were a number of design principles that allowed us to respond to specific and in some cases very detailed demands and specifications from both researchers and participating teachers and LA representatives. We will outline these below: a more detailed and rather more technical account is presented in Carmichael, Procter and James (2005). As the project developed, these principles allowed us to respond to changes in patterns of engagement and to adapt to the 'use cases' that project participants presented, paralleling the iterative, participatory approaches employed across the project as a whole. Once again, this is a novel approach in that the software tools and resources were developed alongside other elements of the project, rather than being developed and deployed from the outset. In this respect it closely resembles the relationship between 'technology' and 'community' described by Verwijs, Mulder, Slagter & Moelaert (2002) in which they argue that while initially at least “technology support should match the primary activities of the community, its size, goals, and the preferences of community members”, the availability of new technologies “may inspire [community members] to explore new interaction styles, thus improving the performance or efficiency of the community”(60-61). Design Principle 1: Small is Beautiful Small components (such as online diary entries – of which more shortly, bibliographical references, personal contact information, descriptions of examples of classroom practice) can be assembled in lots of interesting ways, and it is easier to aggregate content than disaggregate it In many cases, however, a fine balance had to be struck between maintaining meaningful units of analysis and allowing small components to be rearranged in new ways. With the information divided into its smallest descriptive components, we could develop software tools to organize, arrange, sort, search and display these components in a number of different ways. Design Principle 2: Embedded Markup Considered Harmful2 Separating content from formatting allows varied data to be presented in different forms and formats. Had online diary entries been stored as Microsoft Word documents, at some point we would have had to extract the text from its formatting so that it could be imported into qualitative data analysis software packages, for example. Similarly, rather than writing web pages within which content, structure and formatting information were combined, we stored all text content as fragments of XML (extensible markup language) which were then incorporated into page 'templates'. So, for example, individual user diary entries could be incorporated into individual diaries, per-school reports or cross-project reviews. What appeared to be whole new sets of functions, or significant changes within the website, were, in fact, relatively quick and simple to achieve. Design Principle 3: Reduce, Reuse, Recycle The componentisation of the web content and project data more generally also allowed us to maintain single, authoritative versions of content; they might be reused across the website in multiple formats, but only existed in one 'place' online. This overcame the tendency for individuals in collaborative groups all to have their own, slightly different, versions of 2

The title of an important paper by Nelson, T. (1997).

5

documents. We also applied this principle to the code we wrote to produce the web pages – wherever possible we reused program code and techniques, allowing us to build new templates and 'views' of data quickly in response to emerging user requirements. Design Principle 4: Conformance with International Standards Throughout our development work we have adhered to international standards for web resources, metadata, dates and times, and accessibility. This conformance means that not only are our applications capable of accessing other resources, but also that our website content can, where appropriate, be made accessible to a wider (even global) audience. This clearly has implications for research projects wishing to engage with the widest possible audience. The ‘responsive’ approach and the principles outlined above allowed us to address (or at least raise) a range of further issues relating not only to technical support but also to project concerns more generally. With the underpinning principles in place, we found it easier to address the 'use-cases' which the users of the website raised; they were not a substitute for involving project team members, but rather provided a basis upon which we could plan responses to (sometimes surprising) requests and suggestions. As project members themselves became more familiar with the principles underpinning our designs, they couched their requests in terms of rearranging components or providing different 'views' of them. Responsive Design in Action: Online Research Logs One of the key expectations was the engagement of key staff in participating schools (‘school coordinators’) both with and in the ideas and development activities of the project. To this end, each school had an assigned member of the project team whose role was to provide 'critical friendship' (Stenhouse, 1975; Costa and Kallick, 1993). This meant that we as a project we needed to develop strategies to support a triangular relationship involving project researchers, critical friends and school coordinators. At the beginning of the project, the expectation was that logs would take the form of structured Microsoft Word forms downloaded from the website; these would be used by school coordinators to record project activities and any school-based developments. At the same time, the researchers and critical friends would keep diaries (paper or electronic) of their interaction with schools, observations, administrations of research instruments and other information. School coordinators and 'critical friends' alike were concerned that this system was overly dependent either on collection of paper materials from schools and, more importantly, that it allowed little opportunity for dialogue between the different participants. An obvious solution was to use email, but this would have necessitated careful management to ensure that important research data was not lost; we also discovered that many of the school coordinators did not have personal email addresses as LA email systems allowed only a limited number of email addresses per school and barred staff from using webmail systems such as 'Hotmail' in school. Our response to this was to develop a web-based ‘log’ tool to which school and Local Authority coordinators within the project and Project members based in Higher Education institutions could contribute reports on the progress of the project in the settings in which they worked. The design of the logging tool reflected our design principles; log entries were stored as small components for aggregation (principle 1) which were devoid of any formatting information (principle 2). The log entries could be viewed individually, as part of a developing school record, or to help project managers maintain an overview of project 6

progress as a whole. When data analysis began, the school logs could be output in formats suitable for qualitative data analysis using packages such as Atlas/ti (principle 3). Log entry data was stored using well-established standards such as vCalendar (for date information) and Dublin Core (for broader descriptive information) and web pages produced were W3C compliant, in accordance with principle 4. Our early version was little more than a web form replicating the original word documents, with drop-down lists allowing users to select their name, the school name and some broad categories of activity (preliminary visits, in-service training, workshops, data collection, classroom observation etc), together with an area allowing 'free text' entry. By reading through the logs before any meeting with a school coordinator, the critical friend would be prepared and ‘brought up to speed’ about school-based initiatives or research activities in that school, allowing coordination of the development and research aspects of the project.

Figure 1: The 'Log' interface after revisions suggested by participants But how did this online tool respond to developing project requirements and user comments? Almost immediately, participants requested a number of additions and adaptations, most significantly the provision of separate spaces for 'reporting' activity and for more reflective accounts and the development of dialogue between the various individuals concerned (see Figure 1). Some participants used this to maintain links with critical friends and researchers: “School Coordinator still to be decided. Handbook has not been received - could we have one please?”(Log Entry by Head Teacher, School LDW, 12-07-2002)

Others used it to report on development activities and their impact, and to identify organisational factors and barriers to their progress:

7

“Meetings with Eng. Dept have resulted in discussion and review of Dept. Marking policy and, as a result, a revised marking policy, where pupils have greater control over the marking - peer marking implemented and trialed. The decision was made to not grade every piece but to give formative comments instead. Finally… pupils were given marking policy and key to have displayed and to use in front of books and a target sheet, to record targets set as a result of formative comment, to be kept in the back of the exercise books. Pupils only will be given a National Curriculum grade on a termly basis - this was a compromise after discussion with school.” (Log Entry by School Coordinator, School SEK, 18-10-2002) [Workshop] was generally very successful. However, some … staff still need to be made familiar with the basics of Assessment for Learning. Questionnaires will be done here within the next fortnight, probably after [the school inspection] … (Log Entry by School Coordinator, School WHE, 06-01-2003)

Others used the log effectively as a reflective diary to which they contributed regularly, either reporting on their experience of project activity or their own professional learning: “I found describing the class quite difficult and wondered why this was. Perhaps I think of them too much as 'levels' of learning due to the focus on assessment. We have had this discussion in the staff room. Are we thinking too much about children as a 1c or 2a? Also there are so many individuals within a class which you are focusing on as you teach. Do I ever think in 'whole class' terms? I'll keep a note of when I do.” (Log Entry by School Coordinator, School KNG, 21-05-2003) “So used to receiving feedback and discussing the lesson - found it strange just to give my opinion of how it went. I wanted to know how it went as far as [the researcher] was concerned and what [they were] focusing on. I wonder if children feel like this if we do not comment on their work.” (Log Entry by School Coordinator, School KNG, 21-05-2003)

For school-based participants in particular, the increased flexibility allowed them to engage in more depth with project activities, and 68% of all participant log entries comprised not only descriptive but reflective comments, although many reported organisational issues rather than professional learning issues. In contrast, researcher log entries were more likely to be descriptive accounts or reports of activities 'completed' (70%). Critical friends used the logs in a variety of ways and wrote both for a project audience and directly to school coordinators: “I was acutely aware that this is an infants school and the examples I had to hand were drawn from secondary practice. However, teachers were very ready to offer analogous examples from their own pupils. For example misunderstanding that children experience over the different meanings of words … The two examples of children's work brought by teachers …may be worth capturing as examples for the L2L website and to provide examples from work with very young children in relation to this workshop.” (Log entry by Critical Friend, School KNG, 30-09-2002)

They also used the log to offer advice, support and references to sources of information. The wide variation in the nature of the log entries in is keeping with the rather varied findings reported from other studies as to whether contribution to online environments encourages reflective thought – or even accounts of reflective practice. Maxwell et al. (2001, 10) report that while there was some evidence of university-based staff and other teachers developing ‘person-orientated’ critical friend roles in an online discussion space, there was, however, ‘little evidence of the critical analysis associated with the notion of “critical friend”’. In other cases, (for example, Hoel and Gudmundsdottir (1999)) where the purpose and parameters of self-criticism and analysis were, perhaps, more clearly defined, online environments seem to have been more effective at encouraging reflective writing. We were aware that some school coordinators and some critical friends chose not to use the logs, preferring to maintain paper diaries and communicate by telephone or email: 8

I’ve hopeless at filling in the document thing that we’re supposed to all log on. I’m terrible at doing that, I don’t know when I last filled that in. (School Coordinator Interview, GLI) I prefer to phone [the project researcher]. But if I can’t phone [them], then I use the email … or I put the basic “I need to know about…. give me a ring” in an email. (School Coordinator Network Interview, CCH)

We responded to this in two ways; we encouraged project team members to record telephone contacts where possible, and added, in a later version of the log allowed individuals to 'cc' emails to or from schools to a special email address; a small server-side programme periodically transferred these emails to the school logs to which they related, based on a three-letter code included in the email 'subject line'. Once the logs were developing, project members (particularly those with responsibility for several schools, and those involved in preliminary data analysis) requested that a set of management tools be built so that they could gain a quick overview of progress to date (including which parts of the project intervention had taken place, what data had been collected and dates of future visits and meetings) across a group of schools, across a whole LA or across the project as a whole. For a more complete account of the role of the logging tool and the data gathered using it, see Carmichael, Procter and Honour (2003). What we also discovered – both through the logs themselves and during school visits – was that those school coordinators who had made regular use of the logs found them a useful 'running record' of the progress of the project in their school: When I did a summary of everything we had done to show our new co-ordinator, I was amazed at the amount we have covered. It will need careful monitoring and support for the new members of staff to ensure that they can take on board the key practice. (Log entry by School Coordinator, School KNG, 05-01-2004)

Responsive Design in Action: Other Emerging Issues for the Project In retrospect, we were surprised by the extent to which the approaches we developed were not just useful in terms of rapid software development (as in the case of the online logs) but also became a focus for project-wide discussions that took place and threw some aspects of project design and management into high relief. The following are some examples of these broader issues to which we had to respond and where the associated discourse had wider implications across the project. Project Management and Informed Decision-Making At the mid-point in the project, the project team needed to select approximately half of the schools for full data collection and as possible case-study sites. This selection process involved classifying schools according to the likelihood that we would be able to gather nearcomplete data sets from them, based both on their record of involvement up to that point, and the views of researchers and critical friends. We were able to rapidly adapt the school ‘log’ tool, but rather than using it to produce a web page per participating school, a large organizational chart (rather like a project-specific GANTT chart) was generated. This was then used – together with other data – to inform the selection process. Reflecting Concerns about Privacy and Confidentiality The issue of ‘who’ should see ‘what’ on the website was initially conceived in terms of differential access for members of the project team, school-based participants and ‘the

9

public’. It soon became apparent, however, that this did not provide sufficiently ‘fine grained’ distinctions between users, nor did it allow complex patterns of access to be expressed. A key example related to the level of access Local Authority representatives on the project would have to individual school logs. Once this issue had been raised, a meeting of Local Authority staff and teacher members of the advisory group established a confidentiality framework. Our component-based approach allowed us to define (in the metadata of each component) who should see not whole data sets, but specific elements within them. This allowed the confidentiality framework to be applied almost immediately across the web site. In another development, a decision to publish a large set of project resources to schools other than those participating in the project was reflected on the website in a matter of minutes and without any large-scale reorganization. Analysis and Archiving As collection and analysis of data continued within the project, the issue of secure storage of data emerged as an important issue. Dialogue between researchers and developers continued as how best to structure, store and describe datasets, and particular requirements (such as the ability of researchers to retrieve data either by instrument or by research site) were addressed by our component-based approach. Our definition of ‘component’ was also extended to include transcripts, analytical accounts and summaries, images, quantitative data files and bibliographic references, as diverse data were uploaded to the website. This collection now extends to over 1000 resources each of which is, where appropriate, accompanied by a descriptive metadata file. With the funded period of project activity now at an end, a final phase of responsive activity has centred on restructuring web content in such a way that project researchers are able to continue their analysis, collaborative writing tasks and dissemination activities, while at the same time much of the project documentation, including presentations, workshops and reports have now been placed into the public domain. Content has been reformatted and incorporated into a book aimed at teachers interested in developing formative assessment practice in their own schools – illustrated with accounts, derived from online logs and interviews – of how participating project schools engaged in and with the project (James et al, 2006). Reflections On reflection, we can say with some confidence that the management of the project was made easier both by our capability to develop web resources and other software ‘in house’, and by the use of our ‘responsive’ approaches. For some teachers participating in the project, the provision of online support considerably enhanced their experience and provided them with ‘critical friendship’ during what was sometimes a challenging development process. The availability of project resources on the website increased the reach and impact of the project, particularly when combined with face-to-face contact through meetings or professional development activities. Viewed more critically, while some of the school coordinators had engaged fully with the project and used the online log tool to enter into sustained discussions with ‘critical friends’ or as reflective diaries, for others the project website was perceived as simply another available online resource – albeit one where the emphasis was on developing practice rather than on provision of curriculum resources. One aspect of practice that we were not able to develop to any extent was electronic schoolto-school networking and sharing of practice, an issue attributed to prevailing school structures (Cuban, Kirkpatrick and Peck, 2001; Wenger, 2001, 4). Participants in our project 10

also attributed this to competition between schools for resources, staff and examination results; to perceptions of schools being so dissimilar that even effective practice would not 'translate' to other contexts; to poor and incompatible communication systems (especially video conferencing); and to a lack of recognition amongst staff of the value of their professional knowledge, one head teacher suggesting: Teachers are dreadful at recognising, you know, their own professional ability … if you ask a teacher about their strengths and weaknesses, they always tell you the weaknesses straight off. Making people sound confident that they’re skilled or aware of what their skills are is actually essential because they need to analyse something and say, ‘What do I do well?’ which they probably do expand it or pass it on to other people. (L3 HT interview, SEK)

While we were aware of the development of professional knowledge and innovative practice in many project schools, this was generally communicated to other teachers at face-to-face events such as regular school coordinator meetings, after which some schools did, in fact, establish email communication and programmes of reciprocal visits. Alternatively, sharing depended on researchers, critical friends or LA representatives acting as ‘brokers’ of innovative practice between those project schools with which they had contact. In fact, some LA representatives saw this as a key role for them more generally: My role as an assessment advisor is to promote good assessment practice within the LEA whether that is around data and summative assessment or good assessment practice in the classroom … I am going into schools and picking up good practice that I can share with other schools (LA Coordinator Interview, JE) “What we want to try and do is actually get those schools talking to each other more electronically … What I would like to do at that meeting is to set up … that would begin email communication, so that they keep in closer contact with each other.” (LA Coordinator Interview, TG)

So in one respect, schools participating in the project conformed to Hargreaves' model of the 'knowledge creating school' in that they were taking part in a 'managed process of creating new professional knowledge' (2001) However, for a variety of reasons they failed to disseminate that knowledge systematically, or to have it validated by colleagues without the intervention of some kind of 'broker' (see Carmichael, Fox, McCormick, Procter and Honour, 2006 for further discussion of the role of brokers). In the absence of such processes, the dominant model of dissemination remains, in Hargreaves' terms, 'outside-in' (2001, 35) with schools as net 'consumers' rather than 'producers' of professional knowledge. Next Steps: Using the Sakai Virtual Collaboration Environment to Support the 'Knowledge Creating School' One outcome of our work within the Learning how to Learn project has been the extension of some of our online tools and our responsive approach to other projects within the Teaching and Learning Research Programme. A significant difference, however, is that we now employ an existing software platform to address many of the user needs we have identified – rather than developing 'home-grown' solutions as we did in Learning how to Learn. Our chosen platform is Sakai (http://www.sakaiproject.org), an open source community project that allows the configuration of 'worksites' within which sets of 'tools' can be arranged and combined (see Figure 2). The whole system is componentised with content being reusable across tools and worksites; content and formatting are separated; and international standards are applied – so this is an environment within which our commitment to responsiveness can be maintained and developed further.

11

Figure 2: A Sakai 'Worksite' showing multiple collaboration tools. We are currently evaluating Sakai as a platform for supporting social science research, including projects involving school-university collaboration, and have been able to apply the experience gained in the course of the Learning how to Learn Project as we support, develop and research the effectiveness of various online tools and configurations. With the development of 'wiki' and 'web log' tools for use within Sakai, the potential exists to support the kind of extended interaction between teachers, researchers, and critical friends we observed in our own log tool. At the same time it is possible to configure Sakai as an environment within which email is the primary locus of interaction; an 'email archive' and 'new contents' email alerting functions allowing participants to keep abreast of developments and contribute to discussion without any need to regularly log in to worksites. Amongst the projects now participating in our evaluation of Sakai, there are some who use the platform to use it to build central resources to support dissemination; those who use it to provide private, access-controlled discussion spaces where the focus is on interaction rather than documents; and those that use it primarily to support widely distributed groups who communicate mainly by email. Sakai's fine-grained permissions system allows the control of access to resources, and while this is clearly useful for reflecting the kinds of relationships between schools and LA's we describe above, it also allows the phased release of data to progressively larger groups; thus an individual can work privately before inviting others to contribute, and to comment on and validate their activity. This support for 'graded publics' is identified by Altrichter (2005, 22) as a characteristic element of much educational research, and provides a means by which the validation and dissemination of professional knowledge can be supported within and between schools. Furthermore, the flexibility of the platform means that initially it can be configured to replicate or reflect existing structures and working patterns, while offering opportunities to support new patterns of interaction and, potentially, new kinds of relationships between research participants. 12

Our experience of 'moving platforms' and encountering many of the same issues with Sakai as with our own web tools raises an important final issue. It strongly suggests that, while 'use case analysis' is important in identifying how groups of teachers might engage with new technologies, and our four principles are a helpful framework, the most important element of all in the development of effective technological support of collaborative research (regardless of technological platform) is the commitment to make the working practices and preferences of participants themselves an object of systematic enquiry. If we had not identified the low levels of IT use by teachers in participating schools, we might have made assumptions which might have prevented many from taking part in the activities of the project. If we had not engaged in a continuous process of consultation, we might have failed to engage those teachers who relished the opportunity to reflect in detail upon their practice and who had valuable lessons to share. And perhaps most importantly, if we had not provided alternative, non-electronic channels of communication and sites for networking, many teachers would have been denied the experience of participating in, validating and sharing their developing professional knowledge.

13

References Altrichter, H. (2005) 'The Role of the Professional Community in Action Research' Education Action Research 13(1) 11-26. Black, P. & Wiliam, D. (1998) Inside the black box: raising standards through classroom assessment. London, King’s College. Carmichael, P. (2003) 'Teachers as researchers and teachers as software developers: how usecase analysis helps build better educational software'. The Curriculum Journal 14(1) Spring 2003, 105-122. Carmichael, P., Procter, R. and Honour, L. (2003) 'Network Tools across a Distributed Research Project: Reflections on the first two years of 'Learning how to Learn''. Paper presented at the conference of the British Educational Research Association, Edinburgh, September 2003. Carmichael, P., Procter, R. and James, M. (2005) 'Using electronic tools to support a large development and research project: a case study ' Paper presented at TLRP annual conference, November 2005, University of Warwick, Thematic Session on 'Managing Large Projects'. Carmichael, P., Fox, A., McCormick, R., Procter, R. and Honour, L. (2006) 'Teacher networks in and out of school' Research Papers in Education 21(2) 217-234. Costa, A. and Kallick, B. (1993) ‘Through the Lens of a Critical Friend’, Educational Leadership, 51(2), 49-51. Cuban, L., Kirkpatrick, H. and Peck, C. (2001) High access and low use of technologies in high school classrooms: explaining an apparent paradox, American Educational Research Journal, 38 (4) 813-834. Gottesdiener, E. (2002) Requirements by Collaboration. Boston, MA: Addison-Wesley. Hargreaves, D. (2001) Creative Professionalism: the role of teachers in the knowledge society (London: Demos) Hoel, T. and Gudmundsdottir, S. (1999) ‘The REFLECT Project in Norway: interactive pedagogy using email’ Journal of Information Technology for Teacher Education, 8(1); p. 89-110. James, M., Pedder, D., Black, P., Wiliam, D., & McCormick, R. (2003) ‘The Design of the ESRC TLRP “Learning How to Learn” Project’. Paper presented at the 2003 Annual Conference of the British Educational Research Association. Edinburgh. James, M., Black, P., Carmichael, P., Conner, C., Dudley, P., Fox, A., Frost, D., Honour, L., MacBeath, J., McCormick, R., Marshall, B., Pedder, D., Procter, R., Swaffield, S. & Wiliam, D. (2006) Learning How to Learn: tools for schools. TLRP 'Improving Practice' Series. London, Routledge.

14

Maxwell, T.W., Reid, J. McLoughlin, C., Clarke, C and Nicholls, R. (2001) ‘Online support for Action Research in a Teacher Education Internship in rural Australia’ paper presented at the Society for the Provision of Education in Rural Australia Annual Conference, Wagga Wagga, 8-11 July, 2001. Nelson, T. (1997) ‘Embedded Markup Considered Harmful’ World Wide Web Journal 2(4), 129-136. Stenhouse, L. (1975) An Introduction To Curriculum Research And Development. London: Heinemann Educational. Verwijs, C., Mulder, I., Slagter, R., & Moelaert, F. (2002) ‘Leveraging communities with new technologies’ In J. H. E. Andriessen, M. Soekijad, & H. J. Keasberry (Eds.) Support for knowledge sharing in communities (Delft, The Netherlands: Delft University Press), 57-70. Wenger, E. (2001). 'Supporting communities of practice: A survey of community-oriented technologies'. Online at: http://www.ewenger.com/tech

15