The International Journal of Management Education 11 (2013) 119–131
Contents lists available at ScienceDirect
The International Journal of Management Education journal homepage: www.elsevier.com/locate/ijme
An agile method for teaching agile in business schools Marija Cubric Hertfordshire Business School, University of Hertfordshire, UK
a r t i c l e i n f o
a b s t r a c t
Article history: Received 12 October 2012 Received in revised form 6 September 2013 Accepted 11 October 2013
The aim of this paper is to describe, evaluate and discuss a new method for teaching agile project management and similar subjects in higher education. Agile is not only a subject domain in this work, the teaching method itself is based on Scrum, a popular agile methodology mostly used in software development projects. The method is supported by wikis, a natural platform for simulation of software development environments. The findings from the evaluation indicate that the method enables the creation of “significant learning”, which prepares students for life-long learning and increases their employability. However, the knowledge gains, resulting from wiki interactions are found to be more quantitative than qualitative. The results also imply that despite the active promotion of agile values of communication and feedback, issues regarding the teamwork are still emerging. The engagement of the teacher in the learning and teaching process was discovered to be a motivational factor for the team cohesion. This paper could be of interest to anyone planning to teach agile in the higher education settings, but also to a wider academic community interested in applying agile methods in their own teaching practice. Ó 2013 Elsevier Ltd. All rights reserved.
Keywords: Agile CSCL Experiential learning Project management Scrum Wikis
1. Introduction Agile is an “umbrella” term used for a set of closely related software development and project management methodologies, such as Scrum, eXtreme Programming (XP), Kanban, Dynamic Systems Development Method (DSDM) and others. Agile methodologies are based on the values and principles from the “Agile manifesto” (2001), a statement published by a group of leading software practitioners in response to the growing concerns about the failure rate of IT projects (Standish Group, 1994). The failures were mainly attributed to the traditional, process- and documentation-heavy methodologies. The purpose of the Manifesto was to establish an alternative platform for delivery of successful projects, where the emphases is on: “individuals and interactions over processes and tools, working products over comprehensive documentation, customer collaboration over contract negotiation and responding to change over following a plan” (Agile Manifesto, 2001). The most popular agile methodology is Scrum (Schwaber & Beedle, 2002), a lightweight project management framework based on the empirical process control model (Ogunnaike & Ray, 1994) and characterised by frequent iterative and incremental inspection and adaptation. In Scrum, all software features are grouped initially in a “product backlog”. Each development iteration (called “sprint”) starts with the planning meeting where the “product owner” (PO) decides which of the features will be included in the next sprint. All sprints are of equal duration, usually 2 weeks, but they could be shorter or
E-mail address:
[email protected]. 1472-8117/$ – see front matter Ó 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.ijme.2013.10.001
120
M. Cubric / The International Journal of Management Education 11 (2013) 119–131
longer depending on the project. Teams are “self-organising” and during a sprint, the team members work on development of the selected features and coordinate their work through “daily stand-up” meetings. Each sprint ends with a review meeting, where the PO gives feedback to the team on the features developed during the sprint. The “Scrum master” is a member of the team responsible for maintaining the development process and insuring that the team has everything it needs to work effectively. XP (Beck, 2001; Beck & Andres, 2004) is another popular agile methodology, which focuses on software development practices recommended by the leading practitioners: planning game, small releases, coding standards, stories, automated testing, pair-programming, test-driven development, continuous integration, collective code ownership, automated builds, refactoring etc. Both, practitioners and researchers are reporting on many benefits of the agile approach. Organisations are claiming that agile leads to improved time-to-market, increased quality, reduced waste, better predictability and better team moral, although not all of this is supported by empirical evidence (Devine, 2008; Erickson, 2005; Schwaber, Leganza, & D’Silva, 2007). The empirical studies report on benefits of agile in the areas of customer collaboration, work processes for handling defects, learning, thinking ahead for management, focusing on current work and effort estimation for developers (Dybå & Dingsøyr, 2008). These widely recognised benefits are resulting in an increase in popularity of agile methodologies, and have earned them the status of the mainstream software development approach. This was confirmed in the 2010 Forrester report on agile adoption (West & Grant, 2010), where 35% of the survey respondents stated that agile most closely reflects their development process, with the number increasing to 45% with an extended definition of agility. The most recent VerisonOne (2012) annual survey on the state of agile, provides further evidence than “agile is not a fad”: 84% of 4048 survey respondents said that their organisation were practicing agile development, 50% worked in companies that have been practising agile for more than 2 years, 48% worked in companies that have used agile across 5 or more teams, and 41% of companies used agile in 6 or more projects. Given the popularity and the scale of agile adoption, it is not surprising that the industry is seeking more and more graduates with skills in agile and that the demand frequently outpaces the supply (Taft, 2012). According to a recent empirical study on challenges of agile, the lack of suitably trained IT graduates is one of the main problems in the industry today: “We cannot seem to find any graduates who have done anything hands on or even gone beyond one or two lectures on agile methods” (a manager in company “L” quoted in Conboy, Coyle, Wang, & Pikkarainen, 2011:56) The demand for agile practitioners is not restricted only to software industry. According to the Project Management Institute there is a growing demand in organisations for an agile approach to project management (PMI, 2013). However, the lack of suitably trained agile practitioners is even more prominent problem in the project management area, as agile is only occasionally mentioned in business school curriculums. The importance of teaching agile at business schools is further confirmed by recent initiatives on “agile business transformation” in organisations such as BT and IBM (Grout & Bonham, 2012), which are aiming to increase the use of agile techniques across the whole business. In addition to that, an increasing number of non-IT organisations are embedding agile principles and techniques in their practice. The latest examples are: the UK Government (NAO review, 2012), US Government (GAO report, 2012), banking sector (Sarran, 2012) and other highly regulated environments such as the pharmacological industry (Fitzgerald, 2012). This paper reports on experiences from the Hertfordshire Business School, where Agile Project Management has been taught since January 2007 as a separate one-semester module within the Masters programme in Project Management. One of the imperatives in designing the module was to provide students with “hands-on” experience in agile development, as it was felt that the subject is very practical and that only through an experiential approach can students gain knowledge and understanding of “agility”. This in turn presented the main difficulty in the module design, due to the diversity of students’ educational and work experience. It was not obvious what the relevant common factor of students’ domain knowledge was that could serve as a foundation for the “hands-on” learning activities. In other words, students could not simply be asked to develop software, as only a few of them would know how to do it! The “highest common factor” turned out to be the subject they were studying i.e. the agile project management. Hence, the solution was to use the module coursework as a “project” paradigm, where the students were split into small project teams, and guided in how to develop their project in an “agile “ way. In order to simulate the software development environment, wikis were chosen as a development platform for the project. The perceived benefits of adopting an agile approach for teaching agile can be linked to the experiential learning theories of Dewey, Kolb and Piaget, which all state that knowledge develops as a result of direct experience, as well as to Chickering and Gamson’s (1987) principles of good teaching practice which are closely related to the values from the Agile Manifesto. More discussion on this will follow in the Literature review section. The initial solution was first used in the 2007/08 academic year and since then it has been tailored to its present form in four successive academic years through a cyclic process based on Design Science Research (DSR). Prior to 2010, student feedback from the module evaluation and informal discussions indicated that students value regular and prompt feedback, teacher’s interest and enthusiasm for the subject and links with practice. It also indicated that student learning was enhanced with constant review of the subject and early engagement with the module. However, the results (grades) did not fully match student satisfaction with their own learning. Another problem was that cooperation and teamwork remained a problem for some students.
M. Cubric / The International Journal of Management Education 11 (2013) 119–131
121
Table 1 A comparison between Agile Manifesto and Chickering and Gammson’s principles. Agile Manifesto values
Interpretation in L&T context
Chickering and Gammson’s principles
Individuals and interactions over processes and tools
Respecting individual needs and learning styles of students and promoting interactions amongst students and between staff and students
Working software over comprehensive documentation Customer collaboration over contract negotiation
Increasing employability of students and empowering them for life-long active learning Supporting students outside of contact hours, and beyond the prescribed learning objectives
Responding to change over following a plan Deliver working software frequently, from a couple of weeks to a couple of months
Acting quickly to support students’ learning progress and adjusting learning content to address the needs of learners Providing regular and frequent assessment points with the emphasis on formative feedback
Respects diverse talents and ways of learning. Develops reciprocity and cooperation among students. Encourages contacts between students and faculty. Communicates high expectations Encourages active learning Encourages contacts between students and faculty Gives prompt feedback Communicates high expectations Gives prompt feedback Communicates high expectations Gives prompt feedback Emphasizes time on task (*)
The aim of this paper is to assess the impact of the agile Learning and Teaching (L&T) method on student learning and to gain an insight into their understanding of the factors influencing effective teamwork. This paper focuses on the findings from the previous two academic years (2010/11 and 2011/12), which are based on primary data from student surveys and with secondary data from module feedback, wiki usage statistics, student reflections and performance data (grade percentages). 2. Literature review This section contains a review of the relevant literature on teaching agile in higher education settings and the specific literature on learning models and theories which were used to inform the design of the agile L&T method. 2.1. Teaching agile The first systematic review of literature on agile software development published by Dybå and Dingsøyr (2008) covers 36 empirical studies reported by 2005. While the review focus is on the benefits, limitations, and strength of evidence for agile methods, it also includes some interesting findings on students’ perceptions on teaching agile software development. The studies included in the review reported high student satisfaction and enthusiasm with regards to core agile practices, and their positive impact on employability and development of professional skills required by industry. Similar findings were re-iterated in the subsequent literature from 2006 onwards, which also concentrates mainly on the Computer Science and Engineering curriculums, and in particular on experience and lessons learned from teaching agile and individual agile practices (Devedzic & Milenkovic, 2011; Johnston & Johnson, 2010; Lu & DeClue, 2011; Rico & Sayani, 2009); benefits of teaching agile (Coupal & Boechler, 2007; Hazzan & Dubinsky, 2007), difficulties and ways of overcoming them while teaching agile (Lynch et al., 2011). The literature on teaching agile project management is just starting to emerge. For example, Mahnic (2012) reports on lessons learned from teaching Scrum and project planning to software engineering students, concluding that teaching agile is best done through project and practical work and emphasising the role of “product owner” in the teaching process. All of the aforementioned findings are based on students who are software engineering-literate and therefore the learning activities suggested were all based on coding and software development. Satzinger, Batra, and Topi (2007) reported from the Americas Conference on Information Systems (IS) that a panel of 50 IS professionals and educators attending the conference agreed that teaching agile methods within the IS curriculum without focusing on coding is a significant problem. The author of this paper encountered the same problem in teaching agile within the project management curriculum. The literature on teaching agile to project managers is restricted to experience reports from short-term training and coaching of project management practitioners (Griffiths, 2005) with no references to teaching agile in a higher education business school curriculum. This paper attempts to address the current gap in the literature, and to provide some initial insight into the methods used, as well as related benefits and issues in teaching agile to project managers and to those outside the Computer Science curriculum. 2.2. Learning theories The design of the underlying agile L&T method was informed by several influential learning theories, all of which exhibit certain degree of similarity with the agile domain.
122
M. Cubric / The International Journal of Management Education 11 (2013) 119–131
Chickering and Gamson’s (1987) seven principles for good practice in education published in 1987 remain to this day a very useful and popular tool for informing curriculum design. The selection of principles for guiding the design of the agile L&T method was based on their simplicity and practicality. But also because they exhibit a high degree of similarity with the values from the Agile manifesto (Table 1). All but one principle can be related to one of the values from the Manifesto, with an appropriate interpretation in the learning and teaching context. The only exception is the “time-on-task” principle (* in Table 1), which does not have a direct match in the Manifesto values, but it can be easily mapped into the agile principle of frequent and regular software delivery (http://agilemanifesto.org/principles.html). Dalton and Tharp (2002) argued that the principles are incomplete and suggest additional requirements such as that teacher and students should join in productive activities. This criticism has been considered in the design of the agile L&T method, where the teacher has assumed the role of the “product owner” and has worked with the student teams for the duration of the coursework project, within the capacity defined by the role. The agile L&T method considered in this paper is based on the following premises: Learning results from concrete experience Learning is a collaborative process. Relevant to the premises are the theories of experiential learning and collaborative learning. The experiential learning theories (ELT) of Lewin, Dewey, Piaget and Kolb are particularly pertinent to this work, as they all state that knowledge develops as a result of direct experience. Learning is defined as “a process whereby knowledge is created through the transformation of experience” (Kolb, 1984: 38). The ELT’s four stage cyclic learning model (Kolb, Boyatzis, & Mainemelis, 2001) provides a suitable framework for the agile L&T method, because of its iterative and incremental nature:
Concrete experience (CA) – forms the basis for reflection Reflective observation (RO) – reflecting on previous experience Active conceptualisation (AC) – transforming reflections into domain knowledge concepts Active experimentation (AE) – formulating new questions and planning new experience.
It can be easily observed that iterative and incremental software development processes follow a similar pattern, where the AC, AE, CA and RO stages of the learning cycle correspond to the software development activities of coding, creating test cases, testing, and reviewing code respectively. In agile software development the focus is on early “concrete experience” and short cycles, in other words, on early and frequent delivery of software (http://agilemanifesto.org/ principles.html). Collaborative learning (CL) is a learning process that takes place within a group and as a result of group interactions. Although there is no single theory of CL, the current literature often identifies it with the Vygotsky’s theory of social development (1978), a constructivist developmental theory which states that learning does not occur in isolation, but under a guidance from an adult or in collaboration with more capable peers. In the educational context, the “adult ” and “more capable peers” are usually interpreted as teacher and other students. Other benefits of CL regularly cited in the literature include increased information retention, enhanced critical thinking of learners etc. In the review of CL research, Dillenbourg (1999) points out that the benefits of collaborative learning occurring in one context cannot be directly applied in a different context and he further proposes three defining variables (“dimensions”) for collaboration: scale of collaboration, type of learning, and type of collaboration. All findings, according to Dillenbourg, should be situated relative to the three variables. Collaboration and interaction play a crucial role in agile development where business people and developers work together daily throughout the project, with the preference for face-to-face communication (Agile Manifesto, 2001). However, even in agile teams, communication can be a problem, as reported by practitioners and research. According to the VersionOne (2012) survey, 58% respondents stated that teamwork and communication-related issues are the main organisational problems behind any agile project failure. Similar findings are re-iterated in empirical research on agile, where the evidence from the mature agile teams suggests that it is necessary to focus on human and social factors in order to succeed (Dybå & Dingsøyr, 2008). The collaboration in agile teams take place through various practices such as pair-programming, collective code ownership, planning, review and daily meetings. According to Whitworth and Biddle (2007a) planning activities were noted as especially valuable in reducing the tension and conflict in agile teams. They also found that motivation and cohesion in agile teams are strongly linked to clarity of objectives, ease of interaction, and frequent iterative deliveries (Withwort and Biddle, 2007a). Based on the use of wikis, the agile L&T method could be classified to fall under the “computer-supported collaborative learning” (CSCL) and “blended-learning” areas. CSCL as a branch of learning sciences concerned with how people learn together supported by computers (Stahl, Koschmann, & Suthers, 2006) while blended-learning is defined as a thoughtful integration of face-to-face and online learning to optimise student engagement (Garrison & Vaughan, 2007). Garrison, Anderson, and Archers’s (2000) model of blended learning, known as “community of inquiry”, emphasises the importance of social, and teaching presence elements in addition to a cognitive element in a blended learning environment.
M. Cubric / The International Journal of Management Education 11 (2013) 119–131
123
The use of wikis in learning and teaching is not new, and many studies have been done on their use in higher education and on the benefits of so-called “wiki-supported collaborative learning” (Bocconi & Trentin, 2012). The primary purpose of using wikis in the agile L&T method is that they provide a natural platform for simulation of software development tools. These tools, known as “Integrated Development Environment” (IDE), are characterised by the existence of mechanisms for allowing concurrent updates from different users and for the version-control of the content, both of which wikis possess. These two mechanisms also enable “continuous integration”, which is one of the cornerstone practices of agile development, and which states that “at the end of each development episode the code is integrated with the latest release” (Beck, 2001:97). In other words, developers do not wait for the end of the project to integrate their work, but they do it on a daily basis i.e. whenever they have something good enough to integrate. Another useful characteristics of wikis for the use in the agile L&T method is the visibility of individual contributions, through the “History” mechanism. This property is desirable, as according to Whitworth and Biddle’s (2007b) research, regular team awareness of individual activities has a positive impact on the team cohesion. In the case of agile L&T method, the use of computers and the “blend” of activities were both driven by the specific requirements of the agile development process, rather than by the perceived benefits of technology-enhanced learning. Therefore, neither CSCL nor blended-learning per-se are the main focus of this paper. Cress and Kimmerle’s (2008) “co-evolution model” explains the type learning that is taking place with wikis, where knowledge is developed through “co-evolution” of wiki-based interactions and cognitive processes of learners. They distinguish three types of wiki interactions: reading, adding new information and re-organising existing content. They claim that these interactions trigger the cognitive processes of knowledge internalisation, assimilation and accommodation respectively. The latter is especially important for the creation of higher-order qualitative cognitive gains, while the first two lead only to quantitative increases in knowledge. Majchrzak, Wagner, and Yates’ (2006) study on corporate wiki users categorises the users into three groups: “adders”, “synthesizers” and “commentators”, based on the frequency of the corresponding actions: adding new text, integrating and re-organising the existing content or commenting on it respectively. Although their study was conducted in the context of organisational wikis, this classification can be applied to an educational context as well, as these roles are closely related to different learning styles of students. Other theories and ideas related to “wiki-supported collaborative learning” (Bocconi & Trentin, 2012) include Papert’s (1991) and Holmes, Tangney, Fitzgibbon, Savage, and Mehan’s (2001) “learning by making” and Dee Fink’s “significant learning” (2003).
3. Agile Learning and Teaching method The agile L&T method under consideration is based on Scrum (Schwaber & Beedle, 2002) and it is supported by additional XP practices (Beck, 2001; Beck & Andres, 2004) which are directly applicable to the L&T domain, such as small releases, coding standards, pair-programming, continuous integration, collective ownership, and refactoring. The project task is to develop an online wiki-based encyclopedia of Agile Project Management (“agilopedia”) using the knowledge gained in class and that obtained through independent study and reading. The full coursework project specification contains more details regarding the number of articles and standards for structural and presentational aspects of the work, which was in line with the XP practice of “coding standards”. The public nature of “agileopedia” is supported by the constructionism’s view of effective learning which happens in the context where “the learner is consciously engaged in constructing a public entity, whether it’s a sand castle on the beach or a theory of the universe” (Papert, 1991) and where students engage in creating knowledge that can benefit others (Holmes et al., 2001). Wikispaces (www.wikispaces.com) is selected as a development platform (“IDE”) because of its simplicity and the existence of specific features for educators. The “product backlog” consists of a number of topics for the “agileopedia” set by the module leader at the beginning of the semester. Each topic is supposed to be developed on a single wiki page, in a style similar to wikipedia. According to the XP practice on “small releases”, the whole project is scheduled to complete in 5 iterations (“sprints”), with each sprint lasting 2 weeks. In addition to the main development artifact (wiki site) students are asked to submit sprint reports (one per team) specifying various statistics of their sprint work, such as number of edits, number of discussion posts, number of hyperlinks, TurnitIn similarity index etc. Students are divided into groups of 4–5 by self-selection and these groups act as Scrum “teams”. Because of the selfselection, teams are not entirely “cross-functional”, as recommended in Scrum (Schwaber & Beedle, 2002). Students are asked to all work as “developers” and to designate one team member as the “Scrum master” in each sprint. The role of the Scrum master, just like in the Scrum, is to ensure that the team works efficiently through the sprint and to prepare progress reports at the end of each sprint. For the purpose of the project, the role of the teacher is to act as a “product owner” for each of the Scrum teams. The PO is responsible for managing and controlling the product backlog (Schwaber & Beedle, 2002). That involves selection of topics to be included in each sprint (“sprint backlog”) and feedback to the teams at the end of each sprint. Sprint
124
M. Cubric / The International Journal of Management Education 11 (2013) 119–131
backlog is published every two weeks, after the lecture, and includes a list of topics considered to be of “high priority” by the PO. At the beginning of a sprint, the team selects one or two topics from the sprint backlog. This is done on a “first-come-firstserved” basis. During the sprint-planning meeting, the team decides on the allocation of tasks for the sprint. Tasks are development activities specific to the L&T domain and include: literature search, wiki editing, wiki “gardening”, submitting article to TurnitIn, writing progress report, restructuring the work from the previous sprint based on the teacher’s feedback etc. The latter one corresponds to the XP practice “refactoring”. Team members volunteer for the tasks rather than being allocated to the task by the Scrum master. During each sprint, the team work on the selected topic(s) and that work might involve use of XP and Scrum practices such as “pair-programming”, where two members of the team work together on a single article at the same time; or “daily stand-up” meeting, a short 10 min meeting where the team reports on progress and impediments. The XP practices “collective ownership” and “continuous integration” are an inherent part of the method, as each student can work on any articles owned by their team, and the submission of wiki updates is happening on a daily basis rather than at the end of the sprint. At the end of a sprint, the Scrum master prepares and submits the progress report. The work of each team is reviewed by the teacher (the “PO”) who also provides the feedback in a form of wiki page annotations, as well as face-to-face, outside of the lecture time (“sprint review”). The resulting learning and teaching process is shown in Fig. 1. Table 2 illustrated the mapping between the Scrum artefacts and L&T concepts which are used in the agile L&T method, where the values that can vary are shown between “” signs. 4. Research methodology The framework for this study is practice-based research, rooted in Design Science Research (DSR). The DSR is an increasingly popular method in the fields of education, health care, computer science and engineering (Hevner, March, Park, & Ram, 2004; Vaishnavi & Kuechler, 2004). The method allows a researcher to shift between the design of an artifact and its evaluation, through a cyclic process where the new knowledge is constructed in each design cycle. The constructed knowledge is about the resulting artifact and about the process of developing it. The artifact under construction in this research is the agile L&T method, described in Section 3. The method was deployed within the Agile Project Management module and tailored to its present form in five consecutive academic years (2007/08– 2011/12), through various interventions aimed at improving the performance and quality of the method, and subsequently benefiting the students studying the module. For the purpose of this paper only the data from the previous two academic years (2011–12) are considered, corresponding to two snapshots in time: January–May 2011 and January–May 2012, each one representing a different DSR “cycle”. The Student group consisted in total of 50 Master-level students, 24 in 2011 and 26 in 2012. Amongst them, 70% were Business School students and the rest were Computer Science students; 73% were male and 18% were part-time students. A high proportion of students (70%) were international students from eight different countries including Nigeria, India and China. The two cohorts were comparable in size and gender characteristics, but they differed slightly in the proportion of part-time (P/T) students (none in 2011 and 9% in 2012) and participation from Computer Science (11% in 2011 and 4% in 2012).
Fig. 1. An illustration of the Agile Learning and Teaching process.
M. Cubric / The International Journal of Management Education 11 (2013) 119–131
125
Table 2 A mapping of artefacts between Scrum and L&T domains. Scrum domain
L&T domain
Product Owner (PO) Scrum Team Scrum Master Development platform (“IDE”) Product Product backlog Sprint Sprint backlog Task
Teacher A group of students A student (rotating role within the team) A wiki platform such as Wiki-based encyclopaedia of A list of topics for wiki articles -long development of wiki articles Selected set of topics for the wiki articles A learning activity e.g. reading, research, wiki-editing, wiki gardening, bibliography etc.
In this paper the two cohorts are considered as a single group, since the L&T method used with both cohorts was the same, with the exception of the format of the online feedback. This in 2012, was done via direct wiki annotations, rather then copying a wiki page into a MS Word document and then adding comments to the document.
4.1. Evaluation The evaluation of the artifact under the construction (method described in Section 3) is supported by quantitative and qualitative data collected through students’ surveys and various secondary data sources. The mixed-method approach to data analysis was used in order to corroborate the research findings, but also to enable findings which were not initially anticipated (Bryman, 2006). 4.1.1. Students’ survey The survey was composed of 43 items, including questions on previous work and educational experience, characteristics of the project groups, frequency of wiki contributions, type of wiki contributions, impact on learning, and factors influencing group cohesion. The design of the questionnaire was informed by the work of: Whitworth and Biddle (2007a,b) and Majchrzak, Wagner, and Yates (2006). The questionnaire was made available online via the Bristol Online Survey service at the end of the corresponding teaching semester. A prize draw for Amazon vouchers was offered as an incentive for students to participate. The student participants were self-selected. The total number of respondents was N ¼ 34, a response rate of 67% with a slightly better response rate in 2011 (N ¼ 19.79%) compared to 2012 (N ¼ 15.58%). The survey data were analysed using the SPSS v.19. The type of wiki contributions (Table 4), impact on learning (Table 5), and factors influencing group cohesion (Table 6) subscales of the questionnaire all had high reliability, with Chronbach’s a between 0.83 and 0.94. 4.1.2. Secondary data Secondary data used in this paper include wikispaces usage statistics, student grades and qualitative data from students’ reflections and module feedback. Wikispaces usage statistics were used to illustrate the engagement of students. They included number of edits per individual student as well as number of edits per day. The data was exported from wikis in a CSV format and imported to SPSS for further analysis. Students’ grades were calculated as a weighted aggregate of the wiki projects (30%), reflective report (30%) and individual presentation (40%). The grades could be correlated only to the wikispaces statistics, but not to the survey results, because of the anonymous nature of the survey. As mentioned above, the individual reflective reports were part of formal assessment. Students were asked to reflect on the experience gained and lessons learned through the development project, and more specifically, to identify and evaluate specific agile principles and practices used by their team, and to discuss what worked well (and why) as well as what difficulties were encountered by the team and how they could be overcome in future. The consent for using anonymised quotes from the reflections was sought after the publication of the module grades. No student emailed back with any objections. A total number of 48 reports were submitted, 24 in each year. A standard thematic analysis was used for analyzing the data from the reflections. A total of 426 items (quotes) were coded into 19 themes, including issues, benefits, lessons learned, agile practices used, teamwork, high expectations, time on task etc. The focus of this paper is mainly on those items from the reflections which are related to aspects of teamwork and learning, but other items are included also when necessary to illustrate the findings. Qualitative data considered here also include 12 items of anonymous student feedback, submitted within University-wide module evaluation survey.
126
M. Cubric / The International Journal of Management Education 11 (2013) 119–131
4.2. Limitations This study is based on a small population (50 students), which despite a high response rate (67%), led to a small set of data for analysis (34 survey responses). Demographic data were not included in the survey questionnaire as they were not considered relevant for this study and also to reduce the size of the questionnaire. However, this has prevented correlation of demographic attributes with the rest of the data, which might have provided some interesting findings related to the large proportion of international students in the cohort and to the students’ performance. The reflective reports were not anonymous as well as being assessed, which might have introduced a certain level of bias in the qualitative data. 5. Findings and discussion To simplify the presentation in this section, a union of findings from the last two DSR cycles (2011 and 2012) is unveiled, rather than the results of the individual cycles. This is further justified by the results the independent t-test which found only 3 items from the survey to exhibit small but statistically significant differences (p < 0.05) between the two cohorts (see Appendix A for details). Nonetheless these differences are considered sufficiently small and are not essential for the findings reported in this section. Quotes labelled with “R” or “MF” correspond to items from Reflections and Module Feedback respectively. 5.1. Previous experience of participants Majority of students rated their IT literacy skills as high (84.30%), team-working skills as average (44.10%) or high (50%), and academic writing skills as average (64.70%). Responses were polarised with respect to satisfaction with previous groupwork experience: low (20.60%), average (52.90%) and high (26.40%). 5.1.1. Initial “ends” and “means” uncertainty Seventy one percent of students rated their knowledge of agile and experience with wikis as low or very low. Therefore not only that the subject was new to majority of students, but also the tools and the process for developing the coursework were new, indicating high initial “ends” and “means” uncertainty, just like in “real” agile software development teams (Cohn, 2005:83). This was confirmed with the data from students’ reflections, in which 12% of all items coded as issues were related to initial uncertainty, confusion and anxiety due to the lack of knowledge in the first iteration (sprint). The iteration required us to play around with the wiki in order to have a prior knowledge of how it works. This looked very easy but I must confess that it was one of the most challenging iterations because we were still at the “Forming” stage . and decisions had not been made on how we were going to approach the task and complete the group report. In addition, none of us had the experience of using wikispaces . (R-2012) Although this early “concrete experience” (Kolb et al., 2001) has created a feeling of uncertainty for some students, it has also increased an overall sense of “urgency” (Whitworth & Biddle, 2007a) resulting in higher student engagement in the second sprint, where the number of edits compared to the first sprint was significantly increased (Table 3). Other students saw early engagement with the module as a “worthwhile challenge” (R-2012) and a positive experience: In contrast to other modules where there wasn’t as participatory or as involved, with Agile, after a very short amount of time you are actually involved with doing thing (MF-2011). 5.2. Impact on learning In order to assess the impact of the agile L&T method on student learning, the data on student engagement (5.2.1) and performance (5.2.2) are considered as well as the data on students own perceptions on the type of learning taking place (5.2.3) and the impact of the method (5.2.4). Table 3 Distribution of the number of edits per sprint. Sprint
Edits/Sprint in 2011
1 2 3 4 5 Total
351 735 432 360 501 2379
Edit/Sprint in 2012 14.75% 30.90% 18.16% 15.13% 21.06% 100.00%
85 548 487 480 349 1949
4.36% 28.12% 24.99% 24.63% 17.91% 100.00%
M. Cubric / The International Journal of Management Education 11 (2013) 119–131
127
5.2.1. Engagement The individual engagement of students was measured through the number of wiki edits. The statistics shown in Fig. 2 and Table 3 were calculated from the wikispaces data, which provided the number of edits per individual student as well as the number of edits per day. The total number of edit in both academic years was N ¼ 4474 (M ¼ 89.48, SD ¼ 53.98). As it can be seen from Fig. 2 the high standard deviation is partially due to one particularly active student who contributed a total of 293 edits! The higher engagement in 2011 (N ¼ 2402, M ¼ 100.08, SD ¼ 54.75) was partially due to the same aforementioned “outlier”. When this entry was removed, the average for 2011 was reduced to M ¼ 91.70 (SD ¼ 38.79) which was closer to the average from 2012 (N ¼ 2078, M ¼ 79.92, SD ¼ 49.89). As revealed by an independent t-test the difference in number of edits between the two groups was not significant t(48) ¼ 1.346, p ¼ 0.682. The number of edits per sprint varied, but in both years the second sprint produced the highest number of edits as shown in Table 3. Slightly different patterns in engagement after the second sprint can be observed from the same table i.e. 2012 group worked in a slightly more “agile” way and exhibited less of a “student syndrome”. The distribution of a count of edits per sprint shows that the engagement was frequent rather sparse. The small difference in total number of edits between Table 3 (N ¼ 4328) and Fig. 2 (N ¼ 4474) accounts for additional edits taking place outside of the 10-week project timeframe. The average number of individual edits per week is estimated to 8.66 based on 10-weeks long project and the data from Table 3. The survey data corroborate the estimate with 88.24% of students stated that they contributed to the wiki several times per week. All this shows that the majority of students, for the duration of the project, have engaged regularly with the wiki. Clearly, the engagement measured through a number of edits does not directly imply quality of learning, however it does show that at least reading the page content has taken place, as that was necessary in order to situate the new edits within the page. A random check of edits in every sprint has revealed an insignificant number of “empty” edits (e.g. adding a blank space or new line). 5.2.2. Performance The average final grade percentage was M ¼ 56.08 (SD ¼ 13.12). Students from 2012 achieved slightly better grades (M ¼ 56.62, SD ¼ 16.52) than those from 2011 (M ¼ 55.5, SD ¼ 8.36). However, this difference was not significant t(48) ¼ 0.305, p
