Agile Instructional Development Framework

Agile Instructional Development Framework: Strategies for Increasing Learner and Instructional Designer Collaboration Sharon Bratt Grant Macewan University Abstract: This research reports the results of single exploratory case study in which selected components of the Agile Instructional Development (AIDev) Framework are observed in the context of an actual agile instructional development project. The framework is in what Jabardeen has identified as Phase: 7 Validating the conceptual framework in conceptual framework analysis. This phase addresses whether the proposed framework makes sense to other scholars and practitioners (Jabardeen, 2009). Jabardeen recommends dissemination of the framework to an academic audience for discussion and feedback. Several strategies to support the learner’s role as co-designer are identified and summarized. This study concludes with implication for practice and recommendations for future research to validate the efficacy of these strategies.

Introduction “I think being a co-designer of a course is a great way to be really involved in the work that is done. I am a believer that the best results are produced when a person is committed to a project and they include a part of themselves into it. I don't see being a co-designer as a burden or as a lot of extra work, I see it as an opportunity to be creative and to come up with something unique.” --Study participant Basic research proposes new theories, frameworks and models in order to scaffold and guide emerging research as it matures in its field. The creation of a new research tool –such as a conceptual framework -- holds the promise of extending current knowledge or creating new knowledge based on the data derived from the application of that tool. However, such tools must undergo a validation process to support the tool’s claims of rigour and suitability to task. The recently created Agile Instructional Development (AIDev) Framework (Bratt, 2011) which is intended to support efficiency while fostering pedagogical excellence in the design of instruction according to principles of agile software development is in what Jabardeen has identified as Phase :7 Validating the conceptual framework in conceptual framework analysis. This phase addresses whether the proposed framework makes sense to other scholars and practitioners (Jabardeen, 2009). Jabardeen recommends dissemination of the framework to an academic audience for discussion and feedback. The next step in this research agenda is to validate the agile ID conceptual framework in order to judge the applicability of its dimensions, their descriptions and illustrative strategies. The goal of this study is to continue the validation process by identifying and defining new strategies to increase learner and instructional designer collaboration –a component of the AIDev framework (see

Table 1) by involving the learner as co-designer. This research reports the results of single exploratory case study in which selected components of the framework are observed in the context of an actual agile instructional development project.

Table 1: Learner and instructional designer collaboration principles  

Agile ID Principles 

Instructional Design Process  Components 

Learner and instructional designer collaboration 

Design model (approach) 

 Involve instructors and learners in the  requirements activities 

Instructional design team  member roles 

 Place client on design team   Ensure client representative is on‐site full‐time 

Instructional design  processes 

 Use rapid prototyping methodologies with new  clients to provide appearance prototypes   Test design with potential learners 

Instructional design tools 

‐‐ 

The results of this research will be of use to several ID stakeholder groups including instructors and designers of instruction as a method of improving their practice, ID development project managers interested in agile methodologies and learners, who are the real consumers of these education products. The validation of the AIDev Framework is the next step toward developing a body of research around first principles and best practices as this methodology matures in its field and its impact is felt in the classroom. Background to the Agile Instructional Development (AIDev) Framework Instructional development (ID) practitioners have shown a growing interest in the use of agile methodologies which have been popularized by the software development industry with the intent to gain efficiencies and produce more effective solutions. The goal of this research is to determine if agile methods can be applied to instructional development. ID and software development involve similar processes (analysis, design, development and evaluation); thus ID has historically adapted software development methodologies to develop their products. (Douglas, 2006; (Rawsthorne, 2005) The popularity of agile methods in the past decade has captured the interest of instructional designers dissatisfied with the constraints of their linear, process-oriented development models. Traditional instructional design models which are variations on the well-known ADDIE framework (Analysis, Design, Development, Implementation, and Evaluation) have been challenged for their inability to meet contemporary learning needs in both business and education. These fundamentally linear models have been criticized for being ineffective and inefficient, and inflexible and outdated. (Gordon & Zemke, 2000; Tripp & Bichelmeyer, 1990; Willis, 1995) A growing interest in “Agile” over traditional methods has resulted in the adoption of these methodologies. Unfortunately these changes in ID practices have outpaced fundamental academic research to guide and inform their development. Given the growing interest in agile methods it would be beneficial to frame agile instructional development in order to facilitate meaningful discussion among researchers and practitioners around first principles and best practices as this methodology matures in its field and its impact is felt in the classroom. Such a framework

that explicitly integrates the learner (and instructor) into the agile ID process has been recently created as illustrated in

Appendix A and is the focus of this case study in which selected concepts and processes associated with learner and instructional designer collaboration will be observed in the context of an actual agile instructional development project and modified as needed, based on new insights.

Research Design Context It is important in case study research to accurately describe the context of the case under investigation so that others may replicate the study or create different research designs to test other theories underlying the case study. In this case, client – designer collaboration is central to the practice of agile instructional development in which the learner is the client. This collaboration may be characterized in multiple ways depending on the scope of the project and the roles of the stakeholders. For example, does the project involve one instructor working with one instructional designer? Or are several involved? Is the instructor also the instructional designer? Is the project limited to the design of a single course or an entire curriculum? Is the mode of delivery traditional or e-learning? Is the learner also considered the client in the context of the instructional development project? This case study defines the learner as the client and the course instructor as the instructional designer. The application of agile instructional development was limited to the design of one course project to be completed within a 12 week academic term in a traditional classroom. The instructor had no prior experience with agile instructional development but was interested in involving the learner in the design of the course on an on-going basis. The following sections describe a single exploratory case study based on a post-secondary course called The Phenomenology of Technology. Two important features of the course determined its selection for use in this study. First, it was a brand new course therefore the instructor was open to the idea of developing the course as the term progressed. Second, the low course enrolment (7 students) was suitable for encouraging learner participation as co-designers. The course, offered by the institution’s computing science department, included a seminar component which was developed and implemented using agile instructional development processes with a specific focus on the role of the learner as co-designer responsible for making project design decisions and providing feedback to the instructor to guide project requirements. The case study used survey methodology to identify and define new strategies to support learner-designer collaboration which is a component of the AIDev Framework. Research Method Yin recommends the use of case study methods when seeking to answer why or how; when the researcher has little control over events; and when the focus is on a contemporary phenomenon rather than historical. (2003) The purpose of this study is to apply selected processes identified in the AIDev Framework to the development of a post-secondary course in computing science in order to: 1. Identify issues which impede the use of agile instructional development processes using one instantiation of the framework. 2. Provide prescriptive methods for involving the learner in the AIDev process. 3. Test the theory that instructional design processes can be successfully adapted from agile development methods. This study seeks to answer how learners can be supported in their role as co-designers. The phenomenon is the instantiation of AIDev; the researcher has minimal control over events Participants Learner feedback is central to agile methods therefore incenting participation is a critical component in the design of the course. Participants were required to complete three web-based questionnaires at regular intervals during the semester as part of their course requirement. Permission was granted by the course instructor to invite students enrolled in a 2nd year post-secondary course in computing science to participate in the study. This sample was selected based on their enrolment in a brand new course which was highly suitable for this AIDev methodology which endeavors to include learners in the

course development throughout the duration of the course. Students were provided with a consent form outlining the goals of the study and their role as participants. Each questionnaire consisted of both Likert-type items as well as comments to provide a brief explanation of their Likert-type response. Participant anonymity was ensured by limiting access to the feedback forum to the research team. Data Collection Yin (2003) identifies six sources of evidence that can be collected during case studies. Physical artifact, artifacts, archival records, interviews, documentation, direct observation, and participant-observation. Yin recommends the use of multiple data sources to strengthen validity. However the exploratory and descriptive nature of this preliminary study suggested that the data sources be limited to survey data in order to determine goodness of fit of the framework and to use the data to guide future research. Therefore the study collected data using three surveys consisting of both Likert-type items as well as open-ended questions. Collaboration with all stakeholders is a key principle in agile development therefore an initial presentation by the researcher and course instructor was given in order provide an overview of the study including its purpose, description of agile principles and methods, and the central role that learners occupy in adapting the course’s design to meet their learning requirements. The aim of the presentation was to foster collaboration and encourage participants to provide feedback to improve the course design based on their expressed needs. The focus of the agile instructional development methodology was the course’s seminar component. The seminar component required students to conduct a phenomenological study on an individual of their choice to determine the impact of technology on their personal, social and/or professional life. Results of the research were presented at the end of the term. The requirements for the study were deliberately loosely defined in order to invite students to serve as co-designers of the project. Design decisions such as presentation formats, evaluation criteria, instructor feedback and deadlines were determined in collaboration with the course instructor. Participants were asked to provide feedback on their role as co-designers of the seminar project based on specific aspects of instructional design:  assignment due dates  evaluation criteria  evaluation method Participants were also asked to comment on:  the potential benefits of co-designing the course  the potential weaknesses of co-designing the course  their contribution to the design of the project requirements  role as co-designer in any future courses  the traditional role of the instructor as course designer Each of the three surveys was administered after a project milestone had been completed, for example students had collected their data and were ready to discuss presentation format options and evaluation criteria. A sample of results from one of the surveys is found in

Appendix A. Consent to participant was provided by all seven students enrolled in the course but only five students completed all three questionnaires. Individual responses were converted into aggregate data from all of the participants. The research findings integrate the data gathered through the Likert–type questions and the open-ended survey responses. The data directly related to the identification of strategies to support the learner’s role as codesigner are presented and discussed. Results The data (and resultant strategies are categorized using selected components of the Quality Assurance Framework for Online Courses (Ining, Saj, & Hamilton, 2010) (show in Figure 1). The components were selected due to their conceptual fit.  Curriculum design  Learning experience  Instructional design  Web design  Teaching and facilitation  Course presentation

Figure 1. Quality Assurance Framework

The first category that was selected is Learning experience which Chao describes as including factors such as learner pre-requisite knowledge, learning styles and preferences, and the dynamics of the learning community The second category is Instructional design which deals with the connection among learning outcomes, course activities, teaching strategies, and the use of media and technology. The final category is Teaching and Facilitation which is “the art of carrying out the curriculum and instructional design plan” …encompassing the instructor’s knowledge and skill in guiding learning (Chao, p.35). Table 2 Strategies for supporting the role of learner as co-designer  

Recommended Strategy 

Rationale 

Learning experience  

Conduct appropriate learner  characteristics assessment (prior  knowledge, learning styles, self‐regulated  learning) 

Learners vary according to preference for  structured versus unstructured learning  environment. Adjust degree of instructor‐ learner collaboration to appropriate level  

Instructional design  

Instructor should have a complete but  interim instructional design  

Learner characteristics and need for initial  course structure necessitate the provision of a  complete course design to serve as a blueprint  which can be modified as needed 

 

Instructor should decide, in advance,  which components of the instructional  design are most suitable for agile  development subject to change based on  learner input 

Participants expressed preferences for co‐ designing specific parts of the course i.e.  evaluation method but not evaluation criteria;  project format but not due dates.   

Teaching and Facilitation  

Provide learners with provisional yet basic  structure at the course level (i.e. course  outline) and the assignment level  (learning objectives, weight, potential due  dates) 

Participants expressed need for initial structure  that could be modified by the class as the course  progressed based on learners’ input and needs.   

 

Provide learners with fundamentals of  instructional design and facilitate  discussions and decision‐making sessions  about the course’s design 

Participants expressed concern that they lacked  to knowledge to confidently make decisions  about the course’s design without the assistance  of the instructor i.e. evaluation criteria 

 

Provide regular feedback on individual  learner’s contribution and progress in the  course. 

While participants accepted the idea of self‐ directed learning they also expressed a desire  for continuous feedback about their learning to  assure that they were making correct decisions. 

Conclusion

Implications for local practice. The purpose of this study was to apply selected processes identified in the AIDev Framework to the development of a post-secondary course in computing science in order to: 1. Identify issues which impede the role of the learner as co-designer in the agile instructional development process using one instantiation of the framework. 2. Provide prescriptive methods for involving the learner in the AIDev process. Learners believe that they can and should contribute to the course design. However, they are selective about which aspects of course they are “qualified” to contribute to. This qualification can be addressed using several strategies. First, provide a complete course design to serve as a provisional blueprint that is considered dynamic and flexible. This interim design enables learners to see the full architecture of the course –literally how all of the pieces fit together and what the final products may look like. This prototype can be used to elicit feedback to adapt the design to reflect the learners’ input. The provision of a prototype is a standard practice in agile software development. Another strategy to address learners’ inexperience with instructional design is to teach the fundamentals of instructional design and facilitate discussions and decision-making sessions about the course’s design throughout the duration of the course. Finally, regular feedback on individual learner’s progress will address concerns about the self-directed nature of acting as co-designer. Implications for broader practice. Institutions such as the University of British Columbia have partnered with the U.S.-based company Coursera to provide high quality, non-credit courses free of charge to a global audience. The instructional design team at UBC have expressed an interest in using agile methods to develop these Massive Online Massive Open Online Courses (MOOCs). Efforts such as these and others may benefit from the use of some or all of the strategies identified in this study to increase the role of the client in the design process. It is important to note that these strategies were derived from data gathered from learners not instructors; as such they may not be as effective or suitable in supporting the role of the instructor as co-designer. This presents an area where further research is recommended. Summary A single case exploratory study was conducted as a continuation of a research agenda to validate the recently created Agile Instructional Development (AIDev) Framework. The instantiation of a course that used selected strategies from the framework provided an opportunity for what Yin describes as a revelatory case study in which the researcher has an opportunity to observe and analyze a phenomenon previously inaccessible to inquiry (Yin, 2003, p. 48). Themes that emerged from the survey data suggest that more structure in the form of a

provisional course design (i.e. blueprint) is needed up front in order to manage learner expectations and information session(s) at the start of the course to “educate” learners on instructional design essentials would increase learners’ confidence in their ability to contribute as co-designers of the course. Limitations of the study include the lack of triangulation from multiple data sources. This could have been addressed by including interviews, focus groups and documenting each participant’s contribution to the design of the course such as evaluation criteria or recommended seminar project format. A final limitation is that these strategies were derived from data gathered from learners not instructors; as such they may not be as effective or suitable when the co-designer is an instructor working with an instructional designer. Recommendations for future research include using a larger sample size of learners and collecting data from more than one source. Similar methods could be applied using instructors as the sample to identify which strategies would assist in their role as co-designer when using agile instructional development.

References Bratt, S. (2011). A framework for agile instructional development. Paper presented at the Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2011, 1830-1839. Retrieved from http://www.editlib.org/p/38993 Douglas, I. (2006). Issues in software engineering of relevance to instructional design. TechTrends: Linking Research & Practice to Improve Learning, 50(5), 28-35. doi:10.1007/s11528-006-0035-z Gordon, J., & Zemke, R. (2000). The attack on ISD. Training, , 43-53. Retrieved from http://www.trainingsupersite.com/publications/archive/training/2000/004/004cv.htm Ining, T., Saj, T. & Hamilton, D. (2013). Using collaborative course development to achieve online course quality standards. International Review of Research in Open and Distance Learning, 11(3). Jabardeen, Y. (2009). Building a conceptual framework: Philosophy, definitions, and procedure. International Journal of Qualitative Methods 2009, 8(4) Rawsthorne, P. (2005). Agile methods of software engineering should continue to have an influence over instructional design methodologies. (Unpublished Cape Breton University & Memorial University of Newfoundland, Retrieved from http://www.rawsthorne.org/bit/docs/RawsthorneAIDFinal.pdf Roytek, M. A. (2010). Enhancing instructional design efficiency: Methodologies employed by instructional designers. British Journal of Educational Technology, 41(2), 170-180. doi:10.1111/j.1467-8535.2008.00902.x Tripp, S. D., & Bichelmeyer, B. (1990). Rapid prototyping: An alternative instructional design strategy. Educational Technology Research and Development, 38(1), pp. 31-44. Retrieved from http://www.jstor.org.ezproxy.macewan.ca/stable/30219925 Willis, J. (1995). A recursive, reflective instructional design model based on constructivist-interpretivist theory. Educational Technology, 35(6), 5-23. Retrieved from http://www.quasar.ualberta.ca/edpy597mappin/readings/m13_willis_2.htm Yin, R. K. (2003). Case study research: Design and methods (3rd ed.). Thousand Oaks: Sage Publications.

Appendix A Table 3 Agile Instructional Development Framework (Bratt, 2011)

  Instructional  Design Process  Components 

Design model  (approach) 

Instructional  design team  member roles 

Agile ID Principles Embrace change  to increase  pedagogical  utility 

Iterate learning  designs  frequently 

‐‐

 

‐‐  Place client on  design team   Ensure client  representative is on‐ site full‐time 

‐‐

 Use rapid   Hold regular team  prototyping  meetings to cross‐ methodologies with  share efficiencies  new clients to  provide appearance  prototypes   Test design with  potential learners 

 Request  client/learner  feedback from  each iteration   Conduct post  release review 

  Adapt to suit  evolving  requirements  



Instructional  design tools 

Continuous  review 

‐‐

‐‐ 

 use electronic  storyboard  templates, not  authoring tools   

Communication 

 Involve instructors  and learners in the  requirements  activities 

‐‐ 

 Ensure instructional  designers have  formal training in  pedagogy   Use of appropriate  instructional design  theory   Let pedagogy not  technology lead the  solution   Rotate instructional  designers through  same‐customer  projects   Rotate instructional  designers through  same‐industry  projects   Employ subject  matter experts with  instructional design  experience  Use non‐linear   Have instructional  ID model  designers also serve  as developers  Conduct  research   Have instructional  concurrently  designers oversee  with  designing completed  development  by subject matter  experts lacking  Deliver learning  design experience  designs in days  not weeks  Use templates,   Let pedagogy not  technology lead the  learning objects,  solution  Automated  instructional   Use instructional  design (AID)  design specialists  technologies  within a given media  to promote  technology  awareness 

Learner and  instructional designer  collaboration 

 Incorporate  rapid  prototyping  methodology 



Instructional  design  processes 

Pedagogical  Excellence 

‐‐

 Use groupware  authoring tools to  support concurrent  development  processes 

‐‐

Appendix B Survey Question 

Participant Response   With the potential for completely different  presentation methods, there's going to have  to be a fair amount of vagueness for the  criteria, or there's going to have to be  different ones for each, and it might be hard  to make it fair for all given presentation  styles.  I think with being able to decide the criteria  for evaluation for the seminar project has  cleared our own vision on what and where  the project will go. I think that this gives us  students a clear picture of what we need to  do.  I am not at all confident in deciding the  criteria used for evaluation of the project  because to me it makes no sense to decide  the criteria on which I will be evaluated on.  It'll probably be fine, but being given the  freedom to choose will mean we'll have to  take time choosing, and there's every  possibility that we might choose wrongly. It  also makes it harder to tell what's 'good  enough'.  The only problem I see with this is that some  people may have access to and knowledge of  much more creative ideas whereas others do  not.  A general consensus and the suggestions of  various ways of formatting the presentation  has given us different varieties of choices to  explore from.  I am confident about deciding the format for  my presentation because instead of limiting it  to a paper or an oral presentation of a paper,  I get to choose a medium or a hybrid of them  to best present my findings. 

Please tell us how we might increase your confidence in co-designing any aspect of this course

Just give me a half an idea what I'm supposed  to do, and I'll deal with it from there.  Please provide a proper outline. We need to  know when everything is due and how much  everything is worth in writing.  Make a final mark breakdown as to where the  majority of the mark lies, and make it  available on course outline for new students  A very small amount of structure to the  course/project outlines will probably make  things a little more clearer for the students  and give them more confidence to provide  with valuable feedback. 

I felt that I did not contribute on the design of  the course in terms of material, and the kinds  of assignments given. My contributions lie  mainly in the design of the assignments and  projects and how they were completed. 

I would say with the collective decisions of the  other class members we have restructured  the course outline that was given to us  before. I have given my input in the decisions  of how the class should be structured in terms  of class assignments, and their dates.  I was encouraged to participate and conduct  the seminar project at all times. Every student  in class was given the chance to form the  project in their own individual sense and use  their creativity on how they want to conduct  the interview, who the participants will be  and how will the project be presented. i think  i have been encouraged to bring out my own  thought process to the project and shape it  the way i want.  I was encouraged to participate and conduct  the seminar project at all times. Every student  in class was given the chance to form the  project in their own individual sense and use  their creativity on how they want to conduct  the interview, who the participants will be  and how will the project be presented. i think  i have been encouraged to bring out my own  thought process to the project and shape it  the way i want.  Well, we got to choose how we presented our  findings and the specific sense of what we  were supposed to look for, but we only had a  vague idea of what these findings were  supposed to be, so it was difficult to know  what counted as enough information, or was  even valid information.