AERA 2004 Proposal
Using Design-based… - 1 -
Using Design-based Research in Design and Research of TechnologyEnhanced Learning Environments Feng Wang
[email protected]
Michael J. Hannafin
[email protected]
The University of Georgia
Abstract In the past decade, a new research paradigm—design-based research—has showed great potential as an ideal alternative research methodology. Design-based research is suitable to both research and design of technology-enhanced learning environments (TELEs). At the beginning of this presentation, we will propose the definition and five characteristics of design-based research research methodology. Then, we will discuss why design-based research is important to both research and design of TELEs. Next, nine principles are provided that are essential to successfully implementing design-based research methodology with TELEs. Finally, we will discuss its implications, challenges, and payoffs to TELE designers.
Introduction The application of technology in education remains unsatisfying; TELEs have not been widely used by students and teachers (Cuban, 1986, 2001; Kent & McNergney, 1999). To address these problems, a new educational research paradigm — design-based research — has showed great potential to change the disappointing disconnect between educational research and design practice. The design-based research paradigm, advanced initially by Ann Brown (1992) and Allan Collins (1992) as design experiments, posits intimate relationships between design and engineering and tends to be both scientific and educational (Kelly, 2003). Design-based research deconstructs the presumed integrity that the research should not be contaminated by the external influence of the research (Barab & Kirshner, 2001). Researchers in design-based research processes collaborate intimately with participants to achieve theoretical and pragmatic goals that will ultimately change educational practices in a maximum extent. For TELE design and research, design-based research methodology is even more promising. Many TELE projects proven spectacularly successful in recent years feature this methodology, such as WISE (Bell & Linn, 2000), Adventures of Jasper Woodbury (Cognition and Technology Group at Vanderbilt, 1992a, 1992b), and CSILE (Cohen & Scardamalia, 1998). The purposes of this presentation are to summarize the key ideas of design-based research, to describe the utility of design-based research for both TELE research and design, to provide principles deemed essential to successful TELE s based on design-based research methodology, and to discuss implications, challenges, and payoffs for TELE designers.
Design-based research introduction We use the term “design-based research” specifically to represent a research paradigm comprising many labels in the literature, including: 1) design experiments
AERA 2004 Proposal
Using Design-based… - 2 -
(Brown, 1992; Collins, 1992), 2) design-based research (The Design-Based Research Collective, 2003), 3)design research (Cobb, 2001; Edelson, 2002; Collins, Joseph, & Bielaczyc, in press, 4) development research (van den Akker, 1999) or developmental research (Richey & Nelson, 1996), 5) formative research (Walker, 1992), and 6) action research (Stringer, 1999). Though some have slightly different foci, the intrinsic ideas are consistent. We propose the following definition: Design-based research is a research methodology aimed to improve educational practices through systematic, flexible, and iterative review, analysis, design, development, and implementation, based upon collaboration among researchers and practitioners in real-world settings, and leading to design principles or theories. We summarize the key ideas of design-based research through five basic characteristics: Pragmatic research goal Design-based research posits synergy between practice and research, propelling the development of practice as much as possible. From a design-based research perspective, research should refine both theory and practice (Collins, Joseph, Bielaczyc, in press); ultimately, the value of theory will be appraised by the extent to which their principles inform and improve practice (Greeno & Collins, 1996; Cobb, Confrey, diSessa, Lehrer, & Schauble, 2003). Grounded research methodology As design-based research is usually conducted in a limited number of settings, the research and design process need to be grounded. Design-based research is conducted in real-world contexts with social interaction rather than laboratory settings with social isolation (Collins, 1999). Before conducting a design-based research, researchers review literature and available design cases to identify gaps to ensure the value of the research (Edelson, 2002). Its theory driven nature is important in that design-based methods are considered more a research paradigm than an evaluation method. Interactive, iterative, and flexible research process Design-based research process is characterized by an iterative cycle of design, enactment or implementation, analysis, and redesign (The Design-Based Research Collective, 2003). Through this cycle, a theory will be gradually formed and updated based on the accumulated data collected in each design iteration as well as implementation experiences of the designer(s) (Edelson, 2002).The interactive, iterative and flexible research process also corresponds to timely factors in the local settings (The Design-Based Research Collective). Tacitly, in the design-based research process, a researcher needs to balance the role as a designer and a researcher. This is not necessarily negative as it helps to balance different solutions and perspectives (van den Akker, 1999). Integrative research methods Design-based researchers use mixed methods to maximum the credibility and adaptability of their methods. Methods used in design-based research are not unique to design-based research, such as survey, expert review, evaluation, case study, interview, inquiry methods, etc. (Richey & Nelson, 1996; McCandliss, Kalchman, & Bryant, 2003). Retrospective analysis and formative evaluation are also considered methods of designbased research (van den Akker, 1999; Edelson, 2002; Cobb et al., 2003). Contextual research results
AERA 2004 Proposal
Using Design-based… - 3 -
The results generated from design-based research take the form of a profile akin to a consumer report (Collins et al., in press) or principles in the form of heuristic statements. The research process, the research findings, and any change from the initial plan are documented; some warrants or guidance on how to use these principles are also provided (Shavelson, Phillips, Towne, & Feuer, 2003). Thus, other researchers or designers can trace the emergence of an innovation or combinations of innovations according to their interests; they also examine contexts or conditions that led to different effects (Baumgartner & Bell, 2002).
Importance of design-based research to TELEs Design and research on TELEs are mutually beneficial. Design processes, especially research-driven designs, can be thought of as research processes. Design is a strategy for developing, refining, and testing theories with design-based research rather than a way to implement theories for testing with traditional research methodologies (Edelson, 2002). Regarding the history of design TELEs, however, design and research are often mismatched. Numerous learning environments were developed using contradictory theory bases; designs are not supported by underlying theories so the outcomes are not traceable to an emerging theory (Collins, 1992). With traditional research methodologies, research is usually conducted after the design processes and research results generated from traditional research methodologies are too abstract to be referable (Cobb, et al., 2003). Thus, it is difficult for TELE designers to identify and apply useful theories to guide their designs. With the design-based methodology, design and research processes are integrated. Design-based research, with its pragmatic goal, grounded methodology, integrative methods, contextual results, and interactive, iterative, flexible processes, not only supports TELE design but also generates useful, practical theories. In the following section, we explicate the importance of design-based approaches to TELEs using three propositions: 1) design-based research can support the design of TELEs; 2) TELE design processes can be research; 3) design-based research is an ideal research methodology on TELEs. There are other important considerations. First, many problems associated with educational research methodologies became apparent especially in the past decade; more and more educational researchers voice the need to employ pragmatic research methodologies capable of narrowing the gap between theory and practice. Some TELE designers also argued the need to integrate research and design on TELEs. Research on TELEs should take the form of design because knowledge generated from the design processes are unique and can support the design of TELEs in a maximum extend. As design-based research can bring so many benefits to TELE designers, we would argue that it’s an ideal way to both design and research of TELEs.
Principles for design-based research In the following section, we discuss the methodology of design-based research through principles essential to the successful implementation of design-based research on TELEs. We do not intend to build a general framework of design-based research, but rather to inform the design-based research within the context of TELEs.
AERA 2004 Proposal • • • • • • • • •
Using Design-based… - 4 -
Support the design with research from the outset Set pragmatic theory goals and initial plans Conduct research in real-world settings without exception Collaborate with participants intimately Use research methods systematically Analyze collected data retrospectively and constantly Refine the design continuously and iteratively Report contextual and usable design principles Underscore the generizability of the design continuously
Challenges for design-based researchers As a relatively new methodology, design-based research has both advantages and limitations; important unanswered questions remain: What epistemology and theory perspectives underlie design-based research initiatives? How do we reconcile the mixed methods needed in the research processes? How do we balance the goal of improving practice and the goal of generating design principles? The pragmatic nature of designbased research also limits the applicability of traditional theories and related constructs which may be necessary in some situations. After all, the culture of present-day practice is affected deeply by empirical research methodologies that appears “scientifically credible” because of the presumed discipline of the inquiry and statistical evidence typically accompanying reports; thus, design-based research may not be as well-suited for administrators or politicians as empirical methodologies given policy debates and emphasis on “Scientifically-based research” (Cobb, 2001). Finally, as design-based research requires documenting the whole research processes in authentic learning environments, the amount of data collected is typically large, requiring both extended time and resources to analyze (Collins, Joseph, & Bielaczyc, in revision).
Payoffs and conclusions Design-based research affords many opportunities for improvement to both design and research on TELEs. We view three areas as most promising: a) propelling the application of TELEs in classrooms, b) productively and effectively generating credible and useful theories, and c) transforming disappointing traditional research and design methodologies. When design-based research is employed by more designers and researchers, its benefits will be more evident. For the field of instructional technology specifically, design-based research is critical to the evolution of its theory base (Richey & Nelson, 1996). TELE designers should collect their efforts to accelerate the coming systematic change brought by the design-based research methodology.
References Barab, S. A., & Kirshner, D. E. (2001). Guest Editors' Introduction: Rethinking methodology in the learning sciences. Journal of the Learning Sciences, 10(1&2), 5-15. Baumgartner, E., & Bell, P. (2002). What will we do with design principles? Design principles and principled design practice. Paper presented at the Annual
AERA 2004 Proposal
Using Design-based… - 5 -
Conference of the American Educational Research Association, New Orleans, Louisiana. Bell, P., & Linn, M. C. (2000). Scientific arguments as learning artifacts: Designing for learning from the Web with KIE. International Journal of Science Education, Special Issue(22), 797-817. Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The Journal of the Learning Sciences, 2(2), 141-178. Cobb, P. (2001). Supporting the improvement of learning and teaching in social and institutional context. In S. Carver & D. Klahr (Eds.), Cognition and instruction: 25 years of progress (pp. 455-478). Cambridge, MA: Lawrence Erlbaum Associates, Inc. Cobb, P., Confrey, J., diSessa, A., Lehrer, R., & Schauble, L. (2003). Design experiments in educational research. Educational Researcher, 32(1), 9-13. Cognition and Technology Group at Vanderbilt (1992a). The Jasper experiment: An exploration of issues in learning and instructional design. Educational Technology Research & Development, 40(1), 65-80. Cognition and Technology Group at Vanderbilt (1992b). The Jasper Series as an example of anchored instruction: Theory, program description, and assessment data. Educational Psychologist, 27(3), 291-315. Cognition and Technology Group at Vanderbilt (1997). The Jasper project: Lessons in curriculum, instruction, assessment, and professional development. Mahwah, NJ: Erlbaum. Cohen, A., & Scardamalia, M. (1998). Discourse about ideas: Monitoring and regulation in face-to-face and computer-mediated environments. Interactive Learning Environments, 6(1-2), 93-113. Collins, A. (1992). Towards a design science of education. In E. Scanlon & T. O’Shea (Eds.), New directions in educational technology (pp. 15-22). Berlin: Springer. Collins, A., Joseph, D., & Bielaczyc, K. (in press). Design Research: Theoretical and Methodological Issues. In J. Campione (Ed.), Volume in honor of Ann Brown. Collins, A., Joseph, D., & Bielaczyc, K. (in revision). Design Research: Theoretical and Methodological Issues. Journal of the Learning Sciences. Cuban, L. (1986). Teachers and machines: The classroom use of technology since 1920. New York: Teachers College Press. Cuban, L. (2001). Oversold and underused: Computers in the classroom. Cambridge, MA: Harvard University Press. Edelson, D. C. (2002). Design research: What we learn when we engage in design. Journal of the Learning Sciences, 11(1), 105-121. Kelly, A. E. (2003). Research as Design. Educational Researcher, 32(1), 3-4. Kent, T. W., & McNergney, R. F. (1999). Will technology really change education: From blackboard to web. Thousand oaks, CA: Corwin Press. McCandliss, B. D., Kalchman, M., & Bryant, P. (2003). Design experiments and laboratory approaches to learning: Steps toward collaborative exchange. Educational Researcher, 32(1), 14-16.
AERA 2004 Proposal
Using Design-based… - 6 -
Richey, R. C., & Nelson, W. A. (1996). Development research. In D. Jonassen (Ed.), Handbook of research for educational communications and technology (pp. 12131245). London: Macmillan. Savery, J. R., & Duffy, T. M. (1996). Problem based learning: An instructional model and its Constructivist framework. In B. G. Wilson (Ed.), Constructivist learning environments: Case studies in instructional design (pp. 135-148). Englewood Cliffs, NJ: Educatioal Technology Publications. Shavelson, R. J., Phillips, D. C., Towne, L., & Feuer, M. J. (2003). On the science of education design studies. Educational Researcher, 32(1), 25-28. Stringer, E. (1999). Action research: A hand book for practitioners (2nd ed.). Thousand Oaks, CA: Sage Publications. The Design-Based Research Collective (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5-8. van den Akker, J. (1999). Principles and methods of development research. In J. van den Akker, N. Nieveen, R. M. Branch, K. L. Gustafson & T. Plomp (Eds.), Design methodology and developmental research in education and training (pp. 1-14). The Netherlands: Kluwer Academic Publishers. Walker, D. F. (1992). Methodological issues in curriculum research. In P. Jackson (Ed.), Handbook of research on curriculum (pp. 98-118). New York: Macmillan.
This paper can be cited as: Wang, F., & Hannafin, M. J. (2004). Using design-based research in design and research of technology-enhanced learning environments. Paper presented at the Annual Meeting of the American Educational Research Association, San Diego, CA.
