Exploring Technology for Writing and Writing Instruction

Exploring Technology for Writing and Writing Instruction Kristine E. Pytash Kent State University, USA Richard E. Ferdig Kent State University, USA

A volume in the Advances in Educational Technologies and Instructional Design (AETID) Book Series

Managing Director: Editorial Director: Production Manager: Publishing Systems Analyst: Development Editor: Assistant Acquisitions Editor: Typesetter: Cover Design:

Lindsay Johnston Joel Gamon Jennifer Yoder Adrienne Freeland Austin DeMarco Kayla Wolfe Christina Barkanic Jason Mull

Published in the United States of America by Information Science Reference (an imprint of IGI Global) 701 E. Chocolate Avenue Hershey PA 17033 Tel: 717-533-8845 Fax: 717-533-8661 E-mail: [email protected] Web site: http://www.igi-global.com Copyright © 2014 by IGI Global. All rights reserved. No part of this publication may be reproduced, stored or distributed in any form or by any means, electronic or mechanical, including photocopying, without written permission from the publisher. Product or company names used in this set are for identification purposes only. Inclusion of the names of the products or companies does not indicate a claim of ownership by IGI Global of the trademark or registered trademark. Library of Congress Cataloging-in-Publication Data Exploring technology for writing and writing instruction / Kristine E. Pytash and Richard E. Ferdig, editors. pages cm Includes bibliographical references and index. Summary: “This book examines the use of writing technologies in early childhood, elementary, secondary, and post-secondary classrooms, as well as in professional development contexts”--Provided by publisher. ISBN 978-1-4666-4341-3 (hardcover) -- ISBN 978-1-4666-4342-0 (ebook) -- ISBN 978-1-4666-4343-7 (print & perpetual access) 1. English language--Rhetoric--Study and teaching--Computer-assisted instruction. 2. Educational technology. 3. English language--Composition and exercises--Study and teaching--Computer-assisted instruction. I. Pytash, Kristine E., 1979- II. Ferdig, Richard E. (Richard Eugene) PE1404.E97 2013 808’.04207--dc23 2013012270 This book is published in the IGI Global book series Advances in Educational Technologies and Instructional Design (AETID) (ISSN: 2326-8905; eISSN: 2326-8913) British Cataloguing in Publication Data A Cataloguing in Publication record for this book is available from the British Library. All work contributed to this book is new, previously-unpublished material. The views expressed in this book are those of the authors, but not necessarily of the publisher.

1

Chapter 1

Game-Based Writing Strategy Practice with the Writing Pal Rod D. Roscoe Arizona State University, USA Russell D. Brandon Arizona State University, USA Erica L. Snow Arizona State University, USA Danielle S. McNamara Arizona State University, USA

ABSTRACT In this chapter, the authors consider the value of educational games to support students’ writing strategy acquisition and practice. Sixty-five high school students participated in a summer program using the Writing Pal, an intelligent tutoring system designed to support adolescents’ persuasive writing across multiple phases of the writing process. Overall, students who interacted with the full W-Pal intelligent tutoring system (i.e., animated strategy lessons, game-based practice, and essay-based practice with feedback) were better able to articulate new writing strategies than students who engaged in intensive essay-based practice by writing and revising twice as many essays with feedback. Importantly, performance within several educational games was found to be a significant predictor of strategy acquisition. The authors argue that these strategy benefits arise from the ways in which strategy-specific, game-based practice activities support the decomposition of task goals, clear operations for achieving those goals, compensation for individual differences, and motivation to practice.

DOI: 10.4018/978-1-4666-4341-3.ch001

Copyright © 2014, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Game-Based Writing Strategy Practice with the Writing Pal

GAME-BASED WRITING STRATEGY PRACTICE WITH THE WRITING PAL

Benefits of Strategy Instruction and Practice

Writing is a difficult process that involves the coordination of complex cognitive tasks and goals, and a central aim of writing instruction is to enable students to meet these myriad challenges (Breetvelt, van den Bergh, & Rijlaarsdam, 1994; Deane et al., 2008; Flower & Hayes, 1981). Increasingly, and as communicated by the contributors in this volume, researchers are exploring ways to enhance such instruction via technology. For example, automated essay scoring and automated writing evaluation systems (Grimes & Warschauer, 2010; Warschauer & Grimes, 2008) have become popular tools that allow teachers to assign more essays to students while providing feedback to the students on key problems and errors. Prior research has also explored how technology can support specific writing skills, such as summarization (Kintsch, Caccamise, Franzke, Johnson, & Dooley, 2007) and argumentation (Wolfe, Britt, Petrovich, Albrecht, & Kopp, 2009). In this chapter, we consider the value of educational games to support the acquisition and practice of writing strategies. This research occurs within the context of the Writing Pal (W-Pal), an intelligent tutoring system (ITS) designed to support adolescent students’ persuasive writing and strategy development across multiple phases and aspects of the writing process (McNamara et al., 2012; Roscoe, Varner, Weston, Crossley, & McNamara, in press). We first briefly describe the strategy instruction framework that informs W-Pal pedagogy and game-based practice. Subsequently, we examine changes in students’ articulation of writing strategies after learning with W-Pal, and how such changes are related to students’ performance within a suite of educational games.

Strategies are effortful and purposeful procedures that a person can apply to achieve a goal or facilitate the accomplishment of a task (Alexander, Graham, & Harris, 1998; Healy, Schneider, & Bourne, 2012). Consistently, decades of research on writing education have identified explicit strategy instruction as a fundamental means of improving student writing across age levels (Graham, McKeown, Kiuhara, & Harris, 2012; Graham & Perin, 2007). In such instruction, students are provided background information about the processes and goals of writing, and then are taught concrete strategies for enacting those processes and goals. For example, de la Paz and Graham (2002) taught middle-school writers to use a PLAN and WRITE strategy. The PLAN mnemonic teaches students to consider the prompt, generate main and supporting ideas, and organize these ideas. The WRITE mnemonic instructs students to implement these plans and to vary their sentence structure and vocabulary. de la Paz and Graham found that students who were taught the PLAN and WRITE strategies generated more plans of higher quality (d = 1.17), wrote longer essays (d = .82), used more sophisticated vocabulary (d = 1.13), and wrote better essays (d = 1.71) compared to a control condition that received traditional writing instruction (i.e., grammar, spelling, vocabulary, idea generation, and organization). Thus, explicit strategy instruction facilitated adolescent students’ development of writing proficiency. We summarize the benefits of strategy instruction using three general principles. First, strategy instruction benefits learning and performance by decomposing complex or challenging processes into manageable sub-goals (Healy et al., 2012).

2


The writing process is multifaceted and involves several interactive, iterative stages (Deane et al., 2008; Flower & Hayes, 1981) such as prewriting (i.e., generating and organizing ideas prior to writing), drafting (i.e., translating initial thoughts and plans into coherent text), and revising (i.e., elaborating and restructuring an essay to improve overall quality). Strategy instruction makes the purposes and procedures of these writing processes more visible and tractable. Thus, students can develop mastery of individual steps and tasks before having to enact the entire writing process. Second, strategy instruction explicates the operations and actions needed to accomplish writing goals. Strategy instruction provides the concrete “how to” information and heuristics that enable students to complete tasks, solve problems, and evaluate outcomes (Alexander et al., 1998). Finally, strategy instruction can help to compensate for students’ individual limitations and challenges (e.g., Graham, Harris, & Mason, 2005). For example, some developing writers lack broad world knowledge or topic knowledge, leading to essays with unsupported claims and insufficient elaboration. Other students may have a less-developed vocabulary, resulting in text that is highly repetitive. However, strategies can help students leverage their available resources to overcome such limitations. Brainstorming strategies can help students probe their prior knowledge more deeply, and certain rhetorical strategies can help students leverage their personal experiences as evidence in lieu of objective or quantitative data. Indeed, research has shown that strategy instruction is especially important for less-skilled writers (Graham et al., 2005). The decomposition of tasks and goals, clear operations, and compensation for individual differences contribute to the educational potential of strategy instruction. However, to fully realize this potential, students must also be given opportunities for sustained and deliberate practice to internalize the newly-acquired strategies (Kellogg, 2008; Kellogg & Whiteford, 2009), and individualized

feedback to guide them in effective and appropriate strategy use (McGarrell & Verbeem, 2007; Shute, 2008; Sommers, 1982). During the initial stages of acquisition, strategy implementation may be slow and error-prone because the steps are misapplied or forgotten. Students might even experience a decrement in performance as they struggle to use the new strategy. However, through deliberate practice in which learners purposefully seek to improve their skills, students can internalize key operations and understand how to apply specific strategies effectively (Ericcson, Krampe, & Tesch-Romer, 1993; Plant, Ericcson, Hill, & Asberg, 2005). Sustained practice over time is also crucial because knowledge and understanding of the newly learned strategies may decay without reinforcement (Carpenter, Pashler, Wixted, & Vul, 2008; Rohrer & Pashler, 2010). Finally, further support for students’ strategy acquisition and development is provided by feedback (Hattie & Temperley, 2007; Shute, 2008). Feedback helps to focus attention on key aspects of the learning task, outcomes, and the relationship between those factors and strategy use. Summative feedback helps students gauge their performance and recognize when they have implemented a strategy incorrectly during practice. Formative feedback provides concrete guidance for how to improve in future practice. Both types of feedback can help students to optimize their strategy acquisition and practice.

Writing Pal and GameBased Practice The W-Pal writing tutor provides strategy instruction and writing practice that is aligned with the strategy framework outlined above. W-Pal provides comprehensive strategy instruction across major phases of writing and provides diverse practice opportunities with feedback. Direct strategy instruction is provided via eight writing strategy modules, which teach strategies for prewriting, drafting, and revising (Table 1). Students are introduced to specific strategies via

3


Table 1. Summary of writing strategy module content and practice games Module

Strategy Content

Practice Games

Prologue

Introduces W-Pal, the animated characters, and discusses the importance of writing

Freewriting (FW)

Covers freewriting strategies for quickly generating essay ideas, arguments, and evidence prior to writing (FAST mnemonic)

Freewrite Flash

Planning (PL)

Covers outlining and graphic organizer strategies for organizing arguments and evidence in an essay

Mastermind Outline Planning Passage

Introduction Building (IB)

Covers strategies for writing introduction paragraph thesis statements, argument previews, and attention-grabbing techniques (TAG mnemonic)

Essay Launcher Dungeon Escape Fix It

Body Building (BB)

Covers strategies for writing topic sentences and providing objective supporting evidence (CASE mnemonic)

RoBoCo Fix It

Conclusion Building (CB)

Covers strategies for restating the thesis, summarizing arguments, closing an essay, and maintain reader interest in conclusion paragraphs (RECAP mnemonic)

Lockdown Dungeon Escape Fix It

Paraphrasing (PA)

Covers strategies for expressing ideas with more precise and varied wording, sentence structure, splitting run-ons, and condensing choppy sentences

Adventurer’s Loot Map Conquest

Cohesion Building (CH)

Covers strategies for adding cohesive cues to text, such as connective phrases, clarifying undefined referents, and threading ideas throughout the text

CON-Artist Undefined & Mined

Revising (RE)

Covers strategies for reviewing an essay for completeness and clarity, and strategies for how to improve an essay by adding, removing, moving, or substituting ideas (ARMS mnemonic)

Speech Writer

Prewriting Phase

Drafting Phase

Revising Phase

short animated videos narrated by pedagogical agents. These videos decompose the goals and operations for each strategy, such as explaining how to preview one’s arguments in the introduction of an essay (see Figure 1). Essay-based practice is common among technologies for computer-based writing instruction (e.g., Kellogg, Whiteford, & Quinlan, 2010; Warschauer & Grimes, 2008), and W-Pal similarly allows students to compose timed, persuasive essays based on a variety of SAT-style prompts. Essays are typed using a straightforward word processor interface and then submitted for automated assessment. This scoring is powered by natural language algorithms utilizing Coh-Metrix and other text analysis tools (McNamara, Crossley, & Roscoe, 2012). Submitted essays initially receive a holistic rating from “Poor” to “Great” (6-point scale), and then receive formative feed4

back that addresses specific writing goals and strategies (Figure 2). This feedback is implemented as a scaffolded series of thresholds as-

Figure 1. Screenshot from the argument previews lesson within the Introduction Building module


Figure 2. Screenshot of automated essay feedback report within W-Pal

sessing Legitimacy, Length, Relevance, Structure, Introduction, Body, Conclusion, and overall Revising. To avoid overwhelming users (Grimes & Warschauer, 2010), W-Pal gives feedback on only one Initial Topic (i.e., first problem detected in the hierarchical series of checks). Students can then request more feedback on that topic or feedback on one additional Next Topic (i.e., next problem detected). One of the challenges of essay-based practice, however, is that students must enact the entire writing process (Braaksma, Rijlaarsdam, van den Bergh, & van Hout-Wolters, 2004; Breetvelt et al., 1994). Consequently, essay-based practice may not help students focus their efforts on mastering specific strategies. Given the goals of W-Pal, it was important to develop practice opportunities that better supported the principles of decomposition, operations, and compensation. Thus, we selected strategies covered in the lessons and constructed strategy-specific identification or generative practice tasks for each module. In identification practice tasks, students examine brief texts to label the strategies used or identify strategies that could be applied to improve the text. For example, students might identify the

undefined referents in a short text or identify the attention-grabbing technique used in an introductory paragraph. In generative practice tasks, students authored short texts while applying one or more strategies. For instance, students might freewrite for several minutes on a given prompt or write a conclusion paragraph that summarizes an essay outline. A further barrier to practice may be motivation or fatigue, which decreases the likelihood of sustained or deliberate practice. For many learning technologies, such as ITSs that require significant training time, student disengagement is a common problem (McNamara, Jackson, & Graesser, 2009). Although computer-based learning tools often initially benefit from novelty effects, such newness can wear thin (e.g., Clark, 1983). Importantly, students also frequently report negative attitudes toward writing, describing the experience as boring, unpleasant, or impossible (Bruning & Horn, 2000; Hidi, Berndorff, & Ainley, 2002; Pajares, 2003). For these reasons, we chose to support strategy practice within W-Pal using educational games, which are argued to leverage students’ enjoyment of gaming to promote deeper engagement (Dondlinger, 2007; Malone & Lepper,

5


1987; Ryan, Rigby, & Przybylski, 2006; Young et al., 2012). Researchers have proposed various frameworks describing players’ goals for gaming, such as earning points, collecting treasures, exploring worlds, discovering game mechanics, or defeating other players (Quick, Atkinson, & Lin, 2012). Other taxonomies describe generalized game features that players may find enjoyable, such as fantasy, narrative, challenge, discovery, and competition (Hunicke, LeBlanc, & Zubek, 2004). Together, such design elements may stimulate students’ continued interest in playing the game and thereby continuing to practice the strategies. W-Pal uniquely offers game-based practice via a suite of 16 educational games (Table 2). Identification and generative practice tasks were embedded in diverse game features and narratives, including elements of competition, puzzlesolving, role-play, and adventure. To exemplify

W-Pal practice, we briefly describe two games. In Adventurer’s Loot, students practice identifying paraphrasing strategies by taking on the role of a treasure hunter (Figure 3). Paraphrasing strategies include the Synonym Strategy (using more varied and precise words), Structure Strategy (reorganizing sentences), Condensing Strategy (combining choppy sentences), and Splitting Strategy (fixing run-on sentences). Students begin by choosing a site, which sends them to an exotic location such as an underground lake. “Clues” are given in the form of an original sentence along with a target paraphrasing strategy. To “decipher” the clue, players must identify which of four answer sentences implements the given strategy. Correct answers earn treasures (score increase) and incorrect answers cause a monster to appear (score decrease). Importantly, students need only consider only a few sentences at a time, which allows them to

Table 2. Brief descriptions of Writing Pal practice games Game

Description

Freewrite Flash

Fill the Idea Meter and earn Idea Flash Cards by freewriting on a prompt.

Mastermind Outline

Repair the Mastermind Mainframe by assembling an outline from given argument and evidence statements.

Planning Passage

Travel to various destinations and earn souvenirs by selecting appropriate arguments and evidence.

Dungeon Escape

Escape by avoiding the guard and rising waters. Select doors by labeling attention-grabbing techniques.

Essay Launcher

Rescue spaceships by selecting thesis statements and attention-grabbers for sample introduction paragraphs.

Fix It

Evaluate paragraphs for missing key elements, such as thesis statements and evidence. Fix the broken circuit board.

RoBoCo

Build robots by writing topic and evidence sentences for a given thesis.

Lockdown

Stop computer hackers by writing conclusions based on a given outline.

Adventurer’s Loot

Explore different locations and obtain treasure by correctly identifying use of paraphrasing strategies.

Map Conquest

Earn flags by identifying paraphrasing strategies, and then use those flags to conquer the game board.

Undefined & Mined

Disarm mines by identifying undefined referents in short texts.

CON-Artist

Catch a thief by following clues. The clues are solved by selecting transition words to link given sentences.

Speech Writer

Help a friend on the debate team revise a speech. Identify the major problems and then edit the speech to improve it.

6


Figure 3. Screenshot of Adventurer’s Loot game within the Paraphrasing module

focus on key features of each paraphrase type. In addition, the best treasures are earned when players answer correctly on their first attempt. Thus, this game rewards thoughtful analysis of the potential answers rather than guesswork. RoBoCo is a generative practice game in which students write topic sentences and evidence for body paragraphs (Figure 4). Students take on the role of a robot designer at the Robot Body Company who must design new robots. Students read a SAT-style writing prompt and then write a topic sentence and several evidence sentences to support that claim. Importantly, these two tasks are somewhat distinct. The topic and evidence sentences are typed into separate interfaces that further decompose the task of writing a body paragraph while maintaining the overall context. After students submit their text, natural language algorithms assess whether the topic sentence is on-topic and contains an argument, and whether evidence sentences are on-topic and contain objective details (e.g., specific names and dates) rather than subjective or hypothetical details (e.g., what-if statements). Students can request strategy

hints throughout the game. Across two rounds, students can earn up to four robot “heads” for good topic sentences and four robot “bodies” for good evidence. Thus, the number of parts earned is an indicator of how well the student is performing. Subsequently, students can use these earned robot parts (if any) to assemble robots that are displayed at the Annual Show. The final score is based on performance of the task and the number of robots built, which is expressed as financial profit that the student has earned for the company. In sum, educational games within W-Pal serve an important and unique role in strategy acquisition and practice. Students are initially exposed to the strategies via the strategy lessons. However, rather than immediately requiring students to implement new strategies within essay compositions, W-Pal allows students to further master each strategy in isolation. Moreover, by embedding such practice in games, some of the affective obstacles of strategy acquisition may be offset. Game-based practice may encourage more delib-

7


Figure 4. Screenshot of RoBoCo game within the Body Building module

erate and sustained practice as students strive to earn better scores or compete with others. To explore the potential of game-based strategy practice, we describe a study that contrasts two modes of writing instruction and practice. One mode follows a typical approach for computerbased writing instruction wherein students write and revise many practice essays with automated feedback. In this case, students are exposed to writing strategies via the feedback received and have ample opportunities to explore these strategies through essay writing. An alternative mode provides direct strategy instruction along with game-based practice, but at the cost of writing practice essays. In this mode, students write half as many essays with feedback, but can practice explicitly-taught strategies via educational games. We hypothesize that the latter mode will better support students’ acquisition of new writing strategies.

8

METHOD Participants High school students (n = 65) from a large, urban area in the southwestern United States participated in a lab-based summer program using W-Pal. The average age of students was 16 years old, with 70.8% female students. With regards to ethnicity, 6.2% of students identified as African-American, 15.4% as Asian, 24.6% as Caucasian, and 44.6% as Hispanic. Average grade level was 10.2 with 35.4% of students reporting a GPA of 3.0 or below. Most students self-identified as native speakers of English (n = 38) although many self-identified as English Language Learners (ELL, n = 27). However, an analysis of prior writing ability (e.g., prestudy essay scores) found no difference between native speakers and ELLs, t(62) = 1.05, p = .30.


Procedures Students attended 10 sessions over 2-4 weeks. A test of writing strategy knowledge was given in the first and final sessions, along measures of writing proficiency, reading comprehension, vocabulary, and writing attitudes. Sessions 2-9 were devoted to training. Students in the W-Pal Condition (n = 33) used the full W-Pal, including essay writing, lessons, and games. In each session, these students wrote and revised one essay and completed one module. Students rated each game immediately after playing. Students in the Essay Condition interacted only with the essay and feedback tools (n = 32). These students wrote and revised two essays per session with feedback. Due to a data logging error, the post-study materials of one Essay condition student were lost, resulting in a final sample of 31 students.

Measures Strategy Knowledge Test Students responded to six open-ended questions regarding strategies “good writers use” when 1) preparing to write, 2) writing introductory paragraphs, 3) writing body paragraphs, 4) writing concluding paragraphs, 5) making an essay more understandable, and 6) revising an essay (see Appendix). Students’ responses were contentcoded to identify the presence of relevant strategy concepts. For example, in response to Question 1 (preparation), a student wrote, “They can brainstorm ideas about what their writing like a list or brainstorm web.” This response contains three strategy concepts: brainstorming, lists, and webs. Another student answered, “In order to prepare, I pre-write or organize my prior knowledge before writing an essay.” This response exhibits three different concepts: prewriting, organization, and use of prior knowledge. Strategy concept scores were computed for each question by tabulating valid strategy concepts. Thus, students’ responses

could earn similar scores by articulating different, yet valid, conceptions of writing strategies. To develop this coding scheme, the entire corpus of responses was reviewed and a standardized template of acceptable strategy concepts was constructed for each question. The resulting templates were detailed, ranging from 26 to 47 strategy concepts per question. To establish reliability, two raters (blind to condition and test phase) independently applied the coding scheme to approximately one-third of the corpus. Initial inter-rater agreement was high or acceptable for all questions: Question 1 (97.7%), Question 2 (94.1%), Question 3 (88.6%), Question 4 (87.3%), Question 5 (78.9%), and Question 6 (86.1%). Subsequently, the complete corpus was coded by the first author and reviewed by coauthors. All disagreements were resolved through collaborative discussion.

Game Perception Surveys and Log Data Embedded within the system were surveys soliciting students’ game perceptions. W-Pal students used a 4-point scale to rate games on enjoyment, helpfulness for learning, ease of gameplay, and graphics. Higher ratings indicated a more positive response. Two additional questions were asked as a check to make sure the games were playable. These usability questions showed that students considered game instructions to be understandable (M = 3.4, SD = 0.4) and game controls to be clear (M = 3.5, SD = 0.4). Students’ game scores were automatically logged by the W-Pal system.

RESULTS Strategy Knowledge We first examined students’ raw strategy concept scores prior to the study and after the study (Table 3). There were no pretest differences for any of the questions (all Fs < 2.50). To assess

9


Table 3. Pretest and posttest strategy concept scores Writing Pal Question

Pretest

Posttest

Essay Pretest

Gain Posttest

F(1,62)

p

1 – Preparation

2.4 (1.4)

2.5 (1.7)

2.4 (1.4)

2.2 (1.2)