let's play! learning music through video games and ...

MUSIC EDUCATION

ity of Illinois Press. ly published 1934). ld music in the I schools: 1838-1988

Tobias, E. S. (2012). Let's play! Learning music through video games and virtual worlds. In G. McPherson & G. Welch (Eds.), Oxford handbook of music education (Vol. 2, pp. 531-548). Oxford, England: Oxford University Press.

CHAPTER

6.3

n Gould reader

de. New York: University

ireness. London:

w to lock down New York:

;ity Press. :>la. NY: Dover. Press. 'S of the I' York: Mason i man. Toronto:

LET'S PLAY! LEARNING MUSIC THROUGH VIDEO GAMES AND VIRTUAL WORLDS EVAN S. TOBIAS

ical ed. by d 1964). :T: Yale University JOs

Angeles:

Magazine. 8. • Durham. NC: Oxford

The imagery and sounds of games such as Guitar Hero· and Rock Band have captured the imaginations of popular media, the public, and would-be rock stars across the world. Discourse and debate surrounding these games, however, typically remain constrained within a dichotomy of virtual versus "real" performance, often excluding music video games' potential as media for musical learning. Cautious music educators will wonder why they would choose to use their limited time putting video game controllers in students' hands instead of instruments. In this chapter I suggest that video games create virtual worlds rich with potential for students to interact with music in new ways. construct musical understanding, and connect their musical engagement and learning between school and home. While video games may not be the primary form of entertainment media for all young people, gameplay is firmly enmeshed in contemporary cultural milieux. Acknowledging musical playas a productive site for constructing musical understanding (Harwood, 1998; Marsh & Young, 2006), music educators might consider the challenges and potential of using video games and gameplay as a means of and resource for teaching and learning music. Though studies pertaining to this focus are emerging (Clements, Cody, & Gibbs, 2008; Lum, 2009), a lacuna in related

53 2

THE OXFORD HANDBOOK OF MUSIC EDUCATION

empirical work compels music educators to envision curricular and pedagogical possibilities of these media. This chapter offers one step in this direction. After outlining the background and current scope of music-focused video games, 1 situate video games in terms of new literacies (Gee, 2004; Lankshear & Knobel, 2006). 1 then draw on research on video games and learning to provide a theoretical framework that supports the use of video games in music education. Along with music-focused video games 1discuss the roles that music plays in video games (Collins, 2008). As studies focusing on video games and music education increase, the principles and approaches for learning music through gameplayand integrating video games in music programs suggested in this chapter and by others should be reexamined and developed further. It is Friday evening, which means Rock Band 3 night at the Jimenez household. Gina asks her mother, Sasha, for permission to purchase two songs by Lady Gaga as downloadable content (DLC). Sasha says yes, adding, "but I'm playing the drums." Sasha sits behind the drum controllers and starts hitting the rubber pads and cymbals with drumsticks. Gripping the microphone controller, Gina reminds her mother to use the plastic kick pedal to play the parts accurately. Alex grabs a guitar controller and begins pressing the color-coded buttons that simulate frets on the guitar neck while flicking the small plastic bar with his thumb, which simulates strumming a guitar's strings. He plans to demonstrate his new skill of shifting his left hand down the neck and using his pinky, having practiced on expert mode for hours on end. Javier glances at the keyboard controller before picking another guitar controller up from the floor. He selects the bass part and tweaks his avatar's appearance.' Gina selects Poker Face, using the mode allowing for any song to be played rather than requiring advancement through music of increasing complexity. After each player chooses one of the five available difficulty levels the music begins. Forty seconds into the song dissonant sounds clang through the speakers resulting from Javier's trouble coordinating his fingers' placement on the buttons and rhythmic strumming to the iconic notation scrolling across the screen. A virtual crowd boos Javier's mistakes and Gina's spirited but inaccurate vocal performance. Sasha's and Alex's accuracy keep the song advancing. Alex gesticulates with his body emphatically, clicking and strumming the guitar parts that closely parallel the original song. After completing Poker Face Sasha exclaims "I rocked!" Their scores and accuracy-based-statistics display across the screen as Alex retorts, "Yeah, Mom, on easy mode. Let's see how you do playing some Metal at medium or difficult."

LET'S PLAY!: LEAH

games featuring E player online gam actions; and "seric a game environm off-the-shelf gam«; 360, PlayStation 3: Music-focuse. involve players us the games' visual [ ments range from Konga, respectiveHero and Rock B games provide pll: the musicality of e controllers, and go correctly played p: Physical perfOJ drum sets, and mt microphone contre Players can manipi and phrasing in \\ troller.3 Several gan online game syster: Beyond its cel immersive envirOI tributing to an ov music is an integr. culture seriously ie ence and engagem sitates an understo experience them.• works related to nc porating video gar

MUSIC AND VIDEO GAMES Video games are digital media that can be played on computers with or without online access, handheld devices, and console systems in conjunction with a screen (Squire, 2008). These media take many forms including commercial-off-the-shelf

Warschauer and \"I print-based texts" nologies (p. 215). TI ing making, funct

MUSIC EDUCATION

ar and pedagogical direction. llsic-focused video 2.004; Lankshear & :irning to provide a n music education. nusic plays in video music education gameplay and apter and by others

:lez household. .gs by Lady .It I'm playing :ting the rubber ntroller, Gina i accurately. -d buttons that )arwith his :l demonstrate oS pinky, having keyboard _oor. He selects

19 to be played complexity. s the music 19h the placement on -ling across the _but inaccurate lancing. Alex the guitar :er Face Sasha !splay across ow you do

with or without :tion with a screen

LET'S PLAY!: LEARNING MUSIC

533

games featuring genres such as adventure, sports, or role play; "massively multiplayer online games" (MMOG), which feature open-ended worlds and player interactions; and "serious games," which aim to educate or present information within a game environment (Squire, 2006, 2008). This chapter focuses on commercialoff-the-shelf games, particularly those played on console systems such as the Xbox 360, PlayStation 3, and Nintendo Wii. Music-focused video games, referred to as rhythm action games, typically involve players using controllers to match rhythmic and pitch content dictated by the games' visual interfaces (Miller, 2009). Peripheral controllers simulating instruments range from maracas and bongos in the games Samba de Amigo and Donkey Konga, respectively, to microphones, guitars, and drum sets in the games Guitar Hero and Rock Band or turntables in BeatMania and DJ Hero. Rhythm action games provide players with a sense that they are performing music and enhance the musicality of one's experience through connecting gameplay to visuals, audio, controllers, and game structures, such as the rewarding of higher point values for correctly played phrases, as opposed to individual notes (SqUire, 2011). Physical performance is made possible through the use ofMIDI keyboards, MIDI drum sets, and modified electric guitars in games such as Rock Band 3 along with microphone controllers that reproduce and track the accuracy of a player's live voice. Players can manipulate musical parameters to some extent, such as altering dynamics and phrasing in Wii Music with the Nintendo Wii Remote, a wireless gestural controller.3 Several games allow people in different locations to collaborate or compete via online game system networks in addition to the phYSical space around the console. Beyond its central role in rhythm action games, music is critical to creating immersive environments and virtual worlds in other game genres. Whether contributing to an overarching narrative or providing feedback to a player's actions, music is an integral element of contemporary video games. While taking gaming culture seriously is a first step toward realizing the potential of the musical experience and engagement offered by video games, developing informed praxis necessitates an understanding of these media and how young people engage with and experience them. To this end the following section delineates theoretical frameworks related to new literacies and learning through gameplay that support incorporating video games and virtual worlds in music programs.

CONTEXTUALIZING VIDEO GAMES THROUGH NEW LITERACIES Warschauer and Ware (2008) argue that defining literacy as "the ability to decode print-based texts" excludes ways of communicating afforded by new digital technologies (p. 215). They explain that beyond decoding, literacy "encompasses meaning making, functional use of texts, and critical analysis" (p. 215). In expanding


LET'S PLAY!: LJ

beyond print-based and static notions of literacy to those more inclusive and appropriate for digital media, educators might think in terms of literacies. This is consistent with Eisner's (1991) proposition that schools should provide students with opportunities to engage with multiple forms of literacy that take into account diverse forms of sensation, meaning, and representation. Thinking in terms of literacies is key to realizing the full potential of video games in music programs. Lankshear and Knobel (2006) define literacies as "socially recognized ways of generating, communicating, and negotiating meaningful content through the medium of encoded texts within contexts of participation in Discourses (or as members of Discourses)" (p. 64). Discourses, in this case, are "ways of behaving, interacting, valuing, thinking, believing, speaking, and often reading and writing, that are accepted as instantiations of particular identities ('or types of people') by specific groups" (Gee, 2008, p. 3). The practices and Discourses related to digital media such as video games can be considered "New Literacies" different from and expanding the concept of conventional print-based literacies through the affordances of digital technologies (Lankshear & Knobel, 2006). This perspective can help educators avoid constraining the use ofvideo games within conventional conceptions of music literacy by addressing the ways young people engage with music and their ensuing music literacies. Whereas traditional notions of music literacy focus primarily on reading and writing music through standard notation and discerning musical attributes, digital technologies such as video games allow for new forms of text and ways of"reading and writing" (Gee, 2004). Engaging with these texts involves multimodality, which accounts for the infinite ways multimedia can be layered, morphed, and combined (Kress & Van Leeuwen, 2001). Music video games are prime examples of multimodal texts in that they integrate music and its visual representations, graphics that create an immersive environment, peripheral controllers, and actions in the physical space outside the game itself. Addressing music and learning in the context ofvideo games requires educators to broaden beyond a focus on reading and writing standard notation and grapple with how people interact with, learn, and do music through digital media and multimodal texts. At stake are music educators' ability to capitalize on a significant aspect of young people's popular culture and capacity to evolve with how people engage with and understand music. These goals are best met with an understanding of the Discourses surrounding video games and knowledge of how games afford learning.

their lems, and solu1 experimentatic and related res. mine better so: reach a particu ers' learning at (Gee, 2004, ZOC Gameplay ical space outsi these two space "in-room," the argue that you each other and term "in-world technologies, aJ Those with spaces characte masters, and ev by those who iJ dispersed arnot edge; a diversit ship that is flex: spaces also fast, of games by rna creation of exte. ing on the desig music educator. making. How If

534

LEARNING THROUGH VIDEO GAMES Given the problem-solving tasks, potential for collaboration, extended engagement, and new literacies incorporated in video games, it is productive to conceptualize them as designed experiences (Squire, 2006, 2008). Players learn through

Cor

While many of 1 ducive to learni experiences ami observing and eJ place around ga to help them can the meanings yo process (Squire, that afford learn

USIC EDUCATION

Ife inclusive and )f literacies. This provide students take into account :ig in terms of Iit;ic programs. recognized ways ltent through the :Jiscourses (or as "{ays of behaving, ding and writing, pes of people') by related to digital jfferent from and 1rough the affor.s perspective can :onventional con-ngage with music

lyon reading and :t1 attributes, digimd ways of "reades multimodality, morphed, and )rime examples of sentations, graphrs, and actions in

requires educators ation and grapple .media and multisignificant aspect ow people engage ierstanding of the :s afford learning.

MES

engageuctive to conceplearn through


535

their identification of and engagement with the patterns, generalizations, problems, and solutions that make up the game's virtual world through repeated effort, experimentation, trial and error, and/or using information provided by the game and related resources (Gee, 2007). Failure is designed to encourage players to determine better solutions to a given problem and allows for multiple opportunities to reach a particular goal (Gee. 2008). Video games are thus designed to scaffold players' learning and meaning making, which are situated and embodied in gameplay (Gee, 2004, 2007). Gameplay often extends beyond the virtual worlds ofvideo games to the physical space outside the games. Stevens, Satwicz, and McCarthy (2008) differentiate these two spaces in terms of "in-game," within the virtual game environment, and "in-room," the physical environment in which the game is played. The authors argue that young people learn through their varied "in-room" interactions with each other and make broader connections to other aspects of their lives, what they term "in-world." These interactions can also occur via the internet and mobile technologies, and often include peer mentoring and collaboration. Those with shared interest in video games often organize and create affinity spaces characterized by: a common endeavor rather than one's identity; newbies, masters, and everyone else sharing a common space; the transformation ofcontent by those who interact with it; knowledge that is individualized, distributed, and dispersed among other spaces, such as websites; the encouragement of tacit knowledge; a diversity of possibilities for participating and gaining status; and leadership that is flexible and changing (Gee, 2004, pp. 85-87). Video games and affinity spaces also foster opportunities for players to engage as producers or codesigners of games by modding (modifying) content ranging from simple alterations to the creation of extensions for others to play (Gee, 2004, 2007; Squire, 2008). By drawing on the design elements, structures, and interactions surrounding video games, music educators can play an important role in students' learning and meaning making. How might this occur in music classrooms?

CONSIDERING THE EDUCATOR'S ROLE While many of the aforementioned aspects of video games and gameplay are conducive to learning, the onus is on music educators to recontextualize students' experiences and play in terms of teaching and learning music. This demands observing and employing the Discourses, conversations, and interactions that take place around gameplay along with facilitating and scaffolding students' learning to help them connect their in-game, in-room, and in-world experiences. Mediating the meanings young people make from interacting with video games is key to this process (Squire, 2006, 2008). By drawing on the characteristics of video games that afford learning, music educators might cultivate environments conducive to


LET'S PLAYl; L

play, affinity spaces, and peer interaction where students would regularly experience collaboration, musical problem solving (Wiggins, 2009), a wide degree of autonomy, and opportunities to struggle through challenges before choosing to obtain assistance (Green, 2008). The remainder of this chapter offers possibilities for putting the aforementioned theories to practice.

relate to andlo instruments? ' lacking, it is il ricular, and

536

Blurred til LEARNING MUSIC THROUGH VIDEO GAMES Developing a pedagogy that addresses the affordances of video games and virtual worlds for teaching and learning music compels music educators to mobilize the conceptual frameworks discussed thus far and incorporate musical engagement that intersects virtual and physical spaces. This section addresses performance issues relevant to music video games and suggests how video games might be contextualized and integrated with engaged and critical listening, musical analysis, creation of original music, and discussion of related sociocultural issues. In practice, these ways of engaging and thinking musically overlap and may occur simultaneously and recursively with students' gameplay. The suggestions embedded throughout the following discussion are starting points and invite additional approaches as related research emerges.

Performing In public discourse on music video games, a dichotomy of real and simulated performance between rhythm action games and "real" instruments offers music educators and students opportunities to reexamine notions of performance. Many rhythm action garners feel as if they are performing, though they distinguish between performing music and playing these games (Lum, 2009; Miller, 2009). Gameplay is often performative, with players enacting the role of rock star both virtually, through choices made in the game regarding performance venues, and in reality, through imitating their avatar's characteristics and physically embodying this persona through theatrics and gestures, converting the in-room space into a virtual stage (Miller, 2009; Squire, 2008). Miller (2009) contextualizes this phenomenon as schizophonic performance, in which "the players and their audience join the game designers and recorded musicians in stitching musical sound and performing body back together" (p. 424). She also notes that some players approach the game competitively, focusing more intently on scoring points than on performing. While rhythm action games provide players with musical, performative, and aesthetic experiences, does the process of accurately triggering a song's musical elements on instrument-shaped controllers, as dictated by the game visualizations, constitute musical performance? What aspects of playing rhythm action games

The inclusion Def Jam Raps some cases bll the games rail formance and articulation, ly representation contour, durat consider how t in general. Sin drum, and tur: Toconside listening, com might reflect ( standing thrm learning and t omy between \ and what playE sense of perf01 contribute to t butes and inne related questio musicking.

Expanding Classrooms pr formance of ac tions and comMusic educatQo ship between t dents comparE possibilities, SI controllers' cosion from mu!: not encompasEmbracing thE

P MUSIC EDUCATION


:mld regularly expe19), a wide degree of ; before choosing to er offers possibilities

relate to and/or impact traditional notions of performance on acoustic and electric instruments? Though empirical evidence informing discussion of these issues is lacking. it is important for music educators to wrestle with the philosophical. curricular. and pedagogical issues surrounding performance and video games.

537

Blurred Lines between Playing Games and Instruments

o GAMES

games and virtual ltors to mobilize the musical engagement iresses performance leo games might be Ding, musical analyiocultural issues. In -rlap and may occur suggestions embednd invite additional

real and simulated aments offers music performance. Many they distinguish :009; Miller, 2009). Ie of rock star both mance venues, and ?hysically embodythe in-room space contextualizes this -ers and their audijng musical sound that some players ::oring points than performative, and g a song's musical arne visualizations. Ihm action games

The inclusion of vocal parts in video games such as Rock Band. Guitar Hero. and Def Jam Rapstar share similarities with aspects of musical performance and in some cases blur the lines between games and instruments. 4 Variations between the games raise questions as to their potential impact on a player's musical performance and understanding outside the game environment. Vocal phrasing. articulation. lyric, and pitch accuracy, for example. might be affected by the visual representations of vocal parts. Students could compare and contrast how melodic contour, durational values, and intonation are represented in different games and consider how this might impact their performance and conceptualization of music in general. Similar issues may be considered in relation to using guitar. keyboard, drum. and turntable controllers. To consider the broader implications of rhythm action games for performance, listening, composing, and other forms of musical engagement, music educators might reflect on how young people make meaning and develop musical understanding through interacting with these media. The potential of video games for learning and teaching music is greatly expanded when we look beyond a dichotomy between virtual and real instruments and consider what game systems teach and what players learn. How do representations of pitch and rhythm impact one's sense of performance. nuance. sensitivity, and/or phrasing? How does gameplay contribute to players' awareness and understanding of the music's stylistic attributes and inner workings? Along with serving as foci for future research. these and related questions might be posed to students for reflecting on their gameplay and musicking.

Expanding Notions of Play and Performance Classrooms provide a rich context for students' gameplay to occur along with performance of acoustic and electric instruments, offering opportunities for connections and comparisons to be made between video game and musical performance. Music educators' questions and prompts could help focus attention on the relationship between the game's visuals and what students perform or hear. By having students compare video games and musical performance in terms of their expressive possibilities. such as vocal phrasing or nuanced dynamic change. the games' and controllers' constraints become sites for inquiry rather than rationales for exclusion from music classrooms. A traditional performance paradigm. however, does not encompass the full potential of including video games in music classrooms. Embracing the multimodal and music literacies afforded by this technology might


LET'S PLAY!: LE

lead music educators to integrate aspects ofgamer culture, such as the creation and use of game mods. The in-room space, for example, leaves openings for performance and creative responses that can be explored by music teachers and students (Smith, 2004; Squire, 2008). Technological developments are expanding the types ofcontrollers and instruments that can be used to create and perform music through video games. Students can play music and games simultaneously by using MIDI and acoustic drum sets for percussion parts and strum and pluck strings rather than pressing buttons on standard controllers with the YouRock digital/MIDI guitar, Fender Rock Band 3 Squier, or projects such as OpenChord.org's initiative to mod electric guitars into game controllers. The game Rocksmith allows electric guitars to be used as controllers. These mods and alternative controllers allow for a range ofcreative performance pOSSibilities in conjunction with and beyond traditional gameplay. Providing opportunities for students to think musically while exploring and imagining game mods may sow seeds that some day advance the use ofinstruments as game controllers, increase the sophistication of game interfaces and designs to account for expressive interpretations of music, or allow musical performance to control gameplay and elements. When legato or staccato articulations change how a character traverses a virtual landscape or pitch fluctuations determine an avatar's balance on a treacherous perch, game and musical play may merge in fascinating ways. Might music education playa critical role in the work of future video game developers, programmers, composers, and players? While extensions and expansions of performance offer interesting possibilities for students to learn music, framing music video games exclusively in terms of performance is limiting in scope and misses possibilities of integrating video games in music classrooms or ensembles through other types of musical engagement. The remainder of this section explores the potential of combining listening, analysis, and creation of video game music with gameplay.

game. Smith (2 to one's experif thinking in ter Miller (2009) i< such as their ab themselves pIa] they listened. 1 the game. Videl standing, partil questions draw are listening to elusive or diffic learning with s1 ing and

Engaged and Critical Listening Players use instrument game controllers to play along with the recording of a chosen song. While the sounds emanating from most controllers are limited to clicks and taps, one's performance, if correct, is heard as parts of the song emitted from speakers. The connection between players' performance, aural feedback, and game visuals can be considered a form ofengaged listening (Campbell, 2004; Lum, 2009). Campbell (2004) describes engaged listening as "the active participation by a listener in some extent of music-making while the recorded (or live) music is sounding" (p. 91). She argues that for many students, "listening must be folded into a means of interactive engagement with the music" (p. 91). Expanding beyond a sole focus of comparing video games to musical performance allows music educators to access new vistas of engaged listening in music classrooms. The use of instrument-shaped controllers in rhythm action games provide players with an embodied form of engagement with the music they play in the

EngagedLh of Video Ga

The multimod electric guitars into ars to be used as conmge ofcreative perforInal gameplay. y while exploring and the use ofinstruments :erfaces and designs to Illsical performance to iculations change how sdetermine an avatar's y merge in fascinating ( of future video game

nteresting possibilities usively in terms of pergrating video games in sical engagement. The ing listening, analysis,

ith the recording of a ltrollers are limited to rts of the song emitted ce, aural feedback, and :Campbell, 2004; Lum, active participation by rded (or live) music is ing must be folded into . Expanding beyond a ce allows music educarooms. action games provide nusic they play in the

539

Engaged Listening and Multimodal Affordances of Video Games The multimodal nature of video games affords connections between what one sees, hears, and does in the game context. Auerbach (2010) details how educators can capitalize on these affordances by helping students connect their embodied experience dancing and reading iconic notation while playing Dance Dance Revolution (DDR) to vocalization, sight-reading, and transcribing music with standard notation. Whether players focus primarily on the visual, the aural, or a combination of both aspects of video game music is speculative at this point, given a lack of research; however, "reading" music in music-focused games seems related to engaged listening. Miller (2009) found that players read Guitar Hero notation as sets of patterns, a strategy learned from prior formal music experience or during similar processes while playing the game DDR. This suggests that beyond eye-hand coordination and fast finger work, playing music video games requires a conceptual framework about the game's musical system on which one bases one's physical actions (Schultz, 2008). Schultz (2008) argues that such a theory is based on the way that rhythm action games map musical time to physical space visually. Games that provide several difficulty levels for each song begin with a skeleton of the musical content at the easy level and add additional content, filling in rhythms and pitches as the difficulty progresses, until one must essentially play the original part (Miller, 2009; Schultz, 2008). Thus, particularly on easier levels, one does not necessarily see what one hears. Though this may frustrate a player who knows a particular song and is required to playa simplified version of one instrumental part, it offers countless possibilities for students to sharpen their abilities to identify and distinguish between the varied tonal, rhythmic, and structural layers in a song. This process can be repeated for each difficulty level, in essence, generating a visual, kinesthetic, and aural gestalt.

i

1

I I i ;j

l!

j

540


Listening Critically Video games offer opportunities for students to listen critically and aesthetically when contextualized in a music classroom. Music educators can frame video games to elicit students' aesthetic preferences and musical thinking, ranging from discussing the mixes created by professional DJs for DJ Hero to evaluating vocal performances from the game SingStar or Def Jam Rapstar that were uploaded to a game system network. This could also apply to students' perceptions of the aesthetic qualities and relevance of music in video games beyond the music-focused genres discussed thus far. Huiberts and van Tol's (2007) "pretty ugly game sound study," for example, encourages garners to submit and describe examples of good (pretty) and bad (ugly) video game music to a website that allows for discussion of the archived submissions. Having students critique and discuss video game music in the classroom combines garner culture with opportunities for developing students' listening and analytical skills. Jessie holds the controller, confused by the numerous buttons and knobs. "The last time I played a video game was in high school," she discloses to her partner, Erik. Erik inserts Uncharted 2; Among Thieves in the console system, replying, "This is a bit different, but you'll figure it out." After a brief animated scene provides context to the game's start, Jessie finds herself, as the character Nathan Drake, hanging precariously from a train teetering off a cliffside, surrounded by lightly falling snow and the whispering sound of wind. New to the game and controller, Jessie takes several minutes before ascertaining how to make Drake climb the dangling train's surface. As she fiddles with the controller trying to climb and jump, the music alternates between a symphonic motive echoing the game's theme music, a woodwind theme whispering along with the sounds of wind and creaking wood, and ominous-sounding drums with string instrument tremolos. Turning a corner while clinging to a metal pipe, she (Drake) slips. The dynamics swell, returning to calm as she gains her balance climbing upward. Suddenly, a French horn and strings enter at a fortissimo, swelling tensely as a chunk of the train hurtles down toward her. She traverses the train car quickly, avoiding the falling wreckage. The prior woodwind theme returns at a mezzo piano. "This is intense," Jessie exclaims.

Considering the Functions of Music in Video Games Music contributes to video games' immersive environments and the illusion that a player is outside her reality and in the virtual world of a video game by setting particular moods, contributing to game narratives, signifying emotions, and playing a preparatory function, such as warning a player that something important is about to take place or focusing her attention on particular game elements (Collins, lo08; Zehnder & Lipscomb, 2006). Understanding how music functions in the multimodal context of video games and gameplay can lead to new ways of perceiving and experiencing music through digital media. Situating analysis and discourse in the context of playing games in class might provide concrete entry points to address varied musical issues and music's interactive nature in video games.

LET'S PLAY!: LEARN

Collins (2008) exph tive in nature. She to gameplay, and ir actions. Dynamic a ing to game paramE by responding to a terms of adaptive, i for students to Iistel The ability for 1 and actions require designed so that it ( isting music, divide (Collins, 2008). FOI could change base' object (Collins, 200 garnes' virtual wod dynamic and respot gameplay and focus critique the functiol expresses a particul: to the player's overa 'Two tools used (Collins, 2008), mil analyzing interacti' programmers to ide components and ev Cue sheets do not d music might occur 1 matches the player'! ponents in a game; throughout gamepl and create cue sheenonlinear nature at and for extending analysis.

Creating Origi Music education aJ of original content rooms could funct role of video game tique along with tc ation of interactive

MUSIC EDUCATION

Iy and aesthetically

:s can frame video

king, ranging from to evaluating vocal It were uploaded to :::eptions of the aesthe music-focused y ugly game sound examples of good "'lS for discussion of ; video game music for developing stu-

.d knobs. "The to her partner, :stem, replying, lated scene .aracter Nathan • surrounded ) the game and :> make Drake TIer trying to 'Ve echoing the he sounds of ring instrument rake) slips. The .ing upward. _g tensely as a in car quickly, IS at a mezzo

iffies

ld the illusion that game by setting motions, and plaything important is elements (Collins, ::: functions in the new ways of per8 analysis and disJ.crete entry points 'e in video games.


541

Collins (2008) explains that much of video game music is nonlinear and interactive in nature. She makes a distinction between adaptive audio, which responds to gameplay, and interactive audio, which changes according to a player's direct actions. Dynamic audio, according to Collins, is both adaptive through responding to game parameters, such as a character's health in the game, and interactive, by responding to a player's direct input, such as moving an object. Thinking in terms of adaptive, interactive, and dynamic audio opens new imaginative spaces for students to listen to, conceptualize, create, and perform music. The ability for video game music to change based on a player's varied choices and actions requires that music either be composed in a nonlinear fashion and designed so that it can be reconstructed in multiple ways or, in the case of preexisting music, divided into parts to be rearranged based on responses to gameplay (Collins, 2008). For example, the music playing at a particular point in a game could change based on the character's location relative to another character or object (Collins, 2008). Thinking through music in the nonlinear context of video games' virtual worlds offers students new ways of conceptualizing music that are dynamic and responsive to someone other than a performer or conductor. Through gameplay and focused listening students might engage with, analyze, describe, and critique the functions and qualities of video game music and the degree to which it expresses a particular mood or narrative, matches a game's design and play, or adds to the player's overall experience. . Two tools used by creators of video game music, cue sheets and emotion maps (Collins, 2008), might be used in a classroom context as graphic organizers for analyzing interactive video game music. Cue sheets are used by composers and programmers to identify a game's components, describe the characteristics of the components and events that might take place, and list associated musical content. Cue sheets do not display music occurring linearly in time, instead outlining when music might occur based on a player's interaction with the game and how the music matches the player's actions. Emotion maps represent the various events and components in a game and trace the emotional trajectory as it rises, falls, and plateaus throughout gameplay (Collins, 2008). Providing students with opportunities to use and create cue sheets and emotion maps allows for a concrete representation of the nonlinear nature of video game music and its relationship to games and gameplay, and for extending students' engagement with video games from play to playful analysis.

Creating Original Video Game Music Music education and video game literacies share a common value in the creation of original content by students and gamers (Gee, 2004; Squire, 2008). Music classrooms could function as interdisciplinary studios in which students embody the role of video game composers and use their gameplay, listening, analysis, and critique along with tools such as cue sheets and emotion maps to inform their creation of interactive video game music. Creating original video game music would

542


LET'S PLAYl: LE

require students to make musical decisions that encompass expressing the mood of an event or scene, appropriate leitmotifs or themes for game components, the relationship of musical content to gameplay and other parts of the music to allow for interactivity, logical loop points in the music, and other issues that arise from their engagement with the game. Students' experience and knowledge of video games would thus intertwine with their musical learning and musicianship. Helping students create music for multimodal, nonlinear, and interactive contexts may require music educators to rethink how they approach compositional pedagogy, integrate technology, and teach concepts such as musical structure and development.

Opportuni1 brass qUintets I trollers in avid and raise comp form music in t] across the game use of video ga music from m\ playing Gabriel to be seen.

Creating Music in the Game Environment In addition to composing music contributing to the immersive environments and virtual worlds of video games, students might compose in-game and create music to be played in a game environment. Games allowing one to create, layer, and remix music seem to be moving from console systems to mobile gaming devices and phones; however, the inclusion of composition devices in game environments, such as in Guitar Hero, offer interesting opportunities for students to both play the game and create original music. Since the introduction ofGuita.r Hero: World Tour, the Guitar Hero game series includes GH Mix, a composition toot allowing players to create original music in the game environment. A player using GH Mix can create vocal, keyboard, rhythm guitar, lead guitar, and drum tracks in preexisting or user-created tonal systems using the guitar and drum controllers. The music is notated with Guitar Hero's icon system and can be uploaded to the game's online network for others to download and perform in the game environment on their own console systems. Issues such as sonic aspects of the music, its visual representation, the extent to which it can be played by the composer and others, and the constraints one faces when attempting to realize one's ideal song can be addressed as students create music in-game. Students can also reflect on and consider solutions to constraints such as GH Mix's limited harmonic and timbral options. With a teacher's gUidance, students creating music in video game environments could work through musical problems, eventually transitioning to other media and tools to create their music. In this way the game acts as a site for musical exploration, decision-making, thinking, and learning. The Rock Band Network, created by the video game company Harmonix, allows original music to be translated into Rock Band songs for play on the Xbox 360 system and in some cases the PlayStation 3. Using the software sequencing program Reaper and a Rock Band plug-in,S one can create and record music using both digital audio and MIDI, adapt the song for each difficulty level, assign the musical content to the game's guitar, drum, and microphone controllers, and have the song played through the Rock Band video game. The ability to translate any type of recorded music into a playable song in Rock Band opens exciting possibilities for music education.

Discussing l

Music video gar cultural issues i twined with ga example, avatar; stereotypes. ThE culture, or ethn reflect related n between video dents to develol games play in th larger musical a deeply and critic

If music classr and emergent f and classroom ronment cond throughout thi Amid many po account possib teaching and Ie The first m ments, and re beyond a dich ing the virtual modded music

OF MUSIC EDUCATION

the mood of components, the relathe music to allow for es that arise from their -vledge of video games icianship. Helping stu"e contexts may require lal pedagogy, integrate )d development.

:sive environments and same and create music Ie to create, layer, and :nobile gaming devices in game environments, :udents to both play the

:Juitar Hero game series reate original music in ocal, keyboard, rhythm tonal systems ted with Guitar Hero's ork for others to downonsole systems. the extent to Ie constraints one faces ssed as students create Kllutions to constraints With a teacher's guidts could work through Il.nd tools to create their ition, decision-making,

e company Harmonix, gs for play on the Xbox ie software sequencing and record music using ficulty level, assign the Ie controllers, and have ability to translate any 'pens exciting possibili-

LET'S PLAYl: LEARNING MUSIC

543

Opportunities for students, whether in rock bands creating original music or brass quintets performing baroque works, to have their music played with controllers in a video game environment offer varied entry points into these musics and raise compelling questions about what it means to create, listen to, and perform music in this context. Whether deciding how to distribute brass quintet parts across the game controllers or visualizing the rhythms of an original riff, students' use of video games in the music classroom affords new ways of interacting with music from multiple viewpoints. The implications of creating, arranging, and playing Gabrieli on a plastic guitar controller or samba on rubber drums are yet to be seen.

Discussing Game-Related Issues Music video games provide a wealth of entry points for integrating relevant sociocultural issues in music classrooms. Music educators can tease out issues intertwined with game design and play for critical investigation (Squire, 2008). For example, avatars might be critiqued in terms of gendered and cultural norms and stereotypes. The games' musical themes and timbres used to evoke a sense of place, culture, or ethnicity can be investigated to determine the degree to which they reflect related musics and draw on cultural tropes. Analysis of the relationship between video games, the music industry, and music distribution may assist students to develop sophisticated understandings of the role that music and video games play in their lives. By using video game music as a springboard for exploring larger musical and sociocultural issues music educators can help students think deeply and critically about media in which they immerse themselves.

MODDING MUSIC EDUCATION If music classrooms are to include multiple music literacies and music situated in and emergent from video games and gameplay, changes in pedagogy, curriculum, and classroom structure are in order. What steps might one take to create an environment conducive to the ways of thinking through and being musical discussed throughout this chapter? In other words, how might we mod music education? Amid many possible modifications to music education the following five take into account possibilities that video games and gameplay have for expanding music teaching and learning. The first mod consists ofallowing video games to coexist with other texts, instruments, and resources used in music classrooms and ensembles. This means moving beyond a dichotomy between virtual and physical musical engagement and allowing the virtual worlds of video games to merge with those of music classrooms. In madded music classrooms one might find guitar, drum, keyboard, and microphone


LET'S PLAYI:

controllers next to guitars, drums, keyboards, and microphones; students playing the game Rock Band and in rock bands; or ensembles playing video game music according to the gameplay projected on a screen as the score. Music educators would playa role in weaving these various experiences together to help students make connections between the intersections of game, social, and classroom spaces. Second, we might embrace a mix of musical and gaming cultures, where students work within and across affinity groups on projects connected to their play (Gee, 2004; Squire, 2008). Contextualizing gameplay within curricular projects and drawing on students' situated experience with video games will prevent games from being relegated to disconnected activities. Project-based music classrooms that foster affinity spaces require flexible teaching and an environment in which students' learning emerges from their play and project work, their knowledge is distributed across the class, and connections are made beyond the classroom (Gee, 2004). In such classrooms some students might playa game and analyze its music on one side of the room while others on the opposite side create or perform original video game music, with students moving between groups to collaborate. This may mean moving away from strict sequential plans or disconnected activities and allowing for a degree of spontaneity and learning that emerges from students' play but is contextualized within a project or unit. While logistical issues abound, careful consideration and planning can provide an environment conducive to this type oflearning and play. Third, music educators might build on the affordances ofvideo games, treating their constraints as learning opportunities. Allowing students to experience, think through, and address the limitations of music video games provides a rich context for constructing musical understanding. Music educators can play an important role in this process by helping students identify and negotiate constraints and affordances while giving them space to generate their own understandings through experience. The fourth mod involves embracing multimodality, nonlinearity, and interactivity as they pertain to video games and music. The multimodal and interactive nature of video games provides alternative paradigms for thinking through music in terms of space, structure, action, and other concepts. Classrooms embracing multimodality, nonlinearity, and interactivity would include a wide variety of media and ways for students to engage with music. Taken-for-granted notions such as scores, instruments, musical development, and playing music might be interpreted widely and reframed in terms of new technologies and literacies. Finally, music educators should act as facilitators. The previously mentioned mods, while alluding to an increased degree of student choice and freedom, do not imply that music classrooms would be unstructured or that music educators have no role. It is critical to find an ideal balance between scaffolding and supporting students' learning without interfering in their work and construction of understanding (Green, 2008; Hargreaves, Marshall, & North, 2003). Just as video games have built-in structures and design, music educators have a role to play in scaffolding students' play and learning by assessing their progress, providing feedback and information, and ensuring an optimal balance between the challenges they face

and their ab This includE their ability While i' they could t in it themse studies to cc ways ofbeir play music "

544

Building a r on musicall and games in PO] gameplaybc pathways as 1.

2.

3.

4.

5. 6.

7.

Detel unde Detel learn Leart one's vice' Gain: inter< Deter Learr andg Deter game

This agenda surveys ofga games, musi. t ive research as game stu, commercial! include videl handheld sys

JP MUSIC EDUCATION

Jnes; students playing ng video game music -:tusic educators would lp students make con:lorn spaces. g cultures, where stumnected to their play -urricular projects and JI prevent games from :c classrooms that fosent in which students' ::>wledge is distributed ;room (Gee, 2004). In e its music on one side :n original video game his may mean moving d allowing for a degree • but is contextualized consideration and oflearning and play. :video games, treating ts to experience, think ; provides a rich con,rs can play an impornegotiate constraints own understandings

mlinearity, and interItimodal and interacfor thinking through :. Classrooms embraclude a wide variety of -granted notions such nusic might be inter;i literacies. ?reviously mentioned and freedom, do not music educators have Iding and supporting mstruction of under). Just as video games >Ie to play in scaffoldoviding feedback and . challenges they face


545

and their abilities to work through the issues at hand (Gee, 2007; Wiggins, 2009). This includes encouraging students to reflect on their experiences and facilitating their ability to make connections between gameplay and musicianship. While it is not necessary for music educators to become video game experts, they could benefit from becoming literate in gaming and its culture by engaging in it themselves. Collaborating with colleagues in disciplines ranging from game studies to computer programming could further one's knowledge and forge new ways of being musical (Lum, 2009). In modding music education we might then play music and video games with our students.

BUILDING A RESEARCH AGENDA Building a research agenda that seeks to better understand video games' impact on musical understanding and experience as well as their potential use in teaching and learning music will assist music educators in addreSSing the ubiquity of video games in popular culture and young persons' lives. This requires research of video gameplay both in and out of school contexts. The following areas suggest pOSSible pathways as starting points on which interested researchers may expand: 1. 2.

3.

4.

5. 6.

7.

Determining how video games affect the development of students' musical understanding Determining what students are learning about music and how they are learning music through engaging with video games Learning if and to what extent playing music in video games impacts one's performance abilities in musical contexts outside video games and vice versa Gaining a deeper understanding of young peoples' Discourse and social interactions surrounding their engagement with music video games Determining affordances and constraints of music video games Learning what takes place in music classrooms that integrate video games and gaming culture Determining how music education and musicianship can inform video game development

This agenda requires a broad spectrum of research methods. Whether conducting surveys ofgarners or ethnographic studies of music classrooms that integrate video games, music educators ought to consider working on interdisciplinary collaborative research projects benefiting from shared expertise of colleagues in fields such as game studies and musicology (Lum, 2009). While this chapter has focused on commercial off-the-shelf games for console systems, a research agenda should also include video games that are played on computers and mobile platforms such as handheld systems and smart phones.


546

LOOKING FORWARD Squire (2008) suggests that

LET'S PLAY!: LE

at the role of au tualizing notio (2006) book on

as games become more culturally entrenched, the idea of using games in education may be passing from an opportunity to an imperative, if we are to create an education system that adequately prepares students for life in an information/knowledge rich economy. (p. 663)

Music educators should consider what it means to teach and learn music as musical engagement becomes increasingly multimodal and interactive. Though video games are neither equivalent to nor replacements for traditional forms of musical performance, listening, analysis, or creation, they are an interactive medium with potential to transform how young people engage with and understand music. Integrating video games in music education requires music educators' willingness to design experiences and contextualize students' gameplay within broader conceptions of musical literacies, engagement, and learning (Gee, 2004; Squire, 2006, 2008). It is up to music educators to adapt to these societal changes in how people learn music and mOdify their pedagogies and curricula in a manner that is thoughtful and informed.

1. All references

World Tour ar 2. An avatar is tt 3. The Move and respectively, p movements. 4. While these vi ratings, the m; appropriate in 5. A plug-in is a I software progl

REFLECTIVE QUESTIONS How might you connect to, recontextualize, and capitalize on the interest and knowledge students bring with them as a result of their immersion in video games and virtual worlds? 2. In what ways do you or could you mod your pedagogy and curriculum to reflect the kinds of experiences and learning that new literacies, video games, and virtual worlds provide? What steps might you need to take for this to occur? 3. Where are the spaces in your curriculum in which game culture and music-making could take place simultaneously? 4. How might your program inform and advance the work of future video game designers, programmers, and composers?

1.

Auerbach, B. (20 to aural skill societymusi. Campbell, P. S. (. York: Oxforc Clements, A. c.,

in virtual arl

Education }; Collins, K. (2008

game music Eisner, E. (1991). Gee, J. P. (2004). York: Routle Gee, J. P. (2007). updated Gee, J. P. (zo08). Green, L. (2008) Aldercott, l: Hargreaves, D. J. first centur)" 147-163· Harwood, E. (19!

00.:

KEY SOURCES To expand one's perspective on video games and education I recommend Gee's (2007) book on videogames and learning, as well as Squire's (2008) chapter on video game literacy. Collins's (2008) book Game Sound offers an excellent in-depth look

Music Educe

SIC EDUCATION

LET'S PLAYI: LEARNING MUSIC

547

at the role of audio in video games. Those interested in broadening and reconceptualizing notions of literacy would benefit from reading Lankshear and Knobel's (2006) book on new literacies.

es in 'e are to :in an

nusk as musical I1gh video games . musical perform with potential ntegrating video :l design experitions of musical It is up to music usic and modify nformed.

NOTES All references to Guitar Hero in this chapter include the fourth iteration, Guitar Hero: World Tour and the versions that followed, up to and including Band Hero. 2. An avatar is the character representing the player in a video game. 3. The Move and Kinect, gestural controllers for the PlayStation 3 and Xbox 360, respectively, portend an expanSion of music-focused video games controlled by players' movements. 4. While these video games have Entertainment Software Rating Board (ESRB) "Teen" ratings, the majority of music in Rapstar contains explicit language that may not be appropriate in certain school contexts. 5. A plug-in is a software program that can be used within the environment of another software program. 1.

REFERENCES on the interest • immersion in &::urriculum video eed to take for Iture and :uture video

:>mmend Gee's Japter on video : in-depth look

Auerbach, B. (2010). Pedagogical applications of the video game Dance Dance Revolution to aural skills instruction. Music Theory Online, 16(1). Retrieved from http://mto. societymusictheory.orgiissues/mto.lo.16.1!mto.1O.16.l.auerbach.html. Campbell, P. S. (2004). Teaching music globally: Experiencing music, expressing culture. New York: Oxford University Press. Clements, A. c., Cody, T., & Gibbs, B. (2008). Interactive gaming: Musical communities in virtual and imagined worlds. Paper presented at the Cultural Diversity in Music Education Nine, Seattle. Collins, K. (2008). Game sound: An introduction to the history, theory, and practice ofvideo game music and sound design. Cambridge, MA: MIT Press. Eisner, E. (1991). Rethinking literacy. Educational Horizons, 69(3), 120-128. Gee, J. P. (2004). Situated language and learning: A critique oftraditional schooling. New York: Routledge. Gee, J. P. (;W07). What video games have to teach us about learning and literacy (rev. & updated ed.). New York: Palgrave Macmillan. Gee. J. P. (2008). Social linguistics and literacies (3rd ed.). London: Routledge. Green, 1. (2008). Music, informal learning and the school: A new classroom pedagogy. Aldercott, UK: Ashgate. Hargreaves, D. J., Marshall, N. A., & North, A. C. (2003). Music education in the twentyfirst century: A psychological perspective. British Journal ofMusic Education, 20(2), 147-163· Harwood, E. (1998). Music learning in context: A playground tale. Research Studies in Music Education, 11(1), 52-60.


Huiberts, S., & van Tol, R. (2007). Pretty ugly game sound study [website]. Retrieved from http://prettyuglygamesoundstudy.com. Kress, G., & Van Leeuwen, T. (2001). Multimodal discourse: The modes and media of contemporary communication. London: Arnold. Lankshear, c., & Knobel, M. (2006). New literacies: Everyday practices and classroom learning (2nd ed.). Berkshire, UK: Open University Press. Lum, C.-H. (2009). Learning from digital natives: Children's thoughts on music video gaming. OrffEcho, 42(1),25-29. Marsh, K, & Young, S. (2006). Musical play. In G. E. McPherson (ed.), The child as musician: A handbook of musical development (pp. 289-310). New York: Oxford University Press. Miller, K (2009). Schizophonic performance: Guitar Hero, Rock Band, and virtual virtuosity. Journal ofthe Society for American Music, 3(4), 395-429. Schultz, P. (2008). Music theory in music games. In K Collins (ed.), From Pac-Man to pop music (pp. 177-188). Aldershot, UK: Ashgate. Smith, J. (2004). I can see tomorrow in your dance: A study of Dance Dance Revolution and music video games. Journal ofPopular Music Studies, 16(1), 58-84. Squire, K. (2006). From content to context; Videogames as designed experience. Educational Researcher, 35(8), 19-29. Squire, K. (2008). Video-game literacy: A literacy of expertise. In J. Coiro, M. Knobel, C. Lankshear, & D. J. Leu (eds.), Handbook ofresearch on new literacies (pp. 635-670). New York: Lawrence Erlbaum. Squire, K. (2011). Video games and learning: Teaching and participatory culture in the digital age. New York: Teachers College Press. Stevens, R., Satwicz, T.• & McCarthy; 1. (2008). In-game, in-room, in-world: Reconnecting video game play to the rest of kids' lives. In K Salen (ed.), The ecology ofgames: Connecting youth, games, and learning (pp. 41-66). Cambridge. MA: MIT Press. Warschauer, M., & Ware, P. (2008). Learning, change, and power: Competing frames of technology and literacy. In J. Coiro, M. Knobel, C. Lankshear, & D. ]. Leu (eds.), Handbook ofresearch on new literacies (pp. 215-240). New York,: Taylor and Francis. Wiggins, J. (2009). Teachingfor musical understanding (2nd ed.). Rochester, MI: Center for Applied Research in Musical Understanding. Zehnder, S. M., & Lipscomb, S. D. (2006). The role of music in video games. In P. Vorderer & J. Bryant (eds.), Playing video games: Motives, responses. and consequences (pp. 241-258). Mahwah, NJ: Lawrence Erlbaum.

Gl

Music is inherentJ pIes of technicall] before them, provi chapter we oudino well-established b. gogical value by Ie Digital media using a computin the practice invoh than direct produ. for acoustic soune such as Ableton L formance ofmusio manipulation, sot artists can create ware employs gen