StrikeCOM: A Multi-Player Online Strategy Game ... - Semantic Scholar

Proceedings of the 38th Hawaii International Conference on System Sciences - 2005

StrikeCOM: A Multi-Player Online Strategy Game for Researching and Teaching Group Dynamics∗ Douglas P. Twitchell

Karl Wiers Mark Adkins Judee K. Burgoon Jay F. Nunamaker, Jr. Center for the Management of Information University of Arizona {dtwitchell, kwiers, madkins, jburgoon, jnunamaker}@cmi.arizona.edu Abstract

StrikeCOM is a multi-player online strategy game designed to create discourse to aid in the examination of the development of group processes, shared awareness, and communication in distributed and face-to-face groups. The game mimics C3ISR (Command, Control, Communication, Intelligence, Surveillance, Reconnaissance) scenarios and information gathering in group activities. The game is most commonly used to create group communication and interaction in multiple communication modes. Built using a Javabased collaborative server platform the game is available for use in almost any computing environment. StrikeCOM has been used as a research tool to study leadership and deception in group decision making. The U.S. Department of Defense is using the tool to teach Network Centric Warfare to battle commanders. Use of StrikeCOM over the last two years has resulted in a number of lessons-learned, including using simple, familiar game interfaces, utilizing full and immediate feedback, and creating a flexible technical design to meet shifting research and teaching needs.

1. Introduction StrikeCOM is a multi-player networked computer game designed and built by the Center for the Management of Information (CMI) at the University of Arizona. The object of StrikeCOM is for a team of players, each with different capabilities, to find targets or information hidden on a game board within a predefined number of turns. StrikeCOM represents a merger between collaborative software tools ∗ Portions of this research were supported by funding from the U. S. Air Force Office of Scientific Research under the U. S. Department of Defense University Research Initiative (Grant #F49620-01-1-0394). The views, opinions, and/or findings in this report are those of the authors and should not be construed as an official Department of Defense position, policy, or decision.

engineered by CMI and gaming concepts from the Center for Naval Analysis’ ScudHunt [9], developed as part of an experiment for the Defense Advanced Research Projects Agency. StrikeCOM is most commonly used as an online turn-based strategy game known as a C3ISR (Command, Control, Communication, Intelligence, Surveillance, Reconnaissance) task designed to examine the development of shared awareness and communication in distributed groups. CMI created the StrikeCOM game to meet requirements for advanced data collection and customizability to investigate deception detection within large groups of people. StrikeCOM is designed for flexibility and allows research into any group process or interaction. StrikeCOM provides a multiplayer game capable of supporting any number of players using any number of assets searching a game board of any definable size for any number of targets and target types that the researcher chooses to define. The game includes time stamps for each player interaction with the game. In addition, the game includes map overlays with adjustable geographic information reliability statistics, the ability to have multiple independent target types, and an optional shared results visualization scheme. The game also allows the researcher to configure the game such that any or all of these features can change as the game progresses. Wargames such as StrikeCOM have been around for decades, and a number of principles of wargaming have been developed both by practitioners and researchers. Perla [8] lays out five fundamental principles of wargame design, which are also similar to the first five of Dunnigan’s [7] ten steps to wargame design. The five principles include: (1) specify objectives; (2) identify players, their game roles, and the decisions they will be expected to make; (3) determine and collect the information and determine the information feedback they will need to make those decisions; (4) devise the tools needed to make the process work; and (5) document the results of the effort. The game’s design reflects each of these principles.

0-7695-2268-8/05/$20.00 (C) 2005 IEEE

1

Proceedings of the 38th Hawaii International Conference on System Sciences - 2005

Objectives. StrikeCOM’s original objective was a decision-making task for researching group communications by requiring research subjects to communicate toward a goal. This task has proven to be well suited for generating group communications. An additional objective of StrikeCOM that has evolved over time is to aid in the teaching of group communication. Roles. The players, their roles, and the decisions required of them are flexible depending on the setup. Information.The evaluation phase, described in Section 2.2 plus the instructions given at the beginning of the game comprise the information the players need to make their decisions. Tools. The Java-based collaborative server and client software are the tools that makes the game process work. Documentation. Along with some limited technical documentation, this paper is an attempt to comply with this principle. Section 2 outlines the process of playing the game including the search, evaluation, and strike phases. Section 3 explains how the software creates the GUI and interaction between the players. Sections 4.1 and 4.2 each describe the experiences we have had with StrikeCOM and its predecessors. Finally, Section 5 concludes with the lessons learned from these experiences.

2. Game Process In the most common C3ISR configuration of the game, a team of three players works together during a fixed number of turns to find three targets that have been hidden on a six-by-six grid game board representing a hypothetical hostile state. Initially, all players receive game instruction through a 10 minute online video. Each player is assigned one or more information assets (e.g., Satellite, Spy, UAV, etc.) with different search capabilities and sensor reliability. The players go through three phases during the game: the search phase, the evaluation phase, and the strike phase. The search phase and evaluation phase are repeated over a number of turns while the strike phase usually happens only once at the end of the game. During each of these phases, the players must coordinate their team strategy through one or more communication media, including face-to-face, audio, video, and the built-in chat window.

2.1. The Search Phase During the search phase, shown in Figure 1, each player on the team must place his or her information assets on the

Figure 1. An Example search phase of StrikeCOM

board. Each asset has different capabilities. In one of the scenarios, for example, the player in charge of the intelligence assets has a “message interceptor” asset and a “spy” asset. The message interceptor asset can search three vertically consecutive cells of the grid, but the spy can only search one. However, depending on the research or teaching goals of the particular game, the spy may have a higher probability of returning a correct result. StrikeCOM includes a number of possible rules for assets such as restricting them to rows, columns, and grid edges.

2.2. The Evaluation Phase After the players have placed all of their assets according to their team’s strategy, they submit each of their assets, after which the game returns the search results to the players. In the most used configuration, there are three types of results returned for each cell searched: likely nothing found, maybe something found, and likely something found. Likely nothing found is represented by a green check, maybe something found is represented by a yellow diamond, and likely something found is represented by a red “X” as shown in Figure 2. The results are returned based on probabilities assigned to the cells, the targets, and the assets. All of the empty cells and targets have a probability for returning each of the results depending on whether a target has been assigned to the cell. Assets also have a probability for each of the results, and when a cell is search the result returned is calculated by a random number compared to the average of the cell’s or

0-7695-2268-8/05/$20.00 (C) 2005 IEEE

2

Proceedings of the 38th Hawaii International Conference on System Sciences - 2005

Figure 2. An example evaluation phase of StrikeCOM

target’s probabilities and the assets probabilities. The asset and cell probabilities are combined so that each has an influence on the outcome. Some cells might be more difficult to search because of terrain or foliage, and an asset such as a human intelligence agent might be more accurate than a spy satellite. Table 1 shows an example probability set up and the effective probability of returning each result. element cell

asset

average

result yes maybe no yes maybe no yes maybe no

target .7 .2 .1 .5 .4 .1 .6 .3 .1

no target .05 .1 .85 .05 .2 .75 .05 .15 .80

Figure 3. An example evaluation phase of StrikeCOM with shared visual

2.3. The Strike Phase The final phase of the game is the strike phase. During this phase, the players attempt to combine the search results from all of the previous turns and decide on the cells to strike. Each player submits his or her own strike plan independent of the other players; however, the game scores are calculated for each individual player and the team as a whole. Because some assets return unreliable results, the team must communicate their individual results to interpret the aggregate board. Upon completion of the strike phase, scores are calculated and displayed for each player and team based on the number of correct target hits compared to the total number of squares destroyed. A perfect score for an individual results from identifying all targets correctly without incorrectly identifying any cells. The team score is calculated as follows:

Table 1. Probabilities assigned to each of the elements of the game along with the average probabilities used in the final calculation of results

total number of hits total number of hits + total number of misses

Optionally, the game can return an aggregate view of the search results on a pane called “Shared Visual”, shown in Figure 3. This view is often turned on and off for teaching and research purposes since its shared view of the game reduces the need for communication between the players of the game. When added later in the game or a subsequent game, players usually notice the added advantage of shared situational awareness.

The scenario described in this section is only the one most usually used. In addition to the basic military-centric C3ISR scenario the game can be modified to fit almost any information collection and sharing situation. One version of the game takes place in the context of an US Air Force Air Operations Center and utilizes detailed maps, asset definitions, and probabilities. Another scenario places the player into the role of a paparazzi searching for Elvis Presley on the streets of Las Vegas.

0-7695-2268-8/05/$20.00 (C) 2005 IEEE

3

(1)

Proceedings of the 38th Hawaii International Conference on System Sciences - 2005

Figure 5. The property tree representing the StrikeCOM game Figure 4. An example strike phase of StrikeCOM

3. Technical Design StrikeCOM is built upon the same collaborative framework as Collaboratus, a Java-based software suite designed to support group collaboration [3]. The collaborative framework itself is based on the idea of a “property.” A property represents a single object in any collaborative system built in the framework. These objects are organized into a hierarchy that reflects the user interface of the collaborative system and can be stored in any SQL database. In StrikeCOM, for example, each turn is represented on the server as a property. The turn property has player properties as children who in turn have asset properties as their children. The clients can connect to these properties that exist in a single place in memory on the server. Any changes made by a user to a property from one client will be propagated as events to all other clients that have also connected to the same property. This setup allows for the events of the game to be efficiently transmitted to all players independent of the user interface they might be using. Figure 5 shows the property tree that represents a game on the server. This tree is also representative of the graphical interface shown in Figures 1, 2, 3, and 4. A game (and there can be many games) has some number of turns, each of which has some number of players. Normally a user has rights to just one player, but he or she may have rights to any subset of the players. The user can control each of the players he or she has rights to. Each player, in turn, has a number of assets each with potentially different abilities. The assets have a grid, upon which a number of potentially different targets are placed.

The number of turns, players, assets, etc. is limited by the server’s and clients’ memory and speed capabilities. The game is also set up to maximize flexibility since experimental requirements often change from study to study. As described above, many new game types can easily be defined. Each game type is defined in an XML file that prescribes the number of turns, the size of the board, the images involved, the number and type of players and assets, and the probabilities involved in the playing of the game. This flexibility has been instrumental in using StrikeCOM for several different studies and learning experiences. In the future we hope to extend this flexibility by allowing a confederate to change the game setup as each turn progresses allowing for moving targets and destroyed assets. The collaborative server, through which all game communications occur, contains a persistence layer that saves each new or changed property and removes deleted ones as they are added, changed, or deleted during the game. The persistence allows the clients and server to be turned off either voluntarily or involuntarily at any time and the game will remain in the same state when later turned back on. This persistence allows the game to be used asynchronously across time with the clients and/or server being turned off and on, and it provides a measure of fault tolerance against operating system errors and power failures. Unfortunately, during intense game play with large number of simultaneous groups, certain Relational Database Management Systems (RDBMSs) may behave sluggishly when faced with the large number of transactions the game produces. For example, when we used Microsoft SQL Server 2000 with approximately 10 simultaneous groups of 3 (30 total players), the RDBMS responded with latency of up to 10 seconds, which is unacceptable in a game environment. How-

0-7695-2268-8/05/$20.00 (C) 2005 IEEE

4

Proceedings of the 38th Hawaii International Conference on System Sciences - 2005

ever, the same number of players had no perceived latency when the RDBMS was switch to MySQL 4.0. The possible reasons for this are numerous, though a likely explanation is the difference in which the two RDBMSs handle transactions–MySQL 4.0 doesn’t use transactions while MS SQL 2000 has an elaborate transaction tracking system. Therefore, any researcher or trainer planning to use StrikeCOM should take into account the abilities of the various SQL servers available. After the game is complete, it can be saved to an XML file which can be used for research or student evaluation purposes. Details about the timing and placement of each of the assets and turns is recorded for later analysis. Additionally, the chat logs are saved. For example, Twitchell, Nunamaker, and Burgoon [11] used these chat logs to uncover uncertainty caused by deception in conversations.

• LRI - Leader appointed, team told that leader controls reliable information • L - Leader appointed, no information concerning information reliability given • RI - No leader appointed, information concerning information reliability given

4. Experience StrikeCOM has been used for two main purposes since its introduction. It was originally built as a tool for researching group interactions, and was useful in several research studies dealing with leadership and deception, which are described in Section 4.1. However, it has turned out that StrikeCOM is also useful in teaching and training environments. The U.S. Department of Defense has recently used StrikeCOM as a tool for teaching its concept of networkcentric warfare, which is explained in Section 4.2.

4.1. Research StrikeCOM was originally created as a research tool. It has been used in two experiments looking studying the effects of both leadership and deception. However, before CMI developed StrikeCOM, two other games, SCUDHunt and BunkerBuster were used for similar research. 4.1.1

use of this form of distributed gaming and experimentation we began to recognize limitations in SCUDHunt’s design. We conducted a five cell experiment examining the effects of the presence of a leader and the presence of reliable task-specific information have on the performance of a team playing a customized computer game. Four-person teams played SCUDHunt with one of the participants as a leader for some of the conditions. Information was provided concerning the reliability of the information one of the players possessed in some of the conditions. The conditions studied, were:

SCUDHunt

The first experiment task used was a modified version of a game that preceeded StrikeCOM called SCUDHunt originally developed by the Center for Naval Analyses as part of an experiment for the Defense Advanced Research Projects Administration (DARPA) [9]. Like StrikeCOM, the game is an online, turn-based strategy game known as a C3ISR (Command, Control, Communication, Intelligence, Surveillance, Reconnaissance) task originally designed to examine the development of shared awareness in distributed groups. The object of the game is for a four-person team, each with different search capabilities, to find three SCUD launchers hidden on a game board with a predefined number of turns. Each player controlled different search assets with different capabilities. As we became more familiar with the

• X - No leader appointed, no information concerning information reliability given • LoRI - Leader appointed, team told that subordinate controls reliable information Four players were necessary to play a single game for the experiment. There were two types of players involved in this experiment. To emphasize the leader role, we randomly selected a Masters of Business Administration student to participate in each group as the leader, since MBA students often stand out from other masters students and undergraduates in that they are primarily professionals with existing leadership experience. The other three students were randomly selected undergraduate students. Students volunteered for the experiment or were offered credit in their classes to participate. None of the students had ever played the game before or had prior knowledge of the conditions being studied. In total we used 30 MBA students and 90 undergraduate students for a total of 30 groups across conditions. The players were randomly assigned to teams from a pool of undergraduate and MBA students. MBA students were assigned to specific leadership roles based on the condition we were running. Undergraduate students were randomly assigned to each team and role. All games were played in a decision support lab designed for group support system experimentation at the Center for the Management of Information. The large lab consists of 30 computers arranged in a semicircle with large display screens at the front of the room. We assigned seats to each role for each game to maximize distance between participants for each game. Additionally, we always ran at least two games simultaneously to minimize the ability of subjects to identify and communicate with their teammates.

0-7695-2268-8/05/$20.00 (C) 2005 IEEE

5

Proceedings of the 38th Hawaii International Conference on System Sciences - 2005

The size of the room allowed us to run up to seven simultaneous games which we restricted to a single condition. Due to subject scheduling constraints, we normally ran between three and five groups at a time. Teams running simultaneously all received the same condition through instructions provided at the start of the game. Due to the complexity of the task, we were forced to develop specialized training material. Printed instructions were distributed and a training video was shown. Condition instructions were given at the end of the training video and some leadership instructions were given in the assigned seating process. After the game completed, subjects were given their game results and asked to complete the post-task survey. The location of the scud launchers was randomized for each game so that even if one team completed the game early, other players in the room could not cheat by looking at the results on their neighbors’ screens. This experiment exposed a few problems concerning the experimental task and methods. First, the lab space and computer configurations used limited us in the distribution of training material and condition assignment. Some training material and condition manipulations were limited to paper, while overall game instructions were given to all participants simultaneously via video projected at the front of the room. Since the training video was shown to all participants simultaneously, we had to wait for all participants to complete their pre-test surveys before we could show the video. This limited all participants to the speed of the slowest person in the room. We also found that attempting to schedule participants from different experience groups, graduate and undergraduate, caused us to lose many subjects, for if we did not have one graduate student for every three undergraduate students, we would have to turn away some subjects. Similarly, if we did not have an exact multiple of four participants we would have to turn away subjects. From this experiment we decided to design a new laboratory and attempt a new study to test some of the lessons learned. 4.1.2

BunkerBuster

We next developed a modified version of the SCUDHunt game, called BunkerBuster, which was developed to address some of the limitations of SCUDHunt and to test some of the experiment design principles needed when using this game type for experimental research. BunkerBuster simplified some of the complexity found in SCUDHunt to allow easier analysis of results, and improved some of the participant observation data collected. In conjunction with the new game, we also configured a new laboratory specifically designed to address the limitations we encountered in the SCUDHunt study. The new laboratory space allowed us greater control over participants, provided many more ob-

servation capabilities, and gave us much more flexibility in our group designs and manipulations. BunkerBuster maintained many similar elements to SCUDHunt. BunkerBuster remained a four player game and each player had different assets, search capabilities, and search reliabilities. What changed from SCUDHunt was the simplification of the story surrounding the game and the encapsulation of the game in a website designed to walk each participant through the entire experiment process. Participants would sit down at a laptop computer attached to a wireless network. The lab assistant would log the participant into the web site based on an id assigned when the participant signed up for the experiment. From this point forward the participant would remain anonymous. All experiment sign-up and identifying information was stored in a database separate from all experiment data. Once logged in, the participant would step through a series of online consent forms, pre-test surveys, game instructions, game, posttest surveys, and out brief. The most important change in our procedures from the SCUDHunt study was that now all game instructions were delivered via pre-recorded video to each participant through their individual computer. Participants wore noise canceling headsets and their computer screens faced away from all other participants. This allowed us to deliver each participant customized instructions including any experimental manipulations. Each player would then connect to the game and commence playing with their team. The BunkerBuster study examined deception in text, audio, and face-to-face communication modalities. One player was given instructions that they were to do everything possible to stop their team from finding targets hidden on the game board. The location of the targets and the instructions were given at the end of the game training video. The participants would play through a single five turn game, separated from their team members by cubicle partitions and communicating with their team through a text chat application or headsets with microphones (boomsets), or face-to-face with their teammates across a round table discussing the game directly. This study was conducted in our new laboratory designed to provide an environment for distributed group work in audio, video, text-chat, and face-to-face modalities. All interactions between participants could be recorded for later analysis. Audio was captured using a professional multichannel digital recording system. This audio system allowed us to record eight individual participants simultaneously. Each participant’s audio could be routed between any other participants. This audio system allowed us to analyze a single individual’s voice without other interference from other group-members voices or outside noise. We also had the ability to relay instructions to participant’s headsets from a central control room if needed. In face to face

0-7695-2268-8/05/$20.00 (C) 2005 IEEE

6

Proceedings of the 38th Hawaii International Conference on System Sciences - 2005

conditions, the group was video taped for later analysis using digital cameras from two angles and combined to one recording using a video mixing board. The use of BunkerBuster combined with the implementation of the new lab environment exposed yet more limitations in our methods and software. The use of a fourplayer game was far too limiting for our purposes. In addition, we clearly needed to let the participants try to play the game by themselves before we made them play with their team. Also, as each player had different search capabilities throughout the game, it was difficult to say that all of the players were on equal footing. We needed huge numbers of players and games to cancel out the many factors introduced by between-player differences. Because of BunkerBuster’s close ties to the design of SCUDHunt, the expandability and long-term usefulness of the BunkerBuster game was limited. From BunkerBuster and SCUDHunt we learned enough about the game systems and experiment procedures to design the game framework called StrikeCOM. 4.1.3

First StrikeCOM Experiment

The first experiments we ran with StrikeCOM focused on deception and leadership. Combined into a single data collection series, we ran over 200 games, over 600 undergraduate participants, for three separate simultaneous studies. A multi-cell experiment was conducted examining the effect the presence of a leader and the presence of reliable task-specific information have on the performance of a team playing a customized computer game. Three-person teams played a networked, turn-based strategy game where each participant played a different role and controlled different information. Players communicated with each other through a simple, online text-chat application or through audio headsets and were unaware of who the other members of their team were or where they were located. Neither the presence of a leader nor the knowledge of role importance/information reliability was necessary to satisfactorily complete the game. We chose to assign one of the participants as a leader for some of the conditions, and some conditions instructed one player to work against their team and deceive their teammates. Conditions involving a deceptive player are used to create examples of deceptive communications. All of the conditions are enumerated in Table 2. A few changes were made to procedures based on past experience. The task required subjects to play two games of StrikeCOM as opposed to the single game in previous studies. The first game was played alone and lasted three turns. The second game was played with the same teams and lasted five turns. This greatly improved the participants’ understanding of the game before they began working with their team. Video data collection was also modified to collect each individual with a separate camera.

A - Control B - Deception C - Emerging Leader D - Leader, Distributed, Strategy E - Leader, Distributed F - Leader, Collocated, Strategy G - Leader, Collocated

Text X X X X X X X

Audio X X X X

FTF X X

Table 2. First experiment design For this round of studies we introduced new symbols to indicate search results. This proved problematic as we did not think about how our symbol choices would impact later analysis. For example, an uncertain result was indicated by a question-mark in a yellow diamond. When players in text conditions saw this they would abbreviate by simply typing “?” which caused havoc with our automated text analysis tools. For future versions of the experiment we removed the “?” from the symbol. Care must be taken when choosing icons so they are not ambiguous and do not cause problems with future analysis. 4.1.4

Second StrikeCOM Experiment

The next experiment we ran centered on deception in the face of a suspicious team member. For this study Air Force Reserve Officer Training Corps (ROTC) students participated in a simulation of an Air Operations Center (AOC). This version of StrikeCOM included a map, military language in the instructions and game interface, and new manipulations of participants during the video instructions. Previous deception studies had a single player deceiving the team and the rest of the team unaware of their activity. In this study, one of the non-deceivers was warned that there was a possible deceiver on the team. The results of this study are reported elsewhere in this conference [4].

4.2. Teaching The networked computer is changing all facets of the military today. The increased emphasis on high technology in the military and increasing pace of change requires the reevaluation of the accepted ideas that technologists hold with regard to the successful development and fielding of automated systems, networks and collaborative tools. CMI partnered with the Department of Defense Office of Force Transformation to develop StrikeCOM as a wargame which will allow users to experience the tenets of Network Centric Warfare (NCW).The leading edge theory on military operations is NCW [2] and requires changes in thinking on how to accomplish missions, interrelate, communicate and acquire systems to support military actions. Adkins and Kruse’s [1]

0-7695-2268-8/05/$20.00 (C) 2005 IEEE

7

Proceedings of the 38th Hawaii International Conference on System Sciences - 2005

simplified description of the Technology Transition Model [5] is that successful adoption of new technologies is based primarily on two factors: (1) Perceived Net Value, the benefit that warfighters expect to experience each time they use a technology combined with the frequency that they anticipate using the technology; and (2) Perceived Complexity, the cognitive effort associated with using the technology. Cebrowski and Garstka [6] in outlining the concept of NCW brought lessons learned in business, economics and technology into the realm of the warfighter. The basic premise was that the leveraging of networks was driving massive changes throughout the world and the military could take advantage of the power of the network to become more responsive, flexible and lethal. NCW has four pillars: 1) Gain information and knowledge advantage; 2) Assured access; 3) Effects based operations; and 4) Forward based forces that give raise to Five tenets: 1) Knowledge of the adversary; 2) Shared situational awareness; 3) Commanders intent; 4) Decentralized execution and 5) Self synchronization. CMI is now working with Cebrowski and Garstka on a series of NCW workshops with case studies of actual operations and training exercises using StrikeCOM. To illustrate the tenets of NCW, the game has been tuned so teams of 3 officers or civilians play using 3 communication media. The first 3 turn game has players sitting next to each other and talking face-to-face, the next 3 turn game is played using chat only with players who are anonymous. These two game situations are common experiences in actual tactical and operational military interactions. Hence, post-hoc analysis of game scores, communication channel, player behavior and interaction reveal a number of critical teaching points for intent, decentralized execution, selfsynchronization and shared situational awareness. After these two games are played and debriefed, a third game is played with five turns and a shared visualization tool. The communication channels are radio and a text chat that “goes down” after a certain turn late in the game. Network centric concepts are evaluated with the training group via a panel of experts after game is completed. StrikeCOM has been used to support at these workshops for the training of NCW concepts with NATO Supreme Allied Command– Transformation, the U.S. Naval Post Graduate School, the Royal Dutch Army, the Singapore Ministry of Defense, and Network Centric Warfare - Enhancing Battlespace Management in London. In addition, the CMI team is working with U.S. Marine Corps Expeditionary Warfare School (EWS) to modify StrikeCOM to support their curriculum. Specifically, StrikeCOM, will be instrumental for both the resident and non resident course students at EWS. StrikeCOM will be modified to support the teaching of basic tenets of Intelligence Preparation of the Battlespace, Information Management, Collaborative Sharing, and the Marine Corps Plan-

ning Process.

5. Conclusion From the use of StrikeCOM in the research studies and military training, we have learned several important lessons concerning game-based research and teaching. These include a simple, easy, and quick to learn interface, familiarity with the game concept, and the use of motivated subjects. A simple interface and game process allows the subjects or students to concentrate on strategizing or learning from the game rather than the technical steps of running the game software itself. The similarity of StrikeCom to the classic Milton Bradley game “Battleship” gives the game a familiar feel that students and subjects in some parts of the world are used to. The simplicity of the game process allows the players to quickly come up with an acceptable strategy. Rarely is any outside knowledge needed to create a satisfactory strategy. Simplicity also has its drawbacks. Students are always aware of the game as an abstract tool since it lacks the immersive three-dimensional interface common in today’s video game market. The lack of engagement this causes sometimes causes less motivated students and subjects to adopt a “let’s just get it done as fast as we can” attitude that can be detrimental to research, but is not necessarily new in teaching situations. Games such as Full Spectrum Command [10] developed at the University of Southern California have a more complex, more immersive interface and game play that may be more suitable for some military training exercises. One comment that was consistent across both the research and teaching StrikeCOM experiences was the need for immediate feedback. The feedback given by each turn’s evaluation was valuable as well as the feedback from the strike plan at the end, which tells the players the targets that were hit, the targets that weren’t hit, and the bombs that missed. Some wargame scenarios used as teaching tools in the U.S. military only present a scenario and give the students the ability to plan a response. Instructors then give feedback on the response plan. However, students are often interested in “unbiased” (not an instructors’s) feedback on how their response plan would work. Instructors can inform students on how good their plan is based on military doctrine, lesson plans, etc., but the game can tell the students how well their plan worked. The plan may be “bad” according to criteria, but it may still work out in the end. This kind of feedback is valuable to the students’ training as a military planner. Another item of valuable feedback we received from a pilot NCW training course was the use of appropriate symbols. Some of the students were confused by the red, yellow, and green symbols to stand for likely something there, maybe something there, and likely not something there, re-

0-7695-2268-8/05/$20.00 (C) 2005 IEEE

8

Proceedings of the 38th Hawaii International Conference on System Sciences - 2005

spectively. Despite clear explanation in the instructions as to the meanings of the symbols, some students thought the red and green symbols were the opposite of their actual meaning. They thought the red symbol meant that something was likely not there and the green meant something was likely there. The cultural use of green for go and red for stop may have influenced their confusion if they were asking should I look in this cell? instead of is this cell clear? On the technical side, in all of the experiments and teaching sessions the game was run as a Java application, though the game can run as an applet. Since it is run as an application, the software must be installed on the computers the subjects or the trainees use. In some situations, especially in the military, installing an application on the available computers might be difficult. It is possible to run it as an applet, but there are several problems with running applets on unknown machines. The biggest problem is that the version of Java installed on the client machine is often the wrong one. The developers of Java have attempted to address this issue with certain HTML tags that tell the client to download the correct version, but this process doesn’t always work. Ideally, the game would work entirely in HTML without any need for a browser plug-in. The problem with pure HTML using the HTTP protocol is that a client can only pull from a server, it does not create a persistent session where the server can push messages to the client. A game like StrikeCOM requires that events on one client be reflected on other clients immediately. Without the ability to push these events to the client, the clients would have to poll the server periodically for changes. If the polling interval is too great, the events will not necessarily happen in an appropriate time. If the polling interval is too small, the server may become overwhelmed. Until this problem is solved, StrikeCOM will remain an application or applet, despite the disadvantages associated with them. Design lessons: • Utilize simple, familiar game processes and interfaces to decrease learning curve • If possible, use a pure HTML interface for interoperability • Use symbols that reflect common cultural beliefs and are not ambiguous • Incorporate flexibility into the technical design to allow for changing objectives • Implement on an appropriate RDBMS Execution lessons: • Incorporate full, immediate feedback • Use in teaching and research situations, not specific training situations

In summary, StrikeCOM may be useful in a number of teaching and research situations. Our experience with StrikeCOM has resulted in a number research findings, teaching outcomes, and lessons learned. Table 3 is a summary of the lessons learned we have presented throughout the paper. Our hope is that StrikeCOM will be useful to future NCW and EWS efforts as well as others in both their teaching and research pursuits.

References [1] M. Adkins and J. Kruse. Case study: Network centric warfare in the U.S. Navy’s Fifth Fleet: Web-supported operational level command and control in Operation Enduring Freedom. Technical report, Center for the Management of Information, 2004. [2] D. S. Alberts, J. Garstka, and F. Stein. Network Centric Warfare: Development and leveraging information superiority. National Defense University Press, Washington D.C., 1999. [3] C. C. Albrecht. A programming framework supporting rapid application development of highly interactive, collaborative applications. Unpublished Ph.D. dissertation, University of Arizona, 2001. [4] D. Biros, M. Hass, M. Adkins, K. Wiers, D. P. Twitchell, J. K. Burgoon, and J. F. Nunamaker Jr. Task performance under deceptive conditions: Using military scenarios in deception detection research. In Thirty-Eighth Annual Hawaii International Conference on System Sciences (CD/ROM), Big Island, Hawaii, 2005. (In press). [5] R. O. Briggs, M. Adkins, D. D. Mittleman, J. Kruse, S. Miller, and J. F. Nunamaker Jr. A technology transition model derived from field investigation of GSS use aboard the U.S.S. Coronado. Journal of Management Information Systems, 15(3):151–196, 1999. [6] A. K. Cebrowski and J. Garstka. Network centric warfare: Its origin and future. Naval Institute Proceedings, 124(1):28–36, 1997. [7] J. F. Dunnigan. Wargames Handbook, Third Edition: How to Play and Design Commercial and Professional Wargames. Writers Club Press, San Jose, third edition, 2000. [8] P. P. Perla. The Art of Wargaming: A Guide for Professionals and Hobbyists. Naval Institute Press, Annapolis, Maryland, 1990. [9] P. P. Perla, M. Markowitz, A. Nofi, C. Weuve, J. Loughran, and M. Stahl. Gaming and shared situation awareness. Technical Report CRM D0002722.A2/Final, Center for Naval Analyses, November 2000. DOD Document. [10] W. Swartout and M. van Lent. Making a game of system design. Communications of the ACM, 46(7):32–39, 2003. [11] D. P. Twitchell, J. F. Nunamaker Jr., and J. K. Burgoon. Using speech act profiling for deception detection. In Lecture Notes in Computer Science 3073: Intelligence and Security Informatics: Proceedings of the Second Symposium on Intelligence and Security Informatics, Tucson, Arizona, 2004. Springer-Verlag.

Table 3. Summary of lessons learned

0-7695-2268-8/05/$20.00 (C) 2005 IEEE

9