Using Content and Text Classification Methods to Characterize Team ...

2010 International Conference on Global Software Engineering

Using Content and Text Classification Methods to Characterize Team Performance Kathleen Swigger, Robert Brazile

George Dafoulas

University of North Texas Denton, USA [email protected], [email protected]

Middlesex University Middlesex, UK [email protected]

Fatma Cemile Serce

Ferda Nur Alpaslan

Atlim University Ankara, Turkey [email protected]

Middle East Tech. University Ankara, Turkey [email protected]

Victor Lopez Universidad Tecnológica de Panamá Panamá City, Panamá [email protected]

Abstract- Because of the critical role that communication plays in a team's ability to coordinate action, the measurement and analysis of online transcripts in order to predict team performance is becoming increasingly important in domains such as global software development. Current approaches rely on human experts to classify and compare groups according to some prescribed categories, resulting in a laborious and error-prone process. To address some of these issues, the authors compared and evaluated two methods for analyzing content generated by student groups engaged in a software development project. A content analysis and semi-automated text classification methods were applied to the communication data from a global software student project involving students from the US, Panama, and Turkey. Both methods were evaluated in terms of the ability to predict team performance. Application of the communication analysis' methods revealed that high performing teams develop consistent patterns of communicating which can be contrasted to lower performing teams.

asynchronous and distributed decisions are often the norm [14, 41], and where group membership and the identity of participants often change over time. Currently there is no overall consensus on how to measure teamwork effectiveness of global software teams, partly because of a lack of empirical data to validate measures [27]. Current approaches for measuring team performance tend to rely on human observers who label group activities according to some prescribed set of categories [20]; a process that is both laborious and error-prone. Other researchers use questionnaires or surveys to determine the effects of specific factors such as orientation, cohesion, roles, and climate on team performance [15, 65, 26, 40, 43, 50]. Although such studies have produced many useful results, they often lack operational relevance, since the data is not analyzed until after the actual event. Recent advances in information retrieval and data mining, however, suggest that automated and/or semi-automated text classification algorithms are effective in finding differences in the communication patterns among individuals and groups [34, 37, 49]. Thus, it seems reasonable to conclude that these same linguistic and data mining techniques might be used to find effective measures for assessing the performance of global software development teams in real time. Thus, the main objective of this paper is to investigate an automated text classification technique and determine if it can effectively predict global software team performance. The context for this work is a computersupported collaborative environment in which computer

Keywords- (global software student teams, collaborative teams, groupware)

I.

INTRODUCTION

The growing importance of teams in organizations has led to an increased interest in research on team performance. Unfortunately, reliable and valid measures of team performance are often difficult to obtain [28, 33]. This is particularly true for team research that involves distributed software development teams – jobs in which 978-0-7695-4122-8/10 $26.00 © 2010 IEEE DOI 10.1109/ICGSE.2010.30

192

science students from Turkey, Panama, and the US collaborate on large programming projects. A current NSF grant has allowed the authors to capture interaction data from the 420 students who have participated in various team programming projects over the past two years. In this paper, the authors examined group transcripts generated during a global software development student project involving students from the US, Panama, and Turkey. An automated text classification technique was used to detect key word patterns and relationships among the documents generated by different teams. More specifically, we tested whether the measures could determine the similarity among strings of text for high versus low performing teams. Since group grades were previously assigned, we were able to apply the techniques and determine if the automated text classification method performed similarly to our current content analysis technique that requires human observers to assign codes to recorded transcripts. Application of the automated text classification technique revealed that it compared favorably to our current communication analysis method which requires the use of human coders. Before describing our approach, we provide some background on empirical findings relevant to analysis of communication and team performance. II.

The quantity of a group’s communications is much easier to collect because it generally consists of counting amounts or duration. But these numbers and amounts do not generally capture the semantics of the communication activities. Both quantity and content approaches have been used widely to analyze communication among groups in business [32], education [24], military [53], and computer science [19] domains. Regardless of which communication technique is used, a researcher must ultimately determine the relationship between the team’s communication activities and its performance. Team research gathered from using quantitative measures such as number and length of words has been equivocal. Some studies report that high performing teams communicate more frequently than low performing teams [6]; others say this finding is not supported [15]. Some studies indicate that overall communication frequency is reduced during peak workload periods [39]; whereas other studies show that communication frequencies increase as workloads increase [52]. Some of these differences may be due to other factors such as the nature of the task or team composition. Studies associated with communication content have also looked for relationships between different cognitive categories (e.g., planning, contributing, socializing, etc.) and team performance [48]. Results from these types of studies tend to be more specific (but perhaps less generalizable) than those associated with quantitative measures. For example, reference [1] found that a team’s use of military terms, acknowledgments, and identifying statements increased as the group gained more experience with each other. Wang et. al., [54] discovered that specific words with student transcripts could be used to identify student roles and high performers. Another useful content analysis approach has been the development and validation of rating scales that can be used to assess the quality of communication behaviors in a team [13, 17]. One such example is the use of behaviorally anchored rating scales that tie specific communication behaviors to a scenario or event [21]. Other methods have categorized communications by type, which often requires doing a post hoc analysis of video, audio, or text records [20, 38]. In summary, research on team or collaborative communication has examined both the quantity and content of group interactions. Though much more time consuming to analyze, content methods tend to provide richer, qualitative results than quantity methods. A major obstacle in doing content analysis is the cost involved in transcribing and coding the texts. Thus, researchers suggest that linguistic and data mining techniques can help eliminate some of these costly human processes [45]. The methods that we applied in this project tried to take advantage of these types of studies by comparing a text mining technique to a human coding system.

RELEVANT RESEARCH

Because of the critical role that communication plays in a team's ability to achieve coordinated action, the measurement and analysis of communication behaviors has been an ongoing focus of team research. Team research studies are widespread, with numerous articles appearing in business [9, 11], psychology [4, 12], medicine [2], computer science [7], and education [51] journals. The most common method for analyzing communication data has focused on low-level quantitative measures such as the duration of a communication or the time of a communication [5, 24]. Other types of communication analysis approaches concentrate on a team’s content and usually involve creating some type of coding scheme that represents different categories of interest such as the intent of a conversation [22], the types of speech in a sentence [10], or the actual meaning of the discussion [25]. The transcribed discourse is then divided into smaller units of meaning, and those pieces of text that correspond to a particular category are tagged [22]. The tagged content can be analyzed either as frequency counts of the categories themselves or as a series of events (called "interaction analysis") [31, 42]. Researchers using the content approach must make sure that they use multiple coders and that there is adequate agreement among all the coders. Transcribing and coding text is obviously very labor intensive, but the advantage of this approach is that group processes are captured, including, in some cases, nonverbal communication [18, 44].

III.

CURRENT CONTEXT

Like many of the researchers cited in the previous section, the authors’ research has focused on ways to 193

poor performers. In order to provide timely remediation, team performance measures need to go beyond simply counting categories after the fact. Moreover, measures for team performance need to be merged into a single taskembedded view so that the relationships between a team’s flow and content can be more easily assessed. By using communication data generated relatively effortlessly as a byproduct of a group interaction, we hope to establish automated techniques that can measure team performance more effectively. A more detailed list of those techniques and measures now follows.

collect and analyze different types of measures of team performance [48, 51]. In fall 2008, the authors began a multi-year project that is exploring ways to increase the effectiveness of globally distributed software learning teams, particularly student programming teams that are composed of individuals who live in different countries and time zones. The universities participating in the research are Middlesex University (MDX) in the UK, the Universidad Tecnológica de Panamá (UTP) in Panama, University of North Texas (UNT) in the US, Middle East Technical University (METU) and Atilim University (AU) in Turkey. Each of the five universities represents a same/different time zone geographical configuration that allows us to determine which specific spatial-temporal factors are important and how they interact in the context of a software development project. During the past two years, computer science students from the five universities have been divided into culturally/spatially diverse work teams and assigned programming projects that are completed using special collaborative software. The collaborative software supports both synchronous and asynchronous communication among the geographically remote groups. More importantly, the collaborative tools have access to a database that logs all communication activities that occur within the environment. This latter feature is particularly important because it allows us to collect data on the groups’ communication activities. Project performance is evaluated with respect to programming accuracy, efficiency, completeness, and style. Participants also complete several surveys that are designed to collect information about a student’s background, experience, attitudes about collaborative work (i.e., cohesion, support, teamwork skills), and culture. In order to determine the relationship between a group’s content and performance, the authors characterize the communications activities among student teams using a series of categories developed by [13]. More than 2896 communication incidents from four different projects have been coded and analyzed in this manner. Since one of the objectives of this particular study was to identify distinct groups of global software learners with similar communication behaviors, we used cluster analysis to show which groups had similar communication patterns. The primary purpose of cluster analysis is to group objects of similar kind into respective categories or classifications. Having associated the clusters with specific patterns of communication behaviors, we then examined the relationship between the clusters and team performance. Our results suggest that communication patterns among global software development learners are related to team performance. More specifically, it appears that communications related to contributing behaviors seem to have had the most positive relationship to high performance [48]. Unfortunately, coding the individual group transcripts after every project is extremely timeconsuming. There is also the problem of coding the text in real time so that the results can be used to help remediate

IV.

METHODOLOGY

A. Content Analysis Technique The instrument that was used to code the group postings/chat is a coding scheme that characterizes a student group’s collaborative behaviors [13]. Curtis and Lawson [13] identify nine different behaviors as being supportive of the collaborative process. Curtis and Lawson first created a set of 15 separate communication activities and then grouped these activities into five communication behavior categories. The planning behavior indicates that the message contains a statement that relates to organizing work, initiating activities, or group skills. The contributing code is assigned to messages that give help, provide feedback, exchange resources, share programming knowledge, challenge others or explain one’s position. Other collaborative behaviors are also noted such as seeking input and reflection. Conversations about social matters that are unrelated to the group task at hand are generally placed in the social interaction category. Using these five categories, the authors code student messages that have been recorded for a programming task. The human coder generally uses a single communication as the unit of analysis. Trained coders categorize messages into the five content categories using the instrument explained above. Each posting is extracted and coded into one of the communication behavior categories: planning, contributing, seeking-input, monitoring/reflecting, and interacting socially. Duplicate codes are assigned whenever an utterance indicated multiple collaborative behaviors. Instructor messages posted by the class instructor or teaching assistant are excluded from the counts. Unclassified messages that do not fit into any of the categories are also not counted. Percent agreement among any two coders is checked for reliability and consistency. Once the codes are assigned to each text, the codes are then clustered to identify characteristics that maximally discriminate among the cases in different segments. The clustering variables are a group’s number of interactions devoted to the five interaction behaviors. Based on a review of clustering techniques, we chose to use a hybrid clustering method to identify the different groups. The hybrid clustering technique uses two methods namely kmeans and Ward’s hierarchical agglomerative clustering. The centers (or centroids) of each cluster are obtained first

194

using Ward’s method [36], a hierarchical cluster analysis technique which is said to be the most likely method to discover any underlying cluster structure. The resulting centroides are then used as the initial seed points for the nonhierarchical k-means cluster analysis.

Documents that are closely correlated in this manner can be said to share similar features. C. The Data Set The specific data set that was used to compare the content analysis with the automated text classification method was obtained from a 2008 global software development team project. A total of 70 students participated in this team project. The 10 students from Universidad Tecnológica de Panamá were enrolled in a database course, the 30 students from Atilim University were enrolled in a Java course, and the 30 students from the University of North Texas were enrolled in a database course. Students from the three universities were grouped together into 15 teams, with between 5-6 students in each group. Each team consisted of approximately 2 students from the US, 2 from Turkey, and 1 student from Panama. Table I summarizes the demographic information for the project. The actual task that was performed in this project was determined, in part, by the curriculum of the courses that were involved in the research for that semester. The students were given an assignment to design, create, and query a database that could maintain a fleet of rental cars. Students were expected to produce an appropriate E-R diagram and test queries for the database as well as develop a Java application that could add and delete data to and from the database. Student teams were also responsible for completing several reports and documentation for their systems. The student teams were given four weeks to complete this project.

B. Automated Text Classification Technique Recent advances in the linguistic field have led researchers (and us) to consider automated methods for categorizing communication text. A typical text categorization problem begins with a set of documents, each of which is represented as individual words. A matrix is then created where the rows of the matrix represent all of the unique words found within the documents, the columns represent each individual document, and the cells represent the frequency of each word as it appears in each document (also called term frequency). In addition, the words can be stemmed, and/or analyzed on different linguistic levels: lexical, morphological, syntactic or semantic [47]. More complex feature representations (e.g., bigrams [8], part-of-speech [2], complex nominals [36], noun phrase chunks [27], and extracted keywords [31] can also be used). The first step in text categorization is to transform the documents, which typically are strings of characters (i.e. words), into a representation suitable for the classification task. The initial analysis of some of our transcripts showed a number of problems with the transcripts generated by student groups: many spelling and grammatical errors were in the text, abbreviations were often used, and phrases were often entered instead of full sentences. Thus, the text files were pre-processed using the Natural Language Tool Kit (NLTK) to clean up some of the problem text. When this step was complete, we applied the different text analysis techniques to the text files using the Wordstat toolkit [55]. Automatic text classification works similar to the content analysis technique described above, except that the algorithm is applied directly to the text and not a coded text. The starting point for applying a text classification method begins with the creation of the vector space model for text data [46]. The basic idea is (a) to extract unique content-bearing words from the set of documents and treat these words as features and (b) to represent each document as a vector of certain weighted word frequencies in this feature space. The results are a vector of the text data known as a word-by-document matrix. In the case of our global software development project, the rows of the matrix represented all the unique words found in the documents for each team, the columns represented the individual documents for a team, and the cells represent the frequency of each word as it appears in each document. The document vectors are then normalized in order to mitigate the effect of differing lengths of documents [47]. Once the data was stored in the matrix, each student team’s document was correlated with other teams’ documents by measuring the cosine between the vectors.

TABLE I. DEMOGRAPHIC INFORMATION OF SUBJECTS Project Teams School

# students

Degree

AU

30

BS

PTU

10

BS

UNT

30

BS

Total

70

AU: Atilim University, Turkey PTU: Universidad Tecnológica de Panamá, Panama UNT: University of North Texas, US

Although the global software student teams used a number of different online collaborative tools, they did most of their team communication using either an open source learning management system called Online Learning and Training (OLAT) or a synchronous design tool called ICE. Both software packages support

195

asynchronous communications such as forums, emails, wikis, file sharing etc., and synchronous communication such as chat and ER design and database tools. Data from the US-Panama-Turkey projects was obtained from the OLAT and ICE systems directly, and from programs that were developed to augment OLAT’s data collection capabilities. The recorded data included information about every communication activity (i.e., message posting, file upload, and wiki entry, along with the date, time, and author of each online activity). D.

E. Results from the Content Analysis Technique The cluster analysis of the coded transcripts generated from Ward’s method suggested two clusters for the global software teams. Fig. 1 shows the clusters for this project and the number of communication behaviors that occurred in each cluster. Teams 1, 11, and 15 constitute Cluster 1, and teams 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, and 14 were aggregated in Cluster 2. Again, these clusters are determined by determining groups that have similar “communication patterns.” Having identified the two clusters, we then examined the relationship between the cluster placement and team performance. It should be remembered that team performance was defined as the grade on the project. A comparison of performance scores (as reported in Table II) between Clusters 1 and 2 show that the top three teams (i.e., 1, 11, and 15) comprise Cluster 1. All other teams were assigned to Cluster 2. A t-test revealed a significant difference between team scores (t = 18.33 mean, t =4.01, df=13, p< .001). Similar to our previous studies [48], contributing behaviors were significantly higher for the highperforming teams in Cluster 1. Social interactions were higher for low performing teams and feedback seeking and feedback giving were the common communication behaviors observed in both clusters.

Team Performance Measures

Team performance metrics have been previously validated in the global software development learning context. Group performance has been defined on five different levels. The first four measures relate to task performance, and the last measure evaluates teamwork knowledge. Categories for task performance include: accuracy (i.e., satisfies requirements and error free), efficiency (number & type of program modules and concise design), thoroughness (includes all necessary elements), and style (follows correct naming conventions, documentation, etc.). After teams complete each project, researchers from each university evaluate their own student projects as well as those from the other participating countries. A mean grade for task performance is then assigned to each student for each category. A team’s performance score is obtained by averaging each group’s individual grades on each task category for all tasks in the assignment. The grades assigned to the participating teams (1-15) in the 2008 project are listed in Table II in the order of their performance. TABLE II. TEAM GRADES FOR PROJECT Team Grades Team Numbers

1 15 11 5 12 13 9 3 7 10 2 4 6 8 14

Grades

89 79 78 74 73 71 69 64 64 64 59 57 56 55 54

Figure 1.

Clusters for Content Analysis Method

F. Results from the automated text classification technique The comparisons of document similarities resulting from the text classification application are reported in Table III. The matrix represents the correlation of a document (a group’s forum and chats) with all other documents in all groups. Table III indicates that there are strong (greater than 0.5) correlations between documents 1, 11, and 15. There are also similarities, although not as great, between Documents 4 and 8. Documents 1, 11, and 15 represent the discussion/chats that occurred within the three high performing teams, whereas Documents 4 and 8 represent discussion/chats in the low performing teams.

196

TABLE III. CORRELATION MATRIX OF DOCUMENTS FOR EACH TEAM

Documents of Teams 1

2

3

4

5

6

7

8

9

10

11

12

13

1

1

2

0.376

1

3

0.362

0.187

1

4

0.319

0.112

0.077

1

5

0.357

0.142

0.209

0.087

1

6

0.087

0.109

0.054

0.155

0.049

1

7

0.172

0.176

0.136

0.019

0.159

0.019

1

8

0.202

0.124

0.082

0.567

0.023

0.022

0.006

1

9

0.365

0.174

0.29

0.311

0.073

0.034

0.129

0.132

1

10

0.281

0.269

0.233

0.028

0.108

0.018

0.088

0.059

0.139

1

11

0.651

0.315

0.317

0.382

0.188

0.107

0.261

0.263

0.397

0.281

0

12

0.208

0.081

0.048

0.121

0.075

0.102

0

0.021

0.049

0.094

0.22

1

13

0.159

0.181

0.048

0.008

0.082

0

0.033

0.044

0.049

0.091

0.106

0.014

1

14

0.201

0.171

0.167

0.014

0.108

0.018

0.224

0.076

0.112

0.145

0.207

0.031

0.111

15

0.596

0.293

0.187

0.073

0.157

0.025

0.121

0.058

0.164

0.178

0.331

0.128

0.144

TABLE IV. WORD FREQUENCY FOR HIGH PERFORMING TEAMS Word

Freq.

Plan

Contrib.

Seek.

Reflect.

THINK

36

.694

.704

.000

.000

Socl Inter. .000

WORK

24

.000

.000

.000

.000

LOOK

20

.000

.000

.664

.263

LIKE

19

.000

.000

.584

CHECK

18

.000

.657

GOOD

18

.000

RENT

15

.000

PUT

13

RETURN GET

Word

Freq.

Plan

Contrib.

Seek.

Reflect.

RETRIEVE

6

.258

.521

.000

.000

Soci. Interact. .000

.878

THING

6

.000

.597

.248

.000

.000

.000

AHEAD

5

.239

.000

.000

.000

.503

.119

.000

CHARGED

5

.340

.000

.000

.000

.789

.256

.000

.000

SET

5

.365

.531

.000

.055

.606

.250

.000

.537

.000

ADDED

4

.000

1.000

.000

.000

.705

.512

.000

.000

.352

LOT

4

.219

.516

.000

.000

.271

.000

.000

.000

.239

.566

RUN

4

.000

.598

.000

.343

.000

10

.000

.620

.000

.000

.291

DONE

4

.000

.000

.860

.000

.298

10

.000

.629

.396

.000

.000

TODAY

4

.000

.000

.000

.416

.096

SUBMIT

9

.000

.646

.000

.327

.181

DOCUMENT

3

.309

.000

.000

.572

.000

THINGS

8

.000

.497

.000

.000

.122

END

3

.000

.000

.000

.000

.482

WORKS

8

.313

.466

.368

.255

.000

FILL

3

.000

.502

.000

.000

.413

AGE

7

.000

.000

.257

.156

.873

HOLD

3

.000

.538

.440

.235

.448

BACK

7

.000

.000

.862

.107

.089

ORDER

3

.455

.289

.000

.574

.000

FULL

6

.000

.731

.296

.000

.915

WORKING

3

.000

.000

.368

.000

.478

POSSIBLE

6

.249

.000

.506

.000

.182

ANSWER

2

.000

.000

.000

.505

.138

READY

2

.520

.490

.000

.000

.000

197

communications, and an automated text classification technique that finds similarities among keyword patterns within uncoded documents. In previous work, the authors applied content analysis techniques to communication behaviors of global software teams and found that there was a relationship between certain types of behaviors and team performance. While these measures were useful, they were also cumbersome and administered after the task was completed. Thus, the authors examined text classification methods that can be applied to transcripts without the use of human intervention. Both the content analysis and automated text classification techniques were applied to data generated from student teams. Both communication analysis methods were evaluated in terms of their ability to predict team performance. Both techniques found that communication among high performing teams was similar. Our future work will move toward examining additional automated methods. We also intend to examine different flow measures to determine how team behavior changes over the life of a project. Moreover, the automated methods described in this paper are designed to be used in an on-line, real time assessment of team performance. The ultimate goal of this project is to provide a system that can automatically detect problems and suggest remediation for students.

We then examined the actual keywords that occurred in the transcriptions of the three high performing teams to determine if there was any relationship between the keywords identified by the automated classification system and those found in the content analysis method. We used WordNet’s Vector- UMN to find the semantic similarities between the Top Teams’ keywords, as identified by the automatic classification system, and the behavior categories used in the content analysis method (i.e., planning, contributing, seeking, monitoring, and social interactions). We went back to the original list of words for our top teams and selected ONLY those words that had correlations greater than .500 with any of the five content categories. The list of these top words and their classifications are presented in Table IV (NOTE: Words with a frequency of 1 are not included). These words were collected and examined in terms of frequency of occurrence in each of the five categories. Figure 3 compares the frequencies of communication behaviors for each category as identified by the automated text classification method versus those identified by the human coded content analysis method. Although the human coded content analysis frequencies were lower (because we tended to hand code phrases rather than specific words), the proportion of communications in each category for each method appear to be fairly similar, except for differences in the social interaction and seeking communication categories. A pairwise comparison of the proportions in each category indicate significant differences in only the social interaction (p< .001) and seeking information (p < .003) categories. An examination of the specific words that were correlated with Social Interaction and Seeking behaviors show that many of the words in these two categories had either very low correlations or were also related to other categories (i.e., social interaction).

ACKNOWLEDGEMENTS This material is based upon work supported by the National Science Foundation under Grant No. 0705638. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. We also wish to thank the students who participated in the study, and the many colleagues (in all four countries who helped make this research possible. REFERENCES [1]

[2]

[3]

Figure 2. Comparison of word category frequencies between content analysis and automated methods

V.

CONCLUSION

[4]

This paper documents an effort that compared and evaluated methods for analyzing the communication content surrounding the collaboration of global software student teams. The methods included a content analysis technique that requires coders to classify different types of

[5]

[6]

198

L. Achille, B. Schulze, and A. Schmidt-Nielsen, “An analysis of communication and the use of military terms in Navy team training,” Military Psychology, vol. 7, pp. 95-107, 1995. A. Aizawa, “Linguistic techniques to improve the performance of automatic text categorization,” In: Proceedings of NLPRS-01, 6th Natural Language Processing Pacific Rim Symposium, pp. 307– 314, 2001. A. Argraw, A. Hulth, and B. Megyesi, “General-Purpose text categorization applied to the medical domain, “ downloaded December 2009, http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.106.4090 C. Bowers, F. Jentsch, E. Salas, and C. Braun, “Analyzing communication sequences for team training needs assessment,” Human Factors, vol. 40, pp. 672- 679, 1998. B. Butler “Membership size, communication activity and sustainability: A resource-based model of online social structure,” Information Systems Research, vol. 12, pp. 346-362, 2001. E. Bradner and G. Mark, “Why distance matters: Effects on cooperation, persuasion, and deception,” Proceedings of 2002

[7]

[8]

[9]

[10]

[11]

[12]

[13] [14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22] [23] [24]

[25]

ACM Conference on Computer Supported Cooperative Work, November, pp. 226-235, 2002. L. Burnell, J. Priest and J. Durrett, “Teaching distributed multidisciplinary software development,” IEEE Software, vol. 29, pp. 86-93, 2002. M. Caropreso, S. Matwin, and F. Sebastiani, “A learnerindependent evaluation of the usefulness of statistical phrases for automated text categorization,” in Text Databases and Document Management, Theory and Practice, pp. 78–102, 2001. T. Carte, L. Chidambaram and A. Becker, “Emergent leadership in self-managed virtual teams – a longitudinal study of concentrated and shared leadership behaviors,” Group Decision and Negotiation, vol. 15, 4, 2006, pp. 323-343. N. Contractor and S. Grant “The emergence of shared interpretations in organizations: a self-organizing systems perspective,” In J. H. Watt and A.C. VanLear (Eds.), Dynamic Patterns in Communication Processes, pp. 215-230. Thousand Oaks, CA: Sage Publications, 1996. C. Cramton and P. Hinds, “Subgroup dynamics in internationally distributed teams: Ethnocentrism or cross-national learning?” Research in Organizational Behavior, vol. 26, pp. 231-263, 2005. H. Cuevas, S. Fiore, E. Salas, and C. Bowers, “Virtual teams as sociotechnical systems. In Virtual and Collaborative Teams, Pennsylvania, S. H. Godar and S. P. Ferris, Eds., Idea Group, 2004, pp. 1-19. D. Curtis and M. Lawson, “Exploring collaborative on-line learning, JALN, vol. 5,1, pp. 21-34, 2001. G. Dafoulas, K. Swigger, R. Brazile, F. Alpaslan, V. Lopez, and F. Serce, “Global teams: Futuristic models of collaborative work for today’s software development industry,” Proceedings of the 42nd Hawaii International Conference on Systems Sciences, Hawaii. 2009. D. Damian J. Chisan, L. Vaidyanathasamy, and Y. Pal, “Requirements engineering and downstream software development: Findings from a case study,“ Journal of Empirical Software Engineering, vol. 10, pp. 255-283, 2005. C. de Souza, J. Froehlich, and P. Dourish, “Seeking the source: Software source code as a social and technical artifact,” In Proceedings of GROUP ’05, pp. 197–206, 2005. B. De Wever, T. Schellens, M. Valcke, and H. Van Keer, “Content analysis schemes to analyze transcripts of online asynchronous discussion groups: A review,” Computers and Education, vol. 46, pp. 6-28, 2006. W. Donaghy, “Nonverbal communication measurement,” in Measurement of Communication Behavior, P. Emmert and L. L. Parker, Eds.,White Plains, NY: Longman, Inc., 1989, pp. 296-332. A. Dutoit and B. Bruegge, “Communication metrics for software development,” IEEE Transactions on Software Engineering, vol. 24, pp. 615-628, 1998. D. Dwyer, R. Oser, E. Salas, and J. Fowlkes, “ Performance measurement in distributed environments: initial results and implications for training,” Military Psychology, vol. 11, pp. 189215, 1999. H. Edwards and V. Sridhar, “Analysis of the effectiveness of global virtual teams in software engineering projects,” Proceedings of the 36th Hawaii International conference on systems sciences (HICSS03), hicss 19b, 2003. P. Emmert and L. Barker, Measurement of Communication Behavior, White Plains, NY: Longman, Inc., 1989. T. Finholt and L. Sproull, “Electronic groups at work,” Organizational Science, vol. 1, pp. 41-64, 1990. M. Fitze, “Discourse and participation in ESL Face-to-Face and written electronic conferences,” Language Learning & Technology, vol. 10, pp. 64-97, 2006. B. Fortuna, E. Mendes, and N. Milic-Frayling, “Improving the classification of newsgroup messages through social network

[26]

[27]

[28]

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

[37]

[38] [39]

[40]

[41]

[42]

[43]

[44]

199

analysis,” Proceedings of the Sixteenth ACM Conference on Information and Knowledge Management, Lisbon, Portugal, pp. 877–880, 2007. R. Fruchter and A. Townsend, “Multi-cultural dimensions and multi-modal communication in distributed, cross-disciplinary teamwork,“ International Journal Engineering Education, vol. 19, pp. 53-61, 2003. J. F¨urnkranz, T. Mitchell, and E. Riloff, “A case study using linguistic phrases for text categorization on the WWW,” Workshop on Learning for Text Categorization, pp. 5-12, 1998. R. Guzo and M. Dickson, “Teams in organizations: Recent research on performance and effectiveness,” Annual Rev. Psychology, vol. 47, pp. 307-338, 1996. M. Holmes, “Optimal matching analysis of negotiation phase sequences in simulated and authentic hostage negotiations,” Communication Reports, vol. 10, pp. 1-8, 1997. A. Hossain, A. Wu, and K, Chung, “Actor centrality correlates to project based coordination,” Proceedings Computer Supported Cooperative Work, Banff, Alberta, pp. 363-372, 2006. P. Kiekel, N. Cooke, P. Foltz, J. Gorman, and M. Martin, “Some promising results of communication-based automatic measures of team cognition, Proceedings of the Human Factors and Ergonomics Society 46th Annual Meeting, 2002. S. Kiesler and L. Sproull, “Group decision making and communication technology,” Organizational Behavior and Human Decision Processes, vol. 52, pp. 96-123, 1992. P. Layzell, O. Brereton and A. French, “Supporting collaboration in distributed software engineering teams,“ The Asia-Pacific Software Engineering Conference, pp. 38-45, 2005. A. Lui, S. Li, and S. Choy, “An evaluation of automatic text categorization in online discussion analysis,” Proceedings of the Seventh IEEE International Conference on Advanced Learning Technologies, Niigata, Japan, pp. 205-209, 2007. R. Mihalcea and S. Hassan, “Using the essence of texts to improve document classification,” Proceedings of the Conference on Recent Advances in Natural Language Processing,, pp. 150-160, 2005. A. Moschitti, and R. Basili, “Complex linguistic features for text classification: A comprehensive study,” Proceedings of ECIR-04, 26th European Conference on Information Retrieval Research, pp. 181–196, 2004. D. Noble, “A cognitive description of collaboration and coordination to help teams identify and fix problems,” Proceedings of the 7th International Command and Control Research Technology Symposium, Quebec, Canada, 2002. J. Orasanu, Shared mental models and crew performance (Report No. CSLTR-46). Princeton, NJ: Princeton University, 1990. R. Oser, C. Prince, B. Morgan, and S. Simpson, An Analysis of Aircrew Communication Patterns and Content, (NTSC Tec. Rep. No. 90-009), 1991. S. Paul, P. Seetharaman, I. Samarah, and P. Mykytn “Impact of heterogeneity and collaborative conflict management style on performance of synchronous global virtual teams, Information and Management,” vol. 41, 3, pp. 303-321, 2004. L. Pelled, K. Eisenhardt, and M. Song, “Getting it together: Temporal coordination and conflict management in global virtual teams,” Academy of Management Journal, vol. 44, pp. 1251-1262, 2001. M. Poole, M. Holmes, R. Watson, and G. DeSanctis, “Group decision support systems and group communication: a comparison of decision making in computer supported and non supported groups,” Communication Research, 20, pp. 176-213, 1993. R. Rico and S. Cohen, “Effects of task interdependence and type of communication on performance in virtual teams,” Journal of Managerial Psychology, 20, 3-4, pp. 261-274, 2005. C. Rose, Y. Wang, Y. Cui, J. Arguello, F. Fisher, A. Weinberger and Stegmann,“ Analyzing collaborative learning processes

[45]

[46]

[47]

[48]

[49]

[50]

automatically: Exploiting the advances of computational linguistics in computer-supported collaborative learning,” unpblished. C. Rose, P. Donmez, G. Gweon, A. Knight, B. Junker, W. Cohen, K. Koedinger, and N. Heffernan, “Automatic and semi-automatic skill coding with a view towards supporting on-line assessment,” Proceedings of AI in Education '05, 2005. G. Salton, A. Singhal, C. Buckley, and M. Mitra, “Automatic text decomposition using text segments and text themes,” In Proceedings of the the Seventh ACM Conference on Hypertext, ACM, New York, 1996. F. Sebastiani, F. Text categorization, in Texting Mining an its Applications, Zanasi, A., Ed., Southampton, UK: WIT Press, 2005, pp.109-129. F. Serce, K. Swigger, F. Alpaslan, R. Brazile, G. Dafoulas, and V. Lopez, “Exploring the communication behavior among global software development learners,” International journal of Computer Applications in Technology, (in press). R. Sparrowe, R. Liden, S. Wayne, and M. Kraimer, “Social networks and the performance of individuals and groups,” The Academy of Management Journal, vol. 44, pp. 316-325, 2005. D. Staples and A. Cameron, “The effect of task design, team characteristics, organization context and team processes on the

[51]

[52]

[53]

[54]

[55]

200

performance and attitudes of virtual team members, “Proceedings of the 38th Hawaii International Conference on Systems Sciences, the big Island, Hawaii, 52a., 2005. R. Stout, “Planning effects on communication strategies: A shared mental models perspective,” Proceedings of the Human Factors and Ergonomics Society, Santa Monica, Ca., pp. 1278-1282, 1992. K. Swigger, F. Alpaslan, R. Brazile, G. Dafoulas, V. Lopez, and F.Serce, “Structural factors that affect global software development learning team performance,” Conference of special interest group on management information systems, ACM, Limerick, Ireland, pp. 187-196, 2009. R. Thornton, “The effects of automation and task difficulty on crew coordination, workload, and performance,” Unpublished doctoral dissertation, Old Dominion University, Norfolk, VA. G. Wang, G. Ou, T. Chen, and Lewis, “Support of group working: tools for the analysis of web-based collaborative working behaviors,” Proceedings of the Second Workshop on Knowledge Economy and Electronic Commerce, downloaded on December 12, 2009, http://moe.ecrc.nsysu.edu.tw/Chinese/workshopC/2004/16.pdf. Wordstat, Provalis Research, v. 5.1.