Groupwork skills in Higher Education 1 1
Running Head: GROUPWORK SKILLS IN HIGHER EDUCATION
2 In Press at Assessment and Evaluation in Higher Education
3 4 5
Development and Validation of the Groupwork Skills Questionnaire (GSQ)
6
for Higher Education
7
Jennifer Cumming
8
University of Birmingham
9
Charlotte Woodcock
10
Staffordshire University
11
Sam J. Cooley
12
University of Birmingham
13
Mark J. G. Holland
14
Newman University
15
Victoria E. Burns
16
University of Birmingham
17 18
Author Note Jennifer Cumming, School of Sport, Exercise and Rehabilitation Sciences, University
19
of Birmingham; Charlotte Woodcock, School of Psychology, Sport and Exercise,
20
Staffordshire University; Sam J. Cooley, School of Sport, Exercise and Rehabilitation
21
Sciences, University of Birmingham; Mark J. G. Holland, Department for Physical Education
22
and Sport Studies, Newman University; Victoria E. Burns, School of Sport, Exercise and
23
Rehabilitation Sciences, University of Birmingham.
24 25
The development of the Groupwork Skills Questionnaire was supported by a research grants to the Birmingham Evaluating Skills Transfer (BEST) Project from the Centre for
Groupwork skills in Higher Education 2 1
Learning and Academic Development (CLAD), the College of Life and Environmental
2
Sciences, and University Sport Birmingham at the University of Birmingham, and the Higher
3
Education Academy. Many thanks to Chloe Arnold, Jessica Fawcett, Daniel Kelly, Orie
4
Lawrence, Joseph Marsh, Emily Novakovic, and Claire Owen for their assistance with either
5
the data collection and/or data entry.
6
Correspondence concerning this article should be addressed to Jennifer Cumming,
7
School of Sport, Exercise and Rehabilitation Sciences, University of Birmingham,
8
Edgbaston, West Midlands, UK, B15 2TT. Telephone: 0121 414 2877. Email:
9
[email protected].
Groupwork skills in Higher Education 3 1
Abstract
2
The aim of the present study was to develop and provide psychometric evidence in support of
3
the Groupwork Skills Questionnaire (GSQ) for measuring task and interpersonal groupwork
4
skills. A 46-item version of the GSQ was initially completed by 672 University students. The
5
number of items was reduced to 15 following exploratory factor analyses and a two-factor
6
model consisting of task and interpersonal groupwork skills was revealed. Confirmatory
7
factor analyses with model re-specification on new data (n = 275 students) established that
8
the best fitting model consisted of 10 items and the same 2 factors (task and interpersonal).
9
Concurrent validity of the GSQ was then determined with 145 participants by demonstrating
10
significant relationships (p < .05) with attitudes towards groupwork and groupwork self-
11
efficacy. Test-retest reliability was examined over a one week interval. Overall, the GSQ
12
demonstrates good validity and reliability, and has potential for both research and
13
pedagogical application.
14 15
Key words: collaborative learning, groupwork, higher education, reliability, selfassessment, University students, validity
Groupwork skills in Higher Education 4 1 2
Self-Assessment of Groupwork Skills in Higher Education Working in groups is an important aspect of the pedagogical repertoire of university
3
courses. Research indicates that group work enhances students’ academic achievement,
4
quality of peer interactions, self-esteem, attitude towards learning activities, social support,
5
and retention. Moreover, working with others is a powerful tool for developing
6
employability skills, such as communication, conflict management, time management, and
7
reflection (Cumming, 2010; Johnson, Johnson, & Smith, 1998). The importance of these
8
skills is also illustrated by their prevalence in the ‘required characteristics’ identified in job
9
advertisements and rankings in employer surveys (Branine, 2008; Confederation of British
10
Industries, 2009).
11
Placing students in small groups, however, does not mean that the benefits of
12
groupwork will automatically be realized. Interpersonal conflict, lack of understanding of
13
group members’ capabilities, poor task definition, lack of shared goals, and social loafing
14
have all been shown to compromise the effectiveness of groupwork (McCorkle et al., 1999).
15
Negative experiences of groupwork can have detrimental impact on immediate group
16
outcomes including task performance, as well as student’s attitudes towards, and engagement
17
in, later peer collaborations (Hillyard, Gillespi, & Littig, 2010). Other factors influencing
18
whether groupwork is a positive and successful experience include those that relate to the
19
individual (e.g., academic ability, personality) as well as the group (e.g., size, diversity),
20
nature of the task (e.g., level of interdependence), and facilitation procedures (e.g., group
21
membership allocation, assessment processes) (Chapman & Van Auken, 2001; Pfaff &
22
Huddleston, 2003; Rhee, Parent, & Basu, 2013).
23
The pre-existing skills that students bring to the group also predict the success of these
24
activities. Indeed, collaborative work is often compromised because students lack groupwork
25
skills and may not understand how to work cooperatively with others (Johnson et al., 1998;
Groupwork skills in Higher Education 5 1
Prichard, Bizo, & Stratford, 2006). Hence, recommendations have been made for more
2
initiatives to develop students’ groupwork skills (Cumming, 2010; Hillyard et al., 2010;
3
Maiden & Perry, 2011). For these to be successful, however, clarity on what is meant by
4
groupwork skills and how these can be assessed within a higher education context is needed.
5
That is, valid and reliable measurement of groupwork skills are needed so that evaluation can
6
be undertaken to determine the success of any implemented changes. Such a measure could
7
also serve a dual purpose as a pedagogical tool to help students become more aware of their
8
interactions in groups.
9
Terms including teamwork, collaborative, or small-group skills have all been used
10
interchangeably to describe the attributes required to work effectively in groups. As Hughes
11
and Jones (2011) have pointed out, it is very hard to distinguish between teams and groups
12
within an academic setting as both are composed of students who vary along the same set of
13
characteristics (e.g., having a shared identity and common goals) (also see Prichard et al.,
14
2006 for similar arguments). For the sake of clarity, groupwork skills is used herein to
15
describe the set of skills that individuals employ to enhance group functioning. Our particular
16
focus is on the groupwork skills employed by individual students so that they can effectively
17
contribute to the many groups and teams they will be part of in education and work settings.
18
Any assessment of groupwork skills must recognize the myriad of behaviours that this
19
encompasses. For the present study, we adopted a conceptual model based on the view that
20
these skills appear to fall under the broad categories of task-management skills and
21
interpersonal skills (Hobson, Strupeck, Griffin, Szostek, & Rominger, 2014; Morgan,
22
Glickman, Woodard, Blaiwes, & Salas, 1986; Stevens & Campion, 1994). Task groupwork
23
skills focus on the successful completion of the task and consist of factors such as an
24
individuals’ ability to establish specific and challenging group goals; to monitor, evaluate,
25
and provide feedback; plan the sequence of activities to be carried out; as well as
Groupwork skills in Higher Education 6 1
manage/coordinate interdependencies among group members. In contrast, interpersonal
2
groupwork skills focus on interactions with other group members, including the ability to
3
effectively resolve conflict, understand information exchange networks and to utilize these
4
networks to enhance communication, as well as to provide social support via positive group
5
interactions and supportive networks for group members.
6
Although self-report measures of groupwork skills do exist for use within higher
7
education, few have established psychometric properties for separately examining task
8
management and interpersonal skills. For example, Van Duzer and McMartin (2000)
9
developed a unidimensional questionnaire consisting of 11 items to measure how well
10
students perceived themselves and their group members to have contributed to task
11
management but not interpersonal aspects of a student project. Researchers have also
12
borrowed measures intended for alternative contexts or written study-specific measures that
13
have not been subject to rigorous development processes. As another example, McClough
14
and Rogelberg (2003) employed the Teamwork Knowledge Skill and Ability (KSA; Stevens
15
& Campion, 1994) Test for selecting work teams to assess undergraduate students’
16
knowledge of interpersonal and self-management behaviours. However, O’Neill, Goffin, and
17
Gellatly (2012) have identified serious problems with both the reliability and validity of this
18
test. Moreover, the teamwork KSA test is a knowledge test based on multiple choice
19
questions that was not designed or intended to be used as a tool for helping students to
20
improve their groupwork skills thereby limiting its applicability (Hughes & Jones, 2011).
21
For the development of groupwork skills within higher education to be evaluated
22
effectively, valid and reliable instruments are required (Macgowan, 2008). To fulfill this
23
need, the present study was conducted with the specific purposes of (a) item identification,
24
(b) establishing face, content, structural and concurrent validity, and (c) determining internal
25
reliability and test re-test reliability. The resulting Groupwork Skills Questionnaire (GSQ) is
Groupwork skills in Higher Education 7 1
intended for future use in higher education as both a research tool, such as measuring changes
2
following an intervention, as well as a self-reflection tool to aid students’ development of
3
their groupwork skills. Method
4 5 6
Samples Four samples were recruited from a large UK University to provide psychometric
7
evidence for the GSQ. Sample 1 consisted of 672 undergraduate and postgraduate students
8
(Mage = 21 years, SD = 3.31; 50.6% female), with the majority from Life Sciences (48%),
9
Engineering (21%), and Business (19%) degree courses. Sample 2 consisted of 725
10
undergraduate students (Mage = 19 years, SD = 1.87; 59% female) who were studying
11
towards degrees in Geography (20.4%), Sport and Exercise Sciences (20.3%), Life Sciences
12
(17.6%), and Arts and Humanities (17%). Participants in sample 3 (N = 148; Mage = 18
13
years, SD = 0.91 45% female) and sample 4 (N = 108; Mage = 18 years, SD = 0.94; 56%
14
female) were different cohorts of first year undergraduate students undertaking a BSc in
15
Sport and Exercise Sciences. Across the four samples, the ethnicity of most participants
16
reported their ethnicity as white (63%), followed by Asian (29%), black (6%) or mixed race
17
(remainder unspecified).
18
Instruments
19
Demographics. In each sample, participants were asked to provide information
20
concerning their age, gender, ethnicity, degree course, and level of study (i.e., undergraduate
21
vs. postgraduate).
22
Groupwork Skills Questionnaire (GSQ). A pool of 52 items was developed based on our
23
conceptualization of groupwork skills (e.g., Hobson et al., 2014; Morgan et al., 1986; Stevens
24
& Campion, 1994). These were either newly written items or items retained/refined from
25
existing measures (e.g., Stevens & Campion, 1994; Van Duzer and McMartin, 2000). Two
Groupwork skills in Higher Education 8 1
expert raters were given the aforementioned definitions of task and interpersonal groupwork
2
skills. They rated the degree to which each proposed item matched its definition using a 5–
3
point scale (1 = poor match, 5 = excellent match), and commented on item wording or
4
content (Dunn, Bouffard, & Rogers, 1999). Items were deleted (n = 6) or changed (n = 15)
5
based on the feedback and further deliberation to ensure relevance to the intended constructs.
6
Further feedback was solicited from 10 3rd year undergraduate students and 5 academic staff
7
members, resulting in minor wording changes and the 46 items administered to sample 1.
8
Following the first round of analyses, the pool of items was further reduced to the 15 items
9
administered to sample 2 and 3, and 10 items administered to sample 4. Please note that only
10
the final 10 items indentified by the CFA carried out with sample 2 were employed in the
11
data analyses of sample 3. Participants were asked to complete each item of the questionnaire
12
as honestly as possible on a 5-point scale (1= never and 5= always) in terms of how often
13
they tend to employ these specific groupwork skills.
14
Attitude Toward Working in a Group (ATWG). This scale measures how students feel
15
about working with others as part of their educational experience, using 8 bipolar items (i.e.,
16
bad/good experience, waste of time/good use of time) rated on a 7-point semantic differential
17
scale (1 = unfavourable; 7 = favourable). It has demonstrated good psychometric properties
18
via principal components factor analysis and adequate internal reliability (Chapman & Van
19
Auken, 2001).
20
Self-efficacy for Groupwork Measure. In line with McClough and Rogelberg (2003),
21
this scale was adapted from the Personal Efficacy Beliefs Scale (PEBS; Riggs, Warka,
22
Babasa, Betancourt, & Hooker, 1994) to measure self-efficacy for groupwork. The 10 items
23
assess a student’s comfort with, and motivation to demonstrate, skills which facilitate
24
working with others (e.g., “I have all the skills needed to perform very well in groups”).
25
These were rated on a 5-point Likert type scale from 1 (strongly disagree) to 5 (strongly
Groupwork skills in Higher Education 9 1
agree). After reverse-coding 6 items, the scores are averaged to form a measure of groupwork
2
self-efficacy. Riggs et al. have reported evidence in support of the validity and reliability of
3
this scale.
4
Procedures
5
For all samples, the participants were contacted directly by a trained research assistant
6
during a lecture or training course. All participants were informed of the nature of the study,
7
and those who agreed to participate were given a letter of information, a consent form and a
8
questionnaire pack consisting of demographics and a version of the GSQ (sample 1 = 46
9
items, sample 2 and 3 = 15 items, and sample 4 = 10 items). In sample 3, participants also
10
completed measures of groupwork attitudes and self-efficacy. In sample 4, participants were
11
re-administered the GSQ after a one-week interval.
12
Data Analysis
13
Data screening procedures to identify missing cases and univariate and/or multivariate
14
outliers were conducted for each sample as appropriate (Tabachnick & Fidell, 2013).
15
Analytical procedures varied according to each sample, however the internal reliability was
16
always calculated using Cronbach alphas.
17
Sample 1. Despite being guided by a conceptual model suggesting groupwork skills to
18
be two dimensional, an exploratory factor analysis (EFA) was deemed appropriate at this
19
initial stage of questionnaire development. The EFA allowed us to investigate whether any
20
further subscales would emerge within the data as well as to reduce the items to meaningful
21
factors. The EFA was conducted with SPSS v21 using a maximum likelihood method of data
22
extraction (Fabrigar, Wegener, MacCallum, & Strathan, 1999). Factors with an eigenvalue
23
over 1.0 were retained for rotation using the oblique method of direct oblimin (Costello &
24
Osborne, 2005). A criterion level of .35 was set for an item to be considered to load on a
25
factor (Tabachnick & Fidel, 2013). Poor loading items (i.e., < .35) were omitted from further
Groupwork skills in Higher Education 10 1
analysis. Item communalities that estimate the proportion of variance explained by
2
underlying factors were also inspected. Items with low communalities (< .40) were
3
scrutinized for inclusion in the questionnaire (Costello & Osborne, 2005). Although several
4
items had low communalities, they aligned with the two-factor conceptual model and were
5
therefore retained for further examination.
6
Sample 2. To further explore and then confirm the validity of the two-factor model,
7
two subsamples were created by randomly dividing the data from sample 2. An exploratory
8
sequential model testing approach to confirmatory factor analysis (CFA) was adopted
9
(Biddle, Markland, Gilbourne, Chatzisarantis, & Sparkes, 2001). All measurement models
10
were tested using AMOS 18 data analysis software with maximum likelihood estimation
11
(ML). To account for multivariate non-normality, factor loadings were assessed using
12
nonparametric bootstrapping to resample the data for parameter estimation. This process was
13
followed using subsample 2.1 until an adequate model was obtained that best represented the
14
data. Model fit was determined by the chi-square statistic, the Standard Root Mean Square
15
Residual (SRMR ≤ .05), Comparative Fit Index (CFI ≥ .90), Tucker-Lewis Index (TLI ≥ .90),
16
and the Root Mean Square Residual (RMSEA ≤ .08) (Jackson, Gillaspy, Purc-Stephenson,
17
2009). The stability of the factor structure was confirmed with a CFA conducted
18
independently with subsample 2.2 with the same criteria for model fit applied.
19
Sample 3. Paired samples T-tests compared differences in students’ task versus
20
interpersonal groupwork skills in sample 3. Bivariate correlations were also used to establish
21
concurrent validity in this sample by examining the relationships between groupwork skills
22
and two other groupwork variables (i.e., attitudes towards groupwork and groupwork self-
23
efficacy).
24 25
Sample 4. Test-retest reliability determined with intraclass correlation coefficients using a two-way random effect model with data from sample 4 (Ntoumanis, 2001). A
Groupwork skills in Higher Education 11 1
repeated measures MANOVA, with task and interpersonal skills as the dependent variables,
2
was also conducted to identify any changes over time (i.e., within subject variable). Results
3 4 5
Data Screening Cases with missing data was deleted in sample 1 (n = 22), leaving 650 cases for
6
analysis. Univariate data distribution was within the accepted tolerance levels for skewness
7
and kurtosis for all samples (i.e., > -2 or < 2). In sample 2, Mardia’s normalized kurtosis
8
coefficient indicating that multivariate non-normality was evident in both data sets
9
(subsample 2.1 = 43.65, subsample 2.2 = 40.96) thus bootstrapping was employed. In sample
10
3, cases were deleted due to a high mahalanobis distance (n = 1; >18.47) and missing values
11
(n = 3), leaving a final sample size of 144. Finally, in sample 4, 1 case was again deleted for
12
having a mahalanobis distance value greater than 18.47, leaving a final sample size of 107.
13
Exploratory Factor Analysis (EFA)
14
The 46 items administered to sample 1 were subjected to EFA. Six iterations were
15
completed to find a two-factor solution accounting for 40.85% of the variance. Aligned with
16
the conceptual model adopted for the present study, Factor 1 consisted of 8 items reflecting
17
task groupwork skills, whereas Factor 2 comprised of items referring to 7 interpersonal skills
18
(see Table 1 for factor loadings and communalities). The bivariate correlation between the
19
two subscales indicate they are measuring similar, but separate, dimensions of groupwork
20
skills (r = .54, p < .001).
21
Confirmatory Factor Analysis (CFA) Table 2 details fit indices for each model tested
22
to establish the factorial validity of the GSQ. For subsample 2.1, the initial single factor
23
models for task (model 1i) and interpersonal groupwork skills (model 2i) indicated
24
improvements to each model could be made; items with poor factor loadings (< .40) were
25
removed from subsequent analysis, resulting in improvements to each single factor model for
Groupwork skills in Higher Education 12 1
task and interpersonal groupwork skills. The refined single factor models (1ii and 2ii) were
2
combined to test a two-factor 13 item measurement model (3i) for the GSQ. Following
3
further item reduction (model 3ii and 3iii), a 10 item two-factor solution achieved adequate or
4
good fit across the different indices (model 3iv). A χ2 difference test revealed that model 3iv
5
had a significantly smaller χ2 value than model 3iii, indicating that this 10 item two-factor
6
model provided the best fit for the data.
7
The independent CFA conducted with independent data from subsample 2.2
8
demonstrated a good model fit to the data, confirming the validity of the 10 item two-factor
9
solution. As recommended, standardized factor loadings were all above .40 (task = .51 to .76,
10
interpersonal = .65 - .76), standardized residual covariances were below 2.58, and
11
modification indices were low (Hair, Anderson, Tatham, & Black, 1998). The interfactor
12
correlation between task and interpersonal was .42, indicating that these subscales are
13
measuring related, but separate, aspects of groupwork.
14
Descriptive Statistics
15
In sample 3, students perceived themselves to use significantly more interpersonal (M
16
= 3.94, SD = .58) than task groupwork skills (M = 3.52, SD = .60; t(143) = 7.58, p < .001),
17
with scores for both types ranging between “sometimes” to “quite often”. Scores were also
18
above the midpoint on their respective scales for both attitudes towards groupwork (M = 5.41,
19
SD = .89) and groupwork self-efficacy (M = 3.59, SD = .56). In sample 4, students again
20
perceived themselves to use significantly more interpersonal than task groupwork skills (time
21
1 = t(106) = 5.03, p < .001; time 2 = t(106) = 4.11, p < .001).
22
Concurrent Validity
23
Positive and significant correlations were found between groupwork self-efficacy and
24
both interpersonal (r = .22, p
