Predicting International Chinese Students Academic

Predicting International Chinese Students Academic Achievement Wang Yixin and Michael Daller Abstract   The present paper investigates the relation between English language ability and academic achievement of international Chinese students in the UK. The study aims to identify reliable measures for detecting academically at-risk students in a time and cost-effective way. In total, 60 students took part and their academic achievement was defined by the grade point average (GPA) that they got after their first year in UK higher education. At the beginning of the academic year we administered C-tests (a gap-filling format) and a writing task (adapted from the IELTS writing task 2) with several measures of lexical richness. We also included their IELTS scores in our analysis. The results showed that the best predictor of GPA was vocabulary knowledge in English in the writing task as measured with Guiraud’s index of lexical richness (types divided by the square root of tokens). Guiraud alone predicted up to 28% (r= .526) of academic achievement (GPA) in different subject areas already 9 months before the students sat the exam. The C-test scores and IELTS also yielded a positive, significant correlation with GPA, where the individual predictive value of C-test was higher than IELTS. Given the time and costs necessary for an IELTS exam, the C-test turns out to be more efficient for the prediction of academic achievement. However, these two variables were excluded in a multiple regression, and only the scores of Guiraud remained in the model.     Key words: International Chinese students, Academic achievement, Vocabulary knowledge, C-test, Guiraud’s index 1. Introduction In the past few decades, the number of Chinese students pursuing higher education abroad has increased rapidly and steadily. According to the Department for Business, Innovation and Skills (BIS, 2013) nearly a fifth of international mobile students were Chinese in 2010. Compared to study at home universities in China, it is rather challenging for these students to study abroad. Apart from the high tuition fees for international students the living expenses in the UK are about 50% higher on average than in China (NUMBEO, 2014). Besides this huge costs, Chinese international students are also facing other challenges such as language, homesick, and a possible culture shock. Study failure is a major concern

to these international students. An earlier and accurate detection of at-risk international students will be beneficial to both students themselves and the host universities. Many factors other than language, such as appropriate learning strategies in the new learning environment, motivation, general acculturation ability, and intelligence are important to international students’ academic achievement. However, there is consent among researchers that English language ability has an important factor for their academic achievement (Graham, 1987; Bellingham, 1993; Johnson and Ngor, 1996; Volet and Renshaw, 1996; Reid et. al. 1996; Briguglio, 2000; Brooks and Adams, 2002; Lee and Greene, 2007). We therefore focus on English language ability in the present study.   Literature review IELTS and TOEFL predictors of academic achievement     Previous researchers have attended to use standardized tests, IELTS (International English Language Testing System) and TOEFL (Test of English as a Foreign Language) for the prediction of study success. Some studies found significant correlations between IELTS/ TOEFL and academic achievement scores (Elder, 1993; Bellingham, 1993; Ferguson and White, 1993; Hill, Storch and Lynch, 1999; Feast, 2002; Yen and Kuzma, 2009); some studies found no significant correlation (for example, Yule and Hoffman, 1990; Gibson and Rusek, 1992; Cotton and Conrow, 1998; Dooey and Oliver; 2002; Cho and Bridgeman, 2012). A meta-analysis carried out by Wongtrirat (2011) analyses on studies that were published between   1987-2009 on the relationship between TOEFL and GPA and course completion. The results show that that the predictive value of TOEFL on GPA for course completion of international students at both the undergraduate and graduate levels was generally very low. This meta-analysis also came to the conclusion that findings about the predictive validity of IELTS for academic achievement are inconclusive.   More recent studies came to similar conclusions (Ferguson and White, 1993; Moore and Morton, 2004; Daller and Phelan, 2013; Yu, 2014).     There are at least three potential explanations for the low predictive validity of these tests. One is a purely mathematical problem with truncated samples. The reason that there is not much variability in the test scores for those students who are admitted to university (and are the potential sample for further studies) is that most universities will not admit students when their English levels are below the benchmark and that the students themselves, they will not delay the start of their course once they past the cut-off score. Therefore from a purely mathematical

point of view high correlations between the test scores and the achievement scores (e.g. GPA) are very unlikely. Ferguson and White (1993) point out that with truncated samples where the range of scores is curtailed, the magnitude of any correlation coefficient is likely to be depressed. Daller and Phelan (2013) make a similar point when they explain that if the IELTS scores are all the same at entry, ‘the correlation between these scores and any measure of study success will automatically be zero’   (2013: 177). A second problem might be a mismatch between the test tasks and the requirement in real university tasks. Moore and Morton (2004), compare the standard IELTS task 2 with a corpus of 155 university assignment tasks in terms of genre, information source, rhetorical function, and object of enquiry, and they found there seem to be important difference between university assignment writing and the writing task required to pass IELTS. This threaten the construct validity of the IELTS task. A third problem might be the fact that world-wide many students attend courses that train specifically for the IELTS exam before they apply to a UK university.   According to Yu (2014), with large amount of Chinese students eager to achieve the required IELTS scores, many test preparation centres’   endeavoured investment in order to find out the ‘short cut’   to improve candidates scores, IELTS has been turned from an assessment of English ability to an effective test to measure learners’   test skills. It turned ”into a game of ’skills and strategies” (Yu, 2014: 25).   Vocabulary knowledge as predictor of academic achievement   Vocabulary knowledge has been suggested closely related with English language ability and academic achievement (Saville-Troike, 1984; Daller & Xue, 2009; Daller & Phelan, 2013; Roche & Harrington, 2013; Harrington & Roche, 2014a, 2014b). Morris and Cobb (2003) examined the correlation between vocabulary knowledge and academic achievement by establishing vocabulary profiles of 122 students’   300word samples of writing. They found that students’   vocabulary profile results correlated significantly with grades and vocabulary profiles proved to be a useful finer assessment of ESL English language proficiency. Harrington and Roche conducted a serious of studies between 2013 and 2014 and focusing on using Times Yes/No vocabulary recognition test to detect academically at-risk students in an undergraduate English-as-a-Lingua-Franca (ELF) university programme in Oman. Vocabulary size was identified as the best predictors of GPA in their studies. Daller and Xue (2009) predicted academic achievement of international students with several measures of lexical richness, such as Guiraud, Guiraud Advanced, and D (see for more details in the

methodology/procedures section). They also used a C-test at the beginning of the academic year, which could predict almost 40% of the number of failed modules at the end of this year. The C-test is a variation of the Cloze test (Taylor, 1953), developed by Klein-Braley in 1981. Whereas the Cloze test is based on whole word deletions, the C-test format deletes every second half of every second word. According Eckes and Grotjahn (2006: 291), the C-test is a test format that yields high reliability. What C-test actually measures has been an issue of debate for many years. Some studies (Little and Singleton, 1992; Stemmer, 1992) indicated that C-test has a clear lexical focus, whilst some researchers (Sigott, 2004; Eckes and Grotjahn, 2006) state that it is a measurement of general language proficiency. However, it can be argued that these are related aspects of language proficiency. Eckes and Grotjahn, (2006) found that C-test scores correlate highly with scores in all aspects of language ability including reading, writing, listening and speaking. With its appealing feature, easy to administer, objective scoring, and high reliability, the C-test has been favoured in many studies and is also used in the present study.   Hypotheses In the present study we investigate the following hypotheses: 1. Vocabulary knowledge as measured by indices of lexical richness is a significant predictor of international students’ academic success. 2. C-test scores are significant predictors of academic success. 3. IELTS scores are less powerful predictors than measures of lexical richness or the C-test scores. Methodology Participants The participants in the present study were international Chinese students at Swansea University from across different disciplines at both undergraduate and master level. The students at undergraduate level were mainly from international 3+1 programmes (3 years at their home university and 1 year in the UK). Thus, for most of them, even they were on final year undergraduate year, it was their first year abroad. In total 60 students took our set of tests in Sep. 2013, when they just started their 1st semester. For 57 students IELTS scores were available at the beginning of the academic year. 26 out of the 60 students also took the tests in May 2014.

Measures We used a C-test and a written task in September 2013 and in May 2014. The C-test was piloted with a group of different student and a final version with five sub-texts was used. In each sub-text every second half of every second word was deleted according to the classical C-test principle (see Klein-Braley, 1997). Specialized vocabulary was reduced or replaced by simple words with similar meaning. Based on the pilot study we arranged the sub-texts of the C-test according to difficulty (starting with the easiest sub-text). In total there were 100 gaps which gave a maximum score of 100. We used exact scoring with only entirely correct answers were accepted. The writing task was adapted from practice IELTS materials (Milton, Bell & Neville, 2001). By given a topic of ‘tourism’, students were asked to produce a written text in the allotted time (30 minutes) but with no lower or upper word limitation, unlike IELTS were normally a minimum of 300 words is required. Students were also told that scores would be given in consider of both quality and quantity of the writing material they produced. Academic performance was measured by the participants’ overall Grade Point Average (GPA) collected at the end of the second semester of their oneyear study with the students’ permission. For master students, their GPA was calculated for practical reasons only from their taught session, excluding the dissertation. Procedures The tests were administered in a pen-paper format. Students were given 25 minutes on C-test and 30 minutes on the writing task. The tests in both testing rounds (Sep. 2013 and May 2014) were exactly the same. The writing task was transcribed in word format to allow for a computerised analysis with the different measures of lexical richness (see below). Spelling mistakes were corrected, abbreviations were extended and proper nouns were deleted to avoid that they were counted as “infrequent words” by some of the programs. The programme “Vocabprofiler” (Cobb, 2002) was used to calculate types and tokens for the writing task. Three measures of lexical richness where then calculated:

• Guiraud’s index = Types/ √Tokens (Guiraud, 1954) • Guiraud Advanced (GA) = Advanced Types/ √Tokens (Daller, van Hout & Treffers-Daller, 2003) • and “D” (Malvern and Richards, 1997) For GA we defined all types that are beyond the 2K level as advanced (based on the British National Corpus and Nation’s Range Programme, (Nation, 2015). “D” is a measure that indicates the lexical richness of a text through a parameter (“D”) that models the distribution of types and tokens in a text. We computed it with the appropriate CLAN command from the CHILDES project (MacWhinney, 2000). In total there were therefore five predictor variables for each testing round: Guiraud, Guiraud Advanced, “D”, C-test and IELTS scores. Results The reliability of the C-tests In order to establish the reliability of the C-tests we computed Cronbach’s alpha as measure of internal consistency. Both C-tests were sufficiently reliable, with a Cronbach’s alpha = .82 for the C-test in Sep. 2013 and .803 for the C-test in May 2014. We also carried out a factor analysis for both C-tests to investigate the dimensionality of the tests. In both cases we obtained only one factor which is a clear indication that the Ctest has only one dimension. This is in line with the previous research which supports the view that the C-test measures general language ability (Sigott, 2004; Eckes and Grotjahn, 2006). In order to get a view on the concurrent validity of the C-test we correlated it with the IELTS scores. The correlation between C-test Sep. 2013 and IELTS was 0.542 (p