The Relationship between using Multiple-Choice Test

Available online at www.sciencedirect.com

Procedia - Social and Behavioral Sciences 70 (2013) 90 – 94

Akdeniz Language Studies Conference 2012

The relationship between using multiple-choice test-taking strategies and general language proficiency levels Narjes Ghafournia* Department of English, Neyshabur Branch, Islamic Azad University, Neyshabur, 9319613668, Iran.

Abstract The study investigated the significant interaction between using multiple-choice test-taking strategies and general English proficiency levels of Iranian EFL students. For this purpose, 137 Iranian BA students majoring in TEFL participated in the study who answered some sections of a TOEFL test and a test-taking strategy questionnaire. The results showed that more proficient test takers used total test-taking strategies as well as different subcategories of the strategies more frequently than less proficient test takers. Concerning the subcategories of test-taking strategies, significant differences were found among the participants in using guessing strategies. © 2012 Authors.by Published Elsevier Ltd. and/or peer-review under responsibility of ALSC 2012 2012The Published ElsevierbyLtd. Selection Selection and peer-review under responsibility of ALSC 2012 Key words: Cognitive psychology; test-wiseness; multiple choice test-taking strategies; general language proficiency level

1. Introduction As cognitive theory of learning matured during the 1970s, the view of learner changed from a passive recipient of knowledge to an active constructor of knowledge acting autonomously through using metacognitive skills (Mayer, 1992). Cognitive theorists consider learning as an evolving process, requiring learners to take responsibility for their learning and act as active processors of information (Barr & Tagg, 1995). The process-oriented view of language learning has been stressed by some scholars (e.g. Ellis, 1994; Mayer, 1992). Despite significant advances in the cognitive process of language learning,

*

Corresponding Author. Tel.: 00989153130060 E-mail address: [email protected]

1877-0428 © 2012 The Authors. Published by Elsevier Ltd.

Selection and peer-review under responsibility of ALSC 2012 doi:10.1016/j.sbspro.2013.01.043

Narjes Ghafournia / Procedia - Social and Behavioral Sciences 70 (2013) 90 – 94

significant questions have remained unanswered. The complicated nature of learning strategies and their interaction with learning process has made it rather difficult for the researchers to reach an agreement on a single theoretical framework within which to work. Thus, each researcher seems to have his/her own definition and classification of language learning strategies. In addition, due to the interrelated nature of learning and test taking, learning and test-taking strategies should be investigated interactively. It is wellknown that there exist some students with substantial knowledge of a foreign/second language yet unable to perform well in the tests. The assumption, as Cohen (1998) asserts, is that success in language tests depend on enough linguistic and strategic foundations. This means if test takers cannot identify what is expected from them, they cannot not reflect their actual linguistic ability. Although test-taking strategies can be used in language tests of different formats, they are of primary importance to multiple-choice questions. Multiple-choice questions are often problematic due to their emphasis on testing salience, isolation of critical information, ignorance of less relevant details, and analysis of language. In addition, multiple-choice tests are extensively used in second/foreign language programs to assess different aspects of language leaning due to high reliability, convenience in scoring, efficiency, and economy. Several studies have also shown that test-taking strategies are most susceptible to multiple-choice items (e.g. Geiger, 1997; Katalin, 2000). 2. Review of Literature and Empirical Background There are ample empirical researches confirming the positive interaction between test performance and use of certain test-taking strategies (e.g. Rupp et al, 2006). Some researchers appreciated substantial contribution of test-taking strategies to the process of construct validation of language tests (e.g. Bachman, 1990; Cohen, 1998). Cohen believed that deficiency of test takers in using appropriate testtaking strategies depreciates the validity of language tests and attenuate the test results. Despite the positive effect of implementing appropriate test-taking strategies on the process of test taking, these strategies are often ignored in many EFL/ESL teaching curricula. This may be due to some reasons. Most teachers may not have realized the effectiveness of teaching these strategies, or they are so much involved in teaching English content that they do not devote any time to teaching the strategies. This leads to the existence of too many students knowing a great deal of English yet unable to perform satisfactorily on language tests. Regarding the significance of multiple-choice test-taking strategies, the study is an attempt to scrutinize if using certain test-taking strategies to answer multiple-choice questions substantially contributes to Iranian EFL learners' test performance. The investigation is done concerning the levels of general English proficiency. The findings are of significance because multiplechoice tests are extensively used to assess different areas of learning English as a foreign language in Iran, particularly at universities. Thus, the following research question is investigated in the study: Is there any significant interaction use of certain test-taking strategies? 3. Method 3.1. Participants

91

92


The participants were 137 Iranian female sophomores, majoring in English Teaching at Islamic Azad University of Mashhad. Their age ranged from 19 to 26. Based on the scores in the TOEFL test, the participants were divided into the three groups of high, intermediate, and low proficiency. The participants whose scores were above +1 standard deviation from the mean were considered as high, between +_1 standard deviation from the mean were considered as intermediate, and below -1 standard deviation from the mean were considered as low levels of proficiency. Thus, there existed 34 high, 62 intermediate, and 41 low proficiency test takers. 3.2. Instruments A TOEFL test (Longman, 2001) was utilized to evaluate the participants' general level of English proficiency. Because the effectiveness of using multiple-choice test-taking strategies was the major concern of the study, only the three sections of the test, which included multiple-choice items were utilised. The sections were Listening Comprehension with 50 items, Structure and Written Expressions with 40 items, and Reading Comprehension with 50 items. The allotted time to answer the test was 115 minutes. In addition, a test-taking strategy questionnaire constructed by the researcher was utilized in the study. The questionnaire was constructed based on the taxonomies of test-taking strategies developed by Mcphail (1981). The questionnaire consisted of 20 statements, contextualizing the use of four groups of test-taking strategies to answer multiple-choice questions. The strategies were time using, error avoidance, guessing, and intent consideration. The questionnaire was organized on a 5-point Likert scale, which the participants had to indicate the frequency of using each strategy through selecting one of the following: a) never 1 b) seldom 2 c) sometimes 3 d) often 4 e) always 5 To validate the questionnaire, some professors in TEFL reviewed it and made some comments to increase the clarity of the items. In addition, the questionnaire was piloted empirically by a similar sample consisting of 30 participants. The reliability estimate of the questionnaire using formula was .972, which is very high. The allotted time to answer the questionnaire was 15 minutes. 3.3. Procedure All the participants initially took the TOEFL test and then completed the test-taking strategy questionnaire. Prior to taking the test and completing the questionnaire, the participants were briefed on the way to answer the test and questionnaire. 3.4. Data analysis Parametric statistical analyses, including descriptive statistics, one-way analysis of variance, Tukey HSD test, and Pearson correlation coefficients were used to investigate the research question. The analyses were done using the 16th version of SPSS software.

93


4. Results and Discussion To explore the relationship between levels of general English proficiency and using different test-taking strategies, the descriptive statistics were calculated. The results of which are indicated in Table 1. Table 1. Descriptive statistics for test-taking strategies Test-taking Strategies

Number

Minimum

Maximum

Mean

Intent Consideration Time Using Guessing Error Avoidance Total

137 137 137 137 137

1.00 4.00 4.00 7.00 24.00

5.00 20.00 19.00 30.00 101.00

2.6934 12.3212 10.9927 19.3139 64.2482

Standard Deviation 1.29797 4.31811 3.65919 5.93686 21.56242

As indicated in Table 1, the mean score of using total test-taking strategies were (M = 64.2452), and among the four subcategories of test-taking strategies, the mean score of error avoidance strategies (M = 19.3139) was the highest, whereas the mean score of intent consideration strategies (M =2.6934) was the lowest. To probe significant differences in using total test-taking strategies as well as the four subcategories of the strategies, one-way ANOVA was run, the results of which are indicated in Table 2. Table 2. Analysis for variance for test-taking strategies and the subcategories Test-taking Strategies Intent Consideration Strategies Time Using Strategies Guessing

Sum of Squares

df

Mean squares

F

Sig.

Between Groups

3.0407

2

1.523

.903

.408

11.521

2

2.760

.306

.737

124.998

2

62.499

4.938

.009

Error Avoidance Strategies

Between Groups Between Groups Between Groups

57.936

2

28.968

Total Test-taking Strategies

Between Groups

490.415

2

245.208

.524

.594

Within groups Total

62741.147

134

468.218

63231.562

136

As indicated in Table 2, no significant differences existed among the three proficiency groups in using total test-taking strategies, F (2,134) =.524, p = .594. Concerning the four subcategories of test-taking strategies, significant differences existed among the mean scores of the three groups in using guessing strategies, F (2,134) = 4.938, p = .000. No significant differences were found among the mean scores of the three groups in using error avoidance, intent consideration, and time using strategies, at p < .05. In general, the findings indicate some differences among the three levels of general English proficiency in using different test-taking strategies. High proficiency test takers used total, error

94


avoidance, and time using test-taking strategies more frequently than intermediate and low proficient test takers. Low proficiency test takers used total and guessing test-taking strategies less frequently than intermediate and high proficiency groups. Although no significant differences were found among the three groups in using total test-taking strategies, significant differences were found among the three groups in using guessing strategies, which were used more frequently by the intermediate group. Thus, the findings justify previous theoretical and empirical evidences acknowledging the significant interaction of test-taking strategies with the process of test taking (e.g. Geiger, 1997; Katalin, 2000). 5. Conclusion With the development of cognitive psychology in 1970s, considerable attention has been devoted to investigating the process of learning and test taking. Since then, many studies conducted to explore individual differences in using language learning and test-taking strategies to discover effective ways to learn second/foreign languages. Although important advances have been made, significant questions have remained unanswered in this regard. The study was an attempt to shed light on the test-taking strategic patterns of Iranian EFL learners in answering multiple-choice questions at the three levels of general English proficiency. The results proved a positive relationship between applying certain test-taking strategies and the level of language proficiency. In other words, more proficient test takers differed from less proficient test takers in using certain test-taking strategies. Thus, strategic-based instruction should be used in foreign language teaching programs to improve substantially the process of language learning. References Bachman, L. F. (1990). Fundamental consideration in language testing. Oxford: Oxford University Press. Barr, R. B. & Tagg, J. (1995). From teaching to learning. Change, 27, 13-25. Cohen, A. D. (1998). Strategies in learning and using a second language. London: Longman. Ellis, R. (1994). The study of second language acquisition. Oxford: Oxford University Press. Geiger, M. A. (1997). An examination of the relationship between answer changing, test-wiseness, and examination performance. The Journal of Experimental Education, 66, 49-60. Katalin, B. (2000). Reflections on the test-taking strategies of 7th and 11th grade Hungarian students of English. ELT Journal, 7. Mayer, R. E. (1992). Cognition and instruction: Their historic meeting within educational psychology. Journal of Educational Psychology, 84, 402-412. Mcphail, I. (1981). Why teach test-wiseness? Journal of Reading, 9, 32-38. Rupp, A. A., Ferne. T., & Choi. H. (2006). How assessing reading comprehension with multiple-choice questions shapes the construct: a cognitive processing perspective. Language Testing, 23, 441-474.