Why it is Necessary to Validate SILL? - Science Direct

Available online at www.sciencedirect.com

Procedia - Social and Behavioral Sciences 70 (2013) 887 – 893

Akdeniz Language Studies Conference 2012

Why it is necessary to validate SILL? *

Abstract The Strategy Inventory for Language Learning (hereafter SILL, Oxford, 1990) is the most frequently employed instrument in language learning strategy research. However; many studies which have used SILL do not report the validity of the instrument in the specific research contexts. This study presents the results of exploratory and confirmatory factor analyses of a previously validated Turkish version of SILL. The instrument was administered to 445 university English language preparatory school students. Exploratory and confirmatory factor analyses suggested a 4-factor model with 16 items. The findings of this study stress the importance of validating the instrument in specific research contexts. access under CC BY-NC-ND license. © 2012 Authors.by Published Elsevier Ltd. Open 2012The Published ElsevierbyLtd. Selection and/or peer-review under responsibility of ALSC 2012 Selection and peer-review under responsibility of ALSC 2012 Keywords: Strategy Inventory for Language Learning, confirmatory factor analysis, Turkish university students

1. Introduction Research on language learning strategies (hereafter, LLSs) in the field of English Language Teaching (hereafter, ELT) has been popular for over three decades (Cohen & Macaro, 2007). The earliest studies, also known collectively as the 'good learner' studies, focused on the identification and classification of -Manzanares, Russo strategies used by successful language learners (see, e.g. 985). As Oxford and Nyikos (1989) pointed out, it was the use of LLSs appropriate to the learners' stage of learning, personality, age, purpose for learning the language,

* Assist. Prof. Dr. . Tel.: +903742541000-1632; fax: +903742534641. E-mail address: [email protected] This study is derived from the MA dissertation of the second author under the supervision of the first author.

1877-0428 © 2012 The Authors. Published by Elsevier Ltd. Open access under CC BY-NC-ND license.

Selection and peer-review under responsibility of ALSC 2012 doi:10.1016/j.sbspro.2013.01.135

888

Amanda Yeşilbursa and Ömer Faruk İpek / Procedia - Social and Behavioral Sciences 70 (2013) 887 – 893

and type of language which enabled them to take responsibility for their own learning by enhancing learner autonomy, independence, and self-directing. A number of LLS taxonomies emerged from the results of these studies. The three most influential Malley and taxono Chamot (1990) classified LLSs under three headings: metacognitive strategies, which are higher order executive skills that plan for, monitor, or evaluate the success of a learning activity; cognitive strategies, which operate directly on information, manipulating it in order to enhance learning; and social/affective strategies, which involve interaction with another person or control over affect. Rubin (1987) proposed three categories: direct strategies, which are cognitive and metacognitive strategies; communication strategies, which are less focused on learning and more focused on communication; and Social strategies, which enable learners to engage in activities that will help them to learn. Oxford (1990), on the other hand, divided LLSs into two major classes: direct and indirect. These two classes were subdivided into six groups (memory, cognitive, and compensation strategies under the direct class; metacognitive, affective, and social strategies under the indirect class. Oxford (1990) developed the Strategy Inventory for Language Learning (hereafter, SILL) based on this distinction. Although a number of taxonomies have been proposed, it is Oxford's SILL (1990) that has been by far the most influential instrument in LLS research internationally. It has been translated into a number of languages, such a Chinese (Hsiao & Oxford, 2002) and Turkish (Demirel, 2009). It has been used in empirical studies to measure overall LLS use (e.g. Griffiths & Parr, 2001), and to investigate the relationships between LLS use and language proficiency (e.g. Park, 1997), language learners' beliefs (e.g. Yang, 1999), and cognitive style (e.g. Grossman, 2011). Although more recently there have been some speculations about the psychometric value of the instrument (Dornyei, 2005), high internal reliabilities have been reported in a number of studies (see Park, 2011, for a review) and SILL continues to be the preferred instrument in LLS research to date (see, e.g. Cesur, 2008; Demirel, 2009; Grossman, 2011). In spite of the popularity of SILL in empirical studies, relatively few studies have investigated the construct validity of the instrument in different contexts. As Park (2011) points out, the studies that have been conducted have yielded inconsistent results. Some studies have supported the original six-factor model. For example, Demirel (2009) used confirmatory factor analysis (hereafter, CFA) to validate a Turkish version of the instrument and found that the data could be explained by the original six-factor model. Using exploratory factor analysis (hereafter, EFA), Cesur (2008) found similar results. Hsiao and Oxford (2002) suggested that either a two-factor model or a six-factor model would be suitable. However indices emerging from the CFA suggested that neither model showed a good fit. Other studies have suggested very different results. For example, based on a sample of native English speakers learning foreign languages such as French and Russian and using EFA, Oxford and Nyikos (1989) found that a five factor structure best explained the data. In an EFA study with Taiwanese learners of English, Yang (1999) reported a six-factor structure. However the factors were different from those of the original SILL. More recently, Park (2011) used CFA to test a single factor model, a two-factor model, and a six-factor model, and found that while the six-factor model improved over the other two, it was not sufficient to explain the data. Despite the large body of research using the instrument, relatively few studies have used a combination of factor analysis techniques. Of the studies mentioned above, some used either EFA alone (Cesur, 2008; Oxford & Nyikos, 1989; Yang, 1999); or CFA alone (Demirel, 2009; Hsiao & Oxford, 1989; Park, 2011). Hence, given the inconsistencies of the results of the studies mentioned above it is the purpose of the current study to investigate the construct validity of SILL using both EFA and CFA techniques with data collected from a sample of Turkish university foreign language preparatory school students. Thus, it will be possible to determine first the latent factor structure of the data with EFA (Costello & Osborne, 2005), then test goodness of fit of the emerging model with CFA (Bandalos, 1996).


2. Methodology 2.1. Participants A total of 445 students (188 males, 277 females) aged between 18-21 years from the English Language Preparatory School of a state-run university in the Western Black Sea region of Turkey completed the Turkish version of SILL (Demirel, 2009). The participants were studying at the A2 level of the Common European Framework of References for Languages (CEFR) at the time of data collection. After completing a year of EFL courses, these students would go on to study in the Faculties of Science and Letters (n=340), Economics and Administration (n=68), and Engineering and Architecture (n=37). 2.2. Data collection instrument (Oxford, 1990). This instrument consists of 50 items which participants respond to via a five point Likert type scale. These items are grouped into the two main categories of direct and indirect strategies. These categories are further divided into three subcategories: memory strategies, compensation strategies, and cognitive strategies; and metacognitive strategies, affective strategies and social strategies. Thus, the original instrument has a 6 factor structure, which was validated using CFA 2.3. Data collection procedures The data collection instrument was administered to the participants six weeks into the autumn semester of the 2011-2012 academic year while the students were studying at A2 level with permission from the Preparatory School administration. The rationale behind waiting until the participants had reached A2 level before administering the data collection instrument was to ensure that each student had had sufficient time to develop an idea about learning English at the Preparatory School. The instrument was administered with the assistance of the instructors teaching the different groups. Participation in the study was voluntary, and the participants were reassured that their responses would remain anonymous and be used for research purposes only. 2.4. Data analysis procedures The usual procedure to test the construct validity of an instrument which has been adapted to a new context is CFA, which is a theory driven, confirmatory technique in which the researcher can determine how well a set of data fits to a hypothetical model (Schreiber, Stage, King, Nora, & Barlow, 2006). In CFA, variables are fixed a priori to load on specific factors (Bandalos, 1996). However, given that previous studies on the factor structure of SILL yielded inconsistent results (Park, 2011), the researchers decided to conduct exploratory factor analysis (EFA) to determine the latent factor structure of the data. An initial EFA with maximum likelihood factor extraction (MLFE) and promax rotation (cut off .40) (Costello & Osborne, 2005) was run to check the factorability of the data and to identify any items with communalities of less than .30, which were subsequently erased. EFA was run again with the remaining items. The goodness of fit of the model which emerged from the EFA was then assessed with CFA. A number of indices are generally used when assessing model fit. In the current study, the most commonly Square Error of Approximation (RMSEA) were calculated (see, e.g. Schreiber, et al., 2006). The

889


890

for CFI, TLI and RMSEA respectively (Kline, 2005). All statistical tests were run with SPSS 13.0 and AMOS 16.0. 3. Results and discussion 3.1. Results of the exploratory factor analysis An initial EFA (MLFE with promax rotation) with the 50 items of SILL showed that the data set was est of sphericity, factorable (Kaiser-Meyerp=.000). In order to ensure the hygiene of the data set, items with communalities of below .30 were deleted. As a result, items 1, 7-12, 14, 15, 18-23, 27, 33, 39, 41-44, and 50 were not included in the ensuing EFA. In other words, 27 items survived the hygiene process. A second EFA of these surviving items revealed that items 2, 13, 24 and 28 failed to load on any of the factors. This left a 23-item model of four factors with initial eigenvalues of greater than 1.00 which accounted for 48.13% of the variance. The pattern matrix of this model is given in Table 1. 3.2. Results of the confirmatory factor analysis The model elicited from the EFA stage underwent CFA in order to assess the goodness of fit. During this stage, items 11, 14-17, 32, and 38 were deleted to improve the fit, leaving a total of 16 items under the 4 factors named Metacognitive strategies (items 30, 31, 34, 35, 36, 37), Social strategies (items 45, 46, 48), Memory strategies (items 3, 4, 5, 6) and Compensation strategies (items 25, 26, 29). The results of the indices indicated that the model had good fit to the data ( RMSEA=.052). Standard coefficients ranged from .42 to .81, and standard errors ranged from .03 to .10. The Cronbach alpha coefficients for each of the factors were .83 (Metacognitive strategies), .64 (Social strategies), .72 (Memory strategies), .73 (Compensation strategies) and .84 for the whole scale, which indicated that the 4-factor model had acceptable to high internal reliability. 3.3. Discussion The results of the EFA yielded four factors. Factor one was a combination of metacognitive and cognitive strategies; factor 2 consisted of a majority of social strategies; factor 3 was entirely memory strategies; and factor 4 compensatory strategies. The current data set did not yield a factor for the affective strategies, which was in parallel with the findings of Yang (1999) and Oxford & Nyikos (1989). The results of the CFA showed that retaining only the metacognitive strategies of the first factor and the social strategies of the second factor improved the model fit. This left four factors: metacognitive, social, memory and compensation strategies.


Table 1. Pattern matrix Factor Item

1

D36

.68

D30

.67

D35

.64

B17

.62

B16

.60

D34

.59

F49

.53

D37

.49

F47

.49

E40

.48

D31

.45

2

F45

.67

F46

.56

F48

.52

D38

.48

D32

.42

3

A3

.71

A4

.68

A5

.55

A6

.55

4

C26

.85

C29

.75

C25

.41

Note: A=memory strategies; B=cognitive strategies; C=compensation strategies; D=metacognitive strategies; E=affective strategies; F=social strategies. The most striking finding of the current study is the fact that the original 50 item instrument was reduced to 16 items following the EFA and CFA procedures. Apart from Cesur (2008), who validated a 47-item form of the instrument, the other studies investigating the factorial structure of SILL confirmed all 50 items (Demirel, 2009; Hsiao & Oxford, 1989; Oxford & Nyikos, 1989; Yang, 1999). The fact that these items were not validated would suggest that they were not relevant to the current context, which would support LoCastr environment. The results of the current study would suggest the necessity of validating SILL before using it in empirical studies in different contexts, because different samples can yield different factor structures. There are a number of limitations to the study. First, it was conducted with a sample from only one

891


892

university. Future research could be conducted with larger samples. Second, the data was collected at one point in the first half of the first semester. It may be possible that the participants use different strategies as they progress through their preparatory year. Thus, future studies could utilize data collected at various points throughout the academic year. References Bandalos, D. (1996). Confirmatory factor analysis. In Stevens, J. Applied multivariate statistics for the social sciences. 3rd edition. Mahwah, NJ: Lawrence Erlbaum Associates, Inc. Cesur, M. O. (2008). A model explaining and predicting the relationship between university prep class arning styles, and academic success in foreign language. (Unpublished doctoral thesis) Cohen, A. D. & Macaro E. (2007). Language learner strategies: 30 years of research and practice. Oxford: Oxford University Press. Costello A. B., & Osborne, J. W. (2007) Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment Research & Evaluation, 10(7), 35-37. Demirel, M. (2009). The validity and reliability study of Turkish version of strategy inventory for language learners. World Applied Sciences Journal, 7(6). 708-713. The psychology of the language learner: Individual differences in second language acquisition. Mahwah, NJ: Lawrence Erlbaum Associates, Inc. Griffiths, C., & Parr, M. (2001). Language-learning strategies: Theory and perception. ELT Journal, 55, 247-254. Hsiao, T-Y., & Oxford, R. L. (2002). Comparing theories of language learning strategies: A confirmatory factor analysis. The Modern Language Journal. 368-383. Kline, R. B. (2005). Principles and practice of structural equation modelling. New York, NY: The Guilford Press. LoCastro, V. (1994). Learning strategies and learning environments. TESOL Quarterly, 28, 409-414. Grossman, D. (2011). A study of cognitive styles and strategy use by successful and unsuccessful adult learners in Switzerland. Unpublished MA thesis. School of Humanities of the University of Birmingham: Birmingham O'Malley, J. M., & Chamot, A. U. (1990). Learning strategies in second language acquisition. Cambridge: Cambridge University Press. M., Chamot, A. U., Stewner-Manzanares, G., Russo, R. strategy applications with students of English as a foreign language. TESOL Quarterly, 19, 3, 557-584.


Oxford, R. L. (1990). Language learning strategies. What every teacher should know. NewYork: Newbury House Oxford, R., & Nyikos, M. (1989). Variables affecting choice of language learning strategies by university students. Modern Language Journal, 3, 291-300. Park, G. P. (1997). Language learning strategies and English proficiency in Korean university students. Foreign Language Annals, 30, 2, 211-221. Park, G. (2011). The validation process of the SILL: A confirmatory factor analysis. English Language Teaching, 4(4.), 21-28. Politzer, R. L., & McGroarty, M. (1985). An exploratory study of learning behaviors and their relationship to gains in linguistic and communicative competence. TESOL Quarterly, 19, 1, 103-123. Rubin, J. (1987). Learner strategies: Theoretical assumptions, research history and typology. In A. L. Wenden, & J. Rubin, (Eds.), Learner Strategies in Language Learning. Cambridge: Prentice Hall International. Schreiber, J. B., Stage, F. K., King, J., Nora, A., & Barlow, E. A. (2006). Reporting structural equation modelling and confirmatory factor analysis results: A review. The Journal of Educational Research, 99, 323-337 Yang, N. System, 27, 515-535.

893