Differentiating Grammar Instruction for Diverse

Differentiating Grammar Instruction for Diverse Grammatical Features Andrew Schenck and Wonkyung Choi* Columbia University and Pai Chai University

Abstract Differentiating Grammar Instruction for Diverse Grammatical Features Andrew Schenck and Wonkyung Choi An intense debate concerning the efficacy of grammar instruction has precluded intensive examination of varied grammar-focused pedagogical techniques. The purpose of this study, therefore, was to holistically examine different types of grammar-based instruction (explicit vs. implicit) and different types of morphosyntactic features. For this quasi-experimental study, three beginner English as a Foreign Language (EFL) classes were selected from a small university in South Korea (N = 102). Group 1 (n = 32) was provided with explicit emphasis of semantic concepts (images); Group 2 (n = 29) was provided with implicit emphasis of grammatical form (sentence examples); and Group 3 (n = 41) was provided with explicit emphasis of grammatical form (sentence examples and explanation). Results suggest that the complexity of form/meaning relationships must be matched to learner proficiency levels when designing explicit instruction. Less proficient learners may benefit from conscious investigation of more simplistic features, such as the lexical plural, which have a one-to-one relationship

* Wonkyung Choi is the Corresponding Author

154

외국학연구 제29집

between form and meaning. Implicit enhancement of curricula may be effective for features more cognitively complex than the learner’s proficiency level. Implicit curricula allow learners to form general hypotheses about grammatical features which are cognitively less burdensome. Keywords: Form, Meaning, Explicit Instruction, Implicit Grammar, Language Proficiency 주제어: 내재적문법학습, 의미론적 복잡성, 통사형태적 복잡성, 의미중심교수법, 형태중심 교수법

I. Introduction There has been a consistent debate concerning the efficacy of grammar-based instruction

and

feedback.

Some

have

posited

that

explicit

emphasis

of

morphosyntactic structures only exerts an ephemeral effect, while others contend that it is an effective and essential means of enhancing the acquisition process (Bitchener, Young, & Cameron, 2005; Ferris, 2004; Truscott, 1996, 1999; Van Beuningen, DeJong, & Kuiken, 2012). It is the intensity of this debate which has driven researchers to two extremes, one supporting an exclusive focus on meaning in language classrooms, and another supporting a distinct focus on grammatical form. The polarization of researchers along two pedagogical ideals has been detrimental to the development of effective language instruction. When singularly employed, neither of the two approaches can provide language learners with all of the oral or written skills they require. Instructional programs with purely meaning-focused curricula enhance communication, but do not help learners improve grammatical accuracy (Williams, 1995). Purely synthetic grammar curricula, in contrast, promote accuracy, but largely hamper processes of natural communication (Long & Robinson, 1998). Presently, research has recognized the

Differentiating Grammar Instruction for Diverse Grammatical Features

155

need to synergistically integrate both grammar and meaning-focused instruction when teaching a language (Renandya, 2013). Meta-analyses have revealed, for example, that both implicit and explicit grammar instruction is an effective means of enhancing accuracy within communicative lessons (Norris & Ortega, 2000; Spada & Tomita, 2010). Although the debate over form or meaning was an important one, its intensity has hampered efforts to examine types of grammar instruction in detail. Proponents of grammar-focused instruction have sought to support their argument in a generic sense, but have neglected to comprehensively investigate the effectiveness

of

instructional

differentiation,

based

upon

factors

such

as

characteristics of a grammatical feature and language proficiency.

II. Literature Review Instead of examining the efficacy of instructional variation for different grammatical features, theorists have consistently developed cognitive explanations for the linguistic phenomena they encounter. In the 1970’s and 80’s, many systematic aspects of language learning were ascertained to be caused by innate, immutable aspects of morphosyntactic development (Brown, 1973; Chomsky, 1975, 1981, 1986; Dulay & Burt, 1974, 1975). The “Natural Order,” for example, which described a seemingly invariant sequence of morphological development, was hypothesized to be the manifestation of an internal grammatical system (Krashen & Terrell, 1983). Some morphological features, such as the third person singular –s, were thought to be more cognitively difficult, explaining later emergence within interlanguage of L2 learners. Like the Natural Order, a regular pattern of noun phrase development, referred to as the Noun Phrase Accessibility Hierarchy (NPAH), was ascertained to be an innate process. This theoretical perspective posited that learners possess inborn abilities which allow them to learn some relative clauses, such as subject phrases (e.g., the man who came to

156


dinner), at earlier stages; other more cognitively difficult clauses, such as genitive phrases (e.g., the family whose house is next door) were thought to be learned at later stages (Izumi, 2002; Keenan & Comrie, 1977). While cognitive theories explaining linguistic development enlightened our understanding of the language learning process, they did not facilitate the development of new, more effective pedagogical techniques for grammar instruction. Because these theories asserted that acquisition is an internal, immutable process of cognitive development, they relegated the role of external stimuli to an inferior position and, thus, suggested that curricular reform or explicit pedagogical techniques were ineffective and superfluous. Although some research, such as that by Pienemann (2005), examined how instructional intervention could be introduced at a cognitively opportune time, it scrutinized only a limited number of grammatical features. As overemphasis of a cognitive paradigm precluded the exploration of crucial external stimuli such as explicit instruction and input enhancement, examination of different pedagogical techniques for disparate grammatical features went largely ignored for a number of years. A study by Williams and Evans (1998) finally examined how types of grammar pedagogy such as instructed emphasis (explicit) and flooded input (implicit), can be changed for highly disparate features such as participial adjectives (e.g., excited/exciting) and the passive. Results of the study revealed

that

different

instructional

techniques

have

different

levels

of

effectiveness depending upon the grammatical feature. Overall, the findings suggested that instructed emphasis of form increases performance of both features (Williams & Evans, 1998). While the results were insightful, the limited study of grammatical

and

instructional

differences

makes

more

research

necessary.

Additional study may allow for the development of a more comprehensive perspective, from which the best pedagogical techniques for each grammatical feature may be ascertained. The purpose of this study, therefore, is to holistically examine the following question:


How do different types of language instruction (explicit vs. implicit)

157

influence

grammatical features with highly disparate attributes?

It is hoped that such comprehensive study will allow for the development of more holistic perspectives, from which the effective differentiation of grammarfocused instruction may be possible.

III. Method 1. Participants For this quasi-experimental study, three university English as a Foreign Language classes were selected from a small university in South Korea (N = 102). Learners were distributed within the following proportions: Group 1 (n = 32), Group 2 (n = 41), and Group 3 (n = 29). All learners were at beginning levels of English proficiency, and had only limited ability to communicate in either oral or written form. These participants were all South Korean nationals who ranged in age from 18 to 21 years (See Table 1 for pre-test result). Table 1. Pre-test Scores Group 1

2

3

Number Mean N Std. Deviation Mean N Std. Deviation Mean N Std. Deviation

Pre-test Article .6853 32 .14037 .7230 41 .10251 .7118 29 .14347

Pre-test Rule-Based Plural .6719 32 .26517 .5915 41 .23556 .6552 29 .22573

Pre-test Lexical Plural .4531 32 .27184 .4837 41 .26824 .4368 29 .30672

158

Total


Mean N Std. Deviation

.7080 102 .12718

.6348 102 .24290

.4608 102 .27867

2. Procedure As pointed out by DeKeyser (2005), grammatical difficulty is accurately based upon the characteristics of two factors, form and meaning (DeKeyser, 2005). Considering these two attributes, three grammatical features associated with the noun phrase were chosen: the regular plural (e.g., cars, libraries, grasses, etc.), the lexical plural (e.g., children, men, etc.) and the definite article “the”. These features were selected because their morphosyntactic and semantic characteristics are highly disparate. While the regular plural has a simple, regular form (-s, -es), the lexical plural has many variants (e.g., men, women, mice). Both plural forms have only one semantic meaning (“many”), whereas the definite article is semantically complex (e.g., unique parts of our world, unique parts of society, parts of a list, parts of a process, etc.). Before treatments were given to the three groups, participants each took a pretest. The pretest contained two fill-in-the-blank worksheets each having 10 – 14 questions (the worksheets were obtained from learnenglishfeelgood.com) to test knowledge of correct article and plural usage. To score each test section, correct answers were divided by the total number of answers. Following the administration of the pretest, three treatments were randomly assigned to three different groups. Each group received an initial lecture using a PowerPoint with examples from the topics covered in Appendix A. The PowerPoint and explanation, however, varied depending upon the treatment. Group 1 received explicit treatment of semantic concepts (e.g., images and graphic organizers with explanation); Group 2 received implicit treatment of morphosyntactic features (e.g., sentence examples); and Group 3 received explicit treatment of morphosyntactic features (e.g., sentence examples with explanation). Following the treatment, all learners received a handout so that


159

they could review and practice the concepts learned during the treatment (See Table 2 for summary). Table 2. Summary of Treatments for Each Group Group 1 Explicit Emphasis on Meaning: PowerPoint mini-lecture with images and semantic maps.

Group 2 Implicit Emphasis on Morphosyntax: PowerPoint mini-lecture with sentence examples.

Handout: The main topics of the semantic map are provided but the information in the bubbles is blanked out. Learners must fill in the bubbles and add their own bubbles. They must then orally make sentence examples.

Handout: Learners see the sentence examples presented in the mini-lecture. They practice speaking the sentence examples. They are then encouraged to make additional sentence examples by speaking.

Group 3 Explicit Emphasis on Morphosyntax: PowerPoint mini-lecture with explanation of grammatical forms along with the same sentence examples presented to Group 2. Handout: Learners see the sentence examples presented in the mini-lecture. They circle all the grammatical features emphasized in the lecture. They practice speaking the sentence examples and make additional sentence examples.

Group 1 received an explicit focus on meaning. The teacher intentionally drew attention to semantic aspects of the target features through describing images and graphic organizers included within a PowerPoint. Learners were shown images such as those in Figure 1. Following the PowerPoint, learners were given a handout with the images and bubbles only. Students had to fill in the bubbles according to the images. They were then asked to expand on the bubbles and make sentence examples using the target features.

160


Figure 1. An example of semantic treatment for the definite article The second group received an implicit focus on form. Using a PowerPoint, learners were each shown sentence examples which contained the same target features as those presented to Group 1. Sentences such as the following were read to students (notice that the target features parallel those presented to the semantic treatment group): 1. I went to the airport last night to pick up my best friend. He just came home. 2. Could you stop by the supermarket on your way home from work? I need some milk. 3. I stopped at the store and bought a new shirt. 4. I went to the movie theater to see a new movie, but it was closed. 5. He gave me an apple. 6. He gave me some apples. Following the PowerPoint, learners were given a handout containing the same sentence examples. Learners were then asked to practice reading the sentences out loud to each other and practice speaking using vocabulary and


161

expressions from the sentences. No explicit explanation or mention of the target grammar was included within this treatment. The third group was shown the same sentences in the PowerPoint and handout as Group 2. Group 3, however, received an explicit grammar explanation.

The

teacher

explained

aspects

of

syntax

and

morphology

(sentence position of the target feature and form) for each feature and then read the same sentence examples as Group 2. Any semantic explanations were avoided. Learners were then given the same sentences in a handout as the implicit group. Group 3, however, had to circle and identify the grammatical features in the handout. After correctly identifying the target features, learners were asked to read the sentences out load and practice making their own sentences. Following the treatment, each group was directly given a post-test (Appendix B). The form of the posttest was identical to that of the pretest, although different worksheets were used (worksheets were obtained from learningenglishfeelgood.com). Because of the high degree of variability in plural forms (due primarily to lexical characteristics), five different plural forms covered in the treatment were also tested within the posttest assessment of conscious plural knowledge use (lives, sheep, mice, houses, and libraries). It was thought that such coverage would more accurately assess the influence of the treatment, since participants could not be expected to use lexical forms they had never seen. After the posttest, descriptive statistics revealing the degree of improvement were calculated and graphically charted. Next, the significance of statistical differences between pretest and posttest for the target grammatical features were determined using an ANOVA.

162


IV. Results and Discussion ANOVA calculations for pretest scores revealed that there was no significant difference between groups, suggesting that participants’ knowledge of target features was roughly equivalent at the outset of the experiment (See Table 3). Table 3. ANOVA Results for the Pre-test Sum of Squares

Definite Article

Regular Plural -s

Lexical Plural

Between Groups Within Groups Total Between Groups Within Groups Total Between Groups Within Groups Total

df

Mean Square

.026

2

.013

1.608

99

.016

1.634

101

.133

2

.067

5.826

99

.059

5.959

101

.040

2

.020

7.803

99

.079

7.843

101

F

Sig.

.806

.449

1.130

.327

.255

.775

Results of posttest scores, in contrast, revealed significant differences between groups. Posttest scores for the definite article, for example, revealed that group performance was significantly different when treatments were varied (See Table 4).

163


Table 4. ANOVA Values for the Post-test Sum of Squares Definite Article

Regular Plural –s

Lexical Plural

Between Groups Within Groups Total Between Groups Within Groups Total Between Groups Within Groups Total

df

Mean Square

.190

2

.095

2.293

99

.023

2.483

101

.072

2

.036

5.525

99

.056

5.597

101

.824

2

.412

9.738

99

.098

10.562

101

F

Sig.

4.096

.020

.648

.525

4.190

.018

ANOVA calculations revealed an F value of 4.096, which was significant to the .05 level. Scores for the lexical plural were likewise significant to the .05 level (F = 4.190; p = .018), further suggesting that the treatments had differential effects. In contrast to other grammatical features, posttest scores of the regular plural revealed no significant gains on the posttest (F = .648; p = .525), suggesting a nominal, or even nonexistent, effect. Further analysis of significant values revealed interesting associations between groups. Using the Scheffe test, two significant relationships were revealed (See Table 5). Table 5. Results of the Scheffe Test on Post-test ANOVA Calculations Dependent Variable

(I) Group Number

(J) Group Number

1

2 3

Definite Article 2

1 3

Mean Difference -.07470 .02371 .07470 .09840*

Std. Error .03590 .03902 .03590 .03693

164


1 -.02371 2 -.09840* 2 .02884 1 3 .06879 1 -.02884 Regular Plural -s 2 3 .03995 1 -.06879 3 2 -.03995 2 .19931* 1 3 .18481 1 -.19931* Lexical Plural 2 3 -.01451 1 -.18481 3 2 .01451 *. The mean difference is significant at the 0.05 level.

.03902 .03693 .05572 .06057 .05572 .05732 .06057 .05732 .07398 .08041 .07398 .07610 .08041 .07610

3

Dependent Variable

Definite Article

Regular Plural -s

Lexical Plural

Sig.

95% Confidence Interval Lower Bound

Upper Bound

.120

-.1639

.0145

.832

-.0733

.1207

.120

-.0145

.1639

.032

.0066

.1902

.832

-.1207

.0733

.032

-.1902

-.0066

.875

-.1096

.1673

.527

-.0817

.2193

.875

-.1673

.1096

.785

-.1025

.1824

.527

-.2193

.0817

.785

-.1824

.1025

.030

.0155

.3832

.076

-.0150

.3846

.030

-.3832

-.0155

.982

-.2036

.1746

.076

-.3846

.0150

.982

-.1746

.2036


165

The first relationship was between Groups 2 and 3. Group 2, which received implicit emphasis, and Group 3, which received explicit emphasis, had posttest scores that were significantly different. Overall, Group 2 revealed a .09840mean gain over Group 3 (p = .032). The improvement suggests that implicit emphasis of grammatical form may be more effective than explicit presentation for the definite article. Since learners within this study were of low proficiency, they may not have been cognitively ready to actively interpret and understand the use of the target feature. When either form or meaning (or the combination of the two) is too complex, learners may benefit from implicit enhancement of input, which

facilitates

natural

formation

of

more

general,

simplistic

principles

appropriate for lower levels of cognitive development. As for the lexical plural feature, Groups 1 and 2 showed a significant difference. Group 1 revealed a mean gain of .19931 over Group 2, which was significant to the .05 probability level (p = .030). This disparity suggests that use of semantically loaded images may be more effective for teaching the lexical grammatical features. As with findings concerning the definite article, effectiveness of the treatment may be a reflection of language proficiency. Low proficiency learners, such as those used within this study, may benefit from lexical plural features that have a 1 to 1 relationship between forms and meaning (e.g. the form, men, always means “many man”). In addition to tests for significance, comprehensive analysis of pretest/posttest gains also provides useful information about treatment effectiveness. Similar to ANOVA calculations of posttest scores, mean gains within Figure 2 appear largest for the lexical plural in Group 1 and the definite article in Group 2. On the whole, images and semantic treatments for Group 1 appear to have helped increase performance on posttest measures. The large discrepancy between the lexical plural and other features, however, suggests that effectiveness of this treatment

depends

upon

characteristics

of

the

grammatical

feature.

Since

participants within this study were early beginners, the use of images was only effective when features with few forms and meanings were emphasized. The utilization of treatments which emphasized grammatical form, such as the

166


implicit and explicit tasks associated with Groups 1 and 2 respectively, revealed different results. The implicit emphasis of form (Group 2) appeared to be more consistently effective for each grammatical feature, in contrast to other explicit treatments of either semantic or syntactic form (Groups 1 and 3). It appears that learners who received the implicit treatment had to simply interpret all of the grammatical features encountered, explaining why improvement was small, but consistent. Unlike the implicit treatment, explicit emphasis of form may require more cognitive resources than the learner is able to employ, since complex and detailed relationships are often explicitly presented to the learner. This explains why grammatical features with more complexity between form/meaning mapping, such as the definite article and regular plural, showed no improvement in Group 3 (See Figure 2). Unlike the lexical plural, the relationship between meaning and form may not be transparent in either the regular plural or definite article. Whereas the lexical plural, mice, will always mean (“many mouse”), the plural –s may have multiple meanings (many house, many shirt, many computer). Similarly, the definite article may have a myriad of meanings associated with singular form the. In conclusion, it appears that explicit treatment is effective when grammatical difficulty is suitably matched to proficiency level. Beginning learners, such as those who participated within this study, may benefit from explicit treatments that emphasize less difficult grammatical features. Low proficiency learners are able to cognitively handle explicit presentation of lexical features that have a simple 1 to 1 relationship between form and meaning. While images seem to be the most effective means to explicitly emphasize a form/meaning relationship that is developmentally appropriate, sentence examples emphasizing form may also be valuable. Explicit treatments that include features with more lexical or semantic variants may be too cognitively complex for beginners to utilize. Implicit emphasis of such features, however, may provide novice learners with examples they can utilize to form general hypotheses or simplistic principles with a lower cognitive load.


167

Figure 2. Average gains on posttests for each group

V. Summary and Conclusion An intense debate concerning the efficacy of grammar instruction has precluded intensive study of instructional differentiation according to the unique characteristics of morphosyntactic features. Investigation of varied pedagogical techniques has been further hampered by an overemphasis of cognitive theories, which have downplayed the role of external stimuli, such as explicit instruction or input enhancement. Results of this study, however, reveal that both explicit and implicit modification of grammar-focused instruction may significantly facilitate understanding and use of target features. Through analysis of results within this study, two main conclusions can be made about the effectiveness of treatments. First of all, the complexity of grammatical form, meaning, and the form/meaning relationship must be matched with learner proficiency level when designing explicit instruction. Less proficient

168


learners can benefit from conscious investigation of more simplistic features, such as the lexical plural, which have a one on one relationship between form and meaning.

More

proficient

learners

may

benefit

from

explicit

instruction

emphasizing features with more variable form/meaning relationships, such as the definite article and plural –s features. Second, implicit enhancement of curricula may be effective for the presentation of features more cognitively complex than the learner’s proficiency level. Such augmentation of curricula allows learners to form general hypotheses about grammatical features that have a less burdensome cognitive load. While results of the present study have revealed important relationships between type of instruction (explicit vs. implicit), characteristics of a grammatical feature (form vs. meaning), and language proficiency, more research is necessary. Additional examination of grammar-focused instruction must be conducted to confirm the results obtained within this study. In addition, further study is required to examine the influences of treatments on different proficiency levels (intermediate and advanced). If more extensive research is conducted, it will provide a holistic perspective which may truly enhance the efficacy of instruction for all learners and for all grammatical features.


169

References Bitchener, J., Young, S., & Cameron, D. (2005). The effect of different types of corrective feedback on ESL student writing. Journal of Second Language Writing, 14(3), 191-205. Brown, R. (1973). A first language. Cambridge, MA: Harvard University Press. Chomsky, N. (1975). The logical structure of linguistic theory. New York, N.Y.: Plenum. Chomsky,

N.

(1981).

Lectures

on

government

and

binding.

Dordrecht,

Netherlands: Foris. Chomsky, N. (1986). Knowledge of language: Its nature, origin, and use. New York, N.Y.: Praeger. DeKeyser, R. M. (2005). What makes learning second‐language grammar difficult? A review of issues. Language Learning, 55(S1), 1-25. Dulay, H.C., & Burt, M.K. (1974). Natural sequences in child language acquisition. Language Learning, 24(1), 37-53. Dulay, H.C., & Burt, M.K. (1975). A new approach to discovering universals of child second language acquisition. In D. Dato (Ed.), Developmental Psycholinguistics (Monograph Series on Language and Linguistics) (pp. 209-33). Washington, D.C.: Georgetown University Press. Ferris, D. R. (2004). The “grammar correction” debate in L2 writing: Where are we, and where do we go from here? (and what do we do in the meantime…?). Journal of Second Language Writing, 13(1), 49-62. Izumi, S. (2002). Output, input enhancement, and the noticing hypothesis: An experimental study on ESL relativization. Studies in Second Language Acquisition, 24, 541-577. Keenan, E., & Comrie, B. (1977). Noun phrase accessibility and Universal Grammar. Linguistic Inquiry, 8, 63–99. Krashen, S.D., & Terrell, T.D. (1983). The natural approach. New York: Alemany Press.

170


Long, M.H., & Robinson, P. (1998). Focus on form: Theory, research, and practice. In C. Doughty and J. Williams (Eds.), Focus on Form in Classroom Second Language Acquisition (pp. 15-41). New York, NY: Cambridge University Press. Norris, J.M. & Ortega, L. (2000). Effectiveness of L2 instruction: A research synthesis and quantitative meta-analysis. Language Learning, 50(3), 417-528. Retrieved from Education Research Complete. Pienemann,

M.

(2005).

Cross-linguistic

aspects

of

processability

theory.

Amsterdam: John Benjamins Publishing Company. Renandya, W.A. (2013). Essential factors affecting EFL learning outcomes. English Teaching, 68(4), 23-41. Spada, N., & Tomita, Y. (2010). Interactions between type of instruction and type of language feature: A meta-analysis. Language Learning, 60(2), 263-308. doi: 10.1111/j.1467-9922.2010.00562.x Truscott, J. (1996). The case against grammar correction in L2 writing classes. Language Learning, 46(2), 327-369. Truscott, J. (1999). The case for “the case against grammar correction in L2 writing classes”: A response to Ferris. Journal of Second Language Writing, 8(2), 111-122. Van Beuningen, C. G., De Jong, N. H., & Kuiken, F. (2012). Evidence on the effectiveness of comprehensive error correction in second language writing. Language Learning, 62(1), 1-41. Williams, J. (1995). Focus on form in communicative language teaching: Research findings and the classroom teacher. TESOL Journal, 4(4), 12-16. Williams, J., & Evans, J. (1998). What kind of focus and on which forms? In C. Doughty and J. Williams (Eds.), Focus on Form in Classroom Second Language Acquisition (pp. 139-155). New York, NY: Cambridge University Press.


171

Appendix A Concepts Covered with Definite Article (8 Concepts) 1. Unique things in our world: the earth, the moon, the sun, the sky 2. Unique things in our society: the homeless, the hungry, the poor 3. Unique things in our neighborhood or city: the airport, the supermarket, the store, the movie theater 4. Unique things in our situation: the cup of coffee on the counter, the bathroom in our dormitory, the sofa, the stove, the swimming pool outside my house 5. Unique things in my life: the best book I have read, the greatest day of my life, the worst day of my life 6. Unique parts of a list: the first thing is, the second thing is, the third thing is 7. Unique parts of a process: the beginning, the middle, the end; 8. Unique elements in a story: the man, the woman

Concepts Covered with Plural (14 Concepts: 7 Lexical Plurals / 7 Regular Plurals) Apples

Women

Sheep

Libraries

Furniture

Mice

Feet

Fish

Toys

Children

Pens

Lives

172


Appendix B Table B1 Post-test Scores Group

Number

Post-test Article

Post-test Rule-Based Plural

Post-test Lexical Plural

Mean

.7375

.7240

.7969

N

32

32

32

.16214

.21001

.21475

Mean

.8122

.6951

.5976

N

41

41

41

.14177

.24973

.34856

Mean

.7138

.6552

.6121

N

29

29

29

.15521

.24369

.35093

Mean

.7608

.6928

.6642

N

102

102

102

.15680

.23540

.32338

1 Std. 2 Std. 3 Std. Total Std.

Deviation

Deviation

Deviation

Deviation

E-mail: [email protected], [email protected]

논문심사일정 투고일자 : 2014.08.15 심사일자 : 2014.09.15 게재일자 : 2014.09.30