LANGUAGE TESTING

Language Testing: Then and Now/49

LANGUAGE TESTING: THEN AND NOW - Ram Ashish

Giri

Department of English Education

Tribhuvan Universtiy, Npeal

Abstract article is a brief historical overview of English language testing, particularly the testing of English as a second or foreign language. It offers a discussion of how dffirent language testing trends have emerged out of the changes taken place in the field of linguistics, particularly, applied linguistics and teaching of English as a second or foreign language. In discussing the trends, the British, North American and Australian contexts, in particular, hqve been considered. Finally, some issues of the present day language testing are reviewed. The

1. Introduction 'Tests' and 'examinations' are ancient practices. Their origin, oflen subject to interpretations, can be traced back to the 'pre-historic' period of education. In the Indian sub-continent, for example, public discourses and disputations or demonstrations of acquired abilities in a given area were a common practice in the'ancient academic period. The Greeks and Romans are also said to have practised some form of academic examinations in their glorious past. About four thousand years ago, the Chinese are believed to have adopted 'examination' in an elaborate form of a measure of ability for the first time. However, the form of examination that is current in academic settings today has evolved from and is a development of the examination that was in practice in the 19tn century. Today, the word'examination'means a series of systematic'tests'of knowledge, skill or of special ability to be carried out by an individual or an authority. So far as language testing is concerned, it is generally assumed that the history of English language testing is as old as the history of English language teaching itself because testing has always been an integral part of any English language teaching (ELT) programme, which probably began in the 15th century with the ordinance promulgated by Henry the 5'n that English should be adopted as the language of royal correspondence in the place of French. This royal ordinance is believed to have facilitated the development of English language teaching methods, writing of teaching and learning materials and designing language testing strategies. In the beginning, there was a serious lack of teaching and leaming materials. So, methods to be adopted, materials to be used and testing strategies to be followed largely depended on the concerned tutors. As a result, the Engli,sh language teaching and testing did not flourish much. In fact, it was only in the 16tn century when serious attempts were made to produce a scholarly description of the English language thereby providing a foundation for its teaching and learning. This has been marked as the formal beginning of teaching English as a foreign language which later was also termed as English as a second language depending on where the teaching of the language took place.

Journal ofNELTA

Vol.

8, No. I-2

December, 2003

50 / Ram Ashish

Giri

Teaching English as second/foreign language was commonplace in the l Tth century due to leligious ad political upheaval which had ,brought a large number of foreigners to Brita;, but comparatively less so in the 18th century as the number of immigrants dramatically dropped following truce in politics. Outside Britain, France and Denmark were the first countries where teaching of English as a foreign language started. Learners in these countries had to depend on French as the medium of instruction because teaching and learning materials did not exist in any other languages. The teaching of the Engliih language entered a new phase with the publication of Johann Christian Fick's 'Practical English Course in 1793, the earliest gratnmar translation course. However, in the third world countries, the teaching and learning of English started in 1797 with the publication of John Miller's The Tutor' English language testing in England took a significant tum in the 1850s when a system of public examinations was established which was controlled by the universities. Purpose of the article The purpose of the article is to offer a brief historical overview of English language testing, particularly testing English as a second or foreign language. The griiisn, North American and Australian contexts, in particular, have been considered. The paper also discusses the trends in language testing that emerged out of the develtpment of linguistics, particularly applied linguistics and teaching English as a secondTforeign language. Finally, some issues in the present day language testing are reviewed. Language testing is an integral part of language teaching and ' both testing and teaching ur. rcL clearly interrelated that it is virtually impossible to work in either field withouf being constantly concerned with the other' (Heaton, 1988.viii). Also, trends in language testlng have tended to follow trends in language teaching which in turn have followed trends in linguistics, pafticularly applied linguistics. Therefore, trends in language testing have been analysed here in terms of the development of a linguistic theory, its subsequent application to ELT and its influence in the emergence of English language teaching and testing approaches and methods' The article is limited to the findings of a preliminary review of some of the literature in the field of language testing, particularly second or foreign language testing. The author does not advocate for or against any particular trend, nor does he make a case for a particular approach to language testing. He simply makes a descriptive analysis of the history of testing English as a second or foreign language. The readers of the article should note that there is a difference between Engiish as a second language and English as a foreign language. The writer uses these terms interchangeably though he uses them to refer to specific context for English language teaching and testing occasionally. There are three sections of the article. The first part is a brief introduction, and a survey of the British, North American, and Australian contexts and the development of the testing English as a second or foreign language. The second section describes

Language Testing: Then and Now/51

different trends in language testing associated with the development periods of linguistic theories and their influence on English language teaching and testing' The thiid section analyses the emergence of various language testing purposes and presents a paradigm that governs the ESL/EFL testing today. A brief reference to the present day bnglish test ofihe School Leaving Certificate (SLC) Examination has been made in the final section of the article.

2. Origin of Testing of English

as a Second or Foreign Language

2.1 The British Tests

Testing of English for foreigners started much earlier in Britain than in America. In fact the Un]versit/of Cambridgi became involved in overseas testing within a few years of the establishment of the public examinations. The university sent out examination papers overseas for the firit time in 1963 to examine ten candidates and continued to develop its overseas examinations in the next four decades of the 19th century. By 1898' the University of Cambridge had 36 colonial centres with 1220 candidates. However, its formal entry in testing the English of foreigners was not until 1913 when it instituted the examination for the bertificate of Proficiency in English meant to be for "foreigners" who sought proof of their practical knowledge of the language with a view to teaching it in foreign schools. i.eflecting the growing interest in the direct method of teaching, which required rand of the language for active classroom use rather than of teachers' reliable "o-t academic or descriptive ability, the certificate examination was, however, modelled on the traditional native speaker language syllabus, with the essay as the key feature. In addition to a paper in phonetics, there were a literature paper and a paper involving translation. Oiaf examinations, writing questions such as letter writing and composition either on a narrative, descriptive or an imaginative topic and reading and writing main points, and a summary of a story were added in the certificate examination in the iubsequent years. The- certificate received a major boost when in 1931 , the University recognised it as the equivalent of the standard of English required of all.students, British or f6reign, before tlieir entrance to the University. With the growing demands of certifrcaiion in English, different levels of the certificate were introduced and later conducted through the British Council. For the Cambridge tests, the issue of reliability was not an important consideration, and so the examinations remained'untouched by psychometric notions. By comparing the College Board's examination of 1931 with the one practised until recently, one can easily see the lack of concern about reliability. While the Board's examination had its concentration on language with true-false questions without curricular concern, the Cambridge examination put emphasis on curriculum with literature and absolute reliance on subjective grading. Therefore, it can be concluded that the Cambridge examinations, which were innovative and almost non-academic in their desire to test language use rather than knowledge about language, are more concerned about what to test (i.e., about curriculum) than about how to test. The ESL/EFL tests of General Certificate of secondary Education (GCSE) are produced and administered for the non-English speaking background (I'JESB) students Journal ofNELTA

Vol. 8, No.

l-2

December. 2003

52 / Ram Ashish

Giri

enrolled or seeking effolment in the British schools. At the higher level, examination boards develop and administer O (ordinary) and A (Advanced) level examinations and also ESL and EFL proficiency tests. The EFL and ESL tests are given to applicants to the British instituti,ons to determine their ESL/EFL proficiency. The International English Language Testing System (IELTS), which is for the students of Year 11 onward, is a measu.e oi English Language proficiency for the students seeking admission for higher education or training in British or Australian universities, and has been jointly developed by University of Cambridge Language Examination Syndicate (UCE"ES), The British Council (BC) and International Development Program (IDP) of Australian Universities. The test reflects the ideas of communicative language teaching and is probably the first standard communicative language test administered over a large population. tbl,ts, which is widely welcomed by the British and Australian universities, consists of two sections - General and Modular. The general section comprises of a listening test and an oral interview intended to test the oral skills' The modular section, on the other hand, is intended to test the written skills, reading and writing. The modules are further divided into sub-modules. The overall format for the modules are the same; they all contain texts from books, journals, reports, etc., related to a specific subject area and involve candidates instudy skills necessary for academic studies. British tests are highly innovative in content and format, and lay great emphasis on validity on the examination construction procedures, which rely on expert judgement. 2.2T|r.e North American Tests The Immigration Act of 1924 allowed visas outside the quotas for foreign students, but the Immigration Department was unhappy that many non-quota immigrant students gained aJmission to ttre United States totally unfit because of their insufficient knowledge of the English language. The Deparlment asked for an indication of the exact knowledge of the English language a student must have before he or she can be accepted in the United States of America (Spolsky 1993:4). th. Coll"ge Entrance Examination Board was asked to develop a test that measured immigrant students' knowledge of English and determined their ability in the language. The American Association of College Registrars also requested the Board to consider an addition to their special examination designed to test the ability of a foreign student in the use of the English language, which later became a mandatory requirement for attendance by American College institutions. Consequently, the Board set up a commission, which recommended an examination to test ability to understand, to read English intelligibly, and to understand spoken English. The aural comprehension test should include simple English prose read slowly, and simple directions given conversationally. The Board put forward, in 1930, the English Competence Examination, which is perhaps the earliest test intended specifically for English as a Second Language in the Unites States. The specifications of the examinations were as follows: Journal ofNELTA

Vol.8, No. l-2

December, 2003

Language Testing: Then and Now/53

1.

Four one-paragraphpassages - One narrative - One historical, topical or journalistic, - One critical, and - One scientific

2. A longer passage dealing with debatable ideas, 3. A direct dictation, and the reproduction from memory of a dictated passage, 4. An oral test with ten topics prepared for the examiner, 5. Composition on a selected topic. In the last two decades, students seeking eruolment in the North American universities have been given one of the three tests, Comprehensive English Language Test (CELT), Michigan English Language Assessment Battery (MELAB) or Test of English as a Foreign Language (TOEFL). Among these, TOEFL is considered to be a secured evidence of English language proficiency by most North American universities. These tests have traditionally involved the assessment of listening, reading, vocabulary and grammar. The TOEFL has also included Test of Written English (TWE) and Test of Spoken English (TSE) as direct measures of oral and writing proficiency. More recently, the TOEFL has been computerised. Students can enter their answers directly on the computer and results are known within days. The test called American Language Institute of Georgetown University (ALIGU) test is another test of academic English given to applicants of scholarships awarded by various US government agencies. For the school level ESL/EFL proficiency, only the Language Assessment Battery GAB), which assesses receptive as well as productive skills, and Secondary Level English Proficiency (SLEP), which assesses receptive skills, are given. 2.3 The Australian Tests The Australian system of education follows the British one and the tests of ESL or EFL are more or less similar to those of the British. Institutes like Australian Council for Educational Research (ACER), Language Testing Rqsearch Centre (LTRC) and National Centre for English Language Teaching and Research (NCELTR) have developed and implemented test of English language proficiency for specific levels, tasks and situations almost all of which are for the students of Non-English Speaking Background O{ESB) studying in ESL programmes in Australia. More recently, Australian Second Language Proficiency Ratings (ASLPRXIngram 1984), a structured interview procedure and rating scale has been designed to measure general proficiency in a language learned as a second or foreign language. Candidates are rated on a descriptive (criterion-referenced) scale in which the language proficiency in the four basic communicative skills are specified by a number, title and behavioural description. The scale consists of descriptions of language behaviour at nine levels - 0 to 5, five being the native-like proficiency. The ASLPR provides three kinds of information about each level - (1) a description of the language behaviour appropriate to the level, 2) a series of examples of observed behaviour and (3) a series of comments explaining the keys features in a given area. Journal ofNELTA

Vol. B, No. 1-2

December, 2003

54 / Ram Ashish

Giri

for use The ASLpR is based on the absolute proflciency ratings and is_designed pemlits with learners whose education and employrnent is more diverse. It, therefore, it is a scale ;i;; i"g*e of differentiation. The ASi-PR is not a test instrument, rather (ILR) Roundtable Language una u se"t of procedures which are based on Interagency Languages/ Foreign Testing Oral Interview derived from the American Council on the rater has the Educational Testing r"*ir. (ACTEFL/ETS)(Alderqol 9t aI. 1987)- The more lit..ry 16.eplace tli suggested tasks or exercises with the ones he or she regards level and uppiJp.i"t.,'provided ttr"aT ttre new tasks and exercises are of equal diff,rculty levels communicative or linguistic complexity. However, no guidance at the functional, provided' is complexity and as to how to make the taski or exercises of equal difficulty in every part of the students/individuals, oi thousands section, To sum up this their world, take various English language proficiency tests each year to- demonstrate are then used ability/proficiency ir-gigiirft as"a f6reign language, the s.cores of which and Australia UK, USA, Uy aiffi.*t institJions"of English Telkine-countries^like offering as such purposes i*iau,fo. ,"r."nirr! tn. .*Aiiates for u tt.tl*btt of different in career' Test admission to an edu"cational programme, employment or advancement the candidates' scores are related to varioui aspects of pioficiency demonstrating In the last writing' and reading speaking, listening iung,rug. ability in different skiun aspects different measure to developed thrJe d"ecad"s, n,r-.rous test batteries have 6een of English language proficiency of the non-native speakers. TOEFL American engUsfr language test batteries such as MELAB, ALIGU and now. IELTS made have been in use in Xsian cointr]es including Nepal for a long time its way to Nepal about 10 Years ago.

"Ingeneral,Britis'handQ.{orth)AmericanEFLproficiencytestsrepresent test development. North American tradition in different aiproaches to language "on

properlies of tests' Issues such as heavily baied _psycho*.tti9 and prediciive validity ar-e of particular interest in this i"ri"uuifitv una oUj".ti"ity of scoring and generalisibility of results play a dominant tradition. Hence"o"turrent often .ot. in the development of test metho?s. Foiexample,-multiple-choice items aretest is if-the even consistency internal used in testing r.L.pii* skills to gain desired high achieve to in orde-r expected to measure communicative competence. Moreover, in inter-rater reliability, ih; ;;" of trained i"or.rr and detailed specific instructions the .onaurting interviJws are highly recommended for testing productive skills in

il&;;g; i6rt ir

tradition.

The British tradition puts emphasis on the specification of test content and content and expert judgement. While reliaUility reieive less attention in this tradition, that generalised of the test designers' It can be face vaiidity are the major "orrceirrs variability in their formats and include various communicative British tests enjoy ;; activities.

Testing 3. Trends in Language "in "second

oi foreign language testing is not a. recent concept. Modelling languageTheorists and laireuage teaching meth6dologists have developed. and used point t"rffi,"ai"ir, *rr"i"ii--urcn"a tte language tEaching models existing-in_particular of time. In others words trends in ling'uage testing have always followed trends in language teaching. Journal ofNELTA

Vol. B. No. I-2

December, 2003

Language Testing: Then and Now/55

Language testing specialistsi theorists have always stressed the need to base the development and use of language tests on a theory of language proficiency. More recently, they have called for the incorporation of a theoretical framework of what language proficiency is with the methods and technology involved in measuring it. Bachman (1990:9) describes the complexity of the problem of defining and measuring ability in the following terms:

All

language tests must be based on a clear definition of language abilities, whether this derives from a language teaching syllabus or a general theory of language

ability, and must utilise some procedure for eliciting language performance. However, there are different ways of looking at language ability, and as a result, there is a wide variation in its definition. These different definitions of language ability focus on different aspects of the ability, and so developing a single language testing design to accommodate all of these aspects is very complicated. Furthermore, the test instruments or methods used to elicit language abilities are themselves based on the assumed level of the abilities, making it unceftain whether the test measures will elicit data to characterise an individual's language performance in non-testing situations. Language testers, therefore, face a complicated dilemma of attempting to measure abilities that are not precisely defined, and of using methods to elicit language abilities that themselves are manifestations of the very same language abilities. In other words, the test methods or data elicitation procedures one uses to measure language abilities are characteristically related to the way one views these abilities. The most important task of a language tester, therefore, is to define language abilities in such a way that the test methods or response elicitation procedures applied to elicit language test performance that is characteristic of language performance in non-testing situations (Bachman 1990:9). Given below, then, is an account of different trends in language testing and the ways the language has been viewed in these trends. 3.1 The Pre-Scientific Trend Language testing cannot be separated from an understanding ofthe nature oflanguage, of language abilities, and therefore, of its learning and teaching. Different theorists of language testing, therefore, viewed language and language abilities differently, which in consequence, have emerged as language testing trends in different but sometimes overlapping periods. Prior to 1960s, for example, language was viewed as a means to approach the target culture and language learning as a way of training the mind (McGarrell 1981). The target language was an object of study, which could be separated from its context for the purpose of teaching and learning. In this 'separatist' view (Davies 1968:216), the relationship between society and language is not considered important. The target language is separated from its use for the purpose of encouraging learners to an exhaustive study of the target literature, and for training their mind through practice in dealing with varied learning situations and materials. Language leaming, therefore, was viewed as a process of learning grammatical rules and vocabulary and applying those rules and words in writing and translation exercises. Language tesling was seen as testing the learning of words and grammatical accuracy through writihg exercises. Journal ofNELTA

VoL B. No.

I-2

December, 2003

56 / Ram Ashish

Giri

This period, in the history of language testing, is considered to be the 'intuitive' stage (Madsin 1983) or the pre-scientific era (Spolsky 1978) in that decision about teaitring and testing mainly depended on the personal discretion or subjective judgement of the teachers or testers. The pre-scientific trend embraced the traditional grammar - translation approach in which learners were required to apply the rules of grammar taught to them deductively and the vocabulaiy they learned by memorisation. The grammar translation course was basically a grammatical progression, which included grammatical structures, and vocabulary need.i to .*pr.s thim, both selected arbitrarily on the basis of principles of easy to difficult, familiar to unfamiliar and concrete to abstract' The items that were viewed as simple, easy and familiar were taught before the ones that were less frequent, abstract, complex or difficult' The merit of this trend of language testing is that it allowed a global evaluation of the learners' ability in the targeilanguage tfirough their composition and writing activities which required learneri to synthesise their knowledge of the rules and gain vogabulary and to produce syntactically acceptable utterances. To help learners coiitrol over formal accuracy, tasks sr,.it ut an analysis of sentence structure, labelling its parts or synthesising parls.into larger units were commonly practised (Ingram 1985)' To reinforce the mastery o,r.r grammatical accuracy further, abundant writing activities letter, prdcis, and open-ended answers based on reading and such as translation, 'ihe testers made their judgement about a. candidate's "rruy, cbmprehension were organised. abiliiy in the target langirage on the basis of the knowledge of grammar and vocabulary exemplified in these forms of writing (Madsen 1983)' This trend is characterised by a lack of testers' concern for statistic analysis, objectivity, or test reliability (Howard 1980). In this app_roach, the onus was placed on ifr. r"US.liive judgement of the learners' language proficiency Ther9f91e,.a language teacher test within the framework of this trend was tfie responsibility of the individual who set his/her own standards or criteria. A typical language test comprised of translation, framing of question answel and composition exercises. Oral tests were rarely given.

3.2 The Psychometric-structuralist trend During 1960s, the behaviourist language learning theories_inspired rytlly by_Skinner's work it 957), and structural linguiitici emerging out of -the work of Fries (1952) and Bloomfield (tl::;, evolved into what Spolsky (r978) calls_ a psychometric- structuralist trend. In this trend, which is also sometimes known as the era of scientific language patterns testing (Madsen 1983), language was viewed as a multitude of discrete-point which could be learned Uy t[e stimulus-response habit formation process (Ingram of the 1985). Language learning was "... a process of^acquiring conscious.control through largely language, a.second phonotrogicil, g-rammaticil, and lexical patterns of rt,tay' ani anulysis of these patterns as a body of knowledge" (Carroll 1983).

of the Severai approachesio language teaching appeared during this period some the notable ones are - the Audioling-uaiapproach, tie mimicry-memorisation method, to approaches structural The oral-aural method, the oral struclural method, and so on. as learning language Language teaching regard language as a set of structures, and

Journal ofNELTA

Vol. B, No.

l-2

December, 2003

Language Testing: Then and Now/57

gaining mastery over the structure through the process of repetition and practice. Language ability is seen as the ability to handle discrete elements of the language system and develop aspects of individual language skills. The discrete-point approach to testing measured language proficiency by testing learners' knowledge of the discrete items of grammar and vocabulary and of discrete items of language skills by taking one item or aspect at a time. Discrete-point tests became the most widely used tests worldwide in 1960s and 1970s and are still popularly practised in many parts of the world. So much so that the psychometric-structuralist trend in testing a second or foreign language became synonymous to objective testing. Lado's work (1961), which has been seminal in introducing objective testing to second or foreign language testing, claimed a universal appeal and received a great deal of support from the linguists, methodologists, teachers, course designers and test developers everywhere in the subsequent years. It "has correctly pointed to the desirability of testing for very specific items of language knowledge and skills, judiciously sampled from the usually enoffnous pool of possible items. This procedure makes it a highly reliable and valid testing" (Spolsky i978 quoted in McGarrell 1981). There has been a lot of criticism of this approach to language teaching and testing also. Ingram (1985) suggests that language is not just the sum of its parts, butthe parts have to bb.mobilised and integrated together to carry out particular tasks in particular situations. Davies (1963) rejects this approach by saying that whole language development cannot and should be equated with the separate development of its constituent parts because the whole is always greater than any one of its constituent parts.

Oller (1979) makes a similar comment in his seminal work in which he discusses the defect of the discrete point approach to language testing by saying that it suffers from several deficiencies, and suggests that the problem with this approach is that it depends on language proficiency being neatly quantifiable. He outlines its deficiency in the following terms: Discrete point analysis necessarily breaks the elements of language apart and tries to teach them (or test them) separately with little or no attention to the way those elements interact in a larger context of communication. What makes it ineffective as a basis for teaching or testing languages is that crucial properties of language are lost when its elements are separated. The fact is that in any system where the parts interact to produce properties and qualities that do not exist in the part separately, the whole is greater than the sum of its parts...organisational constraints themselves become crucial properlies of the system which simply cannot be found in the parts Separately (Oller 197 9 :I 12) language and its social context are complementary to one another. Language knowledge, therefore, must be tested in language contexts to see if a person can communicate appropriately in the given context of situation. Testing formal knowledge of language, i.e. linguistic competence, is necessary but not sufficient to predict that the person can use the language effectively in

As the quotation above implies,

Jourrql of'NELTA

Vol. 8, No.

I-2

December, 2003

58 / Ram Ashish

Giri

a given situation. A person taking a driving test or music test, for example' must of engine demonstrate their driving or perfoining abiliiy as a whole. The knowledge

performer' parts or the keyboard does not necessarily make him or her a good driver or weir (1988) goes on to suggest that grammatical (linguistic) competence is not a good in indicator of one's communlJative skil'is at all. A tester, therefore, should be interested than of rather the development and measurement of communicative competencelinguistic cbmpetence. Weir's suggestion is in line with Spolsky (1978) and Morrow's that, instead of attempting l9 ltilOy commints which *.t. -id. a decade earlier should ,rtuUtir6 a person's knowledge of a language in terms of elements and skills, one setting. attempt to test that person'r ubility to perform in a specified sociolinguistic

3.3 The Psycholinguistic-sociolinguistic trend A learner ii exposEd to the targef language through its skills (listening, speaking,

a language reading and writing) and elements (sounds, words, grammar, etc.). However parts, is learned in its eniirety, i.e. language skills and elements are leamed as related to approach integrative or holistic not as isolated component, oiu iystem. This process unitary is a and languag. t.*tting implils that learning of language -in language "goes "the individual skilis and elements and requires to be mast.".y b"yorlJ thai it "r properl| organised in varying social situations. The socio-linguistic Yt_"Y. of language porit, ihut 6rr.ry utterance oJ"rrrc in a culturally determined but identifiable context of What situation in which individuals engage as participants to perform various roles' factors' they say and how they say it are determinedby the cultural as well as contextual as the known also is which trend, The psycholinguistic-sociolinguistic to the trend opposing an as integrative-ro.lolingniiic trend (McGirrell 1981), emerged language to psyJhometric-structluralist trend or the discrete - point/atomic approach i.rtlng and acknowledge that language proficiency actualised in performance,-more than is the accumulation of dfscrete elements of language, and therefore language learning therefore testing more than gaining control on a set of habits or structures. Language a candidate's should be vlewed as a measure of communicative competence revealed by 1988) 198^0,.Weir (Howard performance in a given social situation in order for making the discussion "ury to follow, three separate aspects of the the three trend has been identified and discussed in separate headings as to bring about (1) the different though often complementary views together- The three aspects are psycholinguisiic aspect, (2) the integralive aspect' and (3) the sociolinguistic aspect'

3.3.1 The psycholinguistic aspect and cognitive-lelTing theories,. psycholinguistics Influenced^by cognit"ive psychology 'an leailing procisses. Unlike the behaviouristic approach views mind as central ti which regards language learning ui un external mechanistic process shaped by as a environmental factJrs, psycholinluistic approach considers language. learning production, ;ilpi;. internal and abstract mental p.bb.tt. underlying the planning of languig9. lhis aspect of language learning, which percfption u"a as language acquisitioi and subseqt*lly second language acquisition, iater'developed "o*piehension linguistics (Chomsky .1957) Ch.omskyan linguistics J.u*, considerabty of Cn"o*skyan -Saussnieun linguistics in the sense that while the former i, iunau-.ntally different from Journal ofNELTA

Vol.8, No. 1-2

December, 2003

Language Testing: Then rnd Now/59

which exists in every normal human being, the later describes it as a physical process of learning language structures and systems. Bloomfield (1933), who is regarded as the mentor of structural linguistics, took a more concrete approach to language leaming and emphasized the process of segmenting and classifring the physical features of utterances (Chomsky's surface structures) and disregarded the underlying structures (or what Chomsky calls deep structures) of language (Crystal 1997). Chomskyan theory claims that every mind is pre-programmed with the ability of learning a language in which internalisation of the underlying rules is the key to successful language learning and adequate exposure to the target language the only condition for it. Chomsky's work provided a basis for describing a child's (learner's) language as a legitimate, rule governed, and consistently evolving system. This means that measures of a child or learner's performance should have predictive ability of hislher competence, which is unseen, unobservable, and abstract. The psycholinguistic theory of language learning in 1960's led to the development of various language-teaching schemes, which treated language learning as a process of acquiring conscious control and understanding of language systems. The theory advocatei that if a learner has a proper degree of cognitive control over the language structures, facility will develop automatically with use of the language in meaningful situations' (McGarrell 1 98 1 :29) Language testing, under this thqory, shifted its emphasis from linguistic accuracy to functional ability. Language tests reflected on problem-solving approaches and were expected to reveal what underlying rules the learners had internalised. Target language errbrs received a positive attitude and were considered to be an indication of the learners' level of the transitional competence (Corder 1967) or 'interlanguage' (Selinker 1972). Reliability and validity were still central to the language testing (McGanell 1981) 3.3.2 The Integrative Aspect The Integrative Aspect proposes to assess the candidate's capacity to use a number of language abilities at the same time. In this respect, the integrative approach to testing contrasts sharply with the discrete-point approach. The approach is commonly used in proficiency tests because it reflects read language use more closely (Banerjee in Byram, 2OOO). Oller (1979) the mentor of what is popularly known as Unitary Competence Hypothesis (UCH) proposed that there is a single, unitary factor that underlies language pioficiency, and argued that argued that language competence is indivisible and that the iour macro skills of listening, speaking reading and writing are so closely irser related that it is appropriate to consider a general language proficiency factor rathpr than to distinguish finely between them. As Oller (1979:112) writes ...the concept of an integrative test was born in contrast with the definition of a discrete point test. If discrete items take languages skill apart, integrative tests put it back iogether. Whereas discrete items attempt to test of language one bit at a time, integrative tests attempt to assess a knowledge -capacity to use many bits all at the same time, and possibly while learner's exercising ieveral presumed components of a grammatical system, and perhaps more than bne of the traditionally recognised skills or aspects of skills. Journal ofNELTA

Vol.8, No. 1-2

December, 2003

60 / Ram Ashish

Giri

can test several Oller's statement above suggests that the integrative apploach assumes that the general traditionally named elements and skills or aspect because it all language elements and language proficiency is a single principal facior underlying of overall language skills. He claims that the cloze or diciation can provide measures as in discrete-point proficiency, which, unlike testing one element or skill at a time The cloze and testing tradition, assesses several elements and/or skills simultaneously' competence' but they are a dictation, as he acknowledges may not test communicative ability' as they require better guide to learners' uplitrrd. and potential communicative pressure, pragmatics' etc' They them to perform under real life constraints such as time mark and have respectable are also practicable to administer, economical to set and degree of reliability (Weir 1988). a perceptual task, a The cloze test has been criticised as a cognitive rather than knowledge, unreliable, its means to evaluate notional knowledge, rather than functional in measuring EFL validity in measuring EFL proficiency questionable, its validity measure of comprehension proficiency questiona[le, not a unitary technique and a crude Morrow (1979), McNamara 1996) lDavi"s t6g3, Weir 1988, Carroll 1980, Davies (1988) who However, a serious criticism of oller's work comes from evidence that there are two says that the proposition of UCH conflicts with the between knowing how to competencies - reception and production. The difference is stronger than the bluned analyse input and knowing how to construct output claims to be the manifestation correspondence between the two processes which oller are not a of a single unitary factor. As for the cloze and dictation tests, they because of their versatile manifestation of ,rarious aspects of the ucH, rather, they, nature, contain most language skills' 3.3.3 The sociolinguistic asPect a language involves The sociolinguistiJ aspect is based on the premise that knowing It involves, alongside the more than the knowledge of language elements and skills. A candidate should' knowledge of language code, ability to use the language in context' and skills communicatively therefore, be tested for his ability to use linguistic elements of ability in using a given and appropriately in a given situation. There are two aspects code (linguistic competence), and (b) language (a) knowledle of the language -which includes knowledge of the culture and factors

knowledge of socio-linguistic t.rt should measure ability to use language in both tlpes of social rules. A languagJ -test measuring the linguistic aspect of the ability only cannot knowledge becausJ a contexts' account for how well the uSeI can communicate in given social (1972) seminal work which The sociolinguistic aspect draws upon Hymes's as its linguistic context' suggested that the social context of a message is as important missing or and missing the socio-cultural clues of a message may result in several projects of misunderstanding.the message altogether. Hymes work inspired and grammar developing new teaching methods' language syllabuses, learning materials Journal ofNELTA

Vol. 8, No.

l-2

December, 2003

Language Testing: Then and Now/61

books. Wilkins (1976), for example, outlined the notional categories of communicative needs. Munby (1981) provided a scheme of elaborate specifications of the learners' target language communicative needs, the Van Ek (i979) devised a communicative language proficiency course for the Threshold Level, and Leech (1975) wrote an elaborate communicative grammar. All this led to the development of teaching schemes aiming at meeting the target language communicative needs of the learners. So far as language testing is concerned, this aspect calls for testing schemes that reveal the learners' ability in using the language in communicative situations. Language tests should evaluate not only the learners' knowledge of the elements and skills but also their ability to comprehend and produce utterances that are both situationally and contextually appropriately.

4

The Communicative Language Testing Several years after Hymes's two-dimensional model of communicative competence, which consisted of both linguistic and sociolinguistic elements, several models- of communicative language teaching and testing appeared in Europe as well as in North America. In 1980s, particularly in early 1990s, language testing models such as those proposed by Morrow (1979), Canale and Swain (1980) and Bachman (1990) advocated that language testing should be concerned with both competence and performance, i.e., with (a) what the learner knows about the form and about how to use it appropriately, and (b) the extent to which s/he actually demonstrates language knowledge in meaningful performance (Weir, 1 990). The communicative approach to language testing, though it is still said to fall in the psycholinguistic-sociolinguistic trend, has following the works of Bachman (1990), Bachman and Palmer (1996) and McNamara (1996), established itself as a promising approach to second language testing. Bachman and Palmer have moved ahead of what Hymes (1972) proposed and have added additional features in communicative competence. They suggest that a communicative language test has four important dimensions, namely (i) specificity of context, (ii) authenticity of materials, (iii) authenticity of test tasks, and (iv) simulation of real life situation, and must operationalise all four types of competencies - linguistic, sociolinguistic, discourse and strategic. In other words, in a communicative language test, communicative performance tasks should be representative of the types of tasks and materials the candidates are likely to encounter in real-life situations and at the same time, corespond to normal language use (Weir 1990). Despite the progress in developing communicative language testing models, only a small number of communicative language tests in second language have been published. The UCLES's Certificates of Communicative Skills in English, and Australian Educational Council's (AEC) ESL Scales and National language and Literacy Institute of Australia's (NLLIA) ESL Development: Language and Literacy in Schools are currently in use in the British and Australian education systems. Journal ofNELTA

Vol.

I, No. I-2

December, 2003

Giri

62 / Ram Ashish

5.

I,anguage Testing TodaY and Today, lung.rug" testi particularly ESL/EFL tests, serve a wide variety of purposes figure the test is given for several formative or summative goals. The purposes' as below shows, can be summarised under general and specific purposes: 5.1 The Purpose of Assessment General

Administrative

Specilic

Description

General Placement

knowing learners' general ability in the language placing learners in ability groups providing proof of language ability promoting learners to a higher level selecting able learners' for a purpose

Certification Promotion Selection

encouraging learners to learn ascertaining areas of difficulties in the language determining learners' readiness for a course Prognosis evidence for progress checking whether learners are making progress feedback to examinee monitoring learners' progress

Motivation Diagnosis lnstructional

Improvement

evaluating teaching or a course

evaluation

reviewing programme achievement

experimentation finding more about language and its learning

Research

kntwledge

knowing about communication and learning strategies

Figure: Language testing Purpose (based on Cohen 1994; and Madsen'1983)

of assessment These purposes are not always mutually exclusive to each other as one form of may also be used for several other purposes. The assessment given for finding evidence

or pfogress, for example, may also be used for researching students' learning styles of purpose communication strategies. A test given for a general purpose may also serve the may also motivating students towards learning. Similarly, a test given for testing achievement be used for certification purposes.

WH- Paradigm of Assessment in The what, how, whe; and by whom of a test, as what is known as the 'WH- Paradigm of why Assessment' (Cajkler and Addleman 1992) below suggests' largely depends on it.

5.2 The

Journal ofNELTA

Vol. 8, No.

I-2

December, 2003

Language Testing: Thcn and Now/63

WHAT [skills or knowledge learned] HOW fcriterion-referenced]

WHEN Icontinuous]

WHO lteachersl

WHICH FORMAT Iintegrative/subj ective]

summative

WHY fformative] discrete-

administrators point/objective

end-of-term

norm-referenced

use of skills/application of knowledge

(Figure # based on Cajkler and Addleman 2000)

As the figure # above suggests, what is assessed (how well the learners are achieving or how well they are learning), how is the results of the assessment is

interpreted (norm-referenced or criterion-referenced), which format/design of is used, and who conducts the assessment and who uses the results are all

assessment

dependent on why (purpose) of the assessment. Of the three general and fourteen specific purposes of language assessment, as outlined in the figure # above, a national test like the English test of the School Leaving Certificate (SLC) Examination is given only for a general administrative purpose. Under this general purpose, it serves two specific purposes - general proficiency and certification or achievement. The general language proficiency is indicative of the learners' target language linguistic competence as well as their ability to use this competence in actual communicative situations. As a general proficiency test, the English test of the SLC, is, as one would expect, a test of general language skills and elements. The SLC reports an aggregate of scores only, and it does not contain information on the outcomes of the test components nor does it provide descriptions of abilities. The institutes, which may use the test results for selection purposes, make their decisions on the basis of the aggregates. Decision about certification is usually made on the basis of the achievement of the target course. What type of certification (first, second or third division) is issued depends on what has been achieved or learned from what has been taught in the course. The English test of the SLC, in this sense, is an achievement test. In other words, the SLC examination is a measure of achievement, the results of which are sometimes used for screening or selection purposes. A score given on a subject is an indication of how well a candidate has taken the examination. It is not usually an indication of how well he has learned durins the course. Journal ofNELTA

Vol. B,

No l-2

December, 2003

64 / Ram Ashish

Ciri

Conclusion 'What to test' in language has always been an important consideration and is something that has undergone reviews over time. The 20th century, particularly the second half witnessed considerable changes and shifts in the theory and practice of language teaching and testing. Though a timeline'for the changes cannot be explicitly drawn, it appears that every decade of the second half of the 20th century experienced same sort o?-change. In language testing 'What is to be tested' follows what goes in the teaching and learning of that language. In other words, language testing tends to follow trends of language teaching

The trends in the second or foreign language testing may be described as analytical, global and communicative. The analytical approach to language testing follows the description of language, which suggests that language consists of several discrete systems and sub-systems, and language learning is gaining mastery over the systems one by one. Testing, then, is employing objective tests, rvhich could be scored consistently. The global approach advocates for combining various language sub-skills and elements in testing because they are inter-related and interdependent, and for integrating the test items into a total language event. The communicative approach, on the other hand, views language as communication and language learning as developing communicative competence, which is essential for enabling learners to use language in the multiple functions it serves in the real life. In communicative testing, the best exams are those that combine various sub-skills as we do when exchanging ideas orally or in writing. From a communicative perspective, language testing in ESL and EFL may roughly be divided into pre-communicative and communicative stages. In the precommunicative language testing, the learners of a second/foreign language have to demonstrate their practical command of skills acquired. This tradition continued through structuralist, discrete-point framework of Lado (1961), pragmatic expectancy g.u-iu, of Oller (Ig7g), integrative test structure of Carroll (1972) and psycholinguistic performance model of Davies (1988). The communicative language iestittg .utr b. linked to the seminal work of Hymes (1972), but a detailed testing framework was only proposed in the much publicised work of Canale and Swain (i980)

which was later elaborated by Bachman (1990) and Bachman and Palmer (1996)' Almost two decades ago, in order for a language test to be eclectic, a synthesis of discrete-point approach and integrative procedures was viewed necessary. Twenty years later on, when the psycholinguistic-sociolinguistic trend is still in vogue and the tommunicative approactr to language testing is becoming increasingly popular, it is assumed that no single language testing model or test 'can accommodate the wide variety of test scenario'. The present research, therefore, is an attempt to justit that an adaptation or combination of models or tests in different language areas and different target situations should be made available in order to meet the local language testing reouirements. Journal ofNELTA

Vol. B, No.

l-2

December, 2003

Language Testing: Then and Now/65

In Nepal, for example, where ELT practitioners are still bound to the previous trend and are not yet fully aware of the ways and means of communicative language testing, a combination of all three aspects of the psycholinguistic-sociolinguistic trend and the communicative approach to language testing can justifiably serve the various purposes of the English test of the SLC. References Alderson J. Clapham, C. and Wall, D. (1995). Language Test Construction and Evaluation. Cambridge. Cambridge University Press. Alderson, J. C., Krahnke, K. J. and Stansfield, C. W. (19S7). Reviews of English Language Proficiency Zesrs. Washington, DC. TESOL. Allen, J. P. B. and Davies, A. (1977). Testing and Experimental Methods. Edinburgh Course in Applied Linguistics. Vol. 4. London: Oxford University Press. Asher, R.E. (1994xed.). The Encyclopaedia of Language and Linguistics. Oxford. Pergamon Press. Bachman, L. F., Davidson, F., Ryan, K. and Choi, I. (1995). Studies in Language Testing. Vol. l-5. Cambridge: Cambridge University Press. Bachman, L. and Palmer, A. (1996). Language testing in practice: Designing and Developing Useful Language Zesls. Oxford: Oxford University Press. Bachman, L. F. (1990). Fundamental Considerations in Language Testing. Oxford: Oxford University Press.

Banerjee, S. (2000). Integrated Testing. In Byram, M. (2000) Routledge Encyclopaedia of Language Teaching and Learning. London. Routledge. Bhadra, S. and Yadav, Y. (1988). Causes of Failures in the Proficiency Certi/icate Examinations. Kathmandu: CERID. Bloomfield, L. (1933). Language. New York. Holt, Rinehart and Winston Brown, J. D., Cook, H. G., Lockhart, C. and Ranes, T. (1991). 'Soutleast Asian Languages Proficiency Examinations ; ENC ED 365 | 52. Brown, J. D. and Yamashita, S. O. (1995).'Language Testing in Japan.' ERICED 4001.13. Cajkler, W. and Addleman, R. (2000). The Practice of Foreign Language Teaching. London. David Fulton. Canale, M. (1983). 'On Some Dimensions of Language Proficiency.'ln Oller, J. (ed.) (1983). Issues in Language Testing Researcft. Rowley: Newbury House. 102-l15.

Canale,

M. (1994). 'On Some Theoretical Framework for Language Proficiency.' In Rivera, C. (ed.)(1994). Language Proficiency and Academic Achievemerl. Clevedon, Avon: Multilingual Matters.

M. and

Swain, M. (1980). Theoretical Bases for Communicative Approaches to Second Language Teaching and Testing. Applied Linguisitcs. Nol. 1, No. l, Pp- l-47. Carroll, B. J. and Hall, P. J. (1985). Make Your Own Language Tests. New York: Pergamon Press. Carroll, J. B. (1983). 'Psychometric Theory and Language Testing.'ln Oller, J. (ed.) (19S3) Issues in L anguage Testing Res earch. Ptowley: Newbury House. 19 5-207 . Canale,

in English in the SLC Examination. Kathmandu: Tribhuvan University. Chomsky, N. (1957). Syntactic Structures. The Hague. Mouton Clapham, C. & Wall, D. (eds.) (1996). Language Testing Update.. England: Centre for Research in Language Education, Lancaster University. Cohen. A. (1994). Testing Language Ability in the Classroom.Rowley: Newbury House Publishers, Inc. Coombe, C. (1998). 'Current Trends in English Language Testing.' ERIC ED 428574. Corder, S.P. (1973). Introducing Applied Linguistics. London. Penguin. Crystal, D. (1997). The Cambridge Encyclopaedia of Language. Cambridge. Cambridge University Press. CERID, (1989). Causes of Failure

Journal ofNELTA

Vol.8, No. I-2

December, 2003

66 / Ram Ashish

Giri

Ministry Curriculum Development Centre (CDC). (lgg7). Secondary Education Curriculum. Kathmandu: of Education, HMG/N. Cziko, G. A. (1984). 'Improving the Psychometric, Criterion-referenced, and Practical Qualities in Integrative Language Tests.' TESOL Quarterly. 16' 3:367-79' Davey, L. dgg1). e Cise for a National Testing System. ENC Practical Assessment, Research and Evaluation. ISSN 1096 0066. Davey, L. and Monty, M. (1991). 'A Case Against

a

National Test.' ERIC Practical Assessment, Research

and Evaluatior, ISSN 1096 0066.

Davies, A.(1963) (ed.). Language Testing Symposium.

A

Psycholinguistic Approach. Oxford' Oxford

University Press.

Uncertainty in Language Testing: An Argument in Favour of Content Validity.' Language Testing. 5. 1:32-48. Davies, A. ( 1990). Principles of Language Testing. oxford. Black Basil. EzuC. (1995). 'The Cost of aNational Examination.' ERIC/AE Ed' 385611' TESOL Farhady, H. (1979).'The Disjunctive Fallacy between Discrete-point and Integrative Tests.' Quarterly. 16. 1:43-59 (1983). Issues in Farhady, H. (1983). 'New Directions for ESL Proficiency Testing.' In Oller, J. (ed.) 34-46' House' Newbury Rowley: Language Testing Research. ,A Text Driven Method for the Deletion Procedure in Cloze Passages.' ERIC EJ Farhady, H. if qgO). 529503. Brace' Fries, C. (1945). Teaching and Learning of English as a Foreign Language' New York' Harcourt Sentences. English of f'ries, C. (1952). The Stn.rcture of English. An Introduction to the Construction Davies,

A. (1983).'Operationalising

New York. Harcourt Brace. B. (1988). Ihriting English Language Tesls. London: ELBS' for Howard, F. (1980). testing-Comiunicative Ploficiency in French as a Second Language: A Search Pp. 272-280. 2, No. vol 36, Review Language Procedures. canadian Modern Hughes, A. (1986). Testingfor Language Teachers. Cambridge: Cambridge University Press.. Hyiies, D. (1972). on Communlcative Competence. In Pride and Holmes (eds.) Sociolinguistics. London. Penguin. Ingram, D.E. (1934). Using Proficiency Rating Scales with High School Foreign Language Learners' Heaton, J.

ERIC ED 249761, Ingram, D. E. (1985). 'Assessing Proficiency: An Overview of Some Aspects of Testing.'In Hyltenstam, K. and pienemann, V. (eOs) (1985). Modelling and Assessing Second Language Acquisition. Clevedon Avon: Multilingual Matters. 'An Intemational.Survey of Research in Language Testing'' ERIC Ed. 192579 Testing. London. Longman (1961). Language Lado, R. Validity of the Cloze Test with Essay Test Among Korean Students.' 'The (1996). Co-ncunent Lee, S. ERIC Ed. 4167 13. Leech, G. (1975). The communicative Grammar of English. London. Longman Co. Leedy, p. D. (1981). How to Read Research and (Jnderstand il. New York: MacMillan Publishing Inc. Madsen, H.s. (19s3). Techniques in Testing. oxford. oxford University Press McGarrell, H. (1931). Language Testing: A Historical Perspective. In Medium Vol' 6, No' 4 McNamara, T. (2000/. Language Testing. Oxford: Oxford University Press. McNamara, T. F. (1996). Measuring Second Language Performance.London: Longman' Jones, R. L. (1980).

Morrow, K. (1979). Communicative Language Testing: Revolution or evolution? In Brumfit, C' and Johnson, K. (eds.). The Communicative Approach to Language Teaching. Oxford. Oxford University Press. (1981). Communicative Syllabus Design. A Sociolinguistic Model for Defining the Content of J. Munby, - purpose-Specific Language Programmes. London. Cambridge University Press.

Journal ofNELTA

Vol. 8, No.

l-2

December, 2003

Language Testing: Then and Now/67

Oller, J. (1979). 'The Psychology

of

Language and Contrastive Linguistics: The Research and the

Debate'. ENC EJ206643. Oller, J. (1983). Issues in Language Testing Research. Rowley: Newbury House. Oller, J. (1919). Language Tests at School: A Pragmatic Apprctach. England: Longman. Oller, J. ( I 99 I ). 'Current Research/Development in Language Testing.' ERIC ED 365146. Oller, J. (1992). 'Foreign Language Testing; Part 2: Its Depth. ADFL Bulletin,22:5-13 Rea-Deakens, P. and Germaine, K. (1996). Evaluation. Oxford: Oxford Universify Press. Selinker, L. (1972) Interlanguage. IRAL Vol 10. Pp-209-231 Skinner, B. F. (1957) Verbal Behaviour. New York: Appleton, Century, Crofts.

B. (1969).'Introduction: Linguists and Language Testers.'In Spolsky, B. (ed.) (1969). Approaches to language Testing. IAashington, D. C. Center of Applied Linguistics. l-19. Spolsky, B. (1975). Language Testing: Art or Science? Key Note Address at Association Internationale de Linguistique Appliquece World Congress, France. Spolsky, B. (1978)(ed.). Approached to Language Testing. Papers in Applied Linguistics. ERIC ED Spolsky,

16548 Spolsky, B. (1993 Testing of English of Foreign Students in 1930. EzuC 2002. Spols\y, B. (i995). Measured Words: The Development of Objective Language Testing. Oxford. Oxford

University Press. Van Ek, J. A. (1979). The Threshold Level. Council ofEurope. Transbourg, 1979 Weir, C.J.( 1988). The Specification, realisation, and validation of an English language proficiency test. In Hughes, A. (ed.) Testing English for University study. ELT Document 127. London. Modern English Publications/The British Council. Weir, G. (1990). Communicative Language Testing. New York; Prentice Hall. White, E.M. (1994). Teaching and Assessing Writing. San Francisco: Jossey - Bass Publishers. Wilde, J. and Sockey, S. (1995). Evaluation Handbook. Albuquerque: Evaluation Assistance Centre, New Mexico High Land University. Wilkins, D. (1976). Notional Syllabus. A Taxonomy and its relevance to Foreign Language Curiculum Development. London. Oxford University Press.

Contact: [email protected] (Currently Mr. Giri is a Doctoral student at Victoria Universtiy, Australia)

Journal ofNELTA

Vol. B, No.

l-2

December. 2003