Errors and non-errors in English-Arabic machine ...

Available online at www.sciencedirect.com

ScienceDirect Procedia Computer Science 00 (2017) 000–000 www.elsevier.com/locate/procedia

3rd International Conference on Arabic Computational Linguistics, ACLing 2017, 5-6 November 2017, Dubai, United Arab Emirates

Errors and non-errors in English-Arabic machine translation of gender-bound constructs in technical texts Emad A. S. Abu-Ayyash* The British University in Dubai, DIAC, Dubai, United Arab Emirates

Abstract This paper has as its main goal investigating the errors and non-errors made by three machine translation (MT) systems in translating gender-bound constructs from English to Arabic in four selected technical texts. The three MT systems used in this study were Systran’s Pure Neural Machine Translation (PNMT), Google Translate (GT) and Microsoft Bing (MB). The target gender-bound constructs were subject-verb agreement, adjectival-noun agreement and pronoun-antecedent agreement, which occurred naturally in four purposefully selected technical texts. The idea behind the choice of technical texts was to reduce the linguistic load found in other genres, such as literary texts, which involve utilising creative linguistic tools that are out of the scope of the present paper. Upon the qualitative examination of the target language texts, the findings revealed that the three MT systems had errors and non-errors in rendering gender-bound constructs from English to Arabic, and that errors transpired in certain co-textual environments. © 2017 The Authors. Published by Elsevier B.V. Peer-review under responsibility of the scientific committee of the 3rd International Conference on Arabic Computational Linguistics. Keywords: Machine translation; gender; gender-bound cnstructs; English-Arabic translation

1. Introduction Attitudes towards machine translation (MT) have been inconsistent and continuously in flux since the 1950s. In essence, adoption of MT as a valid option for translation was tied to the purposes that the field served and the effectiveness of MT in meeting the requirements of different organizational bodies. According to [1] and [2], MT * Corresponding author. Tel.: 00971-50-7225701; fax: 00971-4-2791471. E-mail address: [email protected] 1877-0509 © 2017 The Authors. Published by Elsevier B.V.

Peer-review under responsibility of the scientific committee of the 3rd International Conference on Arabic Computational Linguistics.

Author name / Procedia Computer Science 00 (2017) 000–000

2

loosely passed through a number of stages of acceptance and adoption. When MT was first introduced as a translation option in the 1950s, it was used for military and intelligence purposes, particularly translating the information gathered from and about certain regimes. However, about ten years later, there was an evaluation stage, and MT was criticised heavily in a report written by the National Academy of Sciences ALPAC. In the 1970s, the field of MT lost more battles as MT projects lost government funding. The spiralling rise of MT began in the following decade, and there was an exponentially increasing reliance on MT systems. According to [1], “in 1984, approximately half a million pages of text were translated by machine” (p. 1). Nowadays, with the intricate web of technology, the internet and businesses that need fast and high-quality translations, MT is gaining more and more momentum, but to what effect? In a recent evaluation of neural machine translation, it is reported that fluency has increased, but that the status of adequacy is not yet definitive [16] and that post-editing of MT has become a reality [17]. The issues about adequacy in MT systems are a corollary of the vast differences between language systems at the levels of morphology and syntactic structures. The focus of this study is English and Arabic, which are acknowledged to be substantially different in a number of ways. For example, while the English declarative statement starts with a pronoun or a noun, the Arabic sentence, which, like English, can start with a noun or a pronoun, can also start with a full verb [8]. The pronoun system, which will be discussed further in the present study, is another area of disparity between the two languages. The present study has as its goal investigating to what extent MT systems are able to render texts from English to Arabic when such texts include forms that have different rules, or grammatical representations, in the two languages. The grammatical constructs that have been selected to serve the purpose of this study were all related to gender, which is acknowledged to be an area of dissimilarity between English and Arabic as far as form is concerned [3]. Therefore, my intention in the current paper was to investigate the types of errors and non-errors perpetrated by MT systems apropos the rendition of gender-bound grammatical constructs from English to Arabic. The target forms have been fed into three MT systems within their natural occurrences, or co-texts, and not as individual, discrete units. That is to say, the selected texts where inserted into the three MT systems as are, yet the analysis focused on the rendition of the gender-bound forms within the produced translations. The transliteration system used in this paper is Qalam [15]. 2. Related work With the exponential rise in the demand for accurate translations, MT is again singled out for critical attention and scrutiny across translation research. Concomitant with MT research, there were attempts to identify the levels of linguistic description pertinent to MT. Levels that were under investigation included phonetics and phonology, morphology, word classes and grammatical categories, syntax, lexicon and semantics, and pragmatics and stylistics [4]. However, a shortcoming of MT systems is that they are ‘limited by their relative ignorance of linguistic information’ [5] (p. 953) despite the various attempts to involve programs that can identify word structure and sentence structure [6, 16]. A number of studies have investigated this claim with various foci on MT capabilities with different linguistic levels. At the semantic and syntactic levels of language, [7] examined the notion of ambiguity and how it can be problematic for MT systems. The study identified five divergent areas between English and Arabic at the semantic level as far as ambiguity is concerned. These include category, homograph, transfer, pronoun reference and gender and number. Within the structural divergence between the two languages, [7] identified the areas of word order, tense and aspect, and agreement. These findings dovetail with the de facto assertions that English and Arabic each houses its own distinct linguistic structures and discursive features [8, 9]. At the word classes linguistic level, word ordering and agreement in three MT systems, which were Google, Tarjim and Systran, were investigated [10]. The study investigated the English-Arabic translations of certain structures as per gender, number, case and definiteness. The findings were reported in terms of the errors perpetrated by the MT systems in rendering such structures. The target structures were fed as discrete units into the MT systems, while in reality, these systems are normally fed with longer stretches of texts where the target structures are normally embedded. In fact, the practice of investigating MT systems’ effectiveness using separate sentences as discrete input units has even become a tool of evaluating MT systems [11].

Author name / Procedia Computer Science 00 (2017) 000–000

3

A similar investigation was conducted by [12] on pronouns as referential elements. These have been considered as problematic in English-Arabic translation because of disparities between the two languages in the systems of number and gender [12]. They state that “who” in English can be equivalent to up to nine Arabic referential items based on number and gender. In the three examples below, “who” should be rendered in three different ways in Arabic:   

This is the man who found the answer. This is the lady who found the answer. These are the three men who found the answer.

In the first example, “who” should be rendered into ‫ الذي‬/alladhii/, singular male-referring who, in the second into ‫التي‬ /allatii/, singular female-referring who, and in the third into ‫ الذين‬/alladhiina/, plural male-referring who. Any error in translating referential items will affect the smooth flow of cohesion and the entire meaning. It can be clearly noticed that MT research suffers from dearth at the linguistic level (e.g. syntax and morphology), particularly at the empirical side, which might explain the status of ignorance of MT systems at the linguistic level [5], and which, at the same time, calls for more investigations that could ultimately lead for better translation performance in terms of producing more natural target language texts (TLTs). 3. Method To conduct the present study four technical and scientific English texts were selected to be the source language texts (SLT). The reason behind the choice of such text types was to minimise the language factors that might come into play with other text types, such as metaphors, similes and literary stylistics, only to name some, and which are still unclear and troubled waters for MT systems [13]. The choice of technical texts has made the gender-bound constructs more accessible and easier to analyse. The texts selected for the present study can be found in Appendix A, and Table 1 outlines the general topics of the four texts and the total number of words for each. Table 1. Themes of the selected texts. Source Language Texts

Theme

Total number of words

SLT-1

Speed and velocity

101

SLT-2

Internet security and accelerator service

101

SLT-3

LCD TVs

97

SLT-4

Laser cooling experiments

135

The MT systems that were selected were Systran’s Pure Neural MT (PNMT), Google Translate (GT) and Microsoft Bing (MB). All four SLTs were fed into the three systems on June 5th, 2017 and translated from English to Arabic. The target language texts (TLT) were then qualitatively analysed and juxtaposed in the area of gender-bound constructs. These constructs were further classified into subcategories based on the forms in which they can exist in the target language, which is Arabic. The subcategories of gender-bound constructs include 1) subject-verb agreement, 2) adjectival-noun agreement and 3) pronoun-antecedent agreement. In all these three areas, MT errors and non-errors were analysed and explained. On a cautious note, since the analysis conducted was qualitative, not all errors and non-errors were put under the microscope since the purpose was to enrich the knowledge about the three target constructs in MT translation, rather than quantifying them. 4. Findings and discussion This part has been divided to three subsections based on the classification of gender-bound constructs in the target language into three categories: 1) subject-verb agreement, 2) adjectival-noun agreement and 3) pronoun-antecedent agreement.

Author name / Procedia Computer Science 00 (2017) 000–000

4

4.1. Subject-verb agreement Both English and Arabic house significant disparity in terms of gender-based, subject-verb agreement. While English is neutral to gender as far as subject-verb agreement is concerned, Arabic is not. In order to elucidate this point, consider example (1): (1)

English The boy sings. The girl sings.

Arabic ‫يُغنّي الولد‬ ‫تُغنّي البنت‬

Transliteration /yughannii alwaladu/ /tughannii albintu/

Obviously, in English it is grammatical to use the same form of the verb (sings) with a singular noun regardless of gender. In Arabic, the gender of the subject is a determiner of the verb form. This disparity between the two languages was a problematic area in 13 occurrences for the MT systems. For example, In SLT-1 (Speed and Velocity), the opening subject-verb structure “speed describes…” and the concluding sentence opening “velocity gives…” were mistranslated in four instances and rendered correctly in two instances in the three MT systems as shown in (2): (2)

SLT-1 Speed describes… Velocity gives…

PNMT ‫تصف السرعة‬ /taSifu-ssur`atu/ ‫السرعة يعطيك‬ /assur`atu yu`Tiik/

GT ‫يصف السرعة‬ /yaSifu-ssur`atu/ ‫السرعة تعطيك‬ /assur`atu tu`Tiik/

MB ‫سرعة يصف‬ /sur`ah yaSif/ ‫السرعة يعطيك‬ /assur`atu yu`Tiik/

While MB’s renditions were both inaccurate as far as gender recognition is concerned, GT was accurate in translating “Velocity gives” to ‫ السرعة تُعطيك‬/ assur`atu tu`Tiik/ thus recognising that velocity is feminine in Arabic, but failing to make similar realisation with the structure “Speed describes”, which was inaccurately rendered as ‫يصف‬ ‫ السرعة‬/yaSifu-ssur`atu/. Similarly, PNMT was accurate in the first instance, but erroneous in the second as shown in the caption presented in Figure 1.

Fig. 1. PNMT rendition of “Speed describes…” and “Velocity describes…”.

Considering the four texts, however, it is clear that gender-related, subject-verb agreement constructs were rendered mostly accurately in the three MT systems. The totals number of such constructs was 45 (13 in SLT-1, 8 in SLT-2, 10 in SLT-3, 14 in SLT-4), while the total number of errors perpetrated in rendering them was 13 (PNMT 2 errors, GT 4 errors, MB 7 errors). Nevertheless, the three MT systems were fully accurate in subject-verb agreement structures in which the main verb was a copula (i.e. is, are, be, etc.). In SLT-1, “A simple example would be…” was rendered correctly as far as gender is concerned by the three MT systems as ...‫ مثا ٌل بسيط هو‬/mithaalun basiiTun huwa/, realising that in Arabic, ‫( مثال‬example) is masculine, which was why all the three systems used ‫ هو‬/huwa/ , rather than ‫ هي‬/heya/, which is a

Author name / Procedia Computer Science 00 (2017) 000–000

5

feminine pronoun. The reason behind this high level of accuracy in Subject+Copula-agreement structures is that in Arabic there is no one-to-one equivalence of copulas because they do not exist. The MT systems, therefore, had to rely on the subject alone to determine the gender. In other words, the items to be considered in translating Subject+Copula structures were less complicated than those involved in Subject+Main Verb structures.

4.2. Adjectival-noun agreement Adjectival-noun agreement is another area of disparity between English and Arabic. While the former is neutral (e.g. smart boy, smart girl) since the adjective form is not bound to the gender of the head noun, the latter is not. In Arabic, the form of the adjective changes based on whether the head noun the adjective modifies is masculine or feminine. Therefore, smart boy should be rendered into Arabic as ‫ ولدٌ ذكي‬/waladun thakiyyun/, and smart girl as ٌ‫بنت‬ ٌ‫ ذكيّة‬/bintun thakiyyatun/, thus adding to the adjective the feminine-marker /at/. For all the three MT systems, the rendition of structures involving adjectival-noun agreement was dominantly characterised by non-errors, indicating that the three MT systems have accurately rendered such forms with high levels of accuracy. To elaborate, the total number of adjectival-noun constructs in the four SLTs was 42 (4 in SLT-1, 7 in SLT-2, 12 in SLT-3, 19 in SLT-4), while the total number of errors was only 4 (PNMT 0 errors, GT 3 errors, MB 1 error). Table 2 lists some examples from the four texts where the adjectival-noun structure has been rendered correctly from English to Arabic by the three MT systems. In the target structure boxes (column 2), MAS stands for masculine and FEM stands for feminine. Table 2. Examples of adjectival-noun constructs’ translations. SLT#

SLT structure

SLT-1

A simple example (MAS)

SLT-2

Local area network (FEM)

SLT-3

A new TV (MAS)

SLT-3

A great picture (FEM)

SLT-4

Large samples (FEM)

SLT-4

New Scheme

PNMT

GT

MB

‫مثال بسيط‬ /mithalun baSiiT/ ‫الشبكة المحليّة‬ /ashshabatah almaHalliyyah/ ‫تلفزيون جديد‬ /tilifizyuun jadiid/ ‫صورة رائعة‬ /Suurah raa’i`ah/ ‫العيّنات الكبيرة‬ /al`ayyinaat alkabiirah/ ّ ‫مخطط جديد‬ /mukhaTTaT jadiid/

‫مثال بسيط‬ /mithalun baSiiT/ ‫الشبكة المحليّة‬ /ashshabatah almaHalliyyah/ ‫تلفزيون جديد‬ /tilifizyuun jadiid/ ‫صورة رائعة‬ /Suurah raa’i`ah/ ‫عيّنات كبيرة‬ /`ayyinaat kabiirah/ ‫نظام جديد‬ /niZaam jadiid/

‫مثال بسيط‬ /mithalun baSiiT/ ‫شبكة االتصال المحليّة‬ /shabakatu alittiSal almaHalliyyah/ ‫التلفزيون الجديد‬ /attilifizyuun aljadiid/ ‫صورة رائعة‬ /Suurah raa’i`ah/ ‫العيّنات الكبيرة‬ /al`ayyinaat alkabiirah/ ‫خطة جديدة‬ /khuTTah jadiidah/

It can be noticed that in all the examples introduced in Table 2, the SLT structures have been translated accurately as the gender-based distinctions in the TLT have been realised. This is more evident when certain head nouns were rendered differently by the MT systems to be masculine in some instances and feminine in others. For example, the ّ noun scheme in SLT-4 was rendered by PNMT as ‫مخطط‬ /mukhaTTaT/ and by GT as ‫ نظام‬/niZaam/, and both lexical items are masculine in Arabic. Therefore, the adjective new which modifies the noun in the SLT was rendered as ‫جديد‬ ّ /jadiid/. On the other hand, MB translated the same noun as ‫خطة‬ /khuTTah/, which is a feminine expression in Arabic, and the rendition of the modifying adjective responded correctly with new being translated as ‫ جديدة‬/jadiidah/ instead of ‫ جديد‬/jadiid/. The limited errors that transpired in the translations of adjectival-noun agreement structures were noticed to occur within a co-text that is replete with feminine expressions as both head nouns and adjectives. The errors here occurred in two types of adjectival expressions, which are 1) a gerund modifying a head noun and 2) an adjective within a series of adjectives. To elaborate, as shown in Figure 2, the chunk “a laser beam coming..” (SLT-4) was erroneously rendered by GT as ‫ شعاع الليزر القادمة‬/shu`aa` alleezar alqaadimah/, thus modifying a masculine noun (shu`aa`) with a feminine adjective (alqaadimah).

Author name / Procedia Computer Science 00 (2017) 000–000

6

Fig. 2. GT rendition of “a laser beam coming…” in its co-text.

Figure 3 depicts a similar error perpetrated by MB with a construct from SLT-3 which involves an adjective that is situated within a series of adjectives in “a slim, stylish display…”, where the adjective “slim” was rendered as ‫أنيقة‬ /’aniiqah/ to modify the masculine noun ‫ عرض‬/`arD/ with a feminine adjective.

Fig. 3. MB rendition of “a slim, stylish display…” in its co-text.

Another structure involving adjectival-noun agreement is that involving relative clauses with which as the relative pronoun. In such structures, non-errors prevailed with the three MT systems rendering “which” accurately, realising that Arabic requires gender-based agreement between the relative pronoun and the noun it modifies. For example, in SLT-1, the constructs “the speed at which…” and “the direction in which…” were correctly translated to Arabic as ‫ السرعة التي‬/assur`ah allatii/ and ‫ االتجاه الذي‬/alittijaah allathii/ resepectively by all the three MT systems, which reflects the realisation that the word for speed in Arabic is feminine and the word for direction is masculine, a realisation that was accurately mirrored in translating the relative pronoun which as ‫( الذي‬for the masculine) and ‫( التي‬for the feminine). 4.3. Pronoun-antecedent agreement Probably this area witnessed the most remarkable inconsistency in the renditions of the three MT systems. The pronoun systems in English and Arabic are similar when it comes to the subject pronouns he, she, I, and we as these have one-to-one equivalence in Arabic as the respective renditions of ‫ هو‬/huwa/, ‫ هي‬/hiya/, ‫ أنا‬/’anaa/ and ‫نحن‬ /naHnu/. The remaining three subject pronouns, which are you, they and it can be rendered in different ways to Arabic based on gender and singularity, plurality or even duality [14] as shown in Table 3. It can be clearly noticed from the illustration in Table 3 that it has two equivalences in Arabic based on gender, you five equivalences based on gender and number and they three equivalences based on gender and number. Table 3. Subject pronouns in English and Arabic.

English subject pronoun It

You

Corresponding Arabic pronoun(s) ‫ هو‬/huwa/

Meaning of the Arabic pronoun

‫ هي‬/heya/

Third person singular masculine (things) Third person singular feminine (things)

‫ أنت‬/’anta/

Second person singular masculine

‫ أنت‬/’anti/

Second person singular feminine

‫ أنتما‬/’antuma/ ‫ أنتم‬/’antum/

Second person dual masculine and feminine Second person plural masculine

Author name / Procedia Computer Science 00 (2017) 000–000

They

7

‫ أنتن‬/’antunna/

Second person plural feminine

‫ هم‬/hum/

Third person plural masculine

‫ هن‬/hunna/

Third person plural feminine

‫ هما‬/humaa/

Third person dual masculine and feminine

This huge discrepancy was reflected in the form of inconsistencies in the translation of structures that involve the agreement between such pronouns and their antecedents. The total number of pronoun-antecedent constructs in the four SLTs is eight (3 in SLT-1, 0 in SLT-2, 1 in SLT-3, 4 in SLT-4), and the total number of errors in the rendition of these constructs was nine (PNMT 2 errors, GT 4 errors, MB 3 errors). Table 4 shows how the three MT systems translated similar structures. Table 4. Translations of pronoun-antecedent structures. SLT#

SLT structure

PNMT

GT

MB

SLT-1

Velocity…it involves

SLT-4

atoms…they encountered

‫انها تنطوي‬...‫الوتيرة‬ /alwatiirah…’innaha tanTawii/ ‫واجهوا‬...‫الذرات‬ /aththarraat…waajahuu/

‫انه ينطوي‬...‫السرعة‬ /assur`ah…innahu yanTawii/ ‫واجهت‬...‫الذرات‬ /aththarraat…waajahat/

‫انه ينطوي‬...‫السرعة‬ /assur`ah…innahu yanTawii/ ‫واجهت‬...‫الذرات‬ /aththarraat…waajahat/

What is conspicuous about the renditions provided in Table 4 is that PNMT was accurate in rendering the singular pronoun it into its feminine Arabic equivalence, while GT and MB were not. However, the situation was reversed with the plural pronoun they as GT and MB were accurate in realising the feminine antecedent of the pronoun while PNMT was not. 5. Conclusion Gender-based structures in English and Arabic are an expected problematic area for MT systems due to the differences between the two languages. This paper investigated the English-Arabic errors and non-errors in the translations of three gender-related structures, which were subject-verb agreement, adjectival-noun agreement and pronoun-antecedent agreement, by three MT systems: Systran’s Pure Neural Machine Translation (PNMT), Google translate (GT) and Microsoft Bing (MB). The three structures were embedded within four purposefully selected technical texts that were fed into the three MT systems. Regarding subject-verb agreement, the three systems were mostly accurate in their TLT renditions when the main verb was a full verb, and fully accurate with copulas. As for the adjectival-noun agreement structures, the three systems produced natural Arabic TLTs, realising Arabic sensitivity to these structures. However, certain environments were evidently problematic, namely when a gerund modified a head noun and when an adjective was used in a series of adjectives modifying the same noun. The most noticed inconsistency, though, was noticed in pronoun-antecedent agreement structures probably because of the huge discrepancy between English and Arabic in the pronoun system itself. Statistically, PNMT’s performance was better than the other two. In 94 occurrences of the three constructs, PNMT made 4 errors, GT 11 and MB 11. While these results cannot be generalised due to the smallness of the sample, the present analysis can form the ground of a broader quantitative investigation that detects the errors and non-errors in translating gender-bound constructs across hundreds of texts utilising the findings of the present analysis.

Author name / Procedia Computer Science 00 (2017) 000–000

8

Appendix A. Source language texts SLT-1: Speed describes how fast something is moving. A simple example would be to look at your car’s speedometer while you are driving. This tells you the speed at which you are traveling. It does not tell you the direction in which you are traveling. Velocity is very similar to speed except that it involves a direction as well as speed. To determine the velocity of an object, you would need to know the object’s speed and direction. To measure velocity in a car, you would need a speedometer and a compass. In essence, velocity gives you more information about an object. SLT-2: ISA (Internet Security and Accelerator) Server is built on Windows technology for advanced security performance and management. Network Address Translation (NAT) is one of Windows technologies that work with ISA server to provide better security performance and management capabilities. ISA server extends Windows NAT functionality by enforcing ISA Server policy for Secure NAT clients. NAT is built into Windows and provides a gateway that can hide the Internet Protocol (IP) addresses of internal local area network (LAN) clients from external clients. This is achieved by masking the internal addresses with a different set of addresses that is visible to the outside. SLT-3: LCD TVs deliver a great picture and offer amazing features, all from a slim, stylish display. Whether you’re looking to make a statement or simply introduce a new TV into a small room, LCDs are definitely worth checking out. LCD packs a big bunch in a small package. These slender displays are fairly lightweight and easy to mount on the wall, which also means placement flexibility. Many LCD TVs also feature a PC or laptop with ease. If your TV also has a picture-in-picture function, you don’t have to miss your favorite shows to do it, either. SLT-4: In early laser-cooling experiments inside atom traps, atoms were cooled when they encountered a laser beam coming at them with an energy that was slightly less than what is needed to be absorbed by an unmoving atom. But absorption can occur anyway if the atom’s energy of motion equals the energy by which the laser beam is detuned. Thus, the detuning compensates for the Doppler effect of the moving atom. Now a new cooling scheme is announced, one in which the laser light is not absorbed but scattered. And scattered coherently in such a way that the atom loses a bit of energy in the encounter. The coherence helps to cool large samples because the scattered light circulates in an optical cavity and the scattering from one atom promotes scattering of light from other atoms.

References [1] Slocum, Jonathan. (1985) “A survey of machine translation: its history, current status, and future prospects.” Computational Linguistics 11 (1): 1–17. [2] Wilks, Yorick. (2009) Machine translation: its scope and limits. New York: Springer. [3] Dawood, Mohammed A, and Mohammed, J. Mohammed. (2008) “The ambiguity of gender in English-Arabic translation.” Adab Al-Rafidayn 52(1): 1-20. [4] Trujillo, Arturo. (1999) Translation engines: techniques for machine translation. London: Springer. [5] Doherty, Stephen. (2016) “The impact of translation technologies on the process and product of translation.” International Journal of Communication 10(1): 947-969. [6] Hutchins, John. (1999) “The development and use of machine translation systems and computer-based translation tools.” International Symposium on Machine Translation and Computer Language Information Processing 26-28 June 1999. [7] Akeel, Marwan, and R. B. Mishra. (2013) “Divergence and ambiguity control in an English to Arabic machine translation.” International Journal of Engineering Research and Applications 3(6): 1670-1679. [8] Bahloul, Maher. (2008). Structure and function of the Arabic verb. London: Routledge. [9] Holes, Clive. (2004). Modern Arabic structures, functions, and varieties. Washington, D.C.: Georgetown University Press. [10] Sembok, Tengku Mohd, and Mohammed Abu Shgier. (2008) “Word agreement and ordering in English-Arabic machine translation.” IEEE Xplore Conference: 1-8. [11] Al-Kabi, Mohammed N., Taghreed M. Hailat, Emad M. Al-Shawakfa, and Izzat M. Alsmadi. (2013) “Evaluating English to Arabic machine translation using BLEU.” International Journal of Advanced Computer Science and Applications 4(1): 66-73. [12] Shunnaq, Abdullah, and Mohammed Saraireh. (1998) Issues in translation. Irbid: Irbid National University & Jordanian Translators’ Association. [13] Hutchins, John. (2001). “Machine translation and human translation: in competition or in complementation?” International Journal of Translation 13(1): 5-20. [14] Wightwick, Jane. and Mahmoud Gaafar. (2005) Easy Arabic grammar. New York: McGraw-Hill. [15] Lawson, David R. (2013) “An assessment of Arabic transliteration systems.” Technical Services Quarterly 27(2): 164-177. [16] Castilho, Sheila, Joss Moorkens, Federico Gaspari, Iacer Calixto, John Tensley, and Andy Way. (2017) “Is neural machine translation the new state of the art?” The Prague Bulletin of Mathematical Linguistics 108(1): 109-120. [17] Escartin, Carla Parra, Hanna Bechara, and Constantin Orasan. (2017) “Questing for quality estimation: A user study.” The Prague Bulletin of Mathematical Linguistics 108(1): 343-354.