Principles for the Establishment of a Quality

Principles for the Establishment of a Quality Measurement Framework for Federal Government Translations

March 2018 Gatineau, Quebec (Translated from French by Gail Grant, Translation Bureau) Daniel J. Caron, Ph.D. Research Associate Language Technologies Research Centre (LTRC) Professor and Fellow, CIRANO (Centre for Interuniversity Research and Analysis on Organizations) Research Associate, CEFRIO (Centre facilitating research and innovation in organizations) Adjunct Professor, SPPA, Carleton University Chair Holder, Chaire de recherche en exploitation des ressources informationnelles (research chair in information resources development) École nationale d'administration publique

Executive Summary ................................................................................................................... 3 1.

Introduction ....................................................................................................................... 4

2.

Background ........................................................................................................................ 5

3.

Approach and methodology............................................................................................... 5

4.

3.1.

Treasury Board of Canada Evaluation Requirements.................................................. 5

3.2.

Theory-based approach to evaluation and rapid impact approach to evaluation ...... 6

3.3.

Research methodology ............................................................................................... 6

Context and modelling ....................................................................................................... 8 4.1.

Context and external factors ....................................................................................... 8

4.1.1.

Legal-administrative framework .......................................................................... 8

4.1.2.

Definition of quality ............................................................................................11

4.2.

Role of the Translation Bureau ...................................................................................13

4.2.1. 4.3.

5.

6.

Description of Translation Bureau activities ........................................................13

Discussion of the theory of change, its hypotheses and risks ......................................15

4.3.1.

Hypotheses ........................................................................................................15

4.3.2.

Risks ...................................................................................................................17

4.3.3.

Results chain and intervention modelling ...........................................................18

Principles and quality measurement .................................................................................20 5.1.

General observations .................................................................................................21

5.2.

Proposals ...................................................................................................................22

Conclusion.........................................................................................................................23

Selective Bibliography ..............................................................................................................24 Appendix 1................................................................................................................................30 Appendix 2................................................................................................................................31 Appendix 3................................................................................................................................32 Appendix 4................................................................................................................................33

2

Executive Summary Translation at the federal government meets the historical requirements that are rooted in the country’s institutional functioning. In the Canadian context, the quality of translations is therefore just as important as the texts in their source language. The thinking behind the development of this quality measurement framework is to anchor translation to its constitutional, legislative, administrative and political foundations. Thus, the discussion on quality highlighted the key elements to consider in order to measure it. Beyond its technical dimension, several factors contribute to the quality of a translation, such as the context in which it will be used and also the importance of it being understood by users. The technical dimension itself was seen as difficult to measure for a number of reasons, including the source text, for example. The proposal focuses on five elements related to measuring the quality of a text, which are: editorial correctness, content accuracy, aesthetics of a text, reflection of the socio-cultural context, and expected uses. Once the quality of the text has been built around its constituent elements, the framework proposes a theory of change to produce a quality translation. This theory of change is based on translation activities, outputs (e.g., translated texts) and repercussions on the functioning of Canadian institutions. To produce a quality translation, seven hypotheses have been identified ranging from the guarantee of the availability of sufficient translation capacity on the market to the systematic measurement of quality. The main hypotheses are the existence of sharedresponsibility business model, quality standards and service standards. Without these three elements, it is difficult, even impossible, to ensure quality translations. Lastly, the results chain outlined with its hypotheses made it possible to propose seven guiding principles for measurement purposes and four quality measurement tools.

3

1. Introduction This report presents the bases for establishing quality measurement for federal government translations. The report is part of an evaluative process and is in line with the Treasury Board Secretariat of Canada’s evaluation requirements according to the July 2016 Policy on Results. The issue of quality and its measurement is based on a number of factors that are both technical and legal-administrative. It is necessary to describe the environment that defines the operational framework of translation to understand the many factors that can influence its quality and measurement. As it is an issue that is not only technical, quality measurement must encompass all the elements used to define it. The purpose of the framework is to answer various questions, such as: understanding what is the nature of translation within the federal government and what is the business process to achieve it; identifying who are the key players and stakeholders in the translation file; presenting a theory that explains how existing activities make it possible to achieve the expected results (theory of change) and validating this information to establish indicators and a thorough evaluation methodology. This report has two specific objectives: on one hand, it will be used to establish the shared fundamental principles for the possible development of a measurement strategy that will facilitate the evaluation of the Translation Bureau’s linguistic services, and on the other, this will ensure that the Translation Bureau supports the Government of Canada in its efforts to provide Canadians with services and to communicate with them in their language of choice. To clearly situate the issue, the report first explains the context of the study and the approach and methodology chosen to develop the framework. Then, the report presents the framework chosen as validated by discussions in accordance with the theory-based approach to evaluation. Lastly, the report presents a model of the expected chain of results, the underlying hypotheses and the risks associated with the achievement of the expected results.

4

2. Background The Translation Bureau has been working since 2016 to structure its efforts to improve the quality of its linguistic services. Following the report and recommendations of the House of Commons’ Standing Committee on Official Languages tabled in February 2016, the government proposed some responses in October 2016 and February 2017. To support the government in its commitments related to its proposed improvement approach, the Translation Bureau wishes to develop an evaluation framework to assess the quality of its linguistic services. This framework must be created in compliance with the Government of Canada’s Policy on Results that went into effect on July 1, 20161.

3. Approach and methodology The approach chosen to establish the framework reflects the Treasury Board’s usual evaluation requirements. After listing these requirements, this section describes the evaluation approach chosen and the research methodology used to design the framework.

3.1.

Treasury Board Secretariat of Canada Evaluation Requirements

The Treasury Board Secretariat’s evaluation requirements are presented in the July 2016 policy entitled Policy on Results.2 In the Government of Canada, evaluation is “the systematic and neutral collection and analysis of evidence to judge merit, worth or value. Evaluation informs decision making, improvements, innovation and accountability. Evaluations typically focus on programs, policies and priorities and examine questions related to relevance, effectiveness and efficiency.” For the purposes of this report, we used the Policy on Results and the document entitled Theory-Based Approaches to Evaluation: Concepts and Practices3 and Guide to Rapid Impact Evaluation.4

1

https://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=31300§ion=html https://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=31300§ion=html 3 https://www.canada.ca/en/treasury-board-secretariat/services/audit-evaluation/centreexcellence-evaluation/theory-based-approaches-evaluation-concepts-practices.html 4 https://www.canada.ca/en/treasury-board-secretariat/services/audit-evaluation/centreexcellence-evaluation/guide-rapid-impact-evaluation.html 2

5

3.2.

Theory-based approach to evaluation and rapid impact approach to evaluation

For the purposes of this evaluation, the approach used is primarily based on a theory-based approach to evaluation. This approach was used to establish and validate hypotheses concerning the Translation Bureau’s activities in their overall context of supporting the Canadian official languages program. The Translation Bureau’s activities have a direct impact on the Canadian government’s capacity to fulfill its legal and administrative requirements and obligations regarding the production of documents in Canada’s two official languages through translations services and the operations of institutions in both legal languages through interpretation services. Moreover, the rapid impact evaluation was used to propose a more technical measurement of the quality of translation according to its source, as well as the contribution of translation in building the federal government’s expertise in the matter of longer term terminology.

3.3.

Research methodology

The methodology used to develop the framework is based on four sources of evidence. The rationale for this choice is to ensure that all players and stakeholders have a common understanding of the theory of change that will be evaluated. The methodology reflects the recommendations of the Treasury Board Secretariat’s Centre of Excellence for Evaluation, which suggests several sources of evidence, including the review of key intervention documents (eg, Memoranda to Cabinet, Intervention Terms and Conditions, planning documents, etc), relevant literature reviews (eg, prior evaluations, social science research, etc), discussions with intervention managers, beneficiaries and subject-matter experts. 5 The choice will be made on the basis of the issue and the context. •

State of knowledge

The quality of linguistic services is not a concept whose definition is unique and whose criteria are established and shared by everyone in the same way. There are a number of interpretations, depending on perspectives. The latter sometimes emerge from the professional environment, individuals who use linguistic services, persons responsible for public policies, and accreditation or regulatory organizations. These various perspectives are a key element in understanding “what” and “how” to measure. This review made it possible to identify and categorize the various definitions and to have a common understanding of what quality is from different perspectives.

5

https://www.canada.ca/en/treasury-board-secretariat/services/audit-evaluation/centreexcellence-evaluation/theory-based-approaches-evaluation-concepts-practices.html#toc6l

6

•

Review of background documentation

The second aspect that is essential to building an evaluation framework is the ability to situate the intervention, the offer of linguistic services in this case, in the administrative-institutional environment to which the intervention contributes. This makes it possible to understand the rationale and objectives of the intervention at all levels. To do so, a review was undertaken of government documentation on the Translation Bureau, its creation and, more generally, the objectives of and government needs for linguistic services. It also included elements on the legal interpretation of Canadian bilingualism. As the evaluation framework uses a theory-based approach, this state of knowledge made it possible to formulate some hypotheses and define expected outcomes. •

Interviews with translation industry stakeholders

The third component reflects another Treasury Board Secretariat recommendation in order to enhance our understanding of what quality means and what should be measured on the basis of the identified hypotheses and risks. A series of some 15 semi-structured interviews was completed in order to answer two questions. First, the interviews focused on the definition of quality per se. Second, the interviews addressed the issues involved in maintaining this quality. The objective was to generate themes directly related to the performance of linguistic services, including an initial validation of the hypotheses and risks related to the theory of change. •

Discussion groups

Lastly, a fourth source of evidence consisted in leading two discussion groups around the same issues as those covered in the interviews. The first discussion group was made up of Translation Bureau clients, while the second group was made up of experienced language professionals.

7

4. Context and modelling This section provides context, including program theory, the context in which translation is done within the federal government, the theory of change with its hypotheses and risks, as well intervention modelling.

4.1.

Context and external factors

As the many Parliamentary Committees and multiple publications on the subject have addressed it over the years,6 the quality of translation issue goes beyond the simple aspect of its technical dimension. Designing a framework to evaluate the quality of a translation therefore requires taking into consideration seriously and as thoroughly as possible the environment and context in which translation activities take place.

4.1.1. Legal-administrative framework To this end, the Treasury Board Secretariat states that “Theory-based approaches, more than many other evaluation approaches, pay explicit attention to the context of the intervention. It is acknowledged that contextual factors can help an intervention achieve its objectives or act against the intervention working (...). These factors are often essential in making causal inferences and need to be part of the evaluation design.”7 In the case of the Translation Bureau and its concerns for measuring the quality of its translation activities, the contextual factors are particularly important. We shall consider four of these.

6

In particular, refer to Parliamentary Debates that address the issue in considerable depth. For example, https://www.ourcommons.ca/DocumentViewer/en/42-1/LANG/report-2. 7 https://www.canada.ca/en/treasury-board-secretariat/services/audit-evaluation/centreexcellence-evaluation/theory-based-approaches-evaluation-concepts-practices.html. 8

The Canadian Charter of Rights and Freedoms states that English and French are the official languages of Canada with equal status. Also, the Official Languages Act establishes bilingualism as a practice in federal institutions and in their relationships with the Canadian public. A number of directives, programs, and/or activities resulting from this are led by various players, including the Treasury Board Secretariat, the Office of the Commissioner of Official Languages, Canadian Heritage, Public Services and Procurement Canada, and all federal departments and agencies. Since everyone can work in their language of choice, the production of documents and communications requires translation to ensure respect for the spirit and the lettter of the legal framework for linguistic obligations.

Various hypotheses depending on activities and their repercussions over time

First, as Graphic 1 shows, translation in Canada is rooted in the specific constitutional and legal requirements of official languages. This is the program logic which is much broader and based on the long term. The issue is thereby directly anchored in the Canadian Charter of Rights and Freedoms of the 1982 Graphic 1 Constitution Act. The Long-Term Program Results Chain Charter states: “English Logic and French are the official languages of Canada and Functioning of Canadian society in terms of official languages in compliance with its established legal framework made possible have equality of status and equal rights and Citizens’ access to Government of Canada programs and services in their language of choice at the expected level of quality privileges as to their use in all institutions of the Increased capacity of federal institutions to function internally in both official languages and with Canadian citizens, and to fulfill their Parliament and official languages obligations Government of 8&9 Canada.” This has Various medium-term repercussions depending on outputs generated obligations in terms of the language of work and the production of documents within Various immediate impacts depending on outputs federal institutions. Parliamentary work and Various outputs resulting from activities documents, proceedings depending on the players and their responsibilities before the courts and communications between the public and federal institutions are directly Various departments, Office of the Treasury Board Translation Canadian agencies and Commissioner of Secretariat Bureau Heritage institutions Official Languages affected by the Charter regarding the use of the two official languages. Key players responsible for activities of the Canadian government in matters of official languages These obligations were set out in various ways long before 1982, including in the 1867 Act.10 However, prior to the 1982 Charter, the 1969 Official Languages Act already provided for fairly specific elements of bilingualism, but they had no supralegislative status. In 1988, the Official Languages Act clarified some obligations. For example, it requires [translation] “simultaneous interpretation of oral interventions and the translation of documents…. This Act also provides for the right to be understood by the judge without the help of an interpreter before the courts other than the Supreme Court. And its Part IV, as well as the regulatory authority that it provides make it possible to extend bilingual federal services beyond what section of the 1982 Act requires.”11 With respect to the subject of this evaluation framework, 8

Canadian Charter of Rights and Freedoms, http://laws-lois.justice.gc.ca/eng/Const/page-15.html Brun, H., Tremblay, G., and Brouillet, E., Droit constitutionnel, 6th edition, Éditions Yvon Blais, 2014. The authors discuss the legal requirements of official languages, but also the official language practices of various legislatures that do not reflect complete bilingualism. 10 Idem, p. 856. 11 Idem, p. 881 and the Official Languages Act, http://laws-lois.justice.gc.ca/eng/acts/O-3.01/FullText.html 9

9

particular attention should be paid to Part V of the 1988 Official Languages Act which sets out the obligations related to communications and documentation within the federal government apparatus. These legal obligations provide the basic context for understanding the importance of the role and responsibilities of federal institutions with respect to official languages. Second, many players are involved in the implementation of these obligations, and the Official Languages Act sets out the role of some of them. Within federal institutions, parts VIII and IX are of particular interest. The Act sets a role for the employer, the Treasury Board Secretariat, with respect to official languages. As such, the Act reads as follows: “The Treasury Board has responsibility for the general direction and coordination of the policies and program of the Government of Canada relating to the implementation of Parts IV, V and VI in all federal institutions” with some exceptions.12 These obligations and attributions provide the Treasury Board Secretariat with the authority to establish application principles, propose regulations, provide instructions, and monitor and evaluate the effectiveness of various measures implemented in the government to meet the requirements. There is therefore an expected consistency between the 1982 Act, the Official Languages Act, and the Regulations and administrative policies of federal institutions in matters of official languages. This consistency is also desired for the other aspects of the Act that have a greater impact on citizens and communities throughout Canadian Heritage’s mandate: “the advancement of the equality status and use of English and French and the enhancement and development of the English and French linguistic minority communities in Canada.”13 The third player that plays a prominent role is the Commissioner of Official Languages of Canada whose responsibilities can be summarized as follows: “to take all actions and measures within the authority of the Commissioner with a view to ensuring recognition of the status of each of the official languages and compliance with the spirit and intent of this Act in the administration of the affairs of federal institutions, including any of their activities relating to the advancement of English and French in Canadian society.”14 Lastly, it must be kept in mind that the courts also have a role to play with regard to the enforceable aspects of the laws in question. Third, as will be covered in the following section, the Translation Bureau does not have a “monopoly” over the federal government’s translation activities. It therefore cannot control the quality of translations, except for those it produces itself. The multiple players involved in translation issues can have a direct or indirect impact on translation quality, depending on the mechanisms chosen or that prevail in the way of providing translation and/or interpretation. Fourth, and consequently, it must be kept in mind that the constitutional dimension of the language question makes it a legal issue at the mercy of judicial or quasi-judicial decisions that may influence the expected outcomes.

12

Official Languages Act, subsection 46(1). These exceptions include the “the Senate, House of Commons, Library of Parliament, office of the Senate Ethics Officer, office of the Conflict of Interest and Ethics Commissioner, Parliamentary Protective Service and office of the Parliamentary Budget Officer.” 13 Department of Canadian Heritage Act, S.C. 1995, c. 11, Powers, Duties and Functions of the Minister, 4 (2)(g). 14 Official Languages Act, Part IX, subsection 56(1). 10

In conclusion, it should be noted that the above-mentioned elements indicate that there is an overall consistency in the relationship between linguistic issues from the Charter to the requirements for concrete practices in the functioning of the government and its relationship with citizens. This consistency in terms of instrumentation therefore compels the government to ensure that it has the means available to meet its objectives. Although translation is not an end in itself, it nevertheless remains an essential means to support the fulfillment of linguistic obligations and commitments. It is also already understood that translation needs will vary and that, depending on their nature—legal, programmatic, communicational or even for the purpose of a simple personal understanding of an internal message— will have an influence on needs in terms of quality such as accuracy of meaning, for example.

4.1.2. Definition of quality Quality is an attribute, and it is not intrinsic to the mere existence of a translated text. To measure quality, a translation must be viewed in the context of a process that flows from the source text to the translated text. The definitions of quality vary and have an impact on many dimensions of the final product. That said, it is essential to construct the results chain around the translation, its production mode, the players who carry it out and the rules governing them in order to understand how a quality translation is achieved. The interviews conducted, and the state of knowledge have shown that to understand how different levels of quality can be attained, the process and production conditions of a translation must be examined, that is, all decisions and processes that could influence the level of quality. Then, as we have seen, translation is a means to meet objectives related to Canada’s linguistic obligations and commitments. There is one aspect of quality that is found in the technical nature of the attribute, but also in its most subjective aspect, namely, the perception that users have of it. The definition of quality is therefore different from one person to another. However, let us remember, in the Canadian context, General Murray’s request in a document calling for measures to improve the administration of justice that clearly sets out the issues created by functional bilingualism under the rule of law. Specifically, the request concerned the hiring of a Chief Justice and three judges to support him. It specified that “[all of them should] know French and one of them in particular should be aware of the uses of French.”15 In their research on the issue of the interpretation of bilingual texts, Brun and Coll. point out that [translation] “[t]he interpretation of bilingual texts involves a search from the legislator’s intention starting with a comparison of the two versions, while taking their context into consideration. In terms of constitutional matters, this approach would normally result in the broadest version.”16 Translation quality, in its capacity to render the intention of the author of the text to be translated, is therefore paramount, and it is recognized that this has consequences.

15

Report of the Attorney and Solicitor General Regarding the Civil Government of Quebec, submitted in 1766 to The Right Honourable Lords of the Committee of the Privy Council for Plantation Affairs. 16 Brun and Coll., idem, p. 851. 11

The literature that addresses translation quality can be broken down into several segments. Some are more technical, others are focused on the meaning, and others can be qualified as “pragmatic” or utilitarian. The first element that defines quality is certainly editorial correctness. In this case, both the interviewees and the literature have indicated that there is a “required minimum,” that is, grammatical aspects, syntax and spelling. This first element is somewhat essential, and a quality measurement cannot disregard it. Second, there is the dimension of the editorial accuracy provided in comparison with the source content. Once again, the interviewees and the literature are unanimous about the fact that the translated content must accurately reflect the information and messages communicated in the source text. The third element revealed in the literature and the discussions is the issue of the aesthetics of the text. Aside from the simple message given, there are issues of style in the target language that are also elements of the definition of a quality translation. In other words, and simply put, a wordfor-word translation is not recommended to ensure quality. The translated text must reflect the usual aesthetics of the target language, whenever possible. Fourth, a quality translation must take into consideration the idiomatic expressions of the source and target languages, as well as the socio-cultural context of the examples, illustrations and mental images into which the reader will be drawn. If a translation includes references to known artists, television or radio programs or famous individuals who have made their mark, there could be significant differences between the socio-cultural framework of the source language and that of the target language. To be clearly understood, a translation must take these elements into consideration. Fifth and lastly, the purpose for which a translation is made is important. The classic example is that of instructions on how to use a hazardous tool or machine, or the translation of a legislative text. In these cases, the consequences of a poor translation are quite clear. The requirements could be less demanding depending on how the text will be used. However, opinions differ on this matter. For some, all translations must be of high quality. For others, the context and its uses play a role. These five elements are directly related to the translated text. They can, more or less easily and accurately, be evaluated by analyzing the text. Other elements were brought to light, particularly during the interviews, and they mainly concern questions of context, that is, in what environment a translation is done. In this instance, we are talking about the issue of service standards as quality determinants. For example, these include issues such as deadlines, translators’ specialties, whether there is access to thesauri and translation memories, and possible exchanges between the translator and the author. These aspects are important, but they are related to quality determinants. They will also be used to formulate or explain the hypotheses that underlie a translation model that facilitates the attainment of the desired level of quality. We shall refer to them as “service standards.”

12

Before pursuing the discussion on the development of a measurement model, it is important to position the role of the Translation Bureau as a key, but not unique, player in the delivery of translation within the federal government.

4.2.

Role of the Translation Bureau

While the Translation Bureau is not explicitly mentioned in the legislation as an implementer, it is clear that its mandate, established in 1934, “supports the Government of Canada in its efforts to serve Canadians and communicate with them in both official languages.”17 Specifically, its incorporating legislation, Translation Bureau Act (R.S.C., 1985, c. T-16), states that: 4(1) The Bureau shall collaborate with and act for all departments, boards, agencies and commissions established by Act of Parliament or appointed by order of the Governor in Council and collaborate with and act for both Houses of Parliament in all matters relating to the making and revising of translations from one language into another of documents, including correspondence, reports, proceedings, debates, bills and Acts, and to interpretation, sign-language interpretation and terminology.18 To this end, the Translation Bureau provides various services that enable it to fulfill its mandate. These activities can be summarized as follows: The Translation Bureau provides translation, revision, proofreading, terminology and interpretation services and linguistic advice to federal departments and agencies, Parliament, the Senate and private businesses that have contracts with the federal public service. These services are offered in both official languages, Canada’s Aboriginal languages, over 100 foreign languages and in visual and tactile sign language.19

4.2.1. Description of Translation Bureau activities More specifically, according to Public Services and Procurement Canada’s Report on Plans and Priorities, the Translation Bureau is one of the Department’s nine programs and it includes four sub-programs: •

Program: Linguistic Management and Services •

Sub-programs o Terminology Standardization Program o Translation and Other Linguistic Services o Translation and Interpretation to Parliament

17

https://www.tpsgc-pwgsc.gc.ca/bt-tb/contact-eng.html page consulted on January 13, 2018. R.S.C., 1985, c. T-16. 19 https://www.tpsgc-pwgsc.gc.ca/bt-tb/contact-eng.html page consulted on January 13, 2018. 18

13

o

Conference Interpretation


Increased capacity of federal institutions to function internally in both official languages and with Canadian citizens, and to fulfill their official languages obligations

Various medium-term repercussions depending on outputs

Various immediate repercussions depending on outputs

Various outputs produced by the activities

Language Portal

Terminology Services

Translation Services

Interpretation Services

Translation Bureau Activities

innovation component

Innovation (R&D Activities)

The Canadian Charter of Rights and Freedoms states that English and French are the official languages of Canada with equal status. Also, the Official Languages Act establishes bilingualism as a practice in federal institutions and in their relationships with the Canadian public. A number of directives, programs, and/or activities resulting from this are led by various players, including the Treasury Board Secretariat, the Office of the Commissioner of Official Languages, Canadian Heritage, Public Services and Procurement Canada, and all federal departments and agencies. Since everyone can work in their language of choice, the production of documents and communications requires translation to ensure respect for the spirit and the letter of the legal framework for linguistic obligations.

These four sub-programs comprise several activities designed to deliver the sub-programs and provide tangible results. It must be pointed out that the Translation Bureau’s activities also stem from the Department’s and the Translation Bureau’s support of the Roadmap for Canada’s Official Graphic 2 Languages 2013‒2018.20 As a result of these commitments Long-Term Program Results Chain Logic to Canadians, the Translation Bureau has been mandated to Functioning of Canadian society in terms of official languages in compliance with its established legal framework made possible maintain a Language Portal and conduct research and Citizens’ access to Government of Canada programs and services in their language of choice at the expected level of quality development in matters of translation. As Graphic 2 shows and to situate the Translation Bureau in the broader context of its contribution to the government’s linguistic obligations, the Translation Bureau’s activities have therefore been consolidated under five major themes: • Translation services • Interpretation services • Terminology services • Management of the resources of a Language Portal • A technological

These five major themes are the basis of the Translation Bureau’s contribution to the Canadian government’s language program. It is important to situate the Translation Bureau and its main activities in the broader context of the Canadian government’s language program to clearly understand the Translation Bureau’s contribution to these objectives in comparison with activities stemming from other key players such as, the Treasury Board Secretariat, the Department of Canadian Heritage, and the Office of the Commissioner of Official Languages. Moreover, this also helps to better understand the Translation Bureau’s limits and constraints with regard to the fulfillment of its mandate. Lastly, since the focus of this framework is translation quality, it is important to be able to measure the repercussions of this quality on the immediate uses, but also on the secondary and ultimate uses of translations in the chain of activities of the Canadian government’s language program. For example, is there a link between translation quality and the management of resources in the Language Portal which is used by teachers in Canadian schools? 20

Government of Canada, Education, Immigration and Communities: Roadmap for Canada’s Official Languages 2013‒2018, Ottawa, 2013. 14

Are there any links between translation quality and the quality of programs and services delivered to Canadians?

4.3.

Discussion of the theory of change, its hypotheses and risks

After examining the context in which the Translation Bureau operates, it is important to understand how translation can be done within the federal government, what are hypotheses that can ensure translation quality and what are the risks associated with the translation processes chosen. First, there are several translation processes. Since not all translations are done or revised by the Translation Bureau, it is important to develop a theory of change that clearly reflects this reality. The various processes differ primarily depending on the method chosen to do a translation: a purely mechanical translation done by the client (use of software or a “free translation”), a translation sent to a private firm, a translation sent to and done by the Translation Bureau, or a translation sent to the Translation Bureau but sub-contracted to a private firm. These different processes entail unknowns in terms of requirements such as translators’ professional qualifications, existing control mechanisms in place, including text revisions, or access to translation memories or to various thesauri specific to uncommon vocabulary. Second, it is also crucial to understand the context in which translations are requested. This is where the service standards come into play. For example, are the deadlines reasonable? Should the translation process be initiated immediately upon the creation of a project that will necessarily produce documents to be translated? Is there a communication channel between the designated translator and the author of the text to be translated? Is there sufficient specific information on the expected requirements in terms of the level of quality according to the final use of the text, for example? This background information is important to understand the reasons that can explain the level of quality beyond the competence of a professional translator alone. The translation service chosen, and the service standards are two elements that were raised by the respondents, and they also reflect certain principles specified in the literature regarding the possibility of attaining an appropriate level of quality. These two components are the pillars of the business model that will lead to the achievement of quality translation, or not.

4.3.1. Hypotheses The hypotheses underlying the operation of quality translation services are numerous. However, seven of them are considered critical. The first hypothesis is that there is a translation capacity to meet expectations and the government’s needs. It must be understood that government activities are diversified and cover unique areas such as, defence, intelligence, old age pensions, and environmental policies. Moreover, Graphic 1 shows that the logic of a long-term program is anchored in constitutional and legislative obligations.

15

The second hypothesis is that the federal government has a business model that allows for the existence, maintenance and renewal of this translation capacity in relation to the government’s requirements and needs. The quality requirements are not commercial, but they are constitutional and political. The signals regarding the government’s needs and requirements that it sends to training institutions and potential suppliers are therefore critical to ensure the existence and renewal of this translation capacity. Moreover, this business must allow for the integration of processes leading to quality. Consequently, the third hypothesis is that there must be clearly defined and measurable criteria (standards) of what is meant by “quality.” These criteria need not be strictly of a technical nature, but they must include the five elements mentioned in section 4.1.2 and be integrated into a business model that makes them achievable. Canadian standards exist, and many people have referred to the quality of these standards. However, these standards themselves entail certain requirements regarding their application. If these are the ones that are chosen, they should be carefully analyzed in order to identify the requirements that stem from them so as to create a complete and viable business model. If there are other standards, the same work must also be done. It is clearly indicated that the existence of quality standards must be reflected in the business model and requirements. As observed above, there are many ways to define and measure the quality of a translation. For the players involved in translation processes to work effectively, there must be such a definition and an awareness of the constraints, levers and mechanisms through which these requirements will be achieved. Similarly, these standards will provide trainers with indications of market needs. The players’ central issue is related to the attainment and respect of this translation quality. For example, with regard to the Translation Bureau and for this purpose, the Translation Bureau Regulations (C.R.C., c. 1561) should be kept in mind, which state that: The Bureau shall … (d) ensure that translation or revisions made by it conform, in so far as Canadian usage permits, to “le français universel” or Standard English21 The fourth hypothesis is that these quality standards are accompanied by service standards integrated into a business model than ensures their compliance. The service standards include, for example, the time allotted, access to translation memories and interactions between the translator and the author of the original text. The fifth hypothesis is that the translations are sent to and done by or under the responsibility of an “authority” that becomes accountable and responsible for ensuring that the translations meet the quality requirements. Currently, it is often the Translation Bureau that is perceived as this authority for all federal government translations, but, in practice, the Translation Bureau is and can only be accountable and responsible for translations that go through its own offices. This authority must be the responsibility of the translation service chosen. One way of making it accountable could be the obligation of adding its name at the end of a translation.

21

C.R.C., c. 561, Translation Bureau Regulations, http://laws-lois.justice.gc.ca/PDF/C.R.C.,_c._1561.pdf, page consulted on January 13, 2018. 16

Furthermore, the sixth hypothesis is that translation must be viewed as an exercise where the responsibility is shared among various players. They include the requester, the author of the original text, the translator, the organization (the Translation Bureau or private firms), and the institutional client. As it is a process involving multiple interdependencies, this hypothesis is central to the theory of change. The institutional client’s responsibility could materialize through the implementation of institutional policies guaranteeing quality assurance processes. The seventh hypothesis is that everyone takes part in the building of translation memories, have access to them and are required to use them, and that everyone takes part in the building of thesauri, have access to them and are required to use them. This is important in order to ensure consistency between federal government translations and interpretations. The eighth hypothesis is that, in the translation process, the level of quality is controlled and validated by a clearly identified authority that is held accountable and responsible for translation quality. Finally, the ninth and last hypothesis that we have chosen is that translations must be subject to revisions and ex-post ad hoc independent evaluations to ensure that they meet government standards and satisfy users’ needs.

4.3.2. Risks For the purposes of the framework, we have identified five significant risks related to the attainment of appropriate quality translation. They are not presented in order of priority. The first risk is related to the business model. In this case, it is first essential to distinguish among four translation “sources.” The first source is a translation done by the Translation Bureau. The second source is a translation done by a third party under the responsibility of the Translation Bureau. The third source is a translation done outside the Translation Bureau’s field of operations, that is, it comes from a translator who is outside the Translation Bureau or mandated by the Translation Bureau. Lastly, the fourth source would be a free translation automated by software and or by an individual using his/her own resources. Then it is important to note that the quality assurance mechanisms could differ depending on each one of these sources. Lastly, the risk is related to the fact that if the business model is not clear and does not include appropriate service standards to guarantee quality translation work, the probabilities of ending up with unsatisfactory translations increase. This business model must also allow for the clear identification of accountabilities. The second risk is the absence of clear and shared quality standards. This risk was mentioned frequently under different names, the most frequent of which was the existence of what many referred to as “good enough.” If quality standards are not clearly set out, the risk of downward grading for cost issues could undermine the expected quality. The third risk is related to capacity and work organization. Translation involves first and foremost professionals whose numbers are limited. The government must ensure that the means used for translation are sufficient to meet demand. In addition, it must ensure that it can draw from a large enough pool of suppliers and that this pool is continuously and adequately renewed. Lastly, it

17

must ensure that it reaps the benefits from digital technologies. This must be done in the spirit of organizing the work so as to benefit from technological advances that will make it possible to operate in the most efficient way possible. To this end, the government must mandate a responsible organization, the Translation Bureau in this case, and provide it with an opportunity to benefit from the required means to fuel this thought process and identify concrete and effective technological solutions. The fourth risk is related to service standards. Regardless of the business model chosen, service standards are required, and this also means control and evaluation processes to ensure compliance with these standards. The service standards guarantee the existence and the adaptation of activities and of their sequence in support of the attainment of quality. If such standards do not exist, there is a risk that things will not be done properly and will simply remain incomplete. Finally, the fifth risk is the governance model. Partly related to the business model as mentioned above, the governance model emphasizes the sharing of responsibilities between the translation service and the institutional client. If the institutional client offloads the translation request and makes it the business of the translation service alone without any consideration for service and quality requirements and without taking part in it, it will be very difficult to attain the desired quality. If they are not controlled, these various risks will have consequences. The first one will affect the quality which could then be reflected in the various issues of accountability between the use of a translation and the level of quality required. For example, it should be noted that personal internal use is not the same as official internal use intended for all employees, or as official communication with the public or as describing a program designed for Canadians. Other consequences could be of a political or legal nature if the translation does not make it possible to properly meet the requirements of federal government services.

4.3.3. Results chain and intervention modelling Translation is the result of the implementation of a multi-step process. The purpose of this section is to explain these steps to better understand the cause to effect chain. On the basis of this causal chain, it is possible to better understand the role and importance of hypotheses in a rigorous translation process. First, to enter into production mode, translations required by federal institutions must be sent to a translation service. As this is not mandatory, the Translation Bureau does not receive all translations and it has no control over the process outside its sphere of operations. Hence, a first division is created between what can be “directly” controlled by the government and what cannot be, at least as easily. Requirements and controls can be made for Translation Bureau activities, but this is a more sensitive issue for external firms. However, when translations are done externally, standards and processes can be guaranteed to ensure quality control. In these cases, the vigilance and willingness of the client-institution also play an important role, and this is why a principle of accountability must apply and be part of the business model. Even when translations are done by the Translation Bureau, the client-institution must bear some responsibility in the process since, as it has been mentioned, a number of issues such as the deadlines requested, or

18

the desired uses of the text are the responsibility of the client-institution and not of any translation service. Many respondents mentioned the last-minute practice requiring that a text be divided among several translators, which will not always ensure a high-quality integrated final text. Second, translation activities must necessarily include publishing and quality control activities. These activities ensure that translations meet the quality standards. Third, beyond quality validation, translations feed and are fed by translation memories and thesauri. One of the objectives of this practice is to ensure the standardization of vocabularies used and enrichment of the government’s terminology base. As this is an integral part of the Translation Bureau’s mandate, translations that do not go through the Translation Bureau, be it only for validation purposes, will not automatically access these memories or enrich this terminology base. Language evolves and so does vocabulary. Thus, and for example, with all the issues related to bioethics, climate change, digital technologies, animal mutations, and social environments in continual development, vocabulary is constantly being enriched. As the government must be at the cutting edge of social issues to be in a position to take action and control, it must have a good command of the appropriate vocabulary to do so. Consequently, this brings us to the fourth item which is the use of this terminology base. Not only is terminology useful for future translations, but it is used in all activities and services provided by the Translation Bureau, including interpretation and the Language Portal. Translation therefore affects the terminology base which has an impact on interpretation activities and the sharing of the best terminologies available.

19

Graphic 3 Theory of Change

Long-Term Program Logic

Results Chain The translated documentation used meets the standards certified by the Canadian Government and makes it possible to comply with Canadian society’s official languages socio-legal framework.

The quality of translated documents is evaluated and controlled to ensure that they meet the needs of institutions and Canadians

Citizens’ access to Canadian government information and documentation of interest to them in their language of choice at the expected level of quality. Increased capacity of federal institutions to function internally in both official languages and with Canadian citizens, and to fulfill their official languages obligations.

Documentation needed for the functioning of federal institutions made available in both official languages

Contribution to Language Portal

The service standards include the sharing and use of translation memories and thesauri

The quality of translations is ensured and validated by the translation service chosen and the client The business model clearly identifies accountability with respect to quality control by the supplier Quality standards are known and expected The texts to be translated are sent to a translation service in compliance with service standards Translation capacity and a viable translation business model exist existfaires viable

Facilitated and improved interpretation capacity

the

Enhancement and use: • of federal expertise in terminology • of the Termium linguistic database

Texts, documents, etc translated according to quality standards expected by federal requesters

Production and/or validation of terms

Quality control activities jointly ensured by the client and the translation service

Translation activities carried out by the translation service chosen

Texts produced in one of the two official languages and translation services requested


Hypotheses

Fifth, translation affects the government’s capacity to communicate internally. Translated documentation makes it possible to operate federal institutions in the spirit and the letter of the Canadian constitution. This is the first repercussion. Sixth, these same translations allow the government to interact with its citizens in the country’s two official languages and thereby respect its linguistic commitments and obligations to the public. However, to be successful, a translation must be of quality, which means that the concept in one language must be equivalent to the concept in the other language. There is therefore a need for control and evaluation or quality measurement in its less technical dimension related to understanding.

This must make it possible to function in the spirit and the letter of the Charter, and the Canadian language laws and policies. This is the second repercussion of translation activities. Graphic 3 shows the theory of change with hypotheses in a generic translation model. This means that, in this case, translations are done by various stakeholders without necessarily going through the Translation Bureau. The model reflects the requirements for a quality translation.

5. Principles and quality measurement This section briefly points out a few variations in the definitions and activities of the translation model and proposes the basic principles for measuring quality.

20

5.1. General observations Generally speaking, the interviews allowed for the validation of the various steps of a translation model that could be qualified as the “ideal type.” Graphic 3 is The interviews allowed for the actually based on what a translation model that ensures optimal validation of the hypotheses. They were recognized as vital to attaining quality should be. Of course, there are disparities among the an appropriate quality level. various respondents. The two major disparities concern the definition of quality and the business model. Some respondents defined quality in a more demanding and sometimes a narrower way. In this last instance, the main criteria should be linguistic and be as in line as possible with the best practices of the target language. Others see the need to adapt further to clients and especially to the public for whom the text is intended. This gave rise to many questions that are not directly within the purview of translation, such as the quality of the source texts which varies considerably and does not always meet the requirements of the source language. These issues are real and have an impact on translation work. There is no single answer to these situations. The second element that was clearly indicated is the business model. Several possibilities arose in this case, and they are based on various forms of logic. Some believe that the Translation Bureau should be the only player responsible and accountable for all translations. This would create a focal point for the quality responsibility. Others believe that the Translation Bureau could play the role of quality monitor, regardless of the translation service that does the translation. Some suggest that only certified translators and translators who are members of recognized professional associations be used. Some have pointed out that a number of translation firms have too few certified translators for their business volume and that quality can be adversely affected owing to a lack of sufficient capacity to translate and revise texts. These two elements are important because they are anchored in the hypotheses. These discussions and choices come under the purview of the government, and it must necessarily respond to them if it wishes to ensure a level of quality in translation. However, the responsibility should not remain strictly with a single agent. As mentioned above, this responsibility is shared and must necessarily include the institutional client who must take the means to support the overall translation process, through acceptable practices and compliance with service standards, for example.

21

5.2.

Proposals

On the basis of the interviews, the state of knowledge and the methodological considerations, two series of proposals can be made. But first, a series of seven guiding principles should inform quality measurement. The guiding principles emerging from this framework are the following: 1- Quality is recognized as a shared responsibility, and it is initially the responsibility of the client organization to ensure quality. 2- Quality is multi-dimensional and includes elements related to the quality of the language, to the reflection of the socio-cultural context, and to the uses that are identified, pre-identified and approved by the client. 3- Quality is anchored in a quality standard. 4- Service standards exist, and translation quality depends in part on compliance with them. 5- The quality of a translation is measured in consideration of the business model used. 6- Quality is evaluated systematically, periodically and randomly. 7- A central authority is responsible for conducting random evaluations of translations. The second group of proposals covers all quality measurement mechanisms, while taking hypotheses and the results chain into consideration. In light of these guiding principles, the theory of change and considerations related to the long-term logic of the program, four measures are proposed to obtain a complete and satisfactory picture of quality. To clarify, quality measurement means measuring several dimensions of a translation. First, the technical dimension of quality must be measured. It is certainly the most concrete one because the measurement is based on language rules. The Translation Bureau already has expertise in this area and performs this task on a regular basis. This step must be part of any attempt to measure quality. Second, the “signified” dimension of the quality of translations must be measured through satisfaction surveys. One can ask whether the translations are satisfactory in the eyes of institutional clients. This could be done, for example, through ongoing discussion groups or online panels. Third, it is important to measure the understanding that end users have of quality. Initially, a first group of participants for this measurement could come from within departments in order to measure the quality of translations made of messages intended for employees. Then, a second group of users could be made up of citizens. Once again, online panels across the country would facilitate the measurement of this understanding. Moreover, in both cases, it would be interesting to add equivalent panels in both languages, because it is a matter of understanding the text. This step would enable the government to measure not only the quality of its translations, but also the quality of its communications with citizens. In fact, it’s the only way of knowing whether the translation is of quality, since in the event that neither language group is satisfied with a text, the problem may reside in the original text, as several stakeholders have mentioned. Fourth and lastly, by taking a comparative approach, it would be important to measure the quality of translations on the basis of business models (quality standards and service standards). In other 22

words, the steps taken on the basis of the first three methodologies should be taken in accordance with existing business models.

6. Conclusion The measurement framework discussed and proposed in this report first illustrates the complexity of measuring the quality of a translation. Thus, the very definition of quality includes many aspects to be taken into consideration. The importance of having quality standards, but service standards as well, becomes critical to the achievement of a given level of quality, but also to its measurement. The business model is a central element to be reviewed in the measurement of quality. The various guiding principles emerging from the analysis should constitute the elements of a sufficiently strong and realistic foundation so that quality can be the subject of ongoing measurement. Finally, the proposed measurement strategies demonstrate the importance of developing a complete quality evaluation model, that is, one that takes into consideration the various facets of quality, as discussed in this report.

23

Selective Bibliography AILIA, Certification Preparation Guide: CAN/CGSB-131.10-2008, AILIA Publications, Toronto, Canada 2008, 16 p. AILIA, Guide de préparation à la certification : CAN/CGSB 131.10-2008, Publications AILIA, Toronto, Canada, 2008, 16 p. Al-Qinai, Jamal, “Translation Quality Assessment. Strategies, Parameters and Procedures,ʺ Meta : journal des traducteurs, Presses de l’Université de Montréal, Canada, 2000, vol. 45, no 3, pp. 497519. (The journal is bilingual, but the articles appear only in English or in French. Archibald, James, “Managing Translation Quality in Multilingual Settings: The Example of the European Unionʺ, Circuit – Magazine d’information des langagiers, Ordre des traducteurs, terminologues et interprètes agréés du Québec (OTTIAQ), Montréal, Canada, printemps 2017, numéro 134. (Only in French) Baker, Mona and Gabriela Saldanha (Editors), Routledge Encyclopedia of Translation Studies, 2nd Edition, Routledge, London (UK) & New York (USA), 2009, 680 p. Bargas-Avila, Javier A. and Bruhlmann, Florian, “Measuring User Rated Language Quality: Development and Validation of the User Interface Language Quality Survey (LQS),ʺ International Journal of Human-Computer Studies, Elsevier Big Data, 2016, vol. 91, 40 p. Bowker, Lynne, Computer-Aided Translation Technology: A Practical Introduction, University of Ottawa Press, Canada, 2002, 220 p. Brunette, Louise, « Normeset censure : ne pas confondre», Journal Translation, Terminology, Writing =Revue TTR : traduction, terminologie, rédaction, Canadian Association for Translation Studies, Canada, 2002, vol. 15, no 2, p. 223233. (Like Meta, the journal is bilingual, but the articles appear only in English or in French. The official title does not include the term Revue.) Centre de traduction des organes de l’Union européenne – CITI, Traduire et interpréter pour l’Europe, Office des publications officielles de l’Union européenne, Bruxelles (Belgique) & Luxembourg, 2017, 24 p. Conde, Tomas, “Translation Evaluation on the Surface of Texts: a Preliminary Analysis,ʺ The Journal of Specialised Translation, University of Roehampton, London (UK), 2011, Issue 15, pp. 6986. Cronin, Michale, Translation and Globalization, Routledge, London (UK) & New York (USA), 2003, 208 p.

24

Delisle, Jean et Fiola, Marco A., La traduction raisonnée : manuel d’initiation à la traduction professionnelle de l’anglais vers le français, 3e édition, Presses de l’Université d’Ottawa, Canada, 2013, 720 p. (Not translated) Delisle, Jean, « L’évaluation des traductions par l’historien », Meta: Translators’ Journal = Meta journal des traducteurs, Presses de l’Université de Montréal, Canada, 2001, vol. 46, no 2, p. 209226. Émergences : Formation – Conseil – Expertise, Normes et qualité au travail, Association Émergences, 2017, dossier d’actualité. (Not translated) European Commission – Directorate General for Translation – Communication and Information Unit, Translating for a Multilingual Community, Official Publication of the European Union, Brussels (Belgium) & Luxembourg, 2009, 24 p. European Committee for Standardization, European Quality Standard for Translation Service Providers EN-15038:2006 Standard, CEN Translation Services, Brussels, Belgium, 2006, 20 p. Gémar, Jean-Claude (ed.), The Language of Law and Translation: Essays on Jurilinguistics = Langage du droit et traduction : essais de jurilinguistique, Éditeur officiel du Québec & Linguatech Éditeur Inc., Conseil supérieur de la langue française, Québec (QC), Canada, 1982, 324 p. (The collective has a bilingual title, but the chapters are written in a single language: some chapters are written in English.) Giroux, Hélène et Taylor, James R., « L’évolution du discours sur la qualité : d’une traduction à l’autre », Revue Communication & Organisation, Presses Universitaires de Bordeaux, France, 1999, vol. 15. (Not translated) Gorter, Dirk, Linguistic Landscape: A New Approach to Multilingualism, Multilingual Matters, Clevedon (USA), 2006, 95 p. Gutt, Ernst-August, Translation and Relevance: Cognition and Context, 2nd Edition, Routledge, London (UK) & New York (USA), 2014, 272 p. House, Juliane, “Translation Quality Assessment: Linguistic Description versus Social Evaluation,ʺ Meta: Translators’ Journal = Meta: journal des traducteurs, Presses de l’Université de Montréal, Canada, 2001, vol. 46, no 2, pp. 243-257. House, Juliane, Translation Quality Assessment: Past and Present, Routledge, London (UK) & New York (USA), 2015, 171 p. Huang, Harry J., “Scandals in Translation Quality Assessment,ʺ The Linguistic Association of Canada and the United States (LACUS) Forum XXXIII: Variation, 2009, pp. 283-295. Huntington, Samuel P., The Clash of Civilizations and the Remaking of World Order, Simon and Schuster, New York (USA), 1997, 368 p.

25

International Organization for Standardization, ISO 12616:2002 (en) Translation-Oriented Terminography, ISO Central Secretariat, Geneva, Switzerland, 2002, 27 p. International Organization for Standardization, ISO 17100:2015 (en) Translation Services — Requirements for Translation Services, ISO Central Secretariat, Geneva, Switzerland, 2015, 20 p. International Organization for Standardization, ISO 18587:2017 (en) Translation Services — PostEditing of Machine Translation Output — Requirements, ISO Central Secretariat, Geneva, Switzerland, 2017, 15 p. International Organization for Standardization, ISO/AWI 21999 Translation Quality Assurance and Assessment – Models and Metrics, ISO Central Secretariat, Geneva, Switzerland International Organization for Standardization, ISO/TS 11669: 2012 (en) Translation Projects — General Guidance, ISO Central Secretariat, Geneva, Switzerland, 2012, 35 p. International Organization for Standardization, Quality Management Principles, ISO Central Secretariat, Geneva, Switzerland, 2015, 20 p. Jiménez-Crespo, Miguel A., Translation and Web Localization, Routledge, London (UK) & New York (USA), 2013, 244 p. Kelly, Nataly and Jost Zetzsche, Found in Translation: How Language Shapes Our Lives and Transforms the World, Penguin Group Inc., New York (USA), 2012, 288 p. Kelly, Nataly, Ten Common Myths About Translation Quality, Huffington Post – The Blog, September 17, 2013. Kockaert Hendrik, J. et Winibert Segers, « L’assurance de la qualité des traductions : items sélectionnés et évaluation assistée par ordinateur », Meta: Translators’ Journal = Meta : journal des traducteurs, Presses de l’Université de Montréal, Canada, 2012, vol. 57, no 1, pp. 159-176. Larose, Robert, « Méthodologie de l’évaluation des traductions », Meta: Translators’ Journa; = Meta : journal des traducteurs, Presses de l’Université de Montréal, Canada, 1998, vol. 43, no 2, pp. 163-186. Larose, Robert, « Qualité et efficacité en traduction : réponse à F. W. Sixel », Meta: Translators’ Journal = Meta : journal des traducteurs, Presses de l’Université de Montréal, Canada, 1994, vol. 39, no 2, p. 362-373. Mohammadi Dehcheshme, Maryam, Machine Translation and Translation Memory Systems: An Ethnographic Study of Translators’ Satisfaction, Ph.D. Thesis, University of Ottawa, Canada, 2017, 227 p.

26

Morin-Hernandez, Katell, La révision comme clé de la gestion de la qualité des traductions en contexte professionnel, thèse de doctorat en langues étrangères appliquées, Université Rennes 2, France, 2009, 273 p. (Not translated) Mossop, Brian, Revising and Editing for Translators, 3rd Edition, Routledge, London (UK) & New York (USA), 2014. Munday, Jeremy and Basil Hatim, Translation: An Advanced Resource Book, Routledge, London (UK), 2004, 373 p. Nida, Eugene and Charles Taber, The Theory and Practice of Translation, Leiden: Brill, Netherlands, 1969, 226 pages. Nida, Eugene, Toward a Science of Translation, Leiden: Brill, Netherlands, 1964, 331 pages. Nord, Christiane, Text Analysis in Translation: Theory, Methodology and Didactic Application of a Model for Translation-Oriented Text Analysis, 2nd Edition, Rodopi, Amsterdam (The Netherlands) & New York (USA), 2006, 274 p. Ogden, Charles and I.A. Richards, The Meaning of Meaning: A Study of the Influence of Language Upon Thought and of the Science of Symbolism, Harcourt, Brace & World Inc., New York (USA), 1923, 386 p. Organisation internationale de normalisation, ISO 12616:2002 (fr) Terminographie axée sur la traduction, Secrétariat central de l’ISO, Genève, Suisse, 2002, 28 p. Organisation internationale de normalisation, ISO 17100:2015 (fr) Services de traduction — Exigences relatives aux services de traduction, Secrétariat central de l’ISO, Genève, Suisse, 2015, 20 p. Organisation internationale de normalisation, ISO 18587:2017 (fr) Services de traduction — Postédition d'un texte résultant d'une traduction automatique — Exigences, Secrétariat central de l’ISO, Genève, Suisse, 2017, 16 p. Organisation internationale de normalisation, ISO/TS 11669:2012 (fr) Projets de traduction — Lignes directrices générales, Secrétariat central de l’ISO, Genève, Suisse, 2012, 35 p. Organisation internationale de normalisation, Principes de management de la qualité, Secrétariat central de l’ISO, Genève, Suisse, 2016, 20 p. Paradis, Michel, A Neurolinguistic Theory of Bilingualism, John Benjamins Publishing Co., Amsterdam, The Netherlands, 2004, 299 p. Pepermans, Raymond, « Gestion de la qualité, contrôle de la qualité ou assurance de la qualité?», Terminology Update = L’Actualité terminologique, Government of Canada Publications, Ottawa, 1987, vol. 20, no 2, page 4.

27

Piller, Ingrid, Intercultural Communication: A Critical Introduction, Edinburgh University Press, Scotland, 2011, 256 p. Public Works and Government Services Canada (PWGSC), 2012-603 Evaluation of Translation Bureau Programs - Volume 1: Terminology Standardization Program (Final Report), PWGSC Office of Audit and Evaluation, Ottawa, Canada, 2014, 24 p. Public Works and Government Services Canada (PWGSC), 2012-603 Evaluation of Translation Bureau Programs - Volume 2: Translation and Other Linguistic Services Program (Final Report), PWGSC Office of Audit and Evaluation, Ottawa, Canada, 2014, 41 p. Public Works and Government Services Canada (PWGSC), 2012-603 Evaluation of Translation Bureau Programs - Volume 3: Conference Interpretation Program (Final Report), PWGSC Office of Audit and Evaluation, Ottawa, Canada, 2014, 23 p. Public Works and Government Services Canada (PWGSC), Evaluation of the Language Portal of Canada (Final Report), PWGSC Office of Audit and Evaluation, Ottawa, Canada, 2016, 52 p. Reiss, Katharina, Translation Criticism – The Potentials & Limitations: Categories and Criteria for Translation Quality Assessment, 2nd Edition, Routledge, London (UK) & New York (USA), 2014, 140 p. Sager, Juan C., Language Engineering and Translation: Consequences of Automation, John Benjamins Publishing Co., Amsterdam (The Netherlands), 1994, 345 p. Samuelsson-Brown, Geoffrey, A Practical Guide for Translators, 5th Revised Edition, Multilingual Matters, Bristol (UK), Buffalo (USA) & Toronto (Canada), 2010, 224 p. Schäffner, Christina and Beverly Adab (Editors), Developing Translation Competence, John Benjamins Publishing Co., Amsterdam (The Netherlands), 2000, 244 p. Searle, John R., Speech Acts: An Essay in the Philosophy of Language, Cambridge University Press, UK, 1969, 214 p. Searle, John, The Construction of Social Reality, Simon and Schuster, New York (USA), 1995, 241 p. Shore, Bradd, Culture in Mind: Cognition, Culture, and the Problem of Meaning, Oxford University Press, UK, 1996, 448 p. Shreve, Gregory and Erik Angelone (eds), Translation and Cognition, John Benjamins Publishing Co., Amsterdam, The Netherlands, 2010, 381 p. Sofer, Morry, The Translator’s Handbook, 6th Revised Edition, Schreiber Publishing Inc., Maryland (USA), 2006, 402 p.

28

Steiner, George, After Babel: Aspects of Language and Translation, 2nd Edition, Oxford University Press, UK, 1998, 560 p. Translation Centre for the Bodies of the European Union – ICTI, Interpreting and Translating for Europe, Official Publications Office of the European Union, Brussels (Belgium) & Luxembourg, 2017, 24 p. Travaux publics et services gouvernementaux Canada (TPSGC), 2012-603 Évaluation des programmes du Bureau de la traduction - Volume 1 : Programme de normalisation terminologique (Rapport final), TPSGC – Bureau de la vérification et de l’évaluation, Ottawa, Canada, 2014, 26 p. Travaux publics et services gouvernementaux Canada (TPSGC), 2012-603 Évaluation des programmes du Bureau de la traduction - Volume 2 : Programme de traduction et autres services linguistiques (Rapport final), TPSGC – Bureau de la vérification et de l’évaluation, Ottawa, Canada, 2014, 45 p. Travaux publics et services gouvernementaux Canada (TPSGC), 2012-603 Évaluation des programmes du Bureau de la traduction - Volume 3 : Programme d’interprétation des conférences (Rapport final), TPSGC – Bureau de la vérification et de l’évaluation, Ottawa, Canada, 2014, 25 p. Travaux publics et services gouvernementaux Canada (TPSGC), Évaluation du Portail linguistique du Canada (Rapport final), TPSGC – Bureau de la vérification et de l’évaluation, Ottawa, Canada, 2014, 53 p. United Nations Department of Conference Services, United Nations Editorial Manual: A Compendium of Rules and Directives on United Nations Editorial Style, Publication Policies, Procedures and Practice, United Nations Official Publication, New York (USA), 1983, 534 p. Venuti, Lawrence, The Scandals of Translation: Towards an Ethics of Difference, Routledge, London (UK) & New York (USA), 1998, 224 p. Williams, Malcolm, “The Assessment of Professional Translation Quality: Creating Credibility out of Chaosʺ, Meta: Translators’ Journal = Meta : journal des traducteurs, Presses de l’Université de Montréal, Canada, 1989, vol. 2, no 2, pp. 13-33. Williams, Malcolm, Translation Quality Assessment: An Argumentation-Centred Approach, University of Ottawa Press, Canada, 2004, 210 p.

29

Appendix 1 Linguistic Management and Services Program and Sub-Programs Terminology Standardization Program As the federal government’s terminology standardization and language services authority, the Translation Bureau is mandated to develop, standardize and disseminate terminology for the entire public service. This sub-program is therefore aimed at developing terminology and language standards that promote consistency and quality in the government’s communications with Canadians, strengthen the government’s national and international roles in the language field, and showcase Canada’s collective wealth of linguistic and terminological knowledge.

Translation and Other Linguistic Services This sub-program is aimed at providing a full range of integrated language solutions in Canada’s official languages and Aboriginal languages, as well as in approximately one hundred foreign languages. It ensures efficient and effective delivery of reasonably priced, top-quality translation, revision, proofreading and languageadvice services to the judiciary and federal departments and agencies on a cost-recovery basis.

Translation and Interpretation to Parliament This sub-program is aimed at ensuring, in a timely manner, the delivery of translation and interpretation services and other language services to Parliament to enable it to function in two official languages and in any other language, as needed. Specifically, the Translation Bureau translates and revises documents for the House of Commons, the Senate, the Parliamentary Committees of the Senate and the House of Commons, the offices of MPs and Senators, the administrative services of both Houses, the Library of Parliament, the Conflict of Interest and Ethics Commissioner, and the Office of the Senate Ethics Officer. In addition, it provides official language interpretation of the debates in the House of Commons, the Senate, Cabinet and their committees, as well as of press conferences and the proceedings of parliamentary associations. Moreover, interpreters regularly accompany MPs and Senators who sit on Parliamentary Committees when they travel in Canada and abroad.

Conference Interpretation This sub-program is aimed at providing conference interpretation services in Canada’s official languages and Aboriginal languages, as well as in foreign and visual languages to all federal departments and agencies. The conference interpreters provide interpretation services during events such as bilateral and multilateral talks between heads of state and governments, interdepartmental and intradepartmental conferences, and meetings between federal Ministers and the provincial and their territorial counterparts.

30

Appendix 2

Graphic 1

Functioning of Canadian society in terms of official languages in compliance with its established legal framework made possible


Citizens’ access to Government of Canada programs and services in their language of choice at the expected level of quality


Various medium-term repercussions depending on outputs

Various immediate impacts depending on outputs

Various outputs resulting from activities depending on the players and their responsibilities

Various departments, agencies and institutions

Treasury Board Secretariat

Translation Bureau

Canadian Heritage

Office of the Commissioner of Official Languages

Key players responsible for activities of the Canadian government in matters of official languages

The Canadian Charter of Rights and Freedoms states that English and French are the official languages of Canada with equal status. Also, the Official Languages Act establishes bilingualism as a practice in federal institutions and in their relationships with the Canadian public. A number of directives, programs, and/or activities resulting from this are led by various players, including the Treasury Board Secretariat, the Office of the Commissioner of Official Languages, Canadian Heritage, Public Services and Procurement Canada, and all federal departments and agencies. Since everyone can work in their language of choice, the production of documents and communications requires translation to ensure respect for the spirit and the lettter of the legal framework for linguistic obligations.


Results Chain

31


Results Chain

Citizens’ access to Government of Canada programs and services in their language of choice at the expected level of quality


Language Portal Various medium-term repercussions depending on outputs

Various immediate repercussions depending on outputs

Various outputs produced by the activities

Terminology Services Translation Services Interpretation Services

Translation Bureau Activities Innovation (R&D Activities)


Appendix 3 Graphic 2 Long-Term Program Logic

Functioning of Canadian society in terms of official languages in compliance with its established legal framework made possible

32

Appendix 4

Graphic 3 Theory of Change Results Chain The translated documentation used meets the standards certified by the Canadian Government and makes it possible to comply with Canadian society’s official languages socio-legal framework. The quality of translated documents is evaluated and controlled to ensure that they meet the needs of institutions and Canadians

Citizens’ access to Canadian government information and documentation of interest to them in their language of choice at the expected level of quality. Increased capacity of federal institutions to function internally in both official languages and with Canadian citizens, and to fulfill their official languages obligations.

Documentation needed for the functioning of federal institutions made available in both official languages

Contribution to Language Portal

The service standards include the sharing and use of translation memories and thesauri

The quality of translations is ensured and validated by the translation service chosen and the client The business model clearly identifies accountability with respect to quality control by the supplier Quality standards are known and expected The texts to be translated are sent to a translation service in compliance with service standards Translation capacity and a viable translation business model exist existfaires viable

Facilitated and improved interpretation capacity

the

Enhancement and use: • of federal expertise in terminology • of the Termium linguistic database

Texts, documents, etc translated according to quality standards expected by federal requesters

Production and/or validation of terms

Quality control activities jointly ensured by the client and the translation service

Translation activities carried out by the translation service chosen


Hypotheses


Texts produced in one of the two official languages and translation services requested

33

Principles for the Establishment of a Quality

Principles for the Establishment of a Quality

Suggest Documents

Establishment of a National Program for Quality

Principles for the Establishment of Mediation Structures - HCCH

Senior Quality Assurance Establishment Armaments Recruitment For ...

Quality Assurance Principles for Processors in the

Framework for the establishment of a feasible

Building Principles for a Quality of Information Specification for Sensor ...

The principles of municipal industrial clusters' establishment on the ...

Principles of urban quality of life for a neighborhood - Core

establishment of seed quality control programs - SciELO

Establishment of background water quality conditions

Principles Of Accreditation: Foundations for Quality Enhancement

Rational Establishment of Air Quality Standards - CiteSeerX

Establishment of quality control protocols and ...

Rational Establishment of Air Quality Standards - CiteSeerX

Establishment and application of a high-quality comparative ... - J-Stage

Establishment of Quality Circles Concerning Safe Drug Treatment for ...

Establishment of local diagnostic reference levels for quality control in ...

Pharmacognostic study and establishment of quality parameter for

Establishment of a Quality Management System Based on ISO ... - MDPI

Establishment of a New Quality Control and Vaccine ... - CiteSeerX

The establishment of quality systems in veterinary diagnostic testing ...

Quality Management in Higher Education The principles of quality ...

Quality Principles for the Ergonomics of Human-Computer ... - CiteSeerX

A simplified methodology for the analysis of the establishment of ...