Recent advances in translation tools from a cognitive ...

5th Translation Process Research Workshop

Graz, 03 December 2016

Investigating the cognitive ergonomic aspects of translation tools in a workplace setting Sharon O’Brien [email protected]

Carlos Teixeira [email protected]

Centre for Translation and Textual Studies (CTTS) ADAPT Centre for Digital Content Technology Dublin City University (Ireland)

Motivation  CAT tools used by translators are important aids, but are not always optimally designed (e.g. O’Brien, Ehrensberger-Dow, Hasler and Connolly, forthcoming)  How translators interact with tools is not always ergonomically optimal, e.g. tendinitis, eye strain, mental fatigue, stress (cf. Ehrensberger-Dow and O’Brien 2015)

C. Teixeira & S. O’Brien

TPRW5

2

Motivation  Frequent switching between sub-tasks (Brunette & O’Brien 2011), e.g.  Checking exact match in TM  Checking and editing fuzzy match  Post-editing MT suggestion  Checking metadata  Checking terminology  Switching might also demand multiple trips to various parts of the UI and to other tools and UIs

 Understanding tool interaction from a situated, embodied cognition perspective (Risku 2014) might be useful


TPRW5

3

Goals

Overarching: Identify potential for improvement in translation tools in order to reduce physical and cognitive effort. Stage 1: Observe and describe typical usage of tools by professional translators in a naturalistic environment. Stage 2: Test one specific aspect of translation tools (related to MT post-editing)


TPRW5

4

Context

     

Collaboration with LSP (AlphaCRC) & client (Intel) 10 translators (3 male) Based in open-plan office Translation experience: 2.5 – 35 years Using memoQ; 70% of words were machine translated Target languages: BP, DE, ES, FR, IT, NL, PL, SV


TPRW5

5

Methods


TPRW5

6

Methods Screen recording


TPRW5

7

Eye tracking

Methods


TPRW5

8

Methods

Areas of Interest


TPRW5

9

Methods

Areas of Interest


TPRW5

10

Results

Gaze behaviour


TPRW5

12

Results Tools used by the translators in addition to memoQ Primary tools P01 Firefox

P02

x

P03

P04

P05

P07

x x

Excel

x

Babylon

P11

P13

x

Chrome

Xbench

P06

P15 x

x x

x x

Secondary tools P01

P02

Skype

Windows Explorer

P04

P05

x

P07

P11

P13

x

x

x

x

P15 x

x

x

x

x

Notepad

x

(OS system messages)


P06

x

MS Outlook (email) MS Outlook (calendar)

P03

x

TPRW5

x

13

Software elements used by the translators on the two screens

Main monitor (Left) P01

memoQ

P02

memoQ

P03

memoQ

P05

memoQ

P06

memoQ

memoQ

P07 P11

memoQ

P13

memoQ


Results

memoQ

P04

P15

Secondary monitor (Right)

memoQ

TPRW5

15


Main monitor (Left)


P01

memoQ, memoQ Concordance, memoQ Term lookup

Xbench, Firefox browser for Microsoft Language Portal, Linguee

P02

memoQ

memoQ Concordance

P03

memoQ, memoQ Concordance, Firefox browser for GlobalSight

P04

Babylon dictionary

P05

memoQ, memoQ Concordance

P06

memoQ, memoQ Concordance (sometimes moved to the secondary monitor)

memoQ Concordance (sometimes)

P07

Excel spreadsheet, Chrome browser for Microsoft Language Portal, memoQ Concordance

memoQ

P11

memoQ

memoQ Concordance, Firefox browser for Microsoft Language Portal, GlobalSight, TAUS

P13

memoQ

memoQ Concordance

P15

memoQ Concordance, Firefox browser for Microsoft Language Portal

memoQ


Results


TPRW5

16


Main monitor (Left)


P01

memoQ, memoQ Concordance, memoQ Term lookup, memoQ dialog boxes

Xbench, Firefox browser for Microsoft Language Portal, Linguee, Firefox browser for project allocation

P02

memoQ, email notifications

memoQ Concordance, Windows Explorer, calendar reminder

P03

memoQ, memoQ Concordance, Firefox browser for GlobalSight, memoQ dialog boxes, email notifications

Excel (with info not pertaining to the recorded translation task)

P04

memoQ dialog boxes, Windows Explorer, Babylon dictionary, Skype


P05

memoQ, memoQ Concordance, memoQ dialog boxes

[Nothing]

P06

memoQ, memoQ Concordance (sometimes moved to the secondary monitor), memoQ dialog boxes

memoQ Concordance (sometimes), Chrome browser for GlobalSight, WordReference.com, memoQ dialog boxes (some), email notifications, Windows Explorer

P07

Excel spreadsheet, Chrome browser for Microsoft Language Portal, memoQ Concordance, memoQ dialog boxes (some)

memoQ, memoQ dialog boxes (some)

P11

memoQ, memoQ dialog boxes

memoQ Concordance, Firefox browser for Microsoft Language Portal, GlobalSight, TAUS, Outlook main window, email notifications, Notepad

P13

memoQ, email notifications, Windows Update

memoQ Concordance, Outlook main window

P15

memoQ dialog boxes (some), memoQ Concordance, Firefox browser for Microsoft Language Portal, Skype notifications, Windows system messages

memoQ, memoQ dialog boxes (some)


TPRW5

Results

17

Results Use of screens


TPRW5

18

Tools used for searches

Results

memoQ Concordance

Xbench

Babylon

Excel

Globalsight

Microsoft Language Portal

WordReference.com

Linguee

P01

wireless service NFC Location Screen timeout power-consuming cloud intelligence

-

-

-

cloud intelligence

-

cloud intelligence

-

P02

Location Screen timeout Volume level Data Sync Auto Rotation cloud intelligence wireless service provider timeout

Location NFC Location (again) Screen timeout Vibration Volume level Data Sync Data Connection Auto Rotation cloud intelligence -

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

referred

-

timeout Auto Rotation vibration cloud intelligence -

-

-

-

-

-

-

-

-

-

-

-

P07

Intel Security Battery Optimizer free storage Screen timeout Close apps and extend

-

P11

and above referred to referred Location Screen timeout Timeout Data Sync controlled automatically major cloud intelligence battery life floating notification wireless service provider accessed by Screen timeout timeout Auto Rotation Overview of major power consuming power-consuming CPU usage Floating Intel Security Battery Optimizer Getting started with Minimum system requirements data plan Screen timeout timeout Data Connection Auto Rotation screen timeout Overview of major Reports on local apps CPU usage cloud intelligence battery life

-

P03

P04

P05 P06

P13

P15

Supported platforms Supported Intel Battery Optimizer Data Sync cloud intelligence telephony stack Buddies sensors Location Screen timeout intelligence


TAUS Data Cloud Search

-

-

-

-

Screen timeout volume level Data Sync cloud intelligence Close apps and extend Extend more top power consuming Intel Security Battery Optimizer Location Screen timeout location Data Connection cloud service referred to Screen timeout Screen brightness Auto Rotation be controlled

-

deplete

-

-

Screen timeout power-consuming

-

-

-

referred to

-

-

referred to

-

-

-

floating

-

-

-

-

-

-

Screen timeout (2x) timeout Data Connection Auto Rotation CPU usage cloud intelligence intelligence

-

-

-

TPRW5

19

Results Number of searches per tool per translator

P01

P02

P03

P04

P06

P07

P11

P13

P15

TOTAL

memoQ Concordance

6

6

2

5

4

4

12

10

15

64

GlobalSight

-

-

4

-

7

-

5

-

-

16


1

-

-

-

-

2

1

1

8

13

Xbench

10

-

-

-

-

-

-

-

-

10

Excel

-

-

-

-

-

6

-

-

-

6

Babylon

-

-

-

1

-

-

-

-

-

1

WordReference.com

-

-

-

-

1

-

-

-

-

1

Linguee

1

-

-

-

-

-

-

-

-

1


-

-

-

-

-

-

1

-

-

1

18

6

6

6

12

12

19

11

23

113

TOTAL


TPRW5

20

Results Terms/expressions searched for per translator Terms

P01

P02

Screen timeout

2

1

cloud intelligence

4

1

1

Auto Rotation

1

1

1

Location

2

1

timeout

P03

P04

1

P06

P07

P11

P13

P15

TOTAL

2

3

2

1

4

15

accessed by

2

11

and above

1

1

2

7

be controlled

1

1

1

1 1

1

2

2

1

1 1

7 1

2

6

Terms

P01 P02 P03 P04 P06 P07 P11 P13 P15 1

cloud service

1

1

1

1

1

Data Connection

1

1

power-consuming

1

1

referred to Volume level

5 2 2

4

4 1

1

1

controlled automatically

1 1

1

1

Intel Security Battery Optimizer

2

battery life

1

Close apps and extend

1

floating 1

Extend more

1

1

floating notification

Getting started with

2

3

Intel Battery Optimizer

1

3

local apps

1

2

major

2

intelligence

1

4

2 2 1

2

1

2

2

Overview of major

1

referred Vibration

1 1

wireless service provider


1

2 2

1 1

1

1

1

1

1 1

1

1

1 1

1

1

1

Minimum system requirements

1

1

Reports on

1

1

Screen brightness

1

sensors NFC

1

1

free storage

1

1

deplete

4

1

CPU usage

4

1

1

data plan Data Sync

TOTAL

1

1

1

Supported

1

1

Supported platforms

1

1

top power consuming

1

2

wireless service

1

2

TOTAL

17

TPRW5

1 1

6

7

6

12

12

19

11

23

21

113

Results Average time spent on each of the search tools P01 Xbench

1.6

memoQ Concordance

1.3

P02

P03

P04

P06

P07

P15

3.5

4.2

4.2

3.1

3.4

2.2

3.4

2.7

7.0

Globalsight

10.2

10.3

TAUS Data Cloud Search Microsoft Language Portal

12.9

Linguee

12.9

9.1

Average (sec)

3 7

7.5

9

11.2

11

12.3

19.2

12.9

13 13

WordReference.com


P13

2

Excel

Babylon

P11

21.4 23.7

TPRW5

21 24

22

Results Time per search

P01 Xbench

1.6

memoQ Concordance

1.3

P02

P03

P06

P07

P11

P13

P15

3.5

4.2

4.2

3.1

Excel

3.4

2.2

3.4

2.7

7.0

Globalsight

10.2

10.3



12.9

Linguee

12.9

Average (sec) 2

9.1

3 7

7.5

9

11.2

11

12.3

19.2

12.9

13 13

WordReference.com

21.4

Babylon

Number of searches

P04

21

23.7

24

P01

P02

P03

P04

P06

P07

P11

P13

P15

TOTAL

Xbench

10

-

-

-

-

-

-

-

-

10

memoQ Concordance

6

6

2

5

4

4

12

10

15

64

Excel

-

-

-

-

-

6

-

-

-

6

Globalsight

-

-

4

-

7

-

5

-

-

16


-

-

-

-

-

-

1

-

-

1


1

-

-

-

-

2

1

1

8

13

Linguee

1

-

-

-

-

-

-

-

-

1

WordReference.com

-

-

-

-

1

-

-

-

-

1

Babylon

-

-

-

1

-

-

-

-

-

1

18

6

6

6

12

12

19

11

23

113

TOTAL


TPRW5

23

Results Productivity


TPRW5

24

Results Quality


TPRW5

25

Summary and Discussion Fixations on the main tool  Translators spend most time on the Target text box, so: Should this be BIGGER, not equal size, by DEFAULT?  What would be the effect of leaving TM metadata out?  What would be the effect of presenting MT metadata? Use of screens  Most time is spent on one monitor, so: o Why do we assume that two monitors are better? o Should translators be taught to use their monitor set-up in a more facilitative manner?


TPRW5

26

Summary and Discussion Term/expressions search  This accounted for a considerable amount of shifts between tools and screens, so: o Don’t force translators to move to new and slow tools to check terminology? o Don’t force them to check terminology in multiple tools? Productivity  Despite the time ‘wasted’ jumping between AOIs, tools and screens, the translators are nonetheless very productive, so: Could we cognitively go any faster or is this unreasonable? → Perhaps more ergonomic, then? C. Teixeira & S. O’Brien

TPRW5

27

Summary and Discussion Quality  We identified the importance of QA in the quality scores, so: The tools and processes should allow translators to do self-revision, as this pays off for quality? Intersubject variation  We identified different individual behaviours in most of the aspects we analysed, so: Tools should include more personalisation possibilities?


TPRW5

28

5th Translation Process Research Workshop

Graz, 03 December 2016

Thank you!

Sharon O’Brien [email protected]

Carlos Teixeira [email protected]

References Brunette, Louise, and Sharon O’Brien. 2011. “Quelle ergonomie pour la pratique postéditrice des textes traduits ?”. ILCEA 14: unpaginated. Ehrensberger-Dow, Maureen & Sharon O’Brien. 2015. Ergonomics of the translation workplace: Potential for cognitive friction. Translation Spaces 4 (1), Special Issue: 98-118. O’Brien, Sharon, Maureen Ehrensberger-Dow, Marcel Hasler & Megan Connolly. Forthcoming. Irritating CAT Tool Features That Matter to Translators. Hermes. Risku, Hanna. 2014. “Translation process research as interaction research: From mental to sociocognitive processes”. MonTI - Monographs in Translation and Interpreting Special Issue 1: 331–353.


TPRW5

33