CONTEXT. Kanjingo â a touch-based PE tool for smartphones (O'Brien & Moorkens 2014) .... No context (surrounding segments), Difficult to go back (1).
CIUTI 2018
Edinburgh, 31 May 2018
Exploring the potential for touch and voicebased interactions in a translation tool Carlos S. C. Teixeira, Joss Moorkens, Daniel Turner, Joris Vreeke and Andy Way ADAPT Centre for Digital Content Technology Dublin City University (Ireland)
RATIONALE ▪ CAT tools are important aids, but are not always optimally designed ➢ Source, Target, Toolbar, Metadata occupy 19% of screen space each But:
RATIONALE ▪ Automatic Speech Recognition can help translators work faster and more ergonomically (cf. Zapata & Saint 2017, Ciobanu 2016)
RATIONALE ▪ Touch-enable devices are ubiquitous but touch interaction has not been sufficiently explored in translation tools
CONTEXT Modern CAT tools are moving towards minimalist interfaces
CONTEXT
Kanjingo – a touch-based PE tool for smartphones (O’Brien & Moorkens 2014)
OUR APPROACH ▪ Goal: Full-fledged CAT tool for editing suggestions coming from machine translation (MT) and translation memories (TM) ▪ Allows for interaction using multiple input modes:
▪ Browser-based, developed using the REACT framework ▪ Incorporates accessibility features ▪ Compliant with XLIFF standard (1.1 through 2.0) ▪ Instrumented (logs user activities)
THE TOOL ▪ History o “Born” in 2017 o First usability tests: October 2017 o Second usability tests: April 2018 ▪ Developed using AGILE methodology (Scrum) o Iterative, incremental and evolutionary o Short feedback loop and adaptation cycle (vs. Waterfall model)
INCORPORATION OF USER FEEDBACK ▪ Layout o Give more prominence to target text box and bring it closer to source text box
▪ Voice o Automatic capitalisation and spacing o Better handling of punctuation (brackets, apostrophes, etc.)
INCORPORATION OF USER FEEDBACK
▪Touch o Context band
o Bin to delete o Single touch to edit o More buttons o Better responsiveness
THE TESTS Data collection: ▪ Internal logging feature ▪ Screen recording (Flashback) ▪ [Backup: Keystroke logging (Inputlog)] ▪ Voice recording (interviews)
Participants: ▪ Irish professional translators o First test: 10 (6 FR → EN, 4 ES → EN) o Second test: 8 (4 FR → EN, 4 ES → EN), 5 same as first test
THE TESTS Materials: ▪ Four texts of around 300 words (French, Spanish → English) ▪ Pre-translated with Google Neural MT ▪ First test: Article in a multilingual corporate magazine split into 4 chunks ▪ Second test: Four news articles in Spanish (El país) and French (Le monde)
Tasks: ▪ Keyboard & mouse | Voice | Touch | Free
RESULTS – Edit distance
2018
2017
0.25
0.20
0.20
0.15
0.15
0.10
0.10 0.05
0.05
0.00
0.00 Keyboard & mouse
Voice
Touch
Free
Keyboard & mouse
Voice
Touch
Free
RESULTS – Characters produced (per source word)
2018
2017
4.00
3.00 2.50 2.00 1.50 1.00 0.50 0.00
3.00 2.00 1.00 0.00
Keyboard & mouse In Editor
Voice
In Voice box
Touch In Tile view
Free Total
Keyboard & mouse In Editor
Voice
In Voice box
Touch In Tile view
Free Total
RESULTS – Edit time (per source word) (s)
2018
2017
3.00 2.50 2.00 1.50 1.00 0.50 0.00
2.50 2.00 1.50 1.00 0.50 0.00 Keyboard & mouse
Voice
Touch
Free
Keyboard & mouse
Voice
Touch
Free
RESULTS – Errors (per 1,000 source words) Errors/1,000 words
Task Keyboard & Mouse Grammar Omission Spacing Typo Typo (leftover)
Number of errors
2 2 7 2 1 14
Voice Grammar Meaning Omission Proper name! Punctuation Spacing Typo
12.00
10.00 8.00 6.00
1 2 2 1 3 7 3 19
4.00 2.00 0.00 Keyboard & mouse
Voice
Touch
Free
Touch Capitalisation Grammar Meaning Misplacement Punctuation Repetition Spacing Typo
1 6 3 3 1 1 7 3 25
▪ Some participants made many more errors than others
Free Grammar Literal Spacing
2 1 1
RESULTS – User Feedback Preferred input modes: ▪ 4 → Keyboard & mouse ▪ 4 → Keyboard & mouse + Voice ▪ 1 also liked the tile view ▪ 1 liked the touch screen, but not the tile view
RESULTS – Editor view – User Feedback ▪ No spellcheck anywhere (or defective spellcheck) (2) ▪ No context (surrounding segments), Difficult to go back (1)
▪ Generally good ▪ Some glitches were fixed ▪ New Accept button was good
RESULTS – VOICE – User Feedback Speech recognition: ▪ Did not pick up what I said (sometimes), did not recognise accent (5) ▪ You need to adapt your way of speaking (1) ▪ Especially bad for single words (1) ▪ Does not recognise proper names (Antena 3, Huawei) (2) ▪ Tricky with punctuation, capitalisation, etc. (2) ▪ Recognised well, very well (3) ▪ Allows you to type less (1)
RESULTS – VOICE – User Feedback Interaction: ▪ Sometimes words are inserted in the wrong place (1) ▪ Default box size too small (1) ▪ Button to accept in the voice box too similar to button to accept segment (1) ▪ You have to know what you’re going to say, have a pad on the side (1) ▪ Awkward to say just one word, easier to write it (1) ▪ Slows you down (1) ▪ Intuitive, nice (4)
▪ Good to see the text before inserting (1)
RESULTS – TOUCH – User Feedback (Hardware) Test 1 – Touch screen in laptop mode: ▪ Awkward position (tired arms) ▪ [Tendency to use the physical keyboard]
Test 2 – Touch screen in tablet mode: ▪ Tablet gets hot ▪ Tablet is heavy (~ 1 kg) ▪ [Fan is too noisy] ▪ Tablet is too big! (13.5” screen)
RESULTS – TOUCH – User Feedback ▪ Tiles take too much space on the screen, Spread-out text, Hard to read the sentences (ST and MT, and the changes I've made) (6) ▪ Bin not working as expected: slow, did not delete, wrong words being deleted (4)
▪ Context band too narrow (1) (Context band used? 3 no, 1 yes, 4 not sure) ▪ Unable to select multiple tiles (words), interrupts the flow of work ▪ Wouldn't obey, have to type hard, difficult to place word where you want
RESULTS – TOUCH – User Feedback (cont’d) ▪ Hated it, Didn't like it at all, It was awful (4) ▪ Frustrating, not enjoyable (2) ▪ Difficult to use (1)
▪ On-screen keyboard was good (4) ▪ Tablet mode OK (2) ▪ Responded well (1) ▪ Handier for changing things (1) ▪ Automatic capitalisation good, choice also good (1) ▪ Visual improvement from last time (looks nicer) (1)
DISCUSSION Voice ▪ Speech recognition very good for some, not so much for others --> Training (human and engine)? ▪ What is the best way to combine ASR with TM and MT?
Touch ▪ Limitations of HTML5 (browser-based applications) ▪ Tile view more disruptive than touch interaction ▪ ‘Natural User Interface’ (Microsoft)
▪ Useful for certain use cases?
MOVING FORWARD Test additional existing features: ▪ Accessibility features (W3C labelling standards & universal design principles) ▪ Voice commands
Introduce new features: ▪ Personalisation
Worth testing standard features? ▪ TM, Concordance, Search & Replace, Comments, etc.
CIUTI 2018
Edinburgh, 31 May 2018
Thank you! Current prototype available to test at https://kanjingo.adaptcentre.ie/
References Ciobanu, Dragoș. 2016. “Automatic Speech Recognition in the professional translation process.” Translation Spaces 5(1): 124 –144. O’Brien, Sharon, and Joss Moorkens. 2014. “Towards Intelligent Post-Editing Interfaces”. In Proceedings of the XXth FIT World Congress, Berlin, Germany, 04-06 August, W. Baur, B. Eichner, S. Kalina, N. Kessler, F. Mayer and J. Orsted (eds.) 131– 137. Wigdor, Daniel and Dennis Wixon. 2011. Brave NUI World – Designing Natural User Interfaces for Touch and Gesture. [S.l.] Elsevier monographs Zapata, Julián, and Elizabeth C. Saint. 2017. “Traduire à voix haute : la traduction dictée interactive comme solution ergonomique”. inTRAlinea 19. 31
