D2.2.1: Open desktop libraries enabling built-in

INFORMATION AND COMMUNICATION TECHNOLOGIES (ICT)

AEGIS Collaborative Project 224348

Open desktop libraries enabling built-in accessibility (update for M46) Deliverable No.

D2.2.1(c)

SubProject No. SP2

SubProject Title

Open Accessible Desktop

Workpackage No.

WP2.2

Workpackage Title

Open desktop support libraries to enable built in accessibility

Activity No.

A2.2.1, A2.2.2,

Activity Title

A2.2.1 Open magnification framework; A2.2.2 Feasibility study on open, cross-platform support of desktop accessibility framework on Windows; A2.2.3 Open Text-to-Speech multilingual functionality; A2.2.4 Built-in support for accessibility in a crossplatform Office Suite; A2.2.5 Open interface to assistive technologies

A2.2.3, A2.2.4,

A2.2.5

Authors

Jan Richards (IDRC), Jerry Dimitriou, Gianna Tsakou, Efthimiadis Nikos (SILO), Christophe Strobbe (KUL), Peter Korn (Oracle), Annika Brännström (FPD), Mats Lundälv (SU-DART)

Status:

Final

Dissemination Level

Public

File Name:

AEGIS D2.2.1c_final_opendesktoplibraries

Project start date and duration September 2008, 48 months


PU

Grant Agreement #224348

Table of Contents Version History table............................................................................................................................6 Abbreviations.......................................................................................................................................7 D2.2.1(c) Executive Summary.............................................................................................................9 Executive Summary............................................................................................................................14 1 Introduction......................................................................................................................................1 2 A2.2.1 Open magnification framework ...........................................................................................3 2.1 Software Availability (Part of GNOME Shell).........................................................................3 2.2 Common Magnifier Framework for GNOME..........................................................................4 2.2.1 Inter-process Communication API (D-Bus)......................................................................4 2.2.2 Magnifier interface:...........................................................................................................4 2.2.3 ZoomRegion interface:.....................................................................................................4 2.3 Persistent User Preferences (GSettings)...................................................................................5 2.3.1 Magnifier Preferences.......................................................................................................5 2.3.2 ZoomRegion Preferences..................................................................................................5 2.4 GNOME Shell Magnifier..........................................................................................................5 2.4.1 Implemented Features.......................................................................................................6 2.4.2 Magnifier Preferences Dialog...........................................................................................8 2.4.3 Status of Communication with Orca:..............................................................................11 2.4.4 Testing (Pilots)................................................................................................................11 2.5 Overview of the application ...................................................................................................11 2.6 Relationship to the OAF.........................................................................................................18 2.7 Planned Future Developments................................................................................................19 2.7.1 Extend GNOME Shell Magnifier Feature Set................................................................19 3 A2.2.2 Open, cross-platform support of desktop accessibility framework on Windows ..............20 3.1 Introduction.............................................................................................................................20 3.2 Feasibility Study.....................................................................................................................20 3.3 Conclusions.............................................................................................................................21 4 A2.2.3 Open Text-to-Speech multilingual functionality.................................................................22 4.1 Introduction.............................................................................................................................22 4.2 Current State of the art – Open-source TTS Engines.............................................................22 4.2.1 Available TTS Engine Types..........................................................................................22 4.2.2 Formant Synthesis...........................................................................................................23 4.2.3 Concatenative Synthesis.................................................................................................23 4.2.4 Available Open-source TTS Engines..............................................................................23 4.2.4.1 Festival....................................................................................................................23 4.2.4.1.1.1 FestVox.....................................................................................................24 4.2.4.2 Flite..........................................................................................................................24 4.2.4.3 GnuSpeech...............................................................................................................24 4.2.4.4 FreeTTS...................................................................................................................24 4.2.4.5 eSpeak TTS engine .................................................................................................25 4.3 AEGIS technology selection - eSpeak....................................................................................25 4.3.1 Pre-AEGIS State – Language Quality............................................................................26 4.4 Language Enhancement Methodology...................................................................................26 4.5 Current status of the prototype................................................................................................27 4.6 Testing (Pilots)........................................................................................................................27 4.6.1 First Phase of Pilot Trials................................................................................................27 4.6.2 Second Phase of Pilot Trials............................................................................................28 October 2012

Oracle

ii


PU


4.6.3 Planned Future Developments........................................................................................28 4.6.3.1 Further Language enhancements.............................................................................28 4.6.3.2 Language Enhancement Guide................................................................................28 4.7 Overview of the application....................................................................................................29 4.8 Relationship to the OAF .....................................................................................................36 5 A2.2.3 CCF symbol support extension for Open Office Writer.....................................................37 5.1 Background.............................................................................................................................37 5.2 Technology Selection..............................................................................................................38 5.3 Current status..........................................................................................................................38 5.3.1 Insertion and Switching between symbols/concepts.......................................................39 5.3.2 Display mode..................................................................................................................40 5.3.3 Language and Symbol support........................................................................................41 5.3.4 Testing (Pilots)................................................................................................................42 5.4 Overview of the application....................................................................................................42 5.5 Relationship to the OAF.........................................................................................................48 6 A2.2.4 Built-in support for accessibility in a cross-platform office suite.......................................49 6.1 odt2daisy: "Save as DAISY"..................................................................................................49 6.1.1 Background.....................................................................................................................49 6.1.2 Technology Selection......................................................................................................49 6.1.3 Save as DAISY Functions...............................................................................................50 6.1.4 Requirements for Source Documents..............................................................................50 6.1.5 Current Status of odt2daisy.............................................................................................51 6.1.5.1 Changes since D2.2.1b............................................................................................53 6.1.6 Testing (Pilots)................................................................................................................54 6.1.7 Overview of the application............................................................................................54 6.1.8 Relationship to the OAF.................................................................................................60 6.2 odt2braille: "Export to Braille"...............................................................................................61 6.2.1 Background.....................................................................................................................61 6.2.2 Technology Selection......................................................................................................61 6.2.3 Save as Braille Functions................................................................................................62 6.2.4 Current Status of odt2braille...........................................................................................62 6.2.5 Testing (Pilots)................................................................................................................64 6.2.6 Overview of the application............................................................................................64 6.2.7 Relationship to the OAF.................................................................................................70 6.3 OpenOffice.org Accessibility Checker...................................................................................70 6.3.1 Background.....................................................................................................................70 6.3.2 Technology Selection......................................................................................................70 6.3.3 Accessibility Checker Requirements...............................................................................70 6.3.4 Current Status of the Accessibility Checker....................................................................71 6.3.4.1 Availability and Licence..........................................................................................71 6.3.4.2 Coverage of Accessibility Issues.............................................................................71 6.3.4.3 Metadata: EARL......................................................................................................77 6.3.5 Testing (Pilots)................................................................................................................77 6.3.6 Overview of the application............................................................................................78 6.3.7 Relationship to the OAF.................................................................................................82 6.4 Compatibility of the Extensions with LibreOffice..................................................................83 6.4.1 Licensing.........................................................................................................................83 6.4.2 API Compatibility...........................................................................................................84 October 2012

Oracle

iii


PU


6.5 Relationship of odt2daisy, odt2braille and the ODF Accessibility Checker to the OAF........86 7 A2.2.5 Open interface to assistive technologies.............................................................................87 7.1 Introduction.............................................................................................................................87 7.2 Evaluating options, choosing D-Bus......................................................................................87 7.3 What must change to support D-Bus......................................................................................87 7.4 Dealing with D-Bus challenges..............................................................................................89 7.5 Current status..........................................................................................................................90 7.6 ÆGIS contributions................................................................................................................90 7.7 Overview of the application....................................................................................................91 7.8 Relationship to the Open Accessibility Framework (OAF)....................................................95 7.9 References...............................................................................................................................96 8 Conclusion......................................................................................................................................97 References..........................................................................................................................................99

Index of Ilustrations Illustration 1: Number of Downloads per OOo plugin, Nov 2009 - Aug 2010..................................11 Illustration 2: Number of Downloads per OOo plugin, Sep 2010 - Aug 2011...................................12 Illustration 3: Number of Downloads per OOo plugin, Sep 2011 - Aug 2012...................................13 Illustration 4: Screen shot of GNOME 3.6's Zoom Options dialog with Magnifier preferences are set ..............................................................................................................................................................9 Illustration 5: Screen shot of GNOME 3.6's Zoom Options dialog with Crosshairs preferences are set..........................................................................................................................................................9 Illustration 6: Screen shot of GNOME 3.6's Zoom Options dialog with Color Effects preferences are set........................................................................................................................................................10 Illustration 7: Screenshot of GNOME Shell 3.6 Beta with Lightness Inversion and crosshairs turned ON......................................................................................................................................................10 Illustration 8: Current level of achievement of defined requirements for GNOME Shell Magnifier 17 Illustration 9: OAF view of Desktop, as seen through the 6 steps of accessibility (3 for creation, 3 for use). Text in bold, italicized red indicates where A2.2.1 delivers into the OAF..........................18 Illustration 10: OAF view of Desktop, as seen through the 6 steps of accessibility (3 for creation, 3 for use). Text in bold, italicized red indicates where the eSpeak TTS in A2.2.3 delivers into the OAF....................................................................................................................................................36 Illustration 11: CCF-SymbolServer window floating on top of the Writer document window, and optionally following the text cursor – symbols (here Blissymbols) are inserted above text in the 2 line Ruby Annotation format. Alternative symbol representations may be selected from the CCFSymbolServer display, or by using the shortcut key command..........................................................39 Illustration 12: LibreOffice Writer with CCF-SymbolWriter. SAW 6 is serving as symbol supported alternative input of text, and together these applications are constituting a full AAC system...........40 Illustration 13: No symbol insertion in text: The CCF-SymbolServer (floating on top of the text window) is displaying looked-up concepts and symbols as words are written, and/or as the text cursor is moved within the text. ........................................................................................................41 Illustration 14: Same multilingual document as in Ill. 13, but here a paragraph with Swedish text is processed, as opposed to the English paragraph above......................................................................41 Illustration 15: CCF-SymbolWriter Options dialog...........................................................................42 Illustration 16: OAF view of Desktop, as seen through the 6 steps of accessibility (3 for creation, 3 for use). Text in bold, italicized red indicates where the CCF support in A2.2.4 delivers into the OAF....................................................................................................................................................48 Illustration 17: OAF view of Desktop, as seen through the 6 steps of accessibility (3 for creation, 3 October 2012

Oracle

iv


PU


for use). Text in bold, italicized red indicates where A2.2.4 delivers into the OAF..........................86 Illustration 18: GNOME accessibility framework use of CORBA - source: http://live.gnome.org/Accessibility/BonoboDeprecation...................................................................88 Illustration 19: GNOME accessibility framework use of D-Bus - source: http://live.gnome.org/Accessibility/BonoboDeprecation...................................................................89 Illustration 20: OAF view of Desktop, as seen through the 6 steps of accessibility (3 for creation, 3 for use). Text in bold, italicized red indicates where A2.2.5 delivers into the OAF..........................96

October 2012

Oracle

v


PU


Version History table Version no.

Dates and comments

1

30/06/2012: Final version of D2.2.1(c)

2

2012/10/23 Final version with Peer review comments incorporated

3

2012/10/31 Additional changes and updates made based on peer review; accessibility & spelling check

October 2012

Oracle

vi


PU


Abbreviations Abbreviation ANSI ASCII API AT-SPI (ATSPI) ATK BANA CCF CORBA D-BUS DCOP DAISY GIF GNU GTK HMM HTML ICEB IPC JNI JPEG KDE LDTP2 LGPL MathML MSAA NISO OLPC ODF ODT PDF PEF PNG SSML SoA TTS UEB October 2012

Explanation American National Standards Institute American Standard Code for Information Interchange (character encoding scheme) Application Programming Interface Assistive Technology Service Provider Interface Accessibility ToolKit Braille Authority of North America Concept Coding Framework Common Object Request Broker Architecture Desktop Bus Desktop COmmunication Protocol Digital Accessible Information System, a standard for digital talking books Graphics Interchange Format (bitmapped image format) GNU Project, a free software, mass collaboration project (recursive algorithm: "GNU’s Not Unix") GIMP ToolKit Hidden Markov Model HyperText Markup Language International Council on English Braille Inter-Process Communication Java Native Interfaces bitmapped image format (acronym originally meant "Joint Photographic Experts Group") K Desktop Environment Linux Desktop (GUI Application) Testing Project (GNU LDTP) (GNU) Lesser General Public License Mathematical Markup Language, an XML language for mathematical notation (W3C Recommendation) Microsoft Active Accessibility National Information Standards Organization One Laptop per Child OpenDocument Format for Office Applications (native format of OpenOffice.org) OpenDocument Text, the word processing format in ODF Portable Document Format, a standard for document exchange Portable Embosser Format, XML-based data format for representing braille books, by the DAISY Consortium Portable Network Graphics (bitmapped image format) Speech Synthesis Markup Language State of the Art Text to Speech Unified English Braille Oracle

vii


Abbreviation W3C WCAG WAV XML

October 2012

PU


Explanation World Wide Web Consortium, de facto standardisation body for the Web Web Content Accessibility Guidelines, a W3C Recommendation Waveform Audio File Format Extensible Markup Language, metalanguage for markup languages

Oracle

viii


PU


D2.2.1(c) Executive Summary Majority of the work under WP2.2 “Open Desktop Support Libraries to Built in Accessibility under AEGIS project was done by November 2011 (the date of D2.2.1b); but we decided to amend an additional version of this deliverable for any additional improvements that we anticipated might happen in the final year of the project. In the last year of the project one major improvements happened: • GNOME 3.6 is now the latest stable version of GNOME Shell, it supports “always on accessibility”. • Further adoption and use GNOME Shell Magnifier with the new end-user interface for adjusting colour/contrast settings (noted in Section 2.4 “GNOME Shell Magnifier” and Section 2.7 “Planned Future Developments” below). The text of D2.2.1b was updated to reflect this improvement. Please see section 2 below. In addition to the major improvements above, several other updates were made in the this deliverable: • Section 2 “A2.2.1 Open magnification framework “, specifically Chapter 2.4.4 “Testing (Pilots)“ has been added to report on the User Testing of GNOME. • Section 5 “A2.2.3 CCF symbol support extension for Open Office Writer, specifically Section 5.3 “Current status” has been updated to reflect the latest development work on CCF LibreOffice/OpenOffice extension according to the evaluation of the users' feedback. Also a Chapter 5.3.4 “Testing (Pilots)“ has been added to report on the User Testing of OOo Writer. • The section 6 on odt2daisy and odt2braille, has been updated as follows: • Section 6.1 “odt2daisy: "Save as DAISY"” specifically Section 6.1.5.1 “Changes since D2.2.1b” has been added to reflect the latest changes since D2.2.1b in A2.2.4. • Section 6.1.7 “Overview of the application” was updated according to the evaluation of the users' feedback. Specifically Requirements in the “Table 8: Current level of achievement of defined requirements for odt2daisy” were modified, (Changes made in following requirements: Maintainability, Changeability, Maturity and F-OODB-1: TS5). • Section 6.2 “OpenOffice.org Accessibility Checker” was modified to respect the recent changes. Especially following sections have been updated: • Section 6.2.6 “Overview of the application”: “Table 9: Current level of achievement of defined requirements: UR-V4, UR-C6, Changeability, FOOBP-1: TS4. • Section 6.3 “OpenOffice.org Accessibility Checker” has been updated in following sections: • Section 6.3.6 “Overview of the application”, “Table 11: Current level of achievement of defined requirements for the ODF Accessibility Checker”; Requirement: UR-C6. • Another updates were made in Section 6.4 “Compatibility of the Extensions with LibreOffice”, especially Section 6.4.2 “API Compatibility” with LibreOffice since the releases of Apache OpenOffice.org were not mentioned in the previous version. October 2012

Oracle

ix


PU


In addition to the above also the number of downloads of OOo plugins were as follows: – Odt2Daisy: – November 2009 – August 2010: 3397 – September 2010 – August 2011: 3217 – September 2011 – August 2012: 3301 – Odt2Braille: – November 2009 – August 2010: 310 – September 2010 – August 2011: 4340 – September 2011 – August 2012: 4092 – AccessODF: – November 2009 – August 2010: 0 – September 2010 – August 2011: 0 – September 2011 – August 2012: 1604 Number of Downloads of OOo plugins by month since the first release until the end of the project (August 2012): November 2009 – August 2010 per plugin: Month

odt2daisy

odt2Braille

Nov 2009

367

Dec 2009

308

Jan 2010

325

Feb 2010

259

Mar 2010

250

Apr 2010

515

May 2010

419

Jun 2010

382

Jul 2010

255

17

Aug 2010

317

293

Total

3397

310

AccessODF

0

Table 1: Monthly downloads per OOo plugins, Nov 2009 - Aug 2010

October 2012

Oracle

x


PU


Number of Downloads Nov 09 - Aug 10

Number of Downloads

600 500 400

odt2daisy odt2braille AccessODF

300 200 100 0 Dec 2009 Feb 2010 Apr 2010 Jun 2010 Aug 2010 Nov 2009 Jan 2010 Mar 2010 May 2010 Jul 2010 Month

Illustration 1: Number of Downloads per OOo plugin, Nov 2009 - Aug 2010 September 2010 – August 2011 per plugin: Month odt2daisy

odt2braille

Sep 2010

286

145

Oct 2010

280

81

Nov 2010

270

90

Dec 2010

270

458

Jan 2011

415

512

Feb 2011

311

269

Mar 2011

320

364

Apr 2011

244

972

May 2011

246

485

Jun 2011

194

277

Jul 2011

163

319

Aug 2011

218

368

Total

3217

4340

AccessODF

0

Table 2: Monthly downloads per OOo plugins, Sep 2010 - Aug 2011

October 2012

Oracle

xi


PU


Number of Downloads Sep 10 - Aug 11 1200 Number of Downloads

1000 800


600 400 200 0 Oct 2010 Dec 2010 Feb 2011 Apr 2011 Jun 2011 Aug 2011 Sep 2010 Nov 2010 Jan 2011 Mar 2011 May 2011 Jul 2011 Month

Illustration 2: Number of Downloads per OOo plugin, Sep 2010 - Aug 2011 September 2011 – August 2012 per plugin: Month odt2daisy

odt2braille

Sep 2011

184

300

Oct 2011

166

307

Nov 2011

280

402

198

Dec 2011

264

425

152

Jan 2012

274

420

122

Feb 2012

324

541

171

Mar 2012

383

469

104

Apr 2012

299

271

178

May 2012

381

355

123

Jun 2012

267

210

128

Jul 2012

272

199

175

Aug 2012

207

193

253

Total

3301

4092

1604

AccessODF

Table 3: Monthly downloads per OOo plugins, Sep 2011 - Aug 2012

October 2012

Oracle

xii


PU


Number of downloads Sep 11 - Aug 12 600 Number of Downloads

500 400


300 200 100 0 Oct 2011 Dec 2011 Feb 2012 Apr 2012 June 2012 Aug 2012 Sep 2011 Nov 2011 Jan 2012 Mar 2012 May 2012 Jul 2012 Month

Illustration 3: Number of Downloads per OOo plugin, Sep 2011 - Aug 2012

October 2012

Oracle

xiii


PU


Executive Summary This report summarizes the technical progress made during the first 36 months of developing necessary open assistive technology support libraries to allow assistive technologies on the open desktop to reach parity, and exceed the functionality of the 2 nd generation assistive technologies on the proprietary desktop and its technical difficulties and issues. The document presents technical progress in following areas under WP2.2 “Open Desktop Support Libraries To Enable Built In Accessibility": – Open Magnification Functionality – Feasibility study for an open, cross-platform support of desktop accessibility framework on Windows – Open Text-to-Speech multilingual functionality – CCF symbol support for OpenOffice.org Writer documents – Built-in support for accessibility in a cross-platform office suite – Open interface to assistive technologies With a few exceptions – the feasibility study for the cross-platform support of a desktop accessibility framework on Windows, the OpenOffice.org accessibility checker, and the conceptcoding framework plug-in to OpenOffice.org (a portion of the built-in support for accessibility in a cross-platform Office Suite) – all of the work done in AEGIS in this work package by Month 30 has been delivered to the respective / appropriate open source communities, and in fact is already being used. This is described in chapter 1 below, but in summary: • The Open Magnification Functionality (A2.2.1) is already part of GNOME Shell (and the GNOME open source code responsibility) beginning in GNOME 3.0 (released in April, 2011), where it was picked up by various GNU/Linux distros • For the Open Text-to-Speech multi-lingual functionality work (A2.2.3), an incremental set of improvements to the eSpeak text-to-speech engine for Italian, (Belgian) Dutch, Spanish, Swedish, and Greek was added together with the initial set of improvements reported in 2.2.1a . Those improvements have been incorporated into eSpeak (and its open source code repository), and are already in use. Multi-lingual functionality is part of odt2daisy, where the language encoding of passages of text in OpenDocument format text files is used to select the correct text-to-speech engine for proper pronunciation of multi-lingual documents • Regarding built-in support for accessibility in a cross-platform office suite (A2.2.4), DAISY book generation – odt2daisy – and Braille transcription and production – odt2braille – are already in use by organizations for the blind who are producing accessible content with these AEGIS deliverables in combination with the OpenOffice.org office suite. Both are available for download from the OpenOffice.org extensions website, and source code is published for both on SourceForge. The AEGIS prototype implementation of the open interface to assistive technology (A2.2.5) is part of the GNOME open source code repository, where GNOME community members are continuing to polish this prototype, toward inclusion in GNOME 3 as part of its release in April, 2011.

October 2012

Oracle

xiv


PU


1 Introduction The Open Accessibility Framework sets forth the six steps toward building an accessible ICT world. WP2.2 comprises a collection of Activities that address several of these six steps, primarily – though not exclusively – on the Open Desktop (which we define as the GNOME desktop running on a base operating system of GNU/Linux or UNIX). The GNOME desktop already contains a fairly well developed accessibility framework. In fact, that framework is the primary source or basis from which the OAF was developed. Therefore, this Work Package is focused on two key tasks: 1. filling in some of the missing pieces (to enable assistive technologies on the open desktop to reach parity, and exceed the functionality of the 2nd generation assistive technologies on the proprietary desktop); 2. addressing authoring tool accessibility issues, to help authors create accessible content. These two key tasks are in service of the main objectives of this workpackage: • To develop the necessary open assistive technology support libraries to allow assistive technologies on the open desktop to reach parity, and exceed the functionality of the 2nd generation assistive technologies on the proprietary desktop. • To extend to the proprietary Windows desktop the 3rd generation access approach of the Open Desktop, and thereby enable the development and use of a non-Internet based, accessible cross-platform application development. • To build into a major cross-platform office suite support for the creation and production of accessible content – in large print, Braille, and DAISY book formats. • To support the reading and Brailling of multilingual documents for the blind; pronounced correctly in the appropriate language & locale, and likewise Brailled correctly. The missing pieces on the open GNOME desktop are: • An open magnification framework, which will be used both to provide built-in screen magnification functionality for users with low-vision, but also form the basis for future work on 3rd generation accessibility magnification techniques, which allow for context-sensitive magnification. This is A2.2.1. • Improved open text-to-speech, supporting a variety of assistive technology use cases – including screen reading and DAISY book generation – in a broad range of European languages. This is A2.2.3. Work since D2.2.1a includes improvements for Spanish, Italian and Belgian Dutch. • A re-write of the “plumbing” and inter-process communication for the open accessibility API, allowing it to be used broadly across a wide range of GNU/Linux and UNIX environments, including those with significant memory and processor constraints (such as the One Laptop Per Child XO, and the various Linux Mobile implementations like MeeGo). Further, because the GNOME 3 desktop has depreciated CORBA, this is also necessary to support accessibility on GNOME 3. For authoring tool accessibility support, the work is all contained in A2.2.4, with the development of four open plug-ins to the OpenOffice.org office suite: odt2braille, odt2daisy, the OpenOffice.org accessibility checker, and authoring with Concept Coding Framework support. Work since D2.2.1a October 2012

Oracle

1


PU


includes numerous improvements in odt2daisy, odt2braille and the Accessibility Checker, as well as a substantially updated version of the CCF-SymbolWriter extension and the underpinning CCFSymbolServer. It also includes evaluation of the compatibility of the extensions with LibreOffice, a fork of OpenOffice.org that came into existence in September 2010. A last piece of work was undertaken in this Work Package: a feasibility study on implementing the IAccessible2 interface for the JavaSE platform – to determine whether an external bridge could contain such an implementation (as was done for the existing Java Access Bridge for Windows, which implemented a unique-to-Java accessibility API). This is in support of a pressing accessibility challenge: the “platform-on-a-platform” problem: how to enable cross-platform environments like Java (and Flash and Silverlight) be accessible on all of the platforms they run on. While the web and ARIA are also an example of this general challenge – and are the subject of SP3 – they don't encompass many of the unique challenges of runtime environments (like Java, Flash, Silverlight), and so exploring the problem in Java became an important part of AEGIS.

October 2012

Oracle

2


PU


2 A2.2.1 Open magnification framework Screen magnification (and enhancement) is an assistive technology used primarily by persons with low vision. The term “enhancement” refers to the fact that screen magnification often involves modifications to the visual display that go beyond simply increasing the size of information. For example, the colour of information on the screen may be adjusted or the focus and/or mouse pointer may be made more prominent, etc. Prior to the start of AEGIS development, magnification on the GNOME 2 Desktop was provided by a service called GNOMEMag [GnomeMag]. GNOMEMag is a CORBA service, without its own user interface or stored settings. While any desktop application could call upon GNOMEMag to magnify/enhance the desktop, in practice the Orca screen reader was the only consumer of the GNOMEMag service and Orca stored its own magnification-related settings in a custom format. This meant that low vision users, who may not have wished to install a screen reader, needed to install Orca with speech turned off in order to magnify the screen. The move towards GNOME 3 [Gnome3] presented both challenges and opportunities to this arrangement. GNOME 3 deprecates CORBA in favour of D-Bus and GNOME 3 tightly integrates a Window Manager called GNOME Shell [GnomeShell] to replace GNOME Panel. While it would be possible to update GNOME Mag as it currently is to communicate with Orca via D-Bus rather than CORBA, a more promising approach is to build magnifier functionality directly into GNOME Shell. This has some advantages: 

Magnification can become “part” of GNOME Shell, meaning that it is present without downloading and installing additional software.



GNOME Shell includes a compositing manager and magnification can be thought of as a kind of compositing. This also opens the door to other graphics manipulations that may benefit users with low vision.



Magnification settings can be placed in a common place and magnifier functions can still communicated by a common API (i.e., the Common Magnifier Framework) to other applications (e.g., Orca) that may wish to use them.

2.1 Software Availability (Part of GNOME Shell) The latest stable version of GNOME Shell (3.4) includes the basic magnifier functionality (i.e., magnification, mouse tracking and crosshairs) and programmatic settings as documented below, as well as GNOME Control Center user interface for setting preferences. The planned additional functionality and programmatic settings for lightness, brightness and contrast effects are also planned for GNOME 3.6. Since D2.2.1a: •

the colour controls (“Lightness” inversion, Brightness, Contrast) have been improved,

•

background work has been done to keep the magnifier harmonised with GNOME Shell and Orca. However, as of GNOME 3.4, the system is in transition so that by GNOME 3.8, the Magnifier will be controlled independently of Orca.

October 2012

Oracle

3


PU


Features

Functionality

Programmatic Settings Preferences UI

Magnification, mouse tracking, position, and crosshairs

GNOME 3.0+

GNOME 3.0+

GNOME 3.4+

Lightness, brightness, and contrast effects

GNOME 3.4+

GNOME 3.4+

GNOME 3.6+

Table 4: Features planned in the next GNOME releases

2.2 Common Magnifier Framework for GNOME The Common Magnifier Framework [GnomeMagnifierFramework] defines two magnifier-related mechanisms: an Inter-process Communication API (using D-Bus) and a set of persistent magnifier preference settings (using GSettings). The framework allows multiple magnifiers and consumer of magnification services (e.g., GNOMEMag, GNOME Shell Magnifier, and Orca) to interoperate and maintain consistency of settings.

2.2.1

Inter-process Communication API (D-Bus)

Certain magnifier properties and functions are best managed using inter-process communication, wherein some client invokes magnifier functions. States that are transient and do not involve a user preference are best handled this way. An example is the region of the screen that is currently magnified. That region will vary, for example, as the mouse moves, or as keyboard focus changes. Furthermore, when the user finishes for the day, and logs out, it is very unlikely that they expect to come back to the exact same magnified content when they next reconnect. In short, this situation is not properly characterized as a persistent property of the magnifier. Note that the magnifier is a manager of "zoom regions". Thus, the D-Bus provides an interface for both the magnifier itself, and for the zoom region construct. The functions and properties available for each are outlined below.

2.2.2 •

Magnifier interface: Turning the magnifier on/off. Note: In future, this D-Bus service call may be removed, and handled solely by the user preferences sub-system (GSettings).

• Showing/hiding • Disposing

2.2.3

system mouse cursor:

of the magnifier:

ZoomRegion interface:

A ZoomRegion is the part of the magnifier that has actual screen presence. A ZoomRegion appears as a rectangle on the screen and contains an enhanced view of some region of the desktop. The following are service functions that a client can call to manipulate individual ZoomRegions: • Change • Define

October 2012

the contents of the magnified view.

an update region: Oracle

4

AEGIS D2.2.1c_final_opendesktoplibraries • Move and/or • Dispose of

PU


resize the ZoomRegion view:

the ZoomRegion:

2.3 Persistent User Preferences (GSettings) Some states and functions of the magnifier reflect user preferences, and, as such, persist between invocations of the magnifier. GSettings is used for managing persistent settings in terms of storing, retrieving, modifying, and reacting to preference changes. The following states and properties were part of the magnifier service layer, but, upon reflection, are now considered preferences. They are listed in quasi-GSettings XML syntax.

2.3.1

Magnifier Preferences

• Magnifier • Whether

on/off:

to use a different set of mouse cursor images:

• Enhance the

mouse cursor image in various ways (size, zoom, colour, hotspot).

•

Cross hairs (show/hide, thickness, length, whether to clip around the mouse pointer, colour, and opacity):

•

Magnification factor: Magnification factor, as a preference, reflects the preferred magnification factor of the first ZoomRegion only. It is technically a property of a ZoomRegion, not the Magnifier as a whole.

2.3.2

ZoomRegion Preferences

•

Magnification factor

•

Mouse tracking mode

•

Screen position (full-screen, top-half, bottom-half, right-half, and left-half)

•

Invert-brightness

•

Increasing or decreasing contrast

•

Increasing or decreasing brightness

•

Lens-mode on/off. With lens-mode, the zoom region moves with the mouse to simulate a hand-held magnifying lens.

•

Scroll-at-edges on/off. When scrolling the zoom region's contents due to mouse movements, and the mouse is near an edge of the desktop (e.g., the top), this setting defines whether the contents continue to scroll beyond that edge.

2.4 GNOME Shell Magnifier The Common Magnifier Framework for GNOME (above) describes the basic inter-process communication and setting storage mechanisms for magnifier-related preferences. These mechanisms are implemented in GNOME Shell by the built-in GNOME Shell Magnifier.

October 2012

Oracle

5


2.4.1

PU


Implemented Features

Note that matching features existed in GNOME 2 provided by the GNOME magnification CORBA service. What is new in the current implementation is that (1) D-Bus is used instead of CORBA, (2) a user settings approach (GSettings) is used instead of a service approach, and (3) the magnifier is based on a object model of the desktop (Clutter) instead of simply modifying the display properties of a frame buffer. Further with respect to (2), the Common Magnifier Framework divides functionality clearly between what is a preference, and what is a service call.  Show/hide magnifier  Toggles the magnifier  Show/hide is a user preference, handled by GConf and available through the magnifier's D-Bus interface





GSettings: show-magnifier (boolean value)



D-Bus: org.gnome.Magnifier.setActive(boolean)

Mouse tracking modes: centered, push, proportional, and none. 

Centered mode: The magnified mouse image is displayed at the center of the magnifier, and the magnified contents are shifted within the magnified view. The point directly under the actual mouse pointer is placed under the magnified mouse cursor image. The visual effect is that the magnified contents scroll under the magnified mouse image as the actual mouse is moved around the screen.



Push mode: The magnified contents generally do not move, and the magnified mouse moves as the user moves the actual mouse. However, when the magnified pointer pushes against an edge of the magnified view, the magnified contents are scrolled into view. For example, if the user is moving the mouse towards the right edge of the magnifier view, then the magnified contents are shifted leftwards.



Proportional mode: Similar to centered in that the magnified contents scroll as the mouse is moved. However, the position of the magnified mouse image is not fixed at the center of the magnified view. Instead, its position is representative of where the actual mouse pointer is on the screen. If the actual mouse is in the bottom left quadrant, and, say, half way between the center and bottom of the screen, then the magnified cursor image is in the bottom left quadrant of the magnified view, half way towards the bottom of the magnified view. In this way the position of the magnified mouse cursor provides a clue to the user as to where the real mouse is on the desktop.



None: The contents of the magnified view are fixed, and do not move as the mouse moves. If the actual mouse image is over the content displayed within the magnified view, the magnified mouse is displayed in the corresponding location.



Mouse tracking mode is a user preference, and handled by GSettings: 

mouse_tracking (enum) 

October 2012

nick 'none' (value = 0) Oracle

6






nick 'centered' (value = 1)



nick 'proportional' (value = 2)



nick 'push' (value = 3)


Magnified view optionally positioned to the:       

top half of the screen, bottom half, left half, right half, full screen, or a smaller view scaled to the size of the screen in the bottom left hand corner. View position is a user preference, handled via GConf: 





PU

GSettings: screen-position (enum) 

nick 'full-screen' (value = 1)



nick 'top-half' (value = 2)



nick 'bottom-half' (value = 3)



nick 'left-half' (value = 4)



nick 'right-half' (value = 5)

DBus: org.gnome.Magnifier.ZoomRegion.moveResize(array [x, y, width, height])

System pointer: Hide the actual mouse cursor when it overlaps the magnified view; reveal when it is over non-magnified parts of the desktop.  Always hide it when in full screen mode.  Acquire the sprite image for the system pointer and display a magnified version within the magnified view.  Magnified cursor image mimics changes of the system pointer image. 



Magnified view optionally follows position of the actual mouse and appears as a movable lens. 

Lens mode is a user preference, and handled by GSettings: 



lens-mode (boolean value)

Crosshairs 

October 2012

The mouse cursor, even when magnified, may be difficult to see for some users. A useful addition in this case is a cross hair that is centred on the magnified mouse Oracle

7


PU


image, moves with it, and whose vertical and horizontal "hairs" extend to the edges of the magnifier view.  In terms of preferences, users can:  enable/disable crosshairs:





GSettings: show-cross-hairs (boolean value)



DBus: org.gnome.Magnifier.setCrosswireSize(integer thickness)

Modify the thickness, colour, length, opacity, and clipping of the crosshairs. "Clipping" refers to whether there is a transparent rectangle at the intersection of the vertical and horizontal "hairs".  GSettings:







cross-hairs-thickness (integer value)



cross-hairs-color (string value, e.g., "red" or "#ff00ffff")



cross-hairs-opacity (integer value [0,255])



cross-hairs-length (integer value)



cross-hairs-clip (boolean value)

DBus: 

org.gnome.Magnifier.setCrosswireSize(integer size)



org.gnome.Magnifier.setCrosswireColor(unsigned colour)



org.gnome.Magnifier.setCrosswireLength(integer length)



org.gnome.Magnifier.setCrosswireClip(boolean)

integer

Inverse video, brightness and contrast.  In addition to magnifying the contents, some users prefer higher contrast of the magnified content, varying brightness levels, and/or "inverse video".  Inverse video is implemented as brightness inversion. Using the HSL colour model, this amounts to inverting lightness (L), while leaving hue (H) and saturation (S) unchanged. Lightness ranges from 0% (black) to 100% (white). The effect is that darker colours become lighter, and vice versa; white and black are interchanged; and "pure colours" remain the same. Pure colours are ones whose lightness is 50%.  Overall brightness changes are implemented as changing the contribution of each of the red, green, and blue channels. Brightness ranges from 0% (black) to (100%) white, where 50% represents no change from the standard video output.  Contrast settings are similarly implemented as removing from or adding to each of the red, green, and blue channels. Contrast ranges from 0% (minimal contrast) to 100% (maximum contrast), where 50% represents no change from the standard contrast. 

October 2012

The work is tracked in GNOME Shell's bugzilla data base [GnomeBug639851] Oracle

8

AEGIS D2.2.1c_final_opendesktoplibraries 



PU


GSettings: 

invert-brightness (boolean value)



brightness (hex string value, e.g., "#7f7f7f")

contrast (hex string value, e.g. "#7f7f7f")

2.4.2

Magnifier Preferences Dialog

The GNOME Shell Magnifier settings have been part of GNOME Control Center since GNOME 3.4 (Illustration 4, 5, 6) and have been updated to include the colour enhancement settings for GNOME 3.6 (Illustration 27).

Illustration 4: Screen shot of GNOME 3.6's Zoom Options dialog with Magnifier preferences are set

Illustration 5: Screen shot of GNOME 3.6's Zoom Options dialog with Crosshairs preferences are set

October 2012

Oracle

9


PU


Illustration 6: Screen shot of GNOME 3.6's Zoom Options dialog with Color Effects preferences are set

Illustration 7: Screenshot of GNOME Shell 3.6 Beta with Lightness Inversion and crosshairs turned ON.

October 2012

Oracle

10


2.4.3

PU


Status of Communication with Orca:

Orca implements communication with GNOME Shell Magnifier via the Common Magnifier Framework (introduced above).

2.4.4

Testing (Pilots)

The GNOME Shell Magnifier has been tested with users in the first two pilot evaluation phases of AEGIS. In response to the requests from the pilot tests and other user feedback, the GNOME Shell Magnifier is now a mature and fully integrated part of the GNOME 3 environment, including added features such as more conveniently available settings, Cross Hair focus support and Inverse video, brightness and contrast modes. More details about the Pilot testing will be found in the D1.5.2 Consolidated evaluation report.

2.5 Overview of the application 1. Title of product: GNOME Shell Magnifier 2. Development team: IDRC 3. Relevant WP/Activity development/integration tool place: 2.2.1 4. Purpose: Enabling built-in screen magnification and screen enhancement of the GNOME Desktop. 5. For which user groups and what are the expected benefits for each of them: •

Partly sighted users: The benefit is usability of the GNOME desktop when GNOME Shell is running.

6. Innovations (in comparison to SoA): Built-in magnification and screen enhancement (with no need to buy/download a separate program) that goes well beyond that found in other platforms such as Windows and MacOS that, for example, do not magnify the screen-saver password screen. 7. Major functions and User Interface aspects: 1. Magnifies and enhances (e.g. mouse pointer crosshairs, colour inversion, colour contrast) the entire screen. 2. Magnifier preference settings: Used to set preferences for screen magnification and enhancement. 8. Relevant Use Cases (from D1.1.3): 1. 4.1 Screen magnification for the GNOME Desktop 9. Level of achievement of defined SP2 general requirements (see D2.2.1a): Table 5: Current level of achievement of defined requirements for GNOME Shell Magnifier

October 2012

Oracle

11


SP2 relevant Requirements

PU


Achieved (Yes/No/Partially and short justification)

Relevant User Requirements (from Chapter 3 of D2.1.1) UR-V1: To be able to use a Yes. GNOME Shell Magnifier provides built-in magnification of visual only based tool as easily the GNOME desktop with a range of personalization settings. as possible. For the desktop this is the PC with the monitor as the main output (or input if we also consider touch screens). Low vision users want to have access to the data shown on the computer monitor with an easy, intuitive and unobtrusive way. UR-V2: Low vision users also Yes. GNOME 3.4+ provides customizable font size. GNOME need customisable screen 3.6+ will provide Ccolour contrast settings are only available in contrast and font size in order to the prototype currently being tested. adjust it to their residual vision. UR-V3: The compatibility of Yes. There is no incompatibility between GNOME Shell tools like the screen magnifiers Magnifier and any graphical and/or dynamic applications. with dynamic and graphical applications to allow a full use of the computer. UR-V4: In general for all users Yes. GNOME Shell Magnifier is compatible with other Assistive with visual impairments, the Technologies for GNOME, in particular Orca. compatibility of ATs when using certain operating systems is essential to guarantee their access to the computers. Relevant functional requirements (from section 4.1.1 of D2.1.1) F-GSM-1: The magnifier must magnify any portion of the GNOMEShell desktop, including log-in screens and components that implement ATK, by rerendering of the visual output according to the user's needs.

Yes. GNOME Shell Magnifier magnifies all parts of GNOME Shell including running applications and the screen-saver login screen, but it does not magnify the main login screen since that occurs before GNOME Shell is running.

Relevant non-functional requirements (from section 4.1.4 of D2.1.1)

October 2012

Oracle

12



PU



Efficiency: Efficiency is a Yes. Moving the mouse with the GNOME Shell Magnifier crucial QA for the Open running uses between 3-20% of the CPU versus 3-18% without Magnification Functionality the Magnifier running. The Magnifier is therefore quite efficient. (GNOMEShell Magnifier). The Magnifier should work on top of other applications, and because of that, it should use minimal system resources and update the screen in real time. Time Behaviour: The Response Time of the magnifier should be real time, meaning that the changes of the magnified region should be magnified on-the-fly.

Yes. The magnified mouse movements are generally in sync with actual mouse (moving within Keyboard>Shortcuts>Universal Access, these can be used to reverse erroneous settings

Ease-of-Use: Its way of use Partial. The magnifier is easy and intuitive to use. However, help should be trivial and intuitive, information is still desirable, but is not currently available. but in case there is a need for user learning/training, the system should be able to guide the user, through a help/ system. Accessibility: In terms of Yes. The magnifier preferences dialog is fully keyboard accessibility, the magnifier accessible. should support alternative ways of input, in addition to mouse, such as for example keyboard – based control of the magnifier position and zoom level. Adaptability/ Customizability: It should integrate well with the existing screen reader for GNOME (e.g. Orca).

Partial. The dependency on Orca to control the magnifier has been removed and placed in the GNOME Control Center. There is a temporary problem that the magnifier is not able to track focus. This will be rectified in GNOME 3.8+ when focus tracking is fully implemented in GNOME Shell.

Relevant technical specifications (from section 4.1.1 of D2.1.1)

October 2012

Oracle

15



PU



Partial. GNOME 3.6 is on track for release on 26 Sept 2012. It TS1: The user will be able to will implement bullets: 1, 2, 3, 4, 5, and 7. select and combine from a wide of basic and advanced display options including: 1. magnification factors from 100% to 3600% 2. various magnification pane configurations (full screen, top, bottom, left, right, lens) 3. mouse tracking modes (centred, push, proportional) 4. colour personalisation (e.g. inverted, white-onblack or custom) 5. modified pointer and/or a modified cursor (including crosshairs) 6. animated transitions (for orientation purposes, e.g. from zoom-in to zoomout) 7. ability to quickly change the magnification factor without having to interact with preference dialogs 8. object-based magnification (when magnifier is used with Orca screen reader) 9. ability to select an area or object and fix them in a magnified fashion to a location on screen, while the rest of the view shifts according to user's point of regard.

October 2012

Oracle

16



PU



TS2: context-aware No. This is still being researched. magnification based on the GNOME accessibility infrastructure (e.g. magnifying menus differently from dialog boxes, differently from text content; or magnifying differently based on text size, text font, text attributes). TS3: multiple simultaneous No. This is still being researched. magnification lenses – each tracking a different portion of an active application (e.g. one magnifying the spreadsheet formula bar, another magnifying the spreadsheet status bar, and a third tracking whatever the user is doing within the spreadsheet content). TS4: Preferences are stored Yes. Settings are stored with GSettings which replaced GConf as using GConf. In addition, Orca the GNOME best practice. can set preferences using DBUS. TS5: The magnifier operates as No longer applicable. The Magnifier has transitioned to being a magnification service for the independent of Orca. It's settings are now controlled by the Orca screen reader. In GNOME Control Center. particular, the magnifier: 1. can be started by Orca 2. observes the magnification preferences set in the Orca properties 3. follows the Orca focus TS6: Communication is via Yes. This functionality is implemented. DBUS via the a modified gnome mag API Illustration 8: Current level of achievement of defined requirements for GNOME Shell Magnifier

October 2012

Oracle

17


PU


2.6 Relationship to the OAF The AEGIS Open Accessibility Framework (OAF) defines a generic framework to addressing Information Communication Technology (ICT) accessibility. Specifically, the OAF defines steps of the “value delivery chain” of creating and using accessible (ICT): three “creation” steps, and three “use” steps. The GNOME Shell Magnifier is (A2.2.1) is part of Step 4 - “Platform Support” - and part of Step 6 - “Assistive Technology”.

Illustration 9: OAF view of Desktop, as seen through the 6 steps of accessibility (3 for creation, 3 for use). Text in bold, italicized red indicates where A2.2.1 delivers into the OAF

October 2012

Oracle

18


PU


2.7 Planned Future Developments 2.7.1

Extend GNOME Shell Magnifier Feature Set

The following features are in development: 1. Multiple Displays 

October 2012

Some users have two displays attached to their computer. They designate one of the displays for the "normal" view, and the other display is used for the magnified view. The use of two displays is for orientation. It is easy to lose one's place in a magnified view since much of the context is lost -- the whole screen cannot fit within a magnified view. Instead, the non-magnified display acts as a thumbnail for the user and they can use that as a way to orient themselves.

Oracle

19


PU


3 A2.2.2 Open, cross-platform support of desktop accessibility framework on Windows 3.1 Introduction The existing Java Access Bridge for Windows (v2.01) was an early attempt to solve the "platformon-a-platform" accessibility problem. However, due to the insufficiency of the only Windows accessibility API at the time (Microsoft Active Accessibility - MSAA), the bridge had to be designed to present its own unique API to Windows assistive technologies for making Java applications accessible. Since that time, two new and more comprehensive accessibility APIs have emerged for Windows: •

IAccessible2: an extension of MSAA, initially led by IBM, that was intended to harmonize MSAA with the Java Accessibility API and Assistive Technology Service Provider Interface (AT-SPI)

•

UI Automation: a new approach developed by Microsoft to create a more flexible accessibility API

This presented the opportunity to move away from the unique API presented by the Java Access Bridge, and to better and more appropriately address the "platform-on-a-platform" problem. The current Java Access Bridge is implemented as a standalone “add-on” to the Java platform. This allows a single bridge to function across multiple Java runtime releases and it allows both the Java platform and the bridge to be developed largely independent of each other. For this AEGIS activity, we undertook to investigate the feasibility of the following: •

whether the IAccessible2 API is suitable for expressing Java accessibility information to Windows screen readers (thereby solving the "platform-on-a-platform" problem).

•

whether a bridge to IAccessible2 can be implemented external to the Java platform – similar to the existing Java Access Bridge or whether it needs to be implemented within the Java platform. If the bridge could be implemented externally, it could be developed and maintained by an open source community, including the ATRC/IDRC. If not, it was likely that subsequent development would need to be undertaken by SUN/Oracle.

3.2 Feasibility Study The feasibility study was conducted by the IDRC/ATRC, with technical advice provided by SUN/Oracle. From the beginning, the goal was to attempt to develop the bridge external to the Java platform so it could be maintained in open source, and independent from the Java SE platform. Because of the similarity of IAccessible2 to the Java Accessibility API, and the greater support for IAccesible2 among Windows assistive technologies, the study began with that API. IAccessible2, being based MSAA (which in turn uses COM for interprocess communication), works by sending WM_GETOBJECT messages to the WinProc of each HWND or Window object that the assistive technology is interacting with. These WM_GETOBJECT messages are synchronous – they must be handled immediately, and the assistive technology blocks until the call October 2012

Oracle

20


PU


returns. The call returns with a handle to the object that "speaks accessibility" on behalf of the HWND, and then the remaining accessibility API calls are made to that object – which is an instance of the IAccessible (or IAccessible2) class. The following technical approaches were attempted: 1. Implementing a dummy Window to receive the WM_GETOBJECT calls instead of the Java runtime 2. Making a small extension to the Java runtime, allowing an external library (e.g. a new Java Access Bridge) to handle WM_GETOBJECT messages on behalf of specific HWNDs created by the Java runtime Unfortunately, these approaches were ultimately unsuccessful. It was not possible to create dummy windows, or otherwise intercept the WM_GETOBJECT messages, in a stable and supportable fashion. In exploring the second option, we ran into two issues: the Java runtime security policy – which greatly limits the kinds of code that can be run in response to synchronous Windows messages (no end-user or external code can be run, for risk of it blocking the Java runtime – which alone would be sufficient reason to not allow such an extension), and the fact that the IAccessible and IAccessible2 objects need to persist and return information that is normally contained within the running application – which again could block the Java runtime if such a call was made in response to the synchronous WM_GETOBJECT call. This led to the conclusion that IAccessible2 could only be implemented directly in the Java runtime itself. UI Automation was then considered, but it also depends upon COM when used by non-managed code, which would be the case for the Java runtime.

3.3 Conclusions •

It was determined that the IAccessible2 API (an extension of Microsoft Active Accessibility) is the appropriate Windows accessibility API for an update to the Java Access Bridge. It has gained significant traction in the marketplace – with multiple Windows assistive technologies (e.g., JAWS, NVDA) implementing it, along with a growing number of mainstream applications. In addition, UIA shares with IAccessible2 A the need to develop the bridge internal to the Java platform.

•

It was determined that an update to the Java Access Bridge using IAccessible2 could not be developed outside the Java platform (see above for more details). Therefore, in consultation with Oracle, ATRC/IDRC closed the study and transferred the project to Oracle, where implementation of the IAccessible2 interface is now an internal project.

October 2012

Oracle

21


PU


4 A2.2.3 Open Text-to-Speech multilingual functionality 4.1 Introduction Text to speech, TTS, is the automatic conversion of unrestricted text into speech. The device, or computer program, that does this is a TTS synthesizer, and if it is a computer program it is often called a TTS Engine. A typical TTS engine comprises several components. Depending on the nature of a particular TTS engine, these include those which: Resolve ambiguities in the text, such as converting Mr to Mister, recognizing dates and currencies as differing from ordinary numbers • Convert the text to symbols or phonemes representing the corresponding speech sounds using a lexicon, a set of simple rules, a statistical method predicting likely pronunciations, or a combination of these • Generate prosodic information, which describes how to speak the sounds in terms of their duration and the pitch of the voice • Convert the phonemes and prosody to an audio speech signal •

There are other sub-systems that may also be used, but all TTS engines have the equivalent of the above. Some of these can be very sophisticated and may for instance use natural language processing to resolve text ambiguities. The last stage, the one that turns a phonetic and prosodic description of the desired speech into a speech signal is often used to label the type of TTS Engine, perhaps because this stage broadly determines what the synthesized speech sounds like. Speech synthesis has long been a vital assistive technology tool and its application in this area is significant and widespread. It allows environmental barriers to be removed for people with a wide range of disabilities. The longest application has been in the use of screen readers for people with visual impairment, but text-to-speech systems are now commonly used by people with dyslexia and other reading difficulties as well as by pre-literate youngsters. They are also frequently employed to aid those with severe speech impairment usually through a dedicated voice output communication aid.

4.2 Current State of the art – Open-source TTS Engines 4.2.1

Available TTS Engine Types

The most important qualities of a speech synthesis system are Naturalness and Intelligibility. Naturalness describes how closely the output sounds like human speech, while intelligibility is the ease with which the output is understood. The ideal speech synthesizer is both natural and intelligible. Speech synthesis systems usually try to maximize both characteristics. Speed is also of importance, especially for users with vision impairments. The two primary technologies for generating synthetic speech waveforms are concatenative synthesis and formant synthesis. There are also other types of engines, such as the Hidden Markov Model (HMM) based synthesis, which recently has started gaining momentum, articulatory synthesis and sinewave synthesis. Each technology has advantages and disadvantages and the intended uses of a synthesis system will typically determine which approach is used. October 2012

Oracle

22


4.2.2

PU


Formant Synthesis

Formant synthesis systems synthesise speech using acoustic models of speech production. This means that they model the speech spectrum and its changes in time as we speak, rather than model the production mechanisms themselves. Formant synthesis systems are sometimes referred to as synthesis-by-rule systems or more usually formant synthesisers. In formant synthesis, the synthesized speech output is created using an acoustic model. Parameters such as fundamental frequency, voicing, and noise levels are varied over time to create a waveform of artificial speech. Formant-synthesized speech can be reliably intelligible, even at very high speeds, avoiding the acoustic glitches that commonly plague concatenative systems. High-speed synthesized speech is used by the visually impaired to quickly navigate computers using a screen reader, making them ideal for accessibility use. Formant synthesizers are usually smaller programs than concatenative systems because they do not have a database of speech samples. They can therefore be used in embedded systems, where memory and microprocessor power are especially limited. Because formant-based systems have complete control of all aspects of the output speech, a wide variety of prosodies and intonations can be output, conveying not just questions and statements, but a variety of emotions and tones of voice. The major weakness of rule-based formant synthesis is that the speech does not sound natural. It sounds artificial and robotic-like and would never be mistaken for human speech. This is because of the simplicity of the models used; it is very difficult, if not impossible, to model, those subtleties of speech that give rise to the perception of naturalness.

4.2.3

Concatenative Synthesis

Concatenative synthesis is based on the concatenation of segments of human speech which has been pre-recorded. Generally, concatenative synthesis produces the most natural-sounding synthesized speech. However, differences between natural variations in speech and the nature of the automated techniques for segmenting the waveforms sometimes result in audible glitches in the output. For example if a concatenative engine tries to speak faster than the rate of the pre-recorded speech, glitches in the final result will occur.

4.2.4

Available Open-source TTS Engines

4.2.4.1 Festival

Festival offers a general framework for building speech synthesis systems as well as including examples of various modules. As a whole it offers full text to speech through a number APIs: from shell level, though a Scheme command interpreter, as a C++ library, from Java, and an Emacs interface. Festival is multi-lingual (currently English and Spanish) though English is the most advanced. Other groups release new languages for the system. And full tools and documentation for build new voices are available through Carnegie Mellon's FestVox project. The system is written in C++ and uses the Edinburgh Speech Tools Library for low level architecture and has a Scheme (SIOD) based command interpreter for control. Documentation is given in the FSF texinfo format which can generate, a printed manual, info files and HTML. Festival is free software. Festival and the speech tools are distributed under an X11-type licence allowing unrestricted commercial and non-commercial use alike.

October 2012

Oracle

23


PU


4.2.4.1.1.1 FestVox

The Festvox project aims to make the building of new synthetic voices more systemic and better documented, making it possible for anyone to build a new voice. It is the “voices” project of Festival. FestVox supports Diphone Synthesis, General and Domain Specific Unit Selection and HMM-based synthesis. FestVox has a broad variety of voice databases and languages and many of the Unit Selection and HMM based voices are quite natural, while the smaller ones retain acceptable intelligibility. FestVox is not a TTS Engine, since it cannot get free text as input, but it can be used with the Festival TTS Engine. It is Free / Open Source project and is distributed under an X11-type license allowing unrestricted commercial and non-commercial use alike. Festival, through FestVox, supports many languages through various University contributions, although not all of them are freely available. At the moment 6 different languages are available. 4.2.4.2 Flite

Flite (festival-lite) is a small, fast run-time synthesis engine developed at Carnegie Mellon University and primarily designed for small embedded machines and/or large servers. Flite is designed as an alternative synthesis engine to Festival for voices built using the FestVox suite of voice building tools. Flite offers: • • • • • •

Completely in C (no C++ or Scheme) for portability, size and speed Reimplementation of the core parts of the Festival architecture (HRG) allowing close compatibility between voices built for each system. Support for compiling FestVox voices into Flite voices. Thread safe Scalable voice size with all data const so it can be in ROM Target architectures, ipaq (Linux/WinCE), Palm OS (treo) and smaller

Flite is in basically written and is in its first stages of testing before release, as free software. A small diphone voice based on the CMU KAL voice is included. along with a sample limited domain talking clock. It is distributed under an X11 style license and is Open Source software. 4.2.4.3 GnuSpeech

GnuSpeech is an extensible, text-to-speech computer software package, that produces artificial speech output based on real-time, articulatory, speech-synthesis-by-rules. That is, it converts text strings into phonetic descriptions, aided by a pronouncing dictionary, letter-to-sound rules, and rhythm and intonation models; transforms the phonetic descriptions into parameters for a low-level articulatory speech synthesizer; uses these to drive an articulatory-model of the human vocal tract producing an output suitable for the normal sound output devices used various computer operating systems; and does this at the same or faster rate than the speech is spoken for adult speech. It is free software and it is distributed under the Gnu GPL.

October 2012

Oracle

24


PU


4.2.4.4 FreeTTS

FreeTTS is a speech synthesis system written entirely in the Java programming language. It is based upon Flite: a small run-time speech synthesis engine developed at Carnegie Mellon University. Flite is derived from the Festival Speech Synthesis System from the University of Edinburgh and the FestVox project from Carnegie Mellon University. FreeTTS includes: Core speech synthesis engine • Support for a number of voices: • an 8khz diphone, male, US English voice • a 16khz diphone, male US English voice • a 16khz limited domain, male US English voice • Support for importing voices from FestVox (US English only) • Specific support for importing CMU ARCTIC voices from FestVox (US English only) • Support for MBROLA voices (downloaded separately): • a 16khz female, US English voice • two 16khz male US English voices • Partial support for JSAPI 1.0 • Extensive API documentation FreeTTS can be used as a workstation/desktop TTS engine. For example, the Emacspeak demo works right out of the box with Emacspeak. One of its strengths is that it can be used as a Java Applet or as a Java Webstart program, so there is no need for installation. It has been tested on the Solaris TM Operating Environment, Mac OS X, GNU/Linux and Win32 operating systems. •

4.2.4.5 eSpeak TTS engine

eSpeak is a compact open source software speech synthesizer for English and other languages, for GNU/Linux and Windows. ( http://espeak.sourceforge.net ) eSpeak produces good quality English speech. It uses a different synthesis method from other open source TTS engines, and sounds quite different. It is perhaps not as natural or "smooth", but many users find the articulation clearer and easier to listen to for long periods. It can run as a command line program to speak text from a file or from stdin. A shared library version is also available. Some of the eSpeak characteristics are mentioned below: • • • • • • •

Includes different Voices, whose characteristics can be altered. Can produce speech output as a WAV file. SSML (Speech Synthesis Markup Language) is supported (not complete), and also HTML. Compact size. The program and its data, including many languages, totals about 1 Mbytes. Can translate text to phoneme codes, so it could be adapted as a front end for another speech synthesis engine. Potential for other languages. Several are included in varying stages of progress. Most of them have been created by volunteer native speakers. Development tools available for producing and tuning phoneme data.

October 2012

Oracle

25


PU


•

Written in C.

•

eSpeak supports 28 different languages and 15 Languages that have naïve implementations.

4.3 AEGIS technology selection - eSpeak eSpeak, as a formant based engine, has one main disadvantage: it does not produce a natural sounding voice, but rather a robotic-sounding one, thus reducing its acceptability for people not used to using it. Despite this drawback, in AEGIS, we selected eSpeak as the TTS engine on which we would invest to enhance the accessibility of the Open Accessible Desktop. The main reasons for this selection are the following: •

eSpeak is an free open-source TTS engine with a big community of users, as well as developers.

•

It already supports many languages (many more than any other open source TTS engine) and it is relatively easy to create new or enhance existing language implementations.

•

eSpeak is very lightweight, because it is an efficiently written formant-based engine. This makes it a very good candidate for embedding in mobile environments where memory and processing power are limited (Mobile phones, PDAs etc). It should be noted that one of the AEGIS goals, under WP4.6, is the porting of a light weight TTS engine so that it can be embedded in mobile environments. Once the porting is complete, it will be possible to give access to all the languages that eSpeak already supports via the mobile environment including the enhancements and improvements that will take place under A2.2.3.

4.3.1

Pre-AEGIS State – Language Quality

eSpeak advertises support of more than 28 different languages. While some of them have fully realized implementations, at least half of these languages have many things missing or incorrectly pronounced and intonated. This is because every new language that is added to eSpeak, "inherits" a base phonemene and rules table that comes from the English. For many of the pilot languages like Swedish, English intonation and pronunciation rules had been adopted for many words . Also the numeric ordinals were supported incorrectly, generating a mixture of English numbering and Swedish endings, like e.g. 1:a was sounding like "one-A". For the Greek and Spanish languages, many of the vowels where a lot longer that usually spoken, even when these vowels where unstressed in the words. Consonants like "T" and "R" were pronounced in an "English" way. Ordinals for some of the languages were both badly pronounced while many abbreviations were not pronounced correctly. Dutch was in a much better state before the AEGIS team started working on the language. However, many pronunciation rules were found to be inaccurate thus creating speech output which was unintelligible. Also some of the vowels, like "O", were spoken longer than normal.

4.4 Language Enhancement Methodology One on the first things that the AEGIS team needed to determine was the current status of the pilot language eSpeak implementations. To do this, audio files were used at the first pilot trials phase of October 2012

Oracle

26


PU


the project (as described in section 28), in order to get feedback from the native speakers of the pilot sites regarding quality and issues to be corrected. In parallel, language enhancements were implemented for the Greek language, since this was the native language of the main eSpeak AEGIS development team (SingularLogic). Having the experience from the Greek language enhancement, a preliminary guide on how to work for improving languages was prepared for all AEGIS pilot sites, so that a systematic approach is followed for all language enhancements by all pilot site teams (e.g. ordinals were to be checked first, pronunciations next, etc.). Additionally, an eSpeak workshop was held at the AEGIS first international conference in Seville during which SingularLogic worked with several native speakers. Sessions were organised for each pilot site team in order to train them on how to implement language enhancements or how to provide to the SingularLogic team feedback on issues that needed to be corrected. A lot of feedback was also captured from native speakers that were attending the conference, notably Spanish but also Swedish, French and Icelandic.

4.5 Current status of the prototype Language enhancements for Greek, Swedish, Belgian Dutch, Italian and Spanish have been added into the engine. In more detail, until October 2011, the status of improvements is as follows: •

29 new and corrected rules and 33 dictionary exceptions in Greek language files. These include fixing many intonation errors, number and ordinal pronunciation and abbreviations (e.g. the ordinal “1ος” which means “first” was pronounced “ένα-ος” instead of “πρώτος”). Also the timings of many phonemes, like “A”, was fixed to match the timings and speeds for the Greek language, as well as vowel and consonant sound corrections, to make language sounds “compatible” with native speaker expectations.

•

15 new and corrected rules and 10 dictionary exceptions in the Swedish language files. These corrections mostly concern intonation problems of the Swedish words. Many of the words in the version of Swedish eSpeak on which we began working had English intonation.

•

15 new and corrected rules and 30 dictionary exceptions (mostly abbreviations) in the Spanish language files, like ordinals (e.g. 1o was pronounced uno-o instead of primero). Vowel and consonant sound corrections were also implemented, to make language sounds “compatible” with native speaker expectations.

•

35 new and corrected rules in the Dutch language files. These are mostly pronunciation corrections and vowel sound length corrections (e.g. The “o” at the end of a word was always spoken longer that it should). Many of these changes were accepted for inclusion in the Dutch language of eSpeak. However, a new variant was created (nl-be) that has rules that are compliant with the Belgian Dutch pronunciation but not with the dutch pronunciation in the Netherlands.

These enhancements have been delivered upstream to the main development team of eSpeak (http://espeak.sourceforge.net) and are expected to be integrated in the next version of eSpeak. Enhancements (existing as well as future ones that are planned for the period March – June 2011 will also be evaluated by users during the second phase of pilot trials and during the 2 nd AEGIS international conference.

October 2012

Oracle

27


PU


4.6 Testing (Pilots) 4.6.1

First Phase of Pilot Trials

The main purpose of including eSpeak in the first pilot trials was to collect end users' feedback on the existing status of the eSpeak pilot site languages before we start working on them. We aimed at collecting as many comments as possible on specific words/sounds, etc. that needed to be improved in order to make eSpeak more intelligible in these languages. During the first pilot phase, end users from each pilot site heard a number of pre-generated audio files, that were generated from two different source text files: A text document that consisted of a small part from the snow white fairytale and a more formal document which contained text from the europa.eu site. For each document and each language, 2 different speed and 2 different voice pitches where selected, generating in total 4 audio files. Users were asked to evaluate the audio files and identify problems in language sounds, like problems in intonation, pronunciation and intelligibility in general. The results of the first pilot phase were used for improving the pilot site languages according to users' comments. They will also be used for comparison purposes as the baseline for evaluation of the various language enhancements made by the AEGIS development team.

4.6.2

Second Phase of Pilot Trials

During the second phase of the pilot trials, users listened to the new, improved eSpeak output and made comparisons with the pre-recorded files of the first pilot phase in order to assess improvements. Further comments on additional enhancements were made for the Spanish and Belgian Dutch language that, if implemented, will improve the overall intelligibility of these languages.

4.6.3

Planned Future Developments

4.6.3.1 Further Language enhancements

While the AEGIS team initially gave priority to the pilot site languages, as a more efficient way to have results for the project, we also intend to address more languages if time and resources allow and if native speakers volunteer to help us identify the corrections that need to be made. AEGIS plans to make further enhancements to most AEGIS pilot site languages ( Spanish, Swedish and Dutch) as well as on Greek, Italian, Czech and other languages for which available native speakers will volunteer to provide consultation. Specific consultation sessions with native speakers will be organised for this during project meetings, as well as targeted travels of the SingularLogic development team. Some of these consultation sessions are expected to take place during dedicated eSpeak working sessions at the AEGIS final workshop and international conference that will take place in Brussels, 28 - 30 November 2011. 4.6.3.2 Language Enhancement Guide

Language editing can be a task that may consume a lot of effort and time. During our sessions with the native speakers, we found out many ways to make these sessions more efficient and less October 2012

Oracle

28


PU


stressful for both the speakers and the language developers, for example: •

Starting by assessing the way in which numbering and ordinals are spoken, since these are very frequently incorrectly pronounced or missing from eSpeak.

•

Continuing with abbreviations, signs (like %,$ etc) that may sound wrong.

•

Choosing words that have the same intonations, or similar pronunciation rules, and then fixing these rules.

All of these methods will be delivered to the eSpeak community in the form of a "Language Enhancement Guide" that will be published on the AEGIS web site as well as the eSpeak website.

4.7 Overview of the application 1. Title of product: eSpeak TTS Engine – Language Enhancements 2. Development team: Singular Logic 3. Relevant WP/Activity development/integration tool place: WP 2.2 / A2.2.3 4. Purpose: Enhancement of the eSpeak TTS engine in EU language support. 5. For which user groups and what are the expected benefits for each of them: •

Blind (without useful residual vision): blind users should benefit because speech output is the main output for this group of people.

•

Slight cognitive limitation and low support need (Users with dyslexia/reading impairments): persons with dyslexia can benefit from the availability of DAISY books, and CCF codes. These technologies require support from a TTS engine because of their extensive use of synthetic speech.

•

Speech Impaired: Users with dyslexia or reading impairments would like to use a Desktop PC as a helping tool that will read them books and articles.

6. Innovations (in comparison to SoA): Enhanced support for many EU languages, including Dutch, Swedish, Spanish, Greek, English, to an very intelligible level, as it is reported by the users. 7. Major functions and User Interface aspects: 1. Major functions: Convert text input to audio (speech) output. 2. User interface aspects: No user interface interaction is needed as the TTS engine is a library that provides TTS services to other application with GUIs available. 8. Relevant Use Cases (from D1.1.3): 1. UC 4.5 – Full DAISY book creation 2. UC 4.6 – Multilingual documents through a screen reader 3. UC 4.10 – Graphic Symbol Support for facilitated text comprehension and production in OpenOffice.org 9. Level of achievement of defined SP2 general requirements Table 6: Current level of achievement of defined requirements for TTS

October 2012

Oracle

(see

D2.1.1):

29


PU


SP2 relevant Requirements Achieved (Yes/No/Partially and short justification) Relevant User Requirements (from Chapter 3 of D2.1.1) UR-C2: Users with dyslexia Yes. By complementing the typical TTS reading support with an or reading impairments option to verify the meaning of words with a visual alternative would like to use a Desktop representation. PC as a helping tool that will read them books and articles. UR-V5: Blind users need Yes: eSpeak TTS engine fully provide this output mechanism. alternative output mechanisms. These users need to have full or close to full accessibility to their desktop PC, with the use of their other senses like touch (Braille) and hearing (Screen Reader - TTS), in order to circumvent their vision disability. UR-V7: Blind users require No: This is a restriction of the formant speech synthesis method, as more natural voices in the it is very difficult to create natural tones from it. speech outputs. Computerized voices without emotion and intonation is a frequent problem for new users or children who have to rely and “communicate” with a computer voice all day long. When that voice does not sound natural, this affects the acceptance to the assistive technology (AT). UR-S1: Users with dyslexia Yes: see UR-C2 above. or reading impairments would like to use a Desktop PC as a helping tool that will read them books and articles.

October 2012

Oracle

30


PU


SP2 relevant Requirements Achieved (Yes/No/Partially and short justification) UR-S3:Regarding those Yes: eSpeak TTS provide the speech output engine for technologies users with communication like the AAC and VOCA. problems because of cognitive limitations or learning difficulties, such as aphasia, it is important to mention the difficulties of using operating systems, programs or menus which have many functions and symbols difficult to understand and use. For that reason they require solutions that provide the information in a clear and simple way, and through different channels: •

Symbols and pictures (AAC);

•

VOCAs (Voice Output Communication Aids). This could be done either with realtime communication tools or by providing the user with a voice, using speech synthesis for example. For those with Dyslexia, the usage of speech recognition software is recommended.

UR-S4:These users demand better quality of some speech outputs. The unclear, distorted voice, that speaks too fast, makes it difficult to use the computer for this specific group of users.

Partially: There already many improvements on EU languages that make them more understandable to native speakers even at high speeds. This is an ongoing process as many languages are continuously been optimized.

Relevant functional requirements (from section 4.1.3.2 of D2.1.1)

October 2012

Oracle

31


PU


SP2 relevant Requirements Achieved (Yes/No/Partially and short justification) F-TTS-1: The TTS Engine Yes: This functionality is already provided by the eSpeak TTS should receive text as input, in Engine. any supported language, and send sound data as output either on the speakers through the Desktop Hardware, or to a file. Relevant non-functional requirements (from section 4.2.5 of D2.1.1) Efficiency: The TTS engine works on top of other applications and should have a small processor and memory footprint.

Yes: As a formant based engine, the eSpeak TTS engine has a very small footprint both on installation (no more than 5MB) and memory (no more than 3MB). Also the format speech synthesis has a relevantly low processing overhead that will not exceed 5-10% on modern desktop systems

Time Behaviour: The Yes: Because it is based on format synthesis, the eSpeak TTS engine Response Time of the TTS provides an almost spontaneous response time. Engine should be spontaneous time, meaning that the user should not wait a lot of time for the engine to start “talking”. The response time must be no more than 50ms. Resource Utilization: Yes: As stated above, eSpeak TTS engine has a small memory and Because the TTS engine CPU footprint, so it makes efficient use of a Computers resources works on top of other applications, it should have minimal needs for Processing Capacity, meaning that it should not make the depended applications slower because of high resource utilization. Usage Time: Yes: Using format based synthesis, the usage time is minimized to a Usage Time should also be very low, even lower that the 20% threshold that is stated as a minimal also, which means requirement. that the engine should never need much time to create sound from text.

October 2012

Oracle

32


PU


SP2 relevant Requirements Achieved (Yes/No/Partially and short justification) Portability: Yes: eSpeak works with all the environments the Open Accessibility The TTS Engine is a part of Desktop supports, including GNU/Linux and GNOME, Solaris and the Open Acceptability GNOME and Windows. Desktop, thus it should work on any environment that the OAD work. This makes Portability QA an important attribute for every Application of the Desktop. Installation: It should get installed with and get removed with every installation and uninstallation of an Open Desktop Platform, with minimal configuration.

Yes: eSpeak provides an automated installer for windows and can be seamlessly installed on Linux environments since it is present in all the software package repositories of the major Linux distributions (Debian, Ubuntu, Fedora, SuSE etc), that provide automatic installation.

Reliability: Yes: No reliability problems have been reported from the users of Since the TTS Engine the pilot sites. provides a single function, and a function that is essential for the OAD, this function should be always reliable. Effectiveness: It should be intelligible and as natural as possible, as well as speedy. It should speak out effectively, numbers, abbreviations, numbered /bullet lists.

October 2012

Partially: As this is part of a continuous effort, a lot of improvements regarding intelligibility for a number of EU languages, including Greek, Swedish, Dutch, Spanish, German etc. We are continuously making more improvements en these and also other languages.

Oracle

33


PU


SP2 relevant Requirements Achieved (Yes/No/Partially and short justification) Localization: It should speak more than one language, and it should be easy to add more. The metrics that will be measured are: •The number of the language pronunciation rules that will be added

Partially: This non-functional requirement is also the main focus of the A2.2.3 activity. Already a lot of language rules and exceptions have been added or amended so that languages become more and more intelligible. As this is still a work in progress, it is considered partially implemented.

• The number of exceptions that will be added for each of the languages • The total number of the languages that will be enhanced. The target value regarding the languages is at least 5 (Spanish, Greek, Swedish, Dutch, French). Customizability: Yes: eSpeak already supports many different voice speeds (from 30 Customizability is also key, words per minute to more that 400 words per minute), pitches and especially in term of voice voices (whisper voice, many male and female voices). pitch and speed settings. The user must be able to choose from a range of different pitch and speed values. Attractiveness and Comfort: In terms of attractiveness and comfort, the TTS should be effective and acceptable both for short texts and long texts, such as e.g., for book reading or newspaper article reading.

Partially: The main drawback of the format speech synthesis is that the speech output is rather robotic and thus not natural. This makes eSpeak not that best choice when it comes to book reading, when emotions must also be used. For newspaper reading though, formant synthesis is considered sufficient and intelligible, both for small and large texts.

Relevant technical specifications (from section 4.1.3.4 of D2.1.1) TS1: Yes: Because it is based on format synthesis, the eSpeak TTS engine The TTS Engine should start provides an almost spontaneous response time. speaking very quickly after being given an utterance to speak October 2012

Oracle

34


PU


SP2 relevant Requirements Achieved (Yes/No/Partially and short justification) TS2: Yes: Because it is based on format synthesis, the eSpeak TTS engine It should stop speaking very provides an almost spontaneous response time, both for stopping quickly after being told to and starting. stop TS3: Yes: As noted on the Portability QA. The TTS Engine should be portable, able to run on GNOME Linux/UNIX platforms, as well as mobile platforms, as well as everywhere that OpenOffice.org runs (which adds to the above set Windows & Macintosh). TS4: It should be able to speak intelligibly at a very high rate of speed (at least 450 words per minute, perhaps even as high as 600 words per minute)

October 2012

Yes: eSpeak already supports many different voice speeds (from 30 words per minute to more that 400 words per minute).

Oracle

35

AEGIS D2.2.1b_final

PU


4.8 Relationship to the OAF Creation Define Accessible GNOME Accessibility API - AT-SPI for ATs - ATK for GNOME apps - Bridged JA-API GNOME Keynav - Defines keynav for all UI element types - Defines keynav for desktop GNOME Themeing - Theme engine, defines focus indicator, I-beam, other things OpenDocument Format - Defines necessary document metadata for accessibility

Stock Elements GTK+ widgets - implement ATK (via GAIL) - support keynav - support themeing XUL widgets (FF, TB) - implement ATK (via bridge) - support keynav - support themeing UNO widgets(OOo) - implement ATK (via bridge) - support keynav - support themeing

Java Swing widgets - implement ATK (via bridge) - support keynav (for GTK+ L&F) - support themeing (for GTK+ L&F)

Use Dev/Auth Tool NetBeans IDE - Developer tool - Aids Java Swing dev. of accessible apps Glade UI designer - Developer tool - Aids GNOME/GTK+ dev. of accessible UIs

Platform Support AT-SPI & ATK libraries

AccessX options - Ship with GNOME, StickyKeys, MouseKeys, Etc. (arguably an AT) Theme switcher

OpenOffice.org / LibreOffice

- Authoring tool - Ability to annotate documents with accessiblity metadata - Export to PDF, HTML, DAISY preserving accessibility metadata - Print to Braille - Accessiblity validation (for DAISY at least) - Create documents annotated with concept coding RDF metadata and CCF symbol representation

- Enables user to choose theme AT app launcher - Enables user to choose AT to use at login Built-in AT - Orca, GSMag, eSpeak, Dasher, all part of GNOME by default - Other AT (e.g. Caribou) free download linked from desktop

The App Itself

Assistive Technology

Huge numbers of apps are accessible, either shipping with GNOME desktop or linked as a free download

Orca screen reader - Blind & low-vision AT

- GTK+ apps, - Firefox, Thunderbird - OpenOffice.org - Java Swing apps

GSMag magnifier - Built into GNOMEShell, used by Orca eSpeak TTS - Used by Orca, others BRLTTY braille library - Used by Orca Dasher text entry & Caribou on screen kbd - Physical impaired AT Opengazer library - Eye & gesture tracker, used as mouse by Dasher, Caribou, ...

CCF Symbol Server - Serves users with graphic symbol representation of text content – via the CCF Symbol extension for Ooo (a.o. apps) ...

Illustration 10: OAF view of Desktop, as seen through the 6 steps of accessibility (3 for creation, 3 for use). Text in bold, italicized red indicates where the eSpeak TTS in A2.2.3 delivers into the OAF November 2011

Oracle

36


PU


5 A2.2.3 CCF symbol support extension for Open Office Writer 5.1 Background The Description of Work (DoW) calls for development of plugin functionality, based on Concept Coding, for multi-modal (including graphic symbol) and multilingual language comprehension support in office documents. During the last 10 to 15 years the use of graphic symbols to support literacy development and access to text content has become increasingly wide-spread in special needs education and AAC (Augmentative and Alternative Communication) practices. This popularity is founded on a growing body of positive experience and research studies1. It is also accompanied by the availability and use of a widening range of educational software tools (such as the Widgit Communicate: series, Clicker5, BoardMaker-Speaking Dynamically Pro, and EdWord 2). So far these methods and resources have been confined within the domain of special needs education. As part of the ÆGIS project, graphic symbol support for access to text is now developed for the standard and open source office suite LibreOffice/OpenOfficeLO/OO. This task is a part of the ÆGIS ambition to include people with cognitive difficulties in the efforts towards more general accessibility in standard ICT environments. The graphic symbol support is developed as a plug-in extension primarily for Writer in LO/OO, and will build on the Concept Coding Framework (CCF)3 suggested open standard for multi-modal language support defined in the EU supported WWAAC project 4, and further developed within the Nordic SYMBERED5 and ÆGIS projects. By offering for the first time this kind of multilingual and multi-modal (text + graphic symbols + speech) as a free and open source software extension to a free, open source and cross-platform standard office package, the potential group of users who may benefit from this kind of support will be dramatically enlarged for several reasons: •

Simply by being free software – offering these features also in socio-economically disadvantaged environments where special software may not be afforded

•

By being cross-platform – this kind of functionality is currently hard to find outside of the MS Windows based software market

•

By being generally available – offering opportunities for individuals, families, schools, work settings etc. to familiarise and test whether graphic symbol support might be useful – without having to buy special software

The expectation is not that this will have any substantial negative effects on the market for commercial proprietary applications in this field, as these applications will continue to offer advantages in terms of continuous support and tight integration of features for special needs educational settings that will motivate purchase where this can be afforded.

Light, J. and McNaughton, D. (2008). Literacy and AAC. Retrieved June 20, 2009 from http://aacliteracy.psu.edu 2 Software developers/providers: http://www.widgit.com/, http://www.cricksoft.com/UK/products/clicker/, http://www.mayer-johnson.com/, http://www.oatsoft.org/Software/edword-and-edweb 3 Concept Coding Framework - CCF: www.oatsoft.org/Software/concept-coding-framework-ccf/ 4 The WWAAC project: www.wwaac.eu/ 5 The SYMBERED project: www.symbolnet.org/ 1

October 2012

Oracle

37


PU


5.2 Technology Selection OpenOffice.org, and later LibreOffice and Apache OpenOffice, was chosen for several reasons: •

It is open source

•

It is cross-platform and has a wider user base than its cross-platform, open-source alternative KOffice

•

Functionality can be added through extensions6

•

It supports RDF7 annotation (since OO.org version 3.2 and LibreOffice 3.3, and ODF (Open Document Format)8 v.1.2 and higher) which is the format used for the CCF word-toconcept-to-symbol annotations

•

It supports the Ruby Annotation9 – however not fully, why the symbols need to be transformed into glyphs in a font in order to be displayed correctly.

•

The default download of OpenOffice.org includes Java, which is used for this plugin extension

•

The CCF plugin is written using standard technology for extending LibreOffice/OpenOffice. UNO (Universal Network Objects) is the preferred method to develop extensions. This implementation of CCF is written in Java, and it interacts with UNO via the Java to UNO language bindings. A built-in JavaDB/Derby sql database10 distributed as part of the extension contains all data except the images libraries for the graphic symbol representations. In future releases it is considered to embed them as well in the database to make installation very simple.

The JavaDB (Apache Derby) database was chosen for the CCF database because of its combination of relatively full featured SQL database functionality, light weight, and suitability for integration with a Java based extension for LibreOffice/OpenOffice.

5.3 Current status The CCF LO/OO extension has been tested with users in all three pilot phases of AEGIS. GUI design and underlying technologies have been evaluated and refined continuously along three rounds of user pilot testing. In that process, a number of improvements have been developed according to the evaluation of the user feedback: Since the first version tested in the first pilot phase, a multitude of enhancements have been made, including; Symbols may now be inserted above words in the text (instead of before the corresponding words, which offered a reading experience that was not very user friendly); The separate CCF-SymbolServer panel floating on top of the document window, displaying available symbol alternatives, and offering convenient ways to select the desired concept and symbol representation; Increased symbol lookup speed; Substantially improved language support, and simplified interface and procedures for settings and installation. An important design amendment has been done by separating the basic CCF functionality from the OpenOffice.org extension and providing it in the form of the separate CCF-SymbolServer. It is 6 7 8 9 10

See, for example, the most popular extensions at http://extensions.services.openoffice.org/most_pop_ext. Resource Description Framework (RDF) - www.w3.org/RDF/ ODF – Open Document Format: www.oasis-open.org/committees/tc_home.php?wg_abbrev=office Ruby Annotation - www.w3.org/TR/ruby/ JavaDB (Apache Derby) http://netbeans.org/kb/docs/ide/java-db.html

October 2012

Oracle

38


PU


running locally on the machine where the extension is installed, and offers its service to the LO/OO extension as a client (potentially one of several, SAW 6 being the first other example). Ruby Annotation layout and new display window Since the version tested in the second phase pilots, symbol images are placed on a separate line above the written text using the symbols as glyphs implemented in a bespoke font, aligned by Ruby Annotation. The lookup mechanism is much faster, thanks to a caching mechanism, and a new mode enables visual tracking of symbols without having them inserted into the text content, but displayed in the separate GUI window of the CCF-SymbolServer (offering several optional display modes). It is possible to use the extension in this way only. Whenever placing the cursor at a word, the word's possible symbols (optionally with concept codes and definitions) are displayed in the CCF-SymbolServer GUI window.

5.3.1

Insertion and Switching between symbols/concepts

Words that are typed are looked up in the CCF concept database continuously as letters are typed, and (in Insert mode) inserted as words are ended by a Space character. Each word is (as far as matches are found) annotated with a concept code, and a code point in the Unicode Private Use Area referring to either Bliss or ARASAAC glyphs. Available graphic symbol representations that match the concept of the current word are presented and, according to the user’s preferences, inserted above the words using Ruby Annotation. When words are matched against the CCF concept database, ambiguous word-to-concept/symbol matches will frequently occur. There may also be several symbol synonyms suggested to represent a single concept (commonly occurring in pictorial systems like ARASAAC, more seldom in a systematic and semantic/logic based system like Bliss). In these cases it is possible for the user to choose which of the offered symbol representations to apply either by selecting it with a mouse click in the CCF-SymbolServer display, or by using the keyboard shortcut command (by default Ctrl+G/Cmd+G).

Illustration 11: CCF-SymbolServer window floating on top of the Writer document window, and optionally following the text cursor – symbols (here Blissymbols) are inserted above text in the 2 line Ruby Annotation format. Alternative symbol representations may be selected from the CCFSymbolServer display, or by using the shortcut key command.

October 2012

Oracle

39


PU


For AAC users who are not capable of entering text letter by letter, the text entry will typically be entered word by word, or even as multi-word phrases, by selection from an on-screen symbol display. Several commercial products for this are available on the market (such as The Grid 2, Tobii Communicator and others). Within AEGIS we have provided the upgraded version 6 of SAW (Special Access to Windows) as a free and open source alternative, which is even enhanced to use theCCF-SymbolServer and its databases for its symbol support – see Illustration 12 below.

Illustration 12: LibreOffice Writer with CCF-SymbolWriter. SAW 6 is serving as symbol supported alternative input of text, and together these applications are constituting a full AAC system.

5.3.2

Display mode

The extension now works equally well whether the user enters text – by ordinary letter-by-letter typing, or by selecting and entering whole words/phrases (with the help of Assistive Technology tools such as on-screen symbol or word charts, word prediction etc.). When stepping through the document with the cursor or placing the cursor within any given word in the text, the server window is updated and shows the symbol for the current word, regardless of the document containing symbol encoding or not.

October 2012

Oracle

40


PU


Illustration 13: No symbol insertion in text: The CCF-SymbolServer (floating on top of the text window) is displaying looked-up concepts and symbols as words are written, and/or as the text cursor is moved within the text.

5.3.3

Language and Symbol support

The CCF-SymbolServer and the CCF-SymbolServer extension for LibreOffice/OpenOffice currently support symbol representation in Blissymbolics and ARASAAC. English and Swedish text is fairly well supported, while Spanish and Dutch has a first level of partial coverage. The language setting of the CCF lookup is now automatically picking up the local language setting of the Writer document. This means that the CCF symbol lookup can now automatically cope with multilingual documents in the supported languages (See Illustr. 14 and 15) The CCF symbol supported documents supports integration with Text-to-Speech within Writer and on the system. The work to enhance the quality and coverage of the CCF resources, and move them towards ISO standardisation, will continue beyond the scope of AEGIS. The extension should work on any platform on which OpenOffice (v.3.2 or later) or LibreOffice (v.3.3 or later) runs, including but not limited to, Linux, Windows XP, Windows Vista, Windows 7 and Mac OS X. However, more extensive testing has only been done on the Windows and Mac OS X platforms, plus some limited testing on Ubuntu Linux (10.4 and later).

Illustration 14: Same multilingual document as in Ill. 13, but here a paragraph with Swedish text is processed, as opposed to the English paragraph above. October 2012

Oracle

41


PU


Illustration 15: CCF-SymbolWriter Options dialog

5.3.4

Testing (Pilots)

The CCF extension has been tested with users in all the three pilot evaluation phases of AEGIS. It has evolved from an early prototype to a mature beta version ready for its first public release after the latest updates in response to the requests from the third pilot round. These updates are reflected in the previous sections (5.3, 5.3.1 – 5.3.4) describing the current status of CCF-SymbolWriter and CCF-SymbolServer.

5.4 Overview of the application 1. Title of product: CCF-SymbolWriter for LibreOffice/OpenOffice 2. Development team: FPD (lead: Lars Nordberg, Bengt Farre, Annika Brännström) and SUDART (Mats Lundälv, Sandra Derbring) 3. Relevant WP/Activity development/integration tool place: 2.2.4 4. Purpose: To make the text based environment of a standard Office application suite – LibreOffice/OpenOffice – accessible as a productive tool also for users with more profound problems in relation to text – both in terms of writing and reading. This has been achieved by – in addition to text-to-speech reading support – providing graphical symbol support. Graphic symbols illustrate the meaning of the words as they are entered into the text, or when text content is loaded from a file. CCF-SymbolWriter is offered as a free and open source plugin extension for LibreOffice/OpenOffice 5. For which user groups and what are the expected benefits for each of them: 6. The user groups identified in the use case “Graphic Symbol Support for facilitated text October 2012

Oracle

42


PU


comprehension and production in OpenOffice.org” (deliverable D1.1.3): 7. Primarily users with Cognitive Impairments (3), but also some user with Speech and/or Hearing impairments (5), as well as some persons with Motor impairment having profound problems with text processing (2). 8. Stakeholders of many categories such as family members, teachers, tutors and employers of persons in the bespoke primary user groups, who need tools of multilingual and multi-modal representation to support these users to have better access text content. 9. Developers of software and services with similar functionality to provide products and 10. Innovations (in comparison to SoA): A mainstream (and open source) word processor can now for the first time be used together with open source and freely available graphic symbol support in the form of an extension for OpenOffice/LibreOffice Writer. 11. Major functions and User Interface aspects: 1. Major functions: Words contained in the text are picked up by the CCF-SymbolWriter extension and forwarded to the external CCF-SymbolServer, which looks up each word in its database to find matching concepts and symbol representations. These are presented in the SymbolServer display panel and optionally sent back to the CCF extension for in-line text display and tagging. The lookup is performed during writing (when each word is ended by a Space character), when the text cursor is moved through the text, or during a complete scan of a text document. The speed of the lookup is boosted by a cashing mechanism, minimising the need for new first time database searches. Two symbol libraries (Bliss and ARASAAC) are currently supported. The CCF architecture is open for the addition of more languages and symbol representations. Basic support for Spanish and Dutch is planned for the final version within AEGIS. 2. User interface aspects: The symbol support may either be used alongside with the text document, presented in the separate Symbol Server display, to support the interpretation and comprehension of text content or, if that is preferred, to produce documents with integrated symbol representation . This is achieved by making use of the existing standard Ruby Annotation format supported by ODF/ODT for double layered text formatting and presentation. When several word to concept matches and/or symbol representations are available, the user may select another than the system's default choice by using a shortcut key command. The symbol support outside of, or within, the documents presents no barriers for combinations with other AT, such as TTS reading support, or alternative input via onscreen-keyboards etc., including on-screen symbol displays for symbol based whole word input (making it possible to use Writer as a graphic symbol editor for AAC symbol users). 12. Relevant Use Cases (from D1.1.3): 1. “Graphic Symbol Support for facilitated text comprehension and production in OpenOffice.org” in D1.1.3 13. Level of achievement of defined SP2 general requirements Table 7: Current level of achievement of defined requirements for CCF

October 2012

Oracle

(see

D2.1.1):

43


PU


SP2 relevant Requirements Achieved (Yes/No/Partially and short justification) Relevant User Requirements (from Chapter 3 of D2.1.1) UR-C1: Ability to review information. Use visual examples (diagrams, icons, drawings) in addition to text descriptions.

Yes. The complementary representation of text with symbols provides the user with additional modality for reviewing the meaning of the content. It can also be used as a visual-symbolicpictorial spell check.

Support spell check. UR-C2: Users with dyslexia Yes. By complementing the typical TTS reading support with an or reading impairments option to verify the meaning of words with a visual alternative would like to use a Desktop representation. PC as a helping tool that will read them books and articles. UR-C5: The software’s help menus and some applica tions installation should be easy to use or understand, as well as the confusing error messages that the computers show.

Yes: The CCF Symbol Support extension and server user interfaces are minimalistic and simple. Further simplifications of the CCF menu and Toolbar, the way to access the settings dialog, and of its content, have been made for the final version.

UR-C6: The lack of transla tion to mother tongue in certain applications or pro grams, make it especially difficult for this group of users to use the desktop applications.

Yes/Partially: The multilingual and multi-modal capacities of the CCF extension provides a potential for providing translation and alternative interpretation support across second and first language text content. The CCF extension is currently available with good support for English and Swedish, and partial support for Spanish and Dutch. The coverage of these languages will be expanded, and more languages will be added in later releases after AEGIS.

UR-S1: Users with dyslexia Yes: see UR-C2 above. or reading impairments would like to use a Desktop PC as a helping tool that will read them books and articles. Relevant functional requirements (from section 4.2.2 of D2.1.1)

October 2012

Oracle

44


PU


SP2 relevant Requirements Achieved (Yes/No/Partially and short justification) F-OOCCF-1. The plugin Yes... in Writer as specified. (In Impress so far only by inserting a should offer users the option Writer DDE text object, instead of the standard text frames.) of writing and reading text with support of graphic symbol representations within OpenOffice.org (primarily Writer, if possible Impress and other OO.org applications). F-OOCCF-2 CCF Support for annotations: The plugin will allow a loaded supported document file to be annotated with concept codes – utilising RDF metadata – to allow graphic symbol representation of matched contained words, and to be saved as a concept coded document.

Yes/Partially: RDF metadata annotation is currently only applied when symbols are inserted as images in front of their corresponding words, but not when displayed above words in the Ruby Annotation font format. To be revised for future versions.

F-OOCCF-3 CCF swapping between text/symbol support: The plugin should provide convenient swapping between text/symbol show/hide options, for both display and print-out of concept coded documents.

No: Not yet

Relevant non-functional requirements (from section 4.2.5 of D2.1.1) Maintainability. Those ex tensions will reside and work in the OpenOffice.org envi ronment – and should work everywhere that OpenOffice .org works (e.g. the Open Desktop, Macintosh, Win dows). With any change on the environment, the plugins should be updated too, to support these changes.

October 2012

Partially. The extension is tested for compatibility when a new version of OpenOffice.org or LibreOffice becomes available. So far, there has been only one change in OpenOffice.org that affected compatibility with extensions; This has so far not effected the CCF Symbol Support extension. When The Document Foundation started LibreOffice, they also made a list of existing open source extensions for OpenOffice.org; there seems to be no intention of breaking compatibility for extensions. However, a user-friendly install package and procedure is still only available for MS Windows (XP+), and installation on different Linux distributions have not been extensively tested.

Oracle

45


PU


SP2 relevant Requirements Achieved (Yes/No/Partially and short justification) Changeability. It should be easy to add new functio nality to the extensions, as long as there are changes on the standards supported (like DAISY).

Yes. The split of the CCF support functionality into a) a stand-alone CCF-SymbolServer, and b) the CCF-SymbolWriter extension, has minimised the footprint of the actual extension, and made the overall CCF symbol support functionality more modular and flexible.

Stability. Stability is also important, because it impacts the stability of the office suite as a whole.

Yes The CCF Symbol Support extension is depending on Java. During the 3rd phase beta testing this caused installation problems for 64 bit Windows 7 systems due to immaturity in the general Java support. These have now been overcome. Otherwise good stability.

Testability. Because of the frequent changes on both their code base and the Open Office code base, the testa bility of these extensions will be a top priority.

Yes. Changes in the OpenOffice.org code base has so far not impacted extensions compatibility (see Maintainability, above). The CCF Symbol Support extension creates logs of actions and function calls during its operation in order to facilitate debugging in case of errors.

Reliability. Because the (See comments to sub-attributes below.) plug-ins will work on a highly interactive envi ronment, they should be reliable. The following sub attributes present this in more detail. Maturity. They should act Yes. The foreseen basic functionality of the CCF Symbol Support as a mature product, with a extension is now established and functioning in a stable fashion. small number of failures. (The details of the RDF concept coding annotation in ODT may need another revision, the option of switching the display of symbols on/off in an annotated document remains.) Substantial UI improvements have been made. Error Tolerance. In a case Yes The CCF extension relies on the functioning of Writer and the of an error, it should be able office suite for display and other basic functions, including error to recover, with an automatic recovering etc. restart of the system or with a modal dialog, with the last status. Data Integrity. In case an Partially. See comments above, but remaining issues with the RDF error occurs, the integrity of annotation format details for concept coding needs to be clarified. the data should not be lost. Relevant technical specifications (from section 4.2.2 of D2.1.1)

October 2012

Oracle

46


PU


SP2 relevant Requirements Achieved (Yes/No/Partially and short justification) F-OOCCF-1 CCF Support Yes/Partially. Ongoing refinements for improved quality of concept for OpenOffice.org: TS1. and representations lookup – still more limited coverage for the new The Bliss and ARASAAC symbol fonts display, and in particular for ARASAAC symbols. symbol systems will be supported based on the CCF. Support for additional symbol systems may be added as time and resources allow. F-OOCCF-1 CCF Support for OpenOffice.org: TS2. Initially English and Swedish text will be supported. Support for additional languages will be added as far time and resources allow

Yes. Support also for Spanish and Dutch – on a basic level – has been added. The CCF language setting is now picking up the language settings of the Writer document. This means that multilingual documents (in the supported languages) are smoothly handled by the CCF word-to-concept and symbol representation look-up

F-OOCCF-1 CCF Support Yes. Substantial improvements and simplifications have been made for OpenOffice.org: TS3. to further improve the UI usability and accessibility Preference settings will allow choice among supported languages and symbol representation systems. F-OOCCF-3 CCF swapping between text/symbol support: TS4. When there is an ambiguity in the word-to-CC look-up, the user should be offered a user-friendly interface for viewing and selecting alternative representations.

October 2012

Yes. The support symbol display in annotated documents is now implemented. The feature and interface for viewing and choosing between alternative symbol representations is now provided in the form of the separate CCF Symbol Server display floating on top of the text document window. Alternative symbol representations may be selected either by clicking on the preferred symbol in the CCFSymbolServer display, or by flipping through alternatives with the shortcut key command (which can now be set by the user).

Oracle

47

AEGIS D2.2.1b_final

PU


5.5 Relationship to the OAF

November 2011

Oracle

48

AEGIS D2.2.1b_final

PU


Creation Define Accessible GNOME Accessibility API - AT-SPI for ATs - ATK for GNOME apps - Bridged JA-API GNOME Keynav - Defines keynav for all UI element types - Defines keynav for desktop GNOME Themeing - Theme engine, defines focus indicator, I-beam, other things OpenDocument Format - Defines necessary document metadata for accessibility

Stock Elements GTK+ widgets - implement ATK (via GAIL) - support keynav - support themeing XUL widgets (FF, TB) - implement ATK (via bridge) - support keynav - support themeing UNO widgets(OOo) - implement ATK (via bridge) - support keynav - support themeing

Java Swing widgets - implement ATK (via bridge) - support keynav (for GTK+ L&F) - support themeing (for GTK+ L&F)

Use Dev/Auth Tool

Platform Support

NetBeans IDE - Developer tool - Aids Java Swing dev. of accessible apps

AT-SPI & ATK libraries

AccessX options - Ship with GNOME, StickyKeys, MouseKeys, Etc. (arguably an AT)

Glade UI designer - Developer tool - Aids GNOME/GTK+ dev. of accessible UIs

Theme switcher - Enables user to choose theme

OpenOffice.org / LibreOffice

- Authoring tool - Ability to annotate documents with accessiblity metadata - Export to PDF, HTML, DAISY preserving accessibility metadata - Print to Braille - Accessiblity validation (for DAISY at least) - Create documents annotated with concept coding RDF metadata and CCF symbol representation

AT app launcher - Enables user to choose AT to use at login Built-in AT - Orca, GSMag, eSpeak, Dasher, all part of GNOME by default - Other AT (e.g. Caribou) free download linked from desktop

The App Itself

Assistive Technology

Huge numbers of apps are accessible, either shipping with GNOME desktop or linked as a free download

Orca screen reader - Blind & low-vision AT

- GTK+ apps, - Firefox, Thunderbird - OpenOffice.org - Java Swing apps

GSMag magnifier - Built into GNOMEShell, used by Orca eSpeak TTS - Used by Orca, others BRLTTY braille library - Used by Orca Dasher text entry & Caribou on screen kbd - Physical impaired AT Opengazer library - Eye & gesture tracker, used as mouse by Dasher, Caribou, ...

CCF-SymbolServer - Serves users with graphic symbol representation of text content – via the CCF-SymbolWriter extension for LO/OO, ... SAW 6 (a.o. apps)

Illustration 16: OAF view of Desktop, as seen through the 6 steps of accessibility (3 for creation, 3 for use). Text in bold, italicized red indicates where the CCF support in A2.2.4 delivers into the OAF

November 2011

Oracle

49


PU


6 A2.2.4 Built-in support for accessibility in a cross-platform office suite 6.1 odt2daisy: "Save as DAISY" 6.1.1

Background

The Description of Work (DoW) calls for building direct support for DAISY digital talking books into an open-source office suite. With current products and tools, creating DAISY books is usually a multi-stage process: first, the source document is created in, for example, a word processing application, then, a separate application converts the source document into a DAISY book (for a list of DAISY production tools, see DAISYTools). Any changes made to the source document require a new conversion in the second application. Further, any formatting issues arising from poor choices made in the original (spaces used instead of TABs for alignment, poor or no notation of page breaks, tables lacking clear table headers, etcetera) cannot simply be fixed in the derivative edition, else they will be lost with the next update to the source. The alternative is using dedicated software for producing audio books, such as the DAISY Consortium’s Obi [DAISYObi] and Tobi [DAISYTobi] or the commercial product Dolphin EasyProducer [DolphinProducer], but this type of software is primarily intended for DAISY production centres, not for end users. The goal is therefore to adapt a mainstream office suite to enable users to create DAISY books within a single and more familiar software environment. This has the additional advantage that any accessibility problems that result from the conversion process can be addressed in the same software environment.

6.1.2

Technology Selection

OpenOffice.org was chosen for several reasons: •

it is open source,

•

it is cross-platform and has a wider user base than its cross-platform, open-source alternative KOffice (and its more recent derivative Calligra Suite),

•

functionality can be added OpenOfficePopularExtensions].

through

extensions

[OpenOfficeExtensionsDev,

The “Save as DAISY” extension targets OpenOffice.org version 3.0 and higher. Older versions of OpenOffice.org are no longer available from the official website because the have reached “End-OfLife” status [OpenOfficeEOL]. The default download of OpenOffice.org before it was transferred to Apache included Java11, which is one of the languages that can be used for developing extensions (other languages include C++ and Python) [OpenOfficeExtensionsDev]. See section 6.4 Compatibility of the Extensions with LibreOffice for a discussion on compatibility with LibreOffice. 11 The release notes for Apache OpenOffice.org 3.4.1 state “Apache OpenOffice 3.4.0 and 3.4.1 support Java 7, which is the recommended configuration; but (especially on 64-bit Windows) you might receive warnings about the Java version being defective. In that case, download and install the most current JRE 6 version. Make sure you get the file "Windows x86 Offline (32-bit)". Then configure OpenOffice to use it at "Tools - Options - OpenOffice.org Java".”. See http://www.openoffice.org/development/releases/3.4.1.html October 2012

Oracle

50


6.1.3

PU


Save as DAISY Functions

odt2daisy should generate a DAISY book with full audio. DAISY 3 (ANSI/NISO Z39.86-2005) is the primary version; DAISY 2.02 support would be useful for compatibility with older DAISY players (mostly hardware players), which are still in use. odt2daisy offers two basic export options: save as DAISY XML (i.e. content is converted to DAISY markup but no audio is generated) and save as “Full DAISY” (with audio). The DAISY XML produced by the first option must be valid and suitable for use in a DAISY production tool such as Obi. If the resulting book contains audio, then the audio must be understandable and synchronised with the text. The resulting book must be playable on any DAISY compliant player – for DAISY 3 books by opening the speechgen.opf file, and for DAISY 2.02 books by opening the ncc.html file.

(Some DAISY players, for example AMIS, display only the speechgen.opf file when browsing the content of a DAISY 3 folder.) odt2daisy should support the creation of DAISY books from OpenDocument Text. Ideally, it should also support OpenDocument Presentations as a source format for DAISY books. Supporting OpenDocument Spreadsheets may be less useful due to the primitive handling of tables in many DAISY players: table cells are read in succession without distinction between header cells and table cells, without announcing new rows and without offering the possibility to query the header cell(s) for a data cell. The DAISY conversion process would need to add text or code that compensates for these limitations. One issue that complicates support for spreadsheets and presentations is that OpenOffice.org Calc and OpenOffice.org Impress (respectively) do not support language identification, which is an essential requirement for text-to-speech conversion The extension should prompt the user to input the necessary metadata for the book (Creator, Publisher, Producer). The extension should work on any 32-bit operating system where OpenOffice.org works, provided that there is an available TTS engine that can be used by the extension. The limitation to 32-bit operating systems is due to the fact that the DAISY Pipeline Lite is not available for 64-bit operating systems. Note that there needs to be a TTS “voice” or engine for each language used in the source document. The extension should support mathematical content through support for Mathematical Markup Language (MathML), the scientific format used in both OpenDocument Text and DAISY 3. (DAISY 2.02 did not support mathematical content.) Note that the DAISY Pipeline Lite cannot convert documents containing mathematics without a plugin to create alternative text and images. The DAISY Pipeline Lite asks users to visit http://www.daisy.org/projects/pipeline/other-addins.shtml for information on such plugins for the DAISY Pipeline. The only available add-in is the commercial MathDaisy add-in by DAISY Consortium member Design Science; this add-in works only on Microsoft Windows.

6.1.4

Requirements for Source Documents

In order to create valid and usable DAISY books, the source documents need to fulfil a number of criteria, including: • Language identification for the document as a whole and for any language changes inside the document. October 2012

Oracle

51


PU


A title (with the Title style in Writer) and correctly used headings (with heading styles that correctly identify the hierarchy of the sections: “Heading 1”, “Heading 2”, etcetera. • Tables with correct styles for table headings and table content. Merged cells and nested tables should be avoided. Rows must not break across pages and columns. • Sufficient colour contrast between text and background in images and other objects (see WCAG 2.0 success criterion 1.4.3 and tools such as the Paciello Group’s Contrast Analyser). Colour must not be used as the only visual means of conveying information (see WCAG 2.0 success criterion 1.4.1). When DAISY books are used on players without a screen, colour contrast in images is not relevant, but DAISY books can also be opened in players that produce visual output (some even with highlighting synchronised with the synthesised speech), for example, AMIS and emerson-reader. • Images must be in PNG or JPEG format; DAISY does not support GIF. For vector images, SVG can be used. • Images and other non-text media need to have text alternatives that convey the same meaning. For more detailed requirements, see the course “Creating Accessible PDF Documents with OpenOffice.org Writer”, in the AEGIS training platform (http://aegis.bluepoint-it.ro/login.php? course=42) or the techniques documents by the Accessibility of Office Documents (ADOD) Project at http://adod.idrc.ocad.ca/. Users would benefit from built-in support for accessible authoring; the OpenOffice.org / LibreOffice Accessibility Checker (see 6.3 OpenOffice.org Accessibility Checker) fills this gap. •

6.1.5

Current Status of odt2daisy

odt2daisy 2.0 was released on 9 November 2009 and is available on the open-source repository SourceForge: http://odt2daisy.sourceforge.net/ (site for binary downloads and documentation) and http://sourceforge.net/projects/odt2daisy/ (the developer site). This version succeeded odt2dtbook, which is no longer available. odt2daisy is available under the Lesser General Public License (LGPL) version 3.0 or higher. odt2dtbook supported export to DAISY 3 XML (no speech synthesis), most of MathML and contained templates (with localisation instructions) that were designed for DAISY authoring. odt2daisy 2.0 added the following: •

A suite of 160 test documents that test conversion of ODT features. These test files were integrated with XMLUnit and JUnit and serve as unit tests. The are also used for regression testing.

•

More than 20 bugs were fixed; these bugs were detected by unit tests or by other means.

•

Export to DAISY 2.02 was added for compatibility with older DAISY players.

•

The DAISY Consortium’s DAISY Pipeline Lite was integrated to enable the production of full audio on Window, Mac OS and Linux/Unix. Before October 2009 audio support was limited to monolingual content; a later version of the DAISY Pipeline Lite enabled support for multilingual content. Multilingual support is now available, provided that the languages in the document are correctly identified and that the system where conversion takes place has text-to-speech engines for those languages.

October 2012

Oracle

52

AEGIS D2.2.1c_final_opendesktoplibraries •

PU


6 screen casts that serve as tutorials on the installation and use of odt2daisy.

odt2daisy 2.1 was released on 4 April 2010 and added the following: •

Compatibility with OpenOffice.org 3.2. (This was a documented issue for several OpenOffice.org extensions.)

•

Changes in localisation.

Changes after the release of version 2.1 (reported in advance in M30 version of D2.2.1a) include: •

The bitrate combobox in the odt2daisy dialog was replaced with a drop-down list to prevent input of incorrect bitrate values by the user.

•

Code to detect at run-time whether odt2daisy is installed inside OpenOffice.org or LibreOffice.

•

Support for long descriptions of images. This enhancement was requested by a DAISY and Braille production centre in Flanders.

•

MathML export failed due to a namespace prefix change in ODF; this issue is now resolved.

•

If “Title” in the document properties (see File > Properties > Description in Writer) is empty, odt2daisy uses the text from the first Title element as a suggestion for the title of the DAISY book in the odt2daisy dialog window (feature request 3086596).

•

Better handling of title pages; unit tests for title pages were modified to test correct conversion of title pages.

•

Table caption are no longer converted to a normal paragraph (above or below the table, depending on their location in the source document) but to a caption element inside the table element that is always rendered at the top of the table (as required by the DAISY specification). New test files with table captions for unit testing and regression testing were added to the test suite.

•

Tables with a heading column (in addition to a heading row) no longer drop the first row during conversion (bug 3171439). New test files with heading columns for unit testing and regression testing were added to the test suite.

•

Support for more Dublin Core and DTBook metadata (for example, dtb:sourcePublisher, dtb:sourceEdition, dtb:sourceDate). This enhancement was requested by a DAISY and Braille production centre in Flanders. New test files with metadata for unit testing and regression testing were added to the test suite.

•

When processing the default language of an ODF file, odt2daisy 2.1 only considers the “Western language”. Using an “Asian language” or “CTL language” (complex text layout) as a default language is now supported under certain conditions: when using an “Asian language”, the “Western language” must be set to “None” in Writer and all instances of a Western languages in the document must be identified individually; when using a “CTL language”, the “Western language” and the “Asian language” must be set to “None” (even when “Asian language is disabled) and all instances of a Western languages and Asian languages in the document must be identified individually. These special conditions

October 2012

Oracle

53


PU


originate from the way languages are tagged in ODF: with the attributes fo:language and fo:country for Western languages (typically languages with an alphabet, such as Latin, Cyrillic and Greek), style:language-asian and style:country-asian for Asian languages (Chinese, Japanese, Korean and Vietnamese), and style:language-complex and style:country-complex for CTL languages (Arabic, Hebrew and other languages that use an abjad or an abugida as their script). So far, this has been tested only with Mandarin Chinese content. •

odt2daisy detects image formats not supported by DAISY before starting the DAISY Pipeline Lite; the resulting error dialog provides the names of the images that need to be replaced.

•

Spanish translation of the user interface; improved German, Dutch and Chinese (simplified characters) translation.

•

More JavaDoc documentation in the Java source code to lower the threshold for outside contributors.

odt2daisy is available for Microsoft Windows, Mac OS X, Linux and Solaris, but not on 64-bit operating systems (because the DAISY Pipeline Lite is not available for 64-bit operating systems). It requires OpenOffice 3.0 or higher. More detailed information, for example on supported features, is available at http://odt2daisy.sourceforge.net/support/ (list of supported features) and http://odt2daisy.sourceforge.net/doc/ (manuals, screencasts, localisation documentation, developer documentation, publications). odt2daisy has been integrated into the DAISY Pipeline [DAISYPipeLineScript], into Dedicon’s altText conversion portal and into Create&Convert by the JISC Regional Support Centre Scotland North & East [JiscCreate&Convert]. Dedicon also submitted their changes to the odt2daisy project. odt2daisy is available from the OpenOffice.org extension website [OpenOfficeDAISY]. odt2daisy is also compatible with LibreOffice; it is not clear how long LibreOffice will maintain compatibility for extensions with OpenOffice.org. odt2daisy has also been added to the LibreOffice Extensions website [LiboDAISY]. 6.1.5.1 Changes since D2.2.1b

Since development effort moved to odt2braille and AccessODF, the changes in odt2daisy have been fairly small: •

odt2daisy was released in November 2011, together with a new version of odt2braille and the first version of AccessODF. This release includes better support for tables and table captions, long descriptions, multi-lingual documents and non-Western languages. (These improvements were described in the previous section.)

•

Work has started on “odt2daisy 3”, which focuses on improving the performance for converting long documents (more than 100 pages). This is mainly done by using XSLT 2.0 instead of XSLT 1.0. This code is available at http://odt2daisy.svn.sourceforge.net/viewvc/odt2daisy/odt2daisy3/ but has not yet been integrated into the main trunk.

October 2012

Oracle

54


PU


•

Work has started on a mechanism to allow users select the text-to-speech engine (if more than is available for a language used in the document) and certain parameters of the text-tospeech engine, since this was a frequently requested feature from the evaluations. This improvement requires working around a limitation in the DAISY Pipeline Lite and needs some more testing and the development of a user interface before it can be integrated into odt2daisy.

•

Translations of the user interface were slightly improved, and a Greek translation was added.

6.1.6

Testing (Pilots)

odt2daisy was included in the first evaluation phase as a mature prototype. Comments and evaluation results led to the following recommendations or improvement requests: •

Give users more control over the voice(s) used for text-to-speech: which voice, and possibly also pitch and speed. (This could be part of “advanced settings.) Note: this requires changes in the DAISY Pipeline Lite, which is outside the control of the AEGIS project.

•

Add an evaluation of document structure or accessibility before conversion, as output quality depends on the quality of the source document. Note: accessibility evaluation is now part of a separate prototype, which is internally available as a separate extension for OpenOffice.org Writer (or LibreOffice Writer); see 6.3 OpenOffice.org Accessibility Checker.

•

Improve the choosing and finding of the location of the output DAISY files.

•

Warn the user if the source document uses a language for which the system has no text-tospeech engine installed. Note: this depends on detection of available TTS engines, which is currently done by the DAISY Pipeline Lite. The Pipeline would need to be modified for odt2daisy to get a list of available languages.

•

Provide continuous feedback about the current stage in the execution of tasks. Note: this is already available, except for the part between the DAISY export dialog and the start of the DAISY Pipeline Lite. The DAISY Pipeline Lite provides continuous feedback (both progressbar and text).

•

Make the icons for Full DAISY and DAISY XML in the File menu easier to distinguish. Note: the menu items in the File menu are called “Export as DAISY XML” and “Export as Full DAISY”, so selecting the correct item does not rely on the recognition of icons.

•

Add PDF-to-DAISY functionality. Note: this is far beyond the plans for odt2daisy. (The PDF import extension only imports PDF documents into OOo Draw, i.e. not in a format that can serve as a source for DAISY or other accessible documents.)

•

Remove Pipeline Lite result message if they are not error messages. Note: the DAISY Pipeline Lite does not seem to have parameters to display only error messages. This can be passed on to the developers of the DAISY Pipeline 2.

6.1.7

October 2012

Overview of the application

Oracle

55


PU


1. Title of product: odt2daisy / OpenOffice.org Export As DAISY 2. Development team: Katholieke Universiteit Leuven (K.U.Leuven) - DocArch 3. Relevant WP/Activity development/integration tool place: 2.2.4 4. Purpose: Enabling both end users and DAISY production centres to export OpenDocument Text (ODT) to a digital talking book in the DAISY format. This includes support for multilingual content and for both DAISY 2.02 and DAISY 3.0. The prototype turns OpenOffice.org or LibreOffice Writer into a DAISY production tool, which should lower the barrier to DAISY production. 5. For which user groups and what are the expected benefits for each of them: The user groups identified in the use case “Full DAISY book creation in OpenOffice.org” (deliverable D1.1.3): •

1.b Blind (without useful residual vision): blind users should benefit because making DAISY production easier should increase the number of DAISY books available to them. When OpenOffice.org and LibreOffice solve their own accessibility issues on Microsoft Windows (through implementation of the iAccessible2 API), DAISY production should also become available to blind users.

•

3.a. Slight cognitive limitation and low support need (Users with dyslexia/reading impairments): persons with dyslexia can benefit from the availability of DAISY books, especially when read in a player that supports highlighting that is synchronised with the synthetic speech.

•

Production centres for accessible document formats benefit through the ability to use the OpenDocument Format as a single source for different output formats, typically DAISY and Braille (a production centre in Flanders is working with K.U.Leuven to enable this workflow). Some production centres use odt2daisy simply to save ODT as DAISY XML and import the DAISY XML file into a tool such as Tobi, in which sentence highlighting can be synchronised with the recording of a human voice. Other production centres benefit through the availability of odt2daisy in the DAISY Pipeline, which thereby supports ODT to DAISY conversion.

6. Innovations (in comparison to SoA): A mainstream (and open source) word processor can now be used as a DAISY production tool. (Save as DAISY for Microsoft Word was not available when the AEGIS project was submitted to the European Commission.) 7. Major functions and User Interface aspects: 1. Major functions: Convert OpenDocument Text (ODT) to DAISY 3.0 XML, to Full DAISY (with support for multilingual content) and to DAISY 2.02 (for compatibility with older DAISY players). 2. User interface aspects: The odt2daisy export dialog is keyboard accessible, but due to limitations in OpenOffice.org accessibility on Windows (OpenOffice.org on Windows uses the Java Accessibility API, which is poorly supported by Windows screen readers) and limitations in the OpenOffice.org API for the creation of user interfaces for extensions (for example, labels cannot be linked explicitly with controls), access by blind users on Windows varies between hard and impossible. Another part of the user interface is the progress dialog generated by the DAISY Pipeline Lite; this user interface October 2012

Oracle

56


PU


uses Eclipse Foundation’s Standard Widget Toolkit (SWT). 8. Relevant Use Cases (from D1.1.3): 1. “Full DAISY book creation in OpenOffice.org” in D1.1.3 9. Level of achievement of defined SP2 general requirements (see Table 8: Current level of achievement of defined requirements for odt2daisy

October 2012

Oracle

D2.1.1):

57


PU


SP2 relevant Requirements Achieved (Yes/No/Partially and short justification) Relevant User Requirements (from Chapter 3 of D2.1.1) UR-V4: In general for all users with visual impair ments, the compatibility of ATs when using certain operating systems is essen tial to guarantee their access to the computers.

Partially. Screen reader access to odt2daisy on Windows is limited because of two reasons: (1) OpenOffice.org (and LibreOffice) uses the Java Accessibility API to expose accessibility metadata on Windows but this API is poorly supported by screen readers on this operating system. (IBM donated an IAccessible2 implementation for OpenOffice.org 3.1 to Oracle, but this code had not been fully integrated by the time Oracle donated the OpenOffice.org codebase to the Apache Foundation). The accessibility of OpenOffice.org is better on the GNOME desktop and on Mac OS. (2) The OpenOffice.org UNO API allows extension developers to create user interfaces, but this API does not enable explicit linking between controls and their labels, so screen readers need to infer labels based on their positioning in the user interface.

UR-V5: Blind users need Partially. See UR-V4. alternative output mecha nisms. These users need to have full or close to full accessibility to their desktop PC, with the use of their other senses like touch (Braille) and hearing (Screen Reader - TTS), in order to circumvent their vision disability. UR-V6: They demand good Partially. See UR-V4. compatibility of the screen readers with dynamic or graphical applications or programs. UR-V7: Blind users require more natural voices in the speech outputs. Computer ized voices without emotion and intonation is a frequent problem for new users or children who have to rely and “communicate” with a computer voice all day long. When that voice does not sound natural, this affects the acceptance to the assistive technology (AT).

October 2012

Partially. The quality of the computer voices on a user’s system is outside the control of odt2daisy. For this requirement, odt2daisy depends on improvements in eSpeak (4 A2.2.3 Open Text-to-Speech multilingual functionality).

Oracle

58


PU


SP2 relevant Requirements Achieved (Yes/No/Partially and short justification) UR-C2: Users with dyslexia or reading impairments would like to use a Desktop PC as a helping tool that will read them books and articles.

Yes, though indirectly: odt2daisy enables the export of DAISY books in which sentences are highlighted when the corresponding part in the recording (MP3) is rendered, but it does not enable textto-speech in OpenOffice.org itself.

UR-C5: The software’s help Yes: The odt2daisy user interface is rather minimal (i.e. a simple menus and some applica export options dialog). tions installation should be easy to use or understand, as well as the confusing error messages that the computers show. UR-C6: The lack of transla Yes/Partially: odt2daisy is available in English, French, Dutch, tion to mother tongue in German, Spanish, Catalan, Swedish and a few other languages, but certain applications or pro more volunteer contributions are needed. grams, make it especially difficult for this group of users to use the desktop applications. UR-S1: Users with dyslexia Yes: see UR-C2 above. or reading impairments would like to use a Desktop PC as a helping tool that will read them books and articles. Relevant functional requirements (from section 4.2.2 of D2.1.1) F-OOD-1. The DAISY Yes. Installing odt2daisy adds two items to the File menu: “Export extension should add a as DAISY XML” and “Export as Full DAISY”. “Export as DAISY” option in the OpenOffice File menu. The extension should generate a DAISY book with full audio. Relevant non-functional requirements (from section 4.2.5 of D2.1.1)

October 2012

Oracle

59


PU


SP2 relevant Requirements Achieved (Yes/No/Partially and short justification) Maintainability. Those ex tensions will reside and work in the OpenOffice.org envi ronment – and should work everywhere that OpenOffice .org works (e.g. the Open Desktop, Macintosh, Win dows). With any change on the environment, the plugins should be updated too, to support these changes.

Yes. The extension is tested for compatibility when a new version of OpenOffice.org or LibreOffice becomes available. So far, there has been only one change in OpenOffice.org that affected compatibility with extensions; this issue was fixed on odt2daisy 2.1 (released April 2010). When The Document Foundation started LibreOffice, they also made a list of existing open source extensions for OpenOffice.org; there seems to be no intention of breaking compatibility for extensions. There have been issues to make LibreOffice and OpenOffice.org work with the first releases of Java 7, but these issues are outside the scope of odt2daisy.

Changeability. It should be easy to add new functio nality to the extensions, as long as there are changes on the standards supported (like DAISY).

Yes. The mapping of ODF to DAISY is almost entirely done through XSLT. This requirement implies checking changes to the ODF and DAISY standards. ODF 1.2 was released by OASIS in September 2011, and the successor of DAISY 3 in July 2012 (note that ANSI/NISO Z39.98-2012 has a different number than ANSI/NISO Z39.86-2005, which was reaffirmed for another five years.) odt2daisy has not been updated to these new standards because development effort has shifted to other extensions.

Stability. Stability is also Yes. odt2daisy is mostly written in Java, which has its own important, because it impacts exception handling mechanism. No stability issues have been the stability of the office reported. suite as a whole. Testability. Because of the frequent changes on both their code base and the Open Office code base, the testa bility of these extensions will be a top priority.

Yes. Changes in the OpenOffice.org code base have impacted extensions compatibility only once (see Maintainability, above). The odt2daisy code repository contains more than 160 unit tests to check correct conversion from ODF to DAISY 3.0. odt2daisy creates logs of actions and function calls during every conversion process in order to facilitate debugging in case of errors.

Reliability. Because the (See comments to subattributes below.) plug-ins will work on a highly interactive envi ronment, they should be reliable. The following sub attributes present this in more detail.

October 2012

Oracle

60


PU


SP2 relevant Requirements Achieved (Yes/No/Partially and short justification) Maturity. They should act Partially. In odt2daisy 2.1, using image formats that are not as a mature product, with a supported by DAISY led to an error at the end of the conversion small number of failures. process instead of a warning before the conversion process. However, the ODF Accessibility Checker provides a warning for this. odt2daisy 2.1.2 warns users about unsupported image formats before the conversion to DAISY. Also, some Linux users reported issues with missing or greyed out menu items for odt2daisy in the File menu; a patch for this has been integrated in the code repository. Error Tolerance. In a case of an error, it should be able to recover, with an automatic restart of the system or with a modal dialog, with the last status.

Yes. odt2daisy provides only an export function, and errors during the export process don’t affect the functioning of the office suite. odt2daisy 2.1 does not prevent the input of an incorrect bitrate for DAISY production, but this issue has been fixed in the code repository.

Data Integrity. In case an Yes. odt2daisy does not modify the ODF content before or during error occurs, the integrity of the export process; errors during the process don’t affect the ODF the data should not be lost. content. Relevant technical specifications (from section 4.2.2 of D2.1.1) F-OODB-1: TS1. The extension should prompt the user to input the necessary metadata for the book (Creator, Publisher, Producer).

Yes. These metadata are part of the export options dialog and are saved as metadata in DAISY (dc:Creator, dc:Publisher and dtb:producer, respectively). The next version of odt2daisy will also export other metadata fields defined by DAISY when the author uses these fields in the document properties.

F-OODB-1: TS2. It should support the creation of DAISY books from Open Document Text. (Open Document Spreadsheets and OpenDocument Presen tations are currently no can didate source formats for DAISY books: Open Office.org Calc and Open Office.org Impress (respec tively) do not support language identification, which is an essential require ment for text-to-speech conversion.)

Yes. Note that the OpenDocument Text (ODT) files need to fulfil certain criteria, for example: images should be in PNG or JPG format (GIF, BMP and other binary formats are not supported in DAISY 3.0), the document language and any language changes need to be correctly identified, images and other objects need text alternatives, etcetera.

October 2012

Oracle

61


PU


SP2 relevant Requirements Achieved (Yes/No/Partially and short justification) F-OODB-1: TS3. The resulting book must be played on any DAISY compliant player – for DAISY 3 books by opening the speechgen.opf file, and for DAISY 2.02 books by opening the ncc.html file.

Yes. Testing is mainly done with the DAISY Consortium’s AMIS and with emerson-reader (both open source). Users have not registered bugs related to compatibility with DAISY players.

F-OODB-1: TS4. If the resulting book contains audio, then the audio must be understandable and synchronised with the text.

Yes. However, audio quality depends on the quality of the voices on the user’s system and is outside the control of odt2daisy. Synchronisation between highlighting and audio is available at the sentence level.

F-OODB-1: TS5. The extension should work on any 32-bit operating system where OpenOffice.org works, provided that there is an available TTS Engine that can be used by the extension. Note that there needs to be a TTS “voice” or engine for each language used in the source document.

Yes.(Availability on 64-bit operating systems depends on the DAISY Pipeline Lite user interface, which uses the Standard Widget Toolkit (SWT) and which requires a 32-bit system. Update: The DAISY Pipeline Lite does not work on a 64-bit JVM. Newer versions of the DAISY Pipeline Lite include a launcher that detects the presence of a 32-bit JVM and then tries to use that JVM. This requires that users have a 32-bit JVM on their 64-bit operating system. More testing is needed before this can be made available.)

F-OODB-1: TS6. The ex tension should support mathematical content through support for Mathe matical Markup Language (MathML), the scientific format used in both Open Document Text and DAISY 3. (DAISY 2.02 did not sup port mathematical content.)

Yes. MathML is supported. Due to a namespace prefix change for MathML in ODF, mathematical content was not supported in odt2daisy 2.1 (released in April 2010), but support was been made available again in odt2daisy 2.1.2.

6.1.8

Relationship to the OAF

See 6.5 Relationship of odt2daisy, odt2braille and the ODF Accessibility Checker to the OAF, below.

October 2012

Oracle

62


PU


6.2 odt2braille: "Export to Braille" 6.2.1

Background

The Description of Work (DoW) calls for building direct support for Braille into an open-source office suite. With current products and tools, printing books in Braille is usually a multi-stage process. First the document is created in a word processor. Then, a separate and specialized application is used to translate and reformat the original document so that an accessible edition can be produced. This derivative document can then be independently printed or brailled. However, any changes the author makes to the original means that the derivative edition must be re-created. Further, any formatting issues arising from poor choices made in the original (spaces used instead of TABs for alignment, poor or no notation of page breaks, tables lacking clear table headers, etc.) cannot simply be fixed in the derivative edition, else they will be lost with the next update to the source. However, as the state of the art document formats are proprietary and not publicly documented, the specialized Braille/large-print generating application is unable to propagate back the necessary changes to make the produced document fully accessible. “Save as Braille” functionality in a free and open-source office suite will make Braille production much easier and more affordable for private users, educational institutions etcetera. The extension should serve both ordinary document authors (teachers, ...) and Braille experts (Braille production centres, ...): • •

For ordinary authors, it should be possible to generate Braille according to formal standards, keeping the amount of required user interaction to a minimum. Braille experts should be able to customize the Braille output according to their own established rules. Therefore, enough flexibility must be provided through user preferences.

6.2.2


OpenOffice.org was chosen for several reasons: •

it is open source,

•

it is cross-platform and has a wider user base than its cross-platform, open-source alternative KOffice (and its more recent derivative Calligra Suite),

•

functionality can be added through extensions [OpenOfficePopularExtensions].

The “Save as Braille” extension targets OpenOffice.org version 3.0 and higher. Older versions of OpenOffice.org are no longer available from the official website because the have reached “End-OfLife” status [OpenOfficeEOL]. The default download of OpenOffice.org before it was transferred to Apache included Java 12, which is one of the languages that can be used for developing extensions [OpenOfficeExtensionsDev]. See section 6.4 Compatibility of the Extensions with LibreOffice for a discussion on compatibility 12 The release notes for Apache OpenOffice.org 3.4.1 state “Apache OpenOffice 3.4.0 and 3.4.1 support Java 7, which is the recommended configuration; but (especially on 64-bit Windows) you might receive warnings about the Java version being defective. In that case, download and install the most current JRE 6 version. Make sure you get the file "Windows x86 Offline (32-bit)". Then configure OpenOffice to use it at "Tools - Options - OpenOffice.org Java".”. See http://www.openoffice.org/development/releases/3.4.1.html October 2012

Oracle

63


PU


with LibreOffice.

6.2.3

Save as Braille Functions

The odt2braille extension must be able to “export as” Braille (Grade 2 Braille) at least OpenDocument Text files (from OpenOffice.org Writer), and ideally also OpenDocument Spreadsheets (from OpenOffice.org Calc) and OpenDocument Presentations (from OpenOffice.org Impress). The output of the extension must be suitable for input for Braille embossers. The extension should support common Braille conventions and formats, such as the “Principles of Print to Braille Transcription 1997” maintained by the Shodor Foundation and the Braille Authority of North America (BANA) [BANABraille]; Unified English Braille (UEB), maintained by the International Council on English Braille (ICEB) [ICEBUEB]. The extension should provide a dialogue with the exports options, such as language, contraction level and grade, since Braille is language dependent. The extension should support common features of the OpenDocument Text (ODT) format, including simple tables (though authors should avoid merged cells), lists, headings and table of contents. Users should be prompted to address structure problems at the source that would possibly lead to unwanted and unexpected artefacts in the Braille output. The extension should work on any platform the OpenOffice.org works, including Linux, Windows XP, Windows Vista, Windows 7.

6.2.4

Current Status of odt2braille

odt2braille 0.0.1 was released on 2 July 2010 and is available on the open-source repository SourceForge: http://odt2braille.sourceforge.net/ (site for binary downloads and documentation) and http://sourceforge.net/projects/odt2braille/ (developer site). Version 0.0.2 was released on 31 August 2010. Version 0.0.3 was released on 2 December 2010. Version 0.1.0 was released on 25 February 2011. The next release is planned in the fourth quarter of 2011. odt2braille is available under the Lesser General Public License (LGPL) version 3.0 or higher. odt2braille reuses some code and libraries from other open-source projects: •

liblouisxml, a library for Braille transcription of XML documents,

•

liblouis, a Braille transcription engine used by liblouisxml,

•

Braille Utils, a utility library for embossing and converting PEF-files [BrailleUtils].

pef2text from the DAISY Pipeline (for the conversion of the Portable Embosser Format (PEF) to a generic or embosser-specific format) and the Java OpenDocument Library (JODL) from odt2daisy are no longer used in odt2braille. •

odt2braille supports the following:

•

supported embossers (drivers need to be installed on the user’s system; some embossers are not tested yet): ◦

Braillo: 200, 270, 400S, 400SR, 440SW, 440SWSF; (270, 440SW and 440SWSF are new since D2.2.1a)

October 2012

Oracle

64


PU


CIDAT: Impacto 600, Impacto Texto, Portathiel Blue; (all new since D2.2.1a) ◦ Enabling Technologies: Bookmaker, Braille Express 100, Braille Express 150, BraillePlace, ET, Juliet Classic, Juliet Pro, Juliet Pro 60, Marathon, Romeo 25, Romeo Attache, Romeo Attache Pro, Romeo Pro 50, Romeo Pro LE Narrow, Romeo Pro LE Wide, Thomas, Thomas Pro; (all new since D2.2.1a) ◦ Index Braille: 4Waves Pro, 4X4 Pro V2, 4X4 Pro V3, Basic Blue-Bar, Basic-D V2, Basic-D V3, Basic-D V4, Basic-S V2, Basic-S V3, Everest-D V2, Everest-D V3, Everest-D V4; (V4 embossers are new since D2.2.1a) ◦ Interpoint: Interpoint 55; ◦ Mountbatten: LS, Pro, Writer+; (new since D2.2.1a) ◦ ViewPlus: Cub, Cub Jr., Elite 150, Elite 200, EmFuse, Emprint SpotDot, Max (new since D2.2.1a); export to the Braille ASCII format usually known as BRF, a Spanish version of Braille file format known as BRA, and the Portable Embosser Format (PEF), ◦

• •

in the languages supported by liblouis/liblouisxml (Arabic, Bulgarian, Tibetan, Welsh, Czech, Danish, Esperanto, Spanish, Estonian, Finnish, Irish, Scottish Gaelic, Modern Hebrew, Hindi, Croatian, Hungarian, Armenian, Icelandic, Italian, Lithuanian, Latvian, Maltese, Norwegian, Polish, Portuguese, Romanian, Russian, Slovak, Slovene, Swedish, Turkish, Vietnamese, German, Greek, Koine Greek, English, French, Dutch, Chinese and a large number of Indian languages: Assamese, Bengali, Khasi, Manipuri, Munda, Old Newari, Santali, Awadhi, Bihari, Braj Bashha, Gondi, Konkani, Kurukh, Marathi, Marwari, Nepali, Pali, Sanskrit, Sindhi, Gujarati, Panjabi, Kannada, Malayalam, Oriya, Dravidian, Tamil, Telugu, and a few new languages since D2.2.1a including Serbian, Kurdish and Ethiopic), ◦

with support of some math Braille codes supported by liblouisxml, and the Flemish math Braille code (Woluwe/Notaert),

•

the creation of notesections and a preliminary section containing a title page, a list of special symbols, a transcriber’s notes page and a table of content,

•

volume management (dividing the document into one or more binders)

•

extensive customisation of the Braille output, including:

•

◦

the number of Braille cells per line and the number of lines per page,

◦

page margins,

◦

Braille formatting (position, spacing, alignment, block protection, etc.) of paragraphs, headings, lists, tables (linear versus stair-step format), notes, pictures and the table of contents,

◦

special indication of formatted text,

◦

page numbering.

support for multilingual documents,

October 2012

Oracle

65


PU


•

hyphenation dictionaries for automatic breaking of words,

•

support for 8-dot Braille,

•

a “6 key entry mode” that simulates Braille input with a chorded keyboard (this should work on any operating system, but not with all keyboards),

•

a “dot pattern entry mode” that enables the input of Braille characters by the numbers of the Braille dots,

•

a preview window (which requires Braille fonts),

•

a number of accessibility checks.

odt2braille has been localised in 15 languages (mostly by external contributors): English, Dutch, French, Catalan, Spanish, Portuguese, Brazilian Portuguese, Slovenian, German, Norwegian Bokmål, Italian, Swedish, Polish, Czech and Arabic. odt2braille also has a user manual and screencasts (http://odt2braille.sourceforge.net/support.html) and developer documentation (http://odt2braille.sourceforge.net/develop.html). odt2braille is only available on Windows. Support on Mac OS and Linux/Unix depends on the availability of a future version of liblouisxml (liblouisxml 2.3.0) which contains contributions made by the odt2braille project. The package repository for Ubuntu currently contains liblouisxml 2.2.02; version 2.4.0 will be available in a future Ubuntu version codenamed “oneiric”. The package repository for Debian contains liblouisxml 2.4.0 for testing; odt2braille will be added to the package repository after the release of BrailleUtils 1.2. The Gentoo Linux accessibility project will probably add liblouisxml and liblouis to its repository. Liblouisxml and a newer version of liblouis will probably be added to the Fink repository for Mac OS. odt2braille has been integrated into Create&Convert by the JISC Regional Support Centre Scotland North & East [JiscCreate&Convert]. odt2braille is available from the OpenOffice.org extension website [OpenOfficeBraille]. odt2braille is also compatible with LibreOffice; it is not clear how long LibreOffice will maintain compatibility for extensions with OpenOffice.org, or vice versa (see 6.4 Compatibility of the Extensions with LibreOffice).

6.2.5

Testing (Pilots)

odt2braille was not available for the first evaluation phase; it is included in the second evaluation phase.

6.2.6


1. Title of product: odt2braille 2. Development team: Katholieke Universiteit Leuven (K.U.Leuven) - DocArch 3. Relevant WP/Activity development/integration tool place: 2.2.4 4. Purpose: Enabling both end users and Braille production centres to export OpenDocument Text (ODT) to a Braille format and to print to an embosser. The prototype turns OpenOffice.org or LibreOffice Writer into a Braille production tool, October 2012

Oracle

66


PU


which should lower the barrier to Braille production. 5. For which user groups and what are the expected benefits for each of them: The user groups identified in the use case “Printing braille in OpenOffice.org” (deliverable D1.1.3): •

1.b Blind (without useful residual vision): blind users should benefit because making Braille production easier should improve access to content in Braille. When OpenOffice.org and LibreOffice solve their own accessibility issues on Microsoft Windows (through implementation of the iAccessible2 API), Braille production should also become available to blind users.

•

Production centres for accessible document formats benefit through the ability to use the OpenDocument Format as a single source for different output formats, typically DAISY and Braille (a production centre in Flanders is working with K.U.Leuven to enable this workflow).

6. Innovations (in comparison to SoA): A mainstream (and open source) word processor can now be used as a Braille production tool, while previous Braille tools were limited with regard to word processing functionality. While previously, print and Braille versions needed to be maintained separately – so corrections in the Braille version of a document needed to fed back manually to the print version if applicable – the OpenDocument Format (ODT) can now be used as a single source for both “print” and Braille. 7. Major functions and User Interface aspects: 1. Major functions: Convert OpenDocument Text (ODT) to ASCII-based formats (BRF and BRA), and the Portable Embosser Format (PEF, based on Unicode and XML), print content directly to selected embossers. 2. User interface aspects: odt2braille’s options dialog, emboss dialog and save dialog are keyboard accessible, but due to limitations in OpenOffice.org accessibility on Windows (OpenOffice.org on Windows uses the Java Accessibility API, which is poorly supported by Windows screen readers) and limitations in the OpenOffice.org API for the creation of user interfaces for extensions (for example, labels cannot be linked explicitly with controls), access by blind users on Windows varies between hard and impossible. 8. Relevant Use Cases (from D1.1.3): 1. “Printing braille in OpenOffice.org” in D1.1.3 9. Level of achievement of defined SP2 general requirements (see Table 9: Current level of achievement of defined requirements for odt2braille

October 2012

Oracle

D2.1.1):

67


PU



Partially. Screen reader access to odt2braille on Windows is limited because of two reasons: (1) OpenOffice.org (and LibreOffice) uses the Java Accessibility API to expose accessibility metadata on Windows but this API is poorly supported by screen readers on this operating system. (IBM donated an iAccessible2 implementation for OpenOffice.org 3.1 to Oracle, but this code had not been fully integrated by the time Oracle donated the OpenOffice.org codebase to the Apache Foundation). The accessibility of OpenOffice.org is better on the GNOME desktop and on Mac OS. (2) The OpenOffice.org UNO API allows extension developers to create user interfaces, but this API does not enable explicit linking between controls and their labels, so screen readers need to infer labels based on their positioning in the user interface. There have been issues to make LibreOffice and OpenOffice.org work with the first releases of Java 7, but these issues are outside the scope of odt2braille.

UR-V5: Blind users need Partially. See UR-V4. alternative output mecha nisms. These users need to have full or close to full accessibility to their desktop PC, with the use of their other senses like touch (Braille) and hearing (Screen Reader - TTS), in order to circumvent their vision disability. UR-V6: They demand good Partially. See UR-V4. compatibility of the screen readers with dynamic or graphical applications or programs. UR-C5: The software’s help menus and some applica tions installation should be easy to use or understand, as well as the confusing error messages that the computers show.

October 2012

Partially: odt2braille enables users to customise the Braille output to a very high extent. This is necessary because Braille conventions vary from country to country and sometimes even from organisation to organisation (in the same country). However, these Braille settings can be saved in templates in order to save users the trouble of adjusting the settings for each document or each printing; in that case, the user can simply select “Emboss” or “Export as Braille” from the file menu, which results in a much simpler process.

Oracle

68


PU


SP2 relevant Requirements Achieved (Yes/No/Partially and short justification) UR-C6: The lack of transla tion to mother tongue in certain applications or pro grams, make it especially difficult for this group of users to use the desktop applications.

Yes/Partially: odt2braille 0.1.0 is available in English, French, Dutch, German, Spanish, Catalan, Swedish, Portuguese, Brazilian Portuguese, Italian, Slovenian, Norwegian Bokmål, Polish, Czech and Arabic and a few other languages, but more volunteer contributions for European languages are needed. Update: a Greek translation has been added.

Relevant functional requirements (from section 4.2.1 of D2.1.1) F-OOBP-1. The extension must be able to “export as” Braille (Grade 2 Braille) at least OpenDocument Text files (from OpenOffice.org Writer), and ideally also OpenDocument Spreadsheets (from OpenOffice.org Calc) and OpenDocument Presen tations (from OpenOffice.org Impress).

Yes. Exporting to ASCII-based Braille formats (BRF and the Spanish BRA) is available; exporting to the Portable Embosser Format (PEF) is also available. Braille grade support is typically supported through the open source libraries liblouis and liblouisxml; some languages have more Braille grades than others. Spreadsheets and presentations are currently not supported. Converting spreadsheets to Braille would present challenges for big spreadsheets (just like big tables in word processing documents). Converting presentations to Braille requires very accessible source documents. It is worth noting that neither Calc nor Impress allow users to identify the language of a document or of parts of a document, which is a requirement for outputting contracted Braille.

Relevant non-functional requirements (from section 4.2.5 of D2.1.1) Maintainability. Those ex tensions will reside and work in the OpenOffice.org envi ronment – and should work everywhere that OpenOffice .org works (e.g. the Open Desktop, Macintosh, Win dows). With any change on the environment, the plugins should be updated too, to support these changes.

Yes. The extension is tested for compatibility when a new version of OpenOffice.org or LibreOffice becomes available. During the lifetime of odt2braille there have been no changes in OpenOffice.org that affected compatibility with extensions. When The Document Foundation started LibreOffice, they also made a list of existing open source extensions for OpenOffice.org; there seems to be no intention of breaking compatibility for extensions. For availability across operating systems, see F-OOBP-1: TS4, below.

Changeability. It should be easy to add new functio nality to the extensions, as long as there are changes on the standards supported (like DAISY).

Yes. odt2braille is still changing in response to user requests (usually from Braille production centres). This requirement implies checking changes to the ODF standard; ODF 1.2 was released by OASIS in September 2011, but no work has been done yet to check for and implement changes that are relevant to odt2braille.

Stability. Stability is also Yes. odt2braille is mostly written in Java, which has its own important, because it impacts exception handling mechanism. No stability issues have been the stability of the office reported. suite as a whole.

October 2012

Oracle

69


PU


SP2 relevant Requirements Achieved (Yes/No/Partially and short justification) Testability. Because of the frequent changes on both their code base and the Open Office code base, the testa bility of these extensions will be a top priority.

Partially. odt2braille has been tested (and is still being tested) extensively by a Braille production centre. Unit tests that can be run automatically (e.g. after a new build) are being developed for odt2braille. The unit test are grouped into 3 categories: formatting tests (tests for headings, lists, tables, table of contents, notes, pictures and textboxes are available), translation tests (these are language-specific, currently only tests for Dutch and Dutch math are available) and embosser communication tests (these are being integrated in the Braille Utils library). An embosser beta testing phase that will involve users and manufacturers is being prepared within the Braille Utils project. odt2braille creates logs of actions and function calls during every conversion process in order to facilitate debugging in case of errors.

Reliability. Because the (See comments to subattributes below.) plug-ins will work on a highly interactive envi ronment, they should be reliable. The following sub attributes present this in more detail. Maturity. They should act Partially. odt2braille 0.1.1 is officially still a “beta” product because as a mature product, with a changes requested by users (especially Braille production centres) small number of failures. are still being integrated. On the other hand, one Braille embosser vendor wants to use it as a replacement (or one of the replacements) to its own Braille editing environment. Error Tolerance. In a case Yes. odt2braille provides only an export function, and errors during of an error, it should be able the export process don’t affect the functioning of the office suite. to recover, with an automatic restart of the system or with a modal dialog, with the last status. Data Integrity. In case an Yes. odt2braille saves Braille settings as metadata in the ODF file: error occurs, the integrity of when modifying these settings (i.e. modifying metadata in memory), the data should not be lost. the Save button in Writer becomes active, so saving the Braille settings becomes a normal save action. Integrity is ensured by using the proper OpenOffice.org APIs and ODF elements and attributes. Relevant technical specifications (from section 4.2.1 of D2.1.1)

October 2012

Oracle

70


PU


SP2 relevant Requirements Achieved (Yes/No/Partially and short justification) F-OOBP-1: TS1. The output of the extension must be suitable for input for Braille embossers. The extension should support common Braille conventions and formats, such as the “Principles of Print to Braille Transcription 1997” maintained by the Shodor Foundation and the Braille Authority of North American (BANA); Unified English Braille (UEB), maintained by the International Council on English Braille (ICEB); Portable Embosser Format (PEF), maintained by the DAISY Consortium.

Yes. The odt2braille settings dialog allows a high level of customisation of the Braille output, thereby allowing users to conform to any set of conventions. These settings are saved in the document being edited. They can therefore also be saved in document templates that serve as the basis for other documents produced by an organisation or a Braille production centre. odt2braille also supports the ASCII-based formats BRF and BRA, and the Portable Embosser Format (PEF, an embosser-independent Braille format based on Unicode and XML). See also F-OOBP-1: TS2 below.

F-OOBP-1: TS2.The output Yes. Enabling support for a specific embosser requires information of the extension can be about the embosser’s driver, in order to add an interface for that “printed” and read. driver to odt2braille. Each embosser needs to be added and tested separately. F-OOBP-1: TS3. The extension should provide a dialogue with the exports options, such as language, contraction level and grade, since Braille is language dependent.

Yes. The Braille settings dialog contains these options, in addition to many other ones.

F-OOBP-1: TS4. The extension should work on any platform the OpenOffice.org works, including Linux, Windows XP, Windows Vista, Windows 7.

Partially. odt2braille uses the open source libraries liblouis and liblouisxml for part of the Braille conversion. On Windows, these libraries can be compiled as static binaries that can be included in the odt2braille extension. Compiling static binaries for Linux and Mac OS failed until after the release of odt2braille 0.1.0. However, liblouis and liblouisxml were updated in the package repositories of Debian, Ubuntu and certain other Linux distributions and odt2braille was made available as a Debian package by the Debian Accessibility Project. (This work was still in progress at the time of the previous version of this report.) In August 2012, an odt2braille version for 64-bit versions of Mac OS (Snow Leopard and Lion) was also made available.

October 2012

Oracle

71


PU


SP2 relevant Requirements Achieved (Yes/No/Partially and short justification) F-OOBP-1: TS5. The extension should support common features of the OpenDocument Text (ODT) format, including simple tables (no merged cells), lists, headings and table of contents.

6.2.7

Yes. odt2braille supports these features and much more, for example: separating Braille books into volumes, some mathematical Braille conventions (for example Nemeth and Marburg through liblouis, and the Flemish Woluwe or Notaert convention through an odt2braille package), and frontmatter and rearmatter for books. K.U.Leuven worked with a Flemish DAISY & Braille production centre to find bugs and add features.



6.3 OpenOffice.org Accessibility Checker 6.3.1

Background

OpenOffice.org Writer supports various file formats and can export formats such as PDF and, with the above extensions, DAISY books and Braille. The quality of the exported formats is to a large extent determined by the quality of the source files. In both the source format and the output formats, headings need to be appropriately identified, images and other objects need text alternatives, heading rows in tables must be programmatically determinable, table rows must not break across pages in print-oriented formats, etcetera. Many users of office suites are not aware of accessibility requirements and techniques, or they may occasionally forget certain techniques. They would benefit from support for accessible authoring that is built into an office suite, or that can be easily added through an extension. Microsoft Office 2011 is the first mainstream office suite with a built-in accessibility checker. Similar functionality can also be added to OpenOffice.org and would benefit users of odt2daisy and odt2braille.

6.3.2


OpenOffice.org was chosen because the primary objective is to support users in creating better source documents for odt2daisy and odt2braille. The accessibility checker should work in the same OpenOffice.org versions as these two extensions. See section 6.4 Compatibility of the Extensions with LibreOffice for a discussion on compatibility with LibreOffice.

6.3.3

Accessibility Checker Requirements

The accessibility checker should display error messages for at least the following issues: •

missing alternative text for images,

•

blinking content (except for animated images imported by the author),

•

incorrect use of headings (skipping heading levels or using an incorrect hierarchy) and the

October 2012

Oracle

72


PU


absence of headings, •

empty document title and headings,

•

insufficient contrast between text and background colours,

•

missing identification of the document’s default language,

•

tables that don’t have proper header rows.

The accessibility checker should display warnings for at least the following issues (issues that potentially cause accessibility problems but that need author judgement): •

language changes inside the document (ideally for all languages that are recognised by the language guesser in the office suite),

•

headings that are simulated by means of big bold text,

•

justified text (which causes rivers of white space),

•

simulated unordered lists and fake ordered lists (which the author should turn into proper lists),

•

tables that are simulated by means of tab stops or spaces (which the author should turn into proper tables).

The accessibility checker may also include checks specific to special formats, for example: •

the presence of merged table cells,

•

the format of images: DAISY only supports JPG and PNG (and SVG),

•

the absence of a title page and a preliminary section (for Braille books),

•

Braille volumes that are too long or too short.

However, it is also possible for the accessibility checker to delegate format-specific checks to odt2daisy and odt2braille, and to call those checks through an interface defined by the accessibility checker and implemented by the extensions.

6.3.4

Current Status of the Accessibility Checker

6.3.4.1 Availability and Licence

The accessibility checker is available inside the project as an extension for OpenOffice.org or LibreOffice Writer. The code will be released under the Lesser General Public License 3 (LGPL 3 or later) after sufficient evaluation. 6.3.4.2 Coverage of Accessibility Issues

The evaluation tool can perform checks that are applicable only to Braille or to DAISY; these checks can be enabled or disabled in the options dialog. The accessibility checker currently only evaluates OpenDocument Text (the word processing format); at the time of writing there is no support for presentations or spreadsheets. October 2012

Oracle

73


PU


The checker currently flags the following issues as errors: •

the document contains more than one Title style

•

empty Title style

•

empty heading style (e.g. Heading 1 applied to an empty paragraph)

•

incorrect heading hierarchy (i.e. heading levels are skipped)

•

a formula has no text alternative

•

no default language is identified for the entire document

•

a span of text has no language

•

a hyperlink has no language (in Writer hyperlinks have no language by default, i.e. the language will be identified as “None” in the status bar, unless the author changes the default language for the character style “Internet Link” or sets the language of each individual link)

•

the contrast between foreground text and background is too low (i.e. it does not meet WCAG 2.0 SC 1.4.3, Level AA)

•

an image is not in a format supported by DAISY (e.g. GIF; DAISY supports only PNG, JPG and SVG). (Note: when DAISY 4 [DAISY4Core] adds support for the Windows bitmap format BMP, and DAISY 4 support is added to odt2daisy, the accessibility checker should no longer flag BMP with an error but with a warning.)

The checker currently flags the following issues as warnings: •

the document has no title (identified by a Title style)

•

the document contains no headings

•

the document contains text just below the title with (Title style) formatting that may suggest that is meant to be a subtitle but without using the Subtitle style

•

the document contains more than 6 levels of headings (this requires the use of alternate level markup when exporting to DAISY)

•

images that are linked to a document instead of embedded

•

images with an anchor that is not set to “As Character”

•

images that have no text alternative

•

other objects (except formulas) that have no text alternative

•

hyperlinks where the link text is identical to the URL (hyperlinks should have meaningful link text; in order to provide printable URLs, authors should create footnotes or endnotes with hyperlink URLs that cannot be activated)

•

tables that contain merged cells

•

tables nested inside other tables

October 2012

Oracle

74


PU


•

table heading rows that are not properly identified

•

table rows that break across pages

•

a caption below a table that is split across 2 pages

•

a very big table that is split across 3 or more pages

•

text is justified instead of left-aligned (for left-to-right text) or right-aligned (for right-to-left or bidirectional text)

•

the font size is smaller than 9 points

•

a long span of text is entirely written in uppercase

•

a long span of text is underlined

•

a long span of text is italicized

•

the Title field in the Document Properties is empty

•

the document has no title page (Braille-specific check)

•

the document has no preliminary section (Braille-specific check)

•

text has been formatted to visually look like a heading without using a Heading style

•

special characters or symbols are rendered by means of a special font instead of Unicode

•

blinking text

The table below provides a mapping between WCAG 2.0 success criteria, the Authoring Techniques for Accessible Office Documents from the ADOD project and the ODF checks that are currently supported. Some checks do not map to any WCAG 2.0 criterion, for example the check for image formats that are not supported by DAISY.

October 2012

Oracle

75


WCAG 2.0

PU

ODF Check(s)


Comments

1.1.1 Non-text Content

Error: formula has no text alternative. Warnings: image has no text alternative; object has no text alternative; special characters or symbols are rendered by means of a special font instead of Unicode.

ADOD Technique 3 Some images may be decorative and therefore not require a text alternative.

1.3.1 Info and Relationships

Warnings: table contains merged cells; table cell contains another table; table has no repeatable heading row(s); table rows break across pages or columns; table is very long; long table has caption below table instead of above; caption is not linked to any image or table; text formatting (e.g. big bold text) suggests that a paragraph should have a Heading style.

ADOD Technique 6 (using named styles instead of direct formatting) ADOD Technique 7.1, 7.2, 7.3, 7.4

1.3.2 Meaningful Sequence

Not tested

Requires research on the effect of floating objects and frames on reading order.

1.3.3 Sensory Characteristics

Not tested

ADOD Technique 9.4

1.4.1 Use of Colour

Not tested

ADOD Technique 8 (in charts) ADOD Technique 9.3

1.4.2 Audio Control

Not tested

1.4.3 Contrast (Minimum) (AA)

Error: contrast between foreground text and background is too low.

ADOD Technique 9.2

1.4.4 Resize text (AA)

No test required

Writer has built-in zoom functionality.

1.4.5 Images of Text (AA)

Not tested

ADOD Technique 9.5

1.4.6 Contrast (Enhanced) (See success criterion 1.4.3.) (AAA) 1.4.7 Low or No Background Audio (AAA)

Not applicable

1.4.8 Visual Presentation (AAA)

Warnings: text is justified; font size is ADOD Technique 9.1 smaller than 9 points; long span of WCAG does not require a text in all caps, underlined or italic. minimum font size or prohibit text that is in all caps, underlined or italic.

October 2012

Writer documents can embed audio but not as background sound.

Oracle

76


WCAG 2.0

PU

ODF Check(s)


Comments

1.4.9 Images of Text (No (See success criterion 1.4.5.) Exception) (AAA) 2.1.1 Keyboard

Warning: An image anchor is not set to “As Character”.

ADOD Technique 4

2.1.2 No Keyboard Trap

Not tested/no test required

Unless macros can trap the keyboard, this needs to be handled by Writer.

2.1.3 Keyboard (No Exception) (AAA)

(See success criterion 2.1.1.)

2.2.1 Timing Adjustable

Not applicable

Not applicable, unless macros can create time limits.

2.2.2 Pause, Stop, Hide

Warning: blinking text

ADOD Technique 9.1

2.2.3 No Timing (AAA)

Not applicable

2.2.4 Interruptions (AAA) Not applicable 2.2.5 Re-authenticating (AAA)

Not applicable

2.3.1 Three Flashes or Below Threshold

Not tested

2.3.2 Three Flashes (AAA)

Not tested

2.4.1 Bypass Blocks

Not applicable

2.4.2 Page Titled

Errors: document has more than one Title style; document has an empty Title style. Warnings: document has no Title style; paragraph below Title is formatted in a way that suggest it should have a Subtitle style.

Some documents may not require a title, e.g. letters.

2.4.3 Focus Order

Not tested/not applicable

Writer is an editing environment, and pressing TAB has a different effect than navigation in a browser.

2.4.4 Link Purpose (In Context)

Warning: hyperlink text is identical to ADOD Technique 10.2 hyperlink URL.

2.4.5 Multiple Ways (AA) Warning: Braille edition has no table of contents.

October 2012

Oracle

ADOD Technique 7.5, 7.6

77


WCAG 2.0 2.4.6 Headings and Labels (AA)

PU

ODF Check(s)


Comments

Errors: empty Heading style; ADOD Technique 5 incorrectly ordered headings; heading Some documents may not inside a frame (DAISY check). require headings, e.g. letters. Warnings: document contains no Heading styles; document has more than 6 levels of headings (DAISY).

2.4.7 Focus Visible (AA) Not tested 2.4.8 Location (AAA)

Not applicable

2.4.9 Link Purpose (Link (See success criterion 2.4.4.) Only) (AAA) 2.4.10 Section Headings (AAA)

(See success criterion 2.4.6.)

3.1.1 Language of Page

Error: document has no default language

ADOD Technique 2

3.1.2 Language of Parts (AA)

Errors: span of text has no language; a hyperlink has no language. Warning: paragraph has no identification.

ADOD Technique 2 In Writer, hyperlinks have no language unless the author specifies one.

3.1.3 Unusual Words (AAA)

Not tested

3.1.4 Abbreviations (AAA)

Not tested

Writer does not have a built-in style or mechanism for abbreviated forms.

3.1.5 Reading Level (AAA)

Not tested

ADOD Technique 10.1

3.1.6 Pronunciation (AAA)

Not tested

Relevant to DAISY export. Unclear how this can be addressed in Writer.

4.1.1 Parsing

No test required

Writer takes care of wellformed XML internally.

4.1.2 Name, Role, Value

Not tested

ADOD Technique 6 (using named styles facilitates mapping to roles)

Table 10: Mapping between WCAG 2.0, ADOD and ODF accessibility checks Some WCAG 2.0 success criteria are not listed in the above table: • The success criteria for time-based media (audio, video) under Guideline 1.2. These criteria are rarely relevant to word processing documents but can be relevant to presentations with embedded media. • The success criteria for predictable behaviour of web pages under Guideline 3.2. • The success criteria for input assistance (forms and context-sensitive help) under Guideline October 2012

Oracle

78


PU


3.3. The following warnings do not correspond to any success criterion in WCAG 2.0: • The document contains images in a format not supported by DAISY. • An empty title field in the document properties (useful for DAISY or PDF export). • The document contains a form (form accessibility in ODF requires more research). • Material will be omitted in the Braille edition because it is not placed in any volume, supplement or preliminary section. • Material is omitted in the Braille edition because odt2braille is unable to process it. • In Braille, material is transposed from its original location. This warns the author that lists, text boxes and images inside a table rendered outside the table in the Braille version. • A volume doesn't start at a logical place. Volumes should begin with a heading. • The document contains 8 dot Braille but the file format (for Braille export) does not support it. • The document contains 8 dot Braille but the embosser does not support it. ADOD Techniques 11 (check accessibility), 12 (use accessibility features when saving or exporting to other formats) and 13 (consider using accessibility support applications/plugins) do not correspond to any success criterion in WCAG 2.0. The mapping table shows that many WCAG 2.0 success criteria are not evaluated by the accessibility checker. There are several reasons for this. • The easiest success criteria were implemented first, in order to quickly develop a working prototype for internal evaluation by the ÆGIS project. • For some success criteria, it is unclear how they can be evaluated systematically triggering false positives, for example, the success criteria 1.3.3 (sensory characteristics), 1.4.5 (images of text), 2.4.4 (link purpose) and 2.3.1 (flashing content). • Some success criteria require language-specific resources, notably the success criteria 3.1.3 (unusual words), 3.1.5 (reading level), 3.1.6 (pronunciation) and 3.1.2 (language of parts). For success criterion 3.1.2 it is possible to rely in Writer’s built-in language guessing functions, so the question is now at which level of granularity the language changes should be detected. 6.3.4.3 Metadata: EARL

OpenOffice.org and LibreOffice support the use of RDF (Resource Description Framework) for storing metadata inside ODF files. The accessibility checker stores information on errors and warnings as RDF inside the document. The format used for this purpose is the Evaluation and Report Language (EARL)[Abou-Zahra 2011], an RDF format for describing test results that is being developed by the World Wide Web Consortium (W3C). In addition to the list of errors and warnings, the metadata also describe which errors and warnings have been repaired or ignored, when the most recent accessibility evaluation was performed, and by which evaluation tool. The ODF accessibility checker is designed as a framework into which several evaluation components can be “plugged” in. One such component would be odt2braille, which would check the document for issues that are specific to Braille conversion. The test results from odt2braille would then also be stored in the EARL report.

October 2012

Oracle

79


6.3.5

PU


Testing (Pilots)

The accessibility checker was not available for the first evaluation phase; it was included in the second evaluation phase, where it was tested together with odt2braille.

6.3.6


1. Title of product: ODF Accessibility Checker 2. Development team: Katholieke Universiteit Leuven (K.U.Leuven) - DocArch 3. Relevant WP/Activity development/integration tool place: 2.2.4 4. Purpose: Enabling users to evaluate the accessibility of OpenDocument Text (ODT) without leaving the OpenOffice.org or LibreOffice Writer interface. This is not only useful for the accessibility of word processing documents but also for the accessibility of output formats such as the Portable Document Format (PDF; Writer can output tagged PDF), DAISY and Braille. 5. For which user groups and what are the expected benefits for each of them: The user groups identified in the use case “Accessibility Checking for ODF” (deliverable D1.1.3) and any other user group that benefits from more accessible documents: •

1.b Blind (without useful residual vision): blind users should benefit because the checker verifies features such as text alternatives for images, use of headings and other features that are essential for accessibility and that can can be verified automatically.

•

Production centres for accessible document formats benefit producing DAISY and Braille requires accessible source documents. The accessibility checker complements odt2daisy and odt2braille.

•

Any user groups that benefit from more accessible documents. However, some document features (for example, missing text alternatives) are easier to check than others (for example, many of those related to the readability of text for persons with cognitive impairments).

•

Any other author who wants or needs to produce accessible documents.

6. Innovations (in comparison to SoA): Checking the accessibility of word processing documents is an innovation compared to the state of play before Microsoft released Office 2010. However, the ODF Accessibility Checker goes beyond its counterpart in Microsoft Office by providing more help information and by providing automatic or semi-automatic repair for a number of issues (automatic: for example, empty headings; semi-automatic: for example, the Repair button for an image without a text alternative invokes the dialog for inputting text alternatives). 7. Major functions and User Interface aspects: 1. Major functions: The ODF Accessibility Checker evaluates OpenDocument Text (ODT) and provides a list of errors and a list of warnings. For each error or warning, it lists the instances. For each type of error and warning, the user interface display a name, a description and a repair suggestions. If the error or warning can be fixed automatically, October 2012

Oracle

80


PU


pressing the Repair button fixes the issue and removes the error or warning from the list. If the error and warning can be fixed by changing data or a setting in a Writer dialog, pressing Repair takes the user to that dialog, in order to repair the issue. 2. User interface aspects: The ODF Accessibility Checker uses a toolpanel, a user interface component that became available in writer in OpenOffice.org/LibreOffice 3.3. Due to limitations in OpenOffice.org accessibility on Windows (OpenOffice.org on Windows uses the Java Accessibility API, which is poorly supported by Windows screen readers) and limitations in the OpenOffice.org API for the creation of user interfaces for extensions (for example, labels cannot be linked explicitly with controls), access by blind users on Windows varies between hard and impossible. 8. Relevant Use Cases (from D1.1.3): 1. “Accessibility Checking for ODF” in D1.1.3 9. Level of achievement of defined SP2 general requirements (see D2.1.1): Table 11: Current level of achievement of defined requirements for the ODF Accessibility Checker

October 2012

Oracle

81


PU



Partially. Screen reader access to the ODF Accessibility Checker on Windows is limited because of two reasons: (1) OpenOffice.org (and LibreOffice) uses the Java Accessibility API to expose accessibility metadata on Windows but this API is poorly supported by screen readers on this operating system. (IBM donated an IAccessible2 implementation for OpenOffice.org 3.1 to Oracle, but this code had not been fully integrated by the time Oracle donated the OpenOffice.org codebase to the Apache Foundation). The accessibility of OpenOffice.org is better on the GNOME desktop and on Mac OS. (2) The OpenOffice.org UNO API allows extension developers to create user interfaces, but this API does not enable explicit linking between controls and their labels, so screen readers need to infer labels based on their positioning in the user interface.

UR-V5: Blind users need Partially. See UR-V4. alternative output mecha nisms. These users need to have full or close to full accessibility to their desktop PC, with the use of their other senses like touch (Braille) and hearing (Screen Reader - TTS), in order to circumvent their vision disability. UR-V6: They demand good Partially. See UR-V4. compatibility of the screen readers with dynamic or graphical applications or programs. UR-C5: The software’s help menus and some applica tions installation should be easy to use or understand, as well as the confusing error messages that the computers show.

October 2012

Yes: The user interface is as simple as possible: it contains four fields and five buttons. (The fields are the list of issues, the name of the currently selected issue, its description and repair suggestions. The buttons are: Recheck, Clear, Ignore and Repair.)

Oracle

82


PU


SP2 relevant Requirements Achieved (Yes/No/Partially and short justification) UR-C6: The lack of transla tion to mother tongue in certain applications or pro grams, make it especially difficult for this group of users to use the desktop applications.

No: The accessibility checker was initially made available in English, Dutch and Spanish. Since each error and warning requires a description and repair suggestions, the content that requires translation is considerably larger than for odt2daisy and odt2braille. Translation will require volunteer contributions. There have been issues to make LibreOffice and OpenOffice.org work with the first releases of Java 7, but these issues are outside the scope of odt2braille.

Relevant functional requirements (from section 4.2.4 of D2.1.1) F-OOAT-1. The ODF accessibility extension must be able to check every ODT document for accessibility issues.

Yes. Every document can be evaluated, but longer documents require more time.

Relevant non-functional requirements (from section 4.2.5 of D2.1.1) Maintainability. Those ex Yes. The extension is tested for compatibility when a new version of tensions will reside and work OpenOffice.org or LibreOffice becomes available. in the OpenOffice.org envi ronment – and should work everywhere that OpenOffice .org works (e.g. the Open Desktop, Macintosh, Win dows). With any change on the environment, the plugins should be updated too, to support these changes. Changeability. It should be easy to add new functio nality to the extensions, as long as there are changes on the standards supported (like DAISY).

Yes. The only relevant standards are ODF and the W3C’s Evaluation and Report Language (EARL). This requirement implies checking changes to these standards. Changes in ODF 1.2 will be reviewed when OASIS has approved the new version. The internal metadata format will be reviewed when EARL becomes a W3C Recommendation.

Stability. Stability is also Yes. The extension is entirely written in Java, which has its own important, because it impacts exception handling mechanism. the stability of the office suite as a whole. Testability. Because of the No. A suite of unit tests that can be run automatically needs to be frequent changes on both developed. The developers currently use test documents that their code base and the Open combine several accessibility issues for manual testing. Office code base, the testa bility of these extensions will be a top priority.

October 2012

Oracle

83


PU


SP2 relevant Requirements Achieved (Yes/No/Partially and short justification) Reliability. Because the (See comments to subattributes below.) plug-ins will work on a highly interactive envi ronment, they should be reliable. The following sub attributes present this in more detail. Maturity. They should act No. The extension was released internally for the first time for as a mature product, with a testing in the second evaluation phase. Feedback will enable the small number of failures. developers to create a more mature version. Error Tolerance. In a case of an error, it should be able to recover, with an automatic restart of the system or with a modal dialog, with the last status.OpenOfficeApache

Yes. So far, no issues that affect the functioning of the office suites have been reported, nor failures to run the extension (i.e. during pretesting by test sites in preparation of the second evaluation phase, and during testing by the developers).

Data Integrity. In case an Yes. The ODF Accessibility Checker saves certain metadata in the error occurs, the integrity of ODF file. So far, no issues related to data integrity have been the data should not be lost. reported (i.e. during pre-testing by test sites in preparation of the second evaluation phase, and during testing by the developers). Relevant technical specifications (from section 4.2.1 of D2.1.1) F-OOAT-1 : TS1. The ODF Yes. All checks listed for F-OOAT-1 TS1 have been implemented, as accessibility extension must have a number of checks that were not listed. be able to detect automatically most of the accessibility issues that can be detected in the XML code of the files that make up an ODT file.(...) F-OOAT-1: TS2.The ODF accessibility extension must be able to detect issues that potentially cause accessibility problems but that need author judgement (potentially with contextsensitive help information).

6.3.7

Partially. The test for fake unordered lists has not been implemented. When authors use an asterisk or dash at the start of a line, enter text and press Enter, Writer automatically turns the line into a list item, so this test may not be necessary after all.



October 2012

Oracle

84


PU


6.4 Compatibility of the Extensions with LibreOffice 6.4.1

Licensing

In 1999 Sun Microsystems, Inc. purchased the company StarDivision, including its main product, the office suite StarOffice. In 2000 Sun created the open source project OpenOffice.org based on the StarOffice source code (minus some few parts that were commercially licensed to StarDivision/Sun). In the following year this open source project released OpenOffice.org version 1.0, an open source office suite, under the Sun Industry Standard Source License (SISSL). Several releases followed under this license. Then in 2005 with version 2.0 the license was changed to the more common LGPL v2. License. Since 2000 a number of “forks” were developed based on OpenOffice.org – office suites that were based on snapshots of the OpenOffice.org codebase at one point or another. Perhaps the most well known of these is LibreOffice, a fork created under the auspices of The Document Foundation based on a snapshot of the beta release of OpenOffice.org 3.3 in September, 2010. A number of GNU/Linux distributions switched from OpenOffice.org to LibreOffice, which accounts for much of its popularity. LibreOffice is licensed under LGPL v3. In 2011 Oracle announced that it was discontinuing commercial support for the OpenOffice.org, in April 2011 Oracle announced its intention to move OpenOffice.org to a community-based project [ApacheDonation], and in June 2011 it became an Apache Foundation incubator project, with ownership of OpenOffice.org being donated to the foundation, under the Apache open source license. At the source code level, code released under the LGPL license cannot be reused in software available under the Apache License (the LGPL license has certain restrictions that are not compatible with Apache, such as the ability to use the code in commercial offerings). Code available under the Apache License can be relicensed and reused in software available under the LGPL license (Apache is a “more permissive” license). Hence, LibreOffice will be able to reuse code from the future Apache OpenOffice suite will, but the reverse will not be true. Apache OpenOffice will not be able to reuse code from LibreOffice – unless the developers also donate their code also to the Apache foundation or otherwise also license it under the Apache or an Apache-compatible license. This essentially maintains the original situation that LibreOffice was in with OpenOffice.org – being a “downstream fork”. Compiled libraries (such as OpenOffice.org extensions) that are available under LGPL or any other license allowing binary redistribution can be installed in software available under the Apache License without violating the LGPL. The GNU Lesser General Public License was formerly known as the GNU Library General Public License and was created in order to allow non-copyleft software (e.g. under Apache License or proprietary licenses) to link to copyleft software. odt2daisy and odt2braille are both available under LGPL 3.0. From a licensing point of view, the compiled extensions can be used with the old OpenOffice.org (versions released by Sun or Oracle), with LibreOffice (which uses both LGPLv3 and MPL) and with the future releases of the Apache OpenOffice suite. At the source code level, integration of odt2daisy and odt2braille into LibreOffice should not pose problems if that becomes desirable. Integration of these two extensions in the future Apache OpenOffice would probably require relicensing the code under the Apache License or adopting a dual license (both LGPL 3.0 and the Apache License). However, both odt2daisy and odt2braille use code from other projects that are compatible with LGPL 3.0 but not with the Apache License, so relicensing or dual-licensing the extensions would require consent from the other projects to relicense or dual-license their code. For this reason, integration into LibreOffice is legally an easier approach than integration into Apache OpenOffice, which does not exist at the time October 2012

Oracle

85


PU


of writing. The Accessibility Checker does not use external libraries, so it is free to choose any open-source licence.

6.4.2

API Compatibility

There are currently no known differences between OpenOffice.org 3.3 and LibreOffice with regard to the APIs for developing and running extensions. Both OpenOffice.org and LibreOffice have a downloadable software development kit (SDK) for developing extensions and external tools, so it is technically possible to create incompatibilities between the APIs and SDKs of both projects, should the projects choose to do so. Introducing incompatible changes would create problems for both extension users and extension developers, as it would force them to choose between one office suite or the other. It is possible for an extension to detect in which of the two office suites it is running (such code has been developed for odt2daisy) and to provide alternative code depending on the detected office suite. This is currently not needed. (If the future Apache OpenOffice version introduces changes that break compatibility with the current OpenOffice.org and LibreOffice – for example by replacing the current user interface with an interface borrowed from IBM Lotus Symphony, or even by removing support for extensions – then the situation will need to be reviewed again. The original intention of this section was to discuss compatibility between Oracle’s OpenOffice.org and the Document Foundation’s LibreOffice.) The Document Foundation currently maintains two lines of LibreOffice: the original LibreOffice 3.3 line (version 3.3.3 was released on 16 June 2011) and the 3.4 line (version 3.4.2 was released on 1 August 2011). LibreOffice 3.5 is being prepared but a feature freeze will probably not occur before December 2011; a release would follow 2 months later, so probably in February 2012. This means that until February 2012, there are three targets for compatibility testing of extensions: • OpenOffice.org 3.3 • LibreOffice 3.3.x, • LibreOffice 3.4.x. Update since D2.2.1b: • The Document Foundation currently maintains LibreOffice 3.5.x and LibreOffice 3.6.x. These versions replace LibreOffice 3.3.x and LibreOffice 3.4.x as main targets for compatibility testing (these older versions are no longer available for download). However, testing with these older versions is useful for checking backward compatibility, as some users may not be able to upgrade LibreOffice regularly. • Apache OpenOffice.org released OpenOffice 3.4.0 and 3.4.1, while OpenOffice.org 3.3 is no longer available for download. Although OpenOffice.org is still in “incubating” project (all new projects at the Apache Software Foundation need to go through the Incubator), it is likely to graduate to a normal project. Due to the number of downloads for these releases of Apache OpenOffice.org, these versions also need to be included in compatibility testing. odt2daisy, odt2braille and the accessibility checker are developed with the OpenOffice.org SDK and the OpenOffice.org API plugin for NetBeans [OpenOfficeNetBeans]. This plugin is available for NetBeans 6.8 and older versions (i.e. not for NetBeans 6.9 or 7.0) [NetBeansOpenOffice] and facilitates testing by making it easy to deploy and run his current code in the OpenOffice.org version installed on his system (simply by selecting “Deploy and Run Extension in OpenOffice.org” from the project’s context menu in NetBeans after building the code). This makes daily compatibility testing very easy, at least in the office suite identified in the configuration of the October 2012

Oracle

86


PU


NetBeans plugin. Currently, the following setup can be used for testing compatibility of extensions: • install OpenOffice.org 3.3 and configure the NetBeans plugin to point to the directory where it is installed, so this version of the office suite can be used for daily testing; • install LibreOffice 3.4 on the same system and test compatibility of extensions at regular intervals (and definitely before every release); OpenOffice.org and LibreOffice can be installed on the same operating system without causing conflicts or interference; • install LibreOffice 3.3 on another system and test compatibility of extensions at regular intervals (and definitely before every release). It is also possible to switch the office suites, for example, by using the SDK from LibreOffice and configuring the NetBeans plugin to point to the LibreOffice directory; testing with OpenOffice.org then requires manual installation of the extensions in OpenOffice.org. It is also possible to develop extensions without the OpenOffice.org API plugin for NetBeans. In that case, there is no “Deploy and Run Extension in OpenOffice.org” option in NetBeans, but it is possible to define an Ant task for this. This is the approach taken for odt2braille and AccessODF. The approach with both OpenOffice.org and LibreOffice on the same operating system can still be used.

October 2012

Oracle

87

AEGIS D2.2.1b_final

PU


6.5 Relationship of odt2daisy, odt2braille and the ODF Accessibility Checker to the OAF


November 2011

Oracle

88


PU


7 A2.2.5 Open interface to assistive technologies 7.1 Introduction The GNOME desktop community developed the first comprehensive, 3rd generation, desktop accessibility framework: the GNOME Accessibility API. Introduced nearly 10 years ago, a core component of this framework is the inter-process communication (IPC) mechanism that operates between accessible applications and the assistive technologies that (re-)present them to users with disabilities. Because this communication is about user interface elements which expose an objectoriented accessibility API, the IPC is object-to-object communication – it uses CORBA for that IPC. The open source GNOME desktop has a well-proven accessibility framework – with hundreds of accessible GNOME applications, as well bridge layers that support the accessibility API exposed by non-GNOME applications such as Java applications, Firefox, Thunderbird, and OpenOffice.org. When combined with a UNIX kernel and operating system – such as GNU/Linux – it makes for a very attractive basis for more lightweight computing environments accessible: such as PDAs, Linux-based mobile phones, and the One Laptop-per-Child project. However, these environments lack the memory and/or CPU power to run CORBA. Also, the other common GNU/Linux graphical desktop – KDE – doesn't include CORBA by policy. Finally, the GNOME community has deprecated CORBA and the uses that GNOME made of it – particularly the componentembedding functionality of Bonobo. This long-planned phase-out of Bonobo and CORBA is scheduled to happen in April 2011 with the introduction of GNOME 3. Because of the potential to enable the proven, open source, 3rd generation accessibility framework of GNOME to work in mobile and other small-footprint devices, and also because of the need for the KDE desktop to support accessibility, and finally to ensure that GNOME accessibility continues to work in GNOME 3, this activity has undertaken a replacement of GNOME's use of CORBA.

7.2 Evaluating options, choosing D-Bus There were three potential alternatives for a CORBA replacement: (1) direct TCP/IP socket communication, (2) X-Window communication based on ICE, and (3) D-Bus. TCP/IP socket communication would require developing an object serialization mechanism, and choice of a streams protocol on top of that – essentially we would have had to implement a number of libraries that already duplicate what is provided in D-Bus. While ICE was designed with accessibility in mind, it assumes that the applications are both sitting on top of X Windows – an assumption that isn't true for ports of GNOME/GTK+ and KDE/Qt to Windows. For those reasons, but also primarily because both the GNOME and KDE communities had already adopted D-Bus – as well as the Nokia Maemo and OLPC projects – it was determined that the CORBA replacement should use D-Bus.

7.3 What must change to support D-Bus CORBA is used throughout the GNOME accessibility framework. The diagram below illustrates this: communication from GNOME GTK+ applications, Mozilla applications (Firefox, Thunderbird), UNI applications (OpenOffice.org, LibreOffice, and their plug-ins), and Java applications all use CORBA to communicate with assistive technologies, and the registry daemon. October 2012

Oracle

89


PU


Illustration 18: GNOME accessibility framework use of CORBA - source: http://live.gnome.org/Accessibility/BonoboDeprecation In particular, this means that we need to re-implement the GNOME ATK bridge (which bridges between the Accessibility ToolKit library ATK and CORBA), the Java Access Bridge for GNOME (which likewise bridges between the Java Accessibility API and CORBA), the at-spi registry daemon, then make sure that the assistive technologies that use the Python and C libraries have a DBus alternative. Regarding the latter – pyatspi and cspi – we participated in discussions with the GNOME accessibility community and reached the conclusion that we would no support a D-Bus-based cspi for the foreseeable future. Of the five cspi clients, GOK has been deprecated (replaced by Caribou that is using pyatspi), at-poke has been replaced by the far more functional accerciser, the LDTP testing library is moving to pyatspi, and both Dasher and MouseTweeks make only very little use of cspi and is being modified directly to interact with D-Bus. Another important change is the way that Java applications will be accessible with the shift to DBus. Because the Java SE platform includes a Java implementation of CORBA, the Java Access Bridge for GNOME was a 100% Java library interacting with CORBA. Going forward, we have instead implemented a JNI-based bridge that bridges from the Java Accessibility API directly to ATK (JAW or the Java Accessibility Wrapper). In this way, Java is insulated from the inter-process communication, and simply re-uses the atk-bridge. Doing it this way allowed an earlier implementation of Java, where we could debug the implementation on top of the old CORBA implementation. It also minimizes the number of unique code paths, and therefore the number of bugs. The new diagram is shown below. Communication from GNOME GTK+ applications, Mozilla October 2012

Oracle

90


PU


applications (Firefox, Thunderbird), UNI applications (OpenOffice.org, LibreOffice, and their plugins), and Java applications all the atk-bridge, which in turn uses D-Bus to communicate with assistive technologies, and the registry daemon. A new 'cspi2' is being implemented containing that minimum set of bindings needed by C/C++ language clients. The other change is that activation is done via D-Bus, rather than via gnome-session.

Illustration 19: GNOME accessibility framework use of D-Bus - source: http://live.gnome.org/Accessibility/BonoboDeprecation

7.4 Dealing with D-Bus challenges The fundamental challenge with D-Bus is performance. D-Bus is an outgrowth of the KDE project's DCOP – a light-weight interprocess communication system. DCOP was designed for scripting – allowing one application to make script calls on another (e.g. “print”, “quit”). D-Bus not only replaces DCOP, but several other existing and competing IPC mechanisms, and introduces sufficient object references to allow for a mapping to object-oriented IPC. However, it is far younger that CORBA, and hasn't had the performance tuning of CORBA – particularly the GNOME CORBA ORB ORBit, which was very fast for IPC between processes on the same machine. Meanwhile, one of the main initial use cases for D-Bus was responding to user-interface events, such as when a user inserts a USB thumb drive into a USB port (and not for dispatching thousands of events/second). In collaboration with the GNOME accessibility community, we have taken a number of steps to address these performance challenges. Since D-Bus supports a system having multiple IPC busses, we create a dedicated bus just for accessibility messages. We have also refactored at-spi (now atspi2), putting more information into each API call so that fewer of them need to be made. Remote October 2012

Oracle

91


PU


reference counting has been eliminated (which not only cuts down on IPC calls, but also reduces complexity). Finally, at-spi2 caches a great deal of API information, again limiting the number of IPC calls needed. In this way, we are seeking to deliver comparable end-user performance of assistive technologies – e.g. how quickly speech reacts in a screen reader as a user tabs through the links in a web page – to what they had with CORBA.

7.5 Current status The ÆGIS effort on A2.2.5 is largely done. The work has taken longer than expected due to the significant amount of refactoring needed in at-spi2 to handle the slower IPC round trips. Also the GNOME community realized that their target date for cutting over to D-Bus, along with the myriad other changes planned for GNOME 3 – was too aggressive. As of this writing, GNOME 3 is a few weeks away from release, and builds of GNOME with the accessibility infrastructure using D-Bus are available and undergoing “beta testing”. Because of the significance of this change – somewhat akin to “changing the engines of an airplane while in flight” – we expect that the work won't fully settle out until GNOME 3.2 around September/October 2011. Because many other aspects of the shift to GNOME 3 present other accessibility challenges – such as the new user interface components which need to implement ATK, the immaturity of Caribou as a replacement for GOK, and some issues still to be worked out around the new accessibility preferences – we expect most users needing assistive technologies will wait for GNOME 3.2. All of the technical goals for A2.2.5 have been essentially achieved with regards to developing an open, light-weight inter-process communication mechanism suitable for use on lower-memory and lower-CPU devices. Recent commercial moves away from Linux-based PDAs – such as Nokia's move to Windows Mobile 7 – mean the mobile and PDA future for this work is somewhat cloudy. After the release of GNOME 3 – and likely after GNOME 3.2 – we anticipate renewed interest from the OLPC community in the accessibility architecture. However, it will require a very significant engineering investment in adapting the preferred OLPC user interface Sugar to use ATK/at-spi2, and that work is not within the scope of ÆGIS. Therefore the main “proof” of this work will be the users of GNOME 3 and other desktop variants (such Ubuntu Unity, and the the Xfce desktop which is targeting lower end netbooks) who will be using the D-Bus based accessibility framework later this year.

7.6 ÆGIS contributions More so than most of the ÆGIS activities, the Open interface to assistive technologies work was done in collaboration with the open source community. Under the ÆGIS project, Oracle initiated the move to DBUS from CORBA. This involved an analysis of the various options and what needed to change to move to DBUS (including the use of CORBA by applications, frameworks, and assistive technologies), and initial implementations of most of the necessary libraries. Under ÆGIS, Oracle also developed the Java ATK Wrapper, as well as mechanisms to enable end-users to run both the old (CORBA) and new (DBUS) interfaces alternately on the same system – to facilitate testing. Finally, as the maintainer of both at-spi / at-spi2, JAW, and the GNOME Accessibility Project overall, Oracle coordinated the growing community effort on this activity through early 2010, at which point others in the community took the leadership role (with Oracle continuing to maintain at-spi2 and JAW).

October 2012

Oracle

92


PU


7.7 Overview of the application 1. Title of product: D-Bus based accessibility framework for the Open Desktop 2. Development team: Oracle 3. Relevant WP/Activity development/integration tool place: 2.2.5 4. Purpose: Enabling a broader range of UNIX and GNU/Linux based environments to use the GNOME accessibility framework, including KDE, low-memory/low-power devices like PDAs and the One Laptop per Child, and the new GNOME 3 environment where CORBA has been deprecated. 5. For which user groups and what are the expected benefits for each of them: All existing users with disabilities served today by GNOME 2, including: •

Visually impaired users using the Orca screen reader / magnifier; or who would use Orca if it would work on their non-GNOME environment

•

Motor impaired users using Dasher, GOK, and Caribou; or who would use Orca if it would work on their non-GNOME environment

6. Innovations (in comparison to SoA): A rich, flexibility, extensible accessibility framework with proven assistive technologies being available to low-memory/low-power environments. 7. Major functions and User Interface aspects: 1. There are no user interface aspects to this work, as it is “plumbing” 8. Relevant Use Cases (from D1.1.3): 1. No use case defined for A2.2.5 in D1.1.3 9. Level of achievement of defined SP2 general requirements (see D2.1.1): Table 12: Current level of achievement of defined requirements for AEGIS Open Interface to Assistive Technologies

October 2012

Oracle

93


PU


SP2 relevant Requirements Achieved (Yes/No/Partially and short justification) Relevant User Requirements (from Chapter 3 of D2.1.1) UR-V3: The compatibility of tools like the screen magnifiers with dynamic and graphical applications to allow full use of the computer.

Yes. The D-Bus based accessibility framework replicates the compatibility already achieved in GNOME 2.30, and with the AEGIS-funded work, improves on it directly with Java compatibility (JAW), and in conjunction with the Open Desktop community is bringing compatibility with KDE/Qt based applications as well as the Clutter UI toolkit which is the basis of the GNOME 3 UI.

UR-V6: They demand good compatibility of the screenreaders with dynamic or graphical applications or programs.

Partially: Further Clutter UI work (specifically the accessibility API of Cally and keyboard operability) – which are outside of the scope of A2.2.5 – is needed before users have "full or close to full accessibility to their desktop PC".

UR-V8: These users ask to Partially: Part of the D-Bus based accessibility framework is the have self-sufficiency if the ability to have accessibility able to turn on/off at will. That work is system crashes, so that they still in process. are not able to start, reinstall the system, because it is not possible to use any AT while the operating system does not work. UR-M2: These users are Partially: The GNOME 3 desktop is integrating assistive demanding progress on the technologies, alongside the D-Bus based accessibility framework. integration of accessibility Much of this is beyond the scope of A2.2.5. tools in all the operating systems, where such integration would bring accessibility integrated as a standard feature. Relevant functional requirements (from section 4.1.4 of D2.1.1) F-OIAT-1. The Open Yes. The Orca screen reader and GNOME Shell Magnifier are Interface With Assistive working well with the new architecture. Dasher work was being Technologies should provide completed at the time of this writing. an interface that will act as a communication interface between the third party application that uses it and the Assistive Technologies components, such as the Screen Reader, Dasher etc.

October 2012

Oracle

94


PU


SP2 relevant Requirements Achieved (Yes/No/Partially and short justification) F-OIAT-2. Users should be enabled to switch among input/output alternatives without requiring them to reconfigure or restart the system or applications.

No. The shift to D-Bus is necessary, but not sufficient, to achieve this outcome. The GNOME 3 Shell is largely achieving this, but not the older GTK+ based applications.

Relevant non-functional requirements (from section 4.1.4 of D2.1.1) Efficiency - The efficiency Yes. A bare minimum of resources are used if the interface is turned of the Open Interface has to off. grant an optimal allocation of system resources. The Open Interface should not maintain the control of a resource if it is not being used. This performance has to be strictly observed for mobile phones and low performance devices. Maintainability - This Partially. We believe this has been achieved, but further testing in interface will reside and third environments is necessary to fully confirm this. work in a third environment. Any change in the environment should update the enhancements. Interoperability Interoperability of Open Interface consists in the ability of a system to cooperate and exchange information with other systems or services or products in a complete and error-free way, with reliability and optimization of the resources

Yes. Multiple assistive technologies have been demonstrated to be interoperable with multiple application frameworks, including OpenOffice.org / LibreOffice, Firefox with ARIA, GTK+webkit, Java, etc.

Reliability - Because the Partially. Alongside the GNOME accessibility community, we are enhancements will work on a targeting GNOME 3.2 for "release quality" and reliability of the highly interactive framework, in September of 2011. environment, they should be reliable meaning that they should act as a mature product, with a small number of failures.

October 2012

Oracle

95


PU


SP2 relevant Requirements Achieved (Yes/No/Partially and short justification) Usability - The usability of Partially. Alongside the GNOME accessibility community, we are the Open Interface is based targeting GNOME 3.2 for "release quality" and reliability of the on the concept of the framework, in September of 2011. efficiency and satisfaction of the user that reaches specific objectives in the target context of the program that interacts with the Open Interface. Security - A system has to Partially. Support for administrative user privileged graphical be preserved from potential applications is in place, but somewhat difficult to activate still. risks and violations in order to grant security. Relevant technical specifications (from section 4.1.6 of D2.1.1) TS1: In order to enable effective use of assistive technologies, software should use system-provided input and output methods, wherever possible.

Yes. D-Bus is system provided, and the use of D-Bus has no impact on the use of other i/o methods.

TS2: Users should be Yes. The D-Bus architecture preserves and exposes this aspect of enabled to access object the GNOME Accessibility Framework. labels stored as additional text, whether those labels are visually presented or not. TS3: Notification of events Yes. The D-Bus architecture preserves and exposes this aspect of should be provided to the GNOME Accessibility Framework. assistive software. Events relevant to user interactions should be available to assistive technologies. Such events include, but are not limited to, changes in object status; such as the creation of new objects, change in selection, change in position, as well as changes in attributes such as size and colour.

October 2012

Oracle

96


PU


SP2 relevant Requirements Achieved (Yes/No/Partially and short justification) TS4:Users also need feedback Yes. The D-Bus architecture preserves and exposes this aspect of on input events, such as key- the GNOME Accessibility Framework. presses and mouse button presses, and output events that the system provides, such as writing text to the screen or playing audio information. TS5: Information on Yes. The D-Bus architecture preserves and exposes this aspect of the individual object attributes GNOME Accessibility Framework. should be available to assistive technologies (such as “listeners”). Such attributes, may include, but are not limited to, object name, size, position and current state. These object attributes are typically available to users by inspection/interaction. TS7: A well-defined onYes. The D-Bus architecture preserves and exposes this aspect of screen indication of the the GNOME Accessibility Framework. current focus shall be provided that moves among interactive interface elements as the input focus changes. The focus shall be programmatically exposed so that assistive technology can track focus and focus changes. TS8: Sufficient information Yes. The D-Bus architecture preserves and exposes this aspect of about a user interface element the GNOME Accessibility Framework. including the identity, operation and state of the element shall be available to assistive technology. When an image represents a program element, the information conveyed by the image must also be available in text.

7.8 Relationship to the Open Accessibility Framework (OAF) The AEGIS Open Accessibility Framework (OAF) defines a generic framework to addressing Information Communication Technology (ICT) accessibility. Specifically, the OAF defines steps of October 2012

Oracle

97


PU


the “value delivery chain” of creating and using accessible (ICT): three “creation” steps, and three “use” steps. The Open interfaces to assistive technology (A2.2.5) is part of Step 1 - “Define Accessible” - and part of Step 4 - “Platform support”.


7.9 References The GNOME community has been tracking this work on the GNOME accessibility wiki pages. •

The main page describing this work – under the term “Bonobo deprecation” is: http://live.gnome.org/Accessibility/BonoboDeprecation

•

This is a subset of the larger work of adapting to the GNOME 3 move, described at: http://live.gnome.org/Accessibility/GNOME3

•

For users prepared to live “on the bleeding edge” (as well as developers who want to be able to run both the “old” and the “new”), where to get the new code and how to have both old and new co-exist on the same system is described at: http://live.gnome.org/AccessibilityCORBAToDBusMapping

•

(2) X-Window communication http://trace.wisc.edu/docs/x_win_andice/x_andice.htm

October 2012

Oracle

based

on

ICE

98


PU


8 Conclusion The goals of this Work Package were: • To develop the necessary open assistive technology support libraries to allow assistive technologies on the open desktop to reach parity, and exceed the functionality of the 2nd generation assistive technologies on the proprietary desktop. • To extend to the proprietary Windows desktop the 3rd generation access approach of the Open Desktop, and thereby enable the development and use of a non-Internet based, accessible cross-platform application development. • To build into a major cross-platform office suite support for the creation and production of accessible content – in large print, Braille, Symbol encoded and DAISY book formats. • To support the reading and brailling of multilingual documents for the blind; pronounced correctly in the appropriate language & locale, and likewise brailled correctly. With the exception of the feasibility study of IA2, we have not only fully accomplished these goals, but the prototypes realizing these goals are published in open source, and many are already in use – either being actively incorporated into GNOME (A2.2.1, A2.2.5), or being used by disability organizations (A2.2.4 odt2daisy, odt2braille). The CCF plugin deliverable for OpenOffice.org relies fully on OpenOffice's own accessibility mechanisms. With consideration to the fact that the CCF library to access symbol information does not expose any properties or other mechanisms that should be used by an accessibility interface; no programmatic accessible API is available for the CCF Library. The CCF Library is one of the background technologies for the OpenOffice.org extension, and the CCF Library API is made available to developers who wish to implement applications or services with multi-modal language support, and is not intended to be directly accessed by end users. improvements in eSpeak (A2.2.3), especially for Spanish, Italian and Belgian Dutch, • odt2daisy (A2.2.4), with numerous small changes such as better table support, support for long descriptions for images, non-Western languages and other changes resulting from bug and feature requests reported by users, • odt2braille (A2.2.4), with more embosser support, expanded Braille customisation options and other changes resulting from bug and feature requests reported by users, • the accessibility checker for Writer (A2.2.4), which now covers many more accessibility issues, • verifying compatibility of the extensions – both licence-wise and technically – with LibreOffice, a recent fork of OpenOffice.org. It remains too early to answer the question of “reaching parity with, and then exceeding, existing 2nd generation AT”. The 2nd pilot phase has ended; improvements resulting from the pilot phase feedback will be reported in D2.2.1c. With the planned extension of ÆGIS by another 6 months, additional time will allow us to further improve these prototypes, and obtain yet further review and feedback in the 3rd pilot phase. Please note that all training manuals and courses of all solutions developed in the project, October 2012

Oracle

99


PU


including those described in this Deliverable, can be found at http://aegis.bluepoint-it.ro/login.php; some of which (those addressing the first evaluation phase) constitute part of D5.2.3: “Training course for pilots participants” submitted to the EC during the second year of the project . Therefore are not repeated here in order to avoid exceeding length).

October 2012

Oracle

100


PU


References [Abou-Zahra 2011] Abou-Zahra, Shadi, ed. “Evaluation and Report Language (EARL) 1.0 Schema - W3C Working Draft 10 May 2011.” http://www.w3.org/TR/2011/WD-EARL10-Schema20110510/ [ANSI/NISO Z39.98-2012] DAISY Consortium, NISO. ANSI/NISO Z39.98-2012 – Authoring and Interchange Framework for Adaptive XML Publishing Specification. July 9, 2012. http://www.daisy.org/z3998/2012/z3998-2012.html [ApacheDonation] Oracle Announces Its Intention to Move OpenOffice.org to a Community-based Project http://emeapressoffice.oracle.com/Press-Releases/Oracle-Announces-Its-Intention-toMove-OpenOffice-org-to-a-Community-based-Project-1ca9.aspx [ApacheIncub] Apache Software Foundation. “OpenOffice.org Incubation Status” (no date). http://incubator.apache.org/projects/openofficeorg.html [BANABraille] Braille Authority of North American (BANA) and Shodor Educational Foundation: Braille Formats: Principles of Print to Braille Transcription 1997: http://www.brl.org/formats/index.html [BrailleUtils] Braille Utils. http://code.google.com/p/brailleutils/ [DAISY4Core] DAISY Consortium, NISO. NISO Z39.86-201x Part A — Core Modules. Revision 2011-03-28. http://www.daisy.org/z3986/2011/auth/cm/ [DAISYPipeLineScript] DAISY Consortium: Pipeline Script: ODT to DAISY 3.0 XML: http://data.daisy.org/projects/pipeline/doc/scripts/OdtToDtbook.html [DAISYObi] DAISY Consortium: Obi: DAISY/NISO Audio Authoring Tool: http://www.daisy.org/obi [DAISYTobi] DAISY Consortium: Tobi: a software tool to author DAISY multimedia: http://www.daisy.org/tobi [DAISYTools] DAISY Consortium: Tools & Services: http://www.daisy.org/tools/579 [DolphinProducer] Dolphin: EasyProducer: Digital talking book creation tool (formerly Dolphin Producer): http://www.yourdolphin.com/productdetail.asp?id=10 [Gnome3] The GNOME Project: GNOME 3 – Made of Easy: http://gnome3.org/ [GnomeAccerciser] The GNOME Project: Accerciser: http://live.gnome.org/Accerciser [GnomeAccPreferences] The GNOME Project: New Accessibility Preferences UI: http://live.gnome.org/Accessibility/NewPreferencesGUI#Seeing_Tab [GnomeBug639851] GNOME Bugzilla: Bug 639851 - Magnifier: Add brightness and contrast functionality: October 2012

Oracle

101


PU


https://bugzilla.gnome.org/show_bug.cgi?id=639851 [GnomeMag] The GNOME Project: GNOMEMag: http://live.gnome.org/GnomeMag [GnomeMagnifierFramework] The GNOME Project: Common Magnifier Framework: http://live.gnome.org/Accessibility/MagnificationFramework [GnomeShell] The GNOME Project: GNOME Shell: http://live.gnome.org/GnomeShell [Honline] The H Open Source. “Oracle: OpenOffice.org to become "a Community-based Project" – Update.” 15 April 2011. http://www.h-online.com/open/news/item/Oracle-OpenOffice-org-tobecome-a-Community-based-Project-Update-1228831.html [ICEBUEB] International Council on English Braille (ICEB): Unified English Braille (UEB) Project: http://www.iceb.org/ueb.html [JiscCreate&Convert] Joint Information Systems Committee (JISC) - Regional Support Centre Scotland North & East: Create&Convert: http://www.rsc-ne-scotland.ac.uk/eduapps/createconvert.php [LDTP] Linux Desktop Testing Project (LDTP): http://ldtp.freedesktop.org/wiki [LiboDAISY] odt2daisy. LibreOffice Extensions. http://extensions.libreoffice.org/extensioncenter/odt2daisy/ [Mago] Canonical: Mago – A Desktop Testing Environment. http://mago.ubuntu.com/ [NetBeansOpenOffice] NetBeans. “OpenOffice.org API Plugin - plugin detail” http://plugins.netbeans.org/plugin/2320/openoffice-org-api-plugin [OpenHApache] The H Open Source. “OpenOffice.org accepted into Apache Incubator.” 14 June 2011. http://www.h-online.com/open/news/item/OpenOffice-org-accepted-into-ApacheIncubator-1259664.html [OpenOfficeBraille] OpenOffice.org repository for Extensions: odt2braille: http://extensions.services.openoffice.org/en/project/odt2braille [OpenOfficeDAISY] OpenOffice.org repository for Extensions: Export As DAISY (XML+Audio) | odt2daisy: http://extensions.services.openoffice.org/en/project/odt2daisy [OpenOfficeEOL] OpenOffice.org: Information about releases that have reached "End-Of-Life" status: http://development.openoffice.org/releases/eol.html [OpenOfficeExtensionsDev] OpenOffice.org: Extensions Development: http://wiki.services.openoffice.org/wiki/Extensions_development [OpenOfficeNetBeans] OpenOffice.org: OpenOffice NetBeans Integration. http://wiki.services.openoffice.org/wiki/OpenOffice_NetBeans_Integration [OpenOfficePopularExtensions] OpenOffice.org: Most Popular Extensions: October 2012

Oracle

102


PU


http://extensions.services.openoffice.org/most_pop_ext [Rooney] Rooney, Paula. “OpenOffice.org forsakes Oracle, forms new foundation and fork.” ZDNet, News and Blogs, 28 September 2010. http://www.zdnet.com/blog/opensource/openofficeorg-forsakes-oracle-forms-new-foundation-and-fork/7445 [TDFLic] The Document Foundation. “Get Involved Developing LibreOffice” (no date). http://www.documentfoundation.org/develop/ [VaughanApache] Vaughan-Nichols, Steven J. “Oracle gives OpenOffice to Apache.” ZDNet, News and Blogs, 1 June 2011. http://www.zdnet.com/blog/open-source/oracle-gives-openoffice-toapache/9035 [VaughanIBM] Vaughan-Nichols, Steven J. “IBM throws its source code and support behind OpenOffice.” ZDNet, News and Blogs, 14 July 2011. http://www.zdnet.com/blog/opensource/ibm-throws-its-source-code-and-support-behind-openoffice/9240 [D1.1.3] AEGIS Deliverable: “Use Cases and Application Scenarios” [D2.2.1a] AEGIS Deliverable: “First Draft - Open desktop libraries enabling built-in accessibility“ [D2.2.1b] AEGIS Deliverable: “Update - Open desktop libraries enabling built-in accessibility” [D2.1.1] AEGIS Deliverable: “Open accessible desktop system requirements“ [D5.2.3] AEGIS Deliverable: “Training course for pilots participants” [D1.5.2] AEGIS Deliverable: “consolidated evaluation report”

October 2012

Oracle

103

D2.2.1: Open desktop libraries enabling built-in

D2.2.1: Open desktop libraries enabling built-in

Suggest Documents

D2.2.1: Open desktop libraries enabling built-in accessibility

enabling preos desktop management - Semantic Scholar

enabling preos desktop management - Semantic Scholar

iJITinOffice: Desktop Environment Enabling ... - Semantic Scholar

ENABLING OPEN AGENT INSTITUTIONS

ARDO Builtin 2009 EN+schede.pdf

Situational Open Innovation - Enabling Boundary

ENABLING OPEN INNOVATION PROCESS ...

Enabling High Data Throughput in Desktop Grids Through ...

Enabling Green IT through Building a Virtual Desktop ... - IEEE Xplore

Enabling High Data Throughput in Desktop Grids Through ...

University of Arkansas Libraries ArcGIS Desktop Tutorial Section 2 ...

Are Desktop Computers Still Relevant In Today's Libraries?

University of Arkansas Libraries ArcGIS Desktop Tutorial Section 6 ...

Enabling mobile access to digital libraries in digital divide contexts ...

Situational Open Innovation - Enabling Boundary ...

Enabling open-source cognitively-controlled collaboration ... - CiteSeerX

Open-Source Electronics Platforms as Enabling ... - Adalidda

Enabling Open Innovation through Agile Development

Open Educational Resources: Enabling Universal Education.

enabling reproducible research: open licensing ... - Stanford University

enabling reproducible research: open licensing ... - Stanford University

Open access enabling courses: risking academic ...

Enabling Enhanced Learning in an Open