include Apple's Mac OS X and iOS, Linux systems such as. Valve's upcoming SteamBox, Google's Android, as well as other d
TRANSGAMING INC. WHITE PAPER: BINARY AND SOURCE PORTABILITY TECHNOLOGIES MARCH, 2014
TransGaming’s Binary and Source Portability Technologies
TransGaming specializes in the development of technologies that facilitate the rapid and cost effective deployment of complex software across multiple platforms. The company was founded with the goal of providing ubiquitous portability to the video games sector - an especially relevant objective in an era where development across multiple and diverse platforms has become complex and expensive. Today’s developers face unprecedented challenges in choosing which platforms to target when developing applications meant to be used by a wide consumer audience. The once-dominant Windows® desktop platform has given way to a huge variety of new choices: alternative desktop OS platforms such as Linux and Mac OS X; mobile devices such as phones and tablets; HTML-based web platforms, running on cloud-based servers; and a plethora of embedded CE Copyright © 2014 TransGaming Inc. All rights reserved.
systems, ranging from video game consoles to TV platforms. These choices are here to stay. There is no “magic bullet” that can eliminate the complexity of platform choices for developers, as each platform adds its own unique value to the marketplace, and every user decides which platforms are right for them. On modern projects with budgets that can range as high as $200 million, maximizing return by targeting the widest possible audience is a vital consideration for every developer. We at TransGaming believe that developers are best served by having a wide selection of tools and technologies for portability at their disposal, and we aim to produce technologies that are well-honed and perfectly suited to address some of the most difficult problems in cross-platform development.
1
TransGaming’s Binary and Source Portability Technologies
Classic Approaches to Portability There are many paths to developing cross-platform projects, each of which has its own set of trade-offs. In choosing the right path, one critical factor is that the further along a project is, the more difficult it can be to add support for a new platform.
Middleware Engines In the video games industry, one choice some developers make is to use a middleware engine that includes support for multiple platforms. This choice is generally only possible at the beginning of a project. Once development is underway, it isn’t possible for developers to transition to a middleware engine without completely rewriting their title. This approach also leaves developers at the mercy of the middleware vendor for support of new target platforms, not to mention often requiring hefty license fees, especially if source code is desired. The middleware approach is only really applicable in the games business - markets such as creativity software and 3d modeling don’t generally lend themselves to these kinds of engines.
Internal Abstraction Layers Other developers may rely on an internally-built engine or framework technology. In these cases, supporting additional platforms requires that the internal framework is built with enough abstraction to be able to deal with the different APIs that a new target platform may support. The abstraction layer must be updated for each new target platform, which requires enough developers on staff to be able to handle implementation and debugging work to support the new target. Even for the largest companies, this can be an expensive and challenging endeavour. This approach also requires that significant effort be undertaken in advance - either by using a pre-existing internal framework that can support multiple platforms, or by explicitly designing both the application and framework together.
Rewriting Code The most common approach in situations where an application has not been designed with portability in mind from the beginning, developers may need to perform significant code rewrites to support a new platform. Due to the cost of development time, this is usually the most expensive Copyright © 2014 TransGaming Inc. All rights reserved.
option, especially if it is important to continue to maintain the application across multiple platforms indefinitely. In effect, achieving the goal of maintainable cross-platform software when the original code was only designed to support a single platform requires developers to create new internal platform abstraction frameworks ad-hoc, while simultaneously refactoring existing code on top of this layer.
TransGaming’s Approach - the API Shim Layer TransGaming takes the view that, in many cases, particularly for games, there is usually little need to extensively rewrite and refactor existing working code. Instead of requiring either expensive middleware engines, extensive ahead-of-time planning and framework development, or a costly rewrite of application code, we advocate the use of API shim layers. An API shim layer is essentially a layer of abstraction that implements the programming interfaces available on a source platform on top of a target system’s available capabilities. API shim layers of one kind or another are used extensively throughout the software industry. One good example is Google Play Services. Google’s Play Store faces the problem that mobile device OEMs are sometimes slow to adopt new versions of Android for existing devices. Applications which require features from newer Android releases can still be sold on the Google Play Store for older devices due to the use of the Google Play Services component, which provides a shim layer that implements the new APIs on top of older OS revisions1. Another good example of an API shim layer is how Apple’s Mac OS X provides compatibility with UNIX software. At its heart, Apple’s Mac OS X uses the Mach microkernel, which works very differently than a traditional UNIX kernel, relying primarily on message passing, rather than the UNIX file-based approach. For compatibility with UNIX APIs, Mac OS X has a tightly integrated kernel-level BSD UNIX based shim layer2, which implements the standard UNIX functionality on top of the Mach interfaces. TransGaming has extensive experience in API shim approaches, with very specialized expertise with graphics API shims. TransGaming’s core portability offerings provide shim layer technology to support the Microsoft® Direct3D® 1 https://developer.android.com/google/play-services 2 http://osxbook.com/
2
TransGaming’s Binary and Source Portability Technologies
APIs on platforms that typically provide the industry standard OpenGL® and OpenGL® ES APIs.
Unlike Direct3D, the OpenGL and OpenGL ES APIs are maintained and updated by an industry consortium, the Khronos Group, which is dedicated to the creation of open standards for parallel computing, graphics, and much more. TransGaming is a Contributing Member of the Khronos Group, and has participated actively in the development of new specifications for both OpenGL and OpenGL ES standards. Many of TransGaming’s contributions to Khronos Group specifications relate to improving compatibility between OpenGL and Direct3D. As an industry standard, OpenGL and OpenGL ES are used as the primary 3D graphics APIs by platforms that include Apple’s Mac OS X and iOS, Linux systems such as Valve’s upcoming SteamBox, Google’s Android, as well as other devices ranging from Smart TVs and Set-top Boxes to aircraft avionics and in-car display systems. Fundamentally, TransGaming’s graphics portability shim technologies allow complex 3D rendering code that was originally written for the Windows and Direct3D world to gain new life and traction within the world of new devices reaching the market which use OpenGL as a primary graphics standard. Ironically, TransGaming has also applied this expertise in the opposite direction, as the initial creators of the ANGLE project3, which provides a conformant implementation of the OpenGL ES APIs on top of Direct3D. Google’s Chrome and Mozilla’s Firefox browsers both rely on ANGLE to provide support for WebGL™, an OpenGL-based graphics API for HTML 5 and JavaScript.
Direct3D to OpenGL Graphics Technology Due to the long dominance of Microsoft in the PC marketplace, and the success of the Xbox® console gaming 3 https://code.google.com/p/angleproject/
Copyright © 2014 TransGaming Inc. All rights reserved.
platform, there is a substantial base of games and application software which use the Direct3D APIs instead of OpenGL to perform 3D graphics rendering tasks. Even though mobile devices such as phones and tablets, which typically use OpenGL ES APIs, have pulled far ahead of PCs in market share, Direct3D has remained an API of choice for more graphically sophisticated games and applications due to the raw performance available in PC and console hardware. TransGaming’s portability technology initially targeted Linux and Mac desktop and laptop systems, enabling PC games to run on those systems. Linux and Mac OS X both support the desktop version of OpenGL, which has capabilities that are relatively similar to those of Direct3D. Mobile and other devices using OpenGL ES were typically lower powered, and had significantly lower graphics capabilities than Direct3D. As such, they had not been well suited as potential porting targets for Direct3D content. With the rise of more advanced mobile chips, capable of far better performance from both CPUs and GPUs, that tide may now be turning. Chips such as Apple’s A7, Qualcomm’s SnapDragon 800 series, and NVidia’s Tegra K1 are reaching or beginning to surpass the capabilities present in the previous generation of console gaming devices such as the Xbox 360 and PlayStation® 3. On the Mac side, Apple is gaining more and more market share of consumer desktop and laptop platforms, making it ever more important as a platform for games. The upcoming release of Valve’s SteamOS and 3rd party OEM Steam Machine consoles now mean that Linux is a platform that developers must begin paying serious attention to. Given the spectacular rise of all of these OpenGL based platforms, TransGaming believes that there are tremendous opportunities for developers with more advanced PC and Direct3D games and applications to begin to investigate options for bringing their software to a much wider audience.
Technology Details The Direct3D APIs, along with both the PC and console hardware that they support, have evolved over many years, being constantly updated to add ever more advanced 3D rendering features. Each new version of Direct3D is contained in a dynamic library, or DLL, indicating its version, and these DLLs are implemented at the driver level via a
3
TransGaming’s Binary and Source Portability Technologies
well defined hardware abstraction layer (HAL). TransGaming graphics shim technology currently supports Direct3D versions 7.0 through 9.0c, with support for version 11.x under development. The Direct3D 9 API corresponds to the capabilities of the Xbox 360, while the Direct3D 11.1 API is more representative of those of the newly released Xbox One. On the target platform side, TransGaming’s technology is capable of supporting a wide range of OpenGL and OpenGL ES based devices, ranging from minimally capable fixedfunction OpenGL ES 1.1 hardware all the way to the devices that support Desktop OpenGL 4.x. Based on the target platform OpenGL version and capabilities, our shim technology reports to the application via the Direct3D device caps functionality. This allows applications to change rendering paths depending on which profile or shader model is available to it. Below is a list of rough equivalents of OpenGL versions and Direct3D profiles and shader models: Direct3D 7, Direct3D 8 and Direct3D 9 fixed function
OpenGL ES 1.1
rendering Direct3D 8 and Direct3D 9 Shader Model 2.0 Direct3D 9 Shader Model 3.0
OpenGL ES 2.0 OpenGL 2.x + extensions / OpenGL ES 3.0
Direct3D 10 Shader Model 4.0
OpenGL 3.2
Direct3D 11 Shader Model 5.0
OpenGL 4.4
If a target platform does not have sufficient capabilities at the OpenGL level, TransGaming also offers an alternative with SwiftShader, an optimized software-based rendering solution. More information about SwiftShader can be found here: http://transgaming.com/swiftshader TransGaming’s implementation of the Direct3D APIs provides a front-end library for each of the different versions of Direct3D, and uses an abstraction layer interface similar to the HAL found in Microsoft’s implementation. The HAL interfaces with two subcomponents. One component deals with graphics context management for the target platform (X11/GLX on Linux Desktops, CGL/Cocoa on Mac OS X, or EGL on embedded devices). The other component handles the bulk of the work in translating the Direct3D API Copyright © 2014 TransGaming Inc. All rights reserved.
command stream requests to their OpenGL or OpenGL ES equivalents. The main tasks for the API translation component include: state management, resource loading and possible conversion, shader program translation, thread synchronization, as well as actual rendering.
Challenges in Translation While in many respects, Direct3D and OpenGL APIs appear similar, there are many challenges involved in the Direct3D to OpenGL translation effort that TransGaming’s technology is able to effortlessly handle for developers who choose it. Among these are: • Shader translation - since Direct3D and OpenGL use completely different shading languages for their programmable pipelines, TransGaming’s shim layer must dynamically translate Direct3D shaders into OpenGL shaders. • Texture formats that are not natively supported by either the OpenGL API or the target platform may require complex operations to be rendered on the target platform. These may include swizzling, data unpacking, etc. Memory usage and performance concerns are of the utmost importance here. • Direct3D 7-9 and OpenGL handle texel centres differently when sampling. This requires special consideration when converting shader instructions to OpenGL to avoid sampling issues. • Multisampling / Anti-aliasing support in OpenGL is exposed very differently than its Direct3D counterpart and requires extra considerations when resolving multisampled render targets. • Direct3D uses a right-handed coordinate system for its 3D space while OpenGL uses a left-handed system. This must be carefully considered when applying transformations in both fixed function and programmable mode, especially when converting to clip space coordinates. • There are many examples of undefined default behaviour in Direct3D that clash with similar undefined behaviour in OpenGL. These can be exposed if a Direct3D application renders content without handling the undefined cases. Some examples include: ᵒᵒ The non-debug version of the Direct3D DLL on Windows allow pixel shaders to read shader inputs that were never explicitly written by the corresponding ver-
4
TransGaming’s Binary and Source Portability Technologies
tex shader. On OpenGL this would cause a linking error at runtime, but in Direct3D a default vector (0.0, 0.0, 0.0, 1.0) is silently used as an input value. ᵒᵒ Direct3D shaders implicitly write the fragment coordinate depth even if this is not requested by the shader itself. OpenGL only writes to the depth buffer when requested to do so. ᵒᵒ NaN and Inf values that are the results of calculations in shaders (when a divide-by-zero is encountered, for example) can create visual artifacts in OpenGL that are not present in Direct3D. TransGaming’s technology provides optional guard instructions to explicitly avoid these situations. A more detailed look at some of the differences between Direct3D and OpenGL that TransGaming’s technology surmounts can be found in the ANGLE sample chapter of the recently released book, OpenGL Insights: http://openglinsights.com/. While the description there covers the opposite problem, the core differences between the APIs remain the same.
Additional Features In addition to handling API specification differences such as those described above, TransGaming’s technology provides many additional features that ensure all Direct3D content is rendered in an optimized and robust way that minimizes graphics glitches and performance bottlenecks. Some of these performance features include: • Sophisticated caching schemes to avoid CPU overhead in OpenGL drivers ᵒᵒ Caching of render and sampler states that map to the OpenGL state machine ᵒᵒ Caching of shader constants to avoid unnecessary data loads between draw calls ᵒᵒ Caching of dynamically generated shaders, based on runtime specializations that may be inserted into a shader based on render state settings
• Flexibility for certain rendering features to be added as part of the shader translation process if it proves to be more performant or works around missing features on a target platform. Examples include: ᵒᵒ User clip planes implemented in pixel shaders ᵒᵒ Optional sRGB texture support implemented in pixel shaders • A modular approach to advanced OpenGL extensions availability using a runtime feature table that allows us to use all features available on a given target platform • Full support for both of Direct3D’s older fixed function pipelines on top of dynamically generated OpenGL shaders, allowing legacy content to run well on purely programmable platforms such as OpenGL ES 2.0 and 3.0
Performance Given the amount of data being pushed to the GPU in a typical 3D scene as well as the amount of computation required to render it, performance is of the utmost concern to any graphics developer in the gaming industry or otherwise. TransGaming’s Direct3D to OpenGL portability code has benefited from 30+ developer-years of work in the highly competitive video game space. A significant amount of that effort has been invested in optimizing each line of code, not only to limit the CPU overhead needed for translation, but also to ensure that the data being rendered is updated and presented to the OpenGL APIs in a way that makes the best use of underlying drivers and hardware. The TransGaming team always strives to achieve optimum performance on any target platform - typically, the limiting factor for performance is the target OpenGL implementation, rather than any overhead in the translation layer. Measured CPU overhead for TransGaming graphics translation components typically amounts to 3-5% of overall application CPU time used in a modern game. This should not be surprising - for the most part, modern graphics effects require more work on the GPU side than the CPU side.
• Support for multithreaded Direct3D rendering ᵒᵒ All calls to the underlying OpenGL library can take place in a separate thread, avoiding application code stalls during API translation and OpenGL driver code execution Copyright © 2014 TransGaming Inc. All rights reserved.
5
TransGaming’s Binary and Source Portability Technologies
TransGaming’s Binary Portability Technology: Cider and Cedega TransGaming’s efforts to build API shim technology began with our first commercial product, Cedega. Cedega, originally named WineX, had its origins in the WINE project, an effort to create a full scale binary compatibility system to allow end-users to run arbitrary Windows applications on Linux systems. In 2006, when Apple began using Intel x86-based processors on the Mac platform, TransGaming introduced Cider, which uses the same core technology as Cedega, but aimed at video game developers and publishers, rather than end-users. Cider has been used to bring some of the world’s best-selling PC-based game content to the Mac, including titles from publishers such as Electronic Arts, Activision, Disney, Take Two, and many others. With 2014’s upcoming release of Valve’s Linux-based SteamOS, TransGaming is extending the same developer and publisher based approach used with Cider to the Linux platform, enabling the simultaneous day-and-date deployment of upcoming games to Windows, MacOS, and Linux/SteamOS.
Binary Portability Technology TransGaming’s binary-level portability technology works by combining the shim API technology described above with a full runtime environment which mimics the user-space runtime environment for Microsoft Windows 32-bit applications. This includes everything from user-space memory layout, to exception handling, to support for directly loading Windows PE-format binary applications (EXE and DLL files) into memory to be executed. A Windows application that has been wrapped in TransGaming’s binary-level portability technology is actually executing the final compiled Windows binary code on the new target platform. In addition to the Direct3D translation library described above, the binary portability layer also provides proprietary shim layers for the other multi-media libraries in the Microsoft DirectX API suite, including XInput and DirectInput for mouse, keyboard, and game controller input, and XAudio2 and DirectAudio for sound processing. Together with implementations of the many other ancillary but essential Windows APIs for low-level file, thread, window, and memory management, these provide a complete environment for running Windows multimedia applications, Copyright © 2014 TransGaming Inc. All rights reserved.
including the most performance intensive AAA game titles, on Mac OS X and Linux desktop hosts.
Windows API DLL Stack In addition to the core multi-media libraries described above, the Win32 API also consists of a large collection of system DLLs, each implementing a specific set of related OS functionality, such as file access, threading, and window management. Cider and Cedega provide native implementations of over 200 of the most important of these interfaces. Each of these DLLs provides the externally visible API entry points that a Win32-based program requires to function. The DLL implementations provided by the binary portability layer are themselves written in portable code, and can be compiled to the native shared library format for the target platform. While much of the code to implement the various DLL shim libraries is fully generic, there are a number of components which require platform-specific code on different targets, in order for the Windows application to behave like other programs on the target OS. For example, when targeting Mac OS X, the window management components in the USER32.DLL library interface to the system using Apple’s Cocoa windowing APIs, while on Linux, those same APIs would be implemented to interface with the X11 Windowing toolkit. These implementation differences are transparent to the source application being ported, which only interfaces with the stock Win32 APIs.
Kernel Request Server The final component in the TransGaming binary portability stack is the Kernel Request Server. This is a separate process which acts as a clearinghouse for all kernel level requests from a Windows application. APIs that perform kernel-level functions such as thread creation and synchronization, file handling, kernel object management, etc, are all routed to this global server process to ensure that these functions are appropriately serialized between multiple Windows processes and threads being executed within the portability environment. To ensure that the performance overhead is minimized, many functions are implemented to access the kernel request server’s global structures directly through shared memory, rather than through IPC. This allows the implementation to operate on the shared memory owned by the
6
TransGaming’s Binary and Source Portability Technologies
global kernel request server process, but execute within the context of the calling thread in the requesting application, optimizing performance by a factor of up to twelve times compared to a pure IPC implementation.
platforms using our technology include: • ArenaNet’s Guild Wars 2
Platform Integration Components
• Cryptic Studios’ Star Trek Online
TransGaming’s binary portability technology stacks also include platform integration components, which allow for the ported Windows applications to fit into the application ecosystem on the target platform. For Cider, this includes a number of tight integration points with Mac OS X, including tools for creating .DMG disk archives and installers for application deployment, patchers and updaters to keep the application up-to-date with current releases of the portability stack, integration with the Apple Mac App Store including support for in-app purchases, and integration with 3rd party Mac native libraries such as Steam game store APIs or Google’s Chromium web client library. The same kind of native integration capabilities can also be supported on Linux. The binary portability stack also includes proprietary support for copy protection and DRM schemes, to support games and other applications which require integration of these components.
• EA’s SPORE, The Sims 3, and many others
Commercial Deployments
• Rockstar’s Grand Theft Auto series and Max Payne 3 • CCP’s EVE Online
• EA’s FIFA 12 The binary portability technique has proven particularly successful with massively multiplayer games, or MMOs. TransGaming’s technology allows these games to leverage the existing build and deployment mechanisms that they use under Windows, vastly simplifying the process of handling updates, which can happen as frequently as once per day in some cases. One caveat of the binary portability approach is the limited tightness of integration possible with the target platform, especially in the areas of desktop window management and other OS-level user interface integration. Games have traditionally eschewed reliance on desktop windows and user interface elements, preferring instead to create immersive game-specific user interface components, often in a full-screen environment, so desktop UI integration is not a prime concern. Non-gaming applications, which require a more integrated desktop UI may not find that the binary portability stack is the best approach to targeting new platforms.
TransGaming’s Source Level Portability Technology
Due to the broad base of support for multimedia APIs, and the ease with which applications can be brought to new platforms using this technology, TransGaming has enjoyed tremendous success in the games marketplace with binary portability techniques, with millions of units shipped to end users. Well known brands that have been enabled for new
Copyright © 2014 TransGaming Inc. All rights reserved.
While TransGaming’s binary-level portability approach has been very successful so far, there are fundamental limitations in its ability to reach markets that are not dominated by Intel’s x86 CPU compatibility, such as the rapidly growing mobile phone and tablet space. Furthermore, properly addressing the needs of these markets also typically requires more sophisticated changes to an application’s user interface in order to handle both areas such as touch-based control as well as other mobile device paradigms in areas such as multitasking. In order to address these and other potential markets, in 2014, TransGaming will commercially release new technology that allows application developers to take full
7
TransGaming’s Binary and Source Portability Technologies
advantage of many of our Win32 portability shim technologies at the source code level. This will allow developers to reuse existing code as a starting point, while simultaneously embracing all the features available in new target devices.
Native Platform Compilation of the TransGaming Graphics Library By leveraging our existing implementations of select strategic Windows APIs from our portability stacks, TransGaming is able to bring proven expertise to new markets which do not require binary-transparent portability, but instead will be leveraging native technologies and APIs for better platform integration. Initial targets cover the Direct3D graphics library described above. The core code in TransGaming’s Direct3D to OpenGL translation libraries is platform independent; the code can be recompiled as a library on any operating system and CPU flavour that supports the OpenGL or OpenGL ES graphics APIs. Games which have been ported to Mac OS X or Linux using Cider or Cedega are limited to the 32-bit x86 platform support which those portability stacks provide. Because the binary portability approach is monolithic on a DLL basis, it is sometimes not immediately clear to developers if a given function is not supported by the shim layer. With source level portability technology, any such issues become immediately clear. Any function not supported in the shim layer will result in a build error, rather than a runtime error. This allows developers the flexibility to resolve platform differences in the way that makes sense for their particular application, whether by changing application source code, or by working with TransGaming to extend the shim library for their specific requirements. TransGaming’s source-level portability stack can be provided to customers in source code form, open to any possible target platform and with the potential for custom changes. Alternatively, it can be provided as a pre-built library in native shared object form, licensed on a per target platform basis. In either case, the native platform linker is used to integrate the TransGaming code with the target project; this eliminates the need for the full runtime environment present with the binary portability technology, such as the PE Exectuable Linker/Loader and Kernel Request Server components. This allows for tighter integration of customers application source code with the TransGaming library code, and exposes additional optimization Copyright © 2014 TransGaming Inc. All rights reserved.
opportunities which do not exist in a binary-portability environment. The source-level portability technology can be either statically or dynamically linked, depending on what is best for a customer’s application. TransGaming’s library approach to source-level portability has been field tested to great success by a pioneering Fortune 1000 cloud services provider, which has integrated TransGaming’s Direct3D 9 graphics translation library into applications targeting Mac OS X and embedded ARM Linux platforms. As the library is extended, additional shim layer components from our binary portability technologies will be included in the source portability library, such as the D3DX9 graphics utility library, the XAudio2 and DirectAudio sound APIs, and XInput and DirectInput for controller support.
Target Platform Support The source level portability library can be compiled on any native platform with a modern C/C++ runtime and OpenGL support. The capability level of source Direct3D code supported for a given target platform depends only on the version of OpenGL (Desktop or ES) available on the platform; for details of the mappings supported, see above. The Direct3D 9.0 compatibility library is already production-ready on the Mac OS X and Linux running on embedded ARM hardware. In 2014, our source level portability approach will also support Android and iOS devices, as well as x86-based Linux using X11, such as SteamOS. Future platform support is limited only by the needs of the market. Partners and customers who choose sourcecode level licensing can retarget the library for any platform, allowing a more agile response to changing markets and business requirements.
Integration with Target Platform Engines and Technologies As discussed above, one limitation of the binary portability approach of the Cider and Cedega stacks is the limited flexibility in platform UI integration. Because the binary being ported was written to target the Windows platform, any binary porting technology will have to strike a compromise between full fidelity with the target platform user experience, and complete compatibility with the Windows APIs used in the code being ported. TransGaming’s source level portability technology addresses this limitation, by allowing the downstream
8
TransGaming’s Binary and Source Portability Technologies
developer full control over integration between the TransGaming graphics library and the target platform windowing and presentation environment. While this approach does require more code to be written by the downstream developer, it allows deeper platform integration and case-by-case selection of the tradeoffs to be made between transparent portability and seamless integration. The primary interface between TransGaming’s sourcelevel graphics portability library and the downstream application code is the TG3D context management API. This API allows the application developer to provide a graphics context for the TransGaming OpenGL output within the target platform’s windowing system and user interface toolkit. TG3D provides a generic 3D graphics context handling interface, and defines a set of callbacks which are implemented within the target application. These callbacks are used to instantiate and manipulate the graphics context within the host UI environment. TransGaming has created a number of example graphics context implementations of the TG3D callbacks, targeting an array of different platforms and environments. These include: • X11/EGL and DRM/GBM for embedded Linux
technology, rendering code can usually be left largely untouched. TransGaming can provide developers extensive information on maximizing Direct3D performance on target GL platforms through preferred data formats, shader implementation tips, and other details. In many cases, small, tactical changes can provide significant performance wins and increase the robustness of an application with very little developer effort. Some examples are provided below: • Texture Formats: Source level integration allows developers to target texture formats that have good OpenGL support and can be loaded optimally on the target platform. For example, PVRTC or ETC format texture compression can be used without needing to modify other graphics pipeline code.
• Cocoa for Apple’s Mac OS X
• Surface Updates: Surface updates and copies can be optimized when considering features and extensions available on a target platform. For example, OpenGL ES extensions exist which allow optimized subsurface updates of noncontiguous memory that would otherwise require CPUside memcpy calls. With source-level integration, developers can have greater control over unnecessary copies, improving performance in some cases.
• X11/GLX for Desktop Linux platforms • Android/EGL for Android NDK development These example implementations are provided in source form, even for binary-level licenses of the graphics translation code, allowing downstream developers to modify the existing code to adjust the details of the platform integration on existing supported platforms, or to learn from the techniques used when creating a new implementation for a target platform that is not currently supported by the example suite.
Benefits of Source Level Integration for Graphics In addition to enabling our shim technology on multiple non-x86 platforms, an additional benefit afforded by source level integration is the flexibility it gives the application developer to address performance bottlenecks on target platforms without requiring a full rewrite of their rendering engine. Aside from areas such as audio and input handling, 3D rendering code is typically one of the areas requiring the most platform specific code for many applications and games. When using TransGaming’s source level porting Copyright © 2014 TransGaming Inc. All rights reserved.
• Buffer Updates: Depending on the type of OpenGL platform being targeted, dynamic vertex and index buffers may not behave the same way they do on a Windows platform. Custom integration at the source level affords the opportunity to make sure buffer updates are implemented in a way that avoids driver-specific performance bottlenecks (such as an entire buffer being updated on the GPU when only a small subset of buffer data was changed).
• Undefined Shader Behaviour: As discussed above, there are several cases where undefined behaviours can differ between OpenGL and Direct3D platforms. Although TransGaming’s graphics translation library has safeguards available to avoid these, it may be advantageous, to change undefined behaviour on the application side in order to avoid additional overhead from these safeguards. For example, the release versions of the Direct3D DLLs allows pixel shaders to read shaders inputs that were never explicitly written by the corresponding vertex shader these inputs get assigned the Direct3D default (0, 0, 0, 1) vector. While this is incorrect application behaviour which is flagged by the Direct3D debug DLLs, when running in a binary compatibility environment it may not be as obvious
9
TransGaming’s Binary and Source Portability Technologies
that this is the source of a graphics anomaly on-screen. When using the source-portability approach, this kind of error is easier for developers to notice, as it is unencumbered by the black-box nature of the full runtime environment. In this case, a developer would have the flexibility to modify their shaders to eliminate the incorrect code, rather than relying on the porting library safeguards with their additional overhead.
TransGaming Snap for Application Evaluation and Debugging As part of the source-level portability technology suite, TransGaming provides Snap, a tool which records all application traffic across the Direct3D API boundary on a native Windows platform. These API-level recordings can then be played back on the target platform, using a TG3D implementation to host the Snap rendering context. This record-and-playback process serves two important functions in the application porting lifecycle. At the start of the product cycle, Snap can be used as an evaluation tool. A Snap recording can be taken from the source Windows application running in its native environment, without modification. The recording can then be played back on the target platform without any of the ported application’s code being compiled, allowing rapid evaluation of graphics performance and fidelity on the target platform with minimal coding. If the target platform is already served by one of TransGaming’s example implementations of the TG3D interface, in fact, this can be accomplished without a single line of code; if support does not already exist in the example suite, a minimal evaluation version of the TG3D layer can be completed in less than 1000 lines of code, depending on the platform. Later in the development cycle, this same technique can be used for detailed debugging of graphics problems in the target application, working with TransGaming’s development team, or with platform OpenGL driver engineering teams. These problems may be due to uses of undefined behaviour in the Direct3D API, gaps in the library implementation, or bugs in the existing implementation. By providing a Snap recording which reproduces the observed behaviour along with a bug report, the rendering problems can be observed, tested and corrected by TransGaming developers, without
Copyright © 2014 TransGaming Inc. All rights reserved.
requiring any access to proprietary application code. Snap is built using a shim implementation of the Windows Direct3D DLL library, just like our portability environment. However, instead of translating the Direct3D API calls to a flavour of OpenGL, the calls are recorded, and then optionally passed through to the native Direct3D implementation for rendering. Hooking Snap into an application can be as simple as dropping a copy of the Snap version of the D3D9.dll into the application’s binary directory.
Future Portability Technology While all of the technology described above brings tremendous benefit to developers looking to target new platforms, in many ways it is just the tip of the iceberg compared to what may be possible with TransGaming’s portability technology in the future. In addition to support for other multimedia APIs for audio, input, and more through source-level shim layers, and support for new target platforms, there many other possibilities that TransGaming is exploring:
Application Streaming One potential consideration is to deploy our technology in conjunction with cloud-based application streaming. In a streaming model, applications are run on a server, and a compressed stream of data is sent from the server to a client over the network. Today, anything that involves 3D graphics operations typically requires the server side of the system to include expensive GPU hardware, which not only renders the 3D content locally, but also must encode the resulting video to a format like h.264 in real time. As a result, server hardware for application streaming is expensive, and only support a limited number of clients running simultaneously. If instead, 3D graphics commands are sent directly down to the network layer, a client device could render those commands on its own GPU. Not only would a GPU not be required on the server end, but it may be possible to achieve even better compression ratios for some applications, while avoiding the graphics artifacts often inherent to video-based processing.
10
TransGaming’s Binary and Source Portability Technologies
TransGaming has already delivered related technology to some customers, including the capability to handle both the compressed streaming of Direct3D graphics commands, as well as rendering those commands to an OpenGL ES client device.
AMD’s recently introduced Mantle API may be a good candidate for this work, as it could enable support for Direct3D 11 on platforms ranging from the PlayStation 4 to Valve’s Linux-based SteamBox.
Targeting HTML 5 / WebGL
TransGaming’s technologies have already proven their value in helping a broad range of games and applications reach new target audiences. With the launch of TransGaming’s source-based portability technology later this year, developers will be able to reach even more markets such as ARM-based tablets, phones, new consoles, and embedded devices. At the same time, source-level portability enables developers to create even closer bonds between their ported software and new target platforms, taking advantage of the best of both worlds. On the binary portability side, TransGaming going back to its historical roots and reinvigorating our Linux technology in order to help developers and publishers bring games to Valve’s SteamOS for day-and-date release. With the same level of cooperation that top publishers have relied on with Cider on the Mac, this approach provides a robust alternative to custom redevelopment to target Linux. Whether using source or binary level approaches, with TransGaming’s technology, developers can achieve rapid time to market with ported applications without compromise to platform integration and performance. If you would like to reach us to find out more about either our binary or source level porting technologies, please contact us by email at:
[email protected]
In addition to the potential impact our graphics translation technology may have for the mobile marketplace, another target that we expect to explore with customers is the ability to do rendering of Direct3D content to the web through HTML 5 and WebGL. While TransGaming’s porting technology is built in C and C++, new tools such as emscripten and asm.js have recently become available that would allow both our portability layer as well as games and applications that use it to be compiled to a binary optimized version of the JavaScript language used by web browsers everywhere. When combined with the WebGL graphics interface, this means that it may be possible to port many PC titles to run directly inside a web browser. This is an especially interesting area of interest for TransGaming, given our role in helping to specify the WebGL standard and in implementing the ANGLE portability layer for Google.
Direct3D 11 and Next Generation Consoles Finally, while there exists a substantial library of Direct3D 9 based code and content which developers may wish to retarget to new platforms, with the introduction of new console devices from both Sony and Microsoft in 2013, more and more developers are now using the more advanced Direct3D 11 API for new code. TransGaming has begun development to support these new APIs, and has been active in the Khronos Group in helping to ensure that features required to fully support Direct3D 11 through our shim layer are being exposed in the more recent versions of OpenGL specifications. TransGaming’s efforts to build a Direct3D 11 compatibility library are underway, and our prototype is already capable of rendering basic geometry - we are very excited about the future prospects for this technology. In addition to using OpenGL as a target for this implementation, it is also interesting to consider the potential to target even lower level GPU APIs for Direct3D 11 support. Copyright © 2014 TransGaming Inc. All rights reserved.
Conclusion
11
TransGaming’s Binary and Source Portability Technologies
Copyright Notice
Disclaimer and Limitation of Liability
This document © 2014 TransGaming Inc. All
ALL DESIGN SPECIFICATIONS, DRAWINGS,
rights reserved.
PROGRAMS, SAMPLES, AND OTHER DOCUMENTS (TOGETHER AND SEPARATELY,
Trademark Notice
“MATERIALS”) ARE BEING PROVIDED
TransGaming, Cedega, Cider, SwiftShader, GameTree TV, the TransGaming logo, and the GameTree TV logo are trademarks of TransGaming, Inc. in Canada, the United States, and other countries. Other company and product names may be trademarks of
“AS IS.” TRANSGAMING MAKES NO WARRANTIES, EXPRESSED, IMPLIED, STATUTORY, OR OTHERWISE WITH RESPECT TO THE MATERIALS, AND EXPRESSLY DISCLAIMS ALL IMPLIED WARRANTIES OF NONINFRINGEMENT, MERCHANTABILITY,
the respective companies with which they
AND FITNESS FOR A PARTICULAR PURPOSE.
are associated.
Information furnished is believed to
be accurate and reliable. TransGaming, Inc. assumes no responsibility for the consequences of use of such information or for any infringement of patents or other rights of third parties that may result from its use. No license is granted by implication or otherwise under any patent or patent rights of TransGaming Inc. Specifications mentioned in this document are subject to change without notice. Technologies discussed in this document are not authorized for use in devices or systems without express written approval or license from TransGaming, Inc.
For more information email:
[email protected] or visit: http://transgaming.com/
TransGaming Inc. Overview TransGaming Inc. (TSX-V: TNG) is the global leader in developing and delivering platform-defining social video game experiences to consumers around the world. From engineering essential technologies for the world’s leading companies, to engaging audiences with truly immersive interactive experiences, TransGaming fuels disruptive innovation across the entire spectrum of consumer technology. TransGaming’s core businesses span the digital distribution of games for Smart TVs, next-generation set-top boxes, and the connected living room, as well as technology licensing for cross-platform game enablement, software 3D graphics rendering, and parallel computing. Visit us at www.transgaming.com.
TransGaming Inc. Head Office: 431 King Street West Suite 600 | Toronto, Ontario | M5V 1K4, Canada | (416) 979-9900
Copyright © 2014 TransGaming Inc. All rights reserved.
12
