Flex and Mobile White Paper
Flex and mobile: a vision for building multiscreen applications Revised June 2010 Note: The original version of this whitepaper referred to a project code-named “Slider”. “Slider” was originally planned to be a separate version of the Flex framework that would contain mobile-specific functionality. However, due to the evolution of the mobile device landscape, as well as significant performance improvements in the latest releases of the Adobe® Flash® Player and Adobe® AIR® runtimes, it is now feasible to integrate these mobile development features directly into the core Flex framework. As a result, the mobile features that were originally planned for the separate “Slider” framework will now simply be a part of the next version of Flex, codenamed “Hero”. This updated whitepaper describes the new unified framework vision.
Table of contents 1 Introduction 2 Bringing Flex to mobile devices 4 Benefits to developers 5 Cross-device considerations 6 Feature highlights 7 Conclusion
Introduction
A new class of application is emerging that leverages trends in cloud-based computing, social networking, and the rise of a wide range of Internet-connected devices. These new applications work across multiple kinds of devices to provide the most engaging user experiences on any screen. For example, an application might run in one form as a web application that users browse to occasionally, in another form as a desktop application that runs continuously, and in yet another form as a mobile application that users can check while waiting for the bus. The explosion in ways for users to access content and applications brings a new challenge for designers, developers, and content providers. Fundamentally, the challenge lies in creating and maintaining one or more versions of content or application that will run appropriately within a range of target environments. Managing variations across targets that vary widely—from screen sizes to input mechanisms to performance characteristics—can be complex and costly to develop and maintain. This development challenge brought on by fragmentation will only increase over time as the number of access points and devices continues to grow.
In an effort to provide a common platform to run applications across multiple screens, Adobe initiated the Open Screen Project (OSP). Working with leading device manufacturers and other OSP partners, Adobe is developing and deploying a consistent runtime environment for web content and applications across devices, both within and outside the browser. This common runtime provides a foundation for a consistent development environment. The next step is to extend the Flex framework and tooling to facilitate the development of cross-platform applications on non-desktop devices.
Bringing Flex to mobile devices The next version of the Flex framework, codenamed “Hero”, will enable developers to create application experiences that translate well across platforms, and make it easy to build applications that work well on a wide variety of mobile devices. “Hero” will enable a broad spectrum of organizations, from traditional media brands and agencies to internal IT departments, to build a range of mobile applications: from simple companion applications that complement existing web or desktop applications to more complex applications where multiscreen delivery is a core requirement from the outset.
Designing and developing across screens As mobile applications have evolved over the past few years, it has become clear that designing and developing mobile applications is not simply a matter of taking a desktop application and scaling it down onto a small screen. There are two key reasons why developing for non-desktop devices requires mobile-specific design and development patterns. First and foremost, the input mechanisms, screen real estate, and interaction patterns commonly found on mobile devices differ in significant ways from the desktop. Interaction patterns such as touchscreen gestures, five-way navigation, and function-specific device keys are important factors in building mobile applications. Second, non-desktop devices are expected to be more constrained in terms of both memory and processor speed for the foreseeable future. While it is critical to craft the UI and logic of an application appropriately for each class of devices, it is expensive and time-consuming to develop completely separate applications for individual devices or platforms. By taking advantage of a crossplatform framework, designers and developers can avoid unnecessary overhead, and focus their efforts on those aspects of their applications that truly need to be different across screens.
A unified framework
The rapid increases in performance on smartphone-class devices over the past year, combined with the highly optimized performance of Adobe runtimes on these devices, now make it feasible to support mobile use cases directly with the core Flex framework. While previous generations of smartphones had faster hardware than lower-end feature phones, they were not well suited to highly interactive, graphically intensive content. With the advent of high-resolution screens and touch interaction on mobile devices, the landscape has changed significantly. Touchscreen smartphones now typically have 600MHz-1GHz CPUs and OpenGL ES 2.0-based GPUs to accelerate both raw script performance and rendering. By itself, this provides a significant boost to rich mobile applications. Running both on the desktop and on this new class of devices, the Flash Player 10.1 and Adobe AIR runtimes have introduced significant performance improvements. General improvements to the core Flash virtual machine over the past few releases, including just-in-time (JIT) compilation and improved memory management, have improved script execution speed and reduced application startup times. On mobile devices, the runtimes take advantage of available GPU capabilities to boost rendering performance during animations and scrolling, and include optimizations to reduce CPU usage (and therefore improve battery life) when in the background, among numerous other improvements. (For more information on runtime improvements, see “The Engineering Behind Flash Player 10.1” on the Flash Player team blog.) Taken together, these factors make it possible to build performant mobile applications using Flex without major architectural changes in the framework. Thus, rather than split off a separate framework solely for mobile development, the Flex “Hero” framework will support development on both desktop and mobile devices, and set the stage for extending Flex-based development to other classes of devices in the future. This unified approach will enable developers to leverage much of the power of the existing Flex framework in mobile applications, including key features such as dynamic layout, view states, data access, and Spark UI components. Because performance constraints will continue to be an issue on mobile devices, core aspects of the Flex framework, including built-in skins and item renderers, will be optimized for performance on mobile devices. Some aspects of the full Flex framework that are not optimized for mobile use, such as the pre-Flex 4 “MX” component set, will not be recommended for mobile applications. Additionally, Adobe plans to provide tools and best practices to guide developers in building performant mobile applications. Because Flex is built on top of Flash Player and Adobe AIR, techniques such as those described in Optimizing Performance for the Flash Platform provide a good starting point for optimizing Flex-based applications.
Accelerating mobile development
On top of these standard Flex features, “Hero” will add new capabilities specifically designed for mobile development, such as a construct for managing “screens” of an application, and the ability to scale applications automatically on devices with varying pixel densities. New skins and functionality designed for mobile form factors and input methods (e.g. touch-based scrolling) will be added to a number of existing components, and new components will be added to support mobile-specific UI patterns. Finally, the Flex mobile features will encapsulate design principles and guidelines that make it easy to create applications that work well across a range of native platforms. The explosion of applications available for mobile devices has been driven by a shift in deployment models from free in-browser content to standalone applications that developers and publishers can easily monetize through online markets. The mobile features in “Hero” will therefore initially target standalone applications that can be deployed through Adobe AIR on mobile devices that have a processor speed of 600MHz or more and have OpenGL ES 2.0 hardware acceleration.
Benefits to developers “Hero” will enable developers to build multiscreen applications, leveraging their existing Flex development skills; enable rapid adaptation across devices; help developers create intuitive user interfaces that are easily customized; and bring the productivity of the Adobe tool chain to mobile development.
Use existing skills Traditional web and desktop application developers often need to extend applications to mobile and non-PC devices, but are required to learn new programming models and languages to do so, adding time and cost to projects. “Hero” will enable developers to use existing Flash and Flex skills to more easily build and deploy mobile applications. As developers face continued screen migration—from desktop to smartphone to tablet to set-top box—the ability to draw on familiar tools, languages, and development methodologies will enable them to quickly extend content and applications to the latest emerging platforms.
Rapid adaptation across devices Significant rework is often required to deploy an application on different mobile devices, and developers are finding that it’s not always cost-effective to build and maintain multiple code bases. The basic promise of the mobile features in “Hero” is to minimize the cost of creating and porting applications across devices. “Hero” will enable easy adaptation by creating a structure to address variations in input mechanisms such as back and menu buttons, screen-size and screen-density differences, and layout changes based on horizontal and vertical screen orientation. Developers have the flexibility to create platform- or device-specific code where
desired, but the goal of the mobile features in “Hero” is to keep such code to a minimum. For applications where an entirely different user interface is required to fit with the “soul of the device,” developers can choose to recreate the views or composition of the application, but might still want to share code and core assets across versions. Both “Hero” and the supporting tool chain will facilitate this notion of efficient adaptation, making it a much more fluid process to create multiscreen applications.
Intuitive user experience and easy customization
Developers targeting existing mobile platforms have found that it is not always easy to create rich, custom experiences beyond the conventional platform interface guidelines. On one hand, developers want to stay as close as possible to the native platform human interface guidelines to leverage familiar interaction patterns. On the other hand, as developers and clients look to differentiate, there is an increasing need to have complete creative control over the user experience. “Hero” will aim to balance these two perspectives. It will provide an intuitive mobile application experience “outof-the-box” that fits in well with interface conventions across native platforms. It will also be easy to customize so that developers have complete creative control and can design, prototype, and iterate their own innovative multiscreen experiences.
Productive design and development tools
In addition to delivering mobile framework features in Flex “Hero”, Adobe is working to bring the power and productivity of Adobe® Flash® Builder™ to mobile development, in a future Flash Builder release code-named “Burrito”. Features such as interactive step-through debugging, design view, and build packaging for multiple platforms will be available, providing the same rapid development experience for mobile applications that Flash Builder provides for web and desktop applications today. Over time, other Adobe tools, such as Adobe® Flash® Catalyst™, will also be augmented to target mobile devices through Flex, providing a complete workflow for designing as well as developing mobile applications.
Cross-device considerations
Architecting multiscreen applications from the ground up requires making decisions about how both major and minor variations across device targets are handled. “Hero” will help accomplish this by providing for the following types of variations across target environments.
Large differences Large differences occur when the developer is facing a major difference in screen size or form factor (for example, when going from an application that was designed for the web accessed via the desktop to a mobile application). By providing a common language, programming model, and familiar tools, Adobe will make it easier to make
the transition when large differences between devices exist. Developers can architect applications to share logic and model code where appropriate between desktop and mobile versions, while crafting different UIs appropriate to each class of devices.
Medium differences
Medium differences occur when the screen size differences are more minor (for example, from one mobile platform to another), but the platform-specific user interface conventions, supported device APIs, or the hardware support of the device itself mandates a change in the user interface to fit with the “soul of the device.” Out of the box, the mobile features in “Hero” will provide an experience that works well across multiple device platforms. The runtime itself will also provide unified access to key device APIs, such as multitouch and accelerometer support. On top of this common experience model, developers will have the flexibility to customize aspects of the UI for different platforms through conditional code.
Small differences
Small differences are the minor differences that occur when screen size or resolution is only a few pixels different but still different enough to mandate design adjustments to optimize the experience. The layout management and view states features available in Flex, along with some of the new features listed above, should enable the minor differences to be handled at runtime with a single, compiled application.
Feature highlights
“Hero” will include mobile- and touch-optimized skins and functionality for a number of existing Flex components, as well as new components that support mobile-specific UI patterns. Some key “Hero” feature areas are highlighted below.
Screens
Unlike desktop applications, mobile smartphone applications are typically composed of a set of small, task-focused UI screens. “Hero” will introduce this concept of screens into the Flex framework. The APIs provided by “Hero” will allow the developer to manage the creation and transitions between the various screens of a mobile application. Developers will author their applications one screen at a time, each one in a separate MXML file. Navigation between screens will be managed through a navigation stack. The Flex SDK will provide APIs that allow the developer to push screens onto the stack, where the top-most screen is visible and active. In addition, popping back to previous screens and switching between different sections of the application will be possible through “Hero” APIs. This functionality will allow the developer to manage application history and screen hierarchy easily and efficiently. Accompanying the navigation APIs will be a set of full screen transitions to provide fluid contextual animations when navigating the
application. These will be customizable by developers, allowing them to create the experience they envision. To help manage memory in constrained device environments, screens will be disposed by default when they are no longer visible, and recreated as necessary when the user navigates back to them. A screen data persistence API will allow developers to easily maintain state as screens are disposed and recreated. This is advantageous when creating mobile applications because network data can be cached for fast retrieval when needed. Developers will be able to fine-tune the balance between performance and memory through screen management APIs—for example, the automatic disposal of screens can be turned off if desired.
Application components A set of flexible application navigation components will be layered on top of the core screen architecture, providing a standard application structure that works well across multiple platforms. Developers can choose to structure their applications in multiple sections, and provide navigation and tool actions that are either global to the application or contextual to a given screen. By default, the framework will also handle platform-specific controls such as back and menu buttons and orientation changes. Developers will have the flexibility to override the default behaviors, and to create platform-specific application UI if desired.
Layout management The existing dynamic layout capabilities in Flex can be used to manage pixel-level differences in screen resolution, as well as easily adapt screens for horizontal and vertical orientations. In addition, “Hero” will provide a simple mechanism to handle differing pixel densities across multiple devices, by scaling applications so that UI elements appear at approximately the same physical size on all devices regardless of density. This avoids the problem of UI elements becoming too small for touch interaction on high-density devices.
Adapting standard components to mobile devices “Hero” will include a set of skins for existing Flex components that are appropriately sized for mobile touchscreen devices, along with additional functionality that enables touch and five-way interaction in components such as lists. By leveraging the existing Spark component set, developers can build mobile-appropriate UIs through the same methods they use to build desktop interfaces.
Conclusion
As mobile devices, platforms, and deployment models quickly evolve and device competition heats up in the industry, the need for more common approaches to development and deployment will escalate. The mobile computing environment creates massive new opportunities for businesses to offer new products and services to their targeted user bases, while introducing further complexities into an already
complex computing world. Adobe has a long history of success in providing crossplatform development and deployment environments, and is well equipped to deliver a robust, scalable platform for organizations to leverage as a foundation for this new multiscreen application opportunity. The mobile features in “Hero” will minimize the cost of creating and porting applications across devices in this multiscreen world by enabling developers to use existing skills, to rapidly adapt applications across platforms, and to fully control the user experience.
For more information Mobile development with Flex: www.adobe.com/go/flexmobile Optimizing Flash performance: help.adobe.com/en_US/as3/mobile/ Adobe, the Adobe logo, Adobe AIR, AIR, Flash, and Flex are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States and/or other countries. All other trademarks are the property of their respective owners. © 2010 Adobe Systems Incorporated. All rights reserved. Printed in the USA.
