ENTERTAINMENT COMPUTING
User Interfaces for Mobile Content S.R. Subramanya LGE Mobile Research Byung K. Yi LG Electronics
Mobile device user interfaces play a crucial role in facilitating a pleasant and rich user experience.
M
obile phones, rapidly becoming ubiquitous, now claim about 2 billion users worldwide. Popularity is transforming them from simple voice-communication devices to advanced-communication devices that provide voice, text, and video messaging. Further, mobile phones now embody consumer-electronics features such as built-in cameras, MP3 players, and video capture and play. Increasingly, computing functions such as e-mail, Web access, and spreadsheets have been added to the mobile phones’ repertoire. The next generation of mobile systems will operate on a core Internet Protocol network that supports all popular operating systems and software applications. Their high data rates and low latencies, together with their increasing capabilities, will help mobile phones facilitate the delivery of rich multimedia services such as video telephony; streaming news, sports, and movies; and multimedia messaging, Web browsing, and games. Statistics from the first quarter of 2005 show the use of mobile content
and applications in the US grew across all segments, with mobile games attracting 6.23 million users, photo messages 12.24 million, ring-tone downloads 23.09 million, and text messaging 65.68 million. Analysts expect this enormous growth to continue in the near future, with the worldwide mobile content market reaching $59 billion by 2009, according to Juniper Research (www.juniperresearch.com). User interfaces (UIs) for mobile devices play a crucial role in facilitating a pleasant and rich user experience, which makes the design of simple yet effective interfaces for mobile content a top priority.
MOBILE SYSTEMS LIMITATIONS Several inherent limitations of mobile systems must be taken into account when developing and deploying content and user interfaces. The mobility constraints can be broadly classified as technology- or user-related. Technology-related constraints can be further classified into communications- and device-related. Major communications-related limitations are higher error rates, more dis-
connections and dropped calls, higher noise, and relatively lower bandwidths. Device-related constraints include limited resources such as modest processing power, relatively small memory capacity, smaller display size, smaller-sized and fewer buttons, and limited battery power. Despite technological advances, the need for mobile units to be small, light, and cheap will cause the device-related constraints to persist. User-related constraints include limited attention spans during mobility, changing locations and contexts, users’ idiosyncrasies, and personal preferences.
Mobile content UIs The UI must be designed to facilitate the widespread and increased use of mobile content. Current UI approaches in the computing domain—such as windows, icons, menus, and pointing (WIMP)—are inadequate and inappropriate for mobile applications. Mobile device constraints pose several UI design challenges. UIs for emerging mobile devices consist of hardware and software combinations such as • additional buttons for text input and special functional keys, • touch-sensitive screens, • audio/visual interfaces, and • haptic interfaces to provide virtualreality (VR) experiences. Generally, the UI should be intuitive and easy to use, as the immense popularity of Apple’s iPods and iTunes has shown. Simpler interfaces also conserve mobile devices’ precious battery power. For several mobile services, access requires a username and password, and possibly the user’s name, mailing address, and credit card information. For a typical application, navigating the different menus and entering the required information on a mobile phone keypad requires from 60 to 80 clicks. User profile information for personalized services requires even more user input. Optimizing the menus and user navigation and devising novel April 2006
85
ENTERTAINMENT COMPUTING
means of data input will result in a significant reduction in the number of clicks and amount of battery power consumed.
Typical currentgeneration UIs Most current-generation mobile phones have built-in cameras, MP3 players, and several personal organizer applications—such as a calendar, scheduler, notepad, alarm clock, and calculator—with associated UIs. Some models support terrestrial digital multimedia broadcasting, enabling users to view TV programs while making telephone calls. As the number of built-in devices and applications expand on mobile devices, the provision of appropriate UIs for these devices and applications becomes increasingly important. For example, the Sony-Ericsson M600 uses an innovative design to provide a full QWERTY keyboard using no more keys than a standard mobile phone keypad. This eliminates the “multiple tappings” often required to input a single letter and facilitates easy messaging and e-mail. In addition to a touch screen and handwriting recognition, the M600 provides an interface that supports multiple tasks such as browsing the Internet while making a phone call. It also supports Word, Excel, and PowerPoint editors and an Adobe PDF viewer. The LG SV360 has a one-millionpolygon-per-second graphics accelerator chip that lets users play games with minute and realistic 3D graphics. The relatively larger 2.2-inch LCD screen provides a higher resolution for 3D games than most mobile phones can accommodate. The phone also has acceleration sensors that let users interact more closely and provide them with better controls and a more realistic feel, which can enhance game enjoyment. The Panasonic P901iTV can receive terrestrial digital broadcasts and supports services that combine mobile communications and broadcasts. For example, a user could visit a relevant Web site by clicking on the URL displayed during a broadcast. The dis86
Computer
play automatically adapts to the vertical or horizontal positioning of the handset. The Nokia 9300 provides a row of eight dedicated application buttons in addition to a QWERTY keyboard. The user interface includes a fully featured Web browser, a virtual mouse, and multiple windows. The user interface also provides a control panel for setting up and prioritizing network connections.
User interface design must address two distinct purposes: content production and consumption. USER INTERFACES FOR 3G DEVICES The emerging third-generation phones support many rich multimedia data applications and enhanced voicerelated functions. Thus, their UIs must be designed for two broad aspects: data- and voice-related performance. Typical data applications include TV broadcasts, video streaming, music players, multimedia messaging, instant messaging, e-mail, Web browsing, mobile commerce, mobile multiparty gaming, mobile collaboration, and several location-based services. Voice-related applications include voice-activated calls, avatar- and gesture-based displays during calls, and voice-memo-related functions. UIs for applications such as these, which provide a unified experience, present a difficult design challenge. User interfaces on the hardware front include enhanced keypads, special keys and joysticks for gaming, and interfaces to several integrated devices and to specialized external devices such as sensors and medical monitors. Given that, in the near future, mobile users are expected to produce considerable amounts of content, the UI design must address two distinct purposes: content production and consumption.
UI design for content production Content production tools should facilitate producing quality content on mobile devices as well as transferring content between mobile phones and other consumer electronics devices, PCs, and servers over several networks such as Bluetooth, cellular, and WiFi. Also, tools for processing and manipulating the captured or received content must be developed. These tools should have simple UIs that let users manipulate the mobile device’s resources, both hardware and software, more effectively. For example, the tools should enable the integrated use of resources such as the camera, scanner, voice recorder, optical character recognition, graphics, and animation generators. In addition, the UIs should support interoperability among the different resources and applications. The UI should, for example, easily import a scanned copy of a business card into a user’s contact list. The picture taken from the camera should also be easy to process and import or export onto relevant applications, such as a photo-album-management application.
UI design for content consumption At the consumption end, UIs should cater to users’ special mobility needs. For example, given mobile users’ limited attention spans, the UI should have pronounced auditory and visual cues, especially during hands-free operations. Mobile users’ needs for content type and presentation modes change frequently during mobile operations. These users prefer to have control over several content presentation parameters based on time and place, situation, interest level, and cost considerations. Thus, their presentation parameters include • level of content detail, such as fulllength or summaries; • presentation quality, including resolution, color versus gray scale, and frame rate; and
• presentation-style elements such as sound effects, screen layouts, and visual effects. Mobile users also want control over different proportions of media such as audio, video, animations, graphics, and text, based on mood, personal taste, mobility situations, and cost factors. The same content must be presented with different media combinations to suit different mobility situations. For example, the content might need to be rendered in audio-only form while driving, in audio and video while walking, and it could include text in addition to audio and video while stationary. Thus, UIs and techniques that adapt to different mobility situations and render the appropriate media combinations best suited to the current mobility scenario should be developed.
Design for interactivity Unlike broadcast content, mobile content would increasingly be ondemand in most situations, with high interactivity. Consumers will have frequent and meaningful interactions with the content provider system for a variety of purposes, such as • looking up and interacting with content and service directories; • querying for content of interest; • requesting content at certain times, from various locations, and onto specific devices; • sending delivery-control signals to invoke commands such as browse, play, stop, fast-forward, pause, and rewind; • providing feedback on programs’ content and perceptual quality; • setting or updating user preferences; and • following up on advertisements. For mobile users, these interactions should be faster, with many fewer keystrokes and, sometimes, with voice commands.
Design for immersiveness The availability of powerful graphics coprocessors, digital signal proces-
sors, and 3D displays can facilitate 3D visual effects. In addition, acceleration sensors, touch-screen interfaces, and haptic interfaces would enhance immersiveness in games and VR applications. UIs must be designed to take these developments into account so that they can support premier communications services, mobile collaboration, gaming, and VR applications.
OTHER DESIGN CONSIDERATIONS The Web’s many and varied content repositories, the emergence of mobile Web access, and the variety of location-based services that depend on it underscore the need for effective and efficient Web interfaces. Further, users increasingly treat mobile phones as inherently personal devices, as the more than 2,000 different phone models and the immense popularity of ring tones, ring-back tones, and wallpapers show. Designing and providing customizable interfaces has thus become critical when catering to individual tastes and preferences. The design of optimal navigation and uniformity across several menus, within the same application as well as across different applications, must also be addressed. Developers thus face the challenge of determining the sets of UI components and their features and benefits to satisfy different user needs. Further, developing content consumption models and determining how they relate to UI design will require serious study. UI design poses several challenges such as handling several on-device and external devices and supporting a variety of voice and multimedia data applications while being fast, simple to use, and customizable. Coping with these challenges will require several advances in the tools for UI design and development. Currently, developers use the User Interface Markup Language for static UI elements (structures) and their properties (styles). The VoiceXML standard has been developed for speech-based dialogs. The Synchronized Multimedia Integration Language provides a means
for describing the spatial and timed appearance of complex multimedia content combinations. Further work in the development of platform-independent markup languages that support the multimodal interactions expected in future mobile devices is still needed, however.
A
nalysts expect that third-generation mobile phones and beyond will be conduits for enormous amounts of multimedia content. Several major issues along a variety of dimensions must be addressed in the design and provision of content to these mobile devices. Because they are crucial for enhancing the user experience and enabling the effective consumption of mobile content, UIs are therefore among the major issues that need further study and work. ■
S. R. Subramanya is a senior research scientist at LGE Mobile Research. Contact him at
[email protected].
Byung K. Yi is a senior executive vice president at LG Electronics. Contact him at
[email protected].
Editor: Michael Macedonia,
[email protected].
C
omputer Wants You
Computer is always looking for interesting editorial content. In addition to our theme articles, we have other feature sections such as Perspectives, Computing Practices, and Research Features as well as numerous columns to which you can contribute. Check out our author guidelines at
www.computer.org/computer/author.htm for more information about how to contribute to your magazine.
April 2006
87
