people began compiling playlists and burning CD-ROMs for playback at home stereos and in their cars. But the RipâBurnâ
Future Generation Software Architectures in the Automotive Domain – Draft Paper
End to End Content Delivery using UPnP and WiFi Networking Fuchs, A. and Choy, H. SimpleDevices, Inc. 2121 El Camino Real, 11th Floor San Mateo, CA 94403
[email protected],
[email protected]
Abstract. This paper describes a software architecture and method for media distribution from the Internet via the home PC to the car. This method is based on the Universal Plug and Play (UPnP) standard for connected devices but is adapted to the specific environment in automobiles. Media can be delivered from and to the car in various ways using a “connect and carry” or wireless connectivity paradigm. The method described here deals with the issues surrounding power management, disconnect and connect problems and corruption proof database updates.
Introduction Infotainment in vehicles will undergo a big change in the coming years. The FM radio will be replaced by digital high definition radio, digital audio broadcast, and satellite broadcast systems. The second big change is triggered by the advent of digital media management. Today, 60 million people in the U.S. have downloaded digital music and 59% of those Americans who’ve downloaded music are older than 34. 60% of all U.S. households profess an interest in subscribing to a service that allows them to create their own playlists and have their music in a format suitable to a mobile lifestyle. 46% of U.S. households will have broadband access by 2007 and nearly 30 million households will have data networks by the end of 2007, mostly based on IEEE 802.11x wireless (Wi-Fi) technologies. Already there are 2 million Wi-Fi networks in use as of Q2/2003 compared with 300K in Q3/2001. This trend is clearly mainstream today. Music delivery through the Internet allows endless customization and is already economically feasible to distribute personalized, non-mainstream content. Because nobody wants to listen to music solely at the PC, people began compiling playlists and burning CD-ROMs for playback at home stereos and in their cars. But the Rip–Burn–Carry process is very inconvenient and time intensive.
Connected Services in Mobile Networks - San Diego, CA, USA, January 10-12, 2004
2
Fuchs, A. and Choy, H.
How can Rip–Burn–Carry be done more conveniently? Delivering media from a PC to various receivers throughout the home via digital wireless networks is a very compelling proposition. Already, more than 600 forum members support Universal Plug And Play (UPnP) as the standard for media serving, rendering, and control. UPnP is based on W3C standards such as TCP/IP, UDP/IP, HTTP, SOAP, and XML. UPnP Media Server and AV Protocol describe how a “control point” can access, transfer, and render media files on a networked consumer electronics device. This topology allows a user to maintain a media library on a home PC or entertainment center, and distribute the media to any device in the home and car. Furthermore, UPnP eliminates the need to mix and burn CDs, thereby saving time and improving ease of use.
The Automotive Challenge – Media Delivery to the Car How can the car be connected to this paradigm? This paper describes ways of connecting cars to digital content and lists the requirements for building a successful system. The first requirement is connecting cars to media in the home or Internet. Using cellular data channels would not fulfill the performance and cost requirements. In comparison, IEEE 802.11x based wireless LAN technology provides compelling broadband performance at zero marginal cost per bit delivered. In addition, cars spend a lot of time at home, parked in front of the house. This time can be leveraged to update media in the car. This leads to the second requirement. Cars are “sometimes” connected devices and cannot stream content from the Internet when on the road. They can only be connected at certain places at certain times. Therefore, media must be cached on a hard drive or on flash memory for playback. If the memory is removable, it can be carried between the car and a media server (e.g., home PC). Alternatively, a wireless LAN can connect the car to the home network while the car is parked at the house. Finally, the car can update its media whenever it finds a wireless access point hotspot. The biggest availability of WLAN systems today is at the home. However, the worldwide fee-based hotspot market will grow significantly from 28,000 access locations in 2003 to 145,417 access locations worldwide by 2007. Figure 1 highlights the three different methods to deliver media to the car:
End to End Content Delivery using UPnP and WiFi Networking
3
WLAN hub
Connected PC Connect & Carry
Hot Spot
WLAN hub
Fig. 1. Content Delivery to Cars
Wireless LANs are changing the way information is exchanged between vehicles and the connected infrastructure in homes and beyond. Zero margin communication networks allow bringing of content to the car, updating navigation mapping and pointof-interest data, and will enable consumers to configure their cars from their living room, PC, or anywhere in the world on the Internet. The same mechanism also enables car manufacturers to collect vehicle data (e.g., diagnostics) while a car is operating and upload the data while the car is parked in the garage. In the following sections, we will point out how the UPnP framework enables all three mechanisms of content delivery. The in-car software embodies a UPnP Control Point and a Media Renderer. Application programming interfaces enable car makers to develop custom graphical user interfaces (GUI) and features, and provide an abstraction to vehicle controls and buttons to browse and play media in the car. Specific Media Synchronization features deal with power management issues and the nature of a non-persistent connection.
System Architecture Figure 2 illustrates the system architecture that is needed for end-to-end media delivery. The content delivery backend allows the user to download or stream content from the Internet through the home gateway to the car device. The user can manage the media information (albums, playlists, etc.) in a database on the home gateway. The media can either be owned by the user in form of CDs or delivered from the Internet. The home gateway hosts the Media Server and the Control Point for manipulating the delivery process. Typically, the home gateway is a Personal Computer or a broadband gateway with integrated wireless connectivity. Alternatively, the Media Server function can also be integrated into a wireless LAN
4
Fuchs, A. and Choy, H.
hub device. The car device includes a head unit for the user interface, a media player and communication unit, and a removable storage unit. Because the car is not connected all the time, the storage device can cache the media files and contains a copy of a subset of the media information database. Depending on the memory requirements the media storage can either be flash-memory or a hard disk.
Content Delivery Back-End
Home Gateway
Car Device
Fig. 2. Media Delivery System Architecture
UPnP Software Framework Universal Plug and Play (UPnP) is a software architecture and a standard for pervasive and ad-hoc peer-to-peer network connectivity of intelligent appliances, wireless devices, and PCs of all form factors. UPnP extends the plug and play PC peripheral model to a network of distributed devices from a wide range of vendors with zero or marginal configuration needed. A device can dynamically join a network, obtain an IP address, convey its capabilities, and learn about the presence and capabilities of other devices. Furthermore, a UPnP enabled device can control remote devices, and transfer data to and from remote devices. In addition, devices can subscribe to events that may occur as state variables change. This feature allows the distributed system to react on remote events. Finally, a device can leave a network smoothly and automatically without leaving any unwanted state behind. The UPnP Forum is an industry initiative designed to enable easy and robust connectivity among stand-alone devices and PCs from many different vendors. Currently, about 636 companies have joined the UPnP Forum. The UPnP Forum seeks to develop standards for describing device protocols and XML-based device schemas for the purpose of enabling device-to-device interoperability in a scalable networked environment. UPnP leverages Internet technologies, including Internet Protocol (IP), TCP, UDP, HTTP, and XML. Like the Internet, contracts are based on wire protocols that are declarative, expressed in XML, and communicated via HTTP. IP internetworking is a
End to End Content Delivery using UPnP and WiFi Networking
5
strong choice for UPnP because of its proven ability to span different physical media, to enable real world multiple-vendor interoperation, and to achieve synergy with the Internet and many home and office intranets. UPnP has been explicitly designed to accommodate these environments. What is "universal" about UPnP? No device drivers are needed - common protocols are used instead. UPnP networking is communication system and media independent. In the mobile environment, IP based protocols can be used over a variety of wireless networks and are independent of the specific underlying physical carrier. UPnP devices can be implemented using any programming language, and on any operating system. UPnP does not specify or constrain the design of an API for applications running on control points; OS vendors may create APIs that suit their customer's needs. UPnP enables vendor control over device UI and interaction using the browser as well as conventional application programmatic control. Leveraging UPnP Discovery Protocol services, a media player finds media servers in the home or at other access points. The car media player uses UPnP Content Directory services with synchronization extensions to transfer rich media files from the media server. Multiple media formats can be supported, such as audio, video, and photographs. The PC requires an UPnP media management application that enables users to add and remove media from a media database. A PC UPnP media server then distributes this media to consumer devices in the car and/or across the home. Associated PC software can add value to the media database by: • • • • • • •
Seamlessly adding personalized Internet media Seamlessly adding other media owned by the user Metadata editing Metadata cleaning Content recording/ripping Managing of UPnP devices Other UPnP services
In the following section, components of the UPnP framework are described in more detail. Media Server, Media Renderer, Control Point UPnP framework defines the following software components be implemented on various devices in a distributed fashion. • • •
Media Server Control Point Media Renderer
6
Fuchs, A. and Choy, H.
Media Server functionality provides access to media data. Control Point allows a user to discover and control other devices and the data flow between devices. Media Renderer function allows playback of a variety of rich media formats on a device. Devices with Media Server functionality are used in conjunction with one or more Media Renderer device(s). In a typical UPnP session, a Control Point uses SSDP (UPnP discovery service) to discover audio/video (AV) entertainment content (e.g. video, music, images, etc) on one or more Media Servers. The Control Point also uses the same SSDP service to discover Media Renderers on a home network. Then, a user activates Control Point features (using source unit buttons or an IR remote control) to browse or search within a Media Server to locate a desired piece of content (e.g. movie, song, playlist, photo album, etc.). Control Point then prepares to render that content on a device with an appropriate Media Renderer. After the desired content has been identified, the Control Point determines an appropriate transfer protocol and data format to transfer the content from the Media Server to the Media Renderer. (Transfer protocol examples include IEEE-1394, HTTP GET, RTSP/RTP, etc. Data format examples include MPEG-2, MPEG-4, MP3, WMA, JPEG, etc.) After these transfer parameters have been established, Control Point controls the flow of the content (e.g. Play, Pause, Stop, Seek, etc.). The actual transfer of content is performed directly by the Media Server and Media Renderer. The content transfer occurs independently from the Control Point, and does not involve UPnP specifications. In fact, UPnP specifications indicate that transfer protocols are not within the domain of UPnP. Control Point uses UPnP to set up transfer of the content, but the transfer is performed using a transfer protocol other than UPnP. Content Directory Service Control Points use a Media Server’s Content Directory service to locate desired content. The Content Directory service exposes both a search capability and a browse capability. Searching is useful when the Control Point (via the end-user) knows something about the content it wants to find (e.g. name, artist, type, date created, etc). Browsing is useful for blindly discovering content that a device has to offer. Each content item that is referenced by the Content Directory service includes information about that content, including transfer protocol(s) and file format(s) that the Media Server can use to transfer the content to a Media Renderer.
End to End Content Delivery using UPnP and WiFi Networking
7
Connection Manager Service The Connection Manager service allows a Control Point to establish a connection between a Media Server and Media Renderer, to monitor all connections, and to terminate a connection. AV Transport Service AV Transport service enables control over the transport of audio and video streams. This service defines a common model for A/V transport control suitable for a generic user interface. It can be used to control a wide variety of disc, tape, and solid-state based media devices such as CD players, VCRs and MP3 players. It allows a Control Point to issue audio/video functions such as play, stop, shuffle, etc.
Automotive Extensions In addition to the standard UPnP features mentioned above, varying degrees of connectivity to cars and limited power require additional properties. They include: -
Automated Content Synchronization that leverages a wireless LAN link or portable storage media Content Prioritization System that updates high priority, short shelf life content first Interactive feedback mechanisms Shortcuts and enhanced content navigation mechanisms that allow safe, convenient access to media and metadata in vehicle environments Secure media transfer and digital rights management (DRM) Secure firmware upgrade through wireless LAN
Scheduled, Automated Content Synchronization Because automobile-based devices are “sometimes” connected with limited power environments, special consideration must be given to synchronization of content to/from an automobile. For example, a user may suddenly drive away from the home while an automobile-based device is synchronizing. Therefore, the synchronization protocol must be fault tolerant and not corrupt the database when synchronization is abruptly terminated. Likewise, synchronizing content all night can drain the battery enough to strand a driver. Therefore, the synchronization algorithm must factor in battery usage. One sample synchronization algorithm introduced the concept of a “synchronization budget,” e.g., 120 minutes, in which a device continues to synchronize content until the synchronization budget is depleted. When the automobile is started and is operating, the synchronization algorithm begins to add to the synchronization budget.
8
Fuchs, A. and Choy, H.
Synchronization does not occur again until the synchronization budget is greater than zero. Synchronization factors include: - Synchronization between PC and Automobile - Support for limited power UPnP LAN connected devices - Software algorithms to monitor and manage synchronization cycle and process against available battery power - Content Prioritization System that updates most important, short shelf life content first - Synchronization budget (limited maximum synchronization time) - Filtering synchronization engine to limit content on a specific device - Synchronization Interruption Recovery o Database corruption protection to prevent media corruption when synchronization interrupts occur o Multiple, concurrent synchronization sessions - Manual and automatic synchronization with a user defined timing for interaction with home - Synchronization between PC and Content Provider - Content and user/device specific filtering synchronization engine - Periodically and automatic synchronization of updated local and Internet services, media databases, playlists, and media - Automatically download and synchronize preferred daily content and services from providers like Wall Street Journal, CBS, and others One Touch Content Navigation and Media Renderer Controls One touch navigation and media control features are especially important to minimize distractions in an automobile. -
Jump buttons to navigate quickly within the content database, e.g., directly to Artist and Genre playlists Next/previous track
Disc Jockey Disc jockey features enable users to dynamically set up playlists and bookmarks at a device. These features then enable users to quickly navigate to these pre-configured playlists and bookmarks while on the road. The synchronization with the home PC also allows the OEM to build in additional interactive tagging functions. This way the user can obtain more information about the media at a later time at the home PC.
End to End Content Delivery using UPnP and WiFi Networking
9
Digital Rights Management With the advent and popularity of Internet music stores in 2003 and beyond, playback of protected (encrypted) media in automobile devices will be an important factor in the growth of the automobile device market. Digital rights management (DRM), which is embodied in software on Media Servers and Media Renderers, provides a secure mechanism to transfer media from the Internet, to the PC, and then to devices in the automobile. Many DRM mechanisms in use today are secure enough to be approved by major record labels and movie studios because they employ strong encryption technology, such as Public Key Infrastructure (PKI) – which is the same technology used by banks and governments to protect proprietary information. In an example of how DRM can be deployed in an automobile device, assume the device is assigned a unique serial number. When the device is initialized, the device uses its serial number and other proprietary algorithms to generate a pair of PKI public and private keys. The private key is hidden by the device, but the public key is shared with Media Servers, which use the device’s public key to encrypt media for the automobile device. Thereafter, only that automobile device can use its private key to decrypt the content. Other devices are not able to decrypt and play that content because they do not have the proper private key. Firmware Upgrade The capability to upgrade firmware in an automobile device is an important consideration. Today, the landscape for new DRM mechanisms and media formats is in flux. Firmware upgrades enable an OEM to update the firmware in a device to accommodate new DRM mechanisms, new codecs, etc. An OEM can also easily extend UPnP with a Firmware Upgrade service to upgrade firmware in automobile devices wirelessly using Wi-Fi.
Software Architecture The following figure illustrates one approach to enabling an automobile device with UPnP. The UPnP software is generally independent from a specific hardware and operating system but uses communication services like TCP/IP, UDP/IP, HTTP, etc. and ties into the media rendering and decoding framework. Depending on the specific application, multiple media formats may need to be provided. In addition, the UPnP framework interacts with the Graphical User Interface (GUI) and provides Application Programming Interfaces (APIs) for the automotive media application software.
10
Fuchs, A. and Choy, H.
SimpleDevices, Inc. provides an UPnP software development kit (SDK) which can easily be embedded into an existing automobile device using the following steps: (1) OEM adds GUI code, which calls GUI abstraction application programming interfaces (API) to support browsing and playback of a local media server, then integrates the new GUI dialog boxes with the existing GUI code. (2) OEM adds user input code, which calls user input abstraction APIs to handle browsing and playback functions, e.g., browse left/right/up/down, play, pause, etc. (3) OEM provides code to the Media Renderer plug-in decoder framework to add decoders, e.g., MP3, WMA, JPEG, MPEG-2. Note that an automobile device has an embedded UPnP Media Server, as well as Control Point and Media Renderer. After synchronizing content to an embedded automobile media database, the embedded Control Point browses and plays content from the embedded Media Server database, which is typically stored on a hard disc drive or flash memory. Adding UPnP to an automobile multimedia device should be not be resource intensive and be doable in a short amount of time. SimpleDevices has ported the UPnP stack and the automotive extensions to C and Java environments. User Input Software
GUI Code
LCD Display Software
GUI Abstraction
Input Code Input Abstraction
UPnP Control Point Automobile Digital Media Application Software
UPnP Media Server UPnP Media Renderer Common UPnP Library
Media Database
Media Renderer Plug-In Decoder Framework Drivers
MPEG-1
MPEG-2
MPEG-4
JPEG
WMA
MP3
AAC
RTOS
Hardware
Provided by automobile device OEM
UPnP SDK provided by SimpleDevices
Provided by RTOS vendor
New development
Fig. 3. Software Architecture
End to End Content Delivery using UPnP and WiFi Networking
11
Conclusions The digital media revolution will not only bring better quality HD Radio and Satellite radio to the car, but also allow users to bring their own content that they either have purchased on packaged media or downloaded from the Internet. The concept of caching content on a harddisk or other memory in the car enables the user to not only select between multiple horizontal channels of self compiled content (albums, playlists, etc.) but also to navigate front-to-back in time by skipping, shuffling, or repeating media entries or by jumping to an entire new set of media files. This paper has shown that the new Universal Plug and Play standard can be used to deliver media from the Internet and the home to a device in the car without adding significant complexity to the in-car multimedia device. Thus the consumer can manage and play media in a homogeneous fashion at home, on the person via a portable player, and in the car. Future extensions of the paradigm may include distributing navigation information, recording and reporting diagnostics information, and integrating portable media devices. The feedback mechanism and the tie in with the home PC allows car companies to build stronger and more interactive relationships with their customers.
References Universal Plug and Play Forum, www.upnp.org Jeronimo, M.; Weast, J.: UPnP Design by Example, Intel Press, May 2003 Stevens, R. : TCP/IP Illustrated, Volume 1,2,3, Addison-Wesley, October 2000
