A File is Not a File: Understanding the I/O Behavior of ... - ACM SIGOPS

A File is Not a File: Understanding the I/O Behavior of Apple Desktop Applications Tyler Harter, Chris Dragga, Michael Vaughn, Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau Department of Computer Sciences University of Wisconsin, Madison {harter,

dragga, vaughn, dusseau, remzi}@cs.wisc.edu

ABSTRACT

user interacts. I/O behavior is especially important to understand since it greatly impacts how users perceive overall system latency and application performance [12]. While a study of how users typically exercise these applications would be interesting, the first step is to perform a detailed study of I/O behavior under typical but controlled workload tasks. This style of application study, common in the field of computer architecture [40], is different from the workload study found in systems research, and can yield deeper insight into how the applications are constructed and how file and storage systems need to be designed in response. Home-user applications are fundamentally large and complex, containing millions of lines of code [20]. In contrast, traditional U NIX-based applications are designed to be simple, to perform one task well, and to be strung together to perform more complex tasks [32]. This modular approach of U NIX applications has not prevailed [17]: modern applications are standalone monoliths, providing a rich and continuously evolving set of features to demanding users. Thus, it is beneficial to study each application individually to ascertain its behavior. In this paper, we present the first in-depth analysis of the I/O behavior of modern home-user applications; we focus on productivity applications (for word processing, spreadsheet manipulation, and presentation creation) and multimedia software (for digital music, movie editing, and photo management). Our analysis centers on two Apple software suites: iWork, consisting of Pages, Numbers, and Keynote; and iLife, which contains iPhoto, iTunes, and iMovie. As Apple’s market share grows [38], these applications form the core of an increasingly popular set of workloads; as device convergence continues, similar forms of these applications are likely to access user files from both stationary machines and moving cellular devices. We call our collection the iBench task suite. To investigate the I/O behavior of the iBench suite, we build an instrumentation framework on top of the powerful DTrace tracing system found inside Mac OS X [8]. DTrace allows us not only to monitor system calls made by each traced application, but also to examine stack traces, in-kernel functions such as page-ins and page-outs, and other details required to ensure accuracy and completeness. We also develop an application harness based on AppleScript [3] to drive each application in the repeatable and automated fashion that is key to any study of GUI-based applications [12]. Our careful study of the tasks in the iBench suite has enabled us to make a number of interesting observations about how applications access and manipulate stored data. In addition to confirming standard past findings (e.g., most files are small; most bytes accessed are from large files [4]), we find the following new results. A file is not a file. Modern applications manage large databases of information organized into complex directory trees. Even simple word-processing documents, which appear to users as a “file”, are

We analyze the I/O behavior of iBench, a new collection of productivity and multimedia application workloads. Our analysis reveals a number of differences between iBench and typical file-system workload studies, including the complex organization of modern files, the lack of pure sequential access, the influence of underlying frameworks on I/O patterns, the widespread use of file synchronization and atomic operations, and the prevalence of threads. Our results have strong ramifications for the design of next generation local and cloud-based storage systems.

1.

INTRODUCTION

The design and implementation of file and storage systems has long been at the forefront of computer systems research. Innovations such as namespace-based locality [21], crash consistency via journaling [15, 29] and copy-on-write [7, 34], checksums and redundancy for reliability [5, 7, 26, 30], scalable on-disk structures [37], distributed file systems [16, 35], and scalable clusterbased storage systems [9, 14, 18] have greatly influenced how data is managed and stored within modern computer systems. Much of this work in file systems over the past three decades has been shaped by measurement: the deep and detailed analysis of workloads [4, 10, 11, 16, 19, 25, 33, 36, 39]. One excellent example is found in work on the Andrew File System [16]; detailed analysis of an early AFS prototype led to the next-generation protocol, including the key innovation of callbacks. Measurement helps us understand the systems of today so we can build improved systems for tomorrow. Whereas most studies of file systems focus on the corporate or academic intranet, most file-system users work in the more mundane environment of the home, accessing data via desktop PCs, laptops, and compact devices such as tablet computers and mobile phones. Despite the large number of previous studies, little is known about home-user applications and their I/O patterns. Home-user applications are important today, and their importance will increase as more users store data not only on local devices but also in the cloud. Users expect to run similar applications across desktops, laptops, and phones; therefore, the behavior of these applications will affect virtually every system with which a

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. SOSP '11, October 23-26, 2011, Cascais, Portugal. Copyright © 2011 ACM 978-1-4503-0977-6/11/10 ... $10.00.

71

cessed (e.g., counts and sizes), the access patterns (e.g., read/write, sequentiality, and preallocation), transactional properties (e.g., durability and atomicity), and threading. Third, we describe how these qualitative changes in I/O behavior may impact the design of future systems. Finally, we present the 34 traces from the iBench task suite; by making these traces publicly available and easy to use, we hope to improve the design, implementation, and evaluation of the next generation of local and cloud storage systems:

in actuality small file systems containing many sub-files (e.g., a Microsoft .doc file is actually a FAT file system containing pieces of the document). File systems should be cognizant of such hidden structure in order to lay out and access data in these complex files more effectively. Sequential access is not sequential. Building on the trend noticed by Vogels for Windows NT [39], we observe that even for streaming media workloads, “pure” sequential access is increasingly rare. Since file formats often include metadata in headers, applications often read and re-read the first portion of a file before streaming through its contents. Prefetching and other optimizations might benefit from a deeper knowledge of these file formats. Auxiliary files dominate. Applications help users create, modify, and organize content, but user files represent a small fraction of the files touched by modern applications. Most files are helper files that applications use to provide a rich graphical experience, support multiple languages, and record history and other metadata. File-system placement strategies might reduce seeks by grouping the hundreds of helper files used by an individual application. Writes are often forced. As the importance of home data increases (e.g., family photos), applications are less willing to simply write data and hope it is eventually flushed to disk. We find that most written data is explicitly forced to disk by the application; for example, iPhoto calls fsync thousands of times in even the simplest of tasks. For file systems and storage, the days of delayed writes [22] may be over; new ideas are needed to support applications that desire durability. Renaming is popular. Home-user applications commonly use atomic operations, in particular rename, to present a consistent view of files to users. For file systems, this may mean that transactional capabilities [23] are needed. It may also necessitate a rethinking of traditional means of file locality; for example, placing a file on disk based on its parent directory [21] does not work as expected when the file is first created in a temporary location and then renamed. Multiple threads perform I/O. Virtually all of the applications we study issue I/O requests from a number of threads; a few applications launch I/Os from hundreds of threads. Part of this usage stems from the GUI-based nature of these applications; it is well known that threads are required to perform long-latency operations in the background to keep the GUI responsive [24]. Thus, file and storage systems should be thread-aware so they can better allocate bandwidth. Frameworks influence I/O. Modern applications are often developed in sophisticated IDEs and leverage powerful libraries, such as Cocoa and Carbon. Whereas UNIX-style applications often directly invoke system calls to read and write files, modern libraries put more code between applications and the underlying file system; for example, including "cocoa.h" in a Mac application imports 112,047 lines of code from 689 different files [28]. Thus, the behavior of the framework, and not just the application, determines I/O patterns. We find that the default behavior of some Cocoa APIs induces extra I/O and possibly unnecessary (and costly) synchronizations to disk. In addition, use of different libraries for similar tasks within an application can lead to inconsistent behavior between those tasks. Future storage design should take these libraries and frameworks into account. This paper contains four major contributions. First, we describe a general tracing framework for creating benchmarks based on interactive tasks that home users may perform (e.g., importing songs, exporting video clips, saving documents). Second, we deconstruct the I/O behavior of the tasks in iBench; we quantify the I/O behavior of each task in numerous ways, including the types of files ac-

http://www.cs.wisc.edu/adsl/Traces/ibench The remainder of this paper is organized as follows. We begin by presenting a detailed timeline of the I/O operations performed by one task in the iBench suite; this motivates the need for a systematic study of home-user applications. We next describe our methodology for creating the iBench task suite. We then spend the majority of the paper quantitatively analyzing the I/O characteristics of the full iBench suite. Finally, we summarize the implications of our findings on file-system design.

2.

CASE STUDY

The I/O characteristics of modern home-user applications are distinct from those of U NIX applications studied in the past. To motivate the need for a new study, we investigate the complex I/O behavior of a single representative task. Specifically, we report in detail the I/O performed over time by the Pages (4.0.3) application, a word processor, running on Mac OS X Snow Leopard (10.6.2) as it creates a blank document, inserts 15 JPEG images each of size 2.5MB, and saves the document as a Microsoft .doc file. Figure 1 shows the I/O this task performs (see the caption for a description of the symbols used). The top portion of the figure illustrates the accesses performed over the full lifetime of the task: at a high level, it shows that more than 385 files spanning six different categories are accessed by eleven different threads, with many intervening calls to fsync and rename. The bottom portion of the figure magnifies a short time interval, showing the reads and writes performed by a single thread accessing the primary .doc productivity file. From this one experiment, we illustrate each finding described in the introduction. We first focus on the single access that saves the user’s document (bottom), and then consider the broader context surrounding this file save, where we observe a flurry of accesses to hundreds of helper files (top). A file is not a file. Focusing on the magnified timeline of reads and writes to the productivity .doc file, we see that the file format comprises more than just a simple file. Microsoft .doc files are based on the FAT file system and allow bundling of multiple files in the single .doc file. This .doc file contains a directory (Root), three streams for large data (WordDocument, Data, and 1Table), and a stream for small data (Ministream). Space is allocated in the file with three sections: a file allocation table (FAT), a double-indirect FAT (DIF) region, and a ministream allocation region (Mini). Sequential access is not sequential. The complex FAT-based file format causes random access patterns in several ways: first, the header is updated at the beginning and end of the magnified access; second, data from individual streams is fragmented throughout the file; and third, the 1Table stream is updated before and after each image is appended to the WordDocument stream. Auxiliary files dominate. Although saving the single .doc we have been considering is the sole purpose of this task, we now turn our attention to the top timeline and see that 385 different files are accessed. There are several reasons for this multitude of files. First, Pages provides a rich graphical experience involving many images and other forms of multimedia; together with the 15 inserted JPEGs, this requires 118 multimedia files. Second, users

72

quit

save

add .jpg

add .jpg

add .jpg

add .jpg

add .jpg

add .jpg

add .jpg

add .jpg

add .jpg

add .jpg

add .jpg

add .jpg

add .jpg

add .jpg

add .jpg

new doc

Threads 1, 10 Thread 11 Thread 3 Thread 1 Threads 2, 4, 6, 8 Thread 1

other 17 (38MB) strings 25 (0.3MB)

Threads 1, 10

Files

multimedia 118 (106MB)

Thread 1

sqlite 2 (9KB)

Thread 4 Threads 1-9 Thread 8 Thread 7 Thread 6 Thread 5 Thread 4 Thread 3 Thread 2

plist 218 (0.8MB) Thread 1

productivity 5 (39MB) 0

5

10

15

20

25

30

35

40

45

50

55

60

65

70

Seconds magnify .doc file save 36386

DIF

36384

FAT Mini Root Ministream Root 1Table Data

Compressed (0.3MB) 36102 36100

36094

x3

x6 x36 x3

x6 x39

x46

36096

x6 x39

36098

1Table

Compressed (23.5MB) 12044

WordDocument

x2

File Offset (KB)

36092

12042 12040

1Table

12038 Compressed (11.7MB) WordDocument

16 x2

14 12

1Table

10 WordDocument

4

x5 x2 x36 x9

x36

6

x7 x2 x36 x9 x28

8

1Table Data

2 0 0

5

10

15

20

25

30

35

40

45

50

55

60

65

70

75

80

85

90

95

WordDocument Header 100

Sequential Runs

Figure 1: Pages Saving A Word Document. The top graph shows the 75-second timeline of the entire run, while the bottom graph is a magnified view of seconds 54 to 58. In the top graph, annotations on the left categorize files by type and indicate file count and amount of I/O; annotations on the right show threads. Black bars are file accesses (reads and writes), with thickness logarithmically proportional to bytes of I/O. / is an fsync; \ is a rename; X is both. In the bottom graph, individual reads and writes to the .doc file are shown. Vertical bar position and bar length represent the offset within the file and number of bytes touched. Thick white bars are reads; thin gray bars are writes. Repeated runs are marked with the number of repetitions. Annotations on the right indicate the name of each file section.

73

3.1

want to use Pages in their native language, so application text is not hard-coded into the executable but is instead stored in 25 different .strings files. Third, to save user preferences and other metadata, Pages uses a SQLite database (2 files) and a number of key-value stores (218 .plist files). Writes are often forced; renaming is popular. Pages uses both of these actions to enforce basic transactional guarantees. It uses fsync to flush write data to disk, making it durable; it uses rename to atomically replace old files with new files so that a file never contains inconsistent data. The timeline shows these invocations numerous times. First, Pages regularly uses fsync and rename when updating the key-value store of a .plist file. Second, fsync is used on the SQLite database. Third, for each of the 15 image insertions, Pages calls fsync on a file named “tempData” (classified as “other”) to update its automatic backup. Multiple threads perform I/O. Pages is a multi-threaded application and issues I/O requests from many different threads during the experiment. Using multiple threads for I/O allows Pages to avoid blocking while I/O requests are outstanding. Examining the I/O behavior across threads, we see that Thread 1 performs the most significant portion of I/O, but ten other threads are also involved. In most cases, a single thread exclusively accesses a file, but it is not uncommon for multiple threads to share a file. Frameworks influence I/O. Pages was developed in a rich programming environment where frameworks such as Cocoa or Carbon are used for I/O; these libraries impact I/O patterns in ways the developer might not expect. For example, although the application developers did not bother to use fsync or rename when saving the user’s work in the .doc file, the Cocoa library regularly uses these calls to atomically and durably update relatively unimportant metadata, such as “recently opened” lists stored in .plist files. As another example, when Pages tries to read data in 512-byte chunks from the .doc, each read goes through the STDIO library, which only reads in 4 KB chunks. Thus, when Pages attempts to read one chunk from the 1Table stream, seven unrequested chunks from the WordDocument stream are also incidentally read (offset 12039 KB). In other cases, regions of the .doc file are repeatedly accessed unnecessarily. For example, around the 3KB offset, read/write pairs occur dozens of times. Pages uses a library to write 2-byte words; each time a word is written, the library reads, updates, and writes back an entire 512-byte chunk. Finally, we see evidence of redundancy between libraries: even though Pages has a backing SQLite database for some of its properties, it also uses .plist files, which function across Apple applications as generic property stores. This one detailed experiment has shed light on a number of interesting I/O behaviors that indicate that home-user applications are indeed different than traditional workloads. A new workload suite is needed that more accurately reflects these applications.

Representative

To capture the I/O behavior of home users, iBench models the actions of a “reasonable” user interacting with iPhoto, iTunes, iMovie, Pages, Numbers, and Keynote. Since the research community does not yet have data on the exact distribution of tasks that home users perform, iBench contains tasks that we believe are common and uses files with sizes that can be justified for a reasonable user. iBench contains 34 different tasks, each representing a home user performing one distinct operation. If desired, these tasks could be combined to create more complex workflows and I/O workloads. The six applications and corresponding tasks are as follows. iLife iPhoto 8.1.1 (419): digital photo album and photo manipulation software. iPhoto stores photos in a library that contains the data for the photos (which can be in a variety of formats, including JPG, TIFF, and PNG), a directory of modified files, a directory of scaled down images, and two files of thumbnail images. The library stores metadata in a SQLite database. iBench contains six tasks exercising user actions typical for iPhoto: starting the application and importing, duplicating, editing, viewing, and deleting photos in the library. These tasks modify both the image files and the underlying database. Each of the iPhoto tasks operates on 400 2.5 MB photos, representing a user who has imported 12 megapixel photos (2.5 MB each) from a full 1 GB flash card on his or her camera. iLife iTunes 9.0.3 (15): a media player capable of both audio and video playback. iTunes organizes its files in a private library and supports most common music formats (e.g., MP3, AIFF, WAVE, AAC, and MPEG-4). iTunes does not employ a database, keeping media metadata and playlists in both a binary and an XML file. iBench contains five tasks for iTunes: starting iTunes, importing and playing an album of MP3 songs, and importing and playing an MPEG-4 movie. Importing requires copying files into the library directory and, for music, analyzing each song file for gapless playback. The music tasks operate over an album (or playlist) of ten songs while the movie tasks use a single 3-minute movie. iLife iMovie 8.0.5 (820): video editing software. iMovie stores its data in a library that contains directories for raw footage and projects, and files containing video footage thumbnails. iMovie supports both MPEG-4 and Quicktime files. iBench contains four tasks for iMovie: starting iMovie, importing an MPEG-4 movie, adding a clip from this movie into a project, and exporting a project to MPEG-4. The tasks all use a 3-minute movie because this is a typical length found from home users on video-sharing websites. iWork Pages 4.0.3 (766): a word processor. Pages uses a ZIPbased file format and can export to DOC, PDF, RTF, and basic text. iBench includes eight tasks for Pages: starting up, creating and saving, opening, and exporting documents with and without images and with different formats. The tasks use 15 page documents. iWork Numbers 2.0.3 (332): a spreadsheet application. Numbers organizes its files with a ZIP-based format and exports to XLS and PDF. The four iBench tasks for Numbers include starting Numbers, generating a spreadsheet and saving it, opening the spreadsheet, and exporting that spreadsheet to XLS. To model a possible home user working on a budget, the tasks utilize a five page spreadsheet with one column graph per sheet. iWork Keynote 5.0.3 (791): a presentation and slideshow application. Keynote saves to a .key ZIP-based format and exports to Microsoft’s PPT format. The seven iBench tasks for Keynote include starting Keynote, creating slides with and without images, opening and playing presentations, and exporting to PPT. Each Keynote task uses a 20-slide presentation.

3. IBENCH TASK SUITE Our goal in constructing the iBench task suite is two-fold. First, we would like iBench to be representative of the tasks performed by home users. For this reason, iBench contains popular applications from the iLife and iWork suites for entertainment and productivity. Second, we would like iBench to be relatively simple for others to use for file and storage system analysis. For this reason, we automate the interactions of a home user and collect the resulting traces of I/O system calls. The traces are available online at this site: http://www.cs.wisc.edu/adsl/Traces/ibench. We now describe in more detail how we met these two goals.

74

iPhoto iTunes Keynote

Numbers

Pages

iMovie

iLife iWork

Name Start Imp Dup Edit Del View Start ImpS ImpM PlayS PlayM Start Imp Add Exp Start New NewP Open PDF PDFP DOC DOCP Start New Open XLS Start New NewP Play PlayP PPT PPTP

Description Open iPhoto with library of 400 photos Import 400 photos into empty library Duplicate 400 photos from library Sequentially edit 400 photos from library Sequentially delete 400 photos; empty trash Sequentially view 400 photos Open iTunes with 10 song album Import 10 song album to library Import 3 minute movie to library Play album of 10 songs Play 3 minute movie Open iMovie with 3 minute clip in project Import 3 minute .m4v (20MB) to “Events” Paste 3 minute clip from “Events” to project Export 3 minute video clip Open Pages Create 15 text page document; save as .pages Create 15 JPG document; save as .pages Open 15 text page document Export 15 page document as .pdf Export 15 JPG document as .pdf Export 15 page document as .doc Export 15 JPG document as .doc Open Numbers Save 5 sheets/column graphs as .numbers Open 5 sheet spreadsheet Export 5 sheets/column graphs as .xls Open Keynote Create 20 text slides; save as .key Create 20 JPG slides; save as .key Open and play presentation of 20 text slides Open and play presentation of 20 JPG slides Export 20 text slides as .ppt Export 20 JPG slides as .ppt

Files (MB) 779 (336.7) 5900 (1966.9) 2928 (1963.9) 12119 (4646.7) 15246 (23.0) 2929 (1006.4) 143 (184.4) 68 (204.9) 41 (67.4) 61 (103.6) 56 (77.9) 433 (223.3) 184 (440.1) 210 (58.3) 70 (157.9) 218 (183.7) 135 (1.6) 408 (112.0) 103 (0.8) 107 (1.5) 404 (77.4) 112 (1.0) 385 (111.3) 283 (179.9) 269 (4.9) 119 (1.3) 236 (4.6) 517 (183.0) 637 (12.1) 654 (92.9) 318 (11.5) 321 (45.4) 685 (12.8) 723 (110.6)

Accesses (MB) 828 (25.4) 8709 (3940.3) 5736 (2076.2) 18927 (12182.9) 15247 (25.0) 3347 (1005.0) 195 (9.3) 139 (264.5) 57 (42.9) 80 (90.9) 69 (32.0) 786 (29.4) 383 (122.3) 547 (2.2) 546 (229.9) 228 (2.3) 157 (1.0) 997 (180.9) 109 (0.6) 115 (0.9) 965 (110.9) 121 (1.0) 952 (183.8) 360 (2.6) 313 (2.8) 137 (1.3) 272 (2.7) 681 (1.1) 863 (5.4) 901 (103.3) 385 (4.9) 388 (55.7) 918 (10.1) 996 (124.6)

RD% 78.8 74.4 52.4 69.8 21.8 98.1 54.7 66.3 48.0 96.9 92.3 99.9 55.6 47.8 55.1 99.9 73.3 60.7 99.5 91.0 67.4 87.9 61.1 99.6 90.7 99.8 94.9 99.8 92.4 66.8 99.8 69.6 78.8 57.6

WR% 21.2 25.6 47.6 30.2 78.2 1.9 45.3 33.7 52.0 3.1 7.7 0.1 44.4 52.2 44.9 0.1 26.7 39.3 0.5 9.0 32.6 12.1 38.9 0.4 9.3 0.2 5.1 0.2 7.6 33.2 0.2 30.4 21.2 42.4

Accesses / CPU Sec 151.1 26.7 237.9 19.6 280.9 24.1 72.4 75.2 152.4 0.4 2.2 134.8 29.3 357.8 2.3 97.7 50.8 54.6 57.6 41.3 49.7 44.4 46.3 115.5 9.6 48.7 8.5 229.8 129.1 70.8 95.0 72.4 115.2 61.0

I/O MB / CPU Sec 4.6 12.1 86.1 12.6 0.5 7.2 3.4 143.1 114.6 0.5 1.0 5.0 9.3 1.4 1.0 1.0 0.3 9.9 0.3 0.3 5.7 0.4 8.9 0.8 0.1 0.5 0.1 0.4 0.8 8.1 1.2 10.4 1.3 7.6

Table 1: 34 Tasks of the iBench Suite. The table summarizes the 34 tasks of iBench, specifying the application, a short name for the task, and a longer description of the actions modeled. The I/O is characterized according to the number of files read or written, the sum of the maximum sizes of all accessed files, the number of file accesses that read or write data, the number of bytes read or written, the percentage of I/O bytes that are part of a read (or write), and the rate of I/O per CPU-second in terms of both file accesses and bytes. Each core is counted individually, so at most 2 CPU-seconds can be counted per second on our dual-core test machine. CPU utilization is measured with the UNIX top utility, which in rare cases produces anomalous CPU utilization snapshots; those values are ignored. instrument the entry and exit points of all system calls dealing with the file system; it also records the current state of the system and the parameters passed to and returned from each call. While tracing with DTrace was generally straightforward, we addressed four challenges in collecting the iBench traces. First, file sizes are not always available to DTrace; thus, we record every file’s initial size and compute subsequent file size changes caused by system calls such as write or ftruncate. Second, iTunes uses the ptrace system call to disable tracing; we circumvent this block by using gdb to insert a breakpoint that automatically returns without calling ptrace. Third, the volfs pseudo-file system in HFS+ (Hierarchical File System) allows files to be opened via their inode number instead of a file name; to include pathnames in the trace, we instrument the build path function to obtain the full path when the task is run. Fourth, tracing system calls misses I/O resulting from memory-mapped files; therefore, we purged memory and instrumented kernel page-in functions to measure the amount of memory-mapped file activity. We found that the amount of memory-mapped I/O is negligible in most tasks; we thus do not include this I/O in the iBench traces or analysis. To provide reproducible results, the traces must be run on a single file-system image. Therefore, the iBench suite also contains snapshots of the initial directories to be restored before each run; initial state is critical in file-system benchmarking [1].

Table 1 contains a brief description of each of the 34 iBench tasks as well as the basic I/O characteristics of each task when running on Mac OS X Snow Leopard 10.6.2. The table illustrates that the iBench tasks perform a significant amount of I/O. Most tasks access hundreds of files, which in aggregate contain tens or hundreds of megabytes of data. The tasks typically access files hundreds of times. The tasks perform widely differing amounts of I/O, from less than a megabyte to more than a gigabyte. Most of the tasks perform many more reads than writes. Finally, the tasks exhibit high I/O throughput, often transferring tens of megabytes of data for every second of computation.

3.2

Easy to Use

To enable other system evaluators to easily use these tasks, the iBench suite is packaged as a set of 34 system call traces. To ensure reproducible results, the 34 user tasks were first automated with AppleScript, a general-purpose GUI scripting language. AppleScript provides generic commands to emulate mouse clicks through menus and application-specific commands to capture higher-level operations. Application-specific commands bypass a small amount of I/O by skipping dialog boxes; however, we use them whenever possible for expediency. The system call traces were gathered using DTrace [8], a kernel and user level dynamic instrumentation tool. DTrace is used to

75

4. ANALYSIS OF IBENCH TASKS

Figure 3 displays the percentage of I/O bytes accessed for each file type. In bytes, multimedia I/O dominates most of the iLife tasks, while productivity I/O has a significant presence in the iWork tasks; file descriptors on multimedia and productivity files tend to receive large amounts of I/O. SQLite, Plist, and Strings files have a smaller share of the total I/O in bytes relative to the number of opened files; this implies that tasks access only a small quantity of data for each of these files opened (e.g., several key-value pairs in a .plist). In most tasks, files classified as “Other” receive a more significant portion of the I/O (the exception is iTunes). Summary: Home applications access a wide variety of file types, generally opening multimedia files the most frequently. iLife tasks tend to access bytes primarily from multimedia or files classified as “Other”; iWork tasks access bytes from a broader range of file types, with some emphasis on productivity files.

The iBench task suite enables us to study the I/O behavior of a large set of home-user actions. As shown from the timeline of I/O behavior for one particular task in Section 2, these tasks are likely to access files in complex ways. To characterize this complex behavior in a quantitative manner across the entire suite of 34 tasks, we focus on answering four categories of questions. • What different types of files are accessed and what are the sizes of these files? • How are files accessed for reads and writes? Are files accessed sequentially? Is space preallocated? • What are the transactional properties? Are writes flushed with fsync or performed atomically? • How do multi-threaded applications distribute I/O across different threads?

4.1.2 File Sizes Large and small files present distinct challenges to the file system. For large files, finding contiguous space can be difficult, while for small files, minimizing initial seek time is more important. We investigate two different questions regarding file size. First, what is the distribution of file sizes accessed by each task? Second, what portion of accessed bytes resides in files of various sizes? To answer these questions, we record file sizes when each unique file descriptor is closed. We categorize sizes as very small (< 4KB), small (< 64KB), medium (< 1MB), large (< 10MB), or very large (≥ 10MB). We track how many accesses are to files in each category and how many of the bytes belong to files in each category. Figure 4 shows the number of accesses to files of each size. Accesses to very small files are extremely common, especially for iWork, accounting for over half of all the accesses in every iWork task. Small file accesses have a significant presence in the iLife tasks. The large quantity of very small and small files is due to frequent use of .plist files that store preferences, settings, and other application data; these files often fill just one or two 4 KB pages. Figure 5 shows the proportion of the files in which the bytes of accessed files reside. Large and very large files dominate every startup workload and nearly every task that processes multimedia files. Small files account for few bytes and very small files are essentially negligible. Summary: Agreeing with many previous studies (e.g., [4]), we find that while applications tend to open many very small files (< 4 KB), most of the bytes accessed are in large files (> 1 MB).

Answering these questions has two benefits. First, the answers can guide file and storage system developers to target their systems better to home-user applications. Second, the characterization will help users of iBench to select the most appropriate traces for evaluation and to understand their resulting behavior. All measurements were performed on a Mac Mini running Mac OS X Snow Leopard version 10.6.2 and the HFS+ file system. The machine has 2 GB of memory and a 2.26 GHz Intel Core Duo processor.

4.1

Nature of Files

Our analysis begins by characterizing the high-level behavior of the iBench tasks. In particular, we study the different types of files opened by each iBench task as well as the sizes of those files.

4.1.1 File Types The iLife and iWork applications store data across a variety of files in a number of different formats; for example, iLife applications tend to store their data in libraries (or data directories) unique to each user, while iWork applications organize their documents in proprietary ZIP-based files. The extent to which tasks access different types of files greatly influences their I/O behavior. To understand accesses to different file types, we place each file into one of six categories, based on file name extensions and usage. Multimedia files contain images (e.g., JPEG), songs (e.g., MP3, AIFF), and movies (e.g., MPEG-4). Productivity files are documents (e.g., .pages, DOC, PDF), spreadsheets (e.g., .numbers, XLS), and presentations (e.g., .key, PPT). SQLite files are database files. Plist files are property-list files in XML containing key-value pairs for user preferences and application properties. Strings files contain strings for localization of application text. Finally, Other contains miscellaneous files such as plain text, logs, files without extensions, and binary files. Figure 2 shows the frequencies with which tasks open and access files of each type; most tasks perform hundreds of these accesses. Multimedia file opens are common in all workloads, though they seldom predominate, even in the multimedia-heavy iLife applications. Conversely, opens of productivity files are rare, even in iWork applications that use them; this is likely because most iWork tasks create or view a single productivity file. Because .plist files act as generic helper files, they are relatively common. SQLite files only have a noticeable presence in iPhoto, where they account for a substantial portion of the observed opens. Strings files occupy a significant minority of most workloads (except iPhoto and iTunes). Finally, between 5% and 20% of files are of type “Other” (except for iTunes, where they are more prevalent).

4.2

Access Patterns

We next examine how the nature of file accesses has changed, studying the read and write patterns of home applications. These patterns include whether files are used for reading, writing, or both; whether files are accessed sequentially or randomly; and finally, whether or not blocks are preallocated via hints to the file system.

4.2.1

File Accesses

One basic characteristic of our workloads is the division between reading and writing on open file descriptors. If an application uses an open file only for reading (or only for writing) or performs more activity on file descriptors of a certain type, then the file system may be able to make more intelligent memory and disk allocations. To determine these characteristics, we classify each opened file descriptor based on the types of accesses–read, write, or both read and write–performed during its lifetime. We also ignore the actual flags used when opening the file since we found they do not accurately reflect behavior; in all workloads, almost all write-only file descriptors were opened with O RDWR. We measure both the

76

Numbers Keynote

iTunes iMovie

Pages

Numbers Keynote

1MB 5MB 56MB 5MB 103MB 10MB 125MB Start Play PlayP New NewP PPT PPTP

3MB 1MB 3MB 3MB Start Open New XLS

Numbers Keynote

iPhoto

iTunes iMovie

Pages

184MB 16MB 118MB 16MB 178MB 16MB 178MB Start Play PlayP New NewP PPT PPTP

>=10MB

180MB 2MB 6MB 6MB

184MB 2MB 2MB 293MB 2MB 187MB 1MB 256MB

0%

Start Open New NewP PDF PDFP DOC DOCP

0%