Atomic Write FTL for Robust Flash File System - CiteSeerX

Atomic Write FTL for Robust Flash File System Sunhwa PARK, Ji Hyun YU, Seong Yong OHM Abstract — Flash memory is one of the essential elements for code and data storage in the mobile products. Robustness is very important in the flash memory based file system of the mobile consumer electronic devices, since sudden power failure may happen and it may cause a loss of important user data and a serious trouble which cannot be recovered by user. In this paper, we present a novel technique for a robust flash file system. This method is based on the log block FTL algorithm and guarantees that all the sectors bounded together are written completely or no sectors are written at all for any interruptions, and thus distributed metadata of flash file system can be safely updated, and the integrity of the file system can be safely maintained. Our simulation results show that the FAT file system using our method is maintained safe for sudden power failure events. Index Terms — Flash memory, FTL, Flash File System, Metadata.

I. INTRODUCTION As the size of flash memory element grows rapidly, flash memory based file system is widely adapted in mobile computing devices such as PDA, MP3 player, digital camera, mobile phone, and digital camcorder. Flash memory based file system(shortly, flash file system) for the mobile devices, however, has different requirements with the traditional file systems developed for hard disk systems.. One of the most important requirements of the flash file system is robustness. Users do not treat their mobile devices like their desktop computers, and do not blame themselves for extracting batteries from their mobile phones at any time. Because mobile devices are small and portable, a user may drop it on floor occasionally. Anyhow possibility of sudden power failures in mobile devices is much higher than desktop computer and such the power failures may break file system. The result of broken file system is much critical because it is not easy to fix broken file system of mobile devices. Therefore the research on the robust flash file system is getting more important than before. Flash memory is widely used because of low power consumption, shock resistance, and lightweight property [1]. However flash memory has different characteristics with hard disk. A write operation should be preceded by an erase operation, and the erase operation can be performed only in a different unit much larger than the write unit [2]. Therefore special methods are required to use flash memory as data storage for mobile devices. There are two approaches for this purpose. The first approach is 0-7803-8920-4/05/$20.00 © IEEE

redesigning file system to support the characteristics of the flash memory. There are well known native flash file systems such as JFFS (Journaling Flash File System) [3] and YAFFS (Yet Another Flash Filing System) [4] in Linux domain. These file systems are both based on LFS (Log-Structured File system) [5] which appends logs circularly over the whole storage. Because LFS does not overwrite old sectors and uses garbage collection mechanism instead, LFS can cover special characteristics of flash memory. Moreover, logging structure has a big advantage for crash recovering. However, in spite of the advantages, the log-structured native flash file systems are not commonly used in mobile devices. It is because of two major drawbacks; lack of compatibility, and requiring much resource. Currently most mobile devices use the FAT file system [6] for their storage, and Microsoft Windows Operating System cannot recognize JFFS or YAFFS. Users may want to read or write files of the mobile devices in their desktop computer, but it is not easy for JFFS and YAFFS. The other critical drawback is that the logstructured file system requires too much resource and long mounting time. It is not adequate for mobile devices, since they have usually restricted resources and require fast mounting time. The other approach to use flash memory as data storage of mobile devices is using FTL (flash translation layer) [7]. FTL is a kind of software layer which emulates a hard disk with flash memory. Because FTL provides the same functionalities of hard disk system through its unique mapping algorithm, most existing hard disk based file systems can be implemented with flash memory. FTL has been widely used in commercial products. Major removable memory cards and USB mass storage devices use FTL. The FAT file system is one of the most widely used file systems on FTL. The FAT file system has long history; it is well known, simple, and fast mountable. However, the robustness of the FAT file system is not sufficient for mobile devices. To enhance the robustness, TFS4 [8] and RFS [9] used a journaling technique. It writes a backup of important metadata and operation log before processing operation, and the log is cleared when the operation is successfully completed. These methods certainly enhance the robustness of the FAT file system, but the cost for additional codes and the capacity for logs may not be acceptable for some applications and may cause serious performance degradation. In this paper, we propose a new method called AtomicWriteFTL which can provide perfect robustness of file system on FTL. The method can be applied on the FAT

file system, and does not require too much resource. It is notable that this method is for crash avoidance, not for crash recovery. Therefore, time-consuming crash recovery process is not necessary. Moreover, it is novel in sense that it guarantees that all the sectors bounded together are written completely or no sectors are written at all. We modified the log block FTL algorithm [10] so as to add an atomic write function which enables the file system to update distributed file system metadata safely. File system crash comes from an inconsistency of file system metadata. File system metadata is internal data to manage user data effectively. I-node of EXT2 file system and FAT table of the FAT file system are examples of the file system metadata. The metadata is usually distributed over the storage, and should be consistent. If it is inconsistent, the file system may works abnormally and user data may be lost. In other words, to avoid file system crash, metadata’s consistency must be maintained. However, it is not easy to maintain the consistency when distributed metadata is updated by multiple-sector writes. Sudden power failures during the writes may cause inconsistency of metadata, loss of user data, and more critical faults of file system. Fig. 1 shows how the metadata is updated in the previous method and what the problem is. In this case, it is assumed that two sectors are being updated; one sector is for FAT table and the other is for directory entry. The updating process is composed of two separated sector writes. During the process, we cannot avoid the unstable state such that the one sector is written but not the other. In fact, the situation is worse in case of hard disk file system because hard disk does not guarantee atomicity of only one sector writing. If sudden power failure occurs during sector writing, the contents of sector being written can not be predicted. Generally, FTL guarantees the atomicity of one sector writing, but it is not enough to maintain consistency of distributed file system metadata. FAT Table

Dir Entry

Data

Dir Entry

Data

Stable

1. Write FAT Table

Power Failure Unstable

2. Write FAT Table

Dir Entry

Data

Stable

Fig. 1. Previous metadata updating process

Fig. 2 shows the operation of the proposed method. Two sectors can be written atomically with the AtomicWriteFTL, and unstable state never occurs during the process. AtomicWriteFTL uses the log block FTL algorithm so that a new sector always written to a log block, not to data blocks directly. With this mechanism, rollback of bounded

multiple sectors is possible. We added transaction number to bind multiple sectors in the log block.

FAT Table

Dir Entry

Data

Stable

1. Atomic Write of 2 sectors FAT Table

Dir Entry

Data

Stable

Fig. 2. Metadata updating process by atomic write feature

To evaluate AtomicWriteFTL, we modified the FAT file system source code and used the atomic write function for the metadata updating process. And we used a software flash memory emulator to verify AtomicWriteFTL operation for sudden power failures. With the emulator, we simulated sudden power failures during the file system operation, and checked whether the file system is broken or not. Our experiments show that our method can protect the file system perfectly for any sudden power failures. The organization of the remainder of the paper is as follows. The next section describes related researches, and the background knowledge for our research is given in the following section. The next section describes detailed design of AtomicWriteFTL. Experimental method and result are described in next section. Final section is for the conclusion and the future researches. II. RELATED WORK To use flash memory like a non-volatile RAM, M. Wu and W. Zwaenepoel proposed eNVy architecture [11]. In this research, they used hardware MMU, cleaning processor and battery backuped SRAM, and also proposed several garbage collection policies. A. Kwaguchi et al.[12] tried to use flash memory as file system storage. They tried to find the method to use hard disk file system on flash memory. This paper seems to be the first paper about FTL concept while M systems already used the word, FTL in industrial area. There are few papers on FTL algorithm. Instead, FTL algorithms are opened usually by patents. Tae-Sun Chung et al[13] proposed their own FTL algorithm and summarized the patents about FTL in their paper. Jesung Kim et al [10] proposed the log block FTL algorithm to maximize performance with minimal memory translation table. For native flash file system, JFFS [3] and YAFFS [4] were developed. Both file systems use log-structured file system architecture [5]. Meanwhile, for the robust hard disk file system, C. Chao et al. proposed Mime system [14]. The system has RAID disks and high-level recovery functionalities. Not only atomic write function, but also transaction functions are

provided like DBMS. Logical Disk[15] concept is proposed by W. de Jonge et al. It is very similar to concept of FTL. Like FTL remaps logical requests to physical requests on flash memory, Logical Disk remaps random logical requests to sequential and physical requests to get similar effects with LFS for hard disk system. Recently, there are some commercial FAT file systems for flash memory. Both TFS4[8] and RFS[9] use journal feature to enhance the robustness of the FAT file system. TFAT of Microsoft WinCE.NET uses different method to increase the robustness of the FAT file system [17]. It ensures that the file allocation table remains intact by replacing FAT0 with FAT1 atomically in case that power loss occurs. However, this approach has some drawbacks; it degrades file system performance and it can not protect directory entries. III. BACKGROUND This section covers basic information to understand design of AtomicWriteFTL. A. Flash Memory Flash memory is a type of EEPROM which can be updated by software. The stored content in Flash memory is non-volatile. Therefore it can be used as code or data storage for mobile consumer devices that require small size and low power consumption. Flash memory has three basic operations; read, write, and erase operation. Read and write operations are processed in page unit, and the size of a page is fixed from 1 byte to 2KBytes depending on the product. Erase operation is processed in block unit which consists of multiple pages. The size of a block is generally somewhere between 4KBytes and 128KBytes. For NORtype flash memory, the page size is typically 1 byte, meaning that each byte can be read and written individually. NAND-type flash memory, on the other hand, is optimized more for mass storage, and the page size is typically 512 / 2048 bytes. B. Log Block FTL Algorithm In the log block FTL algorithm, data sectors can exist in two types of blocks; the log block and the data block. Major difference of two block types is the page allocation policy. The log block uses the out-of-place policy, the data block uses the in-place policy. Fig. 3 compares two page allocation policies. Shadowed boxes represent the pages which are written and number of boxes means the logical sector number of the containing sector. In the in-place policy (Fig. 3 (a)), the sector position is fixed. Therefore additional mapping information is not necessary, but the updating process is not easy because flash memory can not be updated without block erasing. The out-of-place policy is better for the sector updating process, but additional mapping information is necessary to indicate where the logical sector is written.

Fig. 3. Page allocation policy

The log block FTL algorithm uses small number of log blocks as cache of large number of data blocks because the out-of-place policy is efficient for writing process but the in-place policy does not requires page mapping table. All writes are done to the log block and the full-log block is merged with the old data block to a new data block. Fig. 4 shows write operation of the log block FTL algorithm. Initially, there is no log block mapped to data blocks like Fig.4 (a). In the Fig.4 (b), a log block is allocated for the data block to write sector 6, 6, 5. Another sector write makes the log block full, then the log block and the old data block are merged to a new data block like Fig.4 (c). And the old log block and the old data block are changed to free blocks. Besides log and data blocks, the log block FTL algorithm uses the map block to maintain logical to physical block mapping information. IV. ATOMICWRITEFTL This section describes the design of AtomicWriteFTL. A. Atomic Write of multiple sectors Atomic write of multiple sectors means that all sectors are written or no sectors are written at all for any interruptions. For hard disk, it is impossible to write even only one sector atomically because old sector is updated by new sector normally. However in FTL, it is possible and not requires much additional cost to write sectors atomically because FTL uses mapping algorithm by nature. Key technique for atomic writes of multiple sectors is a rollback mechanism. For the rollback, old sectors must be preserved until all new sectors written completely. For this reason, we decide to use the log block FTL algorithm which always writes new sector in log block, and remains old sector in data block. By invalidating new sector in log block, old sector in data block can be recovered easily. Fig. 5 compares sector updating method. Fig. 5 (a) overwrites old sectors, but (b) does not overwrite old sectors.

merge operation occurred with the old data block and the full log block to make a new data block. In the merge operation, old sectors in the data block are replaced with new sectors in the log block, and it is an obstacle for atomic writes. Not committed sectors in the full log block should not replace original sectors in the data block for the possible rollback process. For this reason, we introduced a new process, reclaiming log block process. Merge process generates one new data block and two free blocks with one full log block, one old data block, and a free block. In contrast to merge process, reclaiming process generates one new data block, one new log block and one free block with same inputs. Fig. 7 describes the differences between merge and reclaim operations.

Fig. 4. Log block FTL algorithm operation

B. Transaction Number By log block FTL algorithm, we can avoid overwriting of old sectors. But we still need the mechanism to invalidate or confirm multiple new sectors in log block. At the first mounting process of FTL after sudden power failure we should recognize the sectors should be rollbacked. For the purpose, we introduced Transaction Number. Transaction number represents the validity of all sectors which have the number. For start request of atomic writes by AtomicWriteStart(), a new transaction number is generated. Generated transaction number is written in spare area of new data page in log block. After multiple sector writing, AtomicWriteCommit() is called to commit new sectors. In the function, the generated transaction number is marked as valid in map block area. Previously, map block only contains block mapping table. We modified it to also contain validity information of certain transaction number. Multiple sectors which are in log block and have same transaction number can be atomically validated by one page writing. Fig. 6 visualizes this mechanism. C. Reclaiming log block When log block is full in the log block FTL algorithm,

Fig. 5. Sector updating method comparison

1. New Transaction Number (5) is generated 2. Write multiple sectors with the transaction number 3. Valid mark for transaction number 5

5

5

5

5 Log Block

Data Block

Fig. 6. Sector writing with transaction number

Map block

Full Log Block

Free Block

Old Data Block

Co py Copy

Free Block

Free Block New Data Block

(a) Merge operation Copy

Full Log Block

New Log Block

Old Data Block

Co py Copy

Free Block

Free Block New Data Block

(b) Reclaim operation Fig. 7. Merge and reclaim operation

V. EXPERIMENT We implemented our system and performed some experiments for verification. Our experimental results show that our AtomicWriteFTL increases the robustness of the flash file system. A. FAT file system over AtomicWriteFTL We modified the FAT file system source code to use atomic write function. Metadata of the FAT file system is FAT table and directory entries. We modified to call AtomicWriteStart() before updating metadata and to call AtomicWriteCommit() after updating metadata. The modification is simple and easy if a file system updates metadata synchronously without buffer cache. Otherwise, metadata updating with atomic write function may be more complicated, however for mobile flash file systems, metadata is synchronously updated normally for robustness. For exceptional file systems which still want to update metadata asynchronously, Soft updates [16] can be a good solution. B. Software flash memory emulator To test robustness of file system, we had to simulate sudden power failures. For the purpose, we used software flash memory emulator which has same functionalities of flash memory and additional functionality of power-off simulation. Flash memory has three types of operations. Read, write, and erase operation. Power failure during read operation does not influence file system consistency, but power failure during write or erase operation may influence file system consistency. Therefore, the emulator has two types of power-off simulation; page write error, and block erase error on sudden power failures. Page writing is process of changing some bits in the page from 1 to 0. For power error simulation of page writing, randomly chosen

bits are remained in 1 value. Data of the page is not predictable, and may influence next mounting process. Block erasing is process of changing all bits in the block from 0 to 1. For power error simulation of block erasing, randomly chosen bits are remained in 0 value. For triggering power-off of the emulator, we used countdown variable and power-state variable inside the emulator. Power-state variable represent the state of flash memory. If the state is off-state, any writing or erasing requests are ignored. Countdown variable is used to turn off flash memory emulator. If the variable is non-zero and power-state variable is on-state, then every request of page writing and block erasing decreases countdown variable by 1. And when the variable becomes zero, power-off is triggered and requested page or block is affected by poweroff, and the power-state variable is changed to off-state. C. Experimental Results We tested a set of file system operations including file coping, file deletion, directory creation, and directory deletion. We turned off the power of flash memory emulator randomly during the operations. Then, checked file system with chkdisk utility, and verified the contents of every file with original. In the verification process, the file size may be smaller than original file because file writing job may be interrupted by power loss, but existing contents must be identical with original contents. We tested for 10,000 times power failures. On PentiumIV 2G desktop computer, the test took about 28 hours. By the experiment, we can convince that proposed AtomicWriteFTL can provide very high-level robustness of mobile flash file system. VI. CONCLUSION In this paper, we have proposed a novel approach called AtomicWriteFTL. It can be a fundamental solution for the robust flash file system in sense that it keeps the file system safe from the inconsistency. However, our solution is available only for nonremovable media such as NAND and NOR flash memory, because both the file system and FTL should be modified. To apply our solution for removable cards such as memory stick, and MMC card, we have to modify protocols between hast and removable cards for AtomicWriteStart(), AtomicWriteCommit(). Until now, FTL designers have tried to provide the most similar functionalities with hard disk file system. However AtomicWriteFTL includes an atomic write function not supported in hard disk file system. We believe there is no reason that FTL cannot overcome hard disk characteristics, and AtomicWriteFTL is a good example for our belief. In this paper, we focused on verifying the robustness of AtomicWriteFTL. In the near future, we will compare the

performance of AtomicWriteFTL on real target system with other robust flash file systems. REFERENCES [1]

[2] [3] [4] [5]

[6] [7] [8] [9]

[10]

[11]

[12]

[13]

F. Douglis, R. Cáceres, F. Kaashoek, K. Li, B. Marsh, and J. A. Tauber, “Storage alternatives for mobile computers,” in Proceedings of the 1st Symposium on Operating Systems Design and Implementation(OSDI), 1994, pp. 25-37. Samsung Electronics, “16M x 8 Bit NAND Flash Memory,” http://www.samsung.com/.. D. Woodhouse, “JFFS: The Journaling Flash File System,” in Ottawa Linux Symposium, 2001. Aleph One Company, “Yet Another Flash Filing System,” http://www.aleph1.co.uk/yaffs. M. Rosenblum, and J. K. Ousterhout, “The Design and Implementation of a Log-Structured File System,” ACM Transactions on Computer Systems, Vol. 10, No. 1, pp. 26-51. Microsoft, “FAT: General Overview of On-Disk Format,” Ver.1.03, 2000. Intel Corporation, “Understanding the flash translation layer (FTL) specification,” http://developer.intel.com/. Samsung Electronics, “TFS4 Samsung’s integrated flash software solution for NAND flash memory,” http://www.samsung.com/. Samsung Electronics, “RFS Samsung’s flash software solution optimized to use NAND flash memory on Linux based CE device,” http://www.samsung.com/. Jesung Kim, JongMin Kim, Sam H. Noh, Sang Lyul Min, Yookun Cho, “A Space-Efficient Flash Translation Layer for Copactflash System,” in IEEE Transactions on Consumer Electronics, 2002, pp.366-365. M. Wu and W. Zwaenepoel, “eNVy: A Non-Volatile Main Memory Storage System,” in Proceedings of the 6th International Conference of Architectural Support for Programming Languages and Operating Systems, 1994, pp.86-97. A. Kawaguchi, S. Nishioka, and H. Motoda, “Flash-Memory Based File System,” in Proceedings of ’95 Winter USENIX Technical Conference, 1995, pp. 155-164. Tae-Sun Chung, Stein Park, Myung-Jin Jung, Bumsoo Kim, “STAFF: State Transition Applied Fast Flash Translation Layer,” in Proceedings of the Conference on Architecture of Computing Systems, 2004, pp. 199-212.

[14] C. Chao, R. English, D. Jacobson, A. Stepanov, J. Wilkes, “Mime: a high performance parallel storage device with strong recovery guarantees,” HPL-CSP-92-9 rev1, Heulett-Packed Laboratories, CA, Nov. 1992. [15] W. De Jonge, M. Frans Kaashoek, W. C. Hsieh, “The Logical Disk: A New Approach to Improving File Systems,” in Proceedings of the 14th Symposium on Operating Systems Principles, 1993, pp. 15-28. [16] G. R. Ganger, M. K. McKusick, C. A. N. Soules, Y. N. Patt, “Soft Updates: A Solution to the Metadata Update Problems in File Systems,” in ACM Transactions on Computer Systems, Vol. 18, No. 2. May 2000. [17] Microsoft, “Transaction-Safe FAT File System,” http:///msdn.microsoft.com

Sunhwa Park received BS and MS degrees in computer science, both from Seoul Women’s University, Korea, in 1998 and 2000. She is PhD candidate of Seoul Women’s University. Her research interests include flash memory software, hardware software co-design, and general embedded systems Jihyun Yoo received BS and MS degrees in computer science, both from Seoul Women’s University, Korea, in 1998 and 2000. She is PhD candidate of Seoul Women’s University. Her research interests include hardware software codesign, flash memory software and algorithms

Seong-Yong OHM received the B.E., M.S., and Ph.D. degrees in Computer Engineering from Seoul National University, Seoul, Korea in 1985, 1987, and 1992 respectively. Since March 1996, he has been working with Department of Computer Science and Engineering at Seoul Women’s University, Seoul, Korea as Professor. His main interests include high-level synthesis, hardware software co-design, computer graphics, and embedded systems.