International Journal of Digital Content Technology and its Applications Volume 4, Number 9, December 2010
The Hash Postfix Table Based Metadata Management Algorithm for Mass Storage System 1
Xiujuan LI，2Tao CAI，3Shiguang JU School of Computer Science and Telecommunication Engineering, Jiangsu University, Jiangsu, Zhenjiang 212013, China 1 ,First Author
[email protected] 2 ,corresponding
[email protected] 3 ,
[email protected] doi:10.4156/jdcta.vol4. issue9.12
Abstract It is important for mass storage system to distribute access requests dynamically and balanced between several metadata servers (MDS). Based on analyzing the features of metadata management in the mass storage system, this paper designs the structure of metadata management module, proposes the hash postfix table(HPT) based metadata management algorithm, uses HPT to adjust the distribution of access request dynamically, presents the process of metadata querying and HPT quick adjustment. It analyzes the algorithm from the dynamic equilibrium capability and the time and space overhead. At last, it realizes the prototype, using real data sets to evaluating. The results show that the HPT based metadata management algorithm is more effective and flexible, and can avoid the hotspots.
Keywords: Metadata Management, Hash Postfix Table, Mass Storage System 1. Introduction With the explosive growth of data, mass storage system has become a hot subject of researching and development. Data storage and management of mass storage system have become two important issues of metadata management. Managing and searching metadata are principal means in mass storage system, so how to manage metadata flexibly and efficiently is an important issue in mass storage system. Related work. The number of metadata is smaller than the number of data stored in mass storage systems, on the contrary 50% operations are metadata operations only[1]. Therefore, efficient and flexible metadata management algorithm is an important method to improve the performance of mass storage system. Directory sub-tree partition algorithm and the hash-based algorithm are two common metadata management algorithms of mass storage system. MIT Popek et al [2-5] proposed the directory sub-tree partition algorithm, which divided the directory tree into several sub-trees in accordance with the level of directories and distributed across multiple metadata servers (MDS). It reduces the time and space overhead of metadata management, but it still requires large time and space overhead in metadata searching, directory or filename renaming, and access authorization modification. IBM's P.F.Corbett et al [6] proposed a hash-based metadata management algorithm, using the hash function to distribute the metadata into a number of MDS, which has the advantage of less time and space overhead. But it still requires a lot of time and space overhead between directory listing and making changes to access authorization, due to destroying the directory relationship. In 2007 Su Yong [7] presents shared metadata storage pool management algorithm and used hash function to distribute metadata avoiding a hotspot of directory. But the choice of hash function is difficult, improper hash function would reduce performance of network storage system and its adaptability is poor. Therefore, how to reduce time and space overhead and increase adaptability at the same time is important issues in current metadata management algorithm. We introduce hash postfix table to improve dynamic equilibrium distribution capability, and design hash postfix table based metadata management algorithm. This paper is organized as follows: Section 1 gives the structure of mass storage system metadata management module; Section 2 describes process of the hash postfix table based metadata management algorithm process; Section 3 analyzes and compares algorithms in terms
- 97 -
The Hash Postfix Table Based Metadata Management Algorithm for Mass Storage System Xiujuan LI，Tao CAI，Shiguang JU
of the time and space overhead and dynamic equilibrium distribution capability; Section 4 implements the prototype and uses real data sets to test and analyze.
2. The Structure of Metadata Management Module The structure of metadata management module in mass storage system is shown in Figure1, including a number of MDS (containing one main MDS), hash value calculation module, hash postfix table module and host module (client). MDS preserve all metadata of mass storage systems, and the main MDS is responsible for the collection and updating of metadata. Client submits access requests to hash value calculation module and receives the identity of MDS from the hash postfix table module, then client accesses the corresponding MDS for metadata information according to the identity of MDS. Hash value calculation module receives access requests submitted by client, calculates hash value based on metadata identify and transfers it to the hash postfix table. Hash postfix table module uses the hash value to search in hash postfix table, finds the corresponding identity of MDS and feedbacks to the client. 1. Client hashes the pathname pathname
hash index
Hash function
Hash Postfix Table
2. Return the MDS ID 4. Send response to client Client
3. Send metadata request to the target MDS
cMDS
sMDS
cMDS
cMDS
Figure 1. The Architecture of the Hash Postfix Table based Metadata Management Module
3. The Hash Postfix Table Based Metadata Management Algorithm There are some different characteristics between metadata and data in mass storage system. The number of metadata is small, so it is possible to manage metadata efficiently. In mass storage system metadata operations is more than 50% of total operations and a significant number of access requests use metadata only, so metadata management algorithm is very important to improve the performance of mass storage system. The frequency of large amount of data being accessed will declined sharply after a certain period of time in mass storage system, but their metadata access frequency falls late and slowly, maybe their frequency of being accessed will remain high within its life cycle. Therefore, the data management algorithm is not suitable for metadata manage in mass storage system.
The directory sub-tree partition algorithm needs massive migration time and space overhead for migration when renaming directory, and the hot MDS will reduce the performance of mass storage system. The adaptability of hash-based algorithm is poor and hash function selection is difficult. In general, poor dynamic equilibrium distribution ability is the main problems of current metadata management algorithm. This paper presents the hash postfix table, designs the hash postfix table metadata management algorithm and improves the capacity of dynamic equilibrium distribution. The following introduces hash postfix table, then presents the process of metadata querying and hash postfix table quick adjustment.
3.1. The Hash Postfix Table Each record of the hash postfix table is a triple (Hash Value Begin, Hash Value End, Metadata Server ID). Hash Value Begin is the minimum hash value of this interval, Hash Value End is the maximum hash value of this interval, Metadata Server ID is the identity of MDS corresponding to this hash value interval. Each record keeps a hash value interval and the identity of corresponding MDS. While the hash value of access request is in this interval, the corresponding MDS that identified by
- 98 -
International Journal of Digital Content Technology and its Applications Volume 4, Number 9, December 2010
Metadata Server ID will deal with the metadata operation of this access request. If there are four MDS in current metadata management module, the corresponding hash value of the metadata identify is in the interval [0X0000, 0X4fff], maybe the hash postfix table as shown in Table 1. Table 1. The Structure of the Hash Postfix Table Hash value Begin Hash value End Metadata Sever ID 0X0000 0X1000 0X2000 0X3000 0X4000
0X0fff 0X1fff 0X2fff 0X3fff 0X4fff
0 1 2 0 3
If the hash value of access request is in the interval [0X0000, 0X0fff] or [0X3000, 0X3fff], MDS which identify is 0 will deal with metadata operation of this access request. When hash value of the access request is in the interval [0X1000, 0X1fff], MDS which identity is 1 will deal with metadata operation of this access request.
3.2. The Query Process of Metadata In mass storage system, the client receive the user’s access requests firstly, the client submits the requests to one MDS and receives the appropriate metadata information, then it returns metadata back directly or uses the information stored in metadata to access appropriate storage device for data. How to determine which MDS to deal with access requests is a key issue. MDS choosing is directly related to the time and space overhead of access request processing, and the performance of mass storage system. So it is the key problem in our researching. Here we presents the hash postfix table based metadata management algorithm, the metadata searching process is as follows: Step1. Client receives the access request submitted by user and sends the access request to the hash value calculation module. Step2. Hash value calculation module analyzes the access request, uses the identity contained in the metadata (file pathname or file name, etc.) to calculate hash value and sends the hash value to the hash postfix table module. Step3. The hash postfix table module uses the hash value as the keyword to look up the hash postfix table, determines the hash value interval and sends the identify of corresponding MDS back to the client. Step4. Client sends access requests to the corresponding MDS. Step5. The corresponding MDS search metadata information and return back to the client. If a host received an access request of file0 and the hash value of file0 accessing path is 0X2012, and hash postfix table shows as Table 1, the process of metadata querying is shown in Figure2. The identity of corresponding MDS is 2. Host receive the request from client
Send the request to hash postfix table module
Hash value calculation module hash file pathname
Send the hash value (0X2012) to hash postfix table module
Obtain the identity of corresponding MDS? Yes Send the identity of corresponding MDS to client
Host send request to the corresponding MDS
Search the metadata in this MDS
Send information to client
Figure2. The Query Process of Metadata
- 99 -
No
The Hash Postfix Table Based Metadata Management Algorithm for Mass Storage System Xiujuan LI，Tao CAI，Shiguang JU
3.3. Quick Adjustment Process When there are a lot of access requests that need to process in one MDS, the time for one access request waiting will greatly increase, this MDS becomes hotspot, and it will seriously affect the performance of mass storage system. After adding new MDS and deleting existing MDS, the metadata management module also needs large time and space overhead to adjust, otherwise the metadata management module will not take advantage of new MDS or make some mistakes. There is no quick adjustment strategy in current metadata management algorithms. Directory sub-tree partition algorithm adjust sub-directory contained in MDS to eliminate hotspot and adapt to the change of metadata management module, but which requires a lot of time and space overhead. The hash-based algorithm needs to change the hash function to eliminate hotspot and adapt to the changes of metadata management module, which would cause large amount of metadata migration and need a lot of time and space overhead. At the same time, the choice of hash function is very difficult and the hash function change may lead to another hotspot. We adopt the hash postfix table, and a small number of records in the hash postfix table are needed to adjust for metadata management module change. Here we give the quick adjustment process in the view of appearing hotspot MDS, adding MDS and deleting MDS. 3.3.1. Hotspot MDS Appearance We adjust the hash postfix table to eliminate hotspot MDS in metadata management algorithm based on hash postfix table, avoiding the change of hash function and migration of metadata in the MDS. The quick adjustment process of hash postfix table is as follows: Step1. Detecting the ratio between the number of access requests received per unit time and the number of access requests processed. If the ratio is more than a certain threshold, then this MDS is a hotspot, its identify is marked as MDS_HOT_MAX. Step2. Finding the MDS which has the smallest ratio between the number of received access requests per unit time and the number of access requests processed. Its identity is marked as MDS_HOT_MIN. Step3. Reducing the hash value interval in hash postfix table where Metadata Server ID is MDS_HOT_MAX. Step4. If the separation hash value interval is adjacent to the hash value interval which Metadata Server ID is MDS_HOT_MIN, updating the hash value interval of it. Step5. If they are not adjacent, the hash value will be a new record and inserted into hash postfix table, and its Metadata Server ID is MDS_HOT_MIN. The hash postfix table is shown as Table 1. If MDS identified as 0 is hotspot and the MDS identified as 2 has the minimum ratio between the number of access requests received and the number of access requests processed, the hash value interval will be reduced where the identity of MDS is 0 and revised 0X0000~0X0fff to 0X0000~0X07ff. Then looking up the records in the hash postfix table where Metadata Server ID is 2. If its hash value interval [0X0800, 0X0fff] can not merge with the hash value interval [0X2000, 0X2fff], the new record (0X0800, 0X0fff, 2) is added to hash postfix table. After adjustment the new hash postfix table shows as Table 2. Table 2. The New Hash Postfix Table for Hotspot MDS Hash value Begin Hash value End Metadata Sever ID 0X0000 0X1000 0X2000 0X3000 0X4000 0X0800
0X07ff 0X1fff 0X2fff 0X3fff 0X4fff 0X0fff
3.3.2. Adding MDS
- 100 -
0 1 2 0 3 2
International Journal of Digital Content Technology and its Applications Volume 4, Number 9, December 2010
When adding an new MDS, the metadata management module can adapt quickly by adjusting some hash postfix table records. The process of adjusting the hash postfix table is as follows: Step1. Detecting the ratio between the number of access requests received per unit time and the number of access requests processed in MDS, selecting the MDS which has the largest ratio and marking its identify as MDS_ADD_MAX. Step2. Reducing hash value interval which identify of MDS is MDS_ADD_MAX. Step3. Inserting the remaining hash value interval as a new record to hash postfix table, its Metadata Server ID is as identity of new MDS. Assuming that the new adding MDS’s identity is 5, the current contents of the hash postfix table shows as Table 2, where the MDS identified as 2 has the largest ratio between the number of access requests received per unit time and the number of access requests processed. So modifying record (0X0800, 0X0fff, 2)in the hash postfix table, Metadata Server ID is 5, the new record is (0X0800, 0X0fff, 5). The new hash postfix table is shown in Table 3. Table 3. The New Hash Postfix Table for Adding MDS Hash Value Begin Hash Value End Metadata Sever ID 0X0000 0X1000 0X2000 0X3000 0X4000 0X0800
0X07ff 0X1fff 0X2fff 0X3fff 0X4fff 0X0fff
0 1 2 0 3 5
3.3.3. Deleting MDS After deleting an MDS in the metadata management module, the hash postfix table must be adjusted to avoid mistake in metadata distribution. The adjustment steps about hash postfix table are as follows: Step1. Detecting the ratio between the number of access requests received per unit time and the number of access requests processed in MDS, selecting the MDS which has the smallest ratio and marking its identify as MDS_DEL_MIN. Step2. Finding all the records with the identify of deleted MDS in the hash postfix table. Step3. Modifying Metadata Server ID as MDS_DEL_MIN of those records. Step4. Merging the adjacent hash value interval record with same Metadata Server ID. Assuming that the deleting MDS identify is 5, the current hash postfix table shows as Table 3. The MDS identified as 3 has the smallest ratio between the number of access requests received per unit time and the number of access requests processed. Then modifying the record (0X0800, 0X0fff, 5) in hash postfix table, the new record is (0X0800, 0X0fff, 3). The new hash postfix table is shown in Table 4. Table 4. The New Hash Postfix Table for Deleting MDS Hash Value Begin Hash Value End Metadata Sever ID 0X0000 0X1000 0X2000 0X3000 0X4000 0X0800
0X07ff 0X1fff 0X2fff 0X3fff 0X4fff 0X0fff
0 1 2 0 3 3
4. Analysis This paper adopts hash postfix table to manage metadata and designs new metadata management algorithm in mass store systems, which can improve the dynamic equilibrium distribution capability and reduce the time and space overhead. We analyze the efficiency from the aspect of the dynamic distribution capability, the time and space overhead of the metadata management.
- 101 -
The Hash Postfix Table Based Metadata Management Algorithm for Mass Storage System Xiujuan LI，Tao CAI，Shiguang JU
4.1. Dynamic Equilibrium Distribution Capability Current metadata management algorithms have poor dynamic equilibrium distribution capability. Sub-tree partition algorithm can only adjust the sub-directory between MDS of different methods when there is a hotspot problem. On one hand, it needs a lot of time and space overhead when moving metadata among MDS. On the other hand, it needs to update the sub-tree distribution information which may easily cause errors about synchronization problems. The hash-based algorithm needs to change the hash function to eliminate hotspots, but the choice of hash function is very difficult and changing hash function may lead to anther hotspots. Each MDS only stores part of metadata information, the metadata migration needs a lot of time and space overhead. The hash postfix table based metadata management algorithm adopts hash postfix table, which enhances the efficiency of the dynamic equilibrium distribution capability. In the event of hotspots, it only needs to modify the hash postfix table which needs less time and space overhead; it needn’t re-select hash function, which avoids new MDS hotspots due to improper hash function; changing the hash function does not require additional time and space overhead; there is no metadata migration overhead because each MDS preserves all metadata information. When adding an MDS, it only need add corresponding records which require less time and space overhead. When deleting an MDS, it also need less time and space overhead to update the hash value range corresponding to the deleted MDS to other MDS.
4.2. Time and Space Overhead Based on the level of metadata information, directory sub-tree partition algorithm organizes and distributes metadata, which needs a lot of time and space overhead because of traversing information of each sub-directory. The hash-based algorithm uses the hash function calculate the location of the metadata, requiring less time and space overhead, but the choice of hash function is very important. If the collision rate is higher, the time and space overhead will increase dramatically. The hash postfix table based metadata management algorithm uses hash postfix table to adjust the results about calculation, enhancing the ability of the dynamic equilibrium distribution. It needs less time and space overhead than directory sub-tree partition algorithm because it need not traverse all the subdirectories to access information. The hash postfix table based metadata management algorithm has to look up the hash postfix table so that it has more time and space overhead than the hash-based algorithm. But the hash postfix table is simple and has a small number of records so retrieving it needs less time and space overhead, which ensures the increasing overhead of the hash postfix table based metadata management algorithm is much smaller than the hash-based algorithm.
5. Prototype and Evaluation We use the C language to realize the prototype system of hash postfix table based metadata management algorithm in the Linux. Using the pathname of the file to calculate the hash value of metadata identify. Simulating hotspot in a MDS, adding and deleting an MDS to test the quick adjustment of hash postfix table. At the same time the prototype system of hash-based metadata management algorithm is realize. We use real data sets to test.
5.1. Data Sets and Testing Environment Because the current file system has no standard open test data sets, we use the test method in literature [8] to collected 32,557 metadata information in Linux as the data sets of prototype. The testing environment of prototype system are shown in table 5.
- 102 -
International Journal of Digital Content Technology and its Applications Volume 4, Number 9, December 2010
Table 5. Testing Environment CPU
Intel Pentium 4 2.93 GHz
Memory
1024M
OS
Redhat Enterprise 4.0 (kernel:2.6.9-42.14)
Hard disk
Seagate SATA 160G
5.2. Testing and Analysis
2500 2000 1500 1000 500 0
2000 4000 6000 8000 the number of system request
the number of request in MDS4
the number of request in MDS3
We simulated four MDS in prototype system, use file pathname to calculate hash value, and used the hash value as a keyword to look up hash postfix table and obtained the identity MDS. Due to the real-time testing access requests distribution was difficult, we established four testing-point to evaluate the prototype. Selecting 8,000 metadata randomly the data sets to construct the access request sets of two prototype system, and establishing four test points when the number of access requests is 2000, 4000, 6000 and 8000 to obtain the access requests number in each MDS. And using it as a measure of access requests distribution capability of metadata management algorithms. There are four MDS such as MDS1, MDS2, MDS3 and MDS4. The results are shown in Figure3, Figure 4, Figure 5 and Figure 6.
algorithm based on hash algorithm based on hash postfix table
2500
2000 1500 1000
500 0 2000 4000 6000 8000 the number of system request algorithm based on hash algorithm based on hash postfix table
Figure4. The Number of Access Request in cbzxcbz xMDS2 the number of request in MDS4
the number of request in MDS3
Figure3. The Number of Access Request in MDS xcbzxcbbzxcbzxcbbbbbbbbbbbbbbbbbzx 2500 2000 1500 1000 500 0
2500 2000 1500 1000 500 0
2000 4000 6000 8000 the number of system request
2000 4000 6000 8000 the number of system request
algorithm based on hash algorithm based on hash postfix table
algorithm based on hash algorithm based on hash postfix table
Figure5. The Number of Access Request in MDS3 Figure6. The Number of Access Request in xcbzxcbbzxcbzxcbbbbbbbbbbbbbbbbbzx cbzxcb z MDS4 The result showed that the access request number in MDS rising slowly with the hash postfix table based metadata management algorithm. And in the four testing points the access request number of four MDS was close to each other. Using hash-based metadata management algorithm, the access request number in MDS was also rising, but their change was different. So the hash postfix table based metadata management algorithm had better dynamic equilibrium distribution capability and can avoid hotspot. We compared the access request number of each MDS when the number of access request is 2000,
- 103 -
The Hash Postfix Table Based Metadata Management Algorithm for Mass Storage System Xiujuan LI，Tao CAI，Shiguang JU
the number of request of each MDS
4000, 6000 and 8000. The result showed in Figure 7, Figure 8, Figure 9 and Figure 10. the number of request of each MDS
1500
540 520
1000
500
500
480 460 440
MDS1
MDS2
MDS3
0 MDS1
MDS4
MDS2
MDS3
MDS4
algorithm based on hash algorithm based on hash postfix table
algorithm based on hash algorithm based on hash postfix table
the number of request of each MDS
Figure 7. The Number of Access Request is 2000 Figure 8. The Number of Access Request is xcbzxcbbzxcbzxcbbbbbbbbbbbbbbbbbzx cbzxcb z 4000 3000
the number of request of each MDS
1600
2500
1550
2000
1500
1500
1450
1000
1400 1350 MDS1
MDS2
MDS3
MDS4
500 0 MDS1
algorithm based on hash algorithm based on hash postfix table
MDS2
MDS3
MDS4
algorithm based on hash algorithm based on hash postfix table
Figure 9. The Number of Access Request is 6000 Figure 10. The Number of Access Request is xcbzxcbbzxcbzxcbbbbbbbbbbbbbbbbbzx cbzxcb z 8000 From Figure7, Figure8, Figure9 and Figure10, we found that the number of access requests was uneven in four MDS using hash-based metadata management algorithms, and the number of access requests was uniform four MDS using hash postfix table based metadata management. This showed that hash postfix table based metadata management algorithm can homogeneously distribute metadata between several MDS to avoid hotspot.
6. Conclusions Based on the analysis of metadata management features in mass storage system, we designed the structure of metadata management module and proposed hash postfix table based metadata management algorithm, and described the process of metadata querying and quick adjustment. We analyzed the hash postfix table based metadata management algorithm from two aspects: dynamic equilibrium distribution capacity and the time and space overhead. Finally, we implemented a prototype system and evaluated with real data set. The results showed that the hash postfix table based metadata management algorithm is more effective for dynamic equilibrium distribution of access requests, avoiding the hotspot and improving the flexibility of metadata management. Archive storage system is an important application of the mass storage system with its special features, these features make more requirements to the metadata management algorithm. In the future, we prepare to improve the performance of hash-based metadata management algorithm based on special metadata management features of archive storage system.
- 104 -
International Journal of Digital Content Technology and its Applications Volume 4, Number 9, December 2010
7. Acknowledgement This work is supported by the Natural science foundation of Jiangsu Education under Grant No.09KJB520001, senior fund of Jiangsu university under Grant No.09JDG038, the Natural science foundation of Jiangsu under Grant No.2007086 and the Natural science foundation of China under Grant No.60773049/F020703.
8. References [1] [2] [3] [4]
[5] [6] [7]
[8]
D.Roselli, J.Lorch, and T.Anderson. A Comparison of File System Workloads. Proceedings of the 2000 USENIX Annual Technical Conference, page 41-54, June 2000. Popek G J, Rudisin G, Stoughton A, et al. Detection of Mutual Inconsistency in Distributed Systems[J]. IEEE Trans. on Software Engineering,1986,12(11):1067-1075 Satyanaray M, Kistler J J, Kumar P, et al. Coda: A Highly Available File System for Distributed Workstation Environments[J]. IEEE Trans. on Computers, 1990, 39(4): 184-201 Brandt S A, Lan Xue, Miller E L, et al. Efficient metadata management in large distributed file systems: Proceedings of the 20th IEEE / 11th NASA Goddard Conference on Mass Storage System and Technologies[C]. San Diego:[s.n.], 2003:290-297 S.A. Weil, K.T. Pollack, S.A. Brandt, and E.L. Miller. Dynamic Metadata Management for Petabyte-Scale File System. Proc. ACM/IEEE Conf. Supercomputing(SC’ 04), p. 4, 2004. Corbett P F, Feitelso D G. The Vesta parallel file system[J]. ACM Trans. on Computer System, 1996,14(3):225-264 SU Yong, ZHOU Jing-li, YU Sheng-sheng, JIANG Ming-hua, LIU Gang. Research and Design of MDS in Distributed Storage System[J]. Journal of Chinese Computer Systems, 2007, 28(4): 734-737. LIU Zhong, ZHOU Xing-Ming. A Metadata Management Method Based on Directory Path. Journal of software, 2007, 18(2):236-245.
- 105 -
