Paper number: 1568952365
1
Hybrid Open Hash Tables for Network Processors Qing Ye, Dale Parson, and Liang Cheng
Abstract—Packet classification by analyzing the traffic header information is an essential task of network processors. To efficiently store, retrieve and update information, key-based data accessing algorithms such as hashing are usually used to improve the processing performance. In this paper, we propose the hybrid open hash, a composite algorithm that combines open addressing hash tables with the temporal responsiveness of incremental garbage collection. By dynamically switching between a novel table with incremental construction via selective copying of used entries, and an aged table with incremental cleaning via emptying of deleted entries, this algorithm solves the structural problem of conventional open hashing due to remnants left by deleted keys. Thus, it increases the data accessing speed. We also implement and compare our algorithm with Brutil, the latest proposed hashing mechanism designed for real-time embedded systems, from the concurrency issue of multithreading to the performance simulation by taking the real traffic information to construct hashed keys. Our results show that hybrid open hash has better performance. Several possible enhancement approaches are also discussed to improve the performance further. Index Terms—garbage collection, hash algorithm, network processor
N
I. INTRODUCTION
ETWORK Processors (NPs) are special purpose programmable real-time embedded systems with optimized architectural features that aim to perform real-time packet-processing functions in a fast speed manner [1]. NPs are expected to become the core of the fourth generation of network devices such as routers, voice over IP (VoIP) bridges and virtual private network (VPN) gateways, which require a high degree of flexibility to support evolving network services with high packet rates. For example, the APP550 network processor from Agere Systems is designed to process 5 Gigabits per second [2]. Packet classification is one of the essential tasks of NPs. It is required by almost all the ingress and egress modules along the fast-path in a network processor and some applications Qing Ye is with the Laboratory Of Networking Group (LONGLAB), Computer Science and Engineering Department, Lehigh University, Bethlehem, PA. 18015, USA (phone: 610-758-4801; fax: 610-758-4096; email: qiy3@ Lehigh.edu). Dale Parson is with Agere System, Allentown, PA. 18109, USA (e-mail:
[email protected]). Liang Cheng is with the Computer Science and Engineering Department, Lehigh University, Bethlehem, PA. 18015, USA (phone: 610-758-5941; fax: 610-758-4096; e-mail:
[email protected]).
need deep cross-checks of multiple fields in packet headers. Under the strict time limit, the job of packet classification has to be done in the line speed to decrease the processing delay. Key-based hashing algorithm has been proved to be a good approach to help a lot of network systems and applications efficiently store, query and process data. For instance, resource distribution between the peers in Chord is performed based on certain hashing mechanisms [3], and Secure Hashing Algorithms (SHA) is quite accepted for implementing network security [4]. To identify a connection, a combination of 16-bit port number with 32-bit IPv4 address from both the source and the destination can be taken as the hashed key. However, the conventional hashing algorithms including both the open and chained addressing mechanisms suffer from great performance degradation over time that limits their usefulness in real-time systems. In this paper, we are going to propose hybrid open hash table, a composite algorithm that combines open addressing hash table with the temporal responsiveness of incremental garbage collection to solve this performance degradation problem. It is done by dynamically switching between a new table with incremental construction via selective copying of used entries, and an aged table with incremental cleaning via emptying of deleted entries. Simulation results shows that hybrid open hash has better performance of both maximum and average running than Brutil [5], a latest proposed chained hash table designed for real-time embedded systems. The rest of this paper is organized as follows. In section II, existing hash table approaches and their performance degradation problems are discussed. Section III illustrates the details of our hybrid open hash table approach. Simulation results and associated discussions are presented in section IV. Finally, Section V concludes this paper. II. RELATED WORKS A. Conventional Chained and Open hash tables Hash table has been a research topic for more than thirty years. It is one of the most basic data structures implemented in a lot of system-level libraries such as Sun’s Java. In general, a hash table is typically constructed by mapping all possible object keys to a relatively smaller space of buckets, which index the table. During the hashing process, collisions may happen when two keys are indexed to the same bucket by their hashing functions. Based on the structure of buckets and how
Paper number: 1568952365
2
to handle the hashing collision issue, conventional hash tables can be categorized into chained and open addressing mechanisms [6]. Chained hashing algorithm maps a key to a bucket and each bucket maintains a linked list of entries that may contains several objects hashed to the same index. Each entry in the linked list is constructed with the object key, some dependent data, and a pointer to the next entry in the same chain. Searching for an object needs two steps: a single hashing step reach to a bucket followed by linear-time serial search through the linked list. It terminates when finding the object or visiting the linked list exhaustively. Ideally, with a good hashing function to distribute the keys uniformly into the table, each bucket’s list stays short, and the data searching would go quickly. An open hash table, in contrast, maintains at most 1 entry at each bucket. When hashing collision happens, the second key that is hashed to the same bucket will call a rehash function to find a new bucket. Rehashing is continued until an empty entry is found. Searching for an object needs a single hash step to find the start bucket followed by a serial of rehash steps if the desired object is not found at the index. Rather than build explicit linked lists, an open hash table inserts each key into an implicit serial list defined by the hash and rehash indices of the keys. Fig.1 presents the differences between conventional chained and open hash tables. Chained Hash Table
Open Hash Table
Bucket
Entry
Entry
Entry
Bucket
Entry
Bucket
Entry
Entry
Entry
Bucket
Entry
Bucket
Entry
Bucket
Entry
...
Entry
Entry
...
Rehash
Fig. 1. Conventional Chained and Open hash table.
B. Performance degradation problem Comparing to open hash tables, a chained hash mechanism suffers the problem of large processing delays for real-time embedded systems due to the following reasons: --Concurrency issue. Multithreading poses the biggest problem for chained hash tables. When multiple threads have concurrent access to a table, any of them modifying an entry must lock its bucket so that it cannot be read or written at the same time by other threads, particularly when two or more interdependent fields must be modified atomically in order to maintain consistency. This restriction is the well-known critical section problem [7]. Unfortunately, locking a bucket incurs delays for other threads accessing data and it happens quite often in network processing to speed up the overall performance. Open hash tables share the same need to restrict concurrency within a bucket, but the problem is worse for chained hashing because every insertion and deletion requires locking the bucket of interest, resulting in more stalls not encountered with open hashing. --Memory accessing delay. For a chained hash table, the
nature of the linked list data structure inside a bucket will put a colliding key into the end of the chain when collision happens. Thus, the increment of the length of the list also increases the average data accessing time because a searching operation for that new key has to go through all the entries before it meets the desired one. The same scenario happens with open hashing only in the worst cases when the table is heavily loaded. Even in the zero-collision case, at least two memory accesses must occur to inspect a key residing in the table. The first access reads a hash bucket’s pointer to the linked list element, and the second one reads fields from the appropriate entry. Open hashing requires only one memory access to read a noncolliding key from a one-element bucket. However, the advantages of conventional open hash table do not imply it is an approach good enough for embedded systems that require strict predictable bounds on every system call. In fact, conventional open addressing mechanism has a significant flaw of not dealing with deleted entries efficiently, that is as pernicious as chained hash tables’ excessive memory access overhead. In section III, we will illustrate this problem in detail and discuss our solution based on incremental garbage collection. C. Existing improvements Friedman proposed Brutil [5], a new approach to improve the data accessing performance for chained hash tables. To our best knowledge, it is also the latest hashing mechanism exclusively designed for real-time embedded systems. However, Friedman’s work doesn’t target on the above performance degradation problems mentioned in this paper. It concentrates on solving the large delays that arises when a hash table becomes too heavily loaded, resulting in many collisions and a long linked list in each bucket. Brutil’s idea is to pre-allocate a new bigger chained hash table and incrementally insert entries from the existing smaller table into this fresh new one whenever table growth is occurring, while the conventional solution is to copy the whole old table only when a bigger one is needed. In this way, Brutil avoids the large delay between the deletion of old table and the creation of the new table. Friedman also states that open hashing is not well suited to real-time embedded system applications in his paper. Our comparison results will illustrate that clearly the use of hybrid open hashing with garbage collection given in this paper, stand as a counterexample. Knuth [8] gives an algorithm for in-place reorganization of an open hash table to remove deleted entries that requires linear probing for collision resolution. Szymanski [9] improves on Knuth’s algorithm by removing the requirement for linear probing, but both approaches are also monolithic in their table reconstruction. They avoid the memory cost of maintaining two tables during reconstruction, but they do not avoid the worst-case table access time encountered during reconstruction. Other efforts such as the Deitzfelbinger’s dynamic perfect hashing [10] and Fredman’s hash functions for priority queue [11] also discussed how to improve the performance of hashing algorithms. But neither of them targets
Paper number: 1568952365
3
on the real-time embedded systems which is the major concern of this paper. III. HYBRID OPEN HASH TABLE A. Problem of conventional open hash tables As we discussed above, open hash table is a good data accessing approach for network processors except that it suffers from a problem of not dealing with deleted entries efficiently. A simple example can illustrate this performance problem very well. Suppose we are trying to map four-digit keys in range of 0000-9999 to the indices from 00 to 05 for a 6-entry hash table. In the real world, the size of hash tables used by network processors is thousand times larger than this example. The hash function in use is to simply multiply the key’s left two digits by its right two digits to form a four-digit product, from which the middle two digits are taken as the hash index. For example, the key 4567 yields a product 45 × 67 = 3015, with the middle digits giving a hash bucket of 01 for this hash function. Typical network processing uses a much more complicate hashing algorithm, employing bit shift and XOR operations to compress bits into a bucket. We also assume the simplest linear probing that looks in the subsequent available bucket is taken as our rehashing function when collision happens. In initial, the table is empty. Suppose 5 keys in the sequence of 0110, 0210, 0119, 0005, and 0099 are inserted into the table with the results of hash (or rehash) indices of 1, 2, 3, 0 and 4 are used. After that, the first four keys are deleted and only key 0099 is left. The performance problem becomes obvious if we search for 0099 in the table. Conventional open hashing mechanism will first locate at bucket 0 by hashing 0099 and find a deleted entry. Then the rehashing function of linear probing will lead the searching operation visit all the buckets from 1 to 3 till it eventually find 0099 is in bucket 4. Figure 2 illustrates this example. Bucket
Key
State
Bucket
00
0005
used
00
deleted
01
0110
used
01
deleted
02
0210
used
02
deleted
03
0119
used
03
04
0099
used
04
empty
05
05 (a)
Key
State
deleted 0099
used empty
(b)
Fig. 2. (a) Insert 5 different keys into the open hash table by hash/rehash functions (b) Search for 0099 after deleting the first four keys. Arrow represents the rehashing processes.
The problem becomes worse if the desired key is not in the table and when inserting a new key into the table. In the worst case, searching in an open hash table with all buckets in the deleted states has to go through the overall table exhaustively, resulting in large delays that are not acceptable by network processors. B. Hybrid open hash tables with incremental garbage collection A straightforward way for non-real-time applications to
solve this performance issue is to create a new open hash table and only duplicate all the used buckets from the aged table into the fresh one. Thus, no deleted bucket would be in the new table and searching would run smoothly and easily. However, the monolithic reconstruction and reorganization of a new table incurs large execution time penalties that would cause real-time applications such as network processors miss their deadlines. We propose a hybrid open addressing approach to meet the needs of real-time embedded systems. Our idea is inspired by incremental garbage collection. Garbage collection is a technique for automatically recovering storage from a running program’s heap for application data structures that are no longer referenced by the running program. Classic garbage collection algorithms are monolithic — they stop all application processing during reorganization of memory, thereby impeding real-time responsiveness similar to monolithic hash table reorganization. Real-time systems rely on incremental garbage collection, which interleaves its work with application processing in small, constant time-bound steps. Incremental copy collection achieves reorganization by copying application data into a new heap that is initially free of garbage. It is this property of incremental reorganization in the interest of avoiding monolithic stalls that our hybrid algorithm adapts to open hashing. In stead of creating a new open hash table when collision happens often, our approach is to maintain two tables with the same size from the beginning. The current table is responsible for the new insertion of keys and the alternate table is to filter out the deleted entries by incremental garbage collection. The basic idea is to avoid the monolithic reconstruction of a new hash table by continually organizing the current table by copying used entries from the alternate table while skipping over deleted and empty entries. The reorganization of these two tables proceeds in two phases. During the copy phase, searching caused by inserting a new key, retrieving an existing key, or deleting a useless key will first go through the current table and, if it doesn’t find the target, it turns to the alternate table. A new key is always inserted into the current table while the deletion operation is performed in both tables. There is a garbage collector in the alternate table and it is invoked at the end of every hashing operation. In each step, the garbage collector advances an index variable cleanix through the alternate table, one entry per invocation. When a used entry (i.e., a valid key) is found in the alternate table, it copies that key into the current table as a hashed insertion and then continues to the next entry. Eventually, cleanix advances to the end of the alternate table where all the valid keys in the alternate table has been copied into the current table and, in the mean time, all the new keys are put into the current table. Thus, the used and empty entries in the alternate table are skipped. Garbage collector then resets cleanix to the start of the alternate table and hybrid open hash algorithm turns to the clean phase. During the clean phase, only the current table is responsible
Paper number: 1568952365 for performing all the hashing operations. The garbage collector now sets one entry inside the alternate table to be empty in each step. When cleanix advances to the end of the alternate table, the garbage collector also reset cleanix to the start of the alternate table and hybrid open hashing switch the roles of the current and alternate table so that the previous alternate table (which is now completely empty) becomes the current table (for new insertions), and the previous current table now becomes the alternate table to be filtered for deleted entries by having its used entries copied. Then a new copy phase begins and the garbage collector points to the new alternate table. Note that this incremental table reorganization could be invoked by a background thread that runs during lulls in real-time activities to improve the performance. C. Hybrid Open hash enhancement Given the complicated copy-clean-switch process of adding incremental garbage collection onto open hash tables, the performance costs still possible outweigh the benefits. There are two major costs in our algorithm: memory costs of preallocating two large hash tables and the execution time penalties by calling garbage collection too often. The former cost is not a big issue with real network processors since putting enough memories into the system can solve this issue easily. Moreover, in the initial period, the size of consumed memory could be determined by sum of the size of the current and alternate tables. The performance cost of the second issue could be enhanced by controlling the frequency of garbage collection operations. A simple approach is to call garbage collection only when the work load of hashing operations exceeds a predefined threshold. This threshold could be hard coded by programmer or be a configurable variable set up by the software modules of a network processor. Rather than perform garbage collection once per hash operation in the basic algorithm, this enhancement method uses certain fixed threshold. It could effectively reduce the execution penalties of garbage collection but has one obvious limitation of how to determine an appropriate threshold since it depends on the requirement of different applications. An advanced enhancement method is to determine the threshold adaptively in stead of using a fixed value. The adaptive enhancement approach records the current minimum number of table probes for the current minimal-cost hashing operation within a window of fixed number of operations. At the conclusion of the window, the garbage collector sets its threshold to that minimum, discarding its previous threshold value. It then repeats the process and always gets the minimum and adapts to the changes of current work load. A final approach is the timeout enhancement that applies to hash tables where each used entry is valid for only a finite period time after the most recent hash operation on that entry’s key. When an entry’s timeout period has expired, the key in this entry is removed automatically. Each hash operation that locates its keyed entry must update the timestamp field in that entry with the new expiration time for that key. Searches that
4 detect entries with expired timestamps treat those entries as having deleted status and garbage collection does not copy them into the current table. IV. SIMULATION RESULTS To get a clear view of the execution performance, we directly compare our hybrid open hashing algorithm with Brutil, the latest proposed chained hashing approach for realtime embedded systems. Our simulation is done by one PC with an Intel Pentium4 2.4GHz CPU and 256Mbytes RAM. A traffic file with 4,000,000 records generated by a traffic generator of Agere Systems to test the packet classification service for APP550 is taken as the test data. We implement hybrid open hash tables with Java as same as what Brutil is done. To get a fair result, both algorithms run without multithreading and have the same initial size of hash tables. And no enhancement approach mentioned in section III is added to the basic hybrid open table implementation. A combination of IP address and port number from the traffic file is used to generate a 48-bits hashed key. The same hashing operations are performed by both approaches and the running time of Java’s garbage collection is deducted for all the tests during the process of simulations. We first compare the maximum execution time for running one hash operation in these two hash tables since bounded performance is much more important for real-time embedded systems. Fig. 3 presents our results of testing 100,000-500,000 hashing operations.
Fig. 3. Comparison of maximum running time per hashing operation
The total running time of hybrid open hash tables and Brutil is also compared in Fig. 4. From the simulation results, we can easily find that hybrid open hashing algorithm has a bounded execution time no matter how large the number of hashing operations is, while the bound of Brutil increases as the number of hashing operations increases. The total running time also shows that even working with only one thread and no enhancement method is enabled, the overall performance of hybrid open hash tables is better than Brutil.
Paper number: 1568952365
5
The adaptive and timeout enhancement methods are also implemented based on the basic hybrid open hashing algorithm. To compare their performance, 2,000,000 hashing operations from the same traffic files are performed with each method and the maximum, minimum and average running time of each hash operation are recorded. Table I shows the comparison results. We can also find that by taking our proposed enhancement approach, the performance of hybrid open hash table can be improved further.
Foundation (NSF) and a grant from the Commonwealth of Pennsylvania, Department of Community and Economic Development, through the Pennsylvania Infrastructure Technology Alliance (PITA). Our special thanks are given to Scott Friedman for providing us the source codes of Brutil. Any opinions, findings, and conclusions or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of the sponsors of this research. REFERENCES [1]
Fig. 4. Comparison of total running time TABLE I COMPARISON OF ENHANCEMENT METHODS OVER 2,000,000 HASING OPERATIONS (IN MS) Algorithm
Maximum
Minimum
Average
Basic Adaptive Timeout
15 8 6
2 2 2
3.432 3.246 2.496
V. CONCLUSION For real-time embedded systems such as network processors, conventional hash tables have the problems of performance degradation due to concurrency issues with multithreading, large memory accessing delays and not dealing with deleted entries efficiently. In this paper, we propose an approach of combining hybrid open hash tables with incremental garbage collection to meet the needs of real-time applications. By maintaining two hash tables in stead of one table in conventional approaches, hybrid open hashing approach incrementally copy the valid keys from the alternate table into the current table, skipping over the deleted or empty entries. In this way, hashing operations always deal with a table without two many hash collisions. Our simulation results show the performance of hybrid open hash tables is better than Brutil and it still has potentials to be improved further by several enhancement approaches. ACKNOWLEDGMENT This research has been supported by National Science
M. Peyravian and J. Calvignac, “Fundamental architectural considerations for network processors”, IEEE Journal of Computer Networks, pp. 587-600, issue 41, 2003. [2] D. E. Comer, “Network Systems Design using Network Processors, Agere Version”, Upper Saddle River, NJ: Prentice Hall, 2004. [3] I. Stoica, Robert Morris, David Karger, M. Frans Kaashoek, and Hari Balakrishnan, “Chord: a scalable peer-to-peer lookup service for Internet applications”, in the Proceedings of ACM SIGCOMM 2001, San Deigo, CA, August 2001. [4] RFC 3174. http://www.faqs.org/rfcs/rfc3174.html. [5] S. Friedman, N. Leidenfrost, B. Brodie and R. Cytron, “Hashtables for embedded and real-time systems”, in the Proceedings of IEEE Real-time Embedded System Workshop, Dec. 2001. [6] T. H. Cormen, Charles E. Leiserson, and Ronald L. Rivest, “Introduction to Algorithms”, the MIT Press, 1989. [7] B. Carlson, Packets. “challenge next-gen nets”, EETimes, August 2002, http://www.eetimes.com/story/OEG20020802S0033. [8] D. Knuth, “The Art of Computer Programming”, Vol. 3, Searching and Sorting, Reading, MA: Addison-Wesley, 1973. [9] T. Szymanski, “Hash table reorganization”, Journal of Algorithms, 6(3), 1985, pp. 322-335. [10] M. Dietzfelbinger and F. Meyer auf der Hyde, “An optimal parallel dictionary”, in the Prococeeding of ACM Symposium on Parallel Algorithms and Architectures, 1989, pp. 360-368. [11] M. Ajtai, M. Fredman, and J. Komlós, “Hash functions for priority queues”, in the 24th Annual Symposium on Foundations of Computer Science, pp. 299-303, Tucson, Arizona, 7-9 November 1983.
