Tuning Databases for High Performance DENNIS SHASHA New York University and AT&T Bell Laboratories ^
[email protected]&
In fields ranging from arbitrage to tactical missile defense, speed of access to data can determine success or failure. Database tuning is the activity of making a database system run faster. Like optimization activities in other areas of computer science and engineering, database tuning must work within the constraints of its underlying technology. Just as compiler optimizers, for example, cannot directly change the underlying hardware, database tuners cannot change the underlying database management system.1 The tuner can, however, modify table design, select new indices, rearrange transactions, tamper with the operating system, or buy hardware. The goals are to eliminate bottlenecks, decrease the number of accesses to disks, and guarantee response time, at least in a statistical sense. Understanding how to do this well requires deep knowledge of the interaction among the different components of a database management system. This short article illustrates the interaction among just three components: concurrent transaction design, indices, and table design. The goal is to give an appreciation for the challenges and methods of this field of optimization. TUNING TRANSACTIONS
Concurrency control algorithms in database systems attempt to give users the illusion that they are using the data1
In both cases, optimization ideas influence future incarnations of the underlying technology— witness VLIW processors and new database optimizers.
base in isolation from anyone else. Full isolation or serializability is the guarantee that each transaction will appear to execute one at a time as far as its input/ output behavior is concerned. The concurrency control algorithm in predominant use is two-phase locking, comprising two lock types: read (or shared) and write (or exclusive) locks. Two transactions may both hold a shared lock on a data item. If one transaction holds an exclusive lock on a data item, however, then no other transaction may hold any lock on that data item; in this case, the two transactions are said to conflict. Once a transaction obtains a lock, it holds it until it completes or aborts. Tuning concurrency control entails trying to reduce the number and duration of conflicts. Consider, for example, the following code for a purchase application of item i for price p for a company in bankruptcy (for which the cash cannot go below 0). (1) begin transaction (2) if cash , p then roll back transaction (3) inventory (i) :5 inventory(i) 1 p (4) cash :5 cash 2 p (5) commit transaction This code is correct semantically. For example, if the cash remaining is 100 and purchase P1 has item i with price 50 and purchase P2 has item j with price 75, then one of these will roll back. Unfortunately, this transaction design yields poor performance, because every transaction must acquire an exclusive lock on cash from the beginning to avoid deadlock. That will make cash
Copyright © 1996, CRC Press.
ACM Computing Surveys, Vol. 28, No. 1, March 1996
114
•
Dennis Shasha
a serial bottleneck. Because the cash lock will be held during the inventory access and because that access may entail a disk access requiring about 20 milliseconds, throughput will be limited to approximately 50 purchase transactions per second. Even a company in bankruptcy may find this unacceptable. A surprisingly simple rearrangement helps matters greatly. (1) begin transaction (2) inventory (i) :5 inventory(i) 1 p (3) if cash , p then roll back transaction else cash :5 cash 2 p (4) commit transaction Cash is still a hot spot, but now each transaction avoids holding cash while accessing inventory. Because cash is so hot, it will be in the RAM buffer. The lock on cash can be released as soon as the commit occurs. Other techniques are available that “chop” transactions into independent pieces to shorten lock times further [Shasha 1992]. Locks and Data Structures. When teaching databases, most instructors discuss data structures in the absence of concurrency considerations and then discuss concurrency control in the absence of data structure considerations. The two are closely related, however, as the following scenario illustrates. A sequential key is an attribute or set of attributes that are monotonic with time. A typical sequential key is a timestamp or a sequential counter. In a Btree based on a sequential key all inserts go down the same path (rightmost pointers all the way). Because these inserts must exclusively lock the leaf node, they serialize on that node if the inserts come from different transactions. Systems that use record-level locking (e.g., Oracle 7.x) hold such node locks only for a short time (as “latches”), whereas systems with page-level locking (e.g., Sybase System 10) hold such locks until the end of the transaction. In either case, but especially in the pagelevel case, this serialization could seACM Computing Surveys, Vol. 28, No. 1, March 1996
verely hurt performance. Thus B-trees are a bad idea for sequential keys if there is heavy insert traffic and the inserts come from different transactions. By contrast, a hash structure gives good concurrent performance for sequential keys because hashing randomizes the placement of consecutive keys. Tuning Table Design. Table design is the activity of deciding which attributes should appear in which tables in a relational system. Consider a bank whose Account relation has the normalized schema (account id is the key): —Account(account id, balance, name, street, postal code) Consider now this alternative normalized design: —AccountBal(account id, balance) —AccountLoc(account id, name, street, postal code) The second schema results from vertical partitioning of the first (all non-key attributes are partitioned). The second schema has the following benefits for simple account update transactions that access only the id and the balance: —A sparse clustering index on account id of AccountBal may be a level shorter than it would be for Account relation. The reason is that the leaves of the data structure in a sparse index point to data pages. If AccountBal has far fewer pages than the original table, as is likely in this case, then there will be far fewer leaves in the data structure. —More account id-balance pairs fit in memory, thus increasing the hit ratio. Again, the gain is big if AccountBal tuples are much smaller than Account tuples. —Such a partitioning allows the two tables to be physically distributed, perhaps eliminating a bottleneck when the monthly statements have to go out.
Tuning Databases for High Performance Thus the conceptual design chosen may be determined by a combination of the index design choices, the size of fields, and load balancing considerations. Indexing may depend on concurrency situations. And concurrency bottlenecks may be eliminated by better transaction design. Tuning is for generalists. SUMMARY AND RESEARCH RESULTS
Various research and commercial efforts have attempted to automate tuning. Given information about table sizes and access patterns, DEC’s RDB Expert can give advice about index selection, load balancing, and data compression based on system activity. Given similar information, the AT&T Teradata Data Navigator’s Configurator chooses indices and distributes data across different processing nodes in a parallel data server. Human expertise then comes into play only when deep application knowledge is necessary, for example, in the determination of locking isolation levels or in rewriting queries or table designs, or when these tools do not work as advertised (the problems are all NP-complete). DEC and its customers, for example, continued to employ high-priced human tuners even after RDB Expert appeared. The COMFORT project [Weikum et
•
115
al. 1994] at ETH in Zurich has studied lower-level automatic tuning strategies that could (and should) be incorporated directly into a database management system. Using data from Swiss banks as well as synthesized data, the project has proposed paging strategies (in cooperation with Pat and Elizabeth O’Neil) and load-control strategies that demonstrably improve performance. Technological advances such as parallelism and object-orientation add a new dimension to the tuning problem, sometimes making it easier and sometimes harder. Healing a slow database may not require a good bedside manner, but good tuning may earn one the ephemeral gratitude of its many users. REFERENCES COMER, D. 1979. The ubiquitous B-tree. ACM Comput. Surv. 11, 121–137. COPELAND, G., ALEXANDER, W., BOUGHERTY, E., AND KELLER, T. 1988. Data placement in Bubba. In Proceedings of the ACM SIGMOD Conference (May) 99 –108. GRAY, J. AND REUTER, A. 1993. Transaction Processing: Concepts and Techniques. MorganKaufmann Publishers, San Mateo, CA. SHASHA, D. 1992. Database Tuning: a Principled Approach. Prentice-Hall, Englewood Cliffs, NJ. WEIKUM, G., HASSE, C., MOENKEBERG, A., AND ZABBACK, P. 1994. The COMFORT Automatic Tuning Project. Inf. Syst. 19, 5, 381– 432.
ACM Computing Surveys, Vol. 28, No. 1, March 1996
