Autonomic Features of the IBM DB2 Universal Database ... - IEEE Xplore

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART C: APPLICATIONS AND REVIEWS, VOL. 36, NO. 3, MAY 2006

365

Autonomic Features of the IBM DB2 Universal Database for Linux, UNIX, and Windows Christian M. Garcia-Arellano, Sam S. Lightstone, Guy M. Lohman, Volker Markl, and Adam J. Storm

Abstract—The vast majority of the world’s structured data are now stored and managed by relational database management systems (RDBMSs). These systems provide powerful data management capabilities. However, as the power and functionality of these systems has grown, so has the complexity of their administration. In this paper, we show how the IBM DB2 Universal Database for Linux, UNIX, and Windows (DB2 UDB) product has exploited autonomic computing to reduce this administration complexity and become more self-managing. We survey the major autonomic computing features in the DB2 UDB product and describe the benefits, with experimental data in some cases. Index Terms—Automation, autonomic computing, database, DB2, IBM, physical database design, self-managing, tuning, workload.

I. INTRODUCTION INCE the development of the relational model by E.F. Codd at IBM [1], relational databases have become the de facto standard for managing and querying structured data. The rise of the Internet, online transaction processing, online banking, and the ability to connect heterogeneous systems have all contributed to the massive growth in data volumes over the past 15 years. Terabyte-sized databases have become commonplace. Concurrent with this data growth have come dramatic increases in CPU performance, spurred by Moore’s Law, and improvements in disk technology that have dramatically improved data density for disk storage. To help manage the massive volume of data, and exploit the available hardware, modern relational database management systems (RDBMSs) have been enhanced with powerful features to exploit parallel processing, novel data structures, new types of data indexes, and algorithms for managing system workloads. These features are critical to today’s systems, which have hundreds of disks and CPUs, massive volumes of data, and are often required to run 24 h a day, 7 days a week (24 × 7).

S

Manuscript received October 15, 2004; revised April 27, 2005. DB2, DB2 Universal Database, and IBM are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both UNIX is a registered trademark of the Open Group in the United States and other countries. Linux is a trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others. The views expressed in this paper are those of the authors. This paper was recommended by Guest Editors R. Sterritt and T. Bapty. C. M. Garcia-Arellano, S. S. Lightstone, and A. J. Storm are with the IBM Toronto Laboratory, Markham, ON, L6G 1C7 Canada (e-mail: cmgarcia@ca. ibm.com; [email protected]; [email protected]). G. M. Lohman and V. Markl are with the IBM Almaden Research Center, San Jose, CA 95120 USA (e-mail: [email protected]; marklv@us. ibm.com). Digital Object Identifier 10.1109/TSMCC.2006.871572

These powerful capabilities of modern RDBMSs also add complexity, however, and necessitate an entire professional domain of experts, known as database administrators (DBAs), to manage these systems. The complexity problem is not unique to RDBMSs, of course, but is ubiquitous in information technology (IT) today. Some have called this explosive growth of complexity “creeping featurism.” In fact, many companies now spend more money on DBAs to manage their IT systems than they do on the systems themselves. With growing complexity, this model will not survive. The industry is already reaching a breaking point where human DBAs, regardless of their talent and skill, will soon be unable to manage the complexity of these modern systems. The only viable strategic solution in the long term is to develop systems that manage themselves. Such systems are called autonomic systems. The term autonomic computing was coined by IBM Research [2] to draw an analogy to the human body’s autonomic nervous system, which allows the body to adapt to new environments and challenges without any conscious knowledge or effort. Modern IT systems must manage themselves, as the human body does, or they risk being crushed under their own complexity. The acute problem of system complexity has prompted considerable research and development in self-managing, or autonomic, systems by the major IT system vendors. In particular, all the major RDBMS vendors are now focused on selfmanaging features to lower the total cost of ownership (TCO) caused by exponentially increasing system complexity [3]. II. DB2 UDB AUTONOMIC COMPUTING INITIATIVE Observing the trend in increased complexity of RDBMSs, IBM began a significant research and development effort to infuse the DB2 UDB product with more autonomic technology. The result of this effort is that the DB2 UDB for Linux, UNIX, and Windows V8.2 product [6] now boasts some of the best autonomic technology in the industry, which we will describe. To illustrate the scope of the problem space, we describe the database administration responsibilities in two ways, first as a set of operational requirements and secondly as a timeline of ownership. The operational requirements, as illustrated in Fig. 1, are divided into four simple categories: self-configuring, self-healing, self-optimizing, and self-protecting (self-CHOP). Self-configuring refers to function that connects and configures components so that technology is operational with minimal human intervention. For database servers, this can include server and storage topology configuration, installation, initial database design, configuration of the database server (there are several dozen possible configuration options), and physical database design. Self-configuring also includes the ability for a database

1094-6977/$20.00 © 2006 IEEE

366

Fig. 1.


Requirements for database administrators.

server to be easily deployed, configured, and interconnected with other middleware and applications. Self-healing refers to the ability of a system to detect problems or problem trends (which have not yet manifested as problems) and either take corrective action or provide advice and automated notification to an end user, who can then take corrective action. Even with all the sophistication and redundancy in database systems, things can and will occasionally fail, which makes the automated problem determination aspect of selfhealing a critical domain as well. In automated problem determination, an autonomic system detects problem trends (such as disk storage filling up at a detectable rate, from which a storage constraint can be projected) or detects and notifies the DBA when sudden errors arise. Self-optimizing is the domain of system performance tuning. For databases, this includes compiling queries so that they perform well when run, tuning utilities so that they maximize the available resources, and throttling background processes to avoid undesirable impact to the production workload. Selfoptimizing also includes the daily maintenance of the database, which, if neglected, will generally lead to system performance degradation or storage waste. This includes, for example, regular backups, collection of data statistics, and data defragmentation and reclustering. The final functional category is Self-protecting, which describes a system’s ability to anticipate, identify, and protect against security attacks. The scope varies from static defenses such as virus protection and data encryption to more advanced dynamic defenses such as behavior analysis, where the actions taken by the users accessing the RDBMS are analyzed for negative behaviors, tracked, and adaptively unwound. The DB2 UDB product’s self-protecting capabilities are currently exclusively static defenses, as described further, which is consistent with the self-protecting capabilities of all major industrial RDBMSs today. The administration domain can also be described as a timeline of ownership, as illustrated in Fig. 2. We describe the timeline of ownership as having five stages. The ownership process begins with an assessment of requirements for capital investment and capacity planning. In the next phase, the RDBMS is designed. Thirdly, the database is constructed and tuned. Fourthly, the system goes into production, in which continuous tuning and object administration takes place. Finally, in the fifth stage,

changes are made affecting the application or the database server directly. Although IBM’s vision of autonomic systems is to ultimately make these systems completely free of human interaction, in the next several years there are two critical dynamics at play: 1) trust building, as IBM’s autonomic technology begins to earn the trust of DBAs and CIOs alike and 2) technology building, as IBM fleshes out the autonomic features and capabilities of its products. During this time period, which will last for several years, clear reporting interfaces are critical so that human beings can assess what actions autonomic engines within the DB2 UDB product have been taking. Allowing DBAs to disable any given autonomic feature is one of the DB2 UDB product’s philosophies in the introduction of new autonomic features. This keeps ultimate control of the system in the hands of the DBA and allows them to selectively exploit features they are comfortable using. Last but not least, the ultimate goal of automated systems administration is to not only create piece-wise autonomic components (like an autonomic database management system) but also enable large complex stacks of hardware and software to truly cooperate in a comprehensive autonomic system. This cooperative notion can only be achieved through the development of standards and multicomponent technology. IBM’s strategy is to contribute to the definition of open standards for intercomponent collaborative automated administration, so that components can be combined from multiple vendors. As an example, IBM has been working through the Organization for the Advancement of Structured Information Standards (OASIS) [4] to extend the Web Services Distributed Management (WSDM) standards to include structures for administrative event notifications (common base events or CBEs). As a company that produces a wide array of hardware and middleware, IBM is uniquely positioned to enable such standards across a complete solution stack of products in the next several years. In Sections III–VI, we describe the major capabilities in the IBM DB2 UDB V8.2 product in the four self-CHOP categories. Then, in Section VII, we describe some of the ways in which the DB2 UDB product is able to report on its own autonomic behavior and we also touch on how the product integrates into the larger IBM autonomic middleware stack. Finally, in Section VII, we discuss the future vision for autonomic work in the DB2 UDB product.

III. SELF-CONFIGURING AN RDBMS such as DB2 UDB is a very complex system that is engineered to satisfy many purposes and work in a great variety of environments. For example, the RDBMS has to be able to support many types of workloads, integrate with a large variety of other software, and run on a large variety of hardware platforms and operating systems. Additionally, the RDBMS must be able to perform well under the differing amounts of computational resources that may be available for each new installation of the system. In this section, we introduce some of the selfconfiguring capabilities of the product DB2 UDB V8.2.

GARCIA-ARELLANO et al.: AUTONOMIC FEATURES OF THE IBM DB2 UNIVERSAL DATABASE

Fig. 2.

367

Administrative timeline of ownership.

A. Configuration Advisor During the developmental evolution of the DB2 UDB product, many configuration parameters were added to allow the DBA to fine-tune the system to satisfy this large variety of needs. The sum of all these configuration parameters gives great flexibility to the product and gives each customer the opportunity to tune the system for his own specific needs. All this flexibility, however, also added complexity to the system, which increased the TCO of the product. This TCO increase was mostly a result of the fact that companies had to rely more and more on experts that were completely dedicated to configuring and maintaining each installation of the product. The first step taken in the DB2 UDB product toward the ideal of a completely self-configuring system was the Configuration Advisor 1 [11]—a utility that is able to automatically recommend the best values for a large portion of the configuration parameters in the DB2 UDB product that are critical to performance, including those that control the memory distribution, parallelism, I/O optimization, and recovery. This utility bases its recommendations on the responses to seven basic questions about the workload characteristics. The answers to these questions do not require the DBA to have an in-depth knowledge of the system being tuned, because the utility is also able to automatically sense the available hardware resources and database characteristics. The input to the advisor is then combined with a set of mathematical expressions based on expert heuristics to define the estimated ideal settings for the configuration parameters. These rules derived from DB2 experts were collected through a detailed process of surveying a large number of experienced DBAs and leading performance experts on the DB2 UDB product. The mathematical models were refined by a process of extensive internal experimentation at IBM and the formulae were analyzed over ranges of the input parameters to determine how they would extrapolate in small and large systems. The heuristics can often lead to performance that can outperform a hand-tuned system, and generating the configuration requires only a few seconds instead of requiring a time-consuming trial-and-error process. In Fig. 3, we show the performance of the Configuration Advisor on three different workloads. On the first two systems, OLTP 32-bit and OLTP 64-bit, we show that the Configuration Advisor achieves performance levels within 10% of a system tuned by human experts after only seconds of tuning. This result 1 The Configuration Advisor was first released in the DB2 UDB V6 product under the name of Configuration Smart Guide. Several improvements were made to it in the V8 release of the product.

Fig. 3. Performance of three systems each first tuned by hand and then by the configuration advisor.

is significant because the human-tuned systems require weeks of tuning in many cases. Also, this result shows the Advisor’s ability to perform well under different hardware platforms. The third set of results shows how the Configuration Advisor is able to tune an industrial workload. In this test, the Advisor was run against a system that was first tuned by a skilled DBA. After the Advisor was run, the system was tested with the new configuration and performed more than 5% better. This shows how valuable the Advisor can be in real applications, where it can save weeks of tuning and still tune the system as well or better than hand tuning. B. Dynamic Configuration Parameters A second major step towards the ideal of a self-configuring system was first released in the DB2 UDB V8.1 product release. In this release, many of the configuration parameters that, in previous releases, had to be modified offline were made dynamically adjustable. In other words, these configuration changes no longer required stopping and restarting the database manager and thus could be changed without taking the system offline. This gave more flexibility to the administrators of the RDBMS, because now they could adapt the product configuration to changes in the workload or the environment without incurring system downtime. A typical example of a varying workload is a system with two vastly different uses during the same 24-h period. In the first period, online transaction processing (OLTP) occurs during the

368


hours of 8 A.M. to 9 P.M. to process customer transactions. Once the business day concludes, the system is used for decision support (DSS) to process and analyze the results of the daily operations. The problem with this system is that the optimal values for some of the configuration parameters are different for the two stages. For example, in the DSS portion of the day, sorting of data may be more prevalent and as a result the system will perform better if more memory is allocated for sorting. With the combination of the recommendations made by the Configuration Advisor and the dynamically adjustable configuration parameters, the administrator of the RDBMS is now able to change the distribution of the memory without stopping the database manager; thus allowing the company to maintain the database continuously operational (24 × 7). C. Simplified Memory Management The memory configuration is one of the most performancecritical aspects of a database system. One key enhancement was introduced in the DB2 UDB V8.2 product release to build on the dynamically updatable memory configuration. For each of the memory consumers (sorting, data buffering, etc.) in the product there is a configuration parameter that determines the maximum amount of memory that the consumer may use. In previous versions of the DB2 UDB product and historically in all other database products, if a memory consumer uses all of its memory, future requests for memory by that consumer will fail. The simplified memory management changes this behavior. With the new simplified memory management changes, when a memory consumer uses all of its memory, it is able to overflow into a new area of memory called the overflow buffer, which is used for overflow purposes only. The overflow buffer allows the consumer to satisfy its memory allocation without requiring a configuration parameter change. As a result, the overflow buffer reduces the possibility of system failure as a result of a poor memory configuration. This simplification of memory management is particularly useful for the transition times when a workload is changing its characteristics but the configuration of the database has not yet been updated. The overflow buffer will accommodate the new demand until the memory consumer’s size is dynamically updated. This gives the user a longer window to make the configuration change without impacting the workload performance. D. Design Advisor Another important aspect of the configuration of the database system is the physical database design. The definition of the physical database design is crucial in all database systems, because the difference between a good and bad design could mean the difference in performance of several orders of magnitude. The physical design of a database includes the definition of auxiliary objects, such as indexes and materialized query tables (MQTs) 2 , which provide faster access to the data and consequently faster response to queries. The design also includes the 2 MQTs

are stored and maintained query results. They are also known as materialized views.

definition of how the data are physically organized inside a table, such as the definition of a clustering key to determine the physical order of the rows or a partitioning key in a shared-nothing 3 multipartition database, to define what data should be stored in which partition or if the whole table should be replicated to all the partitions. The only way to tackle this configuration problem in the past was to painstakingly try different physical database designs by time-consuming iterative trial and error. For example, when determining which indexes to create, a DBA would have to guess what index might be beneficial for a given workload, create the index (which required sorting all the keys), collect statistics on the index, rebind all the queries, and determine whether the workload performance improved because of that index or not. For large databases, each such iteration could take minutes or even hours to accomplish. The first step to solving this tremendously difficult problem was the Index Advisor, which was released in Version 6.1 of DB2 UDB [20]. The Index Advisor recommends the best set of indexes to minimize the time to execute a given workload. Version 8.2 of DB2 UDB takes several more steps in this direction by extending the Index Advisor to recommend much more than just indexes. The DB2 Design Advisor [10] is the first industrial-strength utility to determine automatically any combination of the four most important physical database design aspects, including indexes, MQTs [8], partitioning keys [13], and multidimensional clustering (MDC) keys [9]. The utility takes as input the workload and any disk space constraints and automatically senses the system characteristics and the data characteristics, via stored database statistics and sampling of the database. The workload can include both read-only queries and update statements, and for each query the utility can benefit from knowing the frequency of occurrence of each statement, which can be automatically determined from the internal query cache of the database. The Design Advisor uses a hybrid approach to combine the recommendations for each of the four aspects. This hybrid approach was engineered to take into account the interdependencies of the design aspects and satisfy the space and time constraints while providing a near-optimal recommendation in the allotted time budget. For example, an MQT, being a stored table, may benefit from additional indexes and will require some partitioning in a partitioned environment. But those indexes also compete with the MQT for disk space. These interdependencies compound the already huge search space of possible solutions for each aspect individually [8], [9], [13]. The Design Advisor exploits some of the existing technologies in the DB2 UDB product to help it produce the best recommendations. The technologies utilized include the following: 1) the cost-based query optimizer, which is used both to recommend likely candidate objects and to do “what if” analysis of the alternative combinations of design objects [20]; 2) the multiquery optimization (MQO) feature originally developed to combine refreshes of MQTs, which is used 3 Shared-nothing architecture consists of a multiprocessor computer system in which disk and memory are not shared by the processors and where communication between processors is done by passing messages.


Fig. 5.

Fig. 4.

Experimental results using the design advisor.

to find commonalities between the supplied query set for choosing MQTs; 3) statistical sampling, which helps the MDC selection feature to better determine cardinalities and correlations for more accurate results, and the MQT selection feature to better estimate the size of a candidate MQT. The Design Advisor also employs innovative compression technology to reduce the workload of queries that was originally supplied into a smaller set of representative queries, which can then be used to derive recommendations. This compression allows the advisor to produce its recommendations in significantly less time than if no compression were used, with minimal reduction in the quality of the design it recommends. An example of the Design Advisor’s value is shown in Fig. 4, which shows experimental results obtained using a database with a TPC-H schema stored on a multinode system and running 22 decision-support queries. The baseline for this experiment was derived from the physical database design of a published TPC-H benchmark, with some small alterations to simulate the design that an average DBA might use. In these experiments, the Design Advisor was able to produce in about 10 min a solution that recommended 20 new indexes, six MDC dimensions, four partitioning changes, and two new materialized views. 4 The Design Advisor estimated that this new recommended physical database design was going to result in an 88.01% improvement in the performance, and the actual improvement after applying the new design was 84.54%, showing that the recommended benefit was accurate. These results illustrate the Design Advisor’s power to reduce several weeks of tuning effort to only 10 min of computation and still achieve significant improvement. E. Simplified Storage Administration DB2 UDB uses an abstraction called table spaces, which are logical groupings of database storage objects (tables, indexes, 4 Note that these recommendations cannot be used in a real TPC-H benchmark because they violate some of the publication rules. For example, the TPC-H benchmark rules do not allow the use of MQTs. In addition, the recommendations were made not taking into account the update operations that are part of the TPC-H benchmark.

369

Database storage model.

etc.) The abstraction is very important as a way to group objects for administration. The DB2 UDB V8.2 product allows DBAs to provide a set of storage paths (i.e., directories in a file system) to the database without a specification of how these storage paths should be allocated to individual table spaces. The DB2 UDB product’s autonomic technology then creates database storage constructs within these user-specified paths for storing database objects. These storage paths and the database storage constructs created on them are then adaptively used and expanded by the database in real time to adapt to database server load and storage volume. Fig. 5 illustrates the conceptual model, showing the abstraction between physical storage paths and logical groups of objects. The automation of storage configuration and dynamic growth of the storage constructs radically reduces the complexity and skill requirement for the DBA. The complexity of this problem being solved is nontrivial, with a significant number of industrial databases now having thousands of database storage objects, and terabytes of real data, in addition to data required for internal auxiliary structures such as indexes and MQTs. IV. SELF-HEALING There are four main features that were developed to make the DB2 UDB product a truly self-healing system that is capable of detecting problems and healing itself. Next we will describe the characteristics of these technologies. A. Health Monitor RDBMSs are getting more and more complex every day, and consequently, the elements to monitor are growing in number and complexity. This complexity makes detecting and understanding how to solve unhealthy conditions difficult. The Health Monitor (HMON) was created to minimize this complexity explosion by automatically monitoring the health of the RDBMS. When in use, the HMON can drastically reduce the amount of time that a user spends on monitoring the health of the RDBMS. The HMON works by continually sensing the state of the different components of the database manager and active databases. It then evaluates a number of predefined health indicators to determine their status relative to the planned or expected behavior. The predefined health indicators cover a wide range of unhealthy conditions such as those related to the health of various database objects, memory configuration, maintenance tasks, logging for

370


recovery or application concurrency. If an unhealthy condition is about to happen or has already occurred, the HMON raises an alarm. After an alarm is raised, the user can invoke the Recommendation Advisor, which will recommend the paths that need to be taken immediately to resolve the unhealthy condition and to avoid it in the future. From the HMON itself, the user can take immediate corrective action by updating the values of the dynamically updatable configuration parameters introduced in DB2 UDB Version 8.1. The following three types of health indicators are currently supported by the HMON. 1) State-based indicators, with a number of predefined acceptable and unacceptable states. For example, it can calculate whether a database backup is required based on the time elapsed and the amount of data changed since the last backup. 2) Collection state-based indicators, which summarize the state of multiple state-based indicators. For example, it can calculate whether the statistics are up-to-date for all tables. 3) Threshold-based indicators, which evaluate a formula based on sensed data. For example, it can calculate the log space utilization level for the database. Each of the health indicators supports different alert levels and the user has the ability to select between several different reporting mechanisms. These reporting mechanisms allow the user to specify how to be contacted if an alert is generated. The reporting mechanisms include having a user notified by e-mail or pager. B. High-Availability Disaster Recovery High-availability disaster recovery (HADR) is a requirement in all business-critical systems because of the serious consequences of not being able to process transactions and the high cost of losing data. The simplest design of an HADR system includes two sites with two identical copies of the database. In the case of a total or partial failure in one of the sites, the other can take over immediately and automatically. The two sites are most commonly in geographically separate locations so that local disasters do not destroy both sites. The first, or primary, location contains the source copy of the database, while the second, or standby, location contains the secondary copy of the database. The technology for HADR should allow the standby system to take over in case of a disaster that affects the availability of the primary system. The DB2 UDB product made major improvements in the HADR technology in V8.2 to allow automatic management of the primary and standby systems, and the technology to automatically reroute all clients to the secondary site when the primary system is unavailable. Transaction logs are a technology that has been used for years in RDBMSs to protect the consistency of the data from failures. The DB2 UDB product keeps the secondary copy of the database synchronized with the first copy by transferring these transaction logs via a TCP/IP network. The standby database continually applies the logs received from the primary site to keep the two copies of the database synchro-

nized. When a client encounters an outage in the primary copy of the database, it will automatically attempt to establish a connection with the standby site. The rerouting technology in the DB2 UDB product will transparently and automatically move all active connections from the primary to the standby server, while maintaining database consistency. C. Other Self-Healing Features The DB2 UDB V8.2 product includes two additional selfhealing features that are worth mentioning: the Fault Monitor and the new Recover utility. The Fault Monitor keeps unclustered DB2 UDB product installations up and running 24 × 7 by detecting and recovering from failures in the same server. The Fault Monitor facility is a set of processes that monitor the availability of multiple instances of the DB2 UDB product. When a Fault Monitor detects that one of the instances is not active, it will automatically restart the instance to make it operational. The new Recover utility allows the user to recover after a failure by simply indicating any point in time to which to recover the database or by indicating that the database should be recovered to the end of the log files. The automation logic within this utility automatically determines the optimal recovery strategy, given the available backup images and transaction logs, and decides which backup image should be restored and which logs should be replayed to minimize recovery time. V. SELF-OPTIMIZING There are several features in the DB2 UDB V8.2 product that serve to optimize system performance, while requiring little or no user intervention. In this section, we describe these features and note how they serve to relieve the administrator of the RDBMS of tedious optimization tasks. A. Query Optimizer An SQL query is a declarative specification of what data are needed, not how to access the data. For a given query, there are typically many ways to compute the final answer set, each having a different cost. The goal of the query optimizer is to find the query plan with the lowest estimated cost that will produce the desired answer set. Deriving query plans is highly nontrivial, since queries referencing only a handful of tables could have thousands of possible query plans and differing plans can have costs that differ by several orders of magnitude. One could argue that the query optimizer was the first autonomic feature in the database domain. Database systems prior to relational databases required experts to understand how the data were structured and specify explicitly how to navigate through those structures, making new database queries prohibitively expensive to develop. As a result, when IBM was developing the first relational database, it spent a considerable amount of time researching query optimization and proposed the first cost-based query optimizer in 1979 [14], [15]. This technology has been continually improved to what exists today in the DB2 UDB product.


371

termining how often they are being modified. The modification pattern allows the feature to then schedule statistics updates so that the statistics never stray too far from reality. C. Automated Statistics Profiling

Fig. 6.

Response time of queries with and without proper statistics.

The query optimizer in the DB2 UDB product is able to determine the best execution plan for even the most complicated decision support queries. The optimizer has two main components: the query rewrite engine, which transforms the original query into an equivalent form that will perform better and is easier to optimize and the access plan optimizer, which estimates the cost of many different query plans before choosing the best plan for the given query. These two pieces allow for queries to execute efficiently without requiring an expert user. B. Automatic Statistics Collection Determining a query execution plan requires the correct set of information, in the form of detailed statistics on the tables being accessed for a given query. These statistics encapsulate important characteristics of a table, such as the number of rows in the table, the average size of each row, and distribution of the data values. With a good set of statistics, the optimizer can find the optimal query plan. Complications can arise, however, if the statistics become out-of-date, which can happen in the presence of updates to the database. Fig. 6 shows the importance of keeping statistics current. In these experiments, queries were run first without the proper statistics and then a second time with a proper set of statistics. The most dramatic impact of the change in statistics is seen in queries 13 and 15, where the queries run more than 50 times slower than when they have the proper statistics. While keeping the database statistics current is obviously important, the task of keeping them current has, in the past, fallen to the administrator of the RDBMS. This has been changed in the DB2 UDB V8.2 product through the creation of a feature called automatic statistics collection [17]. Automatic statistics collection automatically detects which tables require their statistics to be updated and then schedules the statistics updating jobs. If the administrators of the RDBMS want more control, automatic statistics collection allows them to specify which tables require updated statistics and the times of the day that are more amenable to statistics collection. The feature works by monitoring the tables in the database and de-

While determining when statistics require updating is important, it is equally important that when the statistics are updated, the right set of statistics is collected. By analogy, when traveling on a road trip from New York to Los Angeles, one may need detailed information on Route 66 and the interesting sights along the way, but it is unlikely that information on the restaurants in Florida would be useful. Configuration and maintenance of statistics has traditionally been a time-consuming manual operation, requiring that the DBA continually monitor query performance and data changes in order to determine when to refresh the statistics values and when and how to adjust the set of statistics that the RDBMS maintains. Automatic statistics profiling [17] adds a reactive feedback loop to DB2 UDB, which monitors query executions and validates the statistics used during query compilation against actual information observed when executing the query. The feature then suggests modifications to the statistics based on the discrepancy between the statistics and the actual information in order to collect the right statistics at the right level of granularity. Specifically, automatic statistics profiling suggests the number of frequent values used to approximate the distribution of a column and whether to create column group statistics on certain pairs of columns. Automatic statistics profiling thus, over time, learns the right set of statistics for any workload, improving query execution over time. Experiments with automatic statistics profiling show its ability to improve query execution time by orders of magnitude through the recommendation and implementation of proper column group statistics, without any intervention by a DBA [17]. D. Automated Database Backups The data stored in the database may become unusable because of a hardware or software failure. For this reason, the periodic copy of the data stored in the database to a safe location is a basic requirement for any database system. The DB2 UDB V8.2 product allows the administrator of the RDBMS to enable the automatic execution of a periodic backup copy of the data by just specifying the location for this safe copy. In addition to this, the administrator of the RDBMS can specify thresholds on the number of elapsed days or the number of changes to the data since the last backup to indicate to the autonomic system when it is more appropriate to schedule this maintenance task. E. Automatic Scheduling of Maintenance Tasks Some maintenance tasks have to be periodically executed in order to maintain the system in optimal working condition. One example of these tasks is the backup of the database. The data stored in the database may become unusable because of a hardware or software failure, and a periodic copy of the data stored in the database to a safe location is a basic requirement

372


for any database system. In the DB2 UDB V8.2 product, the automatic execution of a periodic backup copy of the data can be enabled by just specifying the location for the storage of the backup copy. The autonomic system will take into account thresholds of data modification and elapsed times to decide when the next execution of the maintenance task is required and will automatically execute the task. Other maintenance tasks like table and index reorganizations (to provide better disk access properties) cannot be performed online because they require exclusive access to the database object and require the existence of a maintenance window. These maintenance windows are only going to restrict the access to the database objects being affected by the maintenance operation during the time the maintenance operation is taking place. Devising an efficient schedule for a set of tasks is difficult because it may not be clear at the outset how long a specific task will take to run. In the DB2 UDB V8.2 product, all maintenance tasks can be automatically scheduled taking execution times into account. The feature allows the administrator of the RDBMS to specify a window in which maintenance tasks can run and the set of tasks to run in the window. The first few times the tasks are run, they are monitored. Gradually, the system learns the characteristics of the tasks and can accurately estimate how long it will take them to run. If the maintenance window specified is not long enough for the utilities to run, then the DBA is notified and can decide either to extend the maintenance window or to reduce the number of utilities to run. Additionally, the feature eliminates the need for customers to write custom scripts to launch utilities during maintenance windows. F. Utilities Throttling In production database systems, utilities such as backups and loading of data are resource intensive and, if not regulated, can cause the production workload to slow down considerably. To avoid slowing down the system, DBAs have in the past designated windows of relative system inactivity in which utilities can be run with little impact to the workload. The trouble with this model is that increasingly customers are finding that these windows of inactivity are shrinking as their businesses need to function 24 h a day in response to customers throughout the world. The 24-h nature of modern business has made it infeasible to schedule utilities based on system inactivity. In response to this need, the DB2 UDB product added utilities throttling. The utilities throttling feature allows the DBA to specify an impact policy. This impact policy specifies the amount that the utility is allowed to impact the workload. Once the utility is started, the workload is monitored with and without the utility running. This monitoring is used to determine how much the utility is slowing down the workload. Once the system is aware of the impact that the utility has on the workload, it slows down the utility so that it will not impact the workload more than the specified impact policy [16]. For example, assume that the DBA has set the impact policy to 50%. This means that the utility is able to impact the workload by at most 50%. If, at the same time that the workload is started,

Fig. 7. Workload performance when running no utility, a throttled utility, and an unthrottled utility.

the system is processing 100 transactions/min, the utilities throttling feature will ensure that the rate of transactions per minute does not fall below 50, provided that the workload demands are maintained. Throughout the life of the utility, the system is being monitored so that, if at some point no new transactions are arriving, the utility will be given 100% of the resources. In Fig. 7 the effect of throttling utilities is shown. In the baseline run, the workload runs on the machine by itself. Then, in a second run, the workload is run again, but a throttled utility is started after 400 s of running and once the system’s performance is relatively stable. After approximately 75 s (at time 475 on the chart), the utility throttling recognizes that the workload is being significantly impacted and scales back the utility. By 550 s, the utility is only impacting the workload at the level specified by the impact limit (in this case 10%). When this “throttled” run is compared to the “unthrottled” run, it is clear that without utility throttling the workload would have been significantly impacted by the utility. The utility throttling feature frees the DBA from the burden of monitoring the system for appropriate times to run utilities. Additionally, the feature provides the DBA with the confidence that they can run a utility at any time and guarantee that the utility will not negatively impact the workload performance beyond the specified amount. G. Self-Optimizing Utilities When performing database backups and restores, or when loading new data into the database, there are several configuration settings that can affect how long the utility will take to complete. For example, when backing up a database, the DBA can specify how much memory to dedicate to the backup job. Additionally, the DBA has the ability to specify the number of processes to be used in the backup job (i.e., the degree of parallelism). While setting these values correctly can greatly affect the execution time of the utility, determining appropriate values can be difficult and requires advanced database skills. In the DB2 UDB product, the DBA has been relieved of this configuration burden. If a backup job is initiated, the system determines the correct amount of memory and parallelism to use


without any intervention by the administrator of the RDBMS. With this new feature, the backup utility has been shown to run several times faster than it had in the past with default settings. In the presence of an advanced DBA, these settings can be overridden and tailored to individual needs. This simplified configuration also exists for the data load utility, automating the number of CPUs exploited, the I/O parallelism, the memory consumption, and the index maintenance strategy; thus, reducing the amount of work required to bring a database online for the first time and to populate tables, data marts, and data warehouses. 5 H. Workload Management Databases can often be accessed by many different users, each with their own specific set of requirements. A system may concurrently have customers attempting to purchase items, a DBA adding new user accounts, and a marketing department trying to analyze the weekly sales data. When this work enters the database management system, each piece of work is treated equally and will be competing for common resources such as CPU cycles, disk bandwidth, memory, and locks on the database data. The trouble with managing such a diverse system is that the work being attempted by each of these users will likely have different priorities. While weekly sales data are important, it should never force the customers to wait when trying to buy a product. The job of the DBA is to prevent low-priority queries from holding up higher priority queries. This can be a tremendously daunting task if left entirely to the DBA. In fact, if workload management is not built directly into the database management system, some levels of priority specification may be impossible. In response to the need for automation in the area of workload management, the Query Patroller (QP) was added to the DB2 UDB product. The QP allows the DBA to generate a set of rules or policies through which the system will be governed. To accomplish this, QP allows the DBA to assign resources to a user or a set of users in a given class. A user here can be a single-person account with access to the database or a whole application used by many people. For each user, or class of users, the DBA can specify the following: 1) the maximum number of concurrent queries that the user can run at any given time; 2) the maximum cost allowed for any query executed by the user; 3) the maximum number of rows that can be returned from any query executed by the user; 4) the priority in which to run queries executed by the user. Limiting the number of concurrent queries, the maximum cost for any given query and the number of rows returned by any one query prevents any given user from monopolizing the system resources. The priority setting then enables the DBA to determine which queries should be accepted in the system based on the user that submitted each query. This provides the DBA 5 These self-managing capabilities were introduced for the data load utility in V5 of the DB2 UDB product and for the backup utility in DB2 UDB V8.2.

373

with the ability to properly control the system and prevent a low-priority user from blocking a higher priority user. When queries are issued to the system, the QP first determines if allowing the query to execute will violate any of the definitions for the user executing the query. Provided that the query will not violate the specified rules, it is allowed to run on the system. If, on the other hand, the query will violate a rule, it is prevented from executing. Preventing a query from executing could be a permanent decision, as in the case where the cost of the query is too high for the given user, or it could be a temporary decision in the case where too many queries are running for a given user. In the latter case, the QP will monitor the system and run the query when the rules are no longer violated. The QP also allows the DBA to create query classes. Query classes allow for the differentiation between queries of different cost. Assume we have a database system that is used to manage a warehouse of products. Although the system is mainly used for transaction processing, i.e., selling products, occasionally the entire warehouse must be queried to perform trend analysis. Since querying the entire warehouse is costly, the DBA may want to limit the number of large queries on the system at any one time so that the transaction processing can still function. Query classes allow the DBA to specify the maximum cost of each query class as well as the number of queries of each class that are able to run on the system at any given time. This provides an extra level of protection for the DBA. VI. SELF-PROTECTING The self-protecting features in the DB2 UDB V8.2 product serve to reduce the administrative burden involved in securing the database system. While in the future, the direction for the DB2 UDB product is to employ truly autonomic self-protecting features, the current self-protecting features in DB2 UDB are strictly static. This is consistent with the self-protecting features of other commercial RDBMSs; self-protecting feature development has tended to lag behind the other three “self” categories. In this section, we discuss the client/server authentication feature in the DB2 UDB product as well as an infrastructure that allows the DBAs to add their own security technology. A. Data Encryption The DB2 UDB product employs a client/server model for data processing. All data are maintained on the server and requests are made to the server by the client to access these data. When a connection is made from the client to the server, authentication takes place to ensure that the client has the necessary permissions to access the database. The authentication type is configurable so that the DBA has some control over the level of security that will be enforced. In server authentication, a commonly used authentication method, the user ID and password must flow from the client to the server, where they are used to authenticate the user. Conversely, in client authentication, the authentication of the user takes place on the client using operating system authentication methods. To help ensure that the data being sent to and from the client are kept secure, a new authentication method was added in

374


the DB2 UDB V8.2 product. The new authentication method, called “DATA ENCRYPT,” encrypts the data at the client before sending and asks the server to reply with an encrypted result set. This eliminates the need for the application programmer to encrypt and decrypt data when dealing with the database server. B. Security Plug-Ins If the existing authentication methods are not able to meet the given customer needs, customers can take matters into their own hands. The DB2 UDB V8.2 product provides an infrastructure that allows users to write their own authentication methods using security plug-ins. These user-created plug-ins can then be loaded by the product and used to enforce customer-tailored security. The security plug-in infrastructure provides a naming convention for security plug-in libraries and functions. Once the administrator of the RDBMS has built (or bought) and appropriately named their security plug-in, they can change the database manager configuration to point to this new library. When the DB2 UDB product performs authentication, it will check to see if the user has provided a security plug-in and if so will perform authentication through this provided library. While not fully autonomic, in that they must first be written by the user before being used, the security plug-ins do allow for autonomic authentication once written. More importantly, though, the security plug-ins allow customers to utilize an autonomic security protocol of their choosing with little integration effort. VII. REPORTING AND INTEGRATION With the increasing demands of business, corporations are no longer installing single software solutions but instead are building software stacks to manage more robust systems. These software stacks often include Web servers, some level of middleware, which could include database software and messaging services, and all of this sits on top of the operating system. Integrating the pieces of this software stack to work together seamlessly can be difficult, but is a customer requirement. IBM has recognized this customer requirement and has added several features to the DB2 UDB product that address the need for greater software stack integration. A. Workload Management One major problem in software stack integration is providing comprehensive top-to-bottom workload management to ensure quality of service requirements across the entire system. It may be that the Web server is asked to allow 1000 Web page accesses per second, but it can only meet this requirement if the underlying database adjusts its configuration to allow for higher throughput. This problem can manifest itself in many different ways and, to be solved, must be addressed in every level of the software stack. IBM has dedicated a substantial amount of effort to solving this problem with eWLM, the IBM enterprise workload management utility, for comprehensive workload management throughout the entire software stack [19]. The project is a mul-

tiyear effort involving both research and development divisions across the corporation. DB2 UDB has added some preliminary support for eWLM to allow for rudimentary workload monitoring. This monitoring allows a user of eWLM to track the number and frequency of transactions that are flowing through the DB2 UDB product. The future vision is that the DB2 UDB product will be fully integrated into the eWLM infrastructure, so that a user of eWLM will be able to set response time goals for a given transaction or transaction class and eWLM will modify the DB2 UDB product’s resource consumption to achieve the workload response time goals. B. Reporting The DB2 UDB product provides two ways to communicate to the administrator of the RDBMS the important events that occur in the system. The first set of events must be transmitted to the administrator when the system becomes unhealthy. These events that are to be sent to the DBA are first evaluated by the HMON, as described in the Section IV. The HMON is the first line of defense for common problems, providing efficient reporting mechanisms, like pager and e-mail, which allow fast resolution of the health issues. Any DBA that receives such a message can access the administration utilities using a Web browser through the Internet from anywhere in the world. From within the Web interface, the user can also get advice from the Recommendation Advisor and solve the problem promptly. The second level of reporting is required at problem determination time. If a system crashes or becomes nonoperational for some other reason, customers typically call IBM Service for help in rectifying the situation. Diagnosing the problem, using the massive amount of logging information that is generated, can be a dauntingly manual task. To allow for easier problem diagnosis, the db2diag utility was created and added to the DB2 UDB product. Additionally, the diagnostic log records generated by the DB2 UDB product were integrated into the Log and Trace Analyzer. The db2diag utility allows for filtering out background noise when trying to analyze a specific problem. It allows for a service analyst to quickly take the full diagnostic log and filter out all but a few messages that may be important in diagnosing the specific problem. This filtering capability reduces significantly the massive amount of data that are dumped in the diagnostic log by the database engine in systems that require continuous (24 × 7) operation and do not stop operation for months. The DB2 UDB product has also integrated with the Log and Trace Analyzer, an Eclipse-based utility [18]. The Log and Trace Analyzer enables the viewing, analyzing, and correlating of log files generated by several applications, including the DB2 UDB product. The utility allows the integration of log and trace files produced by the various components of the software stack. This integrated analysis of the events produced by all layers of the system allows for faster problem determination, which significantly eases the manageability burden of complex systems. For instance, assume that a machine crashes and a call comes into the DB2 UDB product service team because the customer


suspects that product caused the crash. The DB2 UDB product analyst can then examine the operating system log records and determine if the problem was caused by the database, the operating system, or neither. If, in fact, the problem was caused by the operating system, the service analysts from the operating system support team can get involved in the problem determination. This mechanism minimizes the chance of misdiagnosing a problem and sending it to the incorrect team for determination. VIII. FUTURE WORK Some general themes for future research are as follows: 1) full automation of some features that are currently advisory utilities, so that they are enabled first to run unattended and secondly, to take explicit system action rather than just provide expert recommendations; 2) fully automated configuration and tuning of all system resources allocated to the database (memory, network, disk, CPU); 3) automated design for system topology, which is clearly coupled with other physical database design problems that require the design choices to be codependent; 4) expanded workload and resource management for user requests and system-initiated background processing, allowing automated resource management that can be prioritized by user class or application class; 5) further research into learning during query access plan selection; 6) automated problem determination and obviation, in particular surrounding: configuration errors that relate to task failure, storage administration, and workload performance. IX. CONCLUSION The IBM DB2 UDB for Linux, UNIX, and Windows V8.2 product has begun to deliver on the promise of autonomic computing by delivering a critical mass of autonomic capabilities across the four domains of self-configuring, self-healing, selfoptimizing, and self-protecting. Additionally, DB2 UDB has built a coordinated set of technologies for reporting and interfaces (building trust, enabling user control) and enterprise integration. By focusing on traditional database technologies as well as on relevant technology from control theory, statistics, queuing theory, expert systems, and other fields, the DB2 UDB product has developed rich capabilities that distinguish it within the industry for its scope and quality of autonomic features.

375

[3] S. Chaudhuri, B. Dageville, and G. Lohman, “Tutorial 3: self-managing technology in database management systems,” presented at the 30th Int. Conf. VLDB, Toronto, ON, Canada, Aug. 31, 2004. [4] Organization for the Advancement of Structured Information Standards (OASIS). [Online]. Available: http://www.oasisopen.org/home/index.php [5] Transaction Processing Performance Council. [Online]. Available: http://www.tpc.org/ [6] DB2 Universal Database for Linux, UNIX and Windows. [Online]. Available: http://www- 306.ibm.com/software/data/db2/udb/ [7] S. Parekh, K. R. Rose, J. L. Hellerstein, S. Lightstone, M. Huras, and V. Chang, “Throttling utilities in the IBM DB2 universal database server,” presented at the American Control Conf., Boston, MA, Jun. 2004. [8] D. Zilio, G. Lohman, C. Zuzarte, W. Ma, S. Lightstone, B. Cochrane, and H. Pirahesh, “Recommending materialized views and indexes by IBM’s DB2 design advisor,” in Proc. Int. Conf. Autonomic Computing (ICAC04), New York, May 17–18, 2004, pp. 180–188. [9] S. Lightstone and B. Bhattacharjee, “Automating the design of multidimensional clustering tables in relational databases,” in Proc. 30th Int. Conf. VLDB, Toronto, ON, Canada, Aug. 31, 2004, pp. 1170–1181. [10] J. Rao, S. Lightstone, G. Lohman, D. Zilio, A. Storm, C. Garcia-Arellano, and S. Fadden, “DB2 design advisor: Integrated automated physical database design,” in Proc. 30th Int. Conf. VLDB, Toronto, ON, Canada, Aug. 31, 2004, pp. 1087–1097. [11] S. Lightstone, B. Schiefer, and D. Zilio, “Autonomic computing for relational databases: The ten year vision,” presented at the IEEE Workshop on Autonomic Computing Principles and Architectures (AUCOPA 2003), Banff, AB, Aug. 2003. [12] S. Lightstone, E. Kwan, L. Wu, B. Schiefer, and A. Storm, “Automatic configuration for IBM DB2 universal database: Compressing years of performance tuning experience into seconds of execution,” presented at the 10th Conf. Database Systems for Business, Technology, and the Web (BTW 2003), Germany, Univ. Leipzig, Feb. 2003. [13] J. Rao, C. Zhang, N. Megiddo, and G. Lohman, “Automating physical database design in a parallel database,” in Proc. 2002 ACM SIGMOD Int. Conf. Management of Data (SIGMOD 2003), Madison, WI, Jun. 3–6, 2002, pp. 558–559. [14] P. Selinger, M. Astrahan, D. Chamberlin, R. Lorie, and T. Price, “Access path selection in a relational database management system,” in Proc. 1979 ACM SIGMOD Int. Conf. Management of Data (SIGMOD 1979), Boston, MA, May 30, 1979, pp. 23–34. [15] P. Selinger and M. Adiba, “Access path selection in distributed database management systems,” in Proc. Int. Conf. Databases (ICOD 1980), Univ. Aberdeen, Aberdeen, U.K., 1980, pp. 204–215. [16] S. Parekh, K. Rose, J. Hellerstein, S. Lightstone, M. Huras, and V. Chang, “Managing the performance impact of administrative utilities,” Proc. 14th IFIP/IEEE Int. Workshop Distributed Systems: Operations and Management (DSOM 2003), Heidelberg, Germany, Oct. 2003, pp. 130–142. [17] V. Markl, I. Popivanov, S. Lightstone, V. Raman, A. Aboulnaga, P. Haas, and G. Lohman, “Automated statistics collection in DB2 UDB,” in Proc. 30th Int. Conf. VLDB, Toronto, ON, Canada, Aug. 31, 2004, pp. 1146– 1157. [18] IBM alphaWorks. [Online]. Available: http://www.alphaworks.ibm.com. [19] IBM Research, eWLM: Enterprise Workload Management. [Online]. Available: http://www.research.ibm.com/thinkresearch/pages/2002/ 20020529 ewlm.shtml [20] G. Valentin, M. Zuliani, D. Zilio, G. Lohman, and A. Skelley, “DB2 advisor: An optimizer smart enough to recommend its own indexes,” in Proc. IEEE Int. Conf. Data Engineering, San Diego, CA, Mar. 28, 2000, pp. 101–110.

ACKNOWLEDGMENT The authors would like to acknowledge P. Bird, K. McCullough, K. Schlamb, and C. Zuzarte for their help in writing and editing this paper. REFERENCES [1] E. F. Codd, “A relational model of data for large shared data banks,” Commun. ACM, vol. 13, no. 6, pp. 377–387, Jun. 1970. [2] P. Horn (2001). “Autonomic computing: IBM’s perspective on the state of information technology,” Armonk, NY: International Business Machines. [Online]. Available: http://www.ibm.com/research/autonomic

Christian M. Garcia-Arellano received the Information Systems Engineering degree from the Universidad Tecnologica Nacional, Buenos Aires, Argentina, in 1997 and the M.Sc. degree in computer science from the University of Toronto, Toronto, Ontario, Canada, in 2002. He is a member of the DB2 Engine Development team at the IBM Toronto Laboratory, Markham, ON, Canada. In the last three years he has mainly worked on several memoryrelated enhancements for DB2 UDB. These projects have included enhancements to the configuration advisor, sort memory management, and exploring the affects of online memory

376


tuning on a running workload. He has also worked on autonomic projects like the Design Advisor. Currently, he is focused on the development of the selftuning memory manager (STMM) for DB2 UDB.

Autonomic Computing project (formerly known as SMART, self-managing and resource tuning), part of IBM’s company-wide Autonomic Computing initiative. In 2002, he was elected to the IBM Academy of Technology. His current research interests involve query optimization, self-managing database systems, and problem determination.

Sam S. Lightstone received the B.Sc. Eng. degree in electrical engineering from Queen’s University, Kingston, Ontario, Canada, in 1991 and the MMath degree in computer science from the University of Waterloo, Waterloo, Ontario, Canada, in 2004. He is a Senior Technical Staff Member and Development Manager with IBM’s DB2 Universal Database Development Team IBM Toronto Laboratory, Markham, ON, Canada. Since 1991, he has been with IBM. He is the cofounder and Leader of DB2’s autonomic computing R&D effort. He is an IBM Master Inventor with over 25 patents and patents pending; he has published widely on autonomic computing for relational database systems. His current research includes numerous topics in autonomic computing and relational database management systems including automatic physical database design, adaptive self-tuning resources, automatic administration, benchmarking methodologies, and system control. Mr. Lightstone is a member of IBM’s Autonomic Computing Architecture Board and a member of the IEEE Computer Society Task Force on Autonomous and Autonomic Computing. In 2003, he was elected to the Canadian Technical Excellence Council, the Canadian affiliate of the IBM Academy of Technology.

Volker Markl received the Master’s degree in computer science from the Technische Universität München, Munich, Germany, in 1995, degree in business administration from the University Hagen, Hagen, Germany, in 1995 and the Ph.D. degree from the Technische Universität München. He has been working at IBM’s Almaden Research Center, San Jose, CA since 2001, conducting research in query optimization, indexing, and self-managing databases. He is spearheading the LEO project—an effort on autonomic computing with the goal to create a selftuning optimizer for DB2 UDB. Before joining IBM, he was the Deputy Research Group Manager with the Bavarian Research Center for KnowledgeBased Systems (FORWISS), Munich, Germany, leading the MISTRAL and MDA projects. Mr. Markl is the Almaden chair for the IBM Data Management Professional Interest Community (PIC).

Guy M. Lohman received the B.A. degree in mathematics from the Pomona College, Claremont, CA, in 1971, the M.S. and Ph. D. degrees in operations research from the Cornell University, Ithaca, NY, in 1975 and 1977, respectively. He is a Manager of Advanced Optimization in the Advanced Database Solutions Department, IBM Almaden Research Center, San Jose, CA, and has 23 years of experience in relational query optimization. He is the architect of the Optimizer of the DB2 Universal Data Base (UDB) for Linux, Unix, and Windows, and was responsible for its development in Versions 2 and 5, as well as the invention and prototyping of Visual Explain. He also managed the overall effort to incorporate into the DB2 UDB product the Starburst compiler technology that was prototyped at the Almaden Research Center. More recently, he was the coinventor and designer of the DB2 Index Advisor (now called Design Advisor) and cofounder of the DB2

Adam J. Storm received the B.Sc. degree from McMaster University, Hamilton, ON, Canada and the M.Math. degree in computer science from the University of Waterloo, Waterloo, ON, Canada, in 2000. He is a member of the DB2 Engine Development team, IBM Toronto Laboratory, Markham, ON, Canada. In the last five years he has worked on several memory related enhancements for DB2 UDB. These projects have included enhancements to the DB2 Configuration Advisor, creation of the DB2 Memory Tracker, and work on the DB2 Memory Visualizer. He has also worked on several other Autonomic projects at IBM, including the DB2 support tool and the DB2 Design Advisor. Most recently, he has been working on the Self-Tuning Memory Manager (STMM) for DB2 UDB Viper.