Percentages of Rows Read by Queries as an Operational Database Quality Indicator Paweł Lenkiewicz1 and Krzysztof Stencel1,2 1
Polish-Japanese Institute of Information Technology, Warsaw, Poland
[email protected] 2 Institute of Informatics, Warsaw University, Poland
[email protected]
Abstract. Trace files generated during operation of a database provide enormous amounts of information. This information varies from one DBMS to another—some databases produce more information than the others. There are many research projects which aim at analysing workloads of databases (not necessarily the trace files). Many of them work online in parallel with the usual business of a DBMS. Such approaches exclude a holistic tackling of trace files. To date, the research on offline methods had only a partial scope. In this paper we show a comprehensive method to analyse trace files off-line. The aim of this analysis is to indicate tables and queries which are not handled well in current design of the database and the application. Next, we show case-studies performed on two dissimilar database applications which show the potential of the described method.
1 Introduction An operating database may be set to produce trace files. It is a log of all SQL statements performed by the DBMS accompanied with a number of measures like elapsed time, CPU time, number of rows touched, number of logical/physical reads of data blocks. Trace files are sources of mass information on the actual performance of the DBMS. A holistic analysis of such a source is a challenging task. There is a lot of ongoing research on the workload of databases (by the workload we mean the stream of queries performed by a database). If we limit solely to the stream we can perform some online tests to assess the current performance and the need to change the design [1,2,3]. However, it is impossible to analyse the whole trace online, since in this mode we can only assess some simple measures based on the workload stream. Compare it to the basic blood test. A small blood drop contains all information on the diseases you had, you have and most of those which you will have. However, the basic blood test consists in evaluating a relatively small number of simple indicators. This is a tremendous wasting of information: the blood probe contains the whole DNA of the examined human being, but we observe no more than fifty numbers. The same concerns database trace, which contains all the information on the problems the database system has and will have. However, currently available tools and methods allow assessing only an isolated fraction of the problem. N.T. Nguyen et al. (Eds.): Adv. in Intelligent Inform. and Database Systems, SCI 283, pp. 17–27. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com
18
P. Lenkiewicz and K. Stencel
Of course we have excellent advisor methods to recommend indices [4,5,6], recommend merging of indices [7], recommend materialized views [5,8]. Biggest commercial DBMS vendors implemented special advising tools, e.g. Oracle [9], Microsoft SQL Server [10] and IBM DB2 [11]. There are also some good books [12,13] on database tuning. They are invaluable source of practical information on tuning. However, there is no proposal of comprehensive analysis of more than one aspect in the trace file. In this paper we propose one such method. We show how a thorough analysis of the trace file can show sources of potential problems long before these problems start annoying users. The method consists in applying a holistic analysis of the collected trace files and producing a report on strange behaviours of a DBMS. We verified our method in practice by applying it to two OLTP applications with dissimilar usage profiles. One of them was a web application which performs many local queries (i.e. queries always returning a small result set). The other was a thick client application with a lot of local queries and updates, but it also has some bigger reporting queries. The proposed method allowed to improve both applications significantly. The paper is organized as follows. In Section 2 we present the sketch of our method. In Section 3 we characterize the database applications used in our case study. Section 4 describes desired and undesired behaviours which can be diagnosed by our method. Section 5 presents the improvements which have been achieved as the result of applying tuning activities recommended by our method. Section 6 concludes.
2 The Method Our proposed method of database performance monitoring consists of three steps. First, we gather trace information on performed queries with statistical data like the number of rows and pages read. Second, we gather all this information in the form of histograms (one histogram for each table). Third, we clasify table histogram to one of the cases. During our research, the most challenging task was to identify these cases, find the interesting points in those histograms and check the execution plans of queries typical for each such point. It amounted that there were not so many shapes of those histograms and the main indicator is the number of "full scan" reads. In Section 4 we will present the classification of those shapes and the possible execution plan anomalies they indicated. The first step to get the number of rows read consists of collecting a database trace during the most typical operation period of a database application. SQL Server Profiler tool was used for this purpose. It was configured to catch all SQL statements and stored procedures calls in the monitored database. The trace had been saved to the database table. Here we encountered a serious problem, since the table created by SQL Server Profiler did not allow to make more detailed analysis. The trace data is very difficult for analysis. It would be necessary to parse all the statements to get more information about tables affected by this statements and then deduce the number of affected rows. The situation became more complicated, because we did not analyse individual SQL statements but groups of instructions or stored procedure
Percentages of Rows Read by Queries
19
call. Even more difficult issue was caused by a statement which fired a trigger. In his situation it would be necessary to parse all texts of related objects (stored procedures, triggers). Checking the number of rows read from the tables would be too difficult. We decided to do it another way. We’ve created a diagnostic application which uses the SQL Server query optimiser to get execution plans of all statements from the trace. In these plans we could easily find the physical data access paths (table scan, index scan, index seek) and learn the number of rows affected by each read. The application stores obtained data in simple relational database, which make further analysis much easier. This database stores information about batches, particular statements and tables affected by these statements with most inferesting value for us, i.e. the number of rows read from the table. When the trace is transformed to this form, it is possible to start the generation of histograms on the tables’ usage. We used a simple application for this. It reads all the records from the table which stores information about affected tables and using the number of affected rows and total number of rows builds the table with histogram of row reads. The last step is classifying the tables’ histogram to one of the cases basing on its shape. Characteristics of the case shows us possible tables’ anomalies. The classification is described in details in Section 4.
3 Test-Drive Applications The method has been tested on two database applications, which have Microsoft SQL Server 2005 as the backend. Both of these applications are intensively used in Polish-Japanese Institute of Information Technology. These database applications have been chosen because they represent two different, typical usage profiles. The first tested application was the main Institutes’ database system consisting of: deanery module, documents turnover module, virtual deanery, as well as the module of students payments. The majority of system modules are the Windows applications prepared for particular group of users - employees and students. Only small part of the system functionality is available on the web. The database is very often used for reporting and processing large groups of records, so we can find a lot of complex queries and stored procedures in the trace. The second system was e-learning platform used for managing Internet based studies as well as supporting of traditional learning. The system consists of many modules: content management system, discussion forum, files management module, tests, exams, lessons and others. Almost entire system functionality is available through web interface. The trace shows, that there are a lot of simple point queries and joins, what is typical in web applications. During our analysis we have omitted tables with small number of records as well as the tables which occurred in the trace sporadically. Both applications are intensively developed, so the database objects change very often and a lot of new objects are created. Databases are tuned in unsystematic way by developers and administrators.
20
P. Lenkiewicz and K. Stencel
4 Classification of Histograms We checked many queries and their execution plans from the particular points in the histogram and tried to find anomalies. Then we could classify typical histograms and describe possible anomalies for every case. As the result of analysis of histograms obtained by the proposed method, we have found the five most often occurring cases. The majority of tables could be easily classified to one of them. Every case showed or did not show some anomalies. There were also tables not adequate to these cases or showing characteristics of several cases. It can be easily noticed that the most of physical data accesses concern between 0 and 10% of the table records (mainly point queries, multi point queries, joins) as well as between 90 and 100% (full scans). Reads of number of rows from the range 10-90% are rare. It often indicates anomalies, but sometimes it occurs that they are proper, especially within the range 10-30%. The reads of this type concern mainly range queries or columns with bad selectivity. 4.1 Case 1: 100% of Operations Reading Less Than 10% Records This is the best case. The result suggests lack of anomalies regarding the table. Proper selection and use of indexes. Analysis of queries for his type of tables shows that all reads are done by the index seek physical operation. Such tables have properly designed access to the data from the application.
Fig. 1. Histogram, case 1
4.2 Case 2: Significant Predomination of Operations Reading Less Than 10% of Records, Small Amount of Reads Regarding the Whole Table (Less Than 20%), Small Amount of Reads Regarding 10-30% of Records (Less Than 20%) This case regards typical tables, intensively used in different parts of application. Significant predomination of operations reading less than 10% of records suggests the proper selection of indexes and right data access path used by the application.
Percentages of Rows Read by Queries
21
Fig. 2. Histogram, case 2
Small amount of table scan reads is normal, mainly due to execution of reporting queries. The reads regarding 10-30% may, but need not suggest anomalies. Usually they become from queries that join many tables or fetch many rows to application. Example The Student_registration table is used in deanery application very often. Record in the table contains information that a student is registered for a semester of particular type of studies. 12% of physical data access operations are full table scan reads. Some of them are reporting queries, where the full scan is necessary. The rest of them refers to queries with WHERE condition on a foreign key. All foreign keys are indexed, but some of them have small selectivity. Here is a typical query: SELECT Name, Surname, No FROM Person INNER JOIN Student ON Person.IdPerson=Student.IdPerson INNER JOIN Student_registration ON Student.IdPerson=Student_registration.IdPerson WHERE IdStudies=5 ORDER BY Surname, Name There are only about 20 types of studies, but thousands of registrations, so the selectivity is very bad. The query optimiser does not use the index on foreign key and does the full scan read. 16% of reads affect between 10 and 30% of records. These reads results from the queries which read many records to the application or join with another table, but with foreign key with better selectivity. 4.3 Case 3: Predomination of Operations Reading 100% of Records (More Than 60%) The histogram of this type suggests the lack of indexes or non-proper selection of indexes. Especially lack of indexes on foreign keys and columns occurring in WHERE clause. Analysing queries from the trace for this table type, we can observe a lot of table scan operations (lack of indexes) or clustered index scan operations (only default clustered index on the primary key) although we are dealing with point queries or joins.
22
P. Lenkiewicz and K. Stencel
Fig. 3. Histogram, case 3
Example All reads from the table Payment affect all records although there are mainly point queries and range queries. There is only one clustered index on the primary key. Typical query: SELECT Amount, Date FROM Payment WHERE IdPerson = 4537 AND Date BETWEEN ’2009-01-01’ AND ’2009-01-31’ The optimal strategy for this case is adding non clustered index on IdPerson column and clustered index on Date column. 4.4 Case 4: Predomination of Operations Reading Small Amount of Records, but Large Amount of Operations Reading 100% of Records (Up to 50%) This situation may occur in case of queries selecting data from many tables with typically created indexes on the keys: clustered index on the primary key and non clustered indexes on foreign keys. In many cases, especially regarding less selective data, the server will not use non clustered index on the foreign key but will read
Fig. 4. Histogram, case 4
Percentages of Rows Read by Queries
23
100% of records. Performance improvement may be achieved by using clustered index on a foreign key, which enforce change of the index type on the primary key to non clustered. When the table includes many foreign keys, the main problem is to find the one in which the data is less selective and create clustered index on it and the non clustered indexes for the others. The typical situation in this case is an association table with auto-numbered primary key with unused clustered index and with two foreign keys with non clustered indexes, especially when the data in the foreign key columns are badly selective. Example Group_assignment is the association between the Student_registration table and the Group table. The table consist of auto-numbered primary key with default clustered index and two foreign keys with non clustered indexes. The clustered index is never used. 38% of reads are the full scan caused by joins which use indexes with small selectivity. SELECT Name, Surname, No, GroupNo FROM Person p INNER JOIN Student s ON p.IdPerson = s.IdPerson INNER JOIN Student_registration sr ON s.IdPerson = sr.IdPerson INNER JOIN Group_assignment ga ON sr.IdStudent_registration = ga.IdStudent_registration INNER JOIN Group g ON g.IdGroup = ga.IdGroup ORDER BY GroupNo, Surname, Name
4.5 Case 5: Large Amount of Full Scan Operations (More Than 15%). Other Queries Read Small Number of Records This type of situation could occur when an application is written badly. Small amount of full scan operations is acceptable and could be generated by reporting queries, but bigger number of full scans may suggests that the application needlessly reads entire table e.g. to combo boxes, lists, data grids, etc. In this case usually we
Fig. 5. Histogram, case 5
24
P. Lenkiewicz and K. Stencel
can see, that the indexes are created properly. Some tables can be classified as belonging to this case because of the structure of the queries. Mainly these are complex queries containing subqueries, which are difficult for the query optimiser. Rewriting these queries or stored procedures can significantly improve performance. The other example is a query which contains functions in WHERE clause, what makes the existing indexes useless. Creating the functional index or index on computed column can help in this case. Example The main modules’ starting screen loads all students to the data grid, using the query: SELECT Name, Surname, No FROM Person INNER JOIN Student ON Person.IdPerson = Student.IdPerson ORDER BY Surname, Name
The query must read both tables with the full scan and sort them. Most of users ignore the students’ list read to the screen at such a big cost and start their work issuing the query for a particular student they are interested in. The full scan and external sort of the big table Students is thus performed in vain. 4.6 Other Cases Some tables cannot be classified to any of these cases or are able to be classified to several cases. During our works we have found tables, in which some anomalies occurred parallel, for example unnecessary reading all records by the application and incorrect index selection. There were also tables, which were classified incorrectly because of uncharacteristic queries in period of time, when the trace was collected.
5 Achieved Improvements Proposed method will be very good for large, intensively used and developed databases, in which manual, unsystematic methods of database tuning will not be successful. The big advantage of the method is that tuning recommendations can be obtained relatively fast, without large administrative work. It is also possible to use the method incrementally, by adding to the histogram results of analysis of traces taken later. It will allow to analyse consecutively developed database or the database which is used differently in different periods. Our method will not always guarantee exact optimisation recommendations, so cannot replace a database administrator, but in most cases it will make his work easier. In every case all the tables in which a number of row reads varies against standard will be produced. Obtained list of tables as well as list of possible anomalies may be used for manual tuning. It can also suit as an input to other methods of authomatic database tuning. Owing to large reduction of tables list used for tuning, it becomes possible to achieve substantial reduction of time required for authomatic as well as manual methods. Below we list problems, which have been detected.
Percentages of Rows Read by Queries
25
– Lack of indexes. We have detected lack of indexes mainly on foreign keys and columns used in selection criteria. The problem affected 11 tables in our testdrive applications. Adding proper indexes makes a great impact on performance of particular queries and the application. Adding indexes to all detected non indexed tables reduced the total number of physical page reads by 8% for typical hour of database functioning. – Incorrect indexes selection. We have found 7 tables with improper index configuration. In many cases administrators accept default clustered index on the primary key and use non clustered indexes for the remaining columns. Sometimes it is a very bad strategy, mainly for the foreign keys with small selectivity. In many cases query optimiser will not use the non clustered index, but will use the full scan. Tuning all such tables gave us 3% reduction of physical reads in typical hour. – Execution of heavy reporting queries during the hours of intensive activity in database. Using our method, these kind of queries can be identified. In consequence, they may be executed for example during the night. It is also possible to analyse the time schedule of full scan type reads, aimed for finding the optimum time for its execution. The next goal is to schedule the preparation of materialized views, which will help to generate the reports during the day. We have identified a few reports which could be a good target for such optimisation and moved parts or entire of its preparation to the night. It improved the system response time during the peak hours of system activity and shortened the time needed for preparations of some reports. – Faulty written application. We have found a lot of places, where the application reads large amounts of data unnecessarily (including the worse situation: all records from the table). It happens mainly in the case of dictionary tables used in combo boxes, lists, data grids, etc. It was impossible to fix all the problems quickly, but fixing some of them improved the system response time and reduced the average number of physical reads per hour by 4%. – Finding the tables in which the number of reads is small in the relation to the number of modifications. In these tables the number of indexes should be reduced, although speeding up the modifications will slow down the queries. The decision about the proportion between reads and writes is very difficult. The good knowledge of applications is recommended. In our test-drive applications we have found a few intensively updated tables, which are queried rarely. Dropping existing indexes allowed us to save some disk space and speed up modifications (mainly insertions). We reduced the number of page writes from 1 to 4% in some hours of database activity. – Finding places where creating a functional index should be preferable. We have found build-in functions (like CONVERT, DATEPART, DATEADD, SUBSTRING etc.) in many queries in WHERE clause. In many cases the query optimiser cannot use the index in such queries. To fix the problem we must rewrite the query and/or change the application or use the functional index or indexed
26
P. Lenkiewicz and K. Stencel
computed columns. In some queries it allowed us to reduce the number of full scan type reads and speed up the queries significantly. As in every tuning method based on the trace, it is very important to take the workload from the very typical period of database functioning. If the workload contains to little amount of reads from the table, the result can be not authoritative.
6 Conclusion In this paper we have presented an offline approach to analysis of database trace files. We have show that quantitative analysis of the volumes of rows accessed for each table can indicated ill-tuned tables and queries. The case studies included in the paper proved that the proposed method is proficient as it allowed improving studied database applications significantly. The tuning process performed in our case studies was partly manual with some automated tool support. However, vast majority of the activities performed were easy to mechanise. Future work will encompass efforts to automate the advocated approach as a new kind of database tuning advisors.
References 1. Sattler, K.U., Geist, I., Schallehn, E.: Quiet: Continuous query-driven index tuning. In: VLDB, pp. 1129–1132 (2003) 2. Schnaitter, K., Abiteboul, S., Milo, T., Polyzotis, N.: Colt: continuous on-line tuning. In: Chaudhuri, S., Hristidis, V., Polyzotis, N. (eds.) SIGMOD Conference, pp. 793–795. ACM, New York (2006) 3. Bruno, N., Chaudhuri, S.: An online approach to physical design tuning. In: [6], pp. 826–835 4. Chaudhuri, S., Narasayya, V.R.: Autoadmin ’what-if’ index analysis utility. In: Haas, L.M., Tiwary, A. (eds.) SIGMOD Conference, pp. 367–378. ACM Press, New York (1998) 5. Agrawal, S., Chaudhuri, S., Narasayya, V.R.: Automated selection of materialized views and indexes in sql databases. In: Abbadi, A.E., Brodie, M.L., Chakravarthy, S., Dayal, U., Kamel, N., Schlageter, G., Whang, K.Y. (eds.) VLDB, pp. 496–505. Morgan Kaufmann, San Francisco (2000) 6. Proceedings of the 23rd International Conference on Data Engineering, ICDE 2007, The Marmara Hotel, Istanbul, Turkey, April 15-20, 2007. IEEE (2007) 7. Bruno, N., Chaudhuri, S.: Physical design refinement: The “Merge-reduce” approach. In: Ioannidis, Y., Scholl, M.H., Schmidt, J.W., Matthes, F., Hatzopoulos, M., Böhm, K., Kemper, A., Grust, T., Böhm, C. (eds.) EDBT 2006. LNCS, vol. 3896, pp. 386–404. Springer, Heidelberg (2006) 8. Zilio, D.C., Zuzarte, C., Lightstone, S., Ma, W., Lohman, G.M., Cochrane, R., Pirahesh, H., Colby, L.S., Gryz, J., Alton, E., Liang, D., Valentin, G.: Recommending materialized views and indexes with ibm db2 design advisor. In: ICAC, pp. 180–188. IEEE Computer Society, Los Alamitos (2004) 9. Dageville, B., Das, D., Dias, K., Yagoub, K., Zaït, M., Ziauddin, M.: Automatic sql tuning in oracle 10g. In: [14], pp. 1098–1109 (2004)
Percentages of Rows Read by Queries
27
10. Agrawal, S., Chaudhuri, S., Kollár, L., Marathe, A.P., Narasayya, V.R., Syamala, M.: Database tuning advisor for microsoft sql server 2005. In: [14], pp. 1110–1121 (2004) 11. Valentin, G., Zuliani, M., Zilio, D.C., Lohman, G.M., Skelley, A.: Db2 advisor: An optimizer smart enough to recommend its own indexes. In: ICDE, pp. 101–110 (2000) 12. Shasha, D., Bonnet, P.: Database tuning: principles, experiments, and troubleshooting techniques. Morgan Kaufmann Publishers Inc, San Francisco (2003) 13. Lightstone, S.S., Teorey, T.J., Nadeau, T.: Physical Database Design: the database professional’s guide to exploiting indexes, views, storage, and more. The Morgan Kaufmann Series in Data Management Systems. Morgan Kaufmann Publishers Inc, San Francisco (2007) 14. Nascimento, M.A., Özsu, M.T., Kossmann, D., Miller, R.J., Blakeley, J.A., Schiefer, K.B. (eds.) (e)Proceedings of the Thirtieth International Conference on Very Large Data Bases, Toronto, Canada, August 31 - September 3 2004. Morgan Kaufmann, San Francisco (2004)
