An undocumented SQL Server Optimization Tuning Technique

Senior Database Administrator

Koch Business Solutions, LLC  Founder/President Wichita SQL Server Users Group  Specialized in performance tuning, physical and logical systems and database design  12 years vast experience with the SQL Server stack  6 years as an application developer  Supported OLTP and OLAP environments up to 18TB in size and up to thousands of concurrent users

Scenario 1: The Poorly Performing DB

What’s wrong with the production SellALot database? The performance has really gone downhill recently.

No, this application is 3rd party, so we can’t change anything. Is there anything we can do to beef up the server for SellALot?

Well, the performance monitoring tools we have monitoring that database point to resource issues when the database/server are under load. It really seems to be related to certain queries though. Can these queries be changed or indexes identified to make it perform better? We can throw money at the problem, but there’s no guarantee they’ll fix it. I wish there was a better way of knowing what another CPU with 6 cores or another 64GB of RAM would do for us…..

Scenario 2: The Huge DB

Hey DBA, we need to begin testing some new code against the BIgBoI data warehouse database, but don’t want to risk running it against production because we’re not sure how it will perform and we CANNOT have delays in loading or reporting. Can we get a copy of BIgBoI for testing?

Well, that DW database is so big we just don’t have a dev environment big enough to load and try your changes in. I wish we could give you an exact development copy of that huge database with production data in it to tune against.….but we can’t.

Scenario 3: The Confidential Data DB

Hey DBA, can you create a copy of our HR_Prod database on the dev server with fake data in it so we can test the performance of some program and code changes without exposing Human Resources data to the developers? We just need to be sure the new code will compile and be fast, but we can’t have the real data out there.

Hmmm..…..We can extract and scrub the confidential data, but it’s usually time consuming and sometimes difficult. I wish there was a faster and easier way to do it, but I suppose we have no choice if that’s really what you need.

Introducing the undocumented DBCC command

OPTIMIZER_WHATIF

DBCC OPTIMIZER_WHATIF ( {property/cost_number | property_name} [, {integer_value | string_value} ] )

Undocumented = Unsupported = DON’T RUN IN PRODUCTION!

Use Cases:

• Production Database is too large (space or refresh timeframe) • Confidential/Protected Data • Hardware What Ifs

What it can do:

• Instructs the SQL Optimizer to pretend it has a different set of resources to work with during the query plan estimations • In combination with a database clone (discussed next), you can simulate a production environment on nothing more than a laptop with just a few commands, without all of the data on that laptop!

Database factors Involved in Query Plan creation: Table Metadata: Which indexes are available and the metadata of the tables (data type columns, nullable columns etc.). Session SET OPTIONS: SET options may change the cardinality of a predicate or a join. For instance, setting ANSI_NULLS to OFF will make SQL Server evaluates “NULL = NULL” to true and not as UNKNOWN as expected in the default behavior: this change may affect the cardinality of a filter expression or an outer join. Constraints: SQL Server will try to use a foreign key or a check constraint to create a better query plan.

Statistics: During the creation of a query plan, the Query Optimizer can read statistics to use histograms, density, string summary etc. Number of rows and pages in the table: Based on how many rows a logical query operator will process, the Query Optimizer can decide which physical operator will be used (i.e. a hash or a stream aggregation). In addition, the greater the number of pages in the table, the larger the cost associated with I/O operations will be, so the Query Optimizer has to consider the cost of scanning a table with many pages as opposed using a typically more performance friendly seek.

Hardware factors Involved in Query Plan creation: Available physical memory: The cost of sort and hash operators depends on the relative amount of memory that is available to SQL Server. For example, if the size of the data is larger than the cache, the query optimizer knows that the data must always be spooled to disk. However, if the size of the data is much smaller than the cache, it will probably perform the operation in memory. Number of available CPUs: SQL Server considers different optimizations according to the number of CPUs. The cost associated to a parallel operation is different depending on the number of CPUs that are available. Even if a plan is parallel, SQL Server can identify when you have CPU pressure and will decide at the point of executing the plan not to run the actual plan using many threads, but the estimated plan will remain the same. 32 or 64 bit system: On a 32 bit machine you are limited to the user mode address space, so even you have a server with a lot of memory, you cannot necessarily use all that memory in a sort or hash join operation.

Simulate a production DB in 3 easy steps:

1) Generate a script of the database metadata including the statistics information 2) Create the “data-less” database on a test server

3) Simulate the same CPU/Memory hardware from the production server

Generate a script of the database metadata (including the statistics information) How to generate a script of the necessary database metadata to create a statistics-only database in SQL Server 2005 and in SQL Server 2008 http://support.microsoft.com/?kbid=914288 2005 SP2 or higher

Generate a script of the database metadata (including the statistics information) Scripting option

Value to select

Ansi Padding

True

Continue Scripting on Error

True

Generate Script for Dependent Objects

True

Include System Constraint Names

True

Script Collation

True

Script Create (Database/Objects)

True

Script Logins

True

Script Object Level Permissions

True

Script Statistics

Script Statistics and histograms

Script Indexes

True

Script Triggers

True

Create the “data-less” database on a test server

Easy one here…..“fix up” the script generated in the last step and run on your test server to create the DB. Even on a laptop…

Simulate the same CPU/Memory hardware from the production server using OPTIMIZER_WHATIF() Note: Changed settings will be applied only in the session scope

DBCC TRACEON (2588) WITH NO_INFOMSGS --Enable help for undocumented commands DBCC HELP ('OPTIMIZER_WHATIF') WITH NO_INFOMSGS /* dbcc OPTIMIZER_WHATIF ({property/cost_number | property_name} [,{integer_value | string_value} ]) */

Simulate the same CPU/Memory hardware from the production server using OPTIMIZER_WHATIF() Note: Changed settings will be applied only in the session scope

DBCC TRACEON(3604) WITH NO_INFOMSGS GO DBCC OPTIMIZER_WHATIF(0) WITH NO_INFOMSGS; GO Note: Results dependent upon test server SQL Server version

Simulate the same CPU/Memory hardware from the production server using OPTIMIZER_WHATIF()

--40 cores? DBCC OPTIMIZER_WHATIF(1, 40); --Set amount of memory in MB, in this case 512GB DBCC OPTIMIZER_WHATIF(2, 524288);

--64 bit system? DBCC OPTIMIZER_WHATIF(3, 64);

Time to experiment!

 Tuning Without Data Demo  Tuning With Theoretical Hardware Changes