Performance Tuning with DB Time - Oracle

1



Maximizing Database Performance: Performance Tuning with DB Time Kurt Engeleiter, John Beresniewicz, Cecilia Gervasio Oracle America

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle‟s products remains at the sole discretion of Oracle.

3

Agenda

• NEWS: DB Time is not news • DB Time Fundamentals • DB Time Method for Performance Tuning • DB Time Based Tools – ADDM – Enterprise Manager

4



NEWS: DB Time is Not News

5

DB Time is Pervasive, Familiar, Evolutionary • DB Time is the fundamental measure of Oracle performance – Arrived at progressively but designed-in from 10g

• You are likely using DB Time methods already – AWR and ASH reports – Enterprise Manager

• DB Time is performance analysis science matured – Wait event analysis, YAPP and tracing are all its ancestors

6

DB Time on the Database Home Page • Host CPU • Is host CPU-bound? Is this DB responsible?

• Active Sessions • DB Time accumulation by CPU, User I/O, Wait

7

DB Time on the Performance Page • Area under curve is equal to DB time • Wait class breakdown indicates solution space

8

DB Time on the Top Activity Page • Plots DB Time from ASH data • Shows top DB Time consumers by SQL and Session

DB Time

9

DB Time in AWR Report

• Time Model measures and accumulates DB Time through session instrumentation • DB Time and DB CPU most important • Also shows DB Time expended in various sub-operations

10

DB Time in AWR Report

• If you tune by Wait Events you are tuning by DB Time

• Top 5 Timed Events in the AWR report shows CPU time and non-idle waits for foregrounds

11

DB Time in the Grid Control DB Loadmap • Maps DB Time across the enterprise • Cell size proportional to DB Time

12



DB Time Fundamentals

13

Why Do We Care About Time?

• Human time is critical to the enterprise • Systems performance affects business goals – Human time + technology resource time

• “Time is money” • Performance improvement means doing things faster

Performance is always and only about time

14

DB Time Defined

• Total time in database calls by foreground sessions • Includes CPU time, IO time and non-idle wait time • DB Time is not called response time on purpose • Common currency for Oracle performance analysis Database time is total time spent by user processes either actively working or actively waiting in a database call. 15

Session-level Concepts Database Time (DB Time) = Total time session spends in all database calls

Active Session = Session currently spending time in a database call Average Activity of the Session (% Activity) = Ratio of time active to total wall clock time = Sum(Active Time) / Elapsed Time Browse books

Read reviews for a book

Add to cart

Checkout

TIME

= time spent in database

16

Multiple Sessions DB Time = Sum of DB Time over all sessions

Avg. Active Sessions = Sum of Avg. Activity over all sessions = Sum(DB Time) / Elapsed Time User 1 User 2 User 3

User n

TIME = time spent in database

17

Visualizing DB Time

User 1 User 2 User 3 User n

t0

t1

4 3 2 1

Active Sessions over time

TIME

18

Enterprise Manager Performance Page • Active Sessions by wait class over time • Colored area is the amount of DB Time

DB Time on User I/O is always blue

DB Time on CPU is always green

19



DB Time and System Performance

20

System Load and DB Time • More users More database calls DB Time increases

• Larger transactions More SQLs per call DB Time increases

DB time increases as system load increases.

21

System Performance and DB Time • I/O performance degrades I/O wait time increases DB Time increases

• Application performance degrades Longer elapsed time per SQL DB Time increases

DB time increases as performance degrades.

22

Host Performance and DB Time • Host is CPU bound Foregrounds accumulate active run-queue time Wait event times are artificially inflated DB Time increases

Tune for CPU before waits when CPU constrained

23

CPU Run-Queue and DB Time

Recorded wait time

Recorded wait time User 1

Run-queue Db file sequential read

Actual wait time

Run-queue On CPU

Db file sequential read

On CPU

Actual wait time

DB time is inflated when host is CPU-bound

24



The DB Time Method for Performance Tuning

25

The DB Time Method: Short Course

or database Determine where time isjust spent, reduce it! askand ADDM

26

The DB Time Method: Process

1. 2. 3. 4. 5.

Identify performance issue Scope the issue Set goals Data capture AWR handles this Investigate DB time distribution – Identify largest potential improvement

6. Modify system to tune for largest gain 7. Evaluate against goals Performance tuning by removing excess DB time 27

Identify and Scope the Problem • Big spike = big problem! • Container (host) or database issue?

28

Investigate DB Time Distribution • Identify excess DB time – Look for large skews in time that may represent inefficiencies

• System scope: – Resource limits – is problem outside the DB? – Resource contention (latches)

• Application scope: – Service, module, action – SQLID, rowsource – Resource contention (enqueues)

• Session scope: – Long-running SQL – Resource contention (holding enqueues in uncommitted txns)

29

Investigate DB Time distribution • Which wait classes are dominant? • “Top” lists show skew in key dimensions (SQL, session, file, etc.)

30

Identify Time-saving Modifications

• System scope: – Initialization parameter changes – Add resources or capacity

• Application scope: – – – –

SQL Tuning Advisor: SQL profiles Real-time SQL Monitoring Access Advisor: Physical schema layout Re-write SQL

• Session scope: – Kill session for immediate contention relief – Application design changes often long-term solution

31

Critical Success Factors

• Match solution scope to problem scope – Don‟t tweak optimizer parameters before tuning SQL

• Address root causes, not symptoms • Proceed iteratively one fix at a time – Concurrent fixes should be orthogonal

• Measure and validate results at each successive step • Stop when goals are met 32

Advantages of the DB Time Method

• Tunes exactly what affects users: – Time wasted in database

• Data capture not necessary: – „Always on‟ instrumentation framework (AWR, ASH) – No requirement to reproduce problem (unlike tracing)

• Superior coverage of all scope levels – Works for concurrency problems such as locking – Tight integration with kernel-engineered tools

• Maturation and extension of well-proven methodologies

33



DB Time Based Tools:

ADDM Enterprise Manager

34

Tools for Applying DB Time Method

Two use-cases, one method: 1. Tuning steady-state performance – Improve overall workload throughput or response time – Best practice: use ADDM

2. Diagnosing transient performance problems – Confirm and investigate reported performance issues – Best practice: use EM real-time screens

35

Example: Using ADDM • Users complain about bad performance • AWR report indicates some unusual issues on system • What to do?

36

What does the Internet advise? • Let‟s search on „enq: HW contention‟ • 9,000 hits and LOTS of discussion

37

Example: Following the links • Many hits are other people asking the same question – what do I do about this wait event?

38

Solution: ADDM in Enterprise Manager • Performance Page shows a spike of unusual activity • Click on the ADDM analysis icon

39

Solution: Using ADDM • ADDM performance findings are DB Time based • Ranked by impact = percent of DB Time affected • Click on the issue with the highest impact

40

Solution: Using ADDM • ADDM recommendations describe solutions for findings • EM presents „implement‟ actions where possible

41

Best Practice: Enterprise Manager Real-time Interface

• Transient (sub-hour) or immediate time scope – Requires interactivity of UI

• „Click on the big stuff‟ – Data visualizations display skew directly

• Takes some expertise to separate symptoms from root causes

42

Click on the Big Stuff – Follow the DB Time • Spike in DB Time signals possible performance issue • Click through to Top Activity for more details

43

Follow the DB Time – Top Activity Drilldown • Top Activity Page confirms problem and shows highly skewed DB Time by SQL ID. Click on top SQL ID for further details.

44

Click on the Big Stuff – Follow the DB Time • SQL Details confirms this SQL produced the spike in DB Time and shows parallel execution. Click on „SQL Monitoring‟ to see DB Time by execution plan steps.

45

Follow the DB Time – SQL Execution Details

46

DB Time Method Summary • DB time is the fundamental performance metric

• The method allows DB time analysis at many scopes – Proper scoping of problems and solutions is critical to success

• DB time based diagnosis removes value judgments – Scientific method, not sorcerer‟s magic

Performance improvement means doing the same work in less DB Time

47

Oracle Enterprise Manager 11g Resource Center Access Videos, Webcasts, White Papers, and More

Oracle.com/enterprisemanager11g

49

Additional Oracle Enterprise Manager Sessions Wednesday, Sept. 22

Location

• 10:00 a.m. - Manage the Manager: Diagnosing and Tuning Oracle Enterprise Manager

• Moscone S Rm 102

• 11:30 a.m. - Maximizing Database Performance: Performance Tuning with DB Time

• Moscone S Rm 104

• 11:30 a.m. - Make Upgrades Uneventful Using Oracle Enterprise Manager and My Oracle Support

• Moscone S Rm 310

• 12:30pm – Extracting Real Value from Your Data with Apache Hadoop

• Hilton Hotel, Plaza B

• 1:00 p.m.- Reducing the Risk of SOA Transactions

• Marriott Marquis, Salon 6

• 1:00 p.m. - SQL Tuning Roundtable with Oracle Gurus

• Moscone S Rm102

• 4:45 p.m.- Strategies for Monitoring Large Datacenters with Oracle Enterprise Manager

• Moscone S Rm102

• 4:45 p.m.- Oracle SOA Management Best Practices, Tips, and Techniques

• Moscone W L3, Rm 3018

• 4:45 p.m.- Oracle E-Business Suite Technology: Vision, Release Overview, Product Roadmap

• Moscone W L3, Rm 3002 / 3004

50

Additional Oracle Enterprise Manager Sessions Thursday, Sept. 23

Location

• 9:00 a.m. - Oracle WebLogic Server Management for Oracle DBAs

• Marriott Marquis, Salon 9

• 9:00 a.m. - Enabling Database as a Service Through Agile SelfService Provisioning

• Moscone S. Room 102

• 9:00 a.m. - Reduce TCO with Oracle Application Management Pack for Oracle E-Business Suite

• Moscone W L2, Rm 2024

• 10:30 a.m. - Best Practices for Managing Your PeopleSoft Applications

• Marriott Hotel, Golden Gate A

• 10:30 a.m. - Oracle Enterprise Manager Grid Control Deployment Best Practices

• Moscone S. Room 102

• 10:30 a.m. - Managing Sun SPARC Servers with Oracle Enterprise Manager Ops Center

• Moscone S. Room 252

• 10:30 a.m. - Heterogeneous Data Masking: Oracle, SQL Server, and DB2 Database Best Practices

• Moscone S. Room 306

• 12:00 p.m. - Scalable Enterprise Data Processing for the Cloud with Oracle Grid Engine

• Moscone S. Room 310

• 12:00 p.m. - Spot Problems Before Your Users Call: User Experience Monitoring for Oracle Apps

• Marriott Hotel, Golden Gate A

• 12:00 p.m. - Reduce Problem Resolution Time with Oracle Database 11g Diagnostic Framework

• Moscone S. Room 102

51

Additional Oracle Enterprise Manager Sessions Thursday, Sept. 23

Location

• 1:30 p.m. - Patching Enterprisewide Databases: Automation Techniques and Real-World Insights

• Moscone S. Room 310

• 1:30 p.m. - Managing User Experience: Lessons from eBay

• Marriott Hotel, Golden Gate A

• 1:30 p.m. - Deep Java Diagnostics and Performance Tuning: Expert Tips and Techniques

• Marriott Marquis, Salon 9

• 1:30 p.m. - Oracle Enterprise Manager Configuration Management Unleashed: Top 10 Expert Tips

• Marriott Marquis, Salon 6

• 1:30 p.m. - Oracle Enterprise Manager Security Best Practices

• Moscone S. Room 102

• 3:00 p.m - The X-Files: Managing the Oracle Exadata and Highly Available Oracle Databases

• Moscone S. Room 102

• 3:00 p.m. - Monitoring and Diagnosing Oracle RAC Performance with Oracle Enterprise Manager

• Moscone S. Room 310

52

Oracle Enterprise Manager Hands On Labs Monday September 20, 2010 3:30 p.m. - 4:30 p.m.

Database Performance Diagnostics and Tuning

Marriott Hotel, Salon 12/13, YB Level

5:00 p.m. - 6:00 p.m.

Provisioning, Patch Automation, and Configuration Management Pack

Marriott Hotel, Salon 12/13, YB Level

5:00 p.m. - 6:00 p.m.

Oracle Application Mgmt. Pack for Oracle E-Business Suite: Monitor/Clone

Marriott Marquis, Nob Hill

Tuesday September 21, 2010 11:00 a.m.-12:00 p.m.

Using Oracle Application Change Management Pack for Oracle E-Business Suite

Marriott Marquis, Nob Hill

12:30 p.m.-1:30 p.m.

Database and Application Testing

Marriott Hotel, Salon 12/13, YB Level

2:00 p.m. - 3:00 p.m.

Oracle Fusion Middleware Management

Marriott Hotel, Salon 12/13, YB Level

3:30 p.m. - -4:30 p.m.

Provisioning, Patch Automation, and Configuration Management Pack

Marriott Hotel, Salon 12/13, YB Level

Wednesday September 22, 2010 4:45 p.m. - 5:45 p.m.

Database and Application Testing

Marriott Hotel, Salon 12/13, YB Level

4:45 p.m. - 5:45 p.m.

Oracle Application Mgmt. Pack for Oracle E-Business Suite: Monitor/Clone

Marriott Marquis, Nob Hill

Thursday September 23, 2010 9:00 a.m. - 10:00 a.m.

Database Performance Diagnostics and Tuning

Marriott Hotel, Salon 12/13, YB Level

10:30 a.m. - 11:30 a.m.

Oracle Fusion Middleware Management

Marriott Hotel, Salon 12/13, YB Level

53

Oracle Enterprise Manager Demogrounds DEMO TITLE

L O C AT I O N

Oracle Real Application Testing: Database Replay

Moscone West

Oracle Real Application Testing: SQL Performance Analyzer

Moscone West

Self-Managing Database: Automatic Performance Diagnostics

Moscone West

Self-Managing Database: Automatic Fault Diagnostics

Moscone West

Self-Managing Database: Automatic Application and SQL Tuning

Moscone West

Application Quality Management: Application Testing Suite

Moscone South - S022

Real User Monitoring with Oracle Enterprise Manager

Moscone South - S021

Siebel CRM Application Management

Moscone South - S024

Real User Monitoring with Oracle Enterprise Manager

Moscone West

Oracle WebLogic Server Management and Java Diagnostics

Moscone West

SOA Management with Oracle Enterprise Manager

Moscone West

Oracle Business Transaction Management

Moscone West

Push Button Provisioning and Patch Automation

Moscone West

Smart Configuration Management

Moscone West

Oracle Enterprise Manager Ops Center

Moscone West

Managing the Enterprise Private Cloud

Moscone West

System Management, My Oracle Support, and Oracle Enterprise Manager

Moscone West

Self Managing Database: Change Management for DBAs

Moscone West

Oracle Enterprise Manager: Complete Datacenter Management

Moscone West

Self-Managing Database: Data Masking for DBAs

Moscone West

54

55