Myth busters - performance tuning 101 2007

MythBusters - performance tuning 101 Paul Guerin Bachelor of Engineering (Computer) Oracle DBA Certified Professional DBA 10+ years Employers: Origin Energy, Bluescope Steel, BHP Billiton 21 November 2007

Topics Internet references Basics Cost Based Optimiser Cost Based Optimiser statistics Constraints Sorting Indexes SQL tuning and subquery factoring Partitions Partition pruning Coding practices Materialised Views and Parallelisation Other factors influencing performance Import and Export utilities

Internet references Amongst the best reference material is from Oracle: http://asktom.oracle.com http://tahiti.oracle.com http://www.oracle.com/technology/pub/articles/tech_dba.html Numerous other sources on the internet. http://www.mga.com.au http://www.dbspecialists.com http://www.hotsos.com http://www.singingsql.com http://www.oaktable.net/

Basics “ The objective of tuning a system is either to reduce the response time for end users of the system, or to reduce the resources used to process the same work.” The general goals in order are: Minimise the workload (i.e. SQL tuning). Minimise logical I/O (which in turn reduces physical I/O). Minimise sorting. Minimise output returned. Balance the workload. Execute jobs outside normal business hours (including materialised views). Parallelise the workload.

Cost Based Optimiser The database (Oracle, DB2, SQLserver) features an optimiser that predicts the quickest way that the data will be processed. The Cost Based Optimiser (CBO) supersedes the Rule Based Optimiser (RBO). Unlike the RBO, the CBO is not influenced by the order of tables and predicates in the SQL statement. For each SQL statement several execution plans are generated, and the execution plan estimated to have the lowest cost is chosen. The lowest cost execution plan may be comprised of index accesses or full table scans. View the lowest cost execution plan with: explain plan, autotrace, tkprof utility, or v$sql_plan.

Cost Based Optimiser Hints are available to influence the execution plan for an SQL statement. Every database we manage has an override on the nologging hint /*+ APPEND */ to allow: Recovery from a hot backup. Dataguard to function correctly.

Cost Based Optimiser statistics Statistics gathered by the DBMS_STATS package: System: average read time, average number of CPU cycles/sec, max I/O throughput. Table and columns: Number of rows, average row length, number distinct values Index: Average leaf blocks per key, average data blocks per key. Myth: Old statistics always lead to poor performance. Fact: If the data distribution of the object has not changed then the old statistics remain accurate. Myth: Recent statistics always lead to improved performance. Fact: If the data distribution of the object changes after the statistics are gathered then the new statistics become inaccurate. e.g1. gather statistics, then truncate the table. e.g2. truncate the table, gather statistics, insert rows. Fact: If the data distribution of the object has not changed then the new statistics will not lead to the CBO to choose a more efficient execution plan. Note1: Cached SQL is invalidated every time new object statistics are gathered. Note2: ANALYZE TABLE|INDEX will not gather CBO stats in future Oracle releases. Note3: Specify the MONITORING clause when creating tables.

Constraints CBO uses contraints to determine the best execution plans. e.g. use an index with the correct sort instead of sorting from scratch. Use column constraints instead of constraining in the code. Not Null (all columns should be Not Null constrained unless nulls are legitimate) Check Foreign key Primary Note: Create the index before creating the column constraint. Format for constraint: <tablename>_<columnname>_<constraintmnemonic> TIME_PK = PRIMARY KEY for the TIME table. TIM_DAY_NAME_NN = NOT NULL constraint for the TIME table DAY_NAME column. CUSTOMERS_COUNTRY_FK = FOREIGN KEY constraint for the COUNTRIES table COUNTRY_ID column.

Sorting Best practice is to avoid unnecessary sorting. Be aware of implicit sorting using the following clauses: DISTINCT, GROUP BY, UNION Prefer to use a UNION ALL clause instead of a UNION clause. UNION clause: append result sets together then remove duplicates via a sort. UNION ALL clause: only append result sets together – no sorting. If a sort is explicitly required then use an ORDER BY clause. Use ROWNUM to retrieve only the top rows of the sort: SELECT * FROM (SELECT … FROM … ORDER BY …) WHERE rownum<=5; Note: In 9i, the GROUP BY will implicitly sort in the GROUP BY column order where an appropriate index does not exist. However in 10g the GROUP BY will not sort implicitly where an appropriate index does not exist. Indexes can be created to minimise the workload by avoiding a sort….

Indexes Two main types: B-tree non-unique ascending (default) – does not index NULL values. Bitmap – does index NULL values. A b-tree index is often used to enforce data integrity. e.g. Primary key (or unique key) constraints. However a b-tree index may also improve SQL performance by reducing logical I/O and eliminating sorting. Foreign key - important if deleting or updating the primary key of the parent table. Predicate of a WHERE clause. ORDER BY clause providing the index is in the same sort order (ASC or DESC). GROUP BY clause providing the index is in the same sort order. MIN() or MAX() function: index can be ascending or descending (key can be anywhere in concatenated index).

Indexes Indexes to create will depend on how the data is accessed. Single column index ideally for predicates separated by OR operators. Concatenated column index ideally for predicates separated by AND operators: Leading column of the index must match a predicate for the index to be used. Table access is eliminated where the index includes all the columns in the SELECT and WHERE clauses. Column order dependent on WHERE clause: most selective predicates first (generally equalities before ranges). Function-based indexes CREATE INDEX income_ix ON employees(salary + (salary*commission_pct)); CREATE UNIQUE INDEX t_idx ON t ( CASE WHEN source_id IS NOT NULL THEN source_id END, CASE WHEN source_id IS NULL THEN name END );

Indexes Bitmap index (single or concatenated): Columns of low cardinality. i.e. few unique values. Good for the COUNT() function. Performs poorly when column experiences high DML.

Indexes Oracle recommends that unique indexes be created explicitly, and not through enabling a unique constraint on a table (Concepts pg10-30) Use the following naming format for: primary key index: <tablename>_PK unique index: <tablename>_<columnname>_UK non-unique index: <tablename>_<columnname>_IX bitmap index: <tablename>_<columnname>_BIX Note: Don’t unnecessarily create indexes as each one reduces performance for INSERTS, DELETES, and UPDATES of the indexed column.

SQL tuning Minimise the workload: Use a WHERE clause. Improve the selectivity of the WHERE clause. A selective predicate should result in an index access (if available). A non-selective predicate should result in a full table scan. When joining: Add predicates to eliminate many-to-many joins. Remove all unreferenced objects from the FROM clause. GROUP BY, DISTINCT, UNION – all can hide a cartesian product. Minimise rows returned: Do not use a wildcard (*) in the SELECT clause. Use the ROWNUM pseudo column to limit the result set returned. Only update when the value has changed. The 2 nd statement below is potentially faster. UPDATE TABLE … SET x=0; UPDATE TABLE … SET x=0 WHERE x<0 OR x>0; SQL that use the following are not able to use an index: Functions, including implicit type conversions (except if a function-based index exists) Leading wildcard character (%). NULL, <>, !=. SQL that use the following are able to use an index: =, >, <, LIKE, IN, ||, +, NVL Use >-9.99*POWER(10,125) instead of IS NOT NULL. Trailing wildcard character (%). Conditions that compare columns with constants. e.g. use salary>2000 instead of salary*12>24000

SQL tuning … WHERE SUBSTR(ACCOUNT_NAME,1,7) = 'CAPITAL'; -- Full-table scan only: function on the indexed column. … WHERE ACCOUNT_NAME LIKE 'CAPITAL%'; -- Index access possible. … WHERE TRUNC(TRANS_DATE) = TRUNC(SYSDATE); -- Full-table scan only: function … WHERE TRANS_DATE BETWEEN TRUNC(SYSDATE) AND TRUNC(SYSDATE) + .99999; -- Index access possible. … WHERE ACCOUNT_NAME || ACCOUNT_TYPE = 'AMEXA'; -- Full-table scan only: operator with the indexed column. … WHERE ACCOUNT_NAME='AMEX' AND ACCOUNT_TYPE = 'A‘; -- Index access possible. … WHERE AMOUNT + 3000 < 5000; -- Full-table scan only: operator with the indexed column. … WHERE AMOUNT < 2000; -- Index access possible.

SQL tuning … WHERE SUBSTR(ACCOUNT_NAME,1,7) = 'CAPITAL'; -- Full-table scan only: function on the indexed column. … WHERE salary*12 > 24000; -- Full-table scan only: operator with the indexed column. … WHERE salary > 24000/12; -- Index access possible as automatically simplified to: salary > 2000 … WHERE salary > 2000; -- Index access possible. … WHERE contract != 0; -- Full-table scan only: inequality used. … WHERE contract < 0 OR contract > 0; -- Index access possible. … WHERE status != ‘TRUE’; … WHERE status < ‘TRUE’ OR status > ‘TRUE’; -- Index access possible. … WHERE EMP_TYPE = ‘123’; -- Full-table scan only if implicit type casting occurs. … WHERE EMP_TYPE = 123; --

SQL tuning The identical parts of the following DML…. SELECT count(*) FROM all_objects, (select distinct owner username from all_objects ) owners WHERE all_objects.owner = owners.username UNION ALL SELECT count(*) FROM dba_objects, (select distinct owner username from all_objects ) owners WHERE dba_objects.owner = owners.username; … . Are placed in the WITH clause (subquery factoring): WITH owners AS ( select distinct owner username from all_objects ) SELECT count(*) FROM all_objects, owners WHERE all_objects.owner = owners.username UNION ALL SELECT count(*) FROM dba_objects, owners WHERE dba_objects.owner = owners.username;

Partitions A partitioned table can have partitioned or non-partitioned indexes. A non-partitioned table can have partitioned or non-partitioned indexes. A partition can have sub-partitions. Partitioned and non-partitioned index options: Local partitioned index on a partition of a table. Global partitioned index on any or all partitions of a table, and the index partitions can be independent of the partitioned table. e.g. global partitioned index on contract ID column of a partitioned table on date column. Global index on a whole table. i.e. simply an ordinary index.

Partition Pruning Partition pruning is the skipping of unnecessary index and data partitions or subpartitions in a query. Note: the optimizer cannot prune partitions if the SQL statement applies a function to the partitioning column (with the exception of the TO_DATE function). Similarly, the optimizer cannot use an index if the SQL statement applies a function to the indexed column, unless it is a function-based index.

Partition Pruning SELECT * FROM tx.hh_rtl WHERE day BETWEEN '01-OCT-2007' AND ‘31-OCT-2007'; -- Note: day is not a leading column of an index ----------------------------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Pstart | Pstop | ----------------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 1158 | 45162 | 399K (1)| | | | 1 | PARTITION RANGE ITERATOR | | | | | KEY | KEY | | 2 | TABLE ACCESS BY LOCAL INDEX ROWID| HH_RTL | 1158 | 45162 | 399K (1)| KEY | KEY | |* 3 | INDEX SKIP SCAN | HH_RTL$IDX2 | 901 | | 1374 (4)| KEY | KEY | ----------------------------------------------------------------------------------------------------------- SELECT * FROM tx.hh_rtl WHERE day BETWEEN TO_DATE('01-10-2007','dd-mm-yyyy') AND TO_DATE('31-10-2007','dd-mm-yyyy'); ----------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Pstart | Pstop | ----------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 60860 | 2317K| 11082 (1)| | | |* 1 | TABLE ACCESS FULL | HH_RTL | 60860 | 2317K| 11082 (1)| 100 | 100 | ----------------------------------------------------------------------------------------- SELECT * FROM tx.hh_rtl WHERE cnt_id=1; -- Note: tx.hh_rtl is partitioned by day --------------------------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Pstart | Pstop | --------------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 643 | 25077 | 342 (1)| | | | 1 | PARTITION RANGE ALL | | | | | 1 | 163 | | 2 | TABLE ACCESS BY LOCAL INDEX ROWID| HH_RTL | 643 | 25077 | 342 (1)| 1 | 163 | |* 3 | INDEX RANGE SCAN | PK_HH_RTL | 643 | | 332 (1)| 1 | 163 | ---------------------------------------------------------------------------------------------------------

Coding practices Use static SQL (preferably implicit) whenever possible. Also static PL/SQL already bind results in parse once, reuse many times: INSERT INTO t VALUES ( i ); Minimise SQL parsing by using bind variables in dynamic PL/SQL: Native dynamic SQL without binds results in a full hard parse every time: EXECUTE IMMEDIATE ‘INSERT INTO t VALUES (' || i || ')'; Native dynamic SQL with binds results in a hard parse but reuse execution plan: EXECUTE IMMEDIATE ‘INSERT INTO t VALUES (:x)' USING i; Use non-native dynamic SQL (i.e. DBMS_SQL package) over native (i.e. execute immediate) if executing the dynamic statement repeatedly. Use arrays over DBMS_SQL. Move SQL out of triggers into procedures. To reduce parsing: move the SQL into a package, and call the package from the trigger.

Coding practices Do as a single SQL statement, instead of procedurally. Use an exception instead of first checking for the existence of data i.e. count(*)>1. Commit only at transaction boundaries. e.g. at end of procedure. Sequences: Sequences eliminate serialization and improve the concurrency of your application. When an application accesses a sequence in the sequence cache, the sequence numbers are read quickly. Synonyms are for end users not for application schemas.

Coding practices - monitoring Label at start of transaction to identify entry in v$transaction: SET TRANSACTION NAME '<string>'; Populate the v$session and v$sql dynamic views: Module: dbms_application_info.set_module() Action: dbms_application_info.set_action() Client: dbms_application_info.set_client_info() Populate the v$session_longops dynamic view: dbms_application_info.set_session_longops()

Materialised Views and Parallelisation Remember the rules: Minimise the workload. Minimise logical I/O (which in turn reduces physical I/O). Minimise sorting. Minimise output returned. Balance the workload. Execute jobs outside normal business hours (including materialised views). Parallelise the workload. Materialised views do not minimise the workload, so are not a substitute for SQL tuning. Parallel processing does not minimise the workload, so is not a substitute for SQL tuning. Additionally, parallel processing is resource intensive which may impact other users.

Other factors influencing performance Larger blocksizes result in more efficient data reads. Maximum block size for Windows is 16kbytes. Cached data (logical I/O) is accessed quicker than from disk (physical I/O). Database performance is often slower after a cold backup. Row locking. Library cache latching. Object types other than tables and indexes can improve performance: Clustered objects. Index Organised Tables.

Other factors influencing performance Full table scans are influenced by the high water mark. Deleting or truncating does not reset the high water mark. Move the table to reset the high water mark. Note: Resetting the high water mark does not free space to the operating system. i.e. moving the table does not free space to the operating system.

Export and Import utilities Distributed database links are appropriate for small data migrations but don’t migrate metadata. Export and import utilities are best for migrating data and metadata. Export and import objects: Tables with all or some rows Tables without rows Indexes Grants Triggers Constraints Also available: Transportable tablespaces for migrating large data quickly. RMAN duplicates for migrating huge data quickly.

Minimise the workload by: Create indexes that reduce I/O and sorting. Partition large tables and indexes. Write SQL statements to permit index accesses and partition pruning. Use column constraints – especially Not Null. Use DBMS_STATS instead of ANALYZE. Only use materialised views and parallel processing as a last resort.

Myth busters - performance tuning 101 2007

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (9)

Similar to Myth busters - performance tuning 101 2007

Similar to Myth busters - performance tuning 101 2007 (20)

Recently uploaded

Recently uploaded (20)

Myth busters - performance tuning 101 2007