More Related Content
What's hot
What's hot (20)
Viewers also liked
Similar to Indy pass writing efficient queries – part 1 - indexing
Similar to Indy pass writing efficient queries – part 1 - indexing (20)
Recently uploaded
Recently uploaded (20)
Indy pass writing efficient queries – part 1 - indexing
- 1. Writing Efficient Queries – Part 1Using SQL Server Internals to Improve Data AccessEddie WuerchMCT, MCITPPrincipal, Data ManagementExactTargeteddie@indydba.com
- 2. Disk I/O – Key PointsDisk I/O (reads and writes) is usually the slowest component of a systemCompare: Memory and CPU speeds are reported in GHz: billions of actions per secondDisk I/O rates are simply measured in IOPS: Input/output Operations Per SecondFast disks on mostly sequential workloads can get 100-150 IOPSBecause of disk access rates, our first tuning goal is to reduce overall I/O…so let’s look at those data pages, then examine how to process less of them
- 3. Disk I/O – Key Points - IISQL Server is not a black boxMost data is in structured storageThere are many ways to access the data, some ways are significantly faster – and cause less impact on other processes - than othersUnderstanding SQL Server data storage internals will guide you to the faster ways
- 4. Data Storage in SQL ServerThe base unit of SQL Server data storage is the pageAll data – system and user – is stored in pagesEach page is 8KB (8192 bytes)Pages are allocated from files in 64KB (8-page) extents
- 5. Page ProcessingPages are read from disk and processed in memory as an entire 8KB unitExtents are often read in from disk as a single block to reduce I/OAll data is processed in memory, pulled from disk first (processing put on hold) if data is not in memory
- 6. Page TypesSystem Page TypesSpace Management: File Header, PFS, GAM, SGAMChange Management: DCM, BCMData Page TypesIn-row data Index LOB data and Row-overflow dataAll pages are 8KB
- 7. In-Row Data Pages96-byte Page HeaderPage HeaderRow Data
- 9. Starts at 97th byteRow 1…Row 2…Row 3…Row 4…Row-offset table
- 10. Starts at end of page, moves backwards
- 11. Records first-byte offset of each row4… 3… 2… 1…
- 12. Disk Access MethodsThink of a phone book, with each entry as a recordOrdered by Last Name, First Name, MITwo ways to find a record:Use Last Name, First Name to find a number (Index Seek)Look through the entire phone book, one page at a time, scanning each row for data (Table Scan)
- 13. Clustered IndexRepresents the table itselfIndex specifies the physical ordering of that dataOnly 1 allowed per tableMay be unique, does not have to be the primary keyNon-clustered indexAdditional index of dataOver 200 allowedMay be uniqueIndex Types
- 14. The phone book exampleIf a table has a clustered index, the pointer to each row in the table is the clustered index keyThe leaf level of the nonclustered index contains the nonclustered keys and the clustered index keysNonclustered indexes may also include additional non-indexed columns, will be stored at the leaf level of the index
- 15. Index PagesA-K : Page 4L-U : Page 5V-Z : Page 6 456A : Page 22B : Page 23C : Page 24 D….Baa : Page 276Baba : Page 277Base : Page 278Ba…222324252627274275276277278279280281282283
- 16. Index Lookups
- 17. Index Lookups - revistedNonclustered indexSeparate trip through the clustered index for each ncl entry!Clustered index (table)
- 18. Operational Join TypesMerge JoinsHash JoinsLoop Joins
- 19. Join Type ComparisonLoopMergeHashWhen the other two can’t be used
- 20. One trip through one table
- 21. One trip through the other table for each entry in the first table
- 22. Generally the slowest of the three typesOne trip through each tableRequires indexes on both sides, at least one of them must be uniqueUsually the fastest join typeWorks well for very large joinsBuilds join data in tempdb
- 23. Join and Indexing TipsWhen defining an index, if the data is unique, then declare the index as uniqueJoin on keysProvide arguments in WHERE clauses to match available indexesCluster tables on range scansLook for covering indexes
- 24. So How Do I Know?SET STATISTICS IO ON
- 25. So How Do I Know?
- 26. So How Do I Know?sys.dm_db_index_usage_statsUser_seeksUser_scansUser_lookupsUser_updatesSys.dm_db_missing_index_*Not magic, has limitationsMany similar index entries with different INCLUDE statements may indicate a need to revisit the clustered index design
- 27. So How Do I Know?Scan-indicating waitsLots of PAGEIOLATCH_SH and PAGEIOLATCH_EX waits are generated by tables scans that read from diskCX_PACKET waits – related to parallellism often caused by scanning large tables (don’t reduce MAXDOP: fix the scan!)Other processes with SOS_SCHEDULER_YIELD or high signal wait times may be mitigated by reducing CPU load of scans
- 28. So How Do I Know?TempDB activity in instances without much use of temp tables or table variablesSELECT * FROM sys.dm_io_virtual_file_stats(DB_ID(‘TempDB’), NULL)Must track over time, perform time-slice analysisMay indicate additional worktable sort and hash-match activityTracking this for all of your databases shows the amount of I/O your systems are performing, and if the disk systems are keeping up
- 29. ResourcesMicrosoft White PapersSQL Server 2000 I/O Basics (http://technet.microsoft.com/en-us/library/cc966500.aspx)SQL Server I/O Basics, Chapter 2 (http://technet.microsoft.com/en-us/library/cc917726.aspx)SQL Server Waits and Queues (download) (http://technet.microsoft.com/en-us/library/cc966413.aspx)The Waits and Queues document is highly recommended tuning or analyzing workloads
- 30. ResourcesInside SQL Server Book SeriesSQL 2005The Storage Engine (Kalen Delaney)Query Tuning and Optimization (Delany, et. al.)T-SQL Querying (Ben-Gan, Kollar, Sarka)SQL 2008Microsoft SQL Server 2008 Internals (Delaney, Randal, Tripp, Cunningham)T-SQL Querying (Ben-Gan, Kollar, Sarka)