This document discusses SQL query performance analysis and optimization. It covers key topics like indexes, different types of scans (table scan, index scan, index seek), clustered vs non-clustered indexes, covering indexes to optimize queries, and using statistics to help the query optimizer generate efficient execution plans. The goal is to help analyze and tune queries to achieve fast index seeks and optimize performance.
The document contains descriptions of 4 photographs taken by Berk Schneider including locations and camera settings. The first photo shows a light ribbon effect from a flashlight during nighttime photography outside Fondren Library. The second is a close-up of a mountain flower from Colombia. The third details a golden orb spider's web in a Galveston bird sanctuary. The fourth was taken from a moving bus in Sao Paulo, Brazil and features reflective light creating an interesting mood.
LocalEqual: Crowdsource your international growthGuillaume Ang
LocalEqual helps businesses expand abroad by connecting entrepreneurs with skilled locals around the world. Their process involves designing an internationalization campaign with challenges to identify market opportunities, assess local characteristics, and engage qualified local profiles. Locals can submit solutions to win recognition, money, and potentially a franchise by providing local answers to the challenges. This dual approach allows businesses to grow while giving motivated locals entrepreneurial opportunities in their own markets.
The document discusses database normalization through third normal form. It describes eliminating repeating groups and redundant data by creating separate tables for each set of related data identified by a primary key. These tables should have no non-key attributes that are functionally dependent on other non-prime attributes. The document also covers one-to-one, one-to-many, and many-to-many relationships and discusses whether an application is better suited for online transaction processing or online analytical processing.
The document discusses how data is stored and organized in Microsoft SQL Server. It explains that rows in a table will always appear in the order they were inserted by default. It also describes how data is stored across 8KB pages that are grouped into extents for storage management, and how rows cannot span multiple pages. The document also provides an overview of different types of JOINs between tables like INNER, LEFT OUTER, RIGHT OUTER, and FULL OUTER JOINs.
1. Script backup
2. Recovery models
3. Backup type(Full,differential, transactional)
4. Transactional logs
5. Point in time restore
6. Transactional log shipping
7. Recovery of deleted data without any backup
Hugo Baeta from Automattic gave a presentation on how WordPress.com is made. He discussed what WordPress is, how Automattic makes money, their goal of democratizing publishing. He explained that Automattic is located in San Francisco and has loosely grouped teams with an open source attitude. Hugo also covered how WordPress.com is made through talent, passion, autonomy, simple tools, community support and learning from user feedback.
The query optimizer in SQL Server is cost-based and determines the optimal query plan by estimating the cost of different query plans based on cardinality estimates derived from statistics, the cost model for different query operations, and the total estimated execution time. Statistics are important for the query optimizer to generate high-quality query plans, and the optimizer monitors when statistics may be out of date and automatically updates statistics.
Korean culture has a rich tradition of dance, painting, and music. Traditional Korean dances include court dances performed at banquets as well as folk dances featuring masks. Painting styles have been influenced by Buddhism and use ink on paper or silk with natural landscapes as popular subjects. Music is divided into traditional Korean genres and Western styles, with folk music maintaining set rhythms and modes. Popular traditional musical forms include Pansori vocal music, Pungmul percussion music, and Sanjo shifting rhythms and modes.
This document discusses motivation for adult learners. It begins by outlining the learning objectives which are to explain the differences between andragogy and pedagogy, identify characteristics of adult learners, discuss the six factors that motivate adult learners, and discuss three critical learning strategies for instructors. It then provides background on Malcolm Knowles and his development of the concept of andragogy. It outlines the key differences between pedagogy and andragogy, characteristics of adult learners, the six factors that motivate adult learning, tips for effective instructors, and implementing adult learning principles in distance education.
This document discusses database indexing. It provides information on the benefits of indexes, how to create indexes, common misconceptions about indexing, and rules for determining when and how to create indexes. Key points include that indexes improve performance of queries by enabling faster data retrieval and synchronization; indexes should be created on columns frequently filtered in WHERE and JOIN clauses; and the order of columns in an index matters for its effectiveness.
The document discusses SQL query performance analysis. It covers topics like the query optimizer, execution plans, statistics analysis, and different types of queries and indexes. The query optimizer uses cost-based optimization to determine the most efficient execution plan. Statistics help it estimate cardinality and choose good plans, so keeping statistics up to date is important. The goal is to have queries use indexes through seeks instead of scans whenever possible to improve performance.
SQL Server supports two main types of indexes - clustered and nonclustered. A clustered index physically orders the data on disk based on the index key. Only one clustered index is allowed per table. A nonclustered index contains key values and row locators but does not determine the physical order of data. SQL Server supports up to 999 nonclustered indexes per table. The choice of index depends on the query patterns against the table and the desired performance characteristics.
This document discusses various techniques for optimizing SQL queries in SQL Server, including:
1) Using parameterized queries instead of ad-hoc queries to avoid compilation overhead and improve plan caching.
2) Ensuring optimal ordering of predicates in the WHERE clause and creating appropriate indexes to enable index seeks.
3) Understanding how the query optimizer works by estimating cardinality based on statistics and choosing low-cost execution plans.
4) Avoiding parameter sniffing issues and non-deterministic expressions that prevent accurate cardinality estimation.
5) Using features like the Database Tuning Advisor and query profiling tools to identify optimization opportunities.
This document introduces different types of indexes in SQL Server, including clustered, non-clustered, full text, XML, and columnstore indexes. It discusses common misconceptions about indexes, downsides to indexes, and advanced index tuning techniques like fillfactor and padding. The document provides examples of SQL statements for creating indexes and querying index usage statistics.
This document discusses indexing and query optimization in SQL Server. It provides an overview of indexes including clustered and non-clustered indexes. It describes how data is stored at the page and extent level and differences between tables with and without clustered indexes. The document also outlines the query optimization process including parsing, optimization, execution and the cost-based optimizer. Finally, it reviews common execution plan operators like table scans, index scans and seeks and when they would be considered good or bad.
The document discusses various techniques for performance tuning a database including indexing strategies, query optimization, and hardware upgrades. It provides details on different types of indexes like B-Trees, bitmap indexes, and hash indexes. The summary should recommend indexing on high-cardinality fields that are frequently queried, using the query optimizer to evaluate execution plans, and reviewing hardware needs.
Optimizing Query is very important to improve the performance of the database. Analyse query using query execution plan, create cluster index and non-cluster index and create indexed views
Lec 1 = introduction to structured query language (sql)Faisal Anwar
This document provides an introduction to structured query language (SQL). It outlines the basic commands and functions of SQL for data administration and manipulation. SQL functions are divided into two categories: data definition language for creating database objects and tables, and data manipulation language for inserting, updating, deleting, and retrieving data. The document also covers topics such as data types, constraints, indexes, joins, and aggregate functions.
This document introduces structured query language (SQL) and its basic commands and functions. It discusses how SQL is used for data administration, such as creating tables, indexes, and views, and for data manipulation like adding, modifying, deleting, and retrieving data. The two broad categories of SQL functions are data definition language for creating database objects and data manipulation language for inserting, updating, deleting, and retrieving data from tables.
This document provides an introduction to structured query language (SQL). It outlines the basic commands and functions of SQL for data administration and manipulation. These include using SQL to create tables, indexes and views; and to add, modify, delete and retrieve data. The document also discusses SQL queries, constraints, indexes, joins and aggregate functions to extract useful information from databases.
The document provides biographical information about Antonios Chatzipavlis, a SQL Server expert and evangelist. It then summarizes his presentation on statistics and index internals in SQL Server, which covers topics like cardinality estimation, inspecting and updating statistics, index structure and types, and identifying missing indexes. The presentation includes demonstrations of analyzing cardinality estimation and picking the right index key.
This document provides guidance on optimizing database performance through techniques like indexing, query tuning, avoiding unnecessary operations, and following best practices for objects like stored procedures, triggers, views and transactions. It emphasizes strategies like indexing frequently accessed columns, avoiding correlated subqueries and unnecessary joins, tuning queries to select only required columns, and keeping transactions and locks as short as possible.
Data Never Lies Presentation for beginners in data field.pptxTusharAgarwal49094
This document provides an overview of data concepts including:
1. It explores core data concepts such as structured, semi-structured, and unstructured data.
2. It examines roles and responsibilities in data including database administrators.
3. Key concepts of relational data are described like tables, normalization, and indexes.
4. Non-relational data types like key-value stores and document databases are explored.
5. Data analytics concepts such as data ingestion, processing, ETL, and visualization techniques are covered.
How oracle query works (The SQL Optimizers)Hosein Zare
The document discusses how Oracle's query optimizer works and provides tips for writing optimized SQL queries. It covers the rule-based versus cost-based optimizers, common problems like incorrect driving tables, issues with indexes like low cardinality fields, and how to check index selectivity. The document advises analyzing table statistics, writing queries to leverage indexes, avoiding functions in where clauses, and testing on real data rather than execution plan estimates.
Microsoft SQL Server Performance Query Tuning focuses on execution plans and indexes. Execution plans detail how queries will be processed, including index usage and join methods. Common elements include scans, seeks, lookups, nested loops, hash and merge joins, and aggregations. Indexes provide efficient access paths between users and data. Clustered indexes store data in sorted order while nonclustered indexes reference data locations. Tips include limiting indexes, avoiding updates in indexes, and creating indexes for query predicates.
This document provides an overview of optimizing MySQL queries. It discusses optimization at the database and hardware levels, understanding query execution plans, using EXPLAIN to analyze queries, optimizing specific query types like counts and groups, indexing strategies like covering indexes, and partitioning tables for performance. The goal is to help readers write efficient queries and properly structure databases and indexes for high performance.
This document provides an overview of a presentation on building better SQL Server databases. The presentation covers how SQL Server stores and retrieves data by looking under the hood at tables, data pages, and the process of requesting data. It then discusses best practices for database design such as using the right data types, avoiding page splits, and tips for writing efficient T-SQL code. The presentation aims to teach attendees how to design databases for optimal performance and scalability.
SQL is a standard language for accessing and manipulating databases. It allows users to retrieve, insert, update, and delete data as well as create or modify database objects like tables, indexes, and views. Although SQL standards exist, implementations from different database vendors can vary while still supporting core commands. SQL is always used with an RDBMS which structures data into tables with rows and columns to enable relationships between data. Common SQL commands include SELECT, UPDATE, DELETE, INSERT, and CREATE.
This document provides an overview of basic database concepts and terminology. It discusses why databases are useful for storing and organizing large amounts of information, and outlines some key database concepts like records, fields, tables, keys, and Boolean logic. It also provides guidance on how to develop a sample database structure and discusses common database tasks like searching and filtering records.
This document discusses the key challenges and design of a distributed graph index service. It proposes distributing the index across partitions, with each partition storing an independent index. Index keys are designed to encode vertex and edge data partition ID, type and properties. The document outlines writing new data to indexes, choosing the optimal index for queries, scanning partitions in parallel, and rebuilding indexes offline by replaying all operations. Maintaining consistency between indexes and data across versions and partitions is a primary challenge discussed.
This document provides an introduction and overview of Microsoft SQL Server and its main components: the Database Engine, SQL Server Integration Services (SSIS), SQL Server Analysis Services (SSAS), and SQL Server Reporting Services (SSRS). It also briefly discusses the differences between SQL Server and MySQL Server and their respective client tools. The remainder of the document focuses on SQL syntax, explaining clauses such as FROM, WHERE, GROUP BY, HAVING, and ORDER BY and functions like COUNT, AVG, MIN, MAX and SUM. It emphasizes that the order of predicates in the WHERE clause can impact performance.
Microsoft SQL Server is a relational database management system that stores data in tabular format of columns and rows. It has four major components: the Database Engine for storing and retrieving data efficiently; Integration Services (SSIS) for extracting, transforming, and loading data from various sources into destinations; Analysis Services (SSAS) for creating multi-dimensional data structures and performing analysis and data mining; and Reporting Services (SSRS) for generating reports from different tables in various formats like PDF and maps. The SELECT statement is used to retrieve data and has clauses like FROM, WHERE, GROUP BY, HAVING, and ORDER BY. Proper use of filters and indexes in the WHERE clause is important for query performance.
This document discusses SQL query performance analysis. Addhoc queries are non-parameterized queries that SQL Server treats as different statements even if they only differ by parameters. Prepared queries avoid this issue by using parameters. The query optimizer determines the most efficient execution plan based on criteria like cardinality and cost models. Execution plans and contexts are cached to improve performance. Examples are provided showing how join can outperform subquery, order in the WHERE clause matters, and how same outputs can have different execution plans.
The document discusses several key concepts in SQL Server including:
1. Data in tables is stored in pages by default in the order records are inserted (natural order).
2. Pages are the basic unit of storage which are grouped into extents of 8 contiguous pages.
3. The ORDER BY clause allows ordering the results of a SELECT statement in ascending or descending order.
SQL Server stores data in pages that are grouped into extents for management. By default, rows are returned in the order they were inserted. A nonclustered index will use less space than a clustered index since it only stores the key columns rather than the entire row. The ORDER BY clause is used to sort query results. Various types of joins can be used to combine data from multiple tables.
This document discusses four topics related to database queries: ad hoc queries, execution plans, statistics analysis, and deadlock analysis. Ad hoc queries are queries that are not pre-defined while execution plans show how the database will execute a particular query. Statistics analysis examines data distribution and deadlock analysis identifies locking issues between concurrent queries.
Retrieval Augmented Generation Evaluation with RagasZilliz
Retrieval Augmented Generation (RAG) enhances chatbots by incorporating custom data in the prompt. Using large language models (LLMs) as judge has gained prominence in modern RAG systems. This talk will demo Ragas, an open-source automation tool for RAG evaluations. Christy will talk about and demo evaluating a RAG pipeline using Milvus and RAG metrics like context F1-score and answer correctness.
Self-Healing Test Automation Framework - HealeniumKnoldus Inc.
Revolutionize your test automation with Healenium's self-healing framework. Automate test maintenance, reduce flakes, and increase efficiency. Learn how to build a robust test automation foundation. Discover the power of self-healing tests. Transform your testing experience.
Choosing the Best Outlook OST to PST Converter: Key Features and Considerationswebbyacad software
When looking for a good software utility to convert Outlook OST files to PST format, it is important to find one that is easy to use and has useful features. WebbyAcad OST to PST Converter Tool is a great choice because it is simple to use for anyone, whether you are tech-savvy or not. It can smoothly change your files to PST while keeping all your data safe and secure. Plus, it can handle large amounts of data and convert multiple files at once, which can save you a lot of time. It even comes with 24*7 technical support assistance and a free trial, so you can try it out before making a decision. Whether you need to recover, move, or back up your data, Webbyacad OST to PST Converter is a reliable option that gives you all the support you need to manage your Outlook data effectively.
Generative AI technology is a fascinating field that focuses on creating comp...Nohoax Kanont
Generative AI technology is a fascinating field that focuses on creating computer models capable of generating new, original content. It leverages the power of large language models, neural networks, and machine learning to produce content that can mimic human creativity. This technology has seen a surge in innovation and adoption since the introduction of ChatGPT in 2022, leading to significant productivity benefits across various industries. With its ability to generate text, images, video, and audio, generative AI is transforming how we interact with technology and the types of tasks that can be automated.
How UiPath Discovery Suite supports identification of Agentic Process Automat...DianaGray10
📚 Understand the basics of the newly persona-based LLM-powered Agentic Process Automation and discover how existing UiPath Discovery Suite products like Communication Mining, Process Mining, and Task Mining can be leveraged to identify APA candidates.
Topics Covered:
💡 Idea Behind APA: Explore the innovative concept of Agentic Process Automation and its significance in modern workflows.
🔄 How APA is Different from RPA: Learn the key differences between Agentic Process Automation and Robotic Process Automation.
🚀 Discover the Advantages of APA: Uncover the unique benefits of implementing APA in your organization.
🔍 Identifying APA Candidates with UiPath Discovery Products: See how UiPath's Communication Mining, Process Mining, and Task Mining tools can help pinpoint potential APA candidates.
🔮 Discussion on Expected Future Impacts: Engage in a discussion on the potential future impacts of APA on various industries and business processes.
Enhance your knowledge on the forefront of automation technology and stay ahead with Agentic Process Automation. 🧠💼✨
Speakers:
Arun Kumar Asokan, Delivery Director (US) @ qBotica and UiPath MVP
Naveen Chatlapalli, Solution Architect @ Ashling Partners and UiPath MVP
Keynote : AI & Future Of Offensive SecurityPriyanka Aash
In the presentation, the focus is on the transformative impact of artificial intelligence (AI) in cybersecurity, particularly in the context of malware generation and adversarial attacks. AI promises to revolutionize the field by enabling scalable solutions to historically challenging problems such as continuous threat simulation, autonomous attack path generation, and the creation of sophisticated attack payloads. The discussions underscore how AI-powered tools like AI-based penetration testing can outpace traditional methods, enhancing security posture by efficiently identifying and mitigating vulnerabilities across complex attack surfaces. The use of AI in red teaming further amplifies these capabilities, allowing organizations to validate security controls effectively against diverse adversarial scenarios. These advancements not only streamline testing processes but also bolster defense strategies, ensuring readiness against evolving cyber threats.
Top 12 AI Technology Trends For 2024.pdfMarrie Morris
Technology has become an irreplaceable component of our daily lives. The role of AI in technology revolutionizes our lives for the betterment of the future. In this article, we will learn about the top 12 AI technology trends for 2024.
"Hands-on development experience using wasm Blazor", Furdak Vladyslav.pptxFwdays
I will share my personal experience of full-time development on wasm Blazor
What difficulties our team faced: life hacks with Blazor app routing, whether it is necessary to write JavaScript, which technology stack and architectural patterns we chose
What conclusions we made and what mistakes we committed
The Challenge of Interpretability in Generative AI Models.pdfSara Kroft
Navigating the intricacies of generative AI models reveals a pressing challenge: interpretability. Our blog delves into the complexities of understanding how these advanced models make decisions, shedding light on the mechanisms behind their outputs. Explore the latest research, practical implications, and ethical considerations, as we unravel the opaque processes that drive generative AI. Join us in this insightful journey to demystify the black box of artificial intelligence.
Dive into the complexities of generative AI with our blog on interpretability. Find out why making AI models understandable is key to trust and ethical use and discover current efforts to tackle this big challenge.
The History of Embeddings & Multimodal EmbeddingsZilliz
Frank Liu will walk through the history of embeddings and how we got to the cool embedding models used today. He'll end with a demo on how multimodal RAG is used.
It's your unstructured data: How to get your GenAI app to production (and spe...Zilliz
So you've successfully built a GenAI app POC for your company -- now comes the hard part: bringing it to production. Aparavi addresses the challenges of AI projects while addressing data privacy and PII. Our Service for RAG helps AI developers and data scientists to scale their app to 1000s to millions of users using corporate unstructured data. Aparavi’s AI Data Loader cleans, prepares and then loads only the relevant unstructured data for each AI project/app, enabling you to operationalize the creation of GenAI apps easily and accurately while giving you the time to focus on what you really want to do - building a great AI application with useful and relevant context. All within your environment and never having to share private corporate data with anyone - not even Aparavi.
2. What is an index ?
• Index is a way to organize data to make
searching, sorting and grouping fasters
• we need indexing when :
1. WHERE, ON, HAVING clause (Searching)
2. ORDER BY clause (Sorting)
3. GROUP BY clause (Grouping) etc.
3. Table scan:
SELECT * FROM Student WHERE RollNo = 111
Time complexity of table scan is : O(n)
RollNo Name Country Age
101 Greg UK 23
102 Sachin India 21
103 Akaram Pakistan 22
107 Miyabi China 18
108 Marry Russia 27
109 Scott USA 31
110 Benazir Banglades 17
111 Miyabi Japan 24
112 Rahul India 27
113 Nicolus France 19
4. Types of Index
• Table without any index is called Heap
• There are two type of index:
1. Clustered index
2. Non-Clustered index
5. Clustered index
• When we create a clustered index on any
table physical organization of table is changed.
• Now data of table is stored as binary search
tree(B tree).
CREATE UNIQUE [CLUSTERED] INDEX <Name>
ON <ObjectName>(
<ColumnName> [ASC | DESC ] [ ,...n ]
)
7. Types of scanning
• Table scan: It is very slow can and it is used only if table has
not any clustered index.
• Index scan: It is also slow scan. It is used when table has
clustered index and either in WHERE clause non-key columns
are present or query has not been covered (will discuss later)
or both.
• Index Seek: It is very fast. Our goal is to achieve this.
8. Clustered index
• If we create table with primary key, sql server
automatically creates clustered index on that
table
• A table can have only one clustered index .
• Physical order of rows of table is same as
logical order of key columns of clustered
index.
9. Terms of execution plan
• Predicate: It is condition in WHERE clause which is either non-
key column or column which has not been covered.
• Object: It is name of source from where it getting the data. It
can be name of table, Clustered index or non-clustered index
• Output list: It is name of the columns which is getting from
object.
• Seek Predicate: It is condition in WHERE clause which is either
key column or fully covered.
10. Non-clustered index
• It is logical organization of data of table. A non-clustered index
can be of two types.
1. Heap
2. Based on clustered index.
• If table has clustered index then leaf node of non-clustered
index keeps the key columns of clustered index.
• If the table has not any clustered index then leaf node of non-
clustered index keeps RID which unique of each row of table.
13. Covering of queries
• We can specify maximum 16 column names.
• Sum of size of the columns cannot be more than 900 bytes.
• All columns must belong to same table.
• Data type of columns cannot be ntext, text,
varchar (max), nvarchar (max), varbinary (max), xml, or image
• It cannot be non-deterministic computed column.
14. Statistics Analysis
• The query optimizer uses statistics to create query plans that
improve query performance
• A correct statistics will lead to high-quality query plan.
• Auto create and updates applies strictly to single-column
statistics.
• The query optimizer determines when statistics might be out-
of-date by counting the number of data modifications since
the last statistics update and comparing the number of
modifications to a threshold.
15. Goal
• Should we use sub query or inner join?
• Should we use temp table or table variable?
Other tools:
• Sql query profiler
• Database Tuning Advisor
• Resource Governor