SQL Server -Service Broker - Reliable Messaging
•
2 likes•2,424 views
This document discusses how SQL Server Service Broker can help with scaling out databases by enabling reliable messaging between sharded database partitions. It covers concepts like sharding, partitioning, replication, and how Service Broker allows propagating messages between shards to maintain consistency when data is divided across multiple database instances.
Report
Share
More Related Content
What's hot
What's hot (20)
Similar to SQL Server -Service Broker - Reliable Messaging
Similar to SQL Server -Service Broker - Reliable Messaging (20)
More from Sperasoft
More from Sperasoft (20)
Recently uploaded
Recently uploaded (20)
SQL Server -Service Broker - Reliable Messaging
- 1. SQL Server Service Broker Scale out by Reliable Messaging
- 2. • Scalability – what does Scale Out means? • Sharding • Problems of sharding • How does Service Broker help? Agenda
- 3. • Is the ability of a system to handle growing amount of work in a capable manner • or its ability to be enlarged to accommodate that growth http://en.wikipedia.org/wiki/Scalability Scalability
- 4. • Scale up – scale vertically • Scale out – scale horizontally Scalability Methods
- 5. • means vertically • add resources to a single node in a system – CPU – memory – storage – etc Scaling Up
- 6. • means horizontally • add more nodes to a system • in databases that’s called Sharding • Microsoft used another term in SQL Azure - Federations Scaling Out
- 7. • Twitter Gizzard - https://github.com/twitter/gizzard • written in Scala Sharding Frameworks
- 8. Sharding involves two technologies: • Partitioning • Replication Sharding
- 9. • Data is divided into chunks • Stored across computers • Each chunk is small enough that we can manipulate and query efficiently User = “Denis” User = “Eva” Partitioning
- 10. • multiple copies of data are stored across several machines • efficiently responds to tons of queries • resilient to failures User = “Denis” User = “Eva” User = “Denis” User = “Eva” Replication
- 11. Besides Scaling Out: • Disaster Recovery strategies have You to make database backups • Replicated shards are working copies • Need a new backup? Replicate a new Shard! Benefits of Sharding
- 12. • Yes! Sharding is difficult! • We have to think of how to partition our data correctly • How to ensure that all copies of data are consistent? • Write conflicts – no guarantee that operations will apply in order Are there any Problems?
- 13. • “Shared Nothing Architecture” • no ongoing need to retain shared access between shards • each Shard can live in a totally separate – instance – database server – data center – continent Design Principle
- 14. • Orders table is partitioned by Region • Orders references to Users • Users is not partitioned, it is replicated • How to maintain Users in a consistent state? Users (not partitioned) Orders [Region = “US”] Users (not partitioned) Orders [Region = “Russia”] Partitioning Problem
- 15. Wall Statuses User [Name = “Denis”] Wall Statuses User [Name = “Eva”] Front-End POST Denis writes a new Status GET We should see that status on Eva’s wall How do we get Denis’ statuses? Facebook Sample
- 16. Wall Statuses User [Name = “Denis”] Wall Statuses User [Name = “Eva”] Front-End POST Denis writes a new Status POST Duplicate the status? No benefit from Shards then! Wrong! Possible Solution 1
- 17. Wall Statuses User [Name = “Denis”] Wall Statuses User [Name = “Eva”] Front-End POST Denis writes a new Status GET Gets a user data No benefit from Shards then! Wrong! Possible Solution 2
- 18. Wall Statuses User [Name = “Denis”] Wall Statuses User [Name = “Eva”] Front-End POST Denis writes a new Status POST Propogate the status GET All OK Real Solution
- 19. • custom application which reads from one Shard and writes to the others • add all Shards as Linked servers, write stored procedures to write a new record to each remote server • SQL Server Service Broker Possible Technologies
- 20. • allows Internal and External processes to send and receive guaranteed asynchronous messages • messages are sent to Queues in the same database, same instance, same server, another database, another instance, remote server Service Broker
- 21. the largest known implementation is in MySpace • 440 SQL Server instances • over 1,000 databases • 1 Petabyte of data (1 million gigabytes) Performance: • 5,000 messages/second in Labs • 18,000 messages/second in MySpace Implementations
- 22. • Always Transactional • Asynchronous • Queued • All messages are in XML format • Routing • Multicast messages in SQL Server 2012 Advantages
- 23. • No Administration Tools – not a lack - no at all • Queue is a Table in database – index fragmentation when we cannot empty queue • complicated T-SQL syntax and object model Disadvantages
- 24. • Message Types • Contracts • Queues • Services • Routes • Activation • Remote Service Bindings • Dialog Conversation • Conversation Groups Service Broker Concepts
- 25. • Dialog Security – between two Services – encrypts messages in a dialog conversation – provides the remote authorization • Transport Security – establishes an authenticated network connection between two Databases Service Broker Security
- 26. • Enable Service Broker at Database level ALTER DATABASE SampleDatabase SET ENABLE_BROKER; GO • Always enable Service Broker in MSDB system database in each SQL Server instance • Enable SQL Server to communicate to another instance at Database level ALTER DATABASE SampleDatabase SET TRUSTWORTHY ON; GO SQL Server Configuration
- 27. • Create Send Message type CREATE MESSAGE TYPE NewUserRequest VALIDATION = WELL_FORMED_XML; • Create Receive Message Type CREATE MESSAGE TYPE NewUserResponse VALIDATION = WELL_FORMED_XML; Message Types
- 28. • create Contract between two services CREATE CONTRACT NewUserContract ( NewUserRequest SENT BY INITIATOR, NewUserResponse SENT BY TARGET ); Service Contracts
- 29. CREATE QUEUE NewUserReceiveQueue WITH STATUS = ON, ACTIVATION ( PROCEDURE_NAME = OnReceiveNewUser, MAX_QUEUE_READERS = 5, Execute AS 'dbuser‘ ) ; CREATE QUEUE NewUserSendQueue WITH STATUS = ON; Queues
- 30. • Service will be listening for messages in a queue and react only on those which apply to the spicified contract CREATE SERVICE NewUserSendService ON QUEUE NewUserSendQueue ( [NewUserContract] ); CREATE SERVICE NewUserReceiveService ON QUEUE NewUserReceiveQueue ( [NewUserContract] ); Services
- 31. Sender: • begins a transaction • begins a dialog conversation from NewUserSendService to NewUserReceiveService • sends one or more XML messages to that dialog • commits a transaction • Waits for a Reply message • Processes Reply messages when arrived Sending a Messgae
- 32. Receiver (Target): • when a new message is delivered to Receiver's NewUserReceiveQueue the OnReceiveNewUser stored procedure is activated by Service Broker • Stored procedure begins a transaction • receives messages out of the NewUserReceiveQueue queue • sends Reply messages • ends a dialog conversation • Do what you want with that message • commits a transaction Receiving a Message