From the course: Complete Guide to Apache Kafka for Beginners
Unlock the full course today
Join today to access over 23,200 courses taught by industry experts.
Consumer delivery semantics - Kafka Tutorial
From the course: Complete Guide to Apache Kafka for Beginners
Consumer delivery semantics
Hi, this is Stephane from Conduktor and so now let's have a look at delivery semantics in Apache Kafka. So we have first at most once and at most once is when offsets are committed as soon as the message is received and if the processing goes wrong, then the messages will be lost because they won't be read again. So let's have an example. So we are reading a batch from our consumer, from our consumer group, and then right after reading this batch, we commit the offsets. Then we start to process data, for example, sending an email. So we send it for this one, this one and this one. And then all of a sudden this consumer from the consumer group goes away. What happens is that this measured message and this message are not being processed because the consumer crashed before processing them. So the consumer restarts and when it restarts, it's going to read from where the data was less committed. That means that the data is going to be read from here, here and so on. So that means that…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Contents
-
-
-
-
-
-
-
-
-
-
-
(Locked)
OpenSearch consumer: Project overview52s
-
(Locked)
OpenSearch consumer: Project setup3m 32s
-
(Locked)
Setting up OpenSearch on Docker2m 38s
-
(Locked)
Setting up OpenSearch on the Cloud1m 57s
-
OpenSearch 1014m 26s
-
(Locked)
OpenSearch consumer implementation: Part 17m 26s
-
(Locked)
OpenSearch consumer implementation: Part 29m 34s
-
(Locked)
Consumer delivery semantics3m 16s
-
(Locked)
OpenSearch consumer implementation: Part 3 idempotence5m 29s
-
(Locked)
Consumer offsets commit strategies4m 38s
-
(Locked)
OpenSearch consumer implementation: Part 4 delivery semantics4m 52s
-
(Locked)
OpenSearch consumer implementation: Part 5 batching data4m 20s
-
(Locked)
Consumer offset reset behavior2m 6s
-
(Locked)
OpenSearch consumer implementation: Part 6 replaying data2m 57s
-
(Locked)
Consumer internal threads5m
-
(Locked)
Consumer replica fetching: Rack awareness2m 50s
-
(Locked)
-
-
-
-
-