From the course: Complete Guide to Apache Kafka for Beginners
Unlock the full course today
Join today to access over 23,200 courses taught by industry experts.
Consumer internal threads - Kafka Tutorial
From the course: Complete Guide to Apache Kafka for Beginners
Consumer internal threads
Hi, this is Stephane from ConduKtor and here is an advanced lecture. I don't blame you if you don't hang on till the end, that describes how consumer work in Kafka and what can happen when you run with them for a long time. So you have a consumer group application with three consumers and the consumers in a consumer group they talk to something called a consumer group coordinator, and there is an acting broker, and this is used to detect whether or not your consumers are still up. So there is a heartbeat mechanism, okay, and there's a poll mechanism. So the heartbeat thread is going to be your consumers sending messages to the broker once in a while saying they're still alive. Okay. And the poll thread is going to be other brokers thinking your consumers are still alive because there are still requesting data from Apache Kafka. So these two mechanisms together capture a lot of issues that can happen with your consumers. And so we'll have a look at these in details. Okay. But overall…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Contents
-
-
-
-
-
-
-
-
-
-
-
(Locked)
OpenSearch consumer: Project overview52s
-
(Locked)
OpenSearch consumer: Project setup3m 32s
-
(Locked)
Setting up OpenSearch on Docker2m 38s
-
(Locked)
Setting up OpenSearch on the Cloud1m 57s
-
OpenSearch 1014m 26s
-
(Locked)
OpenSearch consumer implementation: Part 17m 26s
-
(Locked)
OpenSearch consumer implementation: Part 29m 34s
-
(Locked)
Consumer delivery semantics3m 16s
-
(Locked)
OpenSearch consumer implementation: Part 3 idempotence5m 29s
-
(Locked)
Consumer offsets commit strategies4m 38s
-
(Locked)
OpenSearch consumer implementation: Part 4 delivery semantics4m 52s
-
(Locked)
OpenSearch consumer implementation: Part 5 batching data4m 20s
-
(Locked)
Consumer offset reset behavior2m 6s
-
(Locked)
OpenSearch consumer implementation: Part 6 replaying data2m 57s
-
(Locked)
Consumer internal threads5m
-
(Locked)
Consumer replica fetching: Rack awareness2m 50s
-
(Locked)
-
-
-
-
-