From the course: Complete Guide to Apache Kafka for Beginners

Unlock the full course today

Join today to access over 23,200 courses taught by industry experts.

Log cleanup policies

Log cleanup policies - Kafka Tutorial

From the course: Complete Guide to Apache Kafka for Beginners

Start my 1-month free trial Buy for my team

Log cleanup policies

“

Hi, this is Stephane from Conduktor and welcome to this section on understanding log cleanup policies. So we've seen that Kafka topics have data that expire and according to your policy and the concept of making data expire is called log cleanup. So we have two policies in Kafka we'll have a look at. The first one is log cleanup policy equals delete, which is the default for all the users topics and is when we delete data based on the age of the data and by default it is a week. So what I've told you before is that, hey, after a week your data in Kafka is gone. This is because of this log cleanup policy delete. You can also delete your logs based on the max size of a log. We'll see this in the next lecture. By default, it's minus one. That means that the size can be infinite, but you only have one week of data. The second one is log cleanup policy compact, and that is actually the default for a specific internal Kafka topic named the consumer offsets topic. So if we have a Kafka…

Contents