From the course: Complete Guide to Apache Kafka for Beginners

Unlock the full course today

Join today to access over 23,200 courses taught by industry experts.

[Advanced] max.block.ms and buffer.memory

[Advanced] max.block.ms and buffer.memory - Kafka Tutorial

From the course: Complete Guide to Apache Kafka for Beginners

Start my 1-month free trial Buy for my team

[Advanced] max.block.ms and buffer.memory

“

So finally, here's one lecture that you're probably not going to use in the beginning or at all in your Kafka life and I hope for you not to, but I'm still going to have it there. So it's advanced. If you want to skip it, that's fine and we're actually not going to implement it. So it's to describe two settings called max block millisecond and buffer memory. So if the producer becomes very, very, very high throughput and the broker cannot respond fast enough to these requests, the records are going to be buffered in memory on your producer. And the buffer size of buffer .memory is 32MB. It is decide the size of the send buffer, and that buffer is going to fill up over time when the producer when the broker is too busy. And then if the broker becomes less busy, it's going to empty back down when the throughput at the broker increases. So the idea is that this buffer is here to queue up messages before sending them into Kafka. Now we can obviously increase the buffer memory if we have…

Contents