From the course: Complete Guide to Apache Kafka for Beginners

Unlock the full course today

Join today to access over 23,200 courses taught by industry experts.

Case study: Logging and metrics aggregation

Case study: Logging and metrics aggregation - Kafka Tutorial

From the course: Complete Guide to Apache Kafka for Beginners

Start my 1-month free trial Buy for my team

Case study: Logging and metrics aggregation

“

So one of the first use cases of Apache Kafka was to do logging and metrics application. So all these applications will send it into Kafka. And then in this kind of deployment, we usually want like high throughput and you have less restriction regarding data loss or replication of data. And so this is why you used to have like acks equals zero and lower replication factor and so on. So then from the logging metrics aggregation, then you're going to send your data into your favorite logging solution, such as Splunk, CloudWatch, the ELK stack and so on. So the architecture is simple. It looks like this. You have Kafka in the middle and you will create, for example, topics for your application logs and other topics for your metrics. Then your application would have some kind of log forwarder, which is a producer that will send all the logs into Apache Kafka. And so this is a very high throughput type of producer. And then you also have metrics collectors. This is also some kind of…

Contents