-
Integrate Kafka with Datadog monitoring using puppet
Hi, Since i was in debt with an article on how to integate Kafka monitoring using Datadog, let me tell you a couple of things about this topic. First of all, we are taking the same config of Kafka with Jolokia that was describe in following article. From the install of the brokers on our…
-
Automigrate data contained in a topic to different Kafka brokers
Hi, Let’s keep it short, you have a cluster composed of three Kafka brokers and you add another two. The details of doing that are pretty straight forward. If they are new, give them a unique broker.id and put the zoookeeper.connect string in order to know on which cluster to register. If they are already…
-
How to deploy Prometheus infrastructure for Kafka monitoring using puppet
Hi, In the last couple of days i worked on deployment of Prometheus server and agent for Kafka monitoring. In that purpose i will share with you the main points that you need to do in order to achieve this. First thing to do is to use the prometheus and grafana modules that you will…
-
Wrong Kafka configuration and deployment using puppet
Hi, I just want to share with you one case that we have last week and that involved some wrong kafka deplyment from puppet that actually filled the filesystems and got our colleagues that were using it for a transport layer on ELK. Lets start with the begining, we have some puppet code to deploy…
-
Small Mirror Maker test between different Kafka clusters
Hi, Today i am trying to show you what i have been playing with for the last day. There was a business case in which some colleagues from Analytics wanted to replicate all the data from other systems in their cluster. We will start with this, two independent configured clusters with 3 servers each (on…
-
Check Kafka JMX node stats using JConsole
Hi, As you probably know, Kafka is already publishing a lot of performance data on JMX to be collected. In order to do this, you will need to install jconsole (for Windows it’s already embedded in the jdk installation, for Linux you can use this article to check it out https://www.garron.me/en/linux/find-which-package-library-belongs.html. After you have done that,…
-
Monitoring Kafka with DataDog
Hi, A very interesting series of articles that should be checked regarding one option of Kafka monitoring with Datadog : https://www.datadoghq.com/blog/monitoring-kafka-performance-metrics/. I will have in the near future a task regarding this, will post the outcome when it’s done. P.S: It was done and you can find the implementation here : Integrate Kafka with Datadog…