-
Adding custom kafka check consumer.lag to datadog from JMX
Hi, We had the necessity to add the consumer.lag check to datadog. Since we did not have access to the kafka.consumer domain which from what i believe it’s on the client side i decided to connect to the kafka node using JMX (so JConsole was the tool). At MBeans tab you will gladly see that…
-
Interesting insight in docker networking mechanism
Hi, This one is not mine, but it’s worth to mention. There are’s always interesting articles in the docker newsletter but i enjoyed very much this series and i highly recommend you to read it and try it: http://techblog.d2-si.eu/2017/04/25/deep-dive-into-docker-overlay-networks-part-1.html I would really want to work more with docker and hope in the future that i…
-
Securing kafka-manager endpoints with iptables rules behind traefik
Hi, One extra addition to my traefik balancing article from http://log-it.tech/2017/08/19/puppet-implementation-traefik-load-balancer-kafka-manager/ is that even so now we have the balancing capability we still need to restrict access to unsecured endpoint. I thought all the code to be deployable on all of the nodes. If this is taken in consideration, our issue with the firewall rules…
-
Fixing the kafka-manager puppet code
Hi, we have a new code version for kafka-manager deploy. I will not give more details, just that now it also has a fact for the kafka-password and also some minor changes. Fact looks like this: require ‘facter’ Facter.add(:kafka_manager_pass) do setcode do file=’/etc/kafka-manager/application.conf’ if File.exist?(file) kafka_manager_pass = Facter::Core::Execution.exec(“cat #{file} | grep basicAuthentication.password | cut -d’=’…
-
Fixing the keystore/trustore distribution code
Hi, There is an extra thing to be added to my article http://log-it.tech/2017/07/27/configure-kafka-truststore-keystore-using-puppet/ As is the code copies the files at each puppet run to the other nodes which not contain the keystore generation code. And to fix this i used yet again another puppet module that should share data between the nodes, you can…
-
Puppet implementation of traefik load balancer for kafka-manager
Hi, It’s time to give the puppet implementation for the traefik small case. It is related to the following article http://log-it.tech/2017/08/08/balancing-requests-kafka-manager-using-traefik/ Starting from that i tried to find a puppet module that can actually install the package more or less accurate and i found this https://forge.puppet.com/praekeltfoundation/traefik Now, for the service install it works, but for defining of…
-
Install puppet gems on puppet master using hiera
Morning, I needed to install a toml-rb gem in order to my puppet traefik module to work and i just want to short post my workaround on doing that automatically. There was some code in our repo for that but it used only hiera array, don’t really know, so i had to write a very…
-
Balancing requests to kafka-manager using traefik
Hi, Just wanted to share with you a quite small and simple config to balance the traffic between three machines that have kafka-manager installed. For this i used traefik since it was new to me and i wanted to gain a little bit of experience with it. It’s an interesting solution but it took me…
-
Configure kafka truststore and keystore using puppet
Hi, Since the last article was about the template needed to generate the truststore and keystore, now it’s time to give you the rest of the fragments for the deployment with puppet. So, We will have the kafka_security gen class that was called on the last post and it should look like this in the…
-
Generate Kafka key/trust store for TLS activation
Morning folks, Just wanted to share with you also the code for script required on generation of keystore and truststore for Kafka. It is not much but it might be helpful someday. So, the erb script should look like this, and i will show you as well how you can integrate this also with puppet:…