-
Correct SSL script for Kafka deployment
Hi, I wrote some time ago a post about certificate generation in order to secure Kafka cluster. Long story short, it was wrong! Here is the correct version that returns O (keystore is correctly generated and used) #!/bin/bash HOST= PASSWORD= KEYSTOREPASS= VALIDITY=365 keytool -keystore kafka.server.temp.keystore.jks -alias $HOST -validity $VALIDITY -genkey -dname “CN=${HOST}, OU=Myteam, O=Mycompany, L=Bucharest…
-
Kernel not compatible with zookeeper version
Morning, It’s important to share this situation with you. This morning i came to the office to see that a cluster that was upgraded/restarted had an issue with Zookeeper instances. Symptoms were clear: instances won’t start completely. But why? After a little bit of investigation, i went to the /var/log/syslog (/var/log/zookeeper did not contain any…
-
Kafka cluster nodes and controller using golang
Hi, Using the golang library for zookeeper from here you can get very easily the nodes that are registered in the cluster controller node. In order to install this module, beside needing to setup the GOPATH you will have also to install packages from linux distro repo called: bzr, gcc, libzookeeper-mt-dev Once all of this…
-
Delete corrupted Kafka topic version 2.0
Hi, We had in the past the situation described in this link use-case-deleting-corrupted-kafka-topic The situation repeated a little bit different this time. Taking a look on the list of topics, there were three topics marked for deletion. None of them had a Leader or Isr, so after a little bit of investigation the conclusion that…
-
Golang example for kafka service restart script
Hi, Not much to say, a pretty decent script for Kafka service restart(i tried to write it for our rolling upgrade procedure) that it’s still work in progress. If there are any changes that needed to be made to it, i will post it. Here is the script: Cheers
-
Fix under replicated partitions with controller restart
Hi, If you have a Kafka cluster with only one broker that has zero under-replicated partitions and the rest have a number that is not equal to that value, than please be aware that it is not properly registered to the cluster. Taking a look in the state-change log on the instance that it’s the…
-
Kafka service problem on upgrade to version 1.1.0
Hi, If you are using version 1.1.0 or want to upgrade to it, and the method is by puppet module provided from voxpopuli, please be aware of this issue. In the template used for the init script that it’s located under /etc/init.d/kafka and as you can also see on the latest version below: https://github.com/voxpupuli/puppet-kafka/blob/master/templates/init.erb There…
-
Non registered Zookeeper – why doesn’t it work?
Morning, If you ever deploy a server via puppet or other automation language that has also zookeeper installed and you already have a working cluster, please be aware of this. Yesterday i rebuilt a node multiple times (there were some errors to fix), and after finally getting it right, the zookeeper instance did not behave…
-
Use case for deleting corrupted Kafka topic
Hi, We had a week ago a case in which the client could not delete a topic from the cluster (Kafka version in this case was 1.0.0). When the topic was listed, there were no leaders assigned for the partitions. It was pretty clear that it would not delete it until we fixed it. First…
-
Log rotate for Kafka Garbage collect without restart
Morning, If you have a Apache Kafka version which is below 1.0.0 and you don’t have garbage collect rotate as shown here: with: -Xloggc:/opt/kafka/bin/../logs/kafkaServer-gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=100M without: -Xloggc:/opt/kafka/bin/../logs/kafkaServer-gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps One option is to modify the parameters in order to include them in the process that starts but…