-
Delete corrupted Kafka topic version 2.0
Hi, We had in the past the situation described in this link use-case-deleting-corrupted-kafka-topic The situation repeated a little bit different this time. Taking a look on the list of topics, there were three topics marked for deletion. None of them had a Leader or Isr, so after a little bit of investigation the conclusion that…
-
Golang example for kafka service restart script
Hi, Not much to say, a pretty decent script for Kafka service restart(i tried to write it for our rolling upgrade procedure) that it’s still work in progress. If there are any changes that needed to be made to it, i will post it. Here is the script: Cheers
-
Fix under replicated partitions with controller restart
Hi, If you have a Kafka cluster with only one broker that has zero under-replicated partitions and the rest have a number that is not equal to that value, than please be aware that it is not properly registered to the cluster. Taking a look in the state-change log on the instance that it’s the…
-
Kafka service problem on upgrade to version 1.1.0
Hi, If you are using version 1.1.0 or want to upgrade to it, and the method is by puppet module provided from voxpopuli, please be aware of this issue. In the template used for the init script that it’s located under /etc/init.d/kafka and as you can also see on the latest version below: https://github.com/voxpupuli/puppet-kafka/blob/master/templates/init.erb There…
-
IBM MQ crtmqm instance issue
Morning, Short notice. If you are trying to create a queue manager and receive following error: AMQ8101: WebSphere MQ error (893) has occurred. and you also have lots of space in you qmgrs and log directory, don’t look further than that. In some cases it happens because the install in /var/mqm is not complete. The…
-
Command to start sysdig container – redundant but useful
Hi, This is more like a easier way to find the command without searching the net: docker run -it –rm –name=sysdig –privileged=true \ –volume=/var/run/docker.sock:/host/var/run/docker.sock \ –volume=/dev:/host/dev \ –volume=/proc:/host/proc:ro \ –volume=/boot:/host/boot:ro \ –volume=/lib/modules:/host/lib/modules:ro \ –volume=/usr:/host/usr:ro \ sysdig/sysdig The actual command on starting a sysdig container. I will get more in depth with some Kafka cluster aggregated…
-
Non registered Zookeeper – why doesn’t it work?
Morning, If you ever deploy a server via puppet or other automation language that has also zookeeper installed and you already have a working cluster, please be aware of this. Yesterday i rebuilt a node multiple times (there were some errors to fix), and after finally getting it right, the zookeeper instance did not behave…
-
Use case for deleting corrupted Kafka topic
Hi, We had a week ago a case in which the client could not delete a topic from the cluster (Kafka version in this case was 1.0.0). When the topic was listed, there were no leaders assigned for the partitions. It was pretty clear that it would not delete it until we fixed it. First…
-
Log rotate for Kafka Garbage collect without restart
Morning, If you have a Apache Kafka version which is below 1.0.0 and you don’t have garbage collect rotate as shown here: with: -Xloggc:/opt/kafka/bin/../logs/kafkaServer-gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=100M without: -Xloggc:/opt/kafka/bin/../logs/kafkaServer-gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps One option is to modify the parameters in order to include them in the process that starts but…
-
Consumer group coordinator in Kafka using some scala script
Morning, Just a small post regarding returning a consumer group coordinator for a specific consumer group. We had the issue that consumer groups are re-balancing and we didn’t knew if it’s related to application logic or the Consumer Group Coordinator was changing and the Kafka cluster was reassign a different one each time. So, a…