-
Kafka_consumer.yaml (python style) and more
Hi, As a followup to the article i posted earlier ( https://log-it.tech/2019/03/15/get-the-info-you-need-from-consumer-group-python-style/ ) , you can use that info to put in into kafka_consumer.yaml for Datadog integration. It’s not elegant by any means, but it works. As an advise, please don’t over complicate thinks more than they need. In the last example i figured i…
-
Get the info you need from consumer group (python style)
Hi, For some of you might be of help. If it’s rubbish, i truly apologize, please leave a comment with improvements 🙂 I will not explain it. It should be self explanatory. Cheers
-
Kafka consumer group info retrieval using Python
Hi, I’ve been playing with kafka-python module to grab the info i need in order to reconfigure Datadog integration. Unfortunately, there is a catch also on this method. And i will show you below. Here is a little bit of not so elegant code. As you will see, print(meta[3]) will return a very ugly binary…
-
Fact for kafka_consumer hash…or kind of
Hi, There is a late requirement that we activate the kafka_consumer functionality of Datadog. Unfortunately this is a challenge if you don’t have a fixed number of consumer groups and topics (on one client we had a couple of hundreds consumer groups) Here is how it should look in the example file So, if you…
-
Python for opening and reading files
Since I started learning Python and hopefully take also a certification, I am trying to do some hands-on. Nothing too complicated, just some basic exercises, for the moment. Here is one of them. They want to though you to open the and file and read from it like: But there is a thing not to…
-
Distributing service conditionally on OS version
Hi, Since we are in the process of migrating to 16.04, my service restart script needed to be deployed with separate builds. In that purpose, i found a fact that would help me, so that my standard file block transformed into this: case $facts[‘os’][‘distro’][‘codename’] { ‘xenial’: { file {“/root/servicerestart”: source => ‘puppet:///modules/profiles/servicerestart-kafka-new’, mode => ‘0755’,…
-
Order Linux processes by memory usage
This one is more for me actually. We have some issues with one puppet instance on which the processes fail, and i wanted to see if there is any way to order them by memory usage. So i searched the net and found this link https://unix.stackexchange.com/questions/92493/sorting-down-processes-by-memory-usage The command is like And it provides you with…
-
Golang logging using USER profile on Mint 19
Hi, I committed on learning Golang and as a part of this task i came to play with logging examples. It seems that if you user syslog.LOG_USER the info is stored in the /var/log/syslog. Here is the code and also the output package main import ( “io” “log” “log/syslog” “os” “path/filepath” ) func main() {…
-
Small go code example for zookeeper resource editing
Hi, We have the task of “service restart coordination” for our Apache Kafka cluster. It’s still a work in progress but if you want to use the zookeeper for some status verification and update, something like this will work as an example. package main import ( “fmt” “io” “launchpad.net/gozk” “os” “strings” “sync” “time” ) const…
-
Don’t delete the Kafka GC logs when they are used
Hi, I made a mistake some time ago, and it’s there to hunt me. Deleting the normal gc logs including the one it’s already used doesn’t solve anything, it just created a more difficult situation. Here is my example: /dev/sda1 50G 42G 5.2G 90% / /opt/kafka/logs# ll total 34M drwxrwxr-x 2 kafka kafka 4.0K Oct…