-
Apache Kafka technical logs to ELK stack
Morning, Just wanted to share with you the following article: https://www.elastic.co/blog/monitoring-kafka-with-elastic-stack-1-filebeat I will try to share my experience once i have all i need to start working on it. As far as i understood from managing Kafka for a while, monitoring is not enough, this is mandatory in order to manage you clusters in a…
-
Important considerations regarding Kafka MirrorMaker
Morning folks, Just wanted to share with you two things that i learned when playing with Kafka MirrorMaker this morning. First is that if you want to put MirrorMaker or other tool that uses Kafka on debug, all you have to do is to modify the tools-log4j.properties file from the config folder of you Apache…
-
List differences between two sftp hosts using golang
Hi, Just as a intermediate post as i wanted to play a little bit with golang, let me show you what i managed to put together in some days. I created a virtual machine on which i installed docker and grabbed a sftp image. You can try first two from Docker Hub, it should work.…
-
How to change root password on Debian – after vacation
Morning, Since i had a vacation and completely forgot all my passwords for Debian VM i fixed it using this article. Very useful! https://pve.proxmox.com/wiki/Root_Password_Reset Cheers!
-
Observer functionality for puppet zookeeper module
Morning, I know it’s been some time since i last posted but i didn’t had the time to play that much. Today i want to share with you the use case in which we needed to modify the module used for the deployment of zookeeper in order to include also observer role. The link that…
-
Memory debug by Heroku guys on Apache Kafka – nice one
Hi, I know, i should write more about my experience with Apache Kafka, have patience, it’s still building, but until then please check this article: https://blog.heroku.com/fixing-kafka-memory-leak Be aware of the things that you want to include in functionalities and code that is written beside Apache Kafka functionalities, it might get you in to trouble. I…
-
Docker statistics – way to investigate performance
Hi, I wish it would be mine but it isn’t. Quite good article from this week newsletter related to container stats from Docker containers: Analyzing Docker container performance with native tools Wish you an enjoyable read. Cheers!
-
Kafka limits implementation using puppet
Morning, I keep my promise and provide you with the two simple blocks that are needed to implement limits that we discussed in article http://log-it.tech/2017/10/16/ubuntu-change-ulimit-kafka-not-ignore/ For the limits module you can use: https://forge.puppet.com/puppetlabs/limits As for the actual puppet implementation, I took the decision not to restart the service immediately. This being said, it’s dead simple…
-
Ubuntu – change ulimit for kafka, do not ignore
Hi, Wanna share with you what managed to take me half a day to clarify. I just read in the following article https://docs.confluent.io/current/kafka/deployment.html#file-descriptors-and-mmap and learned that in order to optimize kafka, you will need to also change the maximum number of open files. It is nice, but our clusters are deployed on Ubuntu and the…
-
Kafka implementation using puppet at IMWorld Bucharest 2017
Hi, I recently had a presentation on how to deploy kafka using puppet and what do you need as a minimum in order to have success in production. Here is the presentation: Hope it is useful. Cheers! Update: There is also an official version from IMWorld which you can find here: And also the article…