-
Puppet gems install workaround after TLS 1.0 switchoff
Hi, It seems that since Ruby disabled the TLS 1.0 protocol, there is an issue with installing custom gems in the puppet server. If you run puppetserver gem environment you will probably see the following output: /opt/puppetlabs/bin/puppetserver gem environment RubyGems Environment: – RUBYGEMS VERSION: 2.4.8 – RUBY VERSION: 1.9.3 (2015-06-10 patchlevel 551) [java] – INSTALLATION…
-
Error 127, not related to Puppet or Golang
Hi, Something from my experience playing with Golang and Puppet code this morning. I wrote a very very simple script to restart a service that you can find here Today i wanted to put it on the machine and run it with puppet, so i wrote a very small class that looked like this: class…
-
Multiple classes block declaration in hiera will not work
Morning, Do not add multiple classes in hiera like this: — classes: – profiles::datadogagent – profiles::updatekafka kafka::security: true kafka::security_default: true kafka::heap_size: 2048 classes: – profiles::pybackuplogs – profiles::group_coordinator Class updatekafka will not be executed. The structure should look like: — classes: – profiles::datadogagent – profiles::updatekafka – profiles::pybackuplogs – profiles::group_coordinator kafka::security: true kafka::security_default: true kafka::heap_size: 2048 Cheers!
-
Log rotate for Kafka Garbage collect without restart
Morning, If you have a Apache Kafka version which is below 1.0.0 and you don’t have garbage collect rotate as shown here: with: -Xloggc:/opt/kafka/bin/../logs/kafkaServer-gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=100M without: -Xloggc:/opt/kafka/bin/../logs/kafkaServer-gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps One option is to modify the parameters in order to include them in the process that starts but…
-
Observer functionality for puppet zookeeper module
Morning, I know it’s been some time since i last posted but i didn’t had the time to play that much. Today i want to share with you the use case in which we needed to modify the module used for the deployment of zookeeper in order to include also observer role. The link that…
-
Kafka limits implementation using puppet
Morning, I keep my promise and provide you with the two simple blocks that are needed to implement limits that we discussed in article http://log-it.tech/2017/10/16/ubuntu-change-ulimit-kafka-not-ignore/ For the limits module you can use: https://forge.puppet.com/puppetlabs/limits As for the actual puppet implementation, I took the decision not to restart the service immediately. This being said, it’s dead simple…
-
Kafka implementation using puppet at IMWorld Bucharest 2017
Hi, I recently had a presentation on how to deploy kafka using puppet and what do you need as a minimum in order to have success in production. Here is the presentation: Hope it is useful. Cheers! Update: There is also an official version from IMWorld which you can find here: And also the article…
-
Eyaml hiera configuration for puppet, as promised
Morning, We managed to configure also the hiera backend in order to have eyaml module active. It is related to the following past article http://log-it.tech/2017/05/29/install-eyaml-module-on-puppet-master/. So in the hiera.yaml you bassicaly need to add the following configuration before hierarchy: :backends: – eyaml – yaml – puppetdb and :eyaml: :datadir: /etc/puppetlabs/hieradata :pkcs7_private_key: /etc/puppetlabs/puppet/eyaml/private_key.pkcs7.pem :pkcs7_public_key: /etc/puppetlabs/puppet/eyaml/public_key.pkcs7.pem :extension:…
-
Implementing logrotate for kafka
Hi, Yes, we will need to implement also logrotate if we want to keep kafka under control. My solution was with puppet, as you probably expected. After i took a look on the documentation related to log4j properties i this i had a configuration figured out that should look like the following erb template #…
-
Securing kafka-manager endpoints with iptables rules behind traefik
Hi, One extra addition to my traefik balancing article from http://log-it.tech/2017/08/19/puppet-implementation-traefik-load-balancer-kafka-manager/ is that even so now we have the balancing capability we still need to restrict access to unsecured endpoint. I thought all the code to be deployable on all of the nodes. If this is taken in consideration, our issue with the firewall rules…