-
Definitive guide to Kafka, confluent edition
Hi, No technical details today. Just wanted to share with you the Definitive guide to Kafka, book provided by our dear and esteem colleagues from Confluent https://www.confluent.io/wp-content/uploads/confluent-kafka-definitive-guide-complete.pdf Thank you, it should be an interesting read. Cheers!
-
Eyaml hiera configuration for puppet, as promised
Morning, We managed to configure also the hiera backend in order to have eyaml module active. It is related to the following past article http://log-it.tech/2017/05/29/install-eyaml-module-on-puppet-master/. So in the hiera.yaml you bassicaly need to add the following configuration before hierarchy: :backends: – eyaml – yaml – puppetdb and :eyaml: :datadir: /etc/puppetlabs/hieradata :pkcs7_private_key: /etc/puppetlabs/puppet/eyaml/private_key.pkcs7.pem :pkcs7_public_key: /etc/puppetlabs/puppet/eyaml/public_key.pkcs7.pem :extension:…
-
Python dictionary construction from process list
Hi, This is out of my expertise but i wanted to shared it anyways. One colleague wanted to help him with the creation of a pair key:value from one command that lists the processes, in python. With a little bit of testing i came to the following form: import os import subprocess from subprocess import…
-
Kafka problem that wasn’t a problem after all
Hi, Do not make my mistake from the last couple of weeks trying to connect to a “secured” kafka cluster using TLS. I wrote following article http://log-it.tech/2017/07/27/configure-kafka-truststore-keystore-using-puppet/ some time ago, and i know that it’s far from bullet proof but it does the job. Now let’s get to the subject, if you want to connect…
-
Configure Jupyter Notebook on Raspberry PI 2 for remote access and scala kernel install
Hi, This is a continuation of the previous article regarding Jupyter Notebook (http://log-it.tech/2017/09/02/installing-jupyter-notebook-raspberry-pi-2/) Let’s start with my modification in order to have an remote connection to it. It first needs a password in the form of password hash. To generate this pass run python cli and execute this code from IPython.lib import passwd;passwd(“your_custom_password”). Once you…
-
Installing Jupyter Notebook on Raspberry PI 2
Morning, Just want to share you that i managed to install the Jupyter Notebook(http://jupyter.org) on a Raspberry PI 2 without any real problems. Beside a microSD card and a Raspberry you need to read this and that would be all. So, you will need a image of Raspbian from https://www.raspberrypi.org/downloads/raspbian/ (i selected the lite version…
-
Implementing logrotate for kafka
Hi, Yes, we will need to implement also logrotate if we want to keep kafka under control. My solution was with puppet, as you probably expected. After i took a look on the documentation related to log4j properties i this i had a configuration figured out that should look like the following erb template #…
-
Adding custom kafka check consumer.lag to datadog from JMX
Hi, We had the necessity to add the consumer.lag check to datadog. Since we did not have access to the kafka.consumer domain which from what i believe it’s on the client side i decided to connect to the kafka node using JMX (so JConsole was the tool). At MBeans tab you will gladly see that…
-
Interesting insight in docker networking mechanism
Hi, This one is not mine, but it’s worth to mention. There are’s always interesting articles in the docker newsletter but i enjoyed very much this series and i highly recommend you to read it and try it: http://techblog.d2-si.eu/2017/04/25/deep-dive-into-docker-overlay-networks-part-1.html I would really want to work more with docker and hope in the future that i…
-
Securing kafka-manager endpoints with iptables rules behind traefik
Hi, One extra addition to my traefik balancing article from http://log-it.tech/2017/08/19/puppet-implementation-traefik-load-balancer-kafka-manager/ is that even so now we have the balancing capability we still need to restrict access to unsecured endpoint. I thought all the code to be deployable on all of the nodes. If this is taken in consideration, our issue with the firewall rules…