-
Using line_profiler in Python
Currently I am reading one of the chapters from Python Data Science Handbook and saw that there are some magic commands that can be used in IPython. One of them gives you the possibility to time the execution and it’s called %timeit. Going further it seems that there are other commands like %prun but it…
-
Merging initial data load with IP info
Getting back to my traffic statistics project, I managed to play with aggregation of the two main collections that had the information I needed. First of all there is the normal code to load data from Mongo and check also the columns that can be used for the join. Which is available below After this,…
-
Loading unique IP’s in MongoDB
Hi, So today I played a little bit with the possibility of storing the unique IP addresses in a separate table. Since I will use a subscription from ip-api.com, it seems that there is an option to query info by batch processing with a limit of 100 IP’s per payload. So, at a first glance…
-
Loading data to a Mongo database for further processing
Hi, Since I needed my data to be available for further processing in a centralized manner, I have decided to store the first draft as well as further queries to location API in a Mongo database. Here is the short code snippet that was used for this task: From the concatenated file, my interest is…
-
Start of the traffic project
So, I managed to gather about 1 GB of records from the pfsense installation and grab them from the box (filter.log files that you can find under /var/log). And I have a list of 16 logs that I need to concatenate. I had a lot of trouble concatenating it since I tried multiple times to…
-
Microsoft Teams blocked by pfBlockerNG
Hi, One short tip to remember. I’ve been struggling for a while now with the fact that pfBlockerNG was blocking my Teams connection for whatever reason. I couldn’t understand what was the correct way to fix this until today. I should have known that there isn’t a range of IPs that can be whitelisted to…
-
Python Kata on Codewars
Hi, Since I pretty much broke the internet trying to solve the following “kata” with pieces of code, lets paste it also here cause it makes me proud. Here is the link to the kata: https://www.codewars.com/kata/5977ef1f945d45158d00011f And also here is my “solution” which took quite a long time to fix: That is all. Cheers!
-
Traffic statistics – new project
Hi, For some time I wanted to understand how the traffic on my networking is actually shaped. To that purpose, at first I purchased a Synology router but it seems that it hasn’t that much traffic logging capabilities, so I kept it and put in front of it the following box. It’s a cool toy…
-
Enable time sync on Manjaro
So I wanted for a while to use and to learn Manjaro and I grabbed Cinnamon 21.1.0 Installation process is pretty straight forward, I setup the correct time zone and installed all of the default packages. Guess what, after rebooting the laptop the timezone was set correctly but the actual time was way off. I…
-
SASL config issue on latest Kafka versions
Hello, Today I want to share with you a problem that we needed to fix when we decided to activate SASL. Normally, the steps are pretty straight forward and you can use Confluent doku or the general Apache Kafka. The main catch is that if you have a certain property in your config file, the…