Year: 2022

  • Start of the traffic project

    So, I managed to gather about 1 GB of records from the pfsense installation and grab them from the box (filter.log files that you can find under /var/log).

    And I have a list of 16 logs that I need to concatenate.

    I had a lot of trouble concatenating it since I tried multiple times to use writelines() method from the file object.

    The code that worked for me:

    outputcsv = open('//Users//tudorsorin//Downloads//var//log//firewallrepo//filter.csv','w')
f = open(f'//Users//tudorsorin//Downloads//var//log//firewallrepo//filter.concat', 'r')
lines = f.readlines()
for line in lines:
    outputcsv.writelines(",".join(line.split(" ")[0:3])+","+line.split(" ")[-1])
f.close()
outputcsv.close()

    The idea is that it’s already in CSV format and all you need to do is to modify the “header” that normally looks like Feb 20 07:58:18 soaretudorhome filterlog[41546]: to something like Feb,20, 07:58:18, and the rest remains the same.

    Suprisingly, if you want to load it directly to a dataframe using pd.read_csv and you don’t force a header it works and I have all the data there with NaN in the fields that are not filled.

    After this is done, we can filter only traffic that is done over ppoe0 which is the WAN interface, and you can easily do that using temp = df[df[‘pppoe0’] == ‘pppoe0’]

    So far so good. I also took a look at a generic pppoe0 line and came to the conclusion that the colums that interest me are [0,1,2,21,22,23,24] which represent the date and source ip, destination ip and ports (source and destination). You can than filter the dataframe by temp = temp.iloc[:, [0,1,2,21, 22, 23, 24]]

    So we finally we have a dateframe that we can work with. Now what remains is to change the table header and try to enhance it with extra info.

    Cheers

    Sorin

  • Microsoft Teams blocked by pfBlockerNG

    Hi,

    One short tip to remember. I’ve been struggling for a while now with the fact that pfBlockerNG was blocking my Teams connection for whatever reason.

    I couldn’t understand what was the correct way to fix this until today. I should have known that there isn’t a range of IPs that can be whitelisted to make it work, and it’s related to the domain that was blocked.

    This became evident today when I took a look at the Reports tab and Alerts subtab and filtered by interface

    In order to fix it, you will need to go to DNSBL tab and expand TLD Exclusion List so that you can add the general domain that should be excluded.

    You could also whitelist each subdomain but since we are talking Microsoft, I think this is easier.

    The way this works, at least from what I understood, is that it will allow all of hostnames with the general domain and only block the ones that are specifically blacklisted.

    That would be all for today,

    Sorin

  • Python Kata on Codewars

    Hi,

    Since I pretty much broke the internet trying to solve the following “kata” with pieces of code, lets paste it also here cause it makes me proud.

    Here is the link to the kata: https://www.codewars.com/kata/5977ef1f945d45158d00011f

    And also here is my “solution” which took quite a long time to fix:

    def sep_str(st): 
    # your code here
    test_list = [[letter for letter in element] for element in st.split()]
    for a in test_list:
        a.extend([""] * (max(map(len, test_list)) - len(a)))
    if test_list:    
        result = [[test_list[j][i] for j in range(len(test_list))] for i in range(len(test_list[0]))]
    else:
        result = []
    return result

    That is all.

    Cheers!

  • Traffic statistics – new project

    Hi,

    For some time I wanted to understand how the traffic on my networking is actually shaped.

    To that purpose, at first I purchased a Synology router but it seems that it hasn’t that much traffic logging capabilities, so I kept it and put in front of it the following box.

    Product Page: 4 Port (2020)

    It’s a cool toy but ultimately I wanted to have Pfsense installed on it and logging activated so that I can gather as much data as possible.

    It’s now installed and hopefully it should be the start of some articles related to the data manipulation and also, maybe, some administration insights.

    Tnx,

    Sorin