Pay attention to the size of ipsets for nftables

I was preparing an ipset for blocking access by geographical region using firewalld. Firewalld can work with either the traditional iptables or the newer nftables.

I created the ipset file using this Makefile stanza:

%.load: ipset_%.txt
        -firewall-cmd --permanent --delete-ipset=$(@:.load=)
        firewall-cmd --permanent --new-ipset=$(@:.load=) --type=hash:net
        firewall-cmd --permanent --ipset=$(@:.load=) --add-entries-from-file=$<
        systemctl reload firewalld

Here % is replaced by the two letter country code for the region I want to allow.

In the public zone I have a rich rule which inverts the test. This is the result of the firewall-cmd I issued:

  <rule family="ipv4">
    <source ipset="ok" invert="True"/>
    <service name="theservice"/>
    <log/>
    <drop/>
  </rule>

Later I discovered that that the countries not in ok weren't blocked. Troubleshooting found these symptoms:

• firewall-cmd --state returned failed, despite systemctl stating that the service had started
• There was a message about python-nftables failing in systemd logs
  
• nft list showed nothing in the chains or rulesets

Searching on these symptoms got hits but none of the solutions worked. The only thing I learnt is that firewalld calls the python3-nftables routines to manipulate the nftables. I thought it might be the top line in the Python script which read #!/usr/bin/python which invokes python2 on my system, but adding 3 made no difference presumably because firewalld calls it with /usr/bin/python3 <name of script>. Someone suggested this python3 module was broken in a particular distro release because it didn't include a needed flag so he went back to iptables as the backend. I didn't want to do this. Also the report was for a previous distro release so over a year old.

Playing around with ipset which deals directly with the kernel to create the sets manually revealed that the default maxelem of 65336 was not sufficient for my set. After some experimentation I figured that I had to increase this parameter at creation. This boiled down to including this final option on the firewall-cmd line:

firewall-cmd --permanent --new-ipset=$(@:.load=) --type=hash:net --option=maxelem=262144

Now restarting firewalld took a bit longer due to having to digest lots of entries, and there was no more failure of python-nftables. Inspecting the ruleset using nft worked and showed that the nftables rule was in place.

It would have been easier if I had understood the relationship of all the components of this setup better, but things are obvious in hindsight.