Lets add some firewalling stuff to #iocaine!

...but what should I call the module/struct that helps me do that? It's going to interface with NFT, so something fungi-adjacent might do. "Canesten" might be an option, but that's probably a trade mark, and "Clotrimazole" doesn't sound all that great.

So I started thinking along another line: the firewall keeps the crawlers away, it's a strong defense system, but one that needs maintenance. And... it kinda maybe feels like "Vaccine" might do as a name.

Like many other names in iocaine, this will hopefully upset just the right kind of people.

@algernon
not uglymonkey?

Hitting a small wall: neli has partial support for Netfilter stuff. Finishing full support has been on its roadmap since... 2020, and keeps getting pushed to later milestones.

This partial support makes it a little complicated to figure out how to combine the existing pieces and write the missing ones.

I guess it doesn't help that I'm not familiar with the netlink protocol either.

It feels like implementing the netlink bits myself would be less trouble in the short term. But... that doesn't feel great.

@algernon
idk if this helps, but with wireshark you can sniff how other programs use netlink
https://mstdn.io/@wolf480pl/115673339022832995

@wolf480pl Not that NFT, the other NFT! I don't want to even mention the monkey one.

@wolf480pl Hm. It might! Thanks!

@algernon
sorry, I had to

Myeah. I think I'm gonna go without neli and will be netlinking on my own. Easier to figure out what to do following a combination of wireshark captures and C code than half-finished Rust.

Once I have something working in place, I can transition it to neli later, I guess.

But, before I start randomly implementing things, lets see how the Roto-side of it would look! Yesterday I had some plans, lets iterate from there.

Firewall.ensure_filter_set("table_name", "set_name", options)?;

This would:

• Create table_name if it doesn't exist.
• Create set_name within the table, if it doesn't exist.
• Create filter_${set_name}_v4 (and _v6) chains, with appropriate rules.

This means iocaine would own the chains, but the table and the sets can be manipulated from outside too, including adding additional chains. This, in turn, would allow one to create a chain that runs before iocaine's filter chains, gotos to another chain, to allow-list certain IPs, or otherwise bypass the filters.

But for simplicity's sake, iocaine will need to own its chains. Firewall.ensure_filter_set() would go as far as deleting these chains if they exist, and re-creating them.

...which in turn highlights another shortcoming of neli: it doesn't support batched requests, something that'd be almost essential here, at setup time.

Here, options would let one configure things like a timeout, set size, whether to use a counter, and the priority of the chains.

I might even make that parameter an Option, with sensible defaults (24h timeout, 1M size, 0 prio, counters enabled). Yes, 1M is a sensible size here, I'm basing this on the current set size on Eru (723k IPv4, 1620 IPv6 addresses). It would be a lot less if I banned CIDRs, but that's complicated to maintain.

To add a new item:

Firewall.block_ip("table_name", "set_name", "address");

Unlike ensure_filter_set(), this always succeeds, because it will add to the set async, and return immediately.

Before I embark on doing this on my own... I'm gonna go and have another look at nftnl. Not pure rust, but... I wouldn't have to implement the whole netlink stuff myself.

But before I do that, lets start at the top, and implement the Roto/Lua interface!

I'm half-tempted to do this:

# init
let firewall =
  Firewall.ensure_filter_set(
    "iocaine", "genUrl", FirewallOptions.default()
  )?;
globals.add("FIREWALL_GENURL", firewall.into_global())?;

# later in main
FIREWALL_GENURL.block_ip(request.header("x-forwarded-for"));

However, this would require fiddling with Globals, and that's a pain point, something I want to rework after 3.3.

Not opposed to doing it for 3.3, but... will stick to the simpler implementation first.

I think I figured out how to use nftnl, so I won't need to implement the netlink stuff myself.

I can create a table so far, so... progress!

Though, the nftnl using AsRef<cstr> is a bit... bleargh. But I can live with that.

Lovely. nftnl's set wrapping is incomplete.

I'm about to flip some tables.

Ho-humm. I might be able to complement it with nftnl-sys. All is not lost yet!

@algernon

The more consonants in a CLI's name without having any vowels, the better. Like ncmpdcpp!

@amin

Why do things need to be difficult.

Like... I find it hard to believe I'd be the first one with a wish to fiddle with nftables (including sets) from Rust, without having to shell out to the nft binary.

And yet, here we are: we have a pure-rust netlink crate that has partial support for netfilter, and no batching. We have low-level bindings for libnft & libmnl which appear to be complete, but they're low-level, very much non-ergonomic bindings. We have nftnl-rs, which wraps most of it nicely, but the set parts in particular are incomplete. Then we have a bunch of vibe-coded stuff that does netlink itself (and are somewhat incomplete themselves, too).

❯ doas nft list table inet iocaine
table inet iocaine {
        set genurlv4 {
                type ipv4_addr
        }
}

Only cost me one unsafe!

unsafe {
    nftnl_set_set_u32(
      set_v4.as_prt().as_ptr(), 
      NFTNL_SET_FLAGS as u16,
      0,
    );
}

I have absolutely no idea why I have to do this, but... this gets the job done.

How I got here, you might ask... You see, I was getting protocol errors, which was weird, because the table is there, and is valid, I just created it. The protocol family is also fine, can't be a problem (but I did try both ProtoFamily::Inet and ProtoFamily::Ipv4). The set id might be a problem, but... other examples used 0, or 1337 and other numbers - none of that worked for me, so that's not it, either.

So I figured, what else could be a problem? What's the part of the Set that nftnl does not expose? Flags. So I went and set it to zero.

And then it stopped  throwing protocol errors.

Aaah, this felt good. This is something vibecoders will never feel! The satisfaction of figuring something out, the thrill of learning.

Time for .

table inet iocaine {
        set genurl_v4 {
                type ipv4_addr
        }

        set genurl_v6 {
                type ipv6_addr
        }

        chain genurl {
                type filter hook input priority filter - 1; policy accept;
                ip saddr @genurl_v4 drop
                ip6 saddr @genurl_v6 drop
        }
}

table inet iocaine {
        set genurl_v4 {
                type ipv4_addr
                timeout 1d
        }

        set genurl_v6 {
                type ipv6_addr
                timeout 1d
        }

        chain genurl {
                type filter hook input priority filter - 1; policy accept;
                ip saddr @genurl_v4 drop
                ip6 saddr @genurl_v6 drop
        }
}

This is nice. Now I only need to do Firewall.block_ip, and we're good to go.

Urgh. Will have to do globals, because I need the Set to add to it, it looks like... unless I can figure out how to look an existing one up.

@algernon
don't you want to put this before conntrack to save resources?

@wolf480pl Eventually, maybe. For now, I'm good with hooking input.

Oh fun. Lifetime schenanigans.

The problem right now is that in practical terms, I have one script that creates the table, sets, and chains and rules - this part is working.

But then I have another script which is supposed to add an address to a set. But this other script does not have access to the variables I used in the first script, because, well, they're separate scripts!

So, I'd have to shovel the Set into a container that I can ferry over to the main script... but then we're looking at concurrency problems and lifetimes, and all kinds of mind melting things.

I don't really want to pass Sets around (concurrency!), so I think I have two options:

• Figure out how to look a set up via nftnl.
• Start a background task that'd keep the Set alive, and pass messages to it.

I'd much prefer the first.

Oof.

Lookup seems to be possible with nftnl-rs, but oh dear, it is messy. It's mostly nftnl-sys really.

Yeah, I think I'm gonna go with background task for now. It's completely invisible from the Roto/Lua side, so I can freely change it later.

Eh, fuck. Doing it with channels means that the Set, on the rust side, will also accumulate all the IP addresses, which wastes a lot of memory.

Buuut... lets get a first version working first, then I'll address the shortcomings.

....and background task involves lifetime stuff too. Wonderful.

I guess I will have to do the lookup stuff myself then. Unfortunate.

Oooor... maybe I can cheat.

Nope.

Now I have a sad.

@algernon@come-from.mad-scientist.club do you fear the lifetime? I'm pretty decent with borrow checker semantics if you need some help

@buherator tbh, I have no problems with lifetimes, as long as I don't have to care about them across async boundaries.

async + lifetimes is what's giving me quite a bit of headache.

@addison Thanks!

Though, not sure if the problem is solvable in a reasonable way. The situation is that I have  two variables, table & set. The latter has an internal borrow of the former, and is as such bound to table's lifetime.

I'd... like to ferry set over to a tokio task, but I can't async move, because then set would outlive table.

@addison IOW, the problem is that tokio::spawn wants the borrow to be 'static.

@algernon@come-from.mad-scientist.club oh yeah oof that is nonnegotiable 😔 but if you're throwing borrows into a spawn, you're probably not awaiting the result anyway, so classic borrowing wouldn't be valid anyway. Arc::downgrade is probably the "right" solution here.

@addison Not sure that'd work either, because I'm using set in the async code only, which borrows table, and I'd have to extend the lifetime of table, not set...

Might be possible, but at that point, it's probably easier to not do this. Instead of trying to ferry these two around, I can "just" look up the set again, and then I have no lifetime issues.

@algernon@come-from.mad-scientist.club oh sorry I meant "throw table in an Arc"

It looks like nftnl-rs has... very little support for querying stuff. Hm.

On the other hand... I can construct most of the required messages using nftnl-rs's types. The missing parts are around Batch, because that doesn't support a request message type.

But! I can wrap the original types! In a type that implements NlMsg, but ignores the MsgType and sets NLM_F_REQUEST | NLM_F_ACK.

Unfortunately, I will have to duplicate some code for this. But I think I know how to do what I need to do.

I could also fork the crate, and add the missing pieces. That might be more efficient.

Unfortunately, upstream is on GitHub. I'm not sending them a PR. Buuuut... I can send an email. They're also on Fedi, and seem to be replying to people too. Hmm.

Choices, choices.