housecarl/docs/design.md

Housecarl's design

Housecarl is a firewall manager with the same shape as Jarl and Steward: one daemon, housecarld, that serves a REST API and a web UI; one CLI, hc, that talks to that API; SQLite for everything. As Jarl does for BIND, it keeps a Linux machine's nftables ruleset: the database is the source of truth, and the ruleset is built from it.

The model

Named objects, and ordered rules that refer to them (rules.md, nat.md):

• Zones: interfaces grouped by what they connect to (wan, lan, wg). External zones face the internet.
• Address sets: IPv4 and IPv6 addresses, prefixes and ranges, mixed. A dynamic set takes entries at run time, each with a timeout that nftables enforces by itself.
• Services: ports of a protocol and ICMP types, by name. Many are built in (ssh, dns, https, wireguard).
• Rules, in order, in four chains: input (to the firewall), forward (through it), output (from it) and early (before connection tracking).
• NAT: port forwards (with hairpin NAT and an optional IPv6 pinhole), source NAT (masquerade or an address), redirects to the firewall.
• Settings: the chains' policies, and the built-ins, the rules every firewall needs (loopback, established connections, ICMPv6, antispoofing, MSS clamping), each of which can be turned off.

IPv4 and IPv6 are handled together: a set holds both, and a rule that names one becomes one or two nftables rules as needed.

How a change is made

Every change goes through one engine (internal/engine):

1. the request becomes a change to a copy of the configuration;
2. the whole result is validated (validation.md), with errors that say what is wrong and how to fix it;
3. the ruleset is rendered (internal/render) and checked with nft -c -f (a dry run by nftables itself);
4. it is applied with nft -f, which replaces Housecarl's table in one transaction: the old rules stay in force until the new ones are all in;
5. the objects, the history entry and the audit record are saved together. If saving fails, the previous ruleset is applied again.

Dry runs, history with undo, users, sessions and scoped API tokens come from the same machinery as in Jarl and Steward.

Reviewing changes: changesets

As in Jarl, a change can be made at once or staged in a changeset, to be reviewed and applied together with others:

• Each change is an operation (create, update, delete or move an object, or change the settings), kept as the API received it. A move staged right after a move of the same object takes its place: moving a rule up three steps is one operation, and moving it back where it was takes it out.
• Staged in a changeset, an operation is checked as it is added: the changeset with it must still be valid, and nftables must accept the ruleset it makes. Nothing is applied.
• The review shows the operations, the problems, and the diff of the ruleset: the file that is loaded at boot, now and after.
• Applying replays the operations on the configuration as it is then, as one change: one check, one transaction, one history entry, undone as a whole. An operation that no longer applies (its object was changed or deleted since) is named, and can be taken out.
• Then the applied changeset is on probation like any change made over the network: unless it is confirmed, it is rolled back.

Interface	Default	Why
Web UI	Review edits: they are collected in a draft changeset, pages show pending rows (green new, yellow changed, struck out deleted), and a bar leads to the Changes page, with the diff and Apply. The switch in the header turns on Live edits, applied at once.	Several related edits, seen together before they are in force.
CLI	Immediate: each command is a change of its own. Stage with -C ID or HOUSECARL_CHANGESET.	Scripts want a clear result per step.
API	Immediate unless a changeset is named (?changeset=ID, X-Housecarl-Changeset).	As the CLI.

An undo is staged like any edit (in review mode, the web UI's Undo collects it for review). Imports and takeovers can't be staged; with a changeset named, they are refused rather than applied.

Its own table only

Housecarl writes one table, inet housecarl, and leaves every other table alone (Docker's, Incus's, libvirt's). It never flushes the whole ruleset. The table is replaced in one script:

table inet housecarl {}
delete table inet housecarl
table inet housecarl { ... }

Two things survive the replacement: the entries of dynamic sets (read before, written back with their remaining timeouts, in the same script), and the counters (housecarld adds the old totals to the new, so hits don't reset with every change). Sets from Jarl are written with the addresses last read from Jarl.

Every rule carries a comment hc:<mark>, such as hc:r12 for rule 12, hc:f3 for port forward 3, hc:b-established for a built-in. Counters and log lines are traced back to their object through it.

Shadow mode

With shadow.enabled in the configuration file (shadow.md), the same configuration is rendered so that it changes nothing (render's Options.Shadow): drops and rejects accept instead, policies accept, NAT only counts, and what alters packets (MSS clamping, the flowtable) and raw rules that do more than accept, drop or reject are left out. This rests on how nftables runs base chains: an accept only ends its own chain, and the packet goes on to the next one on the hook, other tables' included; only a drop is final. Another table, the real firewall, decides.

What would be dropped, rejected or translated is counted under the same marks, and its comment says so ("hc:r12 [would drop] …"), which is how the tracer tells it apart. The filter chains run at filter + 10 (after the other firewall's: what Housecarl would drop is what switching over would break) or - 10; the NAT chains always at - 10, since the kernel asks NAT chains only until one translates.

Every change is checked by nftables in both forms, so leaving shadow mode can't fail on the ruleset. There is no probation, since nothing can lock anyone out; takeovers and closing connections, which would change traffic, are refused. A ruleset reload elsewhere (flush ruleset) deletes Housecarl's table too: housecarld listens to nftables' netlink notifications (internal/tablewatch) and puts it back.

Not locking yourself out

A change made over the network is applied on probation (a changeset when it is applied, a live edit at once): unless it is confirmed within two minutes (Settings; hc confirm, the banner in the web UI, POST /confirm), it is rolled back by itself, and the rollback is recorded in the history. A change that cuts you off undoes itself.

• The client that made a change confirms it by itself once it reaches the API again: the change's answer carries a one-time token and closes its connection (HTTP/2: a GOAWAY), and the web UI and hc send the token straight back. That proves something only on a new connection, since the built-in that accepts established connections would let the old one through whatever the change did. So the token is accepted only from the same user (and API token), the same address, a different connection, and for the newest change. A change that cuts its client off can't be confirmed by it, and rolls back. This proves the API is still reachable, not that SSH or the LAN still work; with the API, you can fix those. Settings → "a person confirms each change" (hc settings set --auto-confirm off) goes back to confirming by hand; hc --no-confirm leaves one change waiting.
• Changes made through the local socket (sudo hc … on the firewall) need no confirmation: whoever makes them is already on the machine. One made while others are pending joins them.
• A new change restarts the clock for everything pending.
• If housecarld stops with changes pending, it rolls them back when it starts.
• Housecarl doesn't force a rule that keeps SSH or the web UI open. You write the rules that let you in; probation is the safety net.

At boot

The ruleset is saved to /var/lib/housecarl/ruleset.nft whenever it is confirmed (or changed through the socket). housecarl-ruleset.service loads it before the network comes up, without housecarld, so the firewall is up even if housecarld can't start. An unconfirmed ruleset is never saved there: a reboot during probation brings back the last confirmed one.

It fails closed. A fallback saved beside it (ruleset-fallback.nft, render.Fallback) is loaded if it doesn't: no sets, no flowtable, interfaces by name only, so there is little in it to fail. It keeps anything new from the external zones out and lets the inside work, which is as far as a ruleset can go without knowing the rules; a fallback that let everything in would leave a router open on the internet, and one that let nothing in would lock out the person who has to fix it. The boot ruleset has no flowtable either: nftables refuses a flowtable naming a device that doesn't exist yet (on Debian 13's kernel), and before the network is up, ppp0 doesn't. housecarld watches the kernel's link messages (internal/linkwatch) and applies again when a flowtable device appears, or comes back as a new interface.

The drop log

Rules with logging on, and the policies (Settings), send packets to netlink log group 4171. housecarld listens there and keeps the last 2000 in memory (hc log, the Log page). Nothing is written to the kernel log.

Importing a ruleset

hc import turns an existing nftables ruleset into zones, sets, services, rules and NAT, and hc takeover deletes the old table once Housecarl's replaces it (import.md).

Working with Steward and Jarl

• Steward's captive portal (parked until now): a guest network's zone lets clients that haven't signed in reach only DNS and the portal, and sends their web traffic to it. When a guest signs in, Steward adds its address to a dynamic set for the session's length, with a token that may only change that set (sets:write:guests). Addresses, not MAC addresses: matching by MAC only works on the guests' own network segment.
• Names instead of addresses: a set can take its addresses from Jarl (jarl.md). housecarld reads the names' A and AAAA records through Jarl's API (CNAMEs followed, one view per name) and keeps the addresses that lie within the zones and networks the set allows; Jarl never calls Housecarl. What was last read is kept (in the database too) while Jarl can't be reached. A change of the addresses applies the ruleset again without a history entry, as dynamic sets' entries aren't in it either. Names Steward makes from leases must not be used: a client chooses its own hostname.
• The web UI's certificate comes through Jarl, as Steward's does (certificates.md).

Not in 0.1

• Interfaces, addresses and routes: Housecarl only runs the firewall.
• iptables rulesets (only nftables can be imported).
• Rules by MAC address, by user, or by DNS name.
• Flow offloading to hardware (the software flowtable is there).