Boot fallback, flowtable devices that come and go, healthz, renames

The boot unit fails closed: if ruleset.nft doesn't load, it loads
ruleset-fallback.nft (render.Fallback), saved beside it from the same
configuration: nothing new in from external zones, established, the
ICMPv6 IPv6 needs and DHCPv6 replies let in, the inside left alone and
masqueraded, and the unit marked failed with a critical message.
housecarld replaces it when it applies; status, the dashboard and
/healthz say while it is in force. no_external_zone now also says the
fallback closes everything without an external zone.

The boot ruleset has no flowtable: nftables on Debian 13's kernel
refuses one naming a device that doesn't exist yet (ppp0 before PPPoE),
and with it the whole file. internal/linkwatch listens to rtnetlink link
messages; housecarld applies again when a flowtable device appears or
comes back as a new interface (by ifindex), under the engine's lock.
docs/flowtable.md: what it does, when to use it and when not, how.

GET /healthz reports firewall (applied, unconfigured, fallback,
missing), changes on probation and the machine's warnings, with 503
unless in force, read at most every five seconds.

Renaming a zone, set or service changes what names it in the same
change (one history entry, one undo); dynamic sets keep their names.
The importer keeps a chain's head in order: once a would-be built-in
stays a rule, the ones after it do too, so the maintainer's ICMPv6
accepts stay behind the invalid drop as written. hc settings set
--flowtable none empties the list.

make test (nft 1.1.3) and make e2e (242 checks, with VLAN
sub-interfaces, the fallback and the flowtable) pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
abrown38andClaude Opus 5.5 committed 2026-10-19 23:16:46 +01:00
1 parent 835a9ba2cc
commit 9d766b9823
38 files changed
+1335 -84

No files matched your search

+1 -1
View File
@@ -74,7 +74,7 @@ Creating answers 201, other changes 200.
| | |
|---|---|
| `GET /healthz` | `{"ok": true, "version"}`, without authentication |
| `GET /healthz` | `ok`, `version`, `firewall` (`applied`, `unconfigured`, `fallback`, `missing`), `probation` (changes waiting to be confirmed), `warnings` (codes); HTTP 503 unless `ok`. Without authentication; read at most every five seconds ([operations.md](operations.md#monitoring)) |
| `GET /status` | `build`, `started`, `applied` (the ruleset is in force), `apply_error`, `problems`, `probation`, `interfaces`, `counts` (objects by kind), `other_tables` (other nftables tables, `"inet main"`) |
| `GET /interfaces` | the machine's interfaces: `name`, `up`, `addresses`, `zone` |
| `GET /matrix[?changeset=ID]` | the zone matrix: `zones`, `cells` (`from`, `to` (a zone, or `firewall`), `chain`, `verdict` (allowed, blocked), `decided_by`, `rule_id`, `exceptions`: `rule_id`, `chain`, `label`, `verdict` (allowed, blocked, logged, unknown), `what`) ([rules.md](rules.md#the-zone-matrix)) |
+2 -2
View File
@@ -148,7 +148,7 @@ by name.
|---|---|
| `zone list`, `zone show ZONE` | |
| `zone add NAME -i IFACE... [--external] [--comment C]` | |
| `zone update ZONE [-i IFACE]... [--add IFACE] [--remove IFACE] [--external[=false]] [--name N] [--comment C]` | `-i` replaces the interfaces |
| `zone update ZONE [-i IFACE]... [--add IFACE] [--remove IFACE] [--external[=false]] [--name N] [--comment C]` | `-i` replaces the interfaces; `--name` renames it, and the rules naming it follow |
| `zone delete ZONE` | refused while rules use it |
| `set list`, `set show SET` | |
| `set add NAME [ENTRY...] [--dynamic] [--timeout 24h] [--comment C]` | |
@@ -208,7 +208,7 @@ See [nat.md](nat.md).
| Command | |
|---|---|
| `settings show` | |
| `settings set [--input-policy\|--forward-policy\|--output-policy accept\|drop] [--log-drops on\|off] [--builtin NAME=on\|off]... [--own-prefix P]... [--antispoof-v4 P]... [--antispoof-v6 P]... [--flowtable DEV]... [--probation 2m] [--auto-confirm on\|off] [--stats-per-device on\|off]` | lists given replace the old ones |
| `settings set [--input-policy\|--forward-policy\|--output-policy accept\|drop] [--log-drops on\|off] [--builtin NAME=on\|off]... [--own-prefix P]... [--antispoof-v4 P]... [--antispoof-v6 P]... [--flowtable DEV]... [--probation 2m] [--auto-confirm on\|off] [--stats-per-device on\|off]` | lists given replace the old ones; `none` empties one (`--flowtable none`: off) |
### probation, history, import
+1 -1
View File
@@ -14,7 +14,7 @@ reads the file when it starts.
| `tls.cert_file`, `tls.key_file` | none | serve HTTPS with this certificate; housecarld rereads it when the file changes, or on `systemctl reload housecarl` |
| `tls.acme.*` | off | get the certificate from Let's Encrypt, with Jarl publishing the challenge ([certificates.md](certificates.md#settings)) |
| `jarl.server`, `jarl.token_file` (or `jarl.token`), `jarl.ca_cert`, `jarl.insecure` | none | the Jarl server for the certificate's challenge, and its token ([jarl.md](jarl.md)) |
| `ruleset` | `/var/lib/housecarl/ruleset.nft` | the confirmed ruleset, which `housecarl-ruleset.service` loads at boot |
| `ruleset` | `/var/lib/housecarl/ruleset.nft` | the confirmed ruleset, which `housecarl-ruleset.service` loads at boot; its fallback is saved beside it as `ruleset-fallback.nft` ([operations.md](operations.md#if-the-ruleset-fails-to-load-at-boot)). Change both paths in the unit too |
| `trusted_proxies` | `[]` | addresses or prefixes of reverse proxies whose `X-Forwarded-For` and `X-Forwarded-Proto` are believed |
| `socket` | `/run/housecarl/housecarl.sock` | the local API socket; `""` turns it off |
| `database` | `/var/lib/housecarl/housecarl.db` | the SQLite database: configuration, history, users, audit log |
+13
View File
@@ -139,6 +139,19 @@ firewall is up even if housecarld can't start. An unconfirmed ruleset is
never saved there: a reboot during probation brings back the last
confirmed one.
It fails closed. A fallback saved beside it (`ruleset-fallback.nft`,
`render.Fallback`) is loaded if it doesn't: no sets, no flowtable,
interfaces by name only, so there is little in it to fail. It keeps
anything new from the external zones out and lets the inside work,
which is as far as a ruleset can go without knowing the rules; a
fallback that let everything in would leave a router open on the
internet, and one that let nothing in would lock out the person who has
to fix it. The boot ruleset has no flowtable either: nftables refuses a
flowtable naming a device that doesn't exist yet (on Debian 13's
kernel), and before the network is up, `ppp0` doesn't. housecarld
watches the kernel's link messages (internal/linkwatch) and applies
again when a flowtable device appears, or comes back as a new interface.
## The drop log
Rules with logging on, and the policies (Settings), send packets to
+181
View File
@@ -0,0 +1,181 @@
# The flowtable: a fast path for forwarded traffic
The flowtable is a shortcut inside the kernel for traffic the firewall
**forwards** (between the internet and your LAN, between zones). It
speeds up routing on a machine whose CPU can't keep up with its line.
It doesn't make the firewall any safer or less safe, but it changes what
some features can see, so read [when not to use it](#when-not-to-use-it)
before you turn it on.
It is off until you name its devices.
## What it does
Without a flowtable, every forwarded packet takes the whole way through
the kernel: the prerouting hooks, the routing decision, Housecarl's
forward chain (the built-ins, then your rules) and postrouting with NAT.
For an established connection almost all of that work only ends in the
`established` built-in accepting it again, packet after packet.
With a flowtable, Housecarl adds one built-in to the forward chain:
```
meta l4proto { tcp, udp } flow add @ft
```
A TCP or UDP connection takes the whole way for its first packets, as
always: your rules decide whether it is allowed at all. Once the kernel
has seen it established (and your rules let it through), it is added
to the flowtable. From then on its packets are caught as soon as they
arrive on one of the flowtable's devices (the *ingress* hook) and sent
straight out on the other side: NAT is still applied, the TTL still
counts down, but every chain in between is skipped.
```
first packets every packet after
in ─► prerouting ─► routing ─► forward ─► postrouting ─► out
│ ▲
└────────────── flowtable (ingress) ─────────────────────┘
```
It is the software flowtable, run by the CPU. Offloading to the network
card's hardware isn't supported ([design.md](design.md#not-in-01)).
## When to use it
Turn it on when **the router's CPU is what limits your speed**:
* a speed test through the router gets less than your line gives (or
less than the same test from the router itself), and
* during it, one CPU core is busy with network work: in `top`, press `1`
for every core, and look at `si` (softirq) near 100 % on a core.
Typical cases: a small or old machine (Atom, Celeron, a Raspberry Pi, an
old Core 2) on a fast line (500 Mbit/s and up), lots of small packets
(gaming, VoIP, many downloads at once), or a machine that is busy with
other work besides routing. The gain is CPU time per packet: often well
over half of the forwarding cost, which shows up as speed only when the
CPU was the limit, and otherwise as a cooler, quieter, more responsive
machine.
## When not to use it
**When the CPU keeps up anyway.** If the speed test already reaches your
line and `si` stays low, the flowtable buys you nothing you'd notice,
and costs you the figures below. Most modern x86 machines route a
gigabit line without it.
**When you count or limit traffic in the forward chain.** The fast path
skips the forward chain, so whatever must see *every* packet of a
connection only sees the first few. Housecarl warns about each:
| Feature | With the flowtable | Warning |
|---|---|---|
| quotas and byte-rate limits on forward rules | only count first packets: a device can go far past its allowance | `flowtable_traffic` |
| forward rules with "every packet" (`--every-packet`) | only see first packets | `flowtable_traffic` |
| statistics' traffic per device (`stats_per_device`) | only count first packets | `stats_flowtable` |
| hit counters of the `established` built-in (forward) | stop rising once a connection is offloaded | |
Rate limits by packets (`--limit 10/second`) and connection limits act
on new connections, so they work the same.
**When other tables must see forwarded traffic.** Offloaded packets skip
the forward hooks of *every* table, not only Housecarl's: Docker's,
Incus', libvirt's or a table of your own. Those normally only judge new
connections, which still pass through them, but a table that counts or
shapes established traffic in its forward chain won't see it.
What keeps working as before:
* **Your rules** decide every new connection: nothing gets offloaded
that your rules didn't let through. A rule you add later doesn't stop
a connection already open, but that is true without the flowtable too
(the `established` built-in accepts it).
* **NAT**: port forwards, hairpin NAT, masquerading and SNAT.
* **The input and output chains**: traffic to and from the firewall
itself never uses the flowtable.
* **Closing connections** (`hc connections close`, Traffic →
Connections) ends their fast path too: the kernel drops a flow with
its tracking entry.
* **Interface statistics** (traffic per zone and interface) come from the
interfaces' own counters, which count every packet.
* **Traffic shaping with `tc`** on the outgoing interface still applies.
* **The log** and **the packet tracer** see what goes through the rules:
new connections, as always. The tracer doesn't see the packets of an
offloaded connection; trace a new one.
* ICMP, and protocols other than TCP and UDP (GRE, ESP, SCTP), always
take the normal way.
## How to turn it on
Name the devices whose traffic should take the fast path, **both
sides**: the interface connections come in on and the one their replies
come in on. For a router that means the external interface and the
inside ones:
```sh
hc settings set --flowtable wan --flowtable enp2s0
hc settings # Flowtable: wan, enp2s0
```
In the web UI: Settings → Flowtable, add each device. The fast path
only catches packets arriving on a listed device, so a connection to or
from an unlisted one gains little or nothing: leave a device out only on
purpose (say, a WireGuard interface whose traffic you'd rather keep on
the normal way).
* Use **exact device names**, not wildcards (`incusbr*` is refused):
the kernel attaches the flowtable to each device by name.
* With **PPPoE**, name the PPP interface (`ppp0`, or the name your
`ppp@` unit gives it), the one in your external zone.
* `--flowtable` replaces the whole list; `hc settings set --flowtable
none` turns the flowtable off.
The change is applied (and on probation, if made over the network) like
any other.
### Devices that come and go
PPP and WireGuard interfaces don't exist until their service brings
them up, and PPPoE makes a new `ppp0` each time it reconnects. Housecarl
handles that:
* A device that doesn't exist yet is left out of the flowtable, with the
warning `flowtable_device_missing`, instead of failing the whole
ruleset.
* housecarld watches the kernel's interface changes: when a flowtable
device appears, or comes back as a new interface after a reconnect,
it applies the ruleset again with it, a couple of seconds later. The
journal says `interfaces changed: applied the ruleset again`.
Applying again keeps the counters, dynamic sets and quotas, as any
change does, and adds no history entry.
* The ruleset saved for boot has no flowtable at all: it loads before
the network is up, when `ppp0` and `wg0` can't exist yet, and
nftables on Debian 13's kernel refuses a flowtable naming a device
that doesn't exist (and with it the whole file). housecarld adds the
flowtable once it starts. Until then, traffic takes the normal way,
through the same rules.
## Checking it works
```sh
sudo nft list flowtables # the devices it is attached to
sudo conntrack -L | grep OFFLOAD # connections on the fast path
```
`conntrack` is in Debian's `conntrack` package. A long download through
the router should show up as `[OFFLOAD]`. Compare a speed test, and `si`
in `top`, before and after.
If nothing gets offloaded, check that both of the connection's devices
are listed, that `flowtable_device_missing` isn't in `hc status`, and
that the connection is TCP or UDP and established.
## Turning it off
```sh
hc settings set --flowtable none
```
Connections on the fast path go back to the normal way with their next
packet.
+5 -2
View File
@@ -50,8 +50,11 @@ Import a ruleset.
established, invalid, the ICMPv6 rules, MSS clamping, the DHCPv6 client
rule, antispoofing) becomes that built-in only when it sits where the
built-in goes, at the head of every chain it covers. Otherwise it stays
a rule where it was, so the ruleset does what it did. Built-ins your
ruleset doesn't have are turned off.
a rule where it was, so the ruleset does what it did. Since built-ins
come before every rule, once one at a chain's head stays a rule, the
ones after it there stay rules too, in order: an invalid drop that
stays a rule keeps the ICMPv6 accepts after it behind it, as they were.
Built-ins your ruleset doesn't have are turned off.
* **DNAT** rules become port forwards, TCP and UDP of the same port
merged into one. The forward-chain accepts that let them through
become part of the port forward, and a masquerade for inside sources
+75 -2
View File
@@ -71,14 +71,63 @@ sudo hc history undo 42
A confirmed ruleset (or one changed through the socket) is saved to
`/var/lib/housecarl/ruleset.nft`, which `housecarl-ruleset.service`
loads before the network comes up. To see what would load:
loads before the network comes up, so before PPPoE, WireGuard or DHCP
bring up their interfaces. To see what would load:
```sh
sudo cat /var/lib/housecarl/ruleset.nft
sudo nft -c -f /var/lib/housecarl/ruleset.nft # check it
```
Dynamic sets start empty after a reboot.
Dynamic sets start empty after a reboot. The saved ruleset has no
flowtable, since its devices may not exist yet: housecarld adds it when
it starts, and whenever a flowtable device appears
([flowtable.md](flowtable.md#devices-that-come-and-go)).
### If the ruleset fails to load at boot
Should `ruleset.nft` fail to load (a damaged file, an nftables or kernel
upgrade that refuses something in it), the machine would otherwise boot
with no firewall at all until housecarld starts. Instead,
`housecarl-ruleset.service` loads a **fallback** that housecarld saves
beside it, `/var/lib/housecarl/ruleset-fallback.nft`, from the same
confirmed configuration:
* **Nothing new gets in from the external zones' interfaces**, to the
firewall or through it: no port forwards, no SSH or web UI from the
internet.
* What the firewall and the inside need to keep working is let in:
established connections, the ICMPv6 IPv6 needs (neighbour discovery,
router advertisements, packet too big), and DHCPv6 replies from
link-local addresses.
* **Everything else is let through**: the LAN, WireGuard and the
firewall itself are reachable from the inside, so you can log in and
fix it.
* IPv4 leaving an external zone is masqueraded (when any source NAT is
configured), so the inside still reaches the internet. MSS clamping
stays, if it is on.
PPPoE itself isn't affected: its discovery and session are Ethernet
frames, and the PPP link's negotiation happens before any IP, neither
of which an `inet` table sees.
The fallback only knows which interfaces face the internet from the
zones marked **external**. With none, it lets nothing but loopback
through, and the warning `no_external_zone` says so.
When the fallback loads, the unit is marked failed, and the journal says
why (as a critical message):
```sh
systemctl status housecarl-ruleset
journalctl -b -u housecarl-ruleset
```
housecarld puts the real ruleset back when it starts, and saves a fresh
`ruleset.nft`. If it can't apply it either, the fallback stays in force:
`hc status` and the dashboard say "NOT in force: the boot fallback is in
force", and `/healthz` answers 503 ([Monitoring](#monitoring)). Fix
what `hc status` or the journal reports, then restart housecarld.
Keep Debian's `nftables.service` disabled (`systemctl disable
nftables`, without `--now`): at boot it loads `/etc/nftables.conf`, and
@@ -224,6 +273,30 @@ database is kept next to it as `housecarl.db.before-restore-<time>`.
database when it starts, and an older one refuses a database a newer one
has upgraded: restore a backup to go back.
## Monitoring
`GET /healthz` needs no login, and answers within a few seconds of any
change (it reads the ruleset at most every five seconds, however often
it is asked):
```sh
$ curl -s https://firewall.example.net:8070/healthz
{"ok":true,"version":"0.1.0","firewall":"applied","probation":0,"warnings":[]}
```
| Field | |
|---|---|
| `ok` | `true` with HTTP 200 when the firewall is as it should be; `false` with HTTP **503** otherwise |
| `firewall` | `applied` (Housecarl's ruleset is in force), `unconfigured` (nothing configured yet, so nothing to apply), `fallback` (the [boot fallback](#if-the-ruleset-fails-to-load-at-boot) is in force), `missing` (no Housecarl table: flushed, or never applied) |
| `probation` | changes waiting to be confirmed, rolled back unless they are |
| `warnings` | codes of problems with the machine, as `hc status` shows them (`nftables_service`) |
For Checkmk, an HTTP check on the URL that expects status 200 (and,
if you like, the string `"firewall":"applied"`) covers housecarld being
down, the fallback and a missing ruleset. Reach it from where the input
rules let the monitor in (a rule for the Checkmk server's address and
port 8070).
## Checks
`housecarld check` validates the configuration in the database against
+32 -3
View File
@@ -21,6 +21,32 @@ A zone may name an interface that doesn't exist yet (a WireGuard
interface that comes up later): its rules match once it appears, and
`hc status` warns until then.
**VLANs.** A VLAN sub-interface is an interface like any other, by its
name: `enp2s0.20` (or whatever your network configuration calls it,
`vlan20`, `br0.20`). Tagged traffic arrives on the sub-interface, and
untagged traffic on the parent, so `enp2s0` alone doesn't cover
`enp2s0.20`. Give each VLAN its own zone to keep them apart:
```sh
hc zone add lan --interface enp2s0 # untagged
hc zone add iot --interface enp2s0.20
hc zone add guest --interface enp2s0.30
hc rule add forward --in iot --out wan
```
or one zone for the parent and all of its VLANs, when they should be
treated alike: `--interface enp2s0 --interface 'enp2s0.*'`. Interface
names are at most 15 characters (the kernel's limit). The flowtable
takes VLAN sub-interfaces too, by exact name
([flowtable.md](flowtable.md)).
**Renaming** a zone (`hc zone update dhcp --name lan`, or Edit on the
Zones page) changes every rule and NAT rule that names it, in the same
change: the history shows them together, and undoing it puts them all
back. Sets and services rename the same way, except dynamic sets: their
entries, and the API tokens and programs that fill them, go by their
name.
## Address sets
A set is a named list of IPv4 and IPv6 addresses, prefixes and ranges,
@@ -183,7 +209,8 @@ first, marked "every packet"). So they can drop, reject or log, but not
accept: an accept there would let packets past every later rule. Let the
traffic in with a rule of its own. The flowtable's fast path skips the
forward chain for established connections, so with it on, forward
quotas and byte rates only count first packets (a warning says so).
quotas and byte rates only count first packets (a warning says so;
[flowtable.md](flowtable.md)).
**Per-address limits group addresses**: each IPv4 address on its own, and
IPv6 addresses by /64, since one device uses several (privacy addresses
@@ -232,8 +259,10 @@ them. The antispoof lists can be changed with `--antispoof-v4` and
The **flowtable** (`--flowtable wan --flowtable eth1`) lets established
TCP and UDP connections forwarded between those devices skip the rules,
which helps on a slow CPU. It needs exact device names. Counters of
forward rules then only count each connection's first packets.
which helps when the CPU can't keep up with the line. It needs exact
device names, and costs quotas, byte rates and traffic per device in the
forward chain their accuracy: see [flowtable.md](flowtable.md) for when
to use it, and when not.
## The zone matrix
+1 -1
View File
@@ -45,7 +45,7 @@ are unique per kind, ignoring case.
| `zone_interface` | error | not an interface name (up to 15 characters; a trailing `*` matches many) |
| `zone_interface_taken` | error | the interface is in another zone already |
| `zone_interface_missing` | warning, *host* | the interface doesn't exist here (yet); its rules match once it does |
| `no_external_zone` | warning | no zone is external: antispoofing, MSS clamping, the DHCPv6 client rule and port forwards without public addresses need one |
| `no_external_zone` | warning | no zone is external: antispoofing, MSS clamping, the DHCPv6 client rule and port forwards without public addresses need one, and the boot fallback lets nothing but loopback through without one ([operations.md](operations.md#if-the-ruleset-fails-to-load-at-boot)) |
## Sets and services