0 Fork 0

Files

T

abrown38andClaude Opus 5.5 de075edd5d Shadow mode: Housecarl beside the firewall in force, changing nothing

shadow.enabled in housecarl.yaml loads Housecarl's ruleset so that it
blocks, rejects and translates nothing: drops and rejects count and
accept (an accept only ends Housecarl's chain), policies accept and log
what they would drop, NAT only counts, MSS clamping and the flowtable
are left out, and raw rules that do more than accept, drop or reject are
left out with a warning. Comments mark what would happen ([would drop]).
Filter chains run after the other firewall's (+10) or before (-10); NAT
chains always first. Every change is checked by nftables in both forms.
No probation; takeovers and closing connections are refused. The boot
fallback filters nothing. internal/tablewatch puts the table back after
a flush ruleset. Mode in status, healthz, whoami, the web UI, the log
and the tracer; hc ruleset --enforcing. docs/shadow.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

2026-10-20 18:11:01 +01:00

6.4 KiB

Raw Blame History

Developing Housecarl

Layout

cmd/housecarld/       the daemon: serve, init, user, check, backup, restore, config
cmd/hc/               the CLI
internal/
  model/              the configuration (State: settings, zones, sets, services, rules, NAT), users, history
  validate/           whole-configuration checks with stable codes (docs/validation.md)
  render/             State → the nftables ruleset text, with each line's object
  nft/                runs nft: check, apply (keeping dynamic set entries and counters), list, the boot file
  nflog/              reads logged packets from netlink log group 4171
  linkwatch/          netlink: interfaces that appear or go (the flowtable's devices)
  tablewatch/         netlink: a table deleted (shadow mode puts Housecarl's back; docs/shadow.md)
  importer/           nft -j list ruleset → State, with notes and raw rules (docs/import.md)
  hostif/             the machine's interfaces
  store/              SQLite (modernc.org/sqlite, no cgo): configuration, history, users, tokens, sessions, audit
  engine/             configuration changes: edit a copy, diff, validate, check and apply, save with history, undo
  server/             the HTTP API and the web UI, authentication, the local socket, probation, the log ring
  api/, client/       JSON types and the Go client shared by server and CLI
  auth/               passwords (argon2id), tokens, sessions, grants and roles
  certs/, jarl/       the web UI's certificate through Jarl (docs/certificates.md)
  config/             housecarld's YAML configuration
  version/            what a build reports
  web/static/         the web UI: plain ES modules, no build step, embedded with go:embed
deploy/               systemd units, sysusers.d and tmpfiles.d files, Debian package scripts
scripts/e2e.sh        end-to-end test with real traffic through a router container
scripts/mkdeb.sh      the Debian packages (make deb)
scripts/logo.py       the web UI's logo from artwork/logo.svg
scripts/banner.py     the README's banner from the web UI's logo and wordmark

design.md describes how the pieces fit.

Building and testing

make            # bin/housecarld, bin/hc, bash completion
make test       # go vet, gofmt check, go test ./...
make e2e        # scripts/e2e.sh: needs Docker
make deb        # dist/housecarl_<version>_<arch>.deb and housecarl-cli_...

Never apply a test ruleset to your own machine's nftables. Tests that need the real nft run it in a container.

Unit tests need no nftables: the API tests (internal/server) use a fake firewall. The render tests compare the ruleset with what is expected, line by line, and the importer tests import the maintainer's router ruleset (internal/importer/testdata) and check the result.

Real nftables: with HOUSECARL_NFT set to a command that runs nft somewhere harmless, the render, nft and importer tests also have nft check (and apply) their rulesets. A debian:trixie image with nftables 1.1.3 (Debian 13's) is the reference:

make e2e         # makes the housecarl-nft image, if missing
export HOUSECARL_NFT="docker run --rm -i --cap-add NET_ADMIN housecarl-nft nft"
export HOUSECARL_NFT_IMAGE=housecarl-nft
make test

Newer nftables accepts things 1.1.3 doesn't (a meter with a timeout for ct count, hour ranges past midnight): keep to what 1.1.3 takes.

The e2e test builds a small network in Docker from the same image:

a router container (housecarld, the hc CLI) on three networks named wan, enp2s0 and wg0 inside it, IPv4 and IPv6;
"the internet" on the WAN side, a LAN client, and a "jellyfin" host.

It imports a copy of the maintainer's ruleset (lab addresses) with housecarld in shadow mode beside it (traffic unchanged, a would-be drop logged and let through, the table back after the old ruleset is reloaded), restarts it enforcing, takes over the old table, and checks with real traffic: port forwards with hairpin NAT and the IPv6 pinhole, masquerading, antispoofing, the drop log, a lockout change that rolls back by itself, a restart, the boot ruleset after a flush, and that other tables are left alone. It makes the housecarl-nft image itself if it is missing.

KEEP=1 leaves the lab running (the web UI on 10.88.10.1:8070); DOCKER_HOST points at another Docker daemon. Root isn't needed on the host.

The systemd sandbox (capabilities, the boot unit) is only tested by installing the package in a container that runs systemd (--privileged --cgroupns=host /sbin/init).

Certificates

As in Steward: internal/certs has unit tests with a fake Jarl and an in-process DNS server; a real order is tested against Jarl's e2e environment and Pebble (Steward's docs/development.md has the steps).

The web UI

Plain JavaScript modules under internal/web/static/, built with the same helpers as Jarl's and Steward's:

All DOM goes through h() in ui.js, never innerHTML.
The Content-Security-Policy allows only the app's own scripts and styles, so no inline styles. Use a class in style.css, or set el.style.prop.
The files are embedded, so rebuild housecarld after changing them.

There are no JavaScript tests: bundle assets/app.js with esbuild to catch syntax and import errors, and look at the pages in a browser against the e2e lab.

Conventions

Every user-visible change goes into CHANGELOG.md under "Unreleased", with the docs updated in the same change.
Errors say what is wrong and how to fix it, naming the object.
New validation rules get a stable code, a test, and a row in validation.md.
Configuration changes go through the engine, never straight to the store, so they are validated, checked by nftables, recorded and undoable.
Schema changes append a migration to internal/store/store.go; never edit a shipped one.
gofmt and go vet clean (make test).

Releasing

As for Jarl and Steward:

Move "Unreleased" in CHANGELOG.md under ## [<version>] - <date>. A release candidate may leave it there.
Put the version in VERSION, and update the README's status line.
Run make test && make e2e.
Commit as "Release X", tag it git tag -a vX -m "Housecarl X", and push.

The tag starts .gitea/workflows/release.yml, which checks the tag against VERSION and the CHANGELOG, runs the tests, builds the tarballs, the CLI for each OS and the .debs, and publishes them (scripts/gitea-release.sh). It needs the repository secret RELEASE_TOKEN (write:repository, write:package).

6.4 KiB Raw Blame History