abrown38andClaude Opus 5.5
de075edd5d
shadow.enabled in housecarl.yaml loads Housecarl's ruleset so that it blocks, rejects and translates nothing: drops and rejects count and accept (an accept only ends Housecarl's chain), policies accept and log what they would drop, NAT only counts, MSS clamping and the flowtable are left out, and raw rules that do more than accept, drop or reject are left out with a warning. Comments mark what would happen ([would drop]). Filter chains run after the other firewall's (+10) or before (-10); NAT chains always first. Every change is checked by nftables in both forms. No probation; takeovers and closing connections are refused. The boot fallback filters nothing. internal/tablewatch puts the table back after a flush ruleset. Mode in status, healthz, whoami, the web UI, the log and the tracer; hc ruleset --enforcing. docs/shadow.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
143 lines
6.4 KiB
Markdown
143 lines
6.4 KiB
Markdown
# Developing Housecarl
|
|
|
|
## Layout
|
|
|
|
```
|
|
cmd/housecarld/ the daemon: serve, init, user, check, backup, restore, config
|
|
cmd/hc/ the CLI
|
|
internal/
|
|
model/ the configuration (State: settings, zones, sets, services, rules, NAT), users, history
|
|
validate/ whole-configuration checks with stable codes (docs/validation.md)
|
|
render/ State → the nftables ruleset text, with each line's object
|
|
nft/ runs nft: check, apply (keeping dynamic set entries and counters), list, the boot file
|
|
nflog/ reads logged packets from netlink log group 4171
|
|
linkwatch/ netlink: interfaces that appear or go (the flowtable's devices)
|
|
tablewatch/ netlink: a table deleted (shadow mode puts Housecarl's back; docs/shadow.md)
|
|
importer/ nft -j list ruleset → State, with notes and raw rules (docs/import.md)
|
|
hostif/ the machine's interfaces
|
|
store/ SQLite (modernc.org/sqlite, no cgo): configuration, history, users, tokens, sessions, audit
|
|
engine/ configuration changes: edit a copy, diff, validate, check and apply, save with history, undo
|
|
server/ the HTTP API and the web UI, authentication, the local socket, probation, the log ring
|
|
api/, client/ JSON types and the Go client shared by server and CLI
|
|
auth/ passwords (argon2id), tokens, sessions, grants and roles
|
|
certs/, jarl/ the web UI's certificate through Jarl (docs/certificates.md)
|
|
config/ housecarld's YAML configuration
|
|
version/ what a build reports
|
|
web/static/ the web UI: plain ES modules, no build step, embedded with go:embed
|
|
deploy/ systemd units, sysusers.d and tmpfiles.d files, Debian package scripts
|
|
scripts/e2e.sh end-to-end test with real traffic through a router container
|
|
scripts/mkdeb.sh the Debian packages (make deb)
|
|
scripts/logo.py the web UI's logo from artwork/logo.svg
|
|
scripts/banner.py the README's banner from the web UI's logo and wordmark
|
|
```
|
|
|
|
[design.md](design.md) describes how the pieces fit.
|
|
|
|
## Building and testing
|
|
|
|
```sh
|
|
make # bin/housecarld, bin/hc, bash completion
|
|
make test # go vet, gofmt check, go test ./...
|
|
make e2e # scripts/e2e.sh: needs Docker
|
|
make deb # dist/housecarl_<version>_<arch>.deb and housecarl-cli_...
|
|
```
|
|
|
|
**Never apply a test ruleset to your own machine's nftables.** Tests that
|
|
need the real `nft` run it in a container.
|
|
|
|
**Unit tests** need no nftables: the API tests (`internal/server`) use a
|
|
fake firewall. The render tests compare the ruleset
|
|
with what is expected, line by line, and the importer tests import the
|
|
maintainer's router ruleset (`internal/importer/testdata`) and check the
|
|
result.
|
|
|
|
**Real nftables**: with `HOUSECARL_NFT` set to a command that runs nft
|
|
somewhere harmless, the render, nft and importer tests also have nft
|
|
check (and apply) their rulesets. A `debian:trixie` image with nftables
|
|
1.1.3 (Debian 13's) is the reference:
|
|
|
|
```sh
|
|
make e2e # makes the housecarl-nft image, if missing
|
|
export HOUSECARL_NFT="docker run --rm -i --cap-add NET_ADMIN housecarl-nft nft"
|
|
export HOUSECARL_NFT_IMAGE=housecarl-nft
|
|
make test
|
|
```
|
|
|
|
Newer nftables accepts things 1.1.3 doesn't (a meter with a timeout for
|
|
`ct count`, hour ranges past midnight): keep to what 1.1.3 takes.
|
|
|
|
**The e2e test** builds a small network in Docker from the same image:
|
|
|
|
* a router container (housecarld, the `hc` CLI) on three networks named
|
|
`wan`, `enp2s0` and `wg0` inside it, IPv4 and IPv6;
|
|
* "the internet" on the WAN side, a LAN client, and a "jellyfin" host.
|
|
|
|
It imports a copy of the maintainer's ruleset (lab addresses) with
|
|
housecarld in shadow mode beside it (traffic unchanged, a would-be drop
|
|
logged and let through, the table back after the old ruleset is
|
|
reloaded), restarts it enforcing, takes over the old table, and checks
|
|
with real traffic: port forwards with
|
|
hairpin NAT and the IPv6 pinhole, masquerading, antispoofing, the drop
|
|
log, a lockout change that rolls back by itself, a restart, the boot
|
|
ruleset after a flush, and that other tables are left alone. It makes
|
|
the `housecarl-nft` image itself if it is missing.
|
|
|
|
`KEEP=1` leaves the lab running (the web UI on `10.88.10.1:8070`);
|
|
`DOCKER_HOST` points at another Docker daemon. Root isn't needed on the
|
|
host.
|
|
|
|
The systemd sandbox (capabilities, the boot unit) is only tested by
|
|
installing the package in a container that runs systemd
|
|
(`--privileged --cgroupns=host /sbin/init`).
|
|
|
|
## Certificates
|
|
|
|
As in Steward: `internal/certs` has unit tests with a fake Jarl and an
|
|
in-process DNS server; a real order is tested against Jarl's e2e
|
|
environment and Pebble (Steward's docs/development.md has the steps).
|
|
|
|
## The web UI
|
|
|
|
Plain JavaScript modules under `internal/web/static/`, built with the
|
|
same helpers as Jarl's and Steward's:
|
|
|
|
* All DOM goes through `h()` in `ui.js`, never `innerHTML`.
|
|
* The Content-Security-Policy allows only the app's own scripts and
|
|
styles, so no inline styles. Use a class in `style.css`, or set
|
|
`el.style.prop`.
|
|
* The files are embedded, so rebuild housecarld after changing them.
|
|
|
|
There are no JavaScript tests: bundle `assets/app.js` with esbuild to
|
|
catch syntax and import errors, and look at the pages in a browser
|
|
against the e2e lab.
|
|
|
|
## Conventions
|
|
|
|
* Every user-visible change goes into `CHANGELOG.md` under "Unreleased",
|
|
with the docs updated in the same change.
|
|
* Errors say what is wrong and how to fix it, naming the object.
|
|
* New validation rules get a stable code, a test, and a row in
|
|
[validation.md](validation.md).
|
|
* Configuration changes go through the engine, never straight to the
|
|
store, so they are validated, checked by nftables, recorded and
|
|
undoable.
|
|
* Schema changes append a migration to `internal/store/store.go`; never
|
|
edit a shipped one.
|
|
* `gofmt` and `go vet` clean (`make test`).
|
|
|
|
## Releasing
|
|
|
|
As for Jarl and Steward:
|
|
|
|
1. Move "Unreleased" in `CHANGELOG.md` under `## [<version>] - <date>`.
|
|
A release candidate may leave it there.
|
|
2. Put the version in `VERSION`, and update the README's status line.
|
|
3. Run `make test && make e2e`.
|
|
4. Commit as "Release X", tag it `git tag -a vX -m "Housecarl X"`, and push.
|
|
|
|
The tag starts `.gitea/workflows/release.yml`, which checks the tag
|
|
against `VERSION` and the CHANGELOG, runs the tests, builds the tarballs,
|
|
the CLI for each OS and the `.deb`s, and publishes them
|
|
(`scripts/gitea-release.sh`). It needs the repository secret
|
|
`RELEASE_TOKEN` (`write:repository`, `write:package`).
|