# Developing Housecarl ## Layout ``` cmd/housecarld/ the daemon: serve, init, user, check, backup, restore, config cmd/hc/ the CLI internal/ model/ the configuration (State: settings, zones, sets, services, rules, NAT), users, history validate/ whole-configuration checks with stable codes (docs/validation.md) render/ State → the nftables ruleset text, with each line's object nft/ runs nft: check, apply (keeping dynamic set entries and counters), list, the boot file nflog/ reads logged packets from netlink log group 4171 importer/ nft -j list ruleset → State, with notes and raw rules (docs/import.md) hostif/ the machine's interfaces store/ SQLite (modernc.org/sqlite, no cgo): configuration, history, users, tokens, sessions, audit engine/ configuration changes: edit a copy, diff, validate, check and apply, save with history, undo server/ the HTTP API and the web UI, authentication, the local socket, probation, the log ring api/, client/ JSON types and the Go client shared by server and CLI auth/ passwords (argon2id), tokens, sessions, grants and roles certs/, jarl/ the web UI's certificate through Jarl (docs/certificates.md) config/ housecarld's YAML configuration version/ what a build reports web/static/ the web UI: plain ES modules, no build step, embedded with go:embed deploy/ systemd units, sysusers.d and tmpfiles.d files, Debian package scripts scripts/e2e.sh end-to-end test with real traffic through a router container scripts/mkdeb.sh the Debian packages (make deb) scripts/logo.py the web UI's logo from artwork/logo.svg scripts/banner.py the README's banner from the web UI's logo and wordmark ``` [design.md](design.md) describes how the pieces fit. ## Building and testing ```sh make # bin/housecarld, bin/hc, bash completion make test # go vet, gofmt check, go test ./... make e2e # scripts/e2e.sh: needs Docker make deb # dist/housecarl__.deb and housecarl-cli_... ``` **Never apply a test ruleset to your own machine's nftables.** Tests that need the real `nft` run it in a container. **Unit tests** need no nftables: the API tests (`internal/server`) use a fake firewall. The render tests compare the ruleset with what is expected, line by line, and the importer tests import the maintainer's router ruleset (`internal/importer/testdata`) and check the result. **Real nftables**: with `HOUSECARL_NFT` set to a command that runs nft somewhere harmless, the render, nft and importer tests also have nft check (and apply) their rulesets. A `debian:trixie` image with nftables 1.1.3 (Debian 13's) is the reference: ```sh make e2e # makes the housecarl-nft image, if missing export HOUSECARL_NFT="docker run --rm -i --cap-add NET_ADMIN housecarl-nft nft" export HOUSECARL_NFT_IMAGE=housecarl-nft make test ``` Newer nftables accepts things 1.1.3 doesn't (a meter with a timeout for `ct count`, hour ranges past midnight): keep to what 1.1.3 takes. **The e2e test** builds a small network in Docker from the same image: * a router container (housecarld, the `hc` CLI) on three networks named `wan`, `enp2s0` and `wg0` inside it, IPv4 and IPv6; * "the internet" on the WAN side, a LAN client, and a "jellyfin" host. It imports a copy of the maintainer's ruleset (lab addresses), takes over the old table, and checks with real traffic: port forwards with hairpin NAT and the IPv6 pinhole, masquerading, antispoofing, the drop log, a lockout change that rolls back by itself, a restart, the boot ruleset after a flush, and that other tables are left alone. It makes the `housecarl-nft` image itself if it is missing. `KEEP=1` leaves the lab running (the web UI on `10.88.10.1:8070`); `DOCKER_HOST` points at another Docker daemon. Root isn't needed on the host. The systemd sandbox (capabilities, the boot unit) is only tested by installing the package in a container that runs systemd (`--privileged --cgroupns=host /sbin/init`). ## Certificates As in Steward: `internal/certs` has unit tests with a fake Jarl and an in-process DNS server; a real order is tested against Jarl's e2e environment and Pebble (Steward's docs/development.md has the steps). ## The web UI Plain JavaScript modules under `internal/web/static/`, built with the same helpers as Jarl's and Steward's: * All DOM goes through `h()` in `ui.js`, never `innerHTML`. * The Content-Security-Policy allows only the app's own scripts and styles, so no inline styles. Use a class in `style.css`, or set `el.style.prop`. * The files are embedded, so rebuild housecarld after changing them. There are no JavaScript tests: bundle `assets/app.js` with esbuild to catch syntax and import errors, and look at the pages in a browser against the e2e lab. ## Conventions * Every user-visible change goes into `CHANGELOG.md` under "Unreleased", with the docs updated in the same change. * Errors say what is wrong and how to fix it, naming the object. * New validation rules get a stable code, a test, and a row in [validation.md](validation.md). * Configuration changes go through the engine, never straight to the store, so they are validated, checked by nftables, recorded and undoable. * Schema changes append a migration to `internal/store/store.go`; never edit a shipped one. * `gofmt` and `go vet` clean (`make test`). ## Releasing As for Jarl and Steward: 1. Move "Unreleased" in `CHANGELOG.md` under `## [] - `. A release candidate may leave it there. 2. Put the version in `VERSION`, and update the README's status line. 3. Run `make test && make e2e`. 4. Commit as "Release X", tag it `git tag -a vX -m "Housecarl X"`, and push. The tag starts `.gitea/workflows/release.yml`, which checks the tag against `VERSION` and the CHANGELOG, runs the tests, builds the tarballs, the CLI for each OS and the `.deb`s, and publishes them (`scripts/gitea-release.sh`). It needs the repository secret `RELEASE_TOKEN` (`write:repository`, `write:package`).