# Working on Jarl

Notes for coding agents (and people) changing this repository. Read this
first. For more depth, see [docs/development.md](docs/development.md) for
layout, building, tests and releasing, and [docs/concepts.md](docs/concepts.md)
for how Jarl works from the outside.

Jarl manages a BIND 9.20 server. It has three parts:

* `jarld`, a daemon with an HTTP API, an embedded web UI and SQLite storage;
* `jarl`, a CLI for that API;
* a Debian package.

It's all Go: module path `jarl`, version in `go.mod`, `CGO_ENABLED=0`. The
SQLite driver is `modernc.org/sqlite`, so there is no cgo. The target is
Debian/Ubuntu with BIND 9.20, using Debian's paths (`/etc/bind`, the `bind`
group, AppArmor).

## Ground rules

* **The database is the source of truth.** Jarl renders `named.conf`,
  `keys.conf` and the zone files from it. Never hand-edit rendered output or
  add code that reads BIND's files back as state; the importer is the only
  exception.
* **Every change goes through the engine** (`internal/engine`). An API
  request becomes ops in `expand.go`/`ops.go`. The engine then runs
  replay → validate → render → `named-checkconf`/`named-checkzone` →
  atomic install → `rndc reload` → verify serials, and rolls back on
  failure. Going through the engine gives a change permissions, dry runs,
  changesets, history and undo for free, so don't write to the store
  directly from handlers.
* **Every user-visible change updates `CHANGELOG.md`** under
  `## [Unreleased]`, and the matching page in `docs/` in the same commit.
  Docs that may need it:

  * `cli.md`, `api.md` and `configuration.md`;
  * `validation.md`, for new validation codes;
  * `metrics.md`;
  * the feature's own page.

  Write docs in plain, direct English, like the existing pages.
* **Error messages** say what is wrong and how to fix it, naming the object.
  For example: `view external is reachable from the internet …; set
  minimal-any yes`.
* **New validation rules** get a stable `code`, a test, and a row in
  `docs/validation.md`.
* **Schema changes** append a migration to the `migrations` slice in
  `internal/store/store.go`, and never edit an old one. The statistics
  database has its own list, `schema` in `internal/stats/store.go`. Add a note
  under "Upgrading" in the CHANGELOG when a downgrade would break.
* `make test` runs `go vet` and a `gofmt -l` check, and both must pass.
  `goimports` isn't installed, so fix imports by hand.

## Building and testing

```sh
make                 # bin/jarld, bin/jarl, bash completion
make test            # vet + gofmt check + go test ./...
JARL_BIND_ROOT=DIR make test   # also runs the integration tests against real BIND
JARL_BIND_ROOT=DIR make e2e    # scripts/e2e.sh: real named + jarld, drives every CLI command
```

* **Real BIND without root.** Get the packages with `apt-get download` and
  unpack them with `dpkg -x` into a directory; that directory is
  `JARL_BIND_ROOT`. The package list is in docs/development.md, under "Tests
  against real BIND". Without `JARL_BIND_ROOT` the integration tests skip,
  so a green `make test` alone proves less than it looks.
* **Run e2e after any engine, render, CLI or server change.** It prints a
  pass count, such as `606/606`; a failure prints the command and its
  output.
  * It uses fixed ports: DNS 25353, rndc 25953, HTTP 28053, lists 28054 and
    metrics 28055. Stop anything left running on them first.
  * `KEEP=1` leaves named and jarld running; the last line of the log has
    their PIDs.
  * `WORK=DIR` must name an empty directory.
* **Certificate tests** run against Pebble in Docker, unless you set
  `PEBBLE=0`. If `docker info` fails, the Pebble block is silently skipped;
  on this machine it needs `DOCKER_HOST=unix:///var/run/docker.sock`.
* **Unix socket paths** are limited to 107 bytes. Put test sockets under
  `/run/user/$UID/`, not under a deep temporary directory.
* **The systemd sandbox isn't tested.** e2e runs outside systemd, so bugs
  that only appear in the sandbox (`RestrictAddressFamilies`,
  capabilities, read-only paths) need a package test. Run `make deb`, then
  install it in a `debian:trixie` container. That container has BIND 9.20;
  run it with `--privileged /sbin/init` when you need the real unit.

## Layout and where things live

docs/development.md has the full layout. Some places that are easy to miss:

* **Engine** (`internal/engine`):
  * `expand.go` turns requests into ops.
  * `linked.go`: linked zones, kept in step across views by `linkSync`.
  * `reverse.go`: PTR fill, auto-PTR and reverse zone suggestions.
  * `compare.go`: compare views.
  * `exposure.go`: fixes for internet exposure.
* **Validation**: `internal/validate` holds all checks and lints.
  `internal/catalog` is the BIND 9.20 grammar and ARM text for every
  statement. Use it rather than hard-coding option knowledge.
* **Rendering**: `internal/render` writes the configuration.
  `generated.go` holds the statements Jarl adds itself, shown to users as
  "Added by Jarl".
* **Server**: `internal/server` has one `h_*.go` file per API area; routes
  are in `routes.go`. The JSON types are in `internal/api`, shared with the
  Go client in `internal/client`.
* **CLI**: `cmd/jarl` is built on cobra. `complete.go` completes the
  placeholders in usage lines (`ZONE`, `VIEW`, …), so name them
  consistently.
* **Statistics**: `internal/stats` is query statistics, kept in a
  **separate** SQLite file, `stats.db`, with its own migrations.
* **Feature packages**: each has its own doc page.
  * `encdns`: encrypted DNS;
  * `certs` and `acme`: certificates and ACME;
  * `blocking`: response policy zones;
  * `dnssec`;
  * `nscheck`: queries to other name servers;
  * `mailcheck`: mail reverse DNS;
  * `totp`: two-factor login;
  * `backup`;
  * `porkbun`: registrar API, with an in-memory `Fake`. Tests never call
    the real API.

## Web UI rules

`internal/web/static/` holds the UI: plain ES modules, no framework, no
build step. The files are embedded with `go:embed`.

* **Rebuild jarld after editing them**, or you'll be testing the old files.
* **Build DOM only with `h()`** from `assets/ui.js`. Never use `innerHTML`
  or `insertAdjacentHTML`. That's how record data can't inject markup.
* **The CSP is `self` only.** That rules out:
  * inline scripts;
  * `style="…"` attributes and inline style strings (`el.setAttribute("style", …)`);
  * external fonts and CDNs.

  Use a CSS class in `style.css`, or set individual properties with
  `el.style.prop = …` (CSSOM is allowed).
* **Colors are CSS custom properties** with light and dark values. Reuse the
  existing tokens.
* **Pages live in `assets/pages/`**, one file per route. The router is in
  `app.js` and the API calls are in `api.js`.
* **What the user wants:** functionality and ease of use matter more than
  visual polish. In pick lists, leave every checkbox unticked by default.

There's no JS test suite. To catch syntax and import errors without a
browser, bundle `assets/app.js` with esbuild, or load the page in headless
Chrome.

## BIND facts that bit us

* named 9.20 crashes on an empty `keys {}` in a `dnssec-policy`. Validation
  refuses it.
* `named-checkconf` doesn't catch a TTL above `max-zone-ttl`, but named then
  refuses to load the zone.
* `responselog` only takes effect when named starts; use `rndc responselog
  on|off` instead.
* `named -g` sends all logging to stderr, so tests use `-f`.
* RPZ client-IP rules see IPv4 clients as v4-mapped (`::ffff:0:0/96`), so
  IPv6 prefix covers must leave that range out.
* named serves DoH in RFC 8484 wire format only, not the JSON API. Test it
  with BIND's `dig +https` / `dig +tls`, not curl or kdig.
* named can't forward over DoH, only over DoT (`forwarders port 853 tls NAME`).
* `rndc reconfig` makes named reload renewed TLS certificate files.

## Git and releases

These are the maintainer's rules; follow them unless told otherwise:

* **Commit only when asked. Never push**; the maintainer pushes, to Gitea
  at gitea.adbraun.lol.
* **Work on `dev-X.Y.Z` branches** (for example `dev-0.6.1`) off `main`.
  Don't name a branch like a tag (`v0.5.2`): `git push` then fails with
  "src refspec matches more than one".
* **A release** is a commit titled `Release X`.
  * It sets `VERSION`.
  * It updates the status line in the README.
  * For a final release, it moves `[Unreleased]` to
    `## [X] - YYYY-MM-DD`. A release candidate (`X-rc.N`) may leave the
    notes under `[Unreleased]`.
  * Tag it annotated: `git tag -a vX -m "Jarl X"`.
  * Before tagging, `make test` and the e2e run must pass.
  * Check the notes with `VERSION=X scripts/gitea-release.sh notes`.
* **Pushing a `v*` tag** runs `.gitea/workflows/release.yml`. It builds the
  tarballs, the CLI for each OS and the `.deb`s, and publishes them.
* **Before a minor release**, test an upgrade from the previous release's
  `.deb`s in a container. Check data, history, answers from named, and that
  an old backup restores.

## Shell hygiene

* **Never kill processes by pattern** with `pkill -f` or
  `pgrep -f … | xargs kill` in the same shell command. The pattern matches
  the shell running it, and kills that shell. Find the PID with `ps` and
  kill it by number.
* **Start long-running test servers in the background** (your tool's
  background mode, not `&` in a shell that exits), so they outlive the
  command.