jarl/docs/development.md

Developing Jarl

Layout

cmd/jarld/            daemon: serve, init, import, adopt, check, render, resync, backup, restore, user
cmd/jarl/             CLI
internal/
  model/              configuration objects (State: views, ACLs, keys, zones, records, options) and changesets
  dnsname/            DNS name handling: FQDNs, relative names, check-names rules, reverse zones
  rr/                 record data parsing/canonicalization (miekg/dns), TXT quoting, the record type catalog
  namedconf/          named.conf lexer, parser and printer; address match lists
  catalog/            every BIND 9.20 statement: grammar (from named.conf(5)) + ARM descriptions, embedded
  validate/           whole-configuration validation and security lints
  render/             State → named.conf, keys.conf and zone files, with a source map for error messages
  bind/               named-checkconf, named-checkzone and rndc wrappers
  engine/             changesets: expand requests into ops, replay, validate, render, check, deploy, reload, verify, revert;
                      import and adopt
  importer/           reads an existing named.conf tree and its zone files
  store/              SQLite (modernc.org/sqlite, no cgo): state, changesets, users, tokens, sessions, audit
  auth/               argon2id passwords, tokens, sessions, grants and roles
  server/             HTTP API, authentication, the web UI
  stats/              query statistics: statistics-channel poller, response-log tailer, collector, separate SQLite store
  web/static/         the web UI: plain ES modules, no build step, embedded with go:embed
  api/, client/       JSON types and the Go client shared by server and CLI
  acme/               ACME DNS-01 challenge records, shared by jarl acme and the httpreq endpoint
  backup/             database backups: write (VACUUM INTO) and check, list, prune by the time in the name
  nscheck/            DNS queries to other name servers: zone serials and the parent's delegation
  config/             jarld's YAML configuration
deploy/               systemd units (jarld, the daily backup service and timer), sysusers.d and tmpfiles.d files
deploy/debian/        the Debian package's control template and postinst/prerm/postrm
contrib/acme.sh/      acme.sh DNS API hook
contrib/lego/         program for lego's exec DNS provider
scripts/e2e.sh        end-to-end test against a real named
scripts/mkdeb.sh      builds the Debian package (make deb)
scripts/logo.py       traces artwork/logo.png into the web UI's logo (logo.svg, logo.js)
scripts/banner.py     builds the README's banner from the web UI's logo and wordmark
artwork/              the logo's master image and the README's banner

The main flow is in internal/engine/engine.go (applyOps): expand → replay → validate → render → check → install → reload → verify → record. concepts.md describes it from the outside.

Building and testing

make            # bin/jarld, bin/jarl (CGO_ENABLED=0, version from ./VERSION),
                # and bin/completion/{jarl,jarld} (bash completion)
make test       # go vet, gofmt check, go test ./...
make e2e        # scripts/e2e.sh
make deb        # dist/jarl_<version>_<arch>.deb (server) and jarl-cli_<version>_<arch>.deb (DEB_ARCH=arm64 etc. cross-builds)

The Debian package

make deb runs scripts/mkdeb.sh: it builds the binaries for DEB_ARCH, installs them with make install-files into a staging tree with Debian's paths (PREFIX=/usr, the unit in /usr/lib/systemd/system), adds deploy/debian/{postinst,prerm,postrm} and a control file filled in from deploy/debian/control.in, and packs it with dpkg-deb --root-owner-group, so no root is needed. The version comes from ./VERSION (a - becomes ~, so 0.2.0-rc1 sorts before 0.2.0; override with DEB_VERSION), and the maintainer from your git name and email (override with DEB_MAINTAINER). The package ships no files in /etc: postinst writes /etc/jarl/jarl.yaml once, so upgrades never touch it.

To try the package without installing it on your machine:

docker run --rm -it -v "$PWD/dist:/pkg:ro" debian:trixie \
  sh -c 'apt-get update && apt-get install -y /pkg/jarl_*_amd64.deb /pkg/jarl-cli_*_amd64.deb && bash'

Shell completion

The completion scripts are generated by cobra (jarl completion bash). cmd/jarl/complete.go adds value completion: it reads each command's usage line (add ZONE NAME TYPE DATA...) and completes the placeholders it knows (ZONE, VIEW, ACL, USER, TSIG KEY, TYPE, server and group names), so a new command whose usage line uses these names gets completion without extra code. Try it without a shell: jarl __complete record add ''.

Tests against real BIND

The render, engine and adopt tests run the real named-checkconf, named-checkzone, named, rndc and dig when they can find them, and skip otherwise. They are found through JARL_BIND_ROOT (an extracted package tree) or on the system. To get them without root or AppArmor in the way, extract the packages:

mkdir -p ~/bind-root/debs && cd ~/bind-root/debs
apt-get download bind9 bind9-utils bind9-libs bind9-dnsutils \
    liblmdb0 libmaxminddb0 liburcu8t64 libuv1t64 libjemalloc2 libjson-c5 libnghttp2-14
for f in *.deb; do dpkg -x "$f" ~/bind-root; done
cd -
JARL_BIND_ROOT=~/bind-root make test
JARL_BIND_ROOT=~/bind-root make e2e

The tests start named unprivileged on random high ports in temporary directories. On Debian/Ubuntu, AppArmor confines the installed /usr/sbin/named to BIND's own directories, so the tests that start named (engine integration tests, make e2e) need the extracted copy. The checker tests work with the installed bind9-utils too.

scripts/e2e.sh builds a legacy BIND setup, starts named, runs jarld init, import, adopt and serve, then drives every CLI command (records, batches, changesets, views, ACLs, TSIG, options, users, tokens, ACME hooks, audit) and checks the answers with dig. KEEP=1 leaves named and jarld running afterwards. The output says where; set JARL_CLI_CONFIG to that directory's cli.yaml to keep using the CLI against it.

The web UI

The UI is plain JavaScript modules under internal/web/static/, with no framework and no build step. The logo's master is artwork/logo.png (blue #1d5287 and white on a transparent background). scripts/logo.py traces it with potrace (needs potrace and ImageMagick) into assets/logo.svg (the favicon) and assets/logo.js (the same paths); change the PNG and rerun the script rather than editing those files. The header and login page draw the logo inline from logo.js (logo() in brand.js), so it appears with the page instead of popping in after a separate download. Static files are served with Cache-Control: no-cache and a content ETag, so a reload revalidates them with cheap 304 answers. The "JARL" wordmark is drawn as SVG paths by wordmark() in assets/brand.js (over a rule on the login page, the letters alone in the header), so the UI needs no web font and the wordmark takes the theme's text and accent colors. The README's banner, artwork/banner.svg, is the login page's logo and wordmark, generated by scripts/banner.py from logo.js and brand.js (logo.py runs it; run it yourself after editing the wordmark). All DOM is built with the h() helper in ui.js, which never uses innerHTML, so record data and comments can't inject markup. The Content-Security-Policy only allows the app's own scripts and styles. To work on it, run jarld and reload the browser: the files are embedded at build time, so rebuild jarld after changes.

Conventions

• Every user-visible change goes into CHANGELOG.md (under "Unreleased"), and the docs in docs/ are updated in the same change.
• Errors that reach users say what is wrong and how to fix it, naming the object.
• New validation rules get a stable code, a test in internal/validate/validate_test.go, and a row in validation.md.
• New API operations go through the engine (internal/engine/expand.go, ops.go) so they get permissions, validation, history and revert for free, and are documented in api.md.
• gofmt and go vet clean (make vet).

Releasing

1. Move the "Unreleased" section of CHANGELOG.md under ## [<version>] - <date>.
2. Put the version in VERSION.
3. make test && JARL_BIND_ROOT=... make e2e (CI runs only make test; it has no BIND).
4. Commit, tag and push: git tag v<version> && git push origin main v<version>.

The tag starts .gitea/workflows/release.yml, which checks that the tag matches VERSION and that CHANGELOG.md has the section, runs make test, builds everything (make dist, make cli-dist, make deb for amd64 and arm64) and runs scripts/gitea-release.sh. That creates the Gitea release (the CHANGELOG section as notes; the tarballs, the CLI for Linux, macOS and Windows, the .debs and SHA256SUMS attached) and uploads the .debs to the owner's Debian registry, distribution stable, component main (install.md). A version with a hyphen (1.0.0-rc.1) becomes a pre-release; its packages get version 1.0.0~rc.1, which apt sorts before 1.0.0.

Pushing the same tag again (git tag -f, git push -f) re-runs it, and the script replaces the attachments and packages, so a failed release can simply be retried.

Once, on Gitea:

• Actions on for the repository (Settings → Repository → Actions), and a runner with the ubuntu-latest label: act_runner's default, which runs jobs in docker.gitea.com/runner-images:ubuntu-latest. The job downloads Go itself (actions/setup-go, from go.mod), so the runner needs internet access, or a mirror of the actions and Go.
• A repository secret RELEASE_TOKEN (Settings → Actions → Secrets): an access token (User settings → Applications) with the write:repository and write:package scopes, from a user who can write to the repository. The job's automatic token can't be relied on to publish packages.

scripts/gitea-release.sh also runs by hand after the same builds, with GITEA_URL, GITEA_REPO (owner/name) and GITEA_TOKEN set; it needs curl, jq and dpkg-deb.