jarl/docs/development.md

Developing Jarl

Layout

cmd/jarld/            daemon: serve, init, import, adopt, check, render, resync, backup, restore, user
cmd/jarl/             CLI
internal/
  model/              configuration objects (State: views, ACLs, keys, zones, records, options) and changesets
  dnsname/            DNS name handling: FQDNs, relative names, check-names rules, reverse zones
  rr/                 record data parsing/canonicalization (miekg/dns), TXT quoting, the record type catalog
  namedconf/          named.conf lexer, parser and printer; address match lists
  iprange/            address ranges (10.0.10.101-10.0.10.199): parsing and the exact prefix cover
  catalog/            every BIND 9.20 statement: grammar (from named.conf(5)) + ARM descriptions, embedded
  validate/           whole-configuration validation and security lints
  render/             State → named.conf, keys.conf and zone files, with a source map for error messages
  bind/               named-checkconf, named-checkzone and rndc wrappers
  engine/             changesets: expand requests into ops, replay, validate, render, check, deploy, reload, verify, revert;
                      import and adopt
  importer/           reads an existing named.conf tree and its zone files
  store/              SQLite (modernc.org/sqlite, no cgo): state, changesets, users, tokens, sessions, audit
  auth/               argon2id passwords, tokens, sessions, grants and roles
  server/             HTTP API, authentication, the web UI
  stats/              query statistics: statistics-channel poller, response-log tailer, collector, separate SQLite store
  web/static/         the web UI: plain ES modules, no build step, embedded with go:embed
  api/, client/       JSON types and the Go client shared by server and CLI
  acme/               ACME DNS-01 challenge records, shared by jarl acme and the httpreq endpoint
  backup/             database backups: write (VACUUM INTO, plus the DNSSEC key files) and check, list, prune by the time in the name
  nscheck/            DNS queries to other name servers: zone serials, the parent's delegation and DS records
  dnssec/             reads named's DNSSEC key and state files; DS records; the signing phases (docs/dnssec.md)
  porkbun/            Porkbun API client (name servers, glue, DS records) and an in-memory fake for tests
  blocking/           blocklist parser, response policy zone writers, and matching a name the way named does
                      (the downloads are in engine/blocklists.go)
  config/             jarld's YAML configuration
  certs/              certificates: Jarl as an ACME client (lego), renewal, files for named
  encdns/             encrypted DNS (DoT/DoH) setup, status and health checks
  mailcheck/          reverse DNS check of the addresses mail is sent from
  totp/               two-factor login codes and recovery codes
  version/            what a build reports (release tag, commit, commits since a tag)
  testutil/           test helpers, chiefly finding BIND's tools (JARL_BIND_ROOT)
deploy/               systemd units (jarld, the daily backup service and timer), sysusers.d and tmpfiles.d files
deploy/debian/        the Debian package's control template and postinst/prerm/postrm
contrib/acme.sh/      acme.sh DNS API hook
contrib/lego/         program for lego's exec DNS provider
scripts/e2e.sh        end-to-end test against a real named
scripts/mkdeb.sh      builds the Debian package (make deb)
scripts/logo.py       traces artwork/logo.png into the web UI's logo (logo.svg, logo.js)
scripts/banner.py     builds the README's banner from the web UI's logo and wordmark
artwork/              the logo's master image and the README's banner

The main flow is in internal/engine/engine.go (applyOps): expand → replay → validate → render → check → install → reload → verify → record. concepts.md describes it from the outside.

Building and testing

make            # bin/jarld, bin/jarl (CGO_ENABLED=0, version from ./VERSION),
                # and bin/completion/{jarl,jarld} (bash completion)
make test       # go vet, gofmt check, go test ./...
make e2e        # scripts/e2e.sh
make deb        # dist/jarl_<version>_<arch>.deb (server) and jarl-cli_<version>_<arch>.deb (DEB_ARCH=arm64 etc. cross-builds)

The Debian package

make deb runs scripts/mkdeb.sh: it builds the binaries for DEB_ARCH, installs them with make install-files into a staging tree with Debian's paths (PREFIX=/usr, the unit in /usr/lib/systemd/system), adds deploy/debian/{postinst,prerm,postrm} and a control file filled in from deploy/debian/control.in, and packs it with dpkg-deb --root-owner-group, so no root is needed. The version comes from ./VERSION (a - becomes ~, so 0.2.0-rc1 sorts before 0.2.0; override with DEB_VERSION), and the maintainer from your git name and email (override with DEB_MAINTAINER). The package ships no files in /etc: postinst writes /etc/jarl/jarl.yaml once, so upgrades never touch it.

To try the package without installing it on your machine:

docker run --rm -it -v "$PWD/dist:/pkg:ro" debian:trixie \
  sh -c 'apt-get update && apt-get install -y /pkg/jarl_*_amd64.deb /pkg/jarl-cli_*_amd64.deb && bash'

Shell completion

The completion scripts are generated by cobra (jarl completion bash). cmd/jarl/complete.go adds value completion: it reads each command's usage line (add ZONE NAME TYPE DATA...) and completes the placeholders it knows (ZONE, VIEW, ACL, USER, TSIG KEY, TYPE, server and group names), so a new command whose usage line uses these names gets completion without extra code. Try it without a shell: jarl __complete record add ''.

Tests against real BIND

The render, engine and adopt tests run the real named-checkconf, named-checkzone, named, rndc and dig when they can find them, and skip otherwise. They are found through JARL_BIND_ROOT (an extracted package tree) or on the system. To get them without root or AppArmor in the way, extract the packages:

mkdir -p ~/bind-root/debs && cd ~/bind-root/debs
apt-get download bind9 bind9-utils bind9-libs bind9-dnsutils \
    liblmdb0 libmaxminddb0 liburcu8t64 libuv1t64 libjemalloc2 libjson-c5 libnghttp2-14
for f in *.deb; do dpkg -x "$f" ~/bind-root; done
cd -
JARL_BIND_ROOT=~/bind-root make test
JARL_BIND_ROOT=~/bind-root make e2e

The tests start named unprivileged on random high ports in temporary directories. On Debian/Ubuntu, AppArmor confines the installed /usr/sbin/named to BIND's own directories, so the tests that start named (engine integration tests, make e2e) need the extracted copy. The checker tests work with the installed bind9-utils too.

scripts/e2e.sh builds a legacy BIND setup, starts named, runs jarld init, import, adopt and serve, then drives every CLI command (records, batches, changesets, views, ACLs, TSIG, options, users, tokens, ACME hooks, DNSSEC signing, backup and restore with keys, audit) and checks the answers with dig. The Porkbun code is tested against porkbun.Fake, an in-memory stand-in for its API; nothing calls the real one. With Docker, certificates are tested end to end against Pebble, Let's Encrypt's test CA (ghcr.io/letsencrypt/pebble, host network, ports 14000 and 15000), which checks the challenges against the test named: obtaining, HTTPS from jarld, DNS over TLS from named, renewal. PEBBLE=0 skips that; the certs package's own tests use a fake DNS server and porkbun.Fake, and never a real CA. KEEP=1 leaves named and jarld running afterwards. The output says where; set JARL_CLI_CONFIG to that directory's cli.yaml to keep using the CLI against it.

A test machine

scripts/testenv.sh turns a fresh Debian/Ubuntu VM or container into a server to try Jarl on, the way a user would. Run it as root from the repository:

sudo scripts/testenv.sh                         # zone example.com
sudo JARL_TEST_ZONE=lab.test scripts/testenv.sh # another zone

It installs BIND 9, make, git, curl and Go (from go.dev, at the version in go.mod, unless a new enough one is installed). It then writes a typical hand-made configuration for Jarl to import: an ACL, options with logging, an internal view (this machine's networks and the private ranges, with recursion) and an external view (everyone else, authoritative only). The zone is in both views with different answers. There is also a reverse zone for the machine's /24. The zone files in /etc/bind/zones use comments, $TTL, $ORIGIN, $GENERATE, legacy type master and most record types. The script checks the result with named-checkconf -z, (re)starts named (in the background where there is no systemd), and asks named a few questions with dig. At the end it prints the steps to build and install the packages, then run init, import and adopt.

JARL_TEST_VIEWS=no leaves out the views, and JARL_TEST_IP sets the machine's address (default: the address of the default route). The old /etc/bind is saved in /root/bind-before-testenv-<time>.tgz. Once Jarl has adopted the configuration, running the script again needs JARL_TEST_FORCE=1. Debian 13 has BIND 9.20; Ubuntu has it from 25.04.

The web UI

The UI is plain JavaScript modules under internal/web/static/, with no framework and no build step. The logo's master is artwork/logo.png (blue #1d5287 and white on a transparent background). scripts/logo.py traces it with potrace (needs potrace and ImageMagick) into assets/logo.svg (the favicon) and assets/logo.js (the same paths); change the PNG and rerun the script rather than editing those files. The header and login page draw the logo inline from logo.js (logo() in brand.js), so it appears with the page instead of popping in after a separate download. Static files are served with Cache-Control: no-cache and a content ETag, so a reload revalidates them with cheap 304 answers. The "JARL" wordmark is drawn as SVG paths by wordmark() in assets/brand.js (over a rule on the login page, the letters alone in the header), so the UI needs no web font and the wordmark takes the theme's text and accent colors. The README's banner, artwork/banner.svg, is the login page's logo and wordmark, generated by scripts/banner.py from logo.js and brand.js (logo.py runs it; run it yourself after editing the wordmark). All DOM is built with the h() helper in ui.js, which never uses innerHTML, so record data and comments can't inject markup. The Content-Security-Policy only allows the app's own scripts and styles. To work on it, run jarld and reload the browser: the files are embedded at build time, so rebuild jarld after changes.

Conventions

• Every user-visible change goes into CHANGELOG.md (under "Unreleased"), and the docs in docs/ are updated in the same change.
• Errors that reach users say what is wrong and how to fix it, naming the object.
• New validation rules get a stable code, a test in internal/validate/validate_test.go, and a row in validation.md.
• New API operations go through the engine (internal/engine/expand.go, ops.go) so they get permissions, validation, history and revert for free, and are documented in api.md.
• gofmt and go vet clean (make vet).

Releasing

1. Move the "Unreleased" section of CHANGELOG.md under ## [<version>] - <date>. (A release candidate may skip this; see below.)
2. Put the version in VERSION.
3. make test && JARL_BIND_ROOT=... make e2e (CI runs only make test; it has no BIND).
4. Commit, tag and push: git tag v<version> && git push origin main v<version>.

The tag starts .gitea/workflows/release.yml, which checks that the tag matches VERSION and that CHANGELOG.md has the section, runs make test, builds everything (make dist, make cli-dist, make deb for amd64 and arm64) and runs scripts/gitea-release.sh. That creates the Gitea release (the CHANGELOG section as notes; the tarballs, the CLI for Linux, macOS and Windows, the .debs and SHA256SUMS attached) and uploads the .debs to the owner's Debian registry, distribution stable, component main (install.md).

Release candidates

A version with a hyphen (0.2.0-rc.1, tag v0.2.0-rc.1) is a pre-release:

• On Gitea it is marked "Pre-Release". Gitea doesn't count it as the latest release: the repository page, /releases/latest and the API's latest release keep pointing at the last stable one.
• The .debs go to the testing distribution, not stable. Their version is 0.2.0~rc.1, which apt sorts after 0.1.0 but before 0.2.0. So in stable an RC would reach every server on its next apt upgrade. Servers that follow only stable never see RCs. A test machine that also lists testing gets each RC, and then the final 0.2.0 from stable, which supersedes it.
• Release notes: its own ## [0.2.0-rc.1] section if there is one; otherwise the section of the version it leads up to (## [0.2.0]), or else ## [Unreleased]. So you can tag RCs while the changes are still under Unreleased, and move them under ## [0.2.0] for the real release. A stable release still needs its own section.

scripts/gitea-release.sh notes prints the notes a release would get (VERSION=0.2.0-rc.1 scripts/gitea-release.sh notes), and the workflow runs it as a check before building.

Which version a build reports

The footer of the web UI, jarld version, jarl --version, jarl status and jarl whoami say what a binary was built from (the footer links to it on Gitea). Only a release build reports a version number:

• A release: make run on the commit tagged v<VERSION> with no tracked file changed. The Makefile then sets RELEASE_TAG, and the footer reads "Powered by Jarl v0.2.0", linking to the release. The release workflow sets RELEASE_TAG itself.
• Any other build is a development build, named by its commit (which Go records from git) rather than VERSION: "Powered by Jarl 1a2b3c4d5e6f", linking to the commit. With uncommitted changes it is marked -modified. The other places add where it stands relative to the last tag, from git describe (the Makefile's DESCRIBE): dev 1a2b3c4d5e6f (3 commits after v0.2.0-rc.1), or dev 1a2b3c4d5e6f-modified (v0.2.0-rc.1 with uncommitted changes).
• No git (a build from a source archive): VERSION, marked as such: 0.2.0 (source build).

RELEASE_TAG=v0.2.0 make forces the first case.

make deb versions the packages the same way. A release gets VERSION with ~ for - (0.2.0~rc.1). A development build gets a version after the last tag: 3 commits past v0.2.0-rc.1 is 0.2.0~rc.1+git3.gca0f148, and uncommitted changes add the build time (.m20261009153000). apt sorts these after that RC and before the next one or 0.2.0. So installing a newer build over an older one, or over the RC, is an upgrade: plain apt install ./dist/… works, without --reinstall. make deb removes older packages of the same architecture from dist/ first.

Pushing the same tag again (git tag -f, git push -f) re-runs it, and the script replaces the attachments and packages, so a failed release can simply be retried.

Once, on Gitea:

• Actions on for the repository (Settings → Repository → Actions), and a runner with the ubuntu-latest label: act_runner's default, which runs jobs in docker.gitea.com/runner-images:ubuntu-latest. The job downloads Go itself (actions/setup-go, from go.mod), so the runner needs internet access, or a mirror of the actions and Go.
• A repository secret RELEASE_TOKEN (Settings → Actions → Secrets): an access token (User settings → Applications) with the write:repository and write:package scopes, from a user who can write to the repository. The job's automatic token can't be relied on to publish packages.

scripts/gitea-release.sh also runs by hand after the same builds, with GITEA_URL, GITEA_REPO (owner/name) and GITEA_TOKEN set; it needs curl, jq and dpkg-deb.