jarl/docs/concepts.md

# How Jarl works

This page explains Jarl's design: where the data lives, how a change reaches
named, and why it works that way. Read it once before you adopt your server.

## The pieces

```
 browser ──(session + CSRF)──┐
                             ├──> jarld ──> SQLite DB (source of truth)
 jarl CLI ──(API token)──────┤      │
 scripts / certbot ──(token)─┘      ├──> /etc/bind/jarl/named.conf, keys.conf
                                    ├──> /var/lib/bind/jarl/*.zone
                                    └──> named-checkconf / named-checkzone / rndc ──> named
```

* **jarld** is one Go binary. It serves the REST API and the web UI (embedded
  in the binary; the UI uses only the public API), and it is the only thing
  that writes BIND's files. It runs as the unprivileged user `jarl` under
  systemd.
* **jarl** is the command line client. It talks to jarld's API with an API
  token, so it works from any machine that can reach jarld.
* **The database** (SQLite, `/var/lib/jarl/jarl.db`) holds the whole BIND
  configuration as structured data: views, ACLs, TSIG keys, zones, records,
  options, and the defaults for new zones. It also holds users, tokens,
  sessions, changesets and the audit log.

## The database is the source of truth

Jarl never edits BIND's files in place. Every time a change is applied it
renders the complete configuration from the database:

| File | Contents |
|------|----------|
| `/etc/bind/jarl/named.conf` | ACLs, options, views, zone statements, top-level statements such as `logging` and `controls` |
| `/etc/bind/jarl/keys.conf` | TSIG keys. They are kept apart so secrets never show up in diffs, history or the audit log |
| `/var/lib/bind/jarl/<view>/<zone>.zone` | One file per primary zone (`_default` when views aren't used) |
| `/var/cache/bind/jarl/...` | Written by named itself for secondary, stub and mirror zones |

`jarld adopt` turns `/etc/bind/named.conf` into a single
`include "/etc/bind/jarl/named.conf";`, so named loads only what Jarl
generates. Zone files go under `/var/lib/bind` because Debian's AppArmor
profile lets named write there. That keeps journals and future DNSSEC
signing working.

Because files are always regenerated, anything you edit by hand is
overwritten. Jarl records a checksum of every file it writes and reports
**drift** when a file changes behind its back. It then refuses to overwrite
the file until you say so (`--force`, or `jarl resync --force`).

Comments are part of the data model: zones, records, ACLs, views, keys and
options can all carry a comment, and Jarl writes it into the generated file.
Comments in imported zone files are kept.

## Changesets: one change or a bundle, it's the same pipeline

Your brief asked whether each change should be checked and applied on its
own, or whether changes should be bundled and applied together. Jarl does
both, using a single mechanism:

* A **changeset** is an ordered list of operations (create a zone, add a
  record, rename an ACL, ...). Applying one is all-or-nothing.
* Every API call that changes something is applied at once **as a one-operation
  changeset** by default. Nothing extra is needed for small edits.
* To bundle changes, create a changeset and *stage* operations into it
  (API: `?changeset=ID` or the `X-Jarl-Changeset` header; CLI: `--changeset ID`
  or `JARL_CHANGESET`). Review the diff, then apply or discard it.
* `POST /api/v1/apply` (CLI: `jarl apply -f changes.yaml`) applies a list of
  operations as one changeset in a single request.
* `?dry_run=1` (CLI: `--dry-run`) runs the whole pipeline except installing
  files and reloading named, and shows the problems and the diff.

**Why bundling matters.** Some changes only make sense together. Turning
`www` from an A record into a CNAME means deleting the A record and adding
the CNAME, and named must never see the zone with neither, or with a
half-finished renumbering. A changeset reaches named in one step or not at
all.

Operations sent **in one request** (`jarl apply -f`, `POST /apply`, or one
staging request with several operations) are validated together, at their
end state, so their order doesn't matter. When operations are staged **one
at a time** (the web UI's review mode, `--changeset` on single commands),
each one is checked as it is added. You get immediate feedback, so delete
the A record before adding the CNAME. At apply time the whole changeset is
validated again against the configuration as it is then.

**Interfaces use different defaults**, because they are used differently:

| Interface | Default | Why |
|-----------|---------|-----|
| Web UI | *Review mode*: edits are collected in a draft changeset, the zone pages show pending rows (green = new, yellow = changed, struck out = deleted), and a banner leads to the review page, which has a diff and an Apply button. Users can switch to "Apply edits immediately". | People click through several related edits and want to see the result before it goes live. |
| CLI | *Immediate*: every command is applied (and validated) on its own. Bundle with `jarl apply -f` or `--changeset`. | Scripts want a clear success or failure per step, and ACME hooks need the record live when the command returns. |
| API | Immediate unless a changeset is named. | Same as the CLI. |

### What happens when a changeset is applied

1. **Replay.** The operations are replayed on a copy of the current
   configuration, checking permissions and (for staged operations) that the
   objects haven't changed since they were staged (optimistic locking via
   object versions). Concurrent edits produce a `409 conflict` instead of
   silently overwriting each other.
2. **Validate.** The resulting configuration is checked as a whole (see
   [validation.md](validation.md)): DNS rules such as CNAME exclusivity,
   glue, delegations and RRset TTLs; references to ACLs and keys; option
   syntax and scope against the BIND 9.20 grammar; and security lints such
   as open resolvers. Errors block the change, while warnings are shown and
   allowed. Problems that already existed are reported separately and don't
   block unrelated changes.
3. **Render and check.** The files are rendered into a staging directory and
   checked with the real `named-checkconf -z` and `named-checkzone`. Their
   messages are mapped back to the object that caused them.
4. **Install.** Changed files are written atomically (temp file + rename),
   keeping the previous contents in memory.
5. **Reload.** `rndc reconfig` if `named.conf` changed, then `rndc reload ZONE`
   for each changed zone.
6. **Verify.** named loads zones asynchronously and an `rndc reload` that
   fails only shows up in named's log. So Jarl polls `rndc zonestatus` until
   every changed zone reports its new serial. If one doesn't within
   `reload_timeout` (15s), Jarl restores the previous files, reloads again
   and reports the failure.
7. **Record.** The changeset is stored with its result, its diff and the
   *undo operations*, so it can be reverted later (`jarl changeset revert ID`,
   or the Revert button in History). A revert is itself a changeset you
   review before applying.

Serials are bumped automatically, and only when a zone's DNS data actually
changes (a comment-only edit doesn't bump it). Each zone follows its serial
policy: `date` (YYYYMMDDnn, the default), `increment` or `unixtime`, using
RFC 1982 arithmetic.

## Model: views, zones, records, options

* **Views** are ordered; named uses the first view whose `match-clients` and
  `match-destinations` match. Once one view exists, named requires every
  zone to be in a view, so creating the first view offers to move existing
  zones into it (`adopt_zones`). See [views.md](views.md) for a split-DNS
  walkthrough.
* **Zones** have a type: `primary`, `secondary`, `stub`, `static-stub`,
  `forward`, `mirror`, `hint`, `redirect`, or `in-view` (share a zone from
  another view). Only primary zones have records in Jarl. The same zone name
  can exist in several views (split DNS); copy one with `clone_from`.
  A CIDR such as `192.168.1.0/24` or `2001:db8::/48` creates the matching
  reverse zone.
* **Records** are stored canonically (owner FQDN, type, TTL or "zone
  default", data in presentation format). TXT data is entered as plain text;
  Jarl quotes it and splits it into 255-byte strings. Relative names are
  resolved against the zone, and `www.example.com` without a final dot in a
  zone `example.com` is caught as the classic `www.example.com.example.com.`
  mistake. Adding an A/AAAA record with `ptr` also maintains the PTR record in
  the matching reverse zone. In reverse zones a record can be named by its
  address (`10.10.0.20`), and a PTR target without a final dot is read as
  the full host name, because a PTR never points into the reverse zone.
* **Options** are named.conf statements attached to a scope: server
  `options { }`, a view, a zone, or the top level (`logging`, `controls`,
  `statistics-channels`, `include`, ...). The UI and CLI know every BIND 9.20
  statement (353 of them, from the ARM): where it may appear, its grammar,
  and whether it is deprecated. Values are written in named.conf syntax, and
  Jarl checks them before named does. Statements that Jarl manages itself
  (`zone`, `view`, `acl`, `key`, a zone's `type`/`file`, a view's
  `match-clients`, ...) can't be set as raw options.
* **Settings** are Jarl's defaults for new zones: name servers, SOA contact
  and timers, default TTL, serial policy.

ACLs and TSIG keys are referenced by name. Jarl tracks every reference
(other ACLs, `match-clients`, `allow-*` options, `primaries ... key`, ...):
deleting something that is still in use is refused with the list of users,
and renaming updates every reference.

## Authentication and permissions

Your brief suggested HTTP basic auth for the web UI, with the web app
holding an API key. Jarl keeps the idea (each person has a username and
password, and every request is tied to a user and checked against that
user's permissions) but uses a login form and a session instead of basic
auth:

* Basic auth sends the password with every request and can't log out.
  Browsers cache it until they restart, and it can't be rate limited
  per user in a friendly way.
* A login form creates a session: an HttpOnly, SameSite=Strict cookie that
  expires after 12 hours (2 hours idle), plus a CSRF token the UI sends with
  every change. Failed logins are rate limited (8 per username and 20 per IP
  per 15 minutes). Passwords are hashed with argon2id.
* The web UI has no key of its own. It calls the same API as the CLI,
  authenticated as the logged-in user, so the audit log always names the
  person rather than "the web app".

The CLI and scripts use **API tokens** (`jarl_<id>_<secret>`). Only a hash
of each token is stored, and the full token is shown once.

* Each token belongs to a user. By default it has the user's permissions,
  or it can be **restricted** to a subset with its own grants and given an
  expiry. A token can never do more than its user.
* **Grants** are granular: resource (`zones`, `records`, `views`, `acls`,
  `tsig`, `options`, `audit`, `*`) × access (`read`/`write`), optionally
  narrowed by zone, view, record name pattern and record type. For example,
  `records:write zones=adbraun.lol names=_acme-challenge.* types=TXT` is
  what the ready-made `acme` role contains.
* Administrators can do everything, including managing users and server
  internals. Admin rights are never available through a restricted token.

Details: [permissions.md](permissions.md).

## Audit log

Jarl records every login (successful, failed or blocked by the rate
limit), every failed token, every changeset (created, staged, applied or
failed, discarded, reverted), token creation and revocation, user change,
password change, TSIG secret reveal and adoption. Each event has the user,
how they authenticated (session, which token, or local), the client IP and
the outcome. Tokens also remember when and from where they were last used. It is in the database
(searchable in the UI and with `jarl audit`) and is also written to the
journal through jarld's log.

## What Jarl deliberately doesn't do (yet)

* **Dynamic updates** (`allow-update`, `update-policy`): named would write
  to zone files that Jarl regenerates, so the validator rejects them on Jarl
  zones. Use the API (for example `jarl acme`) instead of nsupdate.
* **DNSSEC signing**: not managed yet. The zone directory is
  named-writable, so `dnssec-policy` can be added later.
* **Several servers from one jarld**: each jarld manages the named on its
  own host. A second server runs its own jarld with secondary zones, which
  copy the primary's zones by normal AXFR/IXFR (see "A secondary server" in
  [operations.md](operations.md)). The CLI can name both and run a command
  on both at once ([cli.md](cli.md#several-servers-and-groups)).