Compare commits
5
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
aca0abc780 | ||
|
|
9175d75d61 | ||
|
|
48f0082d8b | ||
|
|
d4399cf630 | ||
|
|
d2a25d654d |
No files matched your search
@@ -0,0 +1,197 @@
|
||||
# Working on Jarl
|
||||
|
||||
Notes for coding agents (and people) changing this repository. Read this
|
||||
first. For more depth, see [docs/development.md](docs/development.md) for
|
||||
layout, building, tests and releasing, and [docs/concepts.md](docs/concepts.md)
|
||||
for how Jarl works from the outside.
|
||||
|
||||
Jarl manages a BIND 9.20 server. It has three parts:
|
||||
|
||||
* `jarld`, a daemon with an HTTP API, an embedded web UI and SQLite storage;
|
||||
* `jarl`, a CLI for that API;
|
||||
* a Debian package.
|
||||
|
||||
It's all Go: module path `jarl`, version in `go.mod`, `CGO_ENABLED=0`. The
|
||||
SQLite driver is `modernc.org/sqlite`, so there is no cgo. The target is
|
||||
Debian/Ubuntu with BIND 9.20, using Debian's paths (`/etc/bind`, the `bind`
|
||||
group, AppArmor).
|
||||
|
||||
## Ground rules
|
||||
|
||||
* **The database is the source of truth.** Jarl renders `named.conf`,
|
||||
`keys.conf` and the zone files from it. Never hand-edit rendered output or
|
||||
add code that reads BIND's files back as state; the importer is the only
|
||||
exception.
|
||||
* **Every change goes through the engine** (`internal/engine`). An API
|
||||
request becomes ops in `expand.go`/`ops.go`. The engine then runs
|
||||
replay → validate → render → `named-checkconf`/`named-checkzone` →
|
||||
atomic install → `rndc reload` → verify serials, and rolls back on
|
||||
failure. Going through the engine gives a change permissions, dry runs,
|
||||
changesets, history and undo for free, so don't write to the store
|
||||
directly from handlers.
|
||||
* **Every user-visible change updates `CHANGELOG.md`** under
|
||||
`## [Unreleased]`, and the matching page in `docs/` in the same commit.
|
||||
Docs that may need it:
|
||||
|
||||
* `cli.md`, `api.md` and `configuration.md`;
|
||||
* `validation.md`, for new validation codes;
|
||||
* `metrics.md`;
|
||||
* the feature's own page.
|
||||
|
||||
Write docs in plain, direct English, like the existing pages.
|
||||
* **Error messages** say what is wrong and how to fix it, naming the object.
|
||||
For example: `view external is reachable from the internet …; set
|
||||
minimal-any yes`.
|
||||
* **New validation rules** get a stable `code`, a test, and a row in
|
||||
`docs/validation.md`.
|
||||
* **Schema changes** append a migration to the `migrations` slice in
|
||||
`internal/store/store.go`, and never edit an old one. The statistics
|
||||
database has its own list, `schema` in `internal/stats/store.go`. Add a note
|
||||
under "Upgrading" in the CHANGELOG when a downgrade would break.
|
||||
* `make test` runs `go vet` and a `gofmt -l` check, and both must pass.
|
||||
`goimports` isn't installed, so fix imports by hand.
|
||||
|
||||
## Building and testing
|
||||
|
||||
```sh
|
||||
make # bin/jarld, bin/jarl, bash completion
|
||||
make test # vet + gofmt check + go test ./...
|
||||
JARL_BIND_ROOT=DIR make test # also runs the integration tests against real BIND
|
||||
JARL_BIND_ROOT=DIR make e2e # scripts/e2e.sh: real named + jarld, drives every CLI command
|
||||
```
|
||||
|
||||
* **Real BIND without root.** Get the packages with `apt-get download` and
|
||||
unpack them with `dpkg -x` into a directory; that directory is
|
||||
`JARL_BIND_ROOT`. The package list is in docs/development.md, under "Tests
|
||||
against real BIND". Without `JARL_BIND_ROOT` the integration tests skip,
|
||||
so a green `make test` alone proves less than it looks.
|
||||
* **Run e2e after any engine, render, CLI or server change.** It prints a
|
||||
pass count, such as `606/606`; a failure prints the command and its
|
||||
output.
|
||||
* It uses fixed ports: DNS 25353, rndc 25953, HTTP 28053, lists 28054 and
|
||||
metrics 28055. Stop anything left running on them first.
|
||||
* `KEEP=1` leaves named and jarld running; the last line of the log has
|
||||
their PIDs.
|
||||
* `WORK=DIR` must name an empty directory.
|
||||
* **Certificate tests** run against Pebble in Docker, unless you set
|
||||
`PEBBLE=0`. If `docker info` fails, the Pebble block is silently skipped;
|
||||
on this machine it needs `DOCKER_HOST=unix:///var/run/docker.sock`.
|
||||
* **Unix socket paths** are limited to 107 bytes. Put test sockets under
|
||||
`/run/user/$UID/`, not under a deep temporary directory.
|
||||
* **The systemd sandbox isn't tested.** e2e runs outside systemd, so bugs
|
||||
that only appear in the sandbox (`RestrictAddressFamilies`,
|
||||
capabilities, read-only paths) need a package test. Run `make deb`, then
|
||||
install it in a `debian:trixie` container. That container has BIND 9.20;
|
||||
run it with `--privileged /sbin/init` when you need the real unit.
|
||||
|
||||
## Layout and where things live
|
||||
|
||||
docs/development.md has the full layout. Some places that are easy to miss:
|
||||
|
||||
* **Engine** (`internal/engine`):
|
||||
* `expand.go` turns requests into ops.
|
||||
* `linked.go`: linked zones, kept in step across views by `linkSync`.
|
||||
* `reverse.go`: PTR fill, auto-PTR and reverse zone suggestions.
|
||||
* `compare.go`: compare views.
|
||||
* `exposure.go`: fixes for internet exposure.
|
||||
* **Validation**: `internal/validate` holds all checks and lints.
|
||||
`internal/catalog` is the BIND 9.20 grammar and ARM text for every
|
||||
statement. Use it rather than hard-coding option knowledge.
|
||||
* **Rendering**: `internal/render` writes the configuration.
|
||||
`generated.go` holds the statements Jarl adds itself, shown to users as
|
||||
"Added by Jarl".
|
||||
* **Server**: `internal/server` has one `h_*.go` file per API area; routes
|
||||
are in `routes.go`. The JSON types are in `internal/api`, shared with the
|
||||
Go client in `internal/client`.
|
||||
* **CLI**: `cmd/jarl` is built on cobra. `complete.go` completes the
|
||||
placeholders in usage lines (`ZONE`, `VIEW`, …), so name them
|
||||
consistently.
|
||||
* **Statistics**: `internal/stats` is query statistics, kept in a
|
||||
**separate** SQLite file, `stats.db`, with its own migrations.
|
||||
* **Feature packages**: each has its own doc page.
|
||||
* `encdns`: encrypted DNS;
|
||||
* `certs` and `acme`: certificates and ACME;
|
||||
* `blocking`: response policy zones;
|
||||
* `dnssec`;
|
||||
* `nscheck`: queries to other name servers;
|
||||
* `mailcheck`: mail reverse DNS;
|
||||
* `totp`: two-factor login;
|
||||
* `backup`;
|
||||
* `porkbun`: registrar API, with an in-memory `Fake`. Tests never call
|
||||
the real API.
|
||||
|
||||
## Web UI rules
|
||||
|
||||
`internal/web/static/` holds the UI: plain ES modules, no framework, no
|
||||
build step. The files are embedded with `go:embed`.
|
||||
|
||||
* **Rebuild jarld after editing them**, or you'll be testing the old files.
|
||||
* **Build DOM only with `h()`** from `assets/ui.js`. Never use `innerHTML`
|
||||
or `insertAdjacentHTML`. That's how record data can't inject markup.
|
||||
* **The CSP is `self` only.** That rules out:
|
||||
* inline scripts;
|
||||
* `style="…"` attributes and inline style strings (`el.setAttribute("style", …)`);
|
||||
* external fonts and CDNs.
|
||||
|
||||
Use a CSS class in `style.css`, or set individual properties with
|
||||
`el.style.prop = …` (CSSOM is allowed).
|
||||
* **Colors are CSS custom properties** with light and dark values. Reuse the
|
||||
existing tokens.
|
||||
* **Pages live in `assets/pages/`**, one file per route. The router is in
|
||||
`app.js` and the API calls are in `api.js`.
|
||||
* **What the user wants:** functionality and ease of use matter more than
|
||||
visual polish. In pick lists, leave every checkbox unticked by default.
|
||||
|
||||
There's no JS test suite. To catch syntax and import errors without a
|
||||
browser, bundle `assets/app.js` with esbuild, or load the page in headless
|
||||
Chrome.
|
||||
|
||||
## BIND facts that bit us
|
||||
|
||||
* named 9.20 crashes on an empty `keys {}` in a `dnssec-policy`. Validation
|
||||
refuses it.
|
||||
* `named-checkconf` doesn't catch a TTL above `max-zone-ttl`, but named then
|
||||
refuses to load the zone.
|
||||
* `responselog` only takes effect when named starts; use `rndc responselog
|
||||
on|off` instead.
|
||||
* `named -g` sends all logging to stderr, so tests use `-f`.
|
||||
* RPZ client-IP rules see IPv4 clients as v4-mapped (`::ffff:0:0/96`), so
|
||||
IPv6 prefix covers must leave that range out.
|
||||
* named serves DoH in RFC 8484 wire format only, not the JSON API. Test it
|
||||
with BIND's `dig +https` / `dig +tls`, not curl or kdig.
|
||||
* named can't forward over DoH, only over DoT (`forwarders port 853 tls NAME`).
|
||||
* `rndc reconfig` makes named reload renewed TLS certificate files.
|
||||
|
||||
## Git and releases
|
||||
|
||||
These are the maintainer's rules; follow them unless told otherwise:
|
||||
|
||||
* **Commit only when asked. Never push**; the maintainer pushes, to Gitea
|
||||
at gitea.adbraun.lol.
|
||||
* **Work on `dev-X.Y.Z` branches** (for example `dev-0.6.1`) off `main`.
|
||||
Don't name a branch like a tag (`v0.5.2`): `git push` then fails with
|
||||
"src refspec matches more than one".
|
||||
* **A release** is a commit titled `Release X`.
|
||||
* It sets `VERSION`.
|
||||
* It updates the status line in the README.
|
||||
* For a final release, it moves `[Unreleased]` to
|
||||
`## [X] - YYYY-MM-DD`. A release candidate (`X-rc.N`) may leave the
|
||||
notes under `[Unreleased]`.
|
||||
* Tag it annotated: `git tag -a vX -m "Jarl X"`.
|
||||
* Before tagging, `make test` and the e2e run must pass.
|
||||
* Check the notes with `VERSION=X scripts/gitea-release.sh notes`.
|
||||
* **Pushing a `v*` tag** runs `.gitea/workflows/release.yml`. It builds the
|
||||
tarballs, the CLI for each OS and the `.deb`s, and publishes them.
|
||||
* **Before a minor release**, test an upgrade from the previous release's
|
||||
`.deb`s in a container. Check data, history, answers from named, and that
|
||||
an old backup restores.
|
||||
|
||||
## Shell hygiene
|
||||
|
||||
* **Never kill processes by pattern** with `pkill -f` or
|
||||
`pgrep -f … | xargs kill` in the same shell command. The pattern matches
|
||||
the shell running it, and kills that shell. Find the PID with `ps` and
|
||||
kill it by number.
|
||||
* **Start long-running test servers in the background** (your tool's
|
||||
background mode, not `&` in a shell that exits), so they outlive the
|
||||
command.
|
||||
@@ -8,6 +8,14 @@ All notable changes to Jarl are documented here. The format follows
|
||||
|
||||
### Added
|
||||
|
||||
- `jarl.service` has the alias `jarld.service`, so `systemctl restart
|
||||
jarld` works too. The alias is linked when the unit is enabled; the
|
||||
package relinks a unit enabled by an older version on upgrade.
|
||||
|
||||
## [0.6.0] - 2026-10-13
|
||||
|
||||
### Added
|
||||
|
||||
- Reverse zones made easy (docs/reverse.md):
|
||||
- **Missing reverse zones**: the Zones page and `jarl zone
|
||||
suggest-reverse` list the reverse zones the private views lack for
|
||||
@@ -95,6 +103,9 @@ All notable changes to Jarl are documented here. The format follows
|
||||
|
||||
### Fixed
|
||||
|
||||
- Encrypted DNS: the client hints tested with kdig (Knot's, often not
|
||||
installed) and with curl's JSON API, which named's DoH doesn't speak;
|
||||
they use BIND's `dig +tls` / `dig +https` now.
|
||||
- Server options → Added by Jarl: long statements didn't wrap, and the
|
||||
browser console reported a Content Security Policy violation (an inline
|
||||
style the page's policy blocks).
|
||||
|
||||
@@ -0,0 +1,8 @@
|
||||
@AGENTS.md
|
||||
|
||||
## Claude Code specifics
|
||||
|
||||
- End commit messages with a `Co-Authored-By:` trailer naming the model,
|
||||
e.g. `Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>`.
|
||||
- Keep temporary scripts, extracted BIND trees, screenshots and test
|
||||
servers' work directories in the session scratchpad, not in the repo.
|
||||
@@ -114,6 +114,7 @@ install-files:
|
||||
|
||||
uninstall:
|
||||
rm -f $(DESTDIR)$(BINDIR)/jarld $(DESTDIR)$(BINDIR)/jarl $(DESTDIR)$(UNITDIR)/jarl.service \
|
||||
$(DESTDIR)$(UNITDIR)/jarld.service \
|
||||
$(DESTDIR)$(UNITDIR)/jarl-backup.service $(DESTDIR)$(UNITDIR)/jarl-backup.timer \
|
||||
$(DESTDIR)$(BASHCOMPDIR)/jarl $(DESTDIR)$(BASHCOMPDIR)/jarld \
|
||||
$(DESTDIR)$(SYSUSERSDIR)/jarl.conf $(DESTDIR)$(TMPFILESDIR)/jarl.conf
|
||||
|
||||
@@ -202,8 +202,7 @@ review step.
|
||||
|
||||
## Status
|
||||
|
||||
Version 0.5.2 is the current release, and 0.6.0 is in testing
|
||||
(0.6.0-rc.2): it makes reverse zones easy (suggested from your records,
|
||||
Version 0.6.0 is the current release. It makes reverse zones easy (suggested from your records,
|
||||
created in the right view and filled, PTR records kept in step), rebuilds
|
||||
comparing a zone's views, lets you explore the statistics down to each
|
||||
client and name (a query log), shows the databases' size, and gives
|
||||
|
||||
@@ -38,6 +38,13 @@ if [ "$1" = configure ]; then
|
||||
if systemctl is-enabled --quiet jarl-backup.timer 2>/dev/null; then
|
||||
systemctl start jarl-backup.timer || true
|
||||
fi
|
||||
# The jarld.service alias is linked when jarl.service is enabled;
|
||||
# relink a unit enabled before the alias existed (this neither
|
||||
# starts nor stops jarld).
|
||||
if [ -n "$2" ] && systemctl is-enabled --quiet jarl.service 2>/dev/null \
|
||||
&& [ ! -e /etc/systemd/system/jarld.service ]; then
|
||||
systemctl reenable jarl.service >/dev/null 2>&1 || true
|
||||
fi
|
||||
# An upgrade restarts a running jarld (its database is upgraded
|
||||
# when it starts). A first install starts nothing: Jarl needs
|
||||
# 'jarld init' and 'jarld adopt' first.
|
||||
|
||||
@@ -65,3 +65,5 @@ SystemCallErrorNumber=EPERM
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
# 'systemctl restart jarld' works too (the daemon's binary is jarld).
|
||||
Alias=jarld.service
|
||||
@@ -32,6 +32,12 @@ internal/
|
||||
blocking/ blocklist parser, response policy zone writers, and matching a name the way named does
|
||||
(the downloads are in engine/blocklists.go)
|
||||
config/ jarld's YAML configuration
|
||||
certs/ certificates: Jarl as an ACME client (lego), renewal, files for named
|
||||
encdns/ encrypted DNS (DoT/DoH) setup, status and health checks
|
||||
mailcheck/ reverse DNS check of the addresses mail is sent from
|
||||
totp/ two-factor login codes and recovery codes
|
||||
version/ what a build reports (release tag, commit, commits since a tag)
|
||||
testutil/ test helpers, chiefly finding BIND's tools (JARL_BIND_ROOT)
|
||||
deploy/ systemd units (jarld, the daily backup service and timer), sysusers.d and tmpfiles.d files
|
||||
deploy/debian/ the Debian package's control template and postinst/prerm/postrm
|
||||
contrib/acme.sh/ acme.sh DNS API hook
|
||||
|
||||
+11
-4
@@ -111,14 +111,21 @@ For DoH clients: https://dns.adbraun.lol/dns-query
|
||||
| Chrome, Edge | Settings → Privacy and security → Security → Use secure DNS → custom: the DoH URL |
|
||||
| Windows 11 | Settings → Network → your connection → DNS server assignment → Edit: the server's address, DNS over HTTPS on (manual template), the DoH URL |
|
||||
|
||||
Test from a machine on your network:
|
||||
Test from a machine on your network, with BIND's own `dig` (9.18 or
|
||||
later; `bind9-dnsutils` on Debian):
|
||||
|
||||
```sh
|
||||
kdig +tls-ca +tls-host=dns.adbraun.lol @10.0.10.5 example.com
|
||||
dig +https @10.0.10.5 example.com
|
||||
curl -s -H 'accept: application/dns-json' 'https://dns.adbraun.lol/dns-query?name=example.com'
|
||||
dig +tls +tls-ca +tls-hostname=dns.adbraun.lol @10.0.10.5 example.com # DoT, port 853
|
||||
dig +https +tls-ca +tls-hostname=dns.adbraun.lol @10.0.10.5 example.com # DoH, port 443, /dns-query
|
||||
```
|
||||
|
||||
`+tls-ca` checks the certificate against the system's CAs and
|
||||
`+tls-hostname` against the host name, as clients do; add `-p PORT` for
|
||||
another port. Without `+tls-ca` the query is still encrypted, but the
|
||||
certificate isn't checked. named's DoH speaks the standard wire format
|
||||
(RFC 8484), not the JSON API some public resolvers also offer, so
|
||||
`curl ... accept: application/dns-json` doesn't work against it.
|
||||
|
||||
**Android away from home.** Private DNS in hostname mode applies on every
|
||||
network. Away from home, your server can't be reached, and Android then
|
||||
resolves nothing at all ("Private DNS server cannot be accessed"). Use a
|
||||
|
||||
@@ -317,6 +317,10 @@ systemctl status jarl
|
||||
journalctl -u jarl -f
|
||||
```
|
||||
|
||||
The unit also answers to `jarld` (`Alias=jarld.service`), so
|
||||
`systemctl restart jarld` and `journalctl -u jarl` work alike once it
|
||||
is enabled.
|
||||
|
||||
Open `https://<management-address>:8053/` and log in as `admin`.
|
||||
|
||||
## 9. Set up the CLI
|
||||
|
||||
@@ -35,13 +35,15 @@ function statusPanel(s) {
|
||||
];
|
||||
const addr = (s.dot[0] || s.doh[0] || {}).address || "ADDRESS";
|
||||
const dotPortArg = s.dot[0] && s.dot[0].port !== 853 ? " -p " + s.dot[0].port : "";
|
||||
const dohPortArg = s.doh[0] && s.doh[0].port !== 443 ? " -p " + s.doh[0].port : "";
|
||||
const clients = [];
|
||||
if (s.dot_host) {
|
||||
clients.push(h("h3", null, "DNS over TLS"), copyBox(s.dot_host),
|
||||
h("ul", { class: "small" },
|
||||
h("li", null, h("strong", null, "Android: "), "Settings → Network → Private DNS → hostname: ", h("code", null, s.hostname), ". Away from home Android can't reach this server and then resolves nothing, so use a VPN home (WireGuard) or switch Private DNS to Automatic when you leave."),
|
||||
h("li", null, h("strong", null, "Linux (systemd-resolved): "), h("code", null, "DNS=" + addr + "#" + s.hostname), " and ", h("code", null, "DNSOverTLS=yes"), " in /etc/systemd/resolved.conf (", h("code", null, "opportunistic"), " falls back to plain DNS away from home)."),
|
||||
h("li", null, h("strong", null, "Test: "), h("code", null, "kdig +tls-ca +tls-host=" + s.hostname + dotPortArg + " @" + addr + " example.com"))));
|
||||
h("li", null, h("strong", null, "Test: "), h("code", null, "dig +tls +tls-ca +tls-hostname=" + s.hostname + dotPortArg + " @" + addr + " example.com"),
|
||||
" (BIND's dig; +tls-ca checks the certificate against the system's CAs)")));
|
||||
}
|
||||
if (s.doh_url) {
|
||||
clients.push(h("h3", null, "DNS over HTTPS"), copyBox(s.doh_url),
|
||||
@@ -49,7 +51,7 @@ function statusPanel(s) {
|
||||
h("li", null, h("strong", null, "Firefox: "), "Settings → Privacy & Security → DNS over HTTPS → Increased protection (falls back to normal DNS) → Custom: the URL above."),
|
||||
h("li", null, h("strong", null, "Chrome, Edge: "), "Settings → Privacy and security → Security → Use secure DNS → custom: the URL above."),
|
||||
h("li", null, h("strong", null, "Windows 11: "), "Settings → Network → your connection → DNS server assignment → Edit: " + addr + ", DNS over HTTPS on (manual template), the URL above."),
|
||||
h("li", null, h("strong", null, "Test: "), h("code", null, "curl -s -H 'accept: application/dns-json' '" + s.doh_url + "?name=example.com'"), " (or ", h("code", null, "dig +https @" + addr + " example.com"), ")")));
|
||||
h("li", null, h("strong", null, "Test: "), h("code", null, "dig +https +tls-ca +tls-hostname=" + s.hostname + dohPortArg + " @" + addr + " example.com"))));
|
||||
}
|
||||
return [
|
||||
h("div", { class: "panel" }, h("h2", null, "Set up ", badge("on", "ok")), h("dl", { class: "kv" }, rows)),
|
||||
|
||||
Reference in new issue
Block a user