Compare commits

..
Author SHA1 Message Date
adamandClaude Opus 5.5 ecffc2d5c1 Let jarld recognize this host's addresses under systemd
release / release (push) Successful in 1m58s
The unit's RestrictAddressFamilies lacked AF_NETLINK, which listing the
host's addresses needs, so the name server checks didn't know this
server's own addresses: the Name servers tab queried them like another
server's, and the delegation check didn't sign them with the view key.
Allow AF_NETLINK, and fall back to /proc/net without it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 18:35:06 +01:00
adamandClaude Opus 5.5 9d9ae37367 Release 0.4.0-rc.3
release / release (push) Successful in 1m56s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 18:19:56 +01:00
adamandClaude Opus 5.5 384f6b3060 Give every view a TSIG key that selects it
Each view's match-clients starts with "key jarl-view-NAME" (after
"!key" for the views listed after it), so a query signed with a view's
key lands in that view from any address. The secrets are derived from a
random seed in Jarl's database. The delegation check signs its queries
to this server's own addresses with the key of the zone's view: before,
they got whichever view matched the server's address, usually internal.
jarl view key / GET /views/{id}/key show a key for dig -k (tsig:write,
audited). jarl-view- key names are reserved; the importer drops view
keys from a configuration Jarl wrote.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 18:19:10 +01:00
adamandClaude Opus 5.5 c696d02513 Give each view its own page, and add a CNAME from an A record
Views open a page like a zone's, with Settings, Options and Zones tabs
(and New zone in that view); the Views page lists and orders them. A
and AAAA records get a + CNAME button that fills in the add form.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 18:12:56 +01:00
adamandClaude Opus 5.5 7be0762f48 Server statistics, multi-view record commands, and rc.3 fixes
Added:
- Server statistics: named's memory and CPU (from /proc), cache size,
  sockets and client counts against their limits (statistics channel,
  rndc status), host memory and load, and free disk space. Memory and
  CPU are kept over time as gauges (sum and sample count, averaged in
  timelines). Statistics page Server section, jarl stats server,
  GET /stats/server.
- jarl record add|set|delete|disable|enable --view a,b | all: the same
  change in each view, applied as one changeset.
- Blocking can be paused until resumed (jarl block pause indefinitely,
  "Until I resume", {"indefinitely": true}).

Changed:
- List options (allow-transfer, forwarders, ...) get their braces added
  when given without them.
- The dashboard lists named-checkconf's warnings about the running
  configuration; the CLI prints them after an apply.
- Stats page: tile and card rows fill their width, and the queries and
  failed answers charts share margins so their time axes line up.
- Dashboard: options have their own tile.
- 24-hour times in the web UI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 18:03:05 +01:00
adamandClaude Opus 5.5 b097e10bef Keep options in a fixed order, and say which block a diff hunk is in
Options of a block (server options, a view, a zone) are listed and
written grouped by topic, alphabetical within a group, so the same
option is in the same place everywhere. Top-level statements keep the
order they were added in.

named.conf diffs add the enclosing blocks to each hunk header, like
git's function context: @@ -50,6 +49,8 @@ view "internal" {.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 17:06:22 +01:00
adamandClaude Opus 5.5 73d97ecc0a Release 0.4.0-rc.2
release / release (push) Successful in 1m55s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 16:51:29 +01:00
adamandClaude Opus 5.5 33060f86ce Don't count a per-view counter going down as a restart of named
After a reconfig that changed named's views, the cache and resolver
counters (summed over views) went down, which was taken for a restart:
every counter then counted its total since named started for one minute.
Only a new boot time means a restart now; a counter that goes down
counts 0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 16:51:24 +01:00
adamandClaude Opus 5.5 89663e366b Disable zones instead of deleting them
A disabled zone stays in Jarl with its records, options and history,
still editable, but named doesn't load it; enabling it serves it as it is
then. Signed zones, and zones another view shares, can't be disabled.
jarl zone disable|enable, the zone page's Disable/Enable button.
Database schema 7.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 13:45:55 +00:00
adamandClaude Opus 5.5 59baccd636 Release 0.4.0-rc.1
release / release (push) Successful in 1m53s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 12:56:15 +00:00
adamandClaude Opus 5.5 02c393a6bb Apply pending changes from the banner
An Apply now button on the pending-changes banner applies the draft and
stays on the page; if the changes are refused, a dialog says why.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 12:56:10 +00:00
adamandClaude Opus 5.5 d4756444ac Edit blocklists and rules in the web UI
An Edit button opens a dialog for a list's name, URL, views and comment,
or a rule's action, scope, views and comment. The download-now button is
called Refresh. Checkboxes in dialogs are no longer stretched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 12:50:41 +00:00
adamandClaude Opus 5.5 67f14b030f Make disabled records obvious, and offer the right fix when checking a blocked name
Disabled records are hatched and badged next to their name. Checking a
name blocked by your own rule for that name offers to remove the rule,
instead of an allow rule that would be refused as a duplicate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 12:45:10 +00:00
adamandClaude Opus 5.5 ad8eb3fc3d Show and edit option values laid out as in named.conf
Values with blocks (logging, controls, ...) are shown over several lines
with indentation, and edited that way: the editor grows with its content,
Tab indents, Enter keeps the indentation. With a draft open, option tables
show the draft's values.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 11:55:41 +00:00
adamandClaude Opus 5.5 27d1f9e538 Pause blocking, blocked clients and a blocked line in the statistics, compact view picker
jarl block pause|resume (and the Blocking page) lets every name through
for a while; jarld resumes on time. The lists stay loaded: the rules zone
just passes everything. Statistics gain the clients with the most blocked
queries and blocked queries on the queries chart. The view choice in the
blocking forms is a dropdown.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 11:10:44 +00:00
adamandClaude Opus 5.5 9fc23310bd Add blocking: blocklists and block/allow rules with BIND response policy zones
jarld downloads blocklists (Adblock, hosts or plain domains), checks them
daily and reloads named only when a list changed; your own block and
allow rules take precedence, and the most specific rule wins. Lists and
rules apply per view (default: every view that offers recursion). Web UI
Blocking page with name checks, jarl blocklist and jarl block, the
/blocking API, blocked counts in the statistics. Database schema 6.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 09:24:19 +00:00
adamandClaude Opus 5.5 436cd79ec9 Release 0.3.0
release / release (push) Successful in 1m49s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 23:39:15 +00:00
adamandClaude Opus 5.5 c645dae14f Release 0.3.0-rc.2
release / release (push) Successful in 1m50s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 23:27:56 +00:00
adamandClaude Opus 5.5 403f85486d Mark signed zones with a padlock, and name clients in statistics
The zones list shows a padlock after signed zones, linking to their
DNSSEC tab. Client lists in the statistics show the host name Jarl's
records have for each address (PTR, else A/AAAA); no DNS queries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 23:27:53 +00:00
adamandClaude Opus 5.5 4113d5d6fc Release 0.3.0-rc.1
release / release (push) Successful in 1m47s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 22:46:55 +00:00
adamandClaude Opus 5.5 f716fb8dd1 Script-friendly CLI output, and keep record list rows on one line
-o id and -o field=NAME,... print just the IDs or chosen fields; -q on
show and list prints the IDs. jarl record list shortens data too long
for the terminal (--wide shows it in full).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 22:46:43 +00:00
adamandClaude Opus 5.5 113a6913a2 Give the DS command for a parent zone in Jarl, and allow port 443
The DNSSEC tab and 'jarl zone dnssec status' give the jarl record add
command (and an Add to parent button) when the parent zone is in Jarl.
jarl.service grants CAP_NET_BIND_SERVICE so the web UI can listen on 443.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 22:42:16 +00:00
adamandClaude Opus 5.5 295c549479 Disable records instead of deleting them
A disabled record stays in Jarl with its ID, TTL and comment but is
left out of the zone; the zone file lists it as a comment. Web UI
toggle, jarl record disable|enable, record.disable|enable operations
and API endpoints. Database schema 5.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 22:33:02 +00:00
adamandClaude Opus 5.5 5f04ab523b Separate record groups with lines instead of greying repeated names
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 22:09:55 +00:00
adamandClaude Opus 5.5 c9590cc3d4 Release 0.2.0
release / release (push) Successful in 1m43s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 21:15:13 +00:00
adamandClaude Opus 5.5 bb6b4ab64d Stream the delegation check's queries too
release / release (push) Successful in 1m42s
Every DNS query the name server checks send is now reported from
exchangeHostPort as a dig command line with a short answer (addresses,
referral, DS, NXDOMAIN) and its round-trip time; the serials check's SOA
queries keep their own events with the server's result. The delegation
check streams with ?stream=1 like /serials, and the Name servers tab
shows its queries as they happen, keeps its result and follows a running
check when you come back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 16:00:39 +00:00
adamandClaude Opus 5.5 1d6c28ba69 Release 0.2.0-rc.2
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 15:49:11 +00:00
adamandClaude Opus 5.5 020223d1a5 Show the Name servers check as it happens, and keep its results
The zone's Name servers tab no longer waits for the other servers before
showing: it lists each query as a dig command line with a waiting
counter, fills in the answers as they arrive, then shows the result with
the queries under "Queries sent". Results (serials and delegation) are
kept for 10 minutes so revisiting the tab is instant; "Check again" asks
anew, and a check keeps running when the tab is left.

GET /zones/{zone}/serials?stream=1 streams the queries as
newline-delimited JSON (nscheck.Checker.Watch); without it the endpoint
is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 15:49:11 +00:00
123 changed files with 9495 additions and 420 deletions

No files matched your search

+209
View File
@@ -8,6 +8,199 @@ All notable changes to Jarl are documented here. The format follows
### Added
- View keys: every view gets a TSIG key, `jarl-view-NAME`, at the front of
its match-clients, so a query signed with it lands in that view from any
address. Jarl signs its delegation check's queries to this server's own
addresses with the key of the zone's view: before, they got whichever
view matched the server's own address (usually internal), so the check
could report the wrong serial for the external view. `jarl view key
VIEW` shows a key, for `dig -k` (docs/views.md). The first apply after
upgrading adds the keys to named.conf and keys.conf.
- Server statistics: named's memory (resident, and its own count) and
CPU use, its cache size, sockets, and recursive and TCP clients against
their limits; the host's memory and load; free space where zone files,
logs and Jarl's data are. Memory and CPU are kept over time like the
other statistics. Statistics page → Server, `jarl stats server`, API
`/stats/server`. Read from named's statistics channel, `rndc status` and
/proc, with no extra privileges.
- CLI: the record commands that change records take several views,
`--view internal,external`, or `--view all`, and make the change in each,
as one changeset (all views or none).
- Blocking can be paused until resumed, like Pi-hole's "indefinitely":
`jarl block pause indefinitely`, **Until I resume** on the Blocking page,
`{"indefinitely": true}` in the API.
- Disabled zones: a zone can be disabled instead of deleted. named stops
loading it (its zone statement is left out of named.conf), while Jarl
keeps it with its records, options and history, still editable;
enabling it serves it as it is then. Disabling says what clients get
instead (recursion, or REFUSED). Signed zones can't be disabled. Web
UI: Disable/Enable on the zone page, a badge in the zone list. CLI:
`jarl zone disable|enable`. API: `PATCH /zones/{zone}` with
`{"disabled": true}`. Database schema 7.
- Blocking (docs/blocking.md): named answers "doesn't exist" for ads,
trackers and malware, using BIND's response policy zones.
- Blocklists in Adblock, hosts or plain-domain format, downloaded by jarld
and checked daily (`blocking.refresh`). A list is downloaded only when
the server says it changed, checked with named-checkzone, and reloaded
only when its content differs; a failed update keeps the previous
version and shows on the dashboard.
- Your own block and allow rules, for a domain and the names below it or
the exact name. Rules take precedence over every list, and the most
specific rule wins.
- Lists and rules apply to chosen views, or to every view that offers
recursion. They are configuration: staged, reviewed and reverted like
any change.
- Checking a name says whether it is blocked and by which rule or list
entry, with an Allow button in the web UI.
- Web UI: a Blocking page, with well-known lists to pick and named's
memory per list. CLI: `jarl blocklist` and `jarl block`. API:
`/blocking/lists`, `/blocking/rules`, `/blocking/check`.
- Pausing: `jarl block pause [DURATION]` or the Blocking page lets every
name through for a while (at most a day); blocking resumes by itself.
The lists stay loaded, so it takes effect at once.
- Statistics: blocked queries over time (a line on the queries chart)
and their share, blocked per list, the most blocked names and the
clients with the most blocked queries (`jarl stats top blocked`,
`top blocked-clients`). Blocked answers no longer count as names that
don't exist.
- The view choice in the web UI's forms is a compact dropdown, however
many views there are.
- A `blocking` permission; database schema 6.
### Changed
- Web UI: each view has its own page, like a zone: Settings (match lists,
comment, colour), Options and Zones tabs (with **New zone in VIEW**),
and Delete. The Views page lists them in order, with the arrows that
reorder them; view links everywhere open the view's page.
- Web UI: A and AAAA records have a **+ CNAME** button that fills in the
add form with a CNAME pointing to the record's name, leaving only the
alias's name to type.
- Options are kept in one fixed order, in named.conf, the web UI and the
CLI: grouped by topic (files, listening, access control, recursion and
forwarding, DNSSEC, transfers and notify, then the rest), alphabetical
within a group. The same option is now in the same place in every view
and zone. The first apply after upgrading reorders named.conf (a diff
that only moves lines). Top-level statements keep their order.
- Options that end in a list (`allow-transfer`, `forwarders`,
`listen-on`, ...) can be given without their braces, in the web UI, CLI
and API: `10.0.10.53; key xfer` becomes `{ 10.0.10.53; key xfer; }`.
Before, only some server options (those edited one element per line)
added them.
- The dashboard's configuration checks include named-checkconf's own
warnings about the running configuration (such as `both "recursion no;"
and "allow-recursion" active`), which before only showed while applying.
The CLI now prints them after an apply too.
- Statistics page: cards and tiles fill their rows instead of leaving one
alone beside an empty space, and the queries and failed answers charts
have the same margins, so their time axes line up.
- Dashboard: options have their own tile; "0 TSIG keys · 22 options" read
as if the options belonged to the keys.
- Web UI times use the 24-hour clock.
- Diffs of named.conf say which block each change is in, after the `@@`
line numbers (`@@ -50,6 +49,8 @@ view "internal" {`), like git does
with function names. Moving an option from the server's options into a
view no longer looks like two unrelated edits.
- Web UI options (server, top-level, view and zone): values with blocks,
such as logging, are shown and edited laid out as in named.conf, over
several lines with indentation. The editor grows with its content, Tab
indents and Enter keeps the indentation (Esc, then Tab, moves on). With
a draft open, the tables show the draft's values, so a staged edit stays
visible after Save.
- The pending-changes banner has an Apply now button: it applies your
draft without going to the Changes page, and stays on the page you're
on. If the changes are refused, a dialog says why and links to Changes.
- Blocking page: blocklists and rules have an Edit button (name, URL,
views, comment; a rule's action and scope). The list button that
downloads now is called Refresh, like `jarl blocklist refresh`.
- Records table: disabled records are hatched and have a "disabled" badge
next to their name, instead of only a muted colour and a badge in the
comment column.
### Fixed
- Under the shipped systemd unit, jarld couldn't list the host's
addresses (it needs a netlink socket, which `RestrictAddressFamilies`
didn't allow), so the name server checks didn't recognize this server's
own addresses. The Name servers tab then queried them like another
server's, and the delegation check didn't sign those queries with the
view's key: with views, both got the view matching the server's own
address (usually internal) and reported a wrong serial. The unit now
allows `AF_NETLINK`, and without it jarld reads the addresses from
/proc.
- Statistics: after a reconfig that added, removed or recreated views
(creating the first view, for instance), a minute could count everything
since named started: millions of queries in an hour. A per-view counter
(cache, upstream times, validation) going down was taken for a restart
of named. Now only a new boot time means a restart, and a counter that
goes down counts 0 for that minute.
- Blocking page, checking a name: when your own block rule is for that very
name, it offers to remove the rule instead of an allow rule, which was
refused as a second rule for the same name. It also names the rule that
matched rather than the policy zone entry.
- docs/validation.md lacked the `disabled_conflict` warning added in 0.3.0.
## [0.3.0] - 2026-10-09
### Added
- Disabled records: a record can be disabled instead of deleted. It stays
in Jarl with its ID, TTL and comment but is left out of the zone (the
zone file lists it as a comment), and enabling it puts it back exactly
as it was. Web UI: a Disable/Enable button on each record, disabled
records greyed out with a badge, a filter for them, and a count in the
zone list. CLI: `jarl record disable|enable` (by ID or by name, type and
data, `--ptr` for the PTR record, `--missing-ok`), `record list
--disabled`, and `disable:`/`enable:` entries for `jarl apply -f`. API:
`POST /records/{id}/disable|enable`, `POST
/zones/{zone}/records/disable|enable`, a `disabled` field on records,
`?disabled=` on record lists and `record.disable|enable` operations.
Validation checks the zone as it is served: the last NS record can't be
disabled, enabling a record is checked like adding it, and a disabled
record that a CNAME now blocks gets a warning. Database schema 5.
- DNSSEC for a zone whose parent zone is in Jarl: the DNSSEC tab gives the
`jarl record add PARENT NAME DS ...` command to copy and an "Add to
PARENT" button (staged like any change), links to the parent zone, and
says when the parent has the DS record. `jarl zone dnssec status` prints
the command too, and turning signing off gives the matching `jarl record
delete`.
- CLI output for scripts: `-o id` prints just the IDs of what a command
shows or lists (`jarl zone show adbraun.lol -o id`), `-o
field=view,soa.serial` just those fields, tab separated. `-q` on `show`
and `list` prints the IDs too. With a change (`jarl record add ... -o
id`) they apply to the changed object.
- Statistics: client lists (busiest and refused clients, in the web UI,
`jarl stats top clients|refused` and `jarl stats summary`) show the host
name Jarl's records have for each address: the PTR record's target, or
else the name of an A/AAAA record. Looked up in Jarl's own data, so no
DNS queries; an address without a record shows as before.
- Zones list: a padlock after each signed zone's name, linking to its
DNSSEC tab (open and amber while signing is being turned off).
### Changed
- The web UI and API can listen on ports below 1024, such as 443:
`jarl.service` now gives jarld the `CAP_NET_BIND_SERVICE` capability
(and only that). With a unit that doesn't, jarld's error says what's
missing. See docs/configuration.md, "HTTPS on port 443".
- Records table: names are no longer greyed out when they repeat the row
above (it made those records look inactive). Records of the same name
are separated by a faint line instead, and a full line marks where the
next name starts.
### Fixed
- `jarl record list`: one long value (a DKIM key) padded every row to its
length, so on a terminal each row wrapped onto a second, blank-looking
line. Long data is now shortened to fit the terminal; `--wide` shows it
in full, as does output to a pipe or file.
## [0.2.0] - 2026-10-09
### Added
- DNSSEC signing per zone (docs/dnssec.md): `jarl zone dnssec
enable|status|check-ds|disable|wipe` and the zone page's DNSSEC tab, or
the zone's new `dnssec_policy` field in the API. named signs the zone
@@ -104,6 +297,22 @@ All notable changes to Jarl are documented here. The format follows
- The view in the zone list, and on the zone page, links to that view,
and the views page highlights it. It used to be plain text, and the zone
page linked to the views page as a whole.
- The zone's Name servers tab shows at once and follows the check as it
happens. Each query is listed as a dig command line (`dig @192.0.2.1
example.org. SOA +norec`) with a "waiting 1.8 s" counter until it is
answered. The finished result then replaces the progress, and the queries
stay under "Queries sent". Results, the delegation check's included, are
kept for 10 minutes, so returning to the tab shows them at once; "Check
again" asks anew. A check keeps running if you leave the tab. Behind it,
`GET /zones/{zone}/serials?stream=1` streams the queries as
newline-delimited JSON (docs/api.md).
- The delegation check streams its queries the same way, every one of
them: finding the parent zone, its servers' addresses, the NS and DS
queries to each parent server and the SOA from each delegated server.
Each shows a short answer ("referral to katelyn.ns.cloudflare.com., …",
"DS 2371", "NXDOMAIN"). The serials check now lists its address lookups
as separate A and AAAA queries. `GET /zones/{zone}/delegation?stream=1`
streams them.
- Switching a zone's tabs (Records, Settings, …) no longer blanks the page
while the tab loads: the page changes once the tab is ready, and
"Loading…" appears only if a page takes a while.
+10 -3
View File
@@ -71,6 +71,10 @@ error: www.adbraun.lol has a CNAME, so it can't have any other records (found A)
queries, record types, cache hit rate. Shown in the web UI and the CLI,
and exportable as JSON. Stored as aggregates in a separate database with
retention you set.
* **Blocking** like Pi-hole, done by named itself: blocklists (Adblock,
hosts or plain domains) downloaded and kept up to date, your own block
and allow rules that beat every list, per view, with "why is this
blocked?" checks and blocked counts in the statistics.
* **Let's Encrypt DNS-01** built in: `jarl acme` plus certbot, acme.sh and lego
hooks, an HTTPS-only endpoint for lego's and Traefik's `httpreq` provider,
and CNAME delegation to a dedicated ACME zone.
@@ -116,6 +120,7 @@ in `/etc/jarl/jarl.yaml` to reach it from your management network.
| [docs/dnssec.md](docs/dnssec.md) | DNSSEC signing, going public with your own name servers, Porkbun |
| [docs/acme.md](docs/acme.md) | Let's Encrypt DNS-01 with certbot, acme.sh, lego and Traefik |
| [docs/stats.md](docs/stats.md) | Query statistics: enabling, what is collected, retention |
| [docs/blocking.md](docs/blocking.md) | Blocking ads, trackers and malware: blocklists, your own rules, views, memory |
| [docs/validation.md](docs/validation.md) | Every check Jarl makes |
| [docs/operations.md](docs/operations.md) | Backups, drift, failures, recovery, monitoring |
| [docs/development.md](docs/development.md) | Code layout, tests against real BIND, conventions |
@@ -166,9 +171,11 @@ review step.
## Status
Version 0.2.0 is in testing (0.2.0-rc.1); 0.1.0 is the current release.
0.2.0 adds DNSSEC signing, Porkbun support, two-factor login and view
colours. Jarl is feature-complete for one server's configuration.
Version 0.3.0 is the current release, and 0.4.0 is in testing
(0.4.0-rc.3): it adds blocking (blocklists and your own rules, like
Pi-hole, done by named), disabling zones, server statistics and view keys. 0.3.0 added disabled records, script-friendly
CLI output and client names in the statistics, on top of 0.2.0's DNSSEC
signing, Porkbun support, two-factor login and view colours. Jarl is feature-complete for one server's configuration.
Managing several servers from one jarld and dynamic updates are not
handled yet ([concepts.md](docs/concepts.md#what-jarl-deliberately-doesnt-do-yet)).
+1 -1
View File
@@ -1 +1 @@
0.2.0-rc.1
0.4.0-rc.3
+498
View File
@@ -0,0 +1,498 @@
package main
import (
"context"
"fmt"
"io"
"net/url"
"strings"
"time"
"github.com/spf13/cobra"
"jarl/internal/api"
"jarl/internal/blocking"
)
// ---- blocking (docs/blocking.md)
// viewsBody adds the --view flags to a request body: names, or "auto" for
// every view that offers recursion.
func viewsBody(body map[string]any, views []string) {
if len(views) == 1 && strings.EqualFold(views[0], "auto") {
body["views"] = []string{}
} else if len(views) > 0 {
body["views"] = views
}
}
func viewsText(views []string, auto bool) string {
switch {
case auto:
return "auto (" + strings.Join(views, ", ") + ")"
case len(views) == 0:
return "-"
}
return strings.Join(views, ", ")
}
func blocklistCmd() *cobra.Command {
cmd := &cobra.Command{Use: "blocklist", Aliases: []string{"blocklists"}, Short: "Blocklists: domains to block, downloaded and kept up to date",
Long: `Blocklists are downloaded by jarld and turned into response policy zones, so
named answers "doesn't exist" for the domains on them (docs/blocking.md).
Hosts, plain domain and Adblock (||domain^) lists work. Your own block and
allow rules ('jarl block') take precedence over every list.`}
var views []string
var comment string
list := &cobra.Command{
Use: "list",
Short: "List blocklists and their state",
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newClient()
if err != nil {
return err
}
var lists []api.Blocklist
if err := c.Get(ctx(), "/blocking/lists", nil, &lists); err != nil {
return err
}
return output(lists, func(w io.Writer) {
fmt.Fprintln(w, "NAME\tSTATE\tDOMAINS\tVIEWS\tUPDATED\tMEMORY\tURL")
for _, b := range lists {
domains, updated, mem := "-", "-", "-"
if s := b.Status; s != nil && s.UpdatedAt != nil {
domains, updated = fmt.Sprint(s.Entries), s.UpdatedAt.Local().Format("2006-01-02 15:04")
}
if b.MemoryMB > 0 {
mem = fmt.Sprintf("~%d MB", b.MemoryMB)
}
fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\t%s\t%s\n", b.Name, b.State, domains, viewsText(b.Views, b.AutoViews), updated, mem, b.URL)
}
for _, b := range lists {
if b.Status != nil && b.Status.Error != "" {
fmt.Fprintf(w, "\n%s: %s\n", b.Name, b.Status.Error)
}
}
})
},
}
add := &cobra.Command{
Use: "add NAME URL",
Short: "Add a blocklist (jarld downloads it right away)",
Long: `Add a blocklist. NAME is a short label (letters, digits, hyphens). Without
--view the list applies to every view that offers recursion.
jarl blocklist add hagezi-pro https://raw.githubusercontent.com/hagezi/dns-blocklists/main/adblock/pro.txt
jarl blocklist add stevenblack https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts --view internal`,
Args: cobra.ExactArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newClient()
if err != nil {
return err
}
body := map[string]any{"name": args[0], "url": args[1]}
viewsBody(body, views)
if comment != "" {
body["comment"] = comment
}
resp, err := c.Mutate(ctx(), "POST", "/blocking/lists", nil, body)
if err != nil {
return err
}
if err := report(resp); err != nil || flagQuiet || !resp.Applied {
return err
}
fmt.Println("jarld downloads the list in the background; 'jarl blocklist list' shows when it's in use.")
return nil
},
}
add.Flags().StringArrayVar(&views, "view", nil, "a view it applies to (repeat for several; default: every view that offers recursion)")
add.Flags().StringVar(&comment, "comment", "", "comment")
var rename, newURL string
update := &cobra.Command{
Use: "update LIST",
Short: "Change a blocklist's name, URL, views or comment",
Example: ` jarl blocklist update hagezi --url https://example.org/new-list.txt
jarl blocklist update hagezi --view internal --view guest
jarl blocklist update hagezi --view auto # every view that offers recursion`,
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newClient()
if err != nil {
return err
}
body := map[string]any{}
f := cmd.Flags()
if f.Changed("name") {
body["name"] = rename
}
if f.Changed("url") {
body["url"] = newURL
}
if f.Changed("comment") {
body["comment"] = comment
}
viewsBody(body, views)
if len(body) == 0 {
return fmt.Errorf("nothing to change (see --help)")
}
resp, err := c.Mutate(ctx(), "PATCH", "/blocking/lists/"+url.PathEscape(args[0]), nil, body)
if err != nil {
return err
}
return report(resp)
},
}
update.Flags().StringVar(&rename, "name", "", "new name")
update.Flags().StringVar(&newURL, "url", "", "new URL")
update.Flags().StringArrayVar(&views, "view", nil, `a view it applies to (repeat for several; "auto": every view that offers recursion)`)
update.Flags().StringVar(&comment, "comment", "", "new comment")
toggle := func(disable bool) *cobra.Command {
verb := map[bool]string{true: "disable", false: "enable"}[disable]
return &cobra.Command{
Use: verb + " LIST",
Short: map[bool]string{true: "Stop using a blocklist (it is kept, and re-enabled without downloading)", false: "Use a disabled blocklist again"}[disable],
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newClient()
if err != nil {
return err
}
resp, err := c.Mutate(ctx(), "PATCH", "/blocking/lists/"+url.PathEscape(args[0]), nil, map[string]any{"disabled": disable})
if err != nil {
return err
}
return report(resp)
},
}
}
var yes bool
del := &cobra.Command{
Use: "delete LIST",
Short: "Delete a blocklist",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
if !confirm("Delete blocklist "+args[0]+"?", yes) {
return fmt.Errorf("canceled")
}
c, err := newClient()
if err != nil {
return err
}
resp, err := c.Mutate(ctx(), "DELETE", "/blocking/lists/"+url.PathEscape(args[0]), nil, nil)
if err != nil {
return err
}
return report(resp)
},
}
del.Flags().BoolVarP(&yes, "yes", "y", false, "don't ask")
var force bool
refresh := &cobra.Command{
Use: "refresh [LIST...]",
Short: "Download blocklists now (all enabled ones without LIST)",
Long: `Download blocklists now instead of waiting for their next check
(blocking.refresh in jarl.yaml, daily by default). A list is only replaced
when its content changed; --force downloads it even when the server says
it hasn't.`,
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newClient()
if err != nil {
return err
}
names := args
if len(names) == 0 {
var lists []api.Blocklist
if err := c.Get(ctx(), "/blocking/lists", nil, &lists); err != nil {
return err
}
for _, b := range lists {
if b.State != "disabled" && b.State != "idle" {
names = append(names, b.Name)
}
}
}
if flagDryRun {
fmt.Printf("would download: %s\n", strings.Join(names, ", "))
return nil
}
failed := 0
for _, n := range names {
var b api.Blocklist
start := time.Now()
err := c.Do(ctx(), "POST", "/blocking/lists/"+url.PathEscape(n)+"/refresh", q("force", map[bool]string{true: "1"}[force]), nil, &b)
switch {
case err != nil:
failed++
fmt.Printf("%s: %v\n", n, err)
case b.Status != nil && b.Status.Error != "":
failed++
fmt.Printf("%s: %s (the previous version stays in use)\n", n, b.Status.Error)
case b.Status != nil:
fmt.Printf("%s: %d domains (%d lines skipped), checked in %s\n", n, b.Status.Entries, b.Status.Skipped, time.Since(start).Round(100*time.Millisecond))
}
}
if failed > 0 {
return &checkFailed{fmt.Sprintf("%d of %d lists could not be updated", failed, len(names))}
}
return nil
},
}
refresh.Flags().BoolVar(&force, "force", false, "download even when the server says the list hasn't changed")
cmd.AddCommand(list, add, update, toggle(true), toggle(false), del, refresh)
return cmd
}
func blockCmd() *cobra.Command {
cmd := &cobra.Command{Use: "block", Short: "Your own block and allow rules, and checking what is blocked",
Long: `Rules block a domain, or allow it despite the blocklists ('jarl blocklist').
A rule covers the domain and every name below it, unless --exact. Rules
take precedence over every list, and the most specific rule wins: allow
cdn.example.com overrides block example.com for cdn.example.com and the
names below it (docs/blocking.md).`}
var views []string
var comment, view string
var exact bool
addRule := func(action string) *cobra.Command {
c := &cobra.Command{
Use: map[string]string{"block": "add DOMAIN", "allow": "allow DOMAIN"}[action],
Short: map[string]string{"block": "Block a domain (and the names below it)", "allow": "Allow a domain (and the names below it) that a blocklist blocks"}[action],
Example: map[string]string{
"block": " jarl block add doubleclick.net\n jarl block add telemetry.example.com --exact --view internal",
"allow": " jarl block allow cdn.example.com\n jarl block allow login.example.com --exact",
}[action],
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newClient()
if err != nil {
return err
}
body := map[string]any{"name": args[0], "action": action}
if exact {
body["exact"] = true
}
viewsBody(body, views)
if comment != "" {
body["comment"] = comment
}
resp, err := c.Mutate(ctx(), "POST", "/blocking/rules", nil, body)
if err != nil {
return err
}
return report(resp)
},
}
c.Flags().BoolVar(&exact, "exact", false, "only this name, not the names below it")
c.Flags().StringArrayVar(&views, "view", nil, "a view it applies to (repeat for several; default: every view that offers recursion)")
c.Flags().StringVar(&comment, "comment", "", "comment")
return c
}
list := &cobra.Command{
Use: "list",
Short: "List your block and allow rules",
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newClient()
if err != nil {
return err
}
var rules []api.BlockRule
if err := c.Get(ctx(), "/blocking/rules", nil, &rules); err != nil {
return err
}
note := pausedNote(c)
return output(rules, func(w io.Writer) {
if note != "" {
fmt.Fprintln(w, note)
}
fmt.Fprintln(w, "ID\tACTION\tDOMAIN\tCOVERS\tVIEWS\tCOMMENT")
for _, r := range rules {
covers := "and below"
if r.Exact {
covers = "exact"
}
fmt.Fprintf(w, "%d\t%s\t%s\t%s\t%s\t%s\n", r.ID, r.Action, r.Name, covers, viewsText(r.Views, r.AutoViews), r.Comment)
}
})
},
}
var id int64
var missingOK bool
del := &cobra.Command{
Use: "delete DOMAIN | delete --id ID",
Short: "Delete the rules for a domain (or one rule by ID)",
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newClient()
if err != nil {
return err
}
var resp *api.MutationResponse
switch {
case id > 0:
resp, err = c.Mutate(ctx(), "DELETE", fmt.Sprintf("/blocking/rules/%d", id), nil, nil)
case len(args) == 1:
qv := q("name", args[0], "view", view)
if missingOK {
qv.Set("missing_ok", "1")
}
resp, err = c.Mutate(ctx(), "DELETE", "/blocking/rules", qv, nil)
default:
return fmt.Errorf("give DOMAIN, or --id")
}
if err != nil {
return err
}
if !resp.Applied && !resp.Staged && !resp.DryRun {
fmt.Println("nothing to do")
return nil
}
return report(resp)
},
}
del.Flags().Int64Var(&id, "id", 0, "delete the rule with this ID")
del.Flags().StringVar(&view, "view", "", "only the rules that apply in this view")
del.Flags().BoolVar(&missingOK, "missing-ok", false, "succeed when there is no such rule")
check := &cobra.Command{
Use: "check DOMAIN",
Short: "Show whether a name is blocked, and by which rule or list",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newClient()
if err != nil {
return err
}
var matches []blocking.Match
if err := c.Get(ctx(), "/blocking/check", q("name", args[0]), &matches); err != nil {
return err
}
note := pausedNote(c)
return output(matches, func(w io.Writer) {
if note != "" {
fmt.Fprint(w, note+"While it is, every name is answered. Otherwise:\n")
}
if len(matches) == 0 {
fmt.Fprintln(w, "Blocking isn't set up in any view.")
return
}
for _, m := range matches {
where := ""
if m.View != "" {
where = "view " + m.View + ":\t"
}
switch {
case m.Result == "none":
fmt.Fprintf(w, "%snot blocked\n", where)
case m.List != "":
fmt.Fprintf(w, "%sblocked by blocklist %s (entry %s)\n", where, m.List, m.Entry)
default:
fmt.Fprintf(w, "%s%s by your rule %d (entry %s)\n", where, m.Result, m.RuleID, m.Entry)
}
}
})
},
}
pause := &cobra.Command{
Use: "pause [DURATION | indefinitely]",
Short: "Let every name through for a while (default 10m, at most 24h) or until resumed",
Long: `Pause blocking: every name is answered normally until the time is up,
then blocking resumes by itself. Useful to see whether blocking is what
breaks a site. The lists stay loaded, so pausing and resuming take effect
at once. "indefinitely" pauses until 'jarl block resume'.
jarl block pause # 10 minutes
jarl block pause 1h
jarl block pause indefinitely
jarl block resume`,
Args: cobra.MaximumNArgs(1),
ValidArgs: []string{"indefinitely", "5m", "10m", "30m", "1h", "24h"},
RunE: func(cmd *cobra.Command, args []string) error {
if len(args) == 1 && args[0] == "indefinitely" {
c, err := newClient()
if err != nil {
return err
}
resp, err := c.Mutate(ctx(), "POST", "/blocking/pause", nil, map[string]any{"indefinitely": true})
if err != nil {
return err
}
if err := report(resp); err != nil || flagQuiet || !resp.Applied {
return err
}
fmt.Println("Blocking is paused until you resume it ('jarl block resume').")
return nil
}
d := 10 * time.Minute
if len(args) == 1 {
var err error
if d, err = time.ParseDuration(args[0]); err != nil || d < time.Minute {
return fmt.Errorf("invalid duration %q (like 10m or 2h, or indefinitely)", args[0])
}
}
c, err := newClient()
if err != nil {
return err
}
resp, err := c.Mutate(ctx(), "POST", "/blocking/pause", nil, map[string]any{"minutes": int(d.Round(time.Minute) / time.Minute)})
if err != nil {
return err
}
if err := report(resp); err != nil || flagQuiet || !resp.Applied {
return err
}
fmt.Printf("Blocking is paused until %s.\n", time.Now().Add(d.Round(time.Minute)).Format("15:04"))
return nil
},
}
resume := &cobra.Command{
Use: "resume",
Short: "End a pause: block again now",
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newClient()
if err != nil {
return err
}
resp, err := c.Mutate(ctx(), "POST", "/blocking/resume", nil, nil)
if err != nil {
return err
}
return report(resp)
},
}
cmd.AddCommand(addRule("block"), addRule("allow"), list, del, check, pause, resume)
return cmd
}
// pausedNote returns a line about a pause of blocking, or "".
func pausedNote(c interface {
Get(context.Context, string, url.Values, any) error
}) string {
var st struct {
PausedUntil *time.Time `json:"paused_until"`
PausedIndefinitely bool `json:"paused_indefinitely"`
}
if c.Get(ctx(), "/blocking/status", nil, &st) != nil {
return ""
}
if st.PausedIndefinitely {
return "Blocking is paused until you resume it ('jarl block resume').\n"
}
if st.PausedUntil == nil {
return ""
}
return "Blocking is paused until " + st.PausedUntil.Local().Format("2006-01-02 15:04") + " ('jarl block resume' ends it).\n"
}
+21 -9
View File
@@ -26,6 +26,10 @@ type batchEntry struct {
Delete map[string]any `yaml:"delete,omitempty" json:"delete,omitempty"`
Set map[string]any `yaml:"set,omitempty" json:"set,omitempty"`
Update map[string]any `yaml:"update,omitempty" json:"update,omitempty"`
// Disable and Enable find records like Delete: by id, or by zone,
// name, type and optionally data.
Disable map[string]any `yaml:"disable,omitempty" json:"disable,omitempty"`
Enable map[string]any `yaml:"enable,omitempty" json:"enable,omitempty"`
// Any API operation: op, id, data.
Op string `yaml:"op,omitempty" json:"op,omitempty"`
ID int64 `yaml:"id,omitempty" json:"id,omitempty"`
@@ -57,25 +61,23 @@ func (b *batchFile) requests() ([]api.ChangeRequest, error) {
var out []api.ChangeRequest
for i, e := range b.Changes {
n := 0
for _, set := range []bool{e.Add != nil, e.Delete != nil, e.Set != nil, e.Update != nil, e.Op != ""} {
for _, set := range []bool{e.Add != nil, e.Delete != nil, e.Set != nil, e.Update != nil, e.Disable != nil, e.Enable != nil, e.Op != ""} {
if set {
n++
}
}
if n != 1 {
return nil, fmt.Errorf("change %d: use exactly one of add, delete, set, update or op", i+1)
return nil, fmt.Errorf("change %d: use exactly one of add, delete, set, update, disable, enable or op", i+1)
}
switch {
case e.Add != nil:
out = append(out, api.ChangeRequest{Op: "record.create", Data: rawJSON(e.Add)})
case e.Delete != nil:
d := e.Delete
if id, ok := toInt64(d["id"]); ok {
delete(d, "id")
out = append(out, api.ChangeRequest{Op: "record.delete", ID: id, Data: rawJSON(d)})
} else {
out = append(out, api.ChangeRequest{Op: "record.delete", Data: rawJSON(d)})
}
out = append(out, byIDOrMatch("record.delete", e.Delete))
case e.Disable != nil:
out = append(out, byIDOrMatch("record.disable", e.Disable))
case e.Enable != nil:
out = append(out, byIDOrMatch("record.enable", e.Enable))
case e.Set != nil:
s := e.Set
switch d := s["data"].(type) {
@@ -104,6 +106,16 @@ func (b *batchFile) requests() ([]api.ChangeRequest, error) {
return out, nil
}
// byIDOrMatch is a request for the record with d's id, or for the records
// d matches.
func byIDOrMatch(op string, d map[string]any) api.ChangeRequest {
if id, ok := toInt64(d["id"]); ok {
delete(d, "id")
return api.ChangeRequest{Op: op, ID: id, Data: rawJSON(d)}
}
return api.ChangeRequest{Op: op, Data: rawJSON(d)}
}
// submitBatch applies requests as one changeset, or stages them into the
// changeset given with --changeset.
func submitBatch(c *client.Client, description string, reqs []api.ChangeRequest) error {
+11 -1
View File
@@ -24,7 +24,7 @@ const completeTimeout = 3 * time.Second
// setupCompletion adds value completion to the command tree.
func setupCompletion(root *cobra.Command) {
root.RegisterFlagCompletionFunc("server", completeServers(true))
root.RegisterFlagCompletionFunc("output", cobra.FixedCompletions([]string{"table", "json", "yaml"}, cobra.ShellCompDirectiveNoFileComp))
root.RegisterFlagCompletionFunc("output", cobra.FixedCompletions([]string{"table", "json", "yaml", "id", "field="}, cobra.ShellCompDirectiveNoFileComp|cobra.ShellCompDirectiveNoSpace))
var walk func(c *cobra.Command)
walk = func(c *cobra.Command) {
for _, name := range []string{"view", "user", "zone"} {
@@ -76,6 +76,8 @@ func argCompletion(c *cobra.Command) cobra.CompletionFunc {
return append([]string{"default"}, model.ViewColors...), cobra.ShellCompDirectiveNoFileComp
case p == "ACL":
return lookup(cmd, "acl")
case p == "LIST":
return lookup(cmd, "blocklist")
case p == "USER":
return lookup(cmd, "user")
case p == "KEY" && parent == "tsig":
@@ -199,6 +201,14 @@ func lookup(cmd *cobra.Command, kind string) ([]cobra.Completion, cobra.ShellCom
for _, v := range views {
addName(v.Name, v.Comment)
}
case "blocklist":
var lists []api.Blocklist
if c.Get(ctx(), "/blocking/lists", nil, &lists) != nil {
return nil, none
}
for _, b := range lists {
addName(b.Name, b.URL)
}
case "acl":
var acls []api.ACL
if c.Get(ctx(), "/acls", nil, &acls) != nil {
+39 -1
View File
@@ -201,7 +201,45 @@ func viewCmd() *cobra.Command {
return nil
},
}
cmd.AddCommand(list, create, update, del, order, color)
var keyNamed bool
key := &cobra.Command{
Use: "key VIEW",
Short: "Show the TSIG key that selects a view, to query it from anywhere",
Long: `Every view has a TSIG key (jarl-view-NAME) at the front of its
match-clients: a query signed with it lands in that view, whatever address
it comes from. Jarl signs its own checks of this server with it; use it to
see what a view answers:
jarl view key external # shows a ready dig -y command
jarl view key external --named-conf > external.key
dig -k external.key @SERVER adbraun.lol SOA
Showing it needs tsig:write and is recorded in the audit log.`,
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newClient()
if err != nil {
return err
}
var k struct {
Name string `json:"name"`
Algorithm string `json:"algorithm"`
Secret string `json:"secret"`
}
if err := c.Get(ctx(), "/views/"+url.PathEscape(args[0])+"/key", nil, &k); err != nil {
return err
}
if keyNamed {
fmt.Printf("key %q {\n\talgorithm %s;\n\tsecret %q;\n};\n", k.Name, k.Algorithm, k.Secret)
return nil
}
return output(k, func(w io.Writer) {
fmt.Fprintf(w, "Key:\t%s\nAlgorithm:\t%s\nSecret:\t%s\ndig:\tdig -y %s:%s:%s @SERVER NAME\n", k.Name, k.Algorithm, k.Secret, k.Algorithm, k.Name, k.Secret)
})
},
}
key.Flags().BoolVar(&keyNamed, "named-conf", false, "print a key statement (for dig -k FILE)")
cmd.AddCommand(list, create, update, del, order, color, key)
return cmd
}
+27 -3
View File
@@ -4,6 +4,7 @@ import (
"fmt"
"io"
"os"
"slices"
"strings"
"time"
@@ -110,7 +111,7 @@ A zone whose DS never reached the parent can be wiped at once instead
}
return nil
case st.Policy != "insecure":
if !confirm(fmt.Sprintf("Start turning DNSSEC off for %s? named keeps it signed until you've removed the DS record at your registrar.", displayName(st.Zone)), yes) {
if !confirm(fmt.Sprintf("Start turning DNSSEC off for %s? named keeps it signed until you've removed the DS record from the parent zone.", displayName(st.Zone)), yes) {
return fmt.Errorf("canceled")
}
next = "insecure"
@@ -133,7 +134,9 @@ A zone whose DS never reached the parent can be wiped at once instead
if err := report(resp); err != nil || !resp.Applied || flagQuiet {
return err
}
if next == "insecure" {
if next == "insecure" && st.ParentZone != "" {
fmt.Printf("\nNext: remove the DS record for %s from %s:\n %s\nthen run\n jarl zone dnssec check-ds %s\nRun 'jarl zone dnssec disable %s' again when 'status' says it has finished.\n", displayName(st.Zone), displayName(st.ParentZone), parentDSCommand(st, "delete", ""), args[0]+viewFlag(view), args[0]+viewFlag(view))
} else if next == "insecure" {
fmt.Printf("\nNext: remove the DS record for %s at your registrar, then run\n jarl zone dnssec check-ds %s\nRun 'jarl zone dnssec disable %s' again when 'status' says it has finished.\n", displayName(st.Zone), args[0]+viewFlag(view), args[0]+viewFlag(view))
} else {
fmt.Println("\nDNSSEC is off. The old keys stay in the key directory until you wipe them ('jarl zone dnssec wipe').")
@@ -344,7 +347,7 @@ func printDNSSEC(w io.Writer, r *api.DNSSECResponse) {
if len(ds) > 0 && r.Policy != "insecure" {
where := "at your registrar"
if r.ParentZone != "" {
where = "in the parent zone " + displayName(r.ParentZone) + " (in Jarl: add it as a DS record there)"
where = "in the parent zone " + displayName(r.ParentZone) + ", which is in Jarl: add it as a DS record there"
}
fmt.Fprintf(w, "\nDS record for the parent zone (%s):\n", where)
if !r.Phase.CanPublishDS && r.Phase.DSReadyAt != nil {
@@ -353,10 +356,31 @@ func printDNSSEC(w io.Writer, r *api.DNSSECResponse) {
for _, k := range ds {
fmt.Fprintf(w, " %s\n", strings.ReplaceAll(k.DS.Record, "\t", " "))
fmt.Fprintf(w, " key tag %d, algorithm %d (%s), digest type %d (SHA-256), digest %s\n", k.DS.KeyTag, k.DS.Algorithm, k.AlgorithmName, k.DS.DigestType, k.DS.Digest)
if r.ParentZone != "" {
data := fmt.Sprintf("%d %d %d %s", k.DS.KeyTag, k.DS.Algorithm, k.DS.DigestType, k.DS.Digest)
if slices.ContainsFunc(r.ParentDS, func(d string) bool { return strings.EqualFold(d, data) }) {
fmt.Fprintf(w, " %s has it.\n", displayName(r.ParentZone))
} else {
fmt.Fprintf(w, " %s\n", parentDSCommand(r, "add", data))
}
}
}
}
}
// parentDSCommand is the jarl command that adds (or deletes) the zone's DS
// record in its parent zone in Jarl.
func parentDSCommand(r *api.DNSSECResponse, verb, data string) string {
parts := []string{"jarl record", verb, displayName(r.ParentZone), strings.TrimSuffix(r.Zone, "."+r.ParentZone), "DS"}
if data != "" {
parts = append(parts, data)
}
if r.ParentView != "" {
parts = append(parts, "--view", r.ParentView)
}
return strings.Join(parts, " ")
}
func dash(s string) string {
if s == "" {
return "-"
+107 -5
View File
@@ -3,6 +3,7 @@
package main
import (
"bytes"
"context"
"encoding/json"
"errors"
@@ -11,6 +12,7 @@ import (
"net/url"
"os"
"path/filepath"
"slices"
"strconv"
"strings"
"sync"
@@ -237,10 +239,10 @@ Exit codes: 0 ok, 1 error, 3 not authenticated or not allowed, 4 not found,
pf.Int64VarP(&flagChangeset, "changeset", "C", 0, "stage changes into this changeset instead of applying them (env JARL_CHANGESET)")
pf.BoolVarP(&flagDryRun, "dry-run", "n", false, "validate and show the changes without applying them")
pf.BoolVar(&flagForce, "force", false, "overwrite BIND files that were changed outside Jarl")
pf.StringVarP(&flagOutput, "output", "o", "table", "output format: table, json or yaml")
pf.StringVarP(&flagOutput, "output", "o", "table", "output format: table, json, yaml, id (just the IDs) or field=NAME[,NAME...] (just those fields, e.g. field=view or field=soa.serial)")
pf.BoolVar(&flagInsecure, "insecure", false, "don't verify the server's TLS certificate")
pf.StringVar(&flagCACert, "ca-cert", "", "PEM file with a CA to trust for the server's certificate")
pf.BoolVarP(&flagQuiet, "quiet", "q", false, "only print errors (and IDs where useful)")
pf.BoolVarP(&flagQuiet, "quiet", "q", false, "only print errors and IDs (with show and list: just the IDs, like -o id)")
root.AddGroup(&cobra.Group{ID: "dns", Title: "DNS commands:"}, &cobra.Group{ID: "changes", Title: "Changes:"}, &cobra.Group{ID: "admin", Title: "Administration:"})
add := func(group string, cmds ...*cobra.Command) {
for _, c := range cmds {
@@ -248,7 +250,7 @@ Exit codes: 0 ok, 1 error, 3 not authenticated or not allowed, 4 not found,
root.AddCommand(c)
}
}
add("dns", zoneCmd(), recordCmd(), viewCmd(), aclCmd(), tsigCmd(), optionCmd(), settingsCmd(), acmeCmd(), searchCmd(), statsCmd())
add("dns", zoneCmd(), recordCmd(), viewCmd(), aclCmd(), tsigCmd(), optionCmd(), settingsCmd(), blockCmd(), blocklistCmd(), acmeCmd(), searchCmd(), statsCmd())
add("changes", applyCmd(), changesetCmd(), historyCmd())
add("admin", statusCmd(), whoamiCmd(), userCmd(), tokenCmd(), auditCmd(), checkCmd(), resyncCmd(), configCmd())
setupCompletion(root)
@@ -262,7 +264,15 @@ func ctx() context.Context { return context.Background() }
// ---- output
func output(v any, table func(w io.Writer)) error {
switch flagOutput {
format := flagOutput
quietIDs := flagQuiet && (format == "table" || format == "")
if quietIDs {
format = "id"
}
if fieldFormat(format) {
return outputFields(v, format, quietIDs)
}
switch format {
case "json":
enc := json.NewEncoder(os.Stdout)
enc.SetIndent("", " ")
@@ -286,7 +296,79 @@ func output(v any, table func(w io.Writer)) error {
table(tw)
return tw.Flush()
}
return fmt.Errorf("unknown output format %q (use table, json or yaml)", flagOutput)
return fmt.Errorf("unknown output format %q (use table, json, yaml, id or field=NAME)", flagOutput)
}
// outputFields prints fields of each object (or of the one object), one
// line per object with the fields separated by tabs, for scripts:
// "id" prints the IDs, "field=view,soa.serial" those fields. Names are the
// JSON field names (see -o json); a dot reaches into nested objects. A
// field an object doesn't have prints as empty (the API leaves out empty
// fields).
func outputFields(v any, format string, quiet bool) error {
fields := []string{"id"}
if f, ok := strings.CutPrefix(format, "field="); ok {
fields = strings.Split(f, ",")
}
b, err := json.Marshal(v)
if err != nil {
return err
}
dec := json.NewDecoder(bytes.NewReader(b))
dec.UseNumber()
var x any
if err := dec.Decode(&x); err != nil {
return err
}
items, ok := x.([]any)
if !ok {
items = []any{x}
}
if quiet && !slices.ContainsFunc(items, func(it any) bool { _, ok := lookupField(it, "id"); return ok }) {
return nil // -q on something without IDs: only errors
}
var out strings.Builder
for _, it := range items {
var vals []string
for _, f := range fields {
val, _ := lookupField(it, strings.TrimSpace(f))
vals = append(vals, fieldText(val))
}
out.WriteString(strings.Join(vals, "\t") + "\n")
}
_, err = io.WriteString(os.Stdout, out.String())
return err
}
// fieldFormat reports whether an -o format prints fields (id, field=...).
func fieldFormat(format string) bool {
return format == "id" || strings.HasPrefix(format, "field=")
}
func lookupField(v any, path string) (any, bool) {
for _, part := range strings.Split(path, ".") {
m, ok := v.(map[string]any)
if !ok {
return nil, false
}
if v, ok = m[part]; !ok {
return nil, false
}
}
return v, true
}
func fieldText(v any) string {
switch x := v.(type) {
case nil:
return ""
case string:
return x
case json.Number, bool:
return fmt.Sprint(x)
}
b, _ := json.Marshal(v)
return string(b)
}
func dot(name string) string {
@@ -391,6 +473,14 @@ func report(resp *api.MutationResponse) error {
if flagOutput == "json" || flagOutput == "yaml" {
return output(resp, nil)
}
if fieldFormat(flagOutput) {
// The changed object, if the response has it ("jarl record add
// ... -o id" prints the new record's ID).
if len(resp.Object) > 0 {
return outputFields(resp.Object, flagOutput, false)
}
return outputFields(resp, flagOutput, false)
}
w := os.Stdout
r := client.ParseResult(resp.Result)
switch {
@@ -440,6 +530,18 @@ func report(resp *api.MutationResponse) error {
json.Unmarshal(resp.Problems, &introduced)
}
printProblems(os.Stderr, introduced)
// named-checkconf's own warnings about the new configuration.
if r != nil && (resp.Applied || resp.DryRun) {
for _, m := range r.Check {
if !m.Error && !strings.HasPrefix(m.Text, "loaded serial") && m.Text != "OK" {
where := ""
if m.Object != "" {
where = " (" + m.Object + ")"
}
fmt.Fprintf(os.Stderr, "warning: named-checkconf: %s%s\n", m.Text, where)
}
}
}
for _, warn := range resp.Warnings {
fmt.Fprintln(os.Stderr, "warning:", warn)
}
+78
View File
@@ -5,6 +5,7 @@ import (
"io"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
@@ -22,6 +23,8 @@ changes:
- set: {zone: example.com, name: api, type: A, data: 192.0.2.5}
- set: {zone: example.com, name: gone, type: TXT}
- update: {id: 13, data: 192.0.2.9}
- disable: {zone: example.com, name: www, type: A, data: 192.0.2.1, ptr: true}
- enable: {id: 14}
- op: zone.create
data: {name: lab.example.com}
`
@@ -44,6 +47,8 @@ changes:
{"rrset.replace", 0, `{"data":["192.0.2.5"],"name":"api","type":"A","zone":"example.com"}`},
{"rrset.replace", 0, `{"data":[],"name":"gone","type":"TXT","zone":"example.com"}`},
{"record.update", 13, `{"data":"192.0.2.9"}`},
{"record.disable", 0, `{"data":"192.0.2.1","name":"www","ptr":true,"type":"A","zone":"example.com"}`},
{"record.enable", 14, `{}`},
{"zone.create", 0, `{"name":"lab.example.com"}`},
}
if len(reqs) != len(want) {
@@ -112,6 +117,38 @@ func TestListValue(t *testing.T) {
// filters they send: flags bound to variables shared between subcommands
// must not leak defaults (zone create's --type primary once hid every
// non-primary zone from 'zone list').
func TestDisableCommands(t *testing.T) {
var got string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
got = r.Method + " " + r.URL.Path + "?" + r.URL.RawQuery
w.Header().Set("Content-Type", "application/json")
w.Write([]byte(`{"applied":true,"changeset_id":1}`))
}))
defer srv.Close()
t.Setenv("JARL_CLI_CONFIG", filepath.Join(t.TempDir(), "cli.yaml"))
for _, tc := range []struct {
args []string
want string
}{
{[]string{"record", "disable", "example.com", "www", "A"}, "POST /api/v1/zones/example.com/records/disable?name=www&type=A"},
{[]string{"record", "disable", "example.com", "www", "A", "192.0.2.1", "--ptr", "--view", "internal"}, "POST /api/v1/zones/example.com/records/disable?data=192.0.2.1&name=www&ptr=1&type=A&view=internal"},
{[]string{"record", "enable", "--id", "12", "--ptr"}, "POST /api/v1/records/12/enable?ptr=1"},
{[]string{"record", "enable", "example.com", "old", "CNAME", "--missing-ok"}, "POST /api/v1/zones/example.com/records/enable?missing_ok=1&name=old&type=CNAME"},
} {
got = ""
root := rootCmd()
root.SetArgs(append(tc.args, "--server", srv.URL, "--token", "jarl_test_token", "-o", "json"))
root.SetOut(io.Discard)
if err := root.Execute(); err != nil {
t.Errorf("%v: %v", tc.args, err)
continue
}
if got != tc.want {
t.Errorf("%v sent %q, want %q", tc.args, got, tc.want)
}
}
}
func TestListQueries(t *testing.T) {
var got string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
@@ -154,3 +191,44 @@ func TestListQueries(t *testing.T) {
}
}
}
func TestOutputFields(t *testing.T) {
capture := func(f func() error) string {
t.Helper()
r, w, _ := os.Pipe()
old := os.Stdout
os.Stdout = w
err := f()
os.Stdout = old
w.Close()
b, _ := io.ReadAll(r)
if err != nil {
t.Fatal(err)
}
return string(b)
}
zones := []map[string]any{
{"id": 12, "name": "adbraun.lol.", "view": "external", "soa": map[string]any{"serial": 2026100901}},
{"id": 3, "name": "lab.adbraun.lol.", "soa": map[string]any{"serial": 7}},
}
defer func(o string, q bool) { flagOutput, flagQuiet = o, q }(flagOutput, flagQuiet)
for _, tc := range []struct {
format string
quiet bool
v any
want string
}{
{"id", false, zones, "12\n3\n"},
{"id", false, zones[0], "12\n"},
{"field=name,view,soa.serial", false, zones, "adbraun.lol.\texternal\t2026100901\nlab.adbraun.lol.\t\t7\n"},
{"field=soa", false, zones[1], "{\"serial\":7}\n"},
{"table", true, zones, "12\n3\n"},
{"table", true, map[string]any{"status": "ok"}, ""}, // -q without IDs: nothing
} {
flagOutput, flagQuiet = tc.format, tc.quiet
got := capture(func() error { return output(tc.v, func(io.Writer) { t.Error("table printed") }) })
if got != tc.want {
t.Errorf("-o %s (quiet %v) = %q, want %q", tc.format, tc.quiet, got, tc.want)
}
}
}
+101
View File
@@ -0,0 +1,101 @@
package main
import (
"fmt"
"os"
"strings"
"jarl/internal/api"
"jarl/internal/client"
)
// zoneViews expands a --view value for a record command: "" or one view
// as it is, "a,b" to those views, and "all" to every view that has the
// zone.
func zoneViews(c *client.Client, zone, view string) ([]string, error) {
if view != "all" && !strings.Contains(view, ",") {
return []string{view}, nil
}
if view != "all" {
var out []string
for _, v := range strings.Split(view, ",") {
if v = strings.TrimSpace(v); v != "" {
out = append(out, v)
}
}
return out, nil
}
var zones []api.Zone
if err := c.Get(ctx(), "/zones", nil, &zones); err != nil {
return nil, err
}
want := strings.ToLower(strings.TrimSuffix(zone, "."))
var out []string
for _, z := range zones {
if strings.ToLower(strings.TrimSuffix(z.Display, ".")) == want && z.View != "" {
out = append(out, z.View)
}
}
if len(out) == 0 {
return nil, fmt.Errorf("zone %s is in no view", zone)
}
return out, nil
}
// perView runs a record command once per view. With several views, the
// changes are staged into one changeset and applied together, so either
// every view gets the change or none does (with --changeset, they are
// staged there).
//
// do returns the server's response, or nil when there was nothing to do.
func perView(c *client.Client, views []string, description string, do func(c *client.Client, view string) (*api.MutationResponse, error)) error {
if len(views) == 1 || c.Changeset > 0 {
for _, v := range views {
resp, err := do(c, v)
if err != nil {
if len(views) > 1 {
err = fmt.Errorf("view %s: %w", v, err)
}
return err
}
if resp != nil {
if err := report(resp); err != nil {
return err
}
}
}
return nil
}
var cs api.Changeset
if err := c.Do(ctx(), "POST", "/changesets", nil, map[string]any{"description": description + " (views " + strings.Join(views, ", ") + ")"}, &cs); err != nil {
return err
}
discard := func() { c.Do(ctx(), "POST", fmt.Sprintf("/changesets/%d/discard", cs.ID), nil, nil, nil) }
staging := *c
staging.Changeset, staging.DryRun = cs.ID, false
staged := 0
for _, v := range views {
resp, err := do(&staging, v)
if err != nil {
discard()
fmt.Fprintf(os.Stderr, "view %s failed, so nothing was changed in any view\n", v)
return err
}
if resp != nil {
staged++
}
}
if staged == 0 {
discard()
return nil
}
var resp api.MutationResponse
err := c.Do(ctx(), "POST", fmt.Sprintf("/changesets/%d/apply", cs.ID), nil, map[string]any{"force": c.Force, "dry_run": c.DryRun}, &resp)
if err != nil || c.DryRun {
discard()
}
if err != nil {
return err
}
return report(&resp)
}
+109 -10
View File
@@ -4,6 +4,7 @@ import (
"fmt"
"io"
"math"
"slices"
"strconv"
"strings"
"time"
@@ -91,6 +92,9 @@ export the data.`,
if s.NXDomainRatio != nil {
fmt.Fprintf(w, "NXDOMAIN:\t%.1f%% of queries\n", *s.NXDomainRatio*100)
}
if s.BlockedRatio != nil {
fmt.Fprintf(w, "Blocked:\t%d (%.1f%% of queries): %s\n", s.Blocked, *s.BlockedRatio*100, entryLine(s.Blocklists, s.Blocked, 6))
}
fmt.Fprintf(w, "Result codes:\t%s\n", entryLine(s.RCodes, s.Queries, 6))
fmt.Fprintf(w, "Record types:\t%s\n", entryLine(s.QTypes, s.Queries, 8))
if len(s.Answers) > 0 {
@@ -102,14 +106,18 @@ export the data.`,
if len(s.Zones) > 0 {
fmt.Fprintf(w, "Zones:\t%s\n", strings.ReplaceAll(entryLine(s.Zones, 0, 10), stats.OtherZone+" ", "other "))
}
for _, k := range []string{stats.KindName, stats.KindClient, stats.KindNXDomain} {
for _, k := range []string{stats.KindName, stats.KindClient, stats.KindNXDomain, stats.KindBlocked} {
es := s.Tops[k]
if len(es) == 0 {
continue
}
fmt.Fprintf(w, "\n%s\t\n", map[string]string{stats.KindName: "Top names", stats.KindClient: "Top clients", stats.KindNXDomain: "Top NXDOMAIN names"}[k])
fmt.Fprintf(w, "\n%s\t\n", map[string]string{stats.KindName: "Top names", stats.KindClient: "Top clients", stats.KindNXDomain: "Top NXDOMAIN names", stats.KindBlocked: "Top blocked names"}[k])
for _, e := range es {
fmt.Fprintf(w, " %s\t%d\n", e.Key, e.Count)
key := e.Key
if e.Name != "" {
key += " (" + e.Name + ")"
}
fmt.Fprintf(w, " %s\t%d\n", key, e.Count)
}
}
})
@@ -120,12 +128,14 @@ export the data.`,
var limit int
top := &cobra.Command{
Use: "top [names|clients|nxdomain|servfail|refused]",
Short: "The most queried names, busiest clients, names with errors, refused clients",
Use: "top [names|clients|nxdomain|servfail|refused|blocked|blocked-clients]",
Short: "The most queried names, busiest clients, names with errors, refused clients, blocked names",
Example: ` jarl stats top names --since 7d
jarl stats top clients --limit 50
jarl stats top nxdomain # misconfigured hosts and typos
jarl stats top refused # who is asking for what they may not have`,
jarl stats top refused # who is asking for what they may not have
jarl stats top blocked # what blocking stops most (docs/blocking.md)
jarl stats top blocked-clients # the devices blocking stops most`,
Args: cobra.MaximumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
kind := stats.KindName
@@ -134,6 +144,9 @@ export the data.`,
if kind == "nxdomain" || kind == "servfail" {
kind = strings.ToLower(args[0])
}
if kind == "blocked-client" {
kind = stats.KindBlockedClient
}
}
c, err := newClient()
if err != nil {
@@ -148,13 +161,22 @@ export the data.`,
for _, e := range es {
total += e.Count
}
head := map[string]string{stats.KindClient: "CLIENT", stats.KindRefused: "CLIENT"}[kind]
head := map[string]string{stats.KindClient: "CLIENT", stats.KindRefused: "CLIENT", stats.KindBlockedClient: "CLIENT"}[kind]
if head == "" {
head = "NAME"
}
fmt.Fprintf(w, "#\t%s\tQUERIES\t\n", head)
named := slices.ContainsFunc(es, func(e stats.Entry) bool { return e.Name != "" })
if named {
fmt.Fprintf(w, "#\t%s\tHOST\tQUERIES\t\n", head)
} else {
fmt.Fprintf(w, "#\t%s\tQUERIES\t\n", head)
}
for i, e := range es {
fmt.Fprintf(w, "%d\t%s\t%d\t%s\n", i+1, e.Key, e.Count, bar(e.Count, es[0].Count, 20))
if named {
fmt.Fprintf(w, "%d\t%s\t%s\t%d\t%s\n", i+1, e.Key, dash(e.Name), e.Count, bar(e.Count, es[0].Count, 20))
} else {
fmt.Fprintf(w, "%d\t%s\t%d\t%s\n", i+1, e.Key, e.Count, bar(e.Count, es[0].Count, 20))
}
}
})
},
@@ -303,7 +325,61 @@ export the data.`,
}
sinceFlag(breakdown, &breakdownSince, "24h", sinceHelp)
cmd.AddCommand(status, summary, top, timeline, heatmap, breakdown)
server := &cobra.Command{
Use: "server",
Short: "Show named's memory, CPU, cache and clients, the host's memory and load, and free disk space",
Long: `Server statistics, sampled once a minute: named's memory and CPU use,
cache size, recursive and TCP clients against their limits, the host's
memory and load, and the space left where zone files, logs and Jarl's
data are. Memory and CPU over time: 'jarl stats timeline --key
g:named:rss,g:named:cpu' (averages; CPU in tenths of a percent).`,
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newClient()
if err != nil {
return err
}
var resp struct {
Server *stats.Server `json:"server"`
Named map[string]string `json:"named,omitempty"`
NamedError string `json:"named_error,omitempty"`
Disks []stats.Disk `json:"disks"`
}
if err := c.Get(ctx(), "/stats/server", nil, &resp); err != nil {
return err
}
return output(resp, func(w io.Writer) {
s := resp.Server
if s == nil {
fmt.Fprintln(w, "No sample yet:\tthe first is taken a minute after jarl starts")
return
}
fmt.Fprintf(w, "named:\t%s, up since %s, reconfigured %s\n", s.Version, isoLocal(s.BootTime), isoLocal(s.ConfigTime))
if s.PID > 0 {
fmt.Fprintf(w, "Memory:\t%s resident (named counts %s), %d threads, PID %d\n", sizeStr(s.RSS), sizeStr(s.MemInUse), s.Threads, s.PID)
fmt.Fprintf(w, "CPU:\t%.1f%% of one CPU over the last minute\n", s.CPUPercent)
} else {
fmt.Fprintf(w, "Memory:\tnamed counts %s (its process isn't visible to jarld)\n", sizeStr(s.MemInUse))
}
fmt.Fprintf(w, "Cache:\t%d RRsets, %s\n", s.CacheRRsets, sizeStr(s.CacheMem))
if n := resp.Named; n != nil {
fmt.Fprintf(w, "Recursive clients:\t%s (now/soft limit/limit), most at once %d\n", n["recursive_clients"], s.RecursionHighwater)
fmt.Fprintf(w, "TCP clients:\t%s (now/limit), most at once %d\n", n["tcp_clients"], s.TCPHighwater)
} else if resp.NamedError != "" {
fmt.Fprintf(w, "Clients:\tunknown: %s\n", resp.NamedError)
}
fmt.Fprintf(w, "Sockets:\t%d UDP, %d TCP\n", s.SocketsUDP, s.SocketsTCP)
if h := s.Host; h != nil && h.MemTotal > 0 {
fmt.Fprintf(w, "Host:\t%s of %s memory in use, load %v, %d CPUs, up %s\n", sizeStr(h.MemTotal-h.MemAvailable), sizeStr(h.MemTotal), h.Load, h.CPUs, (time.Duration(h.Uptime) * time.Second).String())
}
for _, d := range resp.Disks {
fmt.Fprintf(w, "Disk:\t%s free of %s for %s (%s)\n", sizeStr(d.Free), sizeStr(d.Total), strings.Join(d.Holds, ", "), d.Path)
}
})
},
}
cmd.AddCommand(status, summary, top, timeline, heatmap, breakdown, server)
return cmd
}
@@ -376,3 +452,26 @@ func sparkline(vals []int64) string {
}
return sb.String()
}
// sizeStr formats a size in binary units.
func sizeStr(n int64) string {
units := []string{"B", "KiB", "MiB", "GiB", "TiB"}
f, i := float64(n), 0
for f >= 1024 && i < len(units)-1 {
f /= 1024
i++
}
if i == 0 || f >= 100 {
return fmt.Sprintf("%.0f %s", f, units[i])
}
return fmt.Sprintf("%.1f %s", f, units[i])
}
// isoLocal shows a time named reports (RFC 3339) in local time.
func isoLocal(s string) string {
t, err := time.Parse(time.RFC3339Nano, s)
if err != nil {
return s
}
return t.Local().Format("2006-01-02 15:04")
}
+223 -30
View File
@@ -3,12 +3,14 @@ package main
import (
"encoding/json"
"fmt"
"golang.org/x/term"
"io"
"net/netip"
"net/url"
"os"
"strconv"
"strings"
"unicode/utf8"
"github.com/spf13/cobra"
@@ -43,7 +45,11 @@ func zoneCmd() *cobra.Command {
if z.Type == "primary" {
serial, recs = fmt.Sprint(z.SOA.Serial), fmt.Sprint(z.Records)
}
fmt.Fprintf(w, "%d\t%s\t%s\t%s\t%s\t%s\n", z.ID, z.Display, z.View, z.Type, recs, serial)
typ := z.Type
if z.Disabled {
typ += " (disabled)"
}
fmt.Fprintf(w, "%d\t%s\t%s\t%s\t%s\t%s\n", z.ID, z.Display, z.View, typ, recs, serial)
}
})
},
@@ -71,11 +77,18 @@ func zoneCmd() *cobra.Command {
fmt.Fprintf(w, "View:\t%s\n", z.View)
}
fmt.Fprintf(w, "Type:\t%s\n", z.Type)
if z.Disabled {
fmt.Fprintf(w, "Disabled:\tyes: named doesn't load it ('jarl zone enable %s')\n", args[0])
}
if z.Type == "primary" {
fmt.Fprintf(w, "Serial:\t%d\n", z.SOA.Serial)
fmt.Fprintf(w, "SOA:\t%s %s refresh %d retry %d expire %d minimum %d\n", z.SOA.MName, z.SOA.RName, z.SOA.Refresh, z.SOA.Retry, z.SOA.Expire, z.SOA.Minimum)
fmt.Fprintf(w, "Default TTL:\t%d\n", z.DefaultTTL)
fmt.Fprintf(w, "Records:\t%d\n", z.Records)
if z.DisabledRecords > 0 {
fmt.Fprintf(w, "Records:\t%d (%d disabled: jarl record list %s --disabled)\n", z.Records, z.DisabledRecords, args[0])
} else {
fmt.Fprintf(w, "Records:\t%d\n", z.Records)
}
}
if z.InView != "" {
fmt.Fprintf(w, "In view:\t%s\n", z.InView)
@@ -300,6 +313,42 @@ and a network between octet boundaries, like a /22, gets one zone per /24
del.Flags().StringVar(&view, "view", "", "view of the zone")
del.Flags().BoolVarP(&yes, "yes", "y", false, "don't ask for confirmation")
toggle := func(disable bool) *cobra.Command {
verb := map[bool]string{true: "disable", false: "enable"}[disable]
long := `Load a disabled zone in named again, as it is in Jarl now (changes made
while it was disabled included).`
if disable {
long = `Disable a zone: it stays in Jarl with its records, options and history,
and can still be edited, but named no longer loads it. Its names are then
answered by recursion where the view offers it (the internet's version of
the zone, if any), and REFUSED elsewhere. Signed zones can't be disabled;
turn signing off first.
jarl zone disable old.adbraun.lol
jarl zone enable old.adbraun.lol`
}
c := &cobra.Command{
Use: verb + " ZONE",
Short: map[bool]string{true: "Stop serving a zone, keeping it in Jarl", false: "Serve a disabled zone again"}[disable],
Long: long,
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newClient()
if err != nil {
return err
}
path, qv := zonePath(args[0], view)
resp, err := c.Mutate(ctx(), "PATCH", path, qv, map[string]any{"disabled": disable})
if err != nil {
return err
}
return report(resp)
},
}
c.Flags().StringVar(&view, "view", "", "view of the zone")
return c
}
fileCmd := &cobra.Command{
Use: "file ZONE",
Short: "Print the zone file Jarl generates",
@@ -367,7 +416,7 @@ equal to the file: records not in the file are deleted.`,
importCmd.Flags().BoolVar(&replace, "replace", false, "delete records that are not in the file")
importCmd.Flags().BoolVar(&useSOA, "use-soa", false, "also take SOA timers and $TTL from the file")
cmd.AddCommand(list, show, create, update, del, fileCmd, importCmd, zoneSerialsCmd(), zoneDelegationCmd(), zoneDNSSECCmd(), zoneRegistrarCmd())
cmd.AddCommand(list, show, create, update, del, toggle(true), toggle(false), fileCmd, importCmd, zoneSerialsCmd(), zoneDelegationCmd(), zoneDNSSECCmd(), zoneRegistrarCmd())
return cmd
}
@@ -400,10 +449,14 @@ func jsonUnmarshal(b []byte, v any) error { return json.Unmarshal(b, v) }
// ---- records
// viewsFlagHelp describes --view of the record commands that change
// records.
const viewsFlagHelp = "view of the zone; several (internal,external) or \"all\" make the same change in each, together"
func recordCmd() *cobra.Command {
cmd := &cobra.Command{Use: "record", Aliases: []string{"records", "rr"}, Short: "Manage records in primary zones"}
var view, name, typ, search, comment, ttlArg string
var ptr, missingOK, yes bool
var ptr, missingOK, yes, onlyDisabled, wide bool
var id int64
list := &cobra.Command{
@@ -419,18 +472,31 @@ func recordCmd() *cobra.Command {
for k, v := range q("name", name, "type", strings.ToUpper(typ), "q", search) {
qv[k] = v
}
if onlyDisabled {
qv.Set("disabled", "1")
}
var recs []api.Record
if err := c.Get(ctx(), path+"/records", qv, &recs); err != nil {
return err
}
return output(recs, func(w io.Writer) {
fmt.Fprintln(w, "ID\tNAME\tTTL\tTYPE\tDATA\tCOMMENT")
rows := [][]string{{"ID", "NAME", "TTL", "TYPE", "DATA", "COMMENT"}}
for _, r := range recs {
ttl := fmt.Sprint(r.EffectiveTTL)
if r.TTL == nil {
ttl += "*"
}
fmt.Fprintf(w, "%d\t%s\t%s\t%s\t%s\t%s\n", r.ID, r.RelativeName, ttl, r.Type, r.Data, r.Comment)
comment := r.Comment
if r.Disabled {
comment = strings.TrimSpace("[disabled] " + comment)
}
rows = append(rows, []string{fmt.Sprint(r.ID), r.RelativeName, ttl, r.Type, r.Data, comment})
}
if !wide {
fitColumn(rows, 4)
}
for _, row := range rows {
fmt.Fprintln(w, strings.Join(row, "\t"))
}
})
},
@@ -439,6 +505,8 @@ func recordCmd() *cobra.Command {
list.Flags().StringVar(&name, "name", "", "only this name")
list.Flags().StringVarP(&typ, "type", "t", "", "only this type")
list.Flags().StringVar(&search, "search", "", "only records whose name, data or comment contains this")
list.Flags().BoolVar(&onlyDisabled, "disabled", false, "only disabled records")
list.Flags().BoolVarP(&wide, "wide", "w", false, "show long data in full (on a terminal it is shortened to fit)")
add := &cobra.Command{
Use: "add ZONE NAME TYPE DATA...",
@@ -467,15 +535,17 @@ and splitting are automatic).
if ptr {
body["ptr"] = true
}
path, qv := zonePath(args[0], view)
resp, err := c.Mutate(ctx(), "POST", path+"/records", qv, body)
views, err := zoneViews(c, args[0], view)
if err != nil {
return err
}
return report(resp)
return perView(c, views, "add "+args[1]+" "+strings.ToUpper(args[2]), func(c *client.Client, view string) (*api.MutationResponse, error) {
path, qv := zonePath(args[0], view)
return c.Mutate(ctx(), "POST", path+"/records", qv, body)
})
},
}
add.Flags().StringVar(&view, "view", "", "view of the zone")
add.Flags().StringVar(&view, "view", "", viewsFlagHelp)
add.Flags().StringVar(&ttlArg, "ttl", "", "TTL in seconds (default: the RRset's or the zone's)")
add.Flags().StringVar(&comment, "comment", "", "comment")
add.Flags().BoolVar(&ptr, "ptr", false, "also add the PTR record (A/AAAA)")
@@ -560,26 +630,28 @@ or one record by ID.
if len(args) == 3 && !confirm(fmt.Sprintf("Delete all %s records of %s?", strings.ToUpper(args[2]), args[1]), yes) {
return fmt.Errorf("canceled")
}
path, qv := zonePath(args[0], view)
qv.Set("name", args[1])
qv.Set("type", args[2])
if len(args) > 3 {
qv.Set("data", strings.Join(args[3:], " "))
}
if missingOK {
qv.Set("missing_ok", "1")
}
if ptr {
qv.Set("ptr", "1")
}
resp, err := c.Mutate(ctx(), "DELETE", path+"/records", qv, nil)
views, err := zoneViews(c, args[0], view)
if err != nil {
return err
}
return report(resp)
return perView(c, views, "delete "+args[1]+" "+strings.ToUpper(args[2]), func(c *client.Client, view string) (*api.MutationResponse, error) {
path, qv := zonePath(args[0], view)
qv.Set("name", args[1])
qv.Set("type", args[2])
if len(args) > 3 {
qv.Set("data", strings.Join(args[3:], " "))
}
if missingOK {
qv.Set("missing_ok", "1")
}
if ptr {
qv.Set("ptr", "1")
}
return c.Mutate(ctx(), "DELETE", path+"/records", qv, nil)
})
},
}
del.Flags().StringVar(&view, "view", "", "view of the zone")
del.Flags().StringVar(&view, "view", "", viewsFlagHelp)
del.Flags().Int64Var(&id, "id", 0, "delete the record with this ID")
del.Flags().BoolVar(&missingOK, "missing-ok", false, "succeed when nothing matches")
del.Flags().BoolVar(&ptr, "ptr", false, "also delete the matching PTR record (A/AAAA)")
@@ -606,19 +678,104 @@ spaces.
if comment != "" {
body["comment"] = comment
}
path, qv := zonePath(args[0], view)
resp, err := c.Mutate(ctx(), "PUT", path+"/rrsets/"+urlPathEscape(args[1])+"/"+urlPathEscape(args[2]), qv, body)
views, err := zoneViews(c, args[0], view)
if err != nil {
return err
}
return report(resp)
return perView(c, views, "set "+args[1]+" "+strings.ToUpper(args[2]), func(c *client.Client, view string) (*api.MutationResponse, error) {
path, qv := zonePath(args[0], view)
return c.Mutate(ctx(), "PUT", path+"/rrsets/"+urlPathEscape(args[1])+"/"+urlPathEscape(args[2]), qv, body)
})
},
}
set.Flags().StringVar(&view, "view", "", "view of the zone")
set.Flags().StringVar(&view, "view", "", viewsFlagHelp)
set.Flags().StringVar(&ttlArg, "ttl", "", "TTL for the RRset")
set.Flags().StringVar(&comment, "comment", "", "comment for new records")
cmd.AddCommand(list, add, update, del, set)
toggle := func(disable bool) *cobra.Command {
verb := "enable"
long := `Enable disabled records again: by name and type (all of them, or only the
one with DATA), or one record by ID. The record is checked as if it were
added, so a CNAME added in the meantime at its name stops it.
jarl record enable example.com www A 192.0.2.10
jarl record enable --id 1234 --ptr`
if disable {
verb = "disable"
long = `Disable records: they stay in Jarl, with their TTL, comment and ID, but are
left out of the zone (the zone file lists them as comments) until they are
enabled again. Select them by name and type (all of them, or only the one
with DATA), or one record by ID.
jarl record disable example.com www A 192.0.2.10 --ptr # and its PTR record
jarl record disable example.com old CNAME
jarl record disable --id 1234
jarl record list example.com --disabled`
}
var tview string
var tid int64
var tptr, tmissing bool
c := &cobra.Command{
Use: verb + " ZONE NAME TYPE [DATA...] | " + verb + " --id ID",
Short: strings.ToUpper(verb[:1]) + verb[1:] + " records",
Long: long,
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newClient()
if err != nil {
return err
}
qv := q()
if tptr {
qv.Set("ptr", "1")
}
if tid > 0 {
resp, err := c.Mutate(ctx(), "POST", "/records/"+strconv.FormatInt(tid, 10)+"/"+verb, qv, nil)
if err != nil {
return err
}
return report(resp)
}
if len(args) < 3 {
return fmt.Errorf("give ZONE NAME TYPE [DATA], or --id")
}
views, err := zoneViews(c, args[0], tview)
if err != nil {
return err
}
return perView(c, views, verb+" "+args[1]+" "+strings.ToUpper(args[2]), func(c *client.Client, view string) (*api.MutationResponse, error) {
path, zq := zonePath(args[0], view)
vq := q()
for k, v := range qv {
vq[k] = v
}
for k, v := range zq {
vq[k] = v
}
vq.Set("name", args[1])
vq.Set("type", args[2])
if len(args) > 3 {
vq.Set("data", strings.Join(args[3:], " "))
}
if tmissing {
vq.Set("missing_ok", "1")
}
resp, err := c.Mutate(ctx(), "POST", path+"/records/"+verb, vq, nil)
if err != nil && tmissing && strings.HasSuffix(err.Error(), "nothing to do") {
fmt.Println("nothing to do")
return nil, nil
}
return resp, err
})
},
}
c.Flags().StringVar(&tview, "view", "", viewsFlagHelp)
c.Flags().Int64Var(&tid, "id", 0, verb+" the record with this ID")
c.Flags().BoolVar(&tmissing, "missing-ok", false, "succeed when nothing matches or the records are already "+verb+"d")
c.Flags().BoolVar(&tptr, "ptr", false, "also "+verb+" the matching PTR record (A/AAAA)")
return c
}
cmd.AddCommand(list, add, update, del, set, toggle(true), toggle(false))
return cmd
}
@@ -636,3 +793,39 @@ func setTTL(body map[string]any, ttl string) error {
}
return nil
}
// fitColumn shortens column col of a table (header row first) so that the
// rows fit the terminal: the table pads every row to its longest cell, so
// one long TXT record would make every row wrap. Nothing is shortened when
// standard output isn't a terminal.
func fitColumn(rows [][]string, col int) {
fd := int(os.Stdout.Fd())
if !term.IsTerminal(fd) {
return
}
width, _, err := term.GetSize(fd)
if err != nil || width <= 0 {
return
}
widths := make([]int, len(rows[0]))
for _, row := range rows {
for i, c := range row {
widths[i] = max(widths[i], utf8.RuneCountInString(c))
}
}
used := 0
for i, w := range widths {
if i != col {
used += w + 2 // the table's padding
}
}
if col == len(widths)-1 {
used -= 2
}
limit := max(width-used-2, 24)
for _, row := range rows[1:] {
if r := []rune(row[col]); len(r) > limit {
row[col] = string(r[:limit-1]) + "…"
}
}
}
+5 -1
View File
@@ -302,7 +302,11 @@ func renderCmd() *cobra.Command {
return err
}
paths := render.Paths{ConfigDir: filepath.Join(out, "config"), ZoneDir: filepath.Join(out, "zones"), CacheDir: filepath.Join(out, "cache")}
o := render.Render(st, paths, renderExtras(a.cfg))
x := renderExtras(a.cfg)
if x.ViewKeySeed, err = engine.ViewKeySeed(context.Background(), a.store); err != nil {
return err
}
o := render.Render(st, paths, x)
for _, f := range o.Files {
if err := os.MkdirAll(filepath.Dir(f.Path), 0o750); err != nil {
return err
+34 -2
View File
@@ -123,6 +123,10 @@ func openApp(allowMissingConfig bool) (*app, error) {
NoReload: cfg.BIND.NoReload,
Extras: renderExtras(cfg),
}, log)
eng.Limits = engine.BlocklistLimits{
Refresh: time.Duration(cfg.Blocking.Refresh), RetryAfter: time.Hour, Timeout: 2 * time.Minute,
MaxBytes: int64(cfg.Blocking.MaxDownloadMB) << 20, MaxEntries: cfg.Blocking.MaxEntries,
}
return &app{cfg: cfg, log: log, store: st, eng: eng, owner: owner}, nil
}
@@ -144,7 +148,7 @@ func statsOptions(cfg *config.Config) stats.Options {
sc := cfg.Stats
o := stats.Options{QueryLog: sc.QueryLog, LogFile: sc.LogFile, StatisticsChannel: sc.StatisticsChannel, TopN: sc.TopN,
RetainMinute: time.Duration(sc.Retention.Minute), RetainHour: time.Duration(sc.Retention.Hour), RetainDay: time.Duration(sc.Retention.Day),
Anonymize: sc.AnonymizeClients}
Anonymize: sc.AnonymizeClients, MainConfig: cfg.BIND.MainConfig}
for _, p := range sc.ExcludeClients {
if pf, err := config.ParsePrefix(p); err == nil {
o.Exclude = append(o.Exclude, pf.Masked())
@@ -163,7 +167,7 @@ func zoneIndex(a *app) stats.ZoneIndex {
}
out := map[string][]string{}
for _, z := range st.Zones {
if z.Type != model.ZonePrimary && z.Type != model.ZoneSecondary && z.Type != model.ZoneMirror {
if z.Type != model.ZonePrimary && z.Type != model.ZoneSecondary && z.Type != model.ZoneMirror || z.Disabled {
continue
}
v := st.ViewName(z)
@@ -323,6 +327,7 @@ func serveCmd() *cobra.Command {
}
}()
go auditRetention(ctx, a)
go blocklistLoop(ctx, a)
a.log.Info("jarld starting", "version", version.String(), "database", a.cfg.Database, "bind_config", a.cfg.BIND.ConfigDir)
err = srv.Serve(ctx, func() { sdNotify("READY=1\nSTATUS=serving") })
sdNotify("STOPPING=1")
@@ -332,6 +337,33 @@ func serveCmd() *cobra.Command {
}
}
// blocklistLoop keeps blocklists up to date (docs/blocking.md): every few
// minutes it downloads the lists that are due, and at once when a list
// was added or changed. It also ends a pause of blocking on time.
func blocklistLoop(ctx context.Context, a *app) {
select {
case <-ctx.Done():
return
case <-time.After(10 * time.Second): // let jarld finish starting
}
for {
a.eng.RefreshDue(ctx)
wait := 5 * time.Minute
if until := a.eng.ResumeBlocking(ctx); !until.IsZero() {
wait = min(wait, max(time.Until(until), time.Second))
}
t := time.NewTimer(wait)
select {
case <-ctx.Done():
t.Stop()
return
case <-t.C:
case <-a.eng.BlocklistKicks():
t.Stop()
}
}
}
func auditRetention(ctx context.Context, a *app) {
if a.cfg.Audit.RetentionDays <= 0 {
return
+7 -3
View File
@@ -40,8 +40,10 @@ ProtectHome=yes
PrivateTmp=yes
PrivateDevices=yes
NoNewPrivileges=yes
CapabilityBoundingSet=
AmbientCapabilities=
# The only privilege jarld gets: listening on ports below 1024 (443 for the
# web UI). Nothing else, and NoNewPrivileges still applies.
CapabilityBoundingSet=CAP_NET_BIND_SERVICE
AmbientCapabilities=CAP_NET_BIND_SERVICE
ProtectKernelTunables=yes
ProtectKernelModules=yes
ProtectKernelLogs=yes
@@ -53,7 +55,9 @@ RestrictRealtime=yes
RestrictSUIDSGID=yes
LockPersonality=yes
MemoryDenyWriteExecute=yes
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
# AF_NETLINK lets jarld list this host's addresses, so its name server
# checks know which addresses are its own.
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6 AF_NETLINK
SystemCallArchitectures=native
# @system-service includes @chown, which jarld needs to give files to group bind.
SystemCallFilter=@system-service
+72 -10
View File
@@ -110,7 +110,8 @@ on the resources below, except users and tokens) works the same way:
Reads accept `?changeset=ID` too. They then show the configuration as it
will be after the changeset, with `"pending": "add" | "change" | "delete"`
on affected zones and records.
on affected zones and records (`"disable"` or `"enable"` for a record
that is only being disabled or enabled).
The response of an applied change:
@@ -159,7 +160,7 @@ may be the name.
| GET | `/zones` | `?view=`, `?type=`, `?q=` (name contains) |
| POST | `/zones` | ZoneInput (below) |
| GET | `/zones/{zone}` | |
| PATCH | `/zones/{zone}` | ZoneInput (only the fields to change) |
| PATCH | `/zones/{zone}` | ZoneInput (only the fields to change); `{"disabled": true}` disables the zone: named stops loading it, Jarl keeps it |
| DELETE | `/zones/{zone}` | deletes the zone with its records and options |
| GET | `/zones/{zone}/file` | the generated zone file (text/plain) |
| GET | `/zones/{zone}/serials` | asks the zone's other name servers for their serial (below) |
@@ -220,6 +221,18 @@ when called ([operations.md](operations.md#checking-name-servers-and-the-delegat
`error` (with `error`). `source` is `NS record`, `also-notify` or
`primaries`; `port` is set for a non-default port.
`/serials?stream=1` and `/delegation?stream=1` answer with
newline-delimited JSON (`application/x-ndjson`) while they work, which is
how the web UI shows a check's progress. Each DNS query is an event when it
is sent and another when it is answered: `{"kind": "start", "id": 2,
"query": "dig @192.0.2.1 example.org. SOA +norec"}`, then `{"kind": "end",
"id": 2, "answer": "serial 2026100900", "rtt_ms": 12}`. `answer` sums the
answer up (addresses, `referral to …`, `NXDOMAIN`, `no such record`);
`error` replaces it when there was none. The serials check's SOA queries
end with `result`, the `servers` entry, instead. ID 0 is `rndc zonestatus`,
which asks this server's named. The last line is `{"kind": "done",
"response": {...}}`, the endpoint's usual response.
`/delegation` returns `parent`, `resolver`, `delegated`, `parent_ns`,
`jarl_ns`, `parent_servers` (each parent server's `status`: `delegated`,
`not_delegated`, `unreachable`, `error`, with `ns` and `glue`), `glue`
@@ -321,14 +334,18 @@ registrar_error`.
| Method | Path | Body / query |
|--------|------|--------------|
| GET | `/zones/{zone}/records` | `?name=`, `?type=`, `?q=` |
| GET | `/zones/{zone}/records` | `?name=`, `?type=`, `?q=`, `?disabled=1` (only disabled records) or `0` (only enabled ones) |
| POST | `/zones/{zone}/records` | a RecordInput, or an array of them (one changeset) |
| DELETE | `/zones/{zone}/records` | `?name=&type=[&data=][&missing_ok=1][&ptr=1]`: delete by match |
| POST | `/zones/{zone}/records/disable` | same query as delete by match: disable the matching records |
| POST | `/zones/{zone}/records/enable` | same query: enable them again |
| PUT | `/zones/{zone}/rrsets/{name}/{type}` | `{"data": ["192.0.2.1", "192.0.2.2"], "ttl": 60, "comment": "..."}`: make the RRset exactly this |
| DELETE | `/zones/{zone}/rrsets/{name}/{type}` | delete the whole RRset |
| GET | `/records/{id}` | |
| PATCH | `/records/{id}` | RecordInput fields to change (`If-Match` supported) |
| DELETE | `/records/{id}` | `?ptr=1` also deletes the matching PTR |
| POST | `/records/{id}/disable` | `?ptr=1` also disables the matching PTR |
| POST | `/records/{id}/enable` | `?ptr=1` also enables the matching PTR |
RecordInput:
@@ -339,12 +356,26 @@ RecordInput:
"data": "192.0.2.10", // zone file syntax; TXT/SPF take plain text
"ttl": 300, // null or absent: the zone's default TTL
"comment": "web server",
"ptr": true // A/AAAA: also create/move/delete the PTR in the reverse zone
"disabled": false, // true: keep the record in Jarl but out of the zone
"ptr": true // A/AAAA: also create/move/delete (or disable/enable) the PTR in the reverse zone
}
```
Responses add `zone`, `view`, `relative_name`, `effective_ttl`, and `text`
(the decoded value of TXT records).
(the decoded value of TXT records). Zones have `disabled_count` when
some of their records are disabled.
**Disabled records** stay in Jarl, with their ID, TTL and comment, but are
left out of the zone: the zone file lists them as comments (`; disabled:
www IN A 192.0.2.10`). Disabling and enabling are ordinary changes (staged,
reviewed, reverted like any other). Validation checks the zone as it is
served, so disabling the zone's last NS record is refused, and enabling a
record is checked as if it were added: a CNAME added at its name in the
meantime stops it (and gives a `disabled_conflict` warning until then).
`PUT /rrsets` makes the RRset exactly the given data: a disabled record
with one of the values is enabled, other disabled records are left alone.
Disabling a record that is already disabled is a 409; with `missing_ok=1`
(by match) it's "nothing to do".
### Views
@@ -355,6 +386,7 @@ Responses add `zone`, `view`, `relative_name`, `effective_ttl`, and `text`
| GET / PATCH / DELETE | `/views/{id}` | DELETE `?cascade=1` also deletes the view's zones |
| POST | `/views/order` | `{"ids": [3, 1, 2]}` |
| PUT | `/views/{id}/color` | `{"color": "green"}`; `""` for the default |
| GET | `/views/{id}/key` | `{name, algorithm, secret}`: the TSIG key that selects the view (docs/views.md); needs `tsig:write`, audited |
`adopt_zones` moves zones that are not in a view into the new view. named
requires every zone to be in a view as soon as one view exists.
@@ -380,6 +412,29 @@ send the request with that changeset (`?changeset=` or
| GET | `/tsig-keys/{id}` | `?secret=1` includes the secret (needs `tsig:write`, audited) |
| PATCH / DELETE | `/tsig-keys/{id}` | PATCH `{"secret": "..."}` rotates |
### Blocking
Needs `blocking:read` (`blocking:write` to change). See
[blocking.md](blocking.md).
| Method | Path | Body / query |
|--------|------|--------------|
| GET | `/blocking/lists` | each with `views` (where it applies), `auto_views` (not chosen: every view that offers recursion), `state` (ok, waiting, error, disabled, idle), `status` (`entries`, `records`, `skipped`, `checked_at`, `updated_at`, `error`) and `memory_mb` (named's memory for it, estimated) |
| POST | `/blocking/lists` | `{"name": "hagezi-multi", "url": "https://...", "views": ["internal"], "comment": ""}` (`views` empty or absent: automatic); jarld downloads it right after |
| GET / PATCH / DELETE | `/blocking/lists/{id or name}` | PATCH any of `name`, `url`, `views`, `disabled`, `comment` |
| POST | `/blocking/lists/{id or name}/refresh` | download now and wait; `?force=1` even when the server says nothing changed. Returns the list; a failed update is in `state` and `status.error` |
| GET | `/blocking/rules` | each with `views` and `auto_views` |
| POST | `/blocking/rules` | `{"name": "doubleclick.net", "action": "block", "exact": false, "views": [], "comment": ""}` (`action` block or allow; `exact`: only the name, not the names below it) |
| PATCH / DELETE | `/blocking/rules/{id}` | |
| DELETE | `/blocking/rules` | `?name=DOMAIN[&view=V][&missing_ok=1]`: the rules for a domain |
| GET | `/blocking/check` | `?name=DOMAIN`: per view with blocking, `{view, result: blocked\|allowed\|none, entry, rule_id, list}` (what applies when blocking isn't paused) |
| GET | `/blocking/status` | `{paused_until, paused_indefinitely}` (null and false when not paused) |
| POST | `/blocking/pause` | `{"minutes": 15}` (1 to 1440): let every name through until then; jarld resumes by itself. `{"indefinitely": true}`: until resumed |
| POST | `/blocking/resume` | end a pause now |
Lists and rules are configuration like the rest: they can be staged, and
listing with `?changeset=` marks them `pending`.
### Options and settings
| Method | Path | Body / query |
@@ -390,7 +445,9 @@ send the request with that changeset (`?changeset=` or
| GET / PATCH | `/settings` | `{"nameservers": [...], "rname": "...", "default_ttl": 3600, "refresh": ..., "retry": ..., "expire": ..., "negative_ttl": ..., "serial_policy": "date"}` |
Option values are named.conf syntax without the final semicolon. They are
checked against the BIND 9.20 grammar for that scope.
checked against the BIND 9.20 grammar for that scope. `position` only
orders top-level statements; the options of a block are always listed and
written in Jarl's fixed order (docs/concepts.md).
### Catalogs and search
@@ -424,12 +481,16 @@ An **Operation** is `{"op": "...", "id": 0, "base_version": 0, "data": {...}}`:
| `record.create` | | RecordInput + `"zone": "adbraun.lol"` (or `zone_id`) and `"view"` |
| `record.update` | record | RecordInput |
| `record.delete` | record, or 0 with `zone`, `name`, `type`, optional `data`, `missing_ok`, `ptr` | |
| `record.disable` / `record.enable` | record, or 0 with the same match fields as `record.delete` | stored as `record.update` operations |
| `rrset.replace` | | `{"zone", "view", "name", "type", "ttl", "data": [...]}` (empty data deletes the RRset) |
| `view.create` / `view.update` / `view.delete` / `view.reorder` | view (update/delete) | ViewInput / `{"ids": [...]}` |
| `acl.create` / `acl.update` / `acl.delete` | ACL | ACL fields |
| `tsig.create` / `tsig.update` / `tsig.delete` | key | key fields |
| `option.create` / `option.update` / `option.delete` | option | option fields |
| `settings.update` | | settings fields |
| `blocklist.create` / `blocklist.update` / `blocklist.delete` | blocklist (update/delete) | list fields; views as `views` (names) or `view_ids` |
| `blockrule.create` / `blockrule.update` / `blockrule.delete` | rule (update/delete) | rule fields, likewise |
| `blocking.pause` | | `{"until": "2026-10-10T15:04:05Z"}`, `{"indefinitely": true}`, or `{}` to resume |
Operations later in a batch can refer to objects created earlier in it by
name (zone and view names). One request can therefore create a view, a zone
@@ -439,7 +500,7 @@ in it and its records.
| Method | Path | Notes |
|--------|------|-------|
| GET | `/status` | named status, adoption, drift, open changesets, current problems, object counts; `expiring_tokens` (tokens that expire within 14 days or expired in the last 14: all for administrators, otherwise the caller's); `backup` (administrators: `last_at`, `last_file`, `last_size`, `last_error`, `last_error_at`, `dir`, `warn_after` in seconds, `stale`) |
| GET | `/status` | named status, adoption, drift, open changesets, current problems, object counts; `expiring_tokens` (tokens that expire within 14 days or expired in the last 14: all for administrators, otherwise the caller's); `backup` (administrators: `last_at`, `last_file`, `last_size`, `last_error`, `last_error_at`, `dir`, `warn_after` in seconds, `stale`); `blocklist_errors` (with `blocking:read`: lists whose last update failed, `{name, error, updated_at}`); `blocking_paused_until` or `blocking_paused_indefinitely` (with `blocking:read`, while paused). `problems` include named-checkconf's warnings about the running configuration (`named_warning`) |
| POST | `/system/check` | validation + named-checkconf of the current configuration (`options:read`) |
| POST | `/system/resync` | `{"dry_run": false, "force": false}`: rewrite every file from the database and reload (admin) |
| GET | `/audit` | `?user=&action=&outcome=&object_type=&changeset=&q=&since=&until=&limit=200&before=ID` |
@@ -452,11 +513,12 @@ Needs `stats:read`. When statistics are disabled, every endpoint but
| Method | Path | Query | Returns |
|--------|------|-------|---------|
| GET | `/stats/status` | | `{enabled, query_log, statistics_channel, log_file, last_poll, last_log_line, log_lines_read, first_data, retention, warnings}` |
| GET | `/stats/summary` | `since` (90m, 24h, 7d, 2w; default 24h), `top` (default 10) | totals, `avg_qps`, `peak`, `rcodes`, `qtypes`, `views`, `zones`, `answers`, `transport`, `cache_hit_ratio`, `nxdomain_ratio`, `tops` |
| GET | `/stats/summary` | `since` (90m, 24h, 7d, 2w; default 24h), `top` (default 10) | totals, `avg_qps`, `peak`, `rcodes`, `qtypes`, `views`, `zones`, `answers`, `transport`, `cache_hit_ratio`, `nxdomain_ratio` (without blocked answers), `blocked`, `blocked_ratio`, `blocklists` (blocked per list; `rules` for your rules), `tops` |
| GET | `/stats/timeline` | `since`, `step` (auto, 1m, 5m, 15m, 1h, 6h, 1d), `key` (repeatable or comma separated; `rcode:*` = every counter of a group; default `queries`) | `{step, times: [unix...], series: [{key, values}]}` |
| GET | `/stats/top` | `kind` (name, client, nxdomain, servfail, refused), `since`, `limit` (default 20) | `[{key, count}]` |
| GET | `/stats/top` | `kind` (name, client, nxdomain, servfail, refused, blocked, blocked_client), `since`, `limit` (default 20) | `[{key, count}]`; client, refused and blocked_client entries add `name`, the host name of Jarl's record for the address (the summary's client list too) |
| GET | `/stats/heatmap` | `since` (default 28d), `key` (default queries) | `{key, from, avg[7][24], samples[7][24]}`, Monday first, local time, `-1` = no data |
| GET | `/stats/breakdown` | `group` (qtype, rcode, view, zone, answer, transport, rtt, dnssec, cache, ratelimit), `since` | `[{key, count}]` |
| GET | `/stats/server` | | `{server: {rss, mem_in_use, cpu_percent, threads, pid, cache_rrsets, cache_mem, sockets_udp, sockets_tcp, recursion_highwater, tcp_highwater, version, boot_time, config_time, host: {mem_total, mem_available, load, cpus, uptime}}, named: {recursive_clients, tcp_clients, zones, cpus, worker_threads}, disks: [{path, holds, total, free}]}`; gauges over time: `/stats/timeline?key=g:named:rss` (averages, -1 without a sample) |
| GET | `/stats/breakdown` | `group` (qtype, rcode, view, zone, answer, transport, rtt, dnssec, cache, ratelimit, blocklist), `since` | `[{key, count}]` |
```sh
curl -sf -H "$AUTH" "$API/stats/top?kind=nxdomain&since=7d&limit=10" | jq -r '.[] | "\(.count)\t\(.key)"'
+267
View File
@@ -0,0 +1,267 @@
# Blocking
Jarl can make named answer "this name doesn't exist" (NXDOMAIN) for ads,
trackers and malware, like Pi-hole does, for every device that uses your
server to resolve names. It has two parts:
- **Blocklists**: lists of domains published by others, which jarld
downloads and keeps up to date.
- **Your rules**: domains you block or allow yourself. They take precedence
over every list.
It uses BIND's own mechanism for this, response policy zones (RPZ), so named
does the blocking itself; jarld only prepares the lists.
- [Quick start](#quick-start)
- [Where blocking applies](#where-blocking-applies)
- [Your rules](#your-rules)
- [Blocklists](#blocklists)
- [Checking a name](#checking-a-name)
- [Pausing](#pausing)
- [Statistics](#statistics)
- [Memory and performance](#memory-and-performance)
- [How it works](#how-it-works)
- [Troubleshooting](#troubleshooting)
## Quick start
In the web UI, open **Blocking**, pick one of the well-known lists, and
click **Add list**. Or with the CLI:
```sh
jarl blocklist add hagezi-multi https://raw.githubusercontent.com/hagezi/dns-blocklists/main/adblock/multi.txt
jarl blocklist list # "ok" once downloaded, usually within seconds
jarl block check ads.doubleclick.net # blocked? by what?
jarl block allow cdn.example.com # something you need was blocked
```
Like every change in Jarl, adding a list or rule can be staged and reviewed
first (the web UI does this by default: apply it under **Changes**).
## Where blocking applies
Blocking changes the answers named gives when it resolves names for a
client (recursion). It doesn't change what the internet sees of your own
zones: other resolvers ask your server without recursion.
**Views.** A list or rule applies to the views you choose, or by default
to every view that offers recursion. With the usual split (an `internal`
view that resolves for your network, an `external` view that only answers
for your zones) that is just `internal`. A view that doesn't offer
recursion gets nothing, and naming one gives a warning. Without views,
blocking applies to the whole server.
**What it can't do:**
- Devices that don't use your server for DNS aren't affected. That includes
browsers using their own DNS over HTTPS, and devices with a hard-coded
resolver (8.8.8.8). A firewall rule that sends port 53 to your server, and
your network's DNS settings, are the usual answers.
- A device that validates DNSSEC itself sees a blocked name as a
validation failure rather than "doesn't exist". It's still blocked.
- A list can contain names in your own zones. They are blocked like any
other; allow them with a rule if that's not what you want.
## Your rules
A rule **blocks** or **allows** a domain and every name below it, or with
`--exact` (web UI: "only this exact name") just the domain itself.
```sh
jarl block add doubleclick.net # doubleclick.net and everything below it
jarl block add telemetry.example.com --exact # just that name
jarl block allow cdn.example.com --view internal # an exception, in one view
jarl block list
jarl block delete doubleclick.net # or: --id ID
```
**Precedence:** your rules come before every blocklist, and among your
rules the most specific one wins. With these rules:
| Rule | |
|------|-|
| block `example.com` | and below |
| allow `cdn.example.com` | exact |
`example.com` and `www.example.com` are blocked, `cdn.example.com` is
allowed, and `img.cdn.example.com` is blocked again (the allow rule was for
the exact name). An allow rule beats a blocklist entry whatever the lists
say.
There is one rule per domain and view; a second one is refused. A rule
takes effect as soon as the change is applied.
## Blocklists
**Formats.** Lists are read line by line; one list may mix formats:
| Line | Blocks |
|------|--------|
| `\|\|ads.example^` (Adblock) | the name and every name below it |
| `*.ads.example` | the name and every name below it |
| `0.0.0.0 ads.example` (hosts file, any address) | just the name |
| `ads.example` (plain domain) | just the name |
Comments (`#`, `!`), headers and blank lines are ignored. Lines that DNS
can't act on are skipped and counted: Adblock exceptions (`@@`), cosmetic
and URL rules, addresses, invalid names, and single labels (a broken list
can't block `com`). Prefer lists in Adblock format: they say which entries
cover the names below them.
**Well-known lists** (sizes in October 2026; they change):
| List | URL | Domains | Memory per view |
|------|-----|--------:|----------------:|
| Hagezi Light | `https://raw.githubusercontent.com/hagezi/dns-blocklists/main/adblock/light.txt` | 55,000 | ~65 MB |
| Hagezi Multi Normal | `https://raw.githubusercontent.com/hagezi/dns-blocklists/main/adblock/multi.txt` | 159,000 | ~190 MB |
| Hagezi Pro | `https://raw.githubusercontent.com/hagezi/dns-blocklists/main/adblock/pro.txt` | 195,000 | ~235 MB |
| OISD Small | `https://small.oisd.nl/` | 58,000 | ~70 MB |
| OISD Big | `https://big.oisd.nl/` | 241,000 | ~290 MB |
| StevenBlack hosts | `https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts` | 73,000 | ~45 MB |
One good list is usually enough; lists overlap a lot. Hagezi Multi Normal
is a balanced start; Light and OISD Small block less but almost never
something you need.
**Updates.** jarld checks each list every `blocking.refresh` (a day by
default) and downloads it only when the server says it changed. A new
version is checked with `named-checkzone` and replaces the old one only
when its content differs; named then reloads it. If a download or check
fails, named keeps using the previous version, jarld tries again within
the hour, and the dashboard shows the error. To update now:
```sh
jarl blocklist refresh # every list; or: refresh hagezi-multi [--force]
```
**Managing lists:**
```sh
jarl blocklist update hagezi-multi --view internal # --view auto: every view with recursion
jarl blocklist update hagezi-multi --url URL --name NEW-NAME
jarl blocklist disable hagezi-multi # stop using it, keep it (enable: back without a download)
jarl blocklist delete hagezi-multi
```
A list's state is **ok** (in use), **waiting** (not downloaded yet),
**error** (the last update failed; the previous version stays in use),
**disabled**, or **idle** (it applies to no view, so it does nothing).
## Checking a name
`jarl block check NAME`, or the box at the top of the Blocking page, says
for each view whether a name is blocked or allowed, and by which rule or
list entry:
```
$ jarl block check img.ads.example.com
view internal: blocked by blocklist hagezi-multi (entry *.ads.example.com)
view guest: allowed by your rule 12 (entry *.ads.example.com)
```
Views without blocking aren't listed.
In the web UI a blocked name gets an **Allow** button, and the statistics'
most blocked names link to this check.
## Pausing
To see whether blocking is what breaks a site, pause it: every name is
answered normally until the time is up (at most a day), then blocking
resumes by itself, also if jarld was restarted in between. A pause can
also last until someone resumes blocking (like Pi-hole's "indefinitely").
```sh
jarl block pause # 10 minutes; or: jarl block pause 1h
jarl block pause indefinitely
jarl block resume # end it now
```
In the web UI, **Pause blocking** is at the top of the Blocking page (with
**Until I resume** among the lengths), and the dashboard says while
blocking is paused. The lists stay loaded (the
rules zone just lets everything through), so pausing and resuming take
effect within a second. Pausing and resuming are recorded in the history
like other changes.
## Statistics
With [query statistics](stats.md) on (including `query_log`), Jarl counts
what blocking does: blocked queries over time (next to all queries) and
as a share of them, which rules and lists blocked them, the most blocked
names, and the clients with the most blocked queries (`jarl stats summary`,
`jarl stats top blocked`, `jarl stats top blocked-clients`, `jarl stats
breakdown blocklist`). Blocked answers are left out of the NXDOMAIN figures, which
are about names that really don't exist.
Without statistics named doesn't log what it blocks at all.
## Memory and performance
**Memory** is the cost to watch. named keeps each list in memory, once per
view it applies to: about 0.6 KB per policy record (an Adblock entry is
two records: the name, and the names below it). The Blocking page and
`jarl blocklist list` show the estimate per list. Hagezi Pro in one view
took named from 25 MB to 260 MB in a test with BIND 9.20; while a list
reloads, the old and new versions briefly coexist.
**Answers aren't slowed**: named looks blocked names up in memory, and
answers them without asking the internet.
**Updating a list** (Hagezi Pro: download, parse, check) takes jarld about
2 seconds. While named loads the new version, the queries that the loading
thread would have answered wait, up to half a second; this only happens
when the list changed, at most once per `blocking.refresh`.
**Other changes are unaffected**: the lists' contents never go through
Jarl's apply, so adding a record takes as long with a list as without.
Downloads are limited by `blocking.max_download_mb` and
`blocking.max_entries` ([configuration.md](configuration.md#blocking)).
## How it works
For each view (or the server, without views) Jarl writes a
`response-policy` statement and the policy zones it lists, in this order:
```
response-policy {
zone "rules.jarl.invalid" min-update-interval 0; # your rules
zone "hagezi-multi.blocklist.jarl.invalid" min-update-interval 0;
} qname-wait-recurse no break-dnssec yes;
```
- Policy zones are named under `.invalid`, so they can't clash with real
zones (zone names under `jarl.invalid` are refused). They allow no
queries or transfers.
- The rules zone of each view is a file next to your zone files
(`/var/lib/bind/jarl/VIEW/rules.jarl.invalid.rpz`) and shows up in the
changeset diff like any other.
- A list's file is `/var/cache/bind/jarl/blocklists/list-ID.rpz`, shared by
the views that use it. Until the first download it is an empty policy
zone, so named always starts. Files of deleted lists are removed.
- `qname-wait-recurse no` blocks without first resolving the real answer;
`break-dnssec yes` also blocks for clients that ask for DNSSEC data;
`min-update-interval 0` applies a new version at once.
- named looks names up in a policy zone like in any zone, so a wildcard
doesn't apply below a name the zone also contains. Jarl writes the zones
so that the most specific rule still wins.
- A `response-policy` option of your own can't be combined with Jarl's
blocking in the same view: Jarl refuses the change and says so.
- Rewrites are logged to Jarl's response log when statistics read it, and
not at all otherwise.
Lists and rules need the `blocking` permission ([permissions.md](permissions.md)).
## Troubleshooting
- **"no domains found in N lines"**: the URL didn't return a list (often an
HTML error page). Check it in a browser.
- **A list has no effect / is "idle"**: no view it applies to offers
recursion. Pick the view your devices use (`--view internal`).
- **Something is still answered**: `jarl block check NAME` shows whether
Jarl blocks it. If it does, the device probably doesn't use your server
(DNS over HTTPS, another resolver), or has the answer cached for a
while.
- **Something you need is blocked**: `jarl block check NAME` shows which
list; `jarl block allow NAME` overrides it.
+93 -8
View File
@@ -139,10 +139,11 @@ to stderr with `-o json` or `-o yaml`, so stdout holds just the documents.
| Flag | Meaning |
|------|---------|
| `-o table\|json\|yaml` | Output format. JSON and YAML have the API's full objects, for scripts. |
| `-o id`, `-o field=NAME[,NAME...]` | Just the IDs, or just those fields, one line per object (fields separated by tabs). See [Scripting tips](#scripting-tips). |
| `-n, --dry-run` | Validate and show the diff without changing anything. |
| `-C, --changeset ID` | Stage the change into an open changeset instead of applying it. |
| `--force` | Overwrite BIND files that were changed outside Jarl (drift). |
| `-q, --quiet` | Print only errors, or just the ID where a command creates something. |
| `-q, --quiet` | Print only errors, or just the IDs: of what a command creates, or what `show` and `list` find (like `-o id`). |
## Exit codes
@@ -181,12 +182,21 @@ jarl zone create consul --type forward --forwarder "127.0.0.1 port 8600"
jarl zone update adbraun.lol --ttl 3600 --contact hostmaster@adbraun.lol --refresh 7200
jarl zone update adbraun.lol --move-to-view internal
jarl zone delete lab.adbraun.lol [-y]
jarl zone disable lab.adbraun.lol [--view V] # named stops serving it; kept (and editable) in Jarl
jarl zone enable lab.adbraun.lol [--view V]
jarl zone file adbraun.lol # the generated zone file
jarl zone import adbraun.lol db.adbraun.lol [--replace] [--use-soa]
jarl zone serials adbraun.lol # are the secondaries (or primaries) on the current serial?
jarl zone delegation adbraun.lol [--resolver 9.9.9.9] # does the parent zone delegate it correctly?
```
**Disabling a zone** leaves it out of named's configuration but keeps it in
Jarl: records, options and history, which can still be edited. Its names
are then answered by recursion where the view offers it (the internet's
version of the zone, if any), and REFUSED elsewhere; `disable` says which.
Enabling it serves it as it is then. A signed zone can't be disabled (turn
signing off first), nor a zone another view shares with `in-view`.
Reverse names follow the dots (the nibbles in IPv6), so a reverse zone
covers a /8, /16 or /24 (a multiple of /4 for IPv6). A network in between is
covered by several zones, created together in one changeset: a /22 by four
@@ -245,7 +255,7 @@ Jarl.)
## Records
```sh
jarl record list adbraun.lol [--name www] [-t A] [--search nas]
jarl record list adbraun.lol [--name www] [-t A] [--search nas] [--disabled]
jarl record add adbraun.lol nas A 192.168.1.20 --ptr --comment "synology"
jarl record add adbraun.lol @ MX 10 mail
jarl record add adbraun.lol @ TXT "v=spf1 mx -all" # plain text, quoting is automatic
@@ -257,11 +267,26 @@ jarl record update 1234 --ttl default
jarl record delete adbraun.lol www A # every A record of www
jarl record delete adbraun.lol www A 192.168.1.10 # just this one
jarl record delete --id 1234 [--ptr]
jarl record disable adbraun.lol www A 192.168.1.10 --ptr # out of the zone, kept in Jarl
jarl record enable adbraun.lol www A 192.168.1.10 --ptr # back again
jarl record disable --id 1234 # by ID; same for enable
# reverse zones: name records by address; the host name needs no final dot
jarl record add 0.10.10.in-addr.arpa 10.10.0.20 PTR nas.adbraun.lol
# a zone in several views (split horizon): the same change in each
jarl record add adbraun.lol mx2 A 203.0.113.5 --view internal,external
jarl record delete adbraun.lol old A --view all -y
```
**Several views.** When a zone exists in more than one view, the record
commands that change records (`add`, `set`, `delete`, `disable`,
`enable`) take `--view a,b` or `--view all` (every view the zone is in).
The changes are applied together as one changeset: if one view refuses
(the zone isn't in it, a CNAME conflict, ...), no view is changed. With
`--changeset` they are staged there instead. Keep split-horizon data that
differs per view (an internal address, say) to one view at a time.
Names are relative to the zone (`@` is the apex) unless they end with a
dot. Data uses zone file syntax, except that TXT/SPF take plain text: Jarl
adds the quotes and splits long values into 255-byte strings. In reverse zones the name can be the address
@@ -273,6 +298,16 @@ by Jarl). Every record type BIND supports can be used. Jarl knows the
syntax of the common ones and checks them strictly, and the rest are
validated by parsing.
**Disabling** a record takes it out of the zone without deleting it: it
keeps its ID, TTL and comment, the zone file lists it as a comment
(`; disabled: www IN A ...`), and `enable` puts it back exactly as it
was. `record list` marks disabled records `[disabled]`, and `zone show`
counts them. The zone is validated as it is served: you can't disable its
last NS record, and enabling a record is checked like adding it (a CNAME
added at the same name in the meantime stops it). `record set` enables a
disabled record whose value it is given, and leaves other disabled
records alone.
## Batches: `jarl apply -f`
Several changes that must succeed or fail together:
@@ -287,6 +322,7 @@ changes:
- set: {zone: adbraun.lol, name: api, type: A, data: [192.168.1.41, 192.168.1.42], ttl: 60}
- update: {id: 1235, comment: "old web server, remove in November"}
- delete: {id: 1240}
- disable: {zone: adbraun.lol, name: old, type: A} # or {id: ...}; enable: likewise
- op: acl.update # any API operation (see api.md)
id: 3
data: {elements: [192.168.1.0/24, 10.10.0.0/16]}
@@ -332,6 +368,7 @@ jarl view order internal external
jarl view update external --match-clients "!homelab,any"
jarl view delete old [--cascade]
jarl view color internal green # web UI colour; "default" resets it
jarl view key external [--named-conf] # the key that selects the view (dig -k), docs/views.md
jarl acl create homelab 192.168.1.0/24 10.10.0.0/16 localhost
jarl acl update homelab --add 10.20.0.0/16 --remove 10.10.0.0/16
@@ -347,12 +384,21 @@ jarl option list [--view V | --zone Z | --toplevel | --all]
jarl option set recursion yes
jarl option set allow-recursion '{ homelab; localhost; }'
jarl option set forwarders '{ 1.1.1.1; 9.9.9.9; }'
jarl option set allow-transfer '{ key xfer; }' --zone adbraun.lol
jarl option set forwarders 1.1.1.1 9.9.9.9 # the same: braces are added to a list
jarl option set allow-transfer 'key xfer; 10.0.10.53' --zone adbraun.lol
jarl option set response-policy '{ zone "rpz.local"; }' --view internal
jarl option set logging '{ channel q { file "/var/log/named/query.log" versions 3 size 20m; }; category queries { q; }; }' --toplevel
jarl option unset forwarders
jarl option describe # every option allowed at this scope
jarl option describe allow-query-cache # grammar and ARM description
```
Options that end in a list (`allow-*`, `forwarders`, `listen-on`, ...) may
be given without the braces: elements are separated by `;`, `,` or spaces,
and `key NAME` and `port N` stay with their element. Options are always
listed in one fixed order (see [concepts.md](concepts.md)).
```sh
jarl settings show
jarl settings set --ns ns1.adbraun.lol.,ns2.adbraun.lol. --contact hostmaster@adbraun.lol --ttl 3600
@@ -404,15 +450,37 @@ working.
```sh
jarl stats status
jarl stats summary --since 7d
jarl stats top names | clients | nxdomain | servfail | refused [--since 24h] [--limit 20]
jarl stats top names | clients | nxdomain | servfail | refused | blocked | blocked-clients [--since 24h] [--limit 20]
jarl stats timeline --since 6h [--key rcode:NXDOMAIN] [--table]
jarl stats heatmap --since 28d
jarl stats breakdown qtype | rcode | view | zone | answer | rtt
jarl stats breakdown qtype | rcode | view | zone | answer | rtt | blocklist
jarl stats server # named's memory, CPU, cache, clients; host; disk space
```
Statistics must be enabled in `jarl.yaml`, and `-o json` exports the data.
See [stats.md](stats.md).
## Blocking
```sh
jarl blocklist list
jarl blocklist add NAME URL [--view V]... # default: every view that offers recursion
jarl blocklist update NAME [--url U] [--name N] [--view V | --view auto] [--comment C]
jarl blocklist disable | enable | delete NAME
jarl blocklist refresh [NAME...] [--force] # download now (all lists without NAME)
jarl block add DOMAIN [--exact] [--view V]... # block it (and the names below it)
jarl block allow DOMAIN [--exact] [--view V]... # an exception that beats every list
jarl block list
jarl block delete DOMAIN [--view V] | --id ID
jarl block check DOMAIN # blocked? by which rule or list?
jarl block pause [DURATION] # let everything through for a while (default 10m, at most 24h)
jarl block pause indefinitely # ... until 'jarl block resume'
jarl block resume
```
See [blocking.md](blocking.md).
## ACME DNS-01
```sh
@@ -426,8 +494,25 @@ See [acme.md](acme.md) for certbot, acme.sh and a dedicated ACME zone.
## Scripting tips
* `-o json` gives full API objects. Pipe them to `jq`.
* `-q` with `create` commands prints just the ID or token.
* `-o id` prints just the IDs of what a `show` or `list` command finds, one
per line, and `-o field=...` picks fields (their names are those of
`-o json`; a dot reaches inside, as in `soa.serial`; a field an object
doesn't have prints as empty). For changes, they apply to the changed
object:
```sh
ZONE=$(jarl zone show adbraun.lol --view external -o id)
jarl record list adbraun.lol --name www -t A -o id | xargs -n1 jarl record disable --id
jarl zone show adbraun.lol -o field=view,soa.serial # external 2026100901
jarl zone list -o field=id,name,view
ID=$(jarl record add adbraun.lol www A 192.0.2.10 -o id) # the new record's ID
```
* `-q` with `create` commands prints just the ID or token, and with `show`
and `list` just the IDs. Commands without IDs (status, checks) print
nothing but errors.
* `jarl record list` on a terminal shortens data too long for a line
(`…`); `--wide` shows it in full, and so does output to a pipe or file.
* Check `$?` against the exit code table.
* `jarl record delete ... --missing-ok` succeeds when nothing matches, which
keeps cleanup scripts idempotent. `jarl record set` is idempotent by nature.
keeps cleanup scripts idempotent. So does `record disable|enable
--missing-ok`, also when the records are already disabled (enabled). `jarl record set` is idempotent by nature.
+9 -1
View File
@@ -68,7 +68,9 @@ both, using a single mechanism:
* `POST /api/v1/apply` (CLI: `jarl apply -f changes.yaml`) applies a list of
operations as one changeset in a single request.
* `?dry_run=1` (CLI: `--dry-run`) runs the whole pipeline except installing
files and reloading named, and shows the problems and the diff.
files and reloading named, and shows the problems and the diff. Each
part of a named.conf diff says which block it is in, after the `@@`
line numbers: `@@ -50,6 +49,8 @@ view "internal" {`.
**Why bundling matters.** Some changes only make sense together. Turning
`www` from an A record into a CNAME means deleting the A record and adding
@@ -159,6 +161,12 @@ RFC 1982 arithmetic.
Jarl checks them before named does. Statements that Jarl manages itself
(`zone`, `view`, `acl`, `key`, a zone's `type`/`file`, a view's
`match-clients`, ...) can't be set as raw options.
Options of a block are kept in one fixed order, the same in named.conf,
the web UI and the CLI: grouped (files, listening, access control,
recursion and forwarding, DNSSEC, transfers and notify, then the rest),
alphabetical within a group. So the same option is in the same place in
every view and zone. Top-level statements stay in the order they were
added.
* **Settings** are Jarl's defaults for new zones: name servers, SOA contact
and timers, default TTL, serial policy.
+38 -1
View File
@@ -18,7 +18,7 @@ The CLI has its own small configuration file, described in [cli.md](cli.md).
| Key | Default | Meaning |
|-----|---------|---------|
| `listen` | `["127.0.0.1:8053"]` | Addresses to listen on (`host:port`, several allowed). Keep these on localhost and management networks. |
| `listen` | `["127.0.0.1:8053"]` | Addresses to listen on (`host:port`, several allowed). Keep these on localhost and management networks. Ports below 1024 (`"192.168.10.2:443"`) work because `jarl.service` gives jarld the `CAP_NET_BIND_SERVICE` capability, and only that ([HTTPS on port 443](#example-https-on-port-443)). |
| `socket` | `/run/jarl/jarl.sock` | A local unix socket for the API. Requests on it from root or jarld's own user need no token and have administrator rights; anyone else must send a token ([permissions.md](permissions.md#the-local-socket)). The `jarl` CLI uses it when no server is configured. At most 107 bytes long; `""` turns it off. If it can't be created, jarld logs a warning and serves the network addresses only. |
| `tls.cert_file`, `tls.key_file` | (empty: plain HTTP) | PEM certificate chain and key. Both or neither. The files are rechecked every 10 minutes and on SIGHUP (`systemctl reload jarl`), so renewed certificates are used without a restart. |
| `trusted_proxies` | `[]` | Addresses or prefixes of reverse proxies whose `X-Forwarded-For` and `X-Forwarded-Proto` (or `Forwarded: proto=`) headers are believed. Needed for correct client IPs in the audit log and for rate limiting behind a proxy, and for knowing that a proxied request came over HTTPS (the session cookie's Secure flag, `acme.httpreq`). From other addresses these headers are ignored. |
@@ -134,6 +134,17 @@ jarl.yaml; chmod 640 jarl.yaml`, as the Debian package installs it); jarld
warns at start when everyone can read it. At porkbun.com, switch on "API
Access" for the domain, and limit the key to the domain if you can.
### Blocking
How jarld downloads blocklists ([blocking.md](blocking.md)). The lists and
rules themselves are configuration in the database.
| Key | Default | Meaning |
|-----|---------|---------|
| `blocking.refresh` | `24h` | How often each blocklist is checked for changes (at least `1h`); it is downloaded only when it changed ([blocking.md](blocking.md)). |
| `blocking.max_download_mb` | `64` | Larger downloads are refused (the previous version stays in use). |
| `blocking.max_entries` | `1000000` | Lists with more domains are refused. |
## Example: management network with TLS
```yaml
@@ -149,6 +160,32 @@ audit:
retention_days: 730
```
## Example: HTTPS on port 443
```yaml
listen:
- "127.0.0.1:8053" # the CLI on the server keeps using this (or the socket)
- "192.168.10.2:443"
tls:
cert_file: /etc/jarl/tls/fullchain.pem
key_file: /etc/jarl/tls/privkey.pem
```
jarld runs as the unprivileged `jarl` user, and binding a port below 1024
needs the `CAP_NET_BIND_SERVICE` capability, which `jarl.service` grants
(`AmbientCapabilities=CAP_NET_BIND_SERVICE`; nothing else). The unit in
Jarl 0.2.0 and earlier doesn't, and jarld then fails with "bind:
permission denied". Upgrading the package installs the new unit; until
then, a drop-in does the same:
```sh
sudo systemctl edit jarl
# [Service]
# CapabilityBoundingSet=CAP_NET_BIND_SERVICE
# AmbientCapabilities=CAP_NET_BIND_SERVICE
sudo systemctl restart jarl
```
## Example: behind nginx
```yaml
+2
View File
@@ -28,6 +28,8 @@ internal/
nscheck/ DNS queries to other name servers: zone serials, the parent's delegation and DS records
dnssec/ reads named's DNSSEC key and state files; DS records; the signing phases (docs/dnssec.md)
porkbun/ Porkbun API client (name servers, glue, DS records) and an in-memory fake for tests
blocking/ blocklist parser, response policy zone writers, and matching a name the way named does
(the downloads are in engine/blocklists.go)
config/ jarld's YAML configuration
deploy/ systemd units (jarld, the daily backup service and timer), sysusers.d and tmpfiles.d files
deploy/debian/ the Debian package's control template and postinst/prerm/postrm
+12 -2
View File
@@ -122,8 +122,18 @@ can make validating resolvers reject cached answers. Then:
[DNSViz](https://dnsviz.net/) gives an independent picture.
If the parent zone is itself a primary zone in Jarl (`home.adbraun.lol`
inside `adbraun.lol`), the DNSSEC tab says so: add the DS as a record in
that zone.
inside `adbraun.lol`), the DS goes in as a record in that zone, and Jarl
gives you both ways to add it: an **Add to adbraun.lol** button on the
DNSSEC tab, and the command, which `jarl zone dnssec status` prints too:
```sh
jarl record add adbraun.lol home DS 51126 13 2 FFBECE3D...
```
Once the parent zone has it, the tab and `status` say so. When turning
signing off, the steps give the matching `jarl record delete adbraun.lol
home DS`. The zone must be delegated first (NS records for `home` in
`adbraun.lol`), or validation refuses the DS.
## Putting your own name servers on the internet
+3 -3
View File
@@ -88,9 +88,9 @@ password <token>
Download the two `.deb` files from the release page, or build them:
```sh
make deb # dist/jarl_0.1.0_amd64.deb and dist/jarl-cli_0.1.0_amd64.deb
make deb # dist/jarl_0.2.0_amd64.deb and dist/jarl-cli_0.2.0_amd64.deb
make deb DEB_ARCH=arm64 # for another architecture (also armhf, i386, ...)
sudo apt install ./dist/jarl_0.1.0_amd64.deb ./dist/jarl-cli_0.1.0_amd64.deb
sudo apt install ./dist/jarl_0.2.0_amd64.deb ./dist/jarl-cli_0.2.0_amd64.deb
```
Give apt both files: `jarl` needs `jarl-cli` of the same version.
@@ -139,7 +139,7 @@ To manage Jarl servers from your laptop or another server, install only
users, services or directories):
```sh
sudo apt install jarl-cli # with the registry above; or: sudo apt install ./jarl-cli_0.1.0_amd64.deb
sudo apt install jarl-cli # with the registry above; or: sudo apt install ./jarl-cli_0.2.0_amd64.deb
jarl config set server https://ns1.adbraun.lol:8053
jarl config set token
```
+2 -1
View File
@@ -135,6 +135,7 @@ RESOURCE:ACCESS [zones=PATTERNS] [views=PATTERNS] [names=PATTERNS] [types=TYPES]
| `options` | server options, top-level statements, the defaults for new zones, `jarl check` |
| `audit` | everyone's audit events and changesets (without it, users see only their own) |
| `stats` | query statistics, including client addresses and the names they look up (read only) |
| `blocking` | blocklists and block/allow rules ([blocking.md](blocking.md)); `write` also downloads lists now |
| `*` | all of the above |
`ACCESS` is `read` or `write` (write includes read). Filters narrow a grant.
@@ -168,7 +169,7 @@ you can then edit.
| Role | Grants |
|------|--------|
| `readonly` | `*:read` (includes statistics) |
| `dns-editor` | `zones:write`, `records:write`, `views:read`, `acls:read`, `options:read` |
| `dns-editor` | `zones:write`, `records:write`, `views:read`, `acls:read`, `options:read`, `blocking:read` |
| `operator` | `*:write` (everything except user management) |
| `acme` | `records:write names=_acme-challenge.* types=TXT` |
+40 -2
View File
@@ -102,22 +102,58 @@ lists, views, zones). named's own counters can't be filtered.
| `view:NAME` | response log | answers per view (`_default` without views) |
| `zone:NAME` | response log | answers per local zone; `zone:-` for names in none of your zones |
| `responses` | response log | lines read (a health check) |
| `blocked`, `blocklist:NAME` | response log | queries [blocking](blocking.md) answered, and by which list (`blocklist:rules` for your own rules) |
**Server** (sampled every minute; stored as gauges, `g:KEY` the sum of
the samples and `gn:KEY` their number, so a chart shows the average):
| Gauge | Source | Meaning |
|-------|--------|---------|
| `g:named:rss` | /proc | named's resident memory (what the system sees), bytes |
| `g:named:mem` | named | the memory named counts itself as using, bytes |
| `g:named:cpu` | /proc | named's CPU use over the minute, in tenths of a percent of one CPU |
| `g:cache:mem`, `g:cache:rrsets` | named | the cache's memory and RRsets, all views |
| `g:host:mem-used`, `g:host:load` | /proc | the host's memory in use, and its 1-minute load average times 100 |
Jarl finds named's process by name (the one whose command line mentions
named's main configuration file); reading its memory and CPU time needs no
privileges. With `/proc` mounted `hidepid`, it isn't visible and those
figures are left out. The current values also include named's sockets, its
recursive and TCP clients against their limits (`rndc status`), the most
at once since it started, and the space left on the file systems holding
the zone files, Jarl's cache, named.conf, the response log and this
database.
**Top lists** (per hour and per day, the `top_n` largest): `name` (most
queried names), `client`, `nxdomain` (names answered NXDOMAIN), `servfail`
(names answered SERVFAIL), `refused` (clients that were refused).
(names answered SERVFAIL), `refused` (clients that were refused), `blocked`
(names blocking answered), `blocked_client` (clients by blocked queries). Blocked answers are NXDOMAIN too, but they
aren't counted as names that don't exist: not in the `nxdomain` list, and
not in the summary's NXDOMAIN ratio.
Nothing else is stored. In particular there's no per-query record, so you
can't look up "who asked for X at 14:02". That is deliberate, both for size
and privacy.
**Client names.** Where the client lists (busiest and refused clients) show
an address Jarl has a record for, they show its host name too: the PTR
record's target, or else the name of an A/AAAA record with that address
(disabled records don't count, and only records you may read). This is
looked up in Jarl's own data when the list is shown, not stored and not
asked of DNS, so it is instant and adds no queries of its own. The name is
what the records say now: with DHCP, an address seen last week may have
been another machine. Anonymized clients (`anonymize_clients`) are
prefixes and get no name.
## Using it
**Web UI**: Statistics. Pick a range (1 hour to 90 days); everything on the
page follows it. Charts have a hover readout (or focus them and use the
arrow keys) and a **Table** button showing the same data as a table. The
heatmap shows the average queries per hour for each weekday and hour over at
least the last 7 days. The page refreshes every minute.
least the last 7 days. The page refreshes every minute. **Server**, at the
bottom, shows named's memory, CPU, cache and clients, the host and the
disks now, and named's memory and CPU over the range.
**CLI**:
@@ -132,6 +168,8 @@ jarl stats timeline --since 6h # sparkline; --table for every bucket
jarl stats timeline --since 7d --key rcode:NXDOMAIN --key rcode:SERVFAIL
jarl stats heatmap --since 28d
jarl stats breakdown qtype | rcode | view | zone | answer | rtt | dnssec | cache | transport | ratelimit
jarl stats server # memory, CPU, cache, clients, host, disks now
jarl stats timeline --since 7d --key g:named:rss --table # named's memory over a week
jarl stats summary --since 30d -o json # export (also -o yaml)
```
+24 -1
View File
@@ -30,6 +30,7 @@ showing them.
| `record_out_of_zone` | error | The name must be inside the zone (catches `www.example.com` without the final dot) |
| `record_duplicate` | error | The same record can't exist twice |
| `cname_conflict` | error | A CNAME must be the only record at its name (RFC 1034), so no CNAME next to A, MX, TXT... |
| `disabled_conflict` | warning | A disabled record that couldn't be enabled again as the zone is now (a CNAME now shares its name) |
| `cname_multiple` | error | A name can have only one CNAME |
| `cname_apex` | error | No CNAME at the zone apex (it needs SOA and NS) |
| `cname_loop` | error | CNAMEs that point back to themselves |
@@ -56,6 +57,8 @@ showing them.
| `zone_missing_option` | error | Secondary, stub and mirror zones need `primaries` |
| `zone_file_required` / `zone_file_invalid` / `zone_file_managed` | error | Hint zones need a file. Forward zones have none. Primary zone files are Jarl's |
| `zone_inview_invalid`, `zone_inview_option` | error | `in-view` zones only inside views, and only with `forward`/`forwarders` |
| `zone_inview_disabled` | error | An `in-view` zone can't share a disabled zone |
| `zone_disabled_signed` | error | A signed zone can't be disabled: turn signing off first |
| `zone_records_unmanaged` | error | Only primary zones can have records in Jarl |
| `zone_serial_policy` | error | `date`, `increment` or `unixtime` |
| `zone_no_forwarders`, `zone_hint_name`, `zone_name_slash` | info / warning | Unusual but legal setups |
@@ -68,7 +71,7 @@ showing them.
| `acl_ref_missing`, `key_ref_missing` | error | Every ACL or TSIG key that is referenced must exist |
| `acl_cycle` | error | ACLs that include each other |
| `acl_duplicate`, `acl_name_invalid`, `acl_empty`, `acl_shadowed` | error / warning | Unique names, built-in names reserved, elements that can never match |
| `key_*` | error / warning | Valid key names, a supported algorithm, a base64 secret of a sensible length. hmac-md5/sha1 are flagged as weak |
| `key_*` | error / warning | Valid key names, a supported algorithm, a base64 secret of a sensible length. hmac-md5/sha1 are flagged as weak. Names starting with `jarl-view-` are reserved for the view keys (`key_name_reserved`) |
| `view_duplicate`, `view_name_invalid`, `view_acl_invalid` | error | Valid, unique views and match lists |
| `view_unreachable` | warning | A view that comes after one matching every client is never used |
| `option_unknown`, `option_scope_invalid` | error | The option must exist in BIND 9.20 and be allowed where it is set (server, view, zone of this type, top level) |
@@ -100,6 +103,19 @@ applied.
| `dnssec_old_keys` | info | Signing a zone whose key directory still has old keys |
| `serial_not_higher` | error | An explicit serial on a signed zone must be above the one named has loaded (automatic serials are raised by themselves) |
## Blocking
From [blocking.md](blocking.md).
| Code | Severity | Rule |
|------|----------|------|
| `blocking_rule_duplicate` | error | One rule per domain and view |
| `blocking_response_policy` | error | No `response-policy` option where Jarl's blocking writes that statement (the view, or the server) |
| `zone_name_reserved` | error | Zone names under `jarl.invalid` are reserved for blocking's policy zones |
| `blocking_view_missing` | error | A list or rule names a view that doesn't exist |
| `blocking_no_view` | warning | A list or rule applies to no view: none offers recursion, and blocking only changes recursive answers |
| `blocking_no_recursion` | warning | A list or rule names a view that doesn't offer recursion, where it has no effect |
## Security lints
| Code | Severity | What it catches |
@@ -116,3 +132,10 @@ values Jarl accepts as raw named.conf syntax, such as `logging` channels
and `rrset-order`, are the main example. Their errors come back as
`validation_failed` (HTTP 422) with the checker's messages in `check`, each
tied to the object it belongs to.
named-checkconf's warnings don't block a change. They are shown with the
change, and the dashboard's configuration checks list them for as long as
they apply (`named_warning`; `named_error` if the running configuration no
longer passes, after a hand edit say). Jarl runs named-checkconf on the
running configuration for this, without loading zones, when the dashboard
asks and the configuration changed, or at most every 10 minutes.
+41 -2
View File
@@ -116,7 +116,44 @@ dig @<public-ip> example.com # from outside: status R
jarl check
```
On the server, `dig -b <source address>` picks which view you test.
On the server, `dig -b <source address>` picks which view you test, or,
from anywhere, sign the query with the view's key (below):
```sh
jarl view key external --named-conf > external.key
dig -k external.key @ns1.adbraun.lol nas.adbraun.lol +short # 203.0.113.20, even from the LAN
```
## View keys
named picks a view by where a query comes from, so a query this server
sends to its own public address usually lands in the internal view: its
source is one of your own networks. That makes checks of the external view
from the server itself (such as Jarl's delegation check) see the wrong
view.
So Jarl gives every view a TSIG key, `jarl-view-NAME`, and puts it at the
front of the view's `match-clients`:
```
view "internal" {
match-clients { !key jarl-view-external; key jarl-view-internal; homelab; };
...
view "external" {
match-clients { key jarl-view-external; any; };
```
A query signed with a view's key lands in that view, whatever its source;
the `!key` stops an earlier view from taking a later view's queries.
Clients don't send these keys, so for them nothing changes. Jarl signs its
own queries to this server's addresses with the key of the zone's view.
The keys live in `keys.conf` with your other keys, never in diffs or in
`jarl tsig list`; their secrets are derived from a random value in Jarl's
database (and in its backups), so they stay the same from one apply to
the next. `jarl view key VIEW`
shows one (it needs `tsig:write` and is audited). Key names starting with
`jarl-view-` are reserved, and the view keys can be used in your own
options (`allow-transfer { key jarl-view-external; }`).
## Living with views
@@ -130,7 +167,9 @@ On the server, `dig -b <source address>` picks which view you test.
`jarl view list` shows it.
* Deleting a view is refused while it still has zones, unless you pass
`--cascade`.
* Give each view a colour (Views → Edit → Colour, or
* Each view has its own page (Views → the view's name): its settings,
options and zones.
* Give each view a colour (the view's page → Settings → Colour, or
`jarl view color internal green`) to tell the zones of each view apart
in the zone list. The colour only affects the web UI, so it applies at
once, with no changeset or deploy. `jarl view color internal default`
+59 -7
View File
@@ -19,11 +19,13 @@ const Prefix = "/api/v1"
// Zone is a zone as returned by the API.
type Zone struct {
model.Zone
Display string `json:"display_name"`
View string `json:"view,omitempty"`
Records int `json:"record_count"`
ZoneFile string `json:"zone_file,omitempty"` // the file named loads
RName string `json:"rname_email,omitempty"`
Display string `json:"display_name"`
View string `json:"view,omitempty"`
Records int `json:"record_count"`
// DisabledRecords is how many of the records are disabled.
DisabledRecords int `json:"disabled_count,omitempty"`
ZoneFile string `json:"zone_file,omitempty"` // the file named loads
RName string `json:"rname_email,omitempty"`
// Registrar names the registrar Jarl can manage the domain at
// ("porkbun"), when one is configured for it.
Registrar string `json:"registrar,omitempty"`
@@ -40,7 +42,8 @@ type Record struct {
EffectiveTTL uint32 `json:"effective_ttl"`
// Text is the decoded value of TXT and SPF records.
Text *string `json:"text,omitempty"`
// Pending is set when listing with a changeset: add, change or delete.
// Pending is set when listing with a changeset: add, change, delete,
// disable or enable.
Pending string `json:"pending,omitempty"`
}
@@ -74,6 +77,10 @@ type Option struct {
Where string `json:"where"` // e.g. "view internal"
Description string `json:"description,omitempty"`
Status string `json:"status,omitempty"`
// Formatted is the value laid out over several lines, as in
// named.conf, when it has a block (for display and editing; Value is
// the same on one line).
Formatted string `json:"formatted,omitempty"`
}
// User is a user with formatted grants.
@@ -239,6 +246,21 @@ type StatusResponse struct {
Backup *BackupStatus `json:"backup,omitempty"`
// TOTP is about two-factor login across users (administrators only).
TOTP *TOTPOverview `json:"totp,omitempty"`
// BlocklistErrors are blocklists whose last update failed (for those
// who may see blocking).
BlocklistErrors []BlocklistError `json:"blocklist_errors,omitempty"`
// BlockingPausedUntil is set while blocking is paused for a time,
// BlockingPausedIndefinitely while it is paused until resumed.
BlockingPausedUntil *time.Time `json:"blocking_paused_until,omitempty"`
BlockingPausedIndefinitely bool `json:"blocking_paused_indefinitely,omitempty"`
}
// BlocklistError is a blocklist whose last update failed. UpdatedAt is
// the version named keeps using (nil: none yet).
type BlocklistError struct {
Name string `json:"name"`
Error string `json:"error"`
UpdatedAt *time.Time `json:"updated_at,omitempty"`
}
// TOTPOverview is what the dashboard reports about two-factor login.
@@ -370,7 +392,12 @@ type DNSSECResponse struct {
*dnssec.Status
// ParentZone is set when the parent zone is a primary zone in Jarl
// (in the same view): the DS record then goes there, as a record.
ParentZone string `json:"parent_zone,omitempty"`
ParentZone string `json:"parent_zone,omitempty"`
ParentZoneID int64 `json:"parent_zone_id,omitempty"`
ParentView string `json:"parent_view,omitempty"`
// ParentDS is the DS records the parent zone in Jarl has for the
// zone (record data, e.g. "51126 13 2 FFBE...").
ParentDS []string `json:"parent_ds,omitempty"`
// Registrar names the registrar Jarl can publish the DS at
// ("porkbun"), when one is configured in jarl.yaml.
Registrar string `json:"registrar,omitempty"`
@@ -463,3 +490,28 @@ type RegistrarChangeResponse struct {
Checks []nscheck.Finding `json:"checks"`
Warnings []string `json:"warnings"`
}
// Blocklist is a blocklist as returned by the API (docs/blocking.md).
type Blocklist struct {
model.Blocklist
// Views are the views it applies to; AutoViews is set when that is
// "every view that offers recursion" (view_ids is empty).
Views []string `json:"views"`
AutoViews bool `json:"auto_views,omitempty"`
// State is ok, waiting (not downloaded yet), error (the last update
// failed; the previous version stays in use), disabled, or idle (it
// applies to no view).
State string `json:"state"`
Status *model.BlocklistStatus `json:"status,omitempty"`
// MemoryMB estimates what named needs for the list, in all its views.
MemoryMB int `json:"memory_mb,omitempty"`
Pending string `json:"pending,omitempty"`
}
// BlockRule is a block or allow rule as returned by the API.
type BlockRule struct {
model.BlockRule
Views []string `json:"views"`
AutoViews bool `json:"auto_views,omitempty"`
Pending string `json:"pending,omitempty"`
}
+12 -10
View File
@@ -12,19 +12,20 @@ import (
// Resources that grants apply to.
const (
ResZones = "zones" // zone definitions and zone options
ResRecords = "records" // records in zones
ResViews = "views" // views and view options
ResACLs = "acls"
ResTSIG = "tsig" // TSIG keys
ResOptions = "options" // server options, top-level statements, settings
ResAudit = "audit" // audit log and everyone's changesets
ResStats = "stats" // query statistics
ResAll = "*"
ResZones = "zones" // zone definitions and zone options
ResRecords = "records" // records in zones
ResViews = "views" // views and view options
ResACLs = "acls"
ResTSIG = "tsig" // TSIG keys
ResOptions = "options" // server options, top-level statements, settings
ResAudit = "audit" // audit log and everyone's changesets
ResStats = "stats" // query statistics
ResBlocking = "blocking" // blocklists and block/allow rules
ResAll = "*"
)
// Resources lists the resources in display order.
var Resources = []string{ResZones, ResRecords, ResViews, ResACLs, ResTSIG, ResOptions, ResAudit, ResStats}
var Resources = []string{ResZones, ResRecords, ResViews, ResACLs, ResTSIG, ResOptions, ResAudit, ResStats, ResBlocking}
// Access levels.
const (
@@ -377,6 +378,7 @@ var RoleTemplates = map[string][]model.Grant{
{Resource: ResViews, Access: Read},
{Resource: ResACLs, Access: Read},
{Resource: ResOptions, Access: Read},
{Resource: ResBlocking, Access: Read},
},
"operator": {{Resource: ResAll, Access: Write}},
"acme": {{Resource: ResRecords, Access: Write, Names: []string{"_acme-challenge.*"}, Types: []string{"TXT"}}},
+22 -1
View File
@@ -235,6 +235,12 @@ func (t *Tools) CheckDS(ctx context.Context, zone, view string, tag uint16, alg
return t.RNDCRun(ctx, args...)
}
// ZoneCommand is the rndc command line for a zone command, as shown to
// users ("rndc zonestatus example.org IN internal").
func ZoneCommand(cmd, zone, view string) string {
return "rndc " + strings.Join(zoneArgs(cmd, zone, view), " ")
}
func zoneArgs(cmd, zone, view string) []string {
z := strings.TrimSuffix(zone, ".")
if z == "" {
@@ -334,7 +340,14 @@ type ServerStatus struct {
ConfigFile string `json:"config_file"`
Zones string `json:"zones"`
Running bool `json:"running"`
Raw string `json:"raw"`
// RecursiveClients is "now/soft limit/hard limit" (the soft limit is
// where named starts dropping the oldest queries), TCPClients
// "now/limit".
RecursiveClients string `json:"recursive_clients,omitempty"`
TCPClients string `json:"tcp_clients,omitempty"`
CPUs string `json:"cpus,omitempty"`
WorkerThreads string `json:"worker_threads,omitempty"`
Raw string `json:"raw"`
}
// Status returns named's status.
@@ -357,6 +370,14 @@ func (t *Tools) Status(ctx context.Context) (*ServerStatus, error) {
st.ConfigFile = v
case ok && k == "number of zones":
st.Zones = v
case ok && k == "recursive clients":
st.RecursiveClients = v
case ok && k == "tcp clients":
st.TCPClients = v
case ok && k == "CPUs found":
st.CPUs = v
case ok && k == "worker threads":
st.WorkerThreads = v
case strings.Contains(line, "server is up and running"):
st.Running = true
}
+180
View File
@@ -0,0 +1,180 @@
package blocking
import (
"errors"
"os"
"path/filepath"
"strings"
"testing"
)
func TestParse(t *testing.T) {
src := "\uFEFF[Adblock Plus]\r\n" + `! Title: test list
# a comment
||ads.example^
||Tracker.Example.^$important
||cosmetic.example^$third-party
@@||allowed.example^
example.org##.banner
|https://url.example/path
0.0.0.0 hosts.example other.example # two names
127.0.0.1 localhost
127.0.0.1 localhost.localdomain
::1 ip6-localhost
0.0.0.0 0.0.0.0
plain.example
*.wild.example
single
1.2.3.4
bad_label!.example
münchen.example
www.ads.example
` + strings.Repeat("a", 64) + ".example\n"
l, err := Parse(strings.NewReader(src), Limits{})
if err != nil {
t.Fatal(err)
}
want := map[string]uint8{
"ads.example": Exact | Below, "tracker.example": Exact | Below,
"hosts.example": Exact, "other.example": Exact, "plain.example": Exact,
"wild.example": Exact | Below, "xn--mnchen-3ya.example": Exact, "www.ads.example": Exact,
}
if len(l.Domains) != len(want) {
t.Errorf("domains = %v", l.Domains)
}
for n, f := range want {
if l.Domains[n] != f {
t.Errorf("%s = %d, want %d", n, l.Domains[n], f)
}
}
// Skipped: $third-party, @@, ##, |url, localhost (x3), 0.0.0.0, single, 1.2.3.4, bad_label!, 64-char label.
if l.Skipped != 12 || l.Lines != 19 {
t.Errorf("lines %d, skipped %d", l.Lines, l.Skipped)
}
Compact(l)
if _, ok := l.Domains["www.ads.example"]; ok {
t.Error("www.ads.example is covered by ||ads.example^ and should be dropped")
}
var sb strings.Builder
n, hash, err := FeedZone(&sb, "test", 7, l)
if err != nil || n != 10 {
t.Fatalf("FeedZone: %d records, %v", n, err)
}
for _, line := range []string{"ads.example\tCNAME\t.\n", "*.ads.example\tCNAME\t.\n", "plain.example\tCNAME\t.\n", "@\tSOA\tlocalhost. hostmaster.localhost. 7 "} {
if !strings.Contains(sb.String(), line) {
t.Errorf("zone lacks %q:\n%s", line, sb.String())
}
}
if strings.Contains(sb.String(), "*.plain.example") || strings.Contains(sb.String(), "$ORIGIN") {
t.Errorf("zone:\n%s", sb.String())
}
// The hash ignores the serial.
var other strings.Builder
if _, h2, _ := FeedZone(&other, "test", 8, l); h2 != hash {
t.Error("hash depends on the serial")
}
}
func TestParseLimits(t *testing.T) {
_, err := Parse(strings.NewReader("a.example\nb.example\na.example\nc.example\n"), Limits{MaxEntries: 2})
if !errors.Is(err, ErrTooManyEntries) {
t.Errorf("err = %v", err)
}
if _, err := Parse(strings.NewReader("a.example\nb.example\na.example\n"), Limits{MaxEntries: 2}); err != nil {
t.Errorf("a repeated name counted against the limit: %v", err)
}
l, _ := Parse(strings.NewReader("long-name.example\nshort.example\n"), Limits{MaxNameLen: 15})
if len(l.Domains) != 1 || l.Skipped != 1 {
t.Errorf("max name length: %v", l.Domains)
}
}
func TestRuleEntries(t *testing.T) {
rules := []Rule{
{Name: "ads.example", Action: ActionBlock}, // and everything below
{Name: "cdn.ads.example", Action: ActionAllow, Exact: true}, // just this name
{Name: "wide.example", Action: ActionBlock}, // and below
{Name: "ok.wide.example", Action: ActionAllow}, // and below
{Name: "deep.x.y.wide.example", Action: ActionAllow, Exact: true}, // two empty names in between
{Name: "solo.example", Action: ActionBlock, Exact: true},
}
e := RuleEntries(rules)
for name, want := range map[string]string{
"ads.example": ActionBlock,
"x.ads.example": ActionBlock,
"cdn.ads.example": ActionAllow,
"y.cdn.ads.example": ActionBlock, // the wildcard above still applies
"ok.wide.example": ActionAllow,
"a.ok.wide.example": ActionAllow,
"y.wide.example": ActionBlock, // exists only because of a deeper rule
"x.y.wide.example": ActionBlock,
"z.x.y.wide.example": ActionBlock,
"deep.x.y.wide.example": ActionAllow,
"below.deep.x.y.wide.example": ActionBlock,
"solo.example": ActionBlock,
"other.example": "",
"sub.solo.example": "",
"example": "",
"unrelated.org": "",
} {
_, act, ok := MatchRules(e, name)
if !ok {
act = ""
}
if act != want {
t.Errorf("%s: %q, want %q", name, act, want)
}
}
z := RulesZoneFile("rules", rules)
if !strings.Contains(z, "cdn.ads.example\tCNAME\trpz-passthru.\n*.cdn.ads.example\tCNAME\t.\n") {
t.Errorf("rules zone:\n%s", z)
}
if strings.Index(z, "ads.example\t") > strings.Index(z, "*.ads.example\t") {
t.Errorf("a name should come before its wildcard:\n%s", z)
}
}
func TestMatchFeed(t *testing.T) {
l, _ := Parse(strings.NewReader("||tracker.example^\nexact.example\n||deep.a.example^\n"), Limits{})
Compact(l)
path := filepath.Join(t.TempDir(), "list.rpz")
f, _ := os.Create(path)
FeedZone(f, "t", 1, l)
f.Close()
for name, want := range map[string]string{
"tracker.example": "tracker.example",
"a.b.tracker.example": "*.tracker.example",
"exact.example": "exact.example",
"sub.exact.example": "",
"x.deep.a.example": "*.deep.a.example",
"a.example": "",
"example": "",
} {
owner, ok, err := MatchFeed(path, name)
if err != nil {
t.Fatal(err)
}
if !ok {
owner = ""
}
if owner != want {
t.Errorf("%s: %q, want %q", name, owner, want)
}
}
}
func TestNames(t *testing.T) {
if ListZone("hagezi") != "hagezi.blocklist.jarl.invalid" || ListName("hagezi.blocklist.jarl.invalid.") != "hagezi" || ListName(RulesZone) != "" || ListName("x.y.blocklist.jarl.invalid") != "" {
t.Error("zone names")
}
if n, err := NormalizeName("Ads.Example.COM."); err != nil || n != "ads.example.com" {
t.Errorf("NormalizeName = %q, %v", n, err)
}
for _, bad := range []string{"com", "1.2.3.4", "a b.example", "*.example.com", ""} {
if _, err := NormalizeName(bad); err == nil {
t.Errorf("%q accepted", bad)
}
}
}
+198
View File
@@ -0,0 +1,198 @@
// Package blocking turns blocklists and block/allow rules into BIND
// response policy zones (RPZ), and answers which rule or list matches a
// name.
//
// A policy zone holds one rule per owner name: "ads.example CNAME ."
// answers NXDOMAIN for ads.example, "*.ads.example CNAME ." for every name
// below it, and "CNAME rpz-passthru." answers normally (an exception).
// named looks names up in a policy zone like in any zone, so wildcards
// follow DNS rules: a wildcard doesn't apply below a name that exists in
// the zone. The writers here add the rules that keep "the most specific
// rule wins" true despite that (see RulesZone).
package blocking
import (
"bufio"
"errors"
"fmt"
"io"
"net/netip"
"strings"
"unicode/utf8"
"jarl/internal/dnsname"
)
// Flags say what a list entry covers.
const (
Exact uint8 = 1 << iota // the name itself
Below // every name below it
)
// List is a parsed blocklist.
type List struct {
// Domains maps names (lowercase, without the final dot) to flags.
Domains map[string]uint8
// Lines is the number of lines with an entry (not comments or blank
// lines); Skipped how many of them were not usable.
Lines, Skipped int
}
// Limits bound what Parse accepts.
type Limits struct {
MaxEntries int // distinct domains (0: no limit)
MaxNameLen int // longest name in characters (0: 253)
}
// ErrTooManyEntries is returned when a list has more than MaxEntries
// domains.
var ErrTooManyEntries = errors.New("too many entries")
// Parse reads a blocklist. Each line is one of:
//
// ||ads.example^ Adblock style: the name and everything below it
// *.ads.example the name and everything below it
// 0.0.0.0 ads.example hosts file (any address; several names allowed)
// ads.example just this name
//
// Comments (#, !), Adblock headers ([Adblock Plus]) and blank lines are
// ignored. Lines that can't be used for DNS blocking (exceptions,
// cosmetic and URL rules, addresses, single-label names, invalid names)
// are counted as skipped.
func Parse(r io.Reader, lim Limits) (*List, error) {
maxLen := lim.MaxNameLen
if maxLen <= 0 || maxLen > 253 {
maxLen = 253
}
l := &List{Domains: map[string]uint8{}}
full := false // a new name was refused because of MaxEntries
add := func(name string, flags uint8) bool {
name, ok := normalize(name, maxLen)
if !ok {
return false
}
if _, seen := l.Domains[name]; !seen && lim.MaxEntries > 0 && len(l.Domains) >= lim.MaxEntries {
full = true
return false
}
l.Domains[name] |= flags
return true
}
sc := bufio.NewScanner(r)
sc.Buffer(make([]byte, 64*1024), 1024*1024)
first := true
for sc.Scan() {
line := sc.Text()
if first {
line = strings.TrimPrefix(line, "\uFEFF")
first = false
}
line = strings.TrimSpace(line)
if line == "" || line[0] == '#' || line[0] == '!' || line[0] == '[' {
continue
}
l.Lines++
ok := false
switch {
case strings.HasPrefix(line, "||"):
ok = add(adblockDomain(line[2:]), Exact|Below)
case strings.ContainsAny(line, "|/$^@") || strings.Contains(line, "##"):
// Exceptions (@@), URL and cosmetic rules: not for DNS.
default:
if i := strings.Index(line, "#"); i >= 0 {
line = strings.TrimSpace(line[:i])
}
fields := strings.Fields(line)
switch {
case len(fields) == 1 && strings.HasPrefix(fields[0], "*."):
ok = add(fields[0][2:], Exact|Below)
case len(fields) == 1:
ok = add(fields[0], Exact)
case len(fields) > 1 && isAddr(fields[0]):
for _, f := range fields[1:] {
if add(f, Exact) {
ok = true
}
}
}
}
if full {
return l, fmt.Errorf("%w: the list has more than %d domains", ErrTooManyEntries, lim.MaxEntries)
}
if !ok {
l.Skipped++
}
}
if err := sc.Err(); err != nil {
return l, err
}
return l, nil
}
// adblockDomain returns the domain of an Adblock rule's body
// ("ads.example^" or "ads.example^$important"), or "" when the rule
// does more than block a domain (other modifiers, paths, wildcards).
func adblockDomain(s string) string {
d, rest, _ := strings.Cut(s, "^")
if rest != "" && rest != "$important" {
return ""
}
if strings.ContainsAny(d, "*/:|$") {
return ""
}
return d
}
func isAddr(s string) bool {
_, err := netip.ParseAddr(s)
return err == nil
}
// localNames are hosts-file names that aren't blocklist entries.
var localNames = map[string]bool{"localhost.localdomain": true, "ip6-localhost": true, "ip6-loopback": true}
// normalize lowercases a name, removes the final dot, converts
// international names to their ASCII form, and checks it is usable: at
// least two labels of letters, digits, hyphens and underscores, and not an
// address.
func normalize(name string, maxLen int) (string, bool) {
name = strings.TrimSuffix(strings.ToLower(strings.TrimSpace(name)), ".")
if name == "" || localNames[name] || isAddr(name) {
return "", false
}
for i := 0; i < len(name); i++ {
if name[i] >= utf8.RuneSelf {
a, err := dnsname.ToASCII(name)
if err != nil {
return "", false
}
name = strings.TrimSuffix(strings.ToLower(a), ".")
break
}
}
if len(name) > maxLen || !strings.Contains(name, ".") {
return "", false
}
for _, label := range strings.Split(name, ".") {
if label == "" || len(label) > 63 {
return "", false
}
for i := 0; i < len(label); i++ {
c := label[i]
if !(c >= 'a' && c <= 'z' || c >= '0' && c <= '9' || c == '-' || c == '_') {
return "", false
}
}
}
return name, true
}
// NormalizeName checks and normalizes a domain the way list entries are
// (for rules entered by hand). The name must have at least two labels.
func NormalizeName(name string) (string, error) {
n, ok := normalize(name, 253-len(RulesZone)-2)
if !ok {
return "", fmt.Errorf("%q is not a domain name that can be blocked (it needs at least two labels of letters, digits, hyphens or underscores, like ads.example.com)", name)
}
return n, nil
}
+313
View File
@@ -0,0 +1,313 @@
package blocking
import (
"bufio"
"crypto/sha256"
"encoding/hex"
"fmt"
"io"
"os"
"sort"
"strings"
)
// Policy zone names. They are under .invalid (RFC 6761), so they can
// never clash with a real zone, and named only uses them internally.
const (
// RulesZone holds the rules entered in Jarl (one copy per view).
RulesZone = "rules.jarl.invalid"
// listSuffix follows a list's name: hagezi.blocklist.jarl.invalid.
listSuffix = ".blocklist.jarl.invalid"
)
// ListZone returns the policy zone name of a blocklist.
func ListZone(name string) string { return name + listSuffix }
// ListName returns the blocklist name of a policy zone ("" for the rules
// zone and zones that aren't Jarl's).
func ListName(zone string) string {
zone = strings.TrimSuffix(strings.ToLower(zone), ".")
if n, ok := strings.CutSuffix(zone, listSuffix); ok && !strings.Contains(n, ".") {
return n
}
return ""
}
// Match says what blocking does with a name in one view.
type Match struct {
View string `json:"view,omitempty"`
Result string `json:"result"` // blocked, allowed or none
Entry string `json:"entry,omitempty"` // the matching entry: the name, or *.parent
RuleID int64 `json:"rule_id,omitempty"`
List string `json:"list,omitempty"` // the blocklist, when a list matched
}
// Actions of rules.
const (
ActionBlock = "block"
ActionAllow = "allow"
)
// cname is the CNAME target of each action: "." answers NXDOMAIN,
// rpz-passthru. answers as if there were no rule.
var cname = map[string]string{ActionBlock: ".", ActionAllow: "rpz-passthru."}
// header starts a policy zone file. There is no $ORIGIN: names are
// relative to the zone, so a list can be renamed without rewriting its
// file. The short TTLs make a change reach clients' caches quickly.
func header(title string, serial uint32) string {
return fmt.Sprintf("; %s\n; Generated by Jarl. DO NOT EDIT.\n$TTL 60\n@\tSOA\tlocalhost. hostmaster.localhost. %d 3600 600 86400 60\n\tNS\tlocalhost.\n", title, serial)
}
// Placeholder is the content of a list's zone file before the list has
// been downloaded: a valid, empty policy zone.
func Placeholder() string {
return header("Blocklist not downloaded yet.", 1)
}
// Rule is a block or allow rule entered by hand.
type Rule struct {
Name string // lowercase, without the final dot
Action string // ActionBlock or ActionAllow
Exact bool // only the name itself, not the names below it
}
// RuleEntries returns the policy zone owners for rules (relative names,
// "*." for wildcards) and their actions, with the entries that make the
// most specific rule win added: named won't apply a wildcard below a name
// that exists in the zone, so every name below a wildcard that exists
// (because a more specific rule mentions it) gets the wildcard's action
// for itself and for the names below it, unless a rule says otherwise.
// Callers make sure no two rules have the same name and scope.
func RuleEntries(rules []Rule) map[string]string {
e := map[string]string{}
for _, r := range rules {
e[r.Name] = r.Action
if !r.Exact {
e["*."+r.Name] = r.Action
}
}
// Every name that exists in the zone: owners and their ancestors.
nodes := map[string]bool{}
for owner := range e {
n := strings.TrimPrefix(owner, "*.")
for {
nodes[n] = true
i := strings.IndexByte(n, '.')
if i < 0 {
break
}
n = n[i+1:]
}
}
ordered := make([]string, 0, len(nodes))
for n := range nodes {
ordered = append(ordered, n)
}
// Shallow names first, so a filled-in wildcard is inherited further
// down.
sort.Slice(ordered, func(i, j int) bool {
di, dj := strings.Count(ordered[i], "."), strings.Count(ordered[j], ".")
if di != dj {
return di < dj
}
return ordered[i] < ordered[j]
})
for _, n := range ordered {
inherited := ""
for a := n; ; {
i := strings.IndexByte(a, '.')
if i < 0 {
break
}
a = a[i+1:]
if act, ok := e["*."+a]; ok {
inherited = act
break
}
}
if inherited == "" {
continue
}
if _, ok := e[n]; !ok {
e[n] = inherited
}
if _, ok := e["*."+n]; !ok {
e["*."+n] = inherited
}
}
return e
}
// PausedZoneFile renders a rules zone that lets every name through, for
// while blocking is paused: the rules zone comes first, and its match
// stops named from looking at the lists.
func PausedZoneFile(title string) string {
return header(title+" (blocking is paused)", 1) + "*\tCNAME\trpz-passthru.\n"
}
// RulesZoneFile renders the rules zone of a view.
func RulesZoneFile(title string, rules []Rule) string {
e := RuleEntries(rules)
owners := make([]string, 0, len(e))
for o := range e {
owners = append(owners, o)
}
sort.Slice(owners, func(i, j int) bool { return ownerLess(owners[i], owners[j]) })
var sb strings.Builder
sb.WriteString(header(title, 1))
for _, o := range owners {
fmt.Fprintf(&sb, "%s\tCNAME\t%s\n", o, cname[e[o]])
}
return sb.String()
}
// ownerLess sorts names by their labels from the right (example,
// ads.example, *.ads.example, cdn.ads.example), so related rules are next
// to each other.
func ownerLess(a, b string) bool {
ka, kb := reverseLabels(a), reverseLabels(b)
return ka < kb
}
func reverseLabels(n string) string {
labels := strings.Split(n, ".")
for i, j := 0, len(labels)-1; i < j; i, j = i+1, j-1 {
labels[i], labels[j] = labels[j], labels[i]
}
// A wildcard sorts right after the name it belongs to.
return strings.ReplaceAll(strings.Join(labels, "\x00"), "*", "\x01")
}
// MatchRules finds the rule entry named would apply to name (lowercase,
// without the final dot) in a rules zone with entries e: the name itself,
// or else the wildcard of its closest existing ancestor. ok is false when
// no entry applies.
func MatchRules(e map[string]string, name string) (owner, action string, ok bool) {
if act, found := e[name]; found {
return name, act, true
}
nodes := map[string]bool{}
for o := range e {
n := strings.TrimPrefix(o, "*.")
for {
nodes[n] = true
i := strings.IndexByte(n, '.')
if i < 0 {
break
}
n = n[i+1:]
}
}
if nodes[name] {
return "", "", false // the name exists without a rule of its own
}
for a := name; ; {
i := strings.IndexByte(a, '.')
if i < 0 {
return "", "", false
}
a = a[i+1:]
if nodes[a] {
if act, found := e["*."+a]; found {
return "*." + a, act, true
}
return "", "", false
}
}
}
// Compact drops entries another entry already covers (a name below a
// name whose subdomains are blocked). That keeps the zone small, and it
// matters for correctness: no name exists below a wildcard, so every
// wildcard applies to all names below it.
func Compact(l *List) {
for n := range l.Domains {
for a := n; ; {
i := strings.IndexByte(a, '.')
if i < 0 {
break
}
a = a[i+1:]
if l.Domains[a]&Below != 0 {
delete(l.Domains, n)
break
}
}
}
}
// FeedZone writes a list's policy zone to w and returns the number of
// records and a hash of them (the serial isn't part of the hash, so it
// tells whether the content changed). l must be compacted.
func FeedZone(w io.Writer, title string, serial uint32, l *List) (records int, hash string, err error) {
names := make([]string, 0, len(l.Domains))
for n := range l.Domains {
names = append(names, n)
}
sort.Strings(names)
bw := bufio.NewWriterSize(w, 256*1024)
h := sha256.New()
if _, err := bw.WriteString(header(title, serial)); err != nil {
return 0, "", err
}
write := func(owner string) error {
line := owner + "\tCNAME\t.\n"
h.Write([]byte(line))
records++
_, err := bw.WriteString(line)
return err
}
for _, n := range names {
f := l.Domains[n]
if f&Exact != 0 {
if err := write(n); err != nil {
return 0, "", err
}
}
if f&Below != 0 {
if err := write("*." + n); err != nil {
return 0, "", err
}
}
}
if err := bw.Flush(); err != nil {
return 0, "", err
}
return records, hex.EncodeToString(h.Sum(nil)), nil
}
// MatchFeed reports which entry of a list's zone file blocks name: the
// name itself, or a wildcard of one of its ancestors (the closest).
// Lists are compacted, so no name exists below a wildcard and the
// closest wildcard ancestor is the one named uses.
func MatchFeed(path, name string) (owner string, ok bool, err error) {
want := map[string]int{name: 0}
depth := 1
for a := name; ; depth++ {
i := strings.IndexByte(a, '.')
if i < 0 {
break
}
a = a[i+1:]
want["*."+a] = depth
}
f, err := os.Open(path)
if err != nil {
return "", false, err
}
defer f.Close()
best := -1
sc := bufio.NewScanner(f)
for sc.Scan() {
line := sc.Text()
o, _, found := strings.Cut(line, "\t")
if !found {
continue
}
if d, hit := want[o]; hit && (best < 0 || d < best) {
best, owner = d, o
}
}
return owner, best >= 0, sc.Err()
}
+77
View File
@@ -427,3 +427,80 @@ func classify(name, grammar string) (string, []string) {
}
return KindRaw, nil
}
// listParam matches an optional parameter before a statement's list, such
// as "[ port <integer> ]" in allow-transfer's grammar.
var listParam = regexp.MustCompile(`\[ ([a-z0-9-]+) `)
// WrapList adds the braces to a list value given without them, for
// statements that end in a list ("allow-transfer [ port <integer> ] {
// <address_match_element>; ... }"): "10.0.0.53" becomes "{ 10.0.0.53; }",
// "port 5353 10.0.0.53; key xfer" becomes "port 5353 { 10.0.0.53; key
// xfer; }". Elements are separated by ";", "," or new lines, or by spaces
// when they are all addresses. Values with a brace, and values of other
// statements, are returned as they are.
func (st *Statement) WrapList(v string) string {
g, _, _ := strings.Cut(st.Grammar, "\n")
open := strings.Index(g, " {")
if strings.Contains(v, "{") || open < 0 || !strings.HasSuffix(g, "; ... };") {
return v
}
params := map[string]bool{}
for _, m := range listParam.FindAllStringSubmatch(g[:open], -1) {
params[m[1]] = true
}
fields := strings.Fields(v)
i := 0
for i+1 < len(fields) && params[strings.ToLower(fields[i])] {
i += 2
}
head := strings.Join(fields[:i], " ")
rest := strings.Join(fields[i:], " ")
var elems []string
if !strings.ContainsAny(rest, ";,\n") {
elems = spaceElements(fields[i:])
} else {
for _, e := range strings.FieldsFunc(rest, func(r rune) bool { return r == ';' || r == ',' || r == '\n' }) {
if e = strings.TrimSpace(e); e != "" {
elems = append(elems, e)
}
}
}
if len(elems) == 0 {
return v
}
list := "{ " + strings.Join(elems, "; ") + "; }"
if head != "" {
return head + " " + list
}
return list
}
// spaceElements splits list elements separated by spaces: "key NAME"
// is one element, and "port N" and "tls NAME" belong to the address
// before them (forwarders { 192.0.2.1 port 53; }).
func spaceElements(fields []string) []string {
var out []string
for i := 0; i < len(fields); i++ {
f := fields[i]
switch strings.ToLower(f) {
case "key", "!key", "!":
if i+1 < len(fields) {
f += " " + fields[i+1]
i++
}
if f[0] == '!' && strings.HasPrefix(strings.ToLower(f), "! key") && i+1 < len(fields) {
f += " " + fields[i+1]
i++
}
case "port", "tls":
if len(out) > 0 && i+1 < len(fields) {
out[len(out)-1] += " " + f + " " + fields[i+1]
i++
continue
}
}
out = append(out, f)
}
return out
}
+25
View File
@@ -77,3 +77,28 @@ func TestCatalog(t *testing.T) {
t.Errorf("zone types = %s", types)
}
}
func TestWrapList(t *testing.T) {
c := Default()
for _, tc := range []struct{ scope, name, in, want string }{
{ScopeOptions, "allow-transfer", "10.0.10.53", "{ 10.0.10.53; }"},
{ScopeOptions, "allow-transfer", "localhost 10.0.10.53 key xfer ! key old !10.0.0.1", "{ localhost; 10.0.10.53; key xfer; ! key old; !10.0.0.1; }"},
{ScopeOptions, "forwarders", "1.1.1.1 port 853 tls cf 9.9.9.9", "{ 1.1.1.1 port 853 tls cf; 9.9.9.9; }"},
{ScopeOptions, "allow-transfer", "localhost; 10.0.10.53", "{ localhost; 10.0.10.53; }"},
{ScopeOptions, "allow-transfer", "port 5353 key xfer, !10.0.0.1", "port 5353 { key xfer; !10.0.0.1; }"},
{ScopeOptions, "allow-transfer", "{ any; }", "{ any; }"},
{ScopeOptions, "allow-query", "10.0.0.0/8 192.168.1.0/24", "{ 10.0.0.0/8; 192.168.1.0/24; }"},
{ScopeOptions, "forwarders", "1.1.1.1 9.9.9.9", "{ 1.1.1.1; 9.9.9.9; }"},
{ScopeOptions, "listen-on", "port 53 10.0.10.5", "port 53 { 10.0.10.5; }"},
{ScopeOptions, "recursion", "yes", "yes"},
{ScopeOptions, "directory", `"/var/cache/bind"`, `"/var/cache/bind"`},
} {
st, ok := c.Lookup(tc.scope, tc.name)
if !ok {
t.Fatalf("%s not found", tc.name)
}
if got := st.WrapList(tc.in); got != tc.want {
t.Errorf("%s %q = %q, want %q", tc.name, tc.in, got, tc.want)
}
}
}
+47 -14
View File
@@ -25,18 +25,39 @@ type Config struct {
TrustedProxies []string `yaml:"trusted_proxies"`
// Socket is a local unix socket for the API, where root and jarld's
// own user need no token ("" for none).
Socket string `yaml:"socket"`
Database string `yaml:"database"`
StagingDir string `yaml:"staging_dir"`
Log Log `yaml:"log"`
Session Session `yaml:"session"`
TOTP TOTP `yaml:"totp"`
BIND BIND `yaml:"bind"`
Audit Audit `yaml:"audit"`
Backup Backup `yaml:"backup"`
Stats Stats `yaml:"stats"`
ACME ACME `yaml:"acme"`
Porkbun Porkbun `yaml:"porkbun"`
Socket string `yaml:"socket"`
Database string `yaml:"database"`
StagingDir string `yaml:"staging_dir"`
Log Log `yaml:"log"`
Session Session `yaml:"session"`
TOTP TOTP `yaml:"totp"`
BIND BIND `yaml:"bind"`
Audit Audit `yaml:"audit"`
Backup Backup `yaml:"backup"`
Stats Stats `yaml:"stats"`
ACME ACME `yaml:"acme"`
Porkbun Porkbun `yaml:"porkbun"`
Blocking Blocking `yaml:"blocking"`
}
// Blocking configures blocklist downloads (docs/blocking.md). The lists
// and rules themselves are configuration in the database.
type Blocking struct {
// Refresh is how often each list is checked for changes; a list is
// only downloaded again when it changed.
Refresh Duration `yaml:"refresh"`
MaxDownloadMB int `yaml:"max_download_mb"`
MaxEntries int `yaml:"max_entries"`
}
func (b *Blocking) check() error {
if b.Refresh < Duration(time.Hour) {
return fmt.Errorf("refresh must be at least 1h (list hosts ask not to be polled more often)")
}
if b.MaxDownloadMB < 1 || b.MaxEntries < 1 {
return fmt.Errorf("max_download_mb and max_entries must be at least 1")
}
return nil
}
// Porkbun lets Jarl manage domains' name servers, glue records and DNSSEC
@@ -300,8 +321,9 @@ func Default() *Config {
Retention: StatsRetention{Minute: Duration(7 * 24 * time.Hour), Hour: Duration(90 * 24 * time.Hour), Day: Duration(730 * 24 * time.Hour)},
TopN: 100,
},
ACME: ACME{HTTPReq: HTTPReq{RequireHTTPS: true}},
Backup: Backup{Dir: "/var/backups/jarl", Retention: Duration(30 * 24 * time.Hour), KeepAtLeast: 7, WarnAfter: Duration(2 * 24 * time.Hour)},
ACME: ACME{HTTPReq: HTTPReq{RequireHTTPS: true}},
Blocking: Blocking{Refresh: Duration(24 * time.Hour), MaxDownloadMB: 64, MaxEntries: 1_000_000},
Backup: Backup{Dir: "/var/backups/jarl", Retention: Duration(30 * 24 * time.Hour), KeepAtLeast: 7, WarnAfter: Duration(2 * 24 * time.Hour)},
}
}
@@ -369,6 +391,9 @@ func (c *Config) check() error {
if err := c.Porkbun.check(); err != nil {
return fmt.Errorf("porkbun: %w", err)
}
if err := c.Blocking.check(); err != nil {
return fmt.Errorf("blocking: %w", err)
}
switch c.Log.Format {
case "text", "json":
default:
@@ -600,4 +625,12 @@ porkbun:
api_key: "" # pk1_...
secret_api_key: "" # sk1_...
domains: [] # only these domains (default: any Jarl zone the key reaches)
# Downloading blocklists (docs/blocking.md). The lists and rules are set up
# in the web UI or with 'jarl blocklist' and 'jarl block'; this only sets
# how jarld downloads them.
blocking:
refresh: 24h # check each list for changes this often (at least 1h)
max_download_mb: 64 # refuse larger downloads
max_entries: 1000000 # refuse lists with more domains
`
+299
View File
@@ -0,0 +1,299 @@
package engine
import (
"context"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strconv"
"strings"
"sync/atomic"
"testing"
"time"
"jarl/internal/blocking"
"jarl/internal/model"
)
func blockRule(name, action string, exact bool, views ...string) Request {
in := BlockRuleInput{Name: &name, Action: &action, Exact: &exact}
if len(views) > 0 {
in.Views = &views
}
return NewRequest(OpBlockRuleCreate, 0, in)
}
func blocklist(name, url string, views ...string) Request {
in := BlocklistInput{Name: &name, URL: &url}
if len(views) > 0 {
in.Views = &views
}
return NewRequest(OpBlocklistCreate, 0, in)
}
// listServer serves a blocklist with an ETag, and counts the downloads
// that sent content.
type listServer struct {
body atomic.Value
full atomic.Int32
calls atomic.Int32
}
func newListServer(t *testing.T, body string) (*listServer, *httptest.Server) {
ls := &listServer{}
ls.body.Store(body)
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
ls.calls.Add(1)
b := ls.body.Load().(string)
etag := `"` + string(rune('a'+len(b)%26)) + "x" + string(rune('a'+len(b)/26%26)) + `"`
if r.Header.Get("If-None-Match") == etag {
w.WriteHeader(http.StatusNotModified)
return
}
w.Header().Set("ETag", etag)
ls.full.Add(1)
w.Write([]byte(b))
}))
t.Cleanup(srv.Close)
return ls, srv
}
func TestBlockingRender(t *testing.T) {
env := newEnv(t)
setupZone(env)
env.submit(env.admin,
NewRequest(OpViewCreate, 0, ViewInput{Name: model.Ptr("internal"), MatchClients: &[]string{"10.0.0.0/8"}, AdoptZones: true}),
NewRequest(OpViewCreate, 0, ViewInput{Name: model.Ptr("external")}),
)
st := env.state()
ext := st.ViewByName("external")
env.submit(env.admin, NewRequest(OpOptionCreate, 0, OptionInput{Scope: model.Ptr(model.ScopeView), ScopeID: &ext.ID, Name: model.Ptr("recursion"), Value: model.Ptr("no")}))
_, srv := newListServer(t, "||ads.example^\n")
res := env.submit(env.admin,
blocklist("ads", srv.URL+"/list.txt"),
blockRule("tracker.example", "block", false),
blockRule("cdn.tracker.example", "allow", true),
)
if res.Ops[1].Summary != "block tracker.example (and below)" || res.Ops[2].Summary != "allow cdn.tracker.example" {
t.Errorf("summaries: %q, %q", res.Ops[1].Summary, res.Ops[2].Summary)
}
conf, _ := os.ReadFile(filepath.Join(env.dir, "etc/named.conf"))
c := string(conf)
// Automatic views: only internal offers recursion.
internal := c[strings.Index(c, `view "internal"`):strings.Index(c, `view "external"`)]
for _, want := range []string{
"response-policy {\n\t\tzone \"rules.jarl.invalid\" min-update-interval 0 log no;\n\t\tzone \"ads.blocklist.jarl.invalid\" min-update-interval 0 log no;\n\t} qname-wait-recurse no break-dnssec yes;",
`zone "rules.jarl.invalid" {`,
"allow-query { none; };",
} {
if !strings.Contains(internal, want) {
t.Errorf("view internal lacks %q:\n%s", want, internal)
}
}
if strings.Contains(c[strings.Index(c, `view "external"`):], "response-policy") {
t.Error("view external, without recursion, got blocking")
}
rules, err := os.ReadFile(filepath.Join(env.dir, "zones/internal/rules.jarl.invalid.rpz"))
if err != nil || !strings.Contains(string(rules), "cdn.tracker.example\tCNAME\trpz-passthru.\n*.cdn.tracker.example\tCNAME\t.\n") {
t.Errorf("rules zone: %v\n%s", err, rules)
}
// The list's file exists as an empty policy zone until downloaded.
b := env.state().BlocklistByName("ads")
listFile := filepath.Join(env.dir, "cache/blocklists/list-"+strconv.FormatInt(b.ID, 10)+".rpz")
if f, err := os.ReadFile(listFile); err != nil || !strings.Contains(string(f), "not downloaded yet") {
t.Errorf("placeholder: %v %s", err, f)
}
// A view a list or rule names can't be deleted.
res = env.submit(env.admin, NewRequest(OpBlocklistUpdate, b.ID, BlocklistInput{Views: &[]string{"external"}}))
if len(res.Problems) != 1 || res.Problems[0].Code != "blocking_no_recursion" {
t.Errorf("a list in a view without recursion: %+v", res.Problems)
}
if e := env.submitErr(env.admin, NewRequest(OpViewDelete, ext.ID, DeleteInput{Cascade: true})); e.Kind != KindConflict || !strings.Contains(e.Message, "blocklist ads") {
t.Errorf("deleting a used view: %+v", e)
}
st = env.state()
// A second rule for the same name and view is refused.
if e := env.submitErr(env.admin, blockRule("tracker.example", "allow", true)); e.Kind != KindValidation || !strings.Contains(e.Message, "two rules for tracker.example") {
t.Errorf("duplicate rule: %+v", e)
}
// So is a response-policy option where Jarl writes one.
in := st.ViewByName("internal")
if e := env.submitErr(env.admin, NewRequest(OpOptionCreate, 0, OptionInput{Scope: model.Ptr(model.ScopeView), ScopeID: &in.ID, Name: model.Ptr("response-policy"), Value: model.Ptr(`{ zone "x"; }`)})); e.Kind != KindValidation {
t.Errorf("response-policy option: %+v", e)
}
// Bad input.
for _, req := range []Request{blocklist("Bad Name", srv.URL), blocklist("ok", "ftp://x/list"), blockRule("com", "block", false), blockRule("a.example", "deny", false), blockRule("a.example", "block", false, "nope")} {
if e := env.submitErr(env.admin, req); e.Kind != KindBadRequest {
t.Errorf("%s: %+v", req.Data, e)
}
}
}
func TestBlocklistRefresh(t *testing.T) {
env := newEnv(t)
setupZone(env)
ls, srv := newListServer(t, "# test\n||ads.example^\n0.0.0.0 tracker.example\nnot a domain line\n")
env.submit(env.admin, blocklist("ads", srv.URL), blockRule("good.ads.example", "allow", false))
b := env.state().BlocklistByName("ads")
ctx := context.Background()
env.e.RefreshDue(ctx)
s, _ := env.st.BlocklistStatuses(ctx)
st := s[b.ID]
if st == nil || st.Error != "" || st.Entries != 2 || st.Records != 3 || st.Skipped != 1 || st.UpdatedAt == nil {
t.Fatalf("status after the first download: %+v", st)
}
file := env.e.cfg.Paths.BlocklistFile(b.ID)
if f, _ := os.ReadFile(file); !strings.Contains(string(f), "*.ads.example\tCNAME\t.\n") || !strings.Contains(string(f), "tracker.example\tCNAME\t.\n") {
t.Errorf("list zone:\n%s", f)
}
// Not due again: nothing is fetched.
env.e.RefreshDue(ctx)
if ls.calls.Load() != 1 {
t.Errorf("downloads after a second RefreshDue: %d", ls.calls.Load())
}
// A cleared cache leaves an empty placeholder: downloaded again at once.
os.Remove(file)
env.e.RefreshDue(ctx)
if ls.calls.Load() != 2 {
t.Errorf("downloads after the file was lost: %d", ls.calls.Load())
}
if f, _ := os.ReadFile(file); !strings.Contains(string(f), "*.ads.example\tCNAME\t.\n") {
t.Errorf("list zone after the file was lost:\n%s", f)
}
// Unchanged on the server: 304, no rewrite.
firstHash := st.SHA256
fullBefore := ls.full.Load()
env.e.RefreshBlocklist(ctx, b.ID, false)
if ls.full.Load() != fullBefore {
t.Errorf("a conditional download fetched the list again")
}
// Changed: rewritten.
ls.body.Store("||ads.example^\n||more.example^\n")
st, err := env.e.RefreshBlocklist(ctx, b.ID, false)
if err != nil || st.Entries != 2 || st.Records != 4 || st.SHA256 == firstHash {
t.Errorf("after a change: %+v, %v", st, err)
}
// A broken download keeps the previous version and reports why.
before, _ := os.ReadFile(file)
ls.body.Store("<html><body>Not found</body></html>\n")
st, err = env.e.RefreshBlocklist(ctx, b.ID, true)
if err == nil || !strings.Contains(st.Error, "no domains found") {
t.Errorf("garbage: %+v, %v", st, err)
}
if after, _ := os.ReadFile(file); string(after) != string(before) {
t.Error("a failed update changed the list's file")
}
// A failed attempt is retried after RetryAfter, not at every check.
calls := ls.calls.Load()
env.e.RefreshDue(ctx)
if ls.calls.Load() != calls {
t.Error("a failed list was tried again before RetryAfter")
}
// Too big.
env.e.Limits.MaxBytes = 10
ls.body.Store("||ads.example^\n||more.example^\n||third.example^\n")
if st, _ = env.e.RefreshBlocklist(ctx, b.ID, true); !strings.Contains(st.Error, "larger than") {
t.Errorf("size limit: %+v", st)
}
env.e.Limits = DefaultBlocklistLimits()
env.e.RefreshBlocklist(ctx, b.ID, true)
// What blocks what.
check := func(name string) blocking.Match {
m, err := env.e.CheckBlocking(env.state(), name)
if err != nil || len(m) != 1 {
t.Fatalf("check %s: %+v %v", name, m, err)
}
return m[0]
}
if m := check("x.ads.example"); m.Result != "blocked" || m.List != "ads" || m.Entry != "*.ads.example" {
t.Errorf("x.ads.example: %+v", m)
}
if m := check("a.good.ads.example"); m.Result != "allowed" || m.RuleID == 0 || m.Entry != "*.good.ads.example" {
t.Errorf("a.good.ads.example: %+v", m)
}
if m := check("fine.example"); m.Result != "none" {
t.Errorf("fine.example: %+v", m)
}
// Deleting the list removes its file and status.
env.submit(env.admin, NewRequest(OpBlocklistDelete, b.ID, nil))
env.e.RefreshDue(ctx)
if _, err := os.Stat(file); !os.IsNotExist(err) {
t.Errorf("the deleted list's file is still there: %v", err)
}
if s, _ := env.st.BlocklistStatuses(ctx); s[b.ID] != nil {
t.Error("the deleted list's status is still there")
}
}
func TestBlockingPause(t *testing.T) {
env := newEnv(t)
setupZone(env)
env.submit(env.admin, blocklist("ads", "https://example.org/list"))
rulesFile := filepath.Join(env.dir, "zones/_default/rules.jarl.invalid.rpz")
if _, err := os.Stat(rulesFile); !os.IsNotExist(err) {
t.Fatal("a rules zone without rules or a pause")
}
until := env.e.Now().Add(10 * time.Minute)
res := env.submit(env.admin, NewRequest(OpBlockingPause, 0, BlockingPauseInput{Until: &until}))
if !strings.HasPrefix(res.Ops[0].Summary, "pause blocking until ") {
t.Errorf("summary = %q", res.Ops[0].Summary)
}
f, err := os.ReadFile(rulesFile)
if err != nil || !strings.HasSuffix(string(f), "NS\tlocalhost.\n*\tCNAME\trpz-passthru.\n") {
t.Fatalf("paused rules zone: %v\n%s", err, f)
}
conf, _ := os.ReadFile(filepath.Join(env.dir, "etc/named.conf"))
if !strings.Contains(string(conf), "zone \"rules.jarl.invalid\" min-update-interval 0 log no;\n\t\tzone \"ads.") {
t.Errorf("the paused rules zone must come first:\n%s", conf)
}
// Not yet time: nothing happens.
if got := env.e.ResumeBlocking(context.Background()); !got.Equal(until) {
t.Errorf("ResumeBlocking before the end = %v", got)
}
// Time is up: resumed, and the rules zone is gone again.
env.submit(env.admin, blockRule("ads.example", "block", false))
env.e.Now = func() time.Time { return until.Add(time.Second) }
if got := env.e.ResumeBlocking(context.Background()); !got.IsZero() || env.state().Settings.BlockingPausedUntil != nil {
t.Fatalf("after the pause: %v, %v", got, env.state().Settings.BlockingPausedUntil)
}
if f, _ := os.ReadFile(rulesFile); !strings.Contains(string(f), "*.ads.example\tCNAME\t.\n") || strings.Contains(string(f), "passthru") {
t.Errorf("rules zone after the pause:\n%s", f)
}
// Reverting a pause resumes blocking.
until = env.e.Now().Add(time.Hour)
res = env.submit(env.admin, NewRequest(OpBlockingPause, 0, BlockingPauseInput{Until: &until}))
rev, err := env.e.Revert(context.Background(), res.ChangesetID, env.admin)
if err != nil {
t.Fatal(err)
}
if _, err := env.e.Apply(context.Background(), rev.ID, env.admin, ApplyOptions{}); err != nil || env.state().Settings.BlockingPausedUntil != nil {
t.Errorf("revert of a pause: %v, %v", err, env.state().Settings.BlockingPausedUntil)
}
// Until resumed: jarld never ends it, a resume does.
res = env.submit(env.admin, NewRequest(OpBlockingPause, 0, BlockingPauseInput{Indefinitely: true}))
if res.Ops[0].Summary != "pause blocking until resumed" || !env.state().Settings.BlockingPaused() {
t.Errorf("indefinite pause: %q", res.Ops[0].Summary)
}
if f, _ := os.ReadFile(rulesFile); !strings.Contains(string(f), "*\tCNAME\trpz-passthru.\n") {
t.Errorf("rules zone while paused indefinitely:\n%s", f)
}
env.e.Now = func() time.Time { return until.Add(1000 * time.Hour) }
if got := env.e.ResumeBlocking(context.Background()); !got.IsZero() || !env.state().Settings.BlockingPausedIndefinitely {
t.Errorf("an indefinite pause ended by itself: %v", got)
}
env.submit(env.admin, NewRequest(OpBlockingPause, 0, BlockingPauseInput{}))
if env.state().Settings.BlockingPaused() {
t.Error("resume didn't end an indefinite pause")
}
// At most a day.
long := env.e.Now().Add(48 * time.Hour)
if e := env.submitErr(env.admin, NewRequest(OpBlockingPause, 0, BlockingPauseInput{Until: &long})); e.Kind != KindBadRequest {
t.Errorf("a two-day pause: %+v", e)
}
}
+403
View File
@@ -0,0 +1,403 @@
package engine
import (
"context"
"errors"
"fmt"
"io"
"io/fs"
"net/http"
"os"
"path/filepath"
"slices"
"strconv"
"strings"
"time"
"jarl/internal/blocking"
"jarl/internal/model"
"jarl/internal/render"
"jarl/internal/version"
)
// BlocklistLimits bound blocklist downloads (blocking in jarl.yaml).
type BlocklistLimits struct {
Refresh time.Duration // how often a list is checked for changes
RetryAfter time.Duration // after a failed attempt
MaxBytes int64 // per download
MaxEntries int // domains per list
Timeout time.Duration // per download
}
// DefaultBlocklistLimits are the limits without configuration.
func DefaultBlocklistLimits() BlocklistLimits {
return BlocklistLimits{Refresh: 24 * time.Hour, RetryAfter: time.Hour, MaxBytes: 64 << 20, MaxEntries: 1_000_000, Timeout: 2 * time.Minute}
}
// KickBlocklists asks the download loop to look for work now (a list was
// added or changed).
func (e *Engine) KickBlocklists() {
select {
case e.blKick <- struct{}{}:
default:
}
}
// BlocklistKicks delivers KickBlocklists' signals to the download loop.
func (e *Engine) BlocklistKicks() <-chan struct{} { return e.blKick }
// ensureBlocklistFiles creates missing blocklist zone files as empty
// policy zones, so named can load its configuration before a list has
// been downloaded. Existing files are never touched.
func (e *Engine) ensureBlocklistFiles(paths []string) error {
var errs []error
for _, path := range paths {
if _, err := os.Stat(path); err == nil {
continue
}
if err := e.deploy.writeNew(path, []byte(blocking.Placeholder()), 0o640); err != nil && !errors.Is(err, fs.ErrExist) {
errs = append(errs, err)
}
}
return errors.Join(errs...)
}
// RefreshDue downloads the blocklists that are due: never tried, URL
// changed, file missing or empty (the cache was cleared), last check older
// than Refresh, or a failed attempt older than RetryAfter. Lists that are
// disabled or apply to no view are skipped. It also removes the files and
// status of deleted lists.
func (e *Engine) RefreshDue(ctx context.Context) {
st, err := e.store.LoadState(ctx)
if err != nil {
e.log.Warn("blocklists: loading the configuration failed", "err", err)
return
}
statuses, err := e.store.BlocklistStatuses(ctx)
if err != nil {
e.log.Warn("blocklists: loading their status failed", "err", err)
return
}
out := render.Render(st, e.cfg.Paths, e.extras(ctx))
if err := e.ensureBlocklistFiles(out.Blocklists); err != nil {
e.log.Warn("blocklists: creating empty zone files failed", "err", err)
}
e.cleanBlocklists(ctx, st, statuses)
now := e.Now()
for _, b := range st.SortedBlocklists() {
if b.Disabled || len(st.BlockingViews(b.ViewIDs)) == 0 {
continue
}
s := statuses[b.ID]
due := s == nil || s.URL != b.URL || s.CheckedAt == nil
if !due {
wait := e.Limits.Refresh
if s.Error != "" {
wait = e.Limits.RetryAfter
}
due = now.Sub(*s.CheckedAt) >= wait || s.Error == "" && isPlaceholder(e.cfg.Paths.BlocklistFile(b.ID))
}
if !due {
continue
}
if ctx.Err() != nil {
return
}
if _, err := e.RefreshBlocklist(ctx, b.ID, false); err != nil && ctx.Err() == nil {
e.log.Warn("blocklist update failed; named keeps using the previous version", "list", b.Name, "err", err)
}
}
}
// isPlaceholder reports whether a list's zone file is missing or still the
// empty policy zone written before its download.
func isPlaceholder(path string) bool {
f, err := os.Open(path)
if err != nil {
return true
}
defer f.Close()
head := make([]byte, len(blocking.Placeholder()))
n, _ := io.ReadFull(f, head)
return string(head[:n]) == blocking.Placeholder()
}
// cleanBlocklists removes the zone files and status of deleted lists.
func (e *Engine) cleanBlocklists(ctx context.Context, st *model.State, statuses map[int64]*model.BlocklistStatus) {
for id := range statuses {
if st.Blocklists[id] == nil {
e.store.DeleteBlocklistStatus(ctx, id)
}
}
dir := e.cfg.Paths.BlocklistsDir()
entries, err := os.ReadDir(dir)
if err != nil {
return
}
for _, ent := range entries {
n := ent.Name()
idText, ok := strings.CutPrefix(strings.TrimSuffix(n, ".rpz"), "list-")
id, err := strconv.ParseInt(idText, 10, 64)
if !ok || !strings.HasSuffix(n, ".rpz") || err != nil || st.Blocklists[id] != nil {
continue
}
if err := os.Remove(filepath.Join(dir, n)); err == nil {
e.log.Info("removed the zone file of a deleted blocklist", "file", n)
}
}
}
// errNotModified means the server said the list hasn't changed.
var errNotModified = errors.New("not modified")
// RefreshBlocklist downloads a list now and, when its content changed,
// replaces its zone file and makes named reload it. force downloads even
// when the server says nothing changed. A failure keeps the previous
// version. It returns the new status.
func (e *Engine) RefreshBlocklist(ctx context.Context, id int64, force bool) (*model.BlocklistStatus, error) {
e.blMu.Lock()
defer e.blMu.Unlock()
st, err := e.store.LoadState(ctx)
if err != nil {
return nil, err
}
b := st.Blocklists[id]
if b == nil {
return nil, errf(KindNotFound, "blocklist %d does not exist", id)
}
statuses, err := e.store.BlocklistStatuses(ctx)
if err != nil {
return nil, err
}
prev := statuses[id]
if prev == nil {
prev = &model.BlocklistStatus{ID: id}
}
file := e.cfg.Paths.BlocklistFile(id)
now := e.Now()
next := *prev
next.CheckedAt = &now
conditional := !force && prev.URL == b.URL && prev.SHA256 != "" && !isPlaceholder(file)
err = e.downloadBlocklist(ctx, st, b, &next, conditional, file)
next.URL, next.Error = b.URL, ""
if err != nil && !errors.Is(err, errNotModified) {
next.Error = err.Error()
}
if serr := e.store.SetBlocklistStatus(ctx, &next); serr != nil && err == nil {
err = serr
}
if errors.Is(err, errNotModified) {
err = nil
}
return &next, err
}
// downloadBlocklist fetches, parses, checks and installs a list, filling
// in s. It returns errNotModified when there is nothing to do.
func (e *Engine) downloadBlocklist(ctx context.Context, st *model.State, b *model.Blocklist, s *model.BlocklistStatus, conditional bool, file string) error {
ctx, cancel := context.WithTimeout(ctx, e.Limits.Timeout)
defer cancel()
req, err := http.NewRequestWithContext(ctx, http.MethodGet, b.URL, nil)
if err != nil {
return err
}
req.Header.Set("User-Agent", "Jarl/"+version.Short()+" (blocklist download)")
if conditional {
if s.ETag != "" {
req.Header.Set("If-None-Match", s.ETag)
}
if s.LastModified != "" {
req.Header.Set("If-Modified-Since", s.LastModified)
}
}
client := e.HTTP
if client == nil {
client = http.DefaultClient
}
resp, err := client.Do(req)
if err != nil {
return fmt.Errorf("download failed: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode == http.StatusNotModified && conditional {
return errNotModified
}
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("download failed: the server answered %s", resp.Status)
}
zone := blocking.ListZone(b.Name)
body := &limitedReader{r: resp.Body, left: e.Limits.MaxBytes}
l, err := blocking.Parse(body, blocking.Limits{MaxEntries: e.Limits.MaxEntries, MaxNameLen: 253 - len(zone) - 3})
if body.over {
return fmt.Errorf("the list is larger than %d MB (blocking.max_download_mb in jarl.yaml)", e.Limits.MaxBytes>>20)
}
if errors.Is(err, blocking.ErrTooManyEntries) {
return fmt.Errorf("%v (blocking.max_entries in jarl.yaml)", err)
}
if err != nil {
return fmt.Errorf("download failed: %w", err)
}
if len(l.Domains) == 0 {
return fmt.Errorf("no domains found in %d lines: is this a blocklist in hosts, domains or Adblock format?", l.Lines)
}
entries, skipped := len(l.Domains), l.Skipped
blocking.Compact(l)
dir := filepath.Dir(file)
if err := e.deploy.EnsureDir(dir, e.deploy.dirMode(dir)); err != nil {
return err
}
tmp, err := os.CreateTemp(dir, ".list-*.rpz")
if err != nil {
return err
}
tmpName := tmp.Name()
defer os.Remove(tmpName) // after a successful rename, there's nothing to remove
records, hash, err := blocking.FeedZone(tmp, fmt.Sprintf("Blocklist %s from %s, %s", b.Name, b.URL, e.Now().UTC().Format(time.RFC3339)), uint32(e.Now().Unix()), l)
if cerr := tmp.Close(); err == nil {
err = cerr
}
if err != nil {
return err
}
s.ETag, s.LastModified = resp.Header.Get("ETag"), resp.Header.Get("Last-Modified")
s.Entries, s.Skipped = entries, skipped
if hash == s.SHA256 && s.URL == b.URL && !isPlaceholder(file) {
return errNotModified // same content: named keeps what it has
}
if e.tools != nil && !e.cfg.SkipCheck {
res, err := e.tools.CheckZone(ctx, zone, tmpName)
if err != nil {
return fmt.Errorf("running named-checkzone failed: %w", err)
}
if !res.OK {
msg := "named-checkzone rejected the list's zone"
for _, m := range res.Messages {
if m.Error {
msg += ": " + m.Text
break
}
}
return errors.New(msg)
}
}
if err := e.deploy.finish(tmpName, file, 0o640); err != nil {
return err
}
now := e.Now()
s.UpdatedAt, s.Records, s.SHA256 = &now, records, hash
e.log.Info("blocklist updated", "list", b.Name, "domains", entries, "records", records, "skipped_lines", skipped)
return e.reloadBlocklist(ctx, st, b)
}
// reloadBlocklist makes named load a list's new file in every view that
// uses it.
func (e *Engine) reloadBlocklist(ctx context.Context, st *model.State, b *model.Blocklist) error {
if e.cfg.NoReload || e.tools == nil {
return nil
}
if adopted, err := e.Adopted(); err != nil || !adopted {
return nil // the file is ready for when named loads Jarl's configuration
}
for _, z := range render.PolicyZones(st, e.cfg.Paths) {
if z.ListID != b.ID {
continue
}
if err := e.tools.ReloadZone(ctx, z.Name, z.View); err != nil {
return fmt.Errorf("the list was downloaded, but named didn't reload it: %w", err)
}
}
return nil
}
// limitedReader reads at most left bytes and notes when there was more.
type limitedReader struct {
r io.Reader
left int64
over bool
}
func (l *limitedReader) Read(p []byte) (int, error) {
if l.left <= 0 {
var one [1]byte
if n, _ := l.r.Read(one[:]); n > 0 {
l.over = true
}
return 0, io.EOF
}
if int64(len(p)) > l.left {
p = p[:l.left]
}
n, err := l.r.Read(p)
l.left -= int64(n)
return n, err
}
// CheckBlocking reports, for every view with blocking, whether name is
// blocked or allowed there, and by which rule or list. It looks names up
// the way named does: rules first, then the lists in order.
func (e *Engine) CheckBlocking(st *model.State, name string) ([]blocking.Match, error) {
n, err := blocking.NormalizeName(name)
if err != nil {
return nil, errf(KindBadRequest, "%v", err)
}
var out []blocking.Match
feeds := map[string]string{} // file -> matching entry ("" none)
var cur *blocking.Match
for _, z := range render.PolicyZones(st, e.cfg.Paths) {
if cur == nil || cur.View != z.View {
out = append(out, blocking.Match{View: z.View, Result: "none"})
cur = &out[len(out)-1]
}
if cur.Result != "none" {
continue // an earlier zone decided
}
if z.ListID == 0 {
owner, act, ok := blocking.MatchRules(blocking.RuleEntries(render.ViewRules(st, z.ViewID)), n)
if ok {
cur.Entry, cur.RuleID = owner, ruleFor(st, z.ViewID, owner)
cur.Result = map[string]string{blocking.ActionBlock: "blocked", blocking.ActionAllow: "allowed"}[act]
}
continue
}
entry, seen := feeds[z.File]
if !seen {
owner, ok, err := blocking.MatchFeed(z.File, n)
if err != nil && !errors.Is(err, fs.ErrNotExist) {
return nil, err
}
if ok {
entry = owner
}
feeds[z.File] = entry
}
if entry != "" {
cur.Result, cur.Entry, cur.List = "blocked", entry, st.Blocklists[z.ListID].Name
}
}
return out, nil
}
// ruleFor finds the rule behind a rules-zone entry: the rule for the
// entry's name, or (for entries added so the most specific rule wins) the
// closest rule above it that covers the names below it.
func ruleFor(st *model.State, viewID int64, owner string) int64 {
name := strings.TrimPrefix(owner, "*.")
wild := name != owner
var best *model.BlockRule
for _, r := range st.SortedBlockRules() {
if !slices.Contains(st.BlockingViews(r.ViewIDs), viewID) {
continue
}
if r.Name == name && (!wild || !r.Exact) {
return r.ID
}
if !r.Exact && strings.HasSuffix(name, "."+r.Name) && (best == nil || len(r.Name) > len(best.Name)) {
best = r
}
}
if best != nil {
return best.ID
}
return 0
}
+312
View File
@@ -0,0 +1,312 @@
package engine
import (
"context"
"encoding/json"
"fmt"
"net/url"
"regexp"
"slices"
"strings"
"time"
"jarl/internal/auth"
"jarl/internal/blocking"
"jarl/internal/model"
)
// ---- blocking: blocklists and rules (docs/blocking.md)
var blocklistNameRe = regexp.MustCompile(`^[a-z0-9]([a-z0-9-]{0,38}[a-z0-9])?$`)
func blockingAction() auth.Action { return auth.Action{Resource: auth.ResBlocking, Write: true} }
// viewIDs resolves the views of a blocklist or rule input: names or IDs,
// in view order, without duplicates. nil, nil means "not given".
func viewIDs(st *model.State, names *[]string, ids *[]int64) ([]int64, error) {
if names == nil && ids == nil {
return nil, nil
}
want := map[int64]bool{}
if names != nil {
for _, n := range *names {
v := st.ViewByName(strings.TrimSpace(n))
if v == nil {
return nil, errf(KindBadRequest, "view %q does not exist", n)
}
want[v.ID] = true
}
}
if ids != nil {
for _, id := range *ids {
if st.Views[id] == nil {
return nil, errf(KindBadRequest, "view %d does not exist", id)
}
want[id] = true
}
}
out := []int64{}
for _, v := range st.OrderedViews() {
if want[v.ID] {
out = append(out, v.ID)
}
}
return out, nil
}
// CheckBlocklistURL checks a blocklist URL: http or https, with a host.
func CheckBlocklistURL(raw string) (string, error) {
raw = strings.TrimSpace(raw)
u, err := url.Parse(raw)
if err != nil || (u.Scheme != "http" && u.Scheme != "https") || u.Host == "" {
return "", fmt.Errorf("%q is not an http or https URL", raw)
}
return raw, nil
}
func applyBlocklistPut(st *model.State, op model.Op, c *applyCtx) error {
if err := c.require(blockingAction()); err != nil {
return err
}
var in BlocklistInput
if err := decode(op.Data, &in); err != nil {
return err
}
var b *model.Blocklist
if op.Kind == OpBlocklistCreate {
if op.ID == 0 || st.Blocklists[op.ID] != nil {
return errf(KindConflict, "blocklist ID %d is already in use", op.ID)
}
if in.Name == nil || in.URL == nil {
return errf(KindBadRequest, "a blocklist needs a name and a URL")
}
b = &model.Blocklist{ID: op.ID, ViewIDs: []int64{}}
} else {
b = st.Blocklists[op.ID]
if b == nil {
return errf(KindNotFound, "blocklist %d does not exist", op.ID)
}
if err := checkVersion(op, b.Version, "blocklist "+b.Name); err != nil {
return err
}
}
if in.Name != nil {
name := strings.ToLower(strings.TrimSpace(*in.Name))
if !blocklistNameRe.MatchString(name) {
return errf(KindBadRequest, "blocklist name %q: use 1 to 40 lowercase letters, digits and hyphens, like hagezi-pro", *in.Name)
}
if other := st.BlocklistByName(name); other != nil && other.ID != b.ID {
return errf(KindConflict, "blocklist %s already exists", name)
}
b.Name = name
}
if in.URL != nil {
u, err := CheckBlocklistURL(*in.URL)
if err != nil {
return errf(KindBadRequest, "%v", err)
}
b.URL = u
}
ids, err := viewIDs(st, in.Views, in.ViewIDs)
if err != nil {
return err
}
if ids != nil {
b.ViewIDs = ids
}
if in.Disabled != nil {
b.Disabled = *in.Disabled
}
if in.Comment != nil {
b.Comment = *in.Comment
}
b.Touch(c.user(), c.now)
st.Blocklists[b.ID] = b
return nil
}
func applyBlocklistDelete(st *model.State, op model.Op, c *applyCtx) error {
if err := c.require(blockingAction()); err != nil {
return err
}
b := st.Blocklists[op.ID]
if b == nil {
return errf(KindNotFound, "blocklist %d does not exist", op.ID)
}
if err := checkVersion(op, b.Version, "blocklist "+b.Name); err != nil {
return err
}
delete(st.Blocklists, b.ID)
return nil
}
func applyBlockRulePut(st *model.State, op model.Op, c *applyCtx) error {
if err := c.require(blockingAction()); err != nil {
return err
}
var in BlockRuleInput
if err := decode(op.Data, &in); err != nil {
return err
}
var r *model.BlockRule
if op.Kind == OpBlockRuleCreate {
if op.ID == 0 || st.BlockRules[op.ID] != nil {
return errf(KindConflict, "rule ID %d is already in use", op.ID)
}
if in.Name == nil {
return errf(KindBadRequest, "a rule needs a domain name")
}
r = &model.BlockRule{ID: op.ID, Action: model.BlockActionBlock, ViewIDs: []int64{}}
} else {
r = st.BlockRules[op.ID]
if r == nil {
return errf(KindNotFound, "rule %d does not exist", op.ID)
}
if err := checkVersion(op, r.Version, "rule "+r.Name); err != nil {
return err
}
}
if in.Name != nil {
n, err := blocking.NormalizeName(*in.Name)
if err != nil {
return errf(KindBadRequest, "%v", err)
}
r.Name = n
}
if in.Action != nil {
a := strings.ToLower(strings.TrimSpace(*in.Action))
if a != model.BlockActionBlock && a != model.BlockActionAllow {
return errf(KindBadRequest, "a rule's action is block or allow, not %q", *in.Action)
}
r.Action = a
}
if in.Exact != nil {
r.Exact = *in.Exact
}
ids, err := viewIDs(st, in.Views, in.ViewIDs)
if err != nil {
return err
}
if ids != nil {
r.ViewIDs = ids
}
if in.Comment != nil {
r.Comment = *in.Comment
}
r.Touch(c.user(), c.now)
st.BlockRules[r.ID] = r
return nil
}
func applyBlockRuleDelete(st *model.State, op model.Op, c *applyCtx) error {
if err := c.require(blockingAction()); err != nil {
return err
}
r := st.BlockRules[op.ID]
if r == nil {
return errf(KindNotFound, "rule %d does not exist", op.ID)
}
if err := checkVersion(op, r.Version, "rule "+r.Name); err != nil {
return err
}
delete(st.BlockRules, r.ID)
return nil
}
// blockingUsers lists the blocklists and rules that name a view.
func blockingUsers(st *model.State, viewID int64) []string {
var out []string
for _, b := range st.SortedBlocklists() {
if slices.Contains(b.ViewIDs, viewID) {
out = append(out, "blocklist "+b.Name)
}
}
for _, r := range st.SortedBlockRules() {
if slices.Contains(r.ViewIDs, viewID) {
out = append(out, ruleLabel(r))
}
}
return out
}
// ruleLabel describes a rule: "block ads.example (and below)".
func ruleLabel(r *model.BlockRule) string {
s := r.Action + " " + r.Name
if !r.Exact {
s += " (and below)"
}
return s
}
// ---- expansion
func (x *expander) blocklistCreate(req Request) ([]model.Op, error) {
return x.simpleCreate(req, OpBlocklistCreate, func(data json.RawMessage) (string, error) {
var in BlocklistInput
if err := decode(data, &in); err != nil {
return "", err
}
if in.Name == nil || in.URL == nil {
return "", errf(KindBadRequest, "a blocklist needs a name and a URL")
}
return "add blocklist " + *in.Name + " (" + *in.URL + ")", nil
})
}
func (x *expander) blockRuleCreate(req Request) ([]model.Op, error) {
return x.simpleCreate(req, OpBlockRuleCreate, func(data json.RawMessage) (string, error) {
var in BlockRuleInput
if err := decode(data, &in); err != nil {
return "", err
}
if in.Name == nil {
return "", errf(KindBadRequest, "a rule needs a domain name")
}
r := &model.BlockRule{Name: strings.ToLower(strings.TrimSuffix(strings.TrimSpace(*in.Name), ".")), Action: model.BlockActionBlock}
if in.Action != nil {
r.Action = strings.ToLower(*in.Action)
}
if in.Exact != nil {
r.Exact = *in.Exact
}
return ruleLabel(r), nil
})
}
func applyBlockingPause(st *model.State, op model.Op, c *applyCtx) error {
if err := c.require(blockingAction()); err != nil {
return err
}
var in BlockingPauseInput
if err := decode(op.Data, &in); err != nil {
return err
}
if in.Until != nil && in.Until.Sub(c.now) > MaxBlockingPause+time.Minute {
return errf(KindBadRequest, "blocking can be paused for at most %s, or until it is resumed", MaxBlockingPause)
}
if in.Indefinitely && in.Until != nil {
return errf(KindBadRequest, "a pause is either until a time or until resumed, not both")
}
st.Settings.BlockingPausedUntil = in.Until
st.Settings.BlockingPausedIndefinitely = in.Indefinitely
return nil
}
// ResumeBlocking ends a pause whose time is up, and returns when the
// current pause ends (zero when blocking isn't paused).
func (e *Engine) ResumeBlocking(ctx context.Context) time.Time {
st, err := e.store.LoadState(ctx)
if err != nil || st.Settings.BlockingPausedUntil == nil {
return time.Time{}
}
until := *st.Settings.BlockingPausedUntil
if e.Now().Before(until) {
return until
}
if _, err := e.Submit(ctx, []Request{NewRequest(OpBlockingPause, 0, BlockingPauseInput{})}, auth.System, "system", "resume blocking after the pause", ApplyOptions{}); err != nil {
e.log.Warn("resuming blocking after the pause failed; it will be tried again", "err", err)
return e.Now().Add(time.Minute)
}
e.log.Info("blocking resumed after the pause")
return time.Time{}
}
+45
View File
@@ -183,6 +183,48 @@ func (d *Deployer) writeAtomic(path string, content []byte, mode fs.FileMode) er
return nil
}
// writeNew creates a file that doesn't exist yet (fs.ErrExist if it
// does), with the deploy group.
func (d *Deployer) writeNew(path string, content []byte, mode fs.FileMode) error {
dir := filepath.Dir(path)
if err := d.EnsureDir(dir, d.dirMode(dir)); err != nil {
return err
}
f, err := os.OpenFile(path, os.O_WRONLY|os.O_CREATE|os.O_EXCL, mode)
if err != nil {
return err
}
_, err = f.Write(content)
if gid, gerr := d.groupID(); err == nil && gerr == nil && gid >= 0 {
err = f.Chown(-1, gid)
}
if cerr := f.Close(); err == nil {
err = cerr
}
if err != nil {
os.Remove(path)
}
return err
}
// finish gives a temporary file written next to path its mode and the
// deploy group, and moves it into place.
func (d *Deployer) finish(tmp, path string, mode fs.FileMode) error {
if err := os.Chmod(tmp, mode); err != nil {
return err
}
gid, err := d.groupID()
if err != nil {
return err
}
if gid >= 0 {
if err := os.Chown(tmp, -1, gid); err != nil {
return fmt.Errorf("chgrp %s: %w", path, err)
}
}
return os.Rename(tmp, path)
}
// Install writes files and removes paths, returning a backup that undoes
// the changes. On error the files already written are restored.
func (d *Deployer) Install(files []*render.File, remove []string, dirs []string) (*Backup, error) {
@@ -293,5 +335,8 @@ func fileDiff(path string, old []byte, existed bool, new []byte, removed, secret
if d != "" && !strings.HasSuffix(d, "\n") {
d += "\n"
}
if strings.HasSuffix(path, ".conf") {
d = labelHunks(d, string(old), string(new))
}
return d
}
+136
View File
@@ -0,0 +1,136 @@
package engine
import (
"regexp"
"strconv"
"strings"
)
// hunkHeader matches a unified diff hunk header; the line counts are left
// out when they are 1.
var hunkHeader = regexp.MustCompile(`^@@ -(\d+)(?:,\d+)? \+(\d+)(?:,\d+)? @@$`)
// labelHunks adds the blocks a hunk's first change is in to its header,
// like git does with function names, so a change to named.conf says where
// it is:
//
// @@ -50,6 +49,8 @@ view "internal" {
//
// Removed lines are looked up in the old file, added ones in the new.
func labelHunks(diff, old, new string) string {
if !strings.Contains(diff, "\n@@ ") {
return diff
}
oldCtx, newCtx := blockContext(old), blockContext(new)
lines := strings.SplitAfter(diff, "\n")
for i, line := range lines {
m := hunkHeader.FindStringSubmatch(strings.TrimSuffix(line, "\n"))
if m == nil {
continue
}
o, _ := strconv.Atoi(m[1])
n, _ := strconv.Atoi(m[2])
label := ""
hunk:
for _, l := range lines[i+1:] {
switch {
case strings.HasPrefix(l, "-"):
label = at(oldCtx, o)
break hunk
case strings.HasPrefix(l, "+"):
label = at(newCtx, n)
break hunk
case strings.HasPrefix(l, " "):
o, n = o+1, n+1
case strings.HasPrefix(l, `\`):
default:
break hunk
}
}
if label != "" {
lines[i] = strings.TrimSuffix(line, "\n") + " " + label + "\n"
}
}
return strings.Join(lines, "")
}
// at returns the context of line n (1-based), or "".
func at(ctx []string, n int) string {
if n < 1 || n > len(ctx) {
return ""
}
return ctx[n-1]
}
// blockContext returns, for each line of a named.conf file, the blocks
// open at its start: `view "internal" { zone "example.com" {`, or "" at
// the top level. It knows named.conf's strings and comments.
func blockContext(text string) []string {
var (
out []string
stack []string
stmt strings.Builder // the current statement, up to a { or ;
inStr bool
inBlock bool // in a /* */ comment
)
label := func() string {
if len(stack) == 0 {
return ""
}
return strings.Join(stack, " { ") + " {"
}
for _, line := range strings.SplitAfter(text, "\n") {
if line == "" {
continue
}
out = append(out, label())
for i := 0; i < len(line); i++ {
c := line[i]
switch {
case inBlock:
if c == '*' && i+1 < len(line) && line[i+1] == '/' {
inBlock = false
i++
}
case inStr:
stmt.WriteByte(c)
if c == '\\' && i+1 < len(line) {
stmt.WriteByte(line[i+1])
i++
} else if c == '"' {
inStr = false
}
case c == '"':
inStr = true
stmt.WriteByte(c)
case c == '#' || c == '/' && i+1 < len(line) && line[i+1] == '/':
i = len(line) // a comment to the end of the line
case c == '/' && i+1 < len(line) && line[i+1] == '*':
inBlock = true
i++
case c == '{':
stack = append(stack, shortLabel(stmt.String()))
stmt.Reset()
case c == '}':
if len(stack) > 0 {
stack = stack[:len(stack)-1]
}
stmt.Reset()
case c == ';':
stmt.Reset()
default:
stmt.WriteByte(c)
}
}
}
return out
}
// shortLabel tidies the text before a { for a hunk header.
func shortLabel(s string) string {
s = strings.Join(strings.Fields(s), " ")
if len(s) > 60 {
s = s[:57] + "..."
}
return s
}
+63
View File
@@ -0,0 +1,63 @@
package engine
import (
"strings"
"testing"
)
func TestLabelHunks(t *testing.T) {
old := `// header
options {
directory "/var/cache/bind";
forwarders { 1.1.1.1; 9.9.9.9; };
listen-on { any; };
recursion no;
version "a { b"; # a { in a string and comments: /* {
dnssec-validation auto;
};
view "internal" {
match-clients { 10.0.0.0/8; };
recursion yes;
allow-transfer { localhost; };
/* a { comment
over lines */
zone "example.com" {
type primary;
file "example.com.db";
notify yes;
};
zone "other.example" { type primary; file "o.db"; };
};
`
new := strings.Replace(old, "\tforwarders { 1.1.1.1; 9.9.9.9; };\n", "", 1)
new = strings.Replace(new, "\tallow-transfer { localhost; };\n", "\tallow-transfer { localhost; };\n\tforward first;\n\tforwarders { 1.1.1.1; 9.9.9.9; };\n", 1)
new = strings.Replace(new, "\t\tnotify yes;\n", "\t\tnotify no;\n", 1)
d := fileDiff("/etc/bind/named.conf", []byte(old), true, []byte(new), false, false)
var headers []string
for _, l := range strings.Split(d, "\n") {
if strings.HasPrefix(l, "@@") {
headers = append(headers, l)
}
}
want := []string{
`@@ -1,7 +1,6 @@ options {`,
`@@ -12,12 +11,14 @@ view "internal" {`,
}
if strings.Join(headers, "\n") != strings.Join(want, "\n") {
t.Errorf("headers:\n%s\n\ndiff:\n%s", strings.Join(headers, "\n"), d)
}
// The first change of a hunk decides, and a change inside a zone
// names both blocks.
new2 := strings.Replace(old, "\t\tnotify yes;\n", "\t\tnotify no;\n", 1)
d = fileDiff("/etc/bind/named.conf", []byte(old), true, []byte(new2), false, false)
if !strings.Contains(d, ` @@ view "internal" { zone "example.com" {`+"\n") {
t.Errorf("zone change:\n%s", d)
}
// Zone files aren't named.conf syntax.
d = fileDiff("/var/lib/bind/example.com.db", []byte(old), true, []byte(new2), false, false)
if strings.Contains(d, "@@ view") {
t.Errorf("zone file labelled:\n%s", d)
}
}
+34 -2
View File
@@ -17,6 +17,9 @@ func DiffOps(from, to *model.State) []model.Op {
add := func(kind string, id int64, base int, data any, summary string) {
ops = append(ops, newOp(kind, id, base, data, summary))
}
if !reflect.DeepEqual(from.Settings.BlockingPausedUntil, to.Settings.BlockingPausedUntil) || from.Settings.BlockingPausedIndefinitely != to.Settings.BlockingPausedIndefinitely {
add(OpBlockingPause, 0, 0, BlockingPauseInput{Until: to.Settings.BlockingPausedUntil, Indefinitely: to.Settings.BlockingPausedIndefinitely}, map[bool]string{true: "pause blocking", false: "resume blocking"}[to.Settings.BlockingPaused()])
}
if !reflect.DeepEqual(stripVersion(from.Settings), stripVersion(to.Settings)) {
s := to.Settings
add(OpSettingsUpdate, 0, 0, SettingsInput{DefaultTTL: &s.DefaultTTL, Nameservers: model.Ptr(append([]string{}, s.Nameservers...)), RName: &s.RName,
@@ -60,7 +63,7 @@ func DiffOps(from, to *model.State) []model.Op {
n := to.Zones[k]
s := n.SOA
in := ZoneInput{Name: &n.Name, ViewID: &n.ViewID, Type: &n.Type, InView: &n.InView, File: &n.File,
DefaultTTL: &n.DefaultTTL, SerialPolicy: &n.SerialPolicy, DNSSECPolicy: &n.DNSSECPolicy, Comment: &n.Comment}
DefaultTTL: &n.DefaultTTL, SerialPolicy: &n.SerialPolicy, DNSSECPolicy: &n.DNSSECPolicy, Comment: &n.Comment, Disabled: &n.Disabled}
if n.Type == model.ZonePrimary {
in.SOA = &SOAInput{MName: &s.MName, RName: &s.RName, Serial: &s.Serial, Refresh: &s.Refresh, Retry: &s.Retry, Expire: &s.Expire, Minimum: &s.Minimum}
}
@@ -88,7 +91,7 @@ func DiffOps(from, to *model.State) []model.Op {
}
for _, k := range sortedKeys(to.Records) {
n := to.Records[k]
in := RecordInput{ZoneID: n.ZoneID, Name: &n.Name, Type: &n.Type, Data: &n.Data, Comment: &n.Comment}
in := RecordInput{ZoneID: n.ZoneID, Name: &n.Name, Type: &n.Type, Data: &n.Data, Comment: &n.Comment, Disabled: &n.Disabled}
if n.TTL != nil {
in.TTL = model.Some(*n.TTL)
} else {
@@ -101,7 +104,35 @@ func DiffOps(from, to *model.State) []model.Op {
add(OpRecordUpdate, k, o.Version, in, fmt.Sprintf("restore %s %s %s", n.Name, n.Type, n.Data))
}
}
for _, k := range sortedKeys(to.Blocklists) {
n := to.Blocklists[k]
in := BlocklistInput{Name: &n.Name, URL: &n.URL, ViewIDs: model.Ptr(append([]int64{}, n.ViewIDs...)), Disabled: &n.Disabled, Comment: &n.Comment}
if o, ok := from.Blocklists[k]; !ok {
add(OpBlocklistCreate, k, 0, in, "add blocklist "+n.Name)
} else if !sameObject(o, n) {
add(OpBlocklistUpdate, k, o.Version, in, "restore blocklist "+n.Name)
}
}
for _, k := range sortedKeys(to.BlockRules) {
n := to.BlockRules[k]
in := BlockRuleInput{Name: &n.Name, Action: &n.Action, Exact: &n.Exact, ViewIDs: model.Ptr(append([]int64{}, n.ViewIDs...)), Comment: &n.Comment}
if o, ok := from.BlockRules[k]; !ok {
add(OpBlockRuleCreate, k, 0, in, "add rule: "+ruleLabel(n))
} else if !sameObject(o, n) {
add(OpBlockRuleUpdate, k, o.Version, in, "restore rule: "+ruleLabel(n))
}
}
// Deletions, most dependent objects first.
for _, k := range sortedKeys(from.BlockRules) {
if _, ok := to.BlockRules[k]; !ok {
add(OpBlockRuleDelete, k, from.BlockRules[k].Version, nil, "delete rule: "+ruleLabel(from.BlockRules[k]))
}
}
for _, k := range sortedKeys(from.Blocklists) {
if _, ok := to.Blocklists[k]; !ok {
add(OpBlocklistDelete, k, from.Blocklists[k].Version, nil, "delete blocklist "+from.Blocklists[k].Name)
}
}
for _, k := range sortedKeys(from.Records) {
if _, ok := to.Records[k]; !ok {
o := from.Records[k]
@@ -156,6 +187,7 @@ func DiffOps(from, to *model.State) []model.Op {
func stripVersion(s model.Settings) model.Settings {
s.Version = 0
s.BlockingPausedUntil, s.BlockingPausedIndefinitely = nil, false // their own operation
return s
}
+176
View File
@@ -0,0 +1,176 @@
package engine
import (
"os"
"path/filepath"
"strings"
"testing"
"jarl/internal/model"
)
func disableReq(id int64, ptr bool) Request {
return NewRequest(OpRecordDisable, id, RecordInput{PTR: ptr})
}
func enableReq(id int64, ptr bool) Request {
return NewRequest(OpRecordEnable, id, RecordInput{PTR: ptr})
}
func (env *testEnv) zoneFile(name string) string {
env.t.Helper()
b, err := os.ReadFile(filepath.Join(env.dir, "zones/_default", name+".zone"))
if err != nil {
env.t.Fatal(err)
}
return string(b)
}
func serialOf(env *testEnv, zone string) uint32 {
return env.state().ZonesNamed(zone)[0].SOA.Serial
}
func TestDisableRecord(t *testing.T) {
env := newEnv(t)
setupZone(env)
env.submit(env.admin, rec("example.com", "www", "A", "192.0.2.10"))
www := findRec(env.state(), "www.example.com.", "A")
serial := serialOf(env, "example.com")
res := env.submit(env.admin, disableReq(www.ID, false))
if len(res.Ops) != 1 || res.Ops[0].Summary != "disable www.example.com. A 192.0.2.10" {
t.Fatalf("ops = %+v", res.Ops)
}
if r := env.state().Records[www.ID]; r == nil || !r.Disabled {
t.Fatalf("record = %+v", r)
}
zf := env.zoneFile("example.com")
if !strings.Contains(zf, "; disabled: www\t\tIN\tA\t192.0.2.10\n") || strings.Contains(zf, "\nwww\t") {
t.Errorf("zone file:\n%s", zf)
}
if s := serialOf(env, "example.com"); s != serial+1 {
t.Errorf("serial after disable = %d, want %d", s, serial+1)
}
// Editing a disabled record changes nothing that is served.
env.submit(env.admin, NewRequest(OpRecordUpdate, www.ID, RecordInput{Comment: model.Ptr("old web server"), Data: model.Ptr("192.0.2.11")}))
if s := serialOf(env, "example.com"); s != serial+1 {
t.Errorf("serial after editing a disabled record = %d", s)
}
// Disabling it again says so.
if e := env.submitErr(env.admin, disableReq(www.ID, false)); e.Kind != KindConflict || !strings.Contains(e.Message, "already disabled") {
t.Errorf("second disable: %+v", e)
}
env.submit(env.admin, enableReq(www.ID, false))
r := env.state().Records[www.ID]
if r.Disabled || r.Data != "192.0.2.11" || r.Comment != "old web server" {
t.Errorf("enabled record = %+v", r)
}
if zf := env.zoneFile("example.com"); !strings.Contains(zf, "\nwww\t\tIN\tA\t192.0.2.11\t; old web server\n") || strings.Contains(zf, "disabled") {
t.Errorf("zone file after enable:\n%s", zf)
}
}
func TestDisableValidation(t *testing.T) {
env := newEnv(t)
setupZone(env)
// The zone must keep a name server.
ns := findRec(env.state(), "example.com.", "NS")
if e := env.submitErr(env.admin, disableReq(ns.ID, false)); e.Kind != KindValidation {
t.Errorf("disabling the only NS: %+v", e)
}
// A disabled record doesn't count for the CNAME rule...
env.submit(env.admin, rec("example.com", "www", "A", "192.0.2.10"))
www := findRec(env.state(), "www.example.com.", "A")
env.submit(env.admin, disableReq(www.ID, false))
res := env.submit(env.admin, rec("example.com", "www", "CNAME", "ns1"))
// ...but there's a warning that it can't come back,
if len(res.Problems) != 1 || res.Problems[0].Code != "disabled_conflict" || res.Problems[0].ObjectID != www.ID {
t.Errorf("problems = %+v", res.Problems)
}
// and enabling it is checked like adding it.
if e := env.submitErr(env.admin, enableReq(www.ID, false)); e.Kind != KindValidation || !strings.Contains(e.Message, "CNAME") {
t.Errorf("enabling next to a CNAME: %+v", e)
}
}
func TestDisableByMatchAndPTR(t *testing.T) {
env := newEnv(t)
setupZone(env)
env.submit(env.admin, NewRequest(OpZoneCreate, 0, ZoneInput{Name: model.Ptr("192.0.2.0/24")}))
env.submit(env.admin, NewRequest(OpRecordCreate, 0, RecordInput{Zone: "example.com", Name: model.Ptr("nas"), Type: model.Ptr("A"), Data: model.Ptr("192.0.2.20"), PTR: true}))
ptrDisabled := func() bool {
return findRec(env.state(), "20.2.0.192.in-addr.arpa.", "PTR").Disabled
}
// By match, with the PTR record.
match := RecordInput{Zone: "example.com", Name: model.Ptr("nas"), Type: model.Ptr("A"), PTR: true}
res := env.submit(env.admin, NewRequest(OpRecordDisable, 0, match))
if len(res.Ops) != 2 || res.Ops[1].Summary != "disable 20.2.0.192.in-addr.arpa. PTR nas.example.com." {
t.Fatalf("ops = %+v", res.Ops)
}
if !findRec(env.state(), "nas.example.com.", "A").Disabled || !ptrDisabled() {
t.Fatal("A or PTR not disabled")
}
env.submit(env.admin, NewRequest(OpRecordEnable, 0, match))
if findRec(env.state(), "nas.example.com.", "A").Disabled || ptrDisabled() {
t.Fatal("A or PTR not enabled")
}
// Without ptr, the PTR record stays as it is.
a := findRec(env.state(), "nas.example.com.", "A")
env.submit(env.admin, disableReq(a.ID, false))
if ptrDisabled() {
t.Error("PTR disabled without ptr")
}
// Data narrows the match; nothing matching is an error unless missing_ok.
env.submit(env.admin, rec("example.com", "api", "A", "192.0.2.30"), rec("example.com", "api", "A", "192.0.2.31"))
one := RecordInput{Zone: "example.com", Name: model.Ptr("api"), Type: model.Ptr("A"), Data: model.Ptr("192.0.2.31")}
env.submit(env.admin, NewRequest(OpRecordDisable, 0, one))
if findRecData(env.state(), "api.example.com.", "192.0.2.30").Disabled || !findRecData(env.state(), "api.example.com.", "192.0.2.31").Disabled {
t.Error("data match wrong")
}
none := RecordInput{Zone: "example.com", Name: model.Ptr("nope"), Type: model.Ptr("A")}
if e := env.submitErr(env.admin, NewRequest(OpRecordDisable, 0, none)); e.Kind != KindNotFound {
t.Errorf("no match: %+v", e)
}
// missing_ok: nothing to do (an error the CLI's --missing-ok accepts).
none.MissingOK = true
if e := env.submitErr(env.admin, NewRequest(OpRecordDisable, 0, none)); e.Message != "nothing to do" {
t.Errorf("missing_ok: %+v", e)
}
}
func TestDisableRRSetAndRevert(t *testing.T) {
env := newEnv(t)
setupZone(env)
env.submit(env.admin, rec("example.com", "api", "A", "192.0.2.30"), rec("example.com", "api", "A", "192.0.2.31"))
r30 := findRecData(env.state(), "api.example.com.", "192.0.2.30")
r31 := findRecData(env.state(), "api.example.com.", "192.0.2.31")
env.submit(env.admin, disableReq(r30.ID, false), disableReq(r31.ID, false))
// Setting the RRset enables a disabled record with a given value and
// leaves the other disabled one alone.
env.submit(env.admin, NewRequest(OpRRSetReplace, 0, RRSetInput{Zone: "example.com", Name: "api", Type: "A", Data: []string{"192.0.2.30", "192.0.2.40"}}))
s := env.state()
if r := s.Records[r30.ID]; r == nil || r.Disabled {
t.Errorf(".30 = %+v, want kept and enabled", r)
}
if r := s.Records[r31.ID]; r == nil || !r.Disabled {
t.Errorf(".31 = %+v, want kept and disabled", r)
}
// Reverting a disabling changeset enables the record again.
res := env.submit(env.admin, disableReq(r30.ID, false))
rev, err := env.e.Revert(env.ctx, res.ChangesetID, env.admin)
if err != nil {
t.Fatal(err)
}
if _, err := env.e.Apply(env.ctx, rev.ID, env.admin, ApplyOptions{}); err != nil {
t.Fatal(err)
}
if env.state().Records[r30.ID].Disabled {
t.Error("revert did not enable the record")
}
}
+93 -9
View File
@@ -14,8 +14,10 @@ import (
"fmt"
"io/fs"
"log/slog"
"net/http"
"os"
"path/filepath"
"slices"
"sort"
"strings"
"sync"
@@ -23,6 +25,7 @@ import (
"jarl/internal/auth"
"jarl/internal/bind"
"jarl/internal/blocking"
"jarl/internal/dnsname"
"jarl/internal/model"
"jarl/internal/namedconf"
@@ -64,6 +67,17 @@ type Engine struct {
deploy *Deployer
mu sync.Mutex // serializes applies
Now func() time.Time
// Blocklist downloads (blocklists.go).
blMu sync.Mutex // serializes downloads
blKick chan struct{} // a list was added or changed
HTTP *http.Client // for downloads; nil: a default client
Limits BlocklistLimits
namedCheck namedCheck // named-checkconf on the running configuration (namedwarn.go)
seedMu sync.Mutex
seed []byte // the view keys' seed (viewkeys.go)
}
// New creates an engine.
@@ -78,6 +92,8 @@ func New(st *store.Store, tools *bind.Tools, cfg Config, log *slog.Logger) *Engi
store: st, tools: tools, cfg: cfg, log: log,
deploy: &Deployer{Group: cfg.FileGroup, NamedWritable: []string{cfg.Paths.ZoneDir, cfg.Paths.CacheDir}},
Now: time.Now,
blKick: make(chan struct{}, 1),
Limits: DefaultBlocklistLimits(),
}
}
@@ -383,6 +399,7 @@ type plan struct {
drift []DriftEntry
configChanged bool
reloadZones []*model.Zone
reloadPolicy []render.PolicyZone // rules zones whose file changed
verify map[int64]uint32
newZones bool
backup *Backup
@@ -423,7 +440,7 @@ func (e *Engine) makePlan(ctx context.Context, base, work *model.State, user str
pl.introduced = append(pl.introduced, problems...)
return pl, nil
}
pl.out = render.Render(work, e.cfg.Paths, e.cfg.Extras)
pl.out = render.Render(work, e.cfg.Paths, e.extras(ctx))
deployed, err := e.store.DeployedFiles(ctx)
if err != nil {
return nil, err
@@ -446,7 +463,7 @@ func (e *Engine) makePlan(ctx context.Context, base, work *model.State, user str
}
}
for path, df := range deployed {
if pl.out.File(path) != nil || df.Kind != render.KindZone {
if pl.out.File(path) != nil || df.Kind != render.KindZone && df.Kind != render.KindPolicy {
continue
}
old, existed, _ := readFile(path)
@@ -459,18 +476,18 @@ func (e *Engine) makePlan(ctx context.Context, base, work *model.State, user str
pl.diff = diff.String()
// Which zones named must reload, and which serials to verify.
baseOut := render.Render(base, e.cfg.Paths, e.cfg.Extras)
baseOut := render.Render(base, e.cfg.Paths, e.extras(ctx))
changedPaths := map[string]bool{}
for _, f := range pl.changed {
changedPaths[f.Path] = true
}
for _, z := range work.SortedZones() {
if z.Type != model.ZonePrimary {
continue
if z.Type != model.ZonePrimary || z.Disabled {
continue // disabled zones aren't in named: nothing to reload
}
path := pl.out.ZoneFiles[z.ID]
old := base.Zones[z.ID]
existedInNamed := old != nil && old.Type == model.ZonePrimary && baseOut.ZoneFiles[old.ID] == path
existedInNamed := old != nil && old.Type == model.ZonePrimary && !old.Disabled && baseOut.ZoneFiles[old.ID] == path
if !existedInNamed {
pl.newZones = true
pl.verify[z.ID] = z.SOA.Serial
@@ -484,6 +501,16 @@ func (e *Engine) makePlan(ctx context.Context, base, work *model.State, user str
if len(work.Zones) != len(base.Zones) {
pl.configChanged = true
}
// Rules zones whose content changed. New ones come with a change to
// named.conf, and reconfig loads them.
for _, z := range pl.out.Policy {
if z.ListID != 0 || !changedPaths[z.File] {
continue
}
if slices.Contains(baseOut.Policy, z) {
pl.reloadPolicy = append(pl.reloadPolicy, z)
}
}
if !e.cfg.SkipCheck {
if err := e.stagingCheck(ctx, pl); err != nil {
@@ -505,7 +532,7 @@ func (e *Engine) stagingCheck(ctx context.Context, pl *plan) error {
}
defer os.RemoveAll(dir)
sp := render.Paths{ConfigDir: filepath.Join(dir, "config"), ZoneDir: filepath.Join(dir, "zones"), CacheDir: filepath.Join(dir, "cache")}
sout := render.Render(pl.work, sp, e.cfg.Extras)
sout := render.Render(pl.work, sp, e.extras(ctx))
for _, d := range sout.Dirs {
if err := os.MkdirAll(d, 0o750); err != nil {
return err
@@ -519,6 +546,16 @@ func (e *Engine) stagingCheck(ctx context.Context, pl *plan) error {
return err
}
}
// Blocklists as empty policy zones: their content is checked when
// they are downloaded.
for _, path := range sout.Blocklists {
if err := os.MkdirAll(filepath.Dir(path), 0o750); err != nil {
return err
}
if err := os.WriteFile(path, []byte(blocking.Placeholder()), 0o640); err != nil {
return err
}
}
main, err := e.stagingMain(dir, sp.MainConfig())
if err != nil {
return err
@@ -796,6 +833,12 @@ func (e *Engine) applyOps(ctx context.Context, cs *model.Changeset, reqs []Reque
res.ChangesetID = cs.ID
res.Applied = true
e.removeDerived(pl)
for _, op := range cs.Ops {
if strings.HasPrefix(op.Kind, "blocklist.") || op.Kind == OpBlockingPause {
e.KickBlocklists()
break
}
}
e.log.Info("changeset applied", "changeset", cs.ID, "user", p.Name(), "ops", len(cs.Ops), "files", len(res.Files), "description", cs.Description)
return res, nil
}
@@ -829,6 +872,9 @@ func (e *Engine) deployAndReload(ctx context.Context, pl *plan, res *Result, rel
return &Error{Kind: KindBIND, Message: "writing configuration files failed (nothing was changed): " + err.Error()}
}
pl.backup = backup
if err := e.ensureBlocklistFiles(pl.out.Blocklists); err != nil {
e.log.Warn("creating a blocklist's empty zone file failed", "err", err)
}
pl.reloaded = reload
if !reload {
return nil
@@ -860,6 +906,11 @@ func (e *Engine) rollback(ctx context.Context, pl *plan) {
e.log.Error("rndc reload after restore failed", "zone", z.Name, "err", err)
}
}
for _, z := range pl.reloadPolicy {
if err := e.tools.ReloadZone(ctx, z.Name, z.View); err != nil {
e.log.Error("rndc reload after restore failed", "zone", z.Name, "err", err)
}
}
}
// namedView returns the view name to pass to rndc for a zone ("" without
@@ -885,6 +936,16 @@ func (e *Engine) reload(ctx context.Context, pl *plan) ([]string, error) {
}
done = append(done, "reload "+zoneDesc(pl.work, z))
}
for _, z := range pl.reloadPolicy {
label := "blocking rules"
if z.View != "" {
label += " [view " + z.View + "]"
}
if err := e.tools.ReloadZone(ctx, z.Name, z.View); err != nil {
return done, fmt.Errorf("reloading %s failed: %v", label, err)
}
done = append(done, "reload "+label)
}
// named reloads zones asynchronously and reports load errors only in
// its log, so wait until it serves the new serials.
deadline := time.Now().Add(e.cfg.ReloadTimeout)
@@ -998,11 +1059,17 @@ func (e *Engine) Resync(ctx context.Context, p *auth.Principal, opts ApplyOption
pl.configChanged = true
pl.reloadZones = nil
for _, z := range work.SortedZones() {
if z.Type == model.ZonePrimary {
if z.Type == model.ZonePrimary && !z.Disabled {
pl.reloadZones = append(pl.reloadZones, z)
pl.verify[z.ID] = z.SOA.Serial
}
}
pl.reloadPolicy = nil
for _, z := range pl.out.Policy {
if z.ListID == 0 {
pl.reloadPolicy = append(pl.reloadPolicy, z)
}
}
reload := !e.cfg.NoReload && !opts.NoReload
if adopted, aerr := e.Adopted(); reload && (aerr != nil || !adopted) {
reload = false
@@ -1113,6 +1180,14 @@ type Status struct {
MainConfig string `json:"main_config"`
}
// NamedStatus asks named for its status (rndc status).
func (e *Engine) NamedStatus(ctx context.Context) (*bind.ServerStatus, error) {
if e.cfg.NoReload || e.tools == nil {
return nil, errors.New("named is not managed by this jarld (no_reload)")
}
return e.tools.Status(ctx)
}
// Status returns the current status.
func (e *Engine) Status(ctx context.Context) (*Status, error) {
st, err := e.store.LoadState(ctx)
@@ -1156,7 +1231,7 @@ func (e *Engine) Status(ctx context.Context) (*Status, error) {
}
}
}
s.Problems = nonNilProblems(e.validate(st))
s.Problems = nonNilProblems(append(e.validate(st), e.namedProblems(ctx, st, deployed)...))
s.Counts = map[string]int{"views": len(st.Views), "zones": len(st.Zones), "records": len(st.Records), "acls": len(st.ACLs), "tsig_keys": len(st.Keys), "options": len(st.Options)}
return s, nil
}
@@ -1222,6 +1297,15 @@ func (e *Engine) WorkingState(ctx context.Context, csID int64, p *auth.Principal
// secondaries and the rest of the world see), not that of the zone file
// Jarl wrote. ok is false when named isn't asked (no_reload) or doesn't
// report a serial (the zone isn't loaded yet).
// LoadedSerialCommand is the command LoadedSerial runs, or "" when it
// doesn't ask named.
func (e *Engine) LoadedSerialCommand(st *model.State, z *model.Zone) string {
if e.cfg.NoReload || e.tools == nil {
return ""
}
return bind.ZoneCommand("zonestatus", z.Name, e.namedView(st, z))
}
func (e *Engine) LoadedSerial(ctx context.Context, st *model.State, z *model.Zone) (serial uint32, ok bool, err error) {
if e.cfg.NoReload || e.tools == nil {
return 0, false, nil
+209 -6
View File
@@ -145,6 +145,12 @@ func (x *expander) expand(req Request) ([]model.Op, error) {
if n := len(x.st.ZoneRecords(z.ID)); n > 0 {
verb += fmt.Sprintf(" (with its %d records)", n)
}
} else if d := zoneToggle(req, z); d != nil {
if *d {
x.warnings = append(x.warnings, disabledZoneEffect(x.st, z))
return []model.Op{x.versioned(req, z.Version, "disable zone "+zoneDesc(x.st, z))}, nil
}
return []model.Op{x.versioned(req, z.Version, "enable zone "+zoneDesc(x.st, z))}, nil
}
return []model.Op{x.versioned(req, z.Version, "zone "+verb+" "+zoneDesc(x.st, z))}, nil
case OpRecordCreate:
@@ -155,6 +161,8 @@ func (x *expander) expand(req Request) ([]model.Op, error) {
return x.recordDelete(req)
case OpRRSetReplace:
return x.rrsetReplace(req)
case OpRecordDisable, OpRecordEnable:
return x.recordSetDisabled(req, req.Op == OpRecordDisable)
case OpViewCreate:
return x.viewCreate(req)
case OpViewUpdate, OpViewDelete:
@@ -220,6 +228,42 @@ func (x *expander) expand(req Request) ([]model.Op, error) {
return []model.Op{x.versioned(req, o.Version, summary)}, nil
case OpSettingsUpdate:
return []model.Op{newOpRaw(OpSettingsUpdate, 0, 0, req.Data, "update settings")}, nil
case OpBlocklistCreate:
return x.blocklistCreate(req)
case OpBlocklistUpdate, OpBlocklistDelete:
b := x.st.Blocklists[req.ID]
if b == nil {
return nil, errf(KindNotFound, "blocklist %d does not exist", req.ID)
}
verb := "change"
if req.Op == OpBlocklistDelete {
verb = "delete"
}
return []model.Op{x.versioned(req, b.Version, verb+" blocklist "+b.Name)}, nil
case OpBlockingPause:
var in BlockingPauseInput
if err := decode(req.Data, &in); err != nil {
return nil, err
}
summary := "resume blocking"
if in.Until != nil {
summary = "pause blocking until " + in.Until.Local().Format("2006-01-02 15:04")
} else if in.Indefinitely {
summary = "pause blocking until resumed"
}
return []model.Op{newOpRaw(OpBlockingPause, 0, 0, req.Data, summary)}, nil
case OpBlockRuleCreate:
return x.blockRuleCreate(req)
case OpBlockRuleUpdate, OpBlockRuleDelete:
r := x.st.BlockRules[req.ID]
if r == nil {
return nil, errf(KindNotFound, "rule %d does not exist", req.ID)
}
verb := "change rule: "
if req.Op == OpBlockRuleDelete {
verb = "delete rule: "
}
return []model.Op{x.versioned(req, r.Version, verb+ruleLabel(r))}, nil
}
return nil, errf(KindBadRequest, "unknown operation %q", req.Op)
}
@@ -413,6 +457,9 @@ func (x *expander) zoneCreate(req Request) ([]model.Op, error) {
if src != nil {
for _, r := range x.st.ZoneRecords(src.ID) {
ri := RecordInput{Name: model.Ptr(r.Name), Type: model.Ptr(r.Type), Data: model.Ptr(r.Data), Comment: model.Ptr(r.Comment)}
if r.Disabled {
ri.Disabled = model.Ptr(true)
}
if r.TTL != nil {
ri.TTL = model.Some(*r.TTL)
} else {
@@ -502,6 +549,10 @@ func (x *expander) ptrCreate(z *model.Zone, owner string, p *rr.Parsed) ([]model
for _, r := range x.st.ZoneRecords(rz.ID) {
if r.Name == ptrName && r.Type == "PTR" {
if strings.EqualFold(r.Data, owner) {
if r.Disabled {
in := RecordInput{Disabled: model.Ptr(false)}
return []model.Op{newOp(OpRecordUpdate, r.ID, r.Version, in, fmt.Sprintf("enable %s PTR %s", r.Name, r.Data))}, nil
}
return nil, nil // already there
}
x.warnings = append(x.warnings, fmt.Sprintf("%s already has a PTR record (%s); adding another one", ptrName, r.Data))
@@ -515,6 +566,34 @@ func (x *expander) ptrCreate(z *model.Zone, owner string, p *rr.Parsed) ([]model
return []model.Op{newOp(OpRecordCreate, ids[0], 0, in, fmt.Sprintf("add %s IN PTR %s", ptrName, owner))}, nil
}
// ptrSetDisabled returns operations disabling or enabling the PTR record
// of an address record.
func (x *expander) ptrSetDisabled(z *model.Zone, owner string, p *rr.Parsed, disabled bool) []model.Op {
addr, ok := addrOf(p)
if !ok {
return nil
}
rz, ptrName := reverseZoneFor(x.st, z.ViewID, addr)
if rz == nil {
return nil
}
var ops []model.Op
for _, r := range x.st.ZoneRecords(rz.ID) {
if r.Name == ptrName && r.Type == "PTR" && strings.EqualFold(r.Data, owner) && r.Disabled != disabled {
in := RecordInput{Disabled: model.Ptr(disabled)}
ops = append(ops, newOp(OpRecordUpdate, r.ID, r.Version, in, fmt.Sprintf("%s %s PTR %s", enableWord(disabled), r.Name, r.Data)))
}
}
return ops
}
func enableWord(disabled bool) string {
if disabled {
return "disable"
}
return "enable"
}
func (x *expander) ptrDelete(z *model.Zone, owner string, p *rr.Parsed) []model.Op {
addr, ok := addrOf(p)
if !ok {
@@ -635,8 +714,15 @@ func (x *expander) recordUpdate(req Request) ([]model.Op, error) {
if in.Comment != nil && *in.Comment != r.Comment {
changes = append(changes, "comment")
}
toggled := in.Disabled != nil && *in.Disabled != r.Disabled
summary := fmt.Sprintf("change %s %s %s", r.Name, r.Type, r.Data)
if len(changes) > 0 {
switch {
case toggled && len(changes) == 0:
summary = fmt.Sprintf("%s %s %s %s", enableWord(*in.Disabled), r.Name, r.Type, r.Data)
case toggled:
changes = append(changes, enableWord(*in.Disabled)+"d")
fallthrough
case len(changes) > 0:
summary += ": " + strings.Join(changes, ", ")
}
base := req.BaseVersion
@@ -644,15 +730,28 @@ func (x *expander) recordUpdate(req Request) ([]model.Op, error) {
base = r.Version
}
ops := []model.Op{newOp(OpRecordUpdate, r.ID, base, in, summary)}
if ptr && (p.Data != r.Data || owner != r.Name) {
disabled := r.Disabled
if in.Disabled != nil {
disabled = *in.Disabled
}
moved := p.Data != r.Data || owner != r.Name
if ptr && moved {
if old, err := rr.Parse(z.Name, r.Name, r.Type, r.Data); err == nil {
ops = append(ops, x.ptrDelete(z, r.Name, old)...)
}
more, err := x.ptrCreate(z, owner, p)
if err != nil {
return nil, err
if !disabled {
more, err := x.ptrCreate(z, owner, p)
if err != nil {
return nil, err
}
ops = append(ops, more...)
}
ops = append(ops, more...)
}
if ptr && !moved && toggled {
if _, ok := addrOf(p); !ok {
return nil, errf(KindBadRequest, "ptr only applies to A and AAAA records")
}
ops = append(ops, x.ptrSetDisabled(z, owner, p, disabled)...)
}
return ops, nil
}
@@ -725,6 +824,82 @@ func (x *expander) recordDelete(req Request) ([]model.Op, error) {
return ops, nil
}
// recordSetDisabled expands record.disable and record.enable: the record
// with the request's ID, or the records matching zone, name, type and
// (optionally) data.
func (x *expander) recordSetDisabled(req Request, disabled bool) ([]model.Op, error) {
var in RecordInput
if err := decode(req.Data, &in); err != nil {
return nil, err
}
var recs []*model.Record
if req.ID != 0 {
r := x.st.Records[req.ID]
if r == nil {
return nil, errf(KindNotFound, "record %d does not exist", req.ID)
}
recs = append(recs, r)
} else {
z, err := x.resolveZone(in.ZoneID, in.Zone, in.View)
if err != nil {
return nil, err
}
if in.Name == nil || in.Type == nil {
return nil, errf(KindBadRequest, "finding records needs a name and a type (or give the record id)")
}
owner, err := dnsname.Owner(*in.Name, z.Name)
if err != nil {
return nil, errf(KindBadRequest, "%v", err)
}
typ, _, err := rr.NormalizeType(*in.Type)
if err != nil {
return nil, errf(KindBadRequest, "%v", err)
}
data, what := "", owner+" "+typ
if in.Data != nil && strings.TrimSpace(*in.Data) != "" {
_, p, err := parseRecord(z, owner, typ, *in.Data)
if err != nil {
return nil, err
}
data, what = p.Data, what+" "+p.Data
}
for _, r := range x.st.ZoneRecords(z.ID) {
if r.Name == owner && r.Type == typ && (data == "" || strings.EqualFold(r.Data, data)) {
recs = append(recs, r)
}
}
if len(recs) == 0 {
if in.MissingOK {
return nil, nil
}
return nil, errf(KindNotFound, "no record %s in zone %s", dnsname.Display(what), dnsname.Display(z.Name))
}
}
var ops []model.Op
for _, r := range recs {
if r.Disabled == disabled {
continue // already so
}
base := 0
if req.ID != 0 {
base = req.BaseVersion
}
more, err := x.recordUpdate(Request{Op: OpRecordUpdate, ID: r.ID, BaseVersion: base, Data: mustJSON(RecordInput{Disabled: model.Ptr(disabled), PTR: in.PTR})})
if err != nil {
return nil, err
}
ops = append(ops, more...)
}
if len(ops) == 0 && !in.MissingOK {
what := fmt.Sprintf("%s %s %s is", recs[0].Name, recs[0].Type, recs[0].Data)
if len(recs) > 1 {
what = fmt.Sprintf("all %d %s %s records are", len(recs), recs[0].Name, recs[0].Type)
}
return nil, errf(KindConflict, "%s already %sd", what, enableWord(disabled))
}
return ops, nil
}
func (x *expander) rrsetReplace(req Request) ([]model.Op, error) {
var in RRSetInput
if err := decode(req.Data, &in); err != nil {
@@ -862,3 +1037,31 @@ func RedactOps(ops []model.Op) []model.Op {
}
return out
}
// zoneToggle returns the new disabled state when a zone update only
// disables or enables the zone (nil otherwise).
func zoneToggle(req Request, z *model.Zone) *bool {
var in ZoneInput
if decode(req.Data, &in) != nil || in.Disabled == nil || *in.Disabled == z.Disabled {
return nil
}
only := ZoneInput{Disabled: in.Disabled}
if string(mustJSON(in)) != string(mustJSON(only)) {
return nil
}
return in.Disabled
}
// disabledZoneEffect says what clients get once named no longer loads a
// zone: in a view that offers recursion, answers from recursion (the
// internet's version of the zone, if it has one); elsewhere, REFUSED.
func disabledZoneEffect(st *model.State, z *model.Zone) string {
where := "this server"
if v := st.ViewName(z); v != "" {
where = "view " + v
}
if st.RecursionOn(z.ViewID) {
return fmt.Sprintf("once zone %s is disabled, clients of %s that may use recursion get its names from recursion: the internet's version of the zone, if there is one", dnsname.Display(z.Name), where)
}
return fmt.Sprintf("once zone %s is disabled, %s answers REFUSED for its names", dnsname.Display(z.Name), where)
}
+57 -3
View File
@@ -2,6 +2,7 @@ package engine
import (
"encoding/json"
"time"
"jarl/internal/model"
)
@@ -29,12 +30,27 @@ const (
OpOptionUpdate = "option.update"
OpOptionDelete = "option.delete"
OpSettingsUpdate = "settings.update"
OpBlocklistCreate = "blocklist.create"
OpBlocklistUpdate = "blocklist.update"
OpBlocklistDelete = "blocklist.delete"
OpBlockRuleCreate = "blockrule.create"
OpBlockRuleUpdate = "blockrule.update"
OpBlockRuleDelete = "blockrule.delete"
OpBlockingPause = "blocking.pause"
// Requests only: disabling and enabling records, by ID or by match
// (zone, name, type and optionally data), become record.update
// operations.
OpRecordDisable = "record.disable"
OpRecordEnable = "record.enable"
)
// OpKinds lists all operation kinds.
var OpKinds = []string{OpZoneCreate, OpZoneUpdate, OpZoneDelete, OpRecordCreate, OpRecordUpdate, OpRecordDelete, OpRRSetReplace,
OpViewCreate, OpViewUpdate, OpViewDelete, OpViewReorder, OpACLCreate, OpACLUpdate, OpACLDelete,
OpTSIGCreate, OpTSIGUpdate, OpTSIGDelete, OpOptionCreate, OpOptionUpdate, OpOptionDelete, OpSettingsUpdate}
OpTSIGCreate, OpTSIGUpdate, OpTSIGDelete, OpOptionCreate, OpOptionUpdate, OpOptionDelete, OpSettingsUpdate,
OpBlocklistCreate, OpBlocklistUpdate, OpBlocklistDelete, OpBlockRuleCreate, OpBlockRuleUpdate, OpBlockRuleDelete, OpBlockingPause}
// Request is an operation as submitted by a client. Requests are expanded
// into one or more stored operations (for example an A record with "ptr"
@@ -92,6 +108,8 @@ type ZoneInput struct {
// finished, off ("").
DNSSECPolicy *string `json:"dnssec_policy,omitempty"`
Comment *string `json:"comment,omitempty"`
// Disabled keeps the zone in Jarl but out of named's configuration.
Disabled *bool `json:"disabled,omitempty"`
// Create only: name servers for the apex NS records (default: the
// configured default name servers), additional records, and a zone to
@@ -121,8 +139,10 @@ type RecordInput struct {
TTL model.Opt[uint32] `json:"ttl,omitzero"` // null: use the zone default
Data *string `json:"data,omitempty"`
Comment *string `json:"comment,omitempty"`
// PTR also creates (or deletes) the matching PTR record in the
// reverse zone, for A and AAAA records.
// Disabled keeps the record in Jarl but out of the zone.
Disabled *bool `json:"disabled,omitempty"`
// PTR also creates (or deletes, disables or enables) the matching PTR
// record in the reverse zone, for A and AAAA records.
PTR bool `json:"ptr,omitempty"`
// MissingOK makes a delete-by-match succeed when nothing matches.
MissingOK bool `json:"missing_ok,omitempty"`
@@ -205,3 +225,37 @@ type SettingsInput struct {
type DeleteInput struct {
Cascade bool `json:"cascade,omitempty"`
}
// BlocklistInput creates or updates a blocklist. Views are given by name
// (Views) or ID (ViewIDs); an empty list means every view that offers
// recursion.
type BlocklistInput struct {
Name *string `json:"name,omitempty"`
URL *string `json:"url,omitempty"`
Views *[]string `json:"views,omitempty"`
ViewIDs *[]int64 `json:"view_ids,omitempty"`
Disabled *bool `json:"disabled,omitempty"`
Comment *string `json:"comment,omitempty"`
}
// BlockRuleInput creates or updates a block or allow rule. Views as for
// BlocklistInput.
type BlockRuleInput struct {
Name *string `json:"name,omitempty"`
Action *string `json:"action,omitempty"` // block (default) or allow
Exact *bool `json:"exact,omitempty"` // only the name, not the names below it
Views *[]string `json:"views,omitempty"`
ViewIDs *[]int64 `json:"view_ids,omitempty"`
Comment *string `json:"comment,omitempty"`
}
// BlockingPauseInput pauses blocking until a time, or resumes it (null).
type BlockingPauseInput struct {
Until *time.Time `json:"until"`
// Indefinitely pauses until someone resumes blocking (Until is nil).
Indefinitely bool `json:"indefinitely,omitempty"`
}
// MaxBlockingPause is the longest pause: blocking shouldn't stay off by
// mistake (disable lists to stop using them).
const MaxBlockingPause = 24 * time.Hour
+77
View File
@@ -0,0 +1,77 @@
package engine
import (
"context"
"fmt"
"os"
"strings"
"sync"
"time"
"jarl/internal/model"
"jarl/internal/render"
"jarl/internal/validate"
)
// namedCheckTTL is how long the result of checking named's running
// configuration is kept when nothing changed.
const namedCheckTTL = 10 * time.Minute
// namedCheck caches the warnings of named-checkconf on the configuration
// named runs, keyed by what it depends on.
type namedCheck struct {
mu sync.Mutex
key string
at time.Time
problems []validate.Problem
}
// namedProblems runs named-checkconf on the configuration named runs
// (without loading zones, which is quick) and returns what it reports as
// problems. An apply shows these too, but they are named's own checks,
// which Jarl doesn't repeat ("both recursion no and allow-recursion
// active"): the dashboard lists them with Jarl's so they don't go unseen
// after the apply. The result is reused while the deployed files and
// named's main file are unchanged.
func (e *Engine) namedProblems(ctx context.Context, st *model.State, deployed map[string]*model.DeployedFile) []validate.Problem {
if e.tools == nil || e.cfg.SkipCheck || e.cfg.MainConfig == "" || len(deployed) == 0 {
return nil
}
key := ""
if fi, err := os.Stat(e.cfg.MainConfig); err == nil {
key = fi.ModTime().String()
}
if f := deployed[e.cfg.Paths.MainConfig()]; f != nil {
key += "|" + f.SHA256
}
c := &e.namedCheck
c.mu.Lock()
defer c.mu.Unlock()
if c.key == key && e.Now().Sub(c.at) < namedCheckTTL {
return c.problems
}
res, err := e.tools.CheckConfig(ctx, e.cfg.MainConfig, false)
if err != nil {
return nil // named-checkconf couldn't run; an apply reports that
}
out := render.Render(st, e.cfg.Paths, e.extras(ctx))
var ps []validate.Problem
for _, m := range res.Messages {
p := validate.Problem{Severity: validate.Warning, Code: "named_warning", View: m.View, Zone: m.Zone}
if m.Error {
p.Severity, p.Code = validate.Error, "named_error"
}
where := ""
if m.File != "" {
where = fmt.Sprintf(" (%s:%d)", m.File, m.Line)
}
if span, ok := out.Lookup(m.File, m.Line); ok {
p.ObjectType, p.ObjectID = span.ObjectType, span.ObjectID
where = " (" + span.Label + ", " + where[2:]
}
p.Message = "named-checkconf: " + strings.TrimSpace(m.Text) + where
ps = append(ps, p)
}
c.key, c.at, c.problems = key, e.Now(), ps
return ps
}
+40 -4
View File
@@ -98,6 +98,16 @@ func applyOp(st *model.State, op model.Op, c *applyCtx) error {
err = applyOptionDelete(st, op, c)
case OpSettingsUpdate:
err = applySettings(st, op, c)
case OpBlocklistCreate, OpBlocklistUpdate:
err = applyBlocklistPut(st, op, c)
case OpBlocklistDelete:
err = applyBlocklistDelete(st, op, c)
case OpBlockRuleCreate, OpBlockRuleUpdate:
err = applyBlockRulePut(st, op, c)
case OpBlockRuleDelete:
err = applyBlockRuleDelete(st, op, c)
case OpBlockingPause:
err = applyBlockingPause(st, op, c)
default:
err = errf(KindBadRequest, "unknown operation %q", op.Kind)
}
@@ -230,6 +240,9 @@ func setZoneFields(st *model.State, z *model.Zone, in *ZoneInput, creating bool)
if in.SerialPolicy != nil {
z.SerialPolicy = strings.ToLower(strings.TrimSpace(*in.SerialPolicy))
}
if in.Disabled != nil {
z.Disabled = *in.Disabled
}
if in.DNSSECPolicy != nil {
p := strings.Trim(strings.TrimSpace(*in.DNSSECPolicy), `"`)
switch strings.ToLower(p) {
@@ -373,7 +386,7 @@ func parseRecord(z *model.Zone, name, typ, data string) (string, *rr.Parsed, err
func rrsetTTL(st *model.State, zoneID int64, name, typ string, exceptID int64) (*uint32, bool) {
for _, r := range st.Records {
if r.ZoneID == zoneID && r.Name == name && r.Type == typ && r.ID != exceptID {
if r.ZoneID == zoneID && r.Name == name && r.Type == typ && r.ID != exceptID && !r.Disabled {
if r.TTL == nil {
return nil, true
}
@@ -422,6 +435,9 @@ func applyRecordCreate(st *model.State, op model.Op, c *applyCtx) error {
if in.Comment != nil {
r.Comment = *in.Comment
}
if in.Disabled != nil {
r.Disabled = *in.Disabled
}
for _, w := range p.Warnings {
c.warn("%s %s: %s", dnsname.Display(owner), p.Type, w)
}
@@ -498,6 +514,9 @@ func applyRecordUpdate(st *model.State, op model.Op, c *applyCtx) error {
if in.Comment != nil {
r.Comment = *in.Comment
}
if in.Disabled != nil {
r.Disabled = *in.Disabled
}
r.Touch(c.user(), c.now)
return nil
}
@@ -617,20 +636,32 @@ func applyRRSetReplace(st *model.State, op model.Op, c *applyCtx) error {
} else if t, ok := rrsetTTL(st, z.ID, owner, typ, 0); ok {
ttl, ttlKnown = t, true
}
// The RRset becomes exactly the given data. A disabled record with
// one of the values is enabled again; other disabled records are left
// alone.
keep := map[string]bool{}
for id, r := range st.Records {
// (In ID order, so which of two identical records is kept doesn't
// depend on map order.)
for _, id := range sortedKeys(st.Records) {
r := st.Records[id]
if r.ZoneID != z.ID || r.Name != owner || r.Type != typ {
continue
}
k := strings.ToLower(r.Data)
if seen[k] && !keep[k] {
keep[k] = true
if ttlKnown && !sameTTL(r.TTL, ttl) {
r.TTL = copyTTL(ttl)
if (ttlKnown && !sameTTL(r.TTL, ttl)) || r.Disabled {
if ttlKnown {
r.TTL = copyTTL(ttl)
}
r.Disabled = false
r.Touch(c.user(), c.now)
}
continue
}
if r.Disabled {
continue
}
delete(st.Records, id)
}
ids := in.NewIDs
@@ -813,6 +844,9 @@ func applyViewDelete(st *model.State, op model.Op, c *applyCtx) error {
if err := decode(op.Data, &in); err != nil {
return err
}
if users := blockingUsers(st, v.ID); len(users) > 0 {
return errf(KindConflict, "view %s is used by %s; change them to other views first", v.Name, strings.Join(users, ", "))
}
zones := st.ZonesInView(v.ID)
if len(zones) > 0 && !in.Cascade {
var names []string
@@ -1093,6 +1127,8 @@ func NormalizeOption(st *model.State, scope string, scopeID int64, name, value s
if (stmt.Kind == catalog.KindString) && !strings.HasPrefix(v, `"`) && !strings.ContainsAny(v, " \t{};") {
v = namedconf.QuoteString(v)
}
// Lists may be given without their braces: "10.0.0.53; key xfer".
v = stmt.WrapList(v)
parsed, err := namedconf.ParseOptionValue(n, v)
if err != nil {
return "", "", errf(KindBadRequest, "option %s: %v", n, err)
+22
View File
@@ -0,0 +1,22 @@
package engine
import (
"testing"
"jarl/internal/model"
)
func TestNormalizeOptionWrapsLists(t *testing.T) {
st := model.NewState()
for in, want := range map[string]string{
"10.0.10.53": "{ 10.0.10.53; }",
"localhost; key xfer;": "{ localhost; key xfer; }",
"port 53 { any; }": "port 53 { any; }",
"localhost 10.0.10.53": "{ localhost; 10.0.10.53; }",
} {
_, v, err := NormalizeOption(st, model.ScopeOptions, 0, "allow-transfer", in)
if err != nil || v != want {
t.Errorf("%q: %q, %v; want %q", in, v, err, want)
}
}
}
+2 -2
View File
@@ -101,8 +101,8 @@ func (e *Engine) guardSerials(ctx context.Context, base, work *model.State, bump
continue
}
old := base.Zones[z.ID]
if old == nil || old.Type != model.ZonePrimary || old.ViewID != z.ViewID {
continue // new to named: nothing loaded to compare with
if old == nil || old.Type != model.ZonePrimary || old.ViewID != z.ViewID || old.Disabled || z.Disabled {
continue // new to named (or not in it): nothing loaded to compare with
}
path := render.ZoneFilePath(work, z, e.cfg.Paths)
if cur, ok, _ := readFile(path); ok && string(cur) == render.ZoneFile(work, z) {
+68
View File
@@ -0,0 +1,68 @@
package engine
import (
"context"
"crypto/rand"
"encoding/base64"
"errors"
"jarl/internal/model"
"jarl/internal/nscheck"
"jarl/internal/render"
"jarl/internal/store"
)
// viewKeySeedMeta is where the seed of the view keys is kept.
const viewKeySeedMeta = "view_key_seed"
// ViewKeySeed returns the seed the view keys' secrets are derived from
// (render/viewkeys.go), creating it the first time.
func ViewKeySeed(ctx context.Context, st *store.Store) ([]byte, error) {
v, err := st.GetMeta(ctx, viewKeySeedMeta)
if err == nil {
return base64.StdEncoding.DecodeString(v)
}
if !errors.Is(err, store.ErrNotFound) {
return nil, err
}
seed := make([]byte, 32)
if _, err := rand.Read(seed); err != nil {
return nil, err
}
if err := st.SetMeta(ctx, viewKeySeedMeta, base64.StdEncoding.EncodeToString(seed)); err != nil {
return nil, err
}
return seed, nil
}
// extras returns what jarld's configuration adds to the rendered
// configuration, with the view keys' seed.
func (e *Engine) extras(ctx context.Context) render.Extras {
x := e.cfg.Extras
e.seedMu.Lock()
defer e.seedMu.Unlock()
if e.seed == nil {
seed, err := ViewKeySeed(ctx, e.store)
if err != nil {
e.log.Warn("view keys: reading their seed failed", "err", err)
return x
}
e.seed = seed
}
x.ViewKeySeed = e.seed
return x
}
// ViewKey returns the TSIG key that selects a zone's view on this server
// (nil without views).
func (e *Engine) ViewKey(ctx context.Context, st *model.State, z *model.Zone) *nscheck.Key {
v := st.Views[z.ViewID]
if v == nil {
return nil
}
x := e.extras(ctx)
if len(x.ViewKeySeed) == 0 {
return nil
}
return &nscheck.Key{Name: render.ViewKeyName(v.Name), Algorithm: render.ViewKeyAlgorithm, Secret: render.ViewKeySecret(x.ViewKeySeed, v.ID)}
}
+81
View File
@@ -0,0 +1,81 @@
package engine
import (
"os"
"path/filepath"
"strings"
"testing"
"jarl/internal/model"
)
func TestZoneDisable(t *testing.T) {
env := newEnv(t)
setupZone(env)
z := env.state().ZonesNamed("example.com")[0]
confPath := filepath.Join(env.dir, "etc/named.conf")
zonePath := filepath.Join(env.dir, "zones/_default/example.com.zone")
res := env.submit(env.admin, NewRequest(OpZoneUpdate, z.ID, ZoneInput{Disabled: model.Ptr(true)}))
if res.Ops[0].Summary != "disable zone example.com" || len(res.Warnings) != 1 || !strings.Contains(res.Warnings[0], "recursion") {
t.Errorf("disable: %q, warnings %v", res.Ops[0].Summary, res.Warnings)
}
conf, _ := os.ReadFile(confPath)
if strings.Contains(string(conf), "zone \"example.com\" {") || !strings.Contains(string(conf), `// zone "example.com" is disabled in Jarl`) {
t.Errorf("named.conf with the zone disabled:\n%s", conf)
}
// Still edited, and its file kept up to date.
env.submit(env.admin, rec("example.com", "www", "A", "192.0.2.10"))
if f, err := os.ReadFile(zonePath); err != nil || !strings.Contains(string(f), "www\t\tIN\tA\t192.0.2.10") {
t.Errorf("zone file while disabled: %v\n%s", err, f)
}
// Enabling brings it back.
res = env.submit(env.admin, NewRequest(OpZoneUpdate, z.ID, ZoneInput{Disabled: model.Ptr(false)}))
if res.Ops[0].Summary != "enable zone example.com" {
t.Errorf("enable: %q", res.Ops[0].Summary)
}
if conf, _ := os.ReadFile(confPath); !strings.Contains(string(conf), "zone \"example.com\" {") {
t.Errorf("named.conf after enabling:\n%s", conf)
}
// Reverting a disable enables the zone again.
res = env.submit(env.admin, NewRequest(OpZoneUpdate, z.ID, ZoneInput{Disabled: model.Ptr(true)}))
rev, err := env.e.Revert(env.ctx, res.ChangesetID, env.admin)
if err != nil {
t.Fatal(err)
}
if _, err := env.e.Apply(env.ctx, rev.ID, env.admin, ApplyOptions{}); err != nil || env.state().Zones[z.ID].Disabled {
t.Errorf("revert of a disable: %v, disabled=%v", err, env.state().Zones[z.ID].Disabled)
}
// Signed zones can't be disabled.
env.submit(env.admin, NewRequest(OpZoneUpdate, z.ID, ZoneInput{DNSSECPolicy: model.Ptr("default")}))
if e := env.submitErr(env.admin, NewRequest(OpZoneUpdate, z.ID, ZoneInput{Disabled: model.Ptr(true)})); e.Kind != KindValidation || !strings.Contains(e.Message, "signed zone can't be disabled") {
t.Errorf("disabling a signed zone: %+v", e)
}
}
func TestZoneDisableInView(t *testing.T) {
env := newEnv(t)
setupZone(env)
env.submit(env.admin,
NewRequest(OpViewCreate, 0, ViewInput{Name: model.Ptr("internal"), MatchClients: &[]string{"10.0.0.0/8"}, AdoptZones: true}),
NewRequest(OpViewCreate, 0, ViewInput{Name: model.Ptr("external")}),
)
st := env.state()
ext := st.ViewByName("external")
env.submit(env.admin, NewRequest(OpZoneCreate, 0, ZoneInput{Name: model.Ptr("example.com"), ViewID: &ext.ID, Type: model.Ptr(model.ZoneInView), InView: model.Ptr("internal")}))
in := st.ZonesNamed("example.com")[0]
// The zone another view shares can't be disabled alone.
if e := env.submitErr(env.admin, NewRequest(OpZoneUpdate, in.ID, ZoneInput{Disabled: model.Ptr(true)})); e.Kind != KindValidation || !strings.Contains(e.Message, "is disabled, so this zone can't share it") {
t.Errorf("disabling a shared zone: %+v", e)
}
// Without recursion, the warning says REFUSED.
env.submit(env.admin, NewRequest(OpOptionCreate, 0, OptionInput{Scope: model.Ptr(model.ScopeView), ScopeID: &ext.ID, Name: model.Ptr("recursion"), Value: model.Ptr("no")}))
env.submit(env.admin, NewRequest(OpZoneCreate, 0, ZoneInput{Name: model.Ptr("public.example"), ViewID: &ext.ID}))
pub := env.state().FindZone("public.example", ext.ID)
res := env.submit(env.admin, NewRequest(OpZoneUpdate, pub.ID, ZoneInput{Disabled: model.Ptr(true)}))
if len(res.Warnings) != 1 || !strings.Contains(res.Warnings[0], "view external answers REFUSED") {
t.Errorf("warnings = %v", res.Warnings)
}
}
+25 -1
View File
@@ -393,6 +393,10 @@ func (im *importer) key(s *namedconf.Stmt) error {
return fmt.Errorf("%s: key without a block", s.Pos)
}
name := model.NormalizeKeyName(s.Arg(0))
if strings.HasPrefix(name, model.ViewKeyPrefix) {
// A view key from a configuration Jarl wrote: Jarl makes its own.
return nil
}
alg, secret := "", ""
if a := namedconf.FindOne(b.Stmts, "algorithm"); a != nil {
alg = strings.ToLower(a.Arg(0))
@@ -433,7 +437,7 @@ func (im *importer) view(s *namedconf.Stmt) error {
switch kw := st.Keyword(); kw {
case "match-clients":
if mb := st.FirstBlock(); mb != nil {
v.MatchClients = nonNil(elementStrings(mb))
v.MatchClients = withoutViewKeys(nonNil(elementStrings(mb)))
}
case "match-destinations":
if mb := st.FirstBlock(); mb != nil {
@@ -807,3 +811,23 @@ func allowsNone(o *namedconf.Stmt) bool {
}
return true
}
// withoutViewKeys drops the view keys Jarl puts in front of match-clients
// (render/viewkeys.go) from a list read from a configuration Jarl wrote,
// and the "any" it adds after them to a view that had no list.
func withoutViewKeys(elems []string) []string {
out := []string{}
dropped := false
for _, e := range elems {
f := strings.Fields(strings.TrimPrefix(strings.TrimSpace(e), "!"))
if len(f) == 2 && f[0] == "key" && strings.HasPrefix(strings.Trim(f[1], `"`), model.ViewKeyPrefix) {
dropped = true
continue
}
out = append(out, e)
}
if dropped && len(out) == 1 && out[0] == "any" {
return []string{}
}
return out
}
+12
View File
@@ -205,3 +205,15 @@ view "internal" {
t.Errorf("imported state has errors: %v", ps)
}
}
func TestWithoutViewKeys(t *testing.T) {
for _, tc := range []struct{ in, want []string }{
{[]string{"!key jarl-view-external", `key "jarl-view-internal"`, "homelab"}, []string{"homelab"}},
{[]string{"key jarl-view-external", "any"}, []string{}},
{[]string{"key xfer", "any"}, []string{"key xfer", "any"}},
} {
if got := withoutViewKeys(tc.in); strings.Join(got, ";") != strings.Join(tc.want, ";") || got == nil {
t.Errorf("%v: %v, want %v", tc.in, got, tc.want)
}
}
}
+136
View File
@@ -0,0 +1,136 @@
package model
import (
"slices"
"sort"
"strings"
"time"
)
// Blocklist is a list of domains to block, downloaded from URL and kept up
// to date by jarld (docs/blocking.md).
type Blocklist struct {
ID int64 `json:"id"`
Name string `json:"name"` // a DNS label: it names the list's policy zone
URL string `json:"url"`
// ViewIDs are the views the list applies to; empty means every view
// that offers recursion (see BlockingViews).
ViewIDs []int64 `json:"view_ids"`
Disabled bool `json:"disabled,omitempty"`
Comment string `json:"comment"`
Meta
}
// Block rule actions.
const (
BlockActionBlock = "block"
BlockActionAllow = "allow"
)
// BlockRule blocks or allows a domain, and unless Exact the names below
// it. Rules take precedence over blocklists.
type BlockRule struct {
ID int64 `json:"id"`
Name string `json:"name"` // lowercase, without the final dot
Action string `json:"action"` // block or allow
Exact bool `json:"exact,omitempty"`
ViewIDs []int64 `json:"view_ids"` // as for Blocklist
Comment string `json:"comment"`
Meta
}
// BlocklistStatus is what jarld knows about a list's downloads. It is not
// configuration: it is kept outside changesets.
type BlocklistStatus struct {
ID int64 `json:"id"`
URL string `json:"url"` // the URL last tried
ETag string `json:"-"`
LastModified string `json:"-"`
CheckedAt *time.Time `json:"checked_at,omitempty"` // last attempt
UpdatedAt *time.Time `json:"updated_at,omitempty"` // last time the content changed
Entries int `json:"entries"` // domains
Records int `json:"records"` // policy zone records
Skipped int `json:"skipped"` // lines that weren't usable
SHA256 string `json:"-"`
Error string `json:"error,omitempty"` // the last attempt failed
}
// BlocklistByName finds a blocklist by name.
func (s *State) BlocklistByName(name string) *Blocklist {
for _, b := range s.Blocklists {
if strings.EqualFold(b.Name, name) {
return b
}
}
return nil
}
// SortedBlocklists returns the blocklists ordered by name.
func (s *State) SortedBlocklists() []*Blocklist {
out := make([]*Blocklist, 0, len(s.Blocklists))
for _, b := range s.Blocklists {
out = append(out, b)
}
sort.Slice(out, func(i, j int) bool { return out[i].Name < out[j].Name })
return out
}
// SortedBlockRules returns the rules ordered by domain, parents first.
func (s *State) SortedBlockRules() []*BlockRule {
out := make([]*BlockRule, 0, len(s.BlockRules))
for _, r := range s.BlockRules {
out = append(out, r)
}
sort.Slice(out, func(i, j int) bool {
a, b := out[i], out[j]
if a.Name != b.Name {
return CanonicalLess(a.Name+".", b.Name+".")
}
if a.Exact != b.Exact {
return !a.Exact
}
return a.ID < b.ID
})
return out
}
// RecursionOn reports whether a view (0: the server without views) offers
// recursion: its recursion option, else the server's, else BIND's default
// (yes).
func (s *State) RecursionOn(viewID int64) bool {
o := s.FindOption(ScopeView, viewID, "recursion")
if viewID == 0 || o == nil {
if g := s.FindOption(ScopeOptions, 0, "recursion"); o == nil {
o = g
}
}
if o == nil {
return true
}
switch strings.ToLower(strings.Trim(strings.TrimSpace(o.Value), `";`)) {
case "no", "false", "0":
return false
}
return true
}
// BlockingViews returns the views a blocklist or rule with viewIDs applies
// to, in view order: the given views, or when none are given every view
// that offers recursion (blocking only affects recursive queries). Without
// views it returns []int64{0}, the whole server, if it offers recursion.
func (s *State) BlockingViews(viewIDs []int64) []int64 {
views := s.OrderedViews()
if len(views) == 0 {
if !s.RecursionOn(0) {
return nil
}
return []int64{0}
}
var out []int64
for _, v := range views {
if len(viewIDs) == 0 && s.RecursionOn(v.ID) || slices.Contains(viewIDs, v.ID) {
out = append(out, v.ID)
}
}
return out
}
+74 -2
View File
@@ -128,6 +128,9 @@ type Zone struct {
// signing is being removed safely (docs/dnssec.md).
DNSSECPolicy string `json:"dnssec_policy,omitempty"`
Comment string `json:"comment"`
// Disabled zones stay in Jarl, records and all, but named doesn't
// load them: their zone statement is left out of named.conf.
Disabled bool `json:"disabled,omitempty"`
Meta
}
@@ -154,6 +157,9 @@ type Record struct {
TTL *uint32 `json:"ttl,omitempty"`
Data string `json:"data"` // canonical presentation format
Comment string `json:"comment"`
// Disabled records stay in Jarl but are left out of the zone (the
// zone file lists them as comments).
Disabled bool `json:"disabled,omitempty"`
Meta
}
@@ -205,7 +211,18 @@ type Settings struct {
Expire uint32 `json:"expire"`
NegativeTTL uint32 `json:"negative_ttl"`
SerialPolicy string `json:"serial_policy"`
Version int `json:"version"`
// BlockingPausedUntil, while set, lets every name through blocking
// (docs/blocking.md); jarld clears it when the time is up.
BlockingPausedUntil *time.Time `json:"blocking_paused_until,omitempty"`
// BlockingPausedIndefinitely pauses blocking until someone resumes it.
BlockingPausedIndefinitely bool `json:"blocking_paused_indefinitely,omitempty"`
Version int `json:"version"`
}
// BlockingPaused reports whether blocking is paused, for a time or until
// resumed.
func (s *Settings) BlockingPaused() bool {
return s.BlockingPausedUntil != nil || s.BlockingPausedIndefinitely
}
// DefaultSettings returns the settings of a fresh installation.
@@ -229,6 +246,9 @@ type State struct {
Zones map[int64]*Zone
Records map[int64]*Record
Options map[int64]*Option
// Blocking (docs/blocking.md).
Blocklists map[int64]*Blocklist
BlockRules map[int64]*BlockRule
}
// NewState returns an empty state with default settings.
@@ -241,6 +261,9 @@ func NewState() *State {
Zones: map[int64]*Zone{},
Records: map[int64]*Record{},
Options: map[int64]*Option{},
Blocklists: map[int64]*Blocklist{},
BlockRules: map[int64]*BlockRule{},
}
}
@@ -254,6 +277,9 @@ func (s *State) Clone() *State {
Zones: make(map[int64]*Zone, len(s.Zones)),
Records: make(map[int64]*Record, len(s.Records)),
Options: make(map[int64]*Option, len(s.Options)),
Blocklists: make(map[int64]*Blocklist, len(s.Blocklists)),
BlockRules: make(map[int64]*BlockRule, len(s.BlockRules)),
}
c.Settings.Nameservers = append([]string(nil), s.Settings.Nameservers...)
for id, v := range s.Views {
@@ -291,6 +317,16 @@ func (s *State) Clone() *State {
n := *o
c.Options[id] = &n
}
for id, b := range s.Blocklists {
n := *b
n.ViewIDs = append([]int64(nil), b.ViewIDs...)
c.Blocklists[id] = &n
}
for id, r := range s.BlockRules {
n := *r
n.ViewIDs = append([]int64(nil), r.ViewIDs...)
c.BlockRules[id] = &n
}
return c
}
@@ -348,6 +384,13 @@ func (s *State) KeyByName(name string) *TSIGKey {
return nil
}
// ViewKeyPrefix starts the names of the TSIG keys Jarl gives views
// (render/viewkeys.go); other keys can't use it.
const ViewKeyPrefix = "jarl-view-"
// ViewKeyName returns the name of a view's key.
func ViewKeyName(view string) string { return ViewKeyPrefix + view }
// NormalizeKeyName returns the canonical form of a key name: lowercase,
// without a trailing dot.
func NormalizeKeyName(name string) string {
@@ -446,6 +489,18 @@ func (s *State) ZoneRecords(zoneID int64) []*Record {
return out
}
// ServedRecords returns the records of a zone that are in the zone file
// (not disabled), in display order.
func (s *State) ServedRecords(zoneID int64) []*Record {
var out []*Record
for _, r := range s.ZoneRecords(zoneID) {
if !r.Disabled {
out = append(out, r)
}
}
return out
}
// RecordsByZone groups all records by zone ID (unsorted).
func (s *State) RecordsByZone() map[int64][]*Record {
out := map[int64][]*Record{}
@@ -455,7 +510,20 @@ func (s *State) RecordsByZone() map[int64][]*Record {
return out
}
// ScopedOptions returns the options of a scope, in position order.
// ServedRecordsByZone is RecordsByZone without disabled records.
func (s *State) ServedRecordsByZone() map[int64][]*Record {
out := map[int64][]*Record{}
for _, r := range s.Records {
if !r.Disabled {
out[r.ZoneID] = append(out[r.ZoneID], r)
}
}
return out
}
// ScopedOptions returns the options of a scope: in a fixed order by name
// for a block (see optionLess), in the order they were added for
// top-level statements, which are written as entered.
func (s *State) ScopedOptions(scope string, scopeID int64) []*Option {
var out []*Option
for _, o := range s.Options {
@@ -463,6 +531,10 @@ func (s *State) ScopedOptions(scope string, scopeID int64) []*Option {
out = append(out, o)
}
}
if scope != ScopeTopLevel {
sort.Slice(out, func(i, j int) bool { return optionLess(out[i], out[j]) })
return out
}
sort.Slice(out, func(i, j int) bool {
if out[i].Position != out[j].Position {
return out[i].Position < out[j].Position
+55
View File
@@ -0,0 +1,55 @@
package model
// optionGroups orders the options of a block (options, a view, a zone) the
// same way everywhere: in named.conf, the web UI and the CLI. Related
// options are grouped; within a group they are alphabetical, and options
// not listed here come last, alphabetically. named doesn't care about the
// order of options in a block.
var optionGroups = [][]string{
// Files and identity.
{"bindkeys-file", "directory", "dump-file", "hostname", "managed-keys-directory", "memstatistics-file", "pid-file", "recursing-file", "secroots-file", "server-id", "session-keyfile", "statistics-file", "version"},
// Listening and source addresses.
{"listen-on", "listen-on-v6", "notify-source", "notify-source-v6", "query-source", "query-source-v6", "transfer-source", "transfer-source-v6"},
// Access control.
{"allow-query", "allow-query-cache", "allow-query-cache-on", "allow-query-on", "allow-recursion", "allow-recursion-on", "allow-update", "allow-update-forwarding", "blackhole", "deny-answer-addresses", "deny-answer-aliases", "update-policy"},
// Recursion, forwarding and the cache.
{"forward", "forwarders", "max-cache-size", "max-cache-ttl", "max-ncache-ttl", "minimal-responses", "prefetch", "qname-minimization", "rate-limit", "recursion", "response-policy", "serve-stale", "stale-answer-enable", "stale-answer-ttl", "stale-cache-enable"},
// DNSSEC.
{"dnssec-accept-expired", "dnssec-must-be-secure", "dnssec-policy", "dnssec-validation", "inline-signing", "key-directory", "trust-anchor-telemetry", "validate-except"},
// Transfers and notify.
{"allow-notify", "allow-transfer", "also-notify", "catalog-zones", "ixfr-from-differences", "masters", "max-transfer-idle-in", "max-transfer-idle-out", "max-transfer-time-in", "max-transfer-time-out", "notify", "notify-to-soa", "primaries", "provide-ixfr", "request-ixfr", "serial-query-rate", "transfer-format", "transfers-in", "transfers-out", "transfers-per-ns"},
}
var optionGroup = func() map[string]int {
m := map[string]int{}
for i, g := range optionGroups {
for _, n := range g {
m[n] = i
}
}
return m
}()
// optionLess orders two options of the same block: by group, then name,
// then (for an option given more than once, like listen-on) the order they
// were added in.
func optionLess(a, b *Option) bool {
ga, ok := optionGroup[a.Name]
if !ok {
ga = len(optionGroups)
}
gb, ok := optionGroup[b.Name]
if !ok {
gb = len(optionGroups)
}
if ga != gb {
return ga < gb
}
if a.Name != b.Name {
return a.Name < b.Name
}
if a.Position != b.Position {
return a.Position < b.Position
}
return a.ID < b.ID
}
+40
View File
@@ -0,0 +1,40 @@
package model
import (
"strings"
"testing"
)
func TestScopedOptionsOrder(t *testing.T) {
s := NewState()
add := func(scope string, names ...string) {
for i, n := range names {
id := int64(len(s.Options) + 1)
s.Options[id] = &Option{ID: id, Scope: scope, Name: n, Position: i}
}
}
// Added in an arbitrary order; listen-on twice.
add(ScopeView, "zzz-unknown", "forwarders", "listen-on", "recursion", "dnssec-validation", "aaa-unknown", "allow-query", "forward", "directory", "listen-on", "allow-transfer")
var got []string
for _, o := range s.ScopedOptions(ScopeView, 0) {
got = append(got, o.Name)
}
want := "directory listen-on listen-on allow-query forward forwarders recursion dnssec-validation allow-transfer aaa-unknown zzz-unknown"
if strings.Join(got, " ") != want {
t.Errorf("order:\n got %s\nwant %s", strings.Join(got, " "), want)
}
// An option given twice keeps the order it was added in.
l := s.ScopedOptions(ScopeView, 0)
if l[1].Position > l[2].Position {
t.Error("repeated listen-on swapped")
}
// Top-level statements stay as entered.
add(ScopeTopLevel, "logging", "acl", "key")
got = nil
for _, o := range s.ScopedOptions(ScopeTopLevel, 0) {
got = append(got, o.Name)
}
if strings.Join(got, " ") != "logging acl key" {
t.Errorf("top level: %v", got)
}
}
+18
View File
@@ -36,6 +36,24 @@ func Compact(items []Item) string {
// CompactStmt returns the statement on a single line with its semicolon.
func CompactStmt(s *Stmt) string { return Compact(s.Items) + ";" }
// FormatValue returns an option's value laid out as named.conf writes it:
// blocks over several lines, indented with tabs. Values without a block,
// and values that don't parse, are returned as they are.
func FormatValue(name, value string) string {
if !strings.Contains(value, "{") {
return value
}
st, err := ParseOptionValue(name, value)
if err != nil {
return value
}
out := strings.TrimSuffix(strings.TrimRight(FormatStmt(st, 0), "\n"), ";")
if rest, ok := strings.CutPrefix(out, st.Keyword()); ok {
return strings.TrimLeft(rest, " ")
}
return value
}
// QuoteString returns s as a named.conf quoted string.
func QuoteString(s string) string {
var sb strings.Builder
+18
View File
@@ -0,0 +1,18 @@
package namedconf
import "testing"
func TestFormatValue(t *testing.T) {
for _, c := range []struct{ name, value, want string }{
{"logging", `{ channel c { file "/var/log/named/x.log" versions 3; print-time yes; }; category default { c; }; }`,
// A short inner block stays on one line, as in named.conf.
"{\n\tchannel c { file \"/var/log/named/x.log\" versions 3; print-time yes; };\n\tcategory default { c; };\n}"},
{"allow-query", `{ allowed_clients; }`, `{ allowed_clients; }`}, // short: unchanged
{"directory", `"/var/cache/bind"`, `"/var/cache/bind"`},
{"logging", `{ broken`, `{ broken`}, // unparsable: as it is
} {
if got := FormatValue(c.name, c.value); got != c.want {
t.Errorf("FormatValue(%s, %q) =\n%s\nwant\n%s", c.name, c.value, got, c.want)
}
}
}
+46
View File
@@ -0,0 +1,46 @@
package nscheck
import (
"net/netip"
"os"
"path/filepath"
"slices"
"testing"
)
// Without netlink (a sandboxed jarld), the host's addresses come from /proc.
func TestProcAddrs(t *testing.T) {
dir := t.TempDir()
os.WriteFile(filepath.Join(dir, "if_inet6"), []byte(
"00000000000000000000000000000001 01 80 10 80 lo\n"+
"2a0280137f3d00100000000000000005 02 40 00 80 eth0\n"), 0o644)
os.WriteFile(filepath.Join(dir, "fib_trie"), []byte(`Main:
+-- 0.0.0.0/0 3 0 5
|-- 0.0.0.0
/0 universe UNICAST
+-- 10.0.10.0/24 2 0 2
|-- 10.0.10.0
/24 link UNICAST
|-- 10.0.10.5
/32 host LOCAL
|-- 10.0.10.255
/32 link BROADCAST
Local:
+-- 0.0.0.0/0 3 0 5
|-- 10.0.10.5
/32 host LOCAL
`), 0o644)
got := procAddrs(dir)
for _, want := range []string{"::1", "2a02:8013:7f3d:10::5", "10.0.10.5"} {
if !slices.Contains(got, netip.MustParseAddr(want)) {
t.Errorf("%s missing from %v", want, got)
}
}
if len(got) != 3 {
t.Errorf("got %v (10.0.10.5 once, no broadcast or network addresses)", got)
}
// And the real ones on this machine: at least loopback.
if len(procAddrs("/proc/net")) == 0 {
t.Log("no addresses in /proc/net here")
}
}
+52
View File
@@ -0,0 +1,52 @@
package nscheck
import (
"context"
"net"
"net/netip"
"testing"
"time"
"github.com/miekg/dns"
)
// A query to this host's own address is signed with LocalKey, so a server
// with views answers from the view that key selects.
func TestLocalKey(t *testing.T) {
zone := "example.com."
key := &Key{Name: "jarl-view-external.", Algorithm: "hmac-sha256", Secret: "c2VjcmV0LXNlY3JldC1zZWNyZXQtc2VjcmV0LXNlY3I="}
pc, err := net.ListenPacket("udp", "127.0.0.1:0")
if err != nil {
t.Skip(err)
}
_, port, _ := net.SplitHostPort(pc.LocalAddr().String())
srv := &dns.Server{PacketConn: pc, TsigSecret: map[string]string{key.Name: key.Secret},
Handler: dns.HandlerFunc(func(w dns.ResponseWriter, req *dns.Msg) {
// Signed with the view key: the "external" view's serial.
serial := "1"
signed := req.IsTsig() != nil && w.TsigStatus() == nil
if signed {
serial = "2"
}
m := new(dns.Msg)
m.SetReply(req)
m.Authoritative = true
m.Answer = []dns.RR{rr(zone + " 3600 IN SOA ns1." + zone + " admin." + zone + " " + serial + " 3600 900 1209600 300")}
if signed {
m.SetTsig(key.Name, dns.HmacSHA256, 300, time.Now().Unix())
}
w.WriteMsg(m)
})}
go srv.ActivateAndServe()
t.Cleanup(func() { srv.Shutdown() })
ref := uint32(2)
c := &Checker{Timeout: time.Second, Port: port, IsLocal: func(netip.Addr) bool { return false }}
if res := c.Serials(context.Background(), zone, &ref, []Target{{Addresses: []string{"127.0.0.1"}}}); res[0].Status != "behind" {
t.Errorf("unsigned: %+v (want the other view's serial)", res[0])
}
c.LocalKey = key
if res := c.Serials(context.Background(), zone, &ref, []Target{{Addresses: []string{"127.0.0.1"}}}); res[0].Status != "in_sync" {
t.Errorf("signed with the view key: %+v", res[0])
}
}
+221 -15
View File
@@ -15,13 +15,18 @@ package nscheck
import (
"context"
"encoding/hex"
"errors"
"fmt"
"net"
"net/netip"
"os"
"path/filepath"
"sort"
"strconv"
"strings"
"sync"
"sync/atomic"
"syscall"
"time"
@@ -93,6 +98,139 @@ type Checker struct {
// IsLocal reports whether an address belongs to this host (default:
// its interface addresses and loopback).
IsLocal func(netip.Addr) bool
// LocalKey, when set, signs the queries sent to this host's own
// addresses (when they are asked): with views, the key that selects
// the zone's view, so the answer comes from that view and not from
// whichever view matches this host's address.
LocalKey *Key
// Watch, if set, is told about each DNS query the checks send and
// its outcome as it happens (the web UI shows them while it waits).
// It is called from several goroutines at once.
Watch func(Event)
}
// Event is a query a check starts or finishes (Checker.Watch).
type Event struct {
// Kind is "start" or "end" (of query ID).
Kind string `json:"kind"`
ID int64 `json:"id"`
// Query describes what is asked, as a dig command line.
Query string `json:"query,omitempty"`
// End: Answer sums up the answer ("192.0.2.1, 2001:db8::1",
// "NXDOMAIN", "referral to a.nic.lol., …"), or Error says why there
// is none. Serials' SOA queries give the server's Result instead.
Answer string `json:"answer,omitempty"`
Error string `json:"error,omitempty"`
RTTMs int64 `json:"rtt_ms,omitempty"`
Result *Result `json:"result,omitempty"`
}
// eventIDs numbers events; 0 is left for the caller's own steps.
var eventIDs atomic.Int64
func (c *Checker) startEvent(query string) int64 {
if c.Watch == nil {
return 0
}
id := eventIDs.Add(1)
c.Watch(Event{Kind: "start", ID: id, Query: query})
return id
}
func (c *Checker) endEvent(e Event) {
if c.Watch != nil {
e.Kind = "end"
c.Watch(e)
}
}
// quietKey in a context stops exchangeHostPort from reporting the query:
// the caller reports it with more to say (Serials' SOA queries).
type quietKey struct{}
// digQuery is the dig command line for m sent to hostport.
func digQuery(m *dns.Msg, hostport string, key *Key) string {
host, port, err := net.SplitHostPort(hostport)
if err != nil {
host, port = hostport, ""
}
if len(m.Question) == 0 {
return "dig @" + host
}
q := m.Question[0]
s := digLine(host, port, q.Name, dns.TypeToString[q.Qtype], !m.RecursionDesired, key)
if o := m.IsEdns0(); o != nil && o.Do() {
s += " +dnssec"
}
return s
}
// sumAnswer sums up a DNS answer in a few words.
func sumAnswer(resp *dns.Msg) string {
if resp.Rcode != dns.RcodeSuccess {
return dns.RcodeToString[resp.Rcode]
}
var vals []string
for _, rr := range resp.Answer {
switch v := rr.(type) {
case *dns.A:
vals = append(vals, v.A.String())
case *dns.AAAA:
vals = append(vals, v.AAAA.String())
case *dns.NS:
vals = append(vals, v.Ns)
case *dns.SOA:
vals = append(vals, "serial "+strconv.FormatUint(uint64(v.Serial), 10))
case *dns.DS:
vals = append(vals, "DS "+strconv.Itoa(int(v.KeyTag)))
case *dns.RRSIG:
default:
vals = append(vals, dns.TypeToString[rr.Header().Rrtype])
}
}
if len(vals) == 0 {
var ns []string
for _, rr := range resp.Ns {
switch v := rr.(type) {
case *dns.NS:
ns = append(ns, v.Ns)
case *dns.SOA:
return "no such record"
}
}
if len(ns) > 0 {
return "referral to " + shorten(ns)
}
return "empty answer"
}
s := shorten(vals)
if !resp.Authoritative && !resp.RecursionAvailable {
s += " (not authoritative)"
}
return s
}
func shorten(vals []string) string {
if len(vals) > 3 {
return strings.Join(vals[:3], ", ") + " and " + strconv.Itoa(len(vals)-3) + " more"
}
return strings.Join(vals, ", ")
}
// digLine is a dig command line for a query (no TSIG secret).
func digLine(server, port, name, types string, norec bool, key *Key) string {
s := "dig @" + server
if port != "" && port != "53" {
s += " -p " + port
}
s += " " + name + " " + types
if norec {
s += " +norec"
}
if key != nil {
s += " -y " + key.Algorithm + ":" + strings.TrimSuffix(key.Name, ".") + ":…"
}
return s
}
func (c *Checker) timeout() time.Duration {
@@ -122,20 +260,67 @@ func IsLocalAddr(a netip.Addr) bool {
if a.IsLoopback() {
return true
}
addrs, err := net.InterfaceAddrs()
if err != nil {
return false
}
for _, ia := range addrs {
if n, ok := ia.(*net.IPNet); ok {
if b, ok := netip.AddrFromSlice(n.IP); ok && b.Unmap() == a.Unmap() {
return true
}
for _, b := range LocalAddrs() {
if b == a.Unmap() {
return true
}
}
return false
}
// LocalAddrs returns this host's interface addresses. Listing them needs a
// netlink socket; where that is not allowed (a systemd unit without
// AF_NETLINK in RestrictAddressFamilies), they are read from /proc.
func LocalAddrs() []netip.Addr {
var out []netip.Addr
if addrs, err := net.InterfaceAddrs(); err == nil {
for _, ia := range addrs {
if n, ok := ia.(*net.IPNet); ok {
if b, ok := netip.AddrFromSlice(n.IP); ok {
out = append(out, b.Unmap())
}
}
}
return out
}
return procAddrs("/proc/net")
}
// procAddrs reads the host's addresses from /proc/net: IPv6 from
// if_inet6, IPv4 from the local (/32 host LOCAL) entries of fib_trie.
func procAddrs(dir string) []netip.Addr {
var out []netip.Addr
if b, err := os.ReadFile(filepath.Join(dir, "if_inet6")); err == nil {
for _, line := range strings.Split(string(b), "\n") {
f := strings.Fields(line)
if len(f) == 0 || len(f[0]) != 32 {
continue
}
raw, err := hex.DecodeString(f[0])
if err != nil {
continue
}
if a, ok := netip.AddrFromSlice(raw); ok {
out = append(out, a)
}
}
}
if b, err := os.ReadFile(filepath.Join(dir, "fib_trie")); err == nil {
seen := map[netip.Addr]bool{}
var last netip.Addr
for _, line := range strings.Split(string(b), "\n") {
t := strings.TrimSpace(line)
if strings.HasPrefix(t, "|-- ") {
last, _ = netip.ParseAddr(strings.TrimPrefix(t, "|-- "))
} else if strings.HasPrefix(t, "/32 host LOCAL") && last.IsValid() && !seen[last] {
seen[last] = true
out = append(out, last)
}
}
}
return out
}
// SystemResolver returns the first nameserver in /etc/resolv.conf as
// host:port.
func SystemResolver() (string, error) {
@@ -162,16 +347,25 @@ func (c *Checker) exchange(ctx context.Context, m *dns.Msg, addr string, key *Ke
return c.exchangeHostPort(ctx, m, net.JoinHostPort(addr, c.port()), key)
}
func (c *Checker) exchangeHostPort(ctx context.Context, m *dns.Msg, hostport string, key *Key) (*dns.Msg, time.Duration, error) {
func (c *Checker) exchangeHostPort(ctx context.Context, m *dns.Msg, hostport string, key *Key) (resp *dns.Msg, rtt time.Duration, err error) {
if c.Watch != nil && ctx.Value(quietKey{}) == nil {
id := c.startEvent(digQuery(m, hostport, key))
defer func() {
e := Event{ID: id, RTTMs: rtt.Milliseconds()}
if err != nil {
e.Error = err.Error()
} else {
e.Answer = sumAnswer(resp)
}
c.endEvent(e)
}()
}
cl := &dns.Client{Timeout: c.timeout()}
if key != nil {
name := dns.Fqdn(key.Name)
cl.TsigSecret = map[string]string{name: key.Secret}
m.SetTsig(name, dns.Fqdn(key.Algorithm), 300, time.Now().Unix())
}
var resp *dns.Msg
var rtt time.Duration
var err error
for try := 0; try < 2; try++ {
resp, rtt, err = cl.ExchangeContext(ctx, m, hostport)
if err == nil || ctx.Err() != nil {
@@ -228,7 +422,7 @@ func (c *Checker) soa(ctx context.Context, zone string, key *Key, ref *uint32, r
if port == "" {
port = c.port()
}
resp, rtt, err := c.exchangeHostPort(ctx, m, net.JoinHostPort(r.Address, port), key)
resp, rtt, err := c.exchangeHostPort(context.WithValue(ctx, quietKey{}, true), m, net.JoinHostPort(r.Address, port), key)
if err != nil {
var ne net.Error
switch {
@@ -320,11 +514,23 @@ func (c *Checker) Serials(ctx context.Context, zone string, ref *uint32, targets
continue
}
wg.Add(1)
port := r.Port
if port == "" {
port = c.port()
}
// Reported here rather than by exchange: the end carries the
// server's status.
key := t.Key
if key == nil && c.LocalKey != nil && IsLocalAddr(ip) {
key = c.LocalKey
}
id := c.startEvent(digLine(r.Address, port, zone, "SOA", true, key))
go func(r Result, key *Key) {
defer wg.Done()
c.soa(ctx, zone, key, ref, &r)
c.endEvent(Event{ID: id, Result: &r, RTTMs: r.RTTMs})
add(r)
}(r, t.Key)
}(r, key)
}
}
wg.Wait()
+62
View File
@@ -6,6 +6,7 @@ import (
"net/netip"
"strconv"
"strings"
"sync"
"testing"
"time"
@@ -307,3 +308,64 @@ func TestDelegationDS(t *testing.T) {
}
}
}
// TestWatch: with Watch set, every query is reported as it starts and
// ends, as a dig command line with a summary of the answer.
func TestWatch(t *testing.T) {
port, res := internet(t, func(q dns.Question) *dns.Msg {
m := new(dns.Msg)
if q.Qtype == dns.TypeNS && q.Name == "adbraun.lol." {
m.Ns = []dns.RR{rr("adbraun.lol. 3600 IN NS ns1.adbraun.lol.")}
m.Extra = []dns.RR{rr("ns1.adbraun.lol. 3600 IN A 127.0.0.12")}
}
return m
})
var mu sync.Mutex
var events []Event
c := checker(port, res)
c.Watch = func(e Event) {
mu.Lock()
events = append(events, e)
mu.Unlock()
}
ref := uint32(7)
c.Delegation(context.Background(), "adbraun.lol.", []string{"ns1.adbraun.lol."}, map[string][]string{"ns1.adbraun.lol.": {"127.0.0.12"}}, &ref, nil)
open := map[int64]string{}
var queries, answers []string
for _, e := range events {
switch e.Kind {
case "start":
if !strings.HasPrefix(e.Query, "dig @") {
t.Errorf("query %q", e.Query)
}
open[e.ID] = e.Query
queries = append(queries, e.Query)
case "end":
q, ok := open[e.ID]
if !ok {
t.Errorf("end without start: %+v", e)
}
delete(open, e.ID)
a := e.Answer + e.Error
if e.Result != nil {
a = e.Result.Status
}
answers = append(answers, q+" => "+a)
}
}
if len(open) != 0 {
t.Errorf("never ended: %v", open)
}
all := strings.Join(answers, "\n")
for _, want := range []string{
" adbraun.lol. NS +norec => referral to ns1.adbraun.lol.", // asking a parent server
"@127.0.0.12 -p " + port + " adbraun.lol. SOA +norec => in_sync", // asking the delegated server
} {
if !strings.Contains(all, want) {
t.Errorf("no %q in\n%s", want, all)
}
}
if len(queries) < 4 {
t.Errorf("only %d queries:\n%s", len(queries), all)
}
}
+167
View File
@@ -0,0 +1,167 @@
package render
import (
"fmt"
"path/filepath"
"slices"
"strings"
"jarl/internal/blocking"
"jarl/internal/model"
"jarl/internal/namedconf"
)
// KindPolicy is the kind of a rendered rules policy zone file.
const KindPolicy = "policy"
// PolicyZone is a response policy zone in named's configuration
// (docs/blocking.md).
type PolicyZone struct {
Name string // without the final dot
ViewID int64 // 0 without views
View string // "" without views
File string
ListID int64 // the blocklist; 0 for the rules zone
}
// BlocklistsDir holds the zone files of downloaded blocklists, one per
// list, shared by the views that use it.
func (p Paths) BlocklistsDir() string { return filepath.Join(p.CacheDir, "blocklists") }
// BlocklistFile returns the zone file of a blocklist. It is named after
// the list's ID, so renaming the list doesn't move it.
func (p Paths) BlocklistFile(id int64) string {
return filepath.Join(p.BlocklistsDir(), fmt.Sprintf("list-%d.rpz", id))
}
// rulesFile returns a view's rules zone file.
func (p Paths) rulesFile(view string) string {
return filepath.Join(p.ZoneDir, ViewDir(view), blocking.RulesZone+".rpz")
}
// PolicyZones returns the policy zones named loads, per view in view
// order, each view's in the order named checks them: the rules zone first
// (rules win over lists), then the enabled lists by name.
func PolicyZones(s *model.State, p Paths) []PolicyZone {
var out []PolicyZone
rules, lists := s.SortedBlockRules(), s.SortedBlocklists()
scopes := []int64{0}
if len(s.Views) > 0 {
scopes = nil
for _, v := range s.OrderedViews() {
scopes = append(scopes, v.ID)
}
}
for _, vid := range scopes {
view := ""
if v := s.Views[vid]; v != nil {
view = v.Name
}
var zones []PolicyZone
hasRules := false
for _, r := range rules {
if slices.Contains(s.BlockingViews(r.ViewIDs), vid) {
hasRules = true
break
}
}
for _, b := range lists {
if !b.Disabled && slices.Contains(s.BlockingViews(b.ViewIDs), vid) {
zones = append(zones, PolicyZone{Name: blocking.ListZone(b.Name), ViewID: vid, View: view, File: p.BlocklistFile(b.ID), ListID: b.ID})
}
}
// While paused, every view with blocking gets a rules zone: it
// lets everything through.
if hasRules || s.Settings.BlockingPaused() && len(zones) > 0 {
out = append(out, PolicyZone{Name: blocking.RulesZone, ViewID: vid, View: view, File: p.rulesFile(view)})
}
out = append(out, zones...)
}
return out
}
// ViewRules returns the rules that apply in a view (0 without views).
func ViewRules(s *model.State, viewID int64) []blocking.Rule {
var out []blocking.Rule
for _, r := range s.SortedBlockRules() {
if slices.Contains(s.BlockingViews(r.ViewIDs), viewID) {
out = append(out, blocking.Rule{Name: r.Name, Action: r.Action, Exact: r.Exact})
}
}
return out
}
// writeResponsePolicy writes the response-policy statement for a view's
// policy zones (or the server's, without views). Rewrites are logged to
// Jarl's response log when statistics read it, and not at all otherwise
// (they would go to named's default log).
func writeResponsePolicy(w *confWriter, zones []PolicyZone, logRewrites bool) {
if len(zones) == 0 {
return
}
w.begin("blocking", 0, "blocking (rules and blocklists)")
w.write("\t// Blocking (docs/blocking.md): rules first, then blocklists.\n\tresponse-policy {\n")
for _, z := range zones {
// Apply a new version at once: Jarl reloads a list only when it
// changed (at most every blocking.refresh), so named's default
// delay for batching frequent updates only adds a wait.
opts := []string{"min-update-interval 0"}
if !logRewrites {
opts = append(opts, "log no")
}
w.write(fmt.Sprintf("\t\tzone %s%s;\n", namedconf.QuoteString(z.Name), prefixSpace(strings.Join(opts, " "))))
}
// Block without waiting for the real answer, and also for clients
// that ask for DNSSEC data.
w.write("\t} qname-wait-recurse no break-dnssec yes;\n")
w.end()
}
func prefixSpace(s string) string {
if s == "" {
return ""
}
return " " + s
}
// writePolicyZones writes the zone statements of policy zones and adds
// their files to the output: rules zone files are rendered, blocklist
// files only listed (jarld downloads them; see Output.Blocklists).
func writePolicyZones(w *confWriter, s *model.State, zones []PolicyZone, out *Output, indent int) {
tabs := strings.Repeat("\t", indent)
for _, z := range zones {
w.write("\n")
label := "blocking: your rules"
if z.ListID != 0 {
label = "blocking: blocklist " + s.Blocklists[z.ListID].Name
}
w.begin("blocking", z.ListID, label)
w.write(fmt.Sprintf("%szone %s {\n%s\ttype primary;\n%s\tfile %s;\n%s\tallow-query { none; };\n%s\tallow-transfer { none; };\n%s\tnotify no;\n%s};\n",
tabs, namedconf.QuoteString(z.Name), tabs, tabs, namedconf.QuoteString(z.File), tabs, tabs, tabs, tabs))
w.end()
if z.ListID != 0 {
out.Blocklists = appendUnique(out.Blocklists, z.File)
continue
}
title := "Blocking rules"
if z.View != "" {
title += " for view " + z.View
}
content := blocking.RulesZoneFile(title, ViewRules(s, z.ViewID))
if s.Settings.BlockingPaused() {
content = blocking.PausedZoneFile(title)
}
out.Files = append(out.Files, &File{Path: z.File, Content: []byte(content), Mode: 0o640, Kind: KindPolicy})
}
}
// scopeZones returns the policy zones of one view (0 without views).
func scopeZones(all []PolicyZone, viewID int64) []PolicyZone {
var out []PolicyZone
for _, z := range all {
if z.ViewID == viewID {
out = append(out, z)
}
}
return out
}
+32 -8
View File
@@ -69,6 +69,11 @@ type Output struct {
Spans map[string][]Span
// ZoneFiles maps zone IDs to the file named loads for the zone.
ZoneFiles map[int64]string
// Policy lists the response policy zones (docs/blocking.md).
Policy []PolicyZone
// Blocklists are the zone files of blocklists named loads. jarld
// downloads them; until then they must exist, as empty policy zones.
Blocklists []string
}
// File returns the rendered file at path, or nil.
@@ -144,7 +149,7 @@ const headerText = "Generated by Jarl. DO NOT EDIT: changes made here are overwr
// from jarld's configuration (e.g. for query statistics).
func Render(s *model.State, p Paths, x Extras) *Output {
out := &Output{Spans: map[string][]Span{}, ZoneFiles: map[int64]string{}}
keys := renderKeys(s)
keys := renderKeys(s) + renderViewKeys(s, x)
out.Files = append(out.Files, &File{Path: p.KeysConfig(), Content: []byte(keys), Mode: 0o640, Kind: KindKeys, Secret: true})
conf, spans := renderConfig(s, p, x, out)
out.Files = append(out.Files, &File{Path: p.MainConfig(), Content: []byte(conf), Mode: 0o640, Kind: KindConfig})
@@ -264,6 +269,9 @@ func renderConfig(s *model.State, p Paths, x Extras, out *Output) (string, []Spa
st := x.Stats
logResponses := st != nil && st.LogFile != ""
pzs := PolicyZones(s, p)
out.Policy = pzs
logRewrites := logResponses && len(pzs) > 0
sawStats, sawLogging := false, false
if top := s.ScopedOptions(model.ScopeTopLevel, 0); len(top) > 0 {
w.write("\n")
@@ -274,7 +282,7 @@ func renderConfig(s *model.State, p Paths, x Extras, out *Output) (string, []Spa
w.write(st.statsStatement(o))
sawStats = true
case logResponses && o.Name == "logging" && !sawLogging:
w.write(st.loggingStatement(o))
w.write(st.loggingStatement(o, logRewrites))
sawLogging = true
default:
w.write(optionText(o, 0))
@@ -289,7 +297,7 @@ func renderConfig(s *model.State, p Paths, x Extras, out *Output) (string, []Spa
w.write(st.statsStatement(nil))
}
if logResponses && !sawLogging {
w.write(st.loggingStatement(nil))
w.write(st.loggingStatement(nil, logRewrites))
}
w.end()
}
@@ -322,10 +330,14 @@ func renderConfig(s *model.State, p Paths, x Extras, out *Output) (string, []Spa
w.write("\t// " + statsComment + "\n\tresponselog yes;\n")
w.end()
}
w.write("};\n")
views := s.OrderedViews()
if len(views) == 0 {
writeResponsePolicy(w, scopeZones(pzs, 0), logResponses)
}
w.write("};\n")
if len(views) == 0 {
writePolicyZones(w, s, scopeZones(pzs, 0), out, 0)
for _, z := range s.ZonesInView(0) {
w.write("\n")
renderZone(w, s, z, p, out, 0)
@@ -338,8 +350,8 @@ func renderConfig(s *model.State, p Paths, x Extras, out *Output) (string, []Spa
w.write(commentBlock("//", v.Comment))
}
w.write(fmt.Sprintf("view %s {\n", namedconf.QuoteString(v.Name)))
if len(v.MatchClients) > 0 {
writeAML(w, "match-clients", v.MatchClients)
if mc := viewMatchClients(s, x, v); len(mc) > 0 {
writeAML(w, "match-clients", mc)
}
if len(v.MatchDestinations) > 0 {
writeAML(w, "match-destinations", v.MatchDestinations)
@@ -352,6 +364,8 @@ func renderConfig(s *model.State, p Paths, x Extras, out *Output) (string, []Spa
w.write(optionText(o, 1))
w.end()
}
writeResponsePolicy(w, scopeZones(pzs, v.ID), logResponses)
writePolicyZones(w, s, scopeZones(pzs, v.ID), out, 1)
for _, z := range s.ZonesInView(v.ID) {
w.write("\n")
renderZone(w, s, z, p, out, 1)
@@ -384,6 +398,12 @@ func renderZone(w *confWriter, s *model.State, z *model.Zone, p Paths, out *Outp
if v := s.ViewName(z); v != "" {
label += " [view " + v + "]"
}
if z.Disabled {
// Kept in Jarl (its zone file too, so enabling it is quick), not
// loaded by named.
w.write(fmt.Sprintf("%s// zone %s is disabled in Jarl: named doesn't load it.\n", tabs, namedconf.QuoteString(dnsname.Display(z.Name))))
return
}
w.begin("zone", z.ID, label)
if z.Comment != "" {
w.write(indentLines(commentBlock("//", z.Comment), indent))
@@ -461,6 +481,10 @@ func ZoneFile(s *model.State, z *model.Zone) string {
if r.Comment != "" {
line += "\t; " + oneLine(r.Comment)
}
if r.Disabled {
// Kept in Jarl, left out of the zone.
line = "; disabled: " + line
}
sb.WriteString(line + "\n")
}
return sb.String()
@@ -478,7 +502,7 @@ func ZoneContentHash(s *model.State, z *model.Zone) string {
h := sha256.New()
soa := z.SOA
fmt.Fprintf(h, "%s|%d|%s|%s|%d|%d|%d|%d\n", z.Name, z.DefaultTTL, soa.MName, soa.RName, soa.Refresh, soa.Retry, soa.Expire, soa.Minimum)
for _, r := range s.ZoneRecords(z.ID) {
for _, r := range s.ServedRecords(z.ID) {
fmt.Fprintf(h, "%s|%d|%s|%s\n", r.Name, r.EffectiveTTL(z), r.Type, r.Data)
}
return hex.EncodeToString(h.Sum(nil))
+20 -10
View File
@@ -15,6 +15,9 @@ type Extras struct {
// Stats, when set, makes named publish what Jarl's statistics
// collector reads.
Stats *StatsHooks
// ViewKeySeed, when set, gives every view a TSIG key that selects it
// (viewkeys.go).
ViewKeySeed []byte
}
// StatsHooks describe the statistics channel and response log Jarl needs.
@@ -69,8 +72,9 @@ func (h *StatsHooks) statsStatement(user *model.Option) string {
}
// loggingStatement returns the logging statement with Jarl's channel and
// the responses category, merged into the user's own if there is one.
func (h *StatsHooks) loggingStatement(user *model.Option) string {
// the responses category (and rpz, for blocking's rewrites), merged into
// the user's own if there is one.
func (h *StatsHooks) loggingStatement(user *model.Option, rpz bool) string {
var st *namedconf.Stmt
if user != nil {
var err error
@@ -84,15 +88,21 @@ func (h *StatsHooks) loggingStatement(user *model.Option) string {
b := st.FirstBlock()
// Channels must be defined before the categories that use them.
b.Stmts = append([]*namedconf.Stmt{h.logChannel()}, b.Stmts...)
merged := false
for _, s := range b.Stmts {
if s.Keyword() == "category" && strings.EqualFold(s.Arg(0), "responses") && s.FirstBlock() != nil {
s.FirstBlock().Stmts = append(s.FirstBlock().Stmts, namedconf.NewStmt(StatsChannelName))
merged = true
}
categories := []string{"responses"}
if rpz {
categories = append(categories, "rpz")
}
if !merged {
b.Stmts = append(b.Stmts, namedconf.NewStmt("category", "responses", namedconf.NewBlock(namedconf.NewStmt(StatsChannelName))))
for _, cat := range categories {
merged := false
for _, s := range b.Stmts {
if s.Keyword() == "category" && strings.EqualFold(s.Arg(0), cat) && s.FirstBlock() != nil {
s.FirstBlock().Stmts = append(s.FirstBlock().Stmts, namedconf.NewStmt(StatsChannelName))
merged = true
}
}
if !merged {
b.Stmts = append(b.Stmts, namedconf.NewStmt("category", cat, namedconf.NewBlock(namedconf.NewStmt(StatsChannelName))))
}
}
prefix := ""
if user != nil {
+84
View File
@@ -0,0 +1,84 @@
package render
import (
"crypto/hmac"
"crypto/sha256"
"encoding/base64"
"strconv"
"strings"
"jarl/internal/model"
"jarl/internal/namedconf"
)
// View keys let Jarl choose a view regardless of where a query comes from:
// each view gets a TSIG key, and its match-clients starts with that key
// (after excluding the keys of the views listed after it, so no earlier
// view takes their queries). Queries signed with a view's key always land
// in that view. Jarl signs its own checks of this server's addresses with
// them (a query from this host would otherwise land in whichever view
// matches this host's address), and you can too: dig -k.
//
// The secrets are derived from a random seed kept in Jarl's database
// (Extras.ViewKeySeed), so they stay the same from one apply to the next
// and need no storage of their own. They are in keys.conf with the other
// keys.
// ViewKeyAlgorithm is the view keys' algorithm.
const ViewKeyAlgorithm = "hmac-sha256"
// ViewKeyName returns the name of a view's key.
func ViewKeyName(view string) string { return model.ViewKeyName(view) }
// IsViewKey reports whether a key name is a view key's.
func IsViewKey(name string) bool {
return strings.HasPrefix(strings.ToLower(strings.TrimSuffix(name, ".")), model.ViewKeyPrefix)
}
// ViewKeySecret returns the secret of the key of the view with this ID.
func ViewKeySecret(seed []byte, viewID int64) string {
m := hmac.New(sha256.New, seed)
m.Write([]byte("jarl view key " + strconv.FormatInt(viewID, 10)))
return base64.StdEncoding.EncodeToString(m.Sum(nil))
}
// viewKeysOn reports whether view keys are rendered.
func viewKeysOn(s *model.State, x Extras) bool { return len(x.ViewKeySeed) > 0 && len(s.Views) > 0 }
// renderViewKeys returns the key statements of the views' keys.
func renderViewKeys(s *model.State, x Extras) string {
if !viewKeysOn(s, x) {
return ""
}
var sb strings.Builder
sb.WriteString("\n" + commentBlock("//", "View keys: a query signed with one of these always lands in its view.\nJarl signs its own checks of this server with them; dig -k works too."))
for _, v := range s.OrderedViews() {
sb.WriteString("key " + namedconf.QuoteString(ViewKeyName(v.Name)) + " {\n\talgorithm " + ViewKeyAlgorithm + ";\n\tsecret " + namedconf.QuoteString(ViewKeySecret(x.ViewKeySeed, v.ID)) + ";\n};\n")
}
return sb.String()
}
// viewMatchClients returns a view's match-clients with the view keys in
// front: "!key" for the keys of the views after it, then its own key, then
// the view's own list ("any" when it has none, which is what an absent
// match-clients means).
func viewMatchClients(s *model.State, x Extras, v *model.View) []string {
if !viewKeysOn(s, x) {
return v.MatchClients
}
var out []string
after := false
for _, o := range s.OrderedViews() {
if after {
out = append(out, "!key "+namedconf.QuoteString(ViewKeyName(o.Name)))
}
if o.ID == v.ID {
after = true
}
}
out = append(out, "key "+namedconf.QuoteString(ViewKeyName(v.Name)))
if len(v.MatchClients) == 0 {
return append(out, "any")
}
return append(out, v.MatchClients...)
}
+51
View File
@@ -0,0 +1,51 @@
package render
import (
"strings"
"testing"
"jarl/internal/model"
)
func TestViewKeys(t *testing.T) {
s := model.NewState()
s.Views[1] = &model.View{ID: 1, Name: "internal", Position: 0, MatchClients: []string{"homelab", "localhost"}}
s.Views[2] = &model.View{ID: 2, Name: "external", Position: 1}
p := Paths{ConfigDir: "/etc/bind/jarl", ZoneDir: "/var/lib/bind/jarl", CacheDir: "/var/cache/bind/jarl"}
seed := []byte("0123456789abcdef0123456789abcdef")
files := func(x Extras) (conf, keys string) {
for _, f := range Render(s, p, x).Files {
switch f.Kind {
case KindConfig:
conf = string(f.Content)
case KindKeys:
keys = string(f.Content)
}
}
return
}
conf, keys := files(Extras{ViewKeySeed: seed})
for _, want := range []string{
// An earlier view mustn't take a later view's key.
"match-clients {\n\t\t!key jarl-view-external;\n\t\tkey jarl-view-internal;\n\t\thomelab;\n\t\tlocalhost;\n\t};",
// A view without a list matched everyone: it still does.
"match-clients { key jarl-view-external; any; };",
} {
if !strings.Contains(conf, want) {
t.Errorf("named.conf lacks\n%s\n---\n%s", want, conf)
}
}
sec := ViewKeySecret(seed, 2)
if !strings.Contains(keys, "key \"jarl-view-external\" {\n\talgorithm hmac-sha256;\n\tsecret \""+sec+"\";\n};") || !strings.Contains(keys, "jarl-view-internal") {
t.Errorf("keys.conf:\n%s", keys)
}
// The same seed gives the same secret; another seed doesn't.
if ViewKeySecret(seed, 2) != sec || ViewKeySecret([]byte("other"), 2) == sec || ViewKeySecret(seed, 1) == sec {
t.Error("secrets aren't derived from the seed and the view")
}
// Without a seed (or without views), nothing changes.
conf, keys = files(Extras{})
if strings.Contains(conf, "jarl-view-") || strings.Contains(keys, "jarl-view-") {
t.Errorf("view keys without a seed:\n%s", conf)
}
}
+111
View File
@@ -0,0 +1,111 @@
package server
import (
"net/http"
"net/http/httptest"
"testing"
)
func TestBlockingAPI(t *testing.T) {
e := newAPIEnv(t)
lists := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/missing" {
http.NotFound(w, r)
return
}
w.Write([]byte("||ads.example^\n0.0.0.0 tracker.example\n"))
}))
defer lists.Close()
b := e.do("POST", "/blocking/lists", e.token, map[string]any{"name": "ads", "url": lists.URL + "/list.txt"}, 201)["object"].(map[string]any)
if b["name"] != "ads" || b["state"] != "waiting" {
t.Errorf("new list = %v", b)
}
got := e.do("POST", "/blocking/lists/ads/refresh", e.token, nil, 200)
st, _ := got["status"].(map[string]any)
if got["state"] != "ok" || st["entries"] != 2.0 || st["records"] != 3.0 {
t.Errorf("after refresh = %v", got)
}
e.do("POST", "/blocking/rules", e.token, map[string]any{"name": "Good.Ads.Example.", "action": "allow"}, 201)
r := e.do("POST", "/blocking/rules", e.token, map[string]any{"name": "more.example", "exact": true}, 201)["object"].(map[string]any)
if r["name"] != "more.example" || r["action"] != "block" || r["exact"] != true {
t.Errorf("new rule = %v", r)
}
for name, want := range map[string]string{"x.ads.example": "blocked", "a.good.ads.example": "allowed", "more.example": "blocked", "sub.more.example": "none"} {
m := e.list("/blocking/check?name="+name, e.token)
if len(m) != 1 || m[0]["result"] != want {
t.Errorf("check %s = %v, want %s", name, m, want)
}
}
if _, status := e.req("GET", "/blocking/check?name=com", e.token, nil); status != 400 {
t.Errorf("check of a TLD = %d", status)
}
// Staged changes show as pending.
cs := itoa(int64(e.do("POST", "/changesets", e.token, map[string]any{"description": "more"}, 201)["id"].(float64)))
e.do("POST", "/blocking/rules?changeset="+cs, e.token, map[string]any{"name": "staged.example"}, 202)
e.do("PATCH", "/blocking/lists/ads?changeset="+cs, e.token, map[string]any{"disabled": true}, 202)
pending := map[string]any{}
for _, x := range e.list("/blocking/rules?changeset="+cs, e.token) {
pending[x["name"].(string)] = x["pending"]
}
if pending["staged.example"] != "add" || pending["more.example"] != nil {
t.Errorf("pending rules = %v", pending)
}
if l := e.list("/blocking/lists?changeset="+cs, e.token); len(l) != 1 || l[0]["pending"] != "change" || l[0]["state"] != "disabled" {
t.Errorf("pending list = %v", l)
}
e.do("POST", "/changesets/"+cs+"/discard", e.token, nil, 204)
// Delete rules by name; a missing one is a 404 unless missing_ok.
e.do("DELETE", "/blocking/rules?name=more.example", e.token, nil, 200)
if _, status := e.req("DELETE", "/blocking/rules?name=more.example", e.token, nil); status != 404 {
t.Errorf("deleting a missing rule = %d", status)
}
e.do("DELETE", "/blocking/rules?name=more.example&missing_ok=1", e.token, nil, 200)
// Readers see blocking but can't change it.
e.do("POST", "/users", e.token, map[string]any{"username": "reader", "password": "reader-password-123", "grant_strings": []string{"blocking:read"}}, 201)
var readerID int64
for _, u := range e.list("/users", e.token) {
if u["username"] == "reader" {
readerID = int64(u["id"].(float64))
}
}
rt := e.do("POST", "/users/"+itoa(readerID)+"/tokens", e.token, map[string]any{"name": "t"}, 201)["token"].(string)
if l := e.list("/blocking/lists", rt); len(l) != 1 {
t.Errorf("reader's lists = %v", l)
}
for _, c := range []struct{ method, path string }{{"POST", "/blocking/rules"}, {"POST", "/blocking/lists/ads/refresh"}, {"DELETE", "/blocking/lists/ads"}} {
if _, status := e.req(c.method, c.path, rt, map[string]any{"name": "x.example"}); status != 403 {
t.Errorf("reader %s %s = %d", c.method, c.path, status)
}
}
// Pausing.
if _, status := e.req("POST", "/blocking/pause", e.token, map[string]any{"minutes": 0}); status != 400 {
t.Errorf("a 0-minute pause = %d", status)
}
e.do("POST", "/blocking/pause", e.token, map[string]any{"minutes": 30}, 201)
if st := e.do("GET", "/blocking/status", e.token, nil, 200); st["paused_until"] == nil {
t.Errorf("status while paused = %v", st)
}
if st := e.do("GET", "/status", e.token, nil, 200); st["blocking_paused_until"] == nil {
t.Errorf("dashboard while paused = %v", st["blocking_paused_until"])
}
e.do("POST", "/blocking/resume", e.token, nil, 201)
if st := e.do("GET", "/blocking/status", e.token, nil, 200); st["paused_until"] != nil {
t.Errorf("status after resuming = %v", st)
}
// A list that can't be downloaded shows on the dashboard.
e.do("POST", "/blocking/lists", e.token, map[string]any{"name": "gone", "url": lists.URL + "/missing"}, 201)
if l := e.do("POST", "/blocking/lists/gone/refresh", e.token, nil, 200); l["state"] != "error" {
t.Errorf("failed list = %v", l)
}
errs, _ := e.do("GET", "/status", e.token, nil, 200)["blocklist_errors"].([]any)
if len(errs) != 1 || errs[0].(map[string]any)["name"] != "gone" {
t.Errorf("status blocklist_errors = %v", errs)
}
e.do("DELETE", "/blocking/lists/gone", e.token, nil, 200)
e.do("DELETE", "/blocking/lists/ads", e.token, nil, 200)
}
+38
View File
@@ -0,0 +1,38 @@
package server
import (
"net/netip"
"testing"
"jarl/internal/model"
)
func TestClientNames(t *testing.T) {
st := model.NewState()
add := func(id int64, name, typ, data string, disabled bool) {
st.Records[id] = &model.Record{ID: id, ZoneID: 1, Name: name, Type: typ, Data: data, Disabled: disabled}
}
add(1, "nas.adbraun.lol.", "A", "192.168.1.20", false)
add(2, "20.1.168.192.in-addr.arpa.", "PTR", "storage.adbraun.lol.", false) // PTR wins over A
add(3, "web.adbraun.lol.", "A", "192.168.1.30", false) // no PTR: the A record's name
add(4, "alias.adbraun.lol.", "A", "192.168.1.30", false) // first in name order wins
add(5, "old.adbraun.lol.", "A", "192.168.1.40", true) // disabled: ignored
add(6, "secret.adbraun.lol.", "A", "192.168.1.50", false) // not readable
add(7, "v6.adbraun.lol.", "AAAA", "2001:db8::7", false)
add(8, "printer.adbraun.lol.", "A", "192.168.1.60", false) // not asked for
want := map[netip.Addr]bool{}
for _, a := range []string{"192.168.1.20", "192.168.1.30", "192.168.1.40", "192.168.1.50", "2001:db8::7", "10.0.0.1"} {
want[netip.MustParseAddr(a)] = true
}
got := clientNames(st, want, func(r *model.Record) bool { return r.ID != 6 })
exp := map[string]string{"192.168.1.20": "storage.adbraun.lol", "192.168.1.30": "alias.adbraun.lol", "2001:db8::7": "v6.adbraun.lol"}
if len(got) != len(exp) {
t.Errorf("got %v", got)
}
for a, n := range exp {
if got[netip.MustParseAddr(a)] != n {
t.Errorf("%s = %q, want %q", a, got[netip.MustParseAddr(a)], n)
}
}
}
+61
View File
@@ -0,0 +1,61 @@
package server
import (
"strings"
"testing"
)
func TestDisableRecords(t *testing.T) {
e := newAPIEnv(t)
e.do("POST", "/zones", e.token, map[string]any{"name": "example.com", "records": []map[string]any{{"name": "ns1", "type": "A", "data": "192.0.2.1"}}}, 201)
www := e.do("POST", "/zones/example.com/records", e.token, map[string]any{"name": "www", "type": "A", "data": "192.0.2.10"}, 201)["object"].(map[string]any)
id := itoa(int64(www["id"].(float64)))
// By ID: the record comes back disabled.
if r := e.do("POST", "/records/"+id+"/disable", e.token, nil, 201)["object"].(map[string]any); r["disabled"] != true {
t.Errorf("disabled record = %v", r)
}
if _, status := e.req("POST", "/records/"+id+"/disable", e.token, nil); status != 409 {
t.Errorf("disabling twice = %d, want 409", status)
}
if z := e.do("GET", "/zones/example.com", e.token, nil, 200); z["disabled_count"] != 1.0 || z["record_count"] != 3.0 {
t.Errorf("zone counts = %v / %v", z["record_count"], z["disabled_count"])
}
off := e.list("/zones/example.com/records?disabled=1", e.token)
on := e.list("/zones/example.com/records?disabled=0", e.token)
if len(off) != 1 || off[0]["name"] != "www.example.com." || len(on) != 2 {
t.Errorf("filtered: %d disabled, %d enabled", len(off), len(on))
}
file := e.do("GET", "/zones/example.com/file", e.token, nil, 200)
if s, _ := file["_raw"].(string); !strings.Contains(s, "; disabled: www") {
t.Errorf("zone file = %v", file)
}
// Staged: listing with the changeset marks the record.
cs := itoa(int64(e.do("POST", "/changesets", e.token, map[string]any{"description": "back"}, 201)["id"].(float64)))
e.do("POST", "/records/"+id+"/enable?changeset="+cs, e.token, nil, 202)
for _, r := range e.list("/zones/example.com/records?changeset="+cs, e.token) {
if r["name"] == "www.example.com." && (r["pending"] != "enable" || r["disabled"] == true) {
t.Errorf("staged record = %v", r)
}
}
e.do("POST", "/changesets/"+cs+"/discard", e.token, nil, 204)
// By match.
e.do("POST", "/zones/example.com/records/enable?name=www&type=A", e.token, nil, 201)
if _, status := e.req("POST", "/zones/example.com/records/disable?name=nope&type=A", e.token, nil); status != 404 {
t.Errorf("no match = %d", status)
}
if _, status := e.req("POST", "/zones/example.com/records/disable?name=www", e.token, nil); status != 400 {
t.Errorf("no type = %d", status)
}
e.do("POST", "/zones/example.com/records/disable?name=www&type=A&data=192.0.2.10", e.token, nil, 201)
if r := e.do("GET", "/records/"+id, e.token, nil, 200); r["disabled"] != true {
t.Errorf("after match disable = %v", r)
}
// And in a batch.
e.do("POST", "/apply", e.token, map[string]any{"ops": []map[string]any{{"op": "record.enable", "id": www["id"]}}}, 201)
if r := e.do("GET", "/records/"+id, e.token, nil, 200); r["disabled"] == true {
t.Errorf("after batch enable = %v", r)
}
}
+1 -1
View File
@@ -157,7 +157,7 @@ func primaryZonesFor(st *model.State, p *auth.Principal) []acme.Zone {
var out []acme.Zone
for _, z := range st.SortedZones() {
vn := st.ViewName(z)
if z.Type == model.ZonePrimary && p.CanSeeZone(z.Name, vn) {
if z.Type == model.ZonePrimary && !z.Disabled && p.CanSeeZone(z.Name, vn) {
out = append(out, acme.Zone{Name: z.Name, View: vn})
}
}
+409
View File
@@ -0,0 +1,409 @@
package server
import (
"context"
"net/http"
"slices"
"strconv"
"strings"
"time"
"jarl/internal/api"
"jarl/internal/auth"
"jarl/internal/blocking"
"jarl/internal/engine"
"jarl/internal/model"
)
// ---- blocking (docs/blocking.md)
// bytesPerRecord is what named 9.20 needs per policy zone record and view,
// measured with lists of a few hundred thousand entries.
const bytesPerRecord = 630
func viewNames(st *model.State, ids []int64) []string {
out := []string{}
for _, id := range ids {
if v := st.Views[id]; v != nil {
out = append(out, v.Name)
}
}
return out
}
func presentBlocklist(st *model.State, b *model.Blocklist, status *model.BlocklistStatus) api.Blocklist {
views := st.BlockingViews(b.ViewIDs)
out := api.Blocklist{Blocklist: *b, Views: viewNames(st, views), AutoViews: len(b.ViewIDs) == 0 && len(st.Views) > 0, Status: status}
switch {
case b.Disabled:
out.State = "disabled"
case len(views) == 0:
out.State = "idle"
case status != nil && status.Error != "":
out.State = "error"
case status == nil || status.UpdatedAt == nil:
out.State = "waiting"
default:
out.State = "ok"
}
if status != nil && !b.Disabled {
out.MemoryMB = int((int64(status.Records)*bytesPerRecord*int64(len(views)) + 1<<19) >> 20)
}
return out
}
func presentBlockRule(st *model.State, r *model.BlockRule) api.BlockRule {
return api.BlockRule{BlockRule: *r, Views: viewNames(st, st.BlockingViews(r.ViewIDs)), AutoViews: len(r.ViewIDs) == 0 && len(st.Views) > 0}
}
func requireBlocking(p *auth.Principal) error {
return require(p, auth.Action{Resource: auth.ResBlocking})
}
func (s *Server) listBlocklists(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
if err := requireBlocking(p); err != nil {
return err
}
st, base, err := s.states(r)
if err != nil {
return err
}
statuses, err := s.store.BlocklistStatuses(r.Context())
if err != nil {
return err
}
out := []api.Blocklist{}
for _, b := range st.SortedBlocklists() {
o := presentBlocklist(st, b, statuses[b.ID])
o.Pending = pending(base, base != nil && base.Blocklists[b.ID] == nil, base != nil && base.Blocklists[b.ID] != nil && base.Blocklists[b.ID].Version != b.Version)
out = append(out, o)
}
if base != nil {
for _, b := range base.SortedBlocklists() {
if st.Blocklists[b.ID] == nil {
o := presentBlocklist(base, b, statuses[b.ID])
o.Pending = "delete"
out = append(out, o)
}
}
}
writeJSON(w, http.StatusOK, out)
return nil
}
// pending returns the pending marker of an object listed with a changeset.
func pending(base *model.State, added, changed bool) string {
switch {
case base == nil:
return ""
case added:
return "add"
case changed:
return "change"
}
return ""
}
func (s *Server) blocklistByPath(r *http.Request, st *model.State) (*model.Blocklist, error) {
ref := r.PathValue("id")
var b *model.Blocklist
if id, err := strconv.ParseInt(ref, 10, 64); err == nil {
b = st.Blocklists[id]
} else {
b = st.BlocklistByName(ref)
}
if b == nil {
return nil, notFound("blocklist %s not found", ref)
}
return b, nil
}
func (s *Server) blocklistObject(ctx context.Context) func(*model.State, int64) any {
return func(st *model.State, id int64) any {
b := st.Blocklists[id]
if b == nil {
return nil
}
statuses, _ := s.store.BlocklistStatuses(ctx)
return presentBlocklist(st, b, statuses[id])
}
}
func (s *Server) getBlocklist(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
if err := requireBlocking(p); err != nil {
return err
}
st, err := s.state(r)
if err != nil {
return err
}
b, err := s.blocklistByPath(r, st)
if err != nil {
return err
}
writeJSON(w, http.StatusOK, s.blocklistObject(r.Context())(st, b.ID))
return nil
}
func (s *Server) createBlocklist(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
body, err := readBody(r)
if err != nil {
return err
}
return s.mutate(w, r, p, "", []engine.Request{request(engine.OpBlocklistCreate, 0, body)}, s.blocklistObject(r.Context()))
}
func (s *Server) updateBlocklist(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
st, err := s.state(r)
if err != nil {
return err
}
b, err := s.blocklistByPath(r, st)
if err != nil {
return err
}
body, err := readBody(r)
if err != nil {
return err
}
id := b.ID
return s.mutate(w, r, p, "", []engine.Request{request(engine.OpBlocklistUpdate, id, body)}, func(st *model.State, _ int64) any { return s.blocklistObject(r.Context())(st, id) })
}
func (s *Server) deleteBlocklist(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
st, err := s.state(r)
if err != nil {
return err
}
b, err := s.blocklistByPath(r, st)
if err != nil {
return err
}
return s.mutate(w, r, p, "", []engine.Request{request(engine.OpBlocklistDelete, b.ID, nil)}, nil)
}
// refreshBlocklist: POST /blocking/lists/{id}/refresh[?force=1] downloads
// the list now and waits for the result.
func (s *Server) refreshBlocklist(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
if err := require(p, auth.Action{Resource: auth.ResBlocking, Write: true}); err != nil {
return err
}
st, err := s.store.LoadState(r.Context())
if err != nil {
return err
}
b, err := s.blocklistByPath(r, st)
if err != nil {
return err
}
ctx, cancel := context.WithTimeout(r.Context(), 3*time.Minute)
defer cancel()
status, rerr := s.eng.RefreshBlocklist(ctx, b.ID, queryBool(r, "force"))
if status == nil {
return rerr
}
writeJSON(w, http.StatusOK, presentBlocklist(st, b, status))
return nil
}
func (s *Server) listBlockRules(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
if err := requireBlocking(p); err != nil {
return err
}
st, base, err := s.states(r)
if err != nil {
return err
}
out := []api.BlockRule{}
for _, rule := range st.SortedBlockRules() {
o := presentBlockRule(st, rule)
o.Pending = pending(base, base != nil && base.BlockRules[rule.ID] == nil, base != nil && base.BlockRules[rule.ID] != nil && base.BlockRules[rule.ID].Version != rule.Version)
out = append(out, o)
}
if base != nil {
for _, rule := range base.SortedBlockRules() {
if st.BlockRules[rule.ID] == nil {
o := presentBlockRule(base, rule)
o.Pending = "delete"
out = append(out, o)
}
}
}
writeJSON(w, http.StatusOK, out)
return nil
}
func ruleObject(st *model.State, id int64) any {
if rule := st.BlockRules[id]; rule != nil {
return presentBlockRule(st, rule)
}
return nil
}
func (s *Server) ruleByPath(r *http.Request, st *model.State) (*model.BlockRule, error) {
id, _ := strconv.ParseInt(r.PathValue("id"), 10, 64)
rule := st.BlockRules[id]
if rule == nil {
return nil, notFound("rule %s not found", r.PathValue("id"))
}
return rule, nil
}
func (s *Server) createBlockRule(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
body, err := readBody(r)
if err != nil {
return err
}
return s.mutate(w, r, p, "", []engine.Request{request(engine.OpBlockRuleCreate, 0, body)}, ruleObject)
}
func (s *Server) updateBlockRule(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
st, err := s.state(r)
if err != nil {
return err
}
rule, err := s.ruleByPath(r, st)
if err != nil {
return err
}
body, err := readBody(r)
if err != nil {
return err
}
id := rule.ID
return s.mutate(w, r, p, "", []engine.Request{request(engine.OpBlockRuleUpdate, id, body)}, func(st *model.State, _ int64) any { return ruleObject(st, id) })
}
func (s *Server) deleteBlockRule(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
st, err := s.state(r)
if err != nil {
return err
}
rule, err := s.ruleByPath(r, st)
if err != nil {
return err
}
return s.mutate(w, r, p, "", []engine.Request{request(engine.OpBlockRuleDelete, rule.ID, nil)}, nil)
}
// deleteBlockRulesByName: DELETE /blocking/rules?name=NAME[&view=VIEW]
// deletes the rules for a domain (in one view, or all).
func (s *Server) deleteBlockRulesByName(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
st, err := s.state(r)
if err != nil {
return err
}
q := r.URL.Query()
name := strings.TrimSuffix(strings.ToLower(strings.TrimSpace(q.Get("name"))), ".")
if name == "" {
return badRequest("give the domain of the rules to delete (name=)")
}
var viewID int64 = -1
if vn := q.Get("view"); vn != "" {
v := st.ViewByName(vn)
if v == nil {
return notFound("view %s not found", vn)
}
viewID = v.ID
}
var reqs []engine.Request
for _, rule := range st.SortedBlockRules() {
if rule.Name != name {
continue
}
if viewID >= 0 && !slices.Contains(st.BlockingViews(rule.ViewIDs), viewID) {
continue
}
reqs = append(reqs, request(engine.OpBlockRuleDelete, rule.ID, nil))
}
if len(reqs) == 0 {
if queryBool(r, "missing_ok") {
writeJSON(w, http.StatusOK, api.MutationResponse{})
return nil
}
return notFound("there is no rule for %s", name)
}
return s.mutate(w, r, p, "", reqs, nil)
}
// checkBlocking: GET /blocking/check?name=NAME says what blocking does
// with a name in each view.
func (s *Server) checkBlocking(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
if err := requireBlocking(p); err != nil {
return err
}
st, err := s.store.LoadState(r.Context())
if err != nil {
return err
}
out, err := s.eng.CheckBlocking(st, r.URL.Query().Get("name"))
if err != nil {
return err
}
if out == nil {
out = []blocking.Match{}
}
writeJSON(w, http.StatusOK, out)
return nil
}
// blocklistErrors lists the enabled blocklists whose last update failed,
// for the dashboard.
func (s *Server) blocklistErrors(ctx context.Context) []api.BlocklistError {
st, err := s.store.LoadState(ctx)
if err != nil || len(st.Blocklists) == 0 {
return nil
}
statuses, err := s.store.BlocklistStatuses(ctx)
if err != nil {
return nil
}
var out []api.BlocklistError
for _, b := range st.SortedBlocklists() {
if s := statuses[b.ID]; s != nil && s.Error != "" && !b.Disabled {
out = append(out, api.BlocklistError{Name: b.Name, Error: s.Error, UpdatedAt: s.UpdatedAt})
}
}
return out
}
// blockingStatus: GET /blocking/status says whether blocking is paused.
func (s *Server) blockingStatus(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
if err := requireBlocking(p); err != nil {
return err
}
st, err := s.store.LoadState(r.Context())
if err != nil {
return err
}
writeJSON(w, http.StatusOK, map[string]any{"paused_until": st.Settings.BlockingPausedUntil, "paused_indefinitely": st.Settings.BlockingPausedIndefinitely})
return nil
}
// pauseBlocking: POST /blocking/pause {"minutes": 10} lets every name
// through until then, {"indefinitely": true} until resumed; POST
// /blocking/resume ends it.
func (s *Server) pauseBlocking(resume bool) func(http.ResponseWriter, *http.Request, *auth.Principal) error {
return func(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
var in engine.BlockingPauseInput
if !resume {
var body struct {
Minutes int `json:"minutes"`
Indefinitely bool `json:"indefinitely"`
}
if err := decodeJSON(r, &body); err != nil {
return err
}
if body.Indefinitely {
in.Indefinitely = true
return s.mutate(w, r, p, "", []engine.Request{engine.NewRequest(engine.OpBlockingPause, 0, in)}, nil)
}
if body.Minutes < 1 || time.Duration(body.Minutes)*time.Minute > engine.MaxBlockingPause {
return badRequest("pause for 1 to %d minutes, or indefinitely", int(engine.MaxBlockingPause/time.Minute))
}
until := time.Now().Add(time.Duration(body.Minutes) * time.Minute).Truncate(time.Second)
in.Until = &until
}
return s.mutate(w, r, p, "", []engine.Request{engine.NewRequest(engine.OpBlockingPause, 0, in)}, nil)
}
}
+7
View File
@@ -279,6 +279,13 @@ func (s *Server) status(w http.ResponseWriter, r *http.Request, p *auth.Principa
resp.Backup = s.backupStatus(r.Context(), now)
resp.TOTP = s.totpOverview(r.Context())
}
if p.Can(auth.Action{Resource: auth.ResBlocking}) {
resp.BlocklistErrors = s.blocklistErrors(r.Context())
if st, err := s.store.LoadState(r.Context()); err == nil {
resp.BlockingPausedUntil = st.Settings.BlockingPausedUntil
resp.BlockingPausedIndefinitely = st.Settings.BlockingPausedIndefinitely
}
}
writeJSON(w, http.StatusOK, resp)
return nil
}
+26
View File
@@ -14,6 +14,7 @@ import (
"jarl/internal/dnsname"
"jarl/internal/engine"
"jarl/internal/model"
"jarl/internal/render"
"jarl/internal/rr"
)
@@ -92,6 +93,31 @@ func (s *Server) getView(w http.ResponseWriter, r *http.Request, p *auth.Princip
return nil
}
// viewKey: GET /views/{id}/key is the TSIG key that selects the view
// (render/viewkeys.go), with its secret: like revealing a TSIG key's, it
// needs tsig:write and is audited.
func (s *Server) viewKey(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
st, err := s.state(r)
if err != nil {
return err
}
v, err := s.viewByPath(r, st, p)
if err != nil {
return err
}
if err := require(p, auth.Action{Resource: auth.ResTSIG, Write: true}); err != nil {
return forbidden("showing a view's key needs tsig:write")
}
seed, err := engine.ViewKeySeed(r.Context(), s.store)
if err != nil {
return err
}
name := render.ViewKeyName(v.Name)
s.eng.Audit(r.Context(), p, "tsig.reveal", "view", v.ID, v.Name, "success", "view key "+name+" shown")
writeJSON(w, http.StatusOK, map[string]string{"name": name, "algorithm": render.ViewKeyAlgorithm, "secret": render.ViewKeySecret(seed, v.ID)})
return nil
}
func (s *Server) viewObject(r *http.Request) func(st *model.State, id int64) any {
return func(st *model.State, id int64) any {
if v := st.Views[id]; v != nil {
+6 -1
View File
@@ -37,7 +37,12 @@ func (s *Server) zoneDNSSEC(w http.ResponseWriter, r *http.Request, p *auth.Prin
}
resp := api.DNSSECResponse{Zone: z.Name, View: st.ViewName(z), Type: z.Type, Status: s.eng.DNSSEC(r.Context(), st, z), Registrar: s.registrarFor(z)}
if pz := parentInJarl(st, z); pz != nil {
resp.ParentZone = pz.Name
resp.ParentZone, resp.ParentZoneID, resp.ParentView = pz.Name, pz.ID, st.ViewName(pz)
for _, rec := range st.ServedRecords(pz.ID) {
if rec.Name == z.Name && rec.Type == "DS" {
resp.ParentDS = append(resp.ParentDS, rec.Data)
}
}
}
writeJSON(w, http.StatusOK, resp)
return nil
+75 -6
View File
@@ -7,11 +7,13 @@ package server
import (
"context"
"encoding/json"
"fmt"
"net"
"net/http"
"net/netip"
"strings"
"sync"
"time"
"jarl/internal/api"
@@ -58,7 +60,9 @@ func (s *Server) zoneSerials(w http.ResponseWriter, r *http.Request, p *auth.Pri
}
ctx, cancel := context.WithTimeout(r.Context(), 15*time.Second)
defer cancel()
resp.Serial, resp.SerialSource, err = s.localSerial(ctx, st, z)
ck := s.checker()
emit := s.watchChecks(w, r, ck)
resp.Serial, resp.SerialSource, err = s.watchedLocalSerial(ctx, emit, st, z)
if err != nil {
resp.Notes = append(resp.Notes, "named on this server didn't report the zone's serial: "+err.Error())
}
@@ -77,15 +81,72 @@ func (s *Server) zoneSerials(w http.ResponseWriter, r *http.Request, p *auth.Pri
if resp.View != "" {
resp.Notes = append(resp.Notes, fmt.Sprintf("The zone is in view %s. Other servers answer from the view that matches this server's address (or key), as they do for its zone transfers and NOTIFYs.", resp.View))
}
resp.Servers = s.checker().Serials(ctx, z.Name, resp.Serial, targets)
resp.Servers = ck.Serials(ctx, z.Name, resp.Serial, targets)
if len(resp.Servers) > 0 && !hasOther(resp.Servers) {
resp.Notes = append(resp.Notes, "Only this server is listed, so there is nothing to compare with.")
}
resp.CheckedAt = time.Now()
if emit != nil {
emit(map[string]any{"kind": "done", "response": resp})
return nil
}
writeJSON(w, http.StatusOK, resp)
return nil
}
// watchChecks: with ?stream=1, /serials and /delegation answer with
// newline-delimited JSON events while they ask the servers (nscheck.Event,
// kind start/end, one pair per DNS query), then {"kind":"done",
// "response":...}. The web UI shows the queries as they go. It returns
// the function that writes an event, or nil without ?stream=1.
func (s *Server) watchChecks(w http.ResponseWriter, r *http.Request, ck *nscheck.Checker) func(any) {
if !queryBool(r, "stream") {
return nil
}
emit := s.streamJSON(w)
ck.Watch = func(e nscheck.Event) { emit(e) }
return emit
}
// watchedLocalSerial is localSerial, reported as event 0 when streaming.
func (s *Server) watchedLocalSerial(ctx context.Context, emit func(any), st *model.State, z *model.Zone) (*uint32, string, error) {
cmd := s.eng.LoadedSerialCommand(st, z)
if emit != nil && cmd != "" {
emit(nscheck.Event{Kind: "start", ID: 0, Query: cmd})
}
serial, src, err := s.localSerial(ctx, st, z)
if emit != nil && cmd != "" {
e := nscheck.Event{Kind: "end", ID: 0}
switch {
case err != nil:
e.Error = err.Error()
case src == "named" && serial != nil:
e.Answer = fmt.Sprintf("serial %d", *serial)
}
emit(e)
}
return serial, src, err
}
// streamJSON starts a newline-delimited JSON response and returns a
// function that writes one value and flushes it; it is safe for
// concurrent use.
func (s *Server) streamJSON(w http.ResponseWriter) func(any) {
w.Header().Set("Content-Type", "application/x-ndjson")
w.Header().Set("Cache-Control", "no-store")
w.Header().Set("X-Accel-Buffering", "no") // nginx: don't buffer
w.WriteHeader(http.StatusOK)
rc := http.NewResponseController(w)
enc := json.NewEncoder(w)
var mu sync.Mutex
return func(v any) {
mu.Lock()
defer mu.Unlock()
enc.Encode(v)
rc.Flush()
}
}
func (s *Server) zoneDelegation(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
st, err := s.state(r)
if err != nil {
@@ -99,6 +160,9 @@ func (s *Server) zoneDelegation(w http.ResponseWriter, r *http.Request, p *auth.
return badRequest("the delegation check compares the parent's delegation with the zone's NS records, so it needs a primary zone (this is a %s zone; check it where its primary is)", z.Type)
}
c := s.checker()
// This server's own addresses are asked too: sign those queries with
// the key of the zone's view, so they get that view's answer.
c.LocalKey = s.eng.ViewKey(r.Context(), st, z)
if v := r.URL.Query().Get("resolver"); v != "" {
// An address only, always port 53: this is a DNS resolver.
a, err := netip.ParseAddr(v)
@@ -108,7 +172,7 @@ func (s *Server) zoneDelegation(w http.ResponseWriter, r *http.Request, p *auth.
c.Resolver = net.JoinHostPort(a.String(), "53")
}
var ns []string
for _, rec := range st.ZoneRecords(z.ID) {
for _, rec := range st.ServedRecords(z.ID) {
if rec.Type == "NS" && rec.Name == z.Name {
ns = append(ns, rec.Data)
}
@@ -121,11 +185,16 @@ func (s *Server) zoneDelegation(w http.ResponseWriter, r *http.Request, p *auth.
}
ctx, cancel := context.WithTimeout(r.Context(), 30*time.Second)
defer cancel()
ref, _, _ := s.localSerial(ctx, st, z)
emit := s.watchChecks(w, r, c)
ref, _, _ := s.watchedLocalSerial(ctx, emit, st, z)
resp := api.DelegationResponse{DelegationReport: c.Delegation(ctx, z.Name, ns, addrs, ref, s.signing(ctx, st, z)), View: st.ViewName(z), Notes: []string{}, CheckedAt: time.Now()}
if resp.View != "" {
resp.Notes = append(resp.Notes, fmt.Sprintf("Compared with the zone in view %s. Resolvers on the internet get the view that matches their address; make sure that's this one.", resp.View))
}
if emit != nil {
emit(map[string]any{"kind": "done", "response": resp})
return nil
}
writeJSON(w, http.StatusOK, resp)
return nil
}
@@ -160,7 +229,7 @@ func (s *Server) localSerial(ctx context.Context, st *model.State, z *model.Zone
// addresses Jarl has for them in the same view (or nil, to be looked up).
func nsTargets(st *model.State, z *model.Zone) []nscheck.Target {
var out []nscheck.Target
for _, rec := range st.ZoneRecords(z.ID) {
for _, rec := range st.ServedRecords(z.ID) {
if rec.Type != "NS" || rec.Name != z.Name {
continue
}
@@ -178,7 +247,7 @@ func jarlAddresses(st *model.State, viewID int64, name string) []string {
if !z.HasRecords() || !dnsname.IsSubdomain(name, z.Name) {
continue
}
for _, rec := range st.ZoneRecords(z.ID) {
for _, rec := range st.ServedRecords(z.ID) {
if rec.Name == name && (rec.Type == "A" || rec.Type == "AAAA") {
out = append(out, rec.Data)
}
+1 -1
View File
@@ -73,7 +73,7 @@ func lowerNoDot(n string) string { return strings.ToLower(strings.TrimSuffix(n,
// trailing dots.
func zoneNS(st *model.State, z *model.Zone) []string {
var out []string
for _, rec := range st.ZoneRecords(z.ID) {
for _, rec := range st.ServedRecords(z.ID) {
if rec.Type == "NS" && rec.Name == z.Name {
out = append(out, lowerNoDot(rec.Data))
}
+127
View File
@@ -2,10 +2,15 @@ package server
import (
"net/http"
"net/netip"
"path/filepath"
"slices"
"strings"
"time"
"jarl/internal/auth"
"jarl/internal/dnsname"
"jarl/internal/model"
"jarl/internal/stats"
)
@@ -53,6 +58,9 @@ func (s *Server) statsSummary(w http.ResponseWriter, r *http.Request, p *auth.Pr
if err != nil {
return err
}
if err := s.nameClients(r, p, sum.Tops[stats.KindClient]); err != nil {
return err
}
writeJSON(w, http.StatusOK, sum)
return nil
}
@@ -75,6 +83,11 @@ func (s *Server) statsTop(w http.ResponseWriter, r *http.Request, p *auth.Princi
if err != nil {
return badRequest("%v", err)
}
if kind == stats.KindClient || kind == stats.KindRefused || kind == stats.KindBlockedClient {
if err := s.nameClients(r, p, es); err != nil {
return err
}
}
writeJSON(w, http.StatusOK, es)
return nil
}
@@ -145,3 +158,117 @@ func (s *Server) statsBreakdown(w http.ResponseWriter, r *http.Request, p *auth.
writeJSON(w, http.StatusOK, es)
return nil
}
// nameClients gives client entries the host name Jarl's records have for
// the address: the PTR record's target, or else the name of an A/AAAA
// record with the address. It sends no DNS queries, and uses only records
// the user may read. Anonymized clients (prefixes) get no name.
func (s *Server) nameClients(r *http.Request, p *auth.Principal, es []stats.Entry) error {
if len(es) == 0 {
return nil
}
want := map[netip.Addr]bool{}
for _, e := range es {
if a, err := netip.ParseAddr(e.Key); err == nil {
want[a.Unmap()] = true
}
}
if len(want) == 0 {
return nil
}
st, err := s.state(r)
if err != nil {
return err
}
names := clientNames(st, want, func(rec *model.Record) bool { return canReadRecord(p, st, rec) })
for i, e := range es {
if a, err := netip.ParseAddr(e.Key); err == nil {
es[i].Name = names[a.Unmap()]
}
}
return nil
}
// clientNames finds names for the addresses in want (see nameClients).
// Where several records match, the first in name order wins.
func clientNames(st *model.State, want map[netip.Addr]bool, readable func(*model.Record) bool) map[netip.Addr]string {
ptr, host := map[netip.Addr]string{}, map[netip.Addr]string{}
var recs []*model.Record
for _, rec := range st.Records {
if !rec.Disabled && (rec.Type == "PTR" || rec.Type == "A" || rec.Type == "AAAA") {
recs = append(recs, rec)
}
}
slices.SortFunc(recs, func(a, b *model.Record) int {
if a.Name != b.Name {
if model.CanonicalLess(a.Name, b.Name) {
return -1
}
return 1
}
return strings.Compare(a.Data, b.Data)
})
for _, rec := range recs {
var a netip.Addr
var name string
if rec.Type == "PTR" {
var ok bool
if a, ok = dnsname.AddrFromReverse(rec.Name); !ok {
continue
}
name = rec.Data
} else {
var err error
if a, err = netip.ParseAddr(rec.Data); err != nil {
continue
}
name = rec.Name
}
a = a.Unmap()
m := host
if rec.Type == "PTR" {
m = ptr
}
if !want[a] || m[a] != "" || !readable(rec) {
continue
}
m[a] = dnsname.Display(name)
}
for a, n := range host {
if ptr[a] == "" {
ptr[a] = n
}
}
return ptr
}
// statsServer: GET /stats/server is the latest server statistics (named's
// memory, CPU, cache and sockets, the host), named's client counts and
// limits (rndc status), and the space left on the file systems Jarl and
// named write to.
func (s *Server) statsServer(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
if _, err := s.statsQuerier(p); err != nil {
return err
}
out := map[string]any{"server": s.cfg.Stats.Server()}
if ns, err := s.eng.NamedStatus(r.Context()); err == nil {
out["named"] = map[string]string{"recursive_clients": ns.RecursiveClients, "tcp_clients": ns.TCPClients, "zones": ns.Zones,
"cpus": ns.CPUs, "worker_threads": ns.WorkerThreads, "last_configured": ns.LastConfigured}
} else {
out["named_error"] = err.Error()
}
paths := s.eng.Config().Paths
dirs := map[string]string{"zone files": paths.ZoneDir, "Jarl's cache and blocklists": paths.CacheDir, "named.conf": paths.ConfigDir}
labels := []string{"zone files", "Jarl's cache and blocklists", "named.conf"}
if lf := s.cfg.Stats.Options().LogFile; lf != "" {
dirs["named's logs"] = filepath.Dir(lf)
labels = append(labels, "named's logs")
}
if db := s.cfg.Stats.Store().Path(); db != "" {
dirs["statistics database"] = filepath.Dir(db)
labels = append(labels, "statistics database")
}
out["disks"] = stats.DiskUsage(labels, dirs)
writeJSON(w, http.StatusOK, out)
return nil
}
+72 -3
View File
@@ -4,6 +4,7 @@ import (
"encoding/json"
"fmt"
"net/http"
"reflect"
"sort"
"strconv"
"strings"
@@ -44,10 +45,17 @@ func (s *Server) zoneByPath(r *http.Request, st *model.State, p *auth.Principal)
return z, nil
}
func recordCounts(st *model.State) map[int64]int {
counts := map[int64]int{}
type recordCount struct{ all, disabled int }
func recordCounts(st *model.State) map[int64]recordCount {
counts := map[int64]recordCount{}
for _, r := range st.Records {
counts[r.ZoneID]++
c := counts[r.ZoneID]
c.all++
if r.Disabled {
c.disabled++
}
counts[r.ZoneID] = c
}
return counts
}
@@ -190,10 +198,14 @@ func (s *Server) listRecords(w http.ResponseWriter, r *http.Request, p *auth.Pri
}
typ := strings.ToUpper(q.Get("type"))
search := strings.ToLower(q.Get("q"))
disabled := q.Get("disabled") // "1": only disabled records, "0": only enabled ones
match := func(rec *model.Record) bool {
if name != "" && rec.Name != name || typ != "" && rec.Type != typ {
return false
}
if disabled != "" && rec.Disabled != (disabled == "1" || disabled == "true") {
return false
}
return search == "" || strings.Contains(rec.Name, search) || strings.Contains(strings.ToLower(rec.Data), search) || strings.Contains(strings.ToLower(rec.Comment), search)
}
recs := st.ZoneRecords(z.ID)
@@ -229,6 +241,13 @@ func (s *Server) listRecords(w http.ResponseWriter, r *http.Request, p *auth.Pri
ro.Pending = "add"
case old.Version != rec.Version:
ro.Pending = "change"
if only := *old; only.Disabled != rec.Disabled {
// Just disabled or enabled: say which.
only.Disabled, only.Meta = rec.Disabled, rec.Meta
if reflect.DeepEqual(only, *rec) {
ro.Pending = map[bool]string{true: "disable", false: "enable"}[rec.Disabled]
}
}
}
}
out = append(out, ro)
@@ -380,6 +399,56 @@ func (s *Server) deleteRecordsByMatch(w http.ResponseWriter, r *http.Request, p
return s.mutate(w, r, p, "", []engine.Request{engine.NewRequest(engine.OpRecordDelete, 0, in)}, nil)
}
// setRecordDisabled: POST /records/{id}/disable or /enable [?ptr=1]
func (s *Server) setRecordDisabled(disabled bool) func(http.ResponseWriter, *http.Request, *auth.Principal) error {
op := disableOp(disabled)
return func(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
st, err := s.state(r)
if err != nil {
return err
}
rec, err := s.recordByPath(r, st, p)
if err != nil {
return err
}
req := engine.NewRequest(op, rec.ID, engine.RecordInput{PTR: queryBool(r, "ptr")})
id := rec.ID
return s.mutate(w, r, p, "", []engine.Request{req}, func(st *model.State, _ int64) any { return recordObject(st, id) })
}
}
// setRecordsDisabled: POST /zones/{zone}/records/disable or /enable
// ?name=&type=[&data=][&missing_ok=1][&ptr=1]
func (s *Server) setRecordsDisabled(disabled bool) func(http.ResponseWriter, *http.Request, *auth.Principal) error {
op := disableOp(disabled)
return func(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
st, err := s.state(r)
if err != nil {
return err
}
z, err := s.zoneByPath(r, st, p)
if err != nil {
return err
}
q := r.URL.Query()
if q.Get("name") == "" || q.Get("type") == "" {
return badRequest("give name and type (and optionally data) of the records")
}
in := engine.RecordInput{ZoneID: z.ID, Name: model.Ptr(q.Get("name")), Type: model.Ptr(q.Get("type")), MissingOK: queryBool(r, "missing_ok"), PTR: queryBool(r, "ptr")}
if d := q.Get("data"); d != "" {
in.Data = &d
}
return s.mutate(w, r, p, "", []engine.Request{engine.NewRequest(op, 0, in)}, nil)
}
}
func disableOp(disabled bool) string {
if disabled {
return engine.OpRecordDisable
}
return engine.OpRecordEnable
}
func (s *Server) putRRSet(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
st, err := s.state(r)
if err != nil {
+58
View File
@@ -1,7 +1,9 @@
package server
import (
"encoding/json"
"net/netip"
"strings"
"testing"
"time"
@@ -69,3 +71,59 @@ func TestZoneSerialsTargets(t *testing.T) {
e.do("GET", "/zones/copy.example/delegation", e.token, nil, 400)
e.do("GET", "/zones/example.com/delegation?resolver=not-an-ip", e.token, nil, 400)
}
func TestZoneSerialsStream(t *testing.T) {
e := newAPIEnv(t)
e.srv.nsChecker = &nscheck.Checker{Timeout: 200 * time.Millisecond, Port: "1", IsLocal: func(a netip.Addr) bool { return a.String() == "127.0.0.21" }}
e.do("POST", "/tsig-keys", e.token, map[string]any{"name": "xfer"}, 201)
e.do("POST", "/zones", e.token, map[string]any{
"name": "example.com",
"nameservers": []string{"ns1.example.com.", "ns2.example.com."},
"records": []map[string]any{{"name": "ns1", "type": "A", "data": "127.0.0.21"}, {"name": "ns2", "type": "A", "data": "127.0.0.22"}},
"options": []map[string]any{{"name": "also-notify", "value": "{ 127.0.0.24 port 5300 key xfer; }"}},
}, 201)
body, status := e.req("GET", "/zones/example.com/serials?stream=1", e.token, nil)
if status != 200 {
t.Fatalf("stream = %d: %s", status, body)
}
var starts, ends []map[string]any
var done map[string]any
for _, line := range strings.Split(strings.TrimSpace(string(body)), "\n") {
var ev map[string]any
if err := json.Unmarshal([]byte(line), &ev); err != nil {
t.Fatalf("line %q: %v", line, err)
}
switch ev["kind"] {
case "start":
starts = append(starts, ev)
case "end":
ends = append(ends, ev)
case "done":
done = ev
}
}
// .21 is this server (not asked); .22 and .24 are.
queries := map[string]bool{}
for _, s := range starts {
queries[s["query"].(string)] = true
}
for _, q := range []string{
"dig @127.0.0.22 -p 1 example.com. SOA +norec",
"dig @127.0.0.24 -p 5300 example.com. SOA +norec -y hmac-sha256:xfer:…",
} {
if !queries[q] {
t.Errorf("no query %q in %v", q, queries)
}
}
if len(starts) != 2 || len(ends) != 2 {
t.Errorf("starts %d, ends %d", len(starts), len(ends))
}
for _, e := range ends {
if r, _ := e["result"].(map[string]any); r == nil || r["status"] != "unreachable" {
t.Errorf("end event %v", e)
}
}
if done == nil || len(done["response"].(map[string]any)["servers"].([]any)) != 3 {
t.Errorf("done = %v", done)
}
}
+6 -2
View File
@@ -9,13 +9,14 @@ import (
"jarl/internal/dnsname"
"jarl/internal/engine"
"jarl/internal/model"
"jarl/internal/namedconf"
"jarl/internal/render"
"jarl/internal/rr"
"jarl/internal/version"
)
func (s *Server) presentZone(st *model.State, z *model.Zone, counts map[int64]int) api.Zone {
out := api.Zone{Zone: *z, Display: dnsname.Display(z.Name), View: st.ViewName(z), Records: counts[z.ID]}
func (s *Server) presentZone(st *model.State, z *model.Zone, counts map[int64]recordCount) api.Zone {
out := api.Zone{Zone: *z, Display: dnsname.Display(z.Name), View: st.ViewName(z), Records: counts[z.ID].all, DisabledRecords: counts[z.ID].disabled}
out.ZoneFile = render.ZoneFilePath(st, z, s.eng.Config().Paths)
if z.Type == model.ZonePrimary {
out.RName = dnsname.EmailFromMailbox(z.SOA.RName)
@@ -100,6 +101,9 @@ func presentOption(st *model.State, o *model.Option) api.Option {
if c, ok := catalog.Default().Lookup(scope, o.Name); ok {
out.Description, out.Status = c.Description, c.Status
}
if f := namedconf.FormatValue(o.Name, o.Value); f != o.Value {
out.Formatted = f
}
return out
}
+22
View File
@@ -52,10 +52,27 @@ func (s *Server) routes() {
m.HandleFunc("POST "+p+"/views", s.authed(s.createView))
m.HandleFunc("POST "+p+"/views/order", s.authed(s.reorderViews))
m.HandleFunc("GET "+p+"/views/{id}", s.authed(s.getView))
m.HandleFunc("GET "+p+"/views/{id}/key", s.authed(s.viewKey))
m.HandleFunc("PATCH "+p+"/views/{id}", s.authed(s.updateView))
m.HandleFunc("DELETE "+p+"/views/{id}", s.authed(s.deleteView))
m.HandleFunc("PUT "+p+"/views/{id}/color", s.authed(s.setViewColor))
// ACLs
// Blocking
m.HandleFunc("GET "+p+"/blocking/lists", s.authed(s.listBlocklists))
m.HandleFunc("POST "+p+"/blocking/lists", s.authed(s.createBlocklist))
m.HandleFunc("GET "+p+"/blocking/lists/{id}", s.authed(s.getBlocklist))
m.HandleFunc("PATCH "+p+"/blocking/lists/{id}", s.authed(s.updateBlocklist))
m.HandleFunc("DELETE "+p+"/blocking/lists/{id}", s.authed(s.deleteBlocklist))
m.HandleFunc("POST "+p+"/blocking/lists/{id}/refresh", s.authed(s.refreshBlocklist))
m.HandleFunc("GET "+p+"/blocking/rules", s.authed(s.listBlockRules))
m.HandleFunc("POST "+p+"/blocking/rules", s.authed(s.createBlockRule))
m.HandleFunc("DELETE "+p+"/blocking/rules", s.authed(s.deleteBlockRulesByName))
m.HandleFunc("PATCH "+p+"/blocking/rules/{id}", s.authed(s.updateBlockRule))
m.HandleFunc("DELETE "+p+"/blocking/rules/{id}", s.authed(s.deleteBlockRule))
m.HandleFunc("GET "+p+"/blocking/check", s.authed(s.checkBlocking))
m.HandleFunc("GET "+p+"/blocking/status", s.authed(s.blockingStatus))
m.HandleFunc("POST "+p+"/blocking/pause", s.authed(s.pauseBlocking(false)))
m.HandleFunc("POST "+p+"/blocking/resume", s.authed(s.pauseBlocking(true)))
m.HandleFunc("GET "+p+"/acls", s.authed(s.listACLs))
m.HandleFunc("POST "+p+"/acls", s.authed(s.createACL))
m.HandleFunc("GET "+p+"/acls/{id}", s.authed(s.getACL))
@@ -88,11 +105,15 @@ func (s *Server) routes() {
m.HandleFunc("GET "+p+"/zones/{zone}/records", s.authed(s.listRecords))
m.HandleFunc("POST "+p+"/zones/{zone}/records", s.authed(s.createRecord))
m.HandleFunc("DELETE "+p+"/zones/{zone}/records", s.authed(s.deleteRecordsByMatch))
m.HandleFunc("POST "+p+"/zones/{zone}/records/disable", s.authed(s.setRecordsDisabled(true)))
m.HandleFunc("POST "+p+"/zones/{zone}/records/enable", s.authed(s.setRecordsDisabled(false)))
m.HandleFunc("PUT "+p+"/zones/{zone}/rrsets/{name}/{type}", s.authed(s.putRRSet))
m.HandleFunc("DELETE "+p+"/zones/{zone}/rrsets/{name}/{type}", s.authed(s.deleteRRSet))
m.HandleFunc("GET "+p+"/records/{id}", s.authed(s.getRecord))
m.HandleFunc("PATCH "+p+"/records/{id}", s.authed(s.updateRecord))
m.HandleFunc("DELETE "+p+"/records/{id}", s.authed(s.deleteRecord))
m.HandleFunc("POST "+p+"/records/{id}/disable", s.authed(s.setRecordDisabled(true)))
m.HandleFunc("POST "+p+"/records/{id}/enable", s.authed(s.setRecordDisabled(false)))
// Options and settings
m.HandleFunc("GET "+p+"/options", s.authed(s.listOptions))
m.HandleFunc("POST "+p+"/options", s.authed(s.createOption))
@@ -136,6 +157,7 @@ func (s *Server) routes() {
m.HandleFunc("GET "+p+"/stats/timeline", s.authed(s.statsTimeline))
m.HandleFunc("GET "+p+"/stats/heatmap", s.authed(s.statsHeatmap))
m.HandleFunc("GET "+p+"/stats/breakdown", s.authed(s.statsBreakdown))
m.HandleFunc("GET "+p+"/stats/server", s.authed(s.statsServer))
m.HandleFunc(p+"/", func(w http.ResponseWriter, r *http.Request) {
writeError(w, http.StatusNotFound, "not_found", "no such API endpoint: "+r.Method+" "+r.URL.Path)
})
+19
View File
@@ -12,8 +12,10 @@ import (
"net/http"
"net/netip"
"os"
"strconv"
"strings"
"sync"
"syscall"
"time"
"jarl/internal/api"
@@ -116,6 +118,16 @@ func New(cfg Config, eng *engine.Engine, log *slog.Logger, web fs.FS) *Server {
}
// Handler returns the server's HTTP handler with all middleware.
// lowPort reports whether addr ("host:port") has a port below 1024.
func lowPort(addr string) bool {
_, port, err := net.SplitHostPort(addr)
if err != nil {
return false
}
n, err := strconv.Atoi(port)
return err == nil && n > 0 && n < 1024
}
func (s *Server) Handler() http.Handler {
return s.recoverer(s.securityHeaders(s.logRequests(s.mux)))
}
@@ -142,6 +154,9 @@ func (s *Server) Serve(ctx context.Context, ready func()) error {
for _, l := range listeners {
l.Close()
}
if errors.Is(err, syscall.EACCES) && lowPort(addr) {
return fmt.Errorf("listening on %s: %w (ports below 1024 need the CAP_NET_BIND_SERVICE capability; Jarl's jarl.service grants it, so check that the installed unit is current and no drop-in removes it: systemctl cat jarl)", addr, err)
}
return fmt.Errorf("listening on %s: %w", addr, err)
}
listeners = append(listeners, ln)
@@ -384,6 +399,10 @@ func (w *statusWriter) WriteHeader(code int) {
w.ResponseWriter.WriteHeader(code)
}
// Unwrap lets http.ResponseController reach the connection (to flush a
// streamed response).
func (w *statusWriter) Unwrap() http.ResponseWriter { return w.ResponseWriter }
func (w *statusWriter) Write(b []byte) (int, error) {
if w.status == 0 {
w.status = http.StatusOK
+11
View File
@@ -58,6 +58,17 @@ func TestStatsAPI(t *testing.T) {
if len(tops) != 2 || tops[0]["key"] != "www.example.com" {
t.Errorf("top names = %v", tops)
}
// Clients get the host name of Jarl's record for their address.
if c := e.list("/stats/top?kind=client&since=2h", e.token); len(c) != 1 || c[0]["name"] != nil {
t.Errorf("unnamed client = %v", c)
}
e.do("POST", "/zones", e.token, map[string]any{"name": "example.com", "records": []map[string]any{{"name": "ns1", "type": "A", "data": "192.0.2.1"}, {"name": "laptop", "type": "A", "data": "10.0.0.7"}}}, 201)
if c := e.list("/stats/top?kind=client&since=2h", e.token); len(c) != 1 || c[0]["name"] != "laptop.example.com" {
t.Errorf("named client = %v", c)
}
if sum := e.do("GET", "/stats/summary?since=2h", e.token, nil, 200); sum["tops"].(map[string]any)["client"].([]any)[0].(map[string]any)["name"] != "laptop.example.com" {
t.Errorf("summary clients = %v", sum["tops"])
}
if b := e.list("/stats/breakdown?group=zone&since=1h", e.token); len(b) != 2 || b[0]["key"] != "example.com" {
t.Errorf("zones = %v", b)
}
+114
View File
@@ -0,0 +1,114 @@
package stats
import (
"context"
"fmt"
"io"
"log/slog"
"net/netip"
"os"
"path/filepath"
"testing"
"time"
)
func TestParseRewrite(t *testing.T) {
loc := time.UTC
// Real lines from named 9.20.24.
rw, ok := ParseRewrite(`09-Oct-2026 23:49:13.089 client @0x7bd607320000 127.0.0.1#56905 (x.ads.example): view internal: rpz QNAME NXDOMAIN rewrite x.ads.example/A/IN via x.ads.example.rules.jarl.invalid`, loc)
if !ok || rw.Client.String() != "127.0.0.1" || rw.Port != "56905" || rw.View != "internal" || rw.Name != "x.ads.example" || rw.Action != "NXDOMAIN" || policySource(rw.Via) != "rules" {
t.Errorf("rules rewrite: %+v (%s)", rw, policySource(rw.Via))
}
rw, ok = ParseRewrite(`09-Oct-2026 23:49:13.378 client @0x7bd607320000 10.0.0.9#57347 (Tracker.Example): rpz QNAME NXDOMAIN rewrite tracker.example/AAAA/IN via tracker.example.hagezi-pro.blocklist.jarl.invalid`, loc)
if !ok || rw.View != "" || rw.Name != "tracker.example" || policySource(rw.Via) != "hagezi-pro" {
t.Errorf("list rewrite: %+v (%s)", rw, policySource(rw.Via))
}
if policySource("ads.example.my-own-rpz.") != "ads.example.my-own-rpz" {
t.Error("a policy zone that isn't Jarl's")
}
if _, ok := ParseRewrite(`09-Oct-2026 23:49:13.378 client @0x7bd607320000 10.0.0.9#57347 (x): view internal: response: x IN A NOERROR 1 0 1 + (10.0.0.2)`, loc); ok {
t.Error("a response line parsed as a rewrite")
}
}
func TestCollectorBlocking(t *testing.T) {
ctx := context.Background()
loc := time.UTC
dir := t.TempDir()
logFile := filepath.Join(dir, "responses.log")
store, err := OpenStore(filepath.Join(dir, "stats.db"))
if err != nil {
t.Fatal(err)
}
defer store.Close()
named := newFakeNamed(t)
clk := &clock{time.Date(2026, 10, 5, 10, 0, 30, 0, loc)}
opts := Options{QueryLog: true, LogFile: logFile, StatisticsChannel: "unused", TopN: 5,
RetainMinute: 7 * 24 * time.Hour, RetainHour: 90 * 24 * time.Hour, RetainDay: 730 * 24 * time.Hour,
Exclude: []netip.Prefix{netip.MustParsePrefix("10.0.0.250/32")}, Location: loc}
c := NewCollector(store, opts, func() map[string][]string { return nil }, slog.New(slog.NewTextHandler(io.Discard, nil)))
c.poller.URL = named.srv.URL
c.now = clk.now
c.start(ctx)
at := time.Date(2026, 10, 5, 10, 0, 40, 0, loc).Format(logTimeLayout)
rewrite := func(client, port, name, action, via string) string {
return fmt.Sprintf("%s client @0x7f0000000000 %s#%s (%s): view internal: rpz QNAME %s rewrite %s/A/IN via %s\n", at, client, port, name, action, name, via)
}
response := func(client, port, name, rcode string) string {
return fmt.Sprintf("%s client @0x7f0000000000 %s#%s (%s): view internal: response: %s IN A %s 0 0 2 +E(0)K (10.0.0.2)\n", at, client, port, name, name, rcode)
}
f, _ := os.Create(logFile)
for _, l := range []string{
rewrite("10.0.0.5", "1001", "ads.example", "NXDOMAIN", "ads.example.hagezi.blocklist.jarl.invalid"),
response("10.0.0.5", "1001", "ads.example", "NXDOMAIN"),
rewrite("10.0.0.5", "1002", "x.ads.example", "NXDOMAIN", "x.ads.example.hagezi.blocklist.jarl.invalid"),
response("10.0.0.5", "1002", "x.ads.example", "NXDOMAIN"),
rewrite("10.0.0.6", "1003", "ads.example", "NXDOMAIN", "ads.example.rules.jarl.invalid"),
response("10.0.0.6", "1003", "ads.example", "NXDOMAIN"),
// An allow rule: answered normally, not blocked.
rewrite("10.0.0.6", "1004", "cdn.ads.example", "PASSTHRU", "cdn.ads.example.rules.jarl.invalid"),
response("10.0.0.6", "1004", "cdn.ads.example", "NOERROR"),
// A real NXDOMAIN.
response("10.0.0.6", "1005", "typo.example", "NXDOMAIN"),
// An excluded client.
rewrite("10.0.0.250", "1006", "ads.example", "NXDOMAIN", "ads.example.hagezi.blocklist.jarl.invalid"),
response("10.0.0.250", "1006", "ads.example", "NXDOMAIN"),
} {
f.WriteString(l)
}
f.Close()
c.readLog()
named.add(6, 4)
clk.t = time.Date(2026, 10, 5, 10, 1, 2, 0, loc)
c.flush(ctx, false)
m := time.Date(2026, 10, 5, 10, 0, 0, 0, loc).Unix()
got, _ := store.sums(ctx, ResMinute, m, m+60, "")
for k, v := range map[string]int64{"blocked": 3, "blocklist:hagezi": 2, "blocklist:rules": 1, "responses": 5} {
if got[k] != v {
t.Errorf("%s = %d, want %d (all: %v)", k, got[k], v, got)
}
}
if b, _ := store.TopsAt(ctx, ResHour, m, KindBlocked); len(b) != 2 || b[0] != (Entry{Key: "ads.example", Count: 2}) {
t.Errorf("blocked names = %v", b)
}
if bc, _ := store.TopsAt(ctx, ResHour, m, KindBlockedClient); len(bc) != 2 || bc[0] != (Entry{Key: "10.0.0.5", Count: 2}) {
t.Errorf("blocked clients = %v", bc)
}
if nx, _ := store.TopsAt(ctx, ResHour, m, KindNXDomain); len(nx) != 1 || nx[0].Key != "typo.example" {
t.Errorf("NXDOMAIN names should leave out blocked ones: %v", nx)
}
q := &Querier{Store: store, Opts: opts, Now: clk.now}
sum, err := q.Summary(ctx, time.Hour, 5)
if err != nil {
t.Fatal(err)
}
// 6 queries, 4 NXDOMAIN of which 3 blocked.
if sum.Blocked != 3 || sum.BlockedRatio == nil || *sum.BlockedRatio != 0.5 || sum.NXDomainRatio == nil || *sum.NXDomainRatio != 1.0/6 {
t.Errorf("summary: blocked %d ratio %v nxdomain %v", sum.Blocked, sum.BlockedRatio, sum.NXDomainRatio)
}
if len(sum.Blocklists) != 2 || sum.Blocklists[0] != (Entry{Key: "hagezi", Count: 2}) || len(sum.Tops[KindBlocked]) != 2 {
t.Errorf("summary lists %v, tops %v", sum.Blocklists, sum.Tops[KindBlocked])
}
}
+82 -7
View File
@@ -12,6 +12,8 @@ import (
"strings"
"sync"
"time"
"jarl/internal/blocking"
)
// Kinds of top lists.
@@ -21,10 +23,13 @@ const (
KindNXDomain = "nxdomain" // names answered with NXDOMAIN
KindServFail = "servfail" // names answered with SERVFAIL
KindRefused = "refused" // clients that were refused
KindBlocked = "blocked" // names blocked (docs/blocking.md)
// KindBlockedClient: clients with the most blocked queries.
KindBlockedClient = "blocked_client"
)
// Kinds lists the top-list kinds.
var Kinds = []string{KindName, KindClient, KindNXDomain, KindServFail, KindRefused}
var Kinds = []string{KindName, KindClient, KindNXDomain, KindServFail, KindRefused, KindBlocked, KindBlockedClient}
// OtherZone is the zone key for names outside every local zone (answered
// by recursion, or refused).
@@ -42,6 +47,9 @@ type Options struct {
Anonymize bool
Exclude []netip.Prefix
Location *time.Location // for days and log timestamps (default: local time)
// MainConfig is named's main configuration file; it tells named's
// process apart from other named processes (for its memory and CPU).
MainConfig string
// EnsureLogging turns named's response logging on (rndc responselog
// on). It is called at start and when named answers queries but no
// log lines arrive.
@@ -99,6 +107,13 @@ type Collector struct {
recentL []int64 // response log lines per minute, last 10 minutes
linesMin int64
ensureAt time.Time
// rewritten holds queries named blocked (client#port|name) until their
// response line arrives.
rewritten map[string]bool
// proc is the last sample of named's process, server the latest server
// statistics (server.go).
proc *namedProc
server *Server
}
// maxKeys bounds the per-hour maps; rarely seen keys are dropped first.
@@ -227,13 +242,17 @@ func (c *Collector) start(ctx context.Context) {
}
c.ensureLogging(ctx)
// The first poll is the baseline for the counters.
if snap, err := c.poller.Poll(ctx); err == nil {
snap, err := c.poller.Poll(ctx)
if err == nil {
c.prev = snap
t := c.now()
c.status.LastPoll = &t
} else {
c.status.PollError = err.Error()
}
c.mu.Lock()
c.sampleServer(-1, c.now(), snap)
c.mu.Unlock()
c.prune(ctx)
c.log.Info("statistics collector started", "query_log", c.opts.QueryLog, "statistics_channel", c.opts.StatisticsChannel)
}
@@ -296,8 +315,15 @@ func (c *Collector) HandleLine(line string) {
func (c *Collector) handleLine(line string) {
ev, ok := ParseResponse(line, c.opts.Location)
if !ok {
if rw, ok := ParseRewrite(line, c.opts.Location); ok {
c.handleRewrite(rw)
}
return
}
blocked := c.rewritten[ev.Client.String()+"#"+ev.Port+"|"+ev.Name]
if blocked {
delete(c.rewritten, ev.Client.String()+"#"+ev.Port+"|"+ev.Name)
}
c.status.LinesRead++
c.linesMin++
t := ev.Time
@@ -324,12 +350,14 @@ func (c *Collector) handleLine(line string) {
tops := hours[hour]
tops[KindName][ev.Name]++
tops[KindClient][client]++
switch ev.RCode {
case "NXDOMAIN":
switch {
case blocked:
// Counted by handleRewrite; not a name that doesn't exist.
case ev.RCode == "NXDOMAIN":
tops[KindNXDomain][ev.Name]++
case "SERVFAIL":
case ev.RCode == "SERVFAIL":
tops[KindServFail][ev.Name]++
case "REFUSED":
case ev.RCode == "REFUSED":
tops[KindRefused][client]++
}
for _, m := range tops {
@@ -339,6 +367,49 @@ func (c *Collector) handleLine(line string) {
}
}
// handleRewrite counts a query that blocking answered: "blocked", and
// "blocklist:NAME" for the list (or "rules") that matched.
func (c *Collector) handleRewrite(rw Rewrite) {
if rw.Action == "PASSTHRU" {
return // an allow rule: answered normally
}
for _, p := range c.opts.Exclude {
if p.Contains(rw.Client) {
return
}
}
if c.rewritten == nil || len(c.rewritten) > 10_000 {
// Rewrites whose response line never came (named only logs
// responses while response logging is on) don't pile up.
c.rewritten = map[string]bool{}
}
c.rewritten[rw.Client.String()+"#"+rw.Port+"|"+rw.Name] = true
minute, hour := c.minuteOf(rw.Time), c.hourOf(rw.Time)
c.count(minute, "blocked", 1)
c.count(minute, "blocklist:"+policySource(rw.Via), 1)
hours := c.hours
if hour < c.liveHour {
hours = c.late
}
hours[hour] = c.hourMaps(hours, hour)
hours[hour][KindBlocked][rw.Name]++
hours[hour][KindBlockedClient][clientKey(rw.Client, c.opts.Anonymize)]++
}
// policySource names what a rewrite came from: a blocklist's name, "rules"
// for the rules entered in Jarl, or the policy zone for others.
func policySource(via string) string {
via = strings.TrimSuffix(via, ".")
if strings.HasSuffix(via, "."+blocking.RulesZone) {
return "rules"
}
if i := strings.LastIndex(via, ".blocklist.jarl.invalid"); i > 0 && i+len(".blocklist.jarl.invalid") == len(via) {
rest := via[:i]
return rest[strings.LastIndex(rest, ".")+1:]
}
return via
}
// shrink drops the rarest keys until the map is well under maxKeys.
func shrink(m map[string]int64) {
for limit := int64(1); len(m) > maxKeys*8/10; limit++ {
@@ -410,6 +481,10 @@ func (c *Collector) flush(ctx context.Context, final bool) {
}
c.prev = snap
}
if !final {
// Gauges are sampled once a minute, for the minute that ended.
c.sampleServer(cur-60, now, snap)
}
b := &Batch{}
first := c.status.FirstData == nil
for m, vals := range c.minutes {
@@ -528,7 +603,7 @@ func (c *Collector) updateWarnings() {
func top(m map[string]int64, n int) []Entry {
out := make([]Entry, 0, len(m))
for k, v := range m {
out = append(out, Entry{k, v})
out = append(out, Entry{Key: k, Count: v})
}
slices.SortFunc(out, func(a, b Entry) int {
if a.Count != b.Count {
+6 -6
View File
@@ -118,7 +118,7 @@ func TestCollector(t *testing.T) {
}
}
names, _ := store.TopsAt(ctx, ResHour, m, KindName)
if len(names) != 3 || names[0] != (Entry{"nas.adbraun.lol", 2}) {
if len(names) != 3 || names[0] != (Entry{Key: "nas.adbraun.lol", Count: 2}) {
t.Errorf("hour names = %v", names)
}
if nx, _ := store.TopsAt(ctx, ResHour, m, KindNXDomain); len(nx) != 1 || nx[0].Key != "nope.adbraun.lol" {
@@ -133,7 +133,7 @@ func TestCollector(t *testing.T) {
clk.t = time.Date(2026, 10, 5, 11, 0, 2, 0, loc)
c.flush(ctx, false)
day := time.Date(2026, 10, 5, 0, 0, 0, 0, loc).Unix()
if d, _ := store.TopsAt(ctx, ResDay, day, KindName); len(d) == 0 || d[0] != (Entry{"nas.adbraun.lol", 3}) {
if d, _ := store.TopsAt(ctx, ResDay, day, KindName); len(d) == 0 || d[0] != (Entry{Key: "nas.adbraun.lol", Count: 3}) {
t.Errorf("day names after the hour = %v", d)
}
if v, _ := store.sums(ctx, ResDay, day, day+1, "queries"); v["queries"] != 7 {
@@ -149,14 +149,14 @@ func TestCollector(t *testing.T) {
c2.readLog()
clk.t = time.Date(2026, 10, 5, 11, 3, 2, 0, loc)
c2.flush(ctx, false)
if h, _ := store.TopsAt(ctx, ResHour, m, KindName); len(h) == 0 || h[0] != (Entry{"nas.adbraun.lol", 4}) {
if h, _ := store.TopsAt(ctx, ResHour, m, KindName); len(h) == 0 || h[0] != (Entry{Key: "nas.adbraun.lol", Count: 4}) {
t.Errorf("hour 10 after late line = %v", h)
}
if d, _ := store.TopsAt(ctx, ResDay, day, KindName); d[0] != (Entry{"nas.adbraun.lol", 4}) {
if d, _ := store.TopsAt(ctx, ResDay, day, KindName); d[0] != (Entry{Key: "nas.adbraun.lol", Count: 4}) {
t.Errorf("day after late line = %v", d)
}
h11 := time.Date(2026, 10, 5, 11, 0, 0, 0, loc).Unix()
if h, _ := store.TopsAt(ctx, ResHour, h11, KindName); len(h) != 1 || h[0] != (Entry{"printer.adbraun.lol", 2}) {
if h, _ := store.TopsAt(ctx, ResHour, h11, KindName); len(h) != 1 || h[0] != (Entry{Key: "printer.adbraun.lol", Count: 2}) {
t.Errorf("hour 11 continued after restart = %v", h)
}
@@ -198,7 +198,7 @@ func TestCollector(t *testing.T) {
t.Errorf("heatmap Monday 10h=%v 11h=%v Thu 3h=%v", hm.Avg[0][10], hm.Avg[0][11], hm.Avg[3][3])
}
b, _ := q.Breakdown(ctx, "zone", 24*time.Hour)
if len(b) != 2 || b[0] != (Entry{"adbraun.lol", 7}) || b[1] != (Entry{OtherZone, 1}) {
if len(b) != 2 || b[0] != (Entry{Key: "adbraun.lol", Count: 7}) || b[1] != (Entry{Key: OtherZone, Count: 1}) {
t.Errorf("zone breakdown = %v", b)
}
+6
View File
@@ -0,0 +1,6 @@
//go:build !unix
package stats
// DiskUsage is not available on this system.
func DiskUsage(labels []string, paths map[string]string) []Disk { return nil }
+39
View File
@@ -0,0 +1,39 @@
//go:build unix
package stats
import (
"fmt"
"os"
"syscall"
)
// DiskUsage returns the space on the file systems holding paths (label ->
// directory), one entry per file system with the labels of the paths on
// it, in the order of labels.
func DiskUsage(labels []string, paths map[string]string) []Disk {
var out []Disk
byDev := map[string]int{}
for _, l := range labels {
p := paths[l]
fi, err := os.Stat(p)
if err != nil {
continue
}
dev := fmt.Sprint(p)
if st, ok := fi.Sys().(*syscall.Stat_t); ok {
dev = fmt.Sprint(st.Dev)
}
if i, ok := byDev[dev]; ok {
out[i].Holds = append(out[i].Holds, l)
continue
}
var fs syscall.Statfs_t
if err := syscall.Statfs(p, &fs); err != nil {
continue
}
byDev[dev] = len(out)
out = append(out, Disk{Path: p, Holds: []string{l}, Total: int64(fs.Blocks) * int64(fs.Bsize), Free: int64(fs.Bavail) * int64(fs.Bsize)})
}
return out
}
+41 -5
View File
@@ -11,18 +11,36 @@ import (
type Event struct {
Time time.Time
Client netip.Addr
Port string // the client's port: with Client and Name, it ties an RPZ rewrite to its response
View string // "" without views
Name string // lowercase, without the final dot ("." for the root)
Type string // e.g. A, AAAA, PTR
RCode string // e.g. NOERROR, NXDOMAIN
}
// Rewrite is a response policy (RPZ) rewrite named logged: blocking.
type Rewrite struct {
Time time.Time
Client netip.Addr
Port string
View string
Name string // the query name, normalized like Event.Name
Action string // NXDOMAIN, NODATA, PASSTHRU, DROP, ...
Via string // the policy zone entry: OWNER.ZONE
}
// responseRe matches named's response log lines (BIND 9.20, print-time):
//
// 07-Oct-2026 19:46:22.195 client @0x7135... 10.10.0.37#51709 (nas.adbraun.lol): view internal: response: nas.adbraun.lol IN A NOERROR 1 0 1 +E(0)K (10.10.0.2)
//
// The "view NAME: " part is missing when no views are configured.
var responseRe = regexp.MustCompile(`^(\d{2}-[A-Za-z]{3}-\d{4} \d{2}:\d{2}:\d{2}\.\d{3}) client @0x[0-9a-fA-F]+ (\S+)#\d+ \(.*?\): (?:view (.+?): )?response: (\S+) (\S+) (\S+) ([A-Z0-9]+) `)
var responseRe = regexp.MustCompile(`^(\d{2}-[A-Za-z]{3}-\d{4} \d{2}:\d{2}:\d{2}\.\d{3}) client @0x[0-9a-fA-F]+ (\S+)#(\d+) \(.*?\): (?:view (.+?): )?response: (\S+) (\S+) (\S+) ([A-Z0-9]+) `)
// rewriteRe matches named's RPZ rewrite lines (category rpz), logged just
// before the response line of the same query:
//
// 09-Oct-2026 23:49:13.698 client @0x7135... 10.10.0.37#54468 (ads.example): view internal: rpz QNAME NXDOMAIN rewrite ads.example/A/IN via ads.example.hagezi.blocklist.jarl.invalid
var rewriteRe = regexp.MustCompile(`^(\d{2}-[A-Za-z]{3}-\d{4} \d{2}:\d{2}:\d{2}\.\d{3}) client @0x[0-9a-fA-F]+ (\S+)#(\d+) \((.*?)\): (?:view (.+?): )?rpz \S+ (\S+) rewrite \S+ via (\S+)`)
const logTimeLayout = "02-Jan-2006 15:04:05.000"
@@ -44,13 +62,31 @@ func ParseResponse(line string, loc *time.Location) (ev Event, ok bool) {
return Event{
Time: t,
Client: client.Unmap(),
View: m[3],
Name: normalizeName(m[4]),
Type: m[6],
RCode: m[7],
Port: m[3],
View: m[4],
Name: normalizeName(m[5]),
Type: m[7],
RCode: m[8],
}, true
}
// ParseRewrite parses an RPZ rewrite log line.
func ParseRewrite(line string, loc *time.Location) (rw Rewrite, ok bool) {
m := rewriteRe.FindStringSubmatch(line)
if m == nil {
return rw, false
}
t, err := time.ParseInLocation(logTimeLayout, m[1], loc)
if err != nil {
return rw, false
}
client, err := netip.ParseAddr(m[2])
if err != nil {
return rw, false
}
return Rewrite{Time: t, Client: client.Unmap(), Port: m[3], Name: normalizeName(m[4]), View: m[5], Action: m[6], Via: strings.ToLower(m[7])}, true
}
func normalizeName(n string) string {
n = strings.ToLower(n)
if n == "." {
+44 -21
View File
@@ -15,19 +15,31 @@ import (
type Snapshot struct {
BootTime string
Values map[string]int64
// Gauges are named's current values (memory, cache size, sockets),
// not counters.
Gauges map[string]int64
Version string
ConfigTime string
}
// serverJSON is the part of /json/v1/server Jarl uses.
// serverJSON is the part of /json/v1 Jarl uses.
type serverJSON struct {
BootTime string `json:"boot-time"`
Opcodes map[string]int64 `json:"opcodes"`
RCodes map[string]int64 `json:"rcodes"`
QTypes map[string]int64 `json:"qtypes"`
NSStats map[string]int64 `json:"nsstats"`
Views map[string]struct {
BootTime string `json:"boot-time"`
ConfigTime string `json:"config-time"`
Version string `json:"version"`
SockStats map[string]int64 `json:"sockstats"`
Memory struct {
InUse int64 `json:"InUse"`
} `json:"memory"`
Opcodes map[string]int64 `json:"opcodes"`
RCodes map[string]int64 `json:"rcodes"`
QTypes map[string]int64 `json:"qtypes"`
NSStats map[string]int64 `json:"nsstats"`
Views map[string]struct {
Resolver struct {
Stats map[string]int64 `json:"stats"`
CacheStats map[string]int64 `json:"cachestats"`
Cache map[string]int64 `json:"cache"` // RRsets in the cache by type
} `json:"resolver"`
} `json:"views"`
}
@@ -77,7 +89,21 @@ func Flatten(body []byte) (*Snapshot, error) {
v[name] = n
}
}
g := map[string]int64{"named:mem": j.Memory.InUse,
"named:recursion-highwater": j.NSStats["RecursHighwater"], "named:tcp-highwater": j.NSStats["TCPConnHighWater"]}
for k, n := range j.SockStats {
switch {
case strings.HasPrefix(k, "UDP") && strings.HasSuffix(k, "Active"):
g["sockets:udp"] += n
case strings.HasPrefix(k, "TCP") && strings.HasSuffix(k, "Active"):
g["sockets:tcp"] += n
}
}
for _, view := range j.Views {
g["cache:mem"] += view.Resolver.CacheStats["TreeMemInUse"] + view.Resolver.CacheStats["HeapMemInUse"]
for _, n := range view.Resolver.Cache {
g["cache:rrsets"] += n
}
for k, name := range rttKeys {
v[name] += view.Resolver.Stats[k]
}
@@ -92,27 +118,24 @@ func Flatten(body []byte) (*Snapshot, error) {
delete(v, k)
}
}
return &Snapshot{BootTime: j.BootTime, Values: v}, nil
return &Snapshot{BootTime: j.BootTime, Values: v, Gauges: g, Version: j.Version, ConfigTime: j.ConfigTime}, nil
}
// Delta returns the change from prev to cur. When named restarted (its
// boot time changed or a counter went down), the counters started again
// from zero, so the current values are the change.
// boot time changed), the counters started again from zero, so the
// current values are the change. Without a restart, a counter can still
// go down: the per-view ones (cache, upstream times, validation) are
// summed over named's views, and a reconfig that adds, removes or
// recreates views changes that sum. Such a counter counts 0 for the
// minute; it must never make every counter count from zero, which would
// add everything since named started (millions of "queries" in a minute).
func Delta(prev, cur *Snapshot) map[string]int64 {
out := map[string]int64{}
restarted := prev.BootTime != cur.BootTime
if !restarted {
for k, n := range cur.Values {
if n < prev.Values[k] {
restarted = true
break
}
}
}
for k, n := range cur.Values {
d := n
if !restarted {
d = n - prev.Values[k]
d = max(n-prev.Values[k], 0)
}
if d != 0 || k == "queries" {
out[k] = d
@@ -123,13 +146,13 @@ func Delta(prev, cur *Snapshot) map[string]int64 {
// Poller reads named's statistics channel.
type Poller struct {
URL string // e.g. http://127.0.0.1:8058/json/v1/server
URL string // e.g. http://127.0.0.1:8058/json/v1 (server, memory and sockets)
Client *http.Client
}
// NewPoller returns a poller for a statistics channel address.
func NewPoller(addr string) *Poller {
return &Poller{URL: "http://" + addr + "/json/v1/server", Client: &http.Client{Timeout: 10 * time.Second}}
return &Poller{URL: "http://" + addr + "/json/v1", Client: &http.Client{Timeout: 10 * time.Second}}
}
// Poll fetches a snapshot.
+10 -4
View File
@@ -47,9 +47,15 @@ func TestFlattenAndDelta(t *testing.T) {
if d := Delta(s2, s3); d["queries"] != 3 || d["qtype:A"] != 3 {
t.Errorf("restart delta = %v", d)
}
// A counter going down also means a restart.
s4 := &Snapshot{BootTime: s2.BootTime, Values: map[string]int64{"queries": 2}}
if d := Delta(s2, s4); d["queries"] != 2 {
t.Errorf("reset delta = %v", d)
// A counter going down without a restart (views changed in a
// reconfig) counts 0, and doesn't make the others count from zero.
s4 := &Snapshot{BootTime: s2.BootTime, Values: map[string]int64{}}
for k, v := range s2.Values {
s4.Values[k] = v
}
s4.Values["queries"] += 4
s4.Values["cache:hits"] = 1 // was 4: a view's cache counters are gone
if d := Delta(s2, s4); d["queries"] != 4 || d["cache:hits"] != 0 || len(d) != 1 {
t.Errorf("delta with a counter going down = %v", d)
}
}
+144
View File
@@ -0,0 +1,144 @@
package stats
import (
"bytes"
"os"
"path/filepath"
"runtime"
"strconv"
"strings"
"time"
)
// Server statistics from /proc: named's process and the host. They need no
// privileges: /proc/PID/stat and /proc/PID/status are readable by every
// user (unless /proc is mounted with hidepid).
// clockTicks is USER_HZ, the unit of the CPU times in /proc/PID/stat. It
// is 100 on every Linux architecture Jarl runs on.
const clockTicks = 100
// procRoot is where /proc is (tests point it elsewhere).
var procRoot = "/proc"
// namedProc is a sample of named's process.
type namedProc struct {
PID int
RSS int64 // bytes
Threads int64
CPU int64 // user + system time, in clock ticks
At time.Time
}
// findNamed returns the PID of named: the process named "named" whose
// command line mentions mainConfig, or else the oldest one (the lowest
// PID). 0 when there is none.
func findNamed(mainConfig string) int {
entries, err := os.ReadDir(procRoot)
if err != nil {
return 0
}
best := 0
for _, e := range entries {
pid, err := strconv.Atoi(e.Name())
if err != nil {
continue
}
comm, err := os.ReadFile(filepath.Join(procRoot, e.Name(), "comm"))
if err != nil || strings.TrimSpace(string(comm)) != "named" {
continue
}
if mainConfig != "" {
if cmd, err := os.ReadFile(filepath.Join(procRoot, e.Name(), "cmdline")); err == nil && bytes.Contains(cmd, []byte(mainConfig)) {
return pid
}
}
if best == 0 || pid < best {
best = pid
}
}
return best
}
// readNamedProc samples a process.
func readNamedProc(pid int, now time.Time) (*namedProc, bool) {
dir := filepath.Join(procRoot, strconv.Itoa(pid))
stat, err := os.ReadFile(filepath.Join(dir, "stat"))
if err != nil {
return nil, false
}
// The command name is in parentheses and may contain spaces: the
// fields that follow start after the last ')'.
i := bytes.LastIndexByte(stat, ')')
if i < 0 {
return nil, false
}
f := strings.Fields(string(stat[i+1:]))
// f[0] is field 3 (state); utime and stime are fields 14 and 15.
if len(f) < 13 {
return nil, false
}
utime, _ := strconv.ParseInt(f[11], 10, 64)
stime, _ := strconv.ParseInt(f[12], 10, 64)
p := &namedProc{PID: pid, CPU: utime + stime, At: now}
status, err := os.ReadFile(filepath.Join(dir, "status"))
if err != nil {
return nil, false
}
for _, line := range strings.Split(string(status), "\n") {
k, v, ok := strings.Cut(line, ":")
if !ok {
continue
}
switch k {
case "VmRSS":
p.RSS = kib(v)
case "Threads":
p.Threads, _ = strconv.ParseInt(strings.TrimSpace(v), 10, 64)
}
}
return p, true
}
// kib parses a "1234 kB" value from /proc into bytes.
func kib(v string) int64 {
n, _ := strconv.ParseInt(strings.TrimSpace(strings.TrimSuffix(strings.TrimSpace(v), "kB")), 10, 64)
return n * 1024
}
// Host is the machine's memory, load and uptime.
type Host struct {
MemTotal int64 `json:"mem_total"`
MemAvailable int64 `json:"mem_available"`
Load []float64 `json:"load"` // 1, 5 and 15 minutes
CPUs int `json:"cpus"`
Uptime int64 `json:"uptime"` // seconds
}
func readHost() *Host {
h := &Host{CPUs: runtime.NumCPU()}
if b, err := os.ReadFile(filepath.Join(procRoot, "meminfo")); err == nil {
for _, line := range strings.Split(string(b), "\n") {
k, v, _ := strings.Cut(line, ":")
switch k {
case "MemTotal":
h.MemTotal = kib(v)
case "MemAvailable":
h.MemAvailable = kib(v)
}
}
}
if b, err := os.ReadFile(filepath.Join(procRoot, "loadavg")); err == nil {
for _, f := range strings.Fields(string(b))[:min(3, len(strings.Fields(string(b))))] {
x, _ := strconv.ParseFloat(f, 64)
h.Load = append(h.Load, x)
}
}
if b, err := os.ReadFile(filepath.Join(procRoot, "uptime")); err == nil {
if f := strings.Fields(string(b)); len(f) > 0 {
x, _ := strconv.ParseFloat(f[0], 64)
h.Uptime = int64(x)
}
}
return h
}
+47 -14
View File
@@ -128,7 +128,7 @@ func sorted(m map[string]int64, prefix string) []Entry {
out := []Entry{}
for k, v := range m {
if strings.HasPrefix(k, prefix) && v != 0 {
out = append(out, Entry{strings.TrimPrefix(k, prefix), v})
out = append(out, Entry{Key: strings.TrimPrefix(k, prefix), Count: v})
}
}
slices.SortFunc(out, func(a, b Entry) int {
@@ -153,14 +153,19 @@ type Summary struct {
Queries int64 `json:"queries"`
Per string `json:"per"` // minute, hour or day
} `json:"peak,omitempty"`
RCodes []Entry `json:"rcodes"`
QTypes []Entry `json:"qtypes"`
Views []Entry `json:"views"`
Zones []Entry `json:"zones"`
Answers []Entry `json:"answers"` // authoritative, recursive
Transport []Entry `json:"transport"` // udp, tcp
CacheHitRatio *float64 `json:"cache_hit_ratio,omitempty"`
RCodes []Entry `json:"rcodes"`
QTypes []Entry `json:"qtypes"`
Views []Entry `json:"views"`
Zones []Entry `json:"zones"`
Answers []Entry `json:"answers"` // authoritative, recursive
Transport []Entry `json:"transport"` // udp, tcp
CacheHitRatio *float64 `json:"cache_hit_ratio,omitempty"`
// NXDomainRatio leaves out answers blocking gave (which are
// NXDOMAIN too); Blocked counts those (docs/blocking.md).
NXDomainRatio *float64 `json:"nxdomain_ratio,omitempty"`
Blocked int64 `json:"blocked"`
BlockedRatio *float64 `json:"blocked_ratio,omitempty"`
Blocklists []Entry `json:"blocklists,omitempty"`
Tops map[string][]Entry `json:"tops,omitempty"`
}
@@ -181,9 +186,14 @@ func (q *Querier) Summary(ctx context.Context, since time.Duration, topLimit int
r := float64(h) / float64(h+m)
s.CacheHitRatio = &r
}
s.Blocked, s.Blocklists = all["blocked"], sorted(all, "blocklist:")
if s.Queries > 0 {
r := float64(all["rcode:NXDOMAIN"]) / float64(s.Queries)
r := float64(max(all["rcode:NXDOMAIN"]-s.Blocked, 0)) / float64(s.Queries)
s.NXDomainRatio = &r
if s.Blocked > 0 {
b := float64(s.Blocked) / float64(s.Queries)
s.BlockedRatio = &b
}
}
per, err := q.Store.series(ctx, res, from, to, "queries")
if err != nil {
@@ -209,7 +219,7 @@ func (q *Querier) Summary(ctx context.Context, since time.Duration, topLimit int
}
if topLimit > 0 {
s.Tops = map[string][]Entry{}
for _, k := range []string{KindName, KindClient, KindNXDomain} {
for _, k := range []string{KindName, KindClient, KindNXDomain, KindBlocked} {
es, err := q.Top(ctx, k, since, topLimit)
if err != nil {
return nil, err
@@ -246,7 +256,7 @@ func (q *Querier) Breakdown(ctx context.Context, group string, since time.Durati
}
// Groups are the counter groups Breakdown accepts.
var Groups = []string{"qtype", "rcode", "view", "zone", "answer", "transport", "rtt", "dnssec", "cache", "ratelimit"}
var Groups = []string{"qtype", "rcode", "view", "zone", "answer", "transport", "rtt", "dnssec", "cache", "ratelimit", "blocklist"}
// Timeline is a set of series over time.
type Timeline struct {
@@ -255,7 +265,8 @@ type Timeline struct {
Series []*Series `json:"series"`
}
// Series is one counter over time.
// Series is one counter over time. For a gauge (g:KEY) the values are
// averages, and -1 where there is no sample.
type Series struct {
Key string `json:"key"`
Values []int64 `json:"values"`
@@ -330,8 +341,8 @@ func (q *Querier) Timeline(ctx context.Context, since time.Duration, step string
expanded = append(expanded, k)
}
tl := &Timeline{Step: st.name, Times: starts, Series: []*Series{}}
for _, k := range expanded {
raw, err := q.Store.series(ctx, st.res, from, to, k)
bucketed := func(key string) ([]int64, error) {
raw, err := q.Store.series(ctx, st.res, from, to, key)
if err != nil {
return nil, err
}
@@ -346,6 +357,28 @@ func (q *Querier) Timeline(ctx context.Context, since time.Duration, step string
vals[i] += v
}
}
return vals, nil
}
for _, k := range expanded {
vals, err := bucketed(k)
if err != nil {
return nil, err
}
// A gauge (g:KEY) is the average of its samples: its sum divided
// by gn:KEY, their number. A bucket without samples is -1.
if g, ok := strings.CutPrefix(k, "g:"); ok {
n, err := bucketed("gn:" + g)
if err != nil {
return nil, err
}
for i := range vals {
if n[i] > 0 {
vals[i] /= n[i]
} else {
vals[i] = -1
}
}
}
tl.Series = append(tl.Series, &Series{Key: k, Values: vals})
}
return tl, nil
+112
View File
@@ -0,0 +1,112 @@
package stats
import "time"
// Server is the latest server statistics: named's process, memory, cache
// and sockets, and the host's memory and load. It is sampled with the
// counters, once a minute.
type Server struct {
At time.Time `json:"at"`
Version string `json:"version,omitempty"`
BootTime string `json:"boot_time,omitempty"`
ConfigTime string `json:"config_time,omitempty"`
// From /proc (0 when named's process isn't visible).
PID int `json:"pid,omitempty"`
RSS int64 `json:"rss,omitempty"` // resident memory, bytes
Threads int64 `json:"threads,omitempty"`
CPUPercent float64 `json:"cpu_percent"` // of one CPU, over the last minute
// From the statistics channel.
MemInUse int64 `json:"mem_in_use"` // named's own count of the memory it uses
CacheRRsets int64 `json:"cache_rrsets"`
CacheMem int64 `json:"cache_mem"`
SocketsUDP int64 `json:"sockets_udp"`
SocketsTCP int64 `json:"sockets_tcp"`
RecursionHighwater int64 `json:"recursion_highwater"` // most recursive clients at once since named started
TCPHighwater int64 `json:"tcp_highwater"`
Host *Host `json:"host"`
}
// Gauge keys: each is stored as two counters, "g:KEY" (the sum of the
// minute samples) and "gn:KEY" (their number), so hours and days add up
// like counters and a bucket's value is their average (see Timeline).
const (
GaugeRSS = "named:rss"
GaugeMem = "named:mem"
GaugeCPU = "named:cpu" // tenths of a percent of one CPU
GaugeCacheMem = "cache:mem"
GaugeCacheRRs = "cache:rrsets"
GaugeHostMem = "host:mem-used"
GaugeHostLoad = "host:load" // the 1-minute load average, times 100
)
// gauge records a sample of a gauge for a minute (none when minute < 0).
func (c *Collector) gauge(minute int64, key string, v int64) {
if minute < 0 {
return
}
c.count(minute, "g:"+key, v)
c.count(minute, "gn:"+key, 1)
}
// sampleServer records the gauges for a minute and keeps the latest
// values (minute -1: only those). snap is nil when named's statistics
// channel didn't answer.
// Called with c.mu held.
func (c *Collector) sampleServer(minute int64, now time.Time, snap *Snapshot) {
srv := &Server{At: now, Host: readHost()}
if c.proc == nil || c.proc.PID == 0 {
c.proc = &namedProc{PID: findNamed(c.opts.MainConfig)}
}
p, ok := readNamedProc(c.proc.PID, now)
if !ok {
// named restarted with a new PID: look again next minute.
c.proc = nil
} else {
srv.PID, srv.RSS, srv.Threads = p.PID, p.RSS, p.Threads
if prev := c.proc; prev.At.Before(now) && !prev.At.IsZero() && p.CPU >= prev.CPU {
// Tenths of a percent of one CPU: ticks / clockTicks is CPU seconds.
permille := (p.CPU - prev.CPU) * 1000 / clockTicks / max(int64(now.Sub(prev.At)/time.Second), 1)
srv.CPUPercent = float64(permille) / 10
c.gauge(minute, GaugeCPU, permille)
}
c.proc = p
c.gauge(minute, GaugeRSS, p.RSS)
}
if h := srv.Host; h.MemTotal > 0 {
c.gauge(minute, GaugeHostMem, h.MemTotal-h.MemAvailable)
}
if h := srv.Host; len(h.Load) > 0 {
c.gauge(minute, GaugeHostLoad, int64(h.Load[0]*100))
}
if snap != nil && snap.Gauges != nil {
g := snap.Gauges
srv.Version, srv.BootTime, srv.ConfigTime = snap.Version, snap.BootTime, snap.ConfigTime
srv.MemInUse, srv.CacheRRsets, srv.CacheMem = g["named:mem"], g["cache:rrsets"], g["cache:mem"]
srv.SocketsUDP, srv.SocketsTCP = g["sockets:udp"], g["sockets:tcp"]
srv.RecursionHighwater, srv.TCPHighwater = g["named:recursion-highwater"], g["named:tcp-highwater"]
c.gauge(minute, GaugeMem, srv.MemInUse)
c.gauge(minute, GaugeCacheMem, srv.CacheMem)
c.gauge(minute, GaugeCacheRRs, srv.CacheRRsets)
}
c.server = srv
}
// Server returns the latest server statistics (nil before the first
// minute).
func (c *Collector) Server() *Server {
c.mu.Lock()
defer c.mu.Unlock()
if c.server == nil {
return nil
}
s := *c.server
return &s
}
// Disk is the space on a file system Jarl and named use.
type Disk struct {
Path string `json:"path"` // the first of the directories on it
Holds []string `json:"holds"` // what is on it: zone files, logs, ...
Total int64 `json:"total"`
Free int64 `json:"free"` // available to unprivileged users
}
+96
View File
@@ -0,0 +1,96 @@
package stats
import (
"context"
"os"
"path/filepath"
"strconv"
"testing"
"time"
)
func TestProc(t *testing.T) {
root := t.TempDir()
old := procRoot
procRoot = root
defer func() { procRoot = old }()
proc := func(pid, comm, cmdline string, utime, stime int) {
d := filepath.Join(root, pid)
os.MkdirAll(d, 0o755)
os.WriteFile(filepath.Join(d, "comm"), []byte(comm+"\n"), 0o644)
os.WriteFile(filepath.Join(d, "cmdline"), []byte(cmdline), 0o644)
// The command name may hold spaces and parentheses.
stat := pid + " (" + comm + " (x)) S 1 1 1 0 -1 4194560 100 0 0 0 " + strconv.Itoa(utime) + " " + strconv.Itoa(stime) + " 0 0 20 0 9 0 100 0 0\n"
os.WriteFile(filepath.Join(d, "stat"), []byte(stat), 0o644)
os.WriteFile(filepath.Join(d, "status"), []byte("Name:\t"+comm+"\nVmRSS:\t 51200 kB\nThreads:\t9\n"), 0o644)
}
proc("90", "named", "/usr/sbin/named\x00-c\x00/etc/other.conf", 1, 1)
proc("120", "named", "/usr/sbin/named\x00-c\x00/etc/bind/named.conf\x00-u\x00bind", 300, 100)
proc("130", "bash", "bash", 0, 0)
if pid := findNamed("/etc/bind/named.conf"); pid != 120 {
t.Errorf("findNamed = %d, want the one with the config", pid)
}
if pid := findNamed("/nowhere.conf"); pid != 90 {
t.Errorf("findNamed without a match = %d, want the oldest", pid)
}
now := time.Unix(1_000_000, 0)
p, ok := readNamedProc(120, now)
if !ok || p.RSS != 51200*1024 || p.Threads != 9 || p.CPU != 400 {
t.Fatalf("readNamedProc = %+v, %v", p, ok)
}
os.WriteFile(filepath.Join(root, "meminfo"), []byte("MemTotal: 4000000 kB\nMemFree: 100 kB\nMemAvailable: 3000000 kB\n"), 0o644)
os.WriteFile(filepath.Join(root, "loadavg"), []byte("0.52 0.40 0.31 1/300 4242\n"), 0o644)
os.WriteFile(filepath.Join(root, "uptime"), []byte("86400.12 1000.0\n"), 0o644)
h := readHost()
if h.MemTotal != 4000000*1024 || h.MemAvailable != 3000000*1024 || len(h.Load) != 3 || h.Load[0] != 0.52 || h.Uptime != 86400 {
t.Errorf("readHost = %+v", h)
}
// Two samples a minute apart: 30 seconds of CPU time is 50%.
c := &Collector{minutes: map[int64]map[string]int64{}, opts: Options{MainConfig: "/etc/bind/named.conf"}}
snap := &Snapshot{Gauges: map[string]int64{"named:mem": 1000, "cache:rrsets": 7, "cache:mem": 300, "sockets:tcp": 2}, Version: "9.20"}
c.sampleServer(-1, now, snap)
if len(c.minutes) != 0 {
t.Errorf("minute -1 recorded gauges: %v", c.minutes)
}
proc("120", "named", "/usr/sbin/named\x00-c\x00/etc/bind/named.conf", 2800, 600) // +3000 ticks = 30 s
c.sampleServer(600, now.Add(time.Minute), snap)
s := c.Server()
if s.PID != 120 || s.CPUPercent != 50 || s.RSS != 51200*1024 || s.CacheRRsets != 7 || s.SocketsTCP != 2 || s.Version != "9.20" {
t.Errorf("server = %+v", s)
}
m := c.minutes[600]
if m["g:"+GaugeCPU] != 500 || m["gn:"+GaugeCPU] != 1 || m["g:"+GaugeMem] != 1000 || m["g:"+GaugeHostLoad] != 52 {
t.Errorf("gauges = %v", m)
}
}
func TestGaugeTimeline(t *testing.T) {
store, err := OpenStore(":memory:")
if err != nil {
t.Fatal(err)
}
defer store.Close()
now := time.Unix(1_800_000_000, 0)
start := now.Unix() - now.Unix()%300 - 600 // two 5-minute buckets ago
b := &Batch{}
for i := int64(0); i < 5; i++ {
b.AddCounters(ResMinute, start+i*60, map[string]int64{"g:named:rss": 100 + i*10, "gn:named:rss": 1})
}
// The next bucket has no samples.
if err := store.Commit(context.Background(), b); err != nil {
t.Fatal(err)
}
q := &Querier{Store: store, Opts: Options{RetainMinute: 7 * 24 * time.Hour, RetainHour: 90 * 24 * time.Hour}, Now: func() time.Time { return now }}
tl, err := q.Timeline(context.Background(), 15*time.Minute, "5m", []string{"g:named:rss"})
if err != nil {
t.Fatal(err)
}
vals := map[int64]int64{}
for i, ts := range tl.Times {
vals[ts] = tl.Series[0].Values[i]
}
if vals[start] != 120 || vals[start+300] != -1 {
t.Errorf("gauge timeline = %v (times %v)", tl.Series[0].Values, tl.Times)
}
}
Loaded 100 of 123 files, more files were not shown because too many files have changed in this diff. Show more