Compare commits

...
4 Commits
Author SHA1 Message Date
adamandClaude Opus 5.5 5443e95897 Release 0.6.0-rc.1
release / release (push) Successful in 2m13s
0.6.0's first release candidate: reverse zones made easy, comparing views rebuilt, exploring the statistics through a query log, storage on the dashboard, encrypted DNS with several host names and views per listen address, option set matching named statements, forwarding over TLS documented. The upgrade from 0.5.2 packages was tested in a Debian 13 container.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 18:44:50 +01:00
adamandClaude Opus 5.5 ea26171875 option set: statements with a name replace only that one; forwarding over TLS documented
jarl option set tls NAME (and key, remote-servers, primaries, http, dnssec-policy, key-store, server, acl, view, zone, parental-agents) matches the name after the keyword too: setting tls cloudflare no longer replaces tls quad9. jarl option unset tls NAME removes one by its name. docs/encrypted-dns.md has forwarding to Quad9 or Cloudflare over TLS: one changeset, the options explained, several providers, checking it works, and why not over HTTPS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 18:39:28 +01:00
adamandClaude Opus 5.5 250e734046 Query log wording: N answers so far, each kept for 7 days
"The query log holds 7 answers (stats.retention.queries)" read like 7 days' worth. The dashboard and the Statistics page now say how many answers it holds and how long each is kept; the dashboard's storage info has query_log_retention.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 18:27:56 +01:00
adamandClaude Opus 5.5 1b102a78f6 Statistics page: the statistics database's size, and a fullness bar on the disk tiles
The server section shows the statistics database's size (with its -wal and -shm) and how many answers the query log holds, linking to the dashboard's Storage panel; the disk tiles get the same bar as the dashboard. GET /stats/server has database {path, bytes, query_log_rows}.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 18:24:32 +01:00
17 changed files with 277 additions and 24 deletions

No files matched your search

+13 -2
View File
@@ -57,6 +57,8 @@ All notable changes to Jarl are documented here. The format follows
starts from where the records are now. The Certificates page and `jarl starts from where the records are now. The Certificates page and `jarl
cert show` say what named uses a certificate for (DNS over TLS or HTTPS, cert show` say what named uses a certificate for (DNS over TLS or HTTPS,
on which addresses and ports). on which addresses and ports).
- docs/encrypted-dns.md: forwarding to Quad9 or Cloudflare over TLS, how to
check it works, and why not over HTTPS.
- Comparing one name across views: `jarl zone diff adbraun.lol nas [A]`, - Comparing one name across views: `jarl zone diff adbraun.lol nas [A]`,
and a **Compare** button on each record. and a **Compare** button on each record.
@@ -69,10 +71,15 @@ All notable changes to Jarl are documented here. The format follows
the clients asking → everything one of them asks. `GET /stats/explore`. the clients asking → everything one of them asks. `GET /stats/explore`.
- **Storage** on the dashboard: the size of Jarl's databases (and how many - **Storage** on the dashboard: the size of Jarl's databases (and how many
answers the query log holds), and per disk Jarl uses a bar of how full it answers the query log holds), and per disk Jarl uses a bar of how full it
is and how much of it the databases take. is and how much of it the databases take. The Statistics page shows the
statistics database's size too, and its disk tiles get the same bar.
### Changed ### Changed
- `jarl option set tls NAME …` (and `key`, `remote-servers`, `http`,
`dnssec-policy`, `server` …) replaces only the statement with that name:
setting `tls cloudflare` used to replace `tls quad9`. `jarl option unset
tls cloudflare` removes one by its name.
- **Compare views** and `jarl zone diff` are rebuilt around the view - **Compare views** and `jarl zone diff` are rebuilt around the view
you're on (docs/views.md#comparing-views): a summary per view, then you're on (docs/views.md#comparing-views): a summary per view, then
what is answered differently (a view's own records in linked zones, what is answered differently (a view's own records in linked zones,
@@ -88,8 +95,12 @@ All notable changes to Jarl are documented here. The format follows
browser console reported a Content Security Policy violation (an inline browser console reported a Content Security Policy violation (an inline
style the page's policy blocks). style the page's policy blocks).
### Upgrading ### Upgrading from 0.5
- Tested: 0.5.2 packages with views, a linked zone, a reverse zone,
blocking, a `tls` statement and statistics, upgraded in place in Debian
13 (BIND 9.20.29): everything kept and answered as before, the new
features worked on the upgraded data, and a 0.5.2 backup restored.
- The database gets two new versions (12 and 13), applied when jarld - The database gets two new versions (12 and 13), applied when jarld
starts; a 0.5 jarld can't open it afterwards, so take a backup first (`sudo starts; a 0.5 jarld can't open it afterwards, so take a backup first (`sudo
jarld backup`). Nothing changes until you turn `auto_ptr` on for a jarld backup`). Nothing changes until you turn `auto_ptr` on for a
+6 -1
View File
@@ -202,7 +202,12 @@ review step.
## Status ## Status
Version 0.5.2 is the current release. It adds Prometheus metrics, Version 0.5.2 is the current release, and 0.6.0 is in testing
(0.6.0-rc.1): it makes reverse zones easy (suggested from your records,
created in the right view and filled, PTR records kept in step), rebuilds
comparing a zone's views, lets you explore the statistics down to each
client and name (a query log), shows the databases' size, and gives
encrypted DNS several host names and views per listen address. 0.5 added Prometheus metrics,
address ranges in ACLs and other address match lists, safer changeset address ranges in ACLs and other address match lists, safer changeset
handling, internet exposure checks with response rate limiting, TLS handling, internet exposure checks with response rate limiting, TLS
certificates Jarl obtains and renews itself, linked zones for split DNS, certificates Jarl obtains and renews itself, linked zones for split DNS,
+1 -1
View File
@@ -1 +1 @@
0.5.2 0.6.0-rc.1
+51 -5
View File
@@ -586,6 +586,12 @@ func optionCmd() *cobra.Command {
Short: "Set an option (replaces an existing option of the same name)", Short: "Set an option (replaces an existing option of the same name)",
Long: `Set an option; VALUE is named.conf syntax without the trailing semicolon. Long: `Set an option; VALUE is named.conf syntax without the trailing semicolon.
It replaces the option of that name. Statements told apart by the name
after the keyword (tls NAME, key NAME, remote-servers NAME, http NAME,
dnssec-policy NAME, server ADDRESS, ...) replace only the one with that
name: setting tls cloudflare leaves tls quad9 alone. --add adds another
instance of any other option that may occur several times.
jarl option set recursion no jarl option set recursion no
jarl option set allow-recursion '{ homelab; localhost; }' jarl option set allow-recursion '{ homelab; localhost; }'
jarl option set forwarders '{ 1.1.1.1; 9.9.9.9; }' jarl option set forwarders '{ 1.1.1.1; 9.9.9.9; }'
@@ -610,7 +616,11 @@ func optionCmd() *cobra.Command {
if cmd.Flags().Changed("comment") { if cmd.Flags().Changed("comment") {
body["comment"] = comment body["comment"] = comment
} }
label := optionLabel(strings.ToLower(args[0]), value)
for _, o := range opts { for _, o := range opts {
if label != "" && optionLabel(o.Name, o.Value) != label {
continue // another tls (key, server, ...) statement
}
if o.Name == strings.ToLower(args[0]) && !multiple { if o.Name == strings.ToLower(args[0]) && !multiple {
resp, err := c.Mutate(ctx(), "PATCH", fmt.Sprintf("/options/%d", o.ID), nil, body) resp, err := c.Mutate(ctx(), "PATCH", fmt.Sprintf("/options/%d", o.ID), nil, body)
if err != nil { if err != nil {
@@ -631,7 +641,7 @@ func optionCmd() *cobra.Command {
set.Flags().StringVar(&comment, "comment", "", "comment") set.Flags().StringVar(&comment, "comment", "", "comment")
set.Flags().BoolVar(&multiple, "add", false, "add another instance instead of replacing (for options that may occur several times)") set.Flags().BoolVar(&multiple, "add", false, "add another instance instead of replacing (for options that may occur several times)")
unset := &cobra.Command{ unset := &cobra.Command{
Use: "unset NAME | unset --id ID", Use: "unset NAME [LABEL] | unset --id ID",
Short: "Remove an option (BIND's default applies again)", Short: "Remove an option (BIND's default applies again)",
RunE: func(cmd *cobra.Command, args []string) error { RunE: func(cmd *cobra.Command, args []string) error {
c, err := newClient() c, err := newClient()
@@ -645,8 +655,8 @@ func optionCmd() *cobra.Command {
} }
return report(resp) return report(resp)
} }
if len(args) != 1 { if len(args) < 1 || len(args) > 2 {
return fmt.Errorf("give the option name or --id") return fmt.Errorf("give the option name (and the name after it, for tls NAME and the like) or --id")
} }
sc, id, err := scope() sc, id, err := scope()
if err != nil { if err != nil {
@@ -658,15 +668,24 @@ func optionCmd() *cobra.Command {
} }
var matches []api.Option var matches []api.Option
for _, o := range opts { for _, o := range opts {
if o.Name == strings.ToLower(args[0]) { if o.Name == strings.ToLower(args[0]) && (len(args) == 1 || strings.EqualFold(optionLabel(o.Name, o.Value), strings.Trim(args[1], `"`))) {
matches = append(matches, o) matches = append(matches, o)
} }
} }
switch len(matches) { switch len(matches) {
case 0: case 0:
return fmt.Errorf("option %s is not set here", args[0]) return fmt.Errorf("option %s is not set here", strings.Join(args, " "))
case 1: case 1:
default: default:
var labels []string
for _, o := range matches {
if l := optionLabel(o.Name, o.Value); l != "" {
labels = append(labels, l)
}
}
if len(labels) == len(matches) {
return fmt.Errorf("option %s is set %d times (%s); say which: jarl option unset %s %s", args[0], len(matches), strings.Join(labels, ", "), args[0], labels[0])
}
return fmt.Errorf("option %s is set %d times; remove one with --id", args[0], len(matches)) return fmt.Errorf("option %s is set %d times; remove one with --id", args[0], len(matches))
} }
resp, err := c.Mutate(ctx(), "DELETE", fmt.Sprintf("/options/%d", matches[0].ID), nil, nil) resp, err := c.Mutate(ctx(), "DELETE", fmt.Sprintf("/options/%d", matches[0].ID), nil, nil)
@@ -847,3 +866,30 @@ func settingsCmd() *cobra.Command {
cmd.AddCommand(show, set) cmd.AddCommand(show, set)
return cmd return cmd
} }
// labelled are the statements told apart by the name after the keyword:
// tls quad9 { ... }, key xfer { ... }, server 192.0.2.1 { ... }.
var labelled = map[string]bool{"tls": true, "http": true, "key": true, "remote-servers": true, "primaries": true, "masters": true,
"parental-agents": true, "dnssec-policy": true, "key-store": true, "server": true, "view": true, "acl": true, "zone": true}
// optionLabel returns the name a labelled statement's value starts with
// ("quad9" for tls quad9 { ... }), lowercase; "" for other statements.
func optionLabel(name, value string) string {
if !labelled[name] {
return ""
}
v := strings.TrimSpace(value)
if v == "" || strings.HasPrefix(v, "{") {
return ""
}
if strings.HasPrefix(v, `"`) {
if end := strings.Index(v[1:], `"`); end >= 0 {
return strings.ToLower(v[1 : end+1])
}
}
end := strings.IndexAny(v, " \t\n{;")
if end < 0 {
end = len(v)
}
return strings.ToLower(v[:end])
}
+17
View File
@@ -232,3 +232,20 @@ func TestOutputFields(t *testing.T) {
} }
} }
} }
func TestOptionLabel(t *testing.T) {
for _, c := range []struct{ name, value, want string }{
{"tls", `quad9 { remote-hostname "dns.quad9.net"; }`, "quad9"},
{"tls", `Cloudflare{ remote-hostname "cloudflare-dns.com"; }`, "cloudflare"},
{"key", `"xfer key" { algorithm hmac-sha256; secret "x"; }`, "xfer key"},
{"server", "192.0.2.1 { keys { xfer; }; }", "192.0.2.1"},
{"remote-servers", "secondaries { 192.0.2.53; }", "secondaries"},
{"forwarders", "{ 9.9.9.9; }", ""},
{"logging", "{ channel x { stderr; }; }", ""},
{"tls", "", ""},
} {
if got := optionLabel(c.name, c.value); got != c.want {
t.Errorf("optionLabel(%q, %q) = %q, want %q", c.name, c.value, got, c.want)
}
}
}
+1 -1
View File
@@ -566,7 +566,7 @@ Needs `stats:read`. When statistics are disabled, every endpoint but
| GET | `/stats/top` | `kind` (name, client, nxdomain, servfail, refused, blocked, blocked_client), `since`, `limit` (default 20) | `[{key, count}]`; client, refused and blocked_client entries add `name`, the host name of Jarl's record for the address (the summary's client list too) | | GET | `/stats/top` | `kind` (name, client, nxdomain, servfail, refused, blocked, blocked_client), `since`, `limit` (default 20) | `[{key, count}]`; client, refused and blocked_client entries add `name`, the host name of Jarl's record for the address (the summary's client list too) |
| GET | `/stats/heatmap` | `since` (default 28d), `key` (default queries) | `{key, from, avg[7][24], samples[7][24]}`, Monday first, local time, `-1` = no data | | GET | `/stats/heatmap` | `since` (default 28d), `key` (default queries) | `{key, from, avg[7][24], samples[7][24]}`, Monday first, local time, `-1` = no data |
| GET | `/stats/explore` | `?client=&name=&domain=&view=&type=&rcode=&blocked=yes\|no\|LIST&since=24h&views=` | The query log for a filter ([stats.md](stats.md#exploring)): `total`, `blocked`, `series` (per `step` seconds: `t`, `count`, `blocked`), `by` (top 15 of each dimension the filter leaves open: client, name, view, type, rcode, blocked; clients with `name`), `recent` (latest 100: ts, client, client_name, view, name, type, rcode, blocked), `oldest`, `retention`, `client_name`. 404 `query_log_off` without a query log | | GET | `/stats/explore` | `?client=&name=&domain=&view=&type=&rcode=&blocked=yes\|no\|LIST&since=24h&views=` | The query log for a filter ([stats.md](stats.md#exploring)): `total`, `blocked`, `series` (per `step` seconds: `t`, `count`, `blocked`), `by` (top 15 of each dimension the filter leaves open: client, name, view, type, rcode, blocked; clients with `name`), `recent` (latest 100: ts, client, client_name, view, name, type, rcode, blocked), `oldest`, `retention`, `client_name`. 404 `query_log_off` without a query log |
| GET | `/stats/server` | | `{server: {rss, mem_in_use, cpu_percent, threads, pid, cache_rrsets, cache_mem, sockets_udp, sockets_tcp, recursion_highwater, tcp_highwater, version, boot_time, config_time, host: {mem_total, mem_available, load, cpus, uptime}}, named: {recursive_clients, tcp_clients, zones, cpus, worker_threads}, disks: [{path, holds, total, free}]}`; gauges over time: `/stats/timeline?key=g:named:rss` (averages, -1 without a sample) | | GET | `/stats/server` | | `{server: {rss, mem_in_use, cpu_percent, threads, pid, cache_rrsets, cache_mem, sockets_udp, sockets_tcp, recursion_highwater, tcp_highwater, version, boot_time, config_time, host: {mem_total, mem_available, load, cpus, uptime}}, named: {recursive_clients, tcp_clients, zones, cpus, worker_threads}, disks: [{path, holds, total, free}], database: {path, bytes, query_log_rows}}`; gauges over time: `/stats/timeline?key=g:named:rss` (averages, -1 without a sample) |
| GET | `/stats/breakdown` | `group` (qtype, rcode, view, zone, answer, transport, rtt, dnssec, cache, ratelimit, blocklist), `since` | `[{key, count}]` | | GET | `/stats/breakdown` | `group` (qtype, rcode, view, zone, answer, transport, rtt, dnssec, cache, ratelimit, blocklist), `since` | `[{key, count}]` |
```sh ```sh
+10
View File
@@ -482,12 +482,22 @@ jarl option set allow-transfer 'key xfer; 10.0.10.53' --zone adbraun.lol
jarl option set allow-query '10.0.20.1-10.0.20.99; localhost' --view internal jarl option set allow-query '10.0.20.1-10.0.20.99; localhost' --view internal
jarl option set response-policy '{ zone "rpz.local"; }' --view internal jarl option set response-policy '{ zone "rpz.local"; }' --view internal
jarl option set logging '{ channel q { file "/var/log/named/query.log" versions 3 size 20m; }; category queries { q; }; }' --toplevel jarl option set logging '{ channel q { file "/var/log/named/query.log" versions 3 size 20m; }; category queries { q; }; }' --toplevel
jarl option set tls 'quad9 { remote-hostname "dns.quad9.net"; ca-file "/etc/ssl/certs/ca-certificates.crt"; }' --toplevel
jarl option set tls 'cloudflare { remote-hostname "cloudflare-dns.com"; }' --toplevel # tls quad9 stays
jarl option unset tls cloudflare --toplevel # statements with a name: say which
jarl option unset forwarders jarl option unset forwarders
jarl option describe # every option allowed at this scope jarl option describe # every option allowed at this scope
jarl option describe allow-query-cache # grammar and ARM description jarl option describe allow-query-cache # grammar and ARM description
jarl option generated # what Jarl adds to named.conf by itself, and why jarl option generated # what Jarl adds to named.conf by itself, and why
``` ```
`option set` replaces the option of that name. Statements told apart by
the name after the keyword (`tls NAME`, `key NAME`, `remote-servers NAME`,
`http NAME`, `dnssec-policy NAME`, `server ADDRESS`, ...) replace only the
one with that name; `--add` adds another instance of any other option that
may occur several times. Forwarding to Quad9 or Cloudflare over TLS:
[encrypted-dns.md](encrypted-dns.md#forwarding-over-tls).
Options that end in a list (`allow-*`, `forwarders`, `listen-on`, ...) may Options that end in a list (`allow-*`, `forwarders`, `listen-on`, ...) may
be given without the braces: elements are separated by `;`, `,` or spaces, be given without the braces: elements are separated by `;`, `,` or spaces,
and `key NAME` and `port N` stay with their element. Options are always and `key NAME` and `port N` stay with their element. Options are always
+60
View File
@@ -14,6 +14,10 @@ host name's address records in your private views.
In the web UI: **Encrypted DNS** (administrators). In the CLI: `jarl In the web UI: **Encrypted DNS** (administrators). In the CLI: `jarl
encrypted-dns`. In the API: `/api/v1/encrypted-dns`. encrypted-dns`. In the API: `/api/v1/encrypted-dns`.
That is encryption between your clients and this server. For encryption
between this server and the resolvers it forwards to (Quad9, Cloudflare),
see [Forwarding over TLS](#forwarding-over-tls) below.
## Setting it up ## Setting it up
```sh ```sh
@@ -151,6 +155,62 @@ What it can say:
| not from a CA this server trusts (warning) | a staging or test CA: clients reject it unless they trust it | | not from a CA this server trusts (warning) | a staging or test CA: clients reject it unless they trust it |
| REFUSED (warning) | it works, but this server's own address gets a view that doesn't answer for the host name | | REFUSED (warning) | it works, but this server's own address gets a view that doesn't answer for the host name |
## Forwarding over TLS
When a view forwards its recursive queries (to Quad9, Cloudflare, your
ISP's resolver), named can send them over TLS, so your ISP doesn't see the
names you look up: only connections to the forwarder on port 853. It takes
two options, a `tls` statement for each provider (the name its certificate
must have) and the view's `forwarders` using it. Stage them in one
changeset, so one revert undoes them:
```sh
CS=$(jarl changeset create "internal: forward to Quad9 over TLS" -q)
jarl option set tls 'quad9 { remote-hostname "dns.quad9.net"; ca-file "/etc/ssl/certs/ca-certificates.crt"; }' --toplevel -C $CS
jarl option set forwarders '{ 9.9.9.9 port 853 tls quad9; 149.112.112.112 port 853 tls quad9; 2620:fe::fe port 853 tls quad9; 2620:fe::9 port 853 tls quad9; }' --view internal -C $CS
jarl option set forward only --view internal -C $CS
jarl changeset diff $CS # what named.conf gets
jarl changeset apply $CS
# undo: jarl changeset apply $(jarl changeset revert $CS -q)
```
* **`remote-hostname` and `ca-file`** make named check the forwarder's
certificate against that name (Debian's CA bundle at that path). Without
them the connection is encrypted but anyone in between could answer.
* **`forward only`** stops named falling back to plain DNS (to the root
servers) when the forwarders don't answer: lookups then fail instead
(SERVFAIL), which you notice, rather than quietly going unencrypted.
* **Several providers**: a `tls` statement each, and their addresses in one
`forwarders` list. Setting `tls cloudflare` doesn't touch `tls quad9`:
`jarl option set` replaces only the statement of that name.
```sh
jarl option set tls 'cloudflare { remote-hostname "cloudflare-dns.com"; ca-file "/etc/ssl/certs/ca-certificates.crt"; }' --toplevel
jarl option set forwarders '{ 9.9.9.9 port 853 tls quad9; 149.112.112.112 port 853 tls quad9; 1.1.1.1 port 853 tls cloudflare; 1.0.0.1 port 853 tls cloudflare; 2606:4700:4700::1111 port 853 tls cloudflare; 2606:4700:4700::1001 port 853 tls cloudflare; }' --view internal
```
named sends each query to whichever forwarder has answered fastest, not
in order, and the providers filter differently (Quad9 blocks known
malicious domains, Cloudflare's 1.1.1.1 doesn't; 1.1.1.2 does, with
`remote-hostname "security.cloudflare-dns.com"`).
* **DNS over HTTPS** can't be used for forwarding: BIND serves DoH to
clients but forwards only over plain DNS or TLS. For a DoH upstream, run
a local proxy (dnscrypt-proxy, cloudflared) and forward to it on
`127.0.0.1`.
* Providers write their DoT address as `tls://dns.quad9.net`: the name is
`remote-hostname`, and the addresses it resolves to go in `forwarders`
(named needs addresses there, not names).
**Checking it works**: Quad9 says how a query reached it.
```sh
dig +short TXT proto.on.quad9.net @YOUR-SERVER # "dot": over TLS
```
`do53-udp` means plain DNS. `ss -tnp | grep :853` shows named's connections
only briefly: it closes them when idle. `tcpdump -ni any port 853` shows
the traffic as it happens.
## What Jarl checks ## What Jarl checks
* A `tls` statement whose files don't exist is an error * A `tls` statement whose files don't exist is an error
+4 -2
View File
@@ -221,8 +221,10 @@ view switches apply here too.
`anonymize_clients` and `exclude_clients` apply to the query log as to `anonymize_clients` and `exclude_clients` apply to the query log as to
everything else. Its size: each answer takes about 150 to 200 bytes with everything else. Its size: each answer takes about 150 to 200 bytes with
its indexes, so 50,000 answers a day for 7 days is about 60 MB, 200,000 a its indexes, so 50,000 answers a day for 7 days is about 60 MB, 200,000 a
day about 250 MB. The dashboard's Storage panel shows the databases' size day about 250 MB. The Statistics page's server section shows the
and how many answers the query log holds. statistics database's size and how many answers the query log holds, next
to the disks (with a bar of how full each is); the dashboard's Storage
panel has all of Jarl's databases.
## Views ## Views
+4 -2
View File
@@ -294,8 +294,10 @@ type Database struct {
Label string `json:"label"` Label string `json:"label"`
Path string `json:"path"` Path string `json:"path"`
Bytes int64 `json:"bytes"` Bytes int64 `json:"bytes"`
// Rows is how many answers the query log holds (statistics only). // Rows is how many answers the query log holds (statistics only),
QueryLogRows *int64 `json:"query_log_rows,omitempty"` // and QueryLogRetention how long it keeps them ("7d").
QueryLogRows *int64 `json:"query_log_rows,omitempty"`
QueryLogRetention string `json:"query_log_retention,omitempty"`
} }
// Disk is a file system Jarl uses: what of Jarl's it holds, its size, // Disk is a file system Jarl uses: what of Jarl's it holds, its size,
+10
View File
@@ -283,6 +283,16 @@ func (s *Server) statsServer(w http.ResponseWriter, r *http.Request, p *auth.Pri
labels = append(labels, "statistics database") labels = append(labels, "statistics database")
} }
out["disks"] = stats.DiskUsage(labels, dirs) out["disks"] = stats.DiskUsage(labels, dirs)
// The statistics database: what the statistics cost on disk.
if db := s.cfg.Stats.Store().Path(); db != "" && db != ":memory:" {
info := map[string]any{"path": db, "bytes": dbBytes(db)}
if keep := s.cfg.Stats.Options().RetainQueries; keep > 0 && s.cfg.Stats.Options().QueryLog {
if n, err := s.cfg.Stats.Store().QueryLogSize(r.Context()); err == nil {
info["query_log_rows"] = n
}
}
out["database"] = info
}
writeJSON(w, http.StatusOK, out) writeJSON(w, http.StatusOK, out)
return nil return nil
} }
+13 -6
View File
@@ -18,12 +18,7 @@ func (s *Server) storage(ctx context.Context) *api.Storage {
if path == "" || path == ":memory:" { if path == "" || path == ":memory:" {
return nil return nil
} }
d := api.Database{Label: label, Path: path} d := api.Database{Label: label, Path: path, Bytes: dbBytes(path)}
for _, suffix := range []string{"", "-wal", "-shm"} {
if fi, err := os.Stat(path + suffix); err == nil {
d.Bytes += fi.Size()
}
}
out.Databases = append(out.Databases, d) out.Databases = append(out.Databases, d)
out.Total += d.Bytes out.Total += d.Bytes
return &out.Databases[len(out.Databases)-1] return &out.Databases[len(out.Databases)-1]
@@ -46,6 +41,7 @@ func (s *Server) storage(ctx context.Context) *api.Storage {
if d := dbFile(statsDB, p); d != nil && s.cfg.Stats.Options().RetainQueries > 0 { if d := dbFile(statsDB, p); d != nil && s.cfg.Stats.Options().RetainQueries > 0 {
if n, err := s.cfg.Stats.Store().QueryLogSize(ctx); err == nil { if n, err := s.cfg.Stats.Store().QueryLogSize(ctx); err == nil {
d.QueryLogRows = &n d.QueryLogRows = &n
d.QueryLogRetention = s.cfg.Stats.Status().Retention.Queries
} }
} }
add(statsDB, filepath.Dir(p)) add(statsDB, filepath.Dir(p))
@@ -70,3 +66,14 @@ func (s *Server) storage(ctx context.Context) *api.Storage {
} }
return out return out
} }
// dbBytes is the size of a SQLite database: its file, -wal and -shm.
func dbBytes(path string) int64 {
var n int64
for _, suffix := range []string{"", "-wal", "-shm"} {
if fi, err := os.Stat(path + suffix); err == nil {
n += fi.Size()
}
}
return n
}
+22
View File
@@ -5,6 +5,7 @@ import (
"encoding/json" "encoding/json"
"io" "io"
"log/slog" "log/slog"
"path/filepath"
"strings" "strings"
"testing" "testing"
"time" "time"
@@ -176,3 +177,24 @@ func TestStatsExploreAPI(t *testing.T) {
t.Errorf("storage %v", st["storage"]) t.Errorf("storage %v", st["storage"])
} }
} }
// /stats/server reports the statistics database's size and the query log.
func TestStatsServerDatabase(t *testing.T) {
ss, err := stats.OpenStore(filepath.Join(t.TempDir(), "stats.db"))
if err != nil {
t.Fatal(err)
}
defer ss.Close()
b := &stats.Batch{}
b.AddQueries([]stats.QueryRow{{Time: time.Now().Unix(), Client: "10.0.0.7", View: "internal", Name: "www.example.com", Type: "A", RCode: "NOERROR"}})
if err := ss.Commit(context.Background(), b); err != nil {
t.Fatal(err)
}
opts := stats.Options{QueryLog: true, StatisticsChannel: "127.0.0.1:1", TopN: 10, RetainMinute: time.Hour, RetainHour: 24 * time.Hour, RetainDay: 24 * time.Hour, RetainQueries: 7 * 24 * time.Hour}
e := newAPIEnv(t, stats.NewCollector(ss, opts, nil, slog.New(slog.NewTextHandler(io.Discard, nil))))
sv := e.do("GET", "/stats/server", e.token, nil, 200)
db, ok := sv["database"].(map[string]any)
if !ok || db["bytes"].(float64) <= 0 || db["query_log_rows"].(float64) != 1 {
t.Errorf("database %v", sv["database"])
}
}
@@ -157,7 +157,7 @@ function storageCard(sg) {
function storagePanel(sg) { function storagePanel(sg) {
const dbRows = sg.databases.map((d) => [h("dt", null, d.label), const dbRows = sg.databases.map((d) => [h("dt", null, d.label),
h("dd", null, h("strong", null, bytes(d.bytes)), " ", h("span", { class: "mono small muted" }, d.path), h("dd", null, h("strong", null, bytes(d.bytes)), " ", h("span", { class: "mono small muted" }, d.path),
d.query_log_rows != null ? h("div", { class: "small muted" }, "the query log holds " + d.query_log_rows.toLocaleString() + " answers (stats.retention.queries)") : null)]); d.query_log_rows != null ? h("div", { class: "small muted" }, "query log: " + d.query_log_rows.toLocaleString() + (d.query_log_rows === 1 ? " answer" : " answers") + " so far, each kept for " + retentionWords(d.query_log_retention) + " (stats.retention.queries)") : null)]);
return h("div", { class: "panel", id: "storage" }, return h("div", { class: "panel", id: "storage" },
h("h2", null, "Storage"), h("h2", null, "Storage"),
h("dl", { class: "kv" }, dbRows, h("dt", null, "Together"), h("dd", null, h("strong", null, bytes(sg.total)))), h("dl", { class: "kv" }, dbRows, h("dt", null, "Together"), h("dd", null, h("strong", null, bytes(sg.total)))),
@@ -183,3 +183,11 @@ function bytes(n) {
while (n >= 1024 && i < units.length - 1) { n /= 1024; i++; } while (n >= 1024 && i < units.length - 1) { n /= 1024; i++; }
return (n >= 100 || i === 0 ? Math.round(n) : n.toFixed(1)) + " " + units[i]; return (n >= 100 || i === 0 ? Math.round(n) : n.toFixed(1)) + " " + units[i];
} }
// retentionWords: "7d" -> "7 days".
function retentionWords(r) {
const m = /^(\d+)([dhm])$/.exec(r || "");
if (!m) return r || "a while";
const word = { d: "day", h: "hour", m: "minute" }[m[2]];
return m[1] + " " + word + (m[1] === "1" ? "" : "s");
}
+34 -3
View File
@@ -145,7 +145,7 @@ export async function render(el, params, query) {
rtt.length ? listCard("Upstream response time", "how long other servers took to answer named's recursive queries", barList(rtt)) : null, rtt.length ? listCard("Upstream response time", "how long other servers took to answer named's recursive queries", barList(rtt)) : null,
b.blocklist.length ? listCard("Blocked by", "which of your rules and blocklists blocked queries", barList(b.blocklist, { label: (k) => (k === "rules" ? "your rules" : k), href: ex((k) => ({ blocked: k })) || (() => "#/blocking") })) : null)); b.blocklist.length ? listCard("Blocked by", "which of your rules and blocklists blocked queries", barList(b.blocklist, { label: (k) => (k === "rules" ? "your rules" : k), href: ex((k) => ({ blocked: k })) || (() => "#/blocking") })) : null));
parts.push(...serverSection(data.server, complete(data.gauges), rangeLabel)); parts.push(...serverSection(data.server, complete(data.gauges), rangeLabel, status));
el.classList.remove("refreshing"); el.classList.remove("refreshing");
mount(el, parts); mount(el, parts);
scheduleRefresh(el, params, query, gen); scheduleRefresh(el, params, query, gen);
@@ -223,7 +223,7 @@ const gaugeKeys = ["g:named:rss", "g:named:mem", "g:cache:mem", "g:named:cpu"];
// serverSection shows named's process, cache, clients, the host and the // serverSection shows named's process, cache, clients, the host and the
// disks now, and named's memory and CPU over time. // disks now, and named's memory and CPU over time.
function serverSection(sv, gauges, rangeLabel) { function serverSection(sv, gauges, rangeLabel, status) {
const srv = (sv && sv.server) || {}; const srv = (sv && sv.server) || {};
const named = (sv && sv.named) || {}; const named = (sv && sv.named) || {};
const host = srv.host || {}; const host = srv.host || {};
@@ -243,7 +243,8 @@ function serverSection(sv, gauges, rangeLabel) {
tc ? tile("TCP clients", tc[0], tc[1]) : null, tc ? tile("TCP clients", tc[0], tc[1]) : null,
tile("Up since", srv.boot_time ? relTime(srv.boot_time).replace(" ago", "") : "–", srv.config_time ? "named started " + fmtTime(srv.boot_time) + " · reconfigured " + relTime(srv.config_time) : ""), tile("Up since", srv.boot_time ? relTime(srv.boot_time).replace(" ago", "") : "–", srv.config_time ? "named started " + fmtTime(srv.boot_time) + " · reconfigured " + relTime(srv.config_time) : ""),
host.mem_total ? tile("Host memory", pct((host.mem_total - host.mem_available) / host.mem_total), "of " + bytes(host.mem_total) + " in use · load " + (host.load || []).map((x) => x.toFixed(2)).join(" ")) : null, host.mem_total ? tile("Host memory", pct((host.mem_total - host.mem_available) / host.mem_total), "of " + bytes(host.mem_total) + " in use · load " + (host.load || []).map((x) => x.toFixed(2)).join(" ")) : null,
...((sv && sv.disks) || []).map((d) => tile("Disk space", bytes(d.free) + " free", h("span", { title: d.path }, "of " + bytes(d.total) + ", for " + d.holds.join(", ")))))); sv && sv.database ? dbTile(sv.database, status) : null,
...((sv && sv.disks) || []).map(diskTile)));
if (sv && sv.named_error) out.push(h("div", { class: "msg info" }, "Client counts need rndc: " + sv.named_error)); if (sv && sv.named_error) out.push(h("div", { class: "msg info" }, "Client counts need rndc: " + sv.named_error));
// Gauges over time: averages, -1 where there was no sample. // Gauges over time: averages, -1 where there was no sample.
const series = (key) => ((gauges.series || []).find((x) => x.key === key) || { values: gauges.times.map(() => -1) }).values.map((v) => (v < 0 ? null : v)); const series = (key) => ((gauges.series || []).find((x) => x.key === key) || { values: gauges.times.map(() => -1) }).values.map((v) => (v < 0 ? null : v));
@@ -312,3 +313,33 @@ function disabledPanel() {
function exploreSince(since) { function exploreSince(since) {
return rangeSecs[since] > 7 * 86400 ? "7d" : since; return rangeSecs[since] > 7 * 86400 ? "7d" : since;
} }
// dbTile is the statistics database's size: what the statistics cost on
// disk (the dashboard's Storage panel has all of Jarl's databases).
function dbTile(db, status) {
const keep = ((status || {}).retention || {}).queries;
const sub = db.query_log_rows != null
? "query log: " + compact(db.query_log_rows) + (db.query_log_rows === 1 ? " answer" : " answers") + " so far, each kept for " + retentionWords(keep)
: "counters and top lists; no query log";
return h("a", { class: "card link", href: "#/", title: db.path + " · all of Jarl's databases: Dashboard → Storage" },
h("div", { class: "label" }, "Statistics database"), h("div", { class: "value" }, bytes(db.bytes)), h("div", { class: "sub" }, sub));
}
// diskTile is a disk's free space, with a bar of how full it is.
function diskTile(d) {
const u = d.total ? (d.total - d.free) / d.total : 0;
return h("div", { class: "card", title: d.path },
h("div", { class: "label" }, "Disk space"),
h("div", { class: "value" }, bytes(d.free) + " free"),
h("div", { class: "disk-bar", role: "img", "aria-label": Math.round(u * 100) + "% used" },
h("div", { class: "seg other" + (u > 0.9 ? " full" : ""), style: { width: u * 100 + "%" } })),
h("div", { class: "sub" }, h("span", { class: u > 0.9 ? "warn-text" : "" }, Math.round(u * 100) + "% of " + bytes(d.total) + " used"), ", for " + d.holds.join(", ")));
}
// retentionWords: "7d" -> "7 days".
function retentionWords(r) {
const m = /^(\d+)([dhm])$/.exec(r || "");
if (!m) return r || "a while";
const word = { d: "day", h: "hour", m: "minute" }[m[2]];
return m[1] + " " + word + (m[1] === "1" ? "" : "s");
}
+1
View File
@@ -613,3 +613,4 @@ table.recent-queries td.oneline-cell { max-width: 40ch; white-space: nowrap; ove
.swatch { display: inline-block; width: 10px; height: 10px; border-radius: 2px; vertical-align: -1px; } .swatch { display: inline-block; width: 10px; height: 10px; border-radius: 2px; vertical-align: -1px; }
.swatch.db { background: var(--accent); } .swatch.db { background: var(--accent); }
.swatch.other { background: var(--muted); opacity: 0.45; } .swatch.other { background: var(--muted); opacity: 0.45; }
.card .disk-bar { margin: 6px 0 4px; height: 8px; }
+21
View File
@@ -497,6 +497,27 @@ run "acknowledged: info, with the reason" 0 $J exposure
contains "info *external *security_open_resolver .*acknowledged by .*e2e: on purpose" contains "info *external *security_open_resolver .*acknowledged by .*e2e: on purpose"
run "take it back" 0 $J exposure unack external security_open_resolver run "take it back" 0 $J exposure unack external security_open_resolver
run "close the resolver again" 0 $J option unset allow-recursion --view external run "close the resolver again" 0 $J option unset allow-recursion --view external
# Statements told apart by their name (tls NAME, key NAME, ...): setting
# one replaces only the one of that name.
run "a tls statement for forwarding" 0 $J option set tls 'e2e-quad9 { remote-hostname "dns.quad9.net"; }' --toplevel
run "a second one, without --add" 0 $J option set tls 'e2e-cf { remote-hostname "cloudflare-dns.com"; }' --toplevel
run "both are there" 0 $J option list --toplevel
contains "e2e-quad9"
contains "e2e-cf"
run "changing one leaves the other" 0 $J option set tls 'e2e-quad9 { remote-hostname "dns11.quad9.net"; }' --toplevel
run "list" 0 $J option list --toplevel
contains "dns11.quad9.net"
contains "e2e-cf"
not_contains '"dns.quad9.net"'
run "unset needs to know which (exit 1)" 1 $J option unset tls --toplevel
contains "say which: jarl option unset tls"
run "forwarders over TLS" 0 $J option set forwarders '{ 9.9.9.9 port 853 tls e2e-quad9; 1.1.1.1 port 853 tls e2e-cf; }' --view internal
run "named.conf has them" 0 grep -q 'tls e2e-quad9' $E/etc-jarl/named.conf
run "take the forwarders out" 0 $J option unset forwarders --view internal
run "unset one by its name" 0 $J option unset tls e2e-cf --toplevel
run "and the other" 0 $J option unset tls e2e-quad9 --toplevel
run "none left" 0 $J option list --toplevel
not_contains "e2e-quad9"
run "fix: hide the version" 0 $J exposure fix server exposure_version run "fix: hide the version" 0 $J exposure fix server exposure_version
expect_dig "version.bind is hidden" "" -c CH version.bind TXT expect_dig "version.bind is hidden" "" -c CH version.bind TXT
run "mark external private" 0 $J view update external --exposure private run "mark external private" 0 $J view update external --exposure private