Release 0.6.0-rc.3

Since rc.2: the Encrypted DNS page and docs test with BIND's dig +tls / +https instead of kdig and curl's JSON API.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

CHANGELOG.md

@@ -57,6 +57,8 @@ All notable changes to Jarl are documented here. The format follows
   starts from where the records are now. The Certificates page and `jarl
   cert show` say what named uses a certificate for (DNS over TLS or HTTPS,
   on which addresses and ports).
+- docs/encrypted-dns.md: forwarding to Quad9 or Cloudflare over TLS, how to
+  check it works, and why not over HTTPS.
 - Comparing one name across views: `jarl zone diff adbraun.lol nas [A]`,
   and a **Compare** button on each record.
 
@@ -66,13 +68,22 @@ All notable changes to Jarl are documented here. The format follows
   an explore page for it: answers over time, the top clients, names,
   views, types, results and blocklists among them (each a link that
   narrows it further), and the latest answers. "NXDOMAIN in external" →
-  the clients asking → everything one of them asks. `GET /stats/explore`.
+  the clients asking → everything one of them asks. A filter bar types
+  filters in, several values each (any of them), `!value` to leave one
+  out and `*.example.com` for a domain and below; **hide** on a list entry
+  leaves it out. **Explore the answers** opens the page from the
+  Statistics page. `GET /stats/explore`.
 - **Storage** on the dashboard: the size of Jarl's databases (and how many
   answers the query log holds), and per disk Jarl uses a bar of how full it
-  is and how much of it the databases take.
+  is and how much of it the databases take. The Statistics page shows the
+  statistics database's size too, and its disk tiles get the same bar.
 
 ### Changed
 
+- `jarl option set tls NAME …` (and `key`, `remote-servers`, `http`,
+  `dnssec-policy`, `server` …) replaces only the statement with that name:
+  setting `tls cloudflare` used to replace `tls quad9`. `jarl option unset
+  tls cloudflare` removes one by its name.
 - **Compare views** and `jarl zone diff` are rebuilt around the view
   you're on (docs/views.md#comparing-views): a summary per view, then
   what is answered differently (a view's own records in linked zones,
@@ -84,12 +95,19 @@ All notable changes to Jarl are documented here. The format follows
 
 ### Fixed
 
+- Encrypted DNS: the client hints tested with kdig (Knot's, often not
+  installed) and with curl's JSON API, which named's DoH doesn't speak;
+  they use BIND's `dig +tls` / `dig +https` now.
 - Server options → Added by Jarl: long statements didn't wrap, and the
   browser console reported a Content Security Policy violation (an inline
   style the page's policy blocks).
 
-### Upgrading
+### Upgrading from 0.5
 
+- Tested: 0.5.2 packages with views, a linked zone, a reverse zone,
+  blocking, a `tls` statement and statistics, upgraded in place in Debian
+  13 (BIND 9.20.29): everything kept and answered as before, the new
+  features worked on the upgraded data, and a 0.5.2 backup restored.
 - The database gets two new versions (12 and 13), applied when jarld
   starts; a 0.5 jarld can't open it afterwards, so take a backup first (`sudo
   jarld backup`). Nothing changes until you turn `auto_ptr` on for a

README.md

@@ -202,7 +202,12 @@ review step.
 
 ## Status
 
-Version 0.5.2 is the current release. It adds Prometheus metrics,
+Version 0.5.2 is the current release, and 0.6.0 is in testing
+(0.6.0-rc.3): it makes reverse zones easy (suggested from your records,
+created in the right view and filled, PTR records kept in step), rebuilds
+comparing a zone's views, lets you explore the statistics down to each
+client and name (a query log), shows the databases' size, and gives
+encrypted DNS several host names and views per listen address. 0.5 added Prometheus metrics,
 address ranges in ACLs and other address match lists, safer changeset
 handling, internet exposure checks with response rate limiting, TLS
 certificates Jarl obtains and renews itself, linked zones for split DNS,

VERSION

@@ -1 +1 @@
-0.5.2
+0.6.0-rc.3

cmd/jarl/config_cmds.go

@@ -586,6 +586,12 @@ func optionCmd() *cobra.Command {
 		Short: "Set an option (replaces an existing option of the same name)",
 		Long: `Set an option; VALUE is named.conf syntax without the trailing semicolon.
 
+It replaces the option of that name. Statements told apart by the name
+after the keyword (tls NAME, key NAME, remote-servers NAME, http NAME,
+dnssec-policy NAME, server ADDRESS, ...) replace only the one with that
+name: setting tls cloudflare leaves tls quad9 alone. --add adds another
+instance of any other option that may occur several times.
+
   jarl option set recursion no
   jarl option set allow-recursion '{ homelab; localhost; }'
   jarl option set forwarders '{ 1.1.1.1; 9.9.9.9; }'
@@ -610,7 +616,11 @@ func optionCmd() *cobra.Command {
 			if cmd.Flags().Changed("comment") {
 				body["comment"] = comment
 			}
+			label := optionLabel(strings.ToLower(args[0]), value)
 			for _, o := range opts {
+				if label != "" && optionLabel(o.Name, o.Value) != label {
+					continue // another tls (key, server, ...) statement
+				}
 				if o.Name == strings.ToLower(args[0]) && !multiple {
 					resp, err := c.Mutate(ctx(), "PATCH", fmt.Sprintf("/options/%d", o.ID), nil, body)
 					if err != nil {
@@ -631,7 +641,7 @@ func optionCmd() *cobra.Command {
 	set.Flags().StringVar(&comment, "comment", "", "comment")
 	set.Flags().BoolVar(&multiple, "add", false, "add another instance instead of replacing (for options that may occur several times)")
 	unset := &cobra.Command{
-		Use:   "unset NAME | unset --id ID",
+		Use:   "unset NAME [LABEL] | unset --id ID",
 		Short: "Remove an option (BIND's default applies again)",
 		RunE: func(cmd *cobra.Command, args []string) error {
 			c, err := newClient()
@@ -645,8 +655,8 @@ func optionCmd() *cobra.Command {
 				}
 				return report(resp)
 			}
-			if len(args) != 1 {
-				return fmt.Errorf("give the option name or --id")
+			if len(args) < 1 || len(args) > 2 {
+				return fmt.Errorf("give the option name (and the name after it, for tls NAME and the like) or --id")
 			}
 			sc, id, err := scope()
 			if err != nil {
@@ -658,15 +668,24 @@ func optionCmd() *cobra.Command {
 			}
 			var matches []api.Option
 			for _, o := range opts {
-				if o.Name == strings.ToLower(args[0]) {
+				if o.Name == strings.ToLower(args[0]) && (len(args) == 1 || strings.EqualFold(optionLabel(o.Name, o.Value), strings.Trim(args[1], `"`))) {
 					matches = append(matches, o)
 				}
 			}
 			switch len(matches) {
 			case 0:
-				return fmt.Errorf("option %s is not set here", args[0])
+				return fmt.Errorf("option %s is not set here", strings.Join(args, " "))
 			case 1:
 			default:
+				var labels []string
+				for _, o := range matches {
+					if l := optionLabel(o.Name, o.Value); l != "" {
+						labels = append(labels, l)
+					}
+				}
+				if len(labels) == len(matches) {
+					return fmt.Errorf("option %s is set %d times (%s); say which: jarl option unset %s %s", args[0], len(matches), strings.Join(labels, ", "), args[0], labels[0])
+				}
 				return fmt.Errorf("option %s is set %d times; remove one with --id", args[0], len(matches))
 			}
 			resp, err := c.Mutate(ctx(), "DELETE", fmt.Sprintf("/options/%d", matches[0].ID), nil, nil)
@@ -847,3 +866,30 @@ func settingsCmd() *cobra.Command {
 	cmd.AddCommand(show, set)
 	return cmd
 }
+
+// labelled are the statements told apart by the name after the keyword:
+// tls quad9 { ... }, key xfer { ... }, server 192.0.2.1 { ... }.
+var labelled = map[string]bool{"tls": true, "http": true, "key": true, "remote-servers": true, "primaries": true, "masters": true,
+	"parental-agents": true, "dnssec-policy": true, "key-store": true, "server": true, "view": true, "acl": true, "zone": true}
+
+// optionLabel returns the name a labelled statement's value starts with
+// ("quad9" for tls quad9 { ... }), lowercase; "" for other statements.
+func optionLabel(name, value string) string {
+	if !labelled[name] {
+		return ""
+	}
+	v := strings.TrimSpace(value)
+	if v == "" || strings.HasPrefix(v, "{") {
+		return ""
+	}
+	if strings.HasPrefix(v, `"`) {
+		if end := strings.Index(v[1:], `"`); end >= 0 {
+			return strings.ToLower(v[1 : end+1])
+		}
+	}
+	end := strings.IndexAny(v, " \t\n{;")
+	if end < 0 {
+		end = len(v)
+	}
+	return strings.ToLower(v[:end])
+}

cmd/jarl/main_test.go

@@ -232,3 +232,20 @@ func TestOutputFields(t *testing.T) {
 		}
 	}
 }
+
+func TestOptionLabel(t *testing.T) {
+	for _, c := range []struct{ name, value, want string }{
+		{"tls", `quad9 { remote-hostname "dns.quad9.net"; }`, "quad9"},
+		{"tls", `Cloudflare{ remote-hostname "cloudflare-dns.com"; }`, "cloudflare"},
+		{"key", `"xfer key" { algorithm hmac-sha256; secret "x"; }`, "xfer key"},
+		{"server", "192.0.2.1 { keys { xfer; }; }", "192.0.2.1"},
+		{"remote-servers", "secondaries { 192.0.2.53; }", "secondaries"},
+		{"forwarders", "{ 9.9.9.9; }", ""},
+		{"logging", "{ channel x { stderr; }; }", ""},
+		{"tls", "", ""},
+	} {
+		if got := optionLabel(c.name, c.value); got != c.want {
+			t.Errorf("optionLabel(%q, %q) = %q, want %q", c.name, c.value, got, c.want)
+		}
+	}
+}

docs/api.md

@@ -565,8 +565,8 @@ Needs `stats:read`. When statistics are disabled, every endpoint but
 | GET | `/stats/timeline` | `since`, `step` (auto, 1m, 5m, 15m, 1h, 6h, 1d), `key` (repeatable or comma separated; `rcode:*` = every counter of a group; default `queries`) | `{step, times: [unix...], series: [{key, values}]}` |
 | GET | `/stats/top` | `kind` (name, client, nxdomain, servfail, refused, blocked, blocked_client), `since`, `limit` (default 20) | `[{key, count}]`; client, refused and blocked_client entries add `name`, the host name of Jarl's record for the address (the summary's client list too) |
 | GET | `/stats/heatmap` | `since` (default 28d), `key` (default queries) | `{key, from, avg[7][24], samples[7][24]}`, Monday first, local time, `-1` = no data |
-| GET | `/stats/explore` | `?client=&name=&domain=&view=&type=&rcode=&blocked=yes\|no\|LIST&since=24h&views=` | The query log for a filter ([stats.md](stats.md#exploring)): `total`, `blocked`, `series` (per `step` seconds: `t`, `count`, `blocked`), `by` (top 15 of each dimension the filter leaves open: client, name, view, type, rcode, blocked; clients with `name`), `recent` (latest 100: ts, client, client_name, view, name, type, rcode, blocked), `oldest`, `retention`, `client_name`. 404 `query_log_off` without a query log |
-| GET | `/stats/server` | | `{server: {rss, mem_in_use, cpu_percent, threads, pid, cache_rrsets, cache_mem, sockets_udp, sockets_tcp, recursion_highwater, tcp_highwater, version, boot_time, config_time, host: {mem_total, mem_available, load, cpus, uptime}}, named: {recursive_clients, tcp_clients, zones, cpus, worker_threads}, disks: [{path, holds, total, free}]}`; gauges over time: `/stats/timeline?key=g:named:rss` (averages, -1 without a sample) |
+| GET | `/stats/explore` | `?client=&name=&domain=&view=&type=&rcode=&blocked=yes\|no\|LIST&since=24h&views=`; each takes several values, comma separated (any of them), `!value` to leave one out, and names `*.example.com` for it and below | The query log for a filter ([stats.md](stats.md#exploring)): `total`, `blocked`, `series` (per `step` seconds: `t`, `count`, `blocked`), `by` (top 15 of each dimension the filter leaves open: client, name, view, type, rcode, blocked; clients with `name`), `recent` (latest 100: ts, client, client_name, view, name, type, rcode, blocked), `oldest`, `retention`, `client_name`. 404 `query_log_off` without a query log |
+| GET | `/stats/server` | | `{server: {rss, mem_in_use, cpu_percent, threads, pid, cache_rrsets, cache_mem, sockets_udp, sockets_tcp, recursion_highwater, tcp_highwater, version, boot_time, config_time, host: {mem_total, mem_available, load, cpus, uptime}}, named: {recursive_clients, tcp_clients, zones, cpus, worker_threads}, disks: [{path, holds, total, free}], database: {path, bytes, query_log_rows}}`; gauges over time: `/stats/timeline?key=g:named:rss` (averages, -1 without a sample) |
 | GET | `/stats/breakdown` | `group` (qtype, rcode, view, zone, answer, transport, rtt, dnssec, cache, ratelimit, blocklist), `since` | `[{key, count}]` |
 
 ```sh

docs/cli.md

@@ -482,12 +482,22 @@ jarl option set allow-transfer 'key xfer; 10.0.10.53' --zone adbraun.lol
 jarl option set allow-query '10.0.20.1-10.0.20.99; localhost' --view internal
 jarl option set response-policy '{ zone "rpz.local"; }' --view internal
 jarl option set logging '{ channel q { file "/var/log/named/query.log" versions 3 size 20m; }; category queries { q; }; }' --toplevel
+jarl option set tls 'quad9 { remote-hostname "dns.quad9.net"; ca-file "/etc/ssl/certs/ca-certificates.crt"; }' --toplevel
+jarl option set tls 'cloudflare { remote-hostname "cloudflare-dns.com"; }' --toplevel   # tls quad9 stays
+jarl option unset tls cloudflare --toplevel          # statements with a name: say which
 jarl option unset forwarders
 jarl option describe                          # every option allowed at this scope
 jarl option describe allow-query-cache        # grammar and ARM description
 jarl option generated                         # what Jarl adds to named.conf by itself, and why
 ```
 
+`option set` replaces the option of that name. Statements told apart by
+the name after the keyword (`tls NAME`, `key NAME`, `remote-servers NAME`,
+`http NAME`, `dnssec-policy NAME`, `server ADDRESS`, ...) replace only the
+one with that name; `--add` adds another instance of any other option that
+may occur several times. Forwarding to Quad9 or Cloudflare over TLS:
+[encrypted-dns.md](encrypted-dns.md#forwarding-over-tls).
+
 Options that end in a list (`allow-*`, `forwarders`, `listen-on`, ...) may
 be given without the braces: elements are separated by `;`, `,` or spaces,
 and `key NAME` and `port N` stay with their element. Options are always

docs/encrypted-dns.md

@@ -14,6 +14,10 @@ host name's address records in your private views.
 In the web UI: **Encrypted DNS** (administrators). In the CLI: `jarl
 encrypted-dns`. In the API: `/api/v1/encrypted-dns`.
 
+That is encryption between your clients and this server. For encryption
+between this server and the resolvers it forwards to (Quad9, Cloudflare),
+see [Forwarding over TLS](#forwarding-over-tls) below.
+
 ## Setting it up
 
 ```sh
@@ -107,14 +111,21 @@ For DoH clients:   https://dns.adbraun.lol/dns-query
 | Chrome, Edge | Settings → Privacy and security → Security → Use secure DNS → custom: the DoH URL |
 | Windows 11 | Settings → Network → your connection → DNS server assignment → Edit: the server's address, DNS over HTTPS on (manual template), the DoH URL |
 
-Test from a machine on your network:
+Test from a machine on your network, with BIND's own `dig` (9.18 or
+later; `bind9-dnsutils` on Debian):
 
 ```sh
-kdig +tls-ca +tls-host=dns.adbraun.lol @10.0.10.5 example.com
-dig +https @10.0.10.5 example.com
-curl -s -H 'accept: application/dns-json' 'https://dns.adbraun.lol/dns-query?name=example.com'
+dig +tls +tls-ca +tls-hostname=dns.adbraun.lol @10.0.10.5 example.com     # DoT, port 853
+dig +https +tls-ca +tls-hostname=dns.adbraun.lol @10.0.10.5 example.com   # DoH, port 443, /dns-query
 ```
 
+`+tls-ca` checks the certificate against the system's CAs and
+`+tls-hostname` against the host name, as clients do; add `-p PORT` for
+another port. Without `+tls-ca` the query is still encrypted, but the
+certificate isn't checked. named's DoH speaks the standard wire format
+(RFC 8484), not the JSON API some public resolvers also offer, so
+`curl ... accept: application/dns-json` doesn't work against it.
+
 **Android away from home.** Private DNS in hostname mode applies on every
 network. Away from home, your server can't be reached, and Android then
 resolves nothing at all ("Private DNS server cannot be accessed"). Use a
@@ -151,6 +162,62 @@ What it can say:
 | not from a CA this server trusts (warning) | a staging or test CA: clients reject it unless they trust it |
 | REFUSED (warning) | it works, but this server's own address gets a view that doesn't answer for the host name |
 
+## Forwarding over TLS
+
+When a view forwards its recursive queries (to Quad9, Cloudflare, your
+ISP's resolver), named can send them over TLS, so your ISP doesn't see the
+names you look up: only connections to the forwarder on port 853. It takes
+two options, a `tls` statement for each provider (the name its certificate
+must have) and the view's `forwarders` using it. Stage them in one
+changeset, so one revert undoes them:
+
+```sh
+CS=$(jarl changeset create "internal: forward to Quad9 over TLS" -q)
+jarl option set tls 'quad9 { remote-hostname "dns.quad9.net"; ca-file "/etc/ssl/certs/ca-certificates.crt"; }' --toplevel -C $CS
+jarl option set forwarders '{ 9.9.9.9 port 853 tls quad9; 149.112.112.112 port 853 tls quad9; 2620:fe::fe port 853 tls quad9; 2620:fe::9 port 853 tls quad9; }' --view internal -C $CS
+jarl option set forward only --view internal -C $CS
+jarl changeset diff $CS        # what named.conf gets
+jarl changeset apply $CS
+# undo: jarl changeset apply $(jarl changeset revert $CS -q)
+```
+
+* **`remote-hostname` and `ca-file`** make named check the forwarder's
+  certificate against that name (Debian's CA bundle at that path). Without
+  them the connection is encrypted but anyone in between could answer.
+* **`forward only`** stops named falling back to plain DNS (to the root
+  servers) when the forwarders don't answer: lookups then fail instead
+  (SERVFAIL), which you notice, rather than quietly going unencrypted.
+* **Several providers**: a `tls` statement each, and their addresses in one
+  `forwarders` list. Setting `tls cloudflare` doesn't touch `tls quad9`:
+  `jarl option set` replaces only the statement of that name.
+
+  ```sh
+  jarl option set tls 'cloudflare { remote-hostname "cloudflare-dns.com"; ca-file "/etc/ssl/certs/ca-certificates.crt"; }' --toplevel
+  jarl option set forwarders '{ 9.9.9.9 port 853 tls quad9; 149.112.112.112 port 853 tls quad9; 1.1.1.1 port 853 tls cloudflare; 1.0.0.1 port 853 tls cloudflare; 2606:4700:4700::1111 port 853 tls cloudflare; 2606:4700:4700::1001 port 853 tls cloudflare; }' --view internal
+  ```
+
+  named sends each query to whichever forwarder has answered fastest, not
+  in order, and the providers filter differently (Quad9 blocks known
+  malicious domains, Cloudflare's 1.1.1.1 doesn't; 1.1.1.2 does, with
+  `remote-hostname "security.cloudflare-dns.com"`).
+* **DNS over HTTPS** can't be used for forwarding: BIND serves DoH to
+  clients but forwards only over plain DNS or TLS. For a DoH upstream, run
+  a local proxy (dnscrypt-proxy, cloudflared) and forward to it on
+  `127.0.0.1`.
+* Providers write their DoT address as `tls://dns.quad9.net`: the name is
+  `remote-hostname`, and the addresses it resolves to go in `forwarders`
+  (named needs addresses there, not names).
+
+**Checking it works**: Quad9 says how a query reached it.
+
+```sh
+dig +short TXT proto.on.quad9.net @YOUR-SERVER     # "dot": over TLS
+```
+
+`do53-udp` means plain DNS. `ss -tnp | grep :853` shows named's connections
+only briefly: it closes them when idle. `tcpdump -ni any port 853` shows
+the traffic as it happens.
+
 ## What Jarl checks
 
 * A `tls` statement whose files don't exist is an error

docs/stats.md

@@ -213,7 +213,18 @@ which shows, for what you clicked:
 * the latest 100 answers, every cell a link.
 
 So "external, NXDOMAIN" leads to the clients asking, a client to everything
-it asks, one of its names to who else asks for it. Filters combine (this
+it asks, one of its names to who else asks for it. The page also opens
+from **Explore the answers** at the top of the Statistics page, and has a
+filter bar to type filters in: clients, names, views, record types,
+results, blocked or not; it suggests values from what is shown.
+
+Each kind of filter takes several values, comma separated, and matches
+any of them: `10.0.0.5, 10.0.0.7` is either client. `!value` leaves one
+out (`!10.0.0.250` hides a monitoring probe), and `*.example.com` is the
+name and every name below it, so `!*.apple.com` drops a chatty device's
+noise. Kinds combine: this client, in this view, NXDOMAIN, not that name.
+In the lists, clicking a value adds it and **hide** leaves it out; every
+value has a chip with a ✕ to take it out again. Filters combine (this
 client, in this view, blocked only) and each has a ✕ to take it out. A name
 can be widened to the names below it (`domain=`). The Statistics page's
 view switches apply here too.
@@ -221,8 +232,10 @@ view switches apply here too.
 `anonymize_clients` and `exclude_clients` apply to the query log as to
 everything else. Its size: each answer takes about 150 to 200 bytes with
 its indexes, so 50,000 answers a day for 7 days is about 60 MB, 200,000 a
-day about 250 MB. The dashboard's Storage panel shows the databases' size
-and how many answers the query log holds.
+day about 250 MB. The Statistics page's server section shows the
+statistics database's size and how many answers the query log holds, next
+to the disks (with a bar of how full each is); the dashboard's Storage
+panel has all of Jarl's databases.
 
 ## Views
 

internal/api/types.go

@@ -294,8 +294,10 @@ type Database struct {
 	Label string `json:"label"`
 	Path  string `json:"path"`
 	Bytes int64  `json:"bytes"`
-	// Rows is how many answers the query log holds (statistics only).
-	QueryLogRows *int64 `json:"query_log_rows,omitempty"`
+	// Rows is how many answers the query log holds (statistics only),
+	// and QueryLogRetention how long it keeps them ("7d").
+	QueryLogRows      *int64 `json:"query_log_rows,omitempty"`
+	QueryLogRetention string `json:"query_log_retention,omitempty"`
 }
 
 // Disk is a file system Jarl uses: what of Jarl's it holds, its size,

internal/server/h_stats.go

@@ -283,6 +283,16 @@ func (s *Server) statsServer(w http.ResponseWriter, r *http.Request, p *auth.Pri
 		labels = append(labels, "statistics database")
 	}
 	out["disks"] = stats.DiskUsage(labels, dirs)
+	// The statistics database: what the statistics cost on disk.
+	if db := s.cfg.Stats.Store().Path(); db != "" && db != ":memory:" {
+		info := map[string]any{"path": db, "bytes": dbBytes(db)}
+		if keep := s.cfg.Stats.Options().RetainQueries; keep > 0 && s.cfg.Stats.Options().QueryLog {
+			if n, err := s.cfg.Stats.Store().QueryLogSize(r.Context()); err == nil {
+				info["query_log_rows"] = n
+			}
+		}
+		out["database"] = info
+	}
 	writeJSON(w, http.StatusOK, out)
 	return nil
 }

internal/server/h_storage.go

@@ -18,12 +18,7 @@ func (s *Server) storage(ctx context.Context) *api.Storage {
 		if path == "" || path == ":memory:" {
 			return nil
 		}
-		d := api.Database{Label: label, Path: path}
-		for _, suffix := range []string{"", "-wal", "-shm"} {
-			if fi, err := os.Stat(path + suffix); err == nil {
-				d.Bytes += fi.Size()
-			}
-		}
+		d := api.Database{Label: label, Path: path, Bytes: dbBytes(path)}
 		out.Databases = append(out.Databases, d)
 		out.Total += d.Bytes
 		return &out.Databases[len(out.Databases)-1]
@@ -46,6 +41,7 @@ func (s *Server) storage(ctx context.Context) *api.Storage {
 			if d := dbFile(statsDB, p); d != nil && s.cfg.Stats.Options().RetainQueries > 0 {
 				if n, err := s.cfg.Stats.Store().QueryLogSize(ctx); err == nil {
 					d.QueryLogRows = &n
+					d.QueryLogRetention = s.cfg.Stats.Status().Retention.Queries
 				}
 			}
 			add(statsDB, filepath.Dir(p))
@@ -70,3 +66,14 @@ func (s *Server) storage(ctx context.Context) *api.Storage {
 	}
 	return out
 }
+
+// dbBytes is the size of a SQLite database: its file, -wal and -shm.
+func dbBytes(path string) int64 {
+	var n int64
+	for _, suffix := range []string{"", "-wal", "-shm"} {
+		if fi, err := os.Stat(path + suffix); err == nil {
+			n += fi.Size()
+		}
+	}
+	return n
+}

internal/server/stats_test.go

@@ -5,6 +5,7 @@ import (
 	"encoding/json"
 	"io"
 	"log/slog"
+	"path/filepath"
 	"strings"
 	"testing"
 	"time"
@@ -176,3 +177,24 @@ func TestStatsExploreAPI(t *testing.T) {
 		t.Errorf("storage %v", st["storage"])
 	}
 }
+
+// /stats/server reports the statistics database's size and the query log.
+func TestStatsServerDatabase(t *testing.T) {
+	ss, err := stats.OpenStore(filepath.Join(t.TempDir(), "stats.db"))
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer ss.Close()
+	b := &stats.Batch{}
+	b.AddQueries([]stats.QueryRow{{Time: time.Now().Unix(), Client: "10.0.0.7", View: "internal", Name: "www.example.com", Type: "A", RCode: "NOERROR"}})
+	if err := ss.Commit(context.Background(), b); err != nil {
+		t.Fatal(err)
+	}
+	opts := stats.Options{QueryLog: true, StatisticsChannel: "127.0.0.1:1", TopN: 10, RetainMinute: time.Hour, RetainHour: 24 * time.Hour, RetainDay: 24 * time.Hour, RetainQueries: 7 * 24 * time.Hour}
+	e := newAPIEnv(t, stats.NewCollector(ss, opts, nil, slog.New(slog.NewTextHandler(io.Discard, nil))))
+	sv := e.do("GET", "/stats/server", e.token, nil, 200)
+	db, ok := sv["database"].(map[string]any)
+	if !ok || db["bytes"].(float64) <= 0 || db["query_log_rows"].(float64) != 1 {
+		t.Errorf("database %v", sv["database"])
+	}
+}

internal/stats/explore.go

@@ -11,7 +11,10 @@ import (
 // last days (stats.query_retention), filtered by any mix of client, name,
 // view, type, result and blocking, broken down by each of them.
 
-// ExploreFilter narrows the query log. Empty fields don't filter.
+// ExploreFilter narrows the query log. Empty fields don't filter. Each
+// field may hold several values, comma separated: any of them; a value
+// written !VALUE excludes it. A name *.example.com is it and the names
+// below it.
 type ExploreFilter struct {
 	Client string // a client key, as the top lists show it
 	Name   string // exactly this name
@@ -58,33 +61,96 @@ type ExplorePoint struct {
 // ErrNoQueryLog is returned when the query log is off.
 var ErrNoQueryLog = fmt.Errorf("the query log is off: it needs stats.query_log and a stats.query_retention above 0 in /etc/jarl/jarl.yaml (docs/stats.md#exploring)")
 
-// where builds the SQL condition and arguments for a filter.
+// values splits a filter field into the values it must be one of and the
+// ones (written !VALUE) it must not be: "10.0.0.5, !10.0.0.9".
+func values(field string, norm func(string) string) (in, out []string) {
+	for _, v := range strings.Split(field, ",") {
+		v = strings.TrimSpace(v)
+		not := strings.HasPrefix(v, "!")
+		v = norm(strings.TrimSpace(strings.TrimPrefix(v, "!")))
+		switch {
+		case v == "":
+		case not:
+			out = append(out, v)
+		default:
+			in = append(in, v)
+		}
+	}
+	return in, out
+}
+
+// nameValues are the name filter's values: Name's, and Domain's as
+// "*.DOMAIN" (the name and the names below it).
+func (f ExploreFilter) nameValues() (in, out []string) {
+	norm := func(v string) string { return strings.ToLower(strings.TrimSuffix(v, ".")) }
+	in, out = values(f.Name, norm)
+	din, dout := values(f.Domain, norm)
+	for _, d := range din {
+		in = append(in, "*."+strings.TrimPrefix(d, "*."))
+	}
+	for _, d := range dout {
+		out = append(out, "*."+strings.TrimPrefix(d, "*."))
+	}
+	return in, out
+}
+
+// fieldValues returns a dimension's filter values.
+func (f ExploreFilter) fieldValues(dim string) (in, out []string) {
+	same := func(v string) string { return v }
+	upper := strings.ToUpper
+	switch dim {
+	case "client":
+		return values(f.Client, same)
+	case "name":
+		return f.nameValues()
+	case "view":
+		return values(f.View, same)
+	case "type":
+		return values(f.Type, upper)
+	case "rcode":
+		return values(f.RCode, upper)
+	case "blocked":
+		return values(f.Blocked, same)
+	}
+	return nil, nil
+}
+
+// cond is the SQL condition for one value of a dimension.
+func cond(dim, v string) (string, []any) {
+	switch {
+	case dim == "name" && strings.HasPrefix(v, "*."):
+		d := strings.TrimPrefix(v, "*.")
+		return "(name = ? OR (length(name) > length(?) AND substr(name, length(name) - length(?)) = ?))", []any{d, d, d, "." + d}
+	case dim == "blocked" && v == "yes":
+		return "blocked != ''", nil
+	case dim == "blocked" && v == "no":
+		return "blocked = ''", nil
+	}
+	return dim + " = ?", []any{v}
+}
+
+// where builds the SQL condition and arguments for a filter: within a
+// dimension any of its values, and none of its !values; the dimensions
+// together.
 func (q *Querier) where(f ExploreFilter, from int64) (string, []any) {
 	conds := []string{"ts >= ?"}
 	args := []any{from}
-	eq := func(col, v string) {
-		if v != "" {
-			conds = append(conds, col+" = ?")
-			args = append(args, v)
+	for _, dim := range Dimensions {
+		in, out := f.fieldValues(dim)
+		if len(in) > 0 {
+			var or []string
+			for _, v := range in {
+				c, a := cond(dim, v)
+				or = append(or, c)
+				args = append(args, a...)
+			}
+			conds = append(conds, "("+strings.Join(or, " OR ")+")")
+		}
+		for _, v := range out {
+			c, a := cond(dim, v)
+			conds = append(conds, "NOT "+c)
+			args = append(args, a...)
 		}
-	}
-	eq("client", f.Client)
-	eq("name", strings.ToLower(strings.TrimSuffix(f.Name, ".")))
-	eq("view", f.View)
-	eq("type", strings.ToUpper(f.Type))
-	eq("rcode", strings.ToUpper(f.RCode))
-	if d := strings.ToLower(strings.TrimSuffix(f.Domain, ".")); d != "" {
-		conds = append(conds, "(name = ? OR (length(name) > length(?) AND substr(name, length(name) - length(?)) = ?))")
-		args = append(args, d, d, d, "."+d)
-	}
-	switch f.Blocked {
-	case "":
-	case "yes":
-		conds = append(conds, "blocked != ''")
-	case "no":
-		conds = append(conds, "blocked = ''")
-	default:
-		eq("blocked", f.Blocked)
 	}
 	if len(q.Views) > 0 {
 		marks := strings.TrimSuffix(strings.Repeat("?,", len(q.Views)), ",")
@@ -143,7 +209,13 @@ func (q *Querier) Explore(ctx context.Context, f ExploreFilter, topN, recent int
 		out.Series = append(out.Series, p)
 	}
 
-	open := map[string]bool{"client": f.Client == "", "name": f.Name == "", "view": f.View == "", "type": f.Type == "", "rcode": f.RCode == "", "blocked": f.Blocked == "" || f.Blocked == "yes"}
+	// A dimension is broken down unless the filter pins it to one value
+	// (several: how they compare; !values leave the rest to see).
+	open := map[string]bool{}
+	for _, dim := range Dimensions {
+		in, _ := f.fieldValues(dim)
+		open[dim] = len(in) != 1 || (dim == "blocked" && in[0] == "yes") || (dim == "name" && strings.HasPrefix(in[0], "*."))
+	}
 	for _, dim := range Dimensions {
 		if !open[dim] {
 			continue

internal/stats/explore_test.go

@@ -84,6 +84,39 @@ func TestExplore(t *testing.T) {
 		t.Errorf("internal NXDOMAIN: %d", ex.Total)
 	}
 
+	// Several values: any of them; !values: none of them.
+	total := func(f ExploreFilter) int64 {
+		t.Helper()
+		ex, err := q.Explore(ctx, f, 10, 50)
+		if err != nil {
+			t.Fatal(err)
+		}
+		return ex.Total
+	}
+	for _, c := range []struct {
+		f    ExploreFilter
+		want int64
+	}{
+		{ExploreFilter{Client: "10.0.0.5, 198.51.100.7"}, 6},
+		{ExploreFilter{Client: "!10.0.0.5"}, 2},
+		{ExploreFilter{RCode: "NXDOMAIN,noerror"}, 6},
+		{ExploreFilter{Name: "*.adbraun.lol"}, 5},
+		{ExploreFilter{Name: "!*.adbraun.lol"}, 1},
+		{ExploreFilter{Name: "www.adbraun.lol,nas.adbraun.lol"}, 2},
+		{ExploreFilter{Name: "*.adbraun.lol, !typo.adbraun.lol", Client: "!198.51.100.7"}, 2},
+		{ExploreFilter{Blocked: "!hagezi"}, 5},
+		{ExploreFilter{Domain: "adbraun.lol", Name: "!nope.adbraun.lol"}, 3},
+	} {
+		if got := total(c.f); got != c.want {
+			t.Errorf("%+v: %d answers, want %d", c.f, got, c.want)
+		}
+	}
+	// Two clients: their breakdown shows them side by side.
+	ex, _ = q.Explore(ctx, ExploreFilter{Client: "10.0.0.5,198.51.100.7"}, 10, 50)
+	if len(ex.By["client"]) != 2 {
+		t.Errorf("two clients' breakdown %+v", ex.By["client"])
+	}
+
 	// Pruning: past the retention they go.
 	clk.t = clk.t.Add(8 * 24 * time.Hour)
 	c.prune(ctx)

internal/web/static/assets/charts.js

@@ -329,7 +329,9 @@ export function heatmapTable(hm, unit) {
 // barList shows ranked entries as labelled horizontal bars in one hue
 // (one series: no legend, the title names it). Values are always printed,
 // so the list is its own table view.
-export function barList(entries, { total, label = (k) => k, empty = "No data yet.", href } = {}) {
+// extra(key), when given, adds something after an entry's label (e.g. a
+// link that excludes it).
+export function barList(entries, { total, label = (k) => k, empty = "No data yet.", href, extra } = {}) {
   if (!entries.length) return h("div", { class: "empty" }, empty);
   // Entries may be in a natural order (e.g. latency buckets), not by size.
   const peak = Math.max(1, ...entries.map((e) => e.count));
@@ -343,7 +345,7 @@ export function barList(entries, { total, label = (k) => k, empty = "No data yet
     return h("li", { title: (e.name ? e.name + " (" + name + ")" : name) + ": " + e.count.toLocaleString() + " (" + share.toFixed(1) + "%)" },
       h("div", { class: "bar-text" },
         link ? h("a", { href: link, class: "bar-label" }, shown) : h("span", { class: "bar-label" }, shown),
-        h("span", { class: "bar-value" }, compact(e.count), h("span", { class: "muted" }, " · " + share.toFixed(share < 10 ? 1 : 0) + "%"))),
+        h("span", { class: "bar-value" }, extra ? [extra(e.key), " "] : null, compact(e.count), h("span", { class: "muted" }, " · " + share.toFixed(share < 10 ? 1 : 0) + "%"))),
       h("div", { class: "bar-track" }, h("div", { class: "bar-fill", style: { width: Math.max(0.5, (e.count / peak) * 100) + "%" } })));
   }));
 }

internal/web/static/assets/pages/dashboard.js

@@ -157,7 +157,7 @@ function storageCard(sg) {
 function storagePanel(sg) {
   const dbRows = sg.databases.map((d) => [h("dt", null, d.label),
     h("dd", null, h("strong", null, bytes(d.bytes)), " ", h("span", { class: "mono small muted" }, d.path),
-      d.query_log_rows != null ? h("div", { class: "small muted" }, "the query log holds " + d.query_log_rows.toLocaleString() + " answers (stats.retention.queries)") : null)]);
+      d.query_log_rows != null ? h("div", { class: "small muted" }, "query log: " + d.query_log_rows.toLocaleString() + (d.query_log_rows === 1 ? " answer" : " answers") + " so far, each kept for " + retentionWords(d.query_log_retention) + " (stats.retention.queries)") : null)]);
   return h("div", { class: "panel", id: "storage" },
     h("h2", null, "Storage"),
     h("dl", { class: "kv" }, dbRows, h("dt", null, "Together"), h("dd", null, h("strong", null, bytes(sg.total)))),
@@ -183,3 +183,11 @@ function bytes(n) {
   while (n >= 1024 && i < units.length - 1) { n /= 1024; i++; }
   return (n >= 100 || i === 0 ? Math.round(n) : n.toFixed(1)) + " " + units[i];
 }
+
+// retentionWords: "7d" -> "7 days".
+function retentionWords(r) {
+  const m = /^(\d+)([dhm])$/.exec(r || "");
+  if (!m) return r || "a while";
+  const word = { d: "day", h: "hour", m: "minute" }[m[2]];
+  return m[1] + " " + word + (m[1] === "1" ? "" : "s");
+}

internal/web/static/assets/pages/encdns.js

@@ -35,13 +35,15 @@ function statusPanel(s) {
   ];
   const addr = (s.dot[0] || s.doh[0] || {}).address || "ADDRESS";
   const dotPortArg = s.dot[0] && s.dot[0].port !== 853 ? " -p " + s.dot[0].port : "";
+  const dohPortArg = s.doh[0] && s.doh[0].port !== 443 ? " -p " + s.doh[0].port : "";
   const clients = [];
   if (s.dot_host) {
     clients.push(h("h3", null, "DNS over TLS"), copyBox(s.dot_host),
       h("ul", { class: "small" },
         h("li", null, h("strong", null, "Android: "), "Settings → Network → Private DNS → hostname: ", h("code", null, s.hostname), ". Away from home Android can't reach this server and then resolves nothing, so use a VPN home (WireGuard) or switch Private DNS to Automatic when you leave."),
         h("li", null, h("strong", null, "Linux (systemd-resolved): "), h("code", null, "DNS=" + addr + "#" + s.hostname), " and ", h("code", null, "DNSOverTLS=yes"), " in /etc/systemd/resolved.conf (", h("code", null, "opportunistic"), " falls back to plain DNS away from home)."),
-        h("li", null, h("strong", null, "Test: "), h("code", null, "kdig +tls-ca +tls-host=" + s.hostname + dotPortArg + " @" + addr + " example.com"))));
+        h("li", null, h("strong", null, "Test: "), h("code", null, "dig +tls +tls-ca +tls-hostname=" + s.hostname + dotPortArg + " @" + addr + " example.com"),
+          " (BIND's dig; +tls-ca checks the certificate against the system's CAs)")));
   }
   if (s.doh_url) {
     clients.push(h("h3", null, "DNS over HTTPS"), copyBox(s.doh_url),
@@ -49,7 +51,7 @@ function statusPanel(s) {
         h("li", null, h("strong", null, "Firefox: "), "Settings → Privacy & Security → DNS over HTTPS → Increased protection (falls back to normal DNS) → Custom: the URL above."),
         h("li", null, h("strong", null, "Chrome, Edge: "), "Settings → Privacy and security → Security → Use secure DNS → custom: the URL above."),
         h("li", null, h("strong", null, "Windows 11: "), "Settings → Network → your connection → DNS server assignment → Edit: " + addr + ", DNS over HTTPS on (manual template), the URL above."),
-        h("li", null, h("strong", null, "Test: "), h("code", null, "curl -s -H 'accept: application/dns-json' '" + s.doh_url + "?name=example.com'"), " (or ", h("code", null, "dig +https @" + addr + " example.com"), ")")));
+        h("li", null, h("strong", null, "Test: "), h("code", null, "dig +https +tls-ca +tls-hostname=" + s.hostname + dohPortArg + " @" + addr + " example.com"))));
   }
   return [
     h("div", { class: "panel" }, h("h2", null, "Set up ", badge("on", "ok")), h("dl", { class: "kv" }, rows)),

internal/web/static/assets/pages/explore.js

@@ -2,7 +2,7 @@
 // last days, filtered by any mix of client, name, view, record type, result
 // and blocking. Every value is a link that adds it to the filter, so a
 // client leads to its names, a name to the clients asking for it, and so on.
-import { h, mount, fmtTime, errorBox, badge, backLink } from "../ui.js";
+import { h, mount, fmtTime, errorBox, badge, backLink, input } from "../ui.js";
 import { get } from "../api.js";
 import { lineChart, chartCard, legend, timeTable, barList, compact } from "../charts.js";
 import { viewsOff } from "./stats.js";
@@ -19,37 +19,61 @@ const ranges = [["1h", 3600], ["6h", 6 * 3600], ["24h", 86400], ["3d", 3 * 86400
 // exploreHref is the explore page for these filters (and range).
 export function exploreHref(filters, since) {
   const q = new URLSearchParams();
-  for (const k of keys) if (filters[k]) q.set(k, filters[k]);
+  for (const k of keys) {
+    const v = Array.isArray(filters[k]) ? filters[k].join(",") : filters[k];
+    if (v) q.set(k, v);
+  }
   if (since && since !== "24h") q.set("since", since);
   return "#/stats/explore" + (q.toString() ? "?" + q : "");
 }
 
 const blockedLabel = (k) => (k === "rules" ? "your rules" : k === "yes" ? "anything" : k === "no" ? "nothing" : k);
 
-export async function render(el, params, query) {
+// A filter is, per kind, a list of values: any of them, and !VALUE for
+// "not this". Names: *.example.com is it and the names below it.
+const kinds = ["client", "name", "view", "type", "rcode", "blocked"];
+const split = (v) => (Array.isArray(v) ? v : String(v || "").split(",")).map((x) => x.trim()).filter(Boolean);
+
+function parse(query) {
   const f = {};
-  for (const k of keys) if (query.get(k)) f[k] = query.get(k);
+  for (const k of kinds) f[k] = split(query.get(k));
+  for (const d of split(query.get("domain"))) f.name.push(d.startsWith("!") ? "!*." + d.slice(1).replace(/^\*\./, "") : "*." + d.replace(/^\*\./, ""));
+  return f;
+}
+const plain = (v) => v.replace(/^!/, "");
+const isNot = (v) => v.startsWith("!");
+function addTo(f, k, v) { const g = copy(f); g[k] = g[k].filter((x) => x !== "!" + v && x !== v).concat(v); return g; }
+function notTo(f, k, v) { const g = copy(f); g[k] = g[k].filter((x) => x !== v && x !== "!" + v).concat("!" + v); return g; }
+function dropFrom(f, k, v) { const g = copy(f); g[k] = g[k].filter((x) => x !== v); return g; }
+function copy(f) { const g = {}; for (const k of kinds) g[k] = [...(f[k] || [])]; return g; }
+const includes = (f, k) => (f[k] || []).filter((v) => !isNot(v));
+const anyFilter = (f) => kinds.some((k) => (f[k] || []).length);
+
+let generation = 0;
+
+export async function render(el, params, query) {
+  const gen = ++generation;
+  const hash = location.hash;
+  const f = parse(query);
   const since = query.get("since") || "24h";
-  const with_ = (more) => exploreHref({ ...f, ...more }, since);
-  const without = (k) => { const g = { ...f }; delete g[k]; return exploreHref(g, since); };
+  const go = (g) => exploreHref(g, since);
+  const q = {};
+  for (const k of kinds) if (f[k].length) q[k] = f[k].join(",");
 
   // The Statistics page's view switches apply here too.
   const off = viewsOff();
-  let views = "";
-  if (off.size) {
-    const all = (await get("/views").catch(() => [])).map((v) => v.name);
-    views = all.filter((v) => !off.has(v)).join(",");
-  }
+  const allViews = (await get("/views").catch(() => [])).map((v) => v.name);
+  const views = off.size ? allViews.filter((v) => !off.has(v)).join(",") : "";
   let ex;
   try {
-    ex = await get("/stats/explore", { ...f, since, views: views || undefined });
+    ex = await get("/stats/explore", { ...q, since, views: views || undefined });
   } catch (e) {
-    mount(el, head(f, without, since, null), e.code === "query_log_off"
+    mount(el, head(f, go, since, null), filterBar(f, since, null, allViews), e.code === "query_log_off"
       ? h("div", { class: "msg info" }, e.message) : errorBox(e));
     return;
   }
 
-  const parts = [head(f, without, since, ex)];
+  const parts = [head(f, go, since, ex), filterBar(f, since, ex, allViews)];
   const maxSecs = retentionSecs(ex.retention);
   parts.push(h("div", { class: "filter-row", role: "group", "aria-label": "Time range" },
     h("span", { class: "muted" }, "Range"),
@@ -62,8 +86,8 @@ export async function render(el, params, query) {
   parts.push(h("div", { class: "cards" },
     tile("Answers", compact(ex.total), "from " + fmtTime(ex.from)),
     tile("Blocked", compact(ex.blocked), ex.total ? pct(ex.blocked / ex.total) + " of them" : ""),
-    nx && !f.rcode ? tile("NXDOMAIN", compact(nx.count), pct(nx.count / Math.max(1, ex.total)) + " of them") : null,
-    !f.client && ex.by.client ? tile("Clients", (ex.by.client.length >= 15 ? "15+" : String(ex.by.client.length)), "asking") : null));
+    nx && !includes(f, "rcode").length ? tile("NXDOMAIN", compact(nx.count), pct(nx.count / Math.max(1, ex.total)) + " of them") : null,
+    includes(f, "client").length !== 1 && ex.by.client ? tile("Clients", (ex.by.client.length >= 15 ? "15+" : String(ex.by.client.length)), "asking") : null));
 
   const times = ex.series.map((p) => p.t);
   const series = [{ label: "Answers", values: ex.series.map((p) => p.count), slot: 1 }];
@@ -72,58 +96,120 @@ export async function render(el, params, query) {
     ex.total ? lineChart({ series, times, step: ex.step, area: true }) : h("div", { class: "empty" }, "No answers match."),
     ex.total ? timeTable(series, times, ex.step) : null, { wide: true, legend: series.length > 1 ? legend(series) : null }));
 
-  // A breakdown for each dimension the filter leaves open: click a value
-  // to add it.
+  // A breakdown for each kind the filter leaves open: click a value to add
+  // it, "hide" to leave it out.
   const lists = dims.filter(([d]) => ex.by[d] && !(d === "blocked" && !ex.blocked)).map(([d, title]) => {
-    const href = (k) => with_(d === "blocked" ? { blocked: k } : { [d]: k });
     const label = d === "blocked" ? blockedLabel : d === "view" ? (k) => (k === "_default" ? "(no views)" : k) : (k) => k;
+    const pinned = (k) => includes(f, d).includes(k);
     return h("div", { class: "card chart-card" }, h("h3", null, title),
-      barList(ex.by[d], { total: d === "blocked" ? ex.blocked : ex.total, href, label, empty: d === "blocked" ? "Nothing blocked." : "None." }));
+      barList(ex.by[d], { total: d === "blocked" ? ex.blocked : ex.total, label, empty: d === "blocked" ? "Nothing blocked." : "None.",
+        href: (k) => (pinned(k) ? null : go(addTo(f, d, k))),
+        extra: (k) => h("a", { class: "bar-hide", href: go(notTo(f, d, k)), title: "Leave " + label(k) + " out" }, "hide") }));
   });
   if (lists.length) parts.push(h("div", { class: "chart-grid" }, lists));
 
-  parts.push(recentPanel(ex, with_));
+  parts.push(recentPanel(ex, (k, v) => go(addTo(f, k, v))));
+  if (gen !== generation) return; // the user moved on
+  el.classList.remove("refreshing");
   mount(el, parts);
+  scheduleRefresh(el, params, query, gen, hash);
 }
 
-// head is the title and the filter chips, each with a ✕ to take it out.
-function head(f, without, since, ex) {
-  const chips = keys.filter((k) => f[k]).map((k) => h("span", { class: "chip filter-chip" },
-    h("span", { class: "muted" }, filterWords[k] + " "),
-    h("strong", { class: "mono" }, k === "blocked" ? blockedLabel(f[k]) : f[k]),
-    k === "client" && ex && ex.client_name ? h("span", { class: "muted" }, " " + ex.client_name) : null,
-    " ", h("a", { href: without(k), title: "Take it out", class: "chip-x", "aria-label": "Remove the " + filterWords[k] + " filter" }, "✕")));
+// scheduleRefresh redraws the page a minute later with the same filters,
+// so the graphs move along; not while a filter is being typed in.
+function scheduleRefresh(el, params, query, gen, hash) {
+  setTimeout(() => {
+    if (gen !== generation || location.hash !== hash || !el.isConnected) return;
+    const form = el.querySelector("form.explore-filters");
+    if (form && (form.contains(document.activeElement) || form.dataset.dirty)) {
+      scheduleRefresh(el, params, query, gen, hash); // try again later
+      return;
+    }
+    el.classList.add("refreshing");
+    render(el, params, query).catch(() => el.classList.remove("refreshing"));
+  }, 60_000);
+}
+
+// filterBar types filters in. Each field takes several values, comma
+// separated (any of them), !value to leave one out, and names *.domain for
+// it and the names below it. It starts from the current filters and
+// suggests values from what is shown.
+function filterBar(f, since, ex, views) {
+  const sugg = (id, entries) => h("datalist", { id }, (entries || []).map((e) => h("option", { value: e.key }, e.name || null)));
+  const field = (k, placeholder, width, list) => input({ value: (f[k] || []).join(", "), placeholder, class: "mono", list, style: { width } });
+  const client = field("client", "any client", "190px", "ex-clients");
+  const name = field("name", "nas.adbraun.lol, *.apple.com", "250px", "ex-names");
+  const view = field("view", "any view", "130px", "ex-views");
+  const type = field("type", "any type", "110px", "ex-types");
+  const rcode = field("rcode", "any result", "130px", "ex-rcodes");
+  const blocked = field("blocked", "yes, no or a list", "140px", "ex-blocked");
+  const apply = () => {
+    const g = { client: split(client.value), name: split(name.value).map((v) => v.toLowerCase().replace(/\.$/, "")), view: split(view.value),
+      type: split(type.value).map((v) => v.toUpperCase()), rcode: split(rcode.value).map((v) => v.toUpperCase()), blocked: split(blocked.value) };
+    location.hash = exploreHref(g, since);
+  };
+  const form = h("form", { class: "explore-filters", onsubmit: (e) => { e.preventDefault(); apply(); }, oninput: () => { form.dataset.dirty = "1"; } },
+    h("label", null, "Client ", client), h("label", null, "Name ", name), h("label", null, "View ", view),
+    h("label", null, "Type ", type), h("label", null, "Result ", rcode), h("label", null, "Blocked ", blocked),
+    h("button", { type: "submit", class: "primary small" }, "Show"),
+    sugg("ex-clients", ex && ex.by.client), sugg("ex-names", ex && ex.by.name), sugg("ex-views", views.map((v) => ({ key: v }))),
+    sugg("ex-types", (ex && ex.by.type) || ["A", "AAAA", "PTR", "MX", "TXT", "HTTPS", "SRV"].map((k) => ({ key: k }))),
+    sugg("ex-rcodes", ["NOERROR", "NXDOMAIN", "SERVFAIL", "REFUSED"].map((k) => ({ key: k }))),
+    sugg("ex-blocked", [{ key: "yes" }, { key: "no" }, ...((ex && ex.by.blocked) || [])]));
+  return h("div", { class: "panel explore-filter-panel" }, form,
+    h("div", { class: "muted small" }, "Several values: comma separated, any of them. !value leaves one out (!10.0.0.250, !*.apple.com). *.example.com is it and the names below it."));
+}
+
+// head is the title and a chip per filter value, each with a ✕.
+function head(f, go, since, ex) {
+  const chips = [];
+  for (const k of kinds) for (const v of f[k] || []) {
+    const val = plain(v);
+    chips.push(h("span", { class: "chip filter-chip" + (isNot(v) ? " not" : "") },
+      h("span", { class: "muted" }, (isNot(v) ? "not " : "") + filterWords[k] + " "),
+      h("strong", { class: "mono" }, k === "blocked" ? blockedLabel(val) : val),
+      k === "client" && !isNot(v) && ex && ex.client_name && includes(f, "client").length === 1 ? h("span", { class: "muted" }, " " + ex.client_name) : null,
+      " ", h("a", { href: go(dropFrom(f, k, v)), title: "Take it out", class: "chip-x", "aria-label": "Remove " + filterWords[k] + " " + val }, "✕")));
+  }
   const extra = [];
-  if (f.name && !f.domain) extra.push(h("a", { href: exploreHref({ ...f, name: undefined, domain: f.name }, since) }, "this name and the names below it"));
-  if ((f.name || f.domain) && (f.blocked === "yes" || (ex && ex.blocked))) extra.push(h("a", { href: "#/blocking?check=" + encodeURIComponent(f.name || f.domain) }, "why is it blocked?"));
-  if (f.name || f.domain) extra.push(h("a", { href: "#/search?q=" + encodeURIComponent(f.name || f.domain) }, "find it in your zones"));
+  const names = includes(f, "name");
+  if (names.length === 1) {
+    const n = names[0], bare = n.replace(/^\*\./, "");
+    if (!n.startsWith("*.")) extra.push(h("a", { href: go(addTo(dropFrom(f, "name", n), "name", "*." + n)) }, "this name and the names below it"));
+    if (includes(f, "blocked").length || (ex && ex.blocked)) extra.push(h("a", { href: "#/blocking?check=" + encodeURIComponent(bare) }, "why is it blocked?"));
+    extra.push(h("a", { href: "#/search?q=" + encodeURIComponent(bare) }, "find it in your zones"));
+  }
   return h("div", { class: "page-head" },
     h("div", { class: "grow" },
       backLink("Statistics", "#/stats"),
       h("h1", null, title(f, ex)),
-      h("div", { class: "muted" }, "Every answer in the query log that matches. Click a client, name, view, type or result to narrow it down; ✕ takes a filter out."),
+      h("div", { class: "muted" }, "Every answer in the query log that matches. Click a value to add it, \"hide\" to leave it out; ✕ takes it out of the filter."),
       chips.length ? h("div", { class: "chips filter-chips" }, chips,
         h("a", { href: exploreHref({}, since), class: "small" }, "Clear all")) : null,
       extra.length ? h("div", { class: "small" }, extra.map((a, i) => [i ? " · " : "", a])) : null));
 }
 
 function title(f, ex) {
-  const n = Object.keys(f).length;
-  if (!n) return "Explore the answers";
-  const who = f.client ? (ex && ex.client_name ? ex.client_name : f.client) : "";
-  if (f.client && n === 1) return "Client " + who;
-  const what = [f.rcode, f.type, f.blocked ? "blocked" + (f.blocked !== "yes" ? " by " + blockedLabel(f.blocked) : "") : "",
-    f.name || (f.domain ? f.domain + " and below" : "")].filter(Boolean).join(" ");
+  if (!anyFilter(f)) return "Explore the answers";
+  const inc = (k) => includes(f, k);
+  const or = (vs, fmt = (x) => x) => vs.map(fmt).join(" or ");
+  const clients = inc("client");
+  const who = clients.length === 1 && ex && ex.client_name ? ex.client_name : or(clients);
+  const onlyClient = clients.length && kinds.every((k) => k === "client" || !(f[k] || []).length);
+  if (onlyClient && clients.length === 1) return "Client " + who;
+  const what = [or(inc("rcode")), or(inc("type")),
+    inc("blocked").length ? "blocked" + (inc("blocked")[0] !== "yes" ? " by " + or(inc("blocked"), blockedLabel) : "") : "",
+    or(inc("name"), (n) => (n.startsWith("*.") ? n.slice(2) + " and below" : n))].filter(Boolean).join(" ");
   const parts = [what || "Answers"];
-  if (f.client) parts.push("for " + who);
-  if (f.view) parts.push("in view " + f.view);
+  if (clients.length) parts.push("for " + who);
+  if (inc("view").length) parts.push("in view " + or(inc("view")));
   return parts.join(" ");
 }
 
 // recentPanel lists the latest answers, every cell a link.
-function recentPanel(ex, with_) {
+function recentPanel(ex, add) {
   if (!ex.recent.length) return null;
-  const a = (text, more, cls) => h("a", { href: with_(more), class: cls || "" }, text);
+  const a = (text, more, cls) => { const [k, v] = Object.entries(more)[0]; return h("a", { href: add(k, v), class: cls || "" }, text); };
   return h("div", { class: "panel" }, h("h2", null, "Latest answers"),
     h("div", { class: "table-wrap" }, h("table", { class: "recent-queries" },
       h("thead", null, h("tr", null, ["Time", "Client", "View", "Name", "Type", "Result"].map((t) => h("th", null, t)))),

internal/web/static/assets/pages/stats.js

@@ -43,7 +43,9 @@ export async function render(el, params, query) {
       h("div", { class: "muted" }, status.enabled
         ? "From named's counters" + (status.query_log ? " and response log" : "") + " · data since " + (status.first_data ? fmtTime(status.first_data) : "–") +
           " · kept: minutes " + status.retention.minute + ", hours " + status.retention.hour + ", days " + status.retention.day
-        : "Query statistics are off")));
+        : "Query statistics are off")),
+    status.enabled && status.retention && status.retention.queries
+      ? h("div", { class: "actions" }, h("a", { class: "button", href: "#/stats/explore", title: "Every answer of the last " + status.retention.queries + ", by client, name, view, type and result" }, "Explore the answers")) : null);
   if (!status.enabled) {
     mount(el, head, disabledPanel());
     return;
@@ -145,7 +147,7 @@ export async function render(el, params, query) {
     rtt.length ? listCard("Upstream response time", "how long other servers took to answer named's recursive queries", barList(rtt)) : null,
     b.blocklist.length ? listCard("Blocked by", "which of your rules and blocklists blocked queries", barList(b.blocklist, { label: (k) => (k === "rules" ? "your rules" : k), href: ex((k) => ({ blocked: k })) || (() => "#/blocking") })) : null));
 
-  parts.push(...serverSection(data.server, complete(data.gauges), rangeLabel));
+  parts.push(...serverSection(data.server, complete(data.gauges), rangeLabel, status));
   el.classList.remove("refreshing");
   mount(el, parts);
   scheduleRefresh(el, params, query, gen);
@@ -181,7 +183,9 @@ async function loadAll(since, heatSince, queryLog, vf, filtered) {
 // keeping the current render (dimmed) until the new one is ready.
 function scheduleRefresh(el, params, query, gen) {
   setTimeout(() => {
-    if (gen !== generation || !location.hash.startsWith("#/stats") || !el.isConnected) return;
+    // Only on the Statistics page itself: not #/stats/explore.
+    const h = location.hash;
+    if (gen !== generation || !(h === "#/stats" || h.startsWith("#/stats?")) || !el.isConnected) return;
     el.classList.add("refreshing");
     render(el, params, query).catch(() => el.classList.remove("refreshing"));
   }, 60_000);
@@ -223,7 +227,7 @@ const gaugeKeys = ["g:named:rss", "g:named:mem", "g:cache:mem", "g:named:cpu"];
 
 // serverSection shows named's process, cache, clients, the host and the
 // disks now, and named's memory and CPU over time.
-function serverSection(sv, gauges, rangeLabel) {
+function serverSection(sv, gauges, rangeLabel, status) {
   const srv = (sv && sv.server) || {};
   const named = (sv && sv.named) || {};
   const host = srv.host || {};
@@ -243,7 +247,8 @@ function serverSection(sv, gauges, rangeLabel) {
     tc ? tile("TCP clients", tc[0], tc[1]) : null,
     tile("Up since", srv.boot_time ? relTime(srv.boot_time).replace(" ago", "") : "–", srv.config_time ? "named started " + fmtTime(srv.boot_time) + " · reconfigured " + relTime(srv.config_time) : ""),
     host.mem_total ? tile("Host memory", pct((host.mem_total - host.mem_available) / host.mem_total), "of " + bytes(host.mem_total) + " in use · load " + (host.load || []).map((x) => x.toFixed(2)).join(" ")) : null,
-    ...((sv && sv.disks) || []).map((d) => tile("Disk space", bytes(d.free) + " free", h("span", { title: d.path }, "of " + bytes(d.total) + ", for " + d.holds.join(", "))))));
+    sv && sv.database ? dbTile(sv.database, status) : null,
+    ...((sv && sv.disks) || []).map(diskTile)));
   if (sv && sv.named_error) out.push(h("div", { class: "msg info" }, "Client counts need rndc: " + sv.named_error));
   // Gauges over time: averages, -1 where there was no sample.
   const series = (key) => ((gauges.series || []).find((x) => x.key === key) || { values: gauges.times.map(() => -1) }).values.map((v) => (v < 0 ? null : v));
@@ -312,3 +317,33 @@ function disabledPanel() {
 function exploreSince(since) {
   return rangeSecs[since] > 7 * 86400 ? "7d" : since;
 }
+
+// dbTile is the statistics database's size: what the statistics cost on
+// disk (the dashboard's Storage panel has all of Jarl's databases).
+function dbTile(db, status) {
+  const keep = ((status || {}).retention || {}).queries;
+  const sub = db.query_log_rows != null
+    ? "query log: " + compact(db.query_log_rows) + (db.query_log_rows === 1 ? " answer" : " answers") + " so far, each kept for " + retentionWords(keep)
+    : "counters and top lists; no query log";
+  return h("a", { class: "card link", href: "#/", title: db.path + " · all of Jarl's databases: Dashboard → Storage" },
+    h("div", { class: "label" }, "Statistics database"), h("div", { class: "value" }, bytes(db.bytes)), h("div", { class: "sub" }, sub));
+}
+
+// diskTile is a disk's free space, with a bar of how full it is.
+function diskTile(d) {
+  const u = d.total ? (d.total - d.free) / d.total : 0;
+  return h("div", { class: "card", title: d.path },
+    h("div", { class: "label" }, "Disk space"),
+    h("div", { class: "value" }, bytes(d.free) + " free"),
+    h("div", { class: "disk-bar", role: "img", "aria-label": Math.round(u * 100) + "% used" },
+      h("div", { class: "seg other" + (u > 0.9 ? " full" : ""), style: { width: u * 100 + "%" } })),
+    h("div", { class: "sub" }, h("span", { class: u > 0.9 ? "warn-text" : "" }, Math.round(u * 100) + "% of " + bytes(d.total) + " used"), ", for " + d.holds.join(", ")));
+}
+
+// retentionWords: "7d" -> "7 days".
+function retentionWords(r) {
+  const m = /^(\d+)([dhm])$/.exec(r || "");
+  if (!m) return r || "a while";
+  const word = { d: "day", h: "hour", m: "minute" }[m[2]];
+  return m[1] + " " + word + (m[1] === "1" ? "" : "s");
+}

internal/web/static/assets/style.css

@@ -613,3 +613,12 @@ table.recent-queries td.oneline-cell { max-width: 40ch; white-space: nowrap; ove
 .swatch { display: inline-block; width: 10px; height: 10px; border-radius: 2px; vertical-align: -1px; }
 .swatch.db { background: var(--accent); }
 .swatch.other { background: var(--muted); opacity: 0.45; }
+.card .disk-bar { margin: 6px 0 4px; height: 8px; }
+
+/* Explore: the filter bar. */
+form.explore-filters { display: flex; flex-wrap: wrap; gap: 6px 12px; align-items: center; }
+form.explore-filters label { color: var(--muted); display: inline-flex; gap: 6px; align-items: center; }
+.explore-filter-panel { padding-top: 10px; padding-bottom: 10px; }
+.chip.filter-chip.not strong { text-decoration: line-through; }
+.bar-list a.bar-hide { font-size: 11px; color: var(--muted); margin-right: 4px; text-decoration: none; opacity: 0.6; }
+.bar-list li:hover a.bar-hide, .bar-list a.bar-hide:focus { opacity: 1; text-decoration: underline; }

scripts/e2e.sh

@@ -497,6 +497,27 @@ run "acknowledged: info, with the reason" 0 $J exposure
 contains "info  *external  *security_open_resolver .*acknowledged by .*e2e: on purpose"
 run "take it back" 0 $J exposure unack external security_open_resolver
 run "close the resolver again" 0 $J option unset allow-recursion --view external
+# Statements told apart by their name (tls NAME, key NAME, ...): setting
+# one replaces only the one of that name.
+run "a tls statement for forwarding" 0 $J option set tls 'e2e-quad9 { remote-hostname "dns.quad9.net"; }' --toplevel
+run "a second one, without --add" 0 $J option set tls 'e2e-cf { remote-hostname "cloudflare-dns.com"; }' --toplevel
+run "both are there" 0 $J option list --toplevel
+contains "e2e-quad9"
+contains "e2e-cf"
+run "changing one leaves the other" 0 $J option set tls 'e2e-quad9 { remote-hostname "dns11.quad9.net"; }' --toplevel
+run "list" 0 $J option list --toplevel
+contains "dns11.quad9.net"
+contains "e2e-cf"
+not_contains '"dns.quad9.net"'
+run "unset needs to know which (exit 1)" 1 $J option unset tls --toplevel
+contains "say which: jarl option unset tls"
+run "forwarders over TLS" 0 $J option set forwarders '{ 9.9.9.9 port 853 tls e2e-quad9; 1.1.1.1 port 853 tls e2e-cf; }' --view internal
+run "named.conf has them" 0 grep -q 'tls e2e-quad9' $E/etc-jarl/named.conf
+run "take the forwarders out" 0 $J option unset forwarders --view internal
+run "unset one by its name" 0 $J option unset tls e2e-cf --toplevel
+run "and the other" 0 $J option unset tls e2e-quad9 --toplevel
+run "none left" 0 $J option list --toplevel
+not_contains "e2e-quad9"
 run "fix: hide the version" 0 $J exposure fix server exposure_version
 expect_dig "version.bind is hidden" "" -c CH version.bind TXT
 run "mark external private" 0 $J view update external --exposure private