Compare commits

...
6 Commits
Author SHA1 Message Date
adamandClaude Opus 5.5 b367e0c830 Release 0.6.1
release / release (push) Successful in 4m42s
Certificates for other hosts (jarl cert get, with a daily timer),
certificate export, PTR record and override fixes in linked zones,
one-view tokens with linked zones, numeric order in reverse zones, and
cert get refusing to take over a name saved for another Jarl server.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 22:42:42 +01:00
adamandClaude Opus 5.5 254867c61d CLI: cert get doesn't take over another server's saved name
Each Jarl server has its own certificate names, but a host keeps one
saved NAME.json (and, by default, one /etc/jarl/certs/NAME). Asking a
second server for the same name overwrote the first one's settings and
files, so the first certificate quietly stopped being renewed. Now that
is refused, as is a --dir holding another certificate's files;
--replace takes the name or directory over. A --dir another saved
certificate uses is always refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 22:42:42 +01:00
adamandClaude Opus 5.5 3ddd32f97f Release 0.6.1-rc.4
release / release (push) Successful in 4m43s
0.6.1's fourth release candidate: the Stats page's range buttons follow
the retention settings, jarl changeset show prints local time, and the CLI
no longer prints "and N more" when the server sends no PTR list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 22:32:19 +01:00
adamandClaude Opus 5.5 0976faba3f CLI: changeset show prints local time
release / release (push) Failing after 18s
Created and Applied were printed in UTC, while history and status print
local time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 17:29:43 +01:00
adamandClaude Opus 5.5 1d535c8063 Stats: range buttons follow the retention settings
The Statistics page offers only the ranges kept (longest of the minute,
hour and day retentions), adds 1y and 2y when the daily data covers them,
caps the heatmap at the hourly retention and links to Explore within
retention.queries. Status and explore send retention_seconds. The query
log error names stats.retention.queries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 15:19:12 +01:00
adamandClaude Opus 5.5 7f3564c0a5 CLI: don't print "and N more" when the server sends no PTR list
A 0.6.1-rc.2 server sends ptr_other_views without ptr_other_views_list,
so zone show printed only "and 1 more" under "Other views:".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 15:05:50 +01:00
17 changed files with 235 additions and 41 deletions

No files matched your search

+25
View File
@@ -6,6 +6,8 @@ All notable changes to Jarl are documented here. The format follows
## [Unreleased]
## [0.6.1] - 2026-10-20
### Added
- Certificates for other hosts: `jarl cert get NAME -d DOMAIN --dir DIR
@@ -42,6 +44,14 @@ All notable changes to Jarl are documented here. The format follows
### Changed
- The Statistics page's range buttons follow `stats.retention`: ranges
longer than any data kept are left out, and **1y** and **2y** appear
when `retention.day` (or `retention.hour`) covers them (with the
default 730d, both do). The busiest-times heatmap says how far back it
really goes (at most `retention.hour`), and links to the Explore page
keep within `retention.queries` instead of assuming 7 days.
`GET /stats/status` and `GET /stats/explore` add `retention_seconds`.
- A reverse zone's missing PTR records count only its own view's A and
AAAA records. An address only another view's records use (an internal
host, seen from the vps view's copy of `10.0.10.in-addr.arpa`) is no
@@ -97,6 +107,21 @@ All notable changes to Jarl are documented here. The format follows
### Fixed
- `jarl cert get NAME` no longer takes over a name this host has saved
for another Jarl server, which overwrote that certificate's settings
(and, with the default `--dir`, its files), so it stopped being
renewed. It is refused, as is a `--dir` holding another certificate's
files; `--replace` takes the name or directory over.
- The "query log is off" error named a setting that doesn't exist,
`stats.query_retention`; it's `stats.retention.queries`.
- `jarl zone show` for a reverse zone printed only "and 1 more" under
"Other views:" when the server was older than the CLI (0.6.1-rc.2
sends the count but not the addresses). It now prints just the count
line then.
- `jarl changeset show` printed its Created and Applied times in UTC,
while `jarl history` and `jarl status` print local time. It uses local
time now too.
- The statistics' "Busiest times" grid drew hours without data taller
than the others (they picked up the page's padding for empty
messages).
+2 -3
View File
@@ -202,9 +202,8 @@ review step.
## Status
Version 0.6.0 is the current release, and 0.6.1 is in testing
(0.6.1-rc.3): it gets certificates onto your other hosts (`jarl cert
get`: Jarl obtains and renews them, a host downloads them and runs a
Version 0.6.1 is the current release. It gets certificates onto your
other hosts (`jarl cert get`: Jarl obtains and renews them, a host downloads them and runs a
hook, a daily timer keeps them current), exports certificates, fixes
PTR records and overrides in linked zones, and makes one-view tokens
(such as ACME tokens for the external view) work with linked zones.
+1 -1
View File
@@ -1 +1 @@
0.6.1-rc.3
0.6.1
+36 -2
View File
@@ -114,6 +114,32 @@ func (sc savedCert) save(dir string) error {
return writeAtomic(filepath.Join(dir, sc.Name+".json"), append(b, '\n'), 0o600)
}
// checkCertClash refuses a certificate that would take over what this
// host keeps for another one: a name saved for another Jarl server
// (each server has its own names, so the server can't refuse it), or a
// directory with another certificate's files. saved[i] is sc's saved
// settings (i < 0: none).
func checkCertClash(saved []savedCert, i int, sc savedCert, replace bool) error {
if i >= 0 && !replace && !sameServer(saved[i].Conn.Server, sc.Conn.Server) {
return fmt.Errorf("%s is saved here for the Jarl server %s, not %s: choose another name, or take it over with --replace", sc.Name, saved[i].Conn.Server, sc.Conn.Server)
}
for j, o := range saved {
if j != i && o.Name != sc.Name && filepath.Clean(o.Dir) == filepath.Clean(sc.Dir) {
return fmt.Errorf("%s holds the files of %s, saved for %s: give %s another --dir", sc.Dir, o.Name, o.Conn.Server, sc.Name)
}
}
if m, err := readMeta(sc.Dir); err == nil && m.Name != "" && m.Name != sc.Name && !replace {
return fmt.Errorf("%s holds the files of the certificate %s: give %s another --dir, or overwrite them with --replace", sc.Dir, m.Name, sc.Name)
}
return nil
}
// sameServer reports whether two server addresses name the same jarld.
func sameServer(a, b string) bool {
norm := func(s string) string { return strings.TrimRight(strings.ToLower(s), "/") }
return norm(a) == norm(b)
}
func readMeta(dir string) (*localMeta, error) {
b, err := os.ReadFile(filepath.Join(dir, metaFile))
if err != nil {
@@ -340,7 +366,7 @@ func runHook(sc savedCert, ci certInfo, renewed bool) error {
func certGetCmd() *cobra.Command {
var domains []string
var dir, hook, keyType, configDir string
var noSave, force bool
var noSave, force, replace bool
var wait time.Duration
cmd := &cobra.Command{
Use: "get [NAME]",
@@ -365,6 +391,10 @@ readable only by you). 'jarl cert get' without a name then does every
saved certificate: run it daily (jarl cert schedule does that with a
systemd timer). Nothing is downloaded until jarld has renewed one.
A name saved here for another Jarl server isn't taken over, nor a --dir
holding another certificate's files, unless --replace is given; a --dir
another saved certificate uses is always refused.
--hook runs with sh after new files are written (and again on the next
run if it failed), with JARL_CERT_NAME, JARL_CERT_DIR, JARL_CERT_CERT,
JARL_CERT_CHAIN, JARL_CERT_FULLCHAIN, JARL_CERT_KEY, JARL_CERT_DOMAINS,
@@ -427,6 +457,9 @@ replaced an older certificate).`,
}
sc.Conn = cn
}
if err := checkCertClash(saved, i, sc, replace); err != nil {
return err
}
todo = []savedCert{sc}
}
var results []*getResult
@@ -469,7 +502,7 @@ replaced an older certificate).`,
}
switch {
case len(failed) == 1 && len(todo) == 1:
return lastErr // its own exit code (3 forbidden, 6 conflict, ...)
return lastErr // its own exit code (3 forbidden, 5 conflict, ...)
case len(failed) > 0:
return fmt.Errorf("failed: %s", strings.Join(failed, ", "))
}
@@ -483,6 +516,7 @@ replaced an older certificate).`,
f.StringVar(&keyType, "key-type", "", "key type of a new certificate: ec256 (default), ec384, rsa2048, rsa3072, rsa4096")
f.BoolVar(&force, "force", false, "download and run the hook even when this host has the current certificate")
f.BoolVar(&noSave, "no-save", false, "don't save the settings for later runs")
f.BoolVar(&replace, "replace", false, "take over a name saved here for another Jarl server, or a --dir holding another certificate's files")
f.DurationVar(&wait, "wait", 15*time.Minute, "how long to wait while jarld obtains it")
f.StringVar(&configDir, "config-dir", "", "where the settings are saved (default /etc/jarl/cert-get as root, ~/.config/jarl/cert-get otherwise)")
_ = cmd.RegisterFlagCompletionFunc("key-type", cobra.FixedCompletions([]string{"ec256", "ec384", "rsa2048", "rsa3072", "rsa4096"}, cobra.ShellCompDirectiveNoFileComp))
+59
View File
@@ -0,0 +1,59 @@
package main
import (
"os"
"path/filepath"
"strings"
"testing"
)
func TestCheckCertClash(t *testing.T) {
tmp := t.TempDir()
a := connection{Server: "https://dns-a.example:8053"}
b := connection{Server: "https://dns-b.example:8053"}
saved := []savedCert{
{Name: "proxy", Dir: filepath.Join(tmp, "proxy"), Conn: a},
{Name: "mail", Dir: filepath.Join(tmp, "mail"), Conn: a},
}
// mail's files are on disk; "other" has files but no saved settings.
if err := (&localMeta{Name: "mail"}).write(mkdir(t, saved[1].Dir)); err != nil {
t.Fatal(err)
}
if err := (&localMeta{Name: "other"}).write(mkdir(t, filepath.Join(tmp, "other"))); err != nil {
t.Fatal(err)
}
cases := []struct {
what string
i int
sc savedCert
replace bool
want string // "" = allowed
}{
{"same server again", 0, savedCert{Name: "proxy", Dir: saved[0].Dir, Conn: a}, false, ""},
{"same server, other spelling", 0, savedCert{Name: "proxy", Dir: saved[0].Dir, Conn: connection{Server: "HTTPS://dns-a.example:8053/"}}, false, ""},
{"other server", 0, savedCert{Name: "proxy", Dir: saved[0].Dir, Conn: b}, false, "saved here for the Jarl server"},
{"other server, replace", 0, savedCert{Name: "proxy", Dir: saved[0].Dir, Conn: b}, true, ""},
{"new name, saved cert's dir", -1, savedCert{Name: "web", Dir: saved[1].Dir, Conn: b}, false, "holds the files of mail"},
{"new name, saved cert's dir, replace", -1, savedCert{Name: "web", Dir: saved[1].Dir + "/", Conn: b}, true, "holds the files of mail"},
{"new name, unsaved cert's files", -1, savedCert{Name: "web", Dir: filepath.Join(tmp, "other"), Conn: b}, false, "certificate other"},
{"new name, unsaved cert's files, replace", -1, savedCert{Name: "web", Dir: filepath.Join(tmp, "other"), Conn: b}, true, ""},
{"new name, empty dir", -1, savedCert{Name: "web", Dir: filepath.Join(tmp, "web"), Conn: b}, false, ""},
}
for _, c := range cases {
err := checkCertClash(saved, c.i, c.sc, c.replace)
switch {
case c.want == "" && err != nil:
t.Errorf("%s: %v", c.what, err)
case c.want != "" && (err == nil || !strings.Contains(err.Error(), c.want)):
t.Errorf("%s: got %v, want %q", c.what, err, c.want)
}
}
}
func mkdir(t *testing.T, dir string) string {
t.Helper()
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatal(err)
}
return dir
}
+2 -2
View File
@@ -272,9 +272,9 @@ func changesetCmd() *cobra.Command {
return err
}
return output(cs, func(w io.Writer) {
fmt.Fprintf(w, "Changeset:\t%d (%s)\nDescription:\t%s\nCreated:\t%s by %s via %s\n", cs.ID, cs.Status, cs.Description, cs.CreatedAt.Format("2006-01-02 15:04"), cs.Username, cs.Via)
fmt.Fprintf(w, "Changeset:\t%d (%s)\nDescription:\t%s\nCreated:\t%s by %s via %s\n", cs.ID, cs.Status, cs.Description, cs.CreatedAt.Local().Format("2006-01-02 15:04"), cs.Username, cs.Via)
if cs.AppliedAt != nil {
fmt.Fprintf(w, "Applied:\t%s by %s\n", cs.AppliedAt.Format("2006-01-02 15:04"), cs.AppliedBy)
fmt.Fprintf(w, "Applied:\t%s by %s\n", cs.AppliedAt.Local().Format("2006-01-02 15:04"), cs.AppliedBy)
}
fmt.Fprintln(w, "\n#\tOPERATION")
for _, op := range cs.Ops {
+25
View File
@@ -0,0 +1,25 @@
package main
import (
"bytes"
"strings"
"testing"
"jarl/internal/api"
)
func TestPrintPTRFills(t *testing.T) {
var b bytes.Buffer
printPTRFills(&b, []api.PTRFill{{Address: "10.0.10.20", Target: "nextcloud.adbraun.lol.", From: "internal"}}, 3)
out := b.String()
if !strings.Contains(out, "10.0.10.20 -> nextcloud.adbraun.lol (view internal)") || !strings.Contains(out, "and 2 more") {
t.Errorf("listed fills:\n%s", out)
}
// An older server sends the count without the list.
b.Reset()
printPTRFills(&b, nil, 1)
if b.Len() != 0 {
t.Errorf("no list from the server printed %q, want nothing", b.String())
}
}
+6 -1
View File
@@ -1183,8 +1183,13 @@ func terminalWidth() int {
}
// printPTRFills lists PTR records a reverse zone lacks under zone show's
// line about them: address -> name, and the view the name is from.
// line about them: address -> name, and the view the name is from. A server
// older than the CLI may send the count without the list; then nothing is
// listed, rather than "and N more" under no addresses.
func printPTRFills(w io.Writer, fills []api.PTRFill, total int) {
if len(fills) == 0 {
return
}
for _, f := range fills {
from := ""
if f.From != "" {
+2 -2
View File
@@ -579,12 +579,12 @@ Needs `stats:read`. When statistics are disabled, every endpoint but
| Method | Path | Query | Returns |
|--------|------|-------|---------|
| GET | `/stats/status` | | `{enabled, query_log, statistics_channel, log_file, last_poll, last_log_line, log_lines_read, first_data, retention, warnings}` |
| GET | `/stats/status` | | `{enabled, query_log, statistics_channel, log_file, last_poll, last_log_line, log_lines_read, first_data, retention, retention_seconds, warnings}`: `retention` has `minute`, `hour`, `day`, `queries` as `"7d"`, `retention_seconds` the same in seconds (`queries` 0 without a query log) |
| GET | `/stats/summary` | `since` (90m, 24h, 7d, 2w; default 24h), `top` (default 10) | totals, `avg_qps`, `peak`, `rcodes`, `qtypes`, `views`, `zones`, `answers`, `transport`, `cache_hit_ratio`, `nxdomain_ratio` (without blocked answers), `blocked`, `blocked_ratio`, `blocklists` (blocked per list; `rules` for your rules), `tops` |
| GET | `/stats/timeline` | `since`, `step` (auto, 1m, 5m, 15m, 1h, 6h, 1d), `key` (repeatable or comma separated; `rcode:*` = every counter of a group; default `queries`) | `{step, times: [unix...], series: [{key, values}]}` |
| GET | `/stats/top` | `kind` (name, client, nxdomain, servfail, refused, blocked, blocked_client), `since`, `limit` (default 20) | `[{key, count}]`; client, refused and blocked_client entries add `name`, the host name of Jarl's record for the address (the summary's client list too) |
| GET | `/stats/heatmap` | `since` (default 28d), `key` (default queries) | `{key, from, avg[7][24], samples[7][24]}`, Monday first, local time, `-1` = no data |
| GET | `/stats/explore` | `?client=&name=&domain=&view=&type=&rcode=&blocked=yes\|no\|LIST&since=24h&views=`; each takes several values, comma separated (any of them), `!value` to leave one out, and names `*.example.com` for it and below | The query log for a filter ([stats.md](stats.md#exploring)): `total`, `blocked`, `series` (per `step` seconds: `t`, `count`, `blocked`), `by` (top 15 of each dimension the filter leaves open: client, name, view, type, rcode, blocked; clients with `name`), `recent` (latest 100: ts, client, client_name, view, name, type, rcode, blocked), `oldest`, `retention`, `client_name`. 404 `query_log_off` without a query log |
| GET | `/stats/explore` | `?client=&name=&domain=&view=&type=&rcode=&blocked=yes\|no\|LIST&since=24h&views=`; each takes several values, comma separated (any of them), `!value` to leave one out, and names `*.example.com` for it and below | The query log for a filter ([stats.md](stats.md#exploring)): `total`, `blocked`, `series` (per `step` seconds: `t`, `count`, `blocked`), `by` (top 15 of each dimension the filter leaves open: client, name, view, type, rcode, blocked; clients with `name`), `recent` (latest 100: ts, client, client_name, view, name, type, rcode, blocked), `oldest`, `retention` (`"7d"`) and `retention_seconds`, `client_name`. 404 `query_log_off` without a query log |
| GET | `/stats/server` | | `{server: {rss, mem_in_use, cpu_percent, threads, pid, cache_rrsets, cache_mem, sockets_udp, sockets_tcp, recursion_highwater, tcp_highwater, version, boot_time, config_time, host: {mem_total, mem_available, load, cpus, uptime}}, named: {recursive_clients, tcp_clients, zones, cpus, worker_threads}, disks: [{path, holds, total, free}], database: {path, bytes, query_log_rows}}`; gauges over time: `/stats/timeline?key=g:named:rss` (averages, -1 without a sample) |
| GET | `/stats/breakdown` | `group` (qtype, rcode, view, zone, answer, transport, rtt, dnssec, cache, ratelimit, blocklist), `since` | `[{key, count}]` |
+7
View File
@@ -292,6 +292,13 @@ jarl cert saved # what this host has, and when it exp
`jarl cert get` without a name does every saved certificate. A token
given as `@FILE` is saved as that reference, not copied. `jarl cert
forget NAME` stops getting one (`--delete` deletes it on jarld too).
* **Several Jarl servers on one host**: each server has its own
certificate names, so two servers may both have a `proxy`. The host
keeps one `proxy` per settings directory, though: asking another server
for a name saved here for the first one is refused, as is a `--dir`
holding another certificate's files. Choose another name or `--dir`, or
pass `--replace` to take the name (or directory) over. A `--dir` that
another saved certificate uses is always refused.
* **The hook** runs with `sh` after new files are written, with
`JARL_CERT_NAME`, `JARL_CERT_DIR`, `JARL_CERT_CERT`, `JARL_CERT_CHAIN`,
`JARL_CERT_FULLCHAIN`, `JARL_CERT_KEY`, `JARL_CERT_DOMAINS`,
+1
View File
@@ -451,6 +451,7 @@ jarl cert export mail --dir /etc/ssl/mail # fullchain.pem and privkey
jarl cert export mail --part bundle # one part to stdout: fullchain, cert, key, bundle
jarl cert get mail -d mail.adbraun.lol --hook 'systemctl reload postfix' # on another host: get, download, hook
jarl cert get # every saved certificate (what the timer runs)
jarl cert get proxy -s other --replace # take over a name saved here for another server
jarl cert saved # this host's certificates and their expiry
jarl cert forget mail [--delete] # stop getting it here
sudo jarl cert schedule [--remove] # daily systemd timer for jarl cert get
+5 -1
View File
@@ -263,7 +263,11 @@ database, roughly in proportion to the number of views.
lists for `retention.hour`, daily ones for `retention.day`. Older data is
deleted every hour.
- Charts use the finest resolution that covers the range: minutes up to 48
hours, hours up to 90 days, days beyond.
hours (and `retention.minute`), hours up to `retention.hour`, days beyond.
- The Statistics page offers only the ranges the data is kept for: up to 90
days with the defaults, plus a year and 2 years when `retention.day` (or
`retention.hour`) is that long. The Explore page offers the ranges up to
`retention.queries`.
- Top lists are exact within an hour (up to 200,000 distinct names or
clients per hour, beyond which the rarest are dropped). Daily lists are
built from the hourly top lists, so for very long tails they are
+11
View File
@@ -95,6 +95,13 @@ type Status struct {
Day string `json:"day"`
Queries string `json:"queries"` // the query log ("" when off)
} `json:"retention"`
// RetentionSeconds is Retention in seconds, for the web UI's ranges.
RetentionSeconds struct {
Minute int64 `json:"minute"`
Hour int64 `json:"hour"`
Day int64 `json:"day"`
Queries int64 `json:"queries"` // 0 when the query log is off
} `json:"retention_seconds"`
}
// Collector gathers statistics into a Store.
@@ -164,8 +171,12 @@ func NewCollector(store *Store, opts Options, zones ZoneIndex, log *slog.Logger)
c.status.Retention.Minute = humanDuration(opts.RetainMinute)
c.status.Retention.Hour = humanDuration(opts.RetainHour)
c.status.Retention.Day = humanDuration(opts.RetainDay)
c.status.RetentionSeconds.Minute = int64(opts.RetainMinute / time.Second)
c.status.RetentionSeconds.Hour = int64(opts.RetainHour / time.Second)
c.status.RetentionSeconds.Day = int64(opts.RetainDay / time.Second)
if opts.QueryLog && opts.RetainQueries > 0 {
c.status.Retention.Queries = humanDuration(opts.RetainQueries)
c.status.RetentionSeconds.Queries = int64(opts.RetainQueries / time.Second)
}
return c
}
+6 -5
View File
@@ -8,7 +8,7 @@ import (
)
// Exploring the query log (docs/stats.md#exploring): every answer of the
// last days (stats.query_retention), filtered by any mix of client, name,
// last days (stats.retention.queries), filtered by any mix of client, name,
// view, type, result and blocking, broken down by each of them.
// ExploreFilter narrows the query log. Empty fields don't filter. Each
@@ -47,8 +47,9 @@ type Exploration struct {
Recent []QueryRow `json:"recent"`
// Oldest is the first answer the query log still has.
Oldest *time.Time `json:"oldest,omitempty"`
// Retention is how long it keeps them.
Retention string `json:"retention"`
// Retention is how long it keeps them, also in seconds.
Retention string `json:"retention"`
RetentionSeconds int64 `json:"retention_seconds"`
}
// ExplorePoint is one step of the series.
@@ -59,7 +60,7 @@ type ExplorePoint struct {
}
// ErrNoQueryLog is returned when the query log is off.
var ErrNoQueryLog = fmt.Errorf("the query log is off: it needs stats.query_log and a stats.query_retention above 0 in /etc/jarl/jarl.yaml (docs/stats.md#exploring)")
var ErrNoQueryLog = fmt.Errorf("the query log is off: it needs stats.query_log and a stats.retention.queries above 0 in /etc/jarl/jarl.yaml (docs/stats.md#exploring)")
// values splits a filter field into the values it must be one of and the
// ones (written !VALUE) it must not be: "10.0.0.5, !10.0.0.9".
@@ -178,7 +179,7 @@ func (q *Querier) Explore(ctx context.Context, f ExploreFilter, topN, recent int
}
from := now.Add(-since).Unix()
out := &Exploration{From: time.Unix(from, 0), To: now, By: map[string][]Entry{}, Series: []ExplorePoint{}, Recent: []QueryRow{},
Retention: humanDuration(q.Opts.RetainQueries)}
Retention: humanDuration(q.Opts.RetainQueries), RetentionSeconds: int64(q.Opts.RetainQueries / time.Second)}
cond, args := q.where(f, from)
db := q.Store.db
+1 -1
View File
@@ -45,7 +45,7 @@ var schema = []string{
PRIMARY KEY (res, ts, kind, key)
) WITHOUT ROWID;
CREATE TABLE meta (key TEXT PRIMARY KEY, value TEXT NOT NULL);`,
// 2: the query log, every answer for a while (stats.query_retention),
// 2: the query log, every answer for a while (stats.retention.queries),
// for drilling down: what a client asked, who asked for a name
// (docs/stats.md#exploring).
`CREATE TABLE queries (
+8 -7
View File
@@ -16,6 +16,13 @@ const filterWords = { client: "client", name: "name", domain: "domain", view: "v
const keys = Object.keys(filterWords);
const ranges = [["1h", 3600], ["6h", 6 * 3600], ["24h", 86400], ["3d", 3 * 86400], ["7d", 7 * 86400], ["30d", 30 * 86400]];
// exploreRange is the longest range here of at most secs ("1h" at least).
export function exploreRange(secs) {
let out = "1h";
for (const [k, s] of ranges) if (s <= secs) out = k;
return out;
}
// exploreHref is the explore page for these filters (and range).
export function exploreHref(filters, since) {
const q = new URLSearchParams();
@@ -74,7 +81,7 @@ export async function render(el, params, query) {
}
const parts = [head(f, go, since, ex), filterBar(f, since, ex, allViews)];
const maxSecs = retentionSecs(ex.retention);
const maxSecs = ex.retention_seconds || 7 * 86400;
parts.push(h("div", { class: "filter-row", role: "group", "aria-label": "Time range" },
h("span", { class: "muted" }, "Range"),
h("div", { class: "segmented" }, ranges.filter(([, s]) => s <= maxSecs).map(([k]) =>
@@ -234,9 +241,3 @@ function stepWord(s) {
if (s % 3600 === 0) return s === 3600 ? "hour" : s / 3600 + " hours";
return s === 60 ? "minute" : s / 60 + " minutes";
}
function retentionSecs(r) {
const m = /^(\d+)([dhm])$/.exec(r || "");
if (!m) return 7 * 86400;
return Number(m[1]) * { d: 86400, h: 3600, m: 60 }[m[2]];
}
+38 -16
View File
@@ -2,10 +2,27 @@ import { h, mount, fmtTime, relTime, errorBox } from "../ui.js";
import { get } from "../api.js";
import { rerender } from "../app.js";
import { lineChart, chartCard, legend, timeTable, heatmap, heatmapTable, barList, compact, stepSeconds, endLabelWidth } from "../charts.js";
import { exploreHref } from "./explore.js";
import { exploreHref, exploreRange } from "./explore.js";
const ranges = [["1h", "1 hour"], ["6h", "6 hours"], ["24h", "24 hours"], ["7d", "7 days"], ["30d", "30 days"], ["90d", "90 days"]];
const rangeSecs = { "1h": 3600, "6h": 6 * 3600, "24h": 86400, "7d": 7 * 86400, "30d": 30 * 86400, "90d": 90 * 86400 };
// Ranges: the since value, its words and its button. Only the ones the
// statistics are kept for are offered (keptSecs).
const ranges = [["1h", "1 hour"], ["6h", "6 hours"], ["24h", "24 hours"], ["7d", "7 days"], ["30d", "30 days"], ["90d", "90 days"],
["365d", "year", "1y"], ["730d", "2 years", "2y"]];
const rangeSecs = { "1h": 3600, "6h": 6 * 3600, "24h": 86400, "7d": 7 * 86400, "30d": 30 * 86400, "90d": 90 * 86400, "365d": 365 * 86400, "730d": 730 * 86400 };
// keptSecs is how far back the statistics go: the longest of the minute,
// hour and day retentions (a server older than retention_seconds: 2 years).
function keptSecs(status) {
const r = status.retention_seconds;
return r ? Math.max(r.minute, r.hour, r.day) : 730 * 86400;
}
// longestWithin is the longest range of at most secs ("1h" at least).
function longestWithin(secs) {
let out = "1h";
for (const [k] of ranges) if (rangeSecs[k] <= secs) out = k;
return out;
}
// Failed answers keep the same color whatever the range: color follows
// the result code, never its rank.
@@ -36,8 +53,9 @@ const perViewGroups = ["qtype", "rcode", "view", "zone", "blocklist"];
export async function render(el, params, query) {
const gen = ++generation;
const want = query && query.get ? query.get("since") : null;
const since = rangeSecs[want] ? want : "24h";
const status = await get("/stats/status");
const kept = keptSecs(status);
const since = rangeSecs[want] && rangeSecs[want] <= kept ? want : longestWithin(Math.min(86400, kept));
const head = h("div", { class: "page-head" },
h("div", { class: "grow" }, h("h1", null, "Statistics"),
h("div", { class: "muted" }, status.enabled
@@ -50,7 +68,10 @@ export async function render(el, params, query) {
mount(el, head, disabledPanel());
return;
}
const heatSince = rangeSecs[since] < 7 * 86400 ? "7d" : since;
// The heatmap needs hourly counters: at least a week of them, at most
// what is kept.
const hourSecs = status.retention_seconds ? status.retention_seconds.hour : 90 * 86400;
const heatSince = longestWithin(Math.min(Math.max(rangeSecs[since], 7 * 86400), hourSecs));
// Which views: all of them unless some are switched off.
const allViews = status.query_log ? (await get("/views").catch(() => [])).map((v) => v.name) : [];
const off = viewsOff();
@@ -58,14 +79,14 @@ export async function render(el, params, query) {
const filtered = allViews.length > 0 && shown.length < allViews.length;
const vf = filtered ? { views: shown.join(",") } : {};
if (filtered && !shown.length) {
mount(el, head, filterRow(since, allViews, off), h("div", { class: "empty" }, "No views chosen: switch one on above."));
mount(el, head, filterRow(since, kept, allViews, off), h("div", { class: "empty" }, "No views chosen: switch one on above."));
return;
}
let data;
try {
data = await loadAll(since, heatSince, status.query_log, vf, filtered);
} catch (e) {
mount(el, head, filterRow(since, allViews, off), errorBox(e));
mount(el, head, filterRow(since, kept, allViews, off), errorBox(e));
return;
}
if (gen !== generation) return; // the user moved on
@@ -74,7 +95,7 @@ export async function render(el, params, query) {
// sudden drop in traffic.
const timeline = complete(data.timeline), errors = complete(data.errors);
const step = stepSeconds[timeline.step];
const parts = [head, filterRow(since, allViews, off)];
const parts = [head, filterRow(since, kept, allViews, off)];
for (const w of status.warnings || []) parts.push(h("div", { class: "msg warn" }, w));
if (filtered) {
parts.push(h("div", { class: "msg info small" }, "Showing view" + (shown.length > 1 ? "s " : " ") + shown.join(", ") + ": answers from named's response log" +
@@ -120,7 +141,7 @@ export async function render(el, params, query) {
// Who asks for what. With the query log, every value leads to the
// explore page: a client to what it asked, a name to who asked.
const ql = !!(status.retention || {}).queries;
const ex = (filters) => (ql ? (k) => exploreHref(filters(k), exploreSince(since)) : null);
const ex = (filters) => (ql ? (k) => exploreHref(filters(k), exploreSince(since, status)) : null);
if (status.query_log) {
const t = data.tops;
parts.push(h("div", { class: "chart-grid" },
@@ -201,11 +222,11 @@ function complete(tl) {
return tl;
}
function filterRow(since, views, off) {
function filterRow(since, kept, views, off) {
const range = h("div", { class: "filter-row", role: "group", "aria-label": "Time range" },
h("span", { class: "muted" }, "Range"),
h("div", { class: "segmented" }, ranges.map(([k, label]) =>
h("a", { href: "#/stats?since=" + k, class: k === since ? "active" : "", "aria-current": k === since ? "true" : null, title: "last " + label }, k))));
h("div", { class: "segmented" }, ranges.filter(([k]) => rangeSecs[k] <= kept).map(([k, label, button]) =>
h("a", { href: "#/stats?since=" + k, class: k === since ? "active" : "", "aria-current": k === since ? "true" : null, title: "last " + label }, button || k))));
if (!views || views.length < 2) return range;
// A toggle per view (together: linked views, say), and all or none.
const set = (next) => { saveViewsOff(next); rerender(); };
@@ -312,10 +333,11 @@ function disabledPanel() {
"(never individual queries) in a separate database, by default for 7 days per minute, 90 days per hour and 2 years per day. See docs/stats.md."));
}
// exploreSince is the Statistics range for the explore page: the query log
// keeps days, not months.
function exploreSince(since) {
return rangeSecs[since] > 7 * 86400 ? "7d" : since;
// exploreSince is the Statistics range for the explore page: at most what
// the query log keeps (days, usually, not months).
function exploreSince(since, status) {
const q = status.retention_seconds ? status.retention_seconds.queries : 7 * 86400;
return exploreRange(Math.min(rangeSecs[since], q));
}
// dbTile is the statistics database's size: what the statistics cost on