Release 0.6.4-rc.2

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

CHANGELOG.md

@@ -8,6 +8,63 @@ All notable changes to Jarl are documented here. The format follows
 
 ### Added
 
+- **Failed logins in the journal.** A failed login (a wrong password, or
+  a wrong two-factor code) and a refused API token are logged as well as
+  audited, with the address first (`msg="login failed" ip=… user=…`,
+  `msg="token refused" ip=…`), for Guard's `jarl` preset or a fail2ban
+  filter to ban such addresses (docs/operations.md).
+- **`jarl cert get --combined PATH`** also writes the key and the full
+  chain in one file (mode 600, written all at once), for HAProxy. It
+  belongs outside `--dir`, so a directory of combined files can be
+  HAProxy's `crt` directory. The hook gets it as `JARL_CERT_COMBINED`.
+  Adding it to a certificate this host already has writes it from the
+  files on disk and runs the hook; `--combined ''` removes it.
+- **`jarl cert get --local`** doesn't ask jarld: it writes the combined
+  file and runs the hook from this host's files, to apply a changed
+  setting or test a hook.
+- **`jarl cert saved NAME`** shows all of one certificate's saved
+  settings (names, key type, dir, combined file, hook, server) and this
+  host's copy. The token is never shown.
+- `jarl cert list` has a PULLED column: when the certificate was last
+  downloaded, with `(old)` while a host hasn't fetched the renewed one.
+  The owner of a certificate obtained with `jarl cert get` now sees their
+  own downloads (`pulled_by`) in `jarl cert list` and `jarl cert show`, not
+  only administrators.
+
+### Changed
+
+- `jarl cert list` shows `get` as USES for a certificate obtained with
+  `jarl cert get`, not `-`.
+- `jarl cert get` without a name refuses `--dir`, `--hook`, `--key-type`,
+  `--combined`, `--replace` and `--no-save`, which change one
+  certificate; they used to be ignored.
+
+## [0.6.3] - 2026-10-21
+
+### Added
+
+- A docs/acme.md section: lego on a host inside a split DNS fails its own
+  propagation check when the challenge is written into the external view
+  only (`JARL_VIEW`, a one-view token, `.../httpreq/view/...`): why, and
+  the ways round it (no view limit, `--dns.propagation.disable-ans` or
+  `--dns.propagation.wait`, Traefik's `propagation.disableANSChecks`).
+- docs/acme.md and `jarl-lego` show lego 5's commands (`lego run ...`,
+  which also renews; `lego migrate` when upgrading), with lego 4's
+  beside them. Jarl's side is the same for both.
+
+## [0.6.2] - 2026-10-21
+
+### Added
+
+- A `certificates` role (template): `certificates:write`, for tokens and
+  users that get certificates for other hosts with `jarl cert get`. In
+  the web UI's user and token forms it is in "Insert a template…", to
+  narrow with `names=`; in the CLI, `--role certificates`.
+
+## [0.6.1] - 2026-10-20
+
+### Added
+
 - Certificates for other hosts: `jarl cert get NAME -d DOMAIN --dir DIR
   --hook CMD` on a mail server or proxy has jarld obtain the certificate
   (and renew it), waits, downloads it when it is newer than the host's
@@ -42,6 +99,36 @@ All notable changes to Jarl are documented here. The format follows
 
 ### Changed
 
+- The Statistics page's range buttons follow `stats.retention`: ranges
+  longer than any data kept are left out, and **1y** and **2y** appear
+  when `retention.day` (or `retention.hour`) covers them (with the
+  default 730d, both do). The busiest-times heatmap says how far back it
+  really goes (at most `retention.hour`), and links to the Explore page
+  keep within `retention.queries` instead of assuming 7 days.
+  `GET /stats/status` and `GET /stats/explore` add `retention_seconds`.
+
+- A reverse zone's missing PTR records count only its own view's A and
+  AAAA records. An address only another view's records use (an internal
+  host, seen from the vps view's copy of `10.0.10.in-addr.arpa`) is no
+  longer reported as missing; it shows apart, as `Other views` in `jarl
+  zone show`, `ptr_other_views` in the API and a note on the zone's page.
+  Filling (`--fill`, `jarl zone fill-ptr`, the page's button) names those
+  from the other view only when asked, with `--other-views`
+  (`fill_other_views`, `?other_views=1`, a checkbox in the New reverse
+  zone dialog), so one view's host names don't leak into another. When
+  there is nothing else to fill, Jarl says how many there are and how to
+  add them. The page and `jarl zone show` list those addresses with the
+  name each would get, and the view it comes from
+  (`ptr_other_views_list`); `jarl zone show` lists the missing ones too.
+
+- Records, zones and validation findings sort numeric labels by value, so
+  `2.10.0.10.in-addr.arpa` comes before `101.10.0.10.in-addr.arpa` in the
+  web UI, the CLI and the rendered zone files. Names still group by label
+  from the right (subdomains stay under their parent); other labels keep
+  the old byte order (`host10` before `host2`). Zone files are rewritten
+  in the new order the next time they change; named doesn't care about
+  record order.
+
 - `cname_chain` no longer warns about one CNAME pointing at another (a
   service name at a host alias is common, and named answers the chain at
   once). A chain of three or more CNAMEs gets an info note that shows
@@ -75,6 +162,21 @@ All notable changes to Jarl are documented here. The format follows
 
 ### Fixed
 
+- `jarl cert get NAME` no longer takes over a name this host has saved
+  for another Jarl server, which overwrote that certificate's settings
+  (and, with the default `--dir`, its files), so it stopped being
+  renewed. It is refused, as is a `--dir` holding another certificate's
+  files; `--replace` takes the name or directory over.
+- The "query log is off" error named a setting that doesn't exist,
+  `stats.query_retention`; it's `stats.retention.queries`.
+- `jarl zone show` for a reverse zone printed only "and 1 more" under
+  "Other views:" when the server was older than the CLI (0.6.1-rc.2
+  sends the count but not the addresses). It now prints just the count
+  line then.
+- `jarl changeset show` printed its Created and Applied times in UTC,
+  while `jarl history` and `jarl status` print local time. It uses local
+  time now too.
+
 - The statistics' "Busiest times" grid drew hours without data taller
   than the others (they picked up the page's padding for empty
   messages).

README.md

@@ -202,12 +202,18 @@ review step.
 
 ## Status
 
-Version 0.6.0 is the current release, and 0.6.1 is in testing
-(0.6.1-rc.1): it gets certificates onto your other hosts (`jarl cert
-get`: Jarl obtains and renews them, a host downloads them and runs a
+Version 0.6.3 is the current release; 0.6.4 is in testing (0.6.4-rc.2):
+failed logins and refused tokens are logged, address first, for Guard
+and fail2ban, and `jarl cert get` can write a combined file for HAProxy
+(`--combined`) and apply settings without jarld (`--local`). 0.6.3 brings the lego docs up to
+lego 5 and explains lego with split DNS. 0.6.2 added a `certificates` role
+for tokens that get certificates for other hosts. 0.6.1 gets certificates onto your
+other hosts (`jarl cert get`: Jarl obtains and renews them, a host downloads them and runs a
 hook, a daily timer keeps them current), exports certificates, fixes
 PTR records and overrides in linked zones, and makes one-view tokens
 (such as ACME tokens for the external view) work with linked zones.
+Reverse zones list addresses in numeric order and no longer report
+another view's hosts as missing PTR records.
 0.6.0 makes reverse zones easy (suggested from your records,
 created in the right view and filled, PTR records kept in step), rebuilds
 comparing a zone's views, lets you explore the statistics down to each

VERSION

@@ -1 +1 @@
-0.6.1-rc.1
+0.6.4-rc.2

cmd/jarl/admin_cmds.go

@@ -392,7 +392,8 @@ ACCESS is read or write, and filters take comma separated glob patterns:
   records:write zones=adbraun.lol names=_acme-challenge.* types=TXT
   *:read
 
-Roles are ready-made grant sets: readonly, dns-editor, operator, acme.`
+Roles are ready-made grant sets: readonly, dns-editor, operator, acme,
+certificates.`
 
 func userCmd() *cobra.Command {
 	cmd := &cobra.Command{Use: "user", Aliases: []string{"users"}, Short: "Manage users (admin)", Long: "Manage users (admin).\n\n" + grantHelp}
@@ -477,7 +478,7 @@ func userCmd() *cobra.Command {
 	}
 	addUserFlags := func(c *cobra.Command) {
 		c.Flags().BoolVar(&admin, "admin", false, "make the user an administrator (--admin=false to remove)")
-		c.Flags().StringVar(&role, "role", "", "replace the grants with a role: readonly, dns-editor, operator or acme")
+		c.Flags().StringVar(&role, "role", "", "replace the grants with a role: readonly, dns-editor, operator, acme or certificates")
 		c.Flags().StringArrayVar(&grants, "grant", nil, `grant (repeatable), e.g. "records:write zones=example.com"`)
 		c.Flags().StringVar(&comment, "comment", "", "comment")
 		c.Flags().IntVar(&certLimit, "cert-limit", -1, "how many certificates the user may own through jarl cert get (-1: certificates.max_per_user in jarl.yaml)")
@@ -758,7 +759,7 @@ func tokenCmd() *cobra.Command {
 		},
 	}
 	create.Flags().StringVar(&user, "user", "", "create the token for this user (admin)")
-	create.Flags().StringVar(&role, "role", "", "restrict the token to a role: readonly, dns-editor, operator or acme")
+	create.Flags().StringVar(&role, "role", "", "restrict the token to a role: readonly, dns-editor, operator, acme or certificates")
 	create.Flags().StringArrayVar(&grants, "grant", nil, "restrict the token to these grants (repeatable)")
 	create.Flags().StringVar(&expires, "expires", "", "lifetime, e.g. 90d or 720h (default: never)")
 	create.Flags().BoolVar(&save, "save", false, "store the new token in the CLI configuration file")

cmd/jarl/certget.go

@@ -2,7 +2,8 @@ package main
 
 // jarl cert get: certificates for other hosts (docs/certificates.md#certificates-for-other-hosts).
 // jarld obtains and renews the certificate; this host asks for it, waits,
-// downloads it when it is newer than its copy, and runs a hook. The
+// downloads it when it is newer than its copy, writes the combined file
+// (--combined) and runs a hook. The
 // settings of each certificate are saved, so a daily timer (jarl cert
 // schedule) can run 'jarl cert get' for all of them.
 
@@ -36,6 +37,9 @@ type savedCert struct {
 	KeyType string   `json:"key_type,omitempty"`
 	Dir     string   `json:"dir"`
 	Hook    string   `json:"hook,omitempty"`
+	// Combined is a file with the key and the full chain together, for
+	// HAProxy (--combined); kept outside Dir.
+	Combined string     `json:"combined,omitempty"`
 	Conn     connection `json:"connection"`
 	SavedAt  time.Time  `json:"saved_at"`
 }
@@ -48,6 +52,9 @@ type localMeta struct {
 	Serial   string    `json:"serial"`
 	NotAfter time.Time `json:"not_after"`
 	Fetched  time.Time `json:"fetched"`
+	// Combined is the combined file written for these files, so a changed
+	// --combined removes the old one.
+	Combined string `json:"combined,omitempty"`
 	// HookFailed is set when the hook failed for these files: the next
 	// run runs it again.
 	HookFailed string `json:"hook_failed,omitempty"`
@@ -114,6 +121,73 @@ func (sc savedCert) save(dir string) error {
 	return writeAtomic(filepath.Join(dir, sc.Name+".json"), append(b, '\n'), 0o600)
 }
 
+// checkCertClash refuses a certificate that would take over what this
+// host keeps for another one: a name saved for another Jarl server
+// (each server has its own names, so the server can't refuse it), or a
+// directory with another certificate's files. saved[i] is sc's saved
+// settings (i < 0: none).
+func checkCertClash(saved []savedCert, i int, sc savedCert, replace bool) error {
+	if i >= 0 && !replace && !sameServer(saved[i].Conn.Server, sc.Conn.Server) {
+		return fmt.Errorf("%s is saved here for the Jarl server %s, not %s: choose another name, or take it over with --replace", sc.Name, saved[i].Conn.Server, sc.Conn.Server)
+	}
+	for j, o := range saved {
+		if j != i && o.Name != sc.Name && filepath.Clean(o.Dir) == filepath.Clean(sc.Dir) {
+			return fmt.Errorf("%s holds the files of %s, saved for %s: give %s another --dir", sc.Dir, o.Name, o.Conn.Server, sc.Name)
+		}
+	}
+	if err := checkCombined(saved, i, sc); err != nil {
+		return err
+	}
+	if m, err := readMeta(sc.Dir); err == nil && m.Name != "" && m.Name != sc.Name && !replace {
+		return fmt.Errorf("%s holds the files of the certificate %s: give %s another --dir, or overwrite them with --replace", sc.Dir, m.Name, sc.Name)
+	}
+	return nil
+}
+
+// checkCombined refuses a --combined path that is a directory, one of
+// the files jarl cert get writes in a certificate's directory, or another
+// saved certificate's combined file.
+func checkCombined(saved []savedCert, i int, sc savedCert) error {
+	if sc.Combined == "" {
+		return nil
+	}
+	if st, err := os.Stat(sc.Combined); err == nil && st.IsDir() {
+		return fmt.Errorf("--combined %s is a directory: give the file's path, such as %s", sc.Combined, filepath.Join(sc.Combined, sc.Name+".pem"))
+	}
+	dir, base := filepath.Split(sc.Combined)
+	for j, o := range append(slices.Clone(saved), sc) {
+		if j == i {
+			continue // sc itself is last
+		}
+		if filepath.Clean(dir) == filepath.Clean(o.Dir) && (base == metaFile || slices.Contains(certFiles, base)) {
+			return fmt.Errorf("--combined %s would overwrite %s's %s: give another path", sc.Combined, o.Name, base)
+		}
+		if j < len(saved) && o.Name != sc.Name && o.Combined != "" && filepath.Clean(o.Combined) == filepath.Clean(sc.Combined) {
+			return fmt.Errorf("--combined %s is the combined file of %s: give %s another path", sc.Combined, o.Name, sc.Name)
+		}
+	}
+	return nil
+}
+
+// sameNames reports whether two lists hold the same names, in any order.
+func sameNames(a, b []string) bool {
+	norm := func(l []string) []string {
+		out := make([]string, len(l))
+		for i, n := range l {
+			out[i] = strings.TrimSuffix(strings.ToLower(n), ".")
+		}
+		slices.Sort(out)
+		return slices.Compact(out)
+	}
+	return slices.Equal(norm(a), norm(b))
+}
+
+// sameServer reports whether two server addresses name the same jarld.
+func sameServer(a, b string) bool {
+	norm := func(s string) string { return strings.TrimRight(strings.ToLower(s), "/") }
+	return norm(a) == norm(b)
+}
+
 func readMeta(dir string) (*localMeta, error) {
 	b, err := os.ReadFile(filepath.Join(dir, metaFile))
 	if err != nil {
@@ -165,12 +239,16 @@ type getResult struct {
 	Serial   string    `json:"serial"`
 	NotAfter time.Time `json:"not_after"`
 	Updated  bool      `json:"updated"` // new files written
+	// Combined is the combined file; CombinedWritten is set when it was
+	// written now.
+	Combined        string `json:"combined,omitempty"`
+	CombinedWritten bool   `json:"combined_written,omitempty"`
 	Hook            string `json:"hook,omitempty"`
 }
 
 // fetchCert asks jarld for the certificate, waits while it is obtained,
-// downloads it when it is newer than this host's copy (or force) and runs
-// the hook.
+// downloads it when it is newer than this host's copy (or force), and
+// then applies it (applyCert).
 func fetchCert(c *client.Client, sc savedCert, force bool, wait time.Duration) (*getResult, error) {
 	var ans requestAnswer
 	err := c.Do(ctx(), "POST", certPath(sc.Name)+"/request", nil, map[string]any{"domains": sc.Domains, "key_type": sc.KeyType}, &ans)
@@ -215,23 +293,68 @@ func fetchCert(c *client.Client, sc savedCert, force bool, wait time.Duration) (
 	if ci.LastError != "" && ci.LastAttempt != nil && (ci.LastSuccess == nil || ci.LastAttempt.After(*ci.LastSuccess)) {
 		fmt.Fprintf(os.Stderr, "%s: jarld's last attempt failed (%s); this is the certificate it has\n", sc.Name, ci.LastError)
 	}
-	res := &getResult{Name: sc.Name, Dir: sc.Dir, Serial: ci.Serial, NotAfter: *ci.NotAfter}
 	meta, _ := readMeta(sc.Dir)
 	have := meta != nil && meta.Serial == ci.Serial && filesPresent(sc.Dir)
-	if have && !force && meta.HookFailed == "" {
-		return res, nil
-	}
 	renewed := meta != nil && meta.Serial != "" && meta.Serial != ci.Serial
-	if !have || force {
+	updated := !have || force
+	if updated {
 		if err := downloadCert(c, sc, ci); err != nil {
 			return nil, err
 		}
-		res.Updated = true
-		meta = &localMeta{Name: sc.Name, Domains: ci.Domains, Serial: ci.Serial, NotAfter: *ci.NotAfter, Fetched: time.Now().UTC()}
+		old := ""
+		if meta != nil {
+			old = meta.Combined
+		}
+		meta = &localMeta{Name: sc.Name, Domains: ci.Domains, Serial: ci.Serial, NotAfter: *ci.NotAfter, Fetched: time.Now().UTC(), Combined: old}
+	}
+	return applyCert(sc, meta, updated, renewed, force)
+}
+
+// localCert applies the files this host has, without asking jarld
+// (--local): it writes the combined file and runs the hook.
+func localCert(sc savedCert) (*getResult, error) {
+	meta, err := readMeta(sc.Dir)
+	if err != nil || !filesPresent(sc.Dir) {
+		return nil, fmt.Errorf("%s: this host has no files for it in %s yet: run jarl cert get %s without --local", sc.Name, sc.Dir, sc.Name)
+	}
+	if meta.Name != "" && meta.Name != sc.Name {
+		return nil, fmt.Errorf("%s holds the files of the certificate %s, not %s", sc.Dir, meta.Name, sc.Name)
+	}
+	return applyCert(sc, meta, false, false, true)
+}
+
+// applyCert brings what lies outside the certificate's directory in line
+// with its files: it writes the combined file when the files are new, its
+// path changed or it is missing (and removes one written for an older
+// path), then runs the hook when anything changed, when it failed last
+// time, or always.
+func applyCert(sc savedCert, meta *localMeta, updated, renewed, always bool) (*getResult, error) {
+	res := &getResult{Name: sc.Name, Dir: sc.Dir, Serial: meta.Serial, NotAfter: meta.NotAfter, Updated: updated, Combined: sc.Combined}
+	changed := updated
+	if meta.Combined != "" && filepath.Clean(meta.Combined) != filepath.Clean(sc.Combined) {
+		if err := os.Remove(meta.Combined); err != nil && !errors.Is(err, os.ErrNotExist) {
+			return nil, fmt.Errorf("%s: removing the old combined file: %w", sc.Name, err)
+		}
+		meta.Combined = ""
+		changed = true
+	}
+	if sc.Combined != "" {
+		_, err := os.Stat(sc.Combined)
+		if updated || always || meta.Combined == "" || err != nil {
+			if err := writeCombined(sc); err != nil {
+				return nil, err
+			}
+			meta.Combined = sc.Combined
+			res.CombinedWritten = true
+			changed = true
+		}
+	}
+	if !changed && !always && meta.HookFailed == "" {
+		return res, nil
 	}
 	if sc.Hook != "" {
 		res.Hook = "ok"
-		if err := runHook(sc, ci, renewed); err != nil {
+		if err := runHook(sc, meta, renewed); err != nil {
 			meta.HookFailed = err.Error()
 			res.Hook = "failed: " + err.Error()
 		} else {
@@ -249,6 +372,31 @@ func fetchCert(c *client.Client, sc savedCert, force bool, wait time.Duration) (
 
 var certFiles = []string{"cert.pem", "chain.pem", "fullchain.pem", "privkey.pem"}
 
+// writeCombined writes the combined file from the certificate's files:
+// the key, then the full chain (as jarl cert export --part bundle), mode
+// 600, all at once.
+func writeCombined(sc savedCert) error {
+	full, err := os.ReadFile(filepath.Join(sc.Dir, "fullchain.pem"))
+	if err != nil {
+		return err
+	}
+	key, err := os.ReadFile(filepath.Join(sc.Dir, "privkey.pem"))
+	if err != nil {
+		return err
+	}
+	if _, err := tls.X509KeyPair(full, key); err != nil {
+		return fmt.Errorf("%s: the files in %s don't match (%v): run jarl cert get %s --force", sc.Name, sc.Dir, err, sc.Name)
+	}
+	if err := os.MkdirAll(filepath.Dir(sc.Combined), 0o755); err != nil {
+		return err
+	}
+	data := append(append(append([]byte{}, key...), '\n'), full...)
+	if err := writeAtomic(sc.Combined, data, 0o600); err != nil {
+		return fmt.Errorf("%s: writing the combined file: %w", sc.Name, err)
+	}
+	return nil
+}
+
 func filesPresent(dir string) bool {
 	for _, f := range certFiles {
 		if _, err := os.Stat(filepath.Join(dir, f)); err != nil {
@@ -313,7 +461,7 @@ func downloadCert(c *client.Client, sc savedCert, ci certInfo) error {
 
 // runHook runs the certificate's hook with sh, its files in the
 // environment.
-func runHook(sc savedCert, ci certInfo, renewed bool) error {
+func runHook(sc savedCert, meta *localMeta, renewed bool) error {
 	c, cancel := context.WithTimeout(context.Background(), 5*time.Minute)
 	defer cancel()
 	cmd := exec.CommandContext(c, "sh", "-c", sc.Hook)
@@ -328,9 +476,10 @@ func runHook(sc savedCert, ci certInfo, renewed bool) error {
 		"JARL_CERT_CHAIN="+filepath.Join(sc.Dir, "chain.pem"),
 		"JARL_CERT_FULLCHAIN="+filepath.Join(sc.Dir, "fullchain.pem"),
 		"JARL_CERT_KEY="+filepath.Join(sc.Dir, "privkey.pem"),
-		"JARL_CERT_DOMAINS="+strings.Join(ci.Domains, " "),
-		"JARL_CERT_SERIAL="+ci.Serial,
-		"JARL_CERT_NOT_AFTER="+ci.NotAfter.UTC().Format(time.RFC3339),
+		"JARL_CERT_COMBINED="+sc.Combined,
+		"JARL_CERT_DOMAINS="+strings.Join(meta.Domains, " "),
+		"JARL_CERT_SERIAL="+meta.Serial,
+		"JARL_CERT_NOT_AFTER="+meta.NotAfter.UTC().Format(time.RFC3339),
 		"JARL_CERT_RENEWED="+ren,
 	)
 	cmd.Stdout, cmd.Stderr = os.Stderr, os.Stderr
@@ -339,8 +488,8 @@ func runHook(sc savedCert, ci certInfo, renewed bool) error {
 
 func certGetCmd() *cobra.Command {
 	var domains []string
-	var dir, hook, keyType, configDir string
-	var noSave, force bool
+	var dir, hook, keyType, combined, configDir string
+	var noSave, force, replace, local bool
 	var wait time.Duration
 	cmd := &cobra.Command{
 		Use:   "get [NAME]",
@@ -348,7 +497,7 @@ func certGetCmd() *cobra.Command {
 		Long: `Gets a certificate for this host (a mail server, a reverse proxy). jarld
 obtains it from the CA and renews it, answering the challenges in its own
 zones; this command asks for it, waits until it is there, downloads it
-when it is newer than this host's copy, and runs --hook.
+when it is newer than this host's copy, writes --combined and runs --hook.
 
 The token needs a certificates grant for the names, and the certificate
 belongs to the token's user: another user can't take the name. Asking
@@ -357,22 +506,40 @@ removing a name gets a new one.
 
 Files in --dir (default /etc/jarl/certs/NAME as root, ./NAME otherwise):
 cert.pem, chain.pem (the intermediates), fullchain.pem, privkey.pem (mode
-600), and jarl-cert.json (serial, expiry, when it was fetched).
+600), and jarl-cert.json (serial, expiry, when it was fetched). Don't
+point HAProxy's crt at --dir: it loads every file there.
 
-The settings (names, --dir, --hook, the server and token) are saved in
+--combined PATH also writes the key and the full chain in one file (mode
+600), as HAProxy wants. Keep it outside --dir; a directory of combined
+files can be HAProxy's crt directory. --combined '' stops writing it and
+removes the file.
+
+The settings (names, --dir, --combined, --hook, the server and token) are
+saved in
 /etc/jarl/cert-get/NAME.json (as root; ~/.config/jarl/cert-get otherwise,
-readable only by you). 'jarl cert get' without a name then does every
+readable only by you). To change one later, run 'jarl cert get NAME' with
+just that option; the others stay. 'jarl cert get' without a name does every
 saved certificate: run it daily (jarl cert schedule does that with a
 systemd timer). Nothing is downloaded until jarld has renewed one.
 
---hook runs with sh after new files are written (and again on the next
-run if it failed), with JARL_CERT_NAME, JARL_CERT_DIR, JARL_CERT_CERT,
-JARL_CERT_CHAIN, JARL_CERT_FULLCHAIN, JARL_CERT_KEY, JARL_CERT_DOMAINS,
-JARL_CERT_SERIAL, JARL_CERT_NOT_AFTER and JARL_CERT_RENEWED (1 when it
-replaced an older certificate).`,
+A name saved here for another Jarl server isn't taken over, nor a --dir
+holding another certificate's files, unless --replace is given; a --dir
+another saved certificate uses is always refused.
+
+--hook runs with sh after new files are written or the combined file
+changes (and again on the next run if it failed), with JARL_CERT_NAME,
+JARL_CERT_DIR, JARL_CERT_CERT, JARL_CERT_CHAIN, JARL_CERT_FULLCHAIN,
+JARL_CERT_KEY, JARL_CERT_COMBINED (empty without --combined),
+JARL_CERT_DOMAINS, JARL_CERT_SERIAL, JARL_CERT_NOT_AFTER and
+JARL_CERT_RENEWED (1 when it replaced an older certificate).
+
+--local doesn't ask jarld: it uses the files this host has, writes the
+combined file and runs the hook. Use it after changing --combined or
+--hook, or to test a hook.`,
 		Example: `  jarl cert get mail -d mail.adbraun.lol --hook 'systemctl reload postfix dovecot'
-  jarl cert get proxy -d adbraun.lol -d '*.adbraun.lol' --dir /etc/haproxy/certs/proxy \
-      --hook 'cat "$JARL_CERT_KEY" "$JARL_CERT_FULLCHAIN" > /etc/haproxy/proxy.pem && systemctl reload haproxy'
+  jarl cert get proxy -d adbraun.lol -d '*.adbraun.lol' \
+      --combined /etc/haproxy/certs/adbraun.lol.pem --hook 'systemctl reload haproxy'
+  jarl cert get proxy --hook 'systemctl reload haproxy' --local   # change a setting, apply it now
   jarl cert get                      # every saved certificate (what the timer runs)
   jarl cert saved                    # what this host has, and when it expires`,
 		Args: cobra.MaximumNArgs(1),
@@ -384,6 +551,9 @@ replaced an older certificate).`,
 			}
 			var todo []savedCert
 			f := cmd.Flags()
+			if local && force {
+				return errors.New("--local and --force don't go together: --local always writes the combined file and runs the hook; --force downloads again from jarld")
+			}
 			switch {
 			case len(args) == 0:
 				if len(domains) > 0 {
@@ -392,6 +562,11 @@ replaced an older certificate).`,
 				if len(saved) == 0 {
 					return fmt.Errorf("no saved certificates in %s: get one first (jarl cert get NAME -d DOMAIN)", cdir)
 				}
+				for _, n := range []string{"dir", "hook", "key-type", "combined", "replace", "no-save"} {
+					if f.Changed(n) {
+						return fmt.Errorf("--%s changes one certificate: give its name (jarl cert get NAME --%s ...)", n, n)
+					}
+				}
 				todo = saved
 			default:
 				name := args[0]
@@ -404,6 +579,9 @@ replaced an older certificate).`,
 				}
 				sc.Name = name
 				if len(domains) > 0 {
+					if local && i >= 0 && !sameNames(domains, sc.Domains) {
+						return fmt.Errorf("%s: new names need a new certificate from jarld: run jarl cert get %s without --local", name, name)
+					}
 					sc.Domains = domains
 				}
 				if f.Changed("dir") || sc.Dir == "" {
@@ -415,6 +593,16 @@ replaced an older certificate).`,
 				if f.Changed("key-type") {
 					sc.KeyType = keyType
 				}
+				if f.Changed("combined") {
+					sc.Combined = ""
+					if combined != "" {
+						abs, err := filepath.Abs(combined)
+						if err != nil {
+							return err
+						}
+						sc.Combined = abs
+					}
+				}
 				if abs, err := filepath.Abs(sc.Dir); err == nil {
 					sc.Dir = abs
 				}
@@ -427,20 +615,28 @@ replaced an older certificate).`,
 					}
 					sc.Conn = cn
 				}
+				if err := checkCertClash(saved, i, sc, replace); err != nil {
+					return err
+				}
 				todo = []savedCert{sc}
 			}
 			var results []*getResult
 			var failed []string
 			var lastErr error
 			for _, sc := range todo {
-				c, err := sc.Conn.open()
-				if err == nil {
 				var r *getResult
+				var err error
+				if local {
+					r, err = localCert(sc)
+				} else {
+					var c *client.Client
+					if c, err = sc.Conn.open(); err == nil {
 						r, err = fetchCert(c, sc, force, wait)
+					}
+				}
 				if r != nil {
 					results = append(results, r)
 				}
-				}
 				if err != nil {
 					if len(todo) > 1 {
 						fmt.Fprintln(os.Stderr, "error:", err)
@@ -459,8 +655,16 @@ replaced an older certificate).`,
 			if err := output(results, func(w io.Writer) {
 				for _, r := range results {
 					what := "up to date"
-					if r.Updated {
+					switch {
+					case r.Updated:
 						what = "new files"
+					case local:
+						what = "this host's files"
+					case r.CombinedWritten:
+						what = "combined file written"
+					}
+					if r.Combined != "" {
+						what += ", combined " + r.Combined
 					}
 					fmt.Fprintf(w, "%s\t%s\t%s, expires %s\t%s\n", r.Name, r.Dir, what, r.NotAfter.Local().Format("2006-01-02"), hookText(r.Hook))
 				}
@@ -469,7 +673,7 @@ replaced an older certificate).`,
 			}
 			switch {
 			case len(failed) == 1 && len(todo) == 1:
-				return lastErr // its own exit code (3 forbidden, 6 conflict, ...)
+				return lastErr // its own exit code (3 forbidden, 5 conflict, ...)
 			case len(failed) > 0:
 				return fmt.Errorf("failed: %s", strings.Join(failed, ", "))
 			}
@@ -479,10 +683,13 @@ replaced an older certificate).`,
 	f := cmd.Flags()
 	f.StringArrayVarP(&domains, "domain", "d", nil, "a name for the certificate (repeatable; a wildcard as '*.adbraun.lol')")
 	f.StringVar(&dir, "dir", "", "directory for the files (default /etc/jarl/certs/NAME as root, ./NAME otherwise)")
-	f.StringVar(&hook, "hook", "", "shell command to run after new files are written")
+	f.StringVar(&hook, "hook", "", "shell command to run after new files are written ('' removes it)")
+	f.StringVar(&combined, "combined", "", "also write the key and full chain in one file at this path, for HAProxy ('' stops it)")
+	f.BoolVar(&local, "local", false, "don't ask jarld: write the combined file and run the hook from this host's files")
 	f.StringVar(&keyType, "key-type", "", "key type of a new certificate: ec256 (default), ec384, rsa2048, rsa3072, rsa4096")
 	f.BoolVar(&force, "force", false, "download and run the hook even when this host has the current certificate")
 	f.BoolVar(&noSave, "no-save", false, "don't save the settings for later runs")
+	f.BoolVar(&replace, "replace", false, "take over a name saved here for another Jarl server, or a --dir holding another certificate's files")
 	f.DurationVar(&wait, "wait", 15*time.Minute, "how long to wait while jarld obtains it")
 	f.StringVar(&configDir, "config-dir", "", "where the settings are saved (default /etc/jarl/cert-get as root, ~/.config/jarl/cert-get otherwise)")
 	_ = cmd.RegisterFlagCompletionFunc("key-type", cobra.FixedCompletions([]string{"ec256", "ec384", "rsa2048", "rsa3072", "rsa4096"}, cobra.ShellCompDirectiveNoFileComp))
@@ -499,9 +706,15 @@ func hookText(h string) string {
 func certSavedCmd() *cobra.Command {
 	var configDir string
 	cmd := &cobra.Command{
-		Use:   "saved",
+		Use:   "saved [NAME]",
 		Short: "The certificates this host gets with jarl cert get, and when they expire",
-		Args:  cobra.NoArgs,
+		Long: `Lists the certificates saved on this host by jarl cert get. With a name,
+shows all of that certificate's settings and what this host has of it.
+The token is never shown.`,
+		Example: `  jarl cert saved
+  jarl cert saved proxy
+  jarl cert saved -o json`,
+		Args: cobra.MaximumNArgs(1),
 		RunE: func(cmd *cobra.Command, args []string) error {
 			cdir := certConfigDir(configDir)
 			saved, err := loadSaved(cdir)
@@ -514,17 +727,27 @@ func certSavedCmd() *cobra.Command {
 			}
 			var rows []row
 			for _, sc := range saved {
+				if len(args) == 1 && sc.Name != args[0] {
+					continue
+				}
 				sc.Conn.Token = "" // never print it
 				m, _ := readMeta(sc.Dir)
 				rows = append(rows, row{sc, m})
 			}
+			if len(args) == 1 {
+				if len(rows) == 0 {
+					return fmt.Errorf("%s isn't saved in %s (jarl cert saved lists what is)", args[0], cdir)
+				}
+				r := rows[0]
+				return output(r, func(w io.Writer) { printSaved(w, r.savedCert, r.Local) })
+			}
 			return output(rows, func(w io.Writer) {
 				if len(rows) == 0 {
 					fmt.Fprintf(w, "No saved certificates in %s.\n", cdir)
 					return
 				}
 				tw := tabwriter.NewWriter(w, 0, 0, 2, ' ', 0)
-				fmt.Fprintln(tw, "NAME\tNAMES\tDIR\tEXPIRES\tFETCHED\tSERVER\tHOOK")
+				fmt.Fprintln(tw, "NAME\tNAMES\tDIR\tCOMBINED\tEXPIRES\tFETCHED\tSERVER\tHOOK")
 				for _, r := range rows {
 					exp, fetched := "-", "-"
 					if r.Local != nil {
@@ -534,7 +757,7 @@ func certSavedCmd() *cobra.Command {
 							fetched += " HOOK FAILED"
 						}
 					}
-					fmt.Fprintf(tw, "%s\t%s\t%s\t%s\t%s\t%s\t%s\n", r.Name, strings.Join(r.Domains, ","), r.Dir, exp, fetched, r.Conn.Server, firstNonEmpty(r.Hook, "-"))
+					fmt.Fprintf(tw, "%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n", r.Name, strings.Join(r.Domains, ","), r.Dir, dash(r.Combined), exp, fetched, r.Conn.Server, firstNonEmpty(r.Hook, "-"))
 				}
 				tw.Flush()
 			})
@@ -544,6 +767,37 @@ func certSavedCmd() *cobra.Command {
 	return cmd
 }
 
+// printSaved shows one saved certificate: its settings, then this
+// host's copy.
+func printSaved(w io.Writer, sc savedCert, m *localMeta) {
+	fmt.Fprintf(w, "Name:\t%s\n", sc.Name)
+	fmt.Fprintf(w, "Names:\t%s\n", strings.Join(sc.Domains, " "))
+	fmt.Fprintf(w, "Key type:\t%s\n", firstNonEmpty(sc.KeyType, "ec256 (default)"))
+	fmt.Fprintf(w, "Dir:\t%s\n", sc.Dir)
+	fmt.Fprintf(w, "Combined:\t%s\n", dash(sc.Combined))
+	fmt.Fprintf(w, "Hook:\t%s\n", dash(sc.Hook))
+	fmt.Fprintf(w, "Server:\t%s\n", sc.Conn.Server)
+	if sc.Conn.CACert != "" {
+		fmt.Fprintf(w, "CA cert:\t%s\n", sc.Conn.CACert)
+	}
+	if sc.Conn.Insecure {
+		fmt.Fprintf(w, "TLS:\tnot verified (--insecure)\n")
+	}
+	if !sc.SavedAt.IsZero() {
+		fmt.Fprintf(w, "Saved:\t%s\n", fmtTime(&sc.SavedAt))
+	}
+	if m == nil {
+		fmt.Fprintf(w, "Files:\tnone yet (jarl cert get %s fetches them)\n", sc.Name)
+		return
+	}
+	fmt.Fprintf(w, "Serial:\t%s\n", m.Serial)
+	fmt.Fprintf(w, "Expires:\t%s (%dd)\n", fmtTime(&m.NotAfter), int(time.Until(m.NotAfter).Hours()/24))
+	fmt.Fprintf(w, "Fetched:\t%s\n", fmtTime(&m.Fetched))
+	if m.HookFailed != "" {
+		fmt.Fprintf(w, "Hook failed:\t%s (the next jarl cert get runs it again)\n", m.HookFailed)
+	}
+}
+
 func certForgetCmd() *cobra.Command {
 	var configDir string
 	var del, yes bool

cmd/jarl/certget_test.go

@@ -0,0 +1,203 @@
+package main
+
+import (
+	"crypto/ecdsa"
+	"crypto/elliptic"
+	"crypto/rand"
+	"crypto/tls"
+	"crypto/x509"
+	"crypto/x509/pkix"
+	"encoding/pem"
+	"math/big"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+	"time"
+)
+
+func TestCheckCertClash(t *testing.T) {
+	tmp := t.TempDir()
+	a := connection{Server: "https://dns-a.example:8053"}
+	b := connection{Server: "https://dns-b.example:8053"}
+	saved := []savedCert{
+		{Name: "proxy", Dir: filepath.Join(tmp, "proxy"), Conn: a},
+		{Name: "mail", Dir: filepath.Join(tmp, "mail"), Conn: a},
+	}
+	// mail's files are on disk; "other" has files but no saved settings.
+	if err := (&localMeta{Name: "mail"}).write(mkdir(t, saved[1].Dir)); err != nil {
+		t.Fatal(err)
+	}
+	if err := (&localMeta{Name: "other"}).write(mkdir(t, filepath.Join(tmp, "other"))); err != nil {
+		t.Fatal(err)
+	}
+	cases := []struct {
+		what    string
+		i       int
+		sc      savedCert
+		replace bool
+		want    string // "" = allowed
+	}{
+		{"same server again", 0, savedCert{Name: "proxy", Dir: saved[0].Dir, Conn: a}, false, ""},
+		{"same server, other spelling", 0, savedCert{Name: "proxy", Dir: saved[0].Dir, Conn: connection{Server: "HTTPS://dns-a.example:8053/"}}, false, ""},
+		{"other server", 0, savedCert{Name: "proxy", Dir: saved[0].Dir, Conn: b}, false, "saved here for the Jarl server"},
+		{"other server, replace", 0, savedCert{Name: "proxy", Dir: saved[0].Dir, Conn: b}, true, ""},
+		{"new name, saved cert's dir", -1, savedCert{Name: "web", Dir: saved[1].Dir, Conn: b}, false, "holds the files of mail"},
+		{"new name, saved cert's dir, replace", -1, savedCert{Name: "web", Dir: saved[1].Dir + "/", Conn: b}, true, "holds the files of mail"},
+		{"new name, unsaved cert's files", -1, savedCert{Name: "web", Dir: filepath.Join(tmp, "other"), Conn: b}, false, "certificate other"},
+		{"new name, unsaved cert's files, replace", -1, savedCert{Name: "web", Dir: filepath.Join(tmp, "other"), Conn: b}, true, ""},
+		{"new name, empty dir", -1, savedCert{Name: "web", Dir: filepath.Join(tmp, "web"), Conn: b}, false, ""},
+	}
+	for _, c := range cases {
+		err := checkCertClash(saved, c.i, c.sc, c.replace)
+		switch {
+		case c.want == "" && err != nil:
+			t.Errorf("%s: %v", c.what, err)
+		case c.want != "" && (err == nil || !strings.Contains(err.Error(), c.want)):
+			t.Errorf("%s: got %v, want %q", c.what, err, c.want)
+		}
+	}
+}
+
+func mkdir(t *testing.T, dir string) string {
+	t.Helper()
+	if err := os.MkdirAll(dir, 0o755); err != nil {
+		t.Fatal(err)
+	}
+	return dir
+}
+
+func TestCheckCombined(t *testing.T) {
+	tmp := t.TempDir()
+	saved := []savedCert{
+		{Name: "proxy", Dir: filepath.Join(tmp, "proxy"), Combined: filepath.Join(tmp, "haproxy", "proxy.pem")},
+		{Name: "mail", Dir: filepath.Join(tmp, "mail")},
+	}
+	mkdir(t, filepath.Join(tmp, "haproxy"))
+	cases := []struct {
+		what string
+		i    int
+		sc   savedCert
+		want string
+	}{
+		{"none", 1, savedCert{Name: "mail", Dir: saved[1].Dir}, ""},
+		{"own again", 0, saved[0], ""},
+		{"beside the others", 1, savedCert{Name: "mail", Dir: saved[1].Dir, Combined: filepath.Join(tmp, "haproxy", "mail.pem")}, ""},
+		{"inside its own dir", 1, savedCert{Name: "mail", Dir: saved[1].Dir, Combined: filepath.Join(saved[1].Dir, "combined.pem")}, ""},
+		{"another's combined", 1, savedCert{Name: "mail", Dir: saved[1].Dir, Combined: saved[0].Combined}, "combined file of proxy"},
+		{"own fullchain", 1, savedCert{Name: "mail", Dir: saved[1].Dir, Combined: filepath.Join(saved[1].Dir, "fullchain.pem")}, "overwrite mail's fullchain.pem"},
+		{"another's key", 1, savedCert{Name: "mail", Dir: saved[1].Dir, Combined: filepath.Join(saved[0].Dir, "privkey.pem")}, "overwrite proxy's privkey.pem"},
+		{"a directory", 1, savedCert{Name: "mail", Dir: saved[1].Dir, Combined: filepath.Join(tmp, "haproxy")}, "is a directory"},
+	}
+	for _, c := range cases {
+		err := checkCombined(saved, c.i, c.sc)
+		switch {
+		case c.want == "" && err != nil:
+			t.Errorf("%s: %v", c.what, err)
+		case c.want != "" && (err == nil || !strings.Contains(err.Error(), c.want)):
+			t.Errorf("%s: got %v, want %q", c.what, err, c.want)
+		}
+	}
+}
+
+// TestApplyCert: the combined file follows --combined, and the hook runs
+// only when something changed (always with --local).
+func TestApplyCert(t *testing.T) {
+	tmp := t.TempDir()
+	dir := mkdir(t, filepath.Join(tmp, "proxy"))
+	full, key := testKeyPair(t)
+	for name, data := range map[string][]byte{"cert.pem": full, "chain.pem": nil, "fullchain.pem": full, "privkey.pem": key} {
+		if err := os.WriteFile(filepath.Join(dir, name), data, 0o600); err != nil {
+			t.Fatal(err)
+		}
+	}
+	if err := (&localMeta{Name: "proxy", Serial: "1", NotAfter: time.Now().Add(time.Hour)}).write(dir); err != nil {
+		t.Fatal(err)
+	}
+	runs := filepath.Join(tmp, "runs")
+	sc := savedCert{Name: "proxy", Dir: dir, Hook: `echo "$JARL_CERT_COMBINED" >> ` + runs}
+	hookRuns := func() []string {
+		b, _ := os.ReadFile(runs)
+		return strings.Fields(string(b))
+	}
+	apply := func(local bool) *getResult {
+		t.Helper()
+		var r *getResult
+		var err error
+		if local {
+			r, err = localCert(sc)
+		} else {
+			meta, _ := readMeta(dir)
+			r, err = applyCert(sc, meta, false, false, false)
+		}
+		if err != nil {
+			t.Fatal(err)
+		}
+		return r
+	}
+
+	if r := apply(false); r.CombinedWritten || len(hookRuns()) != 0 {
+		t.Fatalf("nothing changed, but: %+v, hook ran %d times", r, len(hookRuns()))
+	}
+	first := filepath.Join(tmp, "haproxy", "proxy.pem")
+	sc.Combined = first
+	if r := apply(false); !r.CombinedWritten || len(hookRuns()) != 1 {
+		t.Fatalf("new --combined: %+v, hook ran %d times", r, len(hookRuns()))
+	}
+	b, err := os.ReadFile(first)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if _, err := tls.X509KeyPair(b, b); err != nil || !strings.HasPrefix(string(b), string(key)) {
+		t.Errorf("combined file isn't the key, then the chain: %v", err)
+	}
+	if st, _ := os.Stat(first); st.Mode().Perm() != 0o600 {
+		t.Errorf("combined file mode %v, want 600", st.Mode().Perm())
+	}
+	if r := apply(false); r.CombinedWritten || len(hookRuns()) != 1 {
+		t.Fatalf("second run changed something: %+v, hook ran %d times", r, len(hookRuns()))
+	}
+	os.Remove(first)
+	if r := apply(false); !r.CombinedWritten || len(hookRuns()) != 2 {
+		t.Fatalf("missing combined file not rewritten: %+v", r)
+	}
+	if r := apply(true); !r.CombinedWritten || len(hookRuns()) != 3 {
+		t.Fatalf("--local: %+v, hook ran %d times", r, len(hookRuns()))
+	}
+	second := filepath.Join(tmp, "haproxy", "adbraun.lol.pem")
+	sc.Combined = second
+	apply(false)
+	if _, err := os.Stat(first); !os.IsNotExist(err) {
+		t.Errorf("old combined file %s not removed: %v", first, err)
+	}
+	if _, err := os.Stat(second); err != nil {
+		t.Errorf("new combined file: %v", err)
+	}
+	sc.Combined = ""
+	apply(false)
+	if _, err := os.Stat(second); !os.IsNotExist(err) {
+		t.Errorf("--combined '' left %s: %v", second, err)
+	}
+	// Five runs; the last had JARL_CERT_COMBINED empty, so echo wrote no field.
+	if got := hookRuns(); len(got) != 4 || got[3] != second {
+		t.Errorf("hook runs: %q", got)
+	}
+}
+
+func testKeyPair(t *testing.T) (certPEM, keyPEM []byte) {
+	t.Helper()
+	k, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
+	if err != nil {
+		t.Fatal(err)
+	}
+	tpl := &x509.Certificate{SerialNumber: big.NewInt(1), Subject: pkix.Name{CommonName: "proxy.example"}, NotBefore: time.Now(), NotAfter: time.Now().Add(time.Hour)}
+	der, err := x509.CreateCertificate(rand.Reader, tpl, tpl, &k.PublicKey, k)
+	if err != nil {
+		t.Fatal(err)
+	}
+	kd, err := x509.MarshalECPrivateKey(k)
+	if err != nil {
+		t.Fatal(err)
+	}
+	return pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: der}), pem.EncodeToMemory(&pem.Block{Type: "EC PRIVATE KEY", Bytes: kd})
+}

cmd/jarl/certs.go

@@ -50,6 +50,7 @@ type certInfo struct {
 		Chain string `json:"chain"`
 		Key   string `json:"key"`
 	} `json:"files,omitempty"`
+	OwnerID  int64  `json:"owner_id,omitempty"`
 	Owner    string `json:"owner,omitempty"`
 	PulledBy []struct {
 		Who    string    `json:"who"`
@@ -131,7 +132,7 @@ func certListCmd() *cobra.Command {
 				return err
 			}
 			return output(r.Certificates, func(w io.Writer) {
-				fmt.Fprintln(w, "NAME\tSOURCE\tUSES\tNAMES\tEXPIRES\tSTATUS")
+				fmt.Fprintln(w, "NAME\tSOURCE\tUSES\tNAMES\tEXPIRES\tPULLED\tSTATUS")
 				for _, ci := range r.Certificates {
 					names := ci.Names
 					if len(names) == 0 {
@@ -141,13 +142,32 @@ func certListCmd() *cobra.Command {
 					if ci.IssuingSince != nil {
 						state = "obtaining"
 					}
-					fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\t%s\n", ci.Name, ci.Source, dash(strings.Join(ci.Uses, ",")), strings.Join(names, " "), fmtDate(ci.NotAfter), state)
+					uses := strings.Join(ci.Uses, ",")
+					if uses == "" && ci.OwnerID != 0 {
+						uses = "get" // for another host (jarl cert get)
+					}
+					fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\t%s\t%s\n", ci.Name, ci.Source, dash(uses), strings.Join(names, " "), fmtDate(ci.NotAfter), lastPull(ci), state)
 				}
 			})
 		},
 	}
 }
 
+// lastPull is when the certificate was last downloaded, marked "(old)"
+// while someone who downloaded it still has the one before its renewal.
+func lastPull(ci certInfo) string {
+	if len(ci.PulledBy) == 0 {
+		return "-"
+	}
+	s := fmtDate(&ci.PulledBy[0].At) // newest first
+	for _, p := range ci.PulledBy {
+		if p.Serial != ci.Serial {
+			return s + " (old)"
+		}
+	}
+	return s
+}
+
 func fmtDate(t *time.Time) string {
 	if t == nil {
 		return "-"

cmd/jarl/changes.go

@@ -272,9 +272,9 @@ func changesetCmd() *cobra.Command {
 				return err
 			}
 			return output(cs, func(w io.Writer) {
-				fmt.Fprintf(w, "Changeset:\t%d (%s)\nDescription:\t%s\nCreated:\t%s by %s via %s\n", cs.ID, cs.Status, cs.Description, cs.CreatedAt.Format("2006-01-02 15:04"), cs.Username, cs.Via)
+				fmt.Fprintf(w, "Changeset:\t%d (%s)\nDescription:\t%s\nCreated:\t%s by %s via %s\n", cs.ID, cs.Status, cs.Description, cs.CreatedAt.Local().Format("2006-01-02 15:04"), cs.Username, cs.Via)
 				if cs.AppliedAt != nil {
-					fmt.Fprintf(w, "Applied:\t%s by %s\n", cs.AppliedAt.Format("2006-01-02 15:04"), cs.AppliedBy)
+					fmt.Fprintf(w, "Applied:\t%s by %s\n", cs.AppliedAt.Local().Format("2006-01-02 15:04"), cs.AppliedBy)
 				}
 				fmt.Fprintln(w, "\n#\tOPERATION")
 				for _, op := range cs.Ops {

cmd/jarl/ptrfills_test.go

@@ -0,0 +1,25 @@
+package main
+
+import (
+	"bytes"
+	"strings"
+	"testing"
+
+	"jarl/internal/api"
+)
+
+func TestPrintPTRFills(t *testing.T) {
+	var b bytes.Buffer
+	printPTRFills(&b, []api.PTRFill{{Address: "10.0.10.20", Target: "nextcloud.adbraun.lol.", From: "internal"}}, 3)
+	out := b.String()
+	if !strings.Contains(out, "10.0.10.20 -> nextcloud.adbraun.lol (view internal)") || !strings.Contains(out, "and 2 more") {
+		t.Errorf("listed fills:\n%s", out)
+	}
+
+	// An older server sends the count without the list.
+	b.Reset()
+	printPTRFills(&b, nil, 1)
+	if b.Len() != 0 {
+		t.Errorf("no list from the server printed %q, want nothing", b.String())
+	}
+}

cmd/jarl/zones.go

@@ -8,6 +8,7 @@ import (
 	"net/netip"
 	"net/url"
 	"os"
+	"slices"
 	"strconv"
 	"strings"
 	"unicode/utf8"
@@ -100,6 +101,16 @@ func zoneCmd() *cobra.Command {
 					fmt.Fprintf(w, "Network:\t%s\n", z.Prefix)
 					if z.PTRMissing > 0 {
 						fmt.Fprintf(w, "Missing PTRs:\t%d (addresses the view's records use here; add them: jarl zone fill-ptr %s)\n", z.PTRMissing, args[0])
+						printPTRFills(w, z.PTRMissingList, z.PTRMissing)
+					}
+					if z.PTROtherViews > 0 {
+						what := "addresses"
+						if z.PTROtherViews == 1 {
+							what = "address"
+						}
+						fmt.Fprintf(w, "Other views:\t%d %s with A or AAAA records only in %s, so not missing here (add PTRs for them in this view too: jarl zone fill-ptr %s --other-views)\n",
+							z.PTROtherViews, what, fromViews(z.PTROtherViewsList), args[0])
+						printPTRFills(w, z.PTROtherViewsList, z.PTROtherViews)
 					}
 				}
 				if len(z.LinkedWith) > 0 {
@@ -119,7 +130,7 @@ func zoneCmd() *cobra.Command {
 	var ns, records, primaries, forwarders, opts []string
 	var contact, comment, cloneFrom, file, inView, forward, linkWith string
 	var ttl uint32
-	var fill bool
+	var fill, fillOther bool
 	create := &cobra.Command{
 		Use:   "create NAME",
 		Short: "Create a zone",
@@ -128,7 +139,8 @@ create its reverse zone: a /32 (or /128) gives a zone for that one address,
 and a network between octet boundaries, like a /22, gets one zone per /24
 (created together). Without --view, a reverse zone goes to the private
 view whose records use its addresses; --fill adds a PTR record for each
-of them (docs/reverse.md).
+of them, and --other-views also for the addresses only other views'
+records use, named from there (docs/reverse.md).
 
   jarl zone create example.com --ns ns1.example.com. --record "ns1 A 192.0.2.1"
   jarl zone create 192.168.1.0/24
@@ -169,9 +181,12 @@ of them (docs/reverse.md).
 			if linkWith != "" {
 				body["link_with"] = linkWith
 			}
-			if fill {
+			if fill || fillOther {
 				body["fill"] = true
 			}
+			if fillOther {
+				body["fill_other_views"] = true
+			}
 			var recs []map[string]any
 			for _, r := range records {
 				f := strings.Fields(r)
@@ -245,6 +260,7 @@ of them (docs/reverse.md).
 	create.Flags().StringVar(&linkWith, "link-with", "", "link with the zone of the same name in this view: copy its shared records and keep them in step (docs/views.md#linked-zones)")
 	create.Flags().StringVar(&comment, "comment", "", "comment")
 	create.Flags().BoolVar(&fill, "fill", false, "reverse zones: add PTR records for the view's A and AAAA records in the network")
+	create.Flags().BoolVar(&fillOther, "other-views", false, "reverse zones: --fill, also naming the addresses only other views' records use")
 
 	var refresh, retry, expire, minimum, serial uint32
 	var mname, policy, moveView string
@@ -1166,17 +1182,61 @@ func terminalWidth() int {
 	return width
 }
 
+// printPTRFills lists PTR records a reverse zone lacks under zone show's
+// line about them: address -> name, and the view the name is from. A server
+// older than the CLI may send the count without the list; then nothing is
+// listed, rather than "and N more" under no addresses.
+func printPTRFills(w io.Writer, fills []api.PTRFill, total int) {
+	if len(fills) == 0 {
+		return
+	}
+	for _, f := range fills {
+		from := ""
+		if f.From != "" {
+			from = " (view " + f.From + ")"
+		}
+		fmt.Fprintf(w, "\t  %s -> %s%s\n", f.Address, dnsname.Display(f.Target), from)
+	}
+	if total > len(fills) {
+		fmt.Fprintf(w, "\t  and %d more\n", total-len(fills))
+	}
+}
+
+// fromViews names the views PTR fills take their names from: "view
+// internal", "views internal and lab".
+func fromViews(fills []api.PTRFill) string {
+	var vs []string
+	for _, f := range fills {
+		if f.From != "" && !slices.Contains(vs, f.From) {
+			vs = append(vs, f.From)
+		}
+	}
+	switch len(vs) {
+	case 0:
+		return "other views"
+	case 1:
+		return "view " + vs[0]
+	}
+	return "views " + strings.Join(vs[:len(vs)-1], ", ") + " and " + vs[len(vs)-1]
+}
+
 func zoneFillPTRCmd() *cobra.Command {
 	var view string
+	var otherViews bool
 	cmd := &cobra.Command{
-		Use:   "fill-ptr ZONE [--view VIEW]",
+		Use:   "fill-ptr ZONE [--view VIEW] [--other-views]",
 		Short: "Add the PTR records a reverse zone lacks for its view's A and AAAA records",
 		Long: `Adds a PTR record to a reverse zone for every address in its network that
 the view's A and AAAA records use and that has none yet. An address with
 several names gets the mail or name server's name, else a host's rather
-than the domain's, else the shortest (docs/reverse.md).`,
+than the domain's, else the shortest (docs/reverse.md).
+
+--other-views also adds the addresses in the network that only other
+views' records use, naming them from there. That publishes those views'
+host names in this one, so it is never done unasked.`,
 		Example: `  jarl zone fill-ptr 10.0.10.0/24
-  jarl zone fill-ptr 10.0.10.in-addr.arpa --view internal`,
+  jarl zone fill-ptr 10.0.10.in-addr.arpa --view internal
+  jarl zone fill-ptr 10.0.10.0/24 --view vps --other-views`,
 		Args: cobra.ExactArgs(1),
 		RunE: func(cmd *cobra.Command, args []string) error {
 			c, err := newClient()
@@ -1184,6 +1244,9 @@ than the domain's, else the shortest (docs/reverse.md).`,
 				return err
 			}
 			path, qv := zonePath(args[0], view)
+			if otherViews {
+				qv.Set("other_views", "1")
+			}
 			resp, err := c.Mutate(ctx(), "POST", path+"/fill-ptr", qv, nil)
 			if err != nil {
 				return err
@@ -1192,6 +1255,7 @@ than the domain's, else the shortest (docs/reverse.md).`,
 		},
 	}
 	cmd.Flags().StringVar(&view, "view", "", "view of the zone")
+	cmd.Flags().BoolVar(&otherViews, "other-views", false, "also name the addresses only other views' records use")
 	return cmd
 }
 
@@ -1231,7 +1295,9 @@ again). A view can stop suggestions: jarl view update V
 --all lists every network the views' records use without a reverse zone,
 also where nothing is suggested unasked (a view with suggestions off, a
 dismissed network, a public view), to create one by hand; --view narrows
-the list to one view. Change the size as you need: a /48 suggested for
+the list to one view and adds the networks only other views' records use
+(hidden "records in VIEW"): create one with --fill --other-views to name
+them from there. Change the size as you need: a /48 suggested for
 IPv6 can be created as a /56 or /64 instead.`,
 		Example: `  jarl zone suggest-reverse
   jarl zone suggest-reverse --create

contrib/lego/jarl-lego.sh

@@ -7,12 +7,17 @@
 #
 # Install it, then:
 #   EXEC_PATH=/usr/local/bin/jarl-lego JARL_TOKEN=@/etc/jarl/lego.token \
-#     lego --email you@example.com --dns exec -d example.com -d '*.example.com' run
+#     lego run --accept-tos --email you@example.com --dns exec -d example.com -d '*.example.com'
+#   (lego 4: lego --accept-tos --email ... --dns exec -d ... run, and "renew" to renew)
 #
 # Options (environment):
 #  JARL_SERVER  Jarl server URL, e.g. https://ns1.example.com:8053 (optional if the jarl CLI is configured)
 #  JARL_TOKEN   API token, or @FILE (optional if the jarl CLI is configured)
 #  JARL_VIEW    Only write into this view (optional; default every view that has the zone)
+#               On a host inside a split DNS, lego's own propagation check then can't
+#               see the record: leave it unset, or run lego with
+#               --dns.propagation.disable-ans (lego 5; --dns.propagation-disable-ans in lego 4)
+#               (docs/acme.md, "Split DNS and lego's propagation check").
 #  JARL_BIN     Path of the jarl binary (optional; default "jarl")
 
 JARL_BIN="${JARL_BIN:-jarl}"

docs/acme.md

@@ -98,10 +98,23 @@ sudo install -m 0755 contrib/lego/jarl-lego.sh /usr/local/bin/jarl-lego   # or f
 export EXEC_PATH=/usr/local/bin/jarl-lego
 export JARL_SERVER=https://ns1.adbraun.lol:8053
 export JARL_TOKEN=@/etc/jarl/certbot.token
-lego --email you@adbraun.lol --dns exec -d adbraun.lol -d '*.adbraun.lol' run
+lego run --accept-tos --email you@adbraun.lol --dns exec -d adbraun.lol -d '*.adbraun.lol'
 ```
 
-Renew with the same variables and `lego ... renew` in place of `run`.
+That is lego 5. `lego run` also renews: run the same command daily (a
+cron job or systemd timer), and it renews once a third of the
+certificate's lifetime is left. **lego 4** takes its options before the
+command, and renews with `renew`:
+
+```sh
+lego --accept-tos --email you@adbraun.lol --dns exec -d adbraun.lol -d '*.adbraun.lol' run     # first time
+lego --email you@adbraun.lol --dns exec -d adbraun.lol -d '*.adbraun.lol' renew                # then daily
+```
+
+Upgrading from lego 4 to 5, run `lego migrate` once (with `--path` if you
+keep lego's data somewhere other than `./.lego`) before anything else:
+lego 5 stores accounts and certificates differently. The exec and httpreq
+providers, and so Jarl's side, are the same in both.
 
 * lego runs `jarl-lego present FQDN VALUE` and `jarl-lego cleanup FQDN VALUE`,
   and the script passes them to `jarl acme set|unset --exact`. FQDN is the
@@ -114,7 +127,66 @@ Renew with the same variables and `lego ... renew` in place of `run`.
 * Leave `EXEC_MODE` unset: the script needs lego's default mode, not `RAW`.
 * lego waits until the record is visible on the zone's name servers
   (`EXEC_PROPAGATION_TIMEOUT`, 60 seconds by default). Jarl answers only
-  once named serves the record, so this check passes on its first try.
+  once named serves the record, so this check passes on its first try,
+  as long as the view that answers lego's host has the record: with
+  `JARL_VIEW` on a host inside your network it doesn't. See
+  [Split DNS and lego's propagation check](#split-dns-and-legos-propagation-check).
+
+### Split DNS and lego's propagation check
+
+Before lego asks the CA to look, it checks the record itself, asking the
+zone's name servers (`ns1.adbraun.lol`) directly. With views, they answer
+lego's host from the view that matches *it*. On a host inside your
+network that is the internal view. If the challenge was written into
+another view only (`JARL_VIEW=external`, `--view external`, a token
+limited to `views=external`, or the httpreq endpoint
+`.../httpreq/view/external`), lego never sees it, and gives up after
+its propagation timeout without the CA having looked:
+
+```
+propagation: time limit exceeded: last error: authoritative nameservers:
+NS ns1.adbraun.lol.:53 returned NXDOMAIN for _acme-challenge.stats.adbraun.lol.
+```
+
+A name whose authorization is still valid from an earlier run ("authorization
+already valid; skipping challenge") isn't checked at all, so one name can
+work and the next fail with the same settings.
+
+The ways round it, best first:
+
+1. **Don't limit the view.** Leave `JARL_VIEW` unset (and give the token
+   no `views=` filter, the httpreq endpoint no `/view/...`): the record
+   goes into every view that has the zone, the internal one too, and
+   lego's check passes at once. The challenge is a random value that is
+   useless once the CA has checked it, and it is removed straight after.
+2. **Skip lego's check of the name servers.** The record is already
+   served when Jarl answers, so lego's check adds nothing:
+
+   ```sh
+   lego run --dns exec --dns.propagation.disable-ans ...    # lego 5
+   lego --dns exec --dns.propagation-disable-ans ... run    # lego 4.20 to 4.x
+   lego --dns exec --dns.disable-cp ... run                 # older lego 4
+   ```
+
+   or replace the checks with a fixed wait, which also gives secondary
+   name servers time to catch up: `--dns.propagation.wait 30s` (lego 5;
+   `--dns.propagation-wait 30s` in lego 4.20 and later). In lego 5 these
+   can also be set in the environment: `LEGO_DNS_PROPAGATION_DISABLE_ANS=true`,
+   `LEGO_DNS_PROPAGATION_WAIT=30s`.
+3. **In Traefik**, the same under `dnsChallenge` (Traefik 3.2 and later;
+   older versions have `disablePropagationCheck: true`):
+
+   ```yaml
+   dnsChallenge:
+     provider: httpreq
+     propagation:
+       disableANSChecks: true     # or delayBeforeChecks: 30s
+   ```
+4. **Run lego where the external view answers it**, such as a host outside
+   your network. Rarely the easy option.
+
+certbot (section 3) doesn't check by itself, and acme.sh (section 4)
+checks through public DNS-over-HTTPS resolvers, so neither runs into this.
 
 ## 6. lego and Traefik over HTTP (httpreq), without the jarl CLI
 
@@ -299,14 +371,20 @@ of the `acme` role.
 export HTTPREQ_ENDPOINT=https://ns1.adbraun.lol:8053/api/v1/acme/httpreq
 export HTTPREQ_USERNAME=lego
 export HTTPREQ_PASSWORD_FILE=/etc/jarl/lego.token     # or HTTPREQ_PASSWORD=jarl_...
-lego --email you@adbraun.lol --dns httpreq -d adbraun.lol -d '*.adbraun.lol' run
+lego run --accept-tos --email you@adbraun.lol --dns httpreq -d adbraun.lol -d '*.adbraun.lol'
 ```
 
+That is lego 5; in lego 4 the options go before the command (`lego
+--email ... --dns httpreq -d ... run`, and `renew` to renew). See
+[section 5](#5-lego).
+
 Leave `HTTPREQ_MODE` unset (its `RAW` mode sends a different body, which
 jarld refuses with a message saying so). If jarld's certificate doesn't come from a public
 CA, lego has to trust yours: `SSL_CERT_FILE=/path/to/your-ca.pem` (lego's
 `LEGO_CA_CERTIFICATES` covers only the ACME server). To write into one view only, put
-it in the endpoint: `.../api/v1/acme/httpreq/view/external`.
+it in the endpoint: `.../api/v1/acme/httpreq/view/external`; on a host inside
+your network, read [Split DNS and lego's propagation check](#split-dns-and-legos-propagation-check)
+first.
 
 ### 4. Traefik
 
@@ -412,4 +490,8 @@ the main zone is never modified by automation.
   `dig @ns1.adbraun.lol _acme-challenge.adbraun.lol TXT`. With views, make
   sure the record is in the view that matches the CA (it is by default)
   and that `allow-query` lets the internet query the zone.
+* lego says `propagation: time limit exceeded ... returned NXDOMAIN for
+  _acme-challenge...`: the record is in a view that doesn't answer lego's
+  host, usually `JARL_VIEW=external` on a host inside your network. See
+  [Split DNS and lego's propagation check](#split-dns-and-legos-propagation-check).
 * `jarl audit --user certbot` shows every attempt.

docs/api.md

@@ -191,6 +191,7 @@ ZoneInput:
 {
   "name": "adbraun.lol",              // or a CIDR like "192.168.1.0/24" for a reverse zone
   "fill": true,                       // reverse zones: PTR records for the view's A/AAAA records there
+  "fill_other_views": true,           // with fill: also for addresses only other views' records use
   "view": "internal",                 // or "view_id"; moving a zone = PATCH with another view
   "type": "primary",                  // secondary, stub, static-stub, forward, mirror, hint, redirect, in-view
   "in_view": "internal",              // type in-view only
@@ -212,8 +213,12 @@ ZoneInput:
 
 A reverse zone created without a view goes to the private view whose
 records use its addresses. Zones come back with `prefix` (reverse zones:
-the network) and `ptr_missing` (addresses in it that the view's A and AAAA
-records use, without a PTR record).
+the network), `ptr_missing` (addresses in it that the view's A and AAAA
+records use, without a PTR record) and `ptr_other_views` (addresses in it
+without a PTR record that only other views' records use; not missing),
+with the first 25 of each in `ptr_missing_list` and
+`ptr_other_views_list` (`address`, `name`, `target`, and `from`, the view
+the name comes from).
 
 `/serials` and `/delegation` send DNS queries from the server, and only
 when called ([operations.md](operations.md#checking-name-servers-and-the-delegation)).
@@ -372,7 +377,7 @@ and of any type, in view order (`[{"id": 289, "view": "external", "type":
 |--------|------|------|
 | POST | `/zones/{zone}/link` | `{"with": ["external"]}` (default: every view with that zone). Names and types that are the same everywhere become shared, the rest stay local |
 | POST | `/zones/{zone}/unlink` | Takes the zone out of its link; its records stay, all local |
-| POST | `/zones/{zone}/fill-ptr` | Adds the PTR records a reverse zone lacks for its view's A and AAAA records ([reverse.md](reverse.md)) |
+| POST | `/zones/{zone}/fill-ptr` | Adds the PTR records a reverse zone lacks for its view's A and AAAA records; `?other_views=1` also for the addresses only other views' records use, named from there ([reverse.md](reverse.md)) |
 | GET | `/reverse-zones/suggestions` | The reverse zones the private views lack (`?all=1`: also the ones not suggested unasked, with `hidden`: public view, suggestions off, dismissed; `?view=`: one view): `view_id`, `view`, `prefix`, `zone`, `group` (`yours`, or `public`: only if the address is yours), `why`, `fill` (PTR records it would get), `samples` |
 | POST | `/reverse-zones/dismiss` | `{"view": "internal", "prefix": "185.199.108.153/32"}` (or `"prefixes": [...]` for several): don't suggest it again (`/undismiss` brings it back). A view's `reverse_suggestions: false` (PATCH `/views/{id}`) stops its suggestions |
 | GET | `/zones/{zone}/compare` | The zone compared with the same zone in other views, from its own view ([views.md](views.md#comparing-views)): `?with=a,b` (default: all), `?name=www&type=A` (one name, the same records included), `?all=1`. Returns `base`, `views`, `linked`, `summary` (per other view: `different`, `only_base`, `only_other`, `same`), `different` and `only` (`views`, `rows`) rows (name, `fqdn`, type, per view `records`, `data`, `ids`, `ttl`, `linked`), `same`, `same_rows`. A zone named without its view is compared from the first view that has it |
@@ -574,12 +579,12 @@ Needs `stats:read`. When statistics are disabled, every endpoint but
 
 | Method | Path | Query | Returns |
 |--------|------|-------|---------|
-| GET | `/stats/status` | | `{enabled, query_log, statistics_channel, log_file, last_poll, last_log_line, log_lines_read, first_data, retention, warnings}` |
+| GET | `/stats/status` | | `{enabled, query_log, statistics_channel, log_file, last_poll, last_log_line, log_lines_read, first_data, retention, retention_seconds, warnings}`: `retention` has `minute`, `hour`, `day`, `queries` as `"7d"`, `retention_seconds` the same in seconds (`queries` 0 without a query log) |
 | GET | `/stats/summary` | `since` (90m, 24h, 7d, 2w; default 24h), `top` (default 10) | totals, `avg_qps`, `peak`, `rcodes`, `qtypes`, `views`, `zones`, `answers`, `transport`, `cache_hit_ratio`, `nxdomain_ratio` (without blocked answers), `blocked`, `blocked_ratio`, `blocklists` (blocked per list; `rules` for your rules), `tops` |
 | GET | `/stats/timeline` | `since`, `step` (auto, 1m, 5m, 15m, 1h, 6h, 1d), `key` (repeatable or comma separated; `rcode:*` = every counter of a group; default `queries`) | `{step, times: [unix...], series: [{key, values}]}` |
 | GET | `/stats/top` | `kind` (name, client, nxdomain, servfail, refused, blocked, blocked_client), `since`, `limit` (default 20) | `[{key, count}]`; client, refused and blocked_client entries add `name`, the host name of Jarl's record for the address (the summary's client list too) |
 | GET | `/stats/heatmap` | `since` (default 28d), `key` (default queries) | `{key, from, avg[7][24], samples[7][24]}`, Monday first, local time, `-1` = no data |
-| GET | `/stats/explore` | `?client=&name=&domain=&view=&type=&rcode=&blocked=yes\|no\|LIST&since=24h&views=`; each takes several values, comma separated (any of them), `!value` to leave one out, and names `*.example.com` for it and below | The query log for a filter ([stats.md](stats.md#exploring)): `total`, `blocked`, `series` (per `step` seconds: `t`, `count`, `blocked`), `by` (top 15 of each dimension the filter leaves open: client, name, view, type, rcode, blocked; clients with `name`), `recent` (latest 100: ts, client, client_name, view, name, type, rcode, blocked), `oldest`, `retention`, `client_name`. 404 `query_log_off` without a query log |
+| GET | `/stats/explore` | `?client=&name=&domain=&view=&type=&rcode=&blocked=yes\|no\|LIST&since=24h&views=`; each takes several values, comma separated (any of them), `!value` to leave one out, and names `*.example.com` for it and below | The query log for a filter ([stats.md](stats.md#exploring)): `total`, `blocked`, `series` (per `step` seconds: `t`, `count`, `blocked`), `by` (top 15 of each dimension the filter leaves open: client, name, view, type, rcode, blocked; clients with `name`), `recent` (latest 100: ts, client, client_name, view, name, type, rcode, blocked), `oldest`, `retention` (`"7d"`) and `retention_seconds`, `client_name`. 404 `query_log_off` without a query log |
 | GET | `/stats/server` | | `{server: {rss, mem_in_use, cpu_percent, threads, pid, cache_rrsets, cache_mem, sockets_udp, sockets_tcp, recursion_highwater, tcp_highwater, version, boot_time, config_time, host: {mem_total, mem_available, load, cpus, uptime}}, named: {recursive_clients, tcp_clients, zones, cpus, worker_threads}, disks: [{path, holds, total, free}], database: {path, bytes, query_log_rows}}`; gauges over time: `/stats/timeline?key=g:named:rss` (averages, -1 without a sample) |
 | GET | `/stats/breakdown` | `group` (qtype, rcode, view, zone, answer, transport, rtt, dnssec, cache, ratelimit, blocklist), `since` | `[{key, count}]` |
 
@@ -598,7 +603,7 @@ Administrators only. See [certificates.md](certificates.md#api).
 
 | Method | Path | Body |
 |--------|------|------|
-| GET | `/certificates` | `certificates` (with `status`, `next_attempt`, `issuing_since`, `serving`, `files`) and `account` |
+| GET | `/certificates` | `certificates` (with `status`, `next_attempt`, `issuing_since`, `serving`, `files`, and for one obtained with `jarl cert get` `owner_id`, `owner` and `pulled_by`: every download for an administrator, the caller's own for the owner) and `account` |
 | POST | `/certificates` | `{"name", "domains", "uses", "challenge", "key_type", "comment"}`, or `{"name", "source": "file", "cert_file", "key_file", "uses"}`; `"issue": false` doesn't start obtaining it |
 | POST | `/certificates/check` | `{"domains", "challenge"}`: the route of each challenge |
 | GET, PATCH, DELETE | `/certificates/{name}` | |

docs/certificates.md

@@ -274,6 +274,7 @@ export JARL_SERVER=https://jarl.adbraun.lol:8053 JARL_TOKEN=@/root/jarl.token
 jarl cert get mail -d mail.adbraun.lol -d smtp.adbraun.lol --hook 'systemctl reload postfix dovecot'
 sudo jarl cert schedule                    # a daily systemd timer runs 'jarl cert get'
 jarl cert saved                            # what this host has, and when it expires
+jarl cert saved mail                       # all of mail's settings
 ```
 
 * **Asking again** with the same names returns the certificate jarld has:
@@ -286,18 +287,41 @@ jarl cert saved                            # what this host has, and when it exp
   expiry, when it was fetched). Each file is replaced at once, never
   half written. The key is checked against the certificate before
   anything is written.
-* **Saved settings**: the names, `--dir`, `--hook`, the server and the
-  token are saved in `/etc/jarl/cert-get/NAME.json` (as root;
-  `~/.config/jarl/cert-get` otherwise), readable only by their owner.
-  `jarl cert get` without a name does every saved certificate. A token
-  given as `@FILE` is saved as that reference, not copied. `jarl cert
-  forget NAME` stops getting one (`--delete` deletes it on jarld too).
-* **The hook** runs with `sh` after new files are written, with
-  `JARL_CERT_NAME`, `JARL_CERT_DIR`, `JARL_CERT_CERT`, `JARL_CERT_CHAIN`,
-  `JARL_CERT_FULLCHAIN`, `JARL_CERT_KEY`, `JARL_CERT_DOMAINS`,
-  `JARL_CERT_SERIAL`, `JARL_CERT_NOT_AFTER` and `JARL_CERT_RENEWED` (1 when
-  it replaced an older certificate). If it fails, `jarl cert get` exits
-  non-zero and runs it again next time. It has 5 minutes.
+* **A combined file** (`--combined PATH`): the key, then the full chain,
+  in one file (mode 600), as HAProxy wants. See [HAProxy](#haproxy).
+* **Saved settings**: the names, `--dir`, `--combined`, `--hook`, the
+  server and the token are saved in `/etc/jarl/cert-get/NAME.json` (as
+  root; `~/.config/jarl/cert-get` otherwise), readable only by their
+  owner. `jarl cert get` without a name does every saved certificate. A
+  token given as `@FILE` is saved as that reference, not copied. `jarl
+  cert saved NAME` shows them all (the token never). `jarl cert forget
+  NAME` stops getting one (`--delete` deletes it on jarld too).
+* **Changing a setting**: run `jarl cert get NAME` with just that option
+  (`--hook '...'`, `--combined PATH`, `--dir DIR`); the others stay.
+  `--hook ''` removes the hook, `--combined ''` stops writing the
+  combined file and deletes it. The run still asks jarld for the
+  certificate, but downloads nothing while this host has the current one.
+  A new or moved combined file is written from the files this host
+  already has, and the hook runs.
+* **`--local`** doesn't ask jarld at all: it writes the combined file and
+  runs the hook from the files this host has. Use it to apply a changed
+  setting while jarld is unreachable, or to test a hook. New names still
+  need jarld.
+* **Several Jarl servers on one host**: each server has its own
+  certificate names, so two servers may both have a `proxy`. The host
+  keeps one `proxy` per settings directory, though: asking another server
+  for a name saved here for the first one is refused, as is a `--dir`
+  holding another certificate's files. Choose another name or `--dir`, or
+  pass `--replace` to take the name (or directory) over. A `--dir` that
+  another saved certificate uses is always refused.
+* **The hook** runs with `sh` after new files are written or the
+  combined file changes (and with `--local`), with `JARL_CERT_NAME`,
+  `JARL_CERT_DIR`, `JARL_CERT_CERT`, `JARL_CERT_CHAIN`,
+  `JARL_CERT_FULLCHAIN`, `JARL_CERT_KEY`, `JARL_CERT_COMBINED` (empty
+  without `--combined`), `JARL_CERT_DOMAINS`, `JARL_CERT_SERIAL`,
+  `JARL_CERT_NOT_AFTER` and `JARL_CERT_RENEWED` (1 when it replaced an
+  older certificate). If it fails, `jarl cert get` exits non-zero and runs
+  it again next time. It has 5 minutes.
 * **`jarl cert schedule`** (Linux with systemd, as root) writes
   `/etc/systemd/system/jarl-cert-get.service` and `.timer`, which run
   `jarl cert get` daily at a random time within the hour after midnight,
@@ -307,7 +331,44 @@ jarl cert saved                            # what this host has, and when it exp
 **On the Certificates page**, the administrators see each certificate's
 owner and who downloaded it: the token, from where, when, and whether
 that host has the current certificate or hasn't fetched the renewed one
-yet. `jarl cert show NAME` says the same.
+yet. `jarl cert show NAME` says the same. The owner's own tokens see
+their own downloads there too. In `jarl cert list`, such a certificate's
+USES is `get`, and PULLED is the last download, with `(old)` while a host
+hasn't fetched the renewed one.
+
+### HAProxy
+
+HAProxy wants the key and the chain in one file. Give `jarl cert get` a
+`--combined` path outside `--dir`:
+
+```sh
+jarl cert get proxy -d adbraun.lol -d '*.adbraun.lol' \
+    --combined /etc/haproxy/certs/adbraun.lol.pem --hook 'systemctl reload haproxy'
+jarl cert get webmail -d webmail.adbraun.lol \
+    --combined /etc/haproxy/certs/webmail.pem --hook 'systemctl reload haproxy'
+```
+
+```
+frontend https
+    bind :443 ssl crt /etc/haproxy/certs/
+```
+
+HAProxy loads every file in a `crt` directory and picks one by SNI, so
+`/etc/haproxy/certs` should hold only combined files. Don't point `crt`
+at `--dir` (`/etc/jarl/certs/NAME`): HAProxy would try to load
+`cert.pem`, `privkey.pem` and `jarl-cert.json` there too, and refuse
+to start. A `--combined` path that would overwrite a file in a saved
+certificate's directory, or another certificate's combined file, is
+refused.
+
+The combined file is mode 600, owned by the user running `jarl cert get`
+(root under the timer). HAProxy reads its certificates as root before it
+drops privileges, so that's fine.
+
+To add a combined file to a certificate this host already has, run
+`jarl cert get proxy --combined /etc/haproxy/certs/adbraun.lol.pem`: it's
+written from the files on disk and the hook runs, with nothing
+downloaded.
 
 ## Security notes
 

docs/cli.md

@@ -179,6 +179,7 @@ jarl zone create 10.0.8.0/22                           # four zones, 8-11.0.10.i
 jarl zone create 10.0.10.100/32                        # a zone for just that address
 jarl zone create 10.0.10.0/24 --fill                   # in the view whose records use it, with their PTRs
 jarl zone fill-ptr 10.0.10.0/24                        # add the PTRs it lacks (zone commands take a network too)
+jarl zone fill-ptr 10.0.10.0/24 --view vps --other-views   # also name addresses only other views' records use
 jarl zone suggest-reverse [--create]                   # reverse zones your private views lack (--create: yours)
 jarl zone suggest-reverse --dismiss 185.199.108.153/32,185.199.109.153/32 --view internal   # not yours: don't suggest them again
 jarl zone suggest-reverse --all [--view V]             # every network without a reverse zone, also the hidden ones
@@ -450,7 +451,11 @@ jarl cert export mail --dir /etc/ssl/mail            # fullchain.pem and privkey
 jarl cert export mail --part bundle                  # one part to stdout: fullchain, cert, key, bundle
 jarl cert get mail -d mail.adbraun.lol --hook 'systemctl reload postfix'   # on another host: get, download, hook
 jarl cert get                                        # every saved certificate (what the timer runs)
+jarl cert get proxy -s other --replace               # take over a name saved here for another server
+jarl cert get proxy --combined /etc/haproxy/certs/proxy.pem   # also key + chain in one file (HAProxy); '' stops it
+jarl cert get proxy --hook 'systemctl reload haproxy' --local  # change a setting; apply it from this host's files, without jarld
 jarl cert saved                                      # this host's certificates and their expiry
+jarl cert saved proxy                                # all of one certificate's settings (never the token)
 jarl cert forget mail [--delete]                     # stop getting it here
 sudo jarl cert schedule [--remove]                   # daily systemd timer for jarl cert get
 jarl cert delete ns1
@@ -535,6 +540,7 @@ jarl token list --all --expiring [60d]                 # expiring within 14 days
 jarl token create laptop --expires 365d
 jarl token create backup-script --grant "*:read"
 jarl token create acme --user certbot --role acme
+jarl token create web01 --grant "certificates:write names=web01.adbraun.lol"   # jarl cert get on one host (or --role certificates)
 jarl token rotate laptop --save                        # new secret, same token and grants
 jarl token rotate --current --save --grace 10m         # the token this CLI uses
 jarl token revoke 7

docs/operations.md

@@ -360,6 +360,23 @@ cp /etc/bind/named.conf.pre-jarl-<date> /etc/bind/named.conf
 rndc reconfig
 ```
 
+## Failed logins in the journal
+
+A failed login to the web UI or the API (a wrong password, or a wrong
+two-factor code), and a refused API token, are written to the journal as
+well as the audit log, with the address first:
+
+```
+level=WARN msg="login failed" ip=203.0.113.9 user=admin reason="wrong username or password"
+level=WARN msg="token refused" ip=203.0.113.9 reason="wrong token secret"
+```
+
+Guard's `jarl` preset, or a fail2ban filter, can ban such addresses at
+the firewall (with Housecarl's ban lists). The address comes before the
+user name, which a caller chooses, so a pattern anchored on
+`msg="login failed" ip=` can't be fooled by a name. With `log.format:
+json` they are JSON objects with the same fields in the same order.
+
 ## Monitoring
 
 * `GET /healthz` (no authentication) reports that jarld is up.

docs/permissions.md

@@ -185,6 +185,7 @@ you can then edit.
 | `dns-editor` | `zones:write`, `records:write`, `views:read`, `acls:read`, `options:read`, `blocking:read` |
 | `operator` | `*:write` (everything except user management) |
 | `acme` | `records:write names=_acme-challenge.* types=TXT` |
+| `certificates` | `certificates:write`: certificates for other hosts (`jarl cert get`) for any name in Jarl's zones. Narrow it to a host's names, e.g. `certificates:write names=web01.adbraun.lol` ([certificates.md](certificates.md#certificates-for-other-hosts)) |
 
 ```sh
 jarl user create alice --role dns-editor
@@ -210,6 +211,7 @@ The audit log shows the user and the token name.
 jarl token create laptop                                  # full rights of the user, no expiry
 jarl token create ci --grant "records:write zones=lab.adbraun.lol" --expires 90d
 jarl token create certbot --user certbot --role acme       # admin: a token for another user
+jarl token create web01 --grant "certificates:write names=web01.adbraun.lol"   # jarl cert get on one host
 jarl token list --all
 jarl token revoke 12
 ```

docs/reverse.md

@@ -97,10 +97,19 @@ jarl zone create 2a02:8013:7f3d:10::/64 --view internal --fill
 the dialog's View), the list also has the addresses only other views'
 records use, marked "records in external": say a VPS's AAAA record that
 only the external view has. Fill names those from the other view's
-record, when the zone's own view has none for the address, and says so.
-Keeping PTR records in step (below) stays within a zone's own view and
-its linked views: a later change to that external record doesn't reach
-the internal reverse zone.
+record only when asked: `--other-views` (with `--fill` or `jarl zone
+fill-ptr`), or the dialog's **Also for addresses only other views'
+records use**, which is ticked for you when you pick a "records in"
+network. Otherwise one view's host names would turn up in another's
+reverse zone, which is what views are for keeping apart. Jarl says which
+names came from another view. Keeping PTR records in step (below) stays
+within a zone's own view and its linked views: a later change to that
+external record doesn't reach the internal reverse zone.
+
+```sh
+jarl zone create 2600:3c01::f03c:93ff:fe2b:1/128 --view internal --fill --other-views
+jarl zone fill-ptr 10.0.10.0/24 --view vps --other-views
+```
 
 ## Creating and filling
 
@@ -135,7 +144,22 @@ Jarl says which name it chose, and which it didn't, so you can change it.
 A reverse zone's page shows the addresses that still lack a PTR record and
 the name each would get (the first 25), with a button to add them. `jarl
 zone show` shows how many as `Missing PTRs`; the API lists them in
-`ptr_missing_list`.
+`ptr_missing_list`. Only the zone's own view's records count: an address
+that only another view's records use isn't missing (say internal's
+`claude.adbraun.lol` and the vps view's copy of `10.0.10.in-addr.arpa`).
+Those are listed apart, with the name each would get: `Other views` in
+`jarl zone show`, `ptr_other_views` and `ptr_other_views_list` in the
+API, and a quiet note on the page with an **Add PTR records for them in
+view vps** button. Filling never adds them unasked. `jarl zone show`
+lists the missing ones too:
+
+```
+Missing PTRs:  1 (addresses the view's records use here; add them: jarl zone fill-ptr 10.0.10.0/24)
+                 10.0.10.7 -> printer.adbraun.lol
+Other views:   2 addresses with A or AAAA records only in view internal, so not missing here (add PTRs for them in this view too: jarl zone fill-ptr 10.0.10.0/24 --other-views)
+                 10.0.10.26 -> gomft.adbraun.lol (view internal)
+                 10.0.10.101 -> claude.adbraun.lol (view internal)
+```
 
 ## Keeping PTR records in step
 

docs/stats.md

@@ -263,7 +263,11 @@ database, roughly in proportion to the number of views.
   lists for `retention.hour`, daily ones for `retention.day`. Older data is
   deleted every hour.
 - Charts use the finest resolution that covers the range: minutes up to 48
-  hours, hours up to 90 days, days beyond.
+  hours (and `retention.minute`), hours up to `retention.hour`, days beyond.
+- The Statistics page offers only the ranges the data is kept for: up to 90
+  days with the defaults, plus a year and 2 years when `retention.day` (or
+  `retention.hour`) is that long. The Explore page offers the ranges up to
+  `retention.queries`.
 - Top lists are exact within an hour (up to 200,000 distinct names or
   clients per hour, beyond which the rarest are dropped). Daily lists are
   built from the hourly top lists, so for very long tails they are

internal/api/types.go

@@ -48,6 +48,12 @@ type Zone struct {
 	// PTRMissingList are the first 25 of them: the PTR records adding
 	// them would create.
 	PTRMissingList []PTRFill `json:"ptr_missing_list,omitempty"`
+	// PTROtherViews counts the addresses in it, without a PTR record,
+	// that only other views' records use: not missing, but fill-ptr with
+	// other_views=1 adds them, naming them from there. PTROtherViewsList
+	// are the first 25 (From says which view).
+	PTROtherViews     int       `json:"ptr_other_views,omitempty"`
+	PTROtherViewsList []PTRFill `json:"ptr_other_views_list,omitempty"`
 	// Pending is set when listing with a changeset: add, change or delete.
 	Pending string `json:"pending,omitempty"`
 }

internal/auth/auth_test.go

@@ -135,6 +135,16 @@ func TestRoleTemplatesValid(t *testing.T) {
 	}
 }
 
+func TestCertificatesRole(t *testing.T) {
+	p := &Principal{User: &model.User{Grants: RoleTemplates["certificates"]}}
+	if !p.Can(Action{Resource: ResCerts, Write: true, Name: "web01.adbraun.lol"}) {
+		t.Error("the certificates role can't get a certificate")
+	}
+	if p.Can(Action{Resource: ResRecords, Write: true, Zone: "adbraun.lol."}) || p.Can(Action{Resource: ResZones, Zone: "adbraun.lol."}) {
+		t.Error("the certificates role reaches beyond certificates")
+	}
+}
+
 func TestCertificateGrants(t *testing.T) {
 	g, err := ParseGrant("certificates:write names=*.adbraun.lol")
 	if err != nil {

internal/auth/grants.go

@@ -391,4 +391,7 @@ var RoleTemplates = map[string][]model.Grant{
 	},
 	"operator": {{Resource: ResAll, Access: Write}},
 	"acme":     {{Resource: ResRecords, Access: Write, Names: []string{"_acme-challenge.*"}, Types: []string{"TXT"}}},
+	// certificates for other hosts (jarl cert get): narrow it with names=
+	// to the host's own.
+	"certificates": {{Resource: ResCerts, Access: Write}},
 }

internal/engine/expand.go

@@ -471,8 +471,8 @@ func (x *expander) zoneCreate(req Request) ([]model.Op, error) {
 			}
 		}
 	}
-	fill := in.Fill
-	in.Fill = false
+	fill, fillOther := in.Fill, in.FillOtherViews
+	in.Fill, in.FillOtherViews = false, false
 	if fill {
 		if zt != model.ZonePrimary {
 			return nil, errf(KindBadRequest, "fill only applies to primary zones")
@@ -484,10 +484,15 @@ func (x *expander) zoneCreate(req Request) ([]model.Op, error) {
 		if in.ViewID != nil {
 			vid = *in.ViewID
 		}
-		fills := PTRFills(x.st, vid, name, moved)
+		fills := PTRFills(x.st, vid, name, moved, fillOther)
 		if len(fills) == 0 {
 			x.warnings = append(x.warnings, fmt.Sprintf("no A or AAAA record of %s has an address in %s, so there was nothing to fill it with", viewOrServer(x.st, vid), dnsname.Display(name)))
 		}
+		if !fillOther {
+			if h := otherViewsHint(x.st, vid, name, moved, len(fills), "to add PTR records for them in this view too: jarl zone fill-ptr ZONE --other-views"); h != "" {
+				x.warnings = append(x.warnings, h)
+			}
+		}
 		x.warnings = append(x.warnings, fillWarnings(fills)...)
 		for _, f := range fills {
 			in.Records = append(in.Records, RecordInput{Name: model.Ptr(f.Name), Type: model.Ptr("PTR"), Data: model.Ptr(f.Target)})

internal/engine/inputs.go

@@ -163,8 +163,11 @@ type ZoneInput struct {
 	// step (docs/reverse.md).
 	AutoPTR *bool `json:"auto_ptr,omitempty"`
 	// Fill (create only, reverse zones) adds a PTR record for every
-	// address of the view's A and AAAA records in the zone's range.
+	// address of the view's A and AAAA records in the zone's range;
+	// FillOtherViews also for the addresses only other views' records
+	// use, named from there.
 	Fill           bool `json:"fill,omitempty"`
+	FillOtherViews bool `json:"fill_other_views,omitempty"`
 
 	// Create only: name servers for the apex NS records (default: the
 	// configured default name servers), additional records, and a zone to
@@ -224,6 +227,13 @@ type LinkInput struct {
 	With []string `json:"with,omitempty"`
 }
 
+// FillPTRInput is zone.fill_ptr: OtherViews also fills the addresses only
+// other views' records use, naming them from there.
+type FillPTRInput struct {
+	ZoneID     int64 `json:"zone_id,omitempty"`
+	OtherViews bool  `json:"other_views,omitempty"`
+}
+
 // RRSetInput replaces all records of one name and type.
 type RRSetInput struct {
 	ZoneID  int64             `json:"zone_id,omitempty"`

internal/engine/reverse.go

@@ -87,11 +87,12 @@ type PTRFill struct {
 // PTRFills returns the PTR records a reverse zone named zoneName in a view
 // lacks for the addresses of the view's A and AAAA records: one per
 // address, naming its mail or name server host when it is one, otherwise
-// a host rather than the domain itself, then the shortest name. An
-// address no record of the view uses is named from another view's
-// records (fillAddresses), marked From. existing are the zone's records
-// (none for a new zone).
-func PTRFills(st *model.State, viewID int64, zoneName string, existing []*model.Record) []PTRFill {
+// a host rather than the domain itself, then the shortest name. With
+// otherViews, an address no record of the view uses is named from another
+// view's records (fillAddresses), marked From; that is only on request,
+// since it publishes one view's host names in another. existing are the
+// zone's records (none for a new zone).
+func PTRFills(st *model.State, viewID int64, zoneName string, existing []*model.Record, otherViews bool) []PTRFill {
 	prefix, ok := dnsname.PrefixFromReverse(zoneName)
 	if !ok {
 		return nil
@@ -105,7 +106,11 @@ func PTRFills(st *model.State, viewID int64, zoneName string, existing []*model.
 	servers := serverNames(st, viewID)
 	byAddr := map[netip.Addr][]addrUse{}
 	var order []netip.Addr
-	for _, u := range fillAddresses(st, viewID) {
+	addrs := forwardAddresses(st, viewID)
+	if otherViews {
+		addrs = fillAddresses(st, viewID)
+	}
+	for _, u := range addrs {
 		if !prefix.Contains(u.addr) {
 			continue
 		}
@@ -195,9 +200,23 @@ func displayAll(names []string) []string {
 	return out
 }
 
+// otherViewsHint says how many addresses only other views' records use in
+// a reverse zone, and how to add them; "" when there are none.
+func otherViewsHint(st *model.State, viewID int64, zoneName string, existing []*model.Record, own int, how string) string {
+	n := len(PTRFills(st, viewID, zoneName, existing, true)) - own
+	if n <= 0 {
+		return ""
+	}
+	what, has, they := "addresses", "have", "they are"
+	if n == 1 {
+		what, has, they = "address", "has", "it is"
+	}
+	return fmt.Sprintf("%d %s in %s %s A or AAAA records only in other views, so %s not missing here; %s", n, what, dnsname.Display(zoneName), has, they, how)
+}
+
 // zoneFillPTR expands zone.fill_ptr: the PTR records a reverse zone lacks.
 func (x *expander) zoneFillPTR(req Request) ([]model.Op, error) {
-	var in LinkInput
+	var in FillPTRInput
 	if err := decode(req.Data, &in); err != nil {
 		return nil, err
 	}
@@ -215,9 +234,15 @@ func (x *expander) zoneFillPTR(req Request) ([]model.Op, error) {
 	if _, ok := dnsname.PrefixFromReverse(z.Name); !ok {
 		return nil, errf(KindBadRequest, "zone %s isn't a reverse zone of whole octets or nibbles (like 10.0.10.in-addr.arpa)", zoneDesc(x.st, z))
 	}
-	fills := PTRFills(x.st, z.ViewID, z.Name, x.st.ZoneRecords(z.ID))
+	fills := PTRFills(x.st, z.ViewID, z.Name, x.st.ZoneRecords(z.ID), in.OtherViews)
 	if len(fills) == 0 {
-		return nil, errf(KindBadRequest, "nothing to add: every address of %s's A and AAAA records in %s has a PTR record", viewOrServer(x.st, z.ViewID), zoneDesc(x.st, z))
+		msg := fmt.Sprintf("nothing to add: every address of %s's A and AAAA records in %s has a PTR record", viewOrServer(x.st, z.ViewID), zoneDesc(x.st, z))
+		if !in.OtherViews {
+			if h := otherViewsHint(x.st, z.ViewID, z.Name, x.st.ZoneRecords(z.ID), 0, "to add PTR records for them in this view too: --other-views"); h != "" {
+				msg += "; " + h
+			}
+		}
+		return nil, errf(KindBadRequest, "%s", msg)
 	}
 	x.warnings = append(x.warnings, fillWarnings(fills)...)
 	ids, err := x.ids(len(fills))
@@ -415,7 +440,7 @@ func ReverseCandidates(st *model.State, exposures []validate.Exposure, local []n
 				continue
 			}
 			s := ReverseSuggestion{ViewID: sc.id, View: viewName(st, sc.id), Prefix: p.String(), Zone: strings.TrimSuffix(name, "."), Group: group, Hidden: hidden, Why: why, Samples: []string{}}
-			fills := PTRFills(st, sc.id, name, nil)
+			fills := PTRFills(st, sc.id, name, nil, opt.All && opt.View != "")
 			s.Fill = len(fills)
 			for i, f := range fills {
 				if i == 3 {

internal/engine/reverse_test.go

@@ -69,11 +69,11 @@ func TestReverseFillAndSuggest(t *testing.T) {
 	env.submit(env.admin, NewRequest(OpZoneCreate, 0, ZoneInput{Name: model.Ptr("fdab:abab:abab::/48")}))
 	st := env.state()
 	z6 := st.FindZone("b.a.b.a.b.a.b.a.b.a.d.f.ip6.arpa.", st.ViewByName("internal").ID)
-	env.submit(env.admin, NewRequest(OpZoneFillPTR, z6.ID, LinkInput{}))
+	env.submit(env.admin, NewRequest(OpZoneFillPTR, z6.ID, FillPTRInput{}))
 	if p := env.ptrs("internal", z6.Name); len(p) != 1 || !strings.HasSuffix(p[0], " nas.example.com.") {
 		t.Errorf("ip6 fill: %q", p)
 	}
-	if e := env.submitErr(env.admin, NewRequest(OpZoneFillPTR, z6.ID, LinkInput{})); !strings.Contains(e.Message, "nothing to add") {
+	if e := env.submitErr(env.admin, NewRequest(OpZoneFillPTR, z6.ID, FillPTRInput{})); !strings.Contains(e.Message, "nothing to add") {
 		t.Errorf("second fill: %v", e)
 	}
 
@@ -288,15 +288,38 @@ func TestReverseFromOtherView(t *testing.T) {
 			t.Errorf("suggested unasked: %+v", s)
 		}
 	}
+	for _, s := range ReverseCandidates(env.state(), env.e.Exposures(env.state()), nil, ReverseOptions{All: true, View: "internal"}) {
+		if strings.HasPrefix(s.Prefix, "2600:") && (s.Fill != 1 || len(s.Samples) != 1) {
+			t.Errorf("candidate counts the other view's address: %+v", s)
+		}
+	}
+	// Filling doesn't take names from other views unasked: it would
+	// publish one view's host names in another. It says how to.
 	res := env.submit(env.admin, NewRequest(OpZoneCreate, 0, ZoneInput{Name: model.Ptr("2600:3c01::f03c:93ff:fe2b:1/128"), View: model.Ptr("internal"), Fill: true}))
 	st := env.state()
-	var zone string
-	for _, z := range st.ZonesInView(st.ViewByName("internal").ID) {
-		if strings.HasSuffix(z.Name, ".ip6.arpa.") {
-			zone = z.Name
+	var z *model.Zone
+	for _, o := range st.ZonesInView(st.ViewByName("internal").ID) {
+		if strings.HasSuffix(o.Name, ".ip6.arpa.") {
+			z = o
 		}
 	}
-	if p := env.ptrs("internal", zone); len(p) != 1 || !strings.HasSuffix(p[0], " vps.example.com.") {
+	if p := env.ptrs("internal", z.Name); len(p) != 0 {
+		t.Errorf("filled from another view unasked: %q", p)
+	}
+	if !slices.ContainsFunc(res.Warnings, func(w string) bool {
+		return strings.Contains(w, "1 address in") && strings.Contains(w, "--other-views")
+	}) {
+		t.Errorf("warnings %q", res.Warnings)
+	}
+	// Not missing: no record of the zone's own view uses the address.
+	if f := PTRFills(st, z.ViewID, z.Name, st.ZoneRecords(z.ID), false); len(f) != 0 {
+		t.Errorf("missing %+v", f)
+	}
+	if e := env.submitErr(env.admin, NewRequest(OpZoneFillPTR, z.ID, FillPTRInput{})); !strings.Contains(e.Message, "nothing to add") || !strings.Contains(e.Message, "--other-views") {
+		t.Errorf("fill-ptr: %v", e)
+	}
+	res = env.submit(env.admin, NewRequest(OpZoneFillPTR, z.ID, FillPTRInput{OtherViews: true}))
+	if p := env.ptrs("internal", z.Name); len(p) != 1 || !strings.HasSuffix(p[0], " vps.example.com.") {
 		t.Errorf("filled %q", p)
 	}
 	if !slices.ContainsFunc(res.Warnings, func(w string) bool { return strings.Contains(w, "only view external has a record for it") }) {

internal/model/blocking.go

@@ -204,7 +204,7 @@ func (s *State) SortedBlockRules() []*BlockRule {
 	sort.Slice(out, func(i, j int) bool {
 		a, b := out[i], out[j]
 		if a.Name != b.Name {
-			return CanonicalLess(a.Name+".", b.Name+".")
+			return NameLess(a.Name+".", b.Name+".")
 		}
 		if a.Exact != b.Exact {
 			return !a.Exact

internal/model/model.go

@@ -524,14 +524,14 @@ func (s *State) SortedZones() []*Zone {
 		if pi, pj := pos[out[i].ViewID], pos[out[j].ViewID]; pi != pj {
 			return pi < pj
 		}
-		return CanonicalLess(out[i].Name, out[j].Name)
+		return NameLess(out[i].Name, out[j].Name)
 	})
 	return out
 }
 
-// SortZones sorts zones by name in DNS canonical order.
+// SortZones sorts zones by name (see NameLess).
 func SortZones(zs []*Zone) {
-	sort.Slice(zs, func(i, j int) bool { return CanonicalLess(zs[i].Name, zs[j].Name) })
+	sort.Slice(zs, func(i, j int) bool { return NameLess(zs[i].Name, zs[j].Name) })
 }
 
 // FindZone finds a zone by name within a view (viewID 0 for no view).
@@ -646,7 +646,7 @@ func typeOrder(t string) string {
 }
 
 // SortRecords sorts records in zone-file order: apex first, then names in
-// DNS canonical order (so subdomains group together), then by type and
+// NameLess order (so subdomains group together), then by type and
 // data.
 func SortRecords(rs []*Record, z *Zone) {
 	apex := ""
@@ -662,7 +662,7 @@ func SortRecords(rs []*Record, z *Zone) {
 			if b.Name == apex {
 				return false
 			}
-			return CanonicalLess(a.Name, b.Name)
+			return NameLess(a.Name, b.Name)
 		}
 		if a.Type != b.Type {
 			return typeOrder(a.Type) < typeOrder(b.Type)
@@ -674,19 +674,50 @@ func SortRecords(rs []*Record, z *Zone) {
 	})
 }
 
-// CanonicalLess compares names in DNS canonical order (RFC 4034 section
-// 6.1): by label from the right.
-func CanonicalLess(a, b string) bool {
+// NameLess compares names label by label from the right, like DNS
+// canonical order (RFC 4034 section 6.1), except that two all-digit
+// labels compare as numbers, so 2.10.0.10.in-addr.arpa comes before
+// 101.10.0.10.in-addr.arpa. It orders names for people and zone files;
+// BIND does its own canonical ordering where DNSSEC needs it.
+func NameLess(a, b string) bool {
 	la := strings.Split(strings.TrimSuffix(a, "."), ".")
 	lb := strings.Split(strings.TrimSuffix(b, "."), ".")
 	for i, j := len(la)-1, len(lb)-1; i >= 0 && j >= 0; i, j = i-1, j-1 {
-		if la[i] != lb[j] {
-			return la[i] < lb[j]
+		if c := compareLabel(la[i], lb[j]); c != 0 {
+			return c < 0
 		}
 	}
 	return len(la) < len(lb)
 }
 
+func compareLabel(a, b string) int {
+	if a == b {
+		return 0
+	}
+	if allDigits(a) && allDigits(b) {
+		na, nb := strings.TrimLeft(a, "0"), strings.TrimLeft(b, "0")
+		if len(na) != len(nb) {
+			return len(na) - len(nb)
+		}
+		if c := strings.Compare(na, nb); c != 0 {
+			return c
+		}
+	}
+	return strings.Compare(a, b)
+}
+
+func allDigits(s string) bool {
+	if s == "" {
+		return false
+	}
+	for i := 0; i < len(s); i++ {
+		if s[i] < '0' || s[i] > '9' {
+			return false
+		}
+	}
+	return true
+}
+
 // RRSetKey identifies an RRset.
 type RRSetKey struct {
 	Name string

internal/model/nameorder_test.go

@@ -0,0 +1,42 @@
+package model
+
+import (
+	"slices"
+	"testing"
+)
+
+func TestNameLess(t *testing.T) {
+	want := []string{
+		"10.0.10.in-addr.arpa.",
+		"1.10.0.10.in-addr.arpa.",
+		"02.10.0.10.in-addr.arpa.",
+		"2.10.0.10.in-addr.arpa.",
+		"10.10.0.10.in-addr.arpa.",
+		"101.10.0.10.in-addr.arpa.",
+		"x.101.10.0.10.in-addr.arpa.",
+		"254.10.0.10.in-addr.arpa.",
+		"a.10.0.10.in-addr.arpa.",
+	}
+	got := slices.Clone(want)
+	slices.Reverse(got)
+	slices.SortFunc(got, func(a, b string) int {
+		if NameLess(a, b) {
+			return -1
+		}
+		if NameLess(b, a) {
+			return 1
+		}
+		return 0
+	})
+	if !slices.Equal(got, want) {
+		t.Fatalf("got %v\nwant %v", got, want)
+	}
+
+	// Non-numeric labels keep canonical (byte) order.
+	if !NameLess("host10.example.", "host2.example.") {
+		t.Error("host10 should sort before host2")
+	}
+	if !NameLess("example.", "www.example.") {
+		t.Error("parent should sort before child")
+	}
+}

internal/server/authlog_test.go

@@ -0,0 +1,23 @@
+package server
+
+import (
+	"bytes"
+	"log/slog"
+	"strings"
+	"testing"
+)
+
+// TestFailedLoginLogged: a failed login and a refused token are logged
+// with the address first, for Guard's and fail2ban's patterns.
+func TestFailedLoginLogged(t *testing.T) {
+	e := newAPIEnv(t)
+	var buf bytes.Buffer
+	e.srv.log = slog.New(slog.NewTextHandler(&buf, nil))
+	e.req("POST", "/auth/login", "", map[string]string{"username": "x ip=8.8.8.8", "password": "wrong"})
+	e.req("GET", "/zones", "jarl_aaaaaaaaaaaaaaaa_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", nil)
+	for _, want := range []string{`msg="login failed" ip=127.0.0.1 user="x ip=8.8.8.8"`, `msg="token refused" ip=127.0.0.1 reason=`} {
+		if !strings.Contains(buf.String(), want) {
+			t.Errorf("no %q in\n%s", want, buf.String())
+		}
+	}
+}

internal/server/authn.go

@@ -136,6 +136,8 @@ func (s *Server) authenticate(r *http.Request) (*auth.Principal, *model.Session,
 		p, err := s.tokenPrincipal(r.Context(), strings.TrimSpace(tok), ip, r.UserAgent())
 		if err != nil {
 			s.limiter.fail("ip:" + ip)
+			// For Guard and fail2ban: the address first, so a name can't fake it.
+			s.log.Warn("token refused", "ip", ip, "reason", err.Error())
 			s.eng.Audit(r.Context(), &auth.Principal{IP: ip, Agent: r.UserAgent(), Method: "token"}, "auth.token_failed", "", 0, "", "failure", err.Error())
 			return nil, nil, &httpError{http.StatusUnauthorized, "unauthenticated", "invalid API token: " + err.Error()}
 		}
@@ -328,6 +330,8 @@ func (s *Server) login(w http.ResponseWriter, r *http.Request) error {
 		if err == nil && u.Disabled && ok {
 			reason = "account disabled"
 		}
+		// For Guard and fail2ban: the address first, so a name can't fake it.
+		s.log.Warn("login failed", "ip", ip, "user", name, "reason", reason)
 		s.eng.Audit(r.Context(), &auth.Principal{User: &model.User{Username: name}, IP: ip, Agent: r.UserAgent(), Method: "session"}, "auth.login_failed", "user", 0, name, "failure", reason)
 		return &httpError{http.StatusUnauthorized, "login_failed", "wrong username or password"}
 	}

internal/server/dsat_test.go

@@ -93,3 +93,29 @@ func TestPTRMissingList(t *testing.T) {
 		t.Errorf("ptr_missing %v, list %v", z["ptr_missing"], z["ptr_missing_list"])
 	}
 }
+
+// An address only another view's records use isn't missing from a
+// reverse zone: it is counted apart, and filled only when asked.
+func TestPTROtherViews(t *testing.T) {
+	e := newAPIEnv(t)
+	e.do("POST", "/views", e.token, map[string]any{"name": "internal", "match_clients": []string{"10.0.0.0/8"}}, 201)
+	e.do("POST", "/views", e.token, map[string]any{"name": "vps"}, 201)
+	e.do("POST", "/zones", e.token, map[string]any{"name": "adbraun.lol", "view": "internal", "nameservers": []string{"ns1.example.net."},
+		"records": []map[string]any{{"name": "claude", "type": "A", "data": "10.0.10.101"}}}, 201)
+	for _, v := range []string{"internal", "vps"} {
+		e.do("POST", "/zones", e.token, map[string]any{"name": "10.0.10.0/24", "view": v, "nameservers": []string{"ns1.example.net."}}, 201)
+	}
+	if z := e.do("GET", "/zones/10.0.10.in-addr.arpa?view=internal", e.token, nil, 200); z["ptr_missing"] != float64(1) || z["ptr_other_views"] != nil {
+		t.Errorf("internal: ptr_missing %v, ptr_other_views %v", z["ptr_missing"], z["ptr_other_views"])
+	}
+	if z := e.do("GET", "/zones/10.0.10.in-addr.arpa?view=vps", e.token, nil, 200); z["ptr_missing"] != nil || z["ptr_other_views"] != float64(1) {
+		t.Errorf("vps: ptr_missing %v, ptr_other_views %v", z["ptr_missing"], z["ptr_other_views"])
+	} else if l, _ := z["ptr_other_views_list"].([]any); len(l) != 1 || l[0].(map[string]any)["address"] != "10.0.10.101" || l[0].(map[string]any)["target"] != "claude.adbraun.lol." || l[0].(map[string]any)["from"] != "internal" {
+		t.Errorf("vps: ptr_other_views_list %v", z["ptr_other_views_list"])
+	}
+	e.do("POST", "/zones/10.0.10.in-addr.arpa/fill-ptr?view=vps", e.token, nil, 400)
+	e.do("POST", "/zones/10.0.10.in-addr.arpa/fill-ptr?view=vps&other_views=1", e.token, nil, 201)
+	if z := e.do("GET", "/zones/10.0.10.in-addr.arpa?view=vps", e.token, nil, 200); z["ptr_other_views"] != nil {
+		t.Errorf("vps after filling: ptr_other_views %v", z["ptr_other_views"])
+	}
+}

internal/server/h_acme.go

@@ -198,6 +198,8 @@ func (s *Server) httpreqPrincipal(w http.ResponseWriter, r *http.Request) (*auth
 	}
 	fail := func(reason string) error {
 		s.limiter.fail("ip:" + ip)
+		// For Guard and fail2ban: the address first, so a name can't fake it.
+		s.log.Warn("token refused", "ip", ip, "reason", "httpreq: "+reason)
 		s.eng.Audit(r.Context(), &auth.Principal{User: &model.User{Username: user}, IP: ip, Agent: r.UserAgent(), Method: "token"}, "auth.token_failed", "", 0, "", "failure", "httpreq: "+reason)
 		w.Header().Set("WWW-Authenticate", httpreqRealm)
 		return &httpError{http.StatusUnauthorized, "unauthenticated", "invalid credentials: " + reason}

internal/server/h_certs.go

@@ -33,7 +33,8 @@ type certView struct {
 	// IssuingSince is set while Jarl is obtaining it.
 	IssuingSince *time.Time `json:"issuing_since,omitempty"`
 	// Owner is the user a certificate for another host belongs to; PulledBy
-	// who downloaded it (list and get, administrators only).
+	// who downloaded it (list and get; an owner who isn't an administrator
+	// sees only their own downloads).
 	Owner    string                  `json:"owner,omitempty"`
 	PulledBy []model.CertificatePull `json:"pulled_by,omitempty"`
 }
@@ -116,12 +117,27 @@ func (s *Server) certFor(r *http.Request, p *auth.Principal, write bool) (*model
 	return c, nil
 }
 
-// withOwners adds owners and downloads to views (for administrators).
-func (s *Server) withOwners(ctx context.Context, views []certView) error {
+// withOwners adds owners and downloads to views: all of them for an
+// administrator, the principal's own downloads for anyone else (who sees
+// only their own certificates).
+func (s *Server) withOwners(ctx context.Context, p *auth.Principal, views []certView) error {
 	pulls, err := s.store.CertificatePulls(ctx)
 	if err != nil {
 		return err
 	}
+	if !p.IsAdmin() {
+		for i := range views {
+			for _, pl := range pulls[views[i].ID] {
+				if p.User != nil && pl.UserID == p.User.ID {
+					views[i].PulledBy = append(views[i].PulledBy, pl)
+				}
+			}
+			if p.User != nil && views[i].OwnerID != 0 {
+				views[i].Owner = p.User.Username
+			}
+		}
+		return nil
+	}
 	users, err := s.store.ListUsers(ctx)
 	if err != nil {
 		return err
@@ -179,13 +195,13 @@ func (s *Server) certList(w http.ResponseWriter, r *http.Request, p *auth.Princi
 		v.NamedUse = encdns.CertUses(st, s.cfg.Certs.FilesDir(c.Name))
 		out = append(out, v)
 	}
+	if err := s.withOwners(r.Context(), p, out); err != nil {
+		return err
+	}
 	if !p.IsAdmin() {
 		writeJSON(w, http.StatusOK, map[string]any{"certificates": out})
 		return nil
 	}
-	if err := s.withOwners(r.Context(), out); err != nil {
-		return err
-	}
 	writeJSON(w, http.StatusOK, map[string]any{"certificates": out, "account": s.cfg.Certs.AccountInfo(r.Context())})
 	return nil
 }
@@ -205,13 +221,11 @@ func (s *Server) certGet(w http.ResponseWriter, r *http.Request, p *auth.Princip
 	}
 	v := s.presentCert(c)
 	v.NamedUse = encdns.CertUses(st, s.cfg.Certs.FilesDir(c.Name))
-	if p.IsAdmin() {
 	vs := []certView{v}
-		if err := s.withOwners(r.Context(), vs); err != nil {
+	if err := s.withOwners(r.Context(), p, vs); err != nil {
 		return err
 	}
 	v = vs[0]
-	}
 	writeJSON(w, http.StatusOK, v)
 	return nil
 }

internal/server/h_config.go

@@ -637,7 +637,7 @@ func (s *Server) search(w http.ResponseWriter, r *http.Request, p *auth.Principa
 	}
 	sort.Slice(recs, func(i, j int) bool {
 		if recs[i].Name != recs[j].Name {
-			return model.CanonicalLess(recs[i].Name, recs[j].Name)
+			return model.NameLess(recs[i].Name, recs[j].Name)
 		}
 		return recs[i].ID < recs[j].ID
 	})

internal/server/h_reverse.go

@@ -8,7 +8,8 @@ import (
 )
 
 // zoneFillPTR: POST /zones/{zone}/fill-ptr adds the PTR records a reverse
-// zone lacks for its view's A and AAAA records (docs/reverse.md).
+// zone lacks for its view's A and AAAA records; ?other_views=1 also for
+// the addresses only other views' records use (docs/reverse.md).
 func (s *Server) zoneFillPTR(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
 	st, err := s.state(r)
 	if err != nil {
@@ -18,7 +19,7 @@ func (s *Server) zoneFillPTR(w http.ResponseWriter, r *http.Request, p *auth.Pri
 	if err != nil {
 		return err
 	}
-	return s.mutate(w, r, p, "", []engine.Request{engine.NewRequest(engine.OpZoneFillPTR, z.ID, engine.LinkInput{ZoneID: z.ID})}, nil)
+	return s.mutate(w, r, p, "", []engine.Request{engine.NewRequest(engine.OpZoneFillPTR, z.ID, engine.FillPTRInput{ZoneID: z.ID, OtherViews: queryBool(r, "other_views")})}, nil)
 }
 
 // reverseSuggestions: GET /reverse-zones/suggestions lists the reverse

internal/server/h_stats.go

@@ -215,7 +215,7 @@ func clientNames(st *model.State, want map[netip.Addr]bool, readable func(*model
 	}
 	slices.SortFunc(recs, func(a, b *model.Record) int {
 		if a.Name != b.Name {
-			if model.CanonicalLess(a.Name, b.Name) {
+			if model.NameLess(a.Name, b.Name) {
 				return -1
 			}
 			return 1

internal/server/h_totp.go

@@ -206,6 +206,8 @@ func (s *Server) loginTOTP(w http.ResponseWriter, r *http.Request) error {
 	if err != nil {
 		s.limiter.fail("ip:"+ip, userKey)
 		left := s.pendingLogins.fail(in.Challenge)
+		// For Guard and fail2ban: the address first, so a name can't fake it.
+		s.log.Warn("login failed", "ip", ip, "user", u.Username, "reason", "wrong code: "+err.Error())
 		s.eng.Audit(r.Context(), who, "auth.totp_failed", "user", u.ID, u.Username, "failure", err.Error())
 		msg := "wrong code: use the current code from your authenticator app, or a recovery code"
 		if left == 0 {

internal/server/present.go

@@ -41,11 +41,21 @@ func (s *Server) presentZone(st *model.State, z *model.Zone, counts map[int64]re
 	if pf, ok := dnsname.PrefixFromReverse(z.Name); ok {
 		out.Prefix = pf.String()
 		if z.Type == model.ZonePrimary {
-			fills := engine.PTRFills(st, z.ViewID, z.Name, st.ZoneRecords(z.ID))
+			fills := engine.PTRFills(st, z.ViewID, z.Name, st.ZoneRecords(z.ID), false)
 			out.PTRMissing = len(fills)
 			for _, f := range fills[:min(len(fills), 25)] {
 				out.PTRMissingList = append(out.PTRMissingList, api.PTRFill(f))
 			}
+			if z.ViewID != 0 {
+				for _, f := range engine.PTRFills(st, z.ViewID, z.Name, st.ZoneRecords(z.ID), true) {
+					if f.From == "" {
+						continue
+					}
+					if out.PTROtherViews++; out.PTROtherViews <= 25 {
+						out.PTROtherViewsList = append(out.PTROtherViewsList, api.PTRFill(f))
+					}
+				}
+			}
 		}
 	}
 	for _, o := range st.LinkedWith(z) {

internal/stats/collector.go

@@ -95,6 +95,13 @@ type Status struct {
 		Day     string `json:"day"`
 		Queries string `json:"queries"` // the query log ("" when off)
 	} `json:"retention"`
+	// RetentionSeconds is Retention in seconds, for the web UI's ranges.
+	RetentionSeconds struct {
+		Minute  int64 `json:"minute"`
+		Hour    int64 `json:"hour"`
+		Day     int64 `json:"day"`
+		Queries int64 `json:"queries"` // 0 when the query log is off
+	} `json:"retention_seconds"`
 }
 
 // Collector gathers statistics into a Store.
@@ -164,8 +171,12 @@ func NewCollector(store *Store, opts Options, zones ZoneIndex, log *slog.Logger)
 	c.status.Retention.Minute = humanDuration(opts.RetainMinute)
 	c.status.Retention.Hour = humanDuration(opts.RetainHour)
 	c.status.Retention.Day = humanDuration(opts.RetainDay)
+	c.status.RetentionSeconds.Minute = int64(opts.RetainMinute / time.Second)
+	c.status.RetentionSeconds.Hour = int64(opts.RetainHour / time.Second)
+	c.status.RetentionSeconds.Day = int64(opts.RetainDay / time.Second)
 	if opts.QueryLog && opts.RetainQueries > 0 {
 		c.status.Retention.Queries = humanDuration(opts.RetainQueries)
+		c.status.RetentionSeconds.Queries = int64(opts.RetainQueries / time.Second)
 	}
 	return c
 }

internal/stats/explore.go

@@ -8,7 +8,7 @@ import (
 )
 
 // Exploring the query log (docs/stats.md#exploring): every answer of the
-// last days (stats.query_retention), filtered by any mix of client, name,
+// last days (stats.retention.queries), filtered by any mix of client, name,
 // view, type, result and blocking, broken down by each of them.
 
 // ExploreFilter narrows the query log. Empty fields don't filter. Each
@@ -47,8 +47,9 @@ type Exploration struct {
 	Recent []QueryRow `json:"recent"`
 	// Oldest is the first answer the query log still has.
 	Oldest *time.Time `json:"oldest,omitempty"`
-	// Retention is how long it keeps them.
+	// Retention is how long it keeps them, also in seconds.
 	Retention        string `json:"retention"`
+	RetentionSeconds int64  `json:"retention_seconds"`
 }
 
 // ExplorePoint is one step of the series.
@@ -59,7 +60,7 @@ type ExplorePoint struct {
 }
 
 // ErrNoQueryLog is returned when the query log is off.
-var ErrNoQueryLog = fmt.Errorf("the query log is off: it needs stats.query_log and a stats.query_retention above 0 in /etc/jarl/jarl.yaml (docs/stats.md#exploring)")
+var ErrNoQueryLog = fmt.Errorf("the query log is off: it needs stats.query_log and a stats.retention.queries above 0 in /etc/jarl/jarl.yaml (docs/stats.md#exploring)")
 
 // values splits a filter field into the values it must be one of and the
 // ones (written !VALUE) it must not be: "10.0.0.5, !10.0.0.9".
@@ -178,7 +179,7 @@ func (q *Querier) Explore(ctx context.Context, f ExploreFilter, topN, recent int
 	}
 	from := now.Add(-since).Unix()
 	out := &Exploration{From: time.Unix(from, 0), To: now, By: map[string][]Entry{}, Series: []ExplorePoint{}, Recent: []QueryRow{},
-		Retention: humanDuration(q.Opts.RetainQueries)}
+		Retention: humanDuration(q.Opts.RetainQueries), RetentionSeconds: int64(q.Opts.RetainQueries / time.Second)}
 	cond, args := q.where(f, from)
 	db := q.Store.db
 

internal/stats/store.go

@@ -45,7 +45,7 @@ var schema = []string{
 		PRIMARY KEY (res, ts, kind, key)
 	) WITHOUT ROWID;
 	CREATE TABLE meta (key TEXT PRIMARY KEY, value TEXT NOT NULL);`,
-	// 2: the query log, every answer for a while (stats.query_retention),
+	// 2: the query log, every answer for a while (stats.retention.queries),
 	// for drilling down: what a client asked, who asked for a name
 	// (docs/stats.md#exploring).
 	`CREATE TABLE queries (

internal/validate/records.go

@@ -186,7 +186,7 @@ func (c *checker) checkZoneRecords(z *model.Zone) {
 	for n := range ix.types {
 		names = append(names, n)
 	}
-	sort.Slice(names, func(i, j int) bool { return model.CanonicalLess(names[i], names[j]) })
+	sort.Slice(names, func(i, j int) bool { return model.NameLess(names[i], names[j]) })
 
 	for _, name := range names {
 		types := ix.types[name]

internal/validate/validate.go

@@ -189,7 +189,7 @@ func sortProblems(ps []Problem) {
 			return a.Zone < b.Zone
 		}
 		if a.Name != b.Name {
-			return model.CanonicalLess(a.Name, b.Name)
+			return model.NameLess(a.Name, b.Name)
 		}
 		return a.Message < b.Message
 	})

internal/web/static/assets/pages/explore.js

@@ -16,6 +16,13 @@ const filterWords = { client: "client", name: "name", domain: "domain", view: "v
 const keys = Object.keys(filterWords);
 const ranges = [["1h", 3600], ["6h", 6 * 3600], ["24h", 86400], ["3d", 3 * 86400], ["7d", 7 * 86400], ["30d", 30 * 86400]];
 
+// exploreRange is the longest range here of at most secs ("1h" at least).
+export function exploreRange(secs) {
+  let out = "1h";
+  for (const [k, s] of ranges) if (s <= secs) out = k;
+  return out;
+}
+
 // exploreHref is the explore page for these filters (and range).
 export function exploreHref(filters, since) {
   const q = new URLSearchParams();
@@ -74,7 +81,7 @@ export async function render(el, params, query) {
   }
 
   const parts = [head(f, go, since, ex), filterBar(f, since, ex, allViews)];
-  const maxSecs = retentionSecs(ex.retention);
+  const maxSecs = ex.retention_seconds || 7 * 86400;
   parts.push(h("div", { class: "filter-row", role: "group", "aria-label": "Time range" },
     h("span", { class: "muted" }, "Range"),
     h("div", { class: "segmented" }, ranges.filter(([, s]) => s <= maxSecs).map(([k]) =>
@@ -234,9 +241,3 @@ function stepWord(s) {
   if (s % 3600 === 0) return s === 3600 ? "hour" : s / 3600 + " hours";
   return s === 60 ? "minute" : s / 60 + " minutes";
 }
-
-function retentionSecs(r) {
-  const m = /^(\d+)([dhm])$/.exec(r || "");
-  if (!m) return 7 * 86400;
-  return Number(m[1]) * { d: 86400, h: 3600, m: 60 }[m[2]];
-}

internal/web/static/assets/pages/stats.js

@@ -2,10 +2,27 @@ import { h, mount, fmtTime, relTime, errorBox } from "../ui.js";
 import { get } from "../api.js";
 import { rerender } from "../app.js";
 import { lineChart, chartCard, legend, timeTable, heatmap, heatmapTable, barList, compact, stepSeconds, endLabelWidth } from "../charts.js";
-import { exploreHref } from "./explore.js";
+import { exploreHref, exploreRange } from "./explore.js";
 
-const ranges = [["1h", "1 hour"], ["6h", "6 hours"], ["24h", "24 hours"], ["7d", "7 days"], ["30d", "30 days"], ["90d", "90 days"]];
-const rangeSecs = { "1h": 3600, "6h": 6 * 3600, "24h": 86400, "7d": 7 * 86400, "30d": 30 * 86400, "90d": 90 * 86400 };
+// Ranges: the since value, its words and its button. Only the ones the
+// statistics are kept for are offered (keptSecs).
+const ranges = [["1h", "1 hour"], ["6h", "6 hours"], ["24h", "24 hours"], ["7d", "7 days"], ["30d", "30 days"], ["90d", "90 days"],
+  ["365d", "year", "1y"], ["730d", "2 years", "2y"]];
+const rangeSecs = { "1h": 3600, "6h": 6 * 3600, "24h": 86400, "7d": 7 * 86400, "30d": 30 * 86400, "90d": 90 * 86400, "365d": 365 * 86400, "730d": 730 * 86400 };
+
+// keptSecs is how far back the statistics go: the longest of the minute,
+// hour and day retentions (a server older than retention_seconds: 2 years).
+function keptSecs(status) {
+  const r = status.retention_seconds;
+  return r ? Math.max(r.minute, r.hour, r.day) : 730 * 86400;
+}
+
+// longestWithin is the longest range of at most secs ("1h" at least).
+function longestWithin(secs) {
+  let out = "1h";
+  for (const [k] of ranges) if (rangeSecs[k] <= secs) out = k;
+  return out;
+}
 
 // Failed answers keep the same color whatever the range: color follows
 // the result code, never its rank.
@@ -36,8 +53,9 @@ const perViewGroups = ["qtype", "rcode", "view", "zone", "blocklist"];
 export async function render(el, params, query) {
   const gen = ++generation;
   const want = query && query.get ? query.get("since") : null;
-  const since = rangeSecs[want] ? want : "24h";
   const status = await get("/stats/status");
+  const kept = keptSecs(status);
+  const since = rangeSecs[want] && rangeSecs[want] <= kept ? want : longestWithin(Math.min(86400, kept));
   const head = h("div", { class: "page-head" },
     h("div", { class: "grow" }, h("h1", null, "Statistics"),
       h("div", { class: "muted" }, status.enabled
@@ -50,7 +68,10 @@ export async function render(el, params, query) {
     mount(el, head, disabledPanel());
     return;
   }
-  const heatSince = rangeSecs[since] < 7 * 86400 ? "7d" : since;
+  // The heatmap needs hourly counters: at least a week of them, at most
+  // what is kept.
+  const hourSecs = status.retention_seconds ? status.retention_seconds.hour : 90 * 86400;
+  const heatSince = longestWithin(Math.min(Math.max(rangeSecs[since], 7 * 86400), hourSecs));
   // Which views: all of them unless some are switched off.
   const allViews = status.query_log ? (await get("/views").catch(() => [])).map((v) => v.name) : [];
   const off = viewsOff();
@@ -58,14 +79,14 @@ export async function render(el, params, query) {
   const filtered = allViews.length > 0 && shown.length < allViews.length;
   const vf = filtered ? { views: shown.join(",") } : {};
   if (filtered && !shown.length) {
-    mount(el, head, filterRow(since, allViews, off), h("div", { class: "empty" }, "No views chosen: switch one on above."));
+    mount(el, head, filterRow(since, kept, allViews, off), h("div", { class: "empty" }, "No views chosen: switch one on above."));
     return;
   }
   let data;
   try {
     data = await loadAll(since, heatSince, status.query_log, vf, filtered);
   } catch (e) {
-    mount(el, head, filterRow(since, allViews, off), errorBox(e));
+    mount(el, head, filterRow(since, kept, allViews, off), errorBox(e));
     return;
   }
   if (gen !== generation) return; // the user moved on
@@ -74,7 +95,7 @@ export async function render(el, params, query) {
   // sudden drop in traffic.
   const timeline = complete(data.timeline), errors = complete(data.errors);
   const step = stepSeconds[timeline.step];
-  const parts = [head, filterRow(since, allViews, off)];
+  const parts = [head, filterRow(since, kept, allViews, off)];
   for (const w of status.warnings || []) parts.push(h("div", { class: "msg warn" }, w));
   if (filtered) {
     parts.push(h("div", { class: "msg info small" }, "Showing view" + (shown.length > 1 ? "s " : " ") + shown.join(", ") + ": answers from named's response log" +
@@ -120,7 +141,7 @@ export async function render(el, params, query) {
   // Who asks for what. With the query log, every value leads to the
   // explore page: a client to what it asked, a name to who asked.
   const ql = !!(status.retention || {}).queries;
-  const ex = (filters) => (ql ? (k) => exploreHref(filters(k), exploreSince(since)) : null);
+  const ex = (filters) => (ql ? (k) => exploreHref(filters(k), exploreSince(since, status)) : null);
   if (status.query_log) {
     const t = data.tops;
     parts.push(h("div", { class: "chart-grid" },
@@ -201,11 +222,11 @@ function complete(tl) {
   return tl;
 }
 
-function filterRow(since, views, off) {
+function filterRow(since, kept, views, off) {
   const range = h("div", { class: "filter-row", role: "group", "aria-label": "Time range" },
     h("span", { class: "muted" }, "Range"),
-    h("div", { class: "segmented" }, ranges.map(([k, label]) =>
-      h("a", { href: "#/stats?since=" + k, class: k === since ? "active" : "", "aria-current": k === since ? "true" : null, title: "last " + label }, k))));
+    h("div", { class: "segmented" }, ranges.filter(([k]) => rangeSecs[k] <= kept).map(([k, label, button]) =>
+      h("a", { href: "#/stats?since=" + k, class: k === since ? "active" : "", "aria-current": k === since ? "true" : null, title: "last " + label }, button || k))));
   if (!views || views.length < 2) return range;
   // A toggle per view (together: linked views, say), and all or none.
   const set = (next) => { saveViewsOff(next); rerender(); };
@@ -312,10 +333,11 @@ function disabledPanel() {
       "(never individual queries) in a separate database, by default for 7 days per minute, 90 days per hour and 2 years per day. See docs/stats.md."));
 }
 
-// exploreSince is the Statistics range for the explore page: the query log
-// keeps days, not months.
-function exploreSince(since) {
-  return rangeSecs[since] > 7 * 86400 ? "7d" : since;
+// exploreSince is the Statistics range for the explore page: at most what
+// the query log keeps (days, usually, not months).
+function exploreSince(since, status) {
+  const q = status.retention_seconds ? status.retention_seconds.queries : 7 * 86400;
+  return exploreRange(Math.min(rangeSecs[since], q));
 }
 
 // dbTile is the statistics database's size: what the statistics cost on

internal/web/static/assets/pages/users.js

@@ -195,7 +195,8 @@ async function rotateDialog(t, redraw) {
 function coveredBy(user, grant) {
   if (!user || user.admin) return true;
   const [res, acc] = grant.split(/\s+/)[0].split(":");
-  return (user.grants || []).some((g) => (g.resource === "*" || g.resource === res) && (g.access === "write" || acc === "read"));
+  // "*" never covers certificates (they hand out private keys).
+  return (user.grants || []).some((g) => ((g.resource === "*" && res !== "certificates") || g.resource === res) && (g.access === "write" || acc === "read"));
 }
 
 // newTokenForm makes a token for owner (the user it belongs to). Its

internal/web/static/assets/pages/zone.js

@@ -72,7 +72,21 @@ export async function render(el, params, query) {
         h("span", { class: "mono" }, f.address), " → ", h("span", { class: "mono" }, display(f.target)),
         f.from ? h("span", { class: "muted" }, " (from view " + f.from + ")") : null,
         f.others && f.others.length ? h("span", { class: "muted" }, " (also " + f.others.map(display).join(", ") + ")") : null)),
-        zone.ptr_missing > (zone.ptr_missing_list || []).length ? h("li", { class: "muted" }, "and " + (zone.ptr_missing - zone.ptr_missing_list.length) + " more") : null)) : null;
+        zone.ptr_missing > (zone.ptr_missing_list || []).length ? h("li", { class: "muted" }, "and " + (zone.ptr_missing - zone.ptr_missing_list.length) + " more") : null))
+    : null;
+  // Addresses only other views have records for: not missing here, but
+  // they can be added (publishing those views' names in this one).
+  const others = zone.ptr_other_views_list || [];
+  const fromViews = [...new Set(others.map((f) => f.from))];
+  const otherNote = !zone.disabled && zone.ptr_other_views && can("zones", true) ? h("div", { class: "muted small" },
+    plural(zone.ptr_other_views, "address", "addresses") + " in " + zone.prefix + " " + (zone.ptr_other_views === 1 ? "has" : "have") + " A or AAAA records only in view " + fromViews.join(", ") +
+      ", not in view " + zone.view + ", so " + (zone.ptr_other_views === 1 ? "it isn't" : "they aren't") + " counted as missing here. ",
+    h("button", { class: "small", title: "Clients of view " + zone.view + " will then get these names, which come from view " + fromViews.join(", "), onclick: () => fillPTR(zone, true) },
+      "Add PTR records for " + (zone.ptr_other_views === 1 ? "it" : "them") + " in view " + zone.view),
+    h("ul", { class: "small ptr-missing" }, others.map((f) => h("li", null,
+      h("span", { class: "mono" }, f.address), " → ", h("span", { class: "mono" }, display(f.target)),
+      fromViews.length > 1 ? h("span", { class: "muted" }, " (view " + f.from + ")") : null)),
+      zone.ptr_other_views > others.length ? h("li", { class: "muted" }, "and " + (zone.ptr_other_views - others.length) + " more") : null)) : null;
   const tabBar = h("div", { class: "tabs" }, tabs.map(([k, label]) => h("a", { href: "#/zones/" + id + "/" + k, class: k === tab ? "active" : "" }, label)));
   // The header and tabs show at once. Below them, switching tabs keeps the
   // previous tab's content (faded) until the new one is loaded, rather
@@ -83,7 +97,7 @@ export async function render(el, params, query) {
   holder.classList.add("stale");
   body.className = "zone-body";
   body.dataset.zone = id;
-  mount(el, head, banner, tabBar, holder);
+  mount(el, head, banner, otherNote, tabBar, holder);
   if (tab === "records") await recordsTab(body, zone, query);
   else if (tab === "settings") await settingsTab(body, zone);
   else if (tab === "options") await optionsEditor(body, { scope: "zone", scope_id: zone.id, catalogScope: "zone:" + zone.type, title: "Zone options" });
@@ -951,9 +965,9 @@ async function toggleZone(zone) {
 }
 
 // fillPTR adds the PTR records a reverse zone lacks (docs/reverse.md).
-async function fillPTR(zone) {
+async function fillPTR(zone, otherViews) {
   try {
-    const r = await change("POST", "/zones/" + zone.id + "/fill-ptr");
+    const r = await change("POST", "/zones/" + zone.id + "/fill-ptr", undefined, otherViews ? { query: { other_views: "1" } } : {});
     toast(describeResult(r));
     (r.warnings || []).forEach((w) => toast(w, "warn", 10000));
     rerender();

internal/web/static/assets/pages/zones.js

@@ -103,7 +103,10 @@ export async function newZoneDialog(views, zones, settings, preset = {}) {
   cloneSel.addEventListener("change", syncLink);
   const comment = input();
   const fill = checkbox("Add a PTR record for each address the view's A and AAAA records use in this network", true);
-  const fillRow = field("PTR records", fill, "Names already in your zones, so the reverse zone answers from the start. Leave it off to add them yourself.");
+  // Off unless asked (or a network only other views use is picked): it
+  // publishes one view's host names in another.
+  const fillOther = checkbox("Also for addresses only other views' records use, named from there", false);
+  const fillRow = field("PTR records", h("div", null, fill, views.length ? fillOther : null), "Names already in your zones, so the reverse zone answers from the start. Leave it off to add them yourself.");
   const updateClones = () => {
     // Records can only be copied into a single zone.
     const n = (kind === "reverse" ? reverseZoneFor(name.value).name || "-" : name.value.trim().toLowerCase().replace(/\.$/, "")) + ".";
@@ -206,6 +209,7 @@ export async function newZoneDialog(views, zones, settings, preset = {}) {
           name.value = c.prefix;
           sizeFrom = c.samples.length ? c.samples[0].split(" ")[0] : null;
           if (views.length && c.view) view.value = c.view;
+          if ((c.hidden || "").startsWith("records in ")) { fill.input.checked = true; fillOther.input.checked = true; }
           changed();
         } },
           h("span", { class: "mono" }, c.prefix),
@@ -273,7 +277,10 @@ export async function newZoneDialog(views, zones, settings, preset = {}) {
       if (view.value) data.view = view.value;
       const opts = [];
       const list = (ta) => lines(ta.value).map((l) => l.replace(/;$/, "") + ";").join(" ");
-      if (t === "primary" && kind === "reverse" && fill.input.checked) data.fill = true;
+      if (t === "primary" && kind === "reverse" && fill.input.checked) {
+        data.fill = true;
+        if (fillOther.input.checked) data.fill_other_views = true;
+      }
       if (t === "primary") {
         data.nameservers = lines(ns.value);
         if (contact.value.trim()) data.soa = { rname: contact.value.trim() };

scripts/e2e.sh

@@ -703,6 +703,21 @@ run "fill-ptr" 0 $J zone fill-ptr fdab:99::/48
 expect_dig "filled later: IPv6 PTR" "nas.revtest.test." -x fdab:99:0:10::10
 run "fill-ptr again: nothing to add (exit 6)" 6 $J zone fill-ptr fdab:99::/48
 contains "nothing to add"
+# The same network in another view: internal's hosts aren't missing there,
+# and are named from internal only when asked.
+run "the network in external too, filled" 0 $J zone create 10.99.1.0/24 --view external --fill
+contains "A or AAAA records only in other views"
+run "nothing missing there" 0 $J zone show 1.99.10.in-addr.arpa --view external
+not_contains "Missing PTRs"
+contains "^Other views:  *2 addresses with A or AAAA records only in view internal"
+contains "10.99.1.11 -> pi.revtest.test (view internal)"
+run "fill-ptr there: nothing of its own (exit 6)" 6 $J zone fill-ptr 10.99.1.0/24 --view external
+contains "--other-views"
+run "fill-ptr --other-views" 0 $J zone fill-ptr 10.99.1.0/24 --view external --other-views
+contains "only view internal has a record for it"
+run "now named there" 0 $J record list 1.99.10.in-addr.arpa --view external
+contains "pi.revtest.test"
+run "drop the external copy" 0 $J zone delete 1.99.10.in-addr.arpa --view external --yes
 run "keep PTRs in step" 0 $J zone update revtest.test --view internal --auto-ptr
 run "zone show says so" 0 $J zone show revtest.test --view internal
 contains "^PTR records:  *kept in step"
@@ -736,7 +751,7 @@ run "and back on" 0 $J view update internal --reverse-suggestions
 run "an AAAA only in external" 0 $J record add adbraun.test vps AAAA 2600:3c01::f03c:93ff:fe2b:1 --view external
 run "offered for internal" 0 $J zone suggest-reverse --all --view internal
 contains "2600:3c01::f03c:93ff:fe2b:1/128 .*records in external"
-run "created in internal, filled from external's record" 0 $J zone create 2600:3c01::f03c:93ff:fe2b:1/128 --view internal --fill
+run "created in internal, filled from external's record" 0 $J zone create 2600:3c01::f03c:93ff:fe2b:1/128 --view internal --fill --other-views
 expect_dig "internal answers its PTR" "vps.adbraun.test." -x 2600:3c01::f03c:93ff:fe2b:1
 # A zone created inside another takes the parent's records under it.
 run "a /16 reverse zone" 0 $J zone create 10.88.0.0/16 --view internal
@@ -924,6 +939,23 @@ if [ -n "$PEBBLE" ]; then
 	if [ "$(wc -l < $CGD/hook.log)" = 1 ]; then ok "the hook didn't run again"; else bad "hook ran again: $(cat $CGD/hook.log)"; fi
 	run "cert saved" 0 $J cert saved --config-dir $CGD/conf
 	contains "mail  *mail.adbraun.test .*$CGD/mail"
+	run "the owner's cert list: for jarl cert get, downloaded today" 0 $J $CG cert list
+	contains "^mail  *acme  *get  *mail.adbraun.test  *[0-9-]*  *$(date +%Y-%m-%d)  *ok"
+	run "cert get --combined: written from this host's files" 0 $J cert get mail --combined $CGD/haproxy/mail.pem --config-dir $CGD/conf
+	contains "mail .*combined file written, combined $CGD/haproxy/mail.pem"
+	run "the combined file is mode 600" 0 stat -c %a $CGD/haproxy/mail.pem
+	contains "^600$"
+	run "it holds the key" 0 sh -c "openssl pkey -in $CGD/haproxy/mail.pem -pubout | cmp - $CGD/pub.pem"
+	run "and the certificate" 0 sh -c "openssl x509 -in $CGD/haproxy/mail.pem -noout -serial | cmp - \$(openssl x509 -in $CGD/mail/cert.pem -noout -serial > $CGD/serial; echo $CGD/serial)"
+	if [ "$(wc -l < $CGD/hook.log)" = 2 ]; then ok "the hook ran for the combined file"; else bad "hook runs: $(cat $CGD/hook.log)"; fi
+	run "cert saved NAME shows every setting" 0 $J cert saved mail --config-dir $CGD/conf
+	contains "^Combined: *$CGD/haproxy/mail.pem$"
+	contains "^Hook: *echo"
+	run "a combined path inside the files' directory is refused" 1 $J cert get mail --combined $CGD/mail/fullchain.pem --config-dir $CGD/conf
+	contains "would overwrite mail's fullchain.pem"
+	run "cert get --local: the hook runs from this host's files" 0 $J cert get mail --local --config-dir $CGD/conf
+	contains "mail .*this host's files"
+	if [ "$(wc -l < $CGD/hook.log)" = 3 ]; then ok "the hook ran with --local"; else bad "hook runs: $(cat $CGD/hook.log)"; fi
 	run "another user can't take the name (exit 5)" 5 $J --token @$E/hosts2.token cert get mail -d mail.adbraun.test --dir $CGD/x --no-save --config-dir $CGD/conf2
 	contains "taken"
 	run "an ACME token can't get certificates (exit 3)" 3 $J --token @$E/nocert.token cert get x -d x.adbraun.test --dir $CGD/x --no-save --config-dir $CGD/conf2
@@ -936,6 +968,11 @@ if [ -n "$PEBBLE" ]; then
 	contains "mail .*new files"
 	run "the hook ran for the renewal" 0 tail -1 $CGD/hook.log
 	contains "^mail 1 privkey.pem$"
+	run "the combined file has the renewed certificate" 0 sh -c "openssl x509 -in $CGD/haproxy/mail.pem -noout -serial | cmp - \$(openssl x509 -in $CGD/mail/cert.pem -noout -serial > $CGD/serial; echo $CGD/serial)"
+	run "the owner sees the download is current" 0 $J $CG cert list
+	if echo "$OUT" | grep -q "(old)"; then bad "owner's list says (old)"; else ok "no (old) after the download"; fi
+	run "cert get --combined '': the combined file is removed" 0 $J cert get mail --combined '' --config-dir $CGD/conf
+	if [ ! -e $CGD/haproxy/mail.pem ]; then ok "combined file removed"; else bad "combined file still there"; fi
 	run "a name added: obtained again" 0 $J cert get mail -d mail.adbraun.test -d smtp.adbraun.test --config-dir $CGD/conf
 	run "the new certificate has both names" 0 openssl x509 -in $CGD/mail/cert.pem -noout -ext subjectAltName
 	contains "smtp.adbraun.test"