Compare commits
12
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
3fb159cf4d | ||
|
|
71ebed2b73 | ||
|
|
95125e7f50 | ||
|
|
c10ebe0d38 | ||
|
|
3d1911dd38 | ||
|
|
310ed8061e | ||
|
|
b367e0c830 | ||
|
|
254867c61d | ||
|
|
3ddd32f97f | ||
|
|
0976faba3f | ||
|
|
1d535c8063 | ||
|
|
7f3564c0a5 |
No files matched your search
@@ -8,6 +8,38 @@ All notable changes to Jarl are documented here. The format follows
|
||||
|
||||
### Added
|
||||
|
||||
- **Failed logins in the journal.** A failed login (a wrong password, or
|
||||
a wrong two-factor code) and a refused API token are logged as well as
|
||||
audited, with the address first (`msg="login failed" ip=… user=…`,
|
||||
`msg="token refused" ip=…`), for Guard's `jarl` preset or a fail2ban
|
||||
filter to ban such addresses (docs/operations.md).
|
||||
|
||||
## [0.6.3] - 2026-10-21
|
||||
|
||||
### Added
|
||||
|
||||
- A docs/acme.md section: lego on a host inside a split DNS fails its own
|
||||
propagation check when the challenge is written into the external view
|
||||
only (`JARL_VIEW`, a one-view token, `.../httpreq/view/...`): why, and
|
||||
the ways round it (no view limit, `--dns.propagation.disable-ans` or
|
||||
`--dns.propagation.wait`, Traefik's `propagation.disableANSChecks`).
|
||||
- docs/acme.md and `jarl-lego` show lego 5's commands (`lego run ...`,
|
||||
which also renews; `lego migrate` when upgrading), with lego 4's
|
||||
beside them. Jarl's side is the same for both.
|
||||
|
||||
## [0.6.2] - 2026-10-21
|
||||
|
||||
### Added
|
||||
|
||||
- A `certificates` role (template): `certificates:write`, for tokens and
|
||||
users that get certificates for other hosts with `jarl cert get`. In
|
||||
the web UI's user and token forms it is in "Insert a template…", to
|
||||
narrow with `names=`; in the CLI, `--role certificates`.
|
||||
|
||||
## [0.6.1] - 2026-10-20
|
||||
|
||||
### Added
|
||||
|
||||
- Certificates for other hosts: `jarl cert get NAME -d DOMAIN --dir DIR
|
||||
--hook CMD` on a mail server or proxy has jarld obtain the certificate
|
||||
(and renew it), waits, downloads it when it is newer than the host's
|
||||
@@ -42,6 +74,14 @@ All notable changes to Jarl are documented here. The format follows
|
||||
|
||||
### Changed
|
||||
|
||||
- The Statistics page's range buttons follow `stats.retention`: ranges
|
||||
longer than any data kept are left out, and **1y** and **2y** appear
|
||||
when `retention.day` (or `retention.hour`) covers them (with the
|
||||
default 730d, both do). The busiest-times heatmap says how far back it
|
||||
really goes (at most `retention.hour`), and links to the Explore page
|
||||
keep within `retention.queries` instead of assuming 7 days.
|
||||
`GET /stats/status` and `GET /stats/explore` add `retention_seconds`.
|
||||
|
||||
- A reverse zone's missing PTR records count only its own view's A and
|
||||
AAAA records. An address only another view's records use (an internal
|
||||
host, seen from the vps view's copy of `10.0.10.in-addr.arpa`) is no
|
||||
@@ -97,6 +137,21 @@ All notable changes to Jarl are documented here. The format follows
|
||||
|
||||
### Fixed
|
||||
|
||||
- `jarl cert get NAME` no longer takes over a name this host has saved
|
||||
for another Jarl server, which overwrote that certificate's settings
|
||||
(and, with the default `--dir`, its files), so it stopped being
|
||||
renewed. It is refused, as is a `--dir` holding another certificate's
|
||||
files; `--replace` takes the name or directory over.
|
||||
- The "query log is off" error named a setting that doesn't exist,
|
||||
`stats.query_retention`; it's `stats.retention.queries`.
|
||||
- `jarl zone show` for a reverse zone printed only "and 1 more" under
|
||||
"Other views:" when the server was older than the CLI (0.6.1-rc.2
|
||||
sends the count but not the addresses). It now prints just the count
|
||||
line then.
|
||||
- `jarl changeset show` printed its Created and Applied times in UTC,
|
||||
while `jarl history` and `jarl status` print local time. It uses local
|
||||
time now too.
|
||||
|
||||
- The statistics' "Busiest times" grid drew hours without data taller
|
||||
than the others (they picked up the page's padding for empty
|
||||
messages).
|
||||
|
||||
@@ -202,9 +202,12 @@ review step.
|
||||
|
||||
## Status
|
||||
|
||||
Version 0.6.0 is the current release, and 0.6.1 is in testing
|
||||
(0.6.1-rc.3): it gets certificates onto your other hosts (`jarl cert
|
||||
get`: Jarl obtains and renews them, a host downloads them and runs a
|
||||
Version 0.6.3 is the current release; 0.6.4 is in testing (0.6.4-rc.1):
|
||||
failed logins and refused tokens are logged, address first, for Guard
|
||||
and fail2ban. 0.6.3 brings the lego docs up to
|
||||
lego 5 and explains lego with split DNS. 0.6.2 added a `certificates` role
|
||||
for tokens that get certificates for other hosts. 0.6.1 gets certificates onto your
|
||||
other hosts (`jarl cert get`: Jarl obtains and renews them, a host downloads them and runs a
|
||||
hook, a daily timer keeps them current), exports certificates, fixes
|
||||
PTR records and overrides in linked zones, and makes one-view tokens
|
||||
(such as ACME tokens for the external view) work with linked zones.
|
||||
|
||||
@@ -392,7 +392,8 @@ ACCESS is read or write, and filters take comma separated glob patterns:
|
||||
records:write zones=adbraun.lol names=_acme-challenge.* types=TXT
|
||||
*:read
|
||||
|
||||
Roles are ready-made grant sets: readonly, dns-editor, operator, acme.`
|
||||
Roles are ready-made grant sets: readonly, dns-editor, operator, acme,
|
||||
certificates.`
|
||||
|
||||
func userCmd() *cobra.Command {
|
||||
cmd := &cobra.Command{Use: "user", Aliases: []string{"users"}, Short: "Manage users (admin)", Long: "Manage users (admin).\n\n" + grantHelp}
|
||||
@@ -477,7 +478,7 @@ func userCmd() *cobra.Command {
|
||||
}
|
||||
addUserFlags := func(c *cobra.Command) {
|
||||
c.Flags().BoolVar(&admin, "admin", false, "make the user an administrator (--admin=false to remove)")
|
||||
c.Flags().StringVar(&role, "role", "", "replace the grants with a role: readonly, dns-editor, operator or acme")
|
||||
c.Flags().StringVar(&role, "role", "", "replace the grants with a role: readonly, dns-editor, operator, acme or certificates")
|
||||
c.Flags().StringArrayVar(&grants, "grant", nil, `grant (repeatable), e.g. "records:write zones=example.com"`)
|
||||
c.Flags().StringVar(&comment, "comment", "", "comment")
|
||||
c.Flags().IntVar(&certLimit, "cert-limit", -1, "how many certificates the user may own through jarl cert get (-1: certificates.max_per_user in jarl.yaml)")
|
||||
@@ -758,7 +759,7 @@ func tokenCmd() *cobra.Command {
|
||||
},
|
||||
}
|
||||
create.Flags().StringVar(&user, "user", "", "create the token for this user (admin)")
|
||||
create.Flags().StringVar(&role, "role", "", "restrict the token to a role: readonly, dns-editor, operator or acme")
|
||||
create.Flags().StringVar(&role, "role", "", "restrict the token to a role: readonly, dns-editor, operator, acme or certificates")
|
||||
create.Flags().StringArrayVar(&grants, "grant", nil, "restrict the token to these grants (repeatable)")
|
||||
create.Flags().StringVar(&expires, "expires", "", "lifetime, e.g. 90d or 720h (default: never)")
|
||||
create.Flags().BoolVar(&save, "save", false, "store the new token in the CLI configuration file")
|
||||
|
||||
+36
-2
@@ -114,6 +114,32 @@ func (sc savedCert) save(dir string) error {
|
||||
return writeAtomic(filepath.Join(dir, sc.Name+".json"), append(b, '\n'), 0o600)
|
||||
}
|
||||
|
||||
// checkCertClash refuses a certificate that would take over what this
|
||||
// host keeps for another one: a name saved for another Jarl server
|
||||
// (each server has its own names, so the server can't refuse it), or a
|
||||
// directory with another certificate's files. saved[i] is sc's saved
|
||||
// settings (i < 0: none).
|
||||
func checkCertClash(saved []savedCert, i int, sc savedCert, replace bool) error {
|
||||
if i >= 0 && !replace && !sameServer(saved[i].Conn.Server, sc.Conn.Server) {
|
||||
return fmt.Errorf("%s is saved here for the Jarl server %s, not %s: choose another name, or take it over with --replace", sc.Name, saved[i].Conn.Server, sc.Conn.Server)
|
||||
}
|
||||
for j, o := range saved {
|
||||
if j != i && o.Name != sc.Name && filepath.Clean(o.Dir) == filepath.Clean(sc.Dir) {
|
||||
return fmt.Errorf("%s holds the files of %s, saved for %s: give %s another --dir", sc.Dir, o.Name, o.Conn.Server, sc.Name)
|
||||
}
|
||||
}
|
||||
if m, err := readMeta(sc.Dir); err == nil && m.Name != "" && m.Name != sc.Name && !replace {
|
||||
return fmt.Errorf("%s holds the files of the certificate %s: give %s another --dir, or overwrite them with --replace", sc.Dir, m.Name, sc.Name)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// sameServer reports whether two server addresses name the same jarld.
|
||||
func sameServer(a, b string) bool {
|
||||
norm := func(s string) string { return strings.TrimRight(strings.ToLower(s), "/") }
|
||||
return norm(a) == norm(b)
|
||||
}
|
||||
|
||||
func readMeta(dir string) (*localMeta, error) {
|
||||
b, err := os.ReadFile(filepath.Join(dir, metaFile))
|
||||
if err != nil {
|
||||
@@ -340,7 +366,7 @@ func runHook(sc savedCert, ci certInfo, renewed bool) error {
|
||||
func certGetCmd() *cobra.Command {
|
||||
var domains []string
|
||||
var dir, hook, keyType, configDir string
|
||||
var noSave, force bool
|
||||
var noSave, force, replace bool
|
||||
var wait time.Duration
|
||||
cmd := &cobra.Command{
|
||||
Use: "get [NAME]",
|
||||
@@ -365,6 +391,10 @@ readable only by you). 'jarl cert get' without a name then does every
|
||||
saved certificate: run it daily (jarl cert schedule does that with a
|
||||
systemd timer). Nothing is downloaded until jarld has renewed one.
|
||||
|
||||
A name saved here for another Jarl server isn't taken over, nor a --dir
|
||||
holding another certificate's files, unless --replace is given; a --dir
|
||||
another saved certificate uses is always refused.
|
||||
|
||||
--hook runs with sh after new files are written (and again on the next
|
||||
run if it failed), with JARL_CERT_NAME, JARL_CERT_DIR, JARL_CERT_CERT,
|
||||
JARL_CERT_CHAIN, JARL_CERT_FULLCHAIN, JARL_CERT_KEY, JARL_CERT_DOMAINS,
|
||||
@@ -427,6 +457,9 @@ replaced an older certificate).`,
|
||||
}
|
||||
sc.Conn = cn
|
||||
}
|
||||
if err := checkCertClash(saved, i, sc, replace); err != nil {
|
||||
return err
|
||||
}
|
||||
todo = []savedCert{sc}
|
||||
}
|
||||
var results []*getResult
|
||||
@@ -469,7 +502,7 @@ replaced an older certificate).`,
|
||||
}
|
||||
switch {
|
||||
case len(failed) == 1 && len(todo) == 1:
|
||||
return lastErr // its own exit code (3 forbidden, 6 conflict, ...)
|
||||
return lastErr // its own exit code (3 forbidden, 5 conflict, ...)
|
||||
case len(failed) > 0:
|
||||
return fmt.Errorf("failed: %s", strings.Join(failed, ", "))
|
||||
}
|
||||
@@ -483,6 +516,7 @@ replaced an older certificate).`,
|
||||
f.StringVar(&keyType, "key-type", "", "key type of a new certificate: ec256 (default), ec384, rsa2048, rsa3072, rsa4096")
|
||||
f.BoolVar(&force, "force", false, "download and run the hook even when this host has the current certificate")
|
||||
f.BoolVar(&noSave, "no-save", false, "don't save the settings for later runs")
|
||||
f.BoolVar(&replace, "replace", false, "take over a name saved here for another Jarl server, or a --dir holding another certificate's files")
|
||||
f.DurationVar(&wait, "wait", 15*time.Minute, "how long to wait while jarld obtains it")
|
||||
f.StringVar(&configDir, "config-dir", "", "where the settings are saved (default /etc/jarl/cert-get as root, ~/.config/jarl/cert-get otherwise)")
|
||||
_ = cmd.RegisterFlagCompletionFunc("key-type", cobra.FixedCompletions([]string{"ec256", "ec384", "rsa2048", "rsa3072", "rsa4096"}, cobra.ShellCompDirectiveNoFileComp))
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestCheckCertClash(t *testing.T) {
|
||||
tmp := t.TempDir()
|
||||
a := connection{Server: "https://dns-a.example:8053"}
|
||||
b := connection{Server: "https://dns-b.example:8053"}
|
||||
saved := []savedCert{
|
||||
{Name: "proxy", Dir: filepath.Join(tmp, "proxy"), Conn: a},
|
||||
{Name: "mail", Dir: filepath.Join(tmp, "mail"), Conn: a},
|
||||
}
|
||||
// mail's files are on disk; "other" has files but no saved settings.
|
||||
if err := (&localMeta{Name: "mail"}).write(mkdir(t, saved[1].Dir)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := (&localMeta{Name: "other"}).write(mkdir(t, filepath.Join(tmp, "other"))); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cases := []struct {
|
||||
what string
|
||||
i int
|
||||
sc savedCert
|
||||
replace bool
|
||||
want string // "" = allowed
|
||||
}{
|
||||
{"same server again", 0, savedCert{Name: "proxy", Dir: saved[0].Dir, Conn: a}, false, ""},
|
||||
{"same server, other spelling", 0, savedCert{Name: "proxy", Dir: saved[0].Dir, Conn: connection{Server: "HTTPS://dns-a.example:8053/"}}, false, ""},
|
||||
{"other server", 0, savedCert{Name: "proxy", Dir: saved[0].Dir, Conn: b}, false, "saved here for the Jarl server"},
|
||||
{"other server, replace", 0, savedCert{Name: "proxy", Dir: saved[0].Dir, Conn: b}, true, ""},
|
||||
{"new name, saved cert's dir", -1, savedCert{Name: "web", Dir: saved[1].Dir, Conn: b}, false, "holds the files of mail"},
|
||||
{"new name, saved cert's dir, replace", -1, savedCert{Name: "web", Dir: saved[1].Dir + "/", Conn: b}, true, "holds the files of mail"},
|
||||
{"new name, unsaved cert's files", -1, savedCert{Name: "web", Dir: filepath.Join(tmp, "other"), Conn: b}, false, "certificate other"},
|
||||
{"new name, unsaved cert's files, replace", -1, savedCert{Name: "web", Dir: filepath.Join(tmp, "other"), Conn: b}, true, ""},
|
||||
{"new name, empty dir", -1, savedCert{Name: "web", Dir: filepath.Join(tmp, "web"), Conn: b}, false, ""},
|
||||
}
|
||||
for _, c := range cases {
|
||||
err := checkCertClash(saved, c.i, c.sc, c.replace)
|
||||
switch {
|
||||
case c.want == "" && err != nil:
|
||||
t.Errorf("%s: %v", c.what, err)
|
||||
case c.want != "" && (err == nil || !strings.Contains(err.Error(), c.want)):
|
||||
t.Errorf("%s: got %v, want %q", c.what, err, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func mkdir(t *testing.T, dir string) string {
|
||||
t.Helper()
|
||||
if err := os.MkdirAll(dir, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return dir
|
||||
}
|
||||
+2
-2
@@ -272,9 +272,9 @@ func changesetCmd() *cobra.Command {
|
||||
return err
|
||||
}
|
||||
return output(cs, func(w io.Writer) {
|
||||
fmt.Fprintf(w, "Changeset:\t%d (%s)\nDescription:\t%s\nCreated:\t%s by %s via %s\n", cs.ID, cs.Status, cs.Description, cs.CreatedAt.Format("2006-01-02 15:04"), cs.Username, cs.Via)
|
||||
fmt.Fprintf(w, "Changeset:\t%d (%s)\nDescription:\t%s\nCreated:\t%s by %s via %s\n", cs.ID, cs.Status, cs.Description, cs.CreatedAt.Local().Format("2006-01-02 15:04"), cs.Username, cs.Via)
|
||||
if cs.AppliedAt != nil {
|
||||
fmt.Fprintf(w, "Applied:\t%s by %s\n", cs.AppliedAt.Format("2006-01-02 15:04"), cs.AppliedBy)
|
||||
fmt.Fprintf(w, "Applied:\t%s by %s\n", cs.AppliedAt.Local().Format("2006-01-02 15:04"), cs.AppliedBy)
|
||||
}
|
||||
fmt.Fprintln(w, "\n#\tOPERATION")
|
||||
for _, op := range cs.Ops {
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"jarl/internal/api"
|
||||
)
|
||||
|
||||
func TestPrintPTRFills(t *testing.T) {
|
||||
var b bytes.Buffer
|
||||
printPTRFills(&b, []api.PTRFill{{Address: "10.0.10.20", Target: "nextcloud.adbraun.lol.", From: "internal"}}, 3)
|
||||
out := b.String()
|
||||
if !strings.Contains(out, "10.0.10.20 -> nextcloud.adbraun.lol (view internal)") || !strings.Contains(out, "and 2 more") {
|
||||
t.Errorf("listed fills:\n%s", out)
|
||||
}
|
||||
|
||||
// An older server sends the count without the list.
|
||||
b.Reset()
|
||||
printPTRFills(&b, nil, 1)
|
||||
if b.Len() != 0 {
|
||||
t.Errorf("no list from the server printed %q, want nothing", b.String())
|
||||
}
|
||||
}
|
||||
+6
-1
@@ -1183,8 +1183,13 @@ func terminalWidth() int {
|
||||
}
|
||||
|
||||
// printPTRFills lists PTR records a reverse zone lacks under zone show's
|
||||
// line about them: address -> name, and the view the name is from.
|
||||
// line about them: address -> name, and the view the name is from. A server
|
||||
// older than the CLI may send the count without the list; then nothing is
|
||||
// listed, rather than "and N more" under no addresses.
|
||||
func printPTRFills(w io.Writer, fills []api.PTRFill, total int) {
|
||||
if len(fills) == 0 {
|
||||
return
|
||||
}
|
||||
for _, f := range fills {
|
||||
from := ""
|
||||
if f.From != "" {
|
||||
|
||||
@@ -7,12 +7,17 @@
|
||||
#
|
||||
# Install it, then:
|
||||
# EXEC_PATH=/usr/local/bin/jarl-lego JARL_TOKEN=@/etc/jarl/lego.token \
|
||||
# lego --email you@example.com --dns exec -d example.com -d '*.example.com' run
|
||||
# lego run --accept-tos --email you@example.com --dns exec -d example.com -d '*.example.com'
|
||||
# (lego 4: lego --accept-tos --email ... --dns exec -d ... run, and "renew" to renew)
|
||||
#
|
||||
# Options (environment):
|
||||
# JARL_SERVER Jarl server URL, e.g. https://ns1.example.com:8053 (optional if the jarl CLI is configured)
|
||||
# JARL_TOKEN API token, or @FILE (optional if the jarl CLI is configured)
|
||||
# JARL_VIEW Only write into this view (optional; default every view that has the zone)
|
||||
# On a host inside a split DNS, lego's own propagation check then can't
|
||||
# see the record: leave it unset, or run lego with
|
||||
# --dns.propagation.disable-ans (lego 5; --dns.propagation-disable-ans in lego 4)
|
||||
# (docs/acme.md, "Split DNS and lego's propagation check").
|
||||
# JARL_BIN Path of the jarl binary (optional; default "jarl")
|
||||
|
||||
JARL_BIN="${JARL_BIN:-jarl}"
|
||||
|
||||
+87
-5
@@ -98,10 +98,23 @@ sudo install -m 0755 contrib/lego/jarl-lego.sh /usr/local/bin/jarl-lego # or f
|
||||
export EXEC_PATH=/usr/local/bin/jarl-lego
|
||||
export JARL_SERVER=https://ns1.adbraun.lol:8053
|
||||
export JARL_TOKEN=@/etc/jarl/certbot.token
|
||||
lego --email you@adbraun.lol --dns exec -d adbraun.lol -d '*.adbraun.lol' run
|
||||
lego run --accept-tos --email you@adbraun.lol --dns exec -d adbraun.lol -d '*.adbraun.lol'
|
||||
```
|
||||
|
||||
Renew with the same variables and `lego ... renew` in place of `run`.
|
||||
That is lego 5. `lego run` also renews: run the same command daily (a
|
||||
cron job or systemd timer), and it renews once a third of the
|
||||
certificate's lifetime is left. **lego 4** takes its options before the
|
||||
command, and renews with `renew`:
|
||||
|
||||
```sh
|
||||
lego --accept-tos --email you@adbraun.lol --dns exec -d adbraun.lol -d '*.adbraun.lol' run # first time
|
||||
lego --email you@adbraun.lol --dns exec -d adbraun.lol -d '*.adbraun.lol' renew # then daily
|
||||
```
|
||||
|
||||
Upgrading from lego 4 to 5, run `lego migrate` once (with `--path` if you
|
||||
keep lego's data somewhere other than `./.lego`) before anything else:
|
||||
lego 5 stores accounts and certificates differently. The exec and httpreq
|
||||
providers, and so Jarl's side, are the same in both.
|
||||
|
||||
* lego runs `jarl-lego present FQDN VALUE` and `jarl-lego cleanup FQDN VALUE`,
|
||||
and the script passes them to `jarl acme set|unset --exact`. FQDN is the
|
||||
@@ -114,7 +127,66 @@ Renew with the same variables and `lego ... renew` in place of `run`.
|
||||
* Leave `EXEC_MODE` unset: the script needs lego's default mode, not `RAW`.
|
||||
* lego waits until the record is visible on the zone's name servers
|
||||
(`EXEC_PROPAGATION_TIMEOUT`, 60 seconds by default). Jarl answers only
|
||||
once named serves the record, so this check passes on its first try.
|
||||
once named serves the record, so this check passes on its first try,
|
||||
as long as the view that answers lego's host has the record: with
|
||||
`JARL_VIEW` on a host inside your network it doesn't. See
|
||||
[Split DNS and lego's propagation check](#split-dns-and-legos-propagation-check).
|
||||
|
||||
### Split DNS and lego's propagation check
|
||||
|
||||
Before lego asks the CA to look, it checks the record itself, asking the
|
||||
zone's name servers (`ns1.adbraun.lol`) directly. With views, they answer
|
||||
lego's host from the view that matches *it*. On a host inside your
|
||||
network that is the internal view. If the challenge was written into
|
||||
another view only (`JARL_VIEW=external`, `--view external`, a token
|
||||
limited to `views=external`, or the httpreq endpoint
|
||||
`.../httpreq/view/external`), lego never sees it, and gives up after
|
||||
its propagation timeout without the CA having looked:
|
||||
|
||||
```
|
||||
propagation: time limit exceeded: last error: authoritative nameservers:
|
||||
NS ns1.adbraun.lol.:53 returned NXDOMAIN for _acme-challenge.stats.adbraun.lol.
|
||||
```
|
||||
|
||||
A name whose authorization is still valid from an earlier run ("authorization
|
||||
already valid; skipping challenge") isn't checked at all, so one name can
|
||||
work and the next fail with the same settings.
|
||||
|
||||
The ways round it, best first:
|
||||
|
||||
1. **Don't limit the view.** Leave `JARL_VIEW` unset (and give the token
|
||||
no `views=` filter, the httpreq endpoint no `/view/...`): the record
|
||||
goes into every view that has the zone, the internal one too, and
|
||||
lego's check passes at once. The challenge is a random value that is
|
||||
useless once the CA has checked it, and it is removed straight after.
|
||||
2. **Skip lego's check of the name servers.** The record is already
|
||||
served when Jarl answers, so lego's check adds nothing:
|
||||
|
||||
```sh
|
||||
lego run --dns exec --dns.propagation.disable-ans ... # lego 5
|
||||
lego --dns exec --dns.propagation-disable-ans ... run # lego 4.20 to 4.x
|
||||
lego --dns exec --dns.disable-cp ... run # older lego 4
|
||||
```
|
||||
|
||||
or replace the checks with a fixed wait, which also gives secondary
|
||||
name servers time to catch up: `--dns.propagation.wait 30s` (lego 5;
|
||||
`--dns.propagation-wait 30s` in lego 4.20 and later). In lego 5 these
|
||||
can also be set in the environment: `LEGO_DNS_PROPAGATION_DISABLE_ANS=true`,
|
||||
`LEGO_DNS_PROPAGATION_WAIT=30s`.
|
||||
3. **In Traefik**, the same under `dnsChallenge` (Traefik 3.2 and later;
|
||||
older versions have `disablePropagationCheck: true`):
|
||||
|
||||
```yaml
|
||||
dnsChallenge:
|
||||
provider: httpreq
|
||||
propagation:
|
||||
disableANSChecks: true # or delayBeforeChecks: 30s
|
||||
```
|
||||
4. **Run lego where the external view answers it**, such as a host outside
|
||||
your network. Rarely the easy option.
|
||||
|
||||
certbot (section 3) doesn't check by itself, and acme.sh (section 4)
|
||||
checks through public DNS-over-HTTPS resolvers, so neither runs into this.
|
||||
|
||||
## 6. lego and Traefik over HTTP (httpreq), without the jarl CLI
|
||||
|
||||
@@ -299,14 +371,20 @@ of the `acme` role.
|
||||
export HTTPREQ_ENDPOINT=https://ns1.adbraun.lol:8053/api/v1/acme/httpreq
|
||||
export HTTPREQ_USERNAME=lego
|
||||
export HTTPREQ_PASSWORD_FILE=/etc/jarl/lego.token # or HTTPREQ_PASSWORD=jarl_...
|
||||
lego --email you@adbraun.lol --dns httpreq -d adbraun.lol -d '*.adbraun.lol' run
|
||||
lego run --accept-tos --email you@adbraun.lol --dns httpreq -d adbraun.lol -d '*.adbraun.lol'
|
||||
```
|
||||
|
||||
That is lego 5; in lego 4 the options go before the command (`lego
|
||||
--email ... --dns httpreq -d ... run`, and `renew` to renew). See
|
||||
[section 5](#5-lego).
|
||||
|
||||
Leave `HTTPREQ_MODE` unset (its `RAW` mode sends a different body, which
|
||||
jarld refuses with a message saying so). If jarld's certificate doesn't come from a public
|
||||
CA, lego has to trust yours: `SSL_CERT_FILE=/path/to/your-ca.pem` (lego's
|
||||
`LEGO_CA_CERTIFICATES` covers only the ACME server). To write into one view only, put
|
||||
it in the endpoint: `.../api/v1/acme/httpreq/view/external`.
|
||||
it in the endpoint: `.../api/v1/acme/httpreq/view/external`; on a host inside
|
||||
your network, read [Split DNS and lego's propagation check](#split-dns-and-legos-propagation-check)
|
||||
first.
|
||||
|
||||
### 4. Traefik
|
||||
|
||||
@@ -412,4 +490,8 @@ the main zone is never modified by automation.
|
||||
`dig @ns1.adbraun.lol _acme-challenge.adbraun.lol TXT`. With views, make
|
||||
sure the record is in the view that matches the CA (it is by default)
|
||||
and that `allow-query` lets the internet query the zone.
|
||||
* lego says `propagation: time limit exceeded ... returned NXDOMAIN for
|
||||
_acme-challenge...`: the record is in a view that doesn't answer lego's
|
||||
host, usually `JARL_VIEW=external` on a host inside your network. See
|
||||
[Split DNS and lego's propagation check](#split-dns-and-legos-propagation-check).
|
||||
* `jarl audit --user certbot` shows every attempt.
|
||||
+2
-2
@@ -579,12 +579,12 @@ Needs `stats:read`. When statistics are disabled, every endpoint but
|
||||
|
||||
| Method | Path | Query | Returns |
|
||||
|--------|------|-------|---------|
|
||||
| GET | `/stats/status` | | `{enabled, query_log, statistics_channel, log_file, last_poll, last_log_line, log_lines_read, first_data, retention, warnings}` |
|
||||
| GET | `/stats/status` | | `{enabled, query_log, statistics_channel, log_file, last_poll, last_log_line, log_lines_read, first_data, retention, retention_seconds, warnings}`: `retention` has `minute`, `hour`, `day`, `queries` as `"7d"`, `retention_seconds` the same in seconds (`queries` 0 without a query log) |
|
||||
| GET | `/stats/summary` | `since` (90m, 24h, 7d, 2w; default 24h), `top` (default 10) | totals, `avg_qps`, `peak`, `rcodes`, `qtypes`, `views`, `zones`, `answers`, `transport`, `cache_hit_ratio`, `nxdomain_ratio` (without blocked answers), `blocked`, `blocked_ratio`, `blocklists` (blocked per list; `rules` for your rules), `tops` |
|
||||
| GET | `/stats/timeline` | `since`, `step` (auto, 1m, 5m, 15m, 1h, 6h, 1d), `key` (repeatable or comma separated; `rcode:*` = every counter of a group; default `queries`) | `{step, times: [unix...], series: [{key, values}]}` |
|
||||
| GET | `/stats/top` | `kind` (name, client, nxdomain, servfail, refused, blocked, blocked_client), `since`, `limit` (default 20) | `[{key, count}]`; client, refused and blocked_client entries add `name`, the host name of Jarl's record for the address (the summary's client list too) |
|
||||
| GET | `/stats/heatmap` | `since` (default 28d), `key` (default queries) | `{key, from, avg[7][24], samples[7][24]}`, Monday first, local time, `-1` = no data |
|
||||
| GET | `/stats/explore` | `?client=&name=&domain=&view=&type=&rcode=&blocked=yes\|no\|LIST&since=24h&views=`; each takes several values, comma separated (any of them), `!value` to leave one out, and names `*.example.com` for it and below | The query log for a filter ([stats.md](stats.md#exploring)): `total`, `blocked`, `series` (per `step` seconds: `t`, `count`, `blocked`), `by` (top 15 of each dimension the filter leaves open: client, name, view, type, rcode, blocked; clients with `name`), `recent` (latest 100: ts, client, client_name, view, name, type, rcode, blocked), `oldest`, `retention`, `client_name`. 404 `query_log_off` without a query log |
|
||||
| GET | `/stats/explore` | `?client=&name=&domain=&view=&type=&rcode=&blocked=yes\|no\|LIST&since=24h&views=`; each takes several values, comma separated (any of them), `!value` to leave one out, and names `*.example.com` for it and below | The query log for a filter ([stats.md](stats.md#exploring)): `total`, `blocked`, `series` (per `step` seconds: `t`, `count`, `blocked`), `by` (top 15 of each dimension the filter leaves open: client, name, view, type, rcode, blocked; clients with `name`), `recent` (latest 100: ts, client, client_name, view, name, type, rcode, blocked), `oldest`, `retention` (`"7d"`) and `retention_seconds`, `client_name`. 404 `query_log_off` without a query log |
|
||||
| GET | `/stats/server` | | `{server: {rss, mem_in_use, cpu_percent, threads, pid, cache_rrsets, cache_mem, sockets_udp, sockets_tcp, recursion_highwater, tcp_highwater, version, boot_time, config_time, host: {mem_total, mem_available, load, cpus, uptime}}, named: {recursive_clients, tcp_clients, zones, cpus, worker_threads}, disks: [{path, holds, total, free}], database: {path, bytes, query_log_rows}}`; gauges over time: `/stats/timeline?key=g:named:rss` (averages, -1 without a sample) |
|
||||
| GET | `/stats/breakdown` | `group` (qtype, rcode, view, zone, answer, transport, rtt, dnssec, cache, ratelimit, blocklist), `since` | `[{key, count}]` |
|
||||
|
||||
|
||||
@@ -292,6 +292,13 @@ jarl cert saved # what this host has, and when it exp
|
||||
`jarl cert get` without a name does every saved certificate. A token
|
||||
given as `@FILE` is saved as that reference, not copied. `jarl cert
|
||||
forget NAME` stops getting one (`--delete` deletes it on jarld too).
|
||||
* **Several Jarl servers on one host**: each server has its own
|
||||
certificate names, so two servers may both have a `proxy`. The host
|
||||
keeps one `proxy` per settings directory, though: asking another server
|
||||
for a name saved here for the first one is refused, as is a `--dir`
|
||||
holding another certificate's files. Choose another name or `--dir`, or
|
||||
pass `--replace` to take the name (or directory) over. A `--dir` that
|
||||
another saved certificate uses is always refused.
|
||||
* **The hook** runs with `sh` after new files are written, with
|
||||
`JARL_CERT_NAME`, `JARL_CERT_DIR`, `JARL_CERT_CERT`, `JARL_CERT_CHAIN`,
|
||||
`JARL_CERT_FULLCHAIN`, `JARL_CERT_KEY`, `JARL_CERT_DOMAINS`,
|
||||
|
||||
@@ -451,6 +451,7 @@ jarl cert export mail --dir /etc/ssl/mail # fullchain.pem and privkey
|
||||
jarl cert export mail --part bundle # one part to stdout: fullchain, cert, key, bundle
|
||||
jarl cert get mail -d mail.adbraun.lol --hook 'systemctl reload postfix' # on another host: get, download, hook
|
||||
jarl cert get # every saved certificate (what the timer runs)
|
||||
jarl cert get proxy -s other --replace # take over a name saved here for another server
|
||||
jarl cert saved # this host's certificates and their expiry
|
||||
jarl cert forget mail [--delete] # stop getting it here
|
||||
sudo jarl cert schedule [--remove] # daily systemd timer for jarl cert get
|
||||
@@ -536,6 +537,7 @@ jarl token list --all --expiring [60d] # expiring within 14 days
|
||||
jarl token create laptop --expires 365d
|
||||
jarl token create backup-script --grant "*:read"
|
||||
jarl token create acme --user certbot --role acme
|
||||
jarl token create web01 --grant "certificates:write names=web01.adbraun.lol" # jarl cert get on one host (or --role certificates)
|
||||
jarl token rotate laptop --save # new secret, same token and grants
|
||||
jarl token rotate --current --save --grace 10m # the token this CLI uses
|
||||
jarl token revoke 7
|
||||
|
||||
@@ -360,6 +360,23 @@ cp /etc/bind/named.conf.pre-jarl-<date> /etc/bind/named.conf
|
||||
rndc reconfig
|
||||
```
|
||||
|
||||
## Failed logins in the journal
|
||||
|
||||
A failed login to the web UI or the API (a wrong password, or a wrong
|
||||
two-factor code), and a refused API token, are written to the journal as
|
||||
well as the audit log, with the address first:
|
||||
|
||||
```
|
||||
level=WARN msg="login failed" ip=203.0.113.9 user=admin reason="wrong username or password"
|
||||
level=WARN msg="token refused" ip=203.0.113.9 reason="wrong token secret"
|
||||
```
|
||||
|
||||
Guard's `jarl` preset, or a fail2ban filter, can ban such addresses at
|
||||
the firewall (with Housecarl's ban lists). The address comes before the
|
||||
user name, which a caller chooses, so a pattern anchored on
|
||||
`msg="login failed" ip=` can't be fooled by a name. With `log.format:
|
||||
json` they are JSON objects with the same fields in the same order.
|
||||
|
||||
## Monitoring
|
||||
|
||||
* `GET /healthz` (no authentication) reports that jarld is up.
|
||||
|
||||
@@ -185,6 +185,7 @@ you can then edit.
|
||||
| `dns-editor` | `zones:write`, `records:write`, `views:read`, `acls:read`, `options:read`, `blocking:read` |
|
||||
| `operator` | `*:write` (everything except user management) |
|
||||
| `acme` | `records:write names=_acme-challenge.* types=TXT` |
|
||||
| `certificates` | `certificates:write`: certificates for other hosts (`jarl cert get`) for any name in Jarl's zones. Narrow it to a host's names, e.g. `certificates:write names=web01.adbraun.lol` ([certificates.md](certificates.md#certificates-for-other-hosts)) |
|
||||
|
||||
```sh
|
||||
jarl user create alice --role dns-editor
|
||||
@@ -210,6 +211,7 @@ The audit log shows the user and the token name.
|
||||
jarl token create laptop # full rights of the user, no expiry
|
||||
jarl token create ci --grant "records:write zones=lab.adbraun.lol" --expires 90d
|
||||
jarl token create certbot --user certbot --role acme # admin: a token for another user
|
||||
jarl token create web01 --grant "certificates:write names=web01.adbraun.lol" # jarl cert get on one host
|
||||
jarl token list --all
|
||||
jarl token revoke 12
|
||||
```
|
||||
|
||||
+5
-1
@@ -263,7 +263,11 @@ database, roughly in proportion to the number of views.
|
||||
lists for `retention.hour`, daily ones for `retention.day`. Older data is
|
||||
deleted every hour.
|
||||
- Charts use the finest resolution that covers the range: minutes up to 48
|
||||
hours, hours up to 90 days, days beyond.
|
||||
hours (and `retention.minute`), hours up to `retention.hour`, days beyond.
|
||||
- The Statistics page offers only the ranges the data is kept for: up to 90
|
||||
days with the defaults, plus a year and 2 years when `retention.day` (or
|
||||
`retention.hour`) is that long. The Explore page offers the ranges up to
|
||||
`retention.queries`.
|
||||
- Top lists are exact within an hour (up to 200,000 distinct names or
|
||||
clients per hour, beyond which the rarest are dropped). Daily lists are
|
||||
built from the hourly top lists, so for very long tails they are
|
||||
|
||||
@@ -135,6 +135,16 @@ func TestRoleTemplatesValid(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestCertificatesRole(t *testing.T) {
|
||||
p := &Principal{User: &model.User{Grants: RoleTemplates["certificates"]}}
|
||||
if !p.Can(Action{Resource: ResCerts, Write: true, Name: "web01.adbraun.lol"}) {
|
||||
t.Error("the certificates role can't get a certificate")
|
||||
}
|
||||
if p.Can(Action{Resource: ResRecords, Write: true, Zone: "adbraun.lol."}) || p.Can(Action{Resource: ResZones, Zone: "adbraun.lol."}) {
|
||||
t.Error("the certificates role reaches beyond certificates")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCertificateGrants(t *testing.T) {
|
||||
g, err := ParseGrant("certificates:write names=*.adbraun.lol")
|
||||
if err != nil {
|
||||
|
||||
@@ -391,4 +391,7 @@ var RoleTemplates = map[string][]model.Grant{
|
||||
},
|
||||
"operator": {{Resource: ResAll, Access: Write}},
|
||||
"acme": {{Resource: ResRecords, Access: Write, Names: []string{"_acme-challenge.*"}, Types: []string{"TXT"}}},
|
||||
// certificates for other hosts (jarl cert get): narrow it with names=
|
||||
// to the host's own.
|
||||
"certificates": {{Resource: ResCerts, Access: Write}},
|
||||
}
|
||||
@@ -0,0 +1,23 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"log/slog"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// TestFailedLoginLogged: a failed login and a refused token are logged
|
||||
// with the address first, for Guard's and fail2ban's patterns.
|
||||
func TestFailedLoginLogged(t *testing.T) {
|
||||
e := newAPIEnv(t)
|
||||
var buf bytes.Buffer
|
||||
e.srv.log = slog.New(slog.NewTextHandler(&buf, nil))
|
||||
e.req("POST", "/auth/login", "", map[string]string{"username": "x ip=8.8.8.8", "password": "wrong"})
|
||||
e.req("GET", "/zones", "jarl_aaaaaaaaaaaaaaaa_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", nil)
|
||||
for _, want := range []string{`msg="login failed" ip=127.0.0.1 user="x ip=8.8.8.8"`, `msg="token refused" ip=127.0.0.1 reason=`} {
|
||||
if !strings.Contains(buf.String(), want) {
|
||||
t.Errorf("no %q in\n%s", want, buf.String())
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -136,6 +136,8 @@ func (s *Server) authenticate(r *http.Request) (*auth.Principal, *model.Session,
|
||||
p, err := s.tokenPrincipal(r.Context(), strings.TrimSpace(tok), ip, r.UserAgent())
|
||||
if err != nil {
|
||||
s.limiter.fail("ip:" + ip)
|
||||
// For Guard and fail2ban: the address first, so a name can't fake it.
|
||||
s.log.Warn("token refused", "ip", ip, "reason", err.Error())
|
||||
s.eng.Audit(r.Context(), &auth.Principal{IP: ip, Agent: r.UserAgent(), Method: "token"}, "auth.token_failed", "", 0, "", "failure", err.Error())
|
||||
return nil, nil, &httpError{http.StatusUnauthorized, "unauthenticated", "invalid API token: " + err.Error()}
|
||||
}
|
||||
@@ -328,6 +330,8 @@ func (s *Server) login(w http.ResponseWriter, r *http.Request) error {
|
||||
if err == nil && u.Disabled && ok {
|
||||
reason = "account disabled"
|
||||
}
|
||||
// For Guard and fail2ban: the address first, so a name can't fake it.
|
||||
s.log.Warn("login failed", "ip", ip, "user", name, "reason", reason)
|
||||
s.eng.Audit(r.Context(), &auth.Principal{User: &model.User{Username: name}, IP: ip, Agent: r.UserAgent(), Method: "session"}, "auth.login_failed", "user", 0, name, "failure", reason)
|
||||
return &httpError{http.StatusUnauthorized, "login_failed", "wrong username or password"}
|
||||
}
|
||||
|
||||
@@ -198,6 +198,8 @@ func (s *Server) httpreqPrincipal(w http.ResponseWriter, r *http.Request) (*auth
|
||||
}
|
||||
fail := func(reason string) error {
|
||||
s.limiter.fail("ip:" + ip)
|
||||
// For Guard and fail2ban: the address first, so a name can't fake it.
|
||||
s.log.Warn("token refused", "ip", ip, "reason", "httpreq: "+reason)
|
||||
s.eng.Audit(r.Context(), &auth.Principal{User: &model.User{Username: user}, IP: ip, Agent: r.UserAgent(), Method: "token"}, "auth.token_failed", "", 0, "", "failure", "httpreq: "+reason)
|
||||
w.Header().Set("WWW-Authenticate", httpreqRealm)
|
||||
return &httpError{http.StatusUnauthorized, "unauthenticated", "invalid credentials: " + reason}
|
||||
|
||||
@@ -206,6 +206,8 @@ func (s *Server) loginTOTP(w http.ResponseWriter, r *http.Request) error {
|
||||
if err != nil {
|
||||
s.limiter.fail("ip:"+ip, userKey)
|
||||
left := s.pendingLogins.fail(in.Challenge)
|
||||
// For Guard and fail2ban: the address first, so a name can't fake it.
|
||||
s.log.Warn("login failed", "ip", ip, "user", u.Username, "reason", "wrong code: "+err.Error())
|
||||
s.eng.Audit(r.Context(), who, "auth.totp_failed", "user", u.ID, u.Username, "failure", err.Error())
|
||||
msg := "wrong code: use the current code from your authenticator app, or a recovery code"
|
||||
if left == 0 {
|
||||
|
||||
@@ -95,6 +95,13 @@ type Status struct {
|
||||
Day string `json:"day"`
|
||||
Queries string `json:"queries"` // the query log ("" when off)
|
||||
} `json:"retention"`
|
||||
// RetentionSeconds is Retention in seconds, for the web UI's ranges.
|
||||
RetentionSeconds struct {
|
||||
Minute int64 `json:"minute"`
|
||||
Hour int64 `json:"hour"`
|
||||
Day int64 `json:"day"`
|
||||
Queries int64 `json:"queries"` // 0 when the query log is off
|
||||
} `json:"retention_seconds"`
|
||||
}
|
||||
|
||||
// Collector gathers statistics into a Store.
|
||||
@@ -164,8 +171,12 @@ func NewCollector(store *Store, opts Options, zones ZoneIndex, log *slog.Logger)
|
||||
c.status.Retention.Minute = humanDuration(opts.RetainMinute)
|
||||
c.status.Retention.Hour = humanDuration(opts.RetainHour)
|
||||
c.status.Retention.Day = humanDuration(opts.RetainDay)
|
||||
c.status.RetentionSeconds.Minute = int64(opts.RetainMinute / time.Second)
|
||||
c.status.RetentionSeconds.Hour = int64(opts.RetainHour / time.Second)
|
||||
c.status.RetentionSeconds.Day = int64(opts.RetainDay / time.Second)
|
||||
if opts.QueryLog && opts.RetainQueries > 0 {
|
||||
c.status.Retention.Queries = humanDuration(opts.RetainQueries)
|
||||
c.status.RetentionSeconds.Queries = int64(opts.RetainQueries / time.Second)
|
||||
}
|
||||
return c
|
||||
}
|
||||
|
||||
@@ -8,7 +8,7 @@ import (
|
||||
)
|
||||
|
||||
// Exploring the query log (docs/stats.md#exploring): every answer of the
|
||||
// last days (stats.query_retention), filtered by any mix of client, name,
|
||||
// last days (stats.retention.queries), filtered by any mix of client, name,
|
||||
// view, type, result and blocking, broken down by each of them.
|
||||
|
||||
// ExploreFilter narrows the query log. Empty fields don't filter. Each
|
||||
@@ -47,8 +47,9 @@ type Exploration struct {
|
||||
Recent []QueryRow `json:"recent"`
|
||||
// Oldest is the first answer the query log still has.
|
||||
Oldest *time.Time `json:"oldest,omitempty"`
|
||||
// Retention is how long it keeps them.
|
||||
// Retention is how long it keeps them, also in seconds.
|
||||
Retention string `json:"retention"`
|
||||
RetentionSeconds int64 `json:"retention_seconds"`
|
||||
}
|
||||
|
||||
// ExplorePoint is one step of the series.
|
||||
@@ -59,7 +60,7 @@ type ExplorePoint struct {
|
||||
}
|
||||
|
||||
// ErrNoQueryLog is returned when the query log is off.
|
||||
var ErrNoQueryLog = fmt.Errorf("the query log is off: it needs stats.query_log and a stats.query_retention above 0 in /etc/jarl/jarl.yaml (docs/stats.md#exploring)")
|
||||
var ErrNoQueryLog = fmt.Errorf("the query log is off: it needs stats.query_log and a stats.retention.queries above 0 in /etc/jarl/jarl.yaml (docs/stats.md#exploring)")
|
||||
|
||||
// values splits a filter field into the values it must be one of and the
|
||||
// ones (written !VALUE) it must not be: "10.0.0.5, !10.0.0.9".
|
||||
@@ -178,7 +179,7 @@ func (q *Querier) Explore(ctx context.Context, f ExploreFilter, topN, recent int
|
||||
}
|
||||
from := now.Add(-since).Unix()
|
||||
out := &Exploration{From: time.Unix(from, 0), To: now, By: map[string][]Entry{}, Series: []ExplorePoint{}, Recent: []QueryRow{},
|
||||
Retention: humanDuration(q.Opts.RetainQueries)}
|
||||
Retention: humanDuration(q.Opts.RetainQueries), RetentionSeconds: int64(q.Opts.RetainQueries / time.Second)}
|
||||
cond, args := q.where(f, from)
|
||||
db := q.Store.db
|
||||
|
||||
|
||||
@@ -45,7 +45,7 @@ var schema = []string{
|
||||
PRIMARY KEY (res, ts, kind, key)
|
||||
) WITHOUT ROWID;
|
||||
CREATE TABLE meta (key TEXT PRIMARY KEY, value TEXT NOT NULL);`,
|
||||
// 2: the query log, every answer for a while (stats.query_retention),
|
||||
// 2: the query log, every answer for a while (stats.retention.queries),
|
||||
// for drilling down: what a client asked, who asked for a name
|
||||
// (docs/stats.md#exploring).
|
||||
`CREATE TABLE queries (
|
||||
|
||||
@@ -16,6 +16,13 @@ const filterWords = { client: "client", name: "name", domain: "domain", view: "v
|
||||
const keys = Object.keys(filterWords);
|
||||
const ranges = [["1h", 3600], ["6h", 6 * 3600], ["24h", 86400], ["3d", 3 * 86400], ["7d", 7 * 86400], ["30d", 30 * 86400]];
|
||||
|
||||
// exploreRange is the longest range here of at most secs ("1h" at least).
|
||||
export function exploreRange(secs) {
|
||||
let out = "1h";
|
||||
for (const [k, s] of ranges) if (s <= secs) out = k;
|
||||
return out;
|
||||
}
|
||||
|
||||
// exploreHref is the explore page for these filters (and range).
|
||||
export function exploreHref(filters, since) {
|
||||
const q = new URLSearchParams();
|
||||
@@ -74,7 +81,7 @@ export async function render(el, params, query) {
|
||||
}
|
||||
|
||||
const parts = [head(f, go, since, ex), filterBar(f, since, ex, allViews)];
|
||||
const maxSecs = retentionSecs(ex.retention);
|
||||
const maxSecs = ex.retention_seconds || 7 * 86400;
|
||||
parts.push(h("div", { class: "filter-row", role: "group", "aria-label": "Time range" },
|
||||
h("span", { class: "muted" }, "Range"),
|
||||
h("div", { class: "segmented" }, ranges.filter(([, s]) => s <= maxSecs).map(([k]) =>
|
||||
@@ -234,9 +241,3 @@ function stepWord(s) {
|
||||
if (s % 3600 === 0) return s === 3600 ? "hour" : s / 3600 + " hours";
|
||||
return s === 60 ? "minute" : s / 60 + " minutes";
|
||||
}
|
||||
|
||||
function retentionSecs(r) {
|
||||
const m = /^(\d+)([dhm])$/.exec(r || "");
|
||||
if (!m) return 7 * 86400;
|
||||
return Number(m[1]) * { d: 86400, h: 3600, m: 60 }[m[2]];
|
||||
}
|
||||
@@ -2,10 +2,27 @@ import { h, mount, fmtTime, relTime, errorBox } from "../ui.js";
|
||||
import { get } from "../api.js";
|
||||
import { rerender } from "../app.js";
|
||||
import { lineChart, chartCard, legend, timeTable, heatmap, heatmapTable, barList, compact, stepSeconds, endLabelWidth } from "../charts.js";
|
||||
import { exploreHref } from "./explore.js";
|
||||
import { exploreHref, exploreRange } from "./explore.js";
|
||||
|
||||
const ranges = [["1h", "1 hour"], ["6h", "6 hours"], ["24h", "24 hours"], ["7d", "7 days"], ["30d", "30 days"], ["90d", "90 days"]];
|
||||
const rangeSecs = { "1h": 3600, "6h": 6 * 3600, "24h": 86400, "7d": 7 * 86400, "30d": 30 * 86400, "90d": 90 * 86400 };
|
||||
// Ranges: the since value, its words and its button. Only the ones the
|
||||
// statistics are kept for are offered (keptSecs).
|
||||
const ranges = [["1h", "1 hour"], ["6h", "6 hours"], ["24h", "24 hours"], ["7d", "7 days"], ["30d", "30 days"], ["90d", "90 days"],
|
||||
["365d", "year", "1y"], ["730d", "2 years", "2y"]];
|
||||
const rangeSecs = { "1h": 3600, "6h": 6 * 3600, "24h": 86400, "7d": 7 * 86400, "30d": 30 * 86400, "90d": 90 * 86400, "365d": 365 * 86400, "730d": 730 * 86400 };
|
||||
|
||||
// keptSecs is how far back the statistics go: the longest of the minute,
|
||||
// hour and day retentions (a server older than retention_seconds: 2 years).
|
||||
function keptSecs(status) {
|
||||
const r = status.retention_seconds;
|
||||
return r ? Math.max(r.minute, r.hour, r.day) : 730 * 86400;
|
||||
}
|
||||
|
||||
// longestWithin is the longest range of at most secs ("1h" at least).
|
||||
function longestWithin(secs) {
|
||||
let out = "1h";
|
||||
for (const [k] of ranges) if (rangeSecs[k] <= secs) out = k;
|
||||
return out;
|
||||
}
|
||||
|
||||
// Failed answers keep the same color whatever the range: color follows
|
||||
// the result code, never its rank.
|
||||
@@ -36,8 +53,9 @@ const perViewGroups = ["qtype", "rcode", "view", "zone", "blocklist"];
|
||||
export async function render(el, params, query) {
|
||||
const gen = ++generation;
|
||||
const want = query && query.get ? query.get("since") : null;
|
||||
const since = rangeSecs[want] ? want : "24h";
|
||||
const status = await get("/stats/status");
|
||||
const kept = keptSecs(status);
|
||||
const since = rangeSecs[want] && rangeSecs[want] <= kept ? want : longestWithin(Math.min(86400, kept));
|
||||
const head = h("div", { class: "page-head" },
|
||||
h("div", { class: "grow" }, h("h1", null, "Statistics"),
|
||||
h("div", { class: "muted" }, status.enabled
|
||||
@@ -50,7 +68,10 @@ export async function render(el, params, query) {
|
||||
mount(el, head, disabledPanel());
|
||||
return;
|
||||
}
|
||||
const heatSince = rangeSecs[since] < 7 * 86400 ? "7d" : since;
|
||||
// The heatmap needs hourly counters: at least a week of them, at most
|
||||
// what is kept.
|
||||
const hourSecs = status.retention_seconds ? status.retention_seconds.hour : 90 * 86400;
|
||||
const heatSince = longestWithin(Math.min(Math.max(rangeSecs[since], 7 * 86400), hourSecs));
|
||||
// Which views: all of them unless some are switched off.
|
||||
const allViews = status.query_log ? (await get("/views").catch(() => [])).map((v) => v.name) : [];
|
||||
const off = viewsOff();
|
||||
@@ -58,14 +79,14 @@ export async function render(el, params, query) {
|
||||
const filtered = allViews.length > 0 && shown.length < allViews.length;
|
||||
const vf = filtered ? { views: shown.join(",") } : {};
|
||||
if (filtered && !shown.length) {
|
||||
mount(el, head, filterRow(since, allViews, off), h("div", { class: "empty" }, "No views chosen: switch one on above."));
|
||||
mount(el, head, filterRow(since, kept, allViews, off), h("div", { class: "empty" }, "No views chosen: switch one on above."));
|
||||
return;
|
||||
}
|
||||
let data;
|
||||
try {
|
||||
data = await loadAll(since, heatSince, status.query_log, vf, filtered);
|
||||
} catch (e) {
|
||||
mount(el, head, filterRow(since, allViews, off), errorBox(e));
|
||||
mount(el, head, filterRow(since, kept, allViews, off), errorBox(e));
|
||||
return;
|
||||
}
|
||||
if (gen !== generation) return; // the user moved on
|
||||
@@ -74,7 +95,7 @@ export async function render(el, params, query) {
|
||||
// sudden drop in traffic.
|
||||
const timeline = complete(data.timeline), errors = complete(data.errors);
|
||||
const step = stepSeconds[timeline.step];
|
||||
const parts = [head, filterRow(since, allViews, off)];
|
||||
const parts = [head, filterRow(since, kept, allViews, off)];
|
||||
for (const w of status.warnings || []) parts.push(h("div", { class: "msg warn" }, w));
|
||||
if (filtered) {
|
||||
parts.push(h("div", { class: "msg info small" }, "Showing view" + (shown.length > 1 ? "s " : " ") + shown.join(", ") + ": answers from named's response log" +
|
||||
@@ -120,7 +141,7 @@ export async function render(el, params, query) {
|
||||
// Who asks for what. With the query log, every value leads to the
|
||||
// explore page: a client to what it asked, a name to who asked.
|
||||
const ql = !!(status.retention || {}).queries;
|
||||
const ex = (filters) => (ql ? (k) => exploreHref(filters(k), exploreSince(since)) : null);
|
||||
const ex = (filters) => (ql ? (k) => exploreHref(filters(k), exploreSince(since, status)) : null);
|
||||
if (status.query_log) {
|
||||
const t = data.tops;
|
||||
parts.push(h("div", { class: "chart-grid" },
|
||||
@@ -201,11 +222,11 @@ function complete(tl) {
|
||||
return tl;
|
||||
}
|
||||
|
||||
function filterRow(since, views, off) {
|
||||
function filterRow(since, kept, views, off) {
|
||||
const range = h("div", { class: "filter-row", role: "group", "aria-label": "Time range" },
|
||||
h("span", { class: "muted" }, "Range"),
|
||||
h("div", { class: "segmented" }, ranges.map(([k, label]) =>
|
||||
h("a", { href: "#/stats?since=" + k, class: k === since ? "active" : "", "aria-current": k === since ? "true" : null, title: "last " + label }, k))));
|
||||
h("div", { class: "segmented" }, ranges.filter(([k]) => rangeSecs[k] <= kept).map(([k, label, button]) =>
|
||||
h("a", { href: "#/stats?since=" + k, class: k === since ? "active" : "", "aria-current": k === since ? "true" : null, title: "last " + label }, button || k))));
|
||||
if (!views || views.length < 2) return range;
|
||||
// A toggle per view (together: linked views, say), and all or none.
|
||||
const set = (next) => { saveViewsOff(next); rerender(); };
|
||||
@@ -312,10 +333,11 @@ function disabledPanel() {
|
||||
"(never individual queries) in a separate database, by default for 7 days per minute, 90 days per hour and 2 years per day. See docs/stats.md."));
|
||||
}
|
||||
|
||||
// exploreSince is the Statistics range for the explore page: the query log
|
||||
// keeps days, not months.
|
||||
function exploreSince(since) {
|
||||
return rangeSecs[since] > 7 * 86400 ? "7d" : since;
|
||||
// exploreSince is the Statistics range for the explore page: at most what
|
||||
// the query log keeps (days, usually, not months).
|
||||
function exploreSince(since, status) {
|
||||
const q = status.retention_seconds ? status.retention_seconds.queries : 7 * 86400;
|
||||
return exploreRange(Math.min(rangeSecs[since], q));
|
||||
}
|
||||
|
||||
// dbTile is the statistics database's size: what the statistics cost on
|
||||
|
||||
@@ -195,7 +195,8 @@ async function rotateDialog(t, redraw) {
|
||||
function coveredBy(user, grant) {
|
||||
if (!user || user.admin) return true;
|
||||
const [res, acc] = grant.split(/\s+/)[0].split(":");
|
||||
return (user.grants || []).some((g) => (g.resource === "*" || g.resource === res) && (g.access === "write" || acc === "read"));
|
||||
// "*" never covers certificates (they hand out private keys).
|
||||
return (user.grants || []).some((g) => ((g.resource === "*" && res !== "certificates") || g.resource === res) && (g.access === "write" || acc === "read"));
|
||||
}
|
||||
|
||||
// newTokenForm makes a token for owner (the user it belongs to). Its
|
||||
|
||||
Reference in new issue
Block a user