Compare commits

...
11 Commits
Author SHA1 Message Date
adamandClaude Opus 5.5 a3b054cc86 Release 0.5.2
release / release (push) Successful in 2m5s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 21:02:12 +01:00
adamandClaude Opus 5.5 c27a6c10bd Mail reverse DNS check, what Jarl adds to named.conf, lists one per line
jarld checks the reverse DNS of the public zones' MX and SPF addresses as the internet sees it, every 6 hours (Exposure page, jarl exposure mail, jarl_mail_reverse_dns_ok). The Server options page, jarl option generated and GET /options/generated list what Jarl writes into named.conf by itself; stats.rate_limit_log turns off the query-errors logging. Option lists of three or more elements are shown one per line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 21:02:11 +01:00
adamandClaude Opus 5.5 d33f573782 Release 0.5.1
release / release (push) Successful in 2m6s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 20:46:57 +01:00
adamandClaude Opus 5.5 13ffd8cad4 Encrypted DNS health, comparing a zone's views, --local everywhere
jarld checks the DoT and DoH listeners every 10 minutes as a client would (page, dashboard, jarl encrypted-dns health, jarl_encrypted_dns_up). The Compare views tab and jarl zone diff show what differs between a zone's views, with Use this version. --local works on unlinked zones. A changed certificates.dir is followed by encrypted DNS's tls statements. Sharing an already shared version replaces other views' overrides.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 20:46:56 +01:00
adamandClaude Opus 5.5 67ae0091b3 Release 0.5.0
release / release (push) Successful in 2m5s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 20:27:25 +01:00
adamandClaude Opus 5.5 c7e5bbb628 Release 0.5.0-rc.1
release / release (push) Successful in 2m4s
0.4.1's release candidates grew into 0.5.0: exposure checks, certificates, linked zones, encrypted DNS and per-view statistics are new features. The upgrade from 0.4.0 packages was tested in a Debian 13 container. The CLI's filtered stats summary says answers for the busiest minute too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 20:19:38 +01:00
adamandClaude Opus 5.5 740f3cc328 Release 0.4.1-rc.6
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 20:06:18 +01:00
adamandClaude Opus 5.5 5fcf4082c9 Web UI: Server and account menus keep the top bar on one line
Server options, ACLs, TSIG keys, Exposure, Certificates, Encrypted DNS and the audit log are in a Server menu, Account, Users and Log out under the user's name; the right-hand group stays together when it wraps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 20:06:18 +01:00
adamandClaude Opus 5.5 1cec6cc393 Fix "update PTR": add a missing PTR, also in linked views
Ticking update PTR alone did nothing: the web UI didn't send it, and the engine only acted when the name or address changed. Alone it now adds a missing PTR record (or enables it); for linked zones every linked view's reverse zone counts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 20:06:18 +01:00
adamandClaude Opus 5.5 921bdd6c95 Release 0.4.1-rc.5
release / release (push) Successful in 2m3s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 19:29:33 +01:00
adamandClaude Opus 5.5 12820b17bb Statistics per view: toggle views on the Statistics page
Answers, result codes, record types, zones, blocking, rate limiting and the top lists are also counted per view, from named's response log. The Statistics page has a toggle per view with All and None, remembered per browser; jarl stats --view and ?views= on the API. named's own counters stay for the whole server, and the page says so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 19:29:33 +01:00
65 changed files with 3034 additions and 134 deletions

No files matched your search

+2
View File
@@ -3,3 +3,5 @@
*.db
*.db-wal
*.db-shm
/jarl
/jarld
+89
View File
@@ -6,8 +6,73 @@ All notable changes to Jarl are documented here. The format follows
## [Unreleased]
## [0.5.2] - 2026-10-12
### Added
- Mail reverse DNS check: the addresses of the public zones' MX hosts and
SPF records are looked up as the internet sees them, every 6 hours: a
PTR record whose name resolves back to the address, which Gmail and
Outlook require. It says who runs the reverse zone (your provider) and
what to set there, and notes when Jarl has its own reverse zone for the
address, which only your clients see. The Exposure page (Check now),
`jarl exposure mail` and Prometheus `jarl_mail_reverse_dns_ok`.
- Added by Jarl: the Server options page, `jarl option generated` and
`GET /options/generated` list what Jarl writes into named.conf by
itself (keys.conf, the statistics channel and logging, blocking's
response-policy, the view keys), why, and what controls each.
- `stats.rate_limit_log` (default true): `false` stops Jarl adding named's
`query-errors` category to its log channel when rate limiting is set.
### Changed
- Option values with a list of three or more elements (allow-query,
forwarders, listen-on, ...) are shown one element per line.
## [0.5.1] - 2026-10-12
### Added
- Encrypted DNS health: jarld checks the DoT and DoH listeners every 10
minutes as a client would (the certificate is for the host name,
current and the one Jarl last wrote; a question gets an answer) and
says what is wrong: nothing listening, a firewall, the wrong or an
older certificate. On the Encrypted DNS page (Check now), the
dashboard, `jarl encrypted-dns health` and Prometheus
`jarl_encrypted_dns_up`.
- Comparing a zone's views: the Compare views tab and `jarl zone diff`
show the names and types that differ between the views that have a
zone, linked or not, with "Use this version" to share one view's
version. `GET /zones/{zone}/compare`.
### Changed
- `--local` (and `"local": true`) works on zones that aren't linked too:
their records are local anyway. `--shared` there is still refused.
- When `certificates.dir` changes, jarld points the `tls` statements
encrypted DNS wrote at the new place when it starts; named kept the old
copies before, which renewals no longer updated.
### Fixed
- Sharing a view's version of a name that was shared already (Use this
version on the shared side) didn't replace another view's override; it
said there was nothing to do.
## [0.5.0] - 2026-10-12
### Upgrading from 0.4
- The database gets three new versions (internet exposure, certificates,
linked zones), applied when jarld starts; a 0.4 jarld can't open it
afterwards, so take a backup first (`sudo jarld backup`). Backups made
by 0.4 restore into this version. Tested: 0.4.0 packages with views,
zones, records, blocking and statistics, upgraded in place and
restored from a 0.4.0 backup.
- Statistics per view start with this version: earlier figures are for
all views together.
- New exposure warnings (docs/exposure.md) appear for views reachable from
the internet, e.g. "no rate-limit" or "version visible" on an external
view: fix them from the Exposure page, or acknowledge them. A change
@@ -18,8 +83,25 @@ All notable changes to Jarl are documented here. The format follows
parser doesn't know them. Remove the ranges first if you need to
downgrade.
### Changed
- The web UI's top bar had grown too long, and the user name, Users and
Log out ended up on a second line on the left. The server's
configuration and security pages (Server options, ACLs, TSIG keys,
Exposure, Certificates, Encrypted DNS, Audit log) are now in a
**Server** menu, and Account, Users and Log out in a menu under your
name. On narrow screens the search box and your name stay together on
the right.
### Fixed
- "update PTR" when editing a record (`jarl record update ID --ptr`) did
nothing unless the name or address changed too, and the web UI didn't
even send it alone: a reverse zone created after the record never got
its PTR. Alone, it now adds the PTR record if it is missing (or enables
it). For records of linked zones, the reverse zones of every linked view
count, not only the record's own view.
- Option values in the web UI looked different from the editor: a long
line wrapped back to the left edge, losing its indentation, and the
column changed width when Edit opened. Values now keep their layout
@@ -40,6 +122,13 @@ All notable changes to Jarl are documented here. The format follows
### Added
- Statistics per view ([docs/stats.md](docs/stats.md#views)): the
Statistics page has a toggle per view, with All and None, remembered in
the browser, to see some views only or several together (linked views,
say). Answers, result codes, record types, zones, blocking, rate
limiting and the top lists follow it; named's own counters stay for the
whole server, and the page says so. `jarl stats --view`, API
`?views=`. Per-view figures start with this version.
- DNS over TLS and HTTPS ([docs/encrypted-dns.md](docs/encrypted-dns.md)):
`jarl encrypted-dns setup --hostname NAME --address ADDR` (and the
Encrypted DNS page) obtains a certificate for the host name if needed,
+5 -5
View File
@@ -200,11 +200,11 @@ review step.
## Status
Version 0.4.0 is the current release, and 0.4.1 is in testing
(0.4.1-rc.4): it adds Prometheus metrics, address ranges in ACLs and
other address match lists, safer changeset handling, internet exposure
checks with response rate limiting, TLS certificates Jarl obtains and
renews itself, linked zones for split DNS, and DNS over TLS and HTTPS. 0.4.0 added blocking (blocklists and
Version 0.5.2 is the current release. It adds Prometheus metrics,
address ranges in ACLs and other address match lists, safer changeset
handling, internet exposure checks with response rate limiting, TLS
certificates Jarl obtains and renews itself, linked zones for split DNS,
DNS over TLS and HTTPS, and statistics per view. 0.4.0 added blocking (blocklists and
your own rules, like Pi-hole, done by named, with different blocking for
different devices), disabling zones, server statistics and view keys, on
top of 0.3.0's disabled records, script-friendly CLI output and client
+1 -1
View File
@@ -1 +1 @@
0.4.1-rc.4
0.5.2
+29 -1
View File
@@ -721,7 +721,35 @@ func optionCmd() *cobra.Command {
}
scopeArgs(describe)
describe.Flags().StringVar(&zoneType, "zone-type", "primary", "with --zone: the zone type")
cmd.AddCommand(list, set, unset, describe)
generated := &cobra.Command{
Use: "generated",
Short: "What Jarl adds to named.conf by itself, why, and what controls it",
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newClient()
if err != nil {
return err
}
var gen []struct {
Where string `json:"where"`
Text string `json:"text"`
Why string `json:"why"`
Setting string `json:"setting"`
}
if err := c.Get(ctx(), "/options/generated", nil, &gen); err != nil {
return err
}
return output(gen, func(w io.Writer) {
for i, g := range gen {
if i > 0 {
fmt.Fprintln(w)
}
fmt.Fprintf(w, "# %s: %s\n# controlled by: %s\n%s\n", g.Where, g.Why, g.Setting, g.Text)
}
})
},
}
cmd.AddCommand(list, set, unset, describe, generated)
return cmd
}
+71 -1
View File
@@ -6,6 +6,7 @@ import (
"io"
"net/url"
"strings"
"time"
"github.com/spf13/cobra"
)
@@ -24,6 +25,7 @@ type encStatus struct {
DoT []encEndpoint `json:"dot"`
DoH []encEndpoint `json:"doh"`
Other bool `json:"other"`
Health *encHealth `json:"health"`
Addresses []struct {
Address string `json:"address"`
Private bool `json:"private"`
@@ -99,6 +101,9 @@ views.`,
fmt.Fprintf(w, "For DoH clients:\t%s (Firefox, Chrome, Windows 11)\n", s.DoHURL)
}
}
if s.Enabled {
printHealth(w, s.Health)
}
if s.Other {
fmt.Fprintln(w, "Note:\tnamed has DoT or DoH listeners written by hand too; setup leaves them alone")
}
@@ -308,7 +313,38 @@ replaces the earlier setup.`,
return report(resp)
},
}
cmd.AddCommand(setup, check, disable)
health := &cobra.Command{
Use: "health",
Short: "Check the DoT and DoH listeners now, as a client would",
Long: `Connects to each listener, checks the certificate it serves (for the
host name, current, the one Jarl last wrote) and asks a question. jarld
also does this every 10 minutes; the dashboard and Prometheus
(jarl_encrypted_dns_up) show the result. Exits 1 when a listener doesn't
work.`,
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newClient()
if err != nil {
return err
}
var h encHealth
if err := c.Do(ctx(), "POST", "/encrypted-dns/health", nil, map[string]any{}, &h); err != nil {
return err
}
if err := output(h, func(w io.Writer) { printHealth(w, &h) }); err != nil {
return err
}
ok := true
for _, p := range h.Probes {
ok = ok && p.OK
}
if !ok {
return errors.New("encrypted DNS isn't working everywhere")
}
return nil
},
}
cmd.AddCommand(setup, check, disable, health)
return cmd
}
@@ -337,3 +373,37 @@ func contains(list []string, s string) bool {
}
return false
}
type encHealth struct {
Checked time.Time `json:"checked"`
Probes []struct {
Kind string `json:"kind"`
Address string `json:"address"`
Port int `json:"port"`
OK bool `json:"ok"`
Problem string `json:"problem"`
Warning string `json:"warning"`
RCode string `json:"rcode"`
Ms int64 `json:"ms"`
} `json:"probes"`
}
func printHealth(w io.Writer, h *encHealth) bool {
if h == nil {
fmt.Fprintln(w, "Health:\tnot checked yet (jarl encrypted-dns health)")
return true
}
ok := true
for _, p := range h.Probes {
kind := map[string]string{"dot": "DoT", "doh": "DoH"}[p.Kind]
state := fmt.Sprintf("working (%s in %d ms)", p.RCode, p.Ms)
if !p.OK {
state, ok = "NOT WORKING: "+p.Problem, false
} else if p.Warning != "" {
state += "; " + p.Warning
}
fmt.Fprintf(w, "%s %s port %d:\t%s\n", kind, p.Address, p.Port, state)
}
fmt.Fprintf(w, "Checked:\t%s\n", h.Checked.Local().Format("2006-01-02 15:04:05"))
return ok
}
+81 -1
View File
@@ -1,6 +1,7 @@
package main
import (
"errors"
"fmt"
"io"
"strings"
@@ -42,6 +43,54 @@ type exposureReport struct {
Count int64
} `json:"names"`
} `json:"rate_limiting"`
Mail *mailReport `json:"mail"`
}
type mailReport struct {
Checked string `json:"checked"`
Results []struct {
Zone string `json:"zone"`
Address string `json:"address"`
Roles []string `json:"roles"`
PTR []string `json:"ptr"`
OK bool `json:"ok"`
Level string `json:"level"`
Message string `json:"message"`
ManagedBy string `json:"managed_by"`
LocalZone string `json:"local_zone"`
} `json:"results"`
Errors []string `json:"errors"`
}
func printMail(w io.Writer, m *mailReport) {
if len(m.Results) == 0 && len(m.Errors) == 0 {
fmt.Fprintln(w, "No mail addresses: no public zone has an MX record or SPF addresses.")
return
}
fmt.Fprintln(w, "ZONE\tADDRESS\tFOR\tPTR\tSTATUS")
for _, r := range m.Results {
ptr := strings.Join(r.PTR, ",")
if ptr == "" {
ptr = "-"
}
fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\n", r.Zone, r.Address, strings.Join(r.Roles, ", "), ptr, r.Level)
if r.Level != "ok" {
fmt.Fprintf(w, " %s\n", r.Message)
}
if r.LocalZone != "" {
fmt.Fprintf(w, " Jarl's reverse zone %s only answers inside: receivers ask %s\n", r.LocalZone, orOwner(r.ManagedBy))
}
}
for _, e := range m.Errors {
fmt.Fprintf(w, "error: %s\n", e)
}
}
func orOwner(s string) string {
if s == "" {
return "the address's owner"
}
return s
}
// exposureScope turns "server" (or "") into the server, anything else
@@ -115,6 +164,10 @@ fix Jarl can stage for each.`,
}
}
}
if r.Mail != nil {
fmt.Fprintln(w, "\nMAIL REVERSE DNS")
printMail(w, r.Mail)
}
if a := r.RateLimiting; a != nil && len(a.Counts) > 0 {
fmt.Fprintf(w, "\nRATE LIMITING, LAST %s\n", a.Since)
for _, e := range a.Counts {
@@ -186,6 +239,33 @@ fix Jarl can stage for each.`,
return report(resp)
},
}
cmd.AddCommand(fix, ack, unack)
mail := &cobra.Command{
Use: "mail",
Short: "Check the reverse DNS of the addresses that send and receive your mail, now",
Long: `Finds the addresses of the public zones' MX hosts and SPF records and checks,
through public resolvers, that each has a PTR record whose name resolves
back to it, as Gmail and Outlook do. Exits 1 when one doesn't.`,
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newClient()
if err != nil {
return err
}
var m mailReport
if err := c.Do(ctx(), "POST", "/exposure/mail", nil, map[string]any{}, &m); err != nil {
return err
}
if err := output(m, func(w io.Writer) { printMail(w, &m) }); err != nil {
return err
}
for _, r := range m.Results {
if !r.OK {
return errors.New("some mail addresses have no working reverse DNS")
}
}
return nil
},
}
cmd.AddCommand(fix, ack, unack, mail)
return cmd
}
+37 -9
View File
@@ -4,6 +4,7 @@ import (
"fmt"
"io"
"math"
"net/url"
"slices"
"strconv"
"strings"
@@ -21,8 +22,22 @@ func statsCmd() *cobra.Command {
Long: `Query statistics collected from named (see docs/stats.md). They must be
enabled in /etc/jarl/jarl.yaml (stats.enabled: true).
Ranges are given with --since: 90m, 24h, 7d, 2w. Use -o json or -o yaml to
export the data.`,
Ranges are given with --since: 90m, 24h, 7d, 2w. --view limits the figures
to some views (repeat it, or separate with commas): answers from named's
response log, per view; named's own counters stay for the whole server.
Use -o json or -o yaml to export the data.`,
}
var statsViews []string
cmd.PersistentFlags().StringSliceVar(&statsViews, "view", nil, "only these views (repeat, or comma separated)")
// withViews adds the --view filter to a query.
withViews := func(v url.Values) url.Values {
if len(statsViews) > 0 {
if v == nil {
v = url.Values{}
}
v.Set("views", strings.Join(statsViews, ","))
}
return v
}
// Each command has its own --since variable: their defaults differ, and
// a shared variable would take the last default for all of them.
@@ -77,14 +92,27 @@ export the data.`,
return err
}
var s stats.Summary
if err := c.Get(ctx(), "/stats/summary", q("since", summarySince, "top", strconv.Itoa(topN)), &s); err != nil {
if err := c.Get(ctx(), "/stats/summary", withViews(q("since", summarySince, "top", strconv.Itoa(topN))), &s); err != nil {
return err
}
return output(s, func(w io.Writer) {
fmt.Fprintf(w, "Range:\t%s to %s\n", s.From.Local().Format("2006-01-02 15:04"), s.To.Local().Format("2006-01-02 15:04"))
fmt.Fprintf(w, "Queries:\t%d (%.2f per second on average)\n", s.Queries, s.AvgQPS)
if len(s.FilterViews) > 0 {
since := ""
if s.ViewsSince != nil {
since = ", per view since " + s.ViewsSince.Local().Format("2006-01-02 15:04")
}
fmt.Fprintf(w, "Views:\t%s (from the response log%s; named's own counters are left out)\n", strings.Join(s.FilterViews, ", "), since)
fmt.Fprintf(w, "Answers:\t%d (%.2f per second on average)\n", s.Queries, s.AvgQPS)
} else {
fmt.Fprintf(w, "Queries:\t%d (%.2f per second on average)\n", s.Queries, s.AvgQPS)
}
if s.Peak != nil {
fmt.Fprintf(w, "Busiest %s:\t%s with %d queries\n", s.Peak.Per, s.Peak.Time.Local().Format("2006-01-02 15:04"), s.Peak.Queries)
what := "queries"
if len(s.FilterViews) > 0 {
what = "answers"
}
fmt.Fprintf(w, "Busiest %s:\t%s with %d %s\n", s.Peak.Per, s.Peak.Time.Local().Format("2006-01-02 15:04"), s.Peak.Queries, what)
}
if s.CacheHitRatio != nil {
fmt.Fprintf(w, "Cache hit ratio:\t%.1f%%\n", *s.CacheHitRatio*100)
@@ -157,7 +185,7 @@ export the data.`,
return err
}
var es []stats.Entry
if err := c.Get(ctx(), "/stats/top", q("kind", kind, "since", topSince, "limit", strconv.Itoa(limit)), &es); err != nil {
if err := c.Get(ctx(), "/stats/top", withViews(q("kind", kind, "since", topSince, "limit", strconv.Itoa(limit))), &es); err != nil {
return err
}
return output(es, func(w io.Writer) {
@@ -207,7 +235,7 @@ export the data.`,
qv.Add("key", k)
}
var tl stats.Timeline
if err := c.Get(ctx(), "/stats/timeline", qv, &tl); err != nil {
if err := c.Get(ctx(), "/stats/timeline", withViews(qv), &tl); err != nil {
return err
}
return output(tl, func(w io.Writer) {
@@ -256,7 +284,7 @@ export the data.`,
return err
}
var hm stats.Heatmap
if err := c.Get(ctx(), "/stats/heatmap", q("since", heatmapSince, "key", hmKey), &hm); err != nil {
if err := c.Get(ctx(), "/stats/heatmap", withViews(q("since", heatmapSince, "key", hmKey)), &hm); err != nil {
return err
}
return output(hm, func(w io.Writer) {
@@ -308,7 +336,7 @@ export the data.`,
return err
}
var es []stats.Entry
if err := c.Get(ctx(), "/stats/breakdown", q("group", args[0], "since", breakdownSince), &es); err != nil {
if err := c.Get(ctx(), "/stats/breakdown", withViews(q("group", args[0], "since", breakdownSince)), &es); err != nil {
return err
}
return output(es, func(w io.Writer) {
+88 -1
View File
@@ -424,7 +424,7 @@ equal to the file: records not in the file are deleted.`,
importCmd.Flags().BoolVar(&replace, "replace", false, "delete records that are not in the file")
importCmd.Flags().BoolVar(&useSOA, "use-soa", false, "also take SOA timers and $TTL from the file")
cmd.AddCommand(list, show, create, update, del, toggle(true), toggle(false), fileCmd, importCmd, zoneSerialsCmd(), zoneDelegationCmd(), zoneDNSSECCmd(), zoneRegistrarCmd(), zoneLinkCmd(true), zoneLinkCmd(false))
cmd.AddCommand(list, show, create, update, del, toggle(true), toggle(false), fileCmd, importCmd, zoneSerialsCmd(), zoneDelegationCmd(), zoneDNSSECCmd(), zoneRegistrarCmd(), zoneLinkCmd(true), zoneLinkCmd(false), zoneDiffCmd())
return cmd
}
@@ -937,3 +937,90 @@ zone is dissolved.`
cmd.Example = " jarl zone unlink adbraun.lol --view vps"
return cmd
}
// zoneDiffCmd is 'zone diff': a zone's records side by side in the views
// that have it.
func zoneDiffCmd() *cobra.Command {
var views []string
var all bool
cmd := &cobra.Command{
Use: "diff ZONE [--view V --view W] [--all]",
Short: "Show what differs between the views that have a zone",
Long: `Lines up a zone's records in the views that have it (or the views
given), one row per name and type, and shows the ones that differ: a
local override in a linked zone, a record one view lacks. With --all, the
ones that are the same too. In a linked zone, 'jarl record update ID
--shared' makes a view's version every view's.`,
Example: ` jarl zone diff adbraun.lol
jarl zone diff adbraun.lol --view internal --view external --all`,
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newClient()
if err != nil {
return err
}
path, qv := zonePath(args[0], "")
if len(views) > 0 {
qv.Set("views", strings.Join(views, ","))
}
if all {
qv.Set("all", "1")
}
var r struct {
Zone string `json:"zone"`
Views []string `json:"views"`
Linked bool `json:"linked"`
Same int `json:"same"`
Rows []struct {
Name string `json:"name"`
Type string `json:"type"`
Same bool `json:"same"`
Views []struct {
Records []string `json:"records"`
IDs []int64 `json:"ids"`
Linked string `json:"linked"`
} `json:"views"`
} `json:"rows"`
}
if err := c.Get(ctx(), path+"/compare", qv, &r); err != nil {
return err
}
return output(r, func(w io.Writer) {
head := []string{"NAME", "TYPE"}
for _, v := range r.Views {
head = append(head, strings.ToUpper(v))
}
fmt.Fprintln(w, strings.Join(head, "\t"))
for _, row := range r.Rows {
cols := []string{row.Name, row.Type}
for _, vs := range row.Views {
cell := "-"
if len(vs.Records) > 0 {
cell = strings.Join(vs.Records, ", ")
if vs.Linked == "local" {
cell += " [local, id " + fmt.Sprint(vs.IDs[0]) + "]"
}
}
cols = append(cols, cell)
}
fmt.Fprintln(w, strings.Join(cols, "\t"))
}
link := "not linked (jarl zone link " + r.Zone + ")"
if r.Linked {
link = "linked"
}
fmt.Fprintf(w, "\n%d names and types differ, %d are the same; %s\n", len(r.Rows)-boolCount(all, r.Same), r.Same, link)
})
},
}
cmd.Flags().StringArrayVar(&views, "view", nil, "a view to compare (repeat; default: every view with the zone)")
cmd.Flags().BoolVar(&all, "all", false, "also show what is the same everywhere")
return cmd
}
func boolCount(b bool, n int) int {
if b {
return n
}
return 0
}
+4 -1
View File
@@ -188,6 +188,7 @@ func renderExtras(cfg *config.Config) render.Extras {
h := &render.StatsHooks{Channel: netip.MustParseAddrPort(cfg.Stats.StatisticsChannel)}
if cfg.Stats.QueryLog {
h.LogFile, h.LogVersions, h.LogSize = cfg.Stats.LogFile, cfg.Stats.LogVersions, cfg.Stats.LogSize
h.NoRateLimitLog = !cfg.Stats.RateLimitLog
}
x.Stats = h
}
@@ -333,7 +334,9 @@ func serveCmd() *cobra.Command {
TOTPRequire: a.cfg.TOTP.Require, TOTPIssuer: a.cfg.TOTP.IssuerName(),
Metrics: server.Metrics{Enabled: a.cfg.Metrics.Enabled, Listen: a.cfg.Metrics.Listen,
Allow: a.cfg.Metrics.AllowPrefixes(), Token: a.cfg.Metrics.Token},
Certs: cm,
Certs: cm,
Resolvers: a.cfg.ACME.Resolvers,
RateLimitLog: a.cfg.Stats.Enabled && a.cfg.Stats.QueryLog && a.cfg.Stats.RateLimitLog,
HTTPReq: server.HTTPReq{
Enabled: a.cfg.ACME.HTTPReq.Enabled,
AllowPlainHTTP: !a.cfg.ACME.HTTPReq.RequireHTTPS,
+7 -1
View File
@@ -353,6 +353,7 @@ and `linked_with` (the other views).
|--------|------|------|
| POST | `/zones/{zone}/link` | `{"with": ["external"]}` (default: every view with that zone). Names and types that are the same everywhere become shared, the rest stay local |
| POST | `/zones/{zone}/unlink` | Takes the zone out of its link; its records stay, all local |
| GET | `/zones/{zone}/compare` | The zone across the views that have it (`?views=a,b`; `?all=1` also the same rows): `views`, `linked`, `same` (count) and `rows` (name, type, `same`, per view `records`, `ids`, `linked`). The zone may be named without a view |
`POST /zones` takes `"link_with": "VIEW"` to create a zone linked with
that view's zone of the same name.
@@ -469,7 +470,8 @@ listing with `?changeset=` marks them `pending`.
| Method | Path | Body / query |
|--------|------|--------------|
| GET | `/options` | `?scope=options\|view\|zone\|toplevel&scope_id=ID` (none: everything you may see) |
| GET | `/options` | `?scope=options\|view\|zone\|toplevel&scope_id=ID` (none: everything you may see). `formatted` lays out a block over several lines, a list of three or more one element per line |
| GET | `/options/generated` | What Jarl adds to named.conf by itself: `where`, `text`, `why`, `setting` (what turns it on or off) |
| POST | `/options` | `{"scope": "zone", "scope_id": 11, "name": "allow-transfer", "value": "{ key xfer; }", "comment": "", "position": 0}` |
| GET / PATCH / DELETE | `/options/{id}` | |
| GET / PATCH | `/settings` | `{"nameservers": [...], "rname": "...", "default_ttl": 3600, "refresh": ..., "retry": ..., "expire": ..., "negative_ttl": ..., "serial_policy": "date"}` |
@@ -539,6 +541,10 @@ in it and its records.
### Query statistics
Every statistics endpoint below takes `views=internal,vps` to limit the
figures to those views ([stats.md](stats.md#views)); a breakdown that only
exists for the whole server then answers `400 not_per_view`.
Needs `stats:read`. When statistics are disabled, every endpoint but
`status` answers 404 `stats_disabled`. See [stats.md](stats.md).
+4 -1
View File
@@ -193,7 +193,10 @@ jarl option set listen-on 'port 853 tls dot { 10.0.10.5; }' --add
```
A certificate named uses can't be deleted or lose its `dns` use until
named no longer names its files.
named no longer names its files. If you change `certificates.dir`, jarld
writes the certificates to the new place when it starts and points the
`tls` statements encrypted DNS wrote there (a hand-written `tls`
statement you change yourself).
## A certificate you renew yourself
+10 -1
View File
@@ -221,6 +221,7 @@ jarl zone unlink adbraun.lol --view vps
jarl record add adbraun.lol nas A 10.0.10.20 --view internal --local # this view only
jarl record set adbraun.lol www A 10.0.10.5 --view internal --local # override the shared www A here
jarl record update 1234 --local | --shared
jarl zone diff adbraun.lol [--view V --view W] [--all] # what differs between the views
```
When a zone exists in several views, add `--view NAME` to commands that
@@ -307,7 +308,9 @@ adds the quotes and splits long values into 255-byte strings. In reverse zones t
host name (a bare `nas` is refused, since it would mean
`nas.0.10.10.in-addr.arpa.`). `--ptr`
keeps the PTR record in the reverse zone in step (when that zone is managed
by Jarl). Every record type BIND supports can be used. Jarl knows the
by Jarl); `jarl record update ID --ptr` with nothing else adds the PTR record
if it is missing, e.g. after creating the reverse zone. In linked zones, the
reverse zones of every linked view count. Every record type BIND supports can be used. Jarl knows the
syntax of the common ones and checks them strictly, and the rest are
validated by parsing.
@@ -402,6 +405,7 @@ jarl exposure # public views, findings,
jarl exposure fix external exposure_rate_limit # rate limiting, log-only first
jarl exposure fix external exposure_rate_limit_log_only # then start enforcing
jarl exposure fix server exposure_version
jarl exposure mail # reverse DNS of the addresses that send and receive mail
jarl exposure ack external security_open_resolver --reason "public resolver, monitored"
jarl exposure unack external security_open_resolver
jarl view update external --exposure public|private|auto
@@ -415,6 +419,7 @@ See [exposure.md](exposure.md).
jarl encrypted-dns # what is set up, what to give clients
jarl encrypted-dns setup --hostname dns.adbraun.lol --address 10.0.10.5 [--address IPv6] [--doh-port 8443] [--no-doh]
jarl encrypted-dns check --address 10.0.10.5 --doh-port 443
jarl encrypted-dns health # check the listeners now, as a client would
jarl encrypted-dns disable
```
@@ -472,6 +477,7 @@ jarl option set logging '{ channel q { file "/var/log/named/query.log" versions
jarl option unset forwarders
jarl option describe # every option allowed at this scope
jarl option describe allow-query-cache # grammar and ARM description
jarl option generated # what Jarl adds to named.conf by itself, and why
```
Options that end in a list (`allow-*`, `forwarders`, `listen-on`, ...) may
@@ -528,6 +534,9 @@ working.
## Query statistics
`--view internal,vps` (or repeated) limits any `jarl stats` command to those
views ([stats.md](stats.md#views)).
```sh
jarl stats status
jarl stats summary --since 7d
+8
View File
@@ -48,6 +48,14 @@ overwritten. Jarl records a checksum of every file it writes and reports
**drift** when a file changes behind its back. It then refuses to overwrite
the file until you say so (`--force`, or `jarl resync --force`).
Some of named.conf comes from Jarl itself rather than from your options:
the include of keys.conf, the statistics channel and the logging
statistics need, blocking's `response-policy` and policy zones, and the
view keys at the front of each view's `match-clients`. The Server options
page lists them under **Added by Jarl**, with why each is there and what
turns it on or off (`jarl option generated`). Where you have a statement
of your own (`logging`, `statistics-channels`), Jarl merges into it.
Comments are part of the data model: zones, records, ACLs, views, keys and
options can all carry a comment, and Jarl writes it into the generated file.
Comments in imported zone files are kept.
+26
View File
@@ -113,6 +113,31 @@ Clients check the certificate against the host name, so the name must
resolve to this server for them: the setup adds it to your private views.
Open the ports in the server's firewall for your own networks only.
## Health
jarld checks the listeners itself, a minute after it starts and then
every 10 minutes, the way a client would: it connects to each, checks the
certificate named serves (for the host name, current, and the one Jarl
last wrote, so a renewal named hasn't reread shows up) and asks for the
host name. The Encrypted DNS page shows the result per listener (**Check
now** checks again), the dashboard a card and the problems, Prometheus
`jarl_encrypted_dns_up{kind,address,port}`:
```sh
jarl encrypted-dns health # check now; exit 1 when a listener doesn't work
```
What it can say:
| | |
|--|--|
| nothing listens on ADDR port P | named couldn't bind the port: look in named's log (ports below 1024 need named to start as root, as the Debian service does) |
| no answer within 5s | a firewall on the server |
| a certificate for X, not HOST | named serves another certificate |
| an older certificate | named hasn't reread the renewed files (`rndc reconfig`) |
| not from a CA this server trusts (warning) | a staging or test CA: clients reject it unless they trust it |
| REFUSED (warning) | it works, but this server's own address gets a view that doesn't answer for the host name |
## What Jarl checks
* A `tls` statement whose files don't exist is an error
@@ -129,3 +154,4 @@ Open the ports in the server's firewall for your own networks only.
| POST | `/encrypted-dns/check` | `{"addresses", "dot", "doh", "dot_port", "doh_port"}`: `conflicts` (address, port, what) and `free_doh_port` |
| POST | `/encrypted-dns` | `{"hostname", "certificate", "addresses", "dot", "doh", "dot_port", "doh_port", "views"}`: staged or applied like any change; 409 while the certificate hasn't been obtained |
| DELETE | `/encrypted-dns` | Turns it off |
| POST | `/encrypted-dns/health` | Checks the listeners now: `checked`, `probes` (kind, address, port, `ok`, `problem`, `warning`, certificate names, expiry and serial, `trusted`, `rcode`, `ms`). `GET /encrypted-dns` has the last check as `health` |
+36 -2
View File
@@ -107,7 +107,10 @@ is only in its log (category `query-errors`). That is why the activity
needs query statistics; without them, look at named's log for
`would rate limit`. When rate limiting is configured and statistics are
on, Jarl adds `query-errors` to its log channel (keeping named's syslog
destination for it).
destination for it). `stats.rate_limit_log: false` in jarl.yaml leaves
`query-errors` alone; the page then can't show what rate limiting does.
The Server options page lists this, and everything else Jarl adds to
named.conf by itself, under **Added by Jarl** (`jarl option generated`).
The statistics also count it: the `rrl` breakdown (`drop`, `slip`,
`would-drop`, `would-slip`), the `ratelimited` and `ratelimited_name` top
@@ -160,6 +163,36 @@ external`). Set in the server options, it applies to every view that has
none of its own, your internal resolver included, which RRL isn't meant
for.
## Mail
Receivers such as Gmail and Outlook check the address mail comes from: it
needs a PTR record (reverse DNS), and the name it gives must resolve back
to the address. Without that, mail is rejected or marked as spam. The PTR
records of your public addresses live in the reverse zones of whoever
owns the addresses, your hosting provider or ISP: you set them in their
panel, not in Jarl. A reverse zone you create in Jarl only answers your
own clients (unless the owner delegated it to you).
So Jarl checks them as the internet sees them, through the public
resolvers of `acme.resolvers`: the addresses of every public zone's MX
hosts (receiving) and SPF `ip4:`, `ip6:`, `a` and `a:` addresses
(sending). The Exposure page's **Mail reverse DNS** panel shows each
address, its PTR, who runs its reverse zone (the provider, from the
zone's name server) and what to do; jarld checks a minute after it starts
and every 6 hours, and logs an address that goes wrong.
```
$ jarl exposure mail
ZONE ADDRESS FOR PTR STATUS
adbraun.lol 172.235.168.106 SPF ip4, MX mail.adbraun.lol mail.adbraun.lol ok
```
It warns when an address has no PTR record, or when its name doesn't
resolve back to it; notes when the name is an alias (CNAME), or isn't the
MX name (fine if the mail server uses it in HELO). `jarl exposure mail`
exits 1 when an address isn't right; Prometheus has
`jarl_mail_reverse_dns_ok{zone,address}`.
## Confirming a change that opens a view up
A change that *introduces* an open resolver, open zone transfers, cache
@@ -204,7 +237,8 @@ monitoring, response size limits and usually a machine of its own.
| Method | Path | Body |
|--------|------|------|
| GET | `/exposure` | `scopes` (per view, or the server: `public`, `override`, `reason`, `rate_limit` off/log-only/enforcing, `findings` with `fix` where Jarl has one), `server` (server-wide findings), `rate_limiting` (last 24h, with statistics: `counts`, `networks`, `names`) |
| GET | `/exposure` | `scopes` (per view, or the server: `public`, `override`, `reason`, `rate_limit` off/log-only/enforcing, `findings` with `fix` where Jarl has one), `server` (server-wide findings), `rate_limiting` (last 24h, with statistics: `counts`, `networks`, `names`), `rate_limit_log`, `mail` (the last mail check) |
| POST | `/exposure/mail` | Checks the mail addresses' reverse DNS now: `checked`, `results` (`zone`, `address`, `roles`, `ptr`, `ok`, `level` ok/info/warning, `message`, `reverse_zone`, `managed_by`, `local_zone`), `errors` |
| POST | `/exposure/fix` | `{"view": "external", "code": "exposure_rate_limit"}` (`view_id` works too; no view: the server). Staged with `?changeset=` like any change |
| POST | `/exposure/ack` | `{"view": "external", "code": "security_open_resolver", "reason": "..."}` |
| POST | `/exposure/unack` | `{"view": "external", "code": "security_open_resolver"}` |
+11
View File
@@ -132,6 +132,8 @@ same values.
| `jarl_certificate_expiry_timestamp_seconds{certificate,source}` | When each certificate Jarl manages expires ([certificates.md](certificates.md)). |
| `jarl_certificate_failing{certificate,source}` | 1 when obtaining (or reading) the certificate failed the last time, or it has expired. |
| `jarl_certificate_renewal_failures{certificate}` | Failed attempts to obtain a certificate since its last success. |
| `jarl_mail_reverse_dns_ok{zone,address}` | 1 when an address that sends or receives a public zone's mail has a PTR record that resolves back to it, at the last check, every 6 hours ([exposure.md](exposure.md#mail)). |
| `jarl_encrypted_dns_up{kind,address,port}` | 1 when the DoT or DoH listener answered with the right, current certificate at the last check ([encrypted-dns.md](encrypted-dns.md#health)). |
### Queries (with `stats.enabled`)
@@ -232,6 +234,15 @@ groups:
expr: jarl_certificate_expiry_timestamp_seconds - time() < 10 * 86400
annotations:
summary: "Certificate {{ $labels.certificate }} expires within 10 days"
- alert: EncryptedDNSDown
expr: jarl_encrypted_dns_up == 0
for: 20m
annotations:
summary: "DNS over {{ $labels.kind }} on {{ $labels.address }} port {{ $labels.port }} isn't working (see the Encrypted DNS page)"
- alert: MailReverseDNS
expr: jarl_mail_reverse_dns_ok == 0
annotations:
summary: "{{ $labels.address }} ({{ $labels.zone }} mail) has no reverse DNS that resolves back: receivers may reject its mail"
- alert: BlockingPausedForLong
expr: jarl_blocking_paused == 1
for: 24h
+28
View File
@@ -60,6 +60,7 @@ off again. The statistics database stays until you delete it
stats:
enabled: false
query_log: true # per-query data; false = named's counters only
rate_limit_log: true # with a rate-limit: also log what it limits (query-errors)
database: /var/lib/jarl/stats.db
statistics_channel: 127.0.0.1:8058 # must be a loopback address
log_file: /var/log/named/jarl-responses.log
@@ -81,6 +82,13 @@ the response log. Logging every answer costs named some CPU and disk I/O,
which is negligible at homelab volumes. At tens of thousands of queries per
second, consider counters only.
`rate_limit_log` matters only with a `rate-limit` set: Jarl then sends
named's `query-errors` category (where rate limiting logs what it limits,
or would in log-only mode) to the response log too, and keeps it in
syslog. That other query errors come along is the price; `false` stops it,
and the Exposure page can't show rate limiting's activity then
([exposure.md](exposure.md#response-rate-limiting)).
`exclude_clients` drops those clients from the log-derived figures (top
lists, views, zones). named's own counters can't be filtered.
@@ -184,6 +192,26 @@ jarl stats summary --since 30d -o json # export (also -o yaml)
`operator` roles include it, `dns-editor` doesn't. Statistics show client
addresses and what they look up, so grant it with care.
## Views
With views, the Statistics page has a toggle per view (and **All** and
**None**): switch some off to see only the others, for example your
internal and VPS views without the internet's noise on the external one,
or several linked views together. The choice is remembered in your
browser. The CLI takes `--view` on every `jarl stats` command (repeat it,
or separate views with commas), and the API `?views=internal,vps`.
Only what named's response log says can be split by view: answers (shown
as *answers* instead of queries), result codes, record types, zones,
blocking, rate limiting and the top lists. named's own counters (answer
sources, transport, upstream response times, DNSSEC, the cache and the
server) are for the whole server, and the page says so; asking for them
per view (`jarl stats breakdown answer --view internal`) is refused.
Per-view figures are kept from the version that added them (Jarl 0.4.1);
the page says since when. They take some more room in the statistics
database, roughly in proportion to the number of views.
## How data is kept
- Minute counters are kept for `retention.minute`, hourly counters and top
+17
View File
@@ -209,6 +209,23 @@ each view (Jarl lists them). Share the right version afterwards. Unlinking
keeps every record, all local; a link left with one zone is dissolved, as
when the other zone is deleted.
**Comparing the views.** The zone's **Compare views** tab, or `jarl zone
diff adbraun.lol`, lines the views up and shows each name and type that
differs: a view's own records, or a name one view lacks. **Use this
version** (`jarl record update ID --shared`) makes one view's version
every view's. It works for zones that aren't linked too, to see what
linking them would keep local.
```
$ jarl zone diff adbraun.lol
NAME TYPE INTERNAL EXTERNAL
nas A 10.0.10.20 [local, id 812] 203.0.113.20 [local, id 790]
5 names and types differ, 21 are the same; linked
```
`--local` also works on a zone that isn't linked (its records are local
anyway), so scripts don't need to know; `--shared` there is refused.
**Permissions** are checked in every view a change reaches. A token with
a grant for one view only (`views=internal`) can change that view's local
records and make overrides, but not a shared record, since that would
+14
View File
@@ -34,6 +34,9 @@ type Zone struct {
LinkedWith []string `json:"linked_with,omitempty"`
// Links are the zones this one is linked with, with their views.
Links []ZoneLink `json:"links,omitempty"`
// OtherViews are the other views with a primary zone of this name
// (linked or not): what GET /zones/{zone}/compare compares.
OtherViews []string `json:"other_views,omitempty"`
// Pending is set when listing with a changeset: add, change or delete.
Pending string `json:"pending,omitempty"`
}
@@ -266,6 +269,9 @@ type StatusResponse struct {
// Certificates sums up the TLS certificates Jarl manages
// (administrators only, when there are any).
Certificates *CertSummary `json:"certificates,omitempty"`
// EncryptedDNS is the last check of DNS over TLS and HTTPS, when it
// is set up (for those who may see options).
EncryptedDNS *EncDNSSummary `json:"encrypted_dns,omitempty"`
}
// CertSummary is what the dashboard reports about certificates.
@@ -575,3 +581,11 @@ type ZoneLink struct {
ID int64 `json:"id"`
View string `json:"view"`
}
// EncDNSSummary is what the dashboard says about encrypted DNS.
type EncDNSSummary struct {
Checked time.Time `json:"checked"`
OK bool `json:"ok"`
Problems []string `json:"problems"`
Warnings []string `json:"warnings"`
}
+32
View File
@@ -25,6 +25,7 @@ import (
"github.com/miekg/dns"
"jarl/internal/auth"
"jarl/internal/encdns"
"jarl/internal/engine"
"jarl/internal/model"
"jarl/internal/porkbun"
@@ -488,3 +489,34 @@ func TestCleanLegoError(t *testing.T) {
t.Errorf("rate limit: %s", got)
}
}
// When certificates.dir changes, Start writes the certificates to the new
// place and points encrypted DNS's tls statement there.
func TestDirChangeFollowed(t *testing.T) {
e := newEnv(t, false)
ctx := context.Background()
admin := &auth.Principal{User: &model.User{Username: "admin", Admin: true}}
now := time.Now()
certFile, keyFile := filepath.Join(e.dir, "c.pem"), filepath.Join(e.dir, "k.pem")
cert, key := makeCert(t, []string{"dns.example.com"}, now.Add(-time.Hour), now.Add(60*24*time.Hour), true)
os.WriteFile(certFile, cert, 0o600)
os.WriteFile(keyFile, key, 0o600)
if _, err := e.m.Create(ctx, Input{Name: model.Ptr("dns"), Source: model.Ptr("file"), CertFile: &certFile, KeyFile: &keyFile, Uses: &[]string{"dns"}}, admin); err != nil {
t.Fatal(err)
}
e.submit(engine.NewRequest(engine.OpOptionCreate, 0, engine.OptionInput{Scope: model.Ptr(model.ScopeTopLevel), Name: model.Ptr("tls"), Value: model.Ptr(encdns.TLSValue("dns", e.m.cfg.Dir))}))
moved := filepath.Join(e.dir, "tls-moved")
cfg := e.m.cfg
cfg.Dir = moved
m2 := New(cfg, e.st, e.eng, slog.New(slog.NewTextHandler(io.Discard, nil)))
m2.Start(ctx)
if _, err := os.Stat(filepath.Join(moved, "dns", ChainFile)); err != nil {
t.Fatalf("not written to the new place: %v", err)
}
for _, o := range e.state().ScopedOptions(model.ScopeTopLevel, 0) {
if o.Name == "tls" && !strings.Contains(o.Value, moved+"/dns/fullchain.pem") {
t.Errorf("tls statement still points elsewhere: %s", o.Value)
}
}
}
+35
View File
@@ -22,6 +22,7 @@ import (
legolog "github.com/go-acme/lego/v4/log"
"jarl/internal/auth"
"jarl/internal/encdns"
"jarl/internal/engine"
"jarl/internal/model"
"jarl/internal/porkbun"
@@ -766,6 +767,40 @@ func (m *Manager) Start(ctx context.Context) {
}
}
}
m.followDir(ctx, all)
}
// followDir points the tls statements encrypted DNS wrote (jarl-NAME) at
// certificates.dir when it has changed: Start has just written the
// certificates there, and named would otherwise keep serving the old
// copies, which renewals no longer update.
func (m *Manager) followDir(ctx context.Context, all []*model.Certificate) {
st, err := m.store.LoadState(ctx)
if err != nil {
return
}
var reqs []engine.Request
for _, b := range encdns.TLSBlocks(st) {
name, ok := strings.CutPrefix(b.Name, encdns.Prefix)
if !ok || filepath.Dir(b.CertFile) == m.FilesDir(name) {
continue
}
i := slices.IndexFunc(all, func(c *model.Certificate) bool { return c.Name == name })
if i < 0 || !all[i].HasUse(model.CertUseDNS) {
continue
}
if _, err := os.Stat(filepath.Join(m.FilesDir(name), ChainFile)); err != nil {
continue // not written there (yet)
}
reqs = append(reqs, engine.NewRequest(engine.OpOptionUpdate, b.OptionID, engine.OptionInput{Value: model.Ptr(encdns.TLSValue(name, m.cfg.Dir))}))
m.log.Info("certificates.dir changed: pointing named at the new files", "tls", b.Name, "was", filepath.Dir(b.CertFile), "now", m.FilesDir(name))
}
if len(reqs) == 0 {
return
}
if _, err := m.eng.Submit(ctx, reqs, auth.System, "jarld", "certificates.dir changed: tls statements follow it", engine.ApplyOptions{}); err != nil {
m.log.Error("pointing named at the moved certificate files failed; run 'jarl encrypted-dns setup' again", "err", err)
}
}
// Run renews certificates when they are due and rereads file-source ones
+9
View File
@@ -364,6 +364,10 @@ type Stats struct {
// per-zone counts) from named's response log. Without it only named's
// own counters are collected.
QueryLog bool `yaml:"query_log"`
// RateLimitLog sends rate limiting's log lines (named's query-errors
// category) to the response log too, so the Exposure page and the
// statistics can show what was, or would be, limited.
RateLimitLog bool `yaml:"rate_limit_log"`
// Database is the statistics database, kept apart from Jarl's
// configuration database.
Database string `yaml:"database"`
@@ -451,6 +455,7 @@ func Default() *Config {
},
Stats: Stats{
QueryLog: true,
RateLimitLog: true,
Database: "/var/lib/jarl/stats.db",
StatisticsChannel: "127.0.0.1:8058",
LogFile: "/var/log/named/jarl-responses.log",
@@ -736,6 +741,10 @@ stats:
# Per-query data (top names and clients, per view and zone). false
# collects only named's own counters, with no logging overhead.
query_log: true
# With a rate-limit set, also log what rate limiting limits (or would,
# in log-only mode): named's query-errors category, which Jarl then adds
# to its log channel (and keeps in syslog). The Exposure page shows it.
rate_limit_log: true
database: /var/lib/jarl/stats.db
statistics_channel: 127.0.0.1:8058
log_file: /var/log/named/jarl-responses.log
+9 -3
View File
@@ -272,9 +272,7 @@ func Plan(st *model.State, s Setup, certDir string) ([]engine.Request, []string,
return nil, nil, fmt.Errorf("choose at least one address to listen on")
}
tlsName := Prefix + s.Certificate
dir := filepath.Join(certDir, s.Certificate)
tlsValue := fmt.Sprintf(`%s { cert-file "%s"; key-file "%s"; protocols { TLSv1.2; TLSv1.3; }; }`,
tlsName, filepath.Join(dir, "fullchain.pem"), filepath.Join(dir, "privkey.pem"))
tlsValue := TLSValue(s.Certificate, certDir)
var reqs []engine.Request
// Take out the earlier setup's listeners and tls statements.
@@ -393,3 +391,11 @@ func Private(a netip.Addr) bool {
a = a.Unmap()
return a.IsPrivate() || a.IsLoopback() || a.IsLinkLocalUnicast() || netip.MustParsePrefix("100.64.0.0/10").Contains(a)
}
// TLSValue is the tls statement the setup writes for a certificate whose
// files are in certDir/NAME.
func TLSValue(cert, certDir string) string {
dir := filepath.Join(certDir, cert)
return fmt.Sprintf(`%s { cert-file "%s"; key-file "%s"; protocols { TLSv1.2; TLSv1.3; }; }`,
Prefix+cert, filepath.Join(dir, "fullchain.pem"), filepath.Join(dir, "privkey.pem"))
}
+208
View File
@@ -0,0 +1,208 @@
package encdns
import (
"bytes"
"context"
"crypto/tls"
"crypto/x509"
"errors"
"fmt"
"io"
"net"
"net/http"
"strconv"
"strings"
"syscall"
"time"
"github.com/miekg/dns"
)
// Probe is the outcome of checking one DoT or DoH listener the way a
// client would: connect, check the certificate, ask a question.
type Probe struct {
Kind string `json:"kind"` // dot or doh
Address string `json:"address"`
Port int `json:"port"`
// OK: it answered over TLS with a certificate for the host name that
// is current. Problem says what is wrong when not; Warning notes
// something that doesn't stop clients from working everywhere.
OK bool `json:"ok"`
Problem string `json:"problem,omitempty"`
Warning string `json:"warning,omitempty"`
// What the certificate named served says.
CertNames []string `json:"cert_names,omitempty"`
CertNotAfter *time.Time `json:"cert_not_after,omitempty"`
CertSerial string `json:"cert_serial,omitempty"`
// Trusted: it chains to a CA this server trusts (not so for a
// staging or test CA).
Trusted bool `json:"trusted"`
RCode string `json:"rcode,omitempty"`
Millis int64 `json:"ms"`
}
// Health is the last check of every listener.
type Health struct {
Checked time.Time `json:"checked"`
Probes []Probe `json:"probes"`
}
// OK reports whether every listener works.
func (h *Health) OK() bool {
for _, p := range h.Probes {
if !p.OK {
return false
}
}
return true
}
// Check probes every DoT and DoH listener of the setup. hostname is what
// clients ask for, serial the certificate Jarl last wrote for named ("" if
// unknown), roots the CAs to trust (nil: the system's).
func Check(ctx context.Context, s Status, hostname, serial string, roots *x509.CertPool) *Health {
h := &Health{Checked: time.Now(), Probes: []Probe{}}
for _, e := range s.DoT {
h.Probes = append(h.Probes, probe(ctx, "dot", e, hostname, serial, roots))
}
for _, e := range s.DoH {
h.Probes = append(h.Probes, probe(ctx, "doh", e, hostname, serial, roots))
}
return h
}
const probeTimeout = 5 * time.Second
func probe(ctx context.Context, kind string, e Endpoint, hostname, serial string, roots *x509.CertPool) Probe {
p := Probe{Kind: kind, Address: e.Address, Port: e.Port}
addr := net.JoinHostPort(e.Address, strconv.Itoa(e.Port))
ctx, cancel := context.WithTimeout(ctx, probeTimeout)
defer cancel()
start := time.Now()
q := new(dns.Msg)
q.SetQuestion(dns.Fqdn(hostname), dns.TypeA)
q.RecursionDesired = true
var state tls.ConnectionState
var answer *dns.Msg
var err error
cfg := &tls.Config{ServerName: hostname, InsecureSkipVerify: true, MinVersion: tls.VersionTLS12} // checked below, to say what is wrong
if kind == "dot" {
cfg.NextProtos = []string{"dot"}
state, answer, err = askDoT(ctx, addr, cfg, q)
} else {
state, answer, err = askDoH(ctx, addr, hostname, e.Port, cfg, q)
}
p.Millis = time.Since(start).Milliseconds()
if len(state.PeerCertificates) > 0 {
leaf := state.PeerCertificates[0]
p.CertNames, p.CertSerial = leaf.DNSNames, fmt.Sprintf("%x", leaf.SerialNumber)
na := leaf.NotAfter
p.CertNotAfter = &na
inter := x509.NewCertPool()
for _, c := range state.PeerCertificates[1:] {
inter.AddCert(c)
}
_, verr := leaf.Verify(x509.VerifyOptions{Roots: roots, Intermediates: inter, DNSName: hostname})
p.Trusted = verr == nil
switch {
case leaf.VerifyHostname(hostname) != nil:
p.Problem = fmt.Sprintf("named serves a certificate for %s, not %s: clients reject it", strings.Join(leaf.DNSNames, ", "), hostname)
case time.Now().After(leaf.NotAfter):
p.Problem = "named serves an expired certificate (until " + leaf.NotAfter.Format(time.DateOnly) + ")"
case serial != "" && p.CertSerial != serial:
p.Problem = "named still serves an older certificate (serial " + p.CertSerial + ", Jarl has " + serial + "): it hasn't reread the files (rndc reconfig)"
case !p.Trusted:
p.Warning = "the certificate isn't from a CA this server trusts (a staging or test CA?): clients reject it unless they trust that CA"
}
}
switch {
case err != nil:
p.Problem = describe(err, e)
case p.Problem != "":
case answer == nil:
p.Problem = "no DNS answer"
default:
p.RCode = dns.RcodeToString[answer.Rcode]
p.OK = true
if answer.Rcode == dns.RcodeRefused {
p.Warning = strings.TrimPrefix(p.Warning+"; ", "; ") + "it answers, but REFUSED: this server's own address gets a view that doesn't answer for " + hostname
}
}
return p
}
func askDoT(ctx context.Context, addr string, cfg *tls.Config, q *dns.Msg) (tls.ConnectionState, *dns.Msg, error) {
d := &tls.Dialer{NetDialer: &net.Dialer{Timeout: probeTimeout}, Config: cfg}
conn, err := d.DialContext(ctx, "tcp", addr)
if err != nil {
return tls.ConnectionState{}, nil, err
}
defer conn.Close()
tc := conn.(*tls.Conn)
state := tc.ConnectionState()
conn.SetDeadline(time.Now().Add(probeTimeout))
co := &dns.Conn{Conn: conn}
if err := co.WriteMsg(q); err != nil {
return state, nil, err
}
r, err := co.ReadMsg()
return state, r, err
}
func askDoH(ctx context.Context, addr, hostname string, port int, cfg *tls.Config, q *dns.Msg) (tls.ConnectionState, *dns.Msg, error) {
cfg = cfg.Clone()
cfg.NextProtos = []string{"h2", "http/1.1"}
tr := &http.Transport{
TLSClientConfig: cfg,
ForceAttemptHTTP2: true,
// Whatever the host name resolves to here: the listener's address.
DialContext: func(ctx context.Context, network, _ string) (net.Conn, error) {
return (&net.Dialer{Timeout: probeTimeout}).DialContext(ctx, network, addr)
},
}
defer tr.CloseIdleConnections()
wire, err := q.Pack()
if err != nil {
return tls.ConnectionState{}, nil, err
}
url := "https://" + net.JoinHostPort(hostname, strconv.Itoa(port)) + "/dns-query"
req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, bytes.NewReader(wire))
if err != nil {
return tls.ConnectionState{}, nil, err
}
req.Header.Set("Content-Type", "application/dns-message")
req.Header.Set("Accept", "application/dns-message")
resp, err := (&http.Client{Transport: tr, Timeout: probeTimeout}).Do(req)
if err != nil {
return tls.ConnectionState{}, nil, err
}
defer resp.Body.Close()
var state tls.ConnectionState
if resp.TLS != nil {
state = *resp.TLS
}
body, err := io.ReadAll(io.LimitReader(resp.Body, 64<<10))
if err != nil {
return state, nil, err
}
if resp.StatusCode != http.StatusOK {
return state, nil, fmt.Errorf("HTTP %d at /dns-query", resp.StatusCode)
}
r := new(dns.Msg)
if err := r.Unpack(body); err != nil {
return state, nil, fmt.Errorf("not a DNS answer: %w", err)
}
return state, r, nil
}
// describe says what a failed probe means.
func describe(err error, e Endpoint) string {
var ne net.Error
switch {
case errors.Is(err, syscall.ECONNREFUSED):
return fmt.Sprintf("nothing listens on %s port %d: named may not have been able to bind it (look in named's log; ports below 1024 need named to start as root)", e.Address, e.Port)
case errors.As(err, &ne) && ne.Timeout(), errors.Is(err, context.DeadlineExceeded):
return fmt.Sprintf("no answer from %s port %d within %s (a firewall on the server?)", e.Address, e.Port, probeTimeout)
}
return err.Error()
}
+115
View File
@@ -0,0 +1,115 @@
package encdns
import (
"context"
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"crypto/tls"
"crypto/x509"
"crypto/x509/pkix"
"io"
"math/big"
"net"
"net/http"
"strings"
"testing"
"time"
"github.com/miekg/dns"
)
func testCert(t *testing.T, names ...string) (tls.Certificate, *x509.Certificate) {
key, _ := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
tmpl := &x509.Certificate{SerialNumber: big.NewInt(0xabc123), Subject: pkix.Name{CommonName: names[0]}, DNSNames: names,
NotBefore: time.Now().Add(-time.Hour), NotAfter: time.Now().Add(24 * time.Hour), ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth}}
der, err := x509.CreateCertificate(rand.Reader, tmpl, tmpl, &key.PublicKey, key)
if err != nil {
t.Fatal(err)
}
leaf, _ := x509.ParseCertificate(der)
return tls.Certificate{Certificate: [][]byte{der}, PrivateKey: key, Leaf: leaf}, leaf
}
// answer answers every question with 192.0.2.53.
var answer = dns.HandlerFunc(func(w dns.ResponseWriter, r *dns.Msg) {
m := new(dns.Msg)
m.SetReply(r)
m.Answer = append(m.Answer, &dns.A{Hdr: dns.RR_Header{Name: r.Question[0].Name, Rrtype: dns.TypeA, Class: dns.ClassINET, Ttl: 60}, A: net.ParseIP("192.0.2.53")})
w.WriteMsg(m)
})
func listeners(t *testing.T, cert tls.Certificate) (dot, doh Endpoint) {
cfg := &tls.Config{Certificates: []tls.Certificate{cert}}
ln, err := tls.Listen("tcp", "127.0.0.1:0", cfg)
if err != nil {
t.Fatal(err)
}
srv := &dns.Server{Listener: ln, Net: "tcp-tls", Handler: answer}
go srv.ActivateAndServe()
t.Cleanup(func() { srv.Shutdown() })
hl, err := tls.Listen("tcp", "127.0.0.1:0", cfg)
if err != nil {
t.Fatal(err)
}
hs := &http.Server{Handler: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/dns-query" {
http.NotFound(w, r)
return
}
b, _ := io.ReadAll(r.Body)
q := new(dns.Msg)
if q.Unpack(b) != nil {
http.Error(w, "bad", 400)
return
}
m := new(dns.Msg)
m.SetReply(q)
m.Rcode = dns.RcodeSuccess
out, _ := m.Pack()
w.Header().Set("Content-Type", "application/dns-message")
w.Write(out)
})}
go hs.Serve(hl)
t.Cleanup(func() { hs.Close() })
port := func(l net.Listener) int { return l.Addr().(*net.TCPAddr).Port }
return Endpoint{"127.0.0.1", port(ln)}, Endpoint{"127.0.0.1", port(hl)}
}
func TestHealth(t *testing.T) {
ctx := context.Background()
cert, leaf := testCert(t, "dns.example.com")
dot, doh := listeners(t, cert)
roots := x509.NewCertPool()
roots.AddCert(leaf)
st := Status{Enabled: true, DoT: []Endpoint{dot}, DoH: []Endpoint{doh}}
h := Check(ctx, st, "dns.example.com", "abc123", roots)
if !h.OK() || len(h.Probes) != 2 {
t.Fatalf("working setup: %+v", h.Probes)
}
for _, p := range h.Probes {
if !p.Trusted || p.RCode != "NOERROR" || p.Warning != "" || p.CertSerial != "abc123" {
t.Errorf("%s: %+v", p.Kind, p)
}
}
// named hasn't reread a renewed certificate.
if h := Check(ctx, st, "dns.example.com", "def456", roots); h.OK() || !strings.Contains(h.Probes[0].Problem, "older certificate") {
t.Errorf("stale: %+v", h.Probes[0])
}
// The wrong name.
if h := Check(ctx, st, "other.example.com", "", roots); h.OK() || !strings.Contains(h.Probes[0].Problem, "not other.example.com") {
t.Errorf("wrong name: %+v", h.Probes[0])
}
// A CA this server doesn't trust: works, with a warning.
if h := Check(ctx, st, "dns.example.com", "abc123", x509.NewCertPool()); !h.OK() || !strings.Contains(h.Probes[1].Warning, "staging") {
t.Errorf("untrusted: %+v", h.Probes[1])
}
// Nothing listening.
l, _ := net.Listen("tcp", "127.0.0.1:0")
gone := l.Addr().(*net.TCPAddr).Port
l.Close()
if h := Check(ctx, Status{Enabled: true, DoT: []Endpoint{{"127.0.0.1", gone}}}, "dns.example.com", "", roots); h.OK() || !strings.Contains(h.Probes[0].Problem, "nothing listens") {
t.Errorf("nothing listening: %+v", h.Probes[0])
}
}
+104
View File
@@ -0,0 +1,104 @@
package engine
import (
"fmt"
"slices"
"strings"
"jarl/internal/model"
)
// ViewSet is what one view has for a name and type.
type ViewSet struct {
View string `json:"view"`
ZoneID int64 `json:"zone_id"`
Records []string `json:"records"` // data (with TTL and "(disabled)" when set), sorted
IDs []int64 `json:"ids"` // the records, in the same order
Linked string `json:"linked,omitempty"` // shared or local, in a linked zone
}
// CompareRow is one name and type across views.
type CompareRow struct {
Name string `json:"name"` // relative to the zone
FQDN string `json:"fqdn"`
Type string `json:"type"`
Same bool `json:"same"`
Views []ViewSet `json:"views"` // in the order of the zones given
}
// CompareZones lines up the records of zones of the same name in
// different views (docs/views.md#linked-zones), one row per name and type,
// sorted like a zone file.
func CompareZones(st *model.State, zones []*model.Zone) []CompareRow {
type key struct{ name, typ string }
sets := make([]map[key][]*model.Record, len(zones))
var keys []key
seen := map[key]bool{}
var all []*model.Record
for i, z := range zones {
sets[i] = map[key][]*model.Record{}
for _, r := range st.ZoneRecords(z.ID) {
k := key{r.Name, r.Type}
sets[i][k] = append(sets[i][k], r)
if !seen[k] {
seen[k] = true
keys = append(keys, k)
all = append(all, r)
}
}
}
if len(zones) > 0 {
// Zone file order: the apex first, then by name and type.
model.SortRecords(all, zones[0])
keys = keys[:0]
for _, r := range all {
keys = append(keys, key{r.Name, r.Type})
}
}
var out []CompareRow
for _, k := range keys {
row := CompareRow{FQDN: k.name, Type: k.typ, Same: true}
if len(zones) > 0 {
row.Name = relName(k.name, zones[0].Name)
}
var first string
for i, z := range zones {
vs := ViewSet{View: st.ViewName(z), ZoneID: z.ID, Records: []string{}, IDs: []int64{}}
rs := slices.Clone(sets[i][k])
slices.SortFunc(rs, func(a, b *model.Record) int { return strings.Compare(strings.ToLower(a.Data), strings.ToLower(b.Data)) })
for _, r := range rs {
d := r.Data
if r.TTL != nil {
d = fmt.Sprintf("%s (TTL %d)", d, *r.TTL)
}
if r.Disabled {
d += " (disabled)"
}
vs.Records = append(vs.Records, d)
vs.IDs = append(vs.IDs, r.ID)
}
if z.LinkGroup != 0 && len(rs) > 0 {
vs.Linked = "shared"
if rs[0].SharedID == 0 {
vs.Linked = "local"
}
}
sig := strings.Join(vs.Records, "\n")
if i == 0 {
first = sig
} else if sig != first {
row.Same = false
}
row.Views = append(row.Views, vs)
}
out = append(out, row)
}
return out
}
func relName(name, zone string) string {
if name == zone {
return "@"
}
return strings.TrimSuffix(name, "."+zone)
}
+87 -24
View File
@@ -5,6 +5,7 @@ import (
"encoding/json"
"fmt"
"net/netip"
"slices"
"strings"
"time"
@@ -642,11 +643,42 @@ func (x *expander) ptrCreate(z *model.Zone, owner string, p *rr.Parsed) ([]model
if !ok {
return nil, errf(KindBadRequest, "ptr only applies to A and AAAA records")
}
rz, ptrName := reverseZoneFor(x.st, z.ViewID, addr)
if rz == nil {
x.warnings = append(x.warnings, fmt.Sprintf("no reverse zone for %s in this view, so no PTR record was added", addr))
return nil, nil
var ops []model.Op
found := false
for _, vid := range ptrViews(x.st, z) {
rz, ptrName := reverseZoneFor(x.st, vid, addr)
if rz == nil {
continue
}
found = true
more, err := x.ptrCreateIn(rz, ptrName, owner)
if err != nil {
return nil, err
}
ops = append(ops, more...)
}
if !found {
x.warnings = append(x.warnings, fmt.Sprintf("no reverse zone for %s in this view, so no PTR record was added (create one: jarl zone create %s/%d)", addr, addr, addr.BitLen()))
}
return ops, nil
}
// ptrViews are the views whose reverse zones get the PTR records of zone
// z's address records: z's own, and, for a linked zone, the other views
// of the link (the record is in all of them).
func ptrViews(st *model.State, z *model.Zone) []int64 {
out := []int64{z.ViewID}
for _, o := range st.LinkedWith(z) {
if !slices.Contains(out, o.ViewID) {
out = append(out, o.ViewID)
}
}
return out
}
// ptrCreateIn returns operations creating the PTR record ptrName -> owner
// in reverse zone rz (none when it is there; enabling it when disabled).
func (x *expander) ptrCreateIn(rz *model.Zone, ptrName, owner string) ([]model.Op, error) {
for _, r := range x.st.ZoneRecords(rz.ID) {
if r.Name == ptrName && r.Type == "PTR" {
if strings.EqualFold(r.Data, owner) {
@@ -674,10 +706,18 @@ func (x *expander) ptrSetDisabled(z *model.Zone, owner string, p *rr.Parsed, dis
if !ok {
return nil
}
rz, ptrName := reverseZoneFor(x.st, z.ViewID, addr)
if rz == nil {
return nil
var ops []model.Op
for _, vid := range ptrViews(x.st, z) {
rz, ptrName := reverseZoneFor(x.st, vid, addr)
if rz == nil {
continue
}
ops = append(ops, x.ptrSetDisabledIn(rz, ptrName, owner, disabled)...)
}
return ops
}
func (x *expander) ptrSetDisabledIn(rz *model.Zone, ptrName, owner string, disabled bool) []model.Op {
var ops []model.Op
for _, r := range x.st.ZoneRecords(rz.ID) {
if r.Name == ptrName && r.Type == "PTR" && strings.EqualFold(r.Data, owner) && r.Disabled != disabled {
@@ -700,12 +740,16 @@ func (x *expander) ptrDelete(z *model.Zone, owner string, p *rr.Parsed) []model.
if !ok {
return nil
}
rz, ptrName := reverseZoneFor(x.st, z.ViewID, addr)
if rz == nil {
return nil
var ops []model.Op
for _, vid := range ptrViews(x.st, z) {
rz, ptrName := reverseZoneFor(x.st, vid, addr)
if rz == nil {
continue
}
in := RecordInput{ZoneID: rz.ID, Name: model.Ptr(ptrName), Type: model.Ptr("PTR"), Data: model.Ptr(owner), MissingOK: true}
ops = append(ops, newOp(OpRecordDelete, 0, 0, in, fmt.Sprintf("delete %s IN PTR %s (if present)", ptrName, owner)))
}
in := RecordInput{ZoneID: rz.ID, Name: model.Ptr(ptrName), Type: model.Ptr("PTR"), Data: model.Ptr(owner), MissingOK: true}
return []model.Op{newOp(OpRecordDelete, 0, 0, in, fmt.Sprintf("delete %s IN PTR %s (if present)", ptrName, owner))}
return ops
}
// parseWarnings keeps what parsing noticed (e.g. a name read as absolute).
@@ -757,9 +801,10 @@ func (x *expander) recordCreate(req Request) ([]model.Op, error) {
}
in.SharedID = &ids[0]
}
} else if in.Local != nil && *in.Local {
return nil, errf(KindBadRequest, "zone %s isn't linked with other views, so its records are local anyway", zoneDesc(x.st, z))
} else if in.Local != nil && !*in.Local {
return nil, notLinked(x.st, z)
}
// local on a zone that isn't linked: its records are local anyway.
in.Local = nil
ops := []model.Op{newOp(OpRecordCreate, ids[0], 0, in, summary)}
if ptr {
@@ -790,10 +835,10 @@ func (x *expander) recordUpdate(req Request) ([]model.Op, error) {
}
var pre []model.Op
if in.Local != nil {
if z.LinkGroup == 0 {
return nil, errf(KindBadRequest, "zone %s isn't linked with other views, so its records are local anyway", zoneDesc(x.st, z))
if z.LinkGroup == 0 && !*in.Local {
return nil, notLinked(x.st, z)
}
if want := *in.Local; want != (r.SharedID == 0) {
if want := *in.Local; z.LinkGroup != 0 && want != (r.SharedID == 0) {
// The whole set of this name and type changes sides.
for _, o := range x.st.RRset(z.ID, r.Name, r.Type) {
id := int64(0)
@@ -804,10 +849,11 @@ func (x *expander) recordUpdate(req Request) ([]model.Op, error) {
}
pre = append(pre, newOp(OpRecordUpdate, o.ID, o.Version, RecordInput{SharedID: model.Ptr(id)}, what))
}
if !want {
// Sharing a version makes it every view's.
pre = append(pre, x.dropOverrides(z, r.Name, r.Type)...)
}
}
if z.LinkGroup != 0 && !*in.Local {
// Sharing a version makes it every view's, also when it is
// shared already and another view overrides it.
pre = append(pre, x.dropOverrides(z, r.Name, r.Type)...)
}
in.Local = nil
if string(mustJSON(in)) == "{}" {
@@ -885,7 +931,10 @@ func (x *expander) recordUpdate(req Request) ([]model.Op, error) {
if len(pre) > 0 {
base = 0 // the record was just updated by pre
}
ops := append(pre, newOp(OpRecordUpdate, r.ID, base, in, summary))
ops := pre
if string(mustJSON(in)) != "{}" || !ptr {
ops = append(ops, newOp(OpRecordUpdate, r.ID, base, in, summary))
}
disabled := r.Disabled
if in.Disabled != nil {
disabled = *in.Disabled
@@ -909,6 +958,15 @@ func (x *expander) recordUpdate(req Request) ([]model.Op, error) {
}
ops = append(ops, x.ptrSetDisabled(z, owner, p, disabled)...)
}
if ptr && !moved && !toggled && !disabled {
// Nothing else changes: make sure the PTR record is there (the
// reverse zone may be newer than the record).
more, err := x.ptrCreate(z, owner, p)
if err != nil {
return nil, err
}
ops = append(ops, more...)
}
return ops, nil
}
@@ -1097,8 +1155,8 @@ func (x *expander) rrsetReplace(req Request) ([]model.Op, error) {
// Explicitly shared: this version becomes every view's.
pre = x.dropOverrides(z, owner, typ)
}
} else if in.Local != nil && *in.Local {
return nil, errf(KindBadRequest, "zone %s isn't linked with other views, so its records are local anyway", zoneDesc(x.st, z))
} else if in.Local != nil && !*in.Local {
return nil, notLinked(x.st, z)
}
in.Local = nil
if len(canon) > 0 {
@@ -1300,3 +1358,8 @@ func firstNonEmptyStr(a, b string) string {
}
return b
}
// notLinked refuses sharing a record of a zone that isn't linked.
func notLinked(st *model.State, z *model.Zone) error {
return errf(KindBadRequest, "zone %s isn't linked with other views, so there is nothing to share its records with (link it first: jarl zone link)", zoneDesc(st, z))
}
+3 -1
View File
@@ -174,7 +174,9 @@ type RecordInput struct {
// Disabled keeps the record in Jarl but out of the zone.
Disabled *bool `json:"disabled,omitempty"`
// PTR also creates (or deletes, disables or enables) the matching PTR
// record in the reverse zone, for A and AAAA records.
// record in the reverse zone, for A and AAAA records. On an update that
// changes nothing else it adds the PTR record if it is missing. In a
// linked zone, every linked view's reverse zone counts.
PTR bool `json:"ptr,omitempty"`
// MissingOK makes a delete-by-match succeed when nothing matches.
MissingOK bool `json:"missing_ok,omitempty"`
+56
View File
@@ -285,3 +285,59 @@ func TestLinkExisting(t *testing.T) {
t.Errorf("shared from internal: external %q", env.view("external"))
}
}
// --local on a zone that isn't linked is fine (its records are local
// anyway); --shared there is refused.
func TestLocalOnUnlinkedZone(t *testing.T) {
env := linkedEnv(t)
local, shared := true, false
env.submit(env.admin, NewRequest(OpRecordCreate, 0, RecordInput{Zone: "example.com", View: "internal", Name: model.Ptr("a"), Type: model.Ptr("A"), Data: model.Ptr("10.0.0.1"), Local: &local}))
env.submit(env.admin, NewRequest(OpRRSetReplace, 0, RRSetInput{Zone: "example.com", View: "internal", Name: "b", Type: "A", Data: []string{"10.0.0.2"}, Local: &local}))
env.submit(env.admin, NewRequest(OpRecordUpdate, env.recordID("internal", "a", "A"), RecordInput{Data: model.Ptr("10.0.0.3"), Local: &local}))
if v := env.view("internal"); !slices.Contains(v, "a A 10.0.0.3 L") || !slices.Contains(v, "b A 10.0.0.2 L") {
t.Errorf("records: %q", v)
}
if e := env.submitErr(env.admin, NewRequest(OpRecordCreate, 0, RecordInput{Zone: "example.com", View: "internal", Name: model.Ptr("c"), Type: model.Ptr("A"), Data: model.Ptr("10.0.0.4"), Local: &shared})); !strings.Contains(e.Message, "nothing to share") {
t.Errorf("--shared on an unlinked zone: %v", e)
}
}
func TestCompareZones(t *testing.T) {
env := linkedEnv(t)
env.submit(env.admin, NewRequest(OpZoneCreate, 0, ZoneInput{Name: model.Ptr("example.com"), View: model.Ptr("external"), LinkWith: "internal"}))
env.submit(env.admin, NewRequest(OpRRSetReplace, 0, RRSetInput{Zone: "example.com", View: "external", Name: "www", Type: "A", Data: []string{"203.0.113.10"}, Local: model.Ptr(true)}))
env.submit(env.admin, NewRequest(OpRecordCreate, 0, RecordInput{Zone: "example.com", View: "internal", Name: model.Ptr("nas"), Type: model.Ptr("A"), Data: model.Ptr("10.0.0.20"), Local: model.Ptr(true)}))
st := env.state()
zones := st.ZonesNamed("example.com.")
st.SortByView(zones)
diff := map[string][]string{}
same := 0
for _, row := range CompareZones(st, zones) {
if row.Same {
same++
continue
}
var cells []string
for _, vs := range row.Views {
cells = append(cells, vs.View+"="+strings.Join(vs.Records, ",")+"/"+vs.Linked)
}
diff[row.Name+" "+row.Type] = cells
}
want := map[string][]string{
"www A": {"internal=192.0.2.10/shared", "external=203.0.113.10/local"},
"nas A": {"internal=10.0.0.20/local", "external=/"},
}
for k, v := range want {
if !slices.Equal(diff[k], v) {
t.Errorf("%s: %q, want %q", k, diff[k], v)
}
}
if len(diff) != 2 || same < 3 {
t.Errorf("differ %v, same %d", diff, same)
}
// "Use this version" on the shared one replaces external's override.
env.submit(env.admin, NewRequest(OpRecordUpdate, env.recordID("internal", "www", "A"), RecordInput{Local: model.Ptr(false)}))
if v := env.view("external"); !slices.Contains(v, "www A 192.0.2.10 S") || slices.Contains(v, "www A 203.0.113.10 L") {
t.Errorf("sharing the shared version: external %q", v)
}
}
+53
View File
@@ -0,0 +1,53 @@
package engine
import (
"slices"
"strings"
"testing"
"jarl/internal/model"
)
// Ticking "update PTR" on an unchanged address record adds its PTR when
// the reverse zone is there (it was created after the record).
func TestUpdatePTRAddsMissing(t *testing.T) {
env := newEnv(t)
env.submit(env.admin, NewRequest(OpSettingsUpdate, 0, SettingsInput{Nameservers: &[]string{"ns1.example.net."}}))
env.submit(env.admin, NewRequest(OpZoneCreate, 0, ZoneInput{Name: model.Ptr("example.com")}))
env.submit(env.admin, rec("example.com", "ldn", "A", "139.144.144.154"))
env.submit(env.admin, NewRequest(OpZoneCreate, 0, ZoneInput{Name: model.Ptr("139.144.144.154/32")}))
var id int64
for _, r := range env.state().Records {
if r.Name == "ldn.example.com." {
id = r.ID
}
}
env.submit(env.admin, NewRequest(OpRecordUpdate, id, RecordInput{PTR: true}))
got := env.records("154.144.144.139.in-addr.arpa.")
if !slices.ContainsFunc(got, func(s string) bool { return strings.Contains(s, "PTR ldn.example.com.") }) {
t.Errorf("reverse zone = %q", got)
}
}
// In linked zones the PTR goes to the reverse zone of whichever linked
// view has one.
func TestUpdatePTRLinked(t *testing.T) {
env := linkedEnv(t)
env.submit(env.admin, NewRequest(OpZoneCreate, 0, ZoneInput{Name: model.Ptr("example.com"), View: model.Ptr("external"), LinkWith: "internal"}))
env.submit(env.admin, recIn("internal", "ldn", "A", "139.144.144.154"))
env.submit(env.admin, NewRequest(OpZoneCreate, 0, ZoneInput{Name: model.Ptr("139.144.144.154/32"), View: model.Ptr("external")}))
res := env.submit(env.admin, NewRequest(OpRecordUpdate, env.recordID("internal", "ldn", "A"), RecordInput{PTR: true}))
st := env.state()
z := st.FindZone("154.144.144.139.in-addr.arpa.", st.ViewByName("external").ID)
ok := false
for _, r := range st.ZoneRecords(z.ID) {
ok = ok || r.Type == "PTR" && r.Data == "ldn.example.com."
}
if !ok {
t.Errorf("no PTR in external's reverse zone (warnings %q)", res.Warnings)
}
// Again: already there, nothing to do.
if e := env.submitErr(env.admin, NewRequest(OpRecordUpdate, env.recordID("internal", "ldn", "A"), RecordInput{PTR: true})); !strings.Contains(e.Message, "nothing to do") {
t.Errorf("second time: %v", e)
}
}
+6
View File
@@ -66,3 +66,9 @@ func (e *Engine) ViewKey(ctx context.Context, st *model.State, z *model.Zone) *n
}
return &nscheck.Key{Name: render.ViewKeyName(v.Name), Algorithm: render.ViewKeyAlgorithm, Secret: render.ViewKeySecret(x.ViewKeySeed, v.ID)}
}
// GeneratedStatements lists what Jarl adds to named.conf by itself
// (render.GeneratedStatements).
func (e *Engine) GeneratedStatements(ctx context.Context, st *model.State) []render.Generated {
return render.GeneratedStatements(st, e.cfg.Paths, e.extras(ctx))
}
+339
View File
@@ -0,0 +1,339 @@
// Package mailcheck checks the reverse DNS of the addresses that send and
// receive mail for your public zones (docs/exposure.md#mail): receivers
// such as Gmail and Outlook reject or mark mail from an address without a
// PTR record, or whose PTR name doesn't resolve back to it. Those records
// belong to whoever owns the address (your hosting provider or ISP), not
// to Jarl, so the check asks the internet and says who to ask.
package mailcheck
import (
"context"
"errors"
"fmt"
"net/netip"
"slices"
"strings"
"time"
"github.com/miekg/dns"
"golang.org/x/net/publicsuffix"
"jarl/internal/model"
)
// Target is an address mail comes from or goes to, and why it is checked.
type Target struct {
Zone string `json:"zone"`
Address string `json:"address"`
Roles []string `json:"roles"` // e.g. "MX mail.adbraun.lol", "SPF ip4"
Host string `json:"host,omitempty"` // the MX host name, when it is one
}
// Result is the check of one address.
type Result struct {
Target
PTR []string `json:"ptr"` // the names its PTR records give
// OK: a PTR name resolves back to the address (forward-confirmed).
OK bool `json:"ok"`
// Level is ok, info or warning; Message says what to do.
Level string `json:"level"`
Message string `json:"message"`
// ReverseZone and ManagedBy say where the PTR record lives.
ReverseZone string `json:"reverse_zone,omitempty"`
ManagedBy string `json:"managed_by,omitempty"`
// LocalZone is a reverse zone for the address in Jarl ("view/zone"),
// which the internet doesn't ask unless the address's owner delegated
// it to this server.
LocalZone string `json:"local_zone,omitempty"`
}
// Report is a whole check.
type Report struct {
Checked time.Time `json:"checked"`
Results []Result `json:"results"`
// Errors are lookups that failed (no internet, say).
Errors []string `json:"errors,omitempty"`
}
// Targets finds the addresses to check in the public zones: the MX hosts
// (receiving) and the SPF record's addresses (sending). Names are looked
// up later, through the resolver.
func Targets(st *model.State, public func(viewID int64) bool) (addrs []Target, names []Target) {
for _, z := range st.SortedZones() {
if z.Type != model.ZonePrimary || z.Disabled || !public(z.ViewID) {
continue
}
zone := strings.TrimSuffix(z.Name, ".")
for _, r := range st.ZoneRecords(z.ID) {
if r.Name != z.Name || r.Disabled {
continue
}
switch r.Type {
case "MX":
f := strings.Fields(r.Data)
if len(f) == 2 && f[1] != "." {
names = append(names, Target{Zone: zone, Host: strings.TrimSuffix(strings.ToLower(f[1]), "."), Roles: []string{"MX " + strings.TrimSuffix(f[1], ".")}})
}
case "TXT":
text := strings.ReplaceAll(strings.ReplaceAll(r.Data, `" "`, ""), `"`, "")
if !strings.HasPrefix(strings.ToLower(text), "v=spf1") {
continue
}
for _, mech := range strings.Fields(text)[1:] {
m := strings.ToLower(strings.TrimLeft(mech, "+?~"))
switch {
case strings.HasPrefix(m, "ip4:"), strings.HasPrefix(m, "ip6:"):
v := m[4:]
p, err := netip.ParsePrefix(v)
if err != nil {
if a, err := netip.ParseAddr(v); err == nil {
p = netip.PrefixFrom(a, a.BitLen())
} else {
continue
}
}
if p.IsSingleIP() {
addrs = append(addrs, Target{Zone: zone, Address: p.Addr().String(), Roles: []string{"SPF " + m[:3]}})
}
case m == "a":
names = append(names, Target{Zone: zone, Host: zone, Roles: []string{"SPF a"}})
case strings.HasPrefix(m, "a:"):
names = append(names, Target{Zone: zone, Host: strings.TrimSuffix(m[2:], "."), Roles: []string{"SPF " + m}})
}
}
}
}
}
return addrs, names
}
// Resolver asks recursive resolvers (the public ones of acme.resolvers).
type Resolver struct {
Servers []string
Timeout time.Duration
}
func (r Resolver) query(ctx context.Context, name string, qtype uint16) (*dns.Msg, error) {
if len(r.Servers) == 0 {
return nil, errors.New("no resolvers configured (acme.resolvers)")
}
m := new(dns.Msg)
m.SetQuestion(dns.Fqdn(name), qtype)
m.RecursionDesired = true
m.SetEdns0(1232, false)
t := r.Timeout
if t == 0 {
t = 5 * time.Second
}
var last error
for _, s := range r.Servers {
in, _, err := (&dns.Client{Timeout: t}).ExchangeContext(ctx, m, s)
if err == nil && in.Truncated {
in, _, err = (&dns.Client{Timeout: t, Net: "tcp"}).ExchangeContext(ctx, m, s)
}
if err == nil && in.Rcode != dns.RcodeServerFailure && in.Rcode != dns.RcodeRefused {
return in, nil
}
if err != nil {
last = fmt.Errorf("%s: %w", s, err)
} else {
last = fmt.Errorf("%s answered %s", s, dns.RcodeToString[in.Rcode])
}
}
return nil, fmt.Errorf("looking up %s %s: %w", strings.TrimSuffix(name, "."), dns.TypeToString[qtype], last)
}
// addresses returns a name's A and AAAA addresses (after CNAMEs).
func (r Resolver) addresses(ctx context.Context, name string) ([]netip.Addr, bool, error) {
var out []netip.Addr
alias := false
for _, t := range []uint16{dns.TypeA, dns.TypeAAAA} {
in, err := r.query(ctx, name, t)
if err != nil {
return nil, false, err
}
for _, rr := range in.Answer {
switch v := rr.(type) {
case *dns.A:
a, _ := netip.AddrFromSlice(v.A.To4())
out = append(out, a)
case *dns.AAAA:
a, _ := netip.AddrFromSlice(v.AAAA)
out = append(out, a)
case *dns.CNAME:
alias = true
}
}
}
return out, alias, nil
}
// Check looks up every target's reverse DNS.
func Check(ctx context.Context, st *model.State, public func(int64) bool, res Resolver) *Report {
rep := &Report{Checked: time.Now(), Results: []Result{}}
addrs, names := Targets(st, public)
// Names to addresses, as the internet sees them.
for _, n := range names {
as, _, err := res.addresses(ctx, n.Host)
if err != nil {
rep.Errors = append(rep.Errors, err.Error())
continue
}
if len(as) == 0 {
rep.Errors = append(rep.Errors, fmt.Sprintf("%s (%s, %s) has no address as the internet sees it: not published yet, or a typo?", n.Host, n.Zone, strings.Join(n.Roles, ", ")))
}
for _, a := range as {
t := n
t.Address = a.String()
addrs = append(addrs, t)
}
}
// One result per address, with every reason it is checked.
merged := map[string]*Target{}
var order []string
for _, t := range addrs {
k := t.Zone + "|" + t.Address
if m := merged[k]; m != nil {
for _, role := range t.Roles {
if !slices.Contains(m.Roles, role) {
m.Roles = append(m.Roles, role)
}
}
if m.Host == "" {
m.Host = t.Host
}
continue
}
t := t
merged[k] = &t
order = append(order, k)
}
for _, k := range order {
r := checkOne(ctx, *merged[k], res)
r.LocalZone = localReverse(st, r.Address, r.ReverseZone)
rep.Results = append(rep.Results, r)
}
return rep
}
// localReverse returns a reverse zone in Jarl that covers the address,
// unless it is the zone the internet asks (delegated to this server).
func localReverse(st *model.State, addr, internetZone string) string {
rev, err := dns.ReverseAddr(addr)
if err != nil {
return ""
}
rev = strings.ToLower(rev)
best := ""
var bz *model.Zone
for _, z := range st.SortedZones() {
n := strings.ToLower(dns.Fqdn(z.Name))
if z.Type == model.ZonePrimary && dns.IsSubDomain(n, rev) && len(n) > len(best) {
best, bz = n, z
}
}
if bz == nil || strings.EqualFold(strings.TrimSuffix(best, "."), internetZone) {
return ""
}
if v := st.ViewName(bz); v != "" {
return v + "/" + strings.TrimSuffix(bz.Name, ".")
}
return strings.TrimSuffix(bz.Name, ".")
}
func checkOne(ctx context.Context, t Target, res Resolver) Result {
r := Result{Target: t, PTR: []string{}}
a, err := netip.ParseAddr(t.Address)
if err != nil {
r.Level, r.Message = "warning", err.Error()
return r
}
rev, _ := dns.ReverseAddr(a.String())
in, err := res.query(ctx, rev, dns.TypePTR)
if err != nil {
r.Level, r.Message = "warning", err.Error()
return r
}
// Who manages the reverse zone: the SOA in the answer.
for _, sec := range [][]dns.RR{in.Ns, in.Answer} {
for _, rr := range sec {
if soa, ok := rr.(*dns.SOA); ok {
r.ReverseZone, r.ManagedBy = strings.TrimSuffix(soa.Hdr.Name, "."), managedBy(soa.Ns)
}
}
}
if r.ReverseZone == "" {
if s, err := res.query(ctx, rev, dns.TypeSOA); err == nil {
for _, sec := range [][]dns.RR{s.Answer, s.Ns} {
for _, rr := range sec {
if soa, ok := rr.(*dns.SOA); ok {
r.ReverseZone, r.ManagedBy = strings.TrimSuffix(soa.Hdr.Name, "."), managedBy(soa.Ns)
}
}
}
}
}
for _, rr := range in.Answer {
if p, ok := rr.(*dns.PTR); ok {
r.PTR = append(r.PTR, strings.TrimSuffix(strings.ToLower(p.Ptr), "."))
}
}
where := "whoever runs the address's reverse zone (your hosting provider or ISP)"
if r.ManagedBy != "" {
where = "the reverse zone's owner (" + r.ManagedBy + ": your hosting provider's or ISP's panel)"
}
if len(r.PTR) == 0 {
r.Level = "warning"
r.Message = fmt.Sprintf("%s has no PTR record: many receivers reject or mark mail from it. Set its reverse DNS to %s at %s", t.Address, suggestName(t), where)
return r
}
var notes []string
for _, name := range r.PTR {
as, alias, err := res.addresses(ctx, name)
if err != nil {
notes = append(notes, err.Error())
continue
}
if slices.Contains(as, a) {
r.OK = true
if alias {
notes = append(notes, name+" is an alias (CNAME): receivers expect the PTR to name a host with its own A/AAAA record")
}
if t.Host != "" && name != t.Host && strings.HasPrefix(strings.Join(t.Roles, " "), "MX") {
notes = append(notes, fmt.Sprintf("the PTR name is %s, not the MX name %s: fine if the mail server says %s in HELO", name, t.Host, name))
}
}
}
switch {
case !r.OK:
r.Level = "warning"
r.Message = fmt.Sprintf("the PTR record of %s names %s, which doesn't resolve back to %s: receivers check that it does. Point the PTR at a name with an A/AAAA record for %s (set at %s), or add that record", t.Address, strings.Join(r.PTR, ", "), t.Address, t.Address, where)
case len(notes) > 0:
r.Level, r.Message = "info", strings.Join(notes, "; ")
default:
r.Level, r.Message = "ok", fmt.Sprintf("%s → %s → %s", t.Address, strings.Join(r.PTR, ", "), t.Address)
}
return r
}
func suggestName(t Target) string {
if t.Host != "" {
return t.Host
}
return "the host's name (with an A/AAAA record for it)"
}
// managedBy turns the reverse zone's primary name server into its owner's
// domain: ns1.linode.com -> linode.com, dns0.zen.co.uk -> zen.co.uk.
func managedBy(ns string) string {
ns = strings.TrimSuffix(strings.ToLower(ns), ".")
labels := strings.Split(ns, ".")
// The registered domain under the ICANN suffix: the list's private
// part (members.linode.com) names customers, not the provider.
for i := 1; i < len(labels); i++ {
if suffix, icann := publicsuffix.PublicSuffix(strings.Join(labels[i:], ".")); icann && suffix == strings.Join(labels[i:], ".") {
return strings.Join(labels[i-1:], ".")
}
}
return ns
}
+108
View File
@@ -0,0 +1,108 @@
package mailcheck
import (
"context"
"net"
"strings"
"testing"
"github.com/miekg/dns"
"jarl/internal/model"
)
// fakeResolver answers from a fixed set of records, as a recursive
// resolver would, with the reverse zone's SOA in negative answers.
func fakeResolver(t *testing.T, records ...string) string {
t.Helper()
var rrs []dns.RR
for _, s := range records {
rr, err := dns.NewRR(s)
if err != nil {
t.Fatal(err)
}
rrs = append(rrs, rr)
}
soa, _ := dns.NewRR("2.0.192.in-addr.arpa. 3600 IN SOA ns1.linode.com. hostmaster.linode.com. 1 3600 900 1209600 300")
pc, err := net.ListenPacket("udp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
srv := &dns.Server{PacketConn: pc, Handler: dns.HandlerFunc(func(w dns.ResponseWriter, q *dns.Msg) {
m := new(dns.Msg)
m.SetReply(q)
qn := strings.ToLower(q.Question[0].Name)
for _, rr := range rrs {
if strings.ToLower(rr.Header().Name) == qn && (rr.Header().Rrtype == q.Question[0].Qtype || rr.Header().Rrtype == dns.TypeCNAME) {
m.Answer = append(m.Answer, rr)
}
}
if len(m.Answer) == 0 && dns.IsSubDomain(soa.Header().Name, qn) {
if q.Question[0].Qtype == dns.TypeSOA && qn == soa.Header().Name {
m.Answer = append(m.Answer, soa)
} else {
m.Ns = append(m.Ns, soa)
}
}
w.WriteMsg(m)
})}
go srv.ActivateAndServe()
t.Cleanup(func() { srv.Shutdown() })
return pc.LocalAddr().String()
}
func TestCheck(t *testing.T) {
st := model.NewState()
ext := &model.View{ID: 1, Name: "external", MatchClients: []string{"any"}}
in := &model.View{ID: 2, Name: "internal", MatchClients: []string{"lan"}}
st.Views[1], st.Views[2] = ext, in
z := &model.Zone{ID: 10, Name: "example.com.", ViewID: 1, Type: model.ZonePrimary}
zi := &model.Zone{ID: 11, Name: "example.com.", ViewID: 2, Type: model.ZonePrimary}
rev := &model.Zone{ID: 12, Name: "27.2.0.192.in-addr.arpa.", ViewID: 2, Type: model.ZonePrimary}
st.Zones[10], st.Zones[11], st.Zones[12] = z, zi, rev
rec := func(id, zone int64, name, typ, data string) {
st.Records[id] = &model.Record{ID: id, ZoneID: zone, Name: name, Type: typ, Data: data}
}
rec(1, 10, "example.com.", "MX", "10 mail.example.com.")
rec(2, 10, "example.com.", "TXT", `"v=spf1 mx ip4:192.0.2.26 ip4:192.0.2.27" " ip4:198.51.100.0/24 -all"`)
rec(3, 10, "www.example.com.", "MX", "10 ignored.example.com.") // not the apex
rec(4, 11, "example.com.", "MX", "10 internal.example.com.") // private view
rec(5, 10, "example.com.", "TXT", `"google-site-verification=x"`)
rec(6, 10, "example.com.", "MX", "20 backup.example.com.") // not on the internet
res := Resolver{Servers: []string{fakeResolver(t,
"mail.example.com. 300 IN A 192.0.2.25",
"25.2.0.192.in-addr.arpa. 300 IN PTR mail.example.com.",
"27.2.0.192.in-addr.arpa. 300 IN PTR li-27.members.linode.com.",
"li-27.members.linode.com. 300 IN A 192.0.2.99",
)}}
public := func(id int64) bool { return id == 1 }
rep := Check(context.Background(), st, public, res)
if len(rep.Errors) != 1 || !strings.Contains(rep.Errors[0], "backup.example.com (example.com, MX backup.example.com) has no address") {
t.Fatalf("errors %v", rep.Errors)
}
got := map[string]Result{}
for _, r := range rep.Results {
got[r.Address] = r
}
if len(got) != 3 {
t.Fatalf("results %+v", rep.Results)
}
if r := got["192.0.2.25"]; !r.OK || r.Level != "ok" || r.Roles[0] != "MX mail.example.com" || r.ManagedBy != "" && r.ManagedBy != "linode.com" {
t.Errorf("mail: %+v", r)
}
if r := got["192.0.2.26"]; r.OK || r.Level != "warning" || !strings.Contains(r.Message, "no PTR record") || r.ManagedBy != "linode.com" || r.ReverseZone != "2.0.192.in-addr.arpa" {
t.Errorf("no PTR: %+v", r)
}
if r := got["192.0.2.27"]; r.OK || !strings.Contains(r.Message, "doesn't resolve back") || r.LocalZone != "internal/27.2.0.192.in-addr.arpa" {
t.Errorf("not forward-confirmed: %+v", r)
}
}
func TestManagedBy(t *testing.T) {
for in, want := range map[string]string{"ns1.linode.com.": "linode.com", "dns0.zen.co.uk.": "zen.co.uk", "ns1.members.linode.com.": "linode.com", "localhost.": "localhost"} {
if got := managedBy(in); got != want {
t.Errorf("managedBy(%q) = %q, want %q", in, got, want)
}
}
}
+35
View File
@@ -182,3 +182,38 @@ func writeItemsInline(sb *strings.Builder, items []Item) {
}
}
}
// FormatValueList is FormatValue, except that a value whose block is a
// plain list (no nested blocks) of at least min elements is laid out one
// element per line, for reading: allow-query, exempt-clients, forwarders.
func FormatValueList(name, value string, min int) string {
st, err := ParseOptionValue(name, value)
if err != nil {
return FormatValue(name, value)
}
items := st.Items[1:]
blocks := 0
var list *Block
for _, it := range items {
if it.Block != nil {
blocks++
list = it.Block
}
}
if blocks != 1 || len(list.Stmts) < min || blockHasBlocks(list) {
return FormatValue(name, value)
}
var sb strings.Builder
for _, it := range items {
if it.Block == nil {
sb.WriteString(Compact([]Item{it}) + " ")
continue
}
sb.WriteString("{\n")
for _, s := range list.Stmts {
sb.WriteString("\t" + Compact(s.Items) + ";\n")
}
sb.WriteString("} ")
}
return strings.TrimSpace(sb.String())
}
+20
View File
@@ -16,3 +16,23 @@ func TestFormatValue(t *testing.T) {
}
}
}
func TestFormatValueList(t *testing.T) {
cases := []struct{ name, value, want string }{
{"allow-query", "{ homelab; localhost; 10.0.0.0/8; }", "{\n\thomelab;\n\tlocalhost;\n\t10.0.0.0/8;\n}"},
{"allow-query", "{ homelab; localhost; }", "{ homelab; localhost; }"}, // short: as it was
{"listen-on", "port 853 tls dot { 10.0.0.1; 10.0.0.2; 10.0.0.3; }", "port 853 tls dot {\n\t10.0.0.1;\n\t10.0.0.2;\n\t10.0.0.3;\n}"},
{"allow-transfer", "{ !10.0.0.9; key xfer; 10.0.0.0/8; }", "{\n\t!10.0.0.9;\n\tkey xfer;\n\t10.0.0.0/8;\n}"},
{"recursion", "yes", "yes"},
}
for _, c := range cases {
if got := FormatValueList(c.name, c.value, 3); got != c.want {
t.Errorf("%s %q:\n got %q\nwant %q", c.name, c.value, got, c.want)
}
}
// A nested block (rate-limit with exempt-clients) keeps FormatValue's layout.
v := "{ responses-per-second 10; log-only yes; exempt-clients { 192.0.2.1; 192.0.2.2; 192.0.2.3; }; }"
if got := FormatValueList("rate-limit", v, 3); got != FormatValue("rate-limit", v) {
t.Errorf("nested: %q", got)
}
}
+142
View File
@@ -0,0 +1,142 @@
package render
import (
"strings"
"jarl/internal/model"
"jarl/internal/namedconf"
)
// Generated is a statement Jarl adds to named's configuration by itself,
// not from an option you set: why, and what controls it.
type Generated struct {
Where string `json:"where"` // top level, options, or "view NAME"
Text string `json:"text"` // as written in named.conf
Why string `json:"why"` // what it is for
Setting string `json:"setting"` // what turns it on or off
}
// GeneratedStatements lists what Render adds to named.conf beyond the
// options in the database, so nothing there comes as a surprise.
func GeneratedStatements(s *model.State, p Paths, x Extras) []Generated {
var out []Generated
add := func(where, text, why, setting string) {
out = append(out, Generated{Where: where, Text: strings.TrimRight(text, "\n"), Why: why, Setting: setting})
}
add("top level", "include "+namedconf.QuoteString(p.KeysConfig())+";",
"TSIG keys (and the view keys) live in their own file, which only named and jarld can read", "always")
st := x.Stats
logResponses := st != nil && st.LogFile != ""
pzs := PolicyZones(s, p)
if st != nil {
_, user := findTop(s, "statistics-channels")
why := "named publishes its counters for jarld's query statistics, on a loopback address only"
if user {
why += " (added to your statistics-channels statement)"
}
add("top level", formatStmt(namedconf.NewStmt("statistics-channels", namedconf.NewBlock(st.statsInet())), 0), why, "stats.enabled in jarl.yaml")
}
if logResponses {
b := namedconf.NewBlock(st.logChannel())
cat := func(name string, extra ...string) {
ch := namedconf.NewBlock(namedconf.NewStmt(StatsChannelName))
for _, e := range extra {
ch.Stmts = append(ch.Stmts, namedconf.NewStmt(e))
}
b.Stmts = append(b.Stmts, namedconf.NewStmt("category", name, ch))
}
why := []string{"responses: every answer, for the statistics' names, clients and views"}
cat("responses")
if len(pzs) > 0 {
cat("rpz")
why = append(why, "rpz: what blocking blocked")
}
if logRateLimit(s, st) {
cat("query-errors", "default_syslog")
why = append(why, "query-errors: what rate limiting limited, or would in log-only mode, for the Exposure page (also to syslog, as before)")
}
w := "named logs to a file jarld reads (" + strings.Join(why, "; ") + ")"
if _, user := findTop(s, "logging"); user {
w += "; merged into your logging statement"
}
setting := "stats.query_log in jarl.yaml"
if logRateLimit(s, st) || hasRateLimit(s) {
setting += "; query-errors: stats.rate_limit_log"
}
add("top level", formatStmt(namedconf.NewStmt("logging", b), 0), w, setting)
if s.FindOption(model.ScopeOptions, 0, "responselog") == nil {
add("options", "responselog yes;", "named logs answers from the start, for the statistics", "stats.query_log in jarl.yaml")
}
}
views := s.OrderedViews()
scopes := []struct {
id int64
name string
}{{0, "options"}}
if len(views) > 0 {
scopes = scopes[:0]
for _, v := range views {
scopes = append(scopes, struct {
id int64
name string
}{v.ID, "view " + v.Name})
}
}
for _, sc := range scopes {
zs := scopeZones(pzs, sc.id)
if len(zs) == 0 {
continue
}
w := newConfWriter()
writeResponsePolicy(w, zs, logResponses)
writePolicyZones(w, s, zs, &Output{}, 0)
add(sc.name, dedent(w.sb.String()), "blocking: your rules and blocklists, as response policy zones named loads", "the Blocking page (jarl block, jarl blocklist)")
}
if viewKeysOn(s, Extras{ViewKeySeed: x.ViewKeySeed}) {
for _, v := range views {
mc := viewMatchClients(s, x, v)
var keys []string
for _, e := range mc {
if strings.Contains(e, "key ") {
keys = append(keys, e+";")
}
}
add("view "+v.Name, "match-clients { "+strings.Join(keys, " ")+" ... };",
"view keys: a query signed with a view's key gets that view, from any address, and never another view (jarl view key)", "always, when there are views")
}
}
return out
}
func findTop(s *model.State, name string) (*model.Option, bool) {
for _, o := range s.ScopedOptions(model.ScopeTopLevel, 0) {
if o.Name == name {
return o, true
}
}
return nil, false
}
func hasRateLimit(s *model.State) bool {
for _, o := range s.Options {
if o.Name == "rate-limit" {
return true
}
}
return false
}
// logRateLimit reports whether rate limiting's log lines go to the
// response log (docs/exposure.md).
func logRateLimit(s *model.State, st *StatsHooks) bool {
return st != nil && st.LogFile != "" && !st.NoRateLimitLog && hasRateLimit(s)
}
// dedent removes one leading tab from every line.
func dedent(s string) string {
lines := strings.Split(s, "\n")
for i, l := range lines {
lines[i] = strings.TrimPrefix(l, "\t")
}
return strings.Join(lines, "\n")
}
+1 -4
View File
@@ -274,10 +274,7 @@ func renderConfig(s *model.State, p Paths, x Extras, out *Output) (string, []Spa
logRewrites := logResponses && len(pzs) > 0
// Rate limiting logs what it limits (or would, in log-only mode) as
// query-errors; the statistics count it (docs/exposure.md).
logRRL := false
for _, o := range s.Options {
logRRL = logRRL || logResponses && o.Name == "rate-limit"
}
logRRL := logRateLimit(s, st)
sawStats, sawLogging := false, false
if top := s.ScopedOptions(model.ScopeTopLevel, 0); len(top) > 0 {
w.write("\n")
+3
View File
@@ -26,6 +26,9 @@ type StatsHooks struct {
LogFile string // response log; "" disables response logging
LogVersions int
LogSize string
// NoRateLimitLog keeps rate limiting's log lines (category
// query-errors) out of the response log (stats.rate_limit_log: false).
NoRateLimitLog bool
}
// StatsChannelName is the logging channel Jarl adds for responses.
+42
View File
@@ -82,3 +82,45 @@ func TestStatsHooksMerged(t *testing.T) {
func TestStatsHooksWithNamed(t *testing.T) {
checkWithNamed(t, sampleState(t), Extras{Stats: statsHooks()})
}
// Rate limiting's log lines go to the response log unless
// stats.rate_limit_log is off; the list of what Jarl adds says so.
func TestRateLimitLogAndGenerated(t *testing.T) {
s := sampleState(t)
for id, o := range s.Options {
if o.Name == "logging" {
delete(s.Options, id)
}
}
s.Options[20000] = &model.Option{ID: 20000, Scope: model.ScopeOptions, Name: "rate-limit", Value: "{ responses-per-second 10; log-only yes; }", Position: 20000}
p := Paths{ConfigDir: "/etc/bind/jarl", ZoneDir: "/var/lib/bind/jarl", CacheDir: "/var/cache/bind/jarl"}
conf := string(Render(s, p, Extras{Stats: statsHooks()}).File(p.MainConfig()).Content)
if !strings.Contains(conf, "category query-errors { jarl_responses; default_syslog; };") {
t.Errorf("query-errors not logged:\n%s", conf)
}
gen := GeneratedStatements(s, p, Extras{Stats: statsHooks()})
var logging *Generated
for i := range gen {
if strings.HasPrefix(gen[i].Text, "logging") {
logging = &gen[i]
}
}
if logging == nil || !strings.Contains(logging.Text, "query-errors") || !strings.Contains(logging.Setting, "stats.rate_limit_log") {
t.Errorf("generated logging = %+v (all %+v)", logging, gen)
}
if !strings.HasPrefix(gen[0].Text, "include ") || gen[0].Where != "top level" {
t.Errorf("first generated = %+v", gen[0])
}
h := statsHooks()
h.NoRateLimitLog = true
conf = string(Render(s, p, Extras{Stats: h}).File(p.MainConfig()).Content)
if strings.Contains(conf, "query-errors") || !strings.Contains(conf, "category responses") {
t.Errorf("rate_limit_log off but query-errors logged:\n%s", conf)
}
for _, g := range GeneratedStatements(s, p, Extras{Stats: h}) {
if strings.Contains(g.Text, "query-errors") {
t.Errorf("listed while off: %+v", g)
}
}
}
+15
View File
@@ -4,6 +4,7 @@ import (
"context"
"encoding/json"
"errors"
"fmt"
"net/http"
"strconv"
"time"
@@ -285,6 +286,20 @@ func (s *Server) status(w http.ResponseWriter, r *http.Request, p *auth.Principa
resp.TOTP = s.totpOverview(r.Context())
resp.Certificates = s.certSummary(r.Context())
}
if p.Can(auth.Action{Resource: auth.ResOptions}) {
if h := s.encryptedDNSHealth(); h != nil {
sum := &api.EncDNSSummary{Checked: h.Checked, OK: h.OK(), Problems: []string{}, Warnings: []string{}}
for _, pr := range h.Probes {
where := fmt.Sprintf("%s on %s port %d", map[string]string{"dot": "DNS over TLS", "doh": "DNS over HTTPS"}[pr.Kind], pr.Address, pr.Port)
if pr.Problem != "" {
sum.Problems = append(sum.Problems, where+": "+pr.Problem)
} else if pr.Warning != "" {
sum.Warnings = append(sum.Warnings, where+": "+pr.Warning)
}
}
resp.EncryptedDNS = sum
}
}
if p.Can(auth.Action{Resource: auth.ResBlocking}) {
resp.BlocklistErrors = s.blocklistErrors(r.Context())
if st, err := s.store.LoadState(r.Context()); err == nil {
+89
View File
@@ -0,0 +1,89 @@
package server
import (
"net/http"
"strings"
"jarl/internal/auth"
"jarl/internal/engine"
"jarl/internal/model"
)
// zoneCompare: GET /zones/{zone}/compare lines up a zone's records across
// the views that have it (docs/views.md#linked-zones). ?views=a,b picks the
// views (default: every view with a primary zone of that name), ?all=1
// includes the names and types that are the same everywhere.
func (s *Server) zoneCompare(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
st, err := s.state(r)
if err != nil {
return err
}
z, err := s.zoneByPath(r, st, p)
if err != nil {
// A name in several views is what there is to compare.
ref := r.PathValue("zone")
if ref == "root" {
ref = "."
}
name, perr := engine.ParseZoneName(ref)
if zs := st.ZonesNamed(name); perr != nil || len(zs) < 2 {
return err
} else {
z = zs[0]
}
}
var want []string
for _, v := range strings.Split(r.URL.Query().Get("views"), ",") {
if v = strings.TrimSpace(v); v != "" {
want = append(want, v)
}
}
var zones []*model.Zone
for _, o := range st.ZonesNamed(z.Name) {
vn := st.ViewName(o)
if o.Type != model.ZonePrimary || !p.CanSeeZone(o.Name, vn) || !p.Can(auth.Action{Resource: auth.ResRecords, Zone: o.Name, View: vn}) {
continue
}
if len(want) > 0 && !containsFold(want, vn) {
continue
}
zones = append(zones, o)
}
st.SortByView(zones)
if len(zones) < 2 {
return badRequest("zone %s is in only one view (that you may read): there is nothing to compare", engine.Display(z.Name))
}
rows := engine.CompareZones(st, zones)
all := queryBool(r, "all")
out := struct {
Zone string `json:"zone"`
Views []string `json:"views"`
Linked bool `json:"linked"` // all of them in one link
Same int `json:"same"` // names and types the same everywhere
Rows []engine.CompareRow `json:"rows"`
}{Zone: engine.Display(z.Name), Rows: []engine.CompareRow{}, Linked: true}
for _, o := range zones {
out.Views = append(out.Views, st.ViewName(o))
out.Linked = out.Linked && o.LinkGroup != 0 && o.LinkGroup == zones[0].LinkGroup
}
for _, row := range rows {
if row.Same {
out.Same++
if !all {
continue
}
}
out.Rows = append(out.Rows, row)
}
writeJSON(w, http.StatusOK, out)
return nil
}
func containsFold(list []string, s string) bool {
for _, x := range list {
if strings.EqualFold(x, s) {
return true
}
}
return false
}
+19
View File
@@ -659,3 +659,22 @@ func (s *Server) search(w http.ResponseWriter, r *http.Request, p *auth.Principa
writeJSON(w, http.StatusOK, out)
return nil
}
// generatedOptions: GET /options/generated lists what Jarl adds to
// named.conf by itself (the statistics channel, its logging, blocking's
// response-policy, view keys), with why and what controls it.
func (s *Server) generatedOptions(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
if err := require(p, auth.Action{Resource: auth.ResOptions}); err != nil {
return err
}
st, err := s.store.LoadState(r.Context())
if err != nil {
return err
}
out := s.eng.GeneratedStatements(r.Context(), st)
if out == nil {
out = []render.Generated{}
}
writeJSON(w, http.StatusOK, out)
return nil
}
+84
View File
@@ -1,6 +1,7 @@
package server
import (
"context"
"net/http"
"net/netip"
"slices"
@@ -43,6 +44,8 @@ type encStatus struct {
// DoTHost and DoHURL are what to give clients.
DoTHost string `json:"dot_host,omitempty"`
DoHURL string `json:"doh_url,omitempty"`
// Health is the last check of the listeners (every 10 minutes).
Health *encdns.Health `json:"health,omitempty"`
// For the setup form.
Addresses []encAddress `json:"addresses"`
Views []encView `json:"views"`
@@ -104,6 +107,9 @@ func (s *Server) encryptedDNS(w http.ResponseWriter, r *http.Request, p *auth.Pr
out.DoHURL += "/dns-query"
}
}
if out.Enabled {
out.Health = s.encryptedDNSHealth()
}
seen := map[netip.Addr]bool{}
for _, a := range localAddrs() {
a = a.Unmap()
@@ -234,3 +240,81 @@ func (s *Server) encryptedDNSDisable(w http.ResponseWriter, r *http.Request, p *
}
return s.mutate(w, r, p, "turn DNS over TLS and HTTPS off", reqs, nil)
}
// encTarget returns the host name clients connect to and the serial of
// the certificate Jarl last wrote for named.
func (s *Server) encTarget(ctx context.Context, st encdns.Status) (host, serial string) {
if s.cfg.Certs == nil || st.Certificate == "" {
return "", ""
}
c, err := s.cfg.Certs.Get(ctx, st.Certificate)
if err != nil {
return "", ""
}
for _, n := range append(c.Names, c.Domains...) {
if !strings.HasPrefix(n, "*.") {
host = n
break
}
}
return host, c.Serial
}
// checkEncryptedDNS checks the DoT and DoH listeners as a client would and
// keeps the result; a change for the worse is logged.
func (s *Server) checkEncryptedDNS(ctx context.Context) *encdns.Health {
if s.cfg.Certs == nil {
return nil
}
state, err := s.store.LoadState(ctx)
if err != nil {
return nil
}
st := encdns.GetStatus(state, s.cfg.Certs.FilesDir(""))
var h *encdns.Health
if st.Enabled {
if host, serial := s.encTarget(ctx, st); host != "" {
h = encdns.Check(ctx, st, host, serial, s.encRoots)
}
}
s.encHealth.Lock()
prev := s.encHealth.h
s.encHealth.h = h
s.encHealth.Unlock()
if h != nil {
for _, p := range h.Probes {
if p.OK {
continue
}
was := false
if prev != nil {
for _, q := range prev.Probes {
was = was || q.Kind == p.Kind && q.Address == p.Address && q.Port == p.Port && !q.OK
}
}
if !was {
s.log.Warn("encrypted DNS isn't working", "kind", p.Kind, "address", p.Address, "port", p.Port, "problem", p.Problem)
}
}
}
return h
}
func (s *Server) encryptedDNSHealth() *encdns.Health {
s.encHealth.Lock()
defer s.encHealth.Unlock()
return s.encHealth.h
}
// encryptedDNSCheckNow: POST /encrypted-dns/health checks the listeners now.
func (s *Server) encryptedDNSCheckNow(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
if err := s.encReady(p, false); err != nil {
return err
}
h := s.checkEncryptedDNS(r.Context())
if h == nil {
return conflict("encrypted DNS isn't set up")
}
writeJSON(w, http.StatusOK, h)
return nil
}
+8 -1
View File
@@ -1,6 +1,7 @@
package server
import (
"jarl/internal/mailcheck"
"net/http"
"time"
@@ -32,6 +33,11 @@ type exposureReport struct {
// RateLimiting is what rate limiting limited, or would have, in the
// last 24 hours (with query statistics on).
RateLimiting *rateLimitActivity `json:"rate_limiting,omitempty"`
// RateLimitLog is false when stats.rate_limit_log is off: what rate
// limiting does isn't counted then.
RateLimitLog bool `json:"rate_limit_log"`
// Mail is the last check of the mail addresses' reverse DNS.
Mail *mailcheck.Report `json:"mail,omitempty"`
}
type rateLimitActivity struct {
@@ -52,7 +58,7 @@ func (s *Server) exposure(w http.ResponseWriter, r *http.Request, p *auth.Princi
return err
}
problems := s.eng.Validate(st)
out := exposureReport{Scopes: []exposureScope{}, Server: []exposureFinding{}}
out := exposureReport{Scopes: []exposureScope{}, Server: []exposureFinding{}, RateLimitLog: s.cfg.RateLimitLog}
finding := func(pr validate.Problem, viewID int64) exposureFinding {
_, desc := engine.ExposureFix(st, viewID, pr.Code)
return exposureFinding{Problem: pr, Fix: desc}
@@ -96,6 +102,7 @@ func (s *Server) exposure(w http.ResponseWriter, r *http.Request, p *auth.Princi
a.Names, _ = s.statsQ.Top(ctx, stats.KindRateLimitedName, since, 10)
out.RateLimiting = a
}
out.Mail = s.mailReport()
writeJSON(w, http.StatusOK, out)
return nil
}
+83
View File
@@ -0,0 +1,83 @@
package server
import (
"context"
"net/http"
"time"
"jarl/internal/auth"
"jarl/internal/mailcheck"
)
// mailCheckEvery is how often jarld checks the reverse DNS of the
// addresses that send and receive mail (docs/exposure.md#mail).
const mailCheckEvery = 6 * time.Hour
// checkMail checks the reverse DNS of the public zones' mail addresses and
// keeps the report; it logs addresses that became wrong.
func (s *Server) checkMail(ctx context.Context) *mailcheck.Report {
st, err := s.store.LoadState(ctx)
if err != nil {
s.log.Warn("mail reverse DNS check: loading state", "err", err)
return nil
}
public := map[int64]bool{}
for _, x := range s.eng.Exposures(st) {
public[x.ViewID] = x.Public
}
if len(st.Views) == 0 {
public[0] = true
}
ctx, cancel := context.WithTimeout(ctx, 2*time.Minute)
defer cancel()
rep := mailcheck.Check(ctx, st, func(id int64) bool { return public[id] }, mailcheck.Resolver{Servers: s.cfg.Resolvers})
s.mail.Lock()
prev := s.mail.r
s.mail.r = rep
s.mail.Unlock()
was := map[string]bool{}
if prev != nil {
for _, r := range prev.Results {
was[r.Zone+"|"+r.Address] = r.OK
}
}
for _, r := range rep.Results {
if ok, seen := was[r.Zone+"|"+r.Address]; !r.OK && (ok || !seen) {
s.log.Warn("mail reverse DNS", "zone", r.Zone, "address", r.Address, "problem", r.Message)
}
}
return rep
}
func (s *Server) mailReport() *mailcheck.Report {
s.mail.Lock()
defer s.mail.Unlock()
return s.mail.r
}
// mailCheckNow: POST /exposure/mail checks now and returns the report.
func (s *Server) mailCheckNow(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
if err := require(p, auth.Action{Resource: auth.ResOptions}); err != nil {
return err
}
rep := s.checkMail(r.Context())
if rep == nil {
rep = &mailcheck.Report{Checked: time.Now(), Results: []mailcheck.Result{}}
}
writeJSON(w, http.StatusOK, rep)
return nil
}
// mailLoop checks a minute after the start, then every mailCheckEvery.
func (s *Server) mailLoop(ctx context.Context) {
wait := time.Minute
for {
select {
case <-ctx.Done():
return
case <-time.After(wait):
s.checkMail(ctx)
wait = mailCheckEvery
}
}
}
+22 -8
View File
@@ -1,6 +1,7 @@
package server
import (
"errors"
"net/http"
"net/netip"
"path/filepath"
@@ -14,14 +15,24 @@ import (
"jarl/internal/stats"
)
// statsQuerier checks permission and that statistics are enabled.
func (s *Server) statsQuerier(p *auth.Principal) (*stats.Querier, error) {
// statsQuerier checks permission and that statistics are enabled. With
// ?views=internal,vps the figures are for those views only.
func (s *Server) statsQuerier(r *http.Request, p *auth.Principal) (*stats.Querier, error) {
if err := require(p, auth.Action{Resource: auth.ResStats}); err != nil {
return nil, err
}
if s.statsQ == nil {
return nil, &httpError{http.StatusNotFound, "stats_disabled", "query statistics are disabled; set stats.enabled: true in /etc/jarl/jarl.yaml and restart jarl (see docs/stats.md)"}
}
var views []string
for _, v := range strings.Split(r.URL.Query().Get("views"), ",") {
if v = strings.TrimSpace(v); v != "" {
views = append(views, v)
}
}
if len(views) > 0 {
return s.statsQ.ForViews(views), nil
}
return s.statsQ, nil
}
@@ -46,7 +57,7 @@ func (s *Server) statsStatus(w http.ResponseWriter, r *http.Request, p *auth.Pri
}
func (s *Server) statsSummary(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
q, err := s.statsQuerier(p)
q, err := s.statsQuerier(r, p)
if err != nil {
return err
}
@@ -66,7 +77,7 @@ func (s *Server) statsSummary(w http.ResponseWriter, r *http.Request, p *auth.Pr
}
func (s *Server) statsTop(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
q, err := s.statsQuerier(p)
q, err := s.statsQuerier(r, p)
if err != nil {
return err
}
@@ -93,7 +104,7 @@ func (s *Server) statsTop(w http.ResponseWriter, r *http.Request, p *auth.Princi
}
func (s *Server) statsTimeline(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
q, err := s.statsQuerier(p)
q, err := s.statsQuerier(r, p)
if err != nil {
return err
}
@@ -124,7 +135,7 @@ func (s *Server) statsTimeline(w http.ResponseWriter, r *http.Request, p *auth.P
}
func (s *Server) statsHeatmap(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
q, err := s.statsQuerier(p)
q, err := s.statsQuerier(r, p)
if err != nil {
return err
}
@@ -143,7 +154,7 @@ func (s *Server) statsHeatmap(w http.ResponseWriter, r *http.Request, p *auth.Pr
}
func (s *Server) statsBreakdown(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
q, err := s.statsQuerier(p)
q, err := s.statsQuerier(r, p)
if err != nil {
return err
}
@@ -152,6 +163,9 @@ func (s *Server) statsBreakdown(w http.ResponseWriter, r *http.Request, p *auth.
return err
}
es, err := q.Breakdown(r.Context(), r.URL.Query().Get("group"), since)
if errors.Is(err, stats.ErrNotPerView) {
return &httpError{http.StatusBadRequest, "not_per_view", r.URL.Query().Get("group") + ": " + err.Error()}
}
if err != nil {
return badRequest("%v", err)
}
@@ -247,7 +261,7 @@ func clientNames(st *model.State, want map[netip.Addr]bool, readable func(*model
// limits (rndc status), and the space left on the file systems Jarl and
// named write to.
func (s *Server) statsServer(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
if _, err := s.statsQuerier(p); err != nil {
if _, err := s.statsQuerier(r, p); err != nil {
return err
}
out := map[string]any{"server": s.cfg.Stats.Server()}
+14
View File
@@ -289,6 +289,20 @@ func (s *Server) collectState(ctx context.Context, pw *promWriter, now time.Time
}
}
if h := s.encryptedDNSHealth(); h != nil {
for _, pr := range h.Probes {
pw.gauge("jarl_encrypted_dns_up", "1 when the DNS over TLS or HTTPS listener answered with the right, current certificate at the last check (every 10 minutes).", b2f(pr.OK),
"kind", pr.Kind, "address", pr.Address, "port", strconv.Itoa(pr.Port))
}
}
if m := s.mailReport(); m != nil {
for _, r := range m.Results {
pw.gauge("jarl_mail_reverse_dns_ok", "1 when an address that sends or receives a public zone's mail has a PTR record that resolves back to it, at the last check (every 6 hours).", b2f(r.OK),
"zone", r.Zone, "address", r.Address)
}
}
if tokens, err := s.store.ListTokens(ctx, 0); err == nil {
for _, t := range tokens {
if t.ExpiresAt != nil && t.RevokedAt == nil {
+8 -1
View File
@@ -22,6 +22,13 @@ func (s *Server) presentZone(st *model.State, z *model.Zone, counts map[int64]re
out.RName = dnsname.EmailFromMailbox(z.SOA.RName)
}
out.Registrar = s.registrarFor(z)
if z.Type == model.ZonePrimary && z.ViewID != 0 {
for _, o := range st.ZonesNamed(z.Name) {
if o.ID != z.ID && o.Type == model.ZonePrimary {
out.OtherViews = append(out.OtherViews, st.ViewName(o))
}
}
}
for _, o := range st.LinkedWith(z) {
out.LinkedWith = append(out.LinkedWith, st.ViewName(o))
out.Links = append(out.Links, api.ZoneLink{ID: o.ID, View: st.ViewName(o)})
@@ -111,7 +118,7 @@ func presentOption(st *model.State, o *model.Option) api.Option {
if c, ok := catalog.Default().Lookup(scope, o.Name); ok {
out.Description, out.Status = c.Description, c.Status
}
if f := namedconf.FormatValue(o.Name, o.Value); f != o.Value {
if f := namedconf.FormatValueList(o.Name, o.Value, 3); f != o.Value {
out.Formatted = f
}
return out
+4
View File
@@ -111,6 +111,7 @@ func (s *Server) routes() {
m.HandleFunc("DELETE "+p+"/zones/{zone}/registrar/ds/{keytag}", s.authed(s.registrarRemoveDS))
m.HandleFunc("PUT "+p+"/zones/{zone}/registrar/ns", s.authed(s.registrarSetNS))
m.HandleFunc("GET "+p+"/zones/{zone}/file", s.authed(s.zoneFile))
m.HandleFunc("GET "+p+"/zones/{zone}/compare", s.authed(s.zoneCompare))
m.HandleFunc("POST "+p+"/zones/{zone}/link", s.authed(s.zoneLink(true)))
m.HandleFunc("POST "+p+"/zones/{zone}/unlink", s.authed(s.zoneLink(false)))
m.HandleFunc("POST "+p+"/zones/{zone}/import", s.authed(s.importZoneFile))
@@ -128,6 +129,7 @@ func (s *Server) routes() {
m.HandleFunc("POST "+p+"/records/{id}/enable", s.authed(s.setRecordDisabled(false)))
// Options and settings
m.HandleFunc("GET "+p+"/options", s.authed(s.listOptions))
m.HandleFunc("GET "+p+"/options/generated", s.authed(s.generatedOptions))
m.HandleFunc("POST "+p+"/options", s.authed(s.createOption))
m.HandleFunc("GET "+p+"/options/{id}", s.authed(s.getOption))
m.HandleFunc("PATCH "+p+"/options/{id}", s.authed(s.updateOption))
@@ -176,8 +178,10 @@ func (s *Server) routes() {
m.HandleFunc("POST "+p+"/encrypted-dns", s.authed(s.encryptedDNSSetup))
m.HandleFunc("DELETE "+p+"/encrypted-dns", s.authed(s.encryptedDNSDisable))
m.HandleFunc("POST "+p+"/encrypted-dns/check", s.authed(s.encryptedDNSCheck))
m.HandleFunc("POST "+p+"/encrypted-dns/health", s.authed(s.encryptedDNSCheckNow))
m.HandleFunc("GET "+p+"/exposure", s.authed(s.exposure))
m.HandleFunc("POST "+p+"/exposure/fix", s.authed(s.exposureFix))
m.HandleFunc("POST "+p+"/exposure/mail", s.authed(s.mailCheckNow))
m.HandleFunc("POST "+p+"/exposure/ack", s.authed(s.exposureAck(true)))
m.HandleFunc("POST "+p+"/exposure/unack", s.authed(s.exposureAck(false)))
m.HandleFunc("GET "+p+"/stats/status", s.authed(s.statsStatus))
+31
View File
@@ -4,6 +4,7 @@ package server
import (
"context"
"crypto/tls"
"crypto/x509"
"errors"
"fmt"
"io/fs"
@@ -21,7 +22,9 @@ import (
"jarl/internal/api"
"jarl/internal/auth"
"jarl/internal/certs"
"jarl/internal/encdns"
"jarl/internal/engine"
"jarl/internal/mailcheck"
"jarl/internal/nscheck"
"jarl/internal/porkbun"
"jarl/internal/stats"
@@ -61,6 +64,12 @@ type Config struct {
TOTPIssuer string
// Metrics configures /metrics (metrics.go).
Metrics Metrics
// RateLimitLog says rate limiting's log lines reach the statistics
// (stats.rate_limit_log), for the Exposure page.
RateLimitLog bool
// Resolvers are public recursive resolvers (acme.resolvers), to see
// the mail addresses' reverse DNS as receivers do.
Resolvers []string
// Certs manages TLS certificates (nil in tests that don't need it),
// including the one jarld serves (h_certs.go, sniff.go).
Certs *certs.Manager
@@ -90,6 +99,18 @@ type Server struct {
// state part of /metrics.
started time.Time
metricsCache metricsCache
// encHealth is the last check of encrypted DNS's listeners
// (h_encdns.go); encRoots the CAs it trusts (nil: the system's).
encHealth struct {
sync.Mutex
h *encdns.Health
}
encRoots *x509.CertPool
// mail is the last check of the mail addresses' reverse DNS (h_mail.go).
mail struct {
sync.Mutex
r *mailcheck.Report
}
mu sync.Mutex
srvs []*http.Server
@@ -226,6 +247,15 @@ func (s *Server) Serve(ctx context.Context, ready func()) error {
ready()
}
go s.housekeeping(ctx)
go s.mailLoop(ctx)
go func() {
// Once jarld and named have settled after a start.
select {
case <-ctx.Done():
case <-time.After(30 * time.Second):
s.checkEncryptedDNS(ctx)
}
}()
select {
case <-ctx.Done():
case err := <-errc:
@@ -300,6 +330,7 @@ func (s *Server) housekeeping(ctx context.Context) {
}
s.limiter.cleanup()
s.warnExpiringTokens(ctx, time.Now())
s.checkEncryptedDNS(ctx)
if s.certs != nil {
s.certs.reloadIfChanged()
}
+14 -1
View File
@@ -35,7 +35,10 @@ func TestStatsAPI(t *testing.T) {
minute := now.Unix() - now.Unix()%60 - 120
hour := minute - minute%3600
b := &stats.Batch{}
vals := map[string]int64{"queries": 50, "rcode:NOERROR": 45, "rcode:NXDOMAIN": 5, "qtype:A": 30, "qtype:AAAA": 20, "zone:example.com": 40, "zone:-": 10}
vals := map[string]int64{"queries": 50, "rcode:NOERROR": 45, "rcode:NXDOMAIN": 5, "qtype:A": 30, "qtype:AAAA": 20, "zone:example.com": 40, "zone:-": 10,
"view:internal": 30, "view:external": 20, stats.ViewKey("internal", "responses"): 30, stats.ViewKey("external", "responses"): 20,
stats.ViewKey("internal", "rcode:NOERROR"): 30, stats.ViewKey("external", "rcode:NOERROR"): 15, stats.ViewKey("external", "rcode:NXDOMAIN"): 5,
"answer:authoritative": 40}
b.AddCounters(stats.ResMinute, minute, vals)
b.AddCounters(stats.ResHour, hour, vals)
b.SetTops(stats.ResHour, hour, stats.KindName, []stats.Entry{{Key: "www.example.com", Count: 30}, {Key: "mail.example.com", Count: 10}})
@@ -47,6 +50,16 @@ func TestStatsAPI(t *testing.T) {
c := stats.NewCollector(ss, opts, nil, slog.New(slog.NewTextHandler(io.Discard, nil)))
e := newAPIEnv(t, c)
// Some views only.
if sum := e.do("GET", "/stats/summary?since=1h&views=external", e.token, nil, 200); sum["queries"].(float64) != 20 || len(sum["rcodes"].([]any)) != 2 || sum["filter_views"] == nil {
t.Errorf("external summary = %v", sum)
}
if b := e.list("/stats/breakdown?group=rcode&since=1h&views=internal,external", e.token); len(b) != 2 || b[0]["count"].(float64) != 45 {
t.Errorf("rcodes of both views = %v", b)
}
if out, status := e.req("GET", "/stats/breakdown?group=answer&since=1h&views=internal", e.token, nil); status != 400 || !strings.Contains(string(out), "not_per_view") {
t.Errorf("answers per view = %d %s", status, out)
}
if st := e.do("GET", "/stats/status", e.token, nil, 200); st["enabled"] != true {
t.Errorf("status = %v", st)
}
+76 -19
View File
@@ -33,6 +33,13 @@ const (
KindRateLimitedName = "ratelimited_name"
)
// ViewKey is the key of a counter for one view: per-view copies of the
// counters that come from named's response log (docs/stats.md).
func ViewKey(view, key string) string { return "@" + view + "/" + key }
// ViewKind is the kind of a top list for one view.
func ViewKind(kind, view string) string { return kind + "@" + view }
// Kinds lists the top-list kinds.
var Kinds = []string{KindName, KindClient, KindNXDomain, KindServFail, KindRefused, KindBlocked, KindBlockedClient, KindRateLimited, KindRateLimitedName}
@@ -115,6 +122,9 @@ type Collector struct {
// rewritten holds queries named blocked (client#port|name) until their
// response line arrives.
rewritten map[string]bool
// viewsSince is set once per-view counters exist (meta views_since).
viewsSince bool
viewsSinceAt int64
// proc is the last sample of named's process, server the latest server
// statistics (server.go).
proc *namedProc
@@ -212,6 +222,9 @@ func (c *Collector) start(ctx context.Context) {
}
}
}
if v, _ := c.store.Meta(ctx, "views_since"); v != "" {
c.viewsSince = true
}
if v, _ := c.store.Meta(ctx, "first_data"); v != "" {
if n, err := strconv.ParseInt(v, 10, 64); err == nil {
t := time.Unix(n, 0)
@@ -227,7 +240,8 @@ func (c *Collector) start(ctx context.Context) {
if last, err := strconv.ParseInt(merged, 10, 64); err == nil {
from := max(last+3600, cur-48*3600)
for h := from; h < cur; h += 3600 {
for _, k := range Kinds {
kinds, _ := c.store.KindsAt(ctx, ResHour, h)
for _, k := range kinds {
if es, err := c.store.TopsAt(ctx, ResHour, h, k); err == nil && len(es) > 0 {
b.AddTops(ResDay, c.dayOf(time.Unix(h, 0)), k, es, 3*c.opts.TopN)
}
@@ -235,7 +249,8 @@ func (c *Collector) start(ctx context.Context) {
}
}
b.SetMeta("merged_through", strconv.FormatInt(cur-3600, 10))
for _, k := range Kinds {
curKinds, _ := c.store.KindsAt(ctx, ResHour, cur)
for _, k := range curKinds {
es, err := c.store.TopsAt(ctx, ResHour, cur, k)
if err != nil || len(es) == 0 {
continue
@@ -294,7 +309,7 @@ func (c *Collector) count(minute int64, key string, n int64) {
c.minutes[minute] = m
}
m[key] += n
if !strings.HasPrefix(key, "g:") && !strings.HasPrefix(key, "gn:") {
if !strings.HasPrefix(key, "g:") && !strings.HasPrefix(key, "gn:") && !strings.HasPrefix(key, "@") {
c.totals[key] += n
}
}
@@ -365,33 +380,55 @@ func (c *Collector) handleLine(line string) {
if view == "" {
view = "_default"
}
zone := c.zoneOf(ev.Name, ev.View)
c.count(minute, "responses", 1)
c.count(minute, "view:"+view, 1)
c.count(minute, "zone:"+c.zoneOf(ev.Name, ev.View), 1)
c.count(minute, "zone:"+zone, 1)
// The same, per view, for the statistics of chosen views.
c.countView(minute, view, "responses", 1)
c.countView(minute, view, "zone:"+zone, 1)
c.countView(minute, view, "rcode:"+ev.RCode, 1)
c.countView(minute, view, "qtype:"+ev.Type, 1)
client := clientKey(ev.Client, c.opts.Anonymize)
hours := c.hours
if hour < c.liveHour {
hours = c.late
}
hours[hour] = c.hourMaps(hours, hour)
tops := hours[hour]
tops[KindName][ev.Name]++
tops[KindClient][client]++
switch {
case blocked:
// Counted by handleRewrite; not a name that doesn't exist.
case ev.RCode == "NXDOMAIN":
tops[KindNXDomain][ev.Name]++
case ev.RCode == "SERVFAIL":
tops[KindServFail][ev.Name]++
case ev.RCode == "REFUSED":
tops[KindRefused][client]++
}
for _, m := range tops {
add := func(kind, key string) {
hours[hour][kind][key]++
m := c.topsMap(hours, hour, ViewKind(kind, view))
m[key]++
if len(m) > maxKeys {
shrink(m)
}
}
add(KindName, ev.Name)
add(KindClient, client)
switch {
case blocked:
// Counted by handleRewrite; not a name that doesn't exist.
case ev.RCode == "NXDOMAIN":
add(KindNXDomain, ev.Name)
case ev.RCode == "SERVFAIL":
add(KindServFail, ev.Name)
case ev.RCode == "REFUSED":
add(KindRefused, client)
}
for _, k := range Kinds {
if m := hours[hour][k]; len(m) > maxKeys {
shrink(m)
}
}
}
// countView counts a per-view copy of a counter.
func (c *Collector) countView(minute int64, view, key string, n int64) {
if !c.viewsSince {
c.viewsSince = true
c.viewsSinceAt = minute
}
c.count(minute, ViewKey(view, key), n)
}
// handleRewrite counts a query that blocking answered: "blocked", and
@@ -412,15 +449,24 @@ func (c *Collector) handleRewrite(rw Rewrite) {
}
c.rewritten[rw.Client.String()+"#"+rw.Port+"|"+rw.Name] = true
minute, hour := c.minuteOf(rw.Time), c.hourOf(rw.Time)
view := rw.View
if view == "" {
view = "_default"
}
c.count(minute, "blocked", 1)
c.count(minute, "blocklist:"+policySource(rw.Via), 1)
c.countView(minute, view, "blocked", 1)
c.countView(minute, view, "blocklist:"+policySource(rw.Via), 1)
hours := c.hours
if hour < c.liveHour {
hours = c.late
}
hours[hour] = c.hourMaps(hours, hour)
client := clientKey(rw.Client, c.opts.Anonymize)
hours[hour][KindBlocked][rw.Name]++
hours[hour][KindBlockedClient][clientKey(rw.Client, c.opts.Anonymize)]++
hours[hour][KindBlockedClient][client]++
c.topsMap(hours, hour, ViewKind(KindBlocked, view))[rw.Name]++
c.topsMap(hours, hour, ViewKind(KindBlockedClient, view))[client]++
}
// handleRateLimit counts a response rate limiting limited, or would have:
@@ -437,7 +483,12 @@ func (c *Collector) handleRateLimit(rl RateLimit) {
key = "rrl:would-" + rl.Action
}
minute, hour := c.minuteOf(rl.Time), c.hourOf(rl.Time)
view := rl.View
if view == "" {
view = "_default"
}
c.count(minute, key, 1)
c.countView(minute, view, key, 1)
hours := c.hours
if hour < c.liveHour {
hours = c.late
@@ -445,6 +496,8 @@ func (c *Collector) handleRateLimit(rl RateLimit) {
hours[hour] = c.hourMaps(hours, hour)
hours[hour][KindRateLimited][rl.Prefix]++
hours[hour][KindRateLimitedName][rl.Name+" "+rl.Type]++
c.topsMap(hours, hour, ViewKind(KindRateLimited, view))[rl.Prefix]++
c.topsMap(hours, hour, ViewKind(KindRateLimitedName, view))[rl.Name+" "+rl.Type]++
}
// policySource names what a rewrite came from: a blocklist's name, "rules"
@@ -547,6 +600,10 @@ func (c *Collector) flush(ctx context.Context, final bool) {
b.AddCounters(ResHour, c.hourOf(t), vals)
b.AddCounters(ResDay, c.dayOf(t), vals)
delete(c.minutes, m)
if c.viewsSinceAt != 0 {
b.SetMeta("views_since", strconv.FormatInt(c.viewsSinceAt, 10))
c.viewsSinceAt = 0
}
if first && len(vals) > 0 {
ft := t
c.status.FirstData = &ft
+158 -10
View File
@@ -5,6 +5,7 @@ import (
"context"
"fmt"
"slices"
"strconv"
"strings"
"time"
)
@@ -14,6 +15,110 @@ type Querier struct {
Store *Store
Opts Options
Now func() time.Time
// Views limits the figures to these views (none: every view). Only
// what named's response log says can be split by view; named's own
// counters (cache, transport, round trips, ...) can't.
Views []string
}
// ForViews returns a querier limited to views (all views when empty).
func (q *Querier) ForViews(views []string) *Querier {
c := *q
c.Views = views
return &c
}
func (q *Querier) filtered() bool { return len(q.Views) > 0 }
// perView reports whether a counter key exists per view, and its name
// there ("queries" is counted as responses).
func perView(key string) (string, bool) {
if key == "queries" || key == "responses" {
return "responses", true
}
if key == "blocked" {
return key, true
}
for _, p := range []string{"rcode:", "qtype:", "zone:", "blocklist:", "rrl:"} {
if strings.HasPrefix(key, p) || key+":" == p {
return key, true
}
}
return key, false
}
// PerViewGroups are the Breakdown groups that exist per view.
var PerViewGroups = []string{"qtype", "rcode", "view", "zone", "blocklist", "rrl"}
// ErrNotPerView is returned for figures named only keeps for the whole server.
var ErrNotPerView = fmt.Errorf("named keeps this for the whole server only, not per view")
// sums is Store.sums, limited to the querier's views.
func (q *Querier) sums(ctx context.Context, res string, from, to int64, prefix string) (map[string]int64, error) {
if !q.filtered() {
return q.Store.sums(ctx, res, from, to, prefix)
}
if prefix == "view:" {
all, err := q.Store.sums(ctx, res, from, to, prefix)
if err != nil {
return nil, err
}
out := map[string]int64{}
for _, v := range q.Views {
if n := all["view:"+v]; n != 0 {
out["view:"+v] = n
}
}
return out, nil
}
if _, ok := perView(prefix); !ok {
// Not kept per view (named's own counters): the whole server's.
return q.Store.sums(ctx, res, from, to, prefix)
}
out := map[string]int64{}
for _, v := range q.Views {
m, err := q.Store.sums(ctx, res, from, to, ViewKey(v, prefix))
if err != nil {
return nil, err
}
for k, n := range m {
out[strings.TrimPrefix(k, ViewKey(v, ""))] += n
}
}
return out, nil
}
// series is Store.series, limited to the querier's views.
func (q *Querier) series(ctx context.Context, res string, from, to int64, key string) (map[int64]int64, error) {
pv, ok := perView(key)
if !q.filtered() || !ok {
return q.Store.series(ctx, res, from, to, key)
}
out := map[int64]int64{}
for _, v := range q.Views {
m, err := q.Store.series(ctx, res, from, to, ViewKey(v, pv))
if err != nil {
return nil, err
}
for ts, n := range m {
out[ts] += n
}
}
return out, nil
}
// ViewsSince is when per-view figures start (nil before any).
func (q *Querier) ViewsSince(ctx context.Context) *time.Time {
v, err := q.Store.Meta(ctx, "views_since")
if err != nil || v == "" {
return nil
}
n, err := strconv.ParseInt(v, 10, 64)
if err != nil {
return nil
}
t := time.Unix(n, 0)
return &t
}
// NewQuerier returns a querier for a collector's data.
@@ -105,10 +210,15 @@ func (s *Store) series(ctx context.Context, res string, from, to int64, key stri
return out, rows.Err()
}
// tops sums top lists in [from, to).
func (s *Store) tops(ctx context.Context, res, kind string, from, to int64, limit int) ([]Entry, error) {
rows, err := s.db.QueryContext(ctx, `SELECT key, SUM(count) AS n FROM tops WHERE res = ? AND kind = ? AND ts >= ? AND ts < ?
GROUP BY key ORDER BY n DESC, key LIMIT ?`, res, kind, from, to, limit)
// tops sums top lists (of one or more kinds) in [from, to).
func (s *Store) tops(ctx context.Context, res string, kinds []string, from, to int64, limit int) ([]Entry, error) {
args := []any{res}
for _, k := range kinds {
args = append(args, k)
}
args = append(args, from, to, limit)
rows, err := s.db.QueryContext(ctx, `SELECT key, SUM(count) AS n FROM tops WHERE res = ? AND kind IN (`+strings.TrimSuffix(strings.Repeat("?,", len(kinds)), ",")+`) AND ts >= ? AND ts < ?
GROUP BY key ORDER BY n DESC, key LIMIT ?`, args...)
if err != nil {
return nil, err
}
@@ -167,6 +277,11 @@ type Summary struct {
BlockedRatio *float64 `json:"blocked_ratio,omitempty"`
Blocklists []Entry `json:"blocklists,omitempty"`
Tops map[string][]Entry `json:"tops,omitempty"`
// Views and ViewsSince are set when the figures are for some views
// only: queries then counts responses, and named's own counters
// (answers, transport, cache) are left out.
FilterViews []string `json:"filter_views,omitempty"`
ViewsSince *time.Time `json:"views_since,omitempty"`
}
// Summary summarizes the last since.
@@ -179,9 +294,32 @@ func (q *Querier) Summary(ctx context.Context, since time.Duration, topLimit int
if err != nil {
return nil, err
}
if q.filtered() {
// The chosen views' counters, under their usual names.
views := map[string]int64{}
for _, v := range q.Views {
views["view:"+v] = all["view:"+v]
}
picked := map[string]int64{}
for k, n := range all {
for _, v := range q.Views {
if rest, ok := strings.CutPrefix(k, ViewKey(v, "")); ok {
picked[rest] += n
}
}
}
for k, n := range views {
picked[k] = n
}
picked["queries"] = picked["responses"]
all = picked
}
s := &Summary{From: time.Unix(from, 0), To: now, Resolution: res, Queries: all["queries"],
RCodes: sorted(all, "rcode:"), QTypes: sorted(all, "qtype:"), Views: sorted(all, "view:"), Zones: sorted(all, "zone:"),
Answers: sorted(all, "answer:"), Transport: sorted(all, "transport:")}
if q.filtered() {
s.FilterViews, s.ViewsSince = q.Views, q.ViewsSince(ctx)
}
if h, m := all["cache:hits"], all["cache:misses"]; h+m > 0 {
r := float64(h) / float64(h+m)
s.CacheHitRatio = &r
@@ -195,7 +333,7 @@ func (q *Querier) Summary(ctx context.Context, since time.Duration, topLimit int
s.BlockedRatio = &b
}
}
per, err := q.Store.series(ctx, res, from, to, "queries")
per, err := q.series(ctx, res, from, to, "queries")
if err != nil {
return nil, err
}
@@ -237,7 +375,14 @@ func (q *Querier) Top(ctx context.Context, kind string, since time.Duration, lim
}
now := q.Now()
res := q.topsResolution(since)
return q.Store.tops(ctx, res, kind, q.bucketStart(res, now.Add(-since)), now.Unix()+1, limit)
kinds := []string{kind}
if q.filtered() {
kinds = nil
for _, v := range q.Views {
kinds = append(kinds, ViewKind(kind, v))
}
}
return q.Store.tops(ctx, res, kinds, q.bucketStart(res, now.Add(-since)), now.Unix()+1, limit)
}
// Breakdown returns the totals of a counter group (qtype, rcode, view,
@@ -246,9 +391,12 @@ func (q *Querier) Breakdown(ctx context.Context, group string, since time.Durati
if !slices.Contains(Groups, group) {
return nil, fmt.Errorf("unknown group %q (use %s)", group, strings.Join(Groups, ", "))
}
if q.filtered() && !slices.Contains(PerViewGroups, group) {
return nil, ErrNotPerView
}
now := q.Now()
res := q.Resolution(since)
m, err := q.Store.sums(ctx, res, q.bucketStart(res, now.Add(-since)), now.Unix()+1, group+":")
m, err := q.sums(ctx, res, q.bucketStart(res, now.Add(-since)), now.Unix()+1, group+":")
if err != nil {
return nil, err
}
@@ -329,7 +477,7 @@ func (q *Querier) Timeline(ctx context.Context, since time.Duration, step string
var expanded []string
for _, k := range keys {
if g, ok := strings.CutSuffix(k, ":*"); ok {
m, err := q.Store.sums(ctx, st.res, from, to, g+":")
m, err := q.sums(ctx, st.res, from, to, g+":")
if err != nil {
return nil, err
}
@@ -342,7 +490,7 @@ func (q *Querier) Timeline(ctx context.Context, since time.Duration, step string
}
tl := &Timeline{Step: st.name, Times: starts, Series: []*Series{}}
bucketed := func(key string) ([]int64, error) {
raw, err := q.Store.series(ctx, st.res, from, to, key)
raw, err := q.series(ctx, st.res, from, to, key)
if err != nil {
return nil, err
}
@@ -404,7 +552,7 @@ func (q *Querier) Heatmap(ctx context.Context, key string, since time.Duration)
since = min(since, q.Opts.RetainHour)
now := q.Now()
from := q.bucketStart(ResHour, now.Add(-since))
raw, err := q.Store.series(ctx, ResHour, from, now.Unix()+1, key)
raw, err := q.series(ctx, ResHour, from, now.Unix()+1, key)
if err != nil {
return nil, err
}
+19
View File
@@ -264,3 +264,22 @@ func (s *Store) TopsAt(ctx context.Context, res string, ts int64, kind string) (
}
return out, rows.Err()
}
// KindsAt returns the top-list kinds stored for one bucket (per-view
// kinds included).
func (s *Store) KindsAt(ctx context.Context, res string, ts int64) ([]string, error) {
rows, err := s.db.QueryContext(ctx, `SELECT DISTINCT kind FROM tops WHERE res = ? AND ts = ?`, res, ts)
if err != nil {
return nil, err
}
defer rows.Close()
var out []string
for rows.Next() {
var k string
if err := rows.Scan(&k); err != nil {
return nil, err
}
out = append(out, k)
}
return out, rows.Err()
}
+103
View File
@@ -0,0 +1,103 @@
package stats
import (
"context"
"errors"
"io"
"log/slog"
"os"
"path/filepath"
"testing"
"time"
)
// Figures for chosen views: from the response log, per view, and added
// up for several.
func TestViewFilter(t *testing.T) {
ctx := context.Background()
loc := time.FixedZone("test", 0)
dir := t.TempDir()
logFile := filepath.Join(dir, "responses.log")
store, err := OpenStore(filepath.Join(dir, "stats.db"))
if err != nil {
t.Fatal(err)
}
defer store.Close()
named := newFakeNamed(t)
clk := &clock{time.Date(2026, 10, 5, 10, 0, 30, 0, loc)}
opts := Options{QueryLog: true, LogFile: logFile, StatisticsChannel: "unused", TopN: 10,
RetainMinute: 7 * 24 * time.Hour, RetainHour: 90 * 24 * time.Hour, RetainDay: 730 * 24 * time.Hour, Location: loc}
zones := func() map[string][]string {
return map[string][]string{"internal": {"adbraun.lol"}, "external": {"adbraun.lol"}}
}
c := NewCollector(store, opts, zones, slog.New(slog.NewTextHandler(io.Discard, nil)))
c.poller.URL = named.srv.URL
c.now = clk.now
c.start(ctx)
at := time.Date(2026, 10, 5, 10, 0, 40, 0, loc)
lines := []string{
logLine(at, "10.10.0.37", "internal", "nas.adbraun.lol", "A", "NOERROR"),
logLine(at, "10.10.0.37", "internal", "nas.adbraun.lol", "AAAA", "NOERROR"),
logLine(at, "10.10.0.42", "internal", "nope.adbraun.lol", "A", "NXDOMAIN"),
logLine(at, "203.0.113.9", "external", "www.adbraun.lol", "A", "NOERROR"),
logLine(at, "203.0.113.9", "external", "x.example", "A", "REFUSED"),
logLine(at, "192.0.2.5", "vps", "debian.org", "AAAA", "NOERROR"),
}
f, _ := os.Create(logFile)
for _, l := range lines {
f.WriteString(l)
}
f.Close()
c.readLog()
named.add(6, 1)
clk.t = time.Date(2026, 10, 5, 10, 1, 2, 0, loc)
c.flush(ctx, false)
clk.t = time.Date(2026, 10, 5, 11, 0, 2, 0, loc)
c.flush(ctx, false)
all := &Querier{Store: store, Opts: opts, Now: clk.now}
s, err := all.Summary(ctx, 24*time.Hour, 5)
if err != nil || s.Queries != 6 || s.FilterViews != nil {
t.Fatalf("all views: %+v %v", s, err)
}
in := all.ForViews([]string{"internal"})
s, err = in.Summary(ctx, 24*time.Hour, 5)
if err != nil {
t.Fatal(err)
}
if s.Queries != 3 || len(s.Views) != 1 || s.Views[0].Key != "internal" || s.ViewsSince == nil {
t.Errorf("internal: queries %d views %v since %v", s.Queries, s.Views, s.ViewsSince)
}
if len(s.RCodes) != 2 || s.RCodes[0] != (Entry{Key: "NOERROR", Count: 2}) || len(s.QTypes) != 2 {
t.Errorf("internal rcodes %v qtypes %v", s.RCodes, s.QTypes)
}
if s.Tops[KindName][0] != (Entry{Key: "nas.adbraun.lol", Count: 2}) || len(s.Tops[KindNXDomain]) != 1 || len(s.Answers) != 0 {
t.Errorf("internal tops %v answers %v", s.Tops, s.Answers)
}
// Linked views, say: two together.
two := all.ForViews([]string{"internal", "external"})
s, _ = two.Summary(ctx, 24*time.Hour, 5)
if s.Queries != 5 || len(s.Views) != 2 {
t.Errorf("internal+external: %d %v", s.Queries, s.Views)
}
if es, _ := two.Top(ctx, KindClient, 24*time.Hour, 5); len(es) != 3 || es[2].Key != "10.10.0.42" {
t.Errorf("clients of two views: %v", es)
}
if es, _ := two.Breakdown(ctx, "rcode", 24*time.Hour); len(es) != 3 {
t.Errorf("rcodes of two views: %v", es)
}
if _, err := two.Breakdown(ctx, "answer", 24*time.Hour); !errors.Is(err, ErrNotPerView) {
t.Errorf("answers per view: %v", err)
}
tl, err := all.ForViews([]string{"vps"}).Timeline(ctx, 2*time.Hour, "1m", []string{"queries", "rcode:*"})
if err != nil {
t.Fatal(err)
}
var total int64
for _, v := range tl.Series[0].Values {
total += v
}
if total != 1 || len(tl.Series) != 2 || tl.Series[1].Key != "rcode:NOERROR" {
t.Errorf("vps timeline: %d %+v", total, tl.Series)
}
}
+47 -15
View File
@@ -45,21 +45,25 @@ const routes = [
{ pattern: /^encrypted-dns$/, page: encdns, nav: "encdns" },
];
// The top bar: the pages used most, then a menu for the server's
// configuration and security. ["key", "#/href", "Label", "admin"?]
const navItems = [
["dashboard", "#/", "Dashboard"],
["stats", "#/stats", "Statistics"],
["zones", "#/zones", "Zones"],
["blocking", "#/blocking", "Blocking"],
["views", "#/views", "Views"],
["exposure", "#/exposure", "Exposure"],
["certificates", "#/certificates", "Certificates", "admin"],
["encdns", "#/encrypted-dns", "Encrypted DNS", "admin"],
["acls", "#/acls", "ACLs"],
["keys", "#/keys", "TSIG keys"],
["options", "#/options", "Server options"],
["blocking", "#/blocking", "Blocking"],
["changes", "#/changes", "Changes"],
["history", "#/history", "History"],
["audit", "#/audit", "Audit log"],
{ menu: "Server", items: [
["options", "#/options", "Server options"],
["acls", "#/acls", "ACLs"],
["keys", "#/keys", "TSIG keys"],
["exposure", "#/exposure", "Exposure"],
["certificates", "#/certificates", "Certificates", "admin"],
["encdns", "#/encrypted-dns", "Encrypted DNS", "admin"],
["audit", "#/audit", "Audit log"],
] },
];
const app = document.getElementById("app");
@@ -168,12 +172,16 @@ function layout() {
logo(30),
wordmark({ rule: false, height: 19, cls: "brand-name" })),
navEl,
h("div", { class: "spacer" }),
modeEl = h("span"),
h("form", { class: "search", onsubmit: (e) => { e.preventDefault(); if (q.value.trim()) location.hash = "#/search?q=" + encodeURIComponent(q.value.trim()); } }, q),
h("a", { href: "#/account", class: "user", title: (me.permissions || []).join("\n") }, me.user.username),
me.admin ? h("a", { href: "#/users" }, "Users") : null,
h("button", { class: "small", onclick: logout }, "Log out"));
// The right-hand group stays together, on the right, also when it
// has to wrap below the navigation.
h("div", { class: "header-right" },
modeEl = h("span"),
h("form", { class: "search", onsubmit: (e) => { e.preventDefault(); if (q.value.trim()) location.hash = "#/search?q=" + encodeURIComponent(q.value.trim()); } }, q),
menu(h("span", { class: "user", title: (me.permissions || []).join("\n") }, me.user.username), [
h("a", { href: "#/account" }, "Account"),
me.admin ? h("a", { href: "#/users" }, "Users") : null,
h("button", { type: "button", class: "menu-item", onclick: logout }, "Log out"),
], { right: true, active: ["account", "users"].includes(current.nav), label: "Account menu" })));
mount(app, h("div", { class: "shell" }, header, barEl, mainEl, footer(me.build)));
modeHolder = modeEl;
renderNav();
@@ -196,9 +204,33 @@ function footer(b) {
}
function renderNav() {
mount(navEl, navItems.filter((n) => n[3] !== "admin" || state.me.admin).map(([key, href, label]) => h("a", { href, class: current.nav === key ? "active" : "" }, label)));
const allowed = (n) => n[3] !== "admin" || state.me.admin;
const link = ([key, href, label]) => h("a", { href, class: current.nav === key ? "active" : "" }, label);
mount(navEl, navItems.map((n) => {
if (!n.menu) return allowed(n) ? link(n) : null;
const items = n.items.filter(allowed);
return menu(n.menu, items.map(link), { active: items.some((i) => i[0] === current.nav), label: n.menu + " menu" });
}));
}
// menu is a drop-down: a button and its links. It closes when one is
// picked, on a click elsewhere and on Escape.
function menu(label, items, opts = {}) {
const d = h("details", { class: "menu" + (opts.right ? " right" : "") },
h("summary", { class: opts.active ? "active" : "", "aria-label": opts.label || null }, label, h("span", { class: "caret", "aria-hidden": "true" }, "▾")),
h("div", { class: "menu-list", role: "menu" }, items.filter(Boolean)));
d.addEventListener("click", (e) => { if (e.target.closest(".menu-list a, .menu-list button")) d.open = false; });
return d;
}
document.addEventListener("click", (e) => {
for (const d of document.querySelectorAll("details.menu[open]")) if (!d.contains(e.target)) d.open = false;
});
document.addEventListener("keydown", (e) => {
if (e.key !== "Escape") return;
for (const d of document.querySelectorAll("details.menu[open]")) { d.open = false; d.querySelector("summary").focus(); }
});
// renderBar shows the pending-changes bar (only while there are pending
// changes) and the small review/live switch in the header.
function renderBar() {
@@ -18,6 +18,9 @@ export async function render(el) {
st.backup ? backupCard(st.backup) : null,
exposure ? exposureCard(exposure) : null,
st.certificates ? certCard(st.certificates) : null,
st.encrypted_dns ? h("a", { class: "card link", href: "#/encrypted-dns" }, h("div", { class: "label" }, "Encrypted DNS"),
h("div", { class: "value" }, st.encrypted_dns.ok ? badge("working", "ok") : badge("not working", "err")),
h("div", { class: "sub" }, "checked ", time(st.encrypted_dns.checked))) : null,
);
const last = s.last_applied;
const parts = [h("div", { class: "page-head" },
@@ -43,6 +46,9 @@ export async function render(el) {
h("strong", null, "The last backup failed "), time(b.last_error_at), ": " + b.last_error,
h("div", { class: "small" }, "See 'journalctl -u jarl-backup' on the server.")));
}
for (const p of (st.encrypted_dns || {}).problems || []) {
parts.push(h("div", { class: "msg err" }, h("strong", null, "Encrypted DNS: "), p, " ", h("a", { href: "#/encrypted-dns" }, "Encrypted DNS")));
}
for (const c of (st.certificates || {}).problems || []) {
parts.push(h("div", { class: "msg " + (c.state === "expiring" ? "warn" : "err") },
h("strong", null, "Certificate " + c.name + " " + (c.state === "expired" ? "has expired" : c.state === "expiring" ? "expires soon" : "can't be renewed") + ": "), c.message, " ",
+27 -1
View File
@@ -2,7 +2,7 @@
// set up and what to give clients, and a setup that obtains the
// certificate if needed, checks the ports and stages named's
// configuration like any change.
import { h, mount, field, input, badge, toast, errorBox, confirmDialog, copyBox, checkbox } from "../ui.js";
import { h, mount, field, input, badge, toast, errorBox, confirmDialog, copyBox, checkbox, fmtTime, relTime } from "../ui.js";
import { api, get, change, describeResult, state } from "../api.js";
import { rerender } from "../app.js";
@@ -50,6 +50,7 @@ function statusPanel(s) {
}
return [
h("div", { class: "panel" }, h("h2", null, "Set up ", badge("on", "ok")), h("dl", { class: "kv" }, rows)),
healthPanel(s.health),
h("div", { class: "panel" }, h("h2", null, "For your clients"),
h("p", { class: "muted small" }, "Clients check the certificate against the host name, so it must resolve to this server for them (Jarl added it to the private views you chose). Open the ports in the server's firewall for your networks only."),
clients),
@@ -223,3 +224,28 @@ async function disable() {
rerender();
} catch (e) { toast(e.message, "err", 8000); }
}
// healthPanel shows the last check of the listeners: Jarl connects to each
// as a client would, checks the certificate and asks a question.
function healthPanel(hl) {
const kind = { dot: "DNS over TLS", doh: "DNS over HTTPS" };
const out = h("div");
const draw = (x) => {
if (!x) return mount(out, h("div", { class: "muted" }, "Not checked yet: Jarl checks a minute after it starts, then every 10 minutes."));
mount(out, h("ul", { class: "probes" }, x.probes.map((p) => h("li", null,
badge(p.ok ? "working" : "not working", p.ok ? "ok" : "err"), " ",
h("strong", null, kind[p.kind]), " ", h("span", { class: "mono" }, p.address + " port " + p.port),
p.ok ? h("span", { class: "muted small" }, " answered " + p.rcode + " in " + p.ms + " ms" + (p.cert_not_after ? " · certificate until " + fmtTime(p.cert_not_after) : "")) : null,
p.problem ? h("div", { class: "msg err small" }, p.problem) : null,
p.warning ? h("div", { class: "msg warn small" }, p.warning) : null))),
h("div", { class: "muted small" }, "Checked ", relTime(x.checked), "."));
};
draw(hl);
const btn = h("button", { class: "small", onclick: async () => {
btn.disabled = true;
try { draw(await api("POST", "/encrypted-dns/health", { body: {} })); } catch (e) { toast(e.message, "err", 8000); } finally { btn.disabled = false; }
} }, "Check now");
return h("div", { class: "panel" }, h("div", { class: "page-head" }, h("h2", { class: "grow" }, "Health"), h("div", { class: "actions" }, btn)),
h("p", { class: "muted small" }, "Jarl connects to each listener as a client would: the certificate must be for the host name, current, and the one Jarl last wrote; then it asks for the host name."),
out);
}
+35 -3
View File
@@ -1,8 +1,8 @@
// Internet exposure (docs/exposure.md): which views are reachable from
// the internet, why, and what to harden, with fixes staged like any edit
// and findings that can be accepted on purpose.
import { h, mount, table, toast, badge, modal, input, field, errorBox, select, plural } from "../ui.js";
import { get, change, describeResult, can } from "../api.js";
import { h, mount, table, toast, badge, modal, input, field, errorBox, select, plural, relTime } from "../ui.js";
import { api, get, change, describeResult, can } from "../api.js";
import { rerender } from "../app.js";
const sevKind = { warning: "warn", error: "err", info: "" };
@@ -28,7 +28,11 @@ export async function render(el) {
parts.push(h("div", { class: "panel" }, h("h2", null, "Findings ", badge(plural(all.filter((f) => f.severity === "warning").length, "warning"), all.some((f) => f.severity === "warning") ? "warn" : "ok")),
all.length ? h("div", { class: "findings" }, all.map((f) => findingRow(f, writable))) : h("div", { class: "empty" }, r.scopes.some((s) => s.public) ? "Nothing to harden." : "No view is reachable from the internet, so there is nothing to check.")));
if (r.rate_limiting) parts.push(activity(r.rate_limiting, r.scopes));
if (r.rate_limiting && r.rate_limit_log) parts.push(activity(r.rate_limiting, r.scopes));
else if (r.rate_limiting && r.scopes.some((s) => s.public && s.rate_limit !== "off")) {
parts.push(h("div", { class: "msg info" }, "What rate limiting limits isn't counted: stats.rate_limit_log is off in /etc/jarl/jarl.yaml, so its log lines stay in named's syslog only."));
}
if (r.scopes.some((s) => s.public)) parts.push(mailPanel(r.mail));
mount(el, parts);
}
@@ -118,3 +122,31 @@ function activity(a, scopes) {
table([{ label: "Client network (limited or would be)", render: (e) => h("span", { class: "mono" }, e.key) }, { label: "Responses", render: (e) => String(e.count) }], a.networks || []),
table([{ label: "Name and type", render: (e) => h("span", { class: "mono" }, e.key) }, { label: "Responses", render: (e) => String(e.count) }], a.names || []))]);
}
// mailPanel: the reverse DNS of the addresses that send and receive the
// public zones' mail, as receivers check it (docs/exposure.md#mail).
function mailPanel(m) {
const out = h("div");
const lv = { ok: "ok", info: "", warning: "warn" };
const draw = (x) => {
if (!x) return mount(out, h("div", { class: "muted" }, "Not checked yet: Jarl checks a minute after it starts, then every 6 hours."));
const rows = x.results.length ? table([
{ label: "Address", render: (r) => [h("span", { class: "mono" }, r.address), h("div", { class: "muted small" }, r.zone + ": " + r.roles.join(", "))] },
{ label: "PTR", render: (r) => r.ptr.length ? h("span", { class: "mono" }, r.ptr.join(", ")) : h("span", { class: "muted" }, "none") },
{ label: "Reverse zone run by", render: (r) => h("span", { class: "small" }, r.managed_by || "–") },
{ label: "", render: (r) => [badge(r.ok ? (r.level === "info" ? "works, note" : "works") : "problem", lv[r.level]),
r.level !== "ok" ? h("div", { class: "small" }, r.message) : null,
r.local_zone ? h("div", { class: "muted small" }, "Jarl's reverse zone " + r.local_zone + " only answers your own clients: receivers ask " + (r.managed_by || "the address's owner") + ".") : null] },
], x.results) : h("div", { class: "empty" }, "No public zone has an MX record or SPF addresses.");
mount(out, rows, (x.errors || []).map((e) => h("div", { class: "msg err small" }, e)),
h("div", { class: "muted small" }, "Checked ", relTime(x.checked), "."));
};
draw(m);
const btn = h("button", { class: "small", onclick: async () => {
btn.disabled = true;
try { draw(await api("POST", "/exposure/mail", { body: {} })); } catch (e) { toast(e.message, "err", 8000); } finally { btn.disabled = false; }
} }, "Check now");
return h("div", { class: "panel" }, h("div", { class: "page-head" }, h("h2", { class: "grow" }, "Mail reverse DNS"), h("div", { class: "actions" }, btn)),
h("p", { class: "muted small" }, "Receivers such as Gmail and Outlook reject or mark mail from an address without a PTR record whose name resolves back to it. These records belong to whoever owns the address (your hosting provider or ISP), so Jarl looks them up as the internet sees them: the addresses of your public zones' MX hosts and SPF records."),
out);
}
+19 -1
View File
@@ -159,14 +159,32 @@ export async function render(el) {
const settingsEl = h("div");
const serverEl = h("div");
const topEl = h("div");
const genEl = h("div");
mount(el,
h("div", { class: "page-head" }, h("div", { class: "grow" }, h("h1", null, "Server options"),
h("div", { class: "muted" }, "Global named options, top-level statements such as logging and controls, and Jarl's defaults for new zones."))),
settingsEl, serverEl, topEl);
settingsEl, serverEl, topEl, genEl);
settingsForm(settingsEl, settings);
await optionsEditor(serverEl, { scope: "options", catalogScope: "options", title: "options { }", intro: "These apply to the whole server; views and zones can override many of them." });
await optionsEditor(topEl, { scope: "toplevel", catalogScope: "toplevel", title: "Top-level statements",
intro: "Statements outside options { }, such as logging, controls, statistics-channels, server, remote-servers, tls or include. Values are named.conf syntax, e.g. { channel ... ; category ... ; } for logging." });
generatedPanel(genEl, await get("/options/generated").catch(() => []));
}
// generatedPanel lists what Jarl adds to named.conf by itself: read-only,
// with why and what turns it on or off.
function generatedPanel(el, list) {
if (!list.length) return;
mount(el, h("div", { class: "panel" },
h("h2", null, "Added by Jarl"),
h("p", { class: "muted" }, "Jarl writes these into named.conf itself, for its own features. They aren't options you can edit here: each is controlled by the setting shown."),
h("div", { class: "table-wrap" }, h("table", null,
h("thead", null, h("tr", null, h("th", null, "Where"), h("th", null, "Statement"), h("th", null, "Why"), h("th", null, "Controlled by"))),
h("tbody", null, list.map((g) => h("tr", null,
h("td", { class: "small" }, g.where),
h("td", null, h("pre", { class: "code optval", style: "white-space: pre-wrap; word-break: break-all" }, g.text)),
h("td", { class: "small" }, g.why),
h("td", { class: "small mono" }, g.setting))))))));
}
function settingsForm(el, s) {
+61 -15
View File
@@ -1,5 +1,6 @@
import { h, mount, fmtTime, relTime, errorBox } from "../ui.js";
import { get } from "../api.js";
import { rerender } from "../app.js";
import { lineChart, chartCard, legend, timeTable, heatmap, heatmapTable, barList, compact, stepSeconds, endLabelWidth } from "../charts.js";
const ranges = [["1h", "1 hour"], ["6h", "6 hours"], ["24h", "24 hours"], ["7d", "7 days"], ["30d", "30 days"], ["90d", "90 days"]];
@@ -17,6 +18,20 @@ const rttOrder = ["<10ms", "10-100ms", "100-500ms", "500-800ms", "800-1600ms", "
let generation = 0;
// The views shown are remembered per browser: the ones switched off, so a
// new view shows up switched on.
const offKey = "jarl.stats.views.off";
function viewsOff() {
try { return new Set(JSON.parse(localStorage.getItem(offKey) || "[]")); } catch { return new Set(); }
}
function saveViewsOff(set) {
try { localStorage.setItem(offKey, JSON.stringify([...set])); } catch { /* private window: not remembered */ }
}
// Breakdowns that exist per view; the others are named's own counters,
// for the whole server.
const perViewGroups = ["qtype", "rcode", "view", "zone", "blocklist"];
export async function render(el, params, query) {
const gen = ++generation;
const want = query && query.get ? query.get("since") : null;
@@ -33,11 +48,21 @@ export async function render(el, params, query) {
return;
}
const heatSince = rangeSecs[since] < 7 * 86400 ? "7d" : since;
// Which views: all of them unless some are switched off.
const allViews = status.query_log ? (await get("/views").catch(() => [])).map((v) => v.name) : [];
const off = viewsOff();
const shown = allViews.filter((v) => !off.has(v));
const filtered = allViews.length > 0 && shown.length < allViews.length;
const vf = filtered ? { views: shown.join(",") } : {};
if (filtered && !shown.length) {
mount(el, head, filterRow(since, allViews, off), h("div", { class: "empty" }, "No views chosen: switch one on above."));
return;
}
let data;
try {
data = await loadAll(since, heatSince, status.query_log);
data = await loadAll(since, heatSince, status.query_log, vf, filtered);
} catch (e) {
mount(el, head, errorBox(e));
mount(el, head, filterRow(since, allViews, off), errorBox(e));
return;
}
if (gen !== generation) return; // the user moved on
@@ -46,13 +71,17 @@ export async function render(el, params, query) {
// sudden drop in traffic.
const timeline = complete(data.timeline), errors = complete(data.errors);
const step = stepSeconds[timeline.step];
const parts = [head, filterRow(since)];
const parts = [head, filterRow(since, allViews, off)];
for (const w of status.warnings || []) parts.push(h("div", { class: "msg warn" }, w));
if (filtered) {
parts.push(h("div", { class: "msg info small" }, "Showing view" + (shown.length > 1 ? "s " : " ") + shown.join(", ") + ": answers from named's response log" +
(s.views_since ? ", per view since " + fmtTime(s.views_since) : "") + ". named's own figures (answer sources, upstream times, cache, the server) are for the whole server."));
}
// Headline numbers.
const rangeLabel = "in the last " + ranges.find((r) => r[0] === since)[1];
parts.push(h("div", { class: "cards" },
tile("Queries", compact(s.queries), rangeLabel),
tile(filtered ? "Answers" : "Queries", compact(s.queries), rangeLabel),
tile("Per second", s.avg_qps < 10 ? s.avg_qps.toFixed(2) : Math.round(s.avg_qps).toLocaleString(), "on average"),
s.peak ? tile("Busiest " + s.peak.per, compact(s.peak.queries), fmtTime(s.peak.time)) : null,
s.nxdomain_ratio != null ? tile("NXDOMAIN", pct(s.nxdomain_ratio), "of queries named no existing name") : null,
@@ -62,7 +91,7 @@ export async function render(el, params, query) {
// Volume over time: one series, so no legend box; the title names it.
// With blocking, a second line shows how many of them were blocked.
const q = timeline.series.find((x) => x.key === "queries") || { values: timeline.times.map(() => 0) };
const qSeries = [{ label: "Queries", values: q.values, slot: 1 }];
const qSeries = [{ label: filtered ? "Answers" : "Queries", values: q.values, slot: 1 }];
const bl = timeline.series.find((x) => x.key === "blocked");
if (bl && bl.values.some((v) => v > 0)) qSeries.push({ label: "Blocked", values: bl.values, slot: 2 });
// Failed answers: three fixed series, legend plus end labels.
@@ -71,7 +100,8 @@ export async function render(el, params, query) {
// The two charts are one above the other: the same room for labels on
// both sides keeps their time axes in line.
const gutter = { left: 52, right: Math.max(endLabelWidth(qSeries), anyFailures ? endLabelWidth(fSeries) : 0) };
parts.push(chartCard(qSeries.length > 1 ? "Queries and blocked queries" : "Queries", "per " + stepName(timeline.step) + ", " + rangeLabel,
const qWord = filtered ? "Answers" : "Queries";
parts.push(chartCard(qSeries.length > 1 ? qWord + " and blocked queries" : qWord, "per " + stepName(timeline.step) + ", " + rangeLabel,
lineChart({ series: qSeries, times: timeline.times, step, area: true, gutter }), timeTable(qSeries, timeline.times, step),
{ wide: true, legend: qSeries.length > 1 ? legend(qSeries) : null }));
@@ -117,19 +147,20 @@ export async function render(el, params, query) {
scheduleRefresh(el, params, query, gen);
}
async function loadAll(since, heatSince, queryLog) {
async function loadAll(since, heatSince, queryLog, vf, filtered) {
const p = {
summary: get("/stats/summary", { since, top: 0 }),
timeline: get("/stats/timeline", { since, key: "queries,blocked" }),
errors: get("/stats/timeline", { since, key: failures.map((f) => f.key).join(",") }),
heat: get("/stats/heatmap", { since: heatSince }),
summary: get("/stats/summary", { since, top: 0, ...vf }),
timeline: get("/stats/timeline", { since, key: "queries,blocked", ...vf }),
errors: get("/stats/timeline", { since, key: failures.map((f) => f.key).join(","), ...vf }),
heat: get("/stats/heatmap", { since: heatSince, ...vf }),
server: get("/stats/server").catch(() => null),
gauges: get("/stats/timeline", { since, key: gaugeKeys.join(",") }),
};
const groups = ["qtype", "rcode", "view", "zone", "answer", "rtt", "blocklist"];
groups.forEach((g) => { p["b_" + g] = get("/stats/breakdown", { group: g, since }); });
// With some views only, named's own counters aren't asked for.
groups.forEach((g) => { p["b_" + g] = filtered && !perViewGroups.includes(g) ? Promise.resolve([]) : get("/stats/breakdown", { group: g, since, ...vf }); });
if (queryLog) {
for (const k of ["name", "client", "nxdomain", "refused", "servfail", "blocked", "blocked_client"]) p["t_" + k] = get("/stats/top", { kind: k, since, limit: 15 });
for (const k of ["name", "client", "nxdomain", "refused", "servfail", "blocked", "blocked_client"]) p["t_" + k] = get("/stats/top", { kind: k, since, limit: 15, ...vf });
}
const keys = Object.keys(p);
const vals = await Promise.all(Object.values(p));
@@ -162,11 +193,26 @@ function complete(tl) {
return tl;
}
function filterRow(since) {
return h("div", { class: "filter-row", role: "group", "aria-label": "Time range" },
function filterRow(since, views, off) {
const range = h("div", { class: "filter-row", role: "group", "aria-label": "Time range" },
h("span", { class: "muted" }, "Range"),
h("div", { class: "segmented" }, ranges.map(([k, label]) =>
h("a", { href: "#/stats?since=" + k, class: k === since ? "active" : "", "aria-current": k === since ? "true" : null, title: "last " + label }, k))));
if (!views || views.length < 2) return range;
// A toggle per view (together: linked views, say), and all or none.
const set = (next) => { saveViewsOff(next); rerender(); };
const toggles = views.map((v) => {
const on = !off.has(v);
return h("button", { type: "button", class: "chip toggle-chip" + (on ? " on" : ""), "aria-pressed": on ? "true" : "false",
title: on ? "Leave view " + v + " out" : "Include view " + v,
onclick: () => { const n = new Set(off); if (on) n.add(v); else n.delete(v); set(n); } }, v);
});
return h("div", { class: "filter-rows" }, range,
h("div", { class: "filter-row", role: "group", "aria-label": "Views" },
h("span", { class: "muted" }, "Views"),
h("div", { class: "chips" }, toggles),
h("button", { type: "button", class: "small", disabled: off.size === 0, onclick: () => set(new Set()) }, "All"),
h("button", { type: "button", class: "small", disabled: off.size === views.length, onclick: () => set(new Set(views)) }, "None")));
}
const gaugeKeys = ["g:named:rss", "g:named:mem", "g:cache:mem", "g:named:cpu"];
+40 -2
View File
@@ -21,6 +21,7 @@ export async function render(el, params, query) {
zone.type === "primary" ? ["file", "Zone file"] : null,
["primary", "secondary", "mirror"].includes(zone.type) ? ["servers", "Name servers"] : null,
zone.type === "primary" ? ["dnssec", "DNSSEC"] : null,
zone.other_views ? ["compare", "Compare views"] : null,
zone.registrar && state.me && state.me.admin ? ["registrar", "Registrar"] : null,
].filter(Boolean);
const body = h("div");
@@ -64,6 +65,7 @@ export async function render(el, params, query) {
else if (tab === "servers") await serversTab(body, zone);
else if (tab === "dnssec") await dnssecTab(body, zone);
else if (tab === "registrar") await registrarTab(body, zone);
else if (tab === "compare") await compareTab(body, zone);
// A later click (another tab) has taken over the page.
if (seq !== renderSeq || !holder.isConnected) return;
holder.replaceWith(body);
@@ -71,6 +73,40 @@ export async function render(el, params, query) {
if (first) first.focus();
}
// ---- comparing a zone across views
// compareTab shows the names and types that differ between the views that
// have the zone, with a button to share one view's version (linked zones).
async function compareTab(el, zone) {
let all = false;
const out = h("div");
const draw = async () => {
const r = await get("/zones/" + zone.id + "/compare", { all: all ? 1 : "" });
const head = h("tr", null, h("th", null, "Name"), h("th", null, "Type"), r.views.map((v) => h("th", null, v)));
const rows = r.rows.map((row) => h("tr", { class: row.same ? "muted" : "" },
h("td", { class: "mono" }, row.name), h("td", null, h("strong", null, row.type)),
row.views.map((vs) => h("td", { class: "compare-cell" },
vs.records.length ? vs.records.map((d) => h("div", { class: "mono small" }, d)) : h("span", { class: "muted" }, "–"),
vs.linked === "local" ? [badge("only " + vs.view, "accent"), " "] : null,
r.linked && !row.same && vs.records.length ? h("button", { class: "small", title: "Make view " + vs.view + "'s version every view's: the others' records of this name and type are replaced",
onclick: () => share(vs, row) }, "Use this version") : null))));
mount(out,
h("p", { class: "muted" }, r.same + " names and types are the same in " + r.views.join(", ") + (r.rows.length && !all ? "; these differ." : ".") + " ",
r.linked ? "The zones are linked: shared records are kept in step, and a view's own records override them there." : "The zones aren't linked: a change in one doesn't reach the others. Link them (Link… above) to keep what is the same in step; linking changes no answer."),
r.rows.length ? h("div", { class: "table-wrap" }, h("table", { class: "compare" }, h("thead", null, head), h("tbody", null, rows))) : h("div", { class: "empty" }, "Nothing differs."),
h("label", { class: "check" }, h("input", { type: "checkbox", checked: all, onchange: (e) => { all = e.target.checked; draw(); } }), "Show what is the same too"));
};
const share = async (vs, row) => {
if (!await confirmDialog("Use view " + vs.view + "'s " + row.name + " " + row.type, "Every view gets " + vs.records.join(", ") + " for " + row.name + " " + row.type + ", shared from now on; the other views' records of that name and type are replaced.", "Use this version")) return;
try {
toast(describeResult(await change("PATCH", "/records/" + vs.ids[0], { local: false })));
await draw();
} catch (e) { toast(e.message, "err", 8000); }
};
await draw();
mount(el, out);
}
// ---- linked zones (docs/views.md#linked-zones)
// linkButton links the zone with the same zone in other views, or takes
@@ -625,7 +661,7 @@ async function recordsTab(el, zone, query) {
const eTTL = input({ type: "number", value: r.ttl ?? "", placeholder: "default", style: { width: "90px" } });
const eData = input({ value: r.type === "TXT" && r.text !== undefined ? r.text : r.data, class: "mono", style: { width: "100%" } });
const eComment = input({ value: r.comment || "", style: { width: "100%" } });
const ePTR = checkbox("update PTR", false);
const ePTR = checkbox("update PTR", false, { title: "Keep the PTR record in the reverse zone in step: moved with a new name or address, or added if it is missing" });
const eErr = h("div");
const save = async () => {
const body = {};
@@ -638,7 +674,9 @@ async function recordsTab(el, zone, query) {
else if (eTTL.value !== "" && Number(eTTL.value) !== r.ttl) body.ttl = Number(eTTL.value);
if (eComment.value !== (r.comment || "")) body.comment = eComment.value;
if (ePTR.input.checked) body.ptr = true;
if (Object.keys(body).length === 0 || (Object.keys(body).length === 1 && body.ptr)) { editing = null; draw(); return; }
// "update PTR" alone adds a missing PTR record (the reverse zone may
// be newer than the record).
if (Object.keys(body).length === 0) { editing = null; draw(); return; }
try {
const res = await change("PATCH", "/records/" + r.id, body);
toast(describeResult(res));
+45
View File
@@ -522,3 +522,48 @@ table.pick-list tr.pick.selected { background: var(--accent-soft); }
/* Encrypted DNS: a protocol per line with its port. */
.enc-ports > div { margin: 2px 0; }
.enc-ports input[type=number] { display: inline-block; }
/* Statistics: views switched on and off. */
.filter-rows { display: flex; flex-direction: column; gap: 8px; margin-bottom: 12px; }
.filter-rows .filter-row { margin-bottom: 0; }
.filter-row .chips { display: flex; flex-wrap: wrap; gap: 6px; }
.chip.toggle-chip { opacity: 0.55; font-size: 13px; padding: 3px 12px; }
.chip.toggle-chip.on { opacity: 1; text-decoration: none; background: var(--accent-soft); border-color: var(--accent); }
/* Drop-down menus in the top bar (Server, the user). */
details.menu { position: relative; }
details.menu > summary { list-style: none; cursor: pointer; padding: 6px 10px; border-radius: var(--radius); color: var(--text); white-space: nowrap; }
details.menu > summary::-webkit-details-marker { display: none; }
details.menu > summary:hover, details.menu[open] > summary { background: var(--info-soft); }
details.menu > summary.active { background: var(--accent-soft); color: var(--accent); font-weight: 600; }
details.menu > summary .caret { margin-left: 4px; font-size: 0.8em; opacity: 0.7; }
details.menu > summary .user { color: inherit; }
details.menu .menu-list {
position: absolute; top: calc(100% + 4px); left: 0; z-index: 20; min-width: 190px;
display: flex; flex-direction: column; padding: 4px;
background: var(--panel); border: 1px solid var(--border); border-radius: var(--radius);
box-shadow: 0 6px 20px rgba(0, 0, 0, 0.12);
}
details.menu.right .menu-list { left: auto; right: 0; }
details.menu .menu-list a, details.menu .menu-list .menu-item {
display: block; padding: 7px 10px; border-radius: var(--radius); color: var(--text); text-align: left;
background: none; border: 0; font: inherit; cursor: pointer; white-space: nowrap;
}
details.menu .menu-list a:hover, details.menu .menu-list .menu-item:hover { background: var(--info-soft); text-decoration: none; }
details.menu .menu-list a.active { background: var(--accent-soft); color: var(--accent); font-weight: 600; }
nav.main { align-items: center; }
header.top .header-right { display: flex; align-items: center; gap: 12px; margin-left: auto; }
@media (max-width: 1200px) {
header.top form.search input { width: 160px; }
header.top { gap: 10px; }
}
/* Encrypted DNS: the health of each listener. */
ul.probes { list-style: none; padding: 0; margin: 0 0 8px; }
ul.probes > li { padding: 6px 0; border-bottom: 1px solid var(--border); }
ul.probes > li .msg { margin: 6px 0 0; }
/* Comparing a zone across views. */
table.compare td.compare-cell { vertical-align: top; }
table.compare td.compare-cell button { margin-top: 4px; }
table.compare tr.muted td { color: var(--muted); }
+32
View File
@@ -377,6 +377,15 @@ run "user delete" 0 $J user delete certbot -y
run "audit" 0 $J audit --limit 10
contains "token.revoke"
# A reverse zone created after the record: "update PTR" alone adds it.
run "an address record" 0 $J record add adbraun.test ldn A 192.0.2.154
run "its /32 reverse zone, afterwards" 0 $J zone create 192.0.2.154/32
LDN=$($J record list adbraun.test --name ldn -o id)
run "record update --ptr adds the missing PTR" 0 $J record update $LDN --ptr
expect_dig "the PTR is served" "ldn.adbraun.test." -x 192.0.2.154
run "remove the reverse zone again" 0 $J zone delete 154.2.0.192.in-addr.arpa -y
run "and the record" 0 $J record delete adbraun.test ldn A -y
# Views
run "view create (adopts zones)" 0 $J view create internal --match-clients homelab --adopt-zones
run "view create external" 0 $J view create external --match-clients any
@@ -409,11 +418,20 @@ expect_dig "now every view's" "10.1.1.1" -k $E/external.key linked.adbraun.test
run "--view all changes a link once" 0 $J record set adbraun.test linked A 10.1.1.2 --view all
contains "linked with internal"
expect_dig "both views" "10.1.1.2" -k $E/external.key linked.adbraun.test A
run "zone diff: what differs between the views" 0 $J zone diff adbraun.test
contains "ns1 *A *127.0.0.1 \\[local, id [0-9]*\\] *203.0.113.1 \\[local, id [0-9]*\\]"
contains "names and types differ, [0-9]* are the same; linked"
run "the compare API" 0 curl -sf -H "Authorization: Bearer $(cat $E/admin.token)" "http://127.0.0.1:$HTTP/api/v1/zones/adbraun.test/compare?view=internal&all=1"
contains '"linked":true'
run "a shared delete" 0 $J record delete adbraun.test linked A --view external -y
expect_dig "gone from internal too" "" linked.adbraun.test A
run "zone unlink" 0 $J zone unlink adbraun.test --view external
run "unlinked: no link column" 0 $J record list adbraun.test --view external
not_contains "shared"
run "--local on a zone that isn't linked is fine" 0 $J record add adbraun.test solo A 10.9.9.9 --view external --local
run "--shared there is refused (exit 6)" 6 $J record add adbraun.test solo2 A 10.9.9.8 --view external --shared
contains "nothing to share"
run "remove it" 0 $J record delete adbraun.test solo A --view external -y
run "zone create --link-with: a linked copy" 0 $J zone create linkcopy.test --view internal --ns ns1.adbraun.test.
run "a record in it" 0 $J record add linkcopy.test www A 192.0.2.80 --view internal
run "copy it into external, linked" 0 $J zone create linkcopy.test --view external --link-with internal
@@ -435,6 +453,13 @@ contains "rate-limit { responses-per-second 10; log-only yes; };"
run "exposure: log-only now" 0 $J exposure
contains "^external *public *log-only"
contains "exposure_rate_limit_log_only"
# Statistics are off here; render/stats_test.go covers their statements.
run "what Jarl adds by itself" 0 $J option generated
contains "^include .*keys.conf"
contains 'match-clients { !key "jarl-view-external"; key "jarl-view-internal"; ... };'
not_contains "statistics-channels"
run "generated, as JSON" 0 $J option generated -o json
contains '"setting": "always, when there are views"'
run "fix: start enforcing" 0 $J exposure fix external exposure_rate_limit_log_only
contains "responses-per-second 10; }"
# A burst of identical queries in the external view (signed with its key):
@@ -783,6 +808,13 @@ if [ -n "$PEBBLE" ]; then
contains "^192.168.1.20$"
run "plain DNS still answers on port 53's listener" 0 $DIG +short -p $DNS @127.0.0.1 nas.adbraun.test A
contains "^192.168.1.20$"
run "encrypted-dns health: both listeners work" 0 $J encrypted-dns health
contains "DoT 127.0.0.1 port $EDOT: *working (NOERROR"
contains "DoH 127.0.0.1 port $EDOH: *working"
contains "isn't from a CA this server trusts"
sleep 16 # /metrics caches Jarl's state for 15 seconds
run "health metric" 0 curl -sf http://127.0.0.1:$HTTP/metrics
contains "^jarl_encrypted_dns_up{kind=\"dot\",address=\"127.0.0.1\",port=\"$EDOT\"} 1$"
run "a certificate named uses can't be deleted (exit 6)" 6 $J cert delete ns1 --yes
contains "take it out first"
run "encrypted-dns disable" 0 $J encrypted-dns disable