Release 0.2.0-rc.3+git60-ee27884

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

CHANGELOG.md

@@ -8,6 +8,31 @@ All notable changes to Housecarl are documented here. The format follows
 
 ### Added
 
+- An administrator can make an API token for another user in the web
+  UI: New token on the user's row under Users (the CLI's `hc token create
+  --user` and the API could already).
+- **Ban lists** ([docs/bans.md](docs/bans.md)). A program (Guard, which
+  reads logs, or fail2ban) bans an address, of every port or of some,
+  for a time; the ban list says what the address is kept from: the
+  firewall itself (`firewall`), some hosts behind it (`hosts`, port
+  forwards to them included), or `everything`. Bans are dropped ahead of
+  every rule, established connections too, and the banned address's
+  connections are closed. IPv6 addresses are banned by /64 (`group_v6`).
+  Never banned: inside networks, the firewall's own addresses, whoever
+  asks, and the list's `never`. Bans end by themselves, are cut to the
+  list's longest time, survive restarts and reboots, and are audited, but
+  aren't configuration (no history, probation or changesets). `hc
+  banlist`, `hc bans`, `hc ban`, `hc unban`; Traffic → Bans in the web UI;
+  `/banlists` and `/banlists/{ref}/bans` in the API; a `bans` grant that
+  can name lists (`bans:write:web`), in the `viewer` and `operator`
+  roles.
+- **A fail2ban action**, `contrib/fail2ban/action.d/housecarl.conf`
+  (installed under `/usr/share/doc/housecarl/contrib/`): fail2ban's bans
+  go into a ban list, with the jail's port and protocol.
+- **Failed logins in the journal.** A failed login and a refused API
+  token are logged (`msg="login failed" ip=… user=…`), for Guard and
+  fail2ban to act on.
+
 - **Quota usage as a share.** What a quota used shows its percentage
   (`12.3 GB of 50.0 GB used (24%)`, each address's in the quota
   dialog), on the Rules page, the dashboard and in `hc quotas`.
@@ -81,6 +106,33 @@ All notable changes to Housecarl are documented here. The format follows
 
 ### Changed
 
+- The ruleset's diff (Changes, `hc changeset diff`, `--dry-run`) names
+  the chain each part is in on its `@@` line (`@@ -40,6 +40,8 @@ chain
+  forward`), as `git diff` names the function, so a rule added deep in a
+  long chain says whether it is in input, forward or output.
+- **History shows the ruleset's diff.** An entry's page (and `hc history
+  diff ID`, `GET /history/{id}/diff`) shows what the change did to the
+  ruleset, for entries made before this version too: the configuration
+  before and after it is got back from the current one and rendered by
+  this version.
+- Negating a match is said first: the zone, From and To fields (rules
+  and NAT) start with "any of" / "any but" instead of a checkbox under
+  the list, shown once the list has an entry, and an "any but" list is
+  marked with a red bar. An empty list is never saved as negated (it
+  matched everything anyway, but would have negated what was added to
+  it later).
+- docs/rules.md: "Matching everything but a few", on leaving some
+  addresses out with a zone and "any but", and what to do when the range
+  isn't a zone.
+- The rule dialog shows only the zone fields its chain can use: no
+  "Going out on" for input and early, no "Coming in on" for output
+  (they could only be refused). A zone left in a field the chain hides
+  isn't sent.
+- History lists a changed setting by its own fields
+  (`builtins.drop_invalid: false → true`), on the web page and in `hc
+  history show`, instead of the whole `builtins` before and after.
+- Set names starting with `ban_` are now reserved (`set_name_reserved`):
+  ban lists' sets use them.
 - **Rules read as a direction.** A match's zones are written `lan → wan`,
   `any but wan → any`, or with the firewall itself as the other end
   (`lan → firewall` in input, `firewall → wan` in output), instead of
@@ -90,6 +142,10 @@ All notable changes to Housecarl are documented here. The format follows
 
 ### Fixed
 
+- The dashboard's bar for a per-address quota showed only the busiest
+  address, so one address that used up its allowance filled it (100%).
+  The bar is now all the addresses together, and the dashboard and the
+  rule's quota line say how many addresses used theirs up.
 - Changing how a quota counts lost or muddled what it used: per address
   to a total started at nothing, per source to per destination kept the
   sources' use as if it were the destinations', and other groups left

Makefile

@@ -112,6 +112,7 @@ install-files:
 	install -D -m 0644 CHANGELOG.md $(DESTDIR)$(DOCDIR)/CHANGELOG.md
 	install -d $(DESTDIR)$(DOCDIR)/docs
 	install -m 0644 docs/*.md $(DESTDIR)$(DOCDIR)/docs/
+	install -D -m 0644 contrib/fail2ban/action.d/housecarl.conf $(DESTDIR)$(DOCDIR)/contrib/fail2ban/action.d/housecarl.conf
 
 uninstall:
 	rm -f $(DESTDIR)$(BINDIR)/housecarld $(DESTDIR)$(BINDIR)/hc $(DESTDIR)$(UNITDIR)/housecarl.service \

README.md

@@ -53,6 +53,10 @@ table of its own.
   firewall you have, blocking nothing: its hits, log and tracer show what
   it would drop, so you see what switching over would break before you do
   ([docs/shadow.md](docs/shadow.md)).
+* **Ban lists for Guard and fail2ban.** A program on any machine bans an
+  address it caught misbehaving in its logs, and the ban list says what
+  that address is kept from: the firewall, some hosts, or everything
+  ([docs/bans.md](docs/bans.md)).
 * **Easy to watch.** A zone matrix of what is allowed between zones,
   statistics over time (traffic per zone and device, drops, the sources
   blocked most, connections; [docs/stats.md](docs/stats.md)), hits per rule, a live log of what was dropped and
@@ -107,6 +111,7 @@ The web UI is on `http://127.0.0.1:8070/` (see `listen` in
 * [install.md](docs/install.md): packages, building from source, first steps
 * [rules.md](docs/rules.md): zones, address sets, services, rules and the built-ins
 * [nat.md](docs/nat.md): port forwards, hairpin NAT, IPv6 pinholes, source NAT, redirects
+* [bans.md](docs/bans.md): ban lists, where Guard and fail2ban ban addresses
 * [import.md](docs/import.md): moving from an existing nftables ruleset
 * [shadow.md](docs/shadow.md): shadow mode, Housecarl beside your firewall, changing nothing
 * [operations.md](docs/operations.md): probation, the boot and its fallback, hits and the log, monitoring, backups, troubleshooting
@@ -123,14 +128,14 @@ The web UI is on `http://127.0.0.1:8070/` (see `listen` in
 
 ## Status
 
-Housecarl 0.1.1 is in testing (0.1.1-rc.13): zones, address sets and
+Housecarl 0.2.0 is in testing (0.2.0-rc.3+git60-ee27884): zones, address sets and
 services with ordered rules, IPv4 and IPv6, port forwards with hairpin NAT
 and IPv6 pinholes, source NAT and redirects, built-ins, probation for
 network changes (confirmed by the client that made them), the ruleset
 at boot with a fail-closed fallback, hits and the drop log, nftables
 import, changesets reviewed with the ruleset's diff (as in Jarl),
-bandwidth limits and quotas, shadow mode beside another firewall, a web
-UI, REST API and CLI.
+bandwidth limits and quotas, shadow mode beside another firewall, ban
+lists for Guard and fail2ban, a web UI, REST API and CLI.
 
 ## Licence
 

VERSION

@@ -1 +1 @@
-0.1.1-rc.13
+0.2.0-rc.3+git60-ee27884

cmd/hc/admin.go

@@ -66,6 +66,30 @@ func historyCmd() *cobra.Command {
 			})
 		},
 	})
+	cmd.AddCommand(&cobra.Command{
+		Use: "diff ID", Short: "Show what a change did to the ruleset", Args: cobra.ExactArgs(1),
+		Long: `Shows the ruleset's diff for a change: the configuration before and after
+it, rendered by this version of Housecarl.`,
+		RunE: func(cmd *cobra.Command, args []string) error {
+			c, err := newClient()
+			if err != nil {
+				return err
+			}
+			var d struct {
+				Diff string `json:"diff"`
+			}
+			if err := c.Get(ctx(), "/history/"+args[0]+"/diff", nil, &d); err != nil {
+				return err
+			}
+			return output(d, func(w io.Writer) {
+				if d.Diff == "" {
+					fmt.Fprintln(w, "the ruleset didn't change")
+					return
+				}
+				printDiff(w, d.Diff)
+			})
+		},
+	})
 	cmd.AddCommand(&cobra.Command{
 		Use:   "undo ID",
 		Short: "Undo a change",
@@ -88,11 +112,13 @@ change touched has changed since; undo the later changes first.`,
 }
 
 // diffJSON lists the fields that differ between two versions of an
-// object: "name: office -> lab".
+// object: "name: office -> lab". An object within (settings' builtins)
+// is compared field by field: "builtins.drop_invalid: false -> true".
 func diffJSON(before, after json.RawMessage) []string {
 	var a, b map[string]any
 	json.Unmarshal(before, &a)
 	json.Unmarshal(after, &b)
+	a, b = flatten(a, "", map[string]any{}), flatten(b, "", map[string]any{})
 	keys := map[string]bool{}
 	for k := range a {
 		keys[k] = true
@@ -132,6 +158,19 @@ func diffJSON(before, after json.RawMessage) []string {
 	return out
 }
 
+// flatten lifts the fields of objects within m to dotted keys; lists
+// stay whole.
+func flatten(m map[string]any, prefix string, out map[string]any) map[string]any {
+	for k, v := range m {
+		if o, ok := v.(map[string]any); ok && len(o) > 0 {
+			flatten(o, prefix+k+".", out)
+		} else {
+			out[prefix+k] = v
+		}
+	}
+	return out
+}
+
 func whoamiCmd() *cobra.Command {
 	return &cobra.Command{
 		Use: "whoami", Short: "Show who you are to the server", Args: cobra.NoArgs,

cmd/hc/bans.go

@@ -0,0 +1,313 @@
+package main
+
+import (
+	"fmt"
+	"io"
+	"net/url"
+	"strconv"
+	"strings"
+
+	"github.com/spf13/cobra"
+
+	"housecarl/internal/api"
+	"housecarl/internal/model"
+)
+
+// ---- ban lists (docs/bans.md)
+
+// banTime writes a ban list's time setting: 0 is the default, -1 none.
+func banTime(n int, def string) string {
+	switch {
+	case n < 0:
+		return "no limit"
+	case n == 0:
+		return def + " (default)"
+	}
+	return fmtSeconds(n)
+}
+
+// parseBanTime reads a time flag: a duration, or "none" (-1) for no
+// limit.
+func parseBanTime(s string) (int, error) {
+	if strings.EqualFold(s, "none") || s == "-1" {
+		return -1, nil
+	}
+	return parseSeconds(s)
+}
+
+func banlistCmd() *cobra.Command {
+	cmd := &cobra.Command{Use: "banlist", Aliases: []string{"banlists"}, Short: "Ban lists: where Guard and fail2ban ban addresses"}
+	cmd.AddCommand(&cobra.Command{
+		Use: "list", Short: "List the ban lists", Args: cobra.NoArgs,
+		RunE: func(cmd *cobra.Command, args []string) error {
+			bs, err := get[[]model.BanList]("/banlists", nil)
+			if err != nil {
+				return err
+			}
+			bans, _ := get[[]model.Ban]("/bans", nil)
+			count := map[int64]int{}
+			for _, b := range bans {
+				count[b.ListID]++
+			}
+			return output(bs, func(w io.Writer) {
+				fmt.Fprintln(w, "NAME\tSCOPE\tHOSTS\tBANS\tCOMMENT")
+				for _, b := range bs {
+					scope := b.Scope
+					if b.Disabled {
+						scope += " (disabled)"
+					}
+					fmt.Fprintf(w, "%s\t%s\t%s\t%d\t%s\n", b.Name, scope, list(b.Hosts), count[b.ID], b.Comment)
+				}
+			})
+		},
+	})
+	cmd.AddCommand(showCmd("banlists", "banlist", func(w io.Writer, b model.BanList) {
+		fmt.Fprintf(w, "Name:\t%s\nScope:\t%s\n", b.Name, b.Scope)
+		if len(b.Hosts) > 0 {
+			fmt.Fprintf(w, "Hosts:\t%s\n", list(b.Hosts))
+		}
+		fmt.Fprintf(w, "Never:\t%s\n", list(b.Never))
+		fmt.Fprintf(w, "Ban time:\t%s\nLongest:\t%s\n", banTime(b.DefaultTime, "1h"), banTime(b.MaxTime, "7d"))
+		fmt.Fprintf(w, "Most bans:\t%d\nIPv6 banned by:\t/%d\n", b.Entries(), b.V6Group())
+		if b.Disabled {
+			fmt.Fprintln(w, "Disabled:\tyes")
+		}
+		if b.Comment != "" {
+			fmt.Fprintf(w, "Comment:\t%s\n", b.Comment)
+		}
+	}))
+	var scope, comment, banT, maxT string
+	var hosts, never []string
+	var entries, group int
+	var disabled bool
+	setFields := func(fl interface{ Changed(string) bool }, p patch) error {
+		if fl.Changed("scope") {
+			p["scope"] = scope
+		}
+		if fl.Changed("host") {
+			p["hosts"] = hosts
+		}
+		if fl.Changed("never") {
+			p["never"] = never
+		}
+		for _, t := range []struct{ flag, field, v string }{{"time", "default_time", banT}, {"max-time", "max_time", maxT}} {
+			if fl.Changed(t.flag) {
+				n, err := parseBanTime(t.v)
+				if err != nil {
+					return err
+				}
+				p[t.field] = n
+			}
+		}
+		if fl.Changed("max-entries") {
+			p["max_entries"] = entries
+		}
+		if fl.Changed("group-v6") {
+			p["group_v6"] = group
+		}
+		if fl.Changed("comment") {
+			p["comment"] = comment
+		}
+		if fl.Changed("disabled") {
+			p["disabled"] = disabled
+		}
+		return nil
+	}
+	flags := func(c *cobra.Command) {
+		f := c.Flags()
+		f.StringVar(&scope, "scope", "", "what bans keep addresses from: firewall, hosts or everything")
+		f.StringArrayVar(&hosts, "host", nil, "scope hosts: a machine's address, or a set (repeat)")
+		f.StringArrayVar(&never, "never", nil, "an address, prefix or set never banned (repeat)")
+		f.StringVar(&banT, "time", "", "a ban's length when none is asked for (default 1h)")
+		f.StringVar(&maxT, "max-time", "", "the longest ban (default 7d; none for no limit)")
+		f.IntVar(&entries, "max-entries", 0, "the most bans the list holds (default 10000)")
+		f.IntVar(&group, "group-v6", 0, "IPv6 prefix length banned for an address (default 64)")
+		f.StringVar(&comment, "comment", "", "a comment")
+		f.BoolVar(&disabled, "disabled", false, "the list's bans aren't in force")
+	}
+	add := &cobra.Command{
+		Use: "add NAME", Short: "Add a ban list", Args: cobra.ExactArgs(1),
+		Long: `Adds a ban list. A program bans an address into it (Guard, fail2ban's
+housecarl action, hc ban), and the list says what that address is kept
+from:
+
+  hc banlist add router --scope firewall                  # the firewall itself
+  hc banlist add web --scope hosts --host 10.0.10.57 --host 2001:db8:10::57
+  hc banlist add edge --scope everything --never trusted
+
+Give the program a token for its list alone:
+
+  hc user add guard-web --no-password --grant bans:write:web
+  hc token create web1 --user guard-web     # as root on the firewall`,
+		RunE: func(cmd *cobra.Command, args []string) error {
+			p := patch{"name": args[0]}
+			if err := setFields(cmd.Flags(), p); err != nil {
+				return err
+			}
+			return change("POST", "/banlists", p)
+		},
+	}
+	flags(add)
+	cmd.AddCommand(add)
+	update := &cobra.Command{
+		Use: "update BANLIST", Short: "Change a ban list", Args: cobra.ExactArgs(1),
+		Long: `Changes a ban list; --host and --never replace the lists. A ban list
+can't be renamed: tokens and programs know it by its name.`,
+		RunE: func(cmd *cobra.Command, args []string) error {
+			p := patch{}
+			if err := setFields(cmd.Flags(), p); err != nil {
+				return err
+			}
+			return change("PATCH", objPath("banlists", args[0]), p)
+		},
+	}
+	flags(update)
+	cmd.AddCommand(update)
+	cmd.AddCommand(deleteCmd("banlists", "banlist"))
+	return cmd
+}
+
+// bansCmd lists the active bans.
+func bansCmd() *cobra.Command {
+	return &cobra.Command{
+		Use: "bans [BANLIST]", Short: "List the active bans", Args: cobra.MaximumNArgs(1),
+		RunE: func(cmd *cobra.Command, args []string) error {
+			path := "/bans"
+			if len(args) == 1 {
+				path = objPath("banlists", args[0]) + "/bans"
+			}
+			bans, err := get[[]model.Ban](path, nil)
+			if err != nil {
+				return err
+			}
+			return output(bans, func(w io.Writer) {
+				fmt.Fprintln(w, "ID\tLIST\tADDRESS\tPORT\tENDS IN\tBY\tSOURCE\tREASON")
+				for _, b := range bans {
+					port := "all"
+					if b.Protocol != "" {
+						port = fmt.Sprintf("%s/%d", b.Protocol, b.Port)
+					}
+					fmt.Fprintf(w, "%d\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n", b.ID, b.List, b.Address, port, fmtSeconds(b.Timeout), b.Who, orDash(b.Source), orDash(clipText(b.Reason, 60)))
+				}
+			})
+		},
+	}
+}
+
+func clipText(s string, n int) string {
+	if len(s) > n {
+		return s[:n-1] + "…"
+	}
+	return s
+}
+
+// banCmd bans an address by hand.
+func banCmd() *cobra.Command {
+	var tcp, udp []string
+	var t, reason string
+	cmd := &cobra.Command{
+		Use: "ban BANLIST ADDRESS", Short: "Ban an address", Args: cobra.ExactArgs(2),
+		Long: `Bans an address in a ban list: every port, or with --tcp and --udp some
+(numbers or service names). IPv6 addresses are banned with the list's
+grouping (a /64 unless set otherwise).
+
+  hc ban edge 203.0.113.9 --time 1d
+  hc ban web 203.0.113.9 --tcp ssh --tcp 8443 --reason "tried passwords"`,
+		RunE: func(cmd *cobra.Command, args []string) error {
+			in := api.BanInput{Address: args[1], Reason: reason, Source: "hc"}
+			if t != "" {
+				n, err := parseBanTime(t)
+				if err != nil {
+					return err
+				}
+				in.Time = n
+			}
+			c, err := newClient()
+			if err != nil {
+				return err
+			}
+			var out api.BanResult
+			do := func(proto string, ports []string) error {
+				in.Protocol = proto
+				in.Ports = nil
+				if len(ports) > 0 {
+					in.Ports = []byte(strconv.Quote(strings.Join(ports, ",")))
+				}
+				var res api.BanResult
+				if err := c.Do(ctx(), "POST", objPath("banlists", args[0])+"/bans", nil, in, &res); err != nil {
+					return err
+				}
+				out.Bans = append(out.Bans, res.Bans...)
+				out.Closed += res.Closed
+				return nil
+			}
+			switch {
+			case len(tcp) == 0 && len(udp) == 0:
+				err = do("", nil)
+			default:
+				if len(tcp) > 0 {
+					err = do("tcp", tcp)
+				}
+				if err == nil && len(udp) > 0 {
+					err = do("udp", udp)
+				}
+			}
+			if err != nil {
+				return err
+			}
+			return output(out, func(w io.Writer) {
+				for _, b := range out.Bans {
+					what := "banned"
+					if b.Existing {
+						what = "renewed"
+					}
+					port := "every port"
+					if b.Protocol != "" {
+						port = fmt.Sprintf("%s/%d", b.Protocol, b.Port)
+					}
+					ends := "never ends"
+					if b.Timeout > 0 {
+						ends = "ends in " + fmtSeconds(b.Timeout)
+					}
+					fmt.Fprintf(w, "%s %s, %s, in %s (%s)\n", what, b.Address, port, b.List, ends)
+				}
+				if out.Closed > 0 {
+					fmt.Fprintf(w, "%d connections closed\n", out.Closed)
+				}
+			})
+		},
+	}
+	f := cmd.Flags()
+	f.StringArrayVar(&tcp, "tcp", nil, "ban only this TCP port or service (repeat)")
+	f.StringArrayVar(&udp, "udp", nil, "ban only this UDP port or service (repeat)")
+	f.StringVar(&t, "time", "", "how long (1h, 7d; default the list's; none for ever, if the list allows)")
+	f.StringVar(&reason, "reason", "", "why, for the record")
+	return cmd
+}
+
+// unbanCmd lifts bans: by ID, or every ban of an address.
+func unbanCmd() *cobra.Command {
+	return &cobra.Command{
+		Use: "unban BANLIST ADDRESS|ID", Short: "Lift a ban, or all of an address's", Args: cobra.ExactArgs(2),
+		RunE: func(cmd *cobra.Command, args []string) error {
+			c, err := newClient()
+			if err != nil {
+				return err
+			}
+			path, query := objPath("banlists", args[0])+"/bans", url.Values{}
+			if _, err := strconv.ParseInt(args[1], 10, 64); err == nil {
+				path += "/" + args[1]
+			} else {
+				query.Set("address", args[1])
+			}
+			var out map[string]int
+			if err := c.Do(ctx(), "DELETE", path, query, nil, &out); err != nil {
+				return err
+			}
+			if !flagQuiet {
+				fmt.Printf("%d bans lifted\n", out["removed"])
+			}
+			return nil
+		},
+	}
+}

cmd/hc/main.go

@@ -192,7 +192,7 @@ Exit codes: 0 ok, 1 error, 3 not authenticated or not allowed, 4 not found,
 			root.AddCommand(c)
 		}
 	}
-	add("fw", statusCmd(), zoneCmd(), setCmd(), serviceCmd(), ruleCmd(), forwardCmd(), snatCmd(), redirectCmd(), settingsCmd(),
+	add("fw", statusCmd(), zoneCmd(), setCmd(), serviceCmd(), ruleCmd(), forwardCmd(), snatCmd(), redirectCmd(), banlistCmd(), bansCmd(), banCmd(), unbanCmd(), settingsCmd(),
 		rulesetCmd(), countersCmd(), quotasCmd(), logCmd(), connectionsCmd(), traceCmd(), matrixCmd(), statsCmd(), interfaceCmd())
 	add("changes", changesetCmd(), confirmCmd(), rollbackCmd(), historyCmd(), importCmd(), takeoverCmd())
 	add("admin", whoamiCmd(), userCmd(), tokenCmd(), passwordCmd(), certCmd(), auditCmd(), configCmd())

cmd/housecarld/main.go

@@ -156,7 +156,8 @@ func serveCmd() *cobra.Command {
 			host := &hostif.Host{}
 			eng.Env = hostEnv(host)
 			fw := &nft.Firewall{Runner: &nft.Runner{Command: nft.Wrapper()}, Interfaces: host.Exists, BootFile: a.cfg.Ruleset,
-				Shadow: a.cfg.Shadow.Enabled, ShadowBefore: a.cfg.Shadow.Position == config.ShadowBefore}
+				Shadow: a.cfg.Shadow.Enabled, ShadowBefore: a.cfg.Shadow.Position == config.ShadowBefore,
+				Bans: func() []render.Ban { return server.ActiveBans(context.Background(), a.store) }}
 			eng.Firewall = fw
 			if fw.Shadow {
 				a.log.Warn("shadow mode: Housecarl's ruleset is loaded but blocks, rejects and translates nothing; the other firewall decides (docs/shadow.md)", "position", a.cfg.Shadow.Position)

contrib/fail2ban/action.d/housecarl.conf

@@ -0,0 +1,52 @@
+# Housecarl's action for fail2ban: a ban goes into a Housecarl ban list,
+# and the list says what the banned address is kept from (the firewall
+# itself, some hosts behind it, or everything). See docs/bans.md.
+#
+# Copy this file to /etc/fail2ban/action.d/housecarl.conf, make a ban
+# list and a token for it alone on the firewall:
+#
+#   hc banlist add web --scope hosts --host 10.0.10.57
+#   hc user add f2b-web --no-password --grant bans:write:web
+#   hc token create web --user f2b-web        # as root on the firewall
+#
+# put the token in /etc/fail2ban/housecarl.token (mode 0600), and use it
+# in a jail:
+#
+#   [sshd]
+#   enabled = true
+#   action  = housecarl[list=web, url="https://router.lan:8070"]
+#
+# The jail's port and protocol go along: port = ssh bans only tcp/22;
+# fail2ban's default (0:65535) bans every port. IPv6 addresses are banned
+# with the list's grouping (a /64 unless set otherwise). Housecarl never
+# bans inside networks, its own addresses or the list's "never".
+#
+# On the firewall itself, the local socket works too, still with the
+# token: url = http://localhost, curlopts = --unix-socket
+# /run/housecarl/housecarl.sock.
+
+[Definition]
+
+actionstart =
+actionstop =
+actioncheck =
+
+# A ban renews one already there; a bantime of -1 is the list's longest.
+actionban = <_curl> -f -X POST -H "Content-Type: application/json" "<url>/api/v1/banlists/<list>/bans"
+            -d '{"address": "<ip>", "protocol": "<protocol>", "ports": "<port>", "time": <bantime>, "source": "fail2ban/<name>"}'
+
+# A ban Housecarl already ended (its own time ran out) is no error.
+actionunban = <_curl> -X DELETE "<url>/api/v1/banlists/<list>/bans?address=<ip>"
+
+[Init]
+
+# The ban list, and Housecarl's URL (its listen address).
+list =
+url = https://127.0.0.1:8070
+# The API token, in a file only root reads.
+tokenfile = /etc/fail2ban/housecarl.token
+# More curl options: --cacert /etc/fail2ban/housecarl-ca.pem for a
+# private CA, --unix-socket for the local socket.
+curlopts =
+
+_curl = curl -sS --max-time 10 <curlopts> -o /dev/null -H "Authorization: Bearer $(cat <tokenfile>)"

docs/api.md

@@ -57,7 +57,9 @@ Calls that change the configuration answer:
   one). Missing when a person must confirm (`manual_confirm`).
 * `?dry_run=1` checks the change, with nftables too, and returns what it
   would do, with `"applied": false, "dry_run": true`, and `diff`: the
-  ruleset's unified diff.
+  ruleset's unified diff, each `@@` line followed by the chain (or other
+  block of the table) the hunk starts in, as in `@@ -40,6 +40,8 @@ chain
+  forward`.
 * `?changeset=ID` (or the header `X-Housecarl-Changeset: ID`) stages the
   change in that changeset instead ([Changesets](#changesets)): the answer
   has `"staged": true`, `changes` (this operation's) and `changeset`.
@@ -108,15 +110,15 @@ Lists replace the old ones. `probation_seconds` 0 is the default (120).
 
 ### Objects
 
-Zones, sets, services, rules, port forwards, source NAT and redirects
-share one pattern, `/{kind}` with kind `zones`, `sets`, `services`,
-`rules`, `forwards`, `snats` or `redirects`:
+Zones, sets, services, rules, port forwards, source NAT, redirects and
+ban lists share one pattern, `/{kind}` with kind `zones`, `sets`,
+`services`, `rules`, `forwards`, `snats`, `redirects` or `banlists`:
 
 | | |
 |---|---|
 | `GET /{kind}` | all of them, in order; `GET /rules?chain=input` for one chain; `GET /services` includes the built-in ones |
 | `POST /{kind}[?before=ID\|after=ID\|first=1]` | create; rules, forwards and NAT rules go last unless given a `position`, or placed before or after another of their chain, or first, in the same change (one history entry; staged in a changeset, the place goes with it) |
-| `GET /{kind}/{ref}` | by ID, or name for zones, sets, services and forwards |
+| `GET /{kind}/{ref}` | by ID, or name for zones, sets, services, forwards and ban lists |
 | `PATCH /{kind}/{ref}` | change the fields given (a JSON merge: `null` clears) |
 | `DELETE /{kind}/{ref}` | refused while something uses it |
 | `POST /rules/{ref}/reset-quota` | start a rule's quota again now (`config:write`; audited, not staged). Optional body: `{"address": "192.168.1.23"}` starts only that address's again (a per-address quota; an IPv6 address counts for its /64; 404 if it hasn't used the quota); `{"give": 5368709120}` takes that many bytes off what was used instead (the address's, or every address's), until the next reset |
@@ -145,7 +147,18 @@ The objects:
 
 A source NAT rule is a match (as a rule's, without the rule's other
 fields) with `"to"` (empty: masquerade), a redirect a match with
-`"to_port"`. Times are seconds (`timeout`).
+`"to_port"`. A ban list ([bans.md](bans.md)):
+
+```json
+{"name": "web", "scope": "hosts", "hosts": ["10.0.10.57"], "never": ["trusted"],
+ "default_time": 3600, "max_time": 604800, "max_entries": 10000, "group_v6": 64, "comment": "", "disabled": false}
+```
+
+`scope` is `firewall`, `hosts` or `everything`; 0 is the default for the
+numbers, and `max_time` -1 is no limit. A ban list can't be renamed.
+Someone with a `bans` grant but no `config:read` may read the ban lists
+the grant covers (`GET /banlists`, `GET /banlists/{ref}`). Times are
+seconds (`timeout`).
 
 ### Statistics
 
@@ -196,6 +209,22 @@ These need `sets:read` or `sets:write`, for that set
 ([permissions.md](permissions.md)); they aren't on probation or in the
 history, but are audited.
 
+### Bans
+
+| | |
+|---|---|
+| `GET /bans` | the active bans of every ban list you may read: `id`, `list_id`, `list`, `address` (an address, or an IPv6 prefix), `protocol` and `port` (none: every port), `expires` and `timeout` (seconds left; none: no end), `created`, `who`, `source`, `reason` |
+| `GET /banlists/{ref}/bans` | one list's |
+| `POST /banlists/{ref}/bans` | ban: `{"address": "203.0.113.9", "protocol": "tcp", "ports": "ssh", "time": 3600, "source": "guard/ssh", "reason": "…"}`. `ports` is a number, a string (`"22"`, `"http,https"`, `"0:65535"` or `"all"` for every port) or a list; without a protocol a number is TCP and UDP, a service its own protocols. `time` 0 is the list's default, -1 its longest; longer is cut down. 201 with `{"bans": [ban…, "existing": true for a renewed one], "closed": N}` (connections closed) |
+| `DELETE /banlists/{ref}/bans/{id}` | lift one ban |
+| `DELETE /banlists/{ref}/bans?address=A` | lift all of an address's bans (404 if it has none) |
+
+These need `bans:read` or `bans:write` for the list
+([permissions.md](permissions.md)); bans aren't on probation or in the
+history, but are audited (`ban.add`, `ban.remove`). Refusals: `422
+never_ban` (inside, the firewall's own, the caller's own, or in the
+list's `never`), `422 list_full`, `409` for a disabled list.
+
 ### Probation
 
 | | |
@@ -210,6 +239,7 @@ history, but are audited.
 |---|---|
 | `GET /history[?limit=&before=]` | changes, newest first |
 | `GET /history/{id}` | one, with each object before and after |
+| `GET /history/{id}/diff` | `{"diff"}`: what it did to the ruleset, as a unified diff with each hunk's chain (the configuration before and after it, got back from the current one, rendered by this version) |
 | `POST /history/{id}/revert` | undo it; 409 if an object it touched has changed since, or it was undone already |
 | `POST /import/nft` | `{"json": NFT_JSON, "text": "nft -a list ruleset output", "defines": "ruleset file", "table": "inet main", "zones": {"enp2s0": "lan"}, "replace": false}` ([import.md](import.md)); answers with `state` (the imported configuration), `notes` and `raw` too |
 | `POST /takeover` | `{"tables": ["inet main"]}`: delete these tables (administrators); on probation over the network; 409 in shadow mode |

docs/bans.md

@@ -0,0 +1,148 @@
+# Ban lists
+
+A **ban list** is where a program bans addresses: Guard (Housecarl's
+companion, which reads logs), fail2ban with Housecarl's action, or you
+with `hc ban`. The program says only *which address*, and perhaps a
+protocol and ports. The ban list says what that address is kept from.
+So a web server that sees password guessing in its own logs can ban the
+guesser at the router without knowing anything about the router.
+
+## Ban lists
+
+```sh
+hc banlist add router --scope firewall                   # the firewall itself
+hc banlist add web --scope hosts --host 10.0.10.57 --host 2001:db8:10::57
+hc banlist add edge --scope everything --never trusted
+```
+
+| Scope | Banned addresses are kept from | Where the drops are |
+|---|---|---|
+| `firewall` | the firewall itself: its SSH, DNS, web UI | `input` |
+| `hosts` | the list's hosts (addresses or sets), behind the firewall | `forward`, to those addresses |
+| `everything` | the firewall and everything behind it | `input` and `forward` |
+
+A `hosts` list covers port forwards to its hosts as well: the forward
+chain sees a forwarded connection's translated address and port, the
+host's own. A web server's Guard bans `tcp/22`, and that is right even
+when the router forwards its port 2222 to it.
+
+Other settings, with their defaults:
+
+| Setting | Default | |
+|---|---|---|
+| `never` | none | addresses, prefixes or sets this list never bans |
+| `default_time` (`--time`) | 1h | a ban's length when the program asks for none |
+| `max_time` (`--max-time`) | 7d | the longest ban; longer ones are cut to it. `-1` (`none`): no limit, so bans may be permanent |
+| `max_entries` | 10000 | the most bans the list holds; more are refused (`list_full`) |
+| `group_v6` | 64 | the IPv6 prefix length banned for an address. One customer usually has a whole /64 and can move around in it; 128 bans the one address. IPv4 addresses are banned alone |
+| `disabled` | no | the list's bans aren't in force (they are kept) |
+
+A ban list can't be renamed: the tokens and programs that ban into it
+know it by its name. Make a new one instead.
+
+## Bans
+
+```sh
+hc ban edge 203.0.113.9 --time 1d --reason "port scan"
+hc ban web 203.0.113.9 --tcp ssh --tcp 8443
+hc bans                       # every list's
+hc bans web
+hc unban web 203.0.113.9      # all of the address's bans in web
+hc unban web 17               # one, by ID
+```
+
+A ban is of **every port**, or of **a protocol's ports** (numbers or
+service names, built-in or yours). A port without a protocol is banned
+for TCP and UDP; a service name takes the service's protocols. Banning an
+address again renews the ban: its new time counts from now.
+
+Bans are dropped **ahead of every rule** in their chains, and ahead of
+the built-in that lets established connections through, so a ban cuts a
+connection that is already open. Housecarl also closes the banned
+address's tracked connections the ban covers when it bans it, because
+connections offloaded to the flowtable no longer pass the rules at all.
+Replies to connections the inside opened to a banned address are
+dropped too.
+
+Bans are kept in the database, apart from the configuration (like a
+dynamic set's entries): they aren't in the history, can't be undone,
+aren't on probation, and don't go through changesets. They end by
+themselves; every change, restart and reboot puts the active ones back
+with the time they have left. Each ban and unban is in the audit log
+(`ban.add`, `ban.remove`), with the program's source and reason.
+
+### What is never banned
+
+Whatever a list's `never` says, Housecarl refuses (`422 never_ban`) to
+ban:
+
+* the inside: addresses in networks of inside zones' interfaces, and
+  private addresses outside the external zones' networks;
+* the firewall's own addresses, and its own prefixes (settings);
+* the address of whoever asks (a program can't lock itself out);
+* loopback, link-local, multicast and unspecified addresses.
+
+A ban grouped to a prefix (IPv6) is refused when the prefix overlaps any
+of these.
+
+## Programs and tokens
+
+Give each program a token for its list alone. A `bans` grant can name
+ban lists, as a `sets` grant names sets:
+
+```sh
+hc user add guard-web --no-password --grant bans:write:web
+hc token create web --user guard-web          # as root on the firewall
+```
+
+With it, the program may ban into `web` and lift its bans, list them,
+and read the list itself (`GET /banlists/web`: its grouping and times),
+and nothing else. `viewer` has `bans:read`, `operator` `bans:write`.
+
+## Guard
+
+Guard reads log files and the systemd journal on any machine, and bans
+the addresses that misbehave into a ban list (its own documentation).
+Run it on the web server with a token for `web`, and on the router with
+one for `router` (or the local socket).
+
+## fail2ban
+
+Housecarl ships an action for fail2ban,
+`/usr/share/doc/housecarl/contrib/fail2ban/action.d/housecarl.conf`
+(`contrib/` in the source). Copy it to `/etc/fail2ban/action.d/`, put
+the token in `/etc/fail2ban/housecarl.token` (mode 0600), and use it in a
+jail:
+
+```ini
+[sshd]
+enabled = true
+action  = housecarl[list=web, url="https://router.lan:8070"]
+```
+
+The jail's `port` and `protocol` go with each ban: `port = ssh` bans only
+`tcp/22`, fail2ban's default `0:65535` bans every port. A `bantime` of
+`-1` is the list's longest ban. When fail2ban unbans an address Housecarl
+already let go (its own time ran out first), nothing happens. Add
+`curlopts = --cacert /etc/fail2ban/housecarl-ca.pem` for a private CA.
+
+## Shadow mode
+
+In shadow mode the ban rules count what they would drop
+(`[would drop]`) and let it through, like every other rule, and banning
+closes no connections.
+
+## The ruleset
+
+Each list has four sets, `ban_<id>_v4`, `ban_<id>_v6` (every port) and
+`ban_<id>_v4_svc`, `ban_<id>_v6_svc` (address, protocol and port), all
+with timeouts:
+
+```
+ip saddr @ban_3_v4 counter drop comment "hc:ban3"
+ip saddr . meta l4proto . th dport @ban_3_v4_svc counter drop comment "hc:ban3"
+ip6 saddr and ffff:ffff:ffff:ffff:: @ban_3_v6 counter drop comment "hc:ban3"
+```
+
+Set names starting with `ban_` are therefore Housecarl's own
+(`set_name_reserved`).

docs/cli.md

@@ -60,7 +60,8 @@ applied and saved (history 12; undo with 'hc history undo 12')
 ```
 
 `-n` shows what would change without changing it, with the diff of the
-ruleset; nftables checks it too.
+ruleset (each `@@` line names the chain the lines below it are in);
+nftables checks it too.
 
 ## Changesets
 
@@ -204,6 +205,19 @@ accept|drop|reject|log`, `--reject-with tcp-reset`, `--log`,
 
 See [nat.md](nat.md).
 
+### banlist, ban
+
+| Command | |
+|---|---|
+| `banlist list`, `banlist show BANLIST` | ban lists, with their bans' count |
+| `banlist add NAME --scope firewall\|hosts\|everything [--host ADDR\|SET]... [--never ADDR\|SET]... [--time 1h] [--max-time 7d\|none] [--max-entries N] [--group-v6 64] [--disabled] [--comment C]` | |
+| `banlist update BANLIST [flags]`, `banlist delete BANLIST` | `--host` and `--never` replace the lists; no renaming |
+| `bans [BANLIST]` | the active bans |
+| `ban BANLIST ADDRESS [--tcp PORT\|SERVICE]... [--udp …]... [--time 1h\|none] [--reason R]` | ban every port, or some; again renews |
+| `unban BANLIST ADDRESS\|ID` | lift one ban, or all of an address's |
+
+See [bans.md](bans.md).
+
 ### settings
 
 | Command | |
@@ -218,7 +232,8 @@ See [nat.md](nat.md).
 | `confirm` | keep the changes on probation; the ruleset is saved for the boot |
 | `rollback` | undo the changes on probation now |
 | `history list [--limit N]` | changes, newest first |
-| `history show ID` | each object before and after |
+| `history show ID` | each object before and after (a changed setting by its own fields: `builtins.drop_invalid: false -> true`) |
+| `history diff ID` | what the change did to the ruleset |
 | `history undo ID` | apply the change backwards (refused if something it touched has changed since) |
 | `import [FILE\|-] [--text FILE] [--defines FILE] [--table T] [--zone IFACE=NAME]... [--replace]` | [import.md](import.md) |
 | `takeover TABLE... [-y]` | delete tables Housecarl's ruleset replaces (administrators); not in shadow mode |

docs/development.md

@@ -135,6 +135,14 @@ As for Jarl and Steward:
 3. Run `make test && make e2e`.
 4. Commit as "Release X", tag it `git tag -a vX -m "Housecarl X"`, and push.
 
+A fix to a release candidate may go out as a build of it rather than a
+new candidate: `0.2.0-rc.3+git<count>-<commit>`, where `<commit>` is the
+short ID of the last commit before "Release X" and `<count>` is `git
+rev-list --count` of it. The count orders the builds (apt compares it as a
+number; the ID alone would order them at random); each sorts after
+`0.2.0-rc.3` and before `0.2.0-rc.4`. Debian writes it
+`0.2.0~rc.3+git<count>~<commit>`.
+
 The tag starts `.gitea/workflows/release.yml`, which checks the tag
 against `VERSION` and the CHANGELOG, runs the tests, builds the tarballs,
 the CLI for each OS and the `.deb`s, and publishes them

docs/operations.md

@@ -309,6 +309,22 @@ down, the fallback and a missing ruleset. Reach it from where the input
 rules let the monitor in (a rule for the Checkmk server's address and
 port 8070).
 
+## Failed logins in the journal
+
+A failed login to the web UI or the API, and a refused API token, are
+written to the journal as well as the audit log, with the address first:
+
+```
+level=WARN msg="login failed" ip=203.0.113.9 user=admin reason="wrong username or password"
+level=WARN msg="token refused" ip=203.0.113.9 reason="wrong token secret"
+```
+
+Guard's `housecarl` preset, or a fail2ban filter, can ban such addresses
+([bans.md](bans.md)). The address comes before the user name, which a
+caller chooses, so a pattern anchored on `msg="login failed" ip=` can't
+be fooled by a name. (With `log.format: json` they are JSON objects with
+the same fields, in the same order.)
+
 ## Checks
 
 `housecarld check` validates the configuration in the database against

docs/permissions.md

@@ -9,6 +9,7 @@ else gets **grants**, `resource:access`:
 |---|---|---|
 | `config` | settings, zones, sets, services, rules, NAT, the ruleset, history, interfaces, probation, changesets (the web UI shows the forms locked) | change them, stage and apply changesets (your own, or anyone's for administrators), undo, import, confirm and roll back; close connections; start and stop traces |
 | `sets` | dynamic sets' entries | add and remove entries |
+| `bans` | ban lists' bans, and the lists the grant names | ban and unban ([bans.md](bans.md)) |
 | `log` | hit counters, the log, connections, traces | – |
 | `audit` | the audit log | – |
 | `*` | all of the above | |
@@ -20,6 +21,10 @@ program, such as Steward's captive portal, which should be able to let a
 client in and nothing more. A dynamic set's entries need a `sets` grant
 even for those with `config:write`.
 
+A `bans` grant names ban lists the same way: `bans:write:web` may ban
+into the list `web` (and read it) and nothing else, the grant for Guard
+or fail2ban on the web server.
+
 Only administrators may delete other tables (`hc takeover`), and manage
 users and the certificate (`hc cert`, Server → Certificate).
 
@@ -27,8 +32,8 @@ users and the certificate (`hc cert`, Server → Certificate).
 
 | Role | Grants |
 |---|---|
-| `viewer` | `config:read sets:read log:read` |
-| `operator` | `config:write sets:write log:read audit:read` |
+| `viewer` | `config:read sets:read bans:read log:read` |
+| `operator` | `config:write sets:write bans:write log:read audit:read` |
 | `log` | `log:read`, e.g. for a monitoring script |
 
 ```sh
@@ -49,9 +54,13 @@ hc token create portal --user portal --grant sets:write:guests --expires 365d
 ```
 
 The secret is shown once; only its hash is stored. A restricted token can
-only create tokens with a subset of its own grants. The web UI only offers
-roles within what your account may do; a token with more would get no
-more than its user anyway.
+only create tokens with a subset of its own grants.
+
+In the web UI, Account → New token makes one of your own; an
+administrator makes one for another user with New token on that user's
+row under Users (`--user` in the CLI, `POST /users/{id}/tokens` in the
+API). The dialog only offers roles within what the token's user may do; a
+token with more would get no more than its user anyway.
 
 **Rotating** gives a token a new secret, keeping its ID, name and
 permissions, so only the programs that use it need updating:
@@ -82,6 +91,6 @@ on probation like any other.
 
 Logins (and failed ones), users, tokens, every configuration change (with
 its history entry), changesets created, applied and discarded,
-confirmations and rollbacks, imports, takeovers and
-dynamic set entries are recorded with who, from where and how: `session`,
+confirmations and rollbacks, imports, takeovers,
+dynamic set entries and bans are recorded with who, from where and how: `session`,
 `token` or `socket`.

docs/rules.md

@@ -165,6 +165,40 @@ hc rule add forward --in wan --status dnat
 hc rule add forward --in wan -a drop
 ```
 
+### Matching everything but a few
+
+A field matches **any of** its entries, or with `--not-…` ("any but" in
+the web UI) anything else; it can't do both. "The LAN except the NAS" is
+still one rule: the zone says which network, and the addresses say which
+of it to leave out, because zones and addresses are different fields and
+a rule needs all of them to match.
+
+| What you want | One rule |
+|---|---|
+| the LAN, except the NAS | `--in lan --from nas --not-from` |
+| everyone on guest, except one device, has a quota | `--in guest --from tv --not-from --quota 10gbytes,over,daily` |
+| the internet, except one host | `--out wan --to blocked --not-to` |
+
+```sh
+# Each device on the LAN: 50 GB a month, but not the NAS (its backups)
+hc rule add forward --in lan --out wan --from nas --not-from --quota 50gbytes,over,per-source,monthly -a drop
+```
+
+Leaving part of an address range out ("10.0.0.0/8 except 10.0.5.0/24")
+when it isn't a zone takes one of:
+
+* **Two rules, the exception first**, when the exception gets the
+  opposite action: `--from 10.0.5.0/24 -a drop` above `--from
+  10.0.0.0/8`. There is no action that skips a rule and goes on, so this
+  doesn't work when the exception should just fall through to later
+  rules.
+* **Ranges around the hole**: `--from 10.0.0.0-10.0.4.255 --from
+  10.0.6.0-10.255.255.255`.
+* **A [raw rule](#raw-rules)**: `ip saddr 10.0.0.0/8 ip saddr != 10.0.5.0/24 accept`.
+
+There's no need to leave a zone out of other zones: an interface belongs
+to one zone, so "any of lan, guest" already leaves out everything else.
+
 ### Bandwidth and quotas
 
 A **byte rate** limits bandwidth: `--limit 20mbytes/second,over,per-source
@@ -268,6 +302,13 @@ In shadow mode ([shadow.md](shadow.md)) a raw rule's final `drop` or
 (NAT, `meta mark set`, `jump`, `notrack`, a verdict map…) is left out,
 with a warning (`shadow_raw_left_out`).
 
+## Ban lists
+
+Programs such as Guard and fail2ban ban addresses into ban lists, and
+those bans come first of all in the input and forward chains (after
+loopback in input), before the built-ins and every rule. See
+[bans.md](bans.md).
+
 ## Built-ins
 
 Housecarl adds the rules every firewall needs before yours. Each can be

docs/shadow.md

@@ -116,6 +116,9 @@ firewall's NAT still does the work.
 * **No takeover, no closing connections.** `hc takeover` (deleting your
   firewall's tables) and `hc connections close` are refused (409,
   `hc` exits with 5): both would change your traffic.
+* **Bans** ([bans.md](bans.md)) are kept and their rules count what they
+  would drop (`[would drop]`), but nothing is dropped, and banning closes
+  no connections.
 * **The boot.** `housecarl-ruleset.service` loads the shadow ruleset at
   boot. Its fallback ([operations.md](operations.md#if-the-ruleset-fails-to-load-at-boot))
   is an empty table that filters nothing, since your firewall protects the

docs/validation.md

@@ -51,7 +51,7 @@ are unique per kind, ignoring case.
 
 | Code | | When |
 |---|---|---|
-| `set_name_reserved` | error | a set name starting with `meter_`, `conn_`, `quota_` or `stats_`: Housecarl's own sets (rate limits, quotas, statistics) use those |
+| `set_name_reserved` | error | a set name starting with `meter_`, `conn_`, `quota_`, `stats_` or `ban_`: Housecarl's own sets (rate limits, quotas, statistics, ban lists) use those |
 | `set_entry` | error | an entry that isn't an address, prefix (without host bits) or range |
 | `set_empty` | warning | a set (not dynamic) with no entries: rules using it match nothing |
 | `set_dynamic_entries` | error | configured entries on a dynamic set: its entries come at run time |
@@ -138,3 +138,17 @@ configuration: the dashboard and `hc status` show them.
 
 In shadow mode `nftables_service` and `takeover_pending` aren't raised:
 another firewall beside Housecarl is the point there.
+
+## Ban lists
+
+| Code | | When |
+|---|---|---|
+| `banlist_name` | error | not a valid name (as for zones and sets) |
+| `banlist_name_taken` | error | two ban lists with one name |
+| `banlist_scope` | error | a scope other than `firewall`, `hosts` or `everything` |
+| `banlist_hosts` | error | scope `hosts` without hosts, or hosts with another scope |
+| `banlist_address` | error | a host or never entry that isn't an address, prefix or range |
+| `unknown_set` | error | a host or never entry naming a set that doesn't exist |
+| `banlist_time` | error | negative times (but `max_time` -1), or a default longer than the longest |
+| `banlist_entries` | error | `max_entries` below 0 or above 1000000 |
+| `banlist_group` | error | `group_v6` outside 16 to 128 |

internal/api/types.go

@@ -532,3 +532,35 @@ type MatrixException struct {
 	// Marks are the counters behind Hits ("r12", "b-ping", "f3").
 	Marks []string `json:"-"`
 }
+
+// BanInput asks a ban list to ban an address (POST
+// /banlists/{list}/bans).
+type BanInput struct {
+	Address string `json:"address"`
+	// Protocol (tcp, udp) and Ports limit the ban; with neither, every
+	// port. Ports are numbers or service names, in a list or a string
+	// ("22", "ssh", "http,https"); "all", "0:65535" or "1-65535" mean
+	// every port. Ports without a protocol take the service's protocols,
+	// or for a number tcp and udp.
+	Protocol string          `json:"protocol,omitempty"`
+	Ports    json.RawMessage `json:"ports,omitempty"`
+	// Time is the ban's length in seconds: 0 the list's default, -1 its
+	// longest (or for ever). Longer than the list allows is cut down.
+	Time   int    `json:"time,omitempty"`
+	Source string `json:"source,omitempty"`
+	Reason string `json:"reason,omitempty"`
+}
+
+// BanResult is the answer to a ban: the bans made or renewed (one per
+// protocol and port), Existing for each already in force.
+type BanResult struct {
+	Bans []BanMade `json:"bans"`
+	// Closed counts the banned address's connections closed.
+	Closed int `json:"closed"`
+}
+
+// BanMade is a ban made, or renewed (Existing).
+type BanMade struct {
+	model.Ban
+	Existing bool `json:"existing,omitempty"`
+}

internal/auth/auth.go

@@ -147,13 +147,14 @@ func NewSessionID() (id, csrf string) {
 const (
 	ResConfig = "config" // the firewall configuration, its history and imports
 	ResSets   = "sets"   // dynamic sets' entries (a grant may name the sets)
+	ResBans   = "bans"   // ban lists' bans (a grant may name the lists)
 	ResLog    = "log"    // counters, the drop log and the status
 	ResAudit  = "audit"  // the audit log
 	ResAll    = "*"
 )
 
 // Resources lists them in display order.
-var Resources = []string{ResConfig, ResSets, ResLog, ResAudit}
+var Resources = []string{ResConfig, ResSets, ResBans, ResLog, ResAudit}
 
 // Access levels.
 const (
@@ -163,8 +164,8 @@ const (
 
 // Roles are ready-made sets of grants.
 var Roles = map[string][]model.Grant{
-	"viewer":   {{Resource: ResConfig, Access: Read}, {Resource: ResSets, Access: Read}, {Resource: ResLog, Access: Read}},
-	"operator": {{Resource: ResConfig, Access: Write}, {Resource: ResSets, Access: Write}, {Resource: ResLog, Access: Read}, {Resource: ResAudit, Access: Read}},
+	"viewer":   {{Resource: ResConfig, Access: Read}, {Resource: ResSets, Access: Read}, {Resource: ResBans, Access: Read}, {Resource: ResLog, Access: Read}},
+	"operator": {{Resource: ResConfig, Access: Write}, {Resource: ResSets, Access: Write}, {Resource: ResBans, Access: Write}, {Resource: ResLog, Access: Read}, {Resource: ResAudit, Access: Read}},
 	"log":      {{Resource: ResLog, Access: Read}},
 }
 
@@ -179,12 +180,13 @@ func RoleNames() []string {
 }
 
 // ParseGrant reads "resource:access" ("log:read", "*:write"), and for
-// sets optionally the sets it is limited to ("sets:write:guest,portal").
+// sets and bans optionally the sets or ban lists it is limited to
+// ("sets:write:guest,portal", "bans:write:web").
 func ParseGrant(s string) (model.Grant, error) {
 	parts := strings.SplitN(strings.TrimSpace(s), ":", 3)
 	g := model.Grant{Resource: strings.ToLower(parts[0])}
 	if len(parts) < 2 {
-		return g, fmt.Errorf("grant %q: write it as resource:access, e.g. log:read, or sets:write:SET1,SET2", s)
+		return g, fmt.Errorf("grant %q: write it as resource:access, e.g. log:read, sets:write:SET1,SET2 or bans:write:LIST", s)
 	}
 	g.Access = strings.ToLower(parts[1])
 	if len(parts) == 3 {
@@ -211,8 +213,8 @@ func ValidateGrant(g *model.Grant) error {
 	if g.Access != Read && g.Access != Write {
 		return fmt.Errorf("access must be read or write, not %q", g.Access)
 	}
-	if len(g.Sets) > 0 && g.Resource != ResSets {
-		return fmt.Errorf("only a sets grant can name sets (sets:write:%s)", strings.Join(g.Sets, ","))
+	if len(g.Sets) > 0 && g.Resource != ResSets && g.Resource != ResBans {
+		return fmt.Errorf("only a sets or bans grant can name sets or ban lists (sets:write:%s)", strings.Join(g.Sets, ","))
 	}
 	return nil
 }
@@ -272,20 +274,26 @@ func covers(gs []model.Grant, resource string, write bool) bool {
 
 // CanSet reports whether the principal may read or change the entries of
 // a dynamic set: a sets grant covers it if it names no sets, or names it.
-func (p *Principal) CanSet(name string, write bool) bool {
+func (p *Principal) CanSet(name string, write bool) bool { return p.canNamed(ResSets, name, write) }
+
+// CanBan reports whether the principal may read or change the bans of a
+// ban list: a bans grant covers it if it names no lists, or names it.
+func (p *Principal) CanBan(list string, write bool) bool { return p.canNamed(ResBans, list, write) }
+
+func (p *Principal) canNamed(resource, name string, write bool) bool {
 	if p == nil || p.User == nil {
 		return false
 	}
-	allowed := p.User.Admin || coversSet(p.User.Grants, name, write)
+	allowed := p.User.Admin || coversNamed(p.User.Grants, resource, name, write)
 	if p.Token != nil && p.Token.Grants != nil {
-		allowed = allowed && coversSet(p.Token.Grants, name, write)
+		allowed = allowed && coversNamed(p.Token.Grants, resource, name, write)
 	}
 	return allowed
 }
 
-func coversSet(gs []model.Grant, name string, write bool) bool {
+func coversNamed(gs []model.Grant, resource, name string, write bool) bool {
 	for _, g := range gs {
-		if (g.Resource != ResAll && g.Resource != ResSets) || (write && g.Access != Write) {
+		if (g.Resource != ResAll && g.Resource != resource) || (write && g.Access != Write) {
 			continue
 		}
 		if len(g.Sets) == 0 {

internal/engine/engine.go

@@ -322,6 +322,22 @@ func Undo(st *model.State, h *model.HistoryEntry) error {
 	return nil
 }
 
+// Rewind applies a history entry backwards to st without Undo's check:
+// stepping back through the entries newest first brings back the
+// configuration as it was before each.
+func Rewind(st *model.State, h *model.HistoryEntry) error {
+	for i := len(h.Changes) - 1; i >= 0; i-- {
+		c := h.Changes[i]
+		if c.Kind == model.KindSettings && c.Before == nil {
+			continue
+		}
+		if err := put(st, c.Kind, c.ID, c.Before); err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
 // Replace makes the configuration next as a whole (imports).
 func (e *Engine) Replace(ctx context.Context, p *auth.Principal, description string, opts Options, next *model.State) (*Result, error) {
 	e.mu.Lock()
@@ -509,6 +525,12 @@ func Normalize(st *model.State) {
 		normMatch(&x.Match)
 		x.ToPort = normPorts(x.ToPort)
 	}
+	for _, b := range st.BanLists {
+		b.Name = strings.TrimSpace(b.Name)
+		b.Scope = strings.ToLower(strings.TrimSpace(b.Scope))
+		b.Hosts = normAddrs(b.Hosts)
+		b.Never = normAddrs(b.Never)
+	}
 	// Positions are fractions after insertions between rules; renumber
 	// only when two have come too close to put another between.
 	st.Sort()

internal/model/addr.go

@@ -32,6 +32,34 @@ func (a Address) String() string {
 	return a.Prefix.String()
 }
 
+// Contains reports whether the entry holds an address.
+func (a Address) Contains(x netip.Addr) bool {
+	x = x.Unmap()
+	if a.First.IsValid() {
+		return x.Is4() == a.First.Is4() && !x.Less(a.First) && !a.End.Less(x)
+	}
+	return a.Prefix.Contains(x)
+}
+
+// Overlaps reports whether the entry and a prefix share an address.
+func (a Address) Overlaps(p netip.Prefix) bool {
+	if !a.First.IsValid() {
+		return a.Prefix.Overlaps(p)
+	}
+	if a.First.Is4() != p.Addr().Is4() {
+		return false
+	}
+	// The prefix's first and last addresses against the range.
+	first := p.Masked().Addr()
+	last := first
+	b := last.AsSlice()
+	for i := p.Bits(); i < len(b)*8; i++ {
+		b[i/8] |= 0x80 >> (i % 8)
+	}
+	last, _ = netip.AddrFromSlice(b)
+	return !last.Less(a.First) && !a.End.Less(first)
+}
+
 // ParseAddress reads an address, a prefix (host bits must be zero) or a
 // range, IPv4 or IPv6.
 func ParseAddress(s string) (Address, error) {

internal/model/auth.go

@@ -7,9 +7,10 @@ import (
 
 // Grant gives access to one resource (see internal/auth).
 type Grant struct {
-	Resource string `json:"resource"` // config, sets, log, audit, *
+	Resource string `json:"resource"` // config, sets, bans, log, audit, *
 	Access   string `json:"access"`   // read or write (write implies read)
-	// Sets limits a "sets" grant to these dynamic sets (none: all).
+	// Sets limits a "sets" grant to these dynamic sets, and a "bans"
+	// grant to these ban lists (none: all).
 	Sets []string `json:"sets,omitempty"`
 }
 

internal/model/bans.go

@@ -0,0 +1,53 @@
+package model
+
+import (
+	"net/netip"
+	"time"
+)
+
+// Ban is an address a ban list keeps out (docs/bans.md): of every port,
+// or of one protocol and port. Bans are kept apart from the
+// configuration, like a dynamic set's entries: not in the history, and
+// not on probation.
+type Ban struct {
+	ID     int64  `json:"id"`
+	ListID int64  `json:"list_id"`
+	List   string `json:"list"`
+	// Address is what is banned: an address, or an IPv6 prefix of the
+	// list's grouping ("2001:db8:1:2::/64").
+	Address  string `json:"address"`
+	Protocol string `json:"protocol,omitempty"` // tcp or udp; "" every port
+	Port     int    `json:"port,omitempty"`
+	// Expires is when the ban ends (nil: it doesn't); Timeout the
+	// seconds it has left then.
+	Expires *time.Time `json:"expires,omitempty"`
+	Timeout int        `json:"timeout,omitempty"`
+	Created time.Time  `json:"created"`
+	// Who banned it (the user or token's name), Source the program's own
+	// name for why ("guard/ssh", "fail2ban/sshd"), Reason free text (a
+	// log line).
+	Who    string `json:"who"`
+	Source string `json:"source,omitempty"`
+	Reason string `json:"reason,omitempty"`
+}
+
+// Prefix is the banned address as a prefix.
+func (b *Ban) Prefix() (netip.Prefix, error) {
+	if p, err := netip.ParsePrefix(b.Address); err == nil {
+		return p, nil
+	}
+	a, err := netip.ParseAddr(b.Address)
+	if err != nil {
+		return netip.Prefix{}, err
+	}
+	return netip.PrefixFrom(a, a.BitLen()), nil
+}
+
+// BanAddress writes a banned prefix: an address alone when it is one
+// address, else the prefix.
+func BanAddress(p netip.Prefix) string {
+	if p.IsSingleIP() {
+		return p.Addr().String()
+	}
+	return p.String()
+}

internal/model/objects.go

@@ -392,3 +392,103 @@ func (r *Redirect) Label() string {
 	}
 	return fmt.Sprintf("redirect #%d", r.ID)
 }
+
+// Ban list scopes: what a list's bans keep the banned addresses from.
+const (
+	// BanScopeFirewall: the firewall itself (the input chain).
+	BanScopeFirewall = "firewall"
+	// BanScopeHosts: the list's hosts, behind the firewall (the forward
+	// chain, where port forwards' addresses are already translated).
+	BanScopeHosts = "hosts"
+	// BanScopeEverything: the firewall and everything behind it.
+	BanScopeEverything = "everything"
+)
+
+// BanScopes lists the scopes.
+var BanScopes = []string{BanScopeFirewall, BanScopeHosts, BanScopeEverything}
+
+// Ban lists' defaults.
+const (
+	DefaultBanTime    = 3600
+	DefaultBanMaxTime = 7 * 24 * 3600
+	DefaultBanEntries = 10000
+	DefaultBanGroupV6 = 64
+)
+
+// BanList is where a program such as Guard, or fail2ban, bans addresses
+// (docs/bans.md). The program names only the address, and perhaps a
+// protocol and ports; the list says what the address is kept from. The
+// bans themselves are kept apart from the configuration, like a dynamic
+// set's entries.
+type BanList struct {
+	ID    int64  `json:"id"`
+	Name  string `json:"name"`
+	Scope string `json:"scope"`
+	// Hosts are set names or addresses: what scope hosts protects.
+	Hosts []string `json:"hosts,omitempty"`
+	// Never are set names or addresses the list never bans.
+	Never []string `json:"never,omitempty"`
+	// DefaultTime is a ban's length in seconds when none is asked for
+	// (0: an hour); MaxTime the longest (0: a week; -1: no limit, so
+	// bans may be permanent).
+	DefaultTime int `json:"default_time,omitempty"`
+	MaxTime     int `json:"max_time,omitempty"`
+	// MaxEntries limits the bans the list holds (0: 10000).
+	MaxEntries int `json:"max_entries,omitempty"`
+	// GroupV6 is the IPv6 prefix length banned for an address (0: /64,
+	// what one customer usually gets); IPv4 addresses are banned alone.
+	GroupV6  int    `json:"group_v6,omitempty"`
+	Comment  string `json:"comment,omitempty"`
+	Disabled bool   `json:"disabled,omitempty"`
+}
+
+func (b *BanList) GetID() int64  { return b.ID }
+func (b *BanList) Label() string { return b.Name }
+
+// BanTime is the default ban length in seconds.
+func (b *BanList) BanTime() int {
+	if b.DefaultTime > 0 {
+		return b.DefaultTime
+	}
+	return DefaultBanTime
+}
+
+// BanMaxTime is the longest ban in seconds, 0 for no limit.
+func (b *BanList) BanMaxTime() int {
+	switch {
+	case b.MaxTime < 0:
+		return 0
+	case b.MaxTime > 0:
+		return b.MaxTime
+	}
+	return DefaultBanMaxTime
+}
+
+// Entries is the most bans the list holds.
+func (b *BanList) Entries() int {
+	if b.MaxEntries > 0 {
+		return b.MaxEntries
+	}
+	return DefaultBanEntries
+}
+
+// V6Group is the IPv6 prefix length an address is banned with.
+func (b *BanList) V6Group() int {
+	if b.GroupV6 > 0 {
+		return b.GroupV6
+	}
+	return DefaultBanGroupV6
+}
+
+// Covers reports whether the list's bans apply to a chain.
+func (b *BanList) Covers(chain string) bool {
+	switch b.Scope {
+	case BanScopeFirewall:
+		return chain == ChainInput
+	case BanScopeHosts:
+		return chain == ChainForward
+	case BanScopeEverything:
+		return chain == ChainInput || chain == ChainForward
+	}
+	return false
+}

internal/model/state.go

@@ -20,6 +20,7 @@ type State struct {
 	Forwards  []*PortForward `json:"forwards"`
 	SNATs     []*SourceNAT   `json:"snats"`
 	Redirects []*Redirect    `json:"redirects"`
+	BanLists  []*BanList     `json:"ban_lists"`
 }
 
 // Object kinds, for changes and the history.
@@ -32,10 +33,11 @@ const (
 	KindForward  = "forward"
 	KindSNAT     = "snat"
 	KindRedirect = "redirect"
+	KindBanList  = "banlist"
 )
 
 // Kinds lists the object kinds (not settings) in display order.
-var Kinds = []string{KindZone, KindSet, KindService, KindRule, KindForward, KindSNAT, KindRedirect}
+var Kinds = []string{KindZone, KindSet, KindService, KindRule, KindForward, KindSNAT, KindRedirect, KindBanList}
 
 // Object is anything in the State with an ID.
 type Object interface {
@@ -73,6 +75,9 @@ func (st *State) Fill() {
 	if st.Redirects == nil {
 		st.Redirects = []*Redirect{}
 	}
+	if st.BanLists == nil {
+		st.BanLists = []*BanList{}
+	}
 }
 
 // Objects returns the objects of a kind.
@@ -107,6 +112,10 @@ func (st *State) Objects(kind string) []Object {
 		for _, o := range st.Redirects {
 			out = append(out, o)
 		}
+	case KindBanList:
+		for _, o := range st.BanLists {
+			out = append(out, o)
+		}
 	}
 	return out
 }
@@ -141,6 +150,8 @@ func (st *State) Put(kind string, id int64, data json.RawMessage) error {
 		return put(&st.SNATs, id, data)
 	case KindRedirect:
 		return put(&st.Redirects, id, data)
+	case KindBanList:
+		return put(&st.BanLists, id, data)
 	}
 	return &unknownKind{kind}
 }
@@ -187,6 +198,7 @@ func (st *State) Sort() {
 	sort.SliceStable(st.Zones, func(i, j int) bool { return st.Zones[i].Name < st.Zones[j].Name })
 	sort.SliceStable(st.Sets, func(i, j int) bool { return st.Sets[i].Name < st.Sets[j].Name })
 	sort.SliceStable(st.Services, func(i, j int) bool { return st.Services[i].Name < st.Services[j].Name })
+	sort.SliceStable(st.BanLists, func(i, j int) bool { return st.BanLists[i].Name < st.BanLists[j].Name })
 	sort.SliceStable(st.Rules, func(i, j int) bool {
 		a, b := st.Rules[i], st.Rules[j]
 		if a.Chain != b.Chain {
@@ -262,7 +274,7 @@ func (st *State) ByID(kind string, id int64) Object {
 }
 
 // Find finds an object of a kind by ID or name (zones, sets, services,
-// port forwards) or by ID alone (rules, source NAT, redirects).
+// port forwards, ban lists) or by ID alone (rules, source NAT, redirects).
 func (st *State) Find(kind, ref string) Object {
 	for _, o := range st.Objects(kind) {
 		if itoa(o.GetID()) == ref {
@@ -290,6 +302,20 @@ func (st *State) Find(kind, ref string) Object {
 				return f
 			}
 		}
+	case KindBanList:
+		if b := st.BanList(ref); b != nil {
+			return b
+		}
+	}
+	return nil
+}
+
+// BanList finds a ban list by name.
+func (st *State) BanList(name string) *BanList {
+	for _, b := range st.BanLists {
+		if strings.EqualFold(b.Name, name) {
+			return b
+		}
 	}
 	return nil
 }

internal/nft/nft.go

@@ -190,6 +190,30 @@ func (r *Runner) DeleteElement(ctx context.Context, set, addr string) error {
 	return err
 }
 
+// AddBan puts a ban in force: added, or if it is there already, given
+// its new time (nftables keeps an element's old timeout when it is added
+// again, so it is added, deleted and added in one transaction).
+func (r *Runner) AddBan(ctx context.Context, b render.Ban) error {
+	set := render.BanSet(b.List, b.Addr.Addr().Is6(), b.Proto != "")
+	e := render.BanElement(b)
+	t := ""
+	if b.Timeout > 0 {
+		t = fmt.Sprintf(" timeout %ds", b.Timeout)
+	}
+	script := fmt.Sprintf("add element inet %[1]s %[2]s { %[3]s }\ndelete element inet %[1]s %[2]s { %[3]s }\nadd element inet %[1]s %[2]s { %[3]s%[4]s }\n", render.Table, set, e, t)
+	_, err := r.run(ctx, script, "-f", "/dev/stdin")
+	return err
+}
+
+// DeleteBan lifts a ban. One already gone (it ran out) isn't an error.
+func (r *Runner) DeleteBan(ctx context.Context, b render.Ban) error {
+	set := render.BanSet(b.List, b.Addr.Addr().Is6(), b.Proto != "")
+	e := render.BanElement(b)
+	script := fmt.Sprintf("add element inet %[1]s %[2]s { %[3]s }\ndelete element inet %[1]s %[2]s { %[3]s }\n", render.Table, set, e)
+	_, err := r.run(ctx, script, "-f", "/dev/stdin")
+	return err
+}
+
 // ---- reading JSON
 
 // Table is what Housecarl reads back from its table: counters by mark
@@ -367,7 +391,7 @@ func ParseTable(data []byte) (*Table, error) {
 				continue
 			}
 			name := strings.TrimSuffix(strings.TrimSuffix(s.Name, "_v4"), "_v6")
-			if name == s.Name || strings.HasPrefix(s.Name, "meter_") || strings.HasPrefix(s.Name, "conn_") {
+			if name == s.Name || strings.HasPrefix(s.Name, "meter_") || strings.HasPrefix(s.Name, "conn_") || render.IsBanSet(s.Name) {
 				continue
 			}
 			for _, e := range s.Elem {
@@ -428,6 +452,9 @@ type Firewall struct {
 	// quotaRules are the rules with a quota in the ruleset in force, by
 	// ID: what their usage was counted by (nil before the first apply).
 	quotaRules map[int64]*model.Rule
+	// Bans returns the active bans, which every apply puts back (the
+	// database keeps them; nil: none).
+	Bans func() []render.Ban
 	// flowDevs are the flowtable's devices in the ruleset in force, by
 	// name, with their interface index then (0: left out, missing).
 	flowDevs map[string]int
@@ -654,6 +681,9 @@ func (f *Firewall) Apply(ctx context.Context, st *model.State) error {
 	usage.apply(f.adjust)
 	f.adjust = nil
 	opt.Quotas, opt.QuotaSets = usage.Quotas, usage.QuotaSets
+	if f.Bans != nil {
+		opt.Bans = f.Bans()
+	}
 	if old != nil {
 		opt.DeviceSets = old.DeviceSets
 	}
@@ -708,7 +738,16 @@ func pruneCounters(base map[string]Counter, st *model.State) {
 	for _, x := range st.Redirects {
 		ids[fmt.Sprintf("d%d", x.ID)] = true
 	}
+	for _, x := range st.BanLists {
+		ids[render.BanMark(x.ID)] = true
+	}
 	for mark := range base {
+		if strings.HasPrefix(mark, "ban") && strings.Trim(mark[3:], "0123456789") == "" {
+			if !ids[mark] {
+				delete(base, mark)
+			}
+			continue
+		}
 		obj, _, _ := strings.Cut(mark, "-") // f3-hairpin, f3-6 are f3's
 		if len(obj) < 2 || !strings.ContainsRune("rfsd", rune(obj[0])) || strings.Trim(obj[1:], "0123456789") != "" {
 			continue // b-established, p-input
@@ -791,6 +830,14 @@ func (f *Firewall) AddElement(ctx context.Context, set, addr string, timeout int
 	return f.Runner.AddElement(ctx, set, addr, timeout)
 }
 
+// AddBan puts a ban in force (Runner.AddBan).
+func (f *Firewall) AddBan(ctx context.Context, b render.Ban) error { return f.Runner.AddBan(ctx, b) }
+
+// DeleteBan lifts a ban (Runner.DeleteBan).
+func (f *Firewall) DeleteBan(ctx context.Context, b render.Ban) error {
+	return f.Runner.DeleteBan(ctx, b)
+}
+
 // DeleteElement removes an entry from a dynamic set.
 func (f *Firewall) DeleteElement(ctx context.Context, set, addr string) error {
 	return f.Runner.DeleteElement(ctx, set, addr)

internal/nft/nft_test.go

@@ -3,6 +3,7 @@ package nft
 import (
 	"context"
 	"fmt"
+	"net/netip"
 	"os"
 	"os/exec"
 	"path/filepath"
@@ -543,3 +544,66 @@ func TestPruneCounters(t *testing.T) {
 		t.Errorf("kept %v", got)
 	}
 }
+
+// TestBansInContainer: a ban goes in, a second one renews its time, a
+// reload keeps the bans the database has, and lifting one already gone
+// isn't an error.
+func TestBansInContainer(t *testing.T) {
+	image := os.Getenv("HOUSECARL_NFT_IMAGE")
+	if image == "" {
+		t.Skip("HOUSECARL_NFT_IMAGE not set")
+	}
+	name := "housecarl-nft-bans"
+	exec.Command("docker", "rm", "-f", name).Run()
+	if out, err := exec.Command("docker", "run", "-d", "--name", name, "--cap-add", "NET_ADMIN", image, "sleep", "300").CombinedOutput(); err != nil {
+		t.Fatalf("starting the container: %v %s", err, out)
+	}
+	t.Cleanup(func() { exec.Command("docker", "rm", "-f", name).Run() })
+	var active []render.Ban
+	fw := &Firewall{Runner: &Runner{Command: []string{"docker", "exec", "-i", name, "nft"}}, Bans: func() []render.Ban { return active }}
+	ctx := context.Background()
+	st := model.NewState()
+	st.Zones = []*model.Zone{{ID: 1, Name: "wan", Interfaces: []string{"eth0"}, External: true}}
+	st.BanLists = []*model.BanList{{ID: 2, Name: "router", Scope: model.BanScopeFirewall}}
+	if err := fw.Apply(ctx, st); err != nil {
+		t.Fatalf("apply: %v", err)
+	}
+	list := func(set string) string {
+		out, _ := exec.Command("docker", "exec", name, "nft", "list", "set", "inet", "housecarl", set).CombinedOutput()
+		return string(out)
+	}
+	b := render.Ban{List: 2, Addr: netip.MustParsePrefix("203.0.113.9/32"), Timeout: 60}
+	if err := fw.AddBan(ctx, b); err != nil {
+		t.Fatal(err)
+	}
+	b.Timeout = 7200
+	if err := fw.AddBan(ctx, b); err != nil {
+		t.Fatalf("renewing: %v", err)
+	}
+	if got := list("ban_2_v4"); !strings.Contains(got, "203.0.113.9 timeout 2h") {
+		t.Errorf("renewed ban: %s", got)
+	}
+	svc := render.Ban{List: 2, Addr: netip.MustParsePrefix("2001:db8:1:2::/64"), Proto: "tcp", Port: 22}
+	if err := fw.AddBan(ctx, svc); err != nil {
+		t.Fatal(err)
+	}
+	active = []render.Ban{svc}
+	if err := fw.Apply(ctx, st); err != nil {
+		t.Fatalf("reapply: %v", err)
+	}
+	if got := list("ban_2_v6_svc"); !strings.Contains(got, "2001:db8:1:2:: . 6 . 22") {
+		t.Errorf("after a reload, the database's ban: %s", got)
+	}
+	if got := list("ban_2_v4"); strings.Contains(got, "203.0.113.9") {
+		t.Errorf("after a reload, a ban the database doesn't have: %s", got)
+	}
+	if err := fw.DeleteBan(ctx, svc); err != nil {
+		t.Fatal(err)
+	}
+	if err := fw.DeleteBan(ctx, svc); err != nil {
+		t.Errorf("lifting a ban already gone: %v", err)
+	}
+	if got := list("ban_2_v6_svc"); strings.Contains(got, "2001:db8") {
+		t.Errorf("lifted ban: %s", got)
+	}
+}

internal/render/bans.go

@@ -0,0 +1,139 @@
+package render
+
+import (
+	"fmt"
+	"net/netip"
+	"strings"
+
+	"housecarl/internal/model"
+)
+
+// Ban is an active ban, for rendering (Options.Bans): its list, the
+// address banned (an IPv4 address, or an IPv6 prefix of the list's
+// grouping), and for a ban of one port its protocol and port. Timeout is
+// the seconds it has left (0: it doesn't end).
+type Ban struct {
+	List    int64
+	Addr    netip.Prefix
+	Proto   string // "tcp", "udp", or "" for every port
+	Port    int
+	Timeout int
+}
+
+// BanSet is the name of a ban list's set for a family, and with svc
+// true its set of bans of one port.
+func BanSet(list int64, v6, svc bool) string {
+	name := fmt.Sprintf("ban_%d_v4", list)
+	if v6 {
+		name = fmt.Sprintf("ban_%d_v6", list)
+	}
+	if svc {
+		name += "_svc"
+	}
+	return name
+}
+
+// BanMark is the mark (comment "hc:ban3") of a ban list's rules.
+func BanMark(list int64) string { return fmt.Sprintf("ban%d", list) }
+
+// BanElement is the set element of a ban: "203.0.113.9", or
+// "203.0.113.9 . tcp . 22".
+func BanElement(b Ban) string {
+	e := b.Addr.Addr().String()
+	if b.Proto != "" {
+		e += fmt.Sprintf(" . %s . %d", b.Proto, b.Port)
+	}
+	return e
+}
+
+// IsBanSet reports whether a set is a ban list's.
+func IsBanSet(name string) bool { return strings.HasPrefix(name, "ban_") }
+
+// banSets declares the ban lists' sets.
+func (r *renderer) banSets() {
+	for _, b := range r.st.BanLists {
+		if b.Disabled {
+			continue
+		}
+		obj := "ban list " + b.Name
+		for _, v6 := range []bool{false, true} {
+			r.w.emit(obj, "\tset %s { type %s; flags timeout; }", BanSet(b.ID, v6, false), addrType(v6))
+			r.w.emit(obj, "\tset %s { type %s . inet_proto . inet_service; flags timeout; }", BanSet(b.ID, v6, true), addrType(v6))
+		}
+	}
+}
+
+// banRules writes the drops of the ban lists that cover a chain, ahead
+// of everything that accepts: a banned address's established connections
+// are cut too.
+func (r *renderer) banRules(chain string) {
+	for _, b := range r.st.BanLists {
+		if b.Disabled || !b.Covers(chain) {
+			continue
+		}
+		obj := "ban list " + b.Name
+		d4, d6 := r.st.SplitRefs(b.Hosts)
+		for _, fam := range []string{"v4", "v6"} {
+			dsts := []string{""}
+			if b.Scope == model.BanScopeHosts {
+				refs := d4
+				if fam == "v6" {
+					refs = d6
+				}
+				var ok bool
+				if dsts, ok = addrAlts(fam, "daddr", refs, true, false); !ok {
+					continue
+				}
+			}
+			key := ipKeyword(fam) + " saddr"
+			if fam == "v6" && b.V6Group() < 128 {
+				key += " and " + v6Mask(b.V6Group())
+			}
+			v6 := fam == "v6"
+			for _, d := range dsts {
+				r.banDrop(obj, b.ID, strings.Join(nonEmpty([]string{d, key + " @" + BanSet(b.ID, v6, false)}), " "))
+				r.banDrop(obj, b.ID, strings.Join(nonEmpty([]string{d, key + " . meta l4proto . th dport @" + BanSet(b.ID, v6, true)}), " "))
+			}
+		}
+	}
+}
+
+// banDrop writes one drop of a ban list, or in shadow mode counts it.
+func (r *renderer) banDrop(obj string, list int64, match string) {
+	if r.opt.Shadow {
+		r.w.emit(obj, "\t\t%s counter accept comment \"hc:%s %s\"", match, BanMark(list), Would("drop"))
+		return
+	}
+	r.w.emit(obj, "\t\t%s counter drop comment \"hc:%s\"", match, BanMark(list))
+}
+
+// v6Mask is the mask of an IPv6 prefix length: 64 → ffff:ffff:ffff:ffff::.
+func v6Mask(bits int) string {
+	mask := make([]byte, 16)
+	for i := 0; i < bits; i++ {
+		mask[i/8] |= 0x80 >> (i % 8)
+	}
+	m, _ := netip.AddrFromSlice(mask)
+	return m.String()
+}
+
+// banElements adds the active bans, with the time each has left, so a
+// reload keeps them.
+func (r *renderer) banElements() {
+	for _, ban := range r.opt.Bans {
+		var list *model.BanList
+		for _, b := range r.st.BanLists {
+			if b.ID == ban.List && !b.Disabled {
+				list = b
+			}
+		}
+		if list == nil || !ban.Addr.IsValid() {
+			continue
+		}
+		timeout := ""
+		if ban.Timeout > 0 {
+			timeout = fmt.Sprintf(" timeout %ds", ban.Timeout)
+		}
+		r.w.emit("ban list "+list.Name, "add element inet %s %s { %s%s }", Table, BanSet(list.ID, ban.Addr.Addr().Is6(), ban.Proto != ""), BanElement(ban), timeout)
+	}
+}

internal/render/render.go

@@ -50,6 +50,8 @@ type Options struct {
 	// DeviceSets are the per-device traffic counters' entries
 	// (stats_up_v4 …), so a reload keeps them.
 	DeviceSets map[string][]Element
+	// Bans are the ban lists' active bans, added to their sets.
+	Bans []Ban
 	// Shadow renders for shadow mode: nothing is blocked, rejected,
 	// translated or changed (see the package comment).
 	Shadow bool
@@ -137,6 +139,7 @@ func (r *renderer) render() {
 	w.emit("", "delete table inet %s", Table)
 	w.emit("", "table inet %s {", Table)
 	r.sets()
+	r.banSets()
 	r.deviceSets()
 	devices := r.flowtableDevices()
 	if len(devices) > 0 {
@@ -165,6 +168,7 @@ func (r *renderer) render() {
 	w.line = base + chains.line
 	w.emit("", "}")
 	r.elements()
+	r.banElements()
 }
 
 // ---- sets
@@ -382,6 +386,7 @@ func (r *renderer) input() {
 	if b.Loopback {
 		r.builtin("loopback", "iifname \"lo\" accept")
 	}
+	r.banRules(model.ChainInput)
 	// Rules that count traffic see every packet: before the established
 	// connections are accepted.
 	r.rules(model.ChainInput, true)
@@ -411,6 +416,7 @@ func (r *renderer) forward(flowtable bool) {
 	st := r.st
 	b := st.Settings.Builtins
 	r.chain("forward", "forward", "filter", st.Settings.ForwardPolicy)
+	r.banRules(model.ChainForward)
 	ext := st.ExternalInterfaces()
 	// Shadow mode changes no packet.
 	if b.MSSClamp && len(ext) > 0 && !r.opt.Shadow {

internal/render/render_test.go

@@ -3,6 +3,7 @@ package render
 import (
 	"context"
 	"fmt"
+	"net/netip"
 	"os"
 	"os/exec"
 	"strings"
@@ -95,10 +96,28 @@ func router() *model.State {
 	st.Redirects = []*model.Redirect{
 		{ID: next(), Position: 1, Match: model.Match{InZones: []string{"lan"}, NotDests: true, Destinations: []string{"10.0.10.5"}, Services: []string{"udp/53"}}, Comment: "force DNS"},
 	}
+	st.BanLists = []*model.BanList{
+		{ID: next(), Name: "router", Scope: model.BanScopeFirewall, Never: []string{"trusted"}},
+		{ID: next(), Name: "web", Scope: model.BanScopeHosts, Hosts: []string{"10.0.10.57", "2a02:8013:7f3d:10::57", "dns_servers"}},
+		{ID: next(), Name: "edge", Scope: model.BanScopeEverything, GroupV6: 48},
+		{ID: next(), Name: "off", Scope: model.BanScopeEverything, Disabled: true},
+	}
 	st.Sort()
 	return st
 }
 
+// banOptions are active bans in each ban list, of every port and of one.
+func banOptions(st *model.State) []Ban {
+	var out []Ban
+	for _, b := range st.BanLists {
+		out = append(out,
+			Ban{List: b.ID, Addr: netip.MustParsePrefix("203.0.113.9/32"), Timeout: 3000},
+			Ban{List: b.ID, Addr: netip.MustParsePrefix("2001:db8:1:2::/64"), Proto: "tcp", Port: 22},
+			Ban{List: b.ID, Addr: netip.MustParsePrefix("198.51.100.7/32"), Proto: "udp", Port: 53, Timeout: 60})
+	}
+	return out
+}
+
 // quotaOptions carries quota usage, as a reload does: the guest quota's,
 // and two devices' of the per-device quota.
 func quotaOptions(st *model.State) (map[string]int64, map[string][]Element) {
@@ -257,7 +276,7 @@ func TestRenderedRulesetPassesNft(t *testing.T) {
 		q, qs := quotaOptions(st)
 		// Enforcing, and in shadow mode after and before other tables.
 		for _, mode := range []Options{{}, {Shadow: true}, {Shadow: true, ShadowBefore: true}} {
-			opt := Options{Elements: map[string][]Element{"guests": {{Addr: "10.0.30.5", Timeout: 1700}, {Addr: "2a02:8013:7f3d:30::5"}}}, Quotas: q, QuotaSets: qs,
+			opt := Options{Elements: map[string][]Element{"guests": {{Addr: "10.0.30.5", Timeout: 1700}, {Addr: "2a02:8013:7f3d:30::5"}}}, Quotas: q, QuotaSets: qs, Bans: banOptions(st),
 				DeviceSets: map[string][]Element{"stats_up_v4": {{Addr: "10.0.10.5", Timeout: 3000, Packets: 7, Bytes: 7000}}, "stats_down_v6": {{Addr: "2001:db8:1:2::", Packets: 1, Bytes: 100}}},
 				Shadow:     mode.Shadow, ShadowBefore: mode.ShadowBefore}
 			nftCheck(t, cmd, fmt.Sprintf("%s (shadow %v, before %v)", name, mode.Shadow, mode.ShadowBefore), Render(st, opt).Text)
@@ -277,3 +296,58 @@ func nftCheck(t *testing.T, cmd []string, name, text string) {
 		}
 	}
 }
+
+func TestRenderBans(t *testing.T) {
+	st := router()
+	id := map[string]int64{}
+	for _, b := range st.BanLists {
+		id[b.Name] = b.ID
+	}
+	text := Render(st, Options{Bans: banOptions(st)}).Text
+	f := func(format string, args ...any) string { return fmt.Sprintf(format, args...) }
+	r, w, e, off := id["router"], id["web"], id["edge"], id["off"]
+	for _, want := range []string{
+		f("set ban_%d_v4 { type ipv4_addr; flags timeout; }", r),
+		f("set ban_%d_v6_svc { type ipv6_addr . inet_proto . inet_service; flags timeout; }", r),
+		// The firewall: in input, every port and one port, /64 for IPv6.
+		f(`ip saddr @ban_%d_v4 counter drop comment "hc:ban%d"`, r, r),
+		f(`ip saddr . meta l4proto . th dport @ban_%d_v4_svc counter drop comment "hc:ban%d"`, r, r),
+		f(`ip6 saddr and ffff:ffff:ffff:ffff:: @ban_%d_v6 counter drop comment "hc:ban%d"`, r, r),
+		f(`ip6 saddr and ffff:ffff:ffff:ffff:: . meta l4proto . th dport @ban_%d_v6_svc counter drop comment "hc:ban%d"`, r, r),
+		// Hosts: in forward, first, to the hosts (literals and sets).
+		f(`ip daddr 10.0.10.57 ip saddr @ban_%d_v4 counter drop`, w),
+		f(`ip daddr @dns_servers_v4 ip saddr . meta l4proto . th dport @ban_%d_v4_svc counter drop`, w),
+		f(`ip6 daddr 2a02:8013:7f3d:10::57 ip6 saddr and ffff:ffff:ffff:ffff:: @ban_%d_v6 counter drop`, w),
+		// Everything: input and forward, /48.
+		f(`ip6 saddr and ffff:ffff:ffff:: @ban_%d_v6 counter drop`, e),
+		// The bans, with their time.
+		f("add element inet housecarl ban_%d_v4 { 203.0.113.9 timeout 3000s }", r),
+		f("add element inet housecarl ban_%d_v6_svc { 2001:db8:1:2:: . tcp . 22 }", w),
+		f("add element inet housecarl ban_%d_v4_svc { 198.51.100.7 . udp . 53 timeout 60s }", e),
+	} {
+		if !strings.Contains(text, want) {
+			t.Errorf("no %q in\n%s", want, text)
+		}
+	}
+	if n := strings.Count(text, f(`@ban_%d_v4 counter drop`, e)); n != 2 {
+		t.Errorf("scope everything: %d rules of every port in IPv4, want 2 (input and forward)", n)
+	}
+	if n := strings.Count(text, f(`@ban_%d_v4 counter drop`, w)); n != 2 {
+		t.Errorf("scope hosts: %d IPv4 rules of every port, want 2 (forward: the literal host and the set)", n)
+	}
+	// Ahead of the accepts: the first rules after loopback, and of forward.
+	in, _, _ := strings.Cut(text[strings.Index(text, "chain input"):], "established")
+	fw, _, _ := strings.Cut(text[strings.Index(text, "chain forward"):], "maxseg")
+	for _, part := range []string{in, fw} {
+		if strings.Contains(part, " accept comment \"hc:r") {
+			t.Errorf("a rule accepts before the bans:\n%s", part)
+		}
+	}
+	if strings.Contains(text, f("ban_%d_", off)) {
+		t.Errorf("a disabled list is rendered")
+	}
+	shadow := Render(st, Options{Shadow: true, Bans: banOptions(st)}).Text
+	if want := f(`ip saddr @ban_%d_v4 counter accept comment "hc:ban%d [would drop]"`, r, r); !strings.Contains(shadow, want) {
+		t.Errorf("shadow: no %q", want)
+	}
+}

internal/server/authn.go

@@ -102,6 +102,8 @@ func (s *Server) authenticate(r *http.Request) (*auth.Principal, *model.Session,
 		p, err := s.tokenPrincipal(r.Context(), strings.TrimSpace(tok), ip, r.UserAgent())
 		if err != nil {
 			s.limiter.fail("ip:" + ip)
+			// For Guard and fail2ban: the address first, so a name can't fake it.
+			s.log.Warn("token refused", "ip", ip, "reason", err.Error())
 			s.eng.Audit(r.Context(), &auth.Principal{IP: ip, Method: "token"}, "auth.token_failed", "", "failure", err.Error())
 			return nil, nil, &httpError{http.StatusUnauthorized, "unauthenticated", "invalid API token: " + err.Error()}
 		}
@@ -246,6 +248,8 @@ func (s *Server) login(w http.ResponseWriter, r *http.Request) error {
 		if err == nil && u.Disabled && ok {
 			reason = "account disabled"
 		}
+		// For Guard and fail2ban: the address first, so a name can't fake it.
+		s.log.Warn("login failed", "ip", ip, "user", name, "reason", reason)
 		s.eng.Audit(r.Context(), who, "auth.login_failed", "user "+name, "failure", reason)
 		return &httpError{http.StatusUnauthorized, "login_failed", "wrong username or password"}
 	}

internal/server/bans_test.go

@@ -0,0 +1,154 @@
+package server
+
+import (
+	"bytes"
+	"context"
+	"log/slog"
+	"strings"
+	"testing"
+
+	"housecarl/internal/api"
+	"housecarl/internal/model"
+	"housecarl/internal/store"
+)
+
+// TestBans: a program with a token for one ban list bans addresses: of
+// a service's ports, every port, or numbers; renewed when banned again;
+// IPv6 by /64; never the inside, the firewall's own, or the list's
+// never; times cut to the list's longest; the address's connections the
+// ban drops are closed; and lifted by ID or address.
+func TestBans(t *testing.T) {
+	e := newEnv(t)
+	e.call(e.token, "POST", "/zones", model.Zone{Name: "wan", Interfaces: []string{"wan"}, External: true}, nil)
+	e.call(e.token, "POST", "/zones", model.Zone{Name: "lan", Interfaces: []string{"eth1"}}, nil)
+	for _, b := range []model.BanList{
+		{Name: "web", Scope: model.BanScopeHosts, Hosts: []string{"10.0.10.57"}, Never: []string{"198.51.100.0/24"}, MaxTime: 3600},
+		{Name: "router", Scope: model.BanScopeFirewall},
+	} {
+		if code := e.call(e.token, "POST", "/banlists", b, nil); code != 201 {
+			t.Fatalf("ban list %s: %d", b.Name, code)
+		}
+	}
+	fc := &fakeConns{list: []api.Connection{
+		{Family: "ipv4", Protocol: "tcp", Source: "203.0.113.9", SourcePort: 40000, Destination: "217.155.8.194", DestinationPort: 2222, DNATTo: "10.0.10.57", DNATPort: 22},
+		{Family: "ipv4", Protocol: "tcp", Source: "203.0.113.9", SourcePort: 40001, Destination: "10.0.10.58", DestinationPort: 22},
+		{Family: "ipv4", Protocol: "tcp", Source: "203.0.113.9", SourcePort: 40002, Destination: "217.155.8.194", DestinationPort: 22},
+	}}
+	e.srv.SetConntrack(fc)
+	var tok api.Token
+	if code := e.call(e.token, "POST", "/me/tokens", api.TokenInput{Name: "guard", Grants: []string{"bans:write:web"}}, &tok); code != 201 {
+		t.Fatalf("token: %d", code)
+	}
+	ban := func(in api.BanInput, want int) api.BanResult {
+		t.Helper()
+		var out api.BanResult
+		var eb api.ErrorBody
+		code := e.call(tok.Secret, "POST", "/banlists/web/bans", in, &out)
+		if code != want {
+			e.call(tok.Secret, "POST", "/banlists/web/bans", in, &eb)
+			t.Errorf("ban %+v: %d, want %d (%+v)", in, code, want, eb)
+		}
+		return out
+	}
+	out := ban(api.BanInput{Address: "203.0.113.9", Ports: []byte(`"ssh"`), Time: 600, Source: "guard/ssh", Reason: "Failed password"}, 201)
+	if len(out.Bans) != 1 || out.Bans[0].Protocol != "tcp" || out.Bans[0].Port != 22 || out.Bans[0].Existing || out.Bans[0].Timeout != 600 {
+		t.Errorf("ssh ban: %+v", out)
+	}
+	if out.Closed != 1 || len(fc.closed) != 1 || fc.closed[0].DNATTo != "10.0.10.57" {
+		t.Errorf("closed %d: %+v (only the one to the list's host)", out.Closed, fc.closed)
+	}
+	if _, ok := e.fw.bans[`ban_`+itoa(out.Bans[0].ListID)+`_v4_svc 203.0.113.9 . tcp . 22`]; !ok {
+		t.Errorf("not in force: %+v", e.fw.bans)
+	}
+	if again := ban(api.BanInput{Address: "203.0.113.9", Ports: []byte(`22`), Protocol: "tcp"}, 201); len(again.Bans) != 1 || !again.Bans[0].Existing {
+		t.Errorf("banned again: %+v", again)
+	}
+	if v6 := ban(api.BanInput{Address: "2001:db8:1:2::77", Time: 99999}, 201); len(v6.Bans) != 1 || v6.Bans[0].Address != "2001:db8:1:2::/64" ||
+		v6.Bans[0].Protocol != "" || v6.Bans[0].Timeout != 3600 {
+		t.Errorf("IPv6, every port, cut to an hour: %+v", v6)
+	}
+	if n := ban(api.BanInput{Address: "::ffff:203.0.113.10", Ports: []byte(`53`)}, 201); len(n.Bans) != 2 || n.Bans[0].Address != "203.0.113.10" {
+		t.Errorf("a number without a protocol is tcp and udp: %+v", n)
+	}
+	if n := ban(api.BanInput{Address: "203.0.113.11", Protocol: "tcp", Ports: []byte(`["http", "https"]`)}, 201); len(n.Bans) != 2 {
+		t.Errorf("two services: %+v", n)
+	}
+	if n := ban(api.BanInput{Address: "203.0.113.12", Protocol: "tcp", Ports: []byte(`"0:65535"`)}, 201); len(n.Bans) != 1 || n.Bans[0].Protocol != "" {
+		t.Errorf("fail2ban's every port: %+v", n)
+	}
+	ban(api.BanInput{Address: "203.0.113.13", Ports: []byte(`"bogus"`)}, 400)
+	ban(api.BanInput{Address: "203.0.113.13", Protocol: "tcp"}, 400)
+	ban(api.BanInput{Address: "not-an-address"}, 400)
+	for _, a := range []string{"198.51.100.4", "10.0.10.99", "217.155.8.194", "127.0.0.1", "fe80::1", "192.168.5.5"} {
+		var eb api.ErrorBody
+		if code := e.call(tok.Secret, "POST", "/banlists/web/bans", api.BanInput{Address: a}, &eb); code != 422 || eb.Error.Code != "never_ban" {
+			t.Errorf("never ban %s: %d %+v", a, code, eb)
+		}
+	}
+	if code := e.call(tok.Secret, "POST", "/banlists/router/bans", api.BanInput{Address: "203.0.113.9"}, nil); code != 403 {
+		t.Errorf("another list: %d", code)
+	}
+	// The program may read its list, and only its list.
+	var bl model.BanList
+	if code := e.call(tok.Secret, "GET", "/banlists/web", nil, &bl); code != 200 || bl.V6Group() != 64 {
+		t.Errorf("reading its list: %d %+v", code, bl)
+	}
+	var lists []model.BanList
+	if e.call(tok.Secret, "GET", "/banlists", nil, &lists); len(lists) != 1 || lists[0].Name != "web" {
+		t.Errorf("lists: %+v", lists)
+	}
+	if code := e.call(tok.Secret, "GET", "/banlists/router", nil, nil); code != 403 {
+		t.Errorf("reading another list: %d", code)
+	}
+	var bans []model.Ban
+	e.call(tok.Secret, "GET", "/banlists/web/bans", nil, &bans)
+	if len(bans) != 7 || bans[0].List != "web" || bans[0].Who != "admin" || bans[0].Source != "guard/ssh" {
+		t.Errorf("bans: %d %+v", len(bans), bans)
+	}
+	var removed map[string]int
+	if code := e.call(tok.Secret, "DELETE", "/banlists/web/bans?address=2001:db8:1:2::5", nil, &removed); code != 200 || removed["removed"] != 1 {
+		t.Errorf("lift by address: %d %+v", code, removed)
+	}
+	if code := e.call(tok.Secret, "DELETE", "/banlists/web/bans/"+itoa(bans[0].ID), nil, &removed); code != 200 || removed["removed"] != 1 {
+		t.Errorf("lift by ID: %d %+v", code, removed)
+	}
+	if code := e.call(tok.Secret, "DELETE", "/banlists/web/bans?address=203.0.113.200", nil, nil); code != 404 {
+		t.Errorf("lifting what isn't banned: %d", code)
+	}
+	if _, ok := e.fw.bans[`ban_`+itoa(bans[0].ListID)+`_v4_svc 203.0.113.9 . tcp . 22`]; ok {
+		t.Errorf("still in force after lifting")
+	}
+	evs, _ := e.srv.store.ListAudit(context.Background(), store.AuditFilter{Action: "ban."})
+	if len(evs) < 10 || !strings.Contains(evs[len(evs)-1].Detail, "guard/ssh: Failed password") {
+		t.Errorf("audit: %d %+v", len(evs), evs[len(evs)-1])
+	}
+	// The list's name is what programs know it by.
+	if code := e.call(e.token, "PATCH", "/banlists/web", map[string]any{"name": "web2"}, nil); code != 409 {
+		t.Errorf("renaming a ban list: %d", code)
+	}
+	// Every apply puts the bans back; a deleted list's go.
+	if got := ActiveBans(context.Background(), e.srv.store); len(got) != 5 {
+		t.Errorf("active bans: %d", len(got))
+	}
+	if code := e.call(e.token, "DELETE", "/banlists/web", nil, nil); code != 200 {
+		t.Errorf("deleting the list: %d", code)
+	}
+	if got := ActiveBans(context.Background(), e.srv.store); len(got) != 0 {
+		t.Errorf("a deleted list's bans: %d", len(got))
+	}
+}
+
+// TestFailedLoginLogged: a failed login and a refused token are logged
+// with the address first, for Guard's and fail2ban's patterns.
+func TestFailedLoginLogged(t *testing.T) {
+	e := newEnv(t)
+	var buf bytes.Buffer
+	e.srv.log = slog.New(slog.NewTextHandler(&buf, nil))
+	e.call("", "POST", "/auth/login", api.Login{Username: `x ip=8.8.8.8`, Password: "wrong"}, nil)
+	e.call("housecarl_aaaaaaaaaaaaaaaa_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", "GET", "/status", nil, nil)
+	for _, want := range []string{`msg="login failed" ip=127.0.0.1 user="x ip=8.8.8.8"`, `msg="token refused" ip=127.0.0.1 reason=`} {
+		if !strings.Contains(buf.String(), want) {
+			t.Errorf("no %q in\n%s", want, buf.String())
+		}
+	}
+}

internal/server/diff_test.go

@@ -0,0 +1,72 @@
+package server
+
+import (
+	"strings"
+	"testing"
+
+	udiff "github.com/aymanbagabas/go-udiff"
+
+	"housecarl/internal/api"
+	"housecarl/internal/model"
+)
+
+// TestBlockHeaders: each hunk says which chain it is in, even when the
+// chain's own line is far above it.
+func TestBlockHeaders(t *testing.T) {
+	var b strings.Builder
+	b.WriteString("table inet housecarl {\n\tset lan_v4 { type ipv4_addr; }\n\tchain input {\n")
+	for i := 0; i < 20; i++ {
+		b.WriteString("\t\tcounter comment \"in\"\n")
+	}
+	b.WriteString("\t}\n\tchain forward {\n")
+	for i := 0; i < 20; i++ {
+		b.WriteString("\t\tcounter comment \"fw\"\n")
+	}
+	b.WriteString("\t}\n}\n")
+	now := b.String()
+	then := strings.Replace(now, "\t\tcounter comment \"in\"\n\t}", "\t\tcounter comment \"in\"\n\t\taccept comment \"new in\"\n\t}", 1)
+	then = strings.Replace(then, "\t\tcounter comment \"fw\"\n\t}", "\t\taccept comment \"new fw\"\n\t}", 1)
+
+	diff := blockHeaders(udiff.Unified("a", "b", now, then), now, then)
+	var heads []string
+	for _, l := range strings.Split(diff, "\n") {
+		if strings.HasPrefix(l, "@@") {
+			heads = append(heads, l)
+		}
+	}
+	if len(heads) != 2 || !strings.HasSuffix(heads[0], "@@ chain input") || !strings.HasSuffix(heads[1], "@@ chain forward") {
+		t.Fatalf("hunk headers %q, want chain input then chain forward\n%s", heads, diff)
+	}
+	if blockHeaders("", now, then) != "" {
+		t.Fatal("no diff should stay empty")
+	}
+}
+
+// TestHistoryDiff: an entry's diff is its own change, rebuilt from the
+// current configuration however many entries came after it.
+func TestHistoryDiff(t *testing.T) {
+	e := newEnv(t)
+	var mr api.MutationResponse
+	e.call(e.token, "POST", "/zones", model.Zone{Name: "lan", Interfaces: []string{"eth1"}}, nil)
+	e.call(e.token, "POST", "/rules", model.Rule{Chain: "input", Match: model.Match{Services: []string{"ssh"}}, Comment: "first"}, &mr)
+	first := mr.HistoryID
+	e.call(e.token, "POST", "/rules", model.Rule{Chain: "forward", Match: model.Match{Services: []string{"http"}}, Comment: "second"}, nil)
+	e.call(e.token, "PATCH", "/settings", map[string]any{"builtins": map[string]any{"drop_invalid": true}}, nil)
+
+	var out struct{ Diff string }
+	if code := e.call(e.token, "GET", "/history/"+itoa(first)+"/diff", nil, &out); code != 200 {
+		t.Fatalf("diff: %d", code)
+	}
+	var added []string
+	for _, l := range strings.Split(out.Diff, "\n") {
+		if strings.HasPrefix(l, "+") && !strings.HasPrefix(l, "+++") {
+			added = append(added, l)
+		}
+	}
+	if len(added) == 0 || !strings.Contains(strings.Join(added, "\n"), ` first"`) || strings.Contains(out.Diff, ` second"`) || !strings.Contains(out.Diff, "@@ chain input") {
+		t.Errorf("entry %d's diff:\n%s", first, out.Diff)
+	}
+	if code := e.call(e.token, "GET", "/history/9999/diff", nil, nil); code != 404 {
+		t.Errorf("unknown entry: %d", code)
+	}
+}

internal/server/h_bans.go

@@ -0,0 +1,555 @@
+package server
+
+import (
+	"context"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"net/http"
+	"net/netip"
+	"strconv"
+	"strings"
+	"time"
+
+	"housecarl/internal/api"
+	"housecarl/internal/auth"
+	"housecarl/internal/model"
+	"housecarl/internal/render"
+	"housecarl/internal/store"
+)
+
+// Ban lists (docs/bans.md): the lists are configuration objects; their
+// bans are kept apart, like a dynamic set's entries, audited but not in
+// the history or on probation. A ban is saved first, then put in force;
+// every apply puts all of them back (ActiveBans).
+
+// banList finds a ban list the principal may read or ban into.
+func (s *Server) banList(r *http.Request, p *auth.Principal, write bool) (*model.BanList, error) {
+	ref := r.PathValue("ref")
+	o := s.eng.State().Find(model.KindBanList, ref)
+	if o == nil {
+		if !p.CanBan(ref, write) {
+			return nil, banForbidden(ref, write)
+		}
+		return nil, notFound("no ban list %s", ref)
+	}
+	b := o.(*model.BanList)
+	if !p.CanBan(b.Name, write) {
+		return nil, banForbidden(b.Name, write)
+	}
+	return b, nil
+}
+
+func banForbidden(list string, write bool) error {
+	access := "read"
+	if write {
+		access = "write"
+	}
+	return forbidden("permission denied: this needs bans:%s for %s", access, list)
+}
+
+// ActiveBans are the bans to put back with every apply (nft.Firewall's
+// Bans).
+func ActiveBans(ctx context.Context, st *store.Store) []render.Ban {
+	list, err := st.ListBans(ctx, 0)
+	if err != nil {
+		return nil
+	}
+	out := make([]render.Ban, 0, len(list))
+	for _, b := range list {
+		if rb, ok := renderBan(b); ok {
+			out = append(out, rb)
+		}
+	}
+	return out
+}
+
+func renderBan(b *model.Ban) (render.Ban, bool) {
+	p, err := b.Prefix()
+	if err != nil {
+		return render.Ban{}, false
+	}
+	return render.Ban{List: b.ListID, Addr: p, Proto: b.Protocol, Port: b.Port, Timeout: b.Timeout}, true
+}
+
+// listBans answers GET /banlists/{ref}/bans, and GET /bans: every list's
+// the principal may read.
+func (s *Server) listBans(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
+	st := s.eng.State()
+	var id int64
+	if r.PathValue("ref") != "" {
+		b, err := s.banList(r, p, false)
+		if err != nil {
+			return err
+		}
+		id = b.ID
+	}
+	all, err := s.store.ListBans(r.Context(), id)
+	if err != nil {
+		return err
+	}
+	out := []*model.Ban{}
+	for _, b := range all {
+		o := st.ByID(model.KindBanList, b.ListID)
+		if o == nil || !p.CanBan(o.Label(), false) {
+			continue
+		}
+		b.List = o.Label()
+		out = append(out, b)
+	}
+	writeJSON(w, http.StatusOK, out)
+	return nil
+}
+
+// addBan answers POST /banlists/{ref}/bans.
+func (s *Server) addBan(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
+	list, err := s.banList(r, p, true)
+	if err != nil {
+		return err
+	}
+	var in api.BanInput
+	if err := decodeJSON(r, &in); err != nil {
+		return err
+	}
+	st := s.eng.State()
+	if list.Disabled {
+		return conflict("ban list %s is disabled: enable it first", list.Name)
+	}
+	addr, err := netip.ParseAddr(strings.Trim(strings.TrimSpace(in.Address), "[]"))
+	if err != nil {
+		return badRequest("%q isn't an address", in.Address)
+	}
+	addr = addr.Unmap().WithZone("")
+	bits := 32
+	if addr.Is6() {
+		bits = list.V6Group()
+	}
+	prefix := netip.PrefixFrom(addr, bits).Masked()
+	if why := s.neverBan(r.Context(), st, list, p, addr, prefix); why != "" {
+		s.eng.Audit(r.Context(), p, "ban.add", list.Name+" "+model.BanAddress(prefix), "failure", why)
+		return &httpError{http.StatusUnprocessableEntity, "never_ban", fmt.Sprintf("%s isn't banned: %s", model.BanAddress(prefix), why)}
+	}
+	ports, err := banPorts(st, in.Protocol, in.Ports)
+	if err != nil {
+		return badRequest("%v", err)
+	}
+	var expires *time.Time
+	switch secs := banSeconds(list, in.Time); {
+	case secs > 0:
+		t := s.nowFn().Add(time.Duration(secs) * time.Second)
+		expires = &t
+	}
+	if n, err := s.store.CountBans(r.Context(), list.ID); err != nil {
+		return err
+	} else if n+len(ports) > list.Entries() {
+		existing := 0
+		for _, pp := range ports {
+			if s.banExists(r.Context(), list.ID, model.BanAddress(prefix), pp) {
+				existing++
+			}
+		}
+		if n+len(ports)-existing > list.Entries() {
+			return &httpError{http.StatusUnprocessableEntity, "list_full", fmt.Sprintf("ban list %s holds its most bans (%d, max_entries); wait for some to end, or raise it", list.Name, list.Entries())}
+		}
+	}
+	out := api.BanResult{Bans: []api.BanMade{}}
+	for _, pp := range ports {
+		b := &model.Ban{ListID: list.ID, List: list.Name, Address: model.BanAddress(prefix), Protocol: pp.proto, Port: pp.port,
+			Expires: expires, Who: p.Name(), Source: clip(in.Source, 100), Reason: clip(in.Reason, 500)}
+		existing, err := s.store.PutBan(r.Context(), b)
+		if err != nil {
+			return err
+		}
+		if expires != nil {
+			b.Timeout = int(time.Until(*expires).Round(time.Second) / time.Second)
+		}
+		rb, _ := renderBan(b)
+		if err := s.fw.AddBan(r.Context(), rb); err != nil {
+			s.log.Warn("putting a ban in force failed; the next change or restart adds it", "list", list.Name, "address", b.Address, "err", err)
+		}
+		out.Bans = append(out.Bans, api.BanMade{Ban: *b, Existing: existing})
+		detail := b.Source
+		if b.Reason != "" {
+			detail = strings.TrimSpace(detail + ": " + b.Reason)
+		}
+		s.eng.Audit(r.Context(), p, "ban.add", list.Name+" "+banLabel(b), "success", detail)
+	}
+	out.Closed = s.closeBanned(r.Context(), st, list, prefix, ports)
+	writeJSON(w, http.StatusCreated, out)
+	return nil
+}
+
+func (s *Server) banExists(ctx context.Context, list int64, addr string, pp banPort) bool {
+	bans, _ := s.store.ListBans(ctx, list)
+	for _, b := range bans {
+		if b.Address == addr && b.Protocol == pp.proto && b.Port == pp.port {
+			return true
+		}
+	}
+	return false
+}
+
+// banSeconds is a ban's length: what was asked (0 the list's default, -1
+// its longest), cut to the list's longest. 0 means it doesn't end.
+func banSeconds(list *model.BanList, asked int) int {
+	max := list.BanMaxTime()
+	secs := asked
+	switch {
+	case asked == 0:
+		secs = list.BanTime()
+	case asked < 0:
+		secs = max
+	}
+	if max > 0 && secs > max {
+		secs = max
+	}
+	return secs
+}
+
+func banLabel(b *model.Ban) string {
+	if b.Protocol == "" {
+		return b.Address
+	}
+	return fmt.Sprintf("%s %s/%d", b.Address, b.Protocol, b.Port)
+}
+
+func clip(s string, n int) string {
+	s = strings.TrimSpace(s)
+	if len(s) > n {
+		return s[:n]
+	}
+	return s
+}
+
+// banPort is one protocol and port banned; proto "" is every port.
+type banPort struct {
+	proto string
+	port  int
+}
+
+// maxBanPorts is the most ports one ban request may name: a range in a
+// service counts each port.
+const maxBanPorts = 64
+
+// banPorts reads a ban request's protocol and ports (api.BanInput).
+func banPorts(st *model.State, protocol string, raw json.RawMessage) ([]banPort, error) {
+	protocol = strings.ToLower(strings.TrimSpace(protocol))
+	if protocol != "" && protocol != "tcp" && protocol != "udp" && protocol != "all" {
+		return nil, fmt.Errorf("protocol %q: use tcp, udp, or leave it out for every port", protocol)
+	}
+	if protocol == "all" {
+		protocol = ""
+	}
+	var words []string
+	if len(raw) > 0 && string(raw) != "null" {
+		var list []json.RawMessage
+		if json.Unmarshal(raw, &list) != nil {
+			list = []json.RawMessage{raw}
+		}
+		for _, item := range list {
+			var n int
+			var str string
+			switch {
+			case json.Unmarshal(item, &n) == nil:
+				words = append(words, strconv.Itoa(n))
+			case json.Unmarshal(item, &str) == nil:
+				words = append(words, strings.FieldsFunc(str, func(c rune) bool { return c == ',' || c == ' ' })...)
+			default:
+				return nil, fmt.Errorf("ports: give numbers or service names, in a list or a string")
+			}
+		}
+	}
+	if len(words) == 0 {
+		if protocol != "" {
+			return nil, fmt.Errorf("protocol %s without ports: give the ports, or leave the protocol out for every port", protocol)
+		}
+		return []banPort{{}}, nil
+	}
+	protos := []string{"tcp", "udp"}
+	if protocol != "" {
+		protos = []string{protocol}
+	}
+	var out []banPort
+	seen := map[banPort]bool{}
+	add := func(proto string, port int) {
+		bp := banPort{proto, port}
+		if !seen[bp] {
+			seen[bp] = true
+			out = append(out, bp)
+		}
+	}
+	for _, w := range words {
+		lw := strings.ToLower(w)
+		switch lw {
+		case "all", "any", "0:65535", "1:65535", "0-65535", "1-65535":
+			return []banPort{{}}, nil
+		}
+		if n, err := strconv.Atoi(w); err == nil {
+			if n < 1 || n > 65535 {
+				return nil, fmt.Errorf("port %d: use 1 to 65535", n)
+			}
+			for _, p := range protos {
+				add(p, n)
+			}
+			continue
+		}
+		svc := st.Service(w)
+		if svc == nil {
+			return nil, fmt.Errorf("%q is neither a port nor a service (built-in or yours)", w)
+		}
+		matched := false
+		for _, e := range st.ServiceEntries([]string{svc.Name}) {
+			if e.Proto != "tcp" && e.Proto != "udp" || protocol != "" && e.Proto != protocol {
+				continue
+			}
+			for _, part := range strings.Split(e.Ports, ",") {
+				lo, hi, _ := strings.Cut(strings.TrimSpace(part), "-")
+				a, err1 := strconv.Atoi(lo)
+				b := a
+				var err2 error
+				if hi != "" {
+					b, err2 = strconv.Atoi(hi)
+				}
+				if err1 != nil || err2 != nil || b-a >= maxBanPorts {
+					return nil, fmt.Errorf("service %s has the range %s: ban every port instead", svc.Name, part)
+				}
+				for port := a; port <= b; port++ {
+					add(e.Proto, port)
+					matched = true
+				}
+			}
+		}
+		if !matched {
+			return nil, fmt.Errorf("service %s has no %s ports", svc.Name, map[bool]string{true: protocol, false: "tcp or udp"}[protocol != ""])
+		}
+	}
+	if len(out) > maxBanPorts {
+		return nil, fmt.Errorf("%d ports in one ban: ban every port instead (up to %d)", len(out), maxBanPorts)
+	}
+	return out, nil
+}
+
+// neverBan says why an address (banned as prefix) must not be banned, or
+// "": it is the firewall's, an inside network's, the client's own, in the
+// list's never, or not an address one bans.
+func (s *Server) neverBan(ctx context.Context, st *model.State, list *model.BanList, p *auth.Principal, addr netip.Addr, prefix netip.Prefix) string {
+	switch {
+	case addr.IsLoopback(), addr.IsUnspecified(), addr.IsMulticast(), addr.IsLinkLocalUnicast(), addr.IsLinkLocalMulticast():
+		return "it is a loopback, link-local, multicast or unspecified address"
+	}
+	if own, err := netip.ParseAddr(p.IP); err == nil && prefix.Contains(own.Unmap()) {
+		return "it is the address of whoever asked (" + own.Unmap().String() + ")"
+	}
+	inside := s.insideNetworks(st)
+	if inside(addr) || inside(prefix.Addr()) {
+		return "it is inside: in a network of an inside zone, or private"
+	}
+	external := map[string]bool{}
+	for _, z := range st.Zones {
+		external[z.Name] = z.External
+	}
+	for net, zone := range s.connected(st) {
+		if !external[zone] && net.Overlaps(prefix) {
+			return fmt.Sprintf("it overlaps %s, a network of zone %s", net, zone)
+		}
+	}
+	if s.ifaces != nil {
+		if all, err := s.ifaces.All(); err == nil {
+			for name, ps := range all {
+				for _, a := range ps {
+					if prefix.Contains(a.Addr()) {
+						return fmt.Sprintf("it is the firewall's own (%s on %s)", a.Addr(), name)
+					}
+				}
+			}
+		}
+	}
+	for _, o := range st.Settings.OwnPrefixes {
+		if op, err := netip.ParsePrefix(o); err == nil && op.Overlaps(prefix) {
+			return "it is in the firewall's own prefix " + o + " (settings)"
+		}
+	}
+	for _, ref := range list.Never {
+		if why := s.inNever(ctx, st, ref, prefix); why != "" {
+			return why + " (the list's never)"
+		}
+	}
+	return ""
+}
+
+// inNever: is prefix in a never entry (an address, or a set)?
+func (s *Server) inNever(ctx context.Context, st *model.State, ref string, prefix netip.Prefix) string {
+	overlaps := func(entry string) bool {
+		a, err := model.ParseAddress(entry)
+		return err == nil && a.Overlaps(prefix)
+	}
+	if model.IsLiteral(ref) {
+		if overlaps(ref) {
+			return "it is in " + ref
+		}
+		return ""
+	}
+	set := st.Set(ref)
+	if set == nil {
+		return ""
+	}
+	entries := set.Entries
+	if set.Dynamic {
+		if t, err := s.fw.Table(ctx); err == nil {
+			for _, e := range t.Elements[set.Name] {
+				entries = append(entries, e.Addr)
+			}
+		}
+	}
+	for _, e := range entries {
+		if overlaps(e) {
+			return fmt.Sprintf("it is in set %s (%s)", set.Name, e)
+		}
+	}
+	return ""
+}
+
+// closeBanned closes the banned address's connections the list's bans
+// now drop: offloaded to the flowtable, they would go on without
+// reaching the rules. Not in shadow mode, which changes no traffic.
+func (s *Server) closeBanned(ctx context.Context, st *model.State, list *model.BanList, prefix netip.Prefix, ports []banPort) int {
+	if s.cfg.Shadow || s.conns == nil {
+		return 0
+	}
+	conns, err := s.conns.List(ctx)
+	if err != nil {
+		s.log.Warn("listing connections to close a ban's failed", "err", err)
+		return 0
+	}
+	own := map[netip.Addr]bool{}
+	if s.ifaces != nil {
+		if all, err := s.ifaces.All(); err == nil {
+			for _, ps := range all {
+				for _, a := range ps {
+					own[a.Addr()] = true
+				}
+			}
+		}
+	}
+	h4, h6 := st.SplitRefs(list.Hosts)
+	hosts := append(append([]string{}, h4.Literals...), h6.Literals...)
+	for _, name := range append(h4.Sets, h6.Sets...) {
+		if set := st.Set(name); set != nil {
+			hosts = append(hosts, set.Entries...)
+		}
+	}
+	isHost := func(a netip.Addr) bool {
+		for _, h := range hosts {
+			if x, err := model.ParseAddress(h); err == nil && x.Contains(a) {
+				return true
+			}
+		}
+		return false
+	}
+	closed := 0
+	for _, c := range conns {
+		src, err := netip.ParseAddr(c.Source)
+		if err != nil || !prefix.Contains(src.Unmap()) {
+			continue
+		}
+		dst, _ := netip.ParseAddr(c.Destination)
+		port := c.DestinationPort
+		if c.DNATTo != "" {
+			dst, _ = netip.ParseAddr(c.DNATTo)
+			if c.DNATPort != 0 {
+				port = c.DNATPort
+			}
+		}
+		toFirewall := own[dst.Unmap()]
+		switch list.Scope {
+		case model.BanScopeFirewall:
+			if !toFirewall {
+				continue
+			}
+		case model.BanScopeHosts:
+			if toFirewall || !isHost(dst.Unmap()) {
+				continue
+			}
+		}
+		hit := false
+		for _, pp := range ports {
+			hit = hit || pp.proto == "" || pp.proto == c.Protocol && pp.port == port
+		}
+		if !hit {
+			continue
+		}
+		if err := s.conns.Close(ctx, c); err == nil {
+			closed++
+		}
+	}
+	return closed
+}
+
+// deleteBans answers DELETE /banlists/{ref}/bans/{id}, and DELETE
+// /banlists/{ref}/bans?address=A: every ban of that address (any of its
+// grouping), of every port and of each.
+func (s *Server) deleteBans(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
+	list, err := s.banList(r, p, true)
+	if err != nil {
+		return err
+	}
+	var gone []*model.Ban
+	if idText := r.PathValue("id"); idText != "" {
+		id, err := strconv.ParseInt(idText, 10, 64)
+		if err != nil {
+			return badRequest("ban ID %q isn't a number", idText)
+		}
+		b, err := s.store.GetBan(r.Context(), id)
+		if errors.Is(err, store.ErrNotFound) || err == nil && b.ListID != list.ID {
+			return notFound("no ban %d in ban list %s", id, list.Name)
+		}
+		if err != nil {
+			return err
+		}
+		gone = append(gone, b)
+	} else {
+		q := strings.Trim(strings.TrimSpace(r.URL.Query().Get("address")), "[]")
+		a, err := netip.ParseAddr(q)
+		if err != nil {
+			return badRequest("say which ban: DELETE …/bans/ID, or …/bans?address=ADDRESS")
+		}
+		a = a.Unmap()
+		bans, err := s.store.ListBans(r.Context(), list.ID)
+		if err != nil {
+			return err
+		}
+		for _, b := range bans {
+			if pre, err := b.Prefix(); err == nil && pre.Contains(a) {
+				gone = append(gone, b)
+			}
+		}
+		if len(gone) == 0 {
+			return notFound("%s isn't banned in ban list %s", a, list.Name)
+		}
+	}
+	for _, b := range gone {
+		if err := s.store.DeleteBan(r.Context(), b.ID); err != nil {
+			return err
+		}
+		if rb, ok := renderBan(b); ok {
+			if err := s.fw.DeleteBan(r.Context(), rb); err != nil {
+				s.log.Warn("lifting a ban failed; the next change or restart drops it", "list", list.Name, "address", b.Address, "err", err)
+			}
+		}
+		s.eng.Audit(r.Context(), p, "ban.remove", list.Name+" "+banLabel(b), "success", "")
+	}
+	writeJSON(w, http.StatusOK, map[string]int{"removed": len(gone)})
+	return nil
+}
+
+// pruneBans forgets the bans that ended, and those of lists st no longer
+// has.
+func (s *Server) pruneBans(ctx context.Context, st *model.State) {
+	var ids []int64
+	for _, b := range st.BanLists {
+		ids = append(ids, b.ID)
+	}
+	if _, err := s.store.PruneBans(ctx, ids); err != nil {
+		s.log.Warn("pruning bans failed", "err", err)
+	}
+}

internal/server/h_config.go

@@ -101,13 +101,75 @@ func (s *Server) rulesetDiff(before, after *model.State) string {
 	if before == nil || after == nil {
 		return ""
 	}
-	return udiff.Unified("ruleset.nft (now)", "ruleset.nft (after)", s.fw.Render(before).Text, s.fw.Render(after).Text)
+	now, then := s.fw.Render(before).Text, s.fw.Render(after).Text
+	return blockHeaders(udiff.Unified("ruleset.nft (now)", "ruleset.nft (after)", now, then), now, then)
+}
+
+// blockHeaders names, after each hunk's @@ line, the chain (or other
+// block of the table) the hunk starts in, as git names the function:
+// "@@ -40,6 +40,8 @@ chain forward". Without it a change deep in a long
+// chain doesn't say which chain it is in.
+func blockHeaders(diff, now, then string) string {
+	if diff == "" {
+		return diff
+	}
+	oldBlocks, newBlocks := enclosingBlocks(now), enclosingBlocks(then)
+	lines := strings.SplitAfter(diff, "\n")
+	for i, l := range lines {
+		if !strings.HasPrefix(l, "@@ ") {
+			continue
+		}
+		// "@@ -a,b +c,d @@": the new side's start, or the old side's
+		// when nothing of the hunk is left.
+		f := strings.Fields(l)
+		if len(f) < 4 || f[3] != "@@" {
+			continue
+		}
+		block := blockAt(newBlocks, f[2])
+		if block == "" {
+			block = blockAt(oldBlocks, f[1])
+		}
+		if block != "" {
+			lines[i] = strings.TrimRight(l, "\n") + " " + block + "\n"
+		}
+	}
+	return strings.Join(lines, "")
+}
+
+// enclosingBlocks gives each line of a rendered ruleset the block of the
+// table it is in ("chain input"), "" outside one. A block opens with a
+// line of one tab ending in "{" and closes with "\t}".
+func enclosingBlocks(text string) []string {
+	var out []string
+	cur := ""
+	for _, l := range strings.Split(text, "\n") {
+		if strings.HasPrefix(l, "\t") && !strings.HasPrefix(l, "\t\t") && strings.HasSuffix(l, "{") {
+			cur = strings.TrimSpace(strings.TrimSuffix(l, "{"))
+		}
+		out = append(out, cur)
+		if l == "\t}" {
+			cur = ""
+		}
+	}
+	return out
+}
+
+// blockAt is the block at a hunk side's start ("+40,8" or "-40"); lines
+// count from 1, and an empty side (",0") sits after its line.
+func blockAt(blocks []string, side string) string {
+	start, _, _ := strings.Cut(side[1:], ",")
+	n, err := strconv.Atoi(start)
+	if err != nil || n < 1 || n > len(blocks) {
+		return ""
+	}
+	return blocks[n-1]
 }
 
 // kindOf maps the API's plural path names to kinds.
 var kindOf = map[string]string{
 	"zones": model.KindZone, "sets": model.KindSet, "services": model.KindService, "rules": model.KindRule,
 	"forwards": model.KindForward, "snats": model.KindSNAT, "redirects": model.KindRedirect,
+	"banlists": model.KindBanList,
 }
 
 func pathKind(r *http.Request) (string, error) {
@@ -128,13 +190,16 @@ func findObject(st *model.State, kind, ref string) (model.Object, error) {
 // ---- objects
 
 func (s *Server) listObjects(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
-	if err := require(p, auth.ResConfig, false); err != nil {
-		return err
-	}
 	kind, err := pathKind(r)
 	if err != nil {
 		return err
 	}
+	if kind == model.KindBanList && !p.Can(auth.ResConfig, false) {
+		return s.listOwnBanLists(w, p)
+	}
+	if err := require(p, auth.ResConfig, false); err != nil {
+		return err
+	}
 	cur, st, draft, err := s.viewState(w, r)
 	if err != nil {
 		return err
@@ -180,13 +245,22 @@ func (s *Server) listObjects(w http.ResponseWriter, r *http.Request, p *auth.Pri
 }
 
 func (s *Server) getObject(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
-	if err := require(p, auth.ResConfig, false); err != nil {
-		return err
-	}
 	kind, err := pathKind(r)
 	if err != nil {
 		return err
 	}
+	// A program that bans into a list may read it (its grouping, times).
+	if kind == model.KindBanList && !p.Can(auth.ResConfig, false) {
+		b, err := s.banList(r, p, false)
+		if err != nil {
+			return err
+		}
+		writeJSON(w, http.StatusOK, b)
+		return nil
+	}
+	if err := require(p, auth.ResConfig, false); err != nil {
+		return err
+	}
 	_, st, _, err := s.viewState(w, r)
 	if err != nil {
 		return err
@@ -223,6 +297,8 @@ func decodeObject(kind string, data []byte) (model.Object, error) {
 		o = &model.SourceNAT{}
 	case model.KindRedirect:
 		o = &model.Redirect{}
+	case model.KindBanList:
+		o = &model.BanList{}
 	}
 	dec := json.NewDecoder(strings.NewReader(string(data)))
 	dec.DisallowUnknownFields()
@@ -274,6 +350,8 @@ func setID(o model.Object, id int64) {
 		x.ID = id
 	case *model.Redirect:
 		x.ID = id
+	case *model.BanList:
+		x.ID = id
 	}
 }
 
@@ -391,6 +469,11 @@ func usedBy(st *model.State, kind string, o model.Object) string {
 				return f.Label()
 			}
 		}
+		for _, b := range st.BanLists {
+			if has(b.Hosts) || has(b.Never) {
+				return "ban list " + b.Name
+			}
+		}
 	}
 	return ""
 }
@@ -433,6 +516,10 @@ func renameRefs(st *model.State, kind, old, now string) {
 			fix(f.PublicAddresses)
 			fix(f.HairpinSources)
 		}
+		for _, b := range st.BanLists {
+			fix(b.Hosts)
+			fix(b.Never)
+		}
 	}
 }
 
@@ -590,6 +677,38 @@ func (s *Server) getHistory(w http.ResponseWriter, r *http.Request, p *auth.Prin
 	return nil
 }
 
+// historyDiff is the ruleset's diff for one history entry: the
+// configurations before and after it, got back by stepping back from
+// the current one through every later entry, rendered by this version.
+func (s *Server) historyDiff(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
+	if err := require(p, auth.ResConfig, false); err != nil {
+		return err
+	}
+	id, err := pathID(r, "id")
+	if err != nil {
+		return err
+	}
+	hs, err := s.store.HistorySince(r.Context(), id)
+	if err != nil {
+		return err
+	}
+	if len(hs) == 0 || hs[len(hs)-1].ID != id {
+		return notFound("history entry %d not found", id)
+	}
+	st := s.eng.State()
+	var after *model.State
+	for _, h := range hs {
+		if h.ID == id {
+			after = st.Clone()
+		}
+		if err := engine.Rewind(st, h); err != nil {
+			return err
+		}
+	}
+	writeJSON(w, http.StatusOK, map[string]string{"diff": s.rulesetDiff(st, after)})
+	return nil
+}
+
 func (s *Server) revertHistory(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
 	if err := require(p, auth.ResConfig, true); err != nil {
 		return err
@@ -669,3 +788,19 @@ func (s *Server) rollbackChanges(w http.ResponseWriter, r *http.Request, p *auth
 	writeJSON(w, http.StatusOK, map[string]int{"rolled_back": n})
 	return nil
 }
+
+// listOwnBanLists lists the ban lists a principal without config:read
+// may use.
+func (s *Server) listOwnBanLists(w http.ResponseWriter, p *auth.Principal) error {
+	out := []*model.BanList{}
+	for _, b := range s.eng.State().BanLists {
+		if p.CanBan(b.Name, false) {
+			out = append(out, b)
+		}
+	}
+	if len(out) == 0 && !p.Can(auth.ResBans, false) {
+		return require(p, auth.ResConfig, false)
+	}
+	writeJSON(w, http.StatusOK, out)
+	return nil
+}

internal/server/ops.go

@@ -104,6 +104,9 @@ func updateOp(st *model.State, op *model.Op) (int64, error) {
 		return 0, err
 	}
 	setID(next, id)
+	if old, now := o.Label(), next.Label(); old != now && op.Kind == model.KindBanList {
+		return 0, conflict("ban list %s can't be renamed: the API tokens and programs (Guard, fail2ban) that ban into it go by its name; make a new list instead", old)
+	}
 	if old, now := o.Label(), next.Label(); old != now && isNamed(op.Kind) {
 		if set, ok := o.(*model.AddressSet); ok && set.Dynamic {
 			return 0, conflict("set %s is dynamic: its entries, and the API tokens and programs that fill it, go by its name; make a new set instead", old)

internal/server/routes.go

@@ -38,6 +38,11 @@ func (s *Server) routes() {
 	m.HandleFunc("GET "+p+"/sets/{ref}/elements", s.authed(s.listElements))
 	m.HandleFunc("POST "+p+"/sets/{ref}/elements", s.authed(s.addElement))
 	m.HandleFunc("DELETE "+p+"/sets/{ref}/elements/{addr}", s.authed(s.deleteElement))
+	m.HandleFunc("GET "+p+"/bans", s.authed(s.listBans))
+	m.HandleFunc("GET "+p+"/banlists/{ref}/bans", s.authed(s.listBans))
+	m.HandleFunc("POST "+p+"/banlists/{ref}/bans", s.authed(s.addBan))
+	m.HandleFunc("DELETE "+p+"/banlists/{ref}/bans", s.authed(s.deleteBans))
+	m.HandleFunc("DELETE "+p+"/banlists/{ref}/bans/{id}", s.authed(s.deleteBans))
 	m.HandleFunc("GET "+p+"/quotas", s.authed(s.listQuotas))
 	m.HandleFunc("GET "+p+"/connections", s.authed(s.listConnections))
 	m.HandleFunc("POST "+p+"/connections/close", s.authed(s.closeConnections))
@@ -70,6 +75,7 @@ func (s *Server) routes() {
 	m.HandleFunc("GET "+p+"/stats/heatmap", s.authed(s.statsHeatmap))
 	m.HandleFunc("GET "+p+"/history", s.authed(s.listHistory))
 	m.HandleFunc("GET "+p+"/history/{id}", s.authed(s.getHistory))
+	m.HandleFunc("GET "+p+"/history/{id}/diff", s.authed(s.historyDiff))
 	m.HandleFunc("POST "+p+"/history/{id}/revert", s.authed(s.revertHistory))
 	m.HandleFunc("GET "+p+"/probation", s.authed(s.getProbation))
 	m.HandleFunc("POST "+p+"/confirm", s.authed(s.confirmChanges))

internal/server/server.go

@@ -72,6 +72,8 @@ type Firewall interface {
 	Table(ctx context.Context) (*nft.Table, error)
 	AddElement(ctx context.Context, set, addr string, timeout int) error
 	DeleteElement(ctx context.Context, set, addr string) error
+	AddBan(ctx context.Context, b render.Ban) error
+	DeleteBan(ctx context.Context, b render.Ban) error
 	ListRuleset(ctx context.Context) ([]byte, error)
 	DeleteTable(ctx context.Context, family, name string) error
 	TableText(ctx context.Context, family, name string) (string, error)
@@ -150,6 +152,7 @@ func New(cfg Config, eng *engine.Engine, fw Firewall, ifaces Interfaces, log *sl
 			prev(st)
 		}
 		s.SaveQuotaUsage(context.Background())
+		s.pruneBans(context.Background(), st) // the engine is locked: not s.eng.State()
 	}
 	return s
 }
@@ -324,6 +327,7 @@ func (s *Server) housekeeping(ctx context.Context) {
 				s.log.Warn("purging sessions failed", "err", err)
 			}
 			s.limiter.cleanup()
+			s.pruneBans(ctx, s.eng.State())
 			if s.certs != nil {
 				s.certs.reloadIfChanged()
 			}

internal/server/server_test.go

@@ -48,9 +48,25 @@ type fakeFW struct {
 	ruleset    string // nft -j list ruleset ("": a default)
 	quotas     nft.QuotaUsage
 	adjusted   []nft.QuotaAdjust
+	bans       map[string]render.Ban // by element (set and key)
 }
 
 func (f *fakeFW) Check(_ context.Context, st *model.State) error { return nil }
+func (f *fakeFW) AddBan(_ context.Context, b render.Ban) error {
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	if f.bans == nil {
+		f.bans = map[string]render.Ban{}
+	}
+	f.bans[render.BanSet(b.List, b.Addr.Addr().Is6(), b.Proto != "")+" "+render.BanElement(b)] = b
+	return nil
+}
+func (f *fakeFW) DeleteBan(_ context.Context, b render.Ban) error {
+	f.mu.Lock()
+	defer f.mu.Unlock()
+	delete(f.bans, render.BanSet(b.List, b.Addr.Addr().Is6(), b.Proto != "")+" "+render.BanElement(b))
+	return nil
+}
 func (f *fakeFW) Apply(_ context.Context, st *model.State) error {
 	f.mu.Lock()
 	defer f.mu.Unlock()
@@ -1319,3 +1335,30 @@ func TestCreatePlaced(t *testing.T) {
 		t.Errorf("%d rules", n)
 	}
 }
+
+// TestTokenForUser: an administrator makes a token for another user (the
+// web UI's New token on a user's row); that user can't make one for
+// someone else.
+func TestTokenForUser(t *testing.T) {
+	e := newEnv(t)
+	var u api.User
+	if code := e.call(e.token, "POST", "/users", api.UserInput{Username: ptr("portal"), Grants: &[]string{"sets:write"}}, &u); code != 201 {
+		t.Fatalf("user: %d", code)
+	}
+	var tok api.Token
+	if code := e.call(e.token, "POST", "/users/"+itoa(u.ID)+"/tokens", api.TokenInput{Name: "portal", ExpiresIn: "30d"}, &tok); code != 201 || tok.Secret == "" || tok.UserID != u.ID {
+		t.Fatalf("token for portal: %d %+v", code, tok)
+	}
+	if code := e.call(tok.Secret, "GET", "/rules", nil, nil); code != 403 {
+		t.Errorf("the token can do more than its user: %d", code)
+	}
+	var users []api.User
+	e.call(e.token, "GET", "/users", nil, &users)
+	for _, o := range users {
+		if o.ID != u.ID {
+			if code := e.call(tok.Secret, "POST", "/users/"+itoa(o.ID)+"/tokens", api.TokenInput{Name: "x"}, nil); code != 403 {
+				t.Errorf("a token for another user, made by a non-administrator: %d", code)
+			}
+		}
+	}
+}

internal/store/bans.go

@@ -0,0 +1,135 @@
+package store
+
+import (
+	"context"
+	"database/sql"
+	"errors"
+	"time"
+
+	"housecarl/internal/model"
+)
+
+const banCols = `id, list_id, address, protocol, port, expires_at, created_at, who, source, reason`
+
+func scanBan(sc interface{ Scan(...any) error }, now time.Time) (*model.Ban, error) {
+	b := &model.Ban{}
+	var expires sql.NullString
+	var created string
+	if err := sc.Scan(&b.ID, &b.ListID, &b.Address, &b.Protocol, &b.Port, &expires, &created, &b.Who, &b.Source, &b.Reason); err != nil {
+		return nil, err
+	}
+	b.Created = parseTS(created)
+	b.Expires = parseTSPtr(expires)
+	if b.Expires != nil {
+		b.Timeout = max(int(b.Expires.Sub(now).Round(time.Second)/time.Second), 1)
+	}
+	return b, nil
+}
+
+// PutBan saves a ban, or renews the same ban (list, address, protocol
+// and port) with its new end, who and why (an empty source or reason
+// keeps the old). existing tells which; b gets
+// its ID either way.
+func (s *Store) PutBan(ctx context.Context, b *model.Ban) (existing bool, err error) {
+	now := time.Now()
+	err = s.db.QueryRowContext(ctx, `SELECT id FROM bans WHERE list_id = ? AND address = ? AND protocol = ? AND port = ?
+		AND (expires_at IS NULL OR expires_at > ?)`, b.ListID, b.Address, b.Protocol, b.Port, ts(now)).Scan(&b.ID)
+	existing = err == nil
+	if err != nil && !errors.Is(err, sql.ErrNoRows) {
+		return false, err
+	}
+	b.Created = now
+	err = s.db.QueryRowContext(ctx, `INSERT INTO bans (list_id, address, protocol, port, expires_at, created_at, who, source, reason)
+		VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
+		ON CONFLICT (list_id, address, protocol, port) DO UPDATE SET expires_at = excluded.expires_at,
+			who = excluded.who, source = COALESCE(NULLIF(excluded.source, ''), bans.source),
+			reason = COALESCE(NULLIF(excluded.reason, ''), bans.reason),
+			created_at = CASE WHEN bans.expires_at IS NULL OR bans.expires_at > excluded.created_at THEN bans.created_at ELSE excluded.created_at END
+		RETURNING id, created_at`,
+		b.ListID, b.Address, b.Protocol, b.Port, tsPtr(b.Expires), ts(now), b.Who, b.Source, b.Reason).Scan(&b.ID, new(string))
+	return existing, err
+}
+
+// ListBans lists the active bans of a list (0: of all lists), oldest
+// first.
+func (s *Store) ListBans(ctx context.Context, listID int64) ([]*model.Ban, error) {
+	now := time.Now()
+	q := `SELECT ` + banCols + ` FROM bans WHERE (expires_at IS NULL OR expires_at > ?)`
+	args := []any{ts(now)}
+	if listID != 0 {
+		q += ` AND list_id = ?`
+		args = append(args, listID)
+	}
+	rows, err := s.db.QueryContext(ctx, q+` ORDER BY id`, args...)
+	if err != nil {
+		return nil, err
+	}
+	defer rows.Close()
+	out := []*model.Ban{}
+	for rows.Next() {
+		b, err := scanBan(rows, now)
+		if err != nil {
+			return nil, err
+		}
+		out = append(out, b)
+	}
+	return out, rows.Err()
+}
+
+// GetBan loads an active ban.
+func (s *Store) GetBan(ctx context.Context, id int64) (*model.Ban, error) {
+	now := time.Now()
+	b, err := scanBan(s.db.QueryRowContext(ctx, `SELECT `+banCols+` FROM bans WHERE id = ? AND (expires_at IS NULL OR expires_at > ?)`, id, ts(now)), now)
+	if errors.Is(err, sql.ErrNoRows) {
+		return nil, ErrNotFound
+	}
+	return b, err
+}
+
+// DeleteBan removes a ban.
+func (s *Store) DeleteBan(ctx context.Context, id int64) error {
+	_, err := s.db.ExecContext(ctx, `DELETE FROM bans WHERE id = ?`, id)
+	return err
+}
+
+// CountBans counts a list's active bans.
+func (s *Store) CountBans(ctx context.Context, listID int64) (int, error) {
+	var n int
+	err := s.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM bans WHERE list_id = ? AND (expires_at IS NULL OR expires_at > ?)`, listID, ts(time.Now())).Scan(&n)
+	return n, err
+}
+
+// PruneBans deletes the bans that ended, and those of lists that no
+// longer exist (lists names the ones that do).
+func (s *Store) PruneBans(ctx context.Context, lists []int64) (int64, error) {
+	res, err := s.db.ExecContext(ctx, `DELETE FROM bans WHERE expires_at <= ?`, ts(time.Now()))
+	if err != nil {
+		return 0, err
+	}
+	n, _ := res.RowsAffected()
+	rows, err := s.db.QueryContext(ctx, `SELECT DISTINCT list_id FROM bans`)
+	if err != nil {
+		return n, err
+	}
+	var gone []int64
+	keep := map[int64]bool{}
+	for _, id := range lists {
+		keep[id] = true
+	}
+	for rows.Next() {
+		var id int64
+		if rows.Scan(&id) == nil && !keep[id] {
+			gone = append(gone, id)
+		}
+	}
+	rows.Close()
+	for _, id := range gone {
+		res, err := s.db.ExecContext(ctx, `DELETE FROM bans WHERE list_id = ?`, id)
+		if err != nil {
+			return n, err
+		}
+		m, _ := res.RowsAffected()
+		n += m
+	}
+	return n, nil
+}

internal/store/history.go

@@ -63,3 +63,21 @@ func (s *Store) GetHistory(ctx context.Context, id int64) (*model.HistoryEntry,
 	}
 	return e, err
 }
+
+// HistorySince lists the entries from id on, newest first.
+func (s *Store) HistorySince(ctx context.Context, id int64) ([]*model.HistoryEntry, error) {
+	rows, err := s.db.QueryContext(ctx, `SELECT `+historyCols+` FROM history WHERE id >= ? ORDER BY id DESC`, id)
+	if err != nil {
+		return nil, err
+	}
+	defer rows.Close()
+	var out []*model.HistoryEntry
+	for rows.Next() {
+		e, err := scanHistory(rows)
+		if err != nil {
+			return nil, err
+		}
+		out = append(out, e)
+	}
+	return out, rows.Err()
+}

internal/store/store.go

@@ -179,6 +179,23 @@ CREATE TABLE changesets (
 	history_id INTEGER
 );
 CREATE INDEX changesets_status ON changesets (status);
+`,
+	// 3: ban lists' bans (the lists are objects)
+	`
+CREATE TABLE bans (
+	id INTEGER PRIMARY KEY AUTOINCREMENT,
+	list_id INTEGER NOT NULL,
+	address TEXT NOT NULL,
+	protocol TEXT NOT NULL DEFAULT '',
+	port INTEGER NOT NULL DEFAULT 0,
+	expires_at TEXT,
+	created_at TEXT NOT NULL,
+	who TEXT NOT NULL DEFAULT '',
+	source TEXT NOT NULL DEFAULT '',
+	reason TEXT NOT NULL DEFAULT '',
+	UNIQUE (list_id, address, protocol, port)
+);
+CREATE INDEX bans_expires ON bans (expires_at);
 `,
 }
 

internal/store/store_test.go

@@ -103,3 +103,60 @@ func TestUsersAndAudit(t *testing.T) {
 		t.Errorf("audit %+v", evs)
 	}
 }
+
+func TestBans(t *testing.T) {
+	ctx := context.Background()
+	s, err := Open(filepath.Join(t.TempDir(), "housecarl.db"))
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer s.Close()
+	in := func(d time.Duration) *time.Time { t := time.Now().Add(d); return &t }
+	b := &model.Ban{ListID: 7, Address: "203.0.113.9", Expires: in(time.Hour), Who: "guard", Source: "guard/ssh"}
+	if existing, err := s.PutBan(ctx, b); err != nil || existing {
+		t.Fatalf("new ban: %v %v", existing, err)
+	}
+	id := b.ID
+	again := &model.Ban{ListID: 7, Address: "203.0.113.9", Expires: in(2 * time.Hour), Who: "guard", Reason: "again"}
+	if existing, err := s.PutBan(ctx, again); err != nil || !existing || again.ID != id {
+		t.Fatalf("renewed ban: %v %v id %d, want %d", existing, err, again.ID, id)
+	}
+	port := &model.Ban{ListID: 7, Address: "2001:db8:1:2::/64", Protocol: "tcp", Port: 22}
+	if _, err := s.PutBan(ctx, port); err != nil {
+		t.Fatal(err)
+	}
+	gone := &model.Ban{ListID: 7, Address: "198.51.100.1", Expires: in(-time.Second)}
+	if _, err := s.PutBan(ctx, gone); err != nil {
+		t.Fatal(err)
+	}
+	other := &model.Ban{ListID: 8, Address: "198.51.100.2"}
+	if _, err := s.PutBan(ctx, other); err != nil {
+		t.Fatal(err)
+	}
+	list, err := s.ListBans(ctx, 7)
+	if err != nil || len(list) != 2 {
+		t.Fatalf("list 7: %v %+v", err, list)
+	}
+	if list[0].Reason != "again" || list[0].Timeout < 7100 || list[0].Timeout > 7200 || list[1].Expires != nil || list[1].Timeout != 0 {
+		t.Errorf("bans: %+v %+v", list[0], list[1])
+	}
+	if n, _ := s.CountBans(ctx, 7); n != 2 {
+		t.Errorf("count %d", n)
+	}
+	// An ended ban is renewed as new.
+	if existing, _ := s.PutBan(ctx, &model.Ban{ListID: 7, Address: "198.51.100.1", Expires: in(time.Minute)}); existing {
+		t.Error("an ended ban counted as existing")
+	}
+	if n, err := s.PruneBans(ctx, []int64{7}); err != nil || n != 1 {
+		t.Errorf("prune: %d %v (list 8's ban)", n, err)
+	}
+	if all, _ := s.ListBans(ctx, 0); len(all) != 3 {
+		t.Errorf("after pruning: %+v", all)
+	}
+	if err := s.DeleteBan(ctx, id); err != nil {
+		t.Fatal(err)
+	}
+	if _, err := s.GetBan(ctx, id); !errors.Is(err, ErrNotFound) {
+		t.Errorf("deleted ban: %v", err)
+	}
+}

internal/validate/validate.go

@@ -82,6 +82,7 @@ func Check(st *model.State, env *Env) []Problem {
 	for _, x := range st.Redirects {
 		c.redirect(x)
 	}
+	c.banLists()
 	return c.out
 }
 
@@ -233,9 +234,9 @@ func (c *checker) sets() {
 	for _, s := range c.st.Sets {
 		obj := "set " + s.Name
 		c.name("set", obj, s.Name, seen)
-		for _, pre := range []string{"meter_", "conn_", "quota_", "stats_"} {
+		for _, pre := range []string{"meter_", "conn_", "quota_", "stats_", "ban_"} {
 			if strings.HasPrefix(strings.ToLower(s.Name), pre) {
-				c.add(Error, "set_name_reserved", obj, "set names starting with %s are Housecarl's own (rate limits, quotas, statistics); choose another name", pre)
+				c.add(Error, "set_name_reserved", obj, "set names starting with %s are Housecarl's own (rate limits, quotas, statistics, ban lists); choose another name", pre)
 			}
 		}
 		for _, e := range s.Entries {
@@ -612,3 +613,48 @@ func (c *checker) redirect(x *model.Redirect) {
 		}
 	}
 }
+
+func (c *checker) banLists() {
+	seen := map[string]string{}
+	for _, b := range c.st.BanLists {
+		obj := "ban list " + b.Name
+		c.name("banlist", obj, b.Name, seen)
+		known := false
+		for _, sc := range model.BanScopes {
+			known = known || b.Scope == sc
+		}
+		if !known {
+			c.add(Error, "banlist_scope", obj, "the scope is %q; use firewall (the firewall itself), hosts (the list's hosts) or everything", b.Scope)
+		}
+		switch {
+		case b.Scope == model.BanScopeHosts && len(b.Hosts) == 0:
+			c.add(Error, "banlist_hosts", obj, "a list with scope hosts needs its hosts: the addresses, or sets, of the machines it protects")
+		case b.Scope != model.BanScopeHosts && len(b.Hosts) > 0:
+			c.add(Error, "banlist_hosts", obj, "only a list with scope hosts has hosts; leave them out, or use scope hosts")
+		}
+		refs := func(what string, list []string) {
+			for _, r := range list {
+				if !model.IsLiteral(r) && c.st.Set(r) == nil {
+					if _, err := model.ParseAddress(r); strings.ContainsAny(r, "./:") {
+						c.add(Error, "banlist_address", obj, "%s: %v", what, err)
+					} else {
+						c.add(Error, "unknown_set", obj, "%s: there is no set %s", what, r)
+					}
+				}
+			}
+		}
+		refs("hosts", b.Hosts)
+		refs("never", b.Never)
+		if b.DefaultTime < 0 || b.MaxTime < -1 {
+			c.add(Error, "banlist_time", obj, "ban times are seconds: default_time 0 (an hour) or more, max_time 0 (a week), more, or -1 for no limit")
+		} else if max := b.BanMaxTime(); max > 0 && b.BanTime() > max {
+			c.add(Error, "banlist_time", obj, "the default ban time (%ds) is longer than the longest (%ds)", b.BanTime(), max)
+		}
+		if b.MaxEntries < 0 || b.MaxEntries > 1000000 {
+			c.add(Error, "banlist_entries", obj, "max_entries is %d; use 1 to 1000000, or 0 for 10000", b.MaxEntries)
+		}
+		if b.GroupV6 != 0 && (b.GroupV6 < 16 || b.GroupV6 > 128) {
+			c.add(Error, "banlist_group", obj, "group_v6 is /%d; use 16 to 128 (64 bans what one customer usually gets, 128 one address)", b.GroupV6)
+		}
+	}
+}

internal/validate/validate_test.go

@@ -24,7 +24,8 @@ func base() *model.State {
 		},
 		Forwards: []*model.PortForward{{ID: 7, Position: 1, Name: "jellyfin", Protocols: []string{"tcp"}, Ports: "8096", To: "10.0.10.57",
 			PublicAddresses: []string{"203.0.113.7"}, Hairpin: true}},
-		SNATs: []*model.SourceNAT{{ID: 8, Position: 1, Match: model.Match{OutZones: []string{"wan"}, Sources: []string{"trusted"}}}},
+		SNATs:    []*model.SourceNAT{{ID: 8, Position: 1, Match: model.Match{OutZones: []string{"wan"}, Sources: []string{"trusted"}}}},
+		BanLists: []*model.BanList{{ID: 9, Name: "web", Scope: model.BanScopeHosts, Hosts: []string{"10.0.10.57"}, Never: []string{"trusted"}}},
 	}
 }
 
@@ -136,6 +137,17 @@ func TestCodes(t *testing.T) {
 		{"redirect_service", func(st *model.State) {
 			st.Redirects = []*model.Redirect{{ID: 11, Position: 1, Match: model.Match{InZones: []string{"lan"}}}}
 		}},
+		{"banlist_name", func(st *model.State) { st.BanLists[0].Name = "web list" }},
+		{"banlist_scope", func(st *model.State) { st.BanLists[0].Scope = "world" }},
+		{"banlist_hosts", func(st *model.State) { st.BanLists[0].Hosts = nil }},
+		{"banlist_hosts", func(st *model.State) { st.BanLists[0].Scope = model.BanScopeFirewall }},
+		{"banlist_address", func(st *model.State) { st.BanLists[0].Never = []string{"10.0.0.300"} }},
+		{"unknown_set", func(st *model.State) { st.BanLists[0].Hosts = []string{"webservers"} }},
+		{"banlist_time", func(st *model.State) { st.BanLists[0].MaxTime = -2 }},
+		{"banlist_time", func(st *model.State) { st.BanLists[0].DefaultTime, st.BanLists[0].MaxTime = 7200, 3600 }},
+		{"banlist_entries", func(st *model.State) { st.BanLists[0].MaxEntries = -1 }},
+		{"banlist_group", func(st *model.State) { st.BanLists[0].GroupV6 = 8 }},
+		{"set_name_reserved", func(st *model.State) { st.Sets[0].Name = "ban_3" }},
 		{"redirect_to_port", func(st *model.State) {
 			st.Redirects = []*model.Redirect{{ID: 11, Position: 1, Match: model.Match{InZones: []string{"lan"}, Services: []string{"dns"}}, ToPort: "53-54"}}
 		}},

internal/web/static/assets/app.js

@@ -4,6 +4,7 @@ import { wordmark, logo } from "./brand.js";
 import { api, get, state, can, setUnauthorizedHandler, setChangeHandler, loadDraft, onDraftChange, setImmediate, useDraft } from "./api.js";
 import * as changes from "./pages/changes.js";
 import * as connections from "./pages/connections.js";
+import * as bans from "./pages/bans.js";
 import * as tracePage from "./pages/trace.js";
 import * as matrixPage from "./pages/matrix.js";
 import * as statsPage from "./pages/stats.js";
@@ -32,6 +33,7 @@ const routes = [
   { pattern: /^services$/, page: services, nav: "services" },
   { pattern: /^log$/, page: log, nav: "log" },
   { pattern: /^connections$/, page: connections, nav: "connections" },
+  { pattern: /^bans$/, page: bans, nav: "bans" },
   { pattern: /^trace$/, page: tracePage, nav: "trace" },
   { pattern: /^stats$/, page: statsPage, nav: "stats" },
   { pattern: /^settings$/, page: settings, nav: "settings" },
@@ -60,6 +62,7 @@ const navItems = [
     ["stats", "#/stats", "Statistics", "log"],
     ["log", "#/log", "Log", "log"],
     ["connections", "#/connections", "Connections", "log"],
+    ["bans", "#/bans", "Bans", "bans"],
     ["trace", "#/trace", "Trace", "log"],
   ] },
   ["changes", "#/changes", "Changes", "config"],

internal/web/static/assets/pages/account.js

@@ -1,8 +1,8 @@
 // Your account: password and API tokens.
-import { h, mount, table, time, modal, input, field, select, errorBox, toast, copyBox } from "../ui.js";
-import { get, api, state, catalog, can } from "../api.js";
+import { h, mount, table, time, input, field, errorBox, toast } from "../ui.js";
+import { get, api, state, catalog } from "../api.js";
 import { rerender } from "../app.js";
-import { revoke, rotateDialog } from "./users.js";
+import { revoke, rotateDialog, tokenDialog } from "./users.js";
 
 export async function render(el) {
   const me = state.me;
@@ -30,7 +30,7 @@ export async function render(el) {
     h("div", { class: "panel" },
       h("div", { class: "page-head" }, h("div", { class: "grow" }, h("h2", null, "API tokens"),
         h("div", { class: "muted" }, "For the hc CLI and scripts: 'hc config set token'.")),
-        h("button", { onclick: () => newToken(cat) }, "New token")),
+        h("button", { onclick: () => tokenDialog(cat) }, "New token")),
       table([
         { label: "Name", render: (t) => t.name },
         { label: "May", render: (t) => h("span", { class: "mono small" }, t.grant_strings.join(" ")) },
@@ -41,31 +41,3 @@ export async function render(el) {
           h("button", { class: "small reveal", onclick: () => revoke(t) }, "Revoke")] },
       ], tokens, { empty: "No tokens." })));
 }
-
-async function newToken(cat) {
-  const name = input({ placeholder: "laptop, backup script…" });
-  // Only roles within what the user may do: a token can never do more
-  // than its user, so offering more would only mislead.
-  const roles = Object.keys(cat.roles).filter((r) => cat.roles[r].every((g) => {
-    const [res, acc] = g.split(":");
-    return can(res, acc === "write");
-  }));
-  const role = select([["", "everything you may do"], ...roles.map((r) => [r, r + ": " + cat.roles[r].join(" ")])], "");
-  const expires = select([["", "never"], ["30d", "30 days"], ["90d", "90 days"], ["365d", "a year"]], "90d");
-  const err = h("div");
-  await modal("New API token", h("div", null, h("div", { class: "form-grid" }, field("Name", name), field("May do", role), field("Expires", expires)), err), [
-    { label: "Cancel", value: null },
-    { label: "Create", kind: "primary", onclick: async () => {
-      mount(err);
-      try {
-        const t = await api("POST", "/me/tokens", { body: { name: name.value.trim(), role: role.value || undefined, expires_in: expires.value || undefined } });
-        await modal("Token created", h("div", null, h("p", null, "Copy it now: it is not shown again."), copyBox(t.secret)));
-        rerender();
-        return true;
-      } catch (e) {
-        mount(err, errorBox(e));
-        return false;
-      }
-    } },
-  ]);
-}

internal/web/static/assets/pages/bans.js

@@ -0,0 +1,146 @@
+// Ban lists, and the addresses banned in them (docs/bans.md).
+import { h, mount, table, modal, errorBox, field, input, select, checkbox, confirmDialog, badge, toast, relTime } from "../ui.js";
+import { get, api, change, can } from "../api.js";
+import { rerender, onLeave } from "../app.js";
+import { listEditor, fmtSeconds, parseSeconds, rowClass, lookupNames, named, hits, counters } from "../widgets.js";
+
+const scopes = {
+  firewall: "the firewall itself",
+  hosts: "its hosts, behind the firewall",
+  everything: "the firewall and everything behind it",
+};
+
+export async function render(el) {
+  const config = can("config", false);
+  const write = can("config", true);
+  const lists = await get("/banlists");
+  const out = h("div");
+  const draw = async () => {
+    const [bans, cs] = await Promise.all([get("/bans"), counters()]);
+    const names = await lookupNames(bans.map((b) => b.address));
+    const count = {};
+    for (const b of bans) count[b.list_id] = (count[b.list_id] || 0) + 1;
+    mount(out,
+      h("h2", null, "Ban lists"),
+      h("div", { class: "panel" }, table([
+        { label: "Name", render: (l) => h("strong", null, l.name) },
+        { label: "Keeps banned addresses from", render: (l) => [scopes[l.scope] || l.scope, l.hosts && l.hosts.length ? h("span", { class: "mono small" }, ": " + l.hosts.join(", ")) : null] },
+        { label: "Never bans", render: (l) => h("span", { class: "mono small" }, (l.never || []).join(", ") || "–") },
+        { label: "Bans", render: (l) => String(count[l.id] || 0) },
+        { label: "Dropped", render: (l) => hits(cs, "ban" + l.id) },
+        { label: "Comment", render: (l) => l.comment },
+        write ? { label: "", tdClass: "actions-cell", render: (l) => l.pending === "deleted" ? badge("deleted", "err") : [
+          h("button", { class: "small reveal", onclick: () => listDialog(l) }, "Edit"), " ",
+          h("button", { class: "small reveal", onclick: () => del(l) }, "Delete")] } : null,
+      ].filter(Boolean), lists, { empty: "No ban lists yet.", rowClass })),
+      h("h2", null, "Banned addresses"),
+      h("div", { class: "panel" }, table([
+        { label: "Address", render: (b) => h("span", { class: "mono" }, named(names, b.address)) },
+        { label: "List", render: (b) => b.list },
+        { label: "Ports", render: (b) => b.protocol ? b.protocol + "/" + b.port : "all" },
+        { label: "Ends in", render: (b) => b.timeout ? fmtSeconds(b.timeout) : "never" },
+        { label: "Banned", render: (b) => relTime(b.created) },
+        { label: "By", render: (b) => [b.who, b.source ? h("span", { class: "muted small" }, " " + b.source) : null] },
+        { label: "Reason", render: (b) => h("span", { class: "small", title: b.reason || "" }, (b.reason || "").length > 80 ? b.reason.slice(0, 79) + "…" : b.reason || "") },
+        { label: "", tdClass: "actions-cell", render: (b) => mayBan() ? h("button", { class: "small reveal", onclick: () => unban(b, draw) }, "Unban") : null },
+      ], bans, { empty: "Nothing is banned." })));
+  };
+  mount(el,
+    h("div", { class: "page-head" },
+      h("div", { class: "grow" }, h("h1", null, "Bans"), h("div", { class: "muted" },
+        "Programs such as Guard and fail2ban ban addresses into a ban list; the list says what a banned address is kept from. Bans are dropped ahead of every rule, established connections too, and end by themselves.")),
+      h("div", { class: "actions" },
+        lists.length && mayBan() ? h("button", { onclick: () => banDialog(lists, draw) }, "Ban an address…") : null,
+        write ? h("button", { class: "primary", onclick: () => listDialog(null) }, "Add ban list") : null)),
+    config || lists.length ? out : h("div", { class: "empty" }, "No ban lists you may see."));
+  await draw();
+  const t = setInterval(() => { if (!document.hidden && !document.querySelector(".modal")) draw().catch(() => {}); }, 15000);
+  onLeave(() => clearInterval(t));
+}
+
+// mayBan: a bans:write grant may name its lists; the server checks.
+function mayBan() { return can("bans", true); }
+
+async function unban(b, draw) {
+  const what = b.address + (b.protocol ? " (" + b.protocol + "/" + b.port + ")" : "");
+  if (!await confirmDialog("Unban", "Lift the ban on " + what + " in " + b.list + "?", "Unban")) return;
+  try { await api("DELETE", "/banlists/" + encodeURIComponent(b.list) + "/bans/" + b.id); toast("Unbanned " + what); draw(); } catch (e) { toast(e.message, "err"); }
+}
+
+async function del(l) {
+  if (!await confirmDialog("Delete ban list", "Delete ban list " + l.name + "? Its bans are lifted, and programs banning into it are refused.", "Delete", "danger")) return;
+  try { await change("DELETE", "/banlists/" + l.id); rerender(); } catch (e) { modal("Not deleted", errorBox(e)); }
+}
+
+async function banDialog(lists, draw) {
+  const list = select(lists.map((l) => [l.name, l.name + " (" + l.scope + ")"]), lists[0].name);
+  const addr = input({ placeholder: "203.0.113.9 or 2001:db8::9", class: "mono" });
+  const proto = select([["", "every port"], ["tcp", "TCP"], ["udp", "UDP"]], "");
+  const ports = input({ placeholder: "22, ssh, http,https" });
+  const t = input({ placeholder: "the list's default", class: "short" });
+  const reason = input({ placeholder: "why, for the record" });
+  const err = h("div");
+  await modal("Ban an address", h("div", null, h("div", { class: "form-grid" },
+    field("Ban list", list), field("Address", addr, "An IPv6 address is banned with the list's grouping: its /64 unless the list says otherwise."),
+    field("Ports", h("div", { class: "filter-row" }, proto, ports), "Every port, or a protocol's ports (numbers or service names)."),
+    field("For", t, "1h, 1d; empty: the list's default."), field("Reason", reason)), err), [
+    { label: "Cancel", value: null },
+    { label: "Ban", kind: "danger", onclick: async () => {
+      mount(err);
+      const secs = parseSeconds(t.value);
+      if (secs === null) { mount(err, h("div", { class: "msg err" }, "Write the time as 1h, 30m or 1d.")); return false; }
+      if (proto.value && !ports.value.trim()) { mount(err, h("div", { class: "msg err" }, "Give the ports, or ban every port.")); return false; }
+      try {
+        const r = await api("POST", "/banlists/" + encodeURIComponent(list.value) + "/bans", { body: { address: addr.value.trim(), protocol: proto.value,
+          ports: proto.value ? ports.value.trim() : undefined, time: secs || 0, reason: reason.value.trim(), source: "web" } });
+        toast("Banned " + r.bans.map((b) => b.address + (b.protocol ? " " + b.protocol + "/" + b.port : "")).join(", ") + (r.closed ? "; " + r.closed + " connections closed" : ""));
+        draw();
+        return true;
+      } catch (e) { mount(err, errorBox(e)); return false; }
+    } },
+  ], { wide: true });
+}
+
+async function listDialog(l) {
+  l = l || { scope: "firewall", hosts: [], never: [] };
+  const name = input({ value: l.name || "", placeholder: "web", disabled: !!l.id });
+  const scope = select(Object.entries(scopes).map(([k, v]) => [k, k + ": " + v]), l.scope);
+  const hostsEd = listEditor(l.hosts || [], { placeholder: "10.0.10.57, 2001:db8:10::57, or a set", addLabel: "Add host" });
+  const never = listEditor(l.never || [], { placeholder: "198.51.100.0/24, or a set", addLabel: "Add address" });
+  const hostFields = field("Hosts", hostsEd, "The machines this list protects: their addresses, IPv4 and IPv6, or sets. Port forwards to them are covered too.").filter(Boolean);
+  const showHosts = () => { for (const n of hostFields) n.classList.toggle("hidden", scope.value !== "hosts"); };
+  scope.addEventListener("change", showHosts);
+  showHosts();
+  const time = input({ value: l.default_time ? fmtSeconds(l.default_time) : "", placeholder: "1h", class: "short" });
+  const maxTime = input({ value: l.max_time > 0 ? fmtSeconds(l.max_time) : "", placeholder: "7d", class: "short" });
+  const forever = checkbox("no limit: bans may last for ever", l.max_time < 0);
+  const entries = input({ value: l.max_entries || "", placeholder: "10000", class: "short", type: "number" });
+  const group = input({ value: l.group_v6 || "", placeholder: "64", class: "short", type: "number" });
+  const disabled = checkbox("disabled: the list's bans aren't in force", !!l.disabled);
+  const comment = input({ value: l.comment || "" });
+  const err = h("div");
+  await modal(l.id ? "Edit ban list " + l.name : "Add ban list", h("div", null, h("div", { class: "form-grid" },
+    field("Name", name, l.id ? "Programs and tokens know the list by its name, so it can't be changed." : "Programs ban into the list by this name; give each a token for it alone (bans:write:NAME)."),
+    field("Scope", scope, "What a banned address is kept from."), ...hostFields,
+    field("Never ban", never, "Besides what is never banned anyway: inside networks, the firewall's own addresses and whoever asks."),
+    field("Ban time", time, "A ban's length when the program asks for none."),
+    field("Longest ban", maxTime, "Longer bans are cut to this."), h("label"), forever,
+    field("Most bans", entries), field("IPv6 banned by", group, "Prefix length: 64 bans what one customer usually gets, 128 one address."),
+    h("label"), disabled, field("Comment", comment)), err), [
+    { label: "Cancel", value: null },
+    { label: l.id ? "Save" : "Add ban list", kind: "primary", onclick: async () => {
+      mount(err);
+      const t = parseSeconds(time.value), m = parseSeconds(maxTime.value);
+      if (t === null || m === null) { mount(err, h("div", { class: "msg err" }, "Write times as 1h, 30m or 7d.")); return false; }
+      const body = { scope: scope.value, hosts: scope.value === "hosts" ? hostsEd.value() : [], never: never.value(),
+        default_time: t || 0, max_time: forever.input.checked ? -1 : m || 0, max_entries: parseInt(entries.value, 10) || 0,
+        group_v6: parseInt(group.value, 10) || 0, disabled: disabled.input.checked, comment: comment.value.trim() };
+      if (!l.id) body.name = name.value.trim();
+      try {
+        await change(l.id ? "PATCH" : "POST", l.id ? "/banlists/" + l.id : "/banlists", body);
+        rerender();
+        return true;
+      } catch (e) { mount(err, errorBox(e)); return false; }
+    } },
+  ], { wide: true, sticky: true });
+}

internal/web/static/assets/pages/dashboard.js

@@ -72,19 +72,26 @@ export async function render(el) {
 }
 
 // quotaPanel lists every rule's quota with what it used: the total, or
-// for a per-address quota the busiest address (the rest are on the rule).
+// for a per-address quota how many addresses used theirs up and the
+// busiest (the rest are on the rule). A per-address quota's bar is all
+// its addresses together, so one address alone can't fill it.
 function quotaPanel(quotas, names) {
   const perAddress = (q) => q.quota.per_source || q.quota.per_destination;
-  const used = (q) => perAddress(q) ? ((q.addresses || [])[0] || { used: 0 }).used : q.used;
+  const addrs = (q) => q.addresses || [];
+  const usedUp = (q) => addrs(q).filter((a) => a.used >= q.quota.bytes).length;
+  const bar = (q) => perAddress(q)
+    ? usageBar(addrs(q).reduce((n, a) => n + Math.min(a.used, q.quota.bytes), 0), addrs(q).length * q.quota.bytes)
+    : usageBar(q.used, q.quota.bytes);
   return h("div", { class: "panel" },
     h("div", { class: "panel-head" }, h("h2", null, "Quotas"), h("a", { class: "button small", href: "#/stats" }, "Statistics")),
     table([
       { label: "Rule", render: (q) => h("a", { href: "#/rules?rule=" + q.rule_id }, q.rule) },
       { label: "Allowance", render: (q) => fmtBytes(q.quota.bytes) + (q.quota.per_source ? " each source" : q.quota.per_destination ? " each destination" : "") },
       { label: "Used", render: (q) => perAddress(q)
-        ? [plural((q.addresses || []).length, "address", "addresses"), q.addresses && q.addresses.length ? [", busiest ", named(names, q.addresses[0].address), " " + fmtBytes(used(q)) + " (" + pct(used(q), q.quota.bytes) + ")"] : ""]
+        ? [plural(addrs(q).length, "address", "addresses"), usedUp(q) ? ", " + usedUp(q) + " used up" : "",
+          addrs(q).length ? [", busiest ", named(names, addrs(q)[0].address), " " + fmtBytes(addrs(q)[0].used) + " (" + pct(addrs(q)[0].used, q.quota.bytes) + ")"] : ""]
         : fmtBytes(q.used) + " (" + pct(q.used, q.quota.bytes) + ")" },
-      { label: "", render: (q) => usageBar(used(q), q.quota.bytes) },
+      { label: "", render: (q) => addrs(q).length || !perAddress(q) ? bar(q) : "" },
       { label: "Next reset", render: (q) => q.next_reset ? time(q.next_reset) : h("span", { class: "muted" }, "by hand") },
     ], quotas));
 }

internal/web/static/assets/pages/history.js

@@ -1,5 +1,5 @@
 // History: every configuration change, with undo.
-import { h, mount, table, time, confirmDialog, modal, errorBox, badge } from "../ui.js";
+import { h, mount, table, time, confirmDialog, modal, errorBox, badge, diffView } from "../ui.js";
 import { get, change, can, state } from "../api.js";
 import { rerender } from "../app.js";
 
@@ -29,7 +29,7 @@ async function undo(e) {
 }
 
 async function entry(el, id) {
-  const e = await get("/history/" + id);
+  const [e, d] = await Promise.all([get("/history/" + id), get("/history/" + id + "/diff").catch(() => null)]);
   mount(el,
     h("nav", { class: "back-link" }, h("a", { href: "#/history" }, "← History")),
     h("div", { class: "page-head" },
@@ -40,7 +40,11 @@ async function entry(el, id) {
       can("config", true) && !e.reverted_by ? h("div", { class: "actions" }, h("button", { onclick: () => undo(e) }, "Undo")) : null),
     e.changes.map((c) => h("div", { class: "panel" },
       h("h3", null, action(c) + " " + c.kind + " ", h("span", { class: "mono" }, c.label)),
-      fieldsTable(c))));
+      fieldsTable(c))),
+    d ? h("div", { class: "panel" },
+      h("h3", null, "Ruleset"),
+      h("p", { class: "muted small" }, "What this change did to the ruleset, rendered by this version of Housecarl."),
+      diffView(d.diff)) : null);
 }
 
 function action(c) {
@@ -50,9 +54,10 @@ function action(c) {
 }
 
 // fieldsTable lists the fields that differ (all of them for a create or
-// delete).
+// delete). An object within (settings' builtins) is listed by its own
+// fields, "builtins.drop_invalid", so only what changed shows.
 function fieldsTable(c) {
-  const a = c.before || {}, b = c.after || {};
+  const a = flatten(c.before || {}), b = flatten(c.after || {});
   const keys = [...new Set([...Object.keys(a), ...Object.keys(b)])].sort();
   const show = (v, k) => {
     if (v === undefined || v === "") return "–";
@@ -68,3 +73,14 @@ function fieldsTable(c) {
     c.after ? { label: c.before ? "After" : "Value", render: (k) => h("span", { class: "mono small" }, show(b[k], k)) } : null,
   ].filter(Boolean), rows);
 }
+
+// flatten lifts the fields of objects within an object to dotted keys
+// ({builtins: {loopback: true}} → {"builtins.loopback": true}); lists
+// stay whole.
+function flatten(o, prefix = "", out = {}) {
+  for (const [k, v] of Object.entries(o)) {
+    if (v && typeof v === "object" && !Array.isArray(v) && Object.keys(v).length) flatten(v, prefix + k + ".", out);
+    else out[prefix + k] = v;
+  }
+  return out;
+}

internal/web/static/assets/pages/rules.js

@@ -167,8 +167,9 @@ function quotaLine(r, q) {
   let used = fmtBytes(qt.bytes) + each;
   if (q && !each) used = fmtBytes(q.used) + " of " + fmtBytes(qt.bytes) + " used (" + pct(q.used, qt.bytes) + ")";
   else if (q) {
-    const n = (q.addresses || []).length;
-    used += " · " + (n === 1 ? "1 address" : n + " addresses") + (n ? ", busiest " + fmtBytes(q.addresses[0].used) + " (" + pct(q.addresses[0].used, qt.bytes) + ")" : "");
+    const a = q.addresses || [], n = a.length;
+    const up = a.filter((x) => x.used >= qt.bytes).length;
+    used += " · " + (n === 1 ? "1 address" : n + " addresses") + (up ? ", " + up + " used up" : "") + (n ? ", busiest " + fmtBytes(a[0].used) + " (" + pct(a[0].used, qt.bytes) + ")" : "");
   }
   const text = "quota " + used + " · " + resets;
   return h("div", { class: "muted small" }, q && r.pending !== "new"
@@ -259,6 +260,9 @@ export async function ruleDialog(r, chain) {
   r = r || { chain, action: "accept" };
   const ch = select(["input", "forward", "output", "early"], r.chain || chain);
   const match = matchEditor(r, n);
+  const chainZones = () => match.zones(ch.value !== "output", ch.value === "forward" || ch.value === "output");
+  ch.addEventListener("change", chainZones);
+  chainZones();
   const action = select([["accept", "accept"], ["drop", "drop"], ["reject", "reject"], ["log", "log only (and go on)"]], r.action);
   const rejectWith = select([["", "with an ICMP error"], ["tcp-reset", "with a TCP reset (TCP rules only)"]], r.reject_with || "");
   const every = checkbox("every packet: also stop connections already open, not just new ones", r.every_packet);
@@ -370,6 +374,7 @@ export async function ruleDialog(r, chain) {
   const addersLabel = h("label");
   sections.raw.onShow = (on) => {
     for (const n of [...matchNodes, ...actionField, ...logField]) n.classList.toggle("hidden", on);
+    if (!on) chainZones();
     for (const [k, sec] of Object.entries(sections)) {
       if (k === "raw") continue;
       if (on && sec.open) for (const n of sec.nodes) n.classList.add("hidden");

internal/web/static/assets/pages/users.js

@@ -1,6 +1,6 @@
 // Users and everyone's tokens (administrators).
 import { h, mount, table, time, fmtTime, modal, input, field, checkbox, select, errorBox, confirmDialog, toast, badge, copyBox } from "../ui.js";
-import { get, api, catalog } from "../api.js";
+import { get, api, catalog, can } from "../api.js";
 import { rerender } from "../app.js";
 
 export async function render(el) {
@@ -16,6 +16,7 @@ export async function render(el) {
       { label: "Last login", render: (u) => u.last_login_at ? time(u.last_login_at) : "never" },
       { label: "Comment", render: (u) => u.comment },
       { label: "", tdClass: "actions-cell", render: (u) => [
+        h("button", { class: "small reveal", onclick: () => tokenDialog(cat, u) }, "New token"), " ",
         h("button", { class: "small reveal", onclick: () => userDialog(u, cat) }, "Edit"), " ",
         h("button", { class: "small reveal", onclick: () => del(u) }, "Delete")] },
     ], users)),
@@ -32,6 +33,42 @@ export async function render(el) {
     ], tokens, { empty: "No tokens." })));
 }
 
+// tokenDialog makes an API token: the viewer's own, or (administrators)
+// another user's. It shows the secret once.
+export async function tokenDialog(cat, user) {
+  const name = input({ placeholder: "laptop, backup script…" });
+  // Only roles within what the user may do: a token can never do more
+  // than its user, so offering more would only mislead.
+  const may = (res, write) => {
+    if (!user) return can(res, write);
+    if (user.admin) return true;
+    const gs = user.grant_strings || [];
+    return gs.includes(res + ":write") || !write && gs.includes(res + ":read");
+  };
+  const roles = Object.keys(cat.roles).filter((r) => cat.roles[r].every((g) => {
+    const [res, acc] = g.split(":");
+    return may(res, acc === "write");
+  }));
+  const role = select([["", user ? "everything " + user.username + " may do" : "everything you may do"], ...roles.map((r) => [r, r + ": " + cat.roles[r].join(" ")])], "");
+  const expires = select([["", "never"], ["30d", "30 days"], ["90d", "90 days"], ["365d", "a year"]], "90d");
+  const err = h("div");
+  await modal(user ? "New API token for " + user.username : "New API token", h("div", null, h("div", { class: "form-grid" }, field("Name", name), field("May do", role), field("Expires", expires)), err), [
+    { label: "Cancel", value: null },
+    { label: "Create", kind: "primary", onclick: async () => {
+      mount(err);
+      try {
+        const t = await api("POST", user ? "/users/" + user.id + "/tokens" : "/me/tokens", { body: { name: name.value.trim(), role: role.value || undefined, expires_in: expires.value || undefined } });
+        await modal("Token created", h("div", null, h("p", null, "Copy it now: it is not shown again." + (user ? " Give it to " + user.username + "; 'hc config set token' sets it in the CLI." : "")), copyBox(t.secret)));
+        rerender();
+        return true;
+      } catch (e) {
+        mount(err, errorBox(e));
+        return false;
+      }
+    } },
+  ]);
+}
+
 // rotateDialog gives a token a new secret. It keeps its name, ID and
 // permissions; only the programs that use it need the new secret.
 export async function rotateDialog(t) {

internal/web/static/assets/style.css

@@ -379,6 +379,10 @@ pre.wrap { white-space: pre-wrap; word-break: break-all; font-size: 12px; }
 .list-rows { display: flex; flex-direction: column; gap: 6px; margin-bottom: 6px; }
 .list-row { display: flex; gap: 6px; align-items: center; }
 .list-row input { width: 26em; max-width: 100%; }
+.negatable { display: flex; gap: 8px; align-items: flex-start; flex-wrap: wrap; }
+.negatable > .list-editor { flex: 1; min-width: 0; }
+.negatable.negated > select { color: var(--err); border-color: var(--err); font-weight: 600; }
+.negatable.negated .list-rows { border-left: 3px solid var(--err); padding-left: 8px; }
 input.short { width: 7em; }
 .builtins { margin-bottom: 10px; line-height: 1.6; }
 .policy { margin-top: 10px; }
@@ -409,6 +413,7 @@ input.wide { width: 100%; }
 .diff .add { background: var(--add); color: var(--add-text); }
 .diff .del { background: var(--del); color: var(--del-text); }
 .diff .hunk { background: var(--hunk); color: var(--accent); }
+.diff .hunk .block { font-weight: 700; color: var(--text); }
 .diff .file { font-weight: 700; padding-top: 6px; }
 select.inline { width: auto; display: inline-block; }
 /* The rule dialog's optional parts: "+" buttons, and ✕ to close one. */

internal/web/static/assets/ui.js

@@ -200,7 +200,12 @@ export function diffView(text) {
     if (line.startsWith("+++") || line.startsWith("---")) cls = "file";
     else if (line.startsWith("+")) cls = "add";
     else if (line.startsWith("-")) cls = "del";
-    else if (line.startsWith("@@")) cls = "hunk";
+    else if (line.startsWith("@@")) {
+      // "@@ -a,b +c,d @@ chain forward": the chain the hunk is in.
+      const m = line.match(/^(@@ .*? @@)(?: (.+))?$/);
+      box.append(h("div", { class: "hunk" }, m ? [m[1], m[2] ? [" ", h("span", { class: "block" }, m[2])] : null] : line));
+      continue;
+    }
     box.append(h("div", { class: cls }, line || " "));
   }
   return box;

internal/web/static/assets/widgets.js

@@ -1,6 +1,6 @@
 // Widgets shared by several pages: list editors with suggestions, the
 // match editor of rules and NAT, descriptions, hit counts, durations.
-import { h, input, select, field, checkbox } from "./ui.js";
+import { h, input, select, field } from "./ui.js";
 import { get, api } from "./api.js";
 
 // ---- durations
@@ -65,33 +65,60 @@ export function listEditor(values, opts = {}) {
 
 // ---- matches
 
+// negatable puts "any of" / "any but" in front of a list editor, so a
+// field says first whether it matches its entries or everything else;
+// "any but" marks the list. The choice shows once the list has a row
+// (empty is "any"), and an empty list is never negated. value() is the
+// list, negated() the choice.
+function negatable(list, not) {
+  const mode = select([["", "any of"], ["not", "any but"]], not ? "not" : "", { class: "inline" });
+  const box = h("div", { class: "negatable" }, mode, list);
+  const rows = () => list.querySelector(".list-rows").children.length;
+  const update = () => {
+    mode.classList.toggle("hidden", rows() === 0);
+    box.classList.toggle("negated", mode.value === "not" && rows() > 0);
+  };
+  mode.addEventListener("change", update);
+  // Add and remove act on their own click first; this runs after.
+  list.addEventListener("click", update);
+  update();
+  box.value = list.value;
+  box.negated = () => mode.value === "not" && list.value().length > 0;
+  return box;
+}
+
 // matchEditor edits what a rule, source NAT or redirect matches. It
 // returns the form-grid fields; value() is the match's fields.
 export function matchEditor(m, n, opts = {}) {
   m = m || {};
   const zones = (list) => listEditor(list, { suggestions: n.zones, placeholder: "zone", mono: false, addLabel: "Add zone" });
   const addrs = (list) => listEditor(list, { suggestions: n.sets, placeholder: "a set, 192.168.1.0/24, 2001:db8:1::/64, 192.168.1.100-192.168.1.199", addLabel: "Add" });
-  const inZ = zones(m.in_zones), outZ = zones(m.out_zones);
-  const notIn = checkbox("any zone but these", m.not_in), notOut = checkbox("any zone but these", m.not_out);
-  const src = addrs(m.sources), dst = addrs(m.destinations);
-  const notSrc = checkbox("any source but these", m.not_sources), notDst = checkbox("any destination but these", m.not_destinations);
+  const inZ = negatable(zones(m.in_zones), m.not_in), outZ = negatable(zones(m.out_zones), m.not_out);
+  const src = negatable(addrs(m.sources), m.not_sources), dst = negatable(addrs(m.destinations), m.not_destinations);
   const svc = listEditor(m.services, { suggestions: n.services, placeholder: "ssh, https, tcp/8080, udp/1000-2000, icmp/echo-request", addLabel: "Add service" });
   const fam = select([["", "IPv4 and IPv6"], ["ipv4", "IPv4 only"], ["ipv6", "IPv6 only"]], m.family || "");
+  const inField = opts.in !== false ? field("Coming in on", inZ) : null;
+  const outField = opts.out !== false ? field("Going out on", outZ) : null;
   const fields = [
-    opts.in !== false ? field("Coming in on", h("div", null, inZ, notIn)) : null,
-    opts.out !== false ? field("Going out on", h("div", null, outZ, notOut)) : null,
-    field("From", h("div", null, src, notSrc), "Sets, addresses, prefixes or ranges, IPv4 and IPv6 mixed. Empty: any."),
-    field("To", h("div", null, dst, notDst)),
+    inField,
+    outField,
+    field("From", src, "Sets, addresses, prefixes or ranges, IPv4 and IPv6 mixed. Empty: any."),
+    field("To", dst),
     field("Services", svc, "Named services (built in, or yours) or entries such as tcp/8080. Empty: any traffic."),
     field("Family", fam),
   ];
   fields.value = () => {
-    const v = { sources: src.value(), not_sources: notSrc.input.checked, destinations: dst.value(), not_destinations: notDst.input.checked,
+    const v = { sources: src.value(), not_sources: src.negated(), destinations: dst.value(), not_destinations: dst.negated(),
       services: svc.value(), family: fam.value };
-    if (opts.in !== false) Object.assign(v, { in_zones: inZ.value(), not_in: notIn.input.checked });
-    if (opts.out !== false) Object.assign(v, { out_zones: outZ.value(), not_out: notOut.input.checked });
+    // A hidden zone field (the chain has no such interface) matches nothing.
+    if (inField) Object.assign(v, inField[1].classList.contains("hidden") ? { in_zones: [], not_in: false } : { in_zones: inZ.value(), not_in: inZ.negated() });
+    if (outField) Object.assign(v, outField[1].classList.contains("hidden") ? { out_zones: [], not_out: false } : { out_zones: outZ.value(), not_out: outZ.negated() });
     return v;
   };
+  // zones shows the zone fields a chain can use: input and early have no
+  // outgoing interface, output no incoming one.
+  const show = (f, on) => f && f.filter(Boolean).forEach((e) => e.classList.toggle("hidden", !on));
+  fields.zones = (inOn, outOn) => { show(inField, inOn); show(outField, outOn); };
   return fields;
 }
 

scripts/e2e.sh

@@ -480,6 +480,96 @@ run "the VLAN rule goes" 0 H rule delete -y "$VR"
 run "the VLAN zone goes" 0 H zone delete -y iot
 R ip link del enp2s0.30
 
+# ---- ban lists: bans kept from the firewall, a host, or everything
+
+# DNS is open to the internet on the router (tcp/53): a listener answers.
+docker exec -d ${P}-router sh -c 'while [ ! -f /tmp/nodns ]; do echo dns | nc -N -l 0.0.0.0 53; done'
+sleep 1
+run "the router's tcp/53 from the internet" 0 connect net $WAN4 53
+contains "^dns$"
+run "a ban list for the firewall" 0 H banlist add router --scope firewall
+# The lab's internet host shares the router's WAN /64: ban IPv6 addresses
+# alone (a /64 would hold the router's own, and is refused).
+run "a ban list for jellyfin" 0 H banlist add jf --scope hosts --host $JF4 --host $JF6 --group-v6 128
+run "a ban list for everything" 0 H banlist add edge --scope everything --max-time 1h
+run "the lists" 0 H banlist list
+contains "jf  *hosts  *10.88.10.57, 2001:db8:7f3d:10::57  *0"
+# The firewall's own DNS, for the internet host.
+run "ban from the firewall's tcp/53" 0 H ban router $NET4 --tcp dns --reason "e2e"
+contains "banned $NET4, tcp/53, in router"
+run "the router's DNS is cut for it" 1 connect net $WAN4 53
+run "the port forward isn't (scope firewall)" 0 connect net $WAN4 8096
+contains "^jellyfin$"
+run "the ban in the ruleset" 0 R nft list ruleset
+contains "ip saddr . meta l4proto . th dport @ban_[0-9]*_v4_svc counter packets [1-9]"
+run "lift it" 0 H unban router $NET4
+contains "1 bans lifted"
+run "DNS again" 0 connect net $WAN4 53
+contains "^dns$"
+# fail2ban on jellyfin, with Housecarl's action and a token for its list.
+run "a user for jellyfin's fail2ban" 0 H user add f2b-jf --no-password --grant bans:write:jf
+R sh -c "/w/hc token create jf --user f2b-jf > /w/f2b.token 2>/dev/null"
+run "its token" 0 test -s "$WORK/f2b.token"
+# f2b ACTION ip=… port=…: the action's command from contrib/, with the
+# jail's values, as fail2ban runs it (on jellyfin).
+f2b() {
+	local conf="$ROOT/contrib/fail2ban/action.d/housecarl.conf" a=$1; shift
+	local cmd curl
+	cmd=$(awk -v a="$a" '$0 ~ "^"a" *=" {sub(/^[^=]*= */,""); print; on=1; next} on && /^[ \t]/ {sub(/^[ \t]+/,""); print; next} {on=0}' "$conf" | tr '\n' ' ')
+	curl=$(sed -n 's/^_curl *= *//p' "$conf")
+	cmd=${cmd//<_curl>/$curl}
+	local kv
+	for kv in "$@" "list=jf" "url=http://$LAN4:8070" "tokenfile=/w/f2b.token" "curlopts=" "protocol=tcp" "bantime=600" "name=jellyfin"; do
+		cmd=${cmd//<${kv%%=*}>/${kv#*=}}
+	done
+	docker exec ${P}-jf sh -c "$cmd"
+}
+run "fail2ban bans the internet host's https" 0 f2b actionban ip=$NET4 port=https
+run "the port forward to jellyfin is cut for it" 1 connect net $WAN4 8096
+run "the router's DNS isn't (scope hosts)" 0 connect net $WAN4 53
+run "fail2ban bans its IPv6 address" 0 f2b actionban ip=$NET6 port=https
+run "IPv6 to jellyfin's 443 is cut" 1 connect net $JF6 443
+run "the bans" 0 H bans jf
+contains "$NET4  *tcp/443 .*f2b-jf  *fail2ban/jellyfin"
+contains "$NET6  *tcp/443"
+run "never the inside" 22 f2b actionban ip=$CLIENT4 port=https
+run "refused, audited" 0 H audit --action ban.add
+contains "failure.*inside"
+run "fail2ban unbans" 0 f2b actionunban ip=$NET4
+run "the port forward works again" 0 connect net $WAN4 8096
+contains "^jellyfin$"
+run "unbanning what Housecarl already let go is no error" 0 f2b actionunban ip=$NET4
+run "and the IPv6 ban" 0 f2b actionunban ip=$NET6
+run "jellyfin by IPv6 again" 0 connect net $JF6 443
+contains "jellyfin6"
+run "the token can't ban elsewhere" 22 f2b actionban ip=$NET4 port=https list=router
+# Everything, cutting a connection that is open, and kept over a restart.
+R touch /tmp/nodns
+connect net $WAN4 53 >/dev/null 2>&1
+docker exec -d ${P}-router sh -c 'nc -l 0.0.0.0 53 > /w/held.out'
+sleep 1
+docker exec -d ${P}-net sh -c "sleep 30 | nc $WAN4 53"
+sleep 1
+run "an open connection from the internet host" 0 H connections --search "$NET4 :53"
+contains "$NET4:[0-9]*  *$WAN4:53"
+run "ban it from everything" 0 H ban edge $NET4 --time 1d
+contains "banned $NET4, every port, in edge (ends in 1h)"
+contains "[1-9][0-9]* connections closed"
+run "its connection is gone" 0 H connections --search "$NET4 :53"
+not_contains ":53"
+run "nothing gets through for it" 1 connect net $WAN4 8096
+stop
+run "restart housecarld" 0 start
+run "the ban is back after the restart" 0 R nft list ruleset
+contains "$NET4 timeout [0-9hms]* expires"
+run "still nothing for it" 1 connect net $WAN4 8096
+run "lift it" 0 H unban edge $NET4
+run "the ban is out of the ruleset" 0 R nft list ruleset
+not_contains "$NET4 timeout"
+run "the port forward again" 0 connect net $WAN4 8096
+contains "^jellyfin$"
+run "the lists go" 0 sh -c "docker exec ${P}-router /w/hc banlist delete -y router && docker exec ${P}-router /w/hc banlist delete -y jf && docker exec ${P}-router /w/hc banlist delete -y edge"
+
 # ---- renaming a zone others refer to
 
 run "rename the imported dhcp zone" 0 H zone update dhcp --name lan

scripts/gitea-release.sh

@@ -114,7 +114,7 @@ done
 files="$files dist/SHA256SUMS"
 
 # The release (created, or found when publishing again).
-release=$(call 404 "$api/releases/tags/$tag")
+release=$(call 404 "$api/releases/tags/$(printf '%s' "$tag" | sed 's/+/%2B/g')")
 id=$(printf '%s\n' "$release" | jq -r '.id // empty')
 if [ -z "$id" ]; then
 	body=$(jq -n --arg tag "$tag" --arg notes "$notes" --argjson pre "$prerelease" \
@@ -127,7 +127,8 @@ for f in $files; do
 	name=${f#dist/}
 	old=$(printf '%s\n' "$release" | jq -r --arg n "$name" '.assets[]? | select(.name == $n) | .id')
 	[ -z "$old" ] || call "" -X DELETE "$api/releases/$id/assets/$old" >/dev/null
-	call "" -X POST -F "attachment=@$f" "$api/releases/$id/assets?name=$name" >/dev/null
+	# A + (build metadata, 0.2.0-rc.3+1a2b3c4) would be a space in a query.
+	call "" -X POST -F "attachment=@$f" "$api/releases/$id/assets?name=$(printf '%s' "$name" | sed 's/+/%2B/g')" >/dev/null
 	echo "gitea-release: attached $name"
 done