Release 0.1.0-rc.6

Housecarl 0.1.0's sixth release candidate. Since rc.5: the zone
matrix, what the rules do to new connections between zones. make test
(with nft 1.1.3) and make e2e (166 checks) pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

CHANGELOG.md

@@ -35,7 +35,8 @@ the way Jarl and Steward are.
   review shows the problems, nftables' check and the diff of the ruleset.
   The web UI collects edits as pending changes by default (pending rows,
   a bar with Review and Apply now, a Changes page), with a switch for live
-  edits; hc stages with `-C ID` or `HOUSECARL_CHANGESET`, and has `hc
+  edits, and Undo in History is staged like any edit; hc stages with
+  `-C ID` or `HOUSECARL_CHANGESET`, and has `hc
   changeset create/list/show/diff/apply/discard/remove`. An applied
   changeset is on probation like any change. `--dry-run` shows the
   ruleset's diff too (docs/design.md, docs/cli.md).
@@ -55,13 +56,30 @@ the way Jarl and Steward are.
   own table, `inet housecarl`, and nothing else. Dynamic sets' entries
   and the counters carry across.
 - **Probation:** a change made over the network rolls itself back unless
-  confirmed within two minutes (`hc confirm`, a banner in the web UI);
+  confirmed within two minutes. The web UI and `hc` confirm their own
+  changes once they reach Housecarl again over a new connection, from the
+  same address (a change that cuts its client off can't, and rolls back);
+  `--no-confirm` and the "a person confirms each change" setting leave it
+  to `hc confirm` or the banner;
   changes through the local socket are kept at once; a restart rolls
   back what was pending. No rule is forced to keep SSH or the web UI
   open.
 - **At boot:** the confirmed ruleset is loaded by
   `housecarl-ruleset.service` before the network comes up, without
   housecarld. An empty configuration applies nothing.
+- **Warnings about the machine:** while `nftables.service` is enabled
+  (it loads `/etc/nftables.conf` at boot, and flushes every table when
+  stopped), and while the table an import came from still applies.
+- **The zone matrix** (Matrix, `hc matrix`): what the rules do to new
+  connections between zones and to and from the firewall, worked out
+  from the rules in order, with each pair's exceptions; rules about
+  networks connected to other zones are left out of a pair.
+- **Connections and tracing:** the connections the kernel tracks, with
+  their NAT, and closing them so the rules check them again (Traffic →
+  Connections, `hc connections`); a rule option, "every packet", for
+  blocks that stop connections already open; and a packet tracer
+  (Traffic → Trace, `hc trace`) that follows packets through every
+  table's rules and names what accepted or dropped them.
 - **Hits and the log:** packet and byte counters for every rule, NAT rule,
   built-in and policy; packets logged by rules and policies, read from
   netlink log group 4171 (`hc log -f`, the Log page).
@@ -75,7 +93,10 @@ the way Jarl and Steward are.
   verdict maps are expanded into one port forward or rule per element.
 - **The `hc` CLI, the REST API and the web UI** (dashboard, rules per
   chain with hits and reordering, NAT, zones, sets, services, the log,
-  settings, import, history with undo), with users, roles, scoped API
+  settings, import, history with undo; the rule dialog keeps the less
+  used parts, such as rate limits, quotas and time windows, behind "+"
+  buttons; each chain's built-ins are listed, with their hits, and
+  switched in a window), with users, roles, scoped API
   tokens, sessions, the audit log, and the web UI's certificate from
   Let's Encrypt through Jarl, as in Steward.
 - **Packaging:** Debian packages `housecarl` and `housecarl-cli`, a

README.md

@@ -48,8 +48,11 @@ table of its own.
   ruleset into zones, sets, services, rules and port forwards, keeping the
   rules' order, and keeps what it can't translate as raw rules
   ([docs/import.md](docs/import.md)).
-* **Easy to watch.** Hits per rule, a live log of what was dropped and
+* **Easy to watch.** A zone matrix of what is allowed between zones,
-  why, and the history of every change, field by field, with undo.
+  hits per rule, a live log of what was dropped and
+  why, the connections the firewall tracks (and closing them), a packet
+  tracer that names the rule that dropped something, in any table, and
+  the history of every change, field by field, with undo.
 * **Its own table only.** Docker, Incus and libvirt keep theirs. The
   confirmed ruleset is loaded at boot, before the network comes up, even
   if housecarld doesn't start.
@@ -111,10 +114,11 @@ The web UI is on `http://127.0.0.1:8070/` (see `listen` in
 
 ## Status
 
-Housecarl 0.1.0 is in testing (0.1.0-rc.2): zones, address sets and
+Housecarl 0.1.0 is in testing (0.1.0-rc.6): zones, address sets and
 services with ordered rules, IPv4 and IPv6, port forwards with hairpin NAT
 and IPv6 pinholes, source NAT and redirects, built-ins, probation for
-network changes, the ruleset at boot, hits and the drop log, nftables
+network changes (confirmed by the client that made them), the ruleset
+at boot, hits and the drop log, nftables
 import, changesets reviewed with the ruleset's diff (as in Jarl),
 bandwidth limits and quotas, a web UI, REST API and CLI.
 

VERSION

@@ -1 +1 @@
-0.1.0-rc.2
+0.1.0-rc.6

cmd/hc/firewall.go

@@ -125,10 +125,15 @@ func settingsCmd() *cobra.Command {
 				fmt.Fprintf(w, "Antispoof IPv6:\t%s\n", list(s.AntispoofV6))
 				fmt.Fprintf(w, "Flowtable:\t%s\n", list(s.FlowtableDevices))
 				fmt.Fprintf(w, "Probation:\t%s\n", fmtSeconds(s.Probation()))
+				if s.ManualConfirm {
+					fmt.Fprintln(w, "Confirmation:\tby a person ('hc confirm', the web UI's banner)")
+				} else {
+					fmt.Fprintln(w, "Confirmation:\tautomatic, by the client that made the change, once it reaches Housecarl again over a new connection")
+				}
 			})
 		},
 	})
-	var input, forward, output, logDrops, probation string
+	var input, forward, output, logDrops, probation, autoConf string
 	var builtins, own, flowtable, spoof4, spoof6 []string
 	set := &cobra.Command{
 		Use: "set [flags]", Short: "Change settings (only the flags given)", Args: cobra.NoArgs,
@@ -172,6 +177,13 @@ dhcpv6_client, antispoof, mss_clamp.
 					p[map[string]string{"own-prefix": "own_prefixes", "flowtable": "flowtable_devices", "antispoof-v4": "antispoof_v4", "antispoof-v6": "antispoof_v6"}[flag]] = v
 				}
 			}
+			if fl.Changed("auto-confirm") {
+				on, err := onOff("auto-confirm", autoConf)
+				if err != nil {
+					return err
+				}
+				p["manual_confirm"] = !on
+			}
 			if fl.Changed("probation") {
 				n, err := parseSeconds(probation)
 				if err != nil {
@@ -193,6 +205,7 @@ dhcpv6_client, antispoof, mss_clamp.
 	f.StringArrayVar(&spoof4, "antispoof-v4", nil, "IPv4 sources dropped from the internet (replaces)")
 	f.StringArrayVar(&spoof6, "antispoof-v6", nil, "IPv6 sources dropped from the internet (replaces)")
 	f.StringVar(&probation, "probation", "", "how long a network change has to be confirmed (2m)")
+	f.StringVar(&autoConf, "auto-confirm", "", "on: the client that made a change confirms it once it reaches Housecarl again over a new connection; off: a person confirms")
 	cmd.AddCommand(set)
 	return cmd
 }
@@ -510,6 +523,13 @@ come back. Don't forget the boot: disable nftables.service, or empty
 				return err
 			}
 			fmt.Println(out["message"])
+			if tok, _ := out["confirm_token"].(string); tok != "" && !flagNoConfirm {
+				if err := autoConfirm(tok); err != nil {
+					fmt.Printf("couldn't confirm it (%v): confirm with 'hc confirm', or the tables come back\n", err)
+				} else {
+					fmt.Println("confirmed: this client reached Housecarl again over a new connection")
+				}
+			}
 			return nil
 		},
 	}

cmd/hc/main.go

@@ -45,6 +45,10 @@ var (
 	flagInsecure  bool
 	flagCACert    string
 	flagQuiet     bool
+	flagNoConfirm bool
+	// lastClient is the client the command made (to confirm a change by
+	// itself after it).
+	lastClient *client.Client
 )
 
 // DefaultSocket is housecarld's local socket.
@@ -135,6 +139,7 @@ func newClient() (*client.Client, error) {
 	if err := setupChangeset(c); err != nil {
 		return nil, err
 	}
+	lastClient = c
 	return c, nil
 }
 
@@ -151,9 +156,10 @@ Changes are made at once; to review several and apply them together,
 stage them in a changeset ('hc changeset --help', -C ID or
 HOUSECARL_CHANGESET).
 
-A change made over the network is on probation: confirm it with 'hc
+A change made over the network is on probation: hc confirms it by itself
-confirm' (within 2 minutes by default) or it is rolled back by itself, so a
+once it reaches Housecarl again over a new connection, and a change that
-change that cuts you off undoes itself. On the firewall itself, root uses
+cuts you off can't be confirmed, so it is rolled back by itself (within 2
+minutes by default). --no-confirm leaves it to 'hc confirm'. On the firewall itself, root uses
 housecarld's local socket: no token, no probation. From elsewhere, set the
 server and an API token once:
 
@@ -175,6 +181,7 @@ Exit codes: 0 ok, 1 error, 3 not authenticated or not allowed, 4 not found,
 	pf.BoolVar(&flagInsecure, "insecure", false, "don't verify the server's TLS certificate")
 	pf.StringVar(&flagCACert, "ca-cert", "", "PEM file with a CA to trust for the server's certificate")
 	pf.BoolVarP(&flagQuiet, "quiet", "q", false, "print only errors (and IDs of new objects)")
+	pf.BoolVar(&flagNoConfirm, "no-confirm", false, "leave a change on probation instead of confirming it once this client reaches Housecarl again")
 	pf.Int64VarP(&flagChangeset, "changeset", "C", 0, "stage configuration changes in this changeset instead of applying them (env HOUSECARL_CHANGESET)")
 	root.AddGroup(&cobra.Group{ID: "fw", Title: "Firewall:"}, &cobra.Group{ID: "changes", Title: "Changes:"}, &cobra.Group{ID: "admin", Title: "Administration:"})
 	add := func(group string, cmds ...*cobra.Command) {
@@ -184,7 +191,7 @@ Exit codes: 0 ok, 1 error, 3 not authenticated or not allowed, 4 not found,
 		}
 	}
 	add("fw", statusCmd(), zoneCmd(), setCmd(), serviceCmd(), ruleCmd(), forwardCmd(), snatCmd(), redirectCmd(), settingsCmd(),
-		rulesetCmd(), countersCmd(), quotasCmd(), logCmd(), interfaceCmd())
+		rulesetCmd(), countersCmd(), quotasCmd(), logCmd(), connectionsCmd(), traceCmd(), matrixCmd(), interfaceCmd())
 	add("changes", changesetCmd(), confirmCmd(), rollbackCmd(), historyCmd(), importCmd(), takeoverCmd())
 	add("admin", whoamiCmd(), userCmd(), tokenCmd(), passwordCmd(), certCmd(), auditCmd(), configCmd())
 	return root
@@ -299,6 +306,19 @@ func exitCode(err error) int {
 	return 1
 }
 
+// autoConfirm confirms a change with its token: the request goes over a
+// new connection (the server closed the change's), so it gets through
+// only if the new ruleset still lets this client reach Housecarl.
+func autoConfirm(token string) error {
+	if lastClient == nil {
+		return errors.New("no client")
+	}
+	c, cancel := context.WithTimeout(context.Background(), 10*time.Second)
+	defer cancel()
+	var out map[string]int
+	return lastClient.Do(c, "POST", "/confirm", nil, api.ConfirmInput{Token: token}, &out)
+}
+
 // report prints the outcome of a change.
 func report(resp *api.MutationResponse) error {
 	switch flagOutput {
@@ -343,6 +363,13 @@ func report(resp *api.MutationResponse) error {
 			fmt.Fprintln(w)
 		}
 		fmt.Fprintln(w, "dry run: nothing was changed")
+	case resp.Applied && resp.ConfirmBy != nil && resp.ConfirmToken != "" && !flagNoConfirm:
+		if err := autoConfirm(resp.ConfirmToken); err != nil {
+			fmt.Fprintf(w, "applied on probation (history %d), but couldn't confirm it: %v\nIt is rolled back within %s unless confirmed ('hc confirm')\n",
+				resp.HistoryID, err, time.Until(*resp.ConfirmBy).Round(time.Second))
+		} else if !flagQuiet {
+			fmt.Fprintf(w, "applied and confirmed (history %d): this client reached Housecarl again over a new connection\n", resp.HistoryID)
+		}
 	case resp.Applied && resp.ConfirmBy != nil:
 		fmt.Fprintf(w, "applied on probation (history %d): confirm with 'hc confirm' within %s, or it is rolled back\n",
 			resp.HistoryID, time.Until(*resp.ConfirmBy).Round(time.Second))

cmd/hc/rules.go

@@ -99,7 +99,7 @@ type ruleFlags struct {
 	states, statuses, days                      []string
 	hours, limit, quota                         string
 	connLimit, groupV4, groupV6                 int
-	log, disabled                               bool
+	log, disabled, everyPacket                  bool
 	before, after                               int64
 	first                                       bool
 }
@@ -122,6 +122,7 @@ func (r *ruleFlags) register(f *pflag.FlagSet) {
 	f.StringVar(&r.raw, "raw", "", "an nftables rule written by hand instead of the match flags")
 	f.StringVar(&r.comment, "comment", "", "a comment")
 	f.BoolVar(&r.disabled, "disabled", false, "keep the rule but don't use it")
+	f.BoolVar(&r.everyPacket, "every-packet", false, "check every packet, so a drop or reject also stops connections already open")
 }
 
 var amountRe = regexp.MustCompile(`^(\d+)\s*([a-zA-Z]*)$`)
@@ -244,7 +245,7 @@ func humanBytes(n int64) string {
 
 func (r *ruleFlags) rule(chain string) (model.Rule, error) {
 	rule := model.Rule{Chain: chain, Match: r.match(), Action: r.action, RejectWith: r.rejectWith, CtState: r.states, CtStatus: r.statuses,
-		Days: r.days, ConnLimit: r.connLimit, Log: r.log, LogPrefix: r.logPrefix, Raw: r.raw, Comment: r.comment, Disabled: r.disabled}
+		Days: r.days, ConnLimit: r.connLimit, EveryPacket: r.everyPacket, Log: r.log, LogPrefix: r.logPrefix, Raw: r.raw, Comment: r.comment, Disabled: r.disabled}
 	if r.hours != "" {
 		from, to, ok := strings.Cut(r.hours, "-")
 		if !ok {
@@ -285,6 +286,9 @@ func ruleCmd() *cobra.Command {
 					}
 					extra := ruleExtras(r)
 					action := r.Action
+					if r.Raw != "" {
+						action = "-" // the text's own
+					}
 					if r.Log {
 						action += "+log"
 					}
@@ -298,7 +302,11 @@ func ruleCmd() *cobra.Command {
 	}
 	cmd.AddCommand(ls)
 	cmd.AddCommand(showCmd("rules", "rule", func(w io.Writer, r model.Rule) {
+		if r.Raw == "" {
 			fmt.Fprintf(w, "ID:\t%d\nChain:\t%s\nMatch:\t%s%s\nAction:\t%s\n", r.ID, r.Chain, describe(r.Match), ruleExtras(r), r.Action)
+		} else {
+			fmt.Fprintf(w, "ID:\t%d\nChain:\t%s\n", r.ID, r.Chain)
+		}
 		if r.Raw != "" {
 			fmt.Fprintf(w, "Raw:\t%s\n", r.Raw)
 		}
@@ -376,6 +384,7 @@ output (from it) or early (before connection tracking: cheap drops).
 				"status": {"ct_status", r.CtStatus}, "days": {"days", r.Days}, "conn-limit": {"conn_limit", r.ConnLimit},
 				"log": {"log", r.Log}, "log-prefix": {"log_prefix", r.LogPrefix}, "raw": {"raw", r.Raw}, "comment": {"comment", r.Comment},
 				"disabled": {"disabled", r.Disabled}, "group-v4": {"group_v4", r.GroupV4}, "group-v6": {"group_v6", r.GroupV6},
+				"every-packet": {"every_packet", r.EveryPacket},
 			} {
 				if fl.Changed(flag) {
 					p[field[0].(string)] = field[1]
@@ -500,6 +509,9 @@ func ruleExtras(r model.Rule) string {
 	if r.ConnLimit > 0 {
 		x = append(x, fmt.Sprintf("over %d connections", r.ConnLimit))
 	}
+	if r.EveryPacket {
+		x = append(x, "every packet")
+	}
 	if len(x) == 0 {
 		return ""
 	}

cmd/hc/traffic.go

@@ -0,0 +1,283 @@
+package main
+
+import (
+	"context"
+	"fmt"
+	"io"
+	"os"
+	"os/signal"
+	"strconv"
+	"strings"
+	"time"
+
+	"github.com/spf13/cobra"
+
+	"housecarl/internal/api"
+)
+
+// Traffic as it happens: connections the kernel tracks, and tracing
+// packets through the ruleset.
+
+func connectionsCmd() *cobra.Command {
+	var search string
+	var limit int
+	cmd := &cobra.Command{
+		Use: "connections", Aliases: []string{"conns"}, Short: "The connections the firewall tracks, and closing them",
+		Long: `Lists the connections the kernel tracks: what the "established" built-in
+lets through without checking the rules again. A rule added later doesn't
+stop them by itself; close them, and their next packet is checked by the
+rules as it stands.
+
+  hc connections --search 192.168.1.50
+  hc connections --search ":443 tcp"
+  hc connections close --search 192.168.1.50`,
+		Args: cobra.NoArgs,
+		RunE: func(cmd *cobra.Command, args []string) error {
+			cs, err := get[[]api.Connection]("/connections", q("q", search, "limit", strconv.Itoa(limit)))
+			if err != nil {
+				return err
+			}
+			return output(cs, func(w io.Writer) {
+				fmt.Fprintln(w, "PROTO\tFROM\tTO\tNAT\tSTATE\tEXPIRES")
+				for _, c := range cs {
+					fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\t%s\n", c.Protocol, hostPort(c.Source, c.SourcePort), hostPort(c.Destination, c.DestinationPort), natText(c), c.State, fmtSeconds(c.Timeout))
+				}
+			})
+		},
+	}
+	cmd.Flags().StringVar(&search, "search", "", "only those with all these words: addresses, :port, protocol, state")
+	cmd.Flags().IntVar(&limit, "limit", 500, "at most this many")
+	var csearch string
+	var yes bool
+	closeCmd := &cobra.Command{
+		Use: "close --search TEXT", Short: "Close connections: the rules check them again with their next packet", Args: cobra.NoArgs,
+		RunE: func(cmd *cobra.Command, args []string) error {
+			if strings.TrimSpace(csearch) == "" {
+				return fmt.Errorf("say which: --search 192.168.1.50 (see 'hc connections --search' first)")
+			}
+			if !confirm("Close every connection matching "+strconvQuote(csearch)+"?", yes) {
+				return fmt.Errorf("nothing closed")
+			}
+			c, err := newClient()
+			if err != nil {
+				return err
+			}
+			var out map[string]int
+			if err := c.Do(ctx(), "POST", "/connections/close", nil, api.CloseConnections{Search: csearch}, &out); err != nil {
+				return err
+			}
+			if !flagQuiet {
+				fmt.Printf("closed %s\n", plural(out["closed"], "connection"))
+			}
+			return nil
+		},
+	}
+	closeCmd.Flags().StringVar(&csearch, "search", "", "the connections to close, as for 'hc connections --search'")
+	closeCmd.Flags().BoolVarP(&yes, "yes", "y", false, "don't ask")
+	cmd.AddCommand(closeCmd)
+	return cmd
+}
+
+func strconvQuote(s string) string { return strconv.Quote(s) }
+
+func hostPort(addr string, port int) string {
+	if port == 0 {
+		return addr
+	}
+	if strings.Contains(addr, ":") {
+		return fmt.Sprintf("[%s]:%d", addr, port)
+	}
+	return fmt.Sprintf("%s:%d", addr, port)
+}
+
+// natText: "to 192.168.1.10:443" (a port forward), "as 203.0.113.10"
+// (masqueraded).
+func natText(c api.Connection) string {
+	var parts []string
+	if c.DNATTo != "" {
+		port := c.DNATPort
+		if port == 0 {
+			port = c.DestinationPort
+		}
+		parts = append(parts, "to "+hostPort(c.DNATTo, port))
+	}
+	if c.SNATTo != "" {
+		parts = append(parts, "as "+c.SNATTo)
+	}
+	return strings.Join(parts, ", ")
+}
+
+func traceCmd() *cobra.Command {
+	var in api.TraceInput
+	cmd := &cobra.Command{
+		Use: "trace", Short: "Follow packets through every table's rules: what each met, and what decided",
+		Long: `Traces packets from or to an address, on a port (either way), through the
+whole ruleset, Housecarl's and every other table's, for a while: each
+rule and policy they meet, and what accepted or dropped them. Ctrl-C
+stops it.
+
+  hc trace --address 192.168.1.50 --port 443
+  hc trace --port 22 --seconds 120
+  hc trace --address 2001:db8:1::/64 --protocol icmpv6`,
+		Args: cobra.NoArgs,
+		RunE: func(cmd *cobra.Command, args []string) error {
+			c, err := newClient()
+			if err != nil {
+				return err
+			}
+			var st api.TraceStatus
+			if err := c.Do(ctx(), "POST", "/trace", nil, in, &st); err != nil {
+				return err
+			}
+			if !flagQuiet {
+				fmt.Fprintf(os.Stderr, "tracing until %s (Ctrl-C stops)\n", st.Ends.Local().Format("15:04:05"))
+			}
+			sig, stop := signal.NotifyContext(context.Background(), os.Interrupt)
+			defer stop()
+			shown := 0
+			for {
+				select {
+				case <-sig.Done():
+					c.Do(ctx(), "DELETE", "/trace", nil, nil, &st)
+				case <-time.After(time.Second):
+					if err := c.Do(ctx(), "GET", "/trace", nil, nil, &st); err != nil {
+						return err
+					}
+				}
+				for ; shown < len(st.Packets); shown++ {
+					printPacket(os.Stdout, st.Packets[shown])
+				}
+				if !st.Running || sig.Err() != nil {
+					break
+				}
+			}
+			if st.Error != "" {
+				return fmt.Errorf("the trace failed: %s", st.Error)
+			}
+			if !flagQuiet {
+				fmt.Fprintf(os.Stderr, "%d packets traced\n", len(st.Packets))
+			}
+			return nil
+		},
+	}
+	f := cmd.Flags()
+	f.StringVar(&in.Address, "address", "", "packets from or to this address or prefix")
+	f.IntVar(&in.Port, "port", 0, "packets from or to this port")
+	f.StringVar(&in.Protocol, "protocol", "", "tcp, udp, sctp, icmp or icmpv6 (default any)")
+	f.IntVar(&in.Seconds, "seconds", 60, "how long (5 to 600)")
+	return cmd
+}
+
+// printPacket writes a traced packet: its headers, each step, its fate.
+func printPacket(w io.Writer, p api.TracePacket) {
+	what := p.Protocol
+	if p.TCPFlags != "" {
+		what += " " + p.TCPFlags
+	}
+	if p.ICMPType != "" {
+		what += " " + p.ICMPType
+	}
+	where := ""
+	if p.In != "" {
+		where += " in " + p.In
+	}
+	if p.Out != "" {
+		where += " out " + p.Out
+	}
+	fmt.Fprintf(w, "%s  %s -> %s  %s%s\n", p.Time.Local().Format("15:04:05.000"), hostPort(p.Source, p.SourcePort), hostPort(p.Destination, p.DestinationPort), what, where)
+	for _, s := range p.Steps {
+		name := s.Label
+		if name == "" {
+			if s.Kind == "policy" {
+				name = s.Chain + " policy"
+			} else {
+				name = s.Rule
+			}
+		}
+		fmt.Fprintf(w, "    %s %s: %s -> %s\n", s.Table, s.Chain, name, s.Verdict)
+	}
+	decided := p.DecidedBy.Label
+	if decided == "" {
+		decided = strings.TrimSpace(p.DecidedBy.Table + " " + p.DecidedBy.Chain + " " + p.DecidedBy.Kind)
+	}
+	if decided == "" {
+		fmt.Fprintf(w, "    => %s\n", strings.ToUpper(p.Verdict))
+	} else {
+		fmt.Fprintf(w, "    => %s by %s\n", strings.ToUpper(p.Verdict), decided)
+	}
+}
+
+func matrixCmd() *cobra.Command {
+	return &cobra.Command{
+		Use: "matrix [FROM TO]", Short: "What the rules do to new connections between zones",
+		Long: `Shows, for each pair of zones (and the firewall itself), what happens to a
+new connection: allowed or blocked, decided by a rule or the policy, and
+how many rules decide for part of the traffic first (+N: exceptions,
+such as some services or addresses). With FROM and TO (a zone, or
+"firewall"), that pair's exceptions in order, and what decides the rest.
+
+  hc matrix
+  hc matrix guest wan
+  hc matrix lan firewall`,
+		Args: cobra.RangeArgs(0, 2),
+		RunE: func(cmd *cobra.Command, args []string) error {
+			if len(args) == 1 {
+				return fmt.Errorf("give both: FROM and TO")
+			}
+			m, err := get[api.Matrix]("/matrix", nil)
+			if err != nil {
+				return err
+			}
+			if len(args) == 2 {
+				for _, c := range m.Cells {
+					if strings.EqualFold(c.From, args[0]) && strings.EqualFold(c.To, args[1]) {
+						return output(c, func(w io.Writer) { printCell(w, c) })
+					}
+				}
+				return fmt.Errorf("no pair %s → %s (zones: %s, and firewall)", args[0], args[1], strings.Join(m.Zones, ", "))
+			}
+			return output(m, func(w io.Writer) { printMatrix(w, m) })
+		},
+	}
+}
+
+func printMatrix(w io.Writer, m api.Matrix) {
+	cols := append(append([]string{}, m.Zones...), "firewall")
+	cells := map[[2]string]api.MatrixCell{}
+	for _, c := range m.Cells {
+		cells[[2]string{c.From, c.To}] = c
+	}
+	color := isTerminal(os.Stdout) && flagOutput == "table"
+	fmt.Fprint(w, "FROM \\ TO")
+	for _, c := range cols {
+		fmt.Fprintf(w, "\t%s", c)
+	}
+	fmt.Fprintln(w)
+	for _, from := range cols {
+		fmt.Fprint(w, from)
+		for _, to := range cols {
+			c, ok := cells[[2]string{from, to}]
+			text := "-"
+			if ok {
+				text = map[string]string{"allowed": "allow", "blocked": "block"}[c.Verdict]
+				if n := len(c.Exceptions); n > 0 {
+					text += fmt.Sprintf(" +%d", n)
+				}
+				if color {
+					text = map[string]string{"allowed": "\x1b[32m", "blocked": "\x1b[31m"}[c.Verdict] + text + "\x1b[0m"
+				}
+			}
+			fmt.Fprintf(w, "\t%s", text)
+		}
+		fmt.Fprintln(w)
+	}
+	fmt.Fprintln(w, "\nallow/block: new connections no exception covers; +N: rules deciding for part of the traffic first ('hc matrix FROM TO')")
+}
+
+func printCell(w io.Writer, c api.MatrixCell) {
+	fmt.Fprintf(w, "%s → %s (the %s chain), new connections:\n", c.From, c.To, c.Chain)
+	for _, e := range c.Exceptions {
+		fmt.Fprintf(w, "  %s\t%s\t%s\n", e.Verdict, e.What, e.Label)
+	}
+	fmt.Fprintf(w, "  %s\tthe rest\t%s\n", c.Verdict, c.DecidedBy)
+}

cmd/housecarld/main.go

@@ -26,6 +26,7 @@ import (
 	"housecarl/internal/auth"
 	"housecarl/internal/certs"
 	"housecarl/internal/config"
+	"housecarl/internal/conntrack"
 	"housecarl/internal/engine"
 	"housecarl/internal/hostif"
 	"housecarl/internal/jarl"
@@ -35,6 +36,7 @@ import (
 	"housecarl/internal/render"
 	"housecarl/internal/server"
 	"housecarl/internal/store"
+	"housecarl/internal/trace"
 	"housecarl/internal/validate"
 	"housecarl/internal/version"
 	"housecarl/internal/web"
@@ -195,6 +197,9 @@ func serveCmd() *cobra.Command {
 			}
 			go nflog.Listen(ctx, render.LogGroup, a.log, srv.LogPacket)
 			go srv.RunQuotas(ctx)
+			srv.SetConntrack(conntrack.Client{})
+			// A trace left behind by a crash.
+			fw.DeleteTable(ctx, "inet", trace.Table)
 			hup := make(chan os.Signal, 1)
 			signal.Notify(hup, syscall.SIGHUP)
 			go func() {

docs/api.md

@@ -51,6 +51,10 @@ Calls that change the configuration answer:
 * `confirm_by`: the change is on probation, and is rolled back at that
   time unless `POST /confirm` comes first. Missing for changes made
   through the socket.
+* `confirm_token`: the client that made the change can confirm it by
+  itself with `POST /confirm {"token"}`, from the same address, as the
+  same user and token, over a new connection (the answer closes this
+  one). Missing when a person must confirm (`manual_confirm`).
 * `?dry_run=1` checks the change, with nftables too, and returns what it
   would do, with `"applied": false, "dry_run": true`, and `diff`: the
   ruleset's unified diff.
@@ -73,9 +77,15 @@ Creating answers 201, other changes 200.
 | `GET /healthz` | `{"ok": true, "version"}`, without authentication |
 | `GET /status` | `build`, `started`, `applied` (the ruleset is in force), `apply_error`, `problems`, `probation`, `interfaces`, `counts` (objects by kind), `other_tables` (other nftables tables, `"inet main"`) |
 | `GET /interfaces` | the machine's interfaces: `name`, `up`, `addresses`, `zone` |
+| `GET /matrix[?changeset=ID]` | the zone matrix: `zones`, `cells` (`from`, `to` (a zone, or `firewall`), `chain`, `verdict` (allowed, blocked), `decided_by`, `rule_id`, `exceptions`: `rule_id`, `chain`, `label`, `verdict` (allowed, blocked, logged, unknown), `what`) ([rules.md](rules.md#the-zone-matrix)) |
 | `GET /ruleset` | the nftables ruleset Housecarl builds, as text |
 | `GET /counters` | `{"since", "counters": {"r12": {"packets", "bytes"}, ...}}`, by mark ([design.md](design.md#its-own-table-only)); totals since housecarld started |
 | `GET /quotas` | each rule's quota: `rule_id`, `rule`, `quota`, `used` (bytes), `addresses` (`[{"address", "used"}]` for a per-address quota), `last_reset`, `next_reset` |
+| `GET /connections[?q=&limit=]` | tracked connections: `family`, `protocol`, `source`, `source_port`, `destination`, `destination_port` (or `icmp_id`, `icmp_type`, `icmp_code`), `dnat_to`, `dnat_port`, `snat_to`, `state`, `assured`, `timeout` (seconds left), `packets` and `bytes` when the kernel counts them. `q`: all of its words in the addresses, `:port`, protocol, state |
+| `POST /connections/close` | `{"connections": [...]}` (as listed) or `{"search": "192.168.1.50"}`: delete their tracking entries (`config:write`, audited); answers `{"closed": N}` |
+| `POST /trace` | `{"address", "port", "protocol", "seconds"}` (5-600, default 60): start tracing (`config:write`, audited); 409 while one runs |
+| `GET /trace` | the trace running, or the last: `running`, `input`, `started`, `ends`, `error`, `packets`: each with `time`, `in`, `out`, `protocol`, addresses and ports, `tcp_flags`, `icmp_type`, `steps` (`table`, `chain`, `kind` rule or policy, `rule`, `mark`, `label`, `verdict`), `verdict` (accepted, dropped) and `decided_by` |
+| `DELETE /trace` | stop it |
 | `GET /log[?limit=&after=&q=]` | logged packets, newest first: `time`, `prefix`, `in`, `out`, `family`, `protocol`, `source`, `source_port`, `destination`, `destination_port`, `icmp_type`, `tcp_flags`, `length`, `mark` (the rule's). `after` (a time) for those since; `q` searches |
 | `GET /catalog` | built-in services, chains, actions, roles, the log group |
 
@@ -89,7 +99,7 @@ Creating answers 201, other changes 200.
  "builtins": {"loopback": true, "established": true, "drop_invalid": true, "icmpv6": true,
               "ping_internal": true, "dhcpv6_client": true, "antispoof": true, "mss_clamp": false},
  "own_prefixes": ["2001:db8::/48"], "antispoof_v4": ["10.0.0.0/8", "..."], "antispoof_v6": ["fc00::/7", "..."],
- "flowtable_devices": ["wan", "eth1"], "probation_seconds": 120}
+ "flowtable_devices": ["wan", "eth1"], "probation_seconds": 120, "manual_confirm": false}
 ```
 
 Lists replace the old ones. `probation_seconds` 0 is the default (120).
@@ -153,7 +163,10 @@ Changes staged to be reviewed and applied together ([design.md](design.md#review
 Staging (`?changeset=ID` on any change to objects or settings) checks
 the changeset with the new operation, and nftables checks its ruleset;
 an error refuses the operation. Only open changesets take operations, and
-only their owner's (or an administrator's). Undo, `/import/nft` and
+only their owner's (or an administrator's). An undo
+(`POST /history/{id}/revert?changeset=ID`) is staged as one operation,
+which checks when it is replayed that nothing it touches changed since;
+applied, History links it to the entry it undoes. `/import/nft` and
 `/takeover` can't be staged: with a changeset named they answer 400.
 
 Reads of objects, settings and `/ruleset` with `?changeset=ID` show the
@@ -180,7 +193,7 @@ history, but are audited.
 | | |
 |---|---|
 | `GET /probation` | `{"pending": [history IDs], "takeovers": N, "deadline", "seconds_left"}` |
-| `POST /confirm` | keep them: `{"confirmed": N}`; the ruleset is saved for the boot |
+| `POST /confirm` | keep them: `{"confirmed": N}`; the ruleset is saved for the boot. With `{"token"}`, the client confirming its own change (see `confirm_token`): 409 if a newer change is pending, or over the change's own connection; 403 from another user or address |
 | `POST /rollback` | undo them now |
 
 ### History, import, takeover

docs/cli.md

@@ -28,6 +28,7 @@ command and its flags.
 | | `HOUSECARL_SOCKET` | the local socket's path |
 | `--insecure`, `--ca-cert FILE` | | TLS: skip the check, or trust another CA |
 | `-n, --dry-run` | | check and show a change without making it, with the ruleset's diff |
+| `--no-confirm` | | leave a change on probation instead of confirming it once `hc` reaches Housecarl again |
 | `-C, --changeset ID` | `HOUSECARL_CHANGESET` (checked, see [Changesets](#changesets)) | stage configuration changes in this changeset instead of making them |
 | `-o, --output FORMAT` | | `table` (default), `json`, `yaml` or `id` |
 | `-q, --quiet` | | print only errors (and the IDs of new objects) |
@@ -41,14 +42,14 @@ server error.
 Every change is checked as a whole, then by nftables, before it is put
 in force. A refused change says why, with the problem's code
 ([validation.md](validation.md)). An applied one says its history entry
-and any new warnings. Over the network, it is on probation:
+and any new warnings. Over the network, it is on probation, and `hc`
+confirms it by reaching Housecarl again over a new connection
+([operations.md](operations.md#changes-on-probation)):
 
 ```
 $ hc rule add input --from trusted --service ssh --comment "SSH from home"
 created rule input #31 (from trusted · ssh)
-applied on probation (history 12): confirm with 'hc confirm' within 2m0s, or it is rolled back
+applied and confirmed (history 12): this client reached Housecarl again over a new connection
-$ hc confirm
-confirmed 1 change; it is kept and loaded at boot
 ```
 
 Through the socket the change is kept at once:
@@ -116,9 +117,9 @@ it, because an exported ID easily outlives its changeset:
 `-C ID` and an ID argument are used as given (housecarld still refuses a
 changeset that isn't open, or another user's unless you are an
 administrator). Only configuration changes are staged: dynamic set
-entries, `confirm` and `rollback` are made at once, and `history undo`,
+entries, `confirm` and `rollback` are made at once; `history undo` is
-`import` and `takeover` can't be staged, so they are refused while a
+staged like any change; `import` and `takeover` can't be staged, so they
-changeset is named. Changesets staged in the web UI show up here too,
+are refused while a changeset is named. Changesets staged in the web UI show up here too,
 and the other way round.
 
 ## Commands
@@ -136,6 +137,9 @@ by name.
 | `counters` | the hits of every rule, NAT rule, built-in and policy, by mark (`r12`, `f3`, `b-established`, `p-input`) |
 | `quotas` | what each rule's quota used (each address's, for per-address quotas), and when it resets |
 | `log [-f] [--search TEXT] [--limit N]` | the packets logged by rules and policies ([operations.md](operations.md#the-log)) |
+| `connections [--search TEXT] [--limit N]`, `connections close --search TEXT [-y]` | tracked connections; closing makes the rules check them again ([operations.md](operations.md#connections)) |
+| `matrix [FROM TO]` | what the rules do to new connections between zones; with FROM and TO, that pair's rules in order ([rules.md](rules.md#the-zone-matrix)) |
+| `trace [--address A] [--port N] [--protocol P] [--seconds S]` | follow packets through every table's rules ([operations.md](operations.md#tracing)) |
 
 ### zone, set, service
 
@@ -181,7 +185,7 @@ The match flags (`rule add`, `snat add`, `redirect add`):
 And for rules: `--state`, `--status`, `--days`, `--hours`, `--limit
 10/minute|1mbytes/second[,burst=N][,over][,per-source|,per-destination]`,
 `--quota 50gbytes[,over][,per-source|,per-destination][,daily|,weekly|,monthly=DAY]`,
-`--group-v4 N`, `--group-v6 N`, `--conn-limit N`, `-a
+`--group-v4 N`, `--group-v6 N`, `--conn-limit N`, `--every-packet`, `-a
 accept|drop|reject|log`, `--reject-with tcp-reset`, `--log`,
 `--log-prefix`, `--raw RULE`, `--comment`, `--disabled`
 ([rules.md](rules.md#rules)).
@@ -203,7 +207,7 @@ See [nat.md](nat.md).
 | Command | |
 |---|---|
 | `settings show` | |
-| `settings set [--input-policy\|--forward-policy\|--output-policy accept\|drop] [--log-drops on\|off] [--builtin NAME=on\|off]... [--own-prefix P]... [--antispoof-v4 P]... [--antispoof-v6 P]... [--flowtable DEV]... [--probation 2m]` | lists given replace the old ones |
+| `settings set [--input-policy\|--forward-policy\|--output-policy accept\|drop] [--log-drops on\|off] [--builtin NAME=on\|off]... [--own-prefix P]... [--antispoof-v4 P]... [--antispoof-v6 P]... [--flowtable DEV]... [--probation 2m] [--auto-confirm on\|off]` | lists given replace the old ones |
 
 ### probation, history, import
 

docs/design.md

@@ -75,8 +75,9 @@ to be reviewed and applied together with others:
 | CLI | Immediate: each command is a change of its own. Stage with `-C ID` or `HOUSECARL_CHANGESET`. | Scripts want a clear result per step. |
 | API | Immediate unless a changeset is named (`?changeset=ID`, `X-Housecarl-Changeset`). | As the CLI. |
 
-Undo, imports and takeovers can't be staged; with a changeset named, they
+An undo is staged like any edit (in review mode, the web UI's Undo
-are refused rather than applied.
+collects it for review). Imports and takeovers can't be staged; with a
+changeset named, they are refused rather than applied.
 
 ## Its own table only
 
@@ -107,6 +108,19 @@ confirmed within two minutes (Settings; `hc confirm`, the banner in the
 web UI, `POST /confirm`), it is rolled back by itself, and the rollback is
 recorded in the history. A change that cuts you off undoes itself.
 
+* **The client that made a change confirms it by itself** once it
+  reaches the API again: the change's answer carries a one-time token and
+  closes its connection (HTTP/2: a GOAWAY), and the web UI and `hc` send
+  the token straight back. That proves something only on a new
+  connection, since the built-in that accepts established connections
+  would let the old one through whatever the change did. So the token is
+  accepted only from the same user (and API token), the same address, a
+  different connection, and for the newest change. A change that cuts its
+  client off can't be confirmed by it, and rolls back. This proves the
+  API is still reachable, not that SSH or the LAN still work; with the
+  API, you can fix those. Settings → "a person confirms each change"
+  (`hc settings set --auto-confirm off`) goes back to confirming by hand;
+  `hc --no-confirm` leaves one change waiting.
 * Changes made through the local socket (`sudo hc …` on the firewall)
   need no confirmation: whoever makes them is already on the machine. One
   made while others are pending joins them.

docs/import.md

@@ -107,7 +107,10 @@ the old table:
 sudo hc takeover "inet main"
 ```
 
-From now on only Housecarl's rules apply. Over the network, this is on
+From now on only Housecarl's rules apply. Until you do, the dashboard
+and `hc status` warn that the imported table still applies
+(`takeover_pending`); they also warn while `nftables.service` is enabled
+(`nftables_service`). Over the network, this is on
 probation too: unless confirmed, the old table comes back.
 
 ## 4. The boot
@@ -122,6 +125,11 @@ sudo systemctl disable nftables.service
 sudo mv /etc/nftables.conf /etc/nftables.conf.before-housecarl
 ```
 
+Disable it, but **don't stop it** (no `systemctl stop nftables`, no
+`disable --now`): Debian's unit stops by running `nft flush ruleset`,
+which deletes every table, Housecarl's and Docker's included, at once.
+Disabled, it simply doesn't run at the next boot.
+
 `housecarl-ruleset.service` runs after `nftables.service` in any case,
 so a leftover `flush ruleset` can't undo it. Docker, Incus and libvirt
 keep adding their own tables as before.

docs/operations.md

@@ -28,7 +28,25 @@ edits) applies each edit at once instead. With hc, see
 ## Changes on probation
 
 Changes made over the network roll back unless confirmed
-([design.md](design.md#not-locking-yourself-out)):
+([design.md](design.md#not-locking-yourself-out)). The web UI and `hc`
+confirm their own changes once they reach Housecarl again over a new
+connection, so normally nothing waits:
+
+```
+$ hc rule add input --from admins --service ssh
+created rule input #31
+applied and confirmed (history 12): this client reached Housecarl again over a new connection
+```
+
+A change that cuts the client off can't be confirmed that way:
+
+```
+applied on probation (history 13), but couldn't confirm it: … context deadline exceeded
+It is rolled back within 1m50s unless confirmed ('hc confirm')
+```
+
+By hand (with `--no-confirm`, or with Settings → "a person confirms each
+change"):
 
 ```sh
 hc status              # Probation: 2 changes roll back in 97s unless confirmed
@@ -62,10 +80,21 @@ sudo nft -c -f /var/lib/housecarl/ruleset.nft     # check it
 
 Dynamic sets start empty after a reboot.
 
-Keep Debian's `nftables.service` disabled, or `/etc/nftables.conf`
+Keep Debian's `nftables.service` disabled (`systemctl disable
-without `flush ruleset`: otherwise it deletes Housecarl's table at
+nftables`, without `--now`): at boot it loads `/etc/nftables.conf`, and
-boot. `housecarl-ruleset.service` runs after it, which covers the
+stopping or restarting it runs `nft flush ruleset`, which deletes every
-ordinary case.
+table, Housecarl's included. `housecarl-ruleset.service` runs after it
+at boot, which covers the ordinary case, but not a restart of
+`nftables.service` later. Housecarl never reads or writes
+`/etc/nftables.conf`; its ruleset is saved in
+`/var/lib/housecarl/ruleset.nft`. The dashboard and `hc status` warn
+while `nftables.service` is enabled.
+
+Other programs' tables are left alone: fail2ban's (`inet f2b-table`),
+Docker's, Incus's. Housecarl replaces only its own table, so they keep
+their rules through every Housecarl change and restart, and nothing
+needs restarting in a particular order. A packet must get past every
+table on its hook, so a drop in any of them wins.
 
 ## Hits
 
@@ -104,16 +133,62 @@ the last 2000 are kept in memory. They aren't written to the kernel log
 or the journal. A rule's log isn't rate-limited: on a busy rule, give it
 a limit, or log only new connections (`--state new`).
 
+## Connections
+
+The built-in that accepts established connections lets their packets
+through without checking the rules again: a rule added later doesn't
+stop a connection already open. Traffic → Connections (`hc connections`)
+lists what the kernel tracks, with NAT (`to 192.168.1.10:443`: a port
+forward; `as 203.0.113.10`: masqueraded) and the time left:
+
+```sh
+hc connections --search 192.168.1.50
+hc connections --search ":443 tcp"
+hc connections close --search 192.168.1.50
+```
+
+Closing a connection deletes its tracking entry: its next packet is
+checked by the rules as they are now. One they block is stopped; one they
+allow carries on (TCP connections are picked up again mid-stream).
+
+To make a block stop open connections by itself, tick "every packet" on
+the rule (`--every-packet`): it is checked before established connections
+are accepted, like rules with a quota ([rules.md](rules.md#rules)).
+
+## Tracing
+
+Traffic → Trace (`hc trace`) follows packets through the whole ruleset:
+Housecarl's rules and every other table (Docker's, fail2ban's, an old
+`inet main`), each rule and policy they meet, and what accepted or
+dropped them:
+
+```
+$ hc trace --address 192.168.1.50 --port 443 --seconds 60
+14:02:13.120  192.168.1.50:51432 -> 198.51.100.9:443  tcp syn in eth1 out wan
+    inet housecarl forward: built-in: mss-clamp -> continue
+    inet housecarl forward: rule forward #12 (kids at night) -> drop
+    => DROPPED by rule forward #12 (kids at night)
+```
+
+It traces packets from or to the address, on the port (either way), for
+5 to 600 seconds, at most 200 packets and 20 a second. It works with
+nftables' own tracing: a table of its own (`inet housecarl-trace`,
+hooked before everything else) marks the packets, and `nft monitor
+trace` reports them; the table goes when the trace ends. One trace runs
+at a time.
+
 ## When something doesn't get through
 
-1. `hc log -f` with `--log-drops on`: is it dropped by a policy (no rule
+1. Trace it (above): it names the rule or policy that dropped it, in
+   whichever table.
+2. `hc log -f` with `--log-drops on`: is it dropped by a policy (no rule
    accepted it), or by a rule (its number is in the line)?
-2. `hc rule list forward` (or `input`): hits on the rule you expect?
+3. `hc rule list forward` (or `input`): hits on the rule you expect?
-3. `hc ruleset`: the rule as nftables sees it, with the zones' interfaces
+4. `hc ruleset`: the rule as nftables sees it, with the zones' interfaces
    and the sets' families.
-4. `hc status`: other tables. A Docker or Incus table, or an old one,
+5. `hc status`: other tables. A Docker or Incus table, or an old one,
    can drop what Housecarl accepts.
-5. Port forwards: `hc forward list` hits, and the inside host's own
+6. Port forwards: `hc forward list` hits, and the inside host's own
    firewall and default route (answers must go back through Housecarl).
 
 ## Backups

docs/permissions.md

@@ -7,9 +7,9 @@ else gets **grants**, `resource:access`:
 
 | Resource | read | write |
 |---|---|---|
-| `config` | settings, zones, sets, services, rules, NAT, the ruleset, history, interfaces, probation, changesets (the web UI shows the forms locked) | change them, stage and apply changesets (your own, or anyone's for administrators), undo, import, confirm and roll back |
+| `config` | settings, zones, sets, services, rules, NAT, the ruleset, history, interfaces, probation, changesets (the web UI shows the forms locked) | change them, stage and apply changesets (your own, or anyone's for administrators), undo, import, confirm and roll back; close connections; start and stop traces |
 | `sets` | dynamic sets' entries | add and remove entries |
-| `log` | hit counters and the log | – |
+| `log` | hit counters, the log, connections, traces | – |
 | `audit` | the audit log | – |
 | `*` | all of the above | |
 

docs/rules.md

@@ -38,7 +38,10 @@ directly (`--from 192.168.1.3`); a set is better when several rules share
 them.
 
 A **dynamic** set's entries are added at run time, each with a timeout,
-and nftables removes them when it runs out, even with housecarld stopped:
+and nftables removes them when it runs out, even with housecarld stopped.
+It has no configured entries (validation refuses them); its timeout is
+the default for entries added without one, and without a timeout they
+stay until removed:
 
 ```sh
 hc set add guests --dynamic --timeout 24h
@@ -116,6 +119,10 @@ The action is `accept`, `drop`, `reject` (an ICMP error, or with
 `--reject-with tcp-reset` a TCP reset) or `log` (log and go on).
 `--log` logs the packets of any rule ([operations.md](operations.md#the-log)).
 `--disabled` (or `hc rule disable ID`) keeps a rule without using it.
+`--every-packet` ("every packet" in the web UI) checks a drop or reject
+against every packet, before established connections are accepted, so it
+also stops connections already open, not just new ones; it can't accept
+([operations.md](operations.md#connections)).
 
 Examples:
 
@@ -193,8 +200,11 @@ hand, placed like any other:
 hc rule add forward --raw 'iifname != "wan" ct state invalid ip daddr @trusted_v4 accept'
 ```
 
-It is checked by nftables with the rest of the ruleset, and can use the
+It is used exactly as written: its text replaces the rule's zones,
-sets' nftables names (`@trusted_v4`). The importer makes raw rules from
+addresses, services, action, limits and log, so a raw rule with any of
+those set is refused (`raw_conflict`); write them in the text. It is
+checked by nftables with the rest of the ruleset, and can use the sets'
+nftables names (`@trusted_v4`). The importer makes raw rules from
 what it can't translate. It goes into the ruleset as written: add
 `counter` to it to count its packets.
 
@@ -225,6 +235,47 @@ TCP and UDP connections forwarded between those devices skip the rules,
 which helps on a slow CPU. It needs exact device names. Counters of
 forward rules then only count each connection's first packets.
 
+## The zone matrix
+
+The Matrix page (`hc matrix`) sums up what the rules do to **new
+connections** between zones: a row per zone the traffic comes from, a
+column per zone it goes to, and the firewall itself (to it: the input
+chain; from it: output; zone to zone: forward).
+
+```
+$ hc matrix
+FROM \ TO  guest     lan       wan       firewall
+guest      block     block     block +1  block +3
+lan        block     block     allow     block +4
+wan        block +2  block +2  block +1  block +3
+firewall   allow     allow     allow     -
+```
+
+Each cell walks that pair's rules in order, as nftables would, with the
+built-ins that act on new connections (ping from internal zones,
+ICMPv6, the DHCPv6 client, antispoofing, port forwards):
+
+* a rule that applies to the whole pair decides it ("allow", "block");
+* a rule that applies to part of its traffic (some addresses, services,
+  times, rates, a quota) is an **exception** (+N), and the walk goes on;
+* what's left falls to the deciding rule or the chain's policy.
+
+`hc matrix guest wan` (or clicking the cell) lists the exceptions in
+order:
+
+```
+guest → wan (the forward chain), new connections:
+  allowed  http, https   rule forward #14
+  blocked  the rest      forward policy
+```
+
+A rule whose addresses all lie in networks directly connected to other
+zones is left out of a pair: traffic between those zones can't have them
+(`from wg_net` doesn't concern traffic coming in on the LAN). Routed
+networks, mixed lists and dynamic sets might, so they stay. Replies and
+established connections aren't shown: the built-in lets them through. In
+review mode, the matrix shows your pending changes.
+
 ## The ruleset
 
 `hc ruleset` prints what Housecarl builds; the web UI and `hc rule list`

docs/validation.md

@@ -51,6 +51,7 @@ are unique per kind, ignoring case.
 |---|---|---|
 | `set_entry` | error | an entry that isn't an address, prefix (without host bits) or range |
 | `set_empty` | warning | a set (not dynamic) with no entries: rules using it match nothing |
+| `set_dynamic_entries` | error | configured entries on a dynamic set: its entries come at run time |
 | `set_timeout` | error | a timeout on a set that isn't dynamic, or under a second |
 | `service_empty` | error | a service with no entries |
 | `service_entry` | error | an entry that isn't `PROTO/PORTS` (`tcp`, `udp`, `sctp`; a port or range) or `icmp/TYPE`, `icmpv6/TYPE` with a type nftables knows |
@@ -77,6 +78,7 @@ These apply to rules, source NAT and redirects.
 | `rule_chain` | error | a chain that isn't `input`, `forward`, `output` or `early` |
 | `rule_action` | error | an action that isn't `accept`, `drop`, `reject` or `log` |
 | `raw_rule` | error | a raw rule over several lines, or with `;` |
+| `raw_conflict` | error | a raw rule with other fields set (zones, addresses, services, limits, log…): it is used as written, so they would be ignored |
 | `early_conntrack` | error | connection states, limits or a connection limit in the early chain, which runs before connection tracking |
 | `early_reject` | error | reject in the early chain: drop instead |
 | `ct_state`, `ct_status` | error | an unknown connection state or status |
@@ -91,7 +93,8 @@ These apply to rules, source NAT and redirects.
 | `rule_quota_reset` | error | a reset that isn't daily, weekly or monthly; a day with another reset than monthly; a day outside 1-31 |
 | `rule_group` | error | a grouping prefix outside 1-32 (IPv4) or 1-128 (IPv6) |
 | `rule_traffic_accept` | error | a rule with a quota or a byte rate that accepts: it is checked before established connections are accepted, so it can only drop, reject or log |
-| `flowtable_traffic` | warning | a forward rule with a quota or a byte rate while the flowtable is on: it only counts each connection's first packets |
+| `rule_every_packet_accept` | error | a rule checked against every packet (`every_packet`) that accepts: it comes before established connections are accepted, so it can only drop, reject or log |
+| `flowtable_traffic` | warning | a forward rule checked against every packet (a quota, a byte rate, `every_packet`) while the flowtable is on: it only sees each connection's first packets |
 | `reject_with` | error | `tcp-reset` on a rule that matches more than TCP, or with an action other than reject |
 | `log_prefix` | error | a log prefix over 100 characters |
 
@@ -115,3 +118,13 @@ These apply to rules, source NAT and redirects.
 | `snat_everything` | warning | no out zones and no sources: everything leaving the firewall is translated |
 | `redirect_service` | error | no service, or one without TCP, UDP or SCTP ports |
 | `redirect_to_port` | error | the local port isn't one port |
+
+## The machine
+
+These come from housecarld looking at the machine, not at the
+configuration: the dashboard and `hc status` show them.
+
+| Code | | When |
+|---|---|---|
+| `nftables_service` | warning, *host* | Debian's `nftables.service` is enabled: at boot it loads `/etc/nftables.conf`, and stopping or restarting it runs `nft flush ruleset`, deleting Housecarl's table too. Disable it with `systemctl disable nftables` (without `--now`) |
+| `takeover_pending` | warning, *host* | the table you imported from is still in force beside Housecarl's, so a packet must get past both; `hc takeover TABLE` deletes it once Housecarl's rules are right |

internal/api/types.go

@@ -66,6 +66,11 @@ type MutationResponse struct {
 	// ConfirmBy is set when the change is on probation: unless confirmed
 	// (POST /confirm) by then, it is rolled back.
 	ConfirmBy *time.Time `json:"confirm_by,omitempty"`
+	// ConfirmToken lets the client that made the change confirm it by
+	// itself (POST /confirm {"token"}): from the same address, over a new
+	// connection, which proves it can still reach the API. The answer
+	// closes its connection, so the next request opens a new one.
+	ConfirmToken string `json:"confirm_token,omitempty"`
 	// Diff is the ruleset's unified diff, for a dry run.
 	Diff string `json:"diff,omitempty"`
 	// Staged: the change was added to a changeset (Changeset), not
@@ -318,3 +323,137 @@ type QuotaAddress struct {
 	Address string `json:"address"`
 	Used    int64  `json:"used"`
 }
+
+// ConfirmInput confirms the changes on probation. With Token (a change's
+// confirm_token), it is the client that made the change confirming it by
+// itself; without, a person's confirmation.
+type ConfirmInput struct {
+	Token string `json:"token,omitempty"`
+}
+
+// Connection is a connection the kernel tracks (conntrack), as it was
+// first seen: Source:SourcePort → Destination:DestinationPort. NAT shows
+// as DNATTo (where it was really sent: a port forward) and SNATTo (the
+// source it went out with: masquerading).
+type Connection struct {
+	ID              uint32 `json:"id,omitempty"`
+	Family          string `json:"family"`
+	Protocol        string `json:"protocol"`
+	Source          string `json:"source"`
+	SourcePort      int    `json:"source_port,omitempty"`
+	Destination     string `json:"destination"`
+	DestinationPort int    `json:"destination_port,omitempty"`
+	ICMPID          int    `json:"icmp_id,omitempty"`
+	ICMPType        int    `json:"icmp_type,omitempty"`
+	ICMPCode        int    `json:"icmp_code,omitempty"`
+	DNATTo          string `json:"dnat_to,omitempty"`
+	DNATPort        int    `json:"dnat_port,omitempty"`
+	SNATTo          string `json:"snat_to,omitempty"`
+	// State is TCP's (established, time-wait, …); Assured: traffic went
+	// both ways.
+	State   string `json:"state,omitempty"`
+	Assured bool   `json:"assured,omitempty"`
+	// Timeout is the seconds left until it is forgotten unless more
+	// traffic comes.
+	Timeout int    `json:"timeout"`
+	Mark    uint32 `json:"mark,omitempty"`
+	// Packets and Bytes, both ways, when the kernel counts them
+	// (net.netfilter.nf_conntrack_acct).
+	Packets uint64 `json:"packets,omitempty"`
+	Bytes   uint64 `json:"bytes,omitempty"`
+}
+
+// CloseConnections closes connections: these, or every one Search finds
+// (as GET /connections?q= does).
+type CloseConnections struct {
+	Connections []Connection `json:"connections,omitempty"`
+	Search      string       `json:"search,omitempty"`
+}
+
+// TraceInput says what to trace: packets from or to Address (an address
+// or prefix), on Port (either way), of Protocol, for Seconds (0: 60).
+type TraceInput struct {
+	Address  string `json:"address,omitempty"`
+	Port     int    `json:"port,omitempty"`
+	Protocol string `json:"protocol,omitempty"`
+	Seconds  int    `json:"seconds,omitempty"`
+}
+
+// TraceStep is one rule or policy a traced packet met, in any table.
+type TraceStep struct {
+	Table string `json:"table"` // "inet housecarl", "ip nat"…
+	Chain string `json:"chain"`
+	Kind  string `json:"kind"` // rule or policy
+	// Rule is the rule as nftables writes it; Mark its Housecarl mark
+	// ("r12", "b-established"), if it is one of Housecarl's.
+	Rule    string `json:"rule,omitempty"`
+	Mark    string `json:"mark,omitempty"`
+	Verdict string `json:"verdict"` // accept, drop, continue, jump X…
+	// Label names Housecarl's object ("rule input #12 (ssh)").
+	Label string `json:"label,omitempty"`
+}
+
+// TracePacket is one traced packet: its headers as it arrived, every
+// step, and its fate.
+type TracePacket struct {
+	ID              string      `json:"id"`
+	Time            time.Time   `json:"time"`
+	In              string      `json:"in,omitempty"`
+	Out             string      `json:"out,omitempty"`
+	Protocol        string      `json:"protocol,omitempty"`
+	Source          string      `json:"source,omitempty"`
+	SourcePort      int         `json:"source_port,omitempty"`
+	Destination     string      `json:"destination,omitempty"`
+	DestinationPort int         `json:"destination_port,omitempty"`
+	TCPFlags        string      `json:"tcp_flags,omitempty"`
+	ICMPType        string      `json:"icmp_type,omitempty"`
+	Steps           []TraceStep `json:"steps"`
+	// Verdict is accepted or dropped; DecidedBy the step that decided.
+	Verdict   string    `json:"verdict"`
+	DecidedBy TraceStep `json:"decided_by"`
+}
+
+// TraceStatus is the trace running (or the last one).
+type TraceStatus struct {
+	Running bool          `json:"running"`
+	Input   TraceInput    `json:"input"`
+	Started time.Time     `json:"started"`
+	Ends    time.Time     `json:"ends"`
+	Packets []TracePacket `json:"packets"`
+	Error   string        `json:"error,omitempty"`
+}
+
+// Matrix is what the rules do to new connections between zones:
+// zone → zone (the forward chain), zone → the firewall (input) and the
+// firewall → zone (output). "firewall" stands for the machine itself.
+type Matrix struct {
+	Zones []string     `json:"zones"`
+	Cells []MatrixCell `json:"cells"`
+}
+
+// MatrixCell is one pair: what happens to new connections from From to
+// To that no exception covers (Verdict, decided by DecidedBy), and the
+// exceptions: rules and built-ins that decide for some of them (some
+// addresses, services, times…), in the order they are checked.
+type MatrixCell struct {
+	From       string            `json:"from"`
+	To         string            `json:"to"`
+	Chain      string            `json:"chain"`
+	Verdict    string            `json:"verdict"` // allowed or blocked
+	DecidedBy  string            `json:"decided_by"`
+	RuleID     int64             `json:"rule_id,omitempty"` // the deciding rule
+	Exceptions []MatrixException `json:"exceptions"`
+}
+
+// MatrixException is a rule (or built-in) that decides for part of a
+// pair's traffic.
+type MatrixException struct {
+	RuleID int64  `json:"rule_id,omitempty"` // 0: a built-in, or port forwards
+	Chain  string `json:"chain"`
+	Label  string `json:"label"`
+	// Verdict is allowed, blocked, logged (and goes on), or unknown (a raw
+	// rule Housecarl can't read).
+	Verdict string `json:"verdict"`
+	// What it covers: "from trusted · ssh", "Monday 08:00–17:00".
+	What string `json:"what"`
+}

internal/client/client.go

@@ -39,9 +39,9 @@ type Client struct {
 }
 
 // Staged reports whether a request is a configuration change that a
-// changeset takes: objects and settings, and the changes that can't be
+// changeset takes: objects, settings and undo, and the changes that can't
-// staged (undo, import, takeover), which the server then refuses rather
+// be staged (import, takeover), which the server then refuses rather than
-// than apply them. Changesets' own calls and dynamic set entries aren't.
+// apply them. Changesets' own calls and dynamic set entries aren't.
 func Staged(method, path string) bool {
 	if method == http.MethodGet || strings.Contains(path, "/elements") {
 		return false

internal/conntrack/conntrack.go

@@ -0,0 +1,351 @@
+// Package conntrack lists and closes the kernel's tracked connections,
+// through ctnetlink (the netfilter netlink subsystem for connection
+// tracking): what the "established" built-in lets through, and what a
+// change of rules doesn't stop by itself.
+//
+// Closing a connection deletes its tracking entry. Its next packet is
+// then checked by the rules again, as a new connection would be: one the
+// rules now block is stopped, one they allow carries on (TCP connections
+// are picked up again mid-stream).
+package conntrack
+
+import (
+	"context"
+	"encoding/binary"
+	"errors"
+	"fmt"
+	"net/netip"
+	"sync/atomic"
+	"syscall"
+	"time"
+
+	"golang.org/x/sys/unix"
+
+	"housecarl/internal/api"
+)
+
+const (
+	subsysCtnetlink = 1
+	msgNew          = 0
+	msgGet          = 1
+	msgDelete       = 2
+
+	ctaTupleOrig     = 1
+	ctaTupleReply    = 2
+	ctaStatus        = 3
+	ctaProtoinfo     = 4
+	ctaTimeout       = 7
+	ctaMark          = 8
+	ctaCountersOrig  = 9
+	ctaCountersReply = 10
+	ctaID            = 12
+
+	ctaTupleIP    = 1
+	ctaTupleProto = 2
+
+	ctaIPv4Src = 1
+	ctaIPv4Dst = 2
+	ctaIPv6Src = 3
+	ctaIPv6Dst = 4
+
+	ctaProtoNum      = 1
+	ctaProtoSrcPort  = 2
+	ctaProtoDstPort  = 3
+	ctaProtoICMPID   = 4
+	ctaProtoICMPType = 5
+	ctaProtoICMPCode = 6
+	ctaProtoICMP6ID  = 7
+	ctaProtoICMP6Typ = 8
+	ctaProtoICMP6Cod = 9
+
+	ctaProtoinfoTCP      = 1
+	ctaProtoinfoTCPState = 1
+
+	ctaCountersPackets = 1
+	ctaCountersBytes   = 2
+
+	statusAssured = 1 << 2
+	statusSrcNAT  = 1 << 4
+	statusDstNAT  = 1 << 5
+
+	nlaNested   = 0x8000
+	nlaTypeMask = 0x3fff
+)
+
+var tcpStates = []string{"none", "syn-sent", "syn-recv", "established", "fin-wait", "close-wait", "last-ack", "time-wait", "close", "syn-sent2"}
+
+var protoNames = map[uint8]string{1: "icmp", 6: "tcp", 17: "udp", 58: "icmpv6", 132: "sctp", 136: "udplite", 33: "dccp", 47: "gre"}
+
+// Client talks to ctnetlink. The zero value works.
+type Client struct{}
+
+var seq atomic.Uint32
+
+// List returns every tracked connection, IPv4 and IPv6.
+func (Client) List(ctx context.Context) ([]api.Connection, error) {
+	msgs, err := request(ctx, msgGet, unix.NLM_F_REQUEST|unix.NLM_F_DUMP, unix.AF_UNSPEC, nil)
+	if err != nil {
+		return nil, err
+	}
+	out := []api.Connection{}
+	for _, m := range msgs {
+		if c, ok := parse(m); ok {
+			out = append(out, c)
+		}
+	}
+	return out, nil
+}
+
+// Close deletes a connection's tracking entry, found by its original
+// direction (addresses, protocol, ports or ICMP id). Gone already is fine.
+func (Client) Close(ctx context.Context, c api.Connection) error {
+	src, err1 := netip.ParseAddr(c.Source)
+	dst, err2 := netip.ParseAddr(c.Destination)
+	if err1 != nil || err2 != nil || src.Is4() != dst.Is4() {
+		return fmt.Errorf("connection %s -> %s: not a pair of addresses", c.Source, c.Destination)
+	}
+	num := protoNumber(c.Protocol)
+	if num == 0 {
+		return fmt.Errorf("connection %s -> %s: unknown protocol %q", c.Source, c.Destination, c.Protocol)
+	}
+	family, ipSrc, ipDst := uint8(unix.AF_INET), uint16(ctaIPv4Src), uint16(ctaIPv4Dst)
+	if src.Is6() {
+		family, ipSrc, ipDst = unix.AF_INET6, ctaIPv6Src, ctaIPv6Dst
+	}
+	ip := attr(ipSrc, src.AsSlice())
+	ip = append(ip, attr(ipDst, dst.AsSlice())...)
+	proto := attr(ctaProtoNum, []byte{num})
+	switch num {
+	case 1:
+		proto = append(proto, attr(ctaProtoICMPID, be16(uint16(c.ICMPID)))...)
+		proto = append(proto, attr(ctaProtoICMPType, []byte{uint8(c.ICMPType)})...)
+		proto = append(proto, attr(ctaProtoICMPCode, []byte{uint8(c.ICMPCode)})...)
+	case 58:
+		proto = append(proto, attr(ctaProtoICMP6ID, be16(uint16(c.ICMPID)))...)
+		proto = append(proto, attr(ctaProtoICMP6Typ, []byte{uint8(c.ICMPType)})...)
+		proto = append(proto, attr(ctaProtoICMP6Cod, []byte{uint8(c.ICMPCode)})...)
+	default:
+		proto = append(proto, attr(ctaProtoSrcPort, be16(uint16(c.SourcePort)))...)
+		proto = append(proto, attr(ctaProtoDstPort, be16(uint16(c.DestinationPort)))...)
+	}
+	tuple := attr(ctaTupleIP|nlaNested, ip)
+	tuple = append(tuple, attr(ctaTupleProto|nlaNested, proto)...)
+	_, err := request(ctx, msgDelete, unix.NLM_F_REQUEST|unix.NLM_F_ACK, family, attr(ctaTupleOrig|nlaNested, tuple))
+	if errors.Is(err, unix.ENOENT) {
+		return nil
+	}
+	return err
+}
+
+func protoNumber(name string) uint8 {
+	for n, s := range protoNames {
+		if s == name {
+			return n
+		}
+	}
+	return 0
+}
+
+// ---- netlink
+
+func request(ctx context.Context, msg uint16, flags uint16, family uint8, payload []byte) ([][]byte, error) {
+	fd, err := unix.Socket(unix.AF_NETLINK, unix.SOCK_RAW|unix.SOCK_CLOEXEC, unix.NETLINK_NETFILTER)
+	if err != nil {
+		return nil, err
+	}
+	defer unix.Close(fd)
+	if err := unix.Bind(fd, &unix.SockaddrNetlink{Family: unix.AF_NETLINK}); err != nil {
+		return nil, err
+	}
+	deadline := time.Now().Add(10 * time.Second)
+	if d, ok := ctx.Deadline(); ok && d.Before(deadline) {
+		deadline = d
+	}
+	tv := unix.NsecToTimeval(time.Until(deadline).Nanoseconds())
+	unix.SetsockoptTimeval(fd, unix.SOL_SOCKET, unix.SO_RCVTIMEO, &tv)
+	unix.SetsockoptInt(fd, unix.SOL_SOCKET, unix.SO_RCVBUF, 1<<22)
+
+	s := seq.Add(1)
+	b := make([]byte, unix.SizeofNlMsghdr+4, unix.SizeofNlMsghdr+4+len(payload))
+	binary.NativeEndian.PutUint16(b[4:], subsysCtnetlink<<8|msg)
+	binary.NativeEndian.PutUint16(b[6:], flags)
+	binary.NativeEndian.PutUint32(b[8:], s)
+	b[16] = family // nfgenmsg: family, version 0, res_id 0
+	b = append(b, payload...)
+	binary.NativeEndian.PutUint32(b[0:], uint32(len(b)))
+	if err := unix.Sendto(fd, b, 0, &unix.SockaddrNetlink{Family: unix.AF_NETLINK}); err != nil {
+		return nil, err
+	}
+	var out [][]byte
+	buf := make([]byte, 1<<16)
+	for {
+		n, _, err := unix.Recvfrom(fd, buf, 0)
+		if err != nil {
+			if errors.Is(err, unix.EAGAIN) {
+				return nil, errors.New("no answer from connection tracking (is nf_conntrack loaded?)")
+			}
+			return nil, err
+		}
+		for off := 0; off+unix.SizeofNlMsghdr <= n; {
+			l := int(binary.NativeEndian.Uint32(buf[off:]))
+			typ := binary.NativeEndian.Uint16(buf[off+4:])
+			if l < unix.SizeofNlMsghdr || off+l > n {
+				return nil, errors.New("connection tracking: a malformed answer")
+			}
+			body := buf[off+unix.SizeofNlMsghdr : off+l]
+			switch typ {
+			case unix.NLMSG_DONE:
+				return out, nil
+			case unix.NLMSG_ERROR:
+				if len(body) >= 4 {
+					if code := int32(binary.NativeEndian.Uint32(body)); code != 0 {
+						return nil, syscall.Errno(-code)
+					}
+				}
+				return out, nil // the acknowledgement
+			default:
+				out = append(out, append([]byte(nil), body...))
+			}
+			off += (l + 3) &^ 3
+		}
+		if flags&unix.NLM_F_DUMP == 0 && flags&unix.NLM_F_ACK == 0 {
+			return out, nil
+		}
+	}
+}
+
+func attr(typ uint16, data []byte) []byte {
+	l := 4 + len(data)
+	b := make([]byte, (l+3)&^3)
+	binary.NativeEndian.PutUint16(b[0:], uint16(l))
+	binary.NativeEndian.PutUint16(b[2:], typ)
+	copy(b[4:], data)
+	return b
+}
+
+func be16(v uint16) []byte {
+	b := make([]byte, 2)
+	binary.BigEndian.PutUint16(b, v)
+	return b
+}
+
+// attrs splits netlink attributes: type (without the nested flag) → data.
+func attrs(b []byte) map[uint16][]byte {
+	m := map[uint16][]byte{}
+	for len(b) >= 4 {
+		l := int(binary.NativeEndian.Uint16(b[0:]))
+		typ := binary.NativeEndian.Uint16(b[2:]) & nlaTypeMask
+		if l < 4 || l > len(b) {
+			break
+		}
+		m[typ] = b[4:l]
+		b = b[min((l+3)&^3, len(b)):]
+	}
+	return m
+}
+
+// ---- parsing
+
+type tuple struct {
+	src, dst         netip.Addr
+	proto            uint8
+	sport, dport     uint16
+	icmpID           uint16
+	icmpType, icmpCd uint8
+}
+
+func parseTuple(b []byte) (tuple, bool) {
+	var t tuple
+	a := attrs(b)
+	ip := attrs(a[ctaTupleIP])
+	if v, ok := ip[ctaIPv4Src]; ok {
+		t.src, _ = netip.AddrFromSlice(v)
+		t.dst, _ = netip.AddrFromSlice(ip[ctaIPv4Dst])
+	} else if v, ok := ip[ctaIPv6Src]; ok {
+		t.src, _ = netip.AddrFromSlice(v)
+		t.dst, _ = netip.AddrFromSlice(ip[ctaIPv6Dst])
+	}
+	p := attrs(a[ctaTupleProto])
+	if v := p[ctaProtoNum]; len(v) == 1 {
+		t.proto = v[0]
+	}
+	if v := p[ctaProtoSrcPort]; len(v) == 2 {
+		t.sport = binary.BigEndian.Uint16(v)
+	}
+	if v := p[ctaProtoDstPort]; len(v) == 2 {
+		t.dport = binary.BigEndian.Uint16(v)
+	}
+	for _, k := range [][3]uint16{{ctaProtoICMPID, ctaProtoICMPType, ctaProtoICMPCode}, {ctaProtoICMP6ID, ctaProtoICMP6Typ, ctaProtoICMP6Cod}} {
+		if v := p[k[0]]; len(v) == 2 {
+			t.icmpID = binary.BigEndian.Uint16(v)
+		}
+		if v := p[k[1]]; len(v) == 1 {
+			t.icmpType = v[0]
+		}
+		if v := p[k[2]]; len(v) == 1 {
+			t.icmpCd = v[0]
+		}
+	}
+	return t, t.src.IsValid() && t.dst.IsValid()
+}
+
+// parse reads one connection (an nfgenmsg and its attributes).
+func parse(m []byte) (api.Connection, bool) {
+	if len(m) < 4 {
+		return api.Connection{}, false
+	}
+	a := attrs(m[4:])
+	orig, ok1 := parseTuple(a[ctaTupleOrig])
+	reply, ok2 := parseTuple(a[ctaTupleReply])
+	if !ok1 || !ok2 {
+		return api.Connection{}, false
+	}
+	c := api.Connection{
+		Family: "ipv4", Protocol: protoNames[orig.proto],
+		Source: orig.src.String(), SourcePort: int(orig.sport), Destination: orig.dst.String(), DestinationPort: int(orig.dport),
+		ICMPID: int(orig.icmpID), ICMPType: int(orig.icmpType), ICMPCode: int(orig.icmpCd),
+	}
+	if orig.src.Is6() {
+		c.Family = "ipv6"
+	}
+	if c.Protocol == "" {
+		c.Protocol = fmt.Sprint(orig.proto)
+	}
+	// NAT shows in the reply direction: it comes from where the packets
+	// really went, to where they really came from.
+	if reply.src != orig.dst || reply.sport != orig.dport {
+		c.DNATTo = reply.src.String()
+		if reply.sport != orig.dport {
+			c.DNATPort = int(reply.sport)
+		}
+	}
+	if reply.dst != orig.src || reply.dport != orig.sport {
+		c.SNATTo = reply.dst.String()
+	}
+	if v := a[ctaStatus]; len(v) == 4 {
+		st := binary.BigEndian.Uint32(v)
+		c.Assured = st&statusAssured != 0
+	}
+	if v := a[ctaTimeout]; len(v) == 4 {
+		c.Timeout = int(binary.BigEndian.Uint32(v))
+	}
+	if v := a[ctaMark]; len(v) == 4 {
+		c.Mark = binary.BigEndian.Uint32(v)
+	}
+	if v := a[ctaID]; len(v) == 4 {
+		c.ID = binary.BigEndian.Uint32(v)
+	}
+	if v := attrs(attrs(a[ctaProtoinfo])[ctaProtoinfoTCP])[ctaProtoinfoTCPState]; len(v) == 1 && int(v[0]) < len(tcpStates) {
+		c.State = tcpStates[v[0]]
+	}
+	for _, k := range []uint16{ctaCountersOrig, ctaCountersReply} {
+		ct := attrs(a[k])
+		if v := ct[ctaCountersPackets]; len(v) == 8 {
+			c.Packets += binary.BigEndian.Uint64(v)
+		}
+		if v := ct[ctaCountersBytes]; len(v) == 8 {
+			c.Bytes += binary.BigEndian.Uint64(v)
+		}
+	}
+	return c, true
+}

internal/conntrack/conntrack_test.go

@@ -0,0 +1,51 @@
+package conntrack
+
+import (
+	"encoding/binary"
+	"net/netip"
+	"testing"
+)
+
+func nested(typ uint16, parts ...[]byte) []byte {
+	var b []byte
+	for _, p := range parts {
+		b = append(b, p...)
+	}
+	return attr(typ|nlaNested, b)
+}
+
+func tupleAttr(typ uint16, src, dst string, proto uint8, sport, dport uint16) []byte {
+	s, d := netip.MustParseAddr(src), netip.MustParseAddr(dst)
+	return nested(typ,
+		nested(ctaTupleIP, attr(ctaIPv4Src, s.AsSlice()), attr(ctaIPv4Dst, d.AsSlice())),
+		nested(ctaTupleProto, attr(ctaProtoNum, []byte{proto}), attr(ctaProtoSrcPort, be16(sport)), attr(ctaProtoDstPort, be16(dport))))
+}
+
+// A port-forwarded TCP connection from the internet, as ctnetlink
+// describes it: the reply comes from the inside host.
+func TestParse(t *testing.T) {
+	status, timeout := make([]byte, 4), make([]byte, 4)
+	binary.BigEndian.PutUint32(status, statusAssured|statusDstNAT)
+	binary.BigEndian.PutUint32(timeout, 431999)
+	msg := []byte{2, 0, 0, 0} // nfgenmsg: AF_INET
+	msg = append(msg, tupleAttr(ctaTupleOrig, "198.51.100.7", "203.0.113.10", 6, 51432, 8443)...)
+	msg = append(msg, tupleAttr(ctaTupleReply, "192.168.1.10", "198.51.100.7", 6, 443, 51432)...)
+	msg = append(msg, attr(ctaStatus, status)...)
+	msg = append(msg, attr(ctaTimeout, timeout)...)
+	msg = append(msg, nested(ctaProtoinfo, nested(ctaProtoinfoTCP, attr(ctaProtoinfoTCPState, []byte{3})))...)
+	c, ok := parse(msg)
+	if !ok {
+		t.Fatal("not parsed")
+	}
+	if c.Protocol != "tcp" || c.Source != "198.51.100.7" || c.SourcePort != 51432 || c.Destination != "203.0.113.10" || c.DestinationPort != 8443 ||
+		c.DNATTo != "192.168.1.10" || c.DNATPort != 443 || c.SNATTo != "" || c.State != "established" || !c.Assured || c.Timeout != 431999 {
+		t.Errorf("parsed %+v", c)
+	}
+	// Masqueraded on the way out: the reply comes back to the WAN address.
+	msg = []byte{2, 0, 0, 0}
+	msg = append(msg, tupleAttr(ctaTupleOrig, "192.168.1.20", "198.51.100.9", 17, 5353, 53)...)
+	msg = append(msg, tupleAttr(ctaTupleReply, "198.51.100.9", "203.0.113.10", 17, 53, 5353)...)
+	if c, _ := parse(msg); c.SNATTo != "203.0.113.10" || c.DNATTo != "" || c.Protocol != "udp" {
+		t.Errorf("masqueraded: %+v", c)
+	}
+}

internal/engine/engine.go

@@ -137,6 +137,9 @@ type Edit func(st *model.State, newID func() int64) error
 type Options struct {
 	DryRun bool
 	Via    string // web, cli, api, import
+	// RevertOf is the history entry the change undoes (a changeset with a
+	// staged undo), so History links the two.
+	RevertOf int64
 }
 
 // Apply makes a change: edit runs on a copy of the configuration, the
@@ -161,7 +164,7 @@ func (e *Engine) Apply(ctx context.Context, p *auth.Principal, description strin
 	if idErr != nil {
 		return nil, idErr
 	}
-	return e.commit(ctx, p, description, opts, next, 0)
+	return e.commit(ctx, p, description, opts, next, opts.RevertOf)
 }
 
 // commit validates next and saves it as the new state.
@@ -264,8 +267,22 @@ func (e *Engine) Revert(ctx context.Context, p *auth.Principal, id int64, opts O
 	}
 	e.mu.Lock()
 	defer e.mu.Unlock()
-	cur := objects(e.state)
 	next := e.state.Clone()
+	if err := Undo(next, h); err != nil {
+		return nil, err
+	}
+	return e.commit(ctx, p, UndoDescription(h), opts, next, h.ID)
+}
+
+// UndoDescription is an undo's description: "undo 12: create rule …".
+func UndoDescription(h *model.HistoryEntry) string {
+	return fmt.Sprintf("undo %d: %s", h.ID, h.Description)
+}
+
+// Undo applies a history entry backwards to st, if every object it
+// changed is still as it left it (a change made since would be lost).
+func Undo(st *model.State, h *model.HistoryEntry) error {
+	cur := objects(st)
 	for i := len(h.Changes) - 1; i >= 0; i-- {
 		c := h.Changes[i]
 		now := cur[key{c.Kind, c.ID}]
@@ -274,13 +291,13 @@ func (e *Engine) Revert(ctx context.Context, p *auth.Principal, id int64, opts O
 			if what == "" {
 				what = fmt.Sprintf("%s %d", c.Kind, c.ID)
 			}
-			return nil, Errorf(KindConflict, "%s %s has changed since entry %d; undo the later changes first, or change it by hand", c.Kind, what, id)
+			return Errorf(KindConflict, "%s %s has changed since entry %d; undo the later changes first, or change it by hand", c.Kind, what, h.ID)
 		}
-		if err := put(next, c.Kind, c.ID, c.Before); err != nil {
+		if err := put(st, c.Kind, c.ID, c.Before); err != nil {
-			return nil, err
+			return err
 		}
 	}
-	return e.commit(ctx, p, fmt.Sprintf("undo %d: %s", h.ID, h.Description), opts, next, h.ID)
+	return nil
 }
 
 // Replace makes the configuration next as a whole (imports).

internal/importer/importer.go

@@ -39,6 +39,8 @@ type Input struct {
 // Result is the imported configuration and what to know about it.
 type Result struct {
 	State *model.State
+	// Table is the table imported ("inet main").
+	Table string
 	Notes []string
 	// Raw lists the rules that became raw rules.
 	Raw []string
@@ -206,7 +208,7 @@ func Import(in Input) (*Result, error) {
 	im.run()
 	im.nameForwards()
 	im.st.Renumber()
-	return &Result{State: im.st, Notes: im.notes, Raw: im.raw}, nil
+	return &Result{State: im.st, Table: im.table.Family + " " + im.table.Name, Notes: im.notes, Raw: im.raw}, nil
 }
 
 func tableList(ts []nftTable) string {
@@ -1464,7 +1466,8 @@ func (im *importer) rawRule(r *model.Rule, p *parsed) {
 	if r.Action == "" {
 		r.Action = model.ActionAccept
 	}
-	r.Match, r.Raw = model.Match{}, text
+	// Only the text counts: what was parsed of it goes.
+	*r = model.Rule{ID: r.ID, Chain: r.Chain, Position: r.Position, Comment: r.Comment, Action: r.Action, Raw: text}
 	im.st.Rules = append(im.st.Rules, r)
 	why := strings.Join(p.unsupported, ", ")
 	if why == "" {

internal/matrix/matrix.go

@@ -0,0 +1,323 @@
+// Package matrix works out what the rules do to new connections between
+// zones, for an overview: for each pair it walks the rules as nftables
+// would, in order. A rule that applies to the whole pair decides it; one
+// that applies to some of its traffic (some addresses, services, times,
+// rates…) is an exception, and the walk goes on; what is left falls to
+// the chain's policy.
+package matrix
+
+import (
+	"fmt"
+	"net/netip"
+	"strings"
+
+	"housecarl/internal/api"
+	"housecarl/internal/model"
+)
+
+// Firewall stands for the machine itself in the matrix.
+const Firewall = "firewall"
+
+// Options tell Build about the machine.
+type Options struct {
+	// Connected maps the networks directly connected to the firewall's
+	// interfaces to their zones. A rule whose addresses all lie in other
+	// zones' networks can't apply to a pair, and is left out of it.
+	Connected map[netip.Prefix]string
+}
+
+// Build works out the matrix of a configuration.
+func Build(st *model.State, opts ...Options) api.Matrix {
+	var o Options
+	if len(opts) > 0 {
+		o = opts[0]
+	}
+	m := api.Matrix{Zones: []string{}, Cells: []api.MatrixCell{}}
+	for _, z := range st.Zones {
+		m.Zones = append(m.Zones, z.Name)
+	}
+	for _, from := range m.Zones {
+		for _, to := range m.Zones {
+			m.Cells = append(m.Cells, cell(st, o, model.ChainForward, from, to))
+		}
+		m.Cells = append(m.Cells, cell(st, o, model.ChainInput, from, Firewall))
+	}
+	for _, to := range m.Zones {
+		m.Cells = append(m.Cells, cell(st, o, model.ChainOutput, Firewall, to))
+	}
+	return m
+}
+
+// Cell returns one pair (from or to may be Firewall).
+func Cell(st *model.State, o Options, from, to string) (api.MatrixCell, bool) {
+	known := func(z string) bool { return st.Zone(z) != nil }
+	switch {
+	case from == Firewall && known(to):
+		return cell(st, o, model.ChainOutput, from, to), true
+	case to == Firewall && known(from):
+		return cell(st, o, model.ChainInput, from, to), true
+	case known(from) && known(to):
+		return cell(st, o, model.ChainForward, from, to), true
+	}
+	return api.MatrixCell{}, false
+}
+
+func cell(st *model.State, o Options, chain, from, to string) api.MatrixCell {
+	c := api.MatrixCell{From: from, To: to, Chain: chain, Exceptions: []api.MatrixException{}}
+	b := st.Settings.Builtins
+	fromZone := st.Zone(from)
+	external := fromZone != nil && fromZone.External
+	// What comes in passes the early chain first.
+	if chain != model.ChainOutput {
+		if b.Antispoof && external {
+			c.Exceptions = append(c.Exceptions, api.MatrixException{Chain: model.ChainEarly, Label: "built-in: antispoof", Verdict: "blocked",
+				What: "sources that can't come from outside (private ranges, your own prefixes)"})
+		}
+		if decided := walk(st, o, &c, model.ChainEarly, from, ""); decided {
+			return c
+		}
+	}
+	// The built-ins that let some new connections in.
+	switch chain {
+	case model.ChainInput:
+		if b.ICMPv6 {
+			c.Exceptions = append(c.Exceptions, api.MatrixException{Chain: chain, Label: "built-in: icmpv6", Verdict: "allowed", What: "ICMPv6 errors, echo, neighbour discovery, MLD"})
+		}
+		if b.PingInternal && !external {
+			c.Exceptions = append(c.Exceptions, api.MatrixException{Chain: chain, Label: "built-in: ping", Verdict: "allowed", What: "ping (ICMP echo)"})
+		}
+		if b.DHCPv6Client && external {
+			c.Exceptions = append(c.Exceptions, api.MatrixException{Chain: chain, Label: "built-in: dhcpv6-client", Verdict: "allowed", What: "DHCPv6 replies to the firewall"})
+		}
+	case model.ChainForward:
+		if n := forwards(st); n > 0 && external && !st.Zone(to).External {
+			c.Exceptions = append(c.Exceptions, api.MatrixException{Chain: chain, Label: "port forwards", Verdict: "allowed",
+				What: fmt.Sprintf("%d port forward%s to inside hosts", n, map[bool]string{true: "", false: "s"}[n == 1])})
+		}
+	case model.ChainOutput:
+		if st.Settings.OutputPolicy == model.PolicyDrop && b.ICMPv6 {
+			c.Exceptions = append(c.Exceptions, api.MatrixException{Chain: chain, Label: "built-in: icmpv6", Verdict: "allowed", What: "ICMPv6 errors, echo, neighbour discovery, MLD"})
+		}
+	}
+	if walk(st, o, &c, chain, from, to) {
+		return c
+	}
+	policy := map[string]string{model.ChainInput: st.Settings.InputPolicy, model.ChainForward: st.Settings.ForwardPolicy, model.ChainOutput: st.Settings.OutputPolicy}[chain]
+	c.Verdict, c.DecidedBy = verdictOf(policy), chain+" policy"
+	return c
+}
+
+func forwards(st *model.State) int {
+	n := 0
+	for _, f := range st.Forwards {
+		if !f.Disabled && f.To != "" {
+			n++
+		}
+	}
+	return n
+}
+
+// walk goes through a chain's rules for the pair, as nftables checks
+// them (those that check every packet first). It reports whether a rule
+// decided the whole pair.
+func walk(st *model.State, o Options, c *api.MatrixCell, chain, from, to string) bool {
+	var rules []*model.Rule
+	for _, every := range []bool{true, false} {
+		for _, r := range st.Rules {
+			if r.Chain == chain && !r.Disabled && r.ChecksEveryPacket() == every {
+				rules = append(rules, r)
+			}
+		}
+	}
+	for _, r := range rules {
+		if r.Raw != "" {
+			c.Exceptions = append(c.Exceptions, api.MatrixException{RuleID: r.ID, Chain: chain, Label: "rule " + r.Label(), Verdict: "unknown", What: "a raw rule: " + r.Raw})
+			continue
+		}
+		if !zones(r, from, to) || !forNew(r) {
+			continue
+		}
+		// Addresses that all live behind other zones can't come from (or
+		// go to) this pair's.
+		if !r.NotSources && elsewhere(st, o, r.Sources, from) || !r.NotDests && to != Firewall && to != "" && elsewhere(st, o, r.Destinations, to) {
+			continue
+		}
+		if chain == model.ChainEarly && r.Action != model.ActionDrop && r.Action != model.ActionReject {
+			continue // an early accept decides nothing: input or forward still check
+		}
+		v := verdictOf(r.Action)
+		what := conditions(st, r)
+		if r.Action == model.ActionLog || what != "" {
+			if what == "" {
+				what = "everything"
+			}
+			c.Exceptions = append(c.Exceptions, api.MatrixException{RuleID: r.ID, Chain: chain, Label: "rule " + r.Label(), Verdict: v, What: what})
+			continue
+		}
+		c.Verdict, c.DecidedBy, c.RuleID = v, "rule "+r.Label(), r.ID
+		return true
+	}
+	return false
+}
+
+func verdictOf(action string) string {
+	switch action {
+	case model.ActionAccept: // and the accept policy
+		return "allowed"
+	case model.ActionLog:
+		return "logged"
+	}
+	return "blocked"
+}
+
+// elsewhere reports whether every address of refs (addresses, sets) lies
+// in networks connected to zones other than zone: then traffic of zone
+// can't have them. An empty list, a dynamic set, or an address the
+// firewall isn't directly connected to (a routed network) might.
+func elsewhere(st *model.State, o Options, refs []string, zone string) bool {
+	if len(refs) == 0 || len(o.Connected) == 0 || zone == Firewall {
+		return false
+	}
+	var entries []string
+	for _, r := range refs {
+		if s := st.Set(r); s != nil {
+			if s.Dynamic {
+				return false
+			}
+			entries = append(entries, s.Entries...)
+		} else {
+			entries = append(entries, r)
+		}
+	}
+	if len(entries) == 0 {
+		return false
+	}
+	for _, e := range entries {
+		a, err := model.ParseAddress(e)
+		if err != nil {
+			return false
+		}
+		first, last := a.Prefix.Masked().Addr(), lastAddr(a.Prefix)
+		if a.First.IsValid() {
+			first, last = a.First, a.End
+		}
+		owner := ""
+		for net, z := range o.Connected {
+			if net.Contains(first) && net.Contains(last) {
+				owner = z
+				break
+			}
+		}
+		if owner == "" || strings.EqualFold(owner, zone) {
+			return false
+		}
+	}
+	return true
+}
+
+// lastAddr is a prefix's last address.
+func lastAddr(p netip.Prefix) netip.Addr {
+	b := p.Masked().Addr().AsSlice()
+	for i := p.Bits(); i < len(b)*8; i++ {
+		b[i/8] |= 0x80 >> (i % 8)
+	}
+	a, _ := netip.AddrFromSlice(b)
+	return a
+}
+
+// zones reports whether a rule's zones take in the pair.
+func zones(r *model.Rule, from, to string) bool {
+	in := func(list []string, not bool, z string) bool {
+		if len(list) == 0 {
+			return true
+		}
+		for _, x := range list {
+			if strings.EqualFold(x, z) {
+				return !not
+			}
+		}
+		return not
+	}
+	if from == Firewall {
+		if len(r.InZones) > 0 {
+			return false
+		}
+	} else if !in(r.InZones, r.NotIn, from) {
+		return false
+	}
+	if to == "" || to == Firewall {
+		return len(r.OutZones) == 0
+	}
+	return in(r.OutZones, r.NotOut, to)
+}
+
+// forNew reports whether a rule can match a new connection's first
+// packet (its connection states allow "new").
+func forNew(r *model.Rule) bool {
+	if len(r.CtState) == 0 {
+		return true
+	}
+	for _, s := range r.CtState {
+		if s == "new" {
+			return true
+		}
+	}
+	return false
+}
+
+// conditions says what limits a rule to part of a pair's traffic ("" for
+// none: it decides the pair).
+func conditions(st *model.State, r *model.Rule) string {
+	var parts []string
+	list := func(not bool, xs []string) string {
+		s := strings.Join(xs, ", ")
+		if not {
+			return "any but " + s
+		}
+		return s
+	}
+	if len(r.Sources) > 0 {
+		parts = append(parts, "from "+list(r.NotSources, r.Sources))
+	}
+	if len(r.Destinations) > 0 {
+		parts = append(parts, "to "+list(r.NotDests, r.Destinations))
+	}
+	if len(r.Services) > 0 {
+		parts = append(parts, strings.Join(r.Services, ", "))
+	}
+	if r.Family != "" {
+		parts = append(parts, r.Family+" only")
+	}
+	if len(r.CtStatus) > 0 {
+		parts = append(parts, "connections that were "+strings.Join(r.CtStatus, ", ")+"'d")
+	}
+	if len(r.Days) > 0 {
+		parts = append(parts, strings.Join(r.Days, ", "))
+	}
+	if r.HourFrom != "" {
+		parts = append(parts, r.HourFrom+"–"+r.HourTo)
+	}
+	if l := r.Limit; l != nil {
+		unit := ""
+		if l.Unit != "" && l.Unit != "packets" {
+			unit = " " + l.Unit
+		}
+		rate := fmt.Sprintf("%d%s/%s", l.Rate, unit, l.Per)
+		if l.Over {
+			parts = append(parts, "beyond "+rate)
+		} else {
+			parts = append(parts, "within "+rate)
+		}
+	}
+	if r.ConnLimit > 0 {
+		parts = append(parts, fmt.Sprintf("sources with over %d connections", r.ConnLimit))
+	}
+	if q := r.Quota; q != nil {
+		if q.Over {
+			parts = append(parts, "once its quota is used up")
+		} else {
+			parts = append(parts, "while its quota lasts")
+		}
+	}
+	return strings.Join(parts, " · ")
+}

internal/matrix/matrix_test.go

@@ -0,0 +1,112 @@
+package matrix
+
+import (
+	"fmt"
+	"net/netip"
+	"strings"
+	"testing"
+
+	"housecarl/internal/api"
+	"housecarl/internal/model"
+)
+
+func state() *model.State {
+	st := model.NewState()
+	st.Zones = []*model.Zone{
+		{ID: 1, Name: "guest", Interfaces: []string{"eth2"}},
+		{ID: 2, Name: "lan", Interfaces: []string{"eth1"}},
+		{ID: 3, Name: "wan", Interfaces: []string{"wan"}, External: true},
+	}
+	st.Sets = []*model.AddressSet{{ID: 4, Name: "admins", Entries: []string{"192.168.1.0/28"}}}
+	pos := 0.0
+	rule := func(r model.Rule) {
+		pos++
+		r.ID, r.Position = int64(10+pos), pos
+		st.Rules = append(st.Rules, &r)
+	}
+	rule(model.Rule{Chain: "input", Match: model.Match{Sources: []string{"admins"}, Services: []string{"ssh"}}, Action: "accept", Comment: "SSH"})
+	rule(model.Rule{Chain: "input", Match: model.Match{InZones: []string{"lan", "guest"}, Services: []string{"dns"}}, Action: "accept"})
+	rule(model.Rule{Chain: "forward", Match: model.Match{InZones: []string{"lan"}, OutZones: []string{"wan"}}, Action: "accept", Comment: "LAN out"})
+	rule(model.Rule{Chain: "forward", Match: model.Match{InZones: []string{"guest"}, OutZones: []string{"wan"}, Services: []string{"http", "https"}}, Action: "accept"})
+	rule(model.Rule{Chain: "forward", Match: model.Match{InZones: []string{"wan"}}, Action: "drop", Comment: "nothing in"})
+	rule(model.Rule{Chain: "forward", Match: model.Match{InZones: []string{"lan"}}, CtState: []string{"established"}, Action: "drop"}) // not for new
+	rule(model.Rule{Chain: "output", Match: model.Match{OutZones: []string{"wan"}}, Action: "drop", Disabled: true})
+	st.Forwards = []*model.PortForward{{ID: 30, Name: "web", Protocols: []string{"tcp"}, Ports: "443", To: "192.168.1.10"}}
+	st.Settings.Builtins.MSSClamp = true
+	return st
+}
+
+func find(m api.Matrix, from, to string) api.MatrixCell {
+	for _, c := range m.Cells {
+		if c.From == from && c.To == to {
+			return c
+		}
+	}
+	return api.MatrixCell{}
+}
+
+func TestMatrix(t *testing.T) {
+	m := Build(state())
+	if strings.Join(m.Zones, ",") != "guest,lan,wan" || len(m.Cells) != 3*3+3+3 {
+		t.Fatalf("zones %v, %d cells", m.Zones, len(m.Cells))
+	}
+	for _, c := range []struct {
+		from, to, verdict, decided string
+		exceptions                 int
+	}{
+		{"lan", "wan", "allowed", "rule forward #13 (LAN out)", 0},
+		{"guest", "wan", "blocked", "forward policy", 1},              // only web
+		{"wan", "lan", "blocked", "rule forward #15 (nothing in)", 2}, // antispoof, port forwards come first
+		{"lan", "guest", "blocked", "forward policy", 0},              // the established-only rule is no exception
+		{"lan", Firewall, "blocked", "input policy", 4},               // icmpv6, ping, ssh from admins, dns
+		{"wan", Firewall, "blocked", "input policy", 4},               // antispoof, icmpv6, dhcpv6, ssh from admins
+		{Firewall, "wan", "allowed", "output policy", 0},              // the drop is disabled
+	} {
+		got := find(m, c.from, c.to)
+		if got.Verdict != c.verdict || got.DecidedBy != c.decided || len(got.Exceptions) != c.exceptions {
+			t.Errorf("%s → %s: %s by %s, %d exceptions %+v; want %s by %s, %d", c.from, c.to, got.Verdict, got.DecidedBy, len(got.Exceptions), got.Exceptions, c.verdict, c.decided, c.exceptions)
+		}
+	}
+	g := find(m, "guest", "wan").Exceptions[0]
+	if g.Verdict != "allowed" || g.What != "http, https" || g.RuleID != 14 {
+		t.Errorf("the guest exception: %+v", g)
+	}
+	if e := find(m, "wan", "lan").Exceptions; e[0].Label != "built-in: antispoof" || e[1].Label != "port forwards" {
+		t.Errorf("wan → lan exceptions: %+v", e)
+	}
+	if _, ok := Cell(state(), Options{}, "lan", "nowhere"); ok {
+		t.Error("an unknown zone")
+	}
+}
+
+// Rules whose addresses all live behind another zone are left out of a
+// pair; routed (unknown) networks, and mixed lists, stay.
+func TestConnected(t *testing.T) {
+	st := state()
+	st.Sets = append(st.Sets, &model.AddressSet{ID: 5, Name: "guest_net", Entries: []string{"192.168.2.0/24"}},
+		&model.AddressSet{ID: 6, Name: "mixed", Entries: []string{"192.168.2.0/24", "10.20.30.0/24"}})
+	st.Rules = append(st.Rules,
+		&model.Rule{ID: 40, Chain: "forward", Position: 10, Match: model.Match{Sources: []string{"guest_net"}}, Action: "drop", Comment: "guests"},
+		&model.Rule{ID: 41, Chain: "forward", Position: 11, Match: model.Match{Destinations: []string{"192.168.2.5"}}, Action: "accept", Comment: "to a guest"},
+		&model.Rule{ID: 42, Chain: "forward", Position: 12, Match: model.Match{Sources: []string{"mixed"}}, Action: "drop", Comment: "mixed"})
+	o := Options{Connected: map[netip.Prefix]string{netip.MustParsePrefix("192.168.1.0/24"): "lan", netip.MustParsePrefix("192.168.2.0/24"): "guest"}}
+	ids := func(c api.MatrixCell) []int64 {
+		var out []int64
+		for _, e := range c.Exceptions {
+			out = append(out, e.RuleID)
+		}
+		return out
+	}
+	lanWan, _ := Cell(st, o, "lan", "wan")
+	if fmt.Sprint(ids(lanWan)) != "[]" {
+		t.Errorf("lan → wan: %v", ids(lanWan)) // decided by "LAN out" before the others anyway
+	}
+	lanGuest, _ := Cell(st, o, "lan", "guest")
+	if fmt.Sprint(ids(lanGuest)) != "[41 42]" {
+		t.Errorf("lan → guest: %v", ids(lanGuest)) // not "guests": their addresses are guest's
+	}
+	guestLan, _ := Cell(st, o, "guest", "lan")
+	if fmt.Sprint(ids(guestLan)) != "[40 42]" {
+		t.Errorf("guest → lan: %v", ids(guestLan)) // not "to a guest": it is in guest, not lan
+	}
+}

internal/model/changeset.go

@@ -35,13 +35,16 @@ const (
 	OpDelete   = "delete"
 	OpMove     = "move"
 	OpSettings = "settings"
+	// OpRevert undoes a history entry: Data is the entry (its changes),
+	// ID its number.
+	OpRevert = "revert"
 )
 
 // Op is one change, as the API received it, so it can be stored and
 // replayed on whatever the configuration is when it is applied.
 type Op struct {
 	Seq  int    `json:"seq"`
-	Op   string `json:"op"` // create, update, delete, move, settings
+	Op   string `json:"op"` // create, update, delete, move, settings, revert
 	Kind string `json:"kind,omitempty"`
 	// Ref names the object (ID, or name) for update, delete and move.
 	Ref string `json:"ref,omitempty"`

internal/model/objects.go

@@ -143,6 +143,10 @@ type Rule struct {
 	// IPv4 address alone, IPv6 by /64, since one device has several).
 	GroupV4 int `json:"group_v4,omitempty"`
 	GroupV6 int `json:"group_v6,omitempty"`
+	// EveryPacket checks the rule against every packet, before established
+	// connections are accepted, so a drop or reject also stops connections
+	// already open, not just new ones.
+	EveryPacket bool   `json:"every_packet,omitempty"`
 	Action      string `json:"action"`
 	// RejectWith: "" (ICMP port/admin unreachable as fits) or tcp-reset.
 	RejectWith string `json:"reject_with,omitempty"`
@@ -252,6 +256,12 @@ func (r *Rule) CountsTraffic() bool {
 	return bytes
 }
 
+// ChecksEveryPacket reports whether a rule is checked before the built-in
+// that accepts established connections: it counts traffic, or asks to.
+func (r *Rule) ChecksEveryPacket() bool {
+	return r.Raw == "" && (r.EveryPacket || r.CountsTraffic())
+}
+
 // Groups returns a rule's address grouping, defaults applied.
 func (r *Rule) Groups() (v4, v6 int) {
 	v4, v6 = r.GroupV4, r.GroupV6

internal/model/settings.go

@@ -31,6 +31,11 @@ type Settings struct {
 	// ProbationSeconds is how long a change made over the network has to
 	// be confirmed before it is rolled back (0: the default).
 	ProbationSeconds int `json:"probation_seconds,omitempty"`
+	// ManualConfirm: changes made over the network wait for a person to
+	// confirm them. Otherwise the client that made a change confirms it by
+	// itself once it reaches the API again over a new connection, from the
+	// same address: proof that it can still manage the firewall.
+	ManualConfirm bool `json:"manual_confirm,omitempty"`
 }
 
 // Builtins are rules Housecarl adds itself, shown as "added by Housecarl".

internal/nft/nft.go

@@ -8,6 +8,7 @@
 package nft
 
 import (
+	"bufio"
 	"bytes"
 	"context"
 	"encoding/json"
@@ -24,6 +25,7 @@ import (
 
 	"housecarl/internal/model"
 	"housecarl/internal/render"
+	"housecarl/internal/trace"
 )
 
 // Runner runs nft.
@@ -541,6 +543,41 @@ func (f *Firewall) ApplyScript(ctx context.Context, script string) error {
 	return f.Runner.ApplyScript(ctx, script)
 }
 
+// Trace follows packets: it starts `nft monitor trace`, puts the tracing
+// table (script) in, and passes each line to line until ctx ends; then
+// it takes the table out again, whatever happened.
+func (f *Firewall) Trace(ctx context.Context, script string, line func(string)) error {
+	c := f.Runner.cmd(ctx, "monitor", "trace")
+	out, err := c.StdoutPipe()
+	if err != nil {
+		return err
+	}
+	var errb bytes.Buffer
+	c.Stderr = &errb
+	if err := c.Start(); err != nil {
+		return err
+	}
+	defer f.Runner.run(context.Background(), "", "delete", "table", "inet", trace.Table)
+	if err := f.Runner.ApplyScript(ctx, script); err != nil {
+		c.Process.Kill()
+		c.Wait()
+		return fmt.Errorf("starting the trace: %w", err)
+	}
+	sc := bufio.NewScanner(out)
+	sc.Buffer(make([]byte, 64<<10), 1<<20)
+	for sc.Scan() {
+		line(sc.Text())
+	}
+	err = c.Wait()
+	if ctx.Err() != nil {
+		return nil // stopped, or its time was up
+	}
+	if err != nil && errb.Len() > 0 {
+		return fmt.Errorf("nft monitor trace: %s", strings.TrimSpace(errb.String()))
+	}
+	return err
+}
+
 // DeleteTable deletes another table (takeover).
 func (f *Firewall) DeleteTable(ctx context.Context, family, name string) error {
 	if name == render.Table && family == "inet" {

internal/render/render.go

@@ -364,7 +364,7 @@ func (r *renderer) policyTail(chain, policy string) {
 // or the others.
 func (r *renderer) rules(chain string, traffic bool) {
 	for _, rule := range r.st.Rules {
-		if rule.Chain != chain || rule.Disabled || rule.CountsTraffic() != traffic {
+		if rule.Chain != chain || rule.Disabled || rule.ChecksEveryPacket() != traffic {
 			continue
 		}
 		obj := rule.Label()

internal/render/render_test.go

@@ -76,6 +76,7 @@ func router() *model.State {
 		Quota: &model.Quota{Bytes: 50000 << 20, Over: true, Reset: "monthly", Day: 15}, Comment: "guest quota"})
 	rule(model.ChainForward, model.Rule{Match: model.Match{InZones: []string{"lan"}, OutZones: []string{"wan"}}, Action: model.ActionDrop,
 		Quota: &model.Quota{Bytes: 5 << 30, Over: true, PerSource: true, Reset: "daily"}, Comment: "device quota"})
+	rule(model.ChainForward, model.Rule{Match: model.Match{Sources: []string{"10.0.10.66"}}, Action: model.ActionReject, EveryPacket: true, Comment: "blocked now"})
 	rule(model.ChainEarly, model.Rule{Match: model.Match{InZones: []string{"wan"}, Sources: []string{"198.51.100.0/24"}}, Action: model.ActionDrop})
 	rule(model.ChainOutput, model.Rule{Raw: "meta skuid 0 counter accept", Comment: "raw"})
 	st.Forwards = []*model.PortForward{
@@ -186,6 +187,9 @@ func TestRenderRouter(t *testing.T) {
 	if bw := strings.Index(fwd, "limit rate over 5 mbytes/second"); bw > est {
 		t.Error("the byte rate rule comes after established")
 	}
+	if now := strings.Index(fwd, "blocked now"); now < 0 || now > est {
+		t.Error("the every-packet rule comes after established")
+	}
 	// The trusted masquerade has no IPv6 part: masquerading the IPv6 LAN
 	// isn't wanted (its sources are global).
 	if strings.Contains(text, "ip6 saddr @trusted_v6 counter masquerade") {

internal/server/h_changesets.go

@@ -36,8 +36,8 @@ func changesetParam(r *http.Request) (int64, error) {
 	return id, nil
 }
 
-// notStaged refuses a changeset on a change that can't be staged (undo,
+// notStaged refuses a changeset on a change that can't be staged (an
-// import, takeover): it would otherwise be applied at once, unexpectedly.
+// import, a takeover): it would otherwise be applied at once, unexpectedly.
 func notStaged(r *http.Request, what string) error {
 	if id, err := changesetParam(r); err != nil || id == 0 {
 		return err
@@ -102,6 +102,11 @@ func (s *Server) stage(w http.ResponseWriter, r *http.Request, p *auth.Principal
 		return nil
 	}
 	op.Summary = engine.Describe(changes)
+	if op.Op == model.OpRevert {
+		var h model.HistoryEntry
+		json.Unmarshal(op.Data, &h)
+		op.Summary = engine.UndoDescription(&h)
+	}
 	op.AddedAt = s.nowFn()
 	for _, o := range cs.Ops {
 		if o.Seq >= op.Seq {
@@ -353,7 +358,17 @@ func (s *Server) applyChangeset(w http.ResponseWriter, r *http.Request, p *auth.
 	if len(cs.Ops) == 0 {
 		return badRequest("changeset %d has no changes to apply", id)
 	}
-	res, err := s.eng.Apply(r.Context(), p, cs.Description, engine.Options{Via: via(r, p)}, func(st *model.State, _ func() int64) error {
+	opts := engine.Options{Via: via(r, p)}
+	desc := cs.Description
+	for _, o := range cs.Ops {
+		if o.Op == model.OpRevert {
+			opts.RevertOf = o.ID // History links the undo
+			if desc == "" && len(cs.Ops) == 1 {
+				desc = o.Summary
+			}
+		}
+	}
+	res, err := s.eng.Apply(r.Context(), p, desc, opts, func(st *model.State, _ func() int64) error {
 		return replay(st, cs.Ops)
 	})
 	if err != nil {
@@ -368,6 +383,7 @@ func (s *Server) applyChangeset(w http.ResponseWriter, r *http.Request, p *auth.
 	s.eng.Audit(r.Context(), p, "changeset.apply", fmt.Sprintf("changeset %d", id), "success", fmt.Sprintf("history %d", res.HistoryID))
 	s.afterChange(r.Context(), p, res)
 	out := s.mutation(res, nil)
+	out.ConfirmToken = s.issueCheck(w, r, p, res)
 	out.Changeset = cs
 	writeJSON(w, http.StatusOK, out)
 	return nil

internal/server/h_config.go

@@ -4,6 +4,7 @@ import (
 	"encoding/json"
 	"errors"
 	"net/http"
+	"net/netip"
 	"sort"
 	"strings"
 
@@ -12,6 +13,7 @@ import (
 	"housecarl/internal/api"
 	"housecarl/internal/auth"
 	"housecarl/internal/engine"
+	"housecarl/internal/matrix"
 	"housecarl/internal/model"
 	"housecarl/internal/store"
 )
@@ -48,7 +50,9 @@ func (s *Server) change(w http.ResponseWriter, r *http.Request, p *auth.Principa
 		return nil
 	}
 	s.afterChange(r.Context(), p, res)
-	writeJSON(w, s.mutationStatus(res, op.Op == model.OpCreate), s.mutation(res, objectOf(&op, id)))
+	out := s.mutation(res, objectOf(&op, id))
+	out.ConfirmToken = s.issueCheck(w, r, p, res)
+	writeJSON(w, s.mutationStatus(res, op.Op == model.OpCreate), out)
 	return nil
 }
 
@@ -83,8 +87,10 @@ func (s *Server) mutationStatus(res *engine.Result, creates bool) int {
 }
 
 // respondChange answers a change made without an operation (undo).
-func (s *Server) respondChange(w http.ResponseWriter, r *http.Request, res *engine.Result, object func(st *model.State) any, creates bool) {
+func (s *Server) respondChange(w http.ResponseWriter, r *http.Request, p *auth.Principal, res *engine.Result, object func(st *model.State) any, creates bool) {
-	writeJSON(w, s.mutationStatus(res, creates), s.mutation(res, object))
+	out := s.mutation(res, object)
+	out.ConfirmToken = s.issueCheck(w, r, p, res)
+	writeJSON(w, s.mutationStatus(res, creates), out)
 }
 
 // rulesetDiff is the unified diff of the rulesets two configurations
@@ -412,6 +418,41 @@ func (s *Server) updateSettings(w http.ResponseWriter, r *http.Request, p *auth.
 	return s.change(w, r, p, model.Op{Op: model.OpSettings, Data: data})
 }
 
+// ---- the zone matrix
+
+// zoneMatrix answers what the rules do to new connections between zones
+// (internal/matrix); with a changeset named, as it will make them.
+func (s *Server) zoneMatrix(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
+	if err := require(p, auth.ResConfig, false); err != nil {
+		return err
+	}
+	_, st, _, err := s.viewState(w, r)
+	if err != nil {
+		return err
+	}
+	writeJSON(w, http.StatusOK, matrix.Build(st, matrix.Options{Connected: s.connected(st)}))
+	return nil
+}
+
+// connected maps the networks on the firewall's interfaces (link-local
+// and loopback aside) to their zones.
+func (s *Server) connected(st *model.State) map[netip.Prefix]string {
+	out := map[netip.Prefix]string{}
+	for _, i := range s.hostInterfaces(st) {
+		if i.Zone == "" {
+			continue
+		}
+		for _, a := range i.Addresses {
+			p, err := netip.ParsePrefix(a)
+			if err != nil || p.Addr().IsLinkLocalUnicast() || p.Addr().IsLoopback() {
+				continue
+			}
+			out[p.Masked()] = i.Zone
+		}
+	}
+	return out
+}
+
 // ---- the ruleset
 
 func (s *Server) ruleset(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
@@ -465,20 +506,34 @@ func (s *Server) revertHistory(w http.ResponseWriter, r *http.Request, p *auth.P
 	if err := require(p, auth.ResConfig, true); err != nil {
 		return err
 	}
-	if err := notStaged(r, "an undo"); err != nil {
-		return err
-	}
 	id, err := pathID(r, "id")
 	if err != nil {
 		return err
 	}
+	// Staged, an undo is an operation of its own, reviewed like any.
+	if csID, err := changesetParam(r); err != nil {
+		return err
+	} else if csID != 0 {
+		h, err := s.store.GetHistory(r.Context(), id)
+		if errors.Is(err, store.ErrNotFound) {
+			return notFound("history entry %d not found", id)
+		}
+		if err != nil {
+			return err
+		}
+		if h.RevertedBy != 0 {
+			return conflict("history entry %d was already undone by entry %d", id, h.RevertedBy)
+		}
+		data, _ := json.Marshal(h)
+		return s.stage(w, r, p, csID, model.Op{Op: model.OpRevert, ID: id, Data: data})
+	}
 	res, err := s.eng.Revert(r.Context(), p, id, engine.Options{DryRun: queryBool(r, "dry_run"), Via: via(r, p)})
 	if err != nil {
 		s.writeErr(w, r, err, res)
 		return nil
 	}
 	s.afterChange(r.Context(), p, res)
-	s.respondChange(w, r, res, nil, false)
+	s.respondChange(w, r, p, res, nil, false)
 	return nil
 }
 
@@ -496,7 +551,17 @@ func (s *Server) confirmChanges(w http.ResponseWriter, r *http.Request, p *auth.
 	if err := require(p, auth.ResConfig, true); err != nil {
 		return err
 	}
-	n, err := s.confirm(r.Context(), p)
+	var in api.ConfirmInput
+	if r.ContentLength != 0 {
+		if err := decodeJSON(r, &in); err != nil {
+			return err
+		}
+	}
+	confirm := func() (int, error) { return s.confirm(r.Context(), p) }
+	if in.Token != "" {
+		confirm = func() (int, error) { return s.confirmToken(r, p, in.Token) }
+	}
+	n, err := confirm()
 	if err != nil {
 		return err
 	}

internal/server/h_import.go

@@ -51,6 +51,11 @@ func (s *Server) importNft(w http.ResponseWriter, r *http.Request, p *auth.Princ
 		return nil
 	}
 	s.afterChange(r.Context(), p, er)
+	if er.Applied {
+		// Until it is taken over, the old table still applies too: the
+		// status warns (takeover_pending).
+		s.store.SetMeta(r.Context(), importedTableKey, res.Table)
+	}
 	out.MutationResponse = api.MutationResponse{Applied: er.Applied, DryRun: er.DryRun, HistoryID: er.HistoryID, Changes: er.Changes, Warnings: er.Warnings}
 	if out.Changes == nil {
 		out.Changes = []model.Change{}
@@ -58,6 +63,7 @@ func (s *Server) importNft(w http.ResponseWriter, r *http.Request, p *auth.Princ
 	if pr := s.probationStatus(); er.Applied && len(pr.Pending) > 0 {
 		out.ConfirmBy = &pr.Deadline
 	}
+	out.ConfirmToken = s.issueCheck(w, r, p, er)
 	writeJSON(w, http.StatusOK, out)
 	return nil
 }
@@ -148,6 +154,12 @@ func (s *Server) takeover(w http.ResponseWriter, r *http.Request, p *auth.Princi
 	} else if pr := s.probationStatus(); pr.Takeovers > 0 {
 		msg += fmt.Sprintf("; confirm with 'hc confirm' within %ds, or they come back", pr.SecondsLeft)
 	}
-	writeJSON(w, http.StatusOK, map[string]any{"deleted": done, "message": msg})
+	answer := map[string]any{"deleted": done, "message": msg}
+	if !queryBool(r, "dry_run") && len(done) > 0 {
+		if tok := s.issueCheck(w, r, p, nil); tok != "" {
+			answer["confirm_token"] = tok
+		}
+	}
+	writeJSON(w, http.StatusOK, answer)
 	return nil
 }

internal/server/h_status.go

@@ -1,7 +1,10 @@
 package server
 
 import (
+	"context"
+	"fmt"
 	"net/http"
+	"os"
 	"sort"
 	"strings"
 	"sync"
@@ -44,10 +47,43 @@ func (s *Server) status(w http.ResponseWriter, r *http.Request, p *auth.Principa
 	if data, err := s.fw.ListRuleset(r.Context()); err == nil {
 		out.OtherTables = otherTables(data)
 	}
+	out.Problems = append(out.Problems, s.hostProblems(r.Context(), out.OtherTables)...)
 	writeJSON(w, http.StatusOK, out)
 	return nil
 }
 
+const importedTableKey = "imported_table"
+
+// nftablesUnitLinks are where an enabled nftables.service shows (Debian
+// enables it for sysinit.target).
+var nftablesUnitLinks = []string{
+	"/etc/systemd/system/sysinit.target.wants/nftables.service",
+	"/etc/systemd/system/multi-user.target.wants/nftables.service",
+}
+
+// hostProblems are warnings about the machine rather than the
+// configuration: another ruleset that still loads at boot, or the table
+// an import came from, still in force beside Housecarl's.
+func (s *Server) hostProblems(ctx context.Context, others []string) []validate.Problem {
+	var out []validate.Problem
+	for _, link := range s.unitLinks {
+		if _, err := os.Lstat(link); err == nil {
+			out = append(out, validate.Problem{Severity: validate.Warning, Code: "nftables_service", Object: "nftables.service",
+				Message: "nftables.service is enabled: at boot it loads /etc/nftables.conf, and stopping or restarting it flushes every table, Housecarl's included; disable it (systemctl disable nftables, without --now)"})
+			break
+		}
+	}
+	if t, err := s.store.GetMeta(ctx, importedTableKey); err == nil && t != "" {
+		for _, o := range others {
+			if o == t {
+				out = append(out, validate.Problem{Severity: validate.Warning, Code: "takeover_pending", Object: "table " + t,
+					Message: fmt.Sprintf("%s, the table you imported, still applies beside Housecarl's: a packet must get past both; once Housecarl's rules are right, delete it with 'hc takeover %q'", t, t)})
+			}
+		}
+	}
+	return out
+}
+
 // hostInterfaces lists the machine's interfaces with their zones.
 func (s *Server) hostInterfaces(st *model.State) []api.HostInterface {
 	out := []api.HostInterface{}

internal/server/h_traffic.go

@@ -0,0 +1,309 @@
+package server
+
+import (
+	"context"
+	"fmt"
+	"net/http"
+	"sort"
+	"strconv"
+	"strings"
+	"sync"
+	"time"
+
+	"housecarl/internal/api"
+	"housecarl/internal/auth"
+	"housecarl/internal/model"
+	"housecarl/internal/trace"
+)
+
+// Traffic as it happens: the connections the kernel tracks (and closing
+// them, so the rules check them again), and tracing packets through the
+// whole ruleset.
+
+// Conntrack lists and closes tracked connections (internal/conntrack).
+type Conntrack interface {
+	List(ctx context.Context) ([]api.Connection, error)
+	Close(ctx context.Context, c api.Connection) error
+}
+
+// SetConntrack gives the server the kernel's connection tracking.
+func (s *Server) SetConntrack(c Conntrack) { s.conns = c }
+
+func (s *Server) conntrack() (Conntrack, error) {
+	if s.conns == nil {
+		return nil, &httpError{http.StatusServiceUnavailable, "unavailable", "connection tracking isn't available here"}
+	}
+	return s.conns, nil
+}
+
+// connMatches: a search in a connection's addresses, ports (":443"),
+// protocol, state and NAT addresses.
+func connMatches(c api.Connection, q string) bool {
+	if q == "" {
+		return true
+	}
+	hay := strings.ToLower(strings.Join([]string{c.Protocol, c.Source, ":" + strconv.Itoa(c.SourcePort), c.Destination, ":" + strconv.Itoa(c.DestinationPort),
+		c.DNATTo, c.SNATTo, c.State}, " ") + " ")
+	for _, w := range strings.Fields(strings.ToLower(q)) {
+		if strings.HasPrefix(w, ":") {
+			w += " " // a whole port
+		}
+		if !strings.Contains(hay, w) {
+			return false
+		}
+	}
+	return true
+}
+
+func (s *Server) listConnections(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
+	if err := require(p, auth.ResLog, false); err != nil {
+		return err
+	}
+	ct, err := s.conntrack()
+	if err != nil {
+		return err
+	}
+	all, err := ct.List(r.Context())
+	if err != nil {
+		return fmt.Errorf("listing connections: %w", err)
+	}
+	q := r.URL.Query().Get("q")
+	out := []api.Connection{}
+	for _, c := range all {
+		if connMatches(c, q) {
+			out = append(out, c)
+		}
+	}
+	sort.Slice(out, func(i, j int) bool {
+		if out[i].Source != out[j].Source {
+			return out[i].Source < out[j].Source
+		}
+		return out[i].Destination < out[j].Destination
+	})
+	if n := queryInt(r, "limit", 1000); n > 0 && len(out) > n {
+		out = out[:n]
+	}
+	w.Header().Set("X-Total-Count", strconv.Itoa(len(all)))
+	writeJSON(w, http.StatusOK, out)
+	return nil
+}
+
+// closeConnections deletes tracking entries: the rules check those
+// connections again with their next packet.
+func (s *Server) closeConnections(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
+	if err := require(p, auth.ResConfig, true); err != nil {
+		return err
+	}
+	ct, err := s.conntrack()
+	if err != nil {
+		return err
+	}
+	var in api.CloseConnections
+	if err := decodeJSON(r, &in); err != nil {
+		return err
+	}
+	list := in.Connections
+	if strings.TrimSpace(in.Search) != "" {
+		all, err := ct.List(r.Context())
+		if err != nil {
+			return fmt.Errorf("listing connections: %w", err)
+		}
+		for _, c := range all {
+			if connMatches(c, in.Search) {
+				list = append(list, c)
+			}
+		}
+	} else if len(list) == 0 {
+		return badRequest("say which connections: a list, or a search")
+	}
+	closed := 0
+	for _, c := range list {
+		if err := ct.Close(r.Context(), c); err != nil {
+			return fmt.Errorf("closing %s %s -> %s: %w", c.Protocol, c.Source, c.Destination, err)
+		}
+		closed++
+	}
+	what := in.Search
+	if what == "" && len(list) == 1 {
+		what = fmt.Sprintf("%s %s:%d -> %s:%d", list[0].Protocol, list[0].Source, list[0].SourcePort, list[0].Destination, list[0].DestinationPort)
+	}
+	s.eng.Audit(r.Context(), p, "connections.close", what, "success", fmt.Sprintf("%d closed", closed))
+	writeJSON(w, http.StatusOK, map[string]int{"closed": closed})
+	return nil
+}
+
+// ---- tracing
+
+type tracing struct {
+	mu     sync.Mutex
+	status api.TraceStatus
+	tracer *trace.Tracer
+	cancel context.CancelFunc
+}
+
+func (s *Server) startTrace(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
+	if err := require(p, auth.ResConfig, true); err != nil {
+		return err
+	}
+	var in api.TraceInput
+	if err := decodeJSON(r, &in); err != nil {
+		return err
+	}
+	if in.Seconds == 0 {
+		in.Seconds = 60
+	}
+	if in.Seconds < 5 || in.Seconds > 600 {
+		return badRequest("trace for 5 to 600 seconds")
+	}
+	script, err := trace.Script(in)
+	if err != nil {
+		return badRequest("%v", err)
+	}
+	t := &s.tracing
+	t.mu.Lock()
+	if t.status.Running {
+		t.mu.Unlock()
+		return conflict("a trace is running already (started %s); stop it first", t.status.Started.Format("15:04:05"))
+	}
+	ctx, cancel := context.WithTimeout(context.Background(), time.Duration(in.Seconds)*time.Second)
+	now := s.nowFn()
+	t.status = api.TraceStatus{Running: true, Input: in, Started: now, Ends: now.Add(time.Duration(in.Seconds) * time.Second), Packets: []api.TracePacket{}}
+	t.tracer = &trace.Tracer{Now: s.nowFn}
+	t.cancel = cancel
+	t.mu.Unlock()
+	s.eng.Audit(r.Context(), p, "trace.start", traceWhat(in), "success", fmt.Sprintf("%d seconds", in.Seconds))
+	go func() {
+		defer cancel()
+		err := s.fw.Trace(ctx, script, func(line string) {
+			t.mu.Lock()
+			defer t.mu.Unlock()
+			t.tracer.Line(line)
+			if t.tracer.Full() {
+				cancel()
+			}
+		})
+		t.mu.Lock()
+		defer t.mu.Unlock()
+		t.status.Running = false
+		if err != nil {
+			t.status.Error = err.Error()
+			s.log.Warn("tracing failed", "err", err)
+		}
+	}()
+	writeJSON(w, http.StatusOK, s.traceStatus())
+	return nil
+}
+
+func traceWhat(in api.TraceInput) string {
+	var parts []string
+	if in.Address != "" {
+		parts = append(parts, in.Address)
+	}
+	if in.Protocol != "" {
+		parts = append(parts, in.Protocol)
+	}
+	if in.Port != 0 {
+		parts = append(parts, "port "+strconv.Itoa(in.Port))
+	}
+	return strings.Join(parts, " ")
+}
+
+// traceStatus is the trace with each step named after Housecarl's object.
+func (s *Server) traceStatus() api.TraceStatus {
+	t := &s.tracing
+	t.mu.Lock()
+	out := t.status
+	if t.tracer != nil {
+		out.Packets = t.tracer.Packets()
+	}
+	t.mu.Unlock()
+	if out.Packets == nil {
+		out.Packets = []api.TracePacket{}
+	}
+	st := s.eng.State()
+	for i := range out.Packets {
+		p := &out.Packets[i]
+		for j := range p.Steps {
+			p.Steps[j].Label = markLabel(st, p.Steps[j])
+		}
+		p.DecidedBy.Label = markLabel(st, p.DecidedBy)
+	}
+	return out
+}
+
+// markLabel names the Housecarl object behind a step: "rule input #12
+// (ssh)", "built-in: established", "port forward web", "input policy".
+func markLabel(st *model.State, step api.TraceStep) string {
+	if step.Kind == "policy" {
+		if step.Table == "inet housecarl" {
+			return step.Chain + " policy"
+		}
+		return ""
+	}
+	m := step.Mark
+	kind, id := "", int64(0)
+	switch {
+	case m == "":
+		return ""
+	case strings.HasPrefix(m, "b-"):
+		return "built-in: " + strings.TrimPrefix(m, "b-")
+	case strings.HasPrefix(m, "r"):
+		kind = model.KindRule
+	case strings.HasPrefix(m, "f"):
+		kind = model.KindForward
+	case strings.HasPrefix(m, "s"):
+		kind = model.KindSNAT
+	case strings.HasPrefix(m, "d"):
+		kind = model.KindRedirect
+	default:
+		return ""
+	}
+	num := strings.TrimLeft(m[1:], "0123456789")
+	id, _ = strconv.ParseInt(strings.TrimSuffix(m[1:], num), 10, 64)
+	if o := st.ByID(kind, id); o != nil {
+		label := o.Label() // "source NAT #3 (…)", "redirect #4", a forward's name…
+		switch {
+		case kind == model.KindRule:
+			label = "rule " + label
+		case kind == model.KindForward && !strings.HasPrefix(label, "port forward"):
+			label = "port forward " + label
+		}
+		if num != "" {
+			label += " (" + strings.TrimPrefix(num, "-") + ")"
+		}
+		return label
+	}
+	return ""
+}
+
+func (s *Server) getTrace(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
+	if err := require(p, auth.ResLog, false); err != nil {
+		return err
+	}
+	writeJSON(w, http.StatusOK, s.traceStatus())
+	return nil
+}
+
+func (s *Server) stopTrace(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
+	if err := require(p, auth.ResConfig, true); err != nil {
+		return err
+	}
+	t := &s.tracing
+	t.mu.Lock()
+	if t.cancel != nil && t.status.Running {
+		t.cancel()
+	}
+	t.mu.Unlock()
+	// Let the trace end, so the answer shows it stopped.
+	for i := 0; i < 50; i++ {
+		t.mu.Lock()
+		running := t.status.Running
+		t.mu.Unlock()
+		if !running {
+			break
+		}
+		time.Sleep(20 * time.Millisecond)
+	}
+	writeJSON(w, http.StatusOK, s.traceStatus())
+	return nil
+}

internal/server/ops.go

@@ -31,6 +31,12 @@ func applyOp(st *model.State, op *model.Op) (int64, error) {
 		return moveOp(st, op)
 	case model.OpSettings:
 		return 0, settingsOp(st, op)
+	case model.OpRevert:
+		var h model.HistoryEntry
+		if err := json.Unmarshal(op.Data, &h); err != nil {
+			return 0, badRequest("revert: %v", err)
+		}
+		return 0, engine.Undo(st, &h)
 	}
 	return 0, badRequest("unknown operation %q", op.Op)
 }

internal/server/probation.go

@@ -2,8 +2,10 @@ package server
 
 import (
 	"context"
+	"crypto/subtle"
 	"encoding/json"
 	"fmt"
+	"net/http"
 	"sync"
 	"time"
 
@@ -31,6 +33,21 @@ type probation struct {
 	timer    *time.Timer
 	// restores are nft scripts that bring back tables a takeover deleted.
 	restores []string
+	// check is the latest change's confirmation token (reachCheck): only
+	// it can confirm, since the ruleset in force is the one it tests.
+	check *reachCheck
+}
+
+// reachCheck is what a client must show to confirm its change by itself:
+// the token, from the same user (and API token), the same address, and a
+// new connection (another remote address and port than the change's).
+type reachCheck struct {
+	token    string
+	userID   int64
+	username string
+	tokenID  int64
+	ip       string
+	remote   string
 }
 
 // system is the principal of rollbacks.
@@ -88,6 +105,60 @@ func (s *Server) startProbation(ctx context.Context, p *auth.Principal, historyI
 	s.saveProbation(ctx)
 }
 
+// issueCheck gives the client that made a network change on probation a
+// token to confirm it by itself, unless the settings want a person to
+// confirm. The answer then closes its connection (HTTP/2: a GOAWAY), so
+// the confirmation has to open a new one through the new ruleset; an
+// established connection would get through whatever the change did.
+func (s *Server) issueCheck(w http.ResponseWriter, r *http.Request, p *auth.Principal, res *engine.Result) string {
+	if p == nil || p.Method == "socket" || (res != nil && !res.Applied) || s.eng.State().Settings.ManualConfirm {
+		return ""
+	}
+	pr := &s.probation
+	pr.mu.Lock()
+	defer pr.mu.Unlock()
+	if len(pr.pending) == 0 && len(pr.restores) == 0 {
+		return ""
+	}
+	c := &reachCheck{token: auth.RandomString(24), username: p.Name(), ip: s.clientIP(r), remote: r.RemoteAddr}
+	if p.User != nil {
+		c.userID = p.User.ID
+	}
+	if p.Token != nil {
+		c.tokenID = p.Token.ID
+	}
+	pr.check = c
+	w.Header().Set("Connection", "close")
+	return c.token
+}
+
+// confirmToken confirms with a change's token: the proof that the client
+// that made it still reaches the API, from where it made it.
+func (s *Server) confirmToken(r *http.Request, p *auth.Principal, token string) (int, error) {
+	pr := &s.probation
+	pr.mu.Lock()
+	c := pr.check
+	pending := len(pr.pending) + len(pr.restores)
+	pr.mu.Unlock()
+	var tokenID int64
+	if p.Token != nil {
+		tokenID = p.Token.ID
+	}
+	switch {
+	case pending == 0:
+		return 0, nil
+	case c == nil || subtle.ConstantTimeCompare([]byte(c.token), []byte(token)) != 1:
+		return 0, conflict("this confirmation is for an earlier change: a newer one is on probation, and only its own client (or a person) can confirm it")
+	case p.User == nil || p.User.ID != c.userID || p.Name() != c.username || tokenID != c.tokenID:
+		return 0, forbidden("only the client that made the change can confirm it this way")
+	case s.clientIP(r) != c.ip:
+		return 0, forbidden("the confirmation comes from %s, the change from %s: it must come from the same place", s.clientIP(r), c.ip)
+	case r.RemoteAddr == c.remote:
+		return 0, conflict("the confirmation came over the change's own connection, which proves nothing; it must open a new one")
+	}
+	return s.confirmWith(r.Context(), p, fmt.Sprintf("confirmed automatically: %s reached the API again over a new connection", c.ip))
+}
+
 func (s *Server) saveBoot() {
 	if err := s.fw.SaveBoot(s.eng.State()); err != nil {
 		s.log.Error("saving the ruleset for the next boot failed", "err", err)
@@ -110,8 +181,12 @@ func (s *Server) probationStatus() api.Probation {
 	return out
 }
 
-// confirm makes the pending changes permanent.
+// confirm makes the pending changes permanent (a person's confirmation).
 func (s *Server) confirm(ctx context.Context, p *auth.Principal) (int, error) {
+	return s.confirmWith(ctx, p, "")
+}
+
+func (s *Server) confirmWith(ctx context.Context, p *auth.Principal, how string) (int, error) {
 	pr := &s.probation
 	pr.mu.Lock()
 	defer pr.mu.Unlock()
@@ -122,10 +197,14 @@ func (s *Server) confirm(ctx context.Context, p *auth.Principal) (int, error) {
 	if pr.timer != nil {
 		pr.timer.Stop()
 	}
-	pr.pending, pr.restores, pr.deadline, pr.timer = nil, nil, time.Time{}, nil
+	pr.pending, pr.restores, pr.deadline, pr.timer, pr.check = nil, nil, time.Time{}, nil, nil
 	s.saveProbation(ctx)
 	s.saveBoot()
-	s.eng.Audit(ctx, p, "config.confirm", "", "success", fmt.Sprintf("%d changes confirmed", n))
+	detail := fmt.Sprintf("%d changes confirmed", n)
+	if how != "" {
+		detail += "; " + how
+	}
+	s.eng.Audit(ctx, p, "config.confirm", "", "success", detail)
 	return n, nil
 }
 
@@ -136,7 +215,7 @@ func (s *Server) rollback(why string) {
 	pr.mu.Lock()
 	defer pr.mu.Unlock()
 	pending, restores := pr.pending, pr.restores
-	pr.pending, pr.restores, pr.deadline, pr.timer = nil, nil, time.Time{}, nil
+	pr.pending, pr.restores, pr.deadline, pr.timer, pr.check = nil, nil, time.Time{}, nil, nil
 	for i := len(restores) - 1; i >= 0; i-- {
 		if err := s.fw.ApplyScript(ctx, restores[i]); err != nil {
 			s.log.Error("bringing back a table a takeover deleted failed", "err", err)

internal/server/routes.go

@@ -42,6 +42,11 @@ func (s *Server) routes() {
 	m.HandleFunc("POST "+p+"/sets/{ref}/elements", s.authed(s.addElement))
 	m.HandleFunc("DELETE "+p+"/sets/{ref}/elements/{addr}", s.authed(s.deleteElement))
 	m.HandleFunc("GET "+p+"/quotas", s.authed(s.listQuotas))
+	m.HandleFunc("GET "+p+"/connections", s.authed(s.listConnections))
+	m.HandleFunc("POST "+p+"/connections/close", s.authed(s.closeConnections))
+	m.HandleFunc("GET "+p+"/trace", s.authed(s.getTrace))
+	m.HandleFunc("POST "+p+"/trace", s.authed(s.startTrace))
+	m.HandleFunc("DELETE "+p+"/trace", s.authed(s.stopTrace))
 	m.HandleFunc("POST "+p+"/rules/{ref}/reset-quota", s.authed(s.resetQuota))
 	m.HandleFunc("GET "+p+"/changesets", s.authed(s.listChangesets))
 	m.HandleFunc("POST "+p+"/changesets", s.authed(s.createChangeset))
@@ -58,6 +63,7 @@ func (s *Server) routes() {
 	m.HandleFunc("DELETE "+p+"/{kind}/{ref}", s.authed(s.deleteObject))
 	m.HandleFunc("POST "+p+"/{kind}/{ref}/move", s.authed(s.moveObject))
 	m.HandleFunc("GET "+p+"/ruleset", s.authed(s.ruleset))
+	m.HandleFunc("GET "+p+"/matrix", s.authed(s.zoneMatrix))
 	m.HandleFunc("GET "+p+"/history", s.authed(s.listHistory))
 	m.HandleFunc("GET "+p+"/history/{id}", s.authed(s.getHistory))
 	m.HandleFunc("POST "+p+"/history/{id}/revert", s.authed(s.revertHistory))

internal/server/routes_cfg.go

@@ -1,6 +1,10 @@
 package server
 
-import "encoding/json"
+import (
+	"encoding/json"
+
+	"housecarl/internal/trace"
+)
 
 // otherTables lists the tables in an `nft -j list ruleset` other than
 // Housecarl's ("inet main").
@@ -18,7 +22,7 @@ func otherTables(data []byte) []string {
 			continue
 		}
 		var t struct{ Family, Name string }
-		if json.Unmarshal(raw, &t) == nil && !(t.Family == "inet" && t.Name == "housecarl") {
+		if json.Unmarshal(raw, &t) == nil && !(t.Family == "inet" && (t.Name == "housecarl" || t.Name == trace.Table)) {
 			out = append(out, t.Family+" "+t.Name)
 		}
 	}

internal/server/server.go

@@ -67,6 +67,8 @@ type Firewall interface {
 	SaveBoot(st *model.State) error
 	// ResetQuota starts a rule's quota again (perAddress: its sets).
 	ResetQuota(ctx context.Context, ruleID int64, perAddress bool) error
+	// Trace follows packets (internal/trace) until ctx ends.
+	Trace(ctx context.Context, script string, line func(string)) error
 }
 
 // Server is the HTTP server.
@@ -88,6 +90,13 @@ type Server struct {
 
 	probation probation
 	logs      *logRing
+	// unitLinks are where an enabled nftables.service shows (tests change
+	// them).
+	unitLinks []string
+	// conns is the kernel's connection tracking (nil: unavailable).
+	conns Conntrack
+	// tracing is the packet trace running (or the last one).
+	tracing tracing
 	// csMu orders changes to changesets (staging, applying, discarding).
 	csMu  sync.Mutex
 	nowFn func() time.Time
@@ -113,7 +122,7 @@ func New(cfg Config, eng *engine.Engine, fw Firewall, ifaces Interfaces, log *sl
 		log = slog.Default()
 	}
 	s := &Server{cfg: cfg, eng: eng, fw: fw, ifaces: ifaces, store: eng.Store(), log: log, mux: http.NewServeMux(), limiter: newRateLimiter(),
-		web: web, started: time.Now(), logs: newLogRing(2000), nowFn: time.Now}
+		web: web, started: time.Now(), logs: newLogRing(2000), nowFn: time.Now, unitLinks: nftablesUnitLinks}
 	s.routes()
 	return s
 }

internal/server/server_test.go

@@ -37,6 +37,8 @@ type fakeFW struct {
 	deleted    []string
 	restored   []string
 	resets     []string
+	traced     string
+	traceLines []string
 }
 
 func (f *fakeFW) Check(_ context.Context, st *model.State) error { return nil }
@@ -96,6 +98,17 @@ func (f *fakeFW) ResetQuota(_ context.Context, id int64, perAddress bool) error
 	f.resets = append(f.resets, fmt.Sprintf("r%d:%v", id, perAddress))
 	return nil
 }
+func (f *fakeFW) Trace(ctx context.Context, script string, line func(string)) error {
+	f.mu.Lock()
+	f.traced = script
+	lines := f.traceLines
+	f.mu.Unlock()
+	for _, l := range lines {
+		line(l)
+	}
+	<-ctx.Done()
+	return nil
+}
 func (f *fakeFW) TableText(_ context.Context, fam, name string) (string, error) {
 	return "table " + fam + " " + name + " {}", nil
 }
@@ -722,3 +735,262 @@ func TestQuotaResets(t *testing.T) {
 		t.Errorf("reset by hand: %d %v", code, e.fw.resets)
 	}
 }
+
+// TestAutoConfirm: the client that made a change confirms it by itself:
+// with its token, from the same address, over a new connection. Nobody
+// else can, an older token can't, and with manual_confirm there is none.
+func TestAutoConfirm(t *testing.T) {
+	e := newEnv(t)
+	e.srv.probationFor = 5 * time.Second
+	post := func(token, path string, body any) (*http.Response, []byte) {
+		b, _ := json.Marshal(body)
+		req, _ := http.NewRequest("POST", e.ts.URL+api.Prefix+path, bytes.NewReader(b))
+		req.Header.Set("Authorization", "Bearer "+token)
+		req.Header.Set("Content-Type", "application/json")
+		resp, err := http.DefaultClient.Do(req)
+		if err != nil {
+			t.Fatal(err)
+		}
+		defer resp.Body.Close()
+		out, _ := io.ReadAll(resp.Body)
+		return resp, out
+	}
+	resp, body := post(e.token, "/zones", model.Zone{Name: "lan", Interfaces: []string{"eth1"}})
+	var mr api.MutationResponse
+	json.Unmarshal(body, &mr)
+	if mr.ConfirmToken == "" || !resp.Close {
+		t.Fatalf("no token, or the connection stays open: %+v close=%v", mr, resp.Close)
+	}
+	first := mr.ConfirmToken
+	// A newer change: only its token confirms now.
+	_, body = post(e.token, "/zones", model.Zone{Name: "dmz", Interfaces: []string{"eth2"}})
+	json.Unmarshal(body, &mr)
+	if resp, _ := post(e.token, "/confirm", api.ConfirmInput{Token: first}); resp.StatusCode != 409 {
+		t.Errorf("an older token: %d", resp.StatusCode)
+	}
+	// Another user can't use it.
+	ctx := context.Background()
+	other := &model.User{Username: "sam", Grants: auth.Roles["operator"]}
+	e.srv.store.CreateUser(ctx, other)
+	otok, pid, sh := auth.NewToken()
+	e.srv.store.CreateToken(ctx, &model.Token{UserID: other.ID, Name: "t", PublicID: pid, SecretHash: sh})
+	if resp, _ := post(otok, "/confirm", api.ConfirmInput{Token: mr.ConfirmToken}); resp.StatusCode != 403 {
+		t.Errorf("another user's confirmation: %d", resp.StatusCode)
+	}
+	// Not over the change's own connection, nor from elsewhere.
+	c := e.srv.probation.check
+	p := &auth.Principal{User: &model.User{ID: c.userID, Username: c.username}, Method: "token", Token: &model.Token{ID: c.tokenID}}
+	same := httptest.NewRequest("POST", "/", nil)
+	same.RemoteAddr = c.remote
+	if _, err := e.srv.confirmToken(same, p, c.token); err == nil || !strings.Contains(err.Error(), "new one") {
+		t.Errorf("over the same connection: %v", err)
+	}
+	elsewhere := httptest.NewRequest("POST", "/", nil)
+	elsewhere.RemoteAddr = "192.0.2.9:4444"
+	if _, err := e.srv.confirmToken(elsewhere, p, c.token); err == nil || !strings.Contains(err.Error(), "same place") {
+		t.Errorf("from elsewhere: %v", err)
+	}
+	// The client itself, over a new connection: confirmed.
+	resp, body = post(e.token, "/confirm", api.ConfirmInput{Token: mr.ConfirmToken})
+	if resp.StatusCode != 200 || !strings.Contains(string(body), `"confirmed":2`) {
+		t.Fatalf("auto-confirm: %d %s", resp.StatusCode, body)
+	}
+	if pr := e.srv.probationStatus(); len(pr.Pending) != 0 {
+		t.Errorf("still pending: %+v", pr)
+	}
+	// manual_confirm: no token; a person confirms.
+	e.call(e.token, "PATCH", "/settings", map[string]any{"manual_confirm": true}, nil)
+	e.call(e.token, "POST", "/confirm", nil, nil)
+	_, body = post(e.token, "/zones", model.Zone{Name: "guest", Interfaces: []string{"eth3"}})
+	mr = api.MutationResponse{}
+	json.Unmarshal(body, &mr)
+	if mr.ConfirmToken != "" || mr.ConfirmBy == nil {
+		t.Errorf("manual confirmation: %+v", mr)
+	}
+}
+
+// TestStagedUndo: an undo staged in a changeset changes nothing until the
+// changeset is applied; then History links the two entries.
+func TestStagedUndo(t *testing.T) {
+	e := newEnv(t)
+	e.srv.probationFor = 5 * time.Second
+	var mr api.MutationResponse
+	e.call(e.token, "POST", "/zones", model.Zone{Name: "lan", Interfaces: []string{"eth1"}}, &mr)
+	e.call(e.token, "POST", "/confirm", nil, nil)
+	made := mr.HistoryID
+	var cs model.Changeset
+	e.call(e.token, "POST", "/changesets", nil, &cs)
+	q := "?changeset=" + itoa(cs.ID)
+	if code := e.call(e.token, "POST", "/history/"+itoa(made)+"/revert"+q, nil, &mr); code != 200 || !mr.Staged {
+		t.Fatalf("staging the undo: %d %+v", code, mr)
+	}
+	if op := mr.Changeset.Ops[0]; op.Op != model.OpRevert || !strings.HasPrefix(op.Summary, "undo "+itoa(made)+":") {
+		t.Errorf("the operation: %+v", op)
+	}
+	if len(e.eng.State().Zones) != 1 {
+		t.Fatal("a staged undo was applied")
+	}
+	// Staging it again can't work: the changeset already undoes it.
+	var eb api.ErrorBody
+	if code := e.call(e.token, "POST", "/history/"+itoa(made)+"/revert"+q, nil, &eb); code != 409 {
+		t.Errorf("a second undo: %d %+v", code, eb)
+	}
+	if code := e.call(e.token, "POST", "/changesets/"+itoa(cs.ID)+"/apply", nil, &mr); code != 200 || !mr.Applied {
+		t.Fatalf("apply: %d %+v", code, mr)
+	}
+	if len(e.eng.State().Zones) != 0 {
+		t.Error("the undo wasn't applied")
+	}
+	h, _ := e.srv.store.GetHistory(context.Background(), mr.HistoryID)
+	orig, _ := e.srv.store.GetHistory(context.Background(), made)
+	if h.RevertOf != made || orig.RevertedBy != h.ID || !strings.HasPrefix(h.Description, "undo ") {
+		t.Errorf("history: %+v, original %+v", h, orig)
+	}
+}
+
+// TestMatrix: the zone matrix, now and as a changeset will make it.
+func TestMatrixEndpoint(t *testing.T) {
+	e := newEnv(t)
+	e.call(e.token, "POST", "/zones", model.Zone{Name: "lan", Interfaces: []string{"eth1"}}, nil)
+	e.call(e.token, "POST", "/zones", model.Zone{Name: "wan", Interfaces: []string{"wan"}, External: true}, nil)
+	var m api.Matrix
+	e.call(e.token, "GET", "/matrix", nil, &m)
+	cell := func(from, to string) api.MatrixCell {
+		for _, c := range m.Cells {
+			if c.From == from && c.To == to {
+				return c
+			}
+		}
+		return api.MatrixCell{}
+	}
+	if c := cell("lan", "wan"); c.Verdict != "blocked" || c.DecidedBy != "forward policy" {
+		t.Errorf("lan → wan: %+v", c)
+	}
+	var cs model.Changeset
+	e.call(e.token, "POST", "/changesets", nil, &cs)
+	q := "?changeset=" + itoa(cs.ID)
+	e.call(e.token, "POST", "/rules"+q, model.Rule{Chain: "forward", Match: model.Match{InZones: []string{"lan"}, OutZones: []string{"wan"}}, Action: "accept"}, nil)
+	e.call(e.token, "GET", "/matrix"+q, nil, &m)
+	if c := cell("lan", "wan"); c.Verdict != "allowed" {
+		t.Errorf("lan → wan with the changeset: %+v", c)
+	}
+}
+
+// TestHostProblems: an enabled nftables.service, and an imported table
+// still in force, are warned about.
+func TestHostProblems(t *testing.T) {
+	e := newEnv(t)
+	dir := t.TempDir()
+	link := dir + "/nftables.service"
+	e.srv.unitLinks = []string{link}
+	codes := func() string {
+		var st api.Status
+		e.call(e.token, "GET", "/status", nil, &st)
+		var out []string
+		for _, p := range st.Problems {
+			out = append(out, p.Code)
+		}
+		return strings.Join(out, " ")
+	}
+	if c := codes(); strings.Contains(c, "nftables_service") || strings.Contains(c, "takeover_pending") {
+		t.Errorf("warned without cause: %s", c)
+	}
+	os.Symlink("/usr/lib/systemd/system/nftables.service", link) // dangling is fine: Lstat
+	// The fake ruleset lists inet main: imported from it, it still applies.
+	e.srv.store.SetMeta(context.Background(), importedTableKey, "inet main")
+	if c := codes(); !strings.Contains(c, "nftables_service") || !strings.Contains(c, "takeover_pending") {
+		t.Errorf("not warned: %s", c)
+	}
+	e.srv.store.SetMeta(context.Background(), importedTableKey, "inet gone")
+	if c := codes(); strings.Contains(c, "takeover_pending") {
+		t.Errorf("warned about a table that is gone: %s", c)
+	}
+}
+
+type fakeConns struct {
+	list   []api.Connection
+	closed []api.Connection
+}
+
+func (f *fakeConns) List(context.Context) ([]api.Connection, error) { return f.list, nil }
+func (f *fakeConns) Close(_ context.Context, c api.Connection) error {
+	f.closed = append(f.closed, c)
+	return nil
+}
+
+// TestConnections: connections listed and searched; closed one by one, or
+// all a search finds; closing needs config:write, and is audited.
+func TestConnections(t *testing.T) {
+	e := newEnv(t)
+	var code int
+	if code = e.call(e.token, "GET", "/connections", nil, nil); code != 503 {
+		t.Errorf("without conntrack: %d", code)
+	}
+	fc := &fakeConns{list: []api.Connection{
+		{Family: "ipv4", Protocol: "tcp", Source: "192.168.1.50", SourcePort: 51432, Destination: "198.51.100.9", DestinationPort: 443, SNATTo: "203.0.113.10", State: "established"},
+		{Family: "ipv4", Protocol: "udp", Source: "192.168.1.51", SourcePort: 5353, Destination: "192.168.1.1", DestinationPort: 53},
+		{Family: "ipv6", Protocol: "tcp", Source: "2001:db8:1::50", SourcePort: 40000, Destination: "2001:db8:2::9", DestinationPort: 4430},
+	}}
+	e.srv.SetConntrack(fc)
+	var cs []api.Connection
+	e.call(e.token, "GET", "/connections?q=192.168.1.50", nil, &cs)
+	if len(cs) != 1 || cs[0].DestinationPort != 443 {
+		t.Errorf("search by address: %+v", cs)
+	}
+	e.call(e.token, "GET", "/connections?q=:443", nil, &cs)
+	if len(cs) != 1 || cs[0].Source != "192.168.1.50" {
+		t.Errorf("search by port (not :4430): %+v", cs)
+	}
+	var out map[string]int
+	if code := e.call(e.token, "POST", "/connections/close", api.CloseConnections{Search: "tcp"}, &out); code != 200 || out["closed"] != 2 || len(fc.closed) != 2 {
+		t.Errorf("close a search: %d %+v %+v", code, out, fc.closed)
+	}
+	if code := e.call(e.token, "POST", "/connections/close", api.CloseConnections{}, nil); code != 400 {
+		t.Errorf("close nothing said: %d", code)
+	}
+	var tok api.Token
+	e.call(e.token, "POST", "/me/tokens", api.TokenInput{Name: "monitor", Role: "log"}, &tok)
+	if code := e.call(tok.Secret, "GET", "/connections", nil, nil); code != 200 {
+		t.Errorf("listing with log:read: %d", code)
+	}
+	if code := e.call(tok.Secret, "POST", "/connections/close", api.CloseConnections{Search: "udp"}, nil); code != 403 {
+		t.Errorf("closing with log:read: %d", code)
+	}
+}
+
+// TestTrace: a trace runs with the filter's table, collects packets named
+// after Housecarl's objects, and stops when asked; one at a time.
+func TestTrace(t *testing.T) {
+	e := newEnv(t)
+	var mr api.MutationResponse
+	e.call(e.token, "POST", "/rules", model.Rule{Chain: "input", Match: model.Match{Services: []string{"tcp/9999"}}, Action: "drop", Comment: "block 9999"}, &mr)
+	var rule model.Rule
+	json.Unmarshal(mr.Object, &rule)
+	e.fw.traceLines = []string{
+		`trace id 1 inet housecarl input packet: iif "wan" ip saddr 198.51.100.7 ip daddr 203.0.113.10 tcp sport 4000 tcp dport 9999 tcp flags == syn`,
+		`trace id 1 inet housecarl input rule ct state established,related accept comment "hc:b-established" (verdict continue)`,
+		fmt.Sprintf(`trace id 1 inet housecarl input rule tcp dport 9999 counter drop comment "hc:r%d block 9999" (verdict drop)`, rule.ID),
+	}
+	var st api.TraceStatus
+	if code := e.call(e.token, "POST", "/trace", api.TraceInput{Port: 9999, Seconds: 30}, &st); code != 200 || !st.Running {
+		t.Fatalf("start: %d %+v", code, st)
+	}
+	if code := e.call(e.token, "POST", "/trace", api.TraceInput{Port: 22}, nil); code != 409 {
+		t.Errorf("a second trace: %d", code)
+	}
+	time.Sleep(100 * time.Millisecond)
+	e.call(e.token, "GET", "/trace", nil, &st)
+	if len(st.Packets) != 1 || st.Packets[0].Verdict != "dropped" || st.Packets[0].DecidedBy.Label != "rule input #"+itoa(rule.ID)+" (block 9999)" ||
+		st.Packets[0].Steps[0].Label != "built-in: established" {
+		t.Errorf("packets: %+v", st.Packets)
+	}
+	if !strings.Contains(e.fw.traced, "th dport 9999") {
+		t.Errorf("the trace table: %s", e.fw.traced)
+	}
+	if code := e.call(e.token, "DELETE", "/trace", nil, &st); code != 200 || st.Running || len(st.Packets) != 1 {
+		t.Errorf("stop: %d %+v", code, st)
+	}
+	if code := e.call(e.token, "POST", "/trace", api.TraceInput{}, nil); code != 400 {
+		t.Errorf("tracing everything: %d", code)
+	}
+}

internal/trace/trace.go

@@ -0,0 +1,258 @@
+// Package trace follows packets through the whole ruleset, every table's
+// chains, with nftables' own tracing: a table of its own (housecarl-trace,
+// hooked before everything else) marks the packets a filter picks
+// (meta nftrace set 1), and `nft monitor trace` reports each chain and
+// rule they pass, and what decided their fate. Packets are counted per
+// trace id: one packet through the hooks it crosses.
+package trace
+
+import (
+	"fmt"
+	"net/netip"
+	"regexp"
+	"strconv"
+	"strings"
+	"time"
+
+	"housecarl/internal/api"
+)
+
+// Table is the tracing table's name (family inet).
+const Table = "housecarl-trace"
+
+// Script builds the tracing table for a filter: packets from or to the
+// address, on the port (either way), of the protocol. A limit keeps a
+// flood from swamping the trace.
+func Script(in api.TraceInput) (string, error) {
+	var addr netip.Prefix
+	if in.Address != "" {
+		p, err := netip.ParsePrefix(in.Address)
+		if err != nil {
+			a, err2 := netip.ParseAddr(in.Address)
+			if err2 != nil {
+				return "", fmt.Errorf("address %q: give an address or a prefix (192.168.1.50, 2001:db8::/64)", in.Address)
+			}
+			p = netip.PrefixFrom(a, a.BitLen())
+		}
+		addr = p.Masked()
+	}
+	if in.Port < 0 || in.Port > 65535 {
+		return "", fmt.Errorf("port %d: 1 to 65535", in.Port)
+	}
+	proto := strings.ToLower(in.Protocol)
+	switch proto {
+	case "", "tcp", "udp", "sctp", "icmp", "icmpv6":
+	default:
+		return "", fmt.Errorf("protocol %q: tcp, udp, sctp, icmp, icmpv6, or nothing for any", in.Protocol)
+	}
+	if !addr.IsValid() && in.Port == 0 {
+		return "", fmt.Errorf("say what to trace: an address, a port, or both")
+	}
+	if in.Port != 0 && (proto == "icmp" || proto == "icmpv6") {
+		return "", fmt.Errorf("ICMP has no ports")
+	}
+	var l4 string
+	switch {
+	case proto != "":
+		l4 = "meta l4proto " + proto
+	case in.Port != 0:
+		l4 = "meta l4proto { tcp, udp, sctp }"
+	}
+	addrs := []string{""}
+	if addr.IsValid() {
+		ip := "ip"
+		if addr.Addr().Is6() {
+			ip = "ip6"
+		}
+		addrs = []string{ip + " saddr " + addr.String(), ip + " daddr " + addr.String()}
+	}
+	ports := []string{""}
+	if in.Port != 0 {
+		ports = []string{fmt.Sprintf("th dport %d", in.Port), fmt.Sprintf("th sport %d", in.Port)}
+	}
+	var rules []string
+	for _, a := range addrs {
+		for _, p := range ports {
+			parts := []string{}
+			for _, x := range []string{a, l4, p, "limit rate 20/second", "meta nftrace set 1"} {
+				if x != "" {
+					parts = append(parts, x)
+				}
+			}
+			rules = append(rules, "\t\t"+strings.Join(parts, " "))
+		}
+	}
+	body := strings.Join(rules, "\n")
+	return fmt.Sprintf(`table inet %[1]s {}
+delete table inet %[1]s
+table inet %[1]s {
+	chain pre {
+		type filter hook prerouting priority -301; policy accept;
+%[2]s
+	}
+	chain out {
+		type filter hook output priority -301; policy accept;
+%[2]s
+	}
+}
+`, Table, body), nil
+}
+
+// ---- reading `nft monitor trace`
+
+// "trace id 38ff3662 inet housecarl input rule … (verdict accept)"
+var lineRe = regexp.MustCompile(`^trace id (\S+) (\S+) (\S+) (\S+) (packet: (.*)|rule (.*) \(verdict ([^)]*)\)|policy (\S+)|(.*))$`)
+
+var (
+	fieldRe = regexp.MustCompile(`\b(iif|oif) "([^"]*)"`)
+	addrRe  = regexp.MustCompile(`\b(ip6?) (saddr|daddr) (\S+)`)
+	portRe  = regexp.MustCompile(`\b(tcp|udp|sctp) (sport|dport) (\d+)`)
+	flagsRe = regexp.MustCompile(`\btcp flags == (\S+)`)
+	icmpRe  = regexp.MustCompile(`\b(icmp|icmpv6) type (\S+)`)
+	protoRe = regexp.MustCompile(`\bmeta l4proto (\S+)|\bip protocol (\S+)|\bip6 nexthdr (\S+)`)
+	markRe  = regexp.MustCompile(`comment "hc:(\S+)`)
+)
+
+// Tracer collects packets from trace lines.
+type Tracer struct {
+	// Max is how many packets it keeps (0: 200).
+	Max     int
+	Now     func() time.Time
+	packets []*api.TracePacket
+	byID    map[string]*api.TracePacket
+	// seen are the chains a packet entered: the kernel reuses trace ids
+	// (they come from the packet's buffer), so entering one again means
+	// another packet.
+	seen map[*api.TracePacket]map[string]bool
+}
+
+// Packets returns what was traced so far, oldest first.
+func (t *Tracer) Packets() []api.TracePacket {
+	out := make([]api.TracePacket, 0, len(t.packets))
+	for _, p := range t.packets {
+		out = append(out, *p)
+	}
+	return out
+}
+
+// Full reports whether it keeps no more packets.
+func (t *Tracer) Full() bool {
+	max := t.Max
+	if max == 0 {
+		max = 200
+	}
+	return len(t.packets) >= max
+}
+
+// Line reads one line of `nft monitor trace`. The tracing table's own
+// steps are left out.
+func (t *Tracer) Line(line string) {
+	m := lineRe.FindStringSubmatch(strings.TrimSpace(line))
+	if m == nil {
+		return
+	}
+	id, table, chain := m[1], m[2]+" "+m[3], m[4]
+	if m[3] == Table {
+		return
+	}
+	if t.byID == nil {
+		t.byID, t.seen = map[string]*api.TracePacket{}, map[*api.TracePacket]map[string]bool{}
+	}
+	p := t.byID[id]
+	isPacket := m[6] != "" || strings.HasPrefix(m[5], "packet:")
+	if p != nil && isPacket && t.seen[p][table+" "+chain] {
+		p = nil // the same id, another packet
+	}
+	if p == nil {
+		if t.Full() {
+			return
+		}
+		now := time.Now
+		if t.Now != nil {
+			now = t.Now
+		}
+		p = &api.TracePacket{ID: id, Time: now(), Verdict: "accepted"}
+		t.byID[id] = p
+		t.seen[p] = map[string]bool{}
+		t.packets = append(t.packets, p)
+	}
+	switch {
+	case isPacket:
+		t.seen[p][table+" "+chain] = true
+		readPacket(p, m[6])
+	case m[7] != "":
+		step := api.TraceStep{Table: table, Chain: chain, Kind: "rule", Rule: m[7], Verdict: m[8]}
+		if mk := markRe.FindStringSubmatch(m[7]); mk != nil {
+			step.Mark = strings.TrimSuffix(mk[1], `"`)
+		}
+		p.Steps = append(p.Steps, step)
+		decide(p, step)
+	case m[9] != "":
+		step := api.TraceStep{Table: table, Chain: chain, Kind: "policy", Verdict: m[9]}
+		p.Steps = append(p.Steps, step)
+		decide(p, step)
+	}
+}
+
+// decide notes a packet's fate: the first drop (or reject) ends it.
+func decide(p *api.TracePacket, s api.TraceStep) {
+	if p.Verdict == "dropped" {
+		return
+	}
+	switch strings.Fields(s.Verdict + " ")[0] {
+	case "drop", "reject":
+		p.Verdict, p.DecidedBy = "dropped", s
+	case "accept":
+		// What let it through is a filter chain's accept: Housecarl's
+		// input, forward or output (NAT chains accept everything they
+		// see). Another table's accept counts when Housecarl's says
+		// nothing.
+		filter := s.Table == "inet housecarl" && (s.Chain == "input" || s.Chain == "forward" || s.Chain == "output")
+		if filter || p.DecidedBy.Verdict == "" && s.Table != "inet housecarl" {
+			p.DecidedBy = s
+		}
+	}
+}
+
+// readPacket fills in a packet's headers from its "packet:" line (once:
+// the first hook's view, before any NAT).
+func readPacket(p *api.TracePacket, f string) {
+	for _, m := range fieldRe.FindAllStringSubmatch(f, -1) {
+		if m[1] == "iif" && p.In == "" {
+			p.In = m[2]
+		}
+		if m[1] == "oif" && p.Out == "" {
+			p.Out = m[2]
+		}
+	}
+	if p.Source != "" {
+		return
+	}
+	for _, m := range addrRe.FindAllStringSubmatch(f, -1) {
+		if m[2] == "saddr" {
+			p.Source = m[3]
+		} else {
+			p.Destination = m[3]
+		}
+	}
+	for _, m := range portRe.FindAllStringSubmatch(f, -1) {
+		p.Protocol = m[1]
+		n, _ := strconv.Atoi(m[3])
+		if m[2] == "sport" {
+			p.SourcePort = n
+		} else {
+			p.DestinationPort = n
+		}
+	}
+	if m := flagsRe.FindStringSubmatch(f); m != nil {
+		p.TCPFlags = m[1]
+	}
+	if m := icmpRe.FindStringSubmatch(f); m != nil {
+		p.Protocol, p.ICMPType = m[1], m[2]
+	}
+	if p.Protocol == "" {
+		if m := protoRe.FindStringSubmatch(f); m != nil {
+			p.Protocol = strings.TrimSpace(m[1] + m[2] + m[3])
+		}
+	}
+}

internal/trace/trace_test.go

@@ -0,0 +1,89 @@
+package trace
+
+import (
+	"strings"
+	"testing"
+
+	"housecarl/internal/api"
+)
+
+// Lines nft 1.1.3's `nft monitor trace` printed for a connection to a
+// blocked port, and for one that loopback accepted.
+const sample = `trace id 38ff3662 inet housecarl-trace pre packet: iif "wan" ip saddr 198.51.100.7 ip daddr 203.0.113.10 ip dscp cs0 ip ecn not-ect ip ttl 64 ip id 44978 ip protocol 6 ip length 60 tcp sport 43168 tcp dport 9999 tcp flags == syn tcp window 65495
+trace id 38ff3662 inet housecarl-trace pre rule ip saddr 198.51.100.7 meta nftrace set 1 (verdict continue)
+trace id 38ff3662 inet housecarl-trace pre policy accept
+trace id 38ff3662 inet housecarl input packet: iif "wan" ip saddr 198.51.100.7 ip daddr 203.0.113.10 ip dscp cs0 ip ecn not-ect ip ttl 64 ip id 44978 ip protocol 6 ip length 60 tcp sport 43168 tcp dport 9999 tcp flags == syn tcp window 65495
+trace id 38ff3662 inet housecarl input rule ct state established,related counter packets 0 bytes 0 accept comment "hc:b-established" (verdict continue)
+trace id 38ff3662 inet housecarl input rule tcp dport 9999 counter packets 1 bytes 60 drop comment "hc:r12 block 9999" (verdict drop)
+trace id a232d9a0 inet housecarl output packet: oif "lo" ip saddr 127.0.0.1 ip daddr 127.0.0.1 ip protocol 6 tcp sport 43168 tcp dport 22 tcp flags == syn
+trace id a232d9a0 inet housecarl output policy accept
+trace id a232d9a0 ip other input rule ip saddr 127.0.0.1 accept (verdict accept)
+trace id 77 inet housecarl forward packet: iif "lan" oif "wan" ip saddr 192.168.1.20 ip daddr 198.51.100.10 tcp sport 3500 tcp dport 7000 tcp flags == syn
+trace id 77 inet housecarl forward rule ip saddr @trusted_v4 counter accept comment "hc:r40 trusted" (verdict accept)
+trace id 77 inet housecarl srcnat rule oifname "wan" counter masquerade comment "hc:s91" (verdict accept)`
+
+func TestTracer(t *testing.T) {
+	var tr Tracer
+	for _, l := range strings.Split(sample, "\n") {
+		tr.Line(l)
+	}
+	ps := tr.Packets()
+	if len(ps) != 3 {
+		t.Fatalf("packets: %+v", ps)
+	}
+	p := ps[0]
+	if p.In != "wan" || p.Source != "198.51.100.7" || p.DestinationPort != 9999 || p.Protocol != "tcp" || p.TCPFlags != "syn" {
+		t.Errorf("headers: %+v", p)
+	}
+	if len(p.Steps) != 2 || p.Verdict != "dropped" || p.DecidedBy.Mark != "r12" || p.DecidedBy.Chain != "input" {
+		t.Errorf("steps: %+v", p)
+	}
+	if q := ps[1]; q.Verdict != "accepted" || q.Out != "lo" || len(q.Steps) != 2 || q.DecidedBy.Kind != "policy" {
+		t.Errorf("second: %+v", q)
+	}
+	// Accepted by the forward rule, not by source NAT, which came after.
+	if q := ps[2]; q.Verdict != "accepted" || q.DecidedBy.Mark != "r40" {
+		t.Errorf("third: decided by %+v", q.DecidedBy)
+	}
+}
+
+// The kernel reuses trace ids: a connection's next packet can come with
+// the same one. Entering a chain again starts another packet.
+func TestReusedID(t *testing.T) {
+	var tr Tracer
+	for _, l := range strings.Split(`trace id 5 inet housecarl output packet: oif "lo" ip saddr 127.0.0.1 ip daddr 127.0.0.1 tcp sport 49078 tcp dport 8070 tcp flags == syn
+trace id 5 inet housecarl output policy accept
+trace id 5 inet housecarl srcnat packet: oif "lo" ip saddr 127.0.0.1 ip daddr 127.0.0.1 tcp sport 49078 tcp dport 8070 tcp flags == syn
+trace id 5 inet housecarl srcnat policy accept
+trace id 5 inet housecarl output packet: oif "lo" ip saddr 127.0.0.1 ip daddr 127.0.0.1 tcp sport 49078 tcp dport 8070 tcp flags == ack
+trace id 5 inet housecarl output policy accept`, "\n") {
+		tr.Line(l)
+	}
+	ps := tr.Packets()
+	if len(ps) != 2 || len(ps[0].Steps) != 2 || len(ps[1].Steps) != 1 || ps[1].TCPFlags != "ack" {
+		t.Errorf("packets: %+v", ps)
+	}
+}
+
+func TestScript(t *testing.T) {
+	s, err := Script(api.TraceInput{Address: "192.168.1.50", Port: 443})
+	if err != nil {
+		t.Fatal(err)
+	}
+	for _, want := range []string{
+		"table inet housecarl-trace {}\ndelete table inet housecarl-trace\n",
+		"type filter hook prerouting priority -301; policy accept;",
+		"ip saddr 192.168.1.50/32 meta l4proto { tcp, udp, sctp } th dport 443 limit rate 20/second meta nftrace set 1",
+		"ip daddr 192.168.1.50/32 meta l4proto { tcp, udp, sctp } th sport 443 limit rate 20/second meta nftrace set 1",
+	} {
+		if !strings.Contains(s, want) {
+			t.Errorf("missing %q in\n%s", want, s)
+		}
+	}
+	if _, err := Script(api.TraceInput{}); err == nil {
+		t.Error("tracing everything")
+	}
+	if _, err := Script(api.TraceInput{Port: 22, Protocol: "icmp"}); err == nil {
+		t.Error("ICMP with a port")
+	}
+}

internal/validate/validate.go

@@ -85,6 +85,36 @@ func Check(st *model.State, env *Env) []Problem {
 	return c.out
 }
 
+// rawIgnores names what a raw rule has besides its text, which the
+// ruleset wouldn't use (the action too is the text's; it isn't checked).
+func rawIgnores(r *model.Rule) []string {
+	m := r.Match
+	var out []string
+	for _, x := range []struct {
+		name string
+		set  bool
+	}{
+		{"zones", len(m.InZones) > 0 || len(m.OutZones) > 0},
+		{"addresses", len(m.Sources) > 0 || len(m.Destinations) > 0},
+		{"services", len(m.Services) > 0},
+		{"family", m.Family != ""},
+		{"connection state", len(r.CtState) > 0 || len(r.CtStatus) > 0},
+		{"time window", len(r.Days) > 0 || r.HourFrom != "" || r.HourTo != ""},
+		{"rate limit", r.Limit != nil},
+		{"connection limit", r.ConnLimit != 0},
+		{"quota", r.Quota != nil},
+		{"address grouping", r.GroupV4 != 0 || r.GroupV6 != 0},
+		{"log", r.Log || r.LogPrefix != ""},
+		{"reject with", r.RejectWith != ""},
+		{"every packet", r.EveryPacket},
+	} {
+		if x.set {
+			out = append(out, x.name)
+		}
+	}
+	return out
+}
+
 // maxByteRate is the largest byte rate per period nftables (1.1.3) can
 // hold: the kernel's token bucket multiplies the rate by the period.
 var maxByteRate = map[string]int64{"second": 4294967294, "minute": 307445734, "hour": 5124095, "day": 213503}
@@ -202,6 +232,9 @@ func (c *checker) sets() {
 				c.add(Error, "set_entry", obj, "%v", err)
 			}
 		}
+		if len(s.Entries) > 0 && s.Dynamic {
+			c.add(Error, "set_dynamic_entries", obj, "a dynamic set gets its entries at run time (hc set entries add, the Entries button), each with its timeout; leave out the configured ones, or make it a set that isn't dynamic")
+		}
 		if len(s.Entries) == 0 && !s.Dynamic {
 			c.add(Warning, "set_empty", obj, "the set is empty: rules using it match nothing")
 		}
@@ -331,6 +364,9 @@ func (c *checker) rule(r *model.Rule) {
 		if strings.ContainsAny(r.Raw, ";\n\r") {
 			c.add(Error, "raw_rule", obj, "a raw rule is one nftables rule on one line, without ;")
 		}
+		if ignored := rawIgnores(r); len(ignored) > 0 {
+			c.add(Error, "raw_conflict", obj, "a raw rule is used as written, so its %s would be ignored: write them in the nftables rule, or leave them out", strings.Join(ignored, ", "))
+		}
 		return
 	}
 	in := r.Chain != model.ChainOutput
@@ -399,11 +435,14 @@ func (c *checker) rule(r *model.Rule) {
 			c.add(Error, "rule_quota_reset", obj, "the day of the month is 1 to 31 (a later day than a month has means its last day)")
 		}
 	}
-	if r.CountsTraffic() && r.Action == model.ActionAccept {
+	switch {
+	case r.CountsTraffic() && r.Action == model.ActionAccept:
 		c.add(Error, "rule_traffic_accept", obj, "a rule with a quota or a byte rate counts every packet, before established connections are accepted, so it can only drop, reject or log; let the traffic in with a rule of its own")
+	case r.ChecksEveryPacket() && r.Action == model.ActionAccept:
+		c.add(Error, "rule_every_packet_accept", obj, "a rule checked against every packet comes before established connections are accepted, so accepting there would skip every later rule; it can drop, reject or log")
 	}
-	if r.CountsTraffic() && r.Chain == model.ChainForward && len(c.st.Settings.FlowtableDevices) > 0 && !r.Disabled {
+	if r.ChecksEveryPacket() && r.Chain == model.ChainForward && len(c.st.Settings.FlowtableDevices) > 0 && !r.Disabled {
-		c.add(Warning, "flowtable_traffic", obj, "the flowtable's fast path skips the forward chain once a connection is established, so this quota or byte rate only counts each connection's first packets; turn the flowtable off (settings) to count all of it")
+		c.add(Warning, "flowtable_traffic", obj, "the flowtable's fast path skips the forward chain once a connection is established, so this rule only sees each connection's first packets; turn the flowtable off (settings) for it to see all of them")
 	}
 	if r.GroupV4 < 0 || r.GroupV4 > 32 || r.GroupV6 < 0 || r.GroupV6 > 128 {
 		c.add(Error, "rule_group", obj, "group addresses by a prefix length: 1 to 32 for IPv4, 1 to 128 for IPv6")

internal/validate/validate_test.go

@@ -66,6 +66,7 @@ func TestCodes(t *testing.T) {
 		{"set_entry", func(st *model.State) { st.Sets[0].Entries = []string{"10.0.10.5/24"} }},
 		{"set_empty", func(st *model.State) { st.Sets[0].Entries = nil }},
 		{"set_timeout", func(st *model.State) { st.Sets[0].Timeout = 60 }},
+		{"set_dynamic_entries", func(st *model.State) { st.Sets[0].Dynamic = true }},
 		{"service_empty", func(st *model.State) { st.Services[0].Entries = nil }},
 		{"service_entry", func(st *model.State) { st.Services[0].Entries = []string{"tcp/99999"} }},
 		{"unknown_zone", func(st *model.State) { st.Rules[1].InZones = []string{"dmz"} }},
@@ -83,6 +84,7 @@ func TestCodes(t *testing.T) {
 		{"rule_chain", func(st *model.State) { rule(st).Chain = "prerouting" }},
 		{"rule_action", func(st *model.State) { rule(st).Action = "allow" }},
 		{"raw_rule", func(st *model.State) { rule(st).Raw = "tcp dport 22 accept; udp dport 53 accept" }},
+		{"raw_conflict", func(st *model.State) { rule(st).Raw = "tcp dport 22 accept" }}, // it still has sources and services
 		{"early_conntrack", func(st *model.State) { rule(st).Chain = model.ChainEarly; rule(st).CtState = []string{"new"} }},
 		{"early_reject", func(st *model.State) { rule(st).Chain = model.ChainEarly; rule(st).Action = model.ActionReject }},
 		{"ct_state", func(st *model.State) { rule(st).CtState = []string{"fresh"} }},
@@ -102,6 +104,7 @@ func TestCodes(t *testing.T) {
 		{"rule_quota_reset", func(st *model.State) { rule(st).Quota = &model.Quota{Bytes: 1, Reset: "daily", Day: 3} }},
 		{"rule_group", func(st *model.State) { rule(st).GroupV6 = 129 }},
 		{"rule_traffic_accept", func(st *model.State) { rule(st).Quota = &model.Quota{Bytes: 1 << 20} }},
+		{"rule_every_packet_accept", func(st *model.State) { rule(st).EveryPacket = true }},
 		{"flowtable_traffic", func(st *model.State) {
 			st.Settings.FlowtableDevices = []string{"wan"}
 			st.Rules[1].Action, st.Rules[1].Quota = model.ActionDrop, &model.Quota{Bytes: 1 << 20, Over: true}

internal/web/static/assets/api.js

@@ -22,11 +22,13 @@ export class APIError extends Error {
 let onUnauthorized = () => {};
 export function setUnauthorizedHandler(fn) { onUnauthorized = fn; }
 
-// onChange is told about every applied change (the probation banner).
+// onChange is told about every applied change (the probation banner), with
+// the answer (its confirm_token, to confirm it by itself).
 let onChange = () => {};
 export function setChangeHandler(fn) { onChange = fn; }
 
-// api calls the REST API. opts: {body, query, stage (a changeset ID)}.
+// api calls the REST API. opts: {body, query, stage (a changeset ID),
+// timeout (ms)}.
 export async function api(method, path, opts = {}) {
   const headers = { "Accept": "application/json" };
   if (opts.stage) headers["X-Housecarl-Changeset"] = String(opts.stage);
@@ -43,7 +45,13 @@ export async function api(method, path, opts = {}) {
     const s = q.toString();
     if (s) url += (url.includes("?") ? "&" : "?") + s;
   }
-  const resp = await fetch(url, { method, headers, body, credentials: "same-origin" });
+  let signal;
+  if (opts.timeout) {
+    const ac = new AbortController();
+    setTimeout(() => ac.abort(), opts.timeout);
+    signal = ac.signal;
+  }
+  const resp = await fetch(url, { method, headers, body, credentials: "same-origin", signal });
   if (resp.status === 204) return null;
   const ct = resp.headers.get("Content-Type") || "";
   const data = ct.includes("application/json") ? await resp.json() : await resp.text();
@@ -173,11 +181,12 @@ export function report(r) {
     const n = ((r.changeset || {}).ops || []).length;
     toast("Staged: " + ((r.changeset.ops || [])[n - 1] || {}).summary + " — " + (n === 1 ? "1 pending change" : n + " pending changes") + " to review and apply");
   } else if (r.dry_run) toast("Checked by Housecarl and nftables: nothing was changed");
-  else if (r.applied && r.confirm_by) toast("Applied on probation (history #" + r.history_id + "): confirm it in the banner", "warn", 6000);
+  else if (r.applied && r.confirm_by && !r.confirm_token) toast("Applied on probation (history #" + r.history_id + "): confirm it in the banner", "warn", 6000);
+  else if (r.applied && r.confirm_by) toast("Applied (history #" + r.history_id + ")");
   else if (r.applied) toast("Applied and saved (history #" + r.history_id + ")");
   else if (!(r.changes || []).length) toast("Nothing to change");
   for (const w of r.warnings || []) toast(w.message + (w.object ? " (" + w.object + ")" : ""), "warn", 10000);
-  if (r.applied) onChange();
+  if (r.applied) onChange(r);
 }
 
 // can reports whether the user may read (or write) a resource.

internal/web/static/assets/app.js

@@ -3,6 +3,9 @@ import { h, mount, toast, errorBox, modal } from "./ui.js";
 import { wordmark, logo } from "./brand.js";
 import { api, get, state, can, setUnauthorizedHandler, setChangeHandler, loadDraft, onDraftChange, setImmediate, useDraft } from "./api.js";
 import * as changes from "./pages/changes.js";
+import * as connections from "./pages/connections.js";
+import * as tracePage from "./pages/trace.js";
+import * as matrixPage from "./pages/matrix.js";
 import * as dashboard from "./pages/dashboard.js";
 import * as rules from "./pages/rules.js";
 import * as nat from "./pages/nat.js";
@@ -22,10 +25,13 @@ const routes = [
   { pattern: /^$/, page: dashboard, nav: "dashboard" },
   { pattern: /^rules$/, page: rules, nav: "rules" },
   { pattern: /^nat$/, page: nat, nav: "nat" },
+  { pattern: /^matrix$/, page: matrixPage, nav: "matrix" },
   { pattern: /^zones$/, page: zones, nav: "zones" },
   { pattern: /^sets$/, page: sets, nav: "sets" },
   { pattern: /^services$/, page: services, nav: "services" },
   { pattern: /^log$/, page: log, nav: "log" },
+  { pattern: /^connections$/, page: connections, nav: "connections" },
+  { pattern: /^trace$/, page: tracePage, nav: "trace" },
   { pattern: /^settings$/, page: settings, nav: "settings" },
   { pattern: /^import$/, page: importPage, nav: "import" },
   { pattern: /^changes$/, page: changes, nav: "changes" },
@@ -41,13 +47,18 @@ const routes = [
 const navItems = [
   ["dashboard", "#/", "Dashboard"],
   ["rules", "#/rules", "Rules", "config"],
+  ["matrix", "#/matrix", "Matrix", "config"],
   ["nat", "#/nat", "NAT", "config"],
   { menu: "Objects", items: [
     ["zones", "#/zones", "Zones", "config"],
     ["sets", "#/sets", "Address sets", "config"],
     ["services", "#/services", "Services", "config"],
   ] },
+  { menu: "Traffic", items: [
     ["log", "#/log", "Log", "log"],
+    ["connections", "#/connections", "Connections", "log"],
+    ["trace", "#/trace", "Trace", "log"],
+  ] },
   ["changes", "#/changes", "Changes", "config"],
   ["history", "#/history", "History", "config"],
   { menu: "Server", items: [
@@ -158,10 +169,11 @@ export async function applyDraft(btn, d, description) {
   try {
     const r = await api("POST", "/changesets/" + d.id + "/apply", { body: { description: description || "" } });
     useDraft(null);
-    if (r.confirm_by) toast("Applied changeset #" + d.id + " on probation (history #" + r.history_id + "): confirm it in the banner", "warn", 8000);
+    if (r.confirm_by && !r.confirm_token) toast("Applied changeset #" + d.id + " on probation (history #" + r.history_id + "): confirm it in the banner", "warn", 8000);
     else toast("Applied changeset #" + d.id + " (history #" + r.history_id + ")");
     for (const w of r.warnings || []) toast(w.message + (w.object ? " (" + w.object + ")" : ""), "warn", 10000);
-    await refreshProbation();
+    if (r.confirm_token) await autoConfirm(r.confirm_token);
+    else await refreshProbation();
     return r;
   } catch (e) {
     btn.disabled = false;
@@ -176,7 +188,25 @@ export async function applyDraft(btn, d, description) {
 
 // ---- probation: changes made here roll back unless confirmed
 
-let probationTimer = null, deadline = 0, pending = 0;
+let probationTimer = null, deadline = 0, pending = 0, checking = false;
+
+// autoConfirm confirms a change with its token. The server closed the
+// change's connection, so this request opens a new one through the new
+// ruleset: it only gets through if this browser can still reach
+// Housecarl. If it can't, the change rolls back by itself.
+export async function autoConfirm(token) {
+  checking = true;
+  await refreshProbation();
+  try {
+    await api("POST", "/confirm", { body: { token }, timeout: 10000 });
+    checking = false;
+    toast("Confirmed automatically: this browser reached Housecarl again over a new connection");
+  } catch (e) {
+    checking = false;
+    toast("Couldn't confirm the change (" + (e.name === "AbortError" ? "no answer over a new connection" : e.message) + "): unless it is confirmed, it rolls back by itself", "warn", 10000);
+  }
+  await refreshProbation();
+}
 
 // refreshProbation asks whether changes wait for confirmation, and shows
 // the banner with a countdown while they do.
@@ -204,6 +234,10 @@ function drawBanner() {
   }
   bannerEl.classList.remove("hidden");
   const write = can("config", true);
+  if (checking) {
+    mount(bannerEl, h("strong", null, "Checking you can still reach Housecarl…"), " The change is confirmed once this browser gets through over a new connection.");
+    return;
+  }
   mount(bannerEl,
     h("strong", null, pending === 1 ? "1 change is on probation" : pending + " changes are on probation"),
     " — rolled back in ", h("span", { class: "mono" }, left + "s"), " unless you confirm them. If this page stops answering, wait: the firewall puts itself back.",
@@ -303,7 +337,7 @@ export function rerender() {
   if (current.page) route();
 }
 
-setChangeHandler(() => refreshProbation());
+setChangeHandler((r) => r && r.confirm_token ? autoConfirm(r.confirm_token) : refreshProbation());
 
 setUnauthorizedHandler(() => {
   if (state.me) {

internal/web/static/assets/pages/connections.js

@@ -0,0 +1,49 @@
+// Connections: what the kernel tracks, and closing them so the rules
+// check them again.
+import { h, mount, input, debounce, toast, confirmDialog, badge } from "../ui.js";
+import { get, api, can } from "../api.js";
+import { onLeave } from "../app.js";
+import { fmtSeconds } from "../widgets.js";
+
+export async function render(el, params, query) {
+  const search = input({ type: "search", placeholder: "address, :port, protocol, state…", value: query.get("q") || "" });
+  const out = h("div");
+  const count = h("span", { class: "muted small" });
+  const write = can("config", true);
+  let shown = [];
+  const hostPort = (a, p) => !p ? a : a.includes(":") ? "[" + a + "]:" + p : a + ":" + p;
+  const close = async (list, what) => {
+    if (!await confirmDialog("Close connections", "Close " + what + "? Their next packet is checked by the rules as they are now: what they block is stopped, what they allow carries on.", "Close", "danger")) return;
+    try {
+      const r = await api("POST", "/connections/close", { body: list ? { connections: list } : { search: search.value.trim() } });
+      toast("Closed " + r.closed + (r.closed === 1 ? " connection" : " connections"));
+      draw();
+    } catch (e) { toast(e.message, "err"); }
+  };
+  const draw = async () => {
+    shown = await get("/connections", { q: search.value.trim(), limit: 500 });
+    count.textContent = shown.length === 1 ? "1 connection" : shown.length + " connections";
+    mount(out, shown.length ? h("div", { class: "table-wrap" }, h("table", null,
+      h("thead", null, h("tr", null, ["Protocol", "From", "To", "NAT", "State", "Expires", ""].map((t) => h("th", null, t)))),
+      h("tbody", null, shown.map((c) => h("tr", null,
+        h("td", null, c.protocol),
+        h("td", { class: "mono" }, hostPort(c.source, c.source_port)),
+        h("td", { class: "mono" }, hostPort(c.destination, c.destination_port)),
+        h("td", { class: "mono small" }, c.dnat_to ? "to " + hostPort(c.dnat_to, c.dnat_port || c.destination_port) : "", c.snat_to ? (c.dnat_to ? ", " : "") + "as " + c.snat_to : ""),
+        h("td", null, c.state || "", c.assured ? null : [" ", badge("one way", "")]),
+        h("td", { class: "small" }, fmtSeconds(c.timeout)),
+        h("td", { class: "actions-cell" }, write ? h("button", { class: "small reveal", onclick: () => close([c], "this connection") }, "Close") : null)))))) :
+      h("div", { class: "empty" }, search.value.trim() ? "No connection matches." : "No connections tracked."));
+  };
+  search.addEventListener("input", debounce(draw, 300));
+  mount(el,
+    h("div", { class: "page-head" },
+      h("div", { class: "grow" }, h("h1", null, "Connections"),
+        h("div", { class: "muted" }, "The connections the firewall tracks. Their packets are let through by the built-in for established connections, without the rules being checked again, so a rule added later doesn't stop them. Close them, and their next packet is checked by the rules as they are now.")),
+      write ? h("div", { class: "actions" }, h("button", { class: "danger", onclick: () => close(null, search.value.trim() ? "every connection matching “" + search.value.trim() + "”" : "every connection") }, "Close all shown")) : null),
+    h("div", { class: "filter-row" }, search, count),
+    h("div", { class: "panel" }, out));
+  await draw();
+  const t = setInterval(() => { if (!document.hidden && document.activeElement !== search) draw().catch(() => {}); }, 5000);
+  onLeave(() => clearInterval(t));
+}

internal/web/static/assets/pages/history.js

@@ -1,6 +1,6 @@
 // History: every configuration change, with undo.
 import { h, mount, table, time, confirmDialog, modal, errorBox, badge } from "../ui.js";
-import { get, change, can } from "../api.js";
+import { get, change, can, state } from "../api.js";
 import { rerender } from "../app.js";
 
 export async function render(el, params) {
@@ -20,10 +20,10 @@ export async function render(el, params) {
 }
 
 async function undo(e) {
-  if (!await confirmDialog("Undo change #" + e.id, "Undo “" + e.description + "”? This is recorded as a new change, which can be undone too.", "Undo")) return;
+  if (!await confirmDialog("Undo change #" + e.id, "Undo “" + e.description + "”? This is recorded as a new change, which can be undone too." + (!state.immediate ? " It is added to your pending changes, to review and apply." : ""), "Undo")) return;
   try {
-    // An undo can't be staged: it is applied at once.
+    // Like any edit: staged for review, or applied at once in live mode.
-    await change("POST", "/history/" + e.id + "/revert", undefined, { immediate: true });
+    await change("POST", "/history/" + e.id + "/revert");
     rerender();
   } catch (err) { modal("Can't undo", errorBox(err)); }
 }

internal/web/static/assets/pages/import.js

@@ -1,7 +1,7 @@
 // Import: an existing nftables ruleset as Housecarl's configuration.
 import { h, mount, field, textarea, input, checkbox, errorBox, copyBox } from "../ui.js";
 import { api } from "../api.js";
-import { refreshProbation } from "../app.js";
+import { refreshProbation, autoConfirm } from "../app.js";
 
 export async function render(el) {
   const json = textarea("", { rows: 6, class: "mono", placeholder: "the output of: sudo nft -j list ruleset" });
@@ -21,12 +21,13 @@ export async function render(el) {
       mount(out, h("div", { class: "msg " + (dry ? "info" : "ok") },
         dry ? "Would import: " : "Imported: ",
         `${st.zones.length} zones, ${st.sets.length} sets, ${st.services.length} services, ${st.rules.length} rules, ${st.forwards.length} port forwards, ${st.snats.length} source NAT rules, ${st.redirects.length} redirects.`,
-        r.confirm_by ? " It is on probation: confirm it in the banner." : ""),
+        r.confirm_by && !r.confirm_token ? " It is on probation: confirm it in the banner." : ""),
       r.notes.length ? h("div", { class: "panel" }, h("h2", null, "Notes"), h("ul", null, r.notes.map((n) => h("li", null, n)))) : null,
       r.raw.length ? h("div", { class: "panel" }, h("h2", null, "Kept as raw rules"), h("ul", { class: "mono small" }, r.raw.map((n) => h("li", null, n)))) : null,
       !dry ? h("p", null, "Check the ", h("a", { href: "#/rules" }, "rules"), " and ", h("a", { href: "#/nat" }, "NAT"),
         "; both rulesets apply until the old table goes: ", copyBox("sudo hc takeover \"inet main\"")) : null);
-      if (!dry) refreshProbation();
+      if (!dry && r.confirm_token) autoConfirm(r.confirm_token);
+      else if (!dry) refreshProbation();
     } catch (e) { mount(out, errorBox(e)); }
   };
   mount(el,

internal/web/static/assets/pages/matrix.js

@@ -0,0 +1,38 @@
+// Matrix: what the rules do to new connections between zones, and to and
+// from the firewall itself.
+import { h, mount, modal, badge } from "../ui.js";
+import { get, state } from "../api.js";
+
+export async function render(el) {
+  const m = await get("/matrix");
+  const cols = [...m.zones, "firewall"];
+  const cells = new Map(m.cells.map((c) => [c.from + "\u0000" + c.to, c]));
+  const pending = state.draft && !state.immediate && (state.draft.ops || []).length;
+  const name = (z) => z === "firewall" ? h("em", null, "firewall") : z;
+  mount(el,
+    h("div", { class: "page-head" }, h("div", { class: "grow" }, h("h1", null, "Zone matrix"),
+      h("div", { class: "muted" }, "What happens to a new connection from a zone (rows) to a zone (columns), or to and from the firewall itself, worked out from the rules in order: allowed or blocked, and how many rules decide for part of the traffic first (some services, addresses, times). Click a cell for the rules."))),
+    pending ? h("div", { class: "msg info" }, "Shown with your pending changes.") : null,
+    !m.zones.length ? h("div", { class: "panel" }, h("div", { class: "empty" }, "No zones yet: add them under Objects → Zones.")) :
+    h("div", { class: "panel" }, h("div", { class: "table-wrap" }, h("table", { class: "matrix" },
+      h("thead", null, h("tr", null, h("th", { class: "corner" }, "from ↓ to →"), cols.map((c) => h("th", null, name(c))))),
+      h("tbody", null, cols.map((from) => h("tr", null, h("th", null, name(from)), cols.map((to) => {
+        const c = cells.get(from + "\u0000" + to);
+        if (!c) return h("td", { class: "matrix-none" }, "–");
+        const n = c.exceptions.length;
+        return h("td", { class: "matrix-" + c.verdict + (n ? " matrix-some" : ""), title: c.verdict + " by " + c.decided_by + (n ? ", " + n + " exception" + (n === 1 ? "" : "s") + " first" : "") },
+          h("button", { type: "button", class: "matrix-cell", onclick: () => detail(c) }, c.verdict, n ? h("span", { class: "matrix-n" }, "+" + n) : null));
+      })))))),
+      h("p", { class: "muted small" }, "Zone to zone is the forward chain; to the firewall, input; from it, output. Exceptions are rules (and built-ins such as ping or port forwards) that decide for some of the traffic before the rest falls to the deciding rule or the policy. Replies and established connections aren't shown: the built-in lets those through.")));
+}
+
+function detail(c) {
+  const ruleLink = (id, chain, label) => id ? h("a", { href: "#/rules?chain=" + chain, onclick: () => document.querySelector(".modal-back") && document.querySelector(".modal-back").remove() }, label) : label;
+  const v = (verdict) => badge(verdict, { allowed: "ok", blocked: "err", logged: "", unknown: "warn" }[verdict]);
+  modal(c.from + " → " + c.to, h("div", null,
+    h("p", { class: "muted" }, "New connections, checked in this order (the " + c.chain + " chain):"),
+    h("div", { class: "table-wrap" }, h("table", null, h("tbody", null,
+      c.exceptions.map((e) => h("tr", null, h("td", null, v(e.verdict)), h("td", null, e.what), h("td", { class: "small" }, ruleLink(e.rule_id, e.chain, e.label)))),
+      h("tr", { class: "matrix-rest" }, h("td", null, v(c.verdict)), h("td", null, h("strong", null, c.exceptions.length ? "the rest" : "everything")),
+        h("td", { class: "small" }, ruleLink(c.rule_id, c.chain, c.decided_by))))))), [{ label: "Close", value: null }], { wide: true });
+}

internal/web/static/assets/pages/rules.js

@@ -27,25 +27,22 @@ export async function render(el, params, query) {
   const chain = query.get("chain") || "input";
   const [all, settings, cs, qs] = await Promise.all([get("/rules", { chain }), get("/settings"), counters(), quotas()]);
   // As nftables checks them: the rules that count traffic first.
-  const rules = [...all.filter(countsTraffic), ...all.filter((r) => !countsTraffic(r))];
+  const rules = [...all.filter(everyPacket), ...all.filter((r) => !everyPacket(r))];
   const write = can("config", true);
   const info = chains.find((c) => c[0] === chain) || chains[0];
   const policy = settings[chain + "_policy"];
-  const bi = (builtins[chain] || []).filter(([flag]) => flag === "" || settings.builtins[flag]);
+  const actionBadge = (r) => r.raw ? badge("as written", "") : badge(r.action + (r.log ? " + log" : ""), { accept: "ok", drop: "err", reject: "warn", log: "" }[r.action]);
-  const actionBadge = (r) => badge(r.action + (r.log ? " + log" : ""), { accept: "ok", drop: "err", reject: "warn", log: "" }[r.action]);
   mount(el,
     h("div", { class: "page-head" },
       h("div", { class: "grow" }, h("h1", null, "Rules"), h("div", { class: "muted" }, "Checked top to bottom; the first rule that decides wins. ", info[2])),
       write ? h("div", { class: "actions" }, h("button", { class: "primary", onclick: () => ruleDialog(null, chain) }, "Add rule")) : null),
     h("div", { class: "tabs" }, chains.map(([k, label]) => h("a", { href: "#/rules?chain=" + k, class: k === chain ? "active" : "" }, label))),
     h("div", { class: "panel" },
-      bi.length ? h("div", { class: "builtins muted small" }, h("strong", null, "Added by Housecarl first: "),
+      builtinsRow(chain, settings, cs),
-        bi.map(([, mark, text], i) => [i ? " · " : "", text, mark ? [" (", hits(cs, "b-" + mark), ")"] : ""]), " — ",
-        h("a", { href: "#/settings" }, "settings")) : null,
       table([
         { label: "#", render: (r) => r.pending === "deleted" ? "" : String(live(rules).indexOf(r) + 1) },
         { label: "Match", render: (r) => r.raw ? [badge("raw", ""), " ", h("span", { class: "mono small" }, r.raw)]
-          : [countsTraffic(r) ? [badge("every packet", "accent"), " "] : null, describe(r), extras(r, qs[r.id])] },
+          : [everyPacket(r) ? [badge("every packet", "accent"), " "] : null, describe(r), extras(r, qs[r.id])] },
         { label: "Action", render: actionBadge },
         { label: "Hits", render: (r) => hits(cs, "r" + r.id) },
         { label: "Comment", render: (r) => r.comment },
@@ -53,7 +50,7 @@ export async function render(el, params, query) {
           if (r.pending === "deleted") return badge("deleted", "err");
           // Moves stay among the rules of the same kind (counting traffic,
           // or not): nftables checks those first.
-          const rs = live(rules).filter((x) => countsTraffic(x) === countsTraffic(r)), i = rs.indexOf(r);
+          const rs = live(rules).filter((x) => everyPacket(x) === everyPacket(r)), i = rs.indexOf(r);
           return [
           h("button", { class: "small reveal", title: "Move up", disabled: i === 0, onclick: () => move(r, { before: rs[i - 1].id }) }, "↑"), " ",
           h("button", { class: "small reveal", title: "Move down", disabled: i === rs.length - 1, onclick: () => move(r, { after: rs[i + 1].id }) }, "↓"), " ",
@@ -63,14 +60,72 @@ export async function render(el, params, query) {
         ];
         } } : null,
       ].filter(Boolean), rules, { empty: "No rules in this chain yet.", rowClass }),
-      rules.some(countsTraffic) ? h("div", { class: "muted small" }, "Rules marked “every packet” have a quota or a byte rate: they are checked before established connections are accepted, so they count all of a connection's traffic, not only its first packet. They can drop, reject or log, not accept.") : null,
+      rules.some(everyPacket) ? h("div", { class: "muted small" }, "Rules marked “every packet” are checked before established connections are accepted: they see all of a connection's traffic, not only its first packet, so a quota counts all of it and a block stops connections already open. They can drop, reject or log, not accept.") : null,
       h("div", { class: "policy small" }, "Then the policy: ", badge(policy, policy === "drop" ? "err" : "ok"),
         policy === "drop" ? [" — ", hits(cs, "p-" + chain), " packets reached it"] : null)));
 }
 
-// countsTraffic: a quota or a byte rate (as model.Rule.CountsTraffic).
+// everyPacket: checked before established connections are accepted (as
-function countsTraffic(r) {
+// model.Rule.ChecksEveryPacket): asked to, or a quota or a byte rate.
-  return !r.raw && (!!r.quota || !!(r.limit && ["bytes", "kbytes", "mbytes"].includes(r.limit.unit)));
+function everyPacket(r) {
+  return !r.raw && (!!r.every_packet || !!r.quota || !!(r.limit && ["bytes", "kbytes", "mbytes"].includes(r.limit.unit)));
+}
+
+// What each built-in does (a switch covers all its rules).
+const builtinText = {
+  loopback: "Accept everything on lo (the firewall talking to itself)",
+  established: "Accept established and related connections: replies, and the rest of connections a rule let in",
+  drop_invalid: "Drop invalid packets: what connection tracking can't place",
+  icmpv6: "Accept the ICMPv6 that IPv6 needs: errors, echo, neighbour discovery, MLD (RFC 4890)",
+  ping_internal: "Accept ping to the firewall from internal zones",
+  dhcpv6_client: "Accept DHCPv6 replies on external zones (your ISP's prefix delegation)",
+  antispoof: "Drop sources that can't come from outside, on external zones (Settings: the lists and your prefixes)",
+  mss_clamp: "Clamp TCP's MSS to the route's MTU on external zones (needed with PPPoE)",
+};
+
+// builtinsRow sums up the chain's built-ins in one line, with a button to
+// see and switch them.
+function builtinsRow(chain, settings, cs) {
+  const list = builtins[chain] || [];
+  const on = list.filter(([flag]) => flag === "" || settings.builtins[flag]);
+  if (!list.length) return null;
+  return h("div", { class: "builtins muted small" },
+    on.length ? (on.length === 1 ? "Housecarl adds 1 rule before yours. " : "Housecarl adds " + on.length + " rules before yours. ") : "Housecarl adds no rules before yours here. ",
+    h("button", { class: "small", onclick: () => builtinsDialog(chain, settings, cs) }, "Built-ins…"));
+}
+
+// builtinsDialog lists the chain's built-ins: what each does, its hits,
+// and a switch (a setting, so it covers every chain the built-in is in).
+async function builtinsDialog(chain, settings, cs) {
+  const list = builtins[chain] || [];
+  const flags = [...new Set(list.map(([flag]) => flag))];
+  const write = can("config", true);
+  const boxes = {};
+  const rows = flags.map((flag) => {
+    const marks = list.filter(([f]) => f === flag).map(([, mark]) => mark).filter(Boolean);
+    const box = flag ? h("input", { type: "checkbox", checked: settings.builtins[flag], disabled: !write }) : null;
+    if (box) boxes[flag] = box;
+    return { flag, marks, box, text: flag ? builtinText[flag] : list.find(([f]) => f === flag)[2] };
+  });
+  const err = h("div");
+  const body = h("div", null,
+    h("p", { class: "muted" }, "Checked in this order, before your rules (rules with a quota or a byte rate come before the established built-in, so they see every packet). Each is a setting: switching one changes every chain it is in."),
+    table([
+      { label: "On", render: (x) => x.box || h("span", { class: "muted small" }, "always") },
+      { label: "Built-in", render: (x) => x.flag ? x.text : [x.text, h("div", { class: "muted small" }, "Part of each port forward.")] },
+      { label: "Hits", render: (x) => x.marks.length ? hits(cs, ...x.marks.map((m) => "b-" + m)) : h("span", { class: "muted" }, "–") },
+    ], rows),
+    err);
+  await modal("Built-ins of the " + chain + " chain", body, write ? [
+    { label: "Cancel", value: null },
+    { label: "Save", kind: "primary", onclick: async () => {
+      mount(err);
+      const b = {};
+      for (const [flag, box] of Object.entries(boxes)) if (box.checked !== settings.builtins[flag]) b[flag] = box.checked;
+      if (!Object.keys(b).length) return true;
+      try { await change("PATCH", "/settings", { builtins: b }); rerender(); return true; } catch (e) { mount(err, errorBox(e)); return false; }
+    } },
+  ] : [{ label: "Close", value: null }], { wide: true });
 }
 
 function extras(r, q) {
@@ -149,11 +204,15 @@ async function del(r) {
 export async function ruleDialog(r, chain) {
   const n = await names();
   r = r || { chain, action: "accept" };
-  const ch = select([["input", "input (to the firewall)"], ["forward", "forward (through it)"], ["output", "output (from it)"], ["early", "early (before conntrack)"]], r.chain || chain);
+  const ch = select(["input", "forward", "output", "early"], r.chain || chain);
   const match = matchEditor(r, n);
   const action = select([["accept", "accept"], ["drop", "drop"], ["reject", "reject"], ["log", "log only (and go on)"]], r.action);
   const rejectWith = select([["", "with an ICMP error"], ["tcp-reset", "with a TCP reset (TCP rules only)"]], r.reject_with || "");
-  const showReject = () => rejectWith.classList.toggle("hidden", action.value !== "reject");
+  const every = checkbox("every packet: also stop connections already open, not just new ones", r.every_packet);
+  const showReject = () => {
+    rejectWith.classList.toggle("hidden", action.value !== "reject");
+    every.classList.toggle("hidden", action.value === "accept");
+  };
   action.addEventListener("change", showReject);
   showReject();
   const states = listEditor(r.ct_state, { suggestions: ["new", "established", "related", "invalid", "untracked"], placeholder: "new", mono: false, addLabel: "Add state" });
@@ -190,27 +249,95 @@ export async function ruleDialog(r, chain) {
   const comment = input({ value: r.comment || "" });
   const disabled = checkbox("disabled: keep the rule but don't use it", r.disabled);
   const err = h("div");
+
+  // The parts most rules don't need are behind "+" buttons: open when the
+  // rule uses them, closed (and unset) with ✕.
+  const adders = h("div", { class: "adders" });
+  const sections = {};
+  const section = (key, label, open, fields) => {
+    const nodes = fields.flat().filter(Boolean);
+    const sec = { open };
+    const add = h("button", { type: "button", class: "small", onclick: () => show(true) }, "+ " + label);
+    const close = h("button", { type: "button", class: "small close-section", title: "Remove " + label.toLowerCase(), onclick: () => show(false) }, "✕");
+    nodes[0].append(" ", close);
+    const show = (on) => {
+      sec.open = on;
+      for (const n of nodes) n.classList.toggle("hidden", !on);
+      add.classList.toggle("hidden", on);
+      if (sec.onShow) sec.onShow(on);
+      if (on) { const f = nodes.find((n) => n.querySelector && n.querySelector("input, select, textarea")); if (f) f.querySelector("input, select, textarea").focus(); }
+      updateGroup();
+    };
+    adders.append(add, " ");
+    sections[key] = sec;
+    sec.nodes = nodes;
+    sec.init = () => show(open);
+    return nodes;
+  };
+  const stateFields = section("state", "Connection state", (r.ct_state || []).length > 0 || (r.ct_status || []).length > 0, [
+    field("Connection state", states, "new: a connection's first packet; established: its later ones; related: packets that belong to another connection (ICMP errors, FTP data); invalid: what connection tracking can't place. Empty: any."),
+    field("Connection status", statuses, "dnat: the connection was port forwarded; snat: its source was translated. Empty: any."),
+  ]);
+  const timeFields = section("time", "Time window", (r.days || []).length > 0 || !!r.hour_from, [
+    field("Days", days),
+    field("Hours", h("div", null, hourFrom, " to ", hourTo), "22:00 to 06:00 passes midnight. In the firewall's local time."),
+  ]);
+  const rateFields = section("rate", "Rate limit", !!r.limit, [
+    field("Rate limit", h("div", null, rate, " ", unit, " per ", per, " ", burst, " ", limitKey, over),
+      "In bytes, it is a bandwidth (KB or MB per second or minute); traffic beyond it is dropped, not queued. The burst is in the same unit."),
+  ]);
+  const quotaFields = section("quota", "Quota", !!r.quota, [
+    field("Quota", h("div", null, quotaAmount, " ", quotaUnit, " ", quotaKey, " resets ", quotaReset, " ", quotaDay, quotaOver),
+      "An allowance of traffic; Housecarl keeps the count across changes and restarts."),
+  ]);
+  const connFields = section("conn", "Connection limit", !!r.conn_limit, [
+    field("Connection limit", conn, "Match a source that already has more than this many connections."),
+  ]);
+  const rawFields = section("raw", "Write it in nftables", !!r.raw, [
+    field("Raw nftables", raw, "One nftables rule, for what Housecarl can't say, used as written: it replaces the zones, addresses, services, action, limits and log (write them in it). Checked by nftables."),
+  ]);
+  // Grouping only means something with a per-address limit, quota or
+  // connection limit.
+  const groupFields = field("Group addresses", h("div", null, "IPv4 /", group4, " IPv6 /", group6),
+    "Per-address limits, connection limits and quotas count addresses by these prefixes. Default: each IPv4 address, IPv6 by /64 (one device has several addresses).").filter(Boolean);
+  function updateGroup() {
+    if (!sections.raw) return; // still building
+    const on = (sections.rate.open && limitKey.value !== "") || (sections.quota.open && quotaKey.value !== "") || sections.conn.open;
+    for (const n of groupFields) n.classList.toggle("hidden", !on);
+  }
+  limitKey.addEventListener("change", updateGroup);
+  quotaKey.addEventListener("change", updateGroup);
+
+  // Written in nftables, a rule is its text: the fields it replaces go
+  // (and the other parts close).
+  const matchNodes = match.flat().filter(Boolean);
+  const actionField = field("Action", h("div", null, action, " ", rejectWith, every),
+    "Every packet: checked before established connections are accepted, so blocking takes effect on what is already connected.").filter(Boolean);
+  const logField = field("Log", h("div", null, log, logPrefix)).filter(Boolean);
+  const addersLabel = h("label");
+  sections.raw.onShow = (on) => {
+    for (const n of [...matchNodes, ...actionField, ...logField]) n.classList.toggle("hidden", on);
+    for (const [k, sec] of Object.entries(sections)) {
+      if (k === "raw") continue;
+      if (on && sec.open) for (const n of sec.nodes) n.classList.add("hidden");
+      if (on) sec.open = false;
+    }
+    for (const b of adders.children) if (!b.textContent.includes("nftables")) b.classList.toggle("hidden", on);
+    updateGroup();
+  };
   const body = h("div", null,
     h("div", { class: "form-grid" },
       field("Chain", ch),
-      ...match.flat(),
+      ...matchNodes,
-      field("Action", h("div", null, action, " ", rejectWith)),
+      ...actionField,
-      field("Connection state", states),
+      ...stateFields, ...timeFields, ...rateFields, ...quotaFields, ...connFields, ...groupFields, ...rawFields,
-      field("Connection status", statuses, "dnat: the connection was port forwarded."),
+      addersLabel, adders,
-      field("Days", days),
+      ...logField,
-      field("Hours", h("div", null, hourFrom, " to ", hourTo), "22:00 to 06:00 passes midnight."),
-      field("Rate limit", h("div", null, rate, " ", unit, " per ", per, " ", burst, " ", limitKey, over),
-        "Empty: no limit. In bytes, it is a bandwidth (KB or MB per second or minute); traffic beyond it is dropped, not queued. The burst is in the same unit."),
-      field("Quota", h("div", null, quotaAmount, " ", quotaUnit, " ", quotaKey, " resets ", quotaReset, " ", quotaDay, quotaOver),
-        "Empty: none. An allowance of traffic; Housecarl keeps the count across changes and restarts."),
-      field("Group addresses", h("div", null, "IPv4 /", group4, " IPv6 /", group6),
-        "Per-address limits, connection limits and quotas count addresses by these prefixes. Default: each IPv4 address, IPv6 by /64 (one device has several addresses)."),
-      field("Connection limit", conn, "Match a source that already has more than this many connections."),
-      field("Log", h("div", null, log, logPrefix)),
-      field("Raw nftables", raw, "Instead of all the above: one nftables rule, for what Housecarl can't say. Checked by nftables."),
       field("Comment", comment),
       r.id ? [h("label"), disabled] : null),
     err);
+  for (const sec of Object.values(sections)) sec.init();
+  updateGroup();
   await modal(r.id ? "Edit rule" : "Add rule", body, [
     { label: "Cancel", value: null },
     { label: "Check", onclick: () => save(true) },
@@ -219,15 +346,21 @@ export async function ruleDialog(r, chain) {
 
   async function save(dry) {
     mount(err);
-    const v = { chain: ch.value, ...match.value(), action: action.value, reject_with: action.value === "reject" ? rejectWith.value : "",
+    const on = (k) => sections[k].open;
-      ct_state: states.value(), ct_status: statuses.value(), days: days.value(), hour_from: hourFrom.value.trim(), hour_to: hourTo.value.trim(),
+    const isRaw = on("raw");
-      limit: rate.value ? { rate: parseInt(rate.value, 10), unit: unit.value, per: per.value, burst: parseInt(burst.value || "0", 10), over: over.input.checked,
+    const m = isRaw ? { in_zones: [], not_in: false, out_zones: [], not_out: false, sources: [], not_sources: false, destinations: [], not_destinations: false, services: [], family: "" } : match.value();
+    const v = { chain: ch.value, ...m, action: isRaw ? "accept" : action.value, reject_with: !isRaw && action.value === "reject" ? rejectWith.value : "",
+      ct_state: on("state") ? states.value() : [], ct_status: on("state") ? statuses.value() : [],
+      days: on("time") ? days.value() : [], hour_from: on("time") ? hourFrom.value.trim() : "", hour_to: on("time") ? hourTo.value.trim() : "",
+      limit: on("rate") && rate.value ? { rate: parseInt(rate.value, 10), unit: unit.value, per: per.value, burst: parseInt(burst.value || "0", 10), over: over.input.checked,
         per_source: limitKey.value === "source", per_destination: limitKey.value === "destination" } : null,
-      quota: quotaAmount.value ? { bytes: parseInt(quotaAmount.value, 10) * (quotaUnit.value === "gb" ? 1 << 30 : 1 << 20), over: quotaOver.input.checked,
+      quota: on("quota") && quotaAmount.value ? { bytes: parseInt(quotaAmount.value, 10) * (quotaUnit.value === "gb" ? 1 << 30 : 1 << 20), over: quotaOver.input.checked,
         per_source: quotaKey.value === "source", per_destination: quotaKey.value === "destination",
         reset: quotaReset.value, day: quotaReset.value === "monthly" ? parseInt(quotaDay.value || "1", 10) : 0 } : null,
-      group_v4: parseInt(group4.value || "0", 10), group_v6: parseInt(group6.value || "0", 10),
+      group_v4: isRaw ? 0 : parseInt(group4.value || "0", 10), group_v6: isRaw ? 0 : parseInt(group6.value || "0", 10),
-      conn_limit: parseInt(conn.value || "0", 10), log: log.input.checked, log_prefix: logPrefix.value.trim(), raw: raw.value.trim(), comment: comment.value.trim() };
+      conn_limit: on("conn") ? parseInt(conn.value || "0", 10) : 0,
+      every_packet: !isRaw && action.value !== "accept" && every.input.checked, log: !isRaw && log.input.checked, log_prefix: isRaw ? "" : logPrefix.value.trim(),
+      raw: on("raw") ? raw.value.trim() : "", comment: comment.value.trim() };
     if (r.id) v.disabled = disabled.input.checked;
     try {
       await change(r.id ? "PATCH" : "POST", r.id ? "/rules/" + r.id : "/rules", v, { query: dry ? { dry_run: 1 } : undefined });

internal/web/static/assets/pages/sets.js

@@ -31,19 +31,29 @@ async function setDialog(s) {
   s = s || { entries: [] };
   const name = input({ value: s.name || "", placeholder: "trusted" });
   const list = listEditor(s.entries, { placeholder: "192.168.1.0/24, 2001:db8:1::/64, 192.168.1.100-192.168.1.199", addLabel: "Add entry" });
-  const dynamic = checkbox("dynamic: entries are added at run time", s.dynamic);
+  const dynamic = checkbox("dynamic: entries are added at run time (the Entries button, hc set entries, a program with a token), each for a time", s.dynamic);
+  // A dynamic set has no configured entries; only it has a timeout.
+  const entryFields = field("Entries", list).filter(Boolean);
+  const timeoutFields = field("Entry timeout", timeout, "How long an entry stays unless it is added with its own. Empty: until removed (or a reboot).").filter(Boolean);
+  const showKind = () => {
+    for (const n of entryFields) n.classList.toggle("hidden", dynamic.input.checked);
+    for (const n of timeoutFields) n.classList.toggle("hidden", !dynamic.input.checked);
+  };
+  dynamic.input.addEventListener("change", showKind);
+  showKind();
   const timeout = input({ value: s.timeout ? fmtSeconds(s.timeout) : "", placeholder: "1h, 1d (none: until removed)", class: "short" });
   const comment = input({ value: s.comment || "" });
   const err = h("div");
   await modal(s.id ? "Edit set " + s.name : "Add set", h("div", null, h("div", { class: "form-grid" },
-    field("Name", name), field("Entries", list), h("label"), dynamic, field("Entry timeout", timeout, "A dynamic set's default."), field("Comment", comment)), err), [
+    field("Name", name), ...entryFields, h("label"), dynamic, ...timeoutFields, field("Comment", comment)), err), [
     { label: "Cancel", value: null },
     { label: s.id ? "Save" : "Add set", kind: "primary", onclick: async () => {
       mount(err);
       const t = parseSeconds(timeout.value);
       if (t === null) { mount(err, h("div", { class: "msg err" }, "Write the timeout as 1h, 30m or 1d.")); return false; }
       try {
-        await change(s.id ? "PATCH" : "POST", s.id ? "/sets/" + s.id : "/sets", { name: name.value.trim(), entries: list.value(), dynamic: dynamic.input.checked, timeout: t, comment: comment.value.trim() });
+        await change(s.id ? "PATCH" : "POST", s.id ? "/sets/" + s.id : "/sets", { name: name.value.trim(), entries: dynamic.input.checked ? [] : list.value(), dynamic: dynamic.input.checked,
+          timeout: dynamic.input.checked ? t : 0, comment: comment.value.trim() });
         rerender();
         return true;
       } catch (e) { mount(err, errorBox(e)); return false; }

internal/web/static/assets/pages/settings.js

@@ -27,6 +27,7 @@ export async function render(el) {
   const v6 = listEditor(s.antispoof_v6, { addLabel: "Add" });
   const flow = listEditor(s.flowtable_devices, { suggestions: ifs.map((i) => i.name), placeholder: "wan", addLabel: "Add device" });
   const probation = input({ value: fmtSeconds(s.probation_seconds || 120), class: "short" });
+  const manual = checkbox("a person confirms each change (otherwise the browser or hc that made it confirms it by itself, once it reaches Housecarl again over a new connection)", s.manual_confirm);
   const err = h("div");
   const save = async (dry) => {
     mount(err);
@@ -37,7 +38,7 @@ export async function render(el) {
     try {
       const r = await change("PATCH", "/settings", { input_policy: input_.value, forward_policy: forward.value, output_policy: output.value,
         log_policy_drops: logDrops.input.checked, builtins, own_prefixes: own.value(), antispoof_v4: v4.value(), antispoof_v6: v6.value(),
-        flowtable_devices: flow.value(), probation_seconds: p === 120 ? 0 : p }, { query: dry ? { dry_run: 1 } : undefined });
+        flowtable_devices: flow.value(), probation_seconds: p === 120 ? 0 : p, manual_confirm: manual.input.checked }, { query: dry ? { dry_run: 1 } : undefined });
       if (!dry && (r.applied || r.staged)) rerender();
     } catch (e) { mount(err, errorBox(e)); }
   };
@@ -54,7 +55,8 @@ export async function render(el) {
       field("Antispoof IPv4", v4),
       field("Antispoof IPv6", v6),
       field("Flowtable", flow, "Forwarded TCP and UDP between these devices skip the rules once established (fast path). Exact names; none: off."),
-      field("Probation", probation, "How long a change made here has to be confirmed before it is rolled back.")),
+      field("Probation", probation, "How long a change made over the network has to be confirmed before it is rolled back."),
+      h("label"), manual),
     err,
     write ? h("div", { class: "actions form-actions" },
       h("button", { class: "primary", onclick: () => save(false) }, "Save"),

internal/web/static/assets/pages/trace.js

@@ -0,0 +1,77 @@
+// Trace: follow packets through every table's rules, and see what
+// decided their fate.
+import { h, mount, input, select, field, errorBox, badge } from "../ui.js";
+import { get, api, can } from "../api.js";
+import { onLeave } from "../app.js";
+
+export async function render(el) {
+  const address = input({ placeholder: "192.168.1.50, or 2001:db8:1::/64", class: "mono" });
+  const port = input({ placeholder: "443", class: "short", type: "number", min: 1, max: 65535 });
+  const protocol = select([["", "any"], "tcp", "udp", "sctp", "icmp", "icmpv6"], "", { class: "inline" });
+  const seconds = input({ value: "60", class: "short", type: "number", min: 5, max: 600 });
+  const err = h("div");
+  const status = h("div");
+  const packets = h("div");
+  const write = can("config", true);
+  const startBtn = h("button", { class: "primary", onclick: start }, "Start tracing");
+  const stopBtn = h("button", { class: "hidden", onclick: stop }, "Stop");
+  let timer = null;
+
+  async function start() {
+    mount(err);
+    try {
+      await api("POST", "/trace", { body: { address: address.value.trim(), port: parseInt(port.value || "0", 10), protocol: protocol.value, seconds: parseInt(seconds.value || "60", 10) } });
+      poll();
+    } catch (e) { mount(err, errorBox(e)); }
+  }
+  async function stop() {
+    try { show(await api("DELETE", "/trace")); } catch (e) { mount(err, errorBox(e)); }
+  }
+  async function poll() {
+    clearTimeout(timer);
+    let st;
+    try { st = await get("/trace"); } catch (e) { mount(err, errorBox(e)); return; }
+    show(st);
+    if (st.running) timer = setTimeout(poll, 1000);
+  }
+  function show(st) {
+    startBtn.classList.toggle("hidden", st.running || !write);
+    stopBtn.classList.toggle("hidden", !st.running || !write);
+    const what = [st.input.address, st.input.protocol, st.input.port ? "port " + st.input.port : ""].filter(Boolean).join(" ");
+    mount(status, st.started && st.started !== "0001-01-01T00:00:00Z" ? h("div", { class: "msg " + (st.error ? "err" : st.running ? "info" : "ok") },
+      st.running ? ["Tracing ", h("strong", null, what), " until " + new Date(st.ends).toLocaleTimeString() + ". Make the traffic happen now."]
+        : ["Traced ", h("strong", null, what), ": " + (st.packets.length === 1 ? "1 packet" : st.packets.length + " packets") + "."],
+      st.error ? h("div", null, st.error) : null) : null);
+    mount(packets, st.packets.length ? st.packets.slice().reverse().map(packet)
+      : st.running ? h("div", { class: "empty" }, "No packets yet.") : null);
+  }
+  const hostPort = (a, p) => !a ? "" : !p ? a : a.includes(":") ? "[" + a + "]:" + p : a + ":" + p;
+  function packet(p) {
+    const head = [p.protocol, p.tcp_flags, p.icmp_type].filter(Boolean).join(" ");
+    const decided = p.decided_by.label || [p.decided_by.table, p.decided_by.chain, p.decided_by.kind].filter(Boolean).join(" ");
+    return h("div", { class: "panel trace-packet" },
+      h("div", { class: "trace-head" },
+        badge(p.verdict, p.verdict === "dropped" ? "err" : "ok"), " ",
+        h("span", { class: "mono" }, hostPort(p.source, p.source_port), " → ", hostPort(p.destination, p.destination_port)), " ",
+        h("span", { class: "muted" }, head, p.in ? " · in " + p.in : "", p.out ? " · out " + p.out : ""),
+        h("span", { class: "muted small trace-time" }, new Date(p.time).toLocaleTimeString())),
+      decided ? h("div", { class: "small" }, p.verdict === "dropped" ? "Dropped by " : "Accepted by ", h("strong", null, decided)) : null,
+      h("table", { class: "trace-steps" }, h("tbody", null, p.steps.map((s) => h("tr", { class: /^(drop|reject)/.test(s.verdict) ? "step-drop" : s.verdict === "accept" ? "step-accept" : "" },
+        h("td", { class: "muted small" }, s.table + " " + s.chain),
+        h("td", null, s.label ? (s.mark && s.mark.startsWith("r") ? h("a", { href: "#/rules?chain=" + s.chain }, s.label) : s.label)
+          : s.kind === "policy" ? s.chain + " policy" : h("span", { class: "mono small" }, s.rule)),
+        h("td", { class: "small" }, s.verdict))))));
+  }
+
+  mount(el,
+    h("div", { class: "page-head" }, h("div", { class: "grow" }, h("h1", null, "Trace"),
+      h("div", { class: "muted" }, "Follows packets through the whole ruleset, Housecarl's rules and every other table (Docker, fail2ban…): each rule and policy they meet, and what accepted or dropped them. For “why doesn't this get through?”"))),
+    write ? h("div", { class: "panel" }, h("div", { class: "form-grid" },
+      field("Address", address, "Packets from or to it (either way). An address or a prefix."),
+      field("Port", h("div", null, port, " ", protocol), "Packets from or to it. Empty: any."),
+      field("For", h("div", null, seconds, " seconds"), "At most 600; it stops by itself after 200 packets.")),
+      h("div", { class: "actions form-actions" }, startBtn, stopBtn), err) : null,
+    status, packets);
+  await poll();
+  onLeave(() => clearTimeout(timer));
+}

internal/web/static/assets/style.css

@@ -409,3 +409,24 @@ input.wide { width: 100%; }
 .diff .hunk { background: var(--hunk); color: var(--accent); }
 .diff .file { font-weight: 700; padding-top: 6px; }
 select.inline { width: auto; display: inline-block; }
+/* The rule dialog's optional parts: "+" buttons, and ✕ to close one. */
+.adders { display: flex; flex-wrap: wrap; gap: 6px; }
+button.close-section { padding: 0 6px; margin-left: 4px; font-size: 11px; line-height: 1.5; }
+/* Trace: one panel per packet, its steps below. */
+.trace-head { display: flex; flex-wrap: wrap; align-items: center; gap: 6px; margin-bottom: 4px; }
+.trace-time { margin-left: auto; }
+table.trace-steps { width: 100%; margin-top: 6px; }
+table.trace-steps td { padding: 3px 8px; border-top: 1px solid var(--border); }
+tr.step-drop td { background: var(--del); }
+tr.step-accept td { background: var(--add); }
+/* The zone matrix: a cell per pair, coloured by what happens. */
+table.matrix th, table.matrix td { text-align: center; }
+table.matrix tbody th { text-align: left; }
+table.matrix th.corner { font-weight: 400; color: var(--muted); text-transform: none; }
+table.matrix td { padding: 4px; }
+.matrix-cell { width: 100%; min-width: 7em; border: 0; background: transparent; color: inherit; font-weight: 600; padding: 6px 8px; cursor: pointer; }
+td.matrix-allowed { background: var(--add); color: var(--add-text); }
+td.matrix-blocked { background: var(--del); color: var(--del-text); }
+.matrix-n { margin-left: 6px; font-weight: 400; font-size: 12px; color: var(--warn); }
+td.matrix-none { color: var(--muted); }
+tr.matrix-rest td { border-top: 2px solid var(--border); }

scripts/e2e.sh

@@ -172,11 +172,11 @@ run "probation of 10 seconds" 0 H settings set --probation 10s
 TOKEN=$(R /w/housecarld -c /w/housecarl.yaml user token admin e2e 2>/dev/null | tail -1)
 C() { docker exec ${P}-client /w/hc --config /w/cli.yaml --server http://$LAN4:8070 --token "$TOKEN" "$@"; }
 run "the API from the LAN" 0 C whoami
-run "a change over the network is on probation" 0 C zone add guests --interface guest0
+run "a change over the network is on probation" 0 C --no-confirm zone add guests --interface guest0
 contains "applied on probation"
 run "confirmed" 0 C confirm
 contains "confirmed 1 change;"
-run "a change that cuts the client off" 0 C rule disable "$APIRULE"
+run "a change that cuts the client off" 0 C --no-confirm rule disable "$APIRULE"
 contains "on probation"
 run "the client is cut off (a timeout)" 124 docker exec ${P}-client timeout 3 /w/hc --config /w/cli.yaml --server http://$LAN4:8070 --token "$TOKEN" whoami
 sleep 12
@@ -185,6 +185,25 @@ run "the rule is enabled again" 0 H rule show "$APIRULE"
 not_contains "disabled"
 run "the rollback is in the history" 0 H history list
 contains "rollback"
+# Confirmed by the client itself: it reached the API over a new connection.
+run "a change confirmed by its client" 0 C zone add guests2 --interface guest2
+contains "applied and confirmed"
+run "nothing waits" 0 C status
+not_contains "Probation:"
+run "the confirmation is audited" 0 H audit --action config.confirm
+contains "confirmed automatically"
+# A change that cuts its client off can't be confirmed by it: rolled back.
+run "a lockout change, not confirmed" 0 C rule disable "$APIRULE"
+contains "couldn't confirm it"
+sleep 3
+run "rolled back: the client is back" 0 C whoami
+run "the rule is enabled again" 0 H rule show "$APIRULE"
+not_contains "disabled"
+run "a person must confirm, by setting" 0 H settings set --auto-confirm off
+run "then a change waits" 0 C zone add guests3 --interface guest3
+contains "applied on probation"
+run "confirmed by a person" 0 C confirm
+run "back to automatic" 0 H settings set --auto-confirm on
 
 # ---- changesets: staged, reviewed, applied together
 
@@ -209,8 +228,8 @@ run "remove the forward operation" 0 CE changeset remove 3
 contains "has 2 operations"
 run "the diff without it" 0 CE changeset diff
 not_contains "dnat ip to 10.88.10.57"
-run "applied together" 0 CE changeset apply
+run "applied together, and confirmed by this client" 0 CE changeset apply
-contains "applied on probation"
+contains "applied and confirmed"
 contains "HOUSECARL_CHANGESET still names changeset $CS"
 run "one history entry" 0 H history list
 contains "e2e batch"
@@ -221,9 +240,14 @@ contains "^jellyfin$"
 run "a stale HOUSECARL_CHANGESET is refused" 1 CE zone add stale -i stale0
 contains "is applied"
 CS2=$(H changeset create -q)
-run "an undo can't be staged" 6 H -C "$CS2" history undo 1
+UNDO=$(H zone add undome -i undo0 | sed -n 's/.*history \([0-9]*\);.*/\1/p')
+run "an undo is staged" 0 H -C "$CS2" history undo "$UNDO"
+contains "staged in changeset $CS2"
+run "not applied yet: the zone is there" 0 H zone show undome
+run "an import can't be staged" 6 sh -c "echo '{\"nftables\":[]}' | docker exec -i ${P}-router /w/hc -C $CS2 import -"
 contains "can't be staged"
 run "discarded" 0 H changeset discard "$CS2"
+run "the zone stays" 0 H zone show undome
 run "a dry run shows the ruleset's diff" 0 H -n set add e2e_dry 10.1.2.3
 contains "^+.*set e2e_dry_v4"
 
@@ -248,6 +272,53 @@ run "the reset is audited" 0 H audit --action quota.
 contains "quota.reset"
 run "the quota rule goes" 0 H rule delete -y "$QR"
 
+# ---- the zone matrix, of the imported ruleset
+
+run "the matrix" 0 H matrix
+contains "^firewall  *allow"
+run "LAN to the internet" 0 H matrix dhcp wan
+contains "allowed  *from trusted_ips  *rule forward #[0-9]* (Forward from trusted IPs)"
+contains "blocked  *the rest  *forward policy"
+not_contains "wg0_net"
+run "the internet to the firewall" 0 H matrix wan firewall
+contains "blocked  *sources that can't come from outside"
+contains "allowed  *udp/51820"
+
+# ---- connections and tracing
+
+# A connection that stays open, masqueraded on its way out.
+docker exec -d ${P}-client sh -c "sleep 30 | nc $NET4 9000"
+sleep 1
+run "the connection is tracked" 0 H connections --search "$CLIENT4 :9000"
+contains "$CLIENT4:[0-9]*  *$NET4:9000  *as $WAN4"
+run "closed" 0 H connections close --search "$CLIENT4 :9000" -y
+contains "closed [1-9][0-9]* connection"
+run "gone" 0 H connections --search "$CLIENT4 :9000"
+not_contains ":9000"
+# A block added while traffic flows: it stops new connections, not this
+# one, unless it checks every packet.
+docker exec -d ${P}-client sh -c "rm -f /tmp/ncdone; yes | nc $NET4 9000; touch /tmp/ncdone"
+sleep 1
+BR=$(H -o id rule add forward --from $CLIENT4 --service tcp/9000 -a reject --reject-with tcp-reset --comment "e2e block" --first)
+run "a block for new connections" 0 test -n "$BR"
+sleep 2
+run "the open upload carries on" 1 docker exec ${P}-client test -f /tmp/ncdone
+run "the block checks every packet now" 0 H rule update "$BR" --every-packet
+sleep 2
+run "the upload is cut" 0 docker exec ${P}-client test -f /tmp/ncdone
+# Tracing a new connection: dropped, by that rule.
+docker exec -d ${P}-router sh -c "/w/hc trace --address $CLIENT4 --port 9000 --seconds 6 > /w/trace.out 2>&1"
+sleep 2
+connect client $NET4 9000 >/dev/null 2>&1
+sleep 6
+run "the trace" 0 cat "$WORK/trace.out"
+contains "$CLIENT4:[0-9]* -> $NET4:9000  tcp syn"
+contains "inet housecarl forward: rule forward #$BR (e2e block) -> drop"
+contains "=> DROPPED by rule forward #$BR (e2e block)"
+run "no trace table is left" 0 R nft list tables
+not_contains "housecarl-trace"
+run "the block goes" 0 H rule delete -y "$BR"
+
 # ---- the drop log
 
 run "log drops from the internet to the API" 0 H rule add input --in wan --service tcp/8070 -a drop --log --first