Shadow mode: Housecarl beside the firewall in force, changing nothing

shadow.enabled in housecarl.yaml loads Housecarl's ruleset so that it
blocks, rejects and translates nothing: drops and rejects count and
accept (an accept only ends Housecarl's chain), policies accept and log
what they would drop, NAT only counts, MSS clamping and the flowtable
are left out, and raw rules that do more than accept, drop or reject are
left out with a warning. Comments mark what would happen ([would drop]).
Filter chains run after the other firewall's (+10) or before (-10); NAT
chains always first. Every change is checked by nftables in both forms.
No probation; takeovers and closing connections are refused. The boot
fallback filters nothing. internal/tablewatch puts the table back after
a flush ruleset. Mode in status, healthz, whoami, the web UI, the log
and the tracer; hc ruleset --enforcing. docs/shadow.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

docs/api.md

@@ -31,7 +31,7 @@ user name. [permissions.md](permissions.md) has what each call needs.
 | 401 | `unauthenticated`, `session_expired`, `login_failed` | |
 | 403 | `forbidden`, `csrf` | |
 | 404 | `not_found` | |
-| 409 | `conflict` | e.g. undoing a change after the objects changed again |
+| 409 | `conflict` | e.g. undoing a change after the objects changed again; a takeover or closing connections in shadow mode ([shadow.md](shadow.md)) |
 | 422 | `validation` | the change would make the configuration invalid; `problems` lists why ([validation.md](validation.md)) |
 | 422 | `firewall` | nftables refused the ruleset (`nft -c`), or applying it failed; the message has nftables' words. Nothing was changed |
 | 429 | `rate_limited` | |
@@ -50,7 +50,7 @@ Calls that change the configuration answer:
 
 * `confirm_by`: the change is on probation, and is rolled back at that
   time unless `POST /confirm` comes first. Missing for changes made
-  through the socket.
+  through the socket, and in shadow mode ([shadow.md](shadow.md)).
 * `confirm_token`: the client that made the change can confirm it by
   itself with `POST /confirm {"token"}`, from the same address, as the
   same user and token, over a new connection (the answer closes this
@@ -74,19 +74,19 @@ Creating answers 201, other changes 200.
 
 | | |
 |---|---|
-| `GET /healthz` | `ok`, `version`, `firewall` (`applied`, `unconfigured`, `fallback`, `missing`), `probation` (changes waiting to be confirmed), `warnings` (codes); HTTP 503 unless `ok`. Without authentication; read at most every five seconds ([operations.md](operations.md#monitoring)) |
-| `GET /status` | `build`, `started`, `applied` (the ruleset is in force), `apply_error`, `problems`, `probation`, `interfaces`, `counts` (objects by kind), `other_tables` (other nftables tables, `"inet main"`) |
+| `GET /healthz` | `ok`, `version`, `firewall` (`applied`, `unconfigured`, `fallback`, `missing`), `mode` (`enforcing`, or `shadow`: the ruleset blocks nothing, [shadow.md](shadow.md)), `probation` (changes waiting to be confirmed), `warnings` (codes); HTTP 503 unless `ok`. Without authentication; read at most every five seconds ([operations.md](operations.md#monitoring)) |
+| `GET /status` | `build`, `started`, `mode` (`enforcing` or `shadow`), `shadow_position` (`after` or `before`, in shadow mode), `applied` (the ruleset is in force), `apply_error`, `problems`, `probation`, `interfaces`, `counts` (objects by kind), `other_tables` (other nftables tables, `"inet main"`) |
 | `GET /interfaces` | the machine's interfaces: `name`, `up`, `addresses`, `zone` |
 | `GET /matrix[?changeset=ID]` | the zone matrix: `zones`, `cells` (`from`, `to` (a zone, or `firewall`), `chain`, `verdict` (allowed, blocked), `decided_by`, `rule_id`, `exceptions`: `rule_id`, `chain`, `label`, `verdict` (allowed, blocked, logged, unknown), `what`) ([rules.md](rules.md#the-zone-matrix)) |
-| `GET /ruleset` | the nftables ruleset Housecarl builds, as text |
+| `GET /ruleset[?enforcing=1]` | the nftables ruleset Housecarl builds, as text. In shadow mode it is the shadow ruleset; `enforcing=1` gives the one loaded once shadow mode is off |
 | `GET /counters` | `{"since", "counters": {"r12": {"packets", "bytes"}, ...}}`, by mark ([design.md](design.md#its-own-table-only)); totals since housecarld started |
 | `GET /quotas` | each rule's quota: `rule_id`, `rule`, `quota`, `used` (bytes), `addresses` (`[{"address", "used"}]` for a per-address quota), `last_reset`, `next_reset` |
 | `GET /connections[?q=&limit=]` | tracked connections: `family`, `protocol`, `source`, `source_port`, `destination`, `destination_port` (or `icmp_id`, `icmp_type`, `icmp_code`), `dnat_to`, `dnat_port`, `snat_to`, `state`, `assured`, `timeout` (seconds left), `packets` and `bytes` when the kernel counts them. `q`: all of its words in the addresses, `:port`, protocol, state |
-| `POST /connections/close` | `{"connections": [...]}` (as listed) or `{"search": "192.168.1.50"}`: delete their tracking entries (`config:write`, audited); answers `{"closed": N}` |
+| `POST /connections/close` | `{"connections": [...]}` (as listed) or `{"search": "192.168.1.50"}`: delete their tracking entries (`config:write`, audited); answers `{"closed": N}`. 409 in shadow mode |
 | `POST /trace` | `{"address", "port", "protocol", "seconds"}` (5-600, default 60): start tracing (`config:write`, audited); 409 while one runs |
-| `GET /trace` | the trace running, or the last: `running`, `input`, `started`, `ends`, `error`, `packets`: each with `time`, `in`, `out`, `protocol`, addresses and ports, `tcp_flags`, `icmp_type`, `steps` (`table`, `chain`, `kind` rule or policy, `rule`, `mark`, `label`, `verdict`), `verdict` (accepted, dropped) and `decided_by` |
+| `GET /trace` | the trace running, or the last: `running`, `input`, `started`, `ends`, `error`, `packets`: each with `time`, `in`, `out`, `protocol`, addresses and ports, `tcp_flags`, `icmp_type`, `steps` (`table`, `chain`, `kind` rule or policy, `rule`, `mark`, `label`, `verdict`), `verdict` (accepted, dropped) and `decided_by`; in shadow mode `would_drop`, the step where Housecarl would have dropped it |
 | `DELETE /trace` | stop it |
-| `GET /log[?limit=&after=&q=]` | logged packets, newest first: `time`, `prefix`, `in`, `out`, `family`, `protocol`, `source`, `source_port`, `destination`, `destination_port`, `icmp_type`, `tcp_flags`, `length`, `mark` (the rule's). `after` (a time) for those since; `q` searches |
+| `GET /log[?limit=&after=&q=]` | logged packets, newest first: `time`, `prefix`, `in`, `out`, `family`, `protocol`, `source`, `source_port`, `destination`, `destination_port`, `icmp_type`, `tcp_flags`, `length`, `mark` (the rule's), `verdict`, `shadow` (logged in shadow mode: a drop or reject is what would have been done). `after` (a time) for those since; `q` searches |
 | `GET /catalog` | built-in services, chains, actions, roles, the log group |
 
 ### Settings
@@ -211,7 +211,7 @@ history, but are audited.
 | `GET /history/{id}` | one, with each object before and after |
 | `POST /history/{id}/revert` | undo it; 409 if an object it touched has changed since, or it was undone already |
 | `POST /import/nft` | `{"json": NFT_JSON, "text": "nft -a list ruleset output", "defines": "ruleset file", "table": "inet main", "zones": {"enp2s0": "lan"}, "replace": false}` ([import.md](import.md)); answers with `state` (the imported configuration), `notes` and `raw` too |
-| `POST /takeover` | `{"tables": ["inet main"]}`: delete these tables (administrators); on probation over the network |
+| `POST /takeover` | `{"tables": ["inet main"]}`: delete these tables (administrators); on probation over the network; 409 in shadow mode |
 
 ### Certificate