housecarl/CHANGELOG.md

# Changelog

All notable changes to Housecarl are documented here. The format follows
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and versions follow
[Semantic Versioning](https://semver.org/).

## [Unreleased]

### Added

- **Names beside inside addresses.** Connections, the log, quota usage,
  the dashboard's quotas and the statistics' devices and blocked
  sources show each inside address's name, found by a reverse lookup:
  `laptop 192.168.1.20` in the web UI, `192.168.1.20 (laptop)` in `hc`.
  Only inside addresses are looked up (an inside zone's networks, and
  private ones outside the external zones'). Names are cached for 10
  minutes and never stored. `names.enabled`, `names.resolver` and
  `names.cache` in housecarl.yaml; `hc --no-names`; `POST /names`.
- **Quotas on the dashboard.** With any rule holding a quota, the
  dashboard lists them all: the allowance, what is used (for a
  per-address quota, how many addresses and the busiest) with a bar, and
  the next reset. Each opens its rule.
- **Links to a rule open it.** A rule named in the matrix, the log, the
  tracer, statistics or the dashboard's busiest rules opens the Rules
  page on its chain, scrolled to it and highlighted (`#/rules?rule=ID`).
  The Rules page shows each rule's ID (`#410`, as `hc` and labels name
  it) beside its order.
- **A clearer zone matrix.** A cell reads allow, block, or **partly**
  (some new connections let through, the rest not), with the exceptions
  counted by what they do (`+3 -1` instead of `+4`), and what they let
  through: services with the addresses they're limited to (`ssh from
  trusted`, `5 port forwards`), under the grid in `hc matrix`, in each
  cell on the Matrix page. A zone to itself is marked (only traffic routed
  between its interfaces passes the firewall). The details list each
  rule's hits. API: `summary`, `allowed`, `blocked`, `open`,
  `same_zone`, `note`, `hits`, and `short` and `hits` per exception.
  Raw rules that are a match and then accept, drop or reject (without
  interfaces) are read instead of shown as unknown.
- **Shadow mode** (`shadow.enabled` in housecarl.yaml; docs/shadow.md):
  Housecarl's ruleset is loaded beside the firewall you have but blocks,
  rejects and translates nothing. Drops and rejects are counted and accept
  instead (an accept only ends Housecarl's chain), policies accept and
  always log what they would drop, NAT only counts, MSS clamping and the
  flowtable are left out, and raw rules that do more than accept, drop or
  reject are left out with a warning (`shadow_raw_left_out`). Comments
  mark what would happen (`[would drop]`). `shadow.position`: Housecarl's
  filter chains run after the other firewall's (`filter + 10`, the
  default: what it would drop is what switching would break) or before
  (`- 10`); its NAT chains always first. Every change is checked by
  nftables in both forms, so leaving shadow mode can't fail. No
  probation; takeovers and closing connections are refused (409). The
  boot ruleset is the shadow one, and its fallback filters nothing. When
  a reload (`flush ruleset`) deletes Housecarl's table, housecarld puts it
  back (it listens to nftables' netlink notifications). `mode` in
  `/status`, `/healthz` and `/auth/whoami`; a banner in the web UI;
  would-be drops marked in the log (`shadow`) and the tracer
  (`would_drop`); `hc ruleset --enforcing` (`GET /ruleset?enforcing=1`)
  shows the ruleset for after. `shadow_unprotected` warns when no other
  table filters the input hook; `nftables_service` and `takeover_pending`
  aren't raised in shadow mode. The e2e test imports in shadow mode
  before taking over.
- **Zone names at import:** `hc import --zone enp2s0=lan` (repeat;
  interfaces given one name share a zone; Zone names on the Import
  page), and a note listing each zone and where its name came from.
  Defines holding a VLAN sub-interface (`enp2s0.20`) name their zone,
  and `_ifname`, `_device` and `_nic` are taken off define names too
  (docs/import.md, "How things are named").
- Import: sets with `flags timeout` become dynamic sets; a DNAT of
  several ports becomes one port forward per port.

### Fixed

- `hc stats top devices` listed each device twice (once for what it
  received, once for what it sent); it is one row with both again.
- `hc rule add --before/--after/--first` made two changes (create at the
  end, then move), so two history entries, two probations, and the rule
  briefly in force at the end of its chain; staged in a changeset, the
  place was lost. The create now takes the place (`POST
  /rules?before=ID`, `after`, `first`) and makes one change.
- `hc matrix` in a terminal: the colours' escape codes threw the columns
  out of line.
- `hc counters` (`GET /counters`) kept listing the objects of a
  configuration replaced since housecarld started (`hc import
  --replace` gives every object a new ID), beside the current ones; the
  totals of objects that no longer exist are now forgotten.
- Import: two base chains on one hook had their rules imported twice,
  and the second chain's leading built-ins were moved above the first
  chain's rules; they now follow each other by priority (read in a
  fixed order), with the stricter policy, and a note.
- Import: a rule inside a jumped-to chain that stays raw kept only its
  own text and so applied to everything; the jump's condition now goes
  in front. A jump and a rule matching the same field are combined only
  when one's addresses lie within the other's; else the rule stays raw.
  A jump's ports, ICMP types, connection status and hop limit were
  dropped; a limit or log on the jump is no longer silently lost.
- Import: a second match on the same field (`ip saddr A ip saddr != B`)
  replaced the first; `!=` on the hop limit and TCP flags was read as
  `==`. Such rules now stay raw, as written.
- Import: a forward-chain accept to a forwarded host and port without
  `ct status dnat` was folded into the port forward, which only lets
  forwarded connections through; it stays a rule now. An IPv6 accept
  limited to an interface no longer becomes a pinhole open to all.
- Import: raw rules naming a set of the old table made nftables refuse
  the ruleset; address sets are now renamed `NAME_v4`/`NAME_v6`, other
  sets written out with their elements, and what can't be is left out
  with a note. Rules naming a set that isn't imported stay raw instead
  of failing validation.
- Import: a DNAT to an IPv6 address, or with matches a port forward
  can't hold (an interface excluded, a limit, a log), is left out with a
  note instead of being imported without them.

## [0.1.0] - 2026-10-19

The first version of Housecarl, a firewall manager for nftables, built
the way Jarl and Steward are.

### Added

- **Zones, address sets and services:** interfaces grouped by what they
  connect to (wildcards like `incusbr*`; external zones face the
  internet); sets of IPv4 and IPv6 addresses, prefixes and ranges, mixed;
  ports and ICMP types by name, with twenty built in (docs/rules.md).
- **Rules** in four chains (input, forward, output, and early, before
  connection tracking), in order, matching by zone, address, service,
  family, connection state and status, days and hours, a rate (also per
  source address), and a connection count; accept, drop, reject (ICMP or
  TCP reset) or log. Raw nftables rules for the rest.
- **IPv4 and IPv6 together:** one rule becomes the nftables rules each
  family needs.
- **NAT:** port forwards as one object (DNAT, the forward accept, hairpin
  NAT, and optionally an IPv6 pinhole to a host's own port); source NAT
  (masquerade or an address; NAT66 on request); redirects to the firewall
  (docs/nat.md).
- **Built-ins**, each switchable: loopback, established connections,
  invalid packets, ICMPv6 (RFC 4890), ping from internal zones, DHCPv6
  client replies, antispoofing with your own prefixes, MSS clamping, and
  a flowtable fast path.
- **Changesets, as in Jarl:** changes staged to be reviewed and applied
  together, as one history entry. Each is checked as it is staged; the
  review shows the problems, nftables' check and the diff of the ruleset.
  The web UI collects edits as pending changes by default (pending rows,
  a bar with Review and Apply now, a Changes page), with a switch for live
  edits, and Undo in History is staged like any edit; hc stages with
  `-C ID` or `HOUSECARL_CHANGESET`, and has `hc
  changeset create/list/show/diff/apply/discard/remove`. An applied
  changeset is on probation like any change. `--dry-run` shows the
  ruleset's diff too (docs/design.md, docs/cli.md).
- **Bandwidth limits and quotas:** rate limits in bytes (`--limit
  20mbytes/second,over,per-source`), per source or per destination;
  quotas, an allowance of traffic per rule or per address, reset daily,
  weekly, monthly on a chosen day, or by hand, kept across changes,
  restarts and reboots (`hc quotas`, `hc rule reset-quota`, usage on the
  rules page). Rules that count traffic are checked before established
  connections are accepted, so they see every packet. Per-address limits
  group IPv6 by /64 by default (`--group-v4`, `--group-v6`). Meters,
  byte rates and quotas in an imported ruleset carry over
  (docs/rules.md).
- **Changes checked twice and applied atomically:** validation of the
  whole configuration with stable problem codes (docs/validation.md),
  then `nft -c`, then one `nft -f` transaction that replaces Housecarl's
  own table, `inet housecarl`, and nothing else. Dynamic sets' entries
  and the counters carry across.
- **Probation:** a change made over the network rolls itself back unless
  confirmed within two minutes. The web UI and `hc` confirm their own
  changes once they reach Housecarl again over a new connection, from the
  same address (a change that cuts its client off can't, and rolls back);
  `--no-confirm` and the "a person confirms each change" setting leave it
  to `hc confirm` or the banner;
  changes through the local socket are kept at once; a restart rolls
  back what was pending. No rule is forced to keep SSH or the web UI
  open.
- **At boot:** the confirmed ruleset is loaded by
  `housecarl-ruleset.service` before the network comes up, without
  housecarld. An empty configuration applies nothing. It fails closed:
  if the saved ruleset doesn't load, a fallback saved beside it
  (`ruleset-fallback.nft`) is, until housecarld puts the real one back:
  nothing new gets in from external zones, the inside keeps working
  (masqueraded), and the unit is marked failed with a critical message.
  `hc status`, the dashboard and `/healthz` say while the fallback is in
  force; the warning `no_external_zone` says when no zone is external,
  which would make the fallback close everything (docs/operations.md).
- **The flowtable and interfaces that come and go:** the boot ruleset
  has no flowtable (nftables on Debian 13's kernel refuses one naming a
  device that doesn't exist yet, such as `ppp0` before PPPoE is up, and
  with it the whole file); housecarld watches the kernel's link messages
  and applies the ruleset again when a flowtable device appears or comes
  back as a new interface (PPPoE reconnecting). A guide to when the
  flowtable helps, and what it costs (docs/flowtable.md).
- **`GET /healthz` for monitoring:** `ok`, `firewall` (applied,
  unconfigured, fallback, missing), changes on probation and the
  machine's warnings, with HTTP 503 unless the firewall is in force;
  no login needed, read at most every five seconds.
- **Import keeps a chain's head in order:** once a would-be built-in at
  the head of a chain has to stay a rule (an invalid drop the forward
  chain needs after an exception), the built-ins after it stay rules
  too, in their place, instead of moving ahead of it: an ICMPv6 accept
  no longer jumps ahead of the invalid drop it came after.
- **Renaming a zone, set or service** changes the rules, NAT rules and
  port forwards that name it, in the same change (one history entry, one
  undo); before, a name in use couldn't change. Dynamic sets keep their
  names (their tokens and programs go by them).
- VLAN sub-interfaces (`enp2s0.20`) in zones and the flowtable, tested
  with tagged traffic (docs/rules.md).
- `hc settings set --flowtable none` (and `none` for the other lists)
  empties the list.
- **Warnings about the machine:** while `nftables.service` is enabled
  (it loads `/etc/nftables.conf` at boot, and flushes every table when
  stopped), and while the table an import came from still applies.
- **Statistics** (Traffic → Statistics, `hc stats`, `/api/v1/stats/...`):
  traffic per interface and zone, drops by chain and when they happen,
  hits per rule and the rules that matched nothing, the sources blocked
  most and what they tried, connections against the kernel's limit, each
  quota's use, and housecarld's and the host's memory and load, sampled
  every minute into a database of their own (`stats.db`, kept 7 days per
  minute, 90 days per hour, 2 years per day). On by default; traffic per
  inside device is a setting (`--stats-per-device`) that adds counters
  to the forward chain. Two charts on the dashboard; a warning when the
  connection tracking table is nearly full (docs/stats.md).
- Set names starting with `meter_`, `conn_`, `quota_` or `stats_` are
  refused (`set_name_reserved`): Housecarl's own sets use them, and a set
  of yours named so could clash with them.
- **The zone matrix** (Matrix, `hc matrix`): what the rules do to new
  connections between zones and to and from the firewall, worked out
  from the rules in order, with each pair's exceptions; rules about
  networks connected to other zones are left out of a pair.
- **Connections and tracing:** the connections the kernel tracks, with
  their NAT, and closing them so the rules check them again (Traffic →
  Connections, `hc connections`); a rule option, "every packet", for
  blocks that stop connections already open; and a packet tracer
  (Traffic → Trace, `hc trace`) that follows packets through every
  table's rules and names what accepted or dropped them.
- **Hits and the log:** packet and byte counters for every rule, NAT rule,
  built-in and policy; packets logged by rules and policies, read from
  netlink log group 4171 (`hc log -f`, the Log page).
- **Dynamic sets** whose entries expire by themselves, changed through
  their own API with tokens that may be limited to one set
  (`sets:write:guests`).
- **Import** of an existing nftables ruleset (`hc import`): rules keep
  their order, defines name the sets, DNAT becomes port forwards, what
  can't be translated stays as raw rules, with notes; `hc takeover`
  deletes the old table, on probation (docs/import.md). DNAT maps and
  verdict maps are expanded into one port forward or rule per element.
- **The `hc` CLI, the REST API and the web UI** (dashboard, rules per
  chain with hits and reordering, NAT, zones, sets, services, the log,
  settings, import, history with undo; the rule dialog keeps the less
  used parts, such as rate limits, quotas and time windows, behind "+"
  buttons; each chain's built-ins are listed, with their hits, and
  switched in a window), with users, roles, scoped API
  tokens, sessions, the audit log, and the web UI's certificate from
  Let's Encrypt through Jarl, as in Steward.
- **Packaging:** Debian packages `housecarl` and `housecarl-cli`, a
  systemd unit whose only privilege is `CAP_NET_ADMIN`, daily backups.
- **The Housecarl logo:** a carved Nordic doorway in Jarl's blue, with a
  simplified version for small sizes, lighter on dark backgrounds; and a
  HOUSECARL wordmark in Jarl's and Steward's angular letters.