housecarl/docs/import.md

# Importing an nftables ruleset

`hc import` turns the ruleset you have into Housecarl's configuration,
and `hc takeover` deletes the old table once Housecarl's replaces it.
Until then both apply.

## 1. Look first

On the firewall, as root:

```sh
sudo hc import --defines /etc/nftables.conf -n
```

`hc` runs `nft -j list ruleset` (and `nft -a list ruleset`, for the text
of rules it can't translate) and sends both to housecarld. `-n` (dry run)
shows what it would make, without saving anything: zones, sets,
services, rules, port forwards and NAT, then **notes** on every guess it
made, and the rules it **kept raw**.

`--defines` reads your ruleset file's `define` lines, so sets and zones
get your names ([How things are named](#how-things-are-named)).
`--zone enp2s0=lan` names a zone yourself.

From another machine, give the JSON as a file, and the text with
`--text`:

```sh
ssh fw 'sudo nft -j list ruleset' > ruleset.json
ssh fw 'sudo nft -a list ruleset' > ruleset.txt
hc import ruleset.json --text ruleset.txt --defines nftables.conf -n
```

With several tables, `--table "inet main"` picks one; by default every
table but Housecarl's is read. The web UI has the same under Server →
Import a ruleset.

## How things are named

**Zones** are named in this order:

1. **As you say**: `--zone enp2s0=lan` (repeat it; Zone names on the
   web UI's Import page). Interfaces given the same name share one zone
   (`--zone enp2s0=lan --zone enp3s0=lan`).
2. **After a define holding the interface**, with a trailing `_int`,
   `_if`, `_iface`, `_ifname`, `_interface`, `_dev`, `_device` or `_nic`
   taken off: `define wan_int = "ppp0"` makes the zone `wan`, `define
   iot_if = enp2s0.20` (a VLAN) the zone `iot`.
3. **After the interface itself**: `enp2s0`.

If the name is taken, the zone is called `INTERFACE_zone`. A define
names what it was *for* in your old ruleset, which isn't always what the
network is (`define dhcp_int = enp2s0` makes `dhcp`, for your LAN). A
note lists each zone and where its name came from; rename one with
`--zone` and import again, or afterwards with `hc zone update dhcp
--name lan` (its rules follow). Name zones for what the network is for:
`lan`, `iot`, `guest`, `servers`, `vpn`, `wan`.

**Address sets**: a named set of the table keeps its name; a define
holding a list of addresses, or one prefix, gives its name to a set
(`define trusted = { 10.0.10.0/24, 10.1.1.0/24 }` makes `trusted`,
wherever those addresses appear together). A single address stays an
address. When two defines hold the same addresses, the first by name
wins.

**Services**: a define holding several ports names a service (`define
web_ports = { 80, 443 }` makes `web_ports`); one port stays a port
(`tcp/22`).

**Port forwards** are named after their comment, without a leading
"Port forward" (`comment "Port Forward Jellyfin"` makes `jellyfin`);
several ports in one DNAT become one forward each (`web-80`, `web-443`),
and a DNAT map's elements are named after the map (`forwards-443`).

Names that aren't valid are made so: other characters become `_` (`-`
in forward names), and a name must start with a letter.

## What becomes what

* **Interfaces** become zones. Zones are marked external when the rules
  show it: MSS clamping on the way out, DHCPv6 client replies, ping
  accepted from everywhere else.
* **Sets and defines** of addresses become address sets; port lists
  become services.
* **Jumps and gotos** to your own chains are flattened: the jump's
  condition is added to each rule of the chain, in place. When the jump
  and a rule both match sources (or destinations), the narrower is kept
  if one lies within the other; otherwise, and for any rule there that
  stays raw, the jump's condition is written in front of the raw rule's
  text, so it still only applies where the jump did.
* **Rules** keep their order. A rule that is a built-in (loopback,
  established, invalid, the ICMPv6 rules, MSS clamping, the DHCPv6 client
  rule, antispoofing) becomes that built-in only when it sits where the
  built-in goes, at the head of every chain it covers. Otherwise it stays
  a rule where it was, so the ruleset does what it did. Since built-ins
  come before every rule, once one at a chain's head stays a rule, the
  ones after it there stay rules too, in order: an invalid drop that
  stays a rule keeps the ICMPv6 accepts after it behind it, as they were.
  Built-ins your ruleset doesn't have are turned off.
* **DNAT** rules become port forwards, TCP and UDP of the same port
  merged into one, one per port when a rule has several. The
  forward-chain accepts that let them through (`ct status dnat`) become
  part of the port forward; an accept to the same host and port without
  `ct status dnat` lets in more than forwarded connections, and stays a
  rule. A masquerade for inside sources to the same host becomes its
  hairpin. An IPv6 accept to the same port, from everywhere, becomes the
  forward's IPv6 pinhole when there is only one candidate. A DNAT to an
  IPv6 address, or with matches a port forward has no field for, is left
  out with a note.
* **Maps are expanded**, one element at a time. A DNAT map
  (`dnat ip to tcp dport map @forwards`, named or written in the rule)
  becomes one port forward per element; forwards from a named map are
  called after it (`forwards-443`). A verdict map
  (`tcp dport vmap { 22 : accept, 23 : drop }`) becomes one rule per
  element, in place, with `jump`s inlined as above. A lookup matches at
  most one element, so the rules do the same. A map keyed by several
  fields (`ip saddr . tcp dport`) can't be expanded: a rule using a named
  one is left out with a note, to be added by hand.
* **Masquerade and SNAT** become source NAT; **redirect** becomes a
  redirect.
* **Meters** (`update @m { ip saddr limit rate over 3/minute }`,
  `add @m { ip saddr ct count over 20 }`, quotas in sets) become the
  rule's per-source (or per-destination) limit, connection limit or quota;
  a mask in the key (`ip saddr and 255.255.255.0`) becomes its grouping.
  Byte rates and quotas are imported too; a quota imported resets only by
  hand until you give it a period. A rule with a quota or a byte rate that
  accepts stays a raw rule ([rules.md](rules.md#bandwidth-and-quotas)).
* **Sets with `flags timeout`** (filled by a program or a rule) become
  dynamic sets, without their entries of the moment: whatever fills them
  must fill Housecarl's from then on, as a note says.
* **Policies** become the chains' policies.
* **Anything else** becomes a raw rule with its original text, in place:
  a match Housecarl has no field for, two matches on the same field
  (`ip saddr 10.0.0.0/8 ip saddr != 10.0.10.0/24`), `!=` where Housecarl
  only has `==`, a set of interface names. Its set references are made
  to work in Housecarl's table: an imported address set becomes
  `@NAME_v4` or `@NAME_v6`, any other set (interface names, ports) is
  written out with its elements. A raw rule naming a set that can't be
  (a meter, an empty set) is left out with a note.
* **Logging rules** log to Housecarl's log (`hc log`), not the kernel's.

Several base chains on one hook (`input` at priority -10 and at 0) are
put one after the other, by priority, into Housecarl's one chain, with
the stricter policy; a note says to check it, since a packet had to get
past each, and an accept in the first no longer ends there. A prerouting
filter chain after connection tracking becomes the early chain, which
runs before it: a note says so. Chains on hooks Housecarl doesn't manage
(ingress, a bridge table) are left out, with a note.

## 2. Import

```sh
sudo hc import --defines /etc/nftables.conf
sudo hc ruleset                  # what Housecarl now applies
sudo hc status                   # problems, and the other tables
```

Housecarl's table is now in force **alongside** the old one: a packet
must be accepted by both. Nothing should change for your traffic. If
Housecarl already has a configuration, `--replace` replaces it.

To be sure first, import in **shadow mode** ([shadow.md](shadow.md)):
Housecarl's ruleset is then loaded but drops nothing, and its log shows
each packet the old ruleset lets through that Housecarl would drop. When
there are none you mind, leave shadow mode and take over.

Imported over the network (not through the local socket), the import is
on probation like any change: confirm it.

## 3. Take over

Once the rules look right (and their hits in the web UI agree), delete
the old table:

```sh
sudo hc takeover "inet main"
```

From now on only Housecarl's rules apply. Until you do, the dashboard
and `hc status` warn that the imported table still applies
(`takeover_pending`); they also warn while `nftables.service` is enabled
(`nftables_service`). Over the network, this is on
probation too: unless confirmed, the old table comes back.

## 4. The boot

Housecarl's ruleset is loaded at boot by `housecarl-ruleset.service`.
Debian's `nftables.service` loads `/etc/nftables.conf`, which usually
starts with `flush ruleset`: that would delete Housecarl's table, or
bring the old one back. Retire it:

```sh
sudo systemctl disable nftables.service
sudo mv /etc/nftables.conf /etc/nftables.conf.before-housecarl
```

Disable it, but **don't stop it** (no `systemctl stop nftables`, no
`disable --now`): Debian's unit stops by running `nft flush ruleset`,
which deletes every table, Housecarl's and Docker's included, at once.
Disabled, it simply doesn't run at the next boot.

`housecarl-ruleset.service` runs after `nftables.service` in any case,
so a leftover `flush ruleset` can't undo it. Docker, Incus and libvirt
keep adding their own tables as before.

## Checking the result

* `hc rule list input` (and `forward`) side by side with
  `nft -a list table inet main`: the same rules, in the same order, with
  built-ins above.
* The notes: each says what to check.
* `hc counters` after a while: a rule with no hits that had hits before
  is worth a look.