housecarl/docs/jarl.md

# Connecting Housecarl to Jarl

Housecarl uses [Jarl](https://gitea.adbraun.lol/adam/jarl), the BIND
manager, for two things:

* the DNS-01 challenge for the web UI's certificate
  ([certificates.md](certificates.md)): Housecarl writes a TXT record;
* **sets from Jarl** ([below](#sets-from-jarl)): sets whose addresses are
  those of names in Jarl, such as `nas.adbraun.lol`, kept up to date.
  Housecarl only reads.

Housecarl talks to Jarl's REST API with an API token. Give it a **user of its
own**, with **only the permission it needs**. Jarl never calls Housecarl.

## 1. A user for Housecarl, in Jarl

On the Jarl server, make a user without a password. It can't log in to
Jarl's web UI, only use its tokens.

For **sets from Jarl**, grant it reading the records of the zones and views
your names are in:

```sh
jarl user create housecarl --no-password --comment "Housecarl on fw01" \
    --grant "records:read zones=adbraun.lol views=internal"
```

Include every zone a name's CNAMEs lead to as well (`zones=adbraun.lol,wg.adbraun.lol`).
Reading is all it needs: it can't change anything in Jarl.

For the **certificate**, grant it the challenge record of Housecarl's own
name, and nothing else:

```sh
jarl user create housecarl --no-password --comment "Housecarl on fw01" \
    --grant "records:write zones=adbraun.lol names=_acme-challenge.fw.adbraun.lol types=TXT"
```

For both, give both grants (`--grant` twice). `jarl user update housecarl
--grant ... --grant ...` replaces a user's grants with the ones given.

Use your zone for `adbraun.lol`, and the name Housecarl's web UI is reached
by for `fw.adbraun.lol` (`tls.acme.domain`). The grant lets the token add
and remove TXT records at exactly `_acme-challenge.fw.adbraun.lol`:

* nothing else in the zone (`names=`);
* no other record type (`types=TXT`);
* no other zone (`zones=`).

It can see that the zone exists, which it needs to find it, but it can't
read the zone's other records.

Why not something wider:

* **Jarl's `acme` role** (`records:write names=_acme-challenge.* types=TXT`)
  would let the token publish challenges for *any* name in *any* zone, and
  so obtain certificates for all of them. Housecarl needs only its own name.
* **Your own account's token** would carry all your permissions, and
  Housecarl's certificate would stop renewing the day your account changes.
  A user per program keeps the two apart, and the audit log shows
  `housecarl/...` for what Housecarl did.

If the challenge name is a CNAME into a zone kept for challenges, such as
`_acme-challenge.fw.adbraun.lol CNAME fw.acme.adbraun.lol.`, grant the
target instead: `zones=acme.adbraun.lol names=fw.acme.adbraun.lol`.

## 2. The token

Make the token, and copy it to the Housecarl server into a file only the
`housecarl` user can read:

```sh
jarl token create fw01 --user housecarl -q > housecarl.token          # on the Jarl server, or with the jarl CLI anywhere
scp housecarl.token fw01:
ssh fw01 'sudo install -m 0600 -o housecarl -g housecarl housecarl.token /etc/housecarl/jarl.token && rm housecarl.token'
rm housecarl.token
```

A token file is better than putting the token in `housecarl.yaml`: the
configuration file is readable by the `housecarl` group, and gets copied
around more readily.

You can give the token an expiry (`--expires 365d`). Rotate it in Jarl with
`jarl token rotate ID --grace 1d`, then put the new secret in the file and
restart housecarld within the grace period. Since the certificate is renewed
every couple of months, nothing breaks while you do. Sets from Jarl keep
their addresses while the token doesn't work (they warn, see
[below](#when-jarl-cant-be-reached)), so they don't break either.

## 3. Housecarl's settings

In `/etc/housecarl/housecarl.yaml`:

```yaml
jarl:
  server: https://dns.adbraun.lol:8053
  token_file: /etc/housecarl/jarl.token
  # ca_cert: /etc/housecarl/jarl-ca.pem   # if Jarl's certificate isn't from a public CA
  # refresh: 1m                           # how often sets from Jarl read their names
```

Then restart housecarld (`systemctl restart housecarl`). Housecarl needs to
reach Jarl's API over the network: Jarl's `listen` must include an
address Housecarl can reach, ideally over HTTPS.

`hc cert check` ([certificates.md](certificates.md#checking)) tests
the connection, the token's permission, and that the record reaches the
internet.

## Sets from Jarl

A set can take its addresses from Jarl instead of listing them: you name
DNS names, and the set holds their addresses. Rules use it like any other
set. When a record changes in Jarl, the set follows within a minute,
without a change to Housecarl's configuration.

```sh
hc set add nas --jarl nas.adbraun.lol@internal --within lan
hc rule add forward --in wg --to nas --service smb
hc set jarl nas                       # what it holds, name by name
```

In the web UI: Sets → Add set, kind "from Jarl". The **Addresses…**
button shows what the set holds.

A set from Jarl has:

* **names**: one or more names, each `NAME@VIEW`, or `NAME` when only one
  of Jarl's views has it ([Views](#views));
* **within**: where its addresses may be, one or more zones or networks
  ([Within](#within)). It is required.

It has no entries of its own, and isn't dynamic. It can be renamed (it goes
by its names in Jarl, not by its own).

### What a name gives

Housecarl reads the name's records as named would answer them in the
view:

* **Every A and AAAA record** of the name. A name with three addresses
  puts all three in the set, IPv4 and IPv6 alike; a rule using the set
  matches each family with its own addresses.
* **CNAMEs are followed**: `files.adbraun.lol CNAME nas.adbraun.lol.` gives
  nas's addresses. Chains are followed up to 8 CNAMEs, from zone to zone
  (a sub-zone such as `wg.adbraun.lol` too), but always in the **same
  view**, and only into zones Jarl holds. `hc set jarl` shows the chain:
  `files.adbraun.lol → nas.adbraun.lol`.
* **Disabled records** are left out, as named leaves them out.

A name gives **no addresses**, and a warning (`jarl_name`) says why, when:

* it has no A, AAAA or CNAME record (only an MX, say, or nothing);
* only a **wildcard** (`*.adbraun.lol`) would answer for it: wildcards
  aren't followed, so name a record of its own;
* a **CNAME leads out of Jarl** (`host.duckdns.org`), into a zone Jarl
  doesn't hold, or to a name with no address;
* the CNAMEs **loop**, or are more than 8;
* it is **below a delegation**: NS records in the zone hand that part to
  other servers, so Jarl doesn't hold its records;
* its zone is a **secondary, mirror, stub or forward zone** (Jarl doesn't
  hold their records), or **disabled** in Jarl;
* no zone the token can see holds it.

A **reverse zone** isn't involved: Housecarl reads the A and AAAA records
only, never PTRs.

An **in-view zone** (one view serving another's zone) is followed to the
zone it serves: `srv.adbraun.lol@vps` reads the internal view's records if
vps's `adbraun.lol` is an in-view of internal's.

### Views

The same name can have different addresses in different views: Jarl's
`internal` view may give `nas.adbraun.lol` its private address, `external`
a public one. A set reads **one view per name**, written `NAME@VIEW`:

* **Name the view your devices see.** The firewall matches the addresses
  packets carry, and for devices inside that is the internal view's.
* **Without a view**, the name is read from the only view that has a zone
  for it. If zones in **more than one view** hold it, the name gives
  nothing until you name the view (`jarl_name`: "is in Jarl's views
  external, internal: name the view"). This counts every view whose zones
  cover the name, even one where the name has no record: a record added
  there later would otherwise change which view Housecarl reads.
* **Linked zones** make no difference: a shared record has the same
  address in every view, and a view's own records are why the view is
  named.
* **Without views** in Jarl, write the name alone.
* A set can mix views: `--jarl nas.adbraun.lol@internal --jarl
  host.adbraun.lol@vps`.

A wrong view can't widen the set: an address from it that isn't
[within](#within) is left out.

### Within

`--within` says where the set's addresses may be. Each entry is:

* **a zone**: the networks connected to its interfaces (the addresses on
  them, as `hc status` lists them), IPv4 and IPv6;
* **a network, address or range**: `10.10.0.0/24`, `2001:db8:10::/64`,
  `192.0.2.7`.

An address from Jarl that isn't within any of them is **left out** of the
set, and a warning names it (`jarl_outside`: "nas.adbraun.lol has address
192.0.2.9, which is left out: not within lan"). The set's other addresses
stay. So a record changed in Jarl, by mistake or by someone who shouldn't,
can only point the set at addresses you allowed.

Zones grow by themselves: when you add a VLAN to the `lan` zone, or give
an interface another network, its addresses count at once. A new zone
counts only once you add it to `within`. If a zone's interfaces have no
address for a while (at boot, a link down), the networks it had last count.

Some addresses never count, whatever `within` says: unspecified (`0.0.0.0`,
`::`), loopback and multicast ones.

`within 0.0.0.0/0` or `::/0` lets any address in, and gives a warning
(`set_jarl_anywhere`): then whoever can change the names in Jarl decides
what the set holds. Name a zone or a network instead. For an address
outside your networks, such as a VPS's public address, name that address
or its network.

### When the addresses change

Housecarl reads the names every `jarl.refresh` (a minute), right after a
change to Housecarl's configuration (a new set or name), when an interface
changes, and when you ask: `hc set jarl NAME --refresh`, or **Read Jarl
now** in the web UI.

When a set's addresses change, housecarld puts the ruleset in force again
with the new ones. That isn't a change of the configuration: there is no
history entry and no probation. It is recorded in the audit log
(`set.jarl`, "added 10.10.0.22; removed 10.10.0.20") and in the journal.

A new set holds nothing for the moment until its names are read, usually
less than a second.

**Open connections aren't closed** when an address leaves a set, as with
any change to a set: the connection's packets are let through as part of an
established connection (or by the flowtable). Close them with `hc
connections close` if they must stop at once.

### When Jarl can't be reached

A set **keeps the addresses it last had** while Jarl can't be read: Jarl is
down, the network to it is, or the token stopped working. Nothing is
emptied, so rules keep working. A warning says so (`jarl_unreachable`), on
the status page and in `hc status`, and `hc set jarl` shows when Jarl last
answered.

What was last read is saved in the database. After a restart or a reboot,
the sets get those addresses at once, before Jarl is read again, and the
boot ruleset ([operations.md](operations.md)) has them too.

When Jarl **answers** that a record is gone, its addresses go. Only a
failure to read keeps them.

Without `jarl.server` in `housecarl.yaml`, sets from Jarl keep what they
had (nothing if Jarl was never read) and warn (`jarl_not_configured`).

### Who decides what the set holds

A set from Jarl lets **whoever can change those records in Jarl** decide
which addresses a rule matches. Keep that in mind for rules that accept:

* `within` limits it to networks you chose;
* Housecarl's token only reads ([step 1](#1-a-user-for-housecarl-in-jarl));
* give the records' zones only to people you'd let change the firewall,
  and check Jarl's audit log for who changed what.

**Don't use names that clients choose for themselves.** A name a DHCP
server makes from the client's own hostname belongs to whoever asks for
it: any guest could call itself `nas`. Name records that you made by hand,
or that come from a DHCP reservation.

### Seeing what a set holds

```
$ hc set jarl nas
Set:        nas
Within:     lan
Holds:      10.10.0.20, 10.10.0.21
Jarl read:  2026-10-23 14:02:11

NAME                                 VIEW      ADDRESS          USED
files.adbraun.lol → nas.adbraun.lol  internal  10.10.0.20       yes
files.adbraun.lol → nas.adbraun.lol  internal  10.10.0.21       yes
files.adbraun.lol → nas.adbraun.lol  internal  2001:db8:10::20  no: not within lan
```

The web UI's **Addresses…** button on the Sets page shows the same, and the
status page lists the warnings: names that give nothing (`jarl_name`),
addresses left out (`jarl_outside`), Jarl not read (`jarl_unreachable`).
The API is `GET /sets/{name}/jarl` ([api.md](api.md#sets-from-jarl)).