adam/jarl
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases 34 Wiki Activity

Add an httpreq endpoint for lego and Traefik

Browse Source 
jarld can now answer lego's httpreq DNS provider
(POST /api/v1/acme/httpreq/present and /cleanup), so lego and Traefik
publish DNS-01 challenges with only environment variables, without the
jarl CLI. lego's username and API token arrive as HTTP Basic auth,
accepted on these routes only.

The endpoint is off by default (acme.httpreq.enabled) and HTTPS-only by
default: jarld's own TLS, or a trusted proxy that says so with
X-Forwarded-Proto or Forwarded (the last value counts). Plain HTTP can be
allowed from loopback only or everywhere. GET /api/v1/acme/httpreq shows
how jarld sees a request, to check a proxy setup.

jarl acme and the endpoint share their code in internal/acme. Documented
in docs/acme.md, configuration.md, api.md and permissions.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
abrown38andClaude Opus 5.5 committed 2026-10-08 13:48:47 +00:00
1 parent 0221c78c8a
commit 6b44b92975
18 files changed
+1106 -73
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/acme.md
+246 -4
View File
@@ -108,7 +108,249 @@ Renew with the same variables and `lego ... renew` in place of `run`.
(`EXEC_PROPAGATION_TIMEOUT`, 60 seconds by default). Jarl answers only
once named serves the record, so this check passes on its first try.
## 6. A dedicated ACME zone (CNAME delegation)
## 6. lego and Traefik over HTTP (httpreq), without the jarl CLI
lego's `httpreq` provider publishes the challenge with an HTTP request
instead of running a program. jarld can answer those requests itself, so
the machine running lego needs nothing from Jarl: no `jarl` binary, no
script, only a few environment variables. That suits Traefik (which
uses lego inside) and lego in a container.
| | `exec` + `jarl-lego` (section 5) | `httpreq` (this section) |
|--|--|--|
| Needs on lego's host | the `jarl` CLI and the script | nothing |
| Turned on in jarld | always available | `acme.httpreq.enabled: true` |
| How the token travels | Bearer header, from the jarl CLI | HTTP Basic password, so HTTPS is required |
| Works in Traefik | no | yes |
Both end up doing exactly the same thing in Jarl (the shared code is in
`internal/acme`): `jarl acme set|unset --exact FQDN VALUE`.
### How it works
lego is given a base URL, a username and a password. For every name on
the certificate it sends
```http
POST /api/v1/acme/httpreq/present HTTP/1.1
Authorization: Basic base64(lego:jarl_...)
Content-Type: application/json
{"fqdn": "_acme-challenge.adbraun.lol.", "value": "LHDhK3oGRvkiefQnx7OOczTY5Tic_xZ6HcMOc_gmtoM"}
```
and, once the CA has checked the record, the same body to
`/api/v1/acme/httpreq/cleanup`. lego treats any 2xx answer as success.
`fqdn` is the full record name, after lego has followed any CNAME (so CNAME
delegation to an ACME zone, section 7, just works).
For each request jarld:
1. refuses it unless the endpoint is enabled (`404 disabled`);
2. refuses it unless it came over HTTPS (`403 https_required`, see
[HTTPS and reverse proxies](#https-and-reverse-proxies));
3. checks the credentials: the password must be a valid API token and the
username the user that owns it (`401`). Failures count towards the
same per-IP lockout as other token failures and appear in the audit log
as `auth.token_failed`;
4. checks the body: `fqdn` must be a domain name and `value` a DNS-01
challenge value (letters, digits, `-` and `_`) (`400`);
5. finds the longest primary zone that contains `fqdn` among the zones the
token can see, in every view that has it (`404` if there is none);
6. sets or removes the TXT record (TTL 60) in one changeset, checked by the
token's grants like any other change (`403` if they don't allow it),
and answers `200` once named serves the change.
```json
{"ok": true, "fqdn": "_acme-challenge.adbraun.lol.", "changeset_id": 412, "zones": ["adbraun.lol (view external)", "adbraun.lol (view internal)"]}
```
Setting a value that is already there, and removing one that isn't,
both succeed, so lego can retry safely. The changeset description ends in
"(lego httpreq)", so `jarl history` and the audit log show where a change
came from.
HTTP Basic is accepted on these routes only. Everywhere else the API
still takes only Bearer tokens and web sessions.
### 1. Enable the endpoint
In `/etc/jarl/jarl.yaml`:
```yaml
acme:
httpreq:
enabled: true
require_https: true # the default; see below
allow_loopback_http: false # the default
```
Then `sudo systemctl restart jarl`. jarld's log says at startup whether
the endpoint can work, for example `acme.httpreq is enabled but jarld has
no tls and no trusted_proxies, so every request will be refused`.
### HTTPS and reverse proxies
The token is in every request, so by default jarld only accepts requests
that reached it over HTTPS. Everything else in Jarl (the web UI, the CLI,
the rest of the API) works over plain HTTP exactly as before; only this
endpoint checks. jarld decides like this:
| The request came... | Counts as HTTPS when |
|--|--|
| straight to jarld with `tls` configured | always |
| from an address in `trusted_proxies` | that proxy says the client used HTTPS: `X-Forwarded-Proto: https`, or if that header is missing, `Forwarded: proto=https` (RFC 7239) |
| from any other address without TLS | never. `X-Forwarded-Proto` and `Forwarded` are ignored, since anyone can send them |
Proxies append to these headers, so when a header holds a list
(`X-Forwarded-Proto: https, http`) jarld uses the **last** value: the one
the nearest proxy, the one it trusts, added. A value a client made up comes
earlier in the list and is ignored. The same rule decides whether the web
UI's session cookie gets the Secure flag (`session.cookie_secure: auto`).
**Setup A: jarld does TLS itself.** Set `tls.cert_file` and
`tls.key_file` ([configuration.md](configuration.md)). Nothing else is
needed.
**Setup B: behind a reverse proxy that terminates TLS.** Add the proxy's
address (as jarld sees it) to `trusted_proxies`, and make sure the proxy
sets `X-Forwarded-Proto`:
```yaml
listen: ["127.0.0.1:8053"]
trusted_proxies: [127.0.0.1] # the proxy runs on this host
```
* **nginx** doesn't set it by default:
```nginx
location / {
proxy_pass http://127.0.0.1:8053;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
```
* **Caddy** (`reverse_proxy 127.0.0.1:8053`) and **Traefik** set
`X-Forwarded-Proto` themselves, and replace the value a client sent.
* **HAProxy**: `http-request set-header X-Forwarded-Proto https if { ssl_fc }`.
lego then uses the proxy's HTTPS URL as its endpoint.
**Setup C: lego runs on the DNS server itself** and talks to
`http://127.0.0.1:8053`. Traffic over loopback never leaves the machine,
so it can be allowed:
```yaml
acme:
httpreq:
enabled: true
allow_loopback_http: true
```
This accepts plain HTTP from `127.0.0.1` and `::1` only, and only when
that address is **not** in `trusted_proxies`. A trusted proxy on the same
host is judged by its headers instead, otherwise everything it forwarded
from the internet would look local. Don't use `allow_loopback_http` if a
reverse proxy runs on the DNS server but isn't in `trusted_proxies`.
**Turning the check off** (`require_https: false`) accepts plain HTTP
from anywhere. jarld logs a warning at startup. Only do this on a network
where you'd also send passwords in the clear.
When a request is refused for not being HTTPS, the answer explains why,
for example: `this request came over plain HTTP from 192.0.2.9, which sent
X-Forwarded-Proto or Forwarded but isn't in trusted_proxies, so the header
was ignored`. jarld also logs a warning and an `acme.httpreq_refused` audit
event. The token in such a request may have crossed the network
unencrypted; if it did, revoke it (`jarl token revoke ID`) and make a new
one.
**Check a setup** without a token. The status endpoint answers without
authentication and says how jarld sees the request:
```sh
curl https://ns1.adbraun.lol/api/v1/acme/httpreq
{"accepted": true, "enabled": true, "https": true, "transport": "HTTPS to trusted proxy 127.0.0.1 (X-Forwarded-Proto: https)"}
```
Run it from where lego runs, against the URL lego will use.
### 2. A user and token for lego
```sh
jarl user create lego --no-password --role acme --comment "lego / Traefik"
jarl token create lego --user lego --role acme -q > lego.token
```
The username lego sends must be the token's user, `lego` here. For CNAME
delegation into an ACME zone, use the zone grant from section 7 instead
of the `acme` role.
### 3. lego
```sh
export HTTPREQ_ENDPOINT=https://ns1.adbraun.lol:8053/api/v1/acme/httpreq
export HTTPREQ_USERNAME=lego
export HTTPREQ_PASSWORD_FILE=/etc/jarl/lego.token # or HTTPREQ_PASSWORD=jarl_...
lego --email you@adbraun.lol --dns httpreq -d adbraun.lol -d '*.adbraun.lol' run
```
Leave `HTTPREQ_MODE` unset (its `RAW` mode sends a different body, which
jarld refuses with a message saying so). If jarld's certificate doesn't come from a public
CA, lego has to trust yours: `SSL_CERT_FILE=/path/to/your-ca.pem` (lego's
`LEGO_CA_CERTIFICATES` covers only the ACME server). To write into one view only, put
it in the endpoint: `.../api/v1/acme/httpreq/view/external`.
### 4. Traefik
Traefik passes the same variables to lego. Static configuration:
```yaml
certificatesResolvers:
letsencrypt:
acme:
email: you@adbraun.lol
storage: /letsencrypt/acme.json
dnsChallenge:
provider: httpreq
```
and in the Traefik container's environment:
```yaml
environment:
HTTPREQ_ENDPOINT: https://ns1.adbraun.lol:8053/api/v1/acme/httpreq
HTTPREQ_USERNAME: lego
HTTPREQ_PASSWORD_FILE: /run/secrets/jarl_token
```
### Trying it by hand
```sh
TOKEN=$(cat lego.token)
curl -u "lego:$TOKEN" -H 'Content-Type: application/json' \
-d '{"fqdn":"_acme-challenge.adbraun.lol.","value":"test-value"}' \
https://ns1.adbraun.lol:8053/api/v1/acme/httpreq/present
dig @ns1.adbraun.lol _acme-challenge.adbraun.lol TXT +short
curl -u "lego:$TOKEN" -H 'Content-Type: application/json' \
-d '{"fqdn":"_acme-challenge.adbraun.lol.","value":"test-value"}' \
https://ns1.adbraun.lol:8053/api/v1/acme/httpreq/cleanup
```
### Answers and what they mean
| Status and code | Meaning |
|--|--|
| `200` | Done; named serves the change. |
| `400 bad_request` | The body isn't `{"fqdn", "value"}` (or is lego's `RAW` mode), or `fqdn` / `value` is invalid. |
| `401 unauthenticated` | No credentials, an invalid, expired or revoked token, or a username that isn't the token's user. |
| `403 https_required` | Not HTTPS, see above. If the token crossed a network unencrypted, revoke it. |
| `403 forbidden` | The token's grants don't allow this TXT record. |
| `404 disabled` | `acme.httpreq.enabled` is off. |
| `404 not_found` | No primary zone that the token can see contains `fqdn` (in that view). |
| `422` and others | The change failed Jarl's validation or named's check, as with any change. |
| `429 rate_limited` | Too many failed logins from this address; wait a few minutes. |
## 7. A dedicated ACME zone (CNAME delegation)
Instead of letting a token write into your main zone, point each challenge
name at a small zone that exists only for ACME. The token may then write
@@ -141,9 +383,9 @@ target:
The hook passes acme.sh's full record name to `jarl acme --exact`, so both
modes work.
* **lego** follows the CNAME itself (unless `LEGO_DISABLE_CNAME_SUPPORT`
is set) and passes the target to `jarl-lego`, which writes there with
`--exact`. The `acme` role allows only names that start with
* **lego** (`exec` or `httpreq`) follows the CNAME itself (unless `LEGO_DISABLE_CNAME_SUPPORT`
is set) and passes the target to `jarl-lego` or jarld, which write there as
given. The `acme` role allows only names that start with
`_acme-challenge`, so the token needs the grant above, not the role. lego
uses the host's resolvers for the lookup; `--dns.resolvers
ns1.adbraun.lol:53` asks the Jarl server instead.
docs/api.md
+18
View File
@@ -28,6 +28,9 @@ change something (anything but GET/HEAD) must send it as `X-CSRF-Token`.
`GET /auth/whoami` returns the user, the token (if any), the effective
permissions and whether the caller is an administrator.
**HTTP Basic**, only on the lego `httpreq` routes (below): the username is
the token's user and the password the API token.
## Errors
```http
@@ -286,6 +289,21 @@ Needs `stats:read`. When statistics are disabled, every endpoint but
curl -sf -H "$AUTH" "$API/stats/top?kind=nxdomain&since=7d&limit=10" | jq -r '.[] | "\(.count)\t\(.key)"'
```
### ACME: lego's httpreq provider
Off unless `acme.httpreq.enabled` is set, and by default HTTPS only. The
full description is in [acme.md](acme.md#6-lego-and-traefik-over-http-httpreq-without-the-jarl-cli).
| Method | Path | Body |
|--------|------|------|
| GET | `/acme/httpreq` | No authentication. `{"enabled", "https", "accepted", "transport"}`: whether the endpoint is on and how jarld sees this request (HTTPS or not, and why) |
| POST | `/acme/httpreq/present` | `{"fqdn": "_acme-challenge.adbraun.lol.", "value": "..."}`. Sets the TXT value in the longest matching primary zone, in every view that has it |
| POST | `/acme/httpreq/cleanup` | Same body. Removes that value (succeeds if it's already gone) |
| POST | `/acme/httpreq/view/{view}/present`, `.../cleanup` | The same, in one view only |
Authenticate with HTTP Basic (user and API token) or a Bearer token.
Success is `200 {"ok": true, "fqdn": ..., "changeset_id": ..., "zones": [...]}`.
### Users and tokens
| Method | Path | Body |
docs/configuration.md
+21 -1
View File
@@ -20,7 +20,7 @@ The CLI has its own small configuration file, described in [cli.md](cli.md).
|-----|---------|---------|
| `listen` | `["127.0.0.1:8053"]` | Addresses to listen on (`host:port`, several allowed). Keep these on localhost and management networks. |
| `tls.cert_file`, `tls.key_file` | (empty: plain HTTP) | PEM certificate chain and key. Both or neither. The files are rechecked every 10 minutes and on SIGHUP (`systemctl reload jarl`), so renewed certificates are used without a restart. |
| `trusted_proxies` | `[]` | Addresses or prefixes of reverse proxies whose `X-Forwarded-For` and `X-Forwarded-Proto` headers are believed. Needed for correct client IPs in the audit log and for rate limiting behind a proxy. |
| `trusted_proxies` | `[]` | Addresses or prefixes of reverse proxies whose `X-Forwarded-For` and `X-Forwarded-Proto` (or `Forwarded: proto=`) headers are believed. Needed for correct client IPs in the audit log and for rate limiting behind a proxy, and for knowing that a proxied request came over HTTPS (the session cookie's Secure flag, `acme.httpreq`). From other addresses these headers are ignored. |
| `session.lifetime` | `12h` | A web session ends this long after login... |
| `session.idle_timeout` | `2h` | ...or after this long without requests. |
| `session.cookie_secure` | `auto` | `auto` sets the cookie's Secure flag when the request came over HTTPS (directly or via a trusted proxy); `always` / `never` force it. |
@@ -51,6 +51,20 @@ The `stats:` section turns on query statistics (off by default). See
[stats.md](stats.md) for every setting and what it changes in named's
configuration. Changing it takes effect when jarld restarts.
### ACME (lego httpreq endpoint)
An endpoint for lego's `httpreq` DNS provider, so lego and Traefik can
publish DNS-01 challenges without the jarl CLI. See
[acme.md](acme.md#6-lego-and-traefik-over-http-httpreq-without-the-jarl-cli)
for how it works and how to set up HTTPS for it. Changes take effect when
jarld restarts.
| Key | Default | Meaning |
|-----|---------|---------|
| `acme.httpreq.enabled` | `false` | Answer `POST /api/v1/acme/httpreq/present` and `/cleanup`. Off, they answer `404 disabled`. |
| `acme.httpreq.require_https` | `true` | Refuse requests that didn't come over HTTPS: jarld's own `tls`, or a proxy in `trusted_proxies` that sends `X-Forwarded-Proto: https`. The rest of Jarl is unaffected by this setting. `false` accepts plain HTTP from anywhere (logged as a warning at startup). |
| `acme.httpreq.allow_loopback_http` | `false` | Also accept plain HTTP from `127.0.0.1` and `::1` (lego on the DNS server itself), unless that address is in `trusted_proxies`. |
### BIND
| Key | Default | Meaning |
@@ -107,3 +121,9 @@ location / {
proxy_read_timeout 300s; # applying a large changeset can take a while
}
```
`X-Forwarded-Proto` tells jarld that the client used HTTPS. It sets the
session cookie's Secure flag, and the lego `httpreq` endpoint needs it
(Caddy and Traefik send it without being told). Check with
`curl https://YOUR-PROXY/api/v1/acme/httpreq`, whose `transport` field says
how jarld saw the request.
docs/development.md
+1
View File
@@ -23,6 +23,7 @@ internal/
stats/ query statistics: statistics-channel poller, response-log tailer, collector, separate SQLite store
web/static/ the web UI: plain ES modules, no build step, embedded with go:embed
api/, client/ JSON types and the Go client shared by server and CLI
acme/ ACME DNS-01 challenge records, shared by jarl acme and the httpreq endpoint
config/ jarld's YAML configuration
deploy/ systemd unit, sysusers.d and tmpfiles.d files
contrib/acme.sh/ acme.sh DNS API hook
docs/permissions.md
+1
View File
@@ -6,6 +6,7 @@
|-----|-----|-------|
| People in the web UI | Username + password → session cookie | argon2id password hashes. The session cookie is HttpOnly and SameSite=Strict and lasts 12h (2h idle). Every change also needs the session's CSRF token, which the UI sends in `X-CSRF-Token`. Changing a password logs out the user's other sessions. |
| CLI, scripts, certbot | API token: `Authorization: Bearer jarl_...` | Shown once at creation, stored hashed. Optional expiry. Revocable. Last use time and IP are recorded. |
| lego / Traefik (`httpreq`) | HTTP Basic: the token's user and the API token | Only on `/api/v1/acme/httpreq/...`, only when `acme.httpreq.enabled` is set, and by default only over HTTPS ([acme.md](acme.md)). The token's grants apply as usual. |
| Root on the server | `jarld` subcommands (`init`, `user`, `import`, `adopt`, `check`, `resync`, `backup`) | Work directly on the database. They are for bootstrapping and recovery, and are audited as "local". |
API tokens are how you give a script, a tool or a person limited access: