adam/housecarl
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 34 Wiki Activity
Files
83b97ad53843c12e6353a7a7693e96ee43bad8bc
housecarl/docs/api.md
T
abrown38andClaude Opus 5.5 68ba2d06c5 Sets from Jarl: sets holding the addresses of names in Jarl 
A set can take its addresses from Jarl: every A and AAAA record of its
names (NAME@VIEW), CNAMEs followed within Jarl's zones in the same view.
housecarld reads them every jarl.refresh and after each change, keeps
only the addresses within the zones and networks the set allows, and
keeps what it last read (saved in the database and the boot ruleset)
while Jarl can't be reached. A change of the addresses applies the
ruleset again without a history entry and is audited (set.jarl).

hc set add --jarl --within, hc set jarl; the kind "from Jarl" and
Addresses… on the Sets page; GET /sets/{name}/jarl and /jarl/refresh;
validation codes set_jarl_*, status warnings jarl_*; docs/jarl.md.
e2e phase with scripts/fakejarl.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:51:55 +01:00

305 lines
19 KiB
Markdown
Raw Blame History

# The REST API
Everything the web UI and the CLI do goes through this API, under
`/api/v1`. Requests and answers are JSON.
## Authentication
* **API tokens**: `Authorization: Bearer housecarl_...`. Make one in the
web UI (your account), with `hc token create NAME`, or with `housecarld
user token USER NAME` on the server.
* **Sessions**: `POST /auth/login` with `{"username", "password"}` sets a
cookie and returns `csrf_token`. Requests other than GET must send it as
`X-CSRF-Token`.
* **The local socket** (`/run/housecarl/housecarl.sock`): root and
housecarld's own user need nothing; others send a token. Changes made
through it aren't on probation.
Failed logins and token attempts are rate limited per address and per
user name. [permissions.md](permissions.md) has what each call needs.
## Errors
```json
{"error": {"code": "validation", "message": "rule input #31: there is no set trusted",
"problems": [{"severity": "error", "code": "unknown_set", "message": "from: there is no set trusted", "object": "rule input #31"}]}}
```
| Status | Code | |
|---|---|---|
| 400 | `bad_request` | the request can't be read |
| 401 | `unauthenticated`, `session_expired`, `login_failed` | |
| 403 | `forbidden`, `csrf` | |
| 404 | `not_found` | |
| 409 | `conflict` | e.g. undoing a change after the objects changed again; a takeover or closing connections in shadow mode ([shadow.md](shadow.md)) |
| 422 | `validation` | the change would make the configuration invalid; `problems` lists why ([validation.md](validation.md)) |
| 422 | `firewall` | nftables refused the ruleset (`nft -c`), or applying it failed; the message has nftables' words. Nothing was changed |
| 429 | `rate_limited` | |
## Changes
Calls that change the configuration answer:
```json
{"applied": true, "history_id": 7,
"changes": [{"kind": "rule", "id": 31, "label": "input #31 (SSH from home)", "after": {...}}],
"warnings": [{"severity": "warning", "code": "set_empty", "message": "...", "object": "..."}],
"object": {...},
"confirm_by": "2026-10-19T10:02:00Z"}
```
* `confirm_by`: the change is on probation, and is rolled back at that
time unless `POST /confirm` comes first. Missing for changes made
through the socket, and in shadow mode ([shadow.md](shadow.md)).
* `confirm_token`: the client that made the change can confirm it by
itself with `POST /confirm {"token"}`, from the same address, as the
same user and token, over a new connection (the answer closes this
one). Missing when a person must confirm (`manual_confirm`).
* `?dry_run=1` checks the change, with nftables too, and returns what it
would do, with `"applied": false, "dry_run": true`, and `diff`: the
ruleset's unified diff, each `@@` line followed by the chain (or other
block of the table) the hunk starts in, as in `@@ -40,6 +40,8 @@ chain
forward`.
* `?changeset=ID` (or the header `X-Housecarl-Changeset: ID`) stages the
change in that changeset instead ([Changesets](#changesets)): the answer
has `"staged": true`, `changes` (this operation's) and `changeset`.
* `before` is missing for a create and `after` for a delete.
* `warnings` are the problems the change brings; older ones are in
`/status`.
* `object` is the created or changed object.
Creating answers 201, other changes 200.
## Endpoints
### Status
| | |
|---|---|
| `GET /healthz` | `ok`, `version`, `firewall` (`applied`, `unconfigured`, `fallback`, `missing`), `mode` (`enforcing`, or `shadow`: the ruleset blocks nothing, [shadow.md](shadow.md)), `probation` (changes waiting to be confirmed), `warnings` (codes); HTTP 503 unless `ok`. Without authentication; read at most every five seconds ([operations.md](operations.md#monitoring)) |
| `GET /status` | `build`, `started`, `mode` (`enforcing` or `shadow`), `shadow_position` (`after` or `before`, in shadow mode), `applied` (the ruleset is in force), `apply_error`, `problems`, `probation`, `interfaces`, `counts` (objects by kind), `other_tables` (other nftables tables, `"inet main"`) |
| `GET /interfaces` | the machine's interfaces: `name`, `up`, `addresses`, `zone` |
| `GET /matrix[?changeset=ID]` | the zone matrix: `zones`, `cells` (`from`, `to` (a zone, or `firewall`), `chain`, `summary` (allowed, blocked, partly), `verdict` (what the rest gets: allowed, blocked), `decided_by`, `rule_id`, `hits`, `allowed` and `blocked` (exceptions by verdict), `open` (what the allowing exceptions let through, in short), `same_zone` and `note`, `exceptions`: `rule_id`, `chain`, `label`, `verdict` (allowed, blocked, logged, unknown), `what`, `short`, `hits`). `hits` are the rule's or policy's packets since housecarld started, all its traffic (missing: no counter) ([rules.md](rules.md#the-zone-matrix)) |
| `GET /ruleset[?enforcing=1]` | the nftables ruleset Housecarl builds, as text. In shadow mode it is the shadow ruleset; `enforcing=1` gives the one loaded once shadow mode is off |
| `GET /counters` | `{"since", "counters": {"r12": {"packets", "bytes"}, ...}}`, by mark ([design.md](design.md#its-own-table-only)); totals since housecarld started |
| `GET /quotas` | each rule's quota: `rule_id`, `rule`, `quota`, `used` (bytes), `addresses` (`[{"address", "used"}]` for a per-address quota), `last_reset`, `next_reset` |
| `GET /connections[?q=&limit=]` | tracked connections: `family`, `protocol`, `source`, `source_port`, `destination`, `destination_port` (or `icmp_id`, `icmp_type`, `icmp_code`), `dnat_to`, `dnat_port`, `snat_to`, `state`, `assured`, `timeout` (seconds left), `packets` and `bytes` when the kernel counts them. `q`: all of its words in the addresses, `:port`, protocol, state |
| `POST /names` | `{"addresses": [...]}` (at most 2000): `{"enabled", "names": {"192.168.1.20": "laptop.example.net"}}`, the names of the inside addresses among them, by reverse lookup ([configuration.md](configuration.md#names)). Addresses without a name, outside ones, and those still being looked up after a second are left out; ask again for those. `log:read` |
| `POST /connections/close` | `{"connections": [...]}` (as listed) or `{"search": "192.168.1.50"}`: delete their tracking entries (`config:write`, audited); answers `{"closed": N}`. 409 in shadow mode |
| `POST /trace` | `{"address", "port", "protocol", "seconds"}` (5-600, default 60): start tracing (`config:write`, audited); 409 while one runs |
| `GET /trace` | the trace running, or the last: `running`, `input`, `started`, `ends`, `error`, `packets`: each with `time`, `in`, `out`, `protocol`, addresses and ports, `tcp_flags`, `icmp_type`, `steps` (`table`, `chain`, `kind` rule or policy, `rule`, `mark`, `label`, `verdict`), `verdict` (accepted, dropped) and `decided_by`; in shadow mode `would_drop`, the step where Housecarl would have dropped it |
| `DELETE /trace` | stop it |
| `GET /log[?limit=&after=&q=]` | logged packets, newest first: `time`, `prefix`, `in`, `out`, `family`, `protocol`, `source`, `source_port`, `destination`, `destination_port`, `icmp_type`, `tcp_flags`, `length`, `mark` (the rule's), `verdict`, `shadow` (logged in shadow mode: a drop or reject is what would have been done). `after` (a time) for those since; `q` searches |
| `GET /catalog` | built-in services, chains, actions, roles, the log group |
### Settings
`GET /settings`, `PATCH /settings` with any of:
```json
{"input_policy": "drop", "forward_policy": "drop", "output_policy": "accept",
"log_policy_drops": false,
"builtins": {"loopback": true, "established": true, "drop_invalid": true, "icmpv6": true,
"ping_internal": true, "dhcpv6_client": true, "antispoof": true, "mss_clamp": false},
"own_prefixes": ["2001:db8::/48"], "antispoof_v4": ["10.0.0.0/8", "..."], "antispoof_v6": ["fc00::/7", "..."],
"flowtable_devices": ["wan", "eth1"], "probation_seconds": 120, "manual_confirm": false,
"stats_per_device": false}
```
Lists replace the old ones. `probation_seconds` 0 is the default (120).
### Objects
Zones, sets, services, rules, port forwards, source NAT, redirects and
ban lists share one pattern, `/{kind}` with kind `zones`, `sets`,
`services`, `rules`, `forwards`, `snats`, `redirects` or `banlists`:
| | |
|---|---|
| `GET /{kind}` | all of them, in order; `GET /rules?chain=input` for one chain; `GET /services` includes the built-in ones |
| `POST /{kind}[?before=ID\|after=ID\|first=1]` | create; rules, forwards and NAT rules go last unless given a `position`, or placed before or after another of their chain, or first, in the same change (one history entry; staged in a changeset, the place goes with it) |
| `GET /{kind}/{ref}` | by ID, or name for zones, sets, services, forwards and ban lists |
| `PATCH /{kind}/{ref}` | change the fields given (a JSON merge: `null` clears) |
| `DELETE /{kind}/{ref}` | refused while something uses it |
| `POST /rules/{ref}/reset-quota` | start a rule's quota again now (`config:write`; audited, not staged). Optional body: `{"address": "192.168.1.23"}` starts only that address's again (a per-address quota; an IPv6 address counts for its /64; 404 if it hasn't used the quota); `{"give": 5368709120}` takes that many bytes off what was used instead (the address's, or every address's), until the next reset |
| `POST /{kind}/{ref}/move` | `{"before": ID}`, `{"after": ID}` or `{"to": "start"\|"end"}`; a rule can take `"chain"` too |
The objects:
```json
{"name": "lan", "interfaces": ["eth1", "eth1.*"], "external": false, "comment": ""}
{"name": "trusted", "entries": ["192.168.1.0/24", "2001:db8:1::/64"], "dynamic": false, "timeout": 0, "comment": ""}
{"name": "nas", "entries": [], "jarl": {"names": [{"name": "nas.example.com", "view": "internal"}], "within": ["lan", "10.9.0.0/16"]}, "comment": ""}
{"name": "web", "entries": ["tcp/8080", "tcp/8443"], "comment": ""}
{"chain": "input",
"in_zones": ["wan"], "not_in": false, "out_zones": [], "not_out": false,
"sources": ["trusted"], "not_sources": false, "destinations": [], "not_destinations": false,
"services": ["ssh"], "family": "",
"ct_state": ["new"], "ct_status": [], "days": [], "hour_from": "", "hour_to": "",
"limit": {"rate": 3, "unit": "", "per": "minute", "burst": 5, "over": true, "per_source": true, "per_destination": false}, "conn_limit": 0,
"quota": {"bytes": 53687091200, "over": true, "per_source": true, "per_destination": false, "reset": "monthly", "day": 15},
"group_v4": 0, "group_v6": 0,
"action": "drop", "reject_with": "", "log": true, "log_prefix": "",
"raw": "", "comment": "", "disabled": false}
{"name": "web", "protocols": ["tcp"], "ports": "8443", "public_addresses": ["203.0.113.10"], "sources": [],
"to": "192.168.1.10", "to_port": "443", "to6": "2001:db8:1::10", "hairpin": true, "hairpin_sources": [],
"comment": "", "disabled": false}
```
A set with `jarl` takes its addresses from Jarl ([jarl.md](jarl.md#sets-from-jarl)):
`names` are names in Jarl, each with a `view` (left out: the only view that
has it); `within` are zones and addresses its addresses must lie in. It
has no `entries` and isn't `dynamic`; `"jarl": null` makes it an ordinary
set again.
A source NAT rule is a match (as a rule's, without the rule's other
fields) with `"to"` (empty: masquerade), a redirect a match with
`"to_port"`. A ban list ([bans.md](bans.md)):
```json
{"name": "web", "scope": "hosts", "hosts": ["10.0.10.57"], "never": ["trusted"],
"default_time": 3600, "max_time": 604800, "max_entries": 10000, "group_v6": 64, "comment": "", "disabled": false}
```
`scope` is `firewall`, `hosts` or `everything`; 0 is the default for the
numbers, and `max_time` -1 is no limit. A ban list can't be renamed.
Someone with a `bans` grant but no `config:read` may read the ban lists
the grant covers (`GET /banlists`, `GET /banlists/{ref}`). Times are
seconds (`timeout`).
### Statistics
`GET /stats/status`, `/stats/summary`, `/stats/timeline`, `/stats/top`,
`/stats/hits` and `/stats/heatmap` (`log:read`): see
[stats.md](stats.md#the-api).
### Changesets
Changes staged to be reviewed and applied together ([design.md](design.md#reviewing-changes-changesets)).
| | |
|---|---|
| `GET /changesets[?status=open&mine=1&limit=]` | newest first |
| `POST /changesets` | `{"description"}`; answers the changeset (201) |
| `GET /changesets/{id}` | `{"id", "username", "via", "description", "status": "open"\|"applied"\|"discarded", "ops": [{"seq", "op", "kind", "ref", "id", "data", "summary", "added_at"}], "created_at", "updated_at", "history_id"}` |
| `PATCH /changesets/{id}` | `{"description"}` |
| `GET /changesets/{id}/preview` | `{"valid", "changes", "warnings", "error", "diff"}`: what applying it would do; `error.op` names an operation that no longer applies |
| `POST /changesets/{id}/apply` | optional `{"description"}`; answers like a change, with `confirm_by` over the network and `changeset` |
| `POST /changesets/{id}/discard` | |
| `DELETE /changesets/{id}/ops/{seq}` | take one operation out |
Staging (`?changeset=ID` on any change to objects or settings) checks
the changeset with the new operation, and nftables checks its ruleset;
an error refuses the operation. Only open changesets take operations, and
only their owner's (or an administrator's). An undo
(`POST /history/{id}/revert?changeset=ID`) is staged as one operation,
which checks when it is replayed that nothing it touches changed since;
applied, History links it to the entry it undoes. `/import/nft` and
`/takeover` can't be staged: with a changeset named they answer 400.
Reads of objects, settings and `/ruleset` with `?changeset=ID` show the
configuration as the changeset will make it; objects are marked
`"pending": "new"`, `"changed"` or `"deleted"` (deleted ones stay in the
list). A changeset that no longer applies shows the current
configuration, with the reason in the `X-Housecarl-Changeset-Error`
header.
### Dynamic sets' entries
| | |
|---|---|
| `GET /sets/{name}/elements` | `[{"address", "timeout"}]`, `timeout` the seconds left |
| `POST /sets/{name}/elements` | `{"address": "10.0.30.17", "timeout": 3600}`; `timeout` 0 is the set's |
| `DELETE /sets/{name}/elements/{address}` | |
These need `sets:read` or `sets:write`, for that set
([permissions.md](permissions.md)); they aren't on probation or in the
history, but are audited.
### Sets from Jarl
| | |
|---|---|
| `GET /sets/{name}/jarl` | what the set holds and why (below) |
| `POST /sets/{name}/jarl/refresh` | reads Jarl now (for every set from Jarl), then answers as `GET` |
They need `sets:read` for that set. 400 for a set that doesn't come from
Jarl. The answer:
```json
{"set": "nas", "configured": true, "read": "2026-10-23T14:02:11Z", "error": "",
"entries": ["10.10.0.20", "10.10.0.21"], "within": ["lan"],
"names": [{"name": "files.example.com", "view": "", "found_view": "internal", "chain": ["nas.example.com"],
"addresses": [{"address": "10.10.0.20", "used": true},
{"address": "2001:db8:10::20", "used": false, "why": "not within lan"}],
"problem": "", "checked": "2026-10-23T14:02:11Z", "error": ""}]}
```
* `configured`: `housecarl.yaml` names a Jarl server; `read`: when Jarl's
zones were last read; `error`: why the last read failed (the set keeps
its addresses).
* `entries`: the addresses the set holds now.
* For each name: `found_view` is the view it was read from, `chain` the
CNAMEs followed, `addresses` every address Jarl gives it with `used`
(or `why` not), `problem` why it gives none, `checked` when Jarl last
answered for it, and `error` why the last try failed (its addresses are
then those of `checked`).
### Bans
| | |
|---|---|
| `GET /bans` | the active bans of every ban list you may read: `id`, `list_id`, `list`, `address` (an address, or an IPv6 prefix), `protocol` and `port` (none: every port), `expires` and `timeout` (seconds left; none: no end), `created`, `who`, `source`, `reason` |
| `GET /banlists/{ref}/bans` | one list's |
| `POST /banlists/{ref}/bans` | ban: `{"address": "203.0.113.9", "protocol": "tcp", "ports": "ssh", "time": 3600, "source": "guard/ssh", "reason": "…"}`. `ports` is a number, a string (`"22"`, `"http,https"`, `"0:65535"` or `"all"` for every port) or a list; without a protocol a number is TCP and UDP, a service its own protocols. `time` 0 is the list's default, -1 its longest; longer is cut down. 201 with `{"bans": [ban…, "existing": true for a renewed one], "closed": N}` (connections closed) |
| `DELETE /banlists/{ref}/bans/{id}` | lift one ban |
| `DELETE /banlists/{ref}/bans?address=A` | lift all of an address's bans (404 if it has none) |
These need `bans:read` or `bans:write` for the list
([permissions.md](permissions.md)); bans aren't on probation or in the
history, but are audited (`ban.add`, `ban.remove`). Refusals: `422
never_ban` (inside, the firewall's own, the caller's own, or in the
list's `never`), `422 list_full`, `409` for a disabled list.
### Probation
| | |
|---|---|
| `GET /probation` | `{"pending": [history IDs], "takeovers": N, "deadline", "seconds_left"}` |
| `POST /confirm` | keep them: `{"confirmed": N}`; the ruleset is saved for the boot. With `{"token"}`, the client confirming its own change (see `confirm_token`): 409 if a newer change is pending, or over the change's own connection; 403 from another user or address |
| `POST /rollback` | undo them now |
### History, import, takeover
| | |
|---|---|
| `GET /history[?limit=&before=]` | changes, newest first |
| `GET /history/{id}` | one, with each object before and after |
| `GET /history/{id}/diff` | `{"diff"}`: what it did to the ruleset, as a unified diff with each hunk's chain (the configuration before and after it, got back from the current one, rendered by this version) |
| `POST /history/{id}/revert` | undo it; 409 if an object it touched has changed since, or it was undone already |
| `POST /import/nft` | `{"json": NFT_JSON, "text": "nft -a list ruleset output", "defines": "ruleset file", "table": "inet main", "zones": {"enp2s0": "lan"}, "replace": false}` ([import.md](import.md)); answers with `state` (the imported configuration), `notes` and `raw` too |
| `POST /takeover` | `{"tables": ["inet main"]}`: delete these tables (administrators); on probation over the network; 409 in shadow mode |
### Certificate
Administrators only ([certificates.md](certificates.md)).
| | |
|---|---|
| `GET /certificate` | `{"configured", "source": "acme"\|"file"\|"none", "status": {"domain", "directory", "staging", "state": "none"\|"ok"\|"working"\|"error", "not_before", "not_after", "issuer", "renew_at", "last_error", "last_attempt", "next_attempt", "check"}}` |
| `POST /certificate/check` | run the check now (up to a few minutes); answers `{"time", "domain", "ok", "steps": [{"name", "status", "detail", "queries"}]}` |
| `POST /certificate/renew` | get a new certificate now, in the background (202); follow it with `GET /certificate` |
### Users, tokens, audit
| | |
|---|---|
| `POST /auth/login`, `POST /auth/logout`, `GET /auth/whoami` | `whoami` has `token_id` when the caller uses a token |
| `POST /me/password` | `{"current", "password"}` |
| `GET /me/tokens`, `POST /me/tokens` | `{"name", "role", "grants", "expires_in"}`; the answer's `secret` is shown once |
| `GET /tokens`, `DELETE /tokens/{id}` | all tokens (administrators), revoke |
| `POST /tokens/{id}/rotate` | a new secret: `{"grace": "1h", "expires_in": "90d", "no_expiry": false}`, all optional; answers with `secret` and `previous_secret_valid_until` |
| `GET`, `POST /users`; `PATCH`, `DELETE /users/{id}` | administrators; `{"username", "password", "admin", "role", "grants", "disabled", "comment"}` |
| `GET`, `POST /users/{id}/tokens` | |
| `GET /audit[?user=&action=&result=&q=&before=&limit=]` | |
Reference in new issue View Git Blame Copy Permalink