Compare commits
56
Commits
No files matched your search
@@ -30,6 +30,10 @@ Go module `housecarl`, `CGO_ENABLED=0`, Linux only for housecarld.
|
||||
on probation and rolled back unless confirmed (docs/design.md). An
|
||||
empty configuration applies nothing. But never force a rule that keeps
|
||||
SSH or the web UI open: the maintainer doesn't want one.
|
||||
* **Shadow mode changes no packet** (docs/shadow.md): anything new the
|
||||
renderer writes that drops, rejects, translates or alters packets needs
|
||||
its shadow form (counted, marked `[would …]`) or must be left out
|
||||
there. `TestShadow` fails on what slips through.
|
||||
* **Stay within nftables 1.1.3** (Debian 13). The dev machine's newer nft
|
||||
accepts things 1.1.3 refuses; the tests with `HOUSECARL_NFT` and the
|
||||
e2e test use 1.1.3.
|
||||
|
||||
+280
-4
@@ -6,6 +6,197 @@ All notable changes to Housecarl are documented here. The format follows
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
|
||||
- An administrator can make an API token for another user in the web
|
||||
UI: New token on the user's row under Users (the CLI's `hc token create
|
||||
--user` and the API could already).
|
||||
- **Ban lists** ([docs/bans.md](docs/bans.md)). A program (Guard, which
|
||||
reads logs, or fail2ban) bans an address, of every port or of some,
|
||||
for a time; the ban list says what the address is kept from: the
|
||||
firewall itself (`firewall`), some hosts behind it (`hosts`, port
|
||||
forwards to them included), or `everything`. Bans are dropped ahead of
|
||||
every rule, established connections too, and the banned address's
|
||||
connections are closed. IPv6 addresses are banned by /64 (`group_v6`).
|
||||
Never banned: inside networks, the firewall's own addresses, whoever
|
||||
asks, and the list's `never`. Bans end by themselves, are cut to the
|
||||
list's longest time, survive restarts and reboots, and are audited, but
|
||||
aren't configuration (no history, probation or changesets). `hc
|
||||
banlist`, `hc bans`, `hc ban`, `hc unban`; Traffic → Bans in the web UI;
|
||||
`/banlists` and `/banlists/{ref}/bans` in the API; a `bans` grant that
|
||||
can name lists (`bans:write:web`), in the `viewer` and `operator`
|
||||
roles.
|
||||
- **A fail2ban action**, `contrib/fail2ban/action.d/housecarl.conf`
|
||||
(installed under `/usr/share/doc/housecarl/contrib/`): fail2ban's bans
|
||||
go into a ban list, with the jail's port and protocol.
|
||||
- **Failed logins in the journal.** A failed login and a refused API
|
||||
token are logged (`msg="login failed" ip=… user=…`), for Guard and
|
||||
fail2ban to act on.
|
||||
|
||||
- **Quota usage as a share.** What a quota used shows its percentage
|
||||
(`12.3 GB of 50.0 GB used (24%)`, each address's in the quota
|
||||
dialog), on the Rules page, the dashboard and in `hc quotas`.
|
||||
- **Reset one address's quota, or give some back.** `hc rule
|
||||
reset-quota ID --address ADDR` starts only that device's allowance
|
||||
again; `--give 5gbytes` takes that much off what was used instead (one
|
||||
address's, or everyone's): extra allowance until the next reset,
|
||||
without changing the rule. In the web UI, a rule's quota is one
|
||||
line (what it used, how it resets) that opens a dialog with the rest:
|
||||
usage per address, last and next reset, and Reset and Give back for
|
||||
everyone or one address. `POST /rules/{ref}/reset-quota` takes `address` and `give`.
|
||||
- **Names beside inside addresses.** Connections, the log, quota usage,
|
||||
the dashboard's quotas and the statistics' devices and blocked
|
||||
sources show each inside address's name, found by a reverse lookup:
|
||||
`laptop 192.168.1.20` in the web UI, `192.168.1.20 (laptop)` in `hc`.
|
||||
Only inside addresses are looked up (an inside zone's networks, and
|
||||
private ones outside the external zones'). Names are cached for 10
|
||||
minutes and never stored. `names.enabled`, `names.resolver` and
|
||||
`names.cache` in housecarl.yaml; `hc --no-names`; `POST /names`.
|
||||
- **Quotas on the dashboard.** With any rule holding a quota, the
|
||||
dashboard lists them all: the allowance, what is used (for a
|
||||
per-address quota, how many addresses and the busiest) with a bar, and
|
||||
the next reset. Each opens its rule.
|
||||
- **Links to a rule open it.** A rule named in the matrix, the log, the
|
||||
tracer, statistics or the dashboard's busiest rules opens the Rules
|
||||
page on its chain, scrolled to it and highlighted (`#/rules?rule=ID`).
|
||||
The Rules page shows each rule's ID (`#410`, as `hc` and labels name
|
||||
it) beside its order.
|
||||
- **A clearer zone matrix.** A cell reads allow, block, or **partly**
|
||||
(some new connections let through, the rest not), with the exceptions
|
||||
counted by what they do (`+3 -1` instead of `+4`), and what they let
|
||||
through: services with the addresses they're limited to (`ssh from
|
||||
trusted`, `5 port forwards`), under the grid in `hc matrix`, in each
|
||||
cell on the Matrix page. A zone to itself is marked (only traffic routed
|
||||
between its interfaces passes the firewall). The details list each
|
||||
rule's hits. API: `summary`, `allowed`, `blocked`, `open`,
|
||||
`same_zone`, `note`, `hits`, and `short` and `hits` per exception.
|
||||
Raw rules that are a match and then accept, drop or reject (without
|
||||
interfaces) are read instead of shown as unknown.
|
||||
- **Shadow mode** (`shadow.enabled` in housecarl.yaml; docs/shadow.md):
|
||||
Housecarl's ruleset is loaded beside the firewall you have but blocks,
|
||||
rejects and translates nothing. Drops and rejects are counted and accept
|
||||
instead (an accept only ends Housecarl's chain), policies accept and
|
||||
always log what they would drop, NAT only counts, MSS clamping and the
|
||||
flowtable are left out, and raw rules that do more than accept, drop or
|
||||
reject are left out with a warning (`shadow_raw_left_out`). Comments
|
||||
mark what would happen (`[would drop]`). `shadow.position`: Housecarl's
|
||||
filter chains run after the other firewall's (`filter + 10`, the
|
||||
default: what it would drop is what switching would break) or before
|
||||
(`- 10`); its NAT chains always first. Every change is checked by
|
||||
nftables in both forms, so leaving shadow mode can't fail. No
|
||||
probation; takeovers and closing connections are refused (409). The
|
||||
boot ruleset is the shadow one, and its fallback filters nothing. When
|
||||
a reload (`flush ruleset`) deletes Housecarl's table, housecarld puts it
|
||||
back (it listens to nftables' netlink notifications). `mode` in
|
||||
`/status`, `/healthz` and `/auth/whoami`; a banner in the web UI;
|
||||
would-be drops marked in the log (`shadow`) and the tracer
|
||||
(`would_drop`); `hc ruleset --enforcing` (`GET /ruleset?enforcing=1`)
|
||||
shows the ruleset for after. `shadow_unprotected` warns when no other
|
||||
table filters the input hook; `nftables_service` and `takeover_pending`
|
||||
aren't raised in shadow mode. The e2e test imports in shadow mode
|
||||
before taking over.
|
||||
- **Zone names at import:** `hc import --zone enp2s0=lan` (repeat;
|
||||
interfaces given one name share a zone; Zone names on the Import
|
||||
page), and a note listing each zone and where its name came from.
|
||||
Defines holding a VLAN sub-interface (`enp2s0.20`) name their zone,
|
||||
and `_ifname`, `_device` and `_nic` are taken off define names too
|
||||
(docs/import.md, "How things are named").
|
||||
- Import: sets with `flags timeout` become dynamic sets; a DNAT of
|
||||
several ports becomes one port forward per port.
|
||||
|
||||
### Changed
|
||||
|
||||
- The ruleset's diff (Changes, `hc changeset diff`, `--dry-run`) names
|
||||
the chain each part is in on its `@@` line (`@@ -40,6 +40,8 @@ chain
|
||||
forward`), as `git diff` names the function, so a rule added deep in a
|
||||
long chain says whether it is in input, forward or output.
|
||||
- **History shows the ruleset's diff.** An entry's page (and `hc history
|
||||
diff ID`, `GET /history/{id}/diff`) shows what the change did to the
|
||||
ruleset, for entries made before this version too: the configuration
|
||||
before and after it is got back from the current one and rendered by
|
||||
this version.
|
||||
- Negating a match is said first: the zone, From and To fields (rules
|
||||
and NAT) start with "any of" / "any but" instead of a checkbox under
|
||||
the list, shown once the list has an entry, and an "any but" list is
|
||||
marked with a red bar. An empty list is never saved as negated (it
|
||||
matched everything anyway, but would have negated what was added to
|
||||
it later).
|
||||
- The rule dialog shows only the zone fields its chain can use: no
|
||||
"Going out on" for input and early, no "Coming in on" for output
|
||||
(they could only be refused). A zone left in a field the chain hides
|
||||
isn't sent.
|
||||
- History lists a changed setting by its own fields
|
||||
(`builtins.drop_invalid: false → true`), on the web page and in `hc
|
||||
history show`, instead of the whole `builtins` before and after.
|
||||
- Set names starting with `ban_` are now reserved (`set_name_reserved`):
|
||||
ban lists' sets use them.
|
||||
- **Rules read as a direction.** A match's zones are written `lan → wan`,
|
||||
`any but wan → any`, or with the firewall itself as the other end
|
||||
(`lan → firewall` in input, `firewall → wan` in output), instead of
|
||||
`in lan · out wan` and `in not wan`; exclusions read `from any but X`.
|
||||
On the Rules and NAT pages and in `hc`; the flags and JSON stay as
|
||||
they were.
|
||||
|
||||
### Fixed
|
||||
|
||||
- The dashboard's bar for a per-address quota showed only the busiest
|
||||
address, so one address that used up its allowance filled it (100%).
|
||||
The bar is now all the addresses together, and the dashboard and the
|
||||
rule's quota line say how many addresses used theirs up.
|
||||
- Changing how a quota counts lost or muddled what it used: per address
|
||||
to a total started at nothing, per source to per destination kept the
|
||||
sources' use as if it were the destinations', and other groups left
|
||||
the old entries behind. Now a total starts at the addresses' sum,
|
||||
wider groups add up their addresses, and what can't be carried starts
|
||||
again with a warning (`quota_restarts`) on the change.
|
||||
- What the quotas used is saved after every change, not only every five
|
||||
minutes: a reboot soon after a change that counts a quota differently
|
||||
could bring back its usage counted the old way.
|
||||
- Statistics' Quotas card linked every entry to the Rules page, not to
|
||||
its rule; each now opens its rule, named. Per-address quotas were
|
||||
missing from it (and from the `quota:rN` series): they count their
|
||||
addresses' sum.
|
||||
- `hc stats top devices` listed each device twice (once for what it
|
||||
received, once for what it sent); it is one row with both again.
|
||||
- `hc rule add --before/--after/--first` made two changes (create at the
|
||||
end, then move), so two history entries, two probations, and the rule
|
||||
briefly in force at the end of its chain; staged in a changeset, the
|
||||
place was lost. The create now takes the place (`POST
|
||||
/rules?before=ID`, `after`, `first`) and makes one change.
|
||||
- `hc matrix` in a terminal: the colours' escape codes threw the columns
|
||||
out of line.
|
||||
- `hc counters` (`GET /counters`) kept listing the objects of a
|
||||
configuration replaced since housecarld started (`hc import
|
||||
--replace` gives every object a new ID), beside the current ones; the
|
||||
totals of objects that no longer exist are now forgotten.
|
||||
- Import: two base chains on one hook had their rules imported twice,
|
||||
and the second chain's leading built-ins were moved above the first
|
||||
chain's rules; they now follow each other by priority (read in a
|
||||
fixed order), with the stricter policy, and a note.
|
||||
- Import: a rule inside a jumped-to chain that stays raw kept only its
|
||||
own text and so applied to everything; the jump's condition now goes
|
||||
in front. A jump and a rule matching the same field are combined only
|
||||
when one's addresses lie within the other's; else the rule stays raw.
|
||||
A jump's ports, ICMP types, connection status and hop limit were
|
||||
dropped; a limit or log on the jump is no longer silently lost.
|
||||
- Import: a second match on the same field (`ip saddr A ip saddr != B`)
|
||||
replaced the first; `!=` on the hop limit and TCP flags was read as
|
||||
`==`. Such rules now stay raw, as written.
|
||||
- Import: a forward-chain accept to a forwarded host and port without
|
||||
`ct status dnat` was folded into the port forward, which only lets
|
||||
forwarded connections through; it stays a rule now. An IPv6 accept
|
||||
limited to an interface no longer becomes a pinhole open to all.
|
||||
- Import: raw rules naming a set of the old table made nftables refuse
|
||||
the ruleset; address sets are now renamed `NAME_v4`/`NAME_v6`, other
|
||||
sets written out with their elements, and what can't be is left out
|
||||
with a note. Rules naming a set that isn't imported stay raw instead
|
||||
of failing validation.
|
||||
- Import: a DNAT to an IPv6 address, or with matches a port forward
|
||||
can't hold (an interface excluded, a limit, a log), is left out with a
|
||||
note instead of being imported without them.
|
||||
|
||||
## [0.1.0] - 2026-10-19
|
||||
|
||||
The first version of Housecarl, a firewall manager for nftables, built
|
||||
the way Jarl and Steward are.
|
||||
|
||||
@@ -30,19 +221,100 @@ the way Jarl and Steward are.
|
||||
invalid packets, ICMPv6 (RFC 4890), ping from internal zones, DHCPv6
|
||||
client replies, antispoofing with your own prefixes, MSS clamping, and
|
||||
a flowtable fast path.
|
||||
- **Changesets, as in Jarl:** changes staged to be reviewed and applied
|
||||
together, as one history entry. Each is checked as it is staged; the
|
||||
review shows the problems, nftables' check and the diff of the ruleset.
|
||||
The web UI collects edits as pending changes by default (pending rows,
|
||||
a bar with Review and Apply now, a Changes page), with a switch for live
|
||||
edits, and Undo in History is staged like any edit; hc stages with
|
||||
`-C ID` or `HOUSECARL_CHANGESET`, and has `hc
|
||||
changeset create/list/show/diff/apply/discard/remove`. An applied
|
||||
changeset is on probation like any change. `--dry-run` shows the
|
||||
ruleset's diff too (docs/design.md, docs/cli.md).
|
||||
- **Bandwidth limits and quotas:** rate limits in bytes (`--limit
|
||||
20mbytes/second,over,per-source`), per source or per destination;
|
||||
quotas, an allowance of traffic per rule or per address, reset daily,
|
||||
weekly, monthly on a chosen day, or by hand, kept across changes,
|
||||
restarts and reboots (`hc quotas`, `hc rule reset-quota`, usage on the
|
||||
rules page). Rules that count traffic are checked before established
|
||||
connections are accepted, so they see every packet. Per-address limits
|
||||
group IPv6 by /64 by default (`--group-v4`, `--group-v6`). Meters,
|
||||
byte rates and quotas in an imported ruleset carry over
|
||||
(docs/rules.md).
|
||||
- **Changes checked twice and applied atomically:** validation of the
|
||||
whole configuration with stable problem codes (docs/validation.md),
|
||||
then `nft -c`, then one `nft -f` transaction that replaces Housecarl's
|
||||
own table, `inet housecarl`, and nothing else. Dynamic sets' entries
|
||||
and the counters carry across.
|
||||
- **Probation:** a change made over the network rolls itself back unless
|
||||
confirmed within two minutes (`hc confirm`, a banner in the web UI);
|
||||
confirmed within two minutes. The web UI and `hc` confirm their own
|
||||
changes once they reach Housecarl again over a new connection, from the
|
||||
same address (a change that cuts its client off can't, and rolls back);
|
||||
`--no-confirm` and the "a person confirms each change" setting leave it
|
||||
to `hc confirm` or the banner;
|
||||
changes through the local socket are kept at once; a restart rolls
|
||||
back what was pending. No rule is forced to keep SSH or the web UI
|
||||
open.
|
||||
- **At boot:** the confirmed ruleset is loaded by
|
||||
`housecarl-ruleset.service` before the network comes up, without
|
||||
housecarld. An empty configuration applies nothing.
|
||||
housecarld. An empty configuration applies nothing. It fails closed:
|
||||
if the saved ruleset doesn't load, a fallback saved beside it
|
||||
(`ruleset-fallback.nft`) is, until housecarld puts the real one back:
|
||||
nothing new gets in from external zones, the inside keeps working
|
||||
(masqueraded), and the unit is marked failed with a critical message.
|
||||
`hc status`, the dashboard and `/healthz` say while the fallback is in
|
||||
force; the warning `no_external_zone` says when no zone is external,
|
||||
which would make the fallback close everything (docs/operations.md).
|
||||
- **The flowtable and interfaces that come and go:** the boot ruleset
|
||||
has no flowtable (nftables on Debian 13's kernel refuses one naming a
|
||||
device that doesn't exist yet, such as `ppp0` before PPPoE is up, and
|
||||
with it the whole file); housecarld watches the kernel's link messages
|
||||
and applies the ruleset again when a flowtable device appears or comes
|
||||
back as a new interface (PPPoE reconnecting). A guide to when the
|
||||
flowtable helps, and what it costs (docs/flowtable.md).
|
||||
- **`GET /healthz` for monitoring:** `ok`, `firewall` (applied,
|
||||
unconfigured, fallback, missing), changes on probation and the
|
||||
machine's warnings, with HTTP 503 unless the firewall is in force;
|
||||
no login needed, read at most every five seconds.
|
||||
- **Import keeps a chain's head in order:** once a would-be built-in at
|
||||
the head of a chain has to stay a rule (an invalid drop the forward
|
||||
chain needs after an exception), the built-ins after it stay rules
|
||||
too, in their place, instead of moving ahead of it: an ICMPv6 accept
|
||||
no longer jumps ahead of the invalid drop it came after.
|
||||
- **Renaming a zone, set or service** changes the rules, NAT rules and
|
||||
port forwards that name it, in the same change (one history entry, one
|
||||
undo); before, a name in use couldn't change. Dynamic sets keep their
|
||||
names (their tokens and programs go by them).
|
||||
- VLAN sub-interfaces (`enp2s0.20`) in zones and the flowtable, tested
|
||||
with tagged traffic (docs/rules.md).
|
||||
- `hc settings set --flowtable none` (and `none` for the other lists)
|
||||
empties the list.
|
||||
- **Warnings about the machine:** while `nftables.service` is enabled
|
||||
(it loads `/etc/nftables.conf` at boot, and flushes every table when
|
||||
stopped), and while the table an import came from still applies.
|
||||
- **Statistics** (Traffic → Statistics, `hc stats`, `/api/v1/stats/...`):
|
||||
traffic per interface and zone, drops by chain and when they happen,
|
||||
hits per rule and the rules that matched nothing, the sources blocked
|
||||
most and what they tried, connections against the kernel's limit, each
|
||||
quota's use, and housecarld's and the host's memory and load, sampled
|
||||
every minute into a database of their own (`stats.db`, kept 7 days per
|
||||
minute, 90 days per hour, 2 years per day). On by default; traffic per
|
||||
inside device is a setting (`--stats-per-device`) that adds counters
|
||||
to the forward chain. Two charts on the dashboard; a warning when the
|
||||
connection tracking table is nearly full (docs/stats.md).
|
||||
- Set names starting with `meter_`, `conn_`, `quota_` or `stats_` are
|
||||
refused (`set_name_reserved`): Housecarl's own sets use them, and a set
|
||||
of yours named so could clash with them.
|
||||
- **The zone matrix** (Matrix, `hc matrix`): what the rules do to new
|
||||
connections between zones and to and from the firewall, worked out
|
||||
from the rules in order, with each pair's exceptions; rules about
|
||||
networks connected to other zones are left out of a pair.
|
||||
- **Connections and tracing:** the connections the kernel tracks, with
|
||||
their NAT, and closing them so the rules check them again (Traffic →
|
||||
Connections, `hc connections`); a rule option, "every packet", for
|
||||
blocks that stop connections already open; and a packet tracer
|
||||
(Traffic → Trace, `hc trace`) that follows packets through every
|
||||
table's rules and names what accepted or dropped them.
|
||||
- **Hits and the log:** packet and byte counters for every rule, NAT rule,
|
||||
built-in and policy; packets logged by rules and policies, read from
|
||||
netlink log group 4171 (`hc log -f`, the Log page).
|
||||
@@ -52,10 +324,14 @@ the way Jarl and Steward are.
|
||||
- **Import** of an existing nftables ruleset (`hc import`): rules keep
|
||||
their order, defines name the sets, DNAT becomes port forwards, what
|
||||
can't be translated stays as raw rules, with notes; `hc takeover`
|
||||
deletes the old table, on probation (docs/import.md).
|
||||
deletes the old table, on probation (docs/import.md). DNAT maps and
|
||||
verdict maps are expanded into one port forward or rule per element.
|
||||
- **The `hc` CLI, the REST API and the web UI** (dashboard, rules per
|
||||
chain with hits and reordering, NAT, zones, sets, services, the log,
|
||||
settings, import, history with undo), with users, roles, scoped API
|
||||
settings, import, history with undo; the rule dialog keeps the less
|
||||
used parts, such as rate limits, quotas and time windows, behind "+"
|
||||
buttons; each chain's built-ins are listed, with their hits, and
|
||||
switched in a window), with users, roles, scoped API
|
||||
tokens, sessions, the audit log, and the web UI's certificate from
|
||||
Let's Encrypt through Jarl, as in Steward.
|
||||
- **Packaging:** Debian packages `housecarl` and `housecarl-cli`, a
|
||||
|
||||
@@ -112,6 +112,7 @@ install-files:
|
||||
install -D -m 0644 CHANGELOG.md $(DESTDIR)$(DOCDIR)/CHANGELOG.md
|
||||
install -d $(DESTDIR)$(DOCDIR)/docs
|
||||
install -m 0644 docs/*.md $(DESTDIR)$(DOCDIR)/docs/
|
||||
install -D -m 0644 contrib/fail2ban/action.d/housecarl.conf $(DESTDIR)$(DOCDIR)/contrib/fail2ban/action.d/housecarl.conf
|
||||
|
||||
uninstall:
|
||||
rm -f $(DESTDIR)$(BINDIR)/housecarld $(DESTDIR)$(BINDIR)/hc $(DESTDIR)$(UNITDIR)/housecarl.service \
|
||||
|
||||
@@ -21,11 +21,14 @@ table of its own.
|
||||
|
||||
* **Zones, sets and services, then rules.** Name your interfaces (`wan`,
|
||||
`lan`, `wg`), your addresses (`trusted`, IPv4 and IPv6 in one set) and
|
||||
your ports (`jellyfin`, or the built-in `ssh`, `dns`, `https`…), and
|
||||
your ports (`web`, or the built-in `ssh`, `dns`, `https`…), and
|
||||
write ordered rules with them: in, out or through the firewall, by
|
||||
zone, address, service, connection state, time of day, rate per
|
||||
source, or connection count. Anything else can be written as an
|
||||
nftables rule by hand.
|
||||
* **Bandwidth and quotas.** Byte-rate limits per device, and traffic
|
||||
allowances that reset daily, weekly or monthly (or by hand), counted
|
||||
across changes and reboots.
|
||||
* **IPv4 and IPv6 together.** One rule covers both; Housecarl writes the
|
||||
nftables rules each family needs.
|
||||
* **NAT in one object.** A port forward is the DNAT, the forward accept
|
||||
@@ -34,20 +37,36 @@ table of its own.
|
||||
* **The rules every firewall needs**, built in and each one switchable:
|
||||
loopback, established connections, invalid packets, the ICMPv6 that
|
||||
IPv6 needs, antispoofing for your prefixes, MSS clamping for PPPoE, the
|
||||
DHCPv6 client, and a flowtable fast path.
|
||||
DHCPv6 client, and a flowtable fast path
|
||||
([docs/flowtable.md](docs/flowtable.md)).
|
||||
* **Changes you can trust.** Each change is validated as a whole, checked
|
||||
by nftables itself, and applied in one transaction. A change made over
|
||||
by nftables itself, and applied in one transaction. As in Jarl, edits
|
||||
can be collected in a changeset, reviewed with the diff of the ruleset,
|
||||
and applied (or discarded) together. A change made over
|
||||
the network is on probation: unless you confirm it within two minutes,
|
||||
it rolls itself back, so a rule that cuts you off undoes itself.
|
||||
* **Your existing ruleset, imported.** `hc import` turns an nftables
|
||||
ruleset into zones, sets, services, rules and port forwards, keeping the
|
||||
rules' order, and keeps what it can't translate as raw rules
|
||||
([docs/import.md](docs/import.md)).
|
||||
* **Easy to watch.** Hits per rule, a live log of what was dropped and
|
||||
why, and the history of every change, field by field, with undo.
|
||||
* **Try it first, in shadow mode.** Housecarl's ruleset runs beside the
|
||||
firewall you have, blocking nothing: its hits, log and tracer show what
|
||||
it would drop, so you see what switching over would break before you do
|
||||
([docs/shadow.md](docs/shadow.md)).
|
||||
* **Ban lists for Guard and fail2ban.** A program on any machine bans an
|
||||
address it caught misbehaving in its logs, and the ban list says what
|
||||
that address is kept from: the firewall, some hosts, or everything
|
||||
([docs/bans.md](docs/bans.md)).
|
||||
* **Easy to watch.** A zone matrix of what is allowed between zones,
|
||||
statistics over time (traffic per zone and device, drops, the sources
|
||||
blocked most, connections; [docs/stats.md](docs/stats.md)), hits per rule, a live log of what was dropped and
|
||||
why, the connections the firewall tracks (and closing them), a packet
|
||||
tracer that names the rule that dropped something, in any table, and
|
||||
the history of every change, field by field, with undo.
|
||||
* **Its own table only.** Docker, Incus and libvirt keep theirs. The
|
||||
confirmed ruleset is loaded at boot, before the network comes up, even
|
||||
if housecarld doesn't start.
|
||||
if housecarld doesn't start; should it fail to load, a fallback keeps
|
||||
anything new from the internet out until housecarld puts it back.
|
||||
* **Sets other programs can fill.** A dynamic set's entries expire by
|
||||
themselves, and a token can be limited to one set: that is how
|
||||
Steward's captive portal will let a guest in.
|
||||
@@ -75,10 +94,10 @@ sudo hc takeover "inet main"
|
||||
sudo systemctl disable nftables.service
|
||||
|
||||
# ...or start from scratch: let yourself in first
|
||||
sudo hc set add admins 10.0.10.0/24
|
||||
sudo hc set add admins 192.168.1.0/24
|
||||
sudo hc rule add input --from admins --service ssh
|
||||
sudo hc zone add wan --interface ppp0 --external
|
||||
sudo hc zone add lan --interface enp2s0
|
||||
sudo hc zone add lan --interface eth1
|
||||
sudo hc rule add forward --in lan --out wan
|
||||
sudo hc snat add --out wan
|
||||
sudo hc status
|
||||
@@ -92,8 +111,11 @@ The web UI is on `http://127.0.0.1:8070/` (see `listen` in
|
||||
* [install.md](docs/install.md): packages, building from source, first steps
|
||||
* [rules.md](docs/rules.md): zones, address sets, services, rules and the built-ins
|
||||
* [nat.md](docs/nat.md): port forwards, hairpin NAT, IPv6 pinholes, source NAT, redirects
|
||||
* [bans.md](docs/bans.md): ban lists, where Guard and fail2ban ban addresses
|
||||
* [import.md](docs/import.md): moving from an existing nftables ruleset
|
||||
* [operations.md](docs/operations.md): probation, the boot, hits and the log, backups, troubleshooting
|
||||
* [shadow.md](docs/shadow.md): shadow mode, Housecarl beside your firewall, changing nothing
|
||||
* [operations.md](docs/operations.md): probation, the boot and its fallback, hits and the log, monitoring, backups, troubleshooting
|
||||
* [flowtable.md](docs/flowtable.md): the fast path for forwarded traffic, and when (not) to use it
|
||||
* [certificates.md](docs/certificates.md): HTTPS with a certificate from Let's Encrypt, through Jarl
|
||||
* [jarl.md](docs/jarl.md): connecting Housecarl to Jarl, and what its token may do
|
||||
* [cli.md](docs/cli.md): the `hc` command line client
|
||||
@@ -106,11 +128,14 @@ The web UI is on `http://127.0.0.1:8070/` (see `listen` in
|
||||
|
||||
## Status
|
||||
|
||||
Housecarl 0.1.0 is in testing (0.1.0-rc.1): zones, address sets and
|
||||
Housecarl 0.2.0 is in testing (0.2.0-rc.3+5ec943e): zones, address sets and
|
||||
services with ordered rules, IPv4 and IPv6, port forwards with hairpin NAT
|
||||
and IPv6 pinholes, source NAT and redirects, built-ins, probation for
|
||||
network changes, the ruleset at boot, hits and the drop log, nftables
|
||||
import, a web UI, REST API and CLI.
|
||||
network changes (confirmed by the client that made them), the ruleset
|
||||
at boot with a fail-closed fallback, hits and the drop log, nftables
|
||||
import, changesets reviewed with the ruleset's diff (as in Jarl),
|
||||
bandwidth limits and quotas, shadow mode beside another firewall, ban
|
||||
lists for Guard and fail2ban, a web UI, REST API and CLI.
|
||||
|
||||
## Licence
|
||||
|
||||
|
||||
+40
-1
@@ -66,6 +66,30 @@ func historyCmd() *cobra.Command {
|
||||
})
|
||||
},
|
||||
})
|
||||
cmd.AddCommand(&cobra.Command{
|
||||
Use: "diff ID", Short: "Show what a change did to the ruleset", Args: cobra.ExactArgs(1),
|
||||
Long: `Shows the ruleset's diff for a change: the configuration before and after
|
||||
it, rendered by this version of Housecarl.`,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
c, err := newClient()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var d struct {
|
||||
Diff string `json:"diff"`
|
||||
}
|
||||
if err := c.Get(ctx(), "/history/"+args[0]+"/diff", nil, &d); err != nil {
|
||||
return err
|
||||
}
|
||||
return output(d, func(w io.Writer) {
|
||||
if d.Diff == "" {
|
||||
fmt.Fprintln(w, "the ruleset didn't change")
|
||||
return
|
||||
}
|
||||
printDiff(w, d.Diff)
|
||||
})
|
||||
},
|
||||
})
|
||||
cmd.AddCommand(&cobra.Command{
|
||||
Use: "undo ID",
|
||||
Short: "Undo a change",
|
||||
@@ -88,11 +112,13 @@ change touched has changed since; undo the later changes first.`,
|
||||
}
|
||||
|
||||
// diffJSON lists the fields that differ between two versions of an
|
||||
// object: "name: office -> lab".
|
||||
// object: "name: office -> lab". An object within (settings' builtins)
|
||||
// is compared field by field: "builtins.drop_invalid: false -> true".
|
||||
func diffJSON(before, after json.RawMessage) []string {
|
||||
var a, b map[string]any
|
||||
json.Unmarshal(before, &a)
|
||||
json.Unmarshal(after, &b)
|
||||
a, b = flatten(a, "", map[string]any{}), flatten(b, "", map[string]any{})
|
||||
keys := map[string]bool{}
|
||||
for k := range a {
|
||||
keys[k] = true
|
||||
@@ -132,6 +158,19 @@ func diffJSON(before, after json.RawMessage) []string {
|
||||
return out
|
||||
}
|
||||
|
||||
// flatten lifts the fields of objects within m to dotted keys; lists
|
||||
// stay whole.
|
||||
func flatten(m map[string]any, prefix string, out map[string]any) map[string]any {
|
||||
for k, v := range m {
|
||||
if o, ok := v.(map[string]any); ok && len(o) > 0 {
|
||||
flatten(o, prefix+k+".", out)
|
||||
} else {
|
||||
out[prefix+k] = v
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func whoamiCmd() *cobra.Command {
|
||||
return &cobra.Command{
|
||||
Use: "whoami", Short: "Show who you are to the server", Args: cobra.NoArgs,
|
||||
|
||||
+313
@@ -0,0 +1,313 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io"
|
||||
"net/url"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"github.com/spf13/cobra"
|
||||
|
||||
"housecarl/internal/api"
|
||||
"housecarl/internal/model"
|
||||
)
|
||||
|
||||
// ---- ban lists (docs/bans.md)
|
||||
|
||||
// banTime writes a ban list's time setting: 0 is the default, -1 none.
|
||||
func banTime(n int, def string) string {
|
||||
switch {
|
||||
case n < 0:
|
||||
return "no limit"
|
||||
case n == 0:
|
||||
return def + " (default)"
|
||||
}
|
||||
return fmtSeconds(n)
|
||||
}
|
||||
|
||||
// parseBanTime reads a time flag: a duration, or "none" (-1) for no
|
||||
// limit.
|
||||
func parseBanTime(s string) (int, error) {
|
||||
if strings.EqualFold(s, "none") || s == "-1" {
|
||||
return -1, nil
|
||||
}
|
||||
return parseSeconds(s)
|
||||
}
|
||||
|
||||
func banlistCmd() *cobra.Command {
|
||||
cmd := &cobra.Command{Use: "banlist", Aliases: []string{"banlists"}, Short: "Ban lists: where Guard and fail2ban ban addresses"}
|
||||
cmd.AddCommand(&cobra.Command{
|
||||
Use: "list", Short: "List the ban lists", Args: cobra.NoArgs,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
bs, err := get[[]model.BanList]("/banlists", nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
bans, _ := get[[]model.Ban]("/bans", nil)
|
||||
count := map[int64]int{}
|
||||
for _, b := range bans {
|
||||
count[b.ListID]++
|
||||
}
|
||||
return output(bs, func(w io.Writer) {
|
||||
fmt.Fprintln(w, "NAME\tSCOPE\tHOSTS\tBANS\tCOMMENT")
|
||||
for _, b := range bs {
|
||||
scope := b.Scope
|
||||
if b.Disabled {
|
||||
scope += " (disabled)"
|
||||
}
|
||||
fmt.Fprintf(w, "%s\t%s\t%s\t%d\t%s\n", b.Name, scope, list(b.Hosts), count[b.ID], b.Comment)
|
||||
}
|
||||
})
|
||||
},
|
||||
})
|
||||
cmd.AddCommand(showCmd("banlists", "banlist", func(w io.Writer, b model.BanList) {
|
||||
fmt.Fprintf(w, "Name:\t%s\nScope:\t%s\n", b.Name, b.Scope)
|
||||
if len(b.Hosts) > 0 {
|
||||
fmt.Fprintf(w, "Hosts:\t%s\n", list(b.Hosts))
|
||||
}
|
||||
fmt.Fprintf(w, "Never:\t%s\n", list(b.Never))
|
||||
fmt.Fprintf(w, "Ban time:\t%s\nLongest:\t%s\n", banTime(b.DefaultTime, "1h"), banTime(b.MaxTime, "7d"))
|
||||
fmt.Fprintf(w, "Most bans:\t%d\nIPv6 banned by:\t/%d\n", b.Entries(), b.V6Group())
|
||||
if b.Disabled {
|
||||
fmt.Fprintln(w, "Disabled:\tyes")
|
||||
}
|
||||
if b.Comment != "" {
|
||||
fmt.Fprintf(w, "Comment:\t%s\n", b.Comment)
|
||||
}
|
||||
}))
|
||||
var scope, comment, banT, maxT string
|
||||
var hosts, never []string
|
||||
var entries, group int
|
||||
var disabled bool
|
||||
setFields := func(fl interface{ Changed(string) bool }, p patch) error {
|
||||
if fl.Changed("scope") {
|
||||
p["scope"] = scope
|
||||
}
|
||||
if fl.Changed("host") {
|
||||
p["hosts"] = hosts
|
||||
}
|
||||
if fl.Changed("never") {
|
||||
p["never"] = never
|
||||
}
|
||||
for _, t := range []struct{ flag, field, v string }{{"time", "default_time", banT}, {"max-time", "max_time", maxT}} {
|
||||
if fl.Changed(t.flag) {
|
||||
n, err := parseBanTime(t.v)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
p[t.field] = n
|
||||
}
|
||||
}
|
||||
if fl.Changed("max-entries") {
|
||||
p["max_entries"] = entries
|
||||
}
|
||||
if fl.Changed("group-v6") {
|
||||
p["group_v6"] = group
|
||||
}
|
||||
if fl.Changed("comment") {
|
||||
p["comment"] = comment
|
||||
}
|
||||
if fl.Changed("disabled") {
|
||||
p["disabled"] = disabled
|
||||
}
|
||||
return nil
|
||||
}
|
||||
flags := func(c *cobra.Command) {
|
||||
f := c.Flags()
|
||||
f.StringVar(&scope, "scope", "", "what bans keep addresses from: firewall, hosts or everything")
|
||||
f.StringArrayVar(&hosts, "host", nil, "scope hosts: a machine's address, or a set (repeat)")
|
||||
f.StringArrayVar(&never, "never", nil, "an address, prefix or set never banned (repeat)")
|
||||
f.StringVar(&banT, "time", "", "a ban's length when none is asked for (default 1h)")
|
||||
f.StringVar(&maxT, "max-time", "", "the longest ban (default 7d; none for no limit)")
|
||||
f.IntVar(&entries, "max-entries", 0, "the most bans the list holds (default 10000)")
|
||||
f.IntVar(&group, "group-v6", 0, "IPv6 prefix length banned for an address (default 64)")
|
||||
f.StringVar(&comment, "comment", "", "a comment")
|
||||
f.BoolVar(&disabled, "disabled", false, "the list's bans aren't in force")
|
||||
}
|
||||
add := &cobra.Command{
|
||||
Use: "add NAME", Short: "Add a ban list", Args: cobra.ExactArgs(1),
|
||||
Long: `Adds a ban list. A program bans an address into it (Guard, fail2ban's
|
||||
housecarl action, hc ban), and the list says what that address is kept
|
||||
from:
|
||||
|
||||
hc banlist add router --scope firewall # the firewall itself
|
||||
hc banlist add web --scope hosts --host 10.0.10.57 --host 2001:db8:10::57
|
||||
hc banlist add edge --scope everything --never trusted
|
||||
|
||||
Give the program a token for its list alone:
|
||||
|
||||
hc user add guard-web --no-password --grant bans:write:web
|
||||
hc token create web1 --user guard-web # as root on the firewall`,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
p := patch{"name": args[0]}
|
||||
if err := setFields(cmd.Flags(), p); err != nil {
|
||||
return err
|
||||
}
|
||||
return change("POST", "/banlists", p)
|
||||
},
|
||||
}
|
||||
flags(add)
|
||||
cmd.AddCommand(add)
|
||||
update := &cobra.Command{
|
||||
Use: "update BANLIST", Short: "Change a ban list", Args: cobra.ExactArgs(1),
|
||||
Long: `Changes a ban list; --host and --never replace the lists. A ban list
|
||||
can't be renamed: tokens and programs know it by its name.`,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
p := patch{}
|
||||
if err := setFields(cmd.Flags(), p); err != nil {
|
||||
return err
|
||||
}
|
||||
return change("PATCH", objPath("banlists", args[0]), p)
|
||||
},
|
||||
}
|
||||
flags(update)
|
||||
cmd.AddCommand(update)
|
||||
cmd.AddCommand(deleteCmd("banlists", "banlist"))
|
||||
return cmd
|
||||
}
|
||||
|
||||
// bansCmd lists the active bans.
|
||||
func bansCmd() *cobra.Command {
|
||||
return &cobra.Command{
|
||||
Use: "bans [BANLIST]", Short: "List the active bans", Args: cobra.MaximumNArgs(1),
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
path := "/bans"
|
||||
if len(args) == 1 {
|
||||
path = objPath("banlists", args[0]) + "/bans"
|
||||
}
|
||||
bans, err := get[[]model.Ban](path, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return output(bans, func(w io.Writer) {
|
||||
fmt.Fprintln(w, "ID\tLIST\tADDRESS\tPORT\tENDS IN\tBY\tSOURCE\tREASON")
|
||||
for _, b := range bans {
|
||||
port := "all"
|
||||
if b.Protocol != "" {
|
||||
port = fmt.Sprintf("%s/%d", b.Protocol, b.Port)
|
||||
}
|
||||
fmt.Fprintf(w, "%d\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n", b.ID, b.List, b.Address, port, fmtSeconds(b.Timeout), b.Who, orDash(b.Source), orDash(clipText(b.Reason, 60)))
|
||||
}
|
||||
})
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func clipText(s string, n int) string {
|
||||
if len(s) > n {
|
||||
return s[:n-1] + "…"
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
// banCmd bans an address by hand.
|
||||
func banCmd() *cobra.Command {
|
||||
var tcp, udp []string
|
||||
var t, reason string
|
||||
cmd := &cobra.Command{
|
||||
Use: "ban BANLIST ADDRESS", Short: "Ban an address", Args: cobra.ExactArgs(2),
|
||||
Long: `Bans an address in a ban list: every port, or with --tcp and --udp some
|
||||
(numbers or service names). IPv6 addresses are banned with the list's
|
||||
grouping (a /64 unless set otherwise).
|
||||
|
||||
hc ban edge 203.0.113.9 --time 1d
|
||||
hc ban web 203.0.113.9 --tcp ssh --tcp 8443 --reason "tried passwords"`,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
in := api.BanInput{Address: args[1], Reason: reason, Source: "hc"}
|
||||
if t != "" {
|
||||
n, err := parseBanTime(t)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
in.Time = n
|
||||
}
|
||||
c, err := newClient()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var out api.BanResult
|
||||
do := func(proto string, ports []string) error {
|
||||
in.Protocol = proto
|
||||
in.Ports = nil
|
||||
if len(ports) > 0 {
|
||||
in.Ports = []byte(strconv.Quote(strings.Join(ports, ",")))
|
||||
}
|
||||
var res api.BanResult
|
||||
if err := c.Do(ctx(), "POST", objPath("banlists", args[0])+"/bans", nil, in, &res); err != nil {
|
||||
return err
|
||||
}
|
||||
out.Bans = append(out.Bans, res.Bans...)
|
||||
out.Closed += res.Closed
|
||||
return nil
|
||||
}
|
||||
switch {
|
||||
case len(tcp) == 0 && len(udp) == 0:
|
||||
err = do("", nil)
|
||||
default:
|
||||
if len(tcp) > 0 {
|
||||
err = do("tcp", tcp)
|
||||
}
|
||||
if err == nil && len(udp) > 0 {
|
||||
err = do("udp", udp)
|
||||
}
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return output(out, func(w io.Writer) {
|
||||
for _, b := range out.Bans {
|
||||
what := "banned"
|
||||
if b.Existing {
|
||||
what = "renewed"
|
||||
}
|
||||
port := "every port"
|
||||
if b.Protocol != "" {
|
||||
port = fmt.Sprintf("%s/%d", b.Protocol, b.Port)
|
||||
}
|
||||
ends := "never ends"
|
||||
if b.Timeout > 0 {
|
||||
ends = "ends in " + fmtSeconds(b.Timeout)
|
||||
}
|
||||
fmt.Fprintf(w, "%s %s, %s, in %s (%s)\n", what, b.Address, port, b.List, ends)
|
||||
}
|
||||
if out.Closed > 0 {
|
||||
fmt.Fprintf(w, "%d connections closed\n", out.Closed)
|
||||
}
|
||||
})
|
||||
},
|
||||
}
|
||||
f := cmd.Flags()
|
||||
f.StringArrayVar(&tcp, "tcp", nil, "ban only this TCP port or service (repeat)")
|
||||
f.StringArrayVar(&udp, "udp", nil, "ban only this UDP port or service (repeat)")
|
||||
f.StringVar(&t, "time", "", "how long (1h, 7d; default the list's; none for ever, if the list allows)")
|
||||
f.StringVar(&reason, "reason", "", "why, for the record")
|
||||
return cmd
|
||||
}
|
||||
|
||||
// unbanCmd lifts bans: by ID, or every ban of an address.
|
||||
func unbanCmd() *cobra.Command {
|
||||
return &cobra.Command{
|
||||
Use: "unban BANLIST ADDRESS|ID", Short: "Lift a ban, or all of an address's", Args: cobra.ExactArgs(2),
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
c, err := newClient()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
path, query := objPath("banlists", args[0])+"/bans", url.Values{}
|
||||
if _, err := strconv.ParseInt(args[1], 10, 64); err == nil {
|
||||
path += "/" + args[1]
|
||||
} else {
|
||||
query.Set("address", args[1])
|
||||
}
|
||||
var out map[string]int
|
||||
if err := c.Do(ctx(), "DELETE", path, query, nil, &out); err != nil {
|
||||
return err
|
||||
}
|
||||
if !flagQuiet {
|
||||
fmt.Printf("%d bans lifted\n", out["removed"])
|
||||
}
|
||||
return nil
|
||||
},
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,377 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"github.com/spf13/cobra"
|
||||
|
||||
"housecarl/internal/api"
|
||||
"housecarl/internal/client"
|
||||
"housecarl/internal/model"
|
||||
)
|
||||
|
||||
// Changesets: configuration changes staged to be reviewed and applied
|
||||
// together. -C/--changeset stages into a changeset as given (housecarld
|
||||
// still refuses one that isn't open, or another user's unless you are an
|
||||
// administrator). One from HOUSECARL_CHANGESET is checked first: an
|
||||
// exported ID outlives its changeset (it gets applied), and IDs keep
|
||||
// growing, so a stale or mistyped one could name someone else's later. It
|
||||
// is used only when it is open and yours, and hc says on stderr which
|
||||
// changeset that is.
|
||||
|
||||
var flagChangeset int64
|
||||
|
||||
// envChangeset returns the changeset in HOUSECARL_CHANGESET (0 if unset).
|
||||
func envChangeset() (int64, error) {
|
||||
v := os.Getenv("HOUSECARL_CHANGESET")
|
||||
if v == "" {
|
||||
return 0, nil
|
||||
}
|
||||
n, err := strconv.ParseInt(v, 10, 64)
|
||||
if err != nil || n <= 0 {
|
||||
return 0, fmt.Errorf("invalid HOUSECARL_CHANGESET %q", v)
|
||||
}
|
||||
return n, nil
|
||||
}
|
||||
|
||||
const startHint = "export HOUSECARL_CHANGESET=$(hc changeset create -q)"
|
||||
|
||||
// checkEnvChangeset checks the changeset from HOUSECARL_CHANGESET before
|
||||
// it is used. With mustOwn, it must also be open and the caller's own
|
||||
// (for staging into it, applying or discarding it).
|
||||
func checkEnvChangeset(c *client.Client, id int64, mustOwn bool) (*model.Changeset, error) {
|
||||
var cs model.Changeset
|
||||
if err := c.Do(ctx(), "GET", fmt.Sprintf("/changesets/%d", id), nil, nil, &cs); err != nil {
|
||||
var ce *client.Error
|
||||
if errors.As(err, &ce) && ce.Status == 404 {
|
||||
return nil, fmt.Errorf("HOUSECARL_CHANGESET is %d, but there is no changeset %d: start one with '%s', or 'unset HOUSECARL_CHANGESET' to make changes at once", id, id, startHint)
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
if mustOwn {
|
||||
if cs.Status != model.ChangesetOpen {
|
||||
return nil, fmt.Errorf("HOUSECARL_CHANGESET is %d, but changeset %d is %s: 'unset HOUSECARL_CHANGESET' to make changes at once, or start a new one with '%s'", id, id, cs.Status, startHint)
|
||||
}
|
||||
var me api.Whoami
|
||||
if err := c.Do(ctx(), "GET", "/auth/whoami", nil, nil, &me); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
mine := me.User != nil && cs.UserID == me.User.ID && (me.User.ID != 0 || cs.Username == me.User.Username)
|
||||
if !mine {
|
||||
return nil, fmt.Errorf("HOUSECARL_CHANGESET is %d, but changeset %d is %s's (made via %s), not yours, so it isn't used from HOUSECARL_CHANGESET; give it as -C %d to use it anyway, or 'unset HOUSECARL_CHANGESET'", id, id, cs.Username, cs.Via, id)
|
||||
}
|
||||
}
|
||||
if !flagQuiet {
|
||||
desc := cs.Description
|
||||
if desc == "" {
|
||||
desc = "no description"
|
||||
}
|
||||
fmt.Fprintf(os.Stderr, "changeset %d (%s, %s) from HOUSECARL_CHANGESET\n", id, desc, plural(len(cs.Ops), "operation"))
|
||||
}
|
||||
return &cs, nil
|
||||
}
|
||||
|
||||
// setupChangeset makes the client stage configuration changes: in
|
||||
// --changeset's changeset, else HOUSECARL_CHANGESET's (checked before
|
||||
// the first change).
|
||||
func setupChangeset(c *client.Client) error {
|
||||
if flagChangeset > 0 {
|
||||
c.Changeset = flagChangeset
|
||||
return nil
|
||||
}
|
||||
n, err := envChangeset()
|
||||
if err != nil || n == 0 {
|
||||
return err
|
||||
}
|
||||
c.Changeset = n
|
||||
c.BeforeStage = func() error {
|
||||
_, err := checkEnvChangeset(c, n, true)
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// changesetArg returns the changeset a changeset subcommand works on: the
|
||||
// ID argument, else --changeset, else HOUSECARL_CHANGESET (checked).
|
||||
// fromEnv says it came from HOUSECARL_CHANGESET.
|
||||
func changesetArg(c *client.Client, args []string, mustOwn bool) (id string, fromEnv bool, err error) {
|
||||
if len(args) > 0 {
|
||||
return args[0], false, nil
|
||||
}
|
||||
if flagChangeset > 0 {
|
||||
return strconv.FormatInt(flagChangeset, 10), false, nil
|
||||
}
|
||||
n, err := envChangeset()
|
||||
if err != nil {
|
||||
return "", false, err
|
||||
}
|
||||
if n == 0 {
|
||||
return "", false, errors.New("which changeset? Give its ID, or set HOUSECARL_CHANGESET (" + startHint + ")")
|
||||
}
|
||||
if _, err := checkEnvChangeset(c, n, mustOwn); err != nil {
|
||||
return "", false, err
|
||||
}
|
||||
return strconv.FormatInt(n, 10), true, nil
|
||||
}
|
||||
|
||||
// envChangesetDone reminds that HOUSECARL_CHANGESET still names a
|
||||
// changeset that was just applied or discarded.
|
||||
func envChangesetDone(id, what string) {
|
||||
if !flagQuiet {
|
||||
fmt.Fprintf(os.Stderr, "HOUSECARL_CHANGESET still names changeset %s, which is now %s: run 'unset HOUSECARL_CHANGESET' (or export a new one) before making more changes\n", id, what)
|
||||
}
|
||||
}
|
||||
|
||||
func plural(n int, word string) string {
|
||||
if n == 1 {
|
||||
return "1 " + word
|
||||
}
|
||||
return strconv.Itoa(n) + " " + word + "s"
|
||||
}
|
||||
|
||||
// printDiff prints a unified diff, colored on a terminal.
|
||||
func printDiff(w io.Writer, diff string) {
|
||||
color := isTerminal(os.Stdout)
|
||||
for _, line := range strings.Split(strings.TrimSuffix(diff, "\n"), "\n") {
|
||||
switch {
|
||||
case !color:
|
||||
case strings.HasPrefix(line, "+++"), strings.HasPrefix(line, "---"):
|
||||
line = "\x1b[1m" + line + "\x1b[0m"
|
||||
case strings.HasPrefix(line, "+"):
|
||||
line = "\x1b[32m" + line + "\x1b[0m"
|
||||
case strings.HasPrefix(line, "-"):
|
||||
line = "\x1b[31m" + line + "\x1b[0m"
|
||||
case strings.HasPrefix(line, "@@"):
|
||||
line = "\x1b[36m" + line + "\x1b[0m"
|
||||
}
|
||||
fmt.Fprintln(w, line)
|
||||
}
|
||||
}
|
||||
|
||||
func isTerminal(f *os.File) bool {
|
||||
fi, err := f.Stat()
|
||||
return err == nil && fi.Mode()&os.ModeCharDevice != 0
|
||||
}
|
||||
|
||||
func changesetCmd() *cobra.Command {
|
||||
cmd := &cobra.Command{
|
||||
Use: "changeset",
|
||||
Aliases: []string{"changesets", "cs"},
|
||||
Short: "Stage changes to review and apply together",
|
||||
Long: `A changeset collects configuration changes to apply together, as one
|
||||
change in the history. Each change is checked as it is staged; 'hc
|
||||
changeset diff' shows what applying them would do: the problems, nftables'
|
||||
check and the ruleset's diff. Over the network an applied changeset is on
|
||||
probation like any change.
|
||||
|
||||
The web UI stages its edits in a changeset by default; hc and the API
|
||||
make changes at once unless one is named with -C ID or HOUSECARL_CHANGESET.
|
||||
Changesets staged in the web UI show up here too, and the other way round.`,
|
||||
Example: ` export HOUSECARL_CHANGESET=$(hc changeset create "guest network" -q)
|
||||
hc zone add guest -i eth1.30
|
||||
hc rule add forward --in guest --out wan
|
||||
hc changeset diff # HOUSECARL_CHANGESET's
|
||||
hc changeset apply
|
||||
unset HOUSECARL_CHANGESET`,
|
||||
}
|
||||
cmd.AddCommand(&cobra.Command{
|
||||
Use: "create [DESCRIPTION]", Short: "Start a changeset (-q prints only its ID)", Args: cobra.MaximumNArgs(1),
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
c, err := newClient()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
in := api.ChangesetInput{}
|
||||
if len(args) > 0 {
|
||||
in.Description = args[0]
|
||||
}
|
||||
var cs model.Changeset
|
||||
if err := c.Do(ctx(), "POST", "/changesets", nil, in, &cs); err != nil {
|
||||
return err
|
||||
}
|
||||
if flagQuiet || flagOutput == "id" {
|
||||
fmt.Println(cs.ID)
|
||||
return nil
|
||||
}
|
||||
fmt.Printf("created changeset %d\nstage changes with -C %d (or export HOUSECARL_CHANGESET=%d), then 'hc changeset apply'\n", cs.ID, cs.ID, cs.ID)
|
||||
return nil
|
||||
},
|
||||
})
|
||||
var status string
|
||||
var mine bool
|
||||
list := &cobra.Command{
|
||||
Use: "list", Short: "List changesets (open ones by default)", Args: cobra.NoArgs,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
c, err := newClient()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var css []*model.Changeset
|
||||
query := q("status", status)
|
||||
if mine {
|
||||
query.Set("mine", "1")
|
||||
}
|
||||
if err := c.Do(ctx(), "GET", "/changesets", query, nil, &css); err != nil {
|
||||
return err
|
||||
}
|
||||
return output(css, func(w io.Writer) {
|
||||
fmt.Fprintln(w, "ID\tSTATUS\tBY\tVIA\tOPERATIONS\tUPDATED\tDESCRIPTION")
|
||||
for _, cs := range css {
|
||||
fmt.Fprintf(w, "%d\t%s\t%s\t%s\t%d\t%s\t%s\n", cs.ID, cs.Status, cs.Username, cs.Via, len(cs.Ops), cs.UpdatedAt.Local().Format("2006-01-02 15:04"), cs.Description)
|
||||
}
|
||||
})
|
||||
},
|
||||
}
|
||||
list.Flags().StringVar(&status, "status", "open", "open, applied, discarded, or \"\" for all")
|
||||
list.Flags().BoolVar(&mine, "mine", false, "only your own")
|
||||
cmd.AddCommand(list)
|
||||
cmd.AddCommand(&cobra.Command{
|
||||
Use: "show [ID]", Short: "Show a changeset's operations (default: -C or HOUSECARL_CHANGESET)", Args: cobra.MaximumNArgs(1),
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
c, err := newClient()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
id, _, err := changesetArg(c, args, false)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var cs model.Changeset
|
||||
if err := c.Do(ctx(), "GET", "/changesets/"+id, nil, nil, &cs); err != nil {
|
||||
return err
|
||||
}
|
||||
return output(&cs, func(w io.Writer) {
|
||||
fmt.Fprintf(w, "Changeset:\t%d (%s)\n", cs.ID, cs.Status)
|
||||
fmt.Fprintf(w, "By:\t%s via %s, %s\n", cs.Username, cs.Via, cs.CreatedAt.Local().Format("2006-01-02 15:04"))
|
||||
if cs.Description != "" {
|
||||
fmt.Fprintf(w, "Description:\t%s\n", cs.Description)
|
||||
}
|
||||
if cs.HistoryID != 0 {
|
||||
fmt.Fprintf(w, "History:\t%d\n", cs.HistoryID)
|
||||
}
|
||||
fmt.Fprintln(w, "\nOP\tCHANGE")
|
||||
for _, o := range cs.Ops {
|
||||
fmt.Fprintf(w, "%d\t%s\n", o.Seq, o.Summary)
|
||||
}
|
||||
})
|
||||
},
|
||||
})
|
||||
cmd.AddCommand(&cobra.Command{
|
||||
Use: "diff [ID]", Short: "Check an open changeset and show what applying it would change (default: -C or HOUSECARL_CHANGESET)", Args: cobra.MaximumNArgs(1),
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
c, err := newClient()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
id, _, err := changesetArg(c, args, false)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var pv api.ChangesetPreview
|
||||
if err := c.Do(ctx(), "GET", "/changesets/"+id+"/preview", nil, nil, &pv); err != nil {
|
||||
return err
|
||||
}
|
||||
if flagOutput == "json" || flagOutput == "yaml" {
|
||||
return output(&pv, nil)
|
||||
}
|
||||
for _, ch := range pv.Changes {
|
||||
if ch.Kind == model.KindSettings {
|
||||
fmt.Printf("would %s settings\n", ch.Action())
|
||||
} else {
|
||||
fmt.Printf("would %s %s %s\n", ch.Action(), ch.Kind, ch.Label)
|
||||
}
|
||||
}
|
||||
printProblems(os.Stderr, pv.Warnings)
|
||||
if pv.Error != nil {
|
||||
printProblems(os.Stderr, pv.Error.Problems)
|
||||
return fmt.Errorf("changeset %s can't be applied: %s", id, pv.Error.Message)
|
||||
}
|
||||
if !pv.Valid {
|
||||
fmt.Println("nothing to change")
|
||||
return nil
|
||||
}
|
||||
fmt.Println()
|
||||
printDiff(os.Stdout, pv.Diff)
|
||||
fmt.Printf("\nvalid: nftables accepts the new ruleset; apply with 'hc changeset apply %s'\n", id)
|
||||
return nil
|
||||
},
|
||||
})
|
||||
var desc string
|
||||
apply := &cobra.Command{
|
||||
Use: "apply [ID]", Short: "Apply an open changeset (default: -C or HOUSECARL_CHANGESET)", Args: cobra.MaximumNArgs(1),
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
c, err := newClient()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
id, fromEnv, err := changesetArg(c, args, true)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var resp api.MutationResponse
|
||||
if err := c.Do(ctx(), "POST", "/changesets/"+id+"/apply", nil, api.ChangesetInput{Description: desc}, &resp); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := report(&resp); err != nil {
|
||||
return err
|
||||
}
|
||||
if fromEnv {
|
||||
envChangesetDone(id, "applied")
|
||||
}
|
||||
return nil
|
||||
},
|
||||
}
|
||||
apply.Flags().StringVar(&desc, "description", "", "the history entry's description (default: the changeset's)")
|
||||
cmd.AddCommand(apply)
|
||||
cmd.AddCommand(&cobra.Command{
|
||||
Use: "discard [ID]", Short: "Throw away an open changeset (default: -C or HOUSECARL_CHANGESET)", Args: cobra.MaximumNArgs(1),
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
c, err := newClient()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
id, fromEnv, err := changesetArg(c, args, true)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := c.Do(ctx(), "POST", "/changesets/"+id+"/discard", nil, nil, nil); err != nil {
|
||||
return err
|
||||
}
|
||||
if !flagQuiet {
|
||||
fmt.Printf("discarded changeset %s\n", id)
|
||||
}
|
||||
if fromEnv {
|
||||
envChangesetDone(id, "discarded")
|
||||
}
|
||||
return nil
|
||||
},
|
||||
})
|
||||
cmd.AddCommand(&cobra.Command{
|
||||
Use: "remove [ID] OP", Short: "Remove one operation from an open changeset (default: -C or HOUSECARL_CHANGESET)", Args: cobra.RangeArgs(1, 2),
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
c, err := newClient()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
op := args[len(args)-1]
|
||||
id, _, err := changesetArg(c, args[:len(args)-1], true)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var cs model.Changeset
|
||||
if err := c.Do(ctx(), "DELETE", "/changesets/"+id+"/ops/"+op, nil, nil, &cs); err != nil {
|
||||
return err
|
||||
}
|
||||
if !flagQuiet {
|
||||
fmt.Printf("removed operation %s; changeset %s has %s\n", op, id, plural(len(cs.Ops), "operation"))
|
||||
}
|
||||
return nil
|
||||
},
|
||||
})
|
||||
return cmd
|
||||
}
|
||||
+129
-7
@@ -4,6 +4,7 @@ import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/url"
|
||||
"os"
|
||||
"os/exec"
|
||||
"sort"
|
||||
@@ -30,6 +31,9 @@ func statusCmd() *cobra.Command {
|
||||
}
|
||||
return output(st, func(w io.Writer) {
|
||||
fmt.Fprintf(w, "Housecarl:\t%s, up since %s\n", st.Build.Label, st.Started.Local().Format("2006-01-02 15:04"))
|
||||
if st.Mode == api.ModeShadow {
|
||||
fmt.Fprintf(w, "Mode:\tSHADOW: the ruleset blocks, rejects and translates nothing; the other firewall decides (Housecarl's chains run %s it)\n", st.ShadowPosition)
|
||||
}
|
||||
if st.Applied {
|
||||
fmt.Fprintln(w, "Ruleset:\tin force (table inet housecarl)")
|
||||
} else if st.Unconfigured {
|
||||
@@ -125,10 +129,16 @@ func settingsCmd() *cobra.Command {
|
||||
fmt.Fprintf(w, "Antispoof IPv6:\t%s\n", list(s.AntispoofV6))
|
||||
fmt.Fprintf(w, "Flowtable:\t%s\n", list(s.FlowtableDevices))
|
||||
fmt.Fprintf(w, "Probation:\t%s\n", fmtSeconds(s.Probation()))
|
||||
fmt.Fprintf(w, "Traffic per device:\t%s\n", map[bool]string{true: "counted (statistics)", false: "not counted"}[s.StatsPerDevice])
|
||||
if s.ManualConfirm {
|
||||
fmt.Fprintln(w, "Confirmation:\tby a person ('hc confirm', the web UI's banner)")
|
||||
} else {
|
||||
fmt.Fprintln(w, "Confirmation:\tautomatic, by the client that made the change, once it reaches Housecarl again over a new connection")
|
||||
}
|
||||
})
|
||||
},
|
||||
})
|
||||
var input, forward, output, logDrops, probation string
|
||||
var input, forward, output, logDrops, probation, autoConf, perDevice string
|
||||
var builtins, own, flowtable, spoof4, spoof6 []string
|
||||
set := &cobra.Command{
|
||||
Use: "set [flags]", Short: "Change settings (only the flags given)", Args: cobra.NoArgs,
|
||||
@@ -136,9 +146,10 @@ func settingsCmd() *cobra.Command {
|
||||
turned off: loopback, established, drop_invalid, icmpv6, ping_internal,
|
||||
dhcpv6_client, antispoof, mss_clamp.
|
||||
|
||||
hc settings set --builtin mss_clamp=on --own-prefix 2a02:8013:7f3d::/48
|
||||
hc settings set --builtin mss_clamp=on --own-prefix 2001:db8::/48
|
||||
hc settings set --forward-policy drop --log-drops on
|
||||
hc settings set --flowtable wan --flowtable enp2s0`,
|
||||
hc settings set --flowtable wan --flowtable eth1
|
||||
hc settings set --stats-per-device on`,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
fl := cmd.Flags()
|
||||
p := patch{}
|
||||
@@ -169,9 +180,27 @@ dhcpv6_client, antispoof, mss_clamp.
|
||||
}
|
||||
for flag, v := range map[string][]string{"own-prefix": own, "flowtable": flowtable, "antispoof-v4": spoof4, "antispoof-v6": spoof6} {
|
||||
if fl.Changed(flag) {
|
||||
// "none" (or "") empties the list.
|
||||
if len(v) == 1 && (v[0] == "none" || v[0] == "") {
|
||||
v = []string{}
|
||||
}
|
||||
p[map[string]string{"own-prefix": "own_prefixes", "flowtable": "flowtable_devices", "antispoof-v4": "antispoof_v4", "antispoof-v6": "antispoof_v6"}[flag]] = v
|
||||
}
|
||||
}
|
||||
if fl.Changed("stats-per-device") {
|
||||
on, err := onOff("stats-per-device", perDevice)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
p["stats_per_device"] = on
|
||||
}
|
||||
if fl.Changed("auto-confirm") {
|
||||
on, err := onOff("auto-confirm", autoConf)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
p["manual_confirm"] = !on
|
||||
}
|
||||
if fl.Changed("probation") {
|
||||
n, err := parseSeconds(probation)
|
||||
if err != nil {
|
||||
@@ -193,6 +222,8 @@ dhcpv6_client, antispoof, mss_clamp.
|
||||
f.StringArrayVar(&spoof4, "antispoof-v4", nil, "IPv4 sources dropped from the internet (replaces)")
|
||||
f.StringArrayVar(&spoof6, "antispoof-v6", nil, "IPv6 sources dropped from the internet (replaces)")
|
||||
f.StringVar(&probation, "probation", "", "how long a network change has to be confirmed (2m)")
|
||||
f.StringVar(&perDevice, "stats-per-device", "", "count traffic per inside device, for the statistics' top devices: on or off")
|
||||
f.StringVar(&autoConf, "auto-confirm", "", "on: the client that made a change confirms it once it reaches Housecarl again over a new connection; off: a person confirms")
|
||||
cmd.AddCommand(set)
|
||||
return cmd
|
||||
}
|
||||
@@ -210,21 +241,28 @@ func onOff(name, v string) (bool, error) {
|
||||
// ---- the ruleset, counters, the log
|
||||
|
||||
func rulesetCmd() *cobra.Command {
|
||||
return &cobra.Command{
|
||||
var enforcing bool
|
||||
cmd := &cobra.Command{
|
||||
Use: "ruleset", Short: "Print the nftables ruleset Housecarl builds", Args: cobra.NoArgs,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
c, err := newClient()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var q url.Values
|
||||
if enforcing {
|
||||
q = url.Values{"enforcing": {"1"}}
|
||||
}
|
||||
var text string
|
||||
if err := c.Do(ctx(), "GET", "/ruleset", nil, nil, &text); err != nil {
|
||||
if err := c.Do(ctx(), "GET", "/ruleset", q, nil, &text); err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Print(text)
|
||||
return nil
|
||||
},
|
||||
}
|
||||
cmd.Flags().BoolVar(&enforcing, "enforcing", false, "in shadow mode: the ruleset loaded once it is off")
|
||||
return cmd
|
||||
}
|
||||
|
||||
func countersCmd() *cobra.Command {
|
||||
@@ -252,6 +290,63 @@ func countersCmd() *cobra.Command {
|
||||
}
|
||||
}
|
||||
|
||||
func quotasCmd() *cobra.Command {
|
||||
return &cobra.Command{
|
||||
Use: "quotas", Aliases: []string{"quota"}, Short: "What the rules' quotas used, and when they reset", Args: cobra.NoArgs,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
qs, err := get[[]api.QuotaInfo]("/quotas", nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var addrs []string
|
||||
for _, q := range qs {
|
||||
for _, a := range q.Addresses {
|
||||
addrs = append(addrs, a.Address)
|
||||
}
|
||||
}
|
||||
names := lookupNames(addrs...)
|
||||
return output(qs, func(w io.Writer) {
|
||||
fmt.Fprintln(w, "RULE\tUSED\tOF\tRESETS\tNEXT RESET")
|
||||
for _, q := range qs {
|
||||
resets := q.Quota.Reset
|
||||
if resets == "" {
|
||||
resets = "by hand"
|
||||
} else if resets == "monthly" && q.Quota.Day > 1 {
|
||||
resets = fmt.Sprintf("monthly (%d)", q.Quota.Day)
|
||||
}
|
||||
next := "-"
|
||||
if q.NextReset != nil {
|
||||
next = q.NextReset.Local().Format("2006-01-02 15:04")
|
||||
}
|
||||
of := humanBytes(q.Quota.Bytes)
|
||||
if q.Quota.PerSource || q.Quota.PerDestination {
|
||||
of += " each"
|
||||
}
|
||||
used := humanBytes(q.Used)
|
||||
if !q.Quota.PerSource && !q.Quota.PerDestination {
|
||||
used += " (" + percent(q.Used, q.Quota.Bytes) + ")"
|
||||
}
|
||||
fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\n", q.Rule, used, of, resets, next)
|
||||
for _, a := range q.Addresses {
|
||||
fmt.Fprintf(w, " %s\t%s (%s)\t\t\t\n", named(names, a.Address, a.Address), humanBytes(a.Used), percent(a.Used, q.Quota.Bytes))
|
||||
}
|
||||
}
|
||||
})
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// percent writes used as a share of total: "25%", "<1%" for a little.
|
||||
func percent(used, total int64) string {
|
||||
if total <= 0 || used <= 0 {
|
||||
return "0%"
|
||||
}
|
||||
if used*100 < total {
|
||||
return "<1%"
|
||||
}
|
||||
return fmt.Sprintf("%d%%", used*100/total)
|
||||
}
|
||||
|
||||
func logCmd() *cobra.Command {
|
||||
var follow bool
|
||||
var limit int
|
||||
@@ -268,6 +363,11 @@ func logCmd() *cobra.Command {
|
||||
var last time.Time
|
||||
print := func(es []api.LogEntry) {
|
||||
sort.Slice(es, func(i, j int) bool { return es[i].Time.Before(es[j].Time) })
|
||||
var addrs []string
|
||||
for _, e := range es {
|
||||
addrs = append(addrs, e.Source, e.Dest)
|
||||
}
|
||||
names := lookupNames(addrs...)
|
||||
for _, e := range es {
|
||||
last = e.Time
|
||||
src, dst := e.Source, e.Dest
|
||||
@@ -277,6 +377,7 @@ func logCmd() *cobra.Command {
|
||||
if e.DestPort != 0 {
|
||||
dst = joinPort(dst, e.DestPort)
|
||||
}
|
||||
src, dst = named(names, e.Source, src), named(names, e.Dest, dst)
|
||||
what := e.Protocol
|
||||
if e.ICMPType != "" {
|
||||
what += " " + e.ICMPType
|
||||
@@ -284,6 +385,9 @@ func logCmd() *cobra.Command {
|
||||
if e.TCPFlags != "" {
|
||||
what += " " + e.TCPFlags
|
||||
}
|
||||
if e.Shadow && (e.Verdict == "drop" || e.Verdict == "reject") {
|
||||
what += " (shadow mode: would " + e.Verdict + ", let through)"
|
||||
}
|
||||
fmt.Printf("%s %-28s %-6s %-6s %s -> %s %s\n", e.Time.Local().Format("15:04:05"), strings.TrimSpace(e.Prefix), orDash(e.In), orDash(e.Out), src, dst, what)
|
||||
}
|
||||
}
|
||||
@@ -372,6 +476,7 @@ func rollbackCmd() *cobra.Command {
|
||||
func importCmd() *cobra.Command {
|
||||
var replace bool
|
||||
var table, defines, textFile string
|
||||
var zones []string
|
||||
cmd := &cobra.Command{
|
||||
Use: "import [FILE|-]",
|
||||
Short: "Import an nftables ruleset (nft -j JSON) as Housecarl's configuration",
|
||||
@@ -383,7 +488,9 @@ forwards and NAT. The input is nft's JSON; on the firewall:
|
||||
Without a FILE, hc runs 'nft -j list ruleset' and 'nft -a list ruleset'
|
||||
itself (root on the firewall); from elsewhere, give the JSON as FILE and
|
||||
the text with --text. --defines reads the ruleset file's define lines, to
|
||||
name the sets as you did (trusted_ips, wg0_net, …). -n shows the result without
|
||||
name the sets as you did (admins, lan_net, …), and the zones; --zone
|
||||
enp2s0=lan names one yourself (repeat; interfaces given one name share a
|
||||
zone). -n shows the result without
|
||||
saving; what can't be expressed becomes raw rules, listed with notes.
|
||||
Once it looks right, import without -n, check it with 'hc ruleset', and
|
||||
'hc takeover inet main' deletes the old table.`,
|
||||
@@ -411,7 +518,14 @@ Once it looks right, import without -n, check it with 'hc ruleset', and
|
||||
return err
|
||||
}
|
||||
}
|
||||
in := api.ImportInput{JSON: data, Text: string(text), Table: table, Replace: replace}
|
||||
in := api.ImportInput{JSON: data, Text: string(text), Table: table, Replace: replace, Zones: map[string]string{}}
|
||||
for _, z := range zones {
|
||||
iface, name, ok := strings.Cut(z, "=")
|
||||
if !ok || iface == "" || name == "" {
|
||||
return fmt.Errorf("--zone %q: write it as INTERFACE=NAME (enp2s0=lan)", z)
|
||||
}
|
||||
in.Zones[iface] = name
|
||||
}
|
||||
if defines != "" {
|
||||
b, err := os.ReadFile(defines)
|
||||
if err != nil {
|
||||
@@ -446,6 +560,7 @@ Once it looks right, import without -n, check it with 'hc ruleset', and
|
||||
cmd.Flags().StringVar(&table, "table", "", "import only this table (\"inet main\"; default: all but Housecarl's)")
|
||||
cmd.Flags().StringVar(&defines, "defines", "", "the ruleset file, whose defines name the sets")
|
||||
cmd.Flags().StringVar(&textFile, "text", "", "'nft -a list ruleset' output, to keep what can't be expressed as raw rules")
|
||||
cmd.Flags().StringArrayVar(&zones, "zone", nil, "name an interface's zone: INTERFACE=NAME (repeat)")
|
||||
return cmd
|
||||
}
|
||||
|
||||
@@ -475,6 +590,13 @@ come back. Don't forget the boot: disable nftables.service, or empty
|
||||
return err
|
||||
}
|
||||
fmt.Println(out["message"])
|
||||
if tok, _ := out["confirm_token"].(string); tok != "" && !flagNoConfirm {
|
||||
if err := autoConfirm(tok); err != nil {
|
||||
fmt.Printf("couldn't confirm it (%v): confirm with 'hc confirm', or the tables come back\n", err)
|
||||
} else {
|
||||
fmt.Println("confirmed: this client reached Housecarl again over a new connection")
|
||||
}
|
||||
}
|
||||
return nil
|
||||
},
|
||||
}
|
||||
|
||||
+66
-18
@@ -37,14 +37,19 @@ func main() {
|
||||
|
||||
// Global flags.
|
||||
var (
|
||||
flagServer string
|
||||
flagToken string
|
||||
flagConfig string
|
||||
flagDryRun bool
|
||||
flagOutput string
|
||||
flagInsecure bool
|
||||
flagCACert string
|
||||
flagQuiet bool
|
||||
flagServer string
|
||||
flagToken string
|
||||
flagConfig string
|
||||
flagDryRun bool
|
||||
flagOutput string
|
||||
flagInsecure bool
|
||||
flagCACert string
|
||||
flagQuiet bool
|
||||
flagNoConfirm bool
|
||||
flagNoNames bool
|
||||
// lastClient is the client the command made (to confirm a change by
|
||||
// itself after it).
|
||||
lastClient *client.Client
|
||||
)
|
||||
|
||||
// DefaultSocket is housecarld's local socket.
|
||||
@@ -132,6 +137,10 @@ func newClient() (*client.Client, error) {
|
||||
return nil, err
|
||||
}
|
||||
c.DryRun = flagDryRun
|
||||
if err := setupChangeset(c); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
lastClient = c
|
||||
return c, nil
|
||||
}
|
||||
|
||||
@@ -143,11 +152,15 @@ func rootCmd() *cobra.Command {
|
||||
|
||||
Every change is checked as a whole, then by nftables, before it is put in
|
||||
force, and is recorded in the history, where 'hc history undo ID' takes it
|
||||
back. --dry-run shows what a change would do.
|
||||
back. --dry-run shows what a change would do, with the ruleset's diff.
|
||||
Changes are made at once; to review several and apply them together,
|
||||
stage them in a changeset ('hc changeset --help', -C ID or
|
||||
HOUSECARL_CHANGESET).
|
||||
|
||||
A change made over the network is on probation: confirm it with 'hc
|
||||
confirm' (within 2 minutes by default) or it is rolled back by itself, so a
|
||||
change that cuts you off undoes itself. On the firewall itself, root uses
|
||||
A change made over the network is on probation: hc confirms it by itself
|
||||
once it reaches Housecarl again over a new connection, and a change that
|
||||
cuts you off can't be confirmed, so it is rolled back by itself (within 2
|
||||
minutes by default). --no-confirm leaves it to 'hc confirm'. On the firewall itself, root uses
|
||||
housecarld's local socket: no token, no probation. From elsewhere, set the
|
||||
server and an API token once:
|
||||
|
||||
@@ -169,6 +182,9 @@ Exit codes: 0 ok, 1 error, 3 not authenticated or not allowed, 4 not found,
|
||||
pf.BoolVar(&flagInsecure, "insecure", false, "don't verify the server's TLS certificate")
|
||||
pf.StringVar(&flagCACert, "ca-cert", "", "PEM file with a CA to trust for the server's certificate")
|
||||
pf.BoolVarP(&flagQuiet, "quiet", "q", false, "print only errors (and IDs of new objects)")
|
||||
pf.BoolVar(&flagNoConfirm, "no-confirm", false, "leave a change on probation instead of confirming it once this client reaches Housecarl again")
|
||||
pf.BoolVar(&flagNoNames, "no-names", false, "show inside addresses without their names (reverse lookups)")
|
||||
pf.Int64VarP(&flagChangeset, "changeset", "C", 0, "stage configuration changes in this changeset instead of applying them (env HOUSECARL_CHANGESET)")
|
||||
root.AddGroup(&cobra.Group{ID: "fw", Title: "Firewall:"}, &cobra.Group{ID: "changes", Title: "Changes:"}, &cobra.Group{ID: "admin", Title: "Administration:"})
|
||||
add := func(group string, cmds ...*cobra.Command) {
|
||||
for _, c := range cmds {
|
||||
@@ -176,9 +192,9 @@ Exit codes: 0 ok, 1 error, 3 not authenticated or not allowed, 4 not found,
|
||||
root.AddCommand(c)
|
||||
}
|
||||
}
|
||||
add("fw", statusCmd(), zoneCmd(), setCmd(), serviceCmd(), ruleCmd(), forwardCmd(), snatCmd(), redirectCmd(), settingsCmd(),
|
||||
rulesetCmd(), countersCmd(), logCmd(), interfaceCmd())
|
||||
add("changes", confirmCmd(), rollbackCmd(), historyCmd(), importCmd(), takeoverCmd())
|
||||
add("fw", statusCmd(), zoneCmd(), setCmd(), serviceCmd(), ruleCmd(), forwardCmd(), snatCmd(), redirectCmd(), banlistCmd(), bansCmd(), banCmd(), unbanCmd(), settingsCmd(),
|
||||
rulesetCmd(), countersCmd(), quotasCmd(), logCmd(), connectionsCmd(), traceCmd(), matrixCmd(), statsCmd(), interfaceCmd())
|
||||
add("changes", changesetCmd(), confirmCmd(), rollbackCmd(), historyCmd(), importCmd(), takeoverCmd())
|
||||
add("admin", whoamiCmd(), userCmd(), tokenCmd(), passwordCmd(), certCmd(), auditCmd(), configCmd())
|
||||
return root
|
||||
}
|
||||
@@ -237,8 +253,6 @@ func outputIDs(v any) error {
|
||||
if m, ok := it.(map[string]any); ok {
|
||||
if id, ok := m["id"]; ok {
|
||||
fmt.Println(id)
|
||||
} else if ip, ok := m["ip"]; ok {
|
||||
fmt.Println(ip)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -294,6 +308,19 @@ func exitCode(err error) int {
|
||||
return 1
|
||||
}
|
||||
|
||||
// autoConfirm confirms a change with its token: the request goes over a
|
||||
// new connection (the server closed the change's), so it gets through
|
||||
// only if the new ruleset still lets this client reach Housecarl.
|
||||
func autoConfirm(token string) error {
|
||||
if lastClient == nil {
|
||||
return errors.New("no client")
|
||||
}
|
||||
c, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
defer cancel()
|
||||
var out map[string]int
|
||||
return lastClient.Do(c, "POST", "/confirm", nil, api.ConfirmInput{Token: token}, &out)
|
||||
}
|
||||
|
||||
// report prints the outcome of a change.
|
||||
func report(resp *api.MutationResponse) error {
|
||||
switch flagOutput {
|
||||
@@ -310,6 +337,9 @@ func report(resp *api.MutationResponse) error {
|
||||
if resp.DryRun {
|
||||
return "would " + c.Action()
|
||||
}
|
||||
if resp.Staged {
|
||||
return "staged: " + c.Action()
|
||||
}
|
||||
return c.Action() + "d"
|
||||
}
|
||||
if !flagQuiet || resp.DryRun {
|
||||
@@ -322,8 +352,26 @@ func report(resp *api.MutationResponse) error {
|
||||
}
|
||||
}
|
||||
switch {
|
||||
case resp.Staged && resp.Changeset != nil:
|
||||
if !flagQuiet {
|
||||
cs := resp.Changeset
|
||||
fmt.Fprintf(w, "staged in changeset %d as operation %d (%s now); review with 'hc changeset diff %d', apply with 'hc changeset apply %d'\n",
|
||||
cs.ID, cs.Ops[len(cs.Ops)-1].Seq, plural(len(cs.Ops), "operation"), cs.ID, cs.ID)
|
||||
}
|
||||
case resp.DryRun:
|
||||
if resp.Diff != "" && !flagQuiet {
|
||||
fmt.Fprintln(w)
|
||||
printDiff(w, resp.Diff)
|
||||
fmt.Fprintln(w)
|
||||
}
|
||||
fmt.Fprintln(w, "dry run: nothing was changed")
|
||||
case resp.Applied && resp.ConfirmBy != nil && resp.ConfirmToken != "" && !flagNoConfirm:
|
||||
if err := autoConfirm(resp.ConfirmToken); err != nil {
|
||||
fmt.Fprintf(w, "applied on probation (history %d), but couldn't confirm it: %v\nIt is rolled back within %s unless confirmed ('hc confirm')\n",
|
||||
resp.HistoryID, err, time.Until(*resp.ConfirmBy).Round(time.Second))
|
||||
} else if !flagQuiet {
|
||||
fmt.Fprintf(w, "applied and confirmed (history %d): this client reached Housecarl again over a new connection\n", resp.HistoryID)
|
||||
}
|
||||
case resp.Applied && resp.ConfirmBy != nil:
|
||||
fmt.Fprintf(w, "applied on probation (history %d): confirm with 'hc confirm' within %s, or it is rolled back\n",
|
||||
resp.HistoryID, time.Until(*resp.ConfirmBy).Round(time.Second))
|
||||
@@ -332,7 +380,7 @@ func report(resp *api.MutationResponse) error {
|
||||
case !resp.Applied && len(resp.Changes) == 0 && !flagQuiet:
|
||||
fmt.Fprintln(w, "nothing to change")
|
||||
}
|
||||
if flagQuiet && len(resp.Object) > 0 && resp.Applied {
|
||||
if flagQuiet && len(resp.Object) > 0 && (resp.Applied || resp.Staged) {
|
||||
outputIDs(resp.Object)
|
||||
}
|
||||
printProblems(os.Stderr, resp.Warnings)
|
||||
|
||||
+48
-3
@@ -1,8 +1,11 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"regexp"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"housecarl/internal/matrix"
|
||||
"housecarl/internal/model"
|
||||
)
|
||||
|
||||
@@ -32,11 +35,53 @@ func TestLimitAndDescribe(t *testing.T) {
|
||||
if _, err := parseLimit("fast"); err == nil {
|
||||
t.Error("a bad limit accepted")
|
||||
}
|
||||
m := model.Match{InZones: []string{"wan"}, NotSources: true, Sources: []string{"trusted"}, Services: []string{"ssh", "tcp/8080"}}
|
||||
if got := describe(m); got != "in wan from not trusted ssh,tcp/8080" {
|
||||
t.Errorf("describe: %q", got)
|
||||
for _, c := range []struct {
|
||||
m model.Match
|
||||
chain string
|
||||
want string
|
||||
}{
|
||||
{model.Match{InZones: []string{"wan"}, NotSources: true, Sources: []string{"trusted"}, Services: []string{"ssh", "tcp/8080"}}, "input",
|
||||
"wan → firewall from any but trusted ssh,tcp/8080"},
|
||||
{model.Match{InZones: []string{"wan"}, NotIn: true, Destinations: []string{"trusted_ips"}}, "forward", "any but wan → any to trusted_ips"},
|
||||
{model.Match{InZones: []string{"lan"}, OutZones: []string{"wan"}}, "forward", "lan → wan"},
|
||||
{model.Match{OutZones: []string{"wan"}}, "output", "firewall → wan"},
|
||||
{model.Match{OutZones: []string{"wan"}, Sources: []string{"10.0.0.0/8"}}, "", "any → wan from 10.0.0.0/8"},
|
||||
{model.Match{Sources: []string{"trusted"}}, "input", "from trusted"},
|
||||
{model.Match{}, "forward", "everything"},
|
||||
} {
|
||||
if got := describe(c.m, c.chain); got != c.want {
|
||||
t.Errorf("describe(%+v, %s) = %q, want %q", c.m, c.chain, got, c.want)
|
||||
}
|
||||
}
|
||||
for used, want := range map[int64]string{0: "0%", 5: "<1%", 250: "25%", 1000: "100%", 1300: "130%"} {
|
||||
if got := percent(used, 1000); got != want {
|
||||
t.Errorf("percent(%d, 1000) = %q, want %q", used, got, want)
|
||||
}
|
||||
}
|
||||
if got := editList([]string{"a", "b", "c"}, []string{"d"}, []string{"b"}); len(got) != 3 || got[1] != "c" || got[2] != "d" {
|
||||
t.Errorf("editList: %v", got)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMatrixAligned: the grid's columns line up in colour too (the
|
||||
// escape codes take no room on screen).
|
||||
func TestMatrixAligned(t *testing.T) {
|
||||
st := model.NewState()
|
||||
st.Zones = []*model.Zone{{ID: 1, Name: "lan", Interfaces: []string{"eth1"}}, {ID: 2, Name: "wan", Interfaces: []string{"wan"}, External: true}}
|
||||
st.Rules = []*model.Rule{{ID: 3, Chain: model.ChainForward, Position: 1, Action: model.ActionAccept, Match: model.Match{InZones: []string{"lan"}, Services: []string{"https"}}}}
|
||||
st.Settings.InputPolicy, st.Settings.ForwardPolicy = model.PolicyDrop, model.PolicyDrop
|
||||
var b strings.Builder
|
||||
writeMatrix(&b, matrix.Build(st), true)
|
||||
esc := regexp.MustCompile("\x1b\\[[0-9]+m")
|
||||
lines := strings.Split(esc.ReplaceAllString(b.String(), ""), "\n")
|
||||
col := strings.Index(lines[0], "wan")
|
||||
for _, l := range lines[1:4] {
|
||||
if r := []rune(l); len(r) <= col || r[col-1] != ' ' || r[col] == ' ' {
|
||||
t.Errorf("misaligned at column %d:\n%s", col, strings.Join(lines[:4], "\n"))
|
||||
break
|
||||
}
|
||||
}
|
||||
if !strings.Contains(b.String(), "\x1b[33mpartly +1") || !strings.Contains(b.String(), "lan → wan\thttps") {
|
||||
t.Errorf("no partly cell, or no list of what's open:\n%s", b.String())
|
||||
}
|
||||
}
|
||||
+43
-3
@@ -49,6 +49,46 @@ func counters() map[string]api.Counter {
|
||||
return cs.Counters
|
||||
}
|
||||
|
||||
// lookupNames asks for inside addresses' names, for tables: none with
|
||||
// --no-names, another output format, or when they can't be had.
|
||||
func lookupNames(addrs ...string) map[string]string {
|
||||
out := map[string]string{}
|
||||
if flagNoNames || flagOutput != "table" {
|
||||
return out
|
||||
}
|
||||
seen := map[string]bool{}
|
||||
var list []string
|
||||
for _, a := range addrs {
|
||||
if a != "" && !seen[a] {
|
||||
seen[a] = true
|
||||
list = append(list, a)
|
||||
}
|
||||
}
|
||||
if len(list) == 0 {
|
||||
return out
|
||||
}
|
||||
c, err := newClient()
|
||||
if err != nil {
|
||||
return out
|
||||
}
|
||||
var ns api.Names
|
||||
if c.Do(ctx(), "POST", "/names", nil, api.NamesInput{Addresses: list}, &ns) != nil {
|
||||
return out
|
||||
}
|
||||
for a, n := range ns.Names {
|
||||
out[a] = strings.SplitN(n, ".", 2)[0]
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// named adds an address's name to text (the address, or address:port).
|
||||
func named(names map[string]string, addr, text string) string {
|
||||
if n := names[addr]; n != "" {
|
||||
return text + " (" + n + ")"
|
||||
}
|
||||
return text
|
||||
}
|
||||
|
||||
func hits(c map[string]api.Counter, marks ...string) string {
|
||||
var p uint64
|
||||
found := false
|
||||
@@ -144,7 +184,7 @@ many: --interface 'incusbr*'. --external marks a zone facing the internet:
|
||||
antispoofing, MSS clamping and port forwards apply there.
|
||||
|
||||
hc zone add wan --interface wan --external
|
||||
hc zone add lan --interface enp2s0`,
|
||||
hc zone add lan --interface eth1`,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
return change("POST", "/zones", model.Zone{Name: args[0], Interfaces: ifaces, External: external, Comment: comment})
|
||||
},
|
||||
@@ -267,7 +307,7 @@ func setCmd() *cobra.Command {
|
||||
Use: "add NAME [ENTRY...]", Short: "Add a set", Args: cobra.MinimumNArgs(1),
|
||||
Long: `Adds a set of addresses, prefixes and ranges, IPv4 and IPv6 mixed:
|
||||
|
||||
hc set add trusted 10.0.10.0/24 10.20.30.0/24 2a02:8013:7f3d:10::/64
|
||||
hc set add trusted 192.168.1.0/24 192.168.2.0/24 2001:db8:1::/64
|
||||
hc set add guests --dynamic --timeout 24h
|
||||
|
||||
A dynamic set takes entries at run time, each with its own timeout ('hc
|
||||
@@ -425,7 +465,7 @@ func serviceCmd() *cobra.Command {
|
||||
Use: "add NAME ENTRY...", Short: "Add a service", Args: cobra.MinimumNArgs(2),
|
||||
Long: `Adds a service: ports of a protocol, and ICMP types.
|
||||
|
||||
hc service add jellyfin tcp/8096 tcp/8920
|
||||
hc service add web tcp/8080 tcp/8443
|
||||
hc service add games udp/27015-27030 tcp/27015
|
||||
hc service add pmtu icmp/destination-unreachable icmpv6/packet-too-big`,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
|
||||
+258
-54
@@ -3,6 +3,8 @@ package main
|
||||
import (
|
||||
"fmt"
|
||||
"io"
|
||||
"net/url"
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
@@ -62,22 +64,40 @@ func (m *matchFlags) patch(fl *pflag.FlagSet, p patch) {
|
||||
set("family", "family", m.family)
|
||||
}
|
||||
|
||||
// describe sums up a match: "in lan from trusted ssh, dns".
|
||||
func describe(m model.Match) string {
|
||||
// describe sums up a match in a chain ("" for NAT): "lan → firewall
|
||||
// from trusted ssh,dns". Zones are a direction; the firewall itself is
|
||||
// the other end of the input and output chains.
|
||||
func describe(m model.Match, chain string) string {
|
||||
var parts []string
|
||||
side := func(word string, not bool, list []string) {
|
||||
if len(list) == 0 {
|
||||
return
|
||||
}
|
||||
list := func(not bool, xs []string) string {
|
||||
s := strings.Join(xs, ",")
|
||||
if not {
|
||||
word += " not"
|
||||
return "any but " + s
|
||||
}
|
||||
parts = append(parts, word+" "+strings.Join(list, ","))
|
||||
return s
|
||||
}
|
||||
if len(m.InZones) > 0 || len(m.OutZones) > 0 {
|
||||
in, out := "any", "any"
|
||||
if len(m.InZones) > 0 {
|
||||
in = list(m.NotIn, m.InZones)
|
||||
}
|
||||
if len(m.OutZones) > 0 {
|
||||
out = list(m.NotOut, m.OutZones)
|
||||
}
|
||||
switch chain {
|
||||
case model.ChainInput:
|
||||
out = "firewall"
|
||||
case model.ChainOutput:
|
||||
in = "firewall"
|
||||
}
|
||||
parts = append(parts, in+" → "+out)
|
||||
}
|
||||
if len(m.Sources) > 0 {
|
||||
parts = append(parts, "from "+list(m.NotSources, m.Sources))
|
||||
}
|
||||
if len(m.Destinations) > 0 {
|
||||
parts = append(parts, "to "+list(m.NotDests, m.Destinations))
|
||||
}
|
||||
side("in", m.NotIn, m.InZones)
|
||||
side("out", m.NotOut, m.OutZones)
|
||||
side("from", m.NotSources, m.Sources)
|
||||
side("to", m.NotDests, m.Destinations)
|
||||
if len(m.Services) > 0 {
|
||||
parts = append(parts, strings.Join(m.Services, ","))
|
||||
}
|
||||
@@ -96,9 +116,9 @@ type ruleFlags struct {
|
||||
matchFlags
|
||||
action, rejectWith, comment, logPrefix, raw string
|
||||
states, statuses, days []string
|
||||
hours, limit string
|
||||
connLimit int
|
||||
log, disabled bool
|
||||
hours, limit, quota string
|
||||
connLimit, groupV4, groupV6 int
|
||||
log, disabled, everyPacket bool
|
||||
before, after int64
|
||||
first bool
|
||||
}
|
||||
@@ -111,13 +131,37 @@ func (r *ruleFlags) register(f *pflag.FlagSet) {
|
||||
f.StringSliceVar(&r.statuses, "status", nil, "connection status: dnat, snat")
|
||||
f.StringSliceVar(&r.days, "days", nil, "days it applies: Monday,Tuesday,…")
|
||||
f.StringVar(&r.hours, "hours", "", "hours it applies: 08:00-17:30 (22:00-06:00 passes midnight)")
|
||||
f.StringVar(&r.limit, "limit", "", "a rate: 10/minute, with ,burst=N ,over (beyond the rate) ,per-source")
|
||||
f.StringVar(&r.limit, "limit", "", "a rate: 10/minute, or bytes: 1mbytes/second, 500kbytes/minute; with ,burst=N ,over (beyond the rate) ,per-source or ,per-destination")
|
||||
f.IntVar(&r.connLimit, "conn-limit", 0, "match a source with more than this many connections")
|
||||
f.StringVar(&r.quota, "quota", "", "an allowance: 5000mbytes or 5gbytes, with ,over (match once used up) ,per-source or ,per-destination, and ,daily ,weekly or ,monthly=DAY to reset")
|
||||
f.IntVar(&r.groupV4, "group-v4", 0, "per-source/destination limits and quotas count IPv4 addresses by this prefix (default 32: each address)")
|
||||
f.IntVar(&r.groupV6, "group-v6", 0, "…and IPv6 addresses by this prefix (default 64: one device or network)")
|
||||
f.BoolVar(&r.log, "log", false, "log matching packets ('hc log')")
|
||||
f.StringVar(&r.logPrefix, "log-prefix", "", "the log line's prefix")
|
||||
f.StringVar(&r.raw, "raw", "", "an nftables rule written by hand instead of the match flags")
|
||||
f.StringVar(&r.comment, "comment", "", "a comment")
|
||||
f.BoolVar(&r.disabled, "disabled", false, "keep the rule but don't use it")
|
||||
f.BoolVar(&r.everyPacket, "every-packet", false, "check every packet, so a drop or reject also stops connections already open")
|
||||
}
|
||||
|
||||
var amountRe = regexp.MustCompile(`^(\d+)\s*([a-zA-Z]*)$`)
|
||||
|
||||
// byteUnit reads a unit: "" (packets, for rates), bytes, kbytes, mbytes,
|
||||
// gbytes, or b, kb, mb, gb in any case.
|
||||
func byteUnit(u string) (string, bool) {
|
||||
switch strings.ToLower(u) {
|
||||
case "", "packets", "packet", "p":
|
||||
return "", true
|
||||
case "bytes", "byte", "b":
|
||||
return "bytes", true
|
||||
case "kbytes", "kb", "k":
|
||||
return "kbytes", true
|
||||
case "mbytes", "mb", "m":
|
||||
return "mbytes", true
|
||||
case "gbytes", "gb", "g":
|
||||
return "gbytes", true
|
||||
}
|
||||
return "", false
|
||||
}
|
||||
|
||||
func parseLimit(s string) (*model.Limit, error) {
|
||||
@@ -126,32 +170,101 @@ func parseLimit(s string) (*model.Limit, error) {
|
||||
}
|
||||
parts := strings.Split(s, ",")
|
||||
rate, per, ok := strings.Cut(strings.TrimSpace(parts[0]), "/")
|
||||
n, err := strconv.Atoi(rate)
|
||||
if !ok || err != nil {
|
||||
return nil, fmt.Errorf("--limit %q: write it as 10/minute, with ,burst=N ,over ,per-source", s)
|
||||
m := amountRe.FindStringSubmatch(strings.TrimSpace(rate))
|
||||
unit, unitOK := "", false
|
||||
if m != nil {
|
||||
unit, unitOK = byteUnit(m[2])
|
||||
}
|
||||
l := &model.Limit{Rate: n, Per: strings.TrimSuffix(per, "s")}
|
||||
if !ok || m == nil || !unitOK || unit == "gbytes" {
|
||||
return nil, fmt.Errorf("--limit %q: write it as 10/minute, or as bytes 1mbytes/second (bytes, kbytes, mbytes), with ,burst=N ,over ,per-source", s)
|
||||
}
|
||||
n, _ := strconv.Atoi(m[1])
|
||||
l := &model.Limit{Rate: n, Unit: unit, Per: strings.TrimSuffix(per, "s")}
|
||||
for _, p := range parts[1:] {
|
||||
k, v, _ := strings.Cut(strings.TrimSpace(p), "=")
|
||||
switch k {
|
||||
case "burst":
|
||||
if l.Burst, err = strconv.Atoi(v); err != nil {
|
||||
return nil, fmt.Errorf("--limit: burst=%q isn't a number", v)
|
||||
b := amountRe.FindStringSubmatch(v)
|
||||
bu, buOK := "", false
|
||||
if b != nil {
|
||||
bu, buOK = byteUnit(b[2])
|
||||
}
|
||||
if b == nil || !buOK || (b[2] != "" && bu != unit) {
|
||||
return nil, fmt.Errorf("--limit: burst=%q: a number, in the rate's unit", v)
|
||||
}
|
||||
l.Burst, _ = strconv.Atoi(b[1])
|
||||
case "over":
|
||||
l.Over = true
|
||||
case "per-source":
|
||||
l.PerSource = true
|
||||
case "per-destination":
|
||||
l.PerDestination = true
|
||||
default:
|
||||
return nil, fmt.Errorf("--limit: unknown %q (burst=N, over, per-source)", k)
|
||||
return nil, fmt.Errorf("--limit: unknown %q (burst=N, over, per-source, per-destination)", k)
|
||||
}
|
||||
}
|
||||
return l, nil
|
||||
}
|
||||
|
||||
// parseQuota reads --quota: 5000mbytes,over,per-source,monthly=15.
|
||||
func parseQuota(s string) (*model.Quota, error) {
|
||||
if s == "" {
|
||||
return nil, nil
|
||||
}
|
||||
parts := strings.Split(s, ",")
|
||||
m := amountRe.FindStringSubmatch(strings.TrimSpace(parts[0]))
|
||||
unit, ok := "", false
|
||||
if m != nil {
|
||||
unit, ok = byteUnit(m[2])
|
||||
}
|
||||
if m == nil || !ok || unit == "" {
|
||||
return nil, fmt.Errorf("--quota %q: write it as 5000mbytes or 5gbytes, with ,over ,per-source ,daily ,weekly or ,monthly=DAY", s)
|
||||
}
|
||||
n, _ := strconv.ParseInt(m[1], 10, 64)
|
||||
q := &model.Quota{Bytes: n * model.ByteUnits[unit]}
|
||||
for _, p := range parts[1:] {
|
||||
k, v, _ := strings.Cut(strings.TrimSpace(p), "=")
|
||||
switch k {
|
||||
case "over":
|
||||
q.Over = true
|
||||
case "per-source":
|
||||
q.PerSource = true
|
||||
case "per-destination":
|
||||
q.PerDestination = true
|
||||
case "daily", "weekly":
|
||||
q.Reset = k
|
||||
case "monthly":
|
||||
q.Reset = k
|
||||
if v != "" {
|
||||
d, err := strconv.Atoi(v)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("--quota: monthly=%q: the day of the month, 1 to 31", v)
|
||||
}
|
||||
q.Day = d
|
||||
}
|
||||
default:
|
||||
return nil, fmt.Errorf("--quota: unknown %q (over, per-source, per-destination, daily, weekly, monthly=DAY)", k)
|
||||
}
|
||||
}
|
||||
return q, nil
|
||||
}
|
||||
|
||||
// humanBytes writes bytes for people: "1.8 GB".
|
||||
func humanBytes(n int64) string {
|
||||
switch {
|
||||
case n >= 1<<30:
|
||||
return strconv.FormatFloat(float64(n)/(1<<30), 'f', 1, 64) + " GB"
|
||||
case n >= 1<<20:
|
||||
return strconv.FormatFloat(float64(n)/(1<<20), 'f', 1, 64) + " MB"
|
||||
case n >= 1<<10:
|
||||
return strconv.FormatFloat(float64(n)/(1<<10), 'f', 1, 64) + " KB"
|
||||
}
|
||||
return strconv.FormatInt(n, 10) + " bytes"
|
||||
}
|
||||
|
||||
func (r *ruleFlags) rule(chain string) (model.Rule, error) {
|
||||
rule := model.Rule{Chain: chain, Match: r.match(), Action: r.action, RejectWith: r.rejectWith, CtState: r.states, CtStatus: r.statuses,
|
||||
Days: r.days, ConnLimit: r.connLimit, Log: r.log, LogPrefix: r.logPrefix, Raw: r.raw, Comment: r.comment, Disabled: r.disabled}
|
||||
Days: r.days, ConnLimit: r.connLimit, EveryPacket: r.everyPacket, Log: r.log, LogPrefix: r.logPrefix, Raw: r.raw, Comment: r.comment, Disabled: r.disabled}
|
||||
if r.hours != "" {
|
||||
from, to, ok := strings.Cut(r.hours, "-")
|
||||
if !ok {
|
||||
@@ -160,7 +273,12 @@ func (r *ruleFlags) rule(chain string) (model.Rule, error) {
|
||||
rule.HourFrom, rule.HourTo = from, to
|
||||
}
|
||||
l, err := parseLimit(r.limit)
|
||||
if err != nil {
|
||||
return rule, err
|
||||
}
|
||||
rule.Limit = l
|
||||
rule.Quota, err = parseQuota(r.quota)
|
||||
rule.GroupV4, rule.GroupV6 = r.groupV4, r.groupV6
|
||||
return rule, err
|
||||
}
|
||||
|
||||
@@ -181,12 +299,15 @@ func ruleCmd() *cobra.Command {
|
||||
return output(rs, func(w io.Writer) {
|
||||
fmt.Fprintln(w, "CHAIN\tID\tMATCH\tACTION\tHITS\tCOMMENT")
|
||||
for _, r := range rs {
|
||||
what := describe(r.Match)
|
||||
what := describe(r.Match, r.Chain)
|
||||
if r.Raw != "" {
|
||||
what = "raw: " + r.Raw
|
||||
}
|
||||
extra := ruleExtras(r)
|
||||
action := r.Action
|
||||
if r.Raw != "" {
|
||||
action = "-" // the text's own
|
||||
}
|
||||
if r.Log {
|
||||
action += "+log"
|
||||
}
|
||||
@@ -200,7 +321,11 @@ func ruleCmd() *cobra.Command {
|
||||
}
|
||||
cmd.AddCommand(ls)
|
||||
cmd.AddCommand(showCmd("rules", "rule", func(w io.Writer, r model.Rule) {
|
||||
fmt.Fprintf(w, "ID:\t%d\nChain:\t%s\nMatch:\t%s%s\nAction:\t%s\n", r.ID, r.Chain, describe(r.Match), ruleExtras(r), r.Action)
|
||||
if r.Raw == "" {
|
||||
fmt.Fprintf(w, "ID:\t%d\nChain:\t%s\nMatch:\t%s%s\nAction:\t%s\n", r.ID, r.Chain, describe(r.Match, r.Chain), ruleExtras(r), r.Action)
|
||||
} else {
|
||||
fmt.Fprintf(w, "ID:\t%d\nChain:\t%s\n", r.ID, r.Chain)
|
||||
}
|
||||
if r.Raw != "" {
|
||||
fmt.Fprintf(w, "Raw:\t%s\n", r.Raw)
|
||||
}
|
||||
@@ -221,7 +346,7 @@ output (from it) or early (before connection tracking: cheap drops).
|
||||
|
||||
hc rule add input --from trusted --service ssh
|
||||
hc rule add input --in wg --service dns
|
||||
hc rule add forward --in lan --out wan --from 2a02:8013:7f3d:10::/64
|
||||
hc rule add forward --in lan --out wan --from 2001:db8:1::/64
|
||||
hc rule add input --in wan --service ssh --limit 3/minute,burst=5,over,per-source -a drop --log
|
||||
hc rule add forward --in wan -a drop --comment "nothing else gets in"
|
||||
hc rule add forward --raw 'iifname != "wan" ct state invalid ip daddr @trusted_v4 accept'`,
|
||||
@@ -234,25 +359,20 @@ output (from it) or early (before connection tracking: cheap drops).
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var resp api.MutationResponse
|
||||
if err := c.Do(ctx(), "POST", "/rules", nil, r, &resp); err != nil {
|
||||
return err
|
||||
// Placed in the same change: one history entry, and never in
|
||||
// force at the end of the chain.
|
||||
q := url.Values{}
|
||||
switch {
|
||||
case rf.first:
|
||||
q.Set("first", "1")
|
||||
case rf.before != 0:
|
||||
q.Set("before", fmt.Sprint(rf.before))
|
||||
case rf.after != 0:
|
||||
q.Set("after", fmt.Sprint(rf.after))
|
||||
}
|
||||
// Then place it.
|
||||
if resp.Applied && (rf.first || rf.before != 0 || rf.after != 0) {
|
||||
var created model.Rule
|
||||
if err := jsonUnmarshal(resp.Object, &created); err == nil {
|
||||
mv := api.MoveInput{Before: rf.before, After: rf.after}
|
||||
if rf.first {
|
||||
mv.To = "start"
|
||||
}
|
||||
var moved api.MutationResponse
|
||||
if err := c.Do(ctx(), "POST", objPath("rules", fmt.Sprint(created.ID))+"/move", nil, mv, &moved); err != nil {
|
||||
return err
|
||||
}
|
||||
resp.Changes = append(resp.Changes, moved.Changes...)
|
||||
resp.ConfirmBy = moved.ConfirmBy
|
||||
}
|
||||
var resp api.MutationResponse
|
||||
if err := c.Do(ctx(), "POST", "/rules", q, r, &resp); err != nil {
|
||||
return err
|
||||
}
|
||||
return report(&resp)
|
||||
},
|
||||
@@ -277,7 +397,8 @@ output (from it) or early (before connection tracking: cheap drops).
|
||||
"action": {"action", r.Action}, "reject-with": {"reject_with", r.RejectWith}, "state": {"ct_state", r.CtState},
|
||||
"status": {"ct_status", r.CtStatus}, "days": {"days", r.Days}, "conn-limit": {"conn_limit", r.ConnLimit},
|
||||
"log": {"log", r.Log}, "log-prefix": {"log_prefix", r.LogPrefix}, "raw": {"raw", r.Raw}, "comment": {"comment", r.Comment},
|
||||
"disabled": {"disabled", r.Disabled},
|
||||
"disabled": {"disabled", r.Disabled}, "group-v4": {"group_v4", r.GroupV4}, "group-v6": {"group_v6", r.GroupV6},
|
||||
"every-packet": {"every_packet", r.EveryPacket},
|
||||
} {
|
||||
if fl.Changed(flag) {
|
||||
p[field[0].(string)] = field[1]
|
||||
@@ -289,6 +410,9 @@ output (from it) or early (before connection tracking: cheap drops).
|
||||
if fl.Changed("limit") {
|
||||
p["limit"] = r.Limit
|
||||
}
|
||||
if fl.Changed("quota") {
|
||||
p["quota"] = r.Quota
|
||||
}
|
||||
return change("PATCH", objPath("rules", args[0]), p)
|
||||
},
|
||||
}
|
||||
@@ -316,6 +440,57 @@ output (from it) or early (before connection tracking: cheap drops).
|
||||
move.Flags().BoolVar(&last, "last", false, "last in its chain")
|
||||
move.Flags().StringVar(&chain, "chain", "", "to the end of another chain")
|
||||
cmd.AddCommand(move)
|
||||
var resetAddr, give string
|
||||
resetCmd := &cobra.Command{
|
||||
Use: "reset-quota ID [--address ADDR] [--give AMOUNT]", Short: "Start a rule's quota again now, or give some back until its next reset", Args: cobra.ExactArgs(1),
|
||||
Long: `Starts a rule's quota again now: all of it (every address's, for a
|
||||
per-address quota), or with --address one address's. --give takes an
|
||||
amount off what was used instead: extra allowance until the quota's next
|
||||
reset, without changing the rule. Done at once, not staged or on
|
||||
probation.
|
||||
|
||||
hc rule reset-quota 31
|
||||
hc rule reset-quota 31 --address 192.168.1.23
|
||||
hc rule reset-quota 31 --give 5gbytes --address 192.168.1.23`,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
in := api.QuotaReset{Address: resetAddr}
|
||||
if give != "" {
|
||||
bad := fmt.Errorf("--give %q: write it as 500mbytes or 5gbytes", give)
|
||||
m := amountRe.FindStringSubmatch(strings.TrimSpace(give))
|
||||
if m == nil {
|
||||
return bad
|
||||
}
|
||||
unit, ok := byteUnit(m[2])
|
||||
n, _ := strconv.ParseInt(m[1], 10, 64)
|
||||
if !ok || unit == "" || n < 1 {
|
||||
return bad
|
||||
}
|
||||
in.Give = n * model.ByteUnits[unit]
|
||||
}
|
||||
c, err := newClient()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := c.Do(ctx(), "POST", objPath("rules", args[0])+"/reset-quota", nil, in, nil); err != nil {
|
||||
return err
|
||||
}
|
||||
if !flagQuiet {
|
||||
who := ""
|
||||
if resetAddr != "" {
|
||||
who = " for " + resetAddr
|
||||
}
|
||||
if in.Give > 0 {
|
||||
fmt.Printf("gave back %s of the quota of rule %s%s, until its next reset\n", humanBytes(in.Give), args[0], who)
|
||||
} else {
|
||||
fmt.Printf("reset the quota of rule %s%s\n", args[0], who)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
},
|
||||
}
|
||||
resetCmd.Flags().StringVar(&resetAddr, "address", "", "only this address's (a per-address quota; an IPv6 address counts for its /64)")
|
||||
resetCmd.Flags().StringVar(&give, "give", "", "take this much off what was used instead (5gbytes, 500mbytes)")
|
||||
cmd.AddCommand(resetCmd)
|
||||
for _, on := range []bool{true, false} {
|
||||
verb, short := "disable", "Keep a rule but don't use it"
|
||||
if on {
|
||||
@@ -345,18 +520,47 @@ func ruleExtras(r model.Rule) string {
|
||||
x = append(x, r.HourFrom+"-"+r.HourTo)
|
||||
}
|
||||
if l := r.Limit; l != nil {
|
||||
s := fmt.Sprintf("limit %d/%s", l.Rate, l.Per)
|
||||
unit := ""
|
||||
if l.Unit != "" && l.Unit != "packets" {
|
||||
unit = " " + l.Unit
|
||||
}
|
||||
s := fmt.Sprintf("limit %d%s/%s", l.Rate, unit, l.Per)
|
||||
if l.Over {
|
||||
s = "over " + s
|
||||
}
|
||||
if l.PerSource {
|
||||
s += " per source"
|
||||
}
|
||||
if l.PerDestination {
|
||||
s += " per destination"
|
||||
}
|
||||
x = append(x, s)
|
||||
}
|
||||
if q := r.Quota; q != nil {
|
||||
s := "quota " + humanBytes(q.Bytes)
|
||||
if q.Over {
|
||||
s = "over " + s
|
||||
}
|
||||
if q.PerSource {
|
||||
s += " per source"
|
||||
}
|
||||
if q.PerDestination {
|
||||
s += " per destination"
|
||||
}
|
||||
switch {
|
||||
case q.Reset == "monthly" && q.Day > 1:
|
||||
s += fmt.Sprintf(" monthly (%d)", q.Day)
|
||||
case q.Reset != "":
|
||||
s += " " + q.Reset
|
||||
}
|
||||
x = append(x, s)
|
||||
}
|
||||
if r.ConnLimit > 0 {
|
||||
x = append(x, fmt.Sprintf("over %d connections", r.ConnLimit))
|
||||
}
|
||||
if r.EveryPacket {
|
||||
x = append(x, "every packet")
|
||||
}
|
||||
if len(x) == 0 {
|
||||
return ""
|
||||
}
|
||||
@@ -412,9 +616,9 @@ through, and with --hairpin makes the public address work from inside too.
|
||||
--to6 also opens the port to an IPv6 host (no translation there: its own
|
||||
port).
|
||||
|
||||
hc forward add 8096 --to 10.0.10.57:443 --to6 2a02:8013:7f3d:10::57 --public 217.155.8.194 --hairpin --name jellyfin
|
||||
hc forward add 7777-7780 --proto tcp,udp --to 10.0.10.76 --public 217.155.8.194 --hairpin --name terraria
|
||||
hc forward add 25 --to 10.0.10.25 --from trusted`,
|
||||
hc forward add 8443 --to 192.168.1.10:443 --to6 2001:db8:1::10 --public 203.0.113.10 --hairpin --name web
|
||||
hc forward add 27015-27030 --proto tcp,udp --to 192.168.1.20 --public 203.0.113.10 --hairpin --name game
|
||||
hc forward add 25 --to 192.168.1.25 --from trusted`,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
pf.Ports = args[0]
|
||||
if to != "" {
|
||||
@@ -514,7 +718,7 @@ func snatCmd() *cobra.Command {
|
||||
if to == "" {
|
||||
to = "masquerade"
|
||||
}
|
||||
fmt.Fprintf(w, "%d\t%s\t%s\t%s\t%s\n", x.ID, describe(x.Match), to, hits(cs, fmt.Sprintf("s%d", x.ID)), x.Comment)
|
||||
fmt.Fprintf(w, "%d\t%s\t%s\t%s\t%s\n", x.ID, describe(x.Match, ""), to, hits(cs, fmt.Sprintf("s%d", x.ID)), x.Comment)
|
||||
}
|
||||
})
|
||||
},
|
||||
@@ -528,7 +732,7 @@ traffic leaving the firewall. IPv4 only, unless --family ipv6 (NAT66).
|
||||
|
||||
hc snat add --out wan --from trusted
|
||||
hc snat add --out wg --not-out --from wg_net
|
||||
hc snat add --out wan --from fdab:abab:abab::/48 --family ipv6 --snat-to 2a02:8013:7f3d::1`,
|
||||
hc snat add --out wan --from fd00:1::/48 --family ipv6 --snat-to 2001:db8::1`,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
return change("POST", "/snats", model.SourceNAT{Match: mf.match(), To: to, Comment: comment})
|
||||
},
|
||||
@@ -594,7 +798,7 @@ func redirectCmd() *cobra.Command {
|
||||
return output(xs, func(w io.Writer) {
|
||||
fmt.Fprintln(w, "ID\tMATCH\tTO PORT\tHITS\tCOMMENT")
|
||||
for _, x := range xs {
|
||||
fmt.Fprintf(w, "%d\t%s\t%s\t%s\t%s\n", x.ID, describe(x.Match), orDash(x.ToPort), hits(cs, fmt.Sprintf("d%d", x.ID)), x.Comment)
|
||||
fmt.Fprintf(w, "%d\t%s\t%s\t%s\t%s\n", x.ID, describe(x.Match, ""), orDash(x.ToPort), hits(cs, fmt.Sprintf("d%d", x.ID)), x.Comment)
|
||||
}
|
||||
})
|
||||
},
|
||||
@@ -606,7 +810,7 @@ func redirectCmd() *cobra.Command {
|
||||
Long: `Sends matching traffic to a port on the firewall itself, so every client
|
||||
uses its DNS server, say:
|
||||
|
||||
hc redirect add --in lan --service dns --not-to 10.0.10.5 --comment "force DNS"`,
|
||||
hc redirect add --in lan --service dns --not-to 192.168.1.5 --comment "force DNS"`,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
return change("POST", "/redirects", model.Redirect{Match: mf.match(), ToPort: toPort, Comment: comment})
|
||||
},
|
||||
|
||||
+417
@@ -0,0 +1,417 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/spf13/cobra"
|
||||
)
|
||||
|
||||
// Statistics: the figures housecarld keeps over time (docs/stats.md). The
|
||||
// types mirror internal/stats's JSON, so hc doesn't link the database.
|
||||
|
||||
type statEntry struct {
|
||||
Key string `json:"key"`
|
||||
Count int64 `json:"count"`
|
||||
Name string `json:"name,omitempty"`
|
||||
}
|
||||
|
||||
type statTraffic struct {
|
||||
Name string `json:"name"`
|
||||
Received int64 `json:"received"`
|
||||
Sent int64 `json:"sent"`
|
||||
}
|
||||
|
||||
type statSummary struct {
|
||||
Since string `json:"since"`
|
||||
From time.Time `json:"from"`
|
||||
Zones []statTraffic `json:"zones"`
|
||||
Interfaces []statTraffic `json:"interfaces"`
|
||||
Drops int64 `json:"drops"`
|
||||
DropsByChain []statEntry `json:"drops_by_chain"`
|
||||
Rules []statEntry `json:"rules"`
|
||||
PeakConnections int64 `json:"peak_connections"`
|
||||
ConntrackMax int64 `json:"conntrack_max"`
|
||||
BlockedSources []statEntry `json:"blocked_sources"`
|
||||
BlockedPorts []statEntry `json:"blocked_ports"`
|
||||
DevicesUp []statEntry `json:"devices_up"`
|
||||
DevicesDown []statEntry `json:"devices_down"`
|
||||
}
|
||||
|
||||
type statTimeline struct {
|
||||
Step string `json:"step"`
|
||||
Times []int64 `json:"times"`
|
||||
Series []struct {
|
||||
Key string `json:"key"`
|
||||
Values []int64 `json:"values"`
|
||||
} `json:"series"`
|
||||
}
|
||||
|
||||
type statStatus struct {
|
||||
Enabled bool `json:"enabled"`
|
||||
Database string `json:"database"`
|
||||
Size int64 `json:"size"`
|
||||
LastSample *time.Time `json:"last_sample"`
|
||||
SampleError string `json:"sample_error"`
|
||||
FirstData *time.Time `json:"first_data"`
|
||||
PerDevice bool `json:"per_device"`
|
||||
Retention struct {
|
||||
Minute, Hour, Day string
|
||||
} `json:"retention"`
|
||||
Latest *struct {
|
||||
Conntrack int64 `json:"conntrack"`
|
||||
ConntrackMax int64 `json:"conntrack_max"`
|
||||
RSS int64 `json:"rss"`
|
||||
CPUPermille int64 `json:"cpu_permille"`
|
||||
MemUsed int64 `json:"mem_used"`
|
||||
Load float64 `json:"load"`
|
||||
} `json:"latest"`
|
||||
}
|
||||
|
||||
func statsCmd() *cobra.Command {
|
||||
var since string
|
||||
var limit int
|
||||
cmd := &cobra.Command{
|
||||
Use: "stats", Short: "Traffic, drops, blocked sources and devices over time",
|
||||
Long: `Sums up what the firewall did over a while (--since: 90m, 24h, 7d, 4w):
|
||||
traffic per zone, packets dropped, the busiest rules, the most connections
|
||||
at once, the sources blocked most and what they tried, and (with traffic
|
||||
per device on) the devices that used the most.
|
||||
|
||||
hc stats --since 7d
|
||||
hc stats traffic --since 24h
|
||||
hc stats top devices --since 30d
|
||||
hc stats rules --unused`,
|
||||
Args: cobra.NoArgs,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
s, err := get[statSummary]("/stats/summary", q("since", since, "limit", strconv.Itoa(limit)))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return output(s, func(w io.Writer) { printSummary(w, s) })
|
||||
},
|
||||
}
|
||||
cmd.Flags().StringVar(&since, "since", "24h", "the last 90m, 24h, 7d, 4w…")
|
||||
cmd.Flags().IntVar(&limit, "limit", 5, "entries per top list")
|
||||
cmd.AddCommand(statsStatusCmd(), statsTrafficCmd(), statsTopCmd(), statsRulesCmd(), statsConnectionsCmd())
|
||||
return cmd
|
||||
}
|
||||
|
||||
func printSummary(w io.Writer, s statSummary) {
|
||||
fmt.Fprintf(w, "Since %s (%s)\n\n", s.From.Local().Format("2006-01-02 15:04"), s.Since)
|
||||
if len(s.Zones) > 0 {
|
||||
fmt.Fprintln(w, "ZONE\tRECEIVED\tSENT")
|
||||
for _, t := range s.Zones {
|
||||
fmt.Fprintf(w, "%s\t%s\t%s\n", t.Name, humanBytes(t.Received), humanBytes(t.Sent))
|
||||
}
|
||||
fmt.Fprintln(w)
|
||||
}
|
||||
var chains []string
|
||||
for _, e := range s.DropsByChain {
|
||||
chains = append(chains, fmt.Sprintf("%s %s", e.Key, humanCount(uint64(e.Count))))
|
||||
}
|
||||
drops := humanCount(uint64(s.Drops))
|
||||
if len(chains) > 0 {
|
||||
drops += " (" + strings.Join(chains, ", ") + ")"
|
||||
}
|
||||
fmt.Fprintf(w, "Dropped:\t%s packets\n", drops)
|
||||
if s.ConntrackMax > 0 {
|
||||
fmt.Fprintf(w, "Connections:\tat most %d at once (of %d)\n", s.PeakConnections, s.ConntrackMax)
|
||||
}
|
||||
list := func(title string, es []statEntry, unit func(int64) string) {
|
||||
if len(es) == 0 {
|
||||
return
|
||||
}
|
||||
fmt.Fprintf(w, "\n%s\n", title)
|
||||
for _, e := range es {
|
||||
name := e.Key
|
||||
if e.Name != "" {
|
||||
name = e.Name
|
||||
}
|
||||
fmt.Fprintf(w, " %s\t%s\n", name, unit(e.Count))
|
||||
}
|
||||
}
|
||||
packets := func(n int64) string { return humanCount(uint64(n)) + " packets" }
|
||||
list("Busiest rules:", s.Rules, packets)
|
||||
list("Blocked most (logged drops):", s.BlockedSources, packets)
|
||||
list("What they tried:", s.BlockedPorts, packets)
|
||||
list("Devices, sent:", s.DevicesUp, humanBytes)
|
||||
list("Devices, received:", s.DevicesDown, humanBytes)
|
||||
}
|
||||
|
||||
func statsStatusCmd() *cobra.Command {
|
||||
return &cobra.Command{
|
||||
Use: "status", Short: "Whether statistics are collected, since when, and the database", Args: cobra.NoArgs,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
s, err := get[statStatus]("/stats/status", nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return output(s, func(w io.Writer) {
|
||||
if !s.Enabled {
|
||||
fmt.Fprintln(w, "Statistics are off (stats.enabled in housecarl.yaml).")
|
||||
return
|
||||
}
|
||||
fmt.Fprintf(w, "Collecting:\tyes\n")
|
||||
if s.Database != "" {
|
||||
fmt.Fprintf(w, "Database:\t%s (%s)\n", s.Database, humanBytes(s.Size))
|
||||
} else {
|
||||
fmt.Fprintf(w, "Database size:\t%s\n", humanBytes(s.Size))
|
||||
}
|
||||
if s.FirstData != nil {
|
||||
fmt.Fprintf(w, "Since:\t%s\n", s.FirstData.Local().Format("2006-01-02 15:04"))
|
||||
}
|
||||
if s.LastSample != nil {
|
||||
fmt.Fprintf(w, "Last sample:\t%s\n", s.LastSample.Local().Format("15:04:05"))
|
||||
}
|
||||
if s.SampleError != "" {
|
||||
fmt.Fprintf(w, "Error:\t%s\n", s.SampleError)
|
||||
}
|
||||
fmt.Fprintf(w, "Kept:\tper minute %s, hourly %s, daily %s\n", s.Retention.Minute, s.Retention.Hour, s.Retention.Day)
|
||||
fmt.Fprintf(w, "Per device:\t%s\n", map[bool]string{true: "on", false: "off (hc settings set --stats-per-device on)"}[s.PerDevice])
|
||||
if l := s.Latest; l != nil {
|
||||
if l.ConntrackMax > 0 {
|
||||
fmt.Fprintf(w, "Connections now:\t%d of %d\n", l.Conntrack, l.ConntrackMax)
|
||||
}
|
||||
fmt.Fprintf(w, "housecarld:\t%s, CPU %.1f%%\n", humanBytes(l.RSS), float64(l.CPUPermille)/10)
|
||||
fmt.Fprintf(w, "Host:\t%s in use, load %.2f\n", humanBytes(l.MemUsed), l.Load)
|
||||
}
|
||||
})
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func statsTrafficCmd() *cobra.Command {
|
||||
var since, step string
|
||||
var ifaces bool
|
||||
cmd := &cobra.Command{
|
||||
Use: "traffic", Short: "Bytes received from and sent to each zone (or interface) over time", Args: cobra.NoArgs,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
rx, tx := "zrx", "ztx"
|
||||
if ifaces {
|
||||
rx, tx = "rx", "tx"
|
||||
}
|
||||
tl, err := get[statTimeline]("/stats/timeline", q("since", since, "step", step, "keys", rx+":*,"+tx+":*"))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return output(tl, func(w io.Writer) { printTimeline(w, tl, rx, tx) })
|
||||
},
|
||||
}
|
||||
cmd.Flags().StringVar(&since, "since", "24h", "the last 90m, 24h, 7d…")
|
||||
cmd.Flags().StringVar(&step, "step", "auto", "per 1m, 5m, 15m, 1h, 6h or 1d")
|
||||
cmd.Flags().BoolVar(&ifaces, "interfaces", false, "per interface instead of per zone")
|
||||
return cmd
|
||||
}
|
||||
|
||||
// printTimeline writes a row per bucket: each name's received and sent.
|
||||
func printTimeline(w io.Writer, tl statTimeline, rx, tx string) {
|
||||
var names []string
|
||||
vals := map[string][]int64{}
|
||||
for _, s := range tl.Series {
|
||||
vals[s.Key] = s.Values
|
||||
if n, ok := strings.CutPrefix(s.Key, rx+":"); ok {
|
||||
names = append(names, n)
|
||||
}
|
||||
}
|
||||
for _, s := range tl.Series {
|
||||
if n, ok := strings.CutPrefix(s.Key, tx+":"); ok && vals[rx+":"+n] == nil {
|
||||
names = append(names, n)
|
||||
}
|
||||
}
|
||||
if len(names) == 0 {
|
||||
fmt.Fprintln(w, "no traffic recorded yet")
|
||||
return
|
||||
}
|
||||
fmt.Fprint(w, "TIME")
|
||||
for _, n := range names {
|
||||
fmt.Fprintf(w, "\t%s ↓\t%s ↑", n, n)
|
||||
}
|
||||
fmt.Fprintln(w)
|
||||
layout := "01-02 15:04"
|
||||
if tl.Step == "1d" {
|
||||
layout = "2006-01-02"
|
||||
}
|
||||
at := func(k string, i int) int64 {
|
||||
if v := vals[k]; i < len(v) {
|
||||
return v[i]
|
||||
}
|
||||
return 0
|
||||
}
|
||||
for i, t := range tl.Times {
|
||||
fmt.Fprint(w, time.Unix(t, 0).Local().Format(layout))
|
||||
for _, n := range names {
|
||||
fmt.Fprintf(w, "\t%s\t%s", humanBytes(at(rx+":"+n, i)), humanBytes(at(tx+":"+n, i)))
|
||||
}
|
||||
fmt.Fprintln(w)
|
||||
}
|
||||
}
|
||||
|
||||
func statsTopCmd() *cobra.Command {
|
||||
var since string
|
||||
var limit int
|
||||
cmd := &cobra.Command{
|
||||
Use: "top devices|blocked|tried|quota RULE-ID", Short: "The largest: devices by traffic, sources blocked, what they tried, a quota's users",
|
||||
Long: `Top lists over the last while:
|
||||
|
||||
devices inside addresses by traffic to and from external zones (needs
|
||||
hc settings set --stats-per-device on)
|
||||
blocked sources of logged drops (the policies' drops with 'log drops'
|
||||
on, and rules that log)
|
||||
tried what those drops tried: tcp/22, udp/53, icmp/echo-request
|
||||
quota ID who used a per-address quota of rule ID`,
|
||||
Args: cobra.RangeArgs(1, 2),
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
get1 := func(kind string) ([]statEntry, error) {
|
||||
return get[[]statEntry]("/stats/top", q("kind", kind, "since", since, "limit", strconv.Itoa(limit)))
|
||||
}
|
||||
switch args[0] {
|
||||
case "devices":
|
||||
up, err := get1("device_up")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
down, err := get1("device_down")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
out := map[string][2]int64{}
|
||||
var order []string
|
||||
for _, l := range [][]statEntry{down, up} {
|
||||
for _, e := range l {
|
||||
if _, ok := out[e.Key]; !ok {
|
||||
out[e.Key] = [2]int64{}
|
||||
order = append(order, e.Key)
|
||||
}
|
||||
}
|
||||
}
|
||||
for _, e := range down {
|
||||
v := out[e.Key]
|
||||
v[0] = e.Count
|
||||
out[e.Key] = v
|
||||
}
|
||||
for _, e := range up {
|
||||
v := out[e.Key]
|
||||
v[1] = e.Count
|
||||
out[e.Key] = v
|
||||
}
|
||||
names := lookupNames(order...)
|
||||
return output(map[string]any{"up": up, "down": down}, func(w io.Writer) {
|
||||
fmt.Fprintln(w, "DEVICE\tRECEIVED\tSENT")
|
||||
for _, k := range order {
|
||||
fmt.Fprintf(w, "%s\t%s\t%s\n", named(names, k, k), humanBytes(out[k][0]), humanBytes(out[k][1]))
|
||||
}
|
||||
})
|
||||
case "blocked", "tried":
|
||||
kind := map[string]string{"blocked": "blocked_src", "tried": "blocked_port"}[args[0]]
|
||||
es, err := get1(kind)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var names map[string]string
|
||||
if args[0] == "blocked" {
|
||||
names = lookupNames(keys(es)...)
|
||||
}
|
||||
return output(es, func(w io.Writer) {
|
||||
fmt.Fprintln(w, map[string]string{"blocked": "SOURCE", "tried": "WHAT"}[args[0]]+"\tPACKETS")
|
||||
for _, e := range es {
|
||||
fmt.Fprintf(w, "%s\t%d\n", named(names, e.Key, e.Key), e.Count)
|
||||
}
|
||||
})
|
||||
case "quota":
|
||||
if len(args) != 2 {
|
||||
return fmt.Errorf("give the rule's ID: hc stats top quota 12")
|
||||
}
|
||||
es, err := get1("quota_r" + args[1])
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
names := lookupNames(keys(es)...)
|
||||
return output(es, func(w io.Writer) {
|
||||
fmt.Fprintln(w, "ADDRESS\tUSED")
|
||||
for _, e := range es {
|
||||
fmt.Fprintf(w, "%s\t%s\n", named(names, e.Key, e.Key), humanBytes(e.Count))
|
||||
}
|
||||
})
|
||||
}
|
||||
return fmt.Errorf("unknown list %q: devices, blocked, tried or quota", args[0])
|
||||
},
|
||||
}
|
||||
cmd.Flags().StringVar(&since, "since", "24h", "the last 90m, 24h, 7d…")
|
||||
cmd.Flags().IntVar(&limit, "limit", 20, "at most this many")
|
||||
return cmd
|
||||
}
|
||||
|
||||
// keys are a top list's keys.
|
||||
func keys(es []statEntry) []string {
|
||||
var out []string
|
||||
for _, e := range es {
|
||||
out = append(out, e.Key)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func statsRulesCmd() *cobra.Command {
|
||||
var since string
|
||||
var unused bool
|
||||
cmd := &cobra.Command{
|
||||
Use: "rules", Short: "Packets each rule, built-in and policy counted; --unused: rules that matched nothing", Args: cobra.NoArgs,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
es, err := get[[]statEntry]("/stats/hits", q("since", since))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if unused {
|
||||
var keep []statEntry
|
||||
for _, e := range es {
|
||||
if e.Count == 0 && strings.HasPrefix(e.Key, "r") {
|
||||
keep = append(keep, e)
|
||||
}
|
||||
}
|
||||
es = keep
|
||||
}
|
||||
return output(es, func(w io.Writer) {
|
||||
if unused && len(es) == 0 {
|
||||
fmt.Fprintf(w, "every rule matched something in the last %s\n", since)
|
||||
return
|
||||
}
|
||||
fmt.Fprintln(w, "WHAT\tPACKETS")
|
||||
for _, e := range es {
|
||||
fmt.Fprintf(w, "%s\t%d\n", orDash(e.Name), e.Count)
|
||||
}
|
||||
})
|
||||
},
|
||||
}
|
||||
cmd.Flags().StringVar(&since, "since", "30d", "the last 24h, 7d, 30d…")
|
||||
cmd.Flags().BoolVar(&unused, "unused", false, "only the rules that matched nothing")
|
||||
return cmd
|
||||
}
|
||||
|
||||
func statsConnectionsCmd() *cobra.Command {
|
||||
var since, step string
|
||||
cmd := &cobra.Command{
|
||||
Use: "connections", Short: "How many connections the firewall tracked over time, against the limit", Args: cobra.NoArgs,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
tl, err := get[statTimeline]("/stats/timeline", q("since", since, "step", step, "keys", "g:conntrack,g:conntrack_max"))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return output(tl, func(w io.Writer) {
|
||||
fmt.Fprintln(w, "TIME\tCONNECTIONS\tLIMIT")
|
||||
for i, t := range tl.Times {
|
||||
n, m := tl.Series[0].Values[i], tl.Series[1].Values[i]
|
||||
if n < 0 {
|
||||
continue // no sample
|
||||
}
|
||||
fmt.Fprintf(w, "%s\t%d\t%d\n", time.Unix(t, 0).Local().Format("01-02 15:04"), n, m)
|
||||
}
|
||||
})
|
||||
},
|
||||
}
|
||||
cmd.Flags().StringVar(&since, "since", "24h", "the last 90m, 24h, 7d…")
|
||||
cmd.Flags().StringVar(&step, "step", "auto", "per 1m, 5m, 15m, 1h, 6h or 1d")
|
||||
return cmd
|
||||
}
|
||||
@@ -0,0 +1,367 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"os/signal"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode/utf8"
|
||||
|
||||
"github.com/spf13/cobra"
|
||||
|
||||
"housecarl/internal/api"
|
||||
)
|
||||
|
||||
// Traffic as it happens: connections the kernel tracks, and tracing
|
||||
// packets through the ruleset.
|
||||
|
||||
func connectionsCmd() *cobra.Command {
|
||||
var search string
|
||||
var limit int
|
||||
cmd := &cobra.Command{
|
||||
Use: "connections", Aliases: []string{"conns"}, Short: "The connections the firewall tracks, and closing them",
|
||||
Long: `Lists the connections the kernel tracks: what the "established" built-in
|
||||
lets through without checking the rules again. A rule added later doesn't
|
||||
stop them by itself; close them, and their next packet is checked by the
|
||||
rules as it stands.
|
||||
|
||||
hc connections --search 192.168.1.50
|
||||
hc connections --search ":443 tcp"
|
||||
hc connections close --search 192.168.1.50`,
|
||||
Args: cobra.NoArgs,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
cs, err := get[[]api.Connection]("/connections", q("q", search, "limit", strconv.Itoa(limit)))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var addrs []string
|
||||
for _, c := range cs {
|
||||
addrs = append(addrs, c.Source, c.Destination)
|
||||
}
|
||||
names := lookupNames(addrs...)
|
||||
return output(cs, func(w io.Writer) {
|
||||
fmt.Fprintln(w, "PROTO\tFROM\tTO\tNAT\tSTATE\tEXPIRES")
|
||||
for _, c := range cs {
|
||||
fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\t%s\n", c.Protocol, named(names, c.Source, hostPort(c.Source, c.SourcePort)),
|
||||
named(names, c.Destination, hostPort(c.Destination, c.DestinationPort)), natText(c), c.State, fmtSeconds(c.Timeout))
|
||||
}
|
||||
})
|
||||
},
|
||||
}
|
||||
cmd.Flags().StringVar(&search, "search", "", "only those with all these words: addresses, :port, protocol, state")
|
||||
cmd.Flags().IntVar(&limit, "limit", 500, "at most this many")
|
||||
var csearch string
|
||||
var yes bool
|
||||
closeCmd := &cobra.Command{
|
||||
Use: "close --search TEXT", Short: "Close connections: the rules check them again with their next packet", Args: cobra.NoArgs,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
if strings.TrimSpace(csearch) == "" {
|
||||
return fmt.Errorf("say which: --search 192.168.1.50 (see 'hc connections --search' first)")
|
||||
}
|
||||
if !confirm("Close every connection matching "+strconvQuote(csearch)+"?", yes) {
|
||||
return fmt.Errorf("nothing closed")
|
||||
}
|
||||
c, err := newClient()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var out map[string]int
|
||||
if err := c.Do(ctx(), "POST", "/connections/close", nil, api.CloseConnections{Search: csearch}, &out); err != nil {
|
||||
return err
|
||||
}
|
||||
if !flagQuiet {
|
||||
fmt.Printf("closed %s\n", plural(out["closed"], "connection"))
|
||||
}
|
||||
return nil
|
||||
},
|
||||
}
|
||||
closeCmd.Flags().StringVar(&csearch, "search", "", "the connections to close, as for 'hc connections --search'")
|
||||
closeCmd.Flags().BoolVarP(&yes, "yes", "y", false, "don't ask")
|
||||
cmd.AddCommand(closeCmd)
|
||||
return cmd
|
||||
}
|
||||
|
||||
func strconvQuote(s string) string { return strconv.Quote(s) }
|
||||
|
||||
func hostPort(addr string, port int) string {
|
||||
if port == 0 {
|
||||
return addr
|
||||
}
|
||||
if strings.Contains(addr, ":") {
|
||||
return fmt.Sprintf("[%s]:%d", addr, port)
|
||||
}
|
||||
return fmt.Sprintf("%s:%d", addr, port)
|
||||
}
|
||||
|
||||
// natText: "to 192.168.1.10:443" (a port forward), "as 203.0.113.10"
|
||||
// (masqueraded).
|
||||
func natText(c api.Connection) string {
|
||||
var parts []string
|
||||
if c.DNATTo != "" {
|
||||
port := c.DNATPort
|
||||
if port == 0 {
|
||||
port = c.DestinationPort
|
||||
}
|
||||
parts = append(parts, "to "+hostPort(c.DNATTo, port))
|
||||
}
|
||||
if c.SNATTo != "" {
|
||||
parts = append(parts, "as "+c.SNATTo)
|
||||
}
|
||||
return strings.Join(parts, ", ")
|
||||
}
|
||||
|
||||
func traceCmd() *cobra.Command {
|
||||
var in api.TraceInput
|
||||
cmd := &cobra.Command{
|
||||
Use: "trace", Short: "Follow packets through every table's rules: what each met, and what decided",
|
||||
Long: `Traces packets from or to an address, on a port (either way), through the
|
||||
whole ruleset, Housecarl's and every other table's, for a while: each
|
||||
rule and policy they meet, and what accepted or dropped them. Ctrl-C
|
||||
stops it.
|
||||
|
||||
hc trace --address 192.168.1.50 --port 443
|
||||
hc trace --port 22 --seconds 120
|
||||
hc trace --address 2001:db8:1::/64 --protocol icmpv6`,
|
||||
Args: cobra.NoArgs,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
c, err := newClient()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var st api.TraceStatus
|
||||
if err := c.Do(ctx(), "POST", "/trace", nil, in, &st); err != nil {
|
||||
return err
|
||||
}
|
||||
if !flagQuiet {
|
||||
fmt.Fprintf(os.Stderr, "tracing until %s (Ctrl-C stops)\n", st.Ends.Local().Format("15:04:05"))
|
||||
}
|
||||
sig, stop := signal.NotifyContext(context.Background(), os.Interrupt)
|
||||
defer stop()
|
||||
shown := 0
|
||||
for {
|
||||
select {
|
||||
case <-sig.Done():
|
||||
c.Do(ctx(), "DELETE", "/trace", nil, nil, &st)
|
||||
case <-time.After(time.Second):
|
||||
if err := c.Do(ctx(), "GET", "/trace", nil, nil, &st); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
for ; shown < len(st.Packets); shown++ {
|
||||
printPacket(os.Stdout, st.Packets[shown])
|
||||
}
|
||||
if !st.Running || sig.Err() != nil {
|
||||
break
|
||||
}
|
||||
}
|
||||
if st.Error != "" {
|
||||
return fmt.Errorf("the trace failed: %s", st.Error)
|
||||
}
|
||||
if !flagQuiet {
|
||||
fmt.Fprintf(os.Stderr, "%d packets traced\n", len(st.Packets))
|
||||
}
|
||||
return nil
|
||||
},
|
||||
}
|
||||
f := cmd.Flags()
|
||||
f.StringVar(&in.Address, "address", "", "packets from or to this address or prefix")
|
||||
f.IntVar(&in.Port, "port", 0, "packets from or to this port")
|
||||
f.StringVar(&in.Protocol, "protocol", "", "tcp, udp, sctp, icmp or icmpv6 (default any)")
|
||||
f.IntVar(&in.Seconds, "seconds", 60, "how long (5 to 600)")
|
||||
return cmd
|
||||
}
|
||||
|
||||
// printPacket writes a traced packet: its headers, each step, its fate.
|
||||
func printPacket(w io.Writer, p api.TracePacket) {
|
||||
what := p.Protocol
|
||||
if p.TCPFlags != "" {
|
||||
what += " " + p.TCPFlags
|
||||
}
|
||||
if p.ICMPType != "" {
|
||||
what += " " + p.ICMPType
|
||||
}
|
||||
where := ""
|
||||
if p.In != "" {
|
||||
where += " in " + p.In
|
||||
}
|
||||
if p.Out != "" {
|
||||
where += " out " + p.Out
|
||||
}
|
||||
fmt.Fprintf(w, "%s %s -> %s %s%s\n", p.Time.Local().Format("15:04:05.000"), hostPort(p.Source, p.SourcePort), hostPort(p.Destination, p.DestinationPort), what, where)
|
||||
for _, s := range p.Steps {
|
||||
name := s.Label
|
||||
if name == "" {
|
||||
if s.Kind == "policy" {
|
||||
name = s.Chain + " policy"
|
||||
} else {
|
||||
name = s.Rule
|
||||
}
|
||||
}
|
||||
fmt.Fprintf(w, " %s %s: %s -> %s\n", s.Table, s.Chain, name, s.Verdict)
|
||||
}
|
||||
decided := p.DecidedBy.Label
|
||||
if decided == "" {
|
||||
decided = strings.TrimSpace(p.DecidedBy.Table + " " + p.DecidedBy.Chain + " " + p.DecidedBy.Kind)
|
||||
}
|
||||
if decided == "" {
|
||||
fmt.Fprintf(w, " => %s\n", strings.ToUpper(p.Verdict))
|
||||
} else {
|
||||
fmt.Fprintf(w, " => %s by %s\n", strings.ToUpper(p.Verdict), decided)
|
||||
}
|
||||
if wd := p.WouldDrop; wd != nil {
|
||||
name := wd.Label
|
||||
if name == "" {
|
||||
name = strings.TrimSpace(wd.Chain + " " + wd.Mark)
|
||||
}
|
||||
fmt.Fprintf(w, " => shadow mode: Housecarl would have dropped it, by %s\n", name)
|
||||
}
|
||||
}
|
||||
|
||||
func matrixCmd() *cobra.Command {
|
||||
return &cobra.Command{
|
||||
Use: "matrix [FROM TO]", Short: "What the rules do to new connections between zones",
|
||||
Long: `Shows, for each pair of zones (and the firewall itself), what happens to a
|
||||
new connection: allowed or blocked, decided by a rule or the policy, and
|
||||
how many rules decide for part of the traffic first (+N: exceptions,
|
||||
such as some services or addresses). With FROM and TO (a zone, or
|
||||
"firewall"), that pair's exceptions in order, and what decides the rest.
|
||||
|
||||
hc matrix
|
||||
hc matrix guest wan
|
||||
hc matrix lan firewall`,
|
||||
Args: cobra.RangeArgs(0, 2),
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
if len(args) == 1 {
|
||||
return fmt.Errorf("give both: FROM and TO")
|
||||
}
|
||||
m, err := get[api.Matrix]("/matrix", nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if len(args) == 2 {
|
||||
for _, c := range m.Cells {
|
||||
if strings.EqualFold(c.From, args[0]) && strings.EqualFold(c.To, args[1]) {
|
||||
return output(c, func(w io.Writer) { printCell(w, c) })
|
||||
}
|
||||
}
|
||||
return fmt.Errorf("no pair %s → %s (zones: %s, and firewall)", args[0], args[1], strings.Join(m.Zones, ", "))
|
||||
}
|
||||
return output(m, func(w io.Writer) { printMatrix(w, m) })
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func printMatrix(w io.Writer, m api.Matrix) {
|
||||
writeMatrix(w, m, isTerminal(os.Stdout) && flagOutput == "table")
|
||||
}
|
||||
|
||||
// writeMatrix writes the grid, in colour on a terminal.
|
||||
func writeMatrix(w io.Writer, m api.Matrix, color bool) {
|
||||
cols := append(append([]string{}, m.Zones...), "firewall")
|
||||
cells := map[[2]string]api.MatrixCell{}
|
||||
for _, c := range m.Cells {
|
||||
cells[[2]string{c.From, c.To}] = c
|
||||
}
|
||||
// The grid is padded by hand: tabwriter would count the colours'
|
||||
// escape codes as text, and misalign the columns.
|
||||
grid := [][]string{append([]string{"FROM \\ TO"}, cols...)}
|
||||
kinds := [][]string{make([]string, len(cols)+1)}
|
||||
same := false
|
||||
for _, from := range cols {
|
||||
row, kind := []string{from}, []string{""}
|
||||
for _, to := range cols {
|
||||
c, ok := cells[[2]string{from, to}]
|
||||
if !ok {
|
||||
row, kind = append(row, "-"), append(kind, "")
|
||||
continue
|
||||
}
|
||||
text := map[string]string{"allowed": "allow", "blocked": "block", "partly": "partly"}[c.Summary]
|
||||
if c.Allowed > 0 {
|
||||
text += fmt.Sprintf(" +%d", c.Allowed)
|
||||
}
|
||||
if c.Blocked > 0 {
|
||||
text += fmt.Sprintf(" -%d", c.Blocked)
|
||||
}
|
||||
if c.SameZone {
|
||||
text += " *"
|
||||
same = true
|
||||
}
|
||||
row, kind = append(row, text), append(kind, c.Summary)
|
||||
}
|
||||
grid, kinds = append(grid, row), append(kinds, kind)
|
||||
}
|
||||
width := make([]int, len(cols)+1)
|
||||
for _, row := range grid {
|
||||
for i, t := range row {
|
||||
width[i] = max(width[i], utf8.RuneCountInString(t))
|
||||
}
|
||||
}
|
||||
paint := map[string]string{"allowed": "\x1b[32m", "blocked": "\x1b[31m", "partly": "\x1b[33m"}
|
||||
for r, row := range grid {
|
||||
var b strings.Builder
|
||||
for i, t := range row {
|
||||
pad := strings.Repeat(" ", width[i]-utf8.RuneCountInString(t))
|
||||
if i == len(row)-1 {
|
||||
pad = ""
|
||||
} else {
|
||||
pad += " "
|
||||
}
|
||||
if p := paint[kinds[r][i]]; color && p != "" {
|
||||
t = p + t + "\x1b[0m"
|
||||
}
|
||||
b.WriteString(t + pad)
|
||||
}
|
||||
fmt.Fprintln(w, b.String())
|
||||
}
|
||||
// What the partly open pairs let through.
|
||||
var open [][2]string
|
||||
for _, from := range cols {
|
||||
for _, to := range cols {
|
||||
c, ok := cells[[2]string{from, to}]
|
||||
if !ok || len(c.Open) == 0 || c.SameZone {
|
||||
continue
|
||||
}
|
||||
items := c.Open
|
||||
more := ""
|
||||
if len(items) > 6 {
|
||||
items, more = items[:6], fmt.Sprintf("; %d more", len(c.Open)-6)
|
||||
}
|
||||
open = append(open, [2]string{from + " → " + to, strings.Join(items, "; ") + more})
|
||||
}
|
||||
}
|
||||
if len(open) > 0 {
|
||||
fmt.Fprintln(w, "\nLet through for part of the traffic:")
|
||||
for _, o := range open {
|
||||
fmt.Fprintf(w, " %s\t%s\n", o[0], o[1])
|
||||
}
|
||||
}
|
||||
fmt.Fprintln(w, "\nallow, block: every new connection; partly: some are let through, some not.")
|
||||
fmt.Fprintln(w, "+N: rules that let part of the traffic through; -N: rules that block part of it. 'hc matrix FROM TO' lists them.")
|
||||
if same {
|
||||
fmt.Fprintln(w, "*: a zone to itself: only traffic routed between its interfaces passes the firewall.")
|
||||
}
|
||||
}
|
||||
|
||||
func printCell(w io.Writer, c api.MatrixCell) {
|
||||
summary := map[string]string{"allowed": "all allowed", "blocked": "all blocked", "partly": "partly allowed"}[c.Summary]
|
||||
fmt.Fprintf(w, "%s → %s (the %s chain), new connections: %s\n", c.From, c.To, c.Chain, summary)
|
||||
if c.Note != "" {
|
||||
fmt.Fprintf(w, " (%s)\n", c.Note)
|
||||
}
|
||||
hits := func(h *uint64) string {
|
||||
if h == nil {
|
||||
return "-"
|
||||
}
|
||||
return fmt.Sprintf("%d", *h)
|
||||
}
|
||||
fmt.Fprintln(w, " VERDICT\tWHAT\tBY\tHITS")
|
||||
for _, e := range c.Exceptions {
|
||||
fmt.Fprintf(w, " %s\t%s\t%s\t%s\n", e.Verdict, e.What, e.Label, hits(e.Hits))
|
||||
}
|
||||
fmt.Fprintf(w, " %s\tthe rest\t%s\t%s\n", c.Verdict, c.DecidedBy, hits(c.Hits))
|
||||
fmt.Fprintln(w, "\nHits: each rule's packets since housecarld started, in all its traffic (a rule can cover several pairs).")
|
||||
}
|
||||
+109
-1
@@ -6,6 +6,7 @@ package main
|
||||
import (
|
||||
"bufio"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
@@ -25,15 +26,21 @@ import (
|
||||
"housecarl/internal/auth"
|
||||
"housecarl/internal/certs"
|
||||
"housecarl/internal/config"
|
||||
"housecarl/internal/conntrack"
|
||||
"housecarl/internal/engine"
|
||||
"housecarl/internal/hostif"
|
||||
"housecarl/internal/jarl"
|
||||
"housecarl/internal/linkwatch"
|
||||
"housecarl/internal/model"
|
||||
"housecarl/internal/names"
|
||||
"housecarl/internal/nflog"
|
||||
"housecarl/internal/nft"
|
||||
"housecarl/internal/render"
|
||||
"housecarl/internal/server"
|
||||
"housecarl/internal/stats"
|
||||
"housecarl/internal/store"
|
||||
"housecarl/internal/tablewatch"
|
||||
"housecarl/internal/trace"
|
||||
"housecarl/internal/validate"
|
||||
"housecarl/internal/version"
|
||||
"housecarl/internal/web"
|
||||
@@ -148,8 +155,13 @@ func serveCmd() *cobra.Command {
|
||||
}
|
||||
host := &hostif.Host{}
|
||||
eng.Env = hostEnv(host)
|
||||
fw := &nft.Firewall{Runner: &nft.Runner{Command: nft.Wrapper()}, Interfaces: host.Exists, BootFile: a.cfg.Ruleset}
|
||||
fw := &nft.Firewall{Runner: &nft.Runner{Command: nft.Wrapper()}, Interfaces: host.Exists, BootFile: a.cfg.Ruleset,
|
||||
Shadow: a.cfg.Shadow.Enabled, ShadowBefore: a.cfg.Shadow.Position == config.ShadowBefore,
|
||||
Bans: func() []render.Ban { return server.ActiveBans(context.Background(), a.store) }}
|
||||
eng.Firewall = fw
|
||||
if fw.Shadow {
|
||||
a.log.Warn("shadow mode: Housecarl's ruleset is loaded but blocks, rejects and translates nothing; the other firewall decides (docs/shadow.md)", "position", a.cfg.Shadow.Position)
|
||||
}
|
||||
for _, p := range eng.Problems() {
|
||||
a.log.Warn("configuration problem: "+p.Message, "object", p.Object, "code", p.Code)
|
||||
}
|
||||
@@ -166,15 +178,31 @@ func serveCmd() *cobra.Command {
|
||||
acme = certs.New(ctx, certs.Config{ACME: a.cfg.TLS.ACME, Version: version.Short()}, jc, a.store, a.log)
|
||||
go acme.Run(ctx)
|
||||
}
|
||||
var resolver *names.Resolver
|
||||
if a.cfg.Names.Enabled {
|
||||
resolver = names.New(a.cfg.Names.Resolver, time.Duration(a.cfg.Names.Cache))
|
||||
}
|
||||
srv := server.New(server.Config{
|
||||
Listen: a.cfg.Listen, TLSCertFile: a.cfg.TLS.CertFile, TLSKeyFile: a.cfg.TLS.KeyFile, TrustedProxies: a.cfg.Proxies(),
|
||||
SessionLifetime: time.Duration(a.cfg.Session.Lifetime), SessionIdle: time.Duration(a.cfg.Session.IdleTimeout),
|
||||
CookieSecure: a.cfg.Session.CookieSecure, Socket: a.cfg.Socket, ACME: acme,
|
||||
Shadow: a.cfg.Shadow.Enabled, ShadowPosition: a.cfg.Shadow.Position, Names: resolver,
|
||||
}, eng, fw, host, a.log, web.FS())
|
||||
// Put the ruleset in force: what was on probation when
|
||||
// housecarld stopped is rolled back first. An empty
|
||||
// configuration applies nothing: its default policies would
|
||||
// close the machine before any rule lets anyone in.
|
||||
// What the quotas used before housecarld (or the machine)
|
||||
// stopped comes back with the next apply.
|
||||
restoreQuotas := func() {
|
||||
if v, err := a.store.GetMeta(ctx, server.QuotaUsageKey); err == nil && v != "" {
|
||||
var u nft.QuotaUsage
|
||||
if json.Unmarshal([]byte(v), &u) == nil {
|
||||
fw.RestoreQuotas(&u)
|
||||
}
|
||||
}
|
||||
}
|
||||
restoreQuotas()
|
||||
if !srv.ResumeProbation(ctx) {
|
||||
if eng.State().Empty() {
|
||||
a.log.Info("the configuration is empty: no ruleset is applied until the first change (or 'hc import')")
|
||||
@@ -184,7 +212,53 @@ func serveCmd() *cobra.Command {
|
||||
a.log.Error("saving the ruleset for the next boot failed", "err", err)
|
||||
}
|
||||
}
|
||||
// A flowtable device that appears later (ppp0 once PPPoE is up,
|
||||
// or again after it reconnects) is added by applying again.
|
||||
go func() {
|
||||
err := linkwatch.Watch(ctx, func() {
|
||||
host.Forget()
|
||||
if !fw.FlowtableStale(eng.State()) {
|
||||
return
|
||||
}
|
||||
if err := eng.ApplyCurrent(ctx); err != nil {
|
||||
a.log.Error("applying the ruleset again for the flowtable's devices failed", "err", err)
|
||||
return
|
||||
}
|
||||
a.log.Info("interfaces changed: applied the ruleset again, with the flowtable's devices that exist now")
|
||||
})
|
||||
if err != nil {
|
||||
a.log.Warn("not watching the interfaces: a flowtable device that appears later is only added by the next change", "err", err)
|
||||
}
|
||||
}()
|
||||
// In shadow mode the other firewall's ruleset is reloaded now
|
||||
// and then, and a ruleset file usually starts with flush
|
||||
// ruleset, which deletes Housecarl's table too: put it back.
|
||||
if fw.Shadow {
|
||||
go func() {
|
||||
err := tablewatch.Watch(ctx, func() {
|
||||
if _, err := fw.Table(ctx); !errors.Is(err, nft.ErrNoTable) || eng.State().Empty() {
|
||||
return
|
||||
}
|
||||
// The quotas' usage went with the table: from
|
||||
// what was last saved.
|
||||
restoreQuotas()
|
||||
if err := eng.ApplyCurrent(ctx); err != nil {
|
||||
a.log.Error("Housecarl's table was deleted, and putting it back failed", "err", err)
|
||||
return
|
||||
}
|
||||
a.log.Info("Housecarl's table was deleted (flush ruleset?): put it back")
|
||||
})
|
||||
if err != nil {
|
||||
a.log.Warn("not watching nftables: if Housecarl's table is deleted, only the next change or a restart puts it back", "err", err)
|
||||
}
|
||||
}()
|
||||
}
|
||||
go nflog.Listen(ctx, render.LogGroup, a.log, srv.LogPacket)
|
||||
go srv.RunQuotas(ctx)
|
||||
srv.SetConntrack(conntrack.Client{})
|
||||
statsDone := startStats(ctx, a, fw, srv)
|
||||
// A trace left behind by a crash.
|
||||
fw.DeleteTable(ctx, "inet", trace.Table)
|
||||
hup := make(chan os.Signal, 1)
|
||||
signal.Notify(hup, syscall.SIGHUP)
|
||||
go func() {
|
||||
@@ -196,12 +270,46 @@ func serveCmd() *cobra.Command {
|
||||
}()
|
||||
err = srv.Serve(ctx, func() { sdNotify("READY=1") })
|
||||
sdNotify("STOPPING=1")
|
||||
srv.SaveQuotaUsage(context.Background())
|
||||
<-statsDone
|
||||
a.log.Info("stopped")
|
||||
return err
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// startStats opens the statistics database and runs the collector until
|
||||
// ctx ends; the channel closes once it wrote its last figures.
|
||||
func startStats(ctx context.Context, a *app, fw *nft.Firewall, srv *server.Server) <-chan struct{} {
|
||||
done := make(chan struct{})
|
||||
c := a.cfg.Stats
|
||||
if !c.Enabled {
|
||||
close(done)
|
||||
return done
|
||||
}
|
||||
st, err := stats.OpenStore(c.Database)
|
||||
if err != nil {
|
||||
a.log.Error("statistics are off: opening their database failed (deleting it is safe)", "err", err)
|
||||
close(done)
|
||||
return done
|
||||
}
|
||||
opts := stats.Options{TopN: c.TopN, RetainMinute: time.Duration(c.Retention.Minute), RetainHour: time.Duration(c.Retention.Hour), RetainDay: time.Duration(c.Retention.Day)}
|
||||
// Tests sample more often than every minute.
|
||||
if v := os.Getenv("HOUSECARL_STATS_INTERVAL"); v != "" {
|
||||
opts.Interval, _ = time.ParseDuration(v)
|
||||
}
|
||||
src := stats.Sources{Table: fw.Table}
|
||||
srv.StatsSources(&src)
|
||||
col := stats.NewCollector(st, opts, src, a.log)
|
||||
srv.SetStats(col)
|
||||
go func() {
|
||||
defer close(done)
|
||||
defer st.Close()
|
||||
col.Run(ctx)
|
||||
}()
|
||||
return done
|
||||
}
|
||||
|
||||
func sdNotify(state string) {
|
||||
sock := os.Getenv("NOTIFY_SOCKET")
|
||||
if sock == "" {
|
||||
|
||||
@@ -0,0 +1,52 @@
|
||||
# Housecarl's action for fail2ban: a ban goes into a Housecarl ban list,
|
||||
# and the list says what the banned address is kept from (the firewall
|
||||
# itself, some hosts behind it, or everything). See docs/bans.md.
|
||||
#
|
||||
# Copy this file to /etc/fail2ban/action.d/housecarl.conf, make a ban
|
||||
# list and a token for it alone on the firewall:
|
||||
#
|
||||
# hc banlist add web --scope hosts --host 10.0.10.57
|
||||
# hc user add f2b-web --no-password --grant bans:write:web
|
||||
# hc token create web --user f2b-web # as root on the firewall
|
||||
#
|
||||
# put the token in /etc/fail2ban/housecarl.token (mode 0600), and use it
|
||||
# in a jail:
|
||||
#
|
||||
# [sshd]
|
||||
# enabled = true
|
||||
# action = housecarl[list=web, url="https://router.lan:8070"]
|
||||
#
|
||||
# The jail's port and protocol go along: port = ssh bans only tcp/22;
|
||||
# fail2ban's default (0:65535) bans every port. IPv6 addresses are banned
|
||||
# with the list's grouping (a /64 unless set otherwise). Housecarl never
|
||||
# bans inside networks, its own addresses or the list's "never".
|
||||
#
|
||||
# On the firewall itself, the local socket works too, still with the
|
||||
# token: url = http://localhost, curlopts = --unix-socket
|
||||
# /run/housecarl/housecarl.sock.
|
||||
|
||||
[Definition]
|
||||
|
||||
actionstart =
|
||||
actionstop =
|
||||
actioncheck =
|
||||
|
||||
# A ban renews one already there; a bantime of -1 is the list's longest.
|
||||
actionban = <_curl> -f -X POST -H "Content-Type: application/json" "<url>/api/v1/banlists/<list>/bans"
|
||||
-d '{"address": "<ip>", "protocol": "<protocol>", "ports": "<port>", "time": <bantime>, "source": "fail2ban/<name>"}'
|
||||
|
||||
# A ban Housecarl already ended (its own time ran out) is no error.
|
||||
actionunban = <_curl> -X DELETE "<url>/api/v1/banlists/<list>/bans?address=<ip>"
|
||||
|
||||
[Init]
|
||||
|
||||
# The ban list, and Housecarl's URL (its listen address).
|
||||
list =
|
||||
url = https://127.0.0.1:8070
|
||||
# The API token, in a file only root reads.
|
||||
tokenfile = /etc/fail2ban/housecarl.token
|
||||
# More curl options: --cacert /etc/fail2ban/housecarl-ca.pem for a
|
||||
# private CA, --unix-socket for the local socket.
|
||||
curlopts =
|
||||
|
||||
_curl = curl -sS --max-time 10 <curlopts> -o /dev/null -H "Authorization: Bearer $(cat <tokenfile>)"
|
||||
@@ -1,5 +1,8 @@
|
||||
# Housecarl: loads the last confirmed ruleset at boot, before the network
|
||||
# comes up and whether or not housecarld starts.
|
||||
# comes up and whether or not housecarld starts. If it fails to load, the
|
||||
# fallback housecarld saved beside it is loaded instead: nothing new gets
|
||||
# in from external zones (docs/operations.md), and the unit is marked
|
||||
# failed, until housecarld starts and puts the real ruleset back.
|
||||
# Installed as /etc/systemd/system/housecarl-ruleset.service.
|
||||
[Unit]
|
||||
Description=Housecarl firewall ruleset
|
||||
@@ -15,7 +18,8 @@ ConditionPathExists=/var/lib/housecarl/ruleset.nft
|
||||
[Service]
|
||||
Type=oneshot
|
||||
RemainAfterExit=yes
|
||||
ExecStart=/usr/sbin/nft -f /var/lib/housecarl/ruleset.nft
|
||||
# "<2>": logged as critical.
|
||||
ExecStart=/bin/sh -c '/usr/sbin/nft -f /var/lib/housecarl/ruleset.nft && exit 0; echo "<2>the saved ruleset failed to load: loading the fallback (nothing new gets in from external zones) until housecarld puts the real one back" >&2; /usr/sbin/nft -f /var/lib/housecarl/ruleset-fallback.nft; exit 1'
|
||||
|
||||
[Install]
|
||||
WantedBy=sysinit.target
|
||||
+111
-24
@@ -31,7 +31,7 @@ user name. [permissions.md](permissions.md) has what each call needs.
|
||||
| 401 | `unauthenticated`, `session_expired`, `login_failed` | |
|
||||
| 403 | `forbidden`, `csrf` | |
|
||||
| 404 | `not_found` | |
|
||||
| 409 | `conflict` | e.g. undoing a change after the objects changed again |
|
||||
| 409 | `conflict` | e.g. undoing a change after the objects changed again; a takeover or closing connections in shadow mode ([shadow.md](shadow.md)) |
|
||||
| 422 | `validation` | the change would make the configuration invalid; `problems` lists why ([validation.md](validation.md)) |
|
||||
| 422 | `firewall` | nftables refused the ruleset (`nft -c`), or applying it failed; the message has nftables' words. Nothing was changed |
|
||||
| 429 | `rate_limited` | |
|
||||
@@ -50,9 +50,19 @@ Calls that change the configuration answer:
|
||||
|
||||
* `confirm_by`: the change is on probation, and is rolled back at that
|
||||
time unless `POST /confirm` comes first. Missing for changes made
|
||||
through the socket.
|
||||
through the socket, and in shadow mode ([shadow.md](shadow.md)).
|
||||
* `confirm_token`: the client that made the change can confirm it by
|
||||
itself with `POST /confirm {"token"}`, from the same address, as the
|
||||
same user and token, over a new connection (the answer closes this
|
||||
one). Missing when a person must confirm (`manual_confirm`).
|
||||
* `?dry_run=1` checks the change, with nftables too, and returns what it
|
||||
would do, with `"applied": false, "dry_run": true`.
|
||||
would do, with `"applied": false, "dry_run": true`, and `diff`: the
|
||||
ruleset's unified diff, each `@@` line followed by the chain (or other
|
||||
block of the table) the hunk starts in, as in `@@ -40,6 +40,8 @@ chain
|
||||
forward`.
|
||||
* `?changeset=ID` (or the header `X-Housecarl-Changeset: ID`) stages the
|
||||
change in that changeset instead ([Changesets](#changesets)): the answer
|
||||
has `"staged": true`, `changes` (this operation's) and `changeset`.
|
||||
* `before` is missing for a create and `after` for a delete.
|
||||
* `warnings` are the problems the change brings; older ones are in
|
||||
`/status`.
|
||||
@@ -66,12 +76,20 @@ Creating answers 201, other changes 200.
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| `GET /healthz` | `{"ok": true, "version"}`, without authentication |
|
||||
| `GET /status` | `build`, `started`, `applied` (the ruleset is in force), `apply_error`, `problems`, `probation`, `interfaces`, `counts` (objects by kind), `other_tables` (other nftables tables, `"inet main"`) |
|
||||
| `GET /healthz` | `ok`, `version`, `firewall` (`applied`, `unconfigured`, `fallback`, `missing`), `mode` (`enforcing`, or `shadow`: the ruleset blocks nothing, [shadow.md](shadow.md)), `probation` (changes waiting to be confirmed), `warnings` (codes); HTTP 503 unless `ok`. Without authentication; read at most every five seconds ([operations.md](operations.md#monitoring)) |
|
||||
| `GET /status` | `build`, `started`, `mode` (`enforcing` or `shadow`), `shadow_position` (`after` or `before`, in shadow mode), `applied` (the ruleset is in force), `apply_error`, `problems`, `probation`, `interfaces`, `counts` (objects by kind), `other_tables` (other nftables tables, `"inet main"`) |
|
||||
| `GET /interfaces` | the machine's interfaces: `name`, `up`, `addresses`, `zone` |
|
||||
| `GET /ruleset` | the nftables ruleset Housecarl builds, as text |
|
||||
| `GET /matrix[?changeset=ID]` | the zone matrix: `zones`, `cells` (`from`, `to` (a zone, or `firewall`), `chain`, `summary` (allowed, blocked, partly), `verdict` (what the rest gets: allowed, blocked), `decided_by`, `rule_id`, `hits`, `allowed` and `blocked` (exceptions by verdict), `open` (what the allowing exceptions let through, in short), `same_zone` and `note`, `exceptions`: `rule_id`, `chain`, `label`, `verdict` (allowed, blocked, logged, unknown), `what`, `short`, `hits`). `hits` are the rule's or policy's packets since housecarld started, all its traffic (missing: no counter) ([rules.md](rules.md#the-zone-matrix)) |
|
||||
| `GET /ruleset[?enforcing=1]` | the nftables ruleset Housecarl builds, as text. In shadow mode it is the shadow ruleset; `enforcing=1` gives the one loaded once shadow mode is off |
|
||||
| `GET /counters` | `{"since", "counters": {"r12": {"packets", "bytes"}, ...}}`, by mark ([design.md](design.md#its-own-table-only)); totals since housecarld started |
|
||||
| `GET /log[?limit=&after=&q=]` | logged packets, newest first: `time`, `prefix`, `in`, `out`, `family`, `protocol`, `source`, `source_port`, `destination`, `destination_port`, `icmp_type`, `tcp_flags`, `length`, `mark` (the rule's). `after` (a time) for those since; `q` searches |
|
||||
| `GET /quotas` | each rule's quota: `rule_id`, `rule`, `quota`, `used` (bytes), `addresses` (`[{"address", "used"}]` for a per-address quota), `last_reset`, `next_reset` |
|
||||
| `GET /connections[?q=&limit=]` | tracked connections: `family`, `protocol`, `source`, `source_port`, `destination`, `destination_port` (or `icmp_id`, `icmp_type`, `icmp_code`), `dnat_to`, `dnat_port`, `snat_to`, `state`, `assured`, `timeout` (seconds left), `packets` and `bytes` when the kernel counts them. `q`: all of its words in the addresses, `:port`, protocol, state |
|
||||
| `POST /names` | `{"addresses": [...]}` (at most 2000): `{"enabled", "names": {"192.168.1.20": "laptop.example.net"}}`, the names of the inside addresses among them, by reverse lookup ([configuration.md](configuration.md#names)). Addresses without a name, outside ones, and those still being looked up after a second are left out; ask again for those. `log:read` |
|
||||
| `POST /connections/close` | `{"connections": [...]}` (as listed) or `{"search": "192.168.1.50"}`: delete their tracking entries (`config:write`, audited); answers `{"closed": N}`. 409 in shadow mode |
|
||||
| `POST /trace` | `{"address", "port", "protocol", "seconds"}` (5-600, default 60): start tracing (`config:write`, audited); 409 while one runs |
|
||||
| `GET /trace` | the trace running, or the last: `running`, `input`, `started`, `ends`, `error`, `packets`: each with `time`, `in`, `out`, `protocol`, addresses and ports, `tcp_flags`, `icmp_type`, `steps` (`table`, `chain`, `kind` rule or policy, `rule`, `mark`, `label`, `verdict`), `verdict` (accepted, dropped) and `decided_by`; in shadow mode `would_drop`, the step where Housecarl would have dropped it |
|
||||
| `DELETE /trace` | stop it |
|
||||
| `GET /log[?limit=&after=&q=]` | logged packets, newest first: `time`, `prefix`, `in`, `out`, `family`, `protocol`, `source`, `source_port`, `destination`, `destination_port`, `icmp_type`, `tcp_flags`, `length`, `mark` (the rule's), `verdict`, `shadow` (logged in shadow mode: a drop or reject is what would have been done). `after` (a time) for those since; `q` searches |
|
||||
| `GET /catalog` | built-in services, chains, actions, roles, the log group |
|
||||
|
||||
### Settings
|
||||
@@ -83,49 +101,101 @@ Creating answers 201, other changes 200.
|
||||
"log_policy_drops": false,
|
||||
"builtins": {"loopback": true, "established": true, "drop_invalid": true, "icmpv6": true,
|
||||
"ping_internal": true, "dhcpv6_client": true, "antispoof": true, "mss_clamp": false},
|
||||
"own_prefixes": ["2001:db8:7f3d::/48"], "antispoof_v4": ["10.0.0.0/8", "..."], "antispoof_v6": ["fc00::/7", "..."],
|
||||
"flowtable_devices": ["wan", "enp2s0"], "probation_seconds": 120}
|
||||
"own_prefixes": ["2001:db8::/48"], "antispoof_v4": ["10.0.0.0/8", "..."], "antispoof_v6": ["fc00::/7", "..."],
|
||||
"flowtable_devices": ["wan", "eth1"], "probation_seconds": 120, "manual_confirm": false,
|
||||
"stats_per_device": false}
|
||||
```
|
||||
|
||||
Lists replace the old ones. `probation_seconds` 0 is the default (120).
|
||||
|
||||
### Objects
|
||||
|
||||
Zones, sets, services, rules, port forwards, source NAT and redirects
|
||||
share one pattern, `/{kind}` with kind `zones`, `sets`, `services`,
|
||||
`rules`, `forwards`, `snats` or `redirects`:
|
||||
Zones, sets, services, rules, port forwards, source NAT, redirects and
|
||||
ban lists share one pattern, `/{kind}` with kind `zones`, `sets`,
|
||||
`services`, `rules`, `forwards`, `snats`, `redirects` or `banlists`:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| `GET /{kind}` | all of them, in order; `GET /rules?chain=input` for one chain; `GET /services` includes the built-in ones |
|
||||
| `POST /{kind}` | create; rules, forwards and NAT rules go last unless given a `position` |
|
||||
| `GET /{kind}/{ref}` | by ID, or name for zones, sets, services and forwards |
|
||||
| `POST /{kind}[?before=ID\|after=ID\|first=1]` | create; rules, forwards and NAT rules go last unless given a `position`, or placed before or after another of their chain, or first, in the same change (one history entry; staged in a changeset, the place goes with it) |
|
||||
| `GET /{kind}/{ref}` | by ID, or name for zones, sets, services, forwards and ban lists |
|
||||
| `PATCH /{kind}/{ref}` | change the fields given (a JSON merge: `null` clears) |
|
||||
| `DELETE /{kind}/{ref}` | refused while something uses it |
|
||||
| `POST /rules/{ref}/reset-quota` | start a rule's quota again now (`config:write`; audited, not staged). Optional body: `{"address": "192.168.1.23"}` starts only that address's again (a per-address quota; an IPv6 address counts for its /64; 404 if it hasn't used the quota); `{"give": 5368709120}` takes that many bytes off what was used instead (the address's, or every address's), until the next reset |
|
||||
| `POST /{kind}/{ref}/move` | `{"before": ID}`, `{"after": ID}` or `{"to": "start"\|"end"}`; a rule can take `"chain"` too |
|
||||
|
||||
The objects:
|
||||
|
||||
```json
|
||||
{"name": "lan", "interfaces": ["enp2s0", "enp2s0.*"], "external": false, "comment": ""}
|
||||
{"name": "trusted", "entries": ["10.0.10.0/24", "2001:db8:7f3d:10::/64"], "dynamic": false, "timeout": 0, "comment": ""}
|
||||
{"name": "jellyfin", "entries": ["tcp/8096", "tcp/8920"], "comment": ""}
|
||||
{"name": "lan", "interfaces": ["eth1", "eth1.*"], "external": false, "comment": ""}
|
||||
{"name": "trusted", "entries": ["192.168.1.0/24", "2001:db8:1::/64"], "dynamic": false, "timeout": 0, "comment": ""}
|
||||
{"name": "web", "entries": ["tcp/8080", "tcp/8443"], "comment": ""}
|
||||
{"chain": "input",
|
||||
"in_zones": ["wan"], "not_in": false, "out_zones": [], "not_out": false,
|
||||
"sources": ["trusted"], "not_sources": false, "destinations": [], "not_destinations": false,
|
||||
"services": ["ssh"], "family": "",
|
||||
"ct_state": ["new"], "ct_status": [], "days": [], "hour_from": "", "hour_to": "",
|
||||
"limit": {"rate": 3, "per": "minute", "burst": 5, "over": true, "per_source": true}, "conn_limit": 0,
|
||||
"limit": {"rate": 3, "unit": "", "per": "minute", "burst": 5, "over": true, "per_source": true, "per_destination": false}, "conn_limit": 0,
|
||||
"quota": {"bytes": 53687091200, "over": true, "per_source": true, "per_destination": false, "reset": "monthly", "day": 15},
|
||||
"group_v4": 0, "group_v6": 0,
|
||||
"action": "drop", "reject_with": "", "log": true, "log_prefix": "",
|
||||
"raw": "", "comment": "", "disabled": false}
|
||||
{"name": "jellyfin", "protocols": ["tcp"], "ports": "8096", "public_addresses": ["203.0.113.7"], "sources": [],
|
||||
"to": "10.0.10.57", "to_port": "443", "to6": "2001:db8:7f3d:10::57", "hairpin": true, "hairpin_sources": [],
|
||||
{"name": "web", "protocols": ["tcp"], "ports": "8443", "public_addresses": ["203.0.113.10"], "sources": [],
|
||||
"to": "192.168.1.10", "to_port": "443", "to6": "2001:db8:1::10", "hairpin": true, "hairpin_sources": [],
|
||||
"comment": "", "disabled": false}
|
||||
```
|
||||
|
||||
A source NAT rule is a match (as a rule's, without the rule's other
|
||||
fields) with `"to"` (empty: masquerade), a redirect a match with
|
||||
`"to_port"`. Times are seconds (`timeout`).
|
||||
`"to_port"`. A ban list ([bans.md](bans.md)):
|
||||
|
||||
```json
|
||||
{"name": "web", "scope": "hosts", "hosts": ["10.0.10.57"], "never": ["trusted"],
|
||||
"default_time": 3600, "max_time": 604800, "max_entries": 10000, "group_v6": 64, "comment": "", "disabled": false}
|
||||
```
|
||||
|
||||
`scope` is `firewall`, `hosts` or `everything`; 0 is the default for the
|
||||
numbers, and `max_time` -1 is no limit. A ban list can't be renamed.
|
||||
Someone with a `bans` grant but no `config:read` may read the ban lists
|
||||
the grant covers (`GET /banlists`, `GET /banlists/{ref}`). Times are
|
||||
seconds (`timeout`).
|
||||
|
||||
### Statistics
|
||||
|
||||
`GET /stats/status`, `/stats/summary`, `/stats/timeline`, `/stats/top`,
|
||||
`/stats/hits` and `/stats/heatmap` (`log:read`): see
|
||||
[stats.md](stats.md#the-api).
|
||||
|
||||
### Changesets
|
||||
|
||||
Changes staged to be reviewed and applied together ([design.md](design.md#reviewing-changes-changesets)).
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| `GET /changesets[?status=open&mine=1&limit=]` | newest first |
|
||||
| `POST /changesets` | `{"description"}`; answers the changeset (201) |
|
||||
| `GET /changesets/{id}` | `{"id", "username", "via", "description", "status": "open"\|"applied"\|"discarded", "ops": [{"seq", "op", "kind", "ref", "id", "data", "summary", "added_at"}], "created_at", "updated_at", "history_id"}` |
|
||||
| `PATCH /changesets/{id}` | `{"description"}` |
|
||||
| `GET /changesets/{id}/preview` | `{"valid", "changes", "warnings", "error", "diff"}`: what applying it would do; `error.op` names an operation that no longer applies |
|
||||
| `POST /changesets/{id}/apply` | optional `{"description"}`; answers like a change, with `confirm_by` over the network and `changeset` |
|
||||
| `POST /changesets/{id}/discard` | |
|
||||
| `DELETE /changesets/{id}/ops/{seq}` | take one operation out |
|
||||
|
||||
Staging (`?changeset=ID` on any change to objects or settings) checks
|
||||
the changeset with the new operation, and nftables checks its ruleset;
|
||||
an error refuses the operation. Only open changesets take operations, and
|
||||
only their owner's (or an administrator's). An undo
|
||||
(`POST /history/{id}/revert?changeset=ID`) is staged as one operation,
|
||||
which checks when it is replayed that nothing it touches changed since;
|
||||
applied, History links it to the entry it undoes. `/import/nft` and
|
||||
`/takeover` can't be staged: with a changeset named they answer 400.
|
||||
|
||||
Reads of objects, settings and `/ruleset` with `?changeset=ID` show the
|
||||
configuration as the changeset will make it; objects are marked
|
||||
`"pending": "new"`, `"changed"` or `"deleted"` (deleted ones stay in the
|
||||
list). A changeset that no longer applies shows the current
|
||||
configuration, with the reason in the `X-Housecarl-Changeset-Error`
|
||||
header.
|
||||
|
||||
### Dynamic sets' entries
|
||||
|
||||
@@ -139,12 +209,28 @@ These need `sets:read` or `sets:write`, for that set
|
||||
([permissions.md](permissions.md)); they aren't on probation or in the
|
||||
history, but are audited.
|
||||
|
||||
### Bans
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| `GET /bans` | the active bans of every ban list you may read: `id`, `list_id`, `list`, `address` (an address, or an IPv6 prefix), `protocol` and `port` (none: every port), `expires` and `timeout` (seconds left; none: no end), `created`, `who`, `source`, `reason` |
|
||||
| `GET /banlists/{ref}/bans` | one list's |
|
||||
| `POST /banlists/{ref}/bans` | ban: `{"address": "203.0.113.9", "protocol": "tcp", "ports": "ssh", "time": 3600, "source": "guard/ssh", "reason": "…"}`. `ports` is a number, a string (`"22"`, `"http,https"`, `"0:65535"` or `"all"` for every port) or a list; without a protocol a number is TCP and UDP, a service its own protocols. `time` 0 is the list's default, -1 its longest; longer is cut down. 201 with `{"bans": [ban…, "existing": true for a renewed one], "closed": N}` (connections closed) |
|
||||
| `DELETE /banlists/{ref}/bans/{id}` | lift one ban |
|
||||
| `DELETE /banlists/{ref}/bans?address=A` | lift all of an address's bans (404 if it has none) |
|
||||
|
||||
These need `bans:read` or `bans:write` for the list
|
||||
([permissions.md](permissions.md)); bans aren't on probation or in the
|
||||
history, but are audited (`ban.add`, `ban.remove`). Refusals: `422
|
||||
never_ban` (inside, the firewall's own, the caller's own, or in the
|
||||
list's `never`), `422 list_full`, `409` for a disabled list.
|
||||
|
||||
### Probation
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| `GET /probation` | `{"pending": [history IDs], "takeovers": N, "deadline", "seconds_left"}` |
|
||||
| `POST /confirm` | keep them: `{"confirmed": N}`; the ruleset is saved for the boot |
|
||||
| `POST /confirm` | keep them: `{"confirmed": N}`; the ruleset is saved for the boot. With `{"token"}`, the client confirming its own change (see `confirm_token`): 409 if a newer change is pending, or over the change's own connection; 403 from another user or address |
|
||||
| `POST /rollback` | undo them now |
|
||||
|
||||
### History, import, takeover
|
||||
@@ -153,9 +239,10 @@ history, but are audited.
|
||||
|---|---|
|
||||
| `GET /history[?limit=&before=]` | changes, newest first |
|
||||
| `GET /history/{id}` | one, with each object before and after |
|
||||
| `GET /history/{id}/diff` | `{"diff"}`: what it did to the ruleset, as a unified diff with each hunk's chain (the configuration before and after it, got back from the current one, rendered by this version) |
|
||||
| `POST /history/{id}/revert` | undo it; 409 if an object it touched has changed since, or it was undone already |
|
||||
| `POST /import/nft` | `{"json": NFT_JSON, "text": "nft -a list ruleset output", "defines": "ruleset file", "table": "inet main", "replace": false}` ([import.md](import.md)); answers with `state` (the imported configuration), `notes` and `raw` too |
|
||||
| `POST /takeover` | `{"tables": ["inet main"]}`: delete these tables (administrators); on probation over the network |
|
||||
| `POST /import/nft` | `{"json": NFT_JSON, "text": "nft -a list ruleset output", "defines": "ruleset file", "table": "inet main", "zones": {"enp2s0": "lan"}, "replace": false}` ([import.md](import.md)); answers with `state` (the imported configuration), `notes` and `raw` too |
|
||||
| `POST /takeover` | `{"tables": ["inet main"]}`: delete these tables (administrators); on probation over the network; 409 in shadow mode |
|
||||
|
||||
### Certificate
|
||||
|
||||
|
||||
+148
@@ -0,0 +1,148 @@
|
||||
# Ban lists
|
||||
|
||||
A **ban list** is where a program bans addresses: Guard (Housecarl's
|
||||
companion, which reads logs), fail2ban with Housecarl's action, or you
|
||||
with `hc ban`. The program says only *which address*, and perhaps a
|
||||
protocol and ports. The ban list says what that address is kept from.
|
||||
So a web server that sees password guessing in its own logs can ban the
|
||||
guesser at the router without knowing anything about the router.
|
||||
|
||||
## Ban lists
|
||||
|
||||
```sh
|
||||
hc banlist add router --scope firewall # the firewall itself
|
||||
hc banlist add web --scope hosts --host 10.0.10.57 --host 2001:db8:10::57
|
||||
hc banlist add edge --scope everything --never trusted
|
||||
```
|
||||
|
||||
| Scope | Banned addresses are kept from | Where the drops are |
|
||||
|---|---|---|
|
||||
| `firewall` | the firewall itself: its SSH, DNS, web UI | `input` |
|
||||
| `hosts` | the list's hosts (addresses or sets), behind the firewall | `forward`, to those addresses |
|
||||
| `everything` | the firewall and everything behind it | `input` and `forward` |
|
||||
|
||||
A `hosts` list covers port forwards to its hosts as well: the forward
|
||||
chain sees a forwarded connection's translated address and port, the
|
||||
host's own. A web server's Guard bans `tcp/22`, and that is right even
|
||||
when the router forwards its port 2222 to it.
|
||||
|
||||
Other settings, with their defaults:
|
||||
|
||||
| Setting | Default | |
|
||||
|---|---|---|
|
||||
| `never` | none | addresses, prefixes or sets this list never bans |
|
||||
| `default_time` (`--time`) | 1h | a ban's length when the program asks for none |
|
||||
| `max_time` (`--max-time`) | 7d | the longest ban; longer ones are cut to it. `-1` (`none`): no limit, so bans may be permanent |
|
||||
| `max_entries` | 10000 | the most bans the list holds; more are refused (`list_full`) |
|
||||
| `group_v6` | 64 | the IPv6 prefix length banned for an address. One customer usually has a whole /64 and can move around in it; 128 bans the one address. IPv4 addresses are banned alone |
|
||||
| `disabled` | no | the list's bans aren't in force (they are kept) |
|
||||
|
||||
A ban list can't be renamed: the tokens and programs that ban into it
|
||||
know it by its name. Make a new one instead.
|
||||
|
||||
## Bans
|
||||
|
||||
```sh
|
||||
hc ban edge 203.0.113.9 --time 1d --reason "port scan"
|
||||
hc ban web 203.0.113.9 --tcp ssh --tcp 8443
|
||||
hc bans # every list's
|
||||
hc bans web
|
||||
hc unban web 203.0.113.9 # all of the address's bans in web
|
||||
hc unban web 17 # one, by ID
|
||||
```
|
||||
|
||||
A ban is of **every port**, or of **a protocol's ports** (numbers or
|
||||
service names, built-in or yours). A port without a protocol is banned
|
||||
for TCP and UDP; a service name takes the service's protocols. Banning an
|
||||
address again renews the ban: its new time counts from now.
|
||||
|
||||
Bans are dropped **ahead of every rule** in their chains, and ahead of
|
||||
the built-in that lets established connections through, so a ban cuts a
|
||||
connection that is already open. Housecarl also closes the banned
|
||||
address's tracked connections the ban covers when it bans it, because
|
||||
connections offloaded to the flowtable no longer pass the rules at all.
|
||||
Replies to connections the inside opened to a banned address are
|
||||
dropped too.
|
||||
|
||||
Bans are kept in the database, apart from the configuration (like a
|
||||
dynamic set's entries): they aren't in the history, can't be undone,
|
||||
aren't on probation, and don't go through changesets. They end by
|
||||
themselves; every change, restart and reboot puts the active ones back
|
||||
with the time they have left. Each ban and unban is in the audit log
|
||||
(`ban.add`, `ban.remove`), with the program's source and reason.
|
||||
|
||||
### What is never banned
|
||||
|
||||
Whatever a list's `never` says, Housecarl refuses (`422 never_ban`) to
|
||||
ban:
|
||||
|
||||
* the inside: addresses in networks of inside zones' interfaces, and
|
||||
private addresses outside the external zones' networks;
|
||||
* the firewall's own addresses, and its own prefixes (settings);
|
||||
* the address of whoever asks (a program can't lock itself out);
|
||||
* loopback, link-local, multicast and unspecified addresses.
|
||||
|
||||
A ban grouped to a prefix (IPv6) is refused when the prefix overlaps any
|
||||
of these.
|
||||
|
||||
## Programs and tokens
|
||||
|
||||
Give each program a token for its list alone. A `bans` grant can name
|
||||
ban lists, as a `sets` grant names sets:
|
||||
|
||||
```sh
|
||||
hc user add guard-web --no-password --grant bans:write:web
|
||||
hc token create web --user guard-web # as root on the firewall
|
||||
```
|
||||
|
||||
With it, the program may ban into `web` and lift its bans, list them,
|
||||
and read the list itself (`GET /banlists/web`: its grouping and times),
|
||||
and nothing else. `viewer` has `bans:read`, `operator` `bans:write`.
|
||||
|
||||
## Guard
|
||||
|
||||
Guard reads log files and the systemd journal on any machine, and bans
|
||||
the addresses that misbehave into a ban list (its own documentation).
|
||||
Run it on the web server with a token for `web`, and on the router with
|
||||
one for `router` (or the local socket).
|
||||
|
||||
## fail2ban
|
||||
|
||||
Housecarl ships an action for fail2ban,
|
||||
`/usr/share/doc/housecarl/contrib/fail2ban/action.d/housecarl.conf`
|
||||
(`contrib/` in the source). Copy it to `/etc/fail2ban/action.d/`, put
|
||||
the token in `/etc/fail2ban/housecarl.token` (mode 0600), and use it in a
|
||||
jail:
|
||||
|
||||
```ini
|
||||
[sshd]
|
||||
enabled = true
|
||||
action = housecarl[list=web, url="https://router.lan:8070"]
|
||||
```
|
||||
|
||||
The jail's `port` and `protocol` go with each ban: `port = ssh` bans only
|
||||
`tcp/22`, fail2ban's default `0:65535` bans every port. A `bantime` of
|
||||
`-1` is the list's longest ban. When fail2ban unbans an address Housecarl
|
||||
already let go (its own time ran out first), nothing happens. Add
|
||||
`curlopts = --cacert /etc/fail2ban/housecarl-ca.pem` for a private CA.
|
||||
|
||||
## Shadow mode
|
||||
|
||||
In shadow mode the ban rules count what they would drop
|
||||
(`[would drop]`) and let it through, like every other rule, and banning
|
||||
closes no connections.
|
||||
|
||||
## The ruleset
|
||||
|
||||
Each list has four sets, `ban_<id>_v4`, `ban_<id>_v6` (every port) and
|
||||
`ban_<id>_v4_svc`, `ban_<id>_v6_svc` (address, protocol and port), all
|
||||
with timeouts:
|
||||
|
||||
```
|
||||
ip saddr @ban_3_v4 counter drop comment "hc:ban3"
|
||||
ip saddr . meta l4proto . th dport @ban_3_v4_svc counter drop comment "hc:ban3"
|
||||
ip6 saddr and ffff:ffff:ffff:ffff:: @ban_3_v6 counter drop comment "hc:ban3"
|
||||
```
|
||||
|
||||
Set names starting with `ban_` are therefore Housecarl's own
|
||||
(`set_name_reserved`).
|
||||
@@ -23,7 +23,7 @@ on the internet, and Jarl puts it in every view that has your zone.
|
||||
3. **Configure the certificate** in `/etc/housecarl/housecarl.yaml`:
|
||||
|
||||
```yaml
|
||||
listen: ["10.0.10.2:8070"] # an address your browser can reach
|
||||
listen: ["192.168.1.2:8070"] # an address your browser can reach
|
||||
tls:
|
||||
acme:
|
||||
domain: fw.adbraun.lol
|
||||
|
||||
+102
-15
@@ -27,9 +27,12 @@ command and its flags.
|
||||
| `--config FILE` | `HOUSECARL_CLI_CONFIG` | the CLI's configuration file |
|
||||
| | `HOUSECARL_SOCKET` | the local socket's path |
|
||||
| `--insecure`, `--ca-cert FILE` | | TLS: skip the check, or trust another CA |
|
||||
| `-n, --dry-run` | | check and show a change without making it |
|
||||
| `-n, --dry-run` | | check and show a change without making it, with the ruleset's diff |
|
||||
| `--no-confirm` | | leave a change on probation instead of confirming it once `hc` reaches Housecarl again |
|
||||
| `-C, --changeset ID` | `HOUSECARL_CHANGESET` (checked, see [Changesets](#changesets)) | stage configuration changes in this changeset instead of making them |
|
||||
| `-o, --output FORMAT` | | `table` (default), `json`, `yaml` or `id` |
|
||||
| `-q, --quiet` | | print only errors (and the IDs of new objects) |
|
||||
| `--no-names` | | show inside addresses without their names: tables show `192.168.1.20 (laptop)` ([configuration.md](configuration.md#names)) |
|
||||
|
||||
Exit codes: 0 ok, 1 error, 3 not authenticated or not allowed, 4 not
|
||||
found, 5 conflict, 6 invalid or refused by validation or nftables, 7
|
||||
@@ -40,14 +43,14 @@ server error.
|
||||
Every change is checked as a whole, then by nftables, before it is put
|
||||
in force. A refused change says why, with the problem's code
|
||||
([validation.md](validation.md)). An applied one says its history entry
|
||||
and any new warnings. Over the network, it is on probation:
|
||||
and any new warnings. Over the network, it is on probation, and `hc`
|
||||
confirms it by reaching Housecarl again over a new connection
|
||||
([operations.md](operations.md#changes-on-probation)):
|
||||
|
||||
```
|
||||
$ hc rule add input --from trusted --service ssh --comment "SSH from home"
|
||||
created rule input #31 (from trusted · ssh)
|
||||
applied on probation (history 12): confirm with 'hc confirm' within 2m0s, or it is rolled back
|
||||
$ hc confirm
|
||||
confirmed 1 change; it is kept and loaded at boot
|
||||
applied and confirmed (history 12): this client reached Housecarl again over a new connection
|
||||
```
|
||||
|
||||
Through the socket the change is kept at once:
|
||||
@@ -56,8 +59,70 @@ Through the socket the change is kept at once:
|
||||
applied and saved (history 12; undo with 'hc history undo 12')
|
||||
```
|
||||
|
||||
`-n` shows what would change without changing it; nftables checks it
|
||||
too.
|
||||
`-n` shows what would change without changing it, with the diff of the
|
||||
ruleset (each `@@` line names the chain the lines below it are in);
|
||||
nftables checks it too.
|
||||
|
||||
## Changesets
|
||||
|
||||
To review several changes and apply them together, as one entry in the
|
||||
history, stage them in a changeset:
|
||||
|
||||
```sh
|
||||
export HOUSECARL_CHANGESET=$(hc changeset create "guest network" -q)
|
||||
hc zone add guest -i eth1.30
|
||||
hc rule add forward --in guest --out wan
|
||||
hc changeset diff # the operations, problems, nftables' check, the ruleset's diff
|
||||
hc changeset remove 2 # take operation 2 out
|
||||
hc changeset apply # one change; over the network on probation
|
||||
unset HOUSECARL_CHANGESET
|
||||
```
|
||||
|
||||
```
|
||||
$ hc rule add forward --in guest --out wan
|
||||
changeset 7 (guest network, 1 operation) from HOUSECARL_CHANGESET
|
||||
staged: create rule forward #52
|
||||
staged in changeset 7 as operation 2 (2 operations now); review with 'hc changeset diff 7', apply with 'hc changeset apply 7'
|
||||
```
|
||||
|
||||
Each change is checked as it is staged: the changeset with it must stay
|
||||
valid, and nftables must accept the ruleset, so make a set before the
|
||||
rule that names it. Applying checks the whole changeset again, on the
|
||||
configuration as it is then; an operation whose object changed since is
|
||||
named (`operation 3 (update rule input #12) can't be applied: …`), and
|
||||
`hc changeset remove` takes it out.
|
||||
|
||||
| Command | |
|
||||
|---|---|
|
||||
| `changeset create [DESCRIPTION]` | start one; `-q` prints only its ID |
|
||||
| `changeset list [--status open\|applied\|discarded\|""] [--mine]` | open ones by default |
|
||||
| `changeset show [ID]` | its operations |
|
||||
| `changeset diff [ID]` | what applying it would do |
|
||||
| `changeset apply [ID] [--description D]` | apply it |
|
||||
| `changeset discard [ID]` | throw it away |
|
||||
| `changeset remove [ID] OP` | take one operation out |
|
||||
|
||||
`show`, `diff`, `apply`, `discard` and `remove` work on the changeset you
|
||||
name, else the one in `-C`, else the one in `HOUSECARL_CHANGESET`.
|
||||
|
||||
A changeset from `HOUSECARL_CHANGESET` is checked before anything uses
|
||||
it, because an exported ID easily outlives its changeset:
|
||||
|
||||
- Staging into it, and `apply`, `discard` and `remove` on it, need it to
|
||||
be open and yours. Otherwise the command stops and says why (`there is
|
||||
no changeset 99`, `changeset 42 is applied`, `changeset 57 is sam's`).
|
||||
Commands that only read, and `hc changeset create`, don't care.
|
||||
- Each command says on stderr which changeset it uses: `changeset 42
|
||||
(guest network, 2 operations) from HOUSECARL_CHANGESET` (not with `-q`).
|
||||
- After `apply` or `discard`, a reminder to `unset HOUSECARL_CHANGESET`.
|
||||
|
||||
`-C ID` and an ID argument are used as given (housecarld still refuses a
|
||||
changeset that isn't open, or another user's unless you are an
|
||||
administrator). Only configuration changes are staged: dynamic set
|
||||
entries, `confirm` and `rollback` are made at once; `history undo` is
|
||||
staged like any change; `import` and `takeover` can't be staged, so they
|
||||
are refused while a changeset is named. Changesets staged in the web UI show up here too,
|
||||
and the other way round.
|
||||
|
||||
## Commands
|
||||
|
||||
@@ -68,11 +133,16 @@ by name.
|
||||
|
||||
| Command | |
|
||||
|---|---|
|
||||
| `status` | the ruleset (in force or not), changes on probation, problems, interfaces, other tables |
|
||||
| `status` | the mode (in shadow mode), the ruleset (in force or not), changes on probation, problems, interfaces, other tables |
|
||||
| `interfaces` | the machine's interfaces, their addresses and zones |
|
||||
| `ruleset` | the nftables ruleset Housecarl builds |
|
||||
| `ruleset [--enforcing]` | the nftables ruleset Housecarl builds; in shadow mode, `--enforcing` shows the one loaded once it is off ([shadow.md](shadow.md)) |
|
||||
| `counters` | the hits of every rule, NAT rule, built-in and policy, by mark (`r12`, `f3`, `b-established`, `p-input`) |
|
||||
| `quotas` | what each rule's quota used (each address's, for per-address quotas), and when it resets |
|
||||
| `log [-f] [--search TEXT] [--limit N]` | the packets logged by rules and policies ([operations.md](operations.md#the-log)) |
|
||||
| `connections [--search TEXT] [--limit N]`, `connections close --search TEXT [-y]` | tracked connections; closing makes the rules check them again ([operations.md](operations.md#connections)); not in shadow mode |
|
||||
| `stats [--since 24h]` | traffic per zone, drops, the busiest rules, blocked sources, devices; `stats status`, `stats traffic`, `stats top devices\|blocked\|tried\|quota ID`, `stats rules [--unused]`, `stats connections` ([stats.md](stats.md#the-cli)) |
|
||||
| `matrix [FROM TO]` | what the rules do to new connections between zones (allow, block or partly), and what is let through; with FROM and TO, that pair's rules in order, with their hits ([rules.md](rules.md#the-zone-matrix)) |
|
||||
| `trace [--address A] [--port N] [--protocol P] [--seconds S]` | follow packets through every table's rules ([operations.md](operations.md#tracing)) |
|
||||
|
||||
### zone, set, service
|
||||
|
||||
@@ -80,7 +150,7 @@ by name.
|
||||
|---|---|
|
||||
| `zone list`, `zone show ZONE` | |
|
||||
| `zone add NAME -i IFACE... [--external] [--comment C]` | |
|
||||
| `zone update ZONE [-i IFACE]... [--add IFACE] [--remove IFACE] [--external[=false]] [--name N] [--comment C]` | `-i` replaces the interfaces |
|
||||
| `zone update ZONE [-i IFACE]... [--add IFACE] [--remove IFACE] [--external[=false]] [--name N] [--comment C]` | `-i` replaces the interfaces; `--name` renames it, and the rules naming it follow |
|
||||
| `zone delete ZONE` | refused while rules use it |
|
||||
| `set list`, `set show SET` | |
|
||||
| `set add NAME [ENTRY...] [--dynamic] [--timeout 24h] [--comment C]` | |
|
||||
@@ -103,6 +173,7 @@ See [rules.md](rules.md).
|
||||
| `rule update ID [flags]` | only the flags given |
|
||||
| `rule move ID --before ID\|--after ID\|--first\|--last\|--chain CHAIN` | |
|
||||
| `rule enable\|disable ID` | |
|
||||
| `rule reset-quota ID [--address ADDR] [--give 5gbytes]` | start a rule's quota again now, or one address's; `--give` takes an amount off what was used instead, until the next reset |
|
||||
| `rule delete ID` | |
|
||||
|
||||
The match flags (`rule add`, `snat add`, `redirect add`):
|
||||
@@ -115,7 +186,9 @@ The match flags (`rule add`, `snat add`, `redirect add`):
|
||||
| `--family ipv4\|ipv6` | |
|
||||
|
||||
And for rules: `--state`, `--status`, `--days`, `--hours`, `--limit
|
||||
10/minute[,burst=N][,over][,per-source]`, `--conn-limit N`, `-a
|
||||
10/minute|1mbytes/second[,burst=N][,over][,per-source|,per-destination]`,
|
||||
`--quota 50gbytes[,over][,per-source|,per-destination][,daily|,weekly|,monthly=DAY]`,
|
||||
`--group-v4 N`, `--group-v6 N`, `--conn-limit N`, `--every-packet`, `-a
|
||||
accept|drop|reject|log`, `--reject-with tcp-reset`, `--log`,
|
||||
`--log-prefix`, `--raw RULE`, `--comment`, `--disabled`
|
||||
([rules.md](rules.md#rules)).
|
||||
@@ -132,12 +205,25 @@ accept|drop|reject|log`, `--reject-with tcp-reset`, `--log`,
|
||||
|
||||
See [nat.md](nat.md).
|
||||
|
||||
### banlist, ban
|
||||
|
||||
| Command | |
|
||||
|---|---|
|
||||
| `banlist list`, `banlist show BANLIST` | ban lists, with their bans' count |
|
||||
| `banlist add NAME --scope firewall\|hosts\|everything [--host ADDR\|SET]... [--never ADDR\|SET]... [--time 1h] [--max-time 7d\|none] [--max-entries N] [--group-v6 64] [--disabled] [--comment C]` | |
|
||||
| `banlist update BANLIST [flags]`, `banlist delete BANLIST` | `--host` and `--never` replace the lists; no renaming |
|
||||
| `bans [BANLIST]` | the active bans |
|
||||
| `ban BANLIST ADDRESS [--tcp PORT\|SERVICE]... [--udp …]... [--time 1h\|none] [--reason R]` | ban every port, or some; again renews |
|
||||
| `unban BANLIST ADDRESS\|ID` | lift one ban, or all of an address's |
|
||||
|
||||
See [bans.md](bans.md).
|
||||
|
||||
### settings
|
||||
|
||||
| Command | |
|
||||
|---|---|
|
||||
| `settings show` | |
|
||||
| `settings set [--input-policy\|--forward-policy\|--output-policy accept\|drop] [--log-drops on\|off] [--builtin NAME=on\|off]... [--own-prefix P]... [--antispoof-v4 P]... [--antispoof-v6 P]... [--flowtable DEV]... [--probation 2m]` | lists given replace the old ones |
|
||||
| `settings set [--input-policy\|--forward-policy\|--output-policy accept\|drop] [--log-drops on\|off] [--builtin NAME=on\|off]... [--own-prefix P]... [--antispoof-v4 P]... [--antispoof-v6 P]... [--flowtable DEV]... [--probation 2m] [--auto-confirm on\|off] [--stats-per-device on\|off]` | lists given replace the old ones; `none` empties one (`--flowtable none`: off) |
|
||||
|
||||
### probation, history, import
|
||||
|
||||
@@ -146,10 +232,11 @@ See [nat.md](nat.md).
|
||||
| `confirm` | keep the changes on probation; the ruleset is saved for the boot |
|
||||
| `rollback` | undo the changes on probation now |
|
||||
| `history list [--limit N]` | changes, newest first |
|
||||
| `history show ID` | each object before and after |
|
||||
| `history show ID` | each object before and after (a changed setting by its own fields: `builtins.drop_invalid: false -> true`) |
|
||||
| `history diff ID` | what the change did to the ruleset |
|
||||
| `history undo ID` | apply the change backwards (refused if something it touched has changed since) |
|
||||
| `import [FILE\|-] [--text FILE] [--defines FILE] [--table T] [--replace]` | [import.md](import.md) |
|
||||
| `takeover TABLE... [-y]` | delete tables Housecarl's ruleset replaces (administrators) |
|
||||
| `import [FILE\|-] [--text FILE] [--defines FILE] [--table T] [--zone IFACE=NAME]... [--replace]` | [import.md](import.md) |
|
||||
| `takeover TABLE... [-y]` | delete tables Housecarl's ruleset replaces (administrators); not in shadow mode |
|
||||
|
||||
### cert
|
||||
|
||||
|
||||
+35
-3
@@ -14,7 +14,7 @@ reads the file when it starts.
|
||||
| `tls.cert_file`, `tls.key_file` | none | serve HTTPS with this certificate; housecarld rereads it when the file changes, or on `systemctl reload housecarl` |
|
||||
| `tls.acme.*` | off | get the certificate from Let's Encrypt, with Jarl publishing the challenge ([certificates.md](certificates.md#settings)) |
|
||||
| `jarl.server`, `jarl.token_file` (or `jarl.token`), `jarl.ca_cert`, `jarl.insecure` | none | the Jarl server for the certificate's challenge, and its token ([jarl.md](jarl.md)) |
|
||||
| `ruleset` | `/var/lib/housecarl/ruleset.nft` | the confirmed ruleset, which `housecarl-ruleset.service` loads at boot |
|
||||
| `ruleset` | `/var/lib/housecarl/ruleset.nft` | the confirmed ruleset, which `housecarl-ruleset.service` loads at boot; its fallback is saved beside it as `ruleset-fallback.nft` ([operations.md](operations.md#if-the-ruleset-fails-to-load-at-boot)). Change both paths in the unit too |
|
||||
| `trusted_proxies` | `[]` | addresses or prefixes of reverse proxies whose `X-Forwarded-For` and `X-Forwarded-Proto` are believed |
|
||||
| `socket` | `/run/housecarl/housecarl.sock` | the local API socket; `""` turns it off |
|
||||
| `database` | `/var/lib/housecarl/housecarl.db` | the SQLite database: configuration, history, users, audit log |
|
||||
@@ -26,11 +26,43 @@ reads the file when it starts.
|
||||
| `backup.dir` | `/var/backups/housecarl` | where `housecarld backup` writes |
|
||||
| `backup.retention` | `30d` | backups older than this are deleted… |
|
||||
| `backup.keep_at_least` | `7` | …keeping at least this many |
|
||||
| `stats.enabled` | `true` | keep traffic, drops, hits, blocked sources, connections and quotas over time ([stats.md](stats.md)) |
|
||||
| `stats.database` | `/var/lib/housecarl/stats.db` | the statistics' own database; deleting it is safe |
|
||||
| `stats.retention.minute`, `.hour`, `.day` | `7d`, `90d`, `730d` | how long per-minute, hourly and daily figures are kept |
|
||||
| `stats.top_n` | `100` | entries kept per top list, per hour and per day |
|
||||
| `shadow.enabled` | `false` | shadow mode: Housecarl's ruleset is loaded beside the firewall you have, but blocks, rejects and translates nothing; its counters, log and tracer show what it would do. No change is on probation, and takeovers and closing connections are refused ([shadow.md](shadow.md)) |
|
||||
| `shadow.position` | `after` | where Housecarl's filter chains run in shadow mode: `after` the other firewall's (`filter + 10`: it sees only what that one lets through, so what it would drop is what switching would break) or `before` them (`filter - 10`: it sees every packet) |
|
||||
| `names.enabled` | `true` | show inside addresses' names beside them, found by reverse lookups (below) |
|
||||
| `names.resolver` | `""` | the DNS server to ask, such as `10.10.0.1:53`; empty: the system's (`/etc/hosts`, then `/etc/resolv.conf`) |
|
||||
| `names.cache` | `10m` | how long a name, or the lack of one, is kept |
|
||||
|
||||
Durations are written `15s`, `12h` or `30d`.
|
||||
|
||||
Shadow mode is here, and not a firewall setting, so that no user or API
|
||||
token can switch enforcement off; restart housecarld to switch.
|
||||
|
||||
### Names
|
||||
|
||||
The web UI and `hc` show a name beside each **inside** address in
|
||||
connections, the log, quotas and traffic per device: `laptop
|
||||
192.168.1.20`. Housecarl finds it with a reverse (PTR) lookup, so the
|
||||
names are your reverse zones' (and `/etc/hosts`'). An address counts as
|
||||
inside when it is in a network connected to an inside zone's interface,
|
||||
or private (`10.0.0.0/8`, `172.16.0.0/12`, `192.168.0.0/16`, `fc00::/7`)
|
||||
and not in an external zone's network. Outside addresses get no name:
|
||||
their reverse zones belong to whoever holds the addresses, who would
|
||||
choose the names, and looking up every scanner is slow.
|
||||
|
||||
Names are looked up when a page or `hc` asks, cached (a failed lookup is
|
||||
tried again after a minute), and never stored: statistics stay by
|
||||
address. With a split DNS (views), the server asked must answer the
|
||||
firewall as an inside client; set `names.resolver` if the system's
|
||||
resolver doesn't. IPv6 traffic per device is counted per /64, which has
|
||||
no name. `hc --no-names` leaves them out.
|
||||
|
||||
The probation's length is a firewall setting, not here: `hc settings set
|
||||
--probation 5m`.
|
||||
--probation 5m`. So is counting traffic per device for the statistics:
|
||||
`hc settings set --stats-per-device on`.
|
||||
|
||||
## HTTPS
|
||||
|
||||
@@ -40,7 +72,7 @@ Let it get its own certificate through Jarl ([certificates.md](certificates.md))
|
||||
or give it one:
|
||||
|
||||
```yaml
|
||||
listen: ["10.0.10.1:8070"]
|
||||
listen: ["192.168.1.1:8070"]
|
||||
tls:
|
||||
cert_file: /etc/housecarl/tls/fullchain.pem
|
||||
key_file: /etc/housecarl/tls/privkey.pem
|
||||
|
||||
+83
-1
@@ -50,6 +50,35 @@ Every change goes through one engine (`internal/engine`):
|
||||
Dry runs, history with undo, users, sessions and scoped API tokens come
|
||||
from the same machinery as in Jarl and Steward.
|
||||
|
||||
## Reviewing changes: changesets
|
||||
|
||||
As in Jarl, a change can be made at once or staged in a **changeset**,
|
||||
to be reviewed and applied together with others:
|
||||
|
||||
* Each change is an **operation** (create, update, delete or move an
|
||||
object, or change the settings), kept as the API received it.
|
||||
* Staged in a changeset, an operation is checked as it is added: the
|
||||
changeset with it must still be valid, and nftables must accept the
|
||||
ruleset it makes. Nothing is applied.
|
||||
* The **review** shows the operations, the problems, and the **diff of
|
||||
the ruleset**: the file that is loaded at boot, now and after.
|
||||
* **Applying** replays the operations on the configuration as it is then,
|
||||
as one change: one check, one transaction, one history entry, undone as
|
||||
a whole. An operation that no longer applies (its object was changed or
|
||||
deleted since) is named, and can be taken out.
|
||||
* Then the applied changeset is **on probation** like any change made over
|
||||
the network: unless it is confirmed, it is rolled back.
|
||||
|
||||
| Interface | Default | Why |
|
||||
|---|---|---|
|
||||
| Web UI | Review edits: they are collected in a draft changeset, pages show pending rows (green new, yellow changed, struck out deleted), and a bar leads to the Changes page, with the diff and Apply. The switch in the header turns on Live edits, applied at once. | Several related edits, seen together before they are in force. |
|
||||
| CLI | Immediate: each command is a change of its own. Stage with `-C ID` or `HOUSECARL_CHANGESET`. | Scripts want a clear result per step. |
|
||||
| API | Immediate unless a changeset is named (`?changeset=ID`, `X-Housecarl-Changeset`). | As the CLI. |
|
||||
|
||||
An undo is staged like any edit (in review mode, the web UI's Undo
|
||||
collects it for review). Imports and takeovers can't be staged; with a
|
||||
changeset named, they are refused rather than applied.
|
||||
|
||||
## Its own table only
|
||||
|
||||
Housecarl writes one table, `inet housecarl`, and leaves every other
|
||||
@@ -71,13 +100,53 @@ Every rule carries a comment `hc:<mark>`, such as `hc:r12` for rule 12,
|
||||
`hc:f3` for port forward 3, `hc:b-established` for a built-in. Counters
|
||||
and log lines are traced back to their object through it.
|
||||
|
||||
## Shadow mode
|
||||
|
||||
With `shadow.enabled` in the configuration file ([shadow.md](shadow.md)),
|
||||
the same configuration is rendered so that it changes nothing (render's
|
||||
`Options.Shadow`): drops and rejects accept instead, policies accept,
|
||||
NAT only counts, and what alters packets (MSS clamping, the flowtable)
|
||||
and raw rules that do more than accept, drop or reject are left out.
|
||||
This rests on how nftables runs base chains: an `accept` only ends its
|
||||
own chain, and the packet goes on to the next one on the hook, other
|
||||
tables' included; only a drop is final. Another table, the real
|
||||
firewall, decides.
|
||||
|
||||
What would be dropped, rejected or translated is counted under the same
|
||||
marks, and its comment says so (`"hc:r12 [would drop] …"`), which is
|
||||
how the tracer tells it apart. The filter chains run at `filter + 10`
|
||||
(after the other firewall's: what Housecarl would drop is what switching
|
||||
over would break) or `- 10`; the NAT chains always at `- 10`, since the
|
||||
kernel asks NAT chains only until one translates.
|
||||
|
||||
Every change is checked by nftables in both forms, so leaving shadow
|
||||
mode can't fail on the ruleset. There is no probation, since nothing
|
||||
can lock anyone out; takeovers and closing connections, which would
|
||||
change traffic, are refused. A ruleset reload elsewhere (`flush
|
||||
ruleset`) deletes Housecarl's table too: housecarld listens to nftables'
|
||||
netlink notifications (internal/tablewatch) and puts it back.
|
||||
|
||||
## Not locking yourself out
|
||||
|
||||
A change made over the network is applied **on probation**: unless it is
|
||||
A change made over the network is applied **on probation** (a changeset
|
||||
when it is applied, a live edit at once): unless it is
|
||||
confirmed within two minutes (Settings; `hc confirm`, the banner in the
|
||||
web UI, `POST /confirm`), it is rolled back by itself, and the rollback is
|
||||
recorded in the history. A change that cuts you off undoes itself.
|
||||
|
||||
* **The client that made a change confirms it by itself** once it
|
||||
reaches the API again: the change's answer carries a one-time token and
|
||||
closes its connection (HTTP/2: a GOAWAY), and the web UI and `hc` send
|
||||
the token straight back. That proves something only on a new
|
||||
connection, since the built-in that accepts established connections
|
||||
would let the old one through whatever the change did. So the token is
|
||||
accepted only from the same user (and API token), the same address, a
|
||||
different connection, and for the newest change. A change that cuts its
|
||||
client off can't be confirmed by it, and rolls back. This proves the
|
||||
API is still reachable, not that SSH or the LAN still work; with the
|
||||
API, you can fix those. Settings → "a person confirms each change"
|
||||
(`hc settings set --auto-confirm off`) goes back to confirming by hand;
|
||||
`hc --no-confirm` leaves one change waiting.
|
||||
* Changes made through the local socket (`sudo hc …` on the firewall)
|
||||
need no confirmation: whoever makes them is already on the machine. One
|
||||
made while others are pending joins them.
|
||||
@@ -96,6 +165,19 @@ firewall is up even if housecarld can't start. An unconfirmed ruleset is
|
||||
never saved there: a reboot during probation brings back the last
|
||||
confirmed one.
|
||||
|
||||
It fails closed. A fallback saved beside it (`ruleset-fallback.nft`,
|
||||
`render.Fallback`) is loaded if it doesn't: no sets, no flowtable,
|
||||
interfaces by name only, so there is little in it to fail. It keeps
|
||||
anything new from the external zones out and lets the inside work,
|
||||
which is as far as a ruleset can go without knowing the rules; a
|
||||
fallback that let everything in would leave a router open on the
|
||||
internet, and one that let nothing in would lock out the person who has
|
||||
to fix it. The boot ruleset has no flowtable either: nftables refuses a
|
||||
flowtable naming a device that doesn't exist yet (on Debian 13's
|
||||
kernel), and before the network is up, `ppp0` doesn't. housecarld
|
||||
watches the kernel's link messages (internal/linkwatch) and applies
|
||||
again when a flowtable device appears, or comes back as a new interface.
|
||||
|
||||
## The drop log
|
||||
|
||||
Rules with logging on, and the policies (Settings), send packets to
|
||||
|
||||
+7
-2
@@ -11,6 +11,8 @@ internal/
|
||||
render/ State → the nftables ruleset text, with each line's object
|
||||
nft/ runs nft: check, apply (keeping dynamic set entries and counters), list, the boot file
|
||||
nflog/ reads logged packets from netlink log group 4171
|
||||
linkwatch/ netlink: interfaces that appear or go (the flowtable's devices)
|
||||
tablewatch/ netlink: a table deleted (shadow mode puts Housecarl's back; docs/shadow.md)
|
||||
importer/ nft -j list ruleset → State, with notes and raw rules (docs/import.md)
|
||||
hostif/ the machine's interfaces
|
||||
store/ SQLite (modernc.org/sqlite, no cgo): configuration, history, users, tokens, sessions, audit
|
||||
@@ -70,8 +72,11 @@ Newer nftables accepts things 1.1.3 doesn't (a meter with a timeout for
|
||||
`wan`, `enp2s0` and `wg0` inside it, IPv4 and IPv6;
|
||||
* "the internet" on the WAN side, a LAN client, and a "jellyfin" host.
|
||||
|
||||
It imports a copy of the maintainer's ruleset (lab addresses), takes
|
||||
over the old table, and checks with real traffic: port forwards with
|
||||
It imports a copy of the maintainer's ruleset (lab addresses) with
|
||||
housecarld in shadow mode beside it (traffic unchanged, a would-be drop
|
||||
logged and let through, the table back after the old ruleset is
|
||||
reloaded), restarts it enforcing, takes over the old table, and checks
|
||||
with real traffic: port forwards with
|
||||
hairpin NAT and the IPv6 pinhole, masquerading, antispoofing, the drop
|
||||
log, a lockout change that rolls back by itself, a restart, the boot
|
||||
ruleset after a flush, and that other tables are left alone. It makes
|
||||
|
||||
@@ -0,0 +1,187 @@
|
||||
# The flowtable: a fast path for forwarded traffic
|
||||
|
||||
The flowtable is a shortcut inside the kernel for traffic the firewall
|
||||
**forwards** (between the internet and your LAN, between zones). It
|
||||
speeds up routing on a machine whose CPU can't keep up with its line.
|
||||
It doesn't make the firewall any safer or less safe, but it changes what
|
||||
some features can see, so read [when not to use it](#when-not-to-use-it)
|
||||
before you turn it on.
|
||||
|
||||
It is off until you name its devices.
|
||||
|
||||
## What it does
|
||||
|
||||
Without a flowtable, every forwarded packet takes the whole way through
|
||||
the kernel: the prerouting hooks, the routing decision, Housecarl's
|
||||
forward chain (the built-ins, then your rules) and postrouting with NAT.
|
||||
For an established connection almost all of that work only ends in the
|
||||
`established` built-in accepting it again, packet after packet.
|
||||
|
||||
With a flowtable, Housecarl adds one built-in to the forward chain:
|
||||
|
||||
```
|
||||
meta l4proto { tcp, udp } flow add @ft
|
||||
```
|
||||
|
||||
A TCP or UDP connection takes the whole way for its first packets, as
|
||||
always: your rules decide whether it is allowed at all. Once the kernel
|
||||
has seen it established (and your rules let it through), it is added
|
||||
to the flowtable. From then on its packets are caught as soon as they
|
||||
arrive on one of the flowtable's devices (the *ingress* hook) and sent
|
||||
straight out on the other side: NAT is still applied, the TTL still
|
||||
counts down, but every chain in between is skipped.
|
||||
|
||||
```
|
||||
first packets every packet after
|
||||
in ─► prerouting ─► routing ─► forward ─► postrouting ─► out
|
||||
│ ▲
|
||||
└────────────── flowtable (ingress) ─────────────────────┘
|
||||
```
|
||||
|
||||
It is the software flowtable, run by the CPU. Offloading to the network
|
||||
card's hardware isn't supported ([design.md](design.md#not-in-01)).
|
||||
|
||||
## When to use it
|
||||
|
||||
Turn it on when **the router's CPU is what limits your speed**:
|
||||
|
||||
* a speed test through the router gets less than your line gives (or
|
||||
less than the same test from the router itself), and
|
||||
* during it, one CPU core is busy with network work: in `top`, press `1`
|
||||
for every core, and look at `si` (softirq) near 100 % on a core.
|
||||
|
||||
Typical cases: a small or old machine (Atom, Celeron, a Raspberry Pi, an
|
||||
old Core 2) on a fast line (500 Mbit/s and up), lots of small packets
|
||||
(gaming, VoIP, many downloads at once), or a machine that is busy with
|
||||
other work besides routing. The gain is CPU time per packet: often well
|
||||
over half of the forwarding cost, which shows up as speed only when the
|
||||
CPU was the limit, and otherwise as a cooler, quieter, more responsive
|
||||
machine.
|
||||
|
||||
In shadow mode ([shadow.md](shadow.md)) Housecarl leaves its flowtable
|
||||
out: an offloaded connection skips every table's forward chain, the
|
||||
other firewall's too. If that firewall has a flowtable of its own, the
|
||||
same goes for Housecarl's forward chain: it only sees each connection's
|
||||
first packets.
|
||||
|
||||
## When not to use it
|
||||
|
||||
**When the CPU keeps up anyway.** If the speed test already reaches your
|
||||
line and `si` stays low, the flowtable buys you nothing you'd notice,
|
||||
and costs you the figures below. Most modern x86 machines route a
|
||||
gigabit line without it.
|
||||
|
||||
**When you count or limit traffic in the forward chain.** The fast path
|
||||
skips the forward chain, so whatever must see *every* packet of a
|
||||
connection only sees the first few. Housecarl warns about each:
|
||||
|
||||
| Feature | With the flowtable | Warning |
|
||||
|---|---|---|
|
||||
| quotas and byte-rate limits on forward rules | only count first packets: a device can go far past its allowance | `flowtable_traffic` |
|
||||
| forward rules with "every packet" (`--every-packet`) | only see first packets | `flowtable_traffic` |
|
||||
| statistics' traffic per device (`stats_per_device`) | only count first packets | `stats_flowtable` |
|
||||
| hit counters of the `established` built-in (forward) | stop rising once a connection is offloaded | |
|
||||
|
||||
Rate limits by packets (`--limit 10/second`) and connection limits act
|
||||
on new connections, so they work the same.
|
||||
|
||||
**When other tables must see forwarded traffic.** Offloaded packets skip
|
||||
the forward hooks of *every* table, not only Housecarl's: Docker's,
|
||||
Incus', libvirt's or a table of your own. Those normally only judge new
|
||||
connections, which still pass through them, but a table that counts or
|
||||
shapes established traffic in its forward chain won't see it.
|
||||
|
||||
What keeps working as before:
|
||||
|
||||
* **Your rules** decide every new connection: nothing gets offloaded
|
||||
that your rules didn't let through. A rule you add later doesn't stop
|
||||
a connection already open, but that is true without the flowtable too
|
||||
(the `established` built-in accepts it).
|
||||
* **NAT**: port forwards, hairpin NAT, masquerading and SNAT.
|
||||
* **The input and output chains**: traffic to and from the firewall
|
||||
itself never uses the flowtable.
|
||||
* **Closing connections** (`hc connections close`, Traffic →
|
||||
Connections) ends their fast path too: the kernel drops a flow with
|
||||
its tracking entry.
|
||||
* **Interface statistics** (traffic per zone and interface) come from the
|
||||
interfaces' own counters, which count every packet.
|
||||
* **Traffic shaping with `tc`** on the outgoing interface still applies.
|
||||
* **The log** and **the packet tracer** see what goes through the rules:
|
||||
new connections, as always. The tracer doesn't see the packets of an
|
||||
offloaded connection; trace a new one.
|
||||
* ICMP, and protocols other than TCP and UDP (GRE, ESP, SCTP), always
|
||||
take the normal way.
|
||||
|
||||
## How to turn it on
|
||||
|
||||
Name the devices whose traffic should take the fast path, **both
|
||||
sides**: the interface connections come in on and the one their replies
|
||||
come in on. For a router that means the external interface and the
|
||||
inside ones:
|
||||
|
||||
```sh
|
||||
hc settings set --flowtable wan --flowtable enp2s0
|
||||
hc settings # Flowtable: wan, enp2s0
|
||||
```
|
||||
|
||||
In the web UI: Settings → Flowtable, add each device. The fast path
|
||||
only catches packets arriving on a listed device, so a connection to or
|
||||
from an unlisted one gains little or nothing: leave a device out only on
|
||||
purpose (say, a WireGuard interface whose traffic you'd rather keep on
|
||||
the normal way).
|
||||
|
||||
* Use **exact device names**, not wildcards (`incusbr*` is refused):
|
||||
the kernel attaches the flowtable to each device by name.
|
||||
* With **PPPoE**, name the PPP interface (`ppp0`, or the name your
|
||||
`ppp@` unit gives it), the one in your external zone.
|
||||
* `--flowtable` replaces the whole list; `hc settings set --flowtable
|
||||
none` turns the flowtable off.
|
||||
|
||||
The change is applied (and on probation, if made over the network) like
|
||||
any other.
|
||||
|
||||
### Devices that come and go
|
||||
|
||||
PPP and WireGuard interfaces don't exist until their service brings
|
||||
them up, and PPPoE makes a new `ppp0` each time it reconnects. Housecarl
|
||||
handles that:
|
||||
|
||||
* A device that doesn't exist yet is left out of the flowtable, with the
|
||||
warning `flowtable_device_missing`, instead of failing the whole
|
||||
ruleset.
|
||||
* housecarld watches the kernel's interface changes: when a flowtable
|
||||
device appears, or comes back as a new interface after a reconnect,
|
||||
it applies the ruleset again with it, a couple of seconds later. The
|
||||
journal says `interfaces changed: applied the ruleset again`.
|
||||
Applying again keeps the counters, dynamic sets and quotas, as any
|
||||
change does, and adds no history entry.
|
||||
* The ruleset saved for boot has no flowtable at all: it loads before
|
||||
the network is up, when `ppp0` and `wg0` can't exist yet, and
|
||||
nftables on Debian 13's kernel refuses a flowtable naming a device
|
||||
that doesn't exist (and with it the whole file). housecarld adds the
|
||||
flowtable once it starts. Until then, traffic takes the normal way,
|
||||
through the same rules.
|
||||
|
||||
## Checking it works
|
||||
|
||||
```sh
|
||||
sudo nft list flowtables # the devices it is attached to
|
||||
sudo conntrack -L | grep OFFLOAD # connections on the fast path
|
||||
```
|
||||
|
||||
`conntrack` is in Debian's `conntrack` package. A long download through
|
||||
the router should show up as `[OFFLOAD]`. Compare a speed test, and `si`
|
||||
in `top`, before and after.
|
||||
|
||||
If nothing gets offloaded, check that both of the connection's devices
|
||||
are listed, that `flowtable_device_missing` isn't in `hc status`, and
|
||||
that the connection is TCP or UDP and established.
|
||||
|
||||
## Turning it off
|
||||
|
||||
```sh
|
||||
hc settings set --flowtable none
|
||||
```
|
||||
|
||||
Connections on the fast path go back to the normal way with their next
|
||||
packet.
|
||||
+111
-18
@@ -18,11 +18,9 @@ shows what it would make, without saving anything: zones, sets,
|
||||
services, rules, port forwards and NAT, then **notes** on every guess it
|
||||
made, and the rules it **kept raw**.
|
||||
|
||||
`--defines` reads your ruleset file's `define` lines, so sets get your
|
||||
names (`define trusted_ips = { ... }` becomes the set `trusted_ips`).
|
||||
Without it, sets are named after what they hold. Interfaces named by a
|
||||
define name their zone too (`define wan_interface = "ppp0"` makes the
|
||||
zone `wan`); otherwise a zone is named after its interface.
|
||||
`--defines` reads your ruleset file's `define` lines, so sets and zones
|
||||
get your names ([How things are named](#how-things-are-named)).
|
||||
`--zone enp2s0=lan` names a zone yourself.
|
||||
|
||||
From another machine, give the JSON as a file, and the text with
|
||||
`--text`:
|
||||
@@ -37,6 +35,46 @@ With several tables, `--table "inet main"` picks one; by default every
|
||||
table but Housecarl's is read. The web UI has the same under Server →
|
||||
Import a ruleset.
|
||||
|
||||
## How things are named
|
||||
|
||||
**Zones** are named in this order:
|
||||
|
||||
1. **As you say**: `--zone enp2s0=lan` (repeat it; Zone names on the
|
||||
web UI's Import page). Interfaces given the same name share one zone
|
||||
(`--zone enp2s0=lan --zone enp3s0=lan`).
|
||||
2. **After a define holding the interface**, with a trailing `_int`,
|
||||
`_if`, `_iface`, `_ifname`, `_interface`, `_dev`, `_device` or `_nic`
|
||||
taken off: `define wan_int = "ppp0"` makes the zone `wan`, `define
|
||||
iot_if = enp2s0.20` (a VLAN) the zone `iot`.
|
||||
3. **After the interface itself**: `enp2s0`.
|
||||
|
||||
If the name is taken, the zone is called `INTERFACE_zone`. A define
|
||||
names what it was *for* in your old ruleset, which isn't always what the
|
||||
network is (`define dhcp_int = enp2s0` makes `dhcp`, for your LAN). A
|
||||
note lists each zone and where its name came from; rename one with
|
||||
`--zone` and import again, or afterwards with `hc zone update dhcp
|
||||
--name lan` (its rules follow). Name zones for what the network is for:
|
||||
`lan`, `iot`, `guest`, `servers`, `vpn`, `wan`.
|
||||
|
||||
**Address sets**: a named set of the table keeps its name; a define
|
||||
holding a list of addresses, or one prefix, gives its name to a set
|
||||
(`define trusted = { 10.0.10.0/24, 10.1.1.0/24 }` makes `trusted`,
|
||||
wherever those addresses appear together). A single address stays an
|
||||
address. When two defines hold the same addresses, the first by name
|
||||
wins.
|
||||
|
||||
**Services**: a define holding several ports names a service (`define
|
||||
web_ports = { 80, 443 }` makes `web_ports`); one port stays a port
|
||||
(`tcp/22`).
|
||||
|
||||
**Port forwards** are named after their comment, without a leading
|
||||
"Port forward" (`comment "Port Forward Jellyfin"` makes `jellyfin`);
|
||||
several ports in one DNAT become one forward each (`web-80`, `web-443`),
|
||||
and a DNAT map's elements are named after the map (`forwards-443`).
|
||||
|
||||
Names that aren't valid are made so: other characters become `_` (`-`
|
||||
in forward names), and a name must start with a letter.
|
||||
|
||||
## What becomes what
|
||||
|
||||
* **Interfaces** become zones. Zones are marked external when the rules
|
||||
@@ -45,27 +83,69 @@ Import a ruleset.
|
||||
* **Sets and defines** of addresses become address sets; port lists
|
||||
become services.
|
||||
* **Jumps and gotos** to your own chains are flattened: the jump's
|
||||
condition is added to each rule of the chain, in place.
|
||||
condition is added to each rule of the chain, in place. When the jump
|
||||
and a rule both match sources (or destinations), the narrower is kept
|
||||
if one lies within the other; otherwise, and for any rule there that
|
||||
stays raw, the jump's condition is written in front of the raw rule's
|
||||
text, so it still only applies where the jump did.
|
||||
* **Rules** keep their order. A rule that is a built-in (loopback,
|
||||
established, invalid, the ICMPv6 rules, MSS clamping, the DHCPv6 client
|
||||
rule, antispoofing) becomes that built-in only when it sits where the
|
||||
built-in goes, at the head of every chain it covers. Otherwise it stays
|
||||
a rule where it was, so the ruleset does what it did. Built-ins your
|
||||
ruleset doesn't have are turned off.
|
||||
a rule where it was, so the ruleset does what it did. Since built-ins
|
||||
come before every rule, once one at a chain's head stays a rule, the
|
||||
ones after it there stay rules too, in order: an invalid drop that
|
||||
stays a rule keeps the ICMPv6 accepts after it behind it, as they were.
|
||||
Built-ins your ruleset doesn't have are turned off.
|
||||
* **DNAT** rules become port forwards, TCP and UDP of the same port
|
||||
merged into one. The forward-chain accepts that let them through
|
||||
become part of the port forward, and a masquerade for inside sources
|
||||
to the same host becomes its hairpin. An IPv6 accept to the same port
|
||||
becomes the forward's IPv6 pinhole when there is only one candidate.
|
||||
merged into one, one per port when a rule has several. The
|
||||
forward-chain accepts that let them through (`ct status dnat`) become
|
||||
part of the port forward; an accept to the same host and port without
|
||||
`ct status dnat` lets in more than forwarded connections, and stays a
|
||||
rule. A masquerade for inside sources to the same host becomes its
|
||||
hairpin. An IPv6 accept to the same port, from everywhere, becomes the
|
||||
forward's IPv6 pinhole when there is only one candidate. A DNAT to an
|
||||
IPv6 address, or with matches a port forward has no field for, is left
|
||||
out with a note.
|
||||
* **Maps are expanded**, one element at a time. A DNAT map
|
||||
(`dnat ip to tcp dport map @forwards`, named or written in the rule)
|
||||
becomes one port forward per element; forwards from a named map are
|
||||
called after it (`forwards-443`). A verdict map
|
||||
(`tcp dport vmap { 22 : accept, 23 : drop }`) becomes one rule per
|
||||
element, in place, with `jump`s inlined as above. A lookup matches at
|
||||
most one element, so the rules do the same. A map keyed by several
|
||||
fields (`ip saddr . tcp dport`) can't be expanded: a rule using a named
|
||||
one is left out with a note, to be added by hand.
|
||||
* **Masquerade and SNAT** become source NAT; **redirect** becomes a
|
||||
redirect.
|
||||
* **Meters** (`update @m { ip saddr limit rate over 3/minute }`,
|
||||
`add @m { ip saddr ct count over 20 }`, quotas in sets) become the
|
||||
rule's per-source (or per-destination) limit, connection limit or quota;
|
||||
a mask in the key (`ip saddr and 255.255.255.0`) becomes its grouping.
|
||||
Byte rates and quotas are imported too; a quota imported resets only by
|
||||
hand until you give it a period. A rule with a quota or a byte rate that
|
||||
accepts stays a raw rule ([rules.md](rules.md#bandwidth-and-quotas)).
|
||||
* **Sets with `flags timeout`** (filled by a program or a rule) become
|
||||
dynamic sets, without their entries of the moment: whatever fills them
|
||||
must fill Housecarl's from then on, as a note says.
|
||||
* **Policies** become the chains' policies.
|
||||
* **Anything else** becomes a raw rule with its original text, in place.
|
||||
A raw rule naming a set of the old table needs the new set name:
|
||||
Housecarl's are `NAME_v4` and `NAME_v6`, and a note says so.
|
||||
* **Anything else** becomes a raw rule with its original text, in place:
|
||||
a match Housecarl has no field for, two matches on the same field
|
||||
(`ip saddr 10.0.0.0/8 ip saddr != 10.0.10.0/24`), `!=` where Housecarl
|
||||
only has `==`, a set of interface names. Its set references are made
|
||||
to work in Housecarl's table: an imported address set becomes
|
||||
`@NAME_v4` or `@NAME_v6`, any other set (interface names, ports) is
|
||||
written out with its elements. A raw rule naming a set that can't be
|
||||
(a meter, an empty set) is left out with a note.
|
||||
* **Logging rules** log to Housecarl's log (`hc log`), not the kernel's.
|
||||
|
||||
Chains on hooks Housecarl doesn't manage (ingress, a bridge table) are
|
||||
left out, with a note.
|
||||
Several base chains on one hook (`input` at priority -10 and at 0) are
|
||||
put one after the other, by priority, into Housecarl's one chain, with
|
||||
the stricter policy; a note says to check it, since a packet had to get
|
||||
past each, and an accept in the first no longer ends there. A prerouting
|
||||
filter chain after connection tracking becomes the early chain, which
|
||||
runs before it: a note says so. Chains on hooks Housecarl doesn't manage
|
||||
(ingress, a bridge table) are left out, with a note.
|
||||
|
||||
## 2. Import
|
||||
|
||||
@@ -79,6 +159,11 @@ Housecarl's table is now in force **alongside** the old one: a packet
|
||||
must be accepted by both. Nothing should change for your traffic. If
|
||||
Housecarl already has a configuration, `--replace` replaces it.
|
||||
|
||||
To be sure first, import in **shadow mode** ([shadow.md](shadow.md)):
|
||||
Housecarl's ruleset is then loaded but drops nothing, and its log shows
|
||||
each packet the old ruleset lets through that Housecarl would drop. When
|
||||
there are none you mind, leave shadow mode and take over.
|
||||
|
||||
Imported over the network (not through the local socket), the import is
|
||||
on probation like any change: confirm it.
|
||||
|
||||
@@ -91,7 +176,10 @@ the old table:
|
||||
sudo hc takeover "inet main"
|
||||
```
|
||||
|
||||
From now on only Housecarl's rules apply. Over the network, this is on
|
||||
From now on only Housecarl's rules apply. Until you do, the dashboard
|
||||
and `hc status` warn that the imported table still applies
|
||||
(`takeover_pending`); they also warn while `nftables.service` is enabled
|
||||
(`nftables_service`). Over the network, this is on
|
||||
probation too: unless confirmed, the old table comes back.
|
||||
|
||||
## 4. The boot
|
||||
@@ -106,6 +194,11 @@ sudo systemctl disable nftables.service
|
||||
sudo mv /etc/nftables.conf /etc/nftables.conf.before-housecarl
|
||||
```
|
||||
|
||||
Disable it, but **don't stop it** (no `systemctl stop nftables`, no
|
||||
`disable --now`): Debian's unit stops by running `nft flush ruleset`,
|
||||
which deletes every table, Housecarl's and Docker's included, at once.
|
||||
Disabled, it simply doesn't run at the next boot.
|
||||
|
||||
`housecarl-ruleset.service` runs after `nftables.service` in any case,
|
||||
so a leftover `flush ruleset` can't undo it. Docker, Incus and libvirt
|
||||
keep adding their own tables as before.
|
||||
|
||||
+7
-3
@@ -85,20 +85,24 @@ you are changing the firewall you are connected through.
|
||||
sudo systemctl disable nftables.service
|
||||
```
|
||||
|
||||
To try Housecarl beside your firewall first, turn on **shadow mode**
|
||||
before the import: it loads Housecarl's ruleset but blocks nothing,
|
||||
and shows what it would do ([shadow.md](shadow.md)).
|
||||
|
||||
**Or start from scratch**, through the local socket (no probation).
|
||||
The first change puts the ruleset in force, and the input and forward
|
||||
policies drop what no rule accepts, so let yourself in first:
|
||||
|
||||
```sh
|
||||
sudo hc set add admins 10.0.10.0/24
|
||||
sudo hc set add admins 192.168.1.0/24
|
||||
sudo hc rule add input --from admins --service ssh
|
||||
sudo hc rule add input --from admins --service tcp/8070 --comment "Housecarl"
|
||||
sudo hc zone add wan --interface ppp0 --external
|
||||
sudo hc zone add lan --interface enp2s0
|
||||
sudo hc zone add lan --interface eth1
|
||||
sudo hc rule add input --in lan --service dns --service dhcp
|
||||
sudo hc rule add forward --in lan --out wan
|
||||
sudo hc snat add --out wan
|
||||
sudo hc settings set --builtin mss_clamp=on --own-prefix 2001:db8:7f3d::/48
|
||||
sudo hc settings set --builtin mss_clamp=on --own-prefix 2001:db8::/48
|
||||
sudo hc status
|
||||
```
|
||||
|
||||
|
||||
+11
-7
@@ -7,10 +7,14 @@ public port are sent to the host, and let through the forward chain.
|
||||
One object does what takes three rules by hand (the DNAT, the forward
|
||||
accept, and the hairpin masquerade).
|
||||
|
||||
In shadow mode ([shadow.md](shadow.md)) NAT rules translate nothing:
|
||||
they count the connections they would translate, and the other
|
||||
firewall's NAT does the work.
|
||||
|
||||
```sh
|
||||
hc forward add 8096 --to 10.0.10.57:443 --public 203.0.113.7 --hairpin --name jellyfin
|
||||
hc forward add 7777-7780 --proto tcp,udp --to 10.0.10.76 --public 203.0.113.7 --hairpin --name terraria
|
||||
hc forward add 25 --to 10.0.10.25 --from mail-relays --name smtp
|
||||
hc forward add 8443 --to 192.168.1.10:443 --public 203.0.113.10 --hairpin --name web
|
||||
hc forward add 27015-27030 --proto tcp,udp --to 192.168.1.20 --public 203.0.113.10 --hairpin --name game
|
||||
hc forward add 25 --to 192.168.1.25 --from mail-relays --name smtp
|
||||
hc forward list
|
||||
```
|
||||
|
||||
@@ -44,10 +48,10 @@ IPv6 needs no translation: inside hosts have public addresses. `--to6`
|
||||
opens the same port (or the `--to` port, if given) to an IPv6 host:
|
||||
|
||||
```sh
|
||||
hc forward add 8096 --to 10.0.10.57:443 --to6 2001:db8:7f3d:10::57 --public 203.0.113.7 --name jellyfin
|
||||
hc forward add 8443 --to 192.168.1.10:443 --to6 2001:db8:1::10 --public 203.0.113.10 --name web
|
||||
```
|
||||
|
||||
IPv6 clients connect to `[2001:db8:7f3d:10::57]:443` directly. A forward
|
||||
IPv6 clients connect to `[2001:db8:1::10]:443` directly. A forward
|
||||
can have only `--to6`, to open an IPv6 port alone.
|
||||
|
||||
## Source NAT
|
||||
@@ -60,7 +64,7 @@ service; the first that matches decides.
|
||||
```sh
|
||||
hc snat add --out wan --from trusted --comment "the LAN to the internet"
|
||||
hc snat add --out wg0 --not-out --from wg-clients
|
||||
hc snat add --out wan --from fd00:abab::/48 --family ipv6 --snat-to 2001:db8:7f3d::1
|
||||
hc snat add --out wan --from fd00:1::/48 --family ipv6 --snat-to 2001:db8::1
|
||||
```
|
||||
|
||||
Source NAT is IPv4 only unless `--family ipv6` says otherwise: with a
|
||||
@@ -73,7 +77,7 @@ A redirect sends matching traffic to a port on the firewall itself, for
|
||||
example to make every client use the firewall's DNS server:
|
||||
|
||||
```sh
|
||||
hc redirect add --in lan --service dns --not-to 10.0.10.5 --comment "force DNS"
|
||||
hc redirect add --in lan --service dns --not-to 192.168.1.5 --comment "force DNS"
|
||||
hc redirect add --in lan --service http --to-port 3128
|
||||
```
|
||||
|
||||
|
||||
+208
-15
@@ -17,10 +17,37 @@ housecarld logs each change it applies, rollbacks, and what it couldn't
|
||||
apply. When it starts, it applies the configuration again (rolling back
|
||||
first what was on probation when it stopped).
|
||||
|
||||
## Reviewing before applying
|
||||
|
||||
In the web UI, edits are collected as pending changes (the bar under the
|
||||
header), and Changes shows them with the diff of the ruleset; Apply puts
|
||||
them in force together. The switch in the header (Review edits / Live
|
||||
edits) applies each edit at once instead. With hc, see
|
||||
[cli.md](cli.md#changesets).
|
||||
|
||||
## Changes on probation
|
||||
|
||||
Changes made over the network roll back unless confirmed
|
||||
([design.md](design.md#not-locking-yourself-out)):
|
||||
([design.md](design.md#not-locking-yourself-out)); in shadow mode, where
|
||||
a change can't cut anyone off, none do ([shadow.md](shadow.md)). The web UI and `hc`
|
||||
confirm their own changes once they reach Housecarl again over a new
|
||||
connection, so normally nothing waits:
|
||||
|
||||
```
|
||||
$ hc rule add input --from admins --service ssh
|
||||
created rule input #31
|
||||
applied and confirmed (history 12): this client reached Housecarl again over a new connection
|
||||
```
|
||||
|
||||
A change that cuts the client off can't be confirmed that way:
|
||||
|
||||
```
|
||||
applied on probation (history 13), but couldn't confirm it: … context deadline exceeded
|
||||
It is rolled back within 1m50s unless confirmed ('hc confirm')
|
||||
```
|
||||
|
||||
By hand (with `--no-confirm`, or with Settings → "a person confirms each
|
||||
change"):
|
||||
|
||||
```sh
|
||||
hc status # Probation: 2 changes roll back in 97s unless confirmed
|
||||
@@ -45,26 +72,101 @@ sudo hc history undo 42
|
||||
|
||||
A confirmed ruleset (or one changed through the socket) is saved to
|
||||
`/var/lib/housecarl/ruleset.nft`, which `housecarl-ruleset.service`
|
||||
loads before the network comes up. To see what would load:
|
||||
loads before the network comes up, so before PPPoE, WireGuard or DHCP
|
||||
bring up their interfaces. To see what would load:
|
||||
|
||||
```sh
|
||||
sudo cat /var/lib/housecarl/ruleset.nft
|
||||
sudo nft -c -f /var/lib/housecarl/ruleset.nft # check it
|
||||
```
|
||||
|
||||
Dynamic sets start empty after a reboot.
|
||||
In shadow mode both files are shadow mode's, and the fallback filters
|
||||
nothing ([shadow.md](shadow.md#what-works-differently)).
|
||||
|
||||
Keep Debian's `nftables.service` disabled, or `/etc/nftables.conf`
|
||||
without `flush ruleset`: otherwise it deletes Housecarl's table at
|
||||
boot. `housecarl-ruleset.service` runs after it, which covers the
|
||||
ordinary case.
|
||||
Dynamic sets start empty after a reboot. The saved ruleset has no
|
||||
flowtable, since its devices may not exist yet: housecarld adds it when
|
||||
it starts, and whenever a flowtable device appears
|
||||
([flowtable.md](flowtable.md#devices-that-come-and-go)).
|
||||
|
||||
### If the ruleset fails to load at boot
|
||||
|
||||
Should `ruleset.nft` fail to load (a damaged file, an nftables or kernel
|
||||
upgrade that refuses something in it), the machine would otherwise boot
|
||||
with no firewall at all until housecarld starts. Instead,
|
||||
`housecarl-ruleset.service` loads a **fallback** that housecarld saves
|
||||
beside it, `/var/lib/housecarl/ruleset-fallback.nft`, from the same
|
||||
confirmed configuration:
|
||||
|
||||
* **Nothing new gets in from the external zones' interfaces**, to the
|
||||
firewall or through it: no port forwards, no SSH or web UI from the
|
||||
internet.
|
||||
* What the firewall and the inside need to keep working is let in:
|
||||
established connections, the ICMPv6 IPv6 needs (neighbour discovery,
|
||||
router advertisements, packet too big), and DHCPv6 replies from
|
||||
link-local addresses.
|
||||
* **Everything else is let through**: the LAN, WireGuard and the
|
||||
firewall itself are reachable from the inside, so you can log in and
|
||||
fix it.
|
||||
* IPv4 leaving an external zone is masqueraded (when any source NAT is
|
||||
configured), so the inside still reaches the internet. MSS clamping
|
||||
stays, if it is on.
|
||||
|
||||
PPPoE itself isn't affected: its discovery and session are Ethernet
|
||||
frames, and the PPP link's negotiation happens before any IP, neither
|
||||
of which an `inet` table sees.
|
||||
|
||||
The fallback only knows which interfaces face the internet from the
|
||||
zones marked **external**. With none, it lets nothing but loopback
|
||||
through, and the warning `no_external_zone` says so.
|
||||
|
||||
When the fallback loads, the unit is marked failed, and the journal says
|
||||
why (as a critical message):
|
||||
|
||||
```sh
|
||||
systemctl status housecarl-ruleset
|
||||
journalctl -b -u housecarl-ruleset
|
||||
```
|
||||
|
||||
housecarld puts the real ruleset back when it starts, and saves a fresh
|
||||
`ruleset.nft`. If it can't apply it either, the fallback stays in force:
|
||||
`hc status` and the dashboard say "NOT in force: the boot fallback is in
|
||||
force", and `/healthz` answers 503 ([Monitoring](#monitoring)). Fix
|
||||
what `hc status` or the journal reports, then restart housecarld.
|
||||
|
||||
Keep Debian's `nftables.service` disabled (`systemctl disable
|
||||
nftables`, without `--now`): at boot it loads `/etc/nftables.conf`, and
|
||||
stopping or restarting it runs `nft flush ruleset`, which deletes every
|
||||
table, Housecarl's included. `housecarl-ruleset.service` runs after it
|
||||
at boot, which covers the ordinary case, but not a restart of
|
||||
`nftables.service` later. Housecarl never reads or writes
|
||||
`/etc/nftables.conf`; its ruleset is saved in
|
||||
`/var/lib/housecarl/ruleset.nft`. The dashboard and `hc status` warn
|
||||
while `nftables.service` is enabled.
|
||||
|
||||
Other programs' tables are left alone: fail2ban's (`inet f2b-table`),
|
||||
Docker's, Incus's. Housecarl replaces only its own table, so they keep
|
||||
their rules through every Housecarl change and restart, and nothing
|
||||
needs restarting in a particular order. A packet must get past every
|
||||
table on its hook, so a drop in any of them wins.
|
||||
|
||||
## Hits
|
||||
|
||||
Every rule, NAT rule, built-in and policy has a counter. `hc rule list`,
|
||||
`hc forward list` and the web UI show packets; `hc counters` shows them
|
||||
all, with bytes. They count from housecarld's start, and carry across
|
||||
changes to the ruleset.
|
||||
changes to the ruleset. A deleted object's counter goes with it; an
|
||||
import with `--replace` gives every object a new ID, so its counters
|
||||
start again. A disabled rule keeps its count for when it is enabled.
|
||||
|
||||
## Quotas
|
||||
|
||||
`hc quotas` (and the rules page) show what each quota used and when it
|
||||
resets; `hc rule reset-quota ID` starts one again (`--address` one device's,
|
||||
`--give 5gbytes` a top-up until the next reset). housecarld resets
|
||||
quotas at midnight when their period ends (late, if it was stopped then),
|
||||
and saves their usage every five minutes, after every change and when
|
||||
it stops; after a reboot it puts it back. A reboot without a clean stop
|
||||
loses at most the last five minutes' count.
|
||||
|
||||
## The log
|
||||
|
||||
@@ -78,25 +180,75 @@ hc log --search 198.51.100.7
|
||||
```
|
||||
|
||||
```
|
||||
10:02:13 hc input #95 drop: wan - 198.51.100.7:53868 -> 203.0.113.7:22 tcp syn
|
||||
10:02:15 hc forward policy drop: wan enp2s0 198.51.100.9:443 -> 10.0.10.20:51432 tcp
|
||||
10:02:13 hc input #95 drop: wan - 198.51.100.7:53868 -> 203.0.113.10:22 tcp syn
|
||||
10:02:15 hc forward policy drop: wan eth1 198.51.100.9:443 -> 192.168.1.20:51432 tcp
|
||||
```
|
||||
|
||||
In shadow mode ([shadow.md](shadow.md)) what the policies would drop is
|
||||
always logged, in all three chains, and drops are marked as what would
|
||||
have been done: the packets went on.
|
||||
|
||||
The packets go to netlink log group 4171, where housecarld reads them;
|
||||
the last 2000 are kept in memory. They aren't written to the kernel log
|
||||
or the journal. A rule's log isn't rate-limited: on a busy rule, give it
|
||||
a limit, or log only new connections (`--state new`).
|
||||
|
||||
## Connections
|
||||
|
||||
The built-in that accepts established connections lets their packets
|
||||
through without checking the rules again: a rule added later doesn't
|
||||
stop a connection already open. Traffic → Connections (`hc connections`)
|
||||
lists what the kernel tracks, with NAT (`to 192.168.1.10:443`: a port
|
||||
forward; `as 203.0.113.10`: masqueraded) and the time left:
|
||||
|
||||
```sh
|
||||
hc connections --search 192.168.1.50
|
||||
hc connections --search ":443 tcp"
|
||||
hc connections close --search 192.168.1.50
|
||||
```
|
||||
|
||||
Closing a connection deletes its tracking entry: its next packet is
|
||||
checked by the rules as they are now. One they block is stopped; one they
|
||||
allow carries on (TCP connections are picked up again mid-stream).
|
||||
|
||||
To make a block stop open connections by itself, tick "every packet" on
|
||||
the rule (`--every-packet`): it is checked before established connections
|
||||
are accepted, like rules with a quota ([rules.md](rules.md#rules)).
|
||||
|
||||
## Tracing
|
||||
|
||||
Traffic → Trace (`hc trace`) follows packets through the whole ruleset:
|
||||
Housecarl's rules and every other table (Docker's, fail2ban's, an old
|
||||
`inet main`), each rule and policy they meet, and what accepted or
|
||||
dropped them:
|
||||
|
||||
```
|
||||
$ hc trace --address 192.168.1.50 --port 443 --seconds 60
|
||||
14:02:13.120 192.168.1.50:51432 -> 198.51.100.9:443 tcp syn in eth1 out wan
|
||||
inet housecarl forward: built-in: mss-clamp -> continue
|
||||
inet housecarl forward: rule forward #12 (kids at night) -> drop
|
||||
=> DROPPED by rule forward #12 (kids at night)
|
||||
```
|
||||
|
||||
It traces packets from or to the address, on the port (either way), for
|
||||
5 to 600 seconds, at most 200 packets and 20 a second. It works with
|
||||
nftables' own tracing: a table of its own (`inet housecarl-trace`,
|
||||
hooked before everything else) marks the packets, and `nft monitor
|
||||
trace` reports them; the table goes when the trace ends. One trace runs
|
||||
at a time.
|
||||
|
||||
## When something doesn't get through
|
||||
|
||||
1. `hc log -f` with `--log-drops on`: is it dropped by a policy (no rule
|
||||
1. Trace it (above): it names the rule or policy that dropped it, in
|
||||
whichever table.
|
||||
2. `hc log -f` with `--log-drops on`: is it dropped by a policy (no rule
|
||||
accepted it), or by a rule (its number is in the line)?
|
||||
2. `hc rule list forward` (or `input`): hits on the rule you expect?
|
||||
3. `hc ruleset`: the rule as nftables sees it, with the zones' interfaces
|
||||
3. `hc rule list forward` (or `input`): hits on the rule you expect?
|
||||
4. `hc ruleset`: the rule as nftables sees it, with the zones' interfaces
|
||||
and the sets' families.
|
||||
4. `hc status`: other tables. A Docker or Incus table, or an old one,
|
||||
5. `hc status`: other tables. A Docker or Incus table, or an old one,
|
||||
can drop what Housecarl accepts.
|
||||
5. Port forwards: `hc forward list` hits, and the inside host's own
|
||||
6. Port forwards: `hc forward list` hits, and the inside host's own
|
||||
firewall and default route (answers must go back through Housecarl).
|
||||
|
||||
## Backups
|
||||
@@ -132,6 +284,47 @@ database is kept next to it as `housecarl.db.before-restore-<time>`.
|
||||
database when it starts, and an older one refuses a database a newer one
|
||||
has upgraded: restore a backup to go back.
|
||||
|
||||
## Monitoring
|
||||
|
||||
`GET /healthz` needs no login, and answers within a few seconds of any
|
||||
change (it reads the ruleset at most every five seconds, however often
|
||||
it is asked):
|
||||
|
||||
```sh
|
||||
$ curl -s https://firewall.example.net:8070/healthz
|
||||
{"ok":true,"version":"0.1.0","firewall":"applied","mode":"enforcing","probation":0,"warnings":[]}
|
||||
```
|
||||
|
||||
| Field | |
|
||||
|---|---|
|
||||
| `ok` | `true` with HTTP 200 when the firewall is as it should be; `false` with HTTP **503** otherwise |
|
||||
| `firewall` | `applied` (Housecarl's ruleset is in force), `unconfigured` (nothing configured yet, so nothing to apply), `fallback` (the [boot fallback](#if-the-ruleset-fails-to-load-at-boot) is in force), `missing` (no Housecarl table: flushed, or never applied) |
|
||||
| `mode` | `enforcing`, or `shadow`: Housecarl's ruleset is loaded but blocks nothing, and another firewall decides ([shadow.md](shadow.md)). `ok` doesn't depend on it; to be told when a machine isn't enforcing, also expect `"mode":"enforcing"` |
|
||||
| `probation` | changes waiting to be confirmed, rolled back unless they are |
|
||||
| `warnings` | codes of problems with the machine, as `hc status` shows them (`nftables_service`) |
|
||||
|
||||
For Checkmk, an HTTP check on the URL that expects status 200 (and,
|
||||
if you like, the string `"firewall":"applied"`) covers housecarld being
|
||||
down, the fallback and a missing ruleset. Reach it from where the input
|
||||
rules let the monitor in (a rule for the Checkmk server's address and
|
||||
port 8070).
|
||||
|
||||
## Failed logins in the journal
|
||||
|
||||
A failed login to the web UI or the API, and a refused API token, are
|
||||
written to the journal as well as the audit log, with the address first:
|
||||
|
||||
```
|
||||
level=WARN msg="login failed" ip=203.0.113.9 user=admin reason="wrong username or password"
|
||||
level=WARN msg="token refused" ip=203.0.113.9 reason="wrong token secret"
|
||||
```
|
||||
|
||||
Guard's `housecarl` preset, or a fail2ban filter, can ban such addresses
|
||||
([bans.md](bans.md)). The address comes before the user name, which a
|
||||
caller chooses, so a pattern anchored on `msg="login failed" ip=` can't
|
||||
be fooled by a name. (With `log.format: json` they are JSON objects with
|
||||
the same fields, in the same order.)
|
||||
|
||||
## Checks
|
||||
|
||||
`housecarld check` validates the configuration in the database against
|
||||
|
||||
+19
-9
@@ -7,9 +7,10 @@ else gets **grants**, `resource:access`:
|
||||
|
||||
| Resource | read | write |
|
||||
|---|---|---|
|
||||
| `config` | settings, zones, sets, services, rules, NAT, the ruleset, history, interfaces, probation (the web UI shows the forms locked) | change them, undo, import, confirm and roll back |
|
||||
| `config` | settings, zones, sets, services, rules, NAT, the ruleset, history, interfaces, probation, changesets (the web UI shows the forms locked) | change them, stage and apply changesets (your own, or anyone's for administrators), undo, import, confirm and roll back; close connections; start and stop traces |
|
||||
| `sets` | dynamic sets' entries | add and remove entries |
|
||||
| `log` | hit counters and the log | – |
|
||||
| `bans` | ban lists' bans, and the lists the grant names | ban and unban ([bans.md](bans.md)) |
|
||||
| `log` | hit counters, the log, connections, traces | – |
|
||||
| `audit` | the audit log | – |
|
||||
| `*` | all of the above | |
|
||||
|
||||
@@ -20,6 +21,10 @@ program, such as Steward's captive portal, which should be able to let a
|
||||
client in and nothing more. A dynamic set's entries need a `sets` grant
|
||||
even for those with `config:write`.
|
||||
|
||||
A `bans` grant names ban lists the same way: `bans:write:web` may ban
|
||||
into the list `web` (and read it) and nothing else, the grant for Guard
|
||||
or fail2ban on the web server.
|
||||
|
||||
Only administrators may delete other tables (`hc takeover`), and manage
|
||||
users and the certificate (`hc cert`, Server → Certificate).
|
||||
|
||||
@@ -27,8 +32,8 @@ users and the certificate (`hc cert`, Server → Certificate).
|
||||
|
||||
| Role | Grants |
|
||||
|---|---|
|
||||
| `viewer` | `config:read sets:read log:read` |
|
||||
| `operator` | `config:write sets:write log:read audit:read` |
|
||||
| `viewer` | `config:read sets:read bans:read log:read` |
|
||||
| `operator` | `config:write sets:write bans:write log:read audit:read` |
|
||||
| `log` | `log:read`, e.g. for a monitoring script |
|
||||
|
||||
```sh
|
||||
@@ -49,9 +54,13 @@ hc token create portal --user portal --grant sets:write:guests --expires 365d
|
||||
```
|
||||
|
||||
The secret is shown once; only its hash is stored. A restricted token can
|
||||
only create tokens with a subset of its own grants. The web UI only offers
|
||||
roles within what your account may do; a token with more would get no
|
||||
more than its user anyway.
|
||||
only create tokens with a subset of its own grants.
|
||||
|
||||
In the web UI, Account → New token makes one of your own; an
|
||||
administrator makes one for another user with New token on that user's
|
||||
row under Users (`--user` in the CLI, `POST /users/{id}/tokens` in the
|
||||
API). The dialog only offers roles within what the token's user may do; a
|
||||
token with more would get no more than its user anyway.
|
||||
|
||||
**Rotating** gives a token a new secret, keeping its ID, name and
|
||||
permissions, so only the programs that use it need updating:
|
||||
@@ -81,6 +90,7 @@ on probation like any other.
|
||||
## The audit log
|
||||
|
||||
Logins (and failed ones), users, tokens, every configuration change (with
|
||||
its history entry), confirmations and rollbacks, imports, takeovers and
|
||||
dynamic set entries are recorded with who, from where and how: `session`,
|
||||
its history entry), changesets created, applied and discarded,
|
||||
confirmations and rollbacks, imports, takeovers,
|
||||
dynamic set entries and bans are recorded with who, from where and how: `session`,
|
||||
`token` or `socket`.
|
||||
+226
-17
@@ -3,12 +3,12 @@
|
||||
## Zones
|
||||
|
||||
A zone names one or more interfaces, so rules can say "from the LAN"
|
||||
rather than `enp2s0`. An interface belongs to one zone at most. A trailing
|
||||
rather than `eth1`. An interface belongs to one zone at most. A trailing
|
||||
`*` matches many: `incusbr*`.
|
||||
|
||||
```sh
|
||||
hc zone add wan --interface wan --external
|
||||
hc zone add lan --interface enp2s0 --interface 'enp2s0.*'
|
||||
hc zone add lan --interface eth1 --interface 'eth1.*'
|
||||
hc zone add wg --interface wg0
|
||||
```
|
||||
|
||||
@@ -21,29 +21,58 @@ A zone may name an interface that doesn't exist yet (a WireGuard
|
||||
interface that comes up later): its rules match once it appears, and
|
||||
`hc status` warns until then.
|
||||
|
||||
**VLANs.** A VLAN sub-interface is an interface like any other, by its
|
||||
name: `enp2s0.20` (or whatever your network configuration calls it,
|
||||
`vlan20`, `br0.20`). Tagged traffic arrives on the sub-interface, and
|
||||
untagged traffic on the parent, so `enp2s0` alone doesn't cover
|
||||
`enp2s0.20`. Give each VLAN its own zone to keep them apart:
|
||||
|
||||
```sh
|
||||
hc zone add lan --interface enp2s0 # untagged
|
||||
hc zone add iot --interface enp2s0.20
|
||||
hc zone add guest --interface enp2s0.30
|
||||
hc rule add forward --in iot --out wan
|
||||
```
|
||||
|
||||
or one zone for the parent and all of its VLANs, when they should be
|
||||
treated alike: `--interface enp2s0 --interface 'enp2s0.*'`. Interface
|
||||
names are at most 15 characters (the kernel's limit). The flowtable
|
||||
takes VLAN sub-interfaces too, by exact name
|
||||
([flowtable.md](flowtable.md)).
|
||||
|
||||
**Renaming** a zone (`hc zone update dhcp --name lan`, or Edit on the
|
||||
Zones page) changes every rule and NAT rule that names it, in the same
|
||||
change: the history shows them together, and undoing it puts them all
|
||||
back. Sets and services rename the same way, except dynamic sets: their
|
||||
entries, and the API tokens and programs that fill them, go by their
|
||||
name.
|
||||
|
||||
## Address sets
|
||||
|
||||
A set is a named list of IPv4 and IPv6 addresses, prefixes and ranges,
|
||||
mixed:
|
||||
|
||||
```sh
|
||||
hc set add trusted 10.0.10.0/24 10.20.30.0/24 2001:db8:7f3d:10::/64
|
||||
hc set add dns-servers 10.0.10.5 2001:db8:7f3d:10::5 10.0.10.250-10.0.10.254
|
||||
hc set add trusted 192.168.1.0/24 192.168.2.0/24 2001:db8:1::/64
|
||||
hc set add dns-servers 192.168.1.5 2001:db8:1::5 192.168.1.250-192.168.1.254
|
||||
```
|
||||
|
||||
In the ruleset each becomes two nftables sets, `trusted_v4` and
|
||||
`trusted_v6` (one if the set has only one family). A rule that names it
|
||||
becomes one nftables rule per family. Rules can also give addresses
|
||||
directly (`--from 10.0.10.3`); a set is better when several rules share
|
||||
directly (`--from 192.168.1.3`); a set is better when several rules share
|
||||
them.
|
||||
|
||||
A **dynamic** set's entries are added at run time, each with a timeout,
|
||||
and nftables removes them when it runs out, even with housecarld stopped:
|
||||
and nftables removes them when it runs out, even with housecarld stopped.
|
||||
It has no configured entries (validation refuses them); its timeout is
|
||||
the default for entries added without one, and without a timeout they
|
||||
stay until removed:
|
||||
|
||||
```sh
|
||||
hc set add guests --dynamic --timeout 24h
|
||||
hc set entries add guests 10.0.30.17 # for 24 hours
|
||||
hc set entries add guests 2001:db8:7f3d:30::17 --timeout 2h
|
||||
hc set entries add guests 2001:db8:3::17 --timeout 2h
|
||||
hc set entries guests # with the time left
|
||||
hc set entries remove guests 10.0.30.17
|
||||
```
|
||||
@@ -60,7 +89,7 @@ A service names ports of a protocol (`tcp`, `udp`, `sctp`) and ICMP
|
||||
types:
|
||||
|
||||
```sh
|
||||
hc service add jellyfin tcp/8096 tcp/8920
|
||||
hc service add web tcp/8080 tcp/8443
|
||||
hc service add games udp/27015-27030 tcp/27015
|
||||
hc service add pmtu icmp/destination-unreachable icmpv6/packet-too-big
|
||||
```
|
||||
@@ -108,13 +137,18 @@ A rule matches on what it gives, and all of it:
|
||||
| `--state` | connection states: `new`, `established`, `related`, `invalid`, `untracked` |
|
||||
| `--status` | connection status: `dnat` (port forwarded), `snat` |
|
||||
| `--days`, `--hours` | a time window: `--days Monday,Friday --hours 08:00-17:30`; `22:00-06:00` passes midnight. In the firewall's time zone, as the kernel knows it |
|
||||
| `--limit` | a rate: `10/minute`, with `,burst=N`, `,over` (match beyond the rate, to drop a flood) and `,per-source` (a rate for each source address) |
|
||||
| `--limit` | a rate: `10/minute`, or bytes: `1mbytes/second`; with `,burst=N`, `,over` (match beyond the rate) and `,per-source` or `,per-destination` ([Bandwidth and quotas](#bandwidth-and-quotas)) |
|
||||
| `--quota` | an allowance of traffic: `50gbytes,over,per-source,monthly=15` ([Bandwidth and quotas](#bandwidth-and-quotas)) |
|
||||
| `--conn-limit N` | a source with more than N connections |
|
||||
|
||||
The action is `accept`, `drop`, `reject` (an ICMP error, or with
|
||||
`--reject-with tcp-reset` a TCP reset) or `log` (log and go on).
|
||||
`--log` logs the packets of any rule ([operations.md](operations.md#the-log)).
|
||||
`--disabled` (or `hc rule disable ID`) keeps a rule without using it.
|
||||
`--every-packet` ("every packet" in the web UI) checks a drop or reject
|
||||
against every packet, before established connections are accepted, so it
|
||||
also stops connections already open, not just new ones; it can't accept
|
||||
([operations.md](operations.md#connections)).
|
||||
|
||||
Examples:
|
||||
|
||||
@@ -131,6 +165,87 @@ hc rule add forward --in wan --status dnat
|
||||
hc rule add forward --in wan -a drop
|
||||
```
|
||||
|
||||
### Bandwidth and quotas
|
||||
|
||||
A **byte rate** limits bandwidth: `--limit 20mbytes/second,over,per-source
|
||||
-a drop` drops what a device sends beyond 20 MB/s. Units are `bytes`,
|
||||
`kbytes` and `mbytes` (`kb`, `mb` for short); the burst is in the same
|
||||
unit. nftables can only hold byte rates per second (up to 4 GB/s) or per
|
||||
minute (up to 293 MB/minute); validation says so for larger ones. This is
|
||||
policing: traffic beyond the rate is dropped, not queued, so TCP slows to
|
||||
fit, and UDP loses packets. Shaping (queues, fair shares) is `tc`'s work,
|
||||
not a firewall's.
|
||||
|
||||
A **quota** is an allowance of traffic. `over` matches once it is used up,
|
||||
to drop what comes after:
|
||||
|
||||
```sh
|
||||
# Each device on the LAN: 50 GB a month, from the 15th (a billing day)
|
||||
hc rule add forward --in lan --out wan --quota 50gbytes,over,per-source,monthly=15 -a drop
|
||||
# The whole guest network: 10 GB a day together
|
||||
hc rule add forward --in guest --out wan --quota 10gbytes,over,daily -a drop
|
||||
hc quotas # what each used, and the next reset
|
||||
hc rule reset-quota 31 # start again now
|
||||
hc rule reset-quota 31 --address 192.168.1.23 # one device only
|
||||
hc rule reset-quota 31 --give 5gbytes --address 192.168.1.23
|
||||
```
|
||||
|
||||
`--give` is a top-up: it takes that much off what was used (never below
|
||||
zero), so the device gets 5 GB more until the quota's next reset, and the
|
||||
rule stays as it is. In the web UI, the quota's line under the rule
|
||||
opens its dialog: usage (each address's), resets, and **Reset** and
|
||||
**Give back…** for everyone or one address. Like a reset, these are
|
||||
done at once (audited, not staged or on probation); Housecarl puts the
|
||||
ruleset in force again with the new figure.
|
||||
|
||||
In the web UI each quota's use is on its rule (with **Reset**), all of
|
||||
them together on the dashboard, and their history under **Traffic →
|
||||
Statistics**.
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| `per-source`, `per-destination` | each address its own allowance; without, one allowance for everything the rule matches |
|
||||
| `daily`, `weekly`, `monthly=DAY` | when it starts again: at midnight; weekly on Mondays; monthly on that day (the last day of a shorter month). Without one, only by hand |
|
||||
|
||||
nftables counts a quota but never resets it, and forgets it when its
|
||||
table is replaced: Housecarl carries what each quota used across every
|
||||
change and restart, saves it every five minutes (and after each change,
|
||||
and when housecarld stops) for after a reboot, and resets quotas when their period ends.
|
||||
A per-address quota counts an address: a device that gets a new one
|
||||
starts again. With Steward's reservations IPv4 addresses stay; IPv6 is
|
||||
grouped by /64 (below).
|
||||
|
||||
Changing the amount, `over` or the resets keeps what was used. Changing
|
||||
how the quota counts carries what can be carried:
|
||||
|
||||
| Change | What was used |
|
||||
|---|---|
|
||||
| per address to one total | the addresses' sum |
|
||||
| to wider groups (IPv6 /64 to /56) | added up per new group |
|
||||
| a total to per address | starts again: a total can't be split |
|
||||
| per source to per destination (or back) | starts again |
|
||||
| to finer groups (/64 to /128) | starts again: a group can't be split |
|
||||
|
||||
A change that starts a quota again says so (warning `quota_restarts`),
|
||||
when it is staged for review and in `hc` (also with `-n`).
|
||||
|
||||
**Rules with a quota or a byte rate count every packet.** The built-in
|
||||
that accepts established connections comes before your rules, so a rule
|
||||
there only sees each connection's first packet. Rules that count traffic
|
||||
are checked before it instead, in their order (the web UI lists them
|
||||
first, marked "every packet"). So they can drop, reject or log, but not
|
||||
accept: an accept there would let packets past every later rule. Let the
|
||||
traffic in with a rule of its own. The flowtable's fast path skips the
|
||||
forward chain for established connections, so with it on, forward
|
||||
quotas and byte rates only count first packets (a warning says so;
|
||||
[flowtable.md](flowtable.md)).
|
||||
|
||||
**Per-address limits group addresses**: each IPv4 address on its own, and
|
||||
IPv6 addresses by /64, since one device uses several (privacy addresses
|
||||
change). `--group-v4 24` counts each /24 together; `--group-v6 128` each
|
||||
IPv6 address alone. This applies to per-address rates, connection limits
|
||||
and quotas.
|
||||
|
||||
### Raw rules
|
||||
|
||||
For what Housecarl can't say, a rule can be one nftables rule written by
|
||||
@@ -140,11 +255,26 @@ hand, placed like any other:
|
||||
hc rule add forward --raw 'iifname != "wan" ct state invalid ip daddr @trusted_v4 accept'
|
||||
```
|
||||
|
||||
It is checked by nftables with the rest of the ruleset, and can use the
|
||||
sets' nftables names (`@trusted_v4`). The importer makes raw rules from
|
||||
It is used exactly as written: its text replaces the rule's zones,
|
||||
addresses, services, action, limits and log, so a raw rule with any of
|
||||
those set is refused (`raw_conflict`); write them in the text. It is
|
||||
checked by nftables with the rest of the ruleset, and can use the sets'
|
||||
nftables names (`@trusted_v4`). The importer makes raw rules from
|
||||
what it can't translate. It goes into the ruleset as written: add
|
||||
`counter` to it to count its packets.
|
||||
|
||||
In shadow mode ([shadow.md](shadow.md)) a raw rule's final `drop` or
|
||||
`reject` becomes `accept`; a raw rule that does anything else to packets
|
||||
(NAT, `meta mark set`, `jump`, `notrack`, a verdict map…) is left out,
|
||||
with a warning (`shadow_raw_left_out`).
|
||||
|
||||
## Ban lists
|
||||
|
||||
Programs such as Guard and fail2ban ban addresses into ban lists, and
|
||||
those bans come first of all in the input and forward chains (after
|
||||
loopback in input), before the built-ins and every rule. See
|
||||
[bans.md](bans.md).
|
||||
|
||||
## Built-ins
|
||||
|
||||
Housecarl adds the rules every firewall needs before yours. Each can be
|
||||
@@ -162,15 +292,94 @@ UI), and `hc ruleset` shows them:
|
||||
| `antispoof` | on | early | drop sources that can't come from the internet, on external zones: the antispoof lists (private, loopback, documentation and multicast ranges) and your own prefixes |
|
||||
| `mss_clamp` | off | forward | clamp TCP's MSS to the route's MTU on external zones: needed with PPPoE |
|
||||
|
||||
**Your prefixes** (`--own-prefix 2001:db8:7f3d::/48`) are your delegated
|
||||
**Your prefixes** (`--own-prefix 2001:db8::/48`) are your delegated
|
||||
IPv6 prefix and public addresses: nothing from outside should come from
|
||||
them. The antispoof lists can be changed with `--antispoof-v4` and
|
||||
`--antispoof-v6`.
|
||||
|
||||
The **flowtable** (`--flowtable wan --flowtable enp2s0`) lets established
|
||||
The **flowtable** (`--flowtable wan --flowtable eth1`) lets established
|
||||
TCP and UDP connections forwarded between those devices skip the rules,
|
||||
which helps on a slow CPU. It needs exact device names. Counters of
|
||||
forward rules then only count each connection's first packets.
|
||||
which helps when the CPU can't keep up with the line. It needs exact
|
||||
device names, and costs quotas, byte rates and traffic per device in the
|
||||
forward chain their accuracy: see [flowtable.md](flowtable.md) for when
|
||||
to use it, and when not.
|
||||
|
||||
## The zone matrix
|
||||
|
||||
The Matrix page (`hc matrix`) sums up what the rules do to **new
|
||||
connections** between zones: a row per zone the traffic comes from, a
|
||||
column per zone it goes to, and the firewall itself (to it: the input
|
||||
chain; from it: output; zone to zone: forward).
|
||||
|
||||
```
|
||||
$ hc matrix
|
||||
FROM \ TO guest lan wan firewall
|
||||
guest block * block partly +1 partly +4
|
||||
lan block block * allow partly +4
|
||||
wan partly +1 -1 partly +1 -1 block -1 * partly +3 -1
|
||||
firewall allow allow allow -
|
||||
|
||||
Let through for part of the traffic:
|
||||
guest → wan http, https
|
||||
guest → firewall ICMPv6; ping; ssh from admins; dns
|
||||
lan → firewall ICMPv6; ping; ssh from admins; dns
|
||||
wan → guest 1 port forward
|
||||
wan → lan 1 port forward
|
||||
wan → firewall ICMPv6; DHCPv6; ssh from admins
|
||||
|
||||
allow, block: every new connection; partly: some are let through, some not.
|
||||
+N: rules that let part of the traffic through; -N: rules that block part of it. 'hc matrix FROM TO' lists them.
|
||||
*: a zone to itself: only traffic routed between its interfaces passes the firewall.
|
||||
```
|
||||
|
||||
Each cell walks that pair's rules in order, as nftables would, with the
|
||||
built-ins that act on new connections (ping from internal zones,
|
||||
ICMPv6, the DHCPv6 client, antispoofing, port forwards):
|
||||
|
||||
* a rule that applies to the whole pair decides it;
|
||||
* a rule that applies to part of its traffic (some addresses, services,
|
||||
times, rates, a quota) is an **exception**, and the walk goes on;
|
||||
* what's left falls to the deciding rule or the chain's policy.
|
||||
|
||||
A cell reads **allow** or **block** when every new connection gets the
|
||||
same, and **partly** when exceptions let some through while the rest is
|
||||
blocked (or block some while the rest is allowed). **+N** counts the
|
||||
exceptions that let traffic through, **-N** those that block it. Under
|
||||
the grid (in the web UI, in each cell) is what the exceptions let
|
||||
through: their services, with the addresses they are limited to.
|
||||
|
||||
A zone to itself (`*`, faded in the web UI) is mostly traffic that never
|
||||
reaches the firewall: within one interface's network it goes straight
|
||||
from host to host. Only what is routed between the zone's interfaces (or
|
||||
back out of the one it came in on) passes.
|
||||
|
||||
`hc matrix guest wan` (or clicking the cell) lists the exceptions in
|
||||
order, with each rule's hits:
|
||||
|
||||
```
|
||||
guest → wan (the forward chain), new connections: partly allowed
|
||||
VERDICT WHAT BY HITS
|
||||
allowed http, https rule forward #14 1520
|
||||
blocked the rest forward policy 37
|
||||
```
|
||||
|
||||
The hits are the rule's packets since housecarld started, in all of its
|
||||
traffic: a rule without zones counts for every pair it covers. They are
|
||||
how you tell an exception that is used from one that isn't.
|
||||
|
||||
A raw rule is read when it is a match followed by `accept`, `drop` or
|
||||
`reject` (with `counter` and `log` if you like): it counts as an exception
|
||||
for every pair of its chain, shown as `raw: …`. One that matches
|
||||
interfaces (the matrix can't tell which zones), jumps, translates, sets
|
||||
something or uses a verdict map shows as **unknown**: check it yourself.
|
||||
Raw rules have hits only if they say `counter`.
|
||||
|
||||
A rule whose addresses all lie in networks directly connected to other
|
||||
zones is left out of a pair: traffic between those zones can't have them
|
||||
(`from wg_net` doesn't concern traffic coming in on the LAN). Routed
|
||||
networks, mixed lists and dynamic sets might, so they stay. Replies and
|
||||
established connections aren't shown: the built-in lets them through. In
|
||||
review mode, the matrix shows your pending changes.
|
||||
|
||||
## The ruleset
|
||||
|
||||
@@ -179,8 +388,8 @@ show each rule's hits. The table looks like this (shortened):
|
||||
|
||||
```
|
||||
table inet housecarl {
|
||||
set trusted_v4 { type ipv4_addr; flags interval; elements = { 10.0.10.0/24, 10.20.30.0/24 } }
|
||||
set trusted_v6 { type ipv6_addr; flags interval; elements = { 2001:db8:7f3d:10::/64 } }
|
||||
set trusted_v4 { type ipv4_addr; flags interval; elements = { 192.168.1.0/24, 192.168.2.0/24 } }
|
||||
set trusted_v6 { type ipv6_addr; flags interval; elements = { 2001:db8:1::/64 } }
|
||||
chain early { type filter hook prerouting priority raw; policy accept; ... }
|
||||
chain input { type filter hook input priority filter; policy drop;
|
||||
iifname "lo" accept comment "hc:b-loopback"
|
||||
|
||||
+174
@@ -0,0 +1,174 @@
|
||||
# Shadow mode
|
||||
|
||||
In shadow mode Housecarl runs beside the firewall you have now and
|
||||
changes nothing. Its ruleset is loaded, so its counters, log, tracer and
|
||||
statistics show what it *would* do with your real traffic, but it blocks,
|
||||
rejects and translates nothing. Your current firewall keeps deciding.
|
||||
|
||||
Use it to:
|
||||
|
||||
* **try Housecarl** on the real machine before you trust it with the
|
||||
firewall: the web UI, the API, `hc`, changesets, history;
|
||||
* **check an import**: import your ruleset ([import.md](import.md)) and
|
||||
watch whether Housecarl would drop anything your firewall lets through;
|
||||
* **develop against the API**: Steward, scripts or fail2ban can fill
|
||||
dynamic sets and see them used, with nothing at stake.
|
||||
|
||||
## Turning it on
|
||||
|
||||
Shadow mode is set in `/etc/housecarl/housecarl.yaml`
|
||||
([configuration.md](configuration.md)), not through the API, so no user
|
||||
or token can switch enforcement off:
|
||||
|
||||
```yaml
|
||||
shadow:
|
||||
enabled: true
|
||||
position: after # or before; see below
|
||||
```
|
||||
|
||||
housecarld reads the file when it starts:
|
||||
|
||||
```sh
|
||||
sudo systemctl restart housecarl
|
||||
sudo hc status # Mode: SHADOW …
|
||||
```
|
||||
|
||||
The web UI shows a banner on every page, `hc status` a `Mode:` line, and
|
||||
`/healthz` `"mode": "shadow"`.
|
||||
|
||||
## Why it can't let anything through
|
||||
|
||||
In nftables, `accept` only ends the chain it is in. The packet then
|
||||
goes on to the next base chain on the same hook, and other tables' chains
|
||||
are among those. Only a drop is final. Housecarl's table in shadow mode
|
||||
never drops, so it can't let through anything your firewall drops, and it
|
||||
can't block anything your firewall accepts.
|
||||
|
||||
## What changes in the ruleset
|
||||
|
||||
The same configuration is rendered, with everything that acts on packets
|
||||
turned into counting:
|
||||
|
||||
| | Enforcing | Shadow mode |
|
||||
|---|---|---|
|
||||
| Rules that drop or reject | `drop`, `reject` | counted, then `accept` (Housecarl's chain ends as it would have). Their comment says `[would drop]` or `[would reject]` |
|
||||
| The built-ins that drop (invalid packets, antispoofing) | `drop` | counted, `[would drop]` |
|
||||
| Policies | as configured | `accept`. What a drop policy would drop is counted and **always logged**, rate-limited, in all three chains |
|
||||
| Port forwards, source NAT, redirects | translate | counted only (`[would dnat]`, `[would masquerade]`, `[would snat]`, `[would redirect]`) |
|
||||
| MSS clamping | changes SYN packets | left out |
|
||||
| The flowtable | offloads connections | left out: offloaded connections skip *every* table's forward chain, your firewall's too |
|
||||
| Raw rules | as written | as written if they only match, count, log or accept. A final `drop` or `reject` becomes `accept`. Anything else that acts on packets (NAT, `set`, `jump`, `notrack`, a verdict map…) is left out, with a warning (`shadow_raw_left_out`) |
|
||||
| Everything else | | the same: accepts, rate limits, connection limits, quotas, dynamic sets, per-device counters, logging |
|
||||
|
||||
`hc ruleset` prints the shadow ruleset, the one loaded now.
|
||||
`hc ruleset --enforcing` prints the ruleset that will be loaded once
|
||||
shadow mode is off.
|
||||
|
||||
**Every change is checked by nftables twice**: the shadow ruleset, and the
|
||||
enforcing one. A change that nftables would refuse once shadow mode is off
|
||||
is refused now, so leaving shadow mode can't fail on that.
|
||||
|
||||
## Where Housecarl's chains run
|
||||
|
||||
Your firewall's chains are usually at priority `filter` (0), its raw
|
||||
chains at `raw`. `shadow.position` puts Housecarl's filter chains just
|
||||
next to them:
|
||||
|
||||
* **`after`** (the default) runs them at `filter + 10` (and `raw + 10`).
|
||||
Housecarl only sees what your firewall let through. So **every would-drop
|
||||
is traffic that would break if you switched over**: the list to work
|
||||
through before you do. What your firewall drops and Housecarl would
|
||||
accept (a hole) doesn't show; compare the rules for that.
|
||||
* **`before`** runs them at `filter - 10` (and `raw - 10`). Housecarl sees
|
||||
every packet, as it would when enforcing, and its hits and drops are the
|
||||
numbers you would get. Its would-drops include everything your firewall
|
||||
drops too (the internet's background noise).
|
||||
|
||||
If your firewall uses other priorities, say `filter + 20`, "after" is
|
||||
relative to `filter`, not to it: check with `nft list ruleset`.
|
||||
|
||||
NAT chains are different: the kernel asks NAT chains in turn only until
|
||||
one translates. Housecarl's NAT chains always run first (`dstnat - 10`,
|
||||
`srcnat - 10`), whatever the position, or they would never see a
|
||||
connection your firewall translates. They translate nothing, so your
|
||||
firewall's NAT still does the work.
|
||||
|
||||
## What you see
|
||||
|
||||
* **Hits** (`hc counters`, the Rules page): a drop rule's hits are what it
|
||||
would have dropped. NAT counters count connections (only a connection's
|
||||
first packet reaches the NAT chains).
|
||||
* **The log** (`hc log`, the Log page): every packet a drop policy would
|
||||
have dropped, and what rules with logging would have dropped, marked
|
||||
*would drop*. `hc log` adds "(shadow mode: would drop, let through)".
|
||||
* **The tracer** (`hc trace`, the Trace page): the verdict is your
|
||||
firewall's, the real one. When Housecarl would have dropped the packet,
|
||||
the trace says by which rule ("shadow mode: Housecarl would have
|
||||
dropped it, by …").
|
||||
* **Statistics** ([stats.md](stats.md)): drops, drops per chain and the
|
||||
blocked sources are what Housecarl would have dropped.
|
||||
|
||||
## What works differently
|
||||
|
||||
* **No probation.** A change can't cut anyone off, so changes from the
|
||||
network are applied and saved at once, with no confirmation. Changesets
|
||||
work as usual; their ruleset diff is of the shadow ruleset.
|
||||
* **No takeover, no closing connections.** `hc takeover` (deleting your
|
||||
firewall's tables) and `hc connections close` are refused (409,
|
||||
`hc` exits with 5): both would change your traffic.
|
||||
* **Bans** ([bans.md](bans.md)) are kept and their rules count what they
|
||||
would drop (`[would drop]`), but nothing is dropped, and banning closes
|
||||
no connections.
|
||||
* **The boot.** `housecarl-ruleset.service` loads the shadow ruleset at
|
||||
boot. Its fallback ([operations.md](operations.md#if-the-ruleset-fails-to-load-at-boot))
|
||||
is an empty table that filters nothing, since your firewall protects the
|
||||
machine.
|
||||
* **Your firewall reloading.** A ruleset file such as `/etc/nftables.conf`
|
||||
usually starts with `flush ruleset`, and Debian's `nftables.service`
|
||||
runs `nft flush ruleset` when it stops or restarts. Both delete
|
||||
Housecarl's table too. housecarld watches nftables' notifications and
|
||||
puts its table back within a few seconds (the journal says "Housecarl's
|
||||
table was deleted … put it back"). Dynamic sets lose their entries, and
|
||||
quotas go back to what was last saved (every five minutes).
|
||||
* **Warnings.** `nftables_service` and `takeover_pending` aren't raised:
|
||||
another firewall beside Housecarl is the point. Instead:
|
||||
`shadow_unprotected` when no other table has a filter chain on the input
|
||||
hook (nothing protects the machine), and `shadow_raw_left_out` for each
|
||||
raw rule left out ([validation.md](validation.md#the-machine)).
|
||||
* **Quotas count** what passes in shadow mode, and keep counting when
|
||||
you switch over.
|
||||
|
||||
## A trial, step by step
|
||||
|
||||
From a machine whose firewall is `/etc/nftables.conf` with `table inet main`:
|
||||
|
||||
1. Install Housecarl and create the administrator
|
||||
([install.md](install.md)).
|
||||
2. Turn shadow mode on in `/etc/housecarl/housecarl.yaml`, and start
|
||||
housecarld: `sudo systemctl enable --now housecarl`.
|
||||
3. Import your ruleset: `sudo hc import --defines /etc/nftables.conf`
|
||||
([import.md](import.md)). Leave `nftables.service` as it is.
|
||||
4. Use your network for a while, and watch `hc log -f`, `hc counters` and
|
||||
the dashboard. With `position: after`, each would-drop is something
|
||||
your firewall allows and Housecarl wouldn't: fix the rule in Housecarl,
|
||||
or decide it doesn't matter. `hc status` lists the raw rules left out.
|
||||
5. When nothing you need would be dropped, leave shadow mode.
|
||||
|
||||
## Leaving shadow mode
|
||||
|
||||
1. Look at what will be loaded: `sudo hc ruleset --enforcing`.
|
||||
2. Set `shadow.enabled: false` (or remove the section), and
|
||||
`sudo systemctl restart housecarl`. Housecarl's ruleset is now in
|
||||
force **alongside** your firewall: a packet must be accepted by both.
|
||||
The restart applies it at once, not on probation, but your old firewall
|
||||
is still there and still decides too.
|
||||
3. Take over: `sudo hc takeover "inet main"` deletes the old table (on
|
||||
probation over the network), and `sudo systemctl disable nftables`
|
||||
(without `--now`) retires it ([import.md](import.md#3-take-over)).
|
||||
|
||||
## Going back
|
||||
|
||||
Set `shadow.enabled: true` and restart housecarld. If Housecarl was your
|
||||
only firewall, **the machine is now unprotected**: load your other
|
||||
firewall first. Until one filters the input hook, `hc status` and
|
||||
`/healthz` warn (`shadow_unprotected`).
|
||||
+144
@@ -0,0 +1,144 @@
|
||||
# Statistics
|
||||
|
||||
Housecarl keeps what the firewall did over time:
|
||||
|
||||
- traffic per interface and zone, received and sent;
|
||||
- packets dropped, by chain, and when drops happen (a weekday × hour
|
||||
heatmap);
|
||||
- hits per rule, built-in and policy, and the rules that matched nothing;
|
||||
- the sources blocked most and what they tried (from the log);
|
||||
- traffic per inside device (a setting, below);
|
||||
- connections tracked, against the kernel's limit;
|
||||
- what each quota counted;
|
||||
- housecarld's and the machine's memory, CPU and load.
|
||||
|
||||
All of it is in the web UI (**Traffic → Statistics**, and two charts on
|
||||
the dashboard), the CLI (`hc stats`) and the API (`/api/v1/stats/...`),
|
||||
for anyone with `log:read`.
|
||||
|
||||
Statistics are **on by default**. They cost nothing on the packets'
|
||||
path: housecarld reads counters the kernel keeps anyway, once a minute.
|
||||
Only traffic per device adds to the ruleset, so it is a setting you turn
|
||||
on. The figures are kept as aggregates in a database of their own,
|
||||
`/var/lib/housecarl/stats.db`, separate from the configuration. Deleting
|
||||
it is always safe (stop housecarl first), and backups leave it out.
|
||||
|
||||
## Settings
|
||||
|
||||
In `/etc/housecarl/housecarl.yaml` (restart housecarl after a change):
|
||||
|
||||
```yaml
|
||||
stats:
|
||||
enabled: true
|
||||
database: /var/lib/housecarl/stats.db
|
||||
retention:
|
||||
minute: 7d # per-minute figures
|
||||
hour: 90d # hourly figures and top lists
|
||||
day: 730d # daily figures and top lists
|
||||
top_n: 100 # entries kept per top list, per hour and day
|
||||
```
|
||||
|
||||
Old data is removed once an hour. A range picks the finest data that
|
||||
covers it: per minute up to 2 days, hourly up to 90 days, then daily.
|
||||
|
||||
## Traffic per device
|
||||
|
||||
```sh
|
||||
hc settings set --stats-per-device on
|
||||
```
|
||||
|
||||
(or **Settings → Statistics** in the web UI). This is a firewall setting,
|
||||
so it goes through the usual review and probation. Housecarl then adds
|
||||
four sets to its table, and four rules at the top of the forward chain
|
||||
(shown as the built-in `stats`):
|
||||
|
||||
```
|
||||
iifname != "wan" oifname "wan" update @stats_up_v4 { ip saddr }
|
||||
iifname != "wan" oifname "wan" update @stats_up_v6 { ip6 saddr and ffff:ffff:ffff:ffff:: }
|
||||
iifname "wan" oifname != "wan" update @stats_down_v4 { ip daddr }
|
||||
iifname "wan" oifname != "wan" update @stats_down_v6 { ip6 daddr and ffff:ffff:ffff:ffff:: }
|
||||
```
|
||||
|
||||
Each inside address talking to an external zone gets a counter: what it
|
||||
sent out, and what came back to it. IPv6 addresses are counted per /64,
|
||||
since one device uses many addresses. Counting is done in the kernel, a
|
||||
set lookup per forwarded packet. An address quiet for an hour drops out
|
||||
of the sets, and a reload keeps the counts.
|
||||
|
||||
With the flowtable on, established connections skip the forward chain,
|
||||
so only their first packets are counted (warning `stats_flowtable`). Set
|
||||
names starting with `stats_` (and `meter_`, `conn_`, `quota_`) are
|
||||
Housecarl's own and can't be used for your sets.
|
||||
|
||||
## What is collected
|
||||
|
||||
Every minute housecarld samples its sources and stores what changed:
|
||||
|
||||
| Figure | Source |
|
||||
|---|---|
|
||||
| `rx:IF`, `tx:IF` (bytes), `rxp:IF`, `txp:IF` (packets) | the interfaces' counters (`/sys/class/net`); the loopback is left out |
|
||||
| `zrx:ZONE`, `ztx:ZONE` | the same, summed per zone: received from and sent to the zone's interfaces |
|
||||
| `hits:MARK`, `hitb:MARK` | each rule's (`r12`), built-in's (`b-established`), port forward's (`f3`) and policy's (`p-input`) counter, in packets and bytes |
|
||||
| `drop:CHAIN`, `drops` | the packets counted by what drops them: rules that drop or reject, the drop policies, antispoofing (`early`) and invalid packets (`invalid`) |
|
||||
| `quota:r12` | bytes counted against a rule's quota |
|
||||
| `g:conntrack`, `g:conntrack_max` | connections tracked, and the limit (`nf_conntrack_max`) |
|
||||
| `g:rss`, `g:cpu` | housecarld's memory (bytes) and CPU (tenths of a percent of one CPU) |
|
||||
| `g:mem`, `g:load` | the host's memory in use, and its 1-minute load average × 100 |
|
||||
|
||||
`g:` figures are gauges: the API averages them over a bucket.
|
||||
|
||||
Top lists, per hour and per day, the `top_n` largest:
|
||||
|
||||
| List | |
|
||||
|---|---|
|
||||
| `blocked_src` | sources of logged drops |
|
||||
| `blocked_port` | what they tried: `tcp/22`, `udp/53`, `icmp/echo-request` |
|
||||
| `device_up`, `device_down` | bytes per inside address, sent and received (traffic per device) |
|
||||
| `quota_r12` | a per-address quota's use, per address |
|
||||
|
||||
The blocked lists only see what is **logged**: the input and forward
|
||||
policies' drops with `hc settings set --log-drops on`, and rules with
|
||||
logging. The drop counts above see everything.
|
||||
|
||||
In shadow mode ([shadow.md](shadow.md)) the drops, and the blocked
|
||||
lists, are what Housecarl would have dropped: the packets went on. What
|
||||
the policies would drop is always logged there.
|
||||
|
||||
A counter that goes down (an interface that came back, a quota reset)
|
||||
counts again from zero. What happens while housecarld is stopped isn't
|
||||
counted.
|
||||
|
||||
## The CLI
|
||||
|
||||
```sh
|
||||
hc stats --since 7d # summary: traffic per zone, drops, busiest rules, top lists
|
||||
hc stats status # collecting? since when, the database, figures now
|
||||
hc stats traffic --since 24h # per zone over time (--interfaces: per interface)
|
||||
hc stats top devices --since 30d # devices by traffic (blocked, tried, quota RULE-ID)
|
||||
hc stats rules --unused # rules that matched nothing in 30 days
|
||||
hc stats connections --since 24h
|
||||
```
|
||||
|
||||
Ranges are written `90m`, `24h`, `7d` or `4w`. `-o json` gives the API's
|
||||
answer.
|
||||
|
||||
## The API
|
||||
|
||||
| Endpoint | |
|
||||
|---|---|
|
||||
| `GET /stats/status` | `enabled`, `database`, `size`, `first_data`, `last_sample`, `sample_error`, `per_device`, `retention`, `latest` (the newest gauges) |
|
||||
| `GET /stats/summary?since=24h&limit=10` | `zones` and `interfaces` (`name`, `received`, `sent`), `drops`, `drops_by_chain`, `rules` (busiest, named), `peak_connections`, `conntrack_max`, `blocked_sources`, `blocked_ports`, `devices_up`, `devices_down` |
|
||||
| `GET /stats/timeline?keys=zrx:*,drops&since=24h&step=auto` | `step`, `times` (bucket starts), `series` (`key`, `values`); `GROUP:*` means every key of a group; `step` is `auto`, `1m`, `5m`, `15m`, `1h`, `6h` or `1d`; a gauge without a sample is -1 |
|
||||
| `GET /stats/top?kind=device_down&since=7d&limit=20` | a top list (`key`, `count`) |
|
||||
| `GET /stats/hits?since=30d` | packets per rule, built-in and policy (`key`, `count`, `name`), with the enabled rules that counted none |
|
||||
| `GET /stats/heatmap?key=drops&since=28d` | `avg[weekday][hour]` (Monday first, local time; -1 where there is no data) |
|
||||
|
||||
With statistics off, `/stats/status` says `enabled: false` and the others
|
||||
answer 503.
|
||||
|
||||
## Warnings
|
||||
|
||||
When the connection tracking table is over 80% full, the status shows
|
||||
the warning `conntrack_full`: once it is full, the kernel drops new
|
||||
connections. Raise `net.netfilter.nf_conntrack_max`, or find what opens
|
||||
so many on the Connections page.
|
||||
+47
-1
@@ -34,6 +34,8 @@ are unique per kind, ignoring case.
|
||||
| `flowtable_device` | error | a flowtable device with a wildcard: name each device |
|
||||
| `flowtable_device_missing` | warning, *host* | a flowtable device doesn't exist; the flowtable is left out until it does |
|
||||
| `probation_range` | error | the probation isn't 10 to 3600 seconds |
|
||||
| `stats_no_external` | warning | traffic per device is on, but no zone is external: it counts what goes to and comes from external zones |
|
||||
| `stats_flowtable` | warning | traffic per device with the flowtable on: its fast path skips the forward chain, so only each connection's first packets are counted |
|
||||
|
||||
## Zones
|
||||
|
||||
@@ -43,14 +45,16 @@ are unique per kind, ignoring case.
|
||||
| `zone_interface` | error | not an interface name (up to 15 characters; a trailing `*` matches many) |
|
||||
| `zone_interface_taken` | error | the interface is in another zone already |
|
||||
| `zone_interface_missing` | warning, *host* | the interface doesn't exist here (yet); its rules match once it does |
|
||||
| `no_external_zone` | warning | no zone is external: antispoofing, MSS clamping, the DHCPv6 client rule and port forwards without public addresses need one |
|
||||
| `no_external_zone` | warning | no zone is external: antispoofing, MSS clamping, the DHCPv6 client rule and port forwards without public addresses need one, and the boot fallback lets nothing but loopback through without one ([operations.md](operations.md#if-the-ruleset-fails-to-load-at-boot)) |
|
||||
|
||||
## Sets and services
|
||||
|
||||
| Code | | When |
|
||||
|---|---|---|
|
||||
| `set_name_reserved` | error | a set name starting with `meter_`, `conn_`, `quota_`, `stats_` or `ban_`: Housecarl's own sets (rate limits, quotas, statistics, ban lists) use those |
|
||||
| `set_entry` | error | an entry that isn't an address, prefix (without host bits) or range |
|
||||
| `set_empty` | warning | a set (not dynamic) with no entries: rules using it match nothing |
|
||||
| `set_dynamic_entries` | error | configured entries on a dynamic set: its entries come at run time |
|
||||
| `set_timeout` | error | a timeout on a set that isn't dynamic, or under a second |
|
||||
| `service_empty` | error | a service with no entries |
|
||||
| `service_entry` | error | an entry that isn't `PROTO/PORTS` (`tcp`, `udp`, `sctp`; a port or range) or `icmp/TYPE`, `icmpv6/TYPE` with a type nftables knows |
|
||||
@@ -77,12 +81,24 @@ These apply to rules, source NAT and redirects.
|
||||
| `rule_chain` | error | a chain that isn't `input`, `forward`, `output` or `early` |
|
||||
| `rule_action` | error | an action that isn't `accept`, `drop`, `reject` or `log` |
|
||||
| `raw_rule` | error | a raw rule over several lines, or with `;` |
|
||||
| `raw_conflict` | error | a raw rule with other fields set (zones, addresses, services, limits, log…): it is used as written, so they would be ignored |
|
||||
| `early_conntrack` | error | connection states, limits or a connection limit in the early chain, which runs before connection tracking |
|
||||
| `early_reject` | error | reject in the early chain: drop instead |
|
||||
| `ct_state`, `ct_status` | error | an unknown connection state or status |
|
||||
| `rule_day`, `rule_hours` | error | an unknown day, or hours that aren't `HH:MM`, or only one of them |
|
||||
| `rule_limit` | error | a rate that isn't a number per second, minute, hour or day |
|
||||
| `rule_conn_limit` | error | a connection limit under 1 |
|
||||
| `rule_limit_unit` | error | a rate's unit that isn't packets, bytes, kbytes or mbytes |
|
||||
| `rule_limit_bytes` | error | a byte rate nftables can't hold: over 4 GB a second, 293 MB a minute, 4.9 MB an hour or 208 KB a day (the message gives it per second) |
|
||||
| `rule_limit_key` | error | a limit both per source and per destination |
|
||||
| `rule_quota` | error | a quota under 1 byte |
|
||||
| `rule_quota_key` | error | a quota both per source and per destination |
|
||||
| `rule_quota_reset` | error | a reset that isn't daily, weekly or monthly; a day with another reset than monthly; a day outside 1-31 |
|
||||
| `rule_group` | error | a grouping prefix outside 1-32 (IPv4) or 1-128 (IPv6) |
|
||||
| `rule_traffic_accept` | error | a rule with a quota or a byte rate that accepts: it is checked before established connections are accepted, so it can only drop, reject or log |
|
||||
| `rule_every_packet_accept` | error | a rule checked against every packet (`every_packet`) that accepts: it comes before established connections are accepted, so it can only drop, reject or log |
|
||||
| `quota_restarts` | warning | a change makes a rule count its quota so that what it used can't carry over (a total to per address, per source to per destination, finer groups): it starts again. Only on the change that does it |
|
||||
| `flowtable_traffic` | warning | a forward rule checked against every packet (a quota, a byte rate, `every_packet`) while the flowtable is on: it only sees each connection's first packets |
|
||||
| `reject_with` | error | `tcp-reset` on a rule that matches more than TCP, or with an action other than reject |
|
||||
| `log_prefix` | error | a log prefix over 100 characters |
|
||||
|
||||
@@ -106,3 +122,33 @@ These apply to rules, source NAT and redirects.
|
||||
| `snat_everything` | warning | no out zones and no sources: everything leaving the firewall is translated |
|
||||
| `redirect_service` | error | no service, or one without TCP, UDP or SCTP ports |
|
||||
| `redirect_to_port` | error | the local port isn't one port |
|
||||
|
||||
## The machine
|
||||
|
||||
These come from housecarld looking at the machine, not at the
|
||||
configuration: the dashboard and `hc status` show them.
|
||||
|
||||
| Code | | When |
|
||||
|---|---|---|
|
||||
| `nftables_service` | warning, *host* | Debian's `nftables.service` is enabled: at boot it loads `/etc/nftables.conf`, and stopping or restarting it runs `nft flush ruleset`, deleting Housecarl's table too. Disable it with `systemctl disable nftables` (without `--now`) |
|
||||
| `conntrack_full` | warning, *host* | the connection tracking table is over 80% full; once it is full, the kernel drops new connections. Raise `net.netfilter.nf_conntrack_max`, or find what opens so many ([stats.md](stats.md#warnings)) |
|
||||
| `takeover_pending` | warning, *host* | the table you imported from is still in force beside Housecarl's, so a packet must get past both; `hc takeover TABLE` deletes it once Housecarl's rules are right |
|
||||
| `shadow_unprotected` | warning, *host* | shadow mode, and no other table has a filter chain on the input hook: Housecarl's ruleset blocks nothing, and nothing else protects the machine. Load your firewall, or leave shadow mode ([shadow.md](shadow.md)) |
|
||||
| `shadow_raw_left_out` | warning, *host* | shadow mode leaves this raw rule out: it does more to packets than accept, drop or reject (NAT, setting a mark or field, a jump, a verdict map…). Its hits aren't counted, and the rules after it see what it would have taken ([shadow.md](shadow.md#what-changes-in-the-ruleset)) |
|
||||
|
||||
In shadow mode `nftables_service` and `takeover_pending` aren't raised:
|
||||
another firewall beside Housecarl is the point there.
|
||||
|
||||
## Ban lists
|
||||
|
||||
| Code | | When |
|
||||
|---|---|---|
|
||||
| `banlist_name` | error | not a valid name (as for zones and sets) |
|
||||
| `banlist_name_taken` | error | two ban lists with one name |
|
||||
| `banlist_scope` | error | a scope other than `firewall`, `hosts` or `everything` |
|
||||
| `banlist_hosts` | error | scope `hosts` without hosts, or hosts with another scope |
|
||||
| `banlist_address` | error | a host or never entry that isn't an address, prefix or range |
|
||||
| `unknown_set` | error | a host or never entry naming a set that doesn't exist |
|
||||
| `banlist_time` | error | negative times (but `max_time` -1), or a default longer than the longest |
|
||||
| `banlist_entries` | error | `max_entries` below 0 or above 1000000 |
|
||||
| `banlist_group` | error | `group_v6` outside 16 to 128 |
|
||||
@@ -3,12 +3,14 @@ module housecarl
|
||||
go 1.27.1
|
||||
|
||||
require (
|
||||
github.com/aymanbagabas/go-udiff v0.4.1
|
||||
github.com/go-acme/lego/v4 v4.35.2
|
||||
github.com/miekg/dns v1.1.73
|
||||
github.com/spf13/cobra v1.10.2
|
||||
github.com/spf13/pflag v1.0.9
|
||||
go.yaml.in/yaml/v3 v3.0.5
|
||||
golang.org/x/crypto v0.57.0
|
||||
golang.org/x/net v0.58.0
|
||||
golang.org/x/sys v0.48.0
|
||||
golang.org/x/term v0.46.0
|
||||
modernc.org/sqlite v1.60.1
|
||||
)
|
||||
@@ -22,8 +24,7 @@ require (
|
||||
github.com/mattn/go-isatty v0.0.24 // indirect
|
||||
github.com/ncruces/go-strftime v1.0.0 // indirect
|
||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
|
||||
github.com/spf13/pflag v1.0.9 // indirect
|
||||
golang.org/x/sys v0.48.0 // indirect
|
||||
golang.org/x/net v0.58.0 // indirect
|
||||
golang.org/x/text v0.42.0 // indirect
|
||||
modernc.org/libc v1.77.1 // indirect
|
||||
modernc.org/mathutil v1.7.1 // indirect
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
github.com/aymanbagabas/go-udiff v0.4.1 h1:OEIrQ8maEeDBXQDoGCbbTTXYJMYRCRO1fnodZ12Gv5o=
|
||||
github.com/aymanbagabas/go-udiff v0.4.1/go.mod h1:0L9PGwj20lrtmEMeyw4WKJ/TMyDtvAoK9bf2u/mNo3w=
|
||||
github.com/cenkalti/backoff/v5 v5.0.3 h1:ZN+IMa753KfX5hd8vVaMixjnqRZ3y8CuJKRKj1xcsSM=
|
||||
github.com/cenkalti/backoff/v5 v5.0.3/go.mod h1:rkhZdG3JZukswDf7f0cwqPNk4K0sa+F97BxZthm/crw=
|
||||
github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
|
||||
|
||||
+295
-5
@@ -26,6 +26,8 @@ type ErrorInfo struct {
|
||||
Problems []validate.Problem `json:"problems,omitempty"`
|
||||
// Result is the preview of a refused change.
|
||||
Result json.RawMessage `json:"result,omitempty"`
|
||||
// Op is the changeset operation (its seq) that couldn't be applied.
|
||||
Op int `json:"op,omitempty"`
|
||||
}
|
||||
|
||||
// Whoami describes the caller.
|
||||
@@ -38,8 +40,18 @@ type Whoami struct {
|
||||
TokenID int64 `json:"token_id,omitempty"`
|
||||
CSRFToken string `json:"csrf_token,omitempty"`
|
||||
Build version.Build `json:"build"`
|
||||
// Mode is enforcing, or shadow (docs/shadow.md).
|
||||
Mode string `json:"mode"`
|
||||
}
|
||||
|
||||
// Modes: housecarld's ruleset decides, or (shadow mode, the
|
||||
// configuration file's shadow.enabled) it only shows what it would do,
|
||||
// beside another firewall.
|
||||
const (
|
||||
ModeEnforcing = "enforcing"
|
||||
ModeShadow = "shadow"
|
||||
)
|
||||
|
||||
// Login is the login request.
|
||||
type Login struct {
|
||||
Username string `json:"username"`
|
||||
@@ -64,6 +76,34 @@ type MutationResponse struct {
|
||||
// ConfirmBy is set when the change is on probation: unless confirmed
|
||||
// (POST /confirm) by then, it is rolled back.
|
||||
ConfirmBy *time.Time `json:"confirm_by,omitempty"`
|
||||
// ConfirmToken lets the client that made the change confirm it by
|
||||
// itself (POST /confirm {"token"}): from the same address, over a new
|
||||
// connection, which proves it can still reach the API. The answer
|
||||
// closes its connection, so the next request opens a new one.
|
||||
ConfirmToken string `json:"confirm_token,omitempty"`
|
||||
// Diff is the ruleset's unified diff, for a dry run.
|
||||
Diff string `json:"diff,omitempty"`
|
||||
// Staged: the change was added to a changeset (Changeset), not
|
||||
// applied.
|
||||
Staged bool `json:"staged,omitempty"`
|
||||
Changeset *model.Changeset `json:"changeset,omitempty"`
|
||||
}
|
||||
|
||||
// ChangesetInput creates a changeset, or changes its description.
|
||||
type ChangesetInput struct {
|
||||
Description string `json:"description"`
|
||||
}
|
||||
|
||||
// ChangesetPreview is what applying a changeset would do: its changes,
|
||||
// the problems, nftables' check and the ruleset's diff.
|
||||
type ChangesetPreview struct {
|
||||
// Valid: it can be applied as it is.
|
||||
Valid bool `json:"valid"`
|
||||
Changes []model.Change `json:"changes"`
|
||||
Warnings []validate.Problem `json:"warnings,omitempty"`
|
||||
// Error says why it can't be applied (Op: which operation, if one).
|
||||
Error *ErrorInfo `json:"error,omitempty"`
|
||||
Diff string `json:"diff,omitempty"`
|
||||
}
|
||||
|
||||
// HostInterface is one of the server's network interfaces, with the zone
|
||||
@@ -125,15 +165,23 @@ type LogEntry struct {
|
||||
TCPFlags string `json:"tcp_flags,omitempty"`
|
||||
RuleMark string `json:"mark,omitempty"`
|
||||
Verdict string `json:"verdict,omitempty"`
|
||||
// Shadow: logged in shadow mode, where a drop or reject is what would
|
||||
// have been done; the packet went on.
|
||||
Shadow bool `json:"shadow,omitempty"`
|
||||
}
|
||||
|
||||
// Status is the dashboard's overview.
|
||||
type Status struct {
|
||||
Build version.Build `json:"build"`
|
||||
Started time.Time `json:"started"`
|
||||
Database string `json:"database,omitempty"`
|
||||
Applied bool `json:"applied"`
|
||||
ApplyError string `json:"apply_error,omitempty"`
|
||||
Build version.Build `json:"build"`
|
||||
Started time.Time `json:"started"`
|
||||
Database string `json:"database,omitempty"`
|
||||
// Mode is enforcing or shadow; ShadowPosition (shadow mode) after or
|
||||
// before: where Housecarl's filter chains run beside the other
|
||||
// firewall's.
|
||||
Mode string `json:"mode"`
|
||||
ShadowPosition string `json:"shadow_position,omitempty"`
|
||||
Applied bool `json:"applied"`
|
||||
ApplyError string `json:"apply_error,omitempty"`
|
||||
// Unconfigured: the configuration is empty, and no ruleset was
|
||||
// applied yet.
|
||||
Unconfigured bool `json:"unconfigured,omitempty"`
|
||||
@@ -168,6 +216,9 @@ type ImportInput struct {
|
||||
// Replace replaces Housecarl's configuration; without it, the import
|
||||
// is refused when there is one.
|
||||
Replace bool `json:"replace,omitempty"`
|
||||
// Zones names the zones by interface ({"enp2s0": "lan"}); interfaces
|
||||
// given the same name share a zone.
|
||||
Zones map[string]string `json:"zones,omitempty"`
|
||||
}
|
||||
|
||||
// ImportResult reports an import.
|
||||
@@ -274,3 +325,242 @@ type CertificateResponse struct {
|
||||
Source string `json:"source"`
|
||||
Status *CertStatus `json:"status,omitempty"`
|
||||
}
|
||||
|
||||
// QuotaReset is POST /rules/{ref}/reset-quota's optional body. Without
|
||||
// one the whole quota starts again. Address starts one address of a
|
||||
// per-address quota again (any address in its group: 10.0.10.23, or an
|
||||
// IPv6 address for its /64); Give takes that many bytes off what was used
|
||||
// instead (the address's, or every address's), until the next reset.
|
||||
type QuotaReset struct {
|
||||
Address string `json:"address,omitempty"`
|
||||
Give int64 `json:"give,omitempty"`
|
||||
}
|
||||
|
||||
// QuotaInfo is a rule's quota and what it used.
|
||||
type QuotaInfo struct {
|
||||
RuleID int64 `json:"rule_id"`
|
||||
Rule string `json:"rule"`
|
||||
Quota model.Quota `json:"quota"`
|
||||
// Used is a total quota's bytes; Addresses a per-address quota's,
|
||||
// each ("192.168.1.20", or "2001:db8:1::/64").
|
||||
Used int64 `json:"used"`
|
||||
Addresses []QuotaAddress `json:"addresses,omitempty"`
|
||||
LastReset *time.Time `json:"last_reset,omitempty"`
|
||||
NextReset *time.Time `json:"next_reset,omitempty"`
|
||||
}
|
||||
|
||||
// QuotaAddress is one address's (or prefix's) use of a per-address quota.
|
||||
type QuotaAddress struct {
|
||||
Address string `json:"address"`
|
||||
Used int64 `json:"used"`
|
||||
}
|
||||
|
||||
// ConfirmInput confirms the changes on probation. With Token (a change's
|
||||
// confirm_token), it is the client that made the change confirming it by
|
||||
// itself; without, a person's confirmation.
|
||||
type ConfirmInput struct {
|
||||
Token string `json:"token,omitempty"`
|
||||
}
|
||||
|
||||
// Connection is a connection the kernel tracks (conntrack), as it was
|
||||
// first seen: Source:SourcePort → Destination:DestinationPort. NAT shows
|
||||
// as DNATTo (where it was really sent: a port forward) and SNATTo (the
|
||||
// source it went out with: masquerading).
|
||||
type Connection struct {
|
||||
ID uint32 `json:"id,omitempty"`
|
||||
Family string `json:"family"`
|
||||
Protocol string `json:"protocol"`
|
||||
Source string `json:"source"`
|
||||
SourcePort int `json:"source_port,omitempty"`
|
||||
Destination string `json:"destination"`
|
||||
DestinationPort int `json:"destination_port,omitempty"`
|
||||
ICMPID int `json:"icmp_id,omitempty"`
|
||||
ICMPType int `json:"icmp_type,omitempty"`
|
||||
ICMPCode int `json:"icmp_code,omitempty"`
|
||||
DNATTo string `json:"dnat_to,omitempty"`
|
||||
DNATPort int `json:"dnat_port,omitempty"`
|
||||
SNATTo string `json:"snat_to,omitempty"`
|
||||
// State is TCP's (established, time-wait, …); Assured: traffic went
|
||||
// both ways.
|
||||
State string `json:"state,omitempty"`
|
||||
Assured bool `json:"assured,omitempty"`
|
||||
// Timeout is the seconds left until it is forgotten unless more
|
||||
// traffic comes.
|
||||
Timeout int `json:"timeout"`
|
||||
Mark uint32 `json:"mark,omitempty"`
|
||||
// Packets and Bytes, both ways, when the kernel counts them
|
||||
// (net.netfilter.nf_conntrack_acct).
|
||||
Packets uint64 `json:"packets,omitempty"`
|
||||
Bytes uint64 `json:"bytes,omitempty"`
|
||||
}
|
||||
|
||||
// NamesInput asks for the names of addresses (POST /names).
|
||||
type NamesInput struct {
|
||||
Addresses []string `json:"addresses"`
|
||||
}
|
||||
|
||||
// Names are the names found for the inside addresses asked about, by
|
||||
// address; those without one (or still being looked up) are left out.
|
||||
// Enabled is false when names are off (names.enabled).
|
||||
type Names struct {
|
||||
Enabled bool `json:"enabled"`
|
||||
Names map[string]string `json:"names"`
|
||||
}
|
||||
|
||||
// CloseConnections closes connections: these, or every one Search finds
|
||||
// (as GET /connections?q= does).
|
||||
type CloseConnections struct {
|
||||
Connections []Connection `json:"connections,omitempty"`
|
||||
Search string `json:"search,omitempty"`
|
||||
}
|
||||
|
||||
// TraceInput says what to trace: packets from or to Address (an address
|
||||
// or prefix), on Port (either way), of Protocol, for Seconds (0: 60).
|
||||
type TraceInput struct {
|
||||
Address string `json:"address,omitempty"`
|
||||
Port int `json:"port,omitempty"`
|
||||
Protocol string `json:"protocol,omitempty"`
|
||||
Seconds int `json:"seconds,omitempty"`
|
||||
}
|
||||
|
||||
// TraceStep is one rule or policy a traced packet met, in any table.
|
||||
type TraceStep struct {
|
||||
Table string `json:"table"` // "inet housecarl", "ip nat"…
|
||||
Chain string `json:"chain"`
|
||||
Kind string `json:"kind"` // rule or policy
|
||||
// Rule is the rule as nftables writes it; Mark its Housecarl mark
|
||||
// ("r12", "b-established"), if it is one of Housecarl's.
|
||||
Rule string `json:"rule,omitempty"`
|
||||
Mark string `json:"mark,omitempty"`
|
||||
Verdict string `json:"verdict"` // accept, drop, continue, jump X…
|
||||
// Label names Housecarl's object ("rule input #12 (ssh)").
|
||||
Label string `json:"label,omitempty"`
|
||||
}
|
||||
|
||||
// TracePacket is one traced packet: its headers as it arrived, every
|
||||
// step, and its fate.
|
||||
type TracePacket struct {
|
||||
ID string `json:"id"`
|
||||
Time time.Time `json:"time"`
|
||||
In string `json:"in,omitempty"`
|
||||
Out string `json:"out,omitempty"`
|
||||
Protocol string `json:"protocol,omitempty"`
|
||||
Source string `json:"source,omitempty"`
|
||||
SourcePort int `json:"source_port,omitempty"`
|
||||
Destination string `json:"destination,omitempty"`
|
||||
DestinationPort int `json:"destination_port,omitempty"`
|
||||
TCPFlags string `json:"tcp_flags,omitempty"`
|
||||
ICMPType string `json:"icmp_type,omitempty"`
|
||||
Steps []TraceStep `json:"steps"`
|
||||
// Verdict is accepted or dropped; DecidedBy the step that decided.
|
||||
Verdict string `json:"verdict"`
|
||||
DecidedBy TraceStep `json:"decided_by"`
|
||||
// WouldDrop, in shadow mode, is the step of Housecarl's where it
|
||||
// would have dropped (or rejected) the packet, had it been enforcing.
|
||||
WouldDrop *TraceStep `json:"would_drop,omitempty"`
|
||||
}
|
||||
|
||||
// TraceStatus is the trace running (or the last one).
|
||||
type TraceStatus struct {
|
||||
Running bool `json:"running"`
|
||||
Input TraceInput `json:"input"`
|
||||
Started time.Time `json:"started"`
|
||||
Ends time.Time `json:"ends"`
|
||||
Packets []TracePacket `json:"packets"`
|
||||
Error string `json:"error,omitempty"`
|
||||
}
|
||||
|
||||
// Matrix is what the rules do to new connections between zones:
|
||||
// zone → zone (the forward chain), zone → the firewall (input) and the
|
||||
// firewall → zone (output). "firewall" stands for the machine itself.
|
||||
type Matrix struct {
|
||||
Zones []string `json:"zones"`
|
||||
Cells []MatrixCell `json:"cells"`
|
||||
}
|
||||
|
||||
// MatrixCell is one pair: what happens to new connections from From to
|
||||
// To that no exception covers (Verdict, decided by DecidedBy), and the
|
||||
// exceptions: rules and built-ins that decide for some of them (some
|
||||
// addresses, services, times…), in the order they are checked.
|
||||
type MatrixCell struct {
|
||||
From string `json:"from"`
|
||||
To string `json:"to"`
|
||||
Chain string `json:"chain"`
|
||||
Verdict string `json:"verdict"` // allowed or blocked
|
||||
DecidedBy string `json:"decided_by"`
|
||||
RuleID int64 `json:"rule_id,omitempty"` // the deciding rule
|
||||
Exceptions []MatrixException `json:"exceptions"`
|
||||
// Summary is the pair at a glance: allowed or blocked when every new
|
||||
// connection gets the same, partly when some are let through and
|
||||
// some not.
|
||||
Summary string `json:"summary"`
|
||||
// Allowed and Blocked count the exceptions that let part of the
|
||||
// traffic through, and that block part of it.
|
||||
Allowed int `json:"allowed"`
|
||||
Blocked int `json:"blocked"`
|
||||
// Open sums up what the allowing exceptions let through, shortest
|
||||
// first in their order: "ssh", "dns", "5 port forwards".
|
||||
Open []string `json:"open"`
|
||||
// SameZone: from a zone to itself. Note says what of that traffic
|
||||
// passes the firewall at all.
|
||||
SameZone bool `json:"same_zone,omitempty"`
|
||||
Note string `json:"note,omitempty"`
|
||||
// Hits are the deciding rule's (or policy's) packets, all of its
|
||||
// traffic since housecarld started (nil: no counter).
|
||||
Hits *uint64 `json:"hits,omitempty"`
|
||||
// Marks are the counters behind Hits.
|
||||
Marks []string `json:"-"`
|
||||
}
|
||||
|
||||
// MatrixException is a rule (or built-in) that decides for part of a
|
||||
// pair's traffic.
|
||||
type MatrixException struct {
|
||||
RuleID int64 `json:"rule_id,omitempty"` // 0: a built-in, or port forwards
|
||||
Chain string `json:"chain"`
|
||||
Label string `json:"label"`
|
||||
// Verdict is allowed, blocked, logged (and goes on), or unknown (a raw
|
||||
// rule Housecarl can't read).
|
||||
Verdict string `json:"verdict"`
|
||||
// What it covers: "from trusted · ssh", "Monday 08:00–17:00".
|
||||
What string `json:"what"`
|
||||
// Short is What in a word or two, for the overview: its services,
|
||||
// or else its first condition.
|
||||
Short string `json:"short"`
|
||||
// Hits are its packets, all of its traffic since housecarld started
|
||||
// (a rule may apply to several pairs; nil: no counter).
|
||||
Hits *uint64 `json:"hits,omitempty"`
|
||||
// Marks are the counters behind Hits ("r12", "b-ping", "f3").
|
||||
Marks []string `json:"-"`
|
||||
}
|
||||
|
||||
// BanInput asks a ban list to ban an address (POST
|
||||
// /banlists/{list}/bans).
|
||||
type BanInput struct {
|
||||
Address string `json:"address"`
|
||||
// Protocol (tcp, udp) and Ports limit the ban; with neither, every
|
||||
// port. Ports are numbers or service names, in a list or a string
|
||||
// ("22", "ssh", "http,https"); "all", "0:65535" or "1-65535" mean
|
||||
// every port. Ports without a protocol take the service's protocols,
|
||||
// or for a number tcp and udp.
|
||||
Protocol string `json:"protocol,omitempty"`
|
||||
Ports json.RawMessage `json:"ports,omitempty"`
|
||||
// Time is the ban's length in seconds: 0 the list's default, -1 its
|
||||
// longest (or for ever). Longer than the list allows is cut down.
|
||||
Time int `json:"time,omitempty"`
|
||||
Source string `json:"source,omitempty"`
|
||||
Reason string `json:"reason,omitempty"`
|
||||
}
|
||||
|
||||
// BanResult is the answer to a ban: the bans made or renewed (one per
|
||||
// protocol and port), Existing for each already in force.
|
||||
type BanResult struct {
|
||||
Bans []BanMade `json:"bans"`
|
||||
// Closed counts the banned address's connections closed.
|
||||
Closed int `json:"closed"`
|
||||
}
|
||||
|
||||
// BanMade is a ban made, or renewed (Existing).
|
||||
type BanMade struct {
|
||||
model.Ban
|
||||
Existing bool `json:"existing,omitempty"`
|
||||
}
|
||||
+20
-12
@@ -147,13 +147,14 @@ func NewSessionID() (id, csrf string) {
|
||||
const (
|
||||
ResConfig = "config" // the firewall configuration, its history and imports
|
||||
ResSets = "sets" // dynamic sets' entries (a grant may name the sets)
|
||||
ResBans = "bans" // ban lists' bans (a grant may name the lists)
|
||||
ResLog = "log" // counters, the drop log and the status
|
||||
ResAudit = "audit" // the audit log
|
||||
ResAll = "*"
|
||||
)
|
||||
|
||||
// Resources lists them in display order.
|
||||
var Resources = []string{ResConfig, ResSets, ResLog, ResAudit}
|
||||
var Resources = []string{ResConfig, ResSets, ResBans, ResLog, ResAudit}
|
||||
|
||||
// Access levels.
|
||||
const (
|
||||
@@ -163,8 +164,8 @@ const (
|
||||
|
||||
// Roles are ready-made sets of grants.
|
||||
var Roles = map[string][]model.Grant{
|
||||
"viewer": {{Resource: ResConfig, Access: Read}, {Resource: ResSets, Access: Read}, {Resource: ResLog, Access: Read}},
|
||||
"operator": {{Resource: ResConfig, Access: Write}, {Resource: ResSets, Access: Write}, {Resource: ResLog, Access: Read}, {Resource: ResAudit, Access: Read}},
|
||||
"viewer": {{Resource: ResConfig, Access: Read}, {Resource: ResSets, Access: Read}, {Resource: ResBans, Access: Read}, {Resource: ResLog, Access: Read}},
|
||||
"operator": {{Resource: ResConfig, Access: Write}, {Resource: ResSets, Access: Write}, {Resource: ResBans, Access: Write}, {Resource: ResLog, Access: Read}, {Resource: ResAudit, Access: Read}},
|
||||
"log": {{Resource: ResLog, Access: Read}},
|
||||
}
|
||||
|
||||
@@ -179,12 +180,13 @@ func RoleNames() []string {
|
||||
}
|
||||
|
||||
// ParseGrant reads "resource:access" ("log:read", "*:write"), and for
|
||||
// sets optionally the sets it is limited to ("sets:write:guest,portal").
|
||||
// sets and bans optionally the sets or ban lists it is limited to
|
||||
// ("sets:write:guest,portal", "bans:write:web").
|
||||
func ParseGrant(s string) (model.Grant, error) {
|
||||
parts := strings.SplitN(strings.TrimSpace(s), ":", 3)
|
||||
g := model.Grant{Resource: strings.ToLower(parts[0])}
|
||||
if len(parts) < 2 {
|
||||
return g, fmt.Errorf("grant %q: write it as resource:access, e.g. log:read, or sets:write:SET1,SET2", s)
|
||||
return g, fmt.Errorf("grant %q: write it as resource:access, e.g. log:read, sets:write:SET1,SET2 or bans:write:LIST", s)
|
||||
}
|
||||
g.Access = strings.ToLower(parts[1])
|
||||
if len(parts) == 3 {
|
||||
@@ -211,8 +213,8 @@ func ValidateGrant(g *model.Grant) error {
|
||||
if g.Access != Read && g.Access != Write {
|
||||
return fmt.Errorf("access must be read or write, not %q", g.Access)
|
||||
}
|
||||
if len(g.Sets) > 0 && g.Resource != ResSets {
|
||||
return fmt.Errorf("only a sets grant can name sets (sets:write:%s)", strings.Join(g.Sets, ","))
|
||||
if len(g.Sets) > 0 && g.Resource != ResSets && g.Resource != ResBans {
|
||||
return fmt.Errorf("only a sets or bans grant can name sets or ban lists (sets:write:%s)", strings.Join(g.Sets, ","))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -272,20 +274,26 @@ func covers(gs []model.Grant, resource string, write bool) bool {
|
||||
|
||||
// CanSet reports whether the principal may read or change the entries of
|
||||
// a dynamic set: a sets grant covers it if it names no sets, or names it.
|
||||
func (p *Principal) CanSet(name string, write bool) bool {
|
||||
func (p *Principal) CanSet(name string, write bool) bool { return p.canNamed(ResSets, name, write) }
|
||||
|
||||
// CanBan reports whether the principal may read or change the bans of a
|
||||
// ban list: a bans grant covers it if it names no lists, or names it.
|
||||
func (p *Principal) CanBan(list string, write bool) bool { return p.canNamed(ResBans, list, write) }
|
||||
|
||||
func (p *Principal) canNamed(resource, name string, write bool) bool {
|
||||
if p == nil || p.User == nil {
|
||||
return false
|
||||
}
|
||||
allowed := p.User.Admin || coversSet(p.User.Grants, name, write)
|
||||
allowed := p.User.Admin || coversNamed(p.User.Grants, resource, name, write)
|
||||
if p.Token != nil && p.Token.Grants != nil {
|
||||
allowed = allowed && coversSet(p.Token.Grants, name, write)
|
||||
allowed = allowed && coversNamed(p.Token.Grants, resource, name, write)
|
||||
}
|
||||
return allowed
|
||||
}
|
||||
|
||||
func coversSet(gs []model.Grant, name string, write bool) bool {
|
||||
func coversNamed(gs []model.Grant, resource, name string, write bool) bool {
|
||||
for _, g := range gs {
|
||||
if (g.Resource != ResAll && g.Resource != ResSets) || (write && g.Access != Write) {
|
||||
if (g.Resource != ResAll && g.Resource != resource) || (write && g.Access != Write) {
|
||||
continue
|
||||
}
|
||||
if len(g.Sets) == 0 {
|
||||
|
||||
@@ -14,6 +14,7 @@ import (
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
@@ -28,6 +29,29 @@ type Client struct {
|
||||
HTTP *http.Client
|
||||
// DryRun previews changes instead of making them.
|
||||
DryRun bool
|
||||
// Changeset stages configuration changes in that changeset instead
|
||||
// of applying them (X-Housecarl-Changeset).
|
||||
Changeset int64
|
||||
// BeforeStage, if set, runs once before the first change is staged,
|
||||
// and can stop it (the CLI checks HOUSECARL_CHANGESET's changeset).
|
||||
BeforeStage func() error
|
||||
staged bool
|
||||
}
|
||||
|
||||
// Staged reports whether a request is a configuration change that a
|
||||
// changeset takes: objects, settings and undo, and the changes that can't
|
||||
// be staged (import, takeover), which the server then refuses rather than
|
||||
// apply them. Changesets' own calls and dynamic set entries aren't.
|
||||
func Staged(method, path string) bool {
|
||||
if method == http.MethodGet || strings.Contains(path, "/elements") {
|
||||
return false
|
||||
}
|
||||
for _, p := range []string{"/zones", "/sets", "/services", "/rules", "/forwards", "/snats", "/redirects", "/settings", "/history/", "/import/", "/takeover"} {
|
||||
if strings.HasPrefix(path, p) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// Options configure New.
|
||||
@@ -107,6 +131,12 @@ func (c *Client) Do(ctx context.Context, method, path string, query url.Values,
|
||||
}
|
||||
query.Set("dry_run", "1")
|
||||
}
|
||||
stage := c.Changeset != 0 && Staged(method, path)
|
||||
if stage && !c.staged && c.BeforeStage != nil {
|
||||
if err := c.BeforeStage(); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
u := c.BaseURL + api.Prefix + path
|
||||
if len(query) > 0 {
|
||||
u += "?" + query.Encode()
|
||||
@@ -131,6 +161,10 @@ func (c *Client) Do(ctx context.Context, method, path string, query url.Values,
|
||||
if c.Token != "" {
|
||||
req.Header.Set("Authorization", "Bearer "+c.Token)
|
||||
}
|
||||
if stage {
|
||||
req.Header.Set("X-Housecarl-Changeset", strconv.FormatInt(c.Changeset, 10))
|
||||
c.staged = true
|
||||
}
|
||||
resp, err := c.HTTP.Do(req)
|
||||
if err != nil {
|
||||
if c.BaseURL == "http://housecarld" {
|
||||
|
||||
@@ -39,6 +39,57 @@ type Config struct {
|
||||
// Ruleset is where the confirmed ruleset is saved for the boot unit
|
||||
// (housecarl-ruleset.service loads it before the network comes up).
|
||||
Ruleset string `yaml:"ruleset"`
|
||||
// Stats keeps traffic, hits and drops over time (docs/stats.md).
|
||||
Stats Stats `yaml:"stats"`
|
||||
// Shadow loads the ruleset without letting it block, reject or
|
||||
// translate anything, beside the firewall in force (docs/shadow.md).
|
||||
Shadow Shadow `yaml:"shadow"`
|
||||
// Names shows inside addresses' names, by reverse lookups
|
||||
// (docs/configuration.md).
|
||||
Names Names `yaml:"names"`
|
||||
}
|
||||
|
||||
// Names configures the reverse lookups that put names beside inside
|
||||
// addresses in the web UI and hc.
|
||||
type Names struct {
|
||||
Enabled bool `yaml:"enabled"`
|
||||
// Resolver is a DNS server ("10.10.0.1:53"; "": the system's).
|
||||
Resolver string `yaml:"resolver"`
|
||||
// Cache is how long a name is kept.
|
||||
Cache Duration `yaml:"cache"`
|
||||
}
|
||||
|
||||
// Shadow is shadow mode: Housecarl's ruleset is loaded and counts, logs
|
||||
// and traces what it would do, while another firewall decides.
|
||||
type Shadow struct {
|
||||
Enabled bool `yaml:"enabled"`
|
||||
// Position is where Housecarl's filter chains run: after the other
|
||||
// firewall's (it sees what that one lets through, so what it would
|
||||
// drop is what switching would break) or before (it sees everything).
|
||||
Position string `yaml:"position"`
|
||||
}
|
||||
|
||||
// Shadow positions.
|
||||
const (
|
||||
ShadowAfter = "after"
|
||||
ShadowBefore = "before"
|
||||
)
|
||||
|
||||
// Stats configures statistics: a database of their own, aggregates
|
||||
// only, pruned by age.
|
||||
type Stats struct {
|
||||
Enabled bool `yaml:"enabled"`
|
||||
Database string `yaml:"database"`
|
||||
Retention StatsRetention `yaml:"retention"`
|
||||
// TopN is how many entries the top lists keep per hour and day.
|
||||
TopN int `yaml:"top_n"`
|
||||
}
|
||||
|
||||
// StatsRetention is how long each resolution is kept.
|
||||
type StatsRetention struct {
|
||||
Minute Duration `yaml:"minute"`
|
||||
Hour Duration `yaml:"hour"`
|
||||
Day Duration `yaml:"day"`
|
||||
}
|
||||
|
||||
// TLS is the web UI's certificate: from files, or obtained through ACME
|
||||
@@ -183,7 +234,11 @@ func Default() *Config {
|
||||
Log: Log{Level: "info", Format: "text"},
|
||||
Session: Session{Lifetime: Duration(12 * time.Hour), IdleTimeout: Duration(2 * time.Hour), CookieSecure: "auto"},
|
||||
Backup: Backup{Dir: "/var/backups/housecarl", Retention: Duration(30 * 24 * time.Hour), KeepAtLeast: 7},
|
||||
TLS: TLS{ACME: ACME{Directory: "letsencrypt-staging", Resolvers: []string{"1.1.1.1:53", "9.9.9.9:53", "8.8.8.8:53"}, CheckPropagation: true}},
|
||||
Stats: Stats{Enabled: true, Database: "/var/lib/housecarl/stats.db", TopN: 100,
|
||||
Retention: StatsRetention{Minute: Duration(7 * 24 * time.Hour), Hour: Duration(90 * 24 * time.Hour), Day: Duration(730 * 24 * time.Hour)}},
|
||||
Shadow: Shadow{Position: ShadowAfter},
|
||||
Names: Names{Enabled: true, Cache: Duration(10 * time.Minute)},
|
||||
TLS: TLS{ACME: ACME{Directory: "letsencrypt-staging", Resolvers: []string{"1.1.1.1:53", "9.9.9.9:53", "8.8.8.8:53"}, CheckPropagation: true}},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -254,6 +309,17 @@ func (c *Config) check() error {
|
||||
default:
|
||||
return fmt.Errorf("log.level must be debug, info, warn or error")
|
||||
}
|
||||
if c.Shadow.Position != ShadowAfter && c.Shadow.Position != ShadowBefore {
|
||||
return fmt.Errorf("shadow.position must be after or before")
|
||||
}
|
||||
if c.Names.Resolver != "" {
|
||||
if _, err := netip.ParseAddrPort(c.Names.Resolver); err != nil {
|
||||
return fmt.Errorf("names.resolver must be an address and port, such as 10.10.0.1:53")
|
||||
}
|
||||
}
|
||||
if c.Names.Cache <= 0 {
|
||||
return fmt.Errorf("names.cache must be a duration, such as 10m")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -393,4 +459,29 @@ backup:
|
||||
dir: /var/backups/housecarl
|
||||
retention: 30d
|
||||
keep_at_least: 7
|
||||
|
||||
# Statistics (docs/stats.md): traffic per interface and zone, hits and
|
||||
# drops, blocked sources, connections, quotas, kept over time in a database
|
||||
# of their own (aggregates only). Deleting it is always safe. Traffic per
|
||||
# device is a firewall setting: hc settings set --stats-per-device on.
|
||||
stats:
|
||||
enabled: true
|
||||
database: /var/lib/housecarl/stats.db
|
||||
retention:
|
||||
minute: 7d # per-minute figures
|
||||
hour: 90d # hourly figures and top lists
|
||||
day: 730d # daily figures and top lists
|
||||
top_n: 100 # entries kept per top list, per hour and day
|
||||
|
||||
# Shadow mode (docs/shadow.md): Housecarl's ruleset is loaded beside the
|
||||
# firewall you have now, but blocks, rejects and translates nothing. Its
|
||||
# counters, log and tracer show what it would do. No change is on
|
||||
# probation, and takeovers and closing connections are refused. housecarld
|
||||
# reads this when it starts: restart it to switch.
|
||||
shadow:
|
||||
enabled: false
|
||||
# after: Housecarl's filter chains run after the other firewall's, and
|
||||
# see only what it lets through: what Housecarl would drop is what
|
||||
# switching over would break. before: they see every packet.
|
||||
position: after
|
||||
`
|
||||
@@ -35,6 +35,7 @@ func TestErrors(t *testing.T) {
|
||||
"{tls: {acme: {domain: '*.example.com'}}, jarl: {server: https://j, token: x}}": "full name",
|
||||
"{tls: {acme: {domain: d.example.com, directory: http://x}}, jarl: {server: https://j, token: x}}": "https://",
|
||||
"ruleset: relative.nft": "ruleset",
|
||||
"shadow: {position: x}": "shadow.position",
|
||||
}
|
||||
for yml, want := range cases {
|
||||
path := filepath.Join(t.TempDir(), "s.yaml")
|
||||
|
||||
@@ -0,0 +1,351 @@
|
||||
// Package conntrack lists and closes the kernel's tracked connections,
|
||||
// through ctnetlink (the netfilter netlink subsystem for connection
|
||||
// tracking): what the "established" built-in lets through, and what a
|
||||
// change of rules doesn't stop by itself.
|
||||
//
|
||||
// Closing a connection deletes its tracking entry. Its next packet is
|
||||
// then checked by the rules again, as a new connection would be: one the
|
||||
// rules now block is stopped, one they allow carries on (TCP connections
|
||||
// are picked up again mid-stream).
|
||||
package conntrack
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/binary"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/netip"
|
||||
"sync/atomic"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"golang.org/x/sys/unix"
|
||||
|
||||
"housecarl/internal/api"
|
||||
)
|
||||
|
||||
const (
|
||||
subsysCtnetlink = 1
|
||||
msgNew = 0
|
||||
msgGet = 1
|
||||
msgDelete = 2
|
||||
|
||||
ctaTupleOrig = 1
|
||||
ctaTupleReply = 2
|
||||
ctaStatus = 3
|
||||
ctaProtoinfo = 4
|
||||
ctaTimeout = 7
|
||||
ctaMark = 8
|
||||
ctaCountersOrig = 9
|
||||
ctaCountersReply = 10
|
||||
ctaID = 12
|
||||
|
||||
ctaTupleIP = 1
|
||||
ctaTupleProto = 2
|
||||
|
||||
ctaIPv4Src = 1
|
||||
ctaIPv4Dst = 2
|
||||
ctaIPv6Src = 3
|
||||
ctaIPv6Dst = 4
|
||||
|
||||
ctaProtoNum = 1
|
||||
ctaProtoSrcPort = 2
|
||||
ctaProtoDstPort = 3
|
||||
ctaProtoICMPID = 4
|
||||
ctaProtoICMPType = 5
|
||||
ctaProtoICMPCode = 6
|
||||
ctaProtoICMP6ID = 7
|
||||
ctaProtoICMP6Typ = 8
|
||||
ctaProtoICMP6Cod = 9
|
||||
|
||||
ctaProtoinfoTCP = 1
|
||||
ctaProtoinfoTCPState = 1
|
||||
|
||||
ctaCountersPackets = 1
|
||||
ctaCountersBytes = 2
|
||||
|
||||
statusAssured = 1 << 2
|
||||
statusSrcNAT = 1 << 4
|
||||
statusDstNAT = 1 << 5
|
||||
|
||||
nlaNested = 0x8000
|
||||
nlaTypeMask = 0x3fff
|
||||
)
|
||||
|
||||
var tcpStates = []string{"none", "syn-sent", "syn-recv", "established", "fin-wait", "close-wait", "last-ack", "time-wait", "close", "syn-sent2"}
|
||||
|
||||
var protoNames = map[uint8]string{1: "icmp", 6: "tcp", 17: "udp", 58: "icmpv6", 132: "sctp", 136: "udplite", 33: "dccp", 47: "gre"}
|
||||
|
||||
// Client talks to ctnetlink. The zero value works.
|
||||
type Client struct{}
|
||||
|
||||
var seq atomic.Uint32
|
||||
|
||||
// List returns every tracked connection, IPv4 and IPv6.
|
||||
func (Client) List(ctx context.Context) ([]api.Connection, error) {
|
||||
msgs, err := request(ctx, msgGet, unix.NLM_F_REQUEST|unix.NLM_F_DUMP, unix.AF_UNSPEC, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out := []api.Connection{}
|
||||
for _, m := range msgs {
|
||||
if c, ok := parse(m); ok {
|
||||
out = append(out, c)
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// Close deletes a connection's tracking entry, found by its original
|
||||
// direction (addresses, protocol, ports or ICMP id). Gone already is fine.
|
||||
func (Client) Close(ctx context.Context, c api.Connection) error {
|
||||
src, err1 := netip.ParseAddr(c.Source)
|
||||
dst, err2 := netip.ParseAddr(c.Destination)
|
||||
if err1 != nil || err2 != nil || src.Is4() != dst.Is4() {
|
||||
return fmt.Errorf("connection %s -> %s: not a pair of addresses", c.Source, c.Destination)
|
||||
}
|
||||
num := protoNumber(c.Protocol)
|
||||
if num == 0 {
|
||||
return fmt.Errorf("connection %s -> %s: unknown protocol %q", c.Source, c.Destination, c.Protocol)
|
||||
}
|
||||
family, ipSrc, ipDst := uint8(unix.AF_INET), uint16(ctaIPv4Src), uint16(ctaIPv4Dst)
|
||||
if src.Is6() {
|
||||
family, ipSrc, ipDst = unix.AF_INET6, ctaIPv6Src, ctaIPv6Dst
|
||||
}
|
||||
ip := attr(ipSrc, src.AsSlice())
|
||||
ip = append(ip, attr(ipDst, dst.AsSlice())...)
|
||||
proto := attr(ctaProtoNum, []byte{num})
|
||||
switch num {
|
||||
case 1:
|
||||
proto = append(proto, attr(ctaProtoICMPID, be16(uint16(c.ICMPID)))...)
|
||||
proto = append(proto, attr(ctaProtoICMPType, []byte{uint8(c.ICMPType)})...)
|
||||
proto = append(proto, attr(ctaProtoICMPCode, []byte{uint8(c.ICMPCode)})...)
|
||||
case 58:
|
||||
proto = append(proto, attr(ctaProtoICMP6ID, be16(uint16(c.ICMPID)))...)
|
||||
proto = append(proto, attr(ctaProtoICMP6Typ, []byte{uint8(c.ICMPType)})...)
|
||||
proto = append(proto, attr(ctaProtoICMP6Cod, []byte{uint8(c.ICMPCode)})...)
|
||||
default:
|
||||
proto = append(proto, attr(ctaProtoSrcPort, be16(uint16(c.SourcePort)))...)
|
||||
proto = append(proto, attr(ctaProtoDstPort, be16(uint16(c.DestinationPort)))...)
|
||||
}
|
||||
tuple := attr(ctaTupleIP|nlaNested, ip)
|
||||
tuple = append(tuple, attr(ctaTupleProto|nlaNested, proto)...)
|
||||
_, err := request(ctx, msgDelete, unix.NLM_F_REQUEST|unix.NLM_F_ACK, family, attr(ctaTupleOrig|nlaNested, tuple))
|
||||
if errors.Is(err, unix.ENOENT) {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func protoNumber(name string) uint8 {
|
||||
for n, s := range protoNames {
|
||||
if s == name {
|
||||
return n
|
||||
}
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
// ---- netlink
|
||||
|
||||
func request(ctx context.Context, msg uint16, flags uint16, family uint8, payload []byte) ([][]byte, error) {
|
||||
fd, err := unix.Socket(unix.AF_NETLINK, unix.SOCK_RAW|unix.SOCK_CLOEXEC, unix.NETLINK_NETFILTER)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer unix.Close(fd)
|
||||
if err := unix.Bind(fd, &unix.SockaddrNetlink{Family: unix.AF_NETLINK}); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
deadline := time.Now().Add(10 * time.Second)
|
||||
if d, ok := ctx.Deadline(); ok && d.Before(deadline) {
|
||||
deadline = d
|
||||
}
|
||||
tv := unix.NsecToTimeval(time.Until(deadline).Nanoseconds())
|
||||
unix.SetsockoptTimeval(fd, unix.SOL_SOCKET, unix.SO_RCVTIMEO, &tv)
|
||||
unix.SetsockoptInt(fd, unix.SOL_SOCKET, unix.SO_RCVBUF, 1<<22)
|
||||
|
||||
s := seq.Add(1)
|
||||
b := make([]byte, unix.SizeofNlMsghdr+4, unix.SizeofNlMsghdr+4+len(payload))
|
||||
binary.NativeEndian.PutUint16(b[4:], subsysCtnetlink<<8|msg)
|
||||
binary.NativeEndian.PutUint16(b[6:], flags)
|
||||
binary.NativeEndian.PutUint32(b[8:], s)
|
||||
b[16] = family // nfgenmsg: family, version 0, res_id 0
|
||||
b = append(b, payload...)
|
||||
binary.NativeEndian.PutUint32(b[0:], uint32(len(b)))
|
||||
if err := unix.Sendto(fd, b, 0, &unix.SockaddrNetlink{Family: unix.AF_NETLINK}); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var out [][]byte
|
||||
buf := make([]byte, 1<<16)
|
||||
for {
|
||||
n, _, err := unix.Recvfrom(fd, buf, 0)
|
||||
if err != nil {
|
||||
if errors.Is(err, unix.EAGAIN) {
|
||||
return nil, errors.New("no answer from connection tracking (is nf_conntrack loaded?)")
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
for off := 0; off+unix.SizeofNlMsghdr <= n; {
|
||||
l := int(binary.NativeEndian.Uint32(buf[off:]))
|
||||
typ := binary.NativeEndian.Uint16(buf[off+4:])
|
||||
if l < unix.SizeofNlMsghdr || off+l > n {
|
||||
return nil, errors.New("connection tracking: a malformed answer")
|
||||
}
|
||||
body := buf[off+unix.SizeofNlMsghdr : off+l]
|
||||
switch typ {
|
||||
case unix.NLMSG_DONE:
|
||||
return out, nil
|
||||
case unix.NLMSG_ERROR:
|
||||
if len(body) >= 4 {
|
||||
if code := int32(binary.NativeEndian.Uint32(body)); code != 0 {
|
||||
return nil, syscall.Errno(-code)
|
||||
}
|
||||
}
|
||||
return out, nil // the acknowledgement
|
||||
default:
|
||||
out = append(out, append([]byte(nil), body...))
|
||||
}
|
||||
off += (l + 3) &^ 3
|
||||
}
|
||||
if flags&unix.NLM_F_DUMP == 0 && flags&unix.NLM_F_ACK == 0 {
|
||||
return out, nil
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func attr(typ uint16, data []byte) []byte {
|
||||
l := 4 + len(data)
|
||||
b := make([]byte, (l+3)&^3)
|
||||
binary.NativeEndian.PutUint16(b[0:], uint16(l))
|
||||
binary.NativeEndian.PutUint16(b[2:], typ)
|
||||
copy(b[4:], data)
|
||||
return b
|
||||
}
|
||||
|
||||
func be16(v uint16) []byte {
|
||||
b := make([]byte, 2)
|
||||
binary.BigEndian.PutUint16(b, v)
|
||||
return b
|
||||
}
|
||||
|
||||
// attrs splits netlink attributes: type (without the nested flag) → data.
|
||||
func attrs(b []byte) map[uint16][]byte {
|
||||
m := map[uint16][]byte{}
|
||||
for len(b) >= 4 {
|
||||
l := int(binary.NativeEndian.Uint16(b[0:]))
|
||||
typ := binary.NativeEndian.Uint16(b[2:]) & nlaTypeMask
|
||||
if l < 4 || l > len(b) {
|
||||
break
|
||||
}
|
||||
m[typ] = b[4:l]
|
||||
b = b[min((l+3)&^3, len(b)):]
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
// ---- parsing
|
||||
|
||||
type tuple struct {
|
||||
src, dst netip.Addr
|
||||
proto uint8
|
||||
sport, dport uint16
|
||||
icmpID uint16
|
||||
icmpType, icmpCd uint8
|
||||
}
|
||||
|
||||
func parseTuple(b []byte) (tuple, bool) {
|
||||
var t tuple
|
||||
a := attrs(b)
|
||||
ip := attrs(a[ctaTupleIP])
|
||||
if v, ok := ip[ctaIPv4Src]; ok {
|
||||
t.src, _ = netip.AddrFromSlice(v)
|
||||
t.dst, _ = netip.AddrFromSlice(ip[ctaIPv4Dst])
|
||||
} else if v, ok := ip[ctaIPv6Src]; ok {
|
||||
t.src, _ = netip.AddrFromSlice(v)
|
||||
t.dst, _ = netip.AddrFromSlice(ip[ctaIPv6Dst])
|
||||
}
|
||||
p := attrs(a[ctaTupleProto])
|
||||
if v := p[ctaProtoNum]; len(v) == 1 {
|
||||
t.proto = v[0]
|
||||
}
|
||||
if v := p[ctaProtoSrcPort]; len(v) == 2 {
|
||||
t.sport = binary.BigEndian.Uint16(v)
|
||||
}
|
||||
if v := p[ctaProtoDstPort]; len(v) == 2 {
|
||||
t.dport = binary.BigEndian.Uint16(v)
|
||||
}
|
||||
for _, k := range [][3]uint16{{ctaProtoICMPID, ctaProtoICMPType, ctaProtoICMPCode}, {ctaProtoICMP6ID, ctaProtoICMP6Typ, ctaProtoICMP6Cod}} {
|
||||
if v := p[k[0]]; len(v) == 2 {
|
||||
t.icmpID = binary.BigEndian.Uint16(v)
|
||||
}
|
||||
if v := p[k[1]]; len(v) == 1 {
|
||||
t.icmpType = v[0]
|
||||
}
|
||||
if v := p[k[2]]; len(v) == 1 {
|
||||
t.icmpCd = v[0]
|
||||
}
|
||||
}
|
||||
return t, t.src.IsValid() && t.dst.IsValid()
|
||||
}
|
||||
|
||||
// parse reads one connection (an nfgenmsg and its attributes).
|
||||
func parse(m []byte) (api.Connection, bool) {
|
||||
if len(m) < 4 {
|
||||
return api.Connection{}, false
|
||||
}
|
||||
a := attrs(m[4:])
|
||||
orig, ok1 := parseTuple(a[ctaTupleOrig])
|
||||
reply, ok2 := parseTuple(a[ctaTupleReply])
|
||||
if !ok1 || !ok2 {
|
||||
return api.Connection{}, false
|
||||
}
|
||||
c := api.Connection{
|
||||
Family: "ipv4", Protocol: protoNames[orig.proto],
|
||||
Source: orig.src.String(), SourcePort: int(orig.sport), Destination: orig.dst.String(), DestinationPort: int(orig.dport),
|
||||
ICMPID: int(orig.icmpID), ICMPType: int(orig.icmpType), ICMPCode: int(orig.icmpCd),
|
||||
}
|
||||
if orig.src.Is6() {
|
||||
c.Family = "ipv6"
|
||||
}
|
||||
if c.Protocol == "" {
|
||||
c.Protocol = fmt.Sprint(orig.proto)
|
||||
}
|
||||
// NAT shows in the reply direction: it comes from where the packets
|
||||
// really went, to where they really came from.
|
||||
if reply.src != orig.dst || reply.sport != orig.dport {
|
||||
c.DNATTo = reply.src.String()
|
||||
if reply.sport != orig.dport {
|
||||
c.DNATPort = int(reply.sport)
|
||||
}
|
||||
}
|
||||
if reply.dst != orig.src || reply.dport != orig.sport {
|
||||
c.SNATTo = reply.dst.String()
|
||||
}
|
||||
if v := a[ctaStatus]; len(v) == 4 {
|
||||
st := binary.BigEndian.Uint32(v)
|
||||
c.Assured = st&statusAssured != 0
|
||||
}
|
||||
if v := a[ctaTimeout]; len(v) == 4 {
|
||||
c.Timeout = int(binary.BigEndian.Uint32(v))
|
||||
}
|
||||
if v := a[ctaMark]; len(v) == 4 {
|
||||
c.Mark = binary.BigEndian.Uint32(v)
|
||||
}
|
||||
if v := a[ctaID]; len(v) == 4 {
|
||||
c.ID = binary.BigEndian.Uint32(v)
|
||||
}
|
||||
if v := attrs(attrs(a[ctaProtoinfo])[ctaProtoinfoTCP])[ctaProtoinfoTCPState]; len(v) == 1 && int(v[0]) < len(tcpStates) {
|
||||
c.State = tcpStates[v[0]]
|
||||
}
|
||||
for _, k := range []uint16{ctaCountersOrig, ctaCountersReply} {
|
||||
ct := attrs(a[k])
|
||||
if v := ct[ctaCountersPackets]; len(v) == 8 {
|
||||
c.Packets += binary.BigEndian.Uint64(v)
|
||||
}
|
||||
if v := ct[ctaCountersBytes]; len(v) == 8 {
|
||||
c.Bytes += binary.BigEndian.Uint64(v)
|
||||
}
|
||||
}
|
||||
return c, true
|
||||
}
|
||||
@@ -0,0 +1,51 @@
|
||||
package conntrack
|
||||
|
||||
import (
|
||||
"encoding/binary"
|
||||
"net/netip"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func nested(typ uint16, parts ...[]byte) []byte {
|
||||
var b []byte
|
||||
for _, p := range parts {
|
||||
b = append(b, p...)
|
||||
}
|
||||
return attr(typ|nlaNested, b)
|
||||
}
|
||||
|
||||
func tupleAttr(typ uint16, src, dst string, proto uint8, sport, dport uint16) []byte {
|
||||
s, d := netip.MustParseAddr(src), netip.MustParseAddr(dst)
|
||||
return nested(typ,
|
||||
nested(ctaTupleIP, attr(ctaIPv4Src, s.AsSlice()), attr(ctaIPv4Dst, d.AsSlice())),
|
||||
nested(ctaTupleProto, attr(ctaProtoNum, []byte{proto}), attr(ctaProtoSrcPort, be16(sport)), attr(ctaProtoDstPort, be16(dport))))
|
||||
}
|
||||
|
||||
// A port-forwarded TCP connection from the internet, as ctnetlink
|
||||
// describes it: the reply comes from the inside host.
|
||||
func TestParse(t *testing.T) {
|
||||
status, timeout := make([]byte, 4), make([]byte, 4)
|
||||
binary.BigEndian.PutUint32(status, statusAssured|statusDstNAT)
|
||||
binary.BigEndian.PutUint32(timeout, 431999)
|
||||
msg := []byte{2, 0, 0, 0} // nfgenmsg: AF_INET
|
||||
msg = append(msg, tupleAttr(ctaTupleOrig, "198.51.100.7", "203.0.113.10", 6, 51432, 8443)...)
|
||||
msg = append(msg, tupleAttr(ctaTupleReply, "192.168.1.10", "198.51.100.7", 6, 443, 51432)...)
|
||||
msg = append(msg, attr(ctaStatus, status)...)
|
||||
msg = append(msg, attr(ctaTimeout, timeout)...)
|
||||
msg = append(msg, nested(ctaProtoinfo, nested(ctaProtoinfoTCP, attr(ctaProtoinfoTCPState, []byte{3})))...)
|
||||
c, ok := parse(msg)
|
||||
if !ok {
|
||||
t.Fatal("not parsed")
|
||||
}
|
||||
if c.Protocol != "tcp" || c.Source != "198.51.100.7" || c.SourcePort != 51432 || c.Destination != "203.0.113.10" || c.DestinationPort != 8443 ||
|
||||
c.DNATTo != "192.168.1.10" || c.DNATPort != 443 || c.SNATTo != "" || c.State != "established" || !c.Assured || c.Timeout != 431999 {
|
||||
t.Errorf("parsed %+v", c)
|
||||
}
|
||||
// Masqueraded on the way out: the reply comes back to the WAN address.
|
||||
msg = []byte{2, 0, 0, 0}
|
||||
msg = append(msg, tupleAttr(ctaTupleOrig, "192.168.1.20", "198.51.100.9", 17, 5353, 53)...)
|
||||
msg = append(msg, tupleAttr(ctaTupleReply, "198.51.100.9", "203.0.113.10", 17, 53, 5353)...)
|
||||
if c, _ := parse(msg); c.SNATTo != "203.0.113.10" || c.DNATTo != "" || c.Protocol != "udp" {
|
||||
t.Errorf("masqueraded: %+v", c)
|
||||
}
|
||||
}
|
||||
+74
-10
@@ -74,6 +74,9 @@ type Result struct {
|
||||
HistoryID int64 `json:"history_id,omitempty"`
|
||||
Changes []model.Change `json:"changes"`
|
||||
Warnings []validate.Problem `json:"warnings,omitempty"`
|
||||
// Before and After are the configurations the change goes from and
|
||||
// to (After also for a dry run), for diffs and answers.
|
||||
Before, After *model.State `json:"-"`
|
||||
}
|
||||
|
||||
// Engine holds the current configuration.
|
||||
@@ -134,6 +137,9 @@ type Edit func(st *model.State, newID func() int64) error
|
||||
type Options struct {
|
||||
DryRun bool
|
||||
Via string // web, cli, api, import
|
||||
// RevertOf is the history entry the change undoes (a changeset with a
|
||||
// staged undo), so History links the two.
|
||||
RevertOf int64
|
||||
}
|
||||
|
||||
// Apply makes a change: edit runs on a copy of the configuration, the
|
||||
@@ -158,7 +164,7 @@ func (e *Engine) Apply(ctx context.Context, p *auth.Principal, description strin
|
||||
if idErr != nil {
|
||||
return nil, idErr
|
||||
}
|
||||
return e.commit(ctx, p, description, opts, next, 0)
|
||||
return e.commit(ctx, p, description, opts, next, opts.RevertOf)
|
||||
}
|
||||
|
||||
// commit validates next and saves it as the new state.
|
||||
@@ -166,7 +172,7 @@ func (e *Engine) commit(ctx context.Context, p *auth.Principal, description stri
|
||||
Normalize(next)
|
||||
next.Sort()
|
||||
changes := Diff(e.state, next)
|
||||
res := &Result{DryRun: opts.DryRun, Changes: changes}
|
||||
res := &Result{DryRun: opts.DryRun, Changes: changes, Before: e.state.Clone(), After: next.Clone()}
|
||||
if len(changes) == 0 {
|
||||
return res, nil
|
||||
}
|
||||
@@ -185,6 +191,7 @@ func (e *Engine) commit(ctx context.Context, p *auth.Principal, description stri
|
||||
// Only report warnings the change brings; old ones are on the status
|
||||
// page.
|
||||
res.Warnings = newProblems(validate.Check(e.state, e.env()), problems)
|
||||
res.Warnings = append(res.Warnings, quotaRestarts(e.state, next)...)
|
||||
if e.Firewall != nil {
|
||||
if err := e.Firewall.Check(ctx, next); err != nil {
|
||||
return res, &Error{Kind: KindFirewall, Message: "nftables refused the ruleset: " + err.Error()}
|
||||
@@ -224,6 +231,23 @@ func (e *Engine) commit(ctx context.Context, p *auth.Principal, description stri
|
||||
return res, nil
|
||||
}
|
||||
|
||||
// quotaRestarts warns of the rules whose quota starts again because they
|
||||
// now count it in a way what they used can't be carried to.
|
||||
func quotaRestarts(before, after *model.State) []validate.Problem {
|
||||
was := map[int64]*model.Rule{}
|
||||
for _, r := range before.Rules {
|
||||
was[r.ID] = r
|
||||
}
|
||||
var out []validate.Problem
|
||||
for _, r := range after.Rules {
|
||||
if carry, why := model.QuotaCarry(was[r.ID], r); carry == model.QuotaRestarts {
|
||||
out = append(out, validate.Problem{Severity: validate.Warning, Code: "quota_restarts", Object: r.Label(),
|
||||
Message: "its quota starts again: " + why})
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func newProblems(before, after []validate.Problem) []validate.Problem {
|
||||
seen := map[string]bool{}
|
||||
for _, p := range before {
|
||||
@@ -238,12 +262,16 @@ func newProblems(before, after []validate.Problem) []validate.Problem {
|
||||
return out
|
||||
}
|
||||
|
||||
// ApplyCurrent puts the current configuration in force (at start).
|
||||
// ApplyCurrent puts the current configuration in force again: at start,
|
||||
// and when a flowtable device appears. It holds the engine's lock, so a
|
||||
// change can't be applied in between.
|
||||
func (e *Engine) ApplyCurrent(ctx context.Context) error {
|
||||
if e.Firewall == nil {
|
||||
return nil
|
||||
}
|
||||
return e.Firewall.Apply(ctx, e.State())
|
||||
e.mu.Lock()
|
||||
defer e.mu.Unlock()
|
||||
return e.Firewall.Apply(ctx, e.state.Clone())
|
||||
}
|
||||
|
||||
// Revert undoes a history entry, if the objects it changed are still as
|
||||
@@ -261,8 +289,22 @@ func (e *Engine) Revert(ctx context.Context, p *auth.Principal, id int64, opts O
|
||||
}
|
||||
e.mu.Lock()
|
||||
defer e.mu.Unlock()
|
||||
cur := objects(e.state)
|
||||
next := e.state.Clone()
|
||||
if err := Undo(next, h); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return e.commit(ctx, p, UndoDescription(h), opts, next, h.ID)
|
||||
}
|
||||
|
||||
// UndoDescription is an undo's description: "undo 12: create rule …".
|
||||
func UndoDescription(h *model.HistoryEntry) string {
|
||||
return fmt.Sprintf("undo %d: %s", h.ID, h.Description)
|
||||
}
|
||||
|
||||
// Undo applies a history entry backwards to st, if every object it
|
||||
// changed is still as it left it (a change made since would be lost).
|
||||
func Undo(st *model.State, h *model.HistoryEntry) error {
|
||||
cur := objects(st)
|
||||
for i := len(h.Changes) - 1; i >= 0; i-- {
|
||||
c := h.Changes[i]
|
||||
now := cur[key{c.Kind, c.ID}]
|
||||
@@ -271,13 +313,29 @@ func (e *Engine) Revert(ctx context.Context, p *auth.Principal, id int64, opts O
|
||||
if what == "" {
|
||||
what = fmt.Sprintf("%s %d", c.Kind, c.ID)
|
||||
}
|
||||
return nil, Errorf(KindConflict, "%s %s has changed since entry %d; undo the later changes first, or change it by hand", c.Kind, what, id)
|
||||
return Errorf(KindConflict, "%s %s has changed since entry %d; undo the later changes first, or change it by hand", c.Kind, what, h.ID)
|
||||
}
|
||||
if err := put(next, c.Kind, c.ID, c.Before); err != nil {
|
||||
return nil, err
|
||||
if err := put(st, c.Kind, c.ID, c.Before); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return e.commit(ctx, p, fmt.Sprintf("undo %d: %s", h.ID, h.Description), opts, next, h.ID)
|
||||
return nil
|
||||
}
|
||||
|
||||
// Rewind applies a history entry backwards to st without Undo's check:
|
||||
// stepping back through the entries newest first brings back the
|
||||
// configuration as it was before each.
|
||||
func Rewind(st *model.State, h *model.HistoryEntry) error {
|
||||
for i := len(h.Changes) - 1; i >= 0; i-- {
|
||||
c := h.Changes[i]
|
||||
if c.Kind == model.KindSettings && c.Before == nil {
|
||||
continue
|
||||
}
|
||||
if err := put(st, c.Kind, c.ID, c.Before); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Replace makes the configuration next as a whole (imports).
|
||||
@@ -370,7 +428,7 @@ func Diff(a, b *model.State) []model.Change {
|
||||
return out
|
||||
}
|
||||
|
||||
// Describe sums up changes: "create zone lan (enp2s0)", or
|
||||
// Describe sums up changes: "create zone lan (eth1)", or
|
||||
// "3 changes" for many.
|
||||
func Describe(cs []model.Change) string {
|
||||
if len(cs) == 0 {
|
||||
@@ -467,6 +525,12 @@ func Normalize(st *model.State) {
|
||||
normMatch(&x.Match)
|
||||
x.ToPort = normPorts(x.ToPort)
|
||||
}
|
||||
for _, b := range st.BanLists {
|
||||
b.Name = strings.TrimSpace(b.Name)
|
||||
b.Scope = strings.ToLower(strings.TrimSpace(b.Scope))
|
||||
b.Hosts = normAddrs(b.Hosts)
|
||||
b.Never = normAddrs(b.Never)
|
||||
}
|
||||
// Positions are fractions after insertions between rules; renumber
|
||||
// only when two have come too close to put another between.
|
||||
st.Sort()
|
||||
|
||||
@@ -173,3 +173,49 @@ func TestPositionsStayStableUntilCrowded(t *testing.T) {
|
||||
t.Errorf("order %v", order)
|
||||
}
|
||||
}
|
||||
|
||||
// TestQuotaRestartsWarns: a change that counts a quota so what it used
|
||||
// can't carry over warns; one that can (to a total, wider groups) doesn't.
|
||||
func TestQuotaRestartsWarns(t *testing.T) {
|
||||
e := newEngine(t)
|
||||
ctx := context.Background()
|
||||
var id int64
|
||||
if _, err := e.Apply(ctx, admin, "", Options{}, func(st *model.State, newID func() int64) error {
|
||||
id = newID()
|
||||
st.Rules = append(st.Rules, &model.Rule{ID: id, Chain: "forward", Action: "drop", Quota: &model.Quota{Bytes: 1 << 30, Over: true, PerSource: true}})
|
||||
return nil
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
change := func(f func(r *model.Rule)) []string {
|
||||
res, err := e.Apply(ctx, admin, "", Options{DryRun: true}, func(st *model.State, _ func() int64) error {
|
||||
f(st.ByID(model.KindRule, id).(*model.Rule))
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var codes []string
|
||||
for _, w := range res.Warnings {
|
||||
if w.Code == "quota_restarts" {
|
||||
codes = append(codes, w.Message)
|
||||
}
|
||||
}
|
||||
return codes
|
||||
}
|
||||
if w := change(func(r *model.Rule) { r.Quota.PerSource, r.Quota.PerDestination = false, true }); len(w) != 1 || !strings.Contains(w[0], "per source can't be counted per destination") {
|
||||
t.Errorf("per source to per destination: %v", w)
|
||||
}
|
||||
if w := change(func(r *model.Rule) { r.Quota.PerSource = false }); len(w) != 0 {
|
||||
t.Errorf("to a total (summed): %v", w)
|
||||
}
|
||||
if w := change(func(r *model.Rule) { r.GroupV6 = 56 }); len(w) != 0 {
|
||||
t.Errorf("wider groups: %v", w)
|
||||
}
|
||||
if w := change(func(r *model.Rule) { r.GroupV6 = 128 }); len(w) != 1 {
|
||||
t.Errorf("finer groups: %v", w)
|
||||
}
|
||||
if w := change(func(r *model.Rule) { r.Quota.Bytes = 2 << 30 }); len(w) != 0 {
|
||||
t.Errorf("another amount: %v", w)
|
||||
}
|
||||
}
|
||||
@@ -63,6 +63,14 @@ func (h *Host) Up(name string) bool {
|
||||
return h.up[name]
|
||||
}
|
||||
|
||||
// Forget drops what was read, so the next question reads the interfaces
|
||||
// again (after the kernel said one appeared or went).
|
||||
func (h *Host) Forget() {
|
||||
h.mu.Lock()
|
||||
h.at = time.Time{}
|
||||
h.mu.Unlock()
|
||||
}
|
||||
|
||||
// Exists returns the set of interface names (nil if they can't be read).
|
||||
func (h *Host) Exists() map[string]bool {
|
||||
h.mu.Lock()
|
||||
|
||||
@@ -0,0 +1,98 @@
|
||||
package importer
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"os/exec"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"housecarl/internal/render"
|
||||
"housecarl/internal/validate"
|
||||
)
|
||||
|
||||
// TestImportAudit: rulesets the importer must not change the meaning of
|
||||
// (testdata/audit.nft, as nft 1.1.3 lists it).
|
||||
func TestImportAudit(t *testing.T) {
|
||||
js, _ := os.ReadFile("testdata/audit.json")
|
||||
txt, _ := os.ReadFile("testdata/audit.txt")
|
||||
defs, _ := os.ReadFile("testdata/audit.nft")
|
||||
res, err := Import(Input{JSON: js, Text: string(txt), Defines: string(defs), Zones: map[string]string{"enp2s0.20": "iot", "eth9": "x"}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
st := res.State
|
||||
notes := strings.Join(res.Notes, "\n")
|
||||
text := render.Render(st, render.Options{}).Text
|
||||
has := func(what, s string) {
|
||||
t.Helper()
|
||||
if !strings.Contains(text, s) {
|
||||
t.Errorf("%s: no %q in\n%s", what, s, text)
|
||||
}
|
||||
}
|
||||
// Two chains on the input hook: their rules once each, in priority
|
||||
// order; established stays a rule after the first chain's rule.
|
||||
if n := strings.Count(text, "192.0.2.66"); n != 1 {
|
||||
t.Errorf("the early input chain's rule %d times", n)
|
||||
}
|
||||
if strings.Index(text, "192.0.2.66") > strings.Index(text, "ct state established,related counter accept comment \"hc:r") ||
|
||||
!strings.Contains(notes, "both hook input") {
|
||||
t.Errorf("chains on one hook:\n%s", text)
|
||||
}
|
||||
// A second match on a field, and != on the hop limit: raw, as written.
|
||||
has("two source matches", "ip saddr 10.0.0.0/8 ip saddr != 10.0.10.0/24 tcp dport 22 drop")
|
||||
has("hop limit", "ip6 hoplimit != 255 drop")
|
||||
// A jump: a rule within it is the rule's own; one outside it, or raw,
|
||||
// keeps the jump's match in front.
|
||||
has("within the jump", `ip saddr 10.9.0.5 tcp dport 22 counter accept comment "hc:r`)
|
||||
has("outside the jump", "ip saddr 10.9.0.0/24 ip saddr 10.8.0.1 tcp dport 23 accept")
|
||||
has("raw under the jump", "ip saddr 10.9.0.0/24 meta mark 0x00000001 accept")
|
||||
// Sets: a timeout set is dynamic; an interface set is written out in
|
||||
// the raw rule that names it.
|
||||
if s := st.Set("blocklist"); s == nil || !s.Dynamic || s.Timeout != 3600 {
|
||||
t.Errorf("blocklist: %+v", s)
|
||||
}
|
||||
has("dynamic set", "ip saddr @blocklist_v4 counter drop")
|
||||
has("interface set", `iifname { "enp2s0", "enp3s0" } accept`)
|
||||
// An accept to a forwarded host that isn't only for forwarded
|
||||
// connections stays a rule; a DNAT of two ports, two forwards.
|
||||
has("not only forwarded", `ip daddr 10.0.10.37 tcp dport 443 counter accept comment "hc:r`)
|
||||
if len(st.Forwards) != 2 || st.Forwards[0].Name != "web-80" || st.Forwards[1].Name != "web-443" {
|
||||
t.Errorf("forwards %+v", st.Forwards)
|
||||
}
|
||||
// Zones: as given (a VLAN), from defines, and why.
|
||||
if z := st.Zone("iot"); z == nil || z.Interfaces[0] != "enp2s0.20" {
|
||||
t.Errorf("zone iot: %+v", st.Zones)
|
||||
}
|
||||
if st.Zone("lan") == nil || st.Zone("wan") == nil {
|
||||
t.Errorf("zones %+v", st.Zones)
|
||||
}
|
||||
for _, want := range []string{"iot (enp2s0.20: as given)", "lan (enp2s0: the define lan_if)", "--zone eth9=x: no rule names"} {
|
||||
if !strings.Contains(notes, want) {
|
||||
t.Errorf("no note %q in\n%s", want, notes)
|
||||
}
|
||||
}
|
||||
if ps := validate.Check(st, nil); validate.HasErrors(ps) {
|
||||
t.Errorf("problems: %+v", validate.Errors(ps))
|
||||
}
|
||||
// And nftables takes it.
|
||||
cmd := strings.Fields(os.Getenv("HOUSECARL_NFT"))
|
||||
if len(cmd) == 0 {
|
||||
return
|
||||
}
|
||||
c := exec.CommandContext(context.Background(), cmd[0], append(cmd[1:], "-c", "-f", "/dev/stdin")...)
|
||||
c.Stdin = strings.NewReader(text)
|
||||
if out, err := c.CombinedOutput(); err != nil {
|
||||
t.Errorf("nft -c: %v\n%s\n%s", err, out, text)
|
||||
}
|
||||
}
|
||||
|
||||
// TestValidIface: interface names a define may hold, VLANs too.
|
||||
func TestValidIface(t *testing.T) {
|
||||
for s, want := range map[string]bool{"enp2s0": true, "wan": true, "enp2s0.20": true, "br0.4094": true, "veth*": true,
|
||||
"10.0.10.5": false, "example.com": false, "a.b.c": false, "enp2s0.20.30": false, "averyveryverylongname": false} {
|
||||
if validIface(s) != want {
|
||||
t.Errorf("validIface(%q) = %v", s, !want)
|
||||
}
|
||||
}
|
||||
}
|
||||
+946
-162
File diff suppressed because it is too large.
Load diff
@@ -1,6 +1,7 @@
|
||||
package importer
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"os/exec"
|
||||
"strings"
|
||||
@@ -49,11 +50,24 @@ func TestImportRouter(t *testing.T) {
|
||||
if s := st.Set("masq_to_internet"); s == nil {
|
||||
t.Errorf("no set masq_to_internet (sets %v)", setNames(st))
|
||||
}
|
||||
// Built-ins: the ones at the head of their chains.
|
||||
// Built-ins: the ones at the head of their chains, up to the invalid
|
||||
// drop, which stays a rule (below): the ICMP, ICMPv6 and DHCPv6
|
||||
// accepts after it stay rules too, in their order, so nothing the
|
||||
// drop stopped gets through ahead of it.
|
||||
b := st.Settings.Builtins
|
||||
if !b.Loopback || !b.Established || !b.ICMPv6 || !b.PingInternal || !b.DHCPv6Client || !b.MSSClamp || !b.Antispoof {
|
||||
if !b.Loopback || !b.Established || b.ICMPv6 || b.PingInternal || b.DHCPv6Client || !b.MSSClamp || !b.Antispoof {
|
||||
t.Errorf("built-ins %+v", b)
|
||||
}
|
||||
var input []string
|
||||
for _, r := range st.Rules {
|
||||
if r.Chain == model.ChainInput {
|
||||
input = append(input, strings.Join(r.CtState, ",")+"|"+strings.Join(r.Services, ",")+"|"+r.Raw)
|
||||
}
|
||||
}
|
||||
if len(input) < 5 || input[0] != "invalid||" || !strings.HasPrefix(input[1], "|icmp/echo-reply,icmp/echo-request|") ||
|
||||
!strings.Contains(input[2], "icmpv6/packet-too-big") || !strings.Contains(input[3], "ip6 hoplimit 255") || !strings.Contains(input[4], "udp/546") {
|
||||
t.Errorf("input rules start %q", input)
|
||||
}
|
||||
// The forward chain's asymmetric-routing rule sits between the
|
||||
// established and invalid rules: invalid stays rules, in order.
|
||||
if b.DropInvalid {
|
||||
@@ -101,9 +115,16 @@ func TestImportRouter(t *testing.T) {
|
||||
if !wgDNS {
|
||||
t.Errorf("wg0_input not inlined")
|
||||
}
|
||||
// The jump's and the rule's sources both: the rule's are kept, noted.
|
||||
if !strings.Contains(strings.Join(res.Notes, "\n"), "the rule's own (10.50.50.3, 10.50.50.6) are kept") {
|
||||
t.Errorf("no note on the overlapping sources: %q", res.Notes)
|
||||
// The jump's and the rule's sources both: the rule's lie within the
|
||||
// jump's, so they say it all, without a note.
|
||||
var wgSSH bool
|
||||
for _, r := range st.Rules {
|
||||
if r.Chain == model.ChainInput && strings.Join(r.Services, ",") == "tcp/22" && strings.Join(r.Sources, ",") == "10.50.50.3,10.50.50.6" {
|
||||
wgSSH = true
|
||||
}
|
||||
}
|
||||
if !wgSSH || strings.Contains(strings.Join(res.Notes, "\n"), "jump") {
|
||||
t.Errorf("the WireGuard SSH rule: found %v, notes %q", wgSSH, res.Notes)
|
||||
}
|
||||
// The imported configuration is valid, and renders.
|
||||
if ps := validate.Check(st, nil); validate.HasErrors(ps) {
|
||||
@@ -173,3 +194,130 @@ func TestImportedRulesetPassesNft(t *testing.T) {
|
||||
t.Errorf("nft -c: %v\n%s", err, out)
|
||||
}
|
||||
}
|
||||
|
||||
// TestImportMaps: dnat maps become port forwards and verdict maps rules,
|
||||
// one per element (testdata/maps.nft); a map that can't be expanded is
|
||||
// left out with a note, never kept as a raw rule naming it.
|
||||
func TestImportMaps(t *testing.T) {
|
||||
js, _ := os.ReadFile("testdata/maps.json")
|
||||
txt, _ := os.ReadFile("testdata/maps.txt")
|
||||
res, err := Import(Input{JSON: js, Text: string(txt)})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var fwds []string
|
||||
for _, f := range res.State.Forwards {
|
||||
fwds = append(fwds, fmt.Sprintf("%s %s/%s->%s:%s", f.Name, strings.Join(f.Protocols, "+"), f.Ports, f.To, f.ToPort))
|
||||
}
|
||||
var rules []string
|
||||
for _, r := range res.State.Rules {
|
||||
rules = append(rules, fmt.Sprintf("%s in=%s from=%s %s %s%s", r.Chain, strings.Join(r.InZones, ","), strings.Join(r.Sources, ","),
|
||||
strings.Join(r.Services, ","), r.Action, r.Raw))
|
||||
}
|
||||
want := []string{
|
||||
"portforwards-80 tcp/80->192.168.1.50:8080",
|
||||
"portforwards-443 tcp/443->192.168.1.50:",
|
||||
"hosts-2222 tcp/2222->192.168.1.60:",
|
||||
" udp/5000->192.168.1.70:5001",
|
||||
" udp/6000->192.168.1.71:",
|
||||
}
|
||||
if got := strings.Join(fwds, "\n"); got != strings.Join(want, "\n") {
|
||||
t.Errorf("forwards:\n%s\nwant:\n%s", got, strings.Join(want, "\n"))
|
||||
}
|
||||
// In order; the jump in the verdict map is inlined.
|
||||
want = []string{
|
||||
"input in= from= accept",
|
||||
"input in=lan from= tcp/22 accept",
|
||||
"input in=lan from= tcp/23 drop",
|
||||
"input in=lan from=192.168.1.0/24 tcp/80 accept",
|
||||
"input in=wan from= udp/53 accept",
|
||||
"input in=wan from= udp/123 drop",
|
||||
"input in= from=10.0.0.0/8 accept",
|
||||
"input in= from=192.0.2.1 drop",
|
||||
}
|
||||
if got := strings.Join(rules, "\n"); got != strings.Join(want, "\n") {
|
||||
t.Errorf("rules:\n%s\nwant:\n%s", got, strings.Join(want, "\n"))
|
||||
}
|
||||
notes := strings.Join(res.Notes, "\n")
|
||||
for _, n := range []string{"map portforwards: its 2 elements became 2 port forwards", "map hosts: its element became a port forward",
|
||||
"uses the map @pairs, which couldn't be expanded"} {
|
||||
if !strings.Contains(notes, n) {
|
||||
t.Errorf("note %q missing:\n%s", n, notes)
|
||||
}
|
||||
}
|
||||
if len(res.Raw) > 0 {
|
||||
t.Errorf("raw rules: %v", res.Raw)
|
||||
}
|
||||
if ps := validate.Errors(validate.Check(res.State, nil)); len(ps) > 0 {
|
||||
t.Errorf("invalid: %+v", ps)
|
||||
}
|
||||
if cmd := strings.Fields(os.Getenv("HOUSECARL_NFT")); len(cmd) > 0 {
|
||||
c := exec.Command(cmd[0], append(cmd[1:], "-c", "-f", "/dev/stdin")...)
|
||||
c.Stdin = strings.NewReader(render.Render(res.State, render.Options{}).Text)
|
||||
if out, err := c.CombinedOutput(); err != nil {
|
||||
t.Errorf("nft -c: %v\n%s", err, out)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestImportMeters: meters (per-address limits, connection counts and
|
||||
// quotas), byte rates and quotas become rules' limits and quotas
|
||||
// (testdata/meters.nft); the meters' sets aren't imported as address sets.
|
||||
func TestImportMeters(t *testing.T) {
|
||||
js, _ := os.ReadFile("testdata/meters.json")
|
||||
txt, _ := os.ReadFile("testdata/meters.txt")
|
||||
res, err := Import(Input{JSON: js, Text: string(txt)})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var got []string
|
||||
for _, r := range res.State.Rules {
|
||||
s := r.Chain
|
||||
if l := r.Limit; l != nil {
|
||||
s += fmt.Sprintf(" limit %d %s/%s burst %d over=%v src=%v dst=%v", l.Rate, l.Unit, l.Per, l.Burst, l.Over, l.PerSource, l.PerDestination)
|
||||
}
|
||||
if r.ConnLimit > 0 {
|
||||
s += fmt.Sprintf(" conn %d", r.ConnLimit)
|
||||
}
|
||||
if q := r.Quota; q != nil {
|
||||
s += fmt.Sprintf(" quota %d over=%v src=%v", q.Bytes, q.Over, q.PerSource)
|
||||
}
|
||||
if r.GroupV4 != 0 || r.GroupV6 != 0 {
|
||||
s += fmt.Sprintf(" group %d/%d", r.GroupV4, r.GroupV6)
|
||||
}
|
||||
s += " " + r.Action + r.Raw
|
||||
got = append(got, s)
|
||||
}
|
||||
want := []string{
|
||||
"input accept", // established: only in input here, so a rule
|
||||
"input limit 3 /minute burst 5 over=true src=true dst=false drop",
|
||||
"input limit 3 /minute burst 5 over=true src=true dst=false group 0/64 drop",
|
||||
"input accept",
|
||||
"forward conn 20 drop",
|
||||
"forward limit 10240 kbytes/second burst 512 over=true src=false dst=false drop",
|
||||
"forward quota 1048576000 over=true src=true group 24/0 drop",
|
||||
"forward quota 104857600000 over=true src=false drop",
|
||||
"forward accept",
|
||||
}
|
||||
if strings.Join(got, "\n") != strings.Join(want, "\n") {
|
||||
t.Errorf("rules:\n%s\nwant:\n%s\nnotes: %v", strings.Join(got, "\n"), strings.Join(want, "\n"), res.Notes)
|
||||
}
|
||||
for _, s := range res.State.Sets {
|
||||
if s.Name != "trusted" {
|
||||
t.Errorf("a meter became a set: %s", s.Name)
|
||||
}
|
||||
}
|
||||
if len(res.Raw) > 0 {
|
||||
t.Errorf("raw rules: %v", res.Raw)
|
||||
}
|
||||
if ps := validate.Errors(validate.Check(res.State, nil)); len(ps) > 0 {
|
||||
t.Errorf("invalid: %+v", ps)
|
||||
}
|
||||
if cmd := strings.Fields(os.Getenv("HOUSECARL_NFT")); len(cmd) > 0 {
|
||||
c := exec.Command(cmd[0], append(cmd[1:], "-c", "-f", "/dev/stdin")...)
|
||||
c.Stdin = strings.NewReader(render.Render(res.State, render.Options{}).Text)
|
||||
if out, err := c.CombinedOutput(); err != nil {
|
||||
t.Errorf("nft -c: %v\n%s", err, out)
|
||||
}
|
||||
}
|
||||
}
|
||||
+1
@@ -0,0 +1 @@
|
||||
{"nftables": [{"metainfo": {"version": "1.1.3", "release_name": "Commodore Bullmoose #4", "json_schema_version": 1}}, {"table": {"family": "inet", "name": "audit", "handle": 1}}, {"chain": {"family": "inet", "table": "audit", "name": "early", "handle": 1, "type": "filter", "hook": "input", "prio": -10, "policy": "accept"}}, {"chain": {"family": "inet", "table": "audit", "name": "input", "handle": 2, "type": "filter", "hook": "input", "prio": 0, "policy": "drop"}}, {"chain": {"family": "inet", "table": "audit", "name": "admin", "handle": 3}}, {"chain": {"family": "inet", "table": "audit", "name": "forward", "handle": 4, "type": "filter", "hook": "forward", "prio": 0, "policy": "drop"}}, {"chain": {"family": "inet", "table": "audit", "name": "prerouting", "handle": 5, "type": "nat", "hook": "prerouting", "prio": -100, "policy": "accept"}}, {"set": {"family": "inet", "name": "blocklist", "table": "audit", "type": "ipv4_addr", "handle": 6, "flags": ["timeout"], "timeout": 3600}}, {"set": {"family": "inet", "name": "lans", "table": "audit", "type": "ifname", "handle": 7, "elem": ["enp2s0", "enp3s0"]}}, {"rule": {"family": "inet", "table": "audit", "chain": "early", "handle": 8, "comment": "an early input chain", "expr": [{"match": {"op": "==", "left": {"payload": {"protocol": "ip", "field": "saddr"}}, "right": "192.0.2.66"}}, {"drop": null}]}}, {"rule": {"family": "inet", "table": "audit", "chain": "input", "handle": 9, "expr": [{"match": {"op": "in", "left": {"ct": {"key": "state"}}, "right": ["established", "related"]}}, {"accept": null}]}}, {"rule": {"family": "inet", "table": "audit", "chain": "input", "handle": 10, "comment": "dynamic", "expr": [{"match": {"op": "==", "left": {"payload": {"protocol": "ip", "field": "saddr"}}, "right": "@blocklist"}}, {"drop": null}]}}, {"rule": {"family": "inet", "table": "audit", "chain": "input", "handle": 11, "comment": "an interface set", "expr": [{"match": {"op": "==", "left": {"meta": {"key": "iifname"}}, "right": "@lans"}}, {"accept": null}]}}, {"rule": {"family": "inet", "table": "audit", "chain": "input", "handle": 12, "comment": "two source matches", "expr": [{"match": {"op": "==", "left": {"payload": {"protocol": "ip", "field": "saddr"}}, "right": {"prefix": {"addr": "10.0.0.0", "len": 8}}}}, {"match": {"op": "!=", "left": {"payload": {"protocol": "ip", "field": "saddr"}}, "right": {"prefix": {"addr": "10.0.10.0", "len": 24}}}}, {"match": {"op": "==", "left": {"payload": {"protocol": "tcp", "field": "dport"}}, "right": 22}}, {"drop": null}]}}, {"rule": {"family": "inet", "table": "audit", "chain": "input", "handle": 13, "comment": "hop limit not 255", "expr": [{"match": {"op": "==", "left": {"payload": {"protocol": "icmpv6", "field": "type"}}, "right": "nd-neighbor-solicit"}}, {"match": {"op": "!=", "left": {"payload": {"protocol": "ip6", "field": "hoplimit"}}, "right": 255}}, {"drop": null}]}}, {"rule": {"family": "inet", "table": "audit", "chain": "input", "handle": 14, "comment": "DNS from the IoT VLAN", "expr": [{"match": {"op": "==", "left": {"meta": {"key": "iifname"}}, "right": "enp2s0.20"}}, {"match": {"op": "==", "left": {"payload": {"protocol": "udp", "field": "dport"}}, "right": 53}}, {"accept": null}]}}, {"rule": {"family": "inet", "table": "audit", "chain": "input", "handle": 15, "expr": [{"match": {"op": "==", "left": {"payload": {"protocol": "ip", "field": "saddr"}}, "right": {"prefix": {"addr": "10.9.0.0", "len": 24}}}}, {"jump": {"target": "admin"}}]}}, {"rule": {"family": "inet", "table": "audit", "chain": "admin", "handle": 16, "comment": "within the jump", "expr": [{"match": {"op": "==", "left": {"payload": {"protocol": "ip", "field": "saddr"}}, "right": "10.9.0.5"}}, {"match": {"op": "==", "left": {"payload": {"protocol": "tcp", "field": "dport"}}, "right": 22}}, {"accept": null}]}}, {"rule": {"family": "inet", "table": "audit", "chain": "admin", "handle": 17, "comment": "outside the jump", "expr": [{"match": {"op": "==", "left": {"payload": {"protocol": "ip", "field": "saddr"}}, "right": "10.8.0.1"}}, {"match": {"op": "==", "left": {"payload": {"protocol": "tcp", "field": "dport"}}, "right": 23}}, {"accept": null}]}}, {"rule": {"family": "inet", "table": "audit", "chain": "admin", "handle": 18, "comment": "raw under the jump", "expr": [{"match": {"op": "==", "left": {"meta": {"key": "mark"}}, "right": 1}}, {"accept": null}]}}, {"rule": {"family": "inet", "table": "audit", "chain": "forward", "handle": 19, "expr": [{"match": {"op": "in", "left": {"ct": {"key": "state"}}, "right": ["established", "related"]}}, {"accept": null}]}}, {"rule": {"family": "inet", "table": "audit", "chain": "forward", "handle": 20, "comment": "not only forwarded", "expr": [{"match": {"op": "==", "left": {"payload": {"protocol": "ip", "field": "daddr"}}, "right": "10.0.10.37"}}, {"match": {"op": "==", "left": {"payload": {"protocol": "tcp", "field": "dport"}}, "right": 443}}, {"accept": null}]}}, {"rule": {"family": "inet", "table": "audit", "chain": "forward", "hanLine truncated
|
||||
Vendored
+53
@@ -0,0 +1,53 @@
|
||||
#! /sbin/nft -f
|
||||
# Cases the importer must not get wrong (importer_test.go TestImportAudit).
|
||||
|
||||
define wan_if = "wan"
|
||||
define lan_if = "enp2s0"
|
||||
define iot_if = "enp2s0.20"
|
||||
define admins = { 10.9.0.0/24 }
|
||||
|
||||
table inet audit {
|
||||
set blocklist {
|
||||
type ipv4_addr
|
||||
flags timeout
|
||||
timeout 1h
|
||||
}
|
||||
set lans {
|
||||
type ifname
|
||||
elements = { "enp2s0", "enp3s0" }
|
||||
}
|
||||
|
||||
chain early {
|
||||
type filter hook input priority -10; policy accept;
|
||||
ip saddr 192.0.2.66 drop comment "an early input chain"
|
||||
}
|
||||
|
||||
chain input {
|
||||
type filter hook input priority filter; policy drop;
|
||||
ct state established,related accept
|
||||
ip saddr @blocklist drop comment "dynamic"
|
||||
iifname @lans accept comment "an interface set"
|
||||
ip saddr 10.0.0.0/8 ip saddr != 10.0.10.0/24 tcp dport 22 drop comment "two source matches"
|
||||
icmpv6 type nd-neighbor-solicit ip6 hoplimit != 255 drop comment "hop limit not 255"
|
||||
iifname $iot_if udp dport 53 accept comment "DNS from the IoT VLAN"
|
||||
ip saddr $admins jump admin
|
||||
}
|
||||
|
||||
chain admin {
|
||||
ip saddr 10.9.0.5 tcp dport 22 accept comment "within the jump"
|
||||
ip saddr 10.8.0.1 tcp dport 23 accept comment "outside the jump"
|
||||
meta mark 0x00000001 accept comment "raw under the jump"
|
||||
}
|
||||
|
||||
chain forward {
|
||||
type filter hook forward priority filter; policy drop;
|
||||
ct state established,related accept
|
||||
ip daddr 10.0.10.37 tcp dport 443 accept comment "not only forwarded"
|
||||
iifname $lan_if oifname $wan_if accept
|
||||
}
|
||||
|
||||
chain prerouting {
|
||||
type nat hook prerouting priority dstnat; policy accept;
|
||||
ip daddr 203.0.113.1 tcp dport { 80, 443 } dnat ip to 10.0.10.37 comment "web"
|
||||
}
|
||||
}
|
||||
Vendored
+46
@@ -0,0 +1,46 @@
|
||||
table inet audit { # handle 1
|
||||
set blocklist { # handle 6
|
||||
type ipv4_addr
|
||||
timeout 1h
|
||||
}
|
||||
|
||||
set lans { # handle 7
|
||||
type ifname
|
||||
elements = { "enp2s0",
|
||||
"enp3s0" }
|
||||
}
|
||||
|
||||
chain early { # handle 1
|
||||
type filter hook input priority filter - 10; policy accept;
|
||||
ip saddr 192.0.2.66 drop comment "an early input chain" # handle 8
|
||||
}
|
||||
|
||||
chain input { # handle 2
|
||||
type filter hook input priority filter; policy drop;
|
||||
ct state established,related accept # handle 9
|
||||
ip saddr @blocklist drop comment "dynamic" # handle 10
|
||||
iifname @lans accept comment "an interface set" # handle 11
|
||||
ip saddr 10.0.0.0/8 ip saddr != 10.0.10.0/24 tcp dport 22 drop comment "two source matches" # handle 12
|
||||
icmpv6 type nd-neighbor-solicit ip6 hoplimit != 255 drop comment "hop limit not 255" # handle 13
|
||||
iifname "enp2s0.20" udp dport 53 accept comment "DNS from the IoT VLAN" # handle 14
|
||||
ip saddr 10.9.0.0/24 jump admin # handle 15
|
||||
}
|
||||
|
||||
chain admin { # handle 3
|
||||
ip saddr 10.9.0.5 tcp dport 22 accept comment "within the jump" # handle 16
|
||||
ip saddr 10.8.0.1 tcp dport 23 accept comment "outside the jump" # handle 17
|
||||
meta mark 0x00000001 accept comment "raw under the jump" # handle 18
|
||||
}
|
||||
|
||||
chain forward { # handle 4
|
||||
type filter hook forward priority filter; policy drop;
|
||||
ct state established,related accept # handle 19
|
||||
ip daddr 10.0.10.37 tcp dport 443 accept comment "not only forwarded" # handle 20
|
||||
iifname "enp2s0" oifname "wan" accept # handle 21
|
||||
}
|
||||
|
||||
chain prerouting { # handle 5
|
||||
type nat hook prerouting priority dstnat; policy accept;
|
||||
ip daddr 203.0.113.1 tcp dport { 80, 443 } dnat ip to 10.0.10.37 comment "web" # handle 23
|
||||
}
|
||||
}
|
||||
Vendored
+1
@@ -0,0 +1 @@
|
||||
{"nftables": [{"metainfo": {"version": "1.1.3", "release_name": "Commodore Bullmoose #4", "json_schema_version": 1}}, {"table": {"family": "inet", "name": "main", "handle": 3}}, {"chain": {"family": "inet", "table": "main", "name": "web", "handle": 1}}, {"chain": {"family": "inet", "table": "main", "name": "prerouting", "handle": 2, "type": "nat", "hook": "prerouting", "prio": -100, "policy": "accept"}}, {"chain": {"family": "inet", "table": "main", "name": "input", "handle": 3, "type": "filter", "hook": "input", "prio": 0, "policy": "drop"}}, {"map": {"family": "inet", "name": "portforwards", "table": "main", "type": "inet_service", "handle": 4, "map": ["ipv4_addr", "inet_service"], "elem": [[80, {"concat": ["192.168.1.50", 8080]}], [443, {"concat": ["192.168.1.50", 443]}]]}}, {"map": {"family": "inet", "name": "hosts", "table": "main", "type": "inet_service", "handle": 5, "map": "ipv4_addr", "elem": [[2222, "192.168.1.60"]]}}, {"map": {"family": "inet", "name": "lan_services", "table": "main", "type": "inet_service", "handle": 6, "map": "verdict", "elem": [[22, {"accept": null}], [23, {"drop": null}], [80, {"jump": {"target": "web"}}]]}}, {"map": {"family": "inet", "name": "pairs", "table": "main", "type": ["ipv4_addr", "inet_service"], "handle": 7, "map": "verdict", "elem": [[{"concat": ["192.168.1.9", 22]}, {"accept": null}]]}}, {"rule": {"family": "inet", "table": "main", "chain": "web", "handle": 8, "expr": [{"match": {"op": "==", "left": {"payload": {"protocol": "ip", "field": "saddr"}}, "right": {"prefix": {"addr": "192.168.1.0", "len": 24}}}}, {"accept": null}]}}, {"rule": {"family": "inet", "table": "main", "chain": "prerouting", "handle": 9, "comment": "Port forwards", "expr": [{"match": {"op": "==", "left": {"meta": {"key": "iifname"}}, "right": "wan"}}, {"dnat": {"family": "ip", "addr": {"map": {"key": {"payload": {"protocol": "tcp", "field": "dport"}}, "data": "@portforwards"}}}}]}}, {"rule": {"family": "inet", "table": "main", "chain": "prerouting", "handle": 10, "expr": [{"match": {"op": "==", "left": {"meta": {"key": "iifname"}}, "right": "wan"}}, {"dnat": {"family": "ip", "addr": {"map": {"key": {"payload": {"protocol": "tcp", "field": "dport"}}, "data": "@hosts"}}}}]}}, {"rule": {"family": "inet", "table": "main", "chain": "prerouting", "handle": 12, "expr": [{"match": {"op": "==", "left": {"meta": {"key": "iifname"}}, "right": "wan"}}, {"dnat": {"family": "ip", "addr": {"map": {"key": {"payload": {"protocol": "udp", "field": "dport"}}, "data": {"set": [[5000, {"concat": ["192.168.1.70", 5001]}], [6000, {"concat": ["192.168.1.71", 6000]}]]}}}}}]}}, {"rule": {"family": "inet", "table": "main", "chain": "input", "handle": 13, "expr": [{"match": {"op": "in", "left": {"ct": {"key": "state"}}, "right": ["established", "related"]}}, {"accept": null}]}}, {"rule": {"family": "inet", "table": "main", "chain": "input", "handle": 14, "expr": [{"match": {"op": "==", "left": {"meta": {"key": "iifname"}}, "right": "lan"}}, {"vmap": {"key": {"payload": {"protocol": "tcp", "field": "dport"}}, "data": "@lan_services"}}]}}, {"rule": {"family": "inet", "table": "main", "chain": "input", "handle": 16, "expr": [{"match": {"op": "==", "left": {"meta": {"key": "iifname"}}, "right": "wan"}}, {"vmap": {"key": {"payload": {"protocol": "udp", "field": "dport"}}, "data": {"set": [[53, {"accept": null}], [123, {"drop": null}]]}}}]}}, {"rule": {"family": "inet", "table": "main", "chain": "input", "handle": 18, "expr": [{"vmap": {"key": {"payload": {"protocol": "ip", "field": "saddr"}}, "data": {"set": [[{"prefix": {"addr": "10.0.0.0", "len": 8}}, {"accept": null}], ["192.0.2.1", {"drop": null}]]}}}]}}, {"rule": {"family": "inet", "table": "main", "chain": "input", "handle": 19, "expr": [{"vmap": {"key": {"concat": [{"payload": {"protocol": "ip", "field": "saddr"}}, {"payload": {"protocol": "tcp", "field": "dport"}}]}, "data": "@pairs"}}]}}]}
|
||||
Vendored
+36
@@ -0,0 +1,36 @@
|
||||
# Maps and verdict maps, as found in hand-written rulesets (importer_test.go).
|
||||
table inet main {
|
||||
map portforwards {
|
||||
type inet_service : ipv4_addr . inet_service
|
||||
elements = { 443 : 192.168.1.50 . 443, 80 : 192.168.1.50 . 8080 }
|
||||
}
|
||||
map hosts {
|
||||
type inet_service : ipv4_addr
|
||||
elements = { 2222 : 192.168.1.60 }
|
||||
}
|
||||
map lan_services {
|
||||
type inet_service : verdict
|
||||
elements = { 22 : accept, 23 : drop, 80 : jump web }
|
||||
}
|
||||
map pairs {
|
||||
type ipv4_addr . inet_service : verdict
|
||||
elements = { 192.168.1.9 . 22 : accept }
|
||||
}
|
||||
chain web {
|
||||
ip saddr 192.168.1.0/24 accept
|
||||
}
|
||||
chain prerouting {
|
||||
type nat hook prerouting priority dstnat; policy accept;
|
||||
iifname "wan" dnat ip to tcp dport map @portforwards comment "Port forwards"
|
||||
iifname "wan" dnat ip to tcp dport map @hosts
|
||||
iifname "wan" dnat ip to udp dport map { 5000 : 192.168.1.70 . 5001, 6000 : 192.168.1.71 . 6000 }
|
||||
}
|
||||
chain input {
|
||||
type filter hook input priority filter; policy drop;
|
||||
ct state established,related accept
|
||||
iifname "lan" tcp dport vmap @lan_services
|
||||
iifname "wan" udp dport vmap { 53 : accept, 123 : drop }
|
||||
ip saddr vmap { 10.0.0.0/8 : accept, 192.0.2.1 : drop }
|
||||
ip saddr . tcp dport vmap @pairs
|
||||
}
|
||||
}
|
||||
Vendored
+44
@@ -0,0 +1,44 @@
|
||||
table inet main { # handle 3
|
||||
map portforwards { # handle 4
|
||||
type inet_service : ipv4_addr . inet_service
|
||||
elements = { 80 : 192.168.1.50 . 8080,
|
||||
443 : 192.168.1.50 . 443 }
|
||||
}
|
||||
|
||||
map hosts { # handle 5
|
||||
type inet_service : ipv4_addr
|
||||
elements = { 2222 : 192.168.1.60 }
|
||||
}
|
||||
|
||||
map lan_services { # handle 6
|
||||
type inet_service : verdict
|
||||
elements = { 22 : accept,
|
||||
23 : drop,
|
||||
80 : jump web }
|
||||
}
|
||||
|
||||
map pairs { # handle 7
|
||||
type ipv4_addr . inet_service : verdict
|
||||
elements = { 192.168.1.9 . 22 : accept }
|
||||
}
|
||||
|
||||
chain web { # handle 1
|
||||
ip saddr 192.168.1.0/24 accept # handle 8
|
||||
}
|
||||
|
||||
chain prerouting { # handle 2
|
||||
type nat hook prerouting priority dstnat; policy accept;
|
||||
iifname "wan" dnat ip to tcp dport map @portforwards comment "Port forwards" # handle 9
|
||||
iifname "wan" dnat ip to tcp dport map @hosts # handle 10
|
||||
iifname "wan" dnat ip to udp dport map { 5000 : 192.168.1.70 . 5001, 6000 : 192.168.1.71 . 6000 } # handle 12
|
||||
}
|
||||
|
||||
chain input { # handle 3
|
||||
type filter hook input priority filter; policy drop;
|
||||
ct state established,related accept # handle 13
|
||||
iifname "lan" tcp dport vmap @lan_services # handle 14
|
||||
iifname "wan" udp dport vmap { 53 : accept, 123 : drop } # handle 16
|
||||
ip saddr vmap { 10.0.0.0/8 : accept, 192.0.2.1 : drop } # handle 18
|
||||
ip saddr . tcp dport vmap @pairs # handle 19
|
||||
}
|
||||
}
|
||||
+1
@@ -0,0 +1 @@
|
||||
{"nftables": [{"metainfo": {"version": "1.1.3", "release_name": "Commodore Bullmoose #4", "json_schema_version": 1}}, {"table": {"family": "inet", "name": "main", "handle": 1}}, {"chain": {"family": "inet", "table": "main", "name": "input", "handle": 1, "type": "filter", "hook": "input", "prio": 0, "policy": "drop"}}, {"chain": {"family": "inet", "table": "main", "name": "forward", "handle": 2, "type": "filter", "hook": "forward", "prio": 0, "policy": "drop"}}, {"chain": {"family": "inet", "table": "main", "name": "output", "handle": 3, "type": "filter", "hook": "output", "prio": 0, "policy": "accept"}}, {"set": {"family": "inet", "name": "ssh_meter", "table": "main", "type": "ipv4_addr", "handle": 4, "size": 65535, "flags": ["timeout", "dynamic"], "timeout": 60}}, {"set": {"family": "inet", "name": "ssh6_meter", "table": "main", "type": "ipv6_addr", "handle": 5, "size": 65535, "flags": ["timeout", "dynamic"], "timeout": 60}}, {"set": {"family": "inet", "name": "my_connlimit", "table": "main", "type": "ipv4_addr", "handle": 6, "size": 65535, "flags": ["dynamic"]}}, {"set": {"family": "inet", "name": "guest_quota", "table": "main", "type": "ipv4_addr", "handle": 7, "size": 65535, "flags": ["dynamic"]}}, {"set": {"family": "inet", "name": "trusted", "table": "main", "type": "ipv4_addr", "handle": 8, "flags": ["interval"], "elem": [{"prefix": {"addr": "10.0.10.0", "len": 24}}]}}, {"rule": {"family": "inet", "table": "main", "chain": "input", "handle": 9, "expr": [{"match": {"op": "in", "left": {"ct": {"key": "state"}}, "right": ["established", "related"]}}, {"accept": null}]}}, {"rule": {"family": "inet", "table": "main", "chain": "input", "handle": 10, "expr": [{"match": {"op": "==", "left": {"payload": {"protocol": "tcp", "field": "dport"}}, "right": 22}}, {"match": {"op": "in", "left": {"ct": {"key": "state"}}, "right": "new"}}, {"set": {"op": "update", "elem": {"payload": {"protocol": "ip", "field": "saddr"}}, "set": "@ssh_meter", "stmt": [{"limit": {"rate": 3, "burst": 5, "per": "minute", "inv": true}}]}}, {"counter": {"packets": 0, "bytes": 0}}, {"drop": null}]}}, {"rule": {"family": "inet", "table": "main", "chain": "input", "handle": 11, "expr": [{"match": {"op": "==", "left": {"payload": {"protocol": "tcp", "field": "dport"}}, "right": 22}}, {"match": {"op": "in", "left": {"ct": {"key": "state"}}, "right": "new"}}, {"set": {"op": "update", "elem": {"&": [{"payload": {"protocol": "ip6", "field": "saddr"}}, "ffff:ffff:ffff:ffff::"]}, "set": "@ssh6_meter", "stmt": [{"limit": {"rate": 3, "burst": 5, "per": "minute", "inv": true}}]}}, {"counter": {"packets": 0, "bytes": 0}}, {"drop": null}]}}, {"rule": {"family": "inet", "table": "main", "chain": "input", "handle": 12, "expr": [{"match": {"op": "==", "left": {"payload": {"protocol": "ip", "field": "saddr"}}, "right": "@trusted"}}, {"match": {"op": "==", "left": {"payload": {"protocol": "tcp", "field": "dport"}}, "right": 22}}, {"accept": null}]}}, {"rule": {"family": "inet", "table": "main", "chain": "forward", "handle": 13, "expr": [{"match": {"op": "in", "left": {"ct": {"key": "state"}}, "right": "new"}}, {"set": {"op": "add", "elem": {"payload": {"protocol": "ip", "field": "saddr"}}, "set": "@my_connlimit", "stmt": [{"ct count": {"val": 20, "inv": true}}]}}, {"counter": {"packets": 0, "bytes": 0}}, {"drop": null}]}}, {"rule": {"family": "inet", "table": "main", "chain": "forward", "handle": 14, "expr": [{"match": {"op": "==", "left": {"meta": {"key": "iifname"}}, "right": "lan"}}, {"match": {"op": "==", "left": {"meta": {"key": "oifname"}}, "right": "wan"}}, {"limit": {"rate": 10, "burst": 512, "per": "second", "inv": true, "rate_unit": "mbytes", "burst_unit": "kbytes"}}, {"drop": null}]}}, {"rule": {"family": "inet", "table": "main", "chain": "forward", "handle": 15, "expr": [{"match": {"op": "==", "left": {"meta": {"key": "iifname"}}, "right": "guest0"}}, {"match": {"op": "==", "left": {"meta": {"key": "oifname"}}, "right": "wan"}}, {"set": {"op": "add", "elem": {"&": [{"payload": {"protocol": "ip", "field": "saddr"}}, "255.255.255.0"]}, "set": "@guest_quota", "stmt": [{"quota": {"val": 1000, "val_unit": "mbytes", "inv": true}}]}}, {"drop": null}]}}, {"rule": {"family": "inet", "table": "main", "chain": "forward", "handle": 16, "expr": [{"match": {"op": "==", "left": {"meta": {"key": "iifname"}}, "right": "lan"}}, {"match": {"op": "==", "left": {"meta": {"key": "oifname"}}, "right": "wan"}}, {"quota": {"val": 100000, "val_unit": "mbytes", "inv": true}}, {"drop": null}]}}, {"rule": {"family": "inet", "table": "main", "chain": "forward", "handle": 17, "expr": [{"match": {"op": "==", "left": {"meta": {"key": "iifname"}}, "right": "lan"}}, {"match": {"op": "==", "left": {"meta": {"key": "oifname"}}, "right": "wan"}}, {"accept": null}]}}]}
|
||||
+46
@@ -0,0 +1,46 @@
|
||||
# Meters, byte rates and quotas, as hand-written rulesets use them (importer_test.go).
|
||||
table inet main {
|
||||
set ssh_meter {
|
||||
type ipv4_addr
|
||||
flags dynamic,timeout
|
||||
timeout 1m
|
||||
}
|
||||
set ssh6_meter {
|
||||
type ipv6_addr
|
||||
flags dynamic,timeout
|
||||
timeout 1m
|
||||
}
|
||||
set my_connlimit {
|
||||
type ipv4_addr
|
||||
size 65535
|
||||
flags dynamic
|
||||
}
|
||||
set guest_quota {
|
||||
type ipv4_addr
|
||||
size 65535
|
||||
flags dynamic
|
||||
}
|
||||
set trusted {
|
||||
type ipv4_addr
|
||||
flags interval
|
||||
elements = { 10.0.10.0/24 }
|
||||
}
|
||||
chain input {
|
||||
type filter hook input priority filter; policy drop;
|
||||
ct state established,related accept
|
||||
tcp dport 22 ct state new update @ssh_meter { ip saddr limit rate over 3/minute burst 5 packets } counter drop
|
||||
tcp dport 22 ct state new update @ssh6_meter { ip6 saddr and ffff:ffff:ffff:ffff:: limit rate over 3/minute } counter drop
|
||||
ip saddr @trusted tcp dport 22 accept
|
||||
}
|
||||
chain forward {
|
||||
type filter hook forward priority filter; policy drop;
|
||||
ct state new add @my_connlimit { ip saddr ct count over 20 } counter drop
|
||||
iifname "lan" oifname "wan" limit rate over 10 mbytes/second burst 512 kbytes drop
|
||||
iifname "guest0" oifname "wan" add @guest_quota { ip saddr and 255.255.255.0 quota over 1000 mbytes } drop
|
||||
iifname "lan" oifname "wan" quota over 100000 mbytes drop
|
||||
iifname "lan" oifname "wan" accept
|
||||
}
|
||||
chain output {
|
||||
type filter hook output priority filter; policy accept;
|
||||
}
|
||||
}
|
||||
+54
@@ -0,0 +1,54 @@
|
||||
table inet main { # handle 1
|
||||
set ssh_meter { # handle 4
|
||||
type ipv4_addr
|
||||
size 65535
|
||||
flags dynamic,timeout
|
||||
timeout 1m
|
||||
}
|
||||
|
||||
set ssh6_meter { # handle 5
|
||||
type ipv6_addr
|
||||
size 65535
|
||||
flags dynamic,timeout
|
||||
timeout 1m
|
||||
}
|
||||
|
||||
set my_connlimit { # handle 6
|
||||
type ipv4_addr
|
||||
size 65535
|
||||
flags dynamic
|
||||
}
|
||||
|
||||
set guest_quota { # handle 7
|
||||
type ipv4_addr
|
||||
size 65535
|
||||
flags dynamic
|
||||
}
|
||||
|
||||
set trusted { # handle 8
|
||||
type ipv4_addr
|
||||
flags interval
|
||||
elements = { 10.0.10.0/24 }
|
||||
}
|
||||
|
||||
chain input { # handle 1
|
||||
type filter hook input priority filter; policy drop;
|
||||
ct state established,related accept # handle 9
|
||||
tcp dport 22 ct state new update @ssh_meter { ip saddr limit rate over 3/minute burst 5 packets } counter packets 0 bytes 0 drop # handle 10
|
||||
tcp dport 22 ct state new update @ssh6_meter { ip6 saddr & ffff:ffff:ffff:ffff:: limit rate over 3/minute burst 5 packets } counter packets 0 bytes 0 drop # handle 11
|
||||
ip saddr @trusted tcp dport 22 accept # handle 12
|
||||
}
|
||||
|
||||
chain forward { # handle 2
|
||||
type filter hook forward priority filter; policy drop;
|
||||
ct state new add @my_connlimit { ip saddr ct count over 20 } counter packets 0 bytes 0 drop # handle 13
|
||||
iifname "lan" oifname "wan" limit rate over 10 mbytes/second burst 512 kbytes drop # handle 14
|
||||
iifname "guest0" oifname "wan" add @guest_quota { ip saddr & 255.255.255.0 quota over 1000 mbytes } drop # handle 15
|
||||
iifname "lan" oifname "wan" quota over 100000 mbytes drop # handle 16
|
||||
iifname "lan" oifname "wan" accept # handle 17
|
||||
}
|
||||
|
||||
chain output { # handle 3
|
||||
type filter hook output priority filter; policy accept;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,96 @@
|
||||
// Package linkwatch tells when the machine's network interfaces change:
|
||||
// one appears, goes, or comes back (ppp0 when PPPoE reconnects). It
|
||||
// listens to the kernel's rtnetlink link messages and calls back once
|
||||
// they have been quiet for a moment, since one change brings several.
|
||||
package linkwatch
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/binary"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"time"
|
||||
|
||||
"golang.org/x/sys/unix"
|
||||
)
|
||||
|
||||
// Settle is how long the link messages must be quiet before the callback.
|
||||
const Settle = 2 * time.Second
|
||||
|
||||
// Watch calls changed after interfaces changed, until ctx ends. It returns
|
||||
// an error only if it can't listen at all.
|
||||
func Watch(ctx context.Context, changed func()) error {
|
||||
fd, err := unix.Socket(unix.AF_NETLINK, unix.SOCK_RAW|unix.SOCK_CLOEXEC|unix.SOCK_NONBLOCK, unix.NETLINK_ROUTE)
|
||||
if err != nil {
|
||||
return fmt.Errorf("opening a netlink socket: %w", err)
|
||||
}
|
||||
if err := unix.Bind(fd, &unix.SockaddrNetlink{Family: unix.AF_NETLINK, Groups: unix.RTMGRP_LINK}); err != nil {
|
||||
unix.Close(fd)
|
||||
return fmt.Errorf("listening to link changes: %w", err)
|
||||
}
|
||||
// A non-blocking descriptor goes through Go's poller, so closing the
|
||||
// file ends a Read in progress.
|
||||
f := os.NewFile(uintptr(fd), "rtnetlink")
|
||||
go func() {
|
||||
<-ctx.Done()
|
||||
f.Close()
|
||||
}()
|
||||
events := make(chan struct{}, 1)
|
||||
go func() {
|
||||
buf := make([]byte, 64<<10)
|
||||
for {
|
||||
n, err := f.Read(buf)
|
||||
switch {
|
||||
case err != nil && ctx.Err() != nil:
|
||||
close(events)
|
||||
return
|
||||
case errors.Is(err, unix.ENOBUFS):
|
||||
// Messages were lost: something changed anyway.
|
||||
case err != nil:
|
||||
time.Sleep(time.Second)
|
||||
continue
|
||||
case !links(buf[:n]):
|
||||
continue
|
||||
}
|
||||
select {
|
||||
case events <- struct{}{}:
|
||||
default:
|
||||
}
|
||||
}
|
||||
}()
|
||||
var timer <-chan time.Time
|
||||
for {
|
||||
select {
|
||||
case _, ok := <-events:
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
timer = time.After(Settle)
|
||||
case <-timer:
|
||||
timer = nil
|
||||
changed()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// links reports whether a netlink datagram holds a link being added,
|
||||
// changed or deleted.
|
||||
func links(b []byte) bool {
|
||||
for len(b) >= unix.NLMSG_HDRLEN {
|
||||
size := int(binary.NativeEndian.Uint32(b[0:4]))
|
||||
typ := binary.NativeEndian.Uint16(b[4:6])
|
||||
if size < unix.NLMSG_HDRLEN || size > len(b) {
|
||||
return false
|
||||
}
|
||||
if typ == unix.RTM_NEWLINK || typ == unix.RTM_DELLINK {
|
||||
return true
|
||||
}
|
||||
size = (size + unix.NLMSG_ALIGNTO - 1) &^ (unix.NLMSG_ALIGNTO - 1)
|
||||
if size >= len(b) {
|
||||
return false
|
||||
}
|
||||
b = b[size:]
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
package linkwatch
|
||||
|
||||
import (
|
||||
"encoding/binary"
|
||||
"testing"
|
||||
|
||||
"golang.org/x/sys/unix"
|
||||
)
|
||||
|
||||
// msg is a netlink message of a type, with a payload of n bytes.
|
||||
func msg(typ uint16, n int) []byte {
|
||||
b := make([]byte, unix.NLMSG_HDRLEN+n)
|
||||
binary.NativeEndian.PutUint32(b[0:4], uint32(len(b)))
|
||||
binary.NativeEndian.PutUint16(b[4:6], typ)
|
||||
return b
|
||||
}
|
||||
|
||||
func TestLinks(t *testing.T) {
|
||||
for _, c := range []struct {
|
||||
name string
|
||||
b []byte
|
||||
want bool
|
||||
}{
|
||||
{"a new link", msg(unix.RTM_NEWLINK, 16), true},
|
||||
{"a link going", msg(unix.RTM_DELLINK, 16), true},
|
||||
{"an address", msg(unix.RTM_NEWADDR, 16), false},
|
||||
{"a link after an address (padded)", append(msg(unix.RTM_NEWADDR, 13), append(make([]byte, 3), msg(unix.RTM_NEWLINK, 8)...)...), true},
|
||||
{"truncated", msg(unix.RTM_NEWLINK, 16)[:10], false},
|
||||
{"a length beyond the datagram", msg(unix.RTM_NEWLINK, 16)[:unix.NLMSG_HDRLEN+4], false},
|
||||
{"empty", nil, false},
|
||||
} {
|
||||
if got := links(c.b); got != c.want {
|
||||
t.Errorf("%s: %v", c.name, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,546 @@
|
||||
// Package matrix works out what the rules do to new connections between
|
||||
// zones, for an overview: for each pair it walks the rules as nftables
|
||||
// would, in order. A rule that applies to the whole pair decides it; one
|
||||
// that applies to some of its traffic (some addresses, services, times,
|
||||
// rates…) is an exception, and the walk goes on; what is left falls to
|
||||
// the chain's policy.
|
||||
package matrix
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"net/netip"
|
||||
"regexp"
|
||||
"strings"
|
||||
|
||||
"housecarl/internal/api"
|
||||
"housecarl/internal/model"
|
||||
)
|
||||
|
||||
// Firewall stands for the machine itself in the matrix.
|
||||
const Firewall = "firewall"
|
||||
|
||||
// Options tell Build about the machine.
|
||||
type Options struct {
|
||||
// Connected maps the networks directly connected to the firewall's
|
||||
// interfaces to their zones. A rule whose addresses all lie in other
|
||||
// zones' networks can't apply to a pair, and is left out of it.
|
||||
Connected map[netip.Prefix]string
|
||||
}
|
||||
|
||||
// Build works out the matrix of a configuration.
|
||||
func Build(st *model.State, opts ...Options) api.Matrix {
|
||||
var o Options
|
||||
if len(opts) > 0 {
|
||||
o = opts[0]
|
||||
}
|
||||
m := api.Matrix{Zones: []string{}, Cells: []api.MatrixCell{}}
|
||||
for _, z := range st.Zones {
|
||||
m.Zones = append(m.Zones, z.Name)
|
||||
}
|
||||
for _, from := range m.Zones {
|
||||
for _, to := range m.Zones {
|
||||
m.Cells = append(m.Cells, cell(st, o, model.ChainForward, from, to))
|
||||
}
|
||||
m.Cells = append(m.Cells, cell(st, o, model.ChainInput, from, Firewall))
|
||||
}
|
||||
for _, to := range m.Zones {
|
||||
m.Cells = append(m.Cells, cell(st, o, model.ChainOutput, Firewall, to))
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
// Cell returns one pair (from or to may be Firewall).
|
||||
func Cell(st *model.State, o Options, from, to string) (api.MatrixCell, bool) {
|
||||
known := func(z string) bool { return st.Zone(z) != nil }
|
||||
switch {
|
||||
case from == Firewall && known(to):
|
||||
return cell(st, o, model.ChainOutput, from, to), true
|
||||
case to == Firewall && known(from):
|
||||
return cell(st, o, model.ChainInput, from, to), true
|
||||
case known(from) && known(to):
|
||||
return cell(st, o, model.ChainForward, from, to), true
|
||||
}
|
||||
return api.MatrixCell{}, false
|
||||
}
|
||||
|
||||
func cell(st *model.State, o Options, chain, from, to string) api.MatrixCell {
|
||||
c := walkCell(st, o, chain, from, to)
|
||||
sum(&c)
|
||||
if from == to && chain == model.ChainForward {
|
||||
c.SameZone, c.Note = true, sameZoneNote(st.Zone(from))
|
||||
}
|
||||
return c
|
||||
}
|
||||
|
||||
// sum fills in the overview: the summary, the exceptions counted by
|
||||
// verdict, and what the allowing ones let through.
|
||||
func sum(c *api.MatrixCell) {
|
||||
c.Open = []string{}
|
||||
seen := map[string]bool{}
|
||||
for _, e := range c.Exceptions {
|
||||
switch e.Verdict {
|
||||
case "allowed":
|
||||
c.Allowed++
|
||||
if !seen[e.Short] {
|
||||
seen[e.Short] = true
|
||||
c.Open = append(c.Open, e.Short)
|
||||
}
|
||||
case "blocked":
|
||||
c.Blocked++
|
||||
}
|
||||
}
|
||||
c.Summary = c.Verdict
|
||||
if c.Verdict == "blocked" && c.Allowed > 0 || c.Verdict == "allowed" && c.Blocked > 0 {
|
||||
c.Summary = "partly"
|
||||
}
|
||||
}
|
||||
|
||||
// sameZoneNote says what of a zone's traffic to itself the firewall sees:
|
||||
// only what it routes between the zone's interfaces.
|
||||
func sameZoneNote(z *model.Zone) string {
|
||||
if z == nil {
|
||||
return ""
|
||||
}
|
||||
if len(z.Interfaces) == 1 && !strings.HasSuffix(z.Interfaces[0], "*") {
|
||||
return fmt.Sprintf("traffic within %s stays on its own network: it passes the firewall only if routed back out of %s", z.Name, z.Interfaces[0])
|
||||
}
|
||||
return fmt.Sprintf("only traffic routed between %s's interfaces (%s) passes the firewall; within one of them it stays on its own network", z.Name, strings.Join(z.Interfaces, ", "))
|
||||
}
|
||||
|
||||
func walkCell(st *model.State, o Options, chain, from, to string) api.MatrixCell {
|
||||
c := api.MatrixCell{From: from, To: to, Chain: chain, Exceptions: []api.MatrixException{}}
|
||||
b := st.Settings.Builtins
|
||||
fromZone := st.Zone(from)
|
||||
external := fromZone != nil && fromZone.External
|
||||
// What comes in passes the early chain first.
|
||||
if chain != model.ChainOutput {
|
||||
if b.Antispoof && external {
|
||||
c.Exceptions = append(c.Exceptions, api.MatrixException{Chain: model.ChainEarly, Label: "built-in: antispoof", Verdict: "blocked",
|
||||
What: "sources that can't come from outside (private ranges, your own prefixes)", Short: "spoofed sources", Marks: []string{"b-antispoof", "b-antispoof6"}})
|
||||
}
|
||||
if decided := walk(st, o, &c, model.ChainEarly, from, ""); decided {
|
||||
return c
|
||||
}
|
||||
}
|
||||
// The built-ins that let some new connections in.
|
||||
switch chain {
|
||||
case model.ChainInput:
|
||||
if b.ICMPv6 {
|
||||
c.Exceptions = append(c.Exceptions, api.MatrixException{Chain: chain, Label: "built-in: icmpv6", Verdict: "allowed", What: "ICMPv6 errors, echo, neighbour discovery, MLD",
|
||||
Short: "ICMPv6", Marks: []string{"b-icmpv6", "b-icmpv6-nd", "b-icmpv6-mld"}})
|
||||
}
|
||||
if b.PingInternal && !external {
|
||||
c.Exceptions = append(c.Exceptions, api.MatrixException{Chain: chain, Label: "built-in: ping", Verdict: "allowed", What: "ping (ICMP echo)", Short: "ping", Marks: []string{"b-ping"}})
|
||||
}
|
||||
if b.DHCPv6Client && external {
|
||||
c.Exceptions = append(c.Exceptions, api.MatrixException{Chain: chain, Label: "built-in: dhcpv6-client", Verdict: "allowed", What: "DHCPv6 replies to the firewall",
|
||||
Short: "DHCPv6", Marks: []string{"b-dhcpv6-client"}})
|
||||
}
|
||||
case model.ChainForward:
|
||||
if marks := forwards(st); len(marks) > 0 && external && !st.Zone(to).External {
|
||||
n := fmt.Sprintf("%d port forward%s", len(marks), map[bool]string{true: "", false: "s"}[len(marks) == 1])
|
||||
c.Exceptions = append(c.Exceptions, api.MatrixException{Chain: chain, Label: "port forwards", Verdict: "allowed",
|
||||
What: n + " to inside hosts", Short: n, Marks: marks})
|
||||
}
|
||||
case model.ChainOutput:
|
||||
if st.Settings.OutputPolicy == model.PolicyDrop && b.ICMPv6 {
|
||||
c.Exceptions = append(c.Exceptions, api.MatrixException{Chain: chain, Label: "built-in: icmpv6", Verdict: "allowed", What: "ICMPv6 errors, echo, neighbour discovery, MLD",
|
||||
Short: "ICMPv6", Marks: []string{"b-icmpv6"}})
|
||||
}
|
||||
}
|
||||
if walk(st, o, &c, chain, from, to) {
|
||||
return c
|
||||
}
|
||||
policy := map[string]string{model.ChainInput: st.Settings.InputPolicy, model.ChainForward: st.Settings.ForwardPolicy, model.ChainOutput: st.Settings.OutputPolicy}[chain]
|
||||
c.Verdict, c.DecidedBy = verdictOf(policy), chain+" policy"
|
||||
if policy == model.PolicyDrop {
|
||||
c.Marks = []string{"p-" + chain} // only a drop policy is counted
|
||||
}
|
||||
return c
|
||||
}
|
||||
|
||||
// forwards returns the counters' marks of the port forwards that send
|
||||
// connections inside.
|
||||
func forwards(st *model.State) []string {
|
||||
var marks []string
|
||||
for _, f := range st.Forwards {
|
||||
if !f.Disabled && f.To != "" {
|
||||
marks = append(marks, fmt.Sprintf("f%d", f.ID))
|
||||
}
|
||||
}
|
||||
return marks
|
||||
}
|
||||
|
||||
// walk goes through a chain's rules for the pair, as nftables checks
|
||||
// them (those that check every packet first). It reports whether a rule
|
||||
// decided the whole pair.
|
||||
func walk(st *model.State, o Options, c *api.MatrixCell, chain, from, to string) bool {
|
||||
var rules []*model.Rule
|
||||
for _, every := range []bool{true, false} {
|
||||
for _, r := range st.Rules {
|
||||
if r.Chain == chain && !r.Disabled && r.ChecksEveryPacket() == every {
|
||||
rules = append(rules, r)
|
||||
}
|
||||
}
|
||||
}
|
||||
for _, r := range rules {
|
||||
mark := []string{fmt.Sprintf("r%d", r.ID)}
|
||||
if r.Raw != "" {
|
||||
rr := readRaw(r.Raw)
|
||||
switch {
|
||||
case !rr.ok:
|
||||
c.Exceptions = append(c.Exceptions, api.MatrixException{RuleID: r.ID, Chain: chain, Label: "rule " + r.Label(), Verdict: "unknown", What: "a raw rule: " + r.Raw,
|
||||
Short: "a raw rule", Marks: mark})
|
||||
case !rr.forNew || chain == model.ChainEarly && rr.verdict == "allowed":
|
||||
// Not for new connections; an early accept decides nothing.
|
||||
case rr.match == "":
|
||||
c.Verdict, c.DecidedBy, c.RuleID, c.Marks = rr.verdict, "rule "+r.Label(), r.ID, mark
|
||||
return true
|
||||
default:
|
||||
c.Exceptions = append(c.Exceptions, api.MatrixException{RuleID: r.ID, Chain: chain, Label: "rule " + r.Label(), Verdict: rr.verdict, What: "raw: " + rr.match,
|
||||
Short: rr.short, Marks: mark})
|
||||
}
|
||||
continue
|
||||
}
|
||||
if !zones(r, from, to) || !forNew(r) {
|
||||
continue
|
||||
}
|
||||
// Addresses that all live behind other zones can't come from (or
|
||||
// go to) this pair's.
|
||||
if !r.NotSources && elsewhere(st, o, r.Sources, from) || !r.NotDests && to != Firewall && to != "" && elsewhere(st, o, r.Destinations, to) {
|
||||
continue
|
||||
}
|
||||
if chain == model.ChainEarly && r.Action != model.ActionDrop && r.Action != model.ActionReject {
|
||||
continue // an early accept decides nothing: input or forward still check
|
||||
}
|
||||
v := verdictOf(r.Action)
|
||||
what := conditions(st, r)
|
||||
if r.Action == model.ActionLog || what != "" {
|
||||
if what == "" {
|
||||
what = "everything"
|
||||
}
|
||||
c.Exceptions = append(c.Exceptions, api.MatrixException{RuleID: r.ID, Chain: chain, Label: "rule " + r.Label(), Verdict: v, What: what,
|
||||
Short: short(r, what), Marks: mark})
|
||||
continue
|
||||
}
|
||||
c.Verdict, c.DecidedBy, c.RuleID, c.Marks = v, "rule "+r.Label(), r.ID, mark
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func verdictOf(action string) string {
|
||||
switch action {
|
||||
case model.ActionAccept: // and the accept policy
|
||||
return "allowed"
|
||||
case model.ActionLog:
|
||||
return "logged"
|
||||
}
|
||||
return "blocked"
|
||||
}
|
||||
|
||||
// elsewhere reports whether every address of refs (addresses, sets) lies
|
||||
// in networks connected to zones other than zone: then traffic of zone
|
||||
// can't have them. An empty list, a dynamic set, or an address the
|
||||
// firewall isn't directly connected to (a routed network) might.
|
||||
func elsewhere(st *model.State, o Options, refs []string, zone string) bool {
|
||||
if len(refs) == 0 || len(o.Connected) == 0 || zone == Firewall {
|
||||
return false
|
||||
}
|
||||
var entries []string
|
||||
for _, r := range refs {
|
||||
if s := st.Set(r); s != nil {
|
||||
if s.Dynamic {
|
||||
return false
|
||||
}
|
||||
entries = append(entries, s.Entries...)
|
||||
} else {
|
||||
entries = append(entries, r)
|
||||
}
|
||||
}
|
||||
if len(entries) == 0 {
|
||||
return false
|
||||
}
|
||||
for _, e := range entries {
|
||||
a, err := model.ParseAddress(e)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
first, last := a.Prefix.Masked().Addr(), lastAddr(a.Prefix)
|
||||
if a.First.IsValid() {
|
||||
first, last = a.First, a.End
|
||||
}
|
||||
owner := ""
|
||||
for net, z := range o.Connected {
|
||||
if net.Contains(first) && net.Contains(last) {
|
||||
owner = z
|
||||
break
|
||||
}
|
||||
}
|
||||
if owner == "" || strings.EqualFold(owner, zone) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// lastAddr is a prefix's last address.
|
||||
func lastAddr(p netip.Prefix) netip.Addr {
|
||||
b := p.Masked().Addr().AsSlice()
|
||||
for i := p.Bits(); i < len(b)*8; i++ {
|
||||
b[i/8] |= 0x80 >> (i % 8)
|
||||
}
|
||||
a, _ := netip.AddrFromSlice(b)
|
||||
return a
|
||||
}
|
||||
|
||||
// zones reports whether a rule's zones take in the pair.
|
||||
func zones(r *model.Rule, from, to string) bool {
|
||||
in := func(list []string, not bool, z string) bool {
|
||||
if len(list) == 0 {
|
||||
return true
|
||||
}
|
||||
for _, x := range list {
|
||||
if strings.EqualFold(x, z) {
|
||||
return !not
|
||||
}
|
||||
}
|
||||
return not
|
||||
}
|
||||
if from == Firewall {
|
||||
if len(r.InZones) > 0 {
|
||||
return false
|
||||
}
|
||||
} else if !in(r.InZones, r.NotIn, from) {
|
||||
return false
|
||||
}
|
||||
if to == "" || to == Firewall {
|
||||
return len(r.OutZones) == 0
|
||||
}
|
||||
return in(r.OutZones, r.NotOut, to)
|
||||
}
|
||||
|
||||
// forNew reports whether a rule can match a new connection's first
|
||||
// packet (its connection states allow "new").
|
||||
func forNew(r *model.Rule) bool {
|
||||
if len(r.CtState) == 0 {
|
||||
return true
|
||||
}
|
||||
for _, s := range r.CtState {
|
||||
if s == "new" {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// raw is what the matrix can read of a raw rule.
|
||||
type raw struct {
|
||||
ok bool // read: it applies to every pair of its chain
|
||||
forNew bool // it can match a new connection
|
||||
verdict string // allowed or blocked
|
||||
match string // what it matches ("" for everything)
|
||||
short string
|
||||
}
|
||||
|
||||
// rawStops are words that make a raw rule's reach or effect unclear to
|
||||
// the matrix: interfaces (which zones?), jumps, NAT, changes, verdict maps.
|
||||
var rawStops = map[string]bool{
|
||||
"iifname": true, "oifname": true, "iif": true, "oif": true, "iiftype": true, "oiftype": true,
|
||||
"jump": true, "goto": true, "return": true, "vmap": true, "queue": true, "continue": true,
|
||||
"dnat": true, "snat": true, "masquerade": true, "redirect": true, "tproxy": true,
|
||||
"set": true, "notrack": true, "dup": true, "fwd": true, "synproxy": true, "flow": true,
|
||||
}
|
||||
|
||||
// readRaw reads a raw rule that is a match and then accept, drop or
|
||||
// reject: the verdict is the last word (but a reject's "with …"); counter
|
||||
// and log only count and log.
|
||||
func readRaw(text string) raw {
|
||||
toks := fields(text)
|
||||
for i, t := range toks {
|
||||
if t == "comment" {
|
||||
toks = toks[:i]
|
||||
break
|
||||
}
|
||||
}
|
||||
verdict := -1
|
||||
for i, t := range toks {
|
||||
if strings.HasPrefix(t, `"`) {
|
||||
continue
|
||||
}
|
||||
w := strings.Trim(t, ",{}")
|
||||
if rawStops[w] {
|
||||
return raw{}
|
||||
}
|
||||
if w == "accept" || w == "drop" || w == "reject" {
|
||||
verdict = i
|
||||
}
|
||||
}
|
||||
if verdict < 0 || verdict != len(toks)-1 && !(toks[verdict] == "reject" && toks[verdict+1] == "with") {
|
||||
return raw{}
|
||||
}
|
||||
rr := raw{ok: true, forNew: true, verdict: verdictOf(toks[verdict])}
|
||||
var match []string
|
||||
for i := 0; i < verdict; i++ {
|
||||
switch toks[i] {
|
||||
case "counter":
|
||||
continue
|
||||
case "log":
|
||||
// Its options run up to the next match.
|
||||
for i+1 < verdict && map[string]bool{"prefix": true, "level": true, "group": true, "flags": true}[toks[i+1]] {
|
||||
i += 2
|
||||
}
|
||||
continue
|
||||
}
|
||||
match = append(match, toks[i])
|
||||
}
|
||||
rr.match = strings.Join(match, " ")
|
||||
// ct state without new: not for new connections. The states are one
|
||||
// word ("established,related") or a set ("{ established, related }").
|
||||
for i := 0; i+2 < len(match); i++ {
|
||||
if match[i] != "ct" || match[i+1] != "state" {
|
||||
continue
|
||||
}
|
||||
list := match[i+2]
|
||||
if list == "{" || strings.HasPrefix(list, "{") {
|
||||
var b []string
|
||||
for j := i + 2; j < len(match); j++ {
|
||||
b = append(b, match[j])
|
||||
if strings.HasSuffix(match[j], "}") {
|
||||
break
|
||||
}
|
||||
}
|
||||
list = strings.Join(b, " ")
|
||||
}
|
||||
rr.forNew = false
|
||||
for _, st := range strings.FieldsFunc(list, func(c rune) bool { return c == ',' || c == ' ' || c == '{' || c == '}' }) {
|
||||
rr.forNew = rr.forNew || st == "new"
|
||||
}
|
||||
}
|
||||
rr.short = rr.match
|
||||
if m := icmpRe.FindStringSubmatch(rr.match); m != nil {
|
||||
rr.short = m[1]
|
||||
}
|
||||
if len(rr.short) > 40 {
|
||||
rr.short = rr.short[:39] + "…"
|
||||
}
|
||||
return rr
|
||||
}
|
||||
|
||||
// fields splits a rule into words, a quoted string being one.
|
||||
func fields(s string) []string {
|
||||
var out []string
|
||||
var cur strings.Builder
|
||||
quoted := false
|
||||
for _, c := range s {
|
||||
switch {
|
||||
case c == '"':
|
||||
quoted = !quoted
|
||||
cur.WriteRune(c)
|
||||
case !quoted && (c == ' ' || c == '\t'):
|
||||
if cur.Len() > 0 {
|
||||
out = append(out, cur.String())
|
||||
cur.Reset()
|
||||
}
|
||||
default:
|
||||
cur.WriteRune(c)
|
||||
}
|
||||
}
|
||||
if cur.Len() > 0 {
|
||||
out = append(out, cur.String())
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// icmpRe finds an ICMP type match in a raw rule: shown as just icmp or
|
||||
// icmpv6.
|
||||
var icmpRe = regexp.MustCompile(`\b(icmpv6|icmp) type\b`)
|
||||
|
||||
// short sums up an exception for the overview: its services (ICMP types
|
||||
// as just icmp or icmpv6), with the addresses it is limited to, which
|
||||
// matter as much ("tcp/22 from trusted"); times and rates are left to
|
||||
// the details.
|
||||
func short(r *model.Rule, what string) string {
|
||||
var parts []string
|
||||
if len(r.Services) > 0 {
|
||||
var svcs []string
|
||||
seen := map[string]bool{}
|
||||
for _, s := range r.Services {
|
||||
if p, _, ok := strings.Cut(s, "/"); ok && (p == "icmp" || p == "icmpv6") {
|
||||
s = p
|
||||
}
|
||||
if !seen[s] {
|
||||
seen[s] = true
|
||||
svcs = append(svcs, s)
|
||||
}
|
||||
}
|
||||
parts = append(parts, strings.Join(svcs, ", "))
|
||||
}
|
||||
for _, c := range strings.Split(what, " · ") {
|
||||
if strings.HasPrefix(c, "from ") || strings.HasPrefix(c, "to ") {
|
||||
parts = append(parts, c)
|
||||
}
|
||||
}
|
||||
if len(parts) == 0 {
|
||||
first, _, _ := strings.Cut(what, " · ")
|
||||
return first
|
||||
}
|
||||
return strings.Join(parts, " ")
|
||||
}
|
||||
|
||||
// conditions says what limits a rule to part of a pair's traffic ("" for
|
||||
// none: it decides the pair).
|
||||
func conditions(st *model.State, r *model.Rule) string {
|
||||
var parts []string
|
||||
list := func(not bool, xs []string) string {
|
||||
s := strings.Join(xs, ", ")
|
||||
if not {
|
||||
return "any but " + s
|
||||
}
|
||||
return s
|
||||
}
|
||||
if len(r.Sources) > 0 {
|
||||
parts = append(parts, "from "+list(r.NotSources, r.Sources))
|
||||
}
|
||||
if len(r.Destinations) > 0 {
|
||||
parts = append(parts, "to "+list(r.NotDests, r.Destinations))
|
||||
}
|
||||
if len(r.Services) > 0 {
|
||||
parts = append(parts, strings.Join(r.Services, ", "))
|
||||
}
|
||||
if r.Family != "" {
|
||||
parts = append(parts, r.Family+" only")
|
||||
}
|
||||
if len(r.CtStatus) > 0 {
|
||||
parts = append(parts, "connections that were "+strings.Join(r.CtStatus, ", ")+"'d")
|
||||
}
|
||||
if len(r.Days) > 0 {
|
||||
parts = append(parts, strings.Join(r.Days, ", "))
|
||||
}
|
||||
if r.HourFrom != "" {
|
||||
parts = append(parts, r.HourFrom+"–"+r.HourTo)
|
||||
}
|
||||
if l := r.Limit; l != nil {
|
||||
unit := ""
|
||||
if l.Unit != "" && l.Unit != "packets" {
|
||||
unit = " " + l.Unit
|
||||
}
|
||||
rate := fmt.Sprintf("%d%s/%s", l.Rate, unit, l.Per)
|
||||
if l.Over {
|
||||
parts = append(parts, "beyond "+rate)
|
||||
} else {
|
||||
parts = append(parts, "within "+rate)
|
||||
}
|
||||
}
|
||||
if r.ConnLimit > 0 {
|
||||
parts = append(parts, fmt.Sprintf("sources with over %d connections", r.ConnLimit))
|
||||
}
|
||||
if q := r.Quota; q != nil {
|
||||
if q.Over {
|
||||
parts = append(parts, "once its quota is used up")
|
||||
} else {
|
||||
parts = append(parts, "while its quota lasts")
|
||||
}
|
||||
}
|
||||
return strings.Join(parts, " · ")
|
||||
}
|
||||
@@ -0,0 +1,199 @@
|
||||
package matrix
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"net/netip"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"housecarl/internal/api"
|
||||
"housecarl/internal/model"
|
||||
)
|
||||
|
||||
func state() *model.State {
|
||||
st := model.NewState()
|
||||
st.Zones = []*model.Zone{
|
||||
{ID: 1, Name: "guest", Interfaces: []string{"eth2"}},
|
||||
{ID: 2, Name: "lan", Interfaces: []string{"eth1"}},
|
||||
{ID: 3, Name: "wan", Interfaces: []string{"wan"}, External: true},
|
||||
}
|
||||
st.Sets = []*model.AddressSet{{ID: 4, Name: "admins", Entries: []string{"192.168.1.0/28"}}}
|
||||
pos := 0.0
|
||||
rule := func(r model.Rule) {
|
||||
pos++
|
||||
r.ID, r.Position = int64(10+pos), pos
|
||||
st.Rules = append(st.Rules, &r)
|
||||
}
|
||||
rule(model.Rule{Chain: "input", Match: model.Match{Sources: []string{"admins"}, Services: []string{"ssh"}}, Action: "accept", Comment: "SSH"})
|
||||
rule(model.Rule{Chain: "input", Match: model.Match{InZones: []string{"lan", "guest"}, Services: []string{"dns"}}, Action: "accept"})
|
||||
rule(model.Rule{Chain: "forward", Match: model.Match{InZones: []string{"lan"}, OutZones: []string{"wan"}}, Action: "accept", Comment: "LAN out"})
|
||||
rule(model.Rule{Chain: "forward", Match: model.Match{InZones: []string{"guest"}, OutZones: []string{"wan"}, Services: []string{"http", "https"}}, Action: "accept"})
|
||||
rule(model.Rule{Chain: "forward", Match: model.Match{InZones: []string{"wan"}}, Action: "drop", Comment: "nothing in"})
|
||||
rule(model.Rule{Chain: "forward", Match: model.Match{InZones: []string{"lan"}}, CtState: []string{"established"}, Action: "drop"}) // not for new
|
||||
rule(model.Rule{Chain: "output", Match: model.Match{OutZones: []string{"wan"}}, Action: "drop", Disabled: true})
|
||||
st.Forwards = []*model.PortForward{{ID: 30, Name: "web", Protocols: []string{"tcp"}, Ports: "443", To: "192.168.1.10"}}
|
||||
st.Settings.Builtins.MSSClamp = true
|
||||
return st
|
||||
}
|
||||
|
||||
func find(m api.Matrix, from, to string) api.MatrixCell {
|
||||
for _, c := range m.Cells {
|
||||
if c.From == from && c.To == to {
|
||||
return c
|
||||
}
|
||||
}
|
||||
return api.MatrixCell{}
|
||||
}
|
||||
|
||||
func TestMatrix(t *testing.T) {
|
||||
m := Build(state())
|
||||
if strings.Join(m.Zones, ",") != "guest,lan,wan" || len(m.Cells) != 3*3+3+3 {
|
||||
t.Fatalf("zones %v, %d cells", m.Zones, len(m.Cells))
|
||||
}
|
||||
for _, c := range []struct {
|
||||
from, to, verdict, decided string
|
||||
exceptions int
|
||||
}{
|
||||
{"lan", "wan", "allowed", "rule forward #13 (LAN out)", 0},
|
||||
{"guest", "wan", "blocked", "forward policy", 1}, // only web
|
||||
{"wan", "lan", "blocked", "rule forward #15 (nothing in)", 2}, // antispoof, port forwards come first
|
||||
{"lan", "guest", "blocked", "forward policy", 0}, // the established-only rule is no exception
|
||||
{"lan", Firewall, "blocked", "input policy", 4}, // icmpv6, ping, ssh from admins, dns
|
||||
{"wan", Firewall, "blocked", "input policy", 4}, // antispoof, icmpv6, dhcpv6, ssh from admins
|
||||
{Firewall, "wan", "allowed", "output policy", 0}, // the drop is disabled
|
||||
} {
|
||||
got := find(m, c.from, c.to)
|
||||
if got.Verdict != c.verdict || got.DecidedBy != c.decided || len(got.Exceptions) != c.exceptions {
|
||||
t.Errorf("%s → %s: %s by %s, %d exceptions %+v; want %s by %s, %d", c.from, c.to, got.Verdict, got.DecidedBy, len(got.Exceptions), got.Exceptions, c.verdict, c.decided, c.exceptions)
|
||||
}
|
||||
}
|
||||
g := find(m, "guest", "wan").Exceptions[0]
|
||||
if g.Verdict != "allowed" || g.What != "http, https" || g.RuleID != 14 {
|
||||
t.Errorf("the guest exception: %+v", g)
|
||||
}
|
||||
if e := find(m, "wan", "lan").Exceptions; e[0].Label != "built-in: antispoof" || e[1].Label != "port forwards" {
|
||||
t.Errorf("wan → lan exceptions: %+v", e)
|
||||
}
|
||||
if _, ok := Cell(state(), Options{}, "lan", "nowhere"); ok {
|
||||
t.Error("an unknown zone")
|
||||
}
|
||||
}
|
||||
|
||||
// Rules whose addresses all live behind another zone are left out of a
|
||||
// pair; routed (unknown) networks, and mixed lists, stay.
|
||||
func TestConnected(t *testing.T) {
|
||||
st := state()
|
||||
st.Sets = append(st.Sets, &model.AddressSet{ID: 5, Name: "guest_net", Entries: []string{"192.168.2.0/24"}},
|
||||
&model.AddressSet{ID: 6, Name: "mixed", Entries: []string{"192.168.2.0/24", "10.20.30.0/24"}})
|
||||
st.Rules = append(st.Rules,
|
||||
&model.Rule{ID: 40, Chain: "forward", Position: 10, Match: model.Match{Sources: []string{"guest_net"}}, Action: "drop", Comment: "guests"},
|
||||
&model.Rule{ID: 41, Chain: "forward", Position: 11, Match: model.Match{Destinations: []string{"192.168.2.5"}}, Action: "accept", Comment: "to a guest"},
|
||||
&model.Rule{ID: 42, Chain: "forward", Position: 12, Match: model.Match{Sources: []string{"mixed"}}, Action: "drop", Comment: "mixed"})
|
||||
o := Options{Connected: map[netip.Prefix]string{netip.MustParsePrefix("192.168.1.0/24"): "lan", netip.MustParsePrefix("192.168.2.0/24"): "guest"}}
|
||||
ids := func(c api.MatrixCell) []int64 {
|
||||
var out []int64
|
||||
for _, e := range c.Exceptions {
|
||||
out = append(out, e.RuleID)
|
||||
}
|
||||
return out
|
||||
}
|
||||
lanWan, _ := Cell(st, o, "lan", "wan")
|
||||
if fmt.Sprint(ids(lanWan)) != "[]" {
|
||||
t.Errorf("lan → wan: %v", ids(lanWan)) // decided by "LAN out" before the others anyway
|
||||
}
|
||||
lanGuest, _ := Cell(st, o, "lan", "guest")
|
||||
if fmt.Sprint(ids(lanGuest)) != "[41 42]" {
|
||||
t.Errorf("lan → guest: %v", ids(lanGuest)) // not "guests": their addresses are guest's
|
||||
}
|
||||
guestLan, _ := Cell(st, o, "guest", "lan")
|
||||
if fmt.Sprint(ids(guestLan)) != "[40 42]" {
|
||||
t.Errorf("guest → lan: %v", ids(guestLan)) // not "to a guest": it is in guest, not lan
|
||||
}
|
||||
}
|
||||
|
||||
// TestSummary: each pair at a glance, what its exceptions let through,
|
||||
// and the counters behind each line.
|
||||
func TestSummary(t *testing.T) {
|
||||
st := state()
|
||||
st.Settings.InputPolicy = model.PolicyDrop
|
||||
m := Build(st)
|
||||
for _, c := range []struct {
|
||||
from, to, summary string
|
||||
allowed, blocked int
|
||||
open string
|
||||
}{
|
||||
{"lan", "wan", "allowed", 0, 0, ""},
|
||||
{"guest", "wan", "partly", 1, 0, "http, https"},
|
||||
{"wan", "lan", "partly", 1, 1, "1 port forward"},
|
||||
{"lan", "guest", "blocked", 0, 0, ""},
|
||||
{"lan", Firewall, "partly", 4, 0, "ICMPv6|ping|ssh from admins|dns"},
|
||||
} {
|
||||
got := find(m, c.from, c.to)
|
||||
if got.Summary != c.summary || got.Allowed != c.allowed || got.Blocked != c.blocked || strings.Join(got.Open, "|") != c.open {
|
||||
t.Errorf("%s → %s: %s +%d -%d %q; want %s +%d -%d %q", c.from, c.to, got.Summary, got.Allowed, got.Blocked, got.Open, c.summary, c.allowed, c.blocked, c.open)
|
||||
}
|
||||
}
|
||||
if c := find(m, "lan", "lan"); !c.SameZone || !strings.Contains(c.Note, "out of eth1") {
|
||||
t.Errorf("same zone: %+v", c)
|
||||
}
|
||||
if c := find(m, "lan", "wan"); c.SameZone || c.Note != "" {
|
||||
t.Errorf("not the same zone: %+v", c)
|
||||
}
|
||||
wl := find(m, "wan", "lan")
|
||||
if strings.Join(wl.Exceptions[0].Marks, ",") != "b-antispoof,b-antispoof6" || strings.Join(wl.Exceptions[1].Marks, ",") != "f30" || strings.Join(wl.Marks, ",") != "r15" {
|
||||
t.Errorf("marks: %+v", wl)
|
||||
}
|
||||
icmp := &model.Rule{Match: model.Match{Services: []string{"icmp/echo-request", "icmp/echo-reply", "icmpv6/echo-request", "ssh"}}}
|
||||
if s := short(icmp, "x"); s != "icmp, icmpv6, ssh" {
|
||||
t.Errorf("short: %q", s)
|
||||
}
|
||||
ssh := &model.Rule{Match: model.Match{Sources: []string{"trusted"}, Services: []string{"ssh"}}, Days: []string{"mon"}}
|
||||
if s := short(ssh, conditions(st, ssh)); s != "ssh from trusted" {
|
||||
t.Errorf("short with a source: %q", s)
|
||||
}
|
||||
if s := short(&model.Rule{Days: []string{"mon"}}, "mon"); s != "mon" {
|
||||
t.Errorf("short of a time: %q", s)
|
||||
}
|
||||
if c := find(m, "lan", Firewall); strings.Join(c.Marks, ",") != "p-input" {
|
||||
t.Errorf("a drop policy's mark: %v", c.Marks)
|
||||
}
|
||||
}
|
||||
|
||||
// TestRaw: raw rules that are a match and a verdict are read; others stay
|
||||
// unknown.
|
||||
func TestRaw(t *testing.T) {
|
||||
for _, c := range []struct {
|
||||
raw, verdict, short string
|
||||
ok, forNew bool
|
||||
}{
|
||||
{"icmpv6 type { nd-router-solicit, nd-neighbor-advert } ip6 hoplimit 255 accept", "allowed", "icmpv6", true, true},
|
||||
{`tcp dport 22 counter log prefix "ssh: " drop comment "x"`, "blocked", "tcp dport 22", true, true},
|
||||
{"ip saddr 10.0.0.1 reject with icmpx type admin-prohibited", "blocked", "ip saddr 10.0.0.1", true, true},
|
||||
{"ct state established,related accept", "allowed", "ct state established,related", true, false},
|
||||
{"ct state { new, untracked } tcp dport 80 accept", "allowed", "ct state { new, untracked } tcp dport 80", true, true},
|
||||
{"counter accept", "allowed", "", true, true},
|
||||
{`iifname "wan" tcp dport 22 accept`, "", "", false, false},
|
||||
{"tcp dport vmap { 22 : accept }", "", "", false, false},
|
||||
{"meta mark set 1 accept", "", "", false, false},
|
||||
{"tcp dport 22 counter", "", "", false, false},
|
||||
} {
|
||||
rr := readRaw(c.raw)
|
||||
if rr.ok != c.ok || c.ok && (rr.verdict != c.verdict || rr.short != c.short || rr.forNew != c.forNew) {
|
||||
t.Errorf("%q: %+v", c.raw, rr)
|
||||
}
|
||||
}
|
||||
// In a pair: an exception that lets ND in, not "unknown".
|
||||
st := state()
|
||||
st.Rules = append(st.Rules, &model.Rule{ID: 90, Chain: "input", Position: 0.5, Raw: "icmpv6 type { nd-router-solicit } ip6 hoplimit 255 accept"})
|
||||
st.Sort()
|
||||
e := find(Build(st), "wan", Firewall).Exceptions
|
||||
found := false
|
||||
for _, x := range e {
|
||||
if x.RuleID == 90 {
|
||||
found = x.Verdict == "allowed" && x.Short == "icmpv6" && strings.HasPrefix(x.What, "raw: icmpv6 type")
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
t.Errorf("the raw ND rule: %+v", e)
|
||||
}
|
||||
}
|
||||
@@ -32,6 +32,34 @@ func (a Address) String() string {
|
||||
return a.Prefix.String()
|
||||
}
|
||||
|
||||
// Contains reports whether the entry holds an address.
|
||||
func (a Address) Contains(x netip.Addr) bool {
|
||||
x = x.Unmap()
|
||||
if a.First.IsValid() {
|
||||
return x.Is4() == a.First.Is4() && !x.Less(a.First) && !a.End.Less(x)
|
||||
}
|
||||
return a.Prefix.Contains(x)
|
||||
}
|
||||
|
||||
// Overlaps reports whether the entry and a prefix share an address.
|
||||
func (a Address) Overlaps(p netip.Prefix) bool {
|
||||
if !a.First.IsValid() {
|
||||
return a.Prefix.Overlaps(p)
|
||||
}
|
||||
if a.First.Is4() != p.Addr().Is4() {
|
||||
return false
|
||||
}
|
||||
// The prefix's first and last addresses against the range.
|
||||
first := p.Masked().Addr()
|
||||
last := first
|
||||
b := last.AsSlice()
|
||||
for i := p.Bits(); i < len(b)*8; i++ {
|
||||
b[i/8] |= 0x80 >> (i % 8)
|
||||
}
|
||||
last, _ = netip.AddrFromSlice(b)
|
||||
return !last.Less(a.First) && !a.End.Less(first)
|
||||
}
|
||||
|
||||
// ParseAddress reads an address, a prefix (host bits must be zero) or a
|
||||
// range, IPv4 or IPv6.
|
||||
func ParseAddress(s string) (Address, error) {
|
||||
|
||||
@@ -7,9 +7,10 @@ import (
|
||||
|
||||
// Grant gives access to one resource (see internal/auth).
|
||||
type Grant struct {
|
||||
Resource string `json:"resource"` // config, sets, log, audit, *
|
||||
Resource string `json:"resource"` // config, sets, bans, log, audit, *
|
||||
Access string `json:"access"` // read or write (write implies read)
|
||||
// Sets limits a "sets" grant to these dynamic sets (none: all).
|
||||
// Sets limits a "sets" grant to these dynamic sets, and a "bans"
|
||||
// grant to these ban lists (none: all).
|
||||
Sets []string `json:"sets,omitempty"`
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,53 @@
|
||||
package model
|
||||
|
||||
import (
|
||||
"net/netip"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Ban is an address a ban list keeps out (docs/bans.md): of every port,
|
||||
// or of one protocol and port. Bans are kept apart from the
|
||||
// configuration, like a dynamic set's entries: not in the history, and
|
||||
// not on probation.
|
||||
type Ban struct {
|
||||
ID int64 `json:"id"`
|
||||
ListID int64 `json:"list_id"`
|
||||
List string `json:"list"`
|
||||
// Address is what is banned: an address, or an IPv6 prefix of the
|
||||
// list's grouping ("2001:db8:1:2::/64").
|
||||
Address string `json:"address"`
|
||||
Protocol string `json:"protocol,omitempty"` // tcp or udp; "" every port
|
||||
Port int `json:"port,omitempty"`
|
||||
// Expires is when the ban ends (nil: it doesn't); Timeout the
|
||||
// seconds it has left then.
|
||||
Expires *time.Time `json:"expires,omitempty"`
|
||||
Timeout int `json:"timeout,omitempty"`
|
||||
Created time.Time `json:"created"`
|
||||
// Who banned it (the user or token's name), Source the program's own
|
||||
// name for why ("guard/ssh", "fail2ban/sshd"), Reason free text (a
|
||||
// log line).
|
||||
Who string `json:"who"`
|
||||
Source string `json:"source,omitempty"`
|
||||
Reason string `json:"reason,omitempty"`
|
||||
}
|
||||
|
||||
// Prefix is the banned address as a prefix.
|
||||
func (b *Ban) Prefix() (netip.Prefix, error) {
|
||||
if p, err := netip.ParsePrefix(b.Address); err == nil {
|
||||
return p, nil
|
||||
}
|
||||
a, err := netip.ParseAddr(b.Address)
|
||||
if err != nil {
|
||||
return netip.Prefix{}, err
|
||||
}
|
||||
return netip.PrefixFrom(a, a.BitLen()), nil
|
||||
}
|
||||
|
||||
// BanAddress writes a banned prefix: an address alone when it is one
|
||||
// address, else the prefix.
|
||||
func BanAddress(p netip.Prefix) string {
|
||||
if p.IsSingleIP() {
|
||||
return p.Addr().String()
|
||||
}
|
||||
return p.String()
|
||||
}
|
||||
@@ -0,0 +1,62 @@
|
||||
package model
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Changeset statuses.
|
||||
const (
|
||||
ChangesetOpen = "open"
|
||||
ChangesetApplied = "applied"
|
||||
ChangesetDiscarded = "discarded"
|
||||
)
|
||||
|
||||
// Changeset is a list of changes, staged to be reviewed and applied
|
||||
// together (or discarded). Applied, it is one history entry.
|
||||
type Changeset struct {
|
||||
ID int64 `json:"id"`
|
||||
UserID int64 `json:"user_id,omitempty"`
|
||||
Username string `json:"username"`
|
||||
Via string `json:"via"`
|
||||
Description string `json:"description"`
|
||||
Status string `json:"status"`
|
||||
Ops []Op `json:"ops"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
UpdatedAt time.Time `json:"updated_at"`
|
||||
// HistoryID is the history entry an applied changeset became.
|
||||
HistoryID int64 `json:"history_id,omitempty"`
|
||||
}
|
||||
|
||||
// Operations.
|
||||
const (
|
||||
OpCreate = "create"
|
||||
OpUpdate = "update"
|
||||
OpDelete = "delete"
|
||||
OpMove = "move"
|
||||
OpSettings = "settings"
|
||||
// OpRevert undoes a history entry: Data is the entry (its changes),
|
||||
// ID its number.
|
||||
OpRevert = "revert"
|
||||
)
|
||||
|
||||
// Op is one change, as the API received it, so it can be stored and
|
||||
// replayed on whatever the configuration is when it is applied.
|
||||
type Op struct {
|
||||
Seq int `json:"seq"`
|
||||
Op string `json:"op"` // create, update, delete, move, settings, revert
|
||||
Kind string `json:"kind,omitempty"`
|
||||
// Ref names the object (ID, or name) for update, delete and move.
|
||||
Ref string `json:"ref,omitempty"`
|
||||
// ID is the object a create makes, given when it is staged so later
|
||||
// operations can name it.
|
||||
ID int64 `json:"id,omitempty"`
|
||||
// Data is the object (create), the fields to change (update,
|
||||
// settings) or where to move it (move).
|
||||
Data json.RawMessage `json:"data,omitempty"`
|
||||
// Place is where a create puts the object in its order (a move's
|
||||
// data); none: at the end.
|
||||
Place json.RawMessage `json:"place,omitempty"`
|
||||
Summary string `json:"summary"`
|
||||
AddedAt time.Time `json:"added_at"`
|
||||
}
|
||||
+254
-10
@@ -133,9 +133,21 @@ type Rule struct {
|
||||
// LimitOver, beyond) a rate.
|
||||
Limit *Limit `json:"limit,omitempty"`
|
||||
// ConnLimit matches when a source already has more than this many
|
||||
// connections (per source address).
|
||||
ConnLimit int `json:"conn_limit,omitempty"`
|
||||
Action string `json:"action"`
|
||||
// connections (per source address, grouped as GroupV4/GroupV6 say).
|
||||
ConnLimit int `json:"conn_limit,omitempty"`
|
||||
// Quota, if set, is an allowance of bytes: the rule matches once it
|
||||
// is used up (Over), or while it lasts.
|
||||
Quota *Quota `json:"quota,omitempty"`
|
||||
// GroupV4 and GroupV6 group addresses for the per-address limits,
|
||||
// connection limit and quota: by prefix length (0: the default, each
|
||||
// IPv4 address alone, IPv6 by /64, since one device has several).
|
||||
GroupV4 int `json:"group_v4,omitempty"`
|
||||
GroupV6 int `json:"group_v6,omitempty"`
|
||||
// EveryPacket checks the rule against every packet, before established
|
||||
// connections are accepted, so a drop or reject also stops connections
|
||||
// already open, not just new ones.
|
||||
EveryPacket bool `json:"every_packet,omitempty"`
|
||||
Action string `json:"action"`
|
||||
// RejectWith: "" (ICMP port/admin unreachable as fits) or tcp-reset.
|
||||
RejectWith string `json:"reject_with,omitempty"`
|
||||
Log bool `json:"log,omitempty"`
|
||||
@@ -155,18 +167,150 @@ func (r *Rule) Label() string {
|
||||
return fmt.Sprintf("%s #%d", r.Chain, r.ID)
|
||||
}
|
||||
|
||||
// Limit is a rate: Rate packets per Per (second, minute, hour, day), with
|
||||
// a burst.
|
||||
// Limit is a rate: Rate packets (or bytes, with Unit) per Per (second,
|
||||
// minute, hour, day), with a burst.
|
||||
type Limit struct {
|
||||
Rate int `json:"rate"`
|
||||
Per string `json:"per"`
|
||||
Burst int `json:"burst,omitempty"`
|
||||
Rate int `json:"rate"`
|
||||
Per string `json:"per"`
|
||||
// Burst is in the rate's unit.
|
||||
Burst int `json:"burst,omitempty"`
|
||||
// Unit: "" for packets, or bytes, kbytes or mbytes for a byte rate
|
||||
// (bandwidth).
|
||||
Unit string `json:"unit,omitempty"`
|
||||
// Over matches packets beyond the rate instead of within it (to drop
|
||||
// floods).
|
||||
Over bool `json:"over,omitempty"`
|
||||
// PerSource counts each source address on its own (a meter), instead
|
||||
// of all traffic the rule sees together.
|
||||
PerSource bool `json:"per_source,omitempty"`
|
||||
// of all traffic the rule sees together; PerDestination each
|
||||
// destination.
|
||||
PerSource bool `json:"per_source,omitempty"`
|
||||
PerDestination bool `json:"per_destination,omitempty"`
|
||||
}
|
||||
|
||||
// Byte units, as nftables writes them.
|
||||
var ByteUnits = map[string]int64{"bytes": 1, "kbytes": 1 << 10, "mbytes": 1 << 20, "gbytes": 1 << 30}
|
||||
|
||||
// Quota is an allowance of bytes. nftables counts it; Housecarl keeps the
|
||||
// count across changes and restarts, and resets it each period.
|
||||
type Quota struct {
|
||||
Bytes int64 `json:"bytes"`
|
||||
// Over matches once the allowance is used up (to drop); otherwise the
|
||||
// rule matches while it lasts (to accept).
|
||||
Over bool `json:"over,omitempty"`
|
||||
// PerSource gives each source address (grouped as the rule's
|
||||
// GroupV4/GroupV6 say) its own allowance; PerDestination each
|
||||
// destination.
|
||||
PerSource bool `json:"per_source,omitempty"`
|
||||
PerDestination bool `json:"per_destination,omitempty"`
|
||||
// Reset: "" (only by hand), daily, weekly (on Mondays) or monthly (on
|
||||
// Day, 1-31: a short month's last day when it has no such day), at
|
||||
// midnight.
|
||||
Reset string `json:"reset,omitempty"`
|
||||
Day int `json:"day,omitempty"`
|
||||
}
|
||||
|
||||
// Quota resets.
|
||||
var QuotaResets = []string{"daily", "weekly", "monthly"}
|
||||
|
||||
// PerAddress reports whether a quota (or limit) counts each address on its
|
||||
// own, and which: "saddr", "daddr" or "".
|
||||
func (q *Quota) PerAddress() string {
|
||||
switch {
|
||||
case q == nil:
|
||||
return ""
|
||||
case q.PerSource:
|
||||
return "saddr"
|
||||
case q.PerDestination:
|
||||
return "daddr"
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// PerAddress: as Quota's.
|
||||
func (l *Limit) PerAddress() string {
|
||||
switch {
|
||||
case l == nil:
|
||||
return ""
|
||||
case l.PerSource:
|
||||
return "saddr"
|
||||
case l.PerDestination:
|
||||
return "daddr"
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// CountsTraffic reports whether a rule measures traffic (a quota or a byte
|
||||
// rate): it must see every packet of a connection, so it is checked before
|
||||
// the built-in that accepts established connections.
|
||||
func (r *Rule) CountsTraffic() bool {
|
||||
if r.Raw != "" {
|
||||
return false
|
||||
}
|
||||
if r.Quota != nil {
|
||||
return true
|
||||
}
|
||||
if r.Limit == nil {
|
||||
return false
|
||||
}
|
||||
_, bytes := ByteUnits[r.Limit.Unit]
|
||||
return bytes
|
||||
}
|
||||
|
||||
// ChecksEveryPacket reports whether a rule is checked before the built-in
|
||||
// that accepts established connections: it counts traffic, or asks to.
|
||||
func (r *Rule) ChecksEveryPacket() bool {
|
||||
return r.Raw == "" && (r.EveryPacket || r.CountsTraffic())
|
||||
}
|
||||
|
||||
// Groups returns a rule's address grouping, defaults applied.
|
||||
func (r *Rule) Groups() (v4, v6 int) {
|
||||
v4, v6 = r.GroupV4, r.GroupV6
|
||||
if v4 == 0 {
|
||||
v4 = 32
|
||||
}
|
||||
if v6 == 0 {
|
||||
v6 = 64
|
||||
}
|
||||
return v4, v6
|
||||
}
|
||||
|
||||
// What becomes of what a rule's quota used when the rule changes
|
||||
// (QuotaCarry).
|
||||
const (
|
||||
QuotaKept = "" // counted as before: carried as it is
|
||||
QuotaSummed = "summed" // per address to a total: the addresses' sum
|
||||
QuotaRegrouped = "regrouped" // per address in wider groups: added up per group
|
||||
QuotaRestarts = "restarts" // can't be carried: starts again
|
||||
)
|
||||
|
||||
// QuotaCarry says what becomes of what old's quota used when the rule
|
||||
// becomes new (both the same rule, with quotas), and for QuotaRestarts
|
||||
// why. The amount, over and the resets don't matter: only how it counts.
|
||||
func QuotaCarry(old, new *Rule) (carry, why string) {
|
||||
if old == nil || new == nil || old.Quota == nil || new.Quota == nil {
|
||||
return QuotaKept, ""
|
||||
}
|
||||
was, is := old.Quota.PerAddress(), new.Quota.PerAddress()
|
||||
words := map[string]string{"saddr": "per source", "daddr": "per destination", "": "in total"}
|
||||
switch {
|
||||
case was == "" && is == "":
|
||||
return QuotaKept, ""
|
||||
case is == "":
|
||||
return QuotaSummed, ""
|
||||
case was == "":
|
||||
return QuotaRestarts, "a total can't be split by address, so each address starts at nothing"
|
||||
case was != is:
|
||||
return QuotaRestarts, "what was counted " + words[was] + " can't be counted " + words[is]
|
||||
}
|
||||
o4, o6 := old.Groups()
|
||||
n4, n6 := new.Groups()
|
||||
switch {
|
||||
case o4 == n4 && o6 == n6:
|
||||
return QuotaKept, ""
|
||||
case n4 > o4 || n6 > o6:
|
||||
return QuotaRestarts, "addresses are grouped more finely now, and a group's use can't be split"
|
||||
}
|
||||
return QuotaRegrouped, ""
|
||||
}
|
||||
|
||||
// PortForward publishes a service on an inside host: incoming connections
|
||||
@@ -248,3 +392,103 @@ func (r *Redirect) Label() string {
|
||||
}
|
||||
return fmt.Sprintf("redirect #%d", r.ID)
|
||||
}
|
||||
|
||||
// Ban list scopes: what a list's bans keep the banned addresses from.
|
||||
const (
|
||||
// BanScopeFirewall: the firewall itself (the input chain).
|
||||
BanScopeFirewall = "firewall"
|
||||
// BanScopeHosts: the list's hosts, behind the firewall (the forward
|
||||
// chain, where port forwards' addresses are already translated).
|
||||
BanScopeHosts = "hosts"
|
||||
// BanScopeEverything: the firewall and everything behind it.
|
||||
BanScopeEverything = "everything"
|
||||
)
|
||||
|
||||
// BanScopes lists the scopes.
|
||||
var BanScopes = []string{BanScopeFirewall, BanScopeHosts, BanScopeEverything}
|
||||
|
||||
// Ban lists' defaults.
|
||||
const (
|
||||
DefaultBanTime = 3600
|
||||
DefaultBanMaxTime = 7 * 24 * 3600
|
||||
DefaultBanEntries = 10000
|
||||
DefaultBanGroupV6 = 64
|
||||
)
|
||||
|
||||
// BanList is where a program such as Guard, or fail2ban, bans addresses
|
||||
// (docs/bans.md). The program names only the address, and perhaps a
|
||||
// protocol and ports; the list says what the address is kept from. The
|
||||
// bans themselves are kept apart from the configuration, like a dynamic
|
||||
// set's entries.
|
||||
type BanList struct {
|
||||
ID int64 `json:"id"`
|
||||
Name string `json:"name"`
|
||||
Scope string `json:"scope"`
|
||||
// Hosts are set names or addresses: what scope hosts protects.
|
||||
Hosts []string `json:"hosts,omitempty"`
|
||||
// Never are set names or addresses the list never bans.
|
||||
Never []string `json:"never,omitempty"`
|
||||
// DefaultTime is a ban's length in seconds when none is asked for
|
||||
// (0: an hour); MaxTime the longest (0: a week; -1: no limit, so
|
||||
// bans may be permanent).
|
||||
DefaultTime int `json:"default_time,omitempty"`
|
||||
MaxTime int `json:"max_time,omitempty"`
|
||||
// MaxEntries limits the bans the list holds (0: 10000).
|
||||
MaxEntries int `json:"max_entries,omitempty"`
|
||||
// GroupV6 is the IPv6 prefix length banned for an address (0: /64,
|
||||
// what one customer usually gets); IPv4 addresses are banned alone.
|
||||
GroupV6 int `json:"group_v6,omitempty"`
|
||||
Comment string `json:"comment,omitempty"`
|
||||
Disabled bool `json:"disabled,omitempty"`
|
||||
}
|
||||
|
||||
func (b *BanList) GetID() int64 { return b.ID }
|
||||
func (b *BanList) Label() string { return b.Name }
|
||||
|
||||
// BanTime is the default ban length in seconds.
|
||||
func (b *BanList) BanTime() int {
|
||||
if b.DefaultTime > 0 {
|
||||
return b.DefaultTime
|
||||
}
|
||||
return DefaultBanTime
|
||||
}
|
||||
|
||||
// BanMaxTime is the longest ban in seconds, 0 for no limit.
|
||||
func (b *BanList) BanMaxTime() int {
|
||||
switch {
|
||||
case b.MaxTime < 0:
|
||||
return 0
|
||||
case b.MaxTime > 0:
|
||||
return b.MaxTime
|
||||
}
|
||||
return DefaultBanMaxTime
|
||||
}
|
||||
|
||||
// Entries is the most bans the list holds.
|
||||
func (b *BanList) Entries() int {
|
||||
if b.MaxEntries > 0 {
|
||||
return b.MaxEntries
|
||||
}
|
||||
return DefaultBanEntries
|
||||
}
|
||||
|
||||
// V6Group is the IPv6 prefix length an address is banned with.
|
||||
func (b *BanList) V6Group() int {
|
||||
if b.GroupV6 > 0 {
|
||||
return b.GroupV6
|
||||
}
|
||||
return DefaultBanGroupV6
|
||||
}
|
||||
|
||||
// Covers reports whether the list's bans apply to a chain.
|
||||
func (b *BanList) Covers(chain string) bool {
|
||||
switch b.Scope {
|
||||
case BanScopeFirewall:
|
||||
return chain == ChainInput
|
||||
case BanScopeHosts:
|
||||
return chain == ChainForward
|
||||
case BanScopeEverything:
|
||||
return chain == ChainInput || chain == ChainForward
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
package model
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestQuotaCarry(t *testing.T) {
|
||||
rule := func(per string, v4, v6 int) *Rule {
|
||||
return &Rule{GroupV4: v4, GroupV6: v6, Quota: &Quota{Bytes: 1, PerSource: per == "src", PerDestination: per == "dst"}}
|
||||
}
|
||||
for _, c := range []struct {
|
||||
old, new *Rule
|
||||
want string
|
||||
}{
|
||||
{rule("", 0, 0), rule("", 0, 0), QuotaKept},
|
||||
{rule("src", 0, 0), rule("src", 32, 64), QuotaKept}, // the defaults, written out
|
||||
{rule("src", 0, 0), rule("", 0, 0), QuotaSummed},
|
||||
{rule("", 0, 0), rule("src", 0, 0), QuotaRestarts},
|
||||
{rule("src", 0, 0), rule("dst", 0, 0), QuotaRestarts},
|
||||
{rule("src", 0, 0), rule("src", 24, 56), QuotaRegrouped},
|
||||
{rule("src", 0, 0), rule("src", 24, 128), QuotaRestarts},
|
||||
{nil, rule("src", 0, 0), QuotaKept},
|
||||
} {
|
||||
if got, _ := QuotaCarry(c.old, c.new); got != c.want {
|
||||
t.Errorf("%+v -> %+v: %q, want %q", c.old, c.new, got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -31,6 +31,15 @@ type Settings struct {
|
||||
// ProbationSeconds is how long a change made over the network has to
|
||||
// be confirmed before it is rolled back (0: the default).
|
||||
ProbationSeconds int `json:"probation_seconds,omitempty"`
|
||||
// ManualConfirm: changes made over the network wait for a person to
|
||||
// confirm them. Otherwise the client that made a change confirms it by
|
||||
// itself once it reaches the API again over a new connection, from the
|
||||
// same address: proof that it can still manage the firewall.
|
||||
ManualConfirm bool `json:"manual_confirm,omitempty"`
|
||||
|
||||
// StatsPerDevice counts forwarded traffic to and from external zones
|
||||
// per inside address (IPv6 per /64), for the statistics' top devices.
|
||||
StatsPerDevice bool `json:"stats_per_device,omitempty"`
|
||||
}
|
||||
|
||||
// Builtins are rules Housecarl adds itself, shown as "added by Housecarl".
|
||||
|
||||
+28
-2
@@ -20,6 +20,7 @@ type State struct {
|
||||
Forwards []*PortForward `json:"forwards"`
|
||||
SNATs []*SourceNAT `json:"snats"`
|
||||
Redirects []*Redirect `json:"redirects"`
|
||||
BanLists []*BanList `json:"ban_lists"`
|
||||
}
|
||||
|
||||
// Object kinds, for changes and the history.
|
||||
@@ -32,10 +33,11 @@ const (
|
||||
KindForward = "forward"
|
||||
KindSNAT = "snat"
|
||||
KindRedirect = "redirect"
|
||||
KindBanList = "banlist"
|
||||
)
|
||||
|
||||
// Kinds lists the object kinds (not settings) in display order.
|
||||
var Kinds = []string{KindZone, KindSet, KindService, KindRule, KindForward, KindSNAT, KindRedirect}
|
||||
var Kinds = []string{KindZone, KindSet, KindService, KindRule, KindForward, KindSNAT, KindRedirect, KindBanList}
|
||||
|
||||
// Object is anything in the State with an ID.
|
||||
type Object interface {
|
||||
@@ -73,6 +75,9 @@ func (st *State) Fill() {
|
||||
if st.Redirects == nil {
|
||||
st.Redirects = []*Redirect{}
|
||||
}
|
||||
if st.BanLists == nil {
|
||||
st.BanLists = []*BanList{}
|
||||
}
|
||||
}
|
||||
|
||||
// Objects returns the objects of a kind.
|
||||
@@ -107,6 +112,10 @@ func (st *State) Objects(kind string) []Object {
|
||||
for _, o := range st.Redirects {
|
||||
out = append(out, o)
|
||||
}
|
||||
case KindBanList:
|
||||
for _, o := range st.BanLists {
|
||||
out = append(out, o)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
@@ -141,6 +150,8 @@ func (st *State) Put(kind string, id int64, data json.RawMessage) error {
|
||||
return put(&st.SNATs, id, data)
|
||||
case KindRedirect:
|
||||
return put(&st.Redirects, id, data)
|
||||
case KindBanList:
|
||||
return put(&st.BanLists, id, data)
|
||||
}
|
||||
return &unknownKind{kind}
|
||||
}
|
||||
@@ -187,6 +198,7 @@ func (st *State) Sort() {
|
||||
sort.SliceStable(st.Zones, func(i, j int) bool { return st.Zones[i].Name < st.Zones[j].Name })
|
||||
sort.SliceStable(st.Sets, func(i, j int) bool { return st.Sets[i].Name < st.Sets[j].Name })
|
||||
sort.SliceStable(st.Services, func(i, j int) bool { return st.Services[i].Name < st.Services[j].Name })
|
||||
sort.SliceStable(st.BanLists, func(i, j int) bool { return st.BanLists[i].Name < st.BanLists[j].Name })
|
||||
sort.SliceStable(st.Rules, func(i, j int) bool {
|
||||
a, b := st.Rules[i], st.Rules[j]
|
||||
if a.Chain != b.Chain {
|
||||
@@ -262,7 +274,7 @@ func (st *State) ByID(kind string, id int64) Object {
|
||||
}
|
||||
|
||||
// Find finds an object of a kind by ID or name (zones, sets, services,
|
||||
// port forwards) or by ID alone (rules, source NAT, redirects).
|
||||
// port forwards, ban lists) or by ID alone (rules, source NAT, redirects).
|
||||
func (st *State) Find(kind, ref string) Object {
|
||||
for _, o := range st.Objects(kind) {
|
||||
if itoa(o.GetID()) == ref {
|
||||
@@ -290,6 +302,20 @@ func (st *State) Find(kind, ref string) Object {
|
||||
return f
|
||||
}
|
||||
}
|
||||
case KindBanList:
|
||||
if b := st.BanList(ref); b != nil {
|
||||
return b
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// BanList finds a ban list by name.
|
||||
func (st *State) BanList(name string) *BanList {
|
||||
for _, b := range st.BanLists {
|
||||
if strings.EqualFold(b.Name, name) {
|
||||
return b
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -0,0 +1,152 @@
|
||||
// Package names finds the names of addresses by reverse lookups, for
|
||||
// showing them beside the addresses (connections, the log, quotas,
|
||||
// traffic per device). Names are cached and never stored: statistics stay
|
||||
// keyed by address.
|
||||
package names
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"net"
|
||||
"net/netip"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Lookups running at once, and the most names kept.
|
||||
const (
|
||||
parallel = 16
|
||||
maxCache = 20000
|
||||
)
|
||||
|
||||
// Resolver looks up and caches addresses' names.
|
||||
type Resolver struct {
|
||||
lookup func(ctx context.Context, addr string) ([]string, error)
|
||||
// ttl is how long a name (or its absence) is kept; a lookup that
|
||||
// failed (a timeout, a server error) is tried again after retry.
|
||||
ttl, retry time.Duration
|
||||
timeout time.Duration
|
||||
now func() time.Time
|
||||
|
||||
mu sync.Mutex
|
||||
cache map[netip.Addr]entry
|
||||
pending map[netip.Addr]chan struct{}
|
||||
sem chan struct{}
|
||||
}
|
||||
|
||||
type entry struct {
|
||||
name string
|
||||
expires time.Time
|
||||
}
|
||||
|
||||
// New makes a resolver asking server ("10.10.0.1:53"; "" for the
|
||||
// system's, /etc/hosts then /etc/resolv.conf), keeping names for ttl.
|
||||
func New(server string, ttl time.Duration) *Resolver {
|
||||
res := &net.Resolver{PreferGo: true}
|
||||
if server != "" {
|
||||
res.Dial = func(ctx context.Context, network, _ string) (net.Conn, error) {
|
||||
var d net.Dialer
|
||||
return d.DialContext(ctx, network, server)
|
||||
}
|
||||
}
|
||||
return NewFunc(res.LookupAddr, ttl)
|
||||
}
|
||||
|
||||
// NewFunc makes a resolver with its own lookup (tests).
|
||||
func NewFunc(lookup func(ctx context.Context, addr string) ([]string, error), ttl time.Duration) *Resolver {
|
||||
return &Resolver{lookup: lookup, ttl: ttl, retry: time.Minute, timeout: 2 * time.Second, now: time.Now,
|
||||
cache: map[netip.Addr]entry{}, pending: map[netip.Addr]chan struct{}{}, sem: make(chan struct{}, parallel)}
|
||||
}
|
||||
|
||||
// Names returns the names of addrs: those cached, and those looked up
|
||||
// within wait. Lookups still running then carry on and are cached for
|
||||
// the next call. An address without a name is left out.
|
||||
func (r *Resolver) Names(ctx context.Context, addrs []netip.Addr, wait time.Duration) map[netip.Addr]string {
|
||||
out := map[netip.Addr]string{}
|
||||
var waiting []netip.Addr
|
||||
var done []chan struct{}
|
||||
now := r.now()
|
||||
r.mu.Lock()
|
||||
for _, a := range addrs {
|
||||
if e, ok := r.cache[a]; ok && now.Before(e.expires) {
|
||||
if e.name != "" {
|
||||
out[a] = e.name
|
||||
}
|
||||
continue
|
||||
}
|
||||
ch, ok := r.pending[a]
|
||||
if !ok {
|
||||
ch = make(chan struct{})
|
||||
r.pending[a] = ch
|
||||
go r.resolve(a, ch)
|
||||
}
|
||||
waiting = append(waiting, a)
|
||||
done = append(done, ch)
|
||||
}
|
||||
r.mu.Unlock()
|
||||
if len(waiting) == 0 {
|
||||
return out
|
||||
}
|
||||
t := time.NewTimer(wait)
|
||||
defer t.Stop()
|
||||
for _, ch := range done {
|
||||
select {
|
||||
case <-ch:
|
||||
case <-t.C:
|
||||
return r.collect(out, waiting)
|
||||
case <-ctx.Done():
|
||||
return r.collect(out, waiting)
|
||||
}
|
||||
}
|
||||
return r.collect(out, waiting)
|
||||
}
|
||||
|
||||
// collect adds what the lookups found for addrs.
|
||||
func (r *Resolver) collect(out map[netip.Addr]string, addrs []netip.Addr) map[netip.Addr]string {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
for _, a := range addrs {
|
||||
if e, ok := r.cache[a]; ok && e.name != "" {
|
||||
out[a] = e.name
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func (r *Resolver) resolve(a netip.Addr, done chan struct{}) {
|
||||
r.sem <- struct{}{}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), r.timeout)
|
||||
ns, err := r.lookup(ctx, a.String())
|
||||
cancel()
|
||||
<-r.sem
|
||||
e := entry{expires: r.now().Add(r.ttl)}
|
||||
var dnsErr *net.DNSError
|
||||
switch {
|
||||
case err == nil && len(ns) > 0:
|
||||
e.name = strings.TrimSuffix(ns[0], ".")
|
||||
case err != nil && !(errors.As(err, &dnsErr) && dnsErr.IsNotFound):
|
||||
e.expires = r.now().Add(r.retry)
|
||||
}
|
||||
r.mu.Lock()
|
||||
if len(r.cache) >= maxCache {
|
||||
r.prune()
|
||||
}
|
||||
r.cache[a] = e
|
||||
delete(r.pending, a)
|
||||
r.mu.Unlock()
|
||||
close(done)
|
||||
}
|
||||
|
||||
// prune drops expired names, or all of them when none has expired.
|
||||
func (r *Resolver) prune() {
|
||||
now := r.now()
|
||||
for a, e := range r.cache {
|
||||
if !now.Before(e.expires) {
|
||||
delete(r.cache, a)
|
||||
}
|
||||
}
|
||||
if len(r.cache) >= maxCache {
|
||||
clear(r.cache)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,57 @@
|
||||
package names
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"net"
|
||||
"net/netip"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestNames(t *testing.T) {
|
||||
var calls atomic.Int32
|
||||
slow := make(chan struct{})
|
||||
r := NewFunc(func(ctx context.Context, addr string) ([]string, error) {
|
||||
calls.Add(1)
|
||||
switch addr {
|
||||
case "10.0.0.1":
|
||||
return []string{"nas.example.net."}, nil
|
||||
case "10.0.0.2":
|
||||
return nil, &net.DNSError{Err: "no such host", Name: addr, IsNotFound: true}
|
||||
case "10.0.0.3":
|
||||
return nil, errors.New("server failure")
|
||||
case "10.0.0.4":
|
||||
<-slow
|
||||
return []string{"slow.example.net."}, nil
|
||||
}
|
||||
return nil, nil
|
||||
}, time.Hour)
|
||||
a := netip.MustParseAddr
|
||||
all := []netip.Addr{a("10.0.0.1"), a("10.0.0.2"), a("10.0.0.3"), a("10.0.0.4")}
|
||||
|
||||
got := r.Names(context.Background(), all, 200*time.Millisecond)
|
||||
if len(got) != 1 || got[a("10.0.0.1")] != "nas.example.net" {
|
||||
t.Fatalf("first call: %v", got)
|
||||
}
|
||||
// The slow lookup finishes later and is there next time; the others
|
||||
// come from the cache, the failure too until it is retried.
|
||||
close(slow)
|
||||
time.Sleep(50 * time.Millisecond)
|
||||
got = r.Names(context.Background(), all, 200*time.Millisecond)
|
||||
if len(got) != 2 || got[a("10.0.0.4")] != "slow.example.net" {
|
||||
t.Fatalf("second call: %v", got)
|
||||
}
|
||||
if n := calls.Load(); n != 4 {
|
||||
t.Fatalf("%d lookups, want 4 (cached)", n)
|
||||
}
|
||||
|
||||
// A failure is tried again after retry, a missing name after ttl.
|
||||
now := time.Now().Add(2 * time.Minute)
|
||||
r.now = func() time.Time { return now }
|
||||
r.Names(context.Background(), all, 200*time.Millisecond)
|
||||
if n := calls.Load(); n != 5 {
|
||||
t.Fatalf("%d lookups, want 5 (the failure again)", n)
|
||||
}
|
||||
}
|
||||
+479
-10
@@ -8,11 +8,14 @@
|
||||
package nft
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net"
|
||||
"net/netip"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
@@ -24,6 +27,7 @@ import (
|
||||
|
||||
"housecarl/internal/model"
|
||||
"housecarl/internal/render"
|
||||
"housecarl/internal/trace"
|
||||
)
|
||||
|
||||
// Runner runs nft.
|
||||
@@ -186,6 +190,30 @@ func (r *Runner) DeleteElement(ctx context.Context, set, addr string) error {
|
||||
return err
|
||||
}
|
||||
|
||||
// AddBan puts a ban in force: added, or if it is there already, given
|
||||
// its new time (nftables keeps an element's old timeout when it is added
|
||||
// again, so it is added, deleted and added in one transaction).
|
||||
func (r *Runner) AddBan(ctx context.Context, b render.Ban) error {
|
||||
set := render.BanSet(b.List, b.Addr.Addr().Is6(), b.Proto != "")
|
||||
e := render.BanElement(b)
|
||||
t := ""
|
||||
if b.Timeout > 0 {
|
||||
t = fmt.Sprintf(" timeout %ds", b.Timeout)
|
||||
}
|
||||
script := fmt.Sprintf("add element inet %[1]s %[2]s { %[3]s }\ndelete element inet %[1]s %[2]s { %[3]s }\nadd element inet %[1]s %[2]s { %[3]s%[4]s }\n", render.Table, set, e, t)
|
||||
_, err := r.run(ctx, script, "-f", "/dev/stdin")
|
||||
return err
|
||||
}
|
||||
|
||||
// DeleteBan lifts a ban. One already gone (it ran out) isn't an error.
|
||||
func (r *Runner) DeleteBan(ctx context.Context, b render.Ban) error {
|
||||
set := render.BanSet(b.List, b.Addr.Addr().Is6(), b.Proto != "")
|
||||
e := render.BanElement(b)
|
||||
script := fmt.Sprintf("add element inet %[1]s %[2]s { %[3]s }\ndelete element inet %[1]s %[2]s { %[3]s }\n", render.Table, set, e)
|
||||
_, err := r.run(ctx, script, "-f", "/dev/stdin")
|
||||
return err
|
||||
}
|
||||
|
||||
// ---- reading JSON
|
||||
|
||||
// Table is what Housecarl reads back from its table: counters by mark
|
||||
@@ -196,11 +224,62 @@ type Table struct {
|
||||
Counters map[string]Counter
|
||||
// Elements by set name (both families together).
|
||||
Elements map[string][]render.Element
|
||||
// Quotas: the rules' quotas' used bytes (quota_r12), and the
|
||||
// per-address quotas' entries by set (quota_r12_v4).
|
||||
QuotaUsage
|
||||
// DeviceSets are the per-device traffic counters by set
|
||||
// (stats_up_v4, stats_down_v6 …): each address's packets and bytes.
|
||||
DeviceSets map[string][]render.Element
|
||||
// Since is when the counters started (housecarld's first apply).
|
||||
Since time.Time
|
||||
// Tables are all the machine's tables ("inet main"), when the whole
|
||||
// ruleset was read.
|
||||
Tables []string
|
||||
// Fallback: the table is the boot fallback (render.Fallback), not a
|
||||
// ruleset housecarld applied.
|
||||
Fallback bool
|
||||
}
|
||||
|
||||
// QuotaUsage is what the quotas used: kept across reloads, and saved in
|
||||
// the database so a reboot doesn't reset it.
|
||||
type QuotaUsage struct {
|
||||
Quotas map[string]int64 `json:"quotas,omitempty"`
|
||||
QuotaSets map[string][]render.Element `json:"quota_sets,omitempty"`
|
||||
}
|
||||
|
||||
// merge takes the larger usage of each quota and entry from o (what was
|
||||
// saved before a reboot, when the kernel's count started again).
|
||||
func (u *QuotaUsage) merge(o *QuotaUsage) {
|
||||
if o == nil {
|
||||
return
|
||||
}
|
||||
if u.Quotas == nil {
|
||||
u.Quotas = map[string]int64{}
|
||||
}
|
||||
if u.QuotaSets == nil {
|
||||
u.QuotaSets = map[string][]render.Element{}
|
||||
}
|
||||
for k, v := range o.Quotas {
|
||||
if v > u.Quotas[k] {
|
||||
u.Quotas[k] = v
|
||||
}
|
||||
}
|
||||
for set, es := range o.QuotaSets {
|
||||
for _, e := range es {
|
||||
found := false
|
||||
for i := range u.QuotaSets[set] {
|
||||
if u.QuotaSets[set][i].Addr == e.Addr {
|
||||
found = true
|
||||
if e.Used > u.QuotaSets[set][i].Used {
|
||||
u.QuotaSets[set][i].Used = e.Used
|
||||
}
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
u.QuotaSets[set] = append(u.QuotaSets[set], e)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Counter is packets and bytes.
|
||||
@@ -223,9 +302,25 @@ type jsonSet struct {
|
||||
Elem []json.RawMessage `json:"elem"`
|
||||
}
|
||||
|
||||
// jsonQuota is a quota's state: a named quota ({"bytes", "used"}, in
|
||||
// bytes) or one in an element ({"val", "val_unit", "used", "used_unit"}).
|
||||
type jsonQuota struct {
|
||||
Name string `json:"name"`
|
||||
Used int64 `json:"used"`
|
||||
UsedUnit string `json:"used_unit"`
|
||||
}
|
||||
|
||||
func (q jsonQuota) bytes() int64 {
|
||||
if m, ok := model.ByteUnits[q.UsedUnit]; ok {
|
||||
return q.Used * m
|
||||
}
|
||||
return q.Used
|
||||
}
|
||||
|
||||
// ParseTable reads `nft -j list table inet housecarl`.
|
||||
func ParseTable(data []byte) (*Table, error) {
|
||||
t := &Table{Counters: map[string]Counter{}, Elements: map[string][]render.Element{}}
|
||||
t := &Table{Counters: map[string]Counter{}, Elements: map[string][]render.Element{}, DeviceSets: map[string][]render.Element{},
|
||||
QuotaUsage: QuotaUsage{Quotas: map[string]int64{}, QuotaSets: map[string][]render.Element{}}}
|
||||
if len(data) == 0 {
|
||||
return t, nil
|
||||
}
|
||||
@@ -240,6 +335,9 @@ func ParseTable(data []byte) (*Table, error) {
|
||||
continue
|
||||
}
|
||||
mark, _, _ := strings.Cut(strings.TrimPrefix(r.Comment, "hc:"), " ")
|
||||
if strings.HasPrefix(mark, render.FallbackMark+"-") {
|
||||
t.Fallback = true
|
||||
}
|
||||
for _, e := range r.Expr {
|
||||
var c struct {
|
||||
Counter *Counter `json:"counter"`
|
||||
@@ -252,13 +350,48 @@ func ParseTable(data []byte) (*Table, error) {
|
||||
}
|
||||
}
|
||||
}
|
||||
if raw, ok := item["quota"]; ok {
|
||||
var q jsonQuota
|
||||
if json.Unmarshal(raw, &q) == nil && strings.HasPrefix(q.Name, "quota_r") {
|
||||
t.Quotas[q.Name] = q.bytes()
|
||||
}
|
||||
}
|
||||
if raw, ok := item["set"]; ok {
|
||||
var s jsonSet
|
||||
if json.Unmarshal(raw, &s) != nil {
|
||||
continue
|
||||
}
|
||||
if strings.HasPrefix(s.Name, "quota_r") {
|
||||
for _, e := range s.Elem {
|
||||
var w struct {
|
||||
Elem struct {
|
||||
Val string `json:"val"`
|
||||
Quota *jsonQuota `json:"quota"`
|
||||
} `json:"elem"`
|
||||
}
|
||||
if json.Unmarshal(e, &w) == nil && w.Elem.Quota != nil {
|
||||
t.QuotaSets[s.Name] = append(t.QuotaSets[s.Name], render.Element{Addr: w.Elem.Val, Used: w.Elem.Quota.bytes()})
|
||||
}
|
||||
}
|
||||
continue
|
||||
}
|
||||
if render.IsDeviceSet(s.Name) {
|
||||
for _, e := range s.Elem {
|
||||
var w struct {
|
||||
Elem struct {
|
||||
Val string `json:"val"`
|
||||
Expires int `json:"expires"`
|
||||
Counter *Counter `json:"counter"`
|
||||
} `json:"elem"`
|
||||
}
|
||||
if json.Unmarshal(e, &w) == nil && w.Elem.Counter != nil {
|
||||
t.DeviceSets[s.Name] = append(t.DeviceSets[s.Name], render.Element{Addr: w.Elem.Val, Timeout: w.Elem.Expires, Packets: w.Elem.Counter.Packets, Bytes: w.Elem.Counter.Bytes})
|
||||
}
|
||||
}
|
||||
continue
|
||||
}
|
||||
name := strings.TrimSuffix(strings.TrimSuffix(s.Name, "_v4"), "_v6")
|
||||
if name == s.Name || strings.HasPrefix(s.Name, "meter_") || strings.HasPrefix(s.Name, "conn_") {
|
||||
if name == s.Name || strings.HasPrefix(s.Name, "meter_") || strings.HasPrefix(s.Name, "conn_") || render.IsBanSet(s.Name) {
|
||||
continue
|
||||
}
|
||||
for _, e := range s.Elem {
|
||||
@@ -301,16 +434,201 @@ type Firewall struct {
|
||||
// BootFile is where the confirmed ruleset is saved for the boot unit
|
||||
// (housecarl-ruleset.service; "" for none).
|
||||
BootFile string
|
||||
// Shadow is shadow mode: the ruleset changes nothing, beside another
|
||||
// firewall (render.Options; docs/shadow.md). ShadowBefore runs its
|
||||
// filter chains before that one's.
|
||||
Shadow, ShadowBefore bool
|
||||
|
||||
// Replacing the table resets its counters: base keeps the totals of
|
||||
// the tables replaced since housecarld started.
|
||||
mu sync.Mutex
|
||||
base map[string]Counter
|
||||
since time.Time
|
||||
// restore is the quota usage saved before housecarld (or the machine)
|
||||
// stopped, put back by the next apply.
|
||||
restore *QuotaUsage
|
||||
// adjust are quota changes the next apply makes (AdjustQuota).
|
||||
adjust []QuotaAdjust
|
||||
// quotaRules are the rules with a quota in the ruleset in force, by
|
||||
// ID: what their usage was counted by (nil before the first apply).
|
||||
quotaRules map[int64]*model.Rule
|
||||
// Bans returns the active bans, which every apply puts back (the
|
||||
// database keeps them; nil: none).
|
||||
Bans func() []render.Ban
|
||||
// flowDevs are the flowtable's devices in the ruleset in force, by
|
||||
// name, with their interface index then (0: left out, missing).
|
||||
flowDevs map[string]int
|
||||
}
|
||||
|
||||
// ifIndex is an interface's index, 0 if it doesn't exist (tests change it).
|
||||
var ifIndex = func(name string) int {
|
||||
if i, err := net.InterfaceByName(name); err == nil {
|
||||
return i.Index
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
// FlowtableStale reports whether the flowtable in force no longer matches
|
||||
// the machine: a device of st's flowtable appeared, went, or came back as
|
||||
// a new interface (ppp0 after PPPoE reconnects; the kernel drops a
|
||||
// device from the flowtable when it goes). Applying again fixes it.
|
||||
func (f *Firewall) FlowtableStale(st *model.State) bool {
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
if f.flowDevs == nil {
|
||||
return false
|
||||
}
|
||||
for _, d := range st.Settings.FlowtableDevices {
|
||||
if ifIndex(d) != f.flowDevs[d] {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// RestoreQuotas has the next apply put this saved quota usage back where
|
||||
// the kernel's count is lower (after a reboot, the boot ruleset starts
|
||||
// every quota at zero).
|
||||
func (f *Firewall) RestoreQuotas(u *QuotaUsage) {
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
f.restore = u
|
||||
}
|
||||
|
||||
// QuotaAdjust takes Give bytes off what a quota used (never below zero):
|
||||
// the named quota Name, or the entries of the per-address sets Name_v4
|
||||
// and Name_v6, Addr's alone if given (the set's key, as listed).
|
||||
type QuotaAdjust struct {
|
||||
Name string
|
||||
Addr string
|
||||
Give int64
|
||||
}
|
||||
|
||||
// AdjustQuota has the next apply change what a quota used.
|
||||
func (f *Firewall) AdjustQuota(a QuotaAdjust) {
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
f.adjust = append(f.adjust, a)
|
||||
}
|
||||
|
||||
// apply takes what a's say off u. An entry with nothing left used is
|
||||
// dropped, so its quota starts again with its next packet.
|
||||
func (u *QuotaUsage) apply(adj []QuotaAdjust) {
|
||||
less := func(used, give int64) int64 {
|
||||
if give >= used {
|
||||
return 0
|
||||
}
|
||||
return used - give
|
||||
}
|
||||
for _, a := range adj {
|
||||
if used, ok := u.Quotas[a.Name]; ok {
|
||||
u.Quotas[a.Name] = less(used, a.Give)
|
||||
}
|
||||
for _, set := range []string{a.Name + "_v4", a.Name + "_v6"} {
|
||||
var kept []render.Element
|
||||
for _, e := range u.QuotaSets[set] {
|
||||
if a.Addr == "" || e.Addr == a.Addr {
|
||||
e.Used = less(e.Used, a.Give)
|
||||
}
|
||||
if e.Used > 0 {
|
||||
kept = append(kept, e)
|
||||
}
|
||||
}
|
||||
if _, ok := u.QuotaSets[set]; ok {
|
||||
u.QuotaSets[set] = kept
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// carry keeps what quotas used when their rules now count differently
|
||||
// (model.QuotaCarry): per address to a total, the sum; to wider groups,
|
||||
// the groups' sums; what can't be carried starts again. was are the
|
||||
// rules in force before (nil: nothing to compare with).
|
||||
func (u *QuotaUsage) carry(was map[int64]*model.Rule, st *model.State) {
|
||||
for _, r := range st.Rules {
|
||||
old := was[r.ID]
|
||||
carry, _ := model.QuotaCarry(old, r)
|
||||
if carry == model.QuotaKept {
|
||||
continue
|
||||
}
|
||||
name := render.QuotaName(r.ID)
|
||||
sets := []string{name + "_v4", name + "_v6"}
|
||||
switch carry {
|
||||
case model.QuotaSummed:
|
||||
var sum int64
|
||||
for _, set := range sets {
|
||||
for _, e := range u.QuotaSets[set] {
|
||||
sum += e.Used
|
||||
}
|
||||
}
|
||||
if u.Quotas == nil {
|
||||
u.Quotas = map[string]int64{}
|
||||
}
|
||||
u.Quotas[name] = sum
|
||||
case model.QuotaRegrouped:
|
||||
v4, v6 := r.Groups()
|
||||
for _, set := range sets {
|
||||
byKey := map[string]int64{}
|
||||
var order []string
|
||||
for _, e := range u.QuotaSets[set] {
|
||||
a, err := netip.ParseAddr(e.Addr)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
bits := v4
|
||||
if a.Is6() {
|
||||
bits = v6
|
||||
}
|
||||
p, _ := a.Prefix(bits)
|
||||
k := p.Addr().String()
|
||||
if _, ok := byKey[k]; !ok {
|
||||
order = append(order, k)
|
||||
}
|
||||
byKey[k] += e.Used
|
||||
}
|
||||
if len(order) == 0 {
|
||||
continue
|
||||
}
|
||||
var es []render.Element
|
||||
for _, k := range order {
|
||||
es = append(es, render.Element{Addr: k, Used: byKey[k]})
|
||||
}
|
||||
u.QuotaSets[set] = es
|
||||
}
|
||||
continue
|
||||
}
|
||||
// What was counted the other way goes: a total, or the entries.
|
||||
if carry != model.QuotaSummed {
|
||||
delete(u.Quotas, name)
|
||||
}
|
||||
for _, set := range sets {
|
||||
delete(u.QuotaSets, set)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ResetQuota starts a rule's quota again: the named quota, or the
|
||||
// per-address quota sets (both families).
|
||||
func (f *Firewall) ResetQuota(ctx context.Context, ruleID int64, perAddress bool) error {
|
||||
name := render.QuotaName(ruleID)
|
||||
if !perAddress {
|
||||
_, err := f.Runner.run(ctx, "", "reset", "quota", "inet", render.Table, name)
|
||||
return err
|
||||
}
|
||||
for _, fam := range []string{"_v4", "_v6"} {
|
||||
if _, err := f.Runner.run(ctx, "", "flush", "set", "inet", render.Table, name+fam); err != nil {
|
||||
if e, ok := err.(*Error); ok && strings.Contains(e.Output, "No such file or directory") {
|
||||
continue // the rule has no such family
|
||||
}
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (f *Firewall) options() render.Options {
|
||||
var o render.Options
|
||||
o := render.Options{Shadow: f.Shadow, ShadowBefore: f.ShadowBefore}
|
||||
if f.Interfaces != nil {
|
||||
o.Interfaces = f.Interfaces()
|
||||
}
|
||||
@@ -320,9 +638,26 @@ func (f *Firewall) options() render.Options {
|
||||
// Render builds the ruleset for st as Apply would.
|
||||
func (f *Firewall) Render(st *model.State) *render.Ruleset { return render.Render(st, f.options()) }
|
||||
|
||||
// Check implements engine.Firewall.
|
||||
// RenderEnforcing builds the ruleset for st as Apply would outside shadow
|
||||
// mode.
|
||||
func (f *Firewall) RenderEnforcing(st *model.State) *render.Ruleset {
|
||||
o := f.options()
|
||||
o.Shadow, o.ShadowBefore = false, false
|
||||
return render.Render(st, o)
|
||||
}
|
||||
|
||||
// Check implements engine.Firewall. In shadow mode it checks the
|
||||
// enforcing ruleset too, so that leaving shadow mode can't fail.
|
||||
func (f *Firewall) Check(ctx context.Context, st *model.State) error {
|
||||
return f.Runner.Check(ctx, render.Render(st, f.options()))
|
||||
if err := f.Runner.Check(ctx, render.Render(st, f.options())); err != nil {
|
||||
return err
|
||||
}
|
||||
if f.Shadow {
|
||||
if err := f.Runner.Check(ctx, f.RenderEnforcing(st)); err != nil {
|
||||
return fmt.Errorf("the ruleset for when shadow mode is off: %w", err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Apply implements engine.Firewall.
|
||||
@@ -337,9 +672,39 @@ func (f *Firewall) Apply(ctx context.Context, st *model.State) error {
|
||||
old = t
|
||||
}
|
||||
}
|
||||
usage := QuotaUsage{}
|
||||
if old != nil {
|
||||
usage = old.QuotaUsage
|
||||
}
|
||||
usage.merge(f.restore)
|
||||
usage.carry(f.quotaRules, st)
|
||||
usage.apply(f.adjust)
|
||||
f.adjust = nil
|
||||
opt.Quotas, opt.QuotaSets = usage.Quotas, usage.QuotaSets
|
||||
if f.Bans != nil {
|
||||
opt.Bans = f.Bans()
|
||||
}
|
||||
if old != nil {
|
||||
opt.DeviceSets = old.DeviceSets
|
||||
}
|
||||
if err := f.Runner.Apply(ctx, render.Render(st, opt)); err != nil {
|
||||
return err
|
||||
}
|
||||
f.restore = nil
|
||||
f.quotaRules = map[int64]*model.Rule{}
|
||||
for _, r := range st.Rules {
|
||||
if r.Quota != nil {
|
||||
c, q := *r, *r.Quota // only the quota and the grouping are read
|
||||
c.Quota = &q
|
||||
f.quotaRules[r.ID] = &c
|
||||
}
|
||||
}
|
||||
f.flowDevs = map[string]int{}
|
||||
for _, d := range st.Settings.FlowtableDevices {
|
||||
if opt.Interfaces == nil || opt.Interfaces[d] {
|
||||
f.flowDevs[d] = ifIndex(d)
|
||||
}
|
||||
}
|
||||
if f.base == nil {
|
||||
f.base, f.since = map[string]Counter{}, time.Now()
|
||||
}
|
||||
@@ -351,23 +716,84 @@ func (f *Firewall) Apply(ctx context.Context, st *model.State) error {
|
||||
f.base[k] = b
|
||||
}
|
||||
}
|
||||
pruneCounters(f.base, st)
|
||||
return nil
|
||||
}
|
||||
|
||||
// SaveBoot writes the ruleset of st for the boot unit, atomically.
|
||||
// pruneCounters forgets the totals of objects st no longer has (deleted,
|
||||
// or replaced by an import with new IDs), so they aren't listed beside
|
||||
// the current ones. A disabled object keeps its totals for when it comes
|
||||
// back; built-ins and policies are always kept.
|
||||
func pruneCounters(base map[string]Counter, st *model.State) {
|
||||
ids := map[string]bool{}
|
||||
for _, r := range st.Rules {
|
||||
ids[fmt.Sprintf("r%d", r.ID)] = true
|
||||
}
|
||||
for _, x := range st.Forwards {
|
||||
ids[fmt.Sprintf("f%d", x.ID)] = true
|
||||
}
|
||||
for _, x := range st.SNATs {
|
||||
ids[fmt.Sprintf("s%d", x.ID)] = true
|
||||
}
|
||||
for _, x := range st.Redirects {
|
||||
ids[fmt.Sprintf("d%d", x.ID)] = true
|
||||
}
|
||||
for _, x := range st.BanLists {
|
||||
ids[render.BanMark(x.ID)] = true
|
||||
}
|
||||
for mark := range base {
|
||||
if strings.HasPrefix(mark, "ban") && strings.Trim(mark[3:], "0123456789") == "" {
|
||||
if !ids[mark] {
|
||||
delete(base, mark)
|
||||
}
|
||||
continue
|
||||
}
|
||||
obj, _, _ := strings.Cut(mark, "-") // f3-hairpin, f3-6 are f3's
|
||||
if len(obj) < 2 || !strings.ContainsRune("rfsd", rune(obj[0])) || strings.Trim(obj[1:], "0123456789") != "" {
|
||||
continue // b-established, p-input
|
||||
}
|
||||
if !ids[obj] {
|
||||
delete(base, mark)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// SaveBoot writes the ruleset of st for the boot unit, and the fallback
|
||||
// it loads if that fails (FallbackFile), atomically. The boot ruleset has
|
||||
// no flowtable: it loads before the network comes up, when its devices
|
||||
// (ppp0, wg0) don't exist yet and nftables would refuse the whole file;
|
||||
// housecarld adds it once they do. In shadow mode both are shadow mode's:
|
||||
// the fallback then filters nothing.
|
||||
func (f *Firewall) SaveBoot(st *model.State) error {
|
||||
if f.BootFile == "" {
|
||||
return nil
|
||||
}
|
||||
rs := render.Render(st, f.options())
|
||||
tmp := f.BootFile + ".tmp"
|
||||
if err := os.MkdirAll(filepath.Dir(f.BootFile), 0o750); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := os.WriteFile(tmp, []byte(rs.Text), 0o640); err != nil {
|
||||
fallback := render.Fallback(st)
|
||||
if f.Shadow {
|
||||
fallback = render.ShadowFallback()
|
||||
}
|
||||
if err := writeAtomic(FallbackFile(f.BootFile), fallback); err != nil {
|
||||
return err
|
||||
}
|
||||
return os.Rename(tmp, f.BootFile)
|
||||
rs := render.Render(st, render.Options{Interfaces: map[string]bool{}, Shadow: f.Shadow, ShadowBefore: f.ShadowBefore})
|
||||
return writeAtomic(f.BootFile, rs.Text)
|
||||
}
|
||||
|
||||
// FallbackFile is where the fallback for a boot ruleset is saved:
|
||||
// ruleset.nft's is ruleset-fallback.nft beside it.
|
||||
func FallbackFile(boot string) string {
|
||||
return strings.TrimSuffix(boot, ".nft") + "-fallback.nft"
|
||||
}
|
||||
|
||||
func writeAtomic(path, text string) error {
|
||||
tmp := path + ".tmp"
|
||||
if err := os.WriteFile(tmp, []byte(text), 0o640); err != nil {
|
||||
return err
|
||||
}
|
||||
return os.Rename(tmp, path)
|
||||
}
|
||||
|
||||
// Table reads Housecarl's table back, with counters totalled since
|
||||
@@ -404,6 +830,14 @@ func (f *Firewall) AddElement(ctx context.Context, set, addr string, timeout int
|
||||
return f.Runner.AddElement(ctx, set, addr, timeout)
|
||||
}
|
||||
|
||||
// AddBan puts a ban in force (Runner.AddBan).
|
||||
func (f *Firewall) AddBan(ctx context.Context, b render.Ban) error { return f.Runner.AddBan(ctx, b) }
|
||||
|
||||
// DeleteBan lifts a ban (Runner.DeleteBan).
|
||||
func (f *Firewall) DeleteBan(ctx context.Context, b render.Ban) error {
|
||||
return f.Runner.DeleteBan(ctx, b)
|
||||
}
|
||||
|
||||
// DeleteElement removes an entry from a dynamic set.
|
||||
func (f *Firewall) DeleteElement(ctx context.Context, set, addr string) error {
|
||||
return f.Runner.DeleteElement(ctx, set, addr)
|
||||
@@ -422,6 +856,41 @@ func (f *Firewall) ApplyScript(ctx context.Context, script string) error {
|
||||
return f.Runner.ApplyScript(ctx, script)
|
||||
}
|
||||
|
||||
// Trace follows packets: it starts `nft monitor trace`, puts the tracing
|
||||
// table (script) in, and passes each line to line until ctx ends; then
|
||||
// it takes the table out again, whatever happened.
|
||||
func (f *Firewall) Trace(ctx context.Context, script string, line func(string)) error {
|
||||
c := f.Runner.cmd(ctx, "monitor", "trace")
|
||||
out, err := c.StdoutPipe()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var errb bytes.Buffer
|
||||
c.Stderr = &errb
|
||||
if err := c.Start(); err != nil {
|
||||
return err
|
||||
}
|
||||
defer f.Runner.run(context.Background(), "", "delete", "table", "inet", trace.Table)
|
||||
if err := f.Runner.ApplyScript(ctx, script); err != nil {
|
||||
c.Process.Kill()
|
||||
c.Wait()
|
||||
return fmt.Errorf("starting the trace: %w", err)
|
||||
}
|
||||
sc := bufio.NewScanner(out)
|
||||
sc.Buffer(make([]byte, 64<<10), 1<<20)
|
||||
for sc.Scan() {
|
||||
line(sc.Text())
|
||||
}
|
||||
err = c.Wait()
|
||||
if ctx.Err() != nil {
|
||||
return nil // stopped, or its time was up
|
||||
}
|
||||
if err != nil && errb.Len() > 0 {
|
||||
return fmt.Errorf("nft monitor trace: %s", strings.TrimSpace(errb.String()))
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
// DeleteTable deletes another table (takeover).
|
||||
func (f *Firewall) DeleteTable(ctx context.Context, family, name string) error {
|
||||
if name == render.Table && family == "inet" {
|
||||
|
||||
@@ -2,8 +2,13 @@ package nft
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net/netip"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"sort"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
@@ -125,3 +130,480 @@ func TestFirewallInContainer(t *testing.T) {
|
||||
t.Errorf("another table was touched: %s", out)
|
||||
}
|
||||
}
|
||||
|
||||
// TestQuotasInContainer: quotas keep what they used across a reload,
|
||||
// reset by hand, and come back after a reboot from what was saved.
|
||||
func TestQuotasInContainer(t *testing.T) {
|
||||
image := os.Getenv("HOUSECARL_NFT_IMAGE")
|
||||
if image == "" {
|
||||
t.Skip("HOUSECARL_NFT_IMAGE not set")
|
||||
}
|
||||
name := "housecarl-nft-quota-test"
|
||||
exec.Command("docker", "rm", "-f", name).Run()
|
||||
if out, err := exec.Command("docker", "run", "-d", "--name", name, "--cap-add", "NET_ADMIN", image, "sleep", "300").CombinedOutput(); err != nil {
|
||||
t.Fatalf("starting the container: %v %s", err, out)
|
||||
}
|
||||
t.Cleanup(func() { exec.Command("docker", "rm", "-f", name).Run() })
|
||||
fw := &Firewall{Runner: &Runner{Command: []string{"docker", "exec", "-i", name, "nft"}}}
|
||||
ctx := context.Background()
|
||||
st := model.NewState()
|
||||
// A total quota on pings to 127.0.0.2, and one per destination on
|
||||
// pings to 127.0.0.3.
|
||||
st.Rules = []*model.Rule{
|
||||
{ID: 1, Chain: model.ChainOutput, Position: 1, Action: model.ActionDrop, Match: model.Match{Destinations: []string{"127.0.0.2"}},
|
||||
Quota: &model.Quota{Bytes: 10 << 20, Over: true}},
|
||||
{ID: 2, Chain: model.ChainOutput, Position: 2, Action: model.ActionDrop, Match: model.Match{Destinations: []string{"127.0.0.3"}},
|
||||
Quota: &model.Quota{Bytes: 10 << 20, Over: true, PerDestination: true}},
|
||||
}
|
||||
if err := fw.Apply(ctx, st); err != nil {
|
||||
t.Fatalf("apply: %v", err)
|
||||
}
|
||||
ping := func(addr string) {
|
||||
exec.Command("docker", "exec", name, "ping", "-c", "3", "-s", "1000", "-W", "1", addr).Run()
|
||||
}
|
||||
ping("127.0.0.2")
|
||||
ping("127.0.0.3")
|
||||
used := func() (int64, int64) {
|
||||
tb, err := fw.Table(ctx)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var dev int64
|
||||
for _, e := range tb.QuotaSets["quota_r2_v4"] {
|
||||
if e.Addr == "127.0.0.3" {
|
||||
dev = e.Used
|
||||
}
|
||||
}
|
||||
return tb.Quotas["quota_r1"], dev
|
||||
}
|
||||
total, dev := used()
|
||||
if total < 3000 || dev < 3000 {
|
||||
t.Fatalf("used: total %d, per destination %d", total, dev)
|
||||
}
|
||||
// A reload (a change) keeps what was used.
|
||||
st.Rules[0].Comment = "changed"
|
||||
if err := fw.Apply(ctx, st); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if t2, d2 := used(); t2 < total || d2 < dev {
|
||||
t.Errorf("after a reload: total %d (was %d), per destination %d (was %d)", t2, total, d2, dev)
|
||||
}
|
||||
// Some given back, and one address started again, by the next apply.
|
||||
total, dev = used()
|
||||
fw.AdjustQuota(QuotaAdjust{Name: "quota_r1", Give: 1000})
|
||||
fw.AdjustQuota(QuotaAdjust{Name: "quota_r2", Addr: "127.0.0.3", Give: 1 << 62})
|
||||
if err := fw.Apply(ctx, st); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if t2, d2 := used(); t2 > total-1000 || t2 < total-2000 || d2 != 0 {
|
||||
t.Errorf("after giving back: total %d (was %d, gave 1000), per destination %d (was %d, reset)", t2, total, d2, dev)
|
||||
}
|
||||
// Counted in total now: what the addresses used carries over.
|
||||
ping("127.0.0.3")
|
||||
_, dev = used()
|
||||
st.Rules[1].Quota.PerDestination = false
|
||||
if err := fw.Apply(ctx, st); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
tb, _ := fw.Table(ctx)
|
||||
if tb.Quotas["quota_r2"] < dev || dev == 0 {
|
||||
t.Errorf("per destination to a total: %d, was %d", tb.Quotas["quota_r2"], dev)
|
||||
}
|
||||
st.Rules[1].Quota.PerDestination = true
|
||||
if err := fw.Apply(ctx, st); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// Reset by hand.
|
||||
if err := fw.ResetQuota(ctx, 1, false); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := fw.ResetQuota(ctx, 2, true); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if t3, d3 := used(); t3 != 0 || d3 != 0 {
|
||||
t.Errorf("after a reset: total %d, per destination %d", t3, d3)
|
||||
}
|
||||
// A reboot: the table is gone (the boot ruleset has no usage); the
|
||||
// saved usage comes back.
|
||||
ping("127.0.0.2")
|
||||
ping("127.0.0.3")
|
||||
tb, _ = fw.Table(ctx)
|
||||
saved := tb.QuotaUsage
|
||||
exec.Command("docker", "exec", name, "nft", "delete", "table", "inet", "housecarl").Run()
|
||||
fw.RestoreQuotas(&saved)
|
||||
if err := fw.Apply(ctx, st); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if t4, d4 := used(); t4 < saved.Quotas["quota_r1"] || t4 == 0 || d4 == 0 {
|
||||
t.Errorf("after a reboot: total %d, per destination %d (saved %+v)", t4, d4, saved)
|
||||
}
|
||||
}
|
||||
|
||||
// TestQuotaAdjust: giving back takes off what was used, never below
|
||||
// zero; an entry with nothing left goes.
|
||||
func TestQuotaAdjust(t *testing.T) {
|
||||
u := QuotaUsage{Quotas: map[string]int64{"quota_r1": 5000, "quota_r9": 10},
|
||||
QuotaSets: map[string][]render.Element{"quota_r2_v4": {{Addr: "10.0.0.1", Used: 3000}, {Addr: "10.0.0.2", Used: 4000}},
|
||||
"quota_r2_v6": {{Addr: "2001:db8::", Used: 100}}}}
|
||||
u.apply([]QuotaAdjust{{Name: "quota_r1", Give: 2000}, {Name: "quota_r9", Give: 2000}, {Name: "quota_r2", Addr: "10.0.0.2", Give: 1500},
|
||||
{Name: "quota_r2", Give: 100}})
|
||||
if u.Quotas["quota_r1"] != 3000 || u.Quotas["quota_r9"] != 0 {
|
||||
t.Errorf("named: %v", u.Quotas)
|
||||
}
|
||||
want := map[string][]render.Element{"quota_r2_v4": {{Addr: "10.0.0.1", Used: 2900}, {Addr: "10.0.0.2", Used: 2400}}, "quota_r2_v6": nil}
|
||||
if !reflect.DeepEqual(u.QuotaSets, want) {
|
||||
t.Errorf("sets: %+v", u.QuotaSets)
|
||||
}
|
||||
}
|
||||
|
||||
// TestQuotaCarry: usage follows a rule counting its quota differently.
|
||||
func TestQuotaCarry(t *testing.T) {
|
||||
perSource := &model.Rule{ID: 2, Quota: &model.Quota{Bytes: 1 << 30, PerSource: true}}
|
||||
usage := func() QuotaUsage {
|
||||
return QuotaUsage{Quotas: map[string]int64{}, QuotaSets: map[string][]render.Element{
|
||||
"quota_r2_v4": {{Addr: "10.0.0.1", Used: 100}, {Addr: "10.0.0.2", Used: 50}},
|
||||
"quota_r2_v6": {{Addr: "2001:db8:0:1::", Used: 7}, {Addr: "2001:db8:0:2::", Used: 3}}}}
|
||||
}
|
||||
was := map[int64]*model.Rule{2: perSource}
|
||||
with := func(f func(r *model.Rule)) *model.State {
|
||||
r := *perSource
|
||||
q := *perSource.Quota
|
||||
r.Quota = &q
|
||||
f(&r)
|
||||
return &model.State{Rules: []*model.Rule{&r}}
|
||||
}
|
||||
u := usage()
|
||||
u.carry(was, with(func(r *model.Rule) { r.Quota.PerSource = false }))
|
||||
if u.Quotas["quota_r2"] != 160 || len(u.QuotaSets["quota_r2_v4"]) != 0 {
|
||||
t.Errorf("to a total: %+v", u)
|
||||
}
|
||||
u = usage()
|
||||
u.carry(was, with(func(r *model.Rule) { r.GroupV4, r.GroupV6 = 24, 48 }))
|
||||
want := map[string][]render.Element{"quota_r2_v4": {{Addr: "10.0.0.0", Used: 150}}, "quota_r2_v6": {{Addr: "2001:db8::", Used: 10}}}
|
||||
if !reflect.DeepEqual(u.QuotaSets, want) {
|
||||
t.Errorf("wider groups: %+v", u.QuotaSets)
|
||||
}
|
||||
u = usage()
|
||||
u.carry(was, with(func(r *model.Rule) { r.Quota.PerSource, r.Quota.PerDestination = false, true }))
|
||||
if len(u.QuotaSets) != 0 {
|
||||
t.Errorf("per destination now: %+v", u.QuotaSets)
|
||||
}
|
||||
u = usage()
|
||||
u.carry(nil, with(func(r *model.Rule) { r.Quota.PerSource = false }))
|
||||
if len(u.QuotaSets["quota_r2_v4"]) != 2 || u.Quotas["quota_r2"] != 0 {
|
||||
t.Errorf("nothing to compare with (the first apply): %+v", u)
|
||||
}
|
||||
}
|
||||
|
||||
// TestDeviceCountsInContainer: the per-device counters are read back and
|
||||
// survive a reload.
|
||||
func TestDeviceCountsInContainer(t *testing.T) {
|
||||
image := os.Getenv("HOUSECARL_NFT_IMAGE")
|
||||
if image == "" {
|
||||
t.Skip("HOUSECARL_NFT_IMAGE not set")
|
||||
}
|
||||
name := "housecarl-nft-stats-test"
|
||||
exec.Command("docker", "rm", "-f", name).Run()
|
||||
if out, err := exec.Command("docker", "run", "-d", "--name", name, "--cap-add", "NET_ADMIN", image, "sleep", "300").CombinedOutput(); err != nil {
|
||||
t.Fatalf("starting the container: %v %s", err, out)
|
||||
}
|
||||
t.Cleanup(func() { exec.Command("docker", "rm", "-f", name).Run() })
|
||||
fw := &Firewall{Runner: &Runner{Command: []string{"docker", "exec", "-i", name, "nft"}}}
|
||||
ctx := context.Background()
|
||||
st := model.NewState()
|
||||
st.Zones = []*model.Zone{{ID: 1, Name: "wan", Interfaces: []string{"eth0"}, External: true}}
|
||||
st.Settings.StatsPerDevice = true
|
||||
if err := fw.Apply(ctx, st); err != nil {
|
||||
t.Fatalf("apply: %v", err)
|
||||
}
|
||||
if out, err := exec.Command("docker", "exec", name, "nft", "add", "element", "inet", "housecarl", "stats_up_v4", "{ 10.0.10.5 counter packets 7 bytes 7000 }").CombinedOutput(); err != nil {
|
||||
t.Fatalf("adding an element: %v %s", err, out)
|
||||
}
|
||||
read := func() render.Element {
|
||||
tb, err := fw.Table(ctx)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, e := range tb.DeviceSets["stats_up_v4"] {
|
||||
if e.Addr == "10.0.10.5" {
|
||||
return e
|
||||
}
|
||||
}
|
||||
t.Fatalf("no 10.0.10.5 in %+v", tb.DeviceSets)
|
||||
return render.Element{}
|
||||
}
|
||||
if e := read(); e.Packets != 7 || e.Bytes != 7000 || e.Timeout <= 0 {
|
||||
t.Fatalf("read back %+v", e)
|
||||
}
|
||||
st.Settings.InputPolicy = model.PolicyAccept // a change: a reload
|
||||
if err := fw.Apply(ctx, st); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if e := read(); e.Packets != 7 || e.Bytes != 7000 {
|
||||
t.Errorf("after a reload: %+v", e)
|
||||
}
|
||||
}
|
||||
|
||||
// TestFlowtableStale: the flowtable in force goes stale when one of its
|
||||
// devices appears, goes, or comes back as a new interface.
|
||||
func TestFlowtableStale(t *testing.T) {
|
||||
idx := map[string]int{"eth1": 3}
|
||||
old := ifIndex
|
||||
ifIndex = func(name string) int { return idx[name] }
|
||||
t.Cleanup(func() { ifIndex = old })
|
||||
exists := func() map[string]bool {
|
||||
out := map[string]bool{}
|
||||
for n, i := range idx {
|
||||
out[n] = i > 0
|
||||
}
|
||||
return out
|
||||
}
|
||||
// "true" stands in for nft: every apply succeeds, and lists nothing.
|
||||
fw := &Firewall{Runner: &Runner{Command: []string{"true"}}, Interfaces: exists}
|
||||
st := model.NewState()
|
||||
st.Settings.FlowtableDevices = []string{"ppp0", "eth1"}
|
||||
if fw.FlowtableStale(st) {
|
||||
t.Error("stale before anything was applied")
|
||||
}
|
||||
if err := fw.Apply(context.Background(), st); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if fw.FlowtableStale(st) {
|
||||
t.Error("stale right after applying")
|
||||
}
|
||||
idx["ppp0"] = 7 // PPPoE came up
|
||||
if !fw.FlowtableStale(st) {
|
||||
t.Error("ppp0 appeared, yet not stale")
|
||||
}
|
||||
fw.Apply(context.Background(), st)
|
||||
idx["ppp0"] = 9 // and reconnected: a new interface by the same name
|
||||
if !fw.FlowtableStale(st) {
|
||||
t.Error("ppp0 came back as a new interface, yet not stale")
|
||||
}
|
||||
fw.Apply(context.Background(), st)
|
||||
idx["eth9"] = 4 // not a flowtable device
|
||||
if fw.FlowtableStale(st) {
|
||||
t.Error("stale over an interface the flowtable doesn't use")
|
||||
}
|
||||
}
|
||||
|
||||
// TestBootInContainer: the boot ruleset loads although its flowtable
|
||||
// devices don't exist yet, the fallback loads and is recognised, and
|
||||
// housecarld's apply replaces it.
|
||||
func TestBootInContainer(t *testing.T) {
|
||||
image := os.Getenv("HOUSECARL_NFT_IMAGE")
|
||||
if image == "" {
|
||||
t.Skip("HOUSECARL_NFT_IMAGE not set")
|
||||
}
|
||||
name := "housecarl-nft-boot-test"
|
||||
exec.Command("docker", "rm", "-f", name).Run()
|
||||
if out, err := exec.Command("docker", "run", "-d", "--name", name, "--cap-add", "NET_ADMIN", image, "sleep", "300").CombinedOutput(); err != nil {
|
||||
t.Fatalf("starting the container: %v %s", err, out)
|
||||
}
|
||||
t.Cleanup(func() { exec.Command("docker", "rm", "-f", name).Run() })
|
||||
nft := func(script string) error {
|
||||
c := exec.Command("docker", "exec", "-i", name, "nft", "-f", "/dev/stdin")
|
||||
c.Stdin = strings.NewReader(script)
|
||||
if out, err := c.CombinedOutput(); err != nil {
|
||||
return fmt.Errorf("%v: %s", err, out)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
dir := t.TempDir()
|
||||
// ppp0 exists when housecarld saves, but not at boot.
|
||||
fw := &Firewall{Runner: &Runner{Command: []string{"docker", "exec", "-i", name, "nft"}}, BootFile: filepath.Join(dir, "ruleset.nft"),
|
||||
Interfaces: func() map[string]bool { return map[string]bool{"ppp0": true, "eth0": true} }}
|
||||
st := model.NewState()
|
||||
st.Zones = []*model.Zone{{ID: 1, Name: "wan", Interfaces: []string{"ppp0"}, External: true}, {ID: 2, Name: "lan", Interfaces: []string{"eth0"}}}
|
||||
st.SNATs = []*model.SourceNAT{{ID: 3, Match: model.Match{OutZones: []string{"wan"}}}}
|
||||
st.Settings.FlowtableDevices = []string{"ppp0", "eth0"}
|
||||
if err := fw.SaveBoot(st); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
boot, _ := os.ReadFile(fw.BootFile)
|
||||
if strings.Contains(string(boot), "flowtable") {
|
||||
t.Errorf("the boot ruleset has a flowtable:\n%s", boot)
|
||||
}
|
||||
fallback, err := os.ReadFile(filepath.Join(dir, "ruleset-fallback.nft"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := nft(string(boot)); err != nil {
|
||||
t.Fatalf("the boot ruleset doesn't load: %v", err)
|
||||
}
|
||||
ctx := context.Background()
|
||||
if tb, err := fw.Table(ctx); err != nil || tb.Fallback {
|
||||
t.Errorf("after the boot ruleset: %+v %v", tb, err)
|
||||
}
|
||||
if err := nft(string(fallback)); err != nil {
|
||||
t.Fatalf("the fallback doesn't load: %v", err)
|
||||
}
|
||||
if tb, err := fw.Table(ctx); err != nil || !tb.Fallback {
|
||||
t.Errorf("the fallback isn't recognised: %+v %v", tb, err)
|
||||
}
|
||||
fw.Interfaces = func() map[string]bool { return map[string]bool{"eth0": true} }
|
||||
if err := fw.Apply(ctx, st); err != nil {
|
||||
t.Fatalf("apply over the fallback: %v", err)
|
||||
}
|
||||
if tb, err := fw.Table(ctx); err != nil || tb.Fallback {
|
||||
t.Errorf("the fallback stayed after an apply: %+v %v", tb, err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestShadowInContainer: in shadow mode a drop counts but doesn't drop,
|
||||
// the ruleset for leaving shadow mode is checked too, and the boot files
|
||||
// are shadow mode's.
|
||||
func TestShadowInContainer(t *testing.T) {
|
||||
image := os.Getenv("HOUSECARL_NFT_IMAGE")
|
||||
if image == "" {
|
||||
t.Skip("HOUSECARL_NFT_IMAGE not set")
|
||||
}
|
||||
name := "housecarl-nft-shadow-test"
|
||||
exec.Command("docker", "rm", "-f", name).Run()
|
||||
if out, err := exec.Command("docker", "run", "-d", "--name", name, "--cap-add", "NET_ADMIN", image, "sleep", "300").CombinedOutput(); err != nil {
|
||||
t.Fatalf("starting the container: %v %s", err, out)
|
||||
}
|
||||
t.Cleanup(func() { exec.Command("docker", "rm", "-f", name).Run() })
|
||||
dir := t.TempDir()
|
||||
fw := &Firewall{Runner: &Runner{Command: []string{"docker", "exec", "-i", name, "nft"}}, Shadow: true, BootFile: filepath.Join(dir, "ruleset.nft")}
|
||||
ctx := context.Background()
|
||||
st := model.NewState()
|
||||
st.Settings.OutputPolicy = model.PolicyDrop
|
||||
st.Settings.InputPolicy = model.PolicyAccept
|
||||
st.Settings.Builtins.Loopback = false // it would accept the ping first
|
||||
st.Rules = []*model.Rule{{ID: 3, Chain: model.ChainOutput, Position: 1, Action: model.ActionReject, Match: model.Match{Destinations: []string{"127.0.0.2"}}}}
|
||||
if err := fw.Apply(ctx, st); err != nil {
|
||||
t.Fatalf("apply: %v", err)
|
||||
}
|
||||
ping := func() error {
|
||||
return exec.Command("docker", "exec", name, "ping", "-c", "2", "-W", "1", "127.0.0.2").Run()
|
||||
}
|
||||
if err := ping(); err != nil {
|
||||
t.Errorf("shadow mode blocked a ping: %v", err)
|
||||
}
|
||||
tb, err := fw.Table(ctx)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if tb.Counters["r3"].Packets == 0 {
|
||||
t.Errorf("what would be rejected isn't counted: %+v", tb.Counters)
|
||||
}
|
||||
// A rule shadow mode leaves out, which nftables refuses when enforcing.
|
||||
broken := st.Clone()
|
||||
broken.Rules = append(broken.Rules, &model.Rule{ID: 4, Chain: model.ChainInput, Position: 1, Raw: "tcp dport 22 jump nowhere", Comment: "jump"})
|
||||
if err := fw.Check(ctx, broken); err == nil || !strings.Contains(err.Error(), "shadow mode is off") || !strings.Contains(err.Error(), "input #4 (jump)") {
|
||||
t.Errorf("the ruleset for leaving shadow mode wasn't checked: %v", err)
|
||||
}
|
||||
// Enforcing, the same rule rejects.
|
||||
enforcing := &Firewall{Runner: fw.Runner}
|
||||
if err := enforcing.Apply(ctx, st); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if ping() == nil {
|
||||
t.Error("enforcing didn't reject")
|
||||
}
|
||||
// The boot files: both load, and filter nothing.
|
||||
if err := fw.SaveBoot(st); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, f := range []string{"ruleset.nft", "ruleset-fallback.nft"} {
|
||||
b, _ := os.ReadFile(filepath.Join(dir, f))
|
||||
c := exec.Command("docker", "exec", "-i", name, "nft", "-f", "/dev/stdin")
|
||||
c.Stdin = strings.NewReader(string(b))
|
||||
if out, err := c.CombinedOutput(); err != nil {
|
||||
t.Fatalf("%s doesn't load: %v %s", f, err, out)
|
||||
}
|
||||
if err := ping(); err != nil {
|
||||
t.Errorf("%s blocked a ping: %v", f, err)
|
||||
}
|
||||
}
|
||||
if tb, err := fw.Table(ctx); err != nil || !tb.Fallback {
|
||||
t.Errorf("the shadow fallback isn't recognised: %+v %v", tb, err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestPruneCounters: the totals of objects gone (an import with --replace
|
||||
// gives every object a new ID) are forgotten; built-ins, policies and
|
||||
// disabled objects keep theirs.
|
||||
func TestPruneCounters(t *testing.T) {
|
||||
base := map[string]Counter{}
|
||||
for _, m := range []string{"r1", "r2", "r3", "f4", "f4-hairpin", "f4-6", "f5-hairpin", "s6", "d7", "d8", "b-established", "p-input", "fallback-input"} {
|
||||
base[m] = Counter{Packets: 1}
|
||||
}
|
||||
st := model.NewState()
|
||||
st.Rules = []*model.Rule{{ID: 1}, {ID: 3, Disabled: true}}
|
||||
st.Forwards = []*model.PortForward{{ID: 4}}
|
||||
st.Redirects = []*model.Redirect{{ID: 8}}
|
||||
pruneCounters(base, st)
|
||||
var got []string
|
||||
for m := range base {
|
||||
got = append(got, m)
|
||||
}
|
||||
sort.Strings(got)
|
||||
if strings.Join(got, " ") != "b-established d8 f4 f4-6 f4-hairpin fallback-input p-input r1 r3" {
|
||||
t.Errorf("kept %v", got)
|
||||
}
|
||||
}
|
||||
|
||||
// TestBansInContainer: a ban goes in, a second one renews its time, a
|
||||
// reload keeps the bans the database has, and lifting one already gone
|
||||
// isn't an error.
|
||||
func TestBansInContainer(t *testing.T) {
|
||||
image := os.Getenv("HOUSECARL_NFT_IMAGE")
|
||||
if image == "" {
|
||||
t.Skip("HOUSECARL_NFT_IMAGE not set")
|
||||
}
|
||||
name := "housecarl-nft-bans"
|
||||
exec.Command("docker", "rm", "-f", name).Run()
|
||||
if out, err := exec.Command("docker", "run", "-d", "--name", name, "--cap-add", "NET_ADMIN", image, "sleep", "300").CombinedOutput(); err != nil {
|
||||
t.Fatalf("starting the container: %v %s", err, out)
|
||||
}
|
||||
t.Cleanup(func() { exec.Command("docker", "rm", "-f", name).Run() })
|
||||
var active []render.Ban
|
||||
fw := &Firewall{Runner: &Runner{Command: []string{"docker", "exec", "-i", name, "nft"}}, Bans: func() []render.Ban { return active }}
|
||||
ctx := context.Background()
|
||||
st := model.NewState()
|
||||
st.Zones = []*model.Zone{{ID: 1, Name: "wan", Interfaces: []string{"eth0"}, External: true}}
|
||||
st.BanLists = []*model.BanList{{ID: 2, Name: "router", Scope: model.BanScopeFirewall}}
|
||||
if err := fw.Apply(ctx, st); err != nil {
|
||||
t.Fatalf("apply: %v", err)
|
||||
}
|
||||
list := func(set string) string {
|
||||
out, _ := exec.Command("docker", "exec", name, "nft", "list", "set", "inet", "housecarl", set).CombinedOutput()
|
||||
return string(out)
|
||||
}
|
||||
b := render.Ban{List: 2, Addr: netip.MustParsePrefix("203.0.113.9/32"), Timeout: 60}
|
||||
if err := fw.AddBan(ctx, b); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
b.Timeout = 7200
|
||||
if err := fw.AddBan(ctx, b); err != nil {
|
||||
t.Fatalf("renewing: %v", err)
|
||||
}
|
||||
if got := list("ban_2_v4"); !strings.Contains(got, "203.0.113.9 timeout 2h") {
|
||||
t.Errorf("renewed ban: %s", got)
|
||||
}
|
||||
svc := render.Ban{List: 2, Addr: netip.MustParsePrefix("2001:db8:1:2::/64"), Proto: "tcp", Port: 22}
|
||||
if err := fw.AddBan(ctx, svc); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
active = []render.Ban{svc}
|
||||
if err := fw.Apply(ctx, st); err != nil {
|
||||
t.Fatalf("reapply: %v", err)
|
||||
}
|
||||
if got := list("ban_2_v6_svc"); !strings.Contains(got, "2001:db8:1:2:: . 6 . 22") {
|
||||
t.Errorf("after a reload, the database's ban: %s", got)
|
||||
}
|
||||
if got := list("ban_2_v4"); strings.Contains(got, "203.0.113.9") {
|
||||
t.Errorf("after a reload, a ban the database doesn't have: %s", got)
|
||||
}
|
||||
if err := fw.DeleteBan(ctx, svc); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := fw.DeleteBan(ctx, svc); err != nil {
|
||||
t.Errorf("lifting a ban already gone: %v", err)
|
||||
}
|
||||
if got := list("ban_2_v6_svc"); strings.Contains(got, "2001:db8") {
|
||||
t.Errorf("lifted ban: %s", got)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,139 @@
|
||||
package render
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"net/netip"
|
||||
"strings"
|
||||
|
||||
"housecarl/internal/model"
|
||||
)
|
||||
|
||||
// Ban is an active ban, for rendering (Options.Bans): its list, the
|
||||
// address banned (an IPv4 address, or an IPv6 prefix of the list's
|
||||
// grouping), and for a ban of one port its protocol and port. Timeout is
|
||||
// the seconds it has left (0: it doesn't end).
|
||||
type Ban struct {
|
||||
List int64
|
||||
Addr netip.Prefix
|
||||
Proto string // "tcp", "udp", or "" for every port
|
||||
Port int
|
||||
Timeout int
|
||||
}
|
||||
|
||||
// BanSet is the name of a ban list's set for a family, and with svc
|
||||
// true its set of bans of one port.
|
||||
func BanSet(list int64, v6, svc bool) string {
|
||||
name := fmt.Sprintf("ban_%d_v4", list)
|
||||
if v6 {
|
||||
name = fmt.Sprintf("ban_%d_v6", list)
|
||||
}
|
||||
if svc {
|
||||
name += "_svc"
|
||||
}
|
||||
return name
|
||||
}
|
||||
|
||||
// BanMark is the mark (comment "hc:ban3") of a ban list's rules.
|
||||
func BanMark(list int64) string { return fmt.Sprintf("ban%d", list) }
|
||||
|
||||
// BanElement is the set element of a ban: "203.0.113.9", or
|
||||
// "203.0.113.9 . tcp . 22".
|
||||
func BanElement(b Ban) string {
|
||||
e := b.Addr.Addr().String()
|
||||
if b.Proto != "" {
|
||||
e += fmt.Sprintf(" . %s . %d", b.Proto, b.Port)
|
||||
}
|
||||
return e
|
||||
}
|
||||
|
||||
// IsBanSet reports whether a set is a ban list's.
|
||||
func IsBanSet(name string) bool { return strings.HasPrefix(name, "ban_") }
|
||||
|
||||
// banSets declares the ban lists' sets.
|
||||
func (r *renderer) banSets() {
|
||||
for _, b := range r.st.BanLists {
|
||||
if b.Disabled {
|
||||
continue
|
||||
}
|
||||
obj := "ban list " + b.Name
|
||||
for _, v6 := range []bool{false, true} {
|
||||
r.w.emit(obj, "\tset %s { type %s; flags timeout; }", BanSet(b.ID, v6, false), addrType(v6))
|
||||
r.w.emit(obj, "\tset %s { type %s . inet_proto . inet_service; flags timeout; }", BanSet(b.ID, v6, true), addrType(v6))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// banRules writes the drops of the ban lists that cover a chain, ahead
|
||||
// of everything that accepts: a banned address's established connections
|
||||
// are cut too.
|
||||
func (r *renderer) banRules(chain string) {
|
||||
for _, b := range r.st.BanLists {
|
||||
if b.Disabled || !b.Covers(chain) {
|
||||
continue
|
||||
}
|
||||
obj := "ban list " + b.Name
|
||||
d4, d6 := r.st.SplitRefs(b.Hosts)
|
||||
for _, fam := range []string{"v4", "v6"} {
|
||||
dsts := []string{""}
|
||||
if b.Scope == model.BanScopeHosts {
|
||||
refs := d4
|
||||
if fam == "v6" {
|
||||
refs = d6
|
||||
}
|
||||
var ok bool
|
||||
if dsts, ok = addrAlts(fam, "daddr", refs, true, false); !ok {
|
||||
continue
|
||||
}
|
||||
}
|
||||
key := ipKeyword(fam) + " saddr"
|
||||
if fam == "v6" && b.V6Group() < 128 {
|
||||
key += " and " + v6Mask(b.V6Group())
|
||||
}
|
||||
v6 := fam == "v6"
|
||||
for _, d := range dsts {
|
||||
r.banDrop(obj, b.ID, strings.Join(nonEmpty([]string{d, key + " @" + BanSet(b.ID, v6, false)}), " "))
|
||||
r.banDrop(obj, b.ID, strings.Join(nonEmpty([]string{d, key + " . meta l4proto . th dport @" + BanSet(b.ID, v6, true)}), " "))
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// banDrop writes one drop of a ban list, or in shadow mode counts it.
|
||||
func (r *renderer) banDrop(obj string, list int64, match string) {
|
||||
if r.opt.Shadow {
|
||||
r.w.emit(obj, "\t\t%s counter accept comment \"hc:%s %s\"", match, BanMark(list), Would("drop"))
|
||||
return
|
||||
}
|
||||
r.w.emit(obj, "\t\t%s counter drop comment \"hc:%s\"", match, BanMark(list))
|
||||
}
|
||||
|
||||
// v6Mask is the mask of an IPv6 prefix length: 64 → ffff:ffff:ffff:ffff::.
|
||||
func v6Mask(bits int) string {
|
||||
mask := make([]byte, 16)
|
||||
for i := 0; i < bits; i++ {
|
||||
mask[i/8] |= 0x80 >> (i % 8)
|
||||
}
|
||||
m, _ := netip.AddrFromSlice(mask)
|
||||
return m.String()
|
||||
}
|
||||
|
||||
// banElements adds the active bans, with the time each has left, so a
|
||||
// reload keeps them.
|
||||
func (r *renderer) banElements() {
|
||||
for _, ban := range r.opt.Bans {
|
||||
var list *model.BanList
|
||||
for _, b := range r.st.BanLists {
|
||||
if b.ID == ban.List && !b.Disabled {
|
||||
list = b
|
||||
}
|
||||
}
|
||||
if list == nil || !ban.Addr.IsValid() {
|
||||
continue
|
||||
}
|
||||
timeout := ""
|
||||
if ban.Timeout > 0 {
|
||||
timeout = fmt.Sprintf(" timeout %ds", ban.Timeout)
|
||||
}
|
||||
r.w.emit("ban list "+list.Name, "add element inet %s %s { %s%s }", Table, BanSet(list.ID, ban.Addr.Addr().Is6(), ban.Proto != ""), BanElement(ban), timeout)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,89 @@
|
||||
package render
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"housecarl/internal/model"
|
||||
)
|
||||
|
||||
// FallbackMark starts the comment of every fallback rule ("hc:fallback-…"):
|
||||
// how housecarld tells the fallback is in force.
|
||||
const FallbackMark = "fallback"
|
||||
|
||||
// Fallback builds the ruleset housecarl-ruleset.service loads at boot when
|
||||
// the saved one fails to: nothing new gets in from the external zones'
|
||||
// interfaces, everything else is let through, and IPv4 leaving an
|
||||
// external zone is masqueraded (when any source NAT is configured), so the
|
||||
// inside keeps working until housecarld puts the real ruleset back. With
|
||||
// no external zone it can't tell which interface faces the internet, and
|
||||
// lets nothing but loopback through.
|
||||
//
|
||||
// It is deliberately small: no sets, flowtable, meters or quotas, and
|
||||
// interfaces by name only, so it loads before they exist.
|
||||
func Fallback(st *model.State) string {
|
||||
var b strings.Builder
|
||||
line := func(format string, args ...any) {
|
||||
fmt.Fprintf(&b, format, args...)
|
||||
b.WriteByte('\n')
|
||||
}
|
||||
// rule writes a counted rule: what it matches, then its verdict.
|
||||
rule := func(name, match, verdict string) {
|
||||
line("\t\t%s counter %s comment \"hc:%s-%s\"", match, verdict, FallbackMark, name)
|
||||
}
|
||||
ext := st.ExternalInterfaces()
|
||||
line("# Housecarl's fallback: loaded at boot only if ruleset.nft fails to load,")
|
||||
line("# until housecarld starts and puts the real ruleset back.")
|
||||
line("table inet %s {}", Table)
|
||||
line("delete table inet %s", Table)
|
||||
line("table inet %s {", Table)
|
||||
if len(ext) == 0 {
|
||||
// Nothing says which interface faces the internet: close it all.
|
||||
for _, c := range []string{"input", "forward"} {
|
||||
line("\tchain %s {", c)
|
||||
line("\t\ttype filter hook %s priority filter; policy drop;", c)
|
||||
line("\t\tiifname \"lo\" accept")
|
||||
line("\t\tcounter comment \"hc:%s-%s\"", FallbackMark, c)
|
||||
line("\t}")
|
||||
}
|
||||
line("}")
|
||||
return b.String()
|
||||
}
|
||||
e := ifaces(ext)
|
||||
line("\tchain input {")
|
||||
line("\t\ttype filter hook input priority filter; policy accept;")
|
||||
line("\t\tiifname \"lo\" accept")
|
||||
line("\t\tct state established,related accept")
|
||||
rule("icmpv6", fmt.Sprintf("iifname %s icmpv6 type { %s, echo-request, echo-reply }", e, icmpv6Errors), "accept")
|
||||
rule("icmpv6-nd", fmt.Sprintf("iifname %s icmpv6 type { %s } ip6 hoplimit 255", e, icmpv6ND), "accept")
|
||||
rule("icmpv6-mld", fmt.Sprintf("iifname %s ip6 saddr fe80::/10 icmpv6 type { %s }", e, icmpv6MLD), "accept")
|
||||
rule("dhcpv6-client", fmt.Sprintf("iifname %s ip6 saddr fe80::/10 udp dport 546", e), "accept")
|
||||
rule("input", "iifname "+e, "drop")
|
||||
line("\t}")
|
||||
line("\tchain forward {")
|
||||
line("\t\ttype filter hook forward priority filter; policy accept;")
|
||||
if st.Settings.Builtins.MSSClamp {
|
||||
line("\t\toifname %s tcp flags syn tcp option maxseg size set rt mtu", e)
|
||||
}
|
||||
line("\t\tct state established,related accept")
|
||||
rule("forward", "iifname "+e, "drop")
|
||||
line("\t}")
|
||||
if masquerades(st) {
|
||||
line("\tchain srcnat {")
|
||||
line("\t\ttype nat hook postrouting priority srcnat; policy accept;")
|
||||
rule("masquerade", "meta nfproto ipv4 oifname "+e, "masquerade")
|
||||
line("\t}")
|
||||
}
|
||||
line("}")
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// masquerades reports whether any source NAT is configured.
|
||||
func masquerades(st *model.State) bool {
|
||||
for _, x := range st.SNATs {
|
||||
if !x.Disabled {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -0,0 +1,75 @@
|
||||
package render
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"os/exec"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"housecarl/internal/model"
|
||||
)
|
||||
|
||||
func TestFallback(t *testing.T) {
|
||||
st := router()
|
||||
text := Fallback(st)
|
||||
for _, want := range []string{
|
||||
"delete table inet housecarl",
|
||||
"type filter hook input priority filter; policy accept;",
|
||||
`iifname "wan" counter drop comment "hc:fallback-input"`,
|
||||
`iifname "wan" counter drop comment "hc:fallback-forward"`,
|
||||
`iifname "wan" ip6 saddr fe80::/10 udp dport 546 counter accept`,
|
||||
`ip6 hoplimit 255 counter accept`,
|
||||
`oifname "wan" tcp flags syn tcp option maxseg size set rt mtu`,
|
||||
`meta nfproto ipv4 oifname "wan" counter masquerade comment "hc:fallback-masquerade"`,
|
||||
} {
|
||||
if !strings.Contains(text, want) {
|
||||
t.Errorf("no %q in\n%s", want, text)
|
||||
}
|
||||
}
|
||||
// Nothing that could fail to load before the network is up.
|
||||
for _, not := range []string{"flowtable", "@", "\tset ", "enp2s0", "wg0"} {
|
||||
if strings.Contains(text, not) {
|
||||
t.Errorf("%q in the fallback:\n%s", not, text)
|
||||
}
|
||||
}
|
||||
// Established connections are accepted before the drops.
|
||||
if strings.Index(text, "ct state established,related accept") > strings.Index(text, "hc:fallback-input") {
|
||||
t.Errorf("established after the drop:\n%s", text)
|
||||
}
|
||||
|
||||
// No source NAT: no masquerade.
|
||||
st.SNATs = nil
|
||||
if text := Fallback(st); strings.Contains(text, "masquerade") {
|
||||
t.Errorf("masquerade without source NAT:\n%s", text)
|
||||
}
|
||||
|
||||
// No external zone: everything but loopback is dropped.
|
||||
for _, z := range st.Zones {
|
||||
z.External = false
|
||||
}
|
||||
text = Fallback(st)
|
||||
if strings.Count(text, "policy drop;") != 2 || strings.Contains(text, "policy accept") || strings.Contains(text, "established") {
|
||||
t.Errorf("no external zone:\n%s", text)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFallbackPassesNft(t *testing.T) {
|
||||
cmd := strings.Fields(os.Getenv("HOUSECARL_NFT"))
|
||||
if len(cmd) == 0 {
|
||||
t.Skip("HOUSECARL_NFT not set")
|
||||
}
|
||||
closed := router()
|
||||
for _, z := range closed.Zones {
|
||||
z.External = false
|
||||
}
|
||||
wild := router()
|
||||
wild.Zones = append(wild.Zones, &model.Zone{ID: 99, Name: "lte", Interfaces: []string{"wwan*"}, External: true})
|
||||
for name, st := range map[string]*model.State{"router": router(), "closed": closed, "wildcard": wild} {
|
||||
c := exec.CommandContext(context.Background(), cmd[0], append(cmd[1:], "-c", "-f", "/dev/stdin")...)
|
||||
c.Stdin = strings.NewReader(Fallback(st))
|
||||
if out, err := c.CombinedOutput(); err != nil {
|
||||
t.Errorf("%s: nft -c: %v\n%s\n%s", name, err, out, Fallback(st))
|
||||
}
|
||||
}
|
||||
}
|
||||
+286
-34
@@ -8,10 +8,19 @@
|
||||
// counter and a comment "hc:<kind><id>", which is how counters are read
|
||||
// back, and Ruleset.Lines maps each line to its object, for nftables'
|
||||
// error messages.
|
||||
//
|
||||
// In shadow mode (Options.Shadow, docs/shadow.md) the same rules are
|
||||
// written so that they change nothing: drops and rejects accept instead
|
||||
// (an accept only ends Housecarl's chain; the packet goes on to other
|
||||
// tables' chains, which decide), policies accept, NAT only counts, and
|
||||
// what alters packets (MSS clamping, the flowtable) is left out. What
|
||||
// would be dropped, rejected or translated carries a marker in its
|
||||
// comment ("[would drop]"), which the tracer reads.
|
||||
package render
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"net/netip"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
@@ -33,14 +42,43 @@ type Options struct {
|
||||
// Elements are dynamic sets' current entries, re-added in the same
|
||||
// transaction so a reload keeps them: set name → entries.
|
||||
Elements map[string][]Element
|
||||
// Quotas are the rules' quotas' used bytes (quota name, quota_r12),
|
||||
// and QuotaSets the per-address quotas' entries (set name,
|
||||
// quota_r12_v4), so a reload keeps what was used.
|
||||
Quotas map[string]int64
|
||||
QuotaSets map[string][]Element
|
||||
// DeviceSets are the per-device traffic counters' entries
|
||||
// (stats_up_v4 …), so a reload keeps them.
|
||||
DeviceSets map[string][]Element
|
||||
// Bans are the ban lists' active bans, added to their sets.
|
||||
Bans []Ban
|
||||
// Shadow renders for shadow mode: nothing is blocked, rejected,
|
||||
// translated or changed (see the package comment).
|
||||
Shadow bool
|
||||
// ShadowBefore runs the filter chains just before other tables' on
|
||||
// their hooks (priority - 10) instead of just after (+ 10).
|
||||
ShadowBefore bool
|
||||
}
|
||||
|
||||
// Element is a dynamic set's entry with its remaining lifetime.
|
||||
// Would marks, in shadow mode, a rule's comment with what it would do:
|
||||
// "[would drop]".
|
||||
func Would(action string) string { return "[would " + action + "]" }
|
||||
|
||||
// Element is a dynamic set's entry with its remaining lifetime, or a
|
||||
// per-address quota's entry with what it used.
|
||||
type Element struct {
|
||||
Addr string `json:"address"`
|
||||
Timeout int `json:"timeout,omitempty"` // seconds left (0: none)
|
||||
Used int64 `json:"used,omitempty"` // bytes
|
||||
// Packets and Bytes are a per-device counter's.
|
||||
Packets uint64 `json:"packets,omitempty"`
|
||||
Bytes uint64 `json:"bytes,omitempty"`
|
||||
}
|
||||
|
||||
// QuotaName is the name of a rule's quota: the named quota of a total
|
||||
// quota, and (with _v4, _v6) the sets of a per-address one.
|
||||
func QuotaName(ruleID int64) string { return fmt.Sprintf("quota_r%d", ruleID) }
|
||||
|
||||
// Ruleset is the rendered nftables script.
|
||||
type Ruleset struct {
|
||||
Text string
|
||||
@@ -86,17 +124,23 @@ type renderer struct {
|
||||
st *model.State
|
||||
opt Options
|
||||
w *writer
|
||||
// meters are the per-source limit and connection sets the rules need.
|
||||
// meters are the per-address limit, connection and quota sets, and
|
||||
// the named quotas, the rules need.
|
||||
meters []string
|
||||
}
|
||||
|
||||
func (r *renderer) render() {
|
||||
w := r.w
|
||||
w.emit("", "# Built by Housecarl from its database; changes made here are lost.")
|
||||
if r.opt.Shadow {
|
||||
w.emit("", "# Shadow mode: nothing here drops, rejects or translates; other tables decide.")
|
||||
}
|
||||
w.emit("", "table inet %s {}", Table)
|
||||
w.emit("", "delete table inet %s", Table)
|
||||
w.emit("", "table inet %s {", Table)
|
||||
r.sets()
|
||||
r.banSets()
|
||||
r.deviceSets()
|
||||
devices := r.flowtableDevices()
|
||||
if len(devices) > 0 {
|
||||
w.emit("built-in: flowtable", "\tflowtable ft { hook ingress priority filter; devices = { %s }; }", strings.Join(quoteAll(devices), ", "))
|
||||
@@ -124,6 +168,7 @@ func (r *renderer) render() {
|
||||
w.line = base + chains.line
|
||||
w.emit("", "}")
|
||||
r.elements()
|
||||
r.banElements()
|
||||
}
|
||||
|
||||
// ---- sets
|
||||
@@ -177,6 +222,56 @@ func (r *renderer) elements() {
|
||||
}
|
||||
}
|
||||
|
||||
// IsDeviceSet reports whether a set is a per-device traffic counter.
|
||||
func IsDeviceSet(name string) bool {
|
||||
return strings.HasPrefix(name, "stats_up_") || strings.HasPrefix(name, "stats_down_")
|
||||
}
|
||||
|
||||
// deviceSets declares the per-device traffic counters, with what they
|
||||
// counted before a reload.
|
||||
func (r *renderer) deviceSets() {
|
||||
if !r.st.Settings.StatsPerDevice || len(r.st.ExternalInterfaces()) == 0 {
|
||||
return
|
||||
}
|
||||
for _, dir := range []string{"up", "down"} {
|
||||
for _, v6 := range []bool{false, true} {
|
||||
name := setName("stats_"+dir, v6)
|
||||
var elems []string
|
||||
for _, e := range r.opt.DeviceSets[name] {
|
||||
a, err := netip.ParseAddr(e.Addr)
|
||||
if err != nil || a.Is6() != v6 {
|
||||
continue
|
||||
}
|
||||
timeout := ""
|
||||
if e.Timeout > 0 {
|
||||
timeout = fmt.Sprintf(" timeout %ds", e.Timeout)
|
||||
}
|
||||
elems = append(elems, fmt.Sprintf("%s%s counter packets %d bytes %d", a, timeout, e.Packets, e.Bytes))
|
||||
}
|
||||
list := ""
|
||||
if len(elems) > 0 {
|
||||
list = " elements = { " + strings.Join(elems, ", ") + " }"
|
||||
}
|
||||
r.w.emit("built-in: stats", "\tset %s { type %s; flags dynamic,timeout; timeout 1h; size 65535; counter;%s }", name, addrType(v6), list)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// deviceCounts counts forwarded traffic per inside address: what leaves
|
||||
// for an external zone by its source, what comes from one by its
|
||||
// destination (IPv6 by /64).
|
||||
func (r *renderer) deviceCounts() {
|
||||
ext := r.st.ExternalInterfaces()
|
||||
if !r.st.Settings.StatsPerDevice || len(ext) == 0 {
|
||||
return
|
||||
}
|
||||
e := ifaces(ext)
|
||||
r.builtin("stats", "iifname != %s oifname %s update @stats_up_v4 { ip saddr }", e, e)
|
||||
r.builtin("stats", "iifname != %s oifname %s update @stats_up_v6 { ip6 saddr and ffff:ffff:ffff:ffff:: }", e, e)
|
||||
r.builtin("stats", "iifname %s oifname != %s update @stats_down_v4 { ip daddr }", e, e)
|
||||
r.builtin("stats", "iifname %s oifname != %s update @stats_down_v6 { ip6 daddr and ffff:ffff:ffff:ffff:: }", e, e)
|
||||
}
|
||||
|
||||
// SetName is the nftables name of a set's family part.
|
||||
func SetName(name string, v6 bool) string { return setName(name, v6) }
|
||||
|
||||
@@ -195,6 +290,10 @@ func addrType(v6 bool) string {
|
||||
}
|
||||
|
||||
func (r *renderer) flowtableDevices() []string {
|
||||
if r.opt.Shadow {
|
||||
// Offloaded connections skip every table's forward chain.
|
||||
return nil
|
||||
}
|
||||
var out []string
|
||||
for _, d := range r.st.Settings.FlowtableDevices {
|
||||
if r.opt.Interfaces == nil || r.opt.Interfaces[d] {
|
||||
@@ -207,8 +306,19 @@ func (r *renderer) flowtableDevices() []string {
|
||||
// ---- chains
|
||||
|
||||
func (r *renderer) chain(name, hook, prio, policy string) {
|
||||
nat := prio == "dstnat" || prio == "srcnat"
|
||||
if r.opt.Shadow {
|
||||
// NAT chains are asked in turn only until one translates: to
|
||||
// count anything, Housecarl's must come first.
|
||||
if nat || r.opt.ShadowBefore {
|
||||
prio += " - 10"
|
||||
} else {
|
||||
prio += " + 10"
|
||||
}
|
||||
policy = model.PolicyAccept
|
||||
}
|
||||
r.w.emit("", "\tchain %s {", name)
|
||||
r.w.emit("", "\t\ttype %s hook %s priority %s; policy %s;", map[bool]string{true: "nat", false: "filter"}[prio == "dstnat" || prio == "srcnat"], hook, prio, policy)
|
||||
r.w.emit("", "\t\ttype %s hook %s priority %s; policy %s;", map[bool]string{true: "nat", false: "filter"}[nat], hook, prio, policy)
|
||||
}
|
||||
|
||||
func (r *renderer) end() { r.w.emit("", "\t}") }
|
||||
@@ -217,6 +327,16 @@ func (r *renderer) builtin(name, format string, args ...any) {
|
||||
r.w.emit("built-in: "+name, "\t\t"+format+" comment \"hc:b-%s\"", append(args, name)...)
|
||||
}
|
||||
|
||||
// builtinDrop writes a built-in that drops what it matches, or in shadow
|
||||
// mode counts it.
|
||||
func (r *renderer) builtinDrop(name, format string, args ...any) {
|
||||
if r.opt.Shadow {
|
||||
r.w.emit("built-in: "+name, "\t\t"+format+" counter accept comment \"hc:b-%s %s\"", append(args, name, Would("drop"))...)
|
||||
return
|
||||
}
|
||||
r.builtin(name, format+" counter drop", args...)
|
||||
}
|
||||
|
||||
func (r *renderer) early() {
|
||||
st := r.st
|
||||
r.chain("early", "prerouting", "raw", "accept")
|
||||
@@ -232,13 +352,14 @@ func (r *renderer) early() {
|
||||
}
|
||||
}
|
||||
if len(v4) > 0 {
|
||||
r.builtin("antispoof", "iifname %s ip saddr { %s } counter drop", ifaces(ext), strings.Join(v4, ", "))
|
||||
r.builtinDrop("antispoof", "iifname %s ip saddr { %s }", ifaces(ext), strings.Join(v4, ", "))
|
||||
}
|
||||
if len(v6) > 0 {
|
||||
r.builtin("antispoof6", "iifname %s ip6 saddr { %s } counter drop", ifaces(ext), strings.Join(v6, ", "))
|
||||
r.builtinDrop("antispoof6", "iifname %s ip6 saddr { %s }", ifaces(ext), strings.Join(v6, ", "))
|
||||
}
|
||||
}
|
||||
r.rules(model.ChainEarly)
|
||||
r.rules(model.ChainEarly, true)
|
||||
r.rules(model.ChainEarly, false)
|
||||
r.end()
|
||||
}
|
||||
|
||||
@@ -254,7 +375,7 @@ func (r *renderer) connBuiltins() {
|
||||
r.builtin("established", "ct state established,related counter accept")
|
||||
}
|
||||
if b.DropInvalid {
|
||||
r.builtin("invalid", "ct state invalid counter drop")
|
||||
r.builtinDrop("invalid", "ct state invalid")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -265,6 +386,10 @@ func (r *renderer) input() {
|
||||
if b.Loopback {
|
||||
r.builtin("loopback", "iifname \"lo\" accept")
|
||||
}
|
||||
r.banRules(model.ChainInput)
|
||||
// Rules that count traffic see every packet: before the established
|
||||
// connections are accepted.
|
||||
r.rules(model.ChainInput, true)
|
||||
r.connBuiltins()
|
||||
ext := st.ExternalInterfaces()
|
||||
if b.ICMPv6 {
|
||||
@@ -282,7 +407,7 @@ func (r *renderer) input() {
|
||||
if b.DHCPv6Client && len(ext) > 0 {
|
||||
r.builtin("dhcpv6-client", "iifname %s ip6 saddr fe80::/10 udp dport 546 counter accept", ifaces(ext))
|
||||
}
|
||||
r.rules(model.ChainInput)
|
||||
r.rules(model.ChainInput, false)
|
||||
r.policyTail("input", st.Settings.InputPolicy)
|
||||
r.end()
|
||||
}
|
||||
@@ -291,10 +416,14 @@ func (r *renderer) forward(flowtable bool) {
|
||||
st := r.st
|
||||
b := st.Settings.Builtins
|
||||
r.chain("forward", "forward", "filter", st.Settings.ForwardPolicy)
|
||||
r.banRules(model.ChainForward)
|
||||
ext := st.ExternalInterfaces()
|
||||
if b.MSSClamp && len(ext) > 0 {
|
||||
// Shadow mode changes no packet.
|
||||
if b.MSSClamp && len(ext) > 0 && !r.opt.Shadow {
|
||||
r.builtin("mss-clamp", "oifname %s tcp flags syn tcp option maxseg size set rt mtu", ifaces(ext))
|
||||
}
|
||||
r.rules(model.ChainForward, true)
|
||||
r.deviceCounts()
|
||||
if flowtable {
|
||||
r.builtin("flowtable", "meta l4proto { tcp, udp } flow add @ft")
|
||||
}
|
||||
@@ -303,7 +432,7 @@ func (r *renderer) forward(flowtable bool) {
|
||||
r.builtin("icmpv6", "icmpv6 type { %s } counter accept", icmpv6Errors)
|
||||
}
|
||||
r.forwardAccepts()
|
||||
r.rules(model.ChainForward)
|
||||
r.rules(model.ChainForward, false)
|
||||
r.policyTail("forward", st.Settings.ForwardPolicy)
|
||||
r.end()
|
||||
}
|
||||
@@ -311,6 +440,7 @@ func (r *renderer) forward(flowtable bool) {
|
||||
func (r *renderer) output() {
|
||||
st := r.st
|
||||
r.chain("output", "output", "filter", st.Settings.OutputPolicy)
|
||||
r.rules(model.ChainOutput, true)
|
||||
if st.Settings.OutputPolicy == model.PolicyDrop {
|
||||
if st.Settings.Builtins.Loopback {
|
||||
r.builtin("loopback", "oifname \"lo\" accept")
|
||||
@@ -322,34 +452,50 @@ func (r *renderer) output() {
|
||||
r.builtin("icmpv6", "icmpv6 type { %s, echo-request, echo-reply, %s, %s } counter accept", icmpv6Errors, icmpv6ND, icmpv6MLD)
|
||||
}
|
||||
}
|
||||
r.rules(model.ChainOutput)
|
||||
r.rules(model.ChainOutput, false)
|
||||
r.policyTail("output", st.Settings.OutputPolicy)
|
||||
r.end()
|
||||
}
|
||||
|
||||
// policyTail counts (and optionally logs) what reaches the policy.
|
||||
// policyTail counts (and optionally logs) what reaches the policy. In
|
||||
// shadow mode, where the policy accepts, that is what it would drop, and
|
||||
// it is always logged: the log is how shadow mode shows it.
|
||||
func (r *renderer) policyTail(chain, policy string) {
|
||||
if policy != model.PolicyDrop {
|
||||
return
|
||||
}
|
||||
obj := "policy: " + chain
|
||||
if r.st.Settings.LogPolicyDrops && chain != "output" {
|
||||
if r.opt.Shadow || r.st.Settings.LogPolicyDrops && chain != "output" {
|
||||
r.w.emit(obj, "\t\tlimit rate 10/second burst 20 packets log prefix %s group %d", quote("hc "+chain+" policy drop: "), LogGroup)
|
||||
}
|
||||
if r.opt.Shadow {
|
||||
r.w.emit(obj, "\t\tcounter comment \"hc:p-%s %s\"", chain, Would("drop"))
|
||||
return
|
||||
}
|
||||
r.w.emit(obj, "\t\tcounter comment \"hc:p-%s\"", chain)
|
||||
}
|
||||
|
||||
// ---- rules
|
||||
|
||||
func (r *renderer) rules(chain string) {
|
||||
// rules writes a chain's rules: those that count traffic (traffic true),
|
||||
// or the others.
|
||||
func (r *renderer) rules(chain string, traffic bool) {
|
||||
for _, rule := range r.st.Rules {
|
||||
if rule.Chain != chain || rule.Disabled {
|
||||
if rule.Chain != chain || rule.Disabled || rule.ChecksEveryPacket() != traffic {
|
||||
continue
|
||||
}
|
||||
obj := rule.Label()
|
||||
mark := fmt.Sprintf("r%d", rule.ID)
|
||||
if rule.Raw != "" {
|
||||
r.w.emit(obj, "\t\t%s", rule.Raw)
|
||||
raw := rule.Raw
|
||||
if r.opt.Shadow {
|
||||
var ok bool
|
||||
if raw, ok = ShadowRaw(raw); !ok {
|
||||
r.w.emit(obj, "\t\t# left out in shadow mode: %s", rule.Raw)
|
||||
continue
|
||||
}
|
||||
}
|
||||
r.w.emit(obj, "\t\t%s", raw)
|
||||
continue
|
||||
}
|
||||
var tail []string
|
||||
@@ -370,8 +516,8 @@ func (r *renderer) rules(chain string) {
|
||||
tail = append(tail, hours(rule.HourFrom, rule.HourTo))
|
||||
}
|
||||
verdict := r.verdict(rule, mark)
|
||||
perSource := rule.ConnLimit > 0 || rule.Limit != nil && rule.Limit.PerSource
|
||||
for _, line := range r.expand(rule.Match, perSource) {
|
||||
perAddress := rule.ConnLimit > 0 || rule.Limit.PerAddress() != "" || rule.Quota.PerAddress() != ""
|
||||
for _, line := range r.expand(rule.Match, perAddress) {
|
||||
parts := append([]string{}, line.parts...)
|
||||
parts = append(parts, tail...)
|
||||
if l := rule.Limit; l != nil {
|
||||
@@ -380,6 +526,9 @@ func (r *renderer) rules(chain string) {
|
||||
if rule.ConnLimit > 0 {
|
||||
parts = append(parts, r.connLimit(rule, line))
|
||||
}
|
||||
if rule.Quota != nil {
|
||||
parts = append(parts, r.quota(rule, line))
|
||||
}
|
||||
parts = append(parts, verdict)
|
||||
r.w.emit(obj, "\t\t%s", strings.Join(nonEmpty(parts), " "))
|
||||
}
|
||||
@@ -396,34 +545,70 @@ func (r *renderer) verdict(rule *model.Rule, mark string) string {
|
||||
}
|
||||
v = append(v, fmt.Sprintf("log prefix %s group %d", quote(prefix), LogGroup))
|
||||
}
|
||||
switch rule.Action {
|
||||
case model.ActionAccept, model.ActionDrop:
|
||||
text := rule.Comment
|
||||
switch {
|
||||
case r.opt.Shadow && (rule.Action == model.ActionDrop || rule.Action == model.ActionReject):
|
||||
// It ends Housecarl's chain as the drop would, and no more.
|
||||
v = append(v, model.ActionAccept)
|
||||
text = Would(rule.Action) + " " + text
|
||||
case rule.Action == model.ActionAccept, rule.Action == model.ActionDrop:
|
||||
v = append(v, rule.Action)
|
||||
case model.ActionReject:
|
||||
case rule.Action == model.ActionReject:
|
||||
if rule.RejectWith == "tcp-reset" {
|
||||
v = append(v, "reject with tcp reset")
|
||||
} else {
|
||||
v = append(v, "reject")
|
||||
}
|
||||
}
|
||||
v = append(v, "comment "+comment(mark, rule.Comment))
|
||||
v = append(v, "comment "+comment(mark, text))
|
||||
return strings.Join(v, " ")
|
||||
}
|
||||
|
||||
// limit renders a rate limit, per source with a meter set per family.
|
||||
// limit renders a rate limit: of packets or bytes; per source or
|
||||
// destination with a meter set per family, whose entries are refreshed
|
||||
// by each packet (update) and forgotten once a source is quiet.
|
||||
func (r *renderer) limit(rule *model.Rule, line expanded, l *model.Limit) string {
|
||||
rate := fmt.Sprintf("limit rate %s%d/%s", map[bool]string{true: "over ", false: ""}[l.Over], l.Rate, l.Per)
|
||||
if l.Burst > 0 {
|
||||
rate += fmt.Sprintf(" burst %d packets", l.Burst)
|
||||
unit := ""
|
||||
if _, ok := model.ByteUnits[l.Unit]; ok {
|
||||
unit = " " + l.Unit
|
||||
}
|
||||
if !l.PerSource {
|
||||
rate := fmt.Sprintf("limit rate %s%d%s/%s", map[bool]string{true: "over ", false: ""}[l.Over], l.Rate, unit, l.Per)
|
||||
if l.Burst > 0 {
|
||||
burstUnit := " packets"
|
||||
if unit != "" {
|
||||
burstUnit = unit
|
||||
}
|
||||
rate += fmt.Sprintf(" burst %d%s", l.Burst, burstUnit)
|
||||
}
|
||||
addr := l.PerAddress()
|
||||
if addr == "" {
|
||||
return rate
|
||||
}
|
||||
name := fmt.Sprintf("meter_r%d_%s", rule.ID, line.fam)
|
||||
if !r.hasMeter(name) {
|
||||
r.meters = append(r.meters, fmt.Sprintf("\tset %s { type %s; flags dynamic,timeout; timeout 1%s; }", name, addrType(line.fam == "v6"), map[string]string{"second": "m", "minute": "m", "hour": "h", "day": "d"}[l.Per]))
|
||||
}
|
||||
return fmt.Sprintf("add @%s { %s saddr %s }", name, ipKeyword(line.fam), rate)
|
||||
return fmt.Sprintf("update @%s { %s %s }", name, r.addrKey(rule, line.fam, addr), rate)
|
||||
}
|
||||
|
||||
// addrKey is what a per-address meter counts by: "ip saddr", or grouped
|
||||
// by a prefix, "ip6 saddr and ffff:ffff:ffff:ffff::".
|
||||
func (r *renderer) addrKey(rule *model.Rule, fam, addr string) string {
|
||||
v4, v6 := rule.Groups()
|
||||
bits, size := v4, 32
|
||||
if fam == "v6" {
|
||||
bits, size = v6, 128
|
||||
}
|
||||
key := ipKeyword(fam) + " " + addr
|
||||
if bits >= size {
|
||||
return key
|
||||
}
|
||||
mask := make([]byte, size/8)
|
||||
for i := 0; i < bits; i++ {
|
||||
mask[i/8] |= 0x80 >> (i % 8)
|
||||
}
|
||||
m, _ := netip.AddrFromSlice(mask)
|
||||
return key + " and " + m.String()
|
||||
}
|
||||
|
||||
func (r *renderer) connLimit(rule *model.Rule, line expanded) string {
|
||||
@@ -431,12 +616,70 @@ func (r *renderer) connLimit(rule *model.Rule, line expanded) string {
|
||||
if !r.hasMeter(name) {
|
||||
r.meters = append(r.meters, fmt.Sprintf("\tset %s { type %s; flags dynamic; size 65535; }", name, addrType(line.fam == "v6")))
|
||||
}
|
||||
return fmt.Sprintf("add @%s { %s saddr ct count over %d }", name, ipKeyword(line.fam), rule.ConnLimit)
|
||||
return fmt.Sprintf("add @%s { %s ct count over %d }", name, r.addrKey(rule, line.fam, "saddr"), rule.ConnLimit)
|
||||
}
|
||||
|
||||
// quota renders a rule's quota: a named quota shared by the rule's lines
|
||||
// (both families count against one allowance), or per address a set of
|
||||
// quotas per family. What was used comes back from Options, so a reload
|
||||
// doesn't reset it.
|
||||
func (r *renderer) quota(rule *model.Rule, line expanded) string {
|
||||
q := rule.Quota
|
||||
amount := quotaAmount(q)
|
||||
name := QuotaName(rule.ID)
|
||||
addr := q.PerAddress()
|
||||
if addr == "" {
|
||||
if !r.hasMeter(name) {
|
||||
used := ""
|
||||
if u := r.opt.Quotas[name]; u > 0 {
|
||||
used = fmt.Sprintf(" used %d bytes", u)
|
||||
}
|
||||
r.meters = append(r.meters, fmt.Sprintf("\tquota %s { %s%s }", name, amount, used))
|
||||
}
|
||||
return fmt.Sprintf("quota name %s", quote(name))
|
||||
}
|
||||
set := name + "_" + line.fam
|
||||
if !r.hasMeter(set) {
|
||||
var elems []string
|
||||
for _, e := range r.opt.QuotaSets[set] {
|
||||
a, err := netip.ParseAddr(e.Addr)
|
||||
if err != nil || a.Is6() != (line.fam == "v6") {
|
||||
continue
|
||||
}
|
||||
used := ""
|
||||
if e.Used > 0 {
|
||||
used = fmt.Sprintf(" used %d bytes", e.Used)
|
||||
}
|
||||
elems = append(elems, fmt.Sprintf("%s quota %s%s", a, amount, used))
|
||||
}
|
||||
list := ""
|
||||
if len(elems) > 0 {
|
||||
list = " elements = { " + strings.Join(elems, ", ") + " }"
|
||||
}
|
||||
r.meters = append(r.meters, fmt.Sprintf("\tset %s { type %s; flags dynamic; size 65535;%s }", set, addrType(line.fam == "v6"), list))
|
||||
}
|
||||
return fmt.Sprintf("add @%s { %s quota %s }", set, r.addrKey(rule, line.fam, addr), amount)
|
||||
}
|
||||
|
||||
// quotaAmount writes a quota: "over 5000 mbytes" (nftables 1.1.3 has no
|
||||
// gbytes).
|
||||
func quotaAmount(q *model.Quota) string {
|
||||
n, unit := q.Bytes, "bytes"
|
||||
for _, u := range []string{"mbytes", "kbytes"} {
|
||||
if n >= model.ByteUnits[u] && n%model.ByteUnits[u] == 0 {
|
||||
n, unit = n/model.ByteUnits[u], u
|
||||
break
|
||||
}
|
||||
}
|
||||
if q.Over {
|
||||
return fmt.Sprintf("over %d %s", n, unit)
|
||||
}
|
||||
return fmt.Sprintf("until %d %s", n, unit)
|
||||
}
|
||||
|
||||
func (r *renderer) hasMeter(name string) bool {
|
||||
for _, m := range r.meters {
|
||||
if strings.Contains(m, "set "+name+" ") {
|
||||
if strings.Contains(m, "set "+name+" ") || strings.Contains(m, "quota "+name+" ") {
|
||||
return true
|
||||
}
|
||||
}
|
||||
@@ -610,7 +853,7 @@ func (r *renderer) dstnat() {
|
||||
}
|
||||
}
|
||||
}
|
||||
r.w.emit(obj, "\t\t%s counter redirect to :%s comment %s", strings.Join(nonEmpty(line.parts), " "), to, comment(mark, x.Comment))
|
||||
r.w.emit(obj, "\t\t%s counter %s", strings.Join(nonEmpty(line.parts), " "), r.nat("redirect to :"+to, mark, x.Comment))
|
||||
}
|
||||
}
|
||||
ext := st.ExternalInterfaces()
|
||||
@@ -640,7 +883,7 @@ func (r *renderer) dstnat() {
|
||||
for _, wh := range where {
|
||||
for _, sa := range srcAlts {
|
||||
for _, p := range f.Protocols {
|
||||
r.w.emit(obj, "\t\t%s counter dnat ip to %s comment %s", strings.Join(nonEmpty([]string{wh, sa, p + " dport " + f.Ports}), " "), to, comment(mark, f.Name+" "+f.Comment))
|
||||
r.w.emit(obj, "\t\t%s counter %s", strings.Join(nonEmpty([]string{wh, sa, p + " dport " + f.Ports}), " "), r.nat("dnat ip to "+to, mark, f.Name+" "+f.Comment))
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -714,7 +957,7 @@ func (r *renderer) srcnat() {
|
||||
}
|
||||
for _, sa := range srcAlts {
|
||||
for _, p := range f.Protocols {
|
||||
r.w.emit(obj, "\t\t%s counter masquerade comment %s", strings.Join(nonEmpty([]string{sa, "ip daddr " + f.To, p + " dport " + ports, "ct status dnat"}), " "), comment(mark, f.Name))
|
||||
r.w.emit(obj, "\t\t%s counter %s", strings.Join(nonEmpty([]string{sa, "ip daddr " + f.To, p + " dport " + ports, "ct status dnat"}), " "), r.nat("masquerade", mark, f.Name))
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -746,12 +989,21 @@ func (r *renderer) srcnat() {
|
||||
action = "snat ip to " + x.To
|
||||
}
|
||||
}
|
||||
r.w.emit(obj, "\t\t%s counter %s comment %s", strings.Join(nonEmpty(line.parts), " "), action, comment(mark, x.Comment))
|
||||
r.w.emit(obj, "\t\t%s counter %s", strings.Join(nonEmpty(line.parts), " "), r.nat(action, mark, x.Comment))
|
||||
}
|
||||
}
|
||||
r.end()
|
||||
}
|
||||
|
||||
// nat writes a NAT rule's statement and comment; in shadow mode only the
|
||||
// comment, saying what it would do ("[would dnat]").
|
||||
func (r *renderer) nat(action, mark, text string) string {
|
||||
if r.opt.Shadow {
|
||||
return "comment " + comment(mark, Would(strings.Fields(action)[0])+" "+text)
|
||||
}
|
||||
return action + " comment " + comment(mark, text)
|
||||
}
|
||||
|
||||
// ---- helpers
|
||||
|
||||
// hours writes a time window. nftables can't take a range past midnight,
|
||||
|
||||
+184
-11
@@ -2,6 +2,8 @@ package render
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net/netip"
|
||||
"os"
|
||||
"os/exec"
|
||||
"strings"
|
||||
@@ -66,6 +68,17 @@ func router() *model.State {
|
||||
rule(model.ChainForward, model.Rule{Match: model.Match{Destinations: []string{"wg_net"}}, Action: model.ActionDrop})
|
||||
rule(model.ChainForward, model.Rule{Match: model.Match{Services: []string{"ping"}}})
|
||||
rule(model.ChainForward, model.Rule{Match: model.Match{Sources: []string{"wg_net", "trusted", "guests"}}})
|
||||
// Bandwidth: a byte rate, per destination; per /24; quotas, total and
|
||||
// per device.
|
||||
rule(model.ChainForward, model.Rule{Match: model.Match{OutZones: []string{"lan"}}, Action: model.ActionDrop,
|
||||
Limit: &model.Limit{Rate: 5, Unit: "mbytes", Per: "second", Burst: 1, Over: true, PerDestination: true}, Comment: "5 MB/s each"})
|
||||
rule(model.ChainForward, model.Rule{Match: model.Match{InZones: []string{"wan"}}, Action: model.ActionDrop, GroupV4: 24,
|
||||
Limit: &model.Limit{Rate: 100, Unit: "kbytes", Per: "minute", Over: true, PerSource: true}, Comment: "per /24"})
|
||||
rule(model.ChainForward, model.Rule{Match: model.Match{Sources: []string{"guests"}, OutZones: []string{"wan"}}, Action: model.ActionDrop,
|
||||
Quota: &model.Quota{Bytes: 50000 << 20, Over: true, Reset: "monthly", Day: 15}, Comment: "guest quota"})
|
||||
rule(model.ChainForward, model.Rule{Match: model.Match{InZones: []string{"lan"}, OutZones: []string{"wan"}}, Action: model.ActionDrop,
|
||||
Quota: &model.Quota{Bytes: 5 << 30, Over: true, PerSource: true, Reset: "daily"}, Comment: "device quota"})
|
||||
rule(model.ChainForward, model.Rule{Match: model.Match{Sources: []string{"10.0.10.66"}}, Action: model.ActionReject, EveryPacket: true, Comment: "blocked now"})
|
||||
rule(model.ChainEarly, model.Rule{Match: model.Match{InZones: []string{"wan"}, Sources: []string{"198.51.100.0/24"}}, Action: model.ActionDrop})
|
||||
rule(model.ChainOutput, model.Rule{Raw: "meta skuid 0 counter accept", Comment: "raw"})
|
||||
st.Forwards = []*model.PortForward{
|
||||
@@ -83,12 +96,48 @@ func router() *model.State {
|
||||
st.Redirects = []*model.Redirect{
|
||||
{ID: next(), Position: 1, Match: model.Match{InZones: []string{"lan"}, NotDests: true, Destinations: []string{"10.0.10.5"}, Services: []string{"udp/53"}}, Comment: "force DNS"},
|
||||
}
|
||||
st.BanLists = []*model.BanList{
|
||||
{ID: next(), Name: "router", Scope: model.BanScopeFirewall, Never: []string{"trusted"}},
|
||||
{ID: next(), Name: "web", Scope: model.BanScopeHosts, Hosts: []string{"10.0.10.57", "2a02:8013:7f3d:10::57", "dns_servers"}},
|
||||
{ID: next(), Name: "edge", Scope: model.BanScopeEverything, GroupV6: 48},
|
||||
{ID: next(), Name: "off", Scope: model.BanScopeEverything, Disabled: true},
|
||||
}
|
||||
st.Sort()
|
||||
return st
|
||||
}
|
||||
|
||||
// banOptions are active bans in each ban list, of every port and of one.
|
||||
func banOptions(st *model.State) []Ban {
|
||||
var out []Ban
|
||||
for _, b := range st.BanLists {
|
||||
out = append(out,
|
||||
Ban{List: b.ID, Addr: netip.MustParsePrefix("203.0.113.9/32"), Timeout: 3000},
|
||||
Ban{List: b.ID, Addr: netip.MustParsePrefix("2001:db8:1:2::/64"), Proto: "tcp", Port: 22},
|
||||
Ban{List: b.ID, Addr: netip.MustParsePrefix("198.51.100.7/32"), Proto: "udp", Port: 53, Timeout: 60})
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// quotaOptions carries quota usage, as a reload does: the guest quota's,
|
||||
// and two devices' of the per-device quota.
|
||||
func quotaOptions(st *model.State) (map[string]int64, map[string][]Element) {
|
||||
q, sets := map[string]int64{}, map[string][]Element{}
|
||||
for _, r := range st.Rules {
|
||||
switch r.Comment {
|
||||
case "guest quota":
|
||||
q[QuotaName(r.ID)] = 123456
|
||||
case "device quota":
|
||||
sets[QuotaName(r.ID)+"_v4"] = []Element{{Addr: "10.0.10.20", Used: 1 << 20}}
|
||||
sets[QuotaName(r.ID)+"_v6"] = []Element{{Addr: "2a02:8013:7f3d:10::", Used: 2048}}
|
||||
}
|
||||
}
|
||||
return q, sets
|
||||
}
|
||||
|
||||
func TestRenderRouter(t *testing.T) {
|
||||
rs := Render(router(), Options{Elements: map[string][]Element{"guests": {{Addr: "10.0.30.5", Timeout: 1700}}}})
|
||||
st := router()
|
||||
q, qs := quotaOptions(st)
|
||||
rs := Render(st, Options{Elements: map[string][]Element{"guests": {{Addr: "10.0.30.5", Timeout: 1700}}}, Quotas: q, QuotaSets: qs})
|
||||
text := rs.Text
|
||||
for _, want := range []string{
|
||||
"table inet housecarl {}\ndelete table inet housecarl\ntable inet housecarl {",
|
||||
@@ -108,7 +157,16 @@ func TestRenderRouter(t *testing.T) {
|
||||
`ip saddr @trusted_v4 tcp dport { 22, 8053, 8067 } counter accept`,
|
||||
`ip6 saddr @trusted_v6 tcp dport { 22, 8053, 8067 } counter accept`,
|
||||
`iifname "enp2s0" ip6 saddr fe80::/10 udp dport 547 counter accept`,
|
||||
`iifname "wan" meta nfproto ipv4 tcp dport 22 add @meter_r`,
|
||||
`iifname "wan" meta nfproto ipv4 tcp dport 22 update @meter_r`,
|
||||
`ip6 saddr and ffff:ffff:ffff:ffff:: limit rate over 3/minute burst 5 packets }`,
|
||||
`oifname "enp2s0" meta nfproto ipv4 update @meter_r`,
|
||||
`{ ip daddr limit rate over 5 mbytes/second burst 1 mbytes } counter drop comment "hc:r`,
|
||||
`{ ip saddr and 255.255.255.0 limit rate over 100 kbytes/minute } counter drop`,
|
||||
`quota name "quota_r`,
|
||||
`{ over 50000 mbytes used 123456 bytes }`,
|
||||
`flags dynamic; size 65535; elements = { 10.0.10.20 quota over 5120 mbytes used 1048576 bytes } }`,
|
||||
`elements = { 2a02:8013:7f3d:10:: quota over 5120 mbytes used 2048 bytes } }`,
|
||||
`{ ip6 saddr and ffff:ffff:ffff:ffff:: quota over 5120 mbytes } counter drop comment "hc:r`,
|
||||
`ip saddr limit rate over 3/minute burst 5 packets } counter log prefix "hc input #`,
|
||||
`tcp dport 22 add @conn_r`,
|
||||
`counter reject with tcp reset comment`,
|
||||
@@ -139,6 +197,19 @@ func TestRenderRouter(t *testing.T) {
|
||||
t.Errorf("missing %q", want)
|
||||
}
|
||||
}
|
||||
// Rules that count traffic see every packet: in forward, after the
|
||||
// MSS clamp and before established connections are accepted.
|
||||
fwd := text[strings.Index(text, "chain forward {"):]
|
||||
quota, est, mss := strings.Index(fwd, `quota name "quota_r`), strings.Index(fwd, "hc:b-established"), strings.Index(fwd, "hc:b-mss-clamp")
|
||||
if quota < 0 || !(mss < quota && quota < est) {
|
||||
t.Errorf("the quota rule isn't between the MSS clamp and established (%d, %d, %d)", mss, quota, est)
|
||||
}
|
||||
if bw := strings.Index(fwd, "limit rate over 5 mbytes/second"); bw > est {
|
||||
t.Error("the byte rate rule comes after established")
|
||||
}
|
||||
if now := strings.Index(fwd, "blocked now"); now < 0 || now > est {
|
||||
t.Error("the every-packet rule comes after established")
|
||||
}
|
||||
// The trusted masquerade has no IPv6 part: masquerading the IPv6 LAN
|
||||
// isn't wanted (its sources are global).
|
||||
if strings.Contains(text, "ip6 saddr @trusted_v6 counter masquerade") {
|
||||
@@ -155,6 +226,31 @@ func TestRenderRouter(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestRenderDeviceCounts(t *testing.T) {
|
||||
st := router()
|
||||
st.Settings.StatsPerDevice = true
|
||||
text := Render(st, Options{DeviceSets: map[string][]Element{"stats_up_v4": {{Addr: "10.0.10.5", Timeout: 3000, Packets: 7, Bytes: 7000}}}}).Text
|
||||
for _, want := range []string{
|
||||
`set stats_up_v4 { type ipv4_addr; flags dynamic,timeout; timeout 1h; size 65535; counter; elements = { 10.0.10.5 timeout 3000s counter packets 7 bytes 7000 } }`,
|
||||
`set stats_down_v6 { type ipv6_addr; flags dynamic,timeout; timeout 1h; size 65535; counter; }`,
|
||||
`iifname != "wan" oifname "wan" update @stats_up_v4 { ip saddr } comment "hc:b-stats"`,
|
||||
`iifname "wan" oifname != "wan" update @stats_down_v6 { ip6 daddr and ffff:ffff:ffff:ffff:: } comment "hc:b-stats"`,
|
||||
} {
|
||||
if !strings.Contains(text, want) {
|
||||
t.Errorf("missing %q", want)
|
||||
}
|
||||
}
|
||||
// Counted before the established built-in, so every packet is.
|
||||
fwd := text[strings.Index(text, "chain forward {"):]
|
||||
if i := strings.Index(fwd, "update @stats_up_v4"); i < 0 || i > strings.Index(fwd, "hc:b-established") {
|
||||
t.Error("the counting comes after the established built-in")
|
||||
}
|
||||
st.Settings.StatsPerDevice = false
|
||||
if strings.Contains(Render(st, Options{}).Text, "stats_") {
|
||||
t.Error("counting without the setting")
|
||||
}
|
||||
}
|
||||
|
||||
// TestRenderedRulesetPassesNft has the real nft check the router's
|
||||
// ruleset (and a minimal one). Set HOUSECARL_NFT to how nft is run, e.g.
|
||||
//
|
||||
@@ -166,15 +262,92 @@ func TestRenderedRulesetPassesNft(t *testing.T) {
|
||||
if len(cmd) == 0 {
|
||||
t.Skip("HOUSECARL_NFT not set")
|
||||
}
|
||||
for name, st := range map[string]*model.State{"router": router(), "empty": model.NewState()} {
|
||||
rs := Render(st, Options{Elements: map[string][]Element{"guests": {{Addr: "10.0.30.5", Timeout: 1700}, {Addr: "2a02:8013:7f3d:30::5"}}}})
|
||||
c := exec.CommandContext(context.Background(), cmd[0], append(cmd[1:], "-c", "-f", "/dev/stdin")...)
|
||||
c.Stdin = strings.NewReader(rs.Text)
|
||||
if out, err := c.CombinedOutput(); err != nil {
|
||||
t.Errorf("%s: nft -c: %v\n%s", name, err, out)
|
||||
for i, l := range strings.Split(rs.Text, "\n") {
|
||||
t.Logf("%3d %s", i+1, l)
|
||||
}
|
||||
stats := router()
|
||||
stats.Settings.StatsPerDevice = true
|
||||
// VLAN sub-interfaces: zones of one VLAN, of all of a parent's, and
|
||||
// the flowtable on one.
|
||||
vlan := router()
|
||||
vlan.Zones = append(vlan.Zones, &model.Zone{ID: 900, Name: "iot", Interfaces: []string{"enp2s0.20"}},
|
||||
&model.Zone{ID: 901, Name: "vlans", Interfaces: []string{"enp3s0.*"}})
|
||||
vlan.Rules = append(vlan.Rules, &model.Rule{ID: 902, Chain: model.ChainForward, Position: 99, Action: model.ActionAccept,
|
||||
Match: model.Match{InZones: []string{"iot", "vlans"}, OutZones: []string{"wan"}}})
|
||||
vlan.Settings.FlowtableDevices = []string{"lo", "enp2s0.20"}
|
||||
for name, st := range map[string]*model.State{"router": router(), "empty": model.NewState(), "stats": stats, "vlan": vlan} {
|
||||
q, qs := quotaOptions(st)
|
||||
// Enforcing, and in shadow mode after and before other tables.
|
||||
for _, mode := range []Options{{}, {Shadow: true}, {Shadow: true, ShadowBefore: true}} {
|
||||
opt := Options{Elements: map[string][]Element{"guests": {{Addr: "10.0.30.5", Timeout: 1700}, {Addr: "2a02:8013:7f3d:30::5"}}}, Quotas: q, QuotaSets: qs, Bans: banOptions(st),
|
||||
DeviceSets: map[string][]Element{"stats_up_v4": {{Addr: "10.0.10.5", Timeout: 3000, Packets: 7, Bytes: 7000}}, "stats_down_v6": {{Addr: "2001:db8:1:2::", Packets: 1, Bytes: 100}}},
|
||||
Shadow: mode.Shadow, ShadowBefore: mode.ShadowBefore}
|
||||
nftCheck(t, cmd, fmt.Sprintf("%s (shadow %v, before %v)", name, mode.Shadow, mode.ShadowBefore), Render(st, opt).Text)
|
||||
}
|
||||
}
|
||||
nftCheck(t, cmd, "shadow fallback", ShadowFallback())
|
||||
}
|
||||
|
||||
func nftCheck(t *testing.T, cmd []string, name, text string) {
|
||||
t.Helper()
|
||||
c := exec.CommandContext(context.Background(), cmd[0], append(cmd[1:], "-c", "-f", "/dev/stdin")...)
|
||||
c.Stdin = strings.NewReader(text)
|
||||
if out, err := c.CombinedOutput(); err != nil {
|
||||
t.Errorf("%s: nft -c: %v\n%s", name, err, out)
|
||||
for i, l := range strings.Split(text, "\n") {
|
||||
t.Logf("%3d %s", i+1, l)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRenderBans(t *testing.T) {
|
||||
st := router()
|
||||
id := map[string]int64{}
|
||||
for _, b := range st.BanLists {
|
||||
id[b.Name] = b.ID
|
||||
}
|
||||
text := Render(st, Options{Bans: banOptions(st)}).Text
|
||||
f := func(format string, args ...any) string { return fmt.Sprintf(format, args...) }
|
||||
r, w, e, off := id["router"], id["web"], id["edge"], id["off"]
|
||||
for _, want := range []string{
|
||||
f("set ban_%d_v4 { type ipv4_addr; flags timeout; }", r),
|
||||
f("set ban_%d_v6_svc { type ipv6_addr . inet_proto . inet_service; flags timeout; }", r),
|
||||
// The firewall: in input, every port and one port, /64 for IPv6.
|
||||
f(`ip saddr @ban_%d_v4 counter drop comment "hc:ban%d"`, r, r),
|
||||
f(`ip saddr . meta l4proto . th dport @ban_%d_v4_svc counter drop comment "hc:ban%d"`, r, r),
|
||||
f(`ip6 saddr and ffff:ffff:ffff:ffff:: @ban_%d_v6 counter drop comment "hc:ban%d"`, r, r),
|
||||
f(`ip6 saddr and ffff:ffff:ffff:ffff:: . meta l4proto . th dport @ban_%d_v6_svc counter drop comment "hc:ban%d"`, r, r),
|
||||
// Hosts: in forward, first, to the hosts (literals and sets).
|
||||
f(`ip daddr 10.0.10.57 ip saddr @ban_%d_v4 counter drop`, w),
|
||||
f(`ip daddr @dns_servers_v4 ip saddr . meta l4proto . th dport @ban_%d_v4_svc counter drop`, w),
|
||||
f(`ip6 daddr 2a02:8013:7f3d:10::57 ip6 saddr and ffff:ffff:ffff:ffff:: @ban_%d_v6 counter drop`, w),
|
||||
// Everything: input and forward, /48.
|
||||
f(`ip6 saddr and ffff:ffff:ffff:: @ban_%d_v6 counter drop`, e),
|
||||
// The bans, with their time.
|
||||
f("add element inet housecarl ban_%d_v4 { 203.0.113.9 timeout 3000s }", r),
|
||||
f("add element inet housecarl ban_%d_v6_svc { 2001:db8:1:2:: . tcp . 22 }", w),
|
||||
f("add element inet housecarl ban_%d_v4_svc { 198.51.100.7 . udp . 53 timeout 60s }", e),
|
||||
} {
|
||||
if !strings.Contains(text, want) {
|
||||
t.Errorf("no %q in\n%s", want, text)
|
||||
}
|
||||
}
|
||||
if n := strings.Count(text, f(`@ban_%d_v4 counter drop`, e)); n != 2 {
|
||||
t.Errorf("scope everything: %d rules of every port in IPv4, want 2 (input and forward)", n)
|
||||
}
|
||||
if n := strings.Count(text, f(`@ban_%d_v4 counter drop`, w)); n != 2 {
|
||||
t.Errorf("scope hosts: %d IPv4 rules of every port, want 2 (forward: the literal host and the set)", n)
|
||||
}
|
||||
// Ahead of the accepts: the first rules after loopback, and of forward.
|
||||
in, _, _ := strings.Cut(text[strings.Index(text, "chain input"):], "established")
|
||||
fw, _, _ := strings.Cut(text[strings.Index(text, "chain forward"):], "maxseg")
|
||||
for _, part := range []string{in, fw} {
|
||||
if strings.Contains(part, " accept comment \"hc:r") {
|
||||
t.Errorf("a rule accepts before the bans:\n%s", part)
|
||||
}
|
||||
}
|
||||
if strings.Contains(text, f("ban_%d_", off)) {
|
||||
t.Errorf("a disabled list is rendered")
|
||||
}
|
||||
shadow := Render(st, Options{Shadow: true, Bans: banOptions(st)}).Text
|
||||
if want := f(`ip saddr @ban_%d_v4 counter accept comment "hc:ban%d [would drop]"`, r, r); !strings.Contains(shadow, want) {
|
||||
t.Errorf("shadow: no %q", want)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,127 @@
|
||||
package render
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"housecarl/internal/model"
|
||||
)
|
||||
|
||||
// effects are the words of a raw rule's statements that act on packets
|
||||
// (or jump away). In shadow mode a raw rule with any of them is left out,
|
||||
// unless its only one is a final drop or reject, which becomes an accept.
|
||||
var effects = map[string]bool{
|
||||
"drop": true, "reject": true, "queue": true,
|
||||
"dnat": true, "snat": true, "masquerade": true, "redirect": true, "tproxy": true,
|
||||
"notrack": true, "dup": true, "fwd": true, "synproxy": true, "flow": true, "offload": true,
|
||||
"set": true, "reset": true, "jump": true, "goto": true, "return": true,
|
||||
}
|
||||
|
||||
// ShadowRaw rewrites a raw rule for shadow mode: as written when it only
|
||||
// matches, counts, logs or accepts; with its final drop or reject made an
|
||||
// accept (marked "[would drop]" in its comment); ok is false when it does
|
||||
// anything else to packets (NAT, setting a mark or header field, a jump,
|
||||
// a verdict map…), and must be left out.
|
||||
func ShadowRaw(raw string) (out string, ok bool) {
|
||||
toks := tokens(raw)
|
||||
verdict := -1
|
||||
for i := 0; i < len(toks); i++ {
|
||||
w := word(toks[i])
|
||||
if w == "comment" {
|
||||
i++ // its text
|
||||
continue
|
||||
}
|
||||
if !effects[w] {
|
||||
continue
|
||||
}
|
||||
if verdict >= 0 || (w != "drop" && w != "reject") {
|
||||
return "", false
|
||||
}
|
||||
verdict = i
|
||||
// What follows a verdict, up to the comment, must be a reject's
|
||||
// "with …".
|
||||
j := i + 1
|
||||
for j < len(toks) && word(toks[j]) != "comment" {
|
||||
j++
|
||||
}
|
||||
if j > i+1 && (w == "drop" || word(toks[i+1]) != "with") {
|
||||
return "", false
|
||||
}
|
||||
i = j - 1
|
||||
}
|
||||
if verdict < 0 {
|
||||
return raw, true
|
||||
}
|
||||
mark := Would(word(toks[verdict]))
|
||||
head := append(append([]string{}, toks[:verdict]...), "accept")
|
||||
for i := verdict + 1; i < len(toks); i++ {
|
||||
if word(toks[i]) != "comment" {
|
||||
continue
|
||||
}
|
||||
if i+1 < len(toks) {
|
||||
text := strings.Trim(toks[i+1], `"`)
|
||||
return strings.Join(append(head, "comment", quote(strings.TrimSpace(mark+" "+text))), " "), true
|
||||
}
|
||||
}
|
||||
return strings.Join(append(head, "comment", quote(mark)), " "), true
|
||||
}
|
||||
|
||||
// tokens splits a rule into words, a quoted string being one.
|
||||
func tokens(s string) []string {
|
||||
var out []string
|
||||
var cur strings.Builder
|
||||
quoted := false
|
||||
for _, c := range s {
|
||||
switch {
|
||||
case c == '"':
|
||||
quoted = !quoted
|
||||
cur.WriteRune(c)
|
||||
case !quoted && (c == ' ' || c == '\t'):
|
||||
if cur.Len() > 0 {
|
||||
out = append(out, cur.String())
|
||||
cur.Reset()
|
||||
}
|
||||
default:
|
||||
cur.WriteRune(c)
|
||||
}
|
||||
}
|
||||
if cur.Len() > 0 {
|
||||
out = append(out, cur.String())
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// word is a token as a keyword: without the punctuation of sets and maps
|
||||
// around it; a quoted string is no keyword.
|
||||
func word(t string) string {
|
||||
if strings.HasPrefix(t, `"`) {
|
||||
return ""
|
||||
}
|
||||
return strings.ToLower(strings.Trim(t, ",{}:()"))
|
||||
}
|
||||
|
||||
// ShadowLeftOut lists the raw rules shadow mode leaves out (ShadowRaw).
|
||||
func ShadowLeftOut(st *model.State) []*model.Rule {
|
||||
var out []*model.Rule
|
||||
for _, r := range st.Rules {
|
||||
if r.Raw == "" || r.Disabled {
|
||||
continue
|
||||
}
|
||||
if _, ok := ShadowRaw(r.Raw); !ok {
|
||||
out = append(out, r)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// ShadowFallback is the boot fallback in shadow mode: the other firewall
|
||||
// protects the machine, so it is an empty table that filters nothing,
|
||||
// with a rule outside every hook that tells housecarld it is in force.
|
||||
func ShadowFallback() string {
|
||||
var b strings.Builder
|
||||
fmt.Fprintf(&b, "# Housecarl's fallback in shadow mode: loaded at boot only if ruleset.nft\n")
|
||||
fmt.Fprintf(&b, "# fails to load. It filters nothing; the other firewall decides.\n")
|
||||
fmt.Fprintf(&b, "table inet %s {}\ndelete table inet %s\n", Table, Table)
|
||||
fmt.Fprintf(&b, "table inet %s {\n\tchain fallback {\n\t\tcounter comment \"hc:%s-shadow\"\n\t}\n}\n", Table, FallbackMark)
|
||||
return b.String()
|
||||
}
|
||||
@@ -0,0 +1,91 @@
|
||||
package render
|
||||
|
||||
import (
|
||||
"regexp"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"housecarl/internal/model"
|
||||
)
|
||||
|
||||
// TestShadow: in shadow mode nothing in the ruleset drops, rejects,
|
||||
// translates or changes a packet, and what would is counted and marked.
|
||||
func TestShadow(t *testing.T) {
|
||||
st := router()
|
||||
st.Rules = append(st.Rules,
|
||||
&model.Rule{ID: 950, Chain: model.ChainInput, Position: 99, Raw: `tcp dport 9999 counter reject with tcp reset comment "old"`},
|
||||
&model.Rule{ID: 951, Chain: model.ChainForward, Position: 99, Raw: "meta mark set 0x1 accept"})
|
||||
st.Settings.OutputPolicy = model.PolicyDrop
|
||||
text := Render(st, Options{Shadow: true}).Text
|
||||
// What acts on packets: verdicts, NAT, changes, the flowtable.
|
||||
acts := regexp.MustCompile(`(?m)(\b(drop|reject|dnat|snat|masquerade|redirect)\b|maxseg size set|flow add|flowtable|policy drop)[^"\n]*$`)
|
||||
for _, l := range strings.Split(text, "\n") {
|
||||
code, _, _ := strings.Cut(l, "comment ")
|
||||
code = strings.ReplaceAll(code, "ct status dnat", "") // a match
|
||||
if strings.HasPrefix(strings.TrimSpace(l), "#") {
|
||||
continue
|
||||
}
|
||||
if m := acts.FindString(code); m != "" && !strings.Contains(code, "log prefix") {
|
||||
t.Errorf("acts on packets in shadow mode: %s", l)
|
||||
}
|
||||
}
|
||||
for _, want := range []string{
|
||||
"type filter hook input priority filter + 10; policy accept;",
|
||||
"type filter hook prerouting priority raw + 10; policy accept;",
|
||||
"type nat hook prerouting priority dstnat - 10; policy accept;",
|
||||
"type nat hook postrouting priority srcnat - 10; policy accept;",
|
||||
`ct state invalid counter accept comment "hc:b-invalid [would drop]"`,
|
||||
`counter comment "hc:p-input [would drop]"`,
|
||||
`counter comment "hc:p-output [would drop]"`,
|
||||
`log prefix "hc output policy drop: "`,
|
||||
`[would reject]`,
|
||||
`[would dnat] jellyfin`,
|
||||
`[would masquerade]`,
|
||||
`[would redirect]`,
|
||||
`tcp dport 9999 counter accept comment "[would reject] old"`,
|
||||
"# left out in shadow mode: meta mark set 0x1 accept",
|
||||
"# Shadow mode:",
|
||||
} {
|
||||
if !strings.Contains(text, want) {
|
||||
t.Errorf("no %q in\n%s", want, text)
|
||||
}
|
||||
}
|
||||
before := Render(st, Options{Shadow: true, ShadowBefore: true}).Text
|
||||
for _, want := range []string{"priority filter - 10;", "priority raw - 10;", "priority dstnat - 10;"} {
|
||||
if !strings.Contains(before, want) {
|
||||
t.Errorf("before: no %q", want)
|
||||
}
|
||||
}
|
||||
// Enforcing is unchanged by it.
|
||||
if strings.Contains(Render(st, Options{}).Text, "would") {
|
||||
t.Error("shadow markers when enforcing")
|
||||
}
|
||||
if got := ShadowLeftOut(st); len(got) != 1 || got[0].ID != 951 {
|
||||
t.Errorf("left out: %v", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestShadowRaw(t *testing.T) {
|
||||
cases := []struct{ in, out string }{
|
||||
{"meta skuid 0 counter accept", "meta skuid 0 counter accept"},
|
||||
{"tcp dport 22 drop", `tcp dport 22 accept comment "[would drop]"`},
|
||||
{`tcp dport 22 counter drop comment "no ssh"`, `tcp dport 22 counter accept comment "[would drop] no ssh"`},
|
||||
{"ip saddr 10.0.0.1 reject with icmpx type admin-prohibited", `ip saddr 10.0.0.1 accept comment "[would reject]"`},
|
||||
{"tcp dport 22 reject with tcp reset", `tcp dport 22 accept comment "[would reject]"`},
|
||||
{`log prefix "drop me: " group 5 accept`, `log prefix "drop me: " group 5 accept`},
|
||||
{"ip saddr { 10.0.0.1, 10.0.0.2 } counter", "ip saddr { 10.0.0.1, 10.0.0.2 } counter"},
|
||||
{"tcp dport vmap { 22 : drop, 80 : accept }", ""},
|
||||
{"meta mark set 1", ""},
|
||||
{"ip daddr 1.2.3.4 dnat ip to 10.0.0.1", ""},
|
||||
{"tcp dport 22 jump other", ""},
|
||||
{"drop accept", ""},
|
||||
{"meta nftrace set 1", ""},
|
||||
{"ct state new notrack", ""},
|
||||
}
|
||||
for _, c := range cases {
|
||||
out, ok := ShadowRaw(c.in)
|
||||
if ok != (c.out != "") || out != c.out {
|
||||
t.Errorf("%q: got %q %v, want %q", c.in, out, ok, c.out)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -102,6 +102,8 @@ func (s *Server) authenticate(r *http.Request) (*auth.Principal, *model.Session,
|
||||
p, err := s.tokenPrincipal(r.Context(), strings.TrimSpace(tok), ip, r.UserAgent())
|
||||
if err != nil {
|
||||
s.limiter.fail("ip:" + ip)
|
||||
// For Guard and fail2ban: the address first, so a name can't fake it.
|
||||
s.log.Warn("token refused", "ip", ip, "reason", err.Error())
|
||||
s.eng.Audit(r.Context(), &auth.Principal{IP: ip, Method: "token"}, "auth.token_failed", "", "failure", err.Error())
|
||||
return nil, nil, &httpError{http.StatusUnauthorized, "unauthenticated", "invalid API token: " + err.Error()}
|
||||
}
|
||||
@@ -246,6 +248,8 @@ func (s *Server) login(w http.ResponseWriter, r *http.Request) error {
|
||||
if err == nil && u.Disabled && ok {
|
||||
reason = "account disabled"
|
||||
}
|
||||
// For Guard and fail2ban: the address first, so a name can't fake it.
|
||||
s.log.Warn("login failed", "ip", ip, "user", name, "reason", reason)
|
||||
s.eng.Audit(r.Context(), who, "auth.login_failed", "user "+name, "failure", reason)
|
||||
return &httpError{http.StatusUnauthorized, "login_failed", "wrong username or password"}
|
||||
}
|
||||
@@ -275,7 +279,7 @@ func (s *Server) logout(w http.ResponseWriter, r *http.Request, p *auth.Principa
|
||||
}
|
||||
|
||||
func (s *Server) whoamiOf(p *auth.Principal, se *model.Session) api.Whoami {
|
||||
out := api.Whoami{User: p.User, Admin: p.IsAdmin(), Method: p.Method, Permissions: p.Permissions(), Build: version.Info()}
|
||||
out := api.Whoami{User: p.User, Admin: p.IsAdmin(), Method: p.Method, Permissions: p.Permissions(), Build: version.Info(), Mode: s.mode()}
|
||||
if se != nil {
|
||||
out.CSRFToken = se.CSRFToken
|
||||
}
|
||||
|
||||
@@ -0,0 +1,154 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"log/slog"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"housecarl/internal/api"
|
||||
"housecarl/internal/model"
|
||||
"housecarl/internal/store"
|
||||
)
|
||||
|
||||
// TestBans: a program with a token for one ban list bans addresses: of
|
||||
// a service's ports, every port, or numbers; renewed when banned again;
|
||||
// IPv6 by /64; never the inside, the firewall's own, or the list's
|
||||
// never; times cut to the list's longest; the address's connections the
|
||||
// ban drops are closed; and lifted by ID or address.
|
||||
func TestBans(t *testing.T) {
|
||||
e := newEnv(t)
|
||||
e.call(e.token, "POST", "/zones", model.Zone{Name: "wan", Interfaces: []string{"wan"}, External: true}, nil)
|
||||
e.call(e.token, "POST", "/zones", model.Zone{Name: "lan", Interfaces: []string{"eth1"}}, nil)
|
||||
for _, b := range []model.BanList{
|
||||
{Name: "web", Scope: model.BanScopeHosts, Hosts: []string{"10.0.10.57"}, Never: []string{"198.51.100.0/24"}, MaxTime: 3600},
|
||||
{Name: "router", Scope: model.BanScopeFirewall},
|
||||
} {
|
||||
if code := e.call(e.token, "POST", "/banlists", b, nil); code != 201 {
|
||||
t.Fatalf("ban list %s: %d", b.Name, code)
|
||||
}
|
||||
}
|
||||
fc := &fakeConns{list: []api.Connection{
|
||||
{Family: "ipv4", Protocol: "tcp", Source: "203.0.113.9", SourcePort: 40000, Destination: "217.155.8.194", DestinationPort: 2222, DNATTo: "10.0.10.57", DNATPort: 22},
|
||||
{Family: "ipv4", Protocol: "tcp", Source: "203.0.113.9", SourcePort: 40001, Destination: "10.0.10.58", DestinationPort: 22},
|
||||
{Family: "ipv4", Protocol: "tcp", Source: "203.0.113.9", SourcePort: 40002, Destination: "217.155.8.194", DestinationPort: 22},
|
||||
}}
|
||||
e.srv.SetConntrack(fc)
|
||||
var tok api.Token
|
||||
if code := e.call(e.token, "POST", "/me/tokens", api.TokenInput{Name: "guard", Grants: []string{"bans:write:web"}}, &tok); code != 201 {
|
||||
t.Fatalf("token: %d", code)
|
||||
}
|
||||
ban := func(in api.BanInput, want int) api.BanResult {
|
||||
t.Helper()
|
||||
var out api.BanResult
|
||||
var eb api.ErrorBody
|
||||
code := e.call(tok.Secret, "POST", "/banlists/web/bans", in, &out)
|
||||
if code != want {
|
||||
e.call(tok.Secret, "POST", "/banlists/web/bans", in, &eb)
|
||||
t.Errorf("ban %+v: %d, want %d (%+v)", in, code, want, eb)
|
||||
}
|
||||
return out
|
||||
}
|
||||
out := ban(api.BanInput{Address: "203.0.113.9", Ports: []byte(`"ssh"`), Time: 600, Source: "guard/ssh", Reason: "Failed password"}, 201)
|
||||
if len(out.Bans) != 1 || out.Bans[0].Protocol != "tcp" || out.Bans[0].Port != 22 || out.Bans[0].Existing || out.Bans[0].Timeout != 600 {
|
||||
t.Errorf("ssh ban: %+v", out)
|
||||
}
|
||||
if out.Closed != 1 || len(fc.closed) != 1 || fc.closed[0].DNATTo != "10.0.10.57" {
|
||||
t.Errorf("closed %d: %+v (only the one to the list's host)", out.Closed, fc.closed)
|
||||
}
|
||||
if _, ok := e.fw.bans[`ban_`+itoa(out.Bans[0].ListID)+`_v4_svc 203.0.113.9 . tcp . 22`]; !ok {
|
||||
t.Errorf("not in force: %+v", e.fw.bans)
|
||||
}
|
||||
if again := ban(api.BanInput{Address: "203.0.113.9", Ports: []byte(`22`), Protocol: "tcp"}, 201); len(again.Bans) != 1 || !again.Bans[0].Existing {
|
||||
t.Errorf("banned again: %+v", again)
|
||||
}
|
||||
if v6 := ban(api.BanInput{Address: "2001:db8:1:2::77", Time: 99999}, 201); len(v6.Bans) != 1 || v6.Bans[0].Address != "2001:db8:1:2::/64" ||
|
||||
v6.Bans[0].Protocol != "" || v6.Bans[0].Timeout != 3600 {
|
||||
t.Errorf("IPv6, every port, cut to an hour: %+v", v6)
|
||||
}
|
||||
if n := ban(api.BanInput{Address: "::ffff:203.0.113.10", Ports: []byte(`53`)}, 201); len(n.Bans) != 2 || n.Bans[0].Address != "203.0.113.10" {
|
||||
t.Errorf("a number without a protocol is tcp and udp: %+v", n)
|
||||
}
|
||||
if n := ban(api.BanInput{Address: "203.0.113.11", Protocol: "tcp", Ports: []byte(`["http", "https"]`)}, 201); len(n.Bans) != 2 {
|
||||
t.Errorf("two services: %+v", n)
|
||||
}
|
||||
if n := ban(api.BanInput{Address: "203.0.113.12", Protocol: "tcp", Ports: []byte(`"0:65535"`)}, 201); len(n.Bans) != 1 || n.Bans[0].Protocol != "" {
|
||||
t.Errorf("fail2ban's every port: %+v", n)
|
||||
}
|
||||
ban(api.BanInput{Address: "203.0.113.13", Ports: []byte(`"bogus"`)}, 400)
|
||||
ban(api.BanInput{Address: "203.0.113.13", Protocol: "tcp"}, 400)
|
||||
ban(api.BanInput{Address: "not-an-address"}, 400)
|
||||
for _, a := range []string{"198.51.100.4", "10.0.10.99", "217.155.8.194", "127.0.0.1", "fe80::1", "192.168.5.5"} {
|
||||
var eb api.ErrorBody
|
||||
if code := e.call(tok.Secret, "POST", "/banlists/web/bans", api.BanInput{Address: a}, &eb); code != 422 || eb.Error.Code != "never_ban" {
|
||||
t.Errorf("never ban %s: %d %+v", a, code, eb)
|
||||
}
|
||||
}
|
||||
if code := e.call(tok.Secret, "POST", "/banlists/router/bans", api.BanInput{Address: "203.0.113.9"}, nil); code != 403 {
|
||||
t.Errorf("another list: %d", code)
|
||||
}
|
||||
// The program may read its list, and only its list.
|
||||
var bl model.BanList
|
||||
if code := e.call(tok.Secret, "GET", "/banlists/web", nil, &bl); code != 200 || bl.V6Group() != 64 {
|
||||
t.Errorf("reading its list: %d %+v", code, bl)
|
||||
}
|
||||
var lists []model.BanList
|
||||
if e.call(tok.Secret, "GET", "/banlists", nil, &lists); len(lists) != 1 || lists[0].Name != "web" {
|
||||
t.Errorf("lists: %+v", lists)
|
||||
}
|
||||
if code := e.call(tok.Secret, "GET", "/banlists/router", nil, nil); code != 403 {
|
||||
t.Errorf("reading another list: %d", code)
|
||||
}
|
||||
var bans []model.Ban
|
||||
e.call(tok.Secret, "GET", "/banlists/web/bans", nil, &bans)
|
||||
if len(bans) != 7 || bans[0].List != "web" || bans[0].Who != "admin" || bans[0].Source != "guard/ssh" {
|
||||
t.Errorf("bans: %d %+v", len(bans), bans)
|
||||
}
|
||||
var removed map[string]int
|
||||
if code := e.call(tok.Secret, "DELETE", "/banlists/web/bans?address=2001:db8:1:2::5", nil, &removed); code != 200 || removed["removed"] != 1 {
|
||||
t.Errorf("lift by address: %d %+v", code, removed)
|
||||
}
|
||||
if code := e.call(tok.Secret, "DELETE", "/banlists/web/bans/"+itoa(bans[0].ID), nil, &removed); code != 200 || removed["removed"] != 1 {
|
||||
t.Errorf("lift by ID: %d %+v", code, removed)
|
||||
}
|
||||
if code := e.call(tok.Secret, "DELETE", "/banlists/web/bans?address=203.0.113.200", nil, nil); code != 404 {
|
||||
t.Errorf("lifting what isn't banned: %d", code)
|
||||
}
|
||||
if _, ok := e.fw.bans[`ban_`+itoa(bans[0].ListID)+`_v4_svc 203.0.113.9 . tcp . 22`]; ok {
|
||||
t.Errorf("still in force after lifting")
|
||||
}
|
||||
evs, _ := e.srv.store.ListAudit(context.Background(), store.AuditFilter{Action: "ban."})
|
||||
if len(evs) < 10 || !strings.Contains(evs[len(evs)-1].Detail, "guard/ssh: Failed password") {
|
||||
t.Errorf("audit: %d %+v", len(evs), evs[len(evs)-1])
|
||||
}
|
||||
// The list's name is what programs know it by.
|
||||
if code := e.call(e.token, "PATCH", "/banlists/web", map[string]any{"name": "web2"}, nil); code != 409 {
|
||||
t.Errorf("renaming a ban list: %d", code)
|
||||
}
|
||||
// Every apply puts the bans back; a deleted list's go.
|
||||
if got := ActiveBans(context.Background(), e.srv.store); len(got) != 5 {
|
||||
t.Errorf("active bans: %d", len(got))
|
||||
}
|
||||
if code := e.call(e.token, "DELETE", "/banlists/web", nil, nil); code != 200 {
|
||||
t.Errorf("deleting the list: %d", code)
|
||||
}
|
||||
if got := ActiveBans(context.Background(), e.srv.store); len(got) != 0 {
|
||||
t.Errorf("a deleted list's bans: %d", len(got))
|
||||
}
|
||||
}
|
||||
|
||||
// TestFailedLoginLogged: a failed login and a refused token are logged
|
||||
// with the address first, for Guard's and fail2ban's patterns.
|
||||
func TestFailedLoginLogged(t *testing.T) {
|
||||
e := newEnv(t)
|
||||
var buf bytes.Buffer
|
||||
e.srv.log = slog.New(slog.NewTextHandler(&buf, nil))
|
||||
e.call("", "POST", "/auth/login", api.Login{Username: `x ip=8.8.8.8`, Password: "wrong"}, nil)
|
||||
e.call("housecarl_aaaaaaaaaaaaaaaa_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", "GET", "/status", nil, nil)
|
||||
for _, want := range []string{`msg="login failed" ip=127.0.0.1 user="x ip=8.8.8.8"`, `msg="token refused" ip=127.0.0.1 reason=`} {
|
||||
if !strings.Contains(buf.String(), want) {
|
||||
t.Errorf("no %q in\n%s", want, buf.String())
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,72 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
udiff "github.com/aymanbagabas/go-udiff"
|
||||
|
||||
"housecarl/internal/api"
|
||||
"housecarl/internal/model"
|
||||
)
|
||||
|
||||
// TestBlockHeaders: each hunk says which chain it is in, even when the
|
||||
// chain's own line is far above it.
|
||||
func TestBlockHeaders(t *testing.T) {
|
||||
var b strings.Builder
|
||||
b.WriteString("table inet housecarl {\n\tset lan_v4 { type ipv4_addr; }\n\tchain input {\n")
|
||||
for i := 0; i < 20; i++ {
|
||||
b.WriteString("\t\tcounter comment \"in\"\n")
|
||||
}
|
||||
b.WriteString("\t}\n\tchain forward {\n")
|
||||
for i := 0; i < 20; i++ {
|
||||
b.WriteString("\t\tcounter comment \"fw\"\n")
|
||||
}
|
||||
b.WriteString("\t}\n}\n")
|
||||
now := b.String()
|
||||
then := strings.Replace(now, "\t\tcounter comment \"in\"\n\t}", "\t\tcounter comment \"in\"\n\t\taccept comment \"new in\"\n\t}", 1)
|
||||
then = strings.Replace(then, "\t\tcounter comment \"fw\"\n\t}", "\t\taccept comment \"new fw\"\n\t}", 1)
|
||||
|
||||
diff := blockHeaders(udiff.Unified("a", "b", now, then), now, then)
|
||||
var heads []string
|
||||
for _, l := range strings.Split(diff, "\n") {
|
||||
if strings.HasPrefix(l, "@@") {
|
||||
heads = append(heads, l)
|
||||
}
|
||||
}
|
||||
if len(heads) != 2 || !strings.HasSuffix(heads[0], "@@ chain input") || !strings.HasSuffix(heads[1], "@@ chain forward") {
|
||||
t.Fatalf("hunk headers %q, want chain input then chain forward\n%s", heads, diff)
|
||||
}
|
||||
if blockHeaders("", now, then) != "" {
|
||||
t.Fatal("no diff should stay empty")
|
||||
}
|
||||
}
|
||||
|
||||
// TestHistoryDiff: an entry's diff is its own change, rebuilt from the
|
||||
// current configuration however many entries came after it.
|
||||
func TestHistoryDiff(t *testing.T) {
|
||||
e := newEnv(t)
|
||||
var mr api.MutationResponse
|
||||
e.call(e.token, "POST", "/zones", model.Zone{Name: "lan", Interfaces: []string{"eth1"}}, nil)
|
||||
e.call(e.token, "POST", "/rules", model.Rule{Chain: "input", Match: model.Match{Services: []string{"ssh"}}, Comment: "first"}, &mr)
|
||||
first := mr.HistoryID
|
||||
e.call(e.token, "POST", "/rules", model.Rule{Chain: "forward", Match: model.Match{Services: []string{"http"}}, Comment: "second"}, nil)
|
||||
e.call(e.token, "PATCH", "/settings", map[string]any{"builtins": map[string]any{"drop_invalid": true}}, nil)
|
||||
|
||||
var out struct{ Diff string }
|
||||
if code := e.call(e.token, "GET", "/history/"+itoa(first)+"/diff", nil, &out); code != 200 {
|
||||
t.Fatalf("diff: %d", code)
|
||||
}
|
||||
var added []string
|
||||
for _, l := range strings.Split(out.Diff, "\n") {
|
||||
if strings.HasPrefix(l, "+") && !strings.HasPrefix(l, "+++") {
|
||||
added = append(added, l)
|
||||
}
|
||||
}
|
||||
if len(added) == 0 || !strings.Contains(strings.Join(added, "\n"), ` first"`) || strings.Contains(out.Diff, ` second"`) || !strings.Contains(out.Diff, "@@ chain input") {
|
||||
t.Errorf("entry %d's diff:\n%s", first, out.Diff)
|
||||
}
|
||||
if code := e.call(e.token, "GET", "/history/9999/diff", nil, nil); code != 404 {
|
||||
t.Errorf("unknown entry: %d", code)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,555 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/netip"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"housecarl/internal/api"
|
||||
"housecarl/internal/auth"
|
||||
"housecarl/internal/model"
|
||||
"housecarl/internal/render"
|
||||
"housecarl/internal/store"
|
||||
)
|
||||
|
||||
// Ban lists (docs/bans.md): the lists are configuration objects; their
|
||||
// bans are kept apart, like a dynamic set's entries, audited but not in
|
||||
// the history or on probation. A ban is saved first, then put in force;
|
||||
// every apply puts all of them back (ActiveBans).
|
||||
|
||||
// banList finds a ban list the principal may read or ban into.
|
||||
func (s *Server) banList(r *http.Request, p *auth.Principal, write bool) (*model.BanList, error) {
|
||||
ref := r.PathValue("ref")
|
||||
o := s.eng.State().Find(model.KindBanList, ref)
|
||||
if o == nil {
|
||||
if !p.CanBan(ref, write) {
|
||||
return nil, banForbidden(ref, write)
|
||||
}
|
||||
return nil, notFound("no ban list %s", ref)
|
||||
}
|
||||
b := o.(*model.BanList)
|
||||
if !p.CanBan(b.Name, write) {
|
||||
return nil, banForbidden(b.Name, write)
|
||||
}
|
||||
return b, nil
|
||||
}
|
||||
|
||||
func banForbidden(list string, write bool) error {
|
||||
access := "read"
|
||||
if write {
|
||||
access = "write"
|
||||
}
|
||||
return forbidden("permission denied: this needs bans:%s for %s", access, list)
|
||||
}
|
||||
|
||||
// ActiveBans are the bans to put back with every apply (nft.Firewall's
|
||||
// Bans).
|
||||
func ActiveBans(ctx context.Context, st *store.Store) []render.Ban {
|
||||
list, err := st.ListBans(ctx, 0)
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
out := make([]render.Ban, 0, len(list))
|
||||
for _, b := range list {
|
||||
if rb, ok := renderBan(b); ok {
|
||||
out = append(out, rb)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func renderBan(b *model.Ban) (render.Ban, bool) {
|
||||
p, err := b.Prefix()
|
||||
if err != nil {
|
||||
return render.Ban{}, false
|
||||
}
|
||||
return render.Ban{List: b.ListID, Addr: p, Proto: b.Protocol, Port: b.Port, Timeout: b.Timeout}, true
|
||||
}
|
||||
|
||||
// listBans answers GET /banlists/{ref}/bans, and GET /bans: every list's
|
||||
// the principal may read.
|
||||
func (s *Server) listBans(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
st := s.eng.State()
|
||||
var id int64
|
||||
if r.PathValue("ref") != "" {
|
||||
b, err := s.banList(r, p, false)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
id = b.ID
|
||||
}
|
||||
all, err := s.store.ListBans(r.Context(), id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
out := []*model.Ban{}
|
||||
for _, b := range all {
|
||||
o := st.ByID(model.KindBanList, b.ListID)
|
||||
if o == nil || !p.CanBan(o.Label(), false) {
|
||||
continue
|
||||
}
|
||||
b.List = o.Label()
|
||||
out = append(out, b)
|
||||
}
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
return nil
|
||||
}
|
||||
|
||||
// addBan answers POST /banlists/{ref}/bans.
|
||||
func (s *Server) addBan(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
list, err := s.banList(r, p, true)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var in api.BanInput
|
||||
if err := decodeJSON(r, &in); err != nil {
|
||||
return err
|
||||
}
|
||||
st := s.eng.State()
|
||||
if list.Disabled {
|
||||
return conflict("ban list %s is disabled: enable it first", list.Name)
|
||||
}
|
||||
addr, err := netip.ParseAddr(strings.Trim(strings.TrimSpace(in.Address), "[]"))
|
||||
if err != nil {
|
||||
return badRequest("%q isn't an address", in.Address)
|
||||
}
|
||||
addr = addr.Unmap().WithZone("")
|
||||
bits := 32
|
||||
if addr.Is6() {
|
||||
bits = list.V6Group()
|
||||
}
|
||||
prefix := netip.PrefixFrom(addr, bits).Masked()
|
||||
if why := s.neverBan(r.Context(), st, list, p, addr, prefix); why != "" {
|
||||
s.eng.Audit(r.Context(), p, "ban.add", list.Name+" "+model.BanAddress(prefix), "failure", why)
|
||||
return &httpError{http.StatusUnprocessableEntity, "never_ban", fmt.Sprintf("%s isn't banned: %s", model.BanAddress(prefix), why)}
|
||||
}
|
||||
ports, err := banPorts(st, in.Protocol, in.Ports)
|
||||
if err != nil {
|
||||
return badRequest("%v", err)
|
||||
}
|
||||
var expires *time.Time
|
||||
switch secs := banSeconds(list, in.Time); {
|
||||
case secs > 0:
|
||||
t := s.nowFn().Add(time.Duration(secs) * time.Second)
|
||||
expires = &t
|
||||
}
|
||||
if n, err := s.store.CountBans(r.Context(), list.ID); err != nil {
|
||||
return err
|
||||
} else if n+len(ports) > list.Entries() {
|
||||
existing := 0
|
||||
for _, pp := range ports {
|
||||
if s.banExists(r.Context(), list.ID, model.BanAddress(prefix), pp) {
|
||||
existing++
|
||||
}
|
||||
}
|
||||
if n+len(ports)-existing > list.Entries() {
|
||||
return &httpError{http.StatusUnprocessableEntity, "list_full", fmt.Sprintf("ban list %s holds its most bans (%d, max_entries); wait for some to end, or raise it", list.Name, list.Entries())}
|
||||
}
|
||||
}
|
||||
out := api.BanResult{Bans: []api.BanMade{}}
|
||||
for _, pp := range ports {
|
||||
b := &model.Ban{ListID: list.ID, List: list.Name, Address: model.BanAddress(prefix), Protocol: pp.proto, Port: pp.port,
|
||||
Expires: expires, Who: p.Name(), Source: clip(in.Source, 100), Reason: clip(in.Reason, 500)}
|
||||
existing, err := s.store.PutBan(r.Context(), b)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if expires != nil {
|
||||
b.Timeout = int(time.Until(*expires).Round(time.Second) / time.Second)
|
||||
}
|
||||
rb, _ := renderBan(b)
|
||||
if err := s.fw.AddBan(r.Context(), rb); err != nil {
|
||||
s.log.Warn("putting a ban in force failed; the next change or restart adds it", "list", list.Name, "address", b.Address, "err", err)
|
||||
}
|
||||
out.Bans = append(out.Bans, api.BanMade{Ban: *b, Existing: existing})
|
||||
detail := b.Source
|
||||
if b.Reason != "" {
|
||||
detail = strings.TrimSpace(detail + ": " + b.Reason)
|
||||
}
|
||||
s.eng.Audit(r.Context(), p, "ban.add", list.Name+" "+banLabel(b), "success", detail)
|
||||
}
|
||||
out.Closed = s.closeBanned(r.Context(), st, list, prefix, ports)
|
||||
writeJSON(w, http.StatusCreated, out)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Server) banExists(ctx context.Context, list int64, addr string, pp banPort) bool {
|
||||
bans, _ := s.store.ListBans(ctx, list)
|
||||
for _, b := range bans {
|
||||
if b.Address == addr && b.Protocol == pp.proto && b.Port == pp.port {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// banSeconds is a ban's length: what was asked (0 the list's default, -1
|
||||
// its longest), cut to the list's longest. 0 means it doesn't end.
|
||||
func banSeconds(list *model.BanList, asked int) int {
|
||||
max := list.BanMaxTime()
|
||||
secs := asked
|
||||
switch {
|
||||
case asked == 0:
|
||||
secs = list.BanTime()
|
||||
case asked < 0:
|
||||
secs = max
|
||||
}
|
||||
if max > 0 && secs > max {
|
||||
secs = max
|
||||
}
|
||||
return secs
|
||||
}
|
||||
|
||||
func banLabel(b *model.Ban) string {
|
||||
if b.Protocol == "" {
|
||||
return b.Address
|
||||
}
|
||||
return fmt.Sprintf("%s %s/%d", b.Address, b.Protocol, b.Port)
|
||||
}
|
||||
|
||||
func clip(s string, n int) string {
|
||||
s = strings.TrimSpace(s)
|
||||
if len(s) > n {
|
||||
return s[:n]
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
// banPort is one protocol and port banned; proto "" is every port.
|
||||
type banPort struct {
|
||||
proto string
|
||||
port int
|
||||
}
|
||||
|
||||
// maxBanPorts is the most ports one ban request may name: a range in a
|
||||
// service counts each port.
|
||||
const maxBanPorts = 64
|
||||
|
||||
// banPorts reads a ban request's protocol and ports (api.BanInput).
|
||||
func banPorts(st *model.State, protocol string, raw json.RawMessage) ([]banPort, error) {
|
||||
protocol = strings.ToLower(strings.TrimSpace(protocol))
|
||||
if protocol != "" && protocol != "tcp" && protocol != "udp" && protocol != "all" {
|
||||
return nil, fmt.Errorf("protocol %q: use tcp, udp, or leave it out for every port", protocol)
|
||||
}
|
||||
if protocol == "all" {
|
||||
protocol = ""
|
||||
}
|
||||
var words []string
|
||||
if len(raw) > 0 && string(raw) != "null" {
|
||||
var list []json.RawMessage
|
||||
if json.Unmarshal(raw, &list) != nil {
|
||||
list = []json.RawMessage{raw}
|
||||
}
|
||||
for _, item := range list {
|
||||
var n int
|
||||
var str string
|
||||
switch {
|
||||
case json.Unmarshal(item, &n) == nil:
|
||||
words = append(words, strconv.Itoa(n))
|
||||
case json.Unmarshal(item, &str) == nil:
|
||||
words = append(words, strings.FieldsFunc(str, func(c rune) bool { return c == ',' || c == ' ' })...)
|
||||
default:
|
||||
return nil, fmt.Errorf("ports: give numbers or service names, in a list or a string")
|
||||
}
|
||||
}
|
||||
}
|
||||
if len(words) == 0 {
|
||||
if protocol != "" {
|
||||
return nil, fmt.Errorf("protocol %s without ports: give the ports, or leave the protocol out for every port", protocol)
|
||||
}
|
||||
return []banPort{{}}, nil
|
||||
}
|
||||
protos := []string{"tcp", "udp"}
|
||||
if protocol != "" {
|
||||
protos = []string{protocol}
|
||||
}
|
||||
var out []banPort
|
||||
seen := map[banPort]bool{}
|
||||
add := func(proto string, port int) {
|
||||
bp := banPort{proto, port}
|
||||
if !seen[bp] {
|
||||
seen[bp] = true
|
||||
out = append(out, bp)
|
||||
}
|
||||
}
|
||||
for _, w := range words {
|
||||
lw := strings.ToLower(w)
|
||||
switch lw {
|
||||
case "all", "any", "0:65535", "1:65535", "0-65535", "1-65535":
|
||||
return []banPort{{}}, nil
|
||||
}
|
||||
if n, err := strconv.Atoi(w); err == nil {
|
||||
if n < 1 || n > 65535 {
|
||||
return nil, fmt.Errorf("port %d: use 1 to 65535", n)
|
||||
}
|
||||
for _, p := range protos {
|
||||
add(p, n)
|
||||
}
|
||||
continue
|
||||
}
|
||||
svc := st.Service(w)
|
||||
if svc == nil {
|
||||
return nil, fmt.Errorf("%q is neither a port nor a service (built-in or yours)", w)
|
||||
}
|
||||
matched := false
|
||||
for _, e := range st.ServiceEntries([]string{svc.Name}) {
|
||||
if e.Proto != "tcp" && e.Proto != "udp" || protocol != "" && e.Proto != protocol {
|
||||
continue
|
||||
}
|
||||
for _, part := range strings.Split(e.Ports, ",") {
|
||||
lo, hi, _ := strings.Cut(strings.TrimSpace(part), "-")
|
||||
a, err1 := strconv.Atoi(lo)
|
||||
b := a
|
||||
var err2 error
|
||||
if hi != "" {
|
||||
b, err2 = strconv.Atoi(hi)
|
||||
}
|
||||
if err1 != nil || err2 != nil || b-a >= maxBanPorts {
|
||||
return nil, fmt.Errorf("service %s has the range %s: ban every port instead", svc.Name, part)
|
||||
}
|
||||
for port := a; port <= b; port++ {
|
||||
add(e.Proto, port)
|
||||
matched = true
|
||||
}
|
||||
}
|
||||
}
|
||||
if !matched {
|
||||
return nil, fmt.Errorf("service %s has no %s ports", svc.Name, map[bool]string{true: protocol, false: "tcp or udp"}[protocol != ""])
|
||||
}
|
||||
}
|
||||
if len(out) > maxBanPorts {
|
||||
return nil, fmt.Errorf("%d ports in one ban: ban every port instead (up to %d)", len(out), maxBanPorts)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// neverBan says why an address (banned as prefix) must not be banned, or
|
||||
// "": it is the firewall's, an inside network's, the client's own, in the
|
||||
// list's never, or not an address one bans.
|
||||
func (s *Server) neverBan(ctx context.Context, st *model.State, list *model.BanList, p *auth.Principal, addr netip.Addr, prefix netip.Prefix) string {
|
||||
switch {
|
||||
case addr.IsLoopback(), addr.IsUnspecified(), addr.IsMulticast(), addr.IsLinkLocalUnicast(), addr.IsLinkLocalMulticast():
|
||||
return "it is a loopback, link-local, multicast or unspecified address"
|
||||
}
|
||||
if own, err := netip.ParseAddr(p.IP); err == nil && prefix.Contains(own.Unmap()) {
|
||||
return "it is the address of whoever asked (" + own.Unmap().String() + ")"
|
||||
}
|
||||
inside := s.insideNetworks(st)
|
||||
if inside(addr) || inside(prefix.Addr()) {
|
||||
return "it is inside: in a network of an inside zone, or private"
|
||||
}
|
||||
external := map[string]bool{}
|
||||
for _, z := range st.Zones {
|
||||
external[z.Name] = z.External
|
||||
}
|
||||
for net, zone := range s.connected(st) {
|
||||
if !external[zone] && net.Overlaps(prefix) {
|
||||
return fmt.Sprintf("it overlaps %s, a network of zone %s", net, zone)
|
||||
}
|
||||
}
|
||||
if s.ifaces != nil {
|
||||
if all, err := s.ifaces.All(); err == nil {
|
||||
for name, ps := range all {
|
||||
for _, a := range ps {
|
||||
if prefix.Contains(a.Addr()) {
|
||||
return fmt.Sprintf("it is the firewall's own (%s on %s)", a.Addr(), name)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
for _, o := range st.Settings.OwnPrefixes {
|
||||
if op, err := netip.ParsePrefix(o); err == nil && op.Overlaps(prefix) {
|
||||
return "it is in the firewall's own prefix " + o + " (settings)"
|
||||
}
|
||||
}
|
||||
for _, ref := range list.Never {
|
||||
if why := s.inNever(ctx, st, ref, prefix); why != "" {
|
||||
return why + " (the list's never)"
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// inNever: is prefix in a never entry (an address, or a set)?
|
||||
func (s *Server) inNever(ctx context.Context, st *model.State, ref string, prefix netip.Prefix) string {
|
||||
overlaps := func(entry string) bool {
|
||||
a, err := model.ParseAddress(entry)
|
||||
return err == nil && a.Overlaps(prefix)
|
||||
}
|
||||
if model.IsLiteral(ref) {
|
||||
if overlaps(ref) {
|
||||
return "it is in " + ref
|
||||
}
|
||||
return ""
|
||||
}
|
||||
set := st.Set(ref)
|
||||
if set == nil {
|
||||
return ""
|
||||
}
|
||||
entries := set.Entries
|
||||
if set.Dynamic {
|
||||
if t, err := s.fw.Table(ctx); err == nil {
|
||||
for _, e := range t.Elements[set.Name] {
|
||||
entries = append(entries, e.Addr)
|
||||
}
|
||||
}
|
||||
}
|
||||
for _, e := range entries {
|
||||
if overlaps(e) {
|
||||
return fmt.Sprintf("it is in set %s (%s)", set.Name, e)
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// closeBanned closes the banned address's connections the list's bans
|
||||
// now drop: offloaded to the flowtable, they would go on without
|
||||
// reaching the rules. Not in shadow mode, which changes no traffic.
|
||||
func (s *Server) closeBanned(ctx context.Context, st *model.State, list *model.BanList, prefix netip.Prefix, ports []banPort) int {
|
||||
if s.cfg.Shadow || s.conns == nil {
|
||||
return 0
|
||||
}
|
||||
conns, err := s.conns.List(ctx)
|
||||
if err != nil {
|
||||
s.log.Warn("listing connections to close a ban's failed", "err", err)
|
||||
return 0
|
||||
}
|
||||
own := map[netip.Addr]bool{}
|
||||
if s.ifaces != nil {
|
||||
if all, err := s.ifaces.All(); err == nil {
|
||||
for _, ps := range all {
|
||||
for _, a := range ps {
|
||||
own[a.Addr()] = true
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
h4, h6 := st.SplitRefs(list.Hosts)
|
||||
hosts := append(append([]string{}, h4.Literals...), h6.Literals...)
|
||||
for _, name := range append(h4.Sets, h6.Sets...) {
|
||||
if set := st.Set(name); set != nil {
|
||||
hosts = append(hosts, set.Entries...)
|
||||
}
|
||||
}
|
||||
isHost := func(a netip.Addr) bool {
|
||||
for _, h := range hosts {
|
||||
if x, err := model.ParseAddress(h); err == nil && x.Contains(a) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
closed := 0
|
||||
for _, c := range conns {
|
||||
src, err := netip.ParseAddr(c.Source)
|
||||
if err != nil || !prefix.Contains(src.Unmap()) {
|
||||
continue
|
||||
}
|
||||
dst, _ := netip.ParseAddr(c.Destination)
|
||||
port := c.DestinationPort
|
||||
if c.DNATTo != "" {
|
||||
dst, _ = netip.ParseAddr(c.DNATTo)
|
||||
if c.DNATPort != 0 {
|
||||
port = c.DNATPort
|
||||
}
|
||||
}
|
||||
toFirewall := own[dst.Unmap()]
|
||||
switch list.Scope {
|
||||
case model.BanScopeFirewall:
|
||||
if !toFirewall {
|
||||
continue
|
||||
}
|
||||
case model.BanScopeHosts:
|
||||
if toFirewall || !isHost(dst.Unmap()) {
|
||||
continue
|
||||
}
|
||||
}
|
||||
hit := false
|
||||
for _, pp := range ports {
|
||||
hit = hit || pp.proto == "" || pp.proto == c.Protocol && pp.port == port
|
||||
}
|
||||
if !hit {
|
||||
continue
|
||||
}
|
||||
if err := s.conns.Close(ctx, c); err == nil {
|
||||
closed++
|
||||
}
|
||||
}
|
||||
return closed
|
||||
}
|
||||
|
||||
// deleteBans answers DELETE /banlists/{ref}/bans/{id}, and DELETE
|
||||
// /banlists/{ref}/bans?address=A: every ban of that address (any of its
|
||||
// grouping), of every port and of each.
|
||||
func (s *Server) deleteBans(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
list, err := s.banList(r, p, true)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var gone []*model.Ban
|
||||
if idText := r.PathValue("id"); idText != "" {
|
||||
id, err := strconv.ParseInt(idText, 10, 64)
|
||||
if err != nil {
|
||||
return badRequest("ban ID %q isn't a number", idText)
|
||||
}
|
||||
b, err := s.store.GetBan(r.Context(), id)
|
||||
if errors.Is(err, store.ErrNotFound) || err == nil && b.ListID != list.ID {
|
||||
return notFound("no ban %d in ban list %s", id, list.Name)
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
gone = append(gone, b)
|
||||
} else {
|
||||
q := strings.Trim(strings.TrimSpace(r.URL.Query().Get("address")), "[]")
|
||||
a, err := netip.ParseAddr(q)
|
||||
if err != nil {
|
||||
return badRequest("say which ban: DELETE …/bans/ID, or …/bans?address=ADDRESS")
|
||||
}
|
||||
a = a.Unmap()
|
||||
bans, err := s.store.ListBans(r.Context(), list.ID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, b := range bans {
|
||||
if pre, err := b.Prefix(); err == nil && pre.Contains(a) {
|
||||
gone = append(gone, b)
|
||||
}
|
||||
}
|
||||
if len(gone) == 0 {
|
||||
return notFound("%s isn't banned in ban list %s", a, list.Name)
|
||||
}
|
||||
}
|
||||
for _, b := range gone {
|
||||
if err := s.store.DeleteBan(r.Context(), b.ID); err != nil {
|
||||
return err
|
||||
}
|
||||
if rb, ok := renderBan(b); ok {
|
||||
if err := s.fw.DeleteBan(r.Context(), rb); err != nil {
|
||||
s.log.Warn("lifting a ban failed; the next change or restart drops it", "list", list.Name, "address", b.Address, "err", err)
|
||||
}
|
||||
}
|
||||
s.eng.Audit(r.Context(), p, "ban.remove", list.Name+" "+banLabel(b), "success", "")
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]int{"removed": len(gone)})
|
||||
return nil
|
||||
}
|
||||
|
||||
// pruneBans forgets the bans that ended, and those of lists st no longer
|
||||
// has.
|
||||
func (s *Server) pruneBans(ctx context.Context, st *model.State) {
|
||||
var ids []int64
|
||||
for _, b := range st.BanLists {
|
||||
ids = append(ids, b.ID)
|
||||
}
|
||||
if _, err := s.store.PruneBans(ctx, ids); err != nil {
|
||||
s.log.Warn("pruning bans failed", "err", err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,484 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"housecarl/internal/api"
|
||||
"housecarl/internal/auth"
|
||||
"housecarl/internal/engine"
|
||||
"housecarl/internal/model"
|
||||
"housecarl/internal/store"
|
||||
)
|
||||
|
||||
// Changesets: changes staged to be reviewed (validation, nftables' check,
|
||||
// the ruleset's diff) and applied together, as one history entry, or
|
||||
// discarded. The web UI stages its edits by default; the CLI and the API
|
||||
// stage when a changeset is named (?changeset=ID, X-Housecarl-Changeset).
|
||||
|
||||
// changesetParam is the changeset a request names (0: none).
|
||||
func changesetParam(r *http.Request) (int64, error) {
|
||||
v := r.URL.Query().Get("changeset")
|
||||
if v == "" {
|
||||
v = r.Header.Get("X-Housecarl-Changeset")
|
||||
}
|
||||
if v == "" {
|
||||
return 0, nil
|
||||
}
|
||||
id, err := strconv.ParseInt(v, 10, 64)
|
||||
if err != nil || id <= 0 {
|
||||
return 0, badRequest("invalid changeset %q", v)
|
||||
}
|
||||
return id, nil
|
||||
}
|
||||
|
||||
// notStaged refuses a changeset on a change that can't be staged (an
|
||||
// import, a takeover): it would otherwise be applied at once, unexpectedly.
|
||||
func notStaged(r *http.Request, what string) error {
|
||||
if id, err := changesetParam(r); err != nil || id == 0 {
|
||||
return err
|
||||
}
|
||||
return badRequest("%s can't be staged in a changeset; leave out --changeset (or unset HOUSECARL_CHANGESET) to do it now", what)
|
||||
}
|
||||
|
||||
// openChangeset loads a changeset to change: it must be open, and the
|
||||
// caller's own unless they are an administrator.
|
||||
func (s *Server) openChangeset(r *http.Request, p *auth.Principal, id int64) (*model.Changeset, error) {
|
||||
cs, err := s.store.GetChangeset(r.Context(), id)
|
||||
if errors.Is(err, store.ErrNotFound) {
|
||||
return nil, notFound("there is no changeset %d", id)
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if cs.Status != model.ChangesetOpen {
|
||||
return nil, conflict("changeset %d is %s", id, cs.Status)
|
||||
}
|
||||
if !ownsChangeset(p, cs) && !p.IsAdmin() {
|
||||
return nil, forbidden("changeset %d is %s's, not yours", id, cs.Username)
|
||||
}
|
||||
return cs, nil
|
||||
}
|
||||
|
||||
func ownsChangeset(p *auth.Principal, cs *model.Changeset) bool {
|
||||
if p == nil || p.User == nil {
|
||||
return false
|
||||
}
|
||||
if p.User.ID != 0 {
|
||||
return cs.UserID == p.User.ID
|
||||
}
|
||||
return cs.UserID == 0 && cs.Username == p.User.Username
|
||||
}
|
||||
|
||||
// stage adds an operation to a changeset. The changeset with it must
|
||||
// still be valid and pass nftables' check; nothing is applied.
|
||||
func (s *Server) stage(w http.ResponseWriter, r *http.Request, p *auth.Principal, id int64, op model.Op) error {
|
||||
s.csMu.Lock()
|
||||
defer s.csMu.Unlock()
|
||||
cs, err := s.openChangeset(r, p, id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
// The operation on its own, after the changeset so far: what it
|
||||
// changes, for its summary.
|
||||
before := s.eng.State()
|
||||
if err := replay(before, cs.Ops); err != nil {
|
||||
return conflict("changeset %d can't be applied as it is (%v); review it first", id, err)
|
||||
}
|
||||
after := before.Clone()
|
||||
objID, err := applyOp(after, &op)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
engine.Normalize(after)
|
||||
after.Sort()
|
||||
changes := engine.Diff(before, after)
|
||||
if len(changes) == 0 {
|
||||
writeJSON(w, http.StatusOK, api.MutationResponse{Changes: []model.Change{}, Changeset: cs})
|
||||
return nil
|
||||
}
|
||||
op.Summary = engine.Describe(changes)
|
||||
if op.Op == model.OpRevert {
|
||||
var h model.HistoryEntry
|
||||
json.Unmarshal(op.Data, &h)
|
||||
op.Summary = engine.UndoDescription(&h)
|
||||
}
|
||||
op.AddedAt = s.nowFn()
|
||||
for _, o := range cs.Ops {
|
||||
if o.Seq >= op.Seq {
|
||||
op.Seq = o.Seq + 1
|
||||
}
|
||||
}
|
||||
if op.Seq == 0 {
|
||||
op.Seq = 1
|
||||
}
|
||||
ops := append(append([]model.Op{}, cs.Ops...), op)
|
||||
dry := queryBool(r, "dry_run")
|
||||
// The whole changeset, checked as it would be applied.
|
||||
res, err := s.eng.Apply(r.Context(), p, "", engine.Options{DryRun: true, Via: via(r, p)}, func(st *model.State, _ func() int64) error {
|
||||
return replay(st, ops)
|
||||
})
|
||||
if err != nil {
|
||||
s.writeErr(w, r, err, res)
|
||||
return nil
|
||||
}
|
||||
if !dry {
|
||||
cs.Ops = ops
|
||||
if err := s.store.UpdateChangeset(r.Context(), cs); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
out := api.MutationResponse{Staged: !dry, DryRun: dry, Changes: changes, Warnings: res.Warnings, Changeset: cs}
|
||||
if object := objectOf(&op, objID); object != nil {
|
||||
if v := object(after); v != nil {
|
||||
out.Object, _ = json.Marshal(v)
|
||||
}
|
||||
}
|
||||
if dry {
|
||||
out.Diff = s.rulesetDiff(res.Before, res.After)
|
||||
}
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Server) createChangeset(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResConfig, true); err != nil {
|
||||
return err
|
||||
}
|
||||
var in api.ChangesetInput
|
||||
if r.ContentLength != 0 {
|
||||
if err := decodeJSON(r, &in); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
cs := &model.Changeset{Username: p.Name(), Via: via(r, p), Description: strings.TrimSpace(in.Description), Status: model.ChangesetOpen}
|
||||
if p.User != nil {
|
||||
cs.UserID = p.User.ID
|
||||
}
|
||||
if err := s.store.CreateChangeset(r.Context(), cs); err != nil {
|
||||
return err
|
||||
}
|
||||
s.eng.Audit(r.Context(), p, "changeset.create", fmt.Sprintf("changeset %d", cs.ID), "success", cs.Description)
|
||||
writeJSON(w, http.StatusCreated, cs)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Server) listChangesets(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResConfig, false); err != nil {
|
||||
return err
|
||||
}
|
||||
var uid int64
|
||||
if queryBool(r, "mine") {
|
||||
uid = -1
|
||||
if p.User != nil && p.User.ID != 0 {
|
||||
uid = p.User.ID
|
||||
}
|
||||
}
|
||||
list, err := s.store.ListChangesets(r.Context(), r.URL.Query().Get("status"), max(uid, 0), queryInt(r, "limit", 100))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if uid == -1 { // the socket's users have no user ID: by name
|
||||
mine := []*model.Changeset{}
|
||||
for _, cs := range list {
|
||||
if ownsChangeset(p, cs) {
|
||||
mine = append(mine, cs)
|
||||
}
|
||||
}
|
||||
list = mine
|
||||
}
|
||||
writeJSON(w, http.StatusOK, list)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Server) getChangeset(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResConfig, false); err != nil {
|
||||
return err
|
||||
}
|
||||
id, err := pathID(r, "id")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
cs, err := s.store.GetChangeset(r.Context(), id)
|
||||
if errors.Is(err, store.ErrNotFound) {
|
||||
return notFound("there is no changeset %d", id)
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
writeJSON(w, http.StatusOK, cs)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Server) updateChangeset(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResConfig, true); err != nil {
|
||||
return err
|
||||
}
|
||||
id, err := pathID(r, "id")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var in api.ChangesetInput
|
||||
if err := decodeJSON(r, &in); err != nil {
|
||||
return err
|
||||
}
|
||||
s.csMu.Lock()
|
||||
defer s.csMu.Unlock()
|
||||
cs, err := s.openChangeset(r, p, id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
cs.Description = strings.TrimSpace(in.Description)
|
||||
if err := s.store.UpdateChangeset(r.Context(), cs); err != nil {
|
||||
return err
|
||||
}
|
||||
writeJSON(w, http.StatusOK, cs)
|
||||
return nil
|
||||
}
|
||||
|
||||
// removeOp takes one operation out of a changeset. Later ones that
|
||||
// needed it show up in the preview.
|
||||
func (s *Server) removeOp(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResConfig, true); err != nil {
|
||||
return err
|
||||
}
|
||||
id, err := pathID(r, "id")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
seq, err := strconv.Atoi(r.PathValue("seq"))
|
||||
if err != nil {
|
||||
return badRequest("invalid operation %q", r.PathValue("seq"))
|
||||
}
|
||||
s.csMu.Lock()
|
||||
defer s.csMu.Unlock()
|
||||
cs, err := s.openChangeset(r, p, id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
kept := []model.Op{}
|
||||
for _, o := range cs.Ops {
|
||||
if o.Seq != seq {
|
||||
kept = append(kept, o)
|
||||
}
|
||||
}
|
||||
if len(kept) == len(cs.Ops) {
|
||||
return notFound("changeset %d has no operation %d", id, seq)
|
||||
}
|
||||
cs.Ops = kept
|
||||
if err := s.store.UpdateChangeset(r.Context(), cs); err != nil {
|
||||
return err
|
||||
}
|
||||
writeJSON(w, http.StatusOK, cs)
|
||||
return nil
|
||||
}
|
||||
|
||||
// previewChangeset says what applying a changeset would do, without
|
||||
// doing it: its changes, problems, nftables' check, the ruleset's diff.
|
||||
func (s *Server) previewChangeset(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResConfig, false); err != nil {
|
||||
return err
|
||||
}
|
||||
id, err := pathID(r, "id")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
cs, err := s.store.GetChangeset(r.Context(), id)
|
||||
if errors.Is(err, store.ErrNotFound) {
|
||||
return notFound("there is no changeset %d", id)
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
res, err := s.eng.Apply(r.Context(), p, "", engine.Options{DryRun: true, Via: via(r, p)}, func(st *model.State, _ func() int64) error {
|
||||
return replay(st, cs.Ops)
|
||||
})
|
||||
out := api.ChangesetPreview{Valid: err == nil && res != nil && len(res.Changes) > 0, Changes: []model.Change{}}
|
||||
if res != nil {
|
||||
if res.Changes != nil {
|
||||
out.Changes = res.Changes
|
||||
}
|
||||
out.Warnings = res.Warnings
|
||||
}
|
||||
if err != nil {
|
||||
out.Error = errorInfo(err)
|
||||
} else if res != nil {
|
||||
out.Diff = s.rulesetDiff(res.Before, res.After)
|
||||
}
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
return nil
|
||||
}
|
||||
|
||||
// errorInfo is an error as the API reports it.
|
||||
func errorInfo(err error) *api.ErrorInfo {
|
||||
var oe *opError
|
||||
if errors.As(err, &oe) {
|
||||
info, _ := oe.info()
|
||||
return &info
|
||||
}
|
||||
var he *httpError
|
||||
if errors.As(err, &he) {
|
||||
return &api.ErrorInfo{Code: he.code, Message: he.msg}
|
||||
}
|
||||
if ee, ok := engine.AsError(err); ok {
|
||||
return &api.ErrorInfo{Code: ee.Kind, Message: ee.Message, Problems: ee.Problems}
|
||||
}
|
||||
return &api.ErrorInfo{Code: "internal", Message: err.Error()}
|
||||
}
|
||||
|
||||
// applyChangeset applies a changeset's operations as one change: one
|
||||
// history entry, and (over the network) on probation like any change.
|
||||
func (s *Server) applyChangeset(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResConfig, true); err != nil {
|
||||
return err
|
||||
}
|
||||
id, err := pathID(r, "id")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var in api.ChangesetInput
|
||||
if r.ContentLength != 0 {
|
||||
if err := decodeJSON(r, &in); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
s.csMu.Lock()
|
||||
defer s.csMu.Unlock()
|
||||
cs, err := s.openChangeset(r, p, id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if d := strings.TrimSpace(in.Description); d != "" {
|
||||
cs.Description = d
|
||||
}
|
||||
if len(cs.Ops) == 0 {
|
||||
return badRequest("changeset %d has no changes to apply", id)
|
||||
}
|
||||
opts := engine.Options{Via: via(r, p)}
|
||||
desc := cs.Description
|
||||
for _, o := range cs.Ops {
|
||||
if o.Op == model.OpRevert {
|
||||
opts.RevertOf = o.ID // History links the undo
|
||||
if desc == "" && len(cs.Ops) == 1 {
|
||||
desc = o.Summary
|
||||
}
|
||||
}
|
||||
}
|
||||
res, err := s.eng.Apply(r.Context(), p, desc, opts, func(st *model.State, _ func() int64) error {
|
||||
return replay(st, cs.Ops)
|
||||
})
|
||||
if err != nil {
|
||||
s.eng.Audit(r.Context(), p, "changeset.apply", fmt.Sprintf("changeset %d", id), "failure", err.Error())
|
||||
s.writeErr(w, r, err, res)
|
||||
return nil
|
||||
}
|
||||
cs.Status, cs.HistoryID = model.ChangesetApplied, res.HistoryID
|
||||
if err := s.store.UpdateChangeset(r.Context(), cs); err != nil {
|
||||
return err
|
||||
}
|
||||
s.eng.Audit(r.Context(), p, "changeset.apply", fmt.Sprintf("changeset %d", id), "success", fmt.Sprintf("history %d", res.HistoryID))
|
||||
s.afterChange(r.Context(), p, res)
|
||||
out := s.mutation(res, nil)
|
||||
out.ConfirmToken = s.issueCheck(w, r, p, res)
|
||||
out.Changeset = cs
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Server) discardChangeset(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResConfig, true); err != nil {
|
||||
return err
|
||||
}
|
||||
id, err := pathID(r, "id")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
s.csMu.Lock()
|
||||
defer s.csMu.Unlock()
|
||||
cs, err := s.openChangeset(r, p, id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
cs.Status = model.ChangesetDiscarded
|
||||
if err := s.store.UpdateChangeset(r.Context(), cs); err != nil {
|
||||
return err
|
||||
}
|
||||
s.eng.Audit(r.Context(), p, "changeset.discard", fmt.Sprintf("changeset %d", id), "success", fmt.Sprintf("%d operations", len(cs.Ops)))
|
||||
writeJSON(w, http.StatusOK, cs)
|
||||
return nil
|
||||
}
|
||||
|
||||
// ---- reading the configuration as a changeset will make it
|
||||
|
||||
// viewState is the configuration a read shows: the current one, or with
|
||||
// a changeset named, as it will be once that is applied (draft true).
|
||||
// A changeset that no longer applies shows the current configuration,
|
||||
// with the reason in the X-Housecarl-Changeset-Error header.
|
||||
func (s *Server) viewState(w http.ResponseWriter, r *http.Request) (cur, view *model.State, draft bool, err error) {
|
||||
cur = s.eng.State()
|
||||
id, err := changesetParam(r)
|
||||
if err != nil || id == 0 {
|
||||
return cur, cur, false, err
|
||||
}
|
||||
cs, err := s.store.GetChangeset(r.Context(), id)
|
||||
if errors.Is(err, store.ErrNotFound) {
|
||||
return nil, nil, false, notFound("there is no changeset %d", id)
|
||||
}
|
||||
if err != nil {
|
||||
return nil, nil, false, err
|
||||
}
|
||||
if cs.Status != model.ChangesetOpen {
|
||||
return cur, cur, false, nil
|
||||
}
|
||||
next := cur.Clone()
|
||||
if err := replay(next, cs.Ops); err != nil {
|
||||
w.Header().Set("X-Housecarl-Changeset-Error", err.Error())
|
||||
return cur, cur, false, nil
|
||||
}
|
||||
engine.Normalize(next)
|
||||
next.Sort()
|
||||
return cur, next, true, nil
|
||||
}
|
||||
|
||||
// withPending lists a kind's objects as a changeset will leave them, each
|
||||
// marked "pending": "new", "changed" or "deleted" (deleted ones stay in
|
||||
// the list, in their place).
|
||||
func withPending(cur, next *model.State, kind string) []any {
|
||||
was := map[int64]string{}
|
||||
for _, o := range cur.Objects(kind) {
|
||||
b, _ := json.Marshal(o)
|
||||
was[o.GetID()] = string(b)
|
||||
}
|
||||
// The deleted ones go back in, so they sort into their places.
|
||||
merged := next.Clone()
|
||||
deleted := map[int64]bool{}
|
||||
for _, o := range cur.Objects(kind) {
|
||||
if merged.ByID(kind, o.GetID()) == nil {
|
||||
b, _ := json.Marshal(o)
|
||||
merged.Put(kind, o.GetID(), b)
|
||||
deleted[o.GetID()] = true
|
||||
}
|
||||
}
|
||||
merged.Sort()
|
||||
out := []any{}
|
||||
for _, o := range merged.Objects(kind) {
|
||||
var m map[string]any
|
||||
b, _ := json.Marshal(o)
|
||||
json.Unmarshal(b, &m)
|
||||
old, existed := was[o.GetID()]
|
||||
switch {
|
||||
case deleted[o.GetID()]:
|
||||
m["pending"] = "deleted"
|
||||
case !existed:
|
||||
m["pending"] = "new"
|
||||
case old != string(b):
|
||||
m["pending"] = "changed"
|
||||
}
|
||||
out = append(out, m)
|
||||
}
|
||||
return out
|
||||
}
|
||||
+404
-186
@@ -4,36 +4,64 @@ import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"net/http"
|
||||
"net/netip"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
udiff "github.com/aymanbagabas/go-udiff"
|
||||
|
||||
"housecarl/internal/api"
|
||||
"housecarl/internal/auth"
|
||||
"housecarl/internal/engine"
|
||||
"housecarl/internal/matrix"
|
||||
"housecarl/internal/model"
|
||||
"housecarl/internal/nft"
|
||||
"housecarl/internal/store"
|
||||
)
|
||||
|
||||
// change applies an edit through the engine and answers it; object picks
|
||||
// what to return from the new state. A network change starts (or joins)
|
||||
// the probation.
|
||||
func (s *Server) change(w http.ResponseWriter, r *http.Request, p *auth.Principal, description string, edit engine.Edit, object func(st *model.State) any) error {
|
||||
// change makes an operation: at once through the engine, or, with a
|
||||
// changeset named (?changeset=ID or X-Housecarl-Changeset), staged in it.
|
||||
// A network change starts (or joins) the probation.
|
||||
func (s *Server) change(w http.ResponseWriter, r *http.Request, p *auth.Principal, op model.Op) error {
|
||||
if err := require(p, auth.ResConfig, true); err != nil {
|
||||
return err
|
||||
}
|
||||
res, err := s.eng.Apply(r.Context(), p, description, engine.Options{DryRun: queryBool(r, "dry_run"), Via: via(r, p)}, edit)
|
||||
if op.Op == model.OpCreate && op.ID == 0 {
|
||||
ids, err := s.store.AllocateIDs(r.Context(), 1)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
op.ID = ids[0]
|
||||
}
|
||||
csID, err := changesetParam(r)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if csID != 0 {
|
||||
return s.stage(w, r, p, csID, op)
|
||||
}
|
||||
var id int64
|
||||
res, err := s.eng.Apply(r.Context(), p, "", engine.Options{DryRun: queryBool(r, "dry_run"), Via: via(r, p)}, func(st *model.State, _ func() int64) error {
|
||||
var err error
|
||||
id, err = applyOp(st, &op)
|
||||
return err
|
||||
})
|
||||
if err != nil {
|
||||
s.writeErr(w, r, err, res)
|
||||
return nil
|
||||
}
|
||||
s.afterChange(r.Context(), p, res)
|
||||
// POST to a collection creates; POST to an object (move) changes it.
|
||||
s.respondChange(w, r, res, object, r.Method == http.MethodPost && r.PathValue("ref") == "")
|
||||
out := s.mutation(res, objectOf(&op, id))
|
||||
out.ConfirmToken = s.issueCheck(w, r, p, res)
|
||||
writeJSON(w, s.mutationStatus(res, op.Op == model.OpCreate), out)
|
||||
return nil
|
||||
}
|
||||
|
||||
// respondChange answers a change: 201 when it created something.
|
||||
func (s *Server) respondChange(w http.ResponseWriter, r *http.Request, res *engine.Result, object func(st *model.State) any, creates bool) {
|
||||
// mutation is the answer to a change: what changed, warnings, the object
|
||||
// (as it is after the change), the probation, and for a dry run the
|
||||
// ruleset's diff.
|
||||
func (s *Server) mutation(res *engine.Result, object func(st *model.State) any) api.MutationResponse {
|
||||
out := api.MutationResponse{Applied: res.Applied, DryRun: res.DryRun, HistoryID: res.HistoryID, Changes: res.Changes, Warnings: res.Warnings}
|
||||
if out.Changes == nil {
|
||||
out.Changes = []model.Change{}
|
||||
@@ -41,22 +69,107 @@ func (s *Server) respondChange(w http.ResponseWriter, r *http.Request, res *engi
|
||||
if pr := s.probationStatus(); res.Applied && (len(pr.Pending) > 0 || pr.Takeovers > 0) {
|
||||
out.ConfirmBy = &pr.Deadline
|
||||
}
|
||||
if object != nil && (res.Applied || res.DryRun) {
|
||||
if v := object(s.eng.State()); v != nil {
|
||||
if object != nil && res.After != nil && (res.Applied || res.DryRun) {
|
||||
if v := object(res.After); v != nil {
|
||||
out.Object, _ = json.Marshal(v)
|
||||
}
|
||||
}
|
||||
status := http.StatusOK
|
||||
if res.Applied && creates {
|
||||
status = http.StatusCreated
|
||||
if res.DryRun && len(res.Changes) > 0 {
|
||||
out.Diff = s.rulesetDiff(res.Before, res.After)
|
||||
}
|
||||
writeJSON(w, status, out)
|
||||
return out
|
||||
}
|
||||
|
||||
// mutationStatus is 201 for an applied create, else 200.
|
||||
func (s *Server) mutationStatus(res *engine.Result, creates bool) int {
|
||||
if res.Applied && creates {
|
||||
return http.StatusCreated
|
||||
}
|
||||
return http.StatusOK
|
||||
}
|
||||
|
||||
// respondChange answers a change made without an operation (undo).
|
||||
func (s *Server) respondChange(w http.ResponseWriter, r *http.Request, p *auth.Principal, res *engine.Result, object func(st *model.State) any, creates bool) {
|
||||
out := s.mutation(res, object)
|
||||
out.ConfirmToken = s.issueCheck(w, r, p, res)
|
||||
writeJSON(w, s.mutationStatus(res, creates), out)
|
||||
}
|
||||
|
||||
// rulesetDiff is the unified diff of the rulesets two configurations
|
||||
// make: the file housecarl-ruleset.service loads, now and after.
|
||||
func (s *Server) rulesetDiff(before, after *model.State) string {
|
||||
if before == nil || after == nil {
|
||||
return ""
|
||||
}
|
||||
now, then := s.fw.Render(before).Text, s.fw.Render(after).Text
|
||||
return blockHeaders(udiff.Unified("ruleset.nft (now)", "ruleset.nft (after)", now, then), now, then)
|
||||
}
|
||||
|
||||
// blockHeaders names, after each hunk's @@ line, the chain (or other
|
||||
// block of the table) the hunk starts in, as git names the function:
|
||||
// "@@ -40,6 +40,8 @@ chain forward". Without it a change deep in a long
|
||||
// chain doesn't say which chain it is in.
|
||||
func blockHeaders(diff, now, then string) string {
|
||||
if diff == "" {
|
||||
return diff
|
||||
}
|
||||
oldBlocks, newBlocks := enclosingBlocks(now), enclosingBlocks(then)
|
||||
lines := strings.SplitAfter(diff, "\n")
|
||||
for i, l := range lines {
|
||||
if !strings.HasPrefix(l, "@@ ") {
|
||||
continue
|
||||
}
|
||||
// "@@ -a,b +c,d @@": the new side's start, or the old side's
|
||||
// when nothing of the hunk is left.
|
||||
f := strings.Fields(l)
|
||||
if len(f) < 4 || f[3] != "@@" {
|
||||
continue
|
||||
}
|
||||
block := blockAt(newBlocks, f[2])
|
||||
if block == "" {
|
||||
block = blockAt(oldBlocks, f[1])
|
||||
}
|
||||
if block != "" {
|
||||
lines[i] = strings.TrimRight(l, "\n") + " " + block + "\n"
|
||||
}
|
||||
}
|
||||
return strings.Join(lines, "")
|
||||
}
|
||||
|
||||
// enclosingBlocks gives each line of a rendered ruleset the block of the
|
||||
// table it is in ("chain input"), "" outside one. A block opens with a
|
||||
// line of one tab ending in "{" and closes with "\t}".
|
||||
func enclosingBlocks(text string) []string {
|
||||
var out []string
|
||||
cur := ""
|
||||
for _, l := range strings.Split(text, "\n") {
|
||||
if strings.HasPrefix(l, "\t") && !strings.HasPrefix(l, "\t\t") && strings.HasSuffix(l, "{") {
|
||||
cur = strings.TrimSpace(strings.TrimSuffix(l, "{"))
|
||||
}
|
||||
out = append(out, cur)
|
||||
if l == "\t}" {
|
||||
cur = ""
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// blockAt is the block at a hunk side's start ("+40,8" or "-40"); lines
|
||||
// count from 1, and an empty side (",0") sits after its line.
|
||||
func blockAt(blocks []string, side string) string {
|
||||
start, _, _ := strings.Cut(side[1:], ",")
|
||||
n, err := strconv.Atoi(start)
|
||||
if err != nil || n < 1 || n > len(blocks) {
|
||||
return ""
|
||||
}
|
||||
return blocks[n-1]
|
||||
}
|
||||
|
||||
// kindOf maps the API's plural path names to kinds.
|
||||
var kindOf = map[string]string{
|
||||
"zones": model.KindZone, "sets": model.KindSet, "services": model.KindService, "rules": model.KindRule,
|
||||
"forwards": model.KindForward, "snats": model.KindSNAT, "redirects": model.KindRedirect,
|
||||
"banlists": model.KindBanList,
|
||||
}
|
||||
|
||||
func pathKind(r *http.Request) (string, error) {
|
||||
@@ -77,23 +190,39 @@ func findObject(st *model.State, kind, ref string) (model.Object, error) {
|
||||
// ---- objects
|
||||
|
||||
func (s *Server) listObjects(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResConfig, false); err != nil {
|
||||
return err
|
||||
}
|
||||
kind, err := pathKind(r)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
st := s.eng.State()
|
||||
if kind == model.KindBanList && !p.Can(auth.ResConfig, false) {
|
||||
return s.listOwnBanLists(w, p)
|
||||
}
|
||||
if err := require(p, auth.ResConfig, false); err != nil {
|
||||
return err
|
||||
}
|
||||
cur, st, draft, err := s.viewState(w, r)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
out := []any{}
|
||||
q := r.URL.Query()
|
||||
for _, o := range st.Objects(kind) {
|
||||
if c := q.Get("chain"); c != "" {
|
||||
if rule, ok := o.(*model.Rule); ok && rule.Chain != c {
|
||||
c := r.URL.Query().Get("chain")
|
||||
if draft {
|
||||
// Marked new, changed or deleted.
|
||||
for _, o := range withPending(cur, st, kind) {
|
||||
if c != "" && kind == model.KindRule && o.(map[string]any)["chain"] != c {
|
||||
continue
|
||||
}
|
||||
out = append(out, o)
|
||||
}
|
||||
} else {
|
||||
for _, o := range st.Objects(kind) {
|
||||
if c != "" {
|
||||
if rule, ok := o.(*model.Rule); ok && rule.Chain != c {
|
||||
continue
|
||||
}
|
||||
}
|
||||
out = append(out, o)
|
||||
}
|
||||
out = append(out, o)
|
||||
}
|
||||
// Services: the built-in ones too, unless overridden.
|
||||
if kind == model.KindService && !queryBool(r, "own") {
|
||||
@@ -102,23 +231,40 @@ func (s *Server) listObjects(w http.ResponseWriter, r *http.Request, p *auth.Pri
|
||||
out = append(out, b)
|
||||
}
|
||||
}
|
||||
sort.SliceStable(out, func(i, j int) bool {
|
||||
return out[i].(*model.Service).Name < out[j].(*model.Service).Name
|
||||
})
|
||||
name := func(x any) string {
|
||||
if m, ok := x.(map[string]any); ok {
|
||||
n, _ := m["name"].(string)
|
||||
return n
|
||||
}
|
||||
return x.(*model.Service).Name
|
||||
}
|
||||
sort.SliceStable(out, func(i, j int) bool { return name(out[i]) < name(out[j]) })
|
||||
}
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Server) getObject(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResConfig, false); err != nil {
|
||||
return err
|
||||
}
|
||||
kind, err := pathKind(r)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
st := s.eng.State()
|
||||
// A program that bans into a list may read it (its grouping, times).
|
||||
if kind == model.KindBanList && !p.Can(auth.ResConfig, false) {
|
||||
b, err := s.banList(r, p, false)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
writeJSON(w, http.StatusOK, b)
|
||||
return nil
|
||||
}
|
||||
if err := require(p, auth.ResConfig, false); err != nil {
|
||||
return err
|
||||
}
|
||||
_, st, _, err := s.viewState(w, r)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
o, err := findObject(st, kind, r.PathValue("ref"))
|
||||
if err != nil && kind == model.KindService {
|
||||
if b := model.BuiltinService(r.PathValue("ref")); b != nil {
|
||||
@@ -151,6 +297,8 @@ func decodeObject(kind string, data []byte) (model.Object, error) {
|
||||
o = &model.SourceNAT{}
|
||||
case model.KindRedirect:
|
||||
o = &model.Redirect{}
|
||||
case model.KindBanList:
|
||||
o = &model.BanList{}
|
||||
}
|
||||
dec := json.NewDecoder(strings.NewReader(string(data)))
|
||||
dec.DisallowUnknownFields()
|
||||
@@ -202,6 +350,8 @@ func setID(o model.Object, id int64) {
|
||||
x.ID = id
|
||||
case *model.Redirect:
|
||||
x.ID = id
|
||||
case *model.BanList:
|
||||
x.ID = id
|
||||
}
|
||||
}
|
||||
|
||||
@@ -235,29 +385,23 @@ func (s *Server) createObject(w http.ResponseWriter, r *http.Request, p *auth.Pr
|
||||
if err := decodeJSON(r, &raw); err != nil {
|
||||
return err
|
||||
}
|
||||
o, err := decodeObject(kind, raw)
|
||||
if err != nil {
|
||||
// Refuse a bad object now, not when it is applied.
|
||||
if _, err := decodeObject(kind, raw); err != nil {
|
||||
return err
|
||||
}
|
||||
if svc, ok := o.(*model.Service); ok {
|
||||
svc.Builtin = false
|
||||
op := model.Op{Op: model.OpCreate, Kind: kind, Data: raw}
|
||||
// ?before=ID, ?after=ID or ?first=1: where it goes in its order.
|
||||
q := r.URL.Query()
|
||||
var place api.MoveInput
|
||||
place.Before, _ = strconv.ParseInt(q.Get("before"), 10, 64)
|
||||
place.After, _ = strconv.ParseInt(q.Get("after"), 10, 64)
|
||||
if queryBool(r, "first") {
|
||||
place.To = "start"
|
||||
}
|
||||
if rule, ok := o.(*model.Rule); ok && rule.Action == "" {
|
||||
rule.Action = model.ActionAccept
|
||||
if place.Before != 0 || place.After != 0 || place.To != "" {
|
||||
op.Place, _ = json.Marshal(place)
|
||||
}
|
||||
if f, ok := o.(*model.PortForward); ok && len(f.Protocols) == 0 {
|
||||
f.Protocols = []string{"tcp"}
|
||||
}
|
||||
var id int64
|
||||
return s.change(w, r, p, "", func(st *model.State, newID func() int64) error {
|
||||
id = newID()
|
||||
setID(o, id)
|
||||
if pos := position(o); pos != nil && *pos == 0 {
|
||||
*pos = lastPosition(st, kind, chainOf(o))
|
||||
}
|
||||
data, _ := json.Marshal(o)
|
||||
return st.Put(kind, id, data)
|
||||
}, func(st *model.State) any { return st.ByID(kind, id) })
|
||||
return s.change(w, r, p, op)
|
||||
}
|
||||
|
||||
func (s *Server) updateObject(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
@@ -269,34 +413,8 @@ func (s *Server) updateObject(w http.ResponseWriter, r *http.Request, p *auth.Pr
|
||||
if err := decodeJSON(r, &patch); err != nil {
|
||||
return err
|
||||
}
|
||||
delete(patch, "id")
|
||||
var id int64
|
||||
return s.change(w, r, p, "", func(st *model.State, _ func() int64) error {
|
||||
o, err := findObject(st, kind, r.PathValue("ref"))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
id = o.GetID()
|
||||
// The patch's fields over the object's.
|
||||
var cur map[string]json.RawMessage
|
||||
b, _ := json.Marshal(o)
|
||||
json.Unmarshal(b, &cur)
|
||||
for k, v := range patch {
|
||||
if string(v) == "null" {
|
||||
delete(cur, k)
|
||||
} else {
|
||||
cur[k] = v
|
||||
}
|
||||
}
|
||||
merged, _ := json.Marshal(cur)
|
||||
next, err := decodeObject(kind, merged)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
setID(next, id)
|
||||
data, _ := json.Marshal(next)
|
||||
return st.Put(kind, id, data)
|
||||
}, func(st *model.State) any { return st.ByID(kind, id) })
|
||||
data, _ := json.Marshal(patch)
|
||||
return s.change(w, r, p, model.Op{Op: model.OpUpdate, Kind: kind, Ref: r.PathValue("ref"), Data: data})
|
||||
}
|
||||
|
||||
func (s *Server) deleteObject(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
@@ -304,16 +422,7 @@ func (s *Server) deleteObject(w http.ResponseWriter, r *http.Request, p *auth.Pr
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return s.change(w, r, p, "", func(st *model.State, _ func() int64) error {
|
||||
o, err := findObject(st, kind, r.PathValue("ref"))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if used := usedBy(st, kind, o); used != "" {
|
||||
return conflict("%s %s is used by %s; change that first", kind, o.Label(), used)
|
||||
}
|
||||
return st.Put(kind, o.GetID(), nil)
|
||||
}, nil)
|
||||
return s.change(w, r, p, model.Op{Op: model.OpDelete, Kind: kind, Ref: r.PathValue("ref")})
|
||||
}
|
||||
|
||||
// usedBy names an object that refers to o (zones, sets and services are
|
||||
@@ -360,12 +469,61 @@ func usedBy(st *model.State, kind string, o model.Object) string {
|
||||
return f.Label()
|
||||
}
|
||||
}
|
||||
for _, b := range st.BanLists {
|
||||
if has(b.Hosts) || has(b.Never) {
|
||||
return "ban list " + b.Name
|
||||
}
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// moveObject reorders a rule or NAT rule: before or after another, or to
|
||||
// the start or end; a rule can change chain at the same time.
|
||||
// renameRefs makes every object that refers to a zone, set or service by
|
||||
// its old name use the new one: renaming it changes them in the same
|
||||
// change, so the history shows (and undoes) them together.
|
||||
func renameRefs(st *model.State, kind, old, now string) {
|
||||
fix := func(list []string) {
|
||||
for i, x := range list {
|
||||
if strings.EqualFold(x, old) {
|
||||
list[i] = now
|
||||
}
|
||||
}
|
||||
}
|
||||
match := func(m *model.Match) {
|
||||
switch kind {
|
||||
case model.KindZone:
|
||||
fix(m.InZones)
|
||||
fix(m.OutZones)
|
||||
case model.KindSet:
|
||||
fix(m.Sources)
|
||||
fix(m.Destinations)
|
||||
case model.KindService:
|
||||
fix(m.Services)
|
||||
}
|
||||
}
|
||||
for _, x := range st.Rules {
|
||||
match(&x.Match)
|
||||
}
|
||||
for _, x := range st.SNATs {
|
||||
match(&x.Match)
|
||||
}
|
||||
for _, x := range st.Redirects {
|
||||
match(&x.Match)
|
||||
}
|
||||
if kind == model.KindSet {
|
||||
for _, f := range st.Forwards {
|
||||
fix(f.Sources)
|
||||
fix(f.PublicAddresses)
|
||||
fix(f.HairpinSources)
|
||||
}
|
||||
for _, b := range st.BanLists {
|
||||
fix(b.Hosts)
|
||||
fix(b.Never)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// moveObject reorders a rule or NAT rule (moveOp).
|
||||
func (s *Server) moveObject(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
kind, err := pathKind(r)
|
||||
if err != nil {
|
||||
@@ -375,72 +533,8 @@ func (s *Server) moveObject(w http.ResponseWriter, r *http.Request, p *auth.Prin
|
||||
if err := decodeJSON(r, &in); err != nil {
|
||||
return err
|
||||
}
|
||||
var id int64
|
||||
return s.change(w, r, p, "", func(st *model.State, _ func() int64) error {
|
||||
o, err := findObject(st, kind, r.PathValue("ref"))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
id = o.GetID()
|
||||
pos := position(o)
|
||||
if pos == nil {
|
||||
return badRequest("only rules, port forwards, source NAT rules and redirects have an order")
|
||||
}
|
||||
if rule, ok := o.(*model.Rule); ok && in.Chain != "" {
|
||||
rule.Chain = in.Chain
|
||||
}
|
||||
chain := chainOf(o)
|
||||
// The others in order.
|
||||
var others []model.Object
|
||||
for _, x := range st.Objects(kind) {
|
||||
if x.GetID() != id && chainOf(x) == chain {
|
||||
others = append(others, x)
|
||||
}
|
||||
}
|
||||
sort.SliceStable(others, func(i, j int) bool { return *position(others[i]) < *position(others[j]) })
|
||||
place := func(i int) { // before others[i] (i == len: at the end)
|
||||
switch {
|
||||
case len(others) == 0:
|
||||
*pos = 1
|
||||
case i == 0:
|
||||
*pos = *position(others[0]) - 1
|
||||
case i == len(others):
|
||||
*pos = *position(others[len(others)-1]) + 1
|
||||
default:
|
||||
*pos = (*position(others[i-1]) + *position(others[i])) / 2
|
||||
}
|
||||
}
|
||||
index := func(ref int64) (int, error) {
|
||||
for i, x := range others {
|
||||
if x.GetID() == ref {
|
||||
return i, nil
|
||||
}
|
||||
}
|
||||
return 0, badRequest("%d isn't another %s of the same chain", ref, kind)
|
||||
}
|
||||
switch {
|
||||
case in.Before != 0:
|
||||
i, err := index(in.Before)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
place(i)
|
||||
case in.After != 0:
|
||||
i, err := index(in.After)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
place(i + 1)
|
||||
case in.To == "start":
|
||||
place(0)
|
||||
case in.To == "end" || in.Chain != "":
|
||||
place(len(others))
|
||||
default:
|
||||
return badRequest("say where: before, after, or to start or end")
|
||||
}
|
||||
data, _ := json.Marshal(o)
|
||||
return st.Put(kind, id, data)
|
||||
}, func(st *model.State) any { return st.ByID(kind, id) })
|
||||
data, _ := json.Marshal(in)
|
||||
return s.change(w, r, p, model.Op{Op: model.OpMove, Kind: kind, Ref: r.PathValue("ref"), Data: data})
|
||||
}
|
||||
|
||||
// ---- settings
|
||||
@@ -449,7 +543,11 @@ func (s *Server) getSettings(w http.ResponseWriter, r *http.Request, p *auth.Pri
|
||||
if err := require(p, auth.ResConfig, false); err != nil {
|
||||
return err
|
||||
}
|
||||
writeJSON(w, http.StatusOK, s.eng.State().Settings)
|
||||
_, st, _, err := s.viewState(w, r)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
writeJSON(w, http.StatusOK, st.Settings)
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -458,35 +556,72 @@ func (s *Server) updateSettings(w http.ResponseWriter, r *http.Request, p *auth.
|
||||
if err := decodeJSON(r, &patch); err != nil {
|
||||
return err
|
||||
}
|
||||
return s.change(w, r, p, "", func(st *model.State, _ func() int64) error {
|
||||
var cur map[string]json.RawMessage
|
||||
b, _ := json.Marshal(st.Settings)
|
||||
json.Unmarshal(b, &cur)
|
||||
for k, v := range patch {
|
||||
// Built-ins merge one by one.
|
||||
if k == "builtins" {
|
||||
var cb, pb map[string]json.RawMessage
|
||||
json.Unmarshal(cur[k], &cb)
|
||||
if err := json.Unmarshal(v, &pb); err != nil {
|
||||
return badRequest("builtins: %v", err)
|
||||
}
|
||||
for bk, bv := range pb {
|
||||
cb[bk] = bv
|
||||
}
|
||||
v, _ = json.Marshal(cb)
|
||||
data, _ := json.Marshal(patch)
|
||||
return s.change(w, r, p, model.Op{Op: model.OpSettings, Data: data})
|
||||
}
|
||||
|
||||
// ---- the zone matrix
|
||||
|
||||
// zoneMatrix answers what the rules do to new connections between zones
|
||||
// (internal/matrix); with a changeset named, as it will make them.
|
||||
func (s *Server) zoneMatrix(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResConfig, false); err != nil {
|
||||
return err
|
||||
}
|
||||
_, st, _, err := s.viewState(w, r)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
m := matrix.Build(st, matrix.Options{Connected: s.connected(st)})
|
||||
if t, err := s.fw.Table(r.Context()); err == nil {
|
||||
matrixHits(&m, t.Counters)
|
||||
}
|
||||
writeJSON(w, http.StatusOK, m)
|
||||
return nil
|
||||
}
|
||||
|
||||
// matrixHits fills in the hits of the matrix's rules, built-ins and
|
||||
// policies from the counters.
|
||||
func matrixHits(m *api.Matrix, counters map[string]nft.Counter) {
|
||||
hits := func(marks []string) *uint64 {
|
||||
var n uint64
|
||||
found := false
|
||||
for _, mk := range marks {
|
||||
if c, ok := counters[mk]; ok {
|
||||
n, found = n+c.Packets, true
|
||||
}
|
||||
cur[k] = v
|
||||
}
|
||||
merged, _ := json.Marshal(cur)
|
||||
var next model.Settings
|
||||
dec := json.NewDecoder(strings.NewReader(string(merged)))
|
||||
dec.DisallowUnknownFields()
|
||||
if err := dec.Decode(&next); err != nil {
|
||||
return badRequest("settings: %v", err)
|
||||
if !found {
|
||||
return nil
|
||||
}
|
||||
st.Settings = next
|
||||
return nil
|
||||
}, func(st *model.State) any { return st.Settings })
|
||||
return &n
|
||||
}
|
||||
for i := range m.Cells {
|
||||
c := &m.Cells[i]
|
||||
c.Hits = hits(c.Marks)
|
||||
for j := range c.Exceptions {
|
||||
c.Exceptions[j].Hits = hits(c.Exceptions[j].Marks)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// connected maps the networks on the firewall's interfaces (link-local
|
||||
// and loopback aside) to their zones.
|
||||
func (s *Server) connected(st *model.State) map[netip.Prefix]string {
|
||||
out := map[netip.Prefix]string{}
|
||||
for _, i := range s.hostInterfaces(st) {
|
||||
if i.Zone == "" {
|
||||
continue
|
||||
}
|
||||
for _, a := range i.Addresses {
|
||||
p, err := netip.ParsePrefix(a)
|
||||
if err != nil || p.Addr().IsLinkLocalUnicast() || p.Addr().IsLoopback() {
|
||||
continue
|
||||
}
|
||||
out[p.Masked()] = i.Zone
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// ---- the ruleset
|
||||
@@ -495,7 +630,15 @@ func (s *Server) ruleset(w http.ResponseWriter, r *http.Request, p *auth.Princip
|
||||
if err := require(p, auth.ResConfig, false); err != nil {
|
||||
return err
|
||||
}
|
||||
rs := s.fw.Render(s.eng.State())
|
||||
_, st, _, err := s.viewState(w, r)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
rs := s.fw.Render(st)
|
||||
// In shadow mode: what would be loaded once it is off.
|
||||
if queryBool(r, "enforcing") {
|
||||
rs = s.fw.RenderEnforcing(st)
|
||||
}
|
||||
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
|
||||
w.Write([]byte(rs.Text))
|
||||
return nil
|
||||
@@ -534,6 +677,38 @@ func (s *Server) getHistory(w http.ResponseWriter, r *http.Request, p *auth.Prin
|
||||
return nil
|
||||
}
|
||||
|
||||
// historyDiff is the ruleset's diff for one history entry: the
|
||||
// configurations before and after it, got back by stepping back from
|
||||
// the current one through every later entry, rendered by this version.
|
||||
func (s *Server) historyDiff(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResConfig, false); err != nil {
|
||||
return err
|
||||
}
|
||||
id, err := pathID(r, "id")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
hs, err := s.store.HistorySince(r.Context(), id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if len(hs) == 0 || hs[len(hs)-1].ID != id {
|
||||
return notFound("history entry %d not found", id)
|
||||
}
|
||||
st := s.eng.State()
|
||||
var after *model.State
|
||||
for _, h := range hs {
|
||||
if h.ID == id {
|
||||
after = st.Clone()
|
||||
}
|
||||
if err := engine.Rewind(st, h); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]string{"diff": s.rulesetDiff(st, after)})
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Server) revertHistory(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResConfig, true); err != nil {
|
||||
return err
|
||||
@@ -542,13 +717,30 @@ func (s *Server) revertHistory(w http.ResponseWriter, r *http.Request, p *auth.P
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
// Staged, an undo is an operation of its own, reviewed like any.
|
||||
if csID, err := changesetParam(r); err != nil {
|
||||
return err
|
||||
} else if csID != 0 {
|
||||
h, err := s.store.GetHistory(r.Context(), id)
|
||||
if errors.Is(err, store.ErrNotFound) {
|
||||
return notFound("history entry %d not found", id)
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if h.RevertedBy != 0 {
|
||||
return conflict("history entry %d was already undone by entry %d", id, h.RevertedBy)
|
||||
}
|
||||
data, _ := json.Marshal(h)
|
||||
return s.stage(w, r, p, csID, model.Op{Op: model.OpRevert, ID: id, Data: data})
|
||||
}
|
||||
res, err := s.eng.Revert(r.Context(), p, id, engine.Options{DryRun: queryBool(r, "dry_run"), Via: via(r, p)})
|
||||
if err != nil {
|
||||
s.writeErr(w, r, err, res)
|
||||
return nil
|
||||
}
|
||||
s.afterChange(r.Context(), p, res)
|
||||
s.respondChange(w, r, res, nil, false)
|
||||
s.respondChange(w, r, p, res, nil, false)
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -566,7 +758,17 @@ func (s *Server) confirmChanges(w http.ResponseWriter, r *http.Request, p *auth.
|
||||
if err := require(p, auth.ResConfig, true); err != nil {
|
||||
return err
|
||||
}
|
||||
n, err := s.confirm(r.Context(), p)
|
||||
var in api.ConfirmInput
|
||||
if r.ContentLength != 0 {
|
||||
if err := decodeJSON(r, &in); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
confirm := func() (int, error) { return s.confirm(r.Context(), p) }
|
||||
if in.Token != "" {
|
||||
confirm = func() (int, error) { return s.confirmToken(r, p, in.Token) }
|
||||
}
|
||||
n, err := confirm()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -586,3 +788,19 @@ func (s *Server) rollbackChanges(w http.ResponseWriter, r *http.Request, p *auth
|
||||
writeJSON(w, http.StatusOK, map[string]int{"rolled_back": n})
|
||||
return nil
|
||||
}
|
||||
|
||||
// listOwnBanLists lists the ban lists a principal without config:read
|
||||
// may use.
|
||||
func (s *Server) listOwnBanLists(w http.ResponseWriter, p *auth.Principal) error {
|
||||
out := []*model.BanList{}
|
||||
for _, b := range s.eng.State().BanLists {
|
||||
if p.CanBan(b.Name, false) {
|
||||
out = append(out, b)
|
||||
}
|
||||
}
|
||||
if len(out) == 0 && !p.Can(auth.ResBans, false) {
|
||||
return require(p, auth.ResConfig, false)
|
||||
}
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,77 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"housecarl/internal/version"
|
||||
)
|
||||
|
||||
// fallbackInForce is what status and /healthz say while the boot fallback
|
||||
// (render.Fallback) is in force.
|
||||
const fallbackInForce = "the boot fallback is in force, not Housecarl's ruleset: the saved ruleset failed to load at boot (journalctl -u housecarl-ruleset), and housecarld couldn't apply it since; it lets nothing new in from external zones (with none, nothing but loopback)"
|
||||
|
||||
// health is GET /healthz, for monitoring (Checkmk, a load balancer). It
|
||||
// needs no authentication, so it says only what a monitor needs, and reads
|
||||
// the ruleset at most every few seconds however often it is asked.
|
||||
type health struct {
|
||||
OK bool `json:"ok"`
|
||||
Version string `json:"version"`
|
||||
Firewall string `json:"firewall"` // applied, unconfigured, fallback, missing
|
||||
// Mode is enforcing, or shadow: the ruleset blocks nothing.
|
||||
Mode string `json:"mode"`
|
||||
// Probation: changes waiting to be confirmed, rolled back otherwise.
|
||||
Probation int `json:"probation"`
|
||||
// Warnings are the codes of the machine's problems (nftables_service).
|
||||
Warnings []string `json:"warnings"`
|
||||
}
|
||||
|
||||
const healthTTL = 5 * time.Second
|
||||
|
||||
type healthCache struct {
|
||||
mu sync.Mutex
|
||||
at time.Time
|
||||
out health
|
||||
}
|
||||
|
||||
func (s *Server) healthz(w http.ResponseWriter, r *http.Request) {
|
||||
c := &s.health
|
||||
c.mu.Lock()
|
||||
if time.Since(c.at) >= healthTTL {
|
||||
c.out, c.at = s.checkHealth(r), time.Now()
|
||||
}
|
||||
out := c.out
|
||||
c.mu.Unlock()
|
||||
code := http.StatusOK
|
||||
if !out.OK {
|
||||
code = http.StatusServiceUnavailable
|
||||
}
|
||||
writeJSON(w, code, out)
|
||||
}
|
||||
|
||||
func (s *Server) checkHealth(r *http.Request) health {
|
||||
out := health{Version: version.String(), Warnings: []string{}, Mode: s.mode()}
|
||||
t, err := s.fw.Table(r.Context())
|
||||
switch {
|
||||
case err != nil && s.eng.State().Empty():
|
||||
out.Firewall, out.OK = "unconfigured", true
|
||||
case err != nil:
|
||||
out.Firewall = "missing"
|
||||
case t.Fallback:
|
||||
out.Firewall = "fallback"
|
||||
default:
|
||||
out.Firewall, out.OK = "applied", true
|
||||
}
|
||||
pr := s.probationStatus()
|
||||
out.Probation = len(pr.Pending) + pr.Takeovers
|
||||
// The other tables matter in shadow mode (shadow_unprotected).
|
||||
var ruleset []byte
|
||||
if s.cfg.Shadow {
|
||||
ruleset, _ = s.fw.ListRuleset(r.Context())
|
||||
}
|
||||
for _, p := range s.hostProblems(r.Context(), ruleset) {
|
||||
out.Warnings = append(out.Warnings, p.Code)
|
||||
}
|
||||
return out
|
||||
}
|
||||
@@ -0,0 +1,65 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"housecarl/internal/api"
|
||||
)
|
||||
|
||||
func TestHealthz(t *testing.T) {
|
||||
e := newEnv(t)
|
||||
link := filepath.Join(t.TempDir(), "nftables.service")
|
||||
e.srv.unitLinks = []string{link}
|
||||
get := func() (int, health) {
|
||||
t.Helper()
|
||||
e.srv.health.at = time.Time{} // no caching between the steps
|
||||
resp, err := http.Get(e.ts.URL + "/healthz")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
var h health
|
||||
json.NewDecoder(resp.Body).Decode(&h)
|
||||
return resp.StatusCode, h
|
||||
}
|
||||
// Nothing configured yet: nothing to be in force.
|
||||
if code, h := get(); code != 200 || !h.OK || h.Firewall != "unconfigured" || h.Version == "" {
|
||||
t.Errorf("unconfigured: %d %+v", code, h)
|
||||
}
|
||||
if code := e.call(e.token, "POST", "/zones", map[string]any{"name": "wan", "interfaces": []string{"wan"}, "external": true}, nil); code != 201 {
|
||||
t.Fatalf("creating a zone: %d", code)
|
||||
}
|
||||
if code, h := get(); code != 200 || h.Firewall != "applied" || h.Probation != 1 || len(h.Warnings) != 0 {
|
||||
t.Errorf("applied, on probation: %d %+v", code, h)
|
||||
}
|
||||
os.WriteFile(link, nil, 0o644)
|
||||
if _, h := get(); strings.Join(h.Warnings, ",") != "nftables_service" {
|
||||
t.Errorf("warnings %v", h.Warnings)
|
||||
}
|
||||
e.fw.mu.Lock()
|
||||
e.fw.fallback = true
|
||||
e.fw.mu.Unlock()
|
||||
if code, h := get(); code != 503 || h.OK || h.Firewall != "fallback" {
|
||||
t.Errorf("fallback: %d %+v", code, h)
|
||||
}
|
||||
var st api.Status
|
||||
e.call(e.token, "GET", "/status", nil, &st)
|
||||
if st.Applied || !strings.Contains(st.ApplyError, "boot fallback") {
|
||||
t.Errorf("status with the fallback: applied %v, %q", st.Applied, st.ApplyError)
|
||||
}
|
||||
// Asked often, the ruleset is read once per few seconds.
|
||||
e.fw.mu.Lock()
|
||||
e.fw.fallback = false
|
||||
e.fw.mu.Unlock()
|
||||
resp, _ := http.Get(e.ts.URL + "/healthz")
|
||||
resp.Body.Close()
|
||||
if resp.StatusCode != 503 {
|
||||
t.Errorf("not cached: %d", resp.StatusCode)
|
||||
}
|
||||
}
|
||||
@@ -18,11 +18,14 @@ func (s *Server) importNft(w http.ResponseWriter, r *http.Request, p *auth.Princ
|
||||
if err := require(p, auth.ResConfig, true); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := notStaged(r, "an import"); err != nil {
|
||||
return err
|
||||
}
|
||||
var in api.ImportInput
|
||||
if err := decodeJSON(r, &in); err != nil {
|
||||
return err
|
||||
}
|
||||
res, err := importer.Import(importer.Input{JSON: in.JSON, Text: in.Text, Defines: in.Defines, Table: in.Table})
|
||||
res, err := importer.Import(importer.Input{JSON: in.JSON, Text: in.Text, Defines: in.Defines, Table: in.Table, Zones: in.Zones})
|
||||
if err != nil {
|
||||
return badRequest("%v", err)
|
||||
}
|
||||
@@ -48,6 +51,12 @@ func (s *Server) importNft(w http.ResponseWriter, r *http.Request, p *auth.Princ
|
||||
return nil
|
||||
}
|
||||
s.afterChange(r.Context(), p, er)
|
||||
if er.Applied {
|
||||
// Until it is taken over, the old table still applies too: the
|
||||
// status warns (takeover_pending; not in shadow mode, where that
|
||||
// is the point).
|
||||
s.store.SetMeta(r.Context(), importedTableKey, res.Table)
|
||||
}
|
||||
out.MutationResponse = api.MutationResponse{Applied: er.Applied, DryRun: er.DryRun, HistoryID: er.HistoryID, Changes: er.Changes, Warnings: er.Warnings}
|
||||
if out.Changes == nil {
|
||||
out.Changes = []model.Change{}
|
||||
@@ -55,6 +64,7 @@ func (s *Server) importNft(w http.ResponseWriter, r *http.Request, p *auth.Princ
|
||||
if pr := s.probationStatus(); er.Applied && len(pr.Pending) > 0 {
|
||||
out.ConfirmBy = &pr.Deadline
|
||||
}
|
||||
out.ConfirmToken = s.issueCheck(w, r, p, er)
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
return nil
|
||||
}
|
||||
@@ -105,6 +115,12 @@ func (s *Server) takeover(w http.ResponseWriter, r *http.Request, p *auth.Princi
|
||||
if err := requireAdmin(p); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := notStaged(r, "a takeover"); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := s.refuseInShadow("delete other tables (the firewall in force)"); err != nil {
|
||||
return err
|
||||
}
|
||||
var in api.TakeoverInput
|
||||
if err := decodeJSON(r, &in); err != nil {
|
||||
return err
|
||||
@@ -142,6 +158,12 @@ func (s *Server) takeover(w http.ResponseWriter, r *http.Request, p *auth.Princi
|
||||
} else if pr := s.probationStatus(); pr.Takeovers > 0 {
|
||||
msg += fmt.Sprintf("; confirm with 'hc confirm' within %ds, or they come back", pr.SecondsLeft)
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]any{"deleted": done, "message": msg})
|
||||
answer := map[string]any{"deleted": done, "message": msg}
|
||||
if !queryBool(r, "dry_run") && len(done) > 0 {
|
||||
if tok := s.issueCheck(w, r, p, nil); tok != "" {
|
||||
answer["confirm_token"] = tok
|
||||
}
|
||||
}
|
||||
writeJSON(w, http.StatusOK, answer)
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,266 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"housecarl/internal/api"
|
||||
"housecarl/internal/auth"
|
||||
"housecarl/internal/model"
|
||||
"housecarl/internal/stats"
|
||||
"housecarl/internal/validate"
|
||||
)
|
||||
|
||||
// Statistics: the firewall's figures over time (internal/stats).
|
||||
|
||||
// StatsSources fills in what the statistics collector needs from the
|
||||
// server: the interfaces' zones, and which counters count drops.
|
||||
func (s *Server) StatsSources(src *stats.Sources) {
|
||||
src.Zones = s.interfaceZones
|
||||
src.Classify = s.classifyMark
|
||||
}
|
||||
|
||||
// SetStats gives the server the statistics collector (nil: statistics
|
||||
// are off).
|
||||
func (s *Server) SetStats(c *stats.Collector) { s.stats = c }
|
||||
|
||||
// interfaceZones maps the machine's interfaces to their zones.
|
||||
func (s *Server) interfaceZones() map[string]string {
|
||||
out := map[string]string{}
|
||||
for _, i := range s.hostInterfaces(s.eng.State()) {
|
||||
if i.Zone != "" {
|
||||
out[i.Name] = i.Zone
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// classifyMark says which chain a counter is in and whether what it
|
||||
// counts is dropped: rules that drop or reject, the drop policies, and
|
||||
// the built-ins that drop (antispoofing, invalid packets).
|
||||
func (s *Server) classifyMark(mark string) (string, bool) {
|
||||
st := s.eng.State()
|
||||
switch {
|
||||
case strings.HasPrefix(mark, "p-"):
|
||||
chain := strings.TrimPrefix(mark, "p-")
|
||||
policy := map[string]string{"input": st.Settings.InputPolicy, "forward": st.Settings.ForwardPolicy, "output": st.Settings.OutputPolicy}[chain]
|
||||
return chain, policy == model.PolicyDrop
|
||||
case strings.HasPrefix(mark, "b-antispoof"):
|
||||
return model.ChainEarly, true
|
||||
case mark == "b-invalid":
|
||||
return "invalid", true
|
||||
case strings.HasPrefix(mark, "r"):
|
||||
num := strings.TrimLeft(mark[1:], "0123456789")
|
||||
id, _ := strconv.ParseInt(strings.TrimSuffix(mark[1:], num), 10, 64)
|
||||
if o, ok := st.ByID(model.KindRule, id).(*model.Rule); ok && o != nil {
|
||||
return o.Chain, o.Action == model.ActionDrop || o.Action == model.ActionReject
|
||||
}
|
||||
}
|
||||
return "", false
|
||||
}
|
||||
|
||||
// markName names a counter's object: "rule input #12 (ssh)", "built-in:
|
||||
// established", "input policy".
|
||||
func markName(st *model.State, mark string) string {
|
||||
if c, ok := strings.CutPrefix(mark, "p-"); ok {
|
||||
return c + " policy"
|
||||
}
|
||||
if l := markLabel(st, api.TraceStep{Kind: "rule", Mark: mark}); l != "" {
|
||||
return l
|
||||
}
|
||||
if len(mark) < 2 {
|
||||
return mark
|
||||
}
|
||||
// An object deleted since it counted.
|
||||
kind := map[byte]string{'r': "rule", 'f': "port forward", 's': "source NAT", 'd': "redirect"}[mark[0]]
|
||||
if num := strings.TrimLeft(mark[1:], "0123456789"); kind != "" && len(num) < len(mark)-1 {
|
||||
return kind + " #" + strings.TrimSuffix(mark[1:], num) + " (deleted)"
|
||||
}
|
||||
return mark
|
||||
}
|
||||
|
||||
func (s *Server) statsCollector() (*stats.Collector, error) {
|
||||
if s.stats == nil {
|
||||
return nil, &httpError{http.StatusServiceUnavailable, "stats_off", "statistics are off: set stats.enabled in housecarl.yaml and restart housecarld"}
|
||||
}
|
||||
return s.stats, nil
|
||||
}
|
||||
|
||||
func (s *Server) statsQuerier(p *auth.Principal) (*stats.Querier, error) {
|
||||
if err := require(p, auth.ResLog, false); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
c, err := s.statsCollector()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
q := stats.NewQuerier(c)
|
||||
q.Now = s.nowFn
|
||||
return q, nil
|
||||
}
|
||||
|
||||
func since(r *http.Request, def string) (string, error) {
|
||||
v := r.URL.Query().Get("since")
|
||||
if v == "" {
|
||||
v = def
|
||||
}
|
||||
if _, err := stats.ParseSince(v); err != nil {
|
||||
return "", badRequest("%v", err)
|
||||
}
|
||||
return v, nil
|
||||
}
|
||||
|
||||
// StatsStatus is the collector's health, with the setting that counts
|
||||
// traffic per device.
|
||||
type StatsStatus struct {
|
||||
stats.Status
|
||||
Latest *stats.Latest `json:"latest,omitempty"`
|
||||
}
|
||||
|
||||
func (s *Server) statsStatus(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResLog, false); err != nil {
|
||||
return err
|
||||
}
|
||||
out := StatsStatus{}
|
||||
if s.stats != nil {
|
||||
out.Status = s.stats.Status()
|
||||
out.Latest = s.stats.Latest()
|
||||
if !p.IsAdmin() {
|
||||
out.Database = ""
|
||||
}
|
||||
}
|
||||
out.PerDevice = s.eng.State().Settings.StatsPerDevice
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Server) statsSummary(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
q, err := s.statsQuerier(p)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
v, err := since(r, "24h")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
d, _ := stats.ParseSince(v)
|
||||
sum, err := q.Summary(r.Context(), d, queryInt(r, "limit", 10))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
st := s.eng.State()
|
||||
for i := range sum.Rules {
|
||||
sum.Rules[i].Name = markName(st, sum.Rules[i].Key)
|
||||
}
|
||||
writeJSON(w, http.StatusOK, sum)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Server) statsTimeline(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
q, err := s.statsQuerier(p)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
v, err := since(r, "24h")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
d, _ := stats.ParseSince(v)
|
||||
var keys []string
|
||||
for _, k := range strings.Split(r.URL.Query().Get("keys"), ",") {
|
||||
if k = strings.TrimSpace(k); k != "" {
|
||||
keys = append(keys, k)
|
||||
}
|
||||
}
|
||||
if len(keys) == 0 {
|
||||
return badRequest("say which figures: keys=zrx:*,ztx:* (docs/stats.md)")
|
||||
}
|
||||
tl, err := q.Timeline(r.Context(), d, r.URL.Query().Get("step"), keys)
|
||||
if err != nil {
|
||||
return badRequest("%v", err)
|
||||
}
|
||||
writeJSON(w, http.StatusOK, tl)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Server) statsTop(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
q, err := s.statsQuerier(p)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
v, err := since(r, "24h")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
d, _ := stats.ParseSince(v)
|
||||
es, err := q.Top(r.Context(), r.URL.Query().Get("kind"), d, queryInt(r, "limit", 20))
|
||||
if err != nil {
|
||||
return badRequest("%v", err)
|
||||
}
|
||||
writeJSON(w, http.StatusOK, es)
|
||||
return nil
|
||||
}
|
||||
|
||||
// statsHits are the packets each rule, built-in and policy counted over
|
||||
// the range, named; rules that counted none are listed too, with 0.
|
||||
func (s *Server) statsHits(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
q, err := s.statsQuerier(p)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
v, err := since(r, "30d")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
d, _ := stats.ParseSince(v)
|
||||
es, err := q.Sums(r.Context(), "hits:", d)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
st := s.eng.State()
|
||||
seen := map[string]bool{}
|
||||
for i := range es {
|
||||
es[i].Name = markName(st, es[i].Key)
|
||||
seen[es[i].Key] = true
|
||||
}
|
||||
for _, rule := range st.Rules {
|
||||
if mark := "r" + strconv.FormatInt(rule.ID, 10); !seen[mark] && !rule.Disabled {
|
||||
es = append(es, stats.Entry{Key: mark, Name: "rule " + rule.Label()})
|
||||
}
|
||||
}
|
||||
writeJSON(w, http.StatusOK, es)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Server) statsHeatmap(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
q, err := s.statsQuerier(p)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
v, err := since(r, "28d")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
d, _ := stats.ParseSince(v)
|
||||
hm, err := q.Heatmap(r.Context(), r.URL.Query().Get("key"), d)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
writeJSON(w, http.StatusOK, hm)
|
||||
return nil
|
||||
}
|
||||
|
||||
// statsProblems warns when the connection tracking table is nearly full:
|
||||
// new connections are dropped once it is.
|
||||
func (s *Server) statsProblems() []validate.Problem {
|
||||
if s.stats == nil {
|
||||
return nil
|
||||
}
|
||||
l := s.stats.Latest()
|
||||
if l == nil || l.ConntrackMax == 0 || l.Conntrack*100 < l.ConntrackMax*80 {
|
||||
return nil
|
||||
}
|
||||
return []validate.Problem{{Severity: validate.Warning, Code: "conntrack_full", Object: "connections",
|
||||
Message: "the connection tracking table is " + strconv.FormatInt(l.Conntrack*100/l.ConntrackMax, 10) + "% full (" + strconv.FormatInt(l.Conntrack, 10) + " of " + strconv.FormatInt(l.ConntrackMax, 10) + "); once it is, new connections are dropped: raise net.netfilter.nf_conntrack_max, or find what opens so many (Connections)"}}
|
||||
}
|
||||
@@ -1,7 +1,10 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"os"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
@@ -24,7 +27,10 @@ func (s *Server) status(w http.ResponseWriter, r *http.Request, p *auth.Principa
|
||||
}
|
||||
st := s.eng.State()
|
||||
out := api.Status{Build: version.Info(), Started: s.started, Problems: s.eng.Problems(), Probation: s.probationStatus(),
|
||||
Interfaces: s.hostInterfaces(st), Counts: map[string]int{}, OtherTables: []string{}}
|
||||
Interfaces: s.hostInterfaces(st), Counts: map[string]int{}, OtherTables: []string{}, Mode: s.mode()}
|
||||
if s.cfg.Shadow {
|
||||
out.ShadowPosition = s.cfg.ShadowPosition
|
||||
}
|
||||
if out.Problems == nil {
|
||||
out.Problems = []validate.Problem{}
|
||||
}
|
||||
@@ -34,20 +40,78 @@ func (s *Server) status(w http.ResponseWriter, r *http.Request, p *auth.Principa
|
||||
for _, k := range model.Kinds {
|
||||
out.Counts[k] = len(st.Objects(k))
|
||||
}
|
||||
if _, err := s.fw.Table(r.Context()); err != nil && st.Empty() {
|
||||
if t, err := s.fw.Table(r.Context()); err != nil && st.Empty() {
|
||||
out.Unconfigured = true
|
||||
} else if err != nil {
|
||||
out.ApplyError = err.Error()
|
||||
} else if t.Fallback {
|
||||
out.ApplyError = fallbackInForce
|
||||
} else {
|
||||
out.Applied = true
|
||||
}
|
||||
if data, err := s.fw.ListRuleset(r.Context()); err == nil {
|
||||
data, err := s.fw.ListRuleset(r.Context())
|
||||
if err == nil {
|
||||
out.OtherTables = otherTables(data)
|
||||
}
|
||||
out.Problems = append(out.Problems, s.hostProblems(r.Context(), data)...)
|
||||
out.Problems = append(out.Problems, s.statsProblems()...)
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
return nil
|
||||
}
|
||||
|
||||
const importedTableKey = "imported_table"
|
||||
|
||||
// shadowProblems are shadow mode's warnings about the machine.
|
||||
func (s *Server) shadowProblems(ruleset []byte) []validate.Problem {
|
||||
var out []validate.Problem
|
||||
if ruleset != nil && len(inputFilters(ruleset)) == 0 {
|
||||
out = append(out, validate.Problem{Severity: validate.Warning, Code: "shadow_unprotected", Object: "shadow mode",
|
||||
Message: "housecarld is in shadow mode, where its ruleset blocks nothing, and no other table filters what reaches this machine (none has a filter chain on the input hook): nothing protects it; load your firewall, or leave shadow mode"})
|
||||
}
|
||||
for _, r := range render.ShadowLeftOut(s.eng.State()) {
|
||||
out = append(out, validate.Problem{Severity: validate.Warning, Code: "shadow_raw_left_out", Object: r.Label(),
|
||||
Message: "shadow mode leaves this raw rule out: it does more to packets than accept, drop or reject (NAT, setting a mark or field, a jump, a verdict map…), so its hits aren't counted and later rules see what it would have taken"})
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// nftablesUnitLinks are where an enabled nftables.service shows (Debian
|
||||
// enables it for sysinit.target).
|
||||
var nftablesUnitLinks = []string{
|
||||
"/etc/systemd/system/sysinit.target.wants/nftables.service",
|
||||
"/etc/systemd/system/multi-user.target.wants/nftables.service",
|
||||
}
|
||||
|
||||
// hostProblems are warnings about the machine rather than the
|
||||
// configuration: another ruleset that still loads at boot, or the table
|
||||
// an import came from, still in force beside Housecarl's. ruleset is the
|
||||
// machine's (nft -j list ruleset; nil: not read). In shadow mode those
|
||||
// are expected; the warnings are that nothing else filters, and the raw
|
||||
// rules left out.
|
||||
func (s *Server) hostProblems(ctx context.Context, ruleset []byte) []validate.Problem {
|
||||
if s.cfg.Shadow {
|
||||
return s.shadowProblems(ruleset)
|
||||
}
|
||||
others := otherTables(ruleset)
|
||||
var out []validate.Problem
|
||||
for _, link := range s.unitLinks {
|
||||
if _, err := os.Lstat(link); err == nil {
|
||||
out = append(out, validate.Problem{Severity: validate.Warning, Code: "nftables_service", Object: "nftables.service",
|
||||
Message: "nftables.service is enabled: at boot it loads /etc/nftables.conf, and stopping or restarting it flushes every table, Housecarl's included; disable it (systemctl disable nftables, without --now)"})
|
||||
break
|
||||
}
|
||||
}
|
||||
if t, err := s.store.GetMeta(ctx, importedTableKey); err == nil && t != "" {
|
||||
for _, o := range others {
|
||||
if o == t {
|
||||
out = append(out, validate.Problem{Severity: validate.Warning, Code: "takeover_pending", Object: "table " + t,
|
||||
Message: fmt.Sprintf("%s, the table you imported, still applies beside Housecarl's: a packet must get past both; once Housecarl's rules are right, delete it with 'hc takeover %q'", t, t)})
|
||||
}
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// hostInterfaces lists the machine's interfaces with their zones.
|
||||
func (s *Server) hostInterfaces(st *model.State) []api.HostInterface {
|
||||
out := []api.HostInterface{}
|
||||
@@ -260,7 +324,13 @@ func (l *logRing) Latest(n int, after time.Time, keep func(api.LogEntry) bool) [
|
||||
|
||||
// LogPacket records a packet the firewall logged (internal/nflog calls
|
||||
// it).
|
||||
func (s *Server) LogPacket(e api.LogEntry) { s.logs.Add(e) }
|
||||
func (s *Server) LogPacket(e api.LogEntry) {
|
||||
e.Shadow = s.cfg.Shadow
|
||||
s.logs.Add(e)
|
||||
if s.stats != nil {
|
||||
s.stats.Packet(e)
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Server) listLog(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResLog, false); err != nil {
|
||||
|
||||
@@ -0,0 +1,396 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/netip"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"housecarl/internal/api"
|
||||
"housecarl/internal/auth"
|
||||
"housecarl/internal/model"
|
||||
"housecarl/internal/trace"
|
||||
)
|
||||
|
||||
// Traffic as it happens: the connections the kernel tracks (and closing
|
||||
// them, so the rules check them again), and tracing packets through the
|
||||
// whole ruleset.
|
||||
|
||||
// Conntrack lists and closes tracked connections (internal/conntrack).
|
||||
type Conntrack interface {
|
||||
List(ctx context.Context) ([]api.Connection, error)
|
||||
Close(ctx context.Context, c api.Connection) error
|
||||
}
|
||||
|
||||
// SetConntrack gives the server the kernel's connection tracking.
|
||||
func (s *Server) SetConntrack(c Conntrack) { s.conns = c }
|
||||
|
||||
func (s *Server) conntrack() (Conntrack, error) {
|
||||
if s.conns == nil {
|
||||
return nil, &httpError{http.StatusServiceUnavailable, "unavailable", "connection tracking isn't available here"}
|
||||
}
|
||||
return s.conns, nil
|
||||
}
|
||||
|
||||
// connMatches: a search in a connection's addresses, ports (":443"),
|
||||
// protocol, state and NAT addresses.
|
||||
func connMatches(c api.Connection, q string) bool {
|
||||
if q == "" {
|
||||
return true
|
||||
}
|
||||
hay := strings.ToLower(strings.Join([]string{c.Protocol, c.Source, ":" + strconv.Itoa(c.SourcePort), c.Destination, ":" + strconv.Itoa(c.DestinationPort),
|
||||
c.DNATTo, c.SNATTo, c.State}, " ") + " ")
|
||||
for _, w := range strings.Fields(strings.ToLower(q)) {
|
||||
if strings.HasPrefix(w, ":") {
|
||||
w += " " // a whole port
|
||||
}
|
||||
if !strings.Contains(hay, w) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func (s *Server) listConnections(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResLog, false); err != nil {
|
||||
return err
|
||||
}
|
||||
ct, err := s.conntrack()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
all, err := ct.List(r.Context())
|
||||
if err != nil {
|
||||
return fmt.Errorf("listing connections: %w", err)
|
||||
}
|
||||
q := r.URL.Query().Get("q")
|
||||
out := []api.Connection{}
|
||||
for _, c := range all {
|
||||
if connMatches(c, q) {
|
||||
out = append(out, c)
|
||||
}
|
||||
}
|
||||
sort.Slice(out, func(i, j int) bool {
|
||||
if out[i].Source != out[j].Source {
|
||||
return out[i].Source < out[j].Source
|
||||
}
|
||||
return out[i].Destination < out[j].Destination
|
||||
})
|
||||
if n := queryInt(r, "limit", 1000); n > 0 && len(out) > n {
|
||||
out = out[:n]
|
||||
}
|
||||
w.Header().Set("X-Total-Count", strconv.Itoa(len(all)))
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
return nil
|
||||
}
|
||||
|
||||
// lookupNames finds the names of the inside addresses asked about
|
||||
// (docs/configuration.md#names), waiting at most a second for lookups.
|
||||
func (s *Server) lookupNames(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResLog, false); err != nil {
|
||||
return err
|
||||
}
|
||||
var in api.NamesInput
|
||||
if err := decodeJSON(r, &in); err != nil {
|
||||
return err
|
||||
}
|
||||
if len(in.Addresses) > maxNames {
|
||||
return badRequest("ask for at most %d addresses at once", maxNames)
|
||||
}
|
||||
out := api.Names{Enabled: s.cfg.Names != nil, Names: map[string]string{}}
|
||||
if s.cfg.Names == nil {
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
return nil
|
||||
}
|
||||
inside := s.insideNetworks(s.eng.State())
|
||||
asked := map[netip.Addr][]string{}
|
||||
var addrs []netip.Addr
|
||||
for _, a := range in.Addresses {
|
||||
ad, err := netip.ParseAddr(a)
|
||||
if err != nil || !inside(ad.Unmap()) {
|
||||
continue
|
||||
}
|
||||
ad = ad.Unmap()
|
||||
if _, ok := asked[ad]; !ok {
|
||||
addrs = append(addrs, ad)
|
||||
}
|
||||
asked[ad] = append(asked[ad], a)
|
||||
}
|
||||
for a, n := range s.cfg.Names.Names(r.Context(), addrs, time.Second) {
|
||||
for _, as := range asked[a] {
|
||||
out.Names[as] = n
|
||||
}
|
||||
}
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
return nil
|
||||
}
|
||||
|
||||
// maxNames is the most addresses one POST /names may ask about.
|
||||
const maxNames = 2000
|
||||
|
||||
// insideNetworks tells inside addresses: in a network connected to an
|
||||
// inside zone's interface, or private and not in an external zone's.
|
||||
func (s *Server) insideNetworks(st *model.State) func(netip.Addr) bool {
|
||||
external := map[string]bool{}
|
||||
for _, z := range st.Zones {
|
||||
external[z.Name] = z.External
|
||||
}
|
||||
var in, out []netip.Prefix
|
||||
for p, zone := range s.connected(st) {
|
||||
if external[zone] {
|
||||
out = append(out, p)
|
||||
} else {
|
||||
in = append(in, p)
|
||||
}
|
||||
}
|
||||
return func(a netip.Addr) bool {
|
||||
if a.IsLoopback() || a.IsUnspecified() || a.IsMulticast() || a.IsLinkLocalUnicast() {
|
||||
return false
|
||||
}
|
||||
for _, p := range in {
|
||||
if p.Contains(a) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
for _, p := range out {
|
||||
if p.Contains(a) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return a.IsPrivate()
|
||||
}
|
||||
}
|
||||
|
||||
// closeConnections deletes tracking entries: the rules check those
|
||||
// connections again with their next packet.
|
||||
func (s *Server) closeConnections(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResConfig, true); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := s.refuseInShadow("close connections"); err != nil {
|
||||
return err
|
||||
}
|
||||
ct, err := s.conntrack()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var in api.CloseConnections
|
||||
if err := decodeJSON(r, &in); err != nil {
|
||||
return err
|
||||
}
|
||||
list := in.Connections
|
||||
if strings.TrimSpace(in.Search) != "" {
|
||||
all, err := ct.List(r.Context())
|
||||
if err != nil {
|
||||
return fmt.Errorf("listing connections: %w", err)
|
||||
}
|
||||
for _, c := range all {
|
||||
if connMatches(c, in.Search) {
|
||||
list = append(list, c)
|
||||
}
|
||||
}
|
||||
} else if len(list) == 0 {
|
||||
return badRequest("say which connections: a list, or a search")
|
||||
}
|
||||
closed := 0
|
||||
for _, c := range list {
|
||||
if err := ct.Close(r.Context(), c); err != nil {
|
||||
return fmt.Errorf("closing %s %s -> %s: %w", c.Protocol, c.Source, c.Destination, err)
|
||||
}
|
||||
closed++
|
||||
}
|
||||
what := in.Search
|
||||
if what == "" && len(list) == 1 {
|
||||
what = fmt.Sprintf("%s %s:%d -> %s:%d", list[0].Protocol, list[0].Source, list[0].SourcePort, list[0].Destination, list[0].DestinationPort)
|
||||
}
|
||||
s.eng.Audit(r.Context(), p, "connections.close", what, "success", fmt.Sprintf("%d closed", closed))
|
||||
writeJSON(w, http.StatusOK, map[string]int{"closed": closed})
|
||||
return nil
|
||||
}
|
||||
|
||||
// ---- tracing
|
||||
|
||||
type tracing struct {
|
||||
mu sync.Mutex
|
||||
status api.TraceStatus
|
||||
tracer *trace.Tracer
|
||||
cancel context.CancelFunc
|
||||
}
|
||||
|
||||
func (s *Server) startTrace(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResConfig, true); err != nil {
|
||||
return err
|
||||
}
|
||||
var in api.TraceInput
|
||||
if err := decodeJSON(r, &in); err != nil {
|
||||
return err
|
||||
}
|
||||
if in.Seconds == 0 {
|
||||
in.Seconds = 60
|
||||
}
|
||||
if in.Seconds < 5 || in.Seconds > 600 {
|
||||
return badRequest("trace for 5 to 600 seconds")
|
||||
}
|
||||
script, err := trace.Script(in)
|
||||
if err != nil {
|
||||
return badRequest("%v", err)
|
||||
}
|
||||
t := &s.tracing
|
||||
t.mu.Lock()
|
||||
if t.status.Running {
|
||||
t.mu.Unlock()
|
||||
return conflict("a trace is running already (started %s); stop it first", t.status.Started.Format("15:04:05"))
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), time.Duration(in.Seconds)*time.Second)
|
||||
now := s.nowFn()
|
||||
t.status = api.TraceStatus{Running: true, Input: in, Started: now, Ends: now.Add(time.Duration(in.Seconds) * time.Second), Packets: []api.TracePacket{}}
|
||||
t.tracer = &trace.Tracer{Now: s.nowFn}
|
||||
t.cancel = cancel
|
||||
t.mu.Unlock()
|
||||
s.eng.Audit(r.Context(), p, "trace.start", traceWhat(in), "success", fmt.Sprintf("%d seconds", in.Seconds))
|
||||
go func() {
|
||||
defer cancel()
|
||||
err := s.fw.Trace(ctx, script, func(line string) {
|
||||
t.mu.Lock()
|
||||
defer t.mu.Unlock()
|
||||
t.tracer.Line(line)
|
||||
if t.tracer.Full() {
|
||||
cancel()
|
||||
}
|
||||
})
|
||||
t.mu.Lock()
|
||||
defer t.mu.Unlock()
|
||||
t.status.Running = false
|
||||
if err != nil {
|
||||
t.status.Error = err.Error()
|
||||
s.log.Warn("tracing failed", "err", err)
|
||||
}
|
||||
}()
|
||||
writeJSON(w, http.StatusOK, s.traceStatus())
|
||||
return nil
|
||||
}
|
||||
|
||||
func traceWhat(in api.TraceInput) string {
|
||||
var parts []string
|
||||
if in.Address != "" {
|
||||
parts = append(parts, in.Address)
|
||||
}
|
||||
if in.Protocol != "" {
|
||||
parts = append(parts, in.Protocol)
|
||||
}
|
||||
if in.Port != 0 {
|
||||
parts = append(parts, "port "+strconv.Itoa(in.Port))
|
||||
}
|
||||
return strings.Join(parts, " ")
|
||||
}
|
||||
|
||||
// traceStatus is the trace with each step named after Housecarl's object.
|
||||
func (s *Server) traceStatus() api.TraceStatus {
|
||||
t := &s.tracing
|
||||
t.mu.Lock()
|
||||
out := t.status
|
||||
if t.tracer != nil {
|
||||
out.Packets = t.tracer.Packets()
|
||||
}
|
||||
t.mu.Unlock()
|
||||
if out.Packets == nil {
|
||||
out.Packets = []api.TracePacket{}
|
||||
}
|
||||
st := s.eng.State()
|
||||
for i := range out.Packets {
|
||||
p := &out.Packets[i]
|
||||
for j := range p.Steps {
|
||||
p.Steps[j].Label = markLabel(st, p.Steps[j])
|
||||
}
|
||||
p.DecidedBy.Label = markLabel(st, p.DecidedBy)
|
||||
if p.WouldDrop != nil {
|
||||
// A copy: the tracer still holds the step.
|
||||
wd := *p.WouldDrop
|
||||
wd.Label = markLabel(st, wd)
|
||||
p.WouldDrop = &wd
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// markLabel names the Housecarl object behind a step: "rule input #12
|
||||
// (ssh)", "built-in: established", "port forward web", "input policy".
|
||||
func markLabel(st *model.State, step api.TraceStep) string {
|
||||
if step.Kind == "policy" {
|
||||
if step.Table == "inet housecarl" {
|
||||
return step.Chain + " policy"
|
||||
}
|
||||
return ""
|
||||
}
|
||||
m := step.Mark
|
||||
kind, id := "", int64(0)
|
||||
switch {
|
||||
case m == "":
|
||||
return ""
|
||||
case strings.HasPrefix(m, "b-"):
|
||||
return "built-in: " + strings.TrimPrefix(m, "b-")
|
||||
case strings.HasPrefix(m, "r"):
|
||||
kind = model.KindRule
|
||||
case strings.HasPrefix(m, "f"):
|
||||
kind = model.KindForward
|
||||
case strings.HasPrefix(m, "s"):
|
||||
kind = model.KindSNAT
|
||||
case strings.HasPrefix(m, "d"):
|
||||
kind = model.KindRedirect
|
||||
default:
|
||||
return ""
|
||||
}
|
||||
num := strings.TrimLeft(m[1:], "0123456789")
|
||||
id, _ = strconv.ParseInt(strings.TrimSuffix(m[1:], num), 10, 64)
|
||||
if o := st.ByID(kind, id); o != nil {
|
||||
label := o.Label() // "source NAT #3 (…)", "redirect #4", a forward's name…
|
||||
switch {
|
||||
case kind == model.KindRule:
|
||||
label = "rule " + label
|
||||
case kind == model.KindForward && !strings.HasPrefix(label, "port forward"):
|
||||
label = "port forward " + label
|
||||
}
|
||||
if num != "" {
|
||||
label += " (" + strings.TrimPrefix(num, "-") + ")"
|
||||
}
|
||||
return label
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func (s *Server) getTrace(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResLog, false); err != nil {
|
||||
return err
|
||||
}
|
||||
writeJSON(w, http.StatusOK, s.traceStatus())
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Server) stopTrace(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResConfig, true); err != nil {
|
||||
return err
|
||||
}
|
||||
t := &s.tracing
|
||||
t.mu.Lock()
|
||||
if t.cancel != nil && t.status.Running {
|
||||
t.cancel()
|
||||
}
|
||||
t.mu.Unlock()
|
||||
// Let the trace end, so the answer shows it stopped.
|
||||
for i := 0; i < 50; i++ {
|
||||
t.mu.Lock()
|
||||
running := t.status.Running
|
||||
t.mu.Unlock()
|
||||
if !running {
|
||||
break
|
||||
}
|
||||
time.Sleep(20 * time.Millisecond)
|
||||
}
|
||||
writeJSON(w, http.StatusOK, s.traceStatus())
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,295 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"housecarl/internal/api"
|
||||
"housecarl/internal/engine"
|
||||
"housecarl/internal/model"
|
||||
)
|
||||
|
||||
// Every configuration change is an operation (model.Op): applied at once,
|
||||
// or staged in a changeset and replayed when the changeset is applied.
|
||||
// applyOp is the one place an operation becomes an edit.
|
||||
|
||||
// applyOp makes an operation's change to st. It returns the ID of the
|
||||
// object it made or changed (0 for deletes and settings).
|
||||
func applyOp(st *model.State, op *model.Op) (int64, error) {
|
||||
switch op.Op {
|
||||
case model.OpCreate:
|
||||
return createOp(st, op)
|
||||
case model.OpUpdate:
|
||||
return updateOp(st, op)
|
||||
case model.OpDelete:
|
||||
return 0, deleteOp(st, op)
|
||||
case model.OpMove:
|
||||
return moveOp(st, op)
|
||||
case model.OpSettings:
|
||||
return 0, settingsOp(st, op)
|
||||
case model.OpRevert:
|
||||
var h model.HistoryEntry
|
||||
if err := json.Unmarshal(op.Data, &h); err != nil {
|
||||
return 0, badRequest("revert: %v", err)
|
||||
}
|
||||
return 0, engine.Undo(st, &h)
|
||||
}
|
||||
return 0, badRequest("unknown operation %q", op.Op)
|
||||
}
|
||||
|
||||
func createOp(st *model.State, op *model.Op) (int64, error) {
|
||||
o, err := decodeObject(op.Kind, op.Data)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if svc, ok := o.(*model.Service); ok {
|
||||
svc.Builtin = false
|
||||
}
|
||||
if rule, ok := o.(*model.Rule); ok && rule.Action == "" {
|
||||
rule.Action = model.ActionAccept
|
||||
}
|
||||
if f, ok := o.(*model.PortForward); ok && len(f.Protocols) == 0 {
|
||||
f.Protocols = []string{"tcp"}
|
||||
}
|
||||
if st.ByID(op.Kind, op.ID) != nil {
|
||||
return 0, conflict("%s %d exists already", op.Kind, op.ID)
|
||||
}
|
||||
setID(o, op.ID)
|
||||
if pos := position(o); pos != nil && *pos == 0 {
|
||||
*pos = lastPosition(st, op.Kind, chainOf(o))
|
||||
}
|
||||
data, _ := json.Marshal(o)
|
||||
if err := st.Put(op.Kind, op.ID, data); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
// Placed in the same change, so it is never in force anywhere else.
|
||||
if len(op.Place) > 0 {
|
||||
if _, err := moveOp(st, &model.Op{Op: model.OpMove, Kind: op.Kind, Ref: strconv.FormatInt(op.ID, 10), Data: op.Place}); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
}
|
||||
return op.ID, nil
|
||||
}
|
||||
|
||||
func updateOp(st *model.State, op *model.Op) (int64, error) {
|
||||
var patch map[string]json.RawMessage
|
||||
if err := json.Unmarshal(op.Data, &patch); err != nil {
|
||||
return 0, badRequest("%s: %v", op.Kind, err)
|
||||
}
|
||||
delete(patch, "id")
|
||||
o, err := findObject(st, op.Kind, op.Ref)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
id := o.GetID()
|
||||
// The patch's fields over the object's.
|
||||
var cur map[string]json.RawMessage
|
||||
b, _ := json.Marshal(o)
|
||||
json.Unmarshal(b, &cur)
|
||||
for k, v := range patch {
|
||||
if string(v) == "null" {
|
||||
delete(cur, k)
|
||||
} else {
|
||||
cur[k] = v
|
||||
}
|
||||
}
|
||||
merged, _ := json.Marshal(cur)
|
||||
next, err := decodeObject(op.Kind, merged)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
setID(next, id)
|
||||
if old, now := o.Label(), next.Label(); old != now && op.Kind == model.KindBanList {
|
||||
return 0, conflict("ban list %s can't be renamed: the API tokens and programs (Guard, fail2ban) that ban into it go by its name; make a new list instead", old)
|
||||
}
|
||||
if old, now := o.Label(), next.Label(); old != now && isNamed(op.Kind) {
|
||||
if set, ok := o.(*model.AddressSet); ok && set.Dynamic {
|
||||
return 0, conflict("set %s is dynamic: its entries, and the API tokens and programs that fill it, go by its name; make a new set instead", old)
|
||||
}
|
||||
renameRefs(st, op.Kind, old, now)
|
||||
}
|
||||
data, _ := json.Marshal(next)
|
||||
return id, st.Put(op.Kind, id, data)
|
||||
}
|
||||
|
||||
// isNamed reports whether other objects refer to a kind by name.
|
||||
func isNamed(kind string) bool {
|
||||
return kind == model.KindZone || kind == model.KindSet || kind == model.KindService
|
||||
}
|
||||
|
||||
func deleteOp(st *model.State, op *model.Op) error {
|
||||
o, err := findObject(st, op.Kind, op.Ref)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if used := usedBy(st, op.Kind, o); used != "" {
|
||||
return conflict("%s %s is used by %s; change that first", op.Kind, o.Label(), used)
|
||||
}
|
||||
return st.Put(op.Kind, o.GetID(), nil)
|
||||
}
|
||||
|
||||
// moveOp reorders a rule or NAT rule: before or after another, or to the
|
||||
// start or end; a rule can change chain at the same time.
|
||||
func moveOp(st *model.State, op *model.Op) (int64, error) {
|
||||
var in api.MoveInput
|
||||
if err := json.Unmarshal(op.Data, &in); err != nil {
|
||||
return 0, badRequest("move: %v", err)
|
||||
}
|
||||
o, err := findObject(st, op.Kind, op.Ref)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
id := o.GetID()
|
||||
pos := position(o)
|
||||
if pos == nil {
|
||||
return 0, badRequest("only rules, port forwards, source NAT rules and redirects have an order")
|
||||
}
|
||||
if rule, ok := o.(*model.Rule); ok && in.Chain != "" {
|
||||
rule.Chain = in.Chain
|
||||
}
|
||||
chain := chainOf(o)
|
||||
// The others in order.
|
||||
var others []model.Object
|
||||
for _, x := range st.Objects(op.Kind) {
|
||||
if x.GetID() != id && chainOf(x) == chain {
|
||||
others = append(others, x)
|
||||
}
|
||||
}
|
||||
sort.SliceStable(others, func(i, j int) bool { return *position(others[i]) < *position(others[j]) })
|
||||
place := func(i int) { // before others[i] (i == len: at the end)
|
||||
switch {
|
||||
case len(others) == 0:
|
||||
*pos = 1
|
||||
case i == 0:
|
||||
*pos = *position(others[0]) - 1
|
||||
case i == len(others):
|
||||
*pos = *position(others[len(others)-1]) + 1
|
||||
default:
|
||||
*pos = (*position(others[i-1]) + *position(others[i])) / 2
|
||||
}
|
||||
}
|
||||
index := func(ref int64) (int, error) {
|
||||
for i, x := range others {
|
||||
if x.GetID() == ref {
|
||||
return i, nil
|
||||
}
|
||||
}
|
||||
return 0, badRequest("%d isn't another %s of the same chain", ref, op.Kind)
|
||||
}
|
||||
switch {
|
||||
case in.Before != 0:
|
||||
i, err := index(in.Before)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
place(i)
|
||||
case in.After != 0:
|
||||
i, err := index(in.After)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
place(i + 1)
|
||||
case in.To == "start":
|
||||
place(0)
|
||||
case in.To == "end" || in.Chain != "":
|
||||
place(len(others))
|
||||
default:
|
||||
return 0, badRequest("say where: before, after, or to start or end")
|
||||
}
|
||||
data, _ := json.Marshal(o)
|
||||
return id, st.Put(op.Kind, id, data)
|
||||
}
|
||||
|
||||
func settingsOp(st *model.State, op *model.Op) error {
|
||||
var patch map[string]json.RawMessage
|
||||
if err := json.Unmarshal(op.Data, &patch); err != nil {
|
||||
return badRequest("settings: %v", err)
|
||||
}
|
||||
var cur map[string]json.RawMessage
|
||||
b, _ := json.Marshal(st.Settings)
|
||||
json.Unmarshal(b, &cur)
|
||||
for k, v := range patch {
|
||||
// Built-ins merge one by one.
|
||||
if k == "builtins" {
|
||||
var cb, pb map[string]json.RawMessage
|
||||
json.Unmarshal(cur[k], &cb)
|
||||
if err := json.Unmarshal(v, &pb); err != nil {
|
||||
return badRequest("builtins: %v", err)
|
||||
}
|
||||
for bk, bv := range pb {
|
||||
cb[bk] = bv
|
||||
}
|
||||
v, _ = json.Marshal(cb)
|
||||
}
|
||||
cur[k] = v
|
||||
}
|
||||
merged, _ := json.Marshal(cur)
|
||||
var next model.Settings
|
||||
dec := json.NewDecoder(strings.NewReader(string(merged)))
|
||||
dec.DisallowUnknownFields()
|
||||
if err := dec.Decode(&next); err != nil {
|
||||
return badRequest("settings: %v", err)
|
||||
}
|
||||
st.Settings = next
|
||||
return nil
|
||||
}
|
||||
|
||||
// replay applies a changeset's operations to st, in order. An operation
|
||||
// that fails is named in the error (opError).
|
||||
func replay(st *model.State, ops []model.Op) error {
|
||||
for i := range ops {
|
||||
if _, err := applyOp(st, &ops[i]); err != nil {
|
||||
return &opError{seq: ops[i].Seq, summary: ops[i].Summary, err: err}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// opError: an operation of a changeset that can't be applied (the
|
||||
// objects it names changed since it was staged, say).
|
||||
type opError struct {
|
||||
seq int
|
||||
summary string
|
||||
err error
|
||||
}
|
||||
|
||||
func (e *opError) Error() string {
|
||||
return fmt.Sprintf("operation %d (%s) can't be applied: %v", e.seq, e.summary, e.err)
|
||||
}
|
||||
|
||||
func (e *opError) Unwrap() error { return e.err }
|
||||
|
||||
// info is the error for the API, with the inner error's status.
|
||||
func (e *opError) info() (api.ErrorInfo, int) {
|
||||
info := api.ErrorInfo{Code: "conflict", Message: e.Error(), Op: e.seq}
|
||||
status := http.StatusConflict
|
||||
var he *httpError
|
||||
if errors.As(e.err, &he) {
|
||||
info.Code, status = he.code, he.status
|
||||
}
|
||||
if ee, ok := engine.AsError(e.err); ok {
|
||||
info.Code, info.Problems, status = ee.Kind, ee.Problems, engineStatus(ee.Kind)
|
||||
}
|
||||
return info, status
|
||||
}
|
||||
|
||||
// objectOf picks what an operation's answer returns from a state.
|
||||
func objectOf(op *model.Op, id int64) func(st *model.State) any {
|
||||
switch op.Op {
|
||||
case model.OpDelete:
|
||||
return nil
|
||||
case model.OpSettings:
|
||||
return func(st *model.State) any { return st.Settings }
|
||||
}
|
||||
return func(st *model.State) any {
|
||||
if o := st.ByID(op.Kind, id); o != nil {
|
||||
return o
|
||||
}
|
||||
return nil
|
||||
}
|
||||
}
|
||||
@@ -2,8 +2,10 @@ package server
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/subtle"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
@@ -31,6 +33,21 @@ type probation struct {
|
||||
timer *time.Timer
|
||||
// restores are nft scripts that bring back tables a takeover deleted.
|
||||
restores []string
|
||||
// check is the latest change's confirmation token (reachCheck): only
|
||||
// it can confirm, since the ruleset in force is the one it tests.
|
||||
check *reachCheck
|
||||
}
|
||||
|
||||
// reachCheck is what a client must show to confirm its change by itself:
|
||||
// the token, from the same user (and API token), the same address, and a
|
||||
// new connection (another remote address and port than the change's).
|
||||
type reachCheck struct {
|
||||
token string
|
||||
userID int64
|
||||
username string
|
||||
tokenID int64
|
||||
ip string
|
||||
remote string
|
||||
}
|
||||
|
||||
// system is the principal of rollbacks.
|
||||
@@ -64,7 +81,8 @@ func (s *Server) startProbation(ctx context.Context, p *auth.Principal, historyI
|
||||
pr := &s.probation
|
||||
pr.mu.Lock()
|
||||
defer pr.mu.Unlock()
|
||||
if p.Method == "socket" && len(pr.pending) == 0 && len(pr.restores) == 0 {
|
||||
// In shadow mode a change can't cut anyone off: it blocks nothing.
|
||||
if s.cfg.Shadow || p.Method == "socket" && len(pr.pending) == 0 && len(pr.restores) == 0 {
|
||||
s.saveBoot()
|
||||
return
|
||||
}
|
||||
@@ -88,6 +106,60 @@ func (s *Server) startProbation(ctx context.Context, p *auth.Principal, historyI
|
||||
s.saveProbation(ctx)
|
||||
}
|
||||
|
||||
// issueCheck gives the client that made a network change on probation a
|
||||
// token to confirm it by itself, unless the settings want a person to
|
||||
// confirm. The answer then closes its connection (HTTP/2: a GOAWAY), so
|
||||
// the confirmation has to open a new one through the new ruleset; an
|
||||
// established connection would get through whatever the change did.
|
||||
func (s *Server) issueCheck(w http.ResponseWriter, r *http.Request, p *auth.Principal, res *engine.Result) string {
|
||||
if p == nil || p.Method == "socket" || (res != nil && !res.Applied) || s.eng.State().Settings.ManualConfirm {
|
||||
return ""
|
||||
}
|
||||
pr := &s.probation
|
||||
pr.mu.Lock()
|
||||
defer pr.mu.Unlock()
|
||||
if len(pr.pending) == 0 && len(pr.restores) == 0 {
|
||||
return ""
|
||||
}
|
||||
c := &reachCheck{token: auth.RandomString(24), username: p.Name(), ip: s.clientIP(r), remote: r.RemoteAddr}
|
||||
if p.User != nil {
|
||||
c.userID = p.User.ID
|
||||
}
|
||||
if p.Token != nil {
|
||||
c.tokenID = p.Token.ID
|
||||
}
|
||||
pr.check = c
|
||||
w.Header().Set("Connection", "close")
|
||||
return c.token
|
||||
}
|
||||
|
||||
// confirmToken confirms with a change's token: the proof that the client
|
||||
// that made it still reaches the API, from where it made it.
|
||||
func (s *Server) confirmToken(r *http.Request, p *auth.Principal, token string) (int, error) {
|
||||
pr := &s.probation
|
||||
pr.mu.Lock()
|
||||
c := pr.check
|
||||
pending := len(pr.pending) + len(pr.restores)
|
||||
pr.mu.Unlock()
|
||||
var tokenID int64
|
||||
if p.Token != nil {
|
||||
tokenID = p.Token.ID
|
||||
}
|
||||
switch {
|
||||
case pending == 0:
|
||||
return 0, nil
|
||||
case c == nil || subtle.ConstantTimeCompare([]byte(c.token), []byte(token)) != 1:
|
||||
return 0, conflict("this confirmation is for an earlier change: a newer one is on probation, and only its own client (or a person) can confirm it")
|
||||
case p.User == nil || p.User.ID != c.userID || p.Name() != c.username || tokenID != c.tokenID:
|
||||
return 0, forbidden("only the client that made the change can confirm it this way")
|
||||
case s.clientIP(r) != c.ip:
|
||||
return 0, forbidden("the confirmation comes from %s, the change from %s: it must come from the same place", s.clientIP(r), c.ip)
|
||||
case r.RemoteAddr == c.remote:
|
||||
return 0, conflict("the confirmation came over the change's own connection, which proves nothing; it must open a new one")
|
||||
}
|
||||
return s.confirmWith(r.Context(), p, fmt.Sprintf("confirmed automatically: %s reached the API again over a new connection", c.ip))
|
||||
}
|
||||
|
||||
func (s *Server) saveBoot() {
|
||||
if err := s.fw.SaveBoot(s.eng.State()); err != nil {
|
||||
s.log.Error("saving the ruleset for the next boot failed", "err", err)
|
||||
@@ -110,8 +182,12 @@ func (s *Server) probationStatus() api.Probation {
|
||||
return out
|
||||
}
|
||||
|
||||
// confirm makes the pending changes permanent.
|
||||
// confirm makes the pending changes permanent (a person's confirmation).
|
||||
func (s *Server) confirm(ctx context.Context, p *auth.Principal) (int, error) {
|
||||
return s.confirmWith(ctx, p, "")
|
||||
}
|
||||
|
||||
func (s *Server) confirmWith(ctx context.Context, p *auth.Principal, how string) (int, error) {
|
||||
pr := &s.probation
|
||||
pr.mu.Lock()
|
||||
defer pr.mu.Unlock()
|
||||
@@ -122,10 +198,14 @@ func (s *Server) confirm(ctx context.Context, p *auth.Principal) (int, error) {
|
||||
if pr.timer != nil {
|
||||
pr.timer.Stop()
|
||||
}
|
||||
pr.pending, pr.restores, pr.deadline, pr.timer = nil, nil, time.Time{}, nil
|
||||
pr.pending, pr.restores, pr.deadline, pr.timer, pr.check = nil, nil, time.Time{}, nil, nil
|
||||
s.saveProbation(ctx)
|
||||
s.saveBoot()
|
||||
s.eng.Audit(ctx, p, "config.confirm", "", "success", fmt.Sprintf("%d changes confirmed", n))
|
||||
detail := fmt.Sprintf("%d changes confirmed", n)
|
||||
if how != "" {
|
||||
detail += "; " + how
|
||||
}
|
||||
s.eng.Audit(ctx, p, "config.confirm", "", "success", detail)
|
||||
return n, nil
|
||||
}
|
||||
|
||||
@@ -136,7 +216,7 @@ func (s *Server) rollback(why string) {
|
||||
pr.mu.Lock()
|
||||
defer pr.mu.Unlock()
|
||||
pending, restores := pr.pending, pr.restores
|
||||
pr.pending, pr.restores, pr.deadline, pr.timer = nil, nil, time.Time{}, nil
|
||||
pr.pending, pr.restores, pr.deadline, pr.timer, pr.check = nil, nil, time.Time{}, nil, nil
|
||||
for i := len(restores) - 1; i >= 0; i-- {
|
||||
if err := s.fw.ApplyScript(ctx, restores[i]); err != nil {
|
||||
s.log.Error("bringing back a table a takeover deleted failed", "err", err)
|
||||
|
||||
@@ -0,0 +1,356 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"math"
|
||||
"net/http"
|
||||
"net/netip"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"housecarl/internal/api"
|
||||
"housecarl/internal/auth"
|
||||
"housecarl/internal/model"
|
||||
"housecarl/internal/nft"
|
||||
"housecarl/internal/render"
|
||||
)
|
||||
|
||||
// Quotas: nftables counts them, and never resets them. housecarld resets
|
||||
// each at the start of its period (daily, weekly, monthly), or when asked,
|
||||
// and saves what they used every few minutes and when it stops, so a
|
||||
// reboot (whose boot ruleset starts them at zero) doesn't lose it.
|
||||
|
||||
const (
|
||||
quotaResetsKey = "quota_resets" // rule mark → when its quota was last reset
|
||||
QuotaUsageKey = "quota_usage" // nft.QuotaUsage, saved
|
||||
)
|
||||
|
||||
// periodStart is when the current period of a reset began (zero for a
|
||||
// quota reset only by hand), in local time.
|
||||
func periodStart(now time.Time, reset string, day int) time.Time {
|
||||
y, m, d := now.Date()
|
||||
midnight := time.Date(y, m, d, 0, 0, 0, 0, now.Location())
|
||||
switch reset {
|
||||
case "daily":
|
||||
return midnight
|
||||
case "weekly":
|
||||
back := (int(now.Weekday()) + 6) % 7 // Monday
|
||||
return midnight.AddDate(0, 0, -back)
|
||||
case "monthly":
|
||||
if day < 1 {
|
||||
day = 1
|
||||
}
|
||||
start := monthDay(y, m, day, now.Location())
|
||||
if start.After(now) {
|
||||
py, pm, _ := time.Date(y, m, 1, 0, 0, 0, 0, now.Location()).AddDate(0, -1, 0).Date()
|
||||
start = monthDay(py, pm, day, now.Location())
|
||||
}
|
||||
return start
|
||||
}
|
||||
return time.Time{}
|
||||
}
|
||||
|
||||
// monthDay is midnight on a day of a month, or on its last day when the
|
||||
// month is shorter.
|
||||
func monthDay(y int, m time.Month, day int, loc *time.Location) time.Time {
|
||||
last := time.Date(y, m+1, 0, 0, 0, 0, 0, loc).Day()
|
||||
if day > last {
|
||||
day = last
|
||||
}
|
||||
return time.Date(y, m, day, 0, 0, 0, 0, loc)
|
||||
}
|
||||
|
||||
// nextReset is when the period after the current one starts.
|
||||
func nextReset(now time.Time, reset string, day int) time.Time {
|
||||
start := periodStart(now, reset, day)
|
||||
switch reset {
|
||||
case "daily":
|
||||
return start.AddDate(0, 0, 1)
|
||||
case "weekly":
|
||||
return start.AddDate(0, 0, 7)
|
||||
case "monthly":
|
||||
y, m, _ := start.Date()
|
||||
ny, nm, _ := time.Date(y, m, 1, 0, 0, 0, 0, now.Location()).AddDate(0, 1, 0).Date()
|
||||
if day < 1 {
|
||||
day = 1
|
||||
}
|
||||
return monthDay(ny, nm, day, now.Location())
|
||||
}
|
||||
return time.Time{}
|
||||
}
|
||||
|
||||
func (s *Server) quotaResets(ctx context.Context) map[string]time.Time {
|
||||
out := map[string]time.Time{}
|
||||
if v, err := s.store.GetMeta(ctx, quotaResetsKey); err == nil && v != "" {
|
||||
json.Unmarshal([]byte(v), &out)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func (s *Server) saveQuotaResets(ctx context.Context, m map[string]time.Time) {
|
||||
b, _ := json.Marshal(m)
|
||||
if err := s.store.SetMeta(ctx, quotaResetsKey, string(b)); err != nil {
|
||||
s.log.Error("saving the quotas' reset times failed", "err", err)
|
||||
}
|
||||
}
|
||||
|
||||
// RunQuotas resets quotas when their period ends and saves their usage,
|
||||
// until ctx ends (then it saves once more).
|
||||
func (s *Server) RunQuotas(ctx context.Context) {
|
||||
s.checkQuotas(ctx)
|
||||
tick := time.NewTicker(time.Minute)
|
||||
defer tick.Stop()
|
||||
n := 0
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
s.SaveQuotaUsage(context.Background())
|
||||
return
|
||||
case <-tick.C:
|
||||
s.checkQuotas(ctx)
|
||||
if n++; n%5 == 0 {
|
||||
s.SaveQuotaUsage(ctx)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// checkQuotas resets the quotas whose period has ended since their last
|
||||
// reset. A quota seen for the first time starts its period now.
|
||||
func (s *Server) checkQuotas(ctx context.Context) {
|
||||
st := s.eng.State()
|
||||
resets := s.quotaResets(ctx)
|
||||
now := s.nowFn()
|
||||
changed := false
|
||||
seen := map[string]bool{}
|
||||
for _, r := range st.Rules {
|
||||
if r.Quota == nil || r.Disabled || r.Raw != "" {
|
||||
continue
|
||||
}
|
||||
key := fmt.Sprintf("r%d", r.ID)
|
||||
seen[key] = true
|
||||
last, ok := resets[key]
|
||||
if !ok {
|
||||
resets[key], changed = now, true
|
||||
continue
|
||||
}
|
||||
start := periodStart(now, r.Quota.Reset, r.Quota.Day)
|
||||
if start.IsZero() || !last.Before(start) {
|
||||
continue
|
||||
}
|
||||
if err := s.fw.ResetQuota(ctx, r.ID, r.Quota.PerAddress() != ""); err != nil {
|
||||
s.log.Error("resetting a quota failed", "rule", r.Label(), "err", err)
|
||||
continue
|
||||
}
|
||||
s.log.Info("quota reset for its new period", "rule", r.Label(), "period", r.Quota.Reset)
|
||||
resets[key], changed = now, true
|
||||
s.forgetQuotaUsage(ctx, r.ID)
|
||||
}
|
||||
for key := range resets {
|
||||
if !seen[key] {
|
||||
delete(resets, key)
|
||||
changed = true
|
||||
}
|
||||
}
|
||||
if changed {
|
||||
s.saveQuotaResets(ctx, resets)
|
||||
}
|
||||
}
|
||||
|
||||
// SaveQuotaUsage saves what the quotas used, for after a reboot.
|
||||
func (s *Server) SaveQuotaUsage(ctx context.Context) {
|
||||
t, err := s.fw.Table(ctx)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
b, _ := json.Marshal(t.QuotaUsage)
|
||||
if err := s.store.SetMeta(ctx, QuotaUsageKey, string(b)); err != nil {
|
||||
s.log.Error("saving the quotas' usage failed", "err", err)
|
||||
}
|
||||
}
|
||||
|
||||
// forgetQuotaUsage drops a reset quota from the saved usage, so a restart
|
||||
// doesn't bring back what it used before.
|
||||
func (s *Server) forgetQuotaUsage(ctx context.Context, ruleID int64) {
|
||||
v, err := s.store.GetMeta(ctx, QuotaUsageKey)
|
||||
if err != nil || v == "" {
|
||||
return
|
||||
}
|
||||
var u nft.QuotaUsage
|
||||
if json.Unmarshal([]byte(v), &u) != nil {
|
||||
return
|
||||
}
|
||||
name := render.QuotaName(ruleID)
|
||||
delete(u.Quotas, name)
|
||||
delete(u.QuotaSets, name+"_v4")
|
||||
delete(u.QuotaSets, name+"_v6")
|
||||
b, _ := json.Marshal(u)
|
||||
s.store.SetMeta(ctx, QuotaUsageKey, string(b))
|
||||
}
|
||||
|
||||
// listQuotas answers what each rule's quota used.
|
||||
func (s *Server) listQuotas(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResLog, false); err != nil {
|
||||
return err
|
||||
}
|
||||
st := s.eng.State()
|
||||
var usage nft.QuotaUsage
|
||||
if t, err := s.fw.Table(r.Context()); err == nil {
|
||||
usage = t.QuotaUsage
|
||||
}
|
||||
resets := s.quotaResets(r.Context())
|
||||
now := s.nowFn()
|
||||
out := []api.QuotaInfo{}
|
||||
for _, rule := range st.Rules {
|
||||
if rule.Quota == nil {
|
||||
continue
|
||||
}
|
||||
q := api.QuotaInfo{RuleID: rule.ID, Rule: rule.Label(), Quota: *rule.Quota}
|
||||
name := render.QuotaName(rule.ID)
|
||||
if rule.Quota.PerAddress() == "" {
|
||||
q.Used = usage.Quotas[name]
|
||||
} else {
|
||||
v4, v6 := rule.Groups()
|
||||
for _, fam := range []string{"_v4", "_v6"} {
|
||||
for _, e := range usage.QuotaSets[name+fam] {
|
||||
addr := e.Addr
|
||||
if fam == "_v4" && v4 < 32 {
|
||||
addr = fmt.Sprintf("%s/%d", addr, v4)
|
||||
} else if fam == "_v6" && v6 < 128 {
|
||||
addr = fmt.Sprintf("%s/%d", addr, v6)
|
||||
}
|
||||
q.Addresses = append(q.Addresses, api.QuotaAddress{Address: addr, Used: e.Used})
|
||||
q.Used += e.Used
|
||||
}
|
||||
}
|
||||
sort.Slice(q.Addresses, func(i, j int) bool { return q.Addresses[i].Used > q.Addresses[j].Used })
|
||||
}
|
||||
if last, ok := resets[fmt.Sprintf("r%d", rule.ID)]; ok {
|
||||
q.LastReset = &last
|
||||
}
|
||||
if next := nextReset(now, rule.Quota.Reset, rule.Quota.Day); !next.IsZero() {
|
||||
q.NextReset = &next
|
||||
}
|
||||
out = append(out, q)
|
||||
}
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
return nil
|
||||
}
|
||||
|
||||
// resetQuota starts a rule's quota again now: all of it, or one
|
||||
// address's; or gives some of what it used back, until its next reset.
|
||||
func (s *Server) resetQuota(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResConfig, true); err != nil {
|
||||
return err
|
||||
}
|
||||
o, err := findObject(s.eng.State(), model.KindRule, r.PathValue("ref"))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
rule := o.(*model.Rule)
|
||||
if rule.Quota == nil {
|
||||
return badRequest("%s has no quota", rule.Label())
|
||||
}
|
||||
var in api.QuotaReset
|
||||
if err := decodeOptional(r, &in); err != nil {
|
||||
return err
|
||||
}
|
||||
if in.Give < 0 {
|
||||
return badRequest("give a number of bytes to give back, 1 or more")
|
||||
}
|
||||
if in.Address == "" && in.Give == 0 {
|
||||
if err := s.fw.ResetQuota(r.Context(), rule.ID, rule.Quota.PerAddress() != ""); err != nil {
|
||||
return err
|
||||
}
|
||||
resets := s.quotaResets(r.Context())
|
||||
resets[fmt.Sprintf("r%d", rule.ID)] = s.nowFn()
|
||||
s.saveQuotaResets(r.Context(), resets)
|
||||
s.forgetQuotaUsage(r.Context(), rule.ID)
|
||||
s.eng.Audit(r.Context(), p, "quota.reset", rule.Label(), "success", "")
|
||||
writeJSON(w, http.StatusOK, map[string]any{"reset": rule.Label()})
|
||||
return nil
|
||||
}
|
||||
name := render.QuotaName(rule.ID)
|
||||
adj := nft.QuotaAdjust{Name: name, Give: in.Give}
|
||||
if in.Address != "" {
|
||||
if rule.Quota.PerAddress() == "" {
|
||||
return badRequest("%s has one quota for everything it matches, not one per address: leave out the address", rule.Label())
|
||||
}
|
||||
key, err := quotaKey(rule, in.Address)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !s.quotaHas(r.Context(), name, key) {
|
||||
return notFound("%s hasn't used the quota of %s since its last reset", in.Address, rule.Label())
|
||||
}
|
||||
adj.Addr = key
|
||||
}
|
||||
if adj.Give == 0 {
|
||||
adj.Give = math.MaxInt64 // all of it: the address starts again
|
||||
}
|
||||
// The next apply writes the quota's new usage into the ruleset.
|
||||
s.fw.AdjustQuota(adj)
|
||||
if err := s.eng.ApplyCurrent(r.Context()); err != nil {
|
||||
return fmt.Errorf("putting the ruleset in force with the quota's new usage: %w", err)
|
||||
}
|
||||
s.SaveQuotaUsage(r.Context())
|
||||
detail, out := "", map[string]any{}
|
||||
if in.Give > 0 {
|
||||
detail = fmt.Sprintf("gave back %d bytes", in.Give)
|
||||
out["given"] = in.Give
|
||||
}
|
||||
if in.Address != "" {
|
||||
detail = strings.TrimSpace(detail + " " + in.Address)
|
||||
out["address"] = in.Address
|
||||
}
|
||||
action := "quota.reset"
|
||||
if in.Give > 0 {
|
||||
action = "quota.give"
|
||||
}
|
||||
s.eng.Audit(r.Context(), p, action, rule.Label(), "success", detail)
|
||||
out["reset"] = rule.Label()
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
return nil
|
||||
}
|
||||
|
||||
// quotaKey is the entry of a per-address quota an address counts under:
|
||||
// the address, or its group's first address (10.0.10.0 for a /24; an
|
||||
// IPv6 address's /64).
|
||||
func quotaKey(rule *model.Rule, s string) (string, error) {
|
||||
var a netip.Addr
|
||||
if p, err := netip.ParsePrefix(s); err == nil {
|
||||
a = p.Addr()
|
||||
} else if a, err = netip.ParseAddr(s); err != nil {
|
||||
return "", badRequest("%q isn't an address", s)
|
||||
}
|
||||
a = a.Unmap()
|
||||
v4, v6 := rule.Groups()
|
||||
bits := v4
|
||||
if a.Is6() {
|
||||
bits = v6
|
||||
}
|
||||
p, err := a.Prefix(bits)
|
||||
if err != nil {
|
||||
return "", badRequest("%q: %v", s, err)
|
||||
}
|
||||
return p.Addr().String(), nil
|
||||
}
|
||||
|
||||
// quotaHas reports whether a per-address quota has an entry for key.
|
||||
func (s *Server) quotaHas(ctx context.Context, name, key string) bool {
|
||||
t, err := s.fw.Table(ctx)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
for _, set := range []string{name + "_v4", name + "_v6"} {
|
||||
for _, e := range t.QuotaSets[set] {
|
||||
if e.Addr == key {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -0,0 +1,84 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"housecarl/internal/api"
|
||||
)
|
||||
|
||||
// TestRename: renaming a zone, set or service changes what refers to it,
|
||||
// in the same change, which undoes as one.
|
||||
func TestRename(t *testing.T) {
|
||||
e := newEnv(t)
|
||||
e.srv.probationFor = 0
|
||||
post := func(path string, body map[string]any) {
|
||||
t.Helper()
|
||||
if code := e.call(e.token, "POST", path, body, nil); code != 201 {
|
||||
t.Fatalf("POST %s: %d", path, code)
|
||||
}
|
||||
}
|
||||
post("/zones", map[string]any{"name": "wan", "interfaces": []string{"wan"}, "external": true})
|
||||
post("/zones", map[string]any{"name": "dhcp", "interfaces": []string{"eth1"}})
|
||||
post("/sets", map[string]any{"name": "trusted_ips", "entries": []string{"10.0.10.0/24"}})
|
||||
post("/sets", map[string]any{"name": "guests", "dynamic": true})
|
||||
post("/services", map[string]any{"name": "jarl", "entries": []string{"tcp/8053"}})
|
||||
post("/rules", map[string]any{"chain": "forward", "in_zones": []string{"dhcp"}, "out_zones": []string{"wan"}, "sources": []string{"trusted_ips"}})
|
||||
post("/rules", map[string]any{"chain": "input", "in_zones": []string{"dhcp"}, "services": []string{"jarl"}})
|
||||
post("/snats", map[string]any{"out_zones": []string{"wan"}, "sources": []string{"trusted_ips"}})
|
||||
post("/forwards", map[string]any{"name": "jf", "ports": "8096", "to": "10.0.10.57", "public_addresses": []string{"217.155.8.194"}, "sources": []string{"trusted_ips"}})
|
||||
|
||||
var res api.MutationResponse
|
||||
if code := e.call(e.token, "PATCH", "/zones/dhcp", map[string]any{"name": "lan"}, &res); code != 200 {
|
||||
t.Fatalf("renaming the zone: %d %+v", code, res)
|
||||
}
|
||||
zoneRename := res.HistoryID
|
||||
e.call(e.token, "PATCH", "/sets/trusted_ips", map[string]any{"name": "trusted"}, &res)
|
||||
setRename := res.HistoryID
|
||||
e.call(e.token, "PATCH", "/services/jarl", map[string]any{"name": "jarl-api"}, &res)
|
||||
serviceRename := res.HistoryID
|
||||
st := e.eng.State()
|
||||
if st.Zone("dhcp") != nil || st.Zone("lan") == nil {
|
||||
t.Fatalf("zones %+v", st.Zones)
|
||||
}
|
||||
for _, r := range st.Rules {
|
||||
if strings.Join(r.InZones, ",") != "lan" {
|
||||
t.Errorf("rule %d in zones %v", r.ID, r.InZones)
|
||||
}
|
||||
if r.Chain == "forward" && strings.Join(r.Sources, ",") != "trusted" {
|
||||
t.Errorf("rule %d sources %v", r.ID, r.Sources)
|
||||
}
|
||||
if r.Chain == "input" && strings.Join(r.Services, ",") != "jarl-api" {
|
||||
t.Errorf("rule %d services %v", r.ID, r.Services)
|
||||
}
|
||||
}
|
||||
if s := st.SNATs[0].Sources; strings.Join(s, ",") != "trusted" {
|
||||
t.Errorf("snat sources %v", s)
|
||||
}
|
||||
if s := st.Forwards[0].Sources; strings.Join(s, ",") != "trusted" {
|
||||
t.Errorf("forward sources %v", s)
|
||||
}
|
||||
if ps := e.eng.Problems(); len(ps) != 0 {
|
||||
for _, p := range ps {
|
||||
if p.Severity == "error" {
|
||||
t.Errorf("problem after renaming: %+v", p)
|
||||
}
|
||||
}
|
||||
}
|
||||
// Undoing the renames (newest first) puts the references back too.
|
||||
for _, id := range []int64{serviceRename, setRename, zoneRename} {
|
||||
if code := e.call(e.token, "POST", "/history/"+itoa(id)+"/revert", nil, nil); code != 200 {
|
||||
t.Fatalf("undo %d: %d", id, code)
|
||||
}
|
||||
}
|
||||
st = e.eng.State()
|
||||
for _, r := range st.Rules {
|
||||
if strings.Join(r.InZones, ",") != "dhcp" || r.Chain == "forward" && r.Sources[0] != "trusted_ips" || r.Chain == "input" && r.Services[0] != "jarl" {
|
||||
t.Errorf("after undo: rule %+v", r.Match)
|
||||
}
|
||||
}
|
||||
// A dynamic set keeps its name: tokens and programs go by it.
|
||||
if code := e.call(e.token, "PATCH", "/sets/guests", map[string]any{"name": "visitors"}, nil); code != 409 {
|
||||
t.Errorf("renaming a dynamic set: %d", code)
|
||||
}
|
||||
}
|
||||
@@ -70,6 +70,12 @@ func engineStatus(kind string) int {
|
||||
// writeErr writes an error response. result, if given, is the preview of
|
||||
// a refused change.
|
||||
func (s *Server) writeErr(w http.ResponseWriter, r *http.Request, err error, result any) {
|
||||
var oe *opError
|
||||
if errors.As(err, &oe) {
|
||||
info, status := oe.info()
|
||||
writeJSON(w, status, api.ErrorBody{Error: info})
|
||||
return
|
||||
}
|
||||
var he *httpError
|
||||
if errors.As(err, &he) {
|
||||
writeError(w, he.status, he.code, he.msg)
|
||||
|
||||
@@ -10,15 +10,12 @@ import (
|
||||
"time"
|
||||
|
||||
"housecarl/internal/api"
|
||||
"housecarl/internal/version"
|
||||
)
|
||||
|
||||
func (s *Server) routes() {
|
||||
m := s.mux
|
||||
p := api.Prefix
|
||||
m.HandleFunc("GET /healthz", func(w http.ResponseWriter, r *http.Request) {
|
||||
writeJSON(w, http.StatusOK, map[string]any{"ok": true, "version": version.String()})
|
||||
})
|
||||
m.HandleFunc("GET /healthz", s.healthz)
|
||||
// Authentication, users and tokens
|
||||
m.HandleFunc("POST "+p+"/auth/login", s.public(s.login))
|
||||
m.HandleFunc("POST "+p+"/auth/logout", s.authed(s.logout))
|
||||
@@ -41,6 +38,27 @@ func (s *Server) routes() {
|
||||
m.HandleFunc("GET "+p+"/sets/{ref}/elements", s.authed(s.listElements))
|
||||
m.HandleFunc("POST "+p+"/sets/{ref}/elements", s.authed(s.addElement))
|
||||
m.HandleFunc("DELETE "+p+"/sets/{ref}/elements/{addr}", s.authed(s.deleteElement))
|
||||
m.HandleFunc("GET "+p+"/bans", s.authed(s.listBans))
|
||||
m.HandleFunc("GET "+p+"/banlists/{ref}/bans", s.authed(s.listBans))
|
||||
m.HandleFunc("POST "+p+"/banlists/{ref}/bans", s.authed(s.addBan))
|
||||
m.HandleFunc("DELETE "+p+"/banlists/{ref}/bans", s.authed(s.deleteBans))
|
||||
m.HandleFunc("DELETE "+p+"/banlists/{ref}/bans/{id}", s.authed(s.deleteBans))
|
||||
m.HandleFunc("GET "+p+"/quotas", s.authed(s.listQuotas))
|
||||
m.HandleFunc("GET "+p+"/connections", s.authed(s.listConnections))
|
||||
m.HandleFunc("POST "+p+"/connections/close", s.authed(s.closeConnections))
|
||||
m.HandleFunc("POST "+p+"/names", s.authed(s.lookupNames))
|
||||
m.HandleFunc("GET "+p+"/trace", s.authed(s.getTrace))
|
||||
m.HandleFunc("POST "+p+"/trace", s.authed(s.startTrace))
|
||||
m.HandleFunc("DELETE "+p+"/trace", s.authed(s.stopTrace))
|
||||
m.HandleFunc("POST "+p+"/rules/{ref}/reset-quota", s.authed(s.resetQuota))
|
||||
m.HandleFunc("GET "+p+"/changesets", s.authed(s.listChangesets))
|
||||
m.HandleFunc("POST "+p+"/changesets", s.authed(s.createChangeset))
|
||||
m.HandleFunc("GET "+p+"/changesets/{id}", s.authed(s.getChangeset))
|
||||
m.HandleFunc("PATCH "+p+"/changesets/{id}", s.authed(s.updateChangeset))
|
||||
m.HandleFunc("GET "+p+"/changesets/{id}/preview", s.authed(s.previewChangeset))
|
||||
m.HandleFunc("POST "+p+"/changesets/{id}/apply", s.authed(s.applyChangeset))
|
||||
m.HandleFunc("POST "+p+"/changesets/{id}/discard", s.authed(s.discardChangeset))
|
||||
m.HandleFunc("DELETE "+p+"/changesets/{id}/ops/{seq}", s.authed(s.removeOp))
|
||||
m.HandleFunc("GET "+p+"/{kind}", s.authed(s.listObjects))
|
||||
m.HandleFunc("POST "+p+"/{kind}", s.authed(s.createObject))
|
||||
m.HandleFunc("GET "+p+"/{kind}/{ref}", s.authed(s.getObject))
|
||||
@@ -48,8 +66,16 @@ func (s *Server) routes() {
|
||||
m.HandleFunc("DELETE "+p+"/{kind}/{ref}", s.authed(s.deleteObject))
|
||||
m.HandleFunc("POST "+p+"/{kind}/{ref}/move", s.authed(s.moveObject))
|
||||
m.HandleFunc("GET "+p+"/ruleset", s.authed(s.ruleset))
|
||||
m.HandleFunc("GET "+p+"/matrix", s.authed(s.zoneMatrix))
|
||||
m.HandleFunc("GET "+p+"/stats/status", s.authed(s.statsStatus))
|
||||
m.HandleFunc("GET "+p+"/stats/summary", s.authed(s.statsSummary))
|
||||
m.HandleFunc("GET "+p+"/stats/timeline", s.authed(s.statsTimeline))
|
||||
m.HandleFunc("GET "+p+"/stats/top", s.authed(s.statsTop))
|
||||
m.HandleFunc("GET "+p+"/stats/hits", s.authed(s.statsHits))
|
||||
m.HandleFunc("GET "+p+"/stats/heatmap", s.authed(s.statsHeatmap))
|
||||
m.HandleFunc("GET "+p+"/history", s.authed(s.listHistory))
|
||||
m.HandleFunc("GET "+p+"/history/{id}", s.authed(s.getHistory))
|
||||
m.HandleFunc("GET "+p+"/history/{id}/diff", s.authed(s.historyDiff))
|
||||
m.HandleFunc("POST "+p+"/history/{id}/revert", s.authed(s.revertHistory))
|
||||
m.HandleFunc("GET "+p+"/probation", s.authed(s.getProbation))
|
||||
m.HandleFunc("POST "+p+"/confirm", s.authed(s.confirmChanges))
|
||||
|
||||
Loaded 100 of 145 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user