Compare commits

...
32 Commits
Author SHA1 Message Date
abrown38andClaude Opus 5.5 507d75829f Release 0.2.0-rc.4+git70-83b97ad
release / release (push) Successful in 3m30s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 14:12:46 +01:00
abrown38andClaude Opus 5.5 83b97ad538 Refuse IPv4-mapped prefixes with an error that says so
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 14:12:46 +01:00
abrown38andClaude Opus 5.5 715cb772e7 Release 0.2.0-rc.4+git68-b4c0988
release / release (push) Successful in 1m7s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 18:06:39 +01:00
abrown38andClaude Opus 5.5 b4c098890c hc forward show: hits, comment, and hairpin without a stray dash
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 18:06:39 +01:00
abrown38andClaude Opus 5.5 1fcc204ff8 Release 0.2.0-rc.4
release / release (push) Successful in 1m6s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:51:55 +01:00
abrown38andClaude Opus 5.5 68ba2d06c5 Sets from Jarl: sets holding the addresses of names in Jarl
A set can take its addresses from Jarl: every A and AAAA record of its
names (NAME@VIEW), CNAMEs followed within Jarl's zones in the same view.
housecarld reads them every jarl.refresh and after each change, keeps
only the addresses within the zones and networks the set allows, and
keeps what it last read (saved in the database and the boot ruleset)
while Jarl can't be reached. A change of the addresses applies the
ruleset again without a history entry and is audited (set.jarl).

hc set add --jarl --within, hc set jarl; the kind "from Jarl" and
Addresses… on the Sets page; GET /sets/{name}/jarl and /jarl/refresh;
validation codes set_jarl_*, status warnings jarl_*; docs/jarl.md.
e2e phase with scripts/fakejarl.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 13:51:55 +01:00
abrown38andClaude Opus 5.5 50dce02746 Release 0.2.0-rc.3+git64-c3bd128
release / release (push) Successful in 1m4s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 12:46:53 +01:00
abrown38andClaude Opus 5.5 c3bd128778 hc user delete: ask first, and take -y
It deleted the user and their tokens at once, while every other delete
asks and takes -y/--yes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 12:46:53 +01:00
abrown38andClaude Opus 5.5 04e623daa1 Release 0.2.0-rc.3+git62-0d16358
release / release (push) Successful in 1m6s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 10:49:41 +01:00
abrown38andClaude Opus 5.5 0d16358617 History diff: keep a stepped-back object in place; number hunks right
Rewind put an object's old form at the end of its list, so a history
entry's diff showed the changed object moving (turning hairpin off for
one forward also moved its forward and DNAT lines). It now sorts after
stepping back. go-udiff v0.4.1 numbers the new side of hunks after one
with a moved line too early, so the chain after @@ could be the one
before; hunkStarts numbers them again. History diffs name their sides
"before N" and "after N".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 10:49:41 +01:00
abrown38andClaude Opus 5.5 540f4b85ac Release 0.2.0-rc.3+git60-ee27884
release / release (push) Successful in 1m4s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 00:15:31 +01:00
abrown38andClaude Opus 5.5 ee278848ee Docs: matching everything but a few; release builds of a candidate
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 00:15:31 +01:00
abrown38andClaude Opus 5.5 6f912dc137 Release 0.2.0-rc.3+5ec943e
release / release (push) Successful in 1m5s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 00:08:21 +01:00
abrown38andClaude Opus 5.5 5ec943ebff Match fields: "any of"/"any but" only with entries, never negate empty
The choice shows once the list has a row (empty is "any"), an empty
list is never sent as negated, a negated list is marked by a red bar
instead of red inputs (which read as errors), and the field's own label
names the choice for screen readers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 00:08:21 +01:00
abrown38andClaude Opus 5.5 4676d17aea Release 0.2.0-rc.3+52e6a98
release / release (push) Successful in 1m5s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 00:03:52 +01:00
abrown38andClaude Opus 5.5 52e6a987cc Match fields say "any of" or "any but" first
The zone, From and To fields start with an any of / any but choice
instead of a checkbox under the list, and an "any but" list is tinted
red. The release script encodes a + (build metadata) in its URLs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 00:03:52 +01:00
abrown38andClaude Opus 5.5 5020ff1185 Release 0.2.0-rc.3
release / release (push) Successful in 1m4s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 23:51:54 +01:00
abrown38andClaude Opus 5.5 e841986a86 Rule dialog: only the zone fields the chain can use
Input and early have no outgoing interface, output no incoming one, so
the dialog hides those fields (they could only be refused) and a zone
left in a hidden field isn't sent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 23:51:54 +01:00
abrown38andClaude Opus 5.5 0e958289de Web UI: administrators make tokens for other users
New token on a user's row under Users, with the Account page's dialog
(now tokenDialog in users.js), offering only roles within that user's
permissions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 23:49:12 +01:00
abrown38andClaude Opus 5.5 9ce4190e01 Release 0.2.0-rc.2
release / release (push) Successful in 1m5s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 23:39:03 +01:00
abrown38andClaude Opus 5.5 92147c8681 Diffs name their chain; history shows the ruleset's diff
The ruleset's diff names the chain each hunk is in on its @@ line, as
git names the function. A history entry shows what it did to the
ruleset (web page, hc history diff, GET /history/{id}/diff), rebuilt
from the current configuration by stepping back through the later
entries. A changed setting is listed by its own fields
(builtins.drop_invalid) instead of the whole object.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 23:39:03 +01:00
abrown38andClaude Opus 5.5 4e35236dba Release 0.2.0-rc.1
release / release (push) Successful in 1m5s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 18:52:51 +01:00
abrown38andClaude Opus 5.5 4af4a5908b Dashboard: a per-address quota's bar counts all its addresses
One address that used up its allowance filled the bar (100%). The bar
is now all the addresses together, and the dashboard and the rule's
quota line say how many used theirs up.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 18:52:51 +01:00
abrown38andClaude Opus 5.5 500bfc7bdd Ban lists: where Guard and fail2ban ban addresses
A program bans an address (every port, or a protocol's ports) for a
time; the ban list says what it is kept from: the firewall, some hosts
behind it, or everything. Bans drop ahead of every rule, close the
address's connections, group IPv6 by /64, are kept in the database
across restarts and audited. Never banned: the inside, the firewall's
own addresses, the caller, the list's never. bans grant (bans:write:LIST),
hc banlist/bans/ban/unban, Traffic -> Bans, a fail2ban action in contrib/,
and failed logins and refused tokens logged for Guard and fail2ban.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 16:29:37 +01:00
abrown38andClaude Opus 5.5 6f56eefae2 Release 0.1.1-rc.13
release / release (push) Successful in 1m5s
Housecarl 0.1.1's thirteenth release candidate. Since rc.12: quota usage
is saved after every change. make test and make e2e (304 checks) pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 15:04:04 +01:00
abrown38andClaude Opus 5.5 aaf0ac5206 Quotas: save their usage after every change
The server hooks the engine's OnChange to save what the quotas used, so
a reboot soon after a change that counts a quota differently doesn't
bring back its usage counted the old way (it was saved every five
minutes and on stop only).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 15:04:04 +01:00
abrown38andClaude Opus 5.5 8fa28d01c2 Release 0.1.1-rc.12
release / release (push) Successful in 1m6s
Housecarl 0.1.1's twelfth release candidate. Since rc.11: quota usage
as a percentage; Statistics' quotas link to their rules and include
per-address quotas. make test and make e2e (304 checks) pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 13:55:43 +01:00
abrown38andClaude Opus 5.5 7d23fb13ed Quotas: usage as a share, and their statistics per rule
What a quota used shows its percentage of the allowance on the Rules
page (line and dialog), the dashboard and in hc quotas. Statistics'
Quotas card links each entry to its rule, named, without the share of
the total; per-address quotas are in it at last: their quota:rN series
is the sum of their addresses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 13:55:43 +01:00
abrown38andClaude Opus 5.5 a169e9aa96 Release 0.1.1-rc.11
release / release (push) Successful in 1m3s
Housecarl 0.1.1's eleventh release candidate. Since rc.10: a quota keeps
what it used when its rule counts it differently (per address to a total:
the sum; wider groups: added up), or starts again with a warning. make
test, the real-nft quota test and make e2e (304 checks) pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 12:57:18 +01:00
abrown38andClaude Opus 5.5 9863dd1cdd Quotas: carry what was used when a rule counts differently
model.QuotaCarry says what becomes of a quota's use when its rule
changes how it counts; the firewall's apply follows it, comparing with
the rules in force: per address to a total starts at the addresses' sum,
wider groups add up their addresses, and what can't be carried (a total
to per address, per source to per destination, finer groups) starts
again, with the engine warning quota_restarts on that change. Before,
per source to per destination kept the sources' use as destinations',
and regrouping left old entries behind.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 12:57:18 +01:00
abrown38andClaude Opus 5.5 46c319eced Release 0.1.1-rc.10
release / release (push) Successful in 1m3s
Housecarl 0.1.1's tenth release candidate. Since rc.9: rules read as a
direction ("lan → wan", "any but wan → any"). make test and make e2e
(297 checks) pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 12:12:57 +01:00
abrown38andClaude Opus 5.5 24dbc0a869 Rules read as a direction
A match's zones are written "lan → wan" (or "any but wan → any", and
"lan → firewall" / "firewall → wan" in the input and output chains)
instead of "in lan · out wan" and "in not wan"; exclusions read "from
any but X". The Rules and NAT pages and hc; flags and JSON unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 12:12:57 +01:00
82 changed files with 6003 additions and 214 deletions

No files matched your search

+116
View File
@@ -8,6 +8,52 @@ All notable changes to Housecarl are documented here. The format follows
### Added
- **Sets from Jarl** ([docs/jarl.md](docs/jarl.md#sets-from-jarl)). A set
can hold the addresses of names in Jarl instead of listing them:
every A and AAAA record of each name, CNAMEs followed (within Jarl's
zones, in the same view), one view per name (`NAME@VIEW`; a name more
than one view covers must say which). housecarld reads them through
Jarl's API every `jarl.refresh` (a minute) and after each change, and
when they change applies the ruleset again, without a history entry
(audited as `set.jarl`). A set uses only the addresses `within` the
zones (their connected networks, which grow with the zone) and
networks it names; others are left out and warned of (`jarl_outside`).
While Jarl can't be read, sets keep the addresses they had, saved in
the database for restarts and in the boot ruleset (`jarl_unreachable`).
Names that give nothing say why (`jarl_name`: no record, a wildcard
only, a CNAME out of Jarl or in a loop, below a delegation, a secondary
zone, more than one view). `hc set add --jarl NAME@VIEW --within ZONE`,
`hc set jarl SET [--refresh]`; the kind "from Jarl" and Addresses… on
the Sets page; `GET /sets/{name}/jarl`, `POST /sets/{name}/jarl/refresh`.
Housecarl's Jarl token needs `records:read` for the zones.
- An administrator can make an API token for another user in the web
UI: New token on the user's row under Users (the CLI's `hc token create
--user` and the API could already).
- **Ban lists** ([docs/bans.md](docs/bans.md)). A program (Guard, which
reads logs, or fail2ban) bans an address, of every port or of some,
for a time; the ban list says what the address is kept from: the
firewall itself (`firewall`), some hosts behind it (`hosts`, port
forwards to them included), or `everything`. Bans are dropped ahead of
every rule, established connections too, and the banned address's
connections are closed. IPv6 addresses are banned by /64 (`group_v6`).
Never banned: inside networks, the firewall's own addresses, whoever
asks, and the list's `never`. Bans end by themselves, are cut to the
list's longest time, survive restarts and reboots, and are audited, but
aren't configuration (no history, probation or changesets). `hc
banlist`, `hc bans`, `hc ban`, `hc unban`; Traffic → Bans in the web UI;
`/banlists` and `/banlists/{ref}/bans` in the API; a `bans` grant that
can name lists (`bans:write:web`), in the `viewer` and `operator`
roles.
- **A fail2ban action**, `contrib/fail2ban/action.d/housecarl.conf`
(installed under `/usr/share/doc/housecarl/contrib/`): fail2ban's bans
go into a ban list, with the jail's port and protocol.
- **Failed logins in the journal.** A failed login and a refused API
token are logged (`msg="login failed" ip=… user=…`), for Guard and
fail2ban to act on.
- **Quota usage as a share.** What a quota used shows its percentage
(`12.3 GB of 50.0 GB used (24%)`, each address's in the quota
dialog), on the Rules page, the dashboard and in `hc quotas`.
- **Reset one address's quota, or give some back.** `hc rule
reset-quota ID --address ADDR` starts only that device's allowance
again; `--give 5gbytes` takes that much off what was used instead (one
@@ -76,8 +122,78 @@ All notable changes to Housecarl are documented here. The format follows
- Import: sets with `flags timeout` become dynamic sets; a DNAT of
several ports becomes one port forward per port.
### Changed
- `hc user delete` asks first, as every other delete does, and takes
`-y`/`--yes` not to; it deleted the user and their tokens at once.
- The ruleset's diff (Changes, `hc changeset diff`, `--dry-run`) names
the chain each part is in on its `@@` line (`@@ -40,6 +40,8 @@ chain
forward`), as `git diff` names the function, so a rule added deep in a
long chain says whether it is in input, forward or output.
- **History shows the ruleset's diff.** An entry's page (and `hc history
diff ID`, `GET /history/{id}/diff`) shows what the change did to the
ruleset, for entries made before this version too: the configuration
before and after it is got back from the current one and rendered by
this version.
- Negating a match is said first: the zone, From and To fields (rules
and NAT) start with "any of" / "any but" instead of a checkbox under
the list, shown once the list has an entry, and an "any but" list is
marked with a red bar. An empty list is never saved as negated (it
matched everything anyway, but would have negated what was added to
it later).
- docs/rules.md: "Matching everything but a few", on leaving some
addresses out with a zone and "any but", and what to do when the range
isn't a zone.
- The rule dialog shows only the zone fields its chain can use: no
"Going out on" for input and early, no "Coming in on" for output
(they could only be refused). A zone left in a field the chain hides
isn't sent.
- History lists a changed setting by its own fields
(`builtins.drop_invalid: false → true`), on the web page and in `hc
history show`, instead of the whole `builtins` before and after.
- Set names starting with `ban_` are now reserved (`set_name_reserved`):
ban lists' sets use them.
- **Rules read as a direction.** A match's zones are written `lan → wan`,
`any but wan → any`, or with the firewall itself as the other end
(`lan → firewall` in input, `firewall → wan` in output), instead of
`in lan · out wan` and `in not wan`; exclusions read `from any but X`.
On the Rules and NAT pages and in `hc`; the flags and JSON stay as
they were.
### Fixed
- An IPv4-mapped prefix (`::ffff:0:0/96`) is refused with an error that
says so; it was refused as "has host bits set; the prefix is invalid
Prefix".
- `hc forward show` shows the forward's hits and comment, and its
hairpin as `no`, `yes` or `yes, from …` (it printed `no -`).
- A history entry's ruleset diff showed the object it changed as moved:
stepping back put the old form at the end of its list, so updating
one port forward (turning hairpin off) also showed its forward and
DNAT lines moving down. The diff is now only what the change did, and
its sides are named `before N` and `after N` instead of `now`.
- Ruleset diffs numbered the hunks after one with a moved line too early
on the new side (`@@ -76,9 +74,9 @@` for `+76,9`), so the chain named
after `@@` could be the one before (`chain output` for a hunk in
`chain dstnat`). go-udiff miscounts them; Housecarl now numbers them
itself.
- The dashboard's bar for a per-address quota showed only the busiest
address, so one address that used up its allowance filled it (100%).
The bar is now all the addresses together, and the dashboard and the
rule's quota line say how many addresses used theirs up.
- Changing how a quota counts lost or muddled what it used: per address
to a total started at nothing, per source to per destination kept the
sources' use as if it were the destinations', and other groups left
the old entries behind. Now a total starts at the addresses' sum,
wider groups add up their addresses, and what can't be carried starts
again with a warning (`quota_restarts`) on the change.
- What the quotas used is saved after every change, not only every five
minutes: a reboot soon after a change that counts a quota differently
could bring back its usage counted the old way.
- Statistics' Quotas card linked every entry to the Rules page, not to
its rule; each now opens its rule, named. Per-address quotas were
missing from it (and from the `quota:rN` series): they count their
addresses' sum.
- `hc stats top devices` listed each device twice (once for what it
received, once for what it sent); it is one row with both again.
- `hc rule add --before/--after/--first` made two changes (create at the
+1
View File
@@ -112,6 +112,7 @@ install-files:
install -D -m 0644 CHANGELOG.md $(DESTDIR)$(DOCDIR)/CHANGELOG.md
install -d $(DESTDIR)$(DOCDIR)/docs
install -m 0644 docs/*.md $(DESTDIR)$(DOCDIR)/docs/
install -D -m 0644 contrib/fail2ban/action.d/housecarl.conf $(DESTDIR)$(DOCDIR)/contrib/fail2ban/action.d/housecarl.conf
uninstall:
rm -f $(DESTDIR)$(BINDIR)/housecarld $(DESTDIR)$(BINDIR)/hc $(DESTDIR)$(UNITDIR)/housecarl.service \
+9 -3
View File
@@ -53,6 +53,10 @@ table of its own.
firewall you have, blocking nothing: its hits, log and tracer show what
it would drop, so you see what switching over would break before you do
([docs/shadow.md](docs/shadow.md)).
* **Ban lists for Guard and fail2ban.** A program on any machine bans an
address it caught misbehaving in its logs, and the ban list says what
that address is kept from: the firewall, some hosts, or everything
([docs/bans.md](docs/bans.md)).
* **Easy to watch.** A zone matrix of what is allowed between zones,
statistics over time (traffic per zone and device, drops, the sources
blocked most, connections; [docs/stats.md](docs/stats.md)), hits per rule, a live log of what was dropped and
@@ -107,6 +111,7 @@ The web UI is on `http://127.0.0.1:8070/` (see `listen` in
* [install.md](docs/install.md): packages, building from source, first steps
* [rules.md](docs/rules.md): zones, address sets, services, rules and the built-ins
* [nat.md](docs/nat.md): port forwards, hairpin NAT, IPv6 pinholes, source NAT, redirects
* [bans.md](docs/bans.md): ban lists, where Guard and fail2ban ban addresses
* [import.md](docs/import.md): moving from an existing nftables ruleset
* [shadow.md](docs/shadow.md): shadow mode, Housecarl beside your firewall, changing nothing
* [operations.md](docs/operations.md): probation, the boot and its fallback, hits and the log, monitoring, backups, troubleshooting
@@ -123,14 +128,15 @@ The web UI is on `http://127.0.0.1:8070/` (see `listen` in
## Status
Housecarl 0.1.1 is in testing (0.1.1-rc.9): zones, address sets and
Housecarl 0.2.0 is in testing (0.2.0-rc.4+git70-83b97ad): zones, address sets and
services with ordered rules, IPv4 and IPv6, port forwards with hairpin NAT
and IPv6 pinholes, source NAT and redirects, built-ins, probation for
network changes (confirmed by the client that made them), the ruleset
at boot with a fail-closed fallback, hits and the drop log, nftables
import, changesets reviewed with the ruleset's diff (as in Jarl),
bandwidth limits and quotas, shadow mode beside another firewall, a web
UI, REST API and CLI.
bandwidth limits and quotas, shadow mode beside another firewall, ban
lists for Guard and fail2ban, sets of names from Jarl, a web UI, REST
API and CLI.
## Licence
+1 -1
View File
@@ -1 +1 @@
0.1.1-rc.9
0.2.0-rc.4+git70-83b97ad
+48 -3
View File
@@ -66,6 +66,30 @@ func historyCmd() *cobra.Command {
})
},
})
cmd.AddCommand(&cobra.Command{
Use: "diff ID", Short: "Show what a change did to the ruleset", Args: cobra.ExactArgs(1),
Long: `Shows the ruleset's diff for a change: the configuration before and after
it, rendered by this version of Housecarl.`,
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newClient()
if err != nil {
return err
}
var d struct {
Diff string `json:"diff"`
}
if err := c.Get(ctx(), "/history/"+args[0]+"/diff", nil, &d); err != nil {
return err
}
return output(d, func(w io.Writer) {
if d.Diff == "" {
fmt.Fprintln(w, "the ruleset didn't change")
return
}
printDiff(w, d.Diff)
})
},
})
cmd.AddCommand(&cobra.Command{
Use: "undo ID",
Short: "Undo a change",
@@ -88,11 +112,13 @@ change touched has changed since; undo the later changes first.`,
}
// diffJSON lists the fields that differ between two versions of an
// object: "name: office -> lab".
// object: "name: office -> lab". An object within (settings' builtins)
// is compared field by field: "builtins.drop_invalid: false -> true".
func diffJSON(before, after json.RawMessage) []string {
var a, b map[string]any
json.Unmarshal(before, &a)
json.Unmarshal(after, &b)
a, b = flatten(a, "", map[string]any{}), flatten(b, "", map[string]any{})
keys := map[string]bool{}
for k := range a {
keys[k] = true
@@ -132,6 +158,19 @@ func diffJSON(before, after json.RawMessage) []string {
return out
}
// flatten lifts the fields of objects within m to dotted keys; lists
// stay whole.
func flatten(m map[string]any, prefix string, out map[string]any) map[string]any {
for k, v := range m {
if o, ok := v.(map[string]any); ok && len(o) > 0 {
flatten(o, prefix+k+".", out)
} else {
out[prefix+k] = v
}
}
return out
}
func whoamiCmd() *cobra.Command {
return &cobra.Command{
Use: "whoami", Short: "Show who you are to the server", Args: cobra.NoArgs,
@@ -310,9 +349,13 @@ operator, log) or grants such as --grant config:read --grant sets:write:guests.
f.BoolVar(&enable, "enable", false, "enable the user again")
f.StringVar(&comment, "comment", "", "a comment")
cmd.AddCommand(update)
cmd.AddCommand(&cobra.Command{
var yes bool
del := &cobra.Command{
Use: "delete USER", Short: "Delete a user and their tokens", Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
if !confirm(fmt.Sprintf("Delete user %s and their tokens?", args[0]), yes) {
return fmt.Errorf("not deleted")
}
id, err := userID(args[0])
if err != nil {
return err
@@ -326,7 +369,9 @@ operator, log) or grants such as --grant config:read --grant sets:write:guests.
}
return nil
},
})
}
del.Flags().BoolVarP(&yes, "yes", "y", false, "don't ask")
cmd.AddCommand(del)
return cmd
}
+313
View File
@@ -0,0 +1,313 @@
package main
import (
"fmt"
"io"
"net/url"
"strconv"
"strings"
"github.com/spf13/cobra"
"housecarl/internal/api"
"housecarl/internal/model"
)
// ---- ban lists (docs/bans.md)
// banTime writes a ban list's time setting: 0 is the default, -1 none.
func banTime(n int, def string) string {
switch {
case n < 0:
return "no limit"
case n == 0:
return def + " (default)"
}
return fmtSeconds(n)
}
// parseBanTime reads a time flag: a duration, or "none" (-1) for no
// limit.
func parseBanTime(s string) (int, error) {
if strings.EqualFold(s, "none") || s == "-1" {
return -1, nil
}
return parseSeconds(s)
}
func banlistCmd() *cobra.Command {
cmd := &cobra.Command{Use: "banlist", Aliases: []string{"banlists"}, Short: "Ban lists: where Guard and fail2ban ban addresses"}
cmd.AddCommand(&cobra.Command{
Use: "list", Short: "List the ban lists", Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, args []string) error {
bs, err := get[[]model.BanList]("/banlists", nil)
if err != nil {
return err
}
bans, _ := get[[]model.Ban]("/bans", nil)
count := map[int64]int{}
for _, b := range bans {
count[b.ListID]++
}
return output(bs, func(w io.Writer) {
fmt.Fprintln(w, "NAME\tSCOPE\tHOSTS\tBANS\tCOMMENT")
for _, b := range bs {
scope := b.Scope
if b.Disabled {
scope += " (disabled)"
}
fmt.Fprintf(w, "%s\t%s\t%s\t%d\t%s\n", b.Name, scope, list(b.Hosts), count[b.ID], b.Comment)
}
})
},
})
cmd.AddCommand(showCmd("banlists", "banlist", func(w io.Writer, b model.BanList) {
fmt.Fprintf(w, "Name:\t%s\nScope:\t%s\n", b.Name, b.Scope)
if len(b.Hosts) > 0 {
fmt.Fprintf(w, "Hosts:\t%s\n", list(b.Hosts))
}
fmt.Fprintf(w, "Never:\t%s\n", list(b.Never))
fmt.Fprintf(w, "Ban time:\t%s\nLongest:\t%s\n", banTime(b.DefaultTime, "1h"), banTime(b.MaxTime, "7d"))
fmt.Fprintf(w, "Most bans:\t%d\nIPv6 banned by:\t/%d\n", b.Entries(), b.V6Group())
if b.Disabled {
fmt.Fprintln(w, "Disabled:\tyes")
}
if b.Comment != "" {
fmt.Fprintf(w, "Comment:\t%s\n", b.Comment)
}
}))
var scope, comment, banT, maxT string
var hosts, never []string
var entries, group int
var disabled bool
setFields := func(fl interface{ Changed(string) bool }, p patch) error {
if fl.Changed("scope") {
p["scope"] = scope
}
if fl.Changed("host") {
p["hosts"] = hosts
}
if fl.Changed("never") {
p["never"] = never
}
for _, t := range []struct{ flag, field, v string }{{"time", "default_time", banT}, {"max-time", "max_time", maxT}} {
if fl.Changed(t.flag) {
n, err := parseBanTime(t.v)
if err != nil {
return err
}
p[t.field] = n
}
}
if fl.Changed("max-entries") {
p["max_entries"] = entries
}
if fl.Changed("group-v6") {
p["group_v6"] = group
}
if fl.Changed("comment") {
p["comment"] = comment
}
if fl.Changed("disabled") {
p["disabled"] = disabled
}
return nil
}
flags := func(c *cobra.Command) {
f := c.Flags()
f.StringVar(&scope, "scope", "", "what bans keep addresses from: firewall, hosts or everything")
f.StringArrayVar(&hosts, "host", nil, "scope hosts: a machine's address, or a set (repeat)")
f.StringArrayVar(&never, "never", nil, "an address, prefix or set never banned (repeat)")
f.StringVar(&banT, "time", "", "a ban's length when none is asked for (default 1h)")
f.StringVar(&maxT, "max-time", "", "the longest ban (default 7d; none for no limit)")
f.IntVar(&entries, "max-entries", 0, "the most bans the list holds (default 10000)")
f.IntVar(&group, "group-v6", 0, "IPv6 prefix length banned for an address (default 64)")
f.StringVar(&comment, "comment", "", "a comment")
f.BoolVar(&disabled, "disabled", false, "the list's bans aren't in force")
}
add := &cobra.Command{
Use: "add NAME", Short: "Add a ban list", Args: cobra.ExactArgs(1),
Long: `Adds a ban list. A program bans an address into it (Guard, fail2ban's
housecarl action, hc ban), and the list says what that address is kept
from:
hc banlist add router --scope firewall # the firewall itself
hc banlist add web --scope hosts --host 10.0.10.57 --host 2001:db8:10::57
hc banlist add edge --scope everything --never trusted
Give the program a token for its list alone:
hc user add guard-web --no-password --grant bans:write:web
hc token create web1 --user guard-web # as root on the firewall`,
RunE: func(cmd *cobra.Command, args []string) error {
p := patch{"name": args[0]}
if err := setFields(cmd.Flags(), p); err != nil {
return err
}
return change("POST", "/banlists", p)
},
}
flags(add)
cmd.AddCommand(add)
update := &cobra.Command{
Use: "update BANLIST", Short: "Change a ban list", Args: cobra.ExactArgs(1),
Long: `Changes a ban list; --host and --never replace the lists. A ban list
can't be renamed: tokens and programs know it by its name.`,
RunE: func(cmd *cobra.Command, args []string) error {
p := patch{}
if err := setFields(cmd.Flags(), p); err != nil {
return err
}
return change("PATCH", objPath("banlists", args[0]), p)
},
}
flags(update)
cmd.AddCommand(update)
cmd.AddCommand(deleteCmd("banlists", "banlist"))
return cmd
}
// bansCmd lists the active bans.
func bansCmd() *cobra.Command {
return &cobra.Command{
Use: "bans [BANLIST]", Short: "List the active bans", Args: cobra.MaximumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
path := "/bans"
if len(args) == 1 {
path = objPath("banlists", args[0]) + "/bans"
}
bans, err := get[[]model.Ban](path, nil)
if err != nil {
return err
}
return output(bans, func(w io.Writer) {
fmt.Fprintln(w, "ID\tLIST\tADDRESS\tPORT\tENDS IN\tBY\tSOURCE\tREASON")
for _, b := range bans {
port := "all"
if b.Protocol != "" {
port = fmt.Sprintf("%s/%d", b.Protocol, b.Port)
}
fmt.Fprintf(w, "%d\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n", b.ID, b.List, b.Address, port, fmtSeconds(b.Timeout), b.Who, orDash(b.Source), orDash(clipText(b.Reason, 60)))
}
})
},
}
}
func clipText(s string, n int) string {
if len(s) > n {
return s[:n-1] + "…"
}
return s
}
// banCmd bans an address by hand.
func banCmd() *cobra.Command {
var tcp, udp []string
var t, reason string
cmd := &cobra.Command{
Use: "ban BANLIST ADDRESS", Short: "Ban an address", Args: cobra.ExactArgs(2),
Long: `Bans an address in a ban list: every port, or with --tcp and --udp some
(numbers or service names). IPv6 addresses are banned with the list's
grouping (a /64 unless set otherwise).
hc ban edge 203.0.113.9 --time 1d
hc ban web 203.0.113.9 --tcp ssh --tcp 8443 --reason "tried passwords"`,
RunE: func(cmd *cobra.Command, args []string) error {
in := api.BanInput{Address: args[1], Reason: reason, Source: "hc"}
if t != "" {
n, err := parseBanTime(t)
if err != nil {
return err
}
in.Time = n
}
c, err := newClient()
if err != nil {
return err
}
var out api.BanResult
do := func(proto string, ports []string) error {
in.Protocol = proto
in.Ports = nil
if len(ports) > 0 {
in.Ports = []byte(strconv.Quote(strings.Join(ports, ",")))
}
var res api.BanResult
if err := c.Do(ctx(), "POST", objPath("banlists", args[0])+"/bans", nil, in, &res); err != nil {
return err
}
out.Bans = append(out.Bans, res.Bans...)
out.Closed += res.Closed
return nil
}
switch {
case len(tcp) == 0 && len(udp) == 0:
err = do("", nil)
default:
if len(tcp) > 0 {
err = do("tcp", tcp)
}
if err == nil && len(udp) > 0 {
err = do("udp", udp)
}
}
if err != nil {
return err
}
return output(out, func(w io.Writer) {
for _, b := range out.Bans {
what := "banned"
if b.Existing {
what = "renewed"
}
port := "every port"
if b.Protocol != "" {
port = fmt.Sprintf("%s/%d", b.Protocol, b.Port)
}
ends := "never ends"
if b.Timeout > 0 {
ends = "ends in " + fmtSeconds(b.Timeout)
}
fmt.Fprintf(w, "%s %s, %s, in %s (%s)\n", what, b.Address, port, b.List, ends)
}
if out.Closed > 0 {
fmt.Fprintf(w, "%d connections closed\n", out.Closed)
}
})
},
}
f := cmd.Flags()
f.StringArrayVar(&tcp, "tcp", nil, "ban only this TCP port or service (repeat)")
f.StringArrayVar(&udp, "udp", nil, "ban only this UDP port or service (repeat)")
f.StringVar(&t, "time", "", "how long (1h, 7d; default the list's; none for ever, if the list allows)")
f.StringVar(&reason, "reason", "", "why, for the record")
return cmd
}
// unbanCmd lifts bans: by ID, or every ban of an address.
func unbanCmd() *cobra.Command {
return &cobra.Command{
Use: "unban BANLIST ADDRESS|ID", Short: "Lift a ban, or all of an address's", Args: cobra.ExactArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newClient()
if err != nil {
return err
}
path, query := objPath("banlists", args[0])+"/bans", url.Values{}
if _, err := strconv.ParseInt(args[1], 10, 64); err == nil {
path += "/" + args[1]
} else {
query.Set("address", args[1])
}
var out map[string]int
if err := c.Do(ctx(), "DELETE", path, query, nil, &out); err != nil {
return err
}
if !flagQuiet {
fmt.Printf("%d bans lifted\n", out["removed"])
}
return nil
},
}
}
+17 -2
View File
@@ -322,9 +322,13 @@ func quotasCmd() *cobra.Command {
if q.Quota.PerSource || q.Quota.PerDestination {
of += " each"
}
fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\n", q.Rule, humanBytes(q.Used), of, resets, next)
used := humanBytes(q.Used)
if !q.Quota.PerSource && !q.Quota.PerDestination {
used += " (" + percent(q.Used, q.Quota.Bytes) + ")"
}
fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\n", q.Rule, used, of, resets, next)
for _, a := range q.Addresses {
fmt.Fprintf(w, " %s\t%s\t\t\t\n", named(names, a.Address, a.Address), humanBytes(a.Used))
fmt.Fprintf(w, " %s\t%s (%s)\t\t\t\n", named(names, a.Address, a.Address), humanBytes(a.Used), percent(a.Used, q.Quota.Bytes))
}
}
})
@@ -332,6 +336,17 @@ func quotasCmd() *cobra.Command {
}
}
// percent writes used as a share of total: "25%", "<1%" for a little.
func percent(used, total int64) string {
if total <= 0 || used <= 0 {
return "0%"
}
if used*100 < total {
return "<1%"
}
return fmt.Sprintf("%d%%", used*100/total)
}
func logCmd() *cobra.Command {
var follow bool
var limit int
+1 -1
View File
@@ -192,7 +192,7 @@ Exit codes: 0 ok, 1 error, 3 not authenticated or not allowed, 4 not found,
root.AddCommand(c)
}
}
add("fw", statusCmd(), zoneCmd(), setCmd(), serviceCmd(), ruleCmd(), forwardCmd(), snatCmd(), redirectCmd(), settingsCmd(),
add("fw", statusCmd(), zoneCmd(), setCmd(), serviceCmd(), ruleCmd(), forwardCmd(), snatCmd(), redirectCmd(), banlistCmd(), bansCmd(), banCmd(), unbanCmd(), settingsCmd(),
rulesetCmd(), countersCmd(), quotasCmd(), logCmd(), connectionsCmd(), traceCmd(), matrixCmd(), statsCmd(), interfaceCmd())
add("changes", changesetCmd(), confirmCmd(), rollbackCmd(), historyCmd(), importCmd(), takeoverCmd())
add("admin", whoamiCmd(), userCmd(), tokenCmd(), passwordCmd(), certCmd(), auditCmd(), configCmd())
+22 -3
View File
@@ -35,9 +35,28 @@ func TestLimitAndDescribe(t *testing.T) {
if _, err := parseLimit("fast"); err == nil {
t.Error("a bad limit accepted")
}
m := model.Match{InZones: []string{"wan"}, NotSources: true, Sources: []string{"trusted"}, Services: []string{"ssh", "tcp/8080"}}
if got := describe(m); got != "in wan from not trusted ssh,tcp/8080" {
t.Errorf("describe: %q", got)
for _, c := range []struct {
m model.Match
chain string
want string
}{
{model.Match{InZones: []string{"wan"}, NotSources: true, Sources: []string{"trusted"}, Services: []string{"ssh", "tcp/8080"}}, "input",
"wan → firewall from any but trusted ssh,tcp/8080"},
{model.Match{InZones: []string{"wan"}, NotIn: true, Destinations: []string{"trusted_ips"}}, "forward", "any but wan → any to trusted_ips"},
{model.Match{InZones: []string{"lan"}, OutZones: []string{"wan"}}, "forward", "lan → wan"},
{model.Match{OutZones: []string{"wan"}}, "output", "firewall → wan"},
{model.Match{OutZones: []string{"wan"}, Sources: []string{"10.0.0.0/8"}}, "", "any → wan from 10.0.0.0/8"},
{model.Match{Sources: []string{"trusted"}}, "input", "from trusted"},
{model.Match{}, "forward", "everything"},
} {
if got := describe(c.m, c.chain); got != c.want {
t.Errorf("describe(%+v, %s) = %q, want %q", c.m, c.chain, got, c.want)
}
}
for used, want := range map[int64]string{0: "0%", 5: "<1%", 250: "25%", 1000: "100%", 1300: "130%"} {
if got := percent(used, 1000); got != want {
t.Errorf("percent(%d, 1000) = %q, want %q", used, got, want)
}
}
if got := editList([]string{"a", "b", "c"}, []string{"d"}, []string{"b"}); len(got) != 3 || got[1] != "c" || got[2] != "d" {
t.Errorf("editList: %v", got)
+132 -1
View File
@@ -271,6 +271,9 @@ func setCmd() *cobra.Command {
fmt.Fprintln(w, "NAME\tENTRIES\tCOMMENT")
for _, s := range ss {
entries := list(s.Entries)
if s.Jarl != nil {
entries = "from Jarl: " + strings.Join(jarlNames(s.Jarl), ", ")
}
if s.Dynamic {
entries = "dynamic"
if s.Timeout > 0 {
@@ -290,6 +293,11 @@ func setCmd() *cobra.Command {
fmt.Fprintf(w, "Timeout:\t%s\n", fmtSeconds(s.Timeout))
}
}
if s.Jarl != nil {
fmt.Fprintf(w, "From Jarl:\t%s\n", strings.Join(jarlNames(s.Jarl), ", "))
fmt.Fprintf(w, "Within:\t%s\n", strings.Join(s.Jarl.Within, ", "))
fmt.Fprintf(w, "Addresses:\thc set jarl %s\n", s.Name)
}
for i, e := range s.Entries {
label := ""
if i == 0 {
@@ -303,18 +311,29 @@ func setCmd() *cobra.Command {
}))
var dynamic bool
var timeout, comment string
var jarlN, within []string
add := &cobra.Command{
Use: "add NAME [ENTRY...]", Short: "Add a set", Args: cobra.MinimumNArgs(1),
Long: `Adds a set of addresses, prefixes and ranges, IPv4 and IPv6 mixed:
hc set add trusted 192.168.1.0/24 192.168.2.0/24 2001:db8:1::/64
hc set add guests --dynamic --timeout 24h
hc set add nas --jarl nas.example.com@internal --within lan
A dynamic set takes entries at run time, each with its own timeout ('hc
set entries', or the API with a token for that set alone: a captive
portal's signed-in clients).`,
portal's signed-in clients).
A Jarl set holds the addresses of names in Jarl (their A and AAAA
records, CNAMEs followed), read again every minute. --jarl takes
NAME@VIEW, or NAME when only one view has it; --within says where its
addresses may be (zones: their connected networks; or networks), and the
others are left out ('hc set jarl NAME' shows them). docs/jarl.md.`,
RunE: func(cmd *cobra.Command, args []string) error {
s := model.AddressSet{Name: args[0], Entries: args[1:], Dynamic: dynamic, Comment: comment}
if len(jarlN) > 0 || len(within) > 0 {
s.Jarl = jarlSource(jarlN, within)
}
if timeout != "" {
n, err := parseSeconds(timeout)
if err != nil {
@@ -327,6 +346,8 @@ portal's signed-in clients).`,
}
add.Flags().BoolVar(&dynamic, "dynamic", false, "entries are added at run time")
add.Flags().StringVar(&timeout, "timeout", "", "a dynamic set's default entry lifetime (1h, 7d)")
add.Flags().StringArrayVar(&jarlN, "jarl", nil, "a name in Jarl whose addresses the set holds: NAME or NAME@VIEW (repeat)")
add.Flags().StringArrayVar(&within, "within", nil, "a Jarl set: a zone or network its addresses must lie in (repeat)")
add.Flags().StringVar(&comment, "comment", "", "a comment")
cmd.AddCommand(add)
var addE, removeE, entries []string
@@ -350,6 +371,23 @@ portal's signed-in clients).`,
}
p["entries"] = editList(cur, addE, removeE)
}
if fl.Changed("jarl") || fl.Changed("within") {
s, err := get[model.AddressSet](objPath("sets", args[0]), nil)
if err != nil {
return err
}
j := s.Jarl
if j == nil {
j = &model.JarlSource{}
}
if fl.Changed("jarl") {
j.Names = jarlSource(jarlN, nil).Names
}
if fl.Changed("within") {
j.Within = within
}
p["jarl"] = j
}
if fl.Changed("timeout") {
n, err := parseSeconds(timeout)
if err != nil {
@@ -371,11 +409,104 @@ portal's signed-in clients).`,
f.StringArrayVar(&addE, "add", nil, "add an entry")
f.StringArrayVar(&removeE, "remove", nil, "remove an entry")
f.StringVar(&timeout, "timeout", "", "a dynamic set's default entry lifetime")
f.StringArrayVar(&jarlN, "jarl", nil, "replace a Jarl set's names: NAME or NAME@VIEW (repeat)")
f.StringArrayVar(&within, "within", nil, "replace where a Jarl set's addresses may be: zones, networks (repeat)")
f.StringVar(&comment, "comment", "", "a comment")
f.StringVar(&name, "name", "", "a new name")
cmd.AddCommand(update)
cmd.AddCommand(deleteCmd("sets", "set"))
cmd.AddCommand(entriesCmd())
cmd.AddCommand(jarlCmd())
return cmd
}
func jarlSource(names, within []string) *model.JarlSource {
j := &model.JarlSource{Names: []model.JarlName{}, Within: within}
for _, n := range names {
j.Names = append(j.Names, model.ParseJarlName(n))
}
return j
}
func jarlNames(j *model.JarlSource) []string {
var out []string
for _, n := range j.Names {
out = append(out, n.String())
}
return out
}
// jarlCmd shows what a Jarl set holds, name by name.
func jarlCmd() *cobra.Command {
var refresh bool
cmd := &cobra.Command{Use: "jarl SET", Short: "What a Jarl set holds: each name's addresses in Jarl, and which are used", Args: cobra.ExactArgs(1),
Long: `Shows a Jarl set's names with the addresses Jarl gives them (CNAMEs
followed), whether each is used or why it is left out (not within the
set's zones and networks), and when Jarl last answered. --refresh reads
Jarl first.`,
RunE: func(cmd *cobra.Command, args []string) error {
c, err := newClient()
if err != nil {
return err
}
var js api.JarlSet
if refresh {
err = c.Do(ctx(), "POST", objPath("sets", args[0])+"/jarl/refresh", nil, nil, &js)
} else {
err = c.Do(ctx(), "GET", objPath("sets", args[0])+"/jarl", nil, nil, &js)
}
if err != nil {
return err
}
return output(js, func(w io.Writer) {
if !js.Configured {
fmt.Fprintln(w, "Jarl isn't configured (jarl.server in housecarl.yaml): the set keeps the addresses it last had.")
}
if js.Error != "" {
fmt.Fprintf(w, "Reading Jarl failed: %s (the set keeps the addresses it had)\n", js.Error)
}
fmt.Fprintf(w, "Set:\t%s\n", js.Set)
fmt.Fprintf(w, "Within:\t%s\n", strings.Join(js.Within, ", "))
fmt.Fprintf(w, "Holds:\t%s\n", list(js.Entries))
if js.Read != nil {
fmt.Fprintf(w, "Jarl read:\t%s\n", js.Read.Local().Format("2006-01-02 15:04:05"))
}
fmt.Fprintln(w)
fmt.Fprintln(w, "NAME\tVIEW\tADDRESS\tUSED")
for _, n := range js.Names {
label := n.Name
if len(n.Chain) > 0 {
label += " → " + strings.Join(n.Chain, " → ")
}
view := n.View
if view == "" {
view = n.Found
}
if view == "" {
view = "-"
}
switch {
case n.Problem != "":
fmt.Fprintf(w, "%s\t%s\t-\tno: %s\n", label, view, n.Problem)
case len(n.Addresses) == 0 && n.Error != "":
fmt.Fprintf(w, "%s\t%s\t-\tnot read: %s\n", label, view, n.Error)
case len(n.Addresses) == 0:
fmt.Fprintf(w, "%s\t%s\t-\tnot read yet\n", label, view)
}
for _, a := range n.Addresses {
used := "yes"
if !a.Used {
used = "no: " + a.Why
}
fmt.Fprintf(w, "%s\t%s\t%s\t%s\n", label, view, a.Address, used)
}
if n.Error != "" && len(n.Addresses) > 0 {
fmt.Fprintf(w, "%s\t%s\t\t(the last read failed: %s; these are from before)\n", label, view, n.Error)
}
}
})
}}
cmd.Flags().BoolVar(&refresh, "refresh", false, "read Jarl now")
return cmd
}
+44 -17
View File
@@ -64,22 +64,40 @@ func (m *matchFlags) patch(fl *pflag.FlagSet, p patch) {
set("family", "family", m.family)
}
// describe sums up a match: "in lan from trusted ssh, dns".
func describe(m model.Match) string {
// describe sums up a match in a chain ("" for NAT): "lan → firewall
// from trusted ssh,dns". Zones are a direction; the firewall itself is
// the other end of the input and output chains.
func describe(m model.Match, chain string) string {
var parts []string
side := func(word string, not bool, list []string) {
if len(list) == 0 {
return
}
list := func(not bool, xs []string) string {
s := strings.Join(xs, ",")
if not {
word += " not"
return "any but " + s
}
parts = append(parts, word+" "+strings.Join(list, ","))
return s
}
if len(m.InZones) > 0 || len(m.OutZones) > 0 {
in, out := "any", "any"
if len(m.InZones) > 0 {
in = list(m.NotIn, m.InZones)
}
if len(m.OutZones) > 0 {
out = list(m.NotOut, m.OutZones)
}
switch chain {
case model.ChainInput:
out = "firewall"
case model.ChainOutput:
in = "firewall"
}
parts = append(parts, in+" → "+out)
}
if len(m.Sources) > 0 {
parts = append(parts, "from "+list(m.NotSources, m.Sources))
}
if len(m.Destinations) > 0 {
parts = append(parts, "to "+list(m.NotDests, m.Destinations))
}
side("in", m.NotIn, m.InZones)
side("out", m.NotOut, m.OutZones)
side("from", m.NotSources, m.Sources)
side("to", m.NotDests, m.Destinations)
if len(m.Services) > 0 {
parts = append(parts, strings.Join(m.Services, ","))
}
@@ -281,7 +299,7 @@ func ruleCmd() *cobra.Command {
return output(rs, func(w io.Writer) {
fmt.Fprintln(w, "CHAIN\tID\tMATCH\tACTION\tHITS\tCOMMENT")
for _, r := range rs {
what := describe(r.Match)
what := describe(r.Match, r.Chain)
if r.Raw != "" {
what = "raw: " + r.Raw
}
@@ -304,7 +322,7 @@ func ruleCmd() *cobra.Command {
cmd.AddCommand(ls)
cmd.AddCommand(showCmd("rules", "rule", func(w io.Writer, r model.Rule) {
if r.Raw == "" {
fmt.Fprintf(w, "ID:\t%d\nChain:\t%s\nMatch:\t%s%s\nAction:\t%s\n", r.ID, r.Chain, describe(r.Match), ruleExtras(r), r.Action)
fmt.Fprintf(w, "ID:\t%d\nChain:\t%s\nMatch:\t%s%s\nAction:\t%s\n", r.ID, r.Chain, describe(r.Match, r.Chain), ruleExtras(r), r.Action)
} else {
fmt.Fprintf(w, "ID:\t%d\nChain:\t%s\n", r.ID, r.Chain)
}
@@ -587,7 +605,16 @@ func forwardCmd() *cobra.Command {
if f.ToPort != "" {
fmt.Fprintf(w, "To port:\t%s\n", f.ToPort)
}
fmt.Fprintf(w, "To (IPv6):\t%s\nFrom:\t%s\nHairpin:\t%s %s\n", orDash(f.To6), list(f.Sources), yesNo(f.Hairpin), list(f.HairpinSources))
hairpin := yesNo(f.Hairpin)
if f.Hairpin && len(f.HairpinSources) > 0 {
hairpin += ", from " + list(f.HairpinSources)
}
fmt.Fprintf(w, "To (IPv6):\t%s\nFrom:\t%s\nHairpin:\t%s\n", orDash(f.To6), list(f.Sources), hairpin)
if f.Comment != "" {
fmt.Fprintf(w, "Comment:\t%s\n", f.Comment)
}
cs := counters()
fmt.Fprintf(w, "Hits:\t%s\n", hits(cs, fmt.Sprintf("f%d", f.ID), fmt.Sprintf("f%d-6", f.ID)))
}))
var pf model.PortForward
var to string
@@ -700,7 +727,7 @@ func snatCmd() *cobra.Command {
if to == "" {
to = "masquerade"
}
fmt.Fprintf(w, "%d\t%s\t%s\t%s\t%s\n", x.ID, describe(x.Match), to, hits(cs, fmt.Sprintf("s%d", x.ID)), x.Comment)
fmt.Fprintf(w, "%d\t%s\t%s\t%s\t%s\n", x.ID, describe(x.Match, ""), to, hits(cs, fmt.Sprintf("s%d", x.ID)), x.Comment)
}
})
},
@@ -780,7 +807,7 @@ func redirectCmd() *cobra.Command {
return output(xs, func(w io.Writer) {
fmt.Fprintln(w, "ID\tMATCH\tTO PORT\tHITS\tCOMMENT")
for _, x := range xs {
fmt.Fprintf(w, "%d\t%s\t%s\t%s\t%s\n", x.ID, describe(x.Match), orDash(x.ToPort), hits(cs, fmt.Sprintf("d%d", x.ID)), x.Comment)
fmt.Fprintf(w, "%d\t%s\t%s\t%s\t%s\n", x.ID, describe(x.Match, ""), orDash(x.ToPort), hits(cs, fmt.Sprintf("d%d", x.ID)), x.Comment)
}
})
},
+35 -6
View File
@@ -30,6 +30,7 @@ import (
"housecarl/internal/engine"
"housecarl/internal/hostif"
"housecarl/internal/jarl"
"housecarl/internal/jarlsets"
"housecarl/internal/linkwatch"
"housecarl/internal/model"
"housecarl/internal/names"
@@ -156,7 +157,8 @@ func serveCmd() *cobra.Command {
host := &hostif.Host{}
eng.Env = hostEnv(host)
fw := &nft.Firewall{Runner: &nft.Runner{Command: nft.Wrapper()}, Interfaces: host.Exists, BootFile: a.cfg.Ruleset,
Shadow: a.cfg.Shadow.Enabled, ShadowBefore: a.cfg.Shadow.Position == config.ShadowBefore}
Shadow: a.cfg.Shadow.Enabled, ShadowBefore: a.cfg.Shadow.Position == config.ShadowBefore,
Bans: func() []render.Ban { return server.ActiveBans(context.Background(), a.store) }}
eng.Firewall = fw
if fw.Shadow {
a.log.Warn("shadow mode: Housecarl's ruleset is loaded but blocks, rejects and translates nothing; the other firewall decides (docs/shadow.md)", "position", a.cfg.Shadow.Position)
@@ -164,19 +166,43 @@ func serveCmd() *cobra.Command {
for _, p := range eng.Problems() {
a.log.Warn("configuration problem: "+p.Message, "object", p.Object, "code", p.Code)
}
var acme *certs.Manager
if a.cfg.TLS.ACME.Domain != "" {
var jc *jarl.Client
if a.cfg.Jarl.Enabled() {
tok, err := a.cfg.Jarl.ReadToken()
if err != nil {
return err
}
jc, err := jarl.New(a.cfg.Jarl.Server, tok, a.cfg.Jarl.CACert, a.cfg.Jarl.Insecure)
if err != nil {
if jc, err = jarl.New(a.cfg.Jarl.Server, tok, a.cfg.Jarl.CACert, a.cfg.Jarl.Insecure); err != nil {
return err
}
}
var acme *certs.Manager
if a.cfg.TLS.ACME.Domain != "" {
if jc == nil {
return errors.New("tls.acme needs Jarl: set jarl.server and its token (docs/certificates.md)")
}
acme = certs.New(ctx, certs.Config{ACME: a.cfg.TLS.ACME, Version: version.Short()}, jc, a.store, a.log)
go acme.Run(ctx)
}
// The sets whose addresses come from Jarl: what was last read
// is put in force at once, Jarl is read again soon after.
jarlSets := &jarlsets.Resolver{Interval: time.Duration(a.cfg.Jarl.Refresh), State: eng.State, Log: a.log,
Apply: func(ctx context.Context) error {
if err := eng.ApplyCurrent(ctx); err != nil {
return err
}
return fw.SaveBoot(eng.State())
},
Audit: func(ctx context.Context, set, detail string) {
eng.Audit(ctx, &auth.Principal{Method: "jarl"}, "set.jarl", set, "success", detail)
},
Load: func(ctx context.Context) (string, error) { return a.store.GetMeta(ctx, jarlsets.MetaKey) },
Save: func(ctx context.Context, v string) error { return a.store.SetMeta(ctx, jarlsets.MetaKey, v) }}
if jc != nil {
jarlSets.Source = jc
}
jarlSets.Restore(ctx)
fw.JarlSets = jarlSets
var resolver *names.Resolver
if a.cfg.Names.Enabled {
resolver = names.New(a.cfg.Names.Resolver, time.Duration(a.cfg.Names.Cache))
@@ -185,8 +211,9 @@ func serveCmd() *cobra.Command {
Listen: a.cfg.Listen, TLSCertFile: a.cfg.TLS.CertFile, TLSKeyFile: a.cfg.TLS.KeyFile, TrustedProxies: a.cfg.Proxies(),
SessionLifetime: time.Duration(a.cfg.Session.Lifetime), SessionIdle: time.Duration(a.cfg.Session.IdleTimeout),
CookieSecure: a.cfg.Session.CookieSecure, Socket: a.cfg.Socket, ACME: acme,
Shadow: a.cfg.Shadow.Enabled, ShadowPosition: a.cfg.Shadow.Position, Names: resolver,
Shadow: a.cfg.Shadow.Enabled, ShadowPosition: a.cfg.Shadow.Position, Names: resolver, JarlSets: jarlSets,
}, eng, fw, host, a.log, web.FS())
jarlSets.Connected = srv.Connected
// Put the ruleset in force: what was on probation when
// housecarld stopped is rolled back first. An empty
// configuration applies nothing: its default policies would
@@ -216,6 +243,7 @@ func serveCmd() *cobra.Command {
go func() {
err := linkwatch.Watch(ctx, func() {
host.Forget()
jarlSets.Kick() // a zone's networks may have changed
if !fw.FlowtableStale(eng.State()) {
return
}
@@ -252,6 +280,7 @@ func serveCmd() *cobra.Command {
}
}()
}
go jarlSets.Run(ctx)
go nflog.Listen(ctx, render.LogGroup, a.log, srv.LogPacket)
go srv.RunQuotas(ctx)
srv.SetConntrack(conntrack.Client{})
+52
View File
@@ -0,0 +1,52 @@
# Housecarl's action for fail2ban: a ban goes into a Housecarl ban list,
# and the list says what the banned address is kept from (the firewall
# itself, some hosts behind it, or everything). See docs/bans.md.
#
# Copy this file to /etc/fail2ban/action.d/housecarl.conf, make a ban
# list and a token for it alone on the firewall:
#
# hc banlist add web --scope hosts --host 10.0.10.57
# hc user add f2b-web --no-password --grant bans:write:web
# hc token create web --user f2b-web # as root on the firewall
#
# put the token in /etc/fail2ban/housecarl.token (mode 0600), and use it
# in a jail:
#
# [sshd]
# enabled = true
# action = housecarl[list=web, url="https://router.lan:8070"]
#
# The jail's port and protocol go along: port = ssh bans only tcp/22;
# fail2ban's default (0:65535) bans every port. IPv6 addresses are banned
# with the list's grouping (a /64 unless set otherwise). Housecarl never
# bans inside networks, its own addresses or the list's "never".
#
# On the firewall itself, the local socket works too, still with the
# token: url = http://localhost, curlopts = --unix-socket
# /run/housecarl/housecarl.sock.
[Definition]
actionstart =
actionstop =
actioncheck =
# A ban renews one already there; a bantime of -1 is the list's longest.
actionban = <_curl> -f -X POST -H "Content-Type: application/json" "<url>/api/v1/banlists/<list>/bans"
-d '{"address": "<ip>", "protocol": "<protocol>", "ports": "<port>", "time": <bantime>, "source": "fail2ban/<name>"}'
# A ban Housecarl already ended (its own time ran out) is no error.
actionunban = <_curl> -X DELETE "<url>/api/v1/banlists/<list>/bans?address=<ip>"
[Init]
# The ban list, and Housecarl's URL (its listen address).
list =
url = https://127.0.0.1:8070
# The API token, in a file only root reads.
tokenfile = /etc/fail2ban/housecarl.token
# More curl options: --cacert /etc/fail2ban/housecarl-ca.pem for a
# private CA, --unix-socket for the local socket.
curlopts =
_curl = curl -sS --max-time 10 <curlopts> -o /dev/null -H "Authorization: Bearer $(cat <tokenfile>)"
+72 -6
View File
@@ -57,7 +57,9 @@ Calls that change the configuration answer:
one). Missing when a person must confirm (`manual_confirm`).
* `?dry_run=1` checks the change, with nftables too, and returns what it
would do, with `"applied": false, "dry_run": true`, and `diff`: the
ruleset's unified diff.
ruleset's unified diff, each `@@` line followed by the chain (or other
block of the table) the hunk starts in, as in `@@ -40,6 +40,8 @@ chain
forward`.
* `?changeset=ID` (or the header `X-Housecarl-Changeset: ID`) stages the
change in that changeset instead ([Changesets](#changesets)): the answer
has `"staged": true`, `changes` (this operation's) and `changeset`.
@@ -108,15 +110,15 @@ Lists replace the old ones. `probation_seconds` 0 is the default (120).
### Objects
Zones, sets, services, rules, port forwards, source NAT and redirects
share one pattern, `/{kind}` with kind `zones`, `sets`, `services`,
`rules`, `forwards`, `snats` or `redirects`:
Zones, sets, services, rules, port forwards, source NAT, redirects and
ban lists share one pattern, `/{kind}` with kind `zones`, `sets`,
`services`, `rules`, `forwards`, `snats`, `redirects` or `banlists`:
| | |
|---|---|
| `GET /{kind}` | all of them, in order; `GET /rules?chain=input` for one chain; `GET /services` includes the built-in ones |
| `POST /{kind}[?before=ID\|after=ID\|first=1]` | create; rules, forwards and NAT rules go last unless given a `position`, or placed before or after another of their chain, or first, in the same change (one history entry; staged in a changeset, the place goes with it) |
| `GET /{kind}/{ref}` | by ID, or name for zones, sets, services and forwards |
| `GET /{kind}/{ref}` | by ID, or name for zones, sets, services, forwards and ban lists |
| `PATCH /{kind}/{ref}` | change the fields given (a JSON merge: `null` clears) |
| `DELETE /{kind}/{ref}` | refused while something uses it |
| `POST /rules/{ref}/reset-quota` | start a rule's quota again now (`config:write`; audited, not staged). Optional body: `{"address": "192.168.1.23"}` starts only that address's again (a per-address quota; an IPv6 address counts for its /64; 404 if it hasn't used the quota); `{"give": 5368709120}` takes that many bytes off what was used instead (the address's, or every address's), until the next reset |
@@ -127,6 +129,7 @@ The objects:
```json
{"name": "lan", "interfaces": ["eth1", "eth1.*"], "external": false, "comment": ""}
{"name": "trusted", "entries": ["192.168.1.0/24", "2001:db8:1::/64"], "dynamic": false, "timeout": 0, "comment": ""}
{"name": "nas", "entries": [], "jarl": {"names": [{"name": "nas.example.com", "view": "internal"}], "within": ["lan", "10.9.0.0/16"]}, "comment": ""}
{"name": "web", "entries": ["tcp/8080", "tcp/8443"], "comment": ""}
{"chain": "input",
"in_zones": ["wan"], "not_in": false, "out_zones": [], "not_out": false,
@@ -143,9 +146,26 @@ The objects:
"comment": "", "disabled": false}
```
A set with `jarl` takes its addresses from Jarl ([jarl.md](jarl.md#sets-from-jarl)):
`names` are names in Jarl, each with a `view` (left out: the only view that
has it); `within` are zones and addresses its addresses must lie in. It
has no `entries` and isn't `dynamic`; `"jarl": null` makes it an ordinary
set again.
A source NAT rule is a match (as a rule's, without the rule's other
fields) with `"to"` (empty: masquerade), a redirect a match with
`"to_port"`. Times are seconds (`timeout`).
`"to_port"`. A ban list ([bans.md](bans.md)):
```json
{"name": "web", "scope": "hosts", "hosts": ["10.0.10.57"], "never": ["trusted"],
"default_time": 3600, "max_time": 604800, "max_entries": 10000, "group_v6": 64, "comment": "", "disabled": false}
```
`scope` is `firewall`, `hosts` or `everything`; 0 is the default for the
numbers, and `max_time` -1 is no limit. A ban list can't be renamed.
Someone with a `bans` grant but no `config:read` may read the ban lists
the grant covers (`GET /banlists`, `GET /banlists/{ref}`). Times are
seconds (`timeout`).
### Statistics
@@ -196,6 +216,51 @@ These need `sets:read` or `sets:write`, for that set
([permissions.md](permissions.md)); they aren't on probation or in the
history, but are audited.
### Sets from Jarl
| | |
|---|---|
| `GET /sets/{name}/jarl` | what the set holds and why (below) |
| `POST /sets/{name}/jarl/refresh` | reads Jarl now (for every set from Jarl), then answers as `GET` |
They need `sets:read` for that set. 400 for a set that doesn't come from
Jarl. The answer:
```json
{"set": "nas", "configured": true, "read": "2026-10-23T14:02:11Z", "error": "",
"entries": ["10.10.0.20", "10.10.0.21"], "within": ["lan"],
"names": [{"name": "files.example.com", "view": "", "found_view": "internal", "chain": ["nas.example.com"],
"addresses": [{"address": "10.10.0.20", "used": true},
{"address": "2001:db8:10::20", "used": false, "why": "not within lan"}],
"problem": "", "checked": "2026-10-23T14:02:11Z", "error": ""}]}
```
* `configured`: `housecarl.yaml` names a Jarl server; `read`: when Jarl's
zones were last read; `error`: why the last read failed (the set keeps
its addresses).
* `entries`: the addresses the set holds now.
* For each name: `found_view` is the view it was read from, `chain` the
CNAMEs followed, `addresses` every address Jarl gives it with `used`
(or `why` not), `problem` why it gives none, `checked` when Jarl last
answered for it, and `error` why the last try failed (its addresses are
then those of `checked`).
### Bans
| | |
|---|---|
| `GET /bans` | the active bans of every ban list you may read: `id`, `list_id`, `list`, `address` (an address, or an IPv6 prefix), `protocol` and `port` (none: every port), `expires` and `timeout` (seconds left; none: no end), `created`, `who`, `source`, `reason` |
| `GET /banlists/{ref}/bans` | one list's |
| `POST /banlists/{ref}/bans` | ban: `{"address": "203.0.113.9", "protocol": "tcp", "ports": "ssh", "time": 3600, "source": "guard/ssh", "reason": "…"}`. `ports` is a number, a string (`"22"`, `"http,https"`, `"0:65535"` or `"all"` for every port) or a list; without a protocol a number is TCP and UDP, a service its own protocols. `time` 0 is the list's default, -1 its longest; longer is cut down. 201 with `{"bans": [ban…, "existing": true for a renewed one], "closed": N}` (connections closed) |
| `DELETE /banlists/{ref}/bans/{id}` | lift one ban |
| `DELETE /banlists/{ref}/bans?address=A` | lift all of an address's bans (404 if it has none) |
These need `bans:read` or `bans:write` for the list
([permissions.md](permissions.md)); bans aren't on probation or in the
history, but are audited (`ban.add`, `ban.remove`). Refusals: `422
never_ban` (inside, the firewall's own, the caller's own, or in the
list's `never`), `422 list_full`, `409` for a disabled list.
### Probation
| | |
@@ -210,6 +275,7 @@ history, but are audited.
|---|---|
| `GET /history[?limit=&before=]` | changes, newest first |
| `GET /history/{id}` | one, with each object before and after |
| `GET /history/{id}/diff` | `{"diff"}`: what it did to the ruleset, as a unified diff with each hunk's chain (the configuration before and after it, got back from the current one, rendered by this version) |
| `POST /history/{id}/revert` | undo it; 409 if an object it touched has changed since, or it was undone already |
| `POST /import/nft` | `{"json": NFT_JSON, "text": "nft -a list ruleset output", "defines": "ruleset file", "table": "inet main", "zones": {"enp2s0": "lan"}, "replace": false}` ([import.md](import.md)); answers with `state` (the imported configuration), `notes` and `raw` too |
| `POST /takeover` | `{"tables": ["inet main"]}`: delete these tables (administrators); on probation over the network; 409 in shadow mode |
+148
View File
@@ -0,0 +1,148 @@
# Ban lists
A **ban list** is where a program bans addresses: Guard (Housecarl's
companion, which reads logs), fail2ban with Housecarl's action, or you
with `hc ban`. The program says only *which address*, and perhaps a
protocol and ports. The ban list says what that address is kept from.
So a web server that sees password guessing in its own logs can ban the
guesser at the router without knowing anything about the router.
## Ban lists
```sh
hc banlist add router --scope firewall # the firewall itself
hc banlist add web --scope hosts --host 10.0.10.57 --host 2001:db8:10::57
hc banlist add edge --scope everything --never trusted
```
| Scope | Banned addresses are kept from | Where the drops are |
|---|---|---|
| `firewall` | the firewall itself: its SSH, DNS, web UI | `input` |
| `hosts` | the list's hosts (addresses or sets), behind the firewall | `forward`, to those addresses |
| `everything` | the firewall and everything behind it | `input` and `forward` |
A `hosts` list covers port forwards to its hosts as well: the forward
chain sees a forwarded connection's translated address and port, the
host's own. A web server's Guard bans `tcp/22`, and that is right even
when the router forwards its port 2222 to it.
Other settings, with their defaults:
| Setting | Default | |
|---|---|---|
| `never` | none | addresses, prefixes or sets this list never bans |
| `default_time` (`--time`) | 1h | a ban's length when the program asks for none |
| `max_time` (`--max-time`) | 7d | the longest ban; longer ones are cut to it. `-1` (`none`): no limit, so bans may be permanent |
| `max_entries` | 10000 | the most bans the list holds; more are refused (`list_full`) |
| `group_v6` | 64 | the IPv6 prefix length banned for an address. One customer usually has a whole /64 and can move around in it; 128 bans the one address. IPv4 addresses are banned alone |
| `disabled` | no | the list's bans aren't in force (they are kept) |
A ban list can't be renamed: the tokens and programs that ban into it
know it by its name. Make a new one instead.
## Bans
```sh
hc ban edge 203.0.113.9 --time 1d --reason "port scan"
hc ban web 203.0.113.9 --tcp ssh --tcp 8443
hc bans # every list's
hc bans web
hc unban web 203.0.113.9 # all of the address's bans in web
hc unban web 17 # one, by ID
```
A ban is of **every port**, or of **a protocol's ports** (numbers or
service names, built-in or yours). A port without a protocol is banned
for TCP and UDP; a service name takes the service's protocols. Banning an
address again renews the ban: its new time counts from now.
Bans are dropped **ahead of every rule** in their chains, and ahead of
the built-in that lets established connections through, so a ban cuts a
connection that is already open. Housecarl also closes the banned
address's tracked connections the ban covers when it bans it, because
connections offloaded to the flowtable no longer pass the rules at all.
Replies to connections the inside opened to a banned address are
dropped too.
Bans are kept in the database, apart from the configuration (like a
dynamic set's entries): they aren't in the history, can't be undone,
aren't on probation, and don't go through changesets. They end by
themselves; every change, restart and reboot puts the active ones back
with the time they have left. Each ban and unban is in the audit log
(`ban.add`, `ban.remove`), with the program's source and reason.
### What is never banned
Whatever a list's `never` says, Housecarl refuses (`422 never_ban`) to
ban:
* the inside: addresses in networks of inside zones' interfaces, and
private addresses outside the external zones' networks;
* the firewall's own addresses, and its own prefixes (settings);
* the address of whoever asks (a program can't lock itself out);
* loopback, link-local, multicast and unspecified addresses.
A ban grouped to a prefix (IPv6) is refused when the prefix overlaps any
of these.
## Programs and tokens
Give each program a token for its list alone. A `bans` grant can name
ban lists, as a `sets` grant names sets:
```sh
hc user add guard-web --no-password --grant bans:write:web
hc token create web --user guard-web # as root on the firewall
```
With it, the program may ban into `web` and lift its bans, list them,
and read the list itself (`GET /banlists/web`: its grouping and times),
and nothing else. `viewer` has `bans:read`, `operator` `bans:write`.
## Guard
Guard reads log files and the systemd journal on any machine, and bans
the addresses that misbehave into a ban list (its own documentation).
Run it on the web server with a token for `web`, and on the router with
one for `router` (or the local socket).
## fail2ban
Housecarl ships an action for fail2ban,
`/usr/share/doc/housecarl/contrib/fail2ban/action.d/housecarl.conf`
(`contrib/` in the source). Copy it to `/etc/fail2ban/action.d/`, put
the token in `/etc/fail2ban/housecarl.token` (mode 0600), and use it in a
jail:
```ini
[sshd]
enabled = true
action = housecarl[list=web, url="https://router.lan:8070"]
```
The jail's `port` and `protocol` go with each ban: `port = ssh` bans only
`tcp/22`, fail2ban's default `0:65535` bans every port. A `bantime` of
`-1` is the list's longest ban. When fail2ban unbans an address Housecarl
already let go (its own time ran out first), nothing happens. Add
`curlopts = --cacert /etc/fail2ban/housecarl-ca.pem` for a private CA.
## Shadow mode
In shadow mode the ban rules count what they would drop
(`[would drop]`) and let it through, like every other rule, and banning
closes no connections.
## The ruleset
Each list has four sets, `ban_<id>_v4`, `ban_<id>_v6` (every port) and
`ban_<id>_v4_svc`, `ban_<id>_v6_svc` (address, protocol and port), all
with timeouts:
```
ip saddr @ban_3_v4 counter drop comment "hc:ban3"
ip saddr . meta l4proto . th dport @ban_3_v4_svc counter drop comment "hc:ban3"
ip6 saddr and ffff:ffff:ffff:ffff:: @ban_3_v6 counter drop comment "hc:ban3"
```
Set names starting with `ban_` are therefore Housecarl's own
(`set_name_reserved`).
+20 -3
View File
@@ -60,7 +60,8 @@ applied and saved (history 12; undo with 'hc history undo 12')
```
`-n` shows what would change without changing it, with the diff of the
ruleset; nftables checks it too.
ruleset (each `@@` line names the chain the lines below it are in);
nftables checks it too.
## Changesets
@@ -153,8 +154,10 @@ by name.
| `zone delete ZONE` | refused while rules use it |
| `set list`, `set show SET` | |
| `set add NAME [ENTRY...] [--dynamic] [--timeout 24h] [--comment C]` | |
| `set update SET [--entry E]... [--add E] [--remove E] [--timeout] [--name] [--comment]` | `--entry` replaces the entries |
| `set add NAME --jarl NAME[@VIEW]... --within ZONE\|NETWORK...` | a set from Jarl: the names' addresses, only those within the zones and networks ([jarl.md](jarl.md#sets-from-jarl)) |
| `set update SET [--entry E]... [--add E] [--remove E] [--timeout] [--jarl N]... [--within W]... [--name] [--comment]` | `--entry` replaces the entries, `--jarl` a Jarl set's names, `--within` where its addresses may be |
| `set entries SET`, `set entries add SET ADDRESS [--timeout 1h]`, `set entries remove SET ADDRESS` | a dynamic set's entries |
| `set jarl SET [--refresh]` | a set from Jarl: each name's addresses in Jarl (CNAMEs followed), used or why not, and when Jarl answered; `--refresh` reads Jarl first |
| `set delete SET` | |
| `service list`, `service show SERVICE` | with the built-in ones |
| `service add NAME ENTRY... [--comment C]` | entries `tcp/22`, `udp/1000-2000`, `icmp/echo-request`, `icmpv6/packet-too-big` |
@@ -204,6 +207,19 @@ accept|drop|reject|log`, `--reject-with tcp-reset`, `--log`,
See [nat.md](nat.md).
### banlist, ban
| Command | |
|---|---|
| `banlist list`, `banlist show BANLIST` | ban lists, with their bans' count |
| `banlist add NAME --scope firewall\|hosts\|everything [--host ADDR\|SET]... [--never ADDR\|SET]... [--time 1h] [--max-time 7d\|none] [--max-entries N] [--group-v6 64] [--disabled] [--comment C]` | |
| `banlist update BANLIST [flags]`, `banlist delete BANLIST` | `--host` and `--never` replace the lists; no renaming |
| `bans [BANLIST]` | the active bans |
| `ban BANLIST ADDRESS [--tcp PORT\|SERVICE]... [--udp …]... [--time 1h\|none] [--reason R]` | ban every port, or some; again renews |
| `unban BANLIST ADDRESS\|ID` | lift one ban, or all of an address's |
See [bans.md](bans.md).
### settings
| Command | |
@@ -218,7 +234,8 @@ See [nat.md](nat.md).
| `confirm` | keep the changes on probation; the ruleset is saved for the boot |
| `rollback` | undo the changes on probation now |
| `history list [--limit N]` | changes, newest first |
| `history show ID` | each object before and after |
| `history show ID` | each object before and after (a changed setting by its own fields: `builtins.drop_invalid: false -> true`) |
| `history diff ID` | what the change did to the ruleset |
| `history undo ID` | apply the change backwards (refused if something it touched has changed since) |
| `import [FILE\|-] [--text FILE] [--defines FILE] [--table T] [--zone IFACE=NAME]... [--replace]` | [import.md](import.md) |
| `takeover TABLE... [-y]` | delete tables Housecarl's ruleset replaces (administrators); not in shadow mode |
+2 -1
View File
@@ -13,7 +13,8 @@ reads the file when it starts.
| `listen` | `["127.0.0.1:8070"]` | addresses of the web UI and API (`host:port`). Keep them on localhost or a management network, and write an input rule that lets you reach them |
| `tls.cert_file`, `tls.key_file` | none | serve HTTPS with this certificate; housecarld rereads it when the file changes, or on `systemctl reload housecarl` |
| `tls.acme.*` | off | get the certificate from Let's Encrypt, with Jarl publishing the challenge ([certificates.md](certificates.md#settings)) |
| `jarl.server`, `jarl.token_file` (or `jarl.token`), `jarl.ca_cert`, `jarl.insecure` | none | the Jarl server for the certificate's challenge, and its token ([jarl.md](jarl.md)) |
| `jarl.server`, `jarl.token_file` (or `jarl.token`), `jarl.ca_cert`, `jarl.insecure` | none | the Jarl server for the certificate's challenge and the sets from Jarl, and its token ([jarl.md](jarl.md)) |
| `jarl.refresh` | `1m` | how often the sets from Jarl read their names (10s or more; [jarl.md](jarl.md#when-the-addresses-change)) |
| `ruleset` | `/var/lib/housecarl/ruleset.nft` | the confirmed ruleset, which `housecarl-ruleset.service` loads at boot; its fallback is saved beside it as `ruleset-fallback.nft` ([operations.md](operations.md#if-the-ruleset-fails-to-load-at-boot)). Change both paths in the unit too |
| `trusted_proxies` | `[]` | addresses or prefixes of reverse proxies whose `X-Forwarded-For` and `X-Forwarded-Proto` are believed |
| `socket` | `/run/housecarl/housecarl.sock` | the local API socket; `""` turns it off |
+11 -3
View File
@@ -94,7 +94,8 @@ table inet housecarl { ... }
Two things survive the replacement: the **entries of dynamic sets**
(read before, written back with their remaining timeouts, in the same
script), and the **counters** (housecarld adds the old totals to the
new, so hits don't reset with every change).
new, so hits don't reset with every change). Sets from Jarl are written
with the addresses last read from Jarl.
Every rule carries a comment `hc:<mark>`, such as `hc:r12` for rule 12,
`hc:f3` for port forward 3, `hc:b-established` for a built-in. Counters
@@ -199,8 +200,15 @@ Housecarl's replaces it ([import.md](import.md)).
may only change that set (`sets:write:guests`). Addresses, not MAC
addresses: matching by MAC only works on the guests' own network
segment.
* **Names instead of addresses**: Steward knows every reservation's
address and Jarl every name, so rules could name a device or a subnet.
* **Names instead of addresses**: a set can take its addresses from Jarl
([jarl.md](jarl.md#sets-from-jarl)). housecarld reads the names' A and
AAAA records through Jarl's API (CNAMEs followed, one view per name)
and keeps the addresses that lie within the zones and networks the set
allows; Jarl never calls Housecarl. What was last read is kept (in the
database too) while Jarl can't be reached. A change of the addresses
applies the ruleset again without a history entry, as dynamic sets'
entries aren't in it either. Names Steward makes from leases must not
be used: a client chooses its own hostname.
* **The web UI's certificate** comes through Jarl, as Steward's does
([certificates.md](certificates.md)).
+11 -1
View File
@@ -20,12 +20,14 @@ internal/
server/ the HTTP API and the web UI, authentication, the local socket, probation, the log ring
api/, client/ JSON types and the Go client shared by server and CLI
auth/ passwords (argon2id), tokens, sessions, grants and roles
certs/, jarl/ the web UI's certificate through Jarl (docs/certificates.md)
certs/, jarl/ the web UI's certificate through Jarl (docs/certificates.md); jarl/ is the Jarl API client
jarlsets/ sets from Jarl: reads the names, follows CNAMEs, keeps what was read, within (docs/jarl.md)
config/ housecarld's YAML configuration
version/ what a build reports
web/static/ the web UI: plain ES modules, no build step, embedded with go:embed
deploy/ systemd units, sysusers.d and tmpfiles.d files, Debian package scripts
scripts/e2e.sh end-to-end test with real traffic through a router container
scripts/fakejarl/ the bit of Jarl's API sets from Jarl read, for the e2e test
scripts/mkdeb.sh the Debian packages (make deb)
scripts/logo.py the web UI's logo from artwork/logo.svg
scripts/banner.py the README's banner from the web UI's logo and wordmark
@@ -135,6 +137,14 @@ As for Jarl and Steward:
3. Run `make test && make e2e`.
4. Commit as "Release X", tag it `git tag -a vX -m "Housecarl X"`, and push.
A fix to a release candidate may go out as a build of it rather than a
new candidate: `0.2.0-rc.3+git<count>-<commit>`, where `<commit>` is the
short ID of the last commit before "Release X" and `<count>` is `git
rev-list --count` of it. The count orders the builds (apt compares it as a
number; the ID alone would order them at random); each sorts after
`0.2.0-rc.3` and before `0.2.0-rc.4`. Debian writes it
`0.2.0~rc.3+git<count>~<commit>`.
The tag starts `.gitea/workflows/release.yml`, which checks the tag
against `VERSION` and the CHANGELOG, runs the tests, builds the tarballs,
the CLI for each OS and the `.deb`s, and publishes them
+219 -7
View File
@@ -1,24 +1,44 @@
# Connecting Housecarl to Jarl
Housecarl uses [Jarl](https://gitea.adbraun.lol/adam/jarl), the BIND
manager, for its DNS records. In this version that means one thing: the
DNS-01 challenge for the web UI's certificate ([certificates.md](certificates.md)).
Naming hosts in rules by their DNS names may use the same connection later.
manager, for two things:
* the DNS-01 challenge for the web UI's certificate
([certificates.md](certificates.md)): Housecarl writes a TXT record;
* **sets from Jarl** ([below](#sets-from-jarl)): sets whose addresses are
those of names in Jarl, such as `nas.adbraun.lol`, kept up to date.
Housecarl only reads.
Housecarl talks to Jarl's REST API with an API token. Give it a **user of its
own**, with **only the permission it needs**.
own**, with **only the permission it needs**. Jarl never calls Housecarl.
## 1. A user for Housecarl, in Jarl
On the Jarl server, make a user without a password. It can't log in to
Jarl's web UI, only use its tokens. Grant it the challenge record of
Housecarl's own name, and nothing else:
Jarl's web UI, only use its tokens.
For **sets from Jarl**, grant it reading the records of the zones and views
your names are in:
```sh
jarl user create housecarl --no-password --comment "Housecarl on fw01" \
--grant "records:read zones=adbraun.lol views=internal"
```
Include every zone a name's CNAMEs lead to as well (`zones=adbraun.lol,wg.adbraun.lol`).
Reading is all it needs: it can't change anything in Jarl.
For the **certificate**, grant it the challenge record of Housecarl's own
name, and nothing else:
```sh
jarl user create housecarl --no-password --comment "Housecarl on fw01" \
--grant "records:write zones=adbraun.lol names=_acme-challenge.fw.adbraun.lol types=TXT"
```
For both, give both grants (`--grant` twice). `jarl user update housecarl
--grant ... --grant ...` replaces a user's grants with the ones given.
Use your zone for `adbraun.lol`, and the name Housecarl's web UI is reached
by for `fw.adbraun.lol` (`tls.acme.domain`). The grant lets the token add
and remove TXT records at exactly `_acme-challenge.fw.adbraun.lol`:
@@ -63,7 +83,9 @@ around more readily.
You can give the token an expiry (`--expires 365d`). Rotate it in Jarl with
`jarl token rotate ID --grace 1d`, then put the new secret in the file and
restart housecarld within the grace period. Since the certificate is renewed
every couple of months, nothing breaks while you do.
every couple of months, nothing breaks while you do. Sets from Jarl keep
their addresses while the token doesn't work (they warn, see
[below](#when-jarl-cant-be-reached)), so they don't break either.
## 3. Housecarl's settings
@@ -74,6 +96,7 @@ jarl:
server: https://dns.adbraun.lol:8053
token_file: /etc/housecarl/jarl.token
# ca_cert: /etc/housecarl/jarl-ca.pem # if Jarl's certificate isn't from a public CA
# refresh: 1m # how often sets from Jarl read their names
```
Then restart housecarld (`systemctl restart housecarl`). Housecarl needs to
@@ -83,3 +106,192 @@ address Housecarl can reach, ideally over HTTPS.
`hc cert check` ([certificates.md](certificates.md#checking)) tests
the connection, the token's permission, and that the record reaches the
internet.
## Sets from Jarl
A set can take its addresses from Jarl instead of listing them: you name
DNS names, and the set holds their addresses. Rules use it like any other
set. When a record changes in Jarl, the set follows within a minute,
without a change to Housecarl's configuration.
```sh
hc set add nas --jarl nas.adbraun.lol@internal --within lan
hc rule add forward --in wg --to nas --service smb
hc set jarl nas # what it holds, name by name
```
In the web UI: Sets → Add set, kind "from Jarl". The **Addresses…**
button shows what the set holds.
A set from Jarl has:
* **names**: one or more names, each `NAME@VIEW`, or `NAME` when only one
of Jarl's views has it ([Views](#views));
* **within**: where its addresses may be, one or more zones or networks
([Within](#within)). It is required.
It has no entries of its own, and isn't dynamic. It can be renamed (it goes
by its names in Jarl, not by its own).
### What a name gives
Housecarl reads the name's records as named would answer them in the
view:
* **Every A and AAAA record** of the name. A name with three addresses
puts all three in the set, IPv4 and IPv6 alike; a rule using the set
matches each family with its own addresses.
* **CNAMEs are followed**: `files.adbraun.lol CNAME nas.adbraun.lol.` gives
nas's addresses. Chains are followed up to 8 CNAMEs, from zone to zone
(a sub-zone such as `wg.adbraun.lol` too), but always in the **same
view**, and only into zones Jarl holds. `hc set jarl` shows the chain:
`files.adbraun.lol → nas.adbraun.lol`.
* **Disabled records** are left out, as named leaves them out.
A name gives **no addresses**, and a warning (`jarl_name`) says why, when:
* it has no A, AAAA or CNAME record (only an MX, say, or nothing);
* only a **wildcard** (`*.adbraun.lol`) would answer for it: wildcards
aren't followed, so name a record of its own;
* a **CNAME leads out of Jarl** (`host.duckdns.org`), into a zone Jarl
doesn't hold, or to a name with no address;
* the CNAMEs **loop**, or are more than 8;
* it is **below a delegation**: NS records in the zone hand that part to
other servers, so Jarl doesn't hold its records;
* its zone is a **secondary, mirror, stub or forward zone** (Jarl doesn't
hold their records), or **disabled** in Jarl;
* no zone the token can see holds it.
A **reverse zone** isn't involved: Housecarl reads the A and AAAA records
only, never PTRs.
An **in-view zone** (one view serving another's zone) is followed to the
zone it serves: `srv.adbraun.lol@vps` reads the internal view's records if
vps's `adbraun.lol` is an in-view of internal's.
### Views
The same name can have different addresses in different views: Jarl's
`internal` view may give `nas.adbraun.lol` its private address, `external`
a public one. A set reads **one view per name**, written `NAME@VIEW`:
* **Name the view your devices see.** The firewall matches the addresses
packets carry, and for devices inside that is the internal view's.
* **Without a view**, the name is read from the only view that has a zone
for it. If zones in **more than one view** hold it, the name gives
nothing until you name the view (`jarl_name`: "is in Jarl's views
external, internal: name the view"). This counts every view whose zones
cover the name, even one where the name has no record: a record added
there later would otherwise change which view Housecarl reads.
* **Linked zones** make no difference: a shared record has the same
address in every view, and a view's own records are why the view is
named.
* **Without views** in Jarl, write the name alone.
* A set can mix views: `--jarl nas.adbraun.lol@internal --jarl
host.adbraun.lol@vps`.
A wrong view can't widen the set: an address from it that isn't
[within](#within) is left out.
### Within
`--within` says where the set's addresses may be. Each entry is:
* **a zone**: the networks connected to its interfaces (the addresses on
them, as `hc status` lists them), IPv4 and IPv6;
* **a network, address or range**: `10.10.0.0/24`, `2001:db8:10::/64`,
`192.0.2.7`.
An address from Jarl that isn't within any of them is **left out** of the
set, and a warning names it (`jarl_outside`: "nas.adbraun.lol has address
192.0.2.9, which is left out: not within lan"). The set's other addresses
stay. So a record changed in Jarl, by mistake or by someone who shouldn't,
can only point the set at addresses you allowed.
Zones grow by themselves: when you add a VLAN to the `lan` zone, or give
an interface another network, its addresses count at once. A new zone
counts only once you add it to `within`. If a zone's interfaces have no
address for a while (at boot, a link down), the networks it had last count.
Some addresses never count, whatever `within` says: unspecified (`0.0.0.0`,
`::`), loopback and multicast ones.
`within 0.0.0.0/0` or `::/0` lets any address in, and gives a warning
(`set_jarl_anywhere`): then whoever can change the names in Jarl decides
what the set holds. Name a zone or a network instead. For an address
outside your networks, such as a VPS's public address, name that address
or its network.
### When the addresses change
Housecarl reads the names every `jarl.refresh` (a minute), right after a
change to Housecarl's configuration (a new set or name), when an interface
changes, and when you ask: `hc set jarl NAME --refresh`, or **Read Jarl
now** in the web UI.
When a set's addresses change, housecarld puts the ruleset in force again
with the new ones. That isn't a change of the configuration: there is no
history entry and no probation. It is recorded in the audit log
(`set.jarl`, "added 10.10.0.22; removed 10.10.0.20") and in the journal.
A new set holds nothing for the moment until its names are read, usually
less than a second.
**Open connections aren't closed** when an address leaves a set, as with
any change to a set: the connection's packets are let through as part of an
established connection (or by the flowtable). Close them with `hc
connections close` if they must stop at once.
### When Jarl can't be reached
A set **keeps the addresses it last had** while Jarl can't be read: Jarl is
down, the network to it is, or the token stopped working. Nothing is
emptied, so rules keep working. A warning says so (`jarl_unreachable`), on
the status page and in `hc status`, and `hc set jarl` shows when Jarl last
answered.
What was last read is saved in the database. After a restart or a reboot,
the sets get those addresses at once, before Jarl is read again, and the
boot ruleset ([operations.md](operations.md)) has them too.
When Jarl **answers** that a record is gone, its addresses go. Only a
failure to read keeps them.
Without `jarl.server` in `housecarl.yaml`, sets from Jarl keep what they
had (nothing if Jarl was never read) and warn (`jarl_not_configured`).
### Who decides what the set holds
A set from Jarl lets **whoever can change those records in Jarl** decide
which addresses a rule matches. Keep that in mind for rules that accept:
* `within` limits it to networks you chose;
* Housecarl's token only reads ([step 1](#1-a-user-for-housecarl-in-jarl));
* give the records' zones only to people you'd let change the firewall,
and check Jarl's audit log for who changed what.
**Don't use names that clients choose for themselves.** A name a DHCP
server makes from the client's own hostname belongs to whoever asks for
it: any guest could call itself `nas`. Name records that you made by hand,
or that come from a DHCP reservation.
### Seeing what a set holds
```
$ hc set jarl nas
Set: nas
Within: lan
Holds: 10.10.0.20, 10.10.0.21
Jarl read: 2026-10-23 14:02:11
NAME VIEW ADDRESS USED
files.adbraun.lol → nas.adbraun.lol internal 10.10.0.20 yes
files.adbraun.lol → nas.adbraun.lol internal 10.10.0.21 yes
files.adbraun.lol → nas.adbraun.lol internal 2001:db8:10::20 no: not within lan
```
The web UI's **Addresses…** button on the Sets page shows the same, and the
status page lists the warnings: names that give nothing (`jarl_name`),
addresses left out (`jarl_outside`), Jarl not read (`jarl_unreachable`).
The API is `GET /sets/{name}/jarl` ([api.md](api.md#sets-from-jarl)).
+23 -4
View File
@@ -83,7 +83,10 @@ sudo nft -c -f /var/lib/housecarl/ruleset.nft # check it
In shadow mode both files are shadow mode's, and the fallback filters
nothing ([shadow.md](shadow.md#what-works-differently)).
Dynamic sets start empty after a reboot. The saved ruleset has no
Dynamic sets start empty after a reboot. Sets from Jarl have the
addresses they last had, in the saved ruleset and again when housecarld
starts, whether Jarl is reachable yet or not ([jarl.md](jarl.md#when-jarl-cant-be-reached)).
The saved ruleset has no
flowtable, since its devices may not exist yet: housecarld adds it when
it starts, and whenever a flowtable device appears
([flowtable.md](flowtable.md#devices-that-come-and-go)).
@@ -164,9 +167,9 @@ start again. A disabled rule keeps its count for when it is enabled.
resets; `hc rule reset-quota ID` starts one again (`--address` one device's,
`--give 5gbytes` a top-up until the next reset). housecarld resets
quotas at midnight when their period ends (late, if it was stopped then),
and saves their usage every five minutes and when it stops; after a
reboot it puts it back. A reboot without a clean stop loses at most the
last five minutes' count.
and saves their usage every five minutes, after every change and when
it stops; after a reboot it puts it back. A reboot without a clean stop
loses at most the last five minutes' count.
## The log
@@ -309,6 +312,22 @@ down, the fallback and a missing ruleset. Reach it from where the input
rules let the monitor in (a rule for the Checkmk server's address and
port 8070).
## Failed logins in the journal
A failed login to the web UI or the API, and a refused API token, are
written to the journal as well as the audit log, with the address first:
```
level=WARN msg="login failed" ip=203.0.113.9 user=admin reason="wrong username or password"
level=WARN msg="token refused" ip=203.0.113.9 reason="wrong token secret"
```
Guard's `housecarl` preset, or a fail2ban filter, can ban such addresses
([bans.md](bans.md)). The address comes before the user name, which a
caller chooses, so a pattern anchored on `msg="login failed" ip=` can't
be fooled by a name. (With `log.format: json` they are JSON objects with
the same fields, in the same order.)
## Checks
`housecarld check` validates the configuration in the database against
+17 -8
View File
@@ -8,7 +8,8 @@ else gets **grants**, `resource:access`:
| Resource | read | write |
|---|---|---|
| `config` | settings, zones, sets, services, rules, NAT, the ruleset, history, interfaces, probation, changesets (the web UI shows the forms locked) | change them, stage and apply changesets (your own, or anyone's for administrators), undo, import, confirm and roll back; close connections; start and stop traces |
| `sets` | dynamic sets' entries | add and remove entries |
| `sets` | dynamic sets' entries, what sets from Jarl hold | add and remove entries |
| `bans` | ban lists' bans, and the lists the grant names | ban and unban ([bans.md](bans.md)) |
| `log` | hit counters, the log, connections, traces | – |
| `audit` | the audit log | – |
| `*` | all of the above | |
@@ -20,6 +21,10 @@ program, such as Steward's captive portal, which should be able to let a
client in and nothing more. A dynamic set's entries need a `sets` grant
even for those with `config:write`.
A `bans` grant names ban lists the same way: `bans:write:web` may ban
into the list `web` (and read it) and nothing else, the grant for Guard
or fail2ban on the web server.
Only administrators may delete other tables (`hc takeover`), and manage
users and the certificate (`hc cert`, Server → Certificate).
@@ -27,8 +32,8 @@ users and the certificate (`hc cert`, Server → Certificate).
| Role | Grants |
|---|---|
| `viewer` | `config:read sets:read log:read` |
| `operator` | `config:write sets:write log:read audit:read` |
| `viewer` | `config:read sets:read bans:read log:read` |
| `operator` | `config:write sets:write bans:write log:read audit:read` |
| `log` | `log:read`, e.g. for a monitoring script |
```sh
@@ -49,9 +54,13 @@ hc token create portal --user portal --grant sets:write:guests --expires 365d
```
The secret is shown once; only its hash is stored. A restricted token can
only create tokens with a subset of its own grants. The web UI only offers
roles within what your account may do; a token with more would get no
more than its user anyway.
only create tokens with a subset of its own grants.
In the web UI, Account → New token makes one of your own; an
administrator makes one for another user with New token on that user's
row under Users (`--user` in the CLI, `POST /users/{id}/tokens` in the
API). The dialog only offers roles within what the token's user may do; a
token with more would get no more than its user anyway.
**Rotating** gives a token a new secret, keeping its ID, name and
permissions, so only the programs that use it need updating:
@@ -82,6 +91,6 @@ on probation like any other.
Logins (and failed ones), users, tokens, every configuration change (with
its history entry), changesets created, applied and discarded,
confirmations and rollbacks, imports, takeovers and
dynamic set entries are recorded with who, from where and how: `session`,
confirmations and rollbacks, imports, takeovers,
dynamic set entries and bans are recorded with who, from where and how: `session`,
`token` or `socket`.
+70 -2
View File
@@ -83,6 +83,19 @@ change to the ruleset, and a restart of housecarld; a reboot empties
them. A token can be limited to one set ([permissions.md](permissions.md)):
that is how another program (Steward's captive portal) lets a client in.
A set **from Jarl** holds the addresses of names in Jarl, the BIND
manager: every A and AAAA record of each name, CNAMEs followed, read again
every minute. Only addresses within the zones and networks it names are
used, so a changed record can't open the firewall to anything else:
```sh
hc set add nas --jarl nas.example.com@internal --within lan
hc set jarl nas # what it holds, and why
```
Views, CNAMEs, what happens when Jarl can't be reached and the rest are in
[jarl.md](jarl.md#sets-from-jarl).
## Services
A service names ports of a protocol (`tcp`, `udp`, `sctp`) and ICMP
@@ -165,6 +178,40 @@ hc rule add forward --in wan --status dnat
hc rule add forward --in wan -a drop
```
### Matching everything but a few
A field matches **any of** its entries, or with `--not-…` ("any but" in
the web UI) anything else; it can't do both. "The LAN except the NAS" is
still one rule: the zone says which network, and the addresses say which
of it to leave out, because zones and addresses are different fields and
a rule needs all of them to match.
| What you want | One rule |
|---|---|
| the LAN, except the NAS | `--in lan --from nas --not-from` |
| everyone on guest, except one device, has a quota | `--in guest --from tv --not-from --quota 10gbytes,over,daily` |
| the internet, except one host | `--out wan --to blocked --not-to` |
```sh
# Each device on the LAN: 50 GB a month, but not the NAS (its backups)
hc rule add forward --in lan --out wan --from nas --not-from --quota 50gbytes,over,per-source,monthly -a drop
```
Leaving part of an address range out ("10.0.0.0/8 except 10.0.5.0/24")
when it isn't a zone takes one of:
* **Two rules, the exception first**, when the exception gets the
opposite action: `--from 10.0.5.0/24 -a drop` above `--from
10.0.0.0/8`. There is no action that skips a rule and goes on, so this
doesn't work when the exception should just fall through to later
rules.
* **Ranges around the hole**: `--from 10.0.0.0-10.0.4.255 --from
10.0.6.0-10.255.255.255`.
* **A [raw rule](#raw-rules)**: `ip saddr 10.0.0.0/8 ip saddr != 10.0.5.0/24 accept`.
There's no need to leave a zone out of other zones: an interface belongs
to one zone, so "any of lan, guest" already leaves out everything else.
### Bandwidth and quotas
A **byte rate** limits bandwidth: `--limit 20mbytes/second,over,per-source
@@ -209,12 +256,26 @@ Statistics**.
nftables counts a quota but never resets it, and forgets it when its
table is replaced: Housecarl carries what each quota used across every
change and restart, saves it every five minutes (and when housecarld
stops) for after a reboot, and resets quotas when their period ends.
change and restart, saves it every five minutes (and after each change,
and when housecarld stops) for after a reboot, and resets quotas when their period ends.
A per-address quota counts an address: a device that gets a new one
starts again. With Steward's reservations IPv4 addresses stay; IPv6 is
grouped by /64 (below).
Changing the amount, `over` or the resets keeps what was used. Changing
how the quota counts carries what can be carried:
| Change | What was used |
|---|---|
| per address to one total | the addresses' sum |
| to wider groups (IPv6 /64 to /56) | added up per new group |
| a total to per address | starts again: a total can't be split |
| per source to per destination (or back) | starts again |
| to finer groups (/64 to /128) | starts again: a group can't be split |
A change that starts a quota again says so (warning `quota_restarts`),
when it is staged for review and in `hc` (also with `-n`).
**Rules with a quota or a byte rate count every packet.** The built-in
that accepts established connections comes before your rules, so a rule
there only sees each connection's first packet. Rules that count traffic
@@ -254,6 +315,13 @@ In shadow mode ([shadow.md](shadow.md)) a raw rule's final `drop` or
(NAT, `meta mark set`, `jump`, `notrack`, a verdict map…) is left out,
with a warning (`shadow_raw_left_out`).
## Ban lists
Programs such as Guard and fail2ban ban addresses into ban lists, and
those bans come first of all in the input and forward chains (after
loopback in input), before the built-ins and every rule. See
[bans.md](bans.md).
## Built-ins
Housecarl adds the rules every firewall needs before yours. Each can be
+3
View File
@@ -116,6 +116,9 @@ firewall's NAT still does the work.
* **No takeover, no closing connections.** `hc takeover` (deleting your
firewall's tables) and `hc connections close` are refused (409,
`hc` exits with 5): both would change your traffic.
* **Bans** ([bans.md](bans.md)) are kept and their rules count what they
would drop (`[would drop]`), but nothing is dropped, and banning closes
no connections.
* **The boot.** `housecarl-ruleset.service` loads the shadow ruleset at
boot. Its fallback ([operations.md](operations.md#if-the-ruleset-fails-to-load-at-boot))
is an empty table that filters nothing, since your firewall protects the
+26 -2
View File
@@ -51,11 +51,16 @@ are unique per kind, ignoring case.
| Code | | When |
|---|---|---|
| `set_name_reserved` | error | a set name starting with `meter_`, `conn_`, `quota_` or `stats_`: Housecarl's own sets (rate limits, quotas, statistics) use those |
| `set_name_reserved` | error | a set name starting with `meter_`, `conn_`, `quota_`, `stats_` or `ban_`: Housecarl's own sets (rate limits, quotas, statistics, ban lists) use those |
| `set_entry` | error | an entry that isn't an address, prefix (without host bits) or range |
| `set_empty` | warning | a set (not dynamic) with no entries: rules using it match nothing |
| `set_empty` | warning | a set (not dynamic, not from Jarl) with no entries: rules using it match nothing |
| `set_dynamic_entries` | error | configured entries on a dynamic set: its entries come at run time |
| `set_timeout` | error | a timeout on a set that isn't dynamic, or under a second |
| `set_jarl_entries` | error | configured entries on a set from Jarl: its addresses are its names' ([jarl.md](jarl.md#sets-from-jarl)) |
| `set_jarl_dynamic` | error | a set both dynamic and from Jarl |
| `set_jarl_names` | error | a set from Jarl without names, with one that isn't a DNS name (a wildcard, say) or a bad view, or with a name twice |
| `set_jarl_within` | error | a set from Jarl without `within`, or with an entry there that is neither a zone nor an address, network or range |
| `set_jarl_anywhere` | warning | a set from Jarl `within 0.0.0.0/0` or `::/0`: any address may come in, so whoever changes the names in Jarl decides what it holds |
| `service_empty` | error | a service with no entries |
| `service_entry` | error | an entry that isn't `PROTO/PORTS` (`tcp`, `udp`, `sctp`; a port or range) or `icmp/TYPE`, `icmpv6/TYPE` with a type nftables knows |
@@ -97,6 +102,7 @@ These apply to rules, source NAT and redirects.
| `rule_group` | error | a grouping prefix outside 1-32 (IPv4) or 1-128 (IPv6) |
| `rule_traffic_accept` | error | a rule with a quota or a byte rate that accepts: it is checked before established connections are accepted, so it can only drop, reject or log |
| `rule_every_packet_accept` | error | a rule checked against every packet (`every_packet`) that accepts: it comes before established connections are accepted, so it can only drop, reject or log |
| `quota_restarts` | warning | a change makes a rule count its quota so that what it used can't carry over (a total to per address, per source to per destination, finer groups): it starts again. Only on the change that does it |
| `flowtable_traffic` | warning | a forward rule checked against every packet (a quota, a byte rate, `every_packet`) while the flowtable is on: it only sees each connection's first packets |
| `reject_with` | error | `tcp-reset` on a rule that matches more than TCP, or with an action other than reject |
| `log_prefix` | error | a log prefix over 100 characters |
@@ -133,7 +139,25 @@ configuration: the dashboard and `hc status` show them.
| `conntrack_full` | warning, *host* | the connection tracking table is over 80% full; once it is full, the kernel drops new connections. Raise `net.netfilter.nf_conntrack_max`, or find what opens so many ([stats.md](stats.md#warnings)) |
| `takeover_pending` | warning, *host* | the table you imported from is still in force beside Housecarl's, so a packet must get past both; `hc takeover TABLE` deletes it once Housecarl's rules are right |
| `shadow_unprotected` | warning, *host* | shadow mode, and no other table has a filter chain on the input hook: Housecarl's ruleset blocks nothing, and nothing else protects the machine. Load your firewall, or leave shadow mode ([shadow.md](shadow.md)) |
| `jarl_name` | warning, *host* | a set from Jarl has a name that gives no addresses, and why: no A, AAAA or CNAME record, a wildcard only, a CNAME out of Jarl's zones or in a loop, below a delegation, a zone Jarl doesn't hold the records of, or in more than one view without one named ([jarl.md](jarl.md#what-a-name-gives)) |
| `jarl_outside` | warning, *host* | an address a name has in Jarl is left out of its set: it isn't within the set's zones and networks ([jarl.md](jarl.md#within)) |
| `jarl_unreachable` | warning, *host* | reading Jarl failed (unreachable, the token refused…), for every name or one; the sets keep the addresses they had ([jarl.md](jarl.md#when-jarl-cant-be-reached)) |
| `jarl_not_configured` | warning, *host* | sets from Jarl, but no `jarl.server` in `housecarl.yaml`: they keep the addresses they last had |
| `shadow_raw_left_out` | warning, *host* | shadow mode leaves this raw rule out: it does more to packets than accept, drop or reject (NAT, setting a mark or field, a jump, a verdict map…). Its hits aren't counted, and the rules after it see what it would have taken ([shadow.md](shadow.md#what-changes-in-the-ruleset)) |
In shadow mode `nftables_service` and `takeover_pending` aren't raised:
another firewall beside Housecarl is the point there.
## Ban lists
| Code | | When |
|---|---|---|
| `banlist_name` | error | not a valid name (as for zones and sets) |
| `banlist_name_taken` | error | two ban lists with one name |
| `banlist_scope` | error | a scope other than `firewall`, `hosts` or `everything` |
| `banlist_hosts` | error | scope `hosts` without hosts, or hosts with another scope |
| `banlist_address` | error | a host or never entry that isn't an address, prefix or range |
| `unknown_set` | error | a host or never entry naming a set that doesn't exist |
| `banlist_time` | error | negative times (but `max_time` -1), or a default longer than the longest |
| `banlist_entries` | error | `max_entries` below 0 or above 1000000 |
| `banlist_group` | error | `group_v6` outside 16 to 128 |
+72
View File
@@ -532,3 +532,75 @@ type MatrixException struct {
// Marks are the counters behind Hits ("r12", "b-ping", "f3").
Marks []string `json:"-"`
}
// BanInput asks a ban list to ban an address (POST
// /banlists/{list}/bans).
type BanInput struct {
Address string `json:"address"`
// Protocol (tcp, udp) and Ports limit the ban; with neither, every
// port. Ports are numbers or service names, in a list or a string
// ("22", "ssh", "http,https"); "all", "0:65535" or "1-65535" mean
// every port. Ports without a protocol take the service's protocols,
// or for a number tcp and udp.
Protocol string `json:"protocol,omitempty"`
Ports json.RawMessage `json:"ports,omitempty"`
// Time is the ban's length in seconds: 0 the list's default, -1 its
// longest (or for ever). Longer than the list allows is cut down.
Time int `json:"time,omitempty"`
Source string `json:"source,omitempty"`
Reason string `json:"reason,omitempty"`
}
// BanResult is the answer to a ban: the bans made or renewed (one per
// protocol and port), Existing for each already in force.
type BanResult struct {
Bans []BanMade `json:"bans"`
// Closed counts the banned address's connections closed.
Closed int `json:"closed"`
}
// BanMade is a ban made, or renewed (Existing).
type BanMade struct {
model.Ban
Existing bool `json:"existing,omitempty"`
}
// JarlSet is what a Jarl set holds and why (GET /sets/{ref}/jarl).
type JarlSet struct {
Set string `json:"set"`
// Configured is whether housecarl.yaml names a Jarl server.
Configured bool `json:"configured"`
// Read is when housecarld last read Jarl's zones; Error why the last
// try failed (the set keeps the addresses it had).
Read *time.Time `json:"read,omitempty"`
Error string `json:"error,omitempty"`
// Entries are the addresses the set holds now.
Entries []string `json:"entries"`
Names []JarlName `json:"names"`
Within []string `json:"within"`
}
// JarlName is one of a Jarl set's names: what Jarl gave for it.
type JarlName struct {
Name string `json:"name"`
View string `json:"view,omitempty"` // as configured ("": the only view that has it)
// Found is the view it was found in.
Found string `json:"found_view,omitempty"`
// Chain are the CNAME targets followed, in order.
Chain []string `json:"chain,omitempty"`
Addresses []JarlAddress `json:"addresses"`
// Problem is why Jarl's records give it no addresses.
Problem string `json:"problem,omitempty"`
// Checked is when Jarl last answered for it; Error why the last try
// failed (its addresses are those of Checked).
Checked *time.Time `json:"checked,omitempty"`
Error string `json:"error,omitempty"`
}
// JarlAddress is an address a name has in Jarl: Used if the set holds
// it, else why not.
type JarlAddress struct {
Address string `json:"address"`
Used bool `json:"used"`
Why string `json:"why,omitempty"`
}
+20 -12
View File
@@ -147,13 +147,14 @@ func NewSessionID() (id, csrf string) {
const (
ResConfig = "config" // the firewall configuration, its history and imports
ResSets = "sets" // dynamic sets' entries (a grant may name the sets)
ResBans = "bans" // ban lists' bans (a grant may name the lists)
ResLog = "log" // counters, the drop log and the status
ResAudit = "audit" // the audit log
ResAll = "*"
)
// Resources lists them in display order.
var Resources = []string{ResConfig, ResSets, ResLog, ResAudit}
var Resources = []string{ResConfig, ResSets, ResBans, ResLog, ResAudit}
// Access levels.
const (
@@ -163,8 +164,8 @@ const (
// Roles are ready-made sets of grants.
var Roles = map[string][]model.Grant{
"viewer": {{Resource: ResConfig, Access: Read}, {Resource: ResSets, Access: Read}, {Resource: ResLog, Access: Read}},
"operator": {{Resource: ResConfig, Access: Write}, {Resource: ResSets, Access: Write}, {Resource: ResLog, Access: Read}, {Resource: ResAudit, Access: Read}},
"viewer": {{Resource: ResConfig, Access: Read}, {Resource: ResSets, Access: Read}, {Resource: ResBans, Access: Read}, {Resource: ResLog, Access: Read}},
"operator": {{Resource: ResConfig, Access: Write}, {Resource: ResSets, Access: Write}, {Resource: ResBans, Access: Write}, {Resource: ResLog, Access: Read}, {Resource: ResAudit, Access: Read}},
"log": {{Resource: ResLog, Access: Read}},
}
@@ -179,12 +180,13 @@ func RoleNames() []string {
}
// ParseGrant reads "resource:access" ("log:read", "*:write"), and for
// sets optionally the sets it is limited to ("sets:write:guest,portal").
// sets and bans optionally the sets or ban lists it is limited to
// ("sets:write:guest,portal", "bans:write:web").
func ParseGrant(s string) (model.Grant, error) {
parts := strings.SplitN(strings.TrimSpace(s), ":", 3)
g := model.Grant{Resource: strings.ToLower(parts[0])}
if len(parts) < 2 {
return g, fmt.Errorf("grant %q: write it as resource:access, e.g. log:read, or sets:write:SET1,SET2", s)
return g, fmt.Errorf("grant %q: write it as resource:access, e.g. log:read, sets:write:SET1,SET2 or bans:write:LIST", s)
}
g.Access = strings.ToLower(parts[1])
if len(parts) == 3 {
@@ -211,8 +213,8 @@ func ValidateGrant(g *model.Grant) error {
if g.Access != Read && g.Access != Write {
return fmt.Errorf("access must be read or write, not %q", g.Access)
}
if len(g.Sets) > 0 && g.Resource != ResSets {
return fmt.Errorf("only a sets grant can name sets (sets:write:%s)", strings.Join(g.Sets, ","))
if len(g.Sets) > 0 && g.Resource != ResSets && g.Resource != ResBans {
return fmt.Errorf("only a sets or bans grant can name sets or ban lists (sets:write:%s)", strings.Join(g.Sets, ","))
}
return nil
}
@@ -272,20 +274,26 @@ func covers(gs []model.Grant, resource string, write bool) bool {
// CanSet reports whether the principal may read or change the entries of
// a dynamic set: a sets grant covers it if it names no sets, or names it.
func (p *Principal) CanSet(name string, write bool) bool {
func (p *Principal) CanSet(name string, write bool) bool { return p.canNamed(ResSets, name, write) }
// CanBan reports whether the principal may read or change the bans of a
// ban list: a bans grant covers it if it names no lists, or names it.
func (p *Principal) CanBan(list string, write bool) bool { return p.canNamed(ResBans, list, write) }
func (p *Principal) canNamed(resource, name string, write bool) bool {
if p == nil || p.User == nil {
return false
}
allowed := p.User.Admin || coversSet(p.User.Grants, name, write)
allowed := p.User.Admin || coversNamed(p.User.Grants, resource, name, write)
if p.Token != nil && p.Token.Grants != nil {
allowed = allowed && coversSet(p.Token.Grants, name, write)
allowed = allowed && coversNamed(p.Token.Grants, resource, name, write)
}
return allowed
}
func coversSet(gs []model.Grant, name string, write bool) bool {
func coversNamed(gs []model.Grant, resource, name string, write bool) bool {
for _, g := range gs {
if (g.Resource != ResAll && g.Resource != ResSets) || (write && g.Access != Write) {
if (g.Resource != ResAll && g.Resource != resource) || (write && g.Access != Write) {
continue
}
if len(g.Sets) == 0 {
+6
View File
@@ -153,6 +153,8 @@ type Jarl struct {
// CACert is a PEM file with a CA to trust for Jarl's certificate.
CACert string `yaml:"ca_cert"`
Insecure bool `yaml:"insecure"`
// Refresh is how often the Jarl sets' names are read (docs/jarl.md).
Refresh Duration `yaml:"refresh"`
}
// Enabled reports whether a Jarl server is configured.
@@ -238,6 +240,7 @@ func Default() *Config {
Retention: StatsRetention{Minute: Duration(7 * 24 * time.Hour), Hour: Duration(90 * 24 * time.Hour), Day: Duration(730 * 24 * time.Hour)}},
Shadow: Shadow{Position: ShadowAfter},
Names: Names{Enabled: true, Cache: Duration(10 * time.Minute)},
Jarl: Jarl{Refresh: Duration(time.Minute)},
TLS: TLS{ACME: ACME{Directory: "letsencrypt-staging", Resolvers: []string{"1.1.1.1:53", "9.9.9.9:53", "8.8.8.8:53"}, CheckPropagation: true}},
}
}
@@ -320,6 +323,9 @@ func (c *Config) check() error {
if c.Names.Cache <= 0 {
return fmt.Errorf("names.cache must be a duration, such as 10m")
}
if c.Jarl.Refresh < Duration(10*time.Second) {
return fmt.Errorf("jarl.refresh must be a duration of 10s or more, such as 1m")
}
return nil
}
+51
View File
@@ -191,6 +191,7 @@ func (e *Engine) commit(ctx context.Context, p *auth.Principal, description stri
// Only report warnings the change brings; old ones are on the status
// page.
res.Warnings = newProblems(validate.Check(e.state, e.env()), problems)
res.Warnings = append(res.Warnings, quotaRestarts(e.state, next)...)
if e.Firewall != nil {
if err := e.Firewall.Check(ctx, next); err != nil {
return res, &Error{Kind: KindFirewall, Message: "nftables refused the ruleset: " + err.Error()}
@@ -230,6 +231,23 @@ func (e *Engine) commit(ctx context.Context, p *auth.Principal, description stri
return res, nil
}
// quotaRestarts warns of the rules whose quota starts again because they
// now count it in a way what they used can't be carried to.
func quotaRestarts(before, after *model.State) []validate.Problem {
was := map[int64]*model.Rule{}
for _, r := range before.Rules {
was[r.ID] = r
}
var out []validate.Problem
for _, r := range after.Rules {
if carry, why := model.QuotaCarry(was[r.ID], r); carry == model.QuotaRestarts {
out = append(out, validate.Problem{Severity: validate.Warning, Code: "quota_restarts", Object: r.Label(),
Message: "its quota starts again: " + why})
}
}
return out
}
func newProblems(before, after []validate.Problem) []validate.Problem {
seen := map[string]bool{}
for _, p := range before {
@@ -304,6 +322,24 @@ func Undo(st *model.State, h *model.HistoryEntry) error {
return nil
}
// Rewind applies a history entry backwards to st without Undo's check:
// stepping back through the entries newest first brings back the
// configuration as it was before each.
func Rewind(st *model.State, h *model.HistoryEntry) error {
for i := len(h.Changes) - 1; i >= 0; i-- {
c := h.Changes[i]
if c.Kind == model.KindSettings && c.Before == nil {
continue
}
if err := put(st, c.Kind, c.ID, c.Before); err != nil {
return err
}
}
// put appends what it brings back; its position puts it in place.
st.Sort()
return nil
}
// Replace makes the configuration next as a whole (imports).
func (e *Engine) Replace(ctx context.Context, p *auth.Principal, description string, opts Options, next *model.State) (*Result, error) {
e.mu.Lock()
@@ -462,6 +498,15 @@ func Normalize(st *model.State) {
for _, x := range st.Sets {
x.Name = strings.TrimSpace(x.Name)
x.Entries = normAddrs(x.Entries)
if j := x.Jarl; j != nil {
for i, n := range j.Names {
j.Names[i] = model.ParseJarlName(n.Name)
if n.View != "" {
j.Names[i].View = strings.TrimSpace(n.View)
}
}
j.Within = normAddrs(j.Within) // zone names stay as they are
}
}
for _, x := range st.Services {
x.Name = strings.TrimSpace(x.Name)
@@ -491,6 +536,12 @@ func Normalize(st *model.State) {
normMatch(&x.Match)
x.ToPort = normPorts(x.ToPort)
}
for _, b := range st.BanLists {
b.Name = strings.TrimSpace(b.Name)
b.Scope = strings.ToLower(strings.TrimSpace(b.Scope))
b.Hosts = normAddrs(b.Hosts)
b.Never = normAddrs(b.Never)
}
// Positions are fractions after insertions between rules; renumber
// only when two have come too close to put another between.
st.Sort()
+46
View File
@@ -173,3 +173,49 @@ func TestPositionsStayStableUntilCrowded(t *testing.T) {
t.Errorf("order %v", order)
}
}
// TestQuotaRestartsWarns: a change that counts a quota so what it used
// can't carry over warns; one that can (to a total, wider groups) doesn't.
func TestQuotaRestartsWarns(t *testing.T) {
e := newEngine(t)
ctx := context.Background()
var id int64
if _, err := e.Apply(ctx, admin, "", Options{}, func(st *model.State, newID func() int64) error {
id = newID()
st.Rules = append(st.Rules, &model.Rule{ID: id, Chain: "forward", Action: "drop", Quota: &model.Quota{Bytes: 1 << 30, Over: true, PerSource: true}})
return nil
}); err != nil {
t.Fatal(err)
}
change := func(f func(r *model.Rule)) []string {
res, err := e.Apply(ctx, admin, "", Options{DryRun: true}, func(st *model.State, _ func() int64) error {
f(st.ByID(model.KindRule, id).(*model.Rule))
return nil
})
if err != nil {
t.Fatal(err)
}
var codes []string
for _, w := range res.Warnings {
if w.Code == "quota_restarts" {
codes = append(codes, w.Message)
}
}
return codes
}
if w := change(func(r *model.Rule) { r.Quota.PerSource, r.Quota.PerDestination = false, true }); len(w) != 1 || !strings.Contains(w[0], "per source can't be counted per destination") {
t.Errorf("per source to per destination: %v", w)
}
if w := change(func(r *model.Rule) { r.Quota.PerSource = false }); len(w) != 0 {
t.Errorf("to a total (summed): %v", w)
}
if w := change(func(r *model.Rule) { r.GroupV6 = 56 }); len(w) != 0 {
t.Errorf("wider groups: %v", w)
}
if w := change(func(r *model.Rule) { r.GroupV6 = 128 }); len(w) != 1 {
t.Errorf("finer groups: %v", w)
}
if w := change(func(r *model.Rule) { r.Quota.Bytes = 2 << 30 }); len(w) != 0 {
t.Errorf("another amount: %v", w)
}
}
+29
View File
@@ -247,3 +247,32 @@ func (c *Client) SetTXT(ctx context.Context, name, value string, zones []Zone, s
}
return c.do(ctx, http.MethodPost, "/apply", batch{Description: desc, Ops: ops, DryRun: dryRun}, nil)
}
// ZoneInfo is a zone as Jarl lists it, of any type.
type ZoneInfo struct {
ID int64 `json:"id"`
Name string `json:"name"` // "example.com."
View string `json:"view,omitempty"`
Type string `json:"type"`
InView string `json:"in_view,omitempty"` // type in-view: the view whose zone it serves
Disabled bool `json:"disabled,omitempty"`
}
// AllZones lists every zone the token can see, of any type.
func (c *Client) AllZones(ctx context.Context) ([]ZoneInfo, error) {
var zs []ZoneInfo
return zs, c.do(ctx, http.MethodGet, "/zones", nil, &zs)
}
// Record is a record as Jarl lists it.
type Record struct {
Name string `json:"name"` // "nas.example.com."
Type string `json:"type"`
Data string `json:"data"`
}
// Records lists a zone's enabled records (zone: its ID).
func (c *Client) Records(ctx context.Context, zone int64) ([]Record, error) {
var rs []Record
return rs, c.do(ctx, http.MethodGet, fmt.Sprintf("/zones/%d/records?disabled=0", zone), nil, &rs)
}
+767
View File
@@ -0,0 +1,767 @@
// Package jarlsets fills the sets whose addresses come from Jarl. It reads
// the A and AAAA records of the sets' names through Jarl's API, following
// CNAMEs within the zones Jarl holds, every minute or so, and keeps what it
// last read (in the database too) for when Jarl can't be reached. The
// firewall gets, for each set, the addresses that lie within what the set
// allows (zones' connected networks, or networks it names); the others are
// left out and reported. When what the sets hold changes, the ruleset is
// applied again (no history entry: the configuration didn't change).
package jarlsets
import (
"context"
"encoding/json"
"fmt"
"log/slog"
"net/netip"
"sort"
"strings"
"sync"
"time"
"housecarl/internal/api"
"housecarl/internal/jarl"
"housecarl/internal/model"
"housecarl/internal/validate"
)
// MetaKey is the database meta key holding what was last read.
const MetaKey = "jarl_sets"
// maxChain is how many CNAMEs are followed for one name.
const maxChain = 8
// Source is Jarl (internal/jarl.Client).
type Source interface {
AllZones(ctx context.Context) ([]jarl.ZoneInfo, error)
Records(ctx context.Context, zone int64) ([]jarl.Record, error)
}
// Lookup is what Jarl gave for one name in one view.
type Lookup struct {
Name string `json:"name"`
View string `json:"view,omitempty"`
Found string `json:"found,omitempty"`
Chain []string `json:"chain,omitempty"`
Addresses []string `json:"addresses,omitempty"`
// Problem is why Jarl's records give no addresses (Jarl answered).
Problem string `json:"problem,omitempty"`
// Checked is when Jarl last answered; Error why the last try failed.
Checked time.Time `json:"checked"`
Error string `json:"error,omitempty"`
}
// saved is what the database keeps.
type saved struct {
Lookups map[string]*Lookup `json:"lookups"`
// Nets are the zones' connected networks last seen, for when a zone's
// interfaces have no address (at boot, a link down).
Nets map[string][]netip.Prefix `json:"nets,omitempty"`
}
// Resolver keeps the Jarl sets' addresses.
type Resolver struct {
// Source is Jarl (nil: not configured; the sets keep what was read).
Source Source
// Interval is how often Jarl is read (default a minute).
Interval time.Duration
// State is the configuration in force.
State func() *model.State
// Connected are the networks connected to the firewall, with their
// zones.
Connected func(st *model.State) map[netip.Prefix]string
// Apply puts the sets' new addresses in force.
Apply func(ctx context.Context) error
// Audit records a change of a set's addresses.
Audit func(ctx context.Context, set, detail string)
// Load and Save read and write the database's copy.
Load func(ctx context.Context) (string, error)
Save func(ctx context.Context, value string) error
Log *slog.Logger
run sync.Mutex // one refresh at a time
mu sync.Mutex
data saved
// applied is what the ruleset in force holds (known after an apply).
applied map[string][]string
known bool
read time.Time
err string
kick chan struct{}
now func() time.Time
}
func (r *Resolver) init() {
if r.data.Lookups == nil {
r.data.Lookups = map[string]*Lookup{}
}
if r.data.Nets == nil {
r.data.Nets = map[string][]netip.Prefix{}
}
if r.kick == nil {
r.kick = make(chan struct{}, 1)
}
if r.now == nil {
r.now = time.Now
}
if r.Log == nil {
r.Log = slog.Default()
}
}
// Restore reads what was saved before housecarld (or the machine)
// stopped. Call it before the first apply.
func (r *Resolver) Restore(ctx context.Context) {
r.mu.Lock()
defer r.mu.Unlock()
r.init()
if r.Load == nil {
return
}
v, err := r.Load(ctx)
if err != nil || v == "" {
return
}
var s saved
if err := json.Unmarshal([]byte(v), &s); err != nil {
r.Log.Warn("the saved Jarl set addresses can't be read; reading Jarl again", "err", err)
return
}
if s.Lookups != nil {
r.data.Lookups = s.Lookups
}
if s.Nets != nil {
r.data.Nets = s.Nets
}
}
// Kick has the next refresh happen now (after a change of the
// configuration: a new name).
func (r *Resolver) Kick() {
r.mu.Lock()
r.init()
k := r.kick
r.mu.Unlock()
select {
case k <- struct{}{}:
default:
}
}
// Run reads Jarl until ctx ends.
func (r *Resolver) Run(ctx context.Context) {
r.mu.Lock()
r.init()
k := r.kick
every := r.Interval
r.mu.Unlock()
if every <= 0 {
every = time.Minute
}
t := time.NewTicker(every)
defer t.Stop()
for {
r.Refresh(ctx)
select {
case <-ctx.Done():
return
case <-t.C:
case <-k:
}
}
}
// Refresh reads Jarl now and puts what changed in force.
func (r *Resolver) Refresh(ctx context.Context) {
r.run.Lock()
defer r.run.Unlock()
st := r.State()
r.read1(ctx, st)
r.sync(ctx, st)
}
// needed are the names the sets of st look up, by key.
func needed(st *model.State) map[string]model.JarlName {
out := map[string]model.JarlName{}
for _, s := range st.Sets {
if s.Jarl == nil {
continue
}
for _, n := range s.Jarl.Names {
out[n.String()] = n
}
}
return out
}
// read1 reads every name from Jarl once.
func (r *Resolver) read1(ctx context.Context, st *model.State) {
names := needed(st)
r.mu.Lock()
r.init()
src := r.Source
r.mu.Unlock()
// The zones' networks, for when they can't be seen (Entries).
r.rememberNets(st)
if src == nil || len(names) == 0 {
r.mu.Lock()
changed := false
for k := range r.data.Lookups {
if _, ok := names[k]; !ok {
delete(r.data.Lookups, k)
changed = true
}
}
r.mu.Unlock()
if changed {
r.save(ctx)
}
return
}
ctx, cancel := context.WithTimeout(ctx, 30*time.Second)
defer cancel()
zones, err := src.AllZones(ctx)
now := r.now()
results := map[string]*Lookup{}
if err == nil {
l := &lookup{ctx: ctx, src: src, zones: zones, records: map[int64][]jarl.Record{}, errs: map[int64]error{}}
for k, n := range names {
results[k] = l.resolve(n)
}
}
r.mu.Lock()
if err != nil {
r.err = err.Error()
} else {
r.err, r.read = "", now
}
for k, n := range names {
old := r.data.Lookups[k]
res := results[k]
switch {
case res != nil && res.Error == "":
res.Checked = now
r.data.Lookups[k] = res
case old != nil:
// Keep what Jarl last said.
old.Error = errText(err, res)
default:
r.data.Lookups[k] = &Lookup{Name: n.Name, View: n.View, Error: errText(err, res)}
}
if err != nil || res != nil && res.Error != "" {
r.Log.Warn("reading a Jarl set's name failed; the set keeps the addresses it had", "name", n.String(), "err", r.data.Lookups[k].Error)
}
}
for k := range r.data.Lookups {
if _, ok := names[k]; !ok {
delete(r.data.Lookups, k)
}
}
r.mu.Unlock()
r.save(ctx)
}
func errText(err error, res *Lookup) string {
if err != nil {
return err.Error()
}
return res.Error
}
func (r *Resolver) save(ctx context.Context) {
if r.Save == nil {
return
}
r.mu.Lock()
b, err := json.Marshal(r.data)
r.mu.Unlock()
if err == nil {
err = r.Save(ctx, string(b))
}
if err != nil {
r.Log.Warn("saving the Jarl set addresses failed", "err", err)
}
}
// rememberNets keeps each zone's connected networks.
func (r *Resolver) rememberNets(st *model.State) {
if r.Connected == nil {
return
}
now := zoneNets(r.Connected(st))
r.mu.Lock()
defer r.mu.Unlock()
for z, ps := range now {
r.data.Nets[z] = ps
}
}
func zoneNets(conn map[netip.Prefix]string) map[string][]netip.Prefix {
out := map[string][]netip.Prefix{}
for p, z := range conn {
out[z] = append(out[z], p)
}
for _, ps := range out {
sort.Slice(ps, func(i, j int) bool { return ps[i].String() < ps[j].String() })
}
return out
}
// sync applies the ruleset again if the sets' addresses changed.
func (r *Resolver) sync(ctx context.Context, st *model.State) {
want := r.Entries(st)
r.mu.Lock()
known, applied := r.known, r.applied
r.mu.Unlock()
// No Jarl sets, nothing to do (and an empty configuration is never
// applied: its policies would close the machine).
if len(want) == 0 || st.Empty() || known && sameEntries(applied, want) || r.Apply == nil {
return
}
if err := r.Apply(ctx); err != nil {
r.Log.Error("putting the Jarl sets' new addresses in force failed", "err", err)
return
}
for _, s := range st.Sets {
if s.Jarl == nil {
continue
}
add, del := diff(applied[s.Name], want[s.Name])
if len(add) == 0 && len(del) == 0 {
continue
}
var parts []string
if len(add) > 0 {
parts = append(parts, "added "+strings.Join(add, ", "))
}
if len(del) > 0 {
parts = append(parts, "removed "+strings.Join(del, ", "))
}
detail := strings.Join(parts, "; ")
r.Log.Info("a Jarl set's addresses changed", "set", s.Name, "change", detail)
if r.Audit != nil {
r.Audit(ctx, s.Name, detail)
}
}
}
func sameEntries(a, b map[string][]string) bool {
if len(a) != len(b) {
return false
}
for k, v := range a {
w, ok := b[k]
if !ok || strings.Join(v, ",") != strings.Join(w, ",") {
return false
}
}
return true
}
func diff(old, now []string) (add, del []string) {
was := map[string]bool{}
for _, a := range old {
was[a] = true
}
is := map[string]bool{}
for _, a := range now {
is[a] = true
if !was[a] {
add = append(add, a)
}
}
for _, a := range old {
if !is[a] {
del = append(del, a)
}
}
return add, del
}
// Applied implements nft.JarlSets.
func (r *Resolver) Applied(entries map[string][]string) {
c := map[string][]string{}
for k, v := range entries {
c[k] = append([]string(nil), v...)
}
r.mu.Lock()
r.applied, r.known = c, true
r.mu.Unlock()
}
// Entries implements nft.JarlSets: every Jarl set of st (empty or not),
// with the addresses its names have that lie within what it allows.
func (r *Resolver) Entries(st *model.State) map[string][]string {
out := map[string][]string{}
nets := r.nets(st)
r.mu.Lock()
defer r.mu.Unlock()
r.init()
for _, s := range st.Sets {
if s.Jarl == nil {
continue
}
seen := map[string]bool{}
list := []string{}
for _, n := range s.Jarl.Names {
lu := r.data.Lookups[n.String()]
if lu == nil {
continue
}
for _, a := range lu.Addresses {
if seen[a] || allowed(a, s.Jarl.Within, nets) != "" {
continue
}
seen[a] = true
list = append(list, a)
}
}
sortAddrs(list)
out[s.Name] = list
}
return out
}
// nets are the zones' connected networks: now, or as last seen.
func (r *Resolver) nets(st *model.State) map[string][]netip.Prefix {
var now map[string][]netip.Prefix
if r.Connected != nil {
now = zoneNets(r.Connected(st))
}
r.mu.Lock()
defer r.mu.Unlock()
r.init()
out := map[string][]netip.Prefix{}
for _, z := range st.Zones {
if ps := now[z.Name]; len(ps) > 0 {
out[z.Name] = ps
} else {
out[z.Name] = r.data.Nets[z.Name]
}
}
return out
}
func sortAddrs(list []string) {
sort.Slice(list, func(i, j int) bool {
a, _ := netip.ParseAddr(list[i])
b, _ := netip.ParseAddr(list[j])
return a.Less(b)
})
}
// allowed says why an address isn't let in ("" if it is).
func allowed(addr string, within []string, nets map[string][]netip.Prefix) string {
a, err := netip.ParseAddr(addr)
if err != nil {
return "not an address"
}
a = a.Unmap()
switch {
case a.IsUnspecified(), a.IsLoopback(), a.IsMulticast():
return "not a host's address (unspecified, loopback or multicast)"
}
for _, w := range within {
if l, err := model.ParseAddress(w); err == nil {
if l.Contains(a) {
return ""
}
continue
}
for _, p := range nets[w] {
if p.Contains(a) {
return ""
}
}
}
return "not within " + strings.Join(within, ", ")
}
// ---- looking names up
// lookup reads Jarl's records for one refresh.
type lookup struct {
ctx context.Context
src Source
zones []jarl.ZoneInfo
records map[int64][]jarl.Record
errs map[int64]error
}
func fqdn(n string) string {
n = strings.ToLower(strings.TrimSpace(n))
if !strings.HasSuffix(n, ".") {
n += "."
}
return n
}
func under(name, zone string) bool {
return zone == "." || name == zone || strings.HasSuffix(name, "."+zone)
}
func short(n string) string { return strings.TrimSuffix(n, ".") }
// zoneFor finds the zone of Jarl's that answers for name in view ("":
// the only view whose zones hold it). It returns the zone, the view, or
// why there is none.
func (l *lookup) zoneFor(name, view string) (*jarl.ZoneInfo, string, string) {
if view == "" {
views := map[string]bool{}
for _, z := range l.zones {
if z.Type != "hint" && under(name, fqdn(z.Name)) {
views[z.View] = true
}
}
switch len(views) {
case 0:
return nil, "", fmt.Sprintf("no zone in Jarl holds %s (or the token can't see it)", short(name))
case 1:
for v := range views {
view = v
}
default:
var vs []string
for v := range views {
vs = append(vs, v)
}
sort.Strings(vs)
return nil, "", fmt.Sprintf("%s is in Jarl's views %s: name the view, such as %s@%s", short(name), strings.Join(vs, ", "), short(name), vs[0])
}
}
var best *jarl.ZoneInfo
for i := range l.zones {
z := &l.zones[i]
zn := fqdn(z.Name)
if z.Type == "hint" || !strings.EqualFold(z.View, view) || !under(name, zn) {
continue
}
if best == nil || len(zn) > len(fqdn(best.Name)) {
best = z
}
}
where := ""
if view != "" {
where = " in view " + view
}
if best == nil {
return nil, view, fmt.Sprintf("no zone%s holds %s in Jarl (or the token can't see it)", where, short(name))
}
if best.Type == "in-view" {
var served *jarl.ZoneInfo
for i := range l.zones {
z := &l.zones[i]
if fqdn(z.Name) == fqdn(best.Name) && strings.EqualFold(z.View, best.InView) && z.Type != "in-view" {
served = z
}
}
if served == nil {
return nil, view, fmt.Sprintf("zone %s%s serves view %s's zone, which the token can't see", short(best.Name), where, best.InView)
}
best = served
}
if best.Disabled {
return nil, view, fmt.Sprintf("zone %s%s is disabled in Jarl", short(best.Name), where)
}
if best.Type != "primary" {
return nil, view, fmt.Sprintf("zone %s%s is a %s zone: Jarl doesn't hold its records", short(best.Name), where, best.Type)
}
return best, view, ""
}
func (l *lookup) zoneRecords(id int64) ([]jarl.Record, error) {
if err, ok := l.errs[id]; ok {
return nil, err
}
if rs, ok := l.records[id]; ok {
return rs, nil
}
rs, err := l.src.Records(l.ctx, id)
if err != nil {
l.errs[id] = err
return nil, err
}
l.records[id] = rs
return rs, nil
}
// resolve looks a name up as named would answer it in the view: its A
// and AAAA records, or those of the names its CNAMEs lead to.
func (l *lookup) resolve(n model.JarlName) *Lookup {
out := &Lookup{Name: n.Name, View: n.View}
name, view := fqdn(n.Name), n.View
seen := map[string]bool{name: true}
for hop := 0; ; hop++ {
z, v, problem := l.zoneFor(name, view)
if hop == 0 {
out.Found = v
}
if problem != "" {
if hop > 0 {
problem = fmt.Sprintf("its CNAME leads to %s, which Housecarl can't follow: %s (only names Jarl holds are followed)", short(name), problem)
}
out.Problem = problem
return out
}
view = v
recs, err := l.zoneRecords(z.ID)
if err != nil {
out.Error = err.Error()
return out
}
apex := fqdn(z.Name)
var addrs []string
cname := ""
for _, rec := range recs {
owner := fqdn(rec.Name)
switch {
case rec.Type == "NS" && owner != apex && under(name, owner):
out.Problem = fmt.Sprintf("%s is below a delegation (NS records at %s): Jarl doesn't hold its records", short(name), short(owner))
return out
case owner != name:
case rec.Type == "A" || rec.Type == "AAAA":
if a, err := netip.ParseAddr(strings.TrimSpace(rec.Data)); err == nil {
addrs = append(addrs, a.Unmap().String())
}
case rec.Type == "CNAME":
cname = target(rec.Data, apex)
}
}
if cname != "" {
if seen[cname] || len(out.Chain) >= maxChain {
out.Problem = fmt.Sprintf("its CNAMEs go round in a loop or are longer than %d (%s)", maxChain, strings.Join(append(out.Chain, short(cname)), " → "))
return out
}
seen[cname] = true
out.Chain = append(out.Chain, short(cname))
name = cname
continue
}
if len(addrs) == 0 {
if hop > 0 {
out.Problem = fmt.Sprintf("its CNAME leads to %s, which has no A or AAAA record in Jarl", short(name))
} else {
out.Problem = fmt.Sprintf("%s has no A, AAAA or CNAME record in Jarl (wildcards aren't followed)", short(name))
}
return out
}
sortAddrs(addrs)
out.Addresses = dedup(addrs)
return out
}
}
// target is a CNAME's target as an FQDN (a relative one is under the
// zone, as in a zone file).
func target(data, apex string) string {
t := strings.ToLower(strings.TrimSpace(data))
if strings.HasSuffix(t, ".") {
return t
}
if t == "@" {
return apex
}
if apex == "." {
return t + "."
}
return t + "." + apex
}
func dedup(in []string) []string {
var out []string
for i, a := range in {
if i == 0 || a != in[i-1] {
out = append(out, a)
}
}
return out
}
// ---- status
// Status is what a Jarl set holds and why.
func (r *Resolver) Status(st *model.State, set *model.AddressSet) api.JarlSet {
out := api.JarlSet{Set: set.Name, Entries: []string{}, Names: []api.JarlName{}, Within: []string{}}
if set.Jarl == nil {
return out
}
out.Within = append(out.Within, set.Jarl.Within...)
out.Entries = r.Entries(st)[set.Name]
nets := r.nets(st)
r.mu.Lock()
defer r.mu.Unlock()
r.init()
out.Configured = r.Source != nil
if !r.read.IsZero() {
t := r.read
out.Read = &t
}
out.Error = r.err
for _, n := range set.Jarl.Names {
jn := api.JarlName{Name: n.Name, View: n.View, Addresses: []api.JarlAddress{}}
if lu := r.data.Lookups[n.String()]; lu != nil {
jn.Found, jn.Chain, jn.Problem, jn.Error = lu.Found, lu.Chain, lu.Problem, lu.Error
if !lu.Checked.IsZero() {
t := lu.Checked
jn.Checked = &t
}
for _, a := range lu.Addresses {
why := allowed(a, set.Jarl.Within, nets)
jn.Addresses = append(jn.Addresses, api.JarlAddress{Address: a, Used: why == "", Why: why})
}
}
out.Names = append(out.Names, jn)
}
return out
}
// Problems are the warnings about the Jarl sets for the status page.
func (r *Resolver) Problems(st *model.State) []validate.Problem {
var out []validate.Problem
var sets []*model.AddressSet
for _, s := range st.Sets {
if s.Jarl != nil {
sets = append(sets, s)
}
}
if len(sets) == 0 {
return nil
}
r.mu.Lock()
configured, readErr := r.Source != nil, r.err
r.mu.Unlock()
if !configured {
out = append(out, validate.Problem{Severity: validate.Warning, Code: "jarl_not_configured", Object: "jarl",
Message: "sets take their addresses from Jarl, but housecarl.yaml names no Jarl server (jarl.server, docs/jarl.md): they keep the addresses they last had, none if Jarl was never read"})
} else if readErr != "" {
out = append(out, validate.Problem{Severity: validate.Warning, Code: "jarl_unreachable", Object: "jarl",
Message: "reading Jarl failed: " + readErr + "; the Jarl sets keep the addresses they had"})
}
for _, s := range sets {
obj := "set " + s.Name
for _, n := range r.Status(st, s).Names {
switch {
case n.Problem != "":
out = append(out, validate.Problem{Severity: validate.Warning, Code: "jarl_name", Object: obj,
Message: n.Name + viewSuffix(n.View) + ": " + n.Problem + "; it adds no addresses"})
case n.Error != "" && readErr == "":
out = append(out, validate.Problem{Severity: validate.Warning, Code: "jarl_unreachable", Object: obj,
Message: n.Name + viewSuffix(n.View) + ": reading it from Jarl failed: " + n.Error + "; it keeps the addresses it had"})
}
for _, a := range n.Addresses {
if !a.Used {
out = append(out, validate.Problem{Severity: validate.Warning, Code: "jarl_outside", Object: obj,
Message: fmt.Sprintf("%s%s has address %s, which is left out: %s", n.Name, viewSuffix(n.View), a.Address, a.Why)})
}
}
}
}
return out
}
func viewSuffix(v string) string {
if v == "" {
return ""
}
return "@" + v
}
+317
View File
@@ -0,0 +1,317 @@
package jarlsets
import (
"context"
"errors"
"io"
"log/slog"
"net/netip"
"reflect"
"strings"
"testing"
"housecarl/internal/jarl"
"housecarl/internal/model"
)
// fakeJarl is a Jarl with a few zones.
type fakeJarl struct {
zones []jarl.ZoneInfo
records map[int64][]jarl.Record
down bool // AllZones fails
broken map[int64]bool // Records fails
calls int
}
func (f *fakeJarl) AllZones(ctx context.Context) ([]jarl.ZoneInfo, error) {
if f.down {
return nil, errors.New("can't reach Jarl")
}
return f.zones, nil
}
func (f *fakeJarl) Records(ctx context.Context, id int64) ([]jarl.Record, error) {
f.calls++
if f.broken[id] {
return nil, errors.New("Jarl answered 500")
}
return f.records[id], nil
}
func rec(name, typ, data string) jarl.Record { return jarl.Record{Name: name, Type: typ, Data: data} }
// newFake: example.com in views internal (1) and external (2), linked
// sub-zone wg.example.com in internal (3), other.org only in internal (4),
// a secondary (5), a disabled zone (6), an in-view zone in vps (7).
func newFake() *fakeJarl {
return &fakeJarl{
zones: []jarl.ZoneInfo{
{ID: 1, Name: "example.com.", View: "internal", Type: "primary"},
{ID: 2, Name: "example.com.", View: "external", Type: "primary"},
{ID: 3, Name: "wg.example.com.", View: "internal", Type: "primary"},
{ID: 4, Name: "other.org.", View: "internal", Type: "primary"},
{ID: 5, Name: "sec.net.", View: "internal", Type: "secondary"},
{ID: 6, Name: "off.net.", View: "internal", Type: "primary", Disabled: true},
{ID: 7, Name: "other.org.", View: "vps", Type: "in-view", InView: "internal"},
{ID: 8, Name: ".", View: "internal", Type: "hint"},
},
records: map[int64][]jarl.Record{
1: {
rec("example.com.", "NS", "ns1.example.com."),
rec("nas.example.com.", "A", "10.10.0.20"),
rec("nas.example.com.", "A", "10.10.0.21"),
rec("nas.example.com.", "AAAA", "2001:db8:10::20"),
rec("files.example.com.", "CNAME", "nas.example.com."),
rec("vpn.example.com.", "CNAME", "peer1.wg.example.com."),
rec("ext.example.com.", "CNAME", "host.duckdns.org."),
rec("loop1.example.com.", "CNAME", "loop2.example.com."),
rec("loop2.example.com.", "CNAME", "loop1"),
rec("rel.example.com.", "CNAME", "nas"),
rec("mail.example.com.", "MX", "10 nas.example.com."),
rec("lab.example.com.", "NS", "ns.lab.example.com."),
rec("www.example.com.", "A", "10.10.0.30"),
rec("bad.example.com.", "A", "127.0.0.1"),
rec("far.example.com.", "A", "192.0.2.9"),
rec("dead.example.com.", "CNAME", "nothing.example.com."),
rec("toother.example.com.", "CNAME", "srv.other.org."),
},
2: {
rec("www.example.com.", "A", "203.0.113.30"),
},
3: {
rec("peer1.wg.example.com.", "A", "10.50.50.2"),
},
4: {
rec("srv.other.org.", "A", "10.10.0.40"),
},
},
broken: map[int64]bool{},
}
}
func resolveOne(t *testing.T, f *fakeJarl, s string) *Lookup {
t.Helper()
l := &lookup{ctx: context.Background(), src: f, zones: f.zones, records: map[int64][]jarl.Record{}, errs: map[int64]error{}}
return l.resolve(model.ParseJarlName(s))
}
func TestResolve(t *testing.T) {
f := newFake()
cases := []struct {
name string
addrs []string
chain []string
problem string // part of it
found string
}{
{name: "nas.example.com@internal", addrs: []string{"10.10.0.20", "10.10.0.21", "2001:db8:10::20"}, found: "internal"},
{name: "NAS.Example.com.@internal", addrs: []string{"10.10.0.20", "10.10.0.21", "2001:db8:10::20"}, found: "internal"},
{name: "files.example.com@internal", addrs: []string{"10.10.0.20", "10.10.0.21", "2001:db8:10::20"}, chain: []string{"nas.example.com"}},
{name: "rel.example.com@internal", addrs: []string{"10.10.0.20", "10.10.0.21", "2001:db8:10::20"}, chain: []string{"nas.example.com"}},
// Into the sub-zone, and into another zone of the same view.
{name: "vpn.example.com@internal", addrs: []string{"10.50.50.2"}, chain: []string{"peer1.wg.example.com"}},
{name: "toother.example.com@internal", addrs: []string{"10.10.0.40"}, chain: []string{"srv.other.org"}},
// Only internal has wg.example.com, but external's example.com
// answers for the name too (that it doesn't exist): the view must be
// named.
{name: "peer1.wg.example.com", problem: "views external, internal"},
{name: "srv.other.org", problem: "views internal, vps"},
// Views give different answers.
{name: "www.example.com@internal", addrs: []string{"10.10.0.30"}},
{name: "www.example.com@external", addrs: []string{"203.0.113.30"}},
{name: "www.example.com", problem: "views external, internal: name the view"},
{name: "www.example.com@vps", problem: "no zone in view vps holds www.example.com"},
// An in-view zone serves the other view's records.
{name: "srv.other.org@vps", addrs: []string{"10.10.0.40"}},
{name: "ext.example.com@internal", chain: []string{"host.duckdns.org"}, problem: "its CNAME leads to host.duckdns.org, which Housecarl can't follow"},
{name: "loop1.example.com@internal", problem: "loop"},
{name: "dead.example.com@internal", chain: []string{"nothing.example.com"}, problem: "nothing.example.com, which has no A or AAAA record"},
{name: "mail.example.com@internal", problem: "has no A, AAAA or CNAME record"},
{name: "missing.example.com@internal", problem: "has no A, AAAA or CNAME record"},
{name: "host.lab.example.com@internal", problem: "below a delegation (NS records at lab.example.com)"},
{name: "a.sec.net", problem: "is a secondary zone"},
{name: "a.off.net", problem: "is disabled"},
{name: "nowhere.test", problem: "no zone in Jarl holds nowhere.test"},
}
for _, c := range cases {
lu := resolveOne(t, f, c.name)
if lu.Error != "" {
t.Errorf("%s: error %s", c.name, lu.Error)
}
if !reflect.DeepEqual(lu.Addresses, c.addrs) {
t.Errorf("%s: addresses %v, want %v (problem %q)", c.name, lu.Addresses, c.addrs, lu.Problem)
}
if c.chain != nil && !reflect.DeepEqual(lu.Chain, c.chain) {
t.Errorf("%s: chain %v, want %v", c.name, lu.Chain, c.chain)
}
if c.problem == "" && lu.Problem != "" || !strings.Contains(lu.Problem, c.problem) {
t.Errorf("%s: problem %q, want %q", c.name, lu.Problem, c.problem)
}
if c.found != "" && lu.Found != c.found {
t.Errorf("%s: found in %q, want %q", c.name, lu.Found, c.found)
}
}
}
func TestResolveNoViews(t *testing.T) {
f := &fakeJarl{zones: []jarl.ZoneInfo{{ID: 1, Name: "lan.", Type: "primary"}},
records: map[int64][]jarl.Record{1: {rec("nas.lan.", "A", "192.168.1.5")}}}
if lu := resolveOne(t, f, "nas.lan"); !reflect.DeepEqual(lu.Addresses, []string{"192.168.1.5"}) || lu.Problem != "" {
t.Errorf("%+v", lu)
}
}
// harness is a resolver over the fake Jarl, with lan = 10.10.0.0/24 and
// wg = 10.50.50.0/24 connected.
type harness struct {
r *Resolver
f *fakeJarl
st *model.State
conn map[netip.Prefix]string
applies int
audits []string
saved string
}
func newHarness(sets ...*model.AddressSet) *harness {
h := &harness{f: newFake(), conn: map[netip.Prefix]string{
netip.MustParsePrefix("10.10.0.0/24"): "lan",
netip.MustParsePrefix("10.50.50.0/24"): "wg",
}}
h.st = &model.State{Zones: []*model.Zone{{Name: "lan"}, {Name: "wg"}, {Name: "wan"}}, Sets: sets}
h.r = &Resolver{Source: h.f, State: func() *model.State { return h.st }, Log: slog.New(slog.NewTextHandler(io.Discard, nil)),
Connected: func(*model.State) map[netip.Prefix]string { return h.conn },
Apply: func(ctx context.Context) error {
h.applies++
h.r.Applied(h.r.Entries(h.st))
return nil
},
Audit: func(ctx context.Context, set, detail string) { h.audits = append(h.audits, set+": "+detail) },
Load: func(ctx context.Context) (string, error) { return h.saved, nil },
Save: func(ctx context.Context, v string) error { h.saved = v; return nil }}
return h
}
func jset(name string, within []string, names ...string) *model.AddressSet {
j := &model.JarlSource{Within: within}
for _, n := range names {
j.Names = append(j.Names, model.ParseJarlName(n))
}
return &model.AddressSet{Name: name, Jarl: j}
}
func TestWithin(t *testing.T) {
h := newHarness(
jset("nas", []string{"lan"}, "nas.example.com@internal", "vpn.example.com@internal", "bad.example.com@internal", "far.example.com@internal"),
jset("wide", []string{"lan", "wg", "192.0.2.0/24", "2001:db8:10::/64"}, "nas.example.com@internal", "vpn.example.com@internal", "far.example.com@internal"),
jset("none", []string{"lan"}, "missing.example.com@internal"),
)
h.r.Refresh(context.Background())
got := h.r.Entries(h.st)
want := map[string][]string{
"nas": {"10.10.0.20", "10.10.0.21"},
"wide": {"10.10.0.20", "10.10.0.21", "10.50.50.2", "192.0.2.9", "2001:db8:10::20"},
"none": {},
}
if !reflect.DeepEqual(got, want) {
t.Fatalf("entries %v, want %v", got, want)
}
st := h.r.Status(h.st, h.st.Sets[0])
why := map[string]string{}
for _, n := range st.Names {
for _, a := range n.Addresses {
why[a.Address] = a.Why
}
}
if why["10.10.0.20"] != "" || !strings.Contains(why["10.50.50.2"], "not within lan") || !strings.Contains(why["127.0.0.1"], "loopback") || why["2001:db8:10::20"] == "" {
t.Errorf("why: %v", why)
}
codes := map[string]int{}
for _, p := range h.r.Problems(h.st) {
codes[p.Code]++
}
// nas: 3 left out (v6, wg, loopback, far = 4); wide: none; none: a name problem.
if codes["jarl_outside"] != 4 || codes["jarl_name"] != 1 || codes["jarl_unreachable"] != 0 {
t.Errorf("problems %v: %+v", codes, h.r.Problems(h.st))
}
// The lan interface loses its address: its networks as last seen count.
h.conn = map[netip.Prefix]string{}
if got := h.r.Entries(h.st)["nas"]; !reflect.DeepEqual(got, []string{"10.10.0.20", "10.10.0.21"}) {
t.Errorf("without lan's address: %v", got)
}
}
func TestKeepsWhatItHad(t *testing.T) {
h := newHarness(jset("nas", []string{"lan", "wg"}, "nas.example.com@internal", "peer1.wg.example.com@internal"))
ctx := context.Background()
h.r.Refresh(ctx)
if h.applies != 1 || len(h.audits) != 1 || !strings.Contains(h.audits[0], "added 10.10.0.20, 10.10.0.21, 10.50.50.2") {
t.Fatalf("first read: %d applies, audits %v", h.applies, h.audits)
}
// Nothing changed: no apply.
h.r.Refresh(ctx)
if h.applies != 1 {
t.Errorf("unchanged: %d applies", h.applies)
}
// Jarl down: the addresses stay, a warning says so.
h.f.down = true
h.r.Refresh(ctx)
if got := h.r.Entries(h.st)["nas"]; len(got) != 3 || h.applies != 1 {
t.Errorf("Jarl down: %v, %d applies", got, h.applies)
}
if ps := h.r.Problems(h.st); len(ps) != 2 || ps[0].Code != "jarl_unreachable" || ps[1].Code != "jarl_outside" {
t.Errorf("Jarl down: %+v", ps)
}
// One zone's records fail: only its name keeps the old addresses.
h.f.down = false
h.f.broken[3] = true
h.f.records[1][1] = rec("nas.example.com.", "A", "10.10.0.22")
h.r.Refresh(ctx)
if got := h.r.Entries(h.st)["nas"]; !reflect.DeepEqual(got, []string{"10.10.0.21", "10.10.0.22", "10.50.50.2"}) {
t.Errorf("one zone down: %v", got)
}
if h.applies != 2 || !strings.Contains(h.audits[1], "added 10.10.0.22; removed 10.10.0.20") {
t.Errorf("audits %v", h.audits)
}
// After a restart, before Jarl is read: what was saved.
r2 := &Resolver{Load: h.r.Load, State: h.r.State, Connected: h.r.Connected}
r2.Restore(ctx)
if got := r2.Entries(h.st)["nas"]; !reflect.DeepEqual(got, []string{"10.10.0.21", "10.10.0.22", "10.50.50.2"}) {
t.Errorf("restored: %v", got)
}
// Jarl answers that the name is gone: its addresses go.
h.f.broken[3] = false
h.f.records[3] = nil
h.r.Refresh(ctx)
if got := h.r.Entries(h.st)["nas"]; !reflect.DeepEqual(got, []string{"10.10.0.21", "10.10.0.22"}) {
t.Errorf("deleted record: %v", got)
}
// A name taken out of the set is forgotten.
h.st.Sets[0].Jarl.Names = h.st.Sets[0].Jarl.Names[:1]
h.r.Refresh(ctx)
if len(h.r.data.Lookups) != 1 {
t.Errorf("lookups %v", h.r.data.Lookups)
}
}
func TestNotConfigured(t *testing.T) {
h := newHarness(jset("nas", []string{"lan"}, "nas.example.com@internal"))
h.r.Source = nil
h.r.Refresh(context.Background())
if ps := h.r.Problems(h.st); len(ps) != 1 || ps[0].Code != "jarl_not_configured" {
t.Errorf("%+v", ps)
}
if got := h.r.Entries(h.st); !reflect.DeepEqual(got, map[string][]string{"nas": {}}) {
t.Errorf("%v", got)
}
}
// TestNoSetsNoApply: without Jarl sets nothing is applied, not even at the
// first read (an empty configuration must never be applied).
func TestNoSetsNoApply(t *testing.T) {
h := newHarness()
h.r.Refresh(context.Background())
if h.applies != 0 {
t.Errorf("%d applies", h.applies)
}
}
+2 -2
View File
@@ -241,7 +241,7 @@ func verdictOf(action string) string {
// elsewhere reports whether every address of refs (addresses, sets) lies
// in networks connected to zones other than zone: then traffic of zone
// can't have them. An empty list, a dynamic set, or an address the
// can't have them. An empty list, a dynamic or Jarl set, or an address the
// firewall isn't directly connected to (a routed network) might.
func elsewhere(st *model.State, o Options, refs []string, zone string) bool {
if len(refs) == 0 || len(o.Connected) == 0 || zone == Firewall {
@@ -250,7 +250,7 @@ func elsewhere(st *model.State, o Options, refs []string, zone string) bool {
var entries []string
for _, r := range refs {
if s := st.Set(r); s != nil {
if s.Dynamic {
if s.Runtime() {
return false
}
entries = append(entries, s.Entries...)
+33 -1
View File
@@ -32,6 +32,34 @@ func (a Address) String() string {
return a.Prefix.String()
}
// Contains reports whether the entry holds an address.
func (a Address) Contains(x netip.Addr) bool {
x = x.Unmap()
if a.First.IsValid() {
return x.Is4() == a.First.Is4() && !x.Less(a.First) && !a.End.Less(x)
}
return a.Prefix.Contains(x)
}
// Overlaps reports whether the entry and a prefix share an address.
func (a Address) Overlaps(p netip.Prefix) bool {
if !a.First.IsValid() {
return a.Prefix.Overlaps(p)
}
if a.First.Is4() != p.Addr().Is4() {
return false
}
// The prefix's first and last addresses against the range.
first := p.Masked().Addr()
last := first
b := last.AsSlice()
for i := p.Bits(); i < len(b)*8; i++ {
b[i/8] |= 0x80 >> (i % 8)
}
last, _ = netip.AddrFromSlice(b)
return !last.Less(a.First) && !a.End.Less(first)
}
// ParseAddress reads an address, a prefix (host bits must be zero) or a
// range, IPv4 or IPv6.
func ParseAddress(s string) (Address, error) {
@@ -56,7 +84,11 @@ func ParseAddress(s string) (Address, error) {
if err != nil {
return Address{}, fmt.Errorf("%q is not an address or prefix", s)
}
p = netip.PrefixFrom(p.Addr().Unmap(), p.Bits())
if p.Addr().Is4In6() {
// Unmapping would leave an IPv4 address with IPv6 bits
// (::ffff:0:0/96 as 0.0.0.0/96).
return Address{}, fmt.Errorf("%q is an IPv4-mapped prefix, which can't be used: write IPv4 prefixes as IPv4 (10.0.0.0/8)", s)
}
if p.Masked() != p {
return Address{}, fmt.Errorf("%q has host bits set; the prefix is %s", s, p.Masked())
}
+28
View File
@@ -0,0 +1,28 @@
package model
import (
"strings"
"testing"
)
func TestParseAddress(t *testing.T) {
for _, c := range []struct{ in, want, err string }{
{in: "10.0.0.0/8", want: "10.0.0.0/8"},
{in: "2001:db8::/32", want: "2001:db8::/32"},
{in: "::ffff:10.0.0.1", want: "10.0.0.1"},
{in: "10.0.0.1/8", err: "host bits"},
{in: "::ffff:0:0/96", err: "IPv4-mapped"},
{in: "::ffff:10.0.0.0/104", err: "IPv4-mapped"},
} {
a, err := ParseAddress(c.in)
if c.err != "" {
if err == nil || !strings.Contains(err.Error(), c.err) {
t.Errorf("%s: error %v, want one saying %q", c.in, err, c.err)
}
continue
}
if err != nil || a.String() != c.want {
t.Errorf("%s: %q, %v; want %q", c.in, a.String(), err, c.want)
}
}
}
+3 -2
View File
@@ -7,9 +7,10 @@ import (
// Grant gives access to one resource (see internal/auth).
type Grant struct {
Resource string `json:"resource"` // config, sets, log, audit, *
Resource string `json:"resource"` // config, sets, bans, log, audit, *
Access string `json:"access"` // read or write (write implies read)
// Sets limits a "sets" grant to these dynamic sets (none: all).
// Sets limits a "sets" grant to these dynamic sets, and a "bans"
// grant to these ban lists (none: all).
Sets []string `json:"sets,omitempty"`
}
+53
View File
@@ -0,0 +1,53 @@
package model
import (
"net/netip"
"time"
)
// Ban is an address a ban list keeps out (docs/bans.md): of every port,
// or of one protocol and port. Bans are kept apart from the
// configuration, like a dynamic set's entries: not in the history, and
// not on probation.
type Ban struct {
ID int64 `json:"id"`
ListID int64 `json:"list_id"`
List string `json:"list"`
// Address is what is banned: an address, or an IPv6 prefix of the
// list's grouping ("2001:db8:1:2::/64").
Address string `json:"address"`
Protocol string `json:"protocol,omitempty"` // tcp or udp; "" every port
Port int `json:"port,omitempty"`
// Expires is when the ban ends (nil: it doesn't); Timeout the
// seconds it has left then.
Expires *time.Time `json:"expires,omitempty"`
Timeout int `json:"timeout,omitempty"`
Created time.Time `json:"created"`
// Who banned it (the user or token's name), Source the program's own
// name for why ("guard/ssh", "fail2ban/sshd"), Reason free text (a
// log line).
Who string `json:"who"`
Source string `json:"source,omitempty"`
Reason string `json:"reason,omitempty"`
}
// Prefix is the banned address as a prefix.
func (b *Ban) Prefix() (netip.Prefix, error) {
if p, err := netip.ParsePrefix(b.Address); err == nil {
return p, nil
}
a, err := netip.ParseAddr(b.Address)
if err != nil {
return netip.Prefix{}, err
}
return netip.PrefixFrom(a, a.BitLen()), nil
}
// BanAddress writes a banned prefix: an address alone when it is one
// address, else the prefix.
func BanAddress(p netip.Prefix) string {
if p.IsSingleIP() {
return p.Addr().String()
}
return p.String()
}
+184 -4
View File
@@ -1,6 +1,9 @@
package model
import "fmt"
import (
"fmt"
"strings"
)
// Chains rules are placed in.
const (
@@ -71,11 +74,49 @@ type AddressSet struct {
Dynamic bool `json:"dynamic,omitempty"`
// Timeout is a dynamic set's default entry lifetime in seconds (0:
// entries stay until removed).
Timeout int `json:"timeout,omitempty"`
Comment string `json:"comment,omitempty"`
Timeout int `json:"timeout,omitempty"`
// Jarl makes the set's entries the addresses of names in Jarl, which
// housecarld reads and keeps up to date (docs/jarl.md).
Jarl *JarlSource `json:"jarl,omitempty"`
Comment string `json:"comment,omitempty"`
}
func (s *AddressSet) GetID() int64 { return s.ID }
func (s *AddressSet) GetID() int64 { return s.ID }
// Runtime reports whether the set's entries come at run time (a dynamic
// set's, a Jarl set's) rather than from the configuration.
func (s *AddressSet) Runtime() bool { return s.Dynamic || s.Jarl != nil }
// JarlSource is where a set's addresses come from in Jarl: the A and AAAA
// records of names (CNAMEs followed), each in a view, kept only where
// Within allows.
type JarlSource struct {
Names []JarlName `json:"names"`
// Within are zone names and addresses (prefixes, ranges): an address
// is used only if it lies in a zone's connected networks or in one of
// the addresses.
Within []string `json:"within"`
}
// JarlName is a name in Jarl, in a view ("": the only view that has it).
type JarlName struct {
Name string `json:"name"` // "nas.example.com", without the final dot
View string `json:"view,omitempty"`
}
// String is "name" or "name@view".
func (n JarlName) String() string {
if n.View == "" {
return n.Name
}
return n.Name + "@" + n.View
}
// ParseJarlName reads "name" or "name@view".
func ParseJarlName(s string) JarlName {
name, view, _ := strings.Cut(strings.TrimSpace(s), "@")
return JarlName{Name: strings.TrimSuffix(strings.ToLower(strings.TrimSpace(name)), "."), View: strings.TrimSpace(view)}
}
func (s *AddressSet) Label() string { return s.Name }
// Service is a named list of ports ("tcp/22", "udp/60000-61000") and ICMP
@@ -274,6 +315,45 @@ func (r *Rule) Groups() (v4, v6 int) {
return v4, v6
}
// What becomes of what a rule's quota used when the rule changes
// (QuotaCarry).
const (
QuotaKept = "" // counted as before: carried as it is
QuotaSummed = "summed" // per address to a total: the addresses' sum
QuotaRegrouped = "regrouped" // per address in wider groups: added up per group
QuotaRestarts = "restarts" // can't be carried: starts again
)
// QuotaCarry says what becomes of what old's quota used when the rule
// becomes new (both the same rule, with quotas), and for QuotaRestarts
// why. The amount, over and the resets don't matter: only how it counts.
func QuotaCarry(old, new *Rule) (carry, why string) {
if old == nil || new == nil || old.Quota == nil || new.Quota == nil {
return QuotaKept, ""
}
was, is := old.Quota.PerAddress(), new.Quota.PerAddress()
words := map[string]string{"saddr": "per source", "daddr": "per destination", "": "in total"}
switch {
case was == "" && is == "":
return QuotaKept, ""
case is == "":
return QuotaSummed, ""
case was == "":
return QuotaRestarts, "a total can't be split by address, so each address starts at nothing"
case was != is:
return QuotaRestarts, "what was counted " + words[was] + " can't be counted " + words[is]
}
o4, o6 := old.Groups()
n4, n6 := new.Groups()
switch {
case o4 == n4 && o6 == n6:
return QuotaKept, ""
case n4 > o4 || n6 > o6:
return QuotaRestarts, "addresses are grouped more finely now, and a group's use can't be split"
}
return QuotaRegrouped, ""
}
// PortForward publishes a service on an inside host: incoming connections
// to a public address and port are sent on (DNAT) to To[:ToPort], let
// through the forward chain, and, with Hairpin, also work from inside.
@@ -353,3 +433,103 @@ func (r *Redirect) Label() string {
}
return fmt.Sprintf("redirect #%d", r.ID)
}
// Ban list scopes: what a list's bans keep the banned addresses from.
const (
// BanScopeFirewall: the firewall itself (the input chain).
BanScopeFirewall = "firewall"
// BanScopeHosts: the list's hosts, behind the firewall (the forward
// chain, where port forwards' addresses are already translated).
BanScopeHosts = "hosts"
// BanScopeEverything: the firewall and everything behind it.
BanScopeEverything = "everything"
)
// BanScopes lists the scopes.
var BanScopes = []string{BanScopeFirewall, BanScopeHosts, BanScopeEverything}
// Ban lists' defaults.
const (
DefaultBanTime = 3600
DefaultBanMaxTime = 7 * 24 * 3600
DefaultBanEntries = 10000
DefaultBanGroupV6 = 64
)
// BanList is where a program such as Guard, or fail2ban, bans addresses
// (docs/bans.md). The program names only the address, and perhaps a
// protocol and ports; the list says what the address is kept from. The
// bans themselves are kept apart from the configuration, like a dynamic
// set's entries.
type BanList struct {
ID int64 `json:"id"`
Name string `json:"name"`
Scope string `json:"scope"`
// Hosts are set names or addresses: what scope hosts protects.
Hosts []string `json:"hosts,omitempty"`
// Never are set names or addresses the list never bans.
Never []string `json:"never,omitempty"`
// DefaultTime is a ban's length in seconds when none is asked for
// (0: an hour); MaxTime the longest (0: a week; -1: no limit, so
// bans may be permanent).
DefaultTime int `json:"default_time,omitempty"`
MaxTime int `json:"max_time,omitempty"`
// MaxEntries limits the bans the list holds (0: 10000).
MaxEntries int `json:"max_entries,omitempty"`
// GroupV6 is the IPv6 prefix length banned for an address (0: /64,
// what one customer usually gets); IPv4 addresses are banned alone.
GroupV6 int `json:"group_v6,omitempty"`
Comment string `json:"comment,omitempty"`
Disabled bool `json:"disabled,omitempty"`
}
func (b *BanList) GetID() int64 { return b.ID }
func (b *BanList) Label() string { return b.Name }
// BanTime is the default ban length in seconds.
func (b *BanList) BanTime() int {
if b.DefaultTime > 0 {
return b.DefaultTime
}
return DefaultBanTime
}
// BanMaxTime is the longest ban in seconds, 0 for no limit.
func (b *BanList) BanMaxTime() int {
switch {
case b.MaxTime < 0:
return 0
case b.MaxTime > 0:
return b.MaxTime
}
return DefaultBanMaxTime
}
// Entries is the most bans the list holds.
func (b *BanList) Entries() int {
if b.MaxEntries > 0 {
return b.MaxEntries
}
return DefaultBanEntries
}
// V6Group is the IPv6 prefix length an address is banned with.
func (b *BanList) V6Group() int {
if b.GroupV6 > 0 {
return b.GroupV6
}
return DefaultBanGroupV6
}
// Covers reports whether the list's bans apply to a chain.
func (b *BanList) Covers(chain string) bool {
switch b.Scope {
case BanScopeFirewall:
return chain == ChainInput
case BanScopeHosts:
return chain == ChainForward
case BanScopeEverything:
return chain == ChainInput || chain == ChainForward
}
return false
}
+26
View File
@@ -0,0 +1,26 @@
package model
import "testing"
func TestQuotaCarry(t *testing.T) {
rule := func(per string, v4, v6 int) *Rule {
return &Rule{GroupV4: v4, GroupV6: v6, Quota: &Quota{Bytes: 1, PerSource: per == "src", PerDestination: per == "dst"}}
}
for _, c := range []struct {
old, new *Rule
want string
}{
{rule("", 0, 0), rule("", 0, 0), QuotaKept},
{rule("src", 0, 0), rule("src", 32, 64), QuotaKept}, // the defaults, written out
{rule("src", 0, 0), rule("", 0, 0), QuotaSummed},
{rule("", 0, 0), rule("src", 0, 0), QuotaRestarts},
{rule("src", 0, 0), rule("dst", 0, 0), QuotaRestarts},
{rule("src", 0, 0), rule("src", 24, 56), QuotaRegrouped},
{rule("src", 0, 0), rule("src", 24, 128), QuotaRestarts},
{nil, rule("src", 0, 0), QuotaKept},
} {
if got, _ := QuotaCarry(c.old, c.new); got != c.want {
t.Errorf("%+v -> %+v: %q, want %q", c.old, c.new, got, c.want)
}
}
}
+5 -4
View File
@@ -1,7 +1,8 @@
package model
// FamilyRefs are a reference list's part for one address family: names of
// sets that have entries of the family (dynamic sets always do), and
// sets that have entries of the family (dynamic and Jarl sets always
// do), and
// literal entries.
type FamilyRefs struct {
Sets []string
@@ -38,10 +39,10 @@ func (st *State) SplitRefs(refs []string) (v4, v6 FamilyRefs) {
return v4, v6
}
// Families reports which families a set holds: a dynamic set may get
// either at run time.
// Families reports which families a set holds: a dynamic or Jarl set may
// get either at run time.
func (s *AddressSet) Families() (v4, v6 bool) {
if s.Dynamic {
if s.Runtime() {
return true, true
}
for _, e := range s.Entries {
+28 -2
View File
@@ -20,6 +20,7 @@ type State struct {
Forwards []*PortForward `json:"forwards"`
SNATs []*SourceNAT `json:"snats"`
Redirects []*Redirect `json:"redirects"`
BanLists []*BanList `json:"ban_lists"`
}
// Object kinds, for changes and the history.
@@ -32,10 +33,11 @@ const (
KindForward = "forward"
KindSNAT = "snat"
KindRedirect = "redirect"
KindBanList = "banlist"
)
// Kinds lists the object kinds (not settings) in display order.
var Kinds = []string{KindZone, KindSet, KindService, KindRule, KindForward, KindSNAT, KindRedirect}
var Kinds = []string{KindZone, KindSet, KindService, KindRule, KindForward, KindSNAT, KindRedirect, KindBanList}
// Object is anything in the State with an ID.
type Object interface {
@@ -73,6 +75,9 @@ func (st *State) Fill() {
if st.Redirects == nil {
st.Redirects = []*Redirect{}
}
if st.BanLists == nil {
st.BanLists = []*BanList{}
}
}
// Objects returns the objects of a kind.
@@ -107,6 +112,10 @@ func (st *State) Objects(kind string) []Object {
for _, o := range st.Redirects {
out = append(out, o)
}
case KindBanList:
for _, o := range st.BanLists {
out = append(out, o)
}
}
return out
}
@@ -141,6 +150,8 @@ func (st *State) Put(kind string, id int64, data json.RawMessage) error {
return put(&st.SNATs, id, data)
case KindRedirect:
return put(&st.Redirects, id, data)
case KindBanList:
return put(&st.BanLists, id, data)
}
return &unknownKind{kind}
}
@@ -187,6 +198,7 @@ func (st *State) Sort() {
sort.SliceStable(st.Zones, func(i, j int) bool { return st.Zones[i].Name < st.Zones[j].Name })
sort.SliceStable(st.Sets, func(i, j int) bool { return st.Sets[i].Name < st.Sets[j].Name })
sort.SliceStable(st.Services, func(i, j int) bool { return st.Services[i].Name < st.Services[j].Name })
sort.SliceStable(st.BanLists, func(i, j int) bool { return st.BanLists[i].Name < st.BanLists[j].Name })
sort.SliceStable(st.Rules, func(i, j int) bool {
a, b := st.Rules[i], st.Rules[j]
if a.Chain != b.Chain {
@@ -262,7 +274,7 @@ func (st *State) ByID(kind string, id int64) Object {
}
// Find finds an object of a kind by ID or name (zones, sets, services,
// port forwards) or by ID alone (rules, source NAT, redirects).
// port forwards, ban lists) or by ID alone (rules, source NAT, redirects).
func (st *State) Find(kind, ref string) Object {
for _, o := range st.Objects(kind) {
if itoa(o.GetID()) == ref {
@@ -290,6 +302,20 @@ func (st *State) Find(kind, ref string) Object {
return f
}
}
case KindBanList:
if b := st.BanList(ref); b != nil {
return b
}
}
return nil
}
// BanList finds a ban list by name.
func (st *State) BanList(name string) *BanList {
for _, b := range st.BanLists {
if strings.EqualFold(b.Name, name) {
return b
}
}
return nil
}
+156 -7
View File
@@ -15,6 +15,7 @@ import (
"errors"
"fmt"
"net"
"net/netip"
"os"
"os/exec"
"path/filepath"
@@ -189,6 +190,30 @@ func (r *Runner) DeleteElement(ctx context.Context, set, addr string) error {
return err
}
// AddBan puts a ban in force: added, or if it is there already, given
// its new time (nftables keeps an element's old timeout when it is added
// again, so it is added, deleted and added in one transaction).
func (r *Runner) AddBan(ctx context.Context, b render.Ban) error {
set := render.BanSet(b.List, b.Addr.Addr().Is6(), b.Proto != "")
e := render.BanElement(b)
t := ""
if b.Timeout > 0 {
t = fmt.Sprintf(" timeout %ds", b.Timeout)
}
script := fmt.Sprintf("add element inet %[1]s %[2]s { %[3]s }\ndelete element inet %[1]s %[2]s { %[3]s }\nadd element inet %[1]s %[2]s { %[3]s%[4]s }\n", render.Table, set, e, t)
_, err := r.run(ctx, script, "-f", "/dev/stdin")
return err
}
// DeleteBan lifts a ban. One already gone (it ran out) isn't an error.
func (r *Runner) DeleteBan(ctx context.Context, b render.Ban) error {
set := render.BanSet(b.List, b.Addr.Addr().Is6(), b.Proto != "")
e := render.BanElement(b)
script := fmt.Sprintf("add element inet %[1]s %[2]s { %[3]s }\ndelete element inet %[1]s %[2]s { %[3]s }\n", render.Table, set, e)
_, err := r.run(ctx, script, "-f", "/dev/stdin")
return err
}
// ---- reading JSON
// Table is what Housecarl reads back from its table: counters by mark
@@ -366,7 +391,7 @@ func ParseTable(data []byte) (*Table, error) {
continue
}
name := strings.TrimSuffix(strings.TrimSuffix(s.Name, "_v4"), "_v6")
if name == s.Name || strings.HasPrefix(s.Name, "meter_") || strings.HasPrefix(s.Name, "conn_") {
if name == s.Name || strings.HasPrefix(s.Name, "meter_") || strings.HasPrefix(s.Name, "conn_") || render.IsBanSet(s.Name) {
continue
}
for _, e := range s.Elem {
@@ -424,11 +449,27 @@ type Firewall struct {
restore *QuotaUsage
// adjust are quota changes the next apply makes (AdjustQuota).
adjust []QuotaAdjust
// quotaRules are the rules with a quota in the ruleset in force, by
// ID: what their usage was counted by (nil before the first apply).
quotaRules map[int64]*model.Rule
// Bans returns the active bans, which every apply puts back (the
// database keeps them; nil: none).
Bans func() []render.Ban
// JarlSets gives the Jarl sets' addresses (nil: none).
JarlSets JarlSets
// flowDevs are the flowtable's devices in the ruleset in force, by
// name, with their interface index then (0: left out, missing).
flowDevs map[string]int
}
// JarlSets are the Jarl sets' addresses (internal/jarlsets).
type JarlSets interface {
// Entries are the addresses for st's Jarl sets: set name → addresses.
Entries(st *model.State) map[string][]string
// Applied is told what an apply put in force.
Applied(entries map[string][]string)
}
// ifIndex is an interface's index, 0 if it doesn't exist (tests change it).
var ifIndex = func(name string) int {
if i, err := net.InterfaceByName(name); err == nil {
@@ -510,6 +551,73 @@ func (u *QuotaUsage) apply(adj []QuotaAdjust) {
}
}
// carry keeps what quotas used when their rules now count differently
// (model.QuotaCarry): per address to a total, the sum; to wider groups,
// the groups' sums; what can't be carried starts again. was are the
// rules in force before (nil: nothing to compare with).
func (u *QuotaUsage) carry(was map[int64]*model.Rule, st *model.State) {
for _, r := range st.Rules {
old := was[r.ID]
carry, _ := model.QuotaCarry(old, r)
if carry == model.QuotaKept {
continue
}
name := render.QuotaName(r.ID)
sets := []string{name + "_v4", name + "_v6"}
switch carry {
case model.QuotaSummed:
var sum int64
for _, set := range sets {
for _, e := range u.QuotaSets[set] {
sum += e.Used
}
}
if u.Quotas == nil {
u.Quotas = map[string]int64{}
}
u.Quotas[name] = sum
case model.QuotaRegrouped:
v4, v6 := r.Groups()
for _, set := range sets {
byKey := map[string]int64{}
var order []string
for _, e := range u.QuotaSets[set] {
a, err := netip.ParseAddr(e.Addr)
if err != nil {
continue
}
bits := v4
if a.Is6() {
bits = v6
}
p, _ := a.Prefix(bits)
k := p.Addr().String()
if _, ok := byKey[k]; !ok {
order = append(order, k)
}
byKey[k] += e.Used
}
if len(order) == 0 {
continue
}
var es []render.Element
for _, k := range order {
es = append(es, render.Element{Addr: k, Used: byKey[k]})
}
u.QuotaSets[set] = es
}
continue
}
// What was counted the other way goes: a total, or the entries.
if carry != model.QuotaSummed {
delete(u.Quotas, name)
}
for _, set := range sets {
delete(u.QuotaSets, set)
}
}
}
// ResetQuota starts a rule's quota again: the named quota, or the
// per-address quota sets (both families).
func (f *Firewall) ResetQuota(ctx context.Context, ruleID int64, perAddress bool) error {
@@ -529,21 +637,24 @@ func (f *Firewall) ResetQuota(ctx context.Context, ruleID int64, perAddress bool
return nil
}
func (f *Firewall) options() render.Options {
func (f *Firewall) options(st *model.State) render.Options {
o := render.Options{Shadow: f.Shadow, ShadowBefore: f.ShadowBefore}
if f.Interfaces != nil {
o.Interfaces = f.Interfaces()
}
if f.JarlSets != nil {
o.JarlSets = f.JarlSets.Entries(st)
}
return o
}
// Render builds the ruleset for st as Apply would.
func (f *Firewall) Render(st *model.State) *render.Ruleset { return render.Render(st, f.options()) }
func (f *Firewall) Render(st *model.State) *render.Ruleset { return render.Render(st, f.options(st)) }
// RenderEnforcing builds the ruleset for st as Apply would outside shadow
// mode.
func (f *Firewall) RenderEnforcing(st *model.State) *render.Ruleset {
o := f.options()
o := f.options(st)
o.Shadow, o.ShadowBefore = false, false
return render.Render(st, o)
}
@@ -551,7 +662,7 @@ func (f *Firewall) RenderEnforcing(st *model.State) *render.Ruleset {
// Check implements engine.Firewall. In shadow mode it checks the
// enforcing ruleset too, so that leaving shadow mode can't fail.
func (f *Firewall) Check(ctx context.Context, st *model.State) error {
if err := f.Runner.Check(ctx, render.Render(st, f.options())); err != nil {
if err := f.Runner.Check(ctx, render.Render(st, f.options(st))); err != nil {
return err
}
if f.Shadow {
@@ -566,7 +677,7 @@ func (f *Firewall) Check(ctx context.Context, st *model.State) error {
func (f *Firewall) Apply(ctx context.Context, st *model.State) error {
f.mu.Lock()
defer f.mu.Unlock()
opt := f.options()
opt := f.options(st)
var old *Table
if data, err := f.Runner.ListTable(ctx); err == nil {
if t, err := ParseTable(data); err == nil {
@@ -579,16 +690,31 @@ func (f *Firewall) Apply(ctx context.Context, st *model.State) error {
usage = old.QuotaUsage
}
usage.merge(f.restore)
usage.carry(f.quotaRules, st)
usage.apply(f.adjust)
f.adjust = nil
opt.Quotas, opt.QuotaSets = usage.Quotas, usage.QuotaSets
if f.Bans != nil {
opt.Bans = f.Bans()
}
if old != nil {
opt.DeviceSets = old.DeviceSets
}
if err := f.Runner.Apply(ctx, render.Render(st, opt)); err != nil {
return err
}
if f.JarlSets != nil {
f.JarlSets.Applied(opt.JarlSets)
}
f.restore = nil
f.quotaRules = map[int64]*model.Rule{}
for _, r := range st.Rules {
if r.Quota != nil {
c, q := *r, *r.Quota // only the quota and the grouping are read
c.Quota = &q
f.quotaRules[r.ID] = &c
}
}
f.flowDevs = map[string]int{}
for _, d := range st.Settings.FlowtableDevices {
if opt.Interfaces == nil || opt.Interfaces[d] {
@@ -628,7 +754,16 @@ func pruneCounters(base map[string]Counter, st *model.State) {
for _, x := range st.Redirects {
ids[fmt.Sprintf("d%d", x.ID)] = true
}
for _, x := range st.BanLists {
ids[render.BanMark(x.ID)] = true
}
for mark := range base {
if strings.HasPrefix(mark, "ban") && strings.Trim(mark[3:], "0123456789") == "" {
if !ids[mark] {
delete(base, mark)
}
continue
}
obj, _, _ := strings.Cut(mark, "-") // f3-hairpin, f3-6 are f3's
if len(obj) < 2 || !strings.ContainsRune("rfsd", rune(obj[0])) || strings.Trim(obj[1:], "0123456789") != "" {
continue // b-established, p-input
@@ -659,7 +794,13 @@ func (f *Firewall) SaveBoot(st *model.State) error {
if err := writeAtomic(FallbackFile(f.BootFile), fallback); err != nil {
return err
}
rs := render.Render(st, render.Options{Interfaces: map[string]bool{}, Shadow: f.Shadow, ShadowBefore: f.ShadowBefore})
opt := render.Options{Interfaces: map[string]bool{}, Shadow: f.Shadow, ShadowBefore: f.ShadowBefore}
if f.JarlSets != nil {
// The addresses last read from Jarl: Jarl may not be reachable yet
// at boot.
opt.JarlSets = f.JarlSets.Entries(st)
}
rs := render.Render(st, opt)
return writeAtomic(f.BootFile, rs.Text)
}
@@ -711,6 +852,14 @@ func (f *Firewall) AddElement(ctx context.Context, set, addr string, timeout int
return f.Runner.AddElement(ctx, set, addr, timeout)
}
// AddBan puts a ban in force (Runner.AddBan).
func (f *Firewall) AddBan(ctx context.Context, b render.Ban) error { return f.Runner.AddBan(ctx, b) }
// DeleteBan lifts a ban (Runner.DeleteBan).
func (f *Firewall) DeleteBan(ctx context.Context, b render.Ban) error {
return f.Runner.DeleteBan(ctx, b)
}
// DeleteElement removes an entry from a dynamic set.
func (f *Firewall) DeleteElement(ctx context.Context, set, addr string) error {
return f.Runner.DeleteElement(ctx, set, addr)
+173 -1
View File
@@ -3,6 +3,7 @@ package nft
import (
"context"
"fmt"
"net/netip"
"os"
"os/exec"
"path/filepath"
@@ -130,6 +131,60 @@ func TestFirewallInContainer(t *testing.T) {
}
}
// fakeJarlSets gives fixed Jarl set addresses.
type fakeJarlSets struct {
entries, applied map[string][]string
}
func (f *fakeJarlSets) Entries(*model.State) map[string][]string { return f.entries }
func (f *fakeJarlSets) Applied(e map[string][]string) { f.applied = e }
// TestJarlSetsInContainer: a Jarl set gets its addresses with each apply,
// and the new ones replace the old.
func TestJarlSetsInContainer(t *testing.T) {
image := os.Getenv("HOUSECARL_NFT_IMAGE")
if image == "" {
t.Skip("HOUSECARL_NFT_IMAGE not set")
}
name := "housecarl-nft-jarl-test"
exec.Command("docker", "rm", "-f", name).Run()
if out, err := exec.Command("docker", "run", "-d", "--name", name, "--cap-add", "NET_ADMIN", image, "sleep", "300").CombinedOutput(); err != nil {
t.Fatalf("starting the container: %v %s", err, out)
}
t.Cleanup(func() { exec.Command("docker", "rm", "-f", name).Run() })
js := &fakeJarlSets{entries: map[string][]string{"nas": {"10.0.0.5", "10.0.0.6", "2001:db8::5"}}}
fw := &Firewall{Runner: &Runner{Command: []string{"docker", "exec", "-i", name, "nft"}}, JarlSets: js}
ctx := context.Background()
st := model.NewState()
st.Zones = []*model.Zone{{ID: 1, Name: "lan", Interfaces: []string{"eth0"}}}
st.Sets = []*model.AddressSet{{ID: 2, Name: "nas", Jarl: &model.JarlSource{Names: []model.JarlName{{Name: "nas.example.com"}}, Within: []string{"lan"}}}}
st.Rules = []*model.Rule{{ID: 3, Chain: model.ChainOutput, Position: 1, Action: model.ActionAccept, Match: model.Match{Destinations: []string{"nas"}}}}
elems := func() string {
tb, err := fw.Table(ctx)
if err != nil {
t.Fatal(err)
}
var got []string
for _, e := range tb.Elements["nas"] {
got = append(got, e.Addr)
}
return strings.Join(got, ",")
}
if err := fw.Apply(ctx, st); err != nil {
t.Fatalf("apply: %v", err)
}
if got := elems(); got != "10.0.0.5,10.0.0.6,2001:db8::5" || len(js.applied["nas"]) != 3 {
t.Errorf("after apply: %s (applied %v)", got, js.applied)
}
js.entries = map[string][]string{"nas": {"10.0.0.7"}}
if err := fw.Apply(ctx, st); err != nil {
t.Fatalf("reapply: %v", err)
}
if got := elems(); got != "10.0.0.7" {
t.Errorf("after the addresses changed: %s", got)
}
}
// TestQuotasInContainer: quotas keep what they used across a reload,
// reset by hand, and come back after a reboot from what was saved.
func TestQuotasInContainer(t *testing.T) {
@@ -197,6 +252,21 @@ func TestQuotasInContainer(t *testing.T) {
if t2, d2 := used(); t2 > total-1000 || t2 < total-2000 || d2 != 0 {
t.Errorf("after giving back: total %d (was %d, gave 1000), per destination %d (was %d, reset)", t2, total, d2, dev)
}
// Counted in total now: what the addresses used carries over.
ping("127.0.0.3")
_, dev = used()
st.Rules[1].Quota.PerDestination = false
if err := fw.Apply(ctx, st); err != nil {
t.Fatal(err)
}
tb, _ := fw.Table(ctx)
if tb.Quotas["quota_r2"] < dev || dev == 0 {
t.Errorf("per destination to a total: %d, was %d", tb.Quotas["quota_r2"], dev)
}
st.Rules[1].Quota.PerDestination = true
if err := fw.Apply(ctx, st); err != nil {
t.Fatal(err)
}
// Reset by hand.
if err := fw.ResetQuota(ctx, 1, false); err != nil {
t.Fatal(err)
@@ -211,7 +281,7 @@ func TestQuotasInContainer(t *testing.T) {
// saved usage comes back.
ping("127.0.0.2")
ping("127.0.0.3")
tb, _ := fw.Table(ctx)
tb, _ = fw.Table(ctx)
saved := tb.QuotaUsage
exec.Command("docker", "exec", name, "nft", "delete", "table", "inet", "housecarl").Run()
fw.RestoreQuotas(&saved)
@@ -240,6 +310,45 @@ func TestQuotaAdjust(t *testing.T) {
}
}
// TestQuotaCarry: usage follows a rule counting its quota differently.
func TestQuotaCarry(t *testing.T) {
perSource := &model.Rule{ID: 2, Quota: &model.Quota{Bytes: 1 << 30, PerSource: true}}
usage := func() QuotaUsage {
return QuotaUsage{Quotas: map[string]int64{}, QuotaSets: map[string][]render.Element{
"quota_r2_v4": {{Addr: "10.0.0.1", Used: 100}, {Addr: "10.0.0.2", Used: 50}},
"quota_r2_v6": {{Addr: "2001:db8:0:1::", Used: 7}, {Addr: "2001:db8:0:2::", Used: 3}}}}
}
was := map[int64]*model.Rule{2: perSource}
with := func(f func(r *model.Rule)) *model.State {
r := *perSource
q := *perSource.Quota
r.Quota = &q
f(&r)
return &model.State{Rules: []*model.Rule{&r}}
}
u := usage()
u.carry(was, with(func(r *model.Rule) { r.Quota.PerSource = false }))
if u.Quotas["quota_r2"] != 160 || len(u.QuotaSets["quota_r2_v4"]) != 0 {
t.Errorf("to a total: %+v", u)
}
u = usage()
u.carry(was, with(func(r *model.Rule) { r.GroupV4, r.GroupV6 = 24, 48 }))
want := map[string][]render.Element{"quota_r2_v4": {{Addr: "10.0.0.0", Used: 150}}, "quota_r2_v6": {{Addr: "2001:db8::", Used: 10}}}
if !reflect.DeepEqual(u.QuotaSets, want) {
t.Errorf("wider groups: %+v", u.QuotaSets)
}
u = usage()
u.carry(was, with(func(r *model.Rule) { r.Quota.PerSource, r.Quota.PerDestination = false, true }))
if len(u.QuotaSets) != 0 {
t.Errorf("per destination now: %+v", u.QuotaSets)
}
u = usage()
u.carry(nil, with(func(r *model.Rule) { r.Quota.PerSource = false }))
if len(u.QuotaSets["quota_r2_v4"]) != 2 || u.Quotas["quota_r2"] != 0 {
t.Errorf("nothing to compare with (the first apply): %+v", u)
}
}
// TestDeviceCountsInContainer: the per-device counters are read back and
// survive a reload.
func TestDeviceCountsInContainer(t *testing.T) {
@@ -489,3 +598,66 @@ func TestPruneCounters(t *testing.T) {
t.Errorf("kept %v", got)
}
}
// TestBansInContainer: a ban goes in, a second one renews its time, a
// reload keeps the bans the database has, and lifting one already gone
// isn't an error.
func TestBansInContainer(t *testing.T) {
image := os.Getenv("HOUSECARL_NFT_IMAGE")
if image == "" {
t.Skip("HOUSECARL_NFT_IMAGE not set")
}
name := "housecarl-nft-bans"
exec.Command("docker", "rm", "-f", name).Run()
if out, err := exec.Command("docker", "run", "-d", "--name", name, "--cap-add", "NET_ADMIN", image, "sleep", "300").CombinedOutput(); err != nil {
t.Fatalf("starting the container: %v %s", err, out)
}
t.Cleanup(func() { exec.Command("docker", "rm", "-f", name).Run() })
var active []render.Ban
fw := &Firewall{Runner: &Runner{Command: []string{"docker", "exec", "-i", name, "nft"}}, Bans: func() []render.Ban { return active }}
ctx := context.Background()
st := model.NewState()
st.Zones = []*model.Zone{{ID: 1, Name: "wan", Interfaces: []string{"eth0"}, External: true}}
st.BanLists = []*model.BanList{{ID: 2, Name: "router", Scope: model.BanScopeFirewall}}
if err := fw.Apply(ctx, st); err != nil {
t.Fatalf("apply: %v", err)
}
list := func(set string) string {
out, _ := exec.Command("docker", "exec", name, "nft", "list", "set", "inet", "housecarl", set).CombinedOutput()
return string(out)
}
b := render.Ban{List: 2, Addr: netip.MustParsePrefix("203.0.113.9/32"), Timeout: 60}
if err := fw.AddBan(ctx, b); err != nil {
t.Fatal(err)
}
b.Timeout = 7200
if err := fw.AddBan(ctx, b); err != nil {
t.Fatalf("renewing: %v", err)
}
if got := list("ban_2_v4"); !strings.Contains(got, "203.0.113.9 timeout 2h") {
t.Errorf("renewed ban: %s", got)
}
svc := render.Ban{List: 2, Addr: netip.MustParsePrefix("2001:db8:1:2::/64"), Proto: "tcp", Port: 22}
if err := fw.AddBan(ctx, svc); err != nil {
t.Fatal(err)
}
active = []render.Ban{svc}
if err := fw.Apply(ctx, st); err != nil {
t.Fatalf("reapply: %v", err)
}
if got := list("ban_2_v6_svc"); !strings.Contains(got, "2001:db8:1:2:: . 6 . 22") {
t.Errorf("after a reload, the database's ban: %s", got)
}
if got := list("ban_2_v4"); strings.Contains(got, "203.0.113.9") {
t.Errorf("after a reload, a ban the database doesn't have: %s", got)
}
if err := fw.DeleteBan(ctx, svc); err != nil {
t.Fatal(err)
}
if err := fw.DeleteBan(ctx, svc); err != nil {
t.Errorf("lifting a ban already gone: %v", err)
}
if got := list("ban_2_v6_svc"); strings.Contains(got, "2001:db8") {
t.Errorf("lifted ban: %s", got)
}
}
+139
View File
@@ -0,0 +1,139 @@
package render
import (
"fmt"
"net/netip"
"strings"
"housecarl/internal/model"
)
// Ban is an active ban, for rendering (Options.Bans): its list, the
// address banned (an IPv4 address, or an IPv6 prefix of the list's
// grouping), and for a ban of one port its protocol and port. Timeout is
// the seconds it has left (0: it doesn't end).
type Ban struct {
List int64
Addr netip.Prefix
Proto string // "tcp", "udp", or "" for every port
Port int
Timeout int
}
// BanSet is the name of a ban list's set for a family, and with svc
// true its set of bans of one port.
func BanSet(list int64, v6, svc bool) string {
name := fmt.Sprintf("ban_%d_v4", list)
if v6 {
name = fmt.Sprintf("ban_%d_v6", list)
}
if svc {
name += "_svc"
}
return name
}
// BanMark is the mark (comment "hc:ban3") of a ban list's rules.
func BanMark(list int64) string { return fmt.Sprintf("ban%d", list) }
// BanElement is the set element of a ban: "203.0.113.9", or
// "203.0.113.9 . tcp . 22".
func BanElement(b Ban) string {
e := b.Addr.Addr().String()
if b.Proto != "" {
e += fmt.Sprintf(" . %s . %d", b.Proto, b.Port)
}
return e
}
// IsBanSet reports whether a set is a ban list's.
func IsBanSet(name string) bool { return strings.HasPrefix(name, "ban_") }
// banSets declares the ban lists' sets.
func (r *renderer) banSets() {
for _, b := range r.st.BanLists {
if b.Disabled {
continue
}
obj := "ban list " + b.Name
for _, v6 := range []bool{false, true} {
r.w.emit(obj, "\tset %s { type %s; flags timeout; }", BanSet(b.ID, v6, false), addrType(v6))
r.w.emit(obj, "\tset %s { type %s . inet_proto . inet_service; flags timeout; }", BanSet(b.ID, v6, true), addrType(v6))
}
}
}
// banRules writes the drops of the ban lists that cover a chain, ahead
// of everything that accepts: a banned address's established connections
// are cut too.
func (r *renderer) banRules(chain string) {
for _, b := range r.st.BanLists {
if b.Disabled || !b.Covers(chain) {
continue
}
obj := "ban list " + b.Name
d4, d6 := r.st.SplitRefs(b.Hosts)
for _, fam := range []string{"v4", "v6"} {
dsts := []string{""}
if b.Scope == model.BanScopeHosts {
refs := d4
if fam == "v6" {
refs = d6
}
var ok bool
if dsts, ok = addrAlts(fam, "daddr", refs, true, false); !ok {
continue
}
}
key := ipKeyword(fam) + " saddr"
if fam == "v6" && b.V6Group() < 128 {
key += " and " + v6Mask(b.V6Group())
}
v6 := fam == "v6"
for _, d := range dsts {
r.banDrop(obj, b.ID, strings.Join(nonEmpty([]string{d, key + " @" + BanSet(b.ID, v6, false)}), " "))
r.banDrop(obj, b.ID, strings.Join(nonEmpty([]string{d, key + " . meta l4proto . th dport @" + BanSet(b.ID, v6, true)}), " "))
}
}
}
}
// banDrop writes one drop of a ban list, or in shadow mode counts it.
func (r *renderer) banDrop(obj string, list int64, match string) {
if r.opt.Shadow {
r.w.emit(obj, "\t\t%s counter accept comment \"hc:%s %s\"", match, BanMark(list), Would("drop"))
return
}
r.w.emit(obj, "\t\t%s counter drop comment \"hc:%s\"", match, BanMark(list))
}
// v6Mask is the mask of an IPv6 prefix length: 64 → ffff:ffff:ffff:ffff::.
func v6Mask(bits int) string {
mask := make([]byte, 16)
for i := 0; i < bits; i++ {
mask[i/8] |= 0x80 >> (i % 8)
}
m, _ := netip.AddrFromSlice(mask)
return m.String()
}
// banElements adds the active bans, with the time each has left, so a
// reload keeps them.
func (r *renderer) banElements() {
for _, ban := range r.opt.Bans {
var list *model.BanList
for _, b := range r.st.BanLists {
if b.ID == ban.List && !b.Disabled {
list = b
}
}
if list == nil || !ban.Addr.IsValid() {
continue
}
timeout := ""
if ban.Timeout > 0 {
timeout = fmt.Sprintf(" timeout %ds", ban.Timeout)
}
r.w.emit("ban list "+list.Name, "add element inet %s %s { %s%s }", Table, BanSet(list.ID, ban.Addr.Addr().Is6(), ban.Proto != ""), BanElement(ban), timeout)
}
}
+33
View File
@@ -50,6 +50,11 @@ type Options struct {
// DeviceSets are the per-device traffic counters' entries
// (stats_up_v4 …), so a reload keeps them.
DeviceSets map[string][]Element
// Bans are the ban lists' active bans, added to their sets.
Bans []Ban
// JarlSets are the Jarl sets' addresses (set name → addresses, both
// families), as Jarl gave them and within allows.
JarlSets map[string][]string
// Shadow renders for shadow mode: nothing is blocked, rejected,
// translated or changed (see the package comment).
Shadow bool
@@ -137,6 +142,7 @@ func (r *renderer) render() {
w.emit("", "delete table inet %s", Table)
w.emit("", "table inet %s {", Table)
r.sets()
r.banSets()
r.deviceSets()
devices := r.flowtableDevices()
if len(devices) > 0 {
@@ -165,6 +171,7 @@ func (r *renderer) render() {
w.line = base + chains.line
w.emit("", "}")
r.elements()
r.banElements()
}
// ---- sets
@@ -182,6 +189,10 @@ func (r *renderer) sets() {
}
continue
}
if s.Jarl != nil {
r.jarlSet(s)
continue
}
has4, has6 := s.Families()
for _, v6 := range []bool{false, true} {
if v6 && !has6 || !v6 && !has4 {
@@ -193,6 +204,26 @@ func (r *renderer) sets() {
}
}
// jarlSet declares a Jarl set's two sets with the addresses it has now
// (none yet: empty, so its rules match nothing).
func (r *renderer) jarlSet(s *model.AddressSet) {
for _, v6 := range []bool{false, true} {
var elems []string
for _, e := range r.opt.JarlSets[s.Name] {
if a, err := model.ParseAddress(e); err == nil && a.IsV6() == v6 {
elems = append(elems, a.String())
}
}
if len(elems) == 0 {
r.w.emit("set "+s.Name, "\tset %s { type %s; }", setName(s.Name, v6), addrType(v6))
continue
}
// Single addresses: no interval flag, so they stay as they are
// (auto-merge would join neighbours into ranges).
r.w.emit("set "+s.Name, "\tset %s { type %s; elements = { %s } }", setName(s.Name, v6), addrType(v6), strings.Join(elems, ", "))
}
}
func (r *renderer) elements() {
names := make([]string, 0, len(r.opt.Elements))
for n := range r.opt.Elements {
@@ -382,6 +413,7 @@ func (r *renderer) input() {
if b.Loopback {
r.builtin("loopback", "iifname \"lo\" accept")
}
r.banRules(model.ChainInput)
// Rules that count traffic see every packet: before the established
// connections are accepted.
r.rules(model.ChainInput, true)
@@ -411,6 +443,7 @@ func (r *renderer) forward(flowtable bool) {
st := r.st
b := st.Settings.Builtins
r.chain("forward", "forward", "filter", st.Settings.ForwardPolicy)
r.banRules(model.ChainForward)
ext := st.ExternalInterfaces()
// Shadow mode changes no packet.
if b.MSSClamp && len(ext) > 0 && !r.opt.Shadow {
+89 -3
View File
@@ -3,6 +3,7 @@ package render
import (
"context"
"fmt"
"net/netip"
"os"
"os/exec"
"strings"
@@ -34,6 +35,8 @@ func router() *model.State {
{ID: next(), Name: "wg_admins", Entries: []string{"10.50.50.3", "10.50.50.6"}},
{ID: next(), Name: "dns_servers", Entries: []string{"10.0.10.5", "217.155.8.194", "fdab:abab:abab:10::5", "2a02:8013:7f3d:10::5"}},
{ID: next(), Name: "guests", Dynamic: true, Timeout: 3600},
{ID: next(), Name: "nas", Jarl: &model.JarlSource{Names: []model.JarlName{{Name: "nas.example.com", View: "internal"}}, Within: []string{"lan"}}},
{ID: next(), Name: "printers", Jarl: &model.JarlSource{Names: []model.JarlName{{Name: "printer.example.com"}}, Within: []string{"lan"}}},
}
st.Services = []*model.Service{
{ID: next(), Name: "jarl", Entries: []string{"tcp/8053"}},
@@ -66,6 +69,7 @@ func router() *model.State {
rule(model.ChainForward, model.Rule{Match: model.Match{InZones: []string{"wan"}}, Action: model.ActionDrop, Comment: "nothing else in"})
rule(model.ChainForward, model.Rule{Match: model.Match{Destinations: []string{"wg_net"}}, Action: model.ActionDrop})
rule(model.ChainForward, model.Rule{Match: model.Match{Services: []string{"ping"}}})
rule(model.ChainForward, model.Rule{Match: model.Match{InZones: []string{"wg"}, Destinations: []string{"nas", "printers"}}, Comment: "from Jarl"})
rule(model.ChainForward, model.Rule{Match: model.Match{Sources: []string{"wg_net", "trusted", "guests"}}})
// Bandwidth: a byte rate, per destination; per /24; quotas, total and
// per device.
@@ -95,10 +99,31 @@ func router() *model.State {
st.Redirects = []*model.Redirect{
{ID: next(), Position: 1, Match: model.Match{InZones: []string{"lan"}, NotDests: true, Destinations: []string{"10.0.10.5"}, Services: []string{"udp/53"}}, Comment: "force DNS"},
}
st.BanLists = []*model.BanList{
{ID: next(), Name: "router", Scope: model.BanScopeFirewall, Never: []string{"trusted"}},
{ID: next(), Name: "web", Scope: model.BanScopeHosts, Hosts: []string{"10.0.10.57", "2a02:8013:7f3d:10::57", "dns_servers"}},
{ID: next(), Name: "edge", Scope: model.BanScopeEverything, GroupV6: 48},
{ID: next(), Name: "off", Scope: model.BanScopeEverything, Disabled: true},
}
st.Sort()
return st
}
// jarlOptions are the Jarl sets' addresses.
var jarlOptions = map[string][]string{"nas": {"10.0.10.20", "10.0.10.21", "2a02:8013:7f3d:10::20"}, "printers": {}}
// banOptions are active bans in each ban list, of every port and of one.
func banOptions(st *model.State) []Ban {
var out []Ban
for _, b := range st.BanLists {
out = append(out,
Ban{List: b.ID, Addr: netip.MustParsePrefix("203.0.113.9/32"), Timeout: 3000},
Ban{List: b.ID, Addr: netip.MustParsePrefix("2001:db8:1:2::/64"), Proto: "tcp", Port: 22},
Ban{List: b.ID, Addr: netip.MustParsePrefix("198.51.100.7/32"), Proto: "udp", Port: 53, Timeout: 60})
}
return out
}
// quotaOptions carries quota usage, as a reload does: the guest quota's,
// and two devices' of the per-device quota.
func quotaOptions(st *model.State) (map[string]int64, map[string][]Element) {
@@ -118,13 +143,19 @@ func quotaOptions(st *model.State) (map[string]int64, map[string][]Element) {
func TestRenderRouter(t *testing.T) {
st := router()
q, qs := quotaOptions(st)
rs := Render(st, Options{Elements: map[string][]Element{"guests": {{Addr: "10.0.30.5", Timeout: 1700}}}, Quotas: q, QuotaSets: qs})
rs := Render(st, Options{Elements: map[string][]Element{"guests": {{Addr: "10.0.30.5", Timeout: 1700}}}, Quotas: q, QuotaSets: qs, JarlSets: jarlOptions})
text := rs.Text
for _, want := range []string{
"table inet housecarl {}\ndelete table inet housecarl\ntable inet housecarl {",
`set trusted_v4 { type ipv4_addr; flags interval; auto-merge; elements = { 10.0.10.0/24, 10.20.30.0/24, 10.50.50.0/24 } }`,
`set trusted_v6 { type ipv6_addr; flags interval; auto-merge; elements = { 2a02:8013:7f3d:10::/64 } }`,
`set guests_v4 { type ipv4_addr; flags timeout; timeout 3600s; }`,
// Jarl sets: their addresses now, or empty.
`set nas_v4 { type ipv4_addr; elements = { 10.0.10.20, 10.0.10.21 } }`,
`set nas_v6 { type ipv6_addr; elements = { 2a02:8013:7f3d:10::20 } }`,
"set printers_v4 { type ipv4_addr; }\n",
`iifname "wg0" ip daddr @nas_v4 counter accept`,
`iifname "wg0" ip6 daddr @printers_v6 counter accept`,
`iifname "wan" ip saddr { 0.0.0.0/8,`,
`iifname "wan" ip6 saddr { ::1/128, ::/128, fc00::/7, 2001:db8::/32, ff00::/8, 2a02:8013:7f3d::/48 } counter drop`,
`oifname "wan" tcp flags syn tcp option maxseg size set rt mtu comment "hc:b-mss-clamp"`,
@@ -257,9 +288,9 @@ func TestRenderedRulesetPassesNft(t *testing.T) {
q, qs := quotaOptions(st)
// Enforcing, and in shadow mode after and before other tables.
for _, mode := range []Options{{}, {Shadow: true}, {Shadow: true, ShadowBefore: true}} {
opt := Options{Elements: map[string][]Element{"guests": {{Addr: "10.0.30.5", Timeout: 1700}, {Addr: "2a02:8013:7f3d:30::5"}}}, Quotas: q, QuotaSets: qs,
opt := Options{Elements: map[string][]Element{"guests": {{Addr: "10.0.30.5", Timeout: 1700}, {Addr: "2a02:8013:7f3d:30::5"}}}, Quotas: q, QuotaSets: qs, Bans: banOptions(st),
DeviceSets: map[string][]Element{"stats_up_v4": {{Addr: "10.0.10.5", Timeout: 3000, Packets: 7, Bytes: 7000}}, "stats_down_v6": {{Addr: "2001:db8:1:2::", Packets: 1, Bytes: 100}}},
Shadow: mode.Shadow, ShadowBefore: mode.ShadowBefore}
JarlSets: jarlOptions, Shadow: mode.Shadow, ShadowBefore: mode.ShadowBefore}
nftCheck(t, cmd, fmt.Sprintf("%s (shadow %v, before %v)", name, mode.Shadow, mode.ShadowBefore), Render(st, opt).Text)
}
}
@@ -277,3 +308,58 @@ func nftCheck(t *testing.T, cmd []string, name, text string) {
}
}
}
func TestRenderBans(t *testing.T) {
st := router()
id := map[string]int64{}
for _, b := range st.BanLists {
id[b.Name] = b.ID
}
text := Render(st, Options{Bans: banOptions(st)}).Text
f := func(format string, args ...any) string { return fmt.Sprintf(format, args...) }
r, w, e, off := id["router"], id["web"], id["edge"], id["off"]
for _, want := range []string{
f("set ban_%d_v4 { type ipv4_addr; flags timeout; }", r),
f("set ban_%d_v6_svc { type ipv6_addr . inet_proto . inet_service; flags timeout; }", r),
// The firewall: in input, every port and one port, /64 for IPv6.
f(`ip saddr @ban_%d_v4 counter drop comment "hc:ban%d"`, r, r),
f(`ip saddr . meta l4proto . th dport @ban_%d_v4_svc counter drop comment "hc:ban%d"`, r, r),
f(`ip6 saddr and ffff:ffff:ffff:ffff:: @ban_%d_v6 counter drop comment "hc:ban%d"`, r, r),
f(`ip6 saddr and ffff:ffff:ffff:ffff:: . meta l4proto . th dport @ban_%d_v6_svc counter drop comment "hc:ban%d"`, r, r),
// Hosts: in forward, first, to the hosts (literals and sets).
f(`ip daddr 10.0.10.57 ip saddr @ban_%d_v4 counter drop`, w),
f(`ip daddr @dns_servers_v4 ip saddr . meta l4proto . th dport @ban_%d_v4_svc counter drop`, w),
f(`ip6 daddr 2a02:8013:7f3d:10::57 ip6 saddr and ffff:ffff:ffff:ffff:: @ban_%d_v6 counter drop`, w),
// Everything: input and forward, /48.
f(`ip6 saddr and ffff:ffff:ffff:: @ban_%d_v6 counter drop`, e),
// The bans, with their time.
f("add element inet housecarl ban_%d_v4 { 203.0.113.9 timeout 3000s }", r),
f("add element inet housecarl ban_%d_v6_svc { 2001:db8:1:2:: . tcp . 22 }", w),
f("add element inet housecarl ban_%d_v4_svc { 198.51.100.7 . udp . 53 timeout 60s }", e),
} {
if !strings.Contains(text, want) {
t.Errorf("no %q in\n%s", want, text)
}
}
if n := strings.Count(text, f(`@ban_%d_v4 counter drop`, e)); n != 2 {
t.Errorf("scope everything: %d rules of every port in IPv4, want 2 (input and forward)", n)
}
if n := strings.Count(text, f(`@ban_%d_v4 counter drop`, w)); n != 2 {
t.Errorf("scope hosts: %d IPv4 rules of every port, want 2 (forward: the literal host and the set)", n)
}
// Ahead of the accepts: the first rules after loopback, and of forward.
in, _, _ := strings.Cut(text[strings.Index(text, "chain input"):], "established")
fw, _, _ := strings.Cut(text[strings.Index(text, "chain forward"):], "maxseg")
for _, part := range []string{in, fw} {
if strings.Contains(part, " accept comment \"hc:r") {
t.Errorf("a rule accepts before the bans:\n%s", part)
}
}
if strings.Contains(text, f("ban_%d_", off)) {
t.Errorf("a disabled list is rendered")
}
shadow := Render(st, Options{Shadow: true, Bans: banOptions(st)}).Text
if want := f(`ip saddr @ban_%d_v4 counter accept comment "hc:ban%d [would drop]"`, r, r); !strings.Contains(shadow, want) {
t.Errorf("shadow: no %q", want)
}
}
+4
View File
@@ -102,6 +102,8 @@ func (s *Server) authenticate(r *http.Request) (*auth.Principal, *model.Session,
p, err := s.tokenPrincipal(r.Context(), strings.TrimSpace(tok), ip, r.UserAgent())
if err != nil {
s.limiter.fail("ip:" + ip)
// For Guard and fail2ban: the address first, so a name can't fake it.
s.log.Warn("token refused", "ip", ip, "reason", err.Error())
s.eng.Audit(r.Context(), &auth.Principal{IP: ip, Method: "token"}, "auth.token_failed", "", "failure", err.Error())
return nil, nil, &httpError{http.StatusUnauthorized, "unauthenticated", "invalid API token: " + err.Error()}
}
@@ -246,6 +248,8 @@ func (s *Server) login(w http.ResponseWriter, r *http.Request) error {
if err == nil && u.Disabled && ok {
reason = "account disabled"
}
// For Guard and fail2ban: the address first, so a name can't fake it.
s.log.Warn("login failed", "ip", ip, "user", name, "reason", reason)
s.eng.Audit(r.Context(), who, "auth.login_failed", "user "+name, "failure", reason)
return &httpError{http.StatusUnauthorized, "login_failed", "wrong username or password"}
}
+154
View File
@@ -0,0 +1,154 @@
package server
import (
"bytes"
"context"
"log/slog"
"strings"
"testing"
"housecarl/internal/api"
"housecarl/internal/model"
"housecarl/internal/store"
)
// TestBans: a program with a token for one ban list bans addresses: of
// a service's ports, every port, or numbers; renewed when banned again;
// IPv6 by /64; never the inside, the firewall's own, or the list's
// never; times cut to the list's longest; the address's connections the
// ban drops are closed; and lifted by ID or address.
func TestBans(t *testing.T) {
e := newEnv(t)
e.call(e.token, "POST", "/zones", model.Zone{Name: "wan", Interfaces: []string{"wan"}, External: true}, nil)
e.call(e.token, "POST", "/zones", model.Zone{Name: "lan", Interfaces: []string{"eth1"}}, nil)
for _, b := range []model.BanList{
{Name: "web", Scope: model.BanScopeHosts, Hosts: []string{"10.0.10.57"}, Never: []string{"198.51.100.0/24"}, MaxTime: 3600},
{Name: "router", Scope: model.BanScopeFirewall},
} {
if code := e.call(e.token, "POST", "/banlists", b, nil); code != 201 {
t.Fatalf("ban list %s: %d", b.Name, code)
}
}
fc := &fakeConns{list: []api.Connection{
{Family: "ipv4", Protocol: "tcp", Source: "203.0.113.9", SourcePort: 40000, Destination: "217.155.8.194", DestinationPort: 2222, DNATTo: "10.0.10.57", DNATPort: 22},
{Family: "ipv4", Protocol: "tcp", Source: "203.0.113.9", SourcePort: 40001, Destination: "10.0.10.58", DestinationPort: 22},
{Family: "ipv4", Protocol: "tcp", Source: "203.0.113.9", SourcePort: 40002, Destination: "217.155.8.194", DestinationPort: 22},
}}
e.srv.SetConntrack(fc)
var tok api.Token
if code := e.call(e.token, "POST", "/me/tokens", api.TokenInput{Name: "guard", Grants: []string{"bans:write:web"}}, &tok); code != 201 {
t.Fatalf("token: %d", code)
}
ban := func(in api.BanInput, want int) api.BanResult {
t.Helper()
var out api.BanResult
var eb api.ErrorBody
code := e.call(tok.Secret, "POST", "/banlists/web/bans", in, &out)
if code != want {
e.call(tok.Secret, "POST", "/banlists/web/bans", in, &eb)
t.Errorf("ban %+v: %d, want %d (%+v)", in, code, want, eb)
}
return out
}
out := ban(api.BanInput{Address: "203.0.113.9", Ports: []byte(`"ssh"`), Time: 600, Source: "guard/ssh", Reason: "Failed password"}, 201)
if len(out.Bans) != 1 || out.Bans[0].Protocol != "tcp" || out.Bans[0].Port != 22 || out.Bans[0].Existing || out.Bans[0].Timeout != 600 {
t.Errorf("ssh ban: %+v", out)
}
if out.Closed != 1 || len(fc.closed) != 1 || fc.closed[0].DNATTo != "10.0.10.57" {
t.Errorf("closed %d: %+v (only the one to the list's host)", out.Closed, fc.closed)
}
if _, ok := e.fw.bans[`ban_`+itoa(out.Bans[0].ListID)+`_v4_svc 203.0.113.9 . tcp . 22`]; !ok {
t.Errorf("not in force: %+v", e.fw.bans)
}
if again := ban(api.BanInput{Address: "203.0.113.9", Ports: []byte(`22`), Protocol: "tcp"}, 201); len(again.Bans) != 1 || !again.Bans[0].Existing {
t.Errorf("banned again: %+v", again)
}
if v6 := ban(api.BanInput{Address: "2001:db8:1:2::77", Time: 99999}, 201); len(v6.Bans) != 1 || v6.Bans[0].Address != "2001:db8:1:2::/64" ||
v6.Bans[0].Protocol != "" || v6.Bans[0].Timeout != 3600 {
t.Errorf("IPv6, every port, cut to an hour: %+v", v6)
}
if n := ban(api.BanInput{Address: "::ffff:203.0.113.10", Ports: []byte(`53`)}, 201); len(n.Bans) != 2 || n.Bans[0].Address != "203.0.113.10" {
t.Errorf("a number without a protocol is tcp and udp: %+v", n)
}
if n := ban(api.BanInput{Address: "203.0.113.11", Protocol: "tcp", Ports: []byte(`["http", "https"]`)}, 201); len(n.Bans) != 2 {
t.Errorf("two services: %+v", n)
}
if n := ban(api.BanInput{Address: "203.0.113.12", Protocol: "tcp", Ports: []byte(`"0:65535"`)}, 201); len(n.Bans) != 1 || n.Bans[0].Protocol != "" {
t.Errorf("fail2ban's every port: %+v", n)
}
ban(api.BanInput{Address: "203.0.113.13", Ports: []byte(`"bogus"`)}, 400)
ban(api.BanInput{Address: "203.0.113.13", Protocol: "tcp"}, 400)
ban(api.BanInput{Address: "not-an-address"}, 400)
for _, a := range []string{"198.51.100.4", "10.0.10.99", "217.155.8.194", "127.0.0.1", "fe80::1", "192.168.5.5"} {
var eb api.ErrorBody
if code := e.call(tok.Secret, "POST", "/banlists/web/bans", api.BanInput{Address: a}, &eb); code != 422 || eb.Error.Code != "never_ban" {
t.Errorf("never ban %s: %d %+v", a, code, eb)
}
}
if code := e.call(tok.Secret, "POST", "/banlists/router/bans", api.BanInput{Address: "203.0.113.9"}, nil); code != 403 {
t.Errorf("another list: %d", code)
}
// The program may read its list, and only its list.
var bl model.BanList
if code := e.call(tok.Secret, "GET", "/banlists/web", nil, &bl); code != 200 || bl.V6Group() != 64 {
t.Errorf("reading its list: %d %+v", code, bl)
}
var lists []model.BanList
if e.call(tok.Secret, "GET", "/banlists", nil, &lists); len(lists) != 1 || lists[0].Name != "web" {
t.Errorf("lists: %+v", lists)
}
if code := e.call(tok.Secret, "GET", "/banlists/router", nil, nil); code != 403 {
t.Errorf("reading another list: %d", code)
}
var bans []model.Ban
e.call(tok.Secret, "GET", "/banlists/web/bans", nil, &bans)
if len(bans) != 7 || bans[0].List != "web" || bans[0].Who != "admin" || bans[0].Source != "guard/ssh" {
t.Errorf("bans: %d %+v", len(bans), bans)
}
var removed map[string]int
if code := e.call(tok.Secret, "DELETE", "/banlists/web/bans?address=2001:db8:1:2::5", nil, &removed); code != 200 || removed["removed"] != 1 {
t.Errorf("lift by address: %d %+v", code, removed)
}
if code := e.call(tok.Secret, "DELETE", "/banlists/web/bans/"+itoa(bans[0].ID), nil, &removed); code != 200 || removed["removed"] != 1 {
t.Errorf("lift by ID: %d %+v", code, removed)
}
if code := e.call(tok.Secret, "DELETE", "/banlists/web/bans?address=203.0.113.200", nil, nil); code != 404 {
t.Errorf("lifting what isn't banned: %d", code)
}
if _, ok := e.fw.bans[`ban_`+itoa(bans[0].ListID)+`_v4_svc 203.0.113.9 . tcp . 22`]; ok {
t.Errorf("still in force after lifting")
}
evs, _ := e.srv.store.ListAudit(context.Background(), store.AuditFilter{Action: "ban."})
if len(evs) < 10 || !strings.Contains(evs[len(evs)-1].Detail, "guard/ssh: Failed password") {
t.Errorf("audit: %d %+v", len(evs), evs[len(evs)-1])
}
// The list's name is what programs know it by.
if code := e.call(e.token, "PATCH", "/banlists/web", map[string]any{"name": "web2"}, nil); code != 409 {
t.Errorf("renaming a ban list: %d", code)
}
// Every apply puts the bans back; a deleted list's go.
if got := ActiveBans(context.Background(), e.srv.store); len(got) != 5 {
t.Errorf("active bans: %d", len(got))
}
if code := e.call(e.token, "DELETE", "/banlists/web", nil, nil); code != 200 {
t.Errorf("deleting the list: %d", code)
}
if got := ActiveBans(context.Background(), e.srv.store); len(got) != 0 {
t.Errorf("a deleted list's bans: %d", len(got))
}
}
// TestFailedLoginLogged: a failed login and a refused token are logged
// with the address first, for Guard's and fail2ban's patterns.
func TestFailedLoginLogged(t *testing.T) {
e := newEnv(t)
var buf bytes.Buffer
e.srv.log = slog.New(slog.NewTextHandler(&buf, nil))
e.call("", "POST", "/auth/login", api.Login{Username: `x ip=8.8.8.8`, Password: "wrong"}, nil)
e.call("housecarl_aaaaaaaaaaaaaaaa_bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", "GET", "/status", nil, nil)
for _, want := range []string{`msg="login failed" ip=127.0.0.1 user="x ip=8.8.8.8"`, `msg="token refused" ip=127.0.0.1 reason=`} {
if !strings.Contains(buf.String(), want) {
t.Errorf("no %q in\n%s", want, buf.String())
}
}
}
+139
View File
@@ -0,0 +1,139 @@
package server
import (
"encoding/json"
"fmt"
"strings"
"testing"
udiff "github.com/aymanbagabas/go-udiff"
"housecarl/internal/api"
"housecarl/internal/model"
)
// TestBlockHeaders: each hunk says which chain it is in, even when the
// chain's own line is far above it.
func TestBlockHeaders(t *testing.T) {
var b strings.Builder
b.WriteString("table inet housecarl {\n\tset lan_v4 { type ipv4_addr; }\n\tchain input {\n")
for i := 0; i < 20; i++ {
b.WriteString("\t\tcounter comment \"in\"\n")
}
b.WriteString("\t}\n\tchain forward {\n")
for i := 0; i < 20; i++ {
b.WriteString("\t\tcounter comment \"fw\"\n")
}
b.WriteString("\t}\n}\n")
now := b.String()
then := strings.Replace(now, "\t\tcounter comment \"in\"\n\t}", "\t\tcounter comment \"in\"\n\t\taccept comment \"new in\"\n\t}", 1)
then = strings.Replace(then, "\t\tcounter comment \"fw\"\n\t}", "\t\taccept comment \"new fw\"\n\t}", 1)
diff := blockHeaders(udiff.Unified("a", "b", now, then), now, then)
var heads []string
for _, l := range strings.Split(diff, "\n") {
if strings.HasPrefix(l, "@@") {
heads = append(heads, l)
}
}
if len(heads) != 2 || !strings.HasSuffix(heads[0], "@@ chain input") || !strings.HasSuffix(heads[1], "@@ chain forward") {
t.Fatalf("hunk headers %q, want chain input then chain forward\n%s", heads, diff)
}
if blockHeaders("", now, then) != "" {
t.Fatal("no diff should stay empty")
}
}
// TestHistoryDiff: an entry's diff is its own change, rebuilt from the
// current configuration however many entries came after it.
func TestHistoryDiff(t *testing.T) {
e := newEnv(t)
var mr api.MutationResponse
e.call(e.token, "POST", "/zones", model.Zone{Name: "lan", Interfaces: []string{"eth1"}}, nil)
e.call(e.token, "POST", "/rules", model.Rule{Chain: "input", Match: model.Match{Services: []string{"ssh"}}, Comment: "first"}, &mr)
first := mr.HistoryID
e.call(e.token, "POST", "/rules", model.Rule{Chain: "forward", Match: model.Match{Services: []string{"http"}}, Comment: "second"}, nil)
e.call(e.token, "PATCH", "/settings", map[string]any{"builtins": map[string]any{"drop_invalid": true}}, nil)
var out struct{ Diff string }
if code := e.call(e.token, "GET", "/history/"+itoa(first)+"/diff", nil, &out); code != 200 {
t.Fatalf("diff: %d", code)
}
var added []string
for _, l := range strings.Split(out.Diff, "\n") {
if strings.HasPrefix(l, "+") && !strings.HasPrefix(l, "+++") {
added = append(added, l)
}
}
if len(added) == 0 || !strings.Contains(strings.Join(added, "\n"), ` first"`) || strings.Contains(out.Diff, ` second"`) || !strings.Contains(out.Diff, "@@ chain input") {
t.Errorf("entry %d's diff:\n%s", first, out.Diff)
}
if code := e.call(e.token, "GET", "/history/9999/diff", nil, nil); code != 404 {
t.Errorf("unknown entry: %d", code)
}
}
// TestHunkStarts: after a hunk with unchanged lines between its edits (a
// moved line), the later hunks' new sides start where they do.
func TestHunkStarts(t *testing.T) {
var a, b strings.Builder
for i := 1; i <= 20; i++ {
fmt.Fprintf(&a, "%d\n", i)
switch i {
case 4:
case 6:
b.WriteString("6\n4\n")
case 14:
default:
fmt.Fprintf(&b, "%d\n", i)
}
}
diff := hunkStarts(udiff.Unified("a", "b", a.String(), b.String()))
var heads []string
for _, l := range strings.Split(diff, "\n") {
if strings.HasPrefix(l, "@@") {
heads = append(heads, l)
}
}
// diff -u: the move nets nothing, the deleted 14 one line.
if want := []string{"@@ -1,9 +1,9 @@", "@@ -11,7 +11,6 @@"}; strings.Join(heads, "|") != strings.Join(want, "|") {
t.Fatalf("hunk headers %q, want %q\n%s", heads, want, diff)
}
}
// TestHistoryDiffUpdate: an update's diff is only what it changed; the
// object stepped back to its old form stays where it was.
func TestHistoryDiffUpdate(t *testing.T) {
e := newEnv(t)
e.call(e.token, "POST", "/zones", model.Zone{Name: "lan", Interfaces: []string{"eth1"}}, nil)
e.call(e.token, "POST", "/zones", model.Zone{Name: "wan", Interfaces: []string{"wan"}, External: true}, nil)
var ids []int64
for i, port := range []string{"8096", "443", "50300", "7777", "25"} {
var mr api.MutationResponse
e.call(e.token, "POST", "/forwards", map[string]any{"name": "f" + port, "ports": port, "to": fmt.Sprintf("10.0.10.%d", 30+i),
"public_addresses": []string{"203.0.113.1"}, "hairpin": true}, &mr)
var f model.PortForward
if err := json.Unmarshal(mr.Object, &f); err != nil {
t.Fatal(err)
}
ids = append(ids, f.ID)
}
var mr api.MutationResponse
if code := e.call(e.token, "PATCH", "/forwards/"+itoa(ids[2]), map[string]any{"hairpin": false}, &mr); code != 200 {
t.Fatalf("update: %d", code)
}
var out struct{ Diff string }
if code := e.call(e.token, "GET", "/history/"+itoa(mr.HistoryID)+"/diff", nil, &out); code != 200 {
t.Fatalf("diff: %d", code)
}
var changed []string
for _, l := range strings.Split(out.Diff, "\n") {
if (strings.HasPrefix(l, "+") || strings.HasPrefix(l, "-")) && !strings.HasPrefix(l, "+++") && !strings.HasPrefix(l, "---") {
changed = append(changed, l)
}
}
if len(changed) != 1 || !strings.HasPrefix(changed[0], "-") || !strings.Contains(changed[0], "-hairpin") ||
!strings.Contains(out.Diff, "(before "+itoa(mr.HistoryID)+")") || !strings.Contains(out.Diff, "@@ chain srcnat") {
t.Errorf("entry %d's diff, want only the hairpin line gone, in chain srcnat:\n%s", mr.HistoryID, out.Diff)
}
}
+555
View File
@@ -0,0 +1,555 @@
package server
import (
"context"
"encoding/json"
"errors"
"fmt"
"net/http"
"net/netip"
"strconv"
"strings"
"time"
"housecarl/internal/api"
"housecarl/internal/auth"
"housecarl/internal/model"
"housecarl/internal/render"
"housecarl/internal/store"
)
// Ban lists (docs/bans.md): the lists are configuration objects; their
// bans are kept apart, like a dynamic set's entries, audited but not in
// the history or on probation. A ban is saved first, then put in force;
// every apply puts all of them back (ActiveBans).
// banList finds a ban list the principal may read or ban into.
func (s *Server) banList(r *http.Request, p *auth.Principal, write bool) (*model.BanList, error) {
ref := r.PathValue("ref")
o := s.eng.State().Find(model.KindBanList, ref)
if o == nil {
if !p.CanBan(ref, write) {
return nil, banForbidden(ref, write)
}
return nil, notFound("no ban list %s", ref)
}
b := o.(*model.BanList)
if !p.CanBan(b.Name, write) {
return nil, banForbidden(b.Name, write)
}
return b, nil
}
func banForbidden(list string, write bool) error {
access := "read"
if write {
access = "write"
}
return forbidden("permission denied: this needs bans:%s for %s", access, list)
}
// ActiveBans are the bans to put back with every apply (nft.Firewall's
// Bans).
func ActiveBans(ctx context.Context, st *store.Store) []render.Ban {
list, err := st.ListBans(ctx, 0)
if err != nil {
return nil
}
out := make([]render.Ban, 0, len(list))
for _, b := range list {
if rb, ok := renderBan(b); ok {
out = append(out, rb)
}
}
return out
}
func renderBan(b *model.Ban) (render.Ban, bool) {
p, err := b.Prefix()
if err != nil {
return render.Ban{}, false
}
return render.Ban{List: b.ListID, Addr: p, Proto: b.Protocol, Port: b.Port, Timeout: b.Timeout}, true
}
// listBans answers GET /banlists/{ref}/bans, and GET /bans: every list's
// the principal may read.
func (s *Server) listBans(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
st := s.eng.State()
var id int64
if r.PathValue("ref") != "" {
b, err := s.banList(r, p, false)
if err != nil {
return err
}
id = b.ID
}
all, err := s.store.ListBans(r.Context(), id)
if err != nil {
return err
}
out := []*model.Ban{}
for _, b := range all {
o := st.ByID(model.KindBanList, b.ListID)
if o == nil || !p.CanBan(o.Label(), false) {
continue
}
b.List = o.Label()
out = append(out, b)
}
writeJSON(w, http.StatusOK, out)
return nil
}
// addBan answers POST /banlists/{ref}/bans.
func (s *Server) addBan(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
list, err := s.banList(r, p, true)
if err != nil {
return err
}
var in api.BanInput
if err := decodeJSON(r, &in); err != nil {
return err
}
st := s.eng.State()
if list.Disabled {
return conflict("ban list %s is disabled: enable it first", list.Name)
}
addr, err := netip.ParseAddr(strings.Trim(strings.TrimSpace(in.Address), "[]"))
if err != nil {
return badRequest("%q isn't an address", in.Address)
}
addr = addr.Unmap().WithZone("")
bits := 32
if addr.Is6() {
bits = list.V6Group()
}
prefix := netip.PrefixFrom(addr, bits).Masked()
if why := s.neverBan(r.Context(), st, list, p, addr, prefix); why != "" {
s.eng.Audit(r.Context(), p, "ban.add", list.Name+" "+model.BanAddress(prefix), "failure", why)
return &httpError{http.StatusUnprocessableEntity, "never_ban", fmt.Sprintf("%s isn't banned: %s", model.BanAddress(prefix), why)}
}
ports, err := banPorts(st, in.Protocol, in.Ports)
if err != nil {
return badRequest("%v", err)
}
var expires *time.Time
switch secs := banSeconds(list, in.Time); {
case secs > 0:
t := s.nowFn().Add(time.Duration(secs) * time.Second)
expires = &t
}
if n, err := s.store.CountBans(r.Context(), list.ID); err != nil {
return err
} else if n+len(ports) > list.Entries() {
existing := 0
for _, pp := range ports {
if s.banExists(r.Context(), list.ID, model.BanAddress(prefix), pp) {
existing++
}
}
if n+len(ports)-existing > list.Entries() {
return &httpError{http.StatusUnprocessableEntity, "list_full", fmt.Sprintf("ban list %s holds its most bans (%d, max_entries); wait for some to end, or raise it", list.Name, list.Entries())}
}
}
out := api.BanResult{Bans: []api.BanMade{}}
for _, pp := range ports {
b := &model.Ban{ListID: list.ID, List: list.Name, Address: model.BanAddress(prefix), Protocol: pp.proto, Port: pp.port,
Expires: expires, Who: p.Name(), Source: clip(in.Source, 100), Reason: clip(in.Reason, 500)}
existing, err := s.store.PutBan(r.Context(), b)
if err != nil {
return err
}
if expires != nil {
b.Timeout = int(time.Until(*expires).Round(time.Second) / time.Second)
}
rb, _ := renderBan(b)
if err := s.fw.AddBan(r.Context(), rb); err != nil {
s.log.Warn("putting a ban in force failed; the next change or restart adds it", "list", list.Name, "address", b.Address, "err", err)
}
out.Bans = append(out.Bans, api.BanMade{Ban: *b, Existing: existing})
detail := b.Source
if b.Reason != "" {
detail = strings.TrimSpace(detail + ": " + b.Reason)
}
s.eng.Audit(r.Context(), p, "ban.add", list.Name+" "+banLabel(b), "success", detail)
}
out.Closed = s.closeBanned(r.Context(), st, list, prefix, ports)
writeJSON(w, http.StatusCreated, out)
return nil
}
func (s *Server) banExists(ctx context.Context, list int64, addr string, pp banPort) bool {
bans, _ := s.store.ListBans(ctx, list)
for _, b := range bans {
if b.Address == addr && b.Protocol == pp.proto && b.Port == pp.port {
return true
}
}
return false
}
// banSeconds is a ban's length: what was asked (0 the list's default, -1
// its longest), cut to the list's longest. 0 means it doesn't end.
func banSeconds(list *model.BanList, asked int) int {
max := list.BanMaxTime()
secs := asked
switch {
case asked == 0:
secs = list.BanTime()
case asked < 0:
secs = max
}
if max > 0 && secs > max {
secs = max
}
return secs
}
func banLabel(b *model.Ban) string {
if b.Protocol == "" {
return b.Address
}
return fmt.Sprintf("%s %s/%d", b.Address, b.Protocol, b.Port)
}
func clip(s string, n int) string {
s = strings.TrimSpace(s)
if len(s) > n {
return s[:n]
}
return s
}
// banPort is one protocol and port banned; proto "" is every port.
type banPort struct {
proto string
port int
}
// maxBanPorts is the most ports one ban request may name: a range in a
// service counts each port.
const maxBanPorts = 64
// banPorts reads a ban request's protocol and ports (api.BanInput).
func banPorts(st *model.State, protocol string, raw json.RawMessage) ([]banPort, error) {
protocol = strings.ToLower(strings.TrimSpace(protocol))
if protocol != "" && protocol != "tcp" && protocol != "udp" && protocol != "all" {
return nil, fmt.Errorf("protocol %q: use tcp, udp, or leave it out for every port", protocol)
}
if protocol == "all" {
protocol = ""
}
var words []string
if len(raw) > 0 && string(raw) != "null" {
var list []json.RawMessage
if json.Unmarshal(raw, &list) != nil {
list = []json.RawMessage{raw}
}
for _, item := range list {
var n int
var str string
switch {
case json.Unmarshal(item, &n) == nil:
words = append(words, strconv.Itoa(n))
case json.Unmarshal(item, &str) == nil:
words = append(words, strings.FieldsFunc(str, func(c rune) bool { return c == ',' || c == ' ' })...)
default:
return nil, fmt.Errorf("ports: give numbers or service names, in a list or a string")
}
}
}
if len(words) == 0 {
if protocol != "" {
return nil, fmt.Errorf("protocol %s without ports: give the ports, or leave the protocol out for every port", protocol)
}
return []banPort{{}}, nil
}
protos := []string{"tcp", "udp"}
if protocol != "" {
protos = []string{protocol}
}
var out []banPort
seen := map[banPort]bool{}
add := func(proto string, port int) {
bp := banPort{proto, port}
if !seen[bp] {
seen[bp] = true
out = append(out, bp)
}
}
for _, w := range words {
lw := strings.ToLower(w)
switch lw {
case "all", "any", "0:65535", "1:65535", "0-65535", "1-65535":
return []banPort{{}}, nil
}
if n, err := strconv.Atoi(w); err == nil {
if n < 1 || n > 65535 {
return nil, fmt.Errorf("port %d: use 1 to 65535", n)
}
for _, p := range protos {
add(p, n)
}
continue
}
svc := st.Service(w)
if svc == nil {
return nil, fmt.Errorf("%q is neither a port nor a service (built-in or yours)", w)
}
matched := false
for _, e := range st.ServiceEntries([]string{svc.Name}) {
if e.Proto != "tcp" && e.Proto != "udp" || protocol != "" && e.Proto != protocol {
continue
}
for _, part := range strings.Split(e.Ports, ",") {
lo, hi, _ := strings.Cut(strings.TrimSpace(part), "-")
a, err1 := strconv.Atoi(lo)
b := a
var err2 error
if hi != "" {
b, err2 = strconv.Atoi(hi)
}
if err1 != nil || err2 != nil || b-a >= maxBanPorts {
return nil, fmt.Errorf("service %s has the range %s: ban every port instead", svc.Name, part)
}
for port := a; port <= b; port++ {
add(e.Proto, port)
matched = true
}
}
}
if !matched {
return nil, fmt.Errorf("service %s has no %s ports", svc.Name, map[bool]string{true: protocol, false: "tcp or udp"}[protocol != ""])
}
}
if len(out) > maxBanPorts {
return nil, fmt.Errorf("%d ports in one ban: ban every port instead (up to %d)", len(out), maxBanPorts)
}
return out, nil
}
// neverBan says why an address (banned as prefix) must not be banned, or
// "": it is the firewall's, an inside network's, the client's own, in the
// list's never, or not an address one bans.
func (s *Server) neverBan(ctx context.Context, st *model.State, list *model.BanList, p *auth.Principal, addr netip.Addr, prefix netip.Prefix) string {
switch {
case addr.IsLoopback(), addr.IsUnspecified(), addr.IsMulticast(), addr.IsLinkLocalUnicast(), addr.IsLinkLocalMulticast():
return "it is a loopback, link-local, multicast or unspecified address"
}
if own, err := netip.ParseAddr(p.IP); err == nil && prefix.Contains(own.Unmap()) {
return "it is the address of whoever asked (" + own.Unmap().String() + ")"
}
inside := s.insideNetworks(st)
if inside(addr) || inside(prefix.Addr()) {
return "it is inside: in a network of an inside zone, or private"
}
external := map[string]bool{}
for _, z := range st.Zones {
external[z.Name] = z.External
}
for net, zone := range s.connected(st) {
if !external[zone] && net.Overlaps(prefix) {
return fmt.Sprintf("it overlaps %s, a network of zone %s", net, zone)
}
}
if s.ifaces != nil {
if all, err := s.ifaces.All(); err == nil {
for name, ps := range all {
for _, a := range ps {
if prefix.Contains(a.Addr()) {
return fmt.Sprintf("it is the firewall's own (%s on %s)", a.Addr(), name)
}
}
}
}
}
for _, o := range st.Settings.OwnPrefixes {
if op, err := netip.ParsePrefix(o); err == nil && op.Overlaps(prefix) {
return "it is in the firewall's own prefix " + o + " (settings)"
}
}
for _, ref := range list.Never {
if why := s.inNever(ctx, st, ref, prefix); why != "" {
return why + " (the list's never)"
}
}
return ""
}
// inNever: is prefix in a never entry (an address, or a set)?
func (s *Server) inNever(ctx context.Context, st *model.State, ref string, prefix netip.Prefix) string {
overlaps := func(entry string) bool {
a, err := model.ParseAddress(entry)
return err == nil && a.Overlaps(prefix)
}
if model.IsLiteral(ref) {
if overlaps(ref) {
return "it is in " + ref
}
return ""
}
set := st.Set(ref)
if set == nil {
return ""
}
entries := set.Entries
if set.Runtime() {
if t, err := s.fw.Table(ctx); err == nil {
for _, e := range t.Elements[set.Name] {
entries = append(entries, e.Addr)
}
}
}
for _, e := range entries {
if overlaps(e) {
return fmt.Sprintf("it is in set %s (%s)", set.Name, e)
}
}
return ""
}
// closeBanned closes the banned address's connections the list's bans
// now drop: offloaded to the flowtable, they would go on without
// reaching the rules. Not in shadow mode, which changes no traffic.
func (s *Server) closeBanned(ctx context.Context, st *model.State, list *model.BanList, prefix netip.Prefix, ports []banPort) int {
if s.cfg.Shadow || s.conns == nil {
return 0
}
conns, err := s.conns.List(ctx)
if err != nil {
s.log.Warn("listing connections to close a ban's failed", "err", err)
return 0
}
own := map[netip.Addr]bool{}
if s.ifaces != nil {
if all, err := s.ifaces.All(); err == nil {
for _, ps := range all {
for _, a := range ps {
own[a.Addr()] = true
}
}
}
}
h4, h6 := st.SplitRefs(list.Hosts)
hosts := append(append([]string{}, h4.Literals...), h6.Literals...)
for _, name := range append(h4.Sets, h6.Sets...) {
if set := st.Set(name); set != nil {
hosts = append(hosts, set.Entries...)
}
}
isHost := func(a netip.Addr) bool {
for _, h := range hosts {
if x, err := model.ParseAddress(h); err == nil && x.Contains(a) {
return true
}
}
return false
}
closed := 0
for _, c := range conns {
src, err := netip.ParseAddr(c.Source)
if err != nil || !prefix.Contains(src.Unmap()) {
continue
}
dst, _ := netip.ParseAddr(c.Destination)
port := c.DestinationPort
if c.DNATTo != "" {
dst, _ = netip.ParseAddr(c.DNATTo)
if c.DNATPort != 0 {
port = c.DNATPort
}
}
toFirewall := own[dst.Unmap()]
switch list.Scope {
case model.BanScopeFirewall:
if !toFirewall {
continue
}
case model.BanScopeHosts:
if toFirewall || !isHost(dst.Unmap()) {
continue
}
}
hit := false
for _, pp := range ports {
hit = hit || pp.proto == "" || pp.proto == c.Protocol && pp.port == port
}
if !hit {
continue
}
if err := s.conns.Close(ctx, c); err == nil {
closed++
}
}
return closed
}
// deleteBans answers DELETE /banlists/{ref}/bans/{id}, and DELETE
// /banlists/{ref}/bans?address=A: every ban of that address (any of its
// grouping), of every port and of each.
func (s *Server) deleteBans(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
list, err := s.banList(r, p, true)
if err != nil {
return err
}
var gone []*model.Ban
if idText := r.PathValue("id"); idText != "" {
id, err := strconv.ParseInt(idText, 10, 64)
if err != nil {
return badRequest("ban ID %q isn't a number", idText)
}
b, err := s.store.GetBan(r.Context(), id)
if errors.Is(err, store.ErrNotFound) || err == nil && b.ListID != list.ID {
return notFound("no ban %d in ban list %s", id, list.Name)
}
if err != nil {
return err
}
gone = append(gone, b)
} else {
q := strings.Trim(strings.TrimSpace(r.URL.Query().Get("address")), "[]")
a, err := netip.ParseAddr(q)
if err != nil {
return badRequest("say which ban: DELETE …/bans/ID, or …/bans?address=ADDRESS")
}
a = a.Unmap()
bans, err := s.store.ListBans(r.Context(), list.ID)
if err != nil {
return err
}
for _, b := range bans {
if pre, err := b.Prefix(); err == nil && pre.Contains(a) {
gone = append(gone, b)
}
}
if len(gone) == 0 {
return notFound("%s isn't banned in ban list %s", a, list.Name)
}
}
for _, b := range gone {
if err := s.store.DeleteBan(r.Context(), b.ID); err != nil {
return err
}
if rb, ok := renderBan(b); ok {
if err := s.fw.DeleteBan(r.Context(), rb); err != nil {
s.log.Warn("lifting a ban failed; the next change or restart drops it", "list", list.Name, "address", b.Address, "err", err)
}
}
s.eng.Audit(r.Context(), p, "ban.remove", list.Name+" "+banLabel(b), "success", "")
}
writeJSON(w, http.StatusOK, map[string]int{"removed": len(gone)})
return nil
}
// pruneBans forgets the bans that ended, and those of lists st no longer
// has.
func (s *Server) pruneBans(ctx context.Context, st *model.State) {
var ids []int64
for _, b := range st.BanLists {
ids = append(ids, b.ID)
}
if _, err := s.store.PruneBans(ctx, ids); err != nil {
s.log.Warn("pruning bans failed", "err", err)
}
}
+208 -7
View File
@@ -3,6 +3,7 @@ package server
import (
"encoding/json"
"errors"
"fmt"
"net/http"
"net/netip"
"sort"
@@ -98,16 +99,138 @@ func (s *Server) respondChange(w http.ResponseWriter, r *http.Request, p *auth.P
// rulesetDiff is the unified diff of the rulesets two configurations
// make: the file housecarl-ruleset.service loads, now and after.
func (s *Server) rulesetDiff(before, after *model.State) string {
return s.labeledDiff(before, after, "now", "after")
}
// labeledDiff is rulesetDiff with its own names for the two sides.
func (s *Server) labeledDiff(before, after *model.State, from, to string) string {
if before == nil || after == nil {
return ""
}
return udiff.Unified("ruleset.nft (now)", "ruleset.nft (after)", s.fw.Render(before).Text, s.fw.Render(after).Text)
now, then := s.fw.Render(before).Text, s.fw.Render(after).Text
diff := hunkStarts(udiff.Unified("ruleset.nft ("+from+")", "ruleset.nft ("+to+")", now, then))
return blockHeaders(diff, now, then)
}
// hunkStarts numbers each hunk's new side again, from its old side and
// the lines the hunks before it added and removed. go-udiff (v0.4.1)
// leaves out of its count the unchanged lines between two edits of one
// hunk, so after a hunk like a moved line every later one started too
// early ("@@ -76,9 +74,9 @@" for "+76,9"), and blockHeaders named the
// chain before it.
func hunkStarts(diff string) string {
lines := strings.SplitAfter(diff, "\n")
delta := 0
for i := 0; i < len(lines); i++ {
f := strings.Fields(lines[i])
if !strings.HasPrefix(lines[i], "@@ ") || len(f) < 4 || f[3] != "@@" {
continue
}
start, _, _ := strings.Cut(f[1][1:], ",")
from, err := strconv.Atoi(start)
if err != nil {
continue
}
oldN, newN := 0, 0
for _, l := range lines[i+1:] {
if strings.HasPrefix(l, "@@ ") {
break
}
switch {
case strings.HasPrefix(l, "-"):
oldN++
case strings.HasPrefix(l, "+"):
newN++
case strings.HasPrefix(l, " "):
oldN++
newN++
}
}
// An empty side is numbered by the line before it.
first := from
if oldN == 0 {
first++
}
to := first + delta
if newN == 0 {
to--
}
side := "+" + strconv.Itoa(to)
if newN > 1 {
side += "," + strconv.Itoa(newN)
}
f[2] = side
lines[i] = strings.Join(f, " ") + "\n"
delta += newN - oldN
}
return strings.Join(lines, "")
}
// blockHeaders names, after each hunk's @@ line, the chain (or other
// block of the table) the hunk starts in, as git names the function:
// "@@ -40,6 +40,8 @@ chain forward". Without it a change deep in a long
// chain doesn't say which chain it is in.
func blockHeaders(diff, now, then string) string {
if diff == "" {
return diff
}
oldBlocks, newBlocks := enclosingBlocks(now), enclosingBlocks(then)
lines := strings.SplitAfter(diff, "\n")
for i, l := range lines {
if !strings.HasPrefix(l, "@@ ") {
continue
}
// "@@ -a,b +c,d @@": the new side's start, or the old side's
// when nothing of the hunk is left.
f := strings.Fields(l)
if len(f) < 4 || f[3] != "@@" {
continue
}
block := blockAt(newBlocks, f[2])
if block == "" {
block = blockAt(oldBlocks, f[1])
}
if block != "" {
lines[i] = strings.TrimRight(l, "\n") + " " + block + "\n"
}
}
return strings.Join(lines, "")
}
// enclosingBlocks gives each line of a rendered ruleset the block of the
// table it is in ("chain input"), "" outside one. A block opens with a
// line of one tab ending in "{" and closes with "\t}".
func enclosingBlocks(text string) []string {
var out []string
cur := ""
for _, l := range strings.Split(text, "\n") {
if strings.HasPrefix(l, "\t") && !strings.HasPrefix(l, "\t\t") && strings.HasSuffix(l, "{") {
cur = strings.TrimSpace(strings.TrimSuffix(l, "{"))
}
out = append(out, cur)
if l == "\t}" {
cur = ""
}
}
return out
}
// blockAt is the block at a hunk side's start ("+40,8" or "-40"); lines
// count from 1, and an empty side (",0") sits after its line.
func blockAt(blocks []string, side string) string {
start, _, _ := strings.Cut(side[1:], ",")
n, err := strconv.Atoi(start)
if err != nil || n < 1 || n > len(blocks) {
return ""
}
return blocks[n-1]
}
// kindOf maps the API's plural path names to kinds.
var kindOf = map[string]string{
"zones": model.KindZone, "sets": model.KindSet, "services": model.KindService, "rules": model.KindRule,
"forwards": model.KindForward, "snats": model.KindSNAT, "redirects": model.KindRedirect,
"banlists": model.KindBanList,
}
func pathKind(r *http.Request) (string, error) {
@@ -128,13 +251,16 @@ func findObject(st *model.State, kind, ref string) (model.Object, error) {
// ---- objects
func (s *Server) listObjects(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
if err := require(p, auth.ResConfig, false); err != nil {
return err
}
kind, err := pathKind(r)
if err != nil {
return err
}
if kind == model.KindBanList && !p.Can(auth.ResConfig, false) {
return s.listOwnBanLists(w, p)
}
if err := require(p, auth.ResConfig, false); err != nil {
return err
}
cur, st, draft, err := s.viewState(w, r)
if err != nil {
return err
@@ -180,13 +306,22 @@ func (s *Server) listObjects(w http.ResponseWriter, r *http.Request, p *auth.Pri
}
func (s *Server) getObject(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
if err := require(p, auth.ResConfig, false); err != nil {
return err
}
kind, err := pathKind(r)
if err != nil {
return err
}
// A program that bans into a list may read it (its grouping, times).
if kind == model.KindBanList && !p.Can(auth.ResConfig, false) {
b, err := s.banList(r, p, false)
if err != nil {
return err
}
writeJSON(w, http.StatusOK, b)
return nil
}
if err := require(p, auth.ResConfig, false); err != nil {
return err
}
_, st, _, err := s.viewState(w, r)
if err != nil {
return err
@@ -223,6 +358,8 @@ func decodeObject(kind string, data []byte) (model.Object, error) {
o = &model.SourceNAT{}
case model.KindRedirect:
o = &model.Redirect{}
case model.KindBanList:
o = &model.BanList{}
}
dec := json.NewDecoder(strings.NewReader(string(data)))
dec.DisallowUnknownFields()
@@ -274,6 +411,8 @@ func setID(o model.Object, id int64) {
x.ID = id
case *model.Redirect:
x.ID = id
case *model.BanList:
x.ID = id
}
}
@@ -391,6 +530,11 @@ func usedBy(st *model.State, kind string, o model.Object) string {
return f.Label()
}
}
for _, b := range st.BanLists {
if has(b.Hosts) || has(b.Never) {
return "ban list " + b.Name
}
}
}
return ""
}
@@ -433,6 +577,10 @@ func renameRefs(st *model.State, kind, old, now string) {
fix(f.PublicAddresses)
fix(f.HairpinSources)
}
for _, b := range st.BanLists {
fix(b.Hosts)
fix(b.Never)
}
}
}
@@ -520,6 +668,10 @@ func matrixHits(m *api.Matrix, counters map[string]nft.Counter) {
// connected maps the networks on the firewall's interfaces (link-local
// and loopback aside) to their zones.
// Connected are the networks connected to the firewall's interfaces,
// with their zones (link-local and loopback left out).
func (s *Server) Connected(st *model.State) map[netip.Prefix]string { return s.connected(st) }
func (s *Server) connected(st *model.State) map[netip.Prefix]string {
out := map[netip.Prefix]string{}
for _, i := range s.hostInterfaces(st) {
@@ -590,6 +742,39 @@ func (s *Server) getHistory(w http.ResponseWriter, r *http.Request, p *auth.Prin
return nil
}
// historyDiff is the ruleset's diff for one history entry: the
// configurations before and after it, got back by stepping back from
// the current one through every later entry, rendered by this version.
func (s *Server) historyDiff(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
if err := require(p, auth.ResConfig, false); err != nil {
return err
}
id, err := pathID(r, "id")
if err != nil {
return err
}
hs, err := s.store.HistorySince(r.Context(), id)
if err != nil {
return err
}
if len(hs) == 0 || hs[len(hs)-1].ID != id {
return notFound("history entry %d not found", id)
}
st := s.eng.State()
var after *model.State
for _, h := range hs {
if h.ID == id {
after = st.Clone()
}
if err := engine.Rewind(st, h); err != nil {
return err
}
}
before, done := fmt.Sprintf("before %d", id), fmt.Sprintf("after %d", id)
writeJSON(w, http.StatusOK, map[string]string{"diff": s.labeledDiff(st, after, before, done)})
return nil
}
func (s *Server) revertHistory(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
if err := require(p, auth.ResConfig, true); err != nil {
return err
@@ -669,3 +854,19 @@ func (s *Server) rollbackChanges(w http.ResponseWriter, r *http.Request, p *auth
writeJSON(w, http.StatusOK, map[string]int{"rolled_back": n})
return nil
}
// listOwnBanLists lists the ban lists a principal without config:read
// may use.
func (s *Server) listOwnBanLists(w http.ResponseWriter, p *auth.Principal) error {
out := []*model.BanList{}
for _, b := range s.eng.State().BanLists {
if p.CanBan(b.Name, false) {
out = append(out, b)
}
}
if len(out) == 0 && !p.Can(auth.ResBans, false) {
return require(p, auth.ResConfig, false)
}
writeJSON(w, http.StatusOK, out)
return nil
}
+55 -3
View File
@@ -84,16 +84,21 @@ var nftablesUnitLinks = []string{
// hostProblems are warnings about the machine rather than the
// configuration: another ruleset that still loads at boot, or the table
// an import came from, still in force beside Housecarl's. ruleset is the
// an import came from, still in force beside Housecarl's, and the Jarl
// sets' names that give no addresses. ruleset is the
// machine's (nft -j list ruleset; nil: not read). In shadow mode those
// are expected; the warnings are that nothing else filters, and the raw
// rules left out.
func (s *Server) hostProblems(ctx context.Context, ruleset []byte) []validate.Problem {
var jarl []validate.Problem
if s.cfg.JarlSets != nil {
jarl = s.cfg.JarlSets.Problems(s.eng.State())
}
if s.cfg.Shadow {
return s.shadowProblems(ruleset)
return append(s.shadowProblems(ruleset), jarl...)
}
others := otherTables(ruleset)
var out []validate.Problem
out := jarl
for _, link := range s.unitLinks {
if _, err := os.Lstat(link); err == nil {
out = append(out, validate.Problem{Severity: validate.Warning, Code: "nftables_service", Object: "nftables.service",
@@ -284,6 +289,53 @@ func (s *Server) deleteElement(w http.ResponseWriter, r *http.Request, p *auth.P
return nil
}
// ---- Jarl sets
// jarlSet is the Jarl set a request names.
func (s *Server) jarlSet(r *http.Request, p *auth.Principal) (*model.State, *model.AddressSet, error) {
name := r.PathValue("ref")
if !p.CanSet(name, false) {
return nil, nil, forbidden("permission denied: this needs sets:read for %s", name)
}
st := s.eng.State()
set := st.Set(name)
if set == nil {
return nil, nil, notFound("no set %s", name)
}
if set.Jarl == nil {
return nil, nil, badRequest("set %s doesn't take its addresses from Jarl", set.Name)
}
if s.cfg.JarlSets == nil {
return nil, nil, conflict("this housecarld doesn't read Jarl sets")
}
return st, set, nil
}
// jarlStatus: GET /sets/{ref}/jarl.
func (s *Server) jarlStatus(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
st, set, err := s.jarlSet(r, p)
if err != nil {
return err
}
writeJSON(w, http.StatusOK, s.cfg.JarlSets.Status(st, set))
return nil
}
// jarlRefresh: POST /sets/{ref}/jarl/refresh reads Jarl now (every Jarl
// set) and answers as GET does.
func (s *Server) jarlRefresh(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
if _, _, err := s.jarlSet(r, p); err != nil {
return err
}
s.cfg.JarlSets.Refresh(r.Context())
st, set, err := s.jarlSet(r, p)
if err != nil {
return err
}
writeJSON(w, http.StatusOK, s.cfg.JarlSets.Status(st, set))
return nil
}
// ---- the log
// logRing keeps the latest logged packets.
+104
View File
@@ -0,0 +1,104 @@
package server
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"housecarl/internal/api"
"housecarl/internal/jarl"
"housecarl/internal/jarlsets"
"housecarl/internal/model"
"housecarl/internal/store"
)
// TestJarlSets: a set from Jarl, through Housecarl's Jarl client against a
// fake Jarl, filled after a change, with its status and warnings.
func TestJarlSets(t *testing.T) {
records := []map[string]string{
{"name": "nas.example.com.", "type": "A", "data": "10.0.10.20"},
{"name": "nas.example.com.", "type": "AAAA", "data": "2001:db8:10::20"},
{"name": "files.example.com.", "type": "CNAME", "data": "nas.example.com."},
{"name": "far.example.com.", "type": "A", "data": "192.0.2.9"},
}
fake := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get("Authorization") != "Bearer jarl-token" {
w.WriteHeader(401)
return
}
switch r.URL.Path {
case "/api/v1/zones":
json.NewEncoder(w).Encode([]map[string]any{{"id": 1, "name": "example.com.", "view": "internal", "type": "primary"}})
case "/api/v1/zones/1/records":
if r.URL.Query().Get("disabled") != "0" {
t.Errorf("records asked without disabled=0: %s", r.URL)
}
json.NewEncoder(w).Encode(records)
default:
w.WriteHeader(404)
}
}))
t.Cleanup(fake.Close)
jc, err := jarl.New(fake.URL, "jarl-token", "", false)
if err != nil {
t.Fatal(err)
}
js := &jarlsets.Resolver{Source: jc}
e := newEnvWith(t, Config{JarlSets: js})
js.State, js.Connected = e.eng.State, e.srv.Connected
js.Apply = func(ctx context.Context) error { return e.eng.ApplyCurrent(ctx) }
js.Audit = func(ctx context.Context, set, detail string) {
e.eng.Audit(ctx, nil, "set.jarl", set, "success", detail)
}
ctx := context.Background()
e.call(e.token, "POST", "/zones", model.Zone{Name: "lan", Interfaces: []string{"eth1"}}, nil) // 10.0.10.0/24
set := model.AddressSet{Name: "nas", Jarl: &model.JarlSource{Names: []model.JarlName{{Name: "files.example.com"}, {Name: "far.example.com", View: "internal"}}, Within: []string{"lan"}}}
if code := e.call(e.token, "POST", "/sets", set, nil); code != 201 {
t.Fatalf("create: %d", code)
}
if code := e.call(e.token, "POST", "/sets", model.AddressSet{Name: "bad", Entries: []string{"10.0.0.1"}, Jarl: set.Jarl}, nil); code != 422 {
t.Errorf("entries in a Jarl set: %d", code)
}
var st api.JarlSet
if code := e.call(e.token, "POST", "/sets/nas/jarl/refresh", nil, &st); code != 200 {
t.Fatalf("refresh: %d", code)
}
if strings.Join(st.Entries, ",") != "10.0.10.20" || len(st.Names) != 2 || !st.Configured || st.Read == nil {
t.Fatalf("status %+v", st)
}
files := st.Names[0]
if files.Found != "internal" || strings.Join(files.Chain, ",") != "nas.example.com" || len(files.Addresses) != 2 ||
!files.Addresses[0].Used || files.Addresses[1].Used || !strings.Contains(files.Addresses[1].Why, "not within lan") {
t.Errorf("files: %+v", files)
}
var status api.Status
e.call(e.token, "GET", "/status", nil, &status)
n := 0
for _, p := range status.Problems {
if p.Code == "jarl_outside" {
n++
}
}
if n != 2 {
t.Errorf("jarl_outside warnings: %d (%+v)", n, status.Problems)
}
// A Jarl set can be renamed (its addresses go by its names in Jarl).
if code := e.call(e.token, "PATCH", "/sets/nas", map[string]any{"name": "storage"}, nil); code != 200 {
t.Errorf("rename: %d", code)
}
if code := e.call(e.token, "GET", "/sets/trusted/jarl", nil, nil); code != 404 {
t.Errorf("no such set: %d", code)
}
e.call(e.token, "POST", "/sets", model.AddressSet{Name: "trusted", Entries: []string{"10.0.10.0/24"}}, nil)
if code := e.call(e.token, "GET", "/sets/trusted/jarl", nil, nil); code != 400 {
t.Errorf("a set not from Jarl: %d", code)
}
evs, _ := e.srv.store.ListAudit(ctx, store.AuditFilter{Action: "set.jarl"})
if len(evs) != 1 || evs[0].Detail != "added 10.0.10.20" {
t.Errorf("audit %+v", evs)
}
}
+3
View File
@@ -104,6 +104,9 @@ func updateOp(st *model.State, op *model.Op) (int64, error) {
return 0, err
}
setID(next, id)
if old, now := o.Label(), next.Label(); old != now && op.Kind == model.KindBanList {
return 0, conflict("ban list %s can't be renamed: the API tokens and programs (Guard, fail2ban) that ban into it go by its name; make a new list instead", old)
}
if old, now := o.Label(), next.Label(); old != now && isNamed(op.Kind) {
if set, ok := o.(*model.AddressSet); ok && set.Dynamic {
return 0, conflict("set %s is dynamic: its entries, and the API tokens and programs that fill it, go by its name; make a new set instead", old)
+8
View File
@@ -38,6 +38,13 @@ func (s *Server) routes() {
m.HandleFunc("GET "+p+"/sets/{ref}/elements", s.authed(s.listElements))
m.HandleFunc("POST "+p+"/sets/{ref}/elements", s.authed(s.addElement))
m.HandleFunc("DELETE "+p+"/sets/{ref}/elements/{addr}", s.authed(s.deleteElement))
m.HandleFunc("GET "+p+"/sets/{ref}/jarl", s.authed(s.jarlStatus))
m.HandleFunc("POST "+p+"/sets/{ref}/jarl/refresh", s.authed(s.jarlRefresh))
m.HandleFunc("GET "+p+"/bans", s.authed(s.listBans))
m.HandleFunc("GET "+p+"/banlists/{ref}/bans", s.authed(s.listBans))
m.HandleFunc("POST "+p+"/banlists/{ref}/bans", s.authed(s.addBan))
m.HandleFunc("DELETE "+p+"/banlists/{ref}/bans", s.authed(s.deleteBans))
m.HandleFunc("DELETE "+p+"/banlists/{ref}/bans/{id}", s.authed(s.deleteBans))
m.HandleFunc("GET "+p+"/quotas", s.authed(s.listQuotas))
m.HandleFunc("GET "+p+"/connections", s.authed(s.listConnections))
m.HandleFunc("POST "+p+"/connections/close", s.authed(s.closeConnections))
@@ -70,6 +77,7 @@ func (s *Server) routes() {
m.HandleFunc("GET "+p+"/stats/heatmap", s.authed(s.statsHeatmap))
m.HandleFunc("GET "+p+"/history", s.authed(s.listHistory))
m.HandleFunc("GET "+p+"/history/{id}", s.authed(s.getHistory))
m.HandleFunc("GET "+p+"/history/{id}/diff", s.authed(s.historyDiff))
m.HandleFunc("POST "+p+"/history/{id}/revert", s.authed(s.revertHistory))
m.HandleFunc("GET "+p+"/probation", s.authed(s.getProbation))
m.HandleFunc("POST "+p+"/confirm", s.authed(s.confirmChanges))
+20
View File
@@ -22,6 +22,7 @@ import (
"housecarl/internal/auth"
"housecarl/internal/certs"
"housecarl/internal/engine"
"housecarl/internal/jarlsets"
"housecarl/internal/model"
"housecarl/internal/names"
"housecarl/internal/nft"
@@ -54,6 +55,8 @@ type Config struct {
ShadowPosition string
// Names looks up inside addresses' names (nil: names are off).
Names *names.Resolver
// JarlSets fills the sets whose addresses come from Jarl (nil: none).
JarlSets *jarlsets.Resolver
}
// Interfaces lists the host's interfaces with their addresses, and
@@ -72,6 +75,8 @@ type Firewall interface {
Table(ctx context.Context) (*nft.Table, error)
AddElement(ctx context.Context, set, addr string, timeout int) error
DeleteElement(ctx context.Context, set, addr string) error
AddBan(ctx context.Context, b render.Ban) error
DeleteBan(ctx context.Context, b render.Ban) error
ListRuleset(ctx context.Context) ([]byte, error)
DeleteTable(ctx context.Context, family, name string) error
TableText(ctx context.Context, family, name string) (string, error)
@@ -141,6 +146,20 @@ func New(cfg Config, eng *engine.Engine, fw Firewall, ifaces Interfaces, log *sl
s := &Server{cfg: cfg, eng: eng, fw: fw, ifaces: ifaces, store: eng.Store(), log: log, mux: http.NewServeMux(), limiter: newRateLimiter(),
web: web, started: time.Now(), logs: newLogRing(2000), nowFn: time.Now, unitLinks: nftablesUnitLinks}
s.routes()
// What the quotas used is saved after every change too, not only every
// five minutes: a change that counts a quota differently converts its
// usage, and a restart must not bring back the old kind.
prev := eng.OnChange
eng.OnChange = func(st *model.State) {
if prev != nil {
prev(st)
}
s.SaveQuotaUsage(context.Background())
s.pruneBans(context.Background(), st) // the engine is locked: not s.eng.State()
if cfg.JarlSets != nil {
cfg.JarlSets.Kick() // a new name: read it now
}
}
return s
}
@@ -314,6 +333,7 @@ func (s *Server) housekeeping(ctx context.Context) {
s.log.Warn("purging sessions failed", "err", err)
}
s.limiter.cleanup()
s.pruneBans(ctx, s.eng.State())
if s.certs != nil {
s.certs.reloadIfChanged()
}
+58
View File
@@ -48,9 +48,25 @@ type fakeFW struct {
ruleset string // nft -j list ruleset ("": a default)
quotas nft.QuotaUsage
adjusted []nft.QuotaAdjust
bans map[string]render.Ban // by element (set and key)
}
func (f *fakeFW) Check(_ context.Context, st *model.State) error { return nil }
func (f *fakeFW) AddBan(_ context.Context, b render.Ban) error {
f.mu.Lock()
defer f.mu.Unlock()
if f.bans == nil {
f.bans = map[string]render.Ban{}
}
f.bans[render.BanSet(b.List, b.Addr.Addr().Is6(), b.Proto != "")+" "+render.BanElement(b)] = b
return nil
}
func (f *fakeFW) DeleteBan(_ context.Context, b render.Ban) error {
f.mu.Lock()
defer f.mu.Unlock()
delete(f.bans, render.BanSet(b.List, b.Addr.Addr().Is6(), b.Proto != "")+" "+render.BanElement(b))
return nil
}
func (f *fakeFW) Apply(_ context.Context, st *model.State) error {
f.mu.Lock()
defer f.mu.Unlock()
@@ -792,6 +808,21 @@ func TestQuotaResets(t *testing.T) {
}
}
// TestQuotaUsageSavedOnChange: a change saves what the quotas used at
// once (a restart must not bring back usage counted the old way).
func TestQuotaUsageSavedOnChange(t *testing.T) {
e := newEnv(t)
e.call(e.token, "POST", "/zones", model.Zone{Name: "lan", Interfaces: []string{"eth1"}}, nil)
e.fw.mu.Lock()
e.fw.quotas = nft.QuotaUsage{Quotas: map[string]int64{"quota_r7": 4242}}
e.fw.mu.Unlock()
e.call(e.token, "POST", "/zones", model.Zone{Name: "dmz", Interfaces: []string{"eth2"}}, nil)
v, err := e.eng.Store().GetMeta(context.Background(), QuotaUsageKey)
if err != nil || !strings.Contains(v, `"quota_r7":4242`) {
t.Errorf("saved usage %q %v", v, err)
}
}
// TestAutoConfirm: the client that made a change confirms it by itself:
// with its token, from the same address, over a new connection. Nobody
// else can, an older token can't, and with manual_confirm there is none.
@@ -1304,3 +1335,30 @@ func TestCreatePlaced(t *testing.T) {
t.Errorf("%d rules", n)
}
}
// TestTokenForUser: an administrator makes a token for another user (the
// web UI's New token on a user's row); that user can't make one for
// someone else.
func TestTokenForUser(t *testing.T) {
e := newEnv(t)
var u api.User
if code := e.call(e.token, "POST", "/users", api.UserInput{Username: ptr("portal"), Grants: &[]string{"sets:write"}}, &u); code != 201 {
t.Fatalf("user: %d", code)
}
var tok api.Token
if code := e.call(e.token, "POST", "/users/"+itoa(u.ID)+"/tokens", api.TokenInput{Name: "portal", ExpiresIn: "30d"}, &tok); code != 201 || tok.Secret == "" || tok.UserID != u.ID {
t.Fatalf("token for portal: %d %+v", code, tok)
}
if code := e.call(tok.Secret, "GET", "/rules", nil, nil); code != 403 {
t.Errorf("the token can do more than its user: %d", code)
}
var users []api.User
e.call(e.token, "GET", "/users", nil, &users)
for _, o := range users {
if o.ID != u.ID {
if code := e.call(tok.Secret, "POST", "/users/"+itoa(o.ID)+"/tokens", api.TokenInput{Name: "x"}, nil); code != 403 {
t.Errorf("a token for another user, made by a non-administrator: %d", code)
}
}
}
}
+4 -1
View File
@@ -402,7 +402,10 @@ func (c *Collector) sample(ctx context.Context, minute int64) map[string]int64 {
for set, es := range t.QuotaSets {
kind := strings.TrimSuffix(strings.TrimSuffix(set, "_v4"), "_v6")
for _, e := range es {
addTop(kind, e.Addr, c.table.of(set+"/"+e.Addr, e.Used))
d := c.table.of(set+"/"+e.Addr, e.Used)
addTop(kind, e.Addr, d)
// A per-address quota's series is its addresses' sum.
add("quota:"+strings.TrimPrefix(kind, "quota_"), d)
}
}
for set, es := range t.DeviceSets {
+6
View File
@@ -85,6 +85,10 @@ func TestCollector(t *testing.T) {
h.table.Counters["r1"] = nft.Counter{Packets: 10, Bytes: 1000}
h.table.Counters["r2"] = nft.Counter{Packets: 100, Bytes: 9000}
h.table.Quotas["quota_r5"] = 1 << 20
if h.table.QuotaSets == nil {
h.table.QuotaSets = map[string][]render.Element{}
}
h.table.QuotaSets["quota_r6_v4"] = []render.Element{{Addr: "10.0.10.5", Used: 1000}}
h.table.DeviceSets["stats_up_v4"] = []render.Element{{Addr: "10.0.10.5", Bytes: 5000}}
store, err := OpenStore(":memory:")
@@ -104,6 +108,7 @@ func TestCollector(t *testing.T) {
h.table.Counters["r1"] = nft.Counter{Packets: 25, Bytes: 2500}
h.table.Counters["p-input"] = nft.Counter{Packets: 3, Bytes: 180} // a new counter counts whole
h.table.Quotas["quota_r5"] = 3 << 20
h.table.QuotaSets["quota_r6_v4"] = []render.Element{{Addr: "10.0.10.5", Used: 2000}, {Addr: "10.0.10.6", Used: 500}} // a new entry counts whole
h.table.DeviceSets["stats_up_v4"] = []render.Element{{Addr: "10.0.10.5", Bytes: 8000}}
h.table.DeviceSets["stats_down_v6"] = []render.Element{{Addr: "2001:db8:1:2::", Bytes: 70000}}
for range 2 {
@@ -142,6 +147,7 @@ func TestCollector(t *testing.T) {
check("drop:forward", sum("drop:")["forward"], 15)
check("drop:input", sum("drop:")["input"], 3)
check("quota:r5", sum("quota:")["r5"], 2<<20)
check("quota:r6 (per address: the sum)", sum("quota:")["r6"], 1500)
s, err := q.Summary(ctx, time.Hour, 10)
if err != nil {
+135
View File
@@ -0,0 +1,135 @@
package store
import (
"context"
"database/sql"
"errors"
"time"
"housecarl/internal/model"
)
const banCols = `id, list_id, address, protocol, port, expires_at, created_at, who, source, reason`
func scanBan(sc interface{ Scan(...any) error }, now time.Time) (*model.Ban, error) {
b := &model.Ban{}
var expires sql.NullString
var created string
if err := sc.Scan(&b.ID, &b.ListID, &b.Address, &b.Protocol, &b.Port, &expires, &created, &b.Who, &b.Source, &b.Reason); err != nil {
return nil, err
}
b.Created = parseTS(created)
b.Expires = parseTSPtr(expires)
if b.Expires != nil {
b.Timeout = max(int(b.Expires.Sub(now).Round(time.Second)/time.Second), 1)
}
return b, nil
}
// PutBan saves a ban, or renews the same ban (list, address, protocol
// and port) with its new end, who and why (an empty source or reason
// keeps the old). existing tells which; b gets
// its ID either way.
func (s *Store) PutBan(ctx context.Context, b *model.Ban) (existing bool, err error) {
now := time.Now()
err = s.db.QueryRowContext(ctx, `SELECT id FROM bans WHERE list_id = ? AND address = ? AND protocol = ? AND port = ?
AND (expires_at IS NULL OR expires_at > ?)`, b.ListID, b.Address, b.Protocol, b.Port, ts(now)).Scan(&b.ID)
existing = err == nil
if err != nil && !errors.Is(err, sql.ErrNoRows) {
return false, err
}
b.Created = now
err = s.db.QueryRowContext(ctx, `INSERT INTO bans (list_id, address, protocol, port, expires_at, created_at, who, source, reason)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT (list_id, address, protocol, port) DO UPDATE SET expires_at = excluded.expires_at,
who = excluded.who, source = COALESCE(NULLIF(excluded.source, ''), bans.source),
reason = COALESCE(NULLIF(excluded.reason, ''), bans.reason),
created_at = CASE WHEN bans.expires_at IS NULL OR bans.expires_at > excluded.created_at THEN bans.created_at ELSE excluded.created_at END
RETURNING id, created_at`,
b.ListID, b.Address, b.Protocol, b.Port, tsPtr(b.Expires), ts(now), b.Who, b.Source, b.Reason).Scan(&b.ID, new(string))
return existing, err
}
// ListBans lists the active bans of a list (0: of all lists), oldest
// first.
func (s *Store) ListBans(ctx context.Context, listID int64) ([]*model.Ban, error) {
now := time.Now()
q := `SELECT ` + banCols + ` FROM bans WHERE (expires_at IS NULL OR expires_at > ?)`
args := []any{ts(now)}
if listID != 0 {
q += ` AND list_id = ?`
args = append(args, listID)
}
rows, err := s.db.QueryContext(ctx, q+` ORDER BY id`, args...)
if err != nil {
return nil, err
}
defer rows.Close()
out := []*model.Ban{}
for rows.Next() {
b, err := scanBan(rows, now)
if err != nil {
return nil, err
}
out = append(out, b)
}
return out, rows.Err()
}
// GetBan loads an active ban.
func (s *Store) GetBan(ctx context.Context, id int64) (*model.Ban, error) {
now := time.Now()
b, err := scanBan(s.db.QueryRowContext(ctx, `SELECT `+banCols+` FROM bans WHERE id = ? AND (expires_at IS NULL OR expires_at > ?)`, id, ts(now)), now)
if errors.Is(err, sql.ErrNoRows) {
return nil, ErrNotFound
}
return b, err
}
// DeleteBan removes a ban.
func (s *Store) DeleteBan(ctx context.Context, id int64) error {
_, err := s.db.ExecContext(ctx, `DELETE FROM bans WHERE id = ?`, id)
return err
}
// CountBans counts a list's active bans.
func (s *Store) CountBans(ctx context.Context, listID int64) (int, error) {
var n int
err := s.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM bans WHERE list_id = ? AND (expires_at IS NULL OR expires_at > ?)`, listID, ts(time.Now())).Scan(&n)
return n, err
}
// PruneBans deletes the bans that ended, and those of lists that no
// longer exist (lists names the ones that do).
func (s *Store) PruneBans(ctx context.Context, lists []int64) (int64, error) {
res, err := s.db.ExecContext(ctx, `DELETE FROM bans WHERE expires_at <= ?`, ts(time.Now()))
if err != nil {
return 0, err
}
n, _ := res.RowsAffected()
rows, err := s.db.QueryContext(ctx, `SELECT DISTINCT list_id FROM bans`)
if err != nil {
return n, err
}
var gone []int64
keep := map[int64]bool{}
for _, id := range lists {
keep[id] = true
}
for rows.Next() {
var id int64
if rows.Scan(&id) == nil && !keep[id] {
gone = append(gone, id)
}
}
rows.Close()
for _, id := range gone {
res, err := s.db.ExecContext(ctx, `DELETE FROM bans WHERE list_id = ?`, id)
if err != nil {
return n, err
}
m, _ := res.RowsAffected()
n += m
}
return n, nil
}
+18
View File
@@ -63,3 +63,21 @@ func (s *Store) GetHistory(ctx context.Context, id int64) (*model.HistoryEntry,
}
return e, err
}
// HistorySince lists the entries from id on, newest first.
func (s *Store) HistorySince(ctx context.Context, id int64) ([]*model.HistoryEntry, error) {
rows, err := s.db.QueryContext(ctx, `SELECT `+historyCols+` FROM history WHERE id >= ? ORDER BY id DESC`, id)
if err != nil {
return nil, err
}
defer rows.Close()
var out []*model.HistoryEntry
for rows.Next() {
e, err := scanHistory(rows)
if err != nil {
return nil, err
}
out = append(out, e)
}
return out, rows.Err()
}
+17
View File
@@ -179,6 +179,23 @@ CREATE TABLE changesets (
history_id INTEGER
);
CREATE INDEX changesets_status ON changesets (status);
`,
// 3: ban lists' bans (the lists are objects)
`
CREATE TABLE bans (
id INTEGER PRIMARY KEY AUTOINCREMENT,
list_id INTEGER NOT NULL,
address TEXT NOT NULL,
protocol TEXT NOT NULL DEFAULT '',
port INTEGER NOT NULL DEFAULT 0,
expires_at TEXT,
created_at TEXT NOT NULL,
who TEXT NOT NULL DEFAULT '',
source TEXT NOT NULL DEFAULT '',
reason TEXT NOT NULL DEFAULT '',
UNIQUE (list_id, address, protocol, port)
);
CREATE INDEX bans_expires ON bans (expires_at);
`,
}
+57
View File
@@ -103,3 +103,60 @@ func TestUsersAndAudit(t *testing.T) {
t.Errorf("audit %+v", evs)
}
}
func TestBans(t *testing.T) {
ctx := context.Background()
s, err := Open(filepath.Join(t.TempDir(), "housecarl.db"))
if err != nil {
t.Fatal(err)
}
defer s.Close()
in := func(d time.Duration) *time.Time { t := time.Now().Add(d); return &t }
b := &model.Ban{ListID: 7, Address: "203.0.113.9", Expires: in(time.Hour), Who: "guard", Source: "guard/ssh"}
if existing, err := s.PutBan(ctx, b); err != nil || existing {
t.Fatalf("new ban: %v %v", existing, err)
}
id := b.ID
again := &model.Ban{ListID: 7, Address: "203.0.113.9", Expires: in(2 * time.Hour), Who: "guard", Reason: "again"}
if existing, err := s.PutBan(ctx, again); err != nil || !existing || again.ID != id {
t.Fatalf("renewed ban: %v %v id %d, want %d", existing, err, again.ID, id)
}
port := &model.Ban{ListID: 7, Address: "2001:db8:1:2::/64", Protocol: "tcp", Port: 22}
if _, err := s.PutBan(ctx, port); err != nil {
t.Fatal(err)
}
gone := &model.Ban{ListID: 7, Address: "198.51.100.1", Expires: in(-time.Second)}
if _, err := s.PutBan(ctx, gone); err != nil {
t.Fatal(err)
}
other := &model.Ban{ListID: 8, Address: "198.51.100.2"}
if _, err := s.PutBan(ctx, other); err != nil {
t.Fatal(err)
}
list, err := s.ListBans(ctx, 7)
if err != nil || len(list) != 2 {
t.Fatalf("list 7: %v %+v", err, list)
}
if list[0].Reason != "again" || list[0].Timeout < 7100 || list[0].Timeout > 7200 || list[1].Expires != nil || list[1].Timeout != 0 {
t.Errorf("bans: %+v %+v", list[0], list[1])
}
if n, _ := s.CountBans(ctx, 7); n != 2 {
t.Errorf("count %d", n)
}
// An ended ban is renewed as new.
if existing, _ := s.PutBan(ctx, &model.Ban{ListID: 7, Address: "198.51.100.1", Expires: in(time.Minute)}); existing {
t.Error("an ended ban counted as existing")
}
if n, err := s.PruneBans(ctx, []int64{7}); err != nil || n != 1 {
t.Errorf("prune: %d %v (list 8's ban)", n, err)
}
if all, _ := s.ListBans(ctx, 0); len(all) != 3 {
t.Errorf("after pruning: %+v", all)
}
if err := s.DeleteBan(ctx, id); err != nil {
t.Fatal(err)
}
if _, err := s.GetBan(ctx, id); !errors.Is(err, ErrNotFound) {
t.Errorf("deleted ban: %v", err)
}
}
+113 -3
View File
@@ -82,6 +82,7 @@ func Check(st *model.State, env *Env) []Problem {
for _, x := range st.Redirects {
c.redirect(x)
}
c.banLists()
return c.out
}
@@ -233,9 +234,9 @@ func (c *checker) sets() {
for _, s := range c.st.Sets {
obj := "set " + s.Name
c.name("set", obj, s.Name, seen)
for _, pre := range []string{"meter_", "conn_", "quota_", "stats_"} {
for _, pre := range []string{"meter_", "conn_", "quota_", "stats_", "ban_"} {
if strings.HasPrefix(strings.ToLower(s.Name), pre) {
c.add(Error, "set_name_reserved", obj, "set names starting with %s are Housecarl's own (rate limits, quotas, statistics); choose another name", pre)
c.add(Error, "set_name_reserved", obj, "set names starting with %s are Housecarl's own (rate limits, quotas, statistics, ban lists); choose another name", pre)
}
}
for _, e := range s.Entries {
@@ -246,15 +247,79 @@ func (c *checker) sets() {
if len(s.Entries) > 0 && s.Dynamic {
c.add(Error, "set_dynamic_entries", obj, "a dynamic set gets its entries at run time (hc set entries add, the Entries button), each with its timeout; leave out the configured ones, or make it a set that isn't dynamic")
}
if len(s.Entries) == 0 && !s.Dynamic {
if len(s.Entries) == 0 && !s.Runtime() {
c.add(Warning, "set_empty", obj, "the set is empty: rules using it match nothing")
}
if s.Jarl != nil {
c.jarlSet(obj, s)
}
if s.Timeout < 0 || s.Timeout > 0 && !s.Dynamic {
c.add(Error, "set_timeout", obj, "only a dynamic set has a timeout, of 1 second or more")
}
}
}
// dnsLabel is one label of a name in Jarl (underscores allowed, as in
// _service names).
var dnsLabel = regexp.MustCompile(`^[a-z0-9_]([a-z0-9_-]{0,61}[a-z0-9_])?$`)
// jarlSet checks a set whose entries come from Jarl.
func (c *checker) jarlSet(obj string, s *model.AddressSet) {
j := s.Jarl
if len(s.Entries) > 0 {
c.add(Error, "set_jarl_entries", obj, "a Jarl set's entries are the addresses of its names in Jarl; leave out the configured ones, or make it a set that doesn't come from Jarl")
}
if s.Dynamic {
c.add(Error, "set_jarl_dynamic", obj, "a set is dynamic or comes from Jarl, not both")
}
if len(j.Names) == 0 {
c.add(Error, "set_jarl_names", obj, "name at least one name in Jarl, such as nas.example.com (or nas.example.com@internal for one view)")
}
seen := map[string]bool{}
for _, n := range j.Names {
if !validDNSName(n.Name) {
c.add(Error, "set_jarl_names", obj, "%q isn't a DNS name; give the full name, such as nas.example.com (wildcards aren't followed)", n.Name)
continue
}
if strings.ContainsAny(n.View, " \t@") {
c.add(Error, "set_jarl_names", obj, "%q isn't a view name: write NAME@VIEW, such as nas.example.com@internal", n.View)
continue
}
if seen[n.String()] {
c.add(Error, "set_jarl_names", obj, "%s is listed twice", n)
}
seen[n.String()] = true
}
if len(j.Within) == 0 {
c.add(Error, "set_jarl_within", obj, "say where its addresses may be (within): zones, whose connected networks count, or networks such as 10.10.0.0/24; an address from Jarl elsewhere is left out, so a changed record can't open the firewall to anything")
}
for _, w := range j.Within {
if a, err := model.ParseAddress(w); err == nil {
if a.Prefix.IsValid() && a.Prefix.Bits() == 0 && !a.First.IsValid() {
c.add(Warning, "set_jarl_anywhere", obj, "within %s lets any address in: whoever can change the names in Jarl decides what the set holds; name zones or networks instead", w)
}
continue
}
if c.st.Zone(w) == nil {
c.add(Error, "set_jarl_within", obj, "within %q: no such zone, and not an address or network", w)
}
}
}
// validDNSName reports whether n is a name a Jarl set can look up: LDH
// labels (and underscores), no wildcard, without the final dot.
func validDNSName(n string) bool {
if n == "" || len(n) > 253 {
return false
}
for _, l := range strings.Split(n, ".") {
if !dnsLabel.MatchString(l) {
return false
}
}
return true
}
func (c *checker) services() {
seen := map[string]string{}
for _, s := range c.st.Services {
@@ -612,3 +677,48 @@ func (c *checker) redirect(x *model.Redirect) {
}
}
}
func (c *checker) banLists() {
seen := map[string]string{}
for _, b := range c.st.BanLists {
obj := "ban list " + b.Name
c.name("banlist", obj, b.Name, seen)
known := false
for _, sc := range model.BanScopes {
known = known || b.Scope == sc
}
if !known {
c.add(Error, "banlist_scope", obj, "the scope is %q; use firewall (the firewall itself), hosts (the list's hosts) or everything", b.Scope)
}
switch {
case b.Scope == model.BanScopeHosts && len(b.Hosts) == 0:
c.add(Error, "banlist_hosts", obj, "a list with scope hosts needs its hosts: the addresses, or sets, of the machines it protects")
case b.Scope != model.BanScopeHosts && len(b.Hosts) > 0:
c.add(Error, "banlist_hosts", obj, "only a list with scope hosts has hosts; leave them out, or use scope hosts")
}
refs := func(what string, list []string) {
for _, r := range list {
if !model.IsLiteral(r) && c.st.Set(r) == nil {
if _, err := model.ParseAddress(r); strings.ContainsAny(r, "./:") {
c.add(Error, "banlist_address", obj, "%s: %v", what, err)
} else {
c.add(Error, "unknown_set", obj, "%s: there is no set %s", what, r)
}
}
}
}
refs("hosts", b.Hosts)
refs("never", b.Never)
if b.DefaultTime < 0 || b.MaxTime < -1 {
c.add(Error, "banlist_time", obj, "ban times are seconds: default_time 0 (an hour) or more, max_time 0 (a week), more, or -1 for no limit")
} else if max := b.BanMaxTime(); max > 0 && b.BanTime() > max {
c.add(Error, "banlist_time", obj, "the default ban time (%ds) is longer than the longest (%ds)", b.BanTime(), max)
}
if b.MaxEntries < 0 || b.MaxEntries > 1000000 {
c.add(Error, "banlist_entries", obj, "max_entries is %d; use 1 to 1000000, or 0 for 10000", b.MaxEntries)
}
if b.GroupV6 != 0 && (b.GroupV6 < 16 || b.GroupV6 > 128) {
c.add(Error, "banlist_group", obj, "group_v6 is /%d; use 16 to 128 (64 bans what one customer usually gets, 128 one address)", b.GroupV6)
}
}
}
+53 -1
View File
@@ -24,7 +24,21 @@ func base() *model.State {
},
Forwards: []*model.PortForward{{ID: 7, Position: 1, Name: "jellyfin", Protocols: []string{"tcp"}, Ports: "8096", To: "10.0.10.57",
PublicAddresses: []string{"203.0.113.7"}, Hairpin: true}},
SNATs: []*model.SourceNAT{{ID: 8, Position: 1, Match: model.Match{OutZones: []string{"wan"}, Sources: []string{"trusted"}}}},
SNATs: []*model.SourceNAT{{ID: 8, Position: 1, Match: model.Match{OutZones: []string{"wan"}, Sources: []string{"trusted"}}}},
BanLists: []*model.BanList{{ID: 9, Name: "web", Scope: model.BanScopeHosts, Hosts: []string{"10.0.10.57"}, Never: []string{"trusted"}}},
}
}
func jarlSource() *model.JarlSource {
return &model.JarlSource{Names: []model.JarlName{{Name: "nas.example.com", View: "internal"}}, Within: []string{"lan", "10.9.0.0/16"}}
}
func TestJarlSetValid(t *testing.T) {
st := base()
st.Sets[0].Entries, st.Sets[0].Jarl = nil, jarlSource()
st.Sets[0].Jarl.Names = append(st.Sets[0].Jarl.Names, model.JarlName{Name: "_svc.files.example.com"})
if ps := Check(st, nil); len(ps) > 0 {
t.Fatalf("problems: %+v", ps)
}
}
@@ -76,6 +90,33 @@ func TestCodes(t *testing.T) {
}},
{"set_timeout", func(st *model.State) { st.Sets[0].Timeout = 60 }},
{"set_dynamic_entries", func(st *model.State) { st.Sets[0].Dynamic = true }},
{"set_jarl_entries", func(st *model.State) { st.Sets[0].Jarl = jarlSource() }},
{"set_jarl_dynamic", func(st *model.State) {
st.Sets[0].Entries, st.Sets[0].Dynamic, st.Sets[0].Jarl = nil, true, jarlSource()
}},
{"set_jarl_names", func(st *model.State) {
st.Sets[0].Entries, st.Sets[0].Jarl = nil, &model.JarlSource{Within: []string{"lan"}}
}},
{"set_jarl_names", func(st *model.State) {
st.Sets[0].Entries, st.Sets[0].Jarl = nil, jarlSource()
st.Sets[0].Jarl.Names[0].Name = "*.example.com"
}},
{"set_jarl_names", func(st *model.State) {
st.Sets[0].Entries, st.Sets[0].Jarl = nil, jarlSource()
st.Sets[0].Jarl.Names = append(st.Sets[0].Jarl.Names, st.Sets[0].Jarl.Names[0])
}},
{"set_jarl_within", func(st *model.State) {
st.Sets[0].Entries, st.Sets[0].Jarl = nil, jarlSource()
st.Sets[0].Jarl.Within = nil
}},
{"set_jarl_within", func(st *model.State) {
st.Sets[0].Entries, st.Sets[0].Jarl = nil, jarlSource()
st.Sets[0].Jarl.Within = []string{"dmz"}
}},
{"set_jarl_anywhere", func(st *model.State) {
st.Sets[0].Entries, st.Sets[0].Jarl = nil, jarlSource()
st.Sets[0].Jarl.Within = []string{"::/0"}
}},
{"service_empty", func(st *model.State) { st.Services[0].Entries = nil }},
{"service_entry", func(st *model.State) { st.Services[0].Entries = []string{"tcp/99999"} }},
{"unknown_zone", func(st *model.State) { st.Rules[1].InZones = []string{"dmz"} }},
@@ -136,6 +177,17 @@ func TestCodes(t *testing.T) {
{"redirect_service", func(st *model.State) {
st.Redirects = []*model.Redirect{{ID: 11, Position: 1, Match: model.Match{InZones: []string{"lan"}}}}
}},
{"banlist_name", func(st *model.State) { st.BanLists[0].Name = "web list" }},
{"banlist_scope", func(st *model.State) { st.BanLists[0].Scope = "world" }},
{"banlist_hosts", func(st *model.State) { st.BanLists[0].Hosts = nil }},
{"banlist_hosts", func(st *model.State) { st.BanLists[0].Scope = model.BanScopeFirewall }},
{"banlist_address", func(st *model.State) { st.BanLists[0].Never = []string{"10.0.0.300"} }},
{"unknown_set", func(st *model.State) { st.BanLists[0].Hosts = []string{"webservers"} }},
{"banlist_time", func(st *model.State) { st.BanLists[0].MaxTime = -2 }},
{"banlist_time", func(st *model.State) { st.BanLists[0].DefaultTime, st.BanLists[0].MaxTime = 7200, 3600 }},
{"banlist_entries", func(st *model.State) { st.BanLists[0].MaxEntries = -1 }},
{"banlist_group", func(st *model.State) { st.BanLists[0].GroupV6 = 8 }},
{"set_name_reserved", func(st *model.State) { st.Sets[0].Name = "ban_3" }},
{"redirect_to_port", func(st *model.State) {
st.Redirects = []*model.Redirect{{ID: 11, Position: 1, Match: model.Match{InZones: []string{"lan"}, Services: []string{"dns"}}, ToPort: "53-54"}}
}},
+3
View File
@@ -4,6 +4,7 @@ import { wordmark, logo } from "./brand.js";
import { api, get, state, can, setUnauthorizedHandler, setChangeHandler, loadDraft, onDraftChange, setImmediate, useDraft } from "./api.js";
import * as changes from "./pages/changes.js";
import * as connections from "./pages/connections.js";
import * as bans from "./pages/bans.js";
import * as tracePage from "./pages/trace.js";
import * as matrixPage from "./pages/matrix.js";
import * as statsPage from "./pages/stats.js";
@@ -32,6 +33,7 @@ const routes = [
{ pattern: /^services$/, page: services, nav: "services" },
{ pattern: /^log$/, page: log, nav: "log" },
{ pattern: /^connections$/, page: connections, nav: "connections" },
{ pattern: /^bans$/, page: bans, nav: "bans" },
{ pattern: /^trace$/, page: tracePage, nav: "trace" },
{ pattern: /^stats$/, page: statsPage, nav: "stats" },
{ pattern: /^settings$/, page: settings, nav: "settings" },
@@ -60,6 +62,7 @@ const navItems = [
["stats", "#/stats", "Statistics", "log"],
["log", "#/log", "Log", "log"],
["connections", "#/connections", "Connections", "log"],
["bans", "#/bans", "Bans", "bans"],
["trace", "#/trace", "Trace", "log"],
] },
["changes", "#/changes", "Changes", "config"],
+2 -2
View File
@@ -331,7 +331,7 @@ export function heatmapTable(hm, unit) {
// so the list is its own table view.
// extra(key), when given, adds something after an entry's label (e.g. a
// link that excludes it).
export function barList(entries, { total, label = (k) => k, empty = "No data yet.", href, extra, format = compact } = {}) {
export function barList(entries, { total, label = (k) => k, empty = "No data yet.", href, extra, format = compact, share: showShare = true } = {}) {
if (!entries.length) return h("div", { class: "empty" }, empty);
// Entries may be in a natural order (e.g. latency buckets), not by size.
const peak = Math.max(1, ...entries.map((e) => e.count));
@@ -345,7 +345,7 @@ export function barList(entries, { total, label = (k) => k, empty = "No data yet
return h("li", { title: (e.name ? e.name + " (" + name + ")" : name) + ": " + format(e.count) + " (" + share.toFixed(1) + "%)" },
h("div", { class: "bar-text" },
link ? h("a", { href: link, class: "bar-label" }, shown) : h("span", { class: "bar-label" }, shown),
h("span", { class: "bar-value" }, extra ? [extra(e.key), " "] : null, format(e.count), h("span", { class: "muted" }, " · " + share.toFixed(share < 10 ? 1 : 0) + "%"))),
h("span", { class: "bar-value" }, extra ? [extra(e.key), " "] : null, format(e.count), showShare ? h("span", { class: "muted" }, " · " + share.toFixed(share < 10 ? 1 : 0) + "%") : null)),
h("div", { class: "bar-track" }, h("div", { class: "bar-fill", style: { width: Math.max(0.5, (e.count / peak) * 100) + "%" } })));
}));
}
+4 -32
View File
@@ -1,8 +1,8 @@
// Your account: password and API tokens.
import { h, mount, table, time, modal, input, field, select, errorBox, toast, copyBox } from "../ui.js";
import { get, api, state, catalog, can } from "../api.js";
import { h, mount, table, time, input, field, errorBox, toast } from "../ui.js";
import { get, api, state, catalog } from "../api.js";
import { rerender } from "../app.js";
import { revoke, rotateDialog } from "./users.js";
import { revoke, rotateDialog, tokenDialog } from "./users.js";
export async function render(el) {
const me = state.me;
@@ -30,7 +30,7 @@ export async function render(el) {
h("div", { class: "panel" },
h("div", { class: "page-head" }, h("div", { class: "grow" }, h("h2", null, "API tokens"),
h("div", { class: "muted" }, "For the hc CLI and scripts: 'hc config set token'.")),
h("button", { onclick: () => newToken(cat) }, "New token")),
h("button", { onclick: () => tokenDialog(cat) }, "New token")),
table([
{ label: "Name", render: (t) => t.name },
{ label: "May", render: (t) => h("span", { class: "mono small" }, t.grant_strings.join(" ")) },
@@ -41,31 +41,3 @@ export async function render(el) {
h("button", { class: "small reveal", onclick: () => revoke(t) }, "Revoke")] },
], tokens, { empty: "No tokens." })));
}
async function newToken(cat) {
const name = input({ placeholder: "laptop, backup script…" });
// Only roles within what the user may do: a token can never do more
// than its user, so offering more would only mislead.
const roles = Object.keys(cat.roles).filter((r) => cat.roles[r].every((g) => {
const [res, acc] = g.split(":");
return can(res, acc === "write");
}));
const role = select([["", "everything you may do"], ...roles.map((r) => [r, r + ": " + cat.roles[r].join(" ")])], "");
const expires = select([["", "never"], ["30d", "30 days"], ["90d", "90 days"], ["365d", "a year"]], "90d");
const err = h("div");
await modal("New API token", h("div", null, h("div", { class: "form-grid" }, field("Name", name), field("May do", role), field("Expires", expires)), err), [
{ label: "Cancel", value: null },
{ label: "Create", kind: "primary", onclick: async () => {
mount(err);
try {
const t = await api("POST", "/me/tokens", { body: { name: name.value.trim(), role: role.value || undefined, expires_in: expires.value || undefined } });
await modal("Token created", h("div", null, h("p", null, "Copy it now: it is not shown again."), copyBox(t.secret)));
rerender();
return true;
} catch (e) {
mount(err, errorBox(e));
return false;
}
} },
]);
}
+146
View File
@@ -0,0 +1,146 @@
// Ban lists, and the addresses banned in them (docs/bans.md).
import { h, mount, table, modal, errorBox, field, input, select, checkbox, confirmDialog, badge, toast, relTime } from "../ui.js";
import { get, api, change, can } from "../api.js";
import { rerender, onLeave } from "../app.js";
import { listEditor, fmtSeconds, parseSeconds, rowClass, lookupNames, named, hits, counters } from "../widgets.js";
const scopes = {
firewall: "the firewall itself",
hosts: "its hosts, behind the firewall",
everything: "the firewall and everything behind it",
};
export async function render(el) {
const config = can("config", false);
const write = can("config", true);
const lists = await get("/banlists");
const out = h("div");
const draw = async () => {
const [bans, cs] = await Promise.all([get("/bans"), counters()]);
const names = await lookupNames(bans.map((b) => b.address));
const count = {};
for (const b of bans) count[b.list_id] = (count[b.list_id] || 0) + 1;
mount(out,
h("h2", null, "Ban lists"),
h("div", { class: "panel" }, table([
{ label: "Name", render: (l) => h("strong", null, l.name) },
{ label: "Keeps banned addresses from", render: (l) => [scopes[l.scope] || l.scope, l.hosts && l.hosts.length ? h("span", { class: "mono small" }, ": " + l.hosts.join(", ")) : null] },
{ label: "Never bans", render: (l) => h("span", { class: "mono small" }, (l.never || []).join(", ") || "–") },
{ label: "Bans", render: (l) => String(count[l.id] || 0) },
{ label: "Dropped", render: (l) => hits(cs, "ban" + l.id) },
{ label: "Comment", render: (l) => l.comment },
write ? { label: "", tdClass: "actions-cell", render: (l) => l.pending === "deleted" ? badge("deleted", "err") : [
h("button", { class: "small reveal", onclick: () => listDialog(l) }, "Edit"), " ",
h("button", { class: "small reveal", onclick: () => del(l) }, "Delete")] } : null,
].filter(Boolean), lists, { empty: "No ban lists yet.", rowClass })),
h("h2", null, "Banned addresses"),
h("div", { class: "panel" }, table([
{ label: "Address", render: (b) => h("span", { class: "mono" }, named(names, b.address)) },
{ label: "List", render: (b) => b.list },
{ label: "Ports", render: (b) => b.protocol ? b.protocol + "/" + b.port : "all" },
{ label: "Ends in", render: (b) => b.timeout ? fmtSeconds(b.timeout) : "never" },
{ label: "Banned", render: (b) => relTime(b.created) },
{ label: "By", render: (b) => [b.who, b.source ? h("span", { class: "muted small" }, " " + b.source) : null] },
{ label: "Reason", render: (b) => h("span", { class: "small", title: b.reason || "" }, (b.reason || "").length > 80 ? b.reason.slice(0, 79) + "…" : b.reason || "") },
{ label: "", tdClass: "actions-cell", render: (b) => mayBan() ? h("button", { class: "small reveal", onclick: () => unban(b, draw) }, "Unban") : null },
], bans, { empty: "Nothing is banned." })));
};
mount(el,
h("div", { class: "page-head" },
h("div", { class: "grow" }, h("h1", null, "Bans"), h("div", { class: "muted" },
"Programs such as Guard and fail2ban ban addresses into a ban list; the list says what a banned address is kept from. Bans are dropped ahead of every rule, established connections too, and end by themselves.")),
h("div", { class: "actions" },
lists.length && mayBan() ? h("button", { onclick: () => banDialog(lists, draw) }, "Ban an address…") : null,
write ? h("button", { class: "primary", onclick: () => listDialog(null) }, "Add ban list") : null)),
config || lists.length ? out : h("div", { class: "empty" }, "No ban lists you may see."));
await draw();
const t = setInterval(() => { if (!document.hidden && !document.querySelector(".modal")) draw().catch(() => {}); }, 15000);
onLeave(() => clearInterval(t));
}
// mayBan: a bans:write grant may name its lists; the server checks.
function mayBan() { return can("bans", true); }
async function unban(b, draw) {
const what = b.address + (b.protocol ? " (" + b.protocol + "/" + b.port + ")" : "");
if (!await confirmDialog("Unban", "Lift the ban on " + what + " in " + b.list + "?", "Unban")) return;
try { await api("DELETE", "/banlists/" + encodeURIComponent(b.list) + "/bans/" + b.id); toast("Unbanned " + what); draw(); } catch (e) { toast(e.message, "err"); }
}
async function del(l) {
if (!await confirmDialog("Delete ban list", "Delete ban list " + l.name + "? Its bans are lifted, and programs banning into it are refused.", "Delete", "danger")) return;
try { await change("DELETE", "/banlists/" + l.id); rerender(); } catch (e) { modal("Not deleted", errorBox(e)); }
}
async function banDialog(lists, draw) {
const list = select(lists.map((l) => [l.name, l.name + " (" + l.scope + ")"]), lists[0].name);
const addr = input({ placeholder: "203.0.113.9 or 2001:db8::9", class: "mono" });
const proto = select([["", "every port"], ["tcp", "TCP"], ["udp", "UDP"]], "");
const ports = input({ placeholder: "22, ssh, http,https" });
const t = input({ placeholder: "the list's default", class: "short" });
const reason = input({ placeholder: "why, for the record" });
const err = h("div");
await modal("Ban an address", h("div", null, h("div", { class: "form-grid" },
field("Ban list", list), field("Address", addr, "An IPv6 address is banned with the list's grouping: its /64 unless the list says otherwise."),
field("Ports", h("div", { class: "filter-row" }, proto, ports), "Every port, or a protocol's ports (numbers or service names)."),
field("For", t, "1h, 1d; empty: the list's default."), field("Reason", reason)), err), [
{ label: "Cancel", value: null },
{ label: "Ban", kind: "danger", onclick: async () => {
mount(err);
const secs = parseSeconds(t.value);
if (secs === null) { mount(err, h("div", { class: "msg err" }, "Write the time as 1h, 30m or 1d.")); return false; }
if (proto.value && !ports.value.trim()) { mount(err, h("div", { class: "msg err" }, "Give the ports, or ban every port.")); return false; }
try {
const r = await api("POST", "/banlists/" + encodeURIComponent(list.value) + "/bans", { body: { address: addr.value.trim(), protocol: proto.value,
ports: proto.value ? ports.value.trim() : undefined, time: secs || 0, reason: reason.value.trim(), source: "web" } });
toast("Banned " + r.bans.map((b) => b.address + (b.protocol ? " " + b.protocol + "/" + b.port : "")).join(", ") + (r.closed ? "; " + r.closed + " connections closed" : ""));
draw();
return true;
} catch (e) { mount(err, errorBox(e)); return false; }
} },
], { wide: true });
}
async function listDialog(l) {
l = l || { scope: "firewall", hosts: [], never: [] };
const name = input({ value: l.name || "", placeholder: "web", disabled: !!l.id });
const scope = select(Object.entries(scopes).map(([k, v]) => [k, k + ": " + v]), l.scope);
const hostsEd = listEditor(l.hosts || [], { placeholder: "10.0.10.57, 2001:db8:10::57, or a set", addLabel: "Add host" });
const never = listEditor(l.never || [], { placeholder: "198.51.100.0/24, or a set", addLabel: "Add address" });
const hostFields = field("Hosts", hostsEd, "The machines this list protects: their addresses, IPv4 and IPv6, or sets. Port forwards to them are covered too.").filter(Boolean);
const showHosts = () => { for (const n of hostFields) n.classList.toggle("hidden", scope.value !== "hosts"); };
scope.addEventListener("change", showHosts);
showHosts();
const time = input({ value: l.default_time ? fmtSeconds(l.default_time) : "", placeholder: "1h", class: "short" });
const maxTime = input({ value: l.max_time > 0 ? fmtSeconds(l.max_time) : "", placeholder: "7d", class: "short" });
const forever = checkbox("no limit: bans may last for ever", l.max_time < 0);
const entries = input({ value: l.max_entries || "", placeholder: "10000", class: "short", type: "number" });
const group = input({ value: l.group_v6 || "", placeholder: "64", class: "short", type: "number" });
const disabled = checkbox("disabled: the list's bans aren't in force", !!l.disabled);
const comment = input({ value: l.comment || "" });
const err = h("div");
await modal(l.id ? "Edit ban list " + l.name : "Add ban list", h("div", null, h("div", { class: "form-grid" },
field("Name", name, l.id ? "Programs and tokens know the list by its name, so it can't be changed." : "Programs ban into the list by this name; give each a token for it alone (bans:write:NAME)."),
field("Scope", scope, "What a banned address is kept from."), ...hostFields,
field("Never ban", never, "Besides what is never banned anyway: inside networks, the firewall's own addresses and whoever asks."),
field("Ban time", time, "A ban's length when the program asks for none."),
field("Longest ban", maxTime, "Longer bans are cut to this."), h("label"), forever,
field("Most bans", entries), field("IPv6 banned by", group, "Prefix length: 64 bans what one customer usually gets, 128 one address."),
h("label"), disabled, field("Comment", comment)), err), [
{ label: "Cancel", value: null },
{ label: l.id ? "Save" : "Add ban list", kind: "primary", onclick: async () => {
mount(err);
const t = parseSeconds(time.value), m = parseSeconds(maxTime.value);
if (t === null || m === null) { mount(err, h("div", { class: "msg err" }, "Write times as 1h, 30m or 7d.")); return false; }
const body = { scope: scope.value, hosts: scope.value === "hosts" ? hostsEd.value() : [], never: never.value(),
default_time: t || 0, max_time: forever.input.checked ? -1 : m || 0, max_entries: parseInt(entries.value, 10) || 0,
group_v6: parseInt(group.value, 10) || 0, disabled: disabled.input.checked, comment: comment.value.trim() };
if (!l.id) body.name = name.value.trim();
try {
await change(l.id ? "PATCH" : "POST", l.id ? "/banlists/" + l.id : "/banlists", body);
rerender();
return true;
} catch (e) { mount(err, errorBox(e)); return false; }
} },
], { wide: true, sticky: true });
}
+13 -6
View File
@@ -3,7 +3,7 @@
import { h, mount, table, badge, problemsList, time, plural } from "../ui.js";
import { get, can } from "../api.js";
import { onLeave } from "../app.js";
import { describe, hits, humanCount, fmtBytes, usageBar, lookupNames, named } from "../widgets.js";
import { describe, hits, humanCount, fmtBytes, usageBar, lookupNames, named, pct } from "../widgets.js";
import { lineChart, chartCard, legend, timeTable, stepSeconds } from "../charts.js";
import { bytes } from "./stats.js";
@@ -72,19 +72,26 @@ export async function render(el) {
}
// quotaPanel lists every rule's quota with what it used: the total, or
// for a per-address quota the busiest address (the rest are on the rule).
// for a per-address quota how many addresses used theirs up and the
// busiest (the rest are on the rule). A per-address quota's bar is all
// its addresses together, so one address alone can't fill it.
function quotaPanel(quotas, names) {
const perAddress = (q) => q.quota.per_source || q.quota.per_destination;
const used = (q) => perAddress(q) ? ((q.addresses || [])[0] || { used: 0 }).used : q.used;
const addrs = (q) => q.addresses || [];
const usedUp = (q) => addrs(q).filter((a) => a.used >= q.quota.bytes).length;
const bar = (q) => perAddress(q)
? usageBar(addrs(q).reduce((n, a) => n + Math.min(a.used, q.quota.bytes), 0), addrs(q).length * q.quota.bytes)
: usageBar(q.used, q.quota.bytes);
return h("div", { class: "panel" },
h("div", { class: "panel-head" }, h("h2", null, "Quotas"), h("a", { class: "button small", href: "#/stats" }, "Statistics")),
table([
{ label: "Rule", render: (q) => h("a", { href: "#/rules?rule=" + q.rule_id }, q.rule) },
{ label: "Allowance", render: (q) => fmtBytes(q.quota.bytes) + (q.quota.per_source ? " each source" : q.quota.per_destination ? " each destination" : "") },
{ label: "Used", render: (q) => perAddress(q)
? [plural((q.addresses || []).length, "address", "addresses"), q.addresses && q.addresses.length ? [", busiest ", named(names, q.addresses[0].address), " " + fmtBytes(used(q))] : ""]
: fmtBytes(q.used) },
{ label: "", render: (q) => usageBar(used(q), q.quota.bytes) },
? [plural(addrs(q).length, "address", "addresses"), usedUp(q) ? ", " + usedUp(q) + " used up" : "",
addrs(q).length ? [", busiest ", named(names, addrs(q)[0].address), " " + fmtBytes(addrs(q)[0].used) + " (" + pct(addrs(q)[0].used, q.quota.bytes) + ")"] : ""]
: fmtBytes(q.used) + " (" + pct(q.used, q.quota.bytes) + ")" },
{ label: "", render: (q) => addrs(q).length || !perAddress(q) ? bar(q) : "" },
{ label: "Next reset", render: (q) => q.next_reset ? time(q.next_reset) : h("span", { class: "muted" }, "by hand") },
], quotas));
}
+21 -5
View File
@@ -1,5 +1,5 @@
// History: every configuration change, with undo.
import { h, mount, table, time, confirmDialog, modal, errorBox, badge } from "../ui.js";
import { h, mount, table, time, confirmDialog, modal, errorBox, badge, diffView } from "../ui.js";
import { get, change, can, state } from "../api.js";
import { rerender } from "../app.js";
@@ -29,7 +29,7 @@ async function undo(e) {
}
async function entry(el, id) {
const e = await get("/history/" + id);
const [e, d] = await Promise.all([get("/history/" + id), get("/history/" + id + "/diff").catch(() => null)]);
mount(el,
h("nav", { class: "back-link" }, h("a", { href: "#/history" }, "← History")),
h("div", { class: "page-head" },
@@ -40,7 +40,11 @@ async function entry(el, id) {
can("config", true) && !e.reverted_by ? h("div", { class: "actions" }, h("button", { onclick: () => undo(e) }, "Undo")) : null),
e.changes.map((c) => h("div", { class: "panel" },
h("h3", null, action(c) + " " + c.kind + " ", h("span", { class: "mono" }, c.label)),
fieldsTable(c))));
fieldsTable(c))),
d ? h("div", { class: "panel" },
h("h3", null, "Ruleset"),
h("p", { class: "muted small" }, "What this change did to the ruleset, rendered by this version of Housecarl."),
diffView(d.diff)) : null);
}
function action(c) {
@@ -50,9 +54,10 @@ function action(c) {
}
// fieldsTable lists the fields that differ (all of them for a create or
// delete).
// delete). An object within (settings' builtins) is listed by its own
// fields, "builtins.drop_invalid", so only what changed shows.
function fieldsTable(c) {
const a = c.before || {}, b = c.after || {};
const a = flatten(c.before || {}), b = flatten(c.after || {});
const keys = [...new Set([...Object.keys(a), ...Object.keys(b)])].sort();
const show = (v, k) => {
if (v === undefined || v === "") return "–";
@@ -68,3 +73,14 @@ function fieldsTable(c) {
c.after ? { label: c.before ? "After" : "Value", render: (k) => h("span", { class: "mono small" }, show(b[k], k)) } : null,
].filter(Boolean), rows);
}
// flatten lifts the fields of objects within an object to dotted keys
// ({builtins: {loopback: true}} → {"builtins.loopback": true}); lists
// stay whole.
function flatten(o, prefix = "", out = {}) {
for (const [k, v] of Object.entries(o)) {
if (v && typeof v === "object" && !Array.isArray(v) && Object.keys(v).length) flatten(v, prefix + k + ".", out);
else out[prefix + k] = v;
}
return out;
}
+11 -5
View File
@@ -2,7 +2,7 @@
import { h, mount, table, modal, errorBox, field, input, select, checkbox, confirmDialog, badge, textarea, toast } from "../ui.js";
import { get, api, change, can } from "../api.js";
import { rerender } from "../app.js";
import { names, matchEditor, describe, hits, counters, listEditor, rowClass, live, fmtBytes, quotas, usageBar, lookupNames, named } from "../widgets.js";
import { names, matchEditor, describe, hits, counters, listEditor, rowClass, live, fmtBytes, quotas, usageBar, lookupNames, named, pct } from "../widgets.js";
const chains = [
["input", "Input", "Traffic to the firewall itself."],
@@ -165,10 +165,11 @@ function quotaLine(r, q) {
const each = qt.per_source ? " each source" : qt.per_destination ? " each destination" : "";
const resets = qt.reset === "monthly" ? "monthly" : qt.reset || "reset by hand";
let used = fmtBytes(qt.bytes) + each;
if (q && !each) used = fmtBytes(q.used) + " of " + fmtBytes(qt.bytes) + " used";
if (q && !each) used = fmtBytes(q.used) + " of " + fmtBytes(qt.bytes) + " used (" + pct(q.used, qt.bytes) + ")";
else if (q) {
const n = (q.addresses || []).length;
used += " · " + (n === 1 ? "1 address" : n + " addresses") + (n ? ", busiest " + fmtBytes(q.addresses[0].used) : "");
const a = q.addresses || [], n = a.length;
const up = a.filter((x) => x.used >= qt.bytes).length;
used += " · " + (n === 1 ? "1 address" : n + " addresses") + (up ? ", " + up + " used up" : "") + (n ? ", busiest " + fmtBytes(a[0].used) + " (" + pct(a[0].used, qt.bytes) + ")" : "");
}
const text = "quota " + used + " · " + resets;
return h("div", { class: "muted small" }, q && r.pending !== "new"
@@ -193,12 +194,13 @@ async function quotaDialog(r, q) {
perAddress ? table([
{ label: "Address", render: (a) => h("span", { class: "mono" }, named(names, a.address)) },
{ label: "Used", render: (a) => fmtBytes(a.used) },
{ label: "", class: "num", tdClass: "num", render: (a) => pct(a.used, qt.bytes) },
{ label: "", render: (a) => usageBar(a.used, qt.bytes) },
write ? { label: "", render: (a) => h("span", { class: "nowrap" },
h("button", { class: "small", onclick: then(() => resetQuota(r, a.address)) }, "Reset"), " ",
h("button", { class: "small", onclick: then(() => giveBack(r, a.address)) }, "Give back…")) } : null,
].filter(Boolean), q.addresses || [], { empty: "No address has used it since the last reset." })
: h("div", null, h("strong", null, fmtBytes(q.used)), " of " + fmtBytes(qt.bytes) + " used", usageBar(q.used, qt.bytes))),
: h("div", null, h("strong", null, fmtBytes(q.used)), " of " + fmtBytes(qt.bytes) + " used (" + pct(q.used, qt.bytes) + ")", usageBar(q.used, qt.bytes))),
[
...(write ? [{ label: perAddress ? "Reset all" : "Reset", value: "reset" }, { label: perAddress ? "Give back to all…" : "Give back…", value: "give" }] : []),
{ label: "Close", value: null },
@@ -258,6 +260,9 @@ export async function ruleDialog(r, chain) {
r = r || { chain, action: "accept" };
const ch = select(["input", "forward", "output", "early"], r.chain || chain);
const match = matchEditor(r, n);
const chainZones = () => match.zones(ch.value !== "output", ch.value === "forward" || ch.value === "output");
ch.addEventListener("change", chainZones);
chainZones();
const action = select([["accept", "accept"], ["drop", "drop"], ["reject", "reject"], ["log", "log only (and go on)"]], r.action);
const rejectWith = select([["", "with an ICMP error"], ["tcp-reset", "with a TCP reset (TCP rules only)"]], r.reject_with || "");
const every = checkbox("every packet: also stop connections already open, not just new ones", r.every_packet);
@@ -369,6 +374,7 @@ export async function ruleDialog(r, chain) {
const addersLabel = h("label");
sections.raw.onShow = (on) => {
for (const n of [...matchNodes, ...actionField, ...logField]) n.classList.toggle("hidden", on);
if (!on) chainZones();
for (const [k, sec] of Object.entries(sections)) {
if (k === "raw") continue;
if (on && sec.open) for (const n of sec.nodes) n.classList.add("hidden");
+73 -13
View File
@@ -1,5 +1,5 @@
// Address sets, and dynamic sets' entries.
import { h, mount, table, modal, errorBox, field, input, checkbox, confirmDialog, badge, toast } from "../ui.js";
// Address sets, dynamic sets' entries, and what Jarl sets hold.
import { h, mount, table, modal, errorBox, field, input, select, confirmDialog, badge, toast, fmtTime } from "../ui.js";
import { get, api, change, can } from "../api.js";
import { rerender } from "../app.js";
import { listEditor, fmtSeconds, parseSeconds, rowClass } from "../widgets.js";
@@ -9,11 +9,14 @@ export async function render(el) {
const write = can("config", true);
mount(el,
h("div", { class: "page-head" },
h("div", { class: "grow" }, h("h1", null, "Address sets"), h("div", { class: "muted" }, "Named lists of IPv4 and IPv6 addresses, prefixes and ranges, mixed. A dynamic set takes entries at run time, each with a timeout (a captive portal's signed-in clients).")),
h("div", { class: "grow" }, h("h1", null, "Address sets"), h("div", { class: "muted" }, "Named lists of IPv4 and IPv6 addresses, prefixes and ranges, mixed. A dynamic set takes entries at run time, each with a timeout (a captive portal's signed-in clients). A Jarl set holds the addresses of names in Jarl, kept up to date.")),
write ? h("div", { class: "actions" }, h("button", { class: "primary", onclick: () => setDialog(null) }, "Add set")) : null),
h("div", { class: "panel" }, table([
{ label: "Name", render: (s) => h("strong", null, s.name) },
{ label: "Entries", render: (s) => s.dynamic ? [badge("dynamic", "accent"), s.timeout ? " timeout " + fmtSeconds(s.timeout) : "", " ",
{ label: "Entries", render: (s) => s.jarl ? [badge("Jarl", "accent"), " ", h("span", { class: "mono small" }, jarlNames(s).join(", ")),
h("span", { class: "muted small" }, " within " + s.jarl.within.join(", ")), " ",
s.pending ? null : h("button", { class: "small", onclick: () => jarlStatus(s) }, "Addresses…")]
: s.dynamic ? [badge("dynamic", "accent"), s.timeout ? " timeout " + fmtSeconds(s.timeout) : "", " ",
s.pending ? null : h("button", { class: "small", onclick: () => entries(s) }, "Entries…")] : h("span", { class: "mono small" }, s.entries.join(", ")) },
{ label: "Comment", render: (s) => s.comment },
write ? { label: "", tdClass: "actions-cell", render: (s) => s.pending === "deleted" ? badge("deleted", "err") : [
@@ -27,33 +30,52 @@ async function del(s) {
try { await change("DELETE", "/sets/" + s.id); rerender(); } catch (e) { modal("Not deleted", errorBox(e)); }
}
const nameOf = (n) => n.view ? n.name + "@" + n.view : n.name;
const jarlNames = (s) => (s.jarl.names || []).map(nameOf);
// parseJarlName reads "name" or "name@view".
function parseJarlName(v) {
const i = v.indexOf("@");
return i < 0 ? { name: v } : { name: v.slice(0, i), view: v.slice(i + 1) };
}
async function setDialog(s) {
s = s || { entries: [] };
const zones = (await get("/zones")).map((z) => z.name);
const name = input({ value: s.name || "", placeholder: "trusted" });
const list = listEditor(s.entries, { placeholder: "192.168.1.0/24, 2001:db8:1::/64, 192.168.1.100-192.168.1.199", addLabel: "Add entry" });
const dynamic = checkbox("dynamic: entries are added at run time (the Entries button, hc set entries, a program with a token), each for a time", s.dynamic);
// A dynamic set has no configured entries; only it has a timeout.
const kind = select([["list", "addresses listed here"], ["dynamic", "dynamic: entries added at run time, each for a time"], ["jarl", "from Jarl: the addresses of names in Jarl"]],
s.jarl ? "jarl" : s.dynamic ? "dynamic" : "list");
const names = listEditor(s.jarl ? jarlNames(s) : [], { placeholder: "nas.example.com@internal", addLabel: "Add name" });
const within = listEditor(s.jarl ? s.jarl.within : [], { placeholder: "lan, or 10.10.0.0/24", addLabel: "Add zone or network", suggestions: zones });
const timeout = input({ value: s.timeout ? fmtSeconds(s.timeout) : "", placeholder: "1h, 1d (none: until removed)", class: "short" });
// Each kind shows its own fields.
const entryFields = field("Entries", list).filter(Boolean);
const timeoutFields = field("Entry timeout", timeout, "How long an entry stays unless it is added with its own. Empty: until removed (or a reboot).").filter(Boolean);
const jarlFields = [
...field("Names in Jarl", names, "The set holds their A and AAAA records' addresses (all of them), CNAMEs followed. Write NAME@VIEW, or just NAME if only one of Jarl's views has it. Read again every minute.").filter(Boolean),
...field("Within", within, "Where its addresses may be: zones (the networks connected to their interfaces) or networks. An address elsewhere is left out, so a changed record in Jarl can't let anything else in.").filter(Boolean),
];
const showKind = () => {
for (const n of entryFields) n.classList.toggle("hidden", dynamic.input.checked);
for (const n of timeoutFields) n.classList.toggle("hidden", !dynamic.input.checked);
for (const n of entryFields) n.classList.toggle("hidden", kind.value !== "list");
for (const n of timeoutFields) n.classList.toggle("hidden", kind.value !== "dynamic");
for (const n of jarlFields) n.classList.toggle("hidden", kind.value !== "jarl");
};
dynamic.input.addEventListener("change", showKind);
kind.addEventListener("change", showKind);
showKind();
const timeout = input({ value: s.timeout ? fmtSeconds(s.timeout) : "", placeholder: "1h, 1d (none: until removed)", class: "short" });
const comment = input({ value: s.comment || "" });
const err = h("div");
await modal(s.id ? "Edit set " + s.name : "Add set", h("div", null, h("div", { class: "form-grid" },
field("Name", name), ...entryFields, h("label"), dynamic, ...timeoutFields, field("Comment", comment)), err), [
field("Name", name), ...field("Kind", kind), ...entryFields, ...timeoutFields, ...jarlFields, field("Comment", comment)), err), [
{ label: "Cancel", value: null },
{ label: s.id ? "Save" : "Add set", kind: "primary", onclick: async () => {
mount(err);
const t = parseSeconds(timeout.value);
if (t === null) { mount(err, h("div", { class: "msg err" }, "Write the timeout as 1h, 30m or 1d.")); return false; }
const k = kind.value;
try {
await change(s.id ? "PATCH" : "POST", s.id ? "/sets/" + s.id : "/sets", { name: name.value.trim(), entries: dynamic.input.checked ? [] : list.value(), dynamic: dynamic.input.checked,
timeout: dynamic.input.checked ? t : 0, comment: comment.value.trim() });
await change(s.id ? "PATCH" : "POST", s.id ? "/sets/" + s.id : "/sets", { name: name.value.trim(), entries: k === "list" ? list.value() : [], dynamic: k === "dynamic",
timeout: k === "dynamic" ? t : 0, jarl: k === "jarl" ? { names: names.value().map(parseJarlName), within: within.value() } : null, comment: comment.value.trim() });
rerender();
return true;
} catch (e) { mount(err, errorBox(e)); return false; }
@@ -61,6 +83,44 @@ async function setDialog(s) {
], { wide: true, sticky: true });
}
// jarlStatus shows what a Jarl set holds: each name's addresses in Jarl,
// used or left out (and why).
async function jarlStatus(s) {
const body = h("div");
const path = "/sets/" + encodeURIComponent(s.name) + "/jarl";
const draw = (js) => {
const rows = [];
for (const n of js.names) {
// A new node for each row: a node can only be in one place.
const label = () => [h("span", { class: "mono" }, n.name), n.chain && n.chain.length ? h("span", { class: "muted small" }, " → " + n.chain.join(" → ")) : null];
const view = n.view || n.found_view || "";
if (n.problem) rows.push({ label, view, address: "", used: h("span", null, badge("no addresses", "err"), " " + n.problem) });
else if (!n.addresses.length) rows.push({ label, view, address: "", used: n.error ? "not read: " + n.error : "not read yet" });
for (const a of n.addresses) rows.push({ label, view, address: a.address, used: a.used ? badge("used", "ok") : h("span", null, badge("left out", "warn"), " " + a.why) });
if (n.error && n.addresses.length) rows.push({ label, view, address: "", used: h("span", { class: "muted" }, "The last read failed (" + n.error + "): these are from " + (n.checked ? fmtTime(n.checked) : "before") + ".") });
}
mount(body,
!js.configured ? h("div", { class: "msg warn" }, "Jarl isn't configured (jarl.server in housecarl.yaml): the set keeps the addresses it last had.") : null,
js.error ? h("div", { class: "msg warn" }, "Reading Jarl failed: " + js.error + ". The set keeps the addresses it had.") : null,
h("p", null, "Holds ", h("span", { class: "mono" }, js.entries.length ? js.entries.join(", ") : "nothing"), " · within " + js.within.join(", "),
js.read ? h("span", { class: "muted" }, " · Jarl read " + fmtTime(js.read)) : null),
table([
{ label: "Name", render: (r) => r.label() },
{ label: "View", render: (r) => r.view },
{ label: "Address", render: (r) => h("span", { class: "mono" }, r.address) },
{ label: "", render: (r) => r.used },
], rows, { empty: "No names." }));
};
draw(await get(path));
await modal("Addresses of " + s.name, body, [
{ label: "Read Jarl now", onclick: async () => {
try { draw(await api("POST", path + "/refresh")); } catch (x) { toast(x.message, "err"); }
return false;
} },
{ label: "Close", value: null },
], { wide: true });
}
// entries shows and changes a dynamic set's entries.
async function entries(s) {
const body = h("div");
+1 -1
View File
@@ -104,7 +104,7 @@ export async function render(el, params, query) {
listCard("What they tried", "the logged drops' protocol and port", barList(s.blocked_ports, { empty: "No logged drops " + rangeLabel + "." })),
listCard("Devices, received", "bytes from external zones, by inside address (IPv6 per /64)", s.devices_down.length ? barList(s.devices_down, { format: bytes, href: connHref }) : devHint),
listCard("Devices, sent", "bytes to external zones", s.devices_up.length ? barList(s.devices_up, { format: bytes, href: connHref }) : devHint.cloneNode(true)),
d.quotas.length ? listCard("Quotas", "bytes counted against each rule's quota, " + rangeLabel, barList(d.quotas, { format: bytes, label: (k) => "rule #" + k.slice(1), href: () => "#/rules" })) : null,
d.quotas.length ? listCard("Quotas", "bytes counted against each rule's quota, " + rangeLabel, barList(d.quotas, { format: bytes, label: (k) => (d.hits.find((x) => x.key === k) || {}).name || "rule #" + k.slice(1), href: ruleHref, share: false })) : null,
listCard("Rules that matched nothing", "in the last 30 days (or since statistics started): candidates to remove",
unused.length ? h("div", null, h("ul", { class: "plain-list" }, unused.slice(0, 12).map((e) => h("li", null, h("a", { href: ruleHref(e.key) }, e.name || e.key)))),
unused.length > 12 ? h("div", { class: "muted small" }, "and " + (unused.length - 12) + " more: hc stats rules --unused") : null) : h("div", { class: "empty" }, "Every rule matched something."))));
+38 -1
View File
@@ -1,6 +1,6 @@
// Users and everyone's tokens (administrators).
import { h, mount, table, time, fmtTime, modal, input, field, checkbox, select, errorBox, confirmDialog, toast, badge, copyBox } from "../ui.js";
import { get, api, catalog } from "../api.js";
import { get, api, catalog, can } from "../api.js";
import { rerender } from "../app.js";
export async function render(el) {
@@ -16,6 +16,7 @@ export async function render(el) {
{ label: "Last login", render: (u) => u.last_login_at ? time(u.last_login_at) : "never" },
{ label: "Comment", render: (u) => u.comment },
{ label: "", tdClass: "actions-cell", render: (u) => [
h("button", { class: "small reveal", onclick: () => tokenDialog(cat, u) }, "New token"), " ",
h("button", { class: "small reveal", onclick: () => userDialog(u, cat) }, "Edit"), " ",
h("button", { class: "small reveal", onclick: () => del(u) }, "Delete")] },
], users)),
@@ -32,6 +33,42 @@ export async function render(el) {
], tokens, { empty: "No tokens." })));
}
// tokenDialog makes an API token: the viewer's own, or (administrators)
// another user's. It shows the secret once.
export async function tokenDialog(cat, user) {
const name = input({ placeholder: "laptop, backup script…" });
// Only roles within what the user may do: a token can never do more
// than its user, so offering more would only mislead.
const may = (res, write) => {
if (!user) return can(res, write);
if (user.admin) return true;
const gs = user.grant_strings || [];
return gs.includes(res + ":write") || !write && gs.includes(res + ":read");
};
const roles = Object.keys(cat.roles).filter((r) => cat.roles[r].every((g) => {
const [res, acc] = g.split(":");
return may(res, acc === "write");
}));
const role = select([["", user ? "everything " + user.username + " may do" : "everything you may do"], ...roles.map((r) => [r, r + ": " + cat.roles[r].join(" ")])], "");
const expires = select([["", "never"], ["30d", "30 days"], ["90d", "90 days"], ["365d", "a year"]], "90d");
const err = h("div");
await modal(user ? "New API token for " + user.username : "New API token", h("div", null, h("div", { class: "form-grid" }, field("Name", name), field("May do", role), field("Expires", expires)), err), [
{ label: "Cancel", value: null },
{ label: "Create", kind: "primary", onclick: async () => {
mount(err);
try {
const t = await api("POST", user ? "/users/" + user.id + "/tokens" : "/me/tokens", { body: { name: name.value.trim(), role: role.value || undefined, expires_in: expires.value || undefined } });
await modal("Token created", h("div", null, h("p", null, "Copy it now: it is not shown again." + (user ? " Give it to " + user.username + "; 'hc config set token' sets it in the CLI." : "")), copyBox(t.secret)));
rerender();
return true;
} catch (e) {
mount(err, errorBox(e));
return false;
}
} },
]);
}
// rotateDialog gives a token a new secret. It keeps its name, ID and
// permissions; only the programs that use it need the new secret.
export async function rotateDialog(t) {
+5
View File
@@ -379,6 +379,10 @@ pre.wrap { white-space: pre-wrap; word-break: break-all; font-size: 12px; }
.list-rows { display: flex; flex-direction: column; gap: 6px; margin-bottom: 6px; }
.list-row { display: flex; gap: 6px; align-items: center; }
.list-row input { width: 26em; max-width: 100%; }
.negatable { display: flex; gap: 8px; align-items: flex-start; flex-wrap: wrap; }
.negatable > .list-editor { flex: 1; min-width: 0; }
.negatable.negated > select { color: var(--err); border-color: var(--err); font-weight: 600; }
.negatable.negated .list-rows { border-left: 3px solid var(--err); padding-left: 8px; }
input.short { width: 7em; }
.builtins { margin-bottom: 10px; line-height: 1.6; }
.policy { margin-top: 10px; }
@@ -409,6 +413,7 @@ input.wide { width: 100%; }
.diff .add { background: var(--add); color: var(--add-text); }
.diff .del { background: var(--del); color: var(--del-text); }
.diff .hunk { background: var(--hunk); color: var(--accent); }
.diff .hunk .block { font-weight: 700; color: var(--text); }
.diff .file { font-weight: 700; padding-top: 6px; }
select.inline { width: auto; display: inline-block; }
/* The rule dialog's optional parts: "+" buttons, and ✕ to close one. */
+6 -1
View File
@@ -200,7 +200,12 @@ export function diffView(text) {
if (line.startsWith("+++") || line.startsWith("---")) cls = "file";
else if (line.startsWith("+")) cls = "add";
else if (line.startsWith("-")) cls = "del";
else if (line.startsWith("@@")) cls = "hunk";
else if (line.startsWith("@@")) {
// "@@ -a,b +c,d @@ chain forward": the chain the hunk is in.
const m = line.match(/^(@@ .*? @@)(?: (.+))?$/);
box.append(h("div", { class: "hunk" }, m ? [m[1], m[2] ? [" ", h("span", { class: "block" }, m[2])] : null] : line));
continue;
}
box.append(h("div", { class: cls }, line || " "));
}
return box;
+56 -17
View File
@@ -1,6 +1,6 @@
// Widgets shared by several pages: list editors with suggestions, the
// match editor of rules and NAT, descriptions, hit counts, durations.
import { h, input, select, field, checkbox } from "./ui.js";
import { h, input, select, field } from "./ui.js";
import { get, api } from "./api.js";
// ---- durations
@@ -65,44 +65,76 @@ export function listEditor(values, opts = {}) {
// ---- matches
// negatable puts "any of" / "any but" in front of a list editor, so a
// field says first whether it matches its entries or everything else;
// "any but" marks the list. The choice shows once the list has a row
// (empty is "any"), and an empty list is never negated. value() is the
// list, negated() the choice.
function negatable(list, not) {
const mode = select([["", "any of"], ["not", "any but"]], not ? "not" : "", { class: "inline" });
const box = h("div", { class: "negatable" }, mode, list);
const rows = () => list.querySelector(".list-rows").children.length;
const update = () => {
mode.classList.toggle("hidden", rows() === 0);
box.classList.toggle("negated", mode.value === "not" && rows() > 0);
};
mode.addEventListener("change", update);
// Add and remove act on their own click first; this runs after.
list.addEventListener("click", update);
update();
box.value = list.value;
box.negated = () => mode.value === "not" && list.value().length > 0;
return box;
}
// matchEditor edits what a rule, source NAT or redirect matches. It
// returns the form-grid fields; value() is the match's fields.
export function matchEditor(m, n, opts = {}) {
m = m || {};
const zones = (list) => listEditor(list, { suggestions: n.zones, placeholder: "zone", mono: false, addLabel: "Add zone" });
const addrs = (list) => listEditor(list, { suggestions: n.sets, placeholder: "a set, 192.168.1.0/24, 2001:db8:1::/64, 192.168.1.100-192.168.1.199", addLabel: "Add" });
const inZ = zones(m.in_zones), outZ = zones(m.out_zones);
const notIn = checkbox("any zone but these", m.not_in), notOut = checkbox("any zone but these", m.not_out);
const src = addrs(m.sources), dst = addrs(m.destinations);
const notSrc = checkbox("any source but these", m.not_sources), notDst = checkbox("any destination but these", m.not_destinations);
const inZ = negatable(zones(m.in_zones), m.not_in), outZ = negatable(zones(m.out_zones), m.not_out);
const src = negatable(addrs(m.sources), m.not_sources), dst = negatable(addrs(m.destinations), m.not_destinations);
const svc = listEditor(m.services, { suggestions: n.services, placeholder: "ssh, https, tcp/8080, udp/1000-2000, icmp/echo-request", addLabel: "Add service" });
const fam = select([["", "IPv4 and IPv6"], ["ipv4", "IPv4 only"], ["ipv6", "IPv6 only"]], m.family || "");
const inField = opts.in !== false ? field("Coming in on", inZ) : null;
const outField = opts.out !== false ? field("Going out on", outZ) : null;
const fields = [
opts.in !== false ? field("Coming in on", h("div", null, inZ, notIn)) : null,
opts.out !== false ? field("Going out on", h("div", null, outZ, notOut)) : null,
field("From", h("div", null, src, notSrc), "Sets, addresses, prefixes or ranges, IPv4 and IPv6 mixed. Empty: any."),
field("To", h("div", null, dst, notDst)),
inField,
outField,
field("From", src, "Sets, addresses, prefixes or ranges, IPv4 and IPv6 mixed. Empty: any."),
field("To", dst),
field("Services", svc, "Named services (built in, or yours) or entries such as tcp/8080. Empty: any traffic."),
field("Family", fam),
];
fields.value = () => {
const v = { sources: src.value(), not_sources: notSrc.input.checked, destinations: dst.value(), not_destinations: notDst.input.checked,
const v = { sources: src.value(), not_sources: src.negated(), destinations: dst.value(), not_destinations: dst.negated(),
services: svc.value(), family: fam.value };
if (opts.in !== false) Object.assign(v, { in_zones: inZ.value(), not_in: notIn.input.checked });
if (opts.out !== false) Object.assign(v, { out_zones: outZ.value(), not_out: notOut.input.checked });
// A hidden zone field (the chain has no such interface) matches nothing.
if (inField) Object.assign(v, inField[1].classList.contains("hidden") ? { in_zones: [], not_in: false } : { in_zones: inZ.value(), not_in: inZ.negated() });
if (outField) Object.assign(v, outField[1].classList.contains("hidden") ? { out_zones: [], not_out: false } : { out_zones: outZ.value(), not_out: outZ.negated() });
return v;
};
// zones shows the zone fields a chain can use: input and early have no
// outgoing interface, output no incoming one.
const show = (f, on) => f && f.filter(Boolean).forEach((e) => e.classList.toggle("hidden", !on));
fields.zones = (inOn, outOn) => { show(inField, inOn); show(outField, outOn); };
return fields;
}
// describe sums up a match: "in lan · from trusted · ssh, dns".
export function describe(m) {
const parts = [];
const side = (word, not, list) => { if (list && list.length) parts.push(word + (not ? " not " : " ") + list.join(", ")); };
side("in", m.not_in, m.in_zones);
side("out", m.not_out, m.out_zones);
side("from", m.not_sources, m.sources);
side("to", m.not_destinations, m.destinations);
const list = (not, xs) => (not ? "any but " : "") + xs.join(", ");
// Zones are a direction, "lan → wan"; the firewall itself is the other
// end of the input and output chains.
const ins = m.in_zones || [], outs = m.out_zones || [];
if (ins.length || outs.length) {
parts.push((m.chain === "output" ? "firewall" : ins.length ? list(m.not_in, ins) : "any") + " → " +
(m.chain === "input" ? "firewall" : outs.length ? list(m.not_out, outs) : "any"));
}
if (m.sources && m.sources.length) parts.push("from " + list(m.not_sources, m.sources));
if (m.destinations && m.destinations.length) parts.push("to " + list(m.not_destinations, m.destinations));
if (m.services && m.services.length) parts.push(m.services.join(", "));
if (m.family) parts.push(m.family + " only");
return parts.length ? parts.join(" · ") : "everything";
@@ -156,6 +188,13 @@ export function fmtBytes(n) {
return n + " bytes";
}
// pct writes used as a share of total: "25%", "<1%" for a little.
export function pct(used, total) {
if (!total || used <= 0) return "0%";
const p = used / total * 100;
return p < 1 ? "<1%" : Math.floor(p) + "%";
}
// usageBar draws used out of total as a bar, red once it is all used.
export function usageBar(used, total) {
const seg = h("div", { class: "seg leased" });
+174 -2
View File
@@ -20,7 +20,8 @@ IMAGE=${E2E_IMAGE:-housecarl-nft}
WORK=${WORK:-$(mktemp -d "${TMPDIR:-/tmp}/housecarl-e2e.XXXXXX")}
docker info >/dev/null 2>&1 || { echo "Docker isn't reachable (set DOCKER_HOST?)"; exit 2; }
(cd "$ROOT" && CGO_ENABLED=0 go build -o "$WORK/housecarld" ./cmd/housecarld && CGO_ENABLED=0 go build -o "$WORK/hc" ./cmd/hc) || exit 1
(cd "$ROOT" && CGO_ENABLED=0 go build -o "$WORK/housecarld" ./cmd/housecarld && CGO_ENABLED=0 go build -o "$WORK/hc" ./cmd/hc &&
CGO_ENABLED=0 go build -o "$WORK/fakejarl" ./scripts/fakejarl) || exit 1
# The image: Debian 13 with nftables and network tools. 'docker build'
# can't reach the network on some hosts, so it is made with run + commit.
if ! docker image inspect "$IMAGE" >/dev/null 2>&1; then
@@ -104,6 +105,7 @@ ruleset: /w/data/ruleset.nft
backup: {dir: /w/backups}
log: {level: debug}
stats: {database: /w/data/stats.db}
jarl: {server: "http://127.0.0.1:8053", token: e2e-jarl-token, refresh: 10s}
EOF
{ cat "$WORK/housecarl.yaml"; echo "shadow: {enabled: true}"; } > "$WORK/housecarl-shadow.yaml"
CONF=housecarl.yaml
@@ -322,7 +324,7 @@ send 2000000 >/dev/null 2>&1
run "used up: the device is cut off" 1 send 1000
run "the usage" 0 H quotas
contains "e2e quota.*1.0 MB each.*by hand"
contains "$CLIENT4 (e2e-client) *1\.[0-9] MB"
contains "$CLIENT4 (e2e-client) *1\.[0-9] MB (1[0-9][0-9]%)"
stop; start
run "after a restart, still used up" 1 send 1000
run "the usage after a restart" 0 H quotas
@@ -342,6 +344,15 @@ run "the device is back" 0 send 1000
run "the reset is audited" 0 H audit --action quota.
contains "quota.reset"
contains "quota.give"
# Counted in total now: what the device used carries over; per
# destination can't take it, and says so.
run "within the quota" 0 send 300000
run "to one total" 0 H rule update "$QR" --quota 1mbytes,over
not_contains "quota_restarts"
run "the total carried over" 0 H quotas
contains "e2e quota) *[0-9.]* KB ([0-9]*%)"
run "per destination starts again" 0 H rule update "$QR" --quota 1mbytes,over,per-destination
contains "quota_restarts"
run "the quota rule goes" 0 H rule delete -y "$QR"
# ---- the zone matrix, of the imported ruleset
@@ -471,6 +482,96 @@ run "the VLAN rule goes" 0 H rule delete -y "$VR"
run "the VLAN zone goes" 0 H zone delete -y iot
R ip link del enp2s0.30
# ---- ban lists: bans kept from the firewall, a host, or everything
# DNS is open to the internet on the router (tcp/53): a listener answers.
docker exec -d ${P}-router sh -c 'while [ ! -f /tmp/nodns ]; do echo dns | nc -N -l 0.0.0.0 53; done'
sleep 1
run "the router's tcp/53 from the internet" 0 connect net $WAN4 53
contains "^dns$"
run "a ban list for the firewall" 0 H banlist add router --scope firewall
# The lab's internet host shares the router's WAN /64: ban IPv6 addresses
# alone (a /64 would hold the router's own, and is refused).
run "a ban list for jellyfin" 0 H banlist add jf --scope hosts --host $JF4 --host $JF6 --group-v6 128
run "a ban list for everything" 0 H banlist add edge --scope everything --max-time 1h
run "the lists" 0 H banlist list
contains "jf *hosts *10.88.10.57, 2001:db8:7f3d:10::57 *0"
# The firewall's own DNS, for the internet host.
run "ban from the firewall's tcp/53" 0 H ban router $NET4 --tcp dns --reason "e2e"
contains "banned $NET4, tcp/53, in router"
run "the router's DNS is cut for it" 1 connect net $WAN4 53
run "the port forward isn't (scope firewall)" 0 connect net $WAN4 8096
contains "^jellyfin$"
run "the ban in the ruleset" 0 R nft list ruleset
contains "ip saddr . meta l4proto . th dport @ban_[0-9]*_v4_svc counter packets [1-9]"
run "lift it" 0 H unban router $NET4
contains "1 bans lifted"
run "DNS again" 0 connect net $WAN4 53
contains "^dns$"
# fail2ban on jellyfin, with Housecarl's action and a token for its list.
run "a user for jellyfin's fail2ban" 0 H user add f2b-jf --no-password --grant bans:write:jf
R sh -c "/w/hc token create jf --user f2b-jf > /w/f2b.token 2>/dev/null"
run "its token" 0 test -s "$WORK/f2b.token"
# f2b ACTION ip=… port=…: the action's command from contrib/, with the
# jail's values, as fail2ban runs it (on jellyfin).
f2b() {
local conf="$ROOT/contrib/fail2ban/action.d/housecarl.conf" a=$1; shift
local cmd curl
cmd=$(awk -v a="$a" '$0 ~ "^"a" *=" {sub(/^[^=]*= */,""); print; on=1; next} on && /^[ \t]/ {sub(/^[ \t]+/,""); print; next} {on=0}' "$conf" | tr '\n' ' ')
curl=$(sed -n 's/^_curl *= *//p' "$conf")
cmd=${cmd//<_curl>/$curl}
local kv
for kv in "$@" "list=jf" "url=http://$LAN4:8070" "tokenfile=/w/f2b.token" "curlopts=" "protocol=tcp" "bantime=600" "name=jellyfin"; do
cmd=${cmd//<${kv%%=*}>/${kv#*=}}
done
docker exec ${P}-jf sh -c "$cmd"
}
run "fail2ban bans the internet host's https" 0 f2b actionban ip=$NET4 port=https
run "the port forward to jellyfin is cut for it" 1 connect net $WAN4 8096
run "the router's DNS isn't (scope hosts)" 0 connect net $WAN4 53
run "fail2ban bans its IPv6 address" 0 f2b actionban ip=$NET6 port=https
run "IPv6 to jellyfin's 443 is cut" 1 connect net $JF6 443
run "the bans" 0 H bans jf
contains "$NET4 *tcp/443 .*f2b-jf *fail2ban/jellyfin"
contains "$NET6 *tcp/443"
run "never the inside" 22 f2b actionban ip=$CLIENT4 port=https
run "refused, audited" 0 H audit --action ban.add
contains "failure.*inside"
run "fail2ban unbans" 0 f2b actionunban ip=$NET4
run "the port forward works again" 0 connect net $WAN4 8096
contains "^jellyfin$"
run "unbanning what Housecarl already let go is no error" 0 f2b actionunban ip=$NET4
run "and the IPv6 ban" 0 f2b actionunban ip=$NET6
run "jellyfin by IPv6 again" 0 connect net $JF6 443
contains "jellyfin6"
run "the token can't ban elsewhere" 22 f2b actionban ip=$NET4 port=https list=router
# Everything, cutting a connection that is open, and kept over a restart.
R touch /tmp/nodns
connect net $WAN4 53 >/dev/null 2>&1
docker exec -d ${P}-router sh -c 'nc -l 0.0.0.0 53 > /w/held.out'
sleep 1
docker exec -d ${P}-net sh -c "sleep 30 | nc $WAN4 53"
sleep 1
run "an open connection from the internet host" 0 H connections --search "$NET4 :53"
contains "$NET4:[0-9]* *$WAN4:53"
run "ban it from everything" 0 H ban edge $NET4 --time 1d
contains "banned $NET4, every port, in edge (ends in 1h)"
contains "[1-9][0-9]* connections closed"
run "its connection is gone" 0 H connections --search "$NET4 :53"
not_contains ":53"
run "nothing gets through for it" 1 connect net $WAN4 8096
stop
run "restart housecarld" 0 start
run "the ban is back after the restart" 0 R nft list ruleset
contains "$NET4 timeout [0-9hms]* expires"
run "still nothing for it" 1 connect net $WAN4 8096
run "lift it" 0 H unban edge $NET4
run "the ban is out of the ruleset" 0 R nft list ruleset
not_contains "$NET4 timeout"
run "the port forward again" 0 connect net $WAN4 8096
contains "^jellyfin$"
run "the lists go" 0 sh -c "docker exec ${P}-router /w/hc banlist delete -y router && docker exec ${P}-router /w/hc banlist delete -y jf && docker exec ${P}-router /w/hc banlist delete -y edge"
# ---- renaming a zone others refer to
run "rename the imported dhcp zone" 0 H zone update dhcp --name lan
@@ -480,6 +581,77 @@ not_contains "dhcp "
run "the ruleset is unchanged by it" 0 R nft list chain inet housecarl input
contains 'iifname "enp2s0" udp dport 67'
# ---- a set from Jarl: names' addresses, kept up to date
# A fake Jarl on the router: example.com in views internal and external.
# jf's AAAA is jellyfin's; ssh.example.com is a CNAME for it; the internet
# host's address is outside the LAN.
jarl_data() { # JF_AAAA
cat > "$WORK/jarl.json" <<EOJ
{"zones": [{"id": 1, "name": "example.com.", "view": "internal", "type": "primary"},
{"id": 2, "name": "example.com.", "view": "external", "type": "primary"}],
"records": {"1": [{"name": "jf.example.com.", "type": "A", "data": "$JF4"},
{"name": "jf.example.com.", "type": "AAAA", "data": "$1"},
{"name": "ssh.example.com.", "type": "CNAME", "data": "jf.example.com."},
{"name": "net.example.com.", "type": "AAAA", "data": "$NET6"}],
"2": [{"name": "ssh.example.com.", "type": "AAAA", "data": "$NET6"}]}}
EOJ
}
jarl_data $JF6
docker exec -d ${P}-router sh -c '/w/fakejarl -listen 127.0.0.1:8053 -data /w/jarl.json -token e2e-jarl-token >> /w/fakejarl.log 2>&1'
sleep 1
run "IPv6 ssh to jellyfin is dropped" 1 connect net $JF6 22
run "a set from Jarl" 0 H set add jfssh --jarl ssh.example.com --within lan
run "a name in two views needs its view" 0 H set jarl jfssh --refresh
contains "in Jarl's views external, internal: name the view"
run "name the views" 0 H set update jfssh --jarl ssh.example.com@internal --jarl net.example.com@internal
JR=$(H -o id rule add forward --in wan --to jfssh --service ssh --first --comment "e2e jarl")
run "the set's addresses, CNAME followed" 0 H set jarl jfssh --refresh
contains "ssh.example.com → jf.example.com *internal *$JF4 *yes"
contains "ssh.example.com → jf.example.com *internal *$JF6 *yes"
contains "net.example.com *internal *$NET6 *no: not within lan"
run "in the ruleset" 0 R nft list set inet housecarl jfssh_v6
contains "$JF6"
not_contains "$NET6"
run "the address outside is warned of" 0 H status
contains "jarl_outside"
run "IPv6 ssh to jellyfin through the Jarl set" 0 connect net $JF6 22
contains "^ssh$"
# The record changes in Jarl: within the refresh, the set follows.
jarl_data 2001:db8:7f3d:10::99
sleep 12
run "the new address, without a change in Housecarl" 0 R nft list set inet housecarl jfssh_v6
contains "2001:db8:7f3d:10::99"
not_contains "$JF6"
run "ssh to the old address is dropped again" 1 connect net $JF6 22
run "no history entry for it" 0 H history list
contains "update set jfssh"
not_contains "10::99"
run "an audit record" 0 H audit --action set.jarl
contains "added 2001:db8:7f3d:10::99"
# Jarl goes away: the set keeps what it had, over a restart too.
jarl_data $JF6
H set jarl jfssh --refresh >/dev/null
R sh -c 'kill $(pidof fakejarl)'
run "Jarl down: the set keeps its addresses" 0 H set jarl jfssh --refresh
contains "Reading Jarl failed"
contains "$JF6 *yes"
run "still through" 0 connect net $JF6 22
contains "^ssh$"
run "warned" 0 H status
contains "jarl_unreachable"
stop
run "restart housecarld, Jarl still down" 0 start
run "the set is back from the database" 0 R nft list set inet housecarl jfssh_v6
contains "$JF6"
run "the boot ruleset has its addresses" 0 R grep -q "$JF6" /w/data/ruleset.nft
run "still through after the restart" 0 connect net $JF6 22
contains "^ssh$"
run "the Jarl rule goes" 0 H rule delete -y "$JR"
run "the Jarl set goes" 0 H set delete -y jfssh
run "no Jarl warning left" 0 H status
not_contains "jarl_"
# ---- restart and boot
stop
+63
View File
@@ -0,0 +1,63 @@
// Command fakejarl answers the part of Jarl's API that Housecarl's Jarl
// sets read (GET /zones, GET /zones/{id}/records), from a JSON file read
// again on each request, so the e2e test can change records:
//
// {"zones": [{"id": 1, "name": "example.com.", "view": "internal", "type": "primary"}],
// "records": {"1": [{"name": "nas.example.com.", "type": "A", "data": "10.0.0.5"}]}}
//
// Usage: fakejarl -listen 127.0.0.1:8053 -data jarl.json -token SECRET
package main
import (
"encoding/json"
"flag"
"log"
"net/http"
"os"
"strings"
)
type data struct {
Zones []map[string]any `json:"zones"`
Records map[string][]map[string]any `json:"records"`
}
func main() {
listen := flag.String("listen", "127.0.0.1:8053", "address to listen on")
file := flag.String("data", "jarl.json", "the zones and records")
token := flag.String("token", "", "the API token Housecarl must send")
flag.Parse()
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get("Authorization") != "Bearer "+*token {
w.WriteHeader(http.StatusUnauthorized)
w.Write([]byte(`{"error": {"code": "unauthorized", "message": "bad token"}}`))
return
}
b, err := os.ReadFile(*file)
var d data
if err == nil {
err = json.Unmarshal(b, &d)
}
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
w.Header().Set("Content-Type", "application/json")
p := strings.TrimPrefix(r.URL.Path, "/api/v1")
switch {
case p == "/zones":
json.NewEncoder(w).Encode(d.Zones)
case strings.HasPrefix(p, "/zones/") && strings.HasSuffix(p, "/records"):
id := strings.TrimSuffix(strings.TrimPrefix(p, "/zones/"), "/records")
recs := d.Records[id]
if recs == nil {
recs = []map[string]any{}
}
json.NewEncoder(w).Encode(recs)
default:
w.WriteHeader(http.StatusNotFound)
w.Write([]byte(`{"error": {"code": "not_found", "message": "not found"}}`))
}
})
log.Fatal(http.ListenAndServe(*listen, nil))
}
+3 -2
View File
@@ -114,7 +114,7 @@ done
files="$files dist/SHA256SUMS"
# The release (created, or found when publishing again).
release=$(call 404 "$api/releases/tags/$tag")
release=$(call 404 "$api/releases/tags/$(printf '%s' "$tag" | sed 's/+/%2B/g')")
id=$(printf '%s\n' "$release" | jq -r '.id // empty')
if [ -z "$id" ]; then
body=$(jq -n --arg tag "$tag" --arg notes "$notes" --argjson pre "$prerelease" \
@@ -127,7 +127,8 @@ for f in $files; do
name=${f#dist/}
old=$(printf '%s\n' "$release" | jq -r --arg n "$name" '.assets[]? | select(.name == $n) | .id')
[ -z "$old" ] || call "" -X DELETE "$api/releases/$id/assets/$old" >/dev/null
call "" -X POST -F "attachment=@$f" "$api/releases/$id/assets?name=$name" >/dev/null
# A + (build metadata, 0.2.0-rc.3+1a2b3c4) would be a space in a query.
call "" -X POST -F "attachment=@$f" "$api/releases/$id/assets?name=$(printf '%s' "$name" | sed 's/+/%2B/g')" >/dev/null
echo "gitea-release: attached $name"
done