Compare commits
18
Commits
No files matched your search
+130
-4
@@ -6,6 +6,47 @@ All notable changes to Housecarl are documented here. The format follows
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
|
||||
- **Zone names at import:** `hc import --zone enp2s0=lan` (repeat;
|
||||
interfaces given one name share a zone; Zone names on the Import
|
||||
page), and a note listing each zone and where its name came from.
|
||||
Defines holding a VLAN sub-interface (`enp2s0.20`) name their zone,
|
||||
and `_ifname`, `_device` and `_nic` are taken off define names too
|
||||
(docs/import.md, "How things are named").
|
||||
- Import: sets with `flags timeout` become dynamic sets; a DNAT of
|
||||
several ports becomes one port forward per port.
|
||||
|
||||
### Fixed
|
||||
|
||||
- Import: two base chains on one hook had their rules imported twice,
|
||||
and the second chain's leading built-ins were moved above the first
|
||||
chain's rules; they now follow each other by priority (read in a
|
||||
fixed order), with the stricter policy, and a note.
|
||||
- Import: a rule inside a jumped-to chain that stays raw kept only its
|
||||
own text and so applied to everything; the jump's condition now goes
|
||||
in front. A jump and a rule matching the same field are combined only
|
||||
when one's addresses lie within the other's; else the rule stays raw.
|
||||
A jump's ports, ICMP types, connection status and hop limit were
|
||||
dropped; a limit or log on the jump is no longer silently lost.
|
||||
- Import: a second match on the same field (`ip saddr A ip saddr != B`)
|
||||
replaced the first; `!=` on the hop limit and TCP flags was read as
|
||||
`==`. Such rules now stay raw, as written.
|
||||
- Import: a forward-chain accept to a forwarded host and port without
|
||||
`ct status dnat` was folded into the port forward, which only lets
|
||||
forwarded connections through; it stays a rule now. An IPv6 accept
|
||||
limited to an interface no longer becomes a pinhole open to all.
|
||||
- Import: raw rules naming a set of the old table made nftables refuse
|
||||
the ruleset; address sets are now renamed `NAME_v4`/`NAME_v6`, other
|
||||
sets written out with their elements, and what can't be is left out
|
||||
with a note. Rules naming a set that isn't imported stay raw instead
|
||||
of failing validation.
|
||||
- Import: a DNAT to an IPv6 address, or with matches a port forward
|
||||
can't hold (an interface excluded, a limit, a log), is left out with a
|
||||
note instead of being imported without them.
|
||||
|
||||
## [0.1.0] - 2026-10-19
|
||||
|
||||
The first version of Housecarl, a firewall manager for nftables, built
|
||||
the way Jarl and Steward are.
|
||||
|
||||
@@ -30,19 +71,100 @@ the way Jarl and Steward are.
|
||||
invalid packets, ICMPv6 (RFC 4890), ping from internal zones, DHCPv6
|
||||
client replies, antispoofing with your own prefixes, MSS clamping, and
|
||||
a flowtable fast path.
|
||||
- **Changesets, as in Jarl:** changes staged to be reviewed and applied
|
||||
together, as one history entry. Each is checked as it is staged; the
|
||||
review shows the problems, nftables' check and the diff of the ruleset.
|
||||
The web UI collects edits as pending changes by default (pending rows,
|
||||
a bar with Review and Apply now, a Changes page), with a switch for live
|
||||
edits, and Undo in History is staged like any edit; hc stages with
|
||||
`-C ID` or `HOUSECARL_CHANGESET`, and has `hc
|
||||
changeset create/list/show/diff/apply/discard/remove`. An applied
|
||||
changeset is on probation like any change. `--dry-run` shows the
|
||||
ruleset's diff too (docs/design.md, docs/cli.md).
|
||||
- **Bandwidth limits and quotas:** rate limits in bytes (`--limit
|
||||
20mbytes/second,over,per-source`), per source or per destination;
|
||||
quotas, an allowance of traffic per rule or per address, reset daily,
|
||||
weekly, monthly on a chosen day, or by hand, kept across changes,
|
||||
restarts and reboots (`hc quotas`, `hc rule reset-quota`, usage on the
|
||||
rules page). Rules that count traffic are checked before established
|
||||
connections are accepted, so they see every packet. Per-address limits
|
||||
group IPv6 by /64 by default (`--group-v4`, `--group-v6`). Meters,
|
||||
byte rates and quotas in an imported ruleset carry over
|
||||
(docs/rules.md).
|
||||
- **Changes checked twice and applied atomically:** validation of the
|
||||
whole configuration with stable problem codes (docs/validation.md),
|
||||
then `nft -c`, then one `nft -f` transaction that replaces Housecarl's
|
||||
own table, `inet housecarl`, and nothing else. Dynamic sets' entries
|
||||
and the counters carry across.
|
||||
- **Probation:** a change made over the network rolls itself back unless
|
||||
confirmed within two minutes (`hc confirm`, a banner in the web UI);
|
||||
confirmed within two minutes. The web UI and `hc` confirm their own
|
||||
changes once they reach Housecarl again over a new connection, from the
|
||||
same address (a change that cuts its client off can't, and rolls back);
|
||||
`--no-confirm` and the "a person confirms each change" setting leave it
|
||||
to `hc confirm` or the banner;
|
||||
changes through the local socket are kept at once; a restart rolls
|
||||
back what was pending. No rule is forced to keep SSH or the web UI
|
||||
open.
|
||||
- **At boot:** the confirmed ruleset is loaded by
|
||||
`housecarl-ruleset.service` before the network comes up, without
|
||||
housecarld. An empty configuration applies nothing.
|
||||
housecarld. An empty configuration applies nothing. It fails closed:
|
||||
if the saved ruleset doesn't load, a fallback saved beside it
|
||||
(`ruleset-fallback.nft`) is, until housecarld puts the real one back:
|
||||
nothing new gets in from external zones, the inside keeps working
|
||||
(masqueraded), and the unit is marked failed with a critical message.
|
||||
`hc status`, the dashboard and `/healthz` say while the fallback is in
|
||||
force; the warning `no_external_zone` says when no zone is external,
|
||||
which would make the fallback close everything (docs/operations.md).
|
||||
- **The flowtable and interfaces that come and go:** the boot ruleset
|
||||
has no flowtable (nftables on Debian 13's kernel refuses one naming a
|
||||
device that doesn't exist yet, such as `ppp0` before PPPoE is up, and
|
||||
with it the whole file); housecarld watches the kernel's link messages
|
||||
and applies the ruleset again when a flowtable device appears or comes
|
||||
back as a new interface (PPPoE reconnecting). A guide to when the
|
||||
flowtable helps, and what it costs (docs/flowtable.md).
|
||||
- **`GET /healthz` for monitoring:** `ok`, `firewall` (applied,
|
||||
unconfigured, fallback, missing), changes on probation and the
|
||||
machine's warnings, with HTTP 503 unless the firewall is in force;
|
||||
no login needed, read at most every five seconds.
|
||||
- **Import keeps a chain's head in order:** once a would-be built-in at
|
||||
the head of a chain has to stay a rule (an invalid drop the forward
|
||||
chain needs after an exception), the built-ins after it stay rules
|
||||
too, in their place, instead of moving ahead of it: an ICMPv6 accept
|
||||
no longer jumps ahead of the invalid drop it came after.
|
||||
- **Renaming a zone, set or service** changes the rules, NAT rules and
|
||||
port forwards that name it, in the same change (one history entry, one
|
||||
undo); before, a name in use couldn't change. Dynamic sets keep their
|
||||
names (their tokens and programs go by them).
|
||||
- VLAN sub-interfaces (`enp2s0.20`) in zones and the flowtable, tested
|
||||
with tagged traffic (docs/rules.md).
|
||||
- `hc settings set --flowtable none` (and `none` for the other lists)
|
||||
empties the list.
|
||||
- **Warnings about the machine:** while `nftables.service` is enabled
|
||||
(it loads `/etc/nftables.conf` at boot, and flushes every table when
|
||||
stopped), and while the table an import came from still applies.
|
||||
- **Statistics** (Traffic → Statistics, `hc stats`, `/api/v1/stats/...`):
|
||||
traffic per interface and zone, drops by chain and when they happen,
|
||||
hits per rule and the rules that matched nothing, the sources blocked
|
||||
most and what they tried, connections against the kernel's limit, each
|
||||
quota's use, and housecarld's and the host's memory and load, sampled
|
||||
every minute into a database of their own (`stats.db`, kept 7 days per
|
||||
minute, 90 days per hour, 2 years per day). On by default; traffic per
|
||||
inside device is a setting (`--stats-per-device`) that adds counters
|
||||
to the forward chain. Two charts on the dashboard; a warning when the
|
||||
connection tracking table is nearly full (docs/stats.md).
|
||||
- Set names starting with `meter_`, `conn_`, `quota_` or `stats_` are
|
||||
refused (`set_name_reserved`): Housecarl's own sets use them, and a set
|
||||
of yours named so could clash with them.
|
||||
- **The zone matrix** (Matrix, `hc matrix`): what the rules do to new
|
||||
connections between zones and to and from the firewall, worked out
|
||||
from the rules in order, with each pair's exceptions; rules about
|
||||
networks connected to other zones are left out of a pair.
|
||||
- **Connections and tracing:** the connections the kernel tracks, with
|
||||
their NAT, and closing them so the rules check them again (Traffic →
|
||||
Connections, `hc connections`); a rule option, "every packet", for
|
||||
blocks that stop connections already open; and a packet tracer
|
||||
(Traffic → Trace, `hc trace`) that follows packets through every
|
||||
table's rules and names what accepted or dropped them.
|
||||
- **Hits and the log:** packet and byte counters for every rule, NAT rule,
|
||||
built-in and policy; packets logged by rules and policies, read from
|
||||
netlink log group 4171 (`hc log -f`, the Log page).
|
||||
@@ -52,10 +174,14 @@ the way Jarl and Steward are.
|
||||
- **Import** of an existing nftables ruleset (`hc import`): rules keep
|
||||
their order, defines name the sets, DNAT becomes port forwards, what
|
||||
can't be translated stays as raw rules, with notes; `hc takeover`
|
||||
deletes the old table, on probation (docs/import.md).
|
||||
deletes the old table, on probation (docs/import.md). DNAT maps and
|
||||
verdict maps are expanded into one port forward or rule per element.
|
||||
- **The `hc` CLI, the REST API and the web UI** (dashboard, rules per
|
||||
chain with hits and reordering, NAT, zones, sets, services, the log,
|
||||
settings, import, history with undo), with users, roles, scoped API
|
||||
settings, import, history with undo; the rule dialog keeps the less
|
||||
used parts, such as rate limits, quotas and time windows, behind "+"
|
||||
buttons; each chain's built-ins are listed, with their hits, and
|
||||
switched in a window), with users, roles, scoped API
|
||||
tokens, sessions, the audit log, and the web UI's certificate from
|
||||
Let's Encrypt through Jarl, as in Steward.
|
||||
- **Packaging:** Debian packages `housecarl` and `housecarl-cli`, a
|
||||
|
||||
@@ -21,11 +21,14 @@ table of its own.
|
||||
|
||||
* **Zones, sets and services, then rules.** Name your interfaces (`wan`,
|
||||
`lan`, `wg`), your addresses (`trusted`, IPv4 and IPv6 in one set) and
|
||||
your ports (`jellyfin`, or the built-in `ssh`, `dns`, `https`…), and
|
||||
your ports (`web`, or the built-in `ssh`, `dns`, `https`…), and
|
||||
write ordered rules with them: in, out or through the firewall, by
|
||||
zone, address, service, connection state, time of day, rate per
|
||||
source, or connection count. Anything else can be written as an
|
||||
nftables rule by hand.
|
||||
* **Bandwidth and quotas.** Byte-rate limits per device, and traffic
|
||||
allowances that reset daily, weekly or monthly (or by hand), counted
|
||||
across changes and reboots.
|
||||
* **IPv4 and IPv6 together.** One rule covers both; Housecarl writes the
|
||||
nftables rules each family needs.
|
||||
* **NAT in one object.** A port forward is the DNAT, the forward accept
|
||||
@@ -34,20 +37,28 @@ table of its own.
|
||||
* **The rules every firewall needs**, built in and each one switchable:
|
||||
loopback, established connections, invalid packets, the ICMPv6 that
|
||||
IPv6 needs, antispoofing for your prefixes, MSS clamping for PPPoE, the
|
||||
DHCPv6 client, and a flowtable fast path.
|
||||
DHCPv6 client, and a flowtable fast path
|
||||
([docs/flowtable.md](docs/flowtable.md)).
|
||||
* **Changes you can trust.** Each change is validated as a whole, checked
|
||||
by nftables itself, and applied in one transaction. A change made over
|
||||
by nftables itself, and applied in one transaction. As in Jarl, edits
|
||||
can be collected in a changeset, reviewed with the diff of the ruleset,
|
||||
and applied (or discarded) together. A change made over
|
||||
the network is on probation: unless you confirm it within two minutes,
|
||||
it rolls itself back, so a rule that cuts you off undoes itself.
|
||||
* **Your existing ruleset, imported.** `hc import` turns an nftables
|
||||
ruleset into zones, sets, services, rules and port forwards, keeping the
|
||||
rules' order, and keeps what it can't translate as raw rules
|
||||
([docs/import.md](docs/import.md)).
|
||||
* **Easy to watch.** Hits per rule, a live log of what was dropped and
|
||||
why, and the history of every change, field by field, with undo.
|
||||
* **Easy to watch.** A zone matrix of what is allowed between zones,
|
||||
statistics over time (traffic per zone and device, drops, the sources
|
||||
blocked most, connections; [docs/stats.md](docs/stats.md)), hits per rule, a live log of what was dropped and
|
||||
why, the connections the firewall tracks (and closing them), a packet
|
||||
tracer that names the rule that dropped something, in any table, and
|
||||
the history of every change, field by field, with undo.
|
||||
* **Its own table only.** Docker, Incus and libvirt keep theirs. The
|
||||
confirmed ruleset is loaded at boot, before the network comes up, even
|
||||
if housecarld doesn't start.
|
||||
if housecarld doesn't start; should it fail to load, a fallback keeps
|
||||
anything new from the internet out until housecarld puts it back.
|
||||
* **Sets other programs can fill.** A dynamic set's entries expire by
|
||||
themselves, and a token can be limited to one set: that is how
|
||||
Steward's captive portal will let a guest in.
|
||||
@@ -75,10 +86,10 @@ sudo hc takeover "inet main"
|
||||
sudo systemctl disable nftables.service
|
||||
|
||||
# ...or start from scratch: let yourself in first
|
||||
sudo hc set add admins 10.0.10.0/24
|
||||
sudo hc set add admins 192.168.1.0/24
|
||||
sudo hc rule add input --from admins --service ssh
|
||||
sudo hc zone add wan --interface ppp0 --external
|
||||
sudo hc zone add lan --interface enp2s0
|
||||
sudo hc zone add lan --interface eth1
|
||||
sudo hc rule add forward --in lan --out wan
|
||||
sudo hc snat add --out wan
|
||||
sudo hc status
|
||||
@@ -93,7 +104,8 @@ The web UI is on `http://127.0.0.1:8070/` (see `listen` in
|
||||
* [rules.md](docs/rules.md): zones, address sets, services, rules and the built-ins
|
||||
* [nat.md](docs/nat.md): port forwards, hairpin NAT, IPv6 pinholes, source NAT, redirects
|
||||
* [import.md](docs/import.md): moving from an existing nftables ruleset
|
||||
* [operations.md](docs/operations.md): probation, the boot, hits and the log, backups, troubleshooting
|
||||
* [operations.md](docs/operations.md): probation, the boot and its fallback, hits and the log, monitoring, backups, troubleshooting
|
||||
* [flowtable.md](docs/flowtable.md): the fast path for forwarded traffic, and when (not) to use it
|
||||
* [certificates.md](docs/certificates.md): HTTPS with a certificate from Let's Encrypt, through Jarl
|
||||
* [jarl.md](docs/jarl.md): connecting Housecarl to Jarl, and what its token may do
|
||||
* [cli.md](docs/cli.md): the `hc` command line client
|
||||
@@ -106,11 +118,13 @@ The web UI is on `http://127.0.0.1:8070/` (see `listen` in
|
||||
|
||||
## Status
|
||||
|
||||
Housecarl 0.1.0 is in testing (0.1.0-rc.1): zones, address sets and
|
||||
Housecarl 0.1.1 is in testing (0.1.1-rc.1): zones, address sets and
|
||||
services with ordered rules, IPv4 and IPv6, port forwards with hairpin NAT
|
||||
and IPv6 pinholes, source NAT and redirects, built-ins, probation for
|
||||
network changes, the ruleset at boot, hits and the drop log, nftables
|
||||
import, a web UI, REST API and CLI.
|
||||
network changes (confirmed by the client that made them), the ruleset
|
||||
at boot with a fail-closed fallback, hits and the drop log, nftables
|
||||
import, changesets reviewed with the ruleset's diff (as in Jarl),
|
||||
bandwidth limits and quotas, a web UI, REST API and CLI.
|
||||
|
||||
## Licence
|
||||
|
||||
|
||||
@@ -0,0 +1,377 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"github.com/spf13/cobra"
|
||||
|
||||
"housecarl/internal/api"
|
||||
"housecarl/internal/client"
|
||||
"housecarl/internal/model"
|
||||
)
|
||||
|
||||
// Changesets: configuration changes staged to be reviewed and applied
|
||||
// together. -C/--changeset stages into a changeset as given (housecarld
|
||||
// still refuses one that isn't open, or another user's unless you are an
|
||||
// administrator). One from HOUSECARL_CHANGESET is checked first: an
|
||||
// exported ID outlives its changeset (it gets applied), and IDs keep
|
||||
// growing, so a stale or mistyped one could name someone else's later. It
|
||||
// is used only when it is open and yours, and hc says on stderr which
|
||||
// changeset that is.
|
||||
|
||||
var flagChangeset int64
|
||||
|
||||
// envChangeset returns the changeset in HOUSECARL_CHANGESET (0 if unset).
|
||||
func envChangeset() (int64, error) {
|
||||
v := os.Getenv("HOUSECARL_CHANGESET")
|
||||
if v == "" {
|
||||
return 0, nil
|
||||
}
|
||||
n, err := strconv.ParseInt(v, 10, 64)
|
||||
if err != nil || n <= 0 {
|
||||
return 0, fmt.Errorf("invalid HOUSECARL_CHANGESET %q", v)
|
||||
}
|
||||
return n, nil
|
||||
}
|
||||
|
||||
const startHint = "export HOUSECARL_CHANGESET=$(hc changeset create -q)"
|
||||
|
||||
// checkEnvChangeset checks the changeset from HOUSECARL_CHANGESET before
|
||||
// it is used. With mustOwn, it must also be open and the caller's own
|
||||
// (for staging into it, applying or discarding it).
|
||||
func checkEnvChangeset(c *client.Client, id int64, mustOwn bool) (*model.Changeset, error) {
|
||||
var cs model.Changeset
|
||||
if err := c.Do(ctx(), "GET", fmt.Sprintf("/changesets/%d", id), nil, nil, &cs); err != nil {
|
||||
var ce *client.Error
|
||||
if errors.As(err, &ce) && ce.Status == 404 {
|
||||
return nil, fmt.Errorf("HOUSECARL_CHANGESET is %d, but there is no changeset %d: start one with '%s', or 'unset HOUSECARL_CHANGESET' to make changes at once", id, id, startHint)
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
if mustOwn {
|
||||
if cs.Status != model.ChangesetOpen {
|
||||
return nil, fmt.Errorf("HOUSECARL_CHANGESET is %d, but changeset %d is %s: 'unset HOUSECARL_CHANGESET' to make changes at once, or start a new one with '%s'", id, id, cs.Status, startHint)
|
||||
}
|
||||
var me api.Whoami
|
||||
if err := c.Do(ctx(), "GET", "/auth/whoami", nil, nil, &me); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
mine := me.User != nil && cs.UserID == me.User.ID && (me.User.ID != 0 || cs.Username == me.User.Username)
|
||||
if !mine {
|
||||
return nil, fmt.Errorf("HOUSECARL_CHANGESET is %d, but changeset %d is %s's (made via %s), not yours, so it isn't used from HOUSECARL_CHANGESET; give it as -C %d to use it anyway, or 'unset HOUSECARL_CHANGESET'", id, id, cs.Username, cs.Via, id)
|
||||
}
|
||||
}
|
||||
if !flagQuiet {
|
||||
desc := cs.Description
|
||||
if desc == "" {
|
||||
desc = "no description"
|
||||
}
|
||||
fmt.Fprintf(os.Stderr, "changeset %d (%s, %s) from HOUSECARL_CHANGESET\n", id, desc, plural(len(cs.Ops), "operation"))
|
||||
}
|
||||
return &cs, nil
|
||||
}
|
||||
|
||||
// setupChangeset makes the client stage configuration changes: in
|
||||
// --changeset's changeset, else HOUSECARL_CHANGESET's (checked before
|
||||
// the first change).
|
||||
func setupChangeset(c *client.Client) error {
|
||||
if flagChangeset > 0 {
|
||||
c.Changeset = flagChangeset
|
||||
return nil
|
||||
}
|
||||
n, err := envChangeset()
|
||||
if err != nil || n == 0 {
|
||||
return err
|
||||
}
|
||||
c.Changeset = n
|
||||
c.BeforeStage = func() error {
|
||||
_, err := checkEnvChangeset(c, n, true)
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// changesetArg returns the changeset a changeset subcommand works on: the
|
||||
// ID argument, else --changeset, else HOUSECARL_CHANGESET (checked).
|
||||
// fromEnv says it came from HOUSECARL_CHANGESET.
|
||||
func changesetArg(c *client.Client, args []string, mustOwn bool) (id string, fromEnv bool, err error) {
|
||||
if len(args) > 0 {
|
||||
return args[0], false, nil
|
||||
}
|
||||
if flagChangeset > 0 {
|
||||
return strconv.FormatInt(flagChangeset, 10), false, nil
|
||||
}
|
||||
n, err := envChangeset()
|
||||
if err != nil {
|
||||
return "", false, err
|
||||
}
|
||||
if n == 0 {
|
||||
return "", false, errors.New("which changeset? Give its ID, or set HOUSECARL_CHANGESET (" + startHint + ")")
|
||||
}
|
||||
if _, err := checkEnvChangeset(c, n, mustOwn); err != nil {
|
||||
return "", false, err
|
||||
}
|
||||
return strconv.FormatInt(n, 10), true, nil
|
||||
}
|
||||
|
||||
// envChangesetDone reminds that HOUSECARL_CHANGESET still names a
|
||||
// changeset that was just applied or discarded.
|
||||
func envChangesetDone(id, what string) {
|
||||
if !flagQuiet {
|
||||
fmt.Fprintf(os.Stderr, "HOUSECARL_CHANGESET still names changeset %s, which is now %s: run 'unset HOUSECARL_CHANGESET' (or export a new one) before making more changes\n", id, what)
|
||||
}
|
||||
}
|
||||
|
||||
func plural(n int, word string) string {
|
||||
if n == 1 {
|
||||
return "1 " + word
|
||||
}
|
||||
return strconv.Itoa(n) + " " + word + "s"
|
||||
}
|
||||
|
||||
// printDiff prints a unified diff, colored on a terminal.
|
||||
func printDiff(w io.Writer, diff string) {
|
||||
color := isTerminal(os.Stdout)
|
||||
for _, line := range strings.Split(strings.TrimSuffix(diff, "\n"), "\n") {
|
||||
switch {
|
||||
case !color:
|
||||
case strings.HasPrefix(line, "+++"), strings.HasPrefix(line, "---"):
|
||||
line = "\x1b[1m" + line + "\x1b[0m"
|
||||
case strings.HasPrefix(line, "+"):
|
||||
line = "\x1b[32m" + line + "\x1b[0m"
|
||||
case strings.HasPrefix(line, "-"):
|
||||
line = "\x1b[31m" + line + "\x1b[0m"
|
||||
case strings.HasPrefix(line, "@@"):
|
||||
line = "\x1b[36m" + line + "\x1b[0m"
|
||||
}
|
||||
fmt.Fprintln(w, line)
|
||||
}
|
||||
}
|
||||
|
||||
func isTerminal(f *os.File) bool {
|
||||
fi, err := f.Stat()
|
||||
return err == nil && fi.Mode()&os.ModeCharDevice != 0
|
||||
}
|
||||
|
||||
func changesetCmd() *cobra.Command {
|
||||
cmd := &cobra.Command{
|
||||
Use: "changeset",
|
||||
Aliases: []string{"changesets", "cs"},
|
||||
Short: "Stage changes to review and apply together",
|
||||
Long: `A changeset collects configuration changes to apply together, as one
|
||||
change in the history. Each change is checked as it is staged; 'hc
|
||||
changeset diff' shows what applying them would do: the problems, nftables'
|
||||
check and the ruleset's diff. Over the network an applied changeset is on
|
||||
probation like any change.
|
||||
|
||||
The web UI stages its edits in a changeset by default; hc and the API
|
||||
make changes at once unless one is named with -C ID or HOUSECARL_CHANGESET.
|
||||
Changesets staged in the web UI show up here too, and the other way round.`,
|
||||
Example: ` export HOUSECARL_CHANGESET=$(hc changeset create "guest network" -q)
|
||||
hc zone add guest -i eth1.30
|
||||
hc rule add forward --in guest --out wan
|
||||
hc changeset diff # HOUSECARL_CHANGESET's
|
||||
hc changeset apply
|
||||
unset HOUSECARL_CHANGESET`,
|
||||
}
|
||||
cmd.AddCommand(&cobra.Command{
|
||||
Use: "create [DESCRIPTION]", Short: "Start a changeset (-q prints only its ID)", Args: cobra.MaximumNArgs(1),
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
c, err := newClient()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
in := api.ChangesetInput{}
|
||||
if len(args) > 0 {
|
||||
in.Description = args[0]
|
||||
}
|
||||
var cs model.Changeset
|
||||
if err := c.Do(ctx(), "POST", "/changesets", nil, in, &cs); err != nil {
|
||||
return err
|
||||
}
|
||||
if flagQuiet || flagOutput == "id" {
|
||||
fmt.Println(cs.ID)
|
||||
return nil
|
||||
}
|
||||
fmt.Printf("created changeset %d\nstage changes with -C %d (or export HOUSECARL_CHANGESET=%d), then 'hc changeset apply'\n", cs.ID, cs.ID, cs.ID)
|
||||
return nil
|
||||
},
|
||||
})
|
||||
var status string
|
||||
var mine bool
|
||||
list := &cobra.Command{
|
||||
Use: "list", Short: "List changesets (open ones by default)", Args: cobra.NoArgs,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
c, err := newClient()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var css []*model.Changeset
|
||||
query := q("status", status)
|
||||
if mine {
|
||||
query.Set("mine", "1")
|
||||
}
|
||||
if err := c.Do(ctx(), "GET", "/changesets", query, nil, &css); err != nil {
|
||||
return err
|
||||
}
|
||||
return output(css, func(w io.Writer) {
|
||||
fmt.Fprintln(w, "ID\tSTATUS\tBY\tVIA\tOPERATIONS\tUPDATED\tDESCRIPTION")
|
||||
for _, cs := range css {
|
||||
fmt.Fprintf(w, "%d\t%s\t%s\t%s\t%d\t%s\t%s\n", cs.ID, cs.Status, cs.Username, cs.Via, len(cs.Ops), cs.UpdatedAt.Local().Format("2006-01-02 15:04"), cs.Description)
|
||||
}
|
||||
})
|
||||
},
|
||||
}
|
||||
list.Flags().StringVar(&status, "status", "open", "open, applied, discarded, or \"\" for all")
|
||||
list.Flags().BoolVar(&mine, "mine", false, "only your own")
|
||||
cmd.AddCommand(list)
|
||||
cmd.AddCommand(&cobra.Command{
|
||||
Use: "show [ID]", Short: "Show a changeset's operations (default: -C or HOUSECARL_CHANGESET)", Args: cobra.MaximumNArgs(1),
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
c, err := newClient()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
id, _, err := changesetArg(c, args, false)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var cs model.Changeset
|
||||
if err := c.Do(ctx(), "GET", "/changesets/"+id, nil, nil, &cs); err != nil {
|
||||
return err
|
||||
}
|
||||
return output(&cs, func(w io.Writer) {
|
||||
fmt.Fprintf(w, "Changeset:\t%d (%s)\n", cs.ID, cs.Status)
|
||||
fmt.Fprintf(w, "By:\t%s via %s, %s\n", cs.Username, cs.Via, cs.CreatedAt.Local().Format("2006-01-02 15:04"))
|
||||
if cs.Description != "" {
|
||||
fmt.Fprintf(w, "Description:\t%s\n", cs.Description)
|
||||
}
|
||||
if cs.HistoryID != 0 {
|
||||
fmt.Fprintf(w, "History:\t%d\n", cs.HistoryID)
|
||||
}
|
||||
fmt.Fprintln(w, "\nOP\tCHANGE")
|
||||
for _, o := range cs.Ops {
|
||||
fmt.Fprintf(w, "%d\t%s\n", o.Seq, o.Summary)
|
||||
}
|
||||
})
|
||||
},
|
||||
})
|
||||
cmd.AddCommand(&cobra.Command{
|
||||
Use: "diff [ID]", Short: "Check an open changeset and show what applying it would change (default: -C or HOUSECARL_CHANGESET)", Args: cobra.MaximumNArgs(1),
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
c, err := newClient()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
id, _, err := changesetArg(c, args, false)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var pv api.ChangesetPreview
|
||||
if err := c.Do(ctx(), "GET", "/changesets/"+id+"/preview", nil, nil, &pv); err != nil {
|
||||
return err
|
||||
}
|
||||
if flagOutput == "json" || flagOutput == "yaml" {
|
||||
return output(&pv, nil)
|
||||
}
|
||||
for _, ch := range pv.Changes {
|
||||
if ch.Kind == model.KindSettings {
|
||||
fmt.Printf("would %s settings\n", ch.Action())
|
||||
} else {
|
||||
fmt.Printf("would %s %s %s\n", ch.Action(), ch.Kind, ch.Label)
|
||||
}
|
||||
}
|
||||
printProblems(os.Stderr, pv.Warnings)
|
||||
if pv.Error != nil {
|
||||
printProblems(os.Stderr, pv.Error.Problems)
|
||||
return fmt.Errorf("changeset %s can't be applied: %s", id, pv.Error.Message)
|
||||
}
|
||||
if !pv.Valid {
|
||||
fmt.Println("nothing to change")
|
||||
return nil
|
||||
}
|
||||
fmt.Println()
|
||||
printDiff(os.Stdout, pv.Diff)
|
||||
fmt.Printf("\nvalid: nftables accepts the new ruleset; apply with 'hc changeset apply %s'\n", id)
|
||||
return nil
|
||||
},
|
||||
})
|
||||
var desc string
|
||||
apply := &cobra.Command{
|
||||
Use: "apply [ID]", Short: "Apply an open changeset (default: -C or HOUSECARL_CHANGESET)", Args: cobra.MaximumNArgs(1),
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
c, err := newClient()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
id, fromEnv, err := changesetArg(c, args, true)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var resp api.MutationResponse
|
||||
if err := c.Do(ctx(), "POST", "/changesets/"+id+"/apply", nil, api.ChangesetInput{Description: desc}, &resp); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := report(&resp); err != nil {
|
||||
return err
|
||||
}
|
||||
if fromEnv {
|
||||
envChangesetDone(id, "applied")
|
||||
}
|
||||
return nil
|
||||
},
|
||||
}
|
||||
apply.Flags().StringVar(&desc, "description", "", "the history entry's description (default: the changeset's)")
|
||||
cmd.AddCommand(apply)
|
||||
cmd.AddCommand(&cobra.Command{
|
||||
Use: "discard [ID]", Short: "Throw away an open changeset (default: -C or HOUSECARL_CHANGESET)", Args: cobra.MaximumNArgs(1),
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
c, err := newClient()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
id, fromEnv, err := changesetArg(c, args, true)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := c.Do(ctx(), "POST", "/changesets/"+id+"/discard", nil, nil, nil); err != nil {
|
||||
return err
|
||||
}
|
||||
if !flagQuiet {
|
||||
fmt.Printf("discarded changeset %s\n", id)
|
||||
}
|
||||
if fromEnv {
|
||||
envChangesetDone(id, "discarded")
|
||||
}
|
||||
return nil
|
||||
},
|
||||
})
|
||||
cmd.AddCommand(&cobra.Command{
|
||||
Use: "remove [ID] OP", Short: "Remove one operation from an open changeset (default: -C or HOUSECARL_CHANGESET)", Args: cobra.RangeArgs(1, 2),
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
c, err := newClient()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
op := args[len(args)-1]
|
||||
id, _, err := changesetArg(c, args[:len(args)-1], true)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var cs model.Changeset
|
||||
if err := c.Do(ctx(), "DELETE", "/changesets/"+id+"/ops/"+op, nil, nil, &cs); err != nil {
|
||||
return err
|
||||
}
|
||||
if !flagQuiet {
|
||||
fmt.Printf("removed operation %s; changeset %s has %s\n", op, id, plural(len(cs.Ops), "operation"))
|
||||
}
|
||||
return nil
|
||||
},
|
||||
})
|
||||
return cmd
|
||||
}
|
||||
+85
-5
@@ -125,10 +125,16 @@ func settingsCmd() *cobra.Command {
|
||||
fmt.Fprintf(w, "Antispoof IPv6:\t%s\n", list(s.AntispoofV6))
|
||||
fmt.Fprintf(w, "Flowtable:\t%s\n", list(s.FlowtableDevices))
|
||||
fmt.Fprintf(w, "Probation:\t%s\n", fmtSeconds(s.Probation()))
|
||||
fmt.Fprintf(w, "Traffic per device:\t%s\n", map[bool]string{true: "counted (statistics)", false: "not counted"}[s.StatsPerDevice])
|
||||
if s.ManualConfirm {
|
||||
fmt.Fprintln(w, "Confirmation:\tby a person ('hc confirm', the web UI's banner)")
|
||||
} else {
|
||||
fmt.Fprintln(w, "Confirmation:\tautomatic, by the client that made the change, once it reaches Housecarl again over a new connection")
|
||||
}
|
||||
})
|
||||
},
|
||||
})
|
||||
var input, forward, output, logDrops, probation string
|
||||
var input, forward, output, logDrops, probation, autoConf, perDevice string
|
||||
var builtins, own, flowtable, spoof4, spoof6 []string
|
||||
set := &cobra.Command{
|
||||
Use: "set [flags]", Short: "Change settings (only the flags given)", Args: cobra.NoArgs,
|
||||
@@ -136,9 +142,10 @@ func settingsCmd() *cobra.Command {
|
||||
turned off: loopback, established, drop_invalid, icmpv6, ping_internal,
|
||||
dhcpv6_client, antispoof, mss_clamp.
|
||||
|
||||
hc settings set --builtin mss_clamp=on --own-prefix 2a02:8013:7f3d::/48
|
||||
hc settings set --builtin mss_clamp=on --own-prefix 2001:db8::/48
|
||||
hc settings set --forward-policy drop --log-drops on
|
||||
hc settings set --flowtable wan --flowtable enp2s0`,
|
||||
hc settings set --flowtable wan --flowtable eth1
|
||||
hc settings set --stats-per-device on`,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
fl := cmd.Flags()
|
||||
p := patch{}
|
||||
@@ -169,9 +176,27 @@ dhcpv6_client, antispoof, mss_clamp.
|
||||
}
|
||||
for flag, v := range map[string][]string{"own-prefix": own, "flowtable": flowtable, "antispoof-v4": spoof4, "antispoof-v6": spoof6} {
|
||||
if fl.Changed(flag) {
|
||||
// "none" (or "") empties the list.
|
||||
if len(v) == 1 && (v[0] == "none" || v[0] == "") {
|
||||
v = []string{}
|
||||
}
|
||||
p[map[string]string{"own-prefix": "own_prefixes", "flowtable": "flowtable_devices", "antispoof-v4": "antispoof_v4", "antispoof-v6": "antispoof_v6"}[flag]] = v
|
||||
}
|
||||
}
|
||||
if fl.Changed("stats-per-device") {
|
||||
on, err := onOff("stats-per-device", perDevice)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
p["stats_per_device"] = on
|
||||
}
|
||||
if fl.Changed("auto-confirm") {
|
||||
on, err := onOff("auto-confirm", autoConf)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
p["manual_confirm"] = !on
|
||||
}
|
||||
if fl.Changed("probation") {
|
||||
n, err := parseSeconds(probation)
|
||||
if err != nil {
|
||||
@@ -193,6 +218,8 @@ dhcpv6_client, antispoof, mss_clamp.
|
||||
f.StringArrayVar(&spoof4, "antispoof-v4", nil, "IPv4 sources dropped from the internet (replaces)")
|
||||
f.StringArrayVar(&spoof6, "antispoof-v6", nil, "IPv6 sources dropped from the internet (replaces)")
|
||||
f.StringVar(&probation, "probation", "", "how long a network change has to be confirmed (2m)")
|
||||
f.StringVar(&perDevice, "stats-per-device", "", "count traffic per inside device, for the statistics' top devices: on or off")
|
||||
f.StringVar(&autoConf, "auto-confirm", "", "on: the client that made a change confirms it once it reaches Housecarl again over a new connection; off: a person confirms")
|
||||
cmd.AddCommand(set)
|
||||
return cmd
|
||||
}
|
||||
@@ -252,6 +279,41 @@ func countersCmd() *cobra.Command {
|
||||
}
|
||||
}
|
||||
|
||||
func quotasCmd() *cobra.Command {
|
||||
return &cobra.Command{
|
||||
Use: "quotas", Aliases: []string{"quota"}, Short: "What the rules' quotas used, and when they reset", Args: cobra.NoArgs,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
qs, err := get[[]api.QuotaInfo]("/quotas", nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return output(qs, func(w io.Writer) {
|
||||
fmt.Fprintln(w, "RULE\tUSED\tOF\tRESETS\tNEXT RESET")
|
||||
for _, q := range qs {
|
||||
resets := q.Quota.Reset
|
||||
if resets == "" {
|
||||
resets = "by hand"
|
||||
} else if resets == "monthly" && q.Quota.Day > 1 {
|
||||
resets = fmt.Sprintf("monthly (%d)", q.Quota.Day)
|
||||
}
|
||||
next := "-"
|
||||
if q.NextReset != nil {
|
||||
next = q.NextReset.Local().Format("2006-01-02 15:04")
|
||||
}
|
||||
of := humanBytes(q.Quota.Bytes)
|
||||
if q.Quota.PerSource || q.Quota.PerDestination {
|
||||
of += " each"
|
||||
}
|
||||
fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\n", q.Rule, humanBytes(q.Used), of, resets, next)
|
||||
for _, a := range q.Addresses {
|
||||
fmt.Fprintf(w, " %s\t%s\t\t\t\n", a.Address, humanBytes(a.Used))
|
||||
}
|
||||
}
|
||||
})
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func logCmd() *cobra.Command {
|
||||
var follow bool
|
||||
var limit int
|
||||
@@ -372,6 +434,7 @@ func rollbackCmd() *cobra.Command {
|
||||
func importCmd() *cobra.Command {
|
||||
var replace bool
|
||||
var table, defines, textFile string
|
||||
var zones []string
|
||||
cmd := &cobra.Command{
|
||||
Use: "import [FILE|-]",
|
||||
Short: "Import an nftables ruleset (nft -j JSON) as Housecarl's configuration",
|
||||
@@ -383,7 +446,9 @@ forwards and NAT. The input is nft's JSON; on the firewall:
|
||||
Without a FILE, hc runs 'nft -j list ruleset' and 'nft -a list ruleset'
|
||||
itself (root on the firewall); from elsewhere, give the JSON as FILE and
|
||||
the text with --text. --defines reads the ruleset file's define lines, to
|
||||
name the sets as you did (trusted_ips, wg0_net, …). -n shows the result without
|
||||
name the sets as you did (admins, lan_net, …), and the zones; --zone
|
||||
enp2s0=lan names one yourself (repeat; interfaces given one name share a
|
||||
zone). -n shows the result without
|
||||
saving; what can't be expressed becomes raw rules, listed with notes.
|
||||
Once it looks right, import without -n, check it with 'hc ruleset', and
|
||||
'hc takeover inet main' deletes the old table.`,
|
||||
@@ -411,7 +476,14 @@ Once it looks right, import without -n, check it with 'hc ruleset', and
|
||||
return err
|
||||
}
|
||||
}
|
||||
in := api.ImportInput{JSON: data, Text: string(text), Table: table, Replace: replace}
|
||||
in := api.ImportInput{JSON: data, Text: string(text), Table: table, Replace: replace, Zones: map[string]string{}}
|
||||
for _, z := range zones {
|
||||
iface, name, ok := strings.Cut(z, "=")
|
||||
if !ok || iface == "" || name == "" {
|
||||
return fmt.Errorf("--zone %q: write it as INTERFACE=NAME (enp2s0=lan)", z)
|
||||
}
|
||||
in.Zones[iface] = name
|
||||
}
|
||||
if defines != "" {
|
||||
b, err := os.ReadFile(defines)
|
||||
if err != nil {
|
||||
@@ -446,6 +518,7 @@ Once it looks right, import without -n, check it with 'hc ruleset', and
|
||||
cmd.Flags().StringVar(&table, "table", "", "import only this table (\"inet main\"; default: all but Housecarl's)")
|
||||
cmd.Flags().StringVar(&defines, "defines", "", "the ruleset file, whose defines name the sets")
|
||||
cmd.Flags().StringVar(&textFile, "text", "", "'nft -a list ruleset' output, to keep what can't be expressed as raw rules")
|
||||
cmd.Flags().StringArrayVar(&zones, "zone", nil, "name an interface's zone: INTERFACE=NAME (repeat)")
|
||||
return cmd
|
||||
}
|
||||
|
||||
@@ -475,6 +548,13 @@ come back. Don't forget the boot: disable nftables.service, or empty
|
||||
return err
|
||||
}
|
||||
fmt.Println(out["message"])
|
||||
if tok, _ := out["confirm_token"].(string); tok != "" && !flagNoConfirm {
|
||||
if err := autoConfirm(tok); err != nil {
|
||||
fmt.Printf("couldn't confirm it (%v): confirm with 'hc confirm', or the tables come back\n", err)
|
||||
} else {
|
||||
fmt.Println("confirmed: this client reached Housecarl again over a new connection")
|
||||
}
|
||||
}
|
||||
return nil
|
||||
},
|
||||
}
|
||||
|
||||
+55
-9
@@ -45,6 +45,10 @@ var (
|
||||
flagInsecure bool
|
||||
flagCACert string
|
||||
flagQuiet bool
|
||||
flagNoConfirm bool
|
||||
// lastClient is the client the command made (to confirm a change by
|
||||
// itself after it).
|
||||
lastClient *client.Client
|
||||
)
|
||||
|
||||
// DefaultSocket is housecarld's local socket.
|
||||
@@ -132,6 +136,10 @@ func newClient() (*client.Client, error) {
|
||||
return nil, err
|
||||
}
|
||||
c.DryRun = flagDryRun
|
||||
if err := setupChangeset(c); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
lastClient = c
|
||||
return c, nil
|
||||
}
|
||||
|
||||
@@ -143,11 +151,15 @@ func rootCmd() *cobra.Command {
|
||||
|
||||
Every change is checked as a whole, then by nftables, before it is put in
|
||||
force, and is recorded in the history, where 'hc history undo ID' takes it
|
||||
back. --dry-run shows what a change would do.
|
||||
back. --dry-run shows what a change would do, with the ruleset's diff.
|
||||
Changes are made at once; to review several and apply them together,
|
||||
stage them in a changeset ('hc changeset --help', -C ID or
|
||||
HOUSECARL_CHANGESET).
|
||||
|
||||
A change made over the network is on probation: confirm it with 'hc
|
||||
confirm' (within 2 minutes by default) or it is rolled back by itself, so a
|
||||
change that cuts you off undoes itself. On the firewall itself, root uses
|
||||
A change made over the network is on probation: hc confirms it by itself
|
||||
once it reaches Housecarl again over a new connection, and a change that
|
||||
cuts you off can't be confirmed, so it is rolled back by itself (within 2
|
||||
minutes by default). --no-confirm leaves it to 'hc confirm'. On the firewall itself, root uses
|
||||
housecarld's local socket: no token, no probation. From elsewhere, set the
|
||||
server and an API token once:
|
||||
|
||||
@@ -169,6 +181,8 @@ Exit codes: 0 ok, 1 error, 3 not authenticated or not allowed, 4 not found,
|
||||
pf.BoolVar(&flagInsecure, "insecure", false, "don't verify the server's TLS certificate")
|
||||
pf.StringVar(&flagCACert, "ca-cert", "", "PEM file with a CA to trust for the server's certificate")
|
||||
pf.BoolVarP(&flagQuiet, "quiet", "q", false, "print only errors (and IDs of new objects)")
|
||||
pf.BoolVar(&flagNoConfirm, "no-confirm", false, "leave a change on probation instead of confirming it once this client reaches Housecarl again")
|
||||
pf.Int64VarP(&flagChangeset, "changeset", "C", 0, "stage configuration changes in this changeset instead of applying them (env HOUSECARL_CHANGESET)")
|
||||
root.AddGroup(&cobra.Group{ID: "fw", Title: "Firewall:"}, &cobra.Group{ID: "changes", Title: "Changes:"}, &cobra.Group{ID: "admin", Title: "Administration:"})
|
||||
add := func(group string, cmds ...*cobra.Command) {
|
||||
for _, c := range cmds {
|
||||
@@ -177,8 +191,8 @@ Exit codes: 0 ok, 1 error, 3 not authenticated or not allowed, 4 not found,
|
||||
}
|
||||
}
|
||||
add("fw", statusCmd(), zoneCmd(), setCmd(), serviceCmd(), ruleCmd(), forwardCmd(), snatCmd(), redirectCmd(), settingsCmd(),
|
||||
rulesetCmd(), countersCmd(), logCmd(), interfaceCmd())
|
||||
add("changes", confirmCmd(), rollbackCmd(), historyCmd(), importCmd(), takeoverCmd())
|
||||
rulesetCmd(), countersCmd(), quotasCmd(), logCmd(), connectionsCmd(), traceCmd(), matrixCmd(), statsCmd(), interfaceCmd())
|
||||
add("changes", changesetCmd(), confirmCmd(), rollbackCmd(), historyCmd(), importCmd(), takeoverCmd())
|
||||
add("admin", whoamiCmd(), userCmd(), tokenCmd(), passwordCmd(), certCmd(), auditCmd(), configCmd())
|
||||
return root
|
||||
}
|
||||
@@ -237,8 +251,6 @@ func outputIDs(v any) error {
|
||||
if m, ok := it.(map[string]any); ok {
|
||||
if id, ok := m["id"]; ok {
|
||||
fmt.Println(id)
|
||||
} else if ip, ok := m["ip"]; ok {
|
||||
fmt.Println(ip)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -294,6 +306,19 @@ func exitCode(err error) int {
|
||||
return 1
|
||||
}
|
||||
|
||||
// autoConfirm confirms a change with its token: the request goes over a
|
||||
// new connection (the server closed the change's), so it gets through
|
||||
// only if the new ruleset still lets this client reach Housecarl.
|
||||
func autoConfirm(token string) error {
|
||||
if lastClient == nil {
|
||||
return errors.New("no client")
|
||||
}
|
||||
c, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
defer cancel()
|
||||
var out map[string]int
|
||||
return lastClient.Do(c, "POST", "/confirm", nil, api.ConfirmInput{Token: token}, &out)
|
||||
}
|
||||
|
||||
// report prints the outcome of a change.
|
||||
func report(resp *api.MutationResponse) error {
|
||||
switch flagOutput {
|
||||
@@ -310,6 +335,9 @@ func report(resp *api.MutationResponse) error {
|
||||
if resp.DryRun {
|
||||
return "would " + c.Action()
|
||||
}
|
||||
if resp.Staged {
|
||||
return "staged: " + c.Action()
|
||||
}
|
||||
return c.Action() + "d"
|
||||
}
|
||||
if !flagQuiet || resp.DryRun {
|
||||
@@ -322,8 +350,26 @@ func report(resp *api.MutationResponse) error {
|
||||
}
|
||||
}
|
||||
switch {
|
||||
case resp.Staged && resp.Changeset != nil:
|
||||
if !flagQuiet {
|
||||
cs := resp.Changeset
|
||||
fmt.Fprintf(w, "staged in changeset %d as operation %d (%s now); review with 'hc changeset diff %d', apply with 'hc changeset apply %d'\n",
|
||||
cs.ID, cs.Ops[len(cs.Ops)-1].Seq, plural(len(cs.Ops), "operation"), cs.ID, cs.ID)
|
||||
}
|
||||
case resp.DryRun:
|
||||
if resp.Diff != "" && !flagQuiet {
|
||||
fmt.Fprintln(w)
|
||||
printDiff(w, resp.Diff)
|
||||
fmt.Fprintln(w)
|
||||
}
|
||||
fmt.Fprintln(w, "dry run: nothing was changed")
|
||||
case resp.Applied && resp.ConfirmBy != nil && resp.ConfirmToken != "" && !flagNoConfirm:
|
||||
if err := autoConfirm(resp.ConfirmToken); err != nil {
|
||||
fmt.Fprintf(w, "applied on probation (history %d), but couldn't confirm it: %v\nIt is rolled back within %s unless confirmed ('hc confirm')\n",
|
||||
resp.HistoryID, err, time.Until(*resp.ConfirmBy).Round(time.Second))
|
||||
} else if !flagQuiet {
|
||||
fmt.Fprintf(w, "applied and confirmed (history %d): this client reached Housecarl again over a new connection\n", resp.HistoryID)
|
||||
}
|
||||
case resp.Applied && resp.ConfirmBy != nil:
|
||||
fmt.Fprintf(w, "applied on probation (history %d): confirm with 'hc confirm' within %s, or it is rolled back\n",
|
||||
resp.HistoryID, time.Until(*resp.ConfirmBy).Round(time.Second))
|
||||
@@ -332,7 +378,7 @@ func report(resp *api.MutationResponse) error {
|
||||
case !resp.Applied && len(resp.Changes) == 0 && !flagQuiet:
|
||||
fmt.Fprintln(w, "nothing to change")
|
||||
}
|
||||
if flagQuiet && len(resp.Object) > 0 && resp.Applied {
|
||||
if flagQuiet && len(resp.Object) > 0 && (resp.Applied || resp.Staged) {
|
||||
outputIDs(resp.Object)
|
||||
}
|
||||
printProblems(os.Stderr, resp.Warnings)
|
||||
|
||||
+3
-3
@@ -144,7 +144,7 @@ many: --interface 'incusbr*'. --external marks a zone facing the internet:
|
||||
antispoofing, MSS clamping and port forwards apply there.
|
||||
|
||||
hc zone add wan --interface wan --external
|
||||
hc zone add lan --interface enp2s0`,
|
||||
hc zone add lan --interface eth1`,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
return change("POST", "/zones", model.Zone{Name: args[0], Interfaces: ifaces, External: external, Comment: comment})
|
||||
},
|
||||
@@ -267,7 +267,7 @@ func setCmd() *cobra.Command {
|
||||
Use: "add NAME [ENTRY...]", Short: "Add a set", Args: cobra.MinimumNArgs(1),
|
||||
Long: `Adds a set of addresses, prefixes and ranges, IPv4 and IPv6 mixed:
|
||||
|
||||
hc set add trusted 10.0.10.0/24 10.20.30.0/24 2a02:8013:7f3d:10::/64
|
||||
hc set add trusted 192.168.1.0/24 192.168.2.0/24 2001:db8:1::/64
|
||||
hc set add guests --dynamic --timeout 24h
|
||||
|
||||
A dynamic set takes entries at run time, each with its own timeout ('hc
|
||||
@@ -425,7 +425,7 @@ func serviceCmd() *cobra.Command {
|
||||
Use: "add NAME ENTRY...", Short: "Add a service", Args: cobra.MinimumNArgs(2),
|
||||
Long: `Adds a service: ports of a protocol, and ICMP types.
|
||||
|
||||
hc service add jellyfin tcp/8096 tcp/8920
|
||||
hc service add web tcp/8080 tcp/8443
|
||||
hc service add games udp/27015-27030 tcp/27015
|
||||
hc service add pmtu icmp/destination-unreachable icmpv6/packet-too-big`,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
|
||||
+175
-20
@@ -3,6 +3,7 @@ package main
|
||||
import (
|
||||
"fmt"
|
||||
"io"
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
@@ -96,9 +97,9 @@ type ruleFlags struct {
|
||||
matchFlags
|
||||
action, rejectWith, comment, logPrefix, raw string
|
||||
states, statuses, days []string
|
||||
hours, limit string
|
||||
connLimit int
|
||||
log, disabled bool
|
||||
hours, limit, quota string
|
||||
connLimit, groupV4, groupV6 int
|
||||
log, disabled, everyPacket bool
|
||||
before, after int64
|
||||
first bool
|
||||
}
|
||||
@@ -111,13 +112,37 @@ func (r *ruleFlags) register(f *pflag.FlagSet) {
|
||||
f.StringSliceVar(&r.statuses, "status", nil, "connection status: dnat, snat")
|
||||
f.StringSliceVar(&r.days, "days", nil, "days it applies: Monday,Tuesday,…")
|
||||
f.StringVar(&r.hours, "hours", "", "hours it applies: 08:00-17:30 (22:00-06:00 passes midnight)")
|
||||
f.StringVar(&r.limit, "limit", "", "a rate: 10/minute, with ,burst=N ,over (beyond the rate) ,per-source")
|
||||
f.StringVar(&r.limit, "limit", "", "a rate: 10/minute, or bytes: 1mbytes/second, 500kbytes/minute; with ,burst=N ,over (beyond the rate) ,per-source or ,per-destination")
|
||||
f.IntVar(&r.connLimit, "conn-limit", 0, "match a source with more than this many connections")
|
||||
f.StringVar(&r.quota, "quota", "", "an allowance: 5000mbytes or 5gbytes, with ,over (match once used up) ,per-source or ,per-destination, and ,daily ,weekly or ,monthly=DAY to reset")
|
||||
f.IntVar(&r.groupV4, "group-v4", 0, "per-source/destination limits and quotas count IPv4 addresses by this prefix (default 32: each address)")
|
||||
f.IntVar(&r.groupV6, "group-v6", 0, "…and IPv6 addresses by this prefix (default 64: one device or network)")
|
||||
f.BoolVar(&r.log, "log", false, "log matching packets ('hc log')")
|
||||
f.StringVar(&r.logPrefix, "log-prefix", "", "the log line's prefix")
|
||||
f.StringVar(&r.raw, "raw", "", "an nftables rule written by hand instead of the match flags")
|
||||
f.StringVar(&r.comment, "comment", "", "a comment")
|
||||
f.BoolVar(&r.disabled, "disabled", false, "keep the rule but don't use it")
|
||||
f.BoolVar(&r.everyPacket, "every-packet", false, "check every packet, so a drop or reject also stops connections already open")
|
||||
}
|
||||
|
||||
var amountRe = regexp.MustCompile(`^(\d+)\s*([a-zA-Z]*)$`)
|
||||
|
||||
// byteUnit reads a unit: "" (packets, for rates), bytes, kbytes, mbytes,
|
||||
// gbytes, or b, kb, mb, gb in any case.
|
||||
func byteUnit(u string) (string, bool) {
|
||||
switch strings.ToLower(u) {
|
||||
case "", "packets", "packet", "p":
|
||||
return "", true
|
||||
case "bytes", "byte", "b":
|
||||
return "bytes", true
|
||||
case "kbytes", "kb", "k":
|
||||
return "kbytes", true
|
||||
case "mbytes", "mb", "m":
|
||||
return "mbytes", true
|
||||
case "gbytes", "gb", "g":
|
||||
return "gbytes", true
|
||||
}
|
||||
return "", false
|
||||
}
|
||||
|
||||
func parseLimit(s string) (*model.Limit, error) {
|
||||
@@ -126,32 +151,101 @@ func parseLimit(s string) (*model.Limit, error) {
|
||||
}
|
||||
parts := strings.Split(s, ",")
|
||||
rate, per, ok := strings.Cut(strings.TrimSpace(parts[0]), "/")
|
||||
n, err := strconv.Atoi(rate)
|
||||
if !ok || err != nil {
|
||||
return nil, fmt.Errorf("--limit %q: write it as 10/minute, with ,burst=N ,over ,per-source", s)
|
||||
m := amountRe.FindStringSubmatch(strings.TrimSpace(rate))
|
||||
unit, unitOK := "", false
|
||||
if m != nil {
|
||||
unit, unitOK = byteUnit(m[2])
|
||||
}
|
||||
l := &model.Limit{Rate: n, Per: strings.TrimSuffix(per, "s")}
|
||||
if !ok || m == nil || !unitOK || unit == "gbytes" {
|
||||
return nil, fmt.Errorf("--limit %q: write it as 10/minute, or as bytes 1mbytes/second (bytes, kbytes, mbytes), with ,burst=N ,over ,per-source", s)
|
||||
}
|
||||
n, _ := strconv.Atoi(m[1])
|
||||
l := &model.Limit{Rate: n, Unit: unit, Per: strings.TrimSuffix(per, "s")}
|
||||
for _, p := range parts[1:] {
|
||||
k, v, _ := strings.Cut(strings.TrimSpace(p), "=")
|
||||
switch k {
|
||||
case "burst":
|
||||
if l.Burst, err = strconv.Atoi(v); err != nil {
|
||||
return nil, fmt.Errorf("--limit: burst=%q isn't a number", v)
|
||||
b := amountRe.FindStringSubmatch(v)
|
||||
bu, buOK := "", false
|
||||
if b != nil {
|
||||
bu, buOK = byteUnit(b[2])
|
||||
}
|
||||
if b == nil || !buOK || (b[2] != "" && bu != unit) {
|
||||
return nil, fmt.Errorf("--limit: burst=%q: a number, in the rate's unit", v)
|
||||
}
|
||||
l.Burst, _ = strconv.Atoi(b[1])
|
||||
case "over":
|
||||
l.Over = true
|
||||
case "per-source":
|
||||
l.PerSource = true
|
||||
case "per-destination":
|
||||
l.PerDestination = true
|
||||
default:
|
||||
return nil, fmt.Errorf("--limit: unknown %q (burst=N, over, per-source)", k)
|
||||
return nil, fmt.Errorf("--limit: unknown %q (burst=N, over, per-source, per-destination)", k)
|
||||
}
|
||||
}
|
||||
return l, nil
|
||||
}
|
||||
|
||||
// parseQuota reads --quota: 5000mbytes,over,per-source,monthly=15.
|
||||
func parseQuota(s string) (*model.Quota, error) {
|
||||
if s == "" {
|
||||
return nil, nil
|
||||
}
|
||||
parts := strings.Split(s, ",")
|
||||
m := amountRe.FindStringSubmatch(strings.TrimSpace(parts[0]))
|
||||
unit, ok := "", false
|
||||
if m != nil {
|
||||
unit, ok = byteUnit(m[2])
|
||||
}
|
||||
if m == nil || !ok || unit == "" {
|
||||
return nil, fmt.Errorf("--quota %q: write it as 5000mbytes or 5gbytes, with ,over ,per-source ,daily ,weekly or ,monthly=DAY", s)
|
||||
}
|
||||
n, _ := strconv.ParseInt(m[1], 10, 64)
|
||||
q := &model.Quota{Bytes: n * model.ByteUnits[unit]}
|
||||
for _, p := range parts[1:] {
|
||||
k, v, _ := strings.Cut(strings.TrimSpace(p), "=")
|
||||
switch k {
|
||||
case "over":
|
||||
q.Over = true
|
||||
case "per-source":
|
||||
q.PerSource = true
|
||||
case "per-destination":
|
||||
q.PerDestination = true
|
||||
case "daily", "weekly":
|
||||
q.Reset = k
|
||||
case "monthly":
|
||||
q.Reset = k
|
||||
if v != "" {
|
||||
d, err := strconv.Atoi(v)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("--quota: monthly=%q: the day of the month, 1 to 31", v)
|
||||
}
|
||||
q.Day = d
|
||||
}
|
||||
default:
|
||||
return nil, fmt.Errorf("--quota: unknown %q (over, per-source, per-destination, daily, weekly, monthly=DAY)", k)
|
||||
}
|
||||
}
|
||||
return q, nil
|
||||
}
|
||||
|
||||
// humanBytes writes bytes for people: "1.8 GB".
|
||||
func humanBytes(n int64) string {
|
||||
switch {
|
||||
case n >= 1<<30:
|
||||
return strconv.FormatFloat(float64(n)/(1<<30), 'f', 1, 64) + " GB"
|
||||
case n >= 1<<20:
|
||||
return strconv.FormatFloat(float64(n)/(1<<20), 'f', 1, 64) + " MB"
|
||||
case n >= 1<<10:
|
||||
return strconv.FormatFloat(float64(n)/(1<<10), 'f', 1, 64) + " KB"
|
||||
}
|
||||
return strconv.FormatInt(n, 10) + " bytes"
|
||||
}
|
||||
|
||||
func (r *ruleFlags) rule(chain string) (model.Rule, error) {
|
||||
rule := model.Rule{Chain: chain, Match: r.match(), Action: r.action, RejectWith: r.rejectWith, CtState: r.states, CtStatus: r.statuses,
|
||||
Days: r.days, ConnLimit: r.connLimit, Log: r.log, LogPrefix: r.logPrefix, Raw: r.raw, Comment: r.comment, Disabled: r.disabled}
|
||||
Days: r.days, ConnLimit: r.connLimit, EveryPacket: r.everyPacket, Log: r.log, LogPrefix: r.logPrefix, Raw: r.raw, Comment: r.comment, Disabled: r.disabled}
|
||||
if r.hours != "" {
|
||||
from, to, ok := strings.Cut(r.hours, "-")
|
||||
if !ok {
|
||||
@@ -160,7 +254,12 @@ func (r *ruleFlags) rule(chain string) (model.Rule, error) {
|
||||
rule.HourFrom, rule.HourTo = from, to
|
||||
}
|
||||
l, err := parseLimit(r.limit)
|
||||
if err != nil {
|
||||
return rule, err
|
||||
}
|
||||
rule.Limit = l
|
||||
rule.Quota, err = parseQuota(r.quota)
|
||||
rule.GroupV4, rule.GroupV6 = r.groupV4, r.groupV6
|
||||
return rule, err
|
||||
}
|
||||
|
||||
@@ -187,6 +286,9 @@ func ruleCmd() *cobra.Command {
|
||||
}
|
||||
extra := ruleExtras(r)
|
||||
action := r.Action
|
||||
if r.Raw != "" {
|
||||
action = "-" // the text's own
|
||||
}
|
||||
if r.Log {
|
||||
action += "+log"
|
||||
}
|
||||
@@ -200,7 +302,11 @@ func ruleCmd() *cobra.Command {
|
||||
}
|
||||
cmd.AddCommand(ls)
|
||||
cmd.AddCommand(showCmd("rules", "rule", func(w io.Writer, r model.Rule) {
|
||||
if r.Raw == "" {
|
||||
fmt.Fprintf(w, "ID:\t%d\nChain:\t%s\nMatch:\t%s%s\nAction:\t%s\n", r.ID, r.Chain, describe(r.Match), ruleExtras(r), r.Action)
|
||||
} else {
|
||||
fmt.Fprintf(w, "ID:\t%d\nChain:\t%s\n", r.ID, r.Chain)
|
||||
}
|
||||
if r.Raw != "" {
|
||||
fmt.Fprintf(w, "Raw:\t%s\n", r.Raw)
|
||||
}
|
||||
@@ -221,7 +327,7 @@ output (from it) or early (before connection tracking: cheap drops).
|
||||
|
||||
hc rule add input --from trusted --service ssh
|
||||
hc rule add input --in wg --service dns
|
||||
hc rule add forward --in lan --out wan --from 2a02:8013:7f3d:10::/64
|
||||
hc rule add forward --in lan --out wan --from 2001:db8:1::/64
|
||||
hc rule add input --in wan --service ssh --limit 3/minute,burst=5,over,per-source -a drop --log
|
||||
hc rule add forward --in wan -a drop --comment "nothing else gets in"
|
||||
hc rule add forward --raw 'iifname != "wan" ct state invalid ip daddr @trusted_v4 accept'`,
|
||||
@@ -277,7 +383,8 @@ output (from it) or early (before connection tracking: cheap drops).
|
||||
"action": {"action", r.Action}, "reject-with": {"reject_with", r.RejectWith}, "state": {"ct_state", r.CtState},
|
||||
"status": {"ct_status", r.CtStatus}, "days": {"days", r.Days}, "conn-limit": {"conn_limit", r.ConnLimit},
|
||||
"log": {"log", r.Log}, "log-prefix": {"log_prefix", r.LogPrefix}, "raw": {"raw", r.Raw}, "comment": {"comment", r.Comment},
|
||||
"disabled": {"disabled", r.Disabled},
|
||||
"disabled": {"disabled", r.Disabled}, "group-v4": {"group_v4", r.GroupV4}, "group-v6": {"group_v6", r.GroupV6},
|
||||
"every-packet": {"every_packet", r.EveryPacket},
|
||||
} {
|
||||
if fl.Changed(flag) {
|
||||
p[field[0].(string)] = field[1]
|
||||
@@ -289,6 +396,9 @@ output (from it) or early (before connection tracking: cheap drops).
|
||||
if fl.Changed("limit") {
|
||||
p["limit"] = r.Limit
|
||||
}
|
||||
if fl.Changed("quota") {
|
||||
p["quota"] = r.Quota
|
||||
}
|
||||
return change("PATCH", objPath("rules", args[0]), p)
|
||||
},
|
||||
}
|
||||
@@ -316,6 +426,22 @@ output (from it) or early (before connection tracking: cheap drops).
|
||||
move.Flags().BoolVar(&last, "last", false, "last in its chain")
|
||||
move.Flags().StringVar(&chain, "chain", "", "to the end of another chain")
|
||||
cmd.AddCommand(move)
|
||||
cmd.AddCommand(&cobra.Command{
|
||||
Use: "reset-quota ID", Short: "Start a rule's quota again now (every address's, for a per-address quota)", Args: cobra.ExactArgs(1),
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
c, err := newClient()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := c.Do(ctx(), "POST", objPath("rules", args[0])+"/reset-quota", nil, nil, nil); err != nil {
|
||||
return err
|
||||
}
|
||||
if !flagQuiet {
|
||||
fmt.Printf("reset the quota of rule %s\n", args[0])
|
||||
}
|
||||
return nil
|
||||
},
|
||||
})
|
||||
for _, on := range []bool{true, false} {
|
||||
verb, short := "disable", "Keep a rule but don't use it"
|
||||
if on {
|
||||
@@ -345,18 +471,47 @@ func ruleExtras(r model.Rule) string {
|
||||
x = append(x, r.HourFrom+"-"+r.HourTo)
|
||||
}
|
||||
if l := r.Limit; l != nil {
|
||||
s := fmt.Sprintf("limit %d/%s", l.Rate, l.Per)
|
||||
unit := ""
|
||||
if l.Unit != "" && l.Unit != "packets" {
|
||||
unit = " " + l.Unit
|
||||
}
|
||||
s := fmt.Sprintf("limit %d%s/%s", l.Rate, unit, l.Per)
|
||||
if l.Over {
|
||||
s = "over " + s
|
||||
}
|
||||
if l.PerSource {
|
||||
s += " per source"
|
||||
}
|
||||
if l.PerDestination {
|
||||
s += " per destination"
|
||||
}
|
||||
x = append(x, s)
|
||||
}
|
||||
if q := r.Quota; q != nil {
|
||||
s := "quota " + humanBytes(q.Bytes)
|
||||
if q.Over {
|
||||
s = "over " + s
|
||||
}
|
||||
if q.PerSource {
|
||||
s += " per source"
|
||||
}
|
||||
if q.PerDestination {
|
||||
s += " per destination"
|
||||
}
|
||||
switch {
|
||||
case q.Reset == "monthly" && q.Day > 1:
|
||||
s += fmt.Sprintf(" monthly (%d)", q.Day)
|
||||
case q.Reset != "":
|
||||
s += " " + q.Reset
|
||||
}
|
||||
x = append(x, s)
|
||||
}
|
||||
if r.ConnLimit > 0 {
|
||||
x = append(x, fmt.Sprintf("over %d connections", r.ConnLimit))
|
||||
}
|
||||
if r.EveryPacket {
|
||||
x = append(x, "every packet")
|
||||
}
|
||||
if len(x) == 0 {
|
||||
return ""
|
||||
}
|
||||
@@ -412,9 +567,9 @@ through, and with --hairpin makes the public address work from inside too.
|
||||
--to6 also opens the port to an IPv6 host (no translation there: its own
|
||||
port).
|
||||
|
||||
hc forward add 8096 --to 10.0.10.57:443 --to6 2a02:8013:7f3d:10::57 --public 217.155.8.194 --hairpin --name jellyfin
|
||||
hc forward add 7777-7780 --proto tcp,udp --to 10.0.10.76 --public 217.155.8.194 --hairpin --name terraria
|
||||
hc forward add 25 --to 10.0.10.25 --from trusted`,
|
||||
hc forward add 8443 --to 192.168.1.10:443 --to6 2001:db8:1::10 --public 203.0.113.10 --hairpin --name web
|
||||
hc forward add 27015-27030 --proto tcp,udp --to 192.168.1.20 --public 203.0.113.10 --hairpin --name game
|
||||
hc forward add 25 --to 192.168.1.25 --from trusted`,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
pf.Ports = args[0]
|
||||
if to != "" {
|
||||
@@ -528,7 +683,7 @@ traffic leaving the firewall. IPv4 only, unless --family ipv6 (NAT66).
|
||||
|
||||
hc snat add --out wan --from trusted
|
||||
hc snat add --out wg --not-out --from wg_net
|
||||
hc snat add --out wan --from fdab:abab:abab::/48 --family ipv6 --snat-to 2a02:8013:7f3d::1`,
|
||||
hc snat add --out wan --from fd00:1::/48 --family ipv6 --snat-to 2001:db8::1`,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
return change("POST", "/snats", model.SourceNAT{Match: mf.match(), To: to, Comment: comment})
|
||||
},
|
||||
@@ -606,7 +761,7 @@ func redirectCmd() *cobra.Command {
|
||||
Long: `Sends matching traffic to a port on the firewall itself, so every client
|
||||
uses its DNS server, say:
|
||||
|
||||
hc redirect add --in lan --service dns --not-to 10.0.10.5 --comment "force DNS"`,
|
||||
hc redirect add --in lan --service dns --not-to 192.168.1.5 --comment "force DNS"`,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
return change("POST", "/redirects", model.Redirect{Match: mf.match(), ToPort: toPort, Comment: comment})
|
||||
},
|
||||
|
||||
+401
@@ -0,0 +1,401 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/spf13/cobra"
|
||||
)
|
||||
|
||||
// Statistics: the figures housecarld keeps over time (docs/stats.md). The
|
||||
// types mirror internal/stats's JSON, so hc doesn't link the database.
|
||||
|
||||
type statEntry struct {
|
||||
Key string `json:"key"`
|
||||
Count int64 `json:"count"`
|
||||
Name string `json:"name,omitempty"`
|
||||
}
|
||||
|
||||
type statTraffic struct {
|
||||
Name string `json:"name"`
|
||||
Received int64 `json:"received"`
|
||||
Sent int64 `json:"sent"`
|
||||
}
|
||||
|
||||
type statSummary struct {
|
||||
Since string `json:"since"`
|
||||
From time.Time `json:"from"`
|
||||
Zones []statTraffic `json:"zones"`
|
||||
Interfaces []statTraffic `json:"interfaces"`
|
||||
Drops int64 `json:"drops"`
|
||||
DropsByChain []statEntry `json:"drops_by_chain"`
|
||||
Rules []statEntry `json:"rules"`
|
||||
PeakConnections int64 `json:"peak_connections"`
|
||||
ConntrackMax int64 `json:"conntrack_max"`
|
||||
BlockedSources []statEntry `json:"blocked_sources"`
|
||||
BlockedPorts []statEntry `json:"blocked_ports"`
|
||||
DevicesUp []statEntry `json:"devices_up"`
|
||||
DevicesDown []statEntry `json:"devices_down"`
|
||||
}
|
||||
|
||||
type statTimeline struct {
|
||||
Step string `json:"step"`
|
||||
Times []int64 `json:"times"`
|
||||
Series []struct {
|
||||
Key string `json:"key"`
|
||||
Values []int64 `json:"values"`
|
||||
} `json:"series"`
|
||||
}
|
||||
|
||||
type statStatus struct {
|
||||
Enabled bool `json:"enabled"`
|
||||
Database string `json:"database"`
|
||||
Size int64 `json:"size"`
|
||||
LastSample *time.Time `json:"last_sample"`
|
||||
SampleError string `json:"sample_error"`
|
||||
FirstData *time.Time `json:"first_data"`
|
||||
PerDevice bool `json:"per_device"`
|
||||
Retention struct {
|
||||
Minute, Hour, Day string
|
||||
} `json:"retention"`
|
||||
Latest *struct {
|
||||
Conntrack int64 `json:"conntrack"`
|
||||
ConntrackMax int64 `json:"conntrack_max"`
|
||||
RSS int64 `json:"rss"`
|
||||
CPUPermille int64 `json:"cpu_permille"`
|
||||
MemUsed int64 `json:"mem_used"`
|
||||
Load float64 `json:"load"`
|
||||
} `json:"latest"`
|
||||
}
|
||||
|
||||
func statsCmd() *cobra.Command {
|
||||
var since string
|
||||
var limit int
|
||||
cmd := &cobra.Command{
|
||||
Use: "stats", Short: "Traffic, drops, blocked sources and devices over time",
|
||||
Long: `Sums up what the firewall did over a while (--since: 90m, 24h, 7d, 4w):
|
||||
traffic per zone, packets dropped, the busiest rules, the most connections
|
||||
at once, the sources blocked most and what they tried, and (with traffic
|
||||
per device on) the devices that used the most.
|
||||
|
||||
hc stats --since 7d
|
||||
hc stats traffic --since 24h
|
||||
hc stats top devices --since 30d
|
||||
hc stats rules --unused`,
|
||||
Args: cobra.NoArgs,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
s, err := get[statSummary]("/stats/summary", q("since", since, "limit", strconv.Itoa(limit)))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return output(s, func(w io.Writer) { printSummary(w, s) })
|
||||
},
|
||||
}
|
||||
cmd.Flags().StringVar(&since, "since", "24h", "the last 90m, 24h, 7d, 4w…")
|
||||
cmd.Flags().IntVar(&limit, "limit", 5, "entries per top list")
|
||||
cmd.AddCommand(statsStatusCmd(), statsTrafficCmd(), statsTopCmd(), statsRulesCmd(), statsConnectionsCmd())
|
||||
return cmd
|
||||
}
|
||||
|
||||
func printSummary(w io.Writer, s statSummary) {
|
||||
fmt.Fprintf(w, "Since %s (%s)\n\n", s.From.Local().Format("2006-01-02 15:04"), s.Since)
|
||||
if len(s.Zones) > 0 {
|
||||
fmt.Fprintln(w, "ZONE\tRECEIVED\tSENT")
|
||||
for _, t := range s.Zones {
|
||||
fmt.Fprintf(w, "%s\t%s\t%s\n", t.Name, humanBytes(t.Received), humanBytes(t.Sent))
|
||||
}
|
||||
fmt.Fprintln(w)
|
||||
}
|
||||
var chains []string
|
||||
for _, e := range s.DropsByChain {
|
||||
chains = append(chains, fmt.Sprintf("%s %s", e.Key, humanCount(uint64(e.Count))))
|
||||
}
|
||||
drops := humanCount(uint64(s.Drops))
|
||||
if len(chains) > 0 {
|
||||
drops += " (" + strings.Join(chains, ", ") + ")"
|
||||
}
|
||||
fmt.Fprintf(w, "Dropped:\t%s packets\n", drops)
|
||||
if s.ConntrackMax > 0 {
|
||||
fmt.Fprintf(w, "Connections:\tat most %d at once (of %d)\n", s.PeakConnections, s.ConntrackMax)
|
||||
}
|
||||
list := func(title string, es []statEntry, unit func(int64) string) {
|
||||
if len(es) == 0 {
|
||||
return
|
||||
}
|
||||
fmt.Fprintf(w, "\n%s\n", title)
|
||||
for _, e := range es {
|
||||
name := e.Key
|
||||
if e.Name != "" {
|
||||
name = e.Name
|
||||
}
|
||||
fmt.Fprintf(w, " %s\t%s\n", name, unit(e.Count))
|
||||
}
|
||||
}
|
||||
packets := func(n int64) string { return humanCount(uint64(n)) + " packets" }
|
||||
list("Busiest rules:", s.Rules, packets)
|
||||
list("Blocked most (logged drops):", s.BlockedSources, packets)
|
||||
list("What they tried:", s.BlockedPorts, packets)
|
||||
list("Devices, sent:", s.DevicesUp, humanBytes)
|
||||
list("Devices, received:", s.DevicesDown, humanBytes)
|
||||
}
|
||||
|
||||
func statsStatusCmd() *cobra.Command {
|
||||
return &cobra.Command{
|
||||
Use: "status", Short: "Whether statistics are collected, since when, and the database", Args: cobra.NoArgs,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
s, err := get[statStatus]("/stats/status", nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return output(s, func(w io.Writer) {
|
||||
if !s.Enabled {
|
||||
fmt.Fprintln(w, "Statistics are off (stats.enabled in housecarl.yaml).")
|
||||
return
|
||||
}
|
||||
fmt.Fprintf(w, "Collecting:\tyes\n")
|
||||
if s.Database != "" {
|
||||
fmt.Fprintf(w, "Database:\t%s (%s)\n", s.Database, humanBytes(s.Size))
|
||||
} else {
|
||||
fmt.Fprintf(w, "Database size:\t%s\n", humanBytes(s.Size))
|
||||
}
|
||||
if s.FirstData != nil {
|
||||
fmt.Fprintf(w, "Since:\t%s\n", s.FirstData.Local().Format("2006-01-02 15:04"))
|
||||
}
|
||||
if s.LastSample != nil {
|
||||
fmt.Fprintf(w, "Last sample:\t%s\n", s.LastSample.Local().Format("15:04:05"))
|
||||
}
|
||||
if s.SampleError != "" {
|
||||
fmt.Fprintf(w, "Error:\t%s\n", s.SampleError)
|
||||
}
|
||||
fmt.Fprintf(w, "Kept:\tper minute %s, hourly %s, daily %s\n", s.Retention.Minute, s.Retention.Hour, s.Retention.Day)
|
||||
fmt.Fprintf(w, "Per device:\t%s\n", map[bool]string{true: "on", false: "off (hc settings set --stats-per-device on)"}[s.PerDevice])
|
||||
if l := s.Latest; l != nil {
|
||||
if l.ConntrackMax > 0 {
|
||||
fmt.Fprintf(w, "Connections now:\t%d of %d\n", l.Conntrack, l.ConntrackMax)
|
||||
}
|
||||
fmt.Fprintf(w, "housecarld:\t%s, CPU %.1f%%\n", humanBytes(l.RSS), float64(l.CPUPermille)/10)
|
||||
fmt.Fprintf(w, "Host:\t%s in use, load %.2f\n", humanBytes(l.MemUsed), l.Load)
|
||||
}
|
||||
})
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func statsTrafficCmd() *cobra.Command {
|
||||
var since, step string
|
||||
var ifaces bool
|
||||
cmd := &cobra.Command{
|
||||
Use: "traffic", Short: "Bytes received from and sent to each zone (or interface) over time", Args: cobra.NoArgs,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
rx, tx := "zrx", "ztx"
|
||||
if ifaces {
|
||||
rx, tx = "rx", "tx"
|
||||
}
|
||||
tl, err := get[statTimeline]("/stats/timeline", q("since", since, "step", step, "keys", rx+":*,"+tx+":*"))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return output(tl, func(w io.Writer) { printTimeline(w, tl, rx, tx) })
|
||||
},
|
||||
}
|
||||
cmd.Flags().StringVar(&since, "since", "24h", "the last 90m, 24h, 7d…")
|
||||
cmd.Flags().StringVar(&step, "step", "auto", "per 1m, 5m, 15m, 1h, 6h or 1d")
|
||||
cmd.Flags().BoolVar(&ifaces, "interfaces", false, "per interface instead of per zone")
|
||||
return cmd
|
||||
}
|
||||
|
||||
// printTimeline writes a row per bucket: each name's received and sent.
|
||||
func printTimeline(w io.Writer, tl statTimeline, rx, tx string) {
|
||||
var names []string
|
||||
vals := map[string][]int64{}
|
||||
for _, s := range tl.Series {
|
||||
vals[s.Key] = s.Values
|
||||
if n, ok := strings.CutPrefix(s.Key, rx+":"); ok {
|
||||
names = append(names, n)
|
||||
}
|
||||
}
|
||||
for _, s := range tl.Series {
|
||||
if n, ok := strings.CutPrefix(s.Key, tx+":"); ok && vals[rx+":"+n] == nil {
|
||||
names = append(names, n)
|
||||
}
|
||||
}
|
||||
if len(names) == 0 {
|
||||
fmt.Fprintln(w, "no traffic recorded yet")
|
||||
return
|
||||
}
|
||||
fmt.Fprint(w, "TIME")
|
||||
for _, n := range names {
|
||||
fmt.Fprintf(w, "\t%s ↓\t%s ↑", n, n)
|
||||
}
|
||||
fmt.Fprintln(w)
|
||||
layout := "01-02 15:04"
|
||||
if tl.Step == "1d" {
|
||||
layout = "2006-01-02"
|
||||
}
|
||||
at := func(k string, i int) int64 {
|
||||
if v := vals[k]; i < len(v) {
|
||||
return v[i]
|
||||
}
|
||||
return 0
|
||||
}
|
||||
for i, t := range tl.Times {
|
||||
fmt.Fprint(w, time.Unix(t, 0).Local().Format(layout))
|
||||
for _, n := range names {
|
||||
fmt.Fprintf(w, "\t%s\t%s", humanBytes(at(rx+":"+n, i)), humanBytes(at(tx+":"+n, i)))
|
||||
}
|
||||
fmt.Fprintln(w)
|
||||
}
|
||||
}
|
||||
|
||||
func statsTopCmd() *cobra.Command {
|
||||
var since string
|
||||
var limit int
|
||||
cmd := &cobra.Command{
|
||||
Use: "top devices|blocked|tried|quota RULE-ID", Short: "The largest: devices by traffic, sources blocked, what they tried, a quota's users",
|
||||
Long: `Top lists over the last while:
|
||||
|
||||
devices inside addresses by traffic to and from external zones (needs
|
||||
hc settings set --stats-per-device on)
|
||||
blocked sources of logged drops (the policies' drops with 'log drops'
|
||||
on, and rules that log)
|
||||
tried what those drops tried: tcp/22, udp/53, icmp/echo-request
|
||||
quota ID who used a per-address quota of rule ID`,
|
||||
Args: cobra.RangeArgs(1, 2),
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
get1 := func(kind string) ([]statEntry, error) {
|
||||
return get[[]statEntry]("/stats/top", q("kind", kind, "since", since, "limit", strconv.Itoa(limit)))
|
||||
}
|
||||
switch args[0] {
|
||||
case "devices":
|
||||
up, err := get1("device_up")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
down, err := get1("device_down")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
out := map[string][2]int64{}
|
||||
var order []string
|
||||
for _, l := range [][]statEntry{down, up} {
|
||||
for _, e := range l {
|
||||
if _, ok := out[e.Key]; !ok {
|
||||
order = append(order, e.Key)
|
||||
}
|
||||
}
|
||||
}
|
||||
for _, e := range down {
|
||||
v := out[e.Key]
|
||||
v[0] = e.Count
|
||||
out[e.Key] = v
|
||||
}
|
||||
for _, e := range up {
|
||||
v := out[e.Key]
|
||||
v[1] = e.Count
|
||||
out[e.Key] = v
|
||||
}
|
||||
return output(map[string]any{"up": up, "down": down}, func(w io.Writer) {
|
||||
fmt.Fprintln(w, "DEVICE\tRECEIVED\tSENT")
|
||||
for _, k := range order {
|
||||
fmt.Fprintf(w, "%s\t%s\t%s\n", k, humanBytes(out[k][0]), humanBytes(out[k][1]))
|
||||
}
|
||||
})
|
||||
case "blocked", "tried":
|
||||
kind := map[string]string{"blocked": "blocked_src", "tried": "blocked_port"}[args[0]]
|
||||
es, err := get1(kind)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return output(es, func(w io.Writer) {
|
||||
fmt.Fprintln(w, map[string]string{"blocked": "SOURCE", "tried": "WHAT"}[args[0]]+"\tPACKETS")
|
||||
for _, e := range es {
|
||||
fmt.Fprintf(w, "%s\t%d\n", e.Key, e.Count)
|
||||
}
|
||||
})
|
||||
case "quota":
|
||||
if len(args) != 2 {
|
||||
return fmt.Errorf("give the rule's ID: hc stats top quota 12")
|
||||
}
|
||||
es, err := get1("quota_r" + args[1])
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return output(es, func(w io.Writer) {
|
||||
fmt.Fprintln(w, "ADDRESS\tUSED")
|
||||
for _, e := range es {
|
||||
fmt.Fprintf(w, "%s\t%s\n", e.Key, humanBytes(e.Count))
|
||||
}
|
||||
})
|
||||
}
|
||||
return fmt.Errorf("unknown list %q: devices, blocked, tried or quota", args[0])
|
||||
},
|
||||
}
|
||||
cmd.Flags().StringVar(&since, "since", "24h", "the last 90m, 24h, 7d…")
|
||||
cmd.Flags().IntVar(&limit, "limit", 20, "at most this many")
|
||||
return cmd
|
||||
}
|
||||
|
||||
func statsRulesCmd() *cobra.Command {
|
||||
var since string
|
||||
var unused bool
|
||||
cmd := &cobra.Command{
|
||||
Use: "rules", Short: "Packets each rule, built-in and policy counted; --unused: rules that matched nothing", Args: cobra.NoArgs,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
es, err := get[[]statEntry]("/stats/hits", q("since", since))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if unused {
|
||||
var keep []statEntry
|
||||
for _, e := range es {
|
||||
if e.Count == 0 && strings.HasPrefix(e.Key, "r") {
|
||||
keep = append(keep, e)
|
||||
}
|
||||
}
|
||||
es = keep
|
||||
}
|
||||
return output(es, func(w io.Writer) {
|
||||
if unused && len(es) == 0 {
|
||||
fmt.Fprintf(w, "every rule matched something in the last %s\n", since)
|
||||
return
|
||||
}
|
||||
fmt.Fprintln(w, "WHAT\tPACKETS")
|
||||
for _, e := range es {
|
||||
fmt.Fprintf(w, "%s\t%d\n", orDash(e.Name), e.Count)
|
||||
}
|
||||
})
|
||||
},
|
||||
}
|
||||
cmd.Flags().StringVar(&since, "since", "30d", "the last 24h, 7d, 30d…")
|
||||
cmd.Flags().BoolVar(&unused, "unused", false, "only the rules that matched nothing")
|
||||
return cmd
|
||||
}
|
||||
|
||||
func statsConnectionsCmd() *cobra.Command {
|
||||
var since, step string
|
||||
cmd := &cobra.Command{
|
||||
Use: "connections", Short: "How many connections the firewall tracked over time, against the limit", Args: cobra.NoArgs,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
tl, err := get[statTimeline]("/stats/timeline", q("since", since, "step", step, "keys", "g:conntrack,g:conntrack_max"))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return output(tl, func(w io.Writer) {
|
||||
fmt.Fprintln(w, "TIME\tCONNECTIONS\tLIMIT")
|
||||
for i, t := range tl.Times {
|
||||
n, m := tl.Series[0].Values[i], tl.Series[1].Values[i]
|
||||
if n < 0 {
|
||||
continue // no sample
|
||||
}
|
||||
fmt.Fprintf(w, "%s\t%d\t%d\n", time.Unix(t, 0).Local().Format("01-02 15:04"), n, m)
|
||||
}
|
||||
})
|
||||
},
|
||||
}
|
||||
cmd.Flags().StringVar(&since, "since", "24h", "the last 90m, 24h, 7d…")
|
||||
cmd.Flags().StringVar(&step, "step", "auto", "per 1m, 5m, 15m, 1h, 6h or 1d")
|
||||
return cmd
|
||||
}
|
||||
@@ -0,0 +1,283 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"os/signal"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/spf13/cobra"
|
||||
|
||||
"housecarl/internal/api"
|
||||
)
|
||||
|
||||
// Traffic as it happens: connections the kernel tracks, and tracing
|
||||
// packets through the ruleset.
|
||||
|
||||
func connectionsCmd() *cobra.Command {
|
||||
var search string
|
||||
var limit int
|
||||
cmd := &cobra.Command{
|
||||
Use: "connections", Aliases: []string{"conns"}, Short: "The connections the firewall tracks, and closing them",
|
||||
Long: `Lists the connections the kernel tracks: what the "established" built-in
|
||||
lets through without checking the rules again. A rule added later doesn't
|
||||
stop them by itself; close them, and their next packet is checked by the
|
||||
rules as it stands.
|
||||
|
||||
hc connections --search 192.168.1.50
|
||||
hc connections --search ":443 tcp"
|
||||
hc connections close --search 192.168.1.50`,
|
||||
Args: cobra.NoArgs,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
cs, err := get[[]api.Connection]("/connections", q("q", search, "limit", strconv.Itoa(limit)))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return output(cs, func(w io.Writer) {
|
||||
fmt.Fprintln(w, "PROTO\tFROM\tTO\tNAT\tSTATE\tEXPIRES")
|
||||
for _, c := range cs {
|
||||
fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\t%s\n", c.Protocol, hostPort(c.Source, c.SourcePort), hostPort(c.Destination, c.DestinationPort), natText(c), c.State, fmtSeconds(c.Timeout))
|
||||
}
|
||||
})
|
||||
},
|
||||
}
|
||||
cmd.Flags().StringVar(&search, "search", "", "only those with all these words: addresses, :port, protocol, state")
|
||||
cmd.Flags().IntVar(&limit, "limit", 500, "at most this many")
|
||||
var csearch string
|
||||
var yes bool
|
||||
closeCmd := &cobra.Command{
|
||||
Use: "close --search TEXT", Short: "Close connections: the rules check them again with their next packet", Args: cobra.NoArgs,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
if strings.TrimSpace(csearch) == "" {
|
||||
return fmt.Errorf("say which: --search 192.168.1.50 (see 'hc connections --search' first)")
|
||||
}
|
||||
if !confirm("Close every connection matching "+strconvQuote(csearch)+"?", yes) {
|
||||
return fmt.Errorf("nothing closed")
|
||||
}
|
||||
c, err := newClient()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var out map[string]int
|
||||
if err := c.Do(ctx(), "POST", "/connections/close", nil, api.CloseConnections{Search: csearch}, &out); err != nil {
|
||||
return err
|
||||
}
|
||||
if !flagQuiet {
|
||||
fmt.Printf("closed %s\n", plural(out["closed"], "connection"))
|
||||
}
|
||||
return nil
|
||||
},
|
||||
}
|
||||
closeCmd.Flags().StringVar(&csearch, "search", "", "the connections to close, as for 'hc connections --search'")
|
||||
closeCmd.Flags().BoolVarP(&yes, "yes", "y", false, "don't ask")
|
||||
cmd.AddCommand(closeCmd)
|
||||
return cmd
|
||||
}
|
||||
|
||||
func strconvQuote(s string) string { return strconv.Quote(s) }
|
||||
|
||||
func hostPort(addr string, port int) string {
|
||||
if port == 0 {
|
||||
return addr
|
||||
}
|
||||
if strings.Contains(addr, ":") {
|
||||
return fmt.Sprintf("[%s]:%d", addr, port)
|
||||
}
|
||||
return fmt.Sprintf("%s:%d", addr, port)
|
||||
}
|
||||
|
||||
// natText: "to 192.168.1.10:443" (a port forward), "as 203.0.113.10"
|
||||
// (masqueraded).
|
||||
func natText(c api.Connection) string {
|
||||
var parts []string
|
||||
if c.DNATTo != "" {
|
||||
port := c.DNATPort
|
||||
if port == 0 {
|
||||
port = c.DestinationPort
|
||||
}
|
||||
parts = append(parts, "to "+hostPort(c.DNATTo, port))
|
||||
}
|
||||
if c.SNATTo != "" {
|
||||
parts = append(parts, "as "+c.SNATTo)
|
||||
}
|
||||
return strings.Join(parts, ", ")
|
||||
}
|
||||
|
||||
func traceCmd() *cobra.Command {
|
||||
var in api.TraceInput
|
||||
cmd := &cobra.Command{
|
||||
Use: "trace", Short: "Follow packets through every table's rules: what each met, and what decided",
|
||||
Long: `Traces packets from or to an address, on a port (either way), through the
|
||||
whole ruleset, Housecarl's and every other table's, for a while: each
|
||||
rule and policy they meet, and what accepted or dropped them. Ctrl-C
|
||||
stops it.
|
||||
|
||||
hc trace --address 192.168.1.50 --port 443
|
||||
hc trace --port 22 --seconds 120
|
||||
hc trace --address 2001:db8:1::/64 --protocol icmpv6`,
|
||||
Args: cobra.NoArgs,
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
c, err := newClient()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var st api.TraceStatus
|
||||
if err := c.Do(ctx(), "POST", "/trace", nil, in, &st); err != nil {
|
||||
return err
|
||||
}
|
||||
if !flagQuiet {
|
||||
fmt.Fprintf(os.Stderr, "tracing until %s (Ctrl-C stops)\n", st.Ends.Local().Format("15:04:05"))
|
||||
}
|
||||
sig, stop := signal.NotifyContext(context.Background(), os.Interrupt)
|
||||
defer stop()
|
||||
shown := 0
|
||||
for {
|
||||
select {
|
||||
case <-sig.Done():
|
||||
c.Do(ctx(), "DELETE", "/trace", nil, nil, &st)
|
||||
case <-time.After(time.Second):
|
||||
if err := c.Do(ctx(), "GET", "/trace", nil, nil, &st); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
for ; shown < len(st.Packets); shown++ {
|
||||
printPacket(os.Stdout, st.Packets[shown])
|
||||
}
|
||||
if !st.Running || sig.Err() != nil {
|
||||
break
|
||||
}
|
||||
}
|
||||
if st.Error != "" {
|
||||
return fmt.Errorf("the trace failed: %s", st.Error)
|
||||
}
|
||||
if !flagQuiet {
|
||||
fmt.Fprintf(os.Stderr, "%d packets traced\n", len(st.Packets))
|
||||
}
|
||||
return nil
|
||||
},
|
||||
}
|
||||
f := cmd.Flags()
|
||||
f.StringVar(&in.Address, "address", "", "packets from or to this address or prefix")
|
||||
f.IntVar(&in.Port, "port", 0, "packets from or to this port")
|
||||
f.StringVar(&in.Protocol, "protocol", "", "tcp, udp, sctp, icmp or icmpv6 (default any)")
|
||||
f.IntVar(&in.Seconds, "seconds", 60, "how long (5 to 600)")
|
||||
return cmd
|
||||
}
|
||||
|
||||
// printPacket writes a traced packet: its headers, each step, its fate.
|
||||
func printPacket(w io.Writer, p api.TracePacket) {
|
||||
what := p.Protocol
|
||||
if p.TCPFlags != "" {
|
||||
what += " " + p.TCPFlags
|
||||
}
|
||||
if p.ICMPType != "" {
|
||||
what += " " + p.ICMPType
|
||||
}
|
||||
where := ""
|
||||
if p.In != "" {
|
||||
where += " in " + p.In
|
||||
}
|
||||
if p.Out != "" {
|
||||
where += " out " + p.Out
|
||||
}
|
||||
fmt.Fprintf(w, "%s %s -> %s %s%s\n", p.Time.Local().Format("15:04:05.000"), hostPort(p.Source, p.SourcePort), hostPort(p.Destination, p.DestinationPort), what, where)
|
||||
for _, s := range p.Steps {
|
||||
name := s.Label
|
||||
if name == "" {
|
||||
if s.Kind == "policy" {
|
||||
name = s.Chain + " policy"
|
||||
} else {
|
||||
name = s.Rule
|
||||
}
|
||||
}
|
||||
fmt.Fprintf(w, " %s %s: %s -> %s\n", s.Table, s.Chain, name, s.Verdict)
|
||||
}
|
||||
decided := p.DecidedBy.Label
|
||||
if decided == "" {
|
||||
decided = strings.TrimSpace(p.DecidedBy.Table + " " + p.DecidedBy.Chain + " " + p.DecidedBy.Kind)
|
||||
}
|
||||
if decided == "" {
|
||||
fmt.Fprintf(w, " => %s\n", strings.ToUpper(p.Verdict))
|
||||
} else {
|
||||
fmt.Fprintf(w, " => %s by %s\n", strings.ToUpper(p.Verdict), decided)
|
||||
}
|
||||
}
|
||||
|
||||
func matrixCmd() *cobra.Command {
|
||||
return &cobra.Command{
|
||||
Use: "matrix [FROM TO]", Short: "What the rules do to new connections between zones",
|
||||
Long: `Shows, for each pair of zones (and the firewall itself), what happens to a
|
||||
new connection: allowed or blocked, decided by a rule or the policy, and
|
||||
how many rules decide for part of the traffic first (+N: exceptions,
|
||||
such as some services or addresses). With FROM and TO (a zone, or
|
||||
"firewall"), that pair's exceptions in order, and what decides the rest.
|
||||
|
||||
hc matrix
|
||||
hc matrix guest wan
|
||||
hc matrix lan firewall`,
|
||||
Args: cobra.RangeArgs(0, 2),
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
if len(args) == 1 {
|
||||
return fmt.Errorf("give both: FROM and TO")
|
||||
}
|
||||
m, err := get[api.Matrix]("/matrix", nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if len(args) == 2 {
|
||||
for _, c := range m.Cells {
|
||||
if strings.EqualFold(c.From, args[0]) && strings.EqualFold(c.To, args[1]) {
|
||||
return output(c, func(w io.Writer) { printCell(w, c) })
|
||||
}
|
||||
}
|
||||
return fmt.Errorf("no pair %s → %s (zones: %s, and firewall)", args[0], args[1], strings.Join(m.Zones, ", "))
|
||||
}
|
||||
return output(m, func(w io.Writer) { printMatrix(w, m) })
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func printMatrix(w io.Writer, m api.Matrix) {
|
||||
cols := append(append([]string{}, m.Zones...), "firewall")
|
||||
cells := map[[2]string]api.MatrixCell{}
|
||||
for _, c := range m.Cells {
|
||||
cells[[2]string{c.From, c.To}] = c
|
||||
}
|
||||
color := isTerminal(os.Stdout) && flagOutput == "table"
|
||||
fmt.Fprint(w, "FROM \\ TO")
|
||||
for _, c := range cols {
|
||||
fmt.Fprintf(w, "\t%s", c)
|
||||
}
|
||||
fmt.Fprintln(w)
|
||||
for _, from := range cols {
|
||||
fmt.Fprint(w, from)
|
||||
for _, to := range cols {
|
||||
c, ok := cells[[2]string{from, to}]
|
||||
text := "-"
|
||||
if ok {
|
||||
text = map[string]string{"allowed": "allow", "blocked": "block"}[c.Verdict]
|
||||
if n := len(c.Exceptions); n > 0 {
|
||||
text += fmt.Sprintf(" +%d", n)
|
||||
}
|
||||
if color {
|
||||
text = map[string]string{"allowed": "\x1b[32m", "blocked": "\x1b[31m"}[c.Verdict] + text + "\x1b[0m"
|
||||
}
|
||||
}
|
||||
fmt.Fprintf(w, "\t%s", text)
|
||||
}
|
||||
fmt.Fprintln(w)
|
||||
}
|
||||
fmt.Fprintln(w, "\nallow/block: new connections no exception covers; +N: rules deciding for part of the traffic first ('hc matrix FROM TO')")
|
||||
}
|
||||
|
||||
func printCell(w io.Writer, c api.MatrixCell) {
|
||||
fmt.Fprintf(w, "%s → %s (the %s chain), new connections:\n", c.From, c.To, c.Chain)
|
||||
for _, e := range c.Exceptions {
|
||||
fmt.Fprintf(w, " %s\t%s\t%s\n", e.Verdict, e.What, e.Label)
|
||||
}
|
||||
fmt.Fprintf(w, " %s\tthe rest\t%s\n", c.Verdict, c.DecidedBy)
|
||||
}
|
||||
@@ -6,6 +6,7 @@ package main
|
||||
import (
|
||||
"bufio"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
@@ -25,15 +26,19 @@ import (
|
||||
"housecarl/internal/auth"
|
||||
"housecarl/internal/certs"
|
||||
"housecarl/internal/config"
|
||||
"housecarl/internal/conntrack"
|
||||
"housecarl/internal/engine"
|
||||
"housecarl/internal/hostif"
|
||||
"housecarl/internal/jarl"
|
||||
"housecarl/internal/linkwatch"
|
||||
"housecarl/internal/model"
|
||||
"housecarl/internal/nflog"
|
||||
"housecarl/internal/nft"
|
||||
"housecarl/internal/render"
|
||||
"housecarl/internal/server"
|
||||
"housecarl/internal/stats"
|
||||
"housecarl/internal/store"
|
||||
"housecarl/internal/trace"
|
||||
"housecarl/internal/validate"
|
||||
"housecarl/internal/version"
|
||||
"housecarl/internal/web"
|
||||
@@ -175,6 +180,14 @@ func serveCmd() *cobra.Command {
|
||||
// housecarld stopped is rolled back first. An empty
|
||||
// configuration applies nothing: its default policies would
|
||||
// close the machine before any rule lets anyone in.
|
||||
// What the quotas used before housecarld (or the machine)
|
||||
// stopped comes back with the next apply.
|
||||
if v, err := a.store.GetMeta(ctx, server.QuotaUsageKey); err == nil && v != "" {
|
||||
var u nft.QuotaUsage
|
||||
if json.Unmarshal([]byte(v), &u) == nil {
|
||||
fw.RestoreQuotas(&u)
|
||||
}
|
||||
}
|
||||
if !srv.ResumeProbation(ctx) {
|
||||
if eng.State().Empty() {
|
||||
a.log.Info("the configuration is empty: no ruleset is applied until the first change (or 'hc import')")
|
||||
@@ -184,7 +197,30 @@ func serveCmd() *cobra.Command {
|
||||
a.log.Error("saving the ruleset for the next boot failed", "err", err)
|
||||
}
|
||||
}
|
||||
// A flowtable device that appears later (ppp0 once PPPoE is up,
|
||||
// or again after it reconnects) is added by applying again.
|
||||
go func() {
|
||||
err := linkwatch.Watch(ctx, func() {
|
||||
host.Forget()
|
||||
if !fw.FlowtableStale(eng.State()) {
|
||||
return
|
||||
}
|
||||
if err := eng.ApplyCurrent(ctx); err != nil {
|
||||
a.log.Error("applying the ruleset again for the flowtable's devices failed", "err", err)
|
||||
return
|
||||
}
|
||||
a.log.Info("interfaces changed: applied the ruleset again, with the flowtable's devices that exist now")
|
||||
})
|
||||
if err != nil {
|
||||
a.log.Warn("not watching the interfaces: a flowtable device that appears later is only added by the next change", "err", err)
|
||||
}
|
||||
}()
|
||||
go nflog.Listen(ctx, render.LogGroup, a.log, srv.LogPacket)
|
||||
go srv.RunQuotas(ctx)
|
||||
srv.SetConntrack(conntrack.Client{})
|
||||
statsDone := startStats(ctx, a, fw, srv)
|
||||
// A trace left behind by a crash.
|
||||
fw.DeleteTable(ctx, "inet", trace.Table)
|
||||
hup := make(chan os.Signal, 1)
|
||||
signal.Notify(hup, syscall.SIGHUP)
|
||||
go func() {
|
||||
@@ -196,12 +232,46 @@ func serveCmd() *cobra.Command {
|
||||
}()
|
||||
err = srv.Serve(ctx, func() { sdNotify("READY=1") })
|
||||
sdNotify("STOPPING=1")
|
||||
srv.SaveQuotaUsage(context.Background())
|
||||
<-statsDone
|
||||
a.log.Info("stopped")
|
||||
return err
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// startStats opens the statistics database and runs the collector until
|
||||
// ctx ends; the channel closes once it wrote its last figures.
|
||||
func startStats(ctx context.Context, a *app, fw *nft.Firewall, srv *server.Server) <-chan struct{} {
|
||||
done := make(chan struct{})
|
||||
c := a.cfg.Stats
|
||||
if !c.Enabled {
|
||||
close(done)
|
||||
return done
|
||||
}
|
||||
st, err := stats.OpenStore(c.Database)
|
||||
if err != nil {
|
||||
a.log.Error("statistics are off: opening their database failed (deleting it is safe)", "err", err)
|
||||
close(done)
|
||||
return done
|
||||
}
|
||||
opts := stats.Options{TopN: c.TopN, RetainMinute: time.Duration(c.Retention.Minute), RetainHour: time.Duration(c.Retention.Hour), RetainDay: time.Duration(c.Retention.Day)}
|
||||
// Tests sample more often than every minute.
|
||||
if v := os.Getenv("HOUSECARL_STATS_INTERVAL"); v != "" {
|
||||
opts.Interval, _ = time.ParseDuration(v)
|
||||
}
|
||||
src := stats.Sources{Table: fw.Table}
|
||||
srv.StatsSources(&src)
|
||||
col := stats.NewCollector(st, opts, src, a.log)
|
||||
srv.SetStats(col)
|
||||
go func() {
|
||||
defer close(done)
|
||||
defer st.Close()
|
||||
col.Run(ctx)
|
||||
}()
|
||||
return done
|
||||
}
|
||||
|
||||
func sdNotify(state string) {
|
||||
sock := os.Getenv("NOTIFY_SOCKET")
|
||||
if sock == "" {
|
||||
|
||||
@@ -1,5 +1,8 @@
|
||||
# Housecarl: loads the last confirmed ruleset at boot, before the network
|
||||
# comes up and whether or not housecarld starts.
|
||||
# comes up and whether or not housecarld starts. If it fails to load, the
|
||||
# fallback housecarld saved beside it is loaded instead: nothing new gets
|
||||
# in from external zones (docs/operations.md), and the unit is marked
|
||||
# failed, until housecarld starts and puts the real ruleset back.
|
||||
# Installed as /etc/systemd/system/housecarl-ruleset.service.
|
||||
[Unit]
|
||||
Description=Housecarl firewall ruleset
|
||||
@@ -15,7 +18,8 @@ ConditionPathExists=/var/lib/housecarl/ruleset.nft
|
||||
[Service]
|
||||
Type=oneshot
|
||||
RemainAfterExit=yes
|
||||
ExecStart=/usr/sbin/nft -f /var/lib/housecarl/ruleset.nft
|
||||
# "<2>": logged as critical.
|
||||
ExecStart=/bin/sh -c '/usr/sbin/nft -f /var/lib/housecarl/ruleset.nft && exit 0; echo "<2>the saved ruleset failed to load: loading the fallback (nothing new gets in from external zones) until housecarld puts the real one back" >&2; /usr/sbin/nft -f /var/lib/housecarl/ruleset-fallback.nft; exit 1'
|
||||
|
||||
[Install]
|
||||
WantedBy=sysinit.target
|
||||
+68
-12
@@ -51,8 +51,16 @@ Calls that change the configuration answer:
|
||||
* `confirm_by`: the change is on probation, and is rolled back at that
|
||||
time unless `POST /confirm` comes first. Missing for changes made
|
||||
through the socket.
|
||||
* `confirm_token`: the client that made the change can confirm it by
|
||||
itself with `POST /confirm {"token"}`, from the same address, as the
|
||||
same user and token, over a new connection (the answer closes this
|
||||
one). Missing when a person must confirm (`manual_confirm`).
|
||||
* `?dry_run=1` checks the change, with nftables too, and returns what it
|
||||
would do, with `"applied": false, "dry_run": true`.
|
||||
would do, with `"applied": false, "dry_run": true`, and `diff`: the
|
||||
ruleset's unified diff.
|
||||
* `?changeset=ID` (or the header `X-Housecarl-Changeset: ID`) stages the
|
||||
change in that changeset instead ([Changesets](#changesets)): the answer
|
||||
has `"staged": true`, `changes` (this operation's) and `changeset`.
|
||||
* `before` is missing for a create and `after` for a delete.
|
||||
* `warnings` are the problems the change brings; older ones are in
|
||||
`/status`.
|
||||
@@ -66,11 +74,18 @@ Creating answers 201, other changes 200.
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| `GET /healthz` | `{"ok": true, "version"}`, without authentication |
|
||||
| `GET /healthz` | `ok`, `version`, `firewall` (`applied`, `unconfigured`, `fallback`, `missing`), `probation` (changes waiting to be confirmed), `warnings` (codes); HTTP 503 unless `ok`. Without authentication; read at most every five seconds ([operations.md](operations.md#monitoring)) |
|
||||
| `GET /status` | `build`, `started`, `applied` (the ruleset is in force), `apply_error`, `problems`, `probation`, `interfaces`, `counts` (objects by kind), `other_tables` (other nftables tables, `"inet main"`) |
|
||||
| `GET /interfaces` | the machine's interfaces: `name`, `up`, `addresses`, `zone` |
|
||||
| `GET /matrix[?changeset=ID]` | the zone matrix: `zones`, `cells` (`from`, `to` (a zone, or `firewall`), `chain`, `verdict` (allowed, blocked), `decided_by`, `rule_id`, `exceptions`: `rule_id`, `chain`, `label`, `verdict` (allowed, blocked, logged, unknown), `what`) ([rules.md](rules.md#the-zone-matrix)) |
|
||||
| `GET /ruleset` | the nftables ruleset Housecarl builds, as text |
|
||||
| `GET /counters` | `{"since", "counters": {"r12": {"packets", "bytes"}, ...}}`, by mark ([design.md](design.md#its-own-table-only)); totals since housecarld started |
|
||||
| `GET /quotas` | each rule's quota: `rule_id`, `rule`, `quota`, `used` (bytes), `addresses` (`[{"address", "used"}]` for a per-address quota), `last_reset`, `next_reset` |
|
||||
| `GET /connections[?q=&limit=]` | tracked connections: `family`, `protocol`, `source`, `source_port`, `destination`, `destination_port` (or `icmp_id`, `icmp_type`, `icmp_code`), `dnat_to`, `dnat_port`, `snat_to`, `state`, `assured`, `timeout` (seconds left), `packets` and `bytes` when the kernel counts them. `q`: all of its words in the addresses, `:port`, protocol, state |
|
||||
| `POST /connections/close` | `{"connections": [...]}` (as listed) or `{"search": "192.168.1.50"}`: delete their tracking entries (`config:write`, audited); answers `{"closed": N}` |
|
||||
| `POST /trace` | `{"address", "port", "protocol", "seconds"}` (5-600, default 60): start tracing (`config:write`, audited); 409 while one runs |
|
||||
| `GET /trace` | the trace running, or the last: `running`, `input`, `started`, `ends`, `error`, `packets`: each with `time`, `in`, `out`, `protocol`, addresses and ports, `tcp_flags`, `icmp_type`, `steps` (`table`, `chain`, `kind` rule or policy, `rule`, `mark`, `label`, `verdict`), `verdict` (accepted, dropped) and `decided_by` |
|
||||
| `DELETE /trace` | stop it |
|
||||
| `GET /log[?limit=&after=&q=]` | logged packets, newest first: `time`, `prefix`, `in`, `out`, `family`, `protocol`, `source`, `source_port`, `destination`, `destination_port`, `icmp_type`, `tcp_flags`, `length`, `mark` (the rule's). `after` (a time) for those since; `q` searches |
|
||||
| `GET /catalog` | built-in services, chains, actions, roles, the log group |
|
||||
|
||||
@@ -83,8 +98,9 @@ Creating answers 201, other changes 200.
|
||||
"log_policy_drops": false,
|
||||
"builtins": {"loopback": true, "established": true, "drop_invalid": true, "icmpv6": true,
|
||||
"ping_internal": true, "dhcpv6_client": true, "antispoof": true, "mss_clamp": false},
|
||||
"own_prefixes": ["2001:db8:7f3d::/48"], "antispoof_v4": ["10.0.0.0/8", "..."], "antispoof_v6": ["fc00::/7", "..."],
|
||||
"flowtable_devices": ["wan", "enp2s0"], "probation_seconds": 120}
|
||||
"own_prefixes": ["2001:db8::/48"], "antispoof_v4": ["10.0.0.0/8", "..."], "antispoof_v6": ["fc00::/7", "..."],
|
||||
"flowtable_devices": ["wan", "eth1"], "probation_seconds": 120, "manual_confirm": false,
|
||||
"stats_per_device": false}
|
||||
```
|
||||
|
||||
Lists replace the old ones. `probation_seconds` 0 is the default (120).
|
||||
@@ -102,24 +118,27 @@ share one pattern, `/{kind}` with kind `zones`, `sets`, `services`,
|
||||
| `GET /{kind}/{ref}` | by ID, or name for zones, sets, services and forwards |
|
||||
| `PATCH /{kind}/{ref}` | change the fields given (a JSON merge: `null` clears) |
|
||||
| `DELETE /{kind}/{ref}` | refused while something uses it |
|
||||
| `POST /rules/{ref}/reset-quota` | start a rule's quota again now (`config:write`; audited, not staged) |
|
||||
| `POST /{kind}/{ref}/move` | `{"before": ID}`, `{"after": ID}` or `{"to": "start"\|"end"}`; a rule can take `"chain"` too |
|
||||
|
||||
The objects:
|
||||
|
||||
```json
|
||||
{"name": "lan", "interfaces": ["enp2s0", "enp2s0.*"], "external": false, "comment": ""}
|
||||
{"name": "trusted", "entries": ["10.0.10.0/24", "2001:db8:7f3d:10::/64"], "dynamic": false, "timeout": 0, "comment": ""}
|
||||
{"name": "jellyfin", "entries": ["tcp/8096", "tcp/8920"], "comment": ""}
|
||||
{"name": "lan", "interfaces": ["eth1", "eth1.*"], "external": false, "comment": ""}
|
||||
{"name": "trusted", "entries": ["192.168.1.0/24", "2001:db8:1::/64"], "dynamic": false, "timeout": 0, "comment": ""}
|
||||
{"name": "web", "entries": ["tcp/8080", "tcp/8443"], "comment": ""}
|
||||
{"chain": "input",
|
||||
"in_zones": ["wan"], "not_in": false, "out_zones": [], "not_out": false,
|
||||
"sources": ["trusted"], "not_sources": false, "destinations": [], "not_destinations": false,
|
||||
"services": ["ssh"], "family": "",
|
||||
"ct_state": ["new"], "ct_status": [], "days": [], "hour_from": "", "hour_to": "",
|
||||
"limit": {"rate": 3, "per": "minute", "burst": 5, "over": true, "per_source": true}, "conn_limit": 0,
|
||||
"limit": {"rate": 3, "unit": "", "per": "minute", "burst": 5, "over": true, "per_source": true, "per_destination": false}, "conn_limit": 0,
|
||||
"quota": {"bytes": 53687091200, "over": true, "per_source": true, "per_destination": false, "reset": "monthly", "day": 15},
|
||||
"group_v4": 0, "group_v6": 0,
|
||||
"action": "drop", "reject_with": "", "log": true, "log_prefix": "",
|
||||
"raw": "", "comment": "", "disabled": false}
|
||||
{"name": "jellyfin", "protocols": ["tcp"], "ports": "8096", "public_addresses": ["203.0.113.7"], "sources": [],
|
||||
"to": "10.0.10.57", "to_port": "443", "to6": "2001:db8:7f3d:10::57", "hairpin": true, "hairpin_sources": [],
|
||||
{"name": "web", "protocols": ["tcp"], "ports": "8443", "public_addresses": ["203.0.113.10"], "sources": [],
|
||||
"to": "192.168.1.10", "to_port": "443", "to6": "2001:db8:1::10", "hairpin": true, "hairpin_sources": [],
|
||||
"comment": "", "disabled": false}
|
||||
```
|
||||
|
||||
@@ -127,6 +146,43 @@ A source NAT rule is a match (as a rule's, without the rule's other
|
||||
fields) with `"to"` (empty: masquerade), a redirect a match with
|
||||
`"to_port"`. Times are seconds (`timeout`).
|
||||
|
||||
### Statistics
|
||||
|
||||
`GET /stats/status`, `/stats/summary`, `/stats/timeline`, `/stats/top`,
|
||||
`/stats/hits` and `/stats/heatmap` (`log:read`): see
|
||||
[stats.md](stats.md#the-api).
|
||||
|
||||
### Changesets
|
||||
|
||||
Changes staged to be reviewed and applied together ([design.md](design.md#reviewing-changes-changesets)).
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| `GET /changesets[?status=open&mine=1&limit=]` | newest first |
|
||||
| `POST /changesets` | `{"description"}`; answers the changeset (201) |
|
||||
| `GET /changesets/{id}` | `{"id", "username", "via", "description", "status": "open"\|"applied"\|"discarded", "ops": [{"seq", "op", "kind", "ref", "id", "data", "summary", "added_at"}], "created_at", "updated_at", "history_id"}` |
|
||||
| `PATCH /changesets/{id}` | `{"description"}` |
|
||||
| `GET /changesets/{id}/preview` | `{"valid", "changes", "warnings", "error", "diff"}`: what applying it would do; `error.op` names an operation that no longer applies |
|
||||
| `POST /changesets/{id}/apply` | optional `{"description"}`; answers like a change, with `confirm_by` over the network and `changeset` |
|
||||
| `POST /changesets/{id}/discard` | |
|
||||
| `DELETE /changesets/{id}/ops/{seq}` | take one operation out |
|
||||
|
||||
Staging (`?changeset=ID` on any change to objects or settings) checks
|
||||
the changeset with the new operation, and nftables checks its ruleset;
|
||||
an error refuses the operation. Only open changesets take operations, and
|
||||
only their owner's (or an administrator's). An undo
|
||||
(`POST /history/{id}/revert?changeset=ID`) is staged as one operation,
|
||||
which checks when it is replayed that nothing it touches changed since;
|
||||
applied, History links it to the entry it undoes. `/import/nft` and
|
||||
`/takeover` can't be staged: with a changeset named they answer 400.
|
||||
|
||||
Reads of objects, settings and `/ruleset` with `?changeset=ID` show the
|
||||
configuration as the changeset will make it; objects are marked
|
||||
`"pending": "new"`, `"changed"` or `"deleted"` (deleted ones stay in the
|
||||
list). A changeset that no longer applies shows the current
|
||||
configuration, with the reason in the `X-Housecarl-Changeset-Error`
|
||||
header.
|
||||
|
||||
### Dynamic sets' entries
|
||||
|
||||
| | |
|
||||
@@ -144,7 +200,7 @@ history, but are audited.
|
||||
| | |
|
||||
|---|---|
|
||||
| `GET /probation` | `{"pending": [history IDs], "takeovers": N, "deadline", "seconds_left"}` |
|
||||
| `POST /confirm` | keep them: `{"confirmed": N}`; the ruleset is saved for the boot |
|
||||
| `POST /confirm` | keep them: `{"confirmed": N}`; the ruleset is saved for the boot. With `{"token"}`, the client confirming its own change (see `confirm_token`): 409 if a newer change is pending, or over the change's own connection; 403 from another user or address |
|
||||
| `POST /rollback` | undo them now |
|
||||
|
||||
### History, import, takeover
|
||||
@@ -154,7 +210,7 @@ history, but are audited.
|
||||
| `GET /history[?limit=&before=]` | changes, newest first |
|
||||
| `GET /history/{id}` | one, with each object before and after |
|
||||
| `POST /history/{id}/revert` | undo it; 409 if an object it touched has changed since, or it was undone already |
|
||||
| `POST /import/nft` | `{"json": NFT_JSON, "text": "nft -a list ruleset output", "defines": "ruleset file", "table": "inet main", "replace": false}` ([import.md](import.md)); answers with `state` (the imported configuration), `notes` and `raw` too |
|
||||
| `POST /import/nft` | `{"json": NFT_JSON, "text": "nft -a list ruleset output", "defines": "ruleset file", "table": "inet main", "zones": {"enp2s0": "lan"}, "replace": false}` ([import.md](import.md)); answers with `state` (the imported configuration), `notes` and `raw` too |
|
||||
| `POST /takeover` | `{"tables": ["inet main"]}`: delete these tables (administrators); on probation over the network |
|
||||
|
||||
### Certificate
|
||||
|
||||
@@ -23,7 +23,7 @@ on the internet, and Jarl puts it in every view that has your zone.
|
||||
3. **Configure the certificate** in `/etc/housecarl/housecarl.yaml`:
|
||||
|
||||
```yaml
|
||||
listen: ["10.0.10.2:8070"] # an address your browser can reach
|
||||
listen: ["192.168.1.2:8070"] # an address your browser can reach
|
||||
tls:
|
||||
acme:
|
||||
domain: fw.adbraun.lol
|
||||
|
||||
+82
-11
@@ -27,7 +27,9 @@ command and its flags.
|
||||
| `--config FILE` | `HOUSECARL_CLI_CONFIG` | the CLI's configuration file |
|
||||
| | `HOUSECARL_SOCKET` | the local socket's path |
|
||||
| `--insecure`, `--ca-cert FILE` | | TLS: skip the check, or trust another CA |
|
||||
| `-n, --dry-run` | | check and show a change without making it |
|
||||
| `-n, --dry-run` | | check and show a change without making it, with the ruleset's diff |
|
||||
| `--no-confirm` | | leave a change on probation instead of confirming it once `hc` reaches Housecarl again |
|
||||
| `-C, --changeset ID` | `HOUSECARL_CHANGESET` (checked, see [Changesets](#changesets)) | stage configuration changes in this changeset instead of making them |
|
||||
| `-o, --output FORMAT` | | `table` (default), `json`, `yaml` or `id` |
|
||||
| `-q, --quiet` | | print only errors (and the IDs of new objects) |
|
||||
|
||||
@@ -40,14 +42,14 @@ server error.
|
||||
Every change is checked as a whole, then by nftables, before it is put
|
||||
in force. A refused change says why, with the problem's code
|
||||
([validation.md](validation.md)). An applied one says its history entry
|
||||
and any new warnings. Over the network, it is on probation:
|
||||
and any new warnings. Over the network, it is on probation, and `hc`
|
||||
confirms it by reaching Housecarl again over a new connection
|
||||
([operations.md](operations.md#changes-on-probation)):
|
||||
|
||||
```
|
||||
$ hc rule add input --from trusted --service ssh --comment "SSH from home"
|
||||
created rule input #31 (from trusted · ssh)
|
||||
applied on probation (history 12): confirm with 'hc confirm' within 2m0s, or it is rolled back
|
||||
$ hc confirm
|
||||
confirmed 1 change; it is kept and loaded at boot
|
||||
applied and confirmed (history 12): this client reached Housecarl again over a new connection
|
||||
```
|
||||
|
||||
Through the socket the change is kept at once:
|
||||
@@ -56,8 +58,69 @@ Through the socket the change is kept at once:
|
||||
applied and saved (history 12; undo with 'hc history undo 12')
|
||||
```
|
||||
|
||||
`-n` shows what would change without changing it; nftables checks it
|
||||
too.
|
||||
`-n` shows what would change without changing it, with the diff of the
|
||||
ruleset; nftables checks it too.
|
||||
|
||||
## Changesets
|
||||
|
||||
To review several changes and apply them together, as one entry in the
|
||||
history, stage them in a changeset:
|
||||
|
||||
```sh
|
||||
export HOUSECARL_CHANGESET=$(hc changeset create "guest network" -q)
|
||||
hc zone add guest -i eth1.30
|
||||
hc rule add forward --in guest --out wan
|
||||
hc changeset diff # the operations, problems, nftables' check, the ruleset's diff
|
||||
hc changeset remove 2 # take operation 2 out
|
||||
hc changeset apply # one change; over the network on probation
|
||||
unset HOUSECARL_CHANGESET
|
||||
```
|
||||
|
||||
```
|
||||
$ hc rule add forward --in guest --out wan
|
||||
changeset 7 (guest network, 1 operation) from HOUSECARL_CHANGESET
|
||||
staged: create rule forward #52
|
||||
staged in changeset 7 as operation 2 (2 operations now); review with 'hc changeset diff 7', apply with 'hc changeset apply 7'
|
||||
```
|
||||
|
||||
Each change is checked as it is staged: the changeset with it must stay
|
||||
valid, and nftables must accept the ruleset, so make a set before the
|
||||
rule that names it. Applying checks the whole changeset again, on the
|
||||
configuration as it is then; an operation whose object changed since is
|
||||
named (`operation 3 (update rule input #12) can't be applied: …`), and
|
||||
`hc changeset remove` takes it out.
|
||||
|
||||
| Command | |
|
||||
|---|---|
|
||||
| `changeset create [DESCRIPTION]` | start one; `-q` prints only its ID |
|
||||
| `changeset list [--status open\|applied\|discarded\|""] [--mine]` | open ones by default |
|
||||
| `changeset show [ID]` | its operations |
|
||||
| `changeset diff [ID]` | what applying it would do |
|
||||
| `changeset apply [ID] [--description D]` | apply it |
|
||||
| `changeset discard [ID]` | throw it away |
|
||||
| `changeset remove [ID] OP` | take one operation out |
|
||||
|
||||
`show`, `diff`, `apply`, `discard` and `remove` work on the changeset you
|
||||
name, else the one in `-C`, else the one in `HOUSECARL_CHANGESET`.
|
||||
|
||||
A changeset from `HOUSECARL_CHANGESET` is checked before anything uses
|
||||
it, because an exported ID easily outlives its changeset:
|
||||
|
||||
- Staging into it, and `apply`, `discard` and `remove` on it, need it to
|
||||
be open and yours. Otherwise the command stops and says why (`there is
|
||||
no changeset 99`, `changeset 42 is applied`, `changeset 57 is sam's`).
|
||||
Commands that only read, and `hc changeset create`, don't care.
|
||||
- Each command says on stderr which changeset it uses: `changeset 42
|
||||
(guest network, 2 operations) from HOUSECARL_CHANGESET` (not with `-q`).
|
||||
- After `apply` or `discard`, a reminder to `unset HOUSECARL_CHANGESET`.
|
||||
|
||||
`-C ID` and an ID argument are used as given (housecarld still refuses a
|
||||
changeset that isn't open, or another user's unless you are an
|
||||
administrator). Only configuration changes are staged: dynamic set
|
||||
entries, `confirm` and `rollback` are made at once; `history undo` is
|
||||
staged like any change; `import` and `takeover` can't be staged, so they
|
||||
are refused while a changeset is named. Changesets staged in the web UI show up here too,
|
||||
and the other way round.
|
||||
|
||||
## Commands
|
||||
|
||||
@@ -72,7 +135,12 @@ by name.
|
||||
| `interfaces` | the machine's interfaces, their addresses and zones |
|
||||
| `ruleset` | the nftables ruleset Housecarl builds |
|
||||
| `counters` | the hits of every rule, NAT rule, built-in and policy, by mark (`r12`, `f3`, `b-established`, `p-input`) |
|
||||
| `quotas` | what each rule's quota used (each address's, for per-address quotas), and when it resets |
|
||||
| `log [-f] [--search TEXT] [--limit N]` | the packets logged by rules and policies ([operations.md](operations.md#the-log)) |
|
||||
| `connections [--search TEXT] [--limit N]`, `connections close --search TEXT [-y]` | tracked connections; closing makes the rules check them again ([operations.md](operations.md#connections)) |
|
||||
| `stats [--since 24h]` | traffic per zone, drops, the busiest rules, blocked sources, devices; `stats status`, `stats traffic`, `stats top devices\|blocked\|tried\|quota ID`, `stats rules [--unused]`, `stats connections` ([stats.md](stats.md#the-cli)) |
|
||||
| `matrix [FROM TO]` | what the rules do to new connections between zones; with FROM and TO, that pair's rules in order ([rules.md](rules.md#the-zone-matrix)) |
|
||||
| `trace [--address A] [--port N] [--protocol P] [--seconds S]` | follow packets through every table's rules ([operations.md](operations.md#tracing)) |
|
||||
|
||||
### zone, set, service
|
||||
|
||||
@@ -80,7 +148,7 @@ by name.
|
||||
|---|---|
|
||||
| `zone list`, `zone show ZONE` | |
|
||||
| `zone add NAME -i IFACE... [--external] [--comment C]` | |
|
||||
| `zone update ZONE [-i IFACE]... [--add IFACE] [--remove IFACE] [--external[=false]] [--name N] [--comment C]` | `-i` replaces the interfaces |
|
||||
| `zone update ZONE [-i IFACE]... [--add IFACE] [--remove IFACE] [--external[=false]] [--name N] [--comment C]` | `-i` replaces the interfaces; `--name` renames it, and the rules naming it follow |
|
||||
| `zone delete ZONE` | refused while rules use it |
|
||||
| `set list`, `set show SET` | |
|
||||
| `set add NAME [ENTRY...] [--dynamic] [--timeout 24h] [--comment C]` | |
|
||||
@@ -103,6 +171,7 @@ See [rules.md](rules.md).
|
||||
| `rule update ID [flags]` | only the flags given |
|
||||
| `rule move ID --before ID\|--after ID\|--first\|--last\|--chain CHAIN` | |
|
||||
| `rule enable\|disable ID` | |
|
||||
| `rule reset-quota ID` | start a rule's quota again now |
|
||||
| `rule delete ID` | |
|
||||
|
||||
The match flags (`rule add`, `snat add`, `redirect add`):
|
||||
@@ -115,7 +184,9 @@ The match flags (`rule add`, `snat add`, `redirect add`):
|
||||
| `--family ipv4\|ipv6` | |
|
||||
|
||||
And for rules: `--state`, `--status`, `--days`, `--hours`, `--limit
|
||||
10/minute[,burst=N][,over][,per-source]`, `--conn-limit N`, `-a
|
||||
10/minute|1mbytes/second[,burst=N][,over][,per-source|,per-destination]`,
|
||||
`--quota 50gbytes[,over][,per-source|,per-destination][,daily|,weekly|,monthly=DAY]`,
|
||||
`--group-v4 N`, `--group-v6 N`, `--conn-limit N`, `--every-packet`, `-a
|
||||
accept|drop|reject|log`, `--reject-with tcp-reset`, `--log`,
|
||||
`--log-prefix`, `--raw RULE`, `--comment`, `--disabled`
|
||||
([rules.md](rules.md#rules)).
|
||||
@@ -137,7 +208,7 @@ See [nat.md](nat.md).
|
||||
| Command | |
|
||||
|---|---|
|
||||
| `settings show` | |
|
||||
| `settings set [--input-policy\|--forward-policy\|--output-policy accept\|drop] [--log-drops on\|off] [--builtin NAME=on\|off]... [--own-prefix P]... [--antispoof-v4 P]... [--antispoof-v6 P]... [--flowtable DEV]... [--probation 2m]` | lists given replace the old ones |
|
||||
| `settings set [--input-policy\|--forward-policy\|--output-policy accept\|drop] [--log-drops on\|off] [--builtin NAME=on\|off]... [--own-prefix P]... [--antispoof-v4 P]... [--antispoof-v6 P]... [--flowtable DEV]... [--probation 2m] [--auto-confirm on\|off] [--stats-per-device on\|off]` | lists given replace the old ones; `none` empties one (`--flowtable none`: off) |
|
||||
|
||||
### probation, history, import
|
||||
|
||||
@@ -148,7 +219,7 @@ See [nat.md](nat.md).
|
||||
| `history list [--limit N]` | changes, newest first |
|
||||
| `history show ID` | each object before and after |
|
||||
| `history undo ID` | apply the change backwards (refused if something it touched has changed since) |
|
||||
| `import [FILE\|-] [--text FILE] [--defines FILE] [--table T] [--replace]` | [import.md](import.md) |
|
||||
| `import [FILE\|-] [--text FILE] [--defines FILE] [--table T] [--zone IFACE=NAME]... [--replace]` | [import.md](import.md) |
|
||||
| `takeover TABLE... [-y]` | delete tables Housecarl's ruleset replaces (administrators) |
|
||||
|
||||
### cert
|
||||
|
||||
@@ -14,7 +14,7 @@ reads the file when it starts.
|
||||
| `tls.cert_file`, `tls.key_file` | none | serve HTTPS with this certificate; housecarld rereads it when the file changes, or on `systemctl reload housecarl` |
|
||||
| `tls.acme.*` | off | get the certificate from Let's Encrypt, with Jarl publishing the challenge ([certificates.md](certificates.md#settings)) |
|
||||
| `jarl.server`, `jarl.token_file` (or `jarl.token`), `jarl.ca_cert`, `jarl.insecure` | none | the Jarl server for the certificate's challenge, and its token ([jarl.md](jarl.md)) |
|
||||
| `ruleset` | `/var/lib/housecarl/ruleset.nft` | the confirmed ruleset, which `housecarl-ruleset.service` loads at boot |
|
||||
| `ruleset` | `/var/lib/housecarl/ruleset.nft` | the confirmed ruleset, which `housecarl-ruleset.service` loads at boot; its fallback is saved beside it as `ruleset-fallback.nft` ([operations.md](operations.md#if-the-ruleset-fails-to-load-at-boot)). Change both paths in the unit too |
|
||||
| `trusted_proxies` | `[]` | addresses or prefixes of reverse proxies whose `X-Forwarded-For` and `X-Forwarded-Proto` are believed |
|
||||
| `socket` | `/run/housecarl/housecarl.sock` | the local API socket; `""` turns it off |
|
||||
| `database` | `/var/lib/housecarl/housecarl.db` | the SQLite database: configuration, history, users, audit log |
|
||||
@@ -26,11 +26,16 @@ reads the file when it starts.
|
||||
| `backup.dir` | `/var/backups/housecarl` | where `housecarld backup` writes |
|
||||
| `backup.retention` | `30d` | backups older than this are deleted… |
|
||||
| `backup.keep_at_least` | `7` | …keeping at least this many |
|
||||
| `stats.enabled` | `true` | keep traffic, drops, hits, blocked sources, connections and quotas over time ([stats.md](stats.md)) |
|
||||
| `stats.database` | `/var/lib/housecarl/stats.db` | the statistics' own database; deleting it is safe |
|
||||
| `stats.retention.minute`, `.hour`, `.day` | `7d`, `90d`, `730d` | how long per-minute, hourly and daily figures are kept |
|
||||
| `stats.top_n` | `100` | entries kept per top list, per hour and per day |
|
||||
|
||||
Durations are written `15s`, `12h` or `30d`.
|
||||
|
||||
The probation's length is a firewall setting, not here: `hc settings set
|
||||
--probation 5m`.
|
||||
--probation 5m`. So is counting traffic per device for the statistics:
|
||||
`hc settings set --stats-per-device on`.
|
||||
|
||||
## HTTPS
|
||||
|
||||
@@ -40,7 +45,7 @@ Let it get its own certificate through Jarl ([certificates.md](certificates.md))
|
||||
or give it one:
|
||||
|
||||
```yaml
|
||||
listen: ["10.0.10.1:8070"]
|
||||
listen: ["192.168.1.1:8070"]
|
||||
tls:
|
||||
cert_file: /etc/housecarl/tls/fullchain.pem
|
||||
key_file: /etc/housecarl/tls/privkey.pem
|
||||
|
||||
+57
-1
@@ -50,6 +50,35 @@ Every change goes through one engine (`internal/engine`):
|
||||
Dry runs, history with undo, users, sessions and scoped API tokens come
|
||||
from the same machinery as in Jarl and Steward.
|
||||
|
||||
## Reviewing changes: changesets
|
||||
|
||||
As in Jarl, a change can be made at once or staged in a **changeset**,
|
||||
to be reviewed and applied together with others:
|
||||
|
||||
* Each change is an **operation** (create, update, delete or move an
|
||||
object, or change the settings), kept as the API received it.
|
||||
* Staged in a changeset, an operation is checked as it is added: the
|
||||
changeset with it must still be valid, and nftables must accept the
|
||||
ruleset it makes. Nothing is applied.
|
||||
* The **review** shows the operations, the problems, and the **diff of
|
||||
the ruleset**: the file that is loaded at boot, now and after.
|
||||
* **Applying** replays the operations on the configuration as it is then,
|
||||
as one change: one check, one transaction, one history entry, undone as
|
||||
a whole. An operation that no longer applies (its object was changed or
|
||||
deleted since) is named, and can be taken out.
|
||||
* Then the applied changeset is **on probation** like any change made over
|
||||
the network: unless it is confirmed, it is rolled back.
|
||||
|
||||
| Interface | Default | Why |
|
||||
|---|---|---|
|
||||
| Web UI | Review edits: they are collected in a draft changeset, pages show pending rows (green new, yellow changed, struck out deleted), and a bar leads to the Changes page, with the diff and Apply. The switch in the header turns on Live edits, applied at once. | Several related edits, seen together before they are in force. |
|
||||
| CLI | Immediate: each command is a change of its own. Stage with `-C ID` or `HOUSECARL_CHANGESET`. | Scripts want a clear result per step. |
|
||||
| API | Immediate unless a changeset is named (`?changeset=ID`, `X-Housecarl-Changeset`). | As the CLI. |
|
||||
|
||||
An undo is staged like any edit (in review mode, the web UI's Undo
|
||||
collects it for review). Imports and takeovers can't be staged; with a
|
||||
changeset named, they are refused rather than applied.
|
||||
|
||||
## Its own table only
|
||||
|
||||
Housecarl writes one table, `inet housecarl`, and leaves every other
|
||||
@@ -73,11 +102,25 @@ and log lines are traced back to their object through it.
|
||||
|
||||
## Not locking yourself out
|
||||
|
||||
A change made over the network is applied **on probation**: unless it is
|
||||
A change made over the network is applied **on probation** (a changeset
|
||||
when it is applied, a live edit at once): unless it is
|
||||
confirmed within two minutes (Settings; `hc confirm`, the banner in the
|
||||
web UI, `POST /confirm`), it is rolled back by itself, and the rollback is
|
||||
recorded in the history. A change that cuts you off undoes itself.
|
||||
|
||||
* **The client that made a change confirms it by itself** once it
|
||||
reaches the API again: the change's answer carries a one-time token and
|
||||
closes its connection (HTTP/2: a GOAWAY), and the web UI and `hc` send
|
||||
the token straight back. That proves something only on a new
|
||||
connection, since the built-in that accepts established connections
|
||||
would let the old one through whatever the change did. So the token is
|
||||
accepted only from the same user (and API token), the same address, a
|
||||
different connection, and for the newest change. A change that cuts its
|
||||
client off can't be confirmed by it, and rolls back. This proves the
|
||||
API is still reachable, not that SSH or the LAN still work; with the
|
||||
API, you can fix those. Settings → "a person confirms each change"
|
||||
(`hc settings set --auto-confirm off`) goes back to confirming by hand;
|
||||
`hc --no-confirm` leaves one change waiting.
|
||||
* Changes made through the local socket (`sudo hc …` on the firewall)
|
||||
need no confirmation: whoever makes them is already on the machine. One
|
||||
made while others are pending joins them.
|
||||
@@ -96,6 +139,19 @@ firewall is up even if housecarld can't start. An unconfirmed ruleset is
|
||||
never saved there: a reboot during probation brings back the last
|
||||
confirmed one.
|
||||
|
||||
It fails closed. A fallback saved beside it (`ruleset-fallback.nft`,
|
||||
`render.Fallback`) is loaded if it doesn't: no sets, no flowtable,
|
||||
interfaces by name only, so there is little in it to fail. It keeps
|
||||
anything new from the external zones out and lets the inside work,
|
||||
which is as far as a ruleset can go without knowing the rules; a
|
||||
fallback that let everything in would leave a router open on the
|
||||
internet, and one that let nothing in would lock out the person who has
|
||||
to fix it. The boot ruleset has no flowtable either: nftables refuses a
|
||||
flowtable naming a device that doesn't exist yet (on Debian 13's
|
||||
kernel), and before the network is up, `ppp0` doesn't. housecarld
|
||||
watches the kernel's link messages (internal/linkwatch) and applies
|
||||
again when a flowtable device appears, or comes back as a new interface.
|
||||
|
||||
## The drop log
|
||||
|
||||
Rules with logging on, and the policies (Settings), send packets to
|
||||
|
||||
@@ -0,0 +1,181 @@
|
||||
# The flowtable: a fast path for forwarded traffic
|
||||
|
||||
The flowtable is a shortcut inside the kernel for traffic the firewall
|
||||
**forwards** (between the internet and your LAN, between zones). It
|
||||
speeds up routing on a machine whose CPU can't keep up with its line.
|
||||
It doesn't make the firewall any safer or less safe, but it changes what
|
||||
some features can see, so read [when not to use it](#when-not-to-use-it)
|
||||
before you turn it on.
|
||||
|
||||
It is off until you name its devices.
|
||||
|
||||
## What it does
|
||||
|
||||
Without a flowtable, every forwarded packet takes the whole way through
|
||||
the kernel: the prerouting hooks, the routing decision, Housecarl's
|
||||
forward chain (the built-ins, then your rules) and postrouting with NAT.
|
||||
For an established connection almost all of that work only ends in the
|
||||
`established` built-in accepting it again, packet after packet.
|
||||
|
||||
With a flowtable, Housecarl adds one built-in to the forward chain:
|
||||
|
||||
```
|
||||
meta l4proto { tcp, udp } flow add @ft
|
||||
```
|
||||
|
||||
A TCP or UDP connection takes the whole way for its first packets, as
|
||||
always: your rules decide whether it is allowed at all. Once the kernel
|
||||
has seen it established (and your rules let it through), it is added
|
||||
to the flowtable. From then on its packets are caught as soon as they
|
||||
arrive on one of the flowtable's devices (the *ingress* hook) and sent
|
||||
straight out on the other side: NAT is still applied, the TTL still
|
||||
counts down, but every chain in between is skipped.
|
||||
|
||||
```
|
||||
first packets every packet after
|
||||
in ─► prerouting ─► routing ─► forward ─► postrouting ─► out
|
||||
│ ▲
|
||||
└────────────── flowtable (ingress) ─────────────────────┘
|
||||
```
|
||||
|
||||
It is the software flowtable, run by the CPU. Offloading to the network
|
||||
card's hardware isn't supported ([design.md](design.md#not-in-01)).
|
||||
|
||||
## When to use it
|
||||
|
||||
Turn it on when **the router's CPU is what limits your speed**:
|
||||
|
||||
* a speed test through the router gets less than your line gives (or
|
||||
less than the same test from the router itself), and
|
||||
* during it, one CPU core is busy with network work: in `top`, press `1`
|
||||
for every core, and look at `si` (softirq) near 100 % on a core.
|
||||
|
||||
Typical cases: a small or old machine (Atom, Celeron, a Raspberry Pi, an
|
||||
old Core 2) on a fast line (500 Mbit/s and up), lots of small packets
|
||||
(gaming, VoIP, many downloads at once), or a machine that is busy with
|
||||
other work besides routing. The gain is CPU time per packet: often well
|
||||
over half of the forwarding cost, which shows up as speed only when the
|
||||
CPU was the limit, and otherwise as a cooler, quieter, more responsive
|
||||
machine.
|
||||
|
||||
## When not to use it
|
||||
|
||||
**When the CPU keeps up anyway.** If the speed test already reaches your
|
||||
line and `si` stays low, the flowtable buys you nothing you'd notice,
|
||||
and costs you the figures below. Most modern x86 machines route a
|
||||
gigabit line without it.
|
||||
|
||||
**When you count or limit traffic in the forward chain.** The fast path
|
||||
skips the forward chain, so whatever must see *every* packet of a
|
||||
connection only sees the first few. Housecarl warns about each:
|
||||
|
||||
| Feature | With the flowtable | Warning |
|
||||
|---|---|---|
|
||||
| quotas and byte-rate limits on forward rules | only count first packets: a device can go far past its allowance | `flowtable_traffic` |
|
||||
| forward rules with "every packet" (`--every-packet`) | only see first packets | `flowtable_traffic` |
|
||||
| statistics' traffic per device (`stats_per_device`) | only count first packets | `stats_flowtable` |
|
||||
| hit counters of the `established` built-in (forward) | stop rising once a connection is offloaded | |
|
||||
|
||||
Rate limits by packets (`--limit 10/second`) and connection limits act
|
||||
on new connections, so they work the same.
|
||||
|
||||
**When other tables must see forwarded traffic.** Offloaded packets skip
|
||||
the forward hooks of *every* table, not only Housecarl's: Docker's,
|
||||
Incus', libvirt's or a table of your own. Those normally only judge new
|
||||
connections, which still pass through them, but a table that counts or
|
||||
shapes established traffic in its forward chain won't see it.
|
||||
|
||||
What keeps working as before:
|
||||
|
||||
* **Your rules** decide every new connection: nothing gets offloaded
|
||||
that your rules didn't let through. A rule you add later doesn't stop
|
||||
a connection already open, but that is true without the flowtable too
|
||||
(the `established` built-in accepts it).
|
||||
* **NAT**: port forwards, hairpin NAT, masquerading and SNAT.
|
||||
* **The input and output chains**: traffic to and from the firewall
|
||||
itself never uses the flowtable.
|
||||
* **Closing connections** (`hc connections close`, Traffic →
|
||||
Connections) ends their fast path too: the kernel drops a flow with
|
||||
its tracking entry.
|
||||
* **Interface statistics** (traffic per zone and interface) come from the
|
||||
interfaces' own counters, which count every packet.
|
||||
* **Traffic shaping with `tc`** on the outgoing interface still applies.
|
||||
* **The log** and **the packet tracer** see what goes through the rules:
|
||||
new connections, as always. The tracer doesn't see the packets of an
|
||||
offloaded connection; trace a new one.
|
||||
* ICMP, and protocols other than TCP and UDP (GRE, ESP, SCTP), always
|
||||
take the normal way.
|
||||
|
||||
## How to turn it on
|
||||
|
||||
Name the devices whose traffic should take the fast path, **both
|
||||
sides**: the interface connections come in on and the one their replies
|
||||
come in on. For a router that means the external interface and the
|
||||
inside ones:
|
||||
|
||||
```sh
|
||||
hc settings set --flowtable wan --flowtable enp2s0
|
||||
hc settings # Flowtable: wan, enp2s0
|
||||
```
|
||||
|
||||
In the web UI: Settings → Flowtable, add each device. The fast path
|
||||
only catches packets arriving on a listed device, so a connection to or
|
||||
from an unlisted one gains little or nothing: leave a device out only on
|
||||
purpose (say, a WireGuard interface whose traffic you'd rather keep on
|
||||
the normal way).
|
||||
|
||||
* Use **exact device names**, not wildcards (`incusbr*` is refused):
|
||||
the kernel attaches the flowtable to each device by name.
|
||||
* With **PPPoE**, name the PPP interface (`ppp0`, or the name your
|
||||
`ppp@` unit gives it), the one in your external zone.
|
||||
* `--flowtable` replaces the whole list; `hc settings set --flowtable
|
||||
none` turns the flowtable off.
|
||||
|
||||
The change is applied (and on probation, if made over the network) like
|
||||
any other.
|
||||
|
||||
### Devices that come and go
|
||||
|
||||
PPP and WireGuard interfaces don't exist until their service brings
|
||||
them up, and PPPoE makes a new `ppp0` each time it reconnects. Housecarl
|
||||
handles that:
|
||||
|
||||
* A device that doesn't exist yet is left out of the flowtable, with the
|
||||
warning `flowtable_device_missing`, instead of failing the whole
|
||||
ruleset.
|
||||
* housecarld watches the kernel's interface changes: when a flowtable
|
||||
device appears, or comes back as a new interface after a reconnect,
|
||||
it applies the ruleset again with it, a couple of seconds later. The
|
||||
journal says `interfaces changed: applied the ruleset again`.
|
||||
Applying again keeps the counters, dynamic sets and quotas, as any
|
||||
change does, and adds no history entry.
|
||||
* The ruleset saved for boot has no flowtable at all: it loads before
|
||||
the network is up, when `ppp0` and `wg0` can't exist yet, and
|
||||
nftables on Debian 13's kernel refuses a flowtable naming a device
|
||||
that doesn't exist (and with it the whole file). housecarld adds the
|
||||
flowtable once it starts. Until then, traffic takes the normal way,
|
||||
through the same rules.
|
||||
|
||||
## Checking it works
|
||||
|
||||
```sh
|
||||
sudo nft list flowtables # the devices it is attached to
|
||||
sudo conntrack -L | grep OFFLOAD # connections on the fast path
|
||||
```
|
||||
|
||||
`conntrack` is in Debian's `conntrack` package. A long download through
|
||||
the router should show up as `[OFFLOAD]`. Compare a speed test, and `si`
|
||||
in `top`, before and after.
|
||||
|
||||
If nothing gets offloaded, check that both of the connection's devices
|
||||
are listed, that `flowtable_device_missing` isn't in `hc status`, and
|
||||
that the connection is TCP or UDP and established.
|
||||
|
||||
## Turning it off
|
||||
|
||||
```sh
|
||||
hc settings set --flowtable none
|
||||
```
|
||||
|
||||
Connections on the fast path go back to the normal way with their next
|
||||
packet.
|
||||
+106
-18
@@ -18,11 +18,9 @@ shows what it would make, without saving anything: zones, sets,
|
||||
services, rules, port forwards and NAT, then **notes** on every guess it
|
||||
made, and the rules it **kept raw**.
|
||||
|
||||
`--defines` reads your ruleset file's `define` lines, so sets get your
|
||||
names (`define trusted_ips = { ... }` becomes the set `trusted_ips`).
|
||||
Without it, sets are named after what they hold. Interfaces named by a
|
||||
define name their zone too (`define wan_interface = "ppp0"` makes the
|
||||
zone `wan`); otherwise a zone is named after its interface.
|
||||
`--defines` reads your ruleset file's `define` lines, so sets and zones
|
||||
get your names ([How things are named](#how-things-are-named)).
|
||||
`--zone enp2s0=lan` names a zone yourself.
|
||||
|
||||
From another machine, give the JSON as a file, and the text with
|
||||
`--text`:
|
||||
@@ -37,6 +35,46 @@ With several tables, `--table "inet main"` picks one; by default every
|
||||
table but Housecarl's is read. The web UI has the same under Server →
|
||||
Import a ruleset.
|
||||
|
||||
## How things are named
|
||||
|
||||
**Zones** are named in this order:
|
||||
|
||||
1. **As you say**: `--zone enp2s0=lan` (repeat it; Zone names on the
|
||||
web UI's Import page). Interfaces given the same name share one zone
|
||||
(`--zone enp2s0=lan --zone enp3s0=lan`).
|
||||
2. **After a define holding the interface**, with a trailing `_int`,
|
||||
`_if`, `_iface`, `_ifname`, `_interface`, `_dev`, `_device` or `_nic`
|
||||
taken off: `define wan_int = "ppp0"` makes the zone `wan`, `define
|
||||
iot_if = enp2s0.20` (a VLAN) the zone `iot`.
|
||||
3. **After the interface itself**: `enp2s0`.
|
||||
|
||||
If the name is taken, the zone is called `INTERFACE_zone`. A define
|
||||
names what it was *for* in your old ruleset, which isn't always what the
|
||||
network is (`define dhcp_int = enp2s0` makes `dhcp`, for your LAN). A
|
||||
note lists each zone and where its name came from; rename one with
|
||||
`--zone` and import again, or afterwards with `hc zone update dhcp
|
||||
--name lan` (its rules follow). Name zones for what the network is for:
|
||||
`lan`, `iot`, `guest`, `servers`, `vpn`, `wan`.
|
||||
|
||||
**Address sets**: a named set of the table keeps its name; a define
|
||||
holding a list of addresses, or one prefix, gives its name to a set
|
||||
(`define trusted = { 10.0.10.0/24, 10.1.1.0/24 }` makes `trusted`,
|
||||
wherever those addresses appear together). A single address stays an
|
||||
address. When two defines hold the same addresses, the first by name
|
||||
wins.
|
||||
|
||||
**Services**: a define holding several ports names a service (`define
|
||||
web_ports = { 80, 443 }` makes `web_ports`); one port stays a port
|
||||
(`tcp/22`).
|
||||
|
||||
**Port forwards** are named after their comment, without a leading
|
||||
"Port forward" (`comment "Port Forward Jellyfin"` makes `jellyfin`);
|
||||
several ports in one DNAT become one forward each (`web-80`, `web-443`),
|
||||
and a DNAT map's elements are named after the map (`forwards-443`).
|
||||
|
||||
Names that aren't valid are made so: other characters become `_` (`-`
|
||||
in forward names), and a name must start with a letter.
|
||||
|
||||
## What becomes what
|
||||
|
||||
* **Interfaces** become zones. Zones are marked external when the rules
|
||||
@@ -45,27 +83,69 @@ Import a ruleset.
|
||||
* **Sets and defines** of addresses become address sets; port lists
|
||||
become services.
|
||||
* **Jumps and gotos** to your own chains are flattened: the jump's
|
||||
condition is added to each rule of the chain, in place.
|
||||
condition is added to each rule of the chain, in place. When the jump
|
||||
and a rule both match sources (or destinations), the narrower is kept
|
||||
if one lies within the other; otherwise, and for any rule there that
|
||||
stays raw, the jump's condition is written in front of the raw rule's
|
||||
text, so it still only applies where the jump did.
|
||||
* **Rules** keep their order. A rule that is a built-in (loopback,
|
||||
established, invalid, the ICMPv6 rules, MSS clamping, the DHCPv6 client
|
||||
rule, antispoofing) becomes that built-in only when it sits where the
|
||||
built-in goes, at the head of every chain it covers. Otherwise it stays
|
||||
a rule where it was, so the ruleset does what it did. Built-ins your
|
||||
ruleset doesn't have are turned off.
|
||||
a rule where it was, so the ruleset does what it did. Since built-ins
|
||||
come before every rule, once one at a chain's head stays a rule, the
|
||||
ones after it there stay rules too, in order: an invalid drop that
|
||||
stays a rule keeps the ICMPv6 accepts after it behind it, as they were.
|
||||
Built-ins your ruleset doesn't have are turned off.
|
||||
* **DNAT** rules become port forwards, TCP and UDP of the same port
|
||||
merged into one. The forward-chain accepts that let them through
|
||||
become part of the port forward, and a masquerade for inside sources
|
||||
to the same host becomes its hairpin. An IPv6 accept to the same port
|
||||
becomes the forward's IPv6 pinhole when there is only one candidate.
|
||||
merged into one, one per port when a rule has several. The
|
||||
forward-chain accepts that let them through (`ct status dnat`) become
|
||||
part of the port forward; an accept to the same host and port without
|
||||
`ct status dnat` lets in more than forwarded connections, and stays a
|
||||
rule. A masquerade for inside sources to the same host becomes its
|
||||
hairpin. An IPv6 accept to the same port, from everywhere, becomes the
|
||||
forward's IPv6 pinhole when there is only one candidate. A DNAT to an
|
||||
IPv6 address, or with matches a port forward has no field for, is left
|
||||
out with a note.
|
||||
* **Maps are expanded**, one element at a time. A DNAT map
|
||||
(`dnat ip to tcp dport map @forwards`, named or written in the rule)
|
||||
becomes one port forward per element; forwards from a named map are
|
||||
called after it (`forwards-443`). A verdict map
|
||||
(`tcp dport vmap { 22 : accept, 23 : drop }`) becomes one rule per
|
||||
element, in place, with `jump`s inlined as above. A lookup matches at
|
||||
most one element, so the rules do the same. A map keyed by several
|
||||
fields (`ip saddr . tcp dport`) can't be expanded: a rule using a named
|
||||
one is left out with a note, to be added by hand.
|
||||
* **Masquerade and SNAT** become source NAT; **redirect** becomes a
|
||||
redirect.
|
||||
* **Meters** (`update @m { ip saddr limit rate over 3/minute }`,
|
||||
`add @m { ip saddr ct count over 20 }`, quotas in sets) become the
|
||||
rule's per-source (or per-destination) limit, connection limit or quota;
|
||||
a mask in the key (`ip saddr and 255.255.255.0`) becomes its grouping.
|
||||
Byte rates and quotas are imported too; a quota imported resets only by
|
||||
hand until you give it a period. A rule with a quota or a byte rate that
|
||||
accepts stays a raw rule ([rules.md](rules.md#bandwidth-and-quotas)).
|
||||
* **Sets with `flags timeout`** (filled by a program or a rule) become
|
||||
dynamic sets, without their entries of the moment: whatever fills them
|
||||
must fill Housecarl's from then on, as a note says.
|
||||
* **Policies** become the chains' policies.
|
||||
* **Anything else** becomes a raw rule with its original text, in place.
|
||||
A raw rule naming a set of the old table needs the new set name:
|
||||
Housecarl's are `NAME_v4` and `NAME_v6`, and a note says so.
|
||||
* **Anything else** becomes a raw rule with its original text, in place:
|
||||
a match Housecarl has no field for, two matches on the same field
|
||||
(`ip saddr 10.0.0.0/8 ip saddr != 10.0.10.0/24`), `!=` where Housecarl
|
||||
only has `==`, a set of interface names. Its set references are made
|
||||
to work in Housecarl's table: an imported address set becomes
|
||||
`@NAME_v4` or `@NAME_v6`, any other set (interface names, ports) is
|
||||
written out with its elements. A raw rule naming a set that can't be
|
||||
(a meter, an empty set) is left out with a note.
|
||||
* **Logging rules** log to Housecarl's log (`hc log`), not the kernel's.
|
||||
|
||||
Chains on hooks Housecarl doesn't manage (ingress, a bridge table) are
|
||||
left out, with a note.
|
||||
Several base chains on one hook (`input` at priority -10 and at 0) are
|
||||
put one after the other, by priority, into Housecarl's one chain, with
|
||||
the stricter policy; a note says to check it, since a packet had to get
|
||||
past each, and an accept in the first no longer ends there. A prerouting
|
||||
filter chain after connection tracking becomes the early chain, which
|
||||
runs before it: a note says so. Chains on hooks Housecarl doesn't manage
|
||||
(ingress, a bridge table) are left out, with a note.
|
||||
|
||||
## 2. Import
|
||||
|
||||
@@ -91,7 +171,10 @@ the old table:
|
||||
sudo hc takeover "inet main"
|
||||
```
|
||||
|
||||
From now on only Housecarl's rules apply. Over the network, this is on
|
||||
From now on only Housecarl's rules apply. Until you do, the dashboard
|
||||
and `hc status` warn that the imported table still applies
|
||||
(`takeover_pending`); they also warn while `nftables.service` is enabled
|
||||
(`nftables_service`). Over the network, this is on
|
||||
probation too: unless confirmed, the old table comes back.
|
||||
|
||||
## 4. The boot
|
||||
@@ -106,6 +189,11 @@ sudo systemctl disable nftables.service
|
||||
sudo mv /etc/nftables.conf /etc/nftables.conf.before-housecarl
|
||||
```
|
||||
|
||||
Disable it, but **don't stop it** (no `systemctl stop nftables`, no
|
||||
`disable --now`): Debian's unit stops by running `nft flush ruleset`,
|
||||
which deletes every table, Housecarl's and Docker's included, at once.
|
||||
Disabled, it simply doesn't run at the next boot.
|
||||
|
||||
`housecarl-ruleset.service` runs after `nftables.service` in any case,
|
||||
so a leftover `flush ruleset` can't undo it. Docker, Incus and libvirt
|
||||
keep adding their own tables as before.
|
||||
|
||||
+3
-3
@@ -90,15 +90,15 @@ you are changing the firewall you are connected through.
|
||||
policies drop what no rule accepts, so let yourself in first:
|
||||
|
||||
```sh
|
||||
sudo hc set add admins 10.0.10.0/24
|
||||
sudo hc set add admins 192.168.1.0/24
|
||||
sudo hc rule add input --from admins --service ssh
|
||||
sudo hc rule add input --from admins --service tcp/8070 --comment "Housecarl"
|
||||
sudo hc zone add wan --interface ppp0 --external
|
||||
sudo hc zone add lan --interface enp2s0
|
||||
sudo hc zone add lan --interface eth1
|
||||
sudo hc rule add input --in lan --service dns --service dhcp
|
||||
sudo hc rule add forward --in lan --out wan
|
||||
sudo hc snat add --out wan
|
||||
sudo hc settings set --builtin mss_clamp=on --own-prefix 2001:db8:7f3d::/48
|
||||
sudo hc settings set --builtin mss_clamp=on --own-prefix 2001:db8::/48
|
||||
sudo hc status
|
||||
```
|
||||
|
||||
|
||||
+7
-7
@@ -8,9 +8,9 @@ One object does what takes three rules by hand (the DNAT, the forward
|
||||
accept, and the hairpin masquerade).
|
||||
|
||||
```sh
|
||||
hc forward add 8096 --to 10.0.10.57:443 --public 203.0.113.7 --hairpin --name jellyfin
|
||||
hc forward add 7777-7780 --proto tcp,udp --to 10.0.10.76 --public 203.0.113.7 --hairpin --name terraria
|
||||
hc forward add 25 --to 10.0.10.25 --from mail-relays --name smtp
|
||||
hc forward add 8443 --to 192.168.1.10:443 --public 203.0.113.10 --hairpin --name web
|
||||
hc forward add 27015-27030 --proto tcp,udp --to 192.168.1.20 --public 203.0.113.10 --hairpin --name game
|
||||
hc forward add 25 --to 192.168.1.25 --from mail-relays --name smtp
|
||||
hc forward list
|
||||
```
|
||||
|
||||
@@ -44,10 +44,10 @@ IPv6 needs no translation: inside hosts have public addresses. `--to6`
|
||||
opens the same port (or the `--to` port, if given) to an IPv6 host:
|
||||
|
||||
```sh
|
||||
hc forward add 8096 --to 10.0.10.57:443 --to6 2001:db8:7f3d:10::57 --public 203.0.113.7 --name jellyfin
|
||||
hc forward add 8443 --to 192.168.1.10:443 --to6 2001:db8:1::10 --public 203.0.113.10 --name web
|
||||
```
|
||||
|
||||
IPv6 clients connect to `[2001:db8:7f3d:10::57]:443` directly. A forward
|
||||
IPv6 clients connect to `[2001:db8:1::10]:443` directly. A forward
|
||||
can have only `--to6`, to open an IPv6 port alone.
|
||||
|
||||
## Source NAT
|
||||
@@ -60,7 +60,7 @@ service; the first that matches decides.
|
||||
```sh
|
||||
hc snat add --out wan --from trusted --comment "the LAN to the internet"
|
||||
hc snat add --out wg0 --not-out --from wg-clients
|
||||
hc snat add --out wan --from fd00:abab::/48 --family ipv6 --snat-to 2001:db8:7f3d::1
|
||||
hc snat add --out wan --from fd00:1::/48 --family ipv6 --snat-to 2001:db8::1
|
||||
```
|
||||
|
||||
Source NAT is IPv4 only unless `--family ipv6` says otherwise: with a
|
||||
@@ -73,7 +73,7 @@ A redirect sends matching traffic to a port on the firewall itself, for
|
||||
example to make every client use the firewall's DNS server:
|
||||
|
||||
```sh
|
||||
hc redirect add --in lan --service dns --not-to 10.0.10.5 --comment "force DNS"
|
||||
hc redirect add --in lan --service dns --not-to 192.168.1.5 --comment "force DNS"
|
||||
hc redirect add --in lan --service http --to-port 3128
|
||||
```
|
||||
|
||||
|
||||
+179
-14
@@ -17,10 +17,36 @@ housecarld logs each change it applies, rollbacks, and what it couldn't
|
||||
apply. When it starts, it applies the configuration again (rolling back
|
||||
first what was on probation when it stopped).
|
||||
|
||||
## Reviewing before applying
|
||||
|
||||
In the web UI, edits are collected as pending changes (the bar under the
|
||||
header), and Changes shows them with the diff of the ruleset; Apply puts
|
||||
them in force together. The switch in the header (Review edits / Live
|
||||
edits) applies each edit at once instead. With hc, see
|
||||
[cli.md](cli.md#changesets).
|
||||
|
||||
## Changes on probation
|
||||
|
||||
Changes made over the network roll back unless confirmed
|
||||
([design.md](design.md#not-locking-yourself-out)):
|
||||
([design.md](design.md#not-locking-yourself-out)). The web UI and `hc`
|
||||
confirm their own changes once they reach Housecarl again over a new
|
||||
connection, so normally nothing waits:
|
||||
|
||||
```
|
||||
$ hc rule add input --from admins --service ssh
|
||||
created rule input #31
|
||||
applied and confirmed (history 12): this client reached Housecarl again over a new connection
|
||||
```
|
||||
|
||||
A change that cuts the client off can't be confirmed that way:
|
||||
|
||||
```
|
||||
applied on probation (history 13), but couldn't confirm it: … context deadline exceeded
|
||||
It is rolled back within 1m50s unless confirmed ('hc confirm')
|
||||
```
|
||||
|
||||
By hand (with `--no-confirm`, or with Settings → "a person confirms each
|
||||
change"):
|
||||
|
||||
```sh
|
||||
hc status # Probation: 2 changes roll back in 97s unless confirmed
|
||||
@@ -45,19 +71,79 @@ sudo hc history undo 42
|
||||
|
||||
A confirmed ruleset (or one changed through the socket) is saved to
|
||||
`/var/lib/housecarl/ruleset.nft`, which `housecarl-ruleset.service`
|
||||
loads before the network comes up. To see what would load:
|
||||
loads before the network comes up, so before PPPoE, WireGuard or DHCP
|
||||
bring up their interfaces. To see what would load:
|
||||
|
||||
```sh
|
||||
sudo cat /var/lib/housecarl/ruleset.nft
|
||||
sudo nft -c -f /var/lib/housecarl/ruleset.nft # check it
|
||||
```
|
||||
|
||||
Dynamic sets start empty after a reboot.
|
||||
Dynamic sets start empty after a reboot. The saved ruleset has no
|
||||
flowtable, since its devices may not exist yet: housecarld adds it when
|
||||
it starts, and whenever a flowtable device appears
|
||||
([flowtable.md](flowtable.md#devices-that-come-and-go)).
|
||||
|
||||
Keep Debian's `nftables.service` disabled, or `/etc/nftables.conf`
|
||||
without `flush ruleset`: otherwise it deletes Housecarl's table at
|
||||
boot. `housecarl-ruleset.service` runs after it, which covers the
|
||||
ordinary case.
|
||||
### If the ruleset fails to load at boot
|
||||
|
||||
Should `ruleset.nft` fail to load (a damaged file, an nftables or kernel
|
||||
upgrade that refuses something in it), the machine would otherwise boot
|
||||
with no firewall at all until housecarld starts. Instead,
|
||||
`housecarl-ruleset.service` loads a **fallback** that housecarld saves
|
||||
beside it, `/var/lib/housecarl/ruleset-fallback.nft`, from the same
|
||||
confirmed configuration:
|
||||
|
||||
* **Nothing new gets in from the external zones' interfaces**, to the
|
||||
firewall or through it: no port forwards, no SSH or web UI from the
|
||||
internet.
|
||||
* What the firewall and the inside need to keep working is let in:
|
||||
established connections, the ICMPv6 IPv6 needs (neighbour discovery,
|
||||
router advertisements, packet too big), and DHCPv6 replies from
|
||||
link-local addresses.
|
||||
* **Everything else is let through**: the LAN, WireGuard and the
|
||||
firewall itself are reachable from the inside, so you can log in and
|
||||
fix it.
|
||||
* IPv4 leaving an external zone is masqueraded (when any source NAT is
|
||||
configured), so the inside still reaches the internet. MSS clamping
|
||||
stays, if it is on.
|
||||
|
||||
PPPoE itself isn't affected: its discovery and session are Ethernet
|
||||
frames, and the PPP link's negotiation happens before any IP, neither
|
||||
of which an `inet` table sees.
|
||||
|
||||
The fallback only knows which interfaces face the internet from the
|
||||
zones marked **external**. With none, it lets nothing but loopback
|
||||
through, and the warning `no_external_zone` says so.
|
||||
|
||||
When the fallback loads, the unit is marked failed, and the journal says
|
||||
why (as a critical message):
|
||||
|
||||
```sh
|
||||
systemctl status housecarl-ruleset
|
||||
journalctl -b -u housecarl-ruleset
|
||||
```
|
||||
|
||||
housecarld puts the real ruleset back when it starts, and saves a fresh
|
||||
`ruleset.nft`. If it can't apply it either, the fallback stays in force:
|
||||
`hc status` and the dashboard say "NOT in force: the boot fallback is in
|
||||
force", and `/healthz` answers 503 ([Monitoring](#monitoring)). Fix
|
||||
what `hc status` or the journal reports, then restart housecarld.
|
||||
|
||||
Keep Debian's `nftables.service` disabled (`systemctl disable
|
||||
nftables`, without `--now`): at boot it loads `/etc/nftables.conf`, and
|
||||
stopping or restarting it runs `nft flush ruleset`, which deletes every
|
||||
table, Housecarl's included. `housecarl-ruleset.service` runs after it
|
||||
at boot, which covers the ordinary case, but not a restart of
|
||||
`nftables.service` later. Housecarl never reads or writes
|
||||
`/etc/nftables.conf`; its ruleset is saved in
|
||||
`/var/lib/housecarl/ruleset.nft`. The dashboard and `hc status` warn
|
||||
while `nftables.service` is enabled.
|
||||
|
||||
Other programs' tables are left alone: fail2ban's (`inet f2b-table`),
|
||||
Docker's, Incus's. Housecarl replaces only its own table, so they keep
|
||||
their rules through every Housecarl change and restart, and nothing
|
||||
needs restarting in a particular order. A packet must get past every
|
||||
table on its hook, so a drop in any of them wins.
|
||||
|
||||
## Hits
|
||||
|
||||
@@ -66,6 +152,15 @@ Every rule, NAT rule, built-in and policy has a counter. `hc rule list`,
|
||||
all, with bytes. They count from housecarld's start, and carry across
|
||||
changes to the ruleset.
|
||||
|
||||
## Quotas
|
||||
|
||||
`hc quotas` (and the rules page) show what each quota used and when it
|
||||
resets; `hc rule reset-quota ID` starts one again. housecarld resets
|
||||
quotas at midnight when their period ends (late, if it was stopped then),
|
||||
and saves their usage every five minutes and when it stops; after a
|
||||
reboot it puts it back. A reboot without a clean stop loses at most the
|
||||
last five minutes' count.
|
||||
|
||||
## The log
|
||||
|
||||
Rules with `--log` (Log in the web UI) and, with `hc settings set
|
||||
@@ -78,8 +173,8 @@ hc log --search 198.51.100.7
|
||||
```
|
||||
|
||||
```
|
||||
10:02:13 hc input #95 drop: wan - 198.51.100.7:53868 -> 203.0.113.7:22 tcp syn
|
||||
10:02:15 hc forward policy drop: wan enp2s0 198.51.100.9:443 -> 10.0.10.20:51432 tcp
|
||||
10:02:13 hc input #95 drop: wan - 198.51.100.7:53868 -> 203.0.113.10:22 tcp syn
|
||||
10:02:15 hc forward policy drop: wan eth1 198.51.100.9:443 -> 192.168.1.20:51432 tcp
|
||||
```
|
||||
|
||||
The packets go to netlink log group 4171, where housecarld reads them;
|
||||
@@ -87,16 +182,62 @@ the last 2000 are kept in memory. They aren't written to the kernel log
|
||||
or the journal. A rule's log isn't rate-limited: on a busy rule, give it
|
||||
a limit, or log only new connections (`--state new`).
|
||||
|
||||
## Connections
|
||||
|
||||
The built-in that accepts established connections lets their packets
|
||||
through without checking the rules again: a rule added later doesn't
|
||||
stop a connection already open. Traffic → Connections (`hc connections`)
|
||||
lists what the kernel tracks, with NAT (`to 192.168.1.10:443`: a port
|
||||
forward; `as 203.0.113.10`: masqueraded) and the time left:
|
||||
|
||||
```sh
|
||||
hc connections --search 192.168.1.50
|
||||
hc connections --search ":443 tcp"
|
||||
hc connections close --search 192.168.1.50
|
||||
```
|
||||
|
||||
Closing a connection deletes its tracking entry: its next packet is
|
||||
checked by the rules as they are now. One they block is stopped; one they
|
||||
allow carries on (TCP connections are picked up again mid-stream).
|
||||
|
||||
To make a block stop open connections by itself, tick "every packet" on
|
||||
the rule (`--every-packet`): it is checked before established connections
|
||||
are accepted, like rules with a quota ([rules.md](rules.md#rules)).
|
||||
|
||||
## Tracing
|
||||
|
||||
Traffic → Trace (`hc trace`) follows packets through the whole ruleset:
|
||||
Housecarl's rules and every other table (Docker's, fail2ban's, an old
|
||||
`inet main`), each rule and policy they meet, and what accepted or
|
||||
dropped them:
|
||||
|
||||
```
|
||||
$ hc trace --address 192.168.1.50 --port 443 --seconds 60
|
||||
14:02:13.120 192.168.1.50:51432 -> 198.51.100.9:443 tcp syn in eth1 out wan
|
||||
inet housecarl forward: built-in: mss-clamp -> continue
|
||||
inet housecarl forward: rule forward #12 (kids at night) -> drop
|
||||
=> DROPPED by rule forward #12 (kids at night)
|
||||
```
|
||||
|
||||
It traces packets from or to the address, on the port (either way), for
|
||||
5 to 600 seconds, at most 200 packets and 20 a second. It works with
|
||||
nftables' own tracing: a table of its own (`inet housecarl-trace`,
|
||||
hooked before everything else) marks the packets, and `nft monitor
|
||||
trace` reports them; the table goes when the trace ends. One trace runs
|
||||
at a time.
|
||||
|
||||
## When something doesn't get through
|
||||
|
||||
1. `hc log -f` with `--log-drops on`: is it dropped by a policy (no rule
|
||||
1. Trace it (above): it names the rule or policy that dropped it, in
|
||||
whichever table.
|
||||
2. `hc log -f` with `--log-drops on`: is it dropped by a policy (no rule
|
||||
accepted it), or by a rule (its number is in the line)?
|
||||
2. `hc rule list forward` (or `input`): hits on the rule you expect?
|
||||
3. `hc ruleset`: the rule as nftables sees it, with the zones' interfaces
|
||||
3. `hc rule list forward` (or `input`): hits on the rule you expect?
|
||||
4. `hc ruleset`: the rule as nftables sees it, with the zones' interfaces
|
||||
and the sets' families.
|
||||
4. `hc status`: other tables. A Docker or Incus table, or an old one,
|
||||
5. `hc status`: other tables. A Docker or Incus table, or an old one,
|
||||
can drop what Housecarl accepts.
|
||||
5. Port forwards: `hc forward list` hits, and the inside host's own
|
||||
6. Port forwards: `hc forward list` hits, and the inside host's own
|
||||
firewall and default route (answers must go back through Housecarl).
|
||||
|
||||
## Backups
|
||||
@@ -132,6 +273,30 @@ database is kept next to it as `housecarl.db.before-restore-<time>`.
|
||||
database when it starts, and an older one refuses a database a newer one
|
||||
has upgraded: restore a backup to go back.
|
||||
|
||||
## Monitoring
|
||||
|
||||
`GET /healthz` needs no login, and answers within a few seconds of any
|
||||
change (it reads the ruleset at most every five seconds, however often
|
||||
it is asked):
|
||||
|
||||
```sh
|
||||
$ curl -s https://firewall.example.net:8070/healthz
|
||||
{"ok":true,"version":"0.1.0","firewall":"applied","probation":0,"warnings":[]}
|
||||
```
|
||||
|
||||
| Field | |
|
||||
|---|---|
|
||||
| `ok` | `true` with HTTP 200 when the firewall is as it should be; `false` with HTTP **503** otherwise |
|
||||
| `firewall` | `applied` (Housecarl's ruleset is in force), `unconfigured` (nothing configured yet, so nothing to apply), `fallback` (the [boot fallback](#if-the-ruleset-fails-to-load-at-boot) is in force), `missing` (no Housecarl table: flushed, or never applied) |
|
||||
| `probation` | changes waiting to be confirmed, rolled back unless they are |
|
||||
| `warnings` | codes of problems with the machine, as `hc status` shows them (`nftables_service`) |
|
||||
|
||||
For Checkmk, an HTTP check on the URL that expects status 200 (and,
|
||||
if you like, the string `"firewall":"applied"`) covers housecarld being
|
||||
down, the fallback and a missing ruleset. Reach it from where the input
|
||||
rules let the monitor in (a rule for the Checkmk server's address and
|
||||
port 8070).
|
||||
|
||||
## Checks
|
||||
|
||||
`housecarld check` validates the configuration in the database against
|
||||
|
||||
+4
-3
@@ -7,9 +7,9 @@ else gets **grants**, `resource:access`:
|
||||
|
||||
| Resource | read | write |
|
||||
|---|---|---|
|
||||
| `config` | settings, zones, sets, services, rules, NAT, the ruleset, history, interfaces, probation (the web UI shows the forms locked) | change them, undo, import, confirm and roll back |
|
||||
| `config` | settings, zones, sets, services, rules, NAT, the ruleset, history, interfaces, probation, changesets (the web UI shows the forms locked) | change them, stage and apply changesets (your own, or anyone's for administrators), undo, import, confirm and roll back; close connections; start and stop traces |
|
||||
| `sets` | dynamic sets' entries | add and remove entries |
|
||||
| `log` | hit counters and the log | – |
|
||||
| `log` | hit counters, the log, connections, traces | – |
|
||||
| `audit` | the audit log | – |
|
||||
| `*` | all of the above | |
|
||||
|
||||
@@ -81,6 +81,7 @@ on probation like any other.
|
||||
## The audit log
|
||||
|
||||
Logins (and failed ones), users, tokens, every configuration change (with
|
||||
its history entry), confirmations and rollbacks, imports, takeovers and
|
||||
its history entry), changesets created, applied and discarded,
|
||||
confirmations and rollbacks, imports, takeovers and
|
||||
dynamic set entries are recorded with who, from where and how: `session`,
|
||||
`token` or `socket`.
|
||||
+150
-17
@@ -3,12 +3,12 @@
|
||||
## Zones
|
||||
|
||||
A zone names one or more interfaces, so rules can say "from the LAN"
|
||||
rather than `enp2s0`. An interface belongs to one zone at most. A trailing
|
||||
rather than `eth1`. An interface belongs to one zone at most. A trailing
|
||||
`*` matches many: `incusbr*`.
|
||||
|
||||
```sh
|
||||
hc zone add wan --interface wan --external
|
||||
hc zone add lan --interface enp2s0 --interface 'enp2s0.*'
|
||||
hc zone add lan --interface eth1 --interface 'eth1.*'
|
||||
hc zone add wg --interface wg0
|
||||
```
|
||||
|
||||
@@ -21,29 +21,58 @@ A zone may name an interface that doesn't exist yet (a WireGuard
|
||||
interface that comes up later): its rules match once it appears, and
|
||||
`hc status` warns until then.
|
||||
|
||||
**VLANs.** A VLAN sub-interface is an interface like any other, by its
|
||||
name: `enp2s0.20` (or whatever your network configuration calls it,
|
||||
`vlan20`, `br0.20`). Tagged traffic arrives on the sub-interface, and
|
||||
untagged traffic on the parent, so `enp2s0` alone doesn't cover
|
||||
`enp2s0.20`. Give each VLAN its own zone to keep them apart:
|
||||
|
||||
```sh
|
||||
hc zone add lan --interface enp2s0 # untagged
|
||||
hc zone add iot --interface enp2s0.20
|
||||
hc zone add guest --interface enp2s0.30
|
||||
hc rule add forward --in iot --out wan
|
||||
```
|
||||
|
||||
or one zone for the parent and all of its VLANs, when they should be
|
||||
treated alike: `--interface enp2s0 --interface 'enp2s0.*'`. Interface
|
||||
names are at most 15 characters (the kernel's limit). The flowtable
|
||||
takes VLAN sub-interfaces too, by exact name
|
||||
([flowtable.md](flowtable.md)).
|
||||
|
||||
**Renaming** a zone (`hc zone update dhcp --name lan`, or Edit on the
|
||||
Zones page) changes every rule and NAT rule that names it, in the same
|
||||
change: the history shows them together, and undoing it puts them all
|
||||
back. Sets and services rename the same way, except dynamic sets: their
|
||||
entries, and the API tokens and programs that fill them, go by their
|
||||
name.
|
||||
|
||||
## Address sets
|
||||
|
||||
A set is a named list of IPv4 and IPv6 addresses, prefixes and ranges,
|
||||
mixed:
|
||||
|
||||
```sh
|
||||
hc set add trusted 10.0.10.0/24 10.20.30.0/24 2001:db8:7f3d:10::/64
|
||||
hc set add dns-servers 10.0.10.5 2001:db8:7f3d:10::5 10.0.10.250-10.0.10.254
|
||||
hc set add trusted 192.168.1.0/24 192.168.2.0/24 2001:db8:1::/64
|
||||
hc set add dns-servers 192.168.1.5 2001:db8:1::5 192.168.1.250-192.168.1.254
|
||||
```
|
||||
|
||||
In the ruleset each becomes two nftables sets, `trusted_v4` and
|
||||
`trusted_v6` (one if the set has only one family). A rule that names it
|
||||
becomes one nftables rule per family. Rules can also give addresses
|
||||
directly (`--from 10.0.10.3`); a set is better when several rules share
|
||||
directly (`--from 192.168.1.3`); a set is better when several rules share
|
||||
them.
|
||||
|
||||
A **dynamic** set's entries are added at run time, each with a timeout,
|
||||
and nftables removes them when it runs out, even with housecarld stopped:
|
||||
and nftables removes them when it runs out, even with housecarld stopped.
|
||||
It has no configured entries (validation refuses them); its timeout is
|
||||
the default for entries added without one, and without a timeout they
|
||||
stay until removed:
|
||||
|
||||
```sh
|
||||
hc set add guests --dynamic --timeout 24h
|
||||
hc set entries add guests 10.0.30.17 # for 24 hours
|
||||
hc set entries add guests 2001:db8:7f3d:30::17 --timeout 2h
|
||||
hc set entries add guests 2001:db8:3::17 --timeout 2h
|
||||
hc set entries guests # with the time left
|
||||
hc set entries remove guests 10.0.30.17
|
||||
```
|
||||
@@ -60,7 +89,7 @@ A service names ports of a protocol (`tcp`, `udp`, `sctp`) and ICMP
|
||||
types:
|
||||
|
||||
```sh
|
||||
hc service add jellyfin tcp/8096 tcp/8920
|
||||
hc service add web tcp/8080 tcp/8443
|
||||
hc service add games udp/27015-27030 tcp/27015
|
||||
hc service add pmtu icmp/destination-unreachable icmpv6/packet-too-big
|
||||
```
|
||||
@@ -108,13 +137,18 @@ A rule matches on what it gives, and all of it:
|
||||
| `--state` | connection states: `new`, `established`, `related`, `invalid`, `untracked` |
|
||||
| `--status` | connection status: `dnat` (port forwarded), `snat` |
|
||||
| `--days`, `--hours` | a time window: `--days Monday,Friday --hours 08:00-17:30`; `22:00-06:00` passes midnight. In the firewall's time zone, as the kernel knows it |
|
||||
| `--limit` | a rate: `10/minute`, with `,burst=N`, `,over` (match beyond the rate, to drop a flood) and `,per-source` (a rate for each source address) |
|
||||
| `--limit` | a rate: `10/minute`, or bytes: `1mbytes/second`; with `,burst=N`, `,over` (match beyond the rate) and `,per-source` or `,per-destination` ([Bandwidth and quotas](#bandwidth-and-quotas)) |
|
||||
| `--quota` | an allowance of traffic: `50gbytes,over,per-source,monthly=15` ([Bandwidth and quotas](#bandwidth-and-quotas)) |
|
||||
| `--conn-limit N` | a source with more than N connections |
|
||||
|
||||
The action is `accept`, `drop`, `reject` (an ICMP error, or with
|
||||
`--reject-with tcp-reset` a TCP reset) or `log` (log and go on).
|
||||
`--log` logs the packets of any rule ([operations.md](operations.md#the-log)).
|
||||
`--disabled` (or `hc rule disable ID`) keeps a rule without using it.
|
||||
`--every-packet` ("every packet" in the web UI) checks a drop or reject
|
||||
against every packet, before established connections are accepted, so it
|
||||
also stops connections already open, not just new ones; it can't accept
|
||||
([operations.md](operations.md#connections)).
|
||||
|
||||
Examples:
|
||||
|
||||
@@ -131,6 +165,59 @@ hc rule add forward --in wan --status dnat
|
||||
hc rule add forward --in wan -a drop
|
||||
```
|
||||
|
||||
### Bandwidth and quotas
|
||||
|
||||
A **byte rate** limits bandwidth: `--limit 20mbytes/second,over,per-source
|
||||
-a drop` drops what a device sends beyond 20 MB/s. Units are `bytes`,
|
||||
`kbytes` and `mbytes` (`kb`, `mb` for short); the burst is in the same
|
||||
unit. nftables can only hold byte rates per second (up to 4 GB/s) or per
|
||||
minute (up to 293 MB/minute); validation says so for larger ones. This is
|
||||
policing: traffic beyond the rate is dropped, not queued, so TCP slows to
|
||||
fit, and UDP loses packets. Shaping (queues, fair shares) is `tc`'s work,
|
||||
not a firewall's.
|
||||
|
||||
A **quota** is an allowance of traffic. `over` matches once it is used up,
|
||||
to drop what comes after:
|
||||
|
||||
```sh
|
||||
# Each device on the LAN: 50 GB a month, from the 15th (a billing day)
|
||||
hc rule add forward --in lan --out wan --quota 50gbytes,over,per-source,monthly=15 -a drop
|
||||
# The whole guest network: 10 GB a day together
|
||||
hc rule add forward --in guest --out wan --quota 10gbytes,over,daily -a drop
|
||||
hc quotas # what each used, and the next reset
|
||||
hc rule reset-quota 31 # start again now
|
||||
```
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| `per-source`, `per-destination` | each address its own allowance; without, one allowance for everything the rule matches |
|
||||
| `daily`, `weekly`, `monthly=DAY` | when it starts again: at midnight; weekly on Mondays; monthly on that day (the last day of a shorter month). Without one, only by hand |
|
||||
|
||||
nftables counts a quota but never resets it, and forgets it when its
|
||||
table is replaced: Housecarl carries what each quota used across every
|
||||
change and restart, saves it every five minutes (and when housecarld
|
||||
stops) for after a reboot, and resets quotas when their period ends.
|
||||
A per-address quota counts an address: a device that gets a new one
|
||||
starts again. With Steward's reservations IPv4 addresses stay; IPv6 is
|
||||
grouped by /64 (below).
|
||||
|
||||
**Rules with a quota or a byte rate count every packet.** The built-in
|
||||
that accepts established connections comes before your rules, so a rule
|
||||
there only sees each connection's first packet. Rules that count traffic
|
||||
are checked before it instead, in their order (the web UI lists them
|
||||
first, marked "every packet"). So they can drop, reject or log, but not
|
||||
accept: an accept there would let packets past every later rule. Let the
|
||||
traffic in with a rule of its own. The flowtable's fast path skips the
|
||||
forward chain for established connections, so with it on, forward
|
||||
quotas and byte rates only count first packets (a warning says so;
|
||||
[flowtable.md](flowtable.md)).
|
||||
|
||||
**Per-address limits group addresses**: each IPv4 address on its own, and
|
||||
IPv6 addresses by /64, since one device uses several (privacy addresses
|
||||
change). `--group-v4 24` counts each /24 together; `--group-v6 128` each
|
||||
IPv6 address alone. This applies to per-address rates, connection limits
|
||||
and quotas.
|
||||
|
||||
### Raw rules
|
||||
|
||||
For what Housecarl can't say, a rule can be one nftables rule written by
|
||||
@@ -140,8 +227,11 @@ hand, placed like any other:
|
||||
hc rule add forward --raw 'iifname != "wan" ct state invalid ip daddr @trusted_v4 accept'
|
||||
```
|
||||
|
||||
It is checked by nftables with the rest of the ruleset, and can use the
|
||||
sets' nftables names (`@trusted_v4`). The importer makes raw rules from
|
||||
It is used exactly as written: its text replaces the rule's zones,
|
||||
addresses, services, action, limits and log, so a raw rule with any of
|
||||
those set is refused (`raw_conflict`); write them in the text. It is
|
||||
checked by nftables with the rest of the ruleset, and can use the sets'
|
||||
nftables names (`@trusted_v4`). The importer makes raw rules from
|
||||
what it can't translate. It goes into the ruleset as written: add
|
||||
`counter` to it to count its packets.
|
||||
|
||||
@@ -162,15 +252,58 @@ UI), and `hc ruleset` shows them:
|
||||
| `antispoof` | on | early | drop sources that can't come from the internet, on external zones: the antispoof lists (private, loopback, documentation and multicast ranges) and your own prefixes |
|
||||
| `mss_clamp` | off | forward | clamp TCP's MSS to the route's MTU on external zones: needed with PPPoE |
|
||||
|
||||
**Your prefixes** (`--own-prefix 2001:db8:7f3d::/48`) are your delegated
|
||||
**Your prefixes** (`--own-prefix 2001:db8::/48`) are your delegated
|
||||
IPv6 prefix and public addresses: nothing from outside should come from
|
||||
them. The antispoof lists can be changed with `--antispoof-v4` and
|
||||
`--antispoof-v6`.
|
||||
|
||||
The **flowtable** (`--flowtable wan --flowtable enp2s0`) lets established
|
||||
The **flowtable** (`--flowtable wan --flowtable eth1`) lets established
|
||||
TCP and UDP connections forwarded between those devices skip the rules,
|
||||
which helps on a slow CPU. It needs exact device names. Counters of
|
||||
forward rules then only count each connection's first packets.
|
||||
which helps when the CPU can't keep up with the line. It needs exact
|
||||
device names, and costs quotas, byte rates and traffic per device in the
|
||||
forward chain their accuracy: see [flowtable.md](flowtable.md) for when
|
||||
to use it, and when not.
|
||||
|
||||
## The zone matrix
|
||||
|
||||
The Matrix page (`hc matrix`) sums up what the rules do to **new
|
||||
connections** between zones: a row per zone the traffic comes from, a
|
||||
column per zone it goes to, and the firewall itself (to it: the input
|
||||
chain; from it: output; zone to zone: forward).
|
||||
|
||||
```
|
||||
$ hc matrix
|
||||
FROM \ TO guest lan wan firewall
|
||||
guest block block block +1 block +3
|
||||
lan block block allow block +4
|
||||
wan block +2 block +2 block +1 block +3
|
||||
firewall allow allow allow -
|
||||
```
|
||||
|
||||
Each cell walks that pair's rules in order, as nftables would, with the
|
||||
built-ins that act on new connections (ping from internal zones,
|
||||
ICMPv6, the DHCPv6 client, antispoofing, port forwards):
|
||||
|
||||
* a rule that applies to the whole pair decides it ("allow", "block");
|
||||
* a rule that applies to part of its traffic (some addresses, services,
|
||||
times, rates, a quota) is an **exception** (+N), and the walk goes on;
|
||||
* what's left falls to the deciding rule or the chain's policy.
|
||||
|
||||
`hc matrix guest wan` (or clicking the cell) lists the exceptions in
|
||||
order:
|
||||
|
||||
```
|
||||
guest → wan (the forward chain), new connections:
|
||||
allowed http, https rule forward #14
|
||||
blocked the rest forward policy
|
||||
```
|
||||
|
||||
A rule whose addresses all lie in networks directly connected to other
|
||||
zones is left out of a pair: traffic between those zones can't have them
|
||||
(`from wg_net` doesn't concern traffic coming in on the LAN). Routed
|
||||
networks, mixed lists and dynamic sets might, so they stay. Replies and
|
||||
established connections aren't shown: the built-in lets them through. In
|
||||
review mode, the matrix shows your pending changes.
|
||||
|
||||
## The ruleset
|
||||
|
||||
@@ -179,8 +312,8 @@ show each rule's hits. The table looks like this (shortened):
|
||||
|
||||
```
|
||||
table inet housecarl {
|
||||
set trusted_v4 { type ipv4_addr; flags interval; elements = { 10.0.10.0/24, 10.20.30.0/24 } }
|
||||
set trusted_v6 { type ipv6_addr; flags interval; elements = { 2001:db8:7f3d:10::/64 } }
|
||||
set trusted_v4 { type ipv4_addr; flags interval; elements = { 192.168.1.0/24, 192.168.2.0/24 } }
|
||||
set trusted_v6 { type ipv6_addr; flags interval; elements = { 2001:db8:1::/64 } }
|
||||
chain early { type filter hook prerouting priority raw; policy accept; ... }
|
||||
chain input { type filter hook input priority filter; policy drop;
|
||||
iifname "lo" accept comment "hc:b-loopback"
|
||||
|
||||
+140
@@ -0,0 +1,140 @@
|
||||
# Statistics
|
||||
|
||||
Housecarl keeps what the firewall did over time:
|
||||
|
||||
- traffic per interface and zone, received and sent;
|
||||
- packets dropped, by chain, and when drops happen (a weekday × hour
|
||||
heatmap);
|
||||
- hits per rule, built-in and policy, and the rules that matched nothing;
|
||||
- the sources blocked most and what they tried (from the log);
|
||||
- traffic per inside device (a setting, below);
|
||||
- connections tracked, against the kernel's limit;
|
||||
- what each quota counted;
|
||||
- housecarld's and the machine's memory, CPU and load.
|
||||
|
||||
All of it is in the web UI (**Traffic → Statistics**, and two charts on
|
||||
the dashboard), the CLI (`hc stats`) and the API (`/api/v1/stats/...`),
|
||||
for anyone with `log:read`.
|
||||
|
||||
Statistics are **on by default**. They cost nothing on the packets'
|
||||
path: housecarld reads counters the kernel keeps anyway, once a minute.
|
||||
Only traffic per device adds to the ruleset, so it is a setting you turn
|
||||
on. The figures are kept as aggregates in a database of their own,
|
||||
`/var/lib/housecarl/stats.db`, separate from the configuration. Deleting
|
||||
it is always safe (stop housecarl first), and backups leave it out.
|
||||
|
||||
## Settings
|
||||
|
||||
In `/etc/housecarl/housecarl.yaml` (restart housecarl after a change):
|
||||
|
||||
```yaml
|
||||
stats:
|
||||
enabled: true
|
||||
database: /var/lib/housecarl/stats.db
|
||||
retention:
|
||||
minute: 7d # per-minute figures
|
||||
hour: 90d # hourly figures and top lists
|
||||
day: 730d # daily figures and top lists
|
||||
top_n: 100 # entries kept per top list, per hour and day
|
||||
```
|
||||
|
||||
Old data is removed once an hour. A range picks the finest data that
|
||||
covers it: per minute up to 2 days, hourly up to 90 days, then daily.
|
||||
|
||||
## Traffic per device
|
||||
|
||||
```sh
|
||||
hc settings set --stats-per-device on
|
||||
```
|
||||
|
||||
(or **Settings → Statistics** in the web UI). This is a firewall setting,
|
||||
so it goes through the usual review and probation. Housecarl then adds
|
||||
four sets to its table, and four rules at the top of the forward chain
|
||||
(shown as the built-in `stats`):
|
||||
|
||||
```
|
||||
iifname != "wan" oifname "wan" update @stats_up_v4 { ip saddr }
|
||||
iifname != "wan" oifname "wan" update @stats_up_v6 { ip6 saddr and ffff:ffff:ffff:ffff:: }
|
||||
iifname "wan" oifname != "wan" update @stats_down_v4 { ip daddr }
|
||||
iifname "wan" oifname != "wan" update @stats_down_v6 { ip6 daddr and ffff:ffff:ffff:ffff:: }
|
||||
```
|
||||
|
||||
Each inside address talking to an external zone gets a counter: what it
|
||||
sent out, and what came back to it. IPv6 addresses are counted per /64,
|
||||
since one device uses many addresses. Counting is done in the kernel, a
|
||||
set lookup per forwarded packet. An address quiet for an hour drops out
|
||||
of the sets, and a reload keeps the counts.
|
||||
|
||||
With the flowtable on, established connections skip the forward chain,
|
||||
so only their first packets are counted (warning `stats_flowtable`). Set
|
||||
names starting with `stats_` (and `meter_`, `conn_`, `quota_`) are
|
||||
Housecarl's own and can't be used for your sets.
|
||||
|
||||
## What is collected
|
||||
|
||||
Every minute housecarld samples its sources and stores what changed:
|
||||
|
||||
| Figure | Source |
|
||||
|---|---|
|
||||
| `rx:IF`, `tx:IF` (bytes), `rxp:IF`, `txp:IF` (packets) | the interfaces' counters (`/sys/class/net`); the loopback is left out |
|
||||
| `zrx:ZONE`, `ztx:ZONE` | the same, summed per zone: received from and sent to the zone's interfaces |
|
||||
| `hits:MARK`, `hitb:MARK` | each rule's (`r12`), built-in's (`b-established`), port forward's (`f3`) and policy's (`p-input`) counter, in packets and bytes |
|
||||
| `drop:CHAIN`, `drops` | the packets counted by what drops them: rules that drop or reject, the drop policies, antispoofing (`early`) and invalid packets (`invalid`) |
|
||||
| `quota:r12` | bytes counted against a rule's quota |
|
||||
| `g:conntrack`, `g:conntrack_max` | connections tracked, and the limit (`nf_conntrack_max`) |
|
||||
| `g:rss`, `g:cpu` | housecarld's memory (bytes) and CPU (tenths of a percent of one CPU) |
|
||||
| `g:mem`, `g:load` | the host's memory in use, and its 1-minute load average × 100 |
|
||||
|
||||
`g:` figures are gauges: the API averages them over a bucket.
|
||||
|
||||
Top lists, per hour and per day, the `top_n` largest:
|
||||
|
||||
| List | |
|
||||
|---|---|
|
||||
| `blocked_src` | sources of logged drops |
|
||||
| `blocked_port` | what they tried: `tcp/22`, `udp/53`, `icmp/echo-request` |
|
||||
| `device_up`, `device_down` | bytes per inside address, sent and received (traffic per device) |
|
||||
| `quota_r12` | a per-address quota's use, per address |
|
||||
|
||||
The blocked lists only see what is **logged**: the input and forward
|
||||
policies' drops with `hc settings set --log-drops on`, and rules with
|
||||
logging. The drop counts above see everything.
|
||||
|
||||
A counter that goes down (an interface that came back, a quota reset)
|
||||
counts again from zero. What happens while housecarld is stopped isn't
|
||||
counted.
|
||||
|
||||
## The CLI
|
||||
|
||||
```sh
|
||||
hc stats --since 7d # summary: traffic per zone, drops, busiest rules, top lists
|
||||
hc stats status # collecting? since when, the database, figures now
|
||||
hc stats traffic --since 24h # per zone over time (--interfaces: per interface)
|
||||
hc stats top devices --since 30d # devices by traffic (blocked, tried, quota RULE-ID)
|
||||
hc stats rules --unused # rules that matched nothing in 30 days
|
||||
hc stats connections --since 24h
|
||||
```
|
||||
|
||||
Ranges are written `90m`, `24h`, `7d` or `4w`. `-o json` gives the API's
|
||||
answer.
|
||||
|
||||
## The API
|
||||
|
||||
| Endpoint | |
|
||||
|---|---|
|
||||
| `GET /stats/status` | `enabled`, `database`, `size`, `first_data`, `last_sample`, `sample_error`, `per_device`, `retention`, `latest` (the newest gauges) |
|
||||
| `GET /stats/summary?since=24h&limit=10` | `zones` and `interfaces` (`name`, `received`, `sent`), `drops`, `drops_by_chain`, `rules` (busiest, named), `peak_connections`, `conntrack_max`, `blocked_sources`, `blocked_ports`, `devices_up`, `devices_down` |
|
||||
| `GET /stats/timeline?keys=zrx:*,drops&since=24h&step=auto` | `step`, `times` (bucket starts), `series` (`key`, `values`); `GROUP:*` means every key of a group; `step` is `auto`, `1m`, `5m`, `15m`, `1h`, `6h` or `1d`; a gauge without a sample is -1 |
|
||||
| `GET /stats/top?kind=device_down&since=7d&limit=20` | a top list (`key`, `count`) |
|
||||
| `GET /stats/hits?since=30d` | packets per rule, built-in and policy (`key`, `count`, `name`), with the enabled rules that counted none |
|
||||
| `GET /stats/heatmap?key=drops&since=28d` | `avg[weekday][hour]` (Monday first, local time; -1 where there is no data) |
|
||||
|
||||
With statistics off, `/stats/status` says `enabled: false` and the others
|
||||
answer 503.
|
||||
|
||||
## Warnings
|
||||
|
||||
When the connection tracking table is over 80% full, the status shows
|
||||
the warning `conntrack_full`: once it is full, the kernel drops new
|
||||
connections. Raise `net.netfilter.nf_conntrack_max`, or find what opens
|
||||
so many on the Connections page.
|
||||
+27
-1
@@ -34,6 +34,8 @@ are unique per kind, ignoring case.
|
||||
| `flowtable_device` | error | a flowtable device with a wildcard: name each device |
|
||||
| `flowtable_device_missing` | warning, *host* | a flowtable device doesn't exist; the flowtable is left out until it does |
|
||||
| `probation_range` | error | the probation isn't 10 to 3600 seconds |
|
||||
| `stats_no_external` | warning | traffic per device is on, but no zone is external: it counts what goes to and comes from external zones |
|
||||
| `stats_flowtable` | warning | traffic per device with the flowtable on: its fast path skips the forward chain, so only each connection's first packets are counted |
|
||||
|
||||
## Zones
|
||||
|
||||
@@ -43,14 +45,16 @@ are unique per kind, ignoring case.
|
||||
| `zone_interface` | error | not an interface name (up to 15 characters; a trailing `*` matches many) |
|
||||
| `zone_interface_taken` | error | the interface is in another zone already |
|
||||
| `zone_interface_missing` | warning, *host* | the interface doesn't exist here (yet); its rules match once it does |
|
||||
| `no_external_zone` | warning | no zone is external: antispoofing, MSS clamping, the DHCPv6 client rule and port forwards without public addresses need one |
|
||||
| `no_external_zone` | warning | no zone is external: antispoofing, MSS clamping, the DHCPv6 client rule and port forwards without public addresses need one, and the boot fallback lets nothing but loopback through without one ([operations.md](operations.md#if-the-ruleset-fails-to-load-at-boot)) |
|
||||
|
||||
## Sets and services
|
||||
|
||||
| Code | | When |
|
||||
|---|---|---|
|
||||
| `set_name_reserved` | error | a set name starting with `meter_`, `conn_`, `quota_` or `stats_`: Housecarl's own sets (rate limits, quotas, statistics) use those |
|
||||
| `set_entry` | error | an entry that isn't an address, prefix (without host bits) or range |
|
||||
| `set_empty` | warning | a set (not dynamic) with no entries: rules using it match nothing |
|
||||
| `set_dynamic_entries` | error | configured entries on a dynamic set: its entries come at run time |
|
||||
| `set_timeout` | error | a timeout on a set that isn't dynamic, or under a second |
|
||||
| `service_empty` | error | a service with no entries |
|
||||
| `service_entry` | error | an entry that isn't `PROTO/PORTS` (`tcp`, `udp`, `sctp`; a port or range) or `icmp/TYPE`, `icmpv6/TYPE` with a type nftables knows |
|
||||
@@ -77,12 +81,23 @@ These apply to rules, source NAT and redirects.
|
||||
| `rule_chain` | error | a chain that isn't `input`, `forward`, `output` or `early` |
|
||||
| `rule_action` | error | an action that isn't `accept`, `drop`, `reject` or `log` |
|
||||
| `raw_rule` | error | a raw rule over several lines, or with `;` |
|
||||
| `raw_conflict` | error | a raw rule with other fields set (zones, addresses, services, limits, log…): it is used as written, so they would be ignored |
|
||||
| `early_conntrack` | error | connection states, limits or a connection limit in the early chain, which runs before connection tracking |
|
||||
| `early_reject` | error | reject in the early chain: drop instead |
|
||||
| `ct_state`, `ct_status` | error | an unknown connection state or status |
|
||||
| `rule_day`, `rule_hours` | error | an unknown day, or hours that aren't `HH:MM`, or only one of them |
|
||||
| `rule_limit` | error | a rate that isn't a number per second, minute, hour or day |
|
||||
| `rule_conn_limit` | error | a connection limit under 1 |
|
||||
| `rule_limit_unit` | error | a rate's unit that isn't packets, bytes, kbytes or mbytes |
|
||||
| `rule_limit_bytes` | error | a byte rate nftables can't hold: over 4 GB a second, 293 MB a minute, 4.9 MB an hour or 208 KB a day (the message gives it per second) |
|
||||
| `rule_limit_key` | error | a limit both per source and per destination |
|
||||
| `rule_quota` | error | a quota under 1 byte |
|
||||
| `rule_quota_key` | error | a quota both per source and per destination |
|
||||
| `rule_quota_reset` | error | a reset that isn't daily, weekly or monthly; a day with another reset than monthly; a day outside 1-31 |
|
||||
| `rule_group` | error | a grouping prefix outside 1-32 (IPv4) or 1-128 (IPv6) |
|
||||
| `rule_traffic_accept` | error | a rule with a quota or a byte rate that accepts: it is checked before established connections are accepted, so it can only drop, reject or log |
|
||||
| `rule_every_packet_accept` | error | a rule checked against every packet (`every_packet`) that accepts: it comes before established connections are accepted, so it can only drop, reject or log |
|
||||
| `flowtable_traffic` | warning | a forward rule checked against every packet (a quota, a byte rate, `every_packet`) while the flowtable is on: it only sees each connection's first packets |
|
||||
| `reject_with` | error | `tcp-reset` on a rule that matches more than TCP, or with an action other than reject |
|
||||
| `log_prefix` | error | a log prefix over 100 characters |
|
||||
|
||||
@@ -106,3 +121,14 @@ These apply to rules, source NAT and redirects.
|
||||
| `snat_everything` | warning | no out zones and no sources: everything leaving the firewall is translated |
|
||||
| `redirect_service` | error | no service, or one without TCP, UDP or SCTP ports |
|
||||
| `redirect_to_port` | error | the local port isn't one port |
|
||||
|
||||
## The machine
|
||||
|
||||
These come from housecarld looking at the machine, not at the
|
||||
configuration: the dashboard and `hc status` show them.
|
||||
|
||||
| Code | | When |
|
||||
|---|---|---|
|
||||
| `nftables_service` | warning, *host* | Debian's `nftables.service` is enabled: at boot it loads `/etc/nftables.conf`, and stopping or restarting it runs `nft flush ruleset`, deleting Housecarl's table too. Disable it with `systemctl disable nftables` (without `--now`) |
|
||||
| `conntrack_full` | warning, *host* | the connection tracking table is over 80% full; once it is full, the kernel drops new connections. Raise `net.netfilter.nf_conntrack_max`, or find what opens so many ([stats.md](stats.md#warnings)) |
|
||||
| `takeover_pending` | warning, *host* | the table you imported from is still in force beside Housecarl's, so a packet must get past both; `hc takeover TABLE` deletes it once Housecarl's rules are right |
|
||||
@@ -3,12 +3,14 @@ module housecarl
|
||||
go 1.27.1
|
||||
|
||||
require (
|
||||
github.com/aymanbagabas/go-udiff v0.4.1
|
||||
github.com/go-acme/lego/v4 v4.35.2
|
||||
github.com/miekg/dns v1.1.73
|
||||
github.com/spf13/cobra v1.10.2
|
||||
github.com/spf13/pflag v1.0.9
|
||||
go.yaml.in/yaml/v3 v3.0.5
|
||||
golang.org/x/crypto v0.57.0
|
||||
golang.org/x/net v0.58.0
|
||||
golang.org/x/sys v0.48.0
|
||||
golang.org/x/term v0.46.0
|
||||
modernc.org/sqlite v1.60.1
|
||||
)
|
||||
@@ -22,8 +24,7 @@ require (
|
||||
github.com/mattn/go-isatty v0.0.24 // indirect
|
||||
github.com/ncruces/go-strftime v1.0.0 // indirect
|
||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
|
||||
github.com/spf13/pflag v1.0.9 // indirect
|
||||
golang.org/x/sys v0.48.0 // indirect
|
||||
golang.org/x/net v0.58.0 // indirect
|
||||
golang.org/x/text v0.42.0 // indirect
|
||||
modernc.org/libc v1.77.1 // indirect
|
||||
modernc.org/mathutil v1.7.1 // indirect
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
github.com/aymanbagabas/go-udiff v0.4.1 h1:OEIrQ8maEeDBXQDoGCbbTTXYJMYRCRO1fnodZ12Gv5o=
|
||||
github.com/aymanbagabas/go-udiff v0.4.1/go.mod h1:0L9PGwj20lrtmEMeyw4WKJ/TMyDtvAoK9bf2u/mNo3w=
|
||||
github.com/cenkalti/backoff/v5 v5.0.3 h1:ZN+IMa753KfX5hd8vVaMixjnqRZ3y8CuJKRKj1xcsSM=
|
||||
github.com/cenkalti/backoff/v5 v5.0.3/go.mod h1:rkhZdG3JZukswDf7f0cwqPNk4K0sa+F97BxZthm/crw=
|
||||
github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
|
||||
|
||||
@@ -26,6 +26,8 @@ type ErrorInfo struct {
|
||||
Problems []validate.Problem `json:"problems,omitempty"`
|
||||
// Result is the preview of a refused change.
|
||||
Result json.RawMessage `json:"result,omitempty"`
|
||||
// Op is the changeset operation (its seq) that couldn't be applied.
|
||||
Op int `json:"op,omitempty"`
|
||||
}
|
||||
|
||||
// Whoami describes the caller.
|
||||
@@ -64,6 +66,34 @@ type MutationResponse struct {
|
||||
// ConfirmBy is set when the change is on probation: unless confirmed
|
||||
// (POST /confirm) by then, it is rolled back.
|
||||
ConfirmBy *time.Time `json:"confirm_by,omitempty"`
|
||||
// ConfirmToken lets the client that made the change confirm it by
|
||||
// itself (POST /confirm {"token"}): from the same address, over a new
|
||||
// connection, which proves it can still reach the API. The answer
|
||||
// closes its connection, so the next request opens a new one.
|
||||
ConfirmToken string `json:"confirm_token,omitempty"`
|
||||
// Diff is the ruleset's unified diff, for a dry run.
|
||||
Diff string `json:"diff,omitempty"`
|
||||
// Staged: the change was added to a changeset (Changeset), not
|
||||
// applied.
|
||||
Staged bool `json:"staged,omitempty"`
|
||||
Changeset *model.Changeset `json:"changeset,omitempty"`
|
||||
}
|
||||
|
||||
// ChangesetInput creates a changeset, or changes its description.
|
||||
type ChangesetInput struct {
|
||||
Description string `json:"description"`
|
||||
}
|
||||
|
||||
// ChangesetPreview is what applying a changeset would do: its changes,
|
||||
// the problems, nftables' check and the ruleset's diff.
|
||||
type ChangesetPreview struct {
|
||||
// Valid: it can be applied as it is.
|
||||
Valid bool `json:"valid"`
|
||||
Changes []model.Change `json:"changes"`
|
||||
Warnings []validate.Problem `json:"warnings,omitempty"`
|
||||
// Error says why it can't be applied (Op: which operation, if one).
|
||||
Error *ErrorInfo `json:"error,omitempty"`
|
||||
Diff string `json:"diff,omitempty"`
|
||||
}
|
||||
|
||||
// HostInterface is one of the server's network interfaces, with the zone
|
||||
@@ -168,6 +198,9 @@ type ImportInput struct {
|
||||
// Replace replaces Housecarl's configuration; without it, the import
|
||||
// is refused when there is one.
|
||||
Replace bool `json:"replace,omitempty"`
|
||||
// Zones names the zones by interface ({"enp2s0": "lan"}); interfaces
|
||||
// given the same name share a zone.
|
||||
Zones map[string]string `json:"zones,omitempty"`
|
||||
}
|
||||
|
||||
// ImportResult reports an import.
|
||||
@@ -274,3 +307,156 @@ type CertificateResponse struct {
|
||||
Source string `json:"source"`
|
||||
Status *CertStatus `json:"status,omitempty"`
|
||||
}
|
||||
|
||||
// QuotaInfo is a rule's quota and what it used.
|
||||
type QuotaInfo struct {
|
||||
RuleID int64 `json:"rule_id"`
|
||||
Rule string `json:"rule"`
|
||||
Quota model.Quota `json:"quota"`
|
||||
// Used is a total quota's bytes; Addresses a per-address quota's,
|
||||
// each ("192.168.1.20", or "2001:db8:1::/64").
|
||||
Used int64 `json:"used"`
|
||||
Addresses []QuotaAddress `json:"addresses,omitempty"`
|
||||
LastReset *time.Time `json:"last_reset,omitempty"`
|
||||
NextReset *time.Time `json:"next_reset,omitempty"`
|
||||
}
|
||||
|
||||
// QuotaAddress is one address's (or prefix's) use of a per-address quota.
|
||||
type QuotaAddress struct {
|
||||
Address string `json:"address"`
|
||||
Used int64 `json:"used"`
|
||||
}
|
||||
|
||||
// ConfirmInput confirms the changes on probation. With Token (a change's
|
||||
// confirm_token), it is the client that made the change confirming it by
|
||||
// itself; without, a person's confirmation.
|
||||
type ConfirmInput struct {
|
||||
Token string `json:"token,omitempty"`
|
||||
}
|
||||
|
||||
// Connection is a connection the kernel tracks (conntrack), as it was
|
||||
// first seen: Source:SourcePort → Destination:DestinationPort. NAT shows
|
||||
// as DNATTo (where it was really sent: a port forward) and SNATTo (the
|
||||
// source it went out with: masquerading).
|
||||
type Connection struct {
|
||||
ID uint32 `json:"id,omitempty"`
|
||||
Family string `json:"family"`
|
||||
Protocol string `json:"protocol"`
|
||||
Source string `json:"source"`
|
||||
SourcePort int `json:"source_port,omitempty"`
|
||||
Destination string `json:"destination"`
|
||||
DestinationPort int `json:"destination_port,omitempty"`
|
||||
ICMPID int `json:"icmp_id,omitempty"`
|
||||
ICMPType int `json:"icmp_type,omitempty"`
|
||||
ICMPCode int `json:"icmp_code,omitempty"`
|
||||
DNATTo string `json:"dnat_to,omitempty"`
|
||||
DNATPort int `json:"dnat_port,omitempty"`
|
||||
SNATTo string `json:"snat_to,omitempty"`
|
||||
// State is TCP's (established, time-wait, …); Assured: traffic went
|
||||
// both ways.
|
||||
State string `json:"state,omitempty"`
|
||||
Assured bool `json:"assured,omitempty"`
|
||||
// Timeout is the seconds left until it is forgotten unless more
|
||||
// traffic comes.
|
||||
Timeout int `json:"timeout"`
|
||||
Mark uint32 `json:"mark,omitempty"`
|
||||
// Packets and Bytes, both ways, when the kernel counts them
|
||||
// (net.netfilter.nf_conntrack_acct).
|
||||
Packets uint64 `json:"packets,omitempty"`
|
||||
Bytes uint64 `json:"bytes,omitempty"`
|
||||
}
|
||||
|
||||
// CloseConnections closes connections: these, or every one Search finds
|
||||
// (as GET /connections?q= does).
|
||||
type CloseConnections struct {
|
||||
Connections []Connection `json:"connections,omitempty"`
|
||||
Search string `json:"search,omitempty"`
|
||||
}
|
||||
|
||||
// TraceInput says what to trace: packets from or to Address (an address
|
||||
// or prefix), on Port (either way), of Protocol, for Seconds (0: 60).
|
||||
type TraceInput struct {
|
||||
Address string `json:"address,omitempty"`
|
||||
Port int `json:"port,omitempty"`
|
||||
Protocol string `json:"protocol,omitempty"`
|
||||
Seconds int `json:"seconds,omitempty"`
|
||||
}
|
||||
|
||||
// TraceStep is one rule or policy a traced packet met, in any table.
|
||||
type TraceStep struct {
|
||||
Table string `json:"table"` // "inet housecarl", "ip nat"…
|
||||
Chain string `json:"chain"`
|
||||
Kind string `json:"kind"` // rule or policy
|
||||
// Rule is the rule as nftables writes it; Mark its Housecarl mark
|
||||
// ("r12", "b-established"), if it is one of Housecarl's.
|
||||
Rule string `json:"rule,omitempty"`
|
||||
Mark string `json:"mark,omitempty"`
|
||||
Verdict string `json:"verdict"` // accept, drop, continue, jump X…
|
||||
// Label names Housecarl's object ("rule input #12 (ssh)").
|
||||
Label string `json:"label,omitempty"`
|
||||
}
|
||||
|
||||
// TracePacket is one traced packet: its headers as it arrived, every
|
||||
// step, and its fate.
|
||||
type TracePacket struct {
|
||||
ID string `json:"id"`
|
||||
Time time.Time `json:"time"`
|
||||
In string `json:"in,omitempty"`
|
||||
Out string `json:"out,omitempty"`
|
||||
Protocol string `json:"protocol,omitempty"`
|
||||
Source string `json:"source,omitempty"`
|
||||
SourcePort int `json:"source_port,omitempty"`
|
||||
Destination string `json:"destination,omitempty"`
|
||||
DestinationPort int `json:"destination_port,omitempty"`
|
||||
TCPFlags string `json:"tcp_flags,omitempty"`
|
||||
ICMPType string `json:"icmp_type,omitempty"`
|
||||
Steps []TraceStep `json:"steps"`
|
||||
// Verdict is accepted or dropped; DecidedBy the step that decided.
|
||||
Verdict string `json:"verdict"`
|
||||
DecidedBy TraceStep `json:"decided_by"`
|
||||
}
|
||||
|
||||
// TraceStatus is the trace running (or the last one).
|
||||
type TraceStatus struct {
|
||||
Running bool `json:"running"`
|
||||
Input TraceInput `json:"input"`
|
||||
Started time.Time `json:"started"`
|
||||
Ends time.Time `json:"ends"`
|
||||
Packets []TracePacket `json:"packets"`
|
||||
Error string `json:"error,omitempty"`
|
||||
}
|
||||
|
||||
// Matrix is what the rules do to new connections between zones:
|
||||
// zone → zone (the forward chain), zone → the firewall (input) and the
|
||||
// firewall → zone (output). "firewall" stands for the machine itself.
|
||||
type Matrix struct {
|
||||
Zones []string `json:"zones"`
|
||||
Cells []MatrixCell `json:"cells"`
|
||||
}
|
||||
|
||||
// MatrixCell is one pair: what happens to new connections from From to
|
||||
// To that no exception covers (Verdict, decided by DecidedBy), and the
|
||||
// exceptions: rules and built-ins that decide for some of them (some
|
||||
// addresses, services, times…), in the order they are checked.
|
||||
type MatrixCell struct {
|
||||
From string `json:"from"`
|
||||
To string `json:"to"`
|
||||
Chain string `json:"chain"`
|
||||
Verdict string `json:"verdict"` // allowed or blocked
|
||||
DecidedBy string `json:"decided_by"`
|
||||
RuleID int64 `json:"rule_id,omitempty"` // the deciding rule
|
||||
Exceptions []MatrixException `json:"exceptions"`
|
||||
}
|
||||
|
||||
// MatrixException is a rule (or built-in) that decides for part of a
|
||||
// pair's traffic.
|
||||
type MatrixException struct {
|
||||
RuleID int64 `json:"rule_id,omitempty"` // 0: a built-in, or port forwards
|
||||
Chain string `json:"chain"`
|
||||
Label string `json:"label"`
|
||||
// Verdict is allowed, blocked, logged (and goes on), or unknown (a raw
|
||||
// rule Housecarl can't read).
|
||||
Verdict string `json:"verdict"`
|
||||
// What it covers: "from trusted · ssh", "Monday 08:00–17:00".
|
||||
What string `json:"what"`
|
||||
}
|
||||
@@ -14,6 +14,7 @@ import (
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
@@ -28,6 +29,29 @@ type Client struct {
|
||||
HTTP *http.Client
|
||||
// DryRun previews changes instead of making them.
|
||||
DryRun bool
|
||||
// Changeset stages configuration changes in that changeset instead
|
||||
// of applying them (X-Housecarl-Changeset).
|
||||
Changeset int64
|
||||
// BeforeStage, if set, runs once before the first change is staged,
|
||||
// and can stop it (the CLI checks HOUSECARL_CHANGESET's changeset).
|
||||
BeforeStage func() error
|
||||
staged bool
|
||||
}
|
||||
|
||||
// Staged reports whether a request is a configuration change that a
|
||||
// changeset takes: objects, settings and undo, and the changes that can't
|
||||
// be staged (import, takeover), which the server then refuses rather than
|
||||
// apply them. Changesets' own calls and dynamic set entries aren't.
|
||||
func Staged(method, path string) bool {
|
||||
if method == http.MethodGet || strings.Contains(path, "/elements") {
|
||||
return false
|
||||
}
|
||||
for _, p := range []string{"/zones", "/sets", "/services", "/rules", "/forwards", "/snats", "/redirects", "/settings", "/history/", "/import/", "/takeover"} {
|
||||
if strings.HasPrefix(path, p) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// Options configure New.
|
||||
@@ -107,6 +131,12 @@ func (c *Client) Do(ctx context.Context, method, path string, query url.Values,
|
||||
}
|
||||
query.Set("dry_run", "1")
|
||||
}
|
||||
stage := c.Changeset != 0 && Staged(method, path)
|
||||
if stage && !c.staged && c.BeforeStage != nil {
|
||||
if err := c.BeforeStage(); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
u := c.BaseURL + api.Prefix + path
|
||||
if len(query) > 0 {
|
||||
u += "?" + query.Encode()
|
||||
@@ -131,6 +161,10 @@ func (c *Client) Do(ctx context.Context, method, path string, query url.Values,
|
||||
if c.Token != "" {
|
||||
req.Header.Set("Authorization", "Bearer "+c.Token)
|
||||
}
|
||||
if stage {
|
||||
req.Header.Set("X-Housecarl-Changeset", strconv.FormatInt(c.Changeset, 10))
|
||||
c.staged = true
|
||||
}
|
||||
resp, err := c.HTTP.Do(req)
|
||||
if err != nil {
|
||||
if c.BaseURL == "http://housecarld" {
|
||||
|
||||
@@ -39,6 +39,25 @@ type Config struct {
|
||||
// Ruleset is where the confirmed ruleset is saved for the boot unit
|
||||
// (housecarl-ruleset.service loads it before the network comes up).
|
||||
Ruleset string `yaml:"ruleset"`
|
||||
// Stats keeps traffic, hits and drops over time (docs/stats.md).
|
||||
Stats Stats `yaml:"stats"`
|
||||
}
|
||||
|
||||
// Stats configures statistics: a database of their own, aggregates
|
||||
// only, pruned by age.
|
||||
type Stats struct {
|
||||
Enabled bool `yaml:"enabled"`
|
||||
Database string `yaml:"database"`
|
||||
Retention StatsRetention `yaml:"retention"`
|
||||
// TopN is how many entries the top lists keep per hour and day.
|
||||
TopN int `yaml:"top_n"`
|
||||
}
|
||||
|
||||
// StatsRetention is how long each resolution is kept.
|
||||
type StatsRetention struct {
|
||||
Minute Duration `yaml:"minute"`
|
||||
Hour Duration `yaml:"hour"`
|
||||
Day Duration `yaml:"day"`
|
||||
}
|
||||
|
||||
// TLS is the web UI's certificate: from files, or obtained through ACME
|
||||
@@ -183,6 +202,8 @@ func Default() *Config {
|
||||
Log: Log{Level: "info", Format: "text"},
|
||||
Session: Session{Lifetime: Duration(12 * time.Hour), IdleTimeout: Duration(2 * time.Hour), CookieSecure: "auto"},
|
||||
Backup: Backup{Dir: "/var/backups/housecarl", Retention: Duration(30 * 24 * time.Hour), KeepAtLeast: 7},
|
||||
Stats: Stats{Enabled: true, Database: "/var/lib/housecarl/stats.db", TopN: 100,
|
||||
Retention: StatsRetention{Minute: Duration(7 * 24 * time.Hour), Hour: Duration(90 * 24 * time.Hour), Day: Duration(730 * 24 * time.Hour)}},
|
||||
TLS: TLS{ACME: ACME{Directory: "letsencrypt-staging", Resolvers: []string{"1.1.1.1:53", "9.9.9.9:53", "8.8.8.8:53"}, CheckPropagation: true}},
|
||||
}
|
||||
}
|
||||
@@ -393,4 +414,17 @@ backup:
|
||||
dir: /var/backups/housecarl
|
||||
retention: 30d
|
||||
keep_at_least: 7
|
||||
|
||||
# Statistics (docs/stats.md): traffic per interface and zone, hits and
|
||||
# drops, blocked sources, connections, quotas, kept over time in a database
|
||||
# of their own (aggregates only). Deleting it is always safe. Traffic per
|
||||
# device is a firewall setting: hc settings set --stats-per-device on.
|
||||
stats:
|
||||
enabled: true
|
||||
database: /var/lib/housecarl/stats.db
|
||||
retention:
|
||||
minute: 7d # per-minute figures
|
||||
hour: 90d # hourly figures and top lists
|
||||
day: 730d # daily figures and top lists
|
||||
top_n: 100 # entries kept per top list, per hour and day
|
||||
`
|
||||
@@ -0,0 +1,351 @@
|
||||
// Package conntrack lists and closes the kernel's tracked connections,
|
||||
// through ctnetlink (the netfilter netlink subsystem for connection
|
||||
// tracking): what the "established" built-in lets through, and what a
|
||||
// change of rules doesn't stop by itself.
|
||||
//
|
||||
// Closing a connection deletes its tracking entry. Its next packet is
|
||||
// then checked by the rules again, as a new connection would be: one the
|
||||
// rules now block is stopped, one they allow carries on (TCP connections
|
||||
// are picked up again mid-stream).
|
||||
package conntrack
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/binary"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/netip"
|
||||
"sync/atomic"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"golang.org/x/sys/unix"
|
||||
|
||||
"housecarl/internal/api"
|
||||
)
|
||||
|
||||
const (
|
||||
subsysCtnetlink = 1
|
||||
msgNew = 0
|
||||
msgGet = 1
|
||||
msgDelete = 2
|
||||
|
||||
ctaTupleOrig = 1
|
||||
ctaTupleReply = 2
|
||||
ctaStatus = 3
|
||||
ctaProtoinfo = 4
|
||||
ctaTimeout = 7
|
||||
ctaMark = 8
|
||||
ctaCountersOrig = 9
|
||||
ctaCountersReply = 10
|
||||
ctaID = 12
|
||||
|
||||
ctaTupleIP = 1
|
||||
ctaTupleProto = 2
|
||||
|
||||
ctaIPv4Src = 1
|
||||
ctaIPv4Dst = 2
|
||||
ctaIPv6Src = 3
|
||||
ctaIPv6Dst = 4
|
||||
|
||||
ctaProtoNum = 1
|
||||
ctaProtoSrcPort = 2
|
||||
ctaProtoDstPort = 3
|
||||
ctaProtoICMPID = 4
|
||||
ctaProtoICMPType = 5
|
||||
ctaProtoICMPCode = 6
|
||||
ctaProtoICMP6ID = 7
|
||||
ctaProtoICMP6Typ = 8
|
||||
ctaProtoICMP6Cod = 9
|
||||
|
||||
ctaProtoinfoTCP = 1
|
||||
ctaProtoinfoTCPState = 1
|
||||
|
||||
ctaCountersPackets = 1
|
||||
ctaCountersBytes = 2
|
||||
|
||||
statusAssured = 1 << 2
|
||||
statusSrcNAT = 1 << 4
|
||||
statusDstNAT = 1 << 5
|
||||
|
||||
nlaNested = 0x8000
|
||||
nlaTypeMask = 0x3fff
|
||||
)
|
||||
|
||||
var tcpStates = []string{"none", "syn-sent", "syn-recv", "established", "fin-wait", "close-wait", "last-ack", "time-wait", "close", "syn-sent2"}
|
||||
|
||||
var protoNames = map[uint8]string{1: "icmp", 6: "tcp", 17: "udp", 58: "icmpv6", 132: "sctp", 136: "udplite", 33: "dccp", 47: "gre"}
|
||||
|
||||
// Client talks to ctnetlink. The zero value works.
|
||||
type Client struct{}
|
||||
|
||||
var seq atomic.Uint32
|
||||
|
||||
// List returns every tracked connection, IPv4 and IPv6.
|
||||
func (Client) List(ctx context.Context) ([]api.Connection, error) {
|
||||
msgs, err := request(ctx, msgGet, unix.NLM_F_REQUEST|unix.NLM_F_DUMP, unix.AF_UNSPEC, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out := []api.Connection{}
|
||||
for _, m := range msgs {
|
||||
if c, ok := parse(m); ok {
|
||||
out = append(out, c)
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// Close deletes a connection's tracking entry, found by its original
|
||||
// direction (addresses, protocol, ports or ICMP id). Gone already is fine.
|
||||
func (Client) Close(ctx context.Context, c api.Connection) error {
|
||||
src, err1 := netip.ParseAddr(c.Source)
|
||||
dst, err2 := netip.ParseAddr(c.Destination)
|
||||
if err1 != nil || err2 != nil || src.Is4() != dst.Is4() {
|
||||
return fmt.Errorf("connection %s -> %s: not a pair of addresses", c.Source, c.Destination)
|
||||
}
|
||||
num := protoNumber(c.Protocol)
|
||||
if num == 0 {
|
||||
return fmt.Errorf("connection %s -> %s: unknown protocol %q", c.Source, c.Destination, c.Protocol)
|
||||
}
|
||||
family, ipSrc, ipDst := uint8(unix.AF_INET), uint16(ctaIPv4Src), uint16(ctaIPv4Dst)
|
||||
if src.Is6() {
|
||||
family, ipSrc, ipDst = unix.AF_INET6, ctaIPv6Src, ctaIPv6Dst
|
||||
}
|
||||
ip := attr(ipSrc, src.AsSlice())
|
||||
ip = append(ip, attr(ipDst, dst.AsSlice())...)
|
||||
proto := attr(ctaProtoNum, []byte{num})
|
||||
switch num {
|
||||
case 1:
|
||||
proto = append(proto, attr(ctaProtoICMPID, be16(uint16(c.ICMPID)))...)
|
||||
proto = append(proto, attr(ctaProtoICMPType, []byte{uint8(c.ICMPType)})...)
|
||||
proto = append(proto, attr(ctaProtoICMPCode, []byte{uint8(c.ICMPCode)})...)
|
||||
case 58:
|
||||
proto = append(proto, attr(ctaProtoICMP6ID, be16(uint16(c.ICMPID)))...)
|
||||
proto = append(proto, attr(ctaProtoICMP6Typ, []byte{uint8(c.ICMPType)})...)
|
||||
proto = append(proto, attr(ctaProtoICMP6Cod, []byte{uint8(c.ICMPCode)})...)
|
||||
default:
|
||||
proto = append(proto, attr(ctaProtoSrcPort, be16(uint16(c.SourcePort)))...)
|
||||
proto = append(proto, attr(ctaProtoDstPort, be16(uint16(c.DestinationPort)))...)
|
||||
}
|
||||
tuple := attr(ctaTupleIP|nlaNested, ip)
|
||||
tuple = append(tuple, attr(ctaTupleProto|nlaNested, proto)...)
|
||||
_, err := request(ctx, msgDelete, unix.NLM_F_REQUEST|unix.NLM_F_ACK, family, attr(ctaTupleOrig|nlaNested, tuple))
|
||||
if errors.Is(err, unix.ENOENT) {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func protoNumber(name string) uint8 {
|
||||
for n, s := range protoNames {
|
||||
if s == name {
|
||||
return n
|
||||
}
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
// ---- netlink
|
||||
|
||||
func request(ctx context.Context, msg uint16, flags uint16, family uint8, payload []byte) ([][]byte, error) {
|
||||
fd, err := unix.Socket(unix.AF_NETLINK, unix.SOCK_RAW|unix.SOCK_CLOEXEC, unix.NETLINK_NETFILTER)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer unix.Close(fd)
|
||||
if err := unix.Bind(fd, &unix.SockaddrNetlink{Family: unix.AF_NETLINK}); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
deadline := time.Now().Add(10 * time.Second)
|
||||
if d, ok := ctx.Deadline(); ok && d.Before(deadline) {
|
||||
deadline = d
|
||||
}
|
||||
tv := unix.NsecToTimeval(time.Until(deadline).Nanoseconds())
|
||||
unix.SetsockoptTimeval(fd, unix.SOL_SOCKET, unix.SO_RCVTIMEO, &tv)
|
||||
unix.SetsockoptInt(fd, unix.SOL_SOCKET, unix.SO_RCVBUF, 1<<22)
|
||||
|
||||
s := seq.Add(1)
|
||||
b := make([]byte, unix.SizeofNlMsghdr+4, unix.SizeofNlMsghdr+4+len(payload))
|
||||
binary.NativeEndian.PutUint16(b[4:], subsysCtnetlink<<8|msg)
|
||||
binary.NativeEndian.PutUint16(b[6:], flags)
|
||||
binary.NativeEndian.PutUint32(b[8:], s)
|
||||
b[16] = family // nfgenmsg: family, version 0, res_id 0
|
||||
b = append(b, payload...)
|
||||
binary.NativeEndian.PutUint32(b[0:], uint32(len(b)))
|
||||
if err := unix.Sendto(fd, b, 0, &unix.SockaddrNetlink{Family: unix.AF_NETLINK}); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var out [][]byte
|
||||
buf := make([]byte, 1<<16)
|
||||
for {
|
||||
n, _, err := unix.Recvfrom(fd, buf, 0)
|
||||
if err != nil {
|
||||
if errors.Is(err, unix.EAGAIN) {
|
||||
return nil, errors.New("no answer from connection tracking (is nf_conntrack loaded?)")
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
for off := 0; off+unix.SizeofNlMsghdr <= n; {
|
||||
l := int(binary.NativeEndian.Uint32(buf[off:]))
|
||||
typ := binary.NativeEndian.Uint16(buf[off+4:])
|
||||
if l < unix.SizeofNlMsghdr || off+l > n {
|
||||
return nil, errors.New("connection tracking: a malformed answer")
|
||||
}
|
||||
body := buf[off+unix.SizeofNlMsghdr : off+l]
|
||||
switch typ {
|
||||
case unix.NLMSG_DONE:
|
||||
return out, nil
|
||||
case unix.NLMSG_ERROR:
|
||||
if len(body) >= 4 {
|
||||
if code := int32(binary.NativeEndian.Uint32(body)); code != 0 {
|
||||
return nil, syscall.Errno(-code)
|
||||
}
|
||||
}
|
||||
return out, nil // the acknowledgement
|
||||
default:
|
||||
out = append(out, append([]byte(nil), body...))
|
||||
}
|
||||
off += (l + 3) &^ 3
|
||||
}
|
||||
if flags&unix.NLM_F_DUMP == 0 && flags&unix.NLM_F_ACK == 0 {
|
||||
return out, nil
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func attr(typ uint16, data []byte) []byte {
|
||||
l := 4 + len(data)
|
||||
b := make([]byte, (l+3)&^3)
|
||||
binary.NativeEndian.PutUint16(b[0:], uint16(l))
|
||||
binary.NativeEndian.PutUint16(b[2:], typ)
|
||||
copy(b[4:], data)
|
||||
return b
|
||||
}
|
||||
|
||||
func be16(v uint16) []byte {
|
||||
b := make([]byte, 2)
|
||||
binary.BigEndian.PutUint16(b, v)
|
||||
return b
|
||||
}
|
||||
|
||||
// attrs splits netlink attributes: type (without the nested flag) → data.
|
||||
func attrs(b []byte) map[uint16][]byte {
|
||||
m := map[uint16][]byte{}
|
||||
for len(b) >= 4 {
|
||||
l := int(binary.NativeEndian.Uint16(b[0:]))
|
||||
typ := binary.NativeEndian.Uint16(b[2:]) & nlaTypeMask
|
||||
if l < 4 || l > len(b) {
|
||||
break
|
||||
}
|
||||
m[typ] = b[4:l]
|
||||
b = b[min((l+3)&^3, len(b)):]
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
// ---- parsing
|
||||
|
||||
type tuple struct {
|
||||
src, dst netip.Addr
|
||||
proto uint8
|
||||
sport, dport uint16
|
||||
icmpID uint16
|
||||
icmpType, icmpCd uint8
|
||||
}
|
||||
|
||||
func parseTuple(b []byte) (tuple, bool) {
|
||||
var t tuple
|
||||
a := attrs(b)
|
||||
ip := attrs(a[ctaTupleIP])
|
||||
if v, ok := ip[ctaIPv4Src]; ok {
|
||||
t.src, _ = netip.AddrFromSlice(v)
|
||||
t.dst, _ = netip.AddrFromSlice(ip[ctaIPv4Dst])
|
||||
} else if v, ok := ip[ctaIPv6Src]; ok {
|
||||
t.src, _ = netip.AddrFromSlice(v)
|
||||
t.dst, _ = netip.AddrFromSlice(ip[ctaIPv6Dst])
|
||||
}
|
||||
p := attrs(a[ctaTupleProto])
|
||||
if v := p[ctaProtoNum]; len(v) == 1 {
|
||||
t.proto = v[0]
|
||||
}
|
||||
if v := p[ctaProtoSrcPort]; len(v) == 2 {
|
||||
t.sport = binary.BigEndian.Uint16(v)
|
||||
}
|
||||
if v := p[ctaProtoDstPort]; len(v) == 2 {
|
||||
t.dport = binary.BigEndian.Uint16(v)
|
||||
}
|
||||
for _, k := range [][3]uint16{{ctaProtoICMPID, ctaProtoICMPType, ctaProtoICMPCode}, {ctaProtoICMP6ID, ctaProtoICMP6Typ, ctaProtoICMP6Cod}} {
|
||||
if v := p[k[0]]; len(v) == 2 {
|
||||
t.icmpID = binary.BigEndian.Uint16(v)
|
||||
}
|
||||
if v := p[k[1]]; len(v) == 1 {
|
||||
t.icmpType = v[0]
|
||||
}
|
||||
if v := p[k[2]]; len(v) == 1 {
|
||||
t.icmpCd = v[0]
|
||||
}
|
||||
}
|
||||
return t, t.src.IsValid() && t.dst.IsValid()
|
||||
}
|
||||
|
||||
// parse reads one connection (an nfgenmsg and its attributes).
|
||||
func parse(m []byte) (api.Connection, bool) {
|
||||
if len(m) < 4 {
|
||||
return api.Connection{}, false
|
||||
}
|
||||
a := attrs(m[4:])
|
||||
orig, ok1 := parseTuple(a[ctaTupleOrig])
|
||||
reply, ok2 := parseTuple(a[ctaTupleReply])
|
||||
if !ok1 || !ok2 {
|
||||
return api.Connection{}, false
|
||||
}
|
||||
c := api.Connection{
|
||||
Family: "ipv4", Protocol: protoNames[orig.proto],
|
||||
Source: orig.src.String(), SourcePort: int(orig.sport), Destination: orig.dst.String(), DestinationPort: int(orig.dport),
|
||||
ICMPID: int(orig.icmpID), ICMPType: int(orig.icmpType), ICMPCode: int(orig.icmpCd),
|
||||
}
|
||||
if orig.src.Is6() {
|
||||
c.Family = "ipv6"
|
||||
}
|
||||
if c.Protocol == "" {
|
||||
c.Protocol = fmt.Sprint(orig.proto)
|
||||
}
|
||||
// NAT shows in the reply direction: it comes from where the packets
|
||||
// really went, to where they really came from.
|
||||
if reply.src != orig.dst || reply.sport != orig.dport {
|
||||
c.DNATTo = reply.src.String()
|
||||
if reply.sport != orig.dport {
|
||||
c.DNATPort = int(reply.sport)
|
||||
}
|
||||
}
|
||||
if reply.dst != orig.src || reply.dport != orig.sport {
|
||||
c.SNATTo = reply.dst.String()
|
||||
}
|
||||
if v := a[ctaStatus]; len(v) == 4 {
|
||||
st := binary.BigEndian.Uint32(v)
|
||||
c.Assured = st&statusAssured != 0
|
||||
}
|
||||
if v := a[ctaTimeout]; len(v) == 4 {
|
||||
c.Timeout = int(binary.BigEndian.Uint32(v))
|
||||
}
|
||||
if v := a[ctaMark]; len(v) == 4 {
|
||||
c.Mark = binary.BigEndian.Uint32(v)
|
||||
}
|
||||
if v := a[ctaID]; len(v) == 4 {
|
||||
c.ID = binary.BigEndian.Uint32(v)
|
||||
}
|
||||
if v := attrs(attrs(a[ctaProtoinfo])[ctaProtoinfoTCP])[ctaProtoinfoTCPState]; len(v) == 1 && int(v[0]) < len(tcpStates) {
|
||||
c.State = tcpStates[v[0]]
|
||||
}
|
||||
for _, k := range []uint16{ctaCountersOrig, ctaCountersReply} {
|
||||
ct := attrs(a[k])
|
||||
if v := ct[ctaCountersPackets]; len(v) == 8 {
|
||||
c.Packets += binary.BigEndian.Uint64(v)
|
||||
}
|
||||
if v := ct[ctaCountersBytes]; len(v) == 8 {
|
||||
c.Bytes += binary.BigEndian.Uint64(v)
|
||||
}
|
||||
}
|
||||
return c, true
|
||||
}
|
||||
@@ -0,0 +1,51 @@
|
||||
package conntrack
|
||||
|
||||
import (
|
||||
"encoding/binary"
|
||||
"net/netip"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func nested(typ uint16, parts ...[]byte) []byte {
|
||||
var b []byte
|
||||
for _, p := range parts {
|
||||
b = append(b, p...)
|
||||
}
|
||||
return attr(typ|nlaNested, b)
|
||||
}
|
||||
|
||||
func tupleAttr(typ uint16, src, dst string, proto uint8, sport, dport uint16) []byte {
|
||||
s, d := netip.MustParseAddr(src), netip.MustParseAddr(dst)
|
||||
return nested(typ,
|
||||
nested(ctaTupleIP, attr(ctaIPv4Src, s.AsSlice()), attr(ctaIPv4Dst, d.AsSlice())),
|
||||
nested(ctaTupleProto, attr(ctaProtoNum, []byte{proto}), attr(ctaProtoSrcPort, be16(sport)), attr(ctaProtoDstPort, be16(dport))))
|
||||
}
|
||||
|
||||
// A port-forwarded TCP connection from the internet, as ctnetlink
|
||||
// describes it: the reply comes from the inside host.
|
||||
func TestParse(t *testing.T) {
|
||||
status, timeout := make([]byte, 4), make([]byte, 4)
|
||||
binary.BigEndian.PutUint32(status, statusAssured|statusDstNAT)
|
||||
binary.BigEndian.PutUint32(timeout, 431999)
|
||||
msg := []byte{2, 0, 0, 0} // nfgenmsg: AF_INET
|
||||
msg = append(msg, tupleAttr(ctaTupleOrig, "198.51.100.7", "203.0.113.10", 6, 51432, 8443)...)
|
||||
msg = append(msg, tupleAttr(ctaTupleReply, "192.168.1.10", "198.51.100.7", 6, 443, 51432)...)
|
||||
msg = append(msg, attr(ctaStatus, status)...)
|
||||
msg = append(msg, attr(ctaTimeout, timeout)...)
|
||||
msg = append(msg, nested(ctaProtoinfo, nested(ctaProtoinfoTCP, attr(ctaProtoinfoTCPState, []byte{3})))...)
|
||||
c, ok := parse(msg)
|
||||
if !ok {
|
||||
t.Fatal("not parsed")
|
||||
}
|
||||
if c.Protocol != "tcp" || c.Source != "198.51.100.7" || c.SourcePort != 51432 || c.Destination != "203.0.113.10" || c.DestinationPort != 8443 ||
|
||||
c.DNATTo != "192.168.1.10" || c.DNATPort != 443 || c.SNATTo != "" || c.State != "established" || !c.Assured || c.Timeout != 431999 {
|
||||
t.Errorf("parsed %+v", c)
|
||||
}
|
||||
// Masqueraded on the way out: the reply comes back to the WAN address.
|
||||
msg = []byte{2, 0, 0, 0}
|
||||
msg = append(msg, tupleAttr(ctaTupleOrig, "192.168.1.20", "198.51.100.9", 17, 5353, 53)...)
|
||||
msg = append(msg, tupleAttr(ctaTupleReply, "198.51.100.9", "203.0.113.10", 17, 53, 5353)...)
|
||||
if c, _ := parse(msg); c.SNATTo != "203.0.113.10" || c.DNATTo != "" || c.Protocol != "udp" {
|
||||
t.Errorf("masqueraded: %+v", c)
|
||||
}
|
||||
}
|
||||
+34
-10
@@ -74,6 +74,9 @@ type Result struct {
|
||||
HistoryID int64 `json:"history_id,omitempty"`
|
||||
Changes []model.Change `json:"changes"`
|
||||
Warnings []validate.Problem `json:"warnings,omitempty"`
|
||||
// Before and After are the configurations the change goes from and
|
||||
// to (After also for a dry run), for diffs and answers.
|
||||
Before, After *model.State `json:"-"`
|
||||
}
|
||||
|
||||
// Engine holds the current configuration.
|
||||
@@ -134,6 +137,9 @@ type Edit func(st *model.State, newID func() int64) error
|
||||
type Options struct {
|
||||
DryRun bool
|
||||
Via string // web, cli, api, import
|
||||
// RevertOf is the history entry the change undoes (a changeset with a
|
||||
// staged undo), so History links the two.
|
||||
RevertOf int64
|
||||
}
|
||||
|
||||
// Apply makes a change: edit runs on a copy of the configuration, the
|
||||
@@ -158,7 +164,7 @@ func (e *Engine) Apply(ctx context.Context, p *auth.Principal, description strin
|
||||
if idErr != nil {
|
||||
return nil, idErr
|
||||
}
|
||||
return e.commit(ctx, p, description, opts, next, 0)
|
||||
return e.commit(ctx, p, description, opts, next, opts.RevertOf)
|
||||
}
|
||||
|
||||
// commit validates next and saves it as the new state.
|
||||
@@ -166,7 +172,7 @@ func (e *Engine) commit(ctx context.Context, p *auth.Principal, description stri
|
||||
Normalize(next)
|
||||
next.Sort()
|
||||
changes := Diff(e.state, next)
|
||||
res := &Result{DryRun: opts.DryRun, Changes: changes}
|
||||
res := &Result{DryRun: opts.DryRun, Changes: changes, Before: e.state.Clone(), After: next.Clone()}
|
||||
if len(changes) == 0 {
|
||||
return res, nil
|
||||
}
|
||||
@@ -238,12 +244,16 @@ func newProblems(before, after []validate.Problem) []validate.Problem {
|
||||
return out
|
||||
}
|
||||
|
||||
// ApplyCurrent puts the current configuration in force (at start).
|
||||
// ApplyCurrent puts the current configuration in force again: at start,
|
||||
// and when a flowtable device appears. It holds the engine's lock, so a
|
||||
// change can't be applied in between.
|
||||
func (e *Engine) ApplyCurrent(ctx context.Context) error {
|
||||
if e.Firewall == nil {
|
||||
return nil
|
||||
}
|
||||
return e.Firewall.Apply(ctx, e.State())
|
||||
e.mu.Lock()
|
||||
defer e.mu.Unlock()
|
||||
return e.Firewall.Apply(ctx, e.state.Clone())
|
||||
}
|
||||
|
||||
// Revert undoes a history entry, if the objects it changed are still as
|
||||
@@ -261,8 +271,22 @@ func (e *Engine) Revert(ctx context.Context, p *auth.Principal, id int64, opts O
|
||||
}
|
||||
e.mu.Lock()
|
||||
defer e.mu.Unlock()
|
||||
cur := objects(e.state)
|
||||
next := e.state.Clone()
|
||||
if err := Undo(next, h); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return e.commit(ctx, p, UndoDescription(h), opts, next, h.ID)
|
||||
}
|
||||
|
||||
// UndoDescription is an undo's description: "undo 12: create rule …".
|
||||
func UndoDescription(h *model.HistoryEntry) string {
|
||||
return fmt.Sprintf("undo %d: %s", h.ID, h.Description)
|
||||
}
|
||||
|
||||
// Undo applies a history entry backwards to st, if every object it
|
||||
// changed is still as it left it (a change made since would be lost).
|
||||
func Undo(st *model.State, h *model.HistoryEntry) error {
|
||||
cur := objects(st)
|
||||
for i := len(h.Changes) - 1; i >= 0; i-- {
|
||||
c := h.Changes[i]
|
||||
now := cur[key{c.Kind, c.ID}]
|
||||
@@ -271,13 +295,13 @@ func (e *Engine) Revert(ctx context.Context, p *auth.Principal, id int64, opts O
|
||||
if what == "" {
|
||||
what = fmt.Sprintf("%s %d", c.Kind, c.ID)
|
||||
}
|
||||
return nil, Errorf(KindConflict, "%s %s has changed since entry %d; undo the later changes first, or change it by hand", c.Kind, what, id)
|
||||
return Errorf(KindConflict, "%s %s has changed since entry %d; undo the later changes first, or change it by hand", c.Kind, what, h.ID)
|
||||
}
|
||||
if err := put(next, c.Kind, c.ID, c.Before); err != nil {
|
||||
return nil, err
|
||||
if err := put(st, c.Kind, c.ID, c.Before); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return e.commit(ctx, p, fmt.Sprintf("undo %d: %s", h.ID, h.Description), opts, next, h.ID)
|
||||
return nil
|
||||
}
|
||||
|
||||
// Replace makes the configuration next as a whole (imports).
|
||||
@@ -370,7 +394,7 @@ func Diff(a, b *model.State) []model.Change {
|
||||
return out
|
||||
}
|
||||
|
||||
// Describe sums up changes: "create zone lan (enp2s0)", or
|
||||
// Describe sums up changes: "create zone lan (eth1)", or
|
||||
// "3 changes" for many.
|
||||
func Describe(cs []model.Change) string {
|
||||
if len(cs) == 0 {
|
||||
|
||||
@@ -63,6 +63,14 @@ func (h *Host) Up(name string) bool {
|
||||
return h.up[name]
|
||||
}
|
||||
|
||||
// Forget drops what was read, so the next question reads the interfaces
|
||||
// again (after the kernel said one appeared or went).
|
||||
func (h *Host) Forget() {
|
||||
h.mu.Lock()
|
||||
h.at = time.Time{}
|
||||
h.mu.Unlock()
|
||||
}
|
||||
|
||||
// Exists returns the set of interface names (nil if they can't be read).
|
||||
func (h *Host) Exists() map[string]bool {
|
||||
h.mu.Lock()
|
||||
|
||||
@@ -0,0 +1,98 @@
|
||||
package importer
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"os/exec"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"housecarl/internal/render"
|
||||
"housecarl/internal/validate"
|
||||
)
|
||||
|
||||
// TestImportAudit: rulesets the importer must not change the meaning of
|
||||
// (testdata/audit.nft, as nft 1.1.3 lists it).
|
||||
func TestImportAudit(t *testing.T) {
|
||||
js, _ := os.ReadFile("testdata/audit.json")
|
||||
txt, _ := os.ReadFile("testdata/audit.txt")
|
||||
defs, _ := os.ReadFile("testdata/audit.nft")
|
||||
res, err := Import(Input{JSON: js, Text: string(txt), Defines: string(defs), Zones: map[string]string{"enp2s0.20": "iot", "eth9": "x"}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
st := res.State
|
||||
notes := strings.Join(res.Notes, "\n")
|
||||
text := render.Render(st, render.Options{}).Text
|
||||
has := func(what, s string) {
|
||||
t.Helper()
|
||||
if !strings.Contains(text, s) {
|
||||
t.Errorf("%s: no %q in\n%s", what, s, text)
|
||||
}
|
||||
}
|
||||
// Two chains on the input hook: their rules once each, in priority
|
||||
// order; established stays a rule after the first chain's rule.
|
||||
if n := strings.Count(text, "192.0.2.66"); n != 1 {
|
||||
t.Errorf("the early input chain's rule %d times", n)
|
||||
}
|
||||
if strings.Index(text, "192.0.2.66") > strings.Index(text, "ct state established,related counter accept comment \"hc:r") ||
|
||||
!strings.Contains(notes, "both hook input") {
|
||||
t.Errorf("chains on one hook:\n%s", text)
|
||||
}
|
||||
// A second match on a field, and != on the hop limit: raw, as written.
|
||||
has("two source matches", "ip saddr 10.0.0.0/8 ip saddr != 10.0.10.0/24 tcp dport 22 drop")
|
||||
has("hop limit", "ip6 hoplimit != 255 drop")
|
||||
// A jump: a rule within it is the rule's own; one outside it, or raw,
|
||||
// keeps the jump's match in front.
|
||||
has("within the jump", `ip saddr 10.9.0.5 tcp dport 22 counter accept comment "hc:r`)
|
||||
has("outside the jump", "ip saddr 10.9.0.0/24 ip saddr 10.8.0.1 tcp dport 23 accept")
|
||||
has("raw under the jump", "ip saddr 10.9.0.0/24 meta mark 0x00000001 accept")
|
||||
// Sets: a timeout set is dynamic; an interface set is written out in
|
||||
// the raw rule that names it.
|
||||
if s := st.Set("blocklist"); s == nil || !s.Dynamic || s.Timeout != 3600 {
|
||||
t.Errorf("blocklist: %+v", s)
|
||||
}
|
||||
has("dynamic set", "ip saddr @blocklist_v4 counter drop")
|
||||
has("interface set", `iifname { "enp2s0", "enp3s0" } accept`)
|
||||
// An accept to a forwarded host that isn't only for forwarded
|
||||
// connections stays a rule; a DNAT of two ports, two forwards.
|
||||
has("not only forwarded", `ip daddr 10.0.10.37 tcp dport 443 counter accept comment "hc:r`)
|
||||
if len(st.Forwards) != 2 || st.Forwards[0].Name != "web-80" || st.Forwards[1].Name != "web-443" {
|
||||
t.Errorf("forwards %+v", st.Forwards)
|
||||
}
|
||||
// Zones: as given (a VLAN), from defines, and why.
|
||||
if z := st.Zone("iot"); z == nil || z.Interfaces[0] != "enp2s0.20" {
|
||||
t.Errorf("zone iot: %+v", st.Zones)
|
||||
}
|
||||
if st.Zone("lan") == nil || st.Zone("wan") == nil {
|
||||
t.Errorf("zones %+v", st.Zones)
|
||||
}
|
||||
for _, want := range []string{"iot (enp2s0.20: as given)", "lan (enp2s0: the define lan_if)", "--zone eth9=x: no rule names"} {
|
||||
if !strings.Contains(notes, want) {
|
||||
t.Errorf("no note %q in\n%s", want, notes)
|
||||
}
|
||||
}
|
||||
if ps := validate.Check(st, nil); validate.HasErrors(ps) {
|
||||
t.Errorf("problems: %+v", validate.Errors(ps))
|
||||
}
|
||||
// And nftables takes it.
|
||||
cmd := strings.Fields(os.Getenv("HOUSECARL_NFT"))
|
||||
if len(cmd) == 0 {
|
||||
return
|
||||
}
|
||||
c := exec.CommandContext(context.Background(), cmd[0], append(cmd[1:], "-c", "-f", "/dev/stdin")...)
|
||||
c.Stdin = strings.NewReader(text)
|
||||
if out, err := c.CombinedOutput(); err != nil {
|
||||
t.Errorf("nft -c: %v\n%s\n%s", err, out, text)
|
||||
}
|
||||
}
|
||||
|
||||
// TestValidIface: interface names a define may hold, VLANs too.
|
||||
func TestValidIface(t *testing.T) {
|
||||
for s, want := range map[string]bool{"enp2s0": true, "wan": true, "enp2s0.20": true, "br0.4094": true, "veth*": true,
|
||||
"10.0.10.5": false, "example.com": false, "a.b.c": false, "enp2s0.20.30": false, "averyveryverylongname": false} {
|
||||
if validIface(s) != want {
|
||||
t.Errorf("validIface(%q) = %v", s, !want)
|
||||
}
|
||||
}
|
||||
}
|
||||
+923
-139
File diff suppressed because it is too large.
Load diff
@@ -1,6 +1,7 @@
|
||||
package importer
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"os/exec"
|
||||
"strings"
|
||||
@@ -49,11 +50,24 @@ func TestImportRouter(t *testing.T) {
|
||||
if s := st.Set("masq_to_internet"); s == nil {
|
||||
t.Errorf("no set masq_to_internet (sets %v)", setNames(st))
|
||||
}
|
||||
// Built-ins: the ones at the head of their chains.
|
||||
// Built-ins: the ones at the head of their chains, up to the invalid
|
||||
// drop, which stays a rule (below): the ICMP, ICMPv6 and DHCPv6
|
||||
// accepts after it stay rules too, in their order, so nothing the
|
||||
// drop stopped gets through ahead of it.
|
||||
b := st.Settings.Builtins
|
||||
if !b.Loopback || !b.Established || !b.ICMPv6 || !b.PingInternal || !b.DHCPv6Client || !b.MSSClamp || !b.Antispoof {
|
||||
if !b.Loopback || !b.Established || b.ICMPv6 || b.PingInternal || b.DHCPv6Client || !b.MSSClamp || !b.Antispoof {
|
||||
t.Errorf("built-ins %+v", b)
|
||||
}
|
||||
var input []string
|
||||
for _, r := range st.Rules {
|
||||
if r.Chain == model.ChainInput {
|
||||
input = append(input, strings.Join(r.CtState, ",")+"|"+strings.Join(r.Services, ",")+"|"+r.Raw)
|
||||
}
|
||||
}
|
||||
if len(input) < 5 || input[0] != "invalid||" || !strings.HasPrefix(input[1], "|icmp/echo-reply,icmp/echo-request|") ||
|
||||
!strings.Contains(input[2], "icmpv6/packet-too-big") || !strings.Contains(input[3], "ip6 hoplimit 255") || !strings.Contains(input[4], "udp/546") {
|
||||
t.Errorf("input rules start %q", input)
|
||||
}
|
||||
// The forward chain's asymmetric-routing rule sits between the
|
||||
// established and invalid rules: invalid stays rules, in order.
|
||||
if b.DropInvalid {
|
||||
@@ -101,9 +115,16 @@ func TestImportRouter(t *testing.T) {
|
||||
if !wgDNS {
|
||||
t.Errorf("wg0_input not inlined")
|
||||
}
|
||||
// The jump's and the rule's sources both: the rule's are kept, noted.
|
||||
if !strings.Contains(strings.Join(res.Notes, "\n"), "the rule's own (10.50.50.3, 10.50.50.6) are kept") {
|
||||
t.Errorf("no note on the overlapping sources: %q", res.Notes)
|
||||
// The jump's and the rule's sources both: the rule's lie within the
|
||||
// jump's, so they say it all, without a note.
|
||||
var wgSSH bool
|
||||
for _, r := range st.Rules {
|
||||
if r.Chain == model.ChainInput && strings.Join(r.Services, ",") == "tcp/22" && strings.Join(r.Sources, ",") == "10.50.50.3,10.50.50.6" {
|
||||
wgSSH = true
|
||||
}
|
||||
}
|
||||
if !wgSSH || strings.Contains(strings.Join(res.Notes, "\n"), "jump") {
|
||||
t.Errorf("the WireGuard SSH rule: found %v, notes %q", wgSSH, res.Notes)
|
||||
}
|
||||
// The imported configuration is valid, and renders.
|
||||
if ps := validate.Check(st, nil); validate.HasErrors(ps) {
|
||||
@@ -173,3 +194,130 @@ func TestImportedRulesetPassesNft(t *testing.T) {
|
||||
t.Errorf("nft -c: %v\n%s", err, out)
|
||||
}
|
||||
}
|
||||
|
||||
// TestImportMaps: dnat maps become port forwards and verdict maps rules,
|
||||
// one per element (testdata/maps.nft); a map that can't be expanded is
|
||||
// left out with a note, never kept as a raw rule naming it.
|
||||
func TestImportMaps(t *testing.T) {
|
||||
js, _ := os.ReadFile("testdata/maps.json")
|
||||
txt, _ := os.ReadFile("testdata/maps.txt")
|
||||
res, err := Import(Input{JSON: js, Text: string(txt)})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var fwds []string
|
||||
for _, f := range res.State.Forwards {
|
||||
fwds = append(fwds, fmt.Sprintf("%s %s/%s->%s:%s", f.Name, strings.Join(f.Protocols, "+"), f.Ports, f.To, f.ToPort))
|
||||
}
|
||||
var rules []string
|
||||
for _, r := range res.State.Rules {
|
||||
rules = append(rules, fmt.Sprintf("%s in=%s from=%s %s %s%s", r.Chain, strings.Join(r.InZones, ","), strings.Join(r.Sources, ","),
|
||||
strings.Join(r.Services, ","), r.Action, r.Raw))
|
||||
}
|
||||
want := []string{
|
||||
"portforwards-80 tcp/80->192.168.1.50:8080",
|
||||
"portforwards-443 tcp/443->192.168.1.50:",
|
||||
"hosts-2222 tcp/2222->192.168.1.60:",
|
||||
" udp/5000->192.168.1.70:5001",
|
||||
" udp/6000->192.168.1.71:",
|
||||
}
|
||||
if got := strings.Join(fwds, "\n"); got != strings.Join(want, "\n") {
|
||||
t.Errorf("forwards:\n%s\nwant:\n%s", got, strings.Join(want, "\n"))
|
||||
}
|
||||
// In order; the jump in the verdict map is inlined.
|
||||
want = []string{
|
||||
"input in= from= accept",
|
||||
"input in=lan from= tcp/22 accept",
|
||||
"input in=lan from= tcp/23 drop",
|
||||
"input in=lan from=192.168.1.0/24 tcp/80 accept",
|
||||
"input in=wan from= udp/53 accept",
|
||||
"input in=wan from= udp/123 drop",
|
||||
"input in= from=10.0.0.0/8 accept",
|
||||
"input in= from=192.0.2.1 drop",
|
||||
}
|
||||
if got := strings.Join(rules, "\n"); got != strings.Join(want, "\n") {
|
||||
t.Errorf("rules:\n%s\nwant:\n%s", got, strings.Join(want, "\n"))
|
||||
}
|
||||
notes := strings.Join(res.Notes, "\n")
|
||||
for _, n := range []string{"map portforwards: its 2 elements became 2 port forwards", "map hosts: its element became a port forward",
|
||||
"uses the map @pairs, which couldn't be expanded"} {
|
||||
if !strings.Contains(notes, n) {
|
||||
t.Errorf("note %q missing:\n%s", n, notes)
|
||||
}
|
||||
}
|
||||
if len(res.Raw) > 0 {
|
||||
t.Errorf("raw rules: %v", res.Raw)
|
||||
}
|
||||
if ps := validate.Errors(validate.Check(res.State, nil)); len(ps) > 0 {
|
||||
t.Errorf("invalid: %+v", ps)
|
||||
}
|
||||
if cmd := strings.Fields(os.Getenv("HOUSECARL_NFT")); len(cmd) > 0 {
|
||||
c := exec.Command(cmd[0], append(cmd[1:], "-c", "-f", "/dev/stdin")...)
|
||||
c.Stdin = strings.NewReader(render.Render(res.State, render.Options{}).Text)
|
||||
if out, err := c.CombinedOutput(); err != nil {
|
||||
t.Errorf("nft -c: %v\n%s", err, out)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestImportMeters: meters (per-address limits, connection counts and
|
||||
// quotas), byte rates and quotas become rules' limits and quotas
|
||||
// (testdata/meters.nft); the meters' sets aren't imported as address sets.
|
||||
func TestImportMeters(t *testing.T) {
|
||||
js, _ := os.ReadFile("testdata/meters.json")
|
||||
txt, _ := os.ReadFile("testdata/meters.txt")
|
||||
res, err := Import(Input{JSON: js, Text: string(txt)})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var got []string
|
||||
for _, r := range res.State.Rules {
|
||||
s := r.Chain
|
||||
if l := r.Limit; l != nil {
|
||||
s += fmt.Sprintf(" limit %d %s/%s burst %d over=%v src=%v dst=%v", l.Rate, l.Unit, l.Per, l.Burst, l.Over, l.PerSource, l.PerDestination)
|
||||
}
|
||||
if r.ConnLimit > 0 {
|
||||
s += fmt.Sprintf(" conn %d", r.ConnLimit)
|
||||
}
|
||||
if q := r.Quota; q != nil {
|
||||
s += fmt.Sprintf(" quota %d over=%v src=%v", q.Bytes, q.Over, q.PerSource)
|
||||
}
|
||||
if r.GroupV4 != 0 || r.GroupV6 != 0 {
|
||||
s += fmt.Sprintf(" group %d/%d", r.GroupV4, r.GroupV6)
|
||||
}
|
||||
s += " " + r.Action + r.Raw
|
||||
got = append(got, s)
|
||||
}
|
||||
want := []string{
|
||||
"input accept", // established: only in input here, so a rule
|
||||
"input limit 3 /minute burst 5 over=true src=true dst=false drop",
|
||||
"input limit 3 /minute burst 5 over=true src=true dst=false group 0/64 drop",
|
||||
"input accept",
|
||||
"forward conn 20 drop",
|
||||
"forward limit 10240 kbytes/second burst 512 over=true src=false dst=false drop",
|
||||
"forward quota 1048576000 over=true src=true group 24/0 drop",
|
||||
"forward quota 104857600000 over=true src=false drop",
|
||||
"forward accept",
|
||||
}
|
||||
if strings.Join(got, "\n") != strings.Join(want, "\n") {
|
||||
t.Errorf("rules:\n%s\nwant:\n%s\nnotes: %v", strings.Join(got, "\n"), strings.Join(want, "\n"), res.Notes)
|
||||
}
|
||||
for _, s := range res.State.Sets {
|
||||
if s.Name != "trusted" {
|
||||
t.Errorf("a meter became a set: %s", s.Name)
|
||||
}
|
||||
}
|
||||
if len(res.Raw) > 0 {
|
||||
t.Errorf("raw rules: %v", res.Raw)
|
||||
}
|
||||
if ps := validate.Errors(validate.Check(res.State, nil)); len(ps) > 0 {
|
||||
t.Errorf("invalid: %+v", ps)
|
||||
}
|
||||
if cmd := strings.Fields(os.Getenv("HOUSECARL_NFT")); len(cmd) > 0 {
|
||||
c := exec.Command(cmd[0], append(cmd[1:], "-c", "-f", "/dev/stdin")...)
|
||||
c.Stdin = strings.NewReader(render.Render(res.State, render.Options{}).Text)
|
||||
if out, err := c.CombinedOutput(); err != nil {
|
||||
t.Errorf("nft -c: %v\n%s", err, out)
|
||||
}
|
||||
}
|
||||
}
|
||||
+1
@@ -0,0 +1 @@
|
||||
{"nftables": [{"metainfo": {"version": "1.1.3", "release_name": "Commodore Bullmoose #4", "json_schema_version": 1}}, {"table": {"family": "inet", "name": "audit", "handle": 1}}, {"chain": {"family": "inet", "table": "audit", "name": "early", "handle": 1, "type": "filter", "hook": "input", "prio": -10, "policy": "accept"}}, {"chain": {"family": "inet", "table": "audit", "name": "input", "handle": 2, "type": "filter", "hook": "input", "prio": 0, "policy": "drop"}}, {"chain": {"family": "inet", "table": "audit", "name": "admin", "handle": 3}}, {"chain": {"family": "inet", "table": "audit", "name": "forward", "handle": 4, "type": "filter", "hook": "forward", "prio": 0, "policy": "drop"}}, {"chain": {"family": "inet", "table": "audit", "name": "prerouting", "handle": 5, "type": "nat", "hook": "prerouting", "prio": -100, "policy": "accept"}}, {"set": {"family": "inet", "name": "blocklist", "table": "audit", "type": "ipv4_addr", "handle": 6, "flags": ["timeout"], "timeout": 3600}}, {"set": {"family": "inet", "name": "lans", "table": "audit", "type": "ifname", "handle": 7, "elem": ["enp2s0", "enp3s0"]}}, {"rule": {"family": "inet", "table": "audit", "chain": "early", "handle": 8, "comment": "an early input chain", "expr": [{"match": {"op": "==", "left": {"payload": {"protocol": "ip", "field": "saddr"}}, "right": "192.0.2.66"}}, {"drop": null}]}}, {"rule": {"family": "inet", "table": "audit", "chain": "input", "handle": 9, "expr": [{"match": {"op": "in", "left": {"ct": {"key": "state"}}, "right": ["established", "related"]}}, {"accept": null}]}}, {"rule": {"family": "inet", "table": "audit", "chain": "input", "handle": 10, "comment": "dynamic", "expr": [{"match": {"op": "==", "left": {"payload": {"protocol": "ip", "field": "saddr"}}, "right": "@blocklist"}}, {"drop": null}]}}, {"rule": {"family": "inet", "table": "audit", "chain": "input", "handle": 11, "comment": "an interface set", "expr": [{"match": {"op": "==", "left": {"meta": {"key": "iifname"}}, "right": "@lans"}}, {"accept": null}]}}, {"rule": {"family": "inet", "table": "audit", "chain": "input", "handle": 12, "comment": "two source matches", "expr": [{"match": {"op": "==", "left": {"payload": {"protocol": "ip", "field": "saddr"}}, "right": {"prefix": {"addr": "10.0.0.0", "len": 8}}}}, {"match": {"op": "!=", "left": {"payload": {"protocol": "ip", "field": "saddr"}}, "right": {"prefix": {"addr": "10.0.10.0", "len": 24}}}}, {"match": {"op": "==", "left": {"payload": {"protocol": "tcp", "field": "dport"}}, "right": 22}}, {"drop": null}]}}, {"rule": {"family": "inet", "table": "audit", "chain": "input", "handle": 13, "comment": "hop limit not 255", "expr": [{"match": {"op": "==", "left": {"payload": {"protocol": "icmpv6", "field": "type"}}, "right": "nd-neighbor-solicit"}}, {"match": {"op": "!=", "left": {"payload": {"protocol": "ip6", "field": "hoplimit"}}, "right": 255}}, {"drop": null}]}}, {"rule": {"family": "inet", "table": "audit", "chain": "input", "handle": 14, "comment": "DNS from the IoT VLAN", "expr": [{"match": {"op": "==", "left": {"meta": {"key": "iifname"}}, "right": "enp2s0.20"}}, {"match": {"op": "==", "left": {"payload": {"protocol": "udp", "field": "dport"}}, "right": 53}}, {"accept": null}]}}, {"rule": {"family": "inet", "table": "audit", "chain": "input", "handle": 15, "expr": [{"match": {"op": "==", "left": {"payload": {"protocol": "ip", "field": "saddr"}}, "right": {"prefix": {"addr": "10.9.0.0", "len": 24}}}}, {"jump": {"target": "admin"}}]}}, {"rule": {"family": "inet", "table": "audit", "chain": "admin", "handle": 16, "comment": "within the jump", "expr": [{"match": {"op": "==", "left": {"payload": {"protocol": "ip", "field": "saddr"}}, "right": "10.9.0.5"}}, {"match": {"op": "==", "left": {"payload": {"protocol": "tcp", "field": "dport"}}, "right": 22}}, {"accept": null}]}}, {"rule": {"family": "inet", "table": "audit", "chain": "admin", "handle": 17, "comment": "outside the jump", "expr": [{"match": {"op": "==", "left": {"payload": {"protocol": "ip", "field": "saddr"}}, "right": "10.8.0.1"}}, {"match": {"op": "==", "left": {"payload": {"protocol": "tcp", "field": "dport"}}, "right": 23}}, {"accept": null}]}}, {"rule": {"family": "inet", "table": "audit", "chain": "admin", "handle": 18, "comment": "raw under the jump", "expr": [{"match": {"op": "==", "left": {"meta": {"key": "mark"}}, "right": 1}}, {"accept": null}]}}, {"rule": {"family": "inet", "table": "audit", "chain": "forward", "handle": 19, "expr": [{"match": {"op": "in", "left": {"ct": {"key": "state"}}, "right": ["established", "related"]}}, {"accept": null}]}}, {"rule": {"family": "inet", "table": "audit", "chain": "forward", "handle": 20, "comment": "not only forwarded", "expr": [{"match": {"op": "==", "left": {"payload": {"protocol": "ip", "field": "daddr"}}, "right": "10.0.10.37"}}, {"match": {"op": "==", "left": {"payload": {"protocol": "tcp", "field": "dport"}}, "right": 443}}, {"accept": null}]}}, {"rule": {"family": "inet", "table": "audit", "chain": "forward", "hanLine truncated
|
||||
Vendored
+53
@@ -0,0 +1,53 @@
|
||||
#! /sbin/nft -f
|
||||
# Cases the importer must not get wrong (importer_test.go TestImportAudit).
|
||||
|
||||
define wan_if = "wan"
|
||||
define lan_if = "enp2s0"
|
||||
define iot_if = "enp2s0.20"
|
||||
define admins = { 10.9.0.0/24 }
|
||||
|
||||
table inet audit {
|
||||
set blocklist {
|
||||
type ipv4_addr
|
||||
flags timeout
|
||||
timeout 1h
|
||||
}
|
||||
set lans {
|
||||
type ifname
|
||||
elements = { "enp2s0", "enp3s0" }
|
||||
}
|
||||
|
||||
chain early {
|
||||
type filter hook input priority -10; policy accept;
|
||||
ip saddr 192.0.2.66 drop comment "an early input chain"
|
||||
}
|
||||
|
||||
chain input {
|
||||
type filter hook input priority filter; policy drop;
|
||||
ct state established,related accept
|
||||
ip saddr @blocklist drop comment "dynamic"
|
||||
iifname @lans accept comment "an interface set"
|
||||
ip saddr 10.0.0.0/8 ip saddr != 10.0.10.0/24 tcp dport 22 drop comment "two source matches"
|
||||
icmpv6 type nd-neighbor-solicit ip6 hoplimit != 255 drop comment "hop limit not 255"
|
||||
iifname $iot_if udp dport 53 accept comment "DNS from the IoT VLAN"
|
||||
ip saddr $admins jump admin
|
||||
}
|
||||
|
||||
chain admin {
|
||||
ip saddr 10.9.0.5 tcp dport 22 accept comment "within the jump"
|
||||
ip saddr 10.8.0.1 tcp dport 23 accept comment "outside the jump"
|
||||
meta mark 0x00000001 accept comment "raw under the jump"
|
||||
}
|
||||
|
||||
chain forward {
|
||||
type filter hook forward priority filter; policy drop;
|
||||
ct state established,related accept
|
||||
ip daddr 10.0.10.37 tcp dport 443 accept comment "not only forwarded"
|
||||
iifname $lan_if oifname $wan_if accept
|
||||
}
|
||||
|
||||
chain prerouting {
|
||||
type nat hook prerouting priority dstnat; policy accept;
|
||||
ip daddr 203.0.113.1 tcp dport { 80, 443 } dnat ip to 10.0.10.37 comment "web"
|
||||
}
|
||||
}
|
||||
Vendored
+46
@@ -0,0 +1,46 @@
|
||||
table inet audit { # handle 1
|
||||
set blocklist { # handle 6
|
||||
type ipv4_addr
|
||||
timeout 1h
|
||||
}
|
||||
|
||||
set lans { # handle 7
|
||||
type ifname
|
||||
elements = { "enp2s0",
|
||||
"enp3s0" }
|
||||
}
|
||||
|
||||
chain early { # handle 1
|
||||
type filter hook input priority filter - 10; policy accept;
|
||||
ip saddr 192.0.2.66 drop comment "an early input chain" # handle 8
|
||||
}
|
||||
|
||||
chain input { # handle 2
|
||||
type filter hook input priority filter; policy drop;
|
||||
ct state established,related accept # handle 9
|
||||
ip saddr @blocklist drop comment "dynamic" # handle 10
|
||||
iifname @lans accept comment "an interface set" # handle 11
|
||||
ip saddr 10.0.0.0/8 ip saddr != 10.0.10.0/24 tcp dport 22 drop comment "two source matches" # handle 12
|
||||
icmpv6 type nd-neighbor-solicit ip6 hoplimit != 255 drop comment "hop limit not 255" # handle 13
|
||||
iifname "enp2s0.20" udp dport 53 accept comment "DNS from the IoT VLAN" # handle 14
|
||||
ip saddr 10.9.0.0/24 jump admin # handle 15
|
||||
}
|
||||
|
||||
chain admin { # handle 3
|
||||
ip saddr 10.9.0.5 tcp dport 22 accept comment "within the jump" # handle 16
|
||||
ip saddr 10.8.0.1 tcp dport 23 accept comment "outside the jump" # handle 17
|
||||
meta mark 0x00000001 accept comment "raw under the jump" # handle 18
|
||||
}
|
||||
|
||||
chain forward { # handle 4
|
||||
type filter hook forward priority filter; policy drop;
|
||||
ct state established,related accept # handle 19
|
||||
ip daddr 10.0.10.37 tcp dport 443 accept comment "not only forwarded" # handle 20
|
||||
iifname "enp2s0" oifname "wan" accept # handle 21
|
||||
}
|
||||
|
||||
chain prerouting { # handle 5
|
||||
type nat hook prerouting priority dstnat; policy accept;
|
||||
ip daddr 203.0.113.1 tcp dport { 80, 443 } dnat ip to 10.0.10.37 comment "web" # handle 23
|
||||
}
|
||||
}
|
||||
Vendored
+1
@@ -0,0 +1 @@
|
||||
{"nftables": [{"metainfo": {"version": "1.1.3", "release_name": "Commodore Bullmoose #4", "json_schema_version": 1}}, {"table": {"family": "inet", "name": "main", "handle": 3}}, {"chain": {"family": "inet", "table": "main", "name": "web", "handle": 1}}, {"chain": {"family": "inet", "table": "main", "name": "prerouting", "handle": 2, "type": "nat", "hook": "prerouting", "prio": -100, "policy": "accept"}}, {"chain": {"family": "inet", "table": "main", "name": "input", "handle": 3, "type": "filter", "hook": "input", "prio": 0, "policy": "drop"}}, {"map": {"family": "inet", "name": "portforwards", "table": "main", "type": "inet_service", "handle": 4, "map": ["ipv4_addr", "inet_service"], "elem": [[80, {"concat": ["192.168.1.50", 8080]}], [443, {"concat": ["192.168.1.50", 443]}]]}}, {"map": {"family": "inet", "name": "hosts", "table": "main", "type": "inet_service", "handle": 5, "map": "ipv4_addr", "elem": [[2222, "192.168.1.60"]]}}, {"map": {"family": "inet", "name": "lan_services", "table": "main", "type": "inet_service", "handle": 6, "map": "verdict", "elem": [[22, {"accept": null}], [23, {"drop": null}], [80, {"jump": {"target": "web"}}]]}}, {"map": {"family": "inet", "name": "pairs", "table": "main", "type": ["ipv4_addr", "inet_service"], "handle": 7, "map": "verdict", "elem": [[{"concat": ["192.168.1.9", 22]}, {"accept": null}]]}}, {"rule": {"family": "inet", "table": "main", "chain": "web", "handle": 8, "expr": [{"match": {"op": "==", "left": {"payload": {"protocol": "ip", "field": "saddr"}}, "right": {"prefix": {"addr": "192.168.1.0", "len": 24}}}}, {"accept": null}]}}, {"rule": {"family": "inet", "table": "main", "chain": "prerouting", "handle": 9, "comment": "Port forwards", "expr": [{"match": {"op": "==", "left": {"meta": {"key": "iifname"}}, "right": "wan"}}, {"dnat": {"family": "ip", "addr": {"map": {"key": {"payload": {"protocol": "tcp", "field": "dport"}}, "data": "@portforwards"}}}}]}}, {"rule": {"family": "inet", "table": "main", "chain": "prerouting", "handle": 10, "expr": [{"match": {"op": "==", "left": {"meta": {"key": "iifname"}}, "right": "wan"}}, {"dnat": {"family": "ip", "addr": {"map": {"key": {"payload": {"protocol": "tcp", "field": "dport"}}, "data": "@hosts"}}}}]}}, {"rule": {"family": "inet", "table": "main", "chain": "prerouting", "handle": 12, "expr": [{"match": {"op": "==", "left": {"meta": {"key": "iifname"}}, "right": "wan"}}, {"dnat": {"family": "ip", "addr": {"map": {"key": {"payload": {"protocol": "udp", "field": "dport"}}, "data": {"set": [[5000, {"concat": ["192.168.1.70", 5001]}], [6000, {"concat": ["192.168.1.71", 6000]}]]}}}}}]}}, {"rule": {"family": "inet", "table": "main", "chain": "input", "handle": 13, "expr": [{"match": {"op": "in", "left": {"ct": {"key": "state"}}, "right": ["established", "related"]}}, {"accept": null}]}}, {"rule": {"family": "inet", "table": "main", "chain": "input", "handle": 14, "expr": [{"match": {"op": "==", "left": {"meta": {"key": "iifname"}}, "right": "lan"}}, {"vmap": {"key": {"payload": {"protocol": "tcp", "field": "dport"}}, "data": "@lan_services"}}]}}, {"rule": {"family": "inet", "table": "main", "chain": "input", "handle": 16, "expr": [{"match": {"op": "==", "left": {"meta": {"key": "iifname"}}, "right": "wan"}}, {"vmap": {"key": {"payload": {"protocol": "udp", "field": "dport"}}, "data": {"set": [[53, {"accept": null}], [123, {"drop": null}]]}}}]}}, {"rule": {"family": "inet", "table": "main", "chain": "input", "handle": 18, "expr": [{"vmap": {"key": {"payload": {"protocol": "ip", "field": "saddr"}}, "data": {"set": [[{"prefix": {"addr": "10.0.0.0", "len": 8}}, {"accept": null}], ["192.0.2.1", {"drop": null}]]}}}]}}, {"rule": {"family": "inet", "table": "main", "chain": "input", "handle": 19, "expr": [{"vmap": {"key": {"concat": [{"payload": {"protocol": "ip", "field": "saddr"}}, {"payload": {"protocol": "tcp", "field": "dport"}}]}, "data": "@pairs"}}]}}]}
|
||||
Vendored
+36
@@ -0,0 +1,36 @@
|
||||
# Maps and verdict maps, as found in hand-written rulesets (importer_test.go).
|
||||
table inet main {
|
||||
map portforwards {
|
||||
type inet_service : ipv4_addr . inet_service
|
||||
elements = { 443 : 192.168.1.50 . 443, 80 : 192.168.1.50 . 8080 }
|
||||
}
|
||||
map hosts {
|
||||
type inet_service : ipv4_addr
|
||||
elements = { 2222 : 192.168.1.60 }
|
||||
}
|
||||
map lan_services {
|
||||
type inet_service : verdict
|
||||
elements = { 22 : accept, 23 : drop, 80 : jump web }
|
||||
}
|
||||
map pairs {
|
||||
type ipv4_addr . inet_service : verdict
|
||||
elements = { 192.168.1.9 . 22 : accept }
|
||||
}
|
||||
chain web {
|
||||
ip saddr 192.168.1.0/24 accept
|
||||
}
|
||||
chain prerouting {
|
||||
type nat hook prerouting priority dstnat; policy accept;
|
||||
iifname "wan" dnat ip to tcp dport map @portforwards comment "Port forwards"
|
||||
iifname "wan" dnat ip to tcp dport map @hosts
|
||||
iifname "wan" dnat ip to udp dport map { 5000 : 192.168.1.70 . 5001, 6000 : 192.168.1.71 . 6000 }
|
||||
}
|
||||
chain input {
|
||||
type filter hook input priority filter; policy drop;
|
||||
ct state established,related accept
|
||||
iifname "lan" tcp dport vmap @lan_services
|
||||
iifname "wan" udp dport vmap { 53 : accept, 123 : drop }
|
||||
ip saddr vmap { 10.0.0.0/8 : accept, 192.0.2.1 : drop }
|
||||
ip saddr . tcp dport vmap @pairs
|
||||
}
|
||||
}
|
||||
Vendored
+44
@@ -0,0 +1,44 @@
|
||||
table inet main { # handle 3
|
||||
map portforwards { # handle 4
|
||||
type inet_service : ipv4_addr . inet_service
|
||||
elements = { 80 : 192.168.1.50 . 8080,
|
||||
443 : 192.168.1.50 . 443 }
|
||||
}
|
||||
|
||||
map hosts { # handle 5
|
||||
type inet_service : ipv4_addr
|
||||
elements = { 2222 : 192.168.1.60 }
|
||||
}
|
||||
|
||||
map lan_services { # handle 6
|
||||
type inet_service : verdict
|
||||
elements = { 22 : accept,
|
||||
23 : drop,
|
||||
80 : jump web }
|
||||
}
|
||||
|
||||
map pairs { # handle 7
|
||||
type ipv4_addr . inet_service : verdict
|
||||
elements = { 192.168.1.9 . 22 : accept }
|
||||
}
|
||||
|
||||
chain web { # handle 1
|
||||
ip saddr 192.168.1.0/24 accept # handle 8
|
||||
}
|
||||
|
||||
chain prerouting { # handle 2
|
||||
type nat hook prerouting priority dstnat; policy accept;
|
||||
iifname "wan" dnat ip to tcp dport map @portforwards comment "Port forwards" # handle 9
|
||||
iifname "wan" dnat ip to tcp dport map @hosts # handle 10
|
||||
iifname "wan" dnat ip to udp dport map { 5000 : 192.168.1.70 . 5001, 6000 : 192.168.1.71 . 6000 } # handle 12
|
||||
}
|
||||
|
||||
chain input { # handle 3
|
||||
type filter hook input priority filter; policy drop;
|
||||
ct state established,related accept # handle 13
|
||||
iifname "lan" tcp dport vmap @lan_services # handle 14
|
||||
iifname "wan" udp dport vmap { 53 : accept, 123 : drop } # handle 16
|
||||
ip saddr vmap { 10.0.0.0/8 : accept, 192.0.2.1 : drop } # handle 18
|
||||
ip saddr . tcp dport vmap @pairs # handle 19
|
||||
}
|
||||
}
|
||||
+1
@@ -0,0 +1 @@
|
||||
{"nftables": [{"metainfo": {"version": "1.1.3", "release_name": "Commodore Bullmoose #4", "json_schema_version": 1}}, {"table": {"family": "inet", "name": "main", "handle": 1}}, {"chain": {"family": "inet", "table": "main", "name": "input", "handle": 1, "type": "filter", "hook": "input", "prio": 0, "policy": "drop"}}, {"chain": {"family": "inet", "table": "main", "name": "forward", "handle": 2, "type": "filter", "hook": "forward", "prio": 0, "policy": "drop"}}, {"chain": {"family": "inet", "table": "main", "name": "output", "handle": 3, "type": "filter", "hook": "output", "prio": 0, "policy": "accept"}}, {"set": {"family": "inet", "name": "ssh_meter", "table": "main", "type": "ipv4_addr", "handle": 4, "size": 65535, "flags": ["timeout", "dynamic"], "timeout": 60}}, {"set": {"family": "inet", "name": "ssh6_meter", "table": "main", "type": "ipv6_addr", "handle": 5, "size": 65535, "flags": ["timeout", "dynamic"], "timeout": 60}}, {"set": {"family": "inet", "name": "my_connlimit", "table": "main", "type": "ipv4_addr", "handle": 6, "size": 65535, "flags": ["dynamic"]}}, {"set": {"family": "inet", "name": "guest_quota", "table": "main", "type": "ipv4_addr", "handle": 7, "size": 65535, "flags": ["dynamic"]}}, {"set": {"family": "inet", "name": "trusted", "table": "main", "type": "ipv4_addr", "handle": 8, "flags": ["interval"], "elem": [{"prefix": {"addr": "10.0.10.0", "len": 24}}]}}, {"rule": {"family": "inet", "table": "main", "chain": "input", "handle": 9, "expr": [{"match": {"op": "in", "left": {"ct": {"key": "state"}}, "right": ["established", "related"]}}, {"accept": null}]}}, {"rule": {"family": "inet", "table": "main", "chain": "input", "handle": 10, "expr": [{"match": {"op": "==", "left": {"payload": {"protocol": "tcp", "field": "dport"}}, "right": 22}}, {"match": {"op": "in", "left": {"ct": {"key": "state"}}, "right": "new"}}, {"set": {"op": "update", "elem": {"payload": {"protocol": "ip", "field": "saddr"}}, "set": "@ssh_meter", "stmt": [{"limit": {"rate": 3, "burst": 5, "per": "minute", "inv": true}}]}}, {"counter": {"packets": 0, "bytes": 0}}, {"drop": null}]}}, {"rule": {"family": "inet", "table": "main", "chain": "input", "handle": 11, "expr": [{"match": {"op": "==", "left": {"payload": {"protocol": "tcp", "field": "dport"}}, "right": 22}}, {"match": {"op": "in", "left": {"ct": {"key": "state"}}, "right": "new"}}, {"set": {"op": "update", "elem": {"&": [{"payload": {"protocol": "ip6", "field": "saddr"}}, "ffff:ffff:ffff:ffff::"]}, "set": "@ssh6_meter", "stmt": [{"limit": {"rate": 3, "burst": 5, "per": "minute", "inv": true}}]}}, {"counter": {"packets": 0, "bytes": 0}}, {"drop": null}]}}, {"rule": {"family": "inet", "table": "main", "chain": "input", "handle": 12, "expr": [{"match": {"op": "==", "left": {"payload": {"protocol": "ip", "field": "saddr"}}, "right": "@trusted"}}, {"match": {"op": "==", "left": {"payload": {"protocol": "tcp", "field": "dport"}}, "right": 22}}, {"accept": null}]}}, {"rule": {"family": "inet", "table": "main", "chain": "forward", "handle": 13, "expr": [{"match": {"op": "in", "left": {"ct": {"key": "state"}}, "right": "new"}}, {"set": {"op": "add", "elem": {"payload": {"protocol": "ip", "field": "saddr"}}, "set": "@my_connlimit", "stmt": [{"ct count": {"val": 20, "inv": true}}]}}, {"counter": {"packets": 0, "bytes": 0}}, {"drop": null}]}}, {"rule": {"family": "inet", "table": "main", "chain": "forward", "handle": 14, "expr": [{"match": {"op": "==", "left": {"meta": {"key": "iifname"}}, "right": "lan"}}, {"match": {"op": "==", "left": {"meta": {"key": "oifname"}}, "right": "wan"}}, {"limit": {"rate": 10, "burst": 512, "per": "second", "inv": true, "rate_unit": "mbytes", "burst_unit": "kbytes"}}, {"drop": null}]}}, {"rule": {"family": "inet", "table": "main", "chain": "forward", "handle": 15, "expr": [{"match": {"op": "==", "left": {"meta": {"key": "iifname"}}, "right": "guest0"}}, {"match": {"op": "==", "left": {"meta": {"key": "oifname"}}, "right": "wan"}}, {"set": {"op": "add", "elem": {"&": [{"payload": {"protocol": "ip", "field": "saddr"}}, "255.255.255.0"]}, "set": "@guest_quota", "stmt": [{"quota": {"val": 1000, "val_unit": "mbytes", "inv": true}}]}}, {"drop": null}]}}, {"rule": {"family": "inet", "table": "main", "chain": "forward", "handle": 16, "expr": [{"match": {"op": "==", "left": {"meta": {"key": "iifname"}}, "right": "lan"}}, {"match": {"op": "==", "left": {"meta": {"key": "oifname"}}, "right": "wan"}}, {"quota": {"val": 100000, "val_unit": "mbytes", "inv": true}}, {"drop": null}]}}, {"rule": {"family": "inet", "table": "main", "chain": "forward", "handle": 17, "expr": [{"match": {"op": "==", "left": {"meta": {"key": "iifname"}}, "right": "lan"}}, {"match": {"op": "==", "left": {"meta": {"key": "oifname"}}, "right": "wan"}}, {"accept": null}]}}]}
|
||||
+46
@@ -0,0 +1,46 @@
|
||||
# Meters, byte rates and quotas, as hand-written rulesets use them (importer_test.go).
|
||||
table inet main {
|
||||
set ssh_meter {
|
||||
type ipv4_addr
|
||||
flags dynamic,timeout
|
||||
timeout 1m
|
||||
}
|
||||
set ssh6_meter {
|
||||
type ipv6_addr
|
||||
flags dynamic,timeout
|
||||
timeout 1m
|
||||
}
|
||||
set my_connlimit {
|
||||
type ipv4_addr
|
||||
size 65535
|
||||
flags dynamic
|
||||
}
|
||||
set guest_quota {
|
||||
type ipv4_addr
|
||||
size 65535
|
||||
flags dynamic
|
||||
}
|
||||
set trusted {
|
||||
type ipv4_addr
|
||||
flags interval
|
||||
elements = { 10.0.10.0/24 }
|
||||
}
|
||||
chain input {
|
||||
type filter hook input priority filter; policy drop;
|
||||
ct state established,related accept
|
||||
tcp dport 22 ct state new update @ssh_meter { ip saddr limit rate over 3/minute burst 5 packets } counter drop
|
||||
tcp dport 22 ct state new update @ssh6_meter { ip6 saddr and ffff:ffff:ffff:ffff:: limit rate over 3/minute } counter drop
|
||||
ip saddr @trusted tcp dport 22 accept
|
||||
}
|
||||
chain forward {
|
||||
type filter hook forward priority filter; policy drop;
|
||||
ct state new add @my_connlimit { ip saddr ct count over 20 } counter drop
|
||||
iifname "lan" oifname "wan" limit rate over 10 mbytes/second burst 512 kbytes drop
|
||||
iifname "guest0" oifname "wan" add @guest_quota { ip saddr and 255.255.255.0 quota over 1000 mbytes } drop
|
||||
iifname "lan" oifname "wan" quota over 100000 mbytes drop
|
||||
iifname "lan" oifname "wan" accept
|
||||
}
|
||||
chain output {
|
||||
type filter hook output priority filter; policy accept;
|
||||
}
|
||||
}
|
||||
+54
@@ -0,0 +1,54 @@
|
||||
table inet main { # handle 1
|
||||
set ssh_meter { # handle 4
|
||||
type ipv4_addr
|
||||
size 65535
|
||||
flags dynamic,timeout
|
||||
timeout 1m
|
||||
}
|
||||
|
||||
set ssh6_meter { # handle 5
|
||||
type ipv6_addr
|
||||
size 65535
|
||||
flags dynamic,timeout
|
||||
timeout 1m
|
||||
}
|
||||
|
||||
set my_connlimit { # handle 6
|
||||
type ipv4_addr
|
||||
size 65535
|
||||
flags dynamic
|
||||
}
|
||||
|
||||
set guest_quota { # handle 7
|
||||
type ipv4_addr
|
||||
size 65535
|
||||
flags dynamic
|
||||
}
|
||||
|
||||
set trusted { # handle 8
|
||||
type ipv4_addr
|
||||
flags interval
|
||||
elements = { 10.0.10.0/24 }
|
||||
}
|
||||
|
||||
chain input { # handle 1
|
||||
type filter hook input priority filter; policy drop;
|
||||
ct state established,related accept # handle 9
|
||||
tcp dport 22 ct state new update @ssh_meter { ip saddr limit rate over 3/minute burst 5 packets } counter packets 0 bytes 0 drop # handle 10
|
||||
tcp dport 22 ct state new update @ssh6_meter { ip6 saddr & ffff:ffff:ffff:ffff:: limit rate over 3/minute burst 5 packets } counter packets 0 bytes 0 drop # handle 11
|
||||
ip saddr @trusted tcp dport 22 accept # handle 12
|
||||
}
|
||||
|
||||
chain forward { # handle 2
|
||||
type filter hook forward priority filter; policy drop;
|
||||
ct state new add @my_connlimit { ip saddr ct count over 20 } counter packets 0 bytes 0 drop # handle 13
|
||||
iifname "lan" oifname "wan" limit rate over 10 mbytes/second burst 512 kbytes drop # handle 14
|
||||
iifname "guest0" oifname "wan" add @guest_quota { ip saddr & 255.255.255.0 quota over 1000 mbytes } drop # handle 15
|
||||
iifname "lan" oifname "wan" quota over 100000 mbytes drop # handle 16
|
||||
iifname "lan" oifname "wan" accept # handle 17
|
||||
}
|
||||
|
||||
chain output { # handle 3
|
||||
type filter hook output priority filter; policy accept;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,96 @@
|
||||
// Package linkwatch tells when the machine's network interfaces change:
|
||||
// one appears, goes, or comes back (ppp0 when PPPoE reconnects). It
|
||||
// listens to the kernel's rtnetlink link messages and calls back once
|
||||
// they have been quiet for a moment, since one change brings several.
|
||||
package linkwatch
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/binary"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"time"
|
||||
|
||||
"golang.org/x/sys/unix"
|
||||
)
|
||||
|
||||
// Settle is how long the link messages must be quiet before the callback.
|
||||
const Settle = 2 * time.Second
|
||||
|
||||
// Watch calls changed after interfaces changed, until ctx ends. It returns
|
||||
// an error only if it can't listen at all.
|
||||
func Watch(ctx context.Context, changed func()) error {
|
||||
fd, err := unix.Socket(unix.AF_NETLINK, unix.SOCK_RAW|unix.SOCK_CLOEXEC|unix.SOCK_NONBLOCK, unix.NETLINK_ROUTE)
|
||||
if err != nil {
|
||||
return fmt.Errorf("opening a netlink socket: %w", err)
|
||||
}
|
||||
if err := unix.Bind(fd, &unix.SockaddrNetlink{Family: unix.AF_NETLINK, Groups: unix.RTMGRP_LINK}); err != nil {
|
||||
unix.Close(fd)
|
||||
return fmt.Errorf("listening to link changes: %w", err)
|
||||
}
|
||||
// A non-blocking descriptor goes through Go's poller, so closing the
|
||||
// file ends a Read in progress.
|
||||
f := os.NewFile(uintptr(fd), "rtnetlink")
|
||||
go func() {
|
||||
<-ctx.Done()
|
||||
f.Close()
|
||||
}()
|
||||
events := make(chan struct{}, 1)
|
||||
go func() {
|
||||
buf := make([]byte, 64<<10)
|
||||
for {
|
||||
n, err := f.Read(buf)
|
||||
switch {
|
||||
case err != nil && ctx.Err() != nil:
|
||||
close(events)
|
||||
return
|
||||
case errors.Is(err, unix.ENOBUFS):
|
||||
// Messages were lost: something changed anyway.
|
||||
case err != nil:
|
||||
time.Sleep(time.Second)
|
||||
continue
|
||||
case !links(buf[:n]):
|
||||
continue
|
||||
}
|
||||
select {
|
||||
case events <- struct{}{}:
|
||||
default:
|
||||
}
|
||||
}
|
||||
}()
|
||||
var timer <-chan time.Time
|
||||
for {
|
||||
select {
|
||||
case _, ok := <-events:
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
timer = time.After(Settle)
|
||||
case <-timer:
|
||||
timer = nil
|
||||
changed()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// links reports whether a netlink datagram holds a link being added,
|
||||
// changed or deleted.
|
||||
func links(b []byte) bool {
|
||||
for len(b) >= unix.NLMSG_HDRLEN {
|
||||
size := int(binary.NativeEndian.Uint32(b[0:4]))
|
||||
typ := binary.NativeEndian.Uint16(b[4:6])
|
||||
if size < unix.NLMSG_HDRLEN || size > len(b) {
|
||||
return false
|
||||
}
|
||||
if typ == unix.RTM_NEWLINK || typ == unix.RTM_DELLINK {
|
||||
return true
|
||||
}
|
||||
size = (size + unix.NLMSG_ALIGNTO - 1) &^ (unix.NLMSG_ALIGNTO - 1)
|
||||
if size >= len(b) {
|
||||
return false
|
||||
}
|
||||
b = b[size:]
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
package linkwatch
|
||||
|
||||
import (
|
||||
"encoding/binary"
|
||||
"testing"
|
||||
|
||||
"golang.org/x/sys/unix"
|
||||
)
|
||||
|
||||
// msg is a netlink message of a type, with a payload of n bytes.
|
||||
func msg(typ uint16, n int) []byte {
|
||||
b := make([]byte, unix.NLMSG_HDRLEN+n)
|
||||
binary.NativeEndian.PutUint32(b[0:4], uint32(len(b)))
|
||||
binary.NativeEndian.PutUint16(b[4:6], typ)
|
||||
return b
|
||||
}
|
||||
|
||||
func TestLinks(t *testing.T) {
|
||||
for _, c := range []struct {
|
||||
name string
|
||||
b []byte
|
||||
want bool
|
||||
}{
|
||||
{"a new link", msg(unix.RTM_NEWLINK, 16), true},
|
||||
{"a link going", msg(unix.RTM_DELLINK, 16), true},
|
||||
{"an address", msg(unix.RTM_NEWADDR, 16), false},
|
||||
{"a link after an address (padded)", append(msg(unix.RTM_NEWADDR, 13), append(make([]byte, 3), msg(unix.RTM_NEWLINK, 8)...)...), true},
|
||||
{"truncated", msg(unix.RTM_NEWLINK, 16)[:10], false},
|
||||
{"a length beyond the datagram", msg(unix.RTM_NEWLINK, 16)[:unix.NLMSG_HDRLEN+4], false},
|
||||
{"empty", nil, false},
|
||||
} {
|
||||
if got := links(c.b); got != c.want {
|
||||
t.Errorf("%s: %v", c.name, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,323 @@
|
||||
// Package matrix works out what the rules do to new connections between
|
||||
// zones, for an overview: for each pair it walks the rules as nftables
|
||||
// would, in order. A rule that applies to the whole pair decides it; one
|
||||
// that applies to some of its traffic (some addresses, services, times,
|
||||
// rates…) is an exception, and the walk goes on; what is left falls to
|
||||
// the chain's policy.
|
||||
package matrix
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"net/netip"
|
||||
"strings"
|
||||
|
||||
"housecarl/internal/api"
|
||||
"housecarl/internal/model"
|
||||
)
|
||||
|
||||
// Firewall stands for the machine itself in the matrix.
|
||||
const Firewall = "firewall"
|
||||
|
||||
// Options tell Build about the machine.
|
||||
type Options struct {
|
||||
// Connected maps the networks directly connected to the firewall's
|
||||
// interfaces to their zones. A rule whose addresses all lie in other
|
||||
// zones' networks can't apply to a pair, and is left out of it.
|
||||
Connected map[netip.Prefix]string
|
||||
}
|
||||
|
||||
// Build works out the matrix of a configuration.
|
||||
func Build(st *model.State, opts ...Options) api.Matrix {
|
||||
var o Options
|
||||
if len(opts) > 0 {
|
||||
o = opts[0]
|
||||
}
|
||||
m := api.Matrix{Zones: []string{}, Cells: []api.MatrixCell{}}
|
||||
for _, z := range st.Zones {
|
||||
m.Zones = append(m.Zones, z.Name)
|
||||
}
|
||||
for _, from := range m.Zones {
|
||||
for _, to := range m.Zones {
|
||||
m.Cells = append(m.Cells, cell(st, o, model.ChainForward, from, to))
|
||||
}
|
||||
m.Cells = append(m.Cells, cell(st, o, model.ChainInput, from, Firewall))
|
||||
}
|
||||
for _, to := range m.Zones {
|
||||
m.Cells = append(m.Cells, cell(st, o, model.ChainOutput, Firewall, to))
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
// Cell returns one pair (from or to may be Firewall).
|
||||
func Cell(st *model.State, o Options, from, to string) (api.MatrixCell, bool) {
|
||||
known := func(z string) bool { return st.Zone(z) != nil }
|
||||
switch {
|
||||
case from == Firewall && known(to):
|
||||
return cell(st, o, model.ChainOutput, from, to), true
|
||||
case to == Firewall && known(from):
|
||||
return cell(st, o, model.ChainInput, from, to), true
|
||||
case known(from) && known(to):
|
||||
return cell(st, o, model.ChainForward, from, to), true
|
||||
}
|
||||
return api.MatrixCell{}, false
|
||||
}
|
||||
|
||||
func cell(st *model.State, o Options, chain, from, to string) api.MatrixCell {
|
||||
c := api.MatrixCell{From: from, To: to, Chain: chain, Exceptions: []api.MatrixException{}}
|
||||
b := st.Settings.Builtins
|
||||
fromZone := st.Zone(from)
|
||||
external := fromZone != nil && fromZone.External
|
||||
// What comes in passes the early chain first.
|
||||
if chain != model.ChainOutput {
|
||||
if b.Antispoof && external {
|
||||
c.Exceptions = append(c.Exceptions, api.MatrixException{Chain: model.ChainEarly, Label: "built-in: antispoof", Verdict: "blocked",
|
||||
What: "sources that can't come from outside (private ranges, your own prefixes)"})
|
||||
}
|
||||
if decided := walk(st, o, &c, model.ChainEarly, from, ""); decided {
|
||||
return c
|
||||
}
|
||||
}
|
||||
// The built-ins that let some new connections in.
|
||||
switch chain {
|
||||
case model.ChainInput:
|
||||
if b.ICMPv6 {
|
||||
c.Exceptions = append(c.Exceptions, api.MatrixException{Chain: chain, Label: "built-in: icmpv6", Verdict: "allowed", What: "ICMPv6 errors, echo, neighbour discovery, MLD"})
|
||||
}
|
||||
if b.PingInternal && !external {
|
||||
c.Exceptions = append(c.Exceptions, api.MatrixException{Chain: chain, Label: "built-in: ping", Verdict: "allowed", What: "ping (ICMP echo)"})
|
||||
}
|
||||
if b.DHCPv6Client && external {
|
||||
c.Exceptions = append(c.Exceptions, api.MatrixException{Chain: chain, Label: "built-in: dhcpv6-client", Verdict: "allowed", What: "DHCPv6 replies to the firewall"})
|
||||
}
|
||||
case model.ChainForward:
|
||||
if n := forwards(st); n > 0 && external && !st.Zone(to).External {
|
||||
c.Exceptions = append(c.Exceptions, api.MatrixException{Chain: chain, Label: "port forwards", Verdict: "allowed",
|
||||
What: fmt.Sprintf("%d port forward%s to inside hosts", n, map[bool]string{true: "", false: "s"}[n == 1])})
|
||||
}
|
||||
case model.ChainOutput:
|
||||
if st.Settings.OutputPolicy == model.PolicyDrop && b.ICMPv6 {
|
||||
c.Exceptions = append(c.Exceptions, api.MatrixException{Chain: chain, Label: "built-in: icmpv6", Verdict: "allowed", What: "ICMPv6 errors, echo, neighbour discovery, MLD"})
|
||||
}
|
||||
}
|
||||
if walk(st, o, &c, chain, from, to) {
|
||||
return c
|
||||
}
|
||||
policy := map[string]string{model.ChainInput: st.Settings.InputPolicy, model.ChainForward: st.Settings.ForwardPolicy, model.ChainOutput: st.Settings.OutputPolicy}[chain]
|
||||
c.Verdict, c.DecidedBy = verdictOf(policy), chain+" policy"
|
||||
return c
|
||||
}
|
||||
|
||||
func forwards(st *model.State) int {
|
||||
n := 0
|
||||
for _, f := range st.Forwards {
|
||||
if !f.Disabled && f.To != "" {
|
||||
n++
|
||||
}
|
||||
}
|
||||
return n
|
||||
}
|
||||
|
||||
// walk goes through a chain's rules for the pair, as nftables checks
|
||||
// them (those that check every packet first). It reports whether a rule
|
||||
// decided the whole pair.
|
||||
func walk(st *model.State, o Options, c *api.MatrixCell, chain, from, to string) bool {
|
||||
var rules []*model.Rule
|
||||
for _, every := range []bool{true, false} {
|
||||
for _, r := range st.Rules {
|
||||
if r.Chain == chain && !r.Disabled && r.ChecksEveryPacket() == every {
|
||||
rules = append(rules, r)
|
||||
}
|
||||
}
|
||||
}
|
||||
for _, r := range rules {
|
||||
if r.Raw != "" {
|
||||
c.Exceptions = append(c.Exceptions, api.MatrixException{RuleID: r.ID, Chain: chain, Label: "rule " + r.Label(), Verdict: "unknown", What: "a raw rule: " + r.Raw})
|
||||
continue
|
||||
}
|
||||
if !zones(r, from, to) || !forNew(r) {
|
||||
continue
|
||||
}
|
||||
// Addresses that all live behind other zones can't come from (or
|
||||
// go to) this pair's.
|
||||
if !r.NotSources && elsewhere(st, o, r.Sources, from) || !r.NotDests && to != Firewall && to != "" && elsewhere(st, o, r.Destinations, to) {
|
||||
continue
|
||||
}
|
||||
if chain == model.ChainEarly && r.Action != model.ActionDrop && r.Action != model.ActionReject {
|
||||
continue // an early accept decides nothing: input or forward still check
|
||||
}
|
||||
v := verdictOf(r.Action)
|
||||
what := conditions(st, r)
|
||||
if r.Action == model.ActionLog || what != "" {
|
||||
if what == "" {
|
||||
what = "everything"
|
||||
}
|
||||
c.Exceptions = append(c.Exceptions, api.MatrixException{RuleID: r.ID, Chain: chain, Label: "rule " + r.Label(), Verdict: v, What: what})
|
||||
continue
|
||||
}
|
||||
c.Verdict, c.DecidedBy, c.RuleID = v, "rule "+r.Label(), r.ID
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func verdictOf(action string) string {
|
||||
switch action {
|
||||
case model.ActionAccept: // and the accept policy
|
||||
return "allowed"
|
||||
case model.ActionLog:
|
||||
return "logged"
|
||||
}
|
||||
return "blocked"
|
||||
}
|
||||
|
||||
// elsewhere reports whether every address of refs (addresses, sets) lies
|
||||
// in networks connected to zones other than zone: then traffic of zone
|
||||
// can't have them. An empty list, a dynamic set, or an address the
|
||||
// firewall isn't directly connected to (a routed network) might.
|
||||
func elsewhere(st *model.State, o Options, refs []string, zone string) bool {
|
||||
if len(refs) == 0 || len(o.Connected) == 0 || zone == Firewall {
|
||||
return false
|
||||
}
|
||||
var entries []string
|
||||
for _, r := range refs {
|
||||
if s := st.Set(r); s != nil {
|
||||
if s.Dynamic {
|
||||
return false
|
||||
}
|
||||
entries = append(entries, s.Entries...)
|
||||
} else {
|
||||
entries = append(entries, r)
|
||||
}
|
||||
}
|
||||
if len(entries) == 0 {
|
||||
return false
|
||||
}
|
||||
for _, e := range entries {
|
||||
a, err := model.ParseAddress(e)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
first, last := a.Prefix.Masked().Addr(), lastAddr(a.Prefix)
|
||||
if a.First.IsValid() {
|
||||
first, last = a.First, a.End
|
||||
}
|
||||
owner := ""
|
||||
for net, z := range o.Connected {
|
||||
if net.Contains(first) && net.Contains(last) {
|
||||
owner = z
|
||||
break
|
||||
}
|
||||
}
|
||||
if owner == "" || strings.EqualFold(owner, zone) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// lastAddr is a prefix's last address.
|
||||
func lastAddr(p netip.Prefix) netip.Addr {
|
||||
b := p.Masked().Addr().AsSlice()
|
||||
for i := p.Bits(); i < len(b)*8; i++ {
|
||||
b[i/8] |= 0x80 >> (i % 8)
|
||||
}
|
||||
a, _ := netip.AddrFromSlice(b)
|
||||
return a
|
||||
}
|
||||
|
||||
// zones reports whether a rule's zones take in the pair.
|
||||
func zones(r *model.Rule, from, to string) bool {
|
||||
in := func(list []string, not bool, z string) bool {
|
||||
if len(list) == 0 {
|
||||
return true
|
||||
}
|
||||
for _, x := range list {
|
||||
if strings.EqualFold(x, z) {
|
||||
return !not
|
||||
}
|
||||
}
|
||||
return not
|
||||
}
|
||||
if from == Firewall {
|
||||
if len(r.InZones) > 0 {
|
||||
return false
|
||||
}
|
||||
} else if !in(r.InZones, r.NotIn, from) {
|
||||
return false
|
||||
}
|
||||
if to == "" || to == Firewall {
|
||||
return len(r.OutZones) == 0
|
||||
}
|
||||
return in(r.OutZones, r.NotOut, to)
|
||||
}
|
||||
|
||||
// forNew reports whether a rule can match a new connection's first
|
||||
// packet (its connection states allow "new").
|
||||
func forNew(r *model.Rule) bool {
|
||||
if len(r.CtState) == 0 {
|
||||
return true
|
||||
}
|
||||
for _, s := range r.CtState {
|
||||
if s == "new" {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// conditions says what limits a rule to part of a pair's traffic ("" for
|
||||
// none: it decides the pair).
|
||||
func conditions(st *model.State, r *model.Rule) string {
|
||||
var parts []string
|
||||
list := func(not bool, xs []string) string {
|
||||
s := strings.Join(xs, ", ")
|
||||
if not {
|
||||
return "any but " + s
|
||||
}
|
||||
return s
|
||||
}
|
||||
if len(r.Sources) > 0 {
|
||||
parts = append(parts, "from "+list(r.NotSources, r.Sources))
|
||||
}
|
||||
if len(r.Destinations) > 0 {
|
||||
parts = append(parts, "to "+list(r.NotDests, r.Destinations))
|
||||
}
|
||||
if len(r.Services) > 0 {
|
||||
parts = append(parts, strings.Join(r.Services, ", "))
|
||||
}
|
||||
if r.Family != "" {
|
||||
parts = append(parts, r.Family+" only")
|
||||
}
|
||||
if len(r.CtStatus) > 0 {
|
||||
parts = append(parts, "connections that were "+strings.Join(r.CtStatus, ", ")+"'d")
|
||||
}
|
||||
if len(r.Days) > 0 {
|
||||
parts = append(parts, strings.Join(r.Days, ", "))
|
||||
}
|
||||
if r.HourFrom != "" {
|
||||
parts = append(parts, r.HourFrom+"–"+r.HourTo)
|
||||
}
|
||||
if l := r.Limit; l != nil {
|
||||
unit := ""
|
||||
if l.Unit != "" && l.Unit != "packets" {
|
||||
unit = " " + l.Unit
|
||||
}
|
||||
rate := fmt.Sprintf("%d%s/%s", l.Rate, unit, l.Per)
|
||||
if l.Over {
|
||||
parts = append(parts, "beyond "+rate)
|
||||
} else {
|
||||
parts = append(parts, "within "+rate)
|
||||
}
|
||||
}
|
||||
if r.ConnLimit > 0 {
|
||||
parts = append(parts, fmt.Sprintf("sources with over %d connections", r.ConnLimit))
|
||||
}
|
||||
if q := r.Quota; q != nil {
|
||||
if q.Over {
|
||||
parts = append(parts, "once its quota is used up")
|
||||
} else {
|
||||
parts = append(parts, "while its quota lasts")
|
||||
}
|
||||
}
|
||||
return strings.Join(parts, " · ")
|
||||
}
|
||||
@@ -0,0 +1,112 @@
|
||||
package matrix
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"net/netip"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"housecarl/internal/api"
|
||||
"housecarl/internal/model"
|
||||
)
|
||||
|
||||
func state() *model.State {
|
||||
st := model.NewState()
|
||||
st.Zones = []*model.Zone{
|
||||
{ID: 1, Name: "guest", Interfaces: []string{"eth2"}},
|
||||
{ID: 2, Name: "lan", Interfaces: []string{"eth1"}},
|
||||
{ID: 3, Name: "wan", Interfaces: []string{"wan"}, External: true},
|
||||
}
|
||||
st.Sets = []*model.AddressSet{{ID: 4, Name: "admins", Entries: []string{"192.168.1.0/28"}}}
|
||||
pos := 0.0
|
||||
rule := func(r model.Rule) {
|
||||
pos++
|
||||
r.ID, r.Position = int64(10+pos), pos
|
||||
st.Rules = append(st.Rules, &r)
|
||||
}
|
||||
rule(model.Rule{Chain: "input", Match: model.Match{Sources: []string{"admins"}, Services: []string{"ssh"}}, Action: "accept", Comment: "SSH"})
|
||||
rule(model.Rule{Chain: "input", Match: model.Match{InZones: []string{"lan", "guest"}, Services: []string{"dns"}}, Action: "accept"})
|
||||
rule(model.Rule{Chain: "forward", Match: model.Match{InZones: []string{"lan"}, OutZones: []string{"wan"}}, Action: "accept", Comment: "LAN out"})
|
||||
rule(model.Rule{Chain: "forward", Match: model.Match{InZones: []string{"guest"}, OutZones: []string{"wan"}, Services: []string{"http", "https"}}, Action: "accept"})
|
||||
rule(model.Rule{Chain: "forward", Match: model.Match{InZones: []string{"wan"}}, Action: "drop", Comment: "nothing in"})
|
||||
rule(model.Rule{Chain: "forward", Match: model.Match{InZones: []string{"lan"}}, CtState: []string{"established"}, Action: "drop"}) // not for new
|
||||
rule(model.Rule{Chain: "output", Match: model.Match{OutZones: []string{"wan"}}, Action: "drop", Disabled: true})
|
||||
st.Forwards = []*model.PortForward{{ID: 30, Name: "web", Protocols: []string{"tcp"}, Ports: "443", To: "192.168.1.10"}}
|
||||
st.Settings.Builtins.MSSClamp = true
|
||||
return st
|
||||
}
|
||||
|
||||
func find(m api.Matrix, from, to string) api.MatrixCell {
|
||||
for _, c := range m.Cells {
|
||||
if c.From == from && c.To == to {
|
||||
return c
|
||||
}
|
||||
}
|
||||
return api.MatrixCell{}
|
||||
}
|
||||
|
||||
func TestMatrix(t *testing.T) {
|
||||
m := Build(state())
|
||||
if strings.Join(m.Zones, ",") != "guest,lan,wan" || len(m.Cells) != 3*3+3+3 {
|
||||
t.Fatalf("zones %v, %d cells", m.Zones, len(m.Cells))
|
||||
}
|
||||
for _, c := range []struct {
|
||||
from, to, verdict, decided string
|
||||
exceptions int
|
||||
}{
|
||||
{"lan", "wan", "allowed", "rule forward #13 (LAN out)", 0},
|
||||
{"guest", "wan", "blocked", "forward policy", 1}, // only web
|
||||
{"wan", "lan", "blocked", "rule forward #15 (nothing in)", 2}, // antispoof, port forwards come first
|
||||
{"lan", "guest", "blocked", "forward policy", 0}, // the established-only rule is no exception
|
||||
{"lan", Firewall, "blocked", "input policy", 4}, // icmpv6, ping, ssh from admins, dns
|
||||
{"wan", Firewall, "blocked", "input policy", 4}, // antispoof, icmpv6, dhcpv6, ssh from admins
|
||||
{Firewall, "wan", "allowed", "output policy", 0}, // the drop is disabled
|
||||
} {
|
||||
got := find(m, c.from, c.to)
|
||||
if got.Verdict != c.verdict || got.DecidedBy != c.decided || len(got.Exceptions) != c.exceptions {
|
||||
t.Errorf("%s → %s: %s by %s, %d exceptions %+v; want %s by %s, %d", c.from, c.to, got.Verdict, got.DecidedBy, len(got.Exceptions), got.Exceptions, c.verdict, c.decided, c.exceptions)
|
||||
}
|
||||
}
|
||||
g := find(m, "guest", "wan").Exceptions[0]
|
||||
if g.Verdict != "allowed" || g.What != "http, https" || g.RuleID != 14 {
|
||||
t.Errorf("the guest exception: %+v", g)
|
||||
}
|
||||
if e := find(m, "wan", "lan").Exceptions; e[0].Label != "built-in: antispoof" || e[1].Label != "port forwards" {
|
||||
t.Errorf("wan → lan exceptions: %+v", e)
|
||||
}
|
||||
if _, ok := Cell(state(), Options{}, "lan", "nowhere"); ok {
|
||||
t.Error("an unknown zone")
|
||||
}
|
||||
}
|
||||
|
||||
// Rules whose addresses all live behind another zone are left out of a
|
||||
// pair; routed (unknown) networks, and mixed lists, stay.
|
||||
func TestConnected(t *testing.T) {
|
||||
st := state()
|
||||
st.Sets = append(st.Sets, &model.AddressSet{ID: 5, Name: "guest_net", Entries: []string{"192.168.2.0/24"}},
|
||||
&model.AddressSet{ID: 6, Name: "mixed", Entries: []string{"192.168.2.0/24", "10.20.30.0/24"}})
|
||||
st.Rules = append(st.Rules,
|
||||
&model.Rule{ID: 40, Chain: "forward", Position: 10, Match: model.Match{Sources: []string{"guest_net"}}, Action: "drop", Comment: "guests"},
|
||||
&model.Rule{ID: 41, Chain: "forward", Position: 11, Match: model.Match{Destinations: []string{"192.168.2.5"}}, Action: "accept", Comment: "to a guest"},
|
||||
&model.Rule{ID: 42, Chain: "forward", Position: 12, Match: model.Match{Sources: []string{"mixed"}}, Action: "drop", Comment: "mixed"})
|
||||
o := Options{Connected: map[netip.Prefix]string{netip.MustParsePrefix("192.168.1.0/24"): "lan", netip.MustParsePrefix("192.168.2.0/24"): "guest"}}
|
||||
ids := func(c api.MatrixCell) []int64 {
|
||||
var out []int64
|
||||
for _, e := range c.Exceptions {
|
||||
out = append(out, e.RuleID)
|
||||
}
|
||||
return out
|
||||
}
|
||||
lanWan, _ := Cell(st, o, "lan", "wan")
|
||||
if fmt.Sprint(ids(lanWan)) != "[]" {
|
||||
t.Errorf("lan → wan: %v", ids(lanWan)) // decided by "LAN out" before the others anyway
|
||||
}
|
||||
lanGuest, _ := Cell(st, o, "lan", "guest")
|
||||
if fmt.Sprint(ids(lanGuest)) != "[41 42]" {
|
||||
t.Errorf("lan → guest: %v", ids(lanGuest)) // not "guests": their addresses are guest's
|
||||
}
|
||||
guestLan, _ := Cell(st, o, "guest", "lan")
|
||||
if fmt.Sprint(ids(guestLan)) != "[40 42]" {
|
||||
t.Errorf("guest → lan: %v", ids(guestLan)) // not "to a guest": it is in guest, not lan
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,59 @@
|
||||
package model
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Changeset statuses.
|
||||
const (
|
||||
ChangesetOpen = "open"
|
||||
ChangesetApplied = "applied"
|
||||
ChangesetDiscarded = "discarded"
|
||||
)
|
||||
|
||||
// Changeset is a list of changes, staged to be reviewed and applied
|
||||
// together (or discarded). Applied, it is one history entry.
|
||||
type Changeset struct {
|
||||
ID int64 `json:"id"`
|
||||
UserID int64 `json:"user_id,omitempty"`
|
||||
Username string `json:"username"`
|
||||
Via string `json:"via"`
|
||||
Description string `json:"description"`
|
||||
Status string `json:"status"`
|
||||
Ops []Op `json:"ops"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
UpdatedAt time.Time `json:"updated_at"`
|
||||
// HistoryID is the history entry an applied changeset became.
|
||||
HistoryID int64 `json:"history_id,omitempty"`
|
||||
}
|
||||
|
||||
// Operations.
|
||||
const (
|
||||
OpCreate = "create"
|
||||
OpUpdate = "update"
|
||||
OpDelete = "delete"
|
||||
OpMove = "move"
|
||||
OpSettings = "settings"
|
||||
// OpRevert undoes a history entry: Data is the entry (its changes),
|
||||
// ID its number.
|
||||
OpRevert = "revert"
|
||||
)
|
||||
|
||||
// Op is one change, as the API received it, so it can be stored and
|
||||
// replayed on whatever the configuration is when it is applied.
|
||||
type Op struct {
|
||||
Seq int `json:"seq"`
|
||||
Op string `json:"op"` // create, update, delete, move, settings, revert
|
||||
Kind string `json:"kind,omitempty"`
|
||||
// Ref names the object (ID, or name) for update, delete and move.
|
||||
Ref string `json:"ref,omitempty"`
|
||||
// ID is the object a create makes, given when it is staged so later
|
||||
// operations can name it.
|
||||
ID int64 `json:"id,omitempty"`
|
||||
// Data is the object (create), the fields to change (update,
|
||||
// settings) or where to move it (move).
|
||||
Data json.RawMessage `json:"data,omitempty"`
|
||||
Summary string `json:"summary"`
|
||||
AddedAt time.Time `json:"added_at"`
|
||||
}
|
||||
+109
-4
@@ -133,8 +133,20 @@ type Rule struct {
|
||||
// LimitOver, beyond) a rate.
|
||||
Limit *Limit `json:"limit,omitempty"`
|
||||
// ConnLimit matches when a source already has more than this many
|
||||
// connections (per source address).
|
||||
// connections (per source address, grouped as GroupV4/GroupV6 say).
|
||||
ConnLimit int `json:"conn_limit,omitempty"`
|
||||
// Quota, if set, is an allowance of bytes: the rule matches once it
|
||||
// is used up (Over), or while it lasts.
|
||||
Quota *Quota `json:"quota,omitempty"`
|
||||
// GroupV4 and GroupV6 group addresses for the per-address limits,
|
||||
// connection limit and quota: by prefix length (0: the default, each
|
||||
// IPv4 address alone, IPv6 by /64, since one device has several).
|
||||
GroupV4 int `json:"group_v4,omitempty"`
|
||||
GroupV6 int `json:"group_v6,omitempty"`
|
||||
// EveryPacket checks the rule against every packet, before established
|
||||
// connections are accepted, so a drop or reject also stops connections
|
||||
// already open, not just new ones.
|
||||
EveryPacket bool `json:"every_packet,omitempty"`
|
||||
Action string `json:"action"`
|
||||
// RejectWith: "" (ICMP port/admin unreachable as fits) or tcp-reset.
|
||||
RejectWith string `json:"reject_with,omitempty"`
|
||||
@@ -155,18 +167,111 @@ func (r *Rule) Label() string {
|
||||
return fmt.Sprintf("%s #%d", r.Chain, r.ID)
|
||||
}
|
||||
|
||||
// Limit is a rate: Rate packets per Per (second, minute, hour, day), with
|
||||
// a burst.
|
||||
// Limit is a rate: Rate packets (or bytes, with Unit) per Per (second,
|
||||
// minute, hour, day), with a burst.
|
||||
type Limit struct {
|
||||
Rate int `json:"rate"`
|
||||
Per string `json:"per"`
|
||||
// Burst is in the rate's unit.
|
||||
Burst int `json:"burst,omitempty"`
|
||||
// Unit: "" for packets, or bytes, kbytes or mbytes for a byte rate
|
||||
// (bandwidth).
|
||||
Unit string `json:"unit,omitempty"`
|
||||
// Over matches packets beyond the rate instead of within it (to drop
|
||||
// floods).
|
||||
Over bool `json:"over,omitempty"`
|
||||
// PerSource counts each source address on its own (a meter), instead
|
||||
// of all traffic the rule sees together.
|
||||
// of all traffic the rule sees together; PerDestination each
|
||||
// destination.
|
||||
PerSource bool `json:"per_source,omitempty"`
|
||||
PerDestination bool `json:"per_destination,omitempty"`
|
||||
}
|
||||
|
||||
// Byte units, as nftables writes them.
|
||||
var ByteUnits = map[string]int64{"bytes": 1, "kbytes": 1 << 10, "mbytes": 1 << 20, "gbytes": 1 << 30}
|
||||
|
||||
// Quota is an allowance of bytes. nftables counts it; Housecarl keeps the
|
||||
// count across changes and restarts, and resets it each period.
|
||||
type Quota struct {
|
||||
Bytes int64 `json:"bytes"`
|
||||
// Over matches once the allowance is used up (to drop); otherwise the
|
||||
// rule matches while it lasts (to accept).
|
||||
Over bool `json:"over,omitempty"`
|
||||
// PerSource gives each source address (grouped as the rule's
|
||||
// GroupV4/GroupV6 say) its own allowance; PerDestination each
|
||||
// destination.
|
||||
PerSource bool `json:"per_source,omitempty"`
|
||||
PerDestination bool `json:"per_destination,omitempty"`
|
||||
// Reset: "" (only by hand), daily, weekly (on Mondays) or monthly (on
|
||||
// Day, 1-31: a short month's last day when it has no such day), at
|
||||
// midnight.
|
||||
Reset string `json:"reset,omitempty"`
|
||||
Day int `json:"day,omitempty"`
|
||||
}
|
||||
|
||||
// Quota resets.
|
||||
var QuotaResets = []string{"daily", "weekly", "monthly"}
|
||||
|
||||
// PerAddress reports whether a quota (or limit) counts each address on its
|
||||
// own, and which: "saddr", "daddr" or "".
|
||||
func (q *Quota) PerAddress() string {
|
||||
switch {
|
||||
case q == nil:
|
||||
return ""
|
||||
case q.PerSource:
|
||||
return "saddr"
|
||||
case q.PerDestination:
|
||||
return "daddr"
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// PerAddress: as Quota's.
|
||||
func (l *Limit) PerAddress() string {
|
||||
switch {
|
||||
case l == nil:
|
||||
return ""
|
||||
case l.PerSource:
|
||||
return "saddr"
|
||||
case l.PerDestination:
|
||||
return "daddr"
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// CountsTraffic reports whether a rule measures traffic (a quota or a byte
|
||||
// rate): it must see every packet of a connection, so it is checked before
|
||||
// the built-in that accepts established connections.
|
||||
func (r *Rule) CountsTraffic() bool {
|
||||
if r.Raw != "" {
|
||||
return false
|
||||
}
|
||||
if r.Quota != nil {
|
||||
return true
|
||||
}
|
||||
if r.Limit == nil {
|
||||
return false
|
||||
}
|
||||
_, bytes := ByteUnits[r.Limit.Unit]
|
||||
return bytes
|
||||
}
|
||||
|
||||
// ChecksEveryPacket reports whether a rule is checked before the built-in
|
||||
// that accepts established connections: it counts traffic, or asks to.
|
||||
func (r *Rule) ChecksEveryPacket() bool {
|
||||
return r.Raw == "" && (r.EveryPacket || r.CountsTraffic())
|
||||
}
|
||||
|
||||
// Groups returns a rule's address grouping, defaults applied.
|
||||
func (r *Rule) Groups() (v4, v6 int) {
|
||||
v4, v6 = r.GroupV4, r.GroupV6
|
||||
if v4 == 0 {
|
||||
v4 = 32
|
||||
}
|
||||
if v6 == 0 {
|
||||
v6 = 64
|
||||
}
|
||||
return v4, v6
|
||||
}
|
||||
|
||||
// PortForward publishes a service on an inside host: incoming connections
|
||||
|
||||
@@ -31,6 +31,15 @@ type Settings struct {
|
||||
// ProbationSeconds is how long a change made over the network has to
|
||||
// be confirmed before it is rolled back (0: the default).
|
||||
ProbationSeconds int `json:"probation_seconds,omitempty"`
|
||||
// ManualConfirm: changes made over the network wait for a person to
|
||||
// confirm them. Otherwise the client that made a change confirms it by
|
||||
// itself once it reaches the API again over a new connection, from the
|
||||
// same address: proof that it can still manage the firewall.
|
||||
ManualConfirm bool `json:"manual_confirm,omitempty"`
|
||||
|
||||
// StatsPerDevice counts forwarded traffic to and from external zones
|
||||
// per inside address (IPv6 per /64), for the statistics' top devices.
|
||||
StatsPerDevice bool `json:"stats_per_device,omitempty"`
|
||||
}
|
||||
|
||||
// Builtins are rules Housecarl adds itself, shown as "added by Housecarl".
|
||||
|
||||
+242
-6
@@ -8,11 +8,13 @@
|
||||
package nft
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
@@ -24,6 +26,7 @@ import (
|
||||
|
||||
"housecarl/internal/model"
|
||||
"housecarl/internal/render"
|
||||
"housecarl/internal/trace"
|
||||
)
|
||||
|
||||
// Runner runs nft.
|
||||
@@ -196,11 +199,62 @@ type Table struct {
|
||||
Counters map[string]Counter
|
||||
// Elements by set name (both families together).
|
||||
Elements map[string][]render.Element
|
||||
// Quotas: the rules' quotas' used bytes (quota_r12), and the
|
||||
// per-address quotas' entries by set (quota_r12_v4).
|
||||
QuotaUsage
|
||||
// DeviceSets are the per-device traffic counters by set
|
||||
// (stats_up_v4, stats_down_v6 …): each address's packets and bytes.
|
||||
DeviceSets map[string][]render.Element
|
||||
// Since is when the counters started (housecarld's first apply).
|
||||
Since time.Time
|
||||
// Tables are all the machine's tables ("inet main"), when the whole
|
||||
// ruleset was read.
|
||||
Tables []string
|
||||
// Fallback: the table is the boot fallback (render.Fallback), not a
|
||||
// ruleset housecarld applied.
|
||||
Fallback bool
|
||||
}
|
||||
|
||||
// QuotaUsage is what the quotas used: kept across reloads, and saved in
|
||||
// the database so a reboot doesn't reset it.
|
||||
type QuotaUsage struct {
|
||||
Quotas map[string]int64 `json:"quotas,omitempty"`
|
||||
QuotaSets map[string][]render.Element `json:"quota_sets,omitempty"`
|
||||
}
|
||||
|
||||
// merge takes the larger usage of each quota and entry from o (what was
|
||||
// saved before a reboot, when the kernel's count started again).
|
||||
func (u *QuotaUsage) merge(o *QuotaUsage) {
|
||||
if o == nil {
|
||||
return
|
||||
}
|
||||
if u.Quotas == nil {
|
||||
u.Quotas = map[string]int64{}
|
||||
}
|
||||
if u.QuotaSets == nil {
|
||||
u.QuotaSets = map[string][]render.Element{}
|
||||
}
|
||||
for k, v := range o.Quotas {
|
||||
if v > u.Quotas[k] {
|
||||
u.Quotas[k] = v
|
||||
}
|
||||
}
|
||||
for set, es := range o.QuotaSets {
|
||||
for _, e := range es {
|
||||
found := false
|
||||
for i := range u.QuotaSets[set] {
|
||||
if u.QuotaSets[set][i].Addr == e.Addr {
|
||||
found = true
|
||||
if e.Used > u.QuotaSets[set][i].Used {
|
||||
u.QuotaSets[set][i].Used = e.Used
|
||||
}
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
u.QuotaSets[set] = append(u.QuotaSets[set], e)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Counter is packets and bytes.
|
||||
@@ -223,9 +277,25 @@ type jsonSet struct {
|
||||
Elem []json.RawMessage `json:"elem"`
|
||||
}
|
||||
|
||||
// jsonQuota is a quota's state: a named quota ({"bytes", "used"}, in
|
||||
// bytes) or one in an element ({"val", "val_unit", "used", "used_unit"}).
|
||||
type jsonQuota struct {
|
||||
Name string `json:"name"`
|
||||
Used int64 `json:"used"`
|
||||
UsedUnit string `json:"used_unit"`
|
||||
}
|
||||
|
||||
func (q jsonQuota) bytes() int64 {
|
||||
if m, ok := model.ByteUnits[q.UsedUnit]; ok {
|
||||
return q.Used * m
|
||||
}
|
||||
return q.Used
|
||||
}
|
||||
|
||||
// ParseTable reads `nft -j list table inet housecarl`.
|
||||
func ParseTable(data []byte) (*Table, error) {
|
||||
t := &Table{Counters: map[string]Counter{}, Elements: map[string][]render.Element{}}
|
||||
t := &Table{Counters: map[string]Counter{}, Elements: map[string][]render.Element{}, DeviceSets: map[string][]render.Element{},
|
||||
QuotaUsage: QuotaUsage{Quotas: map[string]int64{}, QuotaSets: map[string][]render.Element{}}}
|
||||
if len(data) == 0 {
|
||||
return t, nil
|
||||
}
|
||||
@@ -240,6 +310,9 @@ func ParseTable(data []byte) (*Table, error) {
|
||||
continue
|
||||
}
|
||||
mark, _, _ := strings.Cut(strings.TrimPrefix(r.Comment, "hc:"), " ")
|
||||
if strings.HasPrefix(mark, render.FallbackMark+"-") {
|
||||
t.Fallback = true
|
||||
}
|
||||
for _, e := range r.Expr {
|
||||
var c struct {
|
||||
Counter *Counter `json:"counter"`
|
||||
@@ -252,11 +325,46 @@ func ParseTable(data []byte) (*Table, error) {
|
||||
}
|
||||
}
|
||||
}
|
||||
if raw, ok := item["quota"]; ok {
|
||||
var q jsonQuota
|
||||
if json.Unmarshal(raw, &q) == nil && strings.HasPrefix(q.Name, "quota_r") {
|
||||
t.Quotas[q.Name] = q.bytes()
|
||||
}
|
||||
}
|
||||
if raw, ok := item["set"]; ok {
|
||||
var s jsonSet
|
||||
if json.Unmarshal(raw, &s) != nil {
|
||||
continue
|
||||
}
|
||||
if strings.HasPrefix(s.Name, "quota_r") {
|
||||
for _, e := range s.Elem {
|
||||
var w struct {
|
||||
Elem struct {
|
||||
Val string `json:"val"`
|
||||
Quota *jsonQuota `json:"quota"`
|
||||
} `json:"elem"`
|
||||
}
|
||||
if json.Unmarshal(e, &w) == nil && w.Elem.Quota != nil {
|
||||
t.QuotaSets[s.Name] = append(t.QuotaSets[s.Name], render.Element{Addr: w.Elem.Val, Used: w.Elem.Quota.bytes()})
|
||||
}
|
||||
}
|
||||
continue
|
||||
}
|
||||
if render.IsDeviceSet(s.Name) {
|
||||
for _, e := range s.Elem {
|
||||
var w struct {
|
||||
Elem struct {
|
||||
Val string `json:"val"`
|
||||
Expires int `json:"expires"`
|
||||
Counter *Counter `json:"counter"`
|
||||
} `json:"elem"`
|
||||
}
|
||||
if json.Unmarshal(e, &w) == nil && w.Elem.Counter != nil {
|
||||
t.DeviceSets[s.Name] = append(t.DeviceSets[s.Name], render.Element{Addr: w.Elem.Val, Timeout: w.Elem.Expires, Packets: w.Elem.Counter.Packets, Bytes: w.Elem.Counter.Bytes})
|
||||
}
|
||||
}
|
||||
continue
|
||||
}
|
||||
name := strings.TrimSuffix(strings.TrimSuffix(s.Name, "_v4"), "_v6")
|
||||
if name == s.Name || strings.HasPrefix(s.Name, "meter_") || strings.HasPrefix(s.Name, "conn_") {
|
||||
continue
|
||||
@@ -307,6 +415,66 @@ type Firewall struct {
|
||||
mu sync.Mutex
|
||||
base map[string]Counter
|
||||
since time.Time
|
||||
// restore is the quota usage saved before housecarld (or the machine)
|
||||
// stopped, put back by the next apply.
|
||||
restore *QuotaUsage
|
||||
// flowDevs are the flowtable's devices in the ruleset in force, by
|
||||
// name, with their interface index then (0: left out, missing).
|
||||
flowDevs map[string]int
|
||||
}
|
||||
|
||||
// ifIndex is an interface's index, 0 if it doesn't exist (tests change it).
|
||||
var ifIndex = func(name string) int {
|
||||
if i, err := net.InterfaceByName(name); err == nil {
|
||||
return i.Index
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
// FlowtableStale reports whether the flowtable in force no longer matches
|
||||
// the machine: a device of st's flowtable appeared, went, or came back as
|
||||
// a new interface (ppp0 after PPPoE reconnects; the kernel drops a
|
||||
// device from the flowtable when it goes). Applying again fixes it.
|
||||
func (f *Firewall) FlowtableStale(st *model.State) bool {
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
if f.flowDevs == nil {
|
||||
return false
|
||||
}
|
||||
for _, d := range st.Settings.FlowtableDevices {
|
||||
if ifIndex(d) != f.flowDevs[d] {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// RestoreQuotas has the next apply put this saved quota usage back where
|
||||
// the kernel's count is lower (after a reboot, the boot ruleset starts
|
||||
// every quota at zero).
|
||||
func (f *Firewall) RestoreQuotas(u *QuotaUsage) {
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
f.restore = u
|
||||
}
|
||||
|
||||
// ResetQuota starts a rule's quota again: the named quota, or the
|
||||
// per-address quota sets (both families).
|
||||
func (f *Firewall) ResetQuota(ctx context.Context, ruleID int64, perAddress bool) error {
|
||||
name := render.QuotaName(ruleID)
|
||||
if !perAddress {
|
||||
_, err := f.Runner.run(ctx, "", "reset", "quota", "inet", render.Table, name)
|
||||
return err
|
||||
}
|
||||
for _, fam := range []string{"_v4", "_v6"} {
|
||||
if _, err := f.Runner.run(ctx, "", "flush", "set", "inet", render.Table, name+fam); err != nil {
|
||||
if e, ok := err.(*Error); ok && strings.Contains(e.Output, "No such file or directory") {
|
||||
continue // the rule has no such family
|
||||
}
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (f *Firewall) options() render.Options {
|
||||
@@ -337,9 +505,25 @@ func (f *Firewall) Apply(ctx context.Context, st *model.State) error {
|
||||
old = t
|
||||
}
|
||||
}
|
||||
usage := QuotaUsage{}
|
||||
if old != nil {
|
||||
usage = old.QuotaUsage
|
||||
}
|
||||
usage.merge(f.restore)
|
||||
opt.Quotas, opt.QuotaSets = usage.Quotas, usage.QuotaSets
|
||||
if old != nil {
|
||||
opt.DeviceSets = old.DeviceSets
|
||||
}
|
||||
if err := f.Runner.Apply(ctx, render.Render(st, opt)); err != nil {
|
||||
return err
|
||||
}
|
||||
f.restore = nil
|
||||
f.flowDevs = map[string]int{}
|
||||
for _, d := range st.Settings.FlowtableDevices {
|
||||
if opt.Interfaces == nil || opt.Interfaces[d] {
|
||||
f.flowDevs[d] = ifIndex(d)
|
||||
}
|
||||
}
|
||||
if f.base == nil {
|
||||
f.base, f.since = map[string]Counter{}, time.Now()
|
||||
}
|
||||
@@ -354,20 +538,37 @@ func (f *Firewall) Apply(ctx context.Context, st *model.State) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// SaveBoot writes the ruleset of st for the boot unit, atomically.
|
||||
// SaveBoot writes the ruleset of st for the boot unit, and the fallback
|
||||
// it loads if that fails (FallbackFile), atomically. The boot ruleset has
|
||||
// no flowtable: it loads before the network comes up, when its devices
|
||||
// (ppp0, wg0) don't exist yet and nftables would refuse the whole file;
|
||||
// housecarld adds it once they do.
|
||||
func (f *Firewall) SaveBoot(st *model.State) error {
|
||||
if f.BootFile == "" {
|
||||
return nil
|
||||
}
|
||||
rs := render.Render(st, f.options())
|
||||
tmp := f.BootFile + ".tmp"
|
||||
if err := os.MkdirAll(filepath.Dir(f.BootFile), 0o750); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := os.WriteFile(tmp, []byte(rs.Text), 0o640); err != nil {
|
||||
if err := writeAtomic(FallbackFile(f.BootFile), render.Fallback(st)); err != nil {
|
||||
return err
|
||||
}
|
||||
return os.Rename(tmp, f.BootFile)
|
||||
rs := render.Render(st, render.Options{Interfaces: map[string]bool{}})
|
||||
return writeAtomic(f.BootFile, rs.Text)
|
||||
}
|
||||
|
||||
// FallbackFile is where the fallback for a boot ruleset is saved:
|
||||
// ruleset.nft's is ruleset-fallback.nft beside it.
|
||||
func FallbackFile(boot string) string {
|
||||
return strings.TrimSuffix(boot, ".nft") + "-fallback.nft"
|
||||
}
|
||||
|
||||
func writeAtomic(path, text string) error {
|
||||
tmp := path + ".tmp"
|
||||
if err := os.WriteFile(tmp, []byte(text), 0o640); err != nil {
|
||||
return err
|
||||
}
|
||||
return os.Rename(tmp, path)
|
||||
}
|
||||
|
||||
// Table reads Housecarl's table back, with counters totalled since
|
||||
@@ -422,6 +623,41 @@ func (f *Firewall) ApplyScript(ctx context.Context, script string) error {
|
||||
return f.Runner.ApplyScript(ctx, script)
|
||||
}
|
||||
|
||||
// Trace follows packets: it starts `nft monitor trace`, puts the tracing
|
||||
// table (script) in, and passes each line to line until ctx ends; then
|
||||
// it takes the table out again, whatever happened.
|
||||
func (f *Firewall) Trace(ctx context.Context, script string, line func(string)) error {
|
||||
c := f.Runner.cmd(ctx, "monitor", "trace")
|
||||
out, err := c.StdoutPipe()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var errb bytes.Buffer
|
||||
c.Stderr = &errb
|
||||
if err := c.Start(); err != nil {
|
||||
return err
|
||||
}
|
||||
defer f.Runner.run(context.Background(), "", "delete", "table", "inet", trace.Table)
|
||||
if err := f.Runner.ApplyScript(ctx, script); err != nil {
|
||||
c.Process.Kill()
|
||||
c.Wait()
|
||||
return fmt.Errorf("starting the trace: %w", err)
|
||||
}
|
||||
sc := bufio.NewScanner(out)
|
||||
sc.Buffer(make([]byte, 64<<10), 1<<20)
|
||||
for sc.Scan() {
|
||||
line(sc.Text())
|
||||
}
|
||||
err = c.Wait()
|
||||
if ctx.Err() != nil {
|
||||
return nil // stopped, or its time was up
|
||||
}
|
||||
if err != nil && errb.Len() > 0 {
|
||||
return fmt.Errorf("nft monitor trace: %s", strings.TrimSpace(errb.String()))
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
// DeleteTable deletes another table (takeover).
|
||||
func (f *Firewall) DeleteTable(ctx context.Context, family, name string) error {
|
||||
if name == render.Table && family == "inet" {
|
||||
|
||||
@@ -2,8 +2,10 @@ package nft
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
@@ -125,3 +127,241 @@ func TestFirewallInContainer(t *testing.T) {
|
||||
t.Errorf("another table was touched: %s", out)
|
||||
}
|
||||
}
|
||||
|
||||
// TestQuotasInContainer: quotas keep what they used across a reload,
|
||||
// reset by hand, and come back after a reboot from what was saved.
|
||||
func TestQuotasInContainer(t *testing.T) {
|
||||
image := os.Getenv("HOUSECARL_NFT_IMAGE")
|
||||
if image == "" {
|
||||
t.Skip("HOUSECARL_NFT_IMAGE not set")
|
||||
}
|
||||
name := "housecarl-nft-quota-test"
|
||||
exec.Command("docker", "rm", "-f", name).Run()
|
||||
if out, err := exec.Command("docker", "run", "-d", "--name", name, "--cap-add", "NET_ADMIN", image, "sleep", "300").CombinedOutput(); err != nil {
|
||||
t.Fatalf("starting the container: %v %s", err, out)
|
||||
}
|
||||
t.Cleanup(func() { exec.Command("docker", "rm", "-f", name).Run() })
|
||||
fw := &Firewall{Runner: &Runner{Command: []string{"docker", "exec", "-i", name, "nft"}}}
|
||||
ctx := context.Background()
|
||||
st := model.NewState()
|
||||
// A total quota on pings to 127.0.0.2, and one per destination on
|
||||
// pings to 127.0.0.3.
|
||||
st.Rules = []*model.Rule{
|
||||
{ID: 1, Chain: model.ChainOutput, Position: 1, Action: model.ActionDrop, Match: model.Match{Destinations: []string{"127.0.0.2"}},
|
||||
Quota: &model.Quota{Bytes: 10 << 20, Over: true}},
|
||||
{ID: 2, Chain: model.ChainOutput, Position: 2, Action: model.ActionDrop, Match: model.Match{Destinations: []string{"127.0.0.3"}},
|
||||
Quota: &model.Quota{Bytes: 10 << 20, Over: true, PerDestination: true}},
|
||||
}
|
||||
if err := fw.Apply(ctx, st); err != nil {
|
||||
t.Fatalf("apply: %v", err)
|
||||
}
|
||||
ping := func(addr string) {
|
||||
exec.Command("docker", "exec", name, "ping", "-c", "3", "-s", "1000", "-W", "1", addr).Run()
|
||||
}
|
||||
ping("127.0.0.2")
|
||||
ping("127.0.0.3")
|
||||
used := func() (int64, int64) {
|
||||
tb, err := fw.Table(ctx)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var dev int64
|
||||
for _, e := range tb.QuotaSets["quota_r2_v4"] {
|
||||
if e.Addr == "127.0.0.3" {
|
||||
dev = e.Used
|
||||
}
|
||||
}
|
||||
return tb.Quotas["quota_r1"], dev
|
||||
}
|
||||
total, dev := used()
|
||||
if total < 3000 || dev < 3000 {
|
||||
t.Fatalf("used: total %d, per destination %d", total, dev)
|
||||
}
|
||||
// A reload (a change) keeps what was used.
|
||||
st.Rules[0].Comment = "changed"
|
||||
if err := fw.Apply(ctx, st); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if t2, d2 := used(); t2 < total || d2 < dev {
|
||||
t.Errorf("after a reload: total %d (was %d), per destination %d (was %d)", t2, total, d2, dev)
|
||||
}
|
||||
// Reset by hand.
|
||||
if err := fw.ResetQuota(ctx, 1, false); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := fw.ResetQuota(ctx, 2, true); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if t3, d3 := used(); t3 != 0 || d3 != 0 {
|
||||
t.Errorf("after a reset: total %d, per destination %d", t3, d3)
|
||||
}
|
||||
// A reboot: the table is gone (the boot ruleset has no usage); the
|
||||
// saved usage comes back.
|
||||
ping("127.0.0.2")
|
||||
ping("127.0.0.3")
|
||||
tb, _ := fw.Table(ctx)
|
||||
saved := tb.QuotaUsage
|
||||
exec.Command("docker", "exec", name, "nft", "delete", "table", "inet", "housecarl").Run()
|
||||
fw.RestoreQuotas(&saved)
|
||||
if err := fw.Apply(ctx, st); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if t4, d4 := used(); t4 < saved.Quotas["quota_r1"] || t4 == 0 || d4 == 0 {
|
||||
t.Errorf("after a reboot: total %d, per destination %d (saved %+v)", t4, d4, saved)
|
||||
}
|
||||
}
|
||||
|
||||
// TestDeviceCountsInContainer: the per-device counters are read back and
|
||||
// survive a reload.
|
||||
func TestDeviceCountsInContainer(t *testing.T) {
|
||||
image := os.Getenv("HOUSECARL_NFT_IMAGE")
|
||||
if image == "" {
|
||||
t.Skip("HOUSECARL_NFT_IMAGE not set")
|
||||
}
|
||||
name := "housecarl-nft-stats-test"
|
||||
exec.Command("docker", "rm", "-f", name).Run()
|
||||
if out, err := exec.Command("docker", "run", "-d", "--name", name, "--cap-add", "NET_ADMIN", image, "sleep", "300").CombinedOutput(); err != nil {
|
||||
t.Fatalf("starting the container: %v %s", err, out)
|
||||
}
|
||||
t.Cleanup(func() { exec.Command("docker", "rm", "-f", name).Run() })
|
||||
fw := &Firewall{Runner: &Runner{Command: []string{"docker", "exec", "-i", name, "nft"}}}
|
||||
ctx := context.Background()
|
||||
st := model.NewState()
|
||||
st.Zones = []*model.Zone{{ID: 1, Name: "wan", Interfaces: []string{"eth0"}, External: true}}
|
||||
st.Settings.StatsPerDevice = true
|
||||
if err := fw.Apply(ctx, st); err != nil {
|
||||
t.Fatalf("apply: %v", err)
|
||||
}
|
||||
if out, err := exec.Command("docker", "exec", name, "nft", "add", "element", "inet", "housecarl", "stats_up_v4", "{ 10.0.10.5 counter packets 7 bytes 7000 }").CombinedOutput(); err != nil {
|
||||
t.Fatalf("adding an element: %v %s", err, out)
|
||||
}
|
||||
read := func() render.Element {
|
||||
tb, err := fw.Table(ctx)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, e := range tb.DeviceSets["stats_up_v4"] {
|
||||
if e.Addr == "10.0.10.5" {
|
||||
return e
|
||||
}
|
||||
}
|
||||
t.Fatalf("no 10.0.10.5 in %+v", tb.DeviceSets)
|
||||
return render.Element{}
|
||||
}
|
||||
if e := read(); e.Packets != 7 || e.Bytes != 7000 || e.Timeout <= 0 {
|
||||
t.Fatalf("read back %+v", e)
|
||||
}
|
||||
st.Settings.InputPolicy = model.PolicyAccept // a change: a reload
|
||||
if err := fw.Apply(ctx, st); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if e := read(); e.Packets != 7 || e.Bytes != 7000 {
|
||||
t.Errorf("after a reload: %+v", e)
|
||||
}
|
||||
}
|
||||
|
||||
// TestFlowtableStale: the flowtable in force goes stale when one of its
|
||||
// devices appears, goes, or comes back as a new interface.
|
||||
func TestFlowtableStale(t *testing.T) {
|
||||
idx := map[string]int{"eth1": 3}
|
||||
old := ifIndex
|
||||
ifIndex = func(name string) int { return idx[name] }
|
||||
t.Cleanup(func() { ifIndex = old })
|
||||
exists := func() map[string]bool {
|
||||
out := map[string]bool{}
|
||||
for n, i := range idx {
|
||||
out[n] = i > 0
|
||||
}
|
||||
return out
|
||||
}
|
||||
// "true" stands in for nft: every apply succeeds, and lists nothing.
|
||||
fw := &Firewall{Runner: &Runner{Command: []string{"true"}}, Interfaces: exists}
|
||||
st := model.NewState()
|
||||
st.Settings.FlowtableDevices = []string{"ppp0", "eth1"}
|
||||
if fw.FlowtableStale(st) {
|
||||
t.Error("stale before anything was applied")
|
||||
}
|
||||
if err := fw.Apply(context.Background(), st); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if fw.FlowtableStale(st) {
|
||||
t.Error("stale right after applying")
|
||||
}
|
||||
idx["ppp0"] = 7 // PPPoE came up
|
||||
if !fw.FlowtableStale(st) {
|
||||
t.Error("ppp0 appeared, yet not stale")
|
||||
}
|
||||
fw.Apply(context.Background(), st)
|
||||
idx["ppp0"] = 9 // and reconnected: a new interface by the same name
|
||||
if !fw.FlowtableStale(st) {
|
||||
t.Error("ppp0 came back as a new interface, yet not stale")
|
||||
}
|
||||
fw.Apply(context.Background(), st)
|
||||
idx["eth9"] = 4 // not a flowtable device
|
||||
if fw.FlowtableStale(st) {
|
||||
t.Error("stale over an interface the flowtable doesn't use")
|
||||
}
|
||||
}
|
||||
|
||||
// TestBootInContainer: the boot ruleset loads although its flowtable
|
||||
// devices don't exist yet, the fallback loads and is recognised, and
|
||||
// housecarld's apply replaces it.
|
||||
func TestBootInContainer(t *testing.T) {
|
||||
image := os.Getenv("HOUSECARL_NFT_IMAGE")
|
||||
if image == "" {
|
||||
t.Skip("HOUSECARL_NFT_IMAGE not set")
|
||||
}
|
||||
name := "housecarl-nft-boot-test"
|
||||
exec.Command("docker", "rm", "-f", name).Run()
|
||||
if out, err := exec.Command("docker", "run", "-d", "--name", name, "--cap-add", "NET_ADMIN", image, "sleep", "300").CombinedOutput(); err != nil {
|
||||
t.Fatalf("starting the container: %v %s", err, out)
|
||||
}
|
||||
t.Cleanup(func() { exec.Command("docker", "rm", "-f", name).Run() })
|
||||
nft := func(script string) error {
|
||||
c := exec.Command("docker", "exec", "-i", name, "nft", "-f", "/dev/stdin")
|
||||
c.Stdin = strings.NewReader(script)
|
||||
if out, err := c.CombinedOutput(); err != nil {
|
||||
return fmt.Errorf("%v: %s", err, out)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
dir := t.TempDir()
|
||||
// ppp0 exists when housecarld saves, but not at boot.
|
||||
fw := &Firewall{Runner: &Runner{Command: []string{"docker", "exec", "-i", name, "nft"}}, BootFile: filepath.Join(dir, "ruleset.nft"),
|
||||
Interfaces: func() map[string]bool { return map[string]bool{"ppp0": true, "eth0": true} }}
|
||||
st := model.NewState()
|
||||
st.Zones = []*model.Zone{{ID: 1, Name: "wan", Interfaces: []string{"ppp0"}, External: true}, {ID: 2, Name: "lan", Interfaces: []string{"eth0"}}}
|
||||
st.SNATs = []*model.SourceNAT{{ID: 3, Match: model.Match{OutZones: []string{"wan"}}}}
|
||||
st.Settings.FlowtableDevices = []string{"ppp0", "eth0"}
|
||||
if err := fw.SaveBoot(st); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
boot, _ := os.ReadFile(fw.BootFile)
|
||||
if strings.Contains(string(boot), "flowtable") {
|
||||
t.Errorf("the boot ruleset has a flowtable:\n%s", boot)
|
||||
}
|
||||
fallback, err := os.ReadFile(filepath.Join(dir, "ruleset-fallback.nft"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := nft(string(boot)); err != nil {
|
||||
t.Fatalf("the boot ruleset doesn't load: %v", err)
|
||||
}
|
||||
ctx := context.Background()
|
||||
if tb, err := fw.Table(ctx); err != nil || tb.Fallback {
|
||||
t.Errorf("after the boot ruleset: %+v %v", tb, err)
|
||||
}
|
||||
if err := nft(string(fallback)); err != nil {
|
||||
t.Fatalf("the fallback doesn't load: %v", err)
|
||||
}
|
||||
if tb, err := fw.Table(ctx); err != nil || !tb.Fallback {
|
||||
t.Errorf("the fallback isn't recognised: %+v %v", tb, err)
|
||||
}
|
||||
fw.Interfaces = func() map[string]bool { return map[string]bool{"eth0": true} }
|
||||
if err := fw.Apply(ctx, st); err != nil {
|
||||
t.Fatalf("apply over the fallback: %v", err)
|
||||
}
|
||||
if tb, err := fw.Table(ctx); err != nil || tb.Fallback {
|
||||
t.Errorf("the fallback stayed after an apply: %+v %v", tb, err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,89 @@
|
||||
package render
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"housecarl/internal/model"
|
||||
)
|
||||
|
||||
// FallbackMark starts the comment of every fallback rule ("hc:fallback-…"):
|
||||
// how housecarld tells the fallback is in force.
|
||||
const FallbackMark = "fallback"
|
||||
|
||||
// Fallback builds the ruleset housecarl-ruleset.service loads at boot when
|
||||
// the saved one fails to: nothing new gets in from the external zones'
|
||||
// interfaces, everything else is let through, and IPv4 leaving an
|
||||
// external zone is masqueraded (when any source NAT is configured), so the
|
||||
// inside keeps working until housecarld puts the real ruleset back. With
|
||||
// no external zone it can't tell which interface faces the internet, and
|
||||
// lets nothing but loopback through.
|
||||
//
|
||||
// It is deliberately small: no sets, flowtable, meters or quotas, and
|
||||
// interfaces by name only, so it loads before they exist.
|
||||
func Fallback(st *model.State) string {
|
||||
var b strings.Builder
|
||||
line := func(format string, args ...any) {
|
||||
fmt.Fprintf(&b, format, args...)
|
||||
b.WriteByte('\n')
|
||||
}
|
||||
// rule writes a counted rule: what it matches, then its verdict.
|
||||
rule := func(name, match, verdict string) {
|
||||
line("\t\t%s counter %s comment \"hc:%s-%s\"", match, verdict, FallbackMark, name)
|
||||
}
|
||||
ext := st.ExternalInterfaces()
|
||||
line("# Housecarl's fallback: loaded at boot only if ruleset.nft fails to load,")
|
||||
line("# until housecarld starts and puts the real ruleset back.")
|
||||
line("table inet %s {}", Table)
|
||||
line("delete table inet %s", Table)
|
||||
line("table inet %s {", Table)
|
||||
if len(ext) == 0 {
|
||||
// Nothing says which interface faces the internet: close it all.
|
||||
for _, c := range []string{"input", "forward"} {
|
||||
line("\tchain %s {", c)
|
||||
line("\t\ttype filter hook %s priority filter; policy drop;", c)
|
||||
line("\t\tiifname \"lo\" accept")
|
||||
line("\t\tcounter comment \"hc:%s-%s\"", FallbackMark, c)
|
||||
line("\t}")
|
||||
}
|
||||
line("}")
|
||||
return b.String()
|
||||
}
|
||||
e := ifaces(ext)
|
||||
line("\tchain input {")
|
||||
line("\t\ttype filter hook input priority filter; policy accept;")
|
||||
line("\t\tiifname \"lo\" accept")
|
||||
line("\t\tct state established,related accept")
|
||||
rule("icmpv6", fmt.Sprintf("iifname %s icmpv6 type { %s, echo-request, echo-reply }", e, icmpv6Errors), "accept")
|
||||
rule("icmpv6-nd", fmt.Sprintf("iifname %s icmpv6 type { %s } ip6 hoplimit 255", e, icmpv6ND), "accept")
|
||||
rule("icmpv6-mld", fmt.Sprintf("iifname %s ip6 saddr fe80::/10 icmpv6 type { %s }", e, icmpv6MLD), "accept")
|
||||
rule("dhcpv6-client", fmt.Sprintf("iifname %s ip6 saddr fe80::/10 udp dport 546", e), "accept")
|
||||
rule("input", "iifname "+e, "drop")
|
||||
line("\t}")
|
||||
line("\tchain forward {")
|
||||
line("\t\ttype filter hook forward priority filter; policy accept;")
|
||||
if st.Settings.Builtins.MSSClamp {
|
||||
line("\t\toifname %s tcp flags syn tcp option maxseg size set rt mtu", e)
|
||||
}
|
||||
line("\t\tct state established,related accept")
|
||||
rule("forward", "iifname "+e, "drop")
|
||||
line("\t}")
|
||||
if masquerades(st) {
|
||||
line("\tchain srcnat {")
|
||||
line("\t\ttype nat hook postrouting priority srcnat; policy accept;")
|
||||
rule("masquerade", "meta nfproto ipv4 oifname "+e, "masquerade")
|
||||
line("\t}")
|
||||
}
|
||||
line("}")
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// masquerades reports whether any source NAT is configured.
|
||||
func masquerades(st *model.State) bool {
|
||||
for _, x := range st.SNATs {
|
||||
if !x.Disabled {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -0,0 +1,75 @@
|
||||
package render
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"os/exec"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"housecarl/internal/model"
|
||||
)
|
||||
|
||||
func TestFallback(t *testing.T) {
|
||||
st := router()
|
||||
text := Fallback(st)
|
||||
for _, want := range []string{
|
||||
"delete table inet housecarl",
|
||||
"type filter hook input priority filter; policy accept;",
|
||||
`iifname "wan" counter drop comment "hc:fallback-input"`,
|
||||
`iifname "wan" counter drop comment "hc:fallback-forward"`,
|
||||
`iifname "wan" ip6 saddr fe80::/10 udp dport 546 counter accept`,
|
||||
`ip6 hoplimit 255 counter accept`,
|
||||
`oifname "wan" tcp flags syn tcp option maxseg size set rt mtu`,
|
||||
`meta nfproto ipv4 oifname "wan" counter masquerade comment "hc:fallback-masquerade"`,
|
||||
} {
|
||||
if !strings.Contains(text, want) {
|
||||
t.Errorf("no %q in\n%s", want, text)
|
||||
}
|
||||
}
|
||||
// Nothing that could fail to load before the network is up.
|
||||
for _, not := range []string{"flowtable", "@", "\tset ", "enp2s0", "wg0"} {
|
||||
if strings.Contains(text, not) {
|
||||
t.Errorf("%q in the fallback:\n%s", not, text)
|
||||
}
|
||||
}
|
||||
// Established connections are accepted before the drops.
|
||||
if strings.Index(text, "ct state established,related accept") > strings.Index(text, "hc:fallback-input") {
|
||||
t.Errorf("established after the drop:\n%s", text)
|
||||
}
|
||||
|
||||
// No source NAT: no masquerade.
|
||||
st.SNATs = nil
|
||||
if text := Fallback(st); strings.Contains(text, "masquerade") {
|
||||
t.Errorf("masquerade without source NAT:\n%s", text)
|
||||
}
|
||||
|
||||
// No external zone: everything but loopback is dropped.
|
||||
for _, z := range st.Zones {
|
||||
z.External = false
|
||||
}
|
||||
text = Fallback(st)
|
||||
if strings.Count(text, "policy drop;") != 2 || strings.Contains(text, "policy accept") || strings.Contains(text, "established") {
|
||||
t.Errorf("no external zone:\n%s", text)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFallbackPassesNft(t *testing.T) {
|
||||
cmd := strings.Fields(os.Getenv("HOUSECARL_NFT"))
|
||||
if len(cmd) == 0 {
|
||||
t.Skip("HOUSECARL_NFT not set")
|
||||
}
|
||||
closed := router()
|
||||
for _, z := range closed.Zones {
|
||||
z.External = false
|
||||
}
|
||||
wild := router()
|
||||
wild.Zones = append(wild.Zones, &model.Zone{ID: 99, Name: "lte", Interfaces: []string{"wwan*"}, External: true})
|
||||
for name, st := range map[string]*model.State{"router": router(), "closed": closed, "wildcard": wild} {
|
||||
c := exec.CommandContext(context.Background(), cmd[0], append(cmd[1:], "-c", "-f", "/dev/stdin")...)
|
||||
c.Stdin = strings.NewReader(Fallback(st))
|
||||
if out, err := c.CombinedOutput(); err != nil {
|
||||
t.Errorf("%s: nft -c: %v\n%s\n%s", name, err, out, Fallback(st))
|
||||
}
|
||||
}
|
||||
}
|
||||
+189
-18
@@ -12,6 +12,7 @@ package render
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"net/netip"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
@@ -33,14 +34,31 @@ type Options struct {
|
||||
// Elements are dynamic sets' current entries, re-added in the same
|
||||
// transaction so a reload keeps them: set name → entries.
|
||||
Elements map[string][]Element
|
||||
// Quotas are the rules' quotas' used bytes (quota name, quota_r12),
|
||||
// and QuotaSets the per-address quotas' entries (set name,
|
||||
// quota_r12_v4), so a reload keeps what was used.
|
||||
Quotas map[string]int64
|
||||
QuotaSets map[string][]Element
|
||||
// DeviceSets are the per-device traffic counters' entries
|
||||
// (stats_up_v4 …), so a reload keeps them.
|
||||
DeviceSets map[string][]Element
|
||||
}
|
||||
|
||||
// Element is a dynamic set's entry with its remaining lifetime.
|
||||
// Element is a dynamic set's entry with its remaining lifetime, or a
|
||||
// per-address quota's entry with what it used.
|
||||
type Element struct {
|
||||
Addr string `json:"address"`
|
||||
Timeout int `json:"timeout,omitempty"` // seconds left (0: none)
|
||||
Used int64 `json:"used,omitempty"` // bytes
|
||||
// Packets and Bytes are a per-device counter's.
|
||||
Packets uint64 `json:"packets,omitempty"`
|
||||
Bytes uint64 `json:"bytes,omitempty"`
|
||||
}
|
||||
|
||||
// QuotaName is the name of a rule's quota: the named quota of a total
|
||||
// quota, and (with _v4, _v6) the sets of a per-address one.
|
||||
func QuotaName(ruleID int64) string { return fmt.Sprintf("quota_r%d", ruleID) }
|
||||
|
||||
// Ruleset is the rendered nftables script.
|
||||
type Ruleset struct {
|
||||
Text string
|
||||
@@ -86,7 +104,8 @@ type renderer struct {
|
||||
st *model.State
|
||||
opt Options
|
||||
w *writer
|
||||
// meters are the per-source limit and connection sets the rules need.
|
||||
// meters are the per-address limit, connection and quota sets, and
|
||||
// the named quotas, the rules need.
|
||||
meters []string
|
||||
}
|
||||
|
||||
@@ -97,6 +116,7 @@ func (r *renderer) render() {
|
||||
w.emit("", "delete table inet %s", Table)
|
||||
w.emit("", "table inet %s {", Table)
|
||||
r.sets()
|
||||
r.deviceSets()
|
||||
devices := r.flowtableDevices()
|
||||
if len(devices) > 0 {
|
||||
w.emit("built-in: flowtable", "\tflowtable ft { hook ingress priority filter; devices = { %s }; }", strings.Join(quoteAll(devices), ", "))
|
||||
@@ -177,6 +197,56 @@ func (r *renderer) elements() {
|
||||
}
|
||||
}
|
||||
|
||||
// IsDeviceSet reports whether a set is a per-device traffic counter.
|
||||
func IsDeviceSet(name string) bool {
|
||||
return strings.HasPrefix(name, "stats_up_") || strings.HasPrefix(name, "stats_down_")
|
||||
}
|
||||
|
||||
// deviceSets declares the per-device traffic counters, with what they
|
||||
// counted before a reload.
|
||||
func (r *renderer) deviceSets() {
|
||||
if !r.st.Settings.StatsPerDevice || len(r.st.ExternalInterfaces()) == 0 {
|
||||
return
|
||||
}
|
||||
for _, dir := range []string{"up", "down"} {
|
||||
for _, v6 := range []bool{false, true} {
|
||||
name := setName("stats_"+dir, v6)
|
||||
var elems []string
|
||||
for _, e := range r.opt.DeviceSets[name] {
|
||||
a, err := netip.ParseAddr(e.Addr)
|
||||
if err != nil || a.Is6() != v6 {
|
||||
continue
|
||||
}
|
||||
timeout := ""
|
||||
if e.Timeout > 0 {
|
||||
timeout = fmt.Sprintf(" timeout %ds", e.Timeout)
|
||||
}
|
||||
elems = append(elems, fmt.Sprintf("%s%s counter packets %d bytes %d", a, timeout, e.Packets, e.Bytes))
|
||||
}
|
||||
list := ""
|
||||
if len(elems) > 0 {
|
||||
list = " elements = { " + strings.Join(elems, ", ") + " }"
|
||||
}
|
||||
r.w.emit("built-in: stats", "\tset %s { type %s; flags dynamic,timeout; timeout 1h; size 65535; counter;%s }", name, addrType(v6), list)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// deviceCounts counts forwarded traffic per inside address: what leaves
|
||||
// for an external zone by its source, what comes from one by its
|
||||
// destination (IPv6 by /64).
|
||||
func (r *renderer) deviceCounts() {
|
||||
ext := r.st.ExternalInterfaces()
|
||||
if !r.st.Settings.StatsPerDevice || len(ext) == 0 {
|
||||
return
|
||||
}
|
||||
e := ifaces(ext)
|
||||
r.builtin("stats", "iifname != %s oifname %s update @stats_up_v4 { ip saddr }", e, e)
|
||||
r.builtin("stats", "iifname != %s oifname %s update @stats_up_v6 { ip6 saddr and ffff:ffff:ffff:ffff:: }", e, e)
|
||||
r.builtin("stats", "iifname %s oifname != %s update @stats_down_v4 { ip daddr }", e, e)
|
||||
r.builtin("stats", "iifname %s oifname != %s update @stats_down_v6 { ip6 daddr and ffff:ffff:ffff:ffff:: }", e, e)
|
||||
}
|
||||
|
||||
// SetName is the nftables name of a set's family part.
|
||||
func SetName(name string, v6 bool) string { return setName(name, v6) }
|
||||
|
||||
@@ -238,7 +308,8 @@ func (r *renderer) early() {
|
||||
r.builtin("antispoof6", "iifname %s ip6 saddr { %s } counter drop", ifaces(ext), strings.Join(v6, ", "))
|
||||
}
|
||||
}
|
||||
r.rules(model.ChainEarly)
|
||||
r.rules(model.ChainEarly, true)
|
||||
r.rules(model.ChainEarly, false)
|
||||
r.end()
|
||||
}
|
||||
|
||||
@@ -265,6 +336,9 @@ func (r *renderer) input() {
|
||||
if b.Loopback {
|
||||
r.builtin("loopback", "iifname \"lo\" accept")
|
||||
}
|
||||
// Rules that count traffic see every packet: before the established
|
||||
// connections are accepted.
|
||||
r.rules(model.ChainInput, true)
|
||||
r.connBuiltins()
|
||||
ext := st.ExternalInterfaces()
|
||||
if b.ICMPv6 {
|
||||
@@ -282,7 +356,7 @@ func (r *renderer) input() {
|
||||
if b.DHCPv6Client && len(ext) > 0 {
|
||||
r.builtin("dhcpv6-client", "iifname %s ip6 saddr fe80::/10 udp dport 546 counter accept", ifaces(ext))
|
||||
}
|
||||
r.rules(model.ChainInput)
|
||||
r.rules(model.ChainInput, false)
|
||||
r.policyTail("input", st.Settings.InputPolicy)
|
||||
r.end()
|
||||
}
|
||||
@@ -295,6 +369,8 @@ func (r *renderer) forward(flowtable bool) {
|
||||
if b.MSSClamp && len(ext) > 0 {
|
||||
r.builtin("mss-clamp", "oifname %s tcp flags syn tcp option maxseg size set rt mtu", ifaces(ext))
|
||||
}
|
||||
r.rules(model.ChainForward, true)
|
||||
r.deviceCounts()
|
||||
if flowtable {
|
||||
r.builtin("flowtable", "meta l4proto { tcp, udp } flow add @ft")
|
||||
}
|
||||
@@ -303,7 +379,7 @@ func (r *renderer) forward(flowtable bool) {
|
||||
r.builtin("icmpv6", "icmpv6 type { %s } counter accept", icmpv6Errors)
|
||||
}
|
||||
r.forwardAccepts()
|
||||
r.rules(model.ChainForward)
|
||||
r.rules(model.ChainForward, false)
|
||||
r.policyTail("forward", st.Settings.ForwardPolicy)
|
||||
r.end()
|
||||
}
|
||||
@@ -311,6 +387,7 @@ func (r *renderer) forward(flowtable bool) {
|
||||
func (r *renderer) output() {
|
||||
st := r.st
|
||||
r.chain("output", "output", "filter", st.Settings.OutputPolicy)
|
||||
r.rules(model.ChainOutput, true)
|
||||
if st.Settings.OutputPolicy == model.PolicyDrop {
|
||||
if st.Settings.Builtins.Loopback {
|
||||
r.builtin("loopback", "oifname \"lo\" accept")
|
||||
@@ -322,7 +399,7 @@ func (r *renderer) output() {
|
||||
r.builtin("icmpv6", "icmpv6 type { %s, echo-request, echo-reply, %s, %s } counter accept", icmpv6Errors, icmpv6ND, icmpv6MLD)
|
||||
}
|
||||
}
|
||||
r.rules(model.ChainOutput)
|
||||
r.rules(model.ChainOutput, false)
|
||||
r.policyTail("output", st.Settings.OutputPolicy)
|
||||
r.end()
|
||||
}
|
||||
@@ -341,9 +418,11 @@ func (r *renderer) policyTail(chain, policy string) {
|
||||
|
||||
// ---- rules
|
||||
|
||||
func (r *renderer) rules(chain string) {
|
||||
// rules writes a chain's rules: those that count traffic (traffic true),
|
||||
// or the others.
|
||||
func (r *renderer) rules(chain string, traffic bool) {
|
||||
for _, rule := range r.st.Rules {
|
||||
if rule.Chain != chain || rule.Disabled {
|
||||
if rule.Chain != chain || rule.Disabled || rule.ChecksEveryPacket() != traffic {
|
||||
continue
|
||||
}
|
||||
obj := rule.Label()
|
||||
@@ -370,8 +449,8 @@ func (r *renderer) rules(chain string) {
|
||||
tail = append(tail, hours(rule.HourFrom, rule.HourTo))
|
||||
}
|
||||
verdict := r.verdict(rule, mark)
|
||||
perSource := rule.ConnLimit > 0 || rule.Limit != nil && rule.Limit.PerSource
|
||||
for _, line := range r.expand(rule.Match, perSource) {
|
||||
perAddress := rule.ConnLimit > 0 || rule.Limit.PerAddress() != "" || rule.Quota.PerAddress() != ""
|
||||
for _, line := range r.expand(rule.Match, perAddress) {
|
||||
parts := append([]string{}, line.parts...)
|
||||
parts = append(parts, tail...)
|
||||
if l := rule.Limit; l != nil {
|
||||
@@ -380,6 +459,9 @@ func (r *renderer) rules(chain string) {
|
||||
if rule.ConnLimit > 0 {
|
||||
parts = append(parts, r.connLimit(rule, line))
|
||||
}
|
||||
if rule.Quota != nil {
|
||||
parts = append(parts, r.quota(rule, line))
|
||||
}
|
||||
parts = append(parts, verdict)
|
||||
r.w.emit(obj, "\t\t%s", strings.Join(nonEmpty(parts), " "))
|
||||
}
|
||||
@@ -410,20 +492,51 @@ func (r *renderer) verdict(rule *model.Rule, mark string) string {
|
||||
return strings.Join(v, " ")
|
||||
}
|
||||
|
||||
// limit renders a rate limit, per source with a meter set per family.
|
||||
// limit renders a rate limit: of packets or bytes; per source or
|
||||
// destination with a meter set per family, whose entries are refreshed
|
||||
// by each packet (update) and forgotten once a source is quiet.
|
||||
func (r *renderer) limit(rule *model.Rule, line expanded, l *model.Limit) string {
|
||||
rate := fmt.Sprintf("limit rate %s%d/%s", map[bool]string{true: "over ", false: ""}[l.Over], l.Rate, l.Per)
|
||||
if l.Burst > 0 {
|
||||
rate += fmt.Sprintf(" burst %d packets", l.Burst)
|
||||
unit := ""
|
||||
if _, ok := model.ByteUnits[l.Unit]; ok {
|
||||
unit = " " + l.Unit
|
||||
}
|
||||
if !l.PerSource {
|
||||
rate := fmt.Sprintf("limit rate %s%d%s/%s", map[bool]string{true: "over ", false: ""}[l.Over], l.Rate, unit, l.Per)
|
||||
if l.Burst > 0 {
|
||||
burstUnit := " packets"
|
||||
if unit != "" {
|
||||
burstUnit = unit
|
||||
}
|
||||
rate += fmt.Sprintf(" burst %d%s", l.Burst, burstUnit)
|
||||
}
|
||||
addr := l.PerAddress()
|
||||
if addr == "" {
|
||||
return rate
|
||||
}
|
||||
name := fmt.Sprintf("meter_r%d_%s", rule.ID, line.fam)
|
||||
if !r.hasMeter(name) {
|
||||
r.meters = append(r.meters, fmt.Sprintf("\tset %s { type %s; flags dynamic,timeout; timeout 1%s; }", name, addrType(line.fam == "v6"), map[string]string{"second": "m", "minute": "m", "hour": "h", "day": "d"}[l.Per]))
|
||||
}
|
||||
return fmt.Sprintf("add @%s { %s saddr %s }", name, ipKeyword(line.fam), rate)
|
||||
return fmt.Sprintf("update @%s { %s %s }", name, r.addrKey(rule, line.fam, addr), rate)
|
||||
}
|
||||
|
||||
// addrKey is what a per-address meter counts by: "ip saddr", or grouped
|
||||
// by a prefix, "ip6 saddr and ffff:ffff:ffff:ffff::".
|
||||
func (r *renderer) addrKey(rule *model.Rule, fam, addr string) string {
|
||||
v4, v6 := rule.Groups()
|
||||
bits, size := v4, 32
|
||||
if fam == "v6" {
|
||||
bits, size = v6, 128
|
||||
}
|
||||
key := ipKeyword(fam) + " " + addr
|
||||
if bits >= size {
|
||||
return key
|
||||
}
|
||||
mask := make([]byte, size/8)
|
||||
for i := 0; i < bits; i++ {
|
||||
mask[i/8] |= 0x80 >> (i % 8)
|
||||
}
|
||||
m, _ := netip.AddrFromSlice(mask)
|
||||
return key + " and " + m.String()
|
||||
}
|
||||
|
||||
func (r *renderer) connLimit(rule *model.Rule, line expanded) string {
|
||||
@@ -431,12 +544,70 @@ func (r *renderer) connLimit(rule *model.Rule, line expanded) string {
|
||||
if !r.hasMeter(name) {
|
||||
r.meters = append(r.meters, fmt.Sprintf("\tset %s { type %s; flags dynamic; size 65535; }", name, addrType(line.fam == "v6")))
|
||||
}
|
||||
return fmt.Sprintf("add @%s { %s saddr ct count over %d }", name, ipKeyword(line.fam), rule.ConnLimit)
|
||||
return fmt.Sprintf("add @%s { %s ct count over %d }", name, r.addrKey(rule, line.fam, "saddr"), rule.ConnLimit)
|
||||
}
|
||||
|
||||
// quota renders a rule's quota: a named quota shared by the rule's lines
|
||||
// (both families count against one allowance), or per address a set of
|
||||
// quotas per family. What was used comes back from Options, so a reload
|
||||
// doesn't reset it.
|
||||
func (r *renderer) quota(rule *model.Rule, line expanded) string {
|
||||
q := rule.Quota
|
||||
amount := quotaAmount(q)
|
||||
name := QuotaName(rule.ID)
|
||||
addr := q.PerAddress()
|
||||
if addr == "" {
|
||||
if !r.hasMeter(name) {
|
||||
used := ""
|
||||
if u := r.opt.Quotas[name]; u > 0 {
|
||||
used = fmt.Sprintf(" used %d bytes", u)
|
||||
}
|
||||
r.meters = append(r.meters, fmt.Sprintf("\tquota %s { %s%s }", name, amount, used))
|
||||
}
|
||||
return fmt.Sprintf("quota name %s", quote(name))
|
||||
}
|
||||
set := name + "_" + line.fam
|
||||
if !r.hasMeter(set) {
|
||||
var elems []string
|
||||
for _, e := range r.opt.QuotaSets[set] {
|
||||
a, err := netip.ParseAddr(e.Addr)
|
||||
if err != nil || a.Is6() != (line.fam == "v6") {
|
||||
continue
|
||||
}
|
||||
used := ""
|
||||
if e.Used > 0 {
|
||||
used = fmt.Sprintf(" used %d bytes", e.Used)
|
||||
}
|
||||
elems = append(elems, fmt.Sprintf("%s quota %s%s", a, amount, used))
|
||||
}
|
||||
list := ""
|
||||
if len(elems) > 0 {
|
||||
list = " elements = { " + strings.Join(elems, ", ") + " }"
|
||||
}
|
||||
r.meters = append(r.meters, fmt.Sprintf("\tset %s { type %s; flags dynamic; size 65535;%s }", set, addrType(line.fam == "v6"), list))
|
||||
}
|
||||
return fmt.Sprintf("add @%s { %s quota %s }", set, r.addrKey(rule, line.fam, addr), amount)
|
||||
}
|
||||
|
||||
// quotaAmount writes a quota: "over 5000 mbytes" (nftables 1.1.3 has no
|
||||
// gbytes).
|
||||
func quotaAmount(q *model.Quota) string {
|
||||
n, unit := q.Bytes, "bytes"
|
||||
for _, u := range []string{"mbytes", "kbytes"} {
|
||||
if n >= model.ByteUnits[u] && n%model.ByteUnits[u] == 0 {
|
||||
n, unit = n/model.ByteUnits[u], u
|
||||
break
|
||||
}
|
||||
}
|
||||
if q.Over {
|
||||
return fmt.Sprintf("over %d %s", n, unit)
|
||||
}
|
||||
return fmt.Sprintf("until %d %s", n, unit)
|
||||
}
|
||||
|
||||
func (r *renderer) hasMeter(name string) bool {
|
||||
for _, m := range r.meters {
|
||||
if strings.Contains(m, "set "+name+" ") {
|
||||
if strings.Contains(m, "set "+name+" ") || strings.Contains(m, "quota "+name+" ") {
|
||||
return true
|
||||
}
|
||||
}
|
||||
|
||||
@@ -66,6 +66,17 @@ func router() *model.State {
|
||||
rule(model.ChainForward, model.Rule{Match: model.Match{Destinations: []string{"wg_net"}}, Action: model.ActionDrop})
|
||||
rule(model.ChainForward, model.Rule{Match: model.Match{Services: []string{"ping"}}})
|
||||
rule(model.ChainForward, model.Rule{Match: model.Match{Sources: []string{"wg_net", "trusted", "guests"}}})
|
||||
// Bandwidth: a byte rate, per destination; per /24; quotas, total and
|
||||
// per device.
|
||||
rule(model.ChainForward, model.Rule{Match: model.Match{OutZones: []string{"lan"}}, Action: model.ActionDrop,
|
||||
Limit: &model.Limit{Rate: 5, Unit: "mbytes", Per: "second", Burst: 1, Over: true, PerDestination: true}, Comment: "5 MB/s each"})
|
||||
rule(model.ChainForward, model.Rule{Match: model.Match{InZones: []string{"wan"}}, Action: model.ActionDrop, GroupV4: 24,
|
||||
Limit: &model.Limit{Rate: 100, Unit: "kbytes", Per: "minute", Over: true, PerSource: true}, Comment: "per /24"})
|
||||
rule(model.ChainForward, model.Rule{Match: model.Match{Sources: []string{"guests"}, OutZones: []string{"wan"}}, Action: model.ActionDrop,
|
||||
Quota: &model.Quota{Bytes: 50000 << 20, Over: true, Reset: "monthly", Day: 15}, Comment: "guest quota"})
|
||||
rule(model.ChainForward, model.Rule{Match: model.Match{InZones: []string{"lan"}, OutZones: []string{"wan"}}, Action: model.ActionDrop,
|
||||
Quota: &model.Quota{Bytes: 5 << 30, Over: true, PerSource: true, Reset: "daily"}, Comment: "device quota"})
|
||||
rule(model.ChainForward, model.Rule{Match: model.Match{Sources: []string{"10.0.10.66"}}, Action: model.ActionReject, EveryPacket: true, Comment: "blocked now"})
|
||||
rule(model.ChainEarly, model.Rule{Match: model.Match{InZones: []string{"wan"}, Sources: []string{"198.51.100.0/24"}}, Action: model.ActionDrop})
|
||||
rule(model.ChainOutput, model.Rule{Raw: "meta skuid 0 counter accept", Comment: "raw"})
|
||||
st.Forwards = []*model.PortForward{
|
||||
@@ -87,8 +98,26 @@ func router() *model.State {
|
||||
return st
|
||||
}
|
||||
|
||||
// quotaOptions carries quota usage, as a reload does: the guest quota's,
|
||||
// and two devices' of the per-device quota.
|
||||
func quotaOptions(st *model.State) (map[string]int64, map[string][]Element) {
|
||||
q, sets := map[string]int64{}, map[string][]Element{}
|
||||
for _, r := range st.Rules {
|
||||
switch r.Comment {
|
||||
case "guest quota":
|
||||
q[QuotaName(r.ID)] = 123456
|
||||
case "device quota":
|
||||
sets[QuotaName(r.ID)+"_v4"] = []Element{{Addr: "10.0.10.20", Used: 1 << 20}}
|
||||
sets[QuotaName(r.ID)+"_v6"] = []Element{{Addr: "2a02:8013:7f3d:10::", Used: 2048}}
|
||||
}
|
||||
}
|
||||
return q, sets
|
||||
}
|
||||
|
||||
func TestRenderRouter(t *testing.T) {
|
||||
rs := Render(router(), Options{Elements: map[string][]Element{"guests": {{Addr: "10.0.30.5", Timeout: 1700}}}})
|
||||
st := router()
|
||||
q, qs := quotaOptions(st)
|
||||
rs := Render(st, Options{Elements: map[string][]Element{"guests": {{Addr: "10.0.30.5", Timeout: 1700}}}, Quotas: q, QuotaSets: qs})
|
||||
text := rs.Text
|
||||
for _, want := range []string{
|
||||
"table inet housecarl {}\ndelete table inet housecarl\ntable inet housecarl {",
|
||||
@@ -108,7 +137,16 @@ func TestRenderRouter(t *testing.T) {
|
||||
`ip saddr @trusted_v4 tcp dport { 22, 8053, 8067 } counter accept`,
|
||||
`ip6 saddr @trusted_v6 tcp dport { 22, 8053, 8067 } counter accept`,
|
||||
`iifname "enp2s0" ip6 saddr fe80::/10 udp dport 547 counter accept`,
|
||||
`iifname "wan" meta nfproto ipv4 tcp dport 22 add @meter_r`,
|
||||
`iifname "wan" meta nfproto ipv4 tcp dport 22 update @meter_r`,
|
||||
`ip6 saddr and ffff:ffff:ffff:ffff:: limit rate over 3/minute burst 5 packets }`,
|
||||
`oifname "enp2s0" meta nfproto ipv4 update @meter_r`,
|
||||
`{ ip daddr limit rate over 5 mbytes/second burst 1 mbytes } counter drop comment "hc:r`,
|
||||
`{ ip saddr and 255.255.255.0 limit rate over 100 kbytes/minute } counter drop`,
|
||||
`quota name "quota_r`,
|
||||
`{ over 50000 mbytes used 123456 bytes }`,
|
||||
`flags dynamic; size 65535; elements = { 10.0.10.20 quota over 5120 mbytes used 1048576 bytes } }`,
|
||||
`elements = { 2a02:8013:7f3d:10:: quota over 5120 mbytes used 2048 bytes } }`,
|
||||
`{ ip6 saddr and ffff:ffff:ffff:ffff:: quota over 5120 mbytes } counter drop comment "hc:r`,
|
||||
`ip saddr limit rate over 3/minute burst 5 packets } counter log prefix "hc input #`,
|
||||
`tcp dport 22 add @conn_r`,
|
||||
`counter reject with tcp reset comment`,
|
||||
@@ -139,6 +177,19 @@ func TestRenderRouter(t *testing.T) {
|
||||
t.Errorf("missing %q", want)
|
||||
}
|
||||
}
|
||||
// Rules that count traffic see every packet: in forward, after the
|
||||
// MSS clamp and before established connections are accepted.
|
||||
fwd := text[strings.Index(text, "chain forward {"):]
|
||||
quota, est, mss := strings.Index(fwd, `quota name "quota_r`), strings.Index(fwd, "hc:b-established"), strings.Index(fwd, "hc:b-mss-clamp")
|
||||
if quota < 0 || !(mss < quota && quota < est) {
|
||||
t.Errorf("the quota rule isn't between the MSS clamp and established (%d, %d, %d)", mss, quota, est)
|
||||
}
|
||||
if bw := strings.Index(fwd, "limit rate over 5 mbytes/second"); bw > est {
|
||||
t.Error("the byte rate rule comes after established")
|
||||
}
|
||||
if now := strings.Index(fwd, "blocked now"); now < 0 || now > est {
|
||||
t.Error("the every-packet rule comes after established")
|
||||
}
|
||||
// The trusted masquerade has no IPv6 part: masquerading the IPv6 LAN
|
||||
// isn't wanted (its sources are global).
|
||||
if strings.Contains(text, "ip6 saddr @trusted_v6 counter masquerade") {
|
||||
@@ -155,6 +206,31 @@ func TestRenderRouter(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestRenderDeviceCounts(t *testing.T) {
|
||||
st := router()
|
||||
st.Settings.StatsPerDevice = true
|
||||
text := Render(st, Options{DeviceSets: map[string][]Element{"stats_up_v4": {{Addr: "10.0.10.5", Timeout: 3000, Packets: 7, Bytes: 7000}}}}).Text
|
||||
for _, want := range []string{
|
||||
`set stats_up_v4 { type ipv4_addr; flags dynamic,timeout; timeout 1h; size 65535; counter; elements = { 10.0.10.5 timeout 3000s counter packets 7 bytes 7000 } }`,
|
||||
`set stats_down_v6 { type ipv6_addr; flags dynamic,timeout; timeout 1h; size 65535; counter; }`,
|
||||
`iifname != "wan" oifname "wan" update @stats_up_v4 { ip saddr } comment "hc:b-stats"`,
|
||||
`iifname "wan" oifname != "wan" update @stats_down_v6 { ip6 daddr and ffff:ffff:ffff:ffff:: } comment "hc:b-stats"`,
|
||||
} {
|
||||
if !strings.Contains(text, want) {
|
||||
t.Errorf("missing %q", want)
|
||||
}
|
||||
}
|
||||
// Counted before the established built-in, so every packet is.
|
||||
fwd := text[strings.Index(text, "chain forward {"):]
|
||||
if i := strings.Index(fwd, "update @stats_up_v4"); i < 0 || i > strings.Index(fwd, "hc:b-established") {
|
||||
t.Error("the counting comes after the established built-in")
|
||||
}
|
||||
st.Settings.StatsPerDevice = false
|
||||
if strings.Contains(Render(st, Options{}).Text, "stats_") {
|
||||
t.Error("counting without the setting")
|
||||
}
|
||||
}
|
||||
|
||||
// TestRenderedRulesetPassesNft has the real nft check the router's
|
||||
// ruleset (and a minimal one). Set HOUSECARL_NFT to how nft is run, e.g.
|
||||
//
|
||||
@@ -166,8 +242,20 @@ func TestRenderedRulesetPassesNft(t *testing.T) {
|
||||
if len(cmd) == 0 {
|
||||
t.Skip("HOUSECARL_NFT not set")
|
||||
}
|
||||
for name, st := range map[string]*model.State{"router": router(), "empty": model.NewState()} {
|
||||
rs := Render(st, Options{Elements: map[string][]Element{"guests": {{Addr: "10.0.30.5", Timeout: 1700}, {Addr: "2a02:8013:7f3d:30::5"}}}})
|
||||
stats := router()
|
||||
stats.Settings.StatsPerDevice = true
|
||||
// VLAN sub-interfaces: zones of one VLAN, of all of a parent's, and
|
||||
// the flowtable on one.
|
||||
vlan := router()
|
||||
vlan.Zones = append(vlan.Zones, &model.Zone{ID: 900, Name: "iot", Interfaces: []string{"enp2s0.20"}},
|
||||
&model.Zone{ID: 901, Name: "vlans", Interfaces: []string{"enp3s0.*"}})
|
||||
vlan.Rules = append(vlan.Rules, &model.Rule{ID: 902, Chain: model.ChainForward, Position: 99, Action: model.ActionAccept,
|
||||
Match: model.Match{InZones: []string{"iot", "vlans"}, OutZones: []string{"wan"}}})
|
||||
vlan.Settings.FlowtableDevices = []string{"lo", "enp2s0.20"}
|
||||
for name, st := range map[string]*model.State{"router": router(), "empty": model.NewState(), "stats": stats, "vlan": vlan} {
|
||||
q, qs := quotaOptions(st)
|
||||
rs := Render(st, Options{Elements: map[string][]Element{"guests": {{Addr: "10.0.30.5", Timeout: 1700}, {Addr: "2a02:8013:7f3d:30::5"}}}, Quotas: q, QuotaSets: qs,
|
||||
DeviceSets: map[string][]Element{"stats_up_v4": {{Addr: "10.0.10.5", Timeout: 3000, Packets: 7, Bytes: 7000}}, "stats_down_v6": {{Addr: "2001:db8:1:2::", Packets: 1, Bytes: 100}}}})
|
||||
c := exec.CommandContext(context.Background(), cmd[0], append(cmd[1:], "-c", "-f", "/dev/stdin")...)
|
||||
c.Stdin = strings.NewReader(rs.Text)
|
||||
if out, err := c.CombinedOutput(); err != nil {
|
||||
|
||||
@@ -0,0 +1,484 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"housecarl/internal/api"
|
||||
"housecarl/internal/auth"
|
||||
"housecarl/internal/engine"
|
||||
"housecarl/internal/model"
|
||||
"housecarl/internal/store"
|
||||
)
|
||||
|
||||
// Changesets: changes staged to be reviewed (validation, nftables' check,
|
||||
// the ruleset's diff) and applied together, as one history entry, or
|
||||
// discarded. The web UI stages its edits by default; the CLI and the API
|
||||
// stage when a changeset is named (?changeset=ID, X-Housecarl-Changeset).
|
||||
|
||||
// changesetParam is the changeset a request names (0: none).
|
||||
func changesetParam(r *http.Request) (int64, error) {
|
||||
v := r.URL.Query().Get("changeset")
|
||||
if v == "" {
|
||||
v = r.Header.Get("X-Housecarl-Changeset")
|
||||
}
|
||||
if v == "" {
|
||||
return 0, nil
|
||||
}
|
||||
id, err := strconv.ParseInt(v, 10, 64)
|
||||
if err != nil || id <= 0 {
|
||||
return 0, badRequest("invalid changeset %q", v)
|
||||
}
|
||||
return id, nil
|
||||
}
|
||||
|
||||
// notStaged refuses a changeset on a change that can't be staged (an
|
||||
// import, a takeover): it would otherwise be applied at once, unexpectedly.
|
||||
func notStaged(r *http.Request, what string) error {
|
||||
if id, err := changesetParam(r); err != nil || id == 0 {
|
||||
return err
|
||||
}
|
||||
return badRequest("%s can't be staged in a changeset; leave out --changeset (or unset HOUSECARL_CHANGESET) to do it now", what)
|
||||
}
|
||||
|
||||
// openChangeset loads a changeset to change: it must be open, and the
|
||||
// caller's own unless they are an administrator.
|
||||
func (s *Server) openChangeset(r *http.Request, p *auth.Principal, id int64) (*model.Changeset, error) {
|
||||
cs, err := s.store.GetChangeset(r.Context(), id)
|
||||
if errors.Is(err, store.ErrNotFound) {
|
||||
return nil, notFound("there is no changeset %d", id)
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if cs.Status != model.ChangesetOpen {
|
||||
return nil, conflict("changeset %d is %s", id, cs.Status)
|
||||
}
|
||||
if !ownsChangeset(p, cs) && !p.IsAdmin() {
|
||||
return nil, forbidden("changeset %d is %s's, not yours", id, cs.Username)
|
||||
}
|
||||
return cs, nil
|
||||
}
|
||||
|
||||
func ownsChangeset(p *auth.Principal, cs *model.Changeset) bool {
|
||||
if p == nil || p.User == nil {
|
||||
return false
|
||||
}
|
||||
if p.User.ID != 0 {
|
||||
return cs.UserID == p.User.ID
|
||||
}
|
||||
return cs.UserID == 0 && cs.Username == p.User.Username
|
||||
}
|
||||
|
||||
// stage adds an operation to a changeset. The changeset with it must
|
||||
// still be valid and pass nftables' check; nothing is applied.
|
||||
func (s *Server) stage(w http.ResponseWriter, r *http.Request, p *auth.Principal, id int64, op model.Op) error {
|
||||
s.csMu.Lock()
|
||||
defer s.csMu.Unlock()
|
||||
cs, err := s.openChangeset(r, p, id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
// The operation on its own, after the changeset so far: what it
|
||||
// changes, for its summary.
|
||||
before := s.eng.State()
|
||||
if err := replay(before, cs.Ops); err != nil {
|
||||
return conflict("changeset %d can't be applied as it is (%v); review it first", id, err)
|
||||
}
|
||||
after := before.Clone()
|
||||
objID, err := applyOp(after, &op)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
engine.Normalize(after)
|
||||
after.Sort()
|
||||
changes := engine.Diff(before, after)
|
||||
if len(changes) == 0 {
|
||||
writeJSON(w, http.StatusOK, api.MutationResponse{Changes: []model.Change{}, Changeset: cs})
|
||||
return nil
|
||||
}
|
||||
op.Summary = engine.Describe(changes)
|
||||
if op.Op == model.OpRevert {
|
||||
var h model.HistoryEntry
|
||||
json.Unmarshal(op.Data, &h)
|
||||
op.Summary = engine.UndoDescription(&h)
|
||||
}
|
||||
op.AddedAt = s.nowFn()
|
||||
for _, o := range cs.Ops {
|
||||
if o.Seq >= op.Seq {
|
||||
op.Seq = o.Seq + 1
|
||||
}
|
||||
}
|
||||
if op.Seq == 0 {
|
||||
op.Seq = 1
|
||||
}
|
||||
ops := append(append([]model.Op{}, cs.Ops...), op)
|
||||
dry := queryBool(r, "dry_run")
|
||||
// The whole changeset, checked as it would be applied.
|
||||
res, err := s.eng.Apply(r.Context(), p, "", engine.Options{DryRun: true, Via: via(r, p)}, func(st *model.State, _ func() int64) error {
|
||||
return replay(st, ops)
|
||||
})
|
||||
if err != nil {
|
||||
s.writeErr(w, r, err, res)
|
||||
return nil
|
||||
}
|
||||
if !dry {
|
||||
cs.Ops = ops
|
||||
if err := s.store.UpdateChangeset(r.Context(), cs); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
out := api.MutationResponse{Staged: !dry, DryRun: dry, Changes: changes, Warnings: res.Warnings, Changeset: cs}
|
||||
if object := objectOf(&op, objID); object != nil {
|
||||
if v := object(after); v != nil {
|
||||
out.Object, _ = json.Marshal(v)
|
||||
}
|
||||
}
|
||||
if dry {
|
||||
out.Diff = s.rulesetDiff(res.Before, res.After)
|
||||
}
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Server) createChangeset(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResConfig, true); err != nil {
|
||||
return err
|
||||
}
|
||||
var in api.ChangesetInput
|
||||
if r.ContentLength != 0 {
|
||||
if err := decodeJSON(r, &in); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
cs := &model.Changeset{Username: p.Name(), Via: via(r, p), Description: strings.TrimSpace(in.Description), Status: model.ChangesetOpen}
|
||||
if p.User != nil {
|
||||
cs.UserID = p.User.ID
|
||||
}
|
||||
if err := s.store.CreateChangeset(r.Context(), cs); err != nil {
|
||||
return err
|
||||
}
|
||||
s.eng.Audit(r.Context(), p, "changeset.create", fmt.Sprintf("changeset %d", cs.ID), "success", cs.Description)
|
||||
writeJSON(w, http.StatusCreated, cs)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Server) listChangesets(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResConfig, false); err != nil {
|
||||
return err
|
||||
}
|
||||
var uid int64
|
||||
if queryBool(r, "mine") {
|
||||
uid = -1
|
||||
if p.User != nil && p.User.ID != 0 {
|
||||
uid = p.User.ID
|
||||
}
|
||||
}
|
||||
list, err := s.store.ListChangesets(r.Context(), r.URL.Query().Get("status"), max(uid, 0), queryInt(r, "limit", 100))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if uid == -1 { // the socket's users have no user ID: by name
|
||||
mine := []*model.Changeset{}
|
||||
for _, cs := range list {
|
||||
if ownsChangeset(p, cs) {
|
||||
mine = append(mine, cs)
|
||||
}
|
||||
}
|
||||
list = mine
|
||||
}
|
||||
writeJSON(w, http.StatusOK, list)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Server) getChangeset(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResConfig, false); err != nil {
|
||||
return err
|
||||
}
|
||||
id, err := pathID(r, "id")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
cs, err := s.store.GetChangeset(r.Context(), id)
|
||||
if errors.Is(err, store.ErrNotFound) {
|
||||
return notFound("there is no changeset %d", id)
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
writeJSON(w, http.StatusOK, cs)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Server) updateChangeset(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResConfig, true); err != nil {
|
||||
return err
|
||||
}
|
||||
id, err := pathID(r, "id")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var in api.ChangesetInput
|
||||
if err := decodeJSON(r, &in); err != nil {
|
||||
return err
|
||||
}
|
||||
s.csMu.Lock()
|
||||
defer s.csMu.Unlock()
|
||||
cs, err := s.openChangeset(r, p, id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
cs.Description = strings.TrimSpace(in.Description)
|
||||
if err := s.store.UpdateChangeset(r.Context(), cs); err != nil {
|
||||
return err
|
||||
}
|
||||
writeJSON(w, http.StatusOK, cs)
|
||||
return nil
|
||||
}
|
||||
|
||||
// removeOp takes one operation out of a changeset. Later ones that
|
||||
// needed it show up in the preview.
|
||||
func (s *Server) removeOp(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResConfig, true); err != nil {
|
||||
return err
|
||||
}
|
||||
id, err := pathID(r, "id")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
seq, err := strconv.Atoi(r.PathValue("seq"))
|
||||
if err != nil {
|
||||
return badRequest("invalid operation %q", r.PathValue("seq"))
|
||||
}
|
||||
s.csMu.Lock()
|
||||
defer s.csMu.Unlock()
|
||||
cs, err := s.openChangeset(r, p, id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
kept := []model.Op{}
|
||||
for _, o := range cs.Ops {
|
||||
if o.Seq != seq {
|
||||
kept = append(kept, o)
|
||||
}
|
||||
}
|
||||
if len(kept) == len(cs.Ops) {
|
||||
return notFound("changeset %d has no operation %d", id, seq)
|
||||
}
|
||||
cs.Ops = kept
|
||||
if err := s.store.UpdateChangeset(r.Context(), cs); err != nil {
|
||||
return err
|
||||
}
|
||||
writeJSON(w, http.StatusOK, cs)
|
||||
return nil
|
||||
}
|
||||
|
||||
// previewChangeset says what applying a changeset would do, without
|
||||
// doing it: its changes, problems, nftables' check, the ruleset's diff.
|
||||
func (s *Server) previewChangeset(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResConfig, false); err != nil {
|
||||
return err
|
||||
}
|
||||
id, err := pathID(r, "id")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
cs, err := s.store.GetChangeset(r.Context(), id)
|
||||
if errors.Is(err, store.ErrNotFound) {
|
||||
return notFound("there is no changeset %d", id)
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
res, err := s.eng.Apply(r.Context(), p, "", engine.Options{DryRun: true, Via: via(r, p)}, func(st *model.State, _ func() int64) error {
|
||||
return replay(st, cs.Ops)
|
||||
})
|
||||
out := api.ChangesetPreview{Valid: err == nil && res != nil && len(res.Changes) > 0, Changes: []model.Change{}}
|
||||
if res != nil {
|
||||
if res.Changes != nil {
|
||||
out.Changes = res.Changes
|
||||
}
|
||||
out.Warnings = res.Warnings
|
||||
}
|
||||
if err != nil {
|
||||
out.Error = errorInfo(err)
|
||||
} else if res != nil {
|
||||
out.Diff = s.rulesetDiff(res.Before, res.After)
|
||||
}
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
return nil
|
||||
}
|
||||
|
||||
// errorInfo is an error as the API reports it.
|
||||
func errorInfo(err error) *api.ErrorInfo {
|
||||
var oe *opError
|
||||
if errors.As(err, &oe) {
|
||||
info, _ := oe.info()
|
||||
return &info
|
||||
}
|
||||
var he *httpError
|
||||
if errors.As(err, &he) {
|
||||
return &api.ErrorInfo{Code: he.code, Message: he.msg}
|
||||
}
|
||||
if ee, ok := engine.AsError(err); ok {
|
||||
return &api.ErrorInfo{Code: ee.Kind, Message: ee.Message, Problems: ee.Problems}
|
||||
}
|
||||
return &api.ErrorInfo{Code: "internal", Message: err.Error()}
|
||||
}
|
||||
|
||||
// applyChangeset applies a changeset's operations as one change: one
|
||||
// history entry, and (over the network) on probation like any change.
|
||||
func (s *Server) applyChangeset(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResConfig, true); err != nil {
|
||||
return err
|
||||
}
|
||||
id, err := pathID(r, "id")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var in api.ChangesetInput
|
||||
if r.ContentLength != 0 {
|
||||
if err := decodeJSON(r, &in); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
s.csMu.Lock()
|
||||
defer s.csMu.Unlock()
|
||||
cs, err := s.openChangeset(r, p, id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if d := strings.TrimSpace(in.Description); d != "" {
|
||||
cs.Description = d
|
||||
}
|
||||
if len(cs.Ops) == 0 {
|
||||
return badRequest("changeset %d has no changes to apply", id)
|
||||
}
|
||||
opts := engine.Options{Via: via(r, p)}
|
||||
desc := cs.Description
|
||||
for _, o := range cs.Ops {
|
||||
if o.Op == model.OpRevert {
|
||||
opts.RevertOf = o.ID // History links the undo
|
||||
if desc == "" && len(cs.Ops) == 1 {
|
||||
desc = o.Summary
|
||||
}
|
||||
}
|
||||
}
|
||||
res, err := s.eng.Apply(r.Context(), p, desc, opts, func(st *model.State, _ func() int64) error {
|
||||
return replay(st, cs.Ops)
|
||||
})
|
||||
if err != nil {
|
||||
s.eng.Audit(r.Context(), p, "changeset.apply", fmt.Sprintf("changeset %d", id), "failure", err.Error())
|
||||
s.writeErr(w, r, err, res)
|
||||
return nil
|
||||
}
|
||||
cs.Status, cs.HistoryID = model.ChangesetApplied, res.HistoryID
|
||||
if err := s.store.UpdateChangeset(r.Context(), cs); err != nil {
|
||||
return err
|
||||
}
|
||||
s.eng.Audit(r.Context(), p, "changeset.apply", fmt.Sprintf("changeset %d", id), "success", fmt.Sprintf("history %d", res.HistoryID))
|
||||
s.afterChange(r.Context(), p, res)
|
||||
out := s.mutation(res, nil)
|
||||
out.ConfirmToken = s.issueCheck(w, r, p, res)
|
||||
out.Changeset = cs
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Server) discardChangeset(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResConfig, true); err != nil {
|
||||
return err
|
||||
}
|
||||
id, err := pathID(r, "id")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
s.csMu.Lock()
|
||||
defer s.csMu.Unlock()
|
||||
cs, err := s.openChangeset(r, p, id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
cs.Status = model.ChangesetDiscarded
|
||||
if err := s.store.UpdateChangeset(r.Context(), cs); err != nil {
|
||||
return err
|
||||
}
|
||||
s.eng.Audit(r.Context(), p, "changeset.discard", fmt.Sprintf("changeset %d", id), "success", fmt.Sprintf("%d operations", len(cs.Ops)))
|
||||
writeJSON(w, http.StatusOK, cs)
|
||||
return nil
|
||||
}
|
||||
|
||||
// ---- reading the configuration as a changeset will make it
|
||||
|
||||
// viewState is the configuration a read shows: the current one, or with
|
||||
// a changeset named, as it will be once that is applied (draft true).
|
||||
// A changeset that no longer applies shows the current configuration,
|
||||
// with the reason in the X-Housecarl-Changeset-Error header.
|
||||
func (s *Server) viewState(w http.ResponseWriter, r *http.Request) (cur, view *model.State, draft bool, err error) {
|
||||
cur = s.eng.State()
|
||||
id, err := changesetParam(r)
|
||||
if err != nil || id == 0 {
|
||||
return cur, cur, false, err
|
||||
}
|
||||
cs, err := s.store.GetChangeset(r.Context(), id)
|
||||
if errors.Is(err, store.ErrNotFound) {
|
||||
return nil, nil, false, notFound("there is no changeset %d", id)
|
||||
}
|
||||
if err != nil {
|
||||
return nil, nil, false, err
|
||||
}
|
||||
if cs.Status != model.ChangesetOpen {
|
||||
return cur, cur, false, nil
|
||||
}
|
||||
next := cur.Clone()
|
||||
if err := replay(next, cs.Ops); err != nil {
|
||||
w.Header().Set("X-Housecarl-Changeset-Error", err.Error())
|
||||
return cur, cur, false, nil
|
||||
}
|
||||
engine.Normalize(next)
|
||||
next.Sort()
|
||||
return cur, next, true, nil
|
||||
}
|
||||
|
||||
// withPending lists a kind's objects as a changeset will leave them, each
|
||||
// marked "pending": "new", "changed" or "deleted" (deleted ones stay in
|
||||
// the list, in their place).
|
||||
func withPending(cur, next *model.State, kind string) []any {
|
||||
was := map[int64]string{}
|
||||
for _, o := range cur.Objects(kind) {
|
||||
b, _ := json.Marshal(o)
|
||||
was[o.GetID()] = string(b)
|
||||
}
|
||||
// The deleted ones go back in, so they sort into their places.
|
||||
merged := next.Clone()
|
||||
deleted := map[int64]bool{}
|
||||
for _, o := range cur.Objects(kind) {
|
||||
if merged.ByID(kind, o.GetID()) == nil {
|
||||
b, _ := json.Marshal(o)
|
||||
merged.Put(kind, o.GetID(), b)
|
||||
deleted[o.GetID()] = true
|
||||
}
|
||||
}
|
||||
merged.Sort()
|
||||
out := []any{}
|
||||
for _, o := range merged.Objects(kind) {
|
||||
var m map[string]any
|
||||
b, _ := json.Marshal(o)
|
||||
json.Unmarshal(b, &m)
|
||||
old, existed := was[o.GetID()]
|
||||
switch {
|
||||
case deleted[o.GetID()]:
|
||||
m["pending"] = "deleted"
|
||||
case !existed:
|
||||
m["pending"] = "new"
|
||||
case old != string(b):
|
||||
m["pending"] = "changed"
|
||||
}
|
||||
out = append(out, m)
|
||||
}
|
||||
return out
|
||||
}
|
||||
+215
-179
@@ -4,36 +4,62 @@ import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"net/http"
|
||||
"net/netip"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
udiff "github.com/aymanbagabas/go-udiff"
|
||||
|
||||
"housecarl/internal/api"
|
||||
"housecarl/internal/auth"
|
||||
"housecarl/internal/engine"
|
||||
"housecarl/internal/matrix"
|
||||
"housecarl/internal/model"
|
||||
"housecarl/internal/store"
|
||||
)
|
||||
|
||||
// change applies an edit through the engine and answers it; object picks
|
||||
// what to return from the new state. A network change starts (or joins)
|
||||
// the probation.
|
||||
func (s *Server) change(w http.ResponseWriter, r *http.Request, p *auth.Principal, description string, edit engine.Edit, object func(st *model.State) any) error {
|
||||
// change makes an operation: at once through the engine, or, with a
|
||||
// changeset named (?changeset=ID or X-Housecarl-Changeset), staged in it.
|
||||
// A network change starts (or joins) the probation.
|
||||
func (s *Server) change(w http.ResponseWriter, r *http.Request, p *auth.Principal, op model.Op) error {
|
||||
if err := require(p, auth.ResConfig, true); err != nil {
|
||||
return err
|
||||
}
|
||||
res, err := s.eng.Apply(r.Context(), p, description, engine.Options{DryRun: queryBool(r, "dry_run"), Via: via(r, p)}, edit)
|
||||
if op.Op == model.OpCreate && op.ID == 0 {
|
||||
ids, err := s.store.AllocateIDs(r.Context(), 1)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
op.ID = ids[0]
|
||||
}
|
||||
csID, err := changesetParam(r)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if csID != 0 {
|
||||
return s.stage(w, r, p, csID, op)
|
||||
}
|
||||
var id int64
|
||||
res, err := s.eng.Apply(r.Context(), p, "", engine.Options{DryRun: queryBool(r, "dry_run"), Via: via(r, p)}, func(st *model.State, _ func() int64) error {
|
||||
var err error
|
||||
id, err = applyOp(st, &op)
|
||||
return err
|
||||
})
|
||||
if err != nil {
|
||||
s.writeErr(w, r, err, res)
|
||||
return nil
|
||||
}
|
||||
s.afterChange(r.Context(), p, res)
|
||||
// POST to a collection creates; POST to an object (move) changes it.
|
||||
s.respondChange(w, r, res, object, r.Method == http.MethodPost && r.PathValue("ref") == "")
|
||||
out := s.mutation(res, objectOf(&op, id))
|
||||
out.ConfirmToken = s.issueCheck(w, r, p, res)
|
||||
writeJSON(w, s.mutationStatus(res, op.Op == model.OpCreate), out)
|
||||
return nil
|
||||
}
|
||||
|
||||
// respondChange answers a change: 201 when it created something.
|
||||
func (s *Server) respondChange(w http.ResponseWriter, r *http.Request, res *engine.Result, object func(st *model.State) any, creates bool) {
|
||||
// mutation is the answer to a change: what changed, warnings, the object
|
||||
// (as it is after the change), the probation, and for a dry run the
|
||||
// ruleset's diff.
|
||||
func (s *Server) mutation(res *engine.Result, object func(st *model.State) any) api.MutationResponse {
|
||||
out := api.MutationResponse{Applied: res.Applied, DryRun: res.DryRun, HistoryID: res.HistoryID, Changes: res.Changes, Warnings: res.Warnings}
|
||||
if out.Changes == nil {
|
||||
out.Changes = []model.Change{}
|
||||
@@ -41,16 +67,39 @@ func (s *Server) respondChange(w http.ResponseWriter, r *http.Request, res *engi
|
||||
if pr := s.probationStatus(); res.Applied && (len(pr.Pending) > 0 || pr.Takeovers > 0) {
|
||||
out.ConfirmBy = &pr.Deadline
|
||||
}
|
||||
if object != nil && (res.Applied || res.DryRun) {
|
||||
if v := object(s.eng.State()); v != nil {
|
||||
if object != nil && res.After != nil && (res.Applied || res.DryRun) {
|
||||
if v := object(res.After); v != nil {
|
||||
out.Object, _ = json.Marshal(v)
|
||||
}
|
||||
}
|
||||
status := http.StatusOK
|
||||
if res.Applied && creates {
|
||||
status = http.StatusCreated
|
||||
if res.DryRun && len(res.Changes) > 0 {
|
||||
out.Diff = s.rulesetDiff(res.Before, res.After)
|
||||
}
|
||||
writeJSON(w, status, out)
|
||||
return out
|
||||
}
|
||||
|
||||
// mutationStatus is 201 for an applied create, else 200.
|
||||
func (s *Server) mutationStatus(res *engine.Result, creates bool) int {
|
||||
if res.Applied && creates {
|
||||
return http.StatusCreated
|
||||
}
|
||||
return http.StatusOK
|
||||
}
|
||||
|
||||
// respondChange answers a change made without an operation (undo).
|
||||
func (s *Server) respondChange(w http.ResponseWriter, r *http.Request, p *auth.Principal, res *engine.Result, object func(st *model.State) any, creates bool) {
|
||||
out := s.mutation(res, object)
|
||||
out.ConfirmToken = s.issueCheck(w, r, p, res)
|
||||
writeJSON(w, s.mutationStatus(res, creates), out)
|
||||
}
|
||||
|
||||
// rulesetDiff is the unified diff of the rulesets two configurations
|
||||
// make: the file housecarl-ruleset.service loads, now and after.
|
||||
func (s *Server) rulesetDiff(before, after *model.State) string {
|
||||
if before == nil || after == nil {
|
||||
return ""
|
||||
}
|
||||
return udiff.Unified("ruleset.nft (now)", "ruleset.nft (after)", s.fw.Render(before).Text, s.fw.Render(after).Text)
|
||||
}
|
||||
|
||||
// kindOf maps the API's plural path names to kinds.
|
||||
@@ -84,17 +133,30 @@ func (s *Server) listObjects(w http.ResponseWriter, r *http.Request, p *auth.Pri
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
st := s.eng.State()
|
||||
cur, st, draft, err := s.viewState(w, r)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
out := []any{}
|
||||
q := r.URL.Query()
|
||||
c := r.URL.Query().Get("chain")
|
||||
if draft {
|
||||
// Marked new, changed or deleted.
|
||||
for _, o := range withPending(cur, st, kind) {
|
||||
if c != "" && kind == model.KindRule && o.(map[string]any)["chain"] != c {
|
||||
continue
|
||||
}
|
||||
out = append(out, o)
|
||||
}
|
||||
} else {
|
||||
for _, o := range st.Objects(kind) {
|
||||
if c := q.Get("chain"); c != "" {
|
||||
if c != "" {
|
||||
if rule, ok := o.(*model.Rule); ok && rule.Chain != c {
|
||||
continue
|
||||
}
|
||||
}
|
||||
out = append(out, o)
|
||||
}
|
||||
}
|
||||
// Services: the built-in ones too, unless overridden.
|
||||
if kind == model.KindService && !queryBool(r, "own") {
|
||||
for _, b := range model.BuiltinServices() {
|
||||
@@ -102,9 +164,14 @@ func (s *Server) listObjects(w http.ResponseWriter, r *http.Request, p *auth.Pri
|
||||
out = append(out, b)
|
||||
}
|
||||
}
|
||||
sort.SliceStable(out, func(i, j int) bool {
|
||||
return out[i].(*model.Service).Name < out[j].(*model.Service).Name
|
||||
})
|
||||
name := func(x any) string {
|
||||
if m, ok := x.(map[string]any); ok {
|
||||
n, _ := m["name"].(string)
|
||||
return n
|
||||
}
|
||||
return x.(*model.Service).Name
|
||||
}
|
||||
sort.SliceStable(out, func(i, j int) bool { return name(out[i]) < name(out[j]) })
|
||||
}
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
return nil
|
||||
@@ -118,7 +185,10 @@ func (s *Server) getObject(w http.ResponseWriter, r *http.Request, p *auth.Princ
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
st := s.eng.State()
|
||||
_, st, _, err := s.viewState(w, r)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
o, err := findObject(st, kind, r.PathValue("ref"))
|
||||
if err != nil && kind == model.KindService {
|
||||
if b := model.BuiltinService(r.PathValue("ref")); b != nil {
|
||||
@@ -235,29 +305,11 @@ func (s *Server) createObject(w http.ResponseWriter, r *http.Request, p *auth.Pr
|
||||
if err := decodeJSON(r, &raw); err != nil {
|
||||
return err
|
||||
}
|
||||
o, err := decodeObject(kind, raw)
|
||||
if err != nil {
|
||||
// Refuse a bad object now, not when it is applied.
|
||||
if _, err := decodeObject(kind, raw); err != nil {
|
||||
return err
|
||||
}
|
||||
if svc, ok := o.(*model.Service); ok {
|
||||
svc.Builtin = false
|
||||
}
|
||||
if rule, ok := o.(*model.Rule); ok && rule.Action == "" {
|
||||
rule.Action = model.ActionAccept
|
||||
}
|
||||
if f, ok := o.(*model.PortForward); ok && len(f.Protocols) == 0 {
|
||||
f.Protocols = []string{"tcp"}
|
||||
}
|
||||
var id int64
|
||||
return s.change(w, r, p, "", func(st *model.State, newID func() int64) error {
|
||||
id = newID()
|
||||
setID(o, id)
|
||||
if pos := position(o); pos != nil && *pos == 0 {
|
||||
*pos = lastPosition(st, kind, chainOf(o))
|
||||
}
|
||||
data, _ := json.Marshal(o)
|
||||
return st.Put(kind, id, data)
|
||||
}, func(st *model.State) any { return st.ByID(kind, id) })
|
||||
return s.change(w, r, p, model.Op{Op: model.OpCreate, Kind: kind, Data: raw})
|
||||
}
|
||||
|
||||
func (s *Server) updateObject(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
@@ -269,34 +321,8 @@ func (s *Server) updateObject(w http.ResponseWriter, r *http.Request, p *auth.Pr
|
||||
if err := decodeJSON(r, &patch); err != nil {
|
||||
return err
|
||||
}
|
||||
delete(patch, "id")
|
||||
var id int64
|
||||
return s.change(w, r, p, "", func(st *model.State, _ func() int64) error {
|
||||
o, err := findObject(st, kind, r.PathValue("ref"))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
id = o.GetID()
|
||||
// The patch's fields over the object's.
|
||||
var cur map[string]json.RawMessage
|
||||
b, _ := json.Marshal(o)
|
||||
json.Unmarshal(b, &cur)
|
||||
for k, v := range patch {
|
||||
if string(v) == "null" {
|
||||
delete(cur, k)
|
||||
} else {
|
||||
cur[k] = v
|
||||
}
|
||||
}
|
||||
merged, _ := json.Marshal(cur)
|
||||
next, err := decodeObject(kind, merged)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
setID(next, id)
|
||||
data, _ := json.Marshal(next)
|
||||
return st.Put(kind, id, data)
|
||||
}, func(st *model.State) any { return st.ByID(kind, id) })
|
||||
data, _ := json.Marshal(patch)
|
||||
return s.change(w, r, p, model.Op{Op: model.OpUpdate, Kind: kind, Ref: r.PathValue("ref"), Data: data})
|
||||
}
|
||||
|
||||
func (s *Server) deleteObject(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
@@ -304,16 +330,7 @@ func (s *Server) deleteObject(w http.ResponseWriter, r *http.Request, p *auth.Pr
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return s.change(w, r, p, "", func(st *model.State, _ func() int64) error {
|
||||
o, err := findObject(st, kind, r.PathValue("ref"))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if used := usedBy(st, kind, o); used != "" {
|
||||
return conflict("%s %s is used by %s; change that first", kind, o.Label(), used)
|
||||
}
|
||||
return st.Put(kind, o.GetID(), nil)
|
||||
}, nil)
|
||||
return s.change(w, r, p, model.Op{Op: model.OpDelete, Kind: kind, Ref: r.PathValue("ref")})
|
||||
}
|
||||
|
||||
// usedBy names an object that refers to o (zones, sets and services are
|
||||
@@ -364,8 +381,48 @@ func usedBy(st *model.State, kind string, o model.Object) string {
|
||||
return ""
|
||||
}
|
||||
|
||||
// moveObject reorders a rule or NAT rule: before or after another, or to
|
||||
// the start or end; a rule can change chain at the same time.
|
||||
// renameRefs makes every object that refers to a zone, set or service by
|
||||
// its old name use the new one: renaming it changes them in the same
|
||||
// change, so the history shows (and undoes) them together.
|
||||
func renameRefs(st *model.State, kind, old, now string) {
|
||||
fix := func(list []string) {
|
||||
for i, x := range list {
|
||||
if strings.EqualFold(x, old) {
|
||||
list[i] = now
|
||||
}
|
||||
}
|
||||
}
|
||||
match := func(m *model.Match) {
|
||||
switch kind {
|
||||
case model.KindZone:
|
||||
fix(m.InZones)
|
||||
fix(m.OutZones)
|
||||
case model.KindSet:
|
||||
fix(m.Sources)
|
||||
fix(m.Destinations)
|
||||
case model.KindService:
|
||||
fix(m.Services)
|
||||
}
|
||||
}
|
||||
for _, x := range st.Rules {
|
||||
match(&x.Match)
|
||||
}
|
||||
for _, x := range st.SNATs {
|
||||
match(&x.Match)
|
||||
}
|
||||
for _, x := range st.Redirects {
|
||||
match(&x.Match)
|
||||
}
|
||||
if kind == model.KindSet {
|
||||
for _, f := range st.Forwards {
|
||||
fix(f.Sources)
|
||||
fix(f.PublicAddresses)
|
||||
fix(f.HairpinSources)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// moveObject reorders a rule or NAT rule (moveOp).
|
||||
func (s *Server) moveObject(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
kind, err := pathKind(r)
|
||||
if err != nil {
|
||||
@@ -375,72 +432,8 @@ func (s *Server) moveObject(w http.ResponseWriter, r *http.Request, p *auth.Prin
|
||||
if err := decodeJSON(r, &in); err != nil {
|
||||
return err
|
||||
}
|
||||
var id int64
|
||||
return s.change(w, r, p, "", func(st *model.State, _ func() int64) error {
|
||||
o, err := findObject(st, kind, r.PathValue("ref"))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
id = o.GetID()
|
||||
pos := position(o)
|
||||
if pos == nil {
|
||||
return badRequest("only rules, port forwards, source NAT rules and redirects have an order")
|
||||
}
|
||||
if rule, ok := o.(*model.Rule); ok && in.Chain != "" {
|
||||
rule.Chain = in.Chain
|
||||
}
|
||||
chain := chainOf(o)
|
||||
// The others in order.
|
||||
var others []model.Object
|
||||
for _, x := range st.Objects(kind) {
|
||||
if x.GetID() != id && chainOf(x) == chain {
|
||||
others = append(others, x)
|
||||
}
|
||||
}
|
||||
sort.SliceStable(others, func(i, j int) bool { return *position(others[i]) < *position(others[j]) })
|
||||
place := func(i int) { // before others[i] (i == len: at the end)
|
||||
switch {
|
||||
case len(others) == 0:
|
||||
*pos = 1
|
||||
case i == 0:
|
||||
*pos = *position(others[0]) - 1
|
||||
case i == len(others):
|
||||
*pos = *position(others[len(others)-1]) + 1
|
||||
default:
|
||||
*pos = (*position(others[i-1]) + *position(others[i])) / 2
|
||||
}
|
||||
}
|
||||
index := func(ref int64) (int, error) {
|
||||
for i, x := range others {
|
||||
if x.GetID() == ref {
|
||||
return i, nil
|
||||
}
|
||||
}
|
||||
return 0, badRequest("%d isn't another %s of the same chain", ref, kind)
|
||||
}
|
||||
switch {
|
||||
case in.Before != 0:
|
||||
i, err := index(in.Before)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
place(i)
|
||||
case in.After != 0:
|
||||
i, err := index(in.After)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
place(i + 1)
|
||||
case in.To == "start":
|
||||
place(0)
|
||||
case in.To == "end" || in.Chain != "":
|
||||
place(len(others))
|
||||
default:
|
||||
return badRequest("say where: before, after, or to start or end")
|
||||
}
|
||||
data, _ := json.Marshal(o)
|
||||
return st.Put(kind, id, data)
|
||||
}, func(st *model.State) any { return st.ByID(kind, id) })
|
||||
data, _ := json.Marshal(in)
|
||||
return s.change(w, r, p, model.Op{Op: model.OpMove, Kind: kind, Ref: r.PathValue("ref"), Data: data})
|
||||
}
|
||||
|
||||
// ---- settings
|
||||
@@ -449,7 +442,11 @@ func (s *Server) getSettings(w http.ResponseWriter, r *http.Request, p *auth.Pri
|
||||
if err := require(p, auth.ResConfig, false); err != nil {
|
||||
return err
|
||||
}
|
||||
writeJSON(w, http.StatusOK, s.eng.State().Settings)
|
||||
_, st, _, err := s.viewState(w, r)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
writeJSON(w, http.StatusOK, st.Settings)
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -458,35 +455,43 @@ func (s *Server) updateSettings(w http.ResponseWriter, r *http.Request, p *auth.
|
||||
if err := decodeJSON(r, &patch); err != nil {
|
||||
return err
|
||||
}
|
||||
return s.change(w, r, p, "", func(st *model.State, _ func() int64) error {
|
||||
var cur map[string]json.RawMessage
|
||||
b, _ := json.Marshal(st.Settings)
|
||||
json.Unmarshal(b, &cur)
|
||||
for k, v := range patch {
|
||||
// Built-ins merge one by one.
|
||||
if k == "builtins" {
|
||||
var cb, pb map[string]json.RawMessage
|
||||
json.Unmarshal(cur[k], &cb)
|
||||
if err := json.Unmarshal(v, &pb); err != nil {
|
||||
return badRequest("builtins: %v", err)
|
||||
data, _ := json.Marshal(patch)
|
||||
return s.change(w, r, p, model.Op{Op: model.OpSettings, Data: data})
|
||||
}
|
||||
|
||||
// ---- the zone matrix
|
||||
|
||||
// zoneMatrix answers what the rules do to new connections between zones
|
||||
// (internal/matrix); with a changeset named, as it will make them.
|
||||
func (s *Server) zoneMatrix(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResConfig, false); err != nil {
|
||||
return err
|
||||
}
|
||||
for bk, bv := range pb {
|
||||
cb[bk] = bv
|
||||
_, st, _, err := s.viewState(w, r)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
v, _ = json.Marshal(cb)
|
||||
}
|
||||
cur[k] = v
|
||||
}
|
||||
merged, _ := json.Marshal(cur)
|
||||
var next model.Settings
|
||||
dec := json.NewDecoder(strings.NewReader(string(merged)))
|
||||
dec.DisallowUnknownFields()
|
||||
if err := dec.Decode(&next); err != nil {
|
||||
return badRequest("settings: %v", err)
|
||||
}
|
||||
st.Settings = next
|
||||
writeJSON(w, http.StatusOK, matrix.Build(st, matrix.Options{Connected: s.connected(st)}))
|
||||
return nil
|
||||
}, func(st *model.State) any { return st.Settings })
|
||||
}
|
||||
|
||||
// connected maps the networks on the firewall's interfaces (link-local
|
||||
// and loopback aside) to their zones.
|
||||
func (s *Server) connected(st *model.State) map[netip.Prefix]string {
|
||||
out := map[netip.Prefix]string{}
|
||||
for _, i := range s.hostInterfaces(st) {
|
||||
if i.Zone == "" {
|
||||
continue
|
||||
}
|
||||
for _, a := range i.Addresses {
|
||||
p, err := netip.ParsePrefix(a)
|
||||
if err != nil || p.Addr().IsLinkLocalUnicast() || p.Addr().IsLoopback() {
|
||||
continue
|
||||
}
|
||||
out[p.Masked()] = i.Zone
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// ---- the ruleset
|
||||
@@ -495,7 +500,11 @@ func (s *Server) ruleset(w http.ResponseWriter, r *http.Request, p *auth.Princip
|
||||
if err := require(p, auth.ResConfig, false); err != nil {
|
||||
return err
|
||||
}
|
||||
rs := s.fw.Render(s.eng.State())
|
||||
_, st, _, err := s.viewState(w, r)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
rs := s.fw.Render(st)
|
||||
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
|
||||
w.Write([]byte(rs.Text))
|
||||
return nil
|
||||
@@ -542,13 +551,30 @@ func (s *Server) revertHistory(w http.ResponseWriter, r *http.Request, p *auth.P
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
// Staged, an undo is an operation of its own, reviewed like any.
|
||||
if csID, err := changesetParam(r); err != nil {
|
||||
return err
|
||||
} else if csID != 0 {
|
||||
h, err := s.store.GetHistory(r.Context(), id)
|
||||
if errors.Is(err, store.ErrNotFound) {
|
||||
return notFound("history entry %d not found", id)
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if h.RevertedBy != 0 {
|
||||
return conflict("history entry %d was already undone by entry %d", id, h.RevertedBy)
|
||||
}
|
||||
data, _ := json.Marshal(h)
|
||||
return s.stage(w, r, p, csID, model.Op{Op: model.OpRevert, ID: id, Data: data})
|
||||
}
|
||||
res, err := s.eng.Revert(r.Context(), p, id, engine.Options{DryRun: queryBool(r, "dry_run"), Via: via(r, p)})
|
||||
if err != nil {
|
||||
s.writeErr(w, r, err, res)
|
||||
return nil
|
||||
}
|
||||
s.afterChange(r.Context(), p, res)
|
||||
s.respondChange(w, r, res, nil, false)
|
||||
s.respondChange(w, r, p, res, nil, false)
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -566,7 +592,17 @@ func (s *Server) confirmChanges(w http.ResponseWriter, r *http.Request, p *auth.
|
||||
if err := require(p, auth.ResConfig, true); err != nil {
|
||||
return err
|
||||
}
|
||||
n, err := s.confirm(r.Context(), p)
|
||||
var in api.ConfirmInput
|
||||
if r.ContentLength != 0 {
|
||||
if err := decodeJSON(r, &in); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
confirm := func() (int, error) { return s.confirm(r.Context(), p) }
|
||||
if in.Token != "" {
|
||||
confirm = func() (int, error) { return s.confirmToken(r, p, in.Token) }
|
||||
}
|
||||
n, err := confirm()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"housecarl/internal/version"
|
||||
)
|
||||
|
||||
// fallbackInForce is what status and /healthz say while the boot fallback
|
||||
// (render.Fallback) is in force.
|
||||
const fallbackInForce = "the boot fallback is in force, not Housecarl's ruleset: the saved ruleset failed to load at boot (journalctl -u housecarl-ruleset), and housecarld couldn't apply it since; it lets nothing new in from external zones (with none, nothing but loopback)"
|
||||
|
||||
// health is GET /healthz, for monitoring (Checkmk, a load balancer). It
|
||||
// needs no authentication, so it says only what a monitor needs, and reads
|
||||
// the ruleset at most every few seconds however often it is asked.
|
||||
type health struct {
|
||||
OK bool `json:"ok"`
|
||||
Version string `json:"version"`
|
||||
Firewall string `json:"firewall"` // applied, unconfigured, fallback, missing
|
||||
// Probation: changes waiting to be confirmed, rolled back otherwise.
|
||||
Probation int `json:"probation"`
|
||||
// Warnings are the codes of the machine's problems (nftables_service).
|
||||
Warnings []string `json:"warnings"`
|
||||
}
|
||||
|
||||
const healthTTL = 5 * time.Second
|
||||
|
||||
type healthCache struct {
|
||||
mu sync.Mutex
|
||||
at time.Time
|
||||
out health
|
||||
}
|
||||
|
||||
func (s *Server) healthz(w http.ResponseWriter, r *http.Request) {
|
||||
c := &s.health
|
||||
c.mu.Lock()
|
||||
if time.Since(c.at) >= healthTTL {
|
||||
c.out, c.at = s.checkHealth(r), time.Now()
|
||||
}
|
||||
out := c.out
|
||||
c.mu.Unlock()
|
||||
code := http.StatusOK
|
||||
if !out.OK {
|
||||
code = http.StatusServiceUnavailable
|
||||
}
|
||||
writeJSON(w, code, out)
|
||||
}
|
||||
|
||||
func (s *Server) checkHealth(r *http.Request) health {
|
||||
out := health{Version: version.String(), Warnings: []string{}}
|
||||
t, err := s.fw.Table(r.Context())
|
||||
switch {
|
||||
case err != nil && s.eng.State().Empty():
|
||||
out.Firewall, out.OK = "unconfigured", true
|
||||
case err != nil:
|
||||
out.Firewall = "missing"
|
||||
case t.Fallback:
|
||||
out.Firewall = "fallback"
|
||||
default:
|
||||
out.Firewall, out.OK = "applied", true
|
||||
}
|
||||
pr := s.probationStatus()
|
||||
out.Probation = len(pr.Pending) + pr.Takeovers
|
||||
for _, p := range s.hostProblems(r.Context(), nil) {
|
||||
out.Warnings = append(out.Warnings, p.Code)
|
||||
}
|
||||
return out
|
||||
}
|
||||
@@ -0,0 +1,65 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"housecarl/internal/api"
|
||||
)
|
||||
|
||||
func TestHealthz(t *testing.T) {
|
||||
e := newEnv(t)
|
||||
link := filepath.Join(t.TempDir(), "nftables.service")
|
||||
e.srv.unitLinks = []string{link}
|
||||
get := func() (int, health) {
|
||||
t.Helper()
|
||||
e.srv.health.at = time.Time{} // no caching between the steps
|
||||
resp, err := http.Get(e.ts.URL + "/healthz")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
var h health
|
||||
json.NewDecoder(resp.Body).Decode(&h)
|
||||
return resp.StatusCode, h
|
||||
}
|
||||
// Nothing configured yet: nothing to be in force.
|
||||
if code, h := get(); code != 200 || !h.OK || h.Firewall != "unconfigured" || h.Version == "" {
|
||||
t.Errorf("unconfigured: %d %+v", code, h)
|
||||
}
|
||||
if code := e.call(e.token, "POST", "/zones", map[string]any{"name": "wan", "interfaces": []string{"wan"}, "external": true}, nil); code != 201 {
|
||||
t.Fatalf("creating a zone: %d", code)
|
||||
}
|
||||
if code, h := get(); code != 200 || h.Firewall != "applied" || h.Probation != 1 || len(h.Warnings) != 0 {
|
||||
t.Errorf("applied, on probation: %d %+v", code, h)
|
||||
}
|
||||
os.WriteFile(link, nil, 0o644)
|
||||
if _, h := get(); strings.Join(h.Warnings, ",") != "nftables_service" {
|
||||
t.Errorf("warnings %v", h.Warnings)
|
||||
}
|
||||
e.fw.mu.Lock()
|
||||
e.fw.fallback = true
|
||||
e.fw.mu.Unlock()
|
||||
if code, h := get(); code != 503 || h.OK || h.Firewall != "fallback" {
|
||||
t.Errorf("fallback: %d %+v", code, h)
|
||||
}
|
||||
var st api.Status
|
||||
e.call(e.token, "GET", "/status", nil, &st)
|
||||
if st.Applied || !strings.Contains(st.ApplyError, "boot fallback") {
|
||||
t.Errorf("status with the fallback: applied %v, %q", st.Applied, st.ApplyError)
|
||||
}
|
||||
// Asked often, the ruleset is read once per few seconds.
|
||||
e.fw.mu.Lock()
|
||||
e.fw.fallback = false
|
||||
e.fw.mu.Unlock()
|
||||
resp, _ := http.Get(e.ts.URL + "/healthz")
|
||||
resp.Body.Close()
|
||||
if resp.StatusCode != 503 {
|
||||
t.Errorf("not cached: %d", resp.StatusCode)
|
||||
}
|
||||
}
|
||||
@@ -18,11 +18,14 @@ func (s *Server) importNft(w http.ResponseWriter, r *http.Request, p *auth.Princ
|
||||
if err := require(p, auth.ResConfig, true); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := notStaged(r, "an import"); err != nil {
|
||||
return err
|
||||
}
|
||||
var in api.ImportInput
|
||||
if err := decodeJSON(r, &in); err != nil {
|
||||
return err
|
||||
}
|
||||
res, err := importer.Import(importer.Input{JSON: in.JSON, Text: in.Text, Defines: in.Defines, Table: in.Table})
|
||||
res, err := importer.Import(importer.Input{JSON: in.JSON, Text: in.Text, Defines: in.Defines, Table: in.Table, Zones: in.Zones})
|
||||
if err != nil {
|
||||
return badRequest("%v", err)
|
||||
}
|
||||
@@ -48,6 +51,11 @@ func (s *Server) importNft(w http.ResponseWriter, r *http.Request, p *auth.Princ
|
||||
return nil
|
||||
}
|
||||
s.afterChange(r.Context(), p, er)
|
||||
if er.Applied {
|
||||
// Until it is taken over, the old table still applies too: the
|
||||
// status warns (takeover_pending).
|
||||
s.store.SetMeta(r.Context(), importedTableKey, res.Table)
|
||||
}
|
||||
out.MutationResponse = api.MutationResponse{Applied: er.Applied, DryRun: er.DryRun, HistoryID: er.HistoryID, Changes: er.Changes, Warnings: er.Warnings}
|
||||
if out.Changes == nil {
|
||||
out.Changes = []model.Change{}
|
||||
@@ -55,6 +63,7 @@ func (s *Server) importNft(w http.ResponseWriter, r *http.Request, p *auth.Princ
|
||||
if pr := s.probationStatus(); er.Applied && len(pr.Pending) > 0 {
|
||||
out.ConfirmBy = &pr.Deadline
|
||||
}
|
||||
out.ConfirmToken = s.issueCheck(w, r, p, er)
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
return nil
|
||||
}
|
||||
@@ -105,6 +114,9 @@ func (s *Server) takeover(w http.ResponseWriter, r *http.Request, p *auth.Princi
|
||||
if err := requireAdmin(p); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := notStaged(r, "a takeover"); err != nil {
|
||||
return err
|
||||
}
|
||||
var in api.TakeoverInput
|
||||
if err := decodeJSON(r, &in); err != nil {
|
||||
return err
|
||||
@@ -142,6 +154,12 @@ func (s *Server) takeover(w http.ResponseWriter, r *http.Request, p *auth.Princi
|
||||
} else if pr := s.probationStatus(); pr.Takeovers > 0 {
|
||||
msg += fmt.Sprintf("; confirm with 'hc confirm' within %ds, or they come back", pr.SecondsLeft)
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]any{"deleted": done, "message": msg})
|
||||
answer := map[string]any{"deleted": done, "message": msg}
|
||||
if !queryBool(r, "dry_run") && len(done) > 0 {
|
||||
if tok := s.issueCheck(w, r, p, nil); tok != "" {
|
||||
answer["confirm_token"] = tok
|
||||
}
|
||||
}
|
||||
writeJSON(w, http.StatusOK, answer)
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,266 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"housecarl/internal/api"
|
||||
"housecarl/internal/auth"
|
||||
"housecarl/internal/model"
|
||||
"housecarl/internal/stats"
|
||||
"housecarl/internal/validate"
|
||||
)
|
||||
|
||||
// Statistics: the firewall's figures over time (internal/stats).
|
||||
|
||||
// StatsSources fills in what the statistics collector needs from the
|
||||
// server: the interfaces' zones, and which counters count drops.
|
||||
func (s *Server) StatsSources(src *stats.Sources) {
|
||||
src.Zones = s.interfaceZones
|
||||
src.Classify = s.classifyMark
|
||||
}
|
||||
|
||||
// SetStats gives the server the statistics collector (nil: statistics
|
||||
// are off).
|
||||
func (s *Server) SetStats(c *stats.Collector) { s.stats = c }
|
||||
|
||||
// interfaceZones maps the machine's interfaces to their zones.
|
||||
func (s *Server) interfaceZones() map[string]string {
|
||||
out := map[string]string{}
|
||||
for _, i := range s.hostInterfaces(s.eng.State()) {
|
||||
if i.Zone != "" {
|
||||
out[i.Name] = i.Zone
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// classifyMark says which chain a counter is in and whether what it
|
||||
// counts is dropped: rules that drop or reject, the drop policies, and
|
||||
// the built-ins that drop (antispoofing, invalid packets).
|
||||
func (s *Server) classifyMark(mark string) (string, bool) {
|
||||
st := s.eng.State()
|
||||
switch {
|
||||
case strings.HasPrefix(mark, "p-"):
|
||||
chain := strings.TrimPrefix(mark, "p-")
|
||||
policy := map[string]string{"input": st.Settings.InputPolicy, "forward": st.Settings.ForwardPolicy, "output": st.Settings.OutputPolicy}[chain]
|
||||
return chain, policy == model.PolicyDrop
|
||||
case strings.HasPrefix(mark, "b-antispoof"):
|
||||
return model.ChainEarly, true
|
||||
case mark == "b-invalid":
|
||||
return "invalid", true
|
||||
case strings.HasPrefix(mark, "r"):
|
||||
num := strings.TrimLeft(mark[1:], "0123456789")
|
||||
id, _ := strconv.ParseInt(strings.TrimSuffix(mark[1:], num), 10, 64)
|
||||
if o, ok := st.ByID(model.KindRule, id).(*model.Rule); ok && o != nil {
|
||||
return o.Chain, o.Action == model.ActionDrop || o.Action == model.ActionReject
|
||||
}
|
||||
}
|
||||
return "", false
|
||||
}
|
||||
|
||||
// markName names a counter's object: "rule input #12 (ssh)", "built-in:
|
||||
// established", "input policy".
|
||||
func markName(st *model.State, mark string) string {
|
||||
if c, ok := strings.CutPrefix(mark, "p-"); ok {
|
||||
return c + " policy"
|
||||
}
|
||||
if l := markLabel(st, api.TraceStep{Kind: "rule", Mark: mark}); l != "" {
|
||||
return l
|
||||
}
|
||||
if len(mark) < 2 {
|
||||
return mark
|
||||
}
|
||||
// An object deleted since it counted.
|
||||
kind := map[byte]string{'r': "rule", 'f': "port forward", 's': "source NAT", 'd': "redirect"}[mark[0]]
|
||||
if num := strings.TrimLeft(mark[1:], "0123456789"); kind != "" && len(num) < len(mark)-1 {
|
||||
return kind + " #" + strings.TrimSuffix(mark[1:], num) + " (deleted)"
|
||||
}
|
||||
return mark
|
||||
}
|
||||
|
||||
func (s *Server) statsCollector() (*stats.Collector, error) {
|
||||
if s.stats == nil {
|
||||
return nil, &httpError{http.StatusServiceUnavailable, "stats_off", "statistics are off: set stats.enabled in housecarl.yaml and restart housecarld"}
|
||||
}
|
||||
return s.stats, nil
|
||||
}
|
||||
|
||||
func (s *Server) statsQuerier(p *auth.Principal) (*stats.Querier, error) {
|
||||
if err := require(p, auth.ResLog, false); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
c, err := s.statsCollector()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
q := stats.NewQuerier(c)
|
||||
q.Now = s.nowFn
|
||||
return q, nil
|
||||
}
|
||||
|
||||
func since(r *http.Request, def string) (string, error) {
|
||||
v := r.URL.Query().Get("since")
|
||||
if v == "" {
|
||||
v = def
|
||||
}
|
||||
if _, err := stats.ParseSince(v); err != nil {
|
||||
return "", badRequest("%v", err)
|
||||
}
|
||||
return v, nil
|
||||
}
|
||||
|
||||
// StatsStatus is the collector's health, with the setting that counts
|
||||
// traffic per device.
|
||||
type StatsStatus struct {
|
||||
stats.Status
|
||||
Latest *stats.Latest `json:"latest,omitempty"`
|
||||
}
|
||||
|
||||
func (s *Server) statsStatus(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResLog, false); err != nil {
|
||||
return err
|
||||
}
|
||||
out := StatsStatus{}
|
||||
if s.stats != nil {
|
||||
out.Status = s.stats.Status()
|
||||
out.Latest = s.stats.Latest()
|
||||
if !p.IsAdmin() {
|
||||
out.Database = ""
|
||||
}
|
||||
}
|
||||
out.PerDevice = s.eng.State().Settings.StatsPerDevice
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Server) statsSummary(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
q, err := s.statsQuerier(p)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
v, err := since(r, "24h")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
d, _ := stats.ParseSince(v)
|
||||
sum, err := q.Summary(r.Context(), d, queryInt(r, "limit", 10))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
st := s.eng.State()
|
||||
for i := range sum.Rules {
|
||||
sum.Rules[i].Name = markName(st, sum.Rules[i].Key)
|
||||
}
|
||||
writeJSON(w, http.StatusOK, sum)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Server) statsTimeline(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
q, err := s.statsQuerier(p)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
v, err := since(r, "24h")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
d, _ := stats.ParseSince(v)
|
||||
var keys []string
|
||||
for _, k := range strings.Split(r.URL.Query().Get("keys"), ",") {
|
||||
if k = strings.TrimSpace(k); k != "" {
|
||||
keys = append(keys, k)
|
||||
}
|
||||
}
|
||||
if len(keys) == 0 {
|
||||
return badRequest("say which figures: keys=zrx:*,ztx:* (docs/stats.md)")
|
||||
}
|
||||
tl, err := q.Timeline(r.Context(), d, r.URL.Query().Get("step"), keys)
|
||||
if err != nil {
|
||||
return badRequest("%v", err)
|
||||
}
|
||||
writeJSON(w, http.StatusOK, tl)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Server) statsTop(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
q, err := s.statsQuerier(p)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
v, err := since(r, "24h")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
d, _ := stats.ParseSince(v)
|
||||
es, err := q.Top(r.Context(), r.URL.Query().Get("kind"), d, queryInt(r, "limit", 20))
|
||||
if err != nil {
|
||||
return badRequest("%v", err)
|
||||
}
|
||||
writeJSON(w, http.StatusOK, es)
|
||||
return nil
|
||||
}
|
||||
|
||||
// statsHits are the packets each rule, built-in and policy counted over
|
||||
// the range, named; rules that counted none are listed too, with 0.
|
||||
func (s *Server) statsHits(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
q, err := s.statsQuerier(p)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
v, err := since(r, "30d")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
d, _ := stats.ParseSince(v)
|
||||
es, err := q.Sums(r.Context(), "hits:", d)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
st := s.eng.State()
|
||||
seen := map[string]bool{}
|
||||
for i := range es {
|
||||
es[i].Name = markName(st, es[i].Key)
|
||||
seen[es[i].Key] = true
|
||||
}
|
||||
for _, rule := range st.Rules {
|
||||
if mark := "r" + strconv.FormatInt(rule.ID, 10); !seen[mark] && !rule.Disabled {
|
||||
es = append(es, stats.Entry{Key: mark, Name: "rule " + rule.Label()})
|
||||
}
|
||||
}
|
||||
writeJSON(w, http.StatusOK, es)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Server) statsHeatmap(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
q, err := s.statsQuerier(p)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
v, err := since(r, "28d")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
d, _ := stats.ParseSince(v)
|
||||
hm, err := q.Heatmap(r.Context(), r.URL.Query().Get("key"), d)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
writeJSON(w, http.StatusOK, hm)
|
||||
return nil
|
||||
}
|
||||
|
||||
// statsProblems warns when the connection tracking table is nearly full:
|
||||
// new connections are dropped once it is.
|
||||
func (s *Server) statsProblems() []validate.Problem {
|
||||
if s.stats == nil {
|
||||
return nil
|
||||
}
|
||||
l := s.stats.Latest()
|
||||
if l == nil || l.ConntrackMax == 0 || l.Conntrack*100 < l.ConntrackMax*80 {
|
||||
return nil
|
||||
}
|
||||
return []validate.Problem{{Severity: validate.Warning, Code: "conntrack_full", Object: "connections",
|
||||
Message: "the connection tracking table is " + strconv.FormatInt(l.Conntrack*100/l.ConntrackMax, 10) + "% full (" + strconv.FormatInt(l.Conntrack, 10) + " of " + strconv.FormatInt(l.ConntrackMax, 10) + "); once it is, new connections are dropped: raise net.netfilter.nf_conntrack_max, or find what opens so many (Connections)"}}
|
||||
}
|
||||
@@ -1,7 +1,10 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"os"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
@@ -34,20 +37,56 @@ func (s *Server) status(w http.ResponseWriter, r *http.Request, p *auth.Principa
|
||||
for _, k := range model.Kinds {
|
||||
out.Counts[k] = len(st.Objects(k))
|
||||
}
|
||||
if _, err := s.fw.Table(r.Context()); err != nil && st.Empty() {
|
||||
if t, err := s.fw.Table(r.Context()); err != nil && st.Empty() {
|
||||
out.Unconfigured = true
|
||||
} else if err != nil {
|
||||
out.ApplyError = err.Error()
|
||||
} else if t.Fallback {
|
||||
out.ApplyError = fallbackInForce
|
||||
} else {
|
||||
out.Applied = true
|
||||
}
|
||||
if data, err := s.fw.ListRuleset(r.Context()); err == nil {
|
||||
out.OtherTables = otherTables(data)
|
||||
}
|
||||
out.Problems = append(out.Problems, s.hostProblems(r.Context(), out.OtherTables)...)
|
||||
out.Problems = append(out.Problems, s.statsProblems()...)
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
return nil
|
||||
}
|
||||
|
||||
const importedTableKey = "imported_table"
|
||||
|
||||
// nftablesUnitLinks are where an enabled nftables.service shows (Debian
|
||||
// enables it for sysinit.target).
|
||||
var nftablesUnitLinks = []string{
|
||||
"/etc/systemd/system/sysinit.target.wants/nftables.service",
|
||||
"/etc/systemd/system/multi-user.target.wants/nftables.service",
|
||||
}
|
||||
|
||||
// hostProblems are warnings about the machine rather than the
|
||||
// configuration: another ruleset that still loads at boot, or the table
|
||||
// an import came from, still in force beside Housecarl's.
|
||||
func (s *Server) hostProblems(ctx context.Context, others []string) []validate.Problem {
|
||||
var out []validate.Problem
|
||||
for _, link := range s.unitLinks {
|
||||
if _, err := os.Lstat(link); err == nil {
|
||||
out = append(out, validate.Problem{Severity: validate.Warning, Code: "nftables_service", Object: "nftables.service",
|
||||
Message: "nftables.service is enabled: at boot it loads /etc/nftables.conf, and stopping or restarting it flushes every table, Housecarl's included; disable it (systemctl disable nftables, without --now)"})
|
||||
break
|
||||
}
|
||||
}
|
||||
if t, err := s.store.GetMeta(ctx, importedTableKey); err == nil && t != "" {
|
||||
for _, o := range others {
|
||||
if o == t {
|
||||
out = append(out, validate.Problem{Severity: validate.Warning, Code: "takeover_pending", Object: "table " + t,
|
||||
Message: fmt.Sprintf("%s, the table you imported, still applies beside Housecarl's: a packet must get past both; once Housecarl's rules are right, delete it with 'hc takeover %q'", t, t)})
|
||||
}
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// hostInterfaces lists the machine's interfaces with their zones.
|
||||
func (s *Server) hostInterfaces(st *model.State) []api.HostInterface {
|
||||
out := []api.HostInterface{}
|
||||
@@ -260,7 +299,12 @@ func (l *logRing) Latest(n int, after time.Time, keep func(api.LogEntry) bool) [
|
||||
|
||||
// LogPacket records a packet the firewall logged (internal/nflog calls
|
||||
// it).
|
||||
func (s *Server) LogPacket(e api.LogEntry) { s.logs.Add(e) }
|
||||
func (s *Server) LogPacket(e api.LogEntry) {
|
||||
s.logs.Add(e)
|
||||
if s.stats != nil {
|
||||
s.stats.Packet(e)
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Server) listLog(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResLog, false); err != nil {
|
||||
|
||||
@@ -0,0 +1,309 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"housecarl/internal/api"
|
||||
"housecarl/internal/auth"
|
||||
"housecarl/internal/model"
|
||||
"housecarl/internal/trace"
|
||||
)
|
||||
|
||||
// Traffic as it happens: the connections the kernel tracks (and closing
|
||||
// them, so the rules check them again), and tracing packets through the
|
||||
// whole ruleset.
|
||||
|
||||
// Conntrack lists and closes tracked connections (internal/conntrack).
|
||||
type Conntrack interface {
|
||||
List(ctx context.Context) ([]api.Connection, error)
|
||||
Close(ctx context.Context, c api.Connection) error
|
||||
}
|
||||
|
||||
// SetConntrack gives the server the kernel's connection tracking.
|
||||
func (s *Server) SetConntrack(c Conntrack) { s.conns = c }
|
||||
|
||||
func (s *Server) conntrack() (Conntrack, error) {
|
||||
if s.conns == nil {
|
||||
return nil, &httpError{http.StatusServiceUnavailable, "unavailable", "connection tracking isn't available here"}
|
||||
}
|
||||
return s.conns, nil
|
||||
}
|
||||
|
||||
// connMatches: a search in a connection's addresses, ports (":443"),
|
||||
// protocol, state and NAT addresses.
|
||||
func connMatches(c api.Connection, q string) bool {
|
||||
if q == "" {
|
||||
return true
|
||||
}
|
||||
hay := strings.ToLower(strings.Join([]string{c.Protocol, c.Source, ":" + strconv.Itoa(c.SourcePort), c.Destination, ":" + strconv.Itoa(c.DestinationPort),
|
||||
c.DNATTo, c.SNATTo, c.State}, " ") + " ")
|
||||
for _, w := range strings.Fields(strings.ToLower(q)) {
|
||||
if strings.HasPrefix(w, ":") {
|
||||
w += " " // a whole port
|
||||
}
|
||||
if !strings.Contains(hay, w) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func (s *Server) listConnections(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResLog, false); err != nil {
|
||||
return err
|
||||
}
|
||||
ct, err := s.conntrack()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
all, err := ct.List(r.Context())
|
||||
if err != nil {
|
||||
return fmt.Errorf("listing connections: %w", err)
|
||||
}
|
||||
q := r.URL.Query().Get("q")
|
||||
out := []api.Connection{}
|
||||
for _, c := range all {
|
||||
if connMatches(c, q) {
|
||||
out = append(out, c)
|
||||
}
|
||||
}
|
||||
sort.Slice(out, func(i, j int) bool {
|
||||
if out[i].Source != out[j].Source {
|
||||
return out[i].Source < out[j].Source
|
||||
}
|
||||
return out[i].Destination < out[j].Destination
|
||||
})
|
||||
if n := queryInt(r, "limit", 1000); n > 0 && len(out) > n {
|
||||
out = out[:n]
|
||||
}
|
||||
w.Header().Set("X-Total-Count", strconv.Itoa(len(all)))
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
return nil
|
||||
}
|
||||
|
||||
// closeConnections deletes tracking entries: the rules check those
|
||||
// connections again with their next packet.
|
||||
func (s *Server) closeConnections(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResConfig, true); err != nil {
|
||||
return err
|
||||
}
|
||||
ct, err := s.conntrack()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var in api.CloseConnections
|
||||
if err := decodeJSON(r, &in); err != nil {
|
||||
return err
|
||||
}
|
||||
list := in.Connections
|
||||
if strings.TrimSpace(in.Search) != "" {
|
||||
all, err := ct.List(r.Context())
|
||||
if err != nil {
|
||||
return fmt.Errorf("listing connections: %w", err)
|
||||
}
|
||||
for _, c := range all {
|
||||
if connMatches(c, in.Search) {
|
||||
list = append(list, c)
|
||||
}
|
||||
}
|
||||
} else if len(list) == 0 {
|
||||
return badRequest("say which connections: a list, or a search")
|
||||
}
|
||||
closed := 0
|
||||
for _, c := range list {
|
||||
if err := ct.Close(r.Context(), c); err != nil {
|
||||
return fmt.Errorf("closing %s %s -> %s: %w", c.Protocol, c.Source, c.Destination, err)
|
||||
}
|
||||
closed++
|
||||
}
|
||||
what := in.Search
|
||||
if what == "" && len(list) == 1 {
|
||||
what = fmt.Sprintf("%s %s:%d -> %s:%d", list[0].Protocol, list[0].Source, list[0].SourcePort, list[0].Destination, list[0].DestinationPort)
|
||||
}
|
||||
s.eng.Audit(r.Context(), p, "connections.close", what, "success", fmt.Sprintf("%d closed", closed))
|
||||
writeJSON(w, http.StatusOK, map[string]int{"closed": closed})
|
||||
return nil
|
||||
}
|
||||
|
||||
// ---- tracing
|
||||
|
||||
type tracing struct {
|
||||
mu sync.Mutex
|
||||
status api.TraceStatus
|
||||
tracer *trace.Tracer
|
||||
cancel context.CancelFunc
|
||||
}
|
||||
|
||||
func (s *Server) startTrace(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResConfig, true); err != nil {
|
||||
return err
|
||||
}
|
||||
var in api.TraceInput
|
||||
if err := decodeJSON(r, &in); err != nil {
|
||||
return err
|
||||
}
|
||||
if in.Seconds == 0 {
|
||||
in.Seconds = 60
|
||||
}
|
||||
if in.Seconds < 5 || in.Seconds > 600 {
|
||||
return badRequest("trace for 5 to 600 seconds")
|
||||
}
|
||||
script, err := trace.Script(in)
|
||||
if err != nil {
|
||||
return badRequest("%v", err)
|
||||
}
|
||||
t := &s.tracing
|
||||
t.mu.Lock()
|
||||
if t.status.Running {
|
||||
t.mu.Unlock()
|
||||
return conflict("a trace is running already (started %s); stop it first", t.status.Started.Format("15:04:05"))
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), time.Duration(in.Seconds)*time.Second)
|
||||
now := s.nowFn()
|
||||
t.status = api.TraceStatus{Running: true, Input: in, Started: now, Ends: now.Add(time.Duration(in.Seconds) * time.Second), Packets: []api.TracePacket{}}
|
||||
t.tracer = &trace.Tracer{Now: s.nowFn}
|
||||
t.cancel = cancel
|
||||
t.mu.Unlock()
|
||||
s.eng.Audit(r.Context(), p, "trace.start", traceWhat(in), "success", fmt.Sprintf("%d seconds", in.Seconds))
|
||||
go func() {
|
||||
defer cancel()
|
||||
err := s.fw.Trace(ctx, script, func(line string) {
|
||||
t.mu.Lock()
|
||||
defer t.mu.Unlock()
|
||||
t.tracer.Line(line)
|
||||
if t.tracer.Full() {
|
||||
cancel()
|
||||
}
|
||||
})
|
||||
t.mu.Lock()
|
||||
defer t.mu.Unlock()
|
||||
t.status.Running = false
|
||||
if err != nil {
|
||||
t.status.Error = err.Error()
|
||||
s.log.Warn("tracing failed", "err", err)
|
||||
}
|
||||
}()
|
||||
writeJSON(w, http.StatusOK, s.traceStatus())
|
||||
return nil
|
||||
}
|
||||
|
||||
func traceWhat(in api.TraceInput) string {
|
||||
var parts []string
|
||||
if in.Address != "" {
|
||||
parts = append(parts, in.Address)
|
||||
}
|
||||
if in.Protocol != "" {
|
||||
parts = append(parts, in.Protocol)
|
||||
}
|
||||
if in.Port != 0 {
|
||||
parts = append(parts, "port "+strconv.Itoa(in.Port))
|
||||
}
|
||||
return strings.Join(parts, " ")
|
||||
}
|
||||
|
||||
// traceStatus is the trace with each step named after Housecarl's object.
|
||||
func (s *Server) traceStatus() api.TraceStatus {
|
||||
t := &s.tracing
|
||||
t.mu.Lock()
|
||||
out := t.status
|
||||
if t.tracer != nil {
|
||||
out.Packets = t.tracer.Packets()
|
||||
}
|
||||
t.mu.Unlock()
|
||||
if out.Packets == nil {
|
||||
out.Packets = []api.TracePacket{}
|
||||
}
|
||||
st := s.eng.State()
|
||||
for i := range out.Packets {
|
||||
p := &out.Packets[i]
|
||||
for j := range p.Steps {
|
||||
p.Steps[j].Label = markLabel(st, p.Steps[j])
|
||||
}
|
||||
p.DecidedBy.Label = markLabel(st, p.DecidedBy)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// markLabel names the Housecarl object behind a step: "rule input #12
|
||||
// (ssh)", "built-in: established", "port forward web", "input policy".
|
||||
func markLabel(st *model.State, step api.TraceStep) string {
|
||||
if step.Kind == "policy" {
|
||||
if step.Table == "inet housecarl" {
|
||||
return step.Chain + " policy"
|
||||
}
|
||||
return ""
|
||||
}
|
||||
m := step.Mark
|
||||
kind, id := "", int64(0)
|
||||
switch {
|
||||
case m == "":
|
||||
return ""
|
||||
case strings.HasPrefix(m, "b-"):
|
||||
return "built-in: " + strings.TrimPrefix(m, "b-")
|
||||
case strings.HasPrefix(m, "r"):
|
||||
kind = model.KindRule
|
||||
case strings.HasPrefix(m, "f"):
|
||||
kind = model.KindForward
|
||||
case strings.HasPrefix(m, "s"):
|
||||
kind = model.KindSNAT
|
||||
case strings.HasPrefix(m, "d"):
|
||||
kind = model.KindRedirect
|
||||
default:
|
||||
return ""
|
||||
}
|
||||
num := strings.TrimLeft(m[1:], "0123456789")
|
||||
id, _ = strconv.ParseInt(strings.TrimSuffix(m[1:], num), 10, 64)
|
||||
if o := st.ByID(kind, id); o != nil {
|
||||
label := o.Label() // "source NAT #3 (…)", "redirect #4", a forward's name…
|
||||
switch {
|
||||
case kind == model.KindRule:
|
||||
label = "rule " + label
|
||||
case kind == model.KindForward && !strings.HasPrefix(label, "port forward"):
|
||||
label = "port forward " + label
|
||||
}
|
||||
if num != "" {
|
||||
label += " (" + strings.TrimPrefix(num, "-") + ")"
|
||||
}
|
||||
return label
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func (s *Server) getTrace(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResLog, false); err != nil {
|
||||
return err
|
||||
}
|
||||
writeJSON(w, http.StatusOK, s.traceStatus())
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Server) stopTrace(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResConfig, true); err != nil {
|
||||
return err
|
||||
}
|
||||
t := &s.tracing
|
||||
t.mu.Lock()
|
||||
if t.cancel != nil && t.status.Running {
|
||||
t.cancel()
|
||||
}
|
||||
t.mu.Unlock()
|
||||
// Let the trace end, so the answer shows it stopped.
|
||||
for i := 0; i < 50; i++ {
|
||||
t.mu.Lock()
|
||||
running := t.status.Running
|
||||
t.mu.Unlock()
|
||||
if !running {
|
||||
break
|
||||
}
|
||||
time.Sleep(20 * time.Millisecond)
|
||||
}
|
||||
writeJSON(w, http.StatusOK, s.traceStatus())
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,282 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"housecarl/internal/api"
|
||||
"housecarl/internal/engine"
|
||||
"housecarl/internal/model"
|
||||
)
|
||||
|
||||
// Every configuration change is an operation (model.Op): applied at once,
|
||||
// or staged in a changeset and replayed when the changeset is applied.
|
||||
// applyOp is the one place an operation becomes an edit.
|
||||
|
||||
// applyOp makes an operation's change to st. It returns the ID of the
|
||||
// object it made or changed (0 for deletes and settings).
|
||||
func applyOp(st *model.State, op *model.Op) (int64, error) {
|
||||
switch op.Op {
|
||||
case model.OpCreate:
|
||||
return createOp(st, op)
|
||||
case model.OpUpdate:
|
||||
return updateOp(st, op)
|
||||
case model.OpDelete:
|
||||
return 0, deleteOp(st, op)
|
||||
case model.OpMove:
|
||||
return moveOp(st, op)
|
||||
case model.OpSettings:
|
||||
return 0, settingsOp(st, op)
|
||||
case model.OpRevert:
|
||||
var h model.HistoryEntry
|
||||
if err := json.Unmarshal(op.Data, &h); err != nil {
|
||||
return 0, badRequest("revert: %v", err)
|
||||
}
|
||||
return 0, engine.Undo(st, &h)
|
||||
}
|
||||
return 0, badRequest("unknown operation %q", op.Op)
|
||||
}
|
||||
|
||||
func createOp(st *model.State, op *model.Op) (int64, error) {
|
||||
o, err := decodeObject(op.Kind, op.Data)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if svc, ok := o.(*model.Service); ok {
|
||||
svc.Builtin = false
|
||||
}
|
||||
if rule, ok := o.(*model.Rule); ok && rule.Action == "" {
|
||||
rule.Action = model.ActionAccept
|
||||
}
|
||||
if f, ok := o.(*model.PortForward); ok && len(f.Protocols) == 0 {
|
||||
f.Protocols = []string{"tcp"}
|
||||
}
|
||||
if st.ByID(op.Kind, op.ID) != nil {
|
||||
return 0, conflict("%s %d exists already", op.Kind, op.ID)
|
||||
}
|
||||
setID(o, op.ID)
|
||||
if pos := position(o); pos != nil && *pos == 0 {
|
||||
*pos = lastPosition(st, op.Kind, chainOf(o))
|
||||
}
|
||||
data, _ := json.Marshal(o)
|
||||
return op.ID, st.Put(op.Kind, op.ID, data)
|
||||
}
|
||||
|
||||
func updateOp(st *model.State, op *model.Op) (int64, error) {
|
||||
var patch map[string]json.RawMessage
|
||||
if err := json.Unmarshal(op.Data, &patch); err != nil {
|
||||
return 0, badRequest("%s: %v", op.Kind, err)
|
||||
}
|
||||
delete(patch, "id")
|
||||
o, err := findObject(st, op.Kind, op.Ref)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
id := o.GetID()
|
||||
// The patch's fields over the object's.
|
||||
var cur map[string]json.RawMessage
|
||||
b, _ := json.Marshal(o)
|
||||
json.Unmarshal(b, &cur)
|
||||
for k, v := range patch {
|
||||
if string(v) == "null" {
|
||||
delete(cur, k)
|
||||
} else {
|
||||
cur[k] = v
|
||||
}
|
||||
}
|
||||
merged, _ := json.Marshal(cur)
|
||||
next, err := decodeObject(op.Kind, merged)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
setID(next, id)
|
||||
if old, now := o.Label(), next.Label(); old != now && isNamed(op.Kind) {
|
||||
if set, ok := o.(*model.AddressSet); ok && set.Dynamic {
|
||||
return 0, conflict("set %s is dynamic: its entries, and the API tokens and programs that fill it, go by its name; make a new set instead", old)
|
||||
}
|
||||
renameRefs(st, op.Kind, old, now)
|
||||
}
|
||||
data, _ := json.Marshal(next)
|
||||
return id, st.Put(op.Kind, id, data)
|
||||
}
|
||||
|
||||
// isNamed reports whether other objects refer to a kind by name.
|
||||
func isNamed(kind string) bool {
|
||||
return kind == model.KindZone || kind == model.KindSet || kind == model.KindService
|
||||
}
|
||||
|
||||
func deleteOp(st *model.State, op *model.Op) error {
|
||||
o, err := findObject(st, op.Kind, op.Ref)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if used := usedBy(st, op.Kind, o); used != "" {
|
||||
return conflict("%s %s is used by %s; change that first", op.Kind, o.Label(), used)
|
||||
}
|
||||
return st.Put(op.Kind, o.GetID(), nil)
|
||||
}
|
||||
|
||||
// moveOp reorders a rule or NAT rule: before or after another, or to the
|
||||
// start or end; a rule can change chain at the same time.
|
||||
func moveOp(st *model.State, op *model.Op) (int64, error) {
|
||||
var in api.MoveInput
|
||||
if err := json.Unmarshal(op.Data, &in); err != nil {
|
||||
return 0, badRequest("move: %v", err)
|
||||
}
|
||||
o, err := findObject(st, op.Kind, op.Ref)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
id := o.GetID()
|
||||
pos := position(o)
|
||||
if pos == nil {
|
||||
return 0, badRequest("only rules, port forwards, source NAT rules and redirects have an order")
|
||||
}
|
||||
if rule, ok := o.(*model.Rule); ok && in.Chain != "" {
|
||||
rule.Chain = in.Chain
|
||||
}
|
||||
chain := chainOf(o)
|
||||
// The others in order.
|
||||
var others []model.Object
|
||||
for _, x := range st.Objects(op.Kind) {
|
||||
if x.GetID() != id && chainOf(x) == chain {
|
||||
others = append(others, x)
|
||||
}
|
||||
}
|
||||
sort.SliceStable(others, func(i, j int) bool { return *position(others[i]) < *position(others[j]) })
|
||||
place := func(i int) { // before others[i] (i == len: at the end)
|
||||
switch {
|
||||
case len(others) == 0:
|
||||
*pos = 1
|
||||
case i == 0:
|
||||
*pos = *position(others[0]) - 1
|
||||
case i == len(others):
|
||||
*pos = *position(others[len(others)-1]) + 1
|
||||
default:
|
||||
*pos = (*position(others[i-1]) + *position(others[i])) / 2
|
||||
}
|
||||
}
|
||||
index := func(ref int64) (int, error) {
|
||||
for i, x := range others {
|
||||
if x.GetID() == ref {
|
||||
return i, nil
|
||||
}
|
||||
}
|
||||
return 0, badRequest("%d isn't another %s of the same chain", ref, op.Kind)
|
||||
}
|
||||
switch {
|
||||
case in.Before != 0:
|
||||
i, err := index(in.Before)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
place(i)
|
||||
case in.After != 0:
|
||||
i, err := index(in.After)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
place(i + 1)
|
||||
case in.To == "start":
|
||||
place(0)
|
||||
case in.To == "end" || in.Chain != "":
|
||||
place(len(others))
|
||||
default:
|
||||
return 0, badRequest("say where: before, after, or to start or end")
|
||||
}
|
||||
data, _ := json.Marshal(o)
|
||||
return id, st.Put(op.Kind, id, data)
|
||||
}
|
||||
|
||||
func settingsOp(st *model.State, op *model.Op) error {
|
||||
var patch map[string]json.RawMessage
|
||||
if err := json.Unmarshal(op.Data, &patch); err != nil {
|
||||
return badRequest("settings: %v", err)
|
||||
}
|
||||
var cur map[string]json.RawMessage
|
||||
b, _ := json.Marshal(st.Settings)
|
||||
json.Unmarshal(b, &cur)
|
||||
for k, v := range patch {
|
||||
// Built-ins merge one by one.
|
||||
if k == "builtins" {
|
||||
var cb, pb map[string]json.RawMessage
|
||||
json.Unmarshal(cur[k], &cb)
|
||||
if err := json.Unmarshal(v, &pb); err != nil {
|
||||
return badRequest("builtins: %v", err)
|
||||
}
|
||||
for bk, bv := range pb {
|
||||
cb[bk] = bv
|
||||
}
|
||||
v, _ = json.Marshal(cb)
|
||||
}
|
||||
cur[k] = v
|
||||
}
|
||||
merged, _ := json.Marshal(cur)
|
||||
var next model.Settings
|
||||
dec := json.NewDecoder(strings.NewReader(string(merged)))
|
||||
dec.DisallowUnknownFields()
|
||||
if err := dec.Decode(&next); err != nil {
|
||||
return badRequest("settings: %v", err)
|
||||
}
|
||||
st.Settings = next
|
||||
return nil
|
||||
}
|
||||
|
||||
// replay applies a changeset's operations to st, in order. An operation
|
||||
// that fails is named in the error (opError).
|
||||
func replay(st *model.State, ops []model.Op) error {
|
||||
for i := range ops {
|
||||
if _, err := applyOp(st, &ops[i]); err != nil {
|
||||
return &opError{seq: ops[i].Seq, summary: ops[i].Summary, err: err}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// opError: an operation of a changeset that can't be applied (the
|
||||
// objects it names changed since it was staged, say).
|
||||
type opError struct {
|
||||
seq int
|
||||
summary string
|
||||
err error
|
||||
}
|
||||
|
||||
func (e *opError) Error() string {
|
||||
return fmt.Sprintf("operation %d (%s) can't be applied: %v", e.seq, e.summary, e.err)
|
||||
}
|
||||
|
||||
func (e *opError) Unwrap() error { return e.err }
|
||||
|
||||
// info is the error for the API, with the inner error's status.
|
||||
func (e *opError) info() (api.ErrorInfo, int) {
|
||||
info := api.ErrorInfo{Code: "conflict", Message: e.Error(), Op: e.seq}
|
||||
status := http.StatusConflict
|
||||
var he *httpError
|
||||
if errors.As(e.err, &he) {
|
||||
info.Code, status = he.code, he.status
|
||||
}
|
||||
if ee, ok := engine.AsError(e.err); ok {
|
||||
info.Code, info.Problems, status = ee.Kind, ee.Problems, engineStatus(ee.Kind)
|
||||
}
|
||||
return info, status
|
||||
}
|
||||
|
||||
// objectOf picks what an operation's answer returns from a state.
|
||||
func objectOf(op *model.Op, id int64) func(st *model.State) any {
|
||||
switch op.Op {
|
||||
case model.OpDelete:
|
||||
return nil
|
||||
case model.OpSettings:
|
||||
return func(st *model.State) any { return st.Settings }
|
||||
}
|
||||
return func(st *model.State) any {
|
||||
if o := st.ByID(op.Kind, id); o != nil {
|
||||
return o
|
||||
}
|
||||
return nil
|
||||
}
|
||||
}
|
||||
@@ -2,8 +2,10 @@ package server
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/subtle"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
@@ -31,6 +33,21 @@ type probation struct {
|
||||
timer *time.Timer
|
||||
// restores are nft scripts that bring back tables a takeover deleted.
|
||||
restores []string
|
||||
// check is the latest change's confirmation token (reachCheck): only
|
||||
// it can confirm, since the ruleset in force is the one it tests.
|
||||
check *reachCheck
|
||||
}
|
||||
|
||||
// reachCheck is what a client must show to confirm its change by itself:
|
||||
// the token, from the same user (and API token), the same address, and a
|
||||
// new connection (another remote address and port than the change's).
|
||||
type reachCheck struct {
|
||||
token string
|
||||
userID int64
|
||||
username string
|
||||
tokenID int64
|
||||
ip string
|
||||
remote string
|
||||
}
|
||||
|
||||
// system is the principal of rollbacks.
|
||||
@@ -88,6 +105,60 @@ func (s *Server) startProbation(ctx context.Context, p *auth.Principal, historyI
|
||||
s.saveProbation(ctx)
|
||||
}
|
||||
|
||||
// issueCheck gives the client that made a network change on probation a
|
||||
// token to confirm it by itself, unless the settings want a person to
|
||||
// confirm. The answer then closes its connection (HTTP/2: a GOAWAY), so
|
||||
// the confirmation has to open a new one through the new ruleset; an
|
||||
// established connection would get through whatever the change did.
|
||||
func (s *Server) issueCheck(w http.ResponseWriter, r *http.Request, p *auth.Principal, res *engine.Result) string {
|
||||
if p == nil || p.Method == "socket" || (res != nil && !res.Applied) || s.eng.State().Settings.ManualConfirm {
|
||||
return ""
|
||||
}
|
||||
pr := &s.probation
|
||||
pr.mu.Lock()
|
||||
defer pr.mu.Unlock()
|
||||
if len(pr.pending) == 0 && len(pr.restores) == 0 {
|
||||
return ""
|
||||
}
|
||||
c := &reachCheck{token: auth.RandomString(24), username: p.Name(), ip: s.clientIP(r), remote: r.RemoteAddr}
|
||||
if p.User != nil {
|
||||
c.userID = p.User.ID
|
||||
}
|
||||
if p.Token != nil {
|
||||
c.tokenID = p.Token.ID
|
||||
}
|
||||
pr.check = c
|
||||
w.Header().Set("Connection", "close")
|
||||
return c.token
|
||||
}
|
||||
|
||||
// confirmToken confirms with a change's token: the proof that the client
|
||||
// that made it still reaches the API, from where it made it.
|
||||
func (s *Server) confirmToken(r *http.Request, p *auth.Principal, token string) (int, error) {
|
||||
pr := &s.probation
|
||||
pr.mu.Lock()
|
||||
c := pr.check
|
||||
pending := len(pr.pending) + len(pr.restores)
|
||||
pr.mu.Unlock()
|
||||
var tokenID int64
|
||||
if p.Token != nil {
|
||||
tokenID = p.Token.ID
|
||||
}
|
||||
switch {
|
||||
case pending == 0:
|
||||
return 0, nil
|
||||
case c == nil || subtle.ConstantTimeCompare([]byte(c.token), []byte(token)) != 1:
|
||||
return 0, conflict("this confirmation is for an earlier change: a newer one is on probation, and only its own client (or a person) can confirm it")
|
||||
case p.User == nil || p.User.ID != c.userID || p.Name() != c.username || tokenID != c.tokenID:
|
||||
return 0, forbidden("only the client that made the change can confirm it this way")
|
||||
case s.clientIP(r) != c.ip:
|
||||
return 0, forbidden("the confirmation comes from %s, the change from %s: it must come from the same place", s.clientIP(r), c.ip)
|
||||
case r.RemoteAddr == c.remote:
|
||||
return 0, conflict("the confirmation came over the change's own connection, which proves nothing; it must open a new one")
|
||||
}
|
||||
return s.confirmWith(r.Context(), p, fmt.Sprintf("confirmed automatically: %s reached the API again over a new connection", c.ip))
|
||||
}
|
||||
|
||||
func (s *Server) saveBoot() {
|
||||
if err := s.fw.SaveBoot(s.eng.State()); err != nil {
|
||||
s.log.Error("saving the ruleset for the next boot failed", "err", err)
|
||||
@@ -110,8 +181,12 @@ func (s *Server) probationStatus() api.Probation {
|
||||
return out
|
||||
}
|
||||
|
||||
// confirm makes the pending changes permanent.
|
||||
// confirm makes the pending changes permanent (a person's confirmation).
|
||||
func (s *Server) confirm(ctx context.Context, p *auth.Principal) (int, error) {
|
||||
return s.confirmWith(ctx, p, "")
|
||||
}
|
||||
|
||||
func (s *Server) confirmWith(ctx context.Context, p *auth.Principal, how string) (int, error) {
|
||||
pr := &s.probation
|
||||
pr.mu.Lock()
|
||||
defer pr.mu.Unlock()
|
||||
@@ -122,10 +197,14 @@ func (s *Server) confirm(ctx context.Context, p *auth.Principal) (int, error) {
|
||||
if pr.timer != nil {
|
||||
pr.timer.Stop()
|
||||
}
|
||||
pr.pending, pr.restores, pr.deadline, pr.timer = nil, nil, time.Time{}, nil
|
||||
pr.pending, pr.restores, pr.deadline, pr.timer, pr.check = nil, nil, time.Time{}, nil, nil
|
||||
s.saveProbation(ctx)
|
||||
s.saveBoot()
|
||||
s.eng.Audit(ctx, p, "config.confirm", "", "success", fmt.Sprintf("%d changes confirmed", n))
|
||||
detail := fmt.Sprintf("%d changes confirmed", n)
|
||||
if how != "" {
|
||||
detail += "; " + how
|
||||
}
|
||||
s.eng.Audit(ctx, p, "config.confirm", "", "success", detail)
|
||||
return n, nil
|
||||
}
|
||||
|
||||
@@ -136,7 +215,7 @@ func (s *Server) rollback(why string) {
|
||||
pr.mu.Lock()
|
||||
defer pr.mu.Unlock()
|
||||
pending, restores := pr.pending, pr.restores
|
||||
pr.pending, pr.restores, pr.deadline, pr.timer = nil, nil, time.Time{}, nil
|
||||
pr.pending, pr.restores, pr.deadline, pr.timer, pr.check = nil, nil, time.Time{}, nil, nil
|
||||
for i := len(restores) - 1; i >= 0; i-- {
|
||||
if err := s.fw.ApplyScript(ctx, restores[i]); err != nil {
|
||||
s.log.Error("bringing back a table a takeover deleted failed", "err", err)
|
||||
|
||||
@@ -0,0 +1,263 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"sort"
|
||||
"time"
|
||||
|
||||
"housecarl/internal/api"
|
||||
"housecarl/internal/auth"
|
||||
"housecarl/internal/model"
|
||||
"housecarl/internal/nft"
|
||||
"housecarl/internal/render"
|
||||
)
|
||||
|
||||
// Quotas: nftables counts them, and never resets them. housecarld resets
|
||||
// each at the start of its period (daily, weekly, monthly), or when asked,
|
||||
// and saves what they used every few minutes and when it stops, so a
|
||||
// reboot (whose boot ruleset starts them at zero) doesn't lose it.
|
||||
|
||||
const (
|
||||
quotaResetsKey = "quota_resets" // rule mark → when its quota was last reset
|
||||
QuotaUsageKey = "quota_usage" // nft.QuotaUsage, saved
|
||||
)
|
||||
|
||||
// periodStart is when the current period of a reset began (zero for a
|
||||
// quota reset only by hand), in local time.
|
||||
func periodStart(now time.Time, reset string, day int) time.Time {
|
||||
y, m, d := now.Date()
|
||||
midnight := time.Date(y, m, d, 0, 0, 0, 0, now.Location())
|
||||
switch reset {
|
||||
case "daily":
|
||||
return midnight
|
||||
case "weekly":
|
||||
back := (int(now.Weekday()) + 6) % 7 // Monday
|
||||
return midnight.AddDate(0, 0, -back)
|
||||
case "monthly":
|
||||
if day < 1 {
|
||||
day = 1
|
||||
}
|
||||
start := monthDay(y, m, day, now.Location())
|
||||
if start.After(now) {
|
||||
py, pm, _ := time.Date(y, m, 1, 0, 0, 0, 0, now.Location()).AddDate(0, -1, 0).Date()
|
||||
start = monthDay(py, pm, day, now.Location())
|
||||
}
|
||||
return start
|
||||
}
|
||||
return time.Time{}
|
||||
}
|
||||
|
||||
// monthDay is midnight on a day of a month, or on its last day when the
|
||||
// month is shorter.
|
||||
func monthDay(y int, m time.Month, day int, loc *time.Location) time.Time {
|
||||
last := time.Date(y, m+1, 0, 0, 0, 0, 0, loc).Day()
|
||||
if day > last {
|
||||
day = last
|
||||
}
|
||||
return time.Date(y, m, day, 0, 0, 0, 0, loc)
|
||||
}
|
||||
|
||||
// nextReset is when the period after the current one starts.
|
||||
func nextReset(now time.Time, reset string, day int) time.Time {
|
||||
start := periodStart(now, reset, day)
|
||||
switch reset {
|
||||
case "daily":
|
||||
return start.AddDate(0, 0, 1)
|
||||
case "weekly":
|
||||
return start.AddDate(0, 0, 7)
|
||||
case "monthly":
|
||||
y, m, _ := start.Date()
|
||||
ny, nm, _ := time.Date(y, m, 1, 0, 0, 0, 0, now.Location()).AddDate(0, 1, 0).Date()
|
||||
if day < 1 {
|
||||
day = 1
|
||||
}
|
||||
return monthDay(ny, nm, day, now.Location())
|
||||
}
|
||||
return time.Time{}
|
||||
}
|
||||
|
||||
func (s *Server) quotaResets(ctx context.Context) map[string]time.Time {
|
||||
out := map[string]time.Time{}
|
||||
if v, err := s.store.GetMeta(ctx, quotaResetsKey); err == nil && v != "" {
|
||||
json.Unmarshal([]byte(v), &out)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func (s *Server) saveQuotaResets(ctx context.Context, m map[string]time.Time) {
|
||||
b, _ := json.Marshal(m)
|
||||
if err := s.store.SetMeta(ctx, quotaResetsKey, string(b)); err != nil {
|
||||
s.log.Error("saving the quotas' reset times failed", "err", err)
|
||||
}
|
||||
}
|
||||
|
||||
// RunQuotas resets quotas when their period ends and saves their usage,
|
||||
// until ctx ends (then it saves once more).
|
||||
func (s *Server) RunQuotas(ctx context.Context) {
|
||||
s.checkQuotas(ctx)
|
||||
tick := time.NewTicker(time.Minute)
|
||||
defer tick.Stop()
|
||||
n := 0
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
s.SaveQuotaUsage(context.Background())
|
||||
return
|
||||
case <-tick.C:
|
||||
s.checkQuotas(ctx)
|
||||
if n++; n%5 == 0 {
|
||||
s.SaveQuotaUsage(ctx)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// checkQuotas resets the quotas whose period has ended since their last
|
||||
// reset. A quota seen for the first time starts its period now.
|
||||
func (s *Server) checkQuotas(ctx context.Context) {
|
||||
st := s.eng.State()
|
||||
resets := s.quotaResets(ctx)
|
||||
now := s.nowFn()
|
||||
changed := false
|
||||
seen := map[string]bool{}
|
||||
for _, r := range st.Rules {
|
||||
if r.Quota == nil || r.Disabled || r.Raw != "" {
|
||||
continue
|
||||
}
|
||||
key := fmt.Sprintf("r%d", r.ID)
|
||||
seen[key] = true
|
||||
last, ok := resets[key]
|
||||
if !ok {
|
||||
resets[key], changed = now, true
|
||||
continue
|
||||
}
|
||||
start := periodStart(now, r.Quota.Reset, r.Quota.Day)
|
||||
if start.IsZero() || !last.Before(start) {
|
||||
continue
|
||||
}
|
||||
if err := s.fw.ResetQuota(ctx, r.ID, r.Quota.PerAddress() != ""); err != nil {
|
||||
s.log.Error("resetting a quota failed", "rule", r.Label(), "err", err)
|
||||
continue
|
||||
}
|
||||
s.log.Info("quota reset for its new period", "rule", r.Label(), "period", r.Quota.Reset)
|
||||
resets[key], changed = now, true
|
||||
s.forgetQuotaUsage(ctx, r.ID)
|
||||
}
|
||||
for key := range resets {
|
||||
if !seen[key] {
|
||||
delete(resets, key)
|
||||
changed = true
|
||||
}
|
||||
}
|
||||
if changed {
|
||||
s.saveQuotaResets(ctx, resets)
|
||||
}
|
||||
}
|
||||
|
||||
// SaveQuotaUsage saves what the quotas used, for after a reboot.
|
||||
func (s *Server) SaveQuotaUsage(ctx context.Context) {
|
||||
t, err := s.fw.Table(ctx)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
b, _ := json.Marshal(t.QuotaUsage)
|
||||
if err := s.store.SetMeta(ctx, QuotaUsageKey, string(b)); err != nil {
|
||||
s.log.Error("saving the quotas' usage failed", "err", err)
|
||||
}
|
||||
}
|
||||
|
||||
// forgetQuotaUsage drops a reset quota from the saved usage, so a restart
|
||||
// doesn't bring back what it used before.
|
||||
func (s *Server) forgetQuotaUsage(ctx context.Context, ruleID int64) {
|
||||
v, err := s.store.GetMeta(ctx, QuotaUsageKey)
|
||||
if err != nil || v == "" {
|
||||
return
|
||||
}
|
||||
var u nft.QuotaUsage
|
||||
if json.Unmarshal([]byte(v), &u) != nil {
|
||||
return
|
||||
}
|
||||
name := render.QuotaName(ruleID)
|
||||
delete(u.Quotas, name)
|
||||
delete(u.QuotaSets, name+"_v4")
|
||||
delete(u.QuotaSets, name+"_v6")
|
||||
b, _ := json.Marshal(u)
|
||||
s.store.SetMeta(ctx, QuotaUsageKey, string(b))
|
||||
}
|
||||
|
||||
// listQuotas answers what each rule's quota used.
|
||||
func (s *Server) listQuotas(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResLog, false); err != nil {
|
||||
return err
|
||||
}
|
||||
st := s.eng.State()
|
||||
var usage nft.QuotaUsage
|
||||
if t, err := s.fw.Table(r.Context()); err == nil {
|
||||
usage = t.QuotaUsage
|
||||
}
|
||||
resets := s.quotaResets(r.Context())
|
||||
now := s.nowFn()
|
||||
out := []api.QuotaInfo{}
|
||||
for _, rule := range st.Rules {
|
||||
if rule.Quota == nil {
|
||||
continue
|
||||
}
|
||||
q := api.QuotaInfo{RuleID: rule.ID, Rule: rule.Label(), Quota: *rule.Quota}
|
||||
name := render.QuotaName(rule.ID)
|
||||
if rule.Quota.PerAddress() == "" {
|
||||
q.Used = usage.Quotas[name]
|
||||
} else {
|
||||
v4, v6 := rule.Groups()
|
||||
for _, fam := range []string{"_v4", "_v6"} {
|
||||
for _, e := range usage.QuotaSets[name+fam] {
|
||||
addr := e.Addr
|
||||
if fam == "_v4" && v4 < 32 {
|
||||
addr = fmt.Sprintf("%s/%d", addr, v4)
|
||||
} else if fam == "_v6" && v6 < 128 {
|
||||
addr = fmt.Sprintf("%s/%d", addr, v6)
|
||||
}
|
||||
q.Addresses = append(q.Addresses, api.QuotaAddress{Address: addr, Used: e.Used})
|
||||
q.Used += e.Used
|
||||
}
|
||||
}
|
||||
sort.Slice(q.Addresses, func(i, j int) bool { return q.Addresses[i].Used > q.Addresses[j].Used })
|
||||
}
|
||||
if last, ok := resets[fmt.Sprintf("r%d", rule.ID)]; ok {
|
||||
q.LastReset = &last
|
||||
}
|
||||
if next := nextReset(now, rule.Quota.Reset, rule.Quota.Day); !next.IsZero() {
|
||||
q.NextReset = &next
|
||||
}
|
||||
out = append(out, q)
|
||||
}
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
return nil
|
||||
}
|
||||
|
||||
// resetQuota starts a rule's quota again now.
|
||||
func (s *Server) resetQuota(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
|
||||
if err := require(p, auth.ResConfig, true); err != nil {
|
||||
return err
|
||||
}
|
||||
o, err := findObject(s.eng.State(), model.KindRule, r.PathValue("ref"))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
rule := o.(*model.Rule)
|
||||
if rule.Quota == nil {
|
||||
return badRequest("%s has no quota", rule.Label())
|
||||
}
|
||||
if err := s.fw.ResetQuota(r.Context(), rule.ID, rule.Quota.PerAddress() != ""); err != nil {
|
||||
return err
|
||||
}
|
||||
resets := s.quotaResets(r.Context())
|
||||
resets[fmt.Sprintf("r%d", rule.ID)] = s.nowFn()
|
||||
s.saveQuotaResets(r.Context(), resets)
|
||||
s.forgetQuotaUsage(r.Context(), rule.ID)
|
||||
s.eng.Audit(r.Context(), p, "quota.reset", rule.Label(), "success", "")
|
||||
writeJSON(w, http.StatusOK, map[string]any{"reset": rule.Label()})
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,84 @@
|
||||
package server
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"housecarl/internal/api"
|
||||
)
|
||||
|
||||
// TestRename: renaming a zone, set or service changes what refers to it,
|
||||
// in the same change, which undoes as one.
|
||||
func TestRename(t *testing.T) {
|
||||
e := newEnv(t)
|
||||
e.srv.probationFor = 0
|
||||
post := func(path string, body map[string]any) {
|
||||
t.Helper()
|
||||
if code := e.call(e.token, "POST", path, body, nil); code != 201 {
|
||||
t.Fatalf("POST %s: %d", path, code)
|
||||
}
|
||||
}
|
||||
post("/zones", map[string]any{"name": "wan", "interfaces": []string{"wan"}, "external": true})
|
||||
post("/zones", map[string]any{"name": "dhcp", "interfaces": []string{"eth1"}})
|
||||
post("/sets", map[string]any{"name": "trusted_ips", "entries": []string{"10.0.10.0/24"}})
|
||||
post("/sets", map[string]any{"name": "guests", "dynamic": true})
|
||||
post("/services", map[string]any{"name": "jarl", "entries": []string{"tcp/8053"}})
|
||||
post("/rules", map[string]any{"chain": "forward", "in_zones": []string{"dhcp"}, "out_zones": []string{"wan"}, "sources": []string{"trusted_ips"}})
|
||||
post("/rules", map[string]any{"chain": "input", "in_zones": []string{"dhcp"}, "services": []string{"jarl"}})
|
||||
post("/snats", map[string]any{"out_zones": []string{"wan"}, "sources": []string{"trusted_ips"}})
|
||||
post("/forwards", map[string]any{"name": "jf", "ports": "8096", "to": "10.0.10.57", "public_addresses": []string{"217.155.8.194"}, "sources": []string{"trusted_ips"}})
|
||||
|
||||
var res api.MutationResponse
|
||||
if code := e.call(e.token, "PATCH", "/zones/dhcp", map[string]any{"name": "lan"}, &res); code != 200 {
|
||||
t.Fatalf("renaming the zone: %d %+v", code, res)
|
||||
}
|
||||
zoneRename := res.HistoryID
|
||||
e.call(e.token, "PATCH", "/sets/trusted_ips", map[string]any{"name": "trusted"}, &res)
|
||||
setRename := res.HistoryID
|
||||
e.call(e.token, "PATCH", "/services/jarl", map[string]any{"name": "jarl-api"}, &res)
|
||||
serviceRename := res.HistoryID
|
||||
st := e.eng.State()
|
||||
if st.Zone("dhcp") != nil || st.Zone("lan") == nil {
|
||||
t.Fatalf("zones %+v", st.Zones)
|
||||
}
|
||||
for _, r := range st.Rules {
|
||||
if strings.Join(r.InZones, ",") != "lan" {
|
||||
t.Errorf("rule %d in zones %v", r.ID, r.InZones)
|
||||
}
|
||||
if r.Chain == "forward" && strings.Join(r.Sources, ",") != "trusted" {
|
||||
t.Errorf("rule %d sources %v", r.ID, r.Sources)
|
||||
}
|
||||
if r.Chain == "input" && strings.Join(r.Services, ",") != "jarl-api" {
|
||||
t.Errorf("rule %d services %v", r.ID, r.Services)
|
||||
}
|
||||
}
|
||||
if s := st.SNATs[0].Sources; strings.Join(s, ",") != "trusted" {
|
||||
t.Errorf("snat sources %v", s)
|
||||
}
|
||||
if s := st.Forwards[0].Sources; strings.Join(s, ",") != "trusted" {
|
||||
t.Errorf("forward sources %v", s)
|
||||
}
|
||||
if ps := e.eng.Problems(); len(ps) != 0 {
|
||||
for _, p := range ps {
|
||||
if p.Severity == "error" {
|
||||
t.Errorf("problem after renaming: %+v", p)
|
||||
}
|
||||
}
|
||||
}
|
||||
// Undoing the renames (newest first) puts the references back too.
|
||||
for _, id := range []int64{serviceRename, setRename, zoneRename} {
|
||||
if code := e.call(e.token, "POST", "/history/"+itoa(id)+"/revert", nil, nil); code != 200 {
|
||||
t.Fatalf("undo %d: %d", id, code)
|
||||
}
|
||||
}
|
||||
st = e.eng.State()
|
||||
for _, r := range st.Rules {
|
||||
if strings.Join(r.InZones, ",") != "dhcp" || r.Chain == "forward" && r.Sources[0] != "trusted_ips" || r.Chain == "input" && r.Services[0] != "jarl" {
|
||||
t.Errorf("after undo: rule %+v", r.Match)
|
||||
}
|
||||
}
|
||||
// A dynamic set keeps its name: tokens and programs go by it.
|
||||
if code := e.call(e.token, "PATCH", "/sets/guests", map[string]any{"name": "visitors"}, nil); code != 409 {
|
||||
t.Errorf("renaming a dynamic set: %d", code)
|
||||
}
|
||||
}
|
||||
@@ -70,6 +70,12 @@ func engineStatus(kind string) int {
|
||||
// writeErr writes an error response. result, if given, is the preview of
|
||||
// a refused change.
|
||||
func (s *Server) writeErr(w http.ResponseWriter, r *http.Request, err error, result any) {
|
||||
var oe *opError
|
||||
if errors.As(err, &oe) {
|
||||
info, status := oe.info()
|
||||
writeJSON(w, status, api.ErrorBody{Error: info})
|
||||
return
|
||||
}
|
||||
var he *httpError
|
||||
if errors.As(err, &he) {
|
||||
writeError(w, he.status, he.code, he.msg)
|
||||
|
||||
@@ -10,15 +10,12 @@ import (
|
||||
"time"
|
||||
|
||||
"housecarl/internal/api"
|
||||
"housecarl/internal/version"
|
||||
)
|
||||
|
||||
func (s *Server) routes() {
|
||||
m := s.mux
|
||||
p := api.Prefix
|
||||
m.HandleFunc("GET /healthz", func(w http.ResponseWriter, r *http.Request) {
|
||||
writeJSON(w, http.StatusOK, map[string]any{"ok": true, "version": version.String()})
|
||||
})
|
||||
m.HandleFunc("GET /healthz", s.healthz)
|
||||
// Authentication, users and tokens
|
||||
m.HandleFunc("POST "+p+"/auth/login", s.public(s.login))
|
||||
m.HandleFunc("POST "+p+"/auth/logout", s.authed(s.logout))
|
||||
@@ -41,6 +38,21 @@ func (s *Server) routes() {
|
||||
m.HandleFunc("GET "+p+"/sets/{ref}/elements", s.authed(s.listElements))
|
||||
m.HandleFunc("POST "+p+"/sets/{ref}/elements", s.authed(s.addElement))
|
||||
m.HandleFunc("DELETE "+p+"/sets/{ref}/elements/{addr}", s.authed(s.deleteElement))
|
||||
m.HandleFunc("GET "+p+"/quotas", s.authed(s.listQuotas))
|
||||
m.HandleFunc("GET "+p+"/connections", s.authed(s.listConnections))
|
||||
m.HandleFunc("POST "+p+"/connections/close", s.authed(s.closeConnections))
|
||||
m.HandleFunc("GET "+p+"/trace", s.authed(s.getTrace))
|
||||
m.HandleFunc("POST "+p+"/trace", s.authed(s.startTrace))
|
||||
m.HandleFunc("DELETE "+p+"/trace", s.authed(s.stopTrace))
|
||||
m.HandleFunc("POST "+p+"/rules/{ref}/reset-quota", s.authed(s.resetQuota))
|
||||
m.HandleFunc("GET "+p+"/changesets", s.authed(s.listChangesets))
|
||||
m.HandleFunc("POST "+p+"/changesets", s.authed(s.createChangeset))
|
||||
m.HandleFunc("GET "+p+"/changesets/{id}", s.authed(s.getChangeset))
|
||||
m.HandleFunc("PATCH "+p+"/changesets/{id}", s.authed(s.updateChangeset))
|
||||
m.HandleFunc("GET "+p+"/changesets/{id}/preview", s.authed(s.previewChangeset))
|
||||
m.HandleFunc("POST "+p+"/changesets/{id}/apply", s.authed(s.applyChangeset))
|
||||
m.HandleFunc("POST "+p+"/changesets/{id}/discard", s.authed(s.discardChangeset))
|
||||
m.HandleFunc("DELETE "+p+"/changesets/{id}/ops/{seq}", s.authed(s.removeOp))
|
||||
m.HandleFunc("GET "+p+"/{kind}", s.authed(s.listObjects))
|
||||
m.HandleFunc("POST "+p+"/{kind}", s.authed(s.createObject))
|
||||
m.HandleFunc("GET "+p+"/{kind}/{ref}", s.authed(s.getObject))
|
||||
@@ -48,6 +60,13 @@ func (s *Server) routes() {
|
||||
m.HandleFunc("DELETE "+p+"/{kind}/{ref}", s.authed(s.deleteObject))
|
||||
m.HandleFunc("POST "+p+"/{kind}/{ref}/move", s.authed(s.moveObject))
|
||||
m.HandleFunc("GET "+p+"/ruleset", s.authed(s.ruleset))
|
||||
m.HandleFunc("GET "+p+"/matrix", s.authed(s.zoneMatrix))
|
||||
m.HandleFunc("GET "+p+"/stats/status", s.authed(s.statsStatus))
|
||||
m.HandleFunc("GET "+p+"/stats/summary", s.authed(s.statsSummary))
|
||||
m.HandleFunc("GET "+p+"/stats/timeline", s.authed(s.statsTimeline))
|
||||
m.HandleFunc("GET "+p+"/stats/top", s.authed(s.statsTop))
|
||||
m.HandleFunc("GET "+p+"/stats/hits", s.authed(s.statsHits))
|
||||
m.HandleFunc("GET "+p+"/stats/heatmap", s.authed(s.statsHeatmap))
|
||||
m.HandleFunc("GET "+p+"/history", s.authed(s.listHistory))
|
||||
m.HandleFunc("GET "+p+"/history/{id}", s.authed(s.getHistory))
|
||||
m.HandleFunc("POST "+p+"/history/{id}/revert", s.authed(s.revertHistory))
|
||||
|
||||
@@ -1,6 +1,10 @@
|
||||
package server
|
||||
|
||||
import "encoding/json"
|
||||
import (
|
||||
"encoding/json"
|
||||
|
||||
"housecarl/internal/trace"
|
||||
)
|
||||
|
||||
// otherTables lists the tables in an `nft -j list ruleset` other than
|
||||
// Housecarl's ("inet main").
|
||||
@@ -18,7 +22,7 @@ func otherTables(data []byte) []string {
|
||||
continue
|
||||
}
|
||||
var t struct{ Family, Name string }
|
||||
if json.Unmarshal(raw, &t) == nil && !(t.Family == "inet" && t.Name == "housecarl") {
|
||||
if json.Unmarshal(raw, &t) == nil && !(t.Family == "inet" && (t.Name == "housecarl" || t.Name == trace.Table)) {
|
||||
out = append(out, t.Family+" "+t.Name)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -24,6 +24,7 @@ import (
|
||||
"housecarl/internal/model"
|
||||
"housecarl/internal/nft"
|
||||
"housecarl/internal/render"
|
||||
"housecarl/internal/stats"
|
||||
"housecarl/internal/store"
|
||||
"housecarl/internal/version"
|
||||
)
|
||||
@@ -65,6 +66,10 @@ type Firewall interface {
|
||||
TableText(ctx context.Context, family, name string) (string, error)
|
||||
ApplyScript(ctx context.Context, script string) error
|
||||
SaveBoot(st *model.State) error
|
||||
// ResetQuota starts a rule's quota again (perAddress: its sets).
|
||||
ResetQuota(ctx context.Context, ruleID int64, perAddress bool) error
|
||||
// Trace follows packets (internal/trace) until ctx ends.
|
||||
Trace(ctx context.Context, script string, line func(string)) error
|
||||
}
|
||||
|
||||
// Server is the HTTP server.
|
||||
@@ -85,7 +90,19 @@ type Server struct {
|
||||
srvs []*http.Server
|
||||
|
||||
probation probation
|
||||
health healthCache
|
||||
logs *logRing
|
||||
// unitLinks are where an enabled nftables.service shows (tests change
|
||||
// them).
|
||||
unitLinks []string
|
||||
// conns is the kernel's connection tracking (nil: unavailable).
|
||||
conns Conntrack
|
||||
// tracing is the packet trace running (or the last one).
|
||||
tracing tracing
|
||||
// stats collects statistics (nil: off).
|
||||
stats *stats.Collector
|
||||
// csMu orders changes to changesets (staging, applying, discarding).
|
||||
csMu sync.Mutex
|
||||
nowFn func() time.Time
|
||||
// probationFor overrides the settings' probation (tests).
|
||||
probationFor time.Duration
|
||||
@@ -109,7 +126,7 @@ func New(cfg Config, eng *engine.Engine, fw Firewall, ifaces Interfaces, log *sl
|
||||
log = slog.Default()
|
||||
}
|
||||
s := &Server{cfg: cfg, eng: eng, fw: fw, ifaces: ifaces, store: eng.Store(), log: log, mux: http.NewServeMux(), limiter: newRateLimiter(),
|
||||
web: web, started: time.Now(), logs: newLogRing(2000), nowFn: time.Now}
|
||||
web: web, started: time.Now(), logs: newLogRing(2000), nowFn: time.Now, unitLinks: nftablesUnitLinks}
|
||||
s.routes()
|
||||
return s
|
||||
}
|
||||
|
||||
@@ -5,6 +5,7 @@ import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/cookiejar"
|
||||
@@ -22,6 +23,7 @@ import (
|
||||
"housecarl/internal/model"
|
||||
"housecarl/internal/nft"
|
||||
"housecarl/internal/render"
|
||||
"housecarl/internal/stats"
|
||||
"housecarl/internal/store"
|
||||
)
|
||||
|
||||
@@ -35,6 +37,10 @@ type fakeFW struct {
|
||||
applyErr error
|
||||
deleted []string
|
||||
restored []string
|
||||
resets []string
|
||||
traced string
|
||||
traceLines []string
|
||||
fallback bool // the boot fallback is in force
|
||||
}
|
||||
|
||||
func (f *fakeFW) Check(_ context.Context, st *model.State) error { return nil }
|
||||
@@ -59,7 +65,7 @@ func (f *fakeFW) Table(context.Context) (*nft.Table, error) {
|
||||
if len(f.applied) == 0 {
|
||||
return nil, errors.New("No such file or directory: list table inet housecarl")
|
||||
}
|
||||
return &nft.Table{Counters: map[string]nft.Counter{"r1": {Packets: 3, Bytes: 180}}, Elements: f.elements}, nil
|
||||
return &nft.Table{Counters: map[string]nft.Counter{"r1": {Packets: 3, Bytes: 180}}, Elements: f.elements, Fallback: f.fallback}, nil
|
||||
}
|
||||
func (f *fakeFW) AddElement(_ context.Context, set, addr string, timeout int) error {
|
||||
f.mu.Lock()
|
||||
@@ -88,6 +94,23 @@ func (f *fakeFW) DeleteTable(_ context.Context, fam, name string) error {
|
||||
f.deleted = append(f.deleted, fam+" "+name)
|
||||
return nil
|
||||
}
|
||||
func (f *fakeFW) ResetQuota(_ context.Context, id int64, perAddress bool) error {
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
f.resets = append(f.resets, fmt.Sprintf("r%d:%v", id, perAddress))
|
||||
return nil
|
||||
}
|
||||
func (f *fakeFW) Trace(ctx context.Context, script string, line func(string)) error {
|
||||
f.mu.Lock()
|
||||
f.traced = script
|
||||
lines := f.traceLines
|
||||
f.mu.Unlock()
|
||||
for _, l := range lines {
|
||||
line(l)
|
||||
}
|
||||
<-ctx.Done()
|
||||
return nil
|
||||
}
|
||||
func (f *fakeFW) TableText(_ context.Context, fam, name string) (string, error) {
|
||||
return "table " + fam + " " + name + " {}", nil
|
||||
}
|
||||
@@ -538,3 +561,502 @@ func TestImportAndTakeover(t *testing.T) {
|
||||
t.Errorf("taking over Housecarl's own table: %d", code)
|
||||
}
|
||||
}
|
||||
|
||||
// TestChangesets: changes staged in a changeset are checked as they are
|
||||
// added, shown in reads that name the changeset, previewed with the
|
||||
// ruleset's diff, and applied together as one history entry, on
|
||||
// probation; another user can't use the changeset.
|
||||
func TestChangesets(t *testing.T) {
|
||||
e := newEnv(t)
|
||||
e.srv.probationFor = 5 * time.Second
|
||||
e.call(e.token, "POST", "/zones", model.Zone{Name: "lan", Interfaces: []string{"eth1"}}, nil)
|
||||
e.call(e.token, "POST", "/confirm", nil, nil)
|
||||
applied := len(e.fw.applied)
|
||||
var cs model.Changeset
|
||||
if code := e.call(e.token, "POST", "/changesets", api.ChangesetInput{Description: "trusted ssh"}, &cs); code != 201 || cs.Status != "open" {
|
||||
t.Fatalf("create: %d %+v", code, cs)
|
||||
}
|
||||
q := "?changeset=" + itoa(cs.ID)
|
||||
// A rule naming a set the changeset hasn't made yet is refused.
|
||||
var eb api.ErrorBody
|
||||
rule := model.Rule{Chain: "input", Match: model.Match{Sources: []string{"trusted"}, Services: []string{"ssh"}}, Action: "accept"}
|
||||
if code := e.call(e.token, "POST", "/rules"+q, rule, &eb); code != 422 || eb.Error.Code != "validation" {
|
||||
t.Errorf("staging an invalid rule: %d %+v", code, eb)
|
||||
}
|
||||
var mr api.MutationResponse
|
||||
if code := e.call(e.token, "POST", "/sets"+q, model.AddressSet{Name: "trusted", Entries: []string{"10.0.10.0/24"}}, &mr); code != 200 || !mr.Staged || mr.Applied {
|
||||
t.Fatalf("stage set: %d %+v", code, mr)
|
||||
}
|
||||
if code := e.call(e.token, "POST", "/rules"+q, rule, &mr); code != 200 || !mr.Staged || len(mr.Changeset.Ops) != 2 {
|
||||
t.Fatalf("stage rule: %d %+v", code, mr)
|
||||
}
|
||||
var created model.Rule
|
||||
json.Unmarshal(mr.Object, &created)
|
||||
// Later operations can name what the changeset made.
|
||||
if code := e.call(e.token, "PATCH", "/rules/"+itoa(created.ID)+q, map[string]any{"comment": "SSH from home"}, &mr); code != 200 || len(mr.Changeset.Ops) != 3 {
|
||||
t.Fatalf("stage update: %d %+v", code, mr)
|
||||
}
|
||||
if code := e.call(e.token, "DELETE", "/zones/lan"+q, nil, &mr); code != 200 {
|
||||
t.Fatalf("stage delete: %d %+v", code, mr)
|
||||
}
|
||||
// Nothing applied yet.
|
||||
if len(e.fw.applied) != applied || len(e.eng.State().Sets) != 0 {
|
||||
t.Fatalf("staging applied something")
|
||||
}
|
||||
// Reads naming the changeset show it, marked.
|
||||
var zones []map[string]any
|
||||
e.call(e.token, "GET", "/zones"+q, nil, &zones)
|
||||
if len(zones) != 1 || zones[0]["pending"] != "deleted" {
|
||||
t.Errorf("zones in the changeset: %+v", zones)
|
||||
}
|
||||
var rules []map[string]any
|
||||
e.call(e.token, "GET", "/rules"+q+"&chain=input", nil, &rules)
|
||||
if len(rules) != 1 || rules[0]["pending"] != "new" || rules[0]["comment"] != "SSH from home" {
|
||||
t.Errorf("rules in the changeset: %+v", rules)
|
||||
}
|
||||
var plain []map[string]any
|
||||
e.call(e.token, "GET", "/zones", nil, &plain)
|
||||
if len(plain) != 1 || plain[0]["pending"] != nil {
|
||||
t.Errorf("zones without the changeset: %+v", plain)
|
||||
}
|
||||
// The preview: valid, with the ruleset's diff.
|
||||
var pv api.ChangesetPreview
|
||||
e.call(e.token, "GET", "/changesets/"+itoa(cs.ID)+"/preview", nil, &pv)
|
||||
if !pv.Valid || !strings.Contains(pv.Diff, "+\t\tip saddr @trusted_v4 tcp dport 22 counter accept") || !strings.Contains(pv.Diff, "--- ruleset.nft (now)") {
|
||||
t.Errorf("preview: %+v", pv)
|
||||
}
|
||||
// Another user can't stage into it, apply or discard it.
|
||||
ctx := context.Background()
|
||||
other := &model.User{Username: "sam", Grants: auth.Roles["operator"]}
|
||||
e.srv.store.CreateUser(ctx, other)
|
||||
otok, pid, sh := auth.NewToken()
|
||||
e.srv.store.CreateToken(ctx, &model.Token{UserID: other.ID, Name: "t", PublicID: pid, SecretHash: sh})
|
||||
if code := e.call(otok, "POST", "/changesets/"+itoa(cs.ID)+"/apply", nil, &eb); code != 403 || !strings.Contains(eb.Error.Message, "admin's") {
|
||||
t.Errorf("another user's apply: %d %+v", code, eb)
|
||||
}
|
||||
// Removing an operation; then the changeset is applied as one change.
|
||||
var after model.Changeset
|
||||
if code := e.call(e.token, "DELETE", "/changesets/"+itoa(cs.ID)+"/ops/4", nil, &after); code != 200 || len(after.Ops) != 3 {
|
||||
t.Fatalf("remove op: %d %+v", code, after)
|
||||
}
|
||||
if code := e.call(e.token, "POST", "/changesets/"+itoa(cs.ID)+"/apply", nil, &mr); code != 200 || !mr.Applied || mr.ConfirmBy == nil || mr.Changeset.Status != "applied" {
|
||||
t.Fatalf("apply: %d %+v", code, mr)
|
||||
}
|
||||
if len(e.fw.applied) != applied+1 || len(e.eng.State().Rules) != 1 || len(e.eng.State().Zones) != 1 {
|
||||
t.Errorf("applied: %d rulesets, state %+v", len(e.fw.applied)-applied, e.eng.State())
|
||||
}
|
||||
h, _ := e.srv.store.GetHistory(ctx, mr.HistoryID)
|
||||
if h.Description != "trusted ssh" || len(h.Changes) != 2 {
|
||||
t.Errorf("history: %+v", h)
|
||||
}
|
||||
// An applied changeset takes no more.
|
||||
if code := e.call(e.token, "POST", "/sets"+q, model.AddressSet{Name: "x", Entries: []string{"10.0.0.1"}}, &eb); code != 409 || !strings.Contains(eb.Error.Message, "is applied") {
|
||||
t.Errorf("staging into an applied changeset: %d %+v", code, eb)
|
||||
}
|
||||
// A changeset that no longer applies names the operation.
|
||||
var cs2 model.Changeset
|
||||
e.call(e.token, "POST", "/changesets", nil, &cs2)
|
||||
e.call(e.token, "PATCH", "/rules/"+itoa(created.ID)+"?changeset="+itoa(cs2.ID), map[string]any{"comment": "x"}, nil)
|
||||
e.call(e.token, "DELETE", "/rules/"+itoa(created.ID), nil, nil)
|
||||
e.call(e.token, "GET", "/changesets/"+itoa(cs2.ID)+"/preview", nil, &pv)
|
||||
if pv.Valid || pv.Error == nil || pv.Error.Op != 1 || !strings.Contains(pv.Error.Message, "operation 1") {
|
||||
t.Errorf("stale preview: %+v", pv)
|
||||
}
|
||||
if code := e.call(e.token, "POST", "/changesets/"+itoa(cs2.ID)+"/discard", nil, &after); code != 200 || after.Status != "discarded" {
|
||||
t.Errorf("discard: %d %+v", code, after)
|
||||
}
|
||||
// A dry run shows the ruleset's diff.
|
||||
if code := e.call(e.token, "POST", "/sets?dry_run=1", model.AddressSet{Name: "y", Entries: []string{"10.0.0.2"}}, &mr); code != 200 || !strings.Contains(mr.Diff, "+\tset y_v4") {
|
||||
t.Errorf("dry run diff: %d %q", code, mr.Diff)
|
||||
}
|
||||
}
|
||||
|
||||
func TestQuotaPeriods(t *testing.T) {
|
||||
at := func(s string) time.Time { v, _ := time.ParseInLocation("2006-01-02 15:04", s, time.Local); return v }
|
||||
for _, c := range []struct {
|
||||
now, reset string
|
||||
day int
|
||||
start string
|
||||
next string
|
||||
}{
|
||||
{"2026-10-19 10:00", "daily", 0, "2026-10-19 00:00", "2026-10-20 00:00"},
|
||||
{"2026-10-19 10:00", "weekly", 0, "2026-10-19 00:00", "2026-10-26 00:00"}, // a Monday
|
||||
{"2026-10-25 23:59", "weekly", 0, "2026-10-19 00:00", "2026-10-26 00:00"}, // a Sunday
|
||||
{"2026-10-19 10:00", "monthly", 15, "2026-10-15 00:00", "2026-11-15 00:00"},
|
||||
{"2026-10-10 10:00", "monthly", 15, "2026-09-15 00:00", "2026-10-15 00:00"},
|
||||
{"2026-02-28 10:00", "monthly", 31, "2026-02-28 00:00", "2026-03-31 00:00"}, // February has no 31st
|
||||
{"2026-03-15 10:00", "monthly", 31, "2026-02-28 00:00", "2026-03-31 00:00"},
|
||||
{"2026-01-01 00:00", "monthly", 0, "2026-01-01 00:00", "2026-02-01 00:00"},
|
||||
} {
|
||||
now := at(c.now)
|
||||
if got := periodStart(now, c.reset, c.day); !got.Equal(at(c.start)) {
|
||||
t.Errorf("%s %s %d: period from %s, want %s", c.now, c.reset, c.day, got.Format("2006-01-02 15:04"), c.start)
|
||||
}
|
||||
if got := nextReset(now, c.reset, c.day); !got.Equal(at(c.next)) {
|
||||
t.Errorf("%s %s %d: next reset %s, want %s", c.now, c.reset, c.day, got.Format("2006-01-02 15:04"), c.next)
|
||||
}
|
||||
}
|
||||
if !periodStart(time.Now(), "", 0).IsZero() {
|
||||
t.Error("a quota reset by hand has no period")
|
||||
}
|
||||
}
|
||||
|
||||
// TestQuotaResets: a quota starts its period when first seen, is reset
|
||||
// when the next one starts, and by hand.
|
||||
func TestQuotaResets(t *testing.T) {
|
||||
e := newEnv(t)
|
||||
ctx := context.Background()
|
||||
now := time.Date(2026, 10, 19, 23, 50, 0, 0, time.Local)
|
||||
e.srv.nowFn = func() time.Time { return now }
|
||||
var mr api.MutationResponse
|
||||
rule := model.Rule{Chain: "forward", Action: "drop", Quota: &model.Quota{Bytes: 5 << 30, Over: true, PerSource: true, Reset: "daily"}}
|
||||
if code := e.call(e.token, "POST", "/rules", rule, &mr); code != 201 {
|
||||
t.Fatalf("rule: %d %+v", code, mr)
|
||||
}
|
||||
var created model.Rule
|
||||
json.Unmarshal(mr.Object, &created)
|
||||
e.srv.checkQuotas(ctx)
|
||||
if len(e.fw.resets) != 0 {
|
||||
t.Fatalf("reset on first sight: %v", e.fw.resets)
|
||||
}
|
||||
now = now.Add(5 * time.Minute) // the same day
|
||||
e.srv.checkQuotas(ctx)
|
||||
now = now.Add(10 * time.Minute) // past midnight
|
||||
e.srv.checkQuotas(ctx)
|
||||
e.srv.checkQuotas(ctx)
|
||||
want := fmt.Sprintf("r%d:true", created.ID)
|
||||
if strings.Join(e.fw.resets, ",") != want {
|
||||
t.Errorf("resets %v, want one %s", e.fw.resets, want)
|
||||
}
|
||||
var qs []api.QuotaInfo
|
||||
e.call(e.token, "GET", "/quotas", nil, &qs)
|
||||
if len(qs) != 1 || qs[0].NextReset == nil || qs[0].NextReset.Day() != 21 {
|
||||
t.Errorf("quotas %+v", qs)
|
||||
}
|
||||
if code := e.call(e.token, "POST", "/rules/"+itoa(created.ID)+"/reset-quota", nil, nil); code != 200 || len(e.fw.resets) != 2 {
|
||||
t.Errorf("reset by hand: %d %v", code, e.fw.resets)
|
||||
}
|
||||
}
|
||||
|
||||
// TestAutoConfirm: the client that made a change confirms it by itself:
|
||||
// with its token, from the same address, over a new connection. Nobody
|
||||
// else can, an older token can't, and with manual_confirm there is none.
|
||||
func TestAutoConfirm(t *testing.T) {
|
||||
e := newEnv(t)
|
||||
e.srv.probationFor = 5 * time.Second
|
||||
post := func(token, path string, body any) (*http.Response, []byte) {
|
||||
b, _ := json.Marshal(body)
|
||||
req, _ := http.NewRequest("POST", e.ts.URL+api.Prefix+path, bytes.NewReader(b))
|
||||
req.Header.Set("Authorization", "Bearer "+token)
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
out, _ := io.ReadAll(resp.Body)
|
||||
return resp, out
|
||||
}
|
||||
resp, body := post(e.token, "/zones", model.Zone{Name: "lan", Interfaces: []string{"eth1"}})
|
||||
var mr api.MutationResponse
|
||||
json.Unmarshal(body, &mr)
|
||||
if mr.ConfirmToken == "" || !resp.Close {
|
||||
t.Fatalf("no token, or the connection stays open: %+v close=%v", mr, resp.Close)
|
||||
}
|
||||
first := mr.ConfirmToken
|
||||
// A newer change: only its token confirms now.
|
||||
_, body = post(e.token, "/zones", model.Zone{Name: "dmz", Interfaces: []string{"eth2"}})
|
||||
json.Unmarshal(body, &mr)
|
||||
if resp, _ := post(e.token, "/confirm", api.ConfirmInput{Token: first}); resp.StatusCode != 409 {
|
||||
t.Errorf("an older token: %d", resp.StatusCode)
|
||||
}
|
||||
// Another user can't use it.
|
||||
ctx := context.Background()
|
||||
other := &model.User{Username: "sam", Grants: auth.Roles["operator"]}
|
||||
e.srv.store.CreateUser(ctx, other)
|
||||
otok, pid, sh := auth.NewToken()
|
||||
e.srv.store.CreateToken(ctx, &model.Token{UserID: other.ID, Name: "t", PublicID: pid, SecretHash: sh})
|
||||
if resp, _ := post(otok, "/confirm", api.ConfirmInput{Token: mr.ConfirmToken}); resp.StatusCode != 403 {
|
||||
t.Errorf("another user's confirmation: %d", resp.StatusCode)
|
||||
}
|
||||
// Not over the change's own connection, nor from elsewhere.
|
||||
c := e.srv.probation.check
|
||||
p := &auth.Principal{User: &model.User{ID: c.userID, Username: c.username}, Method: "token", Token: &model.Token{ID: c.tokenID}}
|
||||
same := httptest.NewRequest("POST", "/", nil)
|
||||
same.RemoteAddr = c.remote
|
||||
if _, err := e.srv.confirmToken(same, p, c.token); err == nil || !strings.Contains(err.Error(), "new one") {
|
||||
t.Errorf("over the same connection: %v", err)
|
||||
}
|
||||
elsewhere := httptest.NewRequest("POST", "/", nil)
|
||||
elsewhere.RemoteAddr = "192.0.2.9:4444"
|
||||
if _, err := e.srv.confirmToken(elsewhere, p, c.token); err == nil || !strings.Contains(err.Error(), "same place") {
|
||||
t.Errorf("from elsewhere: %v", err)
|
||||
}
|
||||
// The client itself, over a new connection: confirmed.
|
||||
resp, body = post(e.token, "/confirm", api.ConfirmInput{Token: mr.ConfirmToken})
|
||||
if resp.StatusCode != 200 || !strings.Contains(string(body), `"confirmed":2`) {
|
||||
t.Fatalf("auto-confirm: %d %s", resp.StatusCode, body)
|
||||
}
|
||||
if pr := e.srv.probationStatus(); len(pr.Pending) != 0 {
|
||||
t.Errorf("still pending: %+v", pr)
|
||||
}
|
||||
// manual_confirm: no token; a person confirms.
|
||||
e.call(e.token, "PATCH", "/settings", map[string]any{"manual_confirm": true}, nil)
|
||||
e.call(e.token, "POST", "/confirm", nil, nil)
|
||||
_, body = post(e.token, "/zones", model.Zone{Name: "guest", Interfaces: []string{"eth3"}})
|
||||
mr = api.MutationResponse{}
|
||||
json.Unmarshal(body, &mr)
|
||||
if mr.ConfirmToken != "" || mr.ConfirmBy == nil {
|
||||
t.Errorf("manual confirmation: %+v", mr)
|
||||
}
|
||||
}
|
||||
|
||||
// TestStagedUndo: an undo staged in a changeset changes nothing until the
|
||||
// changeset is applied; then History links the two entries.
|
||||
func TestStagedUndo(t *testing.T) {
|
||||
e := newEnv(t)
|
||||
e.srv.probationFor = 5 * time.Second
|
||||
var mr api.MutationResponse
|
||||
e.call(e.token, "POST", "/zones", model.Zone{Name: "lan", Interfaces: []string{"eth1"}}, &mr)
|
||||
e.call(e.token, "POST", "/confirm", nil, nil)
|
||||
made := mr.HistoryID
|
||||
var cs model.Changeset
|
||||
e.call(e.token, "POST", "/changesets", nil, &cs)
|
||||
q := "?changeset=" + itoa(cs.ID)
|
||||
if code := e.call(e.token, "POST", "/history/"+itoa(made)+"/revert"+q, nil, &mr); code != 200 || !mr.Staged {
|
||||
t.Fatalf("staging the undo: %d %+v", code, mr)
|
||||
}
|
||||
if op := mr.Changeset.Ops[0]; op.Op != model.OpRevert || !strings.HasPrefix(op.Summary, "undo "+itoa(made)+":") {
|
||||
t.Errorf("the operation: %+v", op)
|
||||
}
|
||||
if len(e.eng.State().Zones) != 1 {
|
||||
t.Fatal("a staged undo was applied")
|
||||
}
|
||||
// Staging it again can't work: the changeset already undoes it.
|
||||
var eb api.ErrorBody
|
||||
if code := e.call(e.token, "POST", "/history/"+itoa(made)+"/revert"+q, nil, &eb); code != 409 {
|
||||
t.Errorf("a second undo: %d %+v", code, eb)
|
||||
}
|
||||
if code := e.call(e.token, "POST", "/changesets/"+itoa(cs.ID)+"/apply", nil, &mr); code != 200 || !mr.Applied {
|
||||
t.Fatalf("apply: %d %+v", code, mr)
|
||||
}
|
||||
if len(e.eng.State().Zones) != 0 {
|
||||
t.Error("the undo wasn't applied")
|
||||
}
|
||||
h, _ := e.srv.store.GetHistory(context.Background(), mr.HistoryID)
|
||||
orig, _ := e.srv.store.GetHistory(context.Background(), made)
|
||||
if h.RevertOf != made || orig.RevertedBy != h.ID || !strings.HasPrefix(h.Description, "undo ") {
|
||||
t.Errorf("history: %+v, original %+v", h, orig)
|
||||
}
|
||||
}
|
||||
|
||||
// TestMatrix: the zone matrix, now and as a changeset will make it.
|
||||
func TestMatrixEndpoint(t *testing.T) {
|
||||
e := newEnv(t)
|
||||
e.call(e.token, "POST", "/zones", model.Zone{Name: "lan", Interfaces: []string{"eth1"}}, nil)
|
||||
e.call(e.token, "POST", "/zones", model.Zone{Name: "wan", Interfaces: []string{"wan"}, External: true}, nil)
|
||||
var m api.Matrix
|
||||
e.call(e.token, "GET", "/matrix", nil, &m)
|
||||
cell := func(from, to string) api.MatrixCell {
|
||||
for _, c := range m.Cells {
|
||||
if c.From == from && c.To == to {
|
||||
return c
|
||||
}
|
||||
}
|
||||
return api.MatrixCell{}
|
||||
}
|
||||
if c := cell("lan", "wan"); c.Verdict != "blocked" || c.DecidedBy != "forward policy" {
|
||||
t.Errorf("lan → wan: %+v", c)
|
||||
}
|
||||
var cs model.Changeset
|
||||
e.call(e.token, "POST", "/changesets", nil, &cs)
|
||||
q := "?changeset=" + itoa(cs.ID)
|
||||
e.call(e.token, "POST", "/rules"+q, model.Rule{Chain: "forward", Match: model.Match{InZones: []string{"lan"}, OutZones: []string{"wan"}}, Action: "accept"}, nil)
|
||||
e.call(e.token, "GET", "/matrix"+q, nil, &m)
|
||||
if c := cell("lan", "wan"); c.Verdict != "allowed" {
|
||||
t.Errorf("lan → wan with the changeset: %+v", c)
|
||||
}
|
||||
}
|
||||
|
||||
// TestHostProblems: an enabled nftables.service, and an imported table
|
||||
// still in force, are warned about.
|
||||
func TestHostProblems(t *testing.T) {
|
||||
e := newEnv(t)
|
||||
dir := t.TempDir()
|
||||
link := dir + "/nftables.service"
|
||||
e.srv.unitLinks = []string{link}
|
||||
codes := func() string {
|
||||
var st api.Status
|
||||
e.call(e.token, "GET", "/status", nil, &st)
|
||||
var out []string
|
||||
for _, p := range st.Problems {
|
||||
out = append(out, p.Code)
|
||||
}
|
||||
return strings.Join(out, " ")
|
||||
}
|
||||
if c := codes(); strings.Contains(c, "nftables_service") || strings.Contains(c, "takeover_pending") {
|
||||
t.Errorf("warned without cause: %s", c)
|
||||
}
|
||||
os.Symlink("/usr/lib/systemd/system/nftables.service", link) // dangling is fine: Lstat
|
||||
// The fake ruleset lists inet main: imported from it, it still applies.
|
||||
e.srv.store.SetMeta(context.Background(), importedTableKey, "inet main")
|
||||
if c := codes(); !strings.Contains(c, "nftables_service") || !strings.Contains(c, "takeover_pending") {
|
||||
t.Errorf("not warned: %s", c)
|
||||
}
|
||||
e.srv.store.SetMeta(context.Background(), importedTableKey, "inet gone")
|
||||
if c := codes(); strings.Contains(c, "takeover_pending") {
|
||||
t.Errorf("warned about a table that is gone: %s", c)
|
||||
}
|
||||
}
|
||||
|
||||
type fakeConns struct {
|
||||
list []api.Connection
|
||||
closed []api.Connection
|
||||
}
|
||||
|
||||
func (f *fakeConns) List(context.Context) ([]api.Connection, error) { return f.list, nil }
|
||||
func (f *fakeConns) Close(_ context.Context, c api.Connection) error {
|
||||
f.closed = append(f.closed, c)
|
||||
return nil
|
||||
}
|
||||
|
||||
// TestConnections: connections listed and searched; closed one by one, or
|
||||
// all a search finds; closing needs config:write, and is audited.
|
||||
func TestConnections(t *testing.T) {
|
||||
e := newEnv(t)
|
||||
var code int
|
||||
if code = e.call(e.token, "GET", "/connections", nil, nil); code != 503 {
|
||||
t.Errorf("without conntrack: %d", code)
|
||||
}
|
||||
fc := &fakeConns{list: []api.Connection{
|
||||
{Family: "ipv4", Protocol: "tcp", Source: "192.168.1.50", SourcePort: 51432, Destination: "198.51.100.9", DestinationPort: 443, SNATTo: "203.0.113.10", State: "established"},
|
||||
{Family: "ipv4", Protocol: "udp", Source: "192.168.1.51", SourcePort: 5353, Destination: "192.168.1.1", DestinationPort: 53},
|
||||
{Family: "ipv6", Protocol: "tcp", Source: "2001:db8:1::50", SourcePort: 40000, Destination: "2001:db8:2::9", DestinationPort: 4430},
|
||||
}}
|
||||
e.srv.SetConntrack(fc)
|
||||
var cs []api.Connection
|
||||
e.call(e.token, "GET", "/connections?q=192.168.1.50", nil, &cs)
|
||||
if len(cs) != 1 || cs[0].DestinationPort != 443 {
|
||||
t.Errorf("search by address: %+v", cs)
|
||||
}
|
||||
e.call(e.token, "GET", "/connections?q=:443", nil, &cs)
|
||||
if len(cs) != 1 || cs[0].Source != "192.168.1.50" {
|
||||
t.Errorf("search by port (not :4430): %+v", cs)
|
||||
}
|
||||
var out map[string]int
|
||||
if code := e.call(e.token, "POST", "/connections/close", api.CloseConnections{Search: "tcp"}, &out); code != 200 || out["closed"] != 2 || len(fc.closed) != 2 {
|
||||
t.Errorf("close a search: %d %+v %+v", code, out, fc.closed)
|
||||
}
|
||||
if code := e.call(e.token, "POST", "/connections/close", api.CloseConnections{}, nil); code != 400 {
|
||||
t.Errorf("close nothing said: %d", code)
|
||||
}
|
||||
var tok api.Token
|
||||
e.call(e.token, "POST", "/me/tokens", api.TokenInput{Name: "monitor", Role: "log"}, &tok)
|
||||
if code := e.call(tok.Secret, "GET", "/connections", nil, nil); code != 200 {
|
||||
t.Errorf("listing with log:read: %d", code)
|
||||
}
|
||||
if code := e.call(tok.Secret, "POST", "/connections/close", api.CloseConnections{Search: "udp"}, nil); code != 403 {
|
||||
t.Errorf("closing with log:read: %d", code)
|
||||
}
|
||||
}
|
||||
|
||||
// TestTrace: a trace runs with the filter's table, collects packets named
|
||||
// after Housecarl's objects, and stops when asked; one at a time.
|
||||
func TestTrace(t *testing.T) {
|
||||
e := newEnv(t)
|
||||
var mr api.MutationResponse
|
||||
e.call(e.token, "POST", "/rules", model.Rule{Chain: "input", Match: model.Match{Services: []string{"tcp/9999"}}, Action: "drop", Comment: "block 9999"}, &mr)
|
||||
var rule model.Rule
|
||||
json.Unmarshal(mr.Object, &rule)
|
||||
e.fw.traceLines = []string{
|
||||
`trace id 1 inet housecarl input packet: iif "wan" ip saddr 198.51.100.7 ip daddr 203.0.113.10 tcp sport 4000 tcp dport 9999 tcp flags == syn`,
|
||||
`trace id 1 inet housecarl input rule ct state established,related accept comment "hc:b-established" (verdict continue)`,
|
||||
fmt.Sprintf(`trace id 1 inet housecarl input rule tcp dport 9999 counter drop comment "hc:r%d block 9999" (verdict drop)`, rule.ID),
|
||||
}
|
||||
var st api.TraceStatus
|
||||
if code := e.call(e.token, "POST", "/trace", api.TraceInput{Port: 9999, Seconds: 30}, &st); code != 200 || !st.Running {
|
||||
t.Fatalf("start: %d %+v", code, st)
|
||||
}
|
||||
if code := e.call(e.token, "POST", "/trace", api.TraceInput{Port: 22}, nil); code != 409 {
|
||||
t.Errorf("a second trace: %d", code)
|
||||
}
|
||||
time.Sleep(100 * time.Millisecond)
|
||||
e.call(e.token, "GET", "/trace", nil, &st)
|
||||
if len(st.Packets) != 1 || st.Packets[0].Verdict != "dropped" || st.Packets[0].DecidedBy.Label != "rule input #"+itoa(rule.ID)+" (block 9999)" ||
|
||||
st.Packets[0].Steps[0].Label != "built-in: established" {
|
||||
t.Errorf("packets: %+v", st.Packets)
|
||||
}
|
||||
if !strings.Contains(e.fw.traced, "th dport 9999") {
|
||||
t.Errorf("the trace table: %s", e.fw.traced)
|
||||
}
|
||||
if code := e.call(e.token, "DELETE", "/trace", nil, &st); code != 200 || st.Running || len(st.Packets) != 1 {
|
||||
t.Errorf("stop: %d %+v", code, st)
|
||||
}
|
||||
if code := e.call(e.token, "POST", "/trace", api.TraceInput{}, nil); code != 400 {
|
||||
t.Errorf("tracing everything: %d", code)
|
||||
}
|
||||
}
|
||||
|
||||
// TestStatsEndpoints: statistics off, then on: the summary's top lists,
|
||||
// hits per rule (named, with those that counted none), and which
|
||||
// counters count drops.
|
||||
func TestStatsEndpoints(t *testing.T) {
|
||||
e := newEnv(t)
|
||||
if code := e.call(e.token, "GET", "/stats/summary", nil, nil); code != http.StatusServiceUnavailable {
|
||||
t.Errorf("statistics off: %d", code)
|
||||
}
|
||||
var st StatsStatus
|
||||
e.call(e.token, "GET", "/stats/status", nil, &st)
|
||||
if st.Enabled {
|
||||
t.Error("enabled without a collector")
|
||||
}
|
||||
e.call(e.token, "POST", "/zones", model.Zone{Name: "wan", Interfaces: []string{"wan"}, External: true}, nil)
|
||||
e.call(e.token, "POST", "/rules", model.Rule{Chain: "input", Match: model.Match{Sources: []string{"198.51.100.0/24"}}, Action: "drop", Comment: "bad net"}, nil)
|
||||
rule := e.eng.State().Rules[0]
|
||||
|
||||
ss, err := stats.OpenStore(":memory:")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer ss.Close()
|
||||
src := stats.Sources{Table: e.fw.Table, SysNet: t.TempDir(), Proc: t.TempDir()}
|
||||
e.srv.StatsSources(&src)
|
||||
c := stats.NewCollector(ss, stats.Options{RetainMinute: 7 * 24 * time.Hour, RetainHour: 90 * 24 * time.Hour, RetainDay: 730 * 24 * time.Hour}, src, nil)
|
||||
e.srv.SetStats(c)
|
||||
c.Start(context.Background())
|
||||
e.srv.LogPacket(api.LogEntry{Time: time.Now(), Source: "198.51.100.7", Protocol: "tcp", DestPort: 22, Verdict: "drop"})
|
||||
c.Sample(context.Background())
|
||||
|
||||
var sum stats.Summary
|
||||
if code := e.call(e.token, "GET", "/stats/summary?since=1h", nil, &sum); code != http.StatusOK {
|
||||
t.Fatalf("summary: %d", code)
|
||||
}
|
||||
if len(sum.BlockedSources) != 1 || sum.BlockedSources[0].Key != "198.51.100.7" || len(sum.BlockedPorts) != 1 || sum.BlockedPorts[0].Key != "tcp/22" {
|
||||
t.Errorf("summary: %+v", sum)
|
||||
}
|
||||
var hits []stats.Entry
|
||||
e.call(e.token, "GET", "/stats/hits", nil, &hits)
|
||||
if len(hits) != 1 || hits[0].Key != "r"+itoa(rule.ID) || hits[0].Count != 0 || !strings.Contains(hits[0].Name, "bad net") {
|
||||
t.Errorf("hits: %+v", hits)
|
||||
}
|
||||
var top []stats.Entry
|
||||
if code := e.call(e.token, "GET", "/stats/top?kind=blocked_port&since=7d", nil, &top); code != http.StatusOK || len(top) != 1 {
|
||||
t.Errorf("top: %d %+v", code, top)
|
||||
}
|
||||
if code := e.call(e.token, "GET", "/stats/top?kind=nonsense", nil, nil); code != http.StatusBadRequest {
|
||||
t.Errorf("an unknown list: %d", code)
|
||||
}
|
||||
if code := e.call(e.token, "GET", "/stats/timeline?keys=zrx:*,drops&since=2h", nil, nil); code != http.StatusOK {
|
||||
t.Errorf("timeline: %d", code)
|
||||
}
|
||||
e.call(e.token, "GET", "/stats/status", nil, &st)
|
||||
if !st.Enabled || st.LastSample == nil {
|
||||
t.Errorf("status: %+v", st)
|
||||
}
|
||||
for mark, want := range map[string]string{"r" + itoa(rule.ID): "input true", "p-input": "input true", "p-output": "output false", "b-antispoof6": "early true", "b-established": " false"} {
|
||||
chain, drop := e.srv.classifyMark(mark)
|
||||
if got := fmt.Sprint(chain, " ", drop); got != want {
|
||||
t.Errorf("%s: %q, want %q", mark, got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,546 @@
|
||||
package stats
|
||||
|
||||
import (
|
||||
"cmp"
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"slices"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"housecarl/internal/api"
|
||||
"housecarl/internal/nft"
|
||||
)
|
||||
|
||||
// Resolutions and retention, and the sources the figures come from.
|
||||
type Options struct {
|
||||
TopN int
|
||||
RetainMinute time.Duration
|
||||
RetainHour time.Duration
|
||||
RetainDay time.Duration
|
||||
Location *time.Location // for days (default: local time)
|
||||
// Interval is how often the collector samples (default: a minute).
|
||||
Interval time.Duration
|
||||
}
|
||||
|
||||
// Sources are what the collector reads. Nil functions are left out.
|
||||
type Sources struct {
|
||||
// Table reads Housecarl's table: the counters by mark (totals across
|
||||
// reloads), the quotas, the per-device sets.
|
||||
Table func(context.Context) (*nft.Table, error)
|
||||
// Zones maps the interfaces to their zones.
|
||||
Zones func() map[string]string
|
||||
// Classify says which chain a mark's counter is in and whether what
|
||||
// it counts is dropped ("r12" → forward, true).
|
||||
Classify func(mark string) (chain string, drop bool)
|
||||
// SysNet and Proc are /sys/class/net and /proc (tests point them
|
||||
// elsewhere).
|
||||
SysNet, Proc string
|
||||
}
|
||||
|
||||
// Counter keys. Traffic: rx:IF, tx:IF (bytes), rxp:IF, txp:IF (packets),
|
||||
// zrx:ZONE, ztx:ZONE (bytes received from and sent to a zone's
|
||||
// interfaces). Hits: hits:MARK (packets), hitb:MARK (bytes). Drops:
|
||||
// drop:CHAIN and drops (packets). Quotas: quota:r12 (bytes). Gauges
|
||||
// (g:KEY, see gauge) are below.
|
||||
const (
|
||||
GaugeConntrack = "conntrack"
|
||||
GaugeConntrackMax = "conntrack_max"
|
||||
GaugeRSS = "rss" // housecarld's resident memory, bytes
|
||||
GaugeCPU = "cpu" // housecarld's CPU, tenths of a percent of one CPU
|
||||
GaugeMem = "mem" // the host's memory in use, bytes
|
||||
GaugeLoad = "load" // the host's 1-minute load average, times 100
|
||||
)
|
||||
|
||||
// Top-list kinds: per hour and per day, the TopN largest.
|
||||
const (
|
||||
KindBlockedSource = "blocked_src" // logged drops by source address
|
||||
KindBlockedPort = "blocked_port" // logged drops by what they tried: tcp/22, icmp
|
||||
KindDeviceUp = "device_up" // bytes sent to the internet, by inside address
|
||||
KindDeviceDown = "device_down" // bytes received from it
|
||||
// A per-address quota's use by address is "quota_r12".
|
||||
)
|
||||
|
||||
// Kinds are the top lists Top accepts (and quota_rN).
|
||||
var Kinds = []string{KindBlockedSource, KindBlockedPort, KindDeviceUp, KindDeviceDown}
|
||||
|
||||
// Status describes the collector's health.
|
||||
type Status struct {
|
||||
Enabled bool `json:"enabled"`
|
||||
Database string `json:"database,omitempty"`
|
||||
Size int64 `json:"size"`
|
||||
LastSample *time.Time `json:"last_sample,omitempty"`
|
||||
SampleError string `json:"sample_error,omitempty"`
|
||||
FirstData *time.Time `json:"first_data,omitempty"`
|
||||
// PerDevice: the firewall counts traffic per device (a setting).
|
||||
PerDevice bool `json:"per_device"`
|
||||
Retention struct {
|
||||
Minute string `json:"minute"`
|
||||
Hour string `json:"hour"`
|
||||
Day string `json:"day"`
|
||||
} `json:"retention"`
|
||||
}
|
||||
|
||||
// Latest are the newest gauge values.
|
||||
type Latest struct {
|
||||
At time.Time `json:"at"`
|
||||
Conntrack int64 `json:"conntrack"`
|
||||
ConntrackMax int64 `json:"conntrack_max"`
|
||||
RSS int64 `json:"rss"`
|
||||
CPUPermille int64 `json:"cpu_permille"`
|
||||
MemUsed int64 `json:"mem_used"`
|
||||
Load float64 `json:"load"`
|
||||
}
|
||||
|
||||
// Collector gathers statistics into a Store.
|
||||
type Collector struct {
|
||||
opts Options
|
||||
src Sources
|
||||
store *Store
|
||||
log *slog.Logger
|
||||
now func() time.Time
|
||||
|
||||
// Each source's last totals: the next sample stores the difference.
|
||||
ifs, table deltas
|
||||
cpuTicks int64
|
||||
cpuAt time.Time
|
||||
lastPrune time.Time
|
||||
|
||||
mu sync.Mutex
|
||||
hours map[int64]map[string]map[string]int64 // live hours: hour -> kind -> key -> count
|
||||
status Status
|
||||
latest *Latest
|
||||
}
|
||||
|
||||
// maxKeys bounds each kind of an hour's top list in memory.
|
||||
const maxKeys = 100_000
|
||||
|
||||
// NewCollector creates a collector. Run starts it.
|
||||
func NewCollector(store *Store, opts Options, src Sources, log *slog.Logger) *Collector {
|
||||
if opts.Location == nil {
|
||||
opts.Location = time.Local
|
||||
}
|
||||
if log == nil {
|
||||
log = slog.Default()
|
||||
}
|
||||
if opts.TopN <= 0 {
|
||||
opts.TopN = 100
|
||||
}
|
||||
if opts.Interval <= 0 {
|
||||
opts.Interval = time.Minute
|
||||
}
|
||||
if src.SysNet == "" {
|
||||
src.SysNet = "/sys/class/net"
|
||||
}
|
||||
if src.Proc == "" {
|
||||
src.Proc = "/proc"
|
||||
}
|
||||
c := &Collector{opts: opts, src: src, store: store, log: log, now: time.Now, hours: map[int64]map[string]map[string]int64{}}
|
||||
c.status.Enabled = true
|
||||
c.status.Database = store.Path()
|
||||
c.status.Retention.Minute = humanDuration(opts.RetainMinute)
|
||||
c.status.Retention.Hour = humanDuration(opts.RetainHour)
|
||||
c.status.Retention.Day = humanDuration(opts.RetainDay)
|
||||
return c
|
||||
}
|
||||
|
||||
// SetClock replaces the clock (tests).
|
||||
func (c *Collector) SetClock(now func() time.Time) { c.now = now }
|
||||
|
||||
// Store returns the database.
|
||||
func (c *Collector) Store() *Store { return c.store }
|
||||
|
||||
// Options returns the collector's options.
|
||||
func (c *Collector) Options() Options { return c.opts }
|
||||
|
||||
// Status returns the collector's health.
|
||||
func (c *Collector) Status() Status {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
s := c.status
|
||||
s.Size = c.store.Size()
|
||||
return s
|
||||
}
|
||||
|
||||
// Latest returns the newest gauges (nil before the first sample).
|
||||
func (c *Collector) Latest() *Latest {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
if c.latest == nil {
|
||||
return nil
|
||||
}
|
||||
l := *c.latest
|
||||
return &l
|
||||
}
|
||||
|
||||
func (c *Collector) minuteOf(t time.Time) int64 { return t.Unix() - t.Unix()%60 }
|
||||
func (c *Collector) hourOf(t time.Time) int64 { return t.Unix() - t.Unix()%3600 }
|
||||
|
||||
// dayOf returns the start of the local day containing t.
|
||||
func (c *Collector) dayOf(t time.Time) int64 {
|
||||
t = t.In(c.opts.Location)
|
||||
return time.Date(t.Year(), t.Month(), t.Day(), 0, 0, 0, 0, c.opts.Location).Unix()
|
||||
}
|
||||
|
||||
// Run collects until ctx is done.
|
||||
func (c *Collector) Run(ctx context.Context) {
|
||||
c.Start(ctx)
|
||||
for {
|
||||
t := time.NewTimer(time.Until(c.nextSample()))
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
t.Stop()
|
||||
c.write(context.Background(), &Batch{})
|
||||
return
|
||||
case <-t.C:
|
||||
c.Sample(ctx)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// nextSample: two seconds into the next minute, so the minute that just
|
||||
// ended is complete (or after the interval, when it isn't a minute).
|
||||
func (c *Collector) nextSample() time.Time {
|
||||
if c.opts.Interval != time.Minute {
|
||||
return c.now().Add(c.opts.Interval)
|
||||
}
|
||||
return time.Unix(c.minuteOf(c.now())+62, 0)
|
||||
}
|
||||
|
||||
// Start takes the first sample (the baseline the next ones are counted
|
||||
// from), continues the current hour's top lists, and adds hours that
|
||||
// ended while housecarld wasn't running to their days.
|
||||
func (c *Collector) Start(ctx context.Context) {
|
||||
if v, _ := c.store.Meta(ctx, "first_data"); v != "" {
|
||||
if n, err := strconv.ParseInt(v, 10, 64); err == nil {
|
||||
t := time.Unix(n, 0)
|
||||
c.status.FirstData = &t
|
||||
}
|
||||
}
|
||||
cur := c.hourOf(c.now())
|
||||
b := &Batch{}
|
||||
merged, _ := c.store.Meta(ctx, "merged_through")
|
||||
if last, err := strconv.ParseInt(merged, 10, 64); err == nil {
|
||||
for h := max(last+3600, cur-48*3600); h < cur; h += 3600 {
|
||||
kinds, _ := c.store.KindsAt(ctx, ResHour, h)
|
||||
for _, k := range kinds {
|
||||
if es, err := c.store.TopsAt(ctx, ResHour, h, k); err == nil && len(es) > 0 {
|
||||
b.AddTops(ResDay, c.dayOf(time.Unix(h, 0)), k, es, 3*c.opts.TopN)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
b.SetMeta("merged_through", strconv.FormatInt(cur-3600, 10))
|
||||
kinds, _ := c.store.KindsAt(ctx, ResHour, cur)
|
||||
c.mu.Lock()
|
||||
for _, k := range kinds {
|
||||
if es, err := c.store.TopsAt(ctx, ResHour, cur, k); err == nil {
|
||||
m := c.topsMap(cur, k)
|
||||
for _, e := range es {
|
||||
m[e.Key] = e.Count
|
||||
}
|
||||
}
|
||||
}
|
||||
c.mu.Unlock()
|
||||
if err := c.store.Commit(ctx, b); err != nil {
|
||||
c.log.Warn("statistics: catching up failed", "err", err)
|
||||
}
|
||||
c.sample(ctx, -1)
|
||||
c.lastPrune = c.now()
|
||||
c.prune(ctx)
|
||||
c.log.Info("statistics collector started", "database", c.store.Path())
|
||||
}
|
||||
|
||||
// Sample reads every source and writes what changed since the last
|
||||
// sample.
|
||||
func (c *Collector) Sample(ctx context.Context) {
|
||||
now := c.now()
|
||||
// The interval that just ended: with a minute's interval, sampling two
|
||||
// seconds into a minute, the previous minute.
|
||||
minute := c.minuteOf(now.Add(-c.opts.Interval / 2))
|
||||
vals := c.sample(ctx, minute)
|
||||
b := &Batch{}
|
||||
if len(vals) > 0 {
|
||||
t := time.Unix(minute, 0)
|
||||
b.AddCounters(ResMinute, minute, vals)
|
||||
b.AddCounters(ResHour, c.hourOf(t), vals)
|
||||
b.AddCounters(ResDay, c.dayOf(t), vals)
|
||||
c.mu.Lock()
|
||||
if c.status.FirstData == nil {
|
||||
c.status.FirstData = &t
|
||||
b.SetMeta("first_data", strconv.FormatInt(minute, 10))
|
||||
}
|
||||
c.mu.Unlock()
|
||||
}
|
||||
c.write(ctx, b)
|
||||
if now.Sub(c.lastPrune) >= time.Hour {
|
||||
c.lastPrune = now
|
||||
c.prune(ctx)
|
||||
}
|
||||
}
|
||||
|
||||
// write adds the live hours' top lists to b, ends the hours that are
|
||||
// over, and commits.
|
||||
func (c *Collector) write(ctx context.Context, b *Batch) {
|
||||
c.mu.Lock()
|
||||
cur := c.hourOf(c.now())
|
||||
for h, kinds := range c.hours {
|
||||
for k, m := range kinds {
|
||||
b.SetTops(ResHour, h, k, top(m, c.opts.TopN))
|
||||
}
|
||||
if h < cur {
|
||||
// The hour is over: add it to its day.
|
||||
for k, m := range kinds {
|
||||
b.AddTops(ResDay, c.dayOf(time.Unix(h, 0)), k, top(m, c.opts.TopN), 3*c.opts.TopN)
|
||||
}
|
||||
delete(c.hours, h)
|
||||
b.SetMeta("merged_through", strconv.FormatInt(h, 10))
|
||||
}
|
||||
}
|
||||
c.mu.Unlock()
|
||||
if err := c.store.Commit(ctx, b); err != nil {
|
||||
c.log.Error("statistics: writing failed", "err", err)
|
||||
}
|
||||
}
|
||||
|
||||
// deltas turn a source's totals into what changed since its last read.
|
||||
type deltas struct {
|
||||
prev, next map[string]int64
|
||||
based bool
|
||||
}
|
||||
|
||||
// of returns how much key grew. The first read is the baseline (nothing
|
||||
// counts); after it, a new key counts whole (a new interface, rule or
|
||||
// device starts from zero), and a total that went down started again (an
|
||||
// interface came back, a quota was reset, a device's entry expired).
|
||||
func (d *deltas) of(key string, cur int64) int64 {
|
||||
if d.next == nil {
|
||||
d.next = map[string]int64{}
|
||||
}
|
||||
d.next[key] = cur
|
||||
if !d.based {
|
||||
return 0
|
||||
}
|
||||
p, ok := d.prev[key]
|
||||
if !ok || cur < p {
|
||||
return cur
|
||||
}
|
||||
return cur - p
|
||||
}
|
||||
|
||||
// done keeps this read's totals for the next (a failed read skips it, so
|
||||
// its keys aren't new next time).
|
||||
func (d *deltas) done() { d.prev, d.next, d.based = d.next, nil, true }
|
||||
|
||||
// sample reads the sources; minute is the bucket the changes go to (-1:
|
||||
// only the baseline). It returns the counters.
|
||||
func (c *Collector) sample(ctx context.Context, minute int64) map[string]int64 {
|
||||
now := c.now()
|
||||
vals := map[string]int64{}
|
||||
add := func(k string, v int64) {
|
||||
if v != 0 {
|
||||
vals[k] += v
|
||||
}
|
||||
}
|
||||
gauge := func(k string, v int64) {
|
||||
vals["g:"+k] += v
|
||||
vals["gn:"+k]++
|
||||
}
|
||||
tops := map[string]map[string]int64{}
|
||||
addTop := func(kind, key string, v int64) {
|
||||
if v <= 0 {
|
||||
return
|
||||
}
|
||||
if tops[kind] == nil {
|
||||
tops[kind] = map[string]int64{}
|
||||
}
|
||||
tops[kind][key] += v
|
||||
}
|
||||
var errs []string
|
||||
|
||||
// Interfaces, and their zones.
|
||||
var zones map[string]string
|
||||
if c.src.Zones != nil {
|
||||
zones = c.src.Zones()
|
||||
}
|
||||
for name, ic := range readInterfaces(c.src.SysNet) {
|
||||
rx, tx := c.ifs.of("rx:"+name, ic.RxBytes), c.ifs.of("tx:"+name, ic.TxBytes)
|
||||
add("rx:"+name, rx)
|
||||
add("tx:"+name, tx)
|
||||
add("rxp:"+name, c.ifs.of("rxp:"+name, ic.RxPackets))
|
||||
add("txp:"+name, c.ifs.of("txp:"+name, ic.TxPackets))
|
||||
if z := zones[name]; z != "" {
|
||||
add("zrx:"+z, rx)
|
||||
add("ztx:"+z, tx)
|
||||
}
|
||||
}
|
||||
c.ifs.done()
|
||||
|
||||
// Housecarl's table: hits, drops, quotas, devices.
|
||||
if c.src.Table != nil {
|
||||
t, err := c.src.Table(ctx)
|
||||
switch {
|
||||
case err == nil:
|
||||
for mark, ctr := range t.Counters {
|
||||
p := c.table.of("hits:"+mark, int64(ctr.Packets))
|
||||
add("hits:"+mark, p)
|
||||
add("hitb:"+mark, c.table.of("hitb:"+mark, int64(ctr.Bytes)))
|
||||
if c.src.Classify != nil && p > 0 {
|
||||
if chain, drop := c.src.Classify(mark); drop {
|
||||
add("drop:"+chain, p)
|
||||
add("drops", p)
|
||||
}
|
||||
}
|
||||
}
|
||||
for name, used := range t.Quotas {
|
||||
add("quota:"+strings.TrimPrefix(name, "quota_"), c.table.of("quota:"+name, used))
|
||||
}
|
||||
for set, es := range t.QuotaSets {
|
||||
kind := strings.TrimSuffix(strings.TrimSuffix(set, "_v4"), "_v6")
|
||||
for _, e := range es {
|
||||
addTop(kind, e.Addr, c.table.of(set+"/"+e.Addr, e.Used))
|
||||
}
|
||||
}
|
||||
for set, es := range t.DeviceSets {
|
||||
kind := map[bool]string{true: KindDeviceUp, false: KindDeviceDown}[strings.HasPrefix(set, "stats_up")]
|
||||
for _, e := range es {
|
||||
key := e.Addr
|
||||
if strings.HasSuffix(set, "_v6") {
|
||||
key += "/64"
|
||||
}
|
||||
addTop(kind, key, c.table.of(set+"/"+e.Addr, int64(e.Bytes)))
|
||||
}
|
||||
}
|
||||
c.table.done()
|
||||
case errors.Is(err, nft.ErrNoTable):
|
||||
default:
|
||||
errs = append(errs, "reading the table: "+err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
// Gauges.
|
||||
l := &Latest{At: now}
|
||||
if n, m, ok := readConntrack(c.src.Proc); ok {
|
||||
l.Conntrack, l.ConntrackMax = n, m
|
||||
gauge(GaugeConntrack, n)
|
||||
gauge(GaugeConntrackMax, m)
|
||||
}
|
||||
if rss, ticks, ok := readSelf(c.src.Proc); ok {
|
||||
l.RSS = rss
|
||||
gauge(GaugeRSS, rss)
|
||||
if !c.cpuAt.IsZero() && ticks >= c.cpuTicks {
|
||||
secs := max(int64(now.Sub(c.cpuAt)/time.Second), 1)
|
||||
l.CPUPermille = (ticks - c.cpuTicks) * 1000 / clockTicks / secs
|
||||
gauge(GaugeCPU, l.CPUPermille)
|
||||
}
|
||||
c.cpuTicks, c.cpuAt = ticks, now
|
||||
}
|
||||
if mem, load, ok := readHost(c.src.Proc); ok {
|
||||
l.MemUsed, l.Load = mem, load
|
||||
gauge(GaugeMem, mem)
|
||||
gauge(GaugeLoad, int64(load*100))
|
||||
}
|
||||
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
c.latest = l
|
||||
c.status.LastSample = &now
|
||||
c.status.SampleError = strings.Join(errs, "; ")
|
||||
if minute < 0 {
|
||||
return nil
|
||||
}
|
||||
h := c.hourOf(time.Unix(minute, 0))
|
||||
for kind, m := range tops {
|
||||
for k, v := range m {
|
||||
c.addTop(h, kind, k, v)
|
||||
}
|
||||
}
|
||||
return vals
|
||||
}
|
||||
|
||||
// Packet counts a packet the firewall logged: drops by source and by
|
||||
// what they tried.
|
||||
func (c *Collector) Packet(e api.LogEntry) {
|
||||
if e.Verdict != "drop" && e.Verdict != "reject" {
|
||||
return
|
||||
}
|
||||
what := e.Protocol
|
||||
switch {
|
||||
case e.DestPort > 0:
|
||||
what = fmt.Sprintf("%s/%d", e.Protocol, e.DestPort)
|
||||
case e.ICMPType != "":
|
||||
what = e.Protocol + "/" + e.ICMPType
|
||||
}
|
||||
t := e.Time
|
||||
if t.IsZero() {
|
||||
t = c.now()
|
||||
}
|
||||
h := c.hourOf(t)
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
if e.Source != "" {
|
||||
c.addTop(h, KindBlockedSource, e.Source, 1)
|
||||
}
|
||||
if what != "" {
|
||||
c.addTop(h, KindBlockedPort, what, 1)
|
||||
}
|
||||
}
|
||||
|
||||
// addTop counts into an hour's top list. Called with c.mu held.
|
||||
func (c *Collector) addTop(hour int64, kind, key string, n int64) {
|
||||
m := c.topsMap(hour, kind)
|
||||
if _, ok := m[key]; !ok && len(m) >= maxKeys {
|
||||
return
|
||||
}
|
||||
m[key] += n
|
||||
}
|
||||
|
||||
func (c *Collector) topsMap(hour int64, kind string) map[string]int64 {
|
||||
h := c.hours[hour]
|
||||
if h == nil {
|
||||
h = map[string]map[string]int64{}
|
||||
c.hours[hour] = h
|
||||
}
|
||||
m := h[kind]
|
||||
if m == nil {
|
||||
m = map[string]int64{}
|
||||
h[kind] = m
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
// top returns the n largest entries of m.
|
||||
func top(m map[string]int64, n int) []Entry {
|
||||
out := make([]Entry, 0, len(m))
|
||||
for k, v := range m {
|
||||
out = append(out, Entry{Key: k, Count: v})
|
||||
}
|
||||
slices.SortFunc(out, func(a, b Entry) int {
|
||||
if a.Count != b.Count {
|
||||
return cmp.Compare(b.Count, a.Count)
|
||||
}
|
||||
return cmp.Compare(a.Key, b.Key)
|
||||
})
|
||||
if len(out) > n {
|
||||
out = out[:n]
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func (c *Collector) prune(ctx context.Context) {
|
||||
now := c.now().Unix()
|
||||
for res, keep := range map[string]time.Duration{ResMinute: c.opts.RetainMinute, ResHour: c.opts.RetainHour, ResDay: c.opts.RetainDay} {
|
||||
if keep <= 0 {
|
||||
continue
|
||||
}
|
||||
if n, err := c.store.Prune(ctx, res, now-int64(keep.Seconds())); err != nil {
|
||||
c.log.Warn("statistics: pruning failed", "err", err)
|
||||
} else if n > 0 {
|
||||
c.log.Debug("statistics: pruned", "resolution", res, "rows", n)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,238 @@
|
||||
package stats
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"log/slog"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"housecarl/internal/api"
|
||||
"housecarl/internal/nft"
|
||||
"housecarl/internal/render"
|
||||
)
|
||||
|
||||
// fakeHost is a /sys/class/net and a /proc to read from.
|
||||
type fakeHost struct {
|
||||
t *testing.T
|
||||
sys, proc string
|
||||
table *nft.Table
|
||||
tableError error
|
||||
}
|
||||
|
||||
func newFakeHost(t *testing.T) *fakeHost {
|
||||
dir := t.TempDir()
|
||||
h := &fakeHost{t: t, sys: filepath.Join(dir, "sys"), proc: filepath.Join(dir, "proc"),
|
||||
table: &nft.Table{Counters: map[string]nft.Counter{}, QuotaUsage: nft.QuotaUsage{Quotas: map[string]int64{}, QuotaSets: map[string][]render.Element{}},
|
||||
DeviceSets: map[string][]render.Element{}}}
|
||||
h.write("proc/sys/net/netfilter/nf_conntrack_count", "120")
|
||||
h.write("proc/sys/net/netfilter/nf_conntrack_max", "65536")
|
||||
h.write("proc/self/stat", "4242 (housecarld) S 1 4242 4242 0 -1 4194560 3000 0 0 0 150 50 0 0 20 0 9 0 100 1000000 3000")
|
||||
h.write("proc/self/status", "Name:\thousecarld\nVmRSS:\t 20480 kB\nThreads:\t9\n")
|
||||
h.write("proc/meminfo", "MemTotal: 8000000 kB\nMemFree: 1000000 kB\nMemAvailable: 6000000 kB\n")
|
||||
h.write("proc/loadavg", "0.50 0.40 0.30 1/200 4242\n")
|
||||
return h
|
||||
}
|
||||
|
||||
func (h *fakeHost) write(rel, content string) {
|
||||
p := filepath.Join(filepath.Dir(h.sys), rel)
|
||||
if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil {
|
||||
h.t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(p, []byte(content), 0o644); err != nil {
|
||||
h.t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func (h *fakeHost) iface(name string, rx, tx int64) {
|
||||
h.write("sys/"+name+"/statistics/rx_bytes", fmt.Sprint(rx))
|
||||
h.write("sys/"+name+"/statistics/tx_bytes", fmt.Sprint(tx))
|
||||
h.write("sys/"+name+"/statistics/rx_packets", fmt.Sprint(rx/100))
|
||||
h.write("sys/"+name+"/statistics/tx_packets", fmt.Sprint(tx/100))
|
||||
}
|
||||
|
||||
func (h *fakeHost) sources() Sources {
|
||||
return Sources{
|
||||
SysNet: h.sys, Proc: h.proc,
|
||||
Table: func(context.Context) (*nft.Table, error) {
|
||||
if h.tableError != nil {
|
||||
return nil, h.tableError
|
||||
}
|
||||
return h.table, nil
|
||||
},
|
||||
Zones: func() map[string]string { return map[string]string{"eth0": "wan", "eth1": "lan"} },
|
||||
Classify: func(mark string) (string, bool) {
|
||||
switch mark {
|
||||
case "r1":
|
||||
return "forward", true
|
||||
case "p-input":
|
||||
return "input", true
|
||||
}
|
||||
return "input", false
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func TestCollector(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
h := newFakeHost(t)
|
||||
h.iface("eth0", 1000, 500)
|
||||
h.iface("eth1", 2000, 2000)
|
||||
h.iface("lo", 9999, 9999)
|
||||
h.table.Counters["r1"] = nft.Counter{Packets: 10, Bytes: 1000}
|
||||
h.table.Counters["r2"] = nft.Counter{Packets: 100, Bytes: 9000}
|
||||
h.table.Quotas["quota_r5"] = 1 << 20
|
||||
h.table.DeviceSets["stats_up_v4"] = []render.Element{{Addr: "10.0.10.5", Bytes: 5000}}
|
||||
|
||||
store, err := OpenStore(":memory:")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer store.Close()
|
||||
loc := time.FixedZone("test", 0)
|
||||
now := time.Date(2026, 10, 19, 10, 0, 30, 0, loc)
|
||||
c := NewCollector(store, Options{TopN: 10, RetainMinute: 7 * 24 * time.Hour, RetainHour: 90 * 24 * time.Hour, RetainDay: 730 * 24 * time.Hour, Location: loc},
|
||||
h.sources(), slog.New(slog.NewTextHandler(io.Discard, nil)))
|
||||
c.SetClock(func() time.Time { return now })
|
||||
c.Start(ctx)
|
||||
|
||||
// A minute of traffic.
|
||||
h.iface("eth0", 5000, 1500)
|
||||
h.table.Counters["r1"] = nft.Counter{Packets: 25, Bytes: 2500}
|
||||
h.table.Counters["p-input"] = nft.Counter{Packets: 3, Bytes: 180} // a new counter counts whole
|
||||
h.table.Quotas["quota_r5"] = 3 << 20
|
||||
h.table.DeviceSets["stats_up_v4"] = []render.Element{{Addr: "10.0.10.5", Bytes: 8000}}
|
||||
h.table.DeviceSets["stats_down_v6"] = []render.Element{{Addr: "2001:db8:1:2::", Bytes: 70000}}
|
||||
for range 2 {
|
||||
c.Packet(api.LogEntry{Time: now, Source: "198.51.100.7", Protocol: "tcp", DestPort: 22, Verdict: "drop"})
|
||||
}
|
||||
c.Packet(api.LogEntry{Time: now, Source: "198.51.100.8", Protocol: "icmp", ICMPType: "echo-request", Verdict: "drop"})
|
||||
c.Packet(api.LogEntry{Time: now, Source: "10.0.10.5", Protocol: "udp", DestPort: 53, Verdict: "accept"}) // not a drop
|
||||
now = time.Date(2026, 10, 19, 10, 1, 2, 0, loc)
|
||||
c.Sample(ctx)
|
||||
|
||||
q := NewQuerier(c)
|
||||
sum := func(prefix string) map[string]int64 {
|
||||
es, err := q.Sums(ctx, prefix, time.Hour)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
m := map[string]int64{}
|
||||
for _, e := range es {
|
||||
m[e.Key] = e.Count
|
||||
}
|
||||
return m
|
||||
}
|
||||
check := func(what string, got, want int64) {
|
||||
t.Helper()
|
||||
if got != want {
|
||||
t.Errorf("%s = %d, want %d", what, got, want)
|
||||
}
|
||||
}
|
||||
check("rx:eth0", sum("rx:")["eth0"], 4000)
|
||||
check("rx:eth1 (no change)", sum("rx:")["eth1"], 0)
|
||||
check("rx:lo", sum("rx:")["lo"], 0)
|
||||
check("zrx:wan", sum("zrx:")["wan"], 4000)
|
||||
check("ztx:wan", sum("ztx:")["wan"], 1000)
|
||||
check("hits:r1", sum("hits:")["r1"], 15)
|
||||
check("hits:p-input", sum("hits:")["p-input"], 3)
|
||||
check("drop:forward", sum("drop:")["forward"], 15)
|
||||
check("drop:input", sum("drop:")["input"], 3)
|
||||
check("quota:r5", sum("quota:")["r5"], 2<<20)
|
||||
|
||||
s, err := q.Summary(ctx, time.Hour, 10)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
check("drops", s.Drops, 18)
|
||||
if len(s.Zones) == 0 || s.Zones[0].Name != "wan" || s.Zones[0].Received != 4000 {
|
||||
t.Errorf("zones %+v", s.Zones)
|
||||
}
|
||||
if len(s.BlockedSources) != 2 || s.BlockedSources[0] != (Entry{Key: "198.51.100.7", Count: 2}) {
|
||||
t.Errorf("blocked sources %+v", s.BlockedSources)
|
||||
}
|
||||
if len(s.BlockedPorts) != 2 || s.BlockedPorts[0].Key != "tcp/22" || s.BlockedPorts[1].Key != "icmp/echo-request" {
|
||||
t.Errorf("blocked ports %+v", s.BlockedPorts)
|
||||
}
|
||||
if len(s.DevicesUp) != 1 || s.DevicesUp[0] != (Entry{Key: "10.0.10.5", Count: 3000}) {
|
||||
t.Errorf("devices up %+v", s.DevicesUp)
|
||||
}
|
||||
if len(s.DevicesDown) != 1 || s.DevicesDown[0] != (Entry{Key: "2001:db8:1:2::/64", Count: 70000}) {
|
||||
t.Errorf("devices down %+v", s.DevicesDown)
|
||||
}
|
||||
check("peak connections", s.PeakConnections, 120)
|
||||
|
||||
// Gauges average; the CPU needs two samples.
|
||||
tl, err := q.Timeline(ctx, time.Hour, "1m", []string{"g:" + GaugeRSS, "g:" + GaugeMem, "g:" + GaugeLoad, "rx:*"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
last := func(i int) int64 {
|
||||
for j := len(tl.Times) - 1; j >= 0; j-- {
|
||||
if tl.Times[j] == time.Date(2026, 10, 19, 10, 0, 0, 0, loc).Unix() {
|
||||
return tl.Series[i].Values[j]
|
||||
}
|
||||
}
|
||||
return -2
|
||||
}
|
||||
check("rss", last(0), 20480*1024)
|
||||
check("mem", last(1), 2000000*1024)
|
||||
check("load", last(2), 50)
|
||||
if len(tl.Series) != 4 || tl.Series[3].Key != "rx:eth0" {
|
||||
t.Errorf("rx:* expanded to %+v", tl.Series[3:])
|
||||
}
|
||||
|
||||
// Totals that go down started again: an interface came back, a quota
|
||||
// was reset. A table that can't be read is skipped, and its keys
|
||||
// aren't new the next time.
|
||||
h.iface("eth0", 100, 1500)
|
||||
h.table.Quotas["quota_r5"] = 1000
|
||||
h.tableError = fmt.Errorf("nft failed")
|
||||
now = time.Date(2026, 10, 19, 10, 2, 2, 0, loc)
|
||||
c.Sample(ctx)
|
||||
if st := c.Status(); st.SampleError == "" {
|
||||
t.Error("no sample error")
|
||||
}
|
||||
h.tableError = nil
|
||||
now = time.Date(2026, 10, 19, 10, 3, 2, 0, loc)
|
||||
c.Sample(ctx)
|
||||
check("rx:eth0 after a reset", sum("rx:")["eth0"], 4100)
|
||||
check("quota:r5 after a reset", sum("quota:")["r5"], 2<<20+1000)
|
||||
check("hits:r1 after a failed read", sum("hits:")["r1"], 15)
|
||||
|
||||
// The hour ends: its top lists go to its day.
|
||||
now = time.Date(2026, 10, 19, 11, 0, 2, 0, loc)
|
||||
c.Sample(ctx)
|
||||
day, err := store.TopsAt(ctx, ResDay, time.Date(2026, 10, 19, 0, 0, 0, 0, loc).Unix(), KindBlockedSource)
|
||||
if err != nil || len(day) != 2 || day[0].Count != 2 {
|
||||
t.Errorf("the day's blocked sources: %+v %v", day, err)
|
||||
}
|
||||
if es, _ := q.Top(ctx, KindBlockedSource, 30*24*time.Hour, 5); len(es) != 2 {
|
||||
t.Errorf("30 days of blocked sources: %+v", es)
|
||||
}
|
||||
|
||||
// A restart continues the hour's top lists.
|
||||
c.Packet(api.LogEntry{Time: now, Source: "203.0.113.9", Protocol: "tcp", DestPort: 23, Verdict: "drop"})
|
||||
c.write(ctx, &Batch{})
|
||||
c2 := NewCollector(store, c.opts, h.sources(), c.log)
|
||||
c2.SetClock(func() time.Time { return now })
|
||||
c2.Start(ctx)
|
||||
c2.Packet(api.LogEntry{Time: now, Source: "203.0.113.9", Protocol: "tcp", DestPort: 23, Verdict: "drop"})
|
||||
c2.write(ctx, &Batch{})
|
||||
hour, _ := store.TopsAt(ctx, ResHour, time.Date(2026, 10, 19, 11, 0, 0, 0, loc).Unix(), KindBlockedSource)
|
||||
if len(hour) != 1 || hour[0].Count != 2 {
|
||||
t.Errorf("after a restart, the hour's blocked sources: %+v", hour)
|
||||
}
|
||||
|
||||
// The heatmap of drops.
|
||||
hm, err := q.Heatmap(ctx, "", 7*24*time.Hour)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if hm.Avg[0][10] != 18 { // a Monday, 10:00
|
||||
t.Errorf("heatmap Monday 10:00 = %v", hm.Avg[0][10])
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,480 @@
|
||||
package stats
|
||||
|
||||
import (
|
||||
"cmp"
|
||||
"context"
|
||||
"fmt"
|
||||
"slices"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Querier answers questions about stored statistics.
|
||||
type Querier struct {
|
||||
Store *Store
|
||||
Opts Options
|
||||
Now func() time.Time
|
||||
}
|
||||
|
||||
// NewQuerier returns a querier for a collector's data.
|
||||
func NewQuerier(c *Collector) *Querier {
|
||||
return &Querier{Store: c.store, Opts: c.opts, Now: c.now}
|
||||
}
|
||||
|
||||
func (q *Querier) loc() *time.Location {
|
||||
if q.Opts.Location != nil {
|
||||
return q.Opts.Location
|
||||
}
|
||||
return time.Local
|
||||
}
|
||||
|
||||
// Resolution picks the finest stored resolution that covers a time range
|
||||
// starting since ago.
|
||||
func (q *Querier) Resolution(since time.Duration) string {
|
||||
switch {
|
||||
case since <= 48*time.Hour && since <= q.Opts.RetainMinute:
|
||||
return ResMinute
|
||||
case since <= q.Opts.RetainHour:
|
||||
return ResHour
|
||||
}
|
||||
return ResDay
|
||||
}
|
||||
|
||||
// topsResolution picks the resolution for top lists, which exist per hour
|
||||
// and per day.
|
||||
func (q *Querier) topsResolution(since time.Duration) string {
|
||||
if since <= 7*24*time.Hour && since <= q.Opts.RetainHour {
|
||||
return ResHour
|
||||
}
|
||||
return ResDay
|
||||
}
|
||||
|
||||
// bucketStart returns the start of the bucket containing t at a resolution.
|
||||
func (q *Querier) bucketStart(res string, t time.Time) int64 {
|
||||
switch res {
|
||||
case ResMinute:
|
||||
return t.Unix() - t.Unix()%60
|
||||
case ResHour:
|
||||
return t.Unix() - t.Unix()%3600
|
||||
}
|
||||
l := t.In(q.loc())
|
||||
return time.Date(l.Year(), l.Month(), l.Day(), 0, 0, 0, 0, q.loc()).Unix()
|
||||
}
|
||||
|
||||
// sums returns the totals of counters (optionally with a key prefix) in
|
||||
// [from, to).
|
||||
func (s *Store) sums(ctx context.Context, res string, from, to int64, prefix string) (map[string]int64, error) {
|
||||
rows, err := s.db.QueryContext(ctx, `SELECT key, SUM(value) FROM counters WHERE res = ? AND ts >= ? AND ts < ? AND key LIKE ? ESCAPE '\' GROUP BY key`,
|
||||
res, from, to, likePrefix(prefix))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
out := map[string]int64{}
|
||||
for rows.Next() {
|
||||
var k string
|
||||
var v int64
|
||||
if err := rows.Scan(&k, &v); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out[k] = v
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
func likePrefix(p string) string {
|
||||
r := strings.NewReplacer(`\`, `\\`, `%`, `\%`, `_`, `\_`)
|
||||
return r.Replace(p) + "%"
|
||||
}
|
||||
|
||||
// series returns the values of a counter per bucket in [from, to).
|
||||
func (s *Store) series(ctx context.Context, res string, from, to int64, key string) (map[int64]int64, error) {
|
||||
rows, err := s.db.QueryContext(ctx, `SELECT ts, value FROM counters WHERE res = ? AND key = ? AND ts >= ? AND ts < ?`, res, key, from, to)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
out := map[int64]int64{}
|
||||
for rows.Next() {
|
||||
var ts, v int64
|
||||
if err := rows.Scan(&ts, &v); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out[ts] = v
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
// tops sums top lists (of one or more kinds) in [from, to).
|
||||
func (s *Store) tops(ctx context.Context, res string, kinds []string, from, to int64, limit int) ([]Entry, error) {
|
||||
args := []any{res}
|
||||
for _, k := range kinds {
|
||||
args = append(args, k)
|
||||
}
|
||||
args = append(args, from, to, limit)
|
||||
rows, err := s.db.QueryContext(ctx, `SELECT key, SUM(count) AS n FROM tops WHERE res = ? AND kind IN (`+strings.TrimSuffix(strings.Repeat("?,", len(kinds)), ",")+`) AND ts >= ? AND ts < ?
|
||||
GROUP BY key ORDER BY n DESC, key LIMIT ?`, args...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
out := []Entry{}
|
||||
for rows.Next() {
|
||||
var e Entry
|
||||
if err := rows.Scan(&e.Key, &e.Count); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, e)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
func sorted(m map[string]int64, prefix string) []Entry {
|
||||
out := []Entry{}
|
||||
for k, v := range m {
|
||||
if strings.HasPrefix(k, prefix) && v != 0 {
|
||||
out = append(out, Entry{Key: strings.TrimPrefix(k, prefix), Count: v})
|
||||
}
|
||||
}
|
||||
slices.SortFunc(out, func(a, b Entry) int {
|
||||
if a.Count != b.Count {
|
||||
return cmp.Compare(b.Count, a.Count)
|
||||
}
|
||||
return cmp.Compare(a.Key, b.Key)
|
||||
})
|
||||
return out
|
||||
}
|
||||
|
||||
// Top returns the largest entries of a top list over the last since.
|
||||
func (q *Querier) Top(ctx context.Context, kind string, since time.Duration, limit int) ([]Entry, error) {
|
||||
if !slices.Contains(Kinds, kind) && !quotaKind(kind) {
|
||||
return nil, fmt.Errorf("unknown list %q (use %s, or quota_rN)", kind, strings.Join(Kinds, ", "))
|
||||
}
|
||||
now := q.Now()
|
||||
res := q.topsResolution(since)
|
||||
return q.Store.tops(ctx, res, []string{kind}, q.bucketStart(res, now.Add(-since)), now.Unix()+1, limit)
|
||||
}
|
||||
|
||||
func quotaKind(k string) bool {
|
||||
n, ok := strings.CutPrefix(k, "quota_r")
|
||||
_, err := strconv.ParseInt(n, 10, 64)
|
||||
return ok && err == nil
|
||||
}
|
||||
|
||||
// Sums returns the totals of the counters with a prefix over the last
|
||||
// since, largest first, the prefix taken off ("hits:" → "r12").
|
||||
func (q *Querier) Sums(ctx context.Context, prefix string, since time.Duration) ([]Entry, error) {
|
||||
now := q.Now()
|
||||
res := q.Resolution(since)
|
||||
m, err := q.Store.sums(ctx, res, q.bucketStart(res, now.Add(-since)), now.Unix()+1, prefix)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return sorted(m, prefix), nil
|
||||
}
|
||||
|
||||
// Traffic is what an interface or zone received and sent, in bytes.
|
||||
type Traffic struct {
|
||||
Name string `json:"name"`
|
||||
Received int64 `json:"received"`
|
||||
Sent int64 `json:"sent"`
|
||||
}
|
||||
|
||||
// Summary is an overview of the last since.
|
||||
type Summary struct {
|
||||
Since string `json:"since"`
|
||||
From time.Time `json:"from"`
|
||||
Zones []Traffic `json:"zones"`
|
||||
Interfaces []Traffic `json:"interfaces"`
|
||||
// Drops are the packets dropped (rules, built-ins, policies), and by
|
||||
// chain.
|
||||
Drops int64 `json:"drops"`
|
||||
DropsByChain []Entry `json:"drops_by_chain"`
|
||||
// Rules are the busiest rules, built-ins and policies, by packets.
|
||||
Rules []Entry `json:"rules"`
|
||||
// Connections: the most tracked at once (per-bucket averages), and
|
||||
// the table's size.
|
||||
PeakConnections int64 `json:"peak_connections"`
|
||||
ConntrackMax int64 `json:"conntrack_max"`
|
||||
BlockedSources []Entry `json:"blocked_sources"`
|
||||
BlockedPorts []Entry `json:"blocked_ports"`
|
||||
DevicesUp []Entry `json:"devices_up"`
|
||||
DevicesDown []Entry `json:"devices_down"`
|
||||
}
|
||||
|
||||
// Summary sums up the last since, with top lists of up to topLimit.
|
||||
func (q *Querier) Summary(ctx context.Context, since time.Duration, topLimit int) (*Summary, error) {
|
||||
now := q.Now()
|
||||
res := q.Resolution(since)
|
||||
from := q.bucketStart(res, now.Add(-since))
|
||||
all, err := q.Store.sums(ctx, res, from, now.Unix()+1, "")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
s := &Summary{Since: humanDuration(since), From: time.Unix(from, 0), Zones: []Traffic{}, Interfaces: []Traffic{}}
|
||||
traffic := func(rx, tx string) []Traffic {
|
||||
by := map[string]*Traffic{}
|
||||
for k, v := range all {
|
||||
var name string
|
||||
var in bool
|
||||
if n, ok := strings.CutPrefix(k, rx); ok {
|
||||
name, in = n, true
|
||||
} else if n, ok := strings.CutPrefix(k, tx); ok {
|
||||
name = n
|
||||
} else {
|
||||
continue
|
||||
}
|
||||
t := by[name]
|
||||
if t == nil {
|
||||
t = &Traffic{Name: name}
|
||||
by[name] = t
|
||||
}
|
||||
if in {
|
||||
t.Received += v
|
||||
} else {
|
||||
t.Sent += v
|
||||
}
|
||||
}
|
||||
out := []Traffic{}
|
||||
for _, t := range by {
|
||||
out = append(out, *t)
|
||||
}
|
||||
slices.SortFunc(out, func(a, b Traffic) int {
|
||||
if c := cmp.Compare(b.Received+b.Sent, a.Received+a.Sent); c != 0 {
|
||||
return c
|
||||
}
|
||||
return cmp.Compare(a.Name, b.Name)
|
||||
})
|
||||
return out
|
||||
}
|
||||
s.Zones, s.Interfaces = traffic("zrx:", "ztx:"), traffic("rx:", "tx:")
|
||||
s.Drops = all["drops"]
|
||||
s.DropsByChain = sorted(all, "drop:")
|
||||
s.Rules = sorted(all, "hits:")
|
||||
if len(s.Rules) > topLimit {
|
||||
s.Rules = s.Rules[:topLimit]
|
||||
}
|
||||
if tl, err := q.Timeline(ctx, since, "auto", []string{"g:" + GaugeConntrack, "g:" + GaugeConntrackMax}); err == nil {
|
||||
for _, v := range tl.Series[0].Values {
|
||||
s.PeakConnections = max(s.PeakConnections, v)
|
||||
}
|
||||
for _, v := range tl.Series[1].Values {
|
||||
s.ConntrackMax = max(s.ConntrackMax, v)
|
||||
}
|
||||
}
|
||||
for _, l := range []struct {
|
||||
kind string
|
||||
dst *[]Entry
|
||||
}{{KindBlockedSource, &s.BlockedSources}, {KindBlockedPort, &s.BlockedPorts}, {KindDeviceUp, &s.DevicesUp}, {KindDeviceDown, &s.DevicesDown}} {
|
||||
es, err := q.Top(ctx, l.kind, since, topLimit)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
*l.dst = es
|
||||
}
|
||||
return s, nil
|
||||
}
|
||||
|
||||
// Timeline is a set of series over time.
|
||||
type Timeline struct {
|
||||
Step string `json:"step"`
|
||||
Times []int64 `json:"times"` // bucket starts (Unix seconds)
|
||||
Series []*Series `json:"series"`
|
||||
}
|
||||
|
||||
// Series is one counter over time. For a gauge (g:KEY) the values are
|
||||
// averages, and -1 where there is no sample.
|
||||
type Series struct {
|
||||
Key string `json:"key"`
|
||||
Values []int64 `json:"values"`
|
||||
}
|
||||
|
||||
var steps = []struct {
|
||||
name string
|
||||
d time.Duration
|
||||
res string
|
||||
}{
|
||||
{"1m", time.Minute, ResMinute}, {"5m", 5 * time.Minute, ResMinute}, {"15m", 15 * time.Minute, ResMinute},
|
||||
{"1h", time.Hour, ResHour}, {"6h", 6 * time.Hour, ResHour}, {"1d", 24 * time.Hour, ResDay},
|
||||
}
|
||||
|
||||
// Timeline returns counters over the last since. keys are counter names;
|
||||
// a key ending in ":*" means every counter of that group. step is "auto"
|
||||
// or one of 1m, 5m, 15m, 1h, 6h, 1d.
|
||||
func (q *Querier) Timeline(ctx context.Context, since time.Duration, step string, keys []string) (*Timeline, error) {
|
||||
idx := -1
|
||||
for i, st := range steps {
|
||||
ok := st.res == ResMinute && since <= q.Opts.RetainMinute || st.res == ResHour && since <= q.Opts.RetainHour || st.res == ResDay
|
||||
if step == st.name {
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("step %s needs %s data, which is only kept for %s", st.name, map[string]string{ResMinute: "per-minute", ResHour: "hourly"}[st.res], humanDuration(map[string]time.Duration{ResMinute: q.Opts.RetainMinute, ResHour: q.Opts.RetainHour}[st.res]))
|
||||
}
|
||||
idx = i
|
||||
break
|
||||
}
|
||||
if step == "auto" || step == "" {
|
||||
if ok && since/st.d <= 300 {
|
||||
idx = i
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
if idx < 0 {
|
||||
if step != "auto" && step != "" {
|
||||
return nil, fmt.Errorf("unknown step %q (use auto, 1m, 5m, 15m, 1h, 6h or 1d)", step)
|
||||
}
|
||||
idx = len(steps) - 1
|
||||
}
|
||||
st := steps[idx]
|
||||
now := q.Now()
|
||||
// Buckets aligned to the step (days to local midnight).
|
||||
var starts []int64
|
||||
if st.res == ResDay {
|
||||
for t := time.Unix(q.bucketStart(ResDay, now.Add(-since)), 0).In(q.loc()); !t.After(now); t = t.AddDate(0, 0, 1) {
|
||||
starts = append(starts, t.Unix())
|
||||
}
|
||||
} else {
|
||||
sd := int64(st.d.Seconds())
|
||||
first := now.Add(-since).Unix()
|
||||
first -= first % sd
|
||||
for t := first; t <= now.Unix(); t += sd {
|
||||
starts = append(starts, t)
|
||||
}
|
||||
}
|
||||
from, to := starts[0], now.Unix()+1
|
||||
// Expand group keys.
|
||||
var expanded []string
|
||||
for _, k := range keys {
|
||||
if g, ok := strings.CutSuffix(k, ":*"); ok {
|
||||
m, err := q.Store.sums(ctx, st.res, from, to, g+":")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, e := range sorted(m, "") {
|
||||
expanded = append(expanded, e.Key)
|
||||
}
|
||||
continue
|
||||
}
|
||||
expanded = append(expanded, k)
|
||||
}
|
||||
tl := &Timeline{Step: st.name, Times: starts, Series: []*Series{}}
|
||||
bucketed := func(key string) ([]int64, error) {
|
||||
raw, err := q.Store.series(ctx, st.res, from, to, key)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
vals := make([]int64, len(starts))
|
||||
for ts, v := range raw {
|
||||
// The bucket is the last start <= ts.
|
||||
i, found := slices.BinarySearch(starts, ts)
|
||||
if !found {
|
||||
i--
|
||||
}
|
||||
if i >= 0 && i < len(vals) {
|
||||
vals[i] += v
|
||||
}
|
||||
}
|
||||
return vals, nil
|
||||
}
|
||||
for _, k := range expanded {
|
||||
vals, err := bucketed(k)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// A gauge (g:KEY) is the average of its samples: its sum divided
|
||||
// by gn:KEY, their number. A bucket without samples is -1.
|
||||
if g, ok := strings.CutPrefix(k, "g:"); ok {
|
||||
n, err := bucketed("gn:" + g)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for i := range vals {
|
||||
if n[i] > 0 {
|
||||
vals[i] /= n[i]
|
||||
} else {
|
||||
vals[i] = -1
|
||||
}
|
||||
}
|
||||
}
|
||||
tl.Series = append(tl.Series, &Series{Key: k, Values: vals})
|
||||
}
|
||||
return tl, nil
|
||||
}
|
||||
|
||||
// Heatmap is average activity per weekday and hour of day (local time).
|
||||
type Heatmap struct {
|
||||
Key string `json:"key"`
|
||||
// Avg[d][h] is the average per hour on weekday d (0 = Monday) at
|
||||
// hour h; -1 where there is no data.
|
||||
Avg [7][24]float64 `json:"avg"`
|
||||
// Samples[d][h] is how many hours the average is based on.
|
||||
Samples [7][24]int `json:"samples"`
|
||||
From time.Time `json:"from"`
|
||||
}
|
||||
|
||||
// Heatmap averages a counter (default "drops") by weekday and hour over
|
||||
// the last since (at most the hourly retention).
|
||||
func (q *Querier) Heatmap(ctx context.Context, key string, since time.Duration) (*Heatmap, error) {
|
||||
if key == "" {
|
||||
key = "drops"
|
||||
}
|
||||
since = min(since, q.Opts.RetainHour)
|
||||
now := q.Now()
|
||||
from := q.bucketStart(ResHour, now.Add(-since))
|
||||
raw, err := q.Store.series(ctx, ResHour, from, now.Unix()+1, key)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
hm := &Heatmap{Key: key, From: time.Unix(from, 0)}
|
||||
var sum [7][24]int64
|
||||
for ts, v := range raw {
|
||||
t := time.Unix(ts, 0).In(q.loc())
|
||||
d := (int(t.Weekday()) + 6) % 7
|
||||
sum[d][t.Hour()] += v
|
||||
hm.Samples[d][t.Hour()]++
|
||||
}
|
||||
for d := range 7 {
|
||||
for h := range 24 {
|
||||
if n := hm.Samples[d][h]; n > 0 {
|
||||
hm.Avg[d][h] = float64(sum[d][h]) / float64(n)
|
||||
} else {
|
||||
hm.Avg[d][h] = -1
|
||||
}
|
||||
}
|
||||
}
|
||||
return hm, nil
|
||||
}
|
||||
|
||||
// ParseSince parses a range such as 90m, 24h, 7d or 2w.
|
||||
func ParseSince(s string) (time.Duration, error) {
|
||||
if s == "" {
|
||||
return 24 * time.Hour, nil
|
||||
}
|
||||
mult := time.Duration(0)
|
||||
switch {
|
||||
case strings.HasSuffix(s, "d"):
|
||||
mult = 24 * time.Hour
|
||||
case strings.HasSuffix(s, "w"):
|
||||
mult = 7 * 24 * time.Hour
|
||||
}
|
||||
if mult > 0 {
|
||||
var n int
|
||||
if _, err := fmt.Sscanf(s[:len(s)-1], "%d", &n); err != nil || n <= 0 || fmt.Sprint(n) != s[:len(s)-1] {
|
||||
return 0, fmt.Errorf("invalid range %q (use e.g. 90m, 24h, 7d, 2w)", s)
|
||||
}
|
||||
return time.Duration(n) * mult, nil
|
||||
}
|
||||
d, err := time.ParseDuration(s)
|
||||
if err != nil || d <= 0 {
|
||||
return 0, fmt.Errorf("invalid range %q (use e.g. 90m, 24h, 7d, 2w)", s)
|
||||
}
|
||||
return d, nil
|
||||
}
|
||||
|
||||
func humanDuration(d time.Duration) string {
|
||||
if d%(24*time.Hour) == 0 && d > 0 {
|
||||
return strconv.Itoa(int(d/(24*time.Hour))) + "d"
|
||||
}
|
||||
return d.String()
|
||||
}
|
||||
@@ -0,0 +1,136 @@
|
||||
package stats
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// What the collector reads from the kernel without privileges: the
|
||||
// interfaces' counters (/sys/class/net), the connection tracking table's
|
||||
// size, and housecarld's and the host's memory and load (/proc).
|
||||
|
||||
// clockTicks is USER_HZ, the unit of the CPU times in /proc/PID/stat. It
|
||||
// is 100 on every Linux architecture Housecarl runs on.
|
||||
const clockTicks = 100
|
||||
|
||||
// ifCounters are an interface's totals since it came up.
|
||||
type ifCounters struct {
|
||||
RxBytes, TxBytes, RxPackets, TxPackets int64
|
||||
}
|
||||
|
||||
// readInterfaces reads every interface's counters but the loopback's.
|
||||
func readInterfaces(sysNet string) map[string]ifCounters {
|
||||
out := map[string]ifCounters{}
|
||||
entries, err := os.ReadDir(sysNet)
|
||||
if err != nil {
|
||||
return out
|
||||
}
|
||||
for _, e := range entries {
|
||||
name := e.Name()
|
||||
if name == "lo" {
|
||||
continue
|
||||
}
|
||||
dir := filepath.Join(sysNet, name, "statistics")
|
||||
read := func(f string) (int64, bool) {
|
||||
b, err := os.ReadFile(filepath.Join(dir, f))
|
||||
if err != nil {
|
||||
return 0, false
|
||||
}
|
||||
n, err := strconv.ParseInt(strings.TrimSpace(string(b)), 10, 64)
|
||||
return n, err == nil
|
||||
}
|
||||
var c ifCounters
|
||||
var ok bool
|
||||
if c.RxBytes, ok = read("rx_bytes"); !ok {
|
||||
continue
|
||||
}
|
||||
c.TxBytes, _ = read("tx_bytes")
|
||||
c.RxPackets, _ = read("rx_packets")
|
||||
c.TxPackets, _ = read("tx_packets")
|
||||
out[name] = c
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// readConntrack returns how many connections the kernel tracks, and how
|
||||
// many it can (ok false without connection tracking).
|
||||
func readConntrack(proc string) (count, max int64, ok bool) {
|
||||
read := func(f string) (int64, bool) {
|
||||
b, err := os.ReadFile(filepath.Join(proc, "sys/net/netfilter", f))
|
||||
if err != nil {
|
||||
return 0, false
|
||||
}
|
||||
n, err := strconv.ParseInt(strings.TrimSpace(string(b)), 10, 64)
|
||||
return n, err == nil
|
||||
}
|
||||
count, ok1 := read("nf_conntrack_count")
|
||||
max, ok2 := read("nf_conntrack_max")
|
||||
return count, max, ok1 && ok2
|
||||
}
|
||||
|
||||
// readSelf returns housecarld's resident memory (bytes) and CPU time
|
||||
// (clock ticks).
|
||||
func readSelf(proc string) (rss, cpu int64, ok bool) {
|
||||
dir := filepath.Join(proc, "self")
|
||||
stat, err := os.ReadFile(filepath.Join(dir, "stat"))
|
||||
if err != nil {
|
||||
return 0, 0, false
|
||||
}
|
||||
// The command name is in parentheses and may contain spaces: the
|
||||
// fields that follow start after the last ')'.
|
||||
i := bytes.LastIndexByte(stat, ')')
|
||||
if i < 0 {
|
||||
return 0, 0, false
|
||||
}
|
||||
f := strings.Fields(string(stat[i+1:]))
|
||||
// f[0] is field 3 (state); utime and stime are fields 14 and 15.
|
||||
if len(f) < 13 {
|
||||
return 0, 0, false
|
||||
}
|
||||
utime, _ := strconv.ParseInt(f[11], 10, 64)
|
||||
stime, _ := strconv.ParseInt(f[12], 10, 64)
|
||||
status, err := os.ReadFile(filepath.Join(dir, "status"))
|
||||
if err != nil {
|
||||
return 0, 0, false
|
||||
}
|
||||
for _, line := range strings.Split(string(status), "\n") {
|
||||
if k, v, ok := strings.Cut(line, ":"); ok && k == "VmRSS" {
|
||||
rss = kib(v)
|
||||
}
|
||||
}
|
||||
return rss, utime + stime, true
|
||||
}
|
||||
|
||||
// readHost returns the memory in use (bytes) and the 1-minute load
|
||||
// average.
|
||||
func readHost(proc string) (memUsed int64, load float64, ok bool) {
|
||||
b, err := os.ReadFile(filepath.Join(proc, "meminfo"))
|
||||
if err != nil {
|
||||
return 0, 0, false
|
||||
}
|
||||
var total, avail int64
|
||||
for _, line := range strings.Split(string(b), "\n") {
|
||||
k, v, _ := strings.Cut(line, ":")
|
||||
switch k {
|
||||
case "MemTotal":
|
||||
total = kib(v)
|
||||
case "MemAvailable":
|
||||
avail = kib(v)
|
||||
}
|
||||
}
|
||||
if b, err := os.ReadFile(filepath.Join(proc, "loadavg")); err == nil {
|
||||
if f := strings.Fields(string(b)); len(f) > 0 {
|
||||
load, _ = strconv.ParseFloat(f[0], 64)
|
||||
}
|
||||
}
|
||||
return total - avail, load, total > 0
|
||||
}
|
||||
|
||||
// kib parses a "1234 kB" value from /proc into bytes.
|
||||
func kib(v string) int64 {
|
||||
n, _ := strconv.ParseInt(strings.TrimSpace(strings.TrimSuffix(strings.TrimSpace(v), "kB")), 10, 64)
|
||||
return n * 1024
|
||||
}
|
||||
@@ -0,0 +1,304 @@
|
||||
// Package stats keeps the firewall's figures over time: traffic per
|
||||
// interface and zone, hits per rule, drops, blocked sources, connections,
|
||||
// quotas and the machine's health, sampled every minute. It stores
|
||||
// aggregates in a SQLite database of its own, separate from Housecarl's
|
||||
// configuration, so it can be deleted at any time without losing anything
|
||||
// important.
|
||||
package stats
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
|
||||
_ "modernc.org/sqlite" // registers the "sqlite" driver
|
||||
)
|
||||
|
||||
// Resolutions of stored data.
|
||||
const (
|
||||
ResMinute = "m"
|
||||
ResHour = "h"
|
||||
ResDay = "d"
|
||||
)
|
||||
|
||||
// Store is the statistics database.
|
||||
type Store struct {
|
||||
db *sql.DB
|
||||
path string
|
||||
}
|
||||
|
||||
var schema = []string{
|
||||
`CREATE TABLE counters (
|
||||
res TEXT NOT NULL,
|
||||
ts INTEGER NOT NULL,
|
||||
key TEXT NOT NULL,
|
||||
value INTEGER NOT NULL,
|
||||
PRIMARY KEY (res, ts, key)
|
||||
) WITHOUT ROWID;
|
||||
CREATE INDEX counters_by_key ON counters (res, key, ts);
|
||||
CREATE TABLE tops (
|
||||
res TEXT NOT NULL,
|
||||
ts INTEGER NOT NULL,
|
||||
kind TEXT NOT NULL,
|
||||
key TEXT NOT NULL,
|
||||
count INTEGER NOT NULL,
|
||||
PRIMARY KEY (res, ts, kind, key)
|
||||
) WITHOUT ROWID;
|
||||
CREATE TABLE meta (key TEXT PRIMARY KEY, value TEXT NOT NULL);`,
|
||||
}
|
||||
|
||||
// Size is the database's size on disk, with its write-ahead log.
|
||||
func (s *Store) Size() int64 {
|
||||
var n int64
|
||||
for _, f := range []string{s.path, s.path + "-wal"} {
|
||||
if fi, err := os.Stat(f); err == nil {
|
||||
n += fi.Size()
|
||||
}
|
||||
}
|
||||
return n
|
||||
}
|
||||
|
||||
// Oldest is the start of the oldest stored bucket (0 when empty).
|
||||
func (s *Store) Oldest(ctx context.Context) (int64, error) {
|
||||
var ts sql.NullInt64
|
||||
err := s.db.QueryRowContext(ctx, `SELECT MIN(ts) FROM counters`).Scan(&ts)
|
||||
return ts.Int64, err
|
||||
}
|
||||
|
||||
// OpenStore opens (creating if needed) the statistics database. Use
|
||||
// ":memory:" for tests.
|
||||
func OpenStore(path string) (*Store, error) {
|
||||
dsn := "file:" + path + "?_pragma=busy_timeout(10000)&_pragma=synchronous(NORMAL)&_txlock=immediate"
|
||||
if path == ":memory:" {
|
||||
dsn = "file::memory:?_txlock=immediate"
|
||||
} else {
|
||||
dsn += "&_pragma=journal_mode(WAL)"
|
||||
}
|
||||
db, err := sql.Open("sqlite", dsn)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if path == ":memory:" {
|
||||
db.SetMaxOpenConns(1)
|
||||
}
|
||||
s := &Store{db: db, path: path}
|
||||
if err := s.migrate(); err != nil {
|
||||
db.Close()
|
||||
return nil, fmt.Errorf("statistics database %s: %w", path, err)
|
||||
}
|
||||
return s, nil
|
||||
}
|
||||
|
||||
// Path returns the database file ("" in memory).
|
||||
func (s *Store) Path() string {
|
||||
if s.path == ":memory:" {
|
||||
return ""
|
||||
}
|
||||
return s.path
|
||||
}
|
||||
|
||||
// Close closes the database.
|
||||
func (s *Store) Close() error { return s.db.Close() }
|
||||
|
||||
func (s *Store) migrate() error {
|
||||
if _, err := s.db.Exec(`CREATE TABLE IF NOT EXISTS schema_version (version INTEGER NOT NULL)`); err != nil {
|
||||
return err
|
||||
}
|
||||
var v int
|
||||
err := s.db.QueryRow(`SELECT version FROM schema_version`).Scan(&v)
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
if _, err := s.db.Exec(`INSERT INTO schema_version (version) VALUES (0)`); err != nil {
|
||||
return err
|
||||
}
|
||||
} else if err != nil {
|
||||
return err
|
||||
}
|
||||
if v > len(schema) {
|
||||
return fmt.Errorf("schema version %d is newer than this Housecarl; upgrade Housecarl or delete the statistics database", v)
|
||||
}
|
||||
for i := v; i < len(schema); i++ {
|
||||
tx, err := s.db.Begin()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := tx.Exec(schema[i]); err != nil {
|
||||
tx.Rollback()
|
||||
return err
|
||||
}
|
||||
if _, err := tx.Exec(`UPDATE schema_version SET version = ?`, i+1); err != nil {
|
||||
tx.Rollback()
|
||||
return err
|
||||
}
|
||||
if err := tx.Commit(); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Batch collects writes that are committed together.
|
||||
type Batch struct {
|
||||
counters []counterRow
|
||||
tops []topsWrite
|
||||
meta map[string]string
|
||||
}
|
||||
|
||||
type counterRow struct {
|
||||
res string
|
||||
ts int64
|
||||
key string
|
||||
value int64
|
||||
}
|
||||
|
||||
type topsWrite struct {
|
||||
res, kind string
|
||||
ts int64
|
||||
entries []Entry
|
||||
replace bool // replace the (res, ts, kind) rows instead of adding
|
||||
keep int // after adding, keep only the largest entries (0: all)
|
||||
}
|
||||
|
||||
// Entry is a key and its count.
|
||||
type Entry struct {
|
||||
Key string `json:"key"`
|
||||
Count int64 `json:"count"`
|
||||
// Name is what the key stands for (a rule's label, a device's host
|
||||
// name), set by the API.
|
||||
Name string `json:"name,omitempty"`
|
||||
}
|
||||
|
||||
// AddCounters adds values to the counters of a bucket.
|
||||
func (b *Batch) AddCounters(res string, ts int64, values map[string]int64) {
|
||||
for k, v := range values {
|
||||
b.counters = append(b.counters, counterRow{res, ts, k, v})
|
||||
}
|
||||
}
|
||||
|
||||
// SetTops replaces a bucket's top list of a kind.
|
||||
func (b *Batch) SetTops(res string, ts int64, kind string, entries []Entry) {
|
||||
b.tops = append(b.tops, topsWrite{res: res, ts: ts, kind: kind, entries: entries, replace: true})
|
||||
}
|
||||
|
||||
// AddTops adds counts to a bucket's top list and keeps the keep largest.
|
||||
func (b *Batch) AddTops(res string, ts int64, kind string, entries []Entry, keep int) {
|
||||
b.tops = append(b.tops, topsWrite{res: res, ts: ts, kind: kind, entries: entries, keep: keep})
|
||||
}
|
||||
|
||||
// SetMeta stores a value that survives restarts.
|
||||
func (b *Batch) SetMeta(key, value string) {
|
||||
if b.meta == nil {
|
||||
b.meta = map[string]string{}
|
||||
}
|
||||
b.meta[key] = value
|
||||
}
|
||||
|
||||
// Commit writes the batch in one transaction.
|
||||
func (s *Store) Commit(ctx context.Context, b *Batch) error {
|
||||
tx, err := s.db.BeginTx(ctx, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer tx.Rollback()
|
||||
if len(b.counters) > 0 {
|
||||
st, err := tx.PrepareContext(ctx, `INSERT INTO counters (res, ts, key, value) VALUES (?, ?, ?, ?)
|
||||
ON CONFLICT (res, ts, key) DO UPDATE SET value = value + excluded.value`)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, c := range b.counters {
|
||||
if _, err := st.ExecContext(ctx, c.res, c.ts, c.key, c.value); err != nil {
|
||||
st.Close()
|
||||
return err
|
||||
}
|
||||
}
|
||||
st.Close()
|
||||
}
|
||||
for _, t := range b.tops {
|
||||
if t.replace {
|
||||
if _, err := tx.ExecContext(ctx, `DELETE FROM tops WHERE res = ? AND ts = ? AND kind = ?`, t.res, t.ts, t.kind); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
for _, e := range t.entries {
|
||||
if _, err := tx.ExecContext(ctx, `INSERT INTO tops (res, ts, kind, key, count) VALUES (?, ?, ?, ?, ?)
|
||||
ON CONFLICT (res, ts, kind, key) DO UPDATE SET count = count + excluded.count`, t.res, t.ts, t.kind, e.Key, e.Count); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if t.keep > 0 {
|
||||
if _, err := tx.ExecContext(ctx, `DELETE FROM tops WHERE res = ? AND ts = ? AND kind = ? AND key NOT IN (
|
||||
SELECT key FROM tops WHERE res = ? AND ts = ? AND kind = ? ORDER BY count DESC, key LIMIT ?)`,
|
||||
t.res, t.ts, t.kind, t.res, t.ts, t.kind, t.keep); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
for k, v := range b.meta {
|
||||
if _, err := tx.ExecContext(ctx, `INSERT INTO meta (key, value) VALUES (?, ?) ON CONFLICT (key) DO UPDATE SET value = excluded.value`, k, v); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return tx.Commit()
|
||||
}
|
||||
|
||||
// Meta returns a stored value ("" if unset).
|
||||
func (s *Store) Meta(ctx context.Context, key string) (string, error) {
|
||||
var v string
|
||||
err := s.db.QueryRowContext(ctx, `SELECT value FROM meta WHERE key = ?`, key).Scan(&v)
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return "", nil
|
||||
}
|
||||
return v, err
|
||||
}
|
||||
|
||||
// Prune deletes data of a resolution older than before (a Unix time).
|
||||
func (s *Store) Prune(ctx context.Context, res string, before int64) (int64, error) {
|
||||
var n int64
|
||||
for _, table := range []string{"counters", "tops"} {
|
||||
r, err := s.db.ExecContext(ctx, `DELETE FROM `+table+` WHERE res = ? AND ts < ?`, res, before)
|
||||
if err != nil {
|
||||
return n, err
|
||||
}
|
||||
c, _ := r.RowsAffected()
|
||||
n += c
|
||||
}
|
||||
return n, nil
|
||||
}
|
||||
|
||||
// TopsAt returns a bucket's top list of a kind.
|
||||
func (s *Store) TopsAt(ctx context.Context, res string, ts int64, kind string) ([]Entry, error) {
|
||||
rows, err := s.db.QueryContext(ctx, `SELECT key, count FROM tops WHERE res = ? AND ts = ? AND kind = ? ORDER BY count DESC, key`, res, ts, kind)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var out []Entry
|
||||
for rows.Next() {
|
||||
var e Entry
|
||||
if err := rows.Scan(&e.Key, &e.Count); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, e)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
// KindsAt returns the top-list kinds stored for one bucket.
|
||||
func (s *Store) KindsAt(ctx context.Context, res string, ts int64) ([]string, error) {
|
||||
rows, err := s.db.QueryContext(ctx, `SELECT DISTINCT kind FROM tops WHERE res = ? AND ts = ?`, res, ts)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var out []string
|
||||
for rows.Next() {
|
||||
var k string
|
||||
if err := rows.Scan(&k); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, k)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
@@ -0,0 +1,114 @@
|
||||
package store
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"time"
|
||||
|
||||
"housecarl/internal/model"
|
||||
)
|
||||
|
||||
const changesetCols = `id, user_id, username, via, description, status, ops, created_at, updated_at, history_id`
|
||||
|
||||
func scanChangeset(sc interface{ Scan(...any) error }) (*model.Changeset, error) {
|
||||
cs := &model.Changeset{}
|
||||
var uid, hid sql.NullInt64
|
||||
var ops, created, updated string
|
||||
if err := sc.Scan(&cs.ID, &uid, &cs.Username, &cs.Via, &cs.Description, &cs.Status, &ops, &created, &updated, &hid); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
cs.UserID, cs.HistoryID = uid.Int64, hid.Int64
|
||||
cs.CreatedAt, cs.UpdatedAt = parseTS(created), parseTS(updated)
|
||||
if err := json.Unmarshal([]byte(ops), &cs.Ops); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if cs.Ops == nil {
|
||||
cs.Ops = []model.Op{}
|
||||
}
|
||||
return cs, nil
|
||||
}
|
||||
|
||||
// CreateChangeset saves a new changeset and sets its ID.
|
||||
func (s *Store) CreateChangeset(ctx context.Context, cs *model.Changeset) error {
|
||||
now := time.Now()
|
||||
cs.CreatedAt, cs.UpdatedAt = now, now
|
||||
if cs.Ops == nil {
|
||||
cs.Ops = []model.Op{}
|
||||
}
|
||||
ops, _ := json.Marshal(cs.Ops)
|
||||
res, err := s.db.ExecContext(ctx, `INSERT INTO changesets (user_id, username, via, description, status, ops, created_at, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?)`, nullID(cs.UserID), cs.Username, cs.Via, cs.Description, cs.Status, string(ops), ts(now), ts(now))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
cs.ID, err = res.LastInsertId()
|
||||
return err
|
||||
}
|
||||
|
||||
// GetChangeset loads one changeset.
|
||||
func (s *Store) GetChangeset(ctx context.Context, id int64) (*model.Changeset, error) {
|
||||
cs, err := scanChangeset(s.db.QueryRowContext(ctx, `SELECT `+changesetCols+` FROM changesets WHERE id = ?`, id))
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return nil, ErrNotFound
|
||||
}
|
||||
return cs, err
|
||||
}
|
||||
|
||||
// ListChangesets lists changesets, newest first: of a status ("" any),
|
||||
// and of a user (0 anyone's).
|
||||
func (s *Store) ListChangesets(ctx context.Context, status string, userID int64, limit int) ([]*model.Changeset, error) {
|
||||
if limit <= 0 || limit > 500 {
|
||||
limit = 100
|
||||
}
|
||||
q := `SELECT ` + changesetCols + ` FROM changesets WHERE 1=1`
|
||||
var args []any
|
||||
if status != "" {
|
||||
q += ` AND status = ?`
|
||||
args = append(args, status)
|
||||
}
|
||||
if userID != 0 {
|
||||
q += ` AND user_id = ?`
|
||||
args = append(args, userID)
|
||||
}
|
||||
q += ` ORDER BY id DESC LIMIT ?`
|
||||
args = append(args, limit)
|
||||
rows, err := s.db.QueryContext(ctx, q, args...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
out := []*model.Changeset{}
|
||||
for rows.Next() {
|
||||
cs, err := scanChangeset(rows)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, cs)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
// UpdateChangeset saves a changeset's description, status, operations and
|
||||
// history entry.
|
||||
func (s *Store) UpdateChangeset(ctx context.Context, cs *model.Changeset) error {
|
||||
cs.UpdatedAt = time.Now()
|
||||
ops, _ := json.Marshal(cs.Ops)
|
||||
res, err := s.db.ExecContext(ctx, `UPDATE changesets SET description = ?, status = ?, ops = ?, updated_at = ?, history_id = ? WHERE id = ?`,
|
||||
cs.Description, cs.Status, string(ops), ts(cs.UpdatedAt), nullID(cs.HistoryID), cs.ID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if n, _ := res.RowsAffected(); n == 0 {
|
||||
return ErrNotFound
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func nullID(id int64) any {
|
||||
if id == 0 {
|
||||
return nil
|
||||
}
|
||||
return id
|
||||
}
|
||||
@@ -163,6 +163,22 @@ CREATE TABLE audit (
|
||||
detail TEXT NOT NULL DEFAULT ''
|
||||
);
|
||||
CREATE INDEX audit_time ON audit (time);
|
||||
`,
|
||||
// 2: changesets, staged changes to review and apply together
|
||||
`
|
||||
CREATE TABLE changesets (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
user_id INTEGER,
|
||||
username TEXT NOT NULL,
|
||||
via TEXT NOT NULL,
|
||||
description TEXT NOT NULL DEFAULT '',
|
||||
status TEXT NOT NULL,
|
||||
ops TEXT NOT NULL,
|
||||
created_at TEXT NOT NULL,
|
||||
updated_at TEXT NOT NULL,
|
||||
history_id INTEGER
|
||||
);
|
||||
CREATE INDEX changesets_status ON changesets (status);
|
||||
`,
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,258 @@
|
||||
// Package trace follows packets through the whole ruleset, every table's
|
||||
// chains, with nftables' own tracing: a table of its own (housecarl-trace,
|
||||
// hooked before everything else) marks the packets a filter picks
|
||||
// (meta nftrace set 1), and `nft monitor trace` reports each chain and
|
||||
// rule they pass, and what decided their fate. Packets are counted per
|
||||
// trace id: one packet through the hooks it crosses.
|
||||
package trace
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"net/netip"
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"housecarl/internal/api"
|
||||
)
|
||||
|
||||
// Table is the tracing table's name (family inet).
|
||||
const Table = "housecarl-trace"
|
||||
|
||||
// Script builds the tracing table for a filter: packets from or to the
|
||||
// address, on the port (either way), of the protocol. A limit keeps a
|
||||
// flood from swamping the trace.
|
||||
func Script(in api.TraceInput) (string, error) {
|
||||
var addr netip.Prefix
|
||||
if in.Address != "" {
|
||||
p, err := netip.ParsePrefix(in.Address)
|
||||
if err != nil {
|
||||
a, err2 := netip.ParseAddr(in.Address)
|
||||
if err2 != nil {
|
||||
return "", fmt.Errorf("address %q: give an address or a prefix (192.168.1.50, 2001:db8::/64)", in.Address)
|
||||
}
|
||||
p = netip.PrefixFrom(a, a.BitLen())
|
||||
}
|
||||
addr = p.Masked()
|
||||
}
|
||||
if in.Port < 0 || in.Port > 65535 {
|
||||
return "", fmt.Errorf("port %d: 1 to 65535", in.Port)
|
||||
}
|
||||
proto := strings.ToLower(in.Protocol)
|
||||
switch proto {
|
||||
case "", "tcp", "udp", "sctp", "icmp", "icmpv6":
|
||||
default:
|
||||
return "", fmt.Errorf("protocol %q: tcp, udp, sctp, icmp, icmpv6, or nothing for any", in.Protocol)
|
||||
}
|
||||
if !addr.IsValid() && in.Port == 0 {
|
||||
return "", fmt.Errorf("say what to trace: an address, a port, or both")
|
||||
}
|
||||
if in.Port != 0 && (proto == "icmp" || proto == "icmpv6") {
|
||||
return "", fmt.Errorf("ICMP has no ports")
|
||||
}
|
||||
var l4 string
|
||||
switch {
|
||||
case proto != "":
|
||||
l4 = "meta l4proto " + proto
|
||||
case in.Port != 0:
|
||||
l4 = "meta l4proto { tcp, udp, sctp }"
|
||||
}
|
||||
addrs := []string{""}
|
||||
if addr.IsValid() {
|
||||
ip := "ip"
|
||||
if addr.Addr().Is6() {
|
||||
ip = "ip6"
|
||||
}
|
||||
addrs = []string{ip + " saddr " + addr.String(), ip + " daddr " + addr.String()}
|
||||
}
|
||||
ports := []string{""}
|
||||
if in.Port != 0 {
|
||||
ports = []string{fmt.Sprintf("th dport %d", in.Port), fmt.Sprintf("th sport %d", in.Port)}
|
||||
}
|
||||
var rules []string
|
||||
for _, a := range addrs {
|
||||
for _, p := range ports {
|
||||
parts := []string{}
|
||||
for _, x := range []string{a, l4, p, "limit rate 20/second", "meta nftrace set 1"} {
|
||||
if x != "" {
|
||||
parts = append(parts, x)
|
||||
}
|
||||
}
|
||||
rules = append(rules, "\t\t"+strings.Join(parts, " "))
|
||||
}
|
||||
}
|
||||
body := strings.Join(rules, "\n")
|
||||
return fmt.Sprintf(`table inet %[1]s {}
|
||||
delete table inet %[1]s
|
||||
table inet %[1]s {
|
||||
chain pre {
|
||||
type filter hook prerouting priority -301; policy accept;
|
||||
%[2]s
|
||||
}
|
||||
chain out {
|
||||
type filter hook output priority -301; policy accept;
|
||||
%[2]s
|
||||
}
|
||||
}
|
||||
`, Table, body), nil
|
||||
}
|
||||
|
||||
// ---- reading `nft monitor trace`
|
||||
|
||||
// "trace id 38ff3662 inet housecarl input rule … (verdict accept)"
|
||||
var lineRe = regexp.MustCompile(`^trace id (\S+) (\S+) (\S+) (\S+) (packet: (.*)|rule (.*) \(verdict ([^)]*)\)|policy (\S+)|(.*))$`)
|
||||
|
||||
var (
|
||||
fieldRe = regexp.MustCompile(`\b(iif|oif) "([^"]*)"`)
|
||||
addrRe = regexp.MustCompile(`\b(ip6?) (saddr|daddr) (\S+)`)
|
||||
portRe = regexp.MustCompile(`\b(tcp|udp|sctp) (sport|dport) (\d+)`)
|
||||
flagsRe = regexp.MustCompile(`\btcp flags == (\S+)`)
|
||||
icmpRe = regexp.MustCompile(`\b(icmp|icmpv6) type (\S+)`)
|
||||
protoRe = regexp.MustCompile(`\bmeta l4proto (\S+)|\bip protocol (\S+)|\bip6 nexthdr (\S+)`)
|
||||
markRe = regexp.MustCompile(`comment "hc:(\S+)`)
|
||||
)
|
||||
|
||||
// Tracer collects packets from trace lines.
|
||||
type Tracer struct {
|
||||
// Max is how many packets it keeps (0: 200).
|
||||
Max int
|
||||
Now func() time.Time
|
||||
packets []*api.TracePacket
|
||||
byID map[string]*api.TracePacket
|
||||
// seen are the chains a packet entered: the kernel reuses trace ids
|
||||
// (they come from the packet's buffer), so entering one again means
|
||||
// another packet.
|
||||
seen map[*api.TracePacket]map[string]bool
|
||||
}
|
||||
|
||||
// Packets returns what was traced so far, oldest first.
|
||||
func (t *Tracer) Packets() []api.TracePacket {
|
||||
out := make([]api.TracePacket, 0, len(t.packets))
|
||||
for _, p := range t.packets {
|
||||
out = append(out, *p)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// Full reports whether it keeps no more packets.
|
||||
func (t *Tracer) Full() bool {
|
||||
max := t.Max
|
||||
if max == 0 {
|
||||
max = 200
|
||||
}
|
||||
return len(t.packets) >= max
|
||||
}
|
||||
|
||||
// Line reads one line of `nft monitor trace`. The tracing table's own
|
||||
// steps are left out.
|
||||
func (t *Tracer) Line(line string) {
|
||||
m := lineRe.FindStringSubmatch(strings.TrimSpace(line))
|
||||
if m == nil {
|
||||
return
|
||||
}
|
||||
id, table, chain := m[1], m[2]+" "+m[3], m[4]
|
||||
if m[3] == Table {
|
||||
return
|
||||
}
|
||||
if t.byID == nil {
|
||||
t.byID, t.seen = map[string]*api.TracePacket{}, map[*api.TracePacket]map[string]bool{}
|
||||
}
|
||||
p := t.byID[id]
|
||||
isPacket := m[6] != "" || strings.HasPrefix(m[5], "packet:")
|
||||
if p != nil && isPacket && t.seen[p][table+" "+chain] {
|
||||
p = nil // the same id, another packet
|
||||
}
|
||||
if p == nil {
|
||||
if t.Full() {
|
||||
return
|
||||
}
|
||||
now := time.Now
|
||||
if t.Now != nil {
|
||||
now = t.Now
|
||||
}
|
||||
p = &api.TracePacket{ID: id, Time: now(), Verdict: "accepted"}
|
||||
t.byID[id] = p
|
||||
t.seen[p] = map[string]bool{}
|
||||
t.packets = append(t.packets, p)
|
||||
}
|
||||
switch {
|
||||
case isPacket:
|
||||
t.seen[p][table+" "+chain] = true
|
||||
readPacket(p, m[6])
|
||||
case m[7] != "":
|
||||
step := api.TraceStep{Table: table, Chain: chain, Kind: "rule", Rule: m[7], Verdict: m[8]}
|
||||
if mk := markRe.FindStringSubmatch(m[7]); mk != nil {
|
||||
step.Mark = strings.TrimSuffix(mk[1], `"`)
|
||||
}
|
||||
p.Steps = append(p.Steps, step)
|
||||
decide(p, step)
|
||||
case m[9] != "":
|
||||
step := api.TraceStep{Table: table, Chain: chain, Kind: "policy", Verdict: m[9]}
|
||||
p.Steps = append(p.Steps, step)
|
||||
decide(p, step)
|
||||
}
|
||||
}
|
||||
|
||||
// decide notes a packet's fate: the first drop (or reject) ends it.
|
||||
func decide(p *api.TracePacket, s api.TraceStep) {
|
||||
if p.Verdict == "dropped" {
|
||||
return
|
||||
}
|
||||
switch strings.Fields(s.Verdict + " ")[0] {
|
||||
case "drop", "reject":
|
||||
p.Verdict, p.DecidedBy = "dropped", s
|
||||
case "accept":
|
||||
// What let it through is a filter chain's accept: Housecarl's
|
||||
// input, forward or output (NAT chains accept everything they
|
||||
// see). Another table's accept counts when Housecarl's says
|
||||
// nothing.
|
||||
filter := s.Table == "inet housecarl" && (s.Chain == "input" || s.Chain == "forward" || s.Chain == "output")
|
||||
if filter || p.DecidedBy.Verdict == "" && s.Table != "inet housecarl" {
|
||||
p.DecidedBy = s
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// readPacket fills in a packet's headers from its "packet:" line (once:
|
||||
// the first hook's view, before any NAT).
|
||||
func readPacket(p *api.TracePacket, f string) {
|
||||
for _, m := range fieldRe.FindAllStringSubmatch(f, -1) {
|
||||
if m[1] == "iif" && p.In == "" {
|
||||
p.In = m[2]
|
||||
}
|
||||
if m[1] == "oif" && p.Out == "" {
|
||||
p.Out = m[2]
|
||||
}
|
||||
}
|
||||
if p.Source != "" {
|
||||
return
|
||||
}
|
||||
for _, m := range addrRe.FindAllStringSubmatch(f, -1) {
|
||||
if m[2] == "saddr" {
|
||||
p.Source = m[3]
|
||||
} else {
|
||||
p.Destination = m[3]
|
||||
}
|
||||
}
|
||||
for _, m := range portRe.FindAllStringSubmatch(f, -1) {
|
||||
p.Protocol = m[1]
|
||||
n, _ := strconv.Atoi(m[3])
|
||||
if m[2] == "sport" {
|
||||
p.SourcePort = n
|
||||
} else {
|
||||
p.DestinationPort = n
|
||||
}
|
||||
}
|
||||
if m := flagsRe.FindStringSubmatch(f); m != nil {
|
||||
p.TCPFlags = m[1]
|
||||
}
|
||||
if m := icmpRe.FindStringSubmatch(f); m != nil {
|
||||
p.Protocol, p.ICMPType = m[1], m[2]
|
||||
}
|
||||
if p.Protocol == "" {
|
||||
if m := protoRe.FindStringSubmatch(f); m != nil {
|
||||
p.Protocol = strings.TrimSpace(m[1] + m[2] + m[3])
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,89 @@
|
||||
package trace
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"housecarl/internal/api"
|
||||
)
|
||||
|
||||
// Lines nft 1.1.3's `nft monitor trace` printed for a connection to a
|
||||
// blocked port, and for one that loopback accepted.
|
||||
const sample = `trace id 38ff3662 inet housecarl-trace pre packet: iif "wan" ip saddr 198.51.100.7 ip daddr 203.0.113.10 ip dscp cs0 ip ecn not-ect ip ttl 64 ip id 44978 ip protocol 6 ip length 60 tcp sport 43168 tcp dport 9999 tcp flags == syn tcp window 65495
|
||||
trace id 38ff3662 inet housecarl-trace pre rule ip saddr 198.51.100.7 meta nftrace set 1 (verdict continue)
|
||||
trace id 38ff3662 inet housecarl-trace pre policy accept
|
||||
trace id 38ff3662 inet housecarl input packet: iif "wan" ip saddr 198.51.100.7 ip daddr 203.0.113.10 ip dscp cs0 ip ecn not-ect ip ttl 64 ip id 44978 ip protocol 6 ip length 60 tcp sport 43168 tcp dport 9999 tcp flags == syn tcp window 65495
|
||||
trace id 38ff3662 inet housecarl input rule ct state established,related counter packets 0 bytes 0 accept comment "hc:b-established" (verdict continue)
|
||||
trace id 38ff3662 inet housecarl input rule tcp dport 9999 counter packets 1 bytes 60 drop comment "hc:r12 block 9999" (verdict drop)
|
||||
trace id a232d9a0 inet housecarl output packet: oif "lo" ip saddr 127.0.0.1 ip daddr 127.0.0.1 ip protocol 6 tcp sport 43168 tcp dport 22 tcp flags == syn
|
||||
trace id a232d9a0 inet housecarl output policy accept
|
||||
trace id a232d9a0 ip other input rule ip saddr 127.0.0.1 accept (verdict accept)
|
||||
trace id 77 inet housecarl forward packet: iif "lan" oif "wan" ip saddr 192.168.1.20 ip daddr 198.51.100.10 tcp sport 3500 tcp dport 7000 tcp flags == syn
|
||||
trace id 77 inet housecarl forward rule ip saddr @trusted_v4 counter accept comment "hc:r40 trusted" (verdict accept)
|
||||
trace id 77 inet housecarl srcnat rule oifname "wan" counter masquerade comment "hc:s91" (verdict accept)`
|
||||
|
||||
func TestTracer(t *testing.T) {
|
||||
var tr Tracer
|
||||
for _, l := range strings.Split(sample, "\n") {
|
||||
tr.Line(l)
|
||||
}
|
||||
ps := tr.Packets()
|
||||
if len(ps) != 3 {
|
||||
t.Fatalf("packets: %+v", ps)
|
||||
}
|
||||
p := ps[0]
|
||||
if p.In != "wan" || p.Source != "198.51.100.7" || p.DestinationPort != 9999 || p.Protocol != "tcp" || p.TCPFlags != "syn" {
|
||||
t.Errorf("headers: %+v", p)
|
||||
}
|
||||
if len(p.Steps) != 2 || p.Verdict != "dropped" || p.DecidedBy.Mark != "r12" || p.DecidedBy.Chain != "input" {
|
||||
t.Errorf("steps: %+v", p)
|
||||
}
|
||||
if q := ps[1]; q.Verdict != "accepted" || q.Out != "lo" || len(q.Steps) != 2 || q.DecidedBy.Kind != "policy" {
|
||||
t.Errorf("second: %+v", q)
|
||||
}
|
||||
// Accepted by the forward rule, not by source NAT, which came after.
|
||||
if q := ps[2]; q.Verdict != "accepted" || q.DecidedBy.Mark != "r40" {
|
||||
t.Errorf("third: decided by %+v", q.DecidedBy)
|
||||
}
|
||||
}
|
||||
|
||||
// The kernel reuses trace ids: a connection's next packet can come with
|
||||
// the same one. Entering a chain again starts another packet.
|
||||
func TestReusedID(t *testing.T) {
|
||||
var tr Tracer
|
||||
for _, l := range strings.Split(`trace id 5 inet housecarl output packet: oif "lo" ip saddr 127.0.0.1 ip daddr 127.0.0.1 tcp sport 49078 tcp dport 8070 tcp flags == syn
|
||||
trace id 5 inet housecarl output policy accept
|
||||
trace id 5 inet housecarl srcnat packet: oif "lo" ip saddr 127.0.0.1 ip daddr 127.0.0.1 tcp sport 49078 tcp dport 8070 tcp flags == syn
|
||||
trace id 5 inet housecarl srcnat policy accept
|
||||
trace id 5 inet housecarl output packet: oif "lo" ip saddr 127.0.0.1 ip daddr 127.0.0.1 tcp sport 49078 tcp dport 8070 tcp flags == ack
|
||||
trace id 5 inet housecarl output policy accept`, "\n") {
|
||||
tr.Line(l)
|
||||
}
|
||||
ps := tr.Packets()
|
||||
if len(ps) != 2 || len(ps[0].Steps) != 2 || len(ps[1].Steps) != 1 || ps[1].TCPFlags != "ack" {
|
||||
t.Errorf("packets: %+v", ps)
|
||||
}
|
||||
}
|
||||
|
||||
func TestScript(t *testing.T) {
|
||||
s, err := Script(api.TraceInput{Address: "192.168.1.50", Port: 443})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, want := range []string{
|
||||
"table inet housecarl-trace {}\ndelete table inet housecarl-trace\n",
|
||||
"type filter hook prerouting priority -301; policy accept;",
|
||||
"ip saddr 192.168.1.50/32 meta l4proto { tcp, udp, sctp } th dport 443 limit rate 20/second meta nftrace set 1",
|
||||
"ip daddr 192.168.1.50/32 meta l4proto { tcp, udp, sctp } th sport 443 limit rate 20/second meta nftrace set 1",
|
||||
} {
|
||||
if !strings.Contains(s, want) {
|
||||
t.Errorf("missing %q in\n%s", want, s)
|
||||
}
|
||||
}
|
||||
if _, err := Script(api.TraceInput{}); err == nil {
|
||||
t.Error("tracing everything")
|
||||
}
|
||||
if _, err := Script(api.TraceInput{Port: 22, Protocol: "icmp"}); err == nil {
|
||||
t.Error("ICMP with a port")
|
||||
}
|
||||
}
|
||||
@@ -85,6 +85,53 @@ func Check(st *model.State, env *Env) []Problem {
|
||||
return c.out
|
||||
}
|
||||
|
||||
// rawIgnores names what a raw rule has besides its text, which the
|
||||
// ruleset wouldn't use (the action too is the text's; it isn't checked).
|
||||
func rawIgnores(r *model.Rule) []string {
|
||||
m := r.Match
|
||||
var out []string
|
||||
for _, x := range []struct {
|
||||
name string
|
||||
set bool
|
||||
}{
|
||||
{"zones", len(m.InZones) > 0 || len(m.OutZones) > 0},
|
||||
{"addresses", len(m.Sources) > 0 || len(m.Destinations) > 0},
|
||||
{"services", len(m.Services) > 0},
|
||||
{"family", m.Family != ""},
|
||||
{"connection state", len(r.CtState) > 0 || len(r.CtStatus) > 0},
|
||||
{"time window", len(r.Days) > 0 || r.HourFrom != "" || r.HourTo != ""},
|
||||
{"rate limit", r.Limit != nil},
|
||||
{"connection limit", r.ConnLimit != 0},
|
||||
{"quota", r.Quota != nil},
|
||||
{"address grouping", r.GroupV4 != 0 || r.GroupV6 != 0},
|
||||
{"log", r.Log || r.LogPrefix != ""},
|
||||
{"reject with", r.RejectWith != ""},
|
||||
{"every packet", r.EveryPacket},
|
||||
} {
|
||||
if x.set {
|
||||
out = append(out, x.name)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// maxByteRate is the largest byte rate per period nftables (1.1.3) can
|
||||
// hold: the kernel's token bucket multiplies the rate by the period.
|
||||
var maxByteRate = map[string]int64{"second": 4294967294, "minute": 307445734, "hour": 5124095, "day": 213503}
|
||||
|
||||
// byteSize writes bytes the way nftables reads them: "1 mbytes".
|
||||
func byteSize(n int64) string {
|
||||
for _, u := range []string{"mbytes", "kbytes"} {
|
||||
if n >= model.ByteUnits[u] && n%model.ByteUnits[u] == 0 {
|
||||
return fmt.Sprintf("%d %s", n/model.ByteUnits[u], u)
|
||||
}
|
||||
}
|
||||
if n >= 10*model.ByteUnits["kbytes"] {
|
||||
return fmt.Sprintf("%d kbytes", n/model.ByteUnits["kbytes"])
|
||||
}
|
||||
return fmt.Sprintf("%d bytes", n)
|
||||
}
|
||||
|
||||
var nameRe = regexp.MustCompile(`^[A-Za-z][A-Za-z0-9_-]{0,47}$`)
|
||||
|
||||
// ifaceRe: Linux interface names (15 bytes), with an optional trailing *
|
||||
@@ -138,6 +185,12 @@ func (c *checker) settings() {
|
||||
}
|
||||
}
|
||||
}
|
||||
if s.StatsPerDevice && len(c.st.ExternalInterfaces()) == 0 {
|
||||
c.add(Warning, "stats_no_external", "settings", "traffic per device counts what goes to and comes from external zones, and no zone is external (with interfaces)")
|
||||
}
|
||||
if s.StatsPerDevice && len(s.FlowtableDevices) > 0 {
|
||||
c.add(Warning, "stats_flowtable", "settings", "the flowtable's fast path skips the forward chain once a connection is established, so traffic per device only counts each connection's first packets; turn the flowtable off for full figures")
|
||||
}
|
||||
if p := s.ProbationSeconds; p != 0 && (p < 10 || p > 3600) {
|
||||
c.add(Error, "probation_range", "settings", "the probation is %d seconds; use 10 to 3600", p)
|
||||
}
|
||||
@@ -170,8 +223,8 @@ func (c *checker) zones() {
|
||||
}
|
||||
}
|
||||
}
|
||||
if len(c.st.Zones) > 0 && !external {
|
||||
c.add(Warning, "no_external_zone", "zones", "no zone is external: antispoofing, MSS clamping and the DHCPv6 client rule need one, and port forwards without public addresses")
|
||||
if !c.st.Empty() && !external {
|
||||
c.add(Warning, "no_external_zone", "zones", "no zone is external: antispoofing, MSS clamping and the DHCPv6 client rule need one, and port forwards without public addresses; and if the saved ruleset fails to load at boot, the fallback can't tell which interface faces the internet, and lets nothing but loopback through")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -180,11 +233,19 @@ func (c *checker) sets() {
|
||||
for _, s := range c.st.Sets {
|
||||
obj := "set " + s.Name
|
||||
c.name("set", obj, s.Name, seen)
|
||||
for _, pre := range []string{"meter_", "conn_", "quota_", "stats_"} {
|
||||
if strings.HasPrefix(strings.ToLower(s.Name), pre) {
|
||||
c.add(Error, "set_name_reserved", obj, "set names starting with %s are Housecarl's own (rate limits, quotas, statistics); choose another name", pre)
|
||||
}
|
||||
}
|
||||
for _, e := range s.Entries {
|
||||
if _, err := model.ParseAddress(e); err != nil {
|
||||
c.add(Error, "set_entry", obj, "%v", err)
|
||||
}
|
||||
}
|
||||
if len(s.Entries) > 0 && s.Dynamic {
|
||||
c.add(Error, "set_dynamic_entries", obj, "a dynamic set gets its entries at run time (hc set entries add, the Entries button), each with its timeout; leave out the configured ones, or make it a set that isn't dynamic")
|
||||
}
|
||||
if len(s.Entries) == 0 && !s.Dynamic {
|
||||
c.add(Warning, "set_empty", obj, "the set is empty: rules using it match nothing")
|
||||
}
|
||||
@@ -314,6 +375,9 @@ func (c *checker) rule(r *model.Rule) {
|
||||
if strings.ContainsAny(r.Raw, ";\n\r") {
|
||||
c.add(Error, "raw_rule", obj, "a raw rule is one nftables rule on one line, without ;")
|
||||
}
|
||||
if ignored := rawIgnores(r); len(ignored) > 0 {
|
||||
c.add(Error, "raw_conflict", obj, "a raw rule is used as written, so its %s would be ignored: write them in the nftables rule, or leave them out", strings.Join(ignored, ", "))
|
||||
}
|
||||
return
|
||||
}
|
||||
in := r.Chain != model.ChainOutput
|
||||
@@ -348,13 +412,52 @@ func (c *checker) rule(r *model.Rule) {
|
||||
c.add(Error, "rule_hours", obj, "write hours as HH:MM (08:00, 17:30)")
|
||||
}
|
||||
if l := r.Limit; l != nil {
|
||||
if l.Rate < 1 || !limitPer[l.Per] || l.Burst < 0 {
|
||||
c.add(Error, "rule_limit", obj, "a rate is a number of packets per second, minute, hour or day, with an optional burst")
|
||||
_, byteUnit := model.ByteUnits[l.Unit]
|
||||
switch {
|
||||
case l.Rate < 1 || !limitPer[l.Per] || l.Burst < 0:
|
||||
c.add(Error, "rule_limit", obj, "a rate is a number of packets (or bytes, kbytes, mbytes) per second, minute, hour or day, with an optional burst")
|
||||
case l.Unit != "" && l.Unit != "packets" && (!byteUnit || l.Unit == "gbytes"):
|
||||
c.add(Error, "rule_limit_unit", obj, "the rate's unit is %q; use packets, bytes, kbytes or mbytes", l.Unit)
|
||||
case byteUnit && int64(l.Rate)*model.ByteUnits[l.Unit] > maxByteRate[l.Per]:
|
||||
perSecond := int64(l.Rate) * model.ByteUnits[l.Unit] / map[string]int64{"second": 1, "minute": 60, "hour": 3600, "day": 86400}[l.Per]
|
||||
c.add(Error, "rule_limit_bytes", obj, "nftables can't hold a byte rate of %d %s per %s (at most %s per %s); give it per second: %s/second",
|
||||
l.Rate, l.Unit, l.Per, byteSize(maxByteRate[l.Per]), l.Per, byteSize(perSecond))
|
||||
}
|
||||
if l.PerSource && l.PerDestination {
|
||||
c.add(Error, "rule_limit_key", obj, "a limit is per source or per destination, not both")
|
||||
}
|
||||
}
|
||||
if r.ConnLimit < 0 {
|
||||
c.add(Error, "rule_conn_limit", obj, "the connection limit must be 1 or more")
|
||||
}
|
||||
if q := r.Quota; q != nil {
|
||||
if q.Bytes < 1 {
|
||||
c.add(Error, "rule_quota", obj, "a quota is an amount of bytes, 1 or more")
|
||||
}
|
||||
if q.PerSource && q.PerDestination {
|
||||
c.add(Error, "rule_quota_key", obj, "a quota is per source or per destination, not both")
|
||||
}
|
||||
switch {
|
||||
case q.Reset != "" && q.Reset != "daily" && q.Reset != "weekly" && q.Reset != "monthly":
|
||||
c.add(Error, "rule_quota_reset", obj, "the quota resets %q; use daily, weekly, monthly, or nothing (only by hand)", q.Reset)
|
||||
case q.Day != 0 && q.Reset != "monthly":
|
||||
c.add(Error, "rule_quota_reset", obj, "a day of the month is only for a monthly reset")
|
||||
case q.Day < 0 || q.Day > 31:
|
||||
c.add(Error, "rule_quota_reset", obj, "the day of the month is 1 to 31 (a later day than a month has means its last day)")
|
||||
}
|
||||
}
|
||||
switch {
|
||||
case r.CountsTraffic() && r.Action == model.ActionAccept:
|
||||
c.add(Error, "rule_traffic_accept", obj, "a rule with a quota or a byte rate counts every packet, before established connections are accepted, so it can only drop, reject or log; let the traffic in with a rule of its own")
|
||||
case r.ChecksEveryPacket() && r.Action == model.ActionAccept:
|
||||
c.add(Error, "rule_every_packet_accept", obj, "a rule checked against every packet comes before established connections are accepted, so accepting there would skip every later rule; it can drop, reject or log")
|
||||
}
|
||||
if r.ChecksEveryPacket() && r.Chain == model.ChainForward && len(c.st.Settings.FlowtableDevices) > 0 && !r.Disabled {
|
||||
c.add(Warning, "flowtable_traffic", obj, "the flowtable's fast path skips the forward chain once a connection is established, so this rule only sees each connection's first packets; turn the flowtable off (settings) for it to see all of them")
|
||||
}
|
||||
if r.GroupV4 < 0 || r.GroupV4 > 32 || r.GroupV6 < 0 || r.GroupV6 > 128 {
|
||||
c.add(Error, "rule_group", obj, "group addresses by a prefix length: 1 to 32 for IPv4, 1 to 128 for IPv6")
|
||||
}
|
||||
if r.RejectWith != "" {
|
||||
if r.RejectWith != "tcp-reset" {
|
||||
c.add(Error, "reject_with", obj, "reject with is %q; use tcp-reset, or nothing for an ICMP error", r.RejectWith)
|
||||
|
||||
@@ -65,7 +65,17 @@ func TestCodes(t *testing.T) {
|
||||
}},
|
||||
{"set_entry", func(st *model.State) { st.Sets[0].Entries = []string{"10.0.10.5/24"} }},
|
||||
{"set_empty", func(st *model.State) { st.Sets[0].Entries = nil }},
|
||||
{"set_name_reserved", func(st *model.State) { st.Sets[0].Name = "stats_up" }},
|
||||
{"stats_flowtable", func(st *model.State) {
|
||||
st.Settings.StatsPerDevice = true
|
||||
st.Settings.FlowtableDevices = []string{"wan", "eth1"}
|
||||
}},
|
||||
{"stats_no_external", func(st *model.State) {
|
||||
st.Settings.StatsPerDevice = true
|
||||
st.Zones[0].External = false
|
||||
}},
|
||||
{"set_timeout", func(st *model.State) { st.Sets[0].Timeout = 60 }},
|
||||
{"set_dynamic_entries", func(st *model.State) { st.Sets[0].Dynamic = true }},
|
||||
{"service_empty", func(st *model.State) { st.Services[0].Entries = nil }},
|
||||
{"service_entry", func(st *model.State) { st.Services[0].Entries = []string{"tcp/99999"} }},
|
||||
{"unknown_zone", func(st *model.State) { st.Rules[1].InZones = []string{"dmz"} }},
|
||||
@@ -83,6 +93,7 @@ func TestCodes(t *testing.T) {
|
||||
{"rule_chain", func(st *model.State) { rule(st).Chain = "prerouting" }},
|
||||
{"rule_action", func(st *model.State) { rule(st).Action = "allow" }},
|
||||
{"raw_rule", func(st *model.State) { rule(st).Raw = "tcp dport 22 accept; udp dport 53 accept" }},
|
||||
{"raw_conflict", func(st *model.State) { rule(st).Raw = "tcp dport 22 accept" }}, // it still has sources and services
|
||||
{"early_conntrack", func(st *model.State) { rule(st).Chain = model.ChainEarly; rule(st).CtState = []string{"new"} }},
|
||||
{"early_reject", func(st *model.State) { rule(st).Chain = model.ChainEarly; rule(st).Action = model.ActionReject }},
|
||||
{"ct_state", func(st *model.State) { rule(st).CtState = []string{"fresh"} }},
|
||||
@@ -91,6 +102,22 @@ func TestCodes(t *testing.T) {
|
||||
{"rule_hours", func(st *model.State) { rule(st).HourFrom = "08:00" }},
|
||||
{"rule_limit", func(st *model.State) { rule(st).Limit = &model.Limit{Rate: 10, Per: "week"} }},
|
||||
{"rule_conn_limit", func(st *model.State) { rule(st).ConnLimit = -1 }},
|
||||
{"rule_limit_unit", func(st *model.State) { rule(st).Limit = &model.Limit{Rate: 1, Per: "second", Unit: "gbytes"} }},
|
||||
{"rule_limit_bytes", func(st *model.State) { rule(st).Limit = &model.Limit{Rate: 100, Per: "hour", Unit: "mbytes"} }},
|
||||
{"rule_limit_key", func(st *model.State) {
|
||||
rule(st).Limit = &model.Limit{Rate: 1, Per: "second", PerSource: true, PerDestination: true}
|
||||
}},
|
||||
{"rule_quota", func(st *model.State) { rule(st).Quota = &model.Quota{} }},
|
||||
{"rule_quota_key", func(st *model.State) { rule(st).Quota = &model.Quota{Bytes: 1, PerSource: true, PerDestination: true} }},
|
||||
{"rule_quota_reset", func(st *model.State) { rule(st).Quota = &model.Quota{Bytes: 1, Reset: "yearly"} }},
|
||||
{"rule_quota_reset", func(st *model.State) { rule(st).Quota = &model.Quota{Bytes: 1, Reset: "daily", Day: 3} }},
|
||||
{"rule_group", func(st *model.State) { rule(st).GroupV6 = 129 }},
|
||||
{"rule_traffic_accept", func(st *model.State) { rule(st).Quota = &model.Quota{Bytes: 1 << 20} }},
|
||||
{"rule_every_packet_accept", func(st *model.State) { rule(st).EveryPacket = true }},
|
||||
{"flowtable_traffic", func(st *model.State) {
|
||||
st.Settings.FlowtableDevices = []string{"wan"}
|
||||
st.Rules[1].Action, st.Rules[1].Quota = model.ActionDrop, &model.Quota{Bytes: 1 << 20, Over: true}
|
||||
}},
|
||||
{"reject_with", func(st *model.State) { rule(st).RejectWith = "tcp-reset" }},
|
||||
{"log_prefix", func(st *model.State) { rule(st).Log = true; rule(st).LogPrefix = strings.Repeat("x", 101) }},
|
||||
{"forward_protocol", func(st *model.State) { st.Forwards[0].Protocols = []string{"icmp"} }},
|
||||
@@ -138,3 +165,29 @@ func TestHostWarnings(t *testing.T) {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestNoExternalZone(t *testing.T) {
|
||||
st := base()
|
||||
if got := codes(Check(st, nil)); strings.Contains(got, "no_external_zone") {
|
||||
t.Errorf("an external zone, yet: [%s]", got)
|
||||
}
|
||||
st.Zones[0].External = false
|
||||
if got := codes(Check(st, nil)); !strings.Contains(got, "no_external_zone") {
|
||||
t.Errorf("no external zone, and no warning: [%s]", got)
|
||||
}
|
||||
// An empty configuration applies nothing, so has no fallback either.
|
||||
if got := codes(Check(&model.State{Settings: model.DefaultSettings()}, nil)); strings.Contains(got, "no_external_zone") {
|
||||
t.Errorf("empty configuration: [%s]", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestVLANInterfaces(t *testing.T) {
|
||||
st := base()
|
||||
st.Zones[1].Interfaces = []string{"enp2s0.10", "enp2s0.4094", "br0.20", "enp3s0.*"}
|
||||
st.Settings.FlowtableDevices = []string{"enp2s0.10"}
|
||||
for _, p := range Check(st, nil) {
|
||||
if p.Severity == Error {
|
||||
t.Errorf("VLAN interfaces refused: %+v", p)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -5,6 +5,8 @@ export const state = {
|
||||
me: null, // whoami response
|
||||
csrf: "",
|
||||
catalog: null,
|
||||
draft: null, // the open changeset edits are staged into
|
||||
immediate: false, // apply edits immediately instead of staging them
|
||||
};
|
||||
|
||||
export class APIError extends Error {
|
||||
@@ -20,13 +22,16 @@ export class APIError extends Error {
|
||||
let onUnauthorized = () => {};
|
||||
export function setUnauthorizedHandler(fn) { onUnauthorized = fn; }
|
||||
|
||||
// onChange is told about every applied change (the probation banner).
|
||||
// onChange is told about every applied change (the probation banner), with
|
||||
// the answer (its confirm_token, to confirm it by itself).
|
||||
let onChange = () => {};
|
||||
export function setChangeHandler(fn) { onChange = fn; }
|
||||
|
||||
// api calls the REST API. opts: {body, query}.
|
||||
// api calls the REST API. opts: {body, query, stage (a changeset ID),
|
||||
// timeout (ms)}.
|
||||
export async function api(method, path, opts = {}) {
|
||||
const headers = { "Accept": "application/json" };
|
||||
if (opts.stage) headers["X-Housecarl-Changeset"] = String(opts.stage);
|
||||
let body;
|
||||
if (opts.body !== undefined) {
|
||||
headers["Content-Type"] = "application/json";
|
||||
@@ -40,7 +45,13 @@ export async function api(method, path, opts = {}) {
|
||||
const s = q.toString();
|
||||
if (s) url += (url.includes("?") ? "&" : "?") + s;
|
||||
}
|
||||
const resp = await fetch(url, { method, headers, body, credentials: "same-origin" });
|
||||
let signal;
|
||||
if (opts.timeout) {
|
||||
const ac = new AbortController();
|
||||
setTimeout(() => ac.abort(), opts.timeout);
|
||||
signal = ac.signal;
|
||||
}
|
||||
const resp = await fetch(url, { method, headers, body, credentials: "same-origin", signal });
|
||||
if (resp.status === 204) return null;
|
||||
const ct = resp.headers.get("Content-Type") || "";
|
||||
const data = ct.includes("application/json") ? await resp.json() : await resp.text();
|
||||
@@ -51,25 +62,131 @@ export async function api(method, path, opts = {}) {
|
||||
return data;
|
||||
}
|
||||
|
||||
export const get = (path, query) => api("GET", path, { query });
|
||||
// get reads; with pending changes, the configuration as they will make it
|
||||
// (objects marked "pending": new, changed, deleted).
|
||||
export const get = (path, query) => api("GET", path, { query: draftQuery(query) });
|
||||
|
||||
// change sends a configuration change and reports it: a toast with the
|
||||
// history entry (to undo it from History) and one per new warning. It
|
||||
// returns the response.
|
||||
// draftQuery names the draft changeset in a query, unless edits are
|
||||
// applied immediately.
|
||||
export function draftQuery(query = {}) {
|
||||
if (state.draft && !state.immediate) return { ...query, changeset: state.draft.id };
|
||||
return query;
|
||||
}
|
||||
|
||||
// ---- the draft changeset (review mode, as in Jarl)
|
||||
|
||||
const draftKey = () => "housecarl.draft." + (state.me ? state.me.user.username : "");
|
||||
const immediateKey = () => "housecarl.immediate." + (state.me ? state.me.user.username : "");
|
||||
|
||||
const listeners = new Set();
|
||||
export function onDraftChange(fn) { listeners.add(fn); return () => listeners.delete(fn); }
|
||||
function notify() { for (const fn of listeners) fn(); }
|
||||
|
||||
function remember(key, value) {
|
||||
try { if (value === null) localStorage.removeItem(key); else localStorage.setItem(key, value); } catch { /* private window */ }
|
||||
}
|
||||
function recall(key) {
|
||||
try { return localStorage.getItem(key); } catch { return null; }
|
||||
}
|
||||
|
||||
// loadDraft restores the user's draft changeset if it is still open, or
|
||||
// picks up their newest open one made in the web UI.
|
||||
export async function loadDraft() {
|
||||
state.immediate = recall(immediateKey()) === "1";
|
||||
state.draft = null;
|
||||
const id = recall(draftKey());
|
||||
if (id) {
|
||||
try {
|
||||
const cs = await api("GET", "/changesets/" + id);
|
||||
if (cs.status === "open") state.draft = cs;
|
||||
} catch { /* gone */ }
|
||||
}
|
||||
if (!state.draft) {
|
||||
try {
|
||||
const list = await api("GET", "/changesets", { query: { status: "open", mine: 1, limit: 20 } });
|
||||
state.draft = list.find((c) => c.via === "web") || null;
|
||||
} catch { /* no config access */ }
|
||||
}
|
||||
remember(draftKey(), state.draft ? String(state.draft.id) : null);
|
||||
notify();
|
||||
}
|
||||
|
||||
export function setImmediate(on) {
|
||||
state.immediate = on;
|
||||
remember(immediateKey(), on ? "1" : "0");
|
||||
notify();
|
||||
}
|
||||
|
||||
export async function refreshDraft() {
|
||||
if (!state.draft) return;
|
||||
try {
|
||||
const cs = await api("GET", "/changesets/" + state.draft.id);
|
||||
state.draft = cs.status === "open" ? cs : null;
|
||||
} catch { state.draft = null; }
|
||||
if (!state.draft) remember(draftKey(), null);
|
||||
notify();
|
||||
}
|
||||
|
||||
export function useDraft(cs) {
|
||||
state.draft = cs;
|
||||
remember(draftKey(), cs ? String(cs.id) : null);
|
||||
notify();
|
||||
}
|
||||
|
||||
async function ensureDraft() {
|
||||
if (state.draft) return state.draft;
|
||||
const cs = await api("POST", "/changesets", { body: { description: "" } });
|
||||
cs.fresh = true;
|
||||
useDraft(cs);
|
||||
return cs;
|
||||
}
|
||||
|
||||
// dropFreshDraft discards a draft made for an edit that was then refused
|
||||
// (or only checked), so no empty changeset is left open.
|
||||
async function dropFreshDraft(cs) {
|
||||
if (!cs.fresh || (state.draft && state.draft.id !== cs.id) || (state.draft && (state.draft.ops || []).length)) return;
|
||||
try { await api("POST", "/changesets/" + cs.id + "/discard"); } catch { /* leave it */ }
|
||||
useDraft(null);
|
||||
}
|
||||
|
||||
// change sends a configuration change: staged in the draft changeset, or
|
||||
// applied at once in immediate mode (or with opts.immediate). It reports
|
||||
// it (toasts) and returns the response.
|
||||
export async function change(method, path, body, opts = {}) {
|
||||
const r = await api(method, path, { body, query: opts.query });
|
||||
let r;
|
||||
if (state.immediate || opts.immediate) {
|
||||
r = await api(method, path, { body, query: opts.query });
|
||||
} else {
|
||||
const cs = await ensureDraft();
|
||||
try {
|
||||
r = await api(method, path, { body, query: opts.query, stage: cs.id });
|
||||
} catch (e) {
|
||||
if (e.status === 409 && /is (applied|discarded)/.test(e.message)) {
|
||||
useDraft(null);
|
||||
return change(method, path, body, opts);
|
||||
}
|
||||
await dropFreshDraft(cs);
|
||||
throw e;
|
||||
}
|
||||
if (r && r.staged && r.changeset) useDraft(r.changeset);
|
||||
else await dropFreshDraft(cs);
|
||||
}
|
||||
if (!opts.quiet) report(r);
|
||||
return r;
|
||||
}
|
||||
|
||||
export function report(r) {
|
||||
if (!r) return;
|
||||
if (r.dry_run) toast("Checked by Housecarl and nftables: nothing was changed");
|
||||
else if (r.applied && r.confirm_by) toast("Applied on probation (history #" + r.history_id + "): confirm it in the banner", "warn", 6000);
|
||||
if (r.staged) {
|
||||
const n = ((r.changeset || {}).ops || []).length;
|
||||
toast("Staged: " + ((r.changeset.ops || [])[n - 1] || {}).summary + " — " + (n === 1 ? "1 pending change" : n + " pending changes") + " to review and apply");
|
||||
} else if (r.dry_run) toast("Checked by Housecarl and nftables: nothing was changed");
|
||||
else if (r.applied && r.confirm_by && !r.confirm_token) toast("Applied on probation (history #" + r.history_id + "): confirm it in the banner", "warn", 6000);
|
||||
else if (r.applied && r.confirm_by) toast("Applied (history #" + r.history_id + ")");
|
||||
else if (r.applied) toast("Applied and saved (history #" + r.history_id + ")");
|
||||
else if (!(r.changes || []).length) toast("Nothing to change");
|
||||
for (const w of r.warnings || []) toast(w.message + (w.object ? " (" + w.object + ")" : ""), "warn", 10000);
|
||||
if (r.applied) onChange();
|
||||
if (r.applied) onChange(r);
|
||||
}
|
||||
|
||||
// can reports whether the user may read (or write) a resource.
|
||||
|
||||
@@ -1,7 +1,12 @@
|
||||
// Housecarl web UI: router, layout, login and the probation banner.
|
||||
import { h, mount, toast, errorBox } from "./ui.js";
|
||||
import { h, mount, toast, errorBox, modal } from "./ui.js";
|
||||
import { wordmark, logo } from "./brand.js";
|
||||
import { api, get, state, can, setUnauthorizedHandler, setChangeHandler } from "./api.js";
|
||||
import { api, get, state, can, setUnauthorizedHandler, setChangeHandler, loadDraft, onDraftChange, setImmediate, useDraft } from "./api.js";
|
||||
import * as changes from "./pages/changes.js";
|
||||
import * as connections from "./pages/connections.js";
|
||||
import * as tracePage from "./pages/trace.js";
|
||||
import * as matrixPage from "./pages/matrix.js";
|
||||
import * as statsPage from "./pages/stats.js";
|
||||
import * as dashboard from "./pages/dashboard.js";
|
||||
import * as rules from "./pages/rules.js";
|
||||
import * as nat from "./pages/nat.js";
|
||||
@@ -21,12 +26,17 @@ const routes = [
|
||||
{ pattern: /^$/, page: dashboard, nav: "dashboard" },
|
||||
{ pattern: /^rules$/, page: rules, nav: "rules" },
|
||||
{ pattern: /^nat$/, page: nat, nav: "nat" },
|
||||
{ pattern: /^matrix$/, page: matrixPage, nav: "matrix" },
|
||||
{ pattern: /^zones$/, page: zones, nav: "zones" },
|
||||
{ pattern: /^sets$/, page: sets, nav: "sets" },
|
||||
{ pattern: /^services$/, page: services, nav: "services" },
|
||||
{ pattern: /^log$/, page: log, nav: "log" },
|
||||
{ pattern: /^connections$/, page: connections, nav: "connections" },
|
||||
{ pattern: /^trace$/, page: tracePage, nav: "trace" },
|
||||
{ pattern: /^stats$/, page: statsPage, nav: "stats" },
|
||||
{ pattern: /^settings$/, page: settings, nav: "settings" },
|
||||
{ pattern: /^import$/, page: importPage, nav: "import" },
|
||||
{ pattern: /^changes$/, page: changes, nav: "changes" },
|
||||
{ pattern: /^history$/, page: history, nav: "history" },
|
||||
{ pattern: /^history\/(\d+)$/, page: history, nav: "history", params: ["id"] },
|
||||
{ pattern: /^audit$/, page: audit, nav: "audit" },
|
||||
@@ -39,13 +49,20 @@ const routes = [
|
||||
const navItems = [
|
||||
["dashboard", "#/", "Dashboard"],
|
||||
["rules", "#/rules", "Rules", "config"],
|
||||
["matrix", "#/matrix", "Matrix", "config"],
|
||||
["nat", "#/nat", "NAT", "config"],
|
||||
{ menu: "Objects", items: [
|
||||
["zones", "#/zones", "Zones", "config"],
|
||||
["sets", "#/sets", "Address sets", "config"],
|
||||
["services", "#/services", "Services", "config"],
|
||||
] },
|
||||
{ menu: "Traffic", items: [
|
||||
["stats", "#/stats", "Statistics", "log"],
|
||||
["log", "#/log", "Log", "log"],
|
||||
["connections", "#/connections", "Connections", "log"],
|
||||
["trace", "#/trace", "Trace", "log"],
|
||||
] },
|
||||
["changes", "#/changes", "Changes", "config"],
|
||||
["history", "#/history", "History", "config"],
|
||||
{ menu: "Server", items: [
|
||||
["settings", "#/settings", "Settings and built-ins", "config"],
|
||||
@@ -56,7 +73,7 @@ const navItems = [
|
||||
];
|
||||
|
||||
const app = document.getElementById("app");
|
||||
let mainEl, navEl, bannerEl, current = { key: "" }, routeGen = 0;
|
||||
let mainEl, navEl, bannerEl, barEl, modeHolder, current = { key: "" }, routeGen = 0;
|
||||
|
||||
function parseHash() {
|
||||
const raw = location.hash.replace(/^#\/?/, "");
|
||||
@@ -92,6 +109,8 @@ async function loggedIn(me) {
|
||||
state.me = me;
|
||||
state.csrf = me.csrf_token || "";
|
||||
layout();
|
||||
onDraftChange(renderBar);
|
||||
if (can("config", true)) await loadDraft();
|
||||
window.addEventListener("hashchange", route);
|
||||
await route();
|
||||
}
|
||||
@@ -100,6 +119,8 @@ function layout() {
|
||||
navEl = h("nav", { class: "main" });
|
||||
mainEl = h("main");
|
||||
bannerEl = h("div", { class: "probation-banner hidden", role: "alert" });
|
||||
barEl = h("div");
|
||||
modeHolder = can("config", true) ? h("span") : null;
|
||||
const me = state.me;
|
||||
const header = h("header", { class: "top" },
|
||||
h("a", { class: "brand", href: "#/", title: "Dashboard" },
|
||||
@@ -107,19 +128,88 @@ function layout() {
|
||||
wordmark({ rule: false, height: 17, cls: "brand-name" })),
|
||||
navEl,
|
||||
h("div", { class: "header-right" },
|
||||
modeHolder,
|
||||
menu(h("span", { class: "user", title: (me.permissions || []).join("\n") }, me.user.username), [
|
||||
h("a", { href: "#/account" }, "Account"),
|
||||
me.admin ? h("a", { href: "#/users" }, "Users") : null,
|
||||
h("button", { type: "button", class: "menu-item", onclick: logout }, "Log out"),
|
||||
], { right: true, active: ["account", "users"].includes(current.nav), label: "Account menu" })));
|
||||
mount(app, h("div", { class: "shell" }, header, bannerEl, mainEl, footer(me.build)));
|
||||
mount(app, h("div", { class: "shell" }, header, barEl, bannerEl, mainEl, footer(me.build)));
|
||||
renderNav();
|
||||
refreshProbation();
|
||||
}
|
||||
|
||||
// ---- pending changes: edits are staged for review (as in Jarl)
|
||||
|
||||
// renderBar shows the pending-changes bar (while there are pending
|
||||
// changes) and the Review/Live switch in the header.
|
||||
function renderBar() {
|
||||
if (!barEl) return;
|
||||
const d = state.draft;
|
||||
const n = d ? (d.ops || []).length : 0;
|
||||
if (modeHolder) {
|
||||
const live = state.immediate;
|
||||
mount(modeHolder, h("button", {
|
||||
type: "button", class: "mode-toggle" + (live ? " live" : ""), "aria-pressed": live ? "true" : "false",
|
||||
title: live ? "Edits are applied right away (each on probation over the network). Click to collect them for review first."
|
||||
: "Edits are collected as pending changes, reviewed with the ruleset's diff and applied together under Changes. Click to apply edits right away.",
|
||||
onclick: () => { setImmediate(!live); rerender(); },
|
||||
}, h("span", { class: "dot" }), live ? "Live edits" : "Review edits"));
|
||||
}
|
||||
mount(barEl, n > 0 && !state.immediate ? h("div", { class: "pending-bar" },
|
||||
h("strong", null, n === 1 ? "1 pending change" : n + " pending changes"),
|
||||
h("span", { class: "muted" }, "Collected in changeset #" + d.id + "; not in force yet."),
|
||||
h("a", { class: "button small", href: "#/changes" }, "Review"),
|
||||
h("button", { class: "primary small apply-now", title: "Apply the pending changes now, without reviewing them first", onclick: (ev) => applyDraft(ev.target, d) }, "Apply now")) : null);
|
||||
}
|
||||
|
||||
// applyDraft applies the draft changeset from the bar and stays on the
|
||||
// page. If it is refused, a dialog says why and leads to Changes.
|
||||
export async function applyDraft(btn, d, description) {
|
||||
btn.disabled = true;
|
||||
const label = btn.textContent;
|
||||
btn.textContent = "Applying…";
|
||||
try {
|
||||
const r = await api("POST", "/changesets/" + d.id + "/apply", { body: { description: description || "" } });
|
||||
useDraft(null);
|
||||
if (r.confirm_by && !r.confirm_token) toast("Applied changeset #" + d.id + " on probation (history #" + r.history_id + "): confirm it in the banner", "warn", 8000);
|
||||
else toast("Applied changeset #" + d.id + " (history #" + r.history_id + ")");
|
||||
for (const w of r.warnings || []) toast(w.message + (w.object ? " (" + w.object + ")" : ""), "warn", 10000);
|
||||
if (r.confirm_token) await autoConfirm(r.confirm_token);
|
||||
else await refreshProbation();
|
||||
return r;
|
||||
} catch (e) {
|
||||
btn.disabled = false;
|
||||
btn.textContent = label;
|
||||
const review = await modal("The changes were not applied", errorBox(e), [{ label: "Close", value: false }, { label: "Review changes", kind: "primary", value: true }], { wide: true });
|
||||
if (review) location.hash = "#/changes";
|
||||
return null;
|
||||
} finally {
|
||||
rerender();
|
||||
}
|
||||
}
|
||||
|
||||
// ---- probation: changes made here roll back unless confirmed
|
||||
|
||||
let probationTimer = null, deadline = 0, pending = 0;
|
||||
let probationTimer = null, deadline = 0, pending = 0, checking = false;
|
||||
|
||||
// autoConfirm confirms a change with its token. The server closed the
|
||||
// change's connection, so this request opens a new one through the new
|
||||
// ruleset: it only gets through if this browser can still reach
|
||||
// Housecarl. If it can't, the change rolls back by itself.
|
||||
export async function autoConfirm(token) {
|
||||
checking = true;
|
||||
await refreshProbation();
|
||||
try {
|
||||
await api("POST", "/confirm", { body: { token }, timeout: 10000 });
|
||||
checking = false;
|
||||
toast("Confirmed automatically: this browser reached Housecarl again over a new connection");
|
||||
} catch (e) {
|
||||
checking = false;
|
||||
toast("Couldn't confirm the change (" + (e.name === "AbortError" ? "no answer over a new connection" : e.message) + "): unless it is confirmed, it rolls back by itself", "warn", 10000);
|
||||
}
|
||||
await refreshProbation();
|
||||
}
|
||||
|
||||
// refreshProbation asks whether changes wait for confirmation, and shows
|
||||
// the banner with a countdown while they do.
|
||||
@@ -147,6 +237,10 @@ function drawBanner() {
|
||||
}
|
||||
bannerEl.classList.remove("hidden");
|
||||
const write = can("config", true);
|
||||
if (checking) {
|
||||
mount(bannerEl, h("strong", null, "Checking you can still reach Housecarl…"), " The change is confirmed once this browser gets through over a new connection.");
|
||||
return;
|
||||
}
|
||||
mount(bannerEl,
|
||||
h("strong", null, pending === 1 ? "1 change is on probation" : pending + " changes are on probation"),
|
||||
" — rolled back in ", h("span", { class: "mono" }, left + "s"), " unless you confirm them. If this page stops answering, wait: the firewall puts itself back.",
|
||||
@@ -246,7 +340,7 @@ export function rerender() {
|
||||
if (current.page) route();
|
||||
}
|
||||
|
||||
setChangeHandler(() => refreshProbation());
|
||||
setChangeHandler((r) => r && r.confirm_token ? autoConfirm(r.confirm_token) : refreshProbation());
|
||||
|
||||
setUnauthorizedHandler(() => {
|
||||
if (state.me) {
|
||||
|
||||
@@ -0,0 +1,351 @@
|
||||
// Small SVG charts for the statistics page: a line/area chart with a
|
||||
// crosshair tooltip, a weekday x hour heatmap, and bar lists. No library;
|
||||
// colors come from CSS custom properties (--series-N, --heat-N) so light
|
||||
// and dark themes are handled in style.css. All text is inserted as text,
|
||||
// never as markup.
|
||||
import { h } from "./ui.js";
|
||||
|
||||
const NS = "http://www.w3.org/2000/svg";
|
||||
|
||||
function svg(tag, attrs, ...children) {
|
||||
const el = document.createElementNS(NS, tag);
|
||||
for (const [k, v] of Object.entries(attrs || {})) {
|
||||
if (v !== undefined && v !== null && v !== false) el.setAttribute(k, v);
|
||||
}
|
||||
for (const c of children.flat(Infinity)) {
|
||||
if (c === null || c === undefined || c === false) continue;
|
||||
el.append(c instanceof Node ? c : document.createTextNode(String(c)));
|
||||
}
|
||||
return el;
|
||||
}
|
||||
|
||||
// compact formats a count: 1,234 / 12.9K / 4.2M.
|
||||
export function compact(n) {
|
||||
const a = Math.abs(n);
|
||||
if (a >= 1e9) return (n / 1e9).toFixed(a >= 1e10 ? 0 : 1).replace(/\.0$/, "") + "B";
|
||||
if (a >= 1e6) return (n / 1e6).toFixed(a >= 1e7 ? 0 : 1).replace(/\.0$/, "") + "M";
|
||||
if (a >= 1e4) return (n / 1e3).toFixed(a >= 1e5 ? 0 : 1).replace(/\.0$/, "") + "K";
|
||||
return Math.round(n).toLocaleString();
|
||||
}
|
||||
|
||||
// niceTicks returns 0..max in clean steps (1, 2 or 5 x 10^k), at most n+1 ticks.
|
||||
export function niceTicks(max, n = 4) {
|
||||
if (max <= 0) return [0, 1];
|
||||
const raw = max / n;
|
||||
const mag = Math.pow(10, Math.floor(Math.log10(raw)));
|
||||
const step = [1, 2, 5, 10].map((m) => m * mag).find((s) => s >= raw);
|
||||
const ticks = [];
|
||||
for (let v = 0; v < max + step * 0.999; v += step) ticks.push(Math.round(v * 1e6) / 1e6);
|
||||
return ticks;
|
||||
}
|
||||
|
||||
const MIN = 60, HOUR = 3600, DAY = 86400;
|
||||
const tickSteps = [5 * MIN, 10 * MIN, 15 * MIN, 30 * MIN, HOUR, 2 * HOUR, 3 * HOUR, 6 * HOUR, 12 * HOUR, DAY, 2 * DAY, 7 * DAY, 14 * DAY, 30 * DAY];
|
||||
|
||||
// timeTicks picks local-time-aligned ticks between t0 and t1 (Unix seconds).
|
||||
function timeTicks(t0, t1, maxTicks) {
|
||||
const span = t1 - t0;
|
||||
const step = tickSteps.find((s) => span / s <= maxTicks) || 30 * DAY;
|
||||
const out = [];
|
||||
const d = new Date(t0 * 1000);
|
||||
if (step >= DAY) {
|
||||
d.setHours(0, 0, 0, 0);
|
||||
} else {
|
||||
d.setSeconds(0, 0);
|
||||
const mins = d.getHours() * 60 + d.getMinutes();
|
||||
const stepMin = step / 60;
|
||||
d.setHours(0, Math.floor(mins / stepMin) * stepMin, 0, 0);
|
||||
}
|
||||
for (let t = d.getTime() / 1000; t <= t1; ) {
|
||||
if (t >= t0) out.push(t);
|
||||
if (step >= DAY) {
|
||||
const nd = new Date(t * 1000);
|
||||
nd.setDate(nd.getDate() + step / DAY);
|
||||
t = nd.getTime() / 1000;
|
||||
} else t += step;
|
||||
}
|
||||
return { ticks: out, step };
|
||||
}
|
||||
|
||||
const weekdays = ["Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat"];
|
||||
const months = ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"];
|
||||
const pad = (n) => String(n).padStart(2, "0");
|
||||
|
||||
function tickLabel(t, step) {
|
||||
const d = new Date(t * 1000);
|
||||
if (step >= DAY) return step >= 7 * DAY ? months[d.getMonth()] + " " + d.getDate() : weekdays[d.getDay()] + " " + d.getDate();
|
||||
const hm = pad(d.getHours()) + ":" + pad(d.getMinutes());
|
||||
return d.getHours() === 0 && d.getMinutes() === 0 ? weekdays[d.getDay()] + " " + d.getDate() : hm;
|
||||
}
|
||||
|
||||
// longTime describes a bucket for tooltips and tables.
|
||||
export function bucketLabel(t, stepSecs) {
|
||||
const d = new Date(t * 1000);
|
||||
const day = weekdays[d.getDay()] + " " + d.getDate() + " " + months[d.getMonth()];
|
||||
if (stepSecs >= DAY) return day;
|
||||
const end = new Date((t + stepSecs) * 1000);
|
||||
const hm = (x) => pad(x.getHours()) + ":" + pad(x.getMinutes());
|
||||
return day + ", " + hm(d) + (stepSecs > MIN ? "–" + hm(end) : "");
|
||||
}
|
||||
|
||||
export const stepSeconds = { "1m": MIN, "5m": 5 * MIN, "15m": 15 * MIN, "1h": HOUR, "6h": 6 * HOUR, "1d": DAY };
|
||||
|
||||
// tooltip is one floating readout per chart.
|
||||
function tooltip() {
|
||||
const el = h("div", { class: "chart-tip", role: "status" });
|
||||
el.hidden = true;
|
||||
return el;
|
||||
}
|
||||
|
||||
// chartCard wraps a chart with its title and a table-view toggle (the
|
||||
// accessible twin of every chart).
|
||||
export function chartCard(title, subtitle, chart, tableView, opts = {}) {
|
||||
const body = h("div", { class: "chart-body" }, chart);
|
||||
const tableBox = h("div", { class: "chart-table" }, tableView);
|
||||
tableBox.hidden = true;
|
||||
const toggle = h("button", { class: "small link", type: "button", "aria-pressed": "false" }, "Table");
|
||||
toggle.addEventListener("click", () => {
|
||||
const showTable = tableBox.hidden;
|
||||
tableBox.hidden = !showTable;
|
||||
body.hidden = showTable;
|
||||
toggle.textContent = showTable ? "Chart" : "Table";
|
||||
toggle.setAttribute("aria-pressed", String(showTable));
|
||||
});
|
||||
return h("figure", { class: "panel chart-card" + (opts.wide ? " wide" : "") },
|
||||
h("figcaption", { class: "chart-head" },
|
||||
h("div", { class: "grow" }, h("h2", null, title), subtitle ? h("div", { class: "muted small" }, subtitle) : null),
|
||||
opts.legend || null,
|
||||
tableView ? toggle : null),
|
||||
body, tableBox);
|
||||
}
|
||||
|
||||
// legend lists series with a short line key (the mark the series uses).
|
||||
export function legend(series) {
|
||||
return h("ul", { class: "chart-legend" }, series.map((s) =>
|
||||
h("li", null, h("span", { class: "key line s" + s.slot }), s.label)));
|
||||
}
|
||||
|
||||
// endLabelWidth is the room lineChart needs for the end labels of series.
|
||||
export function endLabelWidth(series) {
|
||||
return series.length > 1 && series.length <= 4 ? Math.max(...series.map((s) => s.label.length)) * 7 + 12 : 8;
|
||||
}
|
||||
|
||||
// lineChart draws series over time. series: [{label, values, slot}] (slot
|
||||
// = categorical color 1..3); times: bucket starts (Unix seconds); step in
|
||||
// seconds. With area, the first series gets a 10% wash under its line.
|
||||
// gutter {left, right} sets the least room for the axis labels and the end
|
||||
// labels: charts shown one above the other pass the same gutter, so their
|
||||
// time axes line up. format formats values (axis and readout); a null
|
||||
// value is a gap in the line (no data for that time).
|
||||
export function lineChart({ series, times, step, area = false, height = 220, endLabels = true, gutter = {}, format = compact }) {
|
||||
const root = h("div", { class: "chart", tabindex: "0", "aria-label": series.map((s) => s.label).join(", ") + " over time" });
|
||||
const tip = tooltip();
|
||||
root.append(tip);
|
||||
let width = 0, focusIdx = -1, geom = null;
|
||||
|
||||
function draw() {
|
||||
const w = root.clientWidth;
|
||||
if (!w || w === width) return;
|
||||
width = w;
|
||||
root.querySelector("svg")?.remove();
|
||||
const max = Math.max(1, ...series.flatMap((s) => s.values).filter((v) => v != null));
|
||||
const yTicks = niceTicks(max, 4);
|
||||
const yMax = yTicks[yTicks.length - 1];
|
||||
const labelW = Math.max(...yTicks.map((t) => format(t).length)) * 7 + 10;
|
||||
const endW = endLabels ? endLabelWidth(series) : 8;
|
||||
const m = { top: 10, right: Math.max(endW, gutter.right || 0), bottom: 24, left: Math.max(labelW, gutter.left || 0) };
|
||||
const pw = Math.max(40, w - m.left - m.right), ph = height - m.top - m.bottom;
|
||||
const t0 = times[0], t1 = times[times.length - 1] + step;
|
||||
const x = (t) => m.left + ((t - t0) / Math.max(1, t1 - t0)) * pw;
|
||||
const cx = (i) => x(times[i] + step / 2); // the middle of a bucket
|
||||
const y = (v) => m.top + ph - (v / yMax) * ph;
|
||||
const s = svg("svg", { width: w, height, viewBox: `0 0 ${w} ${height}`, role: "img", "aria-hidden": "true" });
|
||||
// Grid and axes.
|
||||
for (const t of yTicks) {
|
||||
s.append(svg("line", { class: t === 0 ? "axis" : "grid", x1: m.left, x2: m.left + pw, y1: y(t), y2: y(t) }));
|
||||
s.append(svg("text", { class: "tick", x: m.left - 6, y: y(t) + 4, "text-anchor": "end" }, format(t)));
|
||||
}
|
||||
const { ticks, step: tstep } = timeTicks(t0, t1, Math.max(2, Math.floor(pw / 90)));
|
||||
for (const t of ticks) {
|
||||
s.append(svg("text", { class: "tick", x: x(t), y: height - 6, "text-anchor": "middle" }, tickLabel(t, tstep)));
|
||||
}
|
||||
// Series.
|
||||
series.forEach((sr, k) => {
|
||||
// A gap (null) starts a new piece of line.
|
||||
const pts = sr.values.map((v, i) => (v == null ? null : [cx(i), y(v)]));
|
||||
const d = pts.map((p, i) => (p ? (i && pts[i - 1] ? "L" : "M") + p[0].toFixed(1) + " " + p[1].toFixed(1) : "")).join("");
|
||||
if (area && k === 0 && pts.length && pts.every(Boolean)) {
|
||||
s.append(svg("path", { class: "area s" + sr.slot, d: d + `L${pts[pts.length - 1][0].toFixed(1)} ${y(0)}L${pts[0][0].toFixed(1)} ${y(0)}Z` }));
|
||||
}
|
||||
s.append(svg("path", { class: "line s" + sr.slot, d }));
|
||||
// A point without neighbours has no line: show it as a dot.
|
||||
pts.forEach((p, i) => {
|
||||
if (p && !pts[i - 1] && !pts[i + 1]) s.append(svg("circle", { class: "dot s" + sr.slot, cx: p[0], cy: p[1], r: 3 }));
|
||||
});
|
||||
});
|
||||
// Direct end labels when there is room and they don't collide.
|
||||
if (m.right > 8) {
|
||||
const ends = series.map((sr) => ({ label: sr.label, y: y(sr.values.findLast((v) => v != null) || 0) })).sort((a, b) => a.y - b.y);
|
||||
if (ends.every((e, i) => i === 0 || e.y - ends[i - 1].y >= 13)) {
|
||||
for (const e of ends) s.append(svg("text", { class: "end-label", x: m.left + pw + 6, y: e.y + 4 }, e.label));
|
||||
}
|
||||
}
|
||||
// Crosshair (drawn on hover/focus).
|
||||
const cross = svg("line", { class: "crosshair", y1: m.top, y2: m.top + ph, visibility: "hidden" });
|
||||
const dots = series.map((sr) => svg("circle", { class: "dot s" + sr.slot, r: 4, visibility: "hidden" }));
|
||||
s.append(cross, ...dots);
|
||||
// The hit area is the whole plot: readers aim at a time, not a line.
|
||||
const hit = svg("rect", { class: "hit", x: m.left, y: m.top, width: pw, height: ph });
|
||||
s.append(hit);
|
||||
root.prepend(s);
|
||||
geom = { cx, y, m, pw, ph, cross, dots, hit };
|
||||
hit.addEventListener("pointermove", (e) => {
|
||||
const r = s.getBoundingClientRect();
|
||||
const px = e.clientX - r.left;
|
||||
const i = Math.round(((px - m.left) / pw) * (times.length) - 0.5);
|
||||
show(Math.max(0, Math.min(times.length - 1, i)));
|
||||
});
|
||||
hit.addEventListener("pointerleave", hide);
|
||||
if (focusIdx >= 0) show(focusIdx);
|
||||
}
|
||||
|
||||
function show(i) {
|
||||
if (!geom) return;
|
||||
focusIdx = i;
|
||||
const xpos = geom.cx(i);
|
||||
geom.cross.setAttribute("x1", xpos);
|
||||
geom.cross.setAttribute("x2", xpos);
|
||||
geom.cross.setAttribute("visibility", "visible");
|
||||
series.forEach((sr, k) => {
|
||||
geom.dots[k].setAttribute("cx", xpos);
|
||||
geom.dots[k].setAttribute("cy", geom.y(sr.values[i] ?? 0));
|
||||
geom.dots[k].setAttribute("visibility", sr.values[i] == null ? "hidden" : "visible");
|
||||
});
|
||||
tip.replaceChildren(
|
||||
h("div", { class: "tip-head" }, bucketLabel(times[i], step)),
|
||||
...series.map((sr) => h("div", { class: "tip-row" },
|
||||
h("span", { class: "key line s" + sr.slot }), h("strong", null, sr.values[i] == null ? "–" : format === compact ? sr.values[i].toLocaleString() : format(sr.values[i])), " ", h("span", { class: "muted" }, sr.label))));
|
||||
tip.hidden = false;
|
||||
// Keep the readout inside the chart, on the far side of the pointer.
|
||||
const left = xpos > root.clientWidth / 2 ? xpos - tip.offsetWidth - 12 : xpos + 12;
|
||||
tip.style.left = Math.max(0, left) + "px";
|
||||
tip.style.top = geom.m.top + "px";
|
||||
}
|
||||
|
||||
function hide() {
|
||||
if (!geom) return;
|
||||
geom.cross.setAttribute("visibility", "hidden");
|
||||
geom.dots.forEach((d) => d.setAttribute("visibility", "hidden"));
|
||||
tip.hidden = true;
|
||||
focusIdx = -1;
|
||||
}
|
||||
|
||||
// Keyboard: arrows move the crosshair, as hover does.
|
||||
root.addEventListener("focus", () => show(focusIdx >= 0 ? focusIdx : times.length - 1));
|
||||
root.addEventListener("blur", hide);
|
||||
root.addEventListener("keydown", (e) => {
|
||||
if (e.key === "ArrowLeft" || e.key === "ArrowRight") {
|
||||
e.preventDefault();
|
||||
show(Math.max(0, Math.min(times.length - 1, (focusIdx < 0 ? times.length - 1 : focusIdx) + (e.key === "ArrowLeft" ? -1 : 1))));
|
||||
}
|
||||
});
|
||||
new ResizeObserver(draw).observe(root);
|
||||
return root;
|
||||
}
|
||||
|
||||
// timeTable is the table view of a line chart (busiest buckets first is
|
||||
// not useful here: keep time order, newest first).
|
||||
export function timeTable(series, times, step, format) {
|
||||
const cell = (v) => (v == null ? "–" : format ? format(v) : v.toLocaleString());
|
||||
const rows = times.map((t, i) => ({ t, i })).reverse();
|
||||
return h("div", { class: "table-wrap scroll" }, h("table", null,
|
||||
h("thead", null, h("tr", null, h("th", null, "Time"), series.map((s) => h("th", { class: "num" }, s.label)))),
|
||||
h("tbody", null, rows.map((r) => h("tr", null, h("td", null, bucketLabel(r.t, step)),
|
||||
series.map((s) => h("td", { class: "num" }, cell(s.values[r.i]))))))));
|
||||
}
|
||||
|
||||
// heatmap draws average activity by weekday (rows, Monday first) and hour
|
||||
// (columns) with a sequential one-hue ramp; cells without data stay empty.
|
||||
export function heatmap(hm, unit) {
|
||||
const days = ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"];
|
||||
let peak = 0;
|
||||
hm.avg.forEach((row) => row.forEach((v) => { peak = Math.max(peak, v); }));
|
||||
const steps = 7;
|
||||
const level = (v) => (peak <= 0 ? 0 : Math.min(steps - 1, Math.floor((v / peak) * steps)));
|
||||
const tip = tooltip();
|
||||
const grid = h("div", { class: "heatmap", role: "grid", "aria-label": "Average " + unit + " per hour by weekday and hour of day" });
|
||||
grid.append(h("div", { class: "hm-corner" }));
|
||||
for (let hr = 0; hr < 24; hr++) grid.append(h("div", { class: "hm-col" }, hr % 3 === 0 ? pad(hr) : ""));
|
||||
days.forEach((day, d) => {
|
||||
grid.append(h("div", { class: "hm-row" }, day));
|
||||
for (let hr = 0; hr < 24; hr++) {
|
||||
const v = hm.avg[d][hr];
|
||||
const n = hm.samples[d][hr];
|
||||
const text = v < 0 ? day + " " + pad(hr) + ":00 – no data" :
|
||||
day + " " + pad(hr) + ":00 – " + Math.round(v).toLocaleString() + " " + unit + " per hour (average of " + n + (n === 1 ? " hour)" : " hours)");
|
||||
const cell = h("div", { class: "hm-cell " + (v < 0 ? "hm-empty" : "heat-" + level(v)), tabindex: "-1", "aria-label": text });
|
||||
const showTip = () => {
|
||||
tip.textContent = text;
|
||||
tip.hidden = false;
|
||||
const r = cell.getBoundingClientRect(), pr = grid.getBoundingClientRect();
|
||||
tip.style.left = Math.max(0, Math.min(pr.width - tip.offsetWidth, r.left - pr.left - tip.offsetWidth / 2 + r.width / 2)) + "px";
|
||||
tip.style.top = (r.bottom - pr.top + 6) + "px";
|
||||
};
|
||||
cell.addEventListener("pointerenter", showTip);
|
||||
cell.addEventListener("focus", showTip);
|
||||
cell.addEventListener("pointerleave", () => { tip.hidden = true; });
|
||||
cell.addEventListener("blur", () => { tip.hidden = true; });
|
||||
grid.append(cell);
|
||||
}
|
||||
});
|
||||
// Arrow keys move between cells.
|
||||
grid.tabIndex = 0;
|
||||
let cur = -1;
|
||||
const cells = () => [...grid.querySelectorAll(".hm-cell")];
|
||||
grid.addEventListener("keydown", (e) => {
|
||||
const delta = { ArrowLeft: -1, ArrowRight: 1, ArrowUp: -24, ArrowDown: 24 }[e.key];
|
||||
if (delta === undefined) return;
|
||||
e.preventDefault();
|
||||
cur = Math.max(0, Math.min(7 * 24 - 1, (cur < 0 ? 0 : cur) + delta));
|
||||
cells()[cur].focus();
|
||||
});
|
||||
const scale = h("div", { class: "hm-scale" },
|
||||
h("span", { class: "muted small" }, "0"),
|
||||
Array.from({ length: steps }, (_, i) => h("span", { class: "hm-swatch heat-" + i })),
|
||||
h("span", { class: "muted small" }, compact(peak) + " " + unit + "/hour"),
|
||||
h("span", { class: "hm-swatch hm-empty" }), h("span", { class: "muted small" }, "no data"));
|
||||
return h("div", { class: "heatmap-wrap" }, grid, tip, scale);
|
||||
}
|
||||
|
||||
export function heatmapTable(hm, unit) {
|
||||
const days = ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday"];
|
||||
return h("div", { class: "table-wrap scroll" }, h("table", null,
|
||||
h("thead", null, h("tr", null, h("th", null, "Hour"), days.map((d) => h("th", { class: "num" }, d)))),
|
||||
h("tbody", null, Array.from({ length: 24 }, (_, hr) => h("tr", null, h("td", null, pad(hr) + ":00"),
|
||||
days.map((_, d) => h("td", { class: "num" }, hm.avg[d][hr] < 0 ? "–" : Math.round(hm.avg[d][hr]).toLocaleString())))))),
|
||||
h("div", { class: "muted small" }, "Average " + unit + " per hour."));
|
||||
}
|
||||
|
||||
// barList shows ranked entries as labelled horizontal bars in one hue
|
||||
// (one series: no legend, the title names it). Values are always printed,
|
||||
// so the list is its own table view.
|
||||
// extra(key), when given, adds something after an entry's label (e.g. a
|
||||
// link that excludes it).
|
||||
export function barList(entries, { total, label = (k) => k, empty = "No data yet.", href, extra, format = compact } = {}) {
|
||||
if (!entries.length) return h("div", { class: "empty" }, empty);
|
||||
// Entries may be in a natural order (e.g. latency buckets), not by size.
|
||||
const peak = Math.max(1, ...entries.map((e) => e.count));
|
||||
const sum = total || entries.reduce((a, e) => a + e.count, 0);
|
||||
return h("ol", { class: "bar-list" }, entries.map((e) => {
|
||||
const share = sum ? (e.count / sum) * 100 : 0;
|
||||
const name = label(e.key);
|
||||
const link = href ? href(e.key) : null;
|
||||
// Entries with a name (a rule's label): the name, then the key.
|
||||
const shown = e.name ? [e.name, h("span", { class: "muted" }, " " + name)] : name;
|
||||
return h("li", { title: (e.name ? e.name + " (" + name + ")" : name) + ": " + format(e.count) + " (" + share.toFixed(1) + "%)" },
|
||||
h("div", { class: "bar-text" },
|
||||
link ? h("a", { href: link, class: "bar-label" }, shown) : h("span", { class: "bar-label" }, shown),
|
||||
h("span", { class: "bar-value" }, extra ? [extra(e.key), " "] : null, format(e.count), h("span", { class: "muted" }, " · " + share.toFixed(share < 10 ? 1 : 0) + "%"))),
|
||||
h("div", { class: "bar-track" }, h("div", { class: "bar-fill", style: { width: Math.max(0.5, (e.count / peak) * 100) + "%" } })));
|
||||
}));
|
||||
}
|
||||
@@ -0,0 +1,111 @@
|
||||
// Changes: the pending changeset — its operations, the check, and the
|
||||
// ruleset's diff — to apply together or discard. Other open changesets
|
||||
// (from hc or the API) can be taken over here.
|
||||
import { h, mount, table, input, toast, errorBox, confirmDialog, diffView, problemsList, time, plural, badge } from "../ui.js";
|
||||
import { api, state, useDraft, refreshDraft, can } from "../api.js";
|
||||
import { rerender, applyDraft } from "../app.js";
|
||||
|
||||
export async function render(el) {
|
||||
await refreshDraft();
|
||||
const open = await api("GET", "/changesets", { query: { status: "open", limit: 50 } });
|
||||
const d = state.draft;
|
||||
const parts = [h("div", { class: "page-head" },
|
||||
h("div", { class: "grow" }, h("h1", null, "Pending changes"),
|
||||
h("div", { class: "muted" }, "Edits are collected in a changeset. Nothing reaches the firewall until you apply it; then the whole changeset is checked by Housecarl and nftables and put in force in one step, as one entry in the history. Over the network it is on probation: confirm it, or it rolls back.")))];
|
||||
if (state.immediate) {
|
||||
parts.push(h("div", { class: "msg info" }, "Live edits are on (the switch at the top): edits are applied right away, not collected here."));
|
||||
}
|
||||
if (!d || !(d.ops || []).length) {
|
||||
parts.push(h("div", { class: "panel" }, h("div", { class: "empty" }, "No pending changes. Edit rules, NAT, objects or settings and they show up here.")));
|
||||
} else {
|
||||
parts.push(draftPanel(d));
|
||||
}
|
||||
const others = open.filter((c) => !d || c.id !== d.id);
|
||||
if (others.length) {
|
||||
parts.push(h("div", { class: "panel" }, h("h2", null, "Other open changesets"),
|
||||
h("p", { class: "muted small" }, "Made with hc or the API, by others, or earlier here. Switch to one to review and apply it."),
|
||||
table([
|
||||
{ label: "#", render: (c) => String(c.id) },
|
||||
{ label: "Description", render: (c) => c.description || h("span", { class: "muted" }, "(none)") },
|
||||
{ label: "Operations", render: (c) => String((c.ops || []).length) },
|
||||
{ label: "By", render: (c) => [c.username, h("span", { class: "muted small" }, " via " + c.via)] },
|
||||
{ label: "Updated", render: (c) => time(c.updated_at) },
|
||||
can("config", true) ? { label: "", tdClass: "actions-cell", render: (c) => [
|
||||
h("button", { class: "small", onclick: () => { useDraft(c); rerender(); } }, "Switch to"), " ",
|
||||
h("button", { class: "small danger", onclick: async () => {
|
||||
if (!(await confirmDialog("Discard", "Discard changeset #" + c.id + "?", "Discard", "danger"))) return;
|
||||
try { await api("POST", "/changesets/" + c.id + "/discard"); } catch (e) { toast(e.message, "err"); }
|
||||
rerender();
|
||||
} }, "Discard")] } : null,
|
||||
].filter(Boolean), others)));
|
||||
}
|
||||
mount(el, parts);
|
||||
}
|
||||
|
||||
function draftPanel(d) {
|
||||
const ops = d.ops || [];
|
||||
const desc = input({ value: d.description || "", placeholder: "Describe this change (it becomes the history entry's description)", class: "wide" });
|
||||
const preview = h("div", { class: "muted" }, "Checking…");
|
||||
const result = h("div");
|
||||
const applyBtn = h("button", { class: "primary", disabled: true }, "Apply changes");
|
||||
const removeOp = async (seq) => {
|
||||
try {
|
||||
useDraft(await api("DELETE", "/changesets/" + d.id + "/ops/" + seq));
|
||||
rerender();
|
||||
} catch (e) { toast(e.message, "err", 8000); }
|
||||
};
|
||||
const opsTable = table([
|
||||
{ label: "#", render: (o) => String(o.seq) },
|
||||
{ label: "Change", render: (o) => o.summary || o.op },
|
||||
{ label: "Added", render: (o) => time(o.added_at) },
|
||||
{ label: "", tdClass: "actions-cell", render: (o) => h("button", { class: "small", title: "Take this operation out of the changeset", onclick: () => removeOp(o.seq) }, "Remove") },
|
||||
], ops);
|
||||
// failedOp marks the operation that no longer applies, and offers to
|
||||
// take it out.
|
||||
const failedOp = (err) => {
|
||||
const i = ops.findIndex((o) => o.seq === err.op);
|
||||
const rows = opsTable.querySelectorAll("tbody tr");
|
||||
if (i < 0 || !rows[i]) return errorBox({ message: err.message, details: err });
|
||||
rows[i].classList.add("op-failed");
|
||||
return h("div", { class: "msg err" },
|
||||
h("div", null, err.message),
|
||||
h("div", { class: "form-actions" },
|
||||
h("button", { class: "small", onclick: () => removeOp(err.op) }, "Remove operation " + err.op),
|
||||
" ", h("span", { class: "muted small" }, "It is marked in the list above. Something it names changed since it was staged.")));
|
||||
};
|
||||
(async () => {
|
||||
try {
|
||||
const p = await api("GET", "/changesets/" + d.id + "/preview");
|
||||
applyBtn.disabled = !p.valid;
|
||||
mount(preview,
|
||||
p.error ? (p.error.op ? failedOp(p.error) : errorBox({ message: p.error.message, details: p.error })) : null,
|
||||
p.valid ? h("div", { class: "msg ok" }, "Valid: Housecarl's checks pass and nftables accepts the new ruleset.") : null,
|
||||
!p.error && !p.valid ? h("div", { class: "msg info" }, "Together these operations change nothing.") : null,
|
||||
problemsList(p.warnings),
|
||||
h("h3", null, "The ruleset (/var/lib/housecarl/ruleset.nft)"),
|
||||
p.valid ? diffView(p.diff) : h("div", { class: "muted" }, "No diff until the changeset can be applied."));
|
||||
} catch (e) { mount(preview, errorBox(e)); }
|
||||
})();
|
||||
applyBtn.addEventListener("click", async () => {
|
||||
mount(result);
|
||||
if (desc.value !== (d.description || "")) {
|
||||
try { await api("PATCH", "/changesets/" + d.id, { body: { description: desc.value } }); } catch { /* the apply says it */ }
|
||||
}
|
||||
const r = await applyDraft(applyBtn, d, desc.value);
|
||||
if (r && r.history_id) location.hash = "#/history/" + r.history_id;
|
||||
});
|
||||
const discard = h("button", { class: "danger", onclick: async () => {
|
||||
if (!(await confirmDialog("Discard changes", "Throw away all " + plural(ops.length, "pending change") + "?", "Discard", "danger"))) return;
|
||||
try { await api("POST", "/changesets/" + d.id + "/discard"); } catch (e) { toast(e.message, "err"); }
|
||||
useDraft(null);
|
||||
rerender();
|
||||
} }, "Discard all");
|
||||
return h("div", null,
|
||||
h("div", { class: "panel" },
|
||||
h("h2", null, "Changeset #" + d.id + " ", badge(plural(ops.length, "operation"), "warn")),
|
||||
desc,
|
||||
h("div", { class: "form-actions" }, opsTable),
|
||||
h("div", { class: "actions form-actions" }, applyBtn, discard),
|
||||
result),
|
||||
h("div", { class: "panel" }, h("h2", null, "Review"), preview));
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
// Connections: what the kernel tracks, and closing them so the rules
|
||||
// check them again.
|
||||
import { h, mount, input, debounce, toast, confirmDialog, badge } from "../ui.js";
|
||||
import { get, api, can } from "../api.js";
|
||||
import { onLeave } from "../app.js";
|
||||
import { fmtSeconds } from "../widgets.js";
|
||||
|
||||
export async function render(el, params, query) {
|
||||
const search = input({ type: "search", placeholder: "address, :port, protocol, state…", value: query.get("q") || "" });
|
||||
const out = h("div");
|
||||
const count = h("span", { class: "muted small" });
|
||||
const write = can("config", true);
|
||||
let shown = [];
|
||||
const hostPort = (a, p) => !p ? a : a.includes(":") ? "[" + a + "]:" + p : a + ":" + p;
|
||||
const close = async (list, what) => {
|
||||
if (!await confirmDialog("Close connections", "Close " + what + "? Their next packet is checked by the rules as they are now: what they block is stopped, what they allow carries on.", "Close", "danger")) return;
|
||||
try {
|
||||
const r = await api("POST", "/connections/close", { body: list ? { connections: list } : { search: search.value.trim() } });
|
||||
toast("Closed " + r.closed + (r.closed === 1 ? " connection" : " connections"));
|
||||
draw();
|
||||
} catch (e) { toast(e.message, "err"); }
|
||||
};
|
||||
const draw = async () => {
|
||||
shown = await get("/connections", { q: search.value.trim(), limit: 500 });
|
||||
count.textContent = shown.length === 1 ? "1 connection" : shown.length + " connections";
|
||||
mount(out, shown.length ? h("div", { class: "table-wrap" }, h("table", null,
|
||||
h("thead", null, h("tr", null, ["Protocol", "From", "To", "NAT", "State", "Expires", ""].map((t) => h("th", null, t)))),
|
||||
h("tbody", null, shown.map((c) => h("tr", null,
|
||||
h("td", null, c.protocol),
|
||||
h("td", { class: "mono" }, hostPort(c.source, c.source_port)),
|
||||
h("td", { class: "mono" }, hostPort(c.destination, c.destination_port)),
|
||||
h("td", { class: "mono small" }, c.dnat_to ? "to " + hostPort(c.dnat_to, c.dnat_port || c.destination_port) : "", c.snat_to ? (c.dnat_to ? ", " : "") + "as " + c.snat_to : ""),
|
||||
h("td", null, c.state || "", c.assured ? null : [" ", badge("one way", "")]),
|
||||
h("td", { class: "small" }, fmtSeconds(c.timeout)),
|
||||
h("td", { class: "actions-cell" }, write ? h("button", { class: "small reveal", onclick: () => close([c], "this connection") }, "Close") : null)))))) :
|
||||
h("div", { class: "empty" }, search.value.trim() ? "No connection matches." : "No connections tracked."));
|
||||
};
|
||||
search.addEventListener("input", debounce(draw, 300));
|
||||
mount(el,
|
||||
h("div", { class: "page-head" },
|
||||
h("div", { class: "grow" }, h("h1", null, "Connections"),
|
||||
h("div", { class: "muted" }, "The connections the firewall tracks. Their packets are let through by the built-in for established connections, without the rules being checked again, so a rule added later doesn't stop them. Close them, and their next packet is checked by the rules as they are now.")),
|
||||
write ? h("div", { class: "actions" }, h("button", { class: "danger", onclick: () => close(null, search.value.trim() ? "every connection matching “" + search.value.trim() + "”" : "every connection") }, "Close all shown")) : null),
|
||||
h("div", { class: "filter-row" }, search, count),
|
||||
h("div", { class: "panel" }, out));
|
||||
await draw();
|
||||
const t = setInterval(() => { if (!document.hidden && document.activeElement !== search) draw().catch(() => {}); }, 5000);
|
||||
onLeave(() => clearInterval(t));
|
||||
}
|
||||
@@ -4,15 +4,19 @@ import { h, mount, table, badge, problemsList, time } from "../ui.js";
|
||||
import { get, can } from "../api.js";
|
||||
import { onLeave } from "../app.js";
|
||||
import { describe, hits, humanCount } from "../widgets.js";
|
||||
import { lineChart, chartCard, legend, timeTable, stepSeconds } from "../charts.js";
|
||||
import { bytes } from "./stats.js";
|
||||
|
||||
export async function render(el) {
|
||||
const draw = async () => {
|
||||
const st = await get("/status");
|
||||
const config = can("config", false), logs = can("log", false);
|
||||
const [rules, cs, log] = await Promise.all([
|
||||
const [rules, cs, log, tl, zones] = await Promise.all([
|
||||
config ? get("/rules") : [],
|
||||
logs ? get("/counters").then((c) => c.counters).catch(() => ({})) : {},
|
||||
logs ? get("/log", { limit: 8 }).catch(() => []) : [],
|
||||
logs ? get("/stats/timeline", { since: "24h", step: "1h", keys: "zrx:*,ztx:*,drops" }).catch(() => null) : null,
|
||||
config ? get("/zones").catch(() => []) : [],
|
||||
]);
|
||||
const busiest = rules.filter((r) => cs["r" + r.id] && cs["r" + r.id].packets > 0)
|
||||
.sort((a, b) => cs["r" + b.id].packets - cs["r" + a.id].packets).slice(0, 8);
|
||||
@@ -35,6 +39,7 @@ export async function render(el) {
|
||||
card("Dropped by policy", policyDrops("input") + " / " + policyDrops("forward"), "input / forward", logs ? "#/log" : null),
|
||||
card("Problems", st.problems.length, st.problems.length ? "see below" : "none", null)),
|
||||
st.problems.length ? h("div", { class: "panel" }, h("h2", null, "Problems"), problemsList(st.problems)) : null,
|
||||
tl ? statsCharts(tl, zones) : null,
|
||||
h("div", { class: "panel" }, h("h2", null, "Interfaces"), table([
|
||||
{ label: "Interface", render: (i) => [h("span", { class: "dot " + (i.up ? "ok" : "err") }), h("span", { class: "mono" }, h("strong", null, i.name))] },
|
||||
{ label: "Zone", render: (i) => i.zone || h("span", { class: "muted" }, "none") },
|
||||
@@ -63,3 +68,20 @@ export async function render(el) {
|
||||
const t = setInterval(() => { if (!document.hidden) draw().catch(() => {}); }, 15000);
|
||||
onLeave(() => clearInterval(t));
|
||||
}
|
||||
|
||||
// statsCharts: the last 24 hours' traffic with the internet and drops,
|
||||
// per hour (the Statistics page has the rest).
|
||||
function statsCharts(tl, zones) {
|
||||
const ext = new Set(zones.filter((z) => z.external).map((z) => z.name));
|
||||
const step = stepSeconds[tl.step];
|
||||
const total = (prefix) => tl.times.map((_, i) => tl.series.filter((x) => x.key.startsWith(prefix) && ext.has(x.key.slice(prefix.length))).reduce((a, x) => a + x.values[i], 0));
|
||||
const traffic = [{ label: "Downloaded", values: total("zrx:"), slot: 1 }, { label: "Uploaded", values: total("ztx:"), slot: 2 }];
|
||||
const drops = [{ label: "Dropped", values: (tl.series.find((x) => x.key === "drops") || { values: tl.times.map(() => 0) }).values, slot: 2 }];
|
||||
const gutter = { left: 64, right: 8 };
|
||||
return h("div", { class: "chart-grid" },
|
||||
ext.size ? chartCard("Internet traffic", "bytes per hour from and to external zones, last 24 hours",
|
||||
lineChart({ series: traffic, times: tl.times, step, area: true, height: 150, endLabels: false, gutter, format: bytes }),
|
||||
timeTable(traffic, tl.times, step, bytes), { legend: legend(traffic) }) : null,
|
||||
chartCard("Dropped packets", h("span", null, "per hour, last 24 hours · ", h("a", { href: "#/stats" }, "Statistics")),
|
||||
lineChart({ series: drops, times: tl.times, step, area: true, height: 150, endLabels: false, gutter }), timeTable(drops, tl.times, step)));
|
||||
}
|
||||
@@ -1,6 +1,6 @@
|
||||
// History: every configuration change, with undo.
|
||||
import { h, mount, table, time, confirmDialog, modal, errorBox, badge } from "../ui.js";
|
||||
import { get, change, can } from "../api.js";
|
||||
import { get, change, can, state } from "../api.js";
|
||||
import { rerender } from "../app.js";
|
||||
|
||||
export async function render(el, params) {
|
||||
@@ -20,8 +20,9 @@ export async function render(el, params) {
|
||||
}
|
||||
|
||||
async function undo(e) {
|
||||
if (!await confirmDialog("Undo change #" + e.id, "Undo “" + e.description + "”? This is recorded as a new change, which can be undone too.", "Undo")) return;
|
||||
if (!await confirmDialog("Undo change #" + e.id, "Undo “" + e.description + "”? This is recorded as a new change, which can be undone too." + (!state.immediate ? " It is added to your pending changes, to review and apply." : ""), "Undo")) return;
|
||||
try {
|
||||
// Like any edit: staged for review, or applied at once in live mode.
|
||||
await change("POST", "/history/" + e.id + "/revert");
|
||||
rerender();
|
||||
} catch (err) { modal("Can't undo", errorBox(err)); }
|
||||
|
||||
@@ -1,39 +1,48 @@
|
||||
// Import: an existing nftables ruleset as Housecarl's configuration.
|
||||
import { h, mount, field, textarea, input, checkbox, errorBox, copyBox } from "../ui.js";
|
||||
import { api } from "../api.js";
|
||||
import { refreshProbation } from "../app.js";
|
||||
import { refreshProbation, autoConfirm } from "../app.js";
|
||||
|
||||
export async function render(el) {
|
||||
const json = textarea("", { rows: 6, class: "mono", placeholder: "the output of: sudo nft -j list ruleset" });
|
||||
const text = textarea("", { rows: 4, class: "mono", placeholder: "optional, the output of: sudo nft -a list ruleset (keeps what Housecarl can't express as raw rules)" });
|
||||
const defines = textarea("", { rows: 4, class: "mono", placeholder: "optional, your /etc/nftables.conf: its defines name the sets" });
|
||||
const table = input({ placeholder: "inet main (when there are several)", class: "mono" });
|
||||
const zones = input({ placeholder: "enp2s0=lan, wg0=vpn (optional)", class: "mono" });
|
||||
const replace = checkbox("replace Housecarl's configuration", false);
|
||||
const out = h("div");
|
||||
const run = async (dry) => {
|
||||
mount(out);
|
||||
const zoneNames = {};
|
||||
for (const part of zones.value.split(",").map((s) => s.trim()).filter(Boolean)) {
|
||||
const [iface, name] = part.split("=").map((s) => (s || "").trim());
|
||||
if (!iface || !name) { mount(out, h("div", { class: "msg err" }, "Zone names: write each as interface=name (enp2s0=lan).")); return; }
|
||||
zoneNames[iface] = name;
|
||||
}
|
||||
let parsed;
|
||||
try { parsed = JSON.parse(json.value); } catch { mount(out, h("div", { class: "msg err" }, "The ruleset isn't JSON: paste the output of nft -j list ruleset.")); return; }
|
||||
try {
|
||||
const r = await api("POST", "/import/nft", { body: { json: parsed, text: text.value, defines: defines.value, table: table.value.trim(), replace: replace.input.checked },
|
||||
const r = await api("POST", "/import/nft", { body: { json: parsed, text: text.value, defines: defines.value, table: table.value.trim(), zones: zoneNames, replace: replace.input.checked },
|
||||
query: dry ? { dry_run: 1 } : undefined });
|
||||
const st = r.state;
|
||||
mount(out, h("div", { class: "msg " + (dry ? "info" : "ok") },
|
||||
dry ? "Would import: " : "Imported: ",
|
||||
`${st.zones.length} zones, ${st.sets.length} sets, ${st.services.length} services, ${st.rules.length} rules, ${st.forwards.length} port forwards, ${st.snats.length} source NAT rules, ${st.redirects.length} redirects.`,
|
||||
r.confirm_by ? " It is on probation: confirm it in the banner." : ""),
|
||||
r.confirm_by && !r.confirm_token ? " It is on probation: confirm it in the banner." : ""),
|
||||
r.notes.length ? h("div", { class: "panel" }, h("h2", null, "Notes"), h("ul", null, r.notes.map((n) => h("li", null, n)))) : null,
|
||||
r.raw.length ? h("div", { class: "panel" }, h("h2", null, "Kept as raw rules"), h("ul", { class: "mono small" }, r.raw.map((n) => h("li", null, n)))) : null,
|
||||
!dry ? h("p", null, "Check the ", h("a", { href: "#/rules" }, "rules"), " and ", h("a", { href: "#/nat" }, "NAT"),
|
||||
"; both rulesets apply until the old table goes: ", copyBox("sudo hc takeover \"inet main\"")) : null);
|
||||
if (!dry) refreshProbation();
|
||||
if (!dry && r.confirm_token) autoConfirm(r.confirm_token);
|
||||
else if (!dry) refreshProbation();
|
||||
} catch (e) { mount(out, errorBox(e)); }
|
||||
};
|
||||
mount(el,
|
||||
h("div", { class: "page-head" }, h("div", { class: "grow" }, h("h1", null, "Import a ruleset"),
|
||||
h("div", { class: "muted" }, "Turns an nftables ruleset into zones, sets, services, rules, port forwards and NAT. On the firewall, 'sudo hc import --defines /etc/nftables.conf -n' does the same from the command line."))),
|
||||
h("div", { class: "panel" }, h("div", { class: "form-grid" },
|
||||
field("Ruleset (JSON)", json), field("Ruleset (text)", text), field("Ruleset file", defines), field("Table", table), h("label"), replace),
|
||||
field("Ruleset (JSON)", json), field("Ruleset (text)", text), field("Ruleset file", defines), field("Table", table),
|
||||
field("Zone names", zones, "Name zones by what the network is for; interfaces given one name share a zone. Otherwise a define holding the interface names it, or the interface's own name."), h("label"), replace),
|
||||
h("div", { class: "actions form-actions" },
|
||||
h("button", { onclick: () => run(true) }, "Preview"),
|
||||
h("button", { class: "primary", onclick: () => run(false) }, "Import"))),
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
// Matrix: what the rules do to new connections between zones, and to and
|
||||
// from the firewall itself.
|
||||
import { h, mount, modal, badge } from "../ui.js";
|
||||
import { get, state } from "../api.js";
|
||||
|
||||
export async function render(el) {
|
||||
const m = await get("/matrix");
|
||||
const cols = [...m.zones, "firewall"];
|
||||
const cells = new Map(m.cells.map((c) => [c.from + "\u0000" + c.to, c]));
|
||||
const pending = state.draft && !state.immediate && (state.draft.ops || []).length;
|
||||
const name = (z) => z === "firewall" ? h("em", null, "firewall") : z;
|
||||
mount(el,
|
||||
h("div", { class: "page-head" }, h("div", { class: "grow" }, h("h1", null, "Zone matrix"),
|
||||
h("div", { class: "muted" }, "What happens to a new connection from a zone (rows) to a zone (columns), or to and from the firewall itself, worked out from the rules in order: allowed or blocked, and how many rules decide for part of the traffic first (some services, addresses, times). Click a cell for the rules."))),
|
||||
pending ? h("div", { class: "msg info" }, "Shown with your pending changes.") : null,
|
||||
!m.zones.length ? h("div", { class: "panel" }, h("div", { class: "empty" }, "No zones yet: add them under Objects → Zones.")) :
|
||||
h("div", { class: "panel" }, h("div", { class: "table-wrap" }, h("table", { class: "matrix" },
|
||||
h("thead", null, h("tr", null, h("th", { class: "corner" }, "from ↓ to →"), cols.map((c) => h("th", null, name(c))))),
|
||||
h("tbody", null, cols.map((from) => h("tr", null, h("th", null, name(from)), cols.map((to) => {
|
||||
const c = cells.get(from + "\u0000" + to);
|
||||
if (!c) return h("td", { class: "matrix-none" }, "–");
|
||||
const n = c.exceptions.length;
|
||||
return h("td", { class: "matrix-" + c.verdict + (n ? " matrix-some" : ""), title: c.verdict + " by " + c.decided_by + (n ? ", " + n + " exception" + (n === 1 ? "" : "s") + " first" : "") },
|
||||
h("button", { type: "button", class: "matrix-cell", onclick: () => detail(c) }, c.verdict, n ? h("span", { class: "matrix-n" }, "+" + n) : null));
|
||||
})))))),
|
||||
h("p", { class: "muted small" }, "Zone to zone is the forward chain; to the firewall, input; from it, output. Exceptions are rules (and built-ins such as ping or port forwards) that decide for some of the traffic before the rest falls to the deciding rule or the policy. Replies and established connections aren't shown: the built-in lets those through.")));
|
||||
}
|
||||
|
||||
function detail(c) {
|
||||
const ruleLink = (id, chain, label) => id ? h("a", { href: "#/rules?chain=" + chain, onclick: () => document.querySelector(".modal-back") && document.querySelector(".modal-back").remove() }, label) : label;
|
||||
const v = (verdict) => badge(verdict, { allowed: "ok", blocked: "err", logged: "", unknown: "warn" }[verdict]);
|
||||
modal(c.from + " → " + c.to, h("div", null,
|
||||
h("p", { class: "muted" }, "New connections, checked in this order (the " + c.chain + " chain):"),
|
||||
h("div", { class: "table-wrap" }, h("table", null, h("tbody", null,
|
||||
c.exceptions.map((e) => h("tr", null, h("td", null, v(e.verdict)), h("td", null, e.what), h("td", { class: "small" }, ruleLink(e.rule_id, e.chain, e.label)))),
|
||||
h("tr", { class: "matrix-rest" }, h("td", null, v(c.verdict)), h("td", null, h("strong", null, c.exceptions.length ? "the rest" : "everything")),
|
||||
h("td", { class: "small" }, ruleLink(c.rule_id, c.chain, c.decided_by))))))), [{ label: "Close", value: null }], { wide: true });
|
||||
}
|
||||
@@ -3,12 +3,12 @@
|
||||
import { h, mount, table, modal, errorBox, field, input, checkbox, confirmDialog, badge } from "../ui.js";
|
||||
import { get, change, can } from "../api.js";
|
||||
import { rerender } from "../app.js";
|
||||
import { names, matchEditor, describe, hits, counters, listEditor } from "../widgets.js";
|
||||
import { names, matchEditor, describe, hits, counters, listEditor, rowClass } from "../widgets.js";
|
||||
|
||||
export async function render(el) {
|
||||
const [fwds, snats, redirects, cs] = await Promise.all([get("/forwards"), get("/snats"), get("/redirects"), counters()]);
|
||||
const write = can("config", true);
|
||||
const actions = (kind, x, edit) => write ? [
|
||||
const actions = (kind, x, edit) => x.pending === "deleted" ? badge("deleted", "err") : write ? [
|
||||
h("button", { class: "small reveal", onclick: () => edit(x) }, "Edit"), " ",
|
||||
h("button", { class: "small reveal", onclick: () => toggle(kind, x) }, x.disabled ? "Enable" : "Disable"), " ",
|
||||
h("button", { class: "small reveal", onclick: () => del(kind, x) }, "Delete")] : null;
|
||||
@@ -25,7 +25,7 @@ export async function render(el) {
|
||||
{ label: "Hairpin", render: (f) => f.hairpin ? badge((f.hairpin_sources || []).length ? "from " + f.hairpin_sources.join(", ") : "yes", "ok") : "" },
|
||||
{ label: "Hits", render: (f) => hits(cs, "f" + f.id, "f" + f.id + "-6") },
|
||||
{ label: "", tdClass: "actions-cell", render: (f) => actions("forwards", f, forwardDialog) },
|
||||
], fwds, { empty: "No port forwards.", rowClass: (x) => x.disabled ? "disabled" : "" })),
|
||||
], fwds, { empty: "No port forwards.", rowClass })),
|
||||
h("div", { class: "panel" },
|
||||
h("div", { class: "panel-head" }, h("h2", null, "Source NAT"), write ? h("button", { class: "primary small", onclick: () => snatDialog(null) }, "Add source NAT") : null),
|
||||
table([
|
||||
@@ -35,7 +35,7 @@ export async function render(el) {
|
||||
{ label: "Hits", render: (x) => hits(cs, "s" + x.id) },
|
||||
{ label: "Comment", render: (x) => x.comment },
|
||||
{ label: "", tdClass: "actions-cell", render: (x) => actions("snats", x, snatDialog) },
|
||||
], snats, { empty: "No source NAT.", rowClass: (x) => x.disabled ? "disabled" : "" })),
|
||||
], snats, { empty: "No source NAT.", rowClass })),
|
||||
h("div", { class: "panel" },
|
||||
h("div", { class: "panel-head" }, h("h2", null, "Redirects"), write ? h("button", { class: "primary small", onclick: () => redirectDialog(null) }, "Add redirect") : null),
|
||||
table([
|
||||
@@ -44,7 +44,7 @@ export async function render(el) {
|
||||
{ label: "Hits", render: (x) => hits(cs, "d" + x.id) },
|
||||
{ label: "Comment", render: (x) => x.comment },
|
||||
{ label: "", tdClass: "actions-cell", render: (x) => actions("redirects", x, redirectDialog) },
|
||||
], redirects, { empty: "No redirects.", rowClass: (x) => x.disabled ? "disabled" : "" })));
|
||||
], redirects, { empty: "No redirects.", rowClass })));
|
||||
}
|
||||
|
||||
async function toggle(kind, x) {
|
||||
@@ -78,13 +78,13 @@ async function dialog(title, kind, x, fields, value) {
|
||||
async function forwardDialog(f) {
|
||||
const n = await names();
|
||||
f = f || { protocols: ["tcp"], hairpin: true };
|
||||
const name = input({ value: f.name || "", placeholder: "jellyfin" });
|
||||
const name = input({ value: f.name || "", placeholder: "web" });
|
||||
const tcp = checkbox("TCP", f.protocols.includes("tcp")), udp = checkbox("UDP", f.protocols.includes("udp"));
|
||||
const ports = input({ value: f.ports || "", placeholder: "8096 or 7777-7780", class: "mono" });
|
||||
const ports = input({ value: f.ports || "", placeholder: "8080, or a range: 27015-27030", class: "mono" });
|
||||
const pub = listEditor(f.public_addresses, { suggestions: n.sets, placeholder: "your public IPv4 address, or a set", addLabel: "Add address" });
|
||||
const to = input({ value: f.to || "", placeholder: "10.0.10.57", class: "mono" });
|
||||
const to = input({ value: f.to || "", placeholder: "192.168.1.10", class: "mono" });
|
||||
const toPort = input({ value: f.to_port || "", placeholder: "the same", class: "mono short" });
|
||||
const to6 = input({ value: f.to6 || "", placeholder: "2001:db8:10::57 (optional)", class: "mono" });
|
||||
const to6 = input({ value: f.to6 || "", placeholder: "2001:db8:1::10 (optional)", class: "mono" });
|
||||
const src = listEditor(f.sources, { suggestions: n.sets, placeholder: "anywhere", addLabel: "Add source" });
|
||||
const hairpin = checkbox("the public address also works from inside (hairpin NAT)", f.hairpin);
|
||||
const hairSrc = listEditor(f.hairpin_sources, { suggestions: n.sets, placeholder: "all inside sources", addLabel: "Add source" });
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
// Rules: each chain's rules in order, with their hits; add, edit, move.
|
||||
import { h, mount, table, modal, errorBox, field, input, select, checkbox, confirmDialog, badge, textarea } from "../ui.js";
|
||||
import { get, change, can } from "../api.js";
|
||||
import { h, mount, table, modal, errorBox, field, input, select, checkbox, confirmDialog, badge, textarea, toast } from "../ui.js";
|
||||
import { get, api, change, can } from "../api.js";
|
||||
import { rerender } from "../app.js";
|
||||
import { names, matchEditor, describe, hits, counters, listEditor } from "../widgets.js";
|
||||
import { names, matchEditor, describe, hits, counters, listEditor, rowClass, live, fmtBytes, quotas } from "../widgets.js";
|
||||
|
||||
const chains = [
|
||||
["input", "Input", "Traffic to the firewall itself."],
|
||||
@@ -25,49 +25,166 @@ const builtins = {
|
||||
|
||||
export async function render(el, params, query) {
|
||||
const chain = query.get("chain") || "input";
|
||||
const [rules, settings, cs] = await Promise.all([get("/rules", { chain }), get("/settings"), counters()]);
|
||||
const [all, settings, cs, qs] = await Promise.all([get("/rules", { chain }), get("/settings"), counters(), quotas()]);
|
||||
// As nftables checks them: the rules that count traffic first.
|
||||
const rules = [...all.filter(everyPacket), ...all.filter((r) => !everyPacket(r))];
|
||||
const write = can("config", true);
|
||||
const info = chains.find((c) => c[0] === chain) || chains[0];
|
||||
const policy = settings[chain + "_policy"];
|
||||
const bi = (builtins[chain] || []).filter(([flag]) => flag === "" || settings.builtins[flag]);
|
||||
const actionBadge = (r) => badge(r.action + (r.log ? " + log" : ""), { accept: "ok", drop: "err", reject: "warn", log: "" }[r.action]);
|
||||
const actionBadge = (r) => r.raw ? badge("as written", "") : badge(r.action + (r.log ? " + log" : ""), { accept: "ok", drop: "err", reject: "warn", log: "" }[r.action]);
|
||||
mount(el,
|
||||
h("div", { class: "page-head" },
|
||||
h("div", { class: "grow" }, h("h1", null, "Rules"), h("div", { class: "muted" }, "Checked top to bottom; the first rule that decides wins. ", info[2])),
|
||||
write ? h("div", { class: "actions" }, h("button", { class: "primary", onclick: () => ruleDialog(null, chain) }, "Add rule")) : null),
|
||||
h("div", { class: "tabs" }, chains.map(([k, label]) => h("a", { href: "#/rules?chain=" + k, class: k === chain ? "active" : "" }, label))),
|
||||
h("div", { class: "panel" },
|
||||
bi.length ? h("div", { class: "builtins muted small" }, h("strong", null, "Added by Housecarl first: "),
|
||||
bi.map(([, mark, text], i) => [i ? " · " : "", text, mark ? [" (", hits(cs, "b-" + mark), ")"] : ""]), " — ",
|
||||
h("a", { href: "#/settings" }, "settings")) : null,
|
||||
builtinsRow(chain, settings, cs),
|
||||
table([
|
||||
{ label: "#", render: (r, i) => String(i + 1) },
|
||||
{ label: "#", render: (r) => r.pending === "deleted" ? "" : String(live(rules).indexOf(r) + 1) },
|
||||
{ label: "Match", render: (r) => r.raw ? [badge("raw", ""), " ", h("span", { class: "mono small" }, r.raw)]
|
||||
: [describe(r), extras(r)] },
|
||||
: [everyPacket(r) ? [badge("every packet", "accent"), " "] : null, describe(r), extras(r, qs[r.id])] },
|
||||
{ label: "Action", render: actionBadge },
|
||||
{ label: "Hits", render: (r) => hits(cs, "r" + r.id) },
|
||||
{ label: "Comment", render: (r) => r.comment },
|
||||
write ? { label: "", tdClass: "actions-cell", render: (r, i) => [
|
||||
h("button", { class: "small reveal", title: "Move up", disabled: i === 0, onclick: () => move(r, { before: rules[i - 1].id }) }, "↑"), " ",
|
||||
h("button", { class: "small reveal", title: "Move down", disabled: i === rules.length - 1, onclick: () => move(r, { after: rules[i + 1].id }) }, "↓"), " ",
|
||||
write ? { label: "", tdClass: "actions-cell", render: (r) => {
|
||||
if (r.pending === "deleted") return badge("deleted", "err");
|
||||
// Moves stay among the rules of the same kind (counting traffic,
|
||||
// or not): nftables checks those first.
|
||||
const rs = live(rules).filter((x) => everyPacket(x) === everyPacket(r)), i = rs.indexOf(r);
|
||||
return [
|
||||
h("button", { class: "small reveal", title: "Move up", disabled: i === 0, onclick: () => move(r, { before: rs[i - 1].id }) }, "↑"), " ",
|
||||
h("button", { class: "small reveal", title: "Move down", disabled: i === rs.length - 1, onclick: () => move(r, { after: rs[i + 1].id }) }, "↓"), " ",
|
||||
h("button", { class: "small reveal", onclick: () => ruleDialog(r, chain) }, "Edit"), " ",
|
||||
h("button", { class: "small reveal", onclick: () => toggle(r) }, r.disabled ? "Enable" : "Disable"), " ",
|
||||
h("button", { class: "small reveal", onclick: () => del(r) }, "Delete"),
|
||||
] } : null,
|
||||
].filter(Boolean), rules, { empty: "No rules in this chain yet.", rowClass: (r) => r.disabled ? "disabled" : "" }),
|
||||
];
|
||||
} } : null,
|
||||
].filter(Boolean), rules, { empty: "No rules in this chain yet.", rowClass }),
|
||||
rules.some(everyPacket) ? h("div", { class: "muted small" }, "Rules marked “every packet” are checked before established connections are accepted: they see all of a connection's traffic, not only its first packet, so a quota counts all of it and a block stops connections already open. They can drop, reject or log, not accept.") : null,
|
||||
h("div", { class: "policy small" }, "Then the policy: ", badge(policy, policy === "drop" ? "err" : "ok"),
|
||||
policy === "drop" ? [" — ", hits(cs, "p-" + chain), " packets reached it"] : null)));
|
||||
}
|
||||
|
||||
function extras(r) {
|
||||
// everyPacket: checked before established connections are accepted (as
|
||||
// model.Rule.ChecksEveryPacket): asked to, or a quota or a byte rate.
|
||||
function everyPacket(r) {
|
||||
return !r.raw && (!!r.every_packet || !!r.quota || !!(r.limit && ["bytes", "kbytes", "mbytes"].includes(r.limit.unit)));
|
||||
}
|
||||
|
||||
// What each built-in does (a switch covers all its rules).
|
||||
const builtinText = {
|
||||
loopback: "Accept everything on lo (the firewall talking to itself)",
|
||||
established: "Accept established and related connections: replies, and the rest of connections a rule let in",
|
||||
drop_invalid: "Drop invalid packets: what connection tracking can't place",
|
||||
icmpv6: "Accept the ICMPv6 that IPv6 needs: errors, echo, neighbour discovery, MLD (RFC 4890)",
|
||||
ping_internal: "Accept ping to the firewall from internal zones",
|
||||
dhcpv6_client: "Accept DHCPv6 replies on external zones (your ISP's prefix delegation)",
|
||||
antispoof: "Drop sources that can't come from outside, on external zones (Settings: the lists and your prefixes)",
|
||||
mss_clamp: "Clamp TCP's MSS to the route's MTU on external zones (needed with PPPoE)",
|
||||
};
|
||||
|
||||
// builtinsRow sums up the chain's built-ins in one line, with a button to
|
||||
// see and switch them.
|
||||
function builtinsRow(chain, settings, cs) {
|
||||
const list = builtins[chain] || [];
|
||||
const on = list.filter(([flag]) => flag === "" || settings.builtins[flag]);
|
||||
if (!list.length) return null;
|
||||
return h("div", { class: "builtins muted small" },
|
||||
on.length ? (on.length === 1 ? "Housecarl adds 1 rule before yours. " : "Housecarl adds " + on.length + " rules before yours. ") : "Housecarl adds no rules before yours here. ",
|
||||
h("button", { class: "small", onclick: () => builtinsDialog(chain, settings, cs) }, "Built-ins…"));
|
||||
}
|
||||
|
||||
// builtinsDialog lists the chain's built-ins: what each does, its hits,
|
||||
// and a switch (a setting, so it covers every chain the built-in is in).
|
||||
async function builtinsDialog(chain, settings, cs) {
|
||||
const list = builtins[chain] || [];
|
||||
const flags = [...new Set(list.map(([flag]) => flag))];
|
||||
const write = can("config", true);
|
||||
const boxes = {};
|
||||
const rows = flags.map((flag) => {
|
||||
const marks = list.filter(([f]) => f === flag).map(([, mark]) => mark).filter(Boolean);
|
||||
const box = flag ? h("input", { type: "checkbox", checked: settings.builtins[flag], disabled: !write }) : null;
|
||||
if (box) boxes[flag] = box;
|
||||
return { flag, marks, box, text: flag ? builtinText[flag] : list.find(([f]) => f === flag)[2] };
|
||||
});
|
||||
const err = h("div");
|
||||
const body = h("div", null,
|
||||
h("p", { class: "muted" }, "Checked in this order, before your rules (rules with a quota or a byte rate come before the established built-in, so they see every packet). Each is a setting: switching one changes every chain it is in."),
|
||||
table([
|
||||
{ label: "On", render: (x) => x.box || h("span", { class: "muted small" }, "always") },
|
||||
{ label: "Built-in", render: (x) => x.flag ? x.text : [x.text, h("div", { class: "muted small" }, "Part of each port forward.")] },
|
||||
{ label: "Hits", render: (x) => x.marks.length ? hits(cs, ...x.marks.map((m) => "b-" + m)) : h("span", { class: "muted" }, "–") },
|
||||
], rows),
|
||||
err);
|
||||
await modal("Built-ins of the " + chain + " chain", body, write ? [
|
||||
{ label: "Cancel", value: null },
|
||||
{ label: "Save", kind: "primary", onclick: async () => {
|
||||
mount(err);
|
||||
const b = {};
|
||||
for (const [flag, box] of Object.entries(boxes)) if (box.checked !== settings.builtins[flag]) b[flag] = box.checked;
|
||||
if (!Object.keys(b).length) return true;
|
||||
try { await change("PATCH", "/settings", { builtins: b }); rerender(); return true; } catch (e) { mount(err, errorBox(e)); return false; }
|
||||
} },
|
||||
] : [{ label: "Close", value: null }], { wide: true });
|
||||
}
|
||||
|
||||
function extras(r, q) {
|
||||
const x = [];
|
||||
if ((r.ct_state || []).length) x.push("state " + r.ct_state.join(","));
|
||||
if ((r.ct_status || []).length) x.push("status " + r.ct_status.join(","));
|
||||
if ((r.days || []).length) x.push(r.days.join(","));
|
||||
if (r.hour_from) x.push(r.hour_from + "–" + r.hour_to);
|
||||
if (r.limit) x.push((r.limit.over ? "over " : "") + r.limit.rate + "/" + r.limit.per + (r.limit.per_source ? " per source" : ""));
|
||||
if (r.limit) {
|
||||
const l = r.limit;
|
||||
x.push((l.over ? "over " : "") + l.rate + (l.unit && l.unit !== "packets" ? " " + l.unit : "") + "/" + l.per +
|
||||
(l.per_source ? " per source" : l.per_destination ? " per destination" : ""));
|
||||
}
|
||||
if (r.conn_limit) x.push("over " + r.conn_limit + " connections");
|
||||
return x.length ? h("div", { class: "muted small" }, x.join(" · ")) : null;
|
||||
const out = x.length ? [h("div", { class: "muted small" }, x.join(" · "))] : [];
|
||||
if (r.quota) out.push(quotaLine(r, q));
|
||||
return out;
|
||||
}
|
||||
|
||||
// quotaLine shows a quota, and what it used: in total, or per address
|
||||
// (the busiest, and the rest in a dialog).
|
||||
function quotaLine(r, q) {
|
||||
const qt = r.quota;
|
||||
const each = qt.per_source ? " each source" : qt.per_destination ? " each destination" : "";
|
||||
const resets = qt.reset === "monthly" ? "monthly on day " + (qt.day || 1) : qt.reset || "by hand";
|
||||
let used = null;
|
||||
if (q && !qt.per_source && !qt.per_destination) {
|
||||
used = [h("strong", null, fmtBytes(q.used)), " of " + fmtBytes(qt.bytes) + " used"];
|
||||
} else if (q) {
|
||||
const n = (q.addresses || []).length;
|
||||
used = [n === 1 ? "1 address" : n + " addresses", n ? [", busiest ", h("strong", null, fmtBytes(q.addresses[0].used))] : null, " ",
|
||||
n ? h("button", { class: "small", onclick: () => usageDialog(r, q) }, "Usage…") : null];
|
||||
}
|
||||
return h("div", { class: "muted small" }, (qt.over ? "quota over " : "quota ") + fmtBytes(qt.bytes) + each + ", resets " + resets,
|
||||
used ? [" · ", used] : null,
|
||||
q && q.next_reset ? " · next reset " + new Date(q.next_reset).toLocaleString() : null,
|
||||
can("config", true) && q && r.pending !== "new" ? [" ", h("button", { class: "small", onclick: () => resetQuota(r) }, "Reset")] : null);
|
||||
}
|
||||
|
||||
async function usageDialog(r, q) {
|
||||
await modal("Quota usage of " + (r.comment || "rule #" + r.id), h("div", null,
|
||||
h("p", { class: "muted" }, "Each " + (r.quota.per_source ? "source" : "destination") + " has " + fmtBytes(r.quota.bytes) + "."),
|
||||
table([
|
||||
{ label: "Address", render: (a) => h("span", { class: "mono" }, a.address) },
|
||||
{ label: "Used", render: (a) => fmtBytes(a.used) },
|
||||
{ label: "", render: (a) => h("div", { class: "usage" }, h("div", { class: "usage-bar" + (a.used >= r.quota.bytes ? " full" : "") }, bar(a.used / r.quota.bytes))) },
|
||||
], q.addresses)), [{ label: "Close", value: null }], { wide: true });
|
||||
}
|
||||
|
||||
function bar(f) {
|
||||
const seg = h("div", { class: "seg leased" });
|
||||
seg.style.width = Math.min(100, f * 100) + "%";
|
||||
return seg;
|
||||
}
|
||||
|
||||
async function resetQuota(r) {
|
||||
if (!await confirmDialog("Reset the quota", "Start the quota of " + (r.comment || "rule #" + r.id) + " again now" +
|
||||
(r.quota.per_source || r.quota.per_destination ? ", for every address" : "") + "? This is done at once, not staged.", "Reset")) return;
|
||||
try { await api("POST", "/rules/" + r.id + "/reset-quota"); toast("Quota reset"); rerender(); } catch (e) { modal("Not reset", errorBox(e)); }
|
||||
}
|
||||
|
||||
async function move(r, where) {
|
||||
@@ -87,11 +204,15 @@ async function del(r) {
|
||||
export async function ruleDialog(r, chain) {
|
||||
const n = await names();
|
||||
r = r || { chain, action: "accept" };
|
||||
const ch = select([["input", "input (to the firewall)"], ["forward", "forward (through it)"], ["output", "output (from it)"], ["early", "early (before conntrack)"]], r.chain || chain);
|
||||
const ch = select(["input", "forward", "output", "early"], r.chain || chain);
|
||||
const match = matchEditor(r, n);
|
||||
const action = select([["accept", "accept"], ["drop", "drop"], ["reject", "reject"], ["log", "log only (and go on)"]], r.action);
|
||||
const rejectWith = select([["", "with an ICMP error"], ["tcp-reset", "with a TCP reset (TCP rules only)"]], r.reject_with || "");
|
||||
const showReject = () => rejectWith.classList.toggle("hidden", action.value !== "reject");
|
||||
const every = checkbox("every packet: also stop connections already open, not just new ones", r.every_packet);
|
||||
const showReject = () => {
|
||||
rejectWith.classList.toggle("hidden", action.value !== "reject");
|
||||
every.classList.toggle("hidden", action.value === "accept");
|
||||
};
|
||||
action.addEventListener("change", showReject);
|
||||
showReject();
|
||||
const states = listEditor(r.ct_state, { suggestions: ["new", "established", "related", "invalid", "untracked"], placeholder: "new", mono: false, addLabel: "Add state" });
|
||||
@@ -101,33 +222,122 @@ export async function ruleDialog(r, chain) {
|
||||
const hourTo = input({ value: r.hour_to || "", placeholder: "17:30", class: "short" });
|
||||
const l = r.limit || {};
|
||||
const rate = input({ value: l.rate || "", placeholder: "10", class: "short", type: "number", min: 1 });
|
||||
const per = select(["second", "minute", "hour", "day"], l.per || "minute");
|
||||
const unit = select([["", "packets"], ["bytes", "bytes"], ["kbytes", "KB"], ["mbytes", "MB"]], l.unit || "", { class: "inline" });
|
||||
const per = select(["second", "minute", "hour", "day"], l.per || "minute", { class: "inline" });
|
||||
const burst = input({ value: l.burst || "", placeholder: "burst", class: "short", type: "number", min: 0 });
|
||||
const over = checkbox("match beyond the rate (to drop floods)", l.over);
|
||||
const perSource = checkbox("per source address", l.per_source);
|
||||
const over = checkbox("match beyond the rate (to drop floods, or traffic over a bandwidth)", l.over);
|
||||
const keyOf = (x) => x.per_source ? "source" : x.per_destination ? "destination" : "";
|
||||
const keys = [["", "for everything the rule matches"], ["source", "for each source address"], ["destination", "for each destination address"]];
|
||||
const limitKey = select(keys, keyOf(l), { class: "inline" });
|
||||
const qt = r.quota || {};
|
||||
const gb = qt.bytes && qt.bytes % (1 << 30) === 0;
|
||||
const quotaAmount = input({ value: qt.bytes ? qt.bytes / (gb ? 1 << 30 : 1 << 20) : "", placeholder: "50", class: "short", type: "number", min: 1 });
|
||||
const quotaUnit = select([["mb", "MB"], ["gb", "GB"]], gb || !qt.bytes ? "gb" : "mb", { class: "inline" });
|
||||
const quotaOver = checkbox("match once it is used up (to drop); otherwise while it lasts (to accept)", qt.over === undefined ? true : qt.over);
|
||||
const quotaKey = select(keys, keyOf(qt), { class: "inline" });
|
||||
const quotaReset = select([["", "only by hand"], ["daily", "daily, at midnight"], ["weekly", "weekly, on Mondays"], ["monthly", "monthly, on day"]], qt.reset || "", { class: "inline" });
|
||||
const quotaDay = input({ value: qt.day || "", placeholder: "1", class: "short", type: "number", min: 1, max: 31 });
|
||||
const showDay = () => quotaDay.classList.toggle("hidden", quotaReset.value !== "monthly");
|
||||
quotaReset.addEventListener("change", showDay);
|
||||
showDay();
|
||||
const group4 = input({ value: r.group_v4 || "", placeholder: "32", class: "short", type: "number", min: 1, max: 32 });
|
||||
const group6 = input({ value: r.group_v6 || "", placeholder: "64", class: "short", type: "number", min: 1, max: 128 });
|
||||
const conn = input({ value: r.conn_limit || "", placeholder: "e.g. 20", class: "short", type: "number", min: 1 });
|
||||
const log = checkbox("log matching packets (Log page)", r.log);
|
||||
const logPrefix = input({ value: r.log_prefix || "", placeholder: "default: hc CHAIN #ID ACTION:" });
|
||||
const raw = textarea(r.raw || "", { rows: 2, class: "mono", placeholder: "iifname != \"wan\" ct state invalid ip daddr @trusted_v4 accept" });
|
||||
const raw = textarea(r.raw || "", { rows: 2, class: "mono", placeholder: "ip saddr @admins_v4 tcp dport 22 ct state new accept" });
|
||||
const comment = input({ value: r.comment || "" });
|
||||
const disabled = checkbox("disabled: keep the rule but don't use it", r.disabled);
|
||||
const err = h("div");
|
||||
|
||||
// The parts most rules don't need are behind "+" buttons: open when the
|
||||
// rule uses them, closed (and unset) with ✕.
|
||||
const adders = h("div", { class: "adders" });
|
||||
const sections = {};
|
||||
const section = (key, label, open, fields) => {
|
||||
const nodes = fields.flat().filter(Boolean);
|
||||
const sec = { open };
|
||||
const add = h("button", { type: "button", class: "small", onclick: () => show(true) }, "+ " + label);
|
||||
const close = h("button", { type: "button", class: "small close-section", title: "Remove " + label.toLowerCase(), onclick: () => show(false) }, "✕");
|
||||
nodes[0].append(" ", close);
|
||||
const show = (on) => {
|
||||
sec.open = on;
|
||||
for (const n of nodes) n.classList.toggle("hidden", !on);
|
||||
add.classList.toggle("hidden", on);
|
||||
if (sec.onShow) sec.onShow(on);
|
||||
if (on) { const f = nodes.find((n) => n.querySelector && n.querySelector("input, select, textarea")); if (f) f.querySelector("input, select, textarea").focus(); }
|
||||
updateGroup();
|
||||
};
|
||||
adders.append(add, " ");
|
||||
sections[key] = sec;
|
||||
sec.nodes = nodes;
|
||||
sec.init = () => show(open);
|
||||
return nodes;
|
||||
};
|
||||
const stateFields = section("state", "Connection state", (r.ct_state || []).length > 0 || (r.ct_status || []).length > 0, [
|
||||
field("Connection state", states, "new: a connection's first packet; established: its later ones; related: packets that belong to another connection (ICMP errors, FTP data); invalid: what connection tracking can't place. Empty: any."),
|
||||
field("Connection status", statuses, "dnat: the connection was port forwarded; snat: its source was translated. Empty: any."),
|
||||
]);
|
||||
const timeFields = section("time", "Time window", (r.days || []).length > 0 || !!r.hour_from, [
|
||||
field("Days", days),
|
||||
field("Hours", h("div", null, hourFrom, " to ", hourTo), "22:00 to 06:00 passes midnight. In the firewall's local time."),
|
||||
]);
|
||||
const rateFields = section("rate", "Rate limit", !!r.limit, [
|
||||
field("Rate limit", h("div", null, rate, " ", unit, " per ", per, " ", burst, " ", limitKey, over),
|
||||
"In bytes, it is a bandwidth (KB or MB per second or minute); traffic beyond it is dropped, not queued. The burst is in the same unit."),
|
||||
]);
|
||||
const quotaFields = section("quota", "Quota", !!r.quota, [
|
||||
field("Quota", h("div", null, quotaAmount, " ", quotaUnit, " ", quotaKey, " resets ", quotaReset, " ", quotaDay, quotaOver),
|
||||
"An allowance of traffic; Housecarl keeps the count across changes and restarts."),
|
||||
]);
|
||||
const connFields = section("conn", "Connection limit", !!r.conn_limit, [
|
||||
field("Connection limit", conn, "Match a source that already has more than this many connections."),
|
||||
]);
|
||||
const rawFields = section("raw", "Write it in nftables", !!r.raw, [
|
||||
field("Raw nftables", raw, "One nftables rule, for what Housecarl can't say, used as written: it replaces the zones, addresses, services, action, limits and log (write them in it). Checked by nftables."),
|
||||
]);
|
||||
// Grouping only means something with a per-address limit, quota or
|
||||
// connection limit.
|
||||
const groupFields = field("Group addresses", h("div", null, "IPv4 /", group4, " IPv6 /", group6),
|
||||
"Per-address limits, connection limits and quotas count addresses by these prefixes. Default: each IPv4 address, IPv6 by /64 (one device has several addresses).").filter(Boolean);
|
||||
function updateGroup() {
|
||||
if (!sections.raw) return; // still building
|
||||
const on = (sections.rate.open && limitKey.value !== "") || (sections.quota.open && quotaKey.value !== "") || sections.conn.open;
|
||||
for (const n of groupFields) n.classList.toggle("hidden", !on);
|
||||
}
|
||||
limitKey.addEventListener("change", updateGroup);
|
||||
quotaKey.addEventListener("change", updateGroup);
|
||||
|
||||
// Written in nftables, a rule is its text: the fields it replaces go
|
||||
// (and the other parts close).
|
||||
const matchNodes = match.flat().filter(Boolean);
|
||||
const actionField = field("Action", h("div", null, action, " ", rejectWith, every),
|
||||
"Every packet: checked before established connections are accepted, so blocking takes effect on what is already connected.").filter(Boolean);
|
||||
const logField = field("Log", h("div", null, log, logPrefix)).filter(Boolean);
|
||||
const addersLabel = h("label");
|
||||
sections.raw.onShow = (on) => {
|
||||
for (const n of [...matchNodes, ...actionField, ...logField]) n.classList.toggle("hidden", on);
|
||||
for (const [k, sec] of Object.entries(sections)) {
|
||||
if (k === "raw") continue;
|
||||
if (on && sec.open) for (const n of sec.nodes) n.classList.add("hidden");
|
||||
if (on) sec.open = false;
|
||||
}
|
||||
for (const b of adders.children) if (!b.textContent.includes("nftables")) b.classList.toggle("hidden", on);
|
||||
updateGroup();
|
||||
};
|
||||
const body = h("div", null,
|
||||
h("div", { class: "form-grid" },
|
||||
field("Chain", ch),
|
||||
...match.flat(),
|
||||
field("Action", h("div", null, action, " ", rejectWith)),
|
||||
field("Connection state", states),
|
||||
field("Connection status", statuses, "dnat: the connection was port forwarded."),
|
||||
field("Days", days),
|
||||
field("Hours", h("div", null, hourFrom, " to ", hourTo), "22:00 to 06:00 passes midnight."),
|
||||
field("Rate limit", h("div", null, rate, " packets per ", per, " ", burst, over, perSource), "Empty: no limit."),
|
||||
field("Connection limit", conn, "Match a source that already has more than this many connections."),
|
||||
field("Log", h("div", null, log, logPrefix)),
|
||||
field("Raw nftables", raw, "Instead of all the above: one nftables rule, for what Housecarl can't say. Checked by nftables."),
|
||||
...matchNodes,
|
||||
...actionField,
|
||||
...stateFields, ...timeFields, ...rateFields, ...quotaFields, ...connFields, ...groupFields, ...rawFields,
|
||||
addersLabel, adders,
|
||||
...logField,
|
||||
field("Comment", comment),
|
||||
r.id ? [h("label"), disabled] : null),
|
||||
err);
|
||||
for (const sec of Object.values(sections)) sec.init();
|
||||
updateGroup();
|
||||
await modal(r.id ? "Edit rule" : "Add rule", body, [
|
||||
{ label: "Cancel", value: null },
|
||||
{ label: "Check", onclick: () => save(true) },
|
||||
@@ -136,10 +346,21 @@ export async function ruleDialog(r, chain) {
|
||||
|
||||
async function save(dry) {
|
||||
mount(err);
|
||||
const v = { chain: ch.value, ...match.value(), action: action.value, reject_with: action.value === "reject" ? rejectWith.value : "",
|
||||
ct_state: states.value(), ct_status: statuses.value(), days: days.value(), hour_from: hourFrom.value.trim(), hour_to: hourTo.value.trim(),
|
||||
limit: rate.value ? { rate: parseInt(rate.value, 10), per: per.value, burst: parseInt(burst.value || "0", 10), over: over.input.checked, per_source: perSource.input.checked } : null,
|
||||
conn_limit: parseInt(conn.value || "0", 10), log: log.input.checked, log_prefix: logPrefix.value.trim(), raw: raw.value.trim(), comment: comment.value.trim() };
|
||||
const on = (k) => sections[k].open;
|
||||
const isRaw = on("raw");
|
||||
const m = isRaw ? { in_zones: [], not_in: false, out_zones: [], not_out: false, sources: [], not_sources: false, destinations: [], not_destinations: false, services: [], family: "" } : match.value();
|
||||
const v = { chain: ch.value, ...m, action: isRaw ? "accept" : action.value, reject_with: !isRaw && action.value === "reject" ? rejectWith.value : "",
|
||||
ct_state: on("state") ? states.value() : [], ct_status: on("state") ? statuses.value() : [],
|
||||
days: on("time") ? days.value() : [], hour_from: on("time") ? hourFrom.value.trim() : "", hour_to: on("time") ? hourTo.value.trim() : "",
|
||||
limit: on("rate") && rate.value ? { rate: parseInt(rate.value, 10), unit: unit.value, per: per.value, burst: parseInt(burst.value || "0", 10), over: over.input.checked,
|
||||
per_source: limitKey.value === "source", per_destination: limitKey.value === "destination" } : null,
|
||||
quota: on("quota") && quotaAmount.value ? { bytes: parseInt(quotaAmount.value, 10) * (quotaUnit.value === "gb" ? 1 << 30 : 1 << 20), over: quotaOver.input.checked,
|
||||
per_source: quotaKey.value === "source", per_destination: quotaKey.value === "destination",
|
||||
reset: quotaReset.value, day: quotaReset.value === "monthly" ? parseInt(quotaDay.value || "1", 10) : 0 } : null,
|
||||
group_v4: isRaw ? 0 : parseInt(group4.value || "0", 10), group_v6: isRaw ? 0 : parseInt(group6.value || "0", 10),
|
||||
conn_limit: on("conn") ? parseInt(conn.value || "0", 10) : 0,
|
||||
every_packet: !isRaw && action.value !== "accept" && every.input.checked, log: !isRaw && log.input.checked, log_prefix: isRaw ? "" : logPrefix.value.trim(),
|
||||
raw: on("raw") ? raw.value.trim() : "", comment: comment.value.trim() };
|
||||
if (r.id) v.disabled = disabled.input.checked;
|
||||
try {
|
||||
await change(r.id ? "PATCH" : "POST", r.id ? "/rules/" + r.id : "/rules", v, { query: dry ? { dry_run: 1 } : undefined });
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
import { h, mount, table, modal, errorBox, field, input, confirmDialog, badge } from "../ui.js";
|
||||
import { get, change, can } from "../api.js";
|
||||
import { rerender } from "../app.js";
|
||||
import { listEditor } from "../widgets.js";
|
||||
import { listEditor, rowClass } from "../widgets.js";
|
||||
|
||||
export async function render(el) {
|
||||
const services = await get("/services");
|
||||
@@ -16,11 +16,11 @@ export async function render(el) {
|
||||
{ label: "Entries", render: (s) => h("span", { class: "mono small" }, s.entries.join(", ")) },
|
||||
{ label: "", render: (s) => s.builtin ? badge("built in", "") : badge("yours", "accent") },
|
||||
{ label: "Comment", render: (s) => s.comment },
|
||||
write ? { label: "", tdClass: "actions-cell", render: (s) => s.builtin
|
||||
write ? { label: "", tdClass: "actions-cell", render: (s) => s.pending === "deleted" ? badge("deleted", "err") : s.builtin
|
||||
? h("button", { class: "small reveal", onclick: () => serviceDialog({ name: s.name, entries: s.entries }) }, "Change")
|
||||
: [h("button", { class: "small reveal", onclick: () => serviceDialog(s) }, "Edit"), " ",
|
||||
h("button", { class: "small reveal", onclick: () => del(s) }, "Delete")] } : null,
|
||||
].filter(Boolean), services)));
|
||||
].filter(Boolean), services, { rowClass })));
|
||||
}
|
||||
|
||||
async function del(s) {
|
||||
@@ -30,8 +30,8 @@ async function del(s) {
|
||||
|
||||
async function serviceDialog(s) {
|
||||
s = s || { entries: [] };
|
||||
const name = input({ value: s.name || "", placeholder: "jellyfin" });
|
||||
const list = listEditor(s.entries, { suggestions: ["tcp/", "udp/", "icmp/echo-request", "icmpv6/echo-request"], placeholder: "tcp/8096, udp/27015-27030, icmp/echo-request", addLabel: "Add entry" });
|
||||
const name = input({ value: s.name || "", placeholder: "web" });
|
||||
const list = listEditor(s.entries, { suggestions: ["tcp/", "udp/", "icmp/echo-request", "icmpv6/echo-request"], placeholder: "tcp/8080, udp/27015-27030, icmp/echo-request", addLabel: "Add entry" });
|
||||
const comment = input({ value: s.comment || "" });
|
||||
const err = h("div");
|
||||
await modal(s.id ? "Edit service " + s.name : "Add service", h("div", null, h("div", { class: "form-grid" },
|
||||
|
||||
Loaded 100 of 109 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user