Compare commits

..
18 Commits
Author SHA1 Message Date
abrown38andClaude Opus 5.5 6f56eefae2 Release 0.1.1-rc.13
release / release (push) Successful in 1m5s
Housecarl 0.1.1's thirteenth release candidate. Since rc.12: quota usage
is saved after every change. make test and make e2e (304 checks) pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 15:04:04 +01:00
abrown38andClaude Opus 5.5 aaf0ac5206 Quotas: save their usage after every change
The server hooks the engine's OnChange to save what the quotas used, so
a reboot soon after a change that counts a quota differently doesn't
bring back its usage counted the old way (it was saved every five
minutes and on stop only).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 15:04:04 +01:00
abrown38andClaude Opus 5.5 8fa28d01c2 Release 0.1.1-rc.12
release / release (push) Successful in 1m6s
Housecarl 0.1.1's twelfth release candidate. Since rc.11: quota usage
as a percentage; Statistics' quotas link to their rules and include
per-address quotas. make test and make e2e (304 checks) pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 13:55:43 +01:00
abrown38andClaude Opus 5.5 7d23fb13ed Quotas: usage as a share, and their statistics per rule
What a quota used shows its percentage of the allowance on the Rules
page (line and dialog), the dashboard and in hc quotas. Statistics'
Quotas card links each entry to its rule, named, without the share of
the total; per-address quotas are in it at last: their quota:rN series
is the sum of their addresses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 13:55:43 +01:00
abrown38andClaude Opus 5.5 a169e9aa96 Release 0.1.1-rc.11
release / release (push) Successful in 1m3s
Housecarl 0.1.1's eleventh release candidate. Since rc.10: a quota keeps
what it used when its rule counts it differently (per address to a total:
the sum; wider groups: added up), or starts again with a warning. make
test, the real-nft quota test and make e2e (304 checks) pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 12:57:18 +01:00
abrown38andClaude Opus 5.5 9863dd1cdd Quotas: carry what was used when a rule counts differently
model.QuotaCarry says what becomes of a quota's use when its rule
changes how it counts; the firewall's apply follows it, comparing with
the rules in force: per address to a total starts at the addresses' sum,
wider groups add up their addresses, and what can't be carried (a total
to per address, per source to per destination, finer groups) starts
again, with the engine warning quota_restarts on that change. Before,
per source to per destination kept the sources' use as destinations',
and regrouping left old entries behind.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 12:57:18 +01:00
abrown38andClaude Opus 5.5 46c319eced Release 0.1.1-rc.10
release / release (push) Successful in 1m3s
Housecarl 0.1.1's tenth release candidate. Since rc.9: rules read as a
direction ("lan → wan", "any but wan → any"). make test and make e2e
(297 checks) pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 12:12:57 +01:00
abrown38andClaude Opus 5.5 24dbc0a869 Rules read as a direction
A match's zones are written "lan → wan" (or "any but wan → any", and
"lan → firewall" / "firewall → wan" in the input and output chains)
instead of "in lan · out wan" and "in not wan"; exclusions read "from
any but X". The Rules and NAT pages and hc; flags and JSON unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 12:12:57 +01:00
abrown38andClaude Opus 5.5 44b90bd770 Release 0.1.1-rc.9
release / release (push) Successful in 1m3s
Housecarl 0.1.1's ninth release candidate. Since rc.8: a rule's quota is
one line on the Rules page, with its usage, resets and actions in a
dialog. make test passes; the web UI was checked in a browser.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 11:50:04 +01:00
abrown38andClaude Opus 5.5 ff03e76925 Rules: a quota is one line, the rest in its dialog
A rule's quota shows on its row as one line (what it used, how it
resets) that opens a dialog with the rest: the allowance, last and next
reset, usage (each address's, with Reset and Give back), and Reset and
Give back for everyone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 11:50:04 +01:00
abrown38andClaude Opus 5.5 53cd43187e Release 0.1.1-rc.8
release / release (push) Successful in 1m3s
Housecarl 0.1.1's eighth release candidate. Since rc.7: a quota can be
reset for one address, or topped up until its next reset (hc rule
reset-quota --address/--give, Give back in the web UI). make test, the
real-nft quota test and make e2e (297 checks) pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 11:18:13 +01:00
abrown38andClaude Opus 5.5 4c13297f27 Quotas: reset one address, or give some back
POST /rules/{ref}/reset-quota takes an optional address (one entry of a
per-address quota starts again) and give (bytes taken off what was used,
one address's or everyone's, until the next reset). The firewall keeps
the adjustment for its next apply, which writes the new usage into the
ruleset; housecarld applies at once and saves the usage. hc rule
reset-quota --address/--give; Reset and Give back in the web UI, per
address in the usage dialog.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 11:18:13 +01:00
abrown38andClaude Opus 5.5 ba2de59263 Release 0.1.1-rc.7
release / release (push) Successful in 1m4s
Housecarl 0.1.1's seventh release candidate. Since rc.6: names beside
inside addresses, by reverse lookups (names in housecarl.yaml, POST
/names, hc --no-names); hc stats top devices lists each device once.
make test and make e2e (289 checks) pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 10:58:35 +01:00
abrown38andClaude Opus 5.5 42e2013cb2 Names beside inside addresses
Reverse lookups (internal/names, POST /names) put a name beside each
inside address in connections, the log, quota usage, the dashboard's
quotas and the statistics' devices and blocked sources, in the web UI
and hc (--no-names to leave them out). Only inside addresses are looked
up: an inside zone's networks, and private ones outside the external
zones'. Names are cached (names.cache, 10m) and never stored;
names.resolver picks the DNS server.

Also: hc stats top devices listed each device twice; the dashboard's
quotas named the rule's ID twice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 10:58:35 +01:00
abrown38andClaude Opus 5.5 3f24ae0676 Release 0.1.1-rc.6
release / release (push) Successful in 1m5s
Housecarl 0.1.1's sixth release candidate. Since rc.5: the dashboard
lists the rules' quotas. make test passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 10:22:22 +01:00
abrown38andClaude Opus 5.5 1ccca33a30 Dashboard: list the rules' quotas
A Quotas panel with each quota's allowance, use (or, per address, the
count and the busiest), a usage bar and the next reset. The usage bar
moves to widgets.js and the Rules page's usage dialog shares it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 10:20:04 +01:00
abrown38andClaude Opus 5.5 aeaada60ac Release 0.1.1-rc.5
release / release (push) Successful in 1m3s
Housecarl 0.1.1's fifth release candidate. Since rc.4: a rule added
before, after or first is placed in the same change it is created in
(one history entry, kept in a changeset). make test and make e2e (286
checks) pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 21:18:37 +01:00
abrown38andClaude Opus 5.5 05c811bddc Rules: placed in the same change they are created in
hc rule add --before/--after/--first created the rule at the end of its
chain and then moved it: two history entries, two probations, the rule
briefly in force in the wrong place, and in a changeset the place was
lost. POST /rules (and the other ordered kinds) now takes ?before=ID,
?after=ID or ?first=1, kept with the operation when staged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 21:18:37 +01:00
46 changed files with 1472 additions and 116 deletions

No files matched your search

+52
View File
@@ -8,6 +8,29 @@ All notable changes to Housecarl are documented here. The format follows
### Added ### Added
- **Quota usage as a share.** What a quota used shows its percentage
(`12.3 GB of 50.0 GB used (24%)`, each address's in the quota
dialog), on the Rules page, the dashboard and in `hc quotas`.
- **Reset one address's quota, or give some back.** `hc rule
reset-quota ID --address ADDR` starts only that device's allowance
again; `--give 5gbytes` takes that much off what was used instead (one
address's, or everyone's): extra allowance until the next reset,
without changing the rule. In the web UI, a rule's quota is one
line (what it used, how it resets) that opens a dialog with the rest:
usage per address, last and next reset, and Reset and Give back for
everyone or one address. `POST /rules/{ref}/reset-quota` takes `address` and `give`.
- **Names beside inside addresses.** Connections, the log, quota usage,
the dashboard's quotas and the statistics' devices and blocked
sources show each inside address's name, found by a reverse lookup:
`laptop 192.168.1.20` in the web UI, `192.168.1.20 (laptop)` in `hc`.
Only inside addresses are looked up (an inside zone's networks, and
private ones outside the external zones'). Names are cached for 10
minutes and never stored. `names.enabled`, `names.resolver` and
`names.cache` in housecarl.yaml; `hc --no-names`; `POST /names`.
- **Quotas on the dashboard.** With any rule holding a quota, the
dashboard lists them all: the allowance, what is used (for a
per-address quota, how many addresses and the busiest) with a bar, and
the next reset. Each opens its rule.
- **Links to a rule open it.** A rule named in the matrix, the log, the - **Links to a rule open it.** A rule named in the matrix, the log, the
tracer, statistics or the dashboard's busiest rules opens the Rules tracer, statistics or the dashboard's busiest rules opens the Rules
page on its chain, scrolled to it and highlighted (`#/rules?rule=ID`). page on its chain, scrolled to it and highlighted (`#/rules?rule=ID`).
@@ -56,8 +79,37 @@ All notable changes to Housecarl are documented here. The format follows
- Import: sets with `flags timeout` become dynamic sets; a DNAT of - Import: sets with `flags timeout` become dynamic sets; a DNAT of
several ports becomes one port forward per port. several ports becomes one port forward per port.
### Changed
- **Rules read as a direction.** A match's zones are written `lan → wan`,
`any but wan → any`, or with the firewall itself as the other end
(`lan → firewall` in input, `firewall → wan` in output), instead of
`in lan · out wan` and `in not wan`; exclusions read `from any but X`.
On the Rules and NAT pages and in `hc`; the flags and JSON stay as
they were.
### Fixed ### Fixed
- Changing how a quota counts lost or muddled what it used: per address
to a total started at nothing, per source to per destination kept the
sources' use as if it were the destinations', and other groups left
the old entries behind. Now a total starts at the addresses' sum,
wider groups add up their addresses, and what can't be carried starts
again with a warning (`quota_restarts`) on the change.
- What the quotas used is saved after every change, not only every five
minutes: a reboot soon after a change that counts a quota differently
could bring back its usage counted the old way.
- Statistics' Quotas card linked every entry to the Rules page, not to
its rule; each now opens its rule, named. Per-address quotas were
missing from it (and from the `quota:rN` series): they count their
addresses' sum.
- `hc stats top devices` listed each device twice (once for what it
received, once for what it sent); it is one row with both again.
- `hc rule add --before/--after/--first` made two changes (create at the
end, then move), so two history entries, two probations, and the rule
briefly in force at the end of its chain; staged in a changeset, the
place was lost. The create now takes the place (`POST
/rules?before=ID`, `after`, `first`) and makes one change.
- `hc matrix` in a terminal: the colours' escape codes threw the columns - `hc matrix` in a terminal: the colours' escape codes threw the columns
out of line. out of line.
- `hc counters` (`GET /counters`) kept listing the objects of a - `hc counters` (`GET /counters`) kept listing the objects of a
+1 -1
View File
@@ -123,7 +123,7 @@ The web UI is on `http://127.0.0.1:8070/` (see `listen` in
## Status ## Status
Housecarl 0.1.1 is in testing (0.1.1-rc.4): zones, address sets and Housecarl 0.1.1 is in testing (0.1.1-rc.13): zones, address sets and
services with ordered rules, IPv4 and IPv6, port forwards with hairpin NAT services with ordered rules, IPv4 and IPv6, port forwards with hairpin NAT
and IPv6 pinholes, source NAT and redirects, built-ins, probation for and IPv6 pinholes, source NAT and redirects, built-ins, probation for
network changes (confirmed by the client that made them), the ruleset network changes (confirmed by the client that made them), the ruleset
+1 -1
View File
@@ -1 +1 @@
0.1.1-rc.4 0.1.1-rc.13
+30 -2
View File
@@ -298,6 +298,13 @@ func quotasCmd() *cobra.Command {
if err != nil { if err != nil {
return err return err
} }
var addrs []string
for _, q := range qs {
for _, a := range q.Addresses {
addrs = append(addrs, a.Address)
}
}
names := lookupNames(addrs...)
return output(qs, func(w io.Writer) { return output(qs, func(w io.Writer) {
fmt.Fprintln(w, "RULE\tUSED\tOF\tRESETS\tNEXT RESET") fmt.Fprintln(w, "RULE\tUSED\tOF\tRESETS\tNEXT RESET")
for _, q := range qs { for _, q := range qs {
@@ -315,9 +322,13 @@ func quotasCmd() *cobra.Command {
if q.Quota.PerSource || q.Quota.PerDestination { if q.Quota.PerSource || q.Quota.PerDestination {
of += " each" of += " each"
} }
fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\n", q.Rule, humanBytes(q.Used), of, resets, next) used := humanBytes(q.Used)
if !q.Quota.PerSource && !q.Quota.PerDestination {
used += " (" + percent(q.Used, q.Quota.Bytes) + ")"
}
fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\n", q.Rule, used, of, resets, next)
for _, a := range q.Addresses { for _, a := range q.Addresses {
fmt.Fprintf(w, " %s\t%s\t\t\t\n", a.Address, humanBytes(a.Used)) fmt.Fprintf(w, " %s\t%s (%s)\t\t\t\n", named(names, a.Address, a.Address), humanBytes(a.Used), percent(a.Used, q.Quota.Bytes))
} }
} }
}) })
@@ -325,6 +336,17 @@ func quotasCmd() *cobra.Command {
} }
} }
// percent writes used as a share of total: "25%", "<1%" for a little.
func percent(used, total int64) string {
if total <= 0 || used <= 0 {
return "0%"
}
if used*100 < total {
return "<1%"
}
return fmt.Sprintf("%d%%", used*100/total)
}
func logCmd() *cobra.Command { func logCmd() *cobra.Command {
var follow bool var follow bool
var limit int var limit int
@@ -341,6 +363,11 @@ func logCmd() *cobra.Command {
var last time.Time var last time.Time
print := func(es []api.LogEntry) { print := func(es []api.LogEntry) {
sort.Slice(es, func(i, j int) bool { return es[i].Time.Before(es[j].Time) }) sort.Slice(es, func(i, j int) bool { return es[i].Time.Before(es[j].Time) })
var addrs []string
for _, e := range es {
addrs = append(addrs, e.Source, e.Dest)
}
names := lookupNames(addrs...)
for _, e := range es { for _, e := range es {
last = e.Time last = e.Time
src, dst := e.Source, e.Dest src, dst := e.Source, e.Dest
@@ -350,6 +377,7 @@ func logCmd() *cobra.Command {
if e.DestPort != 0 { if e.DestPort != 0 {
dst = joinPort(dst, e.DestPort) dst = joinPort(dst, e.DestPort)
} }
src, dst = named(names, e.Source, src), named(names, e.Dest, dst)
what := e.Protocol what := e.Protocol
if e.ICMPType != "" { if e.ICMPType != "" {
what += " " + e.ICMPType what += " " + e.ICMPType
+2
View File
@@ -46,6 +46,7 @@ var (
flagCACert string flagCACert string
flagQuiet bool flagQuiet bool
flagNoConfirm bool flagNoConfirm bool
flagNoNames bool
// lastClient is the client the command made (to confirm a change by // lastClient is the client the command made (to confirm a change by
// itself after it). // itself after it).
lastClient *client.Client lastClient *client.Client
@@ -182,6 +183,7 @@ Exit codes: 0 ok, 1 error, 3 not authenticated or not allowed, 4 not found,
pf.StringVar(&flagCACert, "ca-cert", "", "PEM file with a CA to trust for the server's certificate") pf.StringVar(&flagCACert, "ca-cert", "", "PEM file with a CA to trust for the server's certificate")
pf.BoolVarP(&flagQuiet, "quiet", "q", false, "print only errors (and IDs of new objects)") pf.BoolVarP(&flagQuiet, "quiet", "q", false, "print only errors (and IDs of new objects)")
pf.BoolVar(&flagNoConfirm, "no-confirm", false, "leave a change on probation instead of confirming it once this client reaches Housecarl again") pf.BoolVar(&flagNoConfirm, "no-confirm", false, "leave a change on probation instead of confirming it once this client reaches Housecarl again")
pf.BoolVar(&flagNoNames, "no-names", false, "show inside addresses without their names (reverse lookups)")
pf.Int64VarP(&flagChangeset, "changeset", "C", 0, "stage configuration changes in this changeset instead of applying them (env HOUSECARL_CHANGESET)") pf.Int64VarP(&flagChangeset, "changeset", "C", 0, "stage configuration changes in this changeset instead of applying them (env HOUSECARL_CHANGESET)")
root.AddGroup(&cobra.Group{ID: "fw", Title: "Firewall:"}, &cobra.Group{ID: "changes", Title: "Changes:"}, &cobra.Group{ID: "admin", Title: "Administration:"}) root.AddGroup(&cobra.Group{ID: "fw", Title: "Firewall:"}, &cobra.Group{ID: "changes", Title: "Changes:"}, &cobra.Group{ID: "admin", Title: "Administration:"})
add := func(group string, cmds ...*cobra.Command) { add := func(group string, cmds ...*cobra.Command) {
+22 -3
View File
@@ -35,9 +35,28 @@ func TestLimitAndDescribe(t *testing.T) {
if _, err := parseLimit("fast"); err == nil { if _, err := parseLimit("fast"); err == nil {
t.Error("a bad limit accepted") t.Error("a bad limit accepted")
} }
m := model.Match{InZones: []string{"wan"}, NotSources: true, Sources: []string{"trusted"}, Services: []string{"ssh", "tcp/8080"}} for _, c := range []struct {
if got := describe(m); got != "in wan from not trusted ssh,tcp/8080" { m model.Match
t.Errorf("describe: %q", got) chain string
want string
}{
{model.Match{InZones: []string{"wan"}, NotSources: true, Sources: []string{"trusted"}, Services: []string{"ssh", "tcp/8080"}}, "input",
"wan → firewall from any but trusted ssh,tcp/8080"},
{model.Match{InZones: []string{"wan"}, NotIn: true, Destinations: []string{"trusted_ips"}}, "forward", "any but wan → any to trusted_ips"},
{model.Match{InZones: []string{"lan"}, OutZones: []string{"wan"}}, "forward", "lan → wan"},
{model.Match{OutZones: []string{"wan"}}, "output", "firewall → wan"},
{model.Match{OutZones: []string{"wan"}, Sources: []string{"10.0.0.0/8"}}, "", "any → wan from 10.0.0.0/8"},
{model.Match{Sources: []string{"trusted"}}, "input", "from trusted"},
{model.Match{}, "forward", "everything"},
} {
if got := describe(c.m, c.chain); got != c.want {
t.Errorf("describe(%+v, %s) = %q, want %q", c.m, c.chain, got, c.want)
}
}
for used, want := range map[int64]string{0: "0%", 5: "<1%", 250: "25%", 1000: "100%", 1300: "130%"} {
if got := percent(used, 1000); got != want {
t.Errorf("percent(%d, 1000) = %q, want %q", used, got, want)
}
} }
if got := editList([]string{"a", "b", "c"}, []string{"d"}, []string{"b"}); len(got) != 3 || got[1] != "c" || got[2] != "d" { if got := editList([]string{"a", "b", "c"}, []string{"d"}, []string{"b"}); len(got) != 3 || got[1] != "c" || got[2] != "d" {
t.Errorf("editList: %v", got) t.Errorf("editList: %v", got)
+40
View File
@@ -49,6 +49,46 @@ func counters() map[string]api.Counter {
return cs.Counters return cs.Counters
} }
// lookupNames asks for inside addresses' names, for tables: none with
// --no-names, another output format, or when they can't be had.
func lookupNames(addrs ...string) map[string]string {
out := map[string]string{}
if flagNoNames || flagOutput != "table" {
return out
}
seen := map[string]bool{}
var list []string
for _, a := range addrs {
if a != "" && !seen[a] {
seen[a] = true
list = append(list, a)
}
}
if len(list) == 0 {
return out
}
c, err := newClient()
if err != nil {
return out
}
var ns api.Names
if c.Do(ctx(), "POST", "/names", nil, api.NamesInput{Addresses: list}, &ns) != nil {
return out
}
for a, n := range ns.Names {
out[a] = strings.SplitN(n, ".", 2)[0]
}
return out
}
// named adds an address's name to text (the address, or address:port).
func named(names map[string]string, addr, text string) string {
if n := names[addr]; n != "" {
return text + " (" + n + ")"
}
return text
}
func hits(c map[string]api.Counter, marks ...string) string { func hits(c map[string]api.Counter, marks ...string) string {
var p uint64 var p uint64
found := false found := false
+87 -38
View File
@@ -3,6 +3,7 @@ package main
import ( import (
"fmt" "fmt"
"io" "io"
"net/url"
"regexp" "regexp"
"strconv" "strconv"
"strings" "strings"
@@ -63,22 +64,40 @@ func (m *matchFlags) patch(fl *pflag.FlagSet, p patch) {
set("family", "family", m.family) set("family", "family", m.family)
} }
// describe sums up a match: "in lan from trusted ssh, dns". // describe sums up a match in a chain ("" for NAT): "lan → firewall
func describe(m model.Match) string { // from trusted ssh,dns". Zones are a direction; the firewall itself is
// the other end of the input and output chains.
func describe(m model.Match, chain string) string {
var parts []string var parts []string
side := func(word string, not bool, list []string) { list := func(not bool, xs []string) string {
if len(list) == 0 { s := strings.Join(xs, ",")
return
}
if not { if not {
word += " not" return "any but " + s
} }
parts = append(parts, word+" "+strings.Join(list, ",")) return s
}
if len(m.InZones) > 0 || len(m.OutZones) > 0 {
in, out := "any", "any"
if len(m.InZones) > 0 {
in = list(m.NotIn, m.InZones)
}
if len(m.OutZones) > 0 {
out = list(m.NotOut, m.OutZones)
}
switch chain {
case model.ChainInput:
out = "firewall"
case model.ChainOutput:
in = "firewall"
}
parts = append(parts, in+" → "+out)
}
if len(m.Sources) > 0 {
parts = append(parts, "from "+list(m.NotSources, m.Sources))
}
if len(m.Destinations) > 0 {
parts = append(parts, "to "+list(m.NotDests, m.Destinations))
} }
side("in", m.NotIn, m.InZones)
side("out", m.NotOut, m.OutZones)
side("from", m.NotSources, m.Sources)
side("to", m.NotDests, m.Destinations)
if len(m.Services) > 0 { if len(m.Services) > 0 {
parts = append(parts, strings.Join(m.Services, ",")) parts = append(parts, strings.Join(m.Services, ","))
} }
@@ -280,7 +299,7 @@ func ruleCmd() *cobra.Command {
return output(rs, func(w io.Writer) { return output(rs, func(w io.Writer) {
fmt.Fprintln(w, "CHAIN\tID\tMATCH\tACTION\tHITS\tCOMMENT") fmt.Fprintln(w, "CHAIN\tID\tMATCH\tACTION\tHITS\tCOMMENT")
for _, r := range rs { for _, r := range rs {
what := describe(r.Match) what := describe(r.Match, r.Chain)
if r.Raw != "" { if r.Raw != "" {
what = "raw: " + r.Raw what = "raw: " + r.Raw
} }
@@ -303,7 +322,7 @@ func ruleCmd() *cobra.Command {
cmd.AddCommand(ls) cmd.AddCommand(ls)
cmd.AddCommand(showCmd("rules", "rule", func(w io.Writer, r model.Rule) { cmd.AddCommand(showCmd("rules", "rule", func(w io.Writer, r model.Rule) {
if r.Raw == "" { if r.Raw == "" {
fmt.Fprintf(w, "ID:\t%d\nChain:\t%s\nMatch:\t%s%s\nAction:\t%s\n", r.ID, r.Chain, describe(r.Match), ruleExtras(r), r.Action) fmt.Fprintf(w, "ID:\t%d\nChain:\t%s\nMatch:\t%s%s\nAction:\t%s\n", r.ID, r.Chain, describe(r.Match, r.Chain), ruleExtras(r), r.Action)
} else { } else {
fmt.Fprintf(w, "ID:\t%d\nChain:\t%s\n", r.ID, r.Chain) fmt.Fprintf(w, "ID:\t%d\nChain:\t%s\n", r.ID, r.Chain)
} }
@@ -340,26 +359,21 @@ output (from it) or early (before connection tracking: cheap drops).
if err != nil { if err != nil {
return err return err
} }
// Placed in the same change: one history entry, and never in
// force at the end of the chain.
q := url.Values{}
switch {
case rf.first:
q.Set("first", "1")
case rf.before != 0:
q.Set("before", fmt.Sprint(rf.before))
case rf.after != 0:
q.Set("after", fmt.Sprint(rf.after))
}
var resp api.MutationResponse var resp api.MutationResponse
if err := c.Do(ctx(), "POST", "/rules", nil, r, &resp); err != nil { if err := c.Do(ctx(), "POST", "/rules", q, r, &resp); err != nil {
return err return err
} }
// Then place it.
if resp.Applied && (rf.first || rf.before != 0 || rf.after != 0) {
var created model.Rule
if err := jsonUnmarshal(resp.Object, &created); err == nil {
mv := api.MoveInput{Before: rf.before, After: rf.after}
if rf.first {
mv.To = "start"
}
var moved api.MutationResponse
if err := c.Do(ctx(), "POST", objPath("rules", fmt.Sprint(created.ID))+"/move", nil, mv, &moved); err != nil {
return err
}
resp.Changes = append(resp.Changes, moved.Changes...)
resp.ConfirmBy = moved.ConfirmBy
}
}
return report(&resp) return report(&resp)
}, },
} }
@@ -426,22 +440,57 @@ output (from it) or early (before connection tracking: cheap drops).
move.Flags().BoolVar(&last, "last", false, "last in its chain") move.Flags().BoolVar(&last, "last", false, "last in its chain")
move.Flags().StringVar(&chain, "chain", "", "to the end of another chain") move.Flags().StringVar(&chain, "chain", "", "to the end of another chain")
cmd.AddCommand(move) cmd.AddCommand(move)
cmd.AddCommand(&cobra.Command{ var resetAddr, give string
Use: "reset-quota ID", Short: "Start a rule's quota again now (every address's, for a per-address quota)", Args: cobra.ExactArgs(1), resetCmd := &cobra.Command{
Use: "reset-quota ID [--address ADDR] [--give AMOUNT]", Short: "Start a rule's quota again now, or give some back until its next reset", Args: cobra.ExactArgs(1),
Long: `Starts a rule's quota again now: all of it (every address's, for a
per-address quota), or with --address one address's. --give takes an
amount off what was used instead: extra allowance until the quota's next
reset, without changing the rule. Done at once, not staged or on
probation.
hc rule reset-quota 31
hc rule reset-quota 31 --address 192.168.1.23
hc rule reset-quota 31 --give 5gbytes --address 192.168.1.23`,
RunE: func(cmd *cobra.Command, args []string) error { RunE: func(cmd *cobra.Command, args []string) error {
in := api.QuotaReset{Address: resetAddr}
if give != "" {
bad := fmt.Errorf("--give %q: write it as 500mbytes or 5gbytes", give)
m := amountRe.FindStringSubmatch(strings.TrimSpace(give))
if m == nil {
return bad
}
unit, ok := byteUnit(m[2])
n, _ := strconv.ParseInt(m[1], 10, 64)
if !ok || unit == "" || n < 1 {
return bad
}
in.Give = n * model.ByteUnits[unit]
}
c, err := newClient() c, err := newClient()
if err != nil { if err != nil {
return err return err
} }
if err := c.Do(ctx(), "POST", objPath("rules", args[0])+"/reset-quota", nil, nil, nil); err != nil { if err := c.Do(ctx(), "POST", objPath("rules", args[0])+"/reset-quota", nil, in, nil); err != nil {
return err return err
} }
if !flagQuiet { if !flagQuiet {
fmt.Printf("reset the quota of rule %s\n", args[0]) who := ""
if resetAddr != "" {
who = " for " + resetAddr
}
if in.Give > 0 {
fmt.Printf("gave back %s of the quota of rule %s%s, until its next reset\n", humanBytes(in.Give), args[0], who)
} else {
fmt.Printf("reset the quota of rule %s%s\n", args[0], who)
}
} }
return nil return nil
}, },
}) }
resetCmd.Flags().StringVar(&resetAddr, "address", "", "only this address's (a per-address quota; an IPv6 address counts for its /64)")
resetCmd.Flags().StringVar(&give, "give", "", "take this much off what was used instead (5gbytes, 500mbytes)")
cmd.AddCommand(resetCmd)
for _, on := range []bool{true, false} { for _, on := range []bool{true, false} {
verb, short := "disable", "Keep a rule but don't use it" verb, short := "disable", "Keep a rule but don't use it"
if on { if on {
@@ -669,7 +718,7 @@ func snatCmd() *cobra.Command {
if to == "" { if to == "" {
to = "masquerade" to = "masquerade"
} }
fmt.Fprintf(w, "%d\t%s\t%s\t%s\t%s\n", x.ID, describe(x.Match), to, hits(cs, fmt.Sprintf("s%d", x.ID)), x.Comment) fmt.Fprintf(w, "%d\t%s\t%s\t%s\t%s\n", x.ID, describe(x.Match, ""), to, hits(cs, fmt.Sprintf("s%d", x.ID)), x.Comment)
} }
}) })
}, },
@@ -749,7 +798,7 @@ func redirectCmd() *cobra.Command {
return output(xs, func(w io.Writer) { return output(xs, func(w io.Writer) {
fmt.Fprintln(w, "ID\tMATCH\tTO PORT\tHITS\tCOMMENT") fmt.Fprintln(w, "ID\tMATCH\tTO PORT\tHITS\tCOMMENT")
for _, x := range xs { for _, x := range xs {
fmt.Fprintf(w, "%d\t%s\t%s\t%s\t%s\n", x.ID, describe(x.Match), orDash(x.ToPort), hits(cs, fmt.Sprintf("d%d", x.ID)), x.Comment) fmt.Fprintf(w, "%d\t%s\t%s\t%s\t%s\n", x.ID, describe(x.Match, ""), orDash(x.ToPort), hits(cs, fmt.Sprintf("d%d", x.ID)), x.Comment)
} }
}) })
}, },
+19 -3
View File
@@ -283,6 +283,7 @@ func statsTopCmd() *cobra.Command {
for _, l := range [][]statEntry{down, up} { for _, l := range [][]statEntry{down, up} {
for _, e := range l { for _, e := range l {
if _, ok := out[e.Key]; !ok { if _, ok := out[e.Key]; !ok {
out[e.Key] = [2]int64{}
order = append(order, e.Key) order = append(order, e.Key)
} }
} }
@@ -297,10 +298,11 @@ func statsTopCmd() *cobra.Command {
v[1] = e.Count v[1] = e.Count
out[e.Key] = v out[e.Key] = v
} }
names := lookupNames(order...)
return output(map[string]any{"up": up, "down": down}, func(w io.Writer) { return output(map[string]any{"up": up, "down": down}, func(w io.Writer) {
fmt.Fprintln(w, "DEVICE\tRECEIVED\tSENT") fmt.Fprintln(w, "DEVICE\tRECEIVED\tSENT")
for _, k := range order { for _, k := range order {
fmt.Fprintf(w, "%s\t%s\t%s\n", k, humanBytes(out[k][0]), humanBytes(out[k][1])) fmt.Fprintf(w, "%s\t%s\t%s\n", named(names, k, k), humanBytes(out[k][0]), humanBytes(out[k][1]))
} }
}) })
case "blocked", "tried": case "blocked", "tried":
@@ -309,10 +311,14 @@ func statsTopCmd() *cobra.Command {
if err != nil { if err != nil {
return err return err
} }
var names map[string]string
if args[0] == "blocked" {
names = lookupNames(keys(es)...)
}
return output(es, func(w io.Writer) { return output(es, func(w io.Writer) {
fmt.Fprintln(w, map[string]string{"blocked": "SOURCE", "tried": "WHAT"}[args[0]]+"\tPACKETS") fmt.Fprintln(w, map[string]string{"blocked": "SOURCE", "tried": "WHAT"}[args[0]]+"\tPACKETS")
for _, e := range es { for _, e := range es {
fmt.Fprintf(w, "%s\t%d\n", e.Key, e.Count) fmt.Fprintf(w, "%s\t%d\n", named(names, e.Key, e.Key), e.Count)
} }
}) })
case "quota": case "quota":
@@ -323,10 +329,11 @@ func statsTopCmd() *cobra.Command {
if err != nil { if err != nil {
return err return err
} }
names := lookupNames(keys(es)...)
return output(es, func(w io.Writer) { return output(es, func(w io.Writer) {
fmt.Fprintln(w, "ADDRESS\tUSED") fmt.Fprintln(w, "ADDRESS\tUSED")
for _, e := range es { for _, e := range es {
fmt.Fprintf(w, "%s\t%s\n", e.Key, humanBytes(e.Count)) fmt.Fprintf(w, "%s\t%s\n", named(names, e.Key, e.Key), humanBytes(e.Count))
} }
}) })
} }
@@ -338,6 +345,15 @@ func statsTopCmd() *cobra.Command {
return cmd return cmd
} }
// keys are a top list's keys.
func keys(es []statEntry) []string {
var out []string
for _, e := range es {
out = append(out, e.Key)
}
return out
}
func statsRulesCmd() *cobra.Command { func statsRulesCmd() *cobra.Command {
var since string var since string
var unused bool var unused bool
+7 -1
View File
@@ -38,10 +38,16 @@ rules as it stands.
if err != nil { if err != nil {
return err return err
} }
var addrs []string
for _, c := range cs {
addrs = append(addrs, c.Source, c.Destination)
}
names := lookupNames(addrs...)
return output(cs, func(w io.Writer) { return output(cs, func(w io.Writer) {
fmt.Fprintln(w, "PROTO\tFROM\tTO\tNAT\tSTATE\tEXPIRES") fmt.Fprintln(w, "PROTO\tFROM\tTO\tNAT\tSTATE\tEXPIRES")
for _, c := range cs { for _, c := range cs {
fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\t%s\n", c.Protocol, hostPort(c.Source, c.SourcePort), hostPort(c.Destination, c.DestinationPort), natText(c), c.State, fmtSeconds(c.Timeout)) fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\t%s\n", c.Protocol, named(names, c.Source, hostPort(c.Source, c.SourcePort)),
named(names, c.Destination, hostPort(c.Destination, c.DestinationPort)), natText(c), c.State, fmtSeconds(c.Timeout))
} }
}) })
}, },
+6 -1
View File
@@ -32,6 +32,7 @@ import (
"housecarl/internal/jarl" "housecarl/internal/jarl"
"housecarl/internal/linkwatch" "housecarl/internal/linkwatch"
"housecarl/internal/model" "housecarl/internal/model"
"housecarl/internal/names"
"housecarl/internal/nflog" "housecarl/internal/nflog"
"housecarl/internal/nft" "housecarl/internal/nft"
"housecarl/internal/render" "housecarl/internal/render"
@@ -176,11 +177,15 @@ func serveCmd() *cobra.Command {
acme = certs.New(ctx, certs.Config{ACME: a.cfg.TLS.ACME, Version: version.Short()}, jc, a.store, a.log) acme = certs.New(ctx, certs.Config{ACME: a.cfg.TLS.ACME, Version: version.Short()}, jc, a.store, a.log)
go acme.Run(ctx) go acme.Run(ctx)
} }
var resolver *names.Resolver
if a.cfg.Names.Enabled {
resolver = names.New(a.cfg.Names.Resolver, time.Duration(a.cfg.Names.Cache))
}
srv := server.New(server.Config{ srv := server.New(server.Config{
Listen: a.cfg.Listen, TLSCertFile: a.cfg.TLS.CertFile, TLSKeyFile: a.cfg.TLS.KeyFile, TrustedProxies: a.cfg.Proxies(), Listen: a.cfg.Listen, TLSCertFile: a.cfg.TLS.CertFile, TLSKeyFile: a.cfg.TLS.KeyFile, TrustedProxies: a.cfg.Proxies(),
SessionLifetime: time.Duration(a.cfg.Session.Lifetime), SessionIdle: time.Duration(a.cfg.Session.IdleTimeout), SessionLifetime: time.Duration(a.cfg.Session.Lifetime), SessionIdle: time.Duration(a.cfg.Session.IdleTimeout),
CookieSecure: a.cfg.Session.CookieSecure, Socket: a.cfg.Socket, ACME: acme, CookieSecure: a.cfg.Session.CookieSecure, Socket: a.cfg.Socket, ACME: acme,
Shadow: a.cfg.Shadow.Enabled, ShadowPosition: a.cfg.Shadow.Position, Shadow: a.cfg.Shadow.Enabled, ShadowPosition: a.cfg.Shadow.Position, Names: resolver,
}, eng, fw, host, a.log, web.FS()) }, eng, fw, host, a.log, web.FS())
// Put the ruleset in force: what was on probation when // Put the ruleset in force: what was on probation when
// housecarld stopped is rolled back first. An empty // housecarld stopped is rolled back first. An empty
+3 -2
View File
@@ -82,6 +82,7 @@ Creating answers 201, other changes 200.
| `GET /counters` | `{"since", "counters": {"r12": {"packets", "bytes"}, ...}}`, by mark ([design.md](design.md#its-own-table-only)); totals since housecarld started | | `GET /counters` | `{"since", "counters": {"r12": {"packets", "bytes"}, ...}}`, by mark ([design.md](design.md#its-own-table-only)); totals since housecarld started |
| `GET /quotas` | each rule's quota: `rule_id`, `rule`, `quota`, `used` (bytes), `addresses` (`[{"address", "used"}]` for a per-address quota), `last_reset`, `next_reset` | | `GET /quotas` | each rule's quota: `rule_id`, `rule`, `quota`, `used` (bytes), `addresses` (`[{"address", "used"}]` for a per-address quota), `last_reset`, `next_reset` |
| `GET /connections[?q=&limit=]` | tracked connections: `family`, `protocol`, `source`, `source_port`, `destination`, `destination_port` (or `icmp_id`, `icmp_type`, `icmp_code`), `dnat_to`, `dnat_port`, `snat_to`, `state`, `assured`, `timeout` (seconds left), `packets` and `bytes` when the kernel counts them. `q`: all of its words in the addresses, `:port`, protocol, state | | `GET /connections[?q=&limit=]` | tracked connections: `family`, `protocol`, `source`, `source_port`, `destination`, `destination_port` (or `icmp_id`, `icmp_type`, `icmp_code`), `dnat_to`, `dnat_port`, `snat_to`, `state`, `assured`, `timeout` (seconds left), `packets` and `bytes` when the kernel counts them. `q`: all of its words in the addresses, `:port`, protocol, state |
| `POST /names` | `{"addresses": [...]}` (at most 2000): `{"enabled", "names": {"192.168.1.20": "laptop.example.net"}}`, the names of the inside addresses among them, by reverse lookup ([configuration.md](configuration.md#names)). Addresses without a name, outside ones, and those still being looked up after a second are left out; ask again for those. `log:read` |
| `POST /connections/close` | `{"connections": [...]}` (as listed) or `{"search": "192.168.1.50"}`: delete their tracking entries (`config:write`, audited); answers `{"closed": N}`. 409 in shadow mode | | `POST /connections/close` | `{"connections": [...]}` (as listed) or `{"search": "192.168.1.50"}`: delete their tracking entries (`config:write`, audited); answers `{"closed": N}`. 409 in shadow mode |
| `POST /trace` | `{"address", "port", "protocol", "seconds"}` (5-600, default 60): start tracing (`config:write`, audited); 409 while one runs | | `POST /trace` | `{"address", "port", "protocol", "seconds"}` (5-600, default 60): start tracing (`config:write`, audited); 409 while one runs |
| `GET /trace` | the trace running, or the last: `running`, `input`, `started`, `ends`, `error`, `packets`: each with `time`, `in`, `out`, `protocol`, addresses and ports, `tcp_flags`, `icmp_type`, `steps` (`table`, `chain`, `kind` rule or policy, `rule`, `mark`, `label`, `verdict`), `verdict` (accepted, dropped) and `decided_by`; in shadow mode `would_drop`, the step where Housecarl would have dropped it | | `GET /trace` | the trace running, or the last: `running`, `input`, `started`, `ends`, `error`, `packets`: each with `time`, `in`, `out`, `protocol`, addresses and ports, `tcp_flags`, `icmp_type`, `steps` (`table`, `chain`, `kind` rule or policy, `rule`, `mark`, `label`, `verdict`), `verdict` (accepted, dropped) and `decided_by`; in shadow mode `would_drop`, the step where Housecarl would have dropped it |
@@ -114,11 +115,11 @@ share one pattern, `/{kind}` with kind `zones`, `sets`, `services`,
| | | | | |
|---|---| |---|---|
| `GET /{kind}` | all of them, in order; `GET /rules?chain=input` for one chain; `GET /services` includes the built-in ones | | `GET /{kind}` | all of them, in order; `GET /rules?chain=input` for one chain; `GET /services` includes the built-in ones |
| `POST /{kind}` | create; rules, forwards and NAT rules go last unless given a `position` | | `POST /{kind}[?before=ID\|after=ID\|first=1]` | create; rules, forwards and NAT rules go last unless given a `position`, or placed before or after another of their chain, or first, in the same change (one history entry; staged in a changeset, the place goes with it) |
| `GET /{kind}/{ref}` | by ID, or name for zones, sets, services and forwards | | `GET /{kind}/{ref}` | by ID, or name for zones, sets, services and forwards |
| `PATCH /{kind}/{ref}` | change the fields given (a JSON merge: `null` clears) | | `PATCH /{kind}/{ref}` | change the fields given (a JSON merge: `null` clears) |
| `DELETE /{kind}/{ref}` | refused while something uses it | | `DELETE /{kind}/{ref}` | refused while something uses it |
| `POST /rules/{ref}/reset-quota` | start a rule's quota again now (`config:write`; audited, not staged) | | `POST /rules/{ref}/reset-quota` | start a rule's quota again now (`config:write`; audited, not staged). Optional body: `{"address": "192.168.1.23"}` starts only that address's again (a per-address quota; an IPv6 address counts for its /64; 404 if it hasn't used the quota); `{"give": 5368709120}` takes that many bytes off what was used instead (the address's, or every address's), until the next reset |
| `POST /{kind}/{ref}/move` | `{"before": ID}`, `{"after": ID}` or `{"to": "start"\|"end"}`; a rule can take `"chain"` too | | `POST /{kind}/{ref}/move` | `{"before": ID}`, `{"after": ID}` or `{"to": "start"\|"end"}`; a rule can take `"chain"` too |
The objects: The objects:
+2 -1
View File
@@ -32,6 +32,7 @@ command and its flags.
| `-C, --changeset ID` | `HOUSECARL_CHANGESET` (checked, see [Changesets](#changesets)) | stage configuration changes in this changeset instead of making them | | `-C, --changeset ID` | `HOUSECARL_CHANGESET` (checked, see [Changesets](#changesets)) | stage configuration changes in this changeset instead of making them |
| `-o, --output FORMAT` | | `table` (default), `json`, `yaml` or `id` | | `-o, --output FORMAT` | | `table` (default), `json`, `yaml` or `id` |
| `-q, --quiet` | | print only errors (and the IDs of new objects) | | `-q, --quiet` | | print only errors (and the IDs of new objects) |
| `--no-names` | | show inside addresses without their names: tables show `192.168.1.20 (laptop)` ([configuration.md](configuration.md#names)) |
Exit codes: 0 ok, 1 error, 3 not authenticated or not allowed, 4 not Exit codes: 0 ok, 1 error, 3 not authenticated or not allowed, 4 not
found, 5 conflict, 6 invalid or refused by validation or nftables, 7 found, 5 conflict, 6 invalid or refused by validation or nftables, 7
@@ -171,7 +172,7 @@ See [rules.md](rules.md).
| `rule update ID [flags]` | only the flags given | | `rule update ID [flags]` | only the flags given |
| `rule move ID --before ID\|--after ID\|--first\|--last\|--chain CHAIN` | | | `rule move ID --before ID\|--after ID\|--first\|--last\|--chain CHAIN` | |
| `rule enable\|disable ID` | | | `rule enable\|disable ID` | |
| `rule reset-quota ID` | start a rule's quota again now | | `rule reset-quota ID [--address ADDR] [--give 5gbytes]` | start a rule's quota again now, or one address's; `--give` takes an amount off what was used instead, until the next reset |
| `rule delete ID` | | | `rule delete ID` | |
The match flags (`rule add`, `snat add`, `redirect add`): The match flags (`rule add`, `snat add`, `redirect add`):
+22
View File
@@ -32,12 +32,34 @@ reads the file when it starts.
| `stats.top_n` | `100` | entries kept per top list, per hour and per day | | `stats.top_n` | `100` | entries kept per top list, per hour and per day |
| `shadow.enabled` | `false` | shadow mode: Housecarl's ruleset is loaded beside the firewall you have, but blocks, rejects and translates nothing; its counters, log and tracer show what it would do. No change is on probation, and takeovers and closing connections are refused ([shadow.md](shadow.md)) | | `shadow.enabled` | `false` | shadow mode: Housecarl's ruleset is loaded beside the firewall you have, but blocks, rejects and translates nothing; its counters, log and tracer show what it would do. No change is on probation, and takeovers and closing connections are refused ([shadow.md](shadow.md)) |
| `shadow.position` | `after` | where Housecarl's filter chains run in shadow mode: `after` the other firewall's (`filter + 10`: it sees only what that one lets through, so what it would drop is what switching would break) or `before` them (`filter - 10`: it sees every packet) | | `shadow.position` | `after` | where Housecarl's filter chains run in shadow mode: `after` the other firewall's (`filter + 10`: it sees only what that one lets through, so what it would drop is what switching would break) or `before` them (`filter - 10`: it sees every packet) |
| `names.enabled` | `true` | show inside addresses' names beside them, found by reverse lookups (below) |
| `names.resolver` | `""` | the DNS server to ask, such as `10.10.0.1:53`; empty: the system's (`/etc/hosts`, then `/etc/resolv.conf`) |
| `names.cache` | `10m` | how long a name, or the lack of one, is kept |
Durations are written `15s`, `12h` or `30d`. Durations are written `15s`, `12h` or `30d`.
Shadow mode is here, and not a firewall setting, so that no user or API Shadow mode is here, and not a firewall setting, so that no user or API
token can switch enforcement off; restart housecarld to switch. token can switch enforcement off; restart housecarld to switch.
### Names
The web UI and `hc` show a name beside each **inside** address in
connections, the log, quotas and traffic per device: `laptop
192.168.1.20`. Housecarl finds it with a reverse (PTR) lookup, so the
names are your reverse zones' (and `/etc/hosts`'). An address counts as
inside when it is in a network connected to an inside zone's interface,
or private (`10.0.0.0/8`, `172.16.0.0/12`, `192.168.0.0/16`, `fc00::/7`)
and not in an external zone's network. Outside addresses get no name:
their reverse zones belong to whoever holds the addresses, who would
choose the names, and looking up every scanner is slow.
Names are looked up when a page or `hc` asks, cached (a failed lookup is
tried again after a minute), and never stored: statistics stay by
address. With a split DNS (views), the server asked must answer the
firewall as an inside client; set `names.resolver` if the system's
resolver doesn't. IPv6 traffic per device is counted per /64, which has
no name. `hc --no-names` leaves them out.
The probation's length is a firewall setting, not here: `hc settings set The probation's length is a firewall setting, not here: `hc settings set
--probation 5m`. So is counting traffic per device for the statistics: --probation 5m`. So is counting traffic per device for the statistics:
`hc settings set --stats-per-device on`. `hc settings set --stats-per-device on`.
+5 -4
View File
@@ -161,11 +161,12 @@ start again. A disabled rule keeps its count for when it is enabled.
## Quotas ## Quotas
`hc quotas` (and the rules page) show what each quota used and when it `hc quotas` (and the rules page) show what each quota used and when it
resets; `hc rule reset-quota ID` starts one again. housecarld resets resets; `hc rule reset-quota ID` starts one again (`--address` one device's,
`--give 5gbytes` a top-up until the next reset). housecarld resets
quotas at midnight when their period ends (late, if it was stopped then), quotas at midnight when their period ends (late, if it was stopped then),
and saves their usage every five minutes and when it stops; after a and saves their usage every five minutes, after every change and when
reboot it puts it back. A reboot without a clean stop loses at most the it stops; after a reboot it puts it back. A reboot without a clean stop
last five minutes' count. loses at most the last five minutes' count.
## The log ## The log
+30 -2
View File
@@ -186,8 +186,22 @@ hc rule add forward --in lan --out wan --quota 50gbytes,over,per-source,monthly=
hc rule add forward --in guest --out wan --quota 10gbytes,over,daily -a drop hc rule add forward --in guest --out wan --quota 10gbytes,over,daily -a drop
hc quotas # what each used, and the next reset hc quotas # what each used, and the next reset
hc rule reset-quota 31 # start again now hc rule reset-quota 31 # start again now
hc rule reset-quota 31 --address 192.168.1.23 # one device only
hc rule reset-quota 31 --give 5gbytes --address 192.168.1.23
``` ```
`--give` is a top-up: it takes that much off what was used (never below
zero), so the device gets 5 GB more until the quota's next reset, and the
rule stays as it is. In the web UI, the quota's line under the rule
opens its dialog: usage (each address's), resets, and **Reset** and
**Give back…** for everyone or one address. Like a reset, these are
done at once (audited, not staged or on probation); Housecarl puts the
ruleset in force again with the new figure.
In the web UI each quota's use is on its rule (with **Reset**), all of
them together on the dashboard, and their history under **Traffic →
Statistics**.
| | | | | |
|---|---| |---|---|
| `per-source`, `per-destination` | each address its own allowance; without, one allowance for everything the rule matches | | `per-source`, `per-destination` | each address its own allowance; without, one allowance for everything the rule matches |
@@ -195,12 +209,26 @@ hc rule reset-quota 31 # start again now
nftables counts a quota but never resets it, and forgets it when its nftables counts a quota but never resets it, and forgets it when its
table is replaced: Housecarl carries what each quota used across every table is replaced: Housecarl carries what each quota used across every
change and restart, saves it every five minutes (and when housecarld change and restart, saves it every five minutes (and after each change,
stops) for after a reboot, and resets quotas when their period ends. and when housecarld stops) for after a reboot, and resets quotas when their period ends.
A per-address quota counts an address: a device that gets a new one A per-address quota counts an address: a device that gets a new one
starts again. With Steward's reservations IPv4 addresses stay; IPv6 is starts again. With Steward's reservations IPv4 addresses stay; IPv6 is
grouped by /64 (below). grouped by /64 (below).
Changing the amount, `over` or the resets keeps what was used. Changing
how the quota counts carries what can be carried:
| Change | What was used |
|---|---|
| per address to one total | the addresses' sum |
| to wider groups (IPv6 /64 to /56) | added up per new group |
| a total to per address | starts again: a total can't be split |
| per source to per destination (or back) | starts again |
| to finer groups (/64 to /128) | starts again: a group can't be split |
A change that starts a quota again says so (warning `quota_restarts`),
when it is staged for review and in `hc` (also with `-n`).
**Rules with a quota or a byte rate count every packet.** The built-in **Rules with a quota or a byte rate count every packet.** The built-in
that accepts established connections comes before your rules, so a rule that accepts established connections comes before your rules, so a rule
there only sees each connection's first packet. Rules that count traffic there only sees each connection's first packet. Rules that count traffic
+1
View File
@@ -97,6 +97,7 @@ These apply to rules, source NAT and redirects.
| `rule_group` | error | a grouping prefix outside 1-32 (IPv4) or 1-128 (IPv6) | | `rule_group` | error | a grouping prefix outside 1-32 (IPv4) or 1-128 (IPv6) |
| `rule_traffic_accept` | error | a rule with a quota or a byte rate that accepts: it is checked before established connections are accepted, so it can only drop, reject or log | | `rule_traffic_accept` | error | a rule with a quota or a byte rate that accepts: it is checked before established connections are accepted, so it can only drop, reject or log |
| `rule_every_packet_accept` | error | a rule checked against every packet (`every_packet`) that accepts: it comes before established connections are accepted, so it can only drop, reject or log | | `rule_every_packet_accept` | error | a rule checked against every packet (`every_packet`) that accepts: it comes before established connections are accepted, so it can only drop, reject or log |
| `quota_restarts` | warning | a change makes a rule count its quota so that what it used can't carry over (a total to per address, per source to per destination, finer groups): it starts again. Only on the change that does it |
| `flowtable_traffic` | warning | a forward rule checked against every packet (a quota, a byte rate, `every_packet`) while the flowtable is on: it only sees each connection's first packets | | `flowtable_traffic` | warning | a forward rule checked against every packet (a quota, a byte rate, `every_packet`) while the flowtable is on: it only sees each connection's first packets |
| `reject_with` | error | `tcp-reset` on a rule that matches more than TCP, or with an action other than reject | | `reject_with` | error | `tcp-reset` on a rule that matches more than TCP, or with an action other than reject |
| `log_prefix` | error | a log prefix over 100 characters | | `log_prefix` | error | a log prefix over 100 characters |
+23
View File
@@ -326,6 +326,16 @@ type CertificateResponse struct {
Status *CertStatus `json:"status,omitempty"` Status *CertStatus `json:"status,omitempty"`
} }
// QuotaReset is POST /rules/{ref}/reset-quota's optional body. Without
// one the whole quota starts again. Address starts one address of a
// per-address quota again (any address in its group: 10.0.10.23, or an
// IPv6 address for its /64); Give takes that many bytes off what was used
// instead (the address's, or every address's), until the next reset.
type QuotaReset struct {
Address string `json:"address,omitempty"`
Give int64 `json:"give,omitempty"`
}
// QuotaInfo is a rule's quota and what it used. // QuotaInfo is a rule's quota and what it used.
type QuotaInfo struct { type QuotaInfo struct {
RuleID int64 `json:"rule_id"` RuleID int64 `json:"rule_id"`
@@ -384,6 +394,19 @@ type Connection struct {
Bytes uint64 `json:"bytes,omitempty"` Bytes uint64 `json:"bytes,omitempty"`
} }
// NamesInput asks for the names of addresses (POST /names).
type NamesInput struct {
Addresses []string `json:"addresses"`
}
// Names are the names found for the inside addresses asked about, by
// address; those without one (or still being looked up) are left out.
// Enabled is false when names are off (names.enabled).
type Names struct {
Enabled bool `json:"enabled"`
Names map[string]string `json:"names"`
}
// CloseConnections closes connections: these, or every one Search finds // CloseConnections closes connections: these, or every one Search finds
// (as GET /connections?q= does). // (as GET /connections?q= does).
type CloseConnections struct { type CloseConnections struct {
+22
View File
@@ -44,6 +44,19 @@ type Config struct {
// Shadow loads the ruleset without letting it block, reject or // Shadow loads the ruleset without letting it block, reject or
// translate anything, beside the firewall in force (docs/shadow.md). // translate anything, beside the firewall in force (docs/shadow.md).
Shadow Shadow `yaml:"shadow"` Shadow Shadow `yaml:"shadow"`
// Names shows inside addresses' names, by reverse lookups
// (docs/configuration.md).
Names Names `yaml:"names"`
}
// Names configures the reverse lookups that put names beside inside
// addresses in the web UI and hc.
type Names struct {
Enabled bool `yaml:"enabled"`
// Resolver is a DNS server ("10.10.0.1:53"; "": the system's).
Resolver string `yaml:"resolver"`
// Cache is how long a name is kept.
Cache Duration `yaml:"cache"`
} }
// Shadow is shadow mode: Housecarl's ruleset is loaded and counts, logs // Shadow is shadow mode: Housecarl's ruleset is loaded and counts, logs
@@ -224,6 +237,7 @@ func Default() *Config {
Stats: Stats{Enabled: true, Database: "/var/lib/housecarl/stats.db", TopN: 100, Stats: Stats{Enabled: true, Database: "/var/lib/housecarl/stats.db", TopN: 100,
Retention: StatsRetention{Minute: Duration(7 * 24 * time.Hour), Hour: Duration(90 * 24 * time.Hour), Day: Duration(730 * 24 * time.Hour)}}, Retention: StatsRetention{Minute: Duration(7 * 24 * time.Hour), Hour: Duration(90 * 24 * time.Hour), Day: Duration(730 * 24 * time.Hour)}},
Shadow: Shadow{Position: ShadowAfter}, Shadow: Shadow{Position: ShadowAfter},
Names: Names{Enabled: true, Cache: Duration(10 * time.Minute)},
TLS: TLS{ACME: ACME{Directory: "letsencrypt-staging", Resolvers: []string{"1.1.1.1:53", "9.9.9.9:53", "8.8.8.8:53"}, CheckPropagation: true}}, TLS: TLS{ACME: ACME{Directory: "letsencrypt-staging", Resolvers: []string{"1.1.1.1:53", "9.9.9.9:53", "8.8.8.8:53"}, CheckPropagation: true}},
} }
} }
@@ -298,6 +312,14 @@ func (c *Config) check() error {
if c.Shadow.Position != ShadowAfter && c.Shadow.Position != ShadowBefore { if c.Shadow.Position != ShadowAfter && c.Shadow.Position != ShadowBefore {
return fmt.Errorf("shadow.position must be after or before") return fmt.Errorf("shadow.position must be after or before")
} }
if c.Names.Resolver != "" {
if _, err := netip.ParseAddrPort(c.Names.Resolver); err != nil {
return fmt.Errorf("names.resolver must be an address and port, such as 10.10.0.1:53")
}
}
if c.Names.Cache <= 0 {
return fmt.Errorf("names.cache must be a duration, such as 10m")
}
return nil return nil
} }
+18
View File
@@ -191,6 +191,7 @@ func (e *Engine) commit(ctx context.Context, p *auth.Principal, description stri
// Only report warnings the change brings; old ones are on the status // Only report warnings the change brings; old ones are on the status
// page. // page.
res.Warnings = newProblems(validate.Check(e.state, e.env()), problems) res.Warnings = newProblems(validate.Check(e.state, e.env()), problems)
res.Warnings = append(res.Warnings, quotaRestarts(e.state, next)...)
if e.Firewall != nil { if e.Firewall != nil {
if err := e.Firewall.Check(ctx, next); err != nil { if err := e.Firewall.Check(ctx, next); err != nil {
return res, &Error{Kind: KindFirewall, Message: "nftables refused the ruleset: " + err.Error()} return res, &Error{Kind: KindFirewall, Message: "nftables refused the ruleset: " + err.Error()}
@@ -230,6 +231,23 @@ func (e *Engine) commit(ctx context.Context, p *auth.Principal, description stri
return res, nil return res, nil
} }
// quotaRestarts warns of the rules whose quota starts again because they
// now count it in a way what they used can't be carried to.
func quotaRestarts(before, after *model.State) []validate.Problem {
was := map[int64]*model.Rule{}
for _, r := range before.Rules {
was[r.ID] = r
}
var out []validate.Problem
for _, r := range after.Rules {
if carry, why := model.QuotaCarry(was[r.ID], r); carry == model.QuotaRestarts {
out = append(out, validate.Problem{Severity: validate.Warning, Code: "quota_restarts", Object: r.Label(),
Message: "its quota starts again: " + why})
}
}
return out
}
func newProblems(before, after []validate.Problem) []validate.Problem { func newProblems(before, after []validate.Problem) []validate.Problem {
seen := map[string]bool{} seen := map[string]bool{}
for _, p := range before { for _, p := range before {
+46
View File
@@ -173,3 +173,49 @@ func TestPositionsStayStableUntilCrowded(t *testing.T) {
t.Errorf("order %v", order) t.Errorf("order %v", order)
} }
} }
// TestQuotaRestartsWarns: a change that counts a quota so what it used
// can't carry over warns; one that can (to a total, wider groups) doesn't.
func TestQuotaRestartsWarns(t *testing.T) {
e := newEngine(t)
ctx := context.Background()
var id int64
if _, err := e.Apply(ctx, admin, "", Options{}, func(st *model.State, newID func() int64) error {
id = newID()
st.Rules = append(st.Rules, &model.Rule{ID: id, Chain: "forward", Action: "drop", Quota: &model.Quota{Bytes: 1 << 30, Over: true, PerSource: true}})
return nil
}); err != nil {
t.Fatal(err)
}
change := func(f func(r *model.Rule)) []string {
res, err := e.Apply(ctx, admin, "", Options{DryRun: true}, func(st *model.State, _ func() int64) error {
f(st.ByID(model.KindRule, id).(*model.Rule))
return nil
})
if err != nil {
t.Fatal(err)
}
var codes []string
for _, w := range res.Warnings {
if w.Code == "quota_restarts" {
codes = append(codes, w.Message)
}
}
return codes
}
if w := change(func(r *model.Rule) { r.Quota.PerSource, r.Quota.PerDestination = false, true }); len(w) != 1 || !strings.Contains(w[0], "per source can't be counted per destination") {
t.Errorf("per source to per destination: %v", w)
}
if w := change(func(r *model.Rule) { r.Quota.PerSource = false }); len(w) != 0 {
t.Errorf("to a total (summed): %v", w)
}
if w := change(func(r *model.Rule) { r.GroupV6 = 56 }); len(w) != 0 {
t.Errorf("wider groups: %v", w)
}
if w := change(func(r *model.Rule) { r.GroupV6 = 128 }); len(w) != 1 {
t.Errorf("finer groups: %v", w)
}
if w := change(func(r *model.Rule) { r.Quota.Bytes = 2 << 30 }); len(w) != 0 {
t.Errorf("another amount: %v", w)
}
}
+3
View File
@@ -54,6 +54,9 @@ type Op struct {
// Data is the object (create), the fields to change (update, // Data is the object (create), the fields to change (update,
// settings) or where to move it (move). // settings) or where to move it (move).
Data json.RawMessage `json:"data,omitempty"` Data json.RawMessage `json:"data,omitempty"`
// Place is where a create puts the object in its order (a move's
// data); none: at the end.
Place json.RawMessage `json:"place,omitempty"`
Summary string `json:"summary"` Summary string `json:"summary"`
AddedAt time.Time `json:"added_at"` AddedAt time.Time `json:"added_at"`
} }
+39
View File
@@ -274,6 +274,45 @@ func (r *Rule) Groups() (v4, v6 int) {
return v4, v6 return v4, v6
} }
// What becomes of what a rule's quota used when the rule changes
// (QuotaCarry).
const (
QuotaKept = "" // counted as before: carried as it is
QuotaSummed = "summed" // per address to a total: the addresses' sum
QuotaRegrouped = "regrouped" // per address in wider groups: added up per group
QuotaRestarts = "restarts" // can't be carried: starts again
)
// QuotaCarry says what becomes of what old's quota used when the rule
// becomes new (both the same rule, with quotas), and for QuotaRestarts
// why. The amount, over and the resets don't matter: only how it counts.
func QuotaCarry(old, new *Rule) (carry, why string) {
if old == nil || new == nil || old.Quota == nil || new.Quota == nil {
return QuotaKept, ""
}
was, is := old.Quota.PerAddress(), new.Quota.PerAddress()
words := map[string]string{"saddr": "per source", "daddr": "per destination", "": "in total"}
switch {
case was == "" && is == "":
return QuotaKept, ""
case is == "":
return QuotaSummed, ""
case was == "":
return QuotaRestarts, "a total can't be split by address, so each address starts at nothing"
case was != is:
return QuotaRestarts, "what was counted " + words[was] + " can't be counted " + words[is]
}
o4, o6 := old.Groups()
n4, n6 := new.Groups()
switch {
case o4 == n4 && o6 == n6:
return QuotaKept, ""
case n4 > o4 || n6 > o6:
return QuotaRestarts, "addresses are grouped more finely now, and a group's use can't be split"
}
return QuotaRegrouped, ""
}
// PortForward publishes a service on an inside host: incoming connections // PortForward publishes a service on an inside host: incoming connections
// to a public address and port are sent on (DNAT) to To[:ToPort], let // to a public address and port are sent on (DNAT) to To[:ToPort], let
// through the forward chain, and, with Hairpin, also work from inside. // through the forward chain, and, with Hairpin, also work from inside.
+26
View File
@@ -0,0 +1,26 @@
package model
import "testing"
func TestQuotaCarry(t *testing.T) {
rule := func(per string, v4, v6 int) *Rule {
return &Rule{GroupV4: v4, GroupV6: v6, Quota: &Quota{Bytes: 1, PerSource: per == "src", PerDestination: per == "dst"}}
}
for _, c := range []struct {
old, new *Rule
want string
}{
{rule("", 0, 0), rule("", 0, 0), QuotaKept},
{rule("src", 0, 0), rule("src", 32, 64), QuotaKept}, // the defaults, written out
{rule("src", 0, 0), rule("", 0, 0), QuotaSummed},
{rule("", 0, 0), rule("src", 0, 0), QuotaRestarts},
{rule("src", 0, 0), rule("dst", 0, 0), QuotaRestarts},
{rule("src", 0, 0), rule("src", 24, 56), QuotaRegrouped},
{rule("src", 0, 0), rule("src", 24, 128), QuotaRestarts},
{nil, rule("src", 0, 0), QuotaKept},
} {
if got, _ := QuotaCarry(c.old, c.new); got != c.want {
t.Errorf("%+v -> %+v: %q, want %q", c.old, c.new, got, c.want)
}
}
}
+152
View File
@@ -0,0 +1,152 @@
// Package names finds the names of addresses by reverse lookups, for
// showing them beside the addresses (connections, the log, quotas,
// traffic per device). Names are cached and never stored: statistics stay
// keyed by address.
package names
import (
"context"
"errors"
"net"
"net/netip"
"strings"
"sync"
"time"
)
// Lookups running at once, and the most names kept.
const (
parallel = 16
maxCache = 20000
)
// Resolver looks up and caches addresses' names.
type Resolver struct {
lookup func(ctx context.Context, addr string) ([]string, error)
// ttl is how long a name (or its absence) is kept; a lookup that
// failed (a timeout, a server error) is tried again after retry.
ttl, retry time.Duration
timeout time.Duration
now func() time.Time
mu sync.Mutex
cache map[netip.Addr]entry
pending map[netip.Addr]chan struct{}
sem chan struct{}
}
type entry struct {
name string
expires time.Time
}
// New makes a resolver asking server ("10.10.0.1:53"; "" for the
// system's, /etc/hosts then /etc/resolv.conf), keeping names for ttl.
func New(server string, ttl time.Duration) *Resolver {
res := &net.Resolver{PreferGo: true}
if server != "" {
res.Dial = func(ctx context.Context, network, _ string) (net.Conn, error) {
var d net.Dialer
return d.DialContext(ctx, network, server)
}
}
return NewFunc(res.LookupAddr, ttl)
}
// NewFunc makes a resolver with its own lookup (tests).
func NewFunc(lookup func(ctx context.Context, addr string) ([]string, error), ttl time.Duration) *Resolver {
return &Resolver{lookup: lookup, ttl: ttl, retry: time.Minute, timeout: 2 * time.Second, now: time.Now,
cache: map[netip.Addr]entry{}, pending: map[netip.Addr]chan struct{}{}, sem: make(chan struct{}, parallel)}
}
// Names returns the names of addrs: those cached, and those looked up
// within wait. Lookups still running then carry on and are cached for
// the next call. An address without a name is left out.
func (r *Resolver) Names(ctx context.Context, addrs []netip.Addr, wait time.Duration) map[netip.Addr]string {
out := map[netip.Addr]string{}
var waiting []netip.Addr
var done []chan struct{}
now := r.now()
r.mu.Lock()
for _, a := range addrs {
if e, ok := r.cache[a]; ok && now.Before(e.expires) {
if e.name != "" {
out[a] = e.name
}
continue
}
ch, ok := r.pending[a]
if !ok {
ch = make(chan struct{})
r.pending[a] = ch
go r.resolve(a, ch)
}
waiting = append(waiting, a)
done = append(done, ch)
}
r.mu.Unlock()
if len(waiting) == 0 {
return out
}
t := time.NewTimer(wait)
defer t.Stop()
for _, ch := range done {
select {
case <-ch:
case <-t.C:
return r.collect(out, waiting)
case <-ctx.Done():
return r.collect(out, waiting)
}
}
return r.collect(out, waiting)
}
// collect adds what the lookups found for addrs.
func (r *Resolver) collect(out map[netip.Addr]string, addrs []netip.Addr) map[netip.Addr]string {
r.mu.Lock()
defer r.mu.Unlock()
for _, a := range addrs {
if e, ok := r.cache[a]; ok && e.name != "" {
out[a] = e.name
}
}
return out
}
func (r *Resolver) resolve(a netip.Addr, done chan struct{}) {
r.sem <- struct{}{}
ctx, cancel := context.WithTimeout(context.Background(), r.timeout)
ns, err := r.lookup(ctx, a.String())
cancel()
<-r.sem
e := entry{expires: r.now().Add(r.ttl)}
var dnsErr *net.DNSError
switch {
case err == nil && len(ns) > 0:
e.name = strings.TrimSuffix(ns[0], ".")
case err != nil && !(errors.As(err, &dnsErr) && dnsErr.IsNotFound):
e.expires = r.now().Add(r.retry)
}
r.mu.Lock()
if len(r.cache) >= maxCache {
r.prune()
}
r.cache[a] = e
delete(r.pending, a)
r.mu.Unlock()
close(done)
}
// prune drops expired names, or all of them when none has expired.
func (r *Resolver) prune() {
now := r.now()
for a, e := range r.cache {
if !now.Before(e.expires) {
delete(r.cache, a)
}
}
if len(r.cache) >= maxCache {
clear(r.cache)
}
}
+57
View File
@@ -0,0 +1,57 @@
package names
import (
"context"
"errors"
"net"
"net/netip"
"sync/atomic"
"testing"
"time"
)
func TestNames(t *testing.T) {
var calls atomic.Int32
slow := make(chan struct{})
r := NewFunc(func(ctx context.Context, addr string) ([]string, error) {
calls.Add(1)
switch addr {
case "10.0.0.1":
return []string{"nas.example.net."}, nil
case "10.0.0.2":
return nil, &net.DNSError{Err: "no such host", Name: addr, IsNotFound: true}
case "10.0.0.3":
return nil, errors.New("server failure")
case "10.0.0.4":
<-slow
return []string{"slow.example.net."}, nil
}
return nil, nil
}, time.Hour)
a := netip.MustParseAddr
all := []netip.Addr{a("10.0.0.1"), a("10.0.0.2"), a("10.0.0.3"), a("10.0.0.4")}
got := r.Names(context.Background(), all, 200*time.Millisecond)
if len(got) != 1 || got[a("10.0.0.1")] != "nas.example.net" {
t.Fatalf("first call: %v", got)
}
// The slow lookup finishes later and is there next time; the others
// come from the cache, the failure too until it is retried.
close(slow)
time.Sleep(50 * time.Millisecond)
got = r.Names(context.Background(), all, 200*time.Millisecond)
if len(got) != 2 || got[a("10.0.0.4")] != "slow.example.net" {
t.Fatalf("second call: %v", got)
}
if n := calls.Load(); n != 4 {
t.Fatalf("%d lookups, want 4 (cached)", n)
}
// A failure is tried again after retry, a missing name after ttl.
now := time.Now().Add(2 * time.Minute)
r.now = func() time.Time { return now }
r.Names(context.Background(), all, 200*time.Millisecond)
if n := calls.Load(); n != 5 {
t.Fatalf("%d lookups, want 5 (the failure again)", n)
}
}
+130
View File
@@ -15,6 +15,7 @@ import (
"errors" "errors"
"fmt" "fmt"
"net" "net"
"net/netip"
"os" "os"
"os/exec" "os/exec"
"path/filepath" "path/filepath"
@@ -422,6 +423,11 @@ type Firewall struct {
// restore is the quota usage saved before housecarld (or the machine) // restore is the quota usage saved before housecarld (or the machine)
// stopped, put back by the next apply. // stopped, put back by the next apply.
restore *QuotaUsage restore *QuotaUsage
// adjust are quota changes the next apply makes (AdjustQuota).
adjust []QuotaAdjust
// quotaRules are the rules with a quota in the ruleset in force, by
// ID: what their usage was counted by (nil before the first apply).
quotaRules map[int64]*model.Rule
// flowDevs are the flowtable's devices in the ruleset in force, by // flowDevs are the flowtable's devices in the ruleset in force, by
// name, with their interface index then (0: left out, missing). // name, with their interface index then (0: left out, missing).
flowDevs map[string]int flowDevs map[string]int
@@ -462,6 +468,119 @@ func (f *Firewall) RestoreQuotas(u *QuotaUsage) {
f.restore = u f.restore = u
} }
// QuotaAdjust takes Give bytes off what a quota used (never below zero):
// the named quota Name, or the entries of the per-address sets Name_v4
// and Name_v6, Addr's alone if given (the set's key, as listed).
type QuotaAdjust struct {
Name string
Addr string
Give int64
}
// AdjustQuota has the next apply change what a quota used.
func (f *Firewall) AdjustQuota(a QuotaAdjust) {
f.mu.Lock()
defer f.mu.Unlock()
f.adjust = append(f.adjust, a)
}
// apply takes what a's say off u. An entry with nothing left used is
// dropped, so its quota starts again with its next packet.
func (u *QuotaUsage) apply(adj []QuotaAdjust) {
less := func(used, give int64) int64 {
if give >= used {
return 0
}
return used - give
}
for _, a := range adj {
if used, ok := u.Quotas[a.Name]; ok {
u.Quotas[a.Name] = less(used, a.Give)
}
for _, set := range []string{a.Name + "_v4", a.Name + "_v6"} {
var kept []render.Element
for _, e := range u.QuotaSets[set] {
if a.Addr == "" || e.Addr == a.Addr {
e.Used = less(e.Used, a.Give)
}
if e.Used > 0 {
kept = append(kept, e)
}
}
if _, ok := u.QuotaSets[set]; ok {
u.QuotaSets[set] = kept
}
}
}
}
// carry keeps what quotas used when their rules now count differently
// (model.QuotaCarry): per address to a total, the sum; to wider groups,
// the groups' sums; what can't be carried starts again. was are the
// rules in force before (nil: nothing to compare with).
func (u *QuotaUsage) carry(was map[int64]*model.Rule, st *model.State) {
for _, r := range st.Rules {
old := was[r.ID]
carry, _ := model.QuotaCarry(old, r)
if carry == model.QuotaKept {
continue
}
name := render.QuotaName(r.ID)
sets := []string{name + "_v4", name + "_v6"}
switch carry {
case model.QuotaSummed:
var sum int64
for _, set := range sets {
for _, e := range u.QuotaSets[set] {
sum += e.Used
}
}
if u.Quotas == nil {
u.Quotas = map[string]int64{}
}
u.Quotas[name] = sum
case model.QuotaRegrouped:
v4, v6 := r.Groups()
for _, set := range sets {
byKey := map[string]int64{}
var order []string
for _, e := range u.QuotaSets[set] {
a, err := netip.ParseAddr(e.Addr)
if err != nil {
continue
}
bits := v4
if a.Is6() {
bits = v6
}
p, _ := a.Prefix(bits)
k := p.Addr().String()
if _, ok := byKey[k]; !ok {
order = append(order, k)
}
byKey[k] += e.Used
}
if len(order) == 0 {
continue
}
var es []render.Element
for _, k := range order {
es = append(es, render.Element{Addr: k, Used: byKey[k]})
}
u.QuotaSets[set] = es
}
continue
}
// What was counted the other way goes: a total, or the entries.
if carry != model.QuotaSummed {
delete(u.Quotas, name)
}
for _, set := range sets {
delete(u.QuotaSets, set)
}
}
}
// ResetQuota starts a rule's quota again: the named quota, or the // ResetQuota starts a rule's quota again: the named quota, or the
// per-address quota sets (both families). // per-address quota sets (both families).
func (f *Firewall) ResetQuota(ctx context.Context, ruleID int64, perAddress bool) error { func (f *Firewall) ResetQuota(ctx context.Context, ruleID int64, perAddress bool) error {
@@ -531,6 +650,9 @@ func (f *Firewall) Apply(ctx context.Context, st *model.State) error {
usage = old.QuotaUsage usage = old.QuotaUsage
} }
usage.merge(f.restore) usage.merge(f.restore)
usage.carry(f.quotaRules, st)
usage.apply(f.adjust)
f.adjust = nil
opt.Quotas, opt.QuotaSets = usage.Quotas, usage.QuotaSets opt.Quotas, opt.QuotaSets = usage.Quotas, usage.QuotaSets
if old != nil { if old != nil {
opt.DeviceSets = old.DeviceSets opt.DeviceSets = old.DeviceSets
@@ -539,6 +661,14 @@ func (f *Firewall) Apply(ctx context.Context, st *model.State) error {
return err return err
} }
f.restore = nil f.restore = nil
f.quotaRules = map[int64]*model.Rule{}
for _, r := range st.Rules {
if r.Quota != nil {
c, q := *r, *r.Quota // only the quota and the grouping are read
c.Quota = &q
f.quotaRules[r.ID] = &c
}
}
f.flowDevs = map[string]int{} f.flowDevs = map[string]int{}
for _, d := range st.Settings.FlowtableDevices { for _, d := range st.Settings.FlowtableDevices {
if opt.Interfaces == nil || opt.Interfaces[d] { if opt.Interfaces == nil || opt.Interfaces[d] {
+83 -1
View File
@@ -6,6 +6,7 @@ import (
"os" "os"
"os/exec" "os/exec"
"path/filepath" "path/filepath"
"reflect"
"sort" "sort"
"strings" "strings"
"testing" "testing"
@@ -186,6 +187,31 @@ func TestQuotasInContainer(t *testing.T) {
if t2, d2 := used(); t2 < total || d2 < dev { if t2, d2 := used(); t2 < total || d2 < dev {
t.Errorf("after a reload: total %d (was %d), per destination %d (was %d)", t2, total, d2, dev) t.Errorf("after a reload: total %d (was %d), per destination %d (was %d)", t2, total, d2, dev)
} }
// Some given back, and one address started again, by the next apply.
total, dev = used()
fw.AdjustQuota(QuotaAdjust{Name: "quota_r1", Give: 1000})
fw.AdjustQuota(QuotaAdjust{Name: "quota_r2", Addr: "127.0.0.3", Give: 1 << 62})
if err := fw.Apply(ctx, st); err != nil {
t.Fatal(err)
}
if t2, d2 := used(); t2 > total-1000 || t2 < total-2000 || d2 != 0 {
t.Errorf("after giving back: total %d (was %d, gave 1000), per destination %d (was %d, reset)", t2, total, d2, dev)
}
// Counted in total now: what the addresses used carries over.
ping("127.0.0.3")
_, dev = used()
st.Rules[1].Quota.PerDestination = false
if err := fw.Apply(ctx, st); err != nil {
t.Fatal(err)
}
tb, _ := fw.Table(ctx)
if tb.Quotas["quota_r2"] < dev || dev == 0 {
t.Errorf("per destination to a total: %d, was %d", tb.Quotas["quota_r2"], dev)
}
st.Rules[1].Quota.PerDestination = true
if err := fw.Apply(ctx, st); err != nil {
t.Fatal(err)
}
// Reset by hand. // Reset by hand.
if err := fw.ResetQuota(ctx, 1, false); err != nil { if err := fw.ResetQuota(ctx, 1, false); err != nil {
t.Fatal(err) t.Fatal(err)
@@ -200,7 +226,7 @@ func TestQuotasInContainer(t *testing.T) {
// saved usage comes back. // saved usage comes back.
ping("127.0.0.2") ping("127.0.0.2")
ping("127.0.0.3") ping("127.0.0.3")
tb, _ := fw.Table(ctx) tb, _ = fw.Table(ctx)
saved := tb.QuotaUsage saved := tb.QuotaUsage
exec.Command("docker", "exec", name, "nft", "delete", "table", "inet", "housecarl").Run() exec.Command("docker", "exec", name, "nft", "delete", "table", "inet", "housecarl").Run()
fw.RestoreQuotas(&saved) fw.RestoreQuotas(&saved)
@@ -212,6 +238,62 @@ func TestQuotasInContainer(t *testing.T) {
} }
} }
// TestQuotaAdjust: giving back takes off what was used, never below
// zero; an entry with nothing left goes.
func TestQuotaAdjust(t *testing.T) {
u := QuotaUsage{Quotas: map[string]int64{"quota_r1": 5000, "quota_r9": 10},
QuotaSets: map[string][]render.Element{"quota_r2_v4": {{Addr: "10.0.0.1", Used: 3000}, {Addr: "10.0.0.2", Used: 4000}},
"quota_r2_v6": {{Addr: "2001:db8::", Used: 100}}}}
u.apply([]QuotaAdjust{{Name: "quota_r1", Give: 2000}, {Name: "quota_r9", Give: 2000}, {Name: "quota_r2", Addr: "10.0.0.2", Give: 1500},
{Name: "quota_r2", Give: 100}})
if u.Quotas["quota_r1"] != 3000 || u.Quotas["quota_r9"] != 0 {
t.Errorf("named: %v", u.Quotas)
}
want := map[string][]render.Element{"quota_r2_v4": {{Addr: "10.0.0.1", Used: 2900}, {Addr: "10.0.0.2", Used: 2400}}, "quota_r2_v6": nil}
if !reflect.DeepEqual(u.QuotaSets, want) {
t.Errorf("sets: %+v", u.QuotaSets)
}
}
// TestQuotaCarry: usage follows a rule counting its quota differently.
func TestQuotaCarry(t *testing.T) {
perSource := &model.Rule{ID: 2, Quota: &model.Quota{Bytes: 1 << 30, PerSource: true}}
usage := func() QuotaUsage {
return QuotaUsage{Quotas: map[string]int64{}, QuotaSets: map[string][]render.Element{
"quota_r2_v4": {{Addr: "10.0.0.1", Used: 100}, {Addr: "10.0.0.2", Used: 50}},
"quota_r2_v6": {{Addr: "2001:db8:0:1::", Used: 7}, {Addr: "2001:db8:0:2::", Used: 3}}}}
}
was := map[int64]*model.Rule{2: perSource}
with := func(f func(r *model.Rule)) *model.State {
r := *perSource
q := *perSource.Quota
r.Quota = &q
f(&r)
return &model.State{Rules: []*model.Rule{&r}}
}
u := usage()
u.carry(was, with(func(r *model.Rule) { r.Quota.PerSource = false }))
if u.Quotas["quota_r2"] != 160 || len(u.QuotaSets["quota_r2_v4"]) != 0 {
t.Errorf("to a total: %+v", u)
}
u = usage()
u.carry(was, with(func(r *model.Rule) { r.GroupV4, r.GroupV6 = 24, 48 }))
want := map[string][]render.Element{"quota_r2_v4": {{Addr: "10.0.0.0", Used: 150}}, "quota_r2_v6": {{Addr: "2001:db8::", Used: 10}}}
if !reflect.DeepEqual(u.QuotaSets, want) {
t.Errorf("wider groups: %+v", u.QuotaSets)
}
u = usage()
u.carry(was, with(func(r *model.Rule) { r.Quota.PerSource, r.Quota.PerDestination = false, true }))
if len(u.QuotaSets) != 0 {
t.Errorf("per destination now: %+v", u.QuotaSets)
}
u = usage()
u.carry(nil, with(func(r *model.Rule) { r.Quota.PerSource = false }))
if len(u.QuotaSets["quota_r2_v4"]) != 2 || u.Quotas["quota_r2"] != 0 {
t.Errorf("nothing to compare with (the first apply): %+v", u)
}
}
// TestDeviceCountsInContainer: the per-device counters are read back and // TestDeviceCountsInContainer: the per-device counters are read back and
// survive a reload. // survive a reload.
func TestDeviceCountsInContainer(t *testing.T) { func TestDeviceCountsInContainer(t *testing.T) {
+14 -1
View File
@@ -6,6 +6,7 @@ import (
"net/http" "net/http"
"net/netip" "net/netip"
"sort" "sort"
"strconv"
"strings" "strings"
udiff "github.com/aymanbagabas/go-udiff" udiff "github.com/aymanbagabas/go-udiff"
@@ -310,7 +311,19 @@ func (s *Server) createObject(w http.ResponseWriter, r *http.Request, p *auth.Pr
if _, err := decodeObject(kind, raw); err != nil { if _, err := decodeObject(kind, raw); err != nil {
return err return err
} }
return s.change(w, r, p, model.Op{Op: model.OpCreate, Kind: kind, Data: raw}) op := model.Op{Op: model.OpCreate, Kind: kind, Data: raw}
// ?before=ID, ?after=ID or ?first=1: where it goes in its order.
q := r.URL.Query()
var place api.MoveInput
place.Before, _ = strconv.ParseInt(q.Get("before"), 10, 64)
place.After, _ = strconv.ParseInt(q.Get("after"), 10, 64)
if queryBool(r, "first") {
place.To = "start"
}
if place.Before != 0 || place.After != 0 || place.To != "" {
op.Place, _ = json.Marshal(place)
}
return s.change(w, r, p, op)
} }
func (s *Server) updateObject(w http.ResponseWriter, r *http.Request, p *auth.Principal) error { func (s *Server) updateObject(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
+78
View File
@@ -4,6 +4,7 @@ import (
"context" "context"
"fmt" "fmt"
"net/http" "net/http"
"net/netip"
"sort" "sort"
"strconv" "strconv"
"strings" "strings"
@@ -88,6 +89,83 @@ func (s *Server) listConnections(w http.ResponseWriter, r *http.Request, p *auth
return nil return nil
} }
// lookupNames finds the names of the inside addresses asked about
// (docs/configuration.md#names), waiting at most a second for lookups.
func (s *Server) lookupNames(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
if err := require(p, auth.ResLog, false); err != nil {
return err
}
var in api.NamesInput
if err := decodeJSON(r, &in); err != nil {
return err
}
if len(in.Addresses) > maxNames {
return badRequest("ask for at most %d addresses at once", maxNames)
}
out := api.Names{Enabled: s.cfg.Names != nil, Names: map[string]string{}}
if s.cfg.Names == nil {
writeJSON(w, http.StatusOK, out)
return nil
}
inside := s.insideNetworks(s.eng.State())
asked := map[netip.Addr][]string{}
var addrs []netip.Addr
for _, a := range in.Addresses {
ad, err := netip.ParseAddr(a)
if err != nil || !inside(ad.Unmap()) {
continue
}
ad = ad.Unmap()
if _, ok := asked[ad]; !ok {
addrs = append(addrs, ad)
}
asked[ad] = append(asked[ad], a)
}
for a, n := range s.cfg.Names.Names(r.Context(), addrs, time.Second) {
for _, as := range asked[a] {
out.Names[as] = n
}
}
writeJSON(w, http.StatusOK, out)
return nil
}
// maxNames is the most addresses one POST /names may ask about.
const maxNames = 2000
// insideNetworks tells inside addresses: in a network connected to an
// inside zone's interface, or private and not in an external zone's.
func (s *Server) insideNetworks(st *model.State) func(netip.Addr) bool {
external := map[string]bool{}
for _, z := range st.Zones {
external[z.Name] = z.External
}
var in, out []netip.Prefix
for p, zone := range s.connected(st) {
if external[zone] {
out = append(out, p)
} else {
in = append(in, p)
}
}
return func(a netip.Addr) bool {
if a.IsLoopback() || a.IsUnspecified() || a.IsMulticast() || a.IsLinkLocalUnicast() {
return false
}
for _, p := range in {
if p.Contains(a) {
return true
}
}
for _, p := range out {
if p.Contains(a) {
return false
}
}
return a.IsPrivate()
}
}
// closeConnections deletes tracking entries: the rules check those // closeConnections deletes tracking entries: the rules check those
// connections again with their next packet. // connections again with their next packet.
func (s *Server) closeConnections(w http.ResponseWriter, r *http.Request, p *auth.Principal) error { func (s *Server) closeConnections(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
+11 -1
View File
@@ -6,6 +6,7 @@ import (
"fmt" "fmt"
"net/http" "net/http"
"sort" "sort"
"strconv"
"strings" "strings"
"housecarl/internal/api" "housecarl/internal/api"
@@ -63,7 +64,16 @@ func createOp(st *model.State, op *model.Op) (int64, error) {
*pos = lastPosition(st, op.Kind, chainOf(o)) *pos = lastPosition(st, op.Kind, chainOf(o))
} }
data, _ := json.Marshal(o) data, _ := json.Marshal(o)
return op.ID, st.Put(op.Kind, op.ID, data) if err := st.Put(op.Kind, op.ID, data); err != nil {
return 0, err
}
// Placed in the same change, so it is never in force anywhere else.
if len(op.Place) > 0 {
if _, err := moveOp(st, &model.Op{Op: model.OpMove, Kind: op.Kind, Ref: strconv.FormatInt(op.ID, 10), Data: op.Place}); err != nil {
return 0, err
}
}
return op.ID, nil
} }
func updateOp(st *model.State, op *model.Op) (int64, error) { func updateOp(st *model.State, op *model.Op) (int64, error) {
+94 -1
View File
@@ -4,8 +4,11 @@ import (
"context" "context"
"encoding/json" "encoding/json"
"fmt" "fmt"
"math"
"net/http" "net/http"
"net/netip"
"sort" "sort"
"strings"
"time" "time"
"housecarl/internal/api" "housecarl/internal/api"
@@ -237,7 +240,8 @@ func (s *Server) listQuotas(w http.ResponseWriter, r *http.Request, p *auth.Prin
return nil return nil
} }
// resetQuota starts a rule's quota again now. // resetQuota starts a rule's quota again now: all of it, or one
// address's; or gives some of what it used back, until its next reset.
func (s *Server) resetQuota(w http.ResponseWriter, r *http.Request, p *auth.Principal) error { func (s *Server) resetQuota(w http.ResponseWriter, r *http.Request, p *auth.Principal) error {
if err := require(p, auth.ResConfig, true); err != nil { if err := require(p, auth.ResConfig, true); err != nil {
return err return err
@@ -250,6 +254,14 @@ func (s *Server) resetQuota(w http.ResponseWriter, r *http.Request, p *auth.Prin
if rule.Quota == nil { if rule.Quota == nil {
return badRequest("%s has no quota", rule.Label()) return badRequest("%s has no quota", rule.Label())
} }
var in api.QuotaReset
if err := decodeOptional(r, &in); err != nil {
return err
}
if in.Give < 0 {
return badRequest("give a number of bytes to give back, 1 or more")
}
if in.Address == "" && in.Give == 0 {
if err := s.fw.ResetQuota(r.Context(), rule.ID, rule.Quota.PerAddress() != ""); err != nil { if err := s.fw.ResetQuota(r.Context(), rule.ID, rule.Quota.PerAddress() != ""); err != nil {
return err return err
} }
@@ -260,4 +272,85 @@ func (s *Server) resetQuota(w http.ResponseWriter, r *http.Request, p *auth.Prin
s.eng.Audit(r.Context(), p, "quota.reset", rule.Label(), "success", "") s.eng.Audit(r.Context(), p, "quota.reset", rule.Label(), "success", "")
writeJSON(w, http.StatusOK, map[string]any{"reset": rule.Label()}) writeJSON(w, http.StatusOK, map[string]any{"reset": rule.Label()})
return nil return nil
}
name := render.QuotaName(rule.ID)
adj := nft.QuotaAdjust{Name: name, Give: in.Give}
if in.Address != "" {
if rule.Quota.PerAddress() == "" {
return badRequest("%s has one quota for everything it matches, not one per address: leave out the address", rule.Label())
}
key, err := quotaKey(rule, in.Address)
if err != nil {
return err
}
if !s.quotaHas(r.Context(), name, key) {
return notFound("%s hasn't used the quota of %s since its last reset", in.Address, rule.Label())
}
adj.Addr = key
}
if adj.Give == 0 {
adj.Give = math.MaxInt64 // all of it: the address starts again
}
// The next apply writes the quota's new usage into the ruleset.
s.fw.AdjustQuota(adj)
if err := s.eng.ApplyCurrent(r.Context()); err != nil {
return fmt.Errorf("putting the ruleset in force with the quota's new usage: %w", err)
}
s.SaveQuotaUsage(r.Context())
detail, out := "", map[string]any{}
if in.Give > 0 {
detail = fmt.Sprintf("gave back %d bytes", in.Give)
out["given"] = in.Give
}
if in.Address != "" {
detail = strings.TrimSpace(detail + " " + in.Address)
out["address"] = in.Address
}
action := "quota.reset"
if in.Give > 0 {
action = "quota.give"
}
s.eng.Audit(r.Context(), p, action, rule.Label(), "success", detail)
out["reset"] = rule.Label()
writeJSON(w, http.StatusOK, out)
return nil
}
// quotaKey is the entry of a per-address quota an address counts under:
// the address, or its group's first address (10.0.10.0 for a /24; an
// IPv6 address's /64).
func quotaKey(rule *model.Rule, s string) (string, error) {
var a netip.Addr
if p, err := netip.ParsePrefix(s); err == nil {
a = p.Addr()
} else if a, err = netip.ParseAddr(s); err != nil {
return "", badRequest("%q isn't an address", s)
}
a = a.Unmap()
v4, v6 := rule.Groups()
bits := v4
if a.Is6() {
bits = v6
}
p, err := a.Prefix(bits)
if err != nil {
return "", badRequest("%q: %v", s, err)
}
return p.Addr().String(), nil
}
// quotaHas reports whether a per-address quota has an entry for key.
func (s *Server) quotaHas(ctx context.Context, name, key string) bool {
t, err := s.fw.Table(ctx)
if err != nil {
return false
}
for _, set := range []string{name + "_v4", name + "_v6"} {
for _, e := range t.QuotaSets[set] {
if e.Addr == key {
return true
}
}
}
return false
} }
+1
View File
@@ -41,6 +41,7 @@ func (s *Server) routes() {
m.HandleFunc("GET "+p+"/quotas", s.authed(s.listQuotas)) m.HandleFunc("GET "+p+"/quotas", s.authed(s.listQuotas))
m.HandleFunc("GET "+p+"/connections", s.authed(s.listConnections)) m.HandleFunc("GET "+p+"/connections", s.authed(s.listConnections))
m.HandleFunc("POST "+p+"/connections/close", s.authed(s.closeConnections)) m.HandleFunc("POST "+p+"/connections/close", s.authed(s.closeConnections))
m.HandleFunc("POST "+p+"/names", s.authed(s.lookupNames))
m.HandleFunc("GET "+p+"/trace", s.authed(s.getTrace)) m.HandleFunc("GET "+p+"/trace", s.authed(s.getTrace))
m.HandleFunc("POST "+p+"/trace", s.authed(s.startTrace)) m.HandleFunc("POST "+p+"/trace", s.authed(s.startTrace))
m.HandleFunc("DELETE "+p+"/trace", s.authed(s.stopTrace)) m.HandleFunc("DELETE "+p+"/trace", s.authed(s.stopTrace))
+15
View File
@@ -23,6 +23,7 @@ import (
"housecarl/internal/certs" "housecarl/internal/certs"
"housecarl/internal/engine" "housecarl/internal/engine"
"housecarl/internal/model" "housecarl/internal/model"
"housecarl/internal/names"
"housecarl/internal/nft" "housecarl/internal/nft"
"housecarl/internal/render" "housecarl/internal/render"
"housecarl/internal/stats" "housecarl/internal/stats"
@@ -51,6 +52,8 @@ type Config struct {
// after or before, for the status. // after or before, for the status.
Shadow bool Shadow bool
ShadowPosition string ShadowPosition string
// Names looks up inside addresses' names (nil: names are off).
Names *names.Resolver
} }
// Interfaces lists the host's interfaces with their addresses, and // Interfaces lists the host's interfaces with their addresses, and
@@ -76,6 +79,8 @@ type Firewall interface {
SaveBoot(st *model.State) error SaveBoot(st *model.State) error
// ResetQuota starts a rule's quota again (perAddress: its sets). // ResetQuota starts a rule's quota again (perAddress: its sets).
ResetQuota(ctx context.Context, ruleID int64, perAddress bool) error ResetQuota(ctx context.Context, ruleID int64, perAddress bool) error
// AdjustQuota has the next apply change what a quota used.
AdjustQuota(a nft.QuotaAdjust)
// Trace follows packets (internal/trace) until ctx ends. // Trace follows packets (internal/trace) until ctx ends.
Trace(ctx context.Context, script string, line func(string)) error Trace(ctx context.Context, script string, line func(string)) error
} }
@@ -136,6 +141,16 @@ func New(cfg Config, eng *engine.Engine, fw Firewall, ifaces Interfaces, log *sl
s := &Server{cfg: cfg, eng: eng, fw: fw, ifaces: ifaces, store: eng.Store(), log: log, mux: http.NewServeMux(), limiter: newRateLimiter(), s := &Server{cfg: cfg, eng: eng, fw: fw, ifaces: ifaces, store: eng.Store(), log: log, mux: http.NewServeMux(), limiter: newRateLimiter(),
web: web, started: time.Now(), logs: newLogRing(2000), nowFn: time.Now, unitLinks: nftablesUnitLinks} web: web, started: time.Now(), logs: newLogRing(2000), nowFn: time.Now, unitLinks: nftablesUnitLinks}
s.routes() s.routes()
// What the quotas used is saved after every change too, not only every
// five minutes: a change that counts a quota differently converts its
// usage, and a restart must not bring back the old kind.
prev := eng.OnChange
eng.OnChange = func(st *model.State) {
if prev != nil {
prev(st)
}
s.SaveQuotaUsage(context.Background())
}
return s return s
} }
+137 -2
View File
@@ -7,11 +7,13 @@ import (
"errors" "errors"
"fmt" "fmt"
"io" "io"
"math"
"net/http" "net/http"
"net/http/cookiejar" "net/http/cookiejar"
"net/http/httptest" "net/http/httptest"
"net/netip" "net/netip"
"os" "os"
"reflect"
"strings" "strings"
"sync" "sync"
"testing" "testing"
@@ -21,6 +23,7 @@ import (
"housecarl/internal/auth" "housecarl/internal/auth"
"housecarl/internal/engine" "housecarl/internal/engine"
"housecarl/internal/model" "housecarl/internal/model"
"housecarl/internal/names"
"housecarl/internal/nft" "housecarl/internal/nft"
"housecarl/internal/render" "housecarl/internal/render"
"housecarl/internal/stats" "housecarl/internal/stats"
@@ -43,6 +46,8 @@ type fakeFW struct {
fallback bool // the boot fallback is in force fallback bool // the boot fallback is in force
shadow bool shadow bool
ruleset string // nft -j list ruleset ("": a default) ruleset string // nft -j list ruleset ("": a default)
quotas nft.QuotaUsage
adjusted []nft.QuotaAdjust
} }
func (f *fakeFW) Check(_ context.Context, st *model.State) error { return nil } func (f *fakeFW) Check(_ context.Context, st *model.State) error { return nil }
@@ -72,7 +77,12 @@ func (f *fakeFW) Table(context.Context) (*nft.Table, error) {
if len(f.applied) == 0 { if len(f.applied) == 0 {
return nil, errors.New("No such file or directory: list table inet housecarl") return nil, errors.New("No such file or directory: list table inet housecarl")
} }
return &nft.Table{Counters: map[string]nft.Counter{"r1": {Packets: 3, Bytes: 180}}, Elements: f.elements, Fallback: f.fallback}, nil return &nft.Table{Counters: map[string]nft.Counter{"r1": {Packets: 3, Bytes: 180}}, Elements: f.elements, Fallback: f.fallback, QuotaUsage: f.quotas}, nil
}
func (f *fakeFW) AdjustQuota(a nft.QuotaAdjust) {
f.mu.Lock()
defer f.mu.Unlock()
f.adjusted = append(f.adjusted, a)
} }
func (f *fakeFW) AddElement(_ context.Context, set, addr string, timeout int) error { func (f *fakeFW) AddElement(_ context.Context, set, addr string, timeout int) error {
f.mu.Lock() f.mu.Lock()
@@ -748,9 +758,53 @@ func TestQuotaResets(t *testing.T) {
if len(qs) != 1 || qs[0].NextReset == nil || qs[0].NextReset.Day() != 21 { if len(qs) != 1 || qs[0].NextReset == nil || qs[0].NextReset.Day() != 21 {
t.Errorf("quotas %+v", qs) t.Errorf("quotas %+v", qs)
} }
if code := e.call(e.token, "POST", "/rules/"+itoa(created.ID)+"/reset-quota", nil, nil); code != 200 || len(e.fw.resets) != 2 { reset := "/rules/" + itoa(created.ID) + "/reset-quota"
if code := e.call(e.token, "POST", reset, nil, nil); code != 200 || len(e.fw.resets) != 2 {
t.Errorf("reset by hand: %d %v", code, e.fw.resets) t.Errorf("reset by hand: %d %v", code, e.fw.resets)
} }
// One address started again, or some given back: the next apply
// (made at once) writes the new usage.
name := render.QuotaName(created.ID)
e.fw.quotas = nft.QuotaUsage{QuotaSets: map[string][]render.Element{name + "_v4": {{Addr: "10.0.10.23", Used: 6 << 30}},
name + "_v6": {{Addr: "2001:db8:1:2::", Used: 1 << 30}}}}
applies := len(e.fw.applied)
if code := e.call(e.token, "POST", reset, api.QuotaReset{Address: "10.0.10.23"}, nil); code != 200 {
t.Errorf("reset one address: %d", code)
}
if code := e.call(e.token, "POST", reset, api.QuotaReset{Address: "2001:db8:1:2:aaaa::7", Give: 2 << 30}, nil); code != 200 {
t.Errorf("give back to an IPv6 address (its /64): %d", code)
}
if code := e.call(e.token, "POST", reset, api.QuotaReset{Give: 1 << 30}, nil); code != 200 {
t.Errorf("give back to everyone: %d", code)
}
want3 := []nft.QuotaAdjust{{Name: name, Addr: "10.0.10.23", Give: math.MaxInt64}, {Name: name, Addr: "2001:db8:1:2::", Give: 2 << 30}, {Name: name, Give: 1 << 30}}
if !reflect.DeepEqual(e.fw.adjusted, want3) || len(e.fw.applied) != applies+3 || len(e.fw.resets) != 2 {
t.Errorf("adjusted %+v, applied %d more, resets %v", e.fw.adjusted, len(e.fw.applied)-applies, e.fw.resets)
}
for _, c := range []struct {
in api.QuotaReset
code int
}{{api.QuotaReset{Address: "10.0.10.99"}, 404}, {api.QuotaReset{Address: "nonsense"}, 400}, {api.QuotaReset{Give: -5}, 400}} {
if code := e.call(e.token, "POST", reset, c.in, nil); code != c.code {
t.Errorf("%+v: %d, want %d", c.in, code, c.code)
}
}
}
// TestQuotaUsageSavedOnChange: a change saves what the quotas used at
// once (a restart must not bring back usage counted the old way).
func TestQuotaUsageSavedOnChange(t *testing.T) {
e := newEnv(t)
e.call(e.token, "POST", "/zones", model.Zone{Name: "lan", Interfaces: []string{"eth1"}}, nil)
e.fw.mu.Lock()
e.fw.quotas = nft.QuotaUsage{Quotas: map[string]int64{"quota_r7": 4242}}
e.fw.mu.Unlock()
e.call(e.token, "POST", "/zones", model.Zone{Name: "dmz", Interfaces: []string{"eth2"}}, nil)
v, err := e.eng.Store().GetMeta(context.Background(), QuotaUsageKey)
if err != nil || !strings.Contains(v, `"quota_r7":4242`) {
t.Errorf("saved usage %q %v", v, err)
}
} }
// TestAutoConfirm: the client that made a change confirms it by itself: // TestAutoConfirm: the client that made a change confirms it by itself:
@@ -982,6 +1036,41 @@ func TestConnections(t *testing.T) {
} }
} }
// TestNames: names only for inside addresses (an inside zone's network,
// or private outside the external zones'), under the address as asked.
func TestNames(t *testing.T) {
var asked []string
var mu sync.Mutex
e := newEnvWith(t, Config{Names: names.NewFunc(func(_ context.Context, a string) ([]string, error) {
mu.Lock()
asked = append(asked, a)
mu.Unlock()
return []string{"host-" + strings.ReplaceAll(a, ".", "-") + ".example.net."}, nil
}, time.Hour)})
e.call(e.token, "POST", "/zones", model.Zone{Name: "wan", Interfaces: []string{"wan"}, External: true}, nil)
e.call(e.token, "POST", "/zones", model.Zone{Name: "lan", Interfaces: []string{"eth1"}}, nil)
var out api.Names
e.call(e.token, "POST", "/names", api.NamesInput{Addresses: []string{"10.0.10.5", "::ffff:10.0.10.5", "192.168.7.7", "217.155.8.194", "8.8.8.8", "fe80::1", "nonsense"}}, &out)
want := map[string]string{"10.0.10.5": "host-10-0-10-5.example.net", "::ffff:10.0.10.5": "host-10-0-10-5.example.net", "192.168.7.7": "host-192-168-7-7.example.net"}
if !out.Enabled || !reflect.DeepEqual(out.Names, want) {
t.Errorf("names: %+v", out)
}
if len(asked) != 2 {
t.Errorf("looked up %v, want the two inside addresses once each", asked)
}
var tok api.Token
e.call(e.token, "POST", "/me/tokens", api.TokenInput{Name: "monitor", Role: "log"}, &tok)
var one, none api.Names
if code := e.call(tok.Secret, "POST", "/names", api.NamesInput{Addresses: []string{"10.0.10.5"}}, &one); code != 200 || len(one.Names) != 1 {
t.Errorf("with log:read: %d %+v", code, one)
}
off := newEnv(t)
off.call(off.token, "POST", "/names", api.NamesInput{Addresses: []string{"10.0.10.5"}}, &none)
if none.Enabled || len(none.Names) != 0 {
t.Errorf("names off: %+v", none)
}
}
// TestTrace: a trace runs with the filter's table, collects packets named // TestTrace: a trace runs with the filter's table, collects packets named
// after Housecarl's objects, and stops when asked; one at a time. // after Housecarl's objects, and stops when asked; one at a time.
func TestTrace(t *testing.T) { func TestTrace(t *testing.T) {
@@ -1184,3 +1273,49 @@ func (e *env) text(path string) string {
b, _ := io.ReadAll(resp.Body) b, _ := io.ReadAll(resp.Body)
return string(b) return string(b)
} }
// TestCreatePlaced: a rule created ?before= another is placed in the same
// change (one history entry), also when staged in a changeset.
func TestCreatePlaced(t *testing.T) {
e := newEnv(t)
var a, b, c api.MutationResponse
e.call(e.token, "POST", "/rules", model.Rule{Chain: "input", Match: model.Match{Services: []string{"ssh"}}}, &a)
e.call(e.token, "POST", "/rules", model.Rule{Chain: "input", Match: model.Match{Services: []string{"dns"}}}, &b)
var first model.Rule
json.Unmarshal(a.Object, &first)
if code := e.call(e.token, "POST", "/rules?before="+itoa(first.ID), model.Rule{Chain: "input", Match: model.Match{Services: []string{"http"}}, Comment: "placed"}, &c); code != 201 {
t.Fatalf("create placed: %d %+v", code, c)
}
order := func() string {
var out []string
for _, r := range e.eng.State().Rules {
out = append(out, strings.Join(r.Services, ""))
}
return strings.Join(out, ",")
}
if o := order(); o != "http,ssh,dns" {
t.Errorf("order: %s", o)
}
var hs []model.HistoryEntry
e.call(e.token, "GET", "/history", nil, &hs)
if len(hs) != 3 || len(hs[0].Changes) != 1 || hs[0].Changes[0].Action() != "create" {
t.Errorf("history: %d entries, newest %+v", len(hs), hs[0])
}
// Staged, the place goes with it.
var cs model.Changeset
e.call(e.token, "POST", "/changesets", nil, &cs)
e.call(e.token, "POST", "/rules?first=1&changeset="+itoa(cs.ID), model.Rule{Chain: "input", Match: model.Match{Services: []string{"ntp"}}}, nil)
if code := e.call(e.token, "POST", "/changesets/"+itoa(cs.ID)+"/apply", nil, nil); code != 200 {
t.Fatalf("apply: %d", code)
}
if o := order(); o != "ntp,http,ssh,dns" {
t.Errorf("order after the changeset: %s", o)
}
// A bad place: refused, nothing created.
if code := e.call(e.token, "POST", "/rules?before=99999", model.Rule{Chain: "input"}, nil); code != 400 {
t.Errorf("before a rule that isn't there: %d", code)
}
if n := len(e.eng.State().Rules); n != 4 {
t.Errorf("%d rules", n)
}
}
+4 -1
View File
@@ -402,7 +402,10 @@ func (c *Collector) sample(ctx context.Context, minute int64) map[string]int64 {
for set, es := range t.QuotaSets { for set, es := range t.QuotaSets {
kind := strings.TrimSuffix(strings.TrimSuffix(set, "_v4"), "_v6") kind := strings.TrimSuffix(strings.TrimSuffix(set, "_v4"), "_v6")
for _, e := range es { for _, e := range es {
addTop(kind, e.Addr, c.table.of(set+"/"+e.Addr, e.Used)) d := c.table.of(set+"/"+e.Addr, e.Used)
addTop(kind, e.Addr, d)
// A per-address quota's series is its addresses' sum.
add("quota:"+strings.TrimPrefix(kind, "quota_"), d)
} }
} }
for set, es := range t.DeviceSets { for set, es := range t.DeviceSets {
+6
View File
@@ -85,6 +85,10 @@ func TestCollector(t *testing.T) {
h.table.Counters["r1"] = nft.Counter{Packets: 10, Bytes: 1000} h.table.Counters["r1"] = nft.Counter{Packets: 10, Bytes: 1000}
h.table.Counters["r2"] = nft.Counter{Packets: 100, Bytes: 9000} h.table.Counters["r2"] = nft.Counter{Packets: 100, Bytes: 9000}
h.table.Quotas["quota_r5"] = 1 << 20 h.table.Quotas["quota_r5"] = 1 << 20
if h.table.QuotaSets == nil {
h.table.QuotaSets = map[string][]render.Element{}
}
h.table.QuotaSets["quota_r6_v4"] = []render.Element{{Addr: "10.0.10.5", Used: 1000}}
h.table.DeviceSets["stats_up_v4"] = []render.Element{{Addr: "10.0.10.5", Bytes: 5000}} h.table.DeviceSets["stats_up_v4"] = []render.Element{{Addr: "10.0.10.5", Bytes: 5000}}
store, err := OpenStore(":memory:") store, err := OpenStore(":memory:")
@@ -104,6 +108,7 @@ func TestCollector(t *testing.T) {
h.table.Counters["r1"] = nft.Counter{Packets: 25, Bytes: 2500} h.table.Counters["r1"] = nft.Counter{Packets: 25, Bytes: 2500}
h.table.Counters["p-input"] = nft.Counter{Packets: 3, Bytes: 180} // a new counter counts whole h.table.Counters["p-input"] = nft.Counter{Packets: 3, Bytes: 180} // a new counter counts whole
h.table.Quotas["quota_r5"] = 3 << 20 h.table.Quotas["quota_r5"] = 3 << 20
h.table.QuotaSets["quota_r6_v4"] = []render.Element{{Addr: "10.0.10.5", Used: 2000}, {Addr: "10.0.10.6", Used: 500}} // a new entry counts whole
h.table.DeviceSets["stats_up_v4"] = []render.Element{{Addr: "10.0.10.5", Bytes: 8000}} h.table.DeviceSets["stats_up_v4"] = []render.Element{{Addr: "10.0.10.5", Bytes: 8000}}
h.table.DeviceSets["stats_down_v6"] = []render.Element{{Addr: "2001:db8:1:2::", Bytes: 70000}} h.table.DeviceSets["stats_down_v6"] = []render.Element{{Addr: "2001:db8:1:2::", Bytes: 70000}}
for range 2 { for range 2 {
@@ -142,6 +147,7 @@ func TestCollector(t *testing.T) {
check("drop:forward", sum("drop:")["forward"], 15) check("drop:forward", sum("drop:")["forward"], 15)
check("drop:input", sum("drop:")["input"], 3) check("drop:input", sum("drop:")["input"], 3)
check("quota:r5", sum("quota:")["r5"], 2<<20) check("quota:r5", sum("quota:")["r5"], 2<<20)
check("quota:r6 (per address: the sum)", sum("quota:")["r6"], 1500)
s, err := q.Summary(ctx, time.Hour, 10) s, err := q.Summary(ctx, time.Hour, 10)
if err != nil { if err != nil {
+2 -2
View File
@@ -331,7 +331,7 @@ export function heatmapTable(hm, unit) {
// so the list is its own table view. // so the list is its own table view.
// extra(key), when given, adds something after an entry's label (e.g. a // extra(key), when given, adds something after an entry's label (e.g. a
// link that excludes it). // link that excludes it).
export function barList(entries, { total, label = (k) => k, empty = "No data yet.", href, extra, format = compact } = {}) { export function barList(entries, { total, label = (k) => k, empty = "No data yet.", href, extra, format = compact, share: showShare = true } = {}) {
if (!entries.length) return h("div", { class: "empty" }, empty); if (!entries.length) return h("div", { class: "empty" }, empty);
// Entries may be in a natural order (e.g. latency buckets), not by size. // Entries may be in a natural order (e.g. latency buckets), not by size.
const peak = Math.max(1, ...entries.map((e) => e.count)); const peak = Math.max(1, ...entries.map((e) => e.count));
@@ -345,7 +345,7 @@ export function barList(entries, { total, label = (k) => k, empty = "No data yet
return h("li", { title: (e.name ? e.name + " (" + name + ")" : name) + ": " + format(e.count) + " (" + share.toFixed(1) + "%)" }, return h("li", { title: (e.name ? e.name + " (" + name + ")" : name) + ": " + format(e.count) + " (" + share.toFixed(1) + "%)" },
h("div", { class: "bar-text" }, h("div", { class: "bar-text" },
link ? h("a", { href: link, class: "bar-label" }, shown) : h("span", { class: "bar-label" }, shown), link ? h("a", { href: link, class: "bar-label" }, shown) : h("span", { class: "bar-label" }, shown),
h("span", { class: "bar-value" }, extra ? [extra(e.key), " "] : null, format(e.count), h("span", { class: "muted" }, " · " + share.toFixed(share < 10 ? 1 : 0) + "%"))), h("span", { class: "bar-value" }, extra ? [extra(e.key), " "] : null, format(e.count), showShare ? h("span", { class: "muted" }, " · " + share.toFixed(share < 10 ? 1 : 0) + "%") : null)),
h("div", { class: "bar-track" }, h("div", { class: "bar-fill", style: { width: Math.max(0.5, (e.count / peak) * 100) + "%" } }))); h("div", { class: "bar-track" }, h("div", { class: "bar-fill", style: { width: Math.max(0.5, (e.count / peak) * 100) + "%" } })));
})); }));
} }
@@ -3,7 +3,7 @@
import { h, mount, input, debounce, toast, confirmDialog, badge } from "../ui.js"; import { h, mount, input, debounce, toast, confirmDialog, badge } from "../ui.js";
import { get, api, can } from "../api.js"; import { get, api, can } from "../api.js";
import { onLeave } from "../app.js"; import { onLeave } from "../app.js";
import { fmtSeconds } from "../widgets.js"; import { fmtSeconds, lookupNames, named } from "../widgets.js";
export async function render(el, params, query) { export async function render(el, params, query) {
const search = input({ type: "search", placeholder: "address, :port, protocol, state…", value: query.get("q") || "" }); const search = input({ type: "search", placeholder: "address, :port, protocol, state…", value: query.get("q") || "" });
@@ -22,14 +22,15 @@ export async function render(el, params, query) {
}; };
const draw = async () => { const draw = async () => {
shown = await get("/connections", { q: search.value.trim(), limit: 500 }); shown = await get("/connections", { q: search.value.trim(), limit: 500 });
const names = await lookupNames(shown.flatMap((c) => [c.source, c.destination, c.dnat_to]));
count.textContent = shown.length === 1 ? "1 connection" : shown.length + " connections"; count.textContent = shown.length === 1 ? "1 connection" : shown.length + " connections";
mount(out, shown.length ? h("div", { class: "table-wrap" }, h("table", null, mount(out, shown.length ? h("div", { class: "table-wrap" }, h("table", null,
h("thead", null, h("tr", null, ["Protocol", "From", "To", "NAT", "State", "Expires", ""].map((t) => h("th", null, t)))), h("thead", null, h("tr", null, ["Protocol", "From", "To", "NAT", "State", "Expires", ""].map((t) => h("th", null, t)))),
h("tbody", null, shown.map((c) => h("tr", null, h("tbody", null, shown.map((c) => h("tr", null,
h("td", null, c.protocol), h("td", null, c.protocol),
h("td", { class: "mono" }, hostPort(c.source, c.source_port)), h("td", { class: "mono" }, named(names, c.source, hostPort(c.source, c.source_port))),
h("td", { class: "mono" }, hostPort(c.destination, c.destination_port)), h("td", { class: "mono" }, named(names, c.destination, hostPort(c.destination, c.destination_port))),
h("td", { class: "mono small" }, c.dnat_to ? "to " + hostPort(c.dnat_to, c.dnat_port || c.destination_port) : "", c.snat_to ? (c.dnat_to ? ", " : "") + "as " + c.snat_to : ""), h("td", { class: "mono small" }, c.dnat_to ? ["to ", named(names, c.dnat_to, hostPort(c.dnat_to, c.dnat_port || c.destination_port))] : "", c.snat_to ? (c.dnat_to ? ", " : "") + "as " + c.snat_to : ""),
h("td", null, c.state || "", c.assured ? null : [" ", badge("one way", "")]), h("td", null, c.state || "", c.assured ? null : [" ", badge("one way", "")]),
h("td", { class: "small" }, fmtSeconds(c.timeout)), h("td", { class: "small" }, fmtSeconds(c.timeout)),
h("td", { class: "actions-cell" }, write ? h("button", { class: "small reveal", onclick: () => close([c], "this connection") }, "Close") : null)))))) : h("td", { class: "actions-cell" }, write ? h("button", { class: "small reveal", onclick: () => close([c], "this connection") }, "Close") : null)))))) :
+25 -5
View File
@@ -1,9 +1,9 @@
// Dashboard: the ruleset's state, problems, interfaces, the busiest rules // Dashboard: the ruleset's state, problems, interfaces, the busiest rules,
// and the latest drops. // the quotas and the latest drops.
import { h, mount, table, badge, problemsList, time } from "../ui.js"; import { h, mount, table, badge, problemsList, time, plural } from "../ui.js";
import { get, can } from "../api.js"; import { get, can } from "../api.js";
import { onLeave } from "../app.js"; import { onLeave } from "../app.js";
import { describe, hits, humanCount } from "../widgets.js"; import { describe, hits, humanCount, fmtBytes, usageBar, lookupNames, named, pct } from "../widgets.js";
import { lineChart, chartCard, legend, timeTable, stepSeconds } from "../charts.js"; import { lineChart, chartCard, legend, timeTable, stepSeconds } from "../charts.js";
import { bytes } from "./stats.js"; import { bytes } from "./stats.js";
@@ -11,12 +11,13 @@ export async function render(el) {
const draw = async () => { const draw = async () => {
const st = await get("/status"); const st = await get("/status");
const config = can("config", false), logs = can("log", false); const config = can("config", false), logs = can("log", false);
const [rules, cs, log, tl, zones] = await Promise.all([ const [rules, cs, log, tl, zones, quotas] = await Promise.all([
config ? get("/rules") : [], config ? get("/rules") : [],
logs ? get("/counters").then((c) => c.counters).catch(() => ({})) : {}, logs ? get("/counters").then((c) => c.counters).catch(() => ({})) : {},
logs ? get("/log", { limit: 8 }).catch(() => []) : [], logs ? get("/log", { limit: 8 }).catch(() => []) : [],
logs ? get("/stats/timeline", { since: "24h", step: "1h", keys: "zrx:*,ztx:*,drops" }).catch(() => null) : null, logs ? get("/stats/timeline", { since: "24h", step: "1h", keys: "zrx:*,ztx:*,drops" }).catch(() => null) : null,
config ? get("/zones").catch(() => []) : [], config ? get("/zones").catch(() => []) : [],
logs ? get("/quotas").catch(() => []) : [],
]); ]);
const busiest = rules.filter((r) => cs["r" + r.id] && cs["r" + r.id].packets > 0) const busiest = rules.filter((r) => cs["r" + r.id] && cs["r" + r.id].packets > 0)
.sort((a, b) => cs["r" + b.id].packets - cs["r" + a.id].packets).slice(0, 8); .sort((a, b) => cs["r" + b.id].packets - cs["r" + a.id].packets).slice(0, 8);
@@ -54,6 +55,7 @@ export async function render(el) {
{ label: "Hits", render: (r) => hits(cs, "r" + r.id) }, { label: "Hits", render: (r) => hits(cs, "r" + r.id) },
{ label: "Comment", render: (r) => r.comment }, { label: "Comment", render: (r) => r.comment },
], busiest)) : null, ], busiest)) : null,
quotas.length ? quotaPanel(quotas, await lookupNames(quotas.map((q) => (q.addresses || [])[0]).filter(Boolean).map((a) => a.address))) : null,
log.length ? h("div", { class: "panel" }, log.length ? h("div", { class: "panel" },
h("div", { class: "panel-head" }, h("h2", null, "Latest logged packets"), h("a", { class: "button small", href: "#/log" }, "The log")), h("div", { class: "panel-head" }, h("h2", null, "Latest logged packets"), h("a", { class: "button small", href: "#/log" }, "The log")),
table([ table([
@@ -69,6 +71,24 @@ export async function render(el) {
onLeave(() => clearInterval(t)); onLeave(() => clearInterval(t));
} }
// quotaPanel lists every rule's quota with what it used: the total, or
// for a per-address quota the busiest address (the rest are on the rule).
function quotaPanel(quotas, names) {
const perAddress = (q) => q.quota.per_source || q.quota.per_destination;
const used = (q) => perAddress(q) ? ((q.addresses || [])[0] || { used: 0 }).used : q.used;
return h("div", { class: "panel" },
h("div", { class: "panel-head" }, h("h2", null, "Quotas"), h("a", { class: "button small", href: "#/stats" }, "Statistics")),
table([
{ label: "Rule", render: (q) => h("a", { href: "#/rules?rule=" + q.rule_id }, q.rule) },
{ label: "Allowance", render: (q) => fmtBytes(q.quota.bytes) + (q.quota.per_source ? " each source" : q.quota.per_destination ? " each destination" : "") },
{ label: "Used", render: (q) => perAddress(q)
? [plural((q.addresses || []).length, "address", "addresses"), q.addresses && q.addresses.length ? [", busiest ", named(names, q.addresses[0].address), " " + fmtBytes(used(q)) + " (" + pct(used(q), q.quota.bytes) + ")"] : ""]
: fmtBytes(q.used) + " (" + pct(q.used, q.quota.bytes) + ")" },
{ label: "", render: (q) => usageBar(used(q), q.quota.bytes) },
{ label: "Next reset", render: (q) => q.next_reset ? time(q.next_reset) : h("span", { class: "muted" }, "by hand") },
], quotas));
}
// statsCharts: the last 24 hours' traffic with the internet and drops, // statsCharts: the last 24 hours' traffic with the internet and drops,
// per hour (the Statistics page has the rest). // per hour (the Statistics page has the rest).
function statsCharts(tl, zones) { function statsCharts(tl, zones) {
+4 -1
View File
@@ -3,6 +3,7 @@
import { h, mount, input, debounce, badge } from "../ui.js"; import { h, mount, input, debounce, badge } from "../ui.js";
import { get, state } from "../api.js"; import { get, state } from "../api.js";
import { onLeave } from "../app.js"; import { onLeave } from "../app.js";
import { lookupNames, named } from "../widgets.js";
export async function render(el, params, query) { export async function render(el, params, query) {
const search = input({ type: "search", placeholder: "address, interface, rule…", value: query.get("q") || "" }); const search = input({ type: "search", placeholder: "address, interface, rule…", value: query.get("q") || "" });
@@ -12,6 +13,7 @@ export async function render(el, params, query) {
const addr = (a, port) => port ? (a.includes(":") ? "[" + a + "]:" + port : a + ":" + port) : a; const addr = (a, port) => port ? (a.includes(":") ? "[" + a + "]:" + port : a + ":" + port) : a;
const draw = async () => { const draw = async () => {
const es = await get("/log", { limit: 300, q: search.value.trim() }); const es = await get("/log", { limit: 300, q: search.value.trim() });
const names = await lookupNames(es.flatMap((e) => [e.source, e.destination]));
mount(out, es.length ? h("div", { class: "table-wrap" }, h("table", { class: "activity" }, mount(out, es.length ? h("div", { class: "table-wrap" }, h("table", { class: "activity" },
h("thead", null, h("tr", null, ["Time", "Why", "In", "Out", "From", "To", "Protocol"].map((t) => h("th", null, t)))), h("thead", null, h("tr", null, ["Time", "Why", "In", "Out", "From", "To", "Protocol"].map((t) => h("th", null, t)))),
h("tbody", null, es.map((e) => h("tr", null, h("tbody", null, es.map((e) => h("tr", null,
@@ -19,7 +21,8 @@ export async function render(el, params, query) {
h("td", null, e.mark && e.mark.startsWith("r") ? h("a", { href: "#/rules?rule=" + e.mark.slice(1), title: "rule #" + e.mark.slice(1) }, e.prefix) : e.prefix, h("td", null, e.mark && e.mark.startsWith("r") ? h("a", { href: "#/rules?rule=" + e.mark.slice(1), title: "rule #" + e.mark.slice(1) }, e.prefix) : e.prefix,
e.shadow && /^(drop|reject)$/.test(e.verdict) ? [" ", badge("would " + e.verdict, "warn")] : null), e.shadow && /^(drop|reject)$/.test(e.verdict) ? [" ", badge("would " + e.verdict, "warn")] : null),
h("td", { class: "mono" }, e.in || ""), h("td", { class: "mono" }, e.out || ""), h("td", { class: "mono" }, e.in || ""), h("td", { class: "mono" }, e.out || ""),
h("td", { class: "mono" }, addr(e.source, e.source_port)), h("td", { class: "mono" }, addr(e.destination, e.destination_port)), h("td", { class: "mono" }, named(names, e.source, addr(e.source, e.source_port))),
h("td", { class: "mono" }, named(names, e.destination, addr(e.destination, e.destination_port))),
h("td", null, e.protocol, e.icmp_type ? " " + e.icmp_type : "", e.tcp_flags ? h("span", { class: "muted small" }, " " + e.tcp_flags) : null)))))) : h("td", null, e.protocol, e.icmp_type ? " " + e.icmp_type : "", e.tcp_flags ? h("span", { class: "muted small" }, " " + e.tcp_flags) : null)))))) :
h("div", { class: "empty" }, state.me.mode === "shadow" h("div", { class: "empty" }, state.me.mode === "shadow"
? "Nothing logged yet. In shadow mode, what the policies would drop is logged; turn logging on for a rule with Edit → Log." ? "Nothing logged yet. In shadow mode, what the policies would drop is logged; turn logging on for a rule with Edit → Log."
+68 -28
View File
@@ -2,7 +2,7 @@
import { h, mount, table, modal, errorBox, field, input, select, checkbox, confirmDialog, badge, textarea, toast } from "../ui.js"; import { h, mount, table, modal, errorBox, field, input, select, checkbox, confirmDialog, badge, textarea, toast } from "../ui.js";
import { get, api, change, can } from "../api.js"; import { get, api, change, can } from "../api.js";
import { rerender } from "../app.js"; import { rerender } from "../app.js";
import { names, matchEditor, describe, hits, counters, listEditor, rowClass, live, fmtBytes, quotas } from "../widgets.js"; import { names, matchEditor, describe, hits, counters, listEditor, rowClass, live, fmtBytes, quotas, usageBar, lookupNames, named, pct } from "../widgets.js";
const chains = [ const chains = [
["input", "Input", "Traffic to the firewall itself."], ["input", "Input", "Traffic to the firewall itself."],
@@ -158,46 +158,86 @@ function extras(r, q) {
return out; return out;
} }
// quotaLine shows a quota, and what it used: in total, or per address // quotaLine sums up a quota on the rule's row: what it used (in total,
// (the busiest, and the rest in a dialog). // or how many addresses and the busiest); the rest is in its dialog.
function quotaLine(r, q) { function quotaLine(r, q) {
const qt = r.quota; const qt = r.quota;
const each = qt.per_source ? " each source" : qt.per_destination ? " each destination" : ""; const each = qt.per_source ? " each source" : qt.per_destination ? " each destination" : "";
const resets = qt.reset === "monthly" ? "monthly on day " + (qt.day || 1) : qt.reset || "by hand"; const resets = qt.reset === "monthly" ? "monthly" : qt.reset || "reset by hand";
let used = null; let used = fmtBytes(qt.bytes) + each;
if (q && !qt.per_source && !qt.per_destination) { if (q && !each) used = fmtBytes(q.used) + " of " + fmtBytes(qt.bytes) + " used (" + pct(q.used, qt.bytes) + ")";
used = [h("strong", null, fmtBytes(q.used)), " of " + fmtBytes(qt.bytes) + " used"]; else if (q) {
} else if (q) {
const n = (q.addresses || []).length; const n = (q.addresses || []).length;
used = [n === 1 ? "1 address" : n + " addresses", n ? [", busiest ", h("strong", null, fmtBytes(q.addresses[0].used))] : null, " ", used += " · " + (n === 1 ? "1 address" : n + " addresses") + (n ? ", busiest " + fmtBytes(q.addresses[0].used) + " (" + pct(q.addresses[0].used, qt.bytes) + ")" : "");
n ? h("button", { class: "small", onclick: () => usageDialog(r, q) }, "Usage…") : null];
} }
return h("div", { class: "muted small" }, (qt.over ? "quota over " : "quota ") + fmtBytes(qt.bytes) + each + ", resets " + resets, const text = "quota " + used + " · " + resets;
used ? [" · ", used] : null, return h("div", { class: "muted small" }, q && r.pending !== "new"
q && q.next_reset ? " · next reset " + new Date(q.next_reset).toLocaleString() : null, ? h("button", { class: "link small", title: "Usage, reset and give back", onclick: () => quotaDialog(r, q) }, text) : text);
can("config", true) && q && r.pending !== "new" ? [" ", h("button", { class: "small", onclick: () => resetQuota(r) }, "Reset")] : null);
} }
async function usageDialog(r, q) { // quotaDialog shows a quota in full: what it used (each address's, for a
await modal("Quota usage of " + (r.comment || "rule #" + r.id), h("div", null, // per-address one), its resets, and resetting or giving back.
h("p", { class: "muted" }, "Each " + (r.quota.per_source ? "source" : "destination") + " has " + fmtBytes(r.quota.bytes) + "."), async function quotaDialog(r, q) {
table([ const qt = r.quota;
{ label: "Address", render: (a) => h("span", { class: "mono" }, a.address) }, const perAddress = qt.per_source || qt.per_destination;
const write = can("config", true);
const resets = qt.reset === "monthly" ? "monthly on day " + (qt.day || 1) : qt.reset || "only by hand";
const when = (t) => t ? new Date(t).toLocaleString() : "–";
const names = perAddress ? await lookupNames((q.addresses || []).map((a) => a.address)) : {};
// A row's action closes this dialog first: it opens one of its own.
const then = (f) => (e) => { e.target.closest(".modal").querySelector(".buttons button:last-child").click(); f(); }; // Close
const v = await modal("Quota of " + (r.comment || "rule #" + r.id), h("div", null,
h("p", null, fmtBytes(qt.bytes) + (perAddress ? (qt.per_source ? " for each source" : " for each destination") : " in all") + "; the rule matches " +
(qt.over ? "once it is used up" : "while it lasts") + ". Resets " + resets + "."),
h("p", { class: "muted small" }, "Last reset " + when(q.last_reset) + " · next reset " + when(q.next_reset)),
perAddress ? table([
{ label: "Address", render: (a) => h("span", { class: "mono" }, named(names, a.address)) },
{ label: "Used", render: (a) => fmtBytes(a.used) }, { label: "Used", render: (a) => fmtBytes(a.used) },
{ label: "", render: (a) => h("div", { class: "usage" }, h("div", { class: "usage-bar" + (a.used >= r.quota.bytes ? " full" : "") }, bar(a.used / r.quota.bytes))) }, { label: "", class: "num", tdClass: "num", render: (a) => pct(a.used, qt.bytes) },
], q.addresses)), [{ label: "Close", value: null }], { wide: true }); { label: "", render: (a) => usageBar(a.used, qt.bytes) },
write ? { label: "", render: (a) => h("span", { class: "nowrap" },
h("button", { class: "small", onclick: then(() => resetQuota(r, a.address)) }, "Reset"), " ",
h("button", { class: "small", onclick: then(() => giveBack(r, a.address)) }, "Give back…")) } : null,
].filter(Boolean), q.addresses || [], { empty: "No address has used it since the last reset." })
: h("div", null, h("strong", null, fmtBytes(q.used)), " of " + fmtBytes(qt.bytes) + " used (" + pct(q.used, qt.bytes) + ")", usageBar(q.used, qt.bytes))),
[
...(write ? [{ label: perAddress ? "Reset all" : "Reset", value: "reset" }, { label: perAddress ? "Give back to all…" : "Give back…", value: "give" }] : []),
{ label: "Close", value: null },
], { wide: perAddress });
if (v === "reset") resetQuota(r);
if (v === "give") giveBack(r);
} }
function bar(f) { // giveBack takes an amount off what a quota used (one address's, or
const seg = h("div", { class: "seg leased" }); // everyone's): extra allowance until its next reset.
seg.style.width = Math.min(100, f * 100) + "%"; async function giveBack(r, address) {
return seg; const amount = input({ type: "number", min: 1, value: "1", class: "short" });
const unit = select([["mb", "MB"], ["gb", "GB"]], "gb", { class: "inline" });
const perAddress = r.quota.per_source || r.quota.per_destination;
const who = address ? " for " + address : perAddress ? " for every address" : "";
await modal("Give back quota", h("div", null,
h("p", null, "Take this much off what " + (r.comment || "rule #" + r.id) + "'s quota used" + who + ": extra allowance until its next reset, without changing the rule. This is done at once, not staged."),
field("Amount", h("div", null, amount, " ", unit))), [
{ label: "Cancel", value: null },
{ label: "Give back", kind: "primary", onclick: async () => {
const n = parseInt(amount.value, 10);
if (!(n > 0)) { toast("Give an amount of 1 or more", "err"); return false; }
const give = n * (unit.value === "gb" ? 1 << 30 : 1 << 20);
try {
await api("POST", "/rules/" + r.id + "/reset-quota", { body: address ? { give, address } : { give } });
toast("Gave back " + fmtBytes(give)); rerender();
} catch (e) { modal("Not given back", errorBox(e)); return false; }
} },
]);
} }
async function resetQuota(r) { async function resetQuota(r, address) {
if (!await confirmDialog("Reset the quota", "Start the quota of " + (r.comment || "rule #" + r.id) + " again now" + if (!await confirmDialog("Reset the quota", "Start the quota of " + (r.comment || "rule #" + r.id) + " again now" +
(r.quota.per_source || r.quota.per_destination ? ", for every address" : "") + "? This is done at once, not staged.", "Reset")) return; (address ? " for " + address : r.quota.per_source || r.quota.per_destination ? ", for every address" : "") + "? This is done at once, not staged.", "Reset")) return;
try { await api("POST", "/rules/" + r.id + "/reset-quota"); toast("Quota reset"); rerender(); } catch (e) { modal("Not reset", errorBox(e)); } try {
await api("POST", "/rules/" + r.id + "/reset-quota", address ? { body: { address } } : {});
toast("Quota reset"); rerender();
} catch (e) { modal("Not reset", errorBox(e)); }
} }
async function move(r, where) { async function move(r, where) {
+3 -1
View File
@@ -3,6 +3,7 @@
import { h, mount, fmtTime, errorBox } from "../ui.js"; import { h, mount, fmtTime, errorBox } from "../ui.js";
import { get } from "../api.js"; import { get } from "../api.js";
import { lineChart, chartCard, legend, timeTable, heatmap, heatmapTable, barList, compact, stepSeconds, endLabelWidth } from "../charts.js"; import { lineChart, chartCard, legend, timeTable, heatmap, heatmapTable, barList, compact, stepSeconds, endLabelWidth } from "../charts.js";
import { withNames } from "../widgets.js";
const ranges = [["1h", "1 hour"], ["6h", "6 hours"], ["24h", "24 hours"], ["7d", "7 days"], ["30d", "30 days"], ["90d", "90 days"], ["365d", "a year"]]; const ranges = [["1h", "1 hour"], ["6h", "6 hours"], ["24h", "24 hours"], ["7d", "7 days"], ["30d", "30 days"], ["90d", "90 days"], ["365d", "a year"]];
const rangeSecs = { "1h": 3600, "6h": 6 * 3600, "24h": 86400, "7d": 7 * 86400, "30d": 30 * 86400, "90d": 90 * 86400, "365d": 365 * 86400 }; const rangeSecs = { "1h": 3600, "6h": 6 * 3600, "24h": 86400, "7d": 7 * 86400, "30d": 30 * 86400, "90d": 90 * 86400, "365d": 365 * 86400 };
@@ -103,7 +104,7 @@ export async function render(el, params, query) {
listCard("What they tried", "the logged drops' protocol and port", barList(s.blocked_ports, { empty: "No logged drops " + rangeLabel + "." })), listCard("What they tried", "the logged drops' protocol and port", barList(s.blocked_ports, { empty: "No logged drops " + rangeLabel + "." })),
listCard("Devices, received", "bytes from external zones, by inside address (IPv6 per /64)", s.devices_down.length ? barList(s.devices_down, { format: bytes, href: connHref }) : devHint), listCard("Devices, received", "bytes from external zones, by inside address (IPv6 per /64)", s.devices_down.length ? barList(s.devices_down, { format: bytes, href: connHref }) : devHint),
listCard("Devices, sent", "bytes to external zones", s.devices_up.length ? barList(s.devices_up, { format: bytes, href: connHref }) : devHint.cloneNode(true)), listCard("Devices, sent", "bytes to external zones", s.devices_up.length ? barList(s.devices_up, { format: bytes, href: connHref }) : devHint.cloneNode(true)),
d.quotas.length ? listCard("Quotas", "bytes counted against each rule's quota, " + rangeLabel, barList(d.quotas, { format: bytes, label: (k) => "rule #" + k.slice(1), href: () => "#/rules" })) : null, d.quotas.length ? listCard("Quotas", "bytes counted against each rule's quota, " + rangeLabel, barList(d.quotas, { format: bytes, label: (k) => (d.hits.find((x) => x.key === k) || {}).name || "rule #" + k.slice(1), href: ruleHref, share: false })) : null,
listCard("Rules that matched nothing", "in the last 30 days (or since statistics started): candidates to remove", listCard("Rules that matched nothing", "in the last 30 days (or since statistics started): candidates to remove",
unused.length ? h("div", null, h("ul", { class: "plain-list" }, unused.slice(0, 12).map((e) => h("li", null, h("a", { href: ruleHref(e.key) }, e.name || e.key)))), unused.length ? h("div", null, h("ul", { class: "plain-list" }, unused.slice(0, 12).map((e) => h("li", null, h("a", { href: ruleHref(e.key) }, e.name || e.key)))),
unused.length > 12 ? h("div", { class: "muted small" }, "and " + (unused.length - 12) + " more: hc stats rules --unused") : null) : h("div", { class: "empty" }, "Every rule matched something.")))); unused.length > 12 ? h("div", { class: "muted small" }, "and " + (unused.length - 12) + " more: hc stats rules --unused") : null) : h("div", { class: "empty" }, "Every rule matched something."))));
@@ -142,6 +143,7 @@ async function loadAll(since, heatSince) {
get("/stats/timeline", { since, keys: "g:conntrack,g:conntrack_max,g:mem,g:rss,g:load,g:cpu" }), get("/stats/timeline", { since, keys: "g:conntrack,g:conntrack_max,g:mem,g:rss,g:load,g:cpu" }),
get("/stats/timeline", { since, keys: "quota:*" }), get("/stats/timeline", { since, keys: "quota:*" }),
]); ]);
await withNames(summary.devices_down, summary.devices_up, summary.blocked_sources);
return { summary, zones, traffic, drops, heat, hits, gauges, return { summary, zones, traffic, drops, heat, hits, gauges,
quotas: quotas.series.map((x) => ({ key: x.key.slice(6), count: x.values.reduce((a, v) => a + Math.max(v, 0), 0) })).filter((e) => e.count > 0).sort((a, b) => b.count - a.count) }; quotas: quotas.series.map((x) => ({ key: x.key.slice(6), count: x.values.reduce((a, v) => a + Math.max(v, 0), 0) })).filter((e) => e.count > 0).sort((a, b) => b.count - a.count) };
} }
+1
View File
@@ -85,6 +85,7 @@ h1 { font-size: 20px; margin: 0 0 4px; }
h2 { font-size: 16px; margin: 18px 0 8px; } h2 { font-size: 16px; margin: 18px 0 8px; }
h3 { font-size: 14px; margin: 14px 0 6px; } h3 { font-size: 14px; margin: 14px 0 6px; }
.muted { color: var(--muted); } .muted { color: var(--muted); }
.host-name { font-weight: 600; }
.small { font-size: 12px; } .small { font-size: 12px; }
.nowrap { white-space: nowrap; } .nowrap { white-space: nowrap; }
.warn-text { color: var(--warn); } .warn-text { color: var(--warn); }
+46 -6
View File
@@ -1,7 +1,7 @@
// Widgets shared by several pages: list editors with suggestions, the // Widgets shared by several pages: list editors with suggestions, the
// match editor of rules and NAT, descriptions, hit counts, durations. // match editor of rules and NAT, descriptions, hit counts, durations.
import { h, input, select, field, checkbox } from "./ui.js"; import { h, input, select, field, checkbox } from "./ui.js";
import { get } from "./api.js"; import { get, api } from "./api.js";
// ---- durations // ---- durations
@@ -98,11 +98,16 @@ export function matchEditor(m, n, opts = {}) {
// describe sums up a match: "in lan · from trusted · ssh, dns". // describe sums up a match: "in lan · from trusted · ssh, dns".
export function describe(m) { export function describe(m) {
const parts = []; const parts = [];
const side = (word, not, list) => { if (list && list.length) parts.push(word + (not ? " not " : " ") + list.join(", ")); }; const list = (not, xs) => (not ? "any but " : "") + xs.join(", ");
side("in", m.not_in, m.in_zones); // Zones are a direction, "lan → wan"; the firewall itself is the other
side("out", m.not_out, m.out_zones); // end of the input and output chains.
side("from", m.not_sources, m.sources); const ins = m.in_zones || [], outs = m.out_zones || [];
side("to", m.not_destinations, m.destinations); if (ins.length || outs.length) {
parts.push((m.chain === "output" ? "firewall" : ins.length ? list(m.not_in, ins) : "any") + " → " +
(m.chain === "input" ? "firewall" : outs.length ? list(m.not_out, outs) : "any"));
}
if (m.sources && m.sources.length) parts.push("from " + list(m.not_sources, m.sources));
if (m.destinations && m.destinations.length) parts.push("to " + list(m.not_destinations, m.destinations));
if (m.services && m.services.length) parts.push(m.services.join(", ")); if (m.services && m.services.length) parts.push(m.services.join(", "));
if (m.family) parts.push(m.family + " only"); if (m.family) parts.push(m.family + " only");
return parts.length ? parts.join(" · ") : "everything"; return parts.length ? parts.join(" · ") : "everything";
@@ -156,6 +161,41 @@ export function fmtBytes(n) {
return n + " bytes"; return n + " bytes";
} }
// pct writes used as a share of total: "25%", "<1%" for a little.
export function pct(used, total) {
if (!total || used <= 0) return "0%";
const p = used / total * 100;
return p < 1 ? "<1%" : Math.floor(p) + "%";
}
// usageBar draws used out of total as a bar, red once it is all used.
export function usageBar(used, total) {
const seg = h("div", { class: "seg leased" });
seg.style.width = Math.min(100, used / total * 100) + "%";
return h("div", { class: "usage" }, h("div", { class: "usage-bar" + (used >= total ? " full" : "") }, seg));
}
// lookupNames asks for inside addresses' names (docs/configuration.md):
// {address: name}, {} when names are off or can't be had.
export async function lookupNames(addrs) {
const list = [...new Set(addrs.filter(Boolean))];
if (!list.length) return {};
try { return (await api("POST", "/names", { body: { addresses: list } })).names || {}; } catch { return {}; }
}
// named shows an address (or text with it, such as address:port) after
// its name's first label; the whole name is in the tooltip.
export function named(names, addr, text = addr) {
const n = names[addr];
return n ? h("span", { title: n + " (" + addr + ")" }, h("span", { class: "host-name" }, n.split(".")[0]), " ", h("span", { class: "muted" }, text)) : text;
}
// withNames sets each top-list entry's name from its key (an address).
export async function withNames(...lists) {
const names = await lookupNames(lists.flat().map((e) => e.key));
for (const e of lists.flat()) if (names[e.key]) e.name = names[e.key].split(".")[0];
}
// quotas loads what the rules' quotas used, by rule ID ({} without // quotas loads what the rules' quotas used, by rule ID ({} without
// log:read). // log:read).
export async function quotas() { export async function quotas() {
+29 -3
View File
@@ -55,6 +55,9 @@ docker run -d --name ${P}-router --hostname router --cap-add NET_ADMIN $sysctls
--network ${P}-wan --ip $WAN4 --ip6 $WAN6 -v "$WORK:/w" "$IMAGE" sleep infinity >/dev/null || exit 1 --network ${P}-wan --ip $WAN4 --ip6 $WAN6 -v "$WORK:/w" "$IMAGE" sleep infinity >/dev/null || exit 1
docker network connect --ip $LAN4 --ip6 $LAN6 ${P}-lan ${P}-router || exit 1 docker network connect --ip $LAN4 --ip6 $LAN6 ${P}-lan ${P}-router || exit 1
docker network connect --ip 10.88.50.1 ${P}-wg ${P}-router || exit 1 docker network connect --ip 10.88.50.1 ${P}-wg ${P}-router || exit 1
# The client has a name for housecarld's reverse lookups (/etc/hosts
# here, a PTR record in real life); the addresses outside get none.
docker exec ${P}-router sh -c "echo '$CLIENT4 e2e-client.lab' >> /etc/hosts" || exit 1
# The router's interfaces get the names of the real one: wan, enp2s0, wg0 # The router's interfaces get the names of the real one: wan, enp2s0, wg0
# (attached in that order: eth0, eth1, eth2). # (attached in that order: eth0, eth1, eth2).
docker exec ${P}-router sh -c 'for p in eth0:wan eth1:enp2s0 eth2:wg0; do i=${p%:*} n=${p#*:}; ip link set $i down && ip link set $i name $n && ip link set $n up; done docker exec ${P}-router sh -c 'for p in eth0:wan eth1:enp2s0 eth2:wg0; do i=${p%:*} n=${p#*:}; ip link set $i down && ip link set $i name $n && ip link set $n up; done
@@ -319,15 +322,35 @@ send 2000000 >/dev/null 2>&1
run "used up: the device is cut off" 1 send 1000 run "used up: the device is cut off" 1 send 1000
run "the usage" 0 H quotas run "the usage" 0 H quotas
contains "e2e quota.*1.0 MB each.*by hand" contains "e2e quota.*1.0 MB each.*by hand"
contains "$CLIENT4 *1\.[0-9] MB" contains "$CLIENT4 (e2e-client) *1\.[0-9] MB (1[0-9][0-9]%)"
stop; start stop; start
run "after a restart, still used up" 1 send 1000 run "after a restart, still used up" 1 send 1000
run "the usage after a restart" 0 H quotas run "the usage after a restart" 0 H quotas
contains "$CLIENT4 *1\.[0-9] MB" contains "$CLIENT4 (e2e-client) *1\.[0-9] MB"
# A top-up: some given back (more than the cut-off packets added), then
# used up again; one address started again; then the whole quota.
run "a top-up for the device" 0 H rule reset-quota "$QR" --give 10mbytes --address $CLIENT4
contains "gave back 10.0 MB"
run "topped up: the device is back" 0 send 1000
send 30000000 >/dev/null 2>&1
run "used up again" 1 send 1000
run "an address that used nothing" 4 H rule reset-quota "$QR" --address 10.88.10.99
run "one address started again" 0 H rule reset-quota "$QR" --address $CLIENT4
run "back again" 0 send 1000
run "reset by hand" 0 H rule reset-quota "$QR" run "reset by hand" 0 H rule reset-quota "$QR"
run "the device is back" 0 send 1000 run "the device is back" 0 send 1000
run "the reset is audited" 0 H audit --action quota. run "the reset is audited" 0 H audit --action quota.
contains "quota.reset" contains "quota.reset"
contains "quota.give"
# Counted in total now: what the device used carries over; per
# destination can't take it, and says so.
run "within the quota" 0 send 300000
run "to one total" 0 H rule update "$QR" --quota 1mbytes,over
not_contains "quota_restarts"
run "the total carried over" 0 H quotas
contains "e2e quota) *[0-9.]* KB ([0-9]*%)"
run "per destination starts again" 0 H rule update "$QR" --quota 1mbytes,over,per-destination
contains "quota_restarts"
run "the quota rule goes" 0 H rule delete -y "$QR" run "the quota rule goes" 0 H rule delete -y "$QR"
# ---- the zone matrix, of the imported ruleset # ---- the zone matrix, of the imported ruleset
@@ -362,7 +385,8 @@ contains "Dropped: *[1-9]"
run "traffic per zone over time" 0 H stats traffic --since 1h --step 1m run "traffic per zone over time" 0 H stats traffic --since 1h --step 1m
contains "wan ↓" contains "wan ↓"
run "the devices" 0 H stats top devices --since 1h run "the devices" 0 H stats top devices --since 1h
contains "^$CLIENT4 *[0-9.]* KB *[0-9.]* KB" contains "^$CLIENT4 (e2e-client) *[0-9.]* KB *[0-9.]* KB"
run "each device once" 0 sh -c "docker exec ${P}-router /w/hc stats top devices --since 1h | grep -c '^$CLIENT4 ' | grep -qx 1"
run "the sources blocked" 0 H stats top blocked --since 1h run "the sources blocked" 0 H stats top blocked --since 1h
contains "^$NET4 *[1-9]" contains "^$NET4 *[1-9]"
run "what they tried" 0 H stats top tried --since 1h run "what they tried" 0 H stats top tried --since 1h
@@ -379,6 +403,8 @@ contains "^[0-9-]* [0-9:]* *[0-9]* *[0-9]*"
docker exec -d ${P}-client sh -c "sleep 30 | nc $NET4 9000" docker exec -d ${P}-client sh -c "sleep 30 | nc $NET4 9000"
sleep 1 sleep 1
run "the connection is tracked" 0 H connections --search "$CLIENT4 :9000" run "the connection is tracked" 0 H connections --search "$CLIENT4 :9000"
contains "$CLIENT4:[0-9]* (e2e-client) *$NET4:9000 *as $WAN4"
run "without names" 0 H connections --search "$CLIENT4 :9000" --no-names
contains "$CLIENT4:[0-9]* *$NET4:9000 *as $WAN4" contains "$CLIENT4:[0-9]* *$NET4:9000 *as $WAN4"
run "closed" 0 H connections close --search "$CLIENT4 :9000" -y run "closed" 0 H connections close --search "$CLIENT4 :9000" -y
contains "closed [1-9][0-9]* connection" contains "closed [1-9][0-9]* connection"