Release 0.2.0-rc.3+git62-0d16358

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

CHANGELOG.md

@@ -115,6 +115,15 @@ All notable changes to Housecarl are documented here. The format follows
   ruleset, for entries made before this version too: the configuration
   before and after it is got back from the current one and rendered by
   this version.
+- Negating a match is said first: the zone, From and To fields (rules
+  and NAT) start with "any of" / "any but" instead of a checkbox under
+  the list, shown once the list has an entry, and an "any but" list is
+  marked with a red bar. An empty list is never saved as negated (it
+  matched everything anyway, but would have negated what was added to
+  it later).
+- docs/rules.md: "Matching everything but a few", on leaving some
+  addresses out with a zone and "any but", and what to do when the range
+  isn't a zone.
 - The rule dialog shows only the zone fields its chain can use: no
   "Going out on" for input and early, no "Coming in on" for output
   (they could only be refused). A zone left in a field the chain hides
@@ -133,6 +142,16 @@ All notable changes to Housecarl are documented here. The format follows
 
 ### Fixed
 
+- A history entry's ruleset diff showed the object it changed as moved:
+  stepping back put the old form at the end of its list, so updating
+  one port forward (turning hairpin off) also showed its forward and
+  DNAT lines moving down. The diff is now only what the change did, and
+  its sides are named `before N` and `after N` instead of `now`.
+- Ruleset diffs numbered the hunks after one with a moved line too early
+  on the new side (`@@ -76,9 +74,9 @@` for `+76,9`), so the chain named
+  after `@@` could be the one before (`chain output` for a hunk in
+  `chain dstnat`). go-udiff miscounts them; Housecarl now numbers them
+  itself.
 - The dashboard's bar for a per-address quota showed only the busiest
   address, so one address that used up its allowance filled it (100%).
   The bar is now all the addresses together, and the dashboard and the

README.md

@@ -128,7 +128,7 @@ The web UI is on `http://127.0.0.1:8070/` (see `listen` in
 
 ## Status
 
-Housecarl 0.2.0 is in testing (0.2.0-rc.3): zones, address sets and
+Housecarl 0.2.0 is in testing (0.2.0-rc.3+git62-0d16358): zones, address sets and
 services with ordered rules, IPv4 and IPv6, port forwards with hairpin NAT
 and IPv6 pinholes, source NAT and redirects, built-ins, probation for
 network changes (confirmed by the client that made them), the ruleset

VERSION

@@ -1 +1 @@
-0.2.0-rc.3
+0.2.0-rc.3+git62-0d16358

docs/development.md

@@ -135,6 +135,14 @@ As for Jarl and Steward:
 3. Run `make test && make e2e`.
 4. Commit as "Release X", tag it `git tag -a vX -m "Housecarl X"`, and push.
 
+A fix to a release candidate may go out as a build of it rather than a
+new candidate: `0.2.0-rc.3+git<count>-<commit>`, where `<commit>` is the
+short ID of the last commit before "Release X" and `<count>` is `git
+rev-list --count` of it. The count orders the builds (apt compares it as a
+number; the ID alone would order them at random); each sorts after
+`0.2.0-rc.3` and before `0.2.0-rc.4`. Debian writes it
+`0.2.0~rc.3+git<count>~<commit>`.
+
 The tag starts `.gitea/workflows/release.yml`, which checks the tag
 against `VERSION` and the CHANGELOG, runs the tests, builds the tarballs,
 the CLI for each OS and the `.deb`s, and publishes them

docs/rules.md

@@ -165,6 +165,40 @@ hc rule add forward --in wan --status dnat
 hc rule add forward --in wan -a drop
 ```
 
+### Matching everything but a few
+
+A field matches **any of** its entries, or with `--not-…` ("any but" in
+the web UI) anything else; it can't do both. "The LAN except the NAS" is
+still one rule: the zone says which network, and the addresses say which
+of it to leave out, because zones and addresses are different fields and
+a rule needs all of them to match.
+
+| What you want | One rule |
+|---|---|
+| the LAN, except the NAS | `--in lan --from nas --not-from` |
+| everyone on guest, except one device, has a quota | `--in guest --from tv --not-from --quota 10gbytes,over,daily` |
+| the internet, except one host | `--out wan --to blocked --not-to` |
+
+```sh
+# Each device on the LAN: 50 GB a month, but not the NAS (its backups)
+hc rule add forward --in lan --out wan --from nas --not-from --quota 50gbytes,over,per-source,monthly -a drop
+```
+
+Leaving part of an address range out ("10.0.0.0/8 except 10.0.5.0/24")
+when it isn't a zone takes one of:
+
+* **Two rules, the exception first**, when the exception gets the
+  opposite action: `--from 10.0.5.0/24 -a drop` above `--from
+  10.0.0.0/8`. There is no action that skips a rule and goes on, so this
+  doesn't work when the exception should just fall through to later
+  rules.
+* **Ranges around the hole**: `--from 10.0.0.0-10.0.4.255 --from
+  10.0.6.0-10.255.255.255`.
+* **A [raw rule](#raw-rules)**: `ip saddr 10.0.0.0/8 ip saddr != 10.0.5.0/24 accept`.
+
+There's no need to leave a zone out of other zones: an interface belongs
+to one zone, so "any of lan, guest" already leaves out everything else.
+
 ### Bandwidth and quotas
 
 A **byte rate** limits bandwidth: `--limit 20mbytes/second,over,per-source

internal/engine/engine.go

@@ -335,6 +335,8 @@ func Rewind(st *model.State, h *model.HistoryEntry) error {
 			return err
 		}
 	}
+	// put appends what it brings back; its position puts it in place.
+	st.Sort()
 	return nil
 }
 

internal/server/diff_test.go

@@ -1,6 +1,8 @@
 package server
 
 import (
+	"encoding/json"
+	"fmt"
 	"strings"
 	"testing"
 
@@ -70,3 +72,68 @@ func TestHistoryDiff(t *testing.T) {
 		t.Errorf("unknown entry: %d", code)
 	}
 }
+
+// TestHunkStarts: after a hunk with unchanged lines between its edits (a
+// moved line), the later hunks' new sides start where they do.
+func TestHunkStarts(t *testing.T) {
+	var a, b strings.Builder
+	for i := 1; i <= 20; i++ {
+		fmt.Fprintf(&a, "%d\n", i)
+		switch i {
+		case 4:
+		case 6:
+			b.WriteString("6\n4\n")
+		case 14:
+		default:
+			fmt.Fprintf(&b, "%d\n", i)
+		}
+	}
+	diff := hunkStarts(udiff.Unified("a", "b", a.String(), b.String()))
+	var heads []string
+	for _, l := range strings.Split(diff, "\n") {
+		if strings.HasPrefix(l, "@@") {
+			heads = append(heads, l)
+		}
+	}
+	// diff -u: the move nets nothing, the deleted 14 one line.
+	if want := []string{"@@ -1,9 +1,9 @@", "@@ -11,7 +11,6 @@"}; strings.Join(heads, "|") != strings.Join(want, "|") {
+		t.Fatalf("hunk headers %q, want %q\n%s", heads, want, diff)
+	}
+}
+
+// TestHistoryDiffUpdate: an update's diff is only what it changed; the
+// object stepped back to its old form stays where it was.
+func TestHistoryDiffUpdate(t *testing.T) {
+	e := newEnv(t)
+	e.call(e.token, "POST", "/zones", model.Zone{Name: "lan", Interfaces: []string{"eth1"}}, nil)
+	e.call(e.token, "POST", "/zones", model.Zone{Name: "wan", Interfaces: []string{"wan"}, External: true}, nil)
+	var ids []int64
+	for i, port := range []string{"8096", "443", "50300", "7777", "25"} {
+		var mr api.MutationResponse
+		e.call(e.token, "POST", "/forwards", map[string]any{"name": "f" + port, "ports": port, "to": fmt.Sprintf("10.0.10.%d", 30+i),
+			"public_addresses": []string{"203.0.113.1"}, "hairpin": true}, &mr)
+		var f model.PortForward
+		if err := json.Unmarshal(mr.Object, &f); err != nil {
+			t.Fatal(err)
+		}
+		ids = append(ids, f.ID)
+	}
+	var mr api.MutationResponse
+	if code := e.call(e.token, "PATCH", "/forwards/"+itoa(ids[2]), map[string]any{"hairpin": false}, &mr); code != 200 {
+		t.Fatalf("update: %d", code)
+	}
+	var out struct{ Diff string }
+	if code := e.call(e.token, "GET", "/history/"+itoa(mr.HistoryID)+"/diff", nil, &out); code != 200 {
+		t.Fatalf("diff: %d", code)
+	}
+	var changed []string
+	for _, l := range strings.Split(out.Diff, "\n") {
+		if (strings.HasPrefix(l, "+") || strings.HasPrefix(l, "-")) && !strings.HasPrefix(l, "+++") && !strings.HasPrefix(l, "---") {
+			changed = append(changed, l)
+		}
+	}
+	if len(changed) != 1 || !strings.HasPrefix(changed[0], "-") || !strings.Contains(changed[0], "-hairpin") ||
+		!strings.Contains(out.Diff, "(before "+itoa(mr.HistoryID)+")") || !strings.Contains(out.Diff, "@@ chain srcnat") {
+		t.Errorf("entry %d's diff, want only the hairpin line gone, in chain srcnat:\n%s", mr.HistoryID, out.Diff)
+	}
+}

internal/server/h_config.go

@@ -3,6 +3,7 @@ package server
 import (
 	"encoding/json"
 	"errors"
+	"fmt"
 	"net/http"
 	"net/netip"
 	"sort"
@@ -98,11 +99,71 @@ func (s *Server) respondChange(w http.ResponseWriter, r *http.Request, p *auth.P
 // rulesetDiff is the unified diff of the rulesets two configurations
 // make: the file housecarl-ruleset.service loads, now and after.
 func (s *Server) rulesetDiff(before, after *model.State) string {
+	return s.labeledDiff(before, after, "now", "after")
+}
+
+// labeledDiff is rulesetDiff with its own names for the two sides.
+func (s *Server) labeledDiff(before, after *model.State, from, to string) string {
 	if before == nil || after == nil {
 		return ""
 	}
 	now, then := s.fw.Render(before).Text, s.fw.Render(after).Text
-	return blockHeaders(udiff.Unified("ruleset.nft (now)", "ruleset.nft (after)", now, then), now, then)
+	diff := hunkStarts(udiff.Unified("ruleset.nft ("+from+")", "ruleset.nft ("+to+")", now, then))
+	return blockHeaders(diff, now, then)
+}
+
+// hunkStarts numbers each hunk's new side again, from its old side and
+// the lines the hunks before it added and removed. go-udiff (v0.4.1)
+// leaves out of its count the unchanged lines between two edits of one
+// hunk, so after a hunk like a moved line every later one started too
+// early ("@@ -76,9 +74,9 @@" for "+76,9"), and blockHeaders named the
+// chain before it.
+func hunkStarts(diff string) string {
+	lines := strings.SplitAfter(diff, "\n")
+	delta := 0
+	for i := 0; i < len(lines); i++ {
+		f := strings.Fields(lines[i])
+		if !strings.HasPrefix(lines[i], "@@ ") || len(f) < 4 || f[3] != "@@" {
+			continue
+		}
+		start, _, _ := strings.Cut(f[1][1:], ",")
+		from, err := strconv.Atoi(start)
+		if err != nil {
+			continue
+		}
+		oldN, newN := 0, 0
+		for _, l := range lines[i+1:] {
+			if strings.HasPrefix(l, "@@ ") {
+				break
+			}
+			switch {
+			case strings.HasPrefix(l, "-"):
+				oldN++
+			case strings.HasPrefix(l, "+"):
+				newN++
+			case strings.HasPrefix(l, " "):
+				oldN++
+				newN++
+			}
+		}
+		// An empty side is numbered by the line before it.
+		first := from
+		if oldN == 0 {
+			first++
+		}
+		to := first + delta
+		if newN == 0 {
+			to--
+		}
+		side := "+" + strconv.Itoa(to)
+		if newN > 1 {
+			side += "," + strconv.Itoa(newN)
+		}
+		f[2] = side
+		lines[i] = strings.Join(f, " ") + "\n"
+		delta += newN - oldN
+	}
+	return strings.Join(lines, "")
 }
 
 // blockHeaders names, after each hunk's @@ line, the chain (or other
@@ -705,7 +766,8 @@ func (s *Server) historyDiff(w http.ResponseWriter, r *http.Request, p *auth.Pri
 			return err
 		}
 	}
-	writeJSON(w, http.StatusOK, map[string]string{"diff": s.rulesetDiff(st, after)})
+	before, done := fmt.Sprintf("before %d", id), fmt.Sprintf("after %d", id)
+	writeJSON(w, http.StatusOK, map[string]string{"diff": s.labeledDiff(st, after, before, done)})
 	return nil
 }
 

internal/web/static/assets/style.css

@@ -379,6 +379,10 @@ pre.wrap { white-space: pre-wrap; word-break: break-all; font-size: 12px; }
 .list-rows { display: flex; flex-direction: column; gap: 6px; margin-bottom: 6px; }
 .list-row { display: flex; gap: 6px; align-items: center; }
 .list-row input { width: 26em; max-width: 100%; }
+.negatable { display: flex; gap: 8px; align-items: flex-start; flex-wrap: wrap; }
+.negatable > .list-editor { flex: 1; min-width: 0; }
+.negatable.negated > select { color: var(--err); border-color: var(--err); font-weight: 600; }
+.negatable.negated .list-rows { border-left: 3px solid var(--err); padding-left: 8px; }
 input.short { width: 7em; }
 .builtins { margin-bottom: 10px; line-height: 1.6; }
 .policy { margin-top: 10px; }

internal/web/static/assets/widgets.js

@@ -1,6 +1,6 @@
 // Widgets shared by several pages: list editors with suggestions, the
 // match editor of rules and NAT, descriptions, hit counts, durations.
-import { h, input, select, field, checkbox } from "./ui.js";
+import { h, input, select, field } from "./ui.js";
 import { get, api } from "./api.js";
 
 // ---- durations
@@ -65,34 +65,54 @@ export function listEditor(values, opts = {}) {
 
 // ---- matches
 
+// negatable puts "any of" / "any but" in front of a list editor, so a
+// field says first whether it matches its entries or everything else;
+// "any but" marks the list. The choice shows once the list has a row
+// (empty is "any"), and an empty list is never negated. value() is the
+// list, negated() the choice.
+function negatable(list, not) {
+  const mode = select([["", "any of"], ["not", "any but"]], not ? "not" : "", { class: "inline" });
+  const box = h("div", { class: "negatable" }, mode, list);
+  const rows = () => list.querySelector(".list-rows").children.length;
+  const update = () => {
+    mode.classList.toggle("hidden", rows() === 0);
+    box.classList.toggle("negated", mode.value === "not" && rows() > 0);
+  };
+  mode.addEventListener("change", update);
+  // Add and remove act on their own click first; this runs after.
+  list.addEventListener("click", update);
+  update();
+  box.value = list.value;
+  box.negated = () => mode.value === "not" && list.value().length > 0;
+  return box;
+}
+
 // matchEditor edits what a rule, source NAT or redirect matches. It
 // returns the form-grid fields; value() is the match's fields.
 export function matchEditor(m, n, opts = {}) {
   m = m || {};
   const zones = (list) => listEditor(list, { suggestions: n.zones, placeholder: "zone", mono: false, addLabel: "Add zone" });
   const addrs = (list) => listEditor(list, { suggestions: n.sets, placeholder: "a set, 192.168.1.0/24, 2001:db8:1::/64, 192.168.1.100-192.168.1.199", addLabel: "Add" });
-  const inZ = zones(m.in_zones), outZ = zones(m.out_zones);
-  const notIn = checkbox("any zone but these", m.not_in), notOut = checkbox("any zone but these", m.not_out);
-  const src = addrs(m.sources), dst = addrs(m.destinations);
-  const notSrc = checkbox("any source but these", m.not_sources), notDst = checkbox("any destination but these", m.not_destinations);
+  const inZ = negatable(zones(m.in_zones), m.not_in), outZ = negatable(zones(m.out_zones), m.not_out);
+  const src = negatable(addrs(m.sources), m.not_sources), dst = negatable(addrs(m.destinations), m.not_destinations);
   const svc = listEditor(m.services, { suggestions: n.services, placeholder: "ssh, https, tcp/8080, udp/1000-2000, icmp/echo-request", addLabel: "Add service" });
   const fam = select([["", "IPv4 and IPv6"], ["ipv4", "IPv4 only"], ["ipv6", "IPv6 only"]], m.family || "");
-  const inField = opts.in !== false ? field("Coming in on", h("div", null, inZ, notIn)) : null;
-  const outField = opts.out !== false ? field("Going out on", h("div", null, outZ, notOut)) : null;
+  const inField = opts.in !== false ? field("Coming in on", inZ) : null;
+  const outField = opts.out !== false ? field("Going out on", outZ) : null;
   const fields = [
     inField,
     outField,
-    field("From", h("div", null, src, notSrc), "Sets, addresses, prefixes or ranges, IPv4 and IPv6 mixed. Empty: any."),
-    field("To", h("div", null, dst, notDst)),
+    field("From", src, "Sets, addresses, prefixes or ranges, IPv4 and IPv6 mixed. Empty: any."),
+    field("To", dst),
     field("Services", svc, "Named services (built in, or yours) or entries such as tcp/8080. Empty: any traffic."),
     field("Family", fam),
   ];
   fields.value = () => {
-    const v = { sources: src.value(), not_sources: notSrc.input.checked, destinations: dst.value(), not_destinations: notDst.input.checked,
+    const v = { sources: src.value(), not_sources: src.negated(), destinations: dst.value(), not_destinations: dst.negated(),
       services: svc.value(), family: fam.value };
     // A hidden zone field (the chain has no such interface) matches nothing.
-    if (inField) Object.assign(v, inField[1].classList.contains("hidden") ? { in_zones: [], not_in: false } : { in_zones: inZ.value(), not_in: notIn.input.checked });
-    if (outField) Object.assign(v, outField[1].classList.contains("hidden") ? { out_zones: [], not_out: false } : { out_zones: outZ.value(), not_out: notOut.input.checked });
+    if (inField) Object.assign(v, inField[1].classList.contains("hidden") ? { in_zones: [], not_in: false } : { in_zones: inZ.value(), not_in: inZ.negated() });
+    if (outField) Object.assign(v, outField[1].classList.contains("hidden") ? { out_zones: [], not_out: false } : { out_zones: outZ.value(), not_out: outZ.negated() });
     return v;
   };
   // zones shows the zone fields a chain can use: input and early have no

scripts/gitea-release.sh

@@ -114,7 +114,7 @@ done
 files="$files dist/SHA256SUMS"
 
 # The release (created, or found when publishing again).
-release=$(call 404 "$api/releases/tags/$tag")
+release=$(call 404 "$api/releases/tags/$(printf '%s' "$tag" | sed 's/+/%2B/g')")
 id=$(printf '%s\n' "$release" | jq -r '.id // empty')
 if [ -z "$id" ]; then
 	body=$(jq -n --arg tag "$tag" --arg notes "$notes" --argjson pre "$prerelease" \
@@ -127,7 +127,8 @@ for f in $files; do
 	name=${f#dist/}
 	old=$(printf '%s\n' "$release" | jq -r --arg n "$name" '.assets[]? | select(.name == $n) | .id')
 	[ -z "$old" ] || call "" -X DELETE "$api/releases/$id/assets/$old" >/dev/null
-	call "" -X POST -F "attachment=@$f" "$api/releases/$id/assets?name=$name" >/dev/null
+	# A + (build metadata, 0.2.0-rc.3+1a2b3c4) would be a space in a query.
+	call "" -X POST -F "attachment=@$f" "$api/releases/$id/assets?name=$(printf '%s' "$name" | sed 's/+/%2B/g')" >/dev/null
 	echo "gitea-release: attached $name"
 done