For the technical audience

Decisions ADR-151 to ADR-172

Every architecture decision record (ADR) behind WrenDNS: what we knew, what we chose, and what it costs. This page is built from the specification on every change, so it's always the current record.

ADR-151 Failover drills: a maintenance window with a drop, a federated coverage reportProposedCLUFLTOBSOPSCLU-014 (owner request).

Context

CLU-014 (owner request). Routers hand devices two DNS servers (for example a Pi and another resolver at 10.50.5.112), but a stub resolver uses its first server until it stops answering, so on a healthy network nothing shows whether the second one works for every device; a passive "single-homed" finding would be guesswork. Design: docs/design/failover-drills.md.

Decision

  • A drill is a maintenance window with a drop (reuses OPS-010/ADR-118: readiness off, alerts muted, out of elections and the health level, the window file, forwarding, handover on the primary). For its targets (addresses or names, a group, or everyone) the node discards UDP queries after client identification and before rate limiting, and closes TCP, DoT, DoH and DoQ connections without an answer: the nearest thing to a dead node. Each counts as a new query status drilled.
  • Query path: one relaxed load of the drill's end time (0: none); only during a drill, one ArcSwap load of the target set and a lookup. Never dropped: [clients] infrastructure, loopback, the node's own and other cluster nodes' addresses, and excluded devices (OBS-022).
  • Bounds: 15 to 300 s, default 60; the cap is not configurable (a long drill is an outage). Ends on expiry or DELETE. A restart cancels a drill (unlike maintenance, which resumes).
  • Preconditions (409 drill_refused with the reason): another node ready and answering in the last 5 minutes; no maintenance on the node and no drill anywhere in the cluster in the last 10 minutes; no pinned or baking rollout (CLU-013); health not severe; targets that resolve to a device seen on the node in the last hour (unless everyone). Not a configuration change: allowed under GitOps authority, audited, operator role; agents need the scope ops:drill (not in the defaults) and a reason.
  • Coverage: a per-client seen table on the telemetry thread (baseline, affinity; 4,096 cap, persisted); a per-client window map on every node while any node drills (learned from the heartbeat's new drill_id), frozen 60 s after the end; the drilled node merges the parts (federated read, retried 5 minutes) into a saved report (last 50 kept) with verdicts followed (answered elsewhere after the first drop, by 5 s after the end) / stuck (≥ 3 dropped, no such answer) / quiet / elsewhere, time to failover, and returned vs sticky after 15 minutes.
  • Interfaces: POST/DELETE/GET /api/v1/nodes/{id}/drill (dryRun), GET /api/v1/drills[/{id}], GET /api/v1/clients/affinity; MCP plan_failover_drill, drill_report; the Cluster page's Drill… and Drills list; metrics; annotations drill.started/drill.ended. Never a health reason.

Consequences

  • The first real drill should be on one laptop for 30 s, then a group, then everyone; never run against a live node without the operator's go.
  • Scheduled drills and a REFUSED mode were open questions; the owner said yes to both (below), and both are built.
  • Devices that never queried during the window are quiet: a drill can't tell an idle device from one without DNS.

Amendments at implementation (T15.2, 2026-10-19):

  • Closing connections: a new Response::Close in wrendns-net. TCP and DoT close the connection; UDP drops; DoH and DoQ answer nothing but keep the connection, because HTTP/2 and QUIC multiplex other queries on it.
  • "Loopback" is the node itself: 127.0.0.1 and ::1 are never dropped, nor held by FLT-017's analytics hold. Other 127.0.0.0/8 addresses are ordinary clients: the cluster e2e's stub resolvers bind them. Masking's infrastructure list (which has all of 127/8) is used without its loopback entries.
  • Who's in the report: the targets seen on the node in the hour before (the baseline's targets), any device the node dropped, and any device another node answered during the window. A non-target the drilled node answered as usual is left out. A target with 1 or 2 unanswered queries and nothing elsewhere is quiet: too little to tell.
  • The window starts at once: the drilled node calls every peer (drill peer call, 1 s deadline) before it drops anything, and the heartbeat's drill_id is the fallback. A new heartbeat flag drills tells an older peer (missing) from one that recorded nothing.
  • Returned: checked from the drilled node's seen table (to the second) 15 minutes after the end.
  • Seen table size: each entry keeps 7 daily counts for the affinity view, about 70 bytes rather than the design's 40 (under 300 KB at the 4,096 cap).
  • Markers: drill.started and drill.ended are new marker kinds; a drill's maintenance window leaves no maintenance.* markers of its own.
  • Test hooks: WRENDNS_DRILL_GAP_SECS, WRENDNS_DRILL_RETURN_SECS, and WRENDNS_DRILL_RETRY_SECS shorten the 10-minute gap, the 15-minute check, and the 5-minute retry for the cluster e2e. WRENDNS_DRILL_IGNORE_HEALTH passes the health check (shown as ignored in the checks): the e2e's nodes have no working upstream, so their health is severe.
  • CLI: wrendns ctl drill start|stop|report over the API (the design's local wrendns drill subcommands are the same calls; only ctl is built).

Owner decisions (2026-10-19, the open questions):

  • Scheduled drills: yes. [drill] schedule = "monthly" with nodes (the nodes that drill themselves), the targets (group or clients, else everyone), for_secs (default 30), hour (default 3) and tz, and mode. The day is in the first week, picked from the node's label and the month (FNV-1a), so it moves from month to month; the node tries once a minute through the hour, skips the month if a precondition fails all hour, and never drills within 7 days of another drill. The report goes out through a new one-off alert kind drill_report (any drill, scheduled or not, reports ended in the last 2 hours) to whatever destinations the rule names, email included. Shared configuration, so nodes says where it runs.
  • REFUSED mode: yes. mode: "refused" (API, form, MCP, ctl --refused, [drill] mode): the node answers REFUSED with extended error 14 (Not Ready, "failover drill") at once, still counted as drilled. Drop stays the default; the preview says the REFUSED drill is faster but less realistic.
  • A live node: yes, laptop first for 30 s, then a group, then everyone.

After the first live drills (owner go):

  • What ran: the laptop was skipped: on Wi-Fi its only DNS server was another resolver (10.50.5.112), so the drilled node never saw it (only the agent's own test lookups). The IOT group for 30 s; then every device except one server (10.50.5.2), for 30 s. That server was left out because its only DNS server was the drilled node and its k3s CoreDNS forwards through it, so dropping it would cut external DNS for the host and its pods.
  • Two verdict rules were too loose, and are tightened (owner yes): an answer elsewhere counts as following only by 5 s after the end (FOLLOW_GRACE_MS): after that the drilled node answers again, so it proves nothing (the IOT drill had a "48.6 s failover" in a 30 s drill). And a target with nothing dropped is quiet, not followed, even when it asked another node (the second drill dropped nothing at all, yet said 5 followed and 100 % coverage). A device that asked the other node both before and after its first drop still follows, with no time. The windows still record 60 s after the end.
  • A drill on a resolver pod (owner go): the pod left the EndpointSlice about 14 s after the start (balancerSwitchMs 12,652; 3 of the first 13 probes on the pods' address timed out, none after). Two changes (owner yes): the chart's readiness probe is every 2 s with 2 failures (about 4 s, for maintenance and real failures too), and the primary relays: a node reaches only the peers it has a stream to (a resolver pod only the primary; the Pi no resolver pod), so the report had missed every node but the drilled one and the primary, on the Pi's drills too. A new peer call relay on the primary passes a window notice or a read (part, list, report, active, seen) on to its peers that the caller didn't reach, and answers with theirs; never a start or end, never a relay. A node that learns of a drill this way has no heartbeat from the drilled node, so its window runs to the drill's end (the cutoff comes from the drilled node's own end), and it holds the one-drill-at-a-time rule until then plus the gap. An older primary refuses the call: the report is as before.

Requirements CLU-013 CLU-014 FLT-017 OBS-022 OPS-010

Builds on ADR-118

ADR-152 Internal names: their own top list, not hiddenProposedFLTOBSOBS-037.

Context

OBS-037. Once the kubelet points every pod at WrenDNS (FLT-017), the dashboard's top lists fill with the cluster's own names: Services asked constantly, and a pod's ndots: 5 search list turns one lookup of api.github.com into three NXDOMAINs under cluster.local first. Reverse lookups of private addresses are the same kind of noise on any network. The only way to hide names was [exclusions] (OBS-022), which also drops them from the query log and exports, and so from the workload view's point. The owner chose a separate list, with private reverse lookups included (2026-10-19).

Decision

  • An internal name is a reverse name in the RFC 6303 private and local ranges (the same as [special] private_ptr_nxdomain), a name at or under cluster.local when [kubernetes] enabled, or one under a suffix in [telemetry] internal_names (at most 100; node-local like the rest of [telemetry], so a replica one version behind never sees it).
  • The aggregator counts an internal name in the hour's internal Space-Saving list (and the group's) instead of domains and nxdomain. The blocked list, the bad-answer lists, a device's own top domains, the time series, the query log, the live view, exports, and /metrics are unchanged: this sorts, it hides nothing.
  • The check runs on the aggregator thread before privacy hashes the name (Aggregates::record_private): a few suffix lookups, no allocation. Top lists rebuilt from a privacy-hashed query log after a restart can't tell and count those names as domains.
  • GET /api/v1/stats/top?kind=internal (with group; not with client), MCP top_items kind: internal, hour_top kind internal. A peer one version behind refuses the kind and is listed in missingNodes.
  • The dashboard's Top domains card switches between Internet (the default) and Internal; the switch asks for one list at a time.

Consequences

The domains and NXDOMAIN lists change meaning for existing installs (less noise); history from before keeps both kinds in domains. A cluster domain other than cluster.local needs [telemetry] internal_names on each node: the chart doesn't write it, so a new chart still works with the previous image.

Requirements FLT-017 OBS-022 OBS-037

ADR-153 Stub zones: a recursive upstream with per-zone hints, a health check its servers answerAcceptedDNSDNS-022 (community request 2026-11-14).

Context

DNS-022 (community request 2026-11-14). Conditional forwarding (DNS-017) asks the configured servers with RD=1 and returns their reply as is. A zone's own authoritative-only servers (BIND with recursion no, NSD, Knot, PowerDNS, internal AD-integrated or departmental masters) answer only their zones: for a delegated sub-domain they return a referral, which reaches the client as an empty NOERROR, and they REFUSE the active health check's . NS, so the upstream is benched and upstream_down flaps while every real lookup works. BIND solves this with stub zones; Unbound with stub-zone. The iterative resolver (ADR-074) already follows referrals with bailiwick and glue rules but always starts at the root, and the spec's health_check_name (spec/04 §5) was never built.

Decision

  • A stub zone is a recursive:// upstream with [upstream.recursive] stub_zone and stub_servers (IP addresses, port 53). It sits in a group and is reached through a [[route]] like any upstream, so groups, strategies, breakers, views, the cache, and the API's forward model need nothing new.
  • The resolver's start() picks, in order: the deepest cached cut strictly below the stub zone (a delegation its servers gave), else the stub's servers, else the root. A cached cut at or above the stub zone is the outside (public) view and never wins: the configured servers are used whatever the public delegation says (split horizon). A DS for the stub apex still goes to the parent.
  • The servers are used as listed (BIND static-stub, Unbound's default stub-prime: no): the zone's NS set isn't fetched from them. Conservative: a NS set listing public or unreachable addresses can't take over, and nothing is learned that the operator didn't write. Priming the NS set (BIND type stub) is deferred until asked for.
  • Stub names never trigger root priming, so a network with no route to the root still answers them at once. Names an answer leads outside the zone (a CNAME target, an out-of-zone server name) are resolved from the root by the same resolver, as recursive:// does; the root never sees the stub zone's own names.
  • Health: Upstream::health_check() probes a stub upstream for liveness only: its servers are asked the zone's SOA directly and any reply except SERVFAIL counts as up, REFUSED included (servers that host only the zones below a stub apex, such as per-/24 reverse zones under a /16 stub, refuse the apex itself; a server that answers SERVFAIL for its own zone is failing, so it doesn't count). Upstream::stub_liveness() is the one rule: the draft-upstream check POST /checks/upstream applies it too. Other upstreams ask Upstream::health_question() (. NS). Client lookups keep their usual accounting: a failed iterative resolution is a failed exchange, so serve-stale still applies to it.
  • Managed forwards (ADR-040) take stub: true: the entry becomes one upstream forward:<domain>#stub (url recursive://, the stub fields) in the group forward:<domain>, with the same route and NTA as a forward. Servers must be IP addresses (udp:// + address is accepted); anything else is refused with a hint. GET /forwards reports stub and the addresses. MCP plan_forward_domain takes stub; the UI's "Send a domain to another server" has a checkbox, named by what it means ("these servers only know this domain").

Consequences

Authoritative-only servers become usable as targets and stop flapping, and delegated sub-domains resolve. Ports other than 53, TSIG, and NS priming aren't supported. A stub zone's out-of-zone CNAME targets are resolved from the root rather than through the default group, which bypasses an encrypted forwarder for those names (documented). Plain forwards still health-check with . NS; making their check ask the route's apex is a separate change. A node one version behind refuses a configuration with the new keys (deny_unknown_fields), as with any new key: in a cluster, upgrade replicas first.

Requirements DNS-017 DNS-022

Builds on ADR-040 ADR-074

ADR-154 A Git-managed node takes over from an emergency primary through the electionAcceptedGitHub issue #3 (v0.6.0; a Kubernetes controller that is Git-managed plus native replicas, automatic failover, config_authority = gitops).

Context

GitHub issue #3 (v0.6.0; a Kubernetes controller that is Git-managed plus native replicas, automatic failover, config_authority = gitops). While the controller is down a replica is elected and becomes an emergency primary (ADR-056, ADR-048): it coordinates on the last version and publishes nothing new. When the controller came back nothing moved the role: the takeover loop (ADR-048, review 05 q2) required promote_plan(..).is_ok(), and promote_plan refuses under automatic failover on purpose (ADR-056: a manual promote would bypass the votes). So the documented "takes over by itself after a minute" never happened with failover = auto, and a GitOps cluster stayed on an emergency primary that can't publish until someone ran a command that was also refused. The owner first accepted letting the takeover bypass that refusal (2026-11-14), then changed course the same day: a bare bypass leaves a primary that can't publish, because under auto a primary writes only while its election lease holds (Cluster::may_publish, ADR-056) and promote creates no election phase; the voters would also keep the lease they granted to the emergency primary.

Decision

under automatic failover the takeover is a handover plus an election, so ADR-056's rules (one vote per epoch, voter-granted leases, pre-vote, fencing) decide who is primary, as for any other failover.

  • The ask. A node that can stand (not the primary, eligible, not a witness, holding the cluster key) and is Git-managed (config_source = "gitops") sends the emergency primary the new elect.handover RPC once it has seen that primary up for the same 60 s as before. The asker is the stream's certificate, as for votes.
  • The answer. The emergency primary agrees only if automatic failover runs, the authority is gitops, it is an emergency primary, and the asker is in the signed registry as eligible and not a witness, is Git-managed (from its Hello, ADR-049), and is up. Otherwise it says why. Agreeing starts the ADR-118 handover without a maintenance window: it stops renewing its lease and keeps serving DNS, stays ready, and keeps its alerts; it follows the new primary after the step-down. The ADR-118 give-up applies: no new primary within 60 s and it renews again (handover_failed). The asker waits 120 s before asking again.
  • The hold-back. Under the gitops authority a node that isn't Git-managed doesn't stand while an eligible, Git-managed node is up (Elector::standing, the ADR-118 switch; it still votes). So when the controller is back, the election goes to it, and when the controller is down for good the replicas stand as before and one becomes the emergency primary. The hold-back is bounded: with no primary up for 60 s it ends and the node stands (an emergency primary beats none, e.g. when the Git-managed node is up but can't win).
  • Manual failover, and fewer than three voters, are unchanged: the takeover is promote (ADR-048), as is a person's promotion.
  • Wire and state. One new RPC kind on the existing stream; no new frame fields, nothing persisted. A node one version behind answers an unknown kind with an error, so the asker logs it and tries again later; the cluster is no worse off than before.
  • Trust. Only a cluster member over mTLS can ask, and the registry (signed by the CA) decides whether it counts. A node that is eligible already holds the cluster key, so the ask widens nothing; the worst a misbehaving eligible node can do is make an emergency primary stop renewing for up to 60 s, which is an election and a re-election, and DNS never waits for it.

Verification

a simulation test (clu_005_a_git_node_is_elected_when_the_emergency_primary_hands_over) with one emergency primary, one Git-managed node and one that holds back: the Git-managed node is elected within one lease window, never while the old primary can write, the other node votes and never asks, and the old primary steps down. The randomized partition simulator already moves nodes in and out of standing (ADR-118), which is the switch the hold-back uses. Unit tests cover when a node defers, how long, when the primary agrees, and when a node can stand to take over. Not tested: a live cluster of Git-managed and native nodes (the failover drill in deploy/cluster/ doesn't set the GitOps authority).

Consequences

with failover = auto and the gitops authority, a returning Git-managed node becomes primary again by itself about a minute after it is seen, plus an election (15 to 25 s of no config writes, DNS unaffected). If it can't win (no cluster key yet), the emergency primary keeps the role and is asked again every two minutes. A node that isn't Git-managed waits up to a minute longer to stand when the primary is lost while a Git-managed node is up. An emergency primary can still keep a cluster coordinated through a long Git-managed outage (certificate renewal waits for a primary), which manual doesn't do; the docs say so and let the owner of the cluster choose.

Amends

ADR-048 (the takeover under auto) and ADR-056 ("under a gitops authority an elected node that isn't Git-managed becomes an emergency primary": now only while no eligible, Git-managed node is up, or after the 60 s bound). Accepted by the owner on 2026-11-15.

Builds on ADR-048 ADR-049 ADR-056 ADR-118

ADR-155 Removing a node: a registry change plus a removed list that travels with every versionAcceptedAGTGitHub issue #5 (v0.6.0).

Context

GitHub issue #5 (v0.6.0). spec/07 lists DELETE /cluster/nodes/{id}, but it was never built, and the registry (the primary's nodes.json) only dropped ephemeral members. A Pi replica rejoined from scratch got a new node ID and left its old record behind as a down, eligible voter: the Cluster page listed it (even as primary, its last state from a period as emergency primary), failover.voters counted it (four voters, three reachable), and WrenDNSClusterPeerDown fired for it for good. Two facts make removal more than deleting a row. A node removed while it still runs reconnects with its certificate (valid for up to 90 days), and record_member re-adds an unknown member as an ephemeral one. And an eligible node holds a copy of the cluster key (ADR-051), which no registry change can recall.

Decision

  • The operation: DELETE /api/v1/cluster/nodes/{id} (admin; agents cluster:admin), wrendns ctl remove-node <id> and a Remove action on the Cluster page. The ID is the Cluster page's node ID, not a site or pod name (a destructive operation by a name that can match several nodes is a hazard). It runs on the primary; a replica's request goes there over the cluster channel (cluster.remove) and is audited as cluster.node.remove with the user and the entry node. It is allowed under a GitOps authority: membership isn't configuration (as maintenance, ADR-118). ?dryRun=true runs every check and says what would happen (AGT-002).
  • Refused: the primary (409 node_is_primary: promote another node first), an ID nobody has (404 node_unknown), a node that is up unless force (409 node_is_up), and, under automatic failover, a primary without its lease (503, like any publish).
  • What it does: the ID goes onto the removed list first, then the record leaves the registry, and the member and stream entries are dropped. A crash between the two leaves a refused node that is still listed, and removing it again finishes. Voters are the registry's eligible nodes and witnesses, so the removed node stops voting at once.
  • The removed list (removed.json, at most 512 IDs, oldest dropped first: node certificates last 90 days) travels in the signed manifest as a new optional field, omitted when empty and ignored by N−1. Every node refuses a stream from a listed ID: at its Hello, and for a stream that was open before, at its next frame. Replicas adopt it with the registry (an empty list from an older primary leaves theirs alone), and a promoted node keeps the one it has. So removal holds for a node that is still running, on every upgraded node, for as long as its certificate would have lasted; it can't renew without a stream.
  • The cluster key is not recalled. The answer says when the node was eligible (heldClusterKey, and a note) and points at wrendns cluster rotate-ca (T5.4c) for a node that is lost or can't be trusted. Removal doesn't rotate the CA: that is a multi-step change on every node, and for the usual case (a record whose machine is gone) there is nothing to protect. Rotating on every removal of an eligible node was rejected as heavy and surprising; a certificate revocation list was rejected because the removed list is the same thing keyed by what the stream authenticates (the node ID).
  • Failover: removing a voter can leave fewer than three, which turns automatic failover into manual (ADR-056). The answer and the dry run say so (automaticFailoverAfter, a note); it isn't refused.
  • Agents: not an MCP tool, like cluster_promote (spec/13): it needs cluster:admin and human approval. The REST route with its dry run is the agent surface.

Consequences

a node that rejoins needs a fresh token and an empty cluster directory (a new ID), as before. A removed node that still runs keeps dialing and being refused; the primary's event list says so (disconnected: this node was removed from the cluster), and its own log says the stream closed. A node one version behind doesn't know the list and accepts the removed node until it is upgraded: upgrade replicas first, as always. IDs are random from the node's key and never reused, so nothing needs undoing. Not covered: a removed node that holds the cluster key can still mint certificates for itself until the CA is rotated, as the answer says. Accepted by the owner on 2026-11-15, with the choices above: removal by ID only through wrendns ctl remove-node (not wrendns cluster remove), force for a node that is up, no CA rotation by removal, no MCP tool, and the Cluster view listing registry members that haven't connected since the node started.

Requirements AGT-002

Builds on ADR-051 ADR-056 ADR-118

Referenced by ADR-903 ADR-960

ADR-156 Support package: redaction that fails closed, keyed pseudonyms that keep structure, and a small log of the process's ownAcceptedOPSOPS-011 (owner request 2026-11-15, GitHub issue #50).

Context

OPS-011 (owner request 2026-11-15, GitHub issue #50). A person who hits a problem hand-copies configuration, logs, and cluster state and hand-redacts each, so they send too little to help or too much to be comfortable. Owner decisions: the default level is strict and fuller logs and the query-log sample are opt-in; the query-log sample is offered; it is not an MCP tool for now. What the code shows: the process writes its log to stderr only (WRENDNS_LOG, default INFO) and keeps none; the systemd unit runs as wrendns without the journal group, container logs are outside the container, and the Helm chart's root filesystem is read-only, so no platform lets it read its own log back, while data_dir is writable everywhere. Secrets sit in the configuration two ways: *_file paths (credentials, passwords, tokens, the bootstrap secret) and inline values (an OIDC client_secret, DoH extra headers). On the Pi the INFO log was 232 lines, about 21 KB, in 24 hours (restarts included).

Decision

  • The package. wrendns-support-<node>-<UTC time>.tar.gz (GitHub takes it as an attachment), built in memory and capped at 20 MiB (the largest files are truncated first, and the manifest says so). It uses our own ustar writer, miniz_oxide's deflate, and a small CRC-32: no new dependency. MANIFEST.json and README.txt (what it is, the level, how to read it) at the root; node/, config/, cluster/, health/, metrics/, stats/, logs/, and queries/ (only when asked).
  • Collectors. Each section is a collector with a 5 s deadline and a byte cap; a slow or failing one shows as missing in the manifest instead of failing the package; the whole run is capped at 30 s. They read the running state through the readers the API uses, from a low-priority thread, never from the query path. No collector can reach the user, token, session, TOTP, or key stores: they take typed views only.
  • Secrets are removed by classification and fail closed. Every configuration field has a class (plain, address, name, url, path, secret) in one table keyed by schema path. The configuration section is built by walking the config's JSON Schema (schemars, already a dependency) with the value. A field with no class is treated as secret, dropped, and listed in the manifest as unclassified, and a test walks the generated schema and fails when any property has no class, so a new field can't ship unclassified. secret fields are removed at every level, none included; a *_file field keeps its path but never the file's content; request headers and any key that looks like a credential are dropped as a second net.
  • Free text (log lines, error strings, the event timeline, the query sample) goes through a redactor in two passes. First the known values from the configuration and the cluster registry (every name, zone, client, group, host name, URL host, address, node and cluster name, e-mail address, and every secret held inline) are replaced exactly, longest match first, with regex-automata (already a dependency). Then hand-written scanners catch the shapes: IPv4 and IPv6, MAC, e-mail, URLs, host-name-like tokens, home paths, PEM blocks, and long hex or base64 runs (24 or more characters, mixed) become <token>. This is best effort, and the package says so.
  • Pseudonyms. A random 256-bit key per package, kept in memory (written, with the map, only to the --keep-map file). Names become device-N, group-N, node-N, pod-N, ns-N, host-N, zone-N.example, user-N@example.invalid, numbered by first encounter in a fixed traversal order, so the same input is the same pseudonym within a package; public domain names keep their last label (site-N.com). Addresses: loopback, unspecified, link-local, multicast, and a short list of well-known public resolvers stay as they are; the rest are permuted prefix-preservingly with a keyed BLAKE3 function (Crypto-PAn style), so addresses that share a /24 (or /64) still share one, and a private address (RFC 1918, ULA, CGNAT) keeps the bits that make it private. A MAC keeps its OUI (the make, not a person) and has the rest permuted.
  • Levels. strict (default): the pseudonyms above, no query names anywhere (counts only), no local zone contents, no free text, WARN and ERROR log lines only. standard: the same, plus INFO log lines, top lists with pseudonymized names, and the query sample (--query-sample N, up to 1,000 rows, never at strict). none: real values (secrets still removed). --reveal NAME keeps one name readable. The manifest records the level, and per file the collector, size, truncation, and the redactions per category (counts, never values).
  • The log. A tracing layer (SupportLog, in the wrendns crate) takes events at INFO and above whatever WRENDNS_LOG says into a ring of 2,000 lines and, through a bounded channel (1,024) to a writer thread, into two files under <data_dir>/logs/ of at most 1 MiB each (rotated when full; directory 0750, files 0640). A full channel drops and counts (the manifest reports the count); a failing write turns the files off and logs it once; with a read-only data directory only the ring remains. The files are on by default (owner, 2026-11-15); [support] persist_logs = false keeps only the ring. The lines are not redacted on disk (they carry client addresses): the redactor runs when a package is made, and the documentation says where they are.
  • Surfaces. wrendns support-package [--level strict|standard|none] [--reveal NAME]... [--query-sample N] [--keep-map FILE] [--list] [-o FILE]. POST /api/v1/system/support-package (admin; the options as the body; ?dryRun=true returns the manifest only; the answer is the archive as an attachment; audited as support.package with the level, never the content). On the System page a section with the three levels explained, a preview of the manifest, and the download. The pseudonym map is a second, short-lived download (owner, 2026-11-15): with keepMap: true the answer carries the package's ID, and GET /api/v1/system/support-package/{id}/map (admin) returns it as plain text, one pseudonym<TAB>original pair per line. The node holds the map in memory only, never writes it to disk, removes it after the first download or after 10 minutes, and keeps one map at a time (a new package replaces it); the page tells the person it holds the real names and isn't for attaching to the issue; the download is audited like the package. The CLI's --keep-map FILE writes it to a file the person chooses. No agent scope and no MCP tool.
  • Cluster (a later phase). --cluster asks each node over the cluster channel (support.collect; the entry node sends its key over the mTLS stream and each node redacts before answering), so unredacted data never leaves the node it is on.

Alternatives rejected

pattern-only redaction (fails open: a secret in a shape nobody listed leaks); an unredacted dump to review by hand (what people do now); uploading to a server (no infrastructure, and the privacy question); tar and flate2 crates (a dependency for about 100 lines); reading the journal or the container log (not readable); random address pseudonyms (they lose the subnet structure triage needs); encrypting to a maintainer key (key custody; possible later).

Consequences

a small amount of disk writing for the log files (about 21 KB a day on the Pi at INFO, measured 2026-11-15; none with persist_logs = false); a classification table to keep current, enforced by a test; none still holds real names, so the page says so; free-text redaction is a best effort, not a guarantee, and the person is told to look the package over; the log files hold client addresses unredacted, mode 0640, which the documentation states. Verification (planned): a fixture with every kind of secret planted, scanned for in raw, decompressed, base64, and URL-encoded forms at each level; the classification-completeness test; a fuzz target for the redactor (never panics, planted values never survive); determinism (same key and input, same bytes); a prefix-preservation property test for addresses; the size cap and truncation; the manifest's counts against a recount; log layer tests (ring bound, rotation, lines surviving a restart, a full channel dropping instead of blocking); make bench-smoke unchanged, since nothing runs on the query path. Decided by the owner (2026-11-15): the UI offers the name map as a short-lived second download; the log files default to on. Accepted 2026-11-16: the owner asked for issue #50 to be built ("address issue 38 and issue 50") after being told that would be read as accepting this ADR as written; the decisions above stand. Built in phases: the log layer first, then the archive, redaction and pseudonyms, then the collectors and the three surfaces; --cluster stays a later phase.

Requirements OPS-011

Referenced by ADR-1005 ADR-163 ADR-172 ADR-903 ADR-910

ADR-157 A calmer dashboard, with the previous one an address awayAcceptedDNSOBSowner request after a user called the dashboard graphs "a tad cluttered" and liked Technitium's smoothed graphs with the exact number on hover, and the wide tables at the bottom on a small screen.

Context

owner request after a user called the dashboard graphs "a tad cluttered" and liked Technitium's smoothed graphs with the exact number on hover, and the wide tables at the bottom on a small screen. The shipped page was 2,577 px tall on a phone: six tiles with rings, sparklines and change pills, a legend that wrapped into a table, a donut, and three full-width lists. A read-only look at real query logs showed why the lists widen: on a Kubernetes primary, a few hundred distinct names over 30 characters carry about three quarters of a day's queries (Kubernetes services, pods' search-list expansions of one real name, 64-hex labels), and none on a Pi. Ten local variants were built on the real data and compared at desktop and phone width; the owner chose "Calm". Decision (proposed):

  • Layout. The six tiles become a row of plain numbers (queries, blocked %, cache hits, upstream p90, active clients, each with its change against the previous period) and a line for NXDOMAIN and SERVFAIL; the chart is the one card; the three lists sit open on the page. Advanced adds, without adding cards: the service level (OBS-016) as a strip under the numbers; the chart's other views, Groups and Upstream, beside Totals and Status; where time goes (OBS-030) and the upstreams as plain sections; devices by type (OBS-025) as a line under the clients; the cache number split into fresh and refreshed (DNS-007).
  • The chart. Smoothed for the eye, exact for the reader: the range is cut into at most one bucket per 6 px (120 at most), each bucket's counts are the exact sums for its interval, drawn through as a monotone curve (it never overshoots); the newest, still-filling bucket is left off so the line doesn't dip. The legend is a row of range totals; hovering shows the interval and each series' exact value (a compact tooltip on a short chart); charts can share a cursor. The Table view (spec/07 §3) and the change markers (OBS-033) stay. The value axis uses compact numbers (10K).
  • The lists. Every row is one line, and what doesn't fit is a tap away: the count opens the full name with a Copy button and, where the display differs from the name, one sentence saying how. The display reads the name (heuristics on the name alone): service.namespace.svc.cluster.local is the service with its namespace as a tag; a pod's search-list expansions of one real name (<real>.<ns>.svc.cluster.local, <real>.svc.cluster.local, <real>.cluster.local) are one row tagged search suffix ×N whose detail lists the variants and whose link searches the real name; hash and UUID labels are shortened; in-addr.arpa and ip6.arpa names show as the address; a pod's own name shows as its workload's. A name that is still too long gives up its subdomain first, a single long label keeps its end, and a tag gives way before the name. The lists ask the API for 25 and show 10 after merging.
  • Loading. The same calls as before, except the Advanced-only measures (latency by path and stage, the service level) are fetched only in the Advanced view, independently of the rest and one fetch at a time: the service level took about 12 s on a slow node, and the page must not wait for it.
  • The previous dashboard stays, unchanged, as DashboardLegacy: #/?view=legacy shows it and remembers the choice in this browser (localStorage, dashboard.view); #/?view=new goes back. The only link to it is a quiet "Switch to the new dashboard" on the old page itself. It is not documented for users yet and is meant to be removed once nobody asks for it (the owner decides when).
  • No API change.

Alternatives rejected

the other nine variants (Sentence, Lens, Share, Multiples, Console, Sections, Rail, Hero, Glance: kept in the lab branch for reference); a visible toggle between two dashboards (it publicises the old one and doubles what we maintain); wrapping long names onto two lines or scrolling the lists sideways (rows of different heights; a scroll bar inside a card on a phone); showing the full name in a tooltip only (a phone has no hover); normalising cluster search suffixes in the server's top lists (they waste three of the top-100 slots per real name; a data-quality improvement for a later ADR); the per-device status bar of issue #30 (it waits for the reporter's backend change, and the row is designed to take a thin bar under the name).

Consequences

a UI-only change (bundle 229.8 KiB gzipped, budget 400); about 600 lines of the old page stay until it is removed; the name rules are heuristics, so a namespace named like a top-level domain (io, dev) reads as part of the name, and the full name is always one tap away; the README and site screenshots are regenerated; the e2e suite is updated to the new structure and gains a test for the old page's address.

Verification

the e2e suite (the dashboard, service level, cache tile, groups, device naming, change markers, internal names, and the legacy address); npm run check clean; the size budget; no sideways scroll at 360 px; the demo network and the live cluster's name shapes looked at.

Requirements DNS-007 OBS-016 OBS-025 OBS-030 OBS-033

ADR-158 The Clients page's list is the last hour, not the clock hourAcceptedAGTThe Clients page's first tab, Seen this hour, read /stats/top?kind=clients, whose Space-Saving list starts again on the clock hour.

Context

The Clients page's first tab, Seen this hour, read /stats/top?kind=clients, whose Space-Saving list starts again on the clock hour. A minute past the hour it held a minute of clients, and the label didn't say so: the owner found "seen this hour" meaningless and asked for "last hour" (2026-11-15). "Last hour" over the clock-hour list would have been wrong in the same way.

Decision

  • The aggregator keeps per client its queries in 5-minute slots over the last 13 (wrendns-telemetry::last_hour): the slot in progress and the 12 before it, so the window always holds the full last hour (60–65 minutes) and a device seen 59 minutes ago is never left out. Updated on the telemetry thread with the other analytics (one hash update per query), never on the query path. At most 4,096 clients; a client leaves once its newest slot leaves the window, checked once per slot, and the table shrinks when a burst of clients has aged out.
  • A live query a whole window older than the newest means the clock was set back: what was counted is dated in the future, so it is dropped and counting starts again (otherwise every new query would be ignored until the clock caught up). Not saved: after a restart the query-log replay that rebuilds the hourly lists (T6.16) refills it, through a path that never moves the window back.
  • GET /api/v1/stats/top?kind=clients&hour=rolling reads it (a new TopHour for /stats/top; /stats/latency keeps Hour). Counts are exact (errorBound 0). hour=rolling with another kind is a 400. With group, the clients whose settings come from that group (their first group), so a group-restricted agent (AGT-004 adds its group) reads its own devices. MCP top_items takes hour: rolling.
  • In a cluster every node sends all its clients (not just its top limit), so the merged counts stay exact; the list is cut afterwards. A peer one version behind doesn't know the read and is listed in missingNodes, and the Clients page says which nodes are left out.
  • The tab is Last hour (heading "Seen in the last hour"); its id stays seen, so #/clients?tab=seen links keep working. The Traffic map's hour picker says previous hour (not "last hour") for the previous clock hour, so "last hour" means one thing in the UI.

Consequences

at the 4,096-client cap about 0.65 MB (80 bytes per client in a table of 8,192 buckets), given back as they age out; a home network's 50 devices take a few KB. In a large cluster each page load moves every node's client list (up to 4,096 rows each). This is a clients-only window: a rolling hour for the other top lists would need a Space-Saving sketch per slot (13 times their memory), so they stay per clock hour, as do the dashboard's active clients and a group's device count, which say "this hour".

Requirements AGT-004

ADR-159 Top clients split by status: a payload on the Space-Saving entryAcceptedOBSOBS-004 (community request 2026-11-15).

Context

OBS-004 (community request 2026-11-15). The dashboard's Top clients bar shows how many queries a device made, not what happened to them: a TV in second place could be mostly blocked telemetry or mostly cache hits, and finding out means a query-log search per device. The status chart already colours blocked, cached, and forwarded queries for the whole network.

Decision

  • SpaceSaving<K, T = ()> carries a payload per entry. offer_with updates it; an entry that replaces the minimum starts a fresh T::default(), so the payload covers the count - error occurrences since the key entered the list. Every other list keeps T = () (zero-sized, no change in memory or work).
  • The hour's client list and each group's client list carry ClientStatus { blocked, cached, forwarded } (three u64; 24 KB per hour for 1,000 clients). Cached takes in stale and refreshed, as the dashboard's status chart does. Other statuses aren't counted: they're the rest of count.
  • TopItem.byStatus (kind=clients, with or without group); absent for other kinds. Federated reads add the splits per key; a peer one version behind sends none, and its queries stay in count only.
  • Each closed hour's split is stored in hour_top as three more kinds, clients_blocked, clients_cached, and clients_forwarded, keyed like clients (its 100 kept clients; zero counts skipped; none at privacy level 2 and above, like clients), so a read over stored hours can add them per client. Today's top-list reads use the current or previous hour from memory; the stored rows are for range reads.
  • The dashboard draws a top client's bar as segments in the status colours (--s-blocked, --s-cached, --s-forwarded, --s-other for the rest), with a legend and the numbers on hover.

Consequences

Up to 300 more hour_top rows an hour, usually far fewer (most clients have no blocked queries). One more field update on the aggregator thread per query (no hot-path change: workers only emit events). The hours rebuilt from the query log at start (privacy level 0) get the split too, since the replay goes through the same path. Grey can include the Space-Saving error bound, which on networks under 1,000 devices is zero.

Requirements OBS-004

ADR-160 Top lists and active clients follow the dashboard's time range, by whole hoursAcceptedOBSOBS-004 (community request 2026-11-15).

Context

OBS-004 (community request 2026-11-15). The dashboard's range picker (15 min to 30 days) changes the charts and totals, but Top domains, Top blocked, Top clients, and the Active clients tile always showed the current clock hour, so they emptied every time the hour turned and never matched the range above them. Each closed hour's top lists are already stored in hour_top for 400 days.

Decision

  • GET /stats/top takes from (and to), parsed like the summary's. The window is every hour it overlaps, counted whole: the stored hours from hour_top (top_range) plus the current hour (and the previous one until it's stored) from memory, summed per key, counts and error bounds alike, reading twice the limit (at least 20) from each source, as the SLO attribution lists do (ADR-127). Hour granularity is said in the UI ("since 14:00") rather than hidden.
  • from doesn't combine with hour, client (per-device domains live in memory only), or group (groups' lists aren't stored): 400 with a hint. The dashboard sends from unless a group is picked, and then says "this hour".
  • Summary.activeClients is the length of the same list over the summary's range (limit 1000), so the tile and its change follow the range. Stored hours hold their busiest 100 clients, so past the last two hours it undercounts on networks with more than 100 devices an hour (documented).
  • Federated: a TopRange read; each node sums its own hours and the lists merge by key (OBS-012). A peer one version behind doesn't know the read and is listed in missingNodes.
  • MCP top_items takes from and to.

Consequences

One SQL aggregate over at most 720 hours × 100 rows per list and dashboard poll (three lists, plus two for the summary and its comparison). The meaning of activeClients changes from "this hour" to "the range's hours"; a client that wants this hour asks with from=-1s. Names that never reached an hour's top 100 don't add up across hours.

Requirements OBS-004 OBS-012

Builds on ADR-127

ADR-161 Muting anomaly findings: a shared entry per standing rule, matched when the finding is created, recorded but not countedAcceptedCLUOBSGitHub issue #38.

Context

GitHub issue #38. Some smart TVs look up the PTR of every address in their /24 once an hour: about 225 NXDOMAIN answers in a minute, 87% of the device's queries, so each node they ask raises an nxdomain_storm finding every hour, WrenDNSDeviceAnomaly fires all day, and the Anomalies page is all TV. Acknowledging (OBS-014) clears one finding; the next hour brings another. What exists doesn't help: ignore_domains doesn't apply to nxdomain_storm and can't name a reverse zone; raising nxdomain_per_minute weakens the check for every device and is per node; [exclusions] for the reverse zone takes those lookups out of the query log and analytics for every device, so a device sweeping the same network would go unseen, which is the case the storm check exists for; an Alertmanager silence can't target a device (the metric carries only kind). Two facts about the code decide the shape: the [telemetry] section is node-local (shared.rs), and the TVs ask both nodes, so the contributor's [[telemetry.anomaly.mute]] would have to be written on every node; and the engine runs on the telemetry thread, which knows addresses, not device names or groups. Decision (proposed):

  • The entry. A top-level, shared [[anomaly_mute]], managed like groups and schedules: name (unique, the key), devices (device names, addresses or networks) and groups (at least one of the two), kinds (at least one of the finding kinds; there is no "all": a blanket mute is ignore-ing the engine, and an explicit list keeps a new kind visible), expires (optional, RFC 3339; the same words as a quick rule's devices, groups, expires, note), note (up to 200 characters). Who made it and when is in the audit log, not in the entry. Files, the API, the UI, MCP and Git all write the same entry; in a Git-managed cluster the API, UI and MCP refuse with 409 gitops_managed and show the TOML to commit, as for other configuration. It travels with the cluster's shared settings (CLU-003), so a mute set on the primary reaches every node, including ones that were down.
  • Matching. Device names and groups are resolved when the configuration applies, from the [[client]] entries (their addresses and networks) and the group's networks and the clients that name it, into a sorted list of address networks. The engine receives one compiled table (networks, a bit per kind, an optional end time) swapped in atomically, like exclusions, and looks a finding's device up once, when the finding is created: a lookup per finding, never per query. A name that matches nothing, an unknown kind, or an empty list is a 422 (the dry run says which). A MAC address is refused with a hint: the address changes with the lease, and the neighbour table isn't on the telemetry thread.
  • What muted means. The finding is recorded as before, with muted: true and mutedBy: <mute name>, so it is listed (with Show muted) and available to the API and MCP. It is not counted in wrendns_anomalies_total{kind}, the sidebar badge, the anomaly alert rule, or the health level, and it is counted in a new wrendns_anomalies_muted_total{kind}. The decision is made at creation against the finding's window start: an expires in the past mutes nothing new, a finding created while the mute held stays muted after it ends, and replaying the same queries with the same mutes gives the same findings.
  • Surfaces. PUT and DELETE /api/v1/anomaly-mutes/{name} (dry run, If-Match, Idempotency-Key, audited as config.anomaly_mute); the mutes are listed by GET /api/v1/config/entries?kind=anomaly_mute, as schedules and alerts are; GET /api/v1/analytics/anomalies gains muted and mutedBy on each finding and muted=true to include them (hidden by default, like acknowledged ones); MCP list_anomaly_mutes, plan_set_anomaly_mute, plan_remove_anomaly_mute (named like plan_remove_local_name), and find_anomalies takes muted. Agent scope: a mute can hide a compromised device, so it has its own write scope, config:write:anomalies (included in config:write:*), rather than riding on ops:anomalies (acknowledging, which changes no configuration) or config:write:groups. The Anomalies page gets Mute for this device… on a finding (the device and kind pre-filled; for 1 day, 1 week, 30 days or until removed, 1 week by default; a note; the preview, a dry run, before saving), a Show muted switch, and a list of mutes with Remove.

Alternatives rejected

[[telemetry.anomaly.mute]] (node-local: the TVs ask both nodes); a per-device ignore_kinds on [[client]] (no expiry, no note, and a device with no [[client]] entry can't have one); acknowledging on a schedule (a finding every hour is the problem); a mute that hides the finding entirely (nothing lost is a rule here: muted findings stay listed and counted apart); matching by MAC now (see above); muting a domain rather than a kind (a different decision, left open).

Consequences

one more shared section (older replicas skip it, CLU-010, and so don't mute until updated: they would still raise the finding, which is the safe direction); a lookup per finding on the telemetry thread; the muted counter's label set is the finding kinds, as wrendns_anomalies_total's. Implementation notes (2026-11-15): the owner approved the shape ("shared top-level is right, go build it"). Two things were added that the proposal didn't say, for the owner to confirm: the config:write:anomalies agent scope above, and chatty_loop is a mutable kind like the others (it is informational and never counted, but a muted one is marked and listed the same). A device name must be a [[client]] with an IP or network in its match; a device known only by the router's DHCP or mDNS name is muted by its address. Accepted 2026-11-16: the owner answered "Go on all 5" to the list that named these two additions as the open confirmation, so the scope and the mutable chatty_loop stand as built. Verification (planned): the engine with a mute table (muted and unmuted devices, expiry, kinds, counters, replay); the config checks (unknown kind, empty lists, names that match nothing, MAC refusal); the API and MCP (dry run, If-Match, 409 under Git, audit); the UI flow end to end; the cluster path (a mute set on the primary reaches a replica).

Requirements CLU-003 CLU-010 OBS-014

Referenced by ADR-162 ADR-907 ADR-910

ADR-162 Editing a quick rule: replace it in place under its ID; the rule keeps its maker and its startAcceptedCLUGitHub issue #57 (a user's request).

Context

GitHub issue #57 (a user's request). The Quick rules page lists each rule with Remove only, and the form only creates, under a new ID. Changing a rule's domain, scope, end or note meant removing it and making another, which leaves a gap with no rule in effect and loses who made it and when. PUT /api/v1/rules/{id} already creates or replaces, but it stamped the writer and the time on every write. Decision (proposed):

  • No new operation. Edit is PUT /api/v1/rules/{id} with the rule's own ID (dry run, If-Match, Idempotency-Key, audited as rule.put with the rule before and after, so the editor is the audit entry's actor). Rules from the config files stay read-only (409).
  • What a replace keeps. The new rule keeps createdBy and created of the one it replaces, so the Quick rules page's Made column says who started the rule and the audit log says who changed it. A stored rule that has none (older data) takes the writer and now, as a new rule does. A client that re-puts a rule to refresh its start no longer does: it removes the rule and makes another.
  • The page. Edit on a rule made through the UI or API opens the form below the row, filled in from a copy taken when Edit was pressed (the list reloads every 10 s and must not reset the form). The preview is a dry run under the same ID, so it simulates the replacement and not an addition. The durations gain Keep the current end (…), first and selected when the rule has a live end time (sent as expires, so an edit that changes the note doesn't extend the rule), and Until I remove it is selected when it has none; any other duration counts from now. The confirmation is a notice on the page, not in the row, which the reload redraws (ADR-161's lesson).
  • No Edit for a rule from a config file, an expired one (the sweep removes it within seconds), or one made through the API for several devices or groups, or devices and groups together (the form holds one scope, so saving would drop the rest silently). The row says to use the API.

Alternatives rejected

remove then make a new rule from the page (a gap with no rule, and the maker and the start are lost); refreshing created on an edit (it would hide who started the rule); updated and updatedBy fields (a new key in the shared configuration, which an older replica refuses, CLU-010, and the audit log already has the editor); a separate PATCH (replace is the contract today); a multi-select scope in the form (a larger change, left open).

Consequences

no configuration schema change and no mixed-version issue; one behaviour change for API clients, that a replace no longer refreshes created; the page has two forms when one is being edited, so the form's test IDs differ (quick-rule-form for a new rule, quick-rule-edit-form for an edit). Accepted 2026-11-16: the owner answered "push it, accept ADR-162" after seeing the behaviour change (a replace no longer refreshes created) and the cases without an Edit.

Verification

a unit test that an edit keeps the maker and the start and that older data takes the writer; the e2e test flt_005 edit a quick rule in place (the form from the rule, the end kept, the old domain no longer allowed, one rule, the maker and start unchanged, the notice, Cancel).

Requirements CLU-010

Builds on ADR-161

ADR-163 A watchdog restarts a node that stops answering and leaves evidence of why: a thread outside the runtime, two checks, a bounded dumpAcceptedNFROBSGitHub issue #7.

Context

GitHub issue #7. A 2-CPU node (an LXC container with a two-worker runtime) stopped answering DNS, the API, metrics and the cluster port while its service stayed active, and stayed that way until someone restarted it, about twelve minutes in the report. The cluster saw its link drop a minute after DNS stopped, as if the runtime's threads stuck one by one. There was no stack to read: the release binary is stripped (ADR-164 gives it a debug file), and nothing on the node looks while it hangs. systemd's Restart=on-failure only acts when the process exits, and /livez is served by the runtime that is stuck. The owner promised the reporter (2026-11-14) "a watchdog so a wedged node restarts itself and leaves a dump", and held 0.7.0 until it exists (2026-11-16). Decision (proposed):

  • Where it runs. One OS thread, wrendns-watchdog, started once the listeners are bound, outside the Tokio runtime and apart from the UDP workers: a check that runs on what is stuck can't see it. It wakes every 5 s. It adds nothing to the query path: no new code in the pipeline, no allocation, locking, or I/O there (02 §3).
  • Two checks. (1) The runtime runs a task: the thread spawns a trivial task on the runtime's handle and waits up to 2 s for it. (2) The listener answers: a plain UDP socket sends the node's own first UDP listener (UDP never has a PROXY header) a query for probe.wrendns.invalid (the name of OBS-020's probes, which special-name handling answers NXDOMAIN without an upstream, and which the telemetry thread keeps out of the query log and analytics; a wildcard bind is asked on loopback); any well-formed reply with the same ID passes. With no UDP listener, only the first check runs.
  • Wedged. A check that has not passed for stall_secs (default 60, 20 to 600), counted from its last pass, or from the thread's start before its first one. A node that is still starting answers (the listeners are bound before the thread starts, and the lists compile in the background), so a long start doesn't count. One pass resets that check, and a stall that has been dumped is not dumped again until a check has passed.
  • What happens. The thread writes the dump, logs at ERROR, flushes the log files, then, with action = "restart" (the default), exits the process with status 70 for the supervisor to restart (systemd Restart=on-failure, Compose restart: unless-stopped, a Kubernetes container). With action = "dump" it writes the dump and leaves the node running. Restarts are limited to max_restarts_per_hour (default 6), kept as timestamps in <data_dir>/watchdog/restarts; past the limit it writes the dump and does not exit, so a false positive can't become an endless loop. A node started by hand (wrendns run in a terminal) just exits.
  • The dump. <data_dir>/watchdog/hang-<UTC time>.txt (directory 0750, files 0640, the newest 5 kept): why (which check, for how long, when each last passed), the build (version, commit, target, build ID), uptime, the thread table from /proc/self/task (id, name, state, what it waits on, CPU time), memory and load, the query counters, the last 200 lines of the support log (ADR-156), the executable's mapping (to turn addresses into offsets), and each thread's stack (below). It holds no client data and no configuration. The support package includes the newest one.
  • Stacks. Each thread reports its own stack: the watchdog sends every thread of the process a signal (tgkill), and the handler stores up to 64 return addresses in a fixed table without allocating or taking a lock. The dump lists the raw addresses; addr2line -e wrendns.debug <address> resolves them with the debug file of ADR-164. The wait is bounded (5 s); a thread that doesn't answer is listed as not answering, which says it is stuck in the kernel. The only unsafe is in wrendns-net, with // SAFETY: comments (NFR-003). The binaries need unwind tables for this (-C force-unwind-tables, size only, no runtime cost).
  • Settings. A node-local [watchdog] section, restart-only like [support]: enabled (true), stall_secs (60), action (restart or dump), max_restarts_per_hour (6). Metrics: wrendns_watchdog_enabled, wrendns_watchdog_check_age_seconds{check} (seconds since each check last passed), and wrendns_watchdog_last_action_timestamp_seconds (the last dump or restart, from the files, 0 for none). The log says at start-up when the previous run ended by the watchdog.
  • Not covered. A node that serves some clients and not others; a hang that spares both the runtime and the UDP listener (the checks ask whether it answers, not whether it answers well); a stall under 60 s.

Alternatives rejected

systemd's WatchdogSec with sd_notify (only systemd; the checks would still be ours); /livez as the check (the stuck runtime serves it; a Kubernetes liveness probe already restarts a pod, but a systemd or Compose install has nothing); a task on the runtime as the checker (it can't see its own stall); abort() for a core dump (needs core limits and systemd-coredump, so most installs would leave nothing); running gdb or eu-stack from the node (the unit's seccomp filter forbids ptrace, and the image has no tools); frame pointers everywhere for cheap unwinding (a cost on the query path); a self-update of the policy from the cluster (this is a node's own safety net).

Consequences

one idle thread waking every 5 s; one probe query per 5 s counted in wrendns_queries_total like OBS-020's; four more settings and three metrics; a real wedge costs a restart (a few seconds) instead of an outage until someone notices; the dump needs the build's debug file to read. A false positive restarts a healthy node once per hour at most six times: the checks only fail when nothing answered for a minute. Accepted 2026-11-16: the owner answered "Push and accept" to the account of what was built, measured and left untested (stack capture on arm64 and armv7 is unproven: the ARM builds link, and the capture is bounded). Verification (planned): the check loop with a stubbed clock and probes (a stall is counted from readiness, a pass resets it, the restart limit); a runtime whose workers are all blocked is seen as stalled and a healthy one is not; the UDP probe against a real listener, a silent socket and one that answers with the wrong ID; the dump's content and its permissions, keeping five; the stack capture in a test that blocks a thread and finds its own function's address; a real-binary run on the build VM, built with a wedge-test Cargo feature (never in a release build) that blocks every runtime worker some seconds after start: the watchdog writes the dump and the process exits with status 70 within stall_secs and one check, as a supervising shell loop sees, and the dump's stacks name the blocked function.

Requirements NFR-003 OBS-020

Builds on ADR-156 ADR-164

Referenced by ADR-164 ADR-168

ADR-164 Debug files for every build: link once with line tables, publish the split-off symbols beside the stripped binaryAcceptedNFRGitHub issue #7.

Context

GitHub issue #7. The release binary is stripped (strip = true), so a stack taken from a hung node (a gdb or eu-stack capture, or the watchdog's dump, ADR-163) is a list of addresses nobody can read. The owner promised the reporter "debug symbols with 0.7.0" (2026-11-14). Two things limit the options: the image must stay under 15 MiB compressed per architecture (NFR-001), and the binaries are built reproducibly with cargo auditable (NFR-005), whose dependency data (.dep-v0) must survive. Decision (proposed):

  • One link, then a split. The Dockerfile's build stage links with strip = none and debug = "line-tables-only" (the profile's environment overrides, in that build only), then uses llvm-objcopy from the pinned toolchain's llvm-tools component: wrendns.debug gets the symbol table and the line tables (--only-keep-debug); the shipped binary is the same link output with the symbol table and debug sections removed, .dep-v0 kept, and a .gnu_debuglink to the debug file's name and CRC. A second build with symbols would have a different layout, so its addresses wouldn't match the shipped binary; -C split-debuginfo keeps no symbol table and isn't supported for fat-LTO static musl builds through zig.
  • No build ID. The zig linker writes none, so the pairing is the release it came from (same version, same architecture) and the debuglink's CRC, which gdb checks.
  • Published as wrendns-<arch>-linux.debug.zst (zstd level 10) for the three architectures, in every release and every edge build, under the signed SHA256SUMS like the binaries and their SBOMs (verify.sh checks that the lines exist). releases.json doesn't list them: a node never fetches one, and self-update takes only its own binary by name.
  • What it costs, measured on the build VM (2026-11-16, the real Docker build for amd64):
    • The shipped binary is 34,754,048 bytes against 34,748,816 (5,232 more, 0.015%): .text is 4,864 bytes larger and everything after it shifts, because turning debug info on changes the linked layout a little (it also adds a one-byte .debug_gdb_scripts reference). It is not byte-identical code, so it was benchmarked rather than assumed: bench-smoke alternating baseline and split binary, ten runs each: cache-hot throughput median 530,993 against 526,753 qps (ranges 482k to 586k and 449k to 557k), cache-hot p50 45.5 against 33.0 µs, miss-heavy p50 217 against 223 µs. No difference beyond the run-to-run spread.
    • The debug file is 141 MB raw, 25.5 MB at zstd 10 (21.4 MB at 19). Three of them add about 75 MB to each edge publish.
    • The stripped binary no longer carries the .comment section (the compiler's identification string).
  • Reproducibility. The nightly two-clean-builds check still compares the binary byte for byte; it reports as a warning if the two debug files differ.
  • Reading a stack: zstd -d wrendns-<arch>-linux.debug.zst -o wrendns.debug, then addr2line -e wrendns.debug -f -C -i <address> (docs/running.md). With line tables, an inlined call shows the line of its own source.

Alternatives rejected

shipping the unstripped binary (141 MB more in every image); full debug info (several times larger, and the line tables answer what a stack needs); a debuginfod server (a service to run); symbols only, no line tables (byte-identical code, and function names alone lose where inside a large function the thread waits).

Consequences

the Docker build adds a few seconds; the image workflow uploads 75 MB more; the release page gets three more files; a stack from any build can be read after the fact if its release's debug file is kept (edge builds are replaced on every push, so a dump from an edge build is read with that build's file while it is still published). Accepted 2026-11-16: the owner answered "Push and accept" after seeing the numbers above, including that the code is not byte-identical and that the images were already within 0.34 MiB of the 15 MiB gate (this adds about 0.04 MiB). Verification (planned): the A/B above; the split binary keeps .dep-v0 and gains the debuglink; addr2line on the debug file resolves a frame the watchdog's e2e produces (ADR-163's deploy/watchdog/e2e.sh on a debug build); the edge image run publishes the three files and the size gate and SBOM checks still pass.

Requirements NFR-001 NFR-005

Builds on ADR-163

Referenced by ADR-163 ADR-906

ADR-165 The last healthy member may use the rest of the budgetAcceptedGitHub issue #35 (0.7.0-edge).

Context

GitHub issue #35 (0.7.0-edge). spec/04 §4 caps each attempt at min(timeout_ms (400 ms by default), 3 × EWMA + 50 ms, 1 s). A forwarder's EWMA reflects its cached answers: about 6.5 ms for NextDNS over DoH on a home network, so the cap is about 70 ms. A name the forwarder has to resolve itself takes much longer: a carrier's Wi-Fi Calling gateway with TTL 0 took 900–1,200 ms at Cloudflare, Quad9 and NextDNS alike. In a group of one, that single attempt timed out at ~70 ms, Group::resolve had nothing left to try, and the client got SERVFAIL ("no upstream answered") after ~100 ms with 1.9 s of the budget unused. On two nodes in a day: 346 SERVFAILs (0.3 % of answers, enough to fire the availability SLO's ticket alert), 735 upstream timeouts, 2 upstream SERVFAILs; every lookup of the Wi-Fi Calling name failed. Larger groups do the same when every member is cut off. timeout_ms can't help: it only lowers the cap, and its 400 ms default is below these answers too.

Decision

  • The adaptive bound and timeout_ms are there to give up on one member and move to the next; hedging (ADR-013) already starts the next member before that. So the attempt on the last member in the attempt order whose breaker is closed may use the query's whole budget (Upstream::timeout_for(true)), bounded by what's left of it, as recursive upstreams already may (ADR-074). Earlier members keep their timeouts, and hedge delays are unchanged.
  • Members whose breaker is open or half-open (last resorts and probes) never get it. An upstream that is really down costs up to the budget per query until three timeouts in a row open its breaker, and then fails as fast as before.
  • No retry of a member that timed out: the one attempt waits longer, so the upstream isn't asked twice and no timeout is recorded against an upstream that was only slow.

Consequences

a slow name is answered in the time it takes, up to the budget, for single-upstream setups too. A blackholed sole upstream makes clients wait up to 2 s for SERVFAIL, not 70–400 ms, until its breaker opens (three queries). Serve-stale (1.8 s) is unchanged. The patient attempt is at most the 2 s budget, under the 3 s after which a silent pooled connection is closed (ADR-148). Tested by ups_006_single_member_waits_for_a_slow_answer and ups_006_last_healthy_member_waits_for_a_slow_answer.

Builds on ADR-013 ADR-074 ADR-148

ADR-166 UniFi: name offline clients from the console's client history and configured clientsAcceptedT8.2, ADR-080 ("not done: UniFi's offline known clients").

Context

T8.2, ADR-080 ("not done: UniFi's offline known clients"). stat/sta lists only connected clients, so a device that has since left (a phone off Wi-Fi, a laptop asleep) loses its name: its queries in the log and the top lists show a bare address, though UniFi still has its alias. Decision (proposed):

  • After stat/sta, each UniFi poll also reads the v2 client history (/v2/api/site/{site}/clients/history?withinHours=<offline_days × 24>, Network 7+, the clients the console remembers within the window, with their last_ip; it answers to an API key) and the configured clients (/api/s/{site}/rest/user: aliases and fixed addresses), with the same API key or login cookie.
  • Both are best effort: a console that can't answer either (an older Network application has no v2 API) still gives its connected clients, and the failure is logged at debug only, so the router isn't marked as failing.
  • A known client is used only if it has a name (an unnamed one adds nothing a bare address doesn't). Merged by MAC: a connected client keeps its address and takes a known name only when it has none. An offline client gets its last address, else its fixed one (only while use_fixedip is on: UniFi keeps fixed_ip after a reservation is turned off), and only if no connected client holds it; between offline clients claiming one address, the most recently seen wins.
  • Bounded (added in review; the first version asked for every remembered client, with no limit). (1) [[router]] offline_days (default 30, 0 for connected clients only, at most 365): the window goes to the console as withinHours, and every row with a last_seen older than it is dropped (a row with no date can't be judged and stays). (2) At most 1,024 offline clients per poll, most recently seen first (OFFLINE_MAX): they ride in every node's answer to devices.names (about 130 bytes each, against the cluster channel's 1 MiB frame) and sit in memory on every node. (3) An offline client is marked (Lease.offline) and is not "seen": it names an address in the query log, but device identification (ADR-117) doesn't count it. Counted there, a remembered device would never age out, and with identify.max_clients (1,024) the places taken by devices that aren't there could push out ones that are.
  • The default is not written to the replicated configuration and the mark is not written for a connected client, so a node on a build before this one (the [[router]] section refuses unknown fields) still reads the shared configuration and the devices.names answers. Roll every node before setting offline_days in the shared configuration.

Consequences

  • Offline devices keep their UniFi names in the UI and the query log. Two more read-only requests per poll (every interval_secs, 300 by default), for the last 30 days only: a site with thousands of remembered clients no longer makes an answer that grows with its whole history. This turns on for every existing UniFi integration at the default; offline_days = 0 restores the old behaviour.
  • A stale last address can name a device UniFi doesn't see now (a guest that left, its address since leased to something unnamed WrenDNS can't tell apart). Names configured in WrenDNS and its own DHCP leases still win.

Builds on ADR-080 ADR-117

Referenced by ADR-080 ADR-942 ADR-973 ADR-983

ADR-167 Domain icons: a brand tile or a monogram, drawn locally and never fetchedAcceptedA user asked for icons left of domain names, from https://icons.duckduckgo.com/ip3/&lt;domain&gt;.ico.

Context

A user asked for icons left of domain names, from https://icons.duckduckgo.com/ip3/<domain>.ico. The owner declined any dependency on a third-party icon service (2026-11-16). Measured that day: the endpoint is unofficial (no published terms, uptime, or rate limit), serves ICO only, and answers a 404 with a grey placeholder for a subdomain or a name with no site (api.github.com, time.nist.gov); and every icon request tells the service which names are in the query log, which shows work names, internal names, and trackers. The UI's content security policy allows images only from 'self' and data:, and at privacy level 1 and up names are hashed (ADR-102), so there would be nothing to look up. Fetching from the sites themselves, from the server, would connect a resolver host to names any client chooses, many of them blocked or malicious (an SSRF and egress risk; review 06-02 fixed one of this kind). A lab on a synthetic demo network, in light and dark and at 390 px, showed the design below on the real lists. Decision (the owner chose it on that lab, with rounded shapes, 2026-11-16):

  • The UI draws a mark left of a domain name in the dashboard's Top domains and Top blocked lists and in the query log's name column. All of it is drawn in the browser from data already in the UI: no request is made for an icon, the content security policy is unchanged, nothing about queries leaves the page, and the DNS path and the API are untouched.
  • A name that belongs to a brand in the map gets the brand's tile: its colour and glyph from Simple Icons (CC0-1.0, pinned at 16.38.0; the marks are their owners' trademarks and only identify the name). The map is curated, from a list in ui/scripts/brand-icons.mjs: 107 of the 133 brands listed have a glyph, mapped from 228 domains and hosts (CDN names too: nflxso.net is Netflix). The longest matching suffix of a name, on whole labels, wins, so notnetflix.com is not Netflix. The script writes the committed data files (brands.ts, map.data.ts): a build never needs the network or the package.
  • Every other name gets a monogram: the first letter of its owner's label, in a tint of a hue taken from a hash of the registrable domain (one lightness and one chroma for every hue, so none outweighs another; both themes). A name of the network itself (.arpa, .lan, .local, .home, .internal, .svc, one label) gets a quiet house. A name the privacy level hid (h and 16 hex digits) gets nothing, and must not pass for a name of the network.
  • A blocked name wears the same mark with most of its colour drained. A near-black mark flips to a light tile in the dark theme.
  • The glyphs are a separate lazy chunk (about 120 KB raw, 51 KiB gzipped), fetched when the first brand tile is needed. Monogram only and Off never fetch it.
  • The choice is Settings → Account → Domain icons: a style (Brand and monogram, the default; Monogram only; Off) and a size (Default, 20 px; Small, 16 px). It is kept per user in the browser (wrendns.icons.<user>), like the detail level: no server state, no API, no configuration, nothing in the cluster.

Consequences

  • The open set has dropped several large companies' marks (Microsoft, Amazon and AWS, LinkedIn, Slack, Disney+, Hulu, OpenAI, Xbox, Minecraft, Nintendo, Adobe, the BBC, Bing; 25 of the 133 listed brands), so their names wear a monogram. 20 of the 27 service presets get a brand tile. A name outside the map is a monogram, never a guess.
  • The map is data we maintain: a brand is added to the list and the data regenerated. docs/third-party-notices.md carries the Simple Icons notice and the trademark disclaimer.
  • Cost: about 6 KiB gzipped in the main chunk and the 51 KiB lazy chunk (the UI is 295.6 KiB of its 400 KiB budget); no server work.
  • Other pages (Clients, a device, Analyze, Explain, Flow, the Traffic map, the Wall) don't carry the mark yet.

Amended 2026-11-17 (the owner: "go on adding the bootstrap icons pack as well as long as it doesn't create too much overhead"): a second pack, for the marks the first one dropped.

  • Bootstrap Icons 1.13.2 (MIT) is the second choice: when Simple Icons has none of a brand's candidate slugs, the generator takes the Bootstrap glyph named with a bi: prefix in the list. Eleven of the 25 missing brands have one: Microsoft (also used for Office, Azure, and Edge names, which have no mark of their own), Teams, Windows, Xbox, Bing, Skype, LinkedIn, Amazon, Slack, OpenAI, Nintendo. Disney+, Hulu, Minecraft, Adobe, the BBC, Walmart, and AWS are in neither pack and stay monograms. A glyph with no colour gets a brand colour chosen by hand in the generator (BOOTSTRAP), and its path sits on a 16 grid, so the data carries the grid size when it isn't 24 and the component reads it.
  • Overhead: none at run time, and the chunk is smaller. The generator now minifies path data with svgo to two decimals (a third of the bytes, under 1% difference at 16 and 20 px), which more than pays for the eleven added glyphs: the glyph data is 86.9 KiB raw and 35.4 KiB gzipped (it was about 118 and 51 for 107 glyphs). svgo, and both packs, are used only when the generator runs (npm i --no-save, pinned in the script, checked by it); nothing is added to package.json, to the build, or to the image.
  • docs/third-party-notices.md carries the Bootstrap Icons licence; the developer page site/dev-icons.html has the recipe for adding a brand or a pack.

Builds on ADR-102

ADR-168 A leaner image: the OpenAPI document compiled in, a gzip-only embedded UI, no CA bundleAcceptedNFRThe image gate is 15 MiB compressed per architecture (NFR-001).

Context

The image gate is 15 MiB compressed per architecture (NFR-001). At 13bb27d the images were 14.84, 14.16, and 14.82 MiB (amd64, arm64, armv7), so one more feature of the size of the domain icons (+0.11 MiB) would have failed the gate on amd64 and armv7. A measured review (2026-11-17: the compressed layer sizes the gate sums, three release builds on the build VM, the BuildKit CI uses) found where the bytes are. The binary is about 97% of the image. Of it, utoipa's OpenAPI document builder was about 0.6 MiB, and the 677 KB document was rebuilt at every request. The embedded UI cost about twice its gzip size, because the build kept a raw copy beside each .gz and the layer's compression can only shrink the raw one. And a CA bundle (0.12 MiB) that no code reads. Decision (the owner chose these, 2026-11-17):

  • GET /api/v1/openapi.json serves docs/api/openapi.json, compiled in with include_str!, not a document built from ApiDoc while running. api_001_committed_openapi_matches_the_code still builds it from the code (a test-only function) and keeps the file equal to it; UPDATE_OPENAPI=1 regenerates the file.
  • npm run build replaces each text file in ui/dist with its .gz when that is smaller (ui/scripts/postbuild.mjs). The server sends the stored file unchanged to a client that accepts gzip, and inflates it for any other (curl without --compressed, a probe) with miniz_oxide, which is already in the tree. A file with no .gz is served as stored, and a damaged .gz is refused (500), never served half-read. This amends ADR-030's size-and-caching bullet ("so no compression crate on the server"): the server now has an inflater, and no new crate.
  • The image has no CA bundle (/etc/ssl/certs/ca-certificates.crt). Every TLS client (upstream DoT and DoH, list fetches, event sinks, SMTP) verifies against the Mozilla roots compiled in with webpki-roots; none reads the system store, and rustls-native-certs is not in the lockfile. A private CA is set with tls_ca as before, which doesn't depend on the bundle's path.

Not done (reviewed): zstd layers (about 2.1 MiB smaller, but Docker before 23.0 and containerd before 1.5 can't pull them, and Debian 12's docker.io is 20.10); removing .eh_frame (the watchdog's stack capture unwinds with it, ADR-163); the linker's identical-code-folding (the cross compiler rejects it); zopfli re-compression of the layer (pipeline and provenance cost); a smaller zoneinfo and SQLite without its unused full-text and R-tree modules (ADR-169); building crates off the DNS path for size (decided after a benchmark).

Consequences

  • The three changes took the images from 14.84, 14.16, and 14.82 MiB to 13.83, 13.08, and 13.54 (amd64, arm64, armv7; read from the size gate of the CI image job on 3027e71; measured per item on the build VM for amd64: OpenAPI -0.60, UI -0.29, CA bundle -0.12), and every future UI file costs about half what it did.
  • Serving the OpenAPI document is faster: no 677 KB document is built per request. The served document can't differ from the committed one, and the drift test is what keeps the committed one equal to the code.
  • A client that doesn't accept gzip costs one inflate of at most about 700 KB per request (rare: browsers always accept gzip).
  • spec/08 §2 no longer lists a CA bundle in the image.

Requirements NFR-001

Builds on ADR-030 ADR-163 ADR-169

Referenced by ADR-030 ADR-169 ADR-170 ADR-915

ADR-169 A compact zoneinfo and SQLite without its unused modulesAcceptedFLTADR-168 left two low-risk cuts undecided.

Context

ADR-168 left two low-risk cuts undecided. The owner chose both on 2026-11-17 ("do items 3 and 4 too") after T23.1 took the images to 13.83, 13.08, and 13.54 MiB. The review had measured them on amd64: a slim zoneinfo -0.25 MiB, SQLite without full-text search and the R-tree index -0.20 MiB. Neither is on the DNS path.

Decision

  • The image's zoneinfo is built by zic -b fat from the package's own tzdata.zi (Dockerfile) instead of copied from /usr/share/zoneinfo. Debian's package ships the database three times (the main directory, posix/, and right/, which carries leap seconds); the build writes it once, with each name a hard link to one file, and copies it with cp -a so the layer stores each file once. Measured with tar and gzip on the build VM (tzdata 2025b): 347 KB for the distribution's directory, 151 KB built this way. The build fails if a zone name the distribution's directory has is missing from it (the host's own localtime and posixrules links and the two extra trees aside). jiff is the only reader (schedule time zones, FLT-010). crates/wrendns-config/tests/zoneinfo.rs builds a directory the same way from the host's tzdata.zi and checks that jiff reads every file, that every name is present, and that the offset, abbreviation, and daylight-saving flag agree with the host's directory for every zone (weekly) and for twelve zones with every kind of rule (every six hours), from 2020 to 2075.
  • zic -b slim was built, tested, and is not used. It is 87 KB, 0.06 MiB smaller still, but jiff 0.2.37 refuses one of its 598 files: America/Ojinaga from tzdata 2025b ("expected last transition in TZif file to have DST offset matching the offset derived from the POSIX time zone transition rule"), so a schedule in that zone would be rejected as an unknown time zone. The first version of this change was slim; the test found it. The fat files, which are what distributions ship, are all accepted. If a later jiff accepts the slim file, the 0.06 MiB can be taken, and the test is the check.
  • SQLite is built without FTS3, FTS5, the R-tree index, and the dbstat table, by LIBSQLITE3_FLAGS (-U flags, which the bundled SQLite's build script applies after its own -D defaults). The flags are in the repository's .cargo/config.toml, not in the Dockerfile, so cargo test runs against the SQLite the image ships and anyone building from source gets the same one. crates/wrendns-store/tests/sqlite_build.rs asserts the four modules are gone and that foreign keys on by default, the JSON functions, STAT4, column metadata, and thread safety stay. No code here uses a virtual table, MATCH, or dbstat. A real image build confirmed it: none of the four option strings is in the shipped binary, and STAT4, column metadata, and the foreign-key default are.

Not done

a slim zoneinfo (above); dropping the other optional parts of the bundled build (load-extension support, soundex, API armor): they cost little and are harder to be sure of.

Consequences

  • About 0.4 MiB less on amd64. A real image build with the slim variant printed 13.32 MiB against 13.83; the fat variant adds about 0.06 MiB. The size gate in CI prints the real numbers.
  • A zone that only a posix/ or right/ path named (nothing here uses one) is no longer in the image. A time zone added to the database later is picked up by rebuilding the image, as before.
  • A SQLite file that defines a full-text or R-tree table still opens (SQLite looks a module up when the table is used); using such a table fails with "no such module".
  • spec/08 §2 says the image carries a compact zoneinfo.

Requirements FLT-010

Builds on ADR-168

Referenced by ADR-168 ADR-170 ADR-915

ADR-170 Crates off the DNS path are built for sizeAcceptedAfter ADR-168 and ADR-169 the images were 13.39, 12.66, and 13.10 MiB of the 15 MiB gate.

Context

After ADR-168 and ADR-169 the images were 13.39, 12.66, and 13.10 MiB of the 15 MiB gate. The 2026-11-17 review found that building the crates that are not on the DNS path with opt-level = "s" instead of 3 makes the binary smaller, and asked for a benchmark first, because performance ranks above footprint. The owner chose to take it if it was safe ("Yes", 2026-11-17) and to measure SQLite's C code at -Os separately. Measured (the build VM; release profile with fat LTO; binaries built from 938c9a9, stripped and gzipped as a stand-in for the layer):

  • The off-path list (26 package specs in the measurement, rusqlite among them): 13,116 to 12,746 KiB, -0.36 MiB. Of the 1,737 functions in the DNS-path crates (wrendns-proto, -net, -cache, -filter, -policy, -telemetry, -upstream), 1,724 are identical in size and 13 differ: startup and configuration code (the preset deserializers, a once-cell initializer, a telemetry drain), none per query. Rotated bench-smoke with an identical-binary control, five rounds, cache-hot median: baseline 526k queries/s, control 500k (-5.0%), this list 510k (-3.2%), inside the control's spread. Miss-heavy: flat.
  • The same list plus wrendns-config: -1.0 MiB in all, but ClientTable::identify (per query) grew from 3,428 to 4,039 bytes, the query handler changed, and cache-hot throughput was 474k (-9.9%, median p50 95 µs against 55). Not taken.
  • SQLite's C code (libsqlite3-sys) at -Os: a further -0.31 MiB, but on the rollup store's own tables and queries (hour_top and rollup_minute, in memory, a copy of the binary as control within 0.5%) the dashboard's week-wide top list was 10.5% slower, minute-blob range reads 11%, hourly point reads 22%, and bulk insert 38%. Not taken.
  • The rusqlite wrapper at "s": top list -0.8%, blobs -0.8%, bulk insert +0.5%, point reads +4.6% (the control's spread was about 4%). The saving is small and it is on that same query path. Not taken.

Decision

the 25 package specs in [profile.release.package.*] of the root Cargo.toml (the HTTP API and its OIDC, OpenAPI, and certificate parts, cluster replication, the git source, the support package, and the config-file parsers) are built with opt-level = "s". wrendns-config, rusqlite, libsqlite3-sys, and every crate that runs per query stay at the release profile's 3. crates/wrendns-config/tests/profile.rs fails if a hot-path crate is added to the list, if an entry is not exactly opt-level = "s", or if an entry no longer matches Cargo.lock (a name@version entry that a dependency update outdates is only a cargo warning, and the saving would be gone without anyone noticing).

Consequences

  • About 0.35 MiB less on every architecture (the size gate in CI prints the real numbers).
  • The API, OIDC, and replication code is somewhat slower per call. It runs per request and per replication message, not per DNS query, and the dashboard's own queries run in SQLite, which is unchanged.
  • A new crate on the DNS path is not on the list by default; one that is added to the list has to pass the test, which names the crates known to be hot.

Builds on ADR-168 ADR-169

Referenced by ADR-986

ADR-171 The query log can be searched for what does not match: a "not" per filter, and a read an older peer doesn't knowAcceptedGitHub issue #58: a user asked for "does not contain" in the query log search.

Context

GitHub issue #58: a user asked for "does not contain" in the query log search. The name filter matches in five ways (contains, exactly, with subdomains, wildcard, regex) and the other filters (client, status, type, response code) are "is one of"; nothing says "everything except", and a regex can't (the engine runs in linear time and has no lookaround). The analysis queries already have != and "not in".

Decision

  • Five filters can be turned around: name (each of its five matches), client, status, type, and response code. They are typed booleans (nameNot, clientNot, statusNot, qtypeNot, rcodeNot) on GET /api/v1/queries, the first four on GET /api/v1/queries/stream; --not-name, --not-client, --not-status, --not-qtype on wrendns qlog search; the Query log page's Match menu ("does not contain", "is not", "is not, or under", "does not match wildcard", "does not match regex"), a "not" box beside Client, Type, and Rcode, and a "not" chip before the statuses. Group, upstream, and slowness are not turned around. The Live view honours them (not regex, which it never had).
  • A flag without its filter is a 400 that names it (as match without name is), not ignored. The page drops such a flag from an old address instead of sending it. A negated rcode keeps the queries that got no response code.
  • The store's Filter carries a Not (five flags). The name and client predicates are resolved against each segment's dictionary into a yes/no per name, so a negated one only flips that set: no extra work per row. What a negation gives up is skipping by bloom filter (a bloom says "maybe present", never "absent from every row"): a negated name or client reads every block the time range and the other filters leave, as "contains", wildcard, and regex already do. A block is skipped for a negated status or response code only when every value in it is excluded (the header masks). Limits, threads, and the idle priority are the existing ones; the docs say to give a time range on a long log.
  • In a cluster, peers decode QueryParams and TailParams leniently, so an older peer would ignore the new flags and answer rows that were not negated into a negated search. A negated search is sent as a new read (QueriesNegated, TailOpenNegated) that an older peer doesn't know, so it is listed among the nodes not reached (as other newer reads are) and never answers wrongly.
  • The address keeps the existing keys and adds the flags (match=exact&nameNot=true): an old link keeps meaning what it did, and an older page opening a new link shows the un-negated search.

Not done

negating group, upstream, or slowness; combining filters with OR; lookaround in regex.

Consequences

  • A negated name search costs about what a "contains" search does; a negated exact name no longer gets the one-block shortcut.
  • The OpenAPI document, the UI types, and the MCP tool schema (search_queries) gain five optional booleans.
  • During a rolling upgrade, a negated search of the whole cluster reaches only the upgraded nodes.

Referenced by ADR-986

ADR-172 A cluster support package: each node redacts its own part under one key, with labels every node agrees onAcceptedGitHub issue #55 (the later phase of #50, ADR-156): in a cluster the problem is often on another node, and a package of the node that answered covers only that one.

Context

GitHub issue #55 (the later phase of #50, ADR-156): in a cluster the problem is often on another node, and a package of the node that answered covers only that one. ADR-156 settled the direction (each node redacts before its data leaves it, the asking node's key travels over the mutually authenticated stream) and left three points: how the key reaches the others, what an older peer does, and the size cap across nodes. One more came up in building it: ADR-156 numbers a name by when it is first met (device-3), so two nodes would give one device two labels, and the issue asks that a device have one name on every node.

Decision

  • POST /api/v1/system/support-package takes cluster: true; wrendns support-package --cluster; a checkbox on the System page. It is the node that was asked that puts the package together. Admin only, no agent scope, no MCP tool, like the single-node package.
  • That node makes a random 256-bit key for the package and sends it in a new read (SupportCollect) over the cluster stream to every reachable member. The key is held in memory for the call and never stored or logged. A member that is ephemeral (a resolver pod) may be asked but may not ask: the key lets its holder undo the address permutation, so only a member the cluster trusts with more than reads may hold one.
  • Each node makes its normal package (collectors, redaction, classification, the same limits) under that key, but with keyed labels: a name's label is device- and eight hex digits of a keyed hash of the name (a few more if two names land on one label), not a number by first encounter. The same name is the same label on every node, in any order, and under another key it is another label. Addresses and MAC addresses were already a keyed permutation. A single-node package keeps its numbers.
  • A node sends back its part: its files as text, its manifest, and the counts, gzip-compressed JSON. The cluster channel carries about a mebibyte per answer, so the node cuts its largest files until the compressed part fits 1,000,000 bytes (the manifest says what was cut), and the asking node reads it with a limit on the unpacked size.
  • The asking node puts the parts side by side (nodes/<node>/... with each node's own MANIFEST.json) under a summary MANIFEST.json (kind: cluster) and a README. The size cap is the single-node 20 MiB divided into equal shares, one per reachable member; a part over its share, or a file whose path is not allowed, is left out and said so. A node that doesn't answer (45 s), is older than this feature (it doesn't know the read, so it answers an error), is busy making another package, or has no open stream is listed with a fixed reason: nothing a peer says is copied into the package. Real node names appear only at the none level; otherwise the asking node names a missing node with the same keyed labels.
  • No pseudonym map: the names are replaced on the nodes they belong to, and a map would have to carry them over the network, so keepMap with cluster is refused (400) and --keep-map with --cluster is an error.

Alternatives rejected

numbering by first encounter and renaming at the asking node (the asking node would hold every node's names, which is what the design avoids); sending each node's data to the asking node to redact (the unredacted data would cross the network); a map the asking node assembles (same); a separate large-transfer channel for parts (the cap and the compression keep a part under a mebibyte, and a node that can't fit is listed, not hidden).

Consequences

  • A cluster package is made with one request, and a device reads the same in every node's files. The labels are hashes, so they say less about order than device-3; the manifest and README explain them.
  • Two different names that hash to one label get different labels on the node that sees both, but another node that sees only one of them may give it the shorter label: a rare ambiguity, harmless to reading.
  • An older node is listed as missing until it is upgraded; a rolling upgrade gives a partial package for a while, and says which nodes.
  • Each part is redacted the same way a single-node package is (best effort for free text), and the README says so.

Verification

pure tests in wrendns-support (labels the same in any order and under one key, different under another; a part survives the wire and a damaged or oversized one is refused; a part is cut to fit; the package lists present, missing, and over-share nodes; paths that escape a node's folder are refused; two nodes with one name get two folders), the request checks, and the existing single-node package tests unchanged.

Builds on ADR-156

Referenced by ADR-986